2024-05-30T16:29:11.897 INFO:root:teuthology version: 0.0.1.dev296+ge1979f0 2024-05-30T16:29:11.898 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140 -- /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/orig.config.yaml 2024-05-30T16:29:11.941 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-30T16:29:12.031 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140 branch: rishabh-clone-stats description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7734140' kernel: flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-clone-stats ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-vol-clone-stats sha1: 019a0d1fdedb1237f85e8a3711761fae4e094286 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 964 sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 sleep_before_teardown: 0 suite: fs:volumes suite_branch: mgr-vol-clone-stats suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 019a0d1fdedb1237f85e8a3711761fae4e094286 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLntZL+JZlpcR/rjkSN8yfgu96A86IvYBCyRGKW1pfr5viYvxvv5BdKfZqCNZ8uazKCxM4uHQ8z/R8kHETOtRfo= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEV3AP8Hs3d69mJfERJGWAj5x9zCw30kGPI2vgWrwz2MfnGSvpnuc0UZM12VEWjiNE+h22c0rIUEOfTMKYf1YLk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e1979f064b0becde2020282c7e87cf3c5e98026c timestamp: 2024-05-30_16:10:36 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-30T16:29:12.032 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa; will attempt to use it 2024-05-30T16:29:12.032 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa/tasks 2024-05-30T16:29:12.032 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-30T16:29:12.034 INFO:teuthology.task.internal:Checking packages... 2024-05-30T16:29:12.052 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9ed025fbf4406fb54980a0647506b15895b6d863' 2024-05-30T16:29:12.052 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-30T16:29:12.052 INFO:teuthology.packaging:ref: None 2024-05-30T16:29:12.052 INFO:teuthology.packaging:tag: None 2024-05-30T16:29:12.053 INFO:teuthology.packaging:branch: rishabh-clone-stats 2024-05-30T16:29:12.053 INFO:teuthology.packaging:sha1: 9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-30T16:29:12.053 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-clone-stats 2024-05-30T16:29:12.276 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3154.g9ed025fb 2024-05-30T16:29:12.278 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-30T16:29:12.284 INFO:teuthology.task.internal:no buildpackages task found 2024-05-30T16:29:12.284 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-30T16:29:12.290 INFO:teuthology.task.internal:Saving configuration 2024-05-30T16:29:12.307 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-30T16:29:12.314 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-30T16:29:12.340 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-30 16:22:46.129388', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLntZL+JZlpcR/rjkSN8yfgu96A86IvYBCyRGKW1pfr5viYvxvv5BdKfZqCNZ8uazKCxM4uHQ8z/R8kHETOtRfo='} 2024-05-30T16:29:12.358 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-30 16:22:46.131345', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEV3AP8Hs3d69mJfERJGWAj5x9zCw30kGPI2vgWrwz2MfnGSvpnuc0UZM12VEWjiNE+h22c0rIUEOfTMKYf1YLk='} 2024-05-30T16:29:12.358 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-30T16:29:12.364 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-30T16:29:12.364 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-30T16:29:12.364 INFO:teuthology.run_tasks:Running task console_log... 2024-05-30T16:29:12.415 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe387f9fbe0>, signals=[15]) 2024-05-30T16:29:12.415 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-30T16:29:12.421 INFO:teuthology.task.internal:Opening connections... 2024-05-30T16:29:12.422 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:29:12.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:29:12.502 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T16:29:12.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:29:12.574 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-30T16:29:12.581 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-05-30T16:29:12.599 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-05-30T16:29:12.600 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-05-30T16:29:12.658 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2024-05-30T16:29:12.658 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2024-05-30T16:29:12.658 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2024-05-30T16:29:12.658 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2024-05-30T16:29:12.658 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2024-05-30T16:29:12.658 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2024-05-30T16:29:12.659 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-30T16:29:12.659 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-05-30T16:29:12.659 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2024-05-30T16:29:12.659 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-30T16:29:12.659 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2024-05-30T16:29:12.659 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-30T16:29:12.659 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-30T16:29:12.659 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-30T16:29:12.660 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-05-30T16:29:12.685 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-05-30T16:29:12.703 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-05-30T16:29:12.703 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-05-30T16:29:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-05-30T16:29:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-05-30T16:29:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-05-30T16:29:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-05-30T16:29:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-05-30T16:29:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-05-30T16:29:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-30T16:29:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-05-30T16:29:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-05-30T16:29:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-30T16:29:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-05-30T16:29:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-30T16:29:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-30T16:29:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-30T16:29:12.761 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-05-30T16:29:12.786 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-30T16:29:12.795 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-30T16:29:12.800 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-30T16:29:12.800 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-05-30T16:29:12.803 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-05-30T16:29:12.819 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-30T16:29:12.825 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-30T16:29:12.825 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-05-30T16:29:12.860 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-05-30T16:29:12.928 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-30T16:29:12.982 INFO:teuthology.run_tasks:Running task kernel... 2024-05-30T16:29:12.996 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-30T16:29:12.996 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, timeout 300 2024-05-30T16:29:12.996 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-30T16:29:12.996 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-30T16:29:13.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T16:29:13.013 DEBUG:teuthology.task.kernel:sha1 for mon.b is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-30T16:29:13.013 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-30T16:29:13.013 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-05-30T16:29:13.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T16:29:13.014 DEBUG:teuthology.task.kernel:sha1 for mon.a is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-30T16:29:13.014 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-30T16:29:13.015 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-05-30T16:29:13.070 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-437.el9.x86_64 2024-05-30T16:29:13.071 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-30T16:29:13.071 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-30T16:29:13.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-30T16:29:13.075 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-437.el9.x86_64 2024-05-30T16:29:13.075 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-30T16:29:13.075 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-30T16:29:13.075 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-30T16:29:13.288 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum upgrade -y linux-firmware 2024-05-30T16:29:13.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum upgrade -y linux-firmware 2024-05-30T16:29:15.436 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-30T16:29:15.471 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-30T16:29:32.837 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-05-30T16:29:33.218 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-05-30T16:29:37.138 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-30T16:29:38.788 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 17 kB 00:00 2024-05-30T16:29:39.327 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-05-30T16:29:39.840 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-30T16:29:40.719 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 17 kB 00:00 2024-05-30T16:29:42.731 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-30T16:29:45.257 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-30T16:29:47.447 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-05-30T16:29:47.448 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-05-30T16:29:47.448 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-05-30T16:29:47.565 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-30T16:29:47.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-30T16:29:47.714 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-30T16:29:47.714 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-30T16:29:47.717 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi067.front.sepia.ceph.com to complete... 2024-05-30T16:29:47.895 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-05-30T16:29:49.983 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-05-30T16:29:49.984 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-05-30T16:29:49.985 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-05-30T16:29:50.081 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-30T16:29:50.081 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-30T16:29:50.225 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-30T16:29:50.225 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-30T16:29:50.228 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi169.front.sepia.ceph.com to complete... 2024-05-30T16:30:07.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30 16:30:07 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-30T16:30:07.142 INFO:teuthology.orchestra.run.smithi067.stderr:FINISHED --2024-05-30 16:30:07-- 2024-05-30T16:30:07.142 INFO:teuthology.orchestra.run.smithi067.stderr:Total wall clock time: 19s 2024-05-30T16:30:07.142 INFO:teuthology.orchestra.run.smithi067.stderr:Downloaded: 1 files, 340M in 19s (17.9 MB/s) 2024-05-30T16:30:07.145 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-30T16:30:07.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-30T16:30:09.341 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying... ######################################## 2024-05-30T16:30:09.472 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing... ######################################## 2024-05-30T16:30:09.594 INFO:teuthology.orchestra.run.smithi067.stdout:Updating / installing... 2024-05-30T16:30:11.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30 16:30:11 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-30T16:30:11.625 INFO:teuthology.orchestra.run.smithi169.stderr:FINISHED --2024-05-30 16:30:11-- 2024-05-30T16:30:11.626 INFO:teuthology.orchestra.run.smithi169.stderr:Total wall clock time: 21s 2024-05-30T16:30:11.626 INFO:teuthology.orchestra.run.smithi169.stderr:Downloaded: 1 files, 340M in 21s (16.3 MB/s) 2024-05-30T16:30:11.628 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-30T16:30:11.628 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-30T16:30:13.694 INFO:teuthology.orchestra.run.smithi169.stdout:Verifying... ######################################## 2024-05-30T16:30:13.855 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing... ######################################## 2024-05-30T16:30:13.970 INFO:teuthology.orchestra.run.smithi169.stdout:Updating / installing... 2024-05-30T16:30:17.880 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-30T16:30:21.950 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-30T16:30:57.214 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-30T16:30:57.215 DEBUG:teuthology.orchestra.run.smithi067:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-30T16:30:57.293 INFO:teuthology.orchestra.run.smithi067.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-30T16:30:57.293 INFO:teuthology.orchestra.run.smithi067.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-30T16:30:57.293 INFO:teuthology.orchestra.run.smithi067.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-30T16:30:57.293 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-30T16:30:57.293 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-30T16:30:57.294 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-30T16:30:57.295 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-30T16:30:57.296 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-30T16:30:57.297 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-30T16:30:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-30T16:30:57.299 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-30T16:30:57.300 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-30T16:30:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-30T16:30:57.302 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-30T16:30:57.303 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-30T16:30:57.304 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-30T16:30:57.305 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-30T16:30:57.306 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-30T16:30:57.307 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-30T16:30:57.308 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-30T16:30:57.309 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-30T16:30:57.310 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-30T16:30:57.311 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-30T16:30:57.312 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-30T16:30:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-30T16:30:57.314 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-30T16:30:57.315 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-30T16:30:57.316 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-30T16:30:57.317 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-30T16:30:57.318 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-30T16:30:57.319 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-30T16:30:57.320 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-30T16:30:57.321 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-30T16:30:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-30T16:30:57.323 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-30T16:30:57.324 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-30T16:30:57.325 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-30T16:30:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-30T16:30:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-30T16:30:57.328 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-30T16:30:57.329 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-30T16:30:57.330 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-30T16:30:57.331 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-30T16:30:57.332 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-30T16:30:57.333 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-30T16:30:57.334 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-30T16:30:57.335 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-30T16:30:57.336 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-30T16:30:57.337 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-30T16:30:57.338 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-30T16:30:57.339 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-30T16:30:57.340 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-30T16:30:57.341 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-30T16:30:57.342 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-30T16:30:57.343 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-30T16:30:57.344 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-30T16:30:57.345 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-30T16:30:57.346 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-30T16:30:57.347 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-30T16:30:57.348 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-30T16:30:57.349 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-30T16:30:57.350 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-30T16:30:57.351 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-30T16:30:57.352 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-30T16:30:57.353 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-30T16:30:57.354 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-30T16:30:57.355 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-30T16:30:57.356 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-30T16:30:57.357 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-30T16:30:57.358 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-30T16:30:57.359 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-30T16:30:57.359 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-30T16:30:57.359 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-30T16:30:57.359 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-30T16:30:57.359 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-30T16:30:57.359 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-30T16:30:57.359 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-30T16:30:57.359 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-30T16:30:57.359 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-30T16:30:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-30T16:30:57.362 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-30T16:30:57.363 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-30T16:30:57.364 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-30T16:30:57.365 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-30T16:30:57.366 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-30T16:30:57.367 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-30T16:30:57.368 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-30T16:30:57.369 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-30T16:30:57.370 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-30T16:30:57.371 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-30T16:30:57.372 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-30T16:30:57.373 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-30T16:30:57.374 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-30T16:30:57.375 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-30T16:30:57.376 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-30T16:30:57.377 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-30T16:30:57.378 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-30T16:30:57.379 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-30T16:30:57.380 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-30T16:30:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-30T16:30:57.382 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-30T16:30:57.383 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-30T16:30:57.384 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-30T16:30:57.385 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-30T16:30:57.386 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-30T16:30:57.387 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-30T16:30:57.388 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-30T16:30:57.389 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-30T16:30:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-30T16:30:57.391 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-30T16:30:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-30T16:30:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-30T16:30:57.394 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-30T16:30:57.395 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-30T16:30:57.396 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-30T16:30:57.397 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-30T16:30:57.398 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-30T16:30:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-30T16:30:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-30T16:30:57.401 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-30T16:30:57.402 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-30T16:30:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-30T16:30:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-30T16:30:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-30T16:30:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-30T16:30:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-30T16:30:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-30T16:30:57.409 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-30T16:30:57.410 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-30T16:30:57.411 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-30T16:30:57.412 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-30T16:30:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-30T16:30:57.414 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-30T16:30:57.415 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-30T16:30:57.416 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-30T16:30:57.417 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-30T16:30:57.418 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-30T16:30:57.419 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-30T16:30:57.420 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-30T16:30:57.421 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-30T16:30:57.422 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-30T16:30:57.423 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-30T16:30:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-30T16:30:57.425 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-30T16:30:57.426 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-30T16:30:57.427 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-30T16:30:57.428 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-30T16:30:57.429 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-30T16:30:57.430 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-30T16:30:57.431 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-30T16:30:57.435 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-30T16:30:57.435 DEBUG:teuthology.orchestra.run.smithi067:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-30T16:30:57.459 INFO:teuthology.orchestra.run.smithi067.stdout:postinstall scriptlet (using /bin/sh): 2024-05-30T16:30:57.459 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-30T16:30:57.459 INFO:teuthology.orchestra.run.smithi067.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-30T16:30:57.459 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-05-30T16:30:57.459 INFO:teuthology.orchestra.run.smithi067.stdout:for file in vmlinuz System.map config; do 2024-05-30T16:30:57.459 INFO:teuthology.orchestra.run.smithi067.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout: fi 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout:done 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-30T16:30:57.460 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-30T16:30:57.461 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-05-30T16:30:57.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-05-30T16:30:57.539 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-05-30T16:30:57.539 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-05-30T16:30:57.539 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2024-05-30T16:30:57.539 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 68.el9 2024-05-30T16:30:57.539 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-05-30T16:30:57.539 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-30T16:30:57.539 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2024-05-30T16:30:57.539 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8264254 2024-05-30T16:30:57.539 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-30T16:30:57.540 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-30T16:30:57.541 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-05-30T16:30:57.541 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:30:57.541 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-05-30T16:30:57.541 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-30T16:30:57.541 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 6.9.0-gb9459a26b088 2024-05-30T16:30:57.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-30T16:30:58.133 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-05-30T16:30:59.539 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-30T16:30:59.574 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-05-30T16:30:59.576 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-05-30T16:30:59.609 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.yhra8WrZQh 2024-05-30T16:30:59.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yhra8WrZQh 2024-05-30T16:30:59.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.yhra8WrZQh 2024-05-30T16:30:59.818 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.yhra8WrZQh is 6KB 2024-05-30T16:30:59.866 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.yhra8WrZQh 2024-05-30T16:30:59.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-05-30T16:30:59.943 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-30T16:30:59.943 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-30T16:30:59.943 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-30T16:30:59.943 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-30T16:30:59.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-30T16:31:00.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-05-30T16:31:01.342 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi169.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-30T16:31:01.342 DEBUG:teuthology.orchestra.run.smithi169:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-30T16:31:01.420 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-30T16:31:01.420 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-30T16:31:01.420 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-30T16:31:01.420 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-30T16:31:01.420 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-30T16:31:01.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-30T16:31:01.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-30T16:31:01.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-30T16:31:01.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-30T16:31:01.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-30T16:31:01.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-30T16:31:01.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-30T16:31:01.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-30T16:31:01.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-30T16:31:01.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-30T16:31:01.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-30T16:31:01.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-30T16:31:01.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-30T16:31:01.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-30T16:31:01.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-30T16:31:01.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-30T16:31:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-30T16:31:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-30T16:31:01.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-30T16:31:01.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-30T16:31:01.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-30T16:31:01.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-30T16:31:01.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-30T16:31:01.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-30T16:31:01.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-30T16:31:01.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-30T16:31:01.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-30T16:31:01.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-30T16:31:01.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-30T16:31:01.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-30T16:31:01.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-30T16:31:01.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-30T16:31:01.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-30T16:31:01.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-30T16:31:01.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-30T16:31:01.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-30T16:31:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-30T16:31:01.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-30T16:31:01.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-30T16:31:01.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-30T16:31:01.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-30T16:31:01.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-30T16:31:01.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-30T16:31:01.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-30T16:31:01.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-30T16:31:01.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-30T16:31:01.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-30T16:31:01.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-30T16:31:01.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-30T16:31:01.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-30T16:31:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-30T16:31:01.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-30T16:31:01.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-30T16:31:01.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-30T16:31:01.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-30T16:31:01.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-30T16:31:01.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-30T16:31:01.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-30T16:31:01.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-30T16:31:01.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-30T16:31:01.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-30T16:31:01.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-30T16:31:01.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-30T16:31:01.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-30T16:31:01.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-30T16:31:01.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-30T16:31:01.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-30T16:31:01.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-30T16:31:01.490 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-30T16:31:01.491 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-30T16:31:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-30T16:31:01.493 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-30T16:31:01.494 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-30T16:31:01.495 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-30T16:31:01.496 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-30T16:31:01.497 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-30T16:31:01.498 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-30T16:31:01.499 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-30T16:31:01.500 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-30T16:31:01.501 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-30T16:31:01.502 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-30T16:31:01.503 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-30T16:31:01.504 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-30T16:31:01.505 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-30T16:31:01.506 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-30T16:31:01.507 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-30T16:31:01.508 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-30T16:31:01.509 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-30T16:31:01.510 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-30T16:31:01.511 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-30T16:31:01.512 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-30T16:31:01.513 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-30T16:31:01.514 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-30T16:31:01.515 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-30T16:31:01.516 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-30T16:31:01.517 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-30T16:31:01.518 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-30T16:31:01.519 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-30T16:31:01.520 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-30T16:31:01.521 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-30T16:31:01.522 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-30T16:31:01.523 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-30T16:31:01.524 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-30T16:31:01.525 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-30T16:31:01.526 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-30T16:31:01.527 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-30T16:31:01.528 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-30T16:31:01.529 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-30T16:31:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-30T16:31:01.531 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-30T16:31:01.532 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-30T16:31:01.533 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-30T16:31:01.534 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-30T16:31:01.535 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-30T16:31:01.536 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-30T16:31:01.537 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-30T16:31:01.538 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-30T16:31:01.539 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-30T16:31:01.540 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-30T16:31:01.541 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-30T16:31:01.542 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-30T16:31:01.543 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-30T16:31:01.544 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-30T16:31:01.545 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-30T16:31:01.546 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-30T16:31:01.547 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-30T16:31:01.548 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-30T16:31:01.549 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-30T16:31:01.550 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-30T16:31:01.551 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-30T16:31:01.552 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-30T16:31:01.553 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-30T16:31:01.554 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-30T16:31:01.554 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-30T16:31:01.554 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-30T16:31:01.554 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-30T16:31:01.557 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-30T16:31:01.557 DEBUG:teuthology.orchestra.run.smithi169:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-30T16:31:01.579 INFO:teuthology.orchestra.run.smithi169.stdout:postinstall scriptlet (using /bin/sh): 2024-05-30T16:31:01.579 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-30T16:31:01.579 INFO:teuthology.orchestra.run.smithi169.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-30T16:31:01.579 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-05-30T16:31:01.579 INFO:teuthology.orchestra.run.smithi169.stdout:for file in vmlinuz System.map config; do 2024-05-30T16:31:01.579 INFO:teuthology.orchestra.run.smithi169.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-30T16:31:01.579 INFO:teuthology.orchestra.run.smithi169.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-30T16:31:01.579 INFO:teuthology.orchestra.run.smithi169.stdout: fi 2024-05-30T16:31:01.579 INFO:teuthology.orchestra.run.smithi169.stdout:done 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-30T16:31:01.580 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-05-30T16:31:01.581 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -qi grub2-tools 2024-05-30T16:31:01.652 INFO:teuthology.orchestra.run.smithi169.stdout:Name : grub2-tools 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Epoch : 1 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Version : 2.06 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Release : 68.el9 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: x86_64 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Group : Unspecified 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Size : 8264254 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:License : GPLv3+ 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Packager : builder@centos.org 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:Vendor : CentOS 2024-05-30T16:31:01.653 INFO:teuthology.orchestra.run.smithi169.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-30T16:31:01.654 INFO:teuthology.orchestra.run.smithi169.stdout:Summary : Support tools for GRUB. 2024-05-30T16:31:01.654 INFO:teuthology.orchestra.run.smithi169.stdout:Description : 2024-05-30T16:31:01.654 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:31:01.654 INFO:teuthology.orchestra.run.smithi169.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-30T16:31:01.654 INFO:teuthology.orchestra.run.smithi169.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-30T16:31:01.654 INFO:teuthology.orchestra.run.smithi169.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-30T16:31:01.654 INFO:teuthology.orchestra.run.smithi169.stdout:hardware devices. 2024-05-30T16:31:01.654 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:31:01.654 INFO:teuthology.orchestra.run.smithi169.stdout:This subpackage provides tools for support of all platforms. 2024-05-30T16:31:01.655 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-30T16:31:01.655 INFO:teuthology.task.kernel:Updating grub on smithi169 to boot 6.9.0-gb9459a26b088 2024-05-30T16:31:01.655 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-30T16:31:02.216 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2024-05-30T16:31:03.598 INFO:teuthology.orchestra.run.smithi169.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-30T16:31:03.630 INFO:teuthology.orchestra.run.smithi169.stderr:done 2024-05-30T16:31:03.632 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-05-30T16:31:03.649 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.NlxcUPodqy 2024-05-30T16:31:03.650 DEBUG:teuthology.orchestra.run.smithi169:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NlxcUPodqy 2024-05-30T16:31:03.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.NlxcUPodqy 2024-05-30T16:31:03.855 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.NlxcUPodqy is 6KB 2024-05-30T16:31:03.903 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.NlxcUPodqy 2024-05-30T16:31:03.917 DEBUG:teuthology.orchestra.run.smithi169:> sudo /bin/ls /boot/loader/entries || true 2024-05-30T16:31:03.979 INFO:teuthology.orchestra.run.smithi169.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-30T16:31:03.979 INFO:teuthology.orchestra.run.smithi169.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-30T16:31:03.979 INFO:teuthology.orchestra.run.smithi169.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-30T16:31:03.980 INFO:teuthology.orchestra.run.smithi169.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-30T16:31:03.981 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-30T16:31:04.134 DEBUG:teuthology.orchestra.run.smithi169:> sudo shutdown -r now 2024-05-30T16:31:30.141 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-30T16:31:30.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-05-30T16:31:30.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:31:34.139 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-30T16:31:34.139 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-30T16:31:34.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:31:48.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-05-30T16:31:52.606 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-05-30T16:31:57.516 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-05-30T16:31:57.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:32:00.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-05-30T16:32:01.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-30T16:32:01.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:32:04.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-05-30T16:32:12.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-05-30T16:32:12.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:32:16.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-30T16:32:16.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:32:31.007 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-05-30T16:32:46.021 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-05-30T16:32:46.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:32:49.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-05-30T16:33:07.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-05-30T16:33:07.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:33:07.520 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-05-30T16:33:08.217 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-05-30T16:33:08.217 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-gb9459a26b088"... 2024-05-30T16:33:08.217 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-05-30T16:33:08.313 INFO:teuthology.orchestra.run.smithi067.stdout:6.9.0-gb9459a26b088 2024-05-30T16:33:08.314 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-30T16:33:08.314 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-30T16:33:08.314 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-30T16:33:09.315 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-30T16:33:09.315 DEBUG:teuthology.orchestra.run.smithi067:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-30T16:33:09.485 INFO:teuthology.orchestra.run.smithi067.stdout:ttyS1 2024-05-30T16:33:09.554 DEBUG:teuthology.parallel:result is None 2024-05-30T16:33:16.678 DEBUG:teuthology.orchestra.remote:timed out 2024-05-30T16:33:31.679 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-30T16:33:31.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:33:32.061 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-05-30T16:33:32.698 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-30T16:33:32.698 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-gb9459a26b088"... 2024-05-30T16:33:32.698 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-05-30T16:33:32.750 INFO:teuthology.orchestra.run.smithi169.stdout:6.9.0-gb9459a26b088 2024-05-30T16:33:32.750 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-30T16:33:32.751 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-30T16:33:32.751 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-30T16:33:33.752 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-30T16:33:33.752 DEBUG:teuthology.orchestra.run.smithi169:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-30T16:33:33.897 INFO:teuthology.orchestra.run.smithi169.stdout:ttyS1 2024-05-30T16:33:33.950 DEBUG:teuthology.parallel:result is None 2024-05-30T16:33:33.950 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-30T16:33:33.959 INFO:teuthology.task.internal:Creating test directory... 2024-05-30T16:33:33.960 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-30T16:33:33.968 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-30T16:33:34.025 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-30T16:33:34.031 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-30T16:33:34.042 INFO:teuthology.task.internal:Creating archive directory... 2024-05-30T16:33:34.042 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-30T16:33:34.049 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-30T16:33:34.122 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-30T16:33:34.129 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-30T16:33:34.129 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-30T16:33:34.136 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-30T16:33:34.230 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-30T16:33:34.238 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-30T16:33:34.263 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-30T16:33:34.270 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-30T16:33:34.274 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-30T16:33:34.281 INFO:teuthology.task.internal:Configuring sudo... 2024-05-30T16:33:34.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-30T16:33:34.288 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-30T16:33:34.374 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-30T16:33:34.384 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-30T16:33:34.384 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-30T16:33:34.391 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-30T16:33:34.444 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-30T16:33:34.528 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-30T16:33:34.626 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-30T16:33:34.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-30T16:33:34.805 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:33:34.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-30T16:33:34.892 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-30T16:33:34.977 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-30T16:33:35.071 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-30T16:33:35.147 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-30T16:33:35.233 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:33:35.233 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-30T16:33:35.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-05-30T16:33:35.327 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-05-30T16:33:35.449 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-30T16:33:35.470 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-30T16:33:35.813 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-30T16:33:35.822 INFO:teuthology.task.internal:Starting timer... 2024-05-30T16:33:35.822 INFO:teuthology.run_tasks:Running task pcp... 2024-05-30T16:33:35.833 INFO:teuthology.run_tasks:Running task selinux... 2024-05-30T16:33:35.843 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-30T16:33:35.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-05-30T16:33:35.977 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-05-30T16:33:35.982 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-05-30T16:33:36.108 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-05-30T16:33:36.113 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-30T16:33:36.113 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-05-30T16:33:36.176 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-05-30T16:33:36.177 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-05-30T16:33:36.233 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-05-30T16:33:36.234 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-05-30T16:33:36.234 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-30T16:33:36.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T16:33:36.309 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-30T16:33:36.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T16:33:36.388 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-30T16:33:36.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-05-30T16:33:36.465 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-05-30T16:33:36.531 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-30T16:33:36.541 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-30T16:33:36.543 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-30T16:33:36.659 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-30T16:33:36.671 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-30T16:33:36.672 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-30T16:40:20.623 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-05-30T16:40:20.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-05-30T16:40:20.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:40:20.740 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-05-30T16:40:20.910 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-05-30T16:40:20.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-30T16:40:20.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-30T16:40:21.027 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-05-30T16:40:21.176 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-05-30T16:40:21.176 INFO:teuthology.run_tasks:Running task clock... 2024-05-30T16:40:21.190 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-30T16:40:21.191 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-30T16:40:21.191 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-30T16:40:21.199 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-30T16:40:21.200 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-30T16:40:21.289 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-30T16:40:21.298 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-30T16:40:21.334 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-30T16:40:21.340 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-30T16:40:21.407 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-05-30T16:40:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-05-30T16:40:21.440 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-05-30T16:40:21.442 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-05-30T16:40:21.482 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-30T16:40:21.487 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-30T16:40:21.523 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-30T16:40:21.532 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-30T16:40:21.655 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-05-30T16:40:21.662 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-30T16:40:21.662 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-05-30T16:40:21.663 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T16:40:21.663 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T16:40:21.663 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T16:40:21.663 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T16:40:21.665 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-05-30T16:40:21.673 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-30T16:40:21.674 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-05-30T16:40:21.674 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T16:40:21.674 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T16:40:21.674 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T16:40:21.674 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T16:40:21.675 INFO:teuthology.run_tasks:Running task install... 2024-05-30T16:40:21.684 DEBUG:teuthology.task.install:project ceph 2024-05-30T16:40:21.684 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'}} 2024-05-30T16:40:21.684 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'} 2024-05-30T16:40:21.685 INFO:teuthology.task.install:Using flavor: default 2024-05-30T16:40:21.690 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-30T16:40:21.690 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-30T16:40:21.690 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863', 'tag': None, 'wait_for_package': False} 2024-05-30T16:40:21.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-30T16:40:21.693 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863', 'tag': None, 'wait_for_package': False} 2024-05-30T16:40:21.693 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-30T16:40:21.842 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/centos/9/flavors/default/ 2024-05-30T16:40:21.842 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.g9ed025fb 2024-05-30T16:40:21.846 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/centos/9/flavors/default/ 2024-05-30T16:40:21.846 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.g9ed025fb 2024-05-30T16:40:22.016 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-30T16:40:22.016 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:40:22.016 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-30T16:40:22.024 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-clone-stats/9ed025fbf4406fb54980a0647506b15895b6d863/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-30T16:40:22.024 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:40:22.024 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-30T16:40:22.112 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-30T16:40:22.113 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9ed025fbf4406fb54980a0647506b15895b6d863/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-30T16:40:22.115 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-30T16:40:22.116 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9ed025fbf4406fb54980a0647506b15895b6d863/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-30T16:40:22.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-30T16:40:22.247 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-30T16:40:22.359 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-30T16:40:22.367 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-30T16:40:22.440 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-05-30T16:40:22.444 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-05-30T16:40:22.452 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2024-05-30T16:40:22.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2024-05-30T16:40:22.763 INFO:teuthology.orchestra.run.smithi169.stdout:45 files removed 2024-05-30T16:40:22.790 INFO:teuthology.orchestra.run.smithi067.stdout:45 files removed 2024-05-30T16:40:22.801 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-30T16:40:22.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-30T16:40:23.627 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 215 kB/s | 83 kB 00:00 2024-05-30T16:40:23.827 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 152 kB/s | 83 kB 00:00 2024-05-30T16:40:24.061 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-30T16:40:24.327 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-30T16:40:24.520 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-30T16:40:24.828 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-05-30T16:40:25.576 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-30T16:40:27.418 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.1 MB 00:02 2024-05-30T16:40:27.644 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-30T16:40:29.661 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-05-30T16:40:32.378 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-05-30T16:40:34.229 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-05-30T16:40:34.346 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 17 kB 00:00 2024-05-30T16:40:35.456 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-05-30T16:40:36.337 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 17 kB 00:00 2024-05-30T16:40:38.661 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-30T16:40:41.814 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-05-30T16:40:44.442 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-30T16:40:44.444 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-30T16:40:44.465 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-30T16:40:44.466 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-30T16:40:44.468 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-30T16:40:44.469 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-30T16:40:44.472 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-30T16:40:44.475 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-30T16:40:44.477 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-30T16:40:44.554 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-05-30T16:40:44.576 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-05-30T16:40:44.576 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-05-30T16:40:44.576 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-05-30T16:40:44.576 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-05-30T16:40:44.576 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-30T16:40:44.576 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 6.6 k 2024-05-30T16:40:44.576 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 5.4 M 2024-05-30T16:40:44.576 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 890 k 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 145 k 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 1.5 M 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 142 k 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 3.5 M 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 7.4 M 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 49 k 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 11 M 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 49 M 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 273 k 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 762 k 2024-05-30T16:40:44.577 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 225 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 27 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 18 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 32 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 746 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 127 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-30T16:40:44.578 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 161 k 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 321 k 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 298 k 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 100 k 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 85 k 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 3.1 M 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 170 k 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-05-30T16:40:44.579 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 3.4 M 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 3.1 M 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-30T16:40:44.580 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 21 M 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 34 k 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 2.4 M 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 252 k 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 4.7 M 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 17 M 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 16 k 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 25 k 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-30T16:40:44.581 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 164 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 501 k 2024-05-30T16:40:44.582 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 5.4 M 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-30T16:40:44.583 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 45 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 135 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-30T16:40:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-30T16:40:44.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-30T16:40:44.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-30T16:40:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-30T16:40:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout:Install 148 Packages 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 11 Packages 2024-05-30T16:40:44.589 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:40:44.590 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 227 M 2024-05-30T16:40:44.590 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-05-30T16:40:44.839 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-05-30T16:40:45.358 INFO:teuthology.orchestra.run.smithi067.stdout:(1/159): ceph-19.0.0-3154.g9ed025fb.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-05-30T16:40:45.574 INFO:teuthology.orchestra.run.smithi067.stdout:(2/159): ceph-fuse-19.0.0-3154.g9ed025fb.el9.x8 4.0 MB/s | 890 kB 00:00 2024-05-30T16:40:45.658 INFO:teuthology.orchestra.run.smithi067.stdout:(3/159): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-05-30T16:40:46.017 INFO:teuthology.orchestra.run.smithi067.stdout:(4/159): ceph-mds-19.0.0-3154.g9ed025fb.el9.x86 6.6 MB/s | 2.4 MB 00:00 2024-05-30T16:40:46.250 INFO:teuthology.orchestra.run.smithi067.stdout:(5/159): ceph-mgr-19.0.0-3154.g9ed025fb.el9.x86 6.4 MB/s | 1.5 MB 00:00 2024-05-30T16:40:46.709 INFO:teuthology.orchestra.run.smithi067.stdout:(6/159): ceph-common-19.0.0-3154.g9ed025fb.el9. 13 MB/s | 21 MB 00:01 2024-05-30T16:40:46.868 INFO:teuthology.orchestra.run.smithi067.stdout:(7/159): ceph-mon-19.0.0-3154.g9ed025fb.el9.x86 7.6 MB/s | 4.7 MB 00:00 2024-05-30T16:40:47.423 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-30T16:40:47.425 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-30T16:40:47.444 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-30T16:40:47.445 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-30T16:40:47.447 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-30T16:40:47.448 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-30T16:40:47.450 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-30T16:40:47.454 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-30T16:40:47.455 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-30T16:40:47.528 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-05-30T16:40:47.552 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-05-30T16:40:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-05-30T16:40:47.552 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-05-30T16:40:47.552 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 6.6 k 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 5.4 M 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 890 k 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 145 k 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 1.5 M 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 142 k 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 3.5 M 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 7.4 M 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 49 k 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 11 M 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 49 M 2024-05-30T16:40:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 273 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 762 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 225 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 27 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 18 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 32 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 746 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 127 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-30T16:40:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 161 k 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 321 k 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 298 k 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 100 k 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 85 k 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 3.1 M 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 170 k 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-30T16:40:47.555 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 3.4 M 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 3.1 M 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-05-30T16:40:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 21 M 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 34 k 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 2.4 M 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 252 k 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 4.7 M 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 17 M 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.g9ed025fb.el9 ceph-noarch 16 k 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 25 k 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-30T16:40:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 164 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 501 k 2024-05-30T16:40:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 5.4 M 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-30T16:40:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 45 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-3154.g9ed025fb.el9 ceph 135 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-30T16:40:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-30T16:40:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-30T16:40:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-30T16:40:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-30T16:40:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 11 Packages 2024-05-30T16:40:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:40:47.566 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 227 M 2024-05-30T16:40:47.566 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-05-30T16:40:47.826 INFO:teuthology.orchestra.run.smithi067.stdout:(8/159): ceph-osd-19.0.0-3154.g9ed025fb.el9.x86 15 MB/s | 17 MB 00:01 2024-05-30T16:40:47.993 INFO:teuthology.orchestra.run.smithi067.stdout:(9/159): ceph-base-19.0.0-3154.g9ed025fb.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-05-30T16:40:48.018 INFO:teuthology.orchestra.run.smithi067.stdout:(10/159): ceph-selinux-19.0.0-3154.g9ed025fb.el 132 kB/s | 25 kB 00:00 2024-05-30T16:40:48.077 INFO:teuthology.orchestra.run.smithi067.stdout:(11/159): cephfs-mirror-19.0.0-3154.g9ed025fb.e 3.8 MB/s | 225 kB 00:00 2024-05-30T16:40:48.127 INFO:teuthology.orchestra.run.smithi067.stdout:(12/159): cephfs-shell-19.0.0-3154.g9ed025fb.el 550 kB/s | 27 kB 00:00 2024-05-30T16:40:48.177 INFO:teuthology.orchestra.run.smithi067.stdout:(13/159): libcephfs-devel-19.0.0-3154.g9ed025fb 648 kB/s | 32 kB 00:00 2024-05-30T16:40:48.369 INFO:teuthology.orchestra.run.smithi067.stdout:(14/159): ceph-radosgw-19.0.0-3154.g9ed025fb.el 7.1 MB/s | 11 MB 00:01 2024-05-30T16:40:48.428 INFO:teuthology.orchestra.run.smithi067.stdout:(15/159): libcephfs2-19.0.0-3154.g9ed025fb.el9. 2.9 MB/s | 746 kB 00:00 2024-05-30T16:40:48.461 INFO:teuthology.orchestra.run.smithi067.stdout:(16/159): libcephsqlite-19.0.0-3154.g9ed025fb.e 1.8 MB/s | 164 kB 00:00 2024-05-30T16:40:48.495 INFO:teuthology.orchestra.run.smithi067.stdout:(17/159): librados-devel-19.0.0-3154.g9ed025fb. 1.9 MB/s | 127 kB 00:00 2024-05-30T16:40:48.532 INFO:teuthology.orchestra.run.smithi169.stdout:(1/159): ceph-19.0.0-3154.g9ed025fb.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-05-30T16:40:48.553 INFO:teuthology.orchestra.run.smithi067.stdout:(18/159): libradosstriper1-19.0.0-3154.g9ed025f 5.4 MB/s | 501 kB 00:00 2024-05-30T16:40:48.637 INFO:teuthology.orchestra.run.smithi067.stdout:(19/159): python3-ceph-argparse-19.0.0-3154.g9e 547 kB/s | 45 kB 00:00 2024-05-30T16:40:48.729 INFO:teuthology.orchestra.run.smithi067.stdout:(20/159): python3-ceph-common-19.0.0-3154.g9ed0 1.5 MB/s | 135 kB 00:00 2024-05-30T16:40:48.888 INFO:teuthology.orchestra.run.smithi067.stdout:(21/159): librgw2-19.0.0-3154.g9ed025fb.el9.x86 14 MB/s | 5.4 MB 00:00 2024-05-30T16:40:48.930 INFO:teuthology.orchestra.run.smithi067.stdout:(22/159): python3-cephfs-19.0.0-3154.g9ed025fb. 809 kB/s | 161 kB 00:00 2024-05-30T16:40:48.972 INFO:teuthology.orchestra.run.smithi067.stdout:(23/159): python3-rados-19.0.0-3154.g9ed025fb.e 3.8 MB/s | 321 kB 00:00 2024-05-30T16:40:49.014 INFO:teuthology.orchestra.run.smithi067.stdout:(24/159): python3-rbd-19.0.0-3154.g9ed025fb.el9 3.5 MB/s | 298 kB 00:00 2024-05-30T16:40:49.015 INFO:teuthology.orchestra.run.smithi169.stdout:(2/159): ceph-fuse-19.0.0-3154.g9ed025fb.el9.x8 1.8 MB/s | 890 kB 00:00 2024-05-30T16:40:49.047 INFO:teuthology.orchestra.run.smithi067.stdout:(25/159): python3-rgw-19.0.0-3154.g9ed025fb.el9 1.3 MB/s | 100 kB 00:00 2024-05-30T16:40:49.081 INFO:teuthology.orchestra.run.smithi067.stdout:(26/159): rbd-fuse-19.0.0-3154.g9ed025fb.el9.x8 1.3 MB/s | 85 kB 00:00 2024-05-30T16:40:49.149 INFO:teuthology.orchestra.run.smithi169.stdout:(3/159): ceph-immutable-object-cache-19.0.0-315 1.1 MB/s | 145 kB 00:00 2024-05-30T16:40:49.348 INFO:teuthology.orchestra.run.smithi067.stdout:(27/159): rbd-mirror-19.0.0-3154.g9ed025fb.el9. 10 MB/s | 3.1 MB 00:00 2024-05-30T16:40:49.398 INFO:teuthology.orchestra.run.smithi067.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3154.g 703 kB/s | 34 kB 00:00 2024-05-30T16:40:49.465 INFO:teuthology.orchestra.run.smithi067.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3154.g9ed025f 2.1 MB/s | 142 kB 00:00 2024-05-30T16:40:49.507 INFO:teuthology.orchestra.run.smithi067.stdout:(30/159): rbd-nbd-19.0.0-3154.g9ed025fb.el9.x86 401 kB/s | 170 kB 00:00 2024-05-30T16:40:49.824 INFO:teuthology.orchestra.run.smithi067.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3154.g9ed02 9.9 MB/s | 3.5 MB 00:00 2024-05-30T16:40:49.908 INFO:teuthology.orchestra.run.smithi067.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3154.g9e 3.0 MB/s | 252 kB 00:00 2024-05-30T16:40:49.991 INFO:teuthology.orchestra.run.smithi067.stdout:(33/159): ceph-mgr-rook-19.0.0-3154.g9ed025fb.e 598 kB/s | 49 kB 00:00 2024-05-30T16:40:50.066 INFO:teuthology.orchestra.run.smithi067.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3154.g9 223 kB/s | 16 kB 00:00 2024-05-30T16:40:50.159 INFO:teuthology.orchestra.run.smithi067.stdout:(35/159): ceph-volume-19.0.0-3154.g9ed025fb.el9 2.9 MB/s | 273 kB 00:00 2024-05-30T16:40:50.275 INFO:teuthology.orchestra.run.smithi067.stdout:(36/159): cephadm-19.0.0-3154.g9ed025fb.el9.noa 6.5 MB/s | 762 kB 00:00 2024-05-30T16:40:50.351 INFO:teuthology.orchestra.run.smithi067.stdout:(37/159): cephfs-top-19.0.0-3154.g9ed025fb.el9. 244 kB/s | 18 kB 00:00 2024-05-30T16:40:50.476 INFO:teuthology.orchestra.run.smithi067.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 650 kB/s | 80 kB 00:00 2024-05-30T16:40:50.599 INFO:teuthology.orchestra.run.smithi169.stdout:(4/159): ceph-mds-19.0.0-3154.g9ed025fb.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-05-30T16:40:50.601 INFO:teuthology.orchestra.run.smithi067.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-05-30T16:40:50.736 INFO:teuthology.orchestra.run.smithi067.stdout:(40/159): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-05-30T16:40:50.768 INFO:teuthology.orchestra.run.smithi067.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-05-30T16:40:51.002 INFO:teuthology.orchestra.run.smithi067.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.2 MB 00:00 2024-05-30T16:40:51.033 INFO:teuthology.orchestra.run.smithi169.stdout:(5/159): ceph-mgr-19.0.0-3154.g9ed025fb.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-05-30T16:40:51.036 INFO:teuthology.orchestra.run.smithi067.stdout:(43/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 848 kB/s | 253 kB 00:00 2024-05-30T16:40:51.070 INFO:teuthology.orchestra.run.smithi067.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-30T16:40:51.103 INFO:teuthology.orchestra.run.smithi067.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-30T16:40:51.129 INFO:teuthology.orchestra.run.smithi067.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-30T16:40:51.162 INFO:teuthology.orchestra.run.smithi067.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-30T16:40:51.308 INFO:teuthology.orchestra.run.smithi169.stdout:(6/159): ceph-base-19.0.0-3154.g9ed025fb.el9.x8 1.8 MB/s | 5.4 MB 00:03 2024-05-30T16:40:51.825 INFO:teuthology.orchestra.run.smithi169.stdout:(7/159): ceph-common-19.0.0-3154.g9ed025fb.el9. 5.9 MB/s | 21 MB 00:03 2024-05-30T16:40:51.946 INFO:teuthology.orchestra.run.smithi067.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 845 kB/s | 689 kB 00:00 2024-05-30T16:40:52.055 INFO:teuthology.orchestra.run.smithi067.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-05-30T16:40:52.088 INFO:teuthology.orchestra.run.smithi067.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 758 kB/s | 107 kB 00:00 2024-05-30T16:40:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 202 kB/s | 22 kB 00:00 2024-05-30T16:40:52.206 INFO:teuthology.orchestra.run.smithi067.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-30T16:40:52.272 INFO:teuthology.orchestra.run.smithi067.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-05-30T16:40:52.381 INFO:teuthology.orchestra.run.smithi067.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 169 kB/s | 18 kB 00:00 2024-05-30T16:40:52.490 INFO:teuthology.orchestra.run.smithi067.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 104 kB/s | 11 kB 00:00 2024-05-30T16:40:52.592 INFO:teuthology.orchestra.run.smithi169.stdout:(8/159): ceph-mon-19.0.0-3154.g9ed025fb.el9.x86 3.0 MB/s | 4.7 MB 00:01 2024-05-30T16:40:52.623 INFO:teuthology.orchestra.run.smithi067.stdout:(56/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.2 MB/s | 3.0 MB 00:00 2024-05-30T16:40:52.657 INFO:teuthology.orchestra.run.smithi067.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 965 kB/s | 160 kB 00:00 2024-05-30T16:40:52.692 INFO:teuthology.orchestra.run.smithi169.stdout:(9/159): ceph-selinux-19.0.0-3154.g9ed025fb.el9 254 kB/s | 25 kB 00:00 2024-05-30T16:40:52.732 INFO:teuthology.orchestra.run.smithi067.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-05-30T16:40:52.858 INFO:teuthology.orchestra.run.smithi067.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 582 kB 00:00 2024-05-30T16:40:52.892 INFO:teuthology.orchestra.run.smithi067.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-05-30T16:40:53.034 INFO:teuthology.orchestra.run.smithi169.stdout:(10/159): ceph-radosgw-19.0.0-3154.g9ed025fb.el 8.8 MB/s | 11 MB 00:01 2024-05-30T16:40:53.101 INFO:teuthology.orchestra.run.smithi169.stdout:(11/159): cephfs-mirror-19.0.0-3154.g9ed025fb.e 3.4 MB/s | 225 kB 00:00 2024-05-30T16:40:53.176 INFO:teuthology.orchestra.run.smithi169.stdout:(12/159): cephfs-shell-19.0.0-3154.g9ed025fb.el 365 kB/s | 27 kB 00:00 2024-05-30T16:40:53.337 INFO:teuthology.orchestra.run.smithi169.stdout:(13/159): libcephfs-devel-19.0.0-3154.g9ed025fb 199 kB/s | 32 kB 00:00 2024-05-30T16:40:53.425 INFO:teuthology.orchestra.run.smithi067.stdout:(61/159): ceph-test-19.0.0-3154.g9ed025fb.el9.x 9.0 MB/s | 49 MB 00:05 2024-05-30T16:40:53.443 INFO:teuthology.orchestra.run.smithi169.stdout:(14/159): libcephfs2-19.0.0-3154.g9ed025fb.el9. 6.9 MB/s | 746 kB 00:00 2024-05-30T16:40:53.450 INFO:teuthology.orchestra.run.smithi067.stdout:(62/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 43 kB/s | 25 kB 00:00 2024-05-30T16:40:53.475 INFO:teuthology.orchestra.run.smithi067.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-05-30T16:40:53.503 INFO:teuthology.orchestra.run.smithi169.stdout:(15/159): libcephsqlite-19.0.0-3154.g9ed025fb.e 2.7 MB/s | 164 kB 00:00 2024-05-30T16:40:53.553 INFO:teuthology.orchestra.run.smithi169.stdout:(16/159): librados-devel-19.0.0-3154.g9ed025fb. 2.5 MB/s | 127 kB 00:00 2024-05-30T16:40:53.567 INFO:teuthology.orchestra.run.smithi067.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-30T16:40:53.592 INFO:teuthology.orchestra.run.smithi067.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 303 kB/s | 35 kB 00:00 2024-05-30T16:40:53.636 INFO:teuthology.orchestra.run.smithi169.stdout:(17/159): libradosstriper1-19.0.0-3154.g9ed025f 5.9 MB/s | 501 kB 00:00 2024-05-30T16:40:54.101 INFO:teuthology.orchestra.run.smithi067.stdout:(66/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 433 kB/s | 292 kB 00:00 2024-05-30T16:40:54.427 INFO:teuthology.orchestra.run.smithi067.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-05-30T16:40:54.487 INFO:teuthology.orchestra.run.smithi169.stdout:(18/159): librgw2-19.0.0-3154.g9ed025fb.el9.x86 6.3 MB/s | 5.4 MB 00:00 2024-05-30T16:40:54.518 INFO:teuthology.orchestra.run.smithi067.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-05-30T16:40:54.568 INFO:teuthology.orchestra.run.smithi067.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-05-30T16:40:54.570 INFO:teuthology.orchestra.run.smithi169.stdout:(19/159): python3-ceph-argparse-19.0.0-3154.g9e 548 kB/s | 45 kB 00:00 2024-05-30T16:40:54.620 INFO:teuthology.orchestra.run.smithi169.stdout:(20/159): python3-ceph-common-19.0.0-3154.g9ed0 2.7 MB/s | 135 kB 00:00 2024-05-30T16:40:54.627 INFO:teuthology.orchestra.run.smithi067.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-30T16:40:54.671 INFO:teuthology.orchestra.run.smithi169.stdout:(21/159): python3-cephfs-19.0.0-3154.g9ed025fb. 3.2 MB/s | 161 kB 00:00 2024-05-30T16:40:54.738 INFO:teuthology.orchestra.run.smithi169.stdout:(22/159): python3-rados-19.0.0-3154.g9ed025fb.e 4.8 MB/s | 321 kB 00:00 2024-05-30T16:40:54.796 INFO:teuthology.orchestra.run.smithi169.stdout:(23/159): python3-rbd-19.0.0-3154.g9ed025fb.el9 5.1 MB/s | 298 kB 00:00 2024-05-30T16:40:54.844 INFO:teuthology.orchestra.run.smithi067.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-05-30T16:40:54.846 INFO:teuthology.orchestra.run.smithi169.stdout:(24/159): python3-rgw-19.0.0-3154.g9ed025fb.el9 2.0 MB/s | 100 kB 00:00 2024-05-30T16:40:54.897 INFO:teuthology.orchestra.run.smithi169.stdout:(25/159): rbd-fuse-19.0.0-3154.g9ed025fb.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-30T16:40:54.953 INFO:teuthology.orchestra.run.smithi067.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 815 kB/s | 87 kB 00:00 2024-05-30T16:40:55.181 INFO:teuthology.orchestra.run.smithi169.stdout:(26/159): rbd-mirror-19.0.0-3154.g9ed025fb.el9. 11 MB/s | 3.1 MB 00:00 2024-05-30T16:40:55.239 INFO:teuthology.orchestra.run.smithi169.stdout:(27/159): rbd-nbd-19.0.0-3154.g9ed025fb.el9.x86 2.9 MB/s | 170 kB 00:00 2024-05-30T16:40:55.306 INFO:teuthology.orchestra.run.smithi169.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3154.g 519 kB/s | 34 kB 00:00 2024-05-30T16:40:55.353 INFO:teuthology.orchestra.run.smithi067.stdout:(73/159): openblas-openmp-0.3.21-2.el9.x86_64.r 2.7 MB/s | 4.7 MB 00:01 2024-05-30T16:40:55.356 INFO:teuthology.orchestra.run.smithi169.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3154.g9ed025f 2.8 MB/s | 142 kB 00:00 2024-05-30T16:40:55.478 INFO:teuthology.orchestra.run.smithi067.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 494 kB/s | 61 kB 00:00 2024-05-30T16:40:55.674 INFO:teuthology.orchestra.run.smithi169.stdout:(30/159): ceph-mgr-dashboard-19.0.0-3154.g9ed02 11 MB/s | 3.5 MB 00:00 2024-05-30T16:40:55.828 INFO:teuthology.orchestra.run.smithi067.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 700 kB/s | 244 kB 00:00 2024-05-30T16:40:55.946 INFO:teuthology.orchestra.run.smithi067.stdout:(76/159): postgresql-server-13.14-1.el9.x86_64. 4.2 MB/s | 5.8 MB 00:01 2024-05-30T16:40:56.046 INFO:teuthology.orchestra.run.smithi067.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 259 kB/s | 26 kB 00:00 2024-05-30T16:40:56.172 INFO:teuthology.orchestra.run.smithi067.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 152 kB/s | 19 kB 00:00 2024-05-30T16:40:56.213 INFO:teuthology.orchestra.run.smithi067.stdout:(79/159): python3-jinja2-2.11.3-6.el9.noarch.rp 646 kB/s | 248 kB 00:00 2024-05-30T16:40:56.280 INFO:teuthology.orchestra.run.smithi067.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-05-30T16:40:56.290 INFO:teuthology.orchestra.run.smithi169.stdout:(31/159): ceph-osd-19.0.0-3154.g9ed025fb.el9.x8 3.4 MB/s | 17 MB 00:04 2024-05-30T16:40:56.322 INFO:teuthology.orchestra.run.smithi067.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-05-30T16:40:56.449 INFO:teuthology.orchestra.run.smithi169.stdout:(32/159): ceph-mgr-diskprediction-local-19.0.0- 9.6 MB/s | 7.4 MB 00:00 2024-05-30T16:40:56.483 INFO:teuthology.orchestra.run.smithi169.stdout:(33/159): ceph-mgr-modules-core-19.0.0-3154.g9e 1.3 MB/s | 252 kB 00:00 2024-05-30T16:40:56.508 INFO:teuthology.orchestra.run.smithi169.stdout:(34/159): ceph-mgr-rook-19.0.0-3154.g9ed025fb.e 859 kB/s | 49 kB 00:00 2024-05-30T16:40:56.550 INFO:teuthology.orchestra.run.smithi169.stdout:(35/159): ceph-prometheus-alerts-19.0.0-3154.g9 249 kB/s | 16 kB 00:00 2024-05-30T16:40:56.600 INFO:teuthology.orchestra.run.smithi169.stdout:(36/159): ceph-volume-19.0.0-3154.g9ed025fb.el9 2.9 MB/s | 273 kB 00:00 2024-05-30T16:40:56.651 INFO:teuthology.orchestra.run.smithi169.stdout:(37/159): cephfs-top-19.0.0-3154.g9ed025fb.el9. 372 kB/s | 18 kB 00:00 2024-05-30T16:40:56.680 INFO:teuthology.orchestra.run.smithi067.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 700 kB/s | 250 kB 00:00 2024-05-30T16:40:56.793 INFO:teuthology.orchestra.run.smithi169.stdout:(38/159): cephadm-19.0.0-3154.g9ed025fb.el9.noa 3.1 MB/s | 762 kB 00:00 2024-05-30T16:40:57.060 INFO:teuthology.orchestra.run.smithi169.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 197 kB/s | 80 kB 00:00 2024-05-30T16:40:57.089 INFO:teuthology.orchestra.run.smithi067.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 545 kB/s | 222 kB 00:00 2024-05-30T16:40:57.251 INFO:teuthology.orchestra.run.smithi169.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-30T16:40:57.281 INFO:teuthology.orchestra.run.smithi067.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 404 kB/s | 77 kB 00:00 2024-05-30T16:40:57.390 INFO:teuthology.orchestra.run.smithi067.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 393 kB/s | 42 kB 00:00 2024-05-30T16:40:57.443 INFO:teuthology.orchestra.run.smithi169.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-05-30T16:40:57.593 INFO:teuthology.orchestra.run.smithi169.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-05-30T16:40:57.666 INFO:teuthology.orchestra.run.smithi067.stdout:(86/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.7 MB/s | 5.1 MB 00:01 2024-05-30T16:40:57.710 INFO:teuthology.orchestra.run.smithi169.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 914 kB/s | 106 kB 00:00 2024-05-30T16:40:57.774 INFO:teuthology.orchestra.run.smithi067.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-05-30T16:40:57.836 INFO:teuthology.orchestra.run.smithi169.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-05-30T16:40:57.857 INFO:teuthology.orchestra.run.smithi067.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 577 kB/s | 269 kB 00:00 2024-05-30T16:40:57.861 INFO:teuthology.orchestra.run.smithi169.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 904 kB/s | 135 kB 00:00 2024-05-30T16:40:57.891 INFO:teuthology.orchestra.run.smithi067.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-05-30T16:40:57.953 INFO:teuthology.orchestra.run.smithi169.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-05-30T16:40:57.966 INFO:teuthology.orchestra.run.smithi067.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 473 kB/s | 51 kB 00:00 2024-05-30T16:40:57.987 INFO:teuthology.orchestra.run.smithi169.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-05-30T16:40:58.000 INFO:teuthology.orchestra.run.smithi067.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 497 kB/s | 54 kB 00:00 2024-05-30T16:40:58.078 INFO:teuthology.orchestra.run.smithi169.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-05-30T16:40:58.117 INFO:teuthology.orchestra.run.smithi067.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 358 kB/s | 42 kB 00:00 2024-05-30T16:40:58.187 INFO:teuthology.orchestra.run.smithi169.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 990 kB/s | 107 kB 00:00 2024-05-30T16:40:58.226 INFO:teuthology.orchestra.run.smithi067.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 410 kB/s | 44 kB 00:00 2024-05-30T16:40:58.262 INFO:teuthology.orchestra.run.smithi169.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-05-30T16:40:58.296 INFO:teuthology.orchestra.run.smithi169.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 201 kB/s | 22 kB 00:00 2024-05-30T16:40:58.334 INFO:teuthology.orchestra.run.smithi067.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-30T16:40:58.421 INFO:teuthology.orchestra.run.smithi169.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-05-30T16:40:58.443 INFO:teuthology.orchestra.run.smithi067.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 334 kB/s | 36 kB 00:00 2024-05-30T16:40:58.455 INFO:teuthology.orchestra.run.smithi169.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 206 kB/s | 33 kB 00:00 2024-05-30T16:40:58.551 INFO:teuthology.orchestra.run.smithi067.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 439 kB/s | 47 kB 00:00 2024-05-30T16:40:58.555 INFO:teuthology.orchestra.run.smithi169.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 183 kB/s | 18 kB 00:00 2024-05-30T16:40:58.664 INFO:teuthology.orchestra.run.smithi169.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-05-30T16:40:58.668 INFO:teuthology.orchestra.run.smithi067.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-05-30T16:40:58.777 INFO:teuthology.orchestra.run.smithi067.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 537 kB/s | 58 kB 00:00 2024-05-30T16:40:58.781 INFO:teuthology.orchestra.run.smithi169.stdout:(56/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.3 MB/s | 3.0 MB 00:00 2024-05-30T16:40:58.815 INFO:teuthology.orchestra.run.smithi169.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-05-30T16:40:58.886 INFO:teuthology.orchestra.run.smithi067.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 870 kB/s | 94 kB 00:00 2024-05-30T16:40:58.898 INFO:teuthology.orchestra.run.smithi169.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 390 kB/s | 45 kB 00:00 2024-05-30T16:40:58.941 INFO:teuthology.orchestra.run.smithi169.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-05-30T16:40:58.994 INFO:teuthology.orchestra.run.smithi067.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 592 kB/s | 64 kB 00:00 2024-05-30T16:40:59.024 INFO:teuthology.orchestra.run.smithi169.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 582 kB 00:00 2024-05-30T16:40:59.041 INFO:teuthology.orchestra.run.smithi169.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-05-30T16:40:59.103 INFO:teuthology.orchestra.run.smithi067.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-05-30T16:40:59.125 INFO:teuthology.orchestra.run.smithi169.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 478 kB/s | 47 kB 00:00 2024-05-30T16:40:59.158 INFO:teuthology.orchestra.run.smithi169.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-05-30T16:40:59.211 INFO:teuthology.orchestra.run.smithi067.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-05-30T16:40:59.234 INFO:teuthology.orchestra.run.smithi169.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-30T16:40:59.267 INFO:teuthology.orchestra.run.smithi169.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 326 kB/s | 35 kB 00:00 2024-05-30T16:40:59.362 INFO:teuthology.orchestra.run.smithi067.stdout:(103/159): python3-babel-2.9.1-2.el9.noarch.rpm 1.4 MB/s | 6.0 MB 00:04 2024-05-30T16:40:59.545 INFO:teuthology.orchestra.run.smithi067.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-05-30T16:40:59.693 INFO:teuthology.orchestra.run.smithi169.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-05-30T16:40:59.896 INFO:teuthology.orchestra.run.smithi067.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-30T16:40:59.934 INFO:teuthology.orchestra.run.smithi169.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-05-30T16:40:59.996 INFO:teuthology.orchestra.run.smithi067.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 687 kB/s | 309 kB 00:00 2024-05-30T16:41:00.021 INFO:teuthology.orchestra.run.smithi067.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-05-30T16:41:00.076 INFO:teuthology.orchestra.run.smithi169.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-05-30T16:41:00.122 INFO:teuthology.orchestra.run.smithi067.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-05-30T16:41:00.185 INFO:teuthology.orchestra.run.smithi169.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-30T16:41:00.222 INFO:teuthology.orchestra.run.smithi067.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 489 kB/s | 49 kB 00:00 2024-05-30T16:41:00.331 INFO:teuthology.orchestra.run.smithi067.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 625 kB/s | 67 kB 00:00 2024-05-30T16:41:00.423 INFO:teuthology.orchestra.run.smithi067.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-30T16:41:00.456 INFO:teuthology.orchestra.run.smithi067.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-05-30T16:41:00.542 INFO:teuthology.orchestra.run.smithi067.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 839 kB 00:00 2024-05-30T16:41:00.582 INFO:teuthology.orchestra.run.smithi067.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-30T16:41:00.632 INFO:teuthology.orchestra.run.smithi067.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 476 kB/s | 43 kB 00:00 2024-05-30T16:41:00.782 INFO:teuthology.orchestra.run.smithi067.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 162 kB/s | 32 kB 00:00 2024-05-30T16:41:00.807 INFO:teuthology.orchestra.run.smithi067.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 86 kB/s | 15 kB 00:00 2024-05-30T16:41:00.883 INFO:teuthology.orchestra.run.smithi067.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-30T16:41:00.916 INFO:teuthology.orchestra.run.smithi067.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-30T16:41:00.969 INFO:teuthology.orchestra.run.smithi169.stdout:(70/159): postgresql-server-13.14-1.el9.x86_64. 7.4 MB/s | 5.8 MB 00:00 2024-05-30T16:41:01.000 INFO:teuthology.orchestra.run.smithi067.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-05-30T16:41:01.034 INFO:teuthology.orchestra.run.smithi067.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-30T16:41:01.134 INFO:teuthology.orchestra.run.smithi067.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-30T16:41:01.218 INFO:teuthology.orchestra.run.smithi067.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-05-30T16:41:01.243 INFO:teuthology.orchestra.run.smithi067.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-05-30T16:41:01.318 INFO:teuthology.orchestra.run.smithi067.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 178 kB/s | 18 kB 00:00 2024-05-30T16:41:01.343 INFO:teuthology.orchestra.run.smithi067.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-05-30T16:41:01.419 INFO:teuthology.orchestra.run.smithi067.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 196 kB/s | 19 kB 00:00 2024-05-30T16:41:01.444 INFO:teuthology.orchestra.run.smithi067.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-05-30T16:41:01.544 INFO:teuthology.orchestra.run.smithi067.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-05-30T16:41:01.578 INFO:teuthology.orchestra.run.smithi067.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 348 kB/s | 46 kB 00:00 2024-05-30T16:41:01.645 INFO:teuthology.orchestra.run.smithi067.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 793 kB/s | 79 kB 00:00 2024-05-30T16:41:01.670 INFO:teuthology.orchestra.run.smithi067.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 630 kB/s | 58 kB 00:00 2024-05-30T16:41:01.677 INFO:teuthology.orchestra.run.smithi169.stdout:(71/159): ceph-test-19.0.0-3154.g9ed025fb.el9.x 5.5 MB/s | 49 MB 00:08 2024-05-30T16:41:01.754 INFO:teuthology.orchestra.run.smithi067.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-30T16:41:01.780 INFO:teuthology.orchestra.run.smithi067.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-05-30T16:41:01.786 INFO:teuthology.orchestra.run.smithi169.stdout:(72/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 4.4 MB 00:02 2024-05-30T16:41:01.855 INFO:teuthology.orchestra.run.smithi067.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 901 kB/s | 90 kB 00:00 2024-05-30T16:41:01.888 INFO:teuthology.orchestra.run.smithi067.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 261 kB/s | 28 kB 00:00 2024-05-30T16:41:01.928 INFO:teuthology.orchestra.run.smithi169.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-05-30T16:41:01.963 INFO:teuthology.orchestra.run.smithi067.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 285 kB/s | 31 kB 00:00 2024-05-30T16:41:01.997 INFO:teuthology.orchestra.run.smithi067.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-30T16:41:02.053 INFO:teuthology.orchestra.run.smithi169.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 493 kB/s | 61 kB 00:00 2024-05-30T16:41:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 548 kB/s | 59 kB 00:00 2024-05-30T16:41:02.114 INFO:teuthology.orchestra.run.smithi067.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 307 kB/s | 36 kB 00:00 2024-05-30T16:41:02.173 INFO:teuthology.orchestra.run.smithi067.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 513 kB/s | 51 kB 00:00 2024-05-30T16:41:02.186 INFO:teuthology.orchestra.run.smithi169.stdout:(75/159): python3-attrs-20.3.0-7.el9.noarch.rpm 172 kB/s | 87 kB 00:00 2024-05-30T16:41:02.220 INFO:teuthology.orchestra.run.smithi169.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-05-30T16:41:02.337 INFO:teuthology.orchestra.run.smithi169.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 221 kB/s | 26 kB 00:00 2024-05-30T16:41:02.399 INFO:teuthology.orchestra.run.smithi067.stdout:(142/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.7 MB/s | 16 MB 00:04 2024-05-30T16:41:02.424 INFO:teuthology.orchestra.run.smithi067.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 359 kB/s | 90 kB 00:00 2024-05-30T16:41:02.457 INFO:teuthology.orchestra.run.smithi067.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 672 kB/s | 230 kB 00:00 2024-05-30T16:41:02.462 INFO:teuthology.orchestra.run.smithi169.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 154 kB/s | 19 kB 00:00 2024-05-30T16:41:02.504 INFO:teuthology.orchestra.run.smithi169.stdout:(79/159): python3-jinja2-2.11.3-6.el9.noarch.rp 783 kB/s | 248 kB 00:00 2024-05-30T16:41:02.588 INFO:teuthology.orchestra.run.smithi169.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-05-30T16:41:02.614 INFO:teuthology.orchestra.run.smithi169.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-05-30T16:41:02.666 INFO:teuthology.orchestra.run.smithi067.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 83 kB/s | 20 kB 00:00 2024-05-30T16:41:02.699 INFO:teuthology.orchestra.run.smithi067.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 794 kB/s | 192 kB 00:00 2024-05-30T16:41:02.722 INFO:teuthology.orchestra.run.smithi169.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-30T16:41:02.831 INFO:teuthology.orchestra.run.smithi169.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-30T16:41:02.939 INFO:teuthology.orchestra.run.smithi169.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 716 kB/s | 77 kB 00:00 2024-05-30T16:41:03.040 INFO:teuthology.orchestra.run.smithi169.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 425 kB/s | 42 kB 00:00 2024-05-30T16:41:03.049 INFO:teuthology.orchestra.run.smithi067.stdout:(147/159): librados2-19.0.0-3154.g9ed025fb.el9. 9.7 MB/s | 3.4 MB 00:00 2024-05-30T16:41:03.100 INFO:teuthology.orchestra.run.smithi067.stdout:(148/159): python3-werkzeug-2.0.3-3.el9.1.noarc 610 kB/s | 427 kB 00:00 2024-05-30T16:41:03.157 INFO:teuthology.orchestra.run.smithi169.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-05-30T16:41:03.274 INFO:teuthology.orchestra.run.smithi169.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-05-30T16:41:03.359 INFO:teuthology.orchestra.run.smithi067.stdout:(149/159): librbd1-19.0.0-3154.g9ed025fb.el9.x8 10 MB/s | 3.1 MB 00:00 2024-05-30T16:41:03.399 INFO:teuthology.orchestra.run.smithi169.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-05-30T16:41:03.409 INFO:teuthology.orchestra.run.smithi067.stdout:(150/159): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-05-30T16:41:03.484 INFO:teuthology.orchestra.run.smithi067.stdout:(151/159): NetworkManager-1.47.91-1.el9.x86_64. 6.0 MB/s | 2.3 MB 00:00 2024-05-30T16:41:03.543 INFO:teuthology.orchestra.run.smithi067.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-05-30T16:41:03.568 INFO:teuthology.orchestra.run.smithi067.stdout:(153/159): NetworkManager-team-1.47.91-1.el9.x8 262 kB/s | 41 kB 00:00 2024-05-30T16:41:03.593 INFO:teuthology.orchestra.run.smithi067.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 742 kB/s | 37 kB 00:00 2024-05-30T16:41:03.643 INFO:teuthology.orchestra.run.smithi067.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.2 MB/s | 260 kB 00:00 2024-05-30T16:41:03.649 INFO:teuthology.orchestra.run.smithi169.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 204 kB/s | 51 kB 00:00 2024-05-30T16:41:03.677 INFO:teuthology.orchestra.run.smithi067.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-05-30T16:41:03.744 INFO:teuthology.orchestra.run.smithi067.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 14 MB/s | 1.3 MB 00:00 2024-05-30T16:41:03.750 INFO:teuthology.orchestra.run.smithi169.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 541 kB/s | 54 kB 00:00 2024-05-30T16:41:03.803 INFO:teuthology.orchestra.run.smithi067.stdout:(158/159): NetworkManager-libnm-1.47.91-1.el9.x 4.1 MB/s | 1.8 MB 00:00 2024-05-30T16:41:04.020 INFO:teuthology.orchestra.run.smithi067.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 2.4 MB 00:00 2024-05-30T16:41:04.024 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-05-30T16:41:04.025 INFO:teuthology.orchestra.run.smithi067.stdout:Total 12 MB/s | 227 MB 00:19 2024-05-30T16:41:05.817 INFO:teuthology.orchestra.run.smithi169.stdout:(91/159): python3-babel-2.9.1-2.el9.noarch.rpm 1.5 MB/s | 6.0 MB 00:04 2024-05-30T16:41:05.925 INFO:teuthology.orchestra.run.smithi169.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 388 kB/s | 42 kB 00:00 2024-05-30T16:41:06.034 INFO:teuthology.orchestra.run.smithi169.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 410 kB/s | 44 kB 00:00 2024-05-30T16:41:06.376 INFO:teuthology.orchestra.run.smithi169.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 648 kB/s | 221 kB 00:00 2024-05-30T16:41:06.498 INFO:teuthology.orchestra.run.smithi169.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 295 kB/s | 36 kB 00:00 2024-05-30T16:41:06.609 INFO:teuthology.orchestra.run.smithi169.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 430 kB/s | 47 kB 00:00 2024-05-30T16:41:07.035 INFO:teuthology.orchestra.run.smithi169.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 718 kB/s | 305 kB 00:00 2024-05-30T16:41:07.144 INFO:teuthology.orchestra.run.smithi169.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 536 kB/s | 58 kB 00:00 2024-05-30T16:41:07.335 INFO:teuthology.orchestra.run.smithi169.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 490 kB/s | 94 kB 00:00 2024-05-30T16:41:07.368 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-05-30T16:41:07.519 INFO:teuthology.orchestra.run.smithi169.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 349 kB/s | 64 kB 00:00 2024-05-30T16:41:07.628 INFO:teuthology.orchestra.run.smithi169.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-05-30T16:41:07.639 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-05-30T16:41:07.639 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-05-30T16:41:07.736 INFO:teuthology.orchestra.run.smithi169.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-05-30T16:41:08.912 INFO:teuthology.orchestra.run.smithi169.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 735 kB/s | 863 kB 00:01 2024-05-30T16:41:09.337 INFO:teuthology.orchestra.run.smithi169.stdout:(104/159): python3-numpy-1.20.1-5.el9.x86_64.rp 773 kB/s | 5.1 MB 00:06 2024-05-30T16:41:09.470 INFO:teuthology.orchestra.run.smithi169.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 983 kB/s | 548 kB 00:00 2024-05-30T16:41:09.579 INFO:teuthology.orchestra.run.smithi169.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-05-30T16:41:09.796 INFO:teuthology.orchestra.run.smithi169.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 673 kB/s | 309 kB 00:00 2024-05-30T16:41:09.896 INFO:teuthology.orchestra.run.smithi169.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 264 kB/s | 26 kB 00:00 2024-05-30T16:41:09.988 INFO:teuthology.orchestra.run.smithi169.stdout:(109/159): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-30T16:41:10.014 INFO:teuthology.orchestra.run.smithi169.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 419 kB/s | 49 kB 00:00 2024-05-30T16:41:10.089 INFO:teuthology.orchestra.run.smithi169.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 678 kB/s | 67 kB 00:00 2024-05-30T16:41:10.123 INFO:teuthology.orchestra.run.smithi169.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-30T16:41:10.206 INFO:teuthology.orchestra.run.smithi169.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 839 kB 00:00 2024-05-30T16:41:10.301 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-05-30T16:41:10.301 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-05-30T16:41:10.306 INFO:teuthology.orchestra.run.smithi169.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 438 kB/s | 43 kB 00:00 2024-05-30T16:41:10.340 INFO:teuthology.orchestra.run.smithi169.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-05-30T16:41:10.399 INFO:teuthology.orchestra.run.smithi169.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 354 kB/s | 32 kB 00:00 2024-05-30T16:41:10.432 INFO:teuthology.orchestra.run.smithi169.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-05-30T16:41:10.508 INFO:teuthology.orchestra.run.smithi169.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-05-30T16:41:10.549 INFO:teuthology.orchestra.run.smithi169.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-30T16:41:10.608 INFO:teuthology.orchestra.run.smithi169.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-30T16:41:10.650 INFO:teuthology.orchestra.run.smithi169.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-30T16:41:10.750 INFO:teuthology.orchestra.run.smithi169.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-30T16:41:10.867 INFO:teuthology.orchestra.run.smithi169.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-05-30T16:41:10.926 INFO:teuthology.orchestra.run.smithi169.stdout:(124/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-05-30T16:41:10.959 INFO:teuthology.orchestra.run.smithi169.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 194 kB/s | 18 kB 00:00 2024-05-30T16:41:11.018 INFO:teuthology.orchestra.run.smithi169.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 253 kB/s | 23 kB 00:00 2024-05-30T16:41:11.052 INFO:teuthology.orchestra.run.smithi169.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 213 kB/s | 19 kB 00:00 2024-05-30T16:41:11.110 INFO:teuthology.orchestra.run.smithi169.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2024-05-30T16:41:11.202 INFO:teuthology.orchestra.run.smithi169.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 508 kB/s | 46 kB 00:00 2024-05-30T16:41:11.253 INFO:teuthology.orchestra.run.smithi169.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-05-30T16:41:11.303 INFO:teuthology.orchestra.run.smithi169.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 792 kB/s | 79 kB 00:00 2024-05-30T16:41:11.336 INFO:teuthology.orchestra.run.smithi169.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 696 kB/s | 58 kB 00:00 2024-05-30T16:41:11.412 INFO:teuthology.orchestra.run.smithi169.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-30T16:41:11.429 INFO:teuthology.orchestra.run.smithi169.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 181 kB/s | 16 kB 00:00 2024-05-30T16:41:11.513 INFO:teuthology.orchestra.run.smithi169.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 901 kB/s | 90 kB 00:00 2024-05-30T16:41:11.530 INFO:teuthology.orchestra.run.smithi169.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-05-30T16:41:11.596 INFO:teuthology.orchestra.run.smithi169.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 372 kB/s | 31 kB 00:00 2024-05-30T16:41:11.630 INFO:teuthology.orchestra.run.smithi169.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-30T16:41:11.689 INFO:teuthology.orchestra.run.smithi169.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 648 kB/s | 59 kB 00:00 2024-05-30T16:41:11.722 INFO:teuthology.orchestra.run.smithi169.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 391 kB/s | 36 kB 00:00 2024-05-30T16:41:11.781 INFO:teuthology.orchestra.run.smithi169.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 559 kB/s | 51 kB 00:00 2024-05-30T16:41:11.823 INFO:teuthology.orchestra.run.smithi169.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2024-05-30T16:41:11.865 INFO:teuthology.orchestra.run.smithi169.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-05-30T16:41:11.963 INFO:teuthology.orchestra.run.smithi169.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.0 MB/s | 427 kB 00:00 2024-05-30T16:41:11.990 INFO:teuthology.orchestra.run.smithi169.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-05-30T16:41:12.066 INFO:teuthology.orchestra.run.smithi169.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 1.8 MB/s | 192 kB 00:00 2024-05-30T16:41:12.808 INFO:teuthology.orchestra.run.smithi169.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-05-30T16:41:13.483 INFO:teuthology.orchestra.run.smithi169.stdout:(148/159): librados2-19.0.0-3154.g9ed025fb.el9. 2.4 MB/s | 3.4 MB 00:01 2024-05-30T16:41:14.359 INFO:teuthology.orchestra.run.smithi169.stdout:(149/159): librbd1-19.0.0-3154.g9ed025fb.el9.x8 2.0 MB/s | 3.1 MB 00:01 2024-05-30T16:41:14.442 INFO:teuthology.orchestra.run.smithi169.stdout:(150/159): NetworkManager-1.47.91-1.el9.x86_64. 2.4 MB/s | 2.3 MB 00:00 2024-05-30T16:41:14.567 INFO:teuthology.orchestra.run.smithi169.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 332 kB/s | 41 kB 00:00 2024-05-30T16:41:14.701 INFO:teuthology.orchestra.run.smithi169.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 1.8 MB/s | 248 kB 00:00 2024-05-30T16:41:14.826 INFO:teuthology.orchestra.run.smithi169.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 295 kB/s | 37 kB 00:00 2024-05-30T16:41:14.951 INFO:teuthology.orchestra.run.smithi169.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.6 MB/s | 207 kB 00:00 2024-05-30T16:41:15.060 INFO:teuthology.orchestra.run.smithi169.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.4 MB/s | 260 kB 00:00 2024-05-30T16:41:15.227 INFO:teuthology.orchestra.run.smithi169.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-30T16:41:15.402 INFO:teuthology.orchestra.run.smithi169.stdout:(157/159): NetworkManager-libnm-1.47.91-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-05-30T16:41:16.186 INFO:teuthology.orchestra.run.smithi169.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 2.4 MB 00:00 2024-05-30T16:41:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-05-30T16:41:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-30T16:41:17.340 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-30T16:41:17.477 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-30T16:41:17.537 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-30T16:41:17.571 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-30T16:41:17.675 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-30T16:41:18.090 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-30T16:41:18.218 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:19.0.0-3154.g9ed025fb.el9.x86_64 8/170 2024-05-30T16:41:18.402 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:19.0.0-3154.g9ed025fb.el9.x86_64 8/170 2024-05-30T16:41:18.435 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:19.0.0-3154.g9ed025fb.el9.x86_64 9/170 2024-05-30T16:41:18.567 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.g9ed025fb.el9.x86_64 9/170 2024-05-30T16:41:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:19.0.0-3154.g9ed025fb.el9.x86_64 10/170 2024-05-30T16:41:18.684 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-30T16:41:18.998 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-30T16:41:19.060 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-30T16:41:19.091 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:19.0.0-3154.g9ed025fb.el9.x86_64 14/170 2024-05-30T16:41:19.244 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.g9ed025fb.el9.x86_64 14/170 2024-05-30T16:41:19.275 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:19.0.0-3154.g9ed025fb.el9.x86 15/170 2024-05-30T16:41:19.395 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.g9ed025fb.el9.x86 15/170 2024-05-30T16:41:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-30T16:41:19.494 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-30T16:41:19.773 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-30T16:41:20.025 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-30T16:41:20.183 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-05-30T16:41:20.271 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-05-30T16:41:20.464 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-05-30T16:41:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-05-30T16:41:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-05-30T16:41:21.102 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-30T16:41:21.229 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-30T16:41:21.269 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-30T16:41:21.317 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-30T16:41:21.432 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-30T16:41:21.483 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-30T16:41:21.573 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-30T16:41:21.918 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-30T16:41:21.974 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-30T16:41:22.120 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-30T16:41:22.199 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-30T16:41:22.239 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-30T16:41:22.301 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-30T16:41:22.302 INFO:teuthology.orchestra.run.smithi169.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 897 kB/s | 16 MB 00:18 2024-05-30T16:41:22.309 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-05-30T16:41:22.310 INFO:teuthology.orchestra.run.smithi169.stdout:Total 6.5 MB/s | 227 MB 00:34 2024-05-30T16:41:22.421 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-30T16:41:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:19.0.0-3154.g9ed025fb.el9. 34/170 2024-05-30T16:41:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.g9ed025fb.el 35/170 2024-05-30T16:41:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:19.0.0-3154.g9ed025fb.el9.x86_6 36/170 2024-05-30T16:41:22.664 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-30T16:41:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-30T16:41:22.851 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-30T16:41:23.037 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-30T16:41:23.105 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-30T16:41:23.839 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-30T16:41:23.947 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-30T16:41:23.987 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-30T16:41:24.052 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-30T16:41:24.114 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-30T16:41:24.177 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:19.0.0-3154.g9ed025fb.el9.x86_6 47/170 2024-05-30T16:41:24.260 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-30T16:41:24.500 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-30T16:41:24.574 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-30T16:41:24.657 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-30T16:41:24.759 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-30T16:41:24.807 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-30T16:41:24.846 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-30T16:41:24.918 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-30T16:41:24.963 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-30T16:41:25.028 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-30T16:41:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-30T16:41:25.305 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-30T16:41:25.319 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-05-30T16:41:25.355 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-30T16:41:25.409 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-30T16:41:25.544 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-30T16:41:25.572 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-05-30T16:41:25.572 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-05-30T16:41:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-30T16:41:25.636 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-30T16:41:25.678 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-30T16:41:25.712 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-30T16:41:25.895 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-30T16:41:26.075 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-30T16:41:26.118 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-30T16:41:26.158 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-30T16:41:26.216 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-30T16:41:26.290 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-30T16:41:26.368 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-30T16:41:26.417 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-30T16:41:26.471 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-30T16:41:26.522 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-30T16:41:26.599 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-30T16:41:26.643 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-30T16:41:26.789 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-30T16:41:26.848 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-30T16:41:26.899 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-30T16:41:26.972 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-30T16:41:27.135 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-30T16:41:27.205 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-30T16:41:28.118 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-05-30T16:41:28.118 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-05-30T16:41:28.204 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-30T16:41:28.263 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-30T16:41:28.296 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-30T16:41:28.423 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-30T16:41:28.488 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-30T16:41:28.543 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-30T16:41:28.658 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-30T16:41:28.698 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-30T16:41:28.945 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-30T16:41:28.969 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-30T16:41:29.095 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-30T16:41:30.023 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-30T16:41:30.164 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-30T16:41:31.905 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-30T16:41:32.001 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-30T16:41:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-30T16:41:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-30T16:41:32.489 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-30T16:41:32.538 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:19.0.0-3154.g9ed025fb.el9.x86_64 103/170 2024-05-30T16:41:32.662 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:19.0.0-3154.g9ed025fb.el9.x86_64 103/170 2024-05-30T16:41:32.695 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 104/170 2024-05-30T16:41:32.754 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-30T16:41:33.339 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-30T16:41:33.366 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:19.0.0-3154.g9ed025fb.el9.x86_64 107/170 2024-05-30T16:41:33.466 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:19.0.0-3154.g9ed025fb.el9.x86_64 107/170 2024-05-30T16:41:33.493 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 108/170 2024-05-30T16:41:34.639 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-05-30T16:41:34.742 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-30T16:41:34.834 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-30T16:41:34.952 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-30T16:41:35.013 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-30T16:41:35.044 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-30T16:41:35.147 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-30T16:41:35.587 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-30T16:41:35.692 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-3154.g9ed025fb.el9.x86_64 8/170 2024-05-30T16:41:35.887 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-3154.g9ed025fb.el9.x86_64 8/170 2024-05-30T16:41:35.918 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-3154.g9ed025fb.el9.x86_64 9/170 2024-05-30T16:41:36.047 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.g9ed025fb.el9.x86_64 9/170 2024-05-30T16:41:36.076 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-3154.g9ed025fb.el9.x86_64 10/170 2024-05-30T16:41:36.123 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-30T16:41:36.278 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 109/170 2024-05-30T16:41:36.322 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 109/170 2024-05-30T16:41:36.416 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-30T16:41:36.491 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-30T16:41:36.523 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-3154.g9ed025fb.el9.x86_64 14/170 2024-05-30T16:41:36.685 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.g9ed025fb.el9.x86_64 14/170 2024-05-30T16:41:36.716 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-3154.g9ed025fb.el9.x86 15/170 2024-05-30T16:41:36.802 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.g9ed025fb.el9.x86 15/170 2024-05-30T16:41:36.834 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-30T16:41:36.906 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-30T16:41:37.005 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 109/170 2024-05-30T16:41:37.051 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:19.0.0-3154.g9ed025fb.el9.x86_64 110/170 2024-05-30T16:41:37.213 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.g9ed025fb.el9.x86_64 110/170 2024-05-30T16:41:37.213 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-30T16:41:37.214 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-30T16:41:37.214 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:41:37.242 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-30T16:41:37.251 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:19.0.0-3154.g9ed025fb.el9.x86_64 111/170 2024-05-30T16:41:37.582 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-30T16:41:37.816 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-05-30T16:41:37.937 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-05-30T16:41:38.127 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-05-30T16:41:38.236 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-05-30T16:41:38.551 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-05-30T16:41:38.747 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-30T16:41:38.868 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-30T16:41:38.909 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-30T16:41:38.957 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-30T16:41:39.059 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-30T16:41:39.104 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-30T16:41:39.201 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-30T16:41:39.253 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-30T16:41:39.291 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-30T16:41:39.412 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-30T16:41:39.479 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-30T16:41:39.511 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-30T16:41:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-30T16:41:39.674 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-30T16:41:39.723 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-3154.g9ed025fb.el9. 34/170 2024-05-30T16:41:39.776 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.g9ed025fb.el 35/170 2024-05-30T16:41:39.814 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-3154.g9ed025fb.el9.x86_6 36/170 2024-05-30T16:41:39.932 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-30T16:41:40.000 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-30T16:41:40.098 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-30T16:41:40.268 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-30T16:41:40.336 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-30T16:41:41.016 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-30T16:41:41.117 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-30T16:41:41.153 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-30T16:41:41.202 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-30T16:41:41.266 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-30T16:41:41.353 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-3154.g9ed025fb.el9.x86_6 47/170 2024-05-30T16:41:41.430 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-30T16:41:41.621 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-30T16:41:41.706 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-30T16:41:41.775 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-30T16:41:41.869 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-30T16:41:41.915 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-30T16:41:41.956 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-30T16:41:42.023 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-30T16:41:42.062 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-30T16:41:42.124 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-30T16:41:42.181 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-30T16:41:42.368 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-30T16:41:42.431 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-30T16:41:42.483 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-30T16:41:42.610 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-30T16:41:42.656 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-30T16:41:42.703 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-30T16:41:42.746 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-30T16:41:42.779 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-30T16:41:42.984 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-30T16:41:43.180 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-30T16:41:43.219 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-30T16:41:43.258 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-30T16:41:43.303 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-30T16:41:43.361 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-30T16:41:43.414 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-30T16:41:43.456 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-30T16:41:43.535 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-30T16:41:43.585 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-30T16:41:43.663 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-30T16:41:43.710 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-30T16:41:43.853 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-30T16:41:43.921 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-30T16:41:43.964 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-30T16:41:44.024 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-30T16:41:44.171 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-30T16:41:44.246 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-30T16:41:45.192 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-30T16:41:45.246 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-30T16:41:45.279 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-30T16:41:45.399 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-30T16:41:45.462 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-30T16:41:45.515 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-30T16:41:45.612 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-30T16:41:45.700 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-30T16:41:45.950 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-30T16:41:45.987 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-30T16:41:46.104 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-30T16:41:46.940 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-30T16:41:47.102 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-30T16:41:48.715 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-30T16:41:48.811 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-30T16:41:48.847 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-30T16:41:48.882 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-30T16:41:49.283 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-30T16:41:49.324 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-3154.g9ed025fb.el9.x86_64 103/170 2024-05-30T16:41:49.444 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-3154.g9ed025fb.el9.x86_64 103/170 2024-05-30T16:41:49.481 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 104/170 2024-05-30T16:41:49.539 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-30T16:41:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-30T16:41:50.118 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-3154.g9ed025fb.el9.x86_64 107/170 2024-05-30T16:41:50.211 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-3154.g9ed025fb.el9.x86_64 107/170 2024-05-30T16:41:50.237 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 108/170 2024-05-30T16:41:51.162 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.g9ed025fb.el9.x86_64 111/170 2024-05-30T16:41:51.162 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2024-05-30T16:41:51.162 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2024-05-30T16:41:51.162 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2024-05-30T16:41:51.162 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2024-05-30T16:41:51.162 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2024-05-30T16:41:51.162 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /root 2024-05-30T16:41:51.162 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2024-05-30T16:41:51.162 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:41:51.403 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:19.0.0-3154.g9ed025fb.el9.noarch 112/170 2024-05-30T16:41:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.g9ed025fb.el9.noarch 112/170 2024-05-30T16:41:51.457 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:41:51.457 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-30T16:41:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:41:52.951 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 109/170 2024-05-30T16:41:53.001 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 109/170 2024-05-30T16:41:53.154 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:19.0.0-3154.g9ed025fb.el9.x86_64 113/170 2024-05-30T16:41:53.236 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.g9ed025fb.el9.x86_64 113/170 2024-05-30T16:41:53.236 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:41:53.237 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-30T16:41:53.237 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-30T16:41:53.237 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-30T16:41:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:41:53.504 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:19.0.0-3154.g9ed025fb.el9.x86_64 114/170 2024-05-30T16:41:53.578 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.g9ed025fb.el9.x86_64 114/170 2024-05-30T16:41:53.579 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:41:53.579 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-30T16:41:53.579 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-30T16:41:53.579 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-30T16:41:53.579 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:41:53.650 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 109/170 2024-05-30T16:41:53.697 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-3154.g9ed025fb.el9.x86_64 110/170 2024-05-30T16:41:53.877 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.g9ed025fb.el9.x86_64 110/170 2024-05-30T16:41:53.877 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-30T16:41:53.877 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-30T16:41:53.877 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:41:53.923 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-3154.g9ed025fb.el9.x86_64 111/170 2024-05-30T16:41:54.088 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:19.0.0-3154.g9ed025fb.el9.x86_64 115/170 2024-05-30T16:41:54.159 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.g9ed025fb.el9.x86_64 115/170 2024-05-30T16:41:54.159 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:41:54.159 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-30T16:41:54.160 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-30T16:41:54.160 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-30T16:41:54.160 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:41:54.299 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-30T16:41:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-30T16:41:54.437 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-30T16:41:54.693 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-30T16:41:54.753 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-30T16:41:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-30T16:41:55.123 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-30T16:41:55.312 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-30T16:41:55.400 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-30T16:41:55.463 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-30T16:41:55.500 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-30T16:41:56.164 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 127/170 2024-05-30T16:41:56.209 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 127/170 2024-05-30T16:41:56.265 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.g9ed025fb.e 128/170 2024-05-30T16:41:56.379 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.g9ed025fb. 129/170 2024-05-30T16:41:56.411 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.g9ed025fb.el9.noa 130/170 2024-05-30T16:41:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.g9ed025fb.el9.noa 130/170 2024-05-30T16:41:57.829 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.g9ed025fb.el9.n 131/170 2024-05-30T16:41:58.995 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.g9ed025fb.el9.n 131/170 2024-05-30T16:41:59.029 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.g9ed 132/170 2024-05-30T16:41:59.186 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.g9ed 132/170 2024-05-30T16:41:59.378 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.g9ed025fb.el 133/170 2024-05-30T16:41:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:19.0.0-3154.g9ed025fb.el9.x86_64 134/170 2024-05-30T16:41:59.642 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.g9ed025fb.el9.x86_64 134/170 2024-05-30T16:41:59.642 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:41:59.642 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-30T16:41:59.643 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-30T16:41:59.643 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-30T16:41:59.643 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:41:59.707 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.g9ed025fb.el9.noarch 135/170 2024-05-30T16:41:59.750 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.g9ed025fb.el9.noarch 135/170 2024-05-30T16:41:59.873 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:19.0.0-3154.g9ed025fb.el9.x86_64 136/170 2024-05-30T16:42:00.052 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 137/170 2024-05-30T16:42:01.164 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-30T16:42:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 139/170 2024-05-30T16:42:01.293 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 139/170 2024-05-30T16:42:01.294 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:01.294 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-30T16:42:01.294 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-30T16:42:01.294 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-30T16:42:01.294 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:01.343 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.g9ed02 140/170 2024-05-30T16:42:01.416 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.g9ed02 140/170 2024-05-30T16:42:01.416 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:01.416 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-30T16:42:01.416 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 141/170 2024-05-30T16:42:01.582 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 141/170 2024-05-30T16:42:01.582 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:01.582 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-30T16:42:01.582 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-30T16:42:01.582 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-30T16:42:01.582 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:01.944 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 142/170 2024-05-30T16:42:02.022 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 142/170 2024-05-30T16:42:02.022 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:02.023 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-30T16:42:02.023 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-30T16:42:02.023 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-30T16:42:02.023 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:06.438 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.g9ed025fb.el9.x86_64 111/170 2024-05-30T16:42:06.438 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-05-30T16:42:06.438 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-05-30T16:42:06.438 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-05-30T16:42:06.438 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-05-30T16:42:06.438 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-05-30T16:42:06.438 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-05-30T16:42:06.439 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-05-30T16:42:06.439 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:06.657 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-3154.g9ed025fb.el9.noarch 112/170 2024-05-30T16:42:06.705 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.g9ed025fb.el9.noarch 112/170 2024-05-30T16:42:06.705 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:06.705 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-30T16:42:06.705 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:07.327 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:19.0.0-3154.g9ed025fb.el9.x86_64 143/170 2024-05-30T16:42:07.383 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 144/170 2024-05-30T16:42:07.448 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 145/170 2024-05-30T16:42:07.485 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-30T16:42:07.523 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-30T16:42:07.837 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-shell-2:19.0.0-3154.g9ed025fb.el9.x86_64 148/170 2024-05-30T16:42:07.897 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-30T16:42:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:19.0.0-3154.g9ed025fb.el9.x86_ 150/170 2024-05-30T16:42:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-30T16:42:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-05-30T16:42:08.079 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-05-30T16:42:08.326 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-3154.g9ed025fb.el9.x86_64 113/170 2024-05-30T16:42:08.406 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.g9ed025fb.el9.x86_64 113/170 2024-05-30T16:42:08.406 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:08.406 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-30T16:42:08.406 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-30T16:42:08.406 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-30T16:42:08.406 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-3154.g9ed025fb.el9.x86_64 114/170 2024-05-30T16:42:08.745 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.g9ed025fb.el9.x86_64 114/170 2024-05-30T16:42:08.745 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:08.745 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-30T16:42:08.745 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-30T16:42:08.745 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-30T16:42:08.746 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-30T16:42:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-30T16:42:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-30T16:42:09.235 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-3154.g9ed025fb.el9.x86_64 115/170 2024-05-30T16:42:09.305 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.g9ed025fb.el9.x86_64 115/170 2024-05-30T16:42:09.305 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:09.305 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-30T16:42:09.305 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-30T16:42:09.305 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-30T16:42:09.305 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:09.428 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-30T16:42:09.488 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-30T16:42:09.569 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-30T16:42:09.806 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-30T16:42:09.873 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-30T16:42:09.961 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-30T16:42:10.067 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-30T16:42:10.232 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-30T16:42:10.290 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-30T16:42:10.336 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-30T16:42:10.397 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-30T16:42:10.583 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:19.0.0-3154.g9ed025fb.el9.noarch 155/170 2024-05-30T16:42:10.697 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-30T16:42:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-30T16:42:10.908 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-30T16:42:10.948 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-30T16:42:10.982 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-30T16:42:11.025 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-30T16:42:11.051 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 127/170 2024-05-30T16:42:11.066 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-30T16:42:11.067 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-30T16:42:11.083 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 127/170 2024-05-30T16:42:11.135 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.g9ed025fb.e 128/170 2024-05-30T16:42:11.199 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-30T16:42:11.236 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.g9ed025fb. 129/170 2024-05-30T16:42:11.236 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-30T16:42:11.237 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-30T16:42:11.268 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.g9ed025fb.el9.noa 130/170 2024-05-30T16:42:11.374 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-30T16:42:11.412 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-30T16:42:11.445 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-30T16:42:11.446 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-30T16:42:11.783 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-30T16:42:11.784 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-30T16:42:11.928 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-30T16:42:11.973 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-30T16:42:12.098 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-30T16:42:12.371 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-30T16:42:12.372 INFO:teuthology.orchestra.run.smithi067.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-30T16:42:12.372 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:12.601 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.g9ed025fb.el9.noa 130/170 2024-05-30T16:42:12.654 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.g9ed025fb.el9.n 131/170 2024-05-30T16:42:13.772 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.g9ed025fb.el9.n 131/170 2024-05-30T16:42:13.804 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.g9ed 132/170 2024-05-30T16:42:13.953 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.g9ed 132/170 2024-05-30T16:42:14.128 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.g9ed025fb.el 133/170 2024-05-30T16:42:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-3154.g9ed025fb.el9.x86_64 134/170 2024-05-30T16:42:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.g9ed025fb.el9.x86_64 134/170 2024-05-30T16:42:14.230 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:14.230 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-30T16:42:14.230 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-30T16:42:14.230 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-30T16:42:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.g9ed025fb.el9.noarch 135/170 2024-05-30T16:42:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.g9ed025fb.el9.noarch 135/170 2024-05-30T16:42:14.468 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-3154.g9ed025fb.el9.x86_64 136/170 2024-05-30T16:42:14.538 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 137/170 2024-05-30T16:42:15.554 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-30T16:42:15.593 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 139/170 2024-05-30T16:42:15.669 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 139/170 2024-05-30T16:42:15.669 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:15.669 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-30T16:42:15.669 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-30T16:42:15.669 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-30T16:42:15.670 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:15.718 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.g9ed02 140/170 2024-05-30T16:42:15.810 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.g9ed02 140/170 2024-05-30T16:42:15.810 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:15.810 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-30T16:42:15.810 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:15.886 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 141/170 2024-05-30T16:42:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 141/170 2024-05-30T16:42:15.983 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:15.983 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-30T16:42:15.983 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-30T16:42:15.983 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-30T16:42:15.983 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:16.361 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 142/170 2024-05-30T16:42:16.439 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 142/170 2024-05-30T16:42:16.439 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-30T16:42:16.439 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-30T16:42:16.439 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-30T16:42:16.439 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-30T16:42:16.439 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:19.0.0-3154.g9ed025fb.el9.x86_64 1/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:19.0.0-3154.g9ed025fb.el9.x86_64 2/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 3/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 4/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.g9ed02 5/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:19.0.0-3154.g9ed025fb.el9.x86_64 6/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:19.0.0-3154.g9ed025fb.el9.x86_64 7/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:19.0.0-3154.g9ed025fb.el9.x86_64 8/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:19.0.0-3154.g9ed025fb.el9.x86_64 9/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 10/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:19.0.0-3154.g9ed025fb.el9.x86_64 11/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:19.0.0-3154.g9ed025fb.el9.x86_64 12/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 13/170 2024-05-30T16:42:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-shell-2:19.0.0-3154.g9ed025fb.el9.x86_64 14/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.g9ed025fb.el9.x86_ 15/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:19.0.0-3154.g9ed025fb.el9.x86_64 16/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:19.0.0-3154.g9ed025fb.el9.x86_64 17/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:19.0.0-3154.g9ed025fb.el9.x86_6 18/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:19.0.0-3154.g9ed025fb.el9.x86 19/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:19.0.0-3154.g9ed025fb.el9.x86_64 20/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.g9ed025fb.el 21/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.g9ed025fb.el9. 22/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:19.0.0-3154.g9ed025fb.el9.x86_6 23/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:19.0.0-3154.g9ed025fb.el9.x86_64 24/170 2024-05-30T16:42:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 25/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 26/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 27/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 28/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 29/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.g9ed025fb. 30/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.g9ed025fb.el9.noa 31/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.g9ed025fb.el9.n 32/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.g9ed 33/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.g9ed025fb.el 34/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.g9ed025fb.el9.noarch 35/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.g9ed025fb.e 36/170 2024-05-30T16:42:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:19.0.0-3154.g9ed025fb.el9.noarch 37/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 38/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:19.0.0-3154.g9ed025fb.el9.noarch 39/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-30T16:42:17.804 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-30T16:42:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-30T16:42:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-30T16:42:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-30T16:42:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-30T16:42:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-30T16:42:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-30T16:42:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:19.0.0-3154.g9ed025fb.el9.x86_64 149/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:19.0.0-3154.g9ed025fb.el9.x86_64 151/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-30T16:42:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-30T16:42:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-30T16:42:20.036 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-30T16:42:20.036 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:20.036 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-05-30T16:42:20.036 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-05-30T16:42:20.036 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-05-30T16:42:20.036 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-05-30T16:42:20.036 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-05-30T16:42:20.036 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.2-8.el9.noarch 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.7.4-5.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.037 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.038 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.4-9.el9.x86_64 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-30T16:42:20.039 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-30T16:42:20.040 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: lua-5.4.4-4.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-30T16:42:20.041 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-13.14-1.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-30T16:42:20.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-30T16:42:20.043 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-30T16:42:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply-3.11-14.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-30T16:42:20.045 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-30T16:42:20.046 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:20.047 INFO:teuthology.orchestra.run.smithi067.stdout: re2-1:20211101-3.el9.x86_64 2024-05-30T16:42:20.048 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-30T16:42:20.048 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-30T16:42:20.048 INFO:teuthology.orchestra.run.smithi067.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-30T16:42:20.048 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-30T16:42:20.048 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-30T16:42:20.048 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-30T16:42:20.048 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:20.048 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-05-30T16:42:20.754 DEBUG:teuthology.parallel:result is None 2024-05-30T16:42:21.543 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-3154.g9ed025fb.el9.x86_64 143/170 2024-05-30T16:42:21.597 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 144/170 2024-05-30T16:42:21.661 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 145/170 2024-05-30T16:42:21.700 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-30T16:42:21.748 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-30T16:42:22.058 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-3154.g9ed025fb.el9.x86_64 148/170 2024-05-30T16:42:22.113 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-30T16:42:22.145 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-3154.g9ed025fb.el9.x86_ 150/170 2024-05-30T16:42:22.185 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-30T16:42:22.230 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-05-30T16:42:22.278 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-05-30T16:42:23.215 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-30T16:42:23.266 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-30T16:42:23.322 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-30T16:42:24.722 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-3154.g9ed025fb.el9.noarch 155/170 2024-05-30T16:42:24.823 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-30T16:42:24.878 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-30T16:42:25.013 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-30T16:42:25.050 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-30T16:42:25.084 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-30T16:42:25.118 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-30T16:42:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-30T16:42:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-30T16:42:25.292 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-30T16:42:25.355 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-30T16:42:25.355 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-30T16:42:25.492 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-30T16:42:25.528 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-30T16:42:25.553 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-30T16:42:25.553 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-30T16:42:25.894 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-30T16:42:25.894 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-30T16:42:26.035 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-30T16:42:26.067 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-30T16:42:26.184 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-30T16:42:26.441 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-30T16:42:26.441 INFO:teuthology.orchestra.run.smithi169.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-30T16:42:26.441 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:32.260 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-3154.g9ed025fb.el9.x86_64 1/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-3154.g9ed025fb.el9.x86_64 2/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 3/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 4/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.g9ed02 5/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-3154.g9ed025fb.el9.x86_64 6/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-3154.g9ed025fb.el9.x86_64 7/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-3154.g9ed025fb.el9.x86_64 8/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-3154.g9ed025fb.el9.x86_64 9/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 10/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-3154.g9ed025fb.el9.x86_64 11/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-3154.g9ed025fb.el9.x86_64 12/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 13/170 2024-05-30T16:42:32.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-3154.g9ed025fb.el9.x86_64 14/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.g9ed025fb.el9.x86_ 15/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-3154.g9ed025fb.el9.x86_64 16/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-3154.g9ed025fb.el9.x86_64 17/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-3154.g9ed025fb.el9.x86_6 18/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-3154.g9ed025fb.el9.x86 19/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-3154.g9ed025fb.el9.x86_64 20/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.g9ed025fb.el 21/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.g9ed025fb.el9. 22/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-3154.g9ed025fb.el9.x86_6 23/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-3154.g9ed025fb.el9.x86_64 24/170 2024-05-30T16:42:32.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 25/170 2024-05-30T16:42:32.274 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 26/170 2024-05-30T16:42:32.274 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 27/170 2024-05-30T16:42:32.274 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 28/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 29/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.g9ed025fb. 30/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.g9ed025fb.el9.noa 31/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.g9ed025fb.el9.n 32/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.g9ed 33/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.g9ed025fb.el 34/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.g9ed025fb.el9.noarch 35/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.g9ed025fb.e 36/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-3154.g9ed025fb.el9.noarch 37/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 38/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-3154.g9ed025fb.el9.noarch 39/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-30T16:42:32.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-30T16:42:32.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-30T16:42:32.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-30T16:42:32.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-30T16:42:32.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-30T16:42:32.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-30T16:42:32.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-30T16:42:32.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-30T16:42:32.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-3154.g9ed025fb.el9.x86_64 149/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-3154.g9ed025fb.el9.x86_64 151/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-05-30T16:42:32.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-30T16:42:32.285 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-30T16:42:32.286 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-30T16:42:32.286 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-30T16:42:34.531 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-30T16:42:34.531 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:34.531 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-05-30T16:42:34.531 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-05-30T16:42:34.531 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-05-30T16:42:34.531 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-05-30T16:42:34.531 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-05-30T16:42:34.531 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-30T16:42:34.531 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.532 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.533 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-3154.g9ed025fb.el9.noarch 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-30T16:42:34.534 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-30T16:42:34.535 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-05-30T16:42:34.536 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-30T16:42:34.537 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-30T16:42:34.538 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-30T16:42:34.539 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-30T16:42:34.540 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-30T16:42:34.541 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.542 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-30T16:42:34.542 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-30T16:42:34.542 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-30T16:42:34.542 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-30T16:42:34.542 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-30T16:42:34.542 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-30T16:42:34.542 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-30T16:42:34.542 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-30T16:42:34.542 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-3154.g9ed025fb.el9.x86_64 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-30T16:42:34.543 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-30T16:42:34.544 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-30T16:42:34.544 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-30T16:42:34.544 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-30T16:42:34.544 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:34.544 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-05-30T16:42:35.177 DEBUG:teuthology.parallel:result is None 2024-05-30T16:42:35.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-30T16:42:35.404 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-30T16:42:35.484 INFO:teuthology.orchestra.run.smithi067.stdout:19.0.0-3154.g9ed025fb.el9 2024-05-30T16:42:35.485 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.g9ed025fb.el9 2024-05-30T16:42:35.485 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.g9ed025fb is installed. 2024-05-30T16:42:35.487 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ed025fbf4406fb54980a0647506b15895b6d863 2024-05-30T16:42:35.626 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-30T16:42:35.685 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-3154.g9ed025fb.el9 2024-05-30T16:42:35.685 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.g9ed025fb.el9 2024-05-30T16:42:35.685 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.g9ed025fb is installed. 2024-05-30T16:42:35.687 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-30T16:42:35.687 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:35.687 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-30T16:42:35.784 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:42:35.784 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-30T16:42:35.862 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-30T16:42:35.862 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:35.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-05-30T16:42:35.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-30T16:42:36.045 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:42:36.046 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-05-30T16:42:36.114 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-30T16:42:36.217 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-30T16:42:36.218 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:36.218 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-30T16:42:36.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-30T16:42:36.445 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:42:36.445 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-30T16:42:36.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-30T16:42:36.622 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-30T16:42:36.622 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:36.623 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-05-30T16:42:36.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-30T16:42:36.806 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:42:36.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-05-30T16:42:36.875 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-30T16:42:36.980 INFO:teuthology.run_tasks:Running task ceph... 2024-05-30T16:42:37.057 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-30T16:42:37.057 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /var/log/ceph 2024-05-30T16:42:37.065 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2024-05-30T16:42:37.152 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-30T16:42:37.152 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-30T16:42:37.159 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-30T16:42:37.242 INFO:tasks.ceph:Creating extra log directories... 2024-05-30T16:42:37.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-30T16:42:37.247 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-30T16:42:37.334 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-30T16:42:37.334 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-30T16:42:37.334 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-30T16:42:37.334 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:37.334 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-30T16:42:37.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-30T16:42:37.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-30T16:42:37.696 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:42:37.696 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-30T16:42:37.739 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-30T16:42:37.922 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-30T16:42:37.987 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-30T16:42:37.987 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-30T16:42:37.987 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140', 'branch': 'rishabh-clone-stats', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7734140', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-clone-stats'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-vol-clone-stats', 'sha1': '019a0d1fdedb1237f85e8a3711761fae4e094286'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 964, 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'mgr-vol-clone-stats', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '019a0d1fdedb1237f85e8a3711761fae4e094286', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi067.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLntZL+JZlpcR/rjkSN8yfgu96A86IvYBCyRGKW1pfr5viYvxvv5BdKfZqCNZ8uazKCxM4uHQ8z/R8kHETOtRfo=', 'smithi169.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEV3AP8Hs3d69mJfERJGWAj5x9zCw30kGPI2vgWrwz2MfnGSvpnuc0UZM12VEWjiNE+h22c0rIUEOfTMKYf1YLk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '9ed025fbf4406fb54980a0647506b15895b6d863', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e1979f064b0becde2020282c7e87cf3c5e98026c', 'timestamp': '2024-05-30_16:10:36', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-30T16:42:37.988 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-30T16:42:37.996 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-30T16:42:38.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/run/ceph 2024-05-30T16:42:38.056 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2024-05-30T16:42:38.140 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:38.140 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-05-30T16:42:38.198 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-30T16:42:38.198 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-05-30T16:42:38.251 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-30T16:42:38.252 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T16:42:38.252 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-30T16:42:38.252 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T16:42:38.252 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-05-30T16:42:38.252 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-05-30 16:42:12.821646212 +0000 2024-05-30T16:42:38.252 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-05-30 16:39:12.447548984 +0000 2024-05-30T16:42:38.252 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-05-30 16:39:12.447548984 +0000 2024-05-30T16:42:38.252 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-05-30 16:39:12.447548984 +0000 2024-05-30T16:42:38.252 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-30T16:42:38.336 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-05-30T16:42:38.336 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-05-30T16:42:38.336 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000381552 s, 1.3 MB/s 2024-05-30T16:42:38.340 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-30T16:42:38.397 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-05-30T16:42:38.453 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-30T16:42:38.453 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T16:42:38.453 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-30T16:42:38.453 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T16:42:38.453 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-05-30T16:42:38.453 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-05-30 16:42:12.823646165 +0000 2024-05-30T16:42:38.453 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-05-30 16:39:13.067534555 +0000 2024-05-30T16:42:38.453 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-05-30 16:39:13.067534555 +0000 2024-05-30T16:42:38.453 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-05-30 16:39:13.067534555 +0000 2024-05-30T16:42:38.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-30T16:42:38.531 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-05-30T16:42:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-05-30T16:42:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000338306 s, 1.5 MB/s 2024-05-30T16:42:38.535 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-30T16:42:38.592 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-05-30T16:42:38.644 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-30T16:42:38.644 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T16:42:38.644 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-05-30T16:42:38.644 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T16:42:38.644 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-05-30T16:42:38.644 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-05-30 16:42:12.823646165 +0000 2024-05-30T16:42:38.644 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-05-30 16:39:13.708519637 +0000 2024-05-30T16:42:38.644 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-05-30 16:39:13.708519637 +0000 2024-05-30T16:42:38.644 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-05-30 16:39:13.708519637 +0000 2024-05-30T16:42:38.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-30T16:42:38.726 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-05-30T16:42:38.726 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-05-30T16:42:38.726 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000370211 s, 1.4 MB/s 2024-05-30T16:42:38.730 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-30T16:42:38.790 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-05-30T16:42:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-30T16:42:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T16:42:38.845 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-05-30T16:42:38.845 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T16:42:38.845 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-05-30T16:42:38.845 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-05-30 16:42:12.824646142 +0000 2024-05-30T16:42:38.845 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-05-30 16:39:14.324505301 +0000 2024-05-30T16:42:38.845 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-05-30 16:39:14.325505277 +0000 2024-05-30T16:42:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-05-30 16:39:14.324505301 +0000 2024-05-30T16:42:38.846 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-30T16:42:38.929 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-05-30T16:42:38.930 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-05-30T16:42:38.930 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000368172 s, 1.4 MB/s 2024-05-30T16:42:38.933 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-30T16:42:38.994 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T16:42:38.995 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:42:38.995 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-05-30T16:42:39.039 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-30T16:42:39.039 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-05-30T16:42:39.119 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-30T16:42:39.120 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T16:42:39.120 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-05-30T16:42:39.120 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T16:42:39.120 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-30T16:42:39.120 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-30 16:42:26.904953811 +0000 2024-05-30T16:42:39.120 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-30 16:39:09.920930225 +0000 2024-05-30T16:42:39.120 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-30 16:39:09.920930225 +0000 2024-05-30T16:42:39.121 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-30 16:39:09.920930225 +0000 2024-05-30T16:42:39.121 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-30T16:42:39.218 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-30T16:42:39.219 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-30T16:42:39.219 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000401829 s, 1.3 MB/s 2024-05-30T16:42:39.221 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-30T16:42:39.264 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-05-30T16:42:39.345 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-30T16:42:39.345 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T16:42:39.346 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 659 Links: 1 2024-05-30T16:42:39.346 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T16:42:39.346 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-30T16:42:39.346 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-30 16:42:26.905953789 +0000 2024-05-30T16:42:39.346 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-30 16:39:10.531916911 +0000 2024-05-30T16:42:39.346 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-30 16:39:10.531916911 +0000 2024-05-30T16:42:39.346 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-30 16:39:10.531916911 +0000 2024-05-30T16:42:39.346 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-30T16:42:39.444 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-30T16:42:39.444 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-30T16:42:39.444 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000326985 s, 1.6 MB/s 2024-05-30T16:42:39.447 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-30T16:42:39.487 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-05-30T16:42:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-30T16:42:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T16:42:39.568 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 678 Links: 1 2024-05-30T16:42:39.568 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T16:42:39.568 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-30T16:42:39.568 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-30 16:42:26.905953789 +0000 2024-05-30T16:42:39.568 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-30 16:39:11.132903814 +0000 2024-05-30T16:42:39.568 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-30 16:39:11.132903814 +0000 2024-05-30T16:42:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-30 16:39:11.132903814 +0000 2024-05-30T16:42:39.569 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-30T16:42:39.678 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-30T16:42:39.678 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-30T16:42:39.678 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000346792 s, 1.5 MB/s 2024-05-30T16:42:39.683 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-30T16:42:39.726 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-05-30T16:42:39.805 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-30T16:42:39.805 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-30T16:42:39.805 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 694 Links: 1 2024-05-30T16:42:39.805 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-30T16:42:39.805 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-05-30T16:42:39.805 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-05-30 16:42:26.906953767 +0000 2024-05-30T16:42:39.805 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-05-30 16:39:11.725890891 +0000 2024-05-30T16:42:39.806 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-05-30 16:39:11.725890891 +0000 2024-05-30T16:42:39.806 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-05-30 16:39:11.725890891 +0000 2024-05-30T16:42:39.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-30T16:42:39.906 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-05-30T16:42:39.906 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-05-30T16:42:39.906 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000285985 s, 1.8 MB/s 2024-05-30T16:42:39.909 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-30T16:42:39.951 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T16:42:39.951 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-30T16:42:39.952 INFO:tasks.ceph:Generating config... 2024-05-30T16:42:39.953 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-30T16:42:39.953 INFO:tasks.ceph:[client] debug client = 20 2024-05-30T16:42:39.953 INFO:tasks.ceph:[client] debug ms = 1 2024-05-30T16:42:39.953 INFO:tasks.ceph:[client] pid file = 2024-05-30T16:42:39.953 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-30T16:42:39.953 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-30T16:42:39.953 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-30T16:42:39.953 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-30T16:42:39.953 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-30T16:42:39.954 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-30T16:42:39.955 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-30T16:42:39.955 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-30T16:42:39.955 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-30T16:42:39.956 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-30T16:42:39.956 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-30T16:42:39.956 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-30T16:42:39.956 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-30T16:42:39.956 INFO:tasks.ceph:Setting up mon.a... 2024-05-30T16:42:39.956 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-30T16:42:40.098 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.keyring 2024-05-30T16:42:40.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-30T16:42:40.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-30T16:42:40.325 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.67'), ('mon.b', '172.21.15.169'), ('mon.c', '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]')] 2024-05-30T16:42:40.325 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.67,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-30T16:42:40.326 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:40.326 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-30T16:42:40.382 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.67 --add b 172.21.15.169 --addv c '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:42:40.542 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: generated fsid faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = quincy 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 0 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:fsid faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:last_changed 2024-05-30T16:42:40.542805+0000 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-05-30T16:42:40.542805+0000 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:min_mon_release 17 (quincy) 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:election_strategy: 1 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:2: [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] mon.c 2024-05-30T16:42:40.543 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-30T16:42:40.546 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-30T16:42:40.600 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d... 2024-05-30T16:42:40.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-30T16:42:40.606 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.746 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.747 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2024-05-30T16:42:40.748 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.67,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2024-05-30T16:42:40.749 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug trim objects = true 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.750 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: bdev async discard = True 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: bdev enable discard = True 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore allocator = bitmap 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2024-05-30T16:42:40.751 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 1/20 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 1/20 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 4/10 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-05-30T16:42:40.752 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: debug finisher = 20 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-05-30T16:42:40.753 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-30T16:42:40.754 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: mon down mkfs grace = 300 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2024-05-30T16:42:40.755 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2024-05-30T16:42:40.756 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-05-30T16:42:40.757 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2024-05-30T16:42:40.757 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = True 2024-05-30T16:42:40.757 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2024-05-30T16:42:40.757 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = True 2024-05-30T16:42:40.757 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-05-30T16:42:40.757 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-05-30T16:42:40.757 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-05-30T16:42:40.757 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-05-30T16:42:40.757 INFO:teuthology.orchestra.run.smithi067.stdout: chdir = "" 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: auth supported = cephx 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: filestore xattr use omap = true 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: mon clock drift allowed = 1.000 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: osd crush chooseleaf type = 0 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: auth debug = true 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on old message = true 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on bug = true 2024-05-30T16:42:40.758 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: mon pg warn max object skew = 0 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: # disable pg_autoscaler by default for new pools 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default size = 2 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow primary affinity = true 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow pg remap = true 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on legacy crush tunables = false 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on crush straw calc version zero = false 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on no sortbitwise = false 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on osd down out interval zero = false 2024-05-30T16:42:40.759 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on too few osds = false 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: mon_allow_pool_size_one = true 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: osd default data pool replay window = 5 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: mon allow pool delete = true 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: mon cluster log file level = debug 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: debug asserts on shutdown = true 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: mon health detail to clog = false 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: mon host = "172.21.15.67,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-05-30T16:42:40.760 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout:[osd] 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: osd journal size = 100 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub load threshold = 5.0 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub max interval = 600 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: osd mclock profile = high_recovery_ops 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: osd recover clone overlap = true 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: osd recovery max chunk = 1048576 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug shutdown = true 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug op order = true 2024-05-30T16:42:40.761 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug verify stray on activate = true 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug trim objects = true 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: osd open classes on start = true 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug pg log writeout = true 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: osd deep scrub update digest min age = 30 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: osd map max advance = 10 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: journal zero on create = true 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: filestore ondisk finisher threads = 3 2024-05-30T16:42:40.762 INFO:teuthology.orchestra.run.smithi067.stdout: filestore apply finisher threads = 3 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: bdev debug aio = true 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug misdirected ops = true 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: bdev async discard = True 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: bdev enable discard = True 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore allocator = bitmap 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore block size = 96636764160 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore fsck on mount = True 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluefs = 1/20 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluestore = 1/20 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: debug osd = 20 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: debug rocksdb = 4/10 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd backfillfull_ratio = 0.85 2024-05-30T16:42:40.763 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd full ratio = 0.9 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd nearfull ratio = 0.8 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: osd failsafe full ratio = 0.95 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: osd objectstore = bluestore 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: osd op complaint time = 180 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr] 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: debug mgr = 20 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2024-05-30T16:42:40.764 INFO:teuthology.orchestra.run.smithi067.stdout: mgr/telemetry/nag = false 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: debug client = 20 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: debug finisher = 20 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout:[mon] 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: debug paxos = 20 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: mon data avail warn = 5 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: mon mgr mkfs grace = 240 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd reporter subtree level = osd 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd prime pg temp = true 2024-05-30T16:42:40.765 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: auth mon ticket ttl = 660 # 11m 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: auth service ticket ttl = 240 # 4m 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: # don't complain about insecure global_id in the test suite 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: # 1m isn't quite enough 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: mon_down_mkfs_grace = 2m 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_filestore_osds = false 2024-05-30T16:42:40.766 INFO:teuthology.orchestra.run.smithi067.stdout: mon down mkfs grace = 300 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: mon op complaint time = 120 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout:[client] 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: rgw cache enabled = true 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable ops log = true 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable usage log = true 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: client mount timeout = 600 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: debug client = 20 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: pid file = "" 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: rados mon op timeout = 900 2024-05-30T16:42:40.767 INFO:teuthology.orchestra.run.smithi067.stdout: rados osd op timeout = 900 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.a] 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.b] 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.c] 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c] 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b] 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d] 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout:[mds] 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout: debug mds = 20 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout: debug mds balancer = 20 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-05-30T16:42:40.768 INFO:teuthology.orchestra.run.smithi067.stdout: mds debug frag = True 2024-05-30T16:42:40.769 INFO:teuthology.orchestra.run.smithi067.stdout: mds debug scatterstat = True 2024-05-30T16:42:40.769 INFO:teuthology.orchestra.run.smithi067.stdout: mds op complaint time = 180 2024-05-30T16:42:40.769 INFO:teuthology.orchestra.run.smithi067.stdout: mds verify scatter = True 2024-05-30T16:42:40.769 INFO:teuthology.orchestra.run.smithi067.stdout: osd op complaint time = 180 2024-05-30T16:42:40.769 INFO:teuthology.orchestra.run.smithi067.stdout: rados mon op timeout = 900 2024-05-30T16:42:40.769 INFO:teuthology.orchestra.run.smithi067.stdout: rados osd op timeout = 900 2024-05-30T16:42:40.783 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-30T16:42:40.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-30T16:42:40.924 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-30T16:42:40.924 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:40.924 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-30T16:42:40.977 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:40.977 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-30T16:42:41.028 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:42:41.028 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:41.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-30T16:42:41.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-30T16:42:41.144 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:42:41.144 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:42:41.246 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T16:42:41.246 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:42:41.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-30T16:42:41.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-30T16:42:41.382 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:42:41.383 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:42:41.477 INFO:tasks.ceph:Setting up mon nodes... 2024-05-30T16:42:41.478 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-30T16:42:41.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-30T16:42:41.658 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-30T16:42:41.670 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-30T16:42:41.828 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-30T16:42:41.838 INFO:tasks.ceph:Setting up mds nodes... 2024-05-30T16:42:41.838 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-30T16:42:42.017 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-30T16:42:42.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-30T16:42:42.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-30T16:42:42.259 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-30T16:42:42.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-30T16:42:42.356 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-30T16:42:42.523 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-30T16:42:42.533 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-30T16:42:42.616 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-30T16:42:42.776 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-30T16:42:42.785 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-30T16:42:42.867 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-30T16:42:42.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-30T16:42:42.994 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-30T16:42:43.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-30T16:42:43.170 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-30T16:42:43.213 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-30T16:42:43.334 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-30T16:42:43.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-30T16:42:43.494 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-30T16:42:43.534 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-30T16:42:43.534 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-30T16:42:43.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-30T16:42:43.628 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T16:42:43.628 INFO:tasks.ceph:role: osd.0 2024-05-30T16:42:43.628 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:42:43.628 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-30T16:42:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T16:42:43.722 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T16:42:43.722 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T16:42:43.723 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-30T16:42:43.723 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T16:42:43.723 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-05-30T16:42:43.723 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T16:42:43.723 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-30T16:42:43.723 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T16:42:43.723 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T16:42:44.127 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-05-30T16:42:44.134 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-05-30T16:42:44.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-30T16:42:44.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-30T16:42:44.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-30T16:42:44.490 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T16:42:44.490 INFO:tasks.ceph:role: osd.1 2024-05-30T16:42:44.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:42:44.490 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-30T16:42:44.586 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T16:42:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T16:42:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T16:42:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-30T16:42:44.587 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T16:42:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-05-30T16:42:44.587 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T16:42:44.587 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-30T16:42:44.587 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T16:42:44.587 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T16:42:44.940 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-05-30T16:42:44.947 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-05-30T16:42:44.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-30T16:42:45.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-30T16:42:45.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-30T16:42:45.255 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T16:42:45.256 INFO:tasks.ceph:role: osd.2 2024-05-30T16:42:45.256 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:42:45.256 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-30T16:42:45.344 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T16:42:45.344 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T16:42:45.344 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T16:42:45.344 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-30T16:42:45.344 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T16:42:45.344 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-05-30T16:42:45.345 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T16:42:45.345 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-30T16:42:45.345 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T16:42:45.345 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T16:42:45.735 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-05-30T16:42:45.742 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-05-30T16:42:45.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-30T16:42:45.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-30T16:42:45.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-30T16:42:46.086 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-30T16:42:46.086 INFO:tasks.ceph:role: osd.3 2024-05-30T16:42:46.086 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:42:46.086 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-30T16:42:46.180 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T16:42:46.180 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T16:42:46.180 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T16:42:46.180 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-30T16:42:46.180 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T16:42:46.180 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-05-30T16:42:46.181 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T16:42:46.181 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-30T16:42:46.181 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T16:42:46.181 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T16:42:46.597 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-05-30T16:42:46.604 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-05-30T16:42:46.604 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-30T16:42:46.727 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-30T16:42:46.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:42:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:46.973+0000 7f7586ae1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-30T16:42:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:46.974+0000 7f7586ae1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-30T16:42:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:46.974+0000 7f7586ae1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-30T16:42:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:46.974+0000 7f7586ae1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-30T16:42:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:46.974+0000 7f7586ae1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-30T16:42:46.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:46.978+0000 7f7586ae1640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-30T16:42:49.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-30T16:42:50.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:42:50.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:50.172+0000 7f446360d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-30T16:42:50.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:50.173+0000 7f446360d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-30T16:42:50.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:50.173+0000 7f446360d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-30T16:42:50.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:50.173+0000 7f446360d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-30T16:42:50.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:50.173+0000 7f446360d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-30T16:42:50.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:50.176+0000 7f446360d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-30T16:42:53.146 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-30T16:42:53.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:42:53.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:53.364+0000 7f5a0c69d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-30T16:42:53.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:53.364+0000 7f5a0c69d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-30T16:42:53.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:53.364+0000 7f5a0c69d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-30T16:42:53.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:53.365+0000 7f5a0c69d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-30T16:42:53.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:53.365+0000 7f5a0c69d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-30T16:42:53.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:53.367+0000 7f5a0c69d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-30T16:42:56.609 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-30T16:42:56.698 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:42:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:56.836+0000 7f37e4d2c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-30T16:42:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:56.837+0000 7f37e4d2c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-30T16:42:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:56.837+0000 7f37e4d2c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-30T16:42:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:56.837+0000 7f37e4d2c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-30T16:42:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:56.837+0000 7f37e4d2c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-30T16:42:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:42:56.839+0000 7f37e4d2c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-30T16:42:59.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-30T16:42:59.902 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-30T16:42:59.993 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T16:42:59.993 INFO:tasks.ceph:role: osd.4 2024-05-30T16:42:59.993 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T16:42:59.994 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T16:43:00.082 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T16:43:00.501 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-05-30T16:43:00.508 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-05-30T16:43:00.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-30T16:43:00.664 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-30T16:43:00.745 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-30T16:43:00.817 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T16:43:00.817 INFO:tasks.ceph:role: osd.5 2024-05-30T16:43:00.817 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T16:43:00.817 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-30T16:43:00.901 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T16:43:00.901 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T16:43:00.901 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T16:43:00.901 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-30T16:43:00.901 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T16:43:00.902 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-05-30T16:43:00.902 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T16:43:00.902 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-30T16:43:00.902 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T16:43:00.902 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T16:43:01.255 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-05-30T16:43:01.261 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-05-30T16:43:01.262 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-30T16:43:01.369 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-30T16:43:01.441 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-30T16:43:01.509 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T16:43:01.509 INFO:tasks.ceph:role: osd.6 2024-05-30T16:43:01.509 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T16:43:01.509 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-30T16:43:01.584 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T16:43:01.584 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T16:43:01.584 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T16:43:01.584 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-30T16:43:01.584 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T16:43:01.584 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-05-30T16:43:01.584 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T16:43:01.584 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-30T16:43:01.584 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T16:43:01.585 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T16:43:01.973 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-05-30T16:43:01.979 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-05-30T16:43:01.980 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-30T16:43:02.092 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-30T16:43:02.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-30T16:43:02.255 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-30T16:43:02.255 INFO:tasks.ceph:role: osd.7 2024-05-30T16:43:02.255 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T16:43:02.255 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-30T16:43:02.342 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-30T16:43:02.342 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-30T16:43:02.342 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-30T16:43:02.342 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-30T16:43:02.345 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-30T16:43:02.345 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-05-30T16:43:02.345 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-30T16:43:02.345 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-30T16:43:02.346 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-30T16:43:02.346 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-30T16:43:02.747 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-05-30T16:43:02.753 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-05-30T16:43:02.753 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-30T16:43:02.922 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-30T16:43:03.010 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:43:03.151 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:03.151+0000 7efdd6032640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-30T16:43:03.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:03.151+0000 7efdd6032640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-30T16:43:03.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:03.152+0000 7efdd6032640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-30T16:43:03.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:03.152+0000 7efdd6032640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-30T16:43:03.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:03.152+0000 7efdd6032640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-30T16:43:03.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:03.155+0000 7efdd6032640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-30T16:43:06.124 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-30T16:43:06.210 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:43:06.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:06.332+0000 7f40fbaf0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-30T16:43:06.334 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:06.333+0000 7f40fbaf0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-30T16:43:06.334 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:06.333+0000 7f40fbaf0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-30T16:43:06.334 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:06.334+0000 7f40fbaf0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-30T16:43:06.334 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:06.334+0000 7f40fbaf0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-30T16:43:06.336 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:06.336+0000 7f40fbaf0640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-30T16:43:07.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:43:07.998 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:43:09.319 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-30T16:43:09.402 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:43:09.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:09.529+0000 7fce05819640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-30T16:43:09.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:09.530+0000 7fce05819640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-30T16:43:09.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:09.530+0000 7fce05819640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-30T16:43:09.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:09.530+0000 7fce05819640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-30T16:43:09.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:09.530+0000 7fce05819640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-30T16:43:09.532 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:09.532+0000 7fce05819640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-30T16:43:12.753 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-30T16:43:12.833 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:43:12.954 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:12.952+0000 7f9314620640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-30T16:43:12.954 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:12.953+0000 7f9314620640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-30T16:43:12.954 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:12.953+0000 7f9314620640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-30T16:43:12.955 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:12.953+0000 7f9314620640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-30T16:43:12.955 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:12.953+0000 7f9314620640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-30T16:43:12.956 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:12.955+0000 7f9314620640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-30T16:43:15.924 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-30T16:43:16.013 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-30T16:43:16.013 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:16.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-30T16:43:16.106 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:16.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-30T16:43:16.194 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:16.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-30T16:43:16.276 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:16.276 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-30T16:43:16.360 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:16.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-30T16:43:16.443 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:16.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-30T16:43:16.532 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:16.532 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-30T16:43:16.621 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:16.621 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-30T16:43:16.710 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:16.710 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-30T16:43:16.791 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:16.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-30T16:43:16.877 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:16.877 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-30T16:43:16.961 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:16.961 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-30T16:43:17.045 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:17.045 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-30T16:43:17.129 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:17.129 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-30T16:43:17.210 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:17.210 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-30T16:43:17.271 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:17.271 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-30T16:43:17.330 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:17.330 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-30T16:43:17.388 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:17.388 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-30T16:43:17.441 INFO:tasks.ceph:Adding keys to all mons... 2024-05-30T16:43:17.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-30T16:43:17.449 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-30T16:43:17.530 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2024-05-30T16:43:17.530 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw== 2024-05-30T16:43:17.530 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-05-30T16:43:17.530 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA== 2024-05-30T16:43:17.530 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-05-30T16:43:17.530 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw== 2024-05-30T16:43:17.530 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2024-05-30T16:43:17.530 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng== 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg== 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w== 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ== 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ== 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw== 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg== 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2024-05-30T16:43:17.531 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A== 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ== 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w== 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A== 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout:[client.2] 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ== 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ== 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout:[client.3] 2024-05-30T16:43:17.532 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A== 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.y] 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw== 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA== 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c] 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw== 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.0] 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng== 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.1] 2024-05-30T16:43:17.533 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg== 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.2] 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w== 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.3] 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ== 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.x] 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ== 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b] 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw== 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d] 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg== 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.4] 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A== 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.5] 2024-05-30T16:43:17.534 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ== 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.6] 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w== 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.7] 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A== 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout:[client.2] 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ== 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout:[client.1] 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ== 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout:[client.3] 2024-05-30T16:43:17.535 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A== 2024-05-30T16:43:17.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-30T16:43:17.585 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-30T16:43:17.725 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-30T16:43:17.730 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-30T16:43:17.869 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-30T16:43:17.875 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-30T16:43:18.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:18.016 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:18.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:18.152 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:18.287 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:18.293 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:18.424 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:18.431 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:18.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-30T16:43:18.573 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-30T16:43:18.709 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-30T16:43:18.715 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-30T16:43:18.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-30T16:43:18.859 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-30T16:43:18.995 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:18.999 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:19.135 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:19.140 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:19.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:19.279 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:19.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:19.421 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-30T16:43:19.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T16:43:19.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T16:43:19.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T16:43:19.709 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T16:43:19.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T16:43:19.856 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T16:43:19.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T16:43:20.003 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-30T16:43:20.144 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-30T16:43:20.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-30T16:43:20.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-30T16:43:20.396 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-30T16:43:20.477 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-30T16:43:20.567 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-30T16:43:20.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-30T16:43:20.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-30T16:43:20.879 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-30T16:43:21.017 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-30T16:43:21.079 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:43:21.087 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-30T16:43:21.139 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-30T16:43:21.139 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-30T16:43:21.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-30T16:43:21.146 INFO:tasks.ceph.mon.a:Started 2024-05-30T16:43:21.147 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-30T16:43:21.147 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-30T16:43:21.176 INFO:tasks.ceph.mon.b:Started 2024-05-30T16:43:21.176 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-30T16:43:21.176 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-30T16:43:21.181 INFO:tasks.ceph.mon.c:Started 2024-05-30T16:43:21.181 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-30T16:43:21.181 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-30T16:43:21.182 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-30T16:43:21.190 INFO:tasks.ceph.mgr.y:Started 2024-05-30T16:43:21.190 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-30T16:43:21.191 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-30T16:43:21.195 INFO:tasks.ceph.mgr.x:Started 2024-05-30T16:43:21.195 INFO:tasks.ceph:Setting crush tunables to default 2024-05-30T16:43:21.195 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd crush tunables default 2024-05-30T16:43:21.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.372+0000 7fc023200640 1 Processor -- start 2024-05-30T16:43:21.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.372+0000 7fc023200640 1 -- start start 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc023200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc023200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc02415ec50 con 0x7fc02415e870 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc023200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0240cff80 con 0x7fc0240d16a0 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc023200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc0240d2150 con 0x7fc0240cfa40 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc022800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc019400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc0240d16a0 0x7fc0240cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:51110/0 (socket says 172.21.15.67:51110) 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 learned_addr learned my addr 172.21.15.67:0/1424803622 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc022800640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 msgr2=0x7fc024169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc022800640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 msgr2=0x7fc024169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:21.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.374+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-30T16:43:21.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.576+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:21.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.576+0000 7fc022800640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 msgr2=0x7fc024169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-30T16:43:21.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.576+0000 7fc022800640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 msgr2=0x7fc024169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:21.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.576+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:21.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.576+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-30T16:43:21.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.977+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:21.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.978+0000 7fc022800640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 msgr2=0x7fc024169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-30T16:43:21.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.978+0000 7fc022800640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 msgr2=0x7fc024169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:21.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.978+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:21.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:21.978+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-30T16:43:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:22.780+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:22.780+0000 7fc022800640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 msgr2=0x7fc024169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-30T16:43:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:22.780+0000 7fc022800640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 msgr2=0x7fc024169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:22.780+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:22.780+0000 7fc022800640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-30T16:43:24.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.374+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 msgr2=0x7fc024169690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:43:24.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.374+0000 7fc01be00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc0240cfa40 0x7fc024169690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:24.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.374+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 >> v1:172.21.15.169:6789/0 conn(0x7fc0240d16a0 legacy=0x7fc0240cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:24.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.374+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 >> v1:172.21.15.67:6789/0 conn(0x7fc02415e870 legacy=0x7fc0240d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:24.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.374+0000 7fc01be00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:24.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.374+0000 7fc01be00640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:24.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.374+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc0240d2150 con 0x7fc0000036e0 2024-05-30T16:43:24.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.374+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc000008d60 con 0x7fc000006180 2024-05-30T16:43:24.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.375+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:24.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.375+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:24.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.375+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 msgr2=0x7fc000008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-30T16:43:24.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.375+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 msgr2=0x7fc000008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:24.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.375+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:24.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.375+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-30T16:43:24.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.376+0000 7fc021e00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 msgr2=0x7fc000005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-30T16:43:24.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.376+0000 7fc021e00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 msgr2=0x7fc000005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:24.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.376+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:24.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.376+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-30T16:43:24.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.575+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:24.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.576+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 msgr2=0x7fc000008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-30T16:43:24.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.576+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 msgr2=0x7fc000008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:24.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.576+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:24.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.576+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-30T16:43:24.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.576+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:24.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.577+0000 7fc021e00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 msgr2=0x7fc000005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-30T16:43:24.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.577+0000 7fc021e00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 msgr2=0x7fc000005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:24.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.577+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:24.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.577+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-30T16:43:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.977+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.978+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 msgr2=0x7fc000008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-30T16:43:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.978+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 msgr2=0x7fc000008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.978+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.978+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-30T16:43:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.978+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.979+0000 7fc021e00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 msgr2=0x7fc000005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-30T16:43:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.979+0000 7fc021e00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 msgr2=0x7fc000005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.979+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:24.979+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-30T16:43:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.780+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.780+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 msgr2=0x7fc000008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-30T16:43:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.780+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 msgr2=0x7fc000008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.780+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:25.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.780+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-30T16:43:25.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.780+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:25.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.781+0000 7fc021e00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 msgr2=0x7fc000005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-30T16:43:25.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.781+0000 7fc021e00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 msgr2=0x7fc000005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-30T16:43:25.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.781+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-30T16:43:25.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:25.781+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-30T16:43:26.792 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:26.790+0000 7f0d1412e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-30T16:43:26.986 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:26.984+0000 7f0d1412e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-30T16:43:27.055 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:27.054+0000 7f0d1412e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-30T16:43:27.366 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:27.364+0000 7f0d1412e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-30T16:43:27.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.373+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 msgr2=0x7fc000008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:43:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.373+0000 7fc01be00640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000006180 0x7fc000008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.373+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 msgr2=0x7fc000005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:43:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.373+0000 7fc01be00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.67:3300/0 conn(0x7fc0000036e0 0x7fc000005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.373+0000 7fc022800640 1 -- 172.21.15.67:0/1424803622 reap_dead start 2024-05-30T16:43:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.373+0000 7fc01be00640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc000006630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.374+0000 7fc01be00640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc000006b70 0x7fc000006f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.374+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0240d0260 con 0x7fc000007560 2024-05-30T16:43:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.374+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc0240d2150 con 0x7fc000006b70 2024-05-30T16:43:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.374+0000 7fc01be00640 1 -- 172.21.15.67:0/1424803622 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc000008d60 con 0x7fc000003b30 2024-05-30T16:43:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.374+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc000006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.374+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> v2:172.21.15.169:3300/0 conn(0x7fc000006b70 0x7fc000006f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.375+0000 7fc021400640 1 -- 172.21.15.67:0/1424803622 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3450777666 0 0) 0x7fc0240d0260 con 0x7fc000007560 2024-05-30T16:43:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.375+0000 7fc021400640 1 -- 172.21.15.67:0/1424803622 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc008003520 con 0x7fc000007560 2024-05-30T16:43:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.375+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 msgr2=0x7fc000006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.376+0000 7fc021400640 1 -- 172.21.15.67:0/1424803622 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3458303510 0 0) 0x7fc008003520 con 0x7fc000007560 2024-05-30T16:43:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.376+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc000006630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.376+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 >> v1:172.21.15.67:6789/0 conn(0x7fc000007560 legacy=0x7fc000007a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.376+0000 7fc019400640 1 -- 172.21.15.67:0/1424803622 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0000083a0 con 0x7fc000006b70 2024-05-30T16:43:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.376+0000 7fc021e00640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc000006630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.379+0000 7fc019400640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 0x7fc000006f50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc00c007d70 tx=0x7fc00c02ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ebbb761d11b7fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.379+0000 7fc021400640 1 -- 172.21.15.67:0/1424803622 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c009950 con 0x7fc000006b70 2024-05-30T16:43:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.379+0000 7fc021400640 1 -- 172.21.15.67:0/1424803622 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c02c500 con 0x7fc000006b70 2024-05-30T16:43:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.379+0000 7fc021400640 1 -- 172.21.15.67:0/1424803622 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c02c8f0 con 0x7fc000006b70 2024-05-30T16:43:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.380+0000 7fc023200640 1 -- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 msgr2=0x7fc000006f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.380+0000 7fc023200640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 0x7fc000006f50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc00c007d70 tx=0x7fc00c02ac70 comp rx=0 tx=0).stop 2024-05-30T16:43:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.381+0000 7fc023200640 1 -- 172.21.15.67:0/1424803622 shutdown_connections 2024-05-30T16:43:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.381+0000 7fc023200640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 0x7fc000006f50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.381+0000 7fc023200640 1 --2- 172.21.15.67:0/1424803622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc000006630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.381+0000 7fc023200640 1 -- 172.21.15.67:0/1424803622 >> 172.21.15.67:0/1424803622 conn(0x7fc02408bfc0 msgr2=0x7fc0240d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.381+0000 7fc023200640 1 -- 172.21.15.67:0/1424803622 shutdown_connections 2024-05-30T16:43:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.381+0000 7fc023200640 1 -- 172.21.15.67:0/1424803622 wait complete. 2024-05-30T16:43:27.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.383+0000 7fc023200640 1 Processor -- start 2024-05-30T16:43:27.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.384+0000 7fc023200640 1 -- start start 2024-05-30T16:43:27.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.385+0000 7fc023200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc024159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.385+0000 7fc023200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 0x7fc02415b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.385+0000 7fc023200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000007560 0x7fc0240b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.385+0000 7fc023200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc0240d4fb0 con 0x7fc000007560 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.385+0000 7fc023200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0240d0820 con 0x7fc000006b70 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.385+0000 7fc023200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc0240cf330 con 0x7fc000003b30 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.385+0000 7fc022800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000007560 0x7fc0240b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.385+0000 7fc022800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000007560 0x7fc0240b9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57382/0 (socket says 172.21.15.67:57382) 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.385+0000 7fc022800640 1 -- 172.21.15.67:0/2909302805 learned_addr learned my addr 172.21.15.67:0/2909302805 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc019400640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 0x7fc02415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc021e00640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc024159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc022800640 1 -- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 msgr2=0x7fc024159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc022800640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc024159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc022800640 1 -- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 msgr2=0x7fc02415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc022800640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 0x7fc02415b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc022800640 1 -- 172.21.15.67:0/2909302805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0240cb7d0 con 0x7fc000007560 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc019400640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 0x7fc02415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc021e00640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc024159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc022800640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000007560 0x7fc0240b9270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc000006100 tx=0x7fc01c0047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc01aa00640 1 -- 172.21.15.67:0/2909302805 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c012a00 con 0x7fc000007560 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc01aa00640 1 -- 172.21.15.67:0/2909302805 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc01c01d070 con 0x7fc000007560 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc01aa00640 1 -- 172.21.15.67:0/2909302805 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c018470 con 0x7fc000007560 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.386+0000 7fc023200640 1 -- 172.21.15.67:0/2909302805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0240cbab0 con 0x7fc000007560 2024-05-30T16:43:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.387+0000 7fc023200640 1 -- 172.21.15.67:0/2909302805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0240cc070 con 0x7fc000007560 2024-05-30T16:43:27.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.387+0000 7fc01aa00640 1 -- 172.21.15.67:0/2909302805 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc01c018610 con 0x7fc000007560 2024-05-30T16:43:27.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.387+0000 7fc01aa00640 1 -- 172.21.15.67:0/2909302805 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc01c022ad0 con 0x7fc000007560 2024-05-30T16:43:27.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.388+0000 7fc023200640 1 -- 172.21.15.67:0/2909302805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0240689e0 con 0x7fc000007560 2024-05-30T16:43:27.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.391+0000 7fc01aa00640 1 -- 172.21.15.67:0/2909302805 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc01c022cd0 con 0x7fc000007560 2024-05-30T16:43:27.448 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:27.447+0000 7f0d1412e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-30T16:43:27.497 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:27.496+0000 7fc0fe093040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-30T16:43:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.510+0000 7fc023200640 1 -- 172.21.15.67:0/2909302805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc024073400 con 0x7fc000007560 2024-05-30T16:43:27.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.514+0000 7fc01aa00640 1 -- 172.21.15.67:0/2909302805 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fc01c022460 con 0x7fc000007560 2024-05-30T16:43:27.516 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-05-30T16:43:27.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.515+0000 7fc023200640 1 -- 172.21.15.67:0/2909302805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000007560 msgr2=0x7fc0240b9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.515+0000 7fc023200640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000007560 0x7fc0240b9270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc000006100 tx=0x7fc01c0047f0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.516+0000 7fc023200640 1 -- 172.21.15.67:0/2909302805 shutdown_connections 2024-05-30T16:43:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.516+0000 7fc023200640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000007560 0x7fc0240b9270 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.516+0000 7fc023200640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc000006b70 0x7fc02415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.516+0000 7fc023200640 1 --2- 172.21.15.67:0/2909302805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc000003b30 0x7fc024159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.516+0000 7fc023200640 1 -- 172.21.15.67:0/2909302805 >> 172.21.15.67:0/2909302805 conn(0x7fc02408bfc0 msgr2=0x7fc0240b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.516+0000 7fc023200640 1 -- 172.21.15.67:0/2909302805 shutdown_connections 2024-05-30T16:43:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:27.516+0000 7fc023200640 1 -- 172.21.15.67:0/2909302805 wait complete. 2024-05-30T16:43:27.535 INFO:tasks.ceph:check_enable_crimson: False 2024-05-30T16:43:27.535 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-30T16:43:27.536 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:27.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-30T16:43:27.603 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:27.604 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-30T16:43:27.681 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:27.680+0000 7fc0fe093040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-30T16:43:27.726 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:27.726 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-30T16:43:27.741 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:27.740+0000 7fc0fe093040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-30T16:43:27.807 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:43:27.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-30T16:43:27.924 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:27.925 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-30T16:43:28.000 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:28.000 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-30T16:43:28.039 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:28.039+0000 7fc0fe093040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-30T16:43:28.069 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:28.069 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-30T16:43:28.096 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:28.096+0000 7fc0fe093040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-30T16:43:28.168 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:43:28.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-30T16:43:28.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 841d017d-87f0-434e-bc72-699abeabcfd2 0 2024-05-30T16:43:28.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.500+0000 7fb0c9400640 1 Processor -- start 2024-05-30T16:43:28.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.500+0000 7fb0c9400640 1 -- start start 2024-05-30T16:43:28.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.502+0000 7fb0c9400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0c40cef30 0x7fb0c40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.502+0000 7fb0c9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c40b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.502+0000 7fb0c9400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0c40d1f90 con 0x7fb0c415e850 2024-05-30T16:43:28.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.502+0000 7fb0c9400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c415ec30 con 0x7fb0c40cef30 2024-05-30T16:43:28.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.502+0000 7fb0c9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c40b8000 con 0x7fb0c40d1b80 2024-05-30T16:43:28.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.502+0000 7fb0c3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c40b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.502+0000 7fb0c3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c40b90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:43280/0 (socket says 172.21.15.169:43280) 2024-05-30T16:43:28.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.502+0000 7fb0c3200640 1 -- 172.21.15.169:0/696744990 learned_addr learned my addr 172.21.15.169:0/696744990 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:28.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.502+0000 7fb0bbe00640 1 --2- 172.21.15.169:0/696744990 >> v2:172.21.15.67:3300/0 conn(0x7fb0c40cef30 0x7fb0c40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.504 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.503+0000 7fb0c3200640 1 -- 172.21.15.169:0/696744990 >> v2:172.21.15.67:3300/0 conn(0x7fb0c40cef30 msgr2=0x7fb0c40d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:28.504 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.503+0000 7fb0c3200640 1 --2- 172.21.15.169:0/696744990 >> v2:172.21.15.67:3300/0 conn(0x7fb0c40cef30 0x7fb0c40d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.504 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.503+0000 7fb0c3200640 1 -- 172.21.15.169:0/696744990 >> v1:172.21.15.67:6789/0 conn(0x7fb0c415e850 legacy=0x7fb0c40b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:28.504 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.503+0000 7fb0c3200640 1 -- 172.21.15.169:0/696744990 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c40b82e0 con 0x7fb0c40d1b80 2024-05-30T16:43:28.504 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.504+0000 7fb0c3200640 1 --2- 172.21.15.169:0/696744990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c40b90a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40cb730 tx=0x7fb0bc02a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d46d232541d0ad server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:28.505 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.504+0000 7fb0c1e00640 1 -- 172.21.15.169:0/696744990 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc036070 con 0x7fb0c40d1b80 2024-05-30T16:43:28.505 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.504+0000 7fb0c1e00640 1 -- 172.21.15.169:0/696744990 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0bc031400 con 0x7fb0c40d1b80 2024-05-30T16:43:28.505 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.504+0000 7fb0c1e00640 1 -- 172.21.15.169:0/696744990 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0317d0 con 0x7fb0c40d1b80 2024-05-30T16:43:28.505 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.505+0000 7fb0c9400640 1 -- 172.21.15.169:0/696744990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 msgr2=0x7fb0c40b90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:28.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.505+0000 7fb0c9400640 1 --2- 172.21.15.169:0/696744990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c40b90a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40cb730 tx=0x7fb0bc02a610 comp rx=0 tx=0).stop 2024-05-30T16:43:28.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.505+0000 7fb0c9400640 1 -- 172.21.15.169:0/696744990 shutdown_connections 2024-05-30T16:43:28.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.505+0000 7fb0c9400640 1 --2- 172.21.15.169:0/696744990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c40b90a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.505+0000 7fb0c9400640 1 --2- 172.21.15.169:0/696744990 >> v2:172.21.15.67:3300/0 conn(0x7fb0c40cef30 0x7fb0c40d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.505+0000 7fb0c9400640 1 -- 172.21.15.169:0/696744990 >> 172.21.15.169:0/696744990 conn(0x7fb0c4067aa0 msgr2=0x7fb0c4067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:28.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.505+0000 7fb0c9400640 1 -- 172.21.15.169:0/696744990 shutdown_connections 2024-05-30T16:43:28.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.505+0000 7fb0c9400640 1 -- 172.21.15.169:0/696744990 wait complete. 2024-05-30T16:43:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.506+0000 7fb0c9400640 1 Processor -- start 2024-05-30T16:43:28.507 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.507+0000 7fb0c9400640 1 -- start start 2024-05-30T16:43:28.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.507+0000 7fb0c9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c40cef30 0x7fb0c41616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0c9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c4160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0c9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0c415e850 0x7fb0c40ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0c9400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c40b95e0 con 0x7fb0c40cef30 2024-05-30T16:43:28.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0c9400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c40b88a0 con 0x7fb0c415e850 2024-05-30T16:43:28.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0c9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c40b9760 con 0x7fb0c40d1b80 2024-05-30T16:43:28.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0bbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c4160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0c3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0c415e850 0x7fb0c40ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0bbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c4160360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:43294/0 (socket says 172.21.15.169:43294) 2024-05-30T16:43:28.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0bbe00640 1 -- 172.21.15.169:0/312490139 learned_addr learned my addr 172.21.15.169:0/312490139 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:28.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0c2800640 1 --2- 172.21.15.169:0/312490139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c40cef30 0x7fb0c41616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0bbe00640 1 -- 172.21.15.169:0/312490139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0c415e850 msgr2=0x7fb0c40ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:28.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0bbe00640 1 --2- 172.21.15.169:0/312490139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0c415e850 0x7fb0c40ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0bbe00640 1 -- 172.21.15.169:0/312490139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c40cef30 msgr2=0x7fb0c41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0bbe00640 1 --2- 172.21.15.169:0/312490139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c40cef30 0x7fb0c41616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.508+0000 7fb0bbe00640 1 -- 172.21.15.169:0/312490139 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c4161c20 con 0x7fb0c40d1b80 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.509+0000 7fb0bbe00640 1 --2- 172.21.15.169:0/312490139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c4160360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00d800 tx=0x7fb0ac00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.509+0000 7fb0baa00640 1 -- 172.21.15.169:0/312490139 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00b6c0 con 0x7fb0c40d1b80 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.509+0000 7fb0baa00640 1 -- 172.21.15.169:0/312490139 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac019070 con 0x7fb0c40d1b80 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.509+0000 7fb0baa00640 1 -- 172.21.15.169:0/312490139 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00ba80 con 0x7fb0c40d1b80 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.509+0000 7fb0c9400640 1 -- 172.21.15.169:0/312490139 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c415f2c0 con 0x7fb0c40d1b80 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.509+0000 7fb0c9400640 1 -- 172.21.15.169:0/312490139 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c415f880 con 0x7fb0c40d1b80 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.509+0000 7fb0baa00640 1 -- 172.21.15.169:0/312490139 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb0ac00bc20 con 0x7fb0c40d1b80 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.509+0000 7fb0baa00640 1 -- 172.21.15.169:0/312490139 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fb0ac0145a0 con 0x7fb0c40d1b80 2024-05-30T16:43:28.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.510+0000 7fb0c9400640 1 -- 172.21.15.169:0/312490139 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c40689e0 con 0x7fb0c40d1b80 2024-05-30T16:43:28.513 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.513+0000 7fb0baa00640 1 -- 172.21.15.169:0/312490139 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb0ac0147a0 con 0x7fb0c40d1b80 2024-05-30T16:43:28.533 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:28.531+0000 7f0d1412e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-30T16:43:28.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.595+0000 7fb0c9400640 1 -- 172.21.15.169:0/312490139 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "841d017d-87f0-434e-bc72-699abeabcfd2", "id": 0} v 0) -- 0x7fb0c41608a0 con 0x7fb0c40d1b80 2024-05-30T16:43:28.601 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.601+0000 7fb0baa00640 1 -- 172.21.15.169:0/312490139 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "841d017d-87f0-434e-bc72-699abeabcfd2", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fb0ac014a50 con 0x7fb0c40d1b80 2024-05-30T16:43:28.602 INFO:teuthology.orchestra.run.smithi169.stdout:0 2024-05-30T16:43:28.603 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.602+0000 7fb0c9400640 1 -- 172.21.15.169:0/312490139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 msgr2=0x7fb0c4160360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:28.603 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.602+0000 7fb0c9400640 1 --2- 172.21.15.169:0/312490139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c4160360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00d800 tx=0x7fb0ac00dcc0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.603+0000 7fb0c9400640 1 -- 172.21.15.169:0/312490139 shutdown_connections 2024-05-30T16:43:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.603+0000 7fb0c9400640 1 --2- 172.21.15.169:0/312490139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0c415e850 0x7fb0c40ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.603+0000 7fb0c9400640 1 --2- 172.21.15.169:0/312490139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0c40d1b80 0x7fb0c4160360 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.603+0000 7fb0c9400640 1 --2- 172.21.15.169:0/312490139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0c40cef30 0x7fb0c41616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.603+0000 7fb0c9400640 1 -- 172.21.15.169:0/312490139 >> 172.21.15.169:0/312490139 conn(0x7fb0c4067aa0 msgr2=0x7fb0c4074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.603+0000 7fb0c9400640 1 -- 172.21.15.169:0/312490139 shutdown_connections 2024-05-30T16:43:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.603+0000 7fb0c9400640 1 -- 172.21.15.169:0/312490139 wait complete. 2024-05-30T16:43:28.619 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new af68158f-3f57-4352-b975-983e7d597f5e 1 2024-05-30T16:43:28.645 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:28.643+0000 7f0d1412e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-30T16:43:28.699 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:28.697+0000 7f0d1412e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-30T16:43:28.786 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.786+0000 7f488a400640 1 Processor -- start 2024-05-30T16:43:28.786 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.786+0000 7f488a400640 1 -- start start 2024-05-30T16:43:28.787 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.787+0000 7f488a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ea40 0x7f48840cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.787+0000 7f488a400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f488415ee20 0x7f488415fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.787+0000 7f488a400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48840d37b0 con 0x7f48840d5190 2024-05-30T16:43:28.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.787+0000 7f488a400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f48840cecf0 con 0x7f488415ee20 2024-05-30T16:43:28.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.787+0000 7f488a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f48840cefd0 con 0x7f488415ea40 2024-05-30T16:43:28.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.787+0000 7f4889000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ea40 0x7f48840cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.787+0000 7f4883e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f48840d5190 0x7f48840d30a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:56812/0 (socket says 172.21.15.169:56812) 2024-05-30T16:43:28.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.787+0000 7f4883e00640 1 -- 172.21.15.169:0/2809424054 learned_addr learned my addr 172.21.15.169:0/2809424054 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-05-30T16:43:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.788+0000 7f4889a00640 1 --2- 172.21.15.169:0/2809424054 >> v2:172.21.15.67:3300/0 conn(0x7f488415ee20 0x7f488415fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.788+0000 7f4889000640 1 -- 172.21.15.169:0/2809424054 >> v2:172.21.15.67:3300/0 conn(0x7f488415ee20 msgr2=0x7f488415fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.788+0000 7f4883400640 1 -- 172.21.15.169:0/2809424054 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3989490565 0 0) 0x7f48840d37b0 con 0x7f48840d5190 2024-05-30T16:43:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.788+0000 7f4889000640 1 --2- 172.21.15.169:0/2809424054 >> v2:172.21.15.67:3300/0 conn(0x7f488415ee20 0x7f488415fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.788+0000 7f4889000640 1 -- 172.21.15.169:0/2809424054 >> v1:172.21.15.169:6789/0 conn(0x7f48840d5190 legacy=0x7f48840d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.788+0000 7f4889000640 1 -- 172.21.15.169:0/2809424054 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48840cf2b0 con 0x7f488415ea40 2024-05-30T16:43:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.788+0000 7f4889a00640 1 --2- 172.21.15.169:0/2809424054 >> v2:172.21.15.67:3300/0 conn(0x7f488415ee20 0x7f488415fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.789+0000 7f4889000640 1 --2- 172.21.15.169:0/2809424054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ea40 0x7f48840cfab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f48780097a0 tx=0x7f487802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8366fce1af68c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.789+0000 7f4883400640 1 -- 172.21.15.169:0/2809424054 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878037070 con 0x7f488415ea40 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.789+0000 7f4883400640 1 -- 172.21.15.169:0/2809424054 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4878032440 con 0x7f488415ea40 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.789+0000 7f4883400640 1 -- 172.21.15.169:0/2809424054 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878032810 con 0x7f488415ea40 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.790+0000 7f488a400640 1 -- 172.21.15.169:0/2809424054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ea40 msgr2=0x7f48840cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.790+0000 7f488a400640 1 --2- 172.21.15.169:0/2809424054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ea40 0x7f48840cfab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f48780097a0 tx=0x7f487802a180 comp rx=0 tx=0).stop 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.790+0000 7f488a400640 1 -- 172.21.15.169:0/2809424054 shutdown_connections 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.790+0000 7f488a400640 1 --2- 172.21.15.169:0/2809424054 >> v2:172.21.15.67:3300/0 conn(0x7f488415ee20 0x7f488415fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.790+0000 7f488a400640 1 --2- 172.21.15.169:0/2809424054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ea40 0x7f48840cfab0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.790+0000 7f488a400640 1 -- 172.21.15.169:0/2809424054 >> 172.21.15.169:0/2809424054 conn(0x7f488408bfc0 msgr2=0x7f48840d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.790+0000 7f488a400640 1 -- 172.21.15.169:0/2809424054 shutdown_connections 2024-05-30T16:43:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.790+0000 7f488a400640 1 -- 172.21.15.169:0/2809424054 wait complete. 2024-05-30T16:43:28.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.792+0000 7f488a400640 1 Processor -- start 2024-05-30T16:43:28.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.792+0000 7f488a400640 1 -- start start 2024-05-30T16:43:28.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f488a400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48840d5190 0x7f48840c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f488a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f488415ea40 0x7f48840cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f488a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ee20 0x7f48840cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:28.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f488a400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f488415f2d0 con 0x7f488415ea40 2024-05-30T16:43:28.794 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f488a400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f48840cfff0 con 0x7f48840d5190 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f488a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f488415f450 con 0x7f488415ee20 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48840d5190 0x7f48840c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48840d5190 0x7f48840c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54254/0 (socket says 172.21.15.169:54254) 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889000640 1 -- 172.21.15.169:0/2038162150 learned_addr learned my addr 172.21.15.169:0/2038162150 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ee20 0x7f48840cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4883e00640 1 --2- 172.21.15.169:0/2038162150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f488415ea40 0x7f48840cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889000640 1 -- 172.21.15.169:0/2038162150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ee20 msgr2=0x7f48840cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889000640 1 --2- 172.21.15.169:0/2038162150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ee20 0x7f48840cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889000640 1 -- 172.21.15.169:0/2038162150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f488415ea40 msgr2=0x7f48840cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889000640 1 --2- 172.21.15.169:0/2038162150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f488415ea40 0x7f48840cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889000640 1 -- 172.21.15.169:0/2038162150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48840c9090 con 0x7f48840d5190 2024-05-30T16:43:28.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.793+0000 7f4889a00640 1 --2- 172.21.15.169:0/2038162150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ee20 0x7f48840cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:28.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.794+0000 7f4889000640 1 --2- 172.21.15.169:0/2038162150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48840d5190 0x7f48840c8b50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48780097a0 tx=0x7f48780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.794+0000 7f4880c00640 1 -- 172.21.15.169:0/2038162150 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878041070 con 0x7f48840d5190 2024-05-30T16:43:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.794+0000 7f4880c00640 1 -- 172.21.15.169:0/2038162150 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487803c870 con 0x7f48840d5190 2024-05-30T16:43:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.794+0000 7f488a400640 1 -- 172.21.15.169:0/2038162150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48841f5370 con 0x7f48840d5190 2024-05-30T16:43:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.794+0000 7f4880c00640 1 -- 172.21.15.169:0/2038162150 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878037040 con 0x7f48840d5190 2024-05-30T16:43:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.794+0000 7f488a400640 1 -- 172.21.15.169:0/2038162150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48841f5930 con 0x7f48840d5190 2024-05-30T16:43:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.795+0000 7f4880c00640 1 -- 172.21.15.169:0/2038162150 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4878044070 con 0x7f48840d5190 2024-05-30T16:43:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.795+0000 7f4880c00640 1 -- 172.21.15.169:0/2038162150 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f487803d3f0 con 0x7f48840d5190 2024-05-30T16:43:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.795+0000 7f488a400640 1 -- 172.21.15.169:0/2038162150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48840690f0 con 0x7f48840d5190 2024-05-30T16:43:28.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.798+0000 7f4880c00640 1 -- 172.21.15.169:0/2038162150 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f48840c9090 con 0x7f48840d5190 2024-05-30T16:43:28.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.878+0000 7f488a400640 1 -- 172.21.15.169:0/2038162150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "af68158f-3f57-4352-b975-983e7d597f5e", "id": 1} v 0) -- 0x7f48841f8570 con 0x7f48840d5190 2024-05-30T16:43:28.885 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.885+0000 7f4880c00640 1 -- 172.21.15.169:0/2038162150 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af68158f-3f57-4352-b975-983e7d597f5e", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f487803cc60 con 0x7f48840d5190 2024-05-30T16:43:28.886 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-05-30T16:43:28.887 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.886+0000 7f488a400640 1 -- 172.21.15.169:0/2038162150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48840d5190 msgr2=0x7f48840c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:28.887 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.886+0000 7f488a400640 1 --2- 172.21.15.169:0/2038162150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48840d5190 0x7f48840c8b50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48780097a0 tx=0x7f48780046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.887 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.887+0000 7f488a400640 1 -- 172.21.15.169:0/2038162150 shutdown_connections 2024-05-30T16:43:28.888 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.887+0000 7f488a400640 1 --2- 172.21.15.169:0/2038162150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f488415ee20 0x7f48840cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.888 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.887+0000 7f488a400640 1 --2- 172.21.15.169:0/2038162150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f488415ea40 0x7f48840cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.888 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.887+0000 7f488a400640 1 --2- 172.21.15.169:0/2038162150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48840d5190 0x7f48840c8b50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:28.888 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.887+0000 7f488a400640 1 -- 172.21.15.169:0/2038162150 >> 172.21.15.169:0/2038162150 conn(0x7f488408bfc0 msgr2=0x7f4884160210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:28.888 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.887+0000 7f488a400640 1 -- 172.21.15.169:0/2038162150 shutdown_connections 2024-05-30T16:43:28.888 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:28.887+0000 7f488a400640 1 -- 172.21.15.169:0/2038162150 wait complete. 2024-05-30T16:43:28.902 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:28.900+0000 7f0d1412e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-30T16:43:28.905 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new d2137b46-ff54-4ae4-a961-cb73f43ba933 2 2024-05-30T16:43:28.974 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:28.972+0000 7f0d1412e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-30T16:43:29.040 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:29.038+0000 7f0d1412e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-30T16:43:29.091 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.090+0000 7f93d8a00640 1 Processor -- start 2024-05-30T16:43:29.091 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.090+0000 7f93d8a00640 1 -- start start 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.091+0000 7f93d8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f93d415e870 0x7f93d40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.091+0000 7f93d8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f93d40d14d0 0x7f93d40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d8a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d40cfce0 con 0x7f93d40cf870 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d8a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f93d40d2150 con 0x7f93d40d14d0 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f93d415ec50 con 0x7f93d415e870 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d3400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f93d415e870 0x7f93d40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d3400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f93d415e870 0x7f93d40d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54264/0 (socket says 172.21.15.169:54264) 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d3400640 1 -- 172.21.15.169:0/3840116210 learned_addr learned my addr 172.21.15.169:0/3840116210 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d3e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f93d40cf870 0x7f93d4162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:56818/0 (socket says 172.21.15.169:56818) 2024-05-30T16:43:29.093 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d2a00640 1 --2- 172.21.15.169:0/3840116210 >> v2:172.21.15.67:3300/0 conn(0x7f93d40d14d0 0x7f93d40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d3400640 1 -- 172.21.15.169:0/3840116210 >> v2:172.21.15.67:3300/0 conn(0x7f93d40d14d0 msgr2=0x7f93d40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:29.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d3400640 1 --2- 172.21.15.169:0/3840116210 >> v2:172.21.15.67:3300/0 conn(0x7f93d40d14d0 0x7f93d40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d3400640 1 -- 172.21.15.169:0/3840116210 >> v1:172.21.15.169:6789/0 conn(0x7f93d40cf870 legacy=0x7f93d4162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.092+0000 7f93d3400640 1 -- 172.21.15.169:0/3840116210 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d40cff80 con 0x7f93d415e870 2024-05-30T16:43:29.094 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.093+0000 7f93d3400640 1 --2- 172.21.15.169:0/3840116210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e870 0x7f93d40d0f90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93d408e720 tx=0x7f93c402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26fd103d447693ec server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:29.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.094+0000 7f93d2000640 1 -- 172.21.15.169:0/3840116210 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4037070 con 0x7f93d415e870 2024-05-30T16:43:29.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.094+0000 7f93d2000640 1 -- 172.21.15.169:0/3840116210 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c4032440 con 0x7f93d415e870 2024-05-30T16:43:29.095 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.094+0000 7f93d2000640 1 -- 172.21.15.169:0/3840116210 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4032810 con 0x7f93d415e870 2024-05-30T16:43:29.095 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:29.094+0000 7fc0fe093040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-30T16:43:29.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.094+0000 7f93d8a00640 1 -- 172.21.15.169:0/3840116210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e870 msgr2=0x7f93d40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.094+0000 7f93d8a00640 1 --2- 172.21.15.169:0/3840116210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e870 0x7f93d40d0f90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93d408e720 tx=0x7f93c402a270 comp rx=0 tx=0).stop 2024-05-30T16:43:29.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.094+0000 7f93d8a00640 1 -- 172.21.15.169:0/3840116210 shutdown_connections 2024-05-30T16:43:29.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.094+0000 7f93d8a00640 1 --2- 172.21.15.169:0/3840116210 >> v2:172.21.15.67:3300/0 conn(0x7f93d40d14d0 0x7f93d40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.094+0000 7f93d8a00640 1 --2- 172.21.15.169:0/3840116210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e870 0x7f93d40d0f90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.094+0000 7f93d8a00640 1 -- 172.21.15.169:0/3840116210 >> 172.21.15.169:0/3840116210 conn(0x7f93d408bfc0 msgr2=0x7f93d40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:29.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.095+0000 7f93d8a00640 1 -- 172.21.15.169:0/3840116210 shutdown_connections 2024-05-30T16:43:29.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.095+0000 7f93d8a00640 1 -- 172.21.15.169:0/3840116210 wait complete. 2024-05-30T16:43:29.097 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.097+0000 7f93d8a00640 1 Processor -- start 2024-05-30T16:43:29.098 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.097+0000 7f93d8a00640 1 -- start start 2024-05-30T16:43:29.100 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40cf870 0x7f93d4158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.100 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40d14d0 0x7f93d40cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.100 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e870 0x7f93d4161170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.100 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d8a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40d0820 con 0x7f93d40d14d0 2024-05-30T16:43:29.100 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40d0540 con 0x7f93d415e870 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f93d40d4fb0 con 0x7f93d40cf870 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40cf870 0x7f93d4158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40cf870 0x7f93d4158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:43318/0 (socket says 172.21.15.169:43318) 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d3400640 1 -- 172.21.15.169:0/3987026544 learned_addr learned my addr 172.21.15.169:0/3987026544 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.099+0000 7f93d3e00640 1 --2- 172.21.15.169:0/3987026544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e870 0x7f93d4161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93d3400640 1 -- 172.21.15.169:0/3987026544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e870 msgr2=0x7f93d4161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93d3400640 1 --2- 172.21.15.169:0/3987026544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e870 0x7f93d4161170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93d2a00640 1 --2- 172.21.15.169:0/3987026544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40d14d0 0x7f93d40cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93d3400640 1 -- 172.21.15.169:0/3987026544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40d14d0 msgr2=0x7f93d40cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93d3400640 1 --2- 172.21.15.169:0/3987026544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40d14d0 0x7f93d40cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.101 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93d3400640 1 -- 172.21.15.169:0/3987026544 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d4159480 con 0x7f93d40cf870 2024-05-30T16:43:29.102 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93d3400640 1 --2- 172.21.15.169:0/3987026544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40cf870 0x7f93d4158f40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f93c402a2b0 tx=0x7f93c40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:29.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93cb400640 1 -- 172.21.15.169:0/3987026544 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4041020 con 0x7f93d40cf870 2024-05-30T16:43:29.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93cb400640 1 -- 172.21.15.169:0/3987026544 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c40306c0 con 0x7f93d40cf870 2024-05-30T16:43:29.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93cb400640 1 -- 172.21.15.169:0/3987026544 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4037040 con 0x7f93d40cf870 2024-05-30T16:43:29.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.100+0000 7f93d8a00640 1 -- 172.21.15.169:0/3987026544 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d40b7e00 con 0x7f93d40cf870 2024-05-30T16:43:29.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.101+0000 7f93d8a00640 1 -- 172.21.15.169:0/3987026544 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d40b83c0 con 0x7f93d40cf870 2024-05-30T16:43:29.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.101+0000 7f93cb400640 1 -- 172.21.15.169:0/3987026544 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f93c4044020 con 0x7f93d40cf870 2024-05-30T16:43:29.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.101+0000 7f93cb400640 1 -- 172.21.15.169:0/3987026544 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f93c403dc40 con 0x7f93d40cf870 2024-05-30T16:43:29.103 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.102+0000 7f93d8a00640 1 -- 172.21.15.169:0/3987026544 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d40689e0 con 0x7f93d40cf870 2024-05-30T16:43:29.105 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.105+0000 7f93cb400640 1 -- 172.21.15.169:0/3987026544 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f93c404b030 con 0x7f93d40cf870 2024-05-30T16:43:29.175 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:29.173+0000 7f0d1412e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-30T16:43:29.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.205+0000 7f93d8a00640 1 -- 172.21.15.169:0/3987026544 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d2137b46-ff54-4ae4-a961-cb73f43ba933", "id": 2} v 0) -- 0x7f93d40ba500 con 0x7f93d40cf870 2024-05-30T16:43:29.213 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.213+0000 7f93cb400640 1 -- 172.21.15.169:0/3987026544 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d2137b46-ff54-4ae4-a961-cb73f43ba933", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f93c403d420 con 0x7f93d40cf870 2024-05-30T16:43:29.213 INFO:teuthology.orchestra.run.smithi169.stdout:2 2024-05-30T16:43:29.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.214+0000 7f93d8a00640 1 -- 172.21.15.169:0/3987026544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40cf870 msgr2=0x7f93d4158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.214+0000 7f93d8a00640 1 --2- 172.21.15.169:0/3987026544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40cf870 0x7f93d4158f40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f93c402a2b0 tx=0x7f93c40046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.214+0000 7f93d8a00640 1 -- 172.21.15.169:0/3987026544 shutdown_connections 2024-05-30T16:43:29.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.214+0000 7f93d8a00640 1 --2- 172.21.15.169:0/3987026544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e870 0x7f93d4161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.214+0000 7f93d8a00640 1 --2- 172.21.15.169:0/3987026544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40d14d0 0x7f93d40cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.214+0000 7f93d8a00640 1 --2- 172.21.15.169:0/3987026544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40cf870 0x7f93d4158f40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.214+0000 7f93d8a00640 1 -- 172.21.15.169:0/3987026544 >> 172.21.15.169:0/3987026544 conn(0x7f93d408bfc0 msgr2=0x7f93d40b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:29.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.214+0000 7f93d8a00640 1 -- 172.21.15.169:0/3987026544 shutdown_connections 2024-05-30T16:43:29.215 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.214+0000 7f93d8a00640 1 -- 172.21.15.169:0/3987026544 wait complete. 2024-05-30T16:43:29.224 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:29.223+0000 7fc0fe093040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-30T16:43:29.231 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new e129317f-acfd-4f49-9b2e-ebb28399566c 3 2024-05-30T16:43:29.278 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:29.277+0000 7fc0fe093040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-30T16:43:29.409 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.408+0000 7ffa1a200640 1 Processor -- start 2024-05-30T16:43:29.409 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.408+0000 7ffa1a200640 1 -- start start 2024-05-30T16:43:29.410 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.409+0000 7ffa1a200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffa1415e870 0x7ffa140d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.410 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.409+0000 7ffa1a200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1415ec50 con 0x7ffa140d14d0 2024-05-30T16:43:29.410 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.409+0000 7ffa1a200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa140cff80 con 0x7ffa140cfa40 2024-05-30T16:43:29.410 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.409+0000 7ffa1a200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ffa140d2150 con 0x7ffa1415e870 2024-05-30T16:43:29.410 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.410+0000 7ffa19800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ffa140cfa40 0x7ffa140d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:56822/0 (socket says 172.21.15.169:56822) 2024-05-30T16:43:29.410 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.410+0000 7ffa19800640 1 -- 172.21.15.169:0/4037879498 learned_addr learned my addr 172.21.15.169:0/4037879498 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.410+0000 7ffa18e00640 1 --2- 172.21.15.169:0/4037879498 >> v2:172.21.15.67:3300/0 conn(0x7ffa1415e870 0x7ffa140d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.410+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2874872701 0 0) 0x7ffa140cff80 con 0x7ffa140cfa40 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.410+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9f8003520 con 0x7ffa140cfa40 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.410+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3452747120 0 0) 0x7ff9f8003520 con 0x7ffa140cfa40 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.410+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 >> v2:172.21.15.67:3300/0 conn(0x7ffa1415e870 msgr2=0x7ffa140d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.410+0000 7ffa13e00640 1 --2- 172.21.15.169:0/4037879498 >> v2:172.21.15.67:3300/0 conn(0x7ffa1415e870 0x7ffa140d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 >> v1:172.21.15.67:6789/0 conn(0x7ffa140d14d0 legacy=0x7ffa140cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa140d0260 con 0x7ffa140cfa40 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ffa0c0019f0 con 0x7ffa140cfa40 2024-05-30T16:43:29.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 >> v1:172.21.15.169:6789/0 conn(0x7ffa140cfa40 legacy=0x7ffa140d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8003a00 0x7ff9f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f8004800 0x7ff9f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f80051c0 0x7ff9f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffa140d2150 con 0x7ff9f8004800 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f800c0c0 con 0x7ff9f80051c0 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f800c2c0 con 0x7ff9f8003a00 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa18e00640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8003a00 0x7ff9f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa19800640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f80051c0 0x7ff9f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa0be00640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f8004800 0x7ff9f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa18e00640 1 -- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f80051c0 msgr2=0x7ff9f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa18e00640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f80051c0 0x7ff9f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.414 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa18e00640 1 -- 172.21.15.169:0/4037879498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f8004800 msgr2=0x7ff9f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:29.415 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa18e00640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f8004800 0x7ff9f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.415 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.411+0000 7ffa18e00640 1 -- 172.21.15.169:0/4037879498 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f8003520 con 0x7ff9f8003a00 2024-05-30T16:43:29.415 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.412+0000 7ffa19800640 1 -- 172.21.15.169:0/4037879498 reap_dead start 2024-05-30T16:43:29.415 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.412+0000 7ffa18e00640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8003a00 0x7ff9f8003e20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009aa0 tx=0x7ff9f8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4367ca8c98a7bdda server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:29.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04036070 con 0x7ff9f8003a00 2024-05-30T16:43:29.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa04031400 con 0x7ff9f8003a00 2024-05-30T16:43:29.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa13e00640 1 -- 172.21.15.169:0/4037879498 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04009670 con 0x7ff9f8003a00 2024-05-30T16:43:29.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa1a200640 1 -- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8003a00 msgr2=0x7ff9f8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa1a200640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8003a00 0x7ff9f8003e20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009aa0 tx=0x7ff9f8005830 comp rx=0 tx=0).stop 2024-05-30T16:43:29.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa1a200640 1 -- 172.21.15.169:0/4037879498 shutdown_connections 2024-05-30T16:43:29.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa1a200640 1 --2- 172.21.15.169:0/4037879498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8003a00 0x7ff9f8003e20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa1a200640 1 -- 172.21.15.169:0/4037879498 >> 172.21.15.169:0/4037879498 conn(0x7ffa1408bfc0 msgr2=0x7ffa140d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:29.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa1a200640 1 -- 172.21.15.169:0/4037879498 shutdown_connections 2024-05-30T16:43:29.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.413+0000 7ffa1a200640 1 -- 172.21.15.169:0/4037879498 wait complete. 2024-05-30T16:43:29.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.415+0000 7ffa1a200640 1 Processor -- start 2024-05-30T16:43:29.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.415+0000 7ffa1a200640 1 -- start start 2024-05-30T16:43:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa1a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f8003a00 0x7ffa14159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa1a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa1415c200 0x7ffa1415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa1a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa1415c680 0x7ffa1415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa1a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffa140d4fb0 con 0x7ff9f8003a00 2024-05-30T16:43:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa1a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa140d0820 con 0x7ffa1415c200 2024-05-30T16:43:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa1a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa140d0f90 con 0x7ffa1415c680 2024-05-30T16:43:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa19800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa1415c680 0x7ffa1415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa19800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa1415c680 0x7ffa1415baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:43338/0 (socket says 172.21.15.169:43338) 2024-05-30T16:43:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa19800640 1 -- 172.21.15.169:0/1582543404 learned_addr learned my addr 172.21.15.169:0/1582543404 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.416+0000 7ffa0be00640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa1415c200 0x7ffa1415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa18e00640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f8003a00 0x7ffa14159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa19800640 1 -- 172.21.15.169:0/1582543404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa1415c200 msgr2=0x7ffa1415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa19800640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa1415c200 0x7ffa1415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa19800640 1 -- 172.21.15.169:0/1582543404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f8003a00 msgr2=0x7ffa14159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa19800640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f8003a00 0x7ffa14159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa19800640 1 -- 172.21.15.169:0/1582543404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa140c8470 con 0x7ffa1415c680 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa0be00640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa1415c200 0x7ffa1415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa19800640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa1415c680 0x7ffa1415baa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffa140cfeb0 tx=0x7ffa0c007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa11600640 1 -- 172.21.15.169:0/1582543404 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c00ab30 con 0x7ffa1415c680 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa11600640 1 -- 172.21.15.169:0/1582543404 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0c018070 con 0x7ffa1415c680 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa11600640 1 -- 172.21.15.169:0/1582543404 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0133f0 con 0x7ffa1415c680 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa1a200640 1 -- 172.21.15.169:0/1582543404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa140c8750 con 0x7ffa1415c680 2024-05-30T16:43:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.417+0000 7ffa1a200640 1 -- 172.21.15.169:0/1582543404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa140c8d10 con 0x7ffa1415c680 2024-05-30T16:43:29.421 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.418+0000 7ffa11600640 1 -- 172.21.15.169:0/1582543404 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffa0c013590 con 0x7ffa1415c680 2024-05-30T16:43:29.421 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.418+0000 7ffa11600640 1 -- 172.21.15.169:0/1582543404 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7ffa0c01d460 con 0x7ffa1415c680 2024-05-30T16:43:29.421 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.418+0000 7ffa1a200640 1 -- 172.21.15.169:0/1582543404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa140689e0 con 0x7ffa1415c680 2024-05-30T16:43:29.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.423+0000 7ffa11600640 1 -- 172.21.15.169:0/1582543404 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffa0c013880 con 0x7ffa1415c680 2024-05-30T16:43:29.498 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:29.497+0000 7fc0fe093040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-30T16:43:29.502 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.502+0000 7ffa1a200640 1 -- 172.21.15.169:0/1582543404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e129317f-acfd-4f49-9b2e-ebb28399566c", "id": 3} v 0) -- 0x7ffa140ce9b0 con 0x7ffa1415c680 2024-05-30T16:43:29.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.510+0000 7ffa11600640 1 -- 172.21.15.169:0/1582543404 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e129317f-acfd-4f49-9b2e-ebb28399566c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7ffa0c013c10 con 0x7ffa1415c680 2024-05-30T16:43:29.511 INFO:teuthology.orchestra.run.smithi169.stdout:3 2024-05-30T16:43:29.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.511+0000 7ffa1a200640 1 -- 172.21.15.169:0/1582543404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa1415c680 msgr2=0x7ffa1415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.511+0000 7ffa1a200640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa1415c680 0x7ffa1415baa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffa140cfeb0 tx=0x7ffa0c007bd0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.511+0000 7ffa1a200640 1 -- 172.21.15.169:0/1582543404 shutdown_connections 2024-05-30T16:43:29.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.511+0000 7ffa1a200640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa1415c680 0x7ffa1415baa0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.511+0000 7ffa1a200640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa1415c200 0x7ffa1415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.511+0000 7ffa1a200640 1 --2- 172.21.15.169:0/1582543404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f8003a00 0x7ffa14159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.511+0000 7ffa1a200640 1 -- 172.21.15.169:0/1582543404 >> 172.21.15.169:0/1582543404 conn(0x7ffa1408bfc0 msgr2=0x7ffa140b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:29.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.512+0000 7ffa1a200640 1 -- 172.21.15.169:0/1582543404 shutdown_connections 2024-05-30T16:43:29.513 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.512+0000 7ffa1a200640 1 -- 172.21.15.169:0/1582543404 wait complete. 2024-05-30T16:43:29.527 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 45b623cb-599d-40fb-bf17-2eb0ae2476e7 4 2024-05-30T16:43:29.571 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:29.570+0000 7fc0fe093040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-30T16:43:29.638 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:29.637+0000 7fc0fe093040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-30T16:43:29.663 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:29.661+0000 7f0d1412e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-30T16:43:29.704 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.704+0000 7feb37000640 1 Processor -- start 2024-05-30T16:43:29.704 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.704+0000 7feb37000640 1 -- start start 2024-05-30T16:43:29.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.705+0000 7feb37000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feb3015b460 0x7feb300cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb37000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feb300b01a0 0x7feb300b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb37000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb300b5320 0x7feb3015ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb37000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7feb300b5790 con 0x7feb3015b460 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb37000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7feb3015b840 con 0x7feb300b01a0 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb37000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feb3006e700 con 0x7feb300b5320 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb35200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feb300b01a0 0x7feb300b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb36600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb300b5320 0x7feb3015ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb35200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feb300b01a0 0x7feb300b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54288/0 (socket says 172.21.15.169:54288) 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb35200640 1 -- 172.21.15.169:0/674002137 learned_addr learned my addr 172.21.15.169:0/674002137 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.706+0000 7feb35c00640 1 --2- 172.21.15.169:0/674002137 >> v2:172.21.15.67:3300/0 conn(0x7feb3015b460 0x7feb300cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.707+0000 7feb36600640 1 -- 172.21.15.169:0/674002137 >> v2:172.21.15.169:3300/0 conn(0x7feb300b01a0 msgr2=0x7feb300b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.707+0000 7feb36600640 1 --2- 172.21.15.169:0/674002137 >> v2:172.21.15.169:3300/0 conn(0x7feb300b01a0 0x7feb300b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.707+0000 7feb36600640 1 -- 172.21.15.169:0/674002137 >> v2:172.21.15.67:3300/0 conn(0x7feb3015b460 msgr2=0x7feb300cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.707+0000 7feb36600640 1 --2- 172.21.15.169:0/674002137 >> v2:172.21.15.67:3300/0 conn(0x7feb3015b460 0x7feb300cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.707+0000 7feb36600640 1 -- 172.21.15.169:0/674002137 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3006e9e0 con 0x7feb300b5320 2024-05-30T16:43:29.709 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.707+0000 7feb35200640 1 --2- 172.21.15.169:0/674002137 >> v2:172.21.15.169:3300/0 conn(0x7feb300b01a0 0x7feb300b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:29.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.708+0000 7feb36600640 1 --2- 172.21.15.169:0/674002137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb300b5320 0x7feb3015ef20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feb300b3c80 tx=0x7feb2802a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25be982f7134331 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:29.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.708+0000 7feb2fe00640 1 -- 172.21.15.169:0/674002137 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28031600 con 0x7feb300b5320 2024-05-30T16:43:29.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.708+0000 7feb2fe00640 1 -- 172.21.15.169:0/674002137 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb280317a0 con 0x7feb300b5320 2024-05-30T16:43:29.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.708+0000 7feb2fe00640 1 -- 172.21.15.169:0/674002137 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28031b70 con 0x7feb300b5320 2024-05-30T16:43:29.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.709+0000 7feb37000640 1 -- 172.21.15.169:0/674002137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb300b5320 msgr2=0x7feb3015ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.709+0000 7feb37000640 1 --2- 172.21.15.169:0/674002137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb300b5320 0x7feb3015ef20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feb300b3c80 tx=0x7feb2802a610 comp rx=0 tx=0).stop 2024-05-30T16:43:29.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.709+0000 7feb37000640 1 -- 172.21.15.169:0/674002137 shutdown_connections 2024-05-30T16:43:29.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.709+0000 7feb37000640 1 --2- 172.21.15.169:0/674002137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb300b5320 0x7feb3015ef20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.711 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.709+0000 7feb37000640 1 --2- 172.21.15.169:0/674002137 >> v2:172.21.15.169:3300/0 conn(0x7feb300b01a0 0x7feb300b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.711 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.709+0000 7feb37000640 1 --2- 172.21.15.169:0/674002137 >> v2:172.21.15.67:3300/0 conn(0x7feb3015b460 0x7feb300cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.711 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.709+0000 7feb37000640 1 -- 172.21.15.169:0/674002137 >> 172.21.15.169:0/674002137 conn(0x7feb300732f0 msgr2=0x7feb3006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:29.711 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.709+0000 7feb37000640 1 -- 172.21.15.169:0/674002137 shutdown_connections 2024-05-30T16:43:29.711 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.709+0000 7feb37000640 1 -- 172.21.15.169:0/674002137 wait complete. 2024-05-30T16:43:29.711 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.711+0000 7feb37000640 1 Processor -- start 2024-05-30T16:43:29.711 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.711+0000 7feb37000640 1 -- start start 2024-05-30T16:43:29.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb37000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb300b01a0 0x7feb30150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb37000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb300b5320 0x7feb3013d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb37000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb3015b460 0x7feb3015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:29.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb37000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feb3006f280 con 0x7feb300b01a0 2024-05-30T16:43:29.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb37000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feb3006efa0 con 0x7feb300b5320 2024-05-30T16:43:29.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb37000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feb3006e3d0 con 0x7feb3015b460 2024-05-30T16:43:29.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb35200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb300b5320 0x7feb3013d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb36600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb3015b460 0x7feb3015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb35200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb300b5320 0x7feb3013d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54302/0 (socket says 172.21.15.169:54302) 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb35200640 1 -- 172.21.15.169:0/761055306 learned_addr learned my addr 172.21.15.169:0/761055306 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.712+0000 7feb35c00640 1 --2- 172.21.15.169:0/761055306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb300b01a0 0x7feb30150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb35200640 1 -- 172.21.15.169:0/761055306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb3015b460 msgr2=0x7feb3015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb35200640 1 --2- 172.21.15.169:0/761055306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb3015b460 0x7feb3015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb35200640 1 -- 172.21.15.169:0/761055306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb300b01a0 msgr2=0x7feb30150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb35200640 1 --2- 172.21.15.169:0/761055306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb300b01a0 0x7feb30150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb35200640 1 -- 172.21.15.169:0/761055306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb301f4650 con 0x7feb300b5320 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb36600640 1 --2- 172.21.15.169:0/761055306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb3015b460 0x7feb3015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb35200640 1 --2- 172.21.15.169:0/761055306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb300b5320 0x7feb3013d9c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb1c006ca0 tx=0x7feb1c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb2d600640 1 -- 172.21.15.169:0/761055306 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c004030 con 0x7feb300b5320 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb2d600640 1 -- 172.21.15.169:0/761055306 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb1c0041d0 con 0x7feb300b5320 2024-05-30T16:43:29.715 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb2d600640 1 -- 172.21.15.169:0/761055306 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c00c910 con 0x7feb300b5320 2024-05-30T16:43:29.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb37000640 1 -- 172.21.15.169:0/761055306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3013df00 con 0x7feb300b5320 2024-05-30T16:43:29.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.713+0000 7feb37000640 1 -- 172.21.15.169:0/761055306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3013e4b0 con 0x7feb300b5320 2024-05-30T16:43:29.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.714+0000 7feb2d600640 1 -- 172.21.15.169:0/761055306 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb1c005990 con 0x7feb300b5320 2024-05-30T16:43:29.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.714+0000 7feb2d600640 1 -- 172.21.15.169:0/761055306 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7feb1c005c80 con 0x7feb300b5320 2024-05-30T16:43:29.716 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.715+0000 7feb37000640 1 -- 172.21.15.169:0/761055306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb300690f0 con 0x7feb300b5320 2024-05-30T16:43:29.717 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.717+0000 7feb2d600640 1 -- 172.21.15.169:0/761055306 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7feb1c017020 con 0x7feb300b5320 2024-05-30T16:43:29.774 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:29.773+0000 7fc0fe093040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-30T16:43:29.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.795+0000 7feb37000640 1 -- 172.21.15.169:0/761055306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "45b623cb-599d-40fb-bf17-2eb0ae2476e7", "id": 4} v 0) -- 0x7feb3013e850 con 0x7feb300b5320 2024-05-30T16:43:29.806 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.805+0000 7feb2d600640 1 -- 172.21.15.169:0/761055306 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "45b623cb-599d-40fb-bf17-2eb0ae2476e7", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7feb1c003610 con 0x7feb300b5320 2024-05-30T16:43:29.806 INFO:teuthology.orchestra.run.smithi169.stdout:4 2024-05-30T16:43:29.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.806+0000 7feb37000640 1 -- 172.21.15.169:0/761055306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb300b5320 msgr2=0x7feb3013d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:29.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.806+0000 7feb37000640 1 --2- 172.21.15.169:0/761055306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb300b5320 0x7feb3013d9c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb1c006ca0 tx=0x7feb1c00ed60 comp rx=0 tx=0).stop 2024-05-30T16:43:29.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.806+0000 7feb37000640 1 -- 172.21.15.169:0/761055306 shutdown_connections 2024-05-30T16:43:29.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.807+0000 7feb37000640 1 --2- 172.21.15.169:0/761055306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb3015b460 0x7feb3015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.807+0000 7feb37000640 1 --2- 172.21.15.169:0/761055306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb300b5320 0x7feb3013d9c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.808 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.807+0000 7feb37000640 1 --2- 172.21.15.169:0/761055306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb300b01a0 0x7feb30150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:29.808 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.807+0000 7feb37000640 1 -- 172.21.15.169:0/761055306 >> 172.21.15.169:0/761055306 conn(0x7feb300732f0 msgr2=0x7feb300cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:29.808 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.807+0000 7feb37000640 1 -- 172.21.15.169:0/761055306 shutdown_connections 2024-05-30T16:43:29.808 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:29.807+0000 7feb37000640 1 -- 172.21.15.169:0/761055306 wait complete. 2024-05-30T16:43:29.824 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7 5 2024-05-30T16:43:29.978 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:29.976+0000 7f0d1412e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-30T16:43:30.043 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:30.041+0000 7f0d1412e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-30T16:43:30.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.048+0000 7f3af6200640 1 Processor -- start 2024-05-30T16:43:30.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.048+0000 7f3af6200640 1 -- start start 2024-05-30T16:43:30.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ea40 0x7f3af00cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af6200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3af00d5190 0x7f3af00d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af6200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3af015ee20 0x7f3af015fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af6200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f3af00cecf0 con 0x7f3af00d5190 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af6200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3af00cefd0 con 0x7f3af015ee20 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3af00cf2b0 con 0x7f3af015ea40 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ea40 0x7f3af00cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af5800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3af015ee20 0x7f3af015fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ea40 0x7f3af00cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:43370/0 (socket says 172.21.15.169:43370) 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.050+0000 7f3af4e00640 1 -- 172.21.15.169:0/1636025035 learned_addr learned my addr 172.21.15.169:0/1636025035 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.051+0000 7f3aefe00640 1 --2- 172.21.15.169:0/1636025035 >> v2:172.21.15.67:3300/0 conn(0x7f3af00d5190 0x7f3af00d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.051+0000 7f3af4e00640 1 -- 172.21.15.169:0/1636025035 >> v2:172.21.15.169:3300/0 conn(0x7f3af015ee20 msgr2=0x7f3af015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.051+0000 7f3af4e00640 1 --2- 172.21.15.169:0/1636025035 >> v2:172.21.15.169:3300/0 conn(0x7f3af015ee20 0x7f3af015fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.051+0000 7f3af4e00640 1 -- 172.21.15.169:0/1636025035 >> v2:172.21.15.67:3300/0 conn(0x7f3af00d5190 msgr2=0x7f3af00d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:30.053 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.051+0000 7f3af4e00640 1 --2- 172.21.15.169:0/1636025035 >> v2:172.21.15.67:3300/0 conn(0x7f3af00d5190 0x7f3af00d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.053 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.051+0000 7f3af4e00640 1 -- 172.21.15.169:0/1636025035 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af00cf590 con 0x7f3af015ea40 2024-05-30T16:43:30.053 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.051+0000 7f3af5800640 1 --2- 172.21.15.169:0/1636025035 >> v2:172.21.15.169:3300/0 conn(0x7f3af015ee20 0x7f3af015fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:30.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.052+0000 7f3af4e00640 1 --2- 172.21.15.169:0/1636025035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ea40 0x7f3af00cfab0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3af00d0f30 tx=0x7f3ae002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a68dc412b052dce1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:30.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.052+0000 7f3aef400640 1 -- 172.21.15.169:0/1636025035 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0037070 con 0x7f3af015ea40 2024-05-30T16:43:30.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.052+0000 7f3aef400640 1 -- 172.21.15.169:0/1636025035 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae0032440 con 0x7f3af015ea40 2024-05-30T16:43:30.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.052+0000 7f3aef400640 1 -- 172.21.15.169:0/1636025035 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0032810 con 0x7f3af015ea40 2024-05-30T16:43:30.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.053+0000 7f3af6200640 1 -- 172.21.15.169:0/1636025035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ea40 msgr2=0x7f3af00cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.053+0000 7f3af6200640 1 --2- 172.21.15.169:0/1636025035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ea40 0x7f3af00cfab0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3af00d0f30 tx=0x7f3ae002a1e0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.053+0000 7f3af6200640 1 -- 172.21.15.169:0/1636025035 shutdown_connections 2024-05-30T16:43:30.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.053+0000 7f3af6200640 1 --2- 172.21.15.169:0/1636025035 >> v2:172.21.15.67:3300/0 conn(0x7f3af00d5190 0x7f3af00d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.053+0000 7f3af6200640 1 --2- 172.21.15.169:0/1636025035 >> v2:172.21.15.169:3300/0 conn(0x7f3af015ee20 0x7f3af015fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.053+0000 7f3af6200640 1 --2- 172.21.15.169:0/1636025035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ea40 0x7f3af00cfab0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.053+0000 7f3af6200640 1 -- 172.21.15.169:0/1636025035 >> 172.21.15.169:0/1636025035 conn(0x7f3af008bfc0 msgr2=0x7f3af00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:30.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.053+0000 7f3af6200640 1 -- 172.21.15.169:0/1636025035 shutdown_connections 2024-05-30T16:43:30.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.053+0000 7f3af6200640 1 -- 172.21.15.169:0/1636025035 wait complete. 2024-05-30T16:43:30.056 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.055+0000 7f3af6200640 1 Processor -- start 2024-05-30T16:43:30.056 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.055+0000 7f3af6200640 1 -- start start 2024-05-30T16:43:30.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af6200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3af00d5190 0x7f3af00b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af015ea40 0x7f3af00b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ee20 0x7f3af01f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af6200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00d3760 con 0x7f3af015ea40 2024-05-30T16:43:30.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af6200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00d35e0 con 0x7f3af00d5190 2024-05-30T16:43:30.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3af015f2d0 con 0x7f3af015ee20 2024-05-30T16:43:30.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3af00d5190 0x7f3af00b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ee20 0x7f3af01f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3af00d5190 0x7f3af00b70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54322/0 (socket says 172.21.15.169:54322) 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af4e00640 1 -- 172.21.15.169:0/336475429 learned_addr learned my addr 172.21.15.169:0/336475429 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3aefe00640 1 --2- 172.21.15.169:0/336475429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af015ea40 0x7f3af00b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af4e00640 1 -- 172.21.15.169:0/336475429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ee20 msgr2=0x7f3af01f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af4e00640 1 --2- 172.21.15.169:0/336475429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ee20 0x7f3af01f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af4e00640 1 -- 172.21.15.169:0/336475429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af015ea40 msgr2=0x7f3af00b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af4e00640 1 --2- 172.21.15.169:0/336475429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af015ea40 0x7f3af00b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af4e00640 1 -- 172.21.15.169:0/336475429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af0150d10 con 0x7f3af00d5190 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.056+0000 7f3af5800640 1 --2- 172.21.15.169:0/336475429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ee20 0x7f3af01f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.057+0000 7f3af4e00640 1 --2- 172.21.15.169:0/336475429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3af00d5190 0x7f3af00b70b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0000c00 tx=0x7f3ae00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.057+0000 7f3aecc00640 1 -- 172.21.15.169:0/336475429 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0041070 con 0x7f3af00d5190 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.057+0000 7f3aecc00640 1 -- 172.21.15.169:0/336475429 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae003c870 con 0x7f3af00d5190 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.057+0000 7f3af6200640 1 -- 172.21.15.169:0/336475429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af00b7740 con 0x7f3af00d5190 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.057+0000 7f3aecc00640 1 -- 172.21.15.169:0/336475429 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0037040 con 0x7f3af00d5190 2024-05-30T16:43:30.059 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.057+0000 7f3af6200640 1 -- 172.21.15.169:0/336475429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af00b7d00 con 0x7f3af00d5190 2024-05-30T16:43:30.060 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.057+0000 7f3aecc00640 1 -- 172.21.15.169:0/336475429 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3ae0044070 con 0x7f3af00d5190 2024-05-30T16:43:30.060 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.058+0000 7f3aecc00640 1 -- 172.21.15.169:0/336475429 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f3ae0046450 con 0x7f3af00d5190 2024-05-30T16:43:30.060 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.058+0000 7f3af6200640 1 -- 172.21.15.169:0/336475429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af00690f0 con 0x7f3af00d5190 2024-05-30T16:43:30.061 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.061+0000 7f3aecc00640 1 -- 172.21.15.169:0/336475429 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3ae003cc60 con 0x7f3af00d5190 2024-05-30T16:43:30.139 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.139+0000 7f3af6200640 1 -- 172.21.15.169:0/336475429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7", "id": 5} v 0) -- 0x7f3af008e620 con 0x7f3af00d5190 2024-05-30T16:43:30.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.154+0000 7f3aecc00640 1 -- 172.21.15.169:0/336475429 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f3ae00468b0 con 0x7f3af00d5190 2024-05-30T16:43:30.155 INFO:teuthology.orchestra.run.smithi169.stdout:5 2024-05-30T16:43:30.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.156+0000 7f3af6200640 1 -- 172.21.15.169:0/336475429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3af00d5190 msgr2=0x7f3af00b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.156+0000 7f3af6200640 1 --2- 172.21.15.169:0/336475429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3af00d5190 0x7f3af00b70b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0000c00 tx=0x7f3ae00046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.156+0000 7f3af6200640 1 -- 172.21.15.169:0/336475429 shutdown_connections 2024-05-30T16:43:30.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.156+0000 7f3af6200640 1 --2- 172.21.15.169:0/336475429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3af015ee20 0x7f3af01f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.156+0000 7f3af6200640 1 --2- 172.21.15.169:0/336475429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3af015ea40 0x7f3af00b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.156+0000 7f3af6200640 1 --2- 172.21.15.169:0/336475429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3af00d5190 0x7f3af00b70b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.156+0000 7f3af6200640 1 -- 172.21.15.169:0/336475429 >> 172.21.15.169:0/336475429 conn(0x7f3af008bfc0 msgr2=0x7f3af01621a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:30.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.156+0000 7f3af6200640 1 -- 172.21.15.169:0/336475429 shutdown_connections 2024-05-30T16:43:30.157 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.156+0000 7f3af6200640 1 -- 172.21.15.169:0/336475429 wait complete. 2024-05-30T16:43:30.174 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 0dd05747-ca8a-4e42-b854-abaa79659a89 6 2024-05-30T16:43:30.183 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:30.181+0000 7f0d1412e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-30T16:43:30.256 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:30.256+0000 7fc0fe093040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-30T16:43:30.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.357+0000 7fd70fa00640 1 Processor -- start 2024-05-30T16:43:30.358 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.357+0000 7fd70fa00640 1 -- start start 2024-05-30T16:43:30.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.359+0000 7fd70fa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7080cef30 0x7fd7080d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.359+0000 7fd70fa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd7080d1b80 0x7fd7080b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.359+0000 7fd70fa00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7080d1f60 con 0x7fd70815e850 2024-05-30T16:43:30.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.359+0000 7fd70fa00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd70815ec30 con 0x7fd7080d1b80 2024-05-30T16:43:30.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.359+0000 7fd70fa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd7080b8000 con 0x7fd7080cef30 2024-05-30T16:43:30.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.359+0000 7fd707e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7080cef30 0x7fd7080d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.359+0000 7fd707e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7080cef30 0x7fd7080d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54328/0 (socket says 172.21.15.169:54328) 2024-05-30T16:43:30.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.359+0000 7fd707e00640 1 -- 172.21.15.169:0/2327334526 learned_addr learned my addr 172.21.15.169:0/2327334526 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:30.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.359+0000 7fd70da00640 1 --2- 172.21.15.169:0/2327334526 >> v2:172.21.15.67:3300/0 conn(0x7fd7080d1b80 0x7fd7080b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.360+0000 7fd707e00640 1 -- 172.21.15.169:0/2327334526 >> v2:172.21.15.67:3300/0 conn(0x7fd7080d1b80 msgr2=0x7fd7080b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:30.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.360+0000 7fd707e00640 1 --2- 172.21.15.169:0/2327334526 >> v2:172.21.15.67:3300/0 conn(0x7fd7080d1b80 0x7fd7080b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.360+0000 7fd707e00640 1 -- 172.21.15.169:0/2327334526 >> v1:172.21.15.67:6789/0 conn(0x7fd70815e850 legacy=0x7fd7080b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:30.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.360+0000 7fd707e00640 1 -- 172.21.15.169:0/2327334526 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7080b82e0 con 0x7fd7080cef30 2024-05-30T16:43:30.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.361+0000 7fd707e00640 1 --2- 172.21.15.169:0/2327334526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7080cef30 0x7fd7080d1640 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7080cb730 tx=0x7fd6f402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=315eb026398312ec server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:30.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.361+0000 7fd707400640 1 -- 172.21.15.169:0/2327334526 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4037070 con 0x7fd7080cef30 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.361+0000 7fd707400640 1 -- 172.21.15.169:0/2327334526 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f4032440 con 0x7fd7080cef30 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.361+0000 7fd707400640 1 -- 172.21.15.169:0/2327334526 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4032810 con 0x7fd7080cef30 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.362+0000 7fd70fa00640 1 -- 172.21.15.169:0/2327334526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7080cef30 msgr2=0x7fd7080d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.362+0000 7fd70fa00640 1 --2- 172.21.15.169:0/2327334526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7080cef30 0x7fd7080d1640 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7080cb730 tx=0x7fd6f402a1a0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.362+0000 7fd70fa00640 1 -- 172.21.15.169:0/2327334526 shutdown_connections 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.362+0000 7fd70fa00640 1 --2- 172.21.15.169:0/2327334526 >> v2:172.21.15.67:3300/0 conn(0x7fd7080d1b80 0x7fd7080b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.362+0000 7fd70fa00640 1 --2- 172.21.15.169:0/2327334526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7080cef30 0x7fd7080d1640 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.362+0000 7fd70fa00640 1 -- 172.21.15.169:0/2327334526 >> 172.21.15.169:0/2327334526 conn(0x7fd708067aa0 msgr2=0x7fd708067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.362+0000 7fd70fa00640 1 -- 172.21.15.169:0/2327334526 shutdown_connections 2024-05-30T16:43:30.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.362+0000 7fd70fa00640 1 -- 172.21.15.169:0/2327334526 wait complete. 2024-05-30T16:43:30.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.364+0000 7fd70fa00640 1 Processor -- start 2024-05-30T16:43:30.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.364+0000 7fd70fa00640 1 -- start start 2024-05-30T16:43:30.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.364+0000 7fd70fa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7080cef30 0x7fd70815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd70fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7080d1b80 0x7fd7080d2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd70fa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70815e850 0x7fd70815fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd70fa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd7080b95e0 con 0x7fd7080cef30 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd70fa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7080b88a0 con 0x7fd70815e850 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd70fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd7080b9760 con 0x7fd7080d1b80 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd707e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7080d1b80 0x7fd7080d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd70da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70815e850 0x7fd70815fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd707e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7080d1b80 0x7fd7080d2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:43392/0 (socket says 172.21.15.169:43392) 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd707e00640 1 -- 172.21.15.169:0/85416383 learned_addr learned my addr 172.21.15.169:0/85416383 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd70d000640 1 --2- 172.21.15.169:0/85416383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7080cef30 0x7fd70815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd707e00640 1 -- 172.21.15.169:0/85416383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70815e850 msgr2=0x7fd70815fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd707e00640 1 --2- 172.21.15.169:0/85416383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70815e850 0x7fd70815fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd707e00640 1 -- 172.21.15.169:0/85416383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7080cef30 msgr2=0x7fd70815a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd707e00640 1 --2- 172.21.15.169:0/85416383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7080cef30 0x7fd70815a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.365+0000 7fd707e00640 1 -- 172.21.15.169:0/85416383 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7080d32e0 con 0x7fd7080d1b80 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.366+0000 7fd707e00640 1 --2- 172.21.15.169:0/85416383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7080d1b80 0x7fd7080d2130 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009960 tx=0x7fd6f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.366+0000 7fd704c00640 1 -- 172.21.15.169:0/85416383 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4041020 con 0x7fd7080d1b80 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.366+0000 7fd704c00640 1 -- 172.21.15.169:0/85416383 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f40307a0 con 0x7fd7080d1b80 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.366+0000 7fd70fa00640 1 -- 172.21.15.169:0/85416383 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7080d35c0 con 0x7fd7080d1b80 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.366+0000 7fd704c00640 1 -- 172.21.15.169:0/85416383 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4037040 con 0x7fd7080d1b80 2024-05-30T16:43:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.366+0000 7fd70fa00640 1 -- 172.21.15.169:0/85416383 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7080ccb30 con 0x7fd7080d1b80 2024-05-30T16:43:30.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.366+0000 7fd704c00640 1 -- 172.21.15.169:0/85416383 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6f4044020 con 0x7fd7080d1b80 2024-05-30T16:43:30.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.367+0000 7fd704c00640 1 -- 172.21.15.169:0/85416383 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fd6f403dce0 con 0x7fd7080d1b80 2024-05-30T16:43:30.369 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.367+0000 7fd70fa00640 1 -- 172.21.15.169:0/85416383 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7080689e0 con 0x7fd7080d1b80 2024-05-30T16:43:30.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.370+0000 7fd704c00640 1 -- 172.21.15.169:0/85416383 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd6f404b030 con 0x7fd7080d1b80 2024-05-30T16:43:30.448 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.448+0000 7fd70fa00640 1 -- 172.21.15.169:0/85416383 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0dd05747-ca8a-4e42-b854-abaa79659a89", "id": 6} v 0) -- 0x7fd7080b7830 con 0x7fd7080d1b80 2024-05-30T16:43:30.458 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.458+0000 7fd704c00640 1 -- 172.21.15.169:0/85416383 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0dd05747-ca8a-4e42-b854-abaa79659a89", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fd6f403d420 con 0x7fd7080d1b80 2024-05-30T16:43:30.458 INFO:teuthology.orchestra.run.smithi169.stdout:6 2024-05-30T16:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.459+0000 7fd70fa00640 1 -- 172.21.15.169:0/85416383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7080d1b80 msgr2=0x7fd7080d2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.459+0000 7fd70fa00640 1 --2- 172.21.15.169:0/85416383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7080d1b80 0x7fd7080d2130 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009960 tx=0x7fd6f40046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.459+0000 7fd70fa00640 1 -- 172.21.15.169:0/85416383 shutdown_connections 2024-05-30T16:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.459+0000 7fd70fa00640 1 --2- 172.21.15.169:0/85416383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd70815e850 0x7fd70815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.459+0000 7fd70fa00640 1 --2- 172.21.15.169:0/85416383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7080d1b80 0x7fd7080d2130 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.459+0000 7fd70fa00640 1 --2- 172.21.15.169:0/85416383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7080cef30 0x7fd70815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.459+0000 7fd70fa00640 1 -- 172.21.15.169:0/85416383 >> 172.21.15.169:0/85416383 conn(0x7fd708067aa0 msgr2=0x7fd708162190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.459+0000 7fd70fa00640 1 -- 172.21.15.169:0/85416383 shutdown_connections 2024-05-30T16:43:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.459+0000 7fd70fa00640 1 -- 172.21.15.169:0/85416383 wait complete. 2024-05-30T16:43:30.477 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new bfddf4bf-a666-4bf3-97d5-93815ab9a518 7 2024-05-30T16:43:30.532 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:30.530+0000 7f0d1412e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-30T16:43:30.575 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:30.575+0000 7fc0fe093040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-30T16:43:30.591 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:30.589+0000 7f0d1412e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-30T16:43:30.642 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.642+0000 7fbb72a00640 1 Processor -- start 2024-05-30T16:43:30.642 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:30.642+0000 7fc0fe093040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-30T16:43:30.643 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.642+0000 7fbb72a00640 1 -- start start 2024-05-30T16:43:30.645 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.644+0000 7fbb72a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb6c15e850 0x7fbb6c0b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.645 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.644+0000 7fbb72a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6c0b8000 con 0x7fbb6c0cef30 2024-05-30T16:43:30.645 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.644+0000 7fbb72a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6c0b82e0 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.645 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.644+0000 7fbb72a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbb6c15ec30 con 0x7fbb6c15e850 2024-05-30T16:43:30.645 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.644+0000 7fbb6be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb6c15e850 0x7fbb6c0b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.645 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.644+0000 7fbb6be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb6c15e850 0x7fbb6c0b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54342/0 (socket says 172.21.15.169:54342) 2024-05-30T16:43:30.645 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.644+0000 7fbb6be00640 1 -- 172.21.15.169:0/181643686 learned_addr learned my addr 172.21.15.169:0/181643686 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:30.646 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.644+0000 7fbb70a00640 1 --1- 172.21.15.169:0/181643686 >> v1:172.21.15.169:6789/0 conn(0x7fbb6c0d1d50 0x7fbb6c0b90a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:56834/0 (socket says 172.21.15.169:56834) 2024-05-30T16:43:30.646 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.645+0000 7fbb6be00640 1 -- 172.21.15.169:0/181643686 >> v1:172.21.15.169:6789/0 conn(0x7fbb6c0d1d50 legacy=0x7fbb6c0b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.646 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.645+0000 7fbb6be00640 1 -- 172.21.15.169:0/181643686 >> v1:172.21.15.67:6789/0 conn(0x7fbb6c0cef30 legacy=0x7fbb6c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.646 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.645+0000 7fbb6be00640 1 -- 172.21.15.169:0/181643686 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c0b85c0 con 0x7fbb6c15e850 2024-05-30T16:43:30.646 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.646+0000 7fbb6be00640 1 --2- 172.21.15.169:0/181643686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 0x7fbb6c0b77b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009870 tx=0x7fbb5c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b74de4abdc3ba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.646+0000 7fbb6aa00640 1 -- 172.21.15.169:0/181643686 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c037070 con 0x7fbb6c15e850 2024-05-30T16:43:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.646+0000 7fbb6aa00640 1 -- 172.21.15.169:0/181643686 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5c032440 con 0x7fbb6c15e850 2024-05-30T16:43:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.646+0000 7fbb6aa00640 1 -- 172.21.15.169:0/181643686 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c032810 con 0x7fbb6c15e850 2024-05-30T16:43:30.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.647+0000 7fbb72a00640 1 -- 172.21.15.169:0/181643686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 msgr2=0x7fbb6c0b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.647+0000 7fbb72a00640 1 --2- 172.21.15.169:0/181643686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 0x7fbb6c0b77b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009870 tx=0x7fbb5c02a270 comp rx=0 tx=0).stop 2024-05-30T16:43:30.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.647+0000 7fbb72a00640 1 -- 172.21.15.169:0/181643686 shutdown_connections 2024-05-30T16:43:30.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.647+0000 7fbb72a00640 1 --2- 172.21.15.169:0/181643686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 0x7fbb6c0b77b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.647+0000 7fbb72a00640 1 -- 172.21.15.169:0/181643686 >> 172.21.15.169:0/181643686 conn(0x7fbb6c067aa0 msgr2=0x7fbb6c067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:30.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.647+0000 7fbb72a00640 1 -- 172.21.15.169:0/181643686 shutdown_connections 2024-05-30T16:43:30.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.647+0000 7fbb72a00640 1 -- 172.21.15.169:0/181643686 wait complete. 2024-05-30T16:43:30.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.649+0000 7fbb72a00640 1 Processor -- start 2024-05-30T16:43:30.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.649+0000 7fbb72a00640 1 -- start start 2024-05-30T16:43:30.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb72a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb6c0cef30 0x7fbb6c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb72a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb6c0d1d50 0x7fbb6c0d4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb72a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 0x7fbb6c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:30.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb72a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c0b97b0 con 0x7fbb6c0cef30 2024-05-30T16:43:30.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb72a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c0b7cf0 con 0x7fbb6c15e850 2024-05-30T16:43:30.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb72a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6c0b9930 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb6b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb6c0d1d50 0x7fbb6c0d4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb70a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 0x7fbb6c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb6b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb6c0d1d50 0x7fbb6c0d4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:43404/0 (socket says 172.21.15.169:43404) 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb6b400640 1 -- 172.21.15.169:0/180980367 learned_addr learned my addr 172.21.15.169:0/180980367 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.650+0000 7fbb6be00640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb6c0cef30 0x7fbb6c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb6b400640 1 -- 172.21.15.169:0/180980367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 msgr2=0x7fbb6c150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb6b400640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 0x7fbb6c150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb6b400640 1 -- 172.21.15.169:0/180980367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb6c0cef30 msgr2=0x7fbb6c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb6b400640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb6c0cef30 0x7fbb6c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb6b400640 1 -- 172.21.15.169:0/180980367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c0d54a0 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb70a00640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 0x7fbb6c150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:30.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb6be00640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb6c0cef30 0x7fbb6c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:43:30.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb6b400640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb6c0d1d50 0x7fbb6c0d4f60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c400 tx=0x7fbb5800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:30.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb63e00640 1 -- 172.21.15.169:0/180980367 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58007020 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb63e00640 1 -- 172.21.15.169:0/180980367 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb580071c0 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb63e00640 1 -- 172.21.15.169:0/180980367 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580075b0 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.651+0000 7fbb72a00640 1 -- 172.21.15.169:0/180980367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6c0ccf60 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.652+0000 7fbb72a00640 1 -- 172.21.15.169:0/180980367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6c0cd800 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.652+0000 7fbb63e00640 1 -- 172.21.15.169:0/180980367 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbb58019020 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.652+0000 7fbb63e00640 1 -- 172.21.15.169:0/180980367 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fbb580067d0 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.655 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.653+0000 7fbb72a00640 1 -- 172.21.15.169:0/180980367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb6c0689e0 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.656 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.655+0000 7fbb63e00640 1 -- 172.21.15.169:0/180980367 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbb58005410 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.734+0000 7fbb72a00640 1 -- 172.21.15.169:0/180980367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bfddf4bf-a666-4bf3-97d5-93815ab9a518", "id": 7} v 0) -- 0x7fbb6c06bb00 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.740 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.740+0000 7fbb63e00640 1 -- 172.21.15.169:0/180980367 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bfddf4bf-a666-4bf3-97d5-93815ab9a518", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fbb58006aa0 con 0x7fbb6c0d1d50 2024-05-30T16:43:30.740 INFO:teuthology.orchestra.run.smithi169.stdout:7 2024-05-30T16:43:30.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.741+0000 7fbb72a00640 1 -- 172.21.15.169:0/180980367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb6c0d1d50 msgr2=0x7fbb6c0d4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:30.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.741+0000 7fbb72a00640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb6c0d1d50 0x7fbb6c0d4f60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c400 tx=0x7fbb5800c8c0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.741+0000 7fbb72a00640 1 -- 172.21.15.169:0/180980367 shutdown_connections 2024-05-30T16:43:30.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.741+0000 7fbb72a00640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c15e850 0x7fbb6c150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.741+0000 7fbb72a00640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb6c0d1d50 0x7fbb6c0d4f60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.741+0000 7fbb72a00640 1 --2- 172.21.15.169:0/180980367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb6c0cef30 0x7fbb6c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:30.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.741+0000 7fbb72a00640 1 -- 172.21.15.169:0/180980367 >> 172.21.15.169:0/180980367 conn(0x7fbb6c067aa0 msgr2=0x7fbb6c074a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:30.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.741+0000 7fbb72a00640 1 -- 172.21.15.169:0/180980367 shutdown_connections 2024-05-30T16:43:30.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:43:30.742+0000 7fbb72a00640 1 -- 172.21.15.169:0/180980367 wait complete. 2024-05-30T16:43:30.758 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-30T16:43:30.758 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-30T16:43:30.766 INFO:tasks.ceph.osd.0:Started 2024-05-30T16:43:30.766 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-30T16:43:30.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-30T16:43:30.773 INFO:tasks.ceph.osd.1:Started 2024-05-30T16:43:30.773 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-30T16:43:30.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-30T16:43:30.778 INFO:tasks.ceph.osd.2:Started 2024-05-30T16:43:30.778 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-30T16:43:30.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-30T16:43:30.783 INFO:tasks.ceph.osd.3:Started 2024-05-30T16:43:30.784 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-30T16:43:30.784 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-30T16:43:30.787 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:30.786+0000 7f0d1412e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-30T16:43:30.789 INFO:tasks.ceph.osd.4:Started 2024-05-30T16:43:30.789 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-30T16:43:30.789 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-30T16:43:30.792 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:30.786+0000 7fc0fe093040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-30T16:43:30.796 INFO:tasks.ceph.osd.5:Started 2024-05-30T16:43:30.796 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-30T16:43:30.796 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-30T16:43:30.801 INFO:tasks.ceph.osd.6:Started 2024-05-30T16:43:30.802 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-30T16:43:30.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-30T16:43:30.807 INFO:tasks.ceph.osd.7:Started 2024-05-30T16:43:30.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:43:31.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.007+0000 7fd24c400640 1 Processor -- start 2024-05-30T16:43:31.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.007+0000 7fd24c400640 1 -- start start 2024-05-30T16:43:31.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.008+0000 7fd24c400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2440b3b50 0x7fd2440d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:31.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.008+0000 7fd24c400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd24415ed20 0x7fd24415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:31.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.008+0000 7fd24c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2440d36d0 con 0x7fd2440d2db0 2024-05-30T16:43:31.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.008+0000 7fd24c400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd2440d5540 con 0x7fd2440b3b50 2024-05-30T16:43:31.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.008+0000 7fd24c400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd244163500 con 0x7fd24415ed20 2024-05-30T16:43:31.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.008+0000 7fd249000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2440b3b50 0x7fd2440d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:31.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.008+0000 7fd249000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2440b3b50 0x7fd2440d3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37672/0 (socket says 172.21.15.67:37672) 2024-05-30T16:43:31.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.008+0000 7fd249000640 1 -- 172.21.15.67:0/1173349597 learned_addr learned my addr 172.21.15.67:0/1173349597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:31.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.008+0000 7fd24a400640 1 --2- 172.21.15.67:0/1173349597 >> v2:172.21.15.169:3300/0 conn(0x7fd24415ed20 0x7fd24415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:31.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.009+0000 7fd24a400640 1 -- 172.21.15.67:0/1173349597 >> v2:172.21.15.67:3300/0 conn(0x7fd2440b3b50 msgr2=0x7fd2440d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:31.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.009+0000 7fd24a400640 1 --2- 172.21.15.67:0/1173349597 >> v2:172.21.15.67:3300/0 conn(0x7fd2440b3b50 0x7fd2440d3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:31.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.009+0000 7fd24a400640 1 -- 172.21.15.67:0/1173349597 >> v1:172.21.15.169:6789/0 conn(0x7fd2440d2db0 legacy=0x7fd2440d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:31.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.009+0000 7fd24a400640 1 -- 172.21.15.67:0/1173349597 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd244162740 con 0x7fd24415ed20 2024-05-30T16:43:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.010+0000 7fd24a400640 1 --2- 172.21.15.67:0/1173349597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24415ed20 0x7fd24415f1b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd2440cec30 tx=0x7fd23c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=292cb582c7ceecc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.010+0000 7fd243e00640 1 -- 172.21.15.67:0/1173349597 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c007740 con 0x7fd24415ed20 2024-05-30T16:43:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.010+0000 7fd243e00640 1 -- 172.21.15.67:0/1173349597 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c0078e0 con 0x7fd24415ed20 2024-05-30T16:43:31.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.010+0000 7fd243e00640 1 -- 172.21.15.67:0/1173349597 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c007cb0 con 0x7fd24415ed20 2024-05-30T16:43:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.011+0000 7fd24c400640 1 -- 172.21.15.67:0/1173349597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24415ed20 msgr2=0x7fd24415f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.011+0000 7fd24c400640 1 --2- 172.21.15.67:0/1173349597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24415ed20 0x7fd24415f1b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd2440cec30 tx=0x7fd23c02a5f0 comp rx=0 tx=0).stop 2024-05-30T16:43:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.011+0000 7fd24c400640 1 -- 172.21.15.67:0/1173349597 shutdown_connections 2024-05-30T16:43:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.011+0000 7fd24c400640 1 --2- 172.21.15.67:0/1173349597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24415ed20 0x7fd24415f1b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.011+0000 7fd24c400640 1 --2- 172.21.15.67:0/1173349597 >> v2:172.21.15.67:3300/0 conn(0x7fd2440b3b50 0x7fd2440d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.011+0000 7fd24c400640 1 -- 172.21.15.67:0/1173349597 >> 172.21.15.67:0/1173349597 conn(0x7fd24408bfc0 msgr2=0x7fd2440d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.011+0000 7fd24c400640 1 -- 172.21.15.67:0/1173349597 shutdown_connections 2024-05-30T16:43:31.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.011+0000 7fd24c400640 1 -- 172.21.15.67:0/1173349597 wait complete. 2024-05-30T16:43:31.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.012+0000 7fd24c400640 1 Processor -- start 2024-05-30T16:43:31.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.013+0000 7fd24c400640 1 -- start start 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.013+0000 7fd24c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2440b3b50 0x7fd2440b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.013+0000 7fd24c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2440d2db0 0x7fd244154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.013+0000 7fd24c400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd24415ed20 0x7fd2440b9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.013+0000 7fd24c400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd244162fe0 con 0x7fd24415ed20 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.013+0000 7fd24c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd244162d00 con 0x7fd2440b3b50 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.013+0000 7fd24c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd2440d04a0 con 0x7fd2440d2db0 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd24a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd24415ed20 0x7fd2440b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd249a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2440b3b50 0x7fd2440b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd24a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd24415ed20 0x7fd2440b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37676/0 (socket says 172.21.15.67:37676) 2024-05-30T16:43:31.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd249000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2440d2db0 0x7fd244154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd24a400640 1 -- 172.21.15.67:0/3243926066 learned_addr learned my addr 172.21.15.67:0/3243926066 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd24a400640 1 -- 172.21.15.67:0/3243926066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2440d2db0 msgr2=0x7fd244154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd24a400640 1 --2- 172.21.15.67:0/3243926066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2440d2db0 0x7fd244154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd24a400640 1 -- 172.21.15.67:0/3243926066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2440b3b50 msgr2=0x7fd2440b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd24a400640 1 --2- 172.21.15.67:0/3243926066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2440b3b50 0x7fd2440b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:31.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.014+0000 7fd24a400640 1 -- 172.21.15.67:0/3243926066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2440cd040 con 0x7fd24415ed20 2024-05-30T16:43:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.015+0000 7fd24a400640 1 --2- 172.21.15.67:0/3243926066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd24415ed20 0x7fd2440b9780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009e50 tx=0x7fd23c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.015+0000 7fd241600640 1 -- 172.21.15.67:0/3243926066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c040070 con 0x7fd24415ed20 2024-05-30T16:43:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.015+0000 7fd241600640 1 -- 172.21.15.67:0/3243926066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c031bf0 con 0x7fd24415ed20 2024-05-30T16:43:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.015+0000 7fd24c400640 1 -- 172.21.15.67:0/3243926066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2440cd320 con 0x7fd24415ed20 2024-05-30T16:43:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.015+0000 7fd241600640 1 -- 172.21.15.67:0/3243926066 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c009450 con 0x7fd24415ed20 2024-05-30T16:43:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.015+0000 7fd24c400640 1 -- 172.21.15.67:0/3243926066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2440cd8e0 con 0x7fd24415ed20 2024-05-30T16:43:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.015+0000 7fd241600640 1 -- 172.21.15.67:0/3243926066 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd23c039040 con 0x7fd24415ed20 2024-05-30T16:43:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.015+0000 7fd241600640 1 -- 172.21.15.67:0/3243926066 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd23c045e00 con 0x7fd24415ed20 2024-05-30T16:43:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.016+0000 7fd24c400640 1 -- 172.21.15.67:0/3243926066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24415d440 con 0x7fd24415ed20 2024-05-30T16:43:31.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.019+0000 7fd241600640 1 -- 172.21.15.67:0/3243926066 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd24415d440 con 0x7fd24415ed20 2024-05-30T16:43:31.021 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:31.019+0000 7f0d1412e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-30T16:43:31.083 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:31.082+0000 7f0d1412e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-30T16:43:31.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.085+0000 7fd24c400640 1 -- 172.21.15.67:0/3243926066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2440cb7e0 con 0x7fd24415ed20 2024-05-30T16:43:31.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.085+0000 7fd241600640 1 -- 172.21.15.67:0/3243926066 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd23c007840 con 0x7fd24415ed20 2024-05-30T16:43:31.087 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:31.087 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":10,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:30.736505+0000","last_up_change":"0.000000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:43:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.087+0000 7fd24c400640 1 -- 172.21.15.67:0/3243926066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd24415ed20 msgr2=0x7fd2440b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.087+0000 7fd24c400640 1 --2- 172.21.15.67:0/3243926066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd24415ed20 0x7fd2440b9780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009e50 tx=0x7fd23c004a40 comp rx=0 tx=0).stop 2024-05-30T16:43:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.087+0000 7fd24c400640 1 -- 172.21.15.67:0/3243926066 shutdown_connections 2024-05-30T16:43:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.087+0000 7fd24c400640 1 --2- 172.21.15.67:0/3243926066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd24415ed20 0x7fd2440b9780 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.087+0000 7fd24c400640 1 --2- 172.21.15.67:0/3243926066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2440d2db0 0x7fd244154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.087+0000 7fd24c400640 1 --2- 172.21.15.67:0/3243926066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2440b3b50 0x7fd2440b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.087+0000 7fd24c400640 1 -- 172.21.15.67:0/3243926066 >> 172.21.15.67:0/3243926066 conn(0x7fd24408bfc0 msgr2=0x7fd24415cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.087+0000 7fd24c400640 1 -- 172.21.15.67:0/3243926066 shutdown_connections 2024-05-30T16:43:31.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:31.087+0000 7fd24c400640 1 -- 172.21.15.67:0/3243926066 wait complete. 2024-05-30T16:43:31.104 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-30T16:43:31.104 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-30T16:43:31.171 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:31.169+0000 7f0d1412e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-30T16:43:31.190 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:31.189+0000 7fc0fe093040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-30T16:43:31.228 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:31.226+0000 7f0d1412e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-30T16:43:31.248 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:31.247+0000 7fc0fe093040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-30T16:43:31.434 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:31.434+0000 7fc0fe093040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-30T16:43:31.471 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-05-30T16:43:31.469+0000 7f9819757640 -1 Falling back to public interface 2024-05-30T16:43:31.486 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-05-30T16:43:31.485+0000 7f7a9d80d640 -1 Falling back to public interface 2024-05-30T16:43:31.492 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-05-30T16:43:31.490+0000 7f488cc0f640 -1 Falling back to public interface 2024-05-30T16:43:31.495 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-05-30T16:43:31.494+0000 7fbaa3a2d640 -1 Falling back to public interface 2024-05-30T16:43:31.503 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-05-30T16:43:31.503+0000 7f64e35a7640 -1 Falling back to public interface 2024-05-30T16:43:31.504 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-05-30T16:43:31.503+0000 7f3d6281c640 -1 Falling back to public interface 2024-05-30T16:43:31.505 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-05-30T16:43:31.505+0000 7fcded25d640 -1 Falling back to public interface 2024-05-30T16:43:31.517 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-05-30T16:43:31.517+0000 7f4609e41640 -1 Falling back to public interface 2024-05-30T16:43:31.609 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:31.608+0000 7fc0fe093040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-30T16:43:31.669 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:31.668+0000 7fc0fe093040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-30T16:43:31.751 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:31.750+0000 7fc0fe093040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-30T16:43:31.807 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:31.806+0000 7fc0fe093040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-30T16:43:31.814 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:31.813+0000 7f0d1412e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-30T16:43:31.911 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-30T16:43:31.954 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:31.953+0000 7f0d1412e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-30T16:43:32.083 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.081+0000 7f8f1ae00640 1 Processor -- start 2024-05-30T16:43:32.083 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.081+0000 7f8f1ae00640 1 -- start start 2024-05-30T16:43:32.084 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.082+0000 7f8f1ae00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f14161c90 0x7f8f140d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:32.084 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.082+0000 7f8f1ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f14162b10 0x7f8f140d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.082+0000 7f8f1ae00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f140cf310 con 0x7f8f140d2c50 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.082+0000 7f8f1ae00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f8f140cfc20 con 0x7f8f14161c90 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.082+0000 7f8f1ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f14163110 con 0x7f8f14162b10 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.082+0000 7f8f13e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f14161c90 0x7f8f140d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.082+0000 7f8f13e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f14161c90 0x7f8f140d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37754/0 (socket says 172.21.15.67:37754) 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.082+0000 7f8f13e00640 1 -- 172.21.15.67:0/3789405420 learned_addr learned my addr 172.21.15.67:0/3789405420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.082+0000 7f8f13400640 1 --1- 172.21.15.67:0/3789405420 >> v1:172.21.15.67:6789/0 conn(0x7f8f140d2c50 0x7f8f14162400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39344/0 (socket says 172.21.15.67:39344) 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.083+0000 7f8f18e00640 1 --2- 172.21.15.67:0/3789405420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f14162b10 0x7f8f140d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.083+0000 7f8f13e00640 1 -- 172.21.15.67:0/3789405420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f14162b10 msgr2=0x7f8f140d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.083+0000 7f8f13e00640 1 --2- 172.21.15.67:0/3789405420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f14162b10 0x7f8f140d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.083+0000 7f8f13e00640 1 -- 172.21.15.67:0/3789405420 >> v1:172.21.15.67:6789/0 conn(0x7f8f140d2c50 legacy=0x7f8f14162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.083+0000 7f8f13e00640 1 -- 172.21.15.67:0/3789405420 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f140cf5d0 con 0x7f8f14161c90 2024-05-30T16:43:32.085 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.083+0000 7f8f12a00640 1 -- 172.21.15.67:0/3789405420 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140574454 0 0) 0x7f8f140cf310 con 0x7f8f140d2c50 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.084+0000 7f8f13e00640 1 --2- 172.21.15.67:0/3789405420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14161c90 0x7f8f140d00b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009870 tx=0x7f8f0402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5754e9a9321ea7ad server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.084+0000 7f8f12a00640 1 -- 172.21.15.67:0/3789405420 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04037070 con 0x7f8f14161c90 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.084+0000 7f8f12a00640 1 -- 172.21.15.67:0/3789405420 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f04032440 con 0x7f8f14161c90 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.084+0000 7f8f12a00640 1 -- 172.21.15.67:0/3789405420 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f04032810 con 0x7f8f14161c90 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.085+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3789405420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14161c90 msgr2=0x7f8f140d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.085+0000 7f8f1ae00640 1 --2- 172.21.15.67:0/3789405420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14161c90 0x7f8f140d00b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009870 tx=0x7f8f0402a1c0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.085+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3789405420 shutdown_connections 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.085+0000 7f8f1ae00640 1 --2- 172.21.15.67:0/3789405420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f14162b10 0x7f8f140d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.085+0000 7f8f1ae00640 1 --2- 172.21.15.67:0/3789405420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14161c90 0x7f8f140d00b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.085+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3789405420 >> 172.21.15.67:0/3789405420 conn(0x7f8f1408bf60 msgr2=0x7f8f140d4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.085+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3789405420 shutdown_connections 2024-05-30T16:43:32.087 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.085+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3789405420 wait complete. 2024-05-30T16:43:32.088 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.086+0000 7f8f1ae00640 1 Processor -- start 2024-05-30T16:43:32.088 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.087+0000 7f8f1ae00640 1 -- start start 2024-05-30T16:43:32.090 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.087+0000 7f8f1ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f140d2c50 0x7f8f140ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:32.090 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.087+0000 7f8f1ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14161c90 0x7f8f140b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.087+0000 7f8f1ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14162b10 0x7f8f14155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.087+0000 7f8f1ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8f14162070 con 0x7f8f14162b10 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.087+0000 7f8f1ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140d05f0 con 0x7f8f14161c90 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.087+0000 7f8f1ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f141621f0 con 0x7f8f140d2c50 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f18e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14162b10 0x7f8f14155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f18e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14162b10 0x7f8f14155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37768/0 (socket says 172.21.15.67:37768) 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f18e00640 1 -- 172.21.15.67:0/3553857748 learned_addr learned my addr 172.21.15.67:0/3553857748 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f13e00640 1 --2- 172.21.15.67:0/3553857748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f140d2c50 0x7f8f140ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f13400640 1 --2- 172.21.15.67:0/3553857748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14161c90 0x7f8f140b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f18e00640 1 -- 172.21.15.67:0/3553857748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f140d2c50 msgr2=0x7f8f140ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f18e00640 1 --2- 172.21.15.67:0/3553857748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f140d2c50 0x7f8f140ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.091 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f18e00640 1 -- 172.21.15.67:0/3553857748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14161c90 msgr2=0x7f8f140b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:32.092 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f18e00640 1 --2- 172.21.15.67:0/3553857748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14161c90 0x7f8f140b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.092 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f18e00640 1 -- 172.21.15.67:0/3553857748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f140b8c60 con 0x7f8f14162b10 2024-05-30T16:43:32.092 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.088+0000 7f8f18e00640 1 --2- 172.21.15.67:0/3553857748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14162b10 0x7f8f14155b90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c007e60 tx=0x7f8f0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:32.092 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.089+0000 7f8f0be00640 1 -- 172.21.15.67:0/3553857748 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c014420 con 0x7f8f14162b10 2024-05-30T16:43:32.092 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.089+0000 7f8f0be00640 1 -- 172.21.15.67:0/3553857748 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c019070 con 0x7f8f14162b10 2024-05-30T16:43:32.092 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.089+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3553857748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f140cd340 con 0x7f8f14162b10 2024-05-30T16:43:32.092 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.089+0000 7f8f0be00640 1 -- 172.21.15.67:0/3553857748 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c014820 con 0x7f8f14162b10 2024-05-30T16:43:32.092 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.089+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3553857748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f140cd900 con 0x7f8f14162b10 2024-05-30T16:43:32.093 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.090+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3553857748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1404fce0 con 0x7f8f14162b10 2024-05-30T16:43:32.093 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.090+0000 7f8f0be00640 1 -- 172.21.15.67:0/3553857748 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 2) ==== 45545+0+0 (secure 0 0 0) 0x7f8f0c0149c0 con 0x7f8f14162b10 2024-05-30T16:43:32.093 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.091+0000 7f8f0be00640 1 -- 172.21.15.67:0/3553857748 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f8f0c04b2b0 con 0x7f8f14162b10 2024-05-30T16:43:32.094 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.093+0000 7f8f0be00640 1 -- 172.21.15.67:0/3553857748 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8f0c0139d0 con 0x7f8f14162b10 2024-05-30T16:43:32.161 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.160+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3553857748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f140d1640 con 0x7f8f14162b10 2024-05-30T16:43:32.162 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.161+0000 7f8f0be00640 1 -- 172.21.15.67:0/3553857748 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f8f0c013c30 con 0x7f8f14162b10 2024-05-30T16:43:32.162 INFO:teuthology.misc.health.smithi067.stdout: 2024-05-30T16:43:32.162 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":10,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:30.736505+0000","last_up_change":"0.000000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:43:32.164 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.162+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3553857748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14162b10 msgr2=0x7f8f14155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:32.164 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.162+0000 7f8f1ae00640 1 --2- 172.21.15.67:0/3553857748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14162b10 0x7f8f14155b90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c007e60 tx=0x7f8f0c00e4b0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.164 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.162+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3553857748 shutdown_connections 2024-05-30T16:43:32.164 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.162+0000 7f8f1ae00640 1 --2- 172.21.15.67:0/3553857748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f14162b10 0x7f8f14155b90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.164 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.162+0000 7f8f1ae00640 1 --2- 172.21.15.67:0/3553857748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f14161c90 0x7f8f140b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.164 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.162+0000 7f8f1ae00640 1 --2- 172.21.15.67:0/3553857748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f140d2c50 0x7f8f140ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:32.164 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.162+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3553857748 >> 172.21.15.67:0/3553857748 conn(0x7f8f1408bf60 msgr2=0x7f8f140d4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:32.164 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.162+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3553857748 shutdown_connections 2024-05-30T16:43:32.164 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:32.162+0000 7f8f1ae00640 1 -- 172.21.15.67:0/3553857748 wait complete. 2024-05-30T16:43:32.177 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-30T16:43:32.353 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:32.352+0000 7fc0fe093040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-30T16:43:32.485 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-05-30T16:43:32.485+0000 7fc0fe093040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-30T16:43:32.990 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:32.989+0000 7f0cb4600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-30T16:43:32.992 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:43:32.990+0000 7f0cb4600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-30T16:43:34.431 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-05-30T16:43:34.430+0000 7f4609e41640 -1 osd.5 0 log_to_monitors true 2024-05-30T16:43:34.606 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-05-30T16:43:34.605+0000 7f64e35a7640 -1 osd.7 0 log_to_monitors true 2024-05-30T16:43:34.940 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-05-30T16:43:34.939+0000 7f3d6281c640 -1 osd.0 0 log_to_monitors true 2024-05-30T16:43:35.069 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-05-30T16:43:35.068+0000 7f9819757640 -1 osd.2 0 log_to_monitors true 2024-05-30T16:43:35.106 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-05-30T16:43:35.105+0000 7f7a9d80d640 -1 osd.6 0 log_to_monitors true 2024-05-30T16:43:35.115 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-05-30T16:43:35.114+0000 7fbaa3a2d640 -1 osd.3 0 log_to_monitors true 2024-05-30T16:43:35.132 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-05-30T16:43:35.132+0000 7f488cc0f640 -1 osd.1 0 log_to_monitors true 2024-05-30T16:43:35.144 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-05-30T16:43:35.144+0000 7fcded25d640 -1 osd.4 0 log_to_monitors true 2024-05-30T16:43:37.403 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-05-30T16:43:37.402+0000 7f3d5d000640 -1 osd.0 0 waiting for initial osdmap 2024-05-30T16:43:37.429 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-05-30T16:43:37.429+0000 7f3d53000640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T16:43:37.639 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-05-30T16:43:37.639+0000 7f4604c00640 -1 osd.5 0 waiting for initial osdmap 2024-05-30T16:43:37.653 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-05-30T16:43:37.653+0000 7f45fac00640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T16:43:37.680 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-05-30T16:43:37.680+0000 7f64de400640 -1 osd.7 0 waiting for initial osdmap 2024-05-30T16:43:37.711 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-05-30T16:43:37.711+0000 7f64d4400640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T16:43:38.142 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:43:38.148 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:43:38.509 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-05-30T16:43:38.509+0000 7f7a98a00640 -1 osd.6 0 waiting for initial osdmap 2024-05-30T16:43:38.515 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-05-30T16:43:38.515+0000 7f7a8e000640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T16:43:38.550 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-05-30T16:43:38.550+0000 7fcde8c00640 -1 osd.4 0 waiting for initial osdmap 2024-05-30T16:43:38.569 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-05-30T16:43:38.569+0000 7fcdde200640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T16:43:38.940 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-05-30T16:43:38.940+0000 7f4888000640 -1 osd.1 0 waiting for initial osdmap 2024-05-30T16:43:38.942 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-05-30T16:43:38.941+0000 7f9814600640 -1 osd.2 0 waiting for initial osdmap 2024-05-30T16:43:38.948 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-05-30T16:43:38.947+0000 7f487d600640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T16:43:38.953 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-05-30T16:43:38.952+0000 7f980a600640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T16:43:38.980 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-05-30T16:43:38.980+0000 7fba9e000640 -1 osd.3 0 waiting for initial osdmap 2024-05-30T16:43:38.983 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-30T16:43:38.988 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-05-30T16:43:38.984+0000 7fba94000640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-30T16:43:39.133 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.132+0000 7f9b7e800640 1 Processor -- start 2024-05-30T16:43:39.133 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.132+0000 7f9b7e800640 1 -- start start 2024-05-30T16:43:39.135 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.134+0000 7f9b7e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b780d2a60 0x7f9b780d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:39.135 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.134+0000 7f9b7e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b7815ee40 con 0x7f9b780cefb0 2024-05-30T16:43:39.135 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.134+0000 7f9b7e800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b780d3380 con 0x7f9b780d4150 2024-05-30T16:43:39.135 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.134+0000 7f9b7e800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9b7815f0e0 con 0x7f9b780d2a60 2024-05-30T16:43:39.136 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.135+0000 7f9b7de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9b780cefb0 0x7f9b780cf3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39350/0 (socket says 172.21.15.67:39350) 2024-05-30T16:43:39.136 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.135+0000 7f9b7de00640 1 -- 172.21.15.67:0/1436011804 learned_addr learned my addr 172.21.15.67:0/1436011804 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:39.136 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.135+0000 7f9b7d400640 1 --2- 172.21.15.67:0/1436011804 >> v2:172.21.15.67:3300/0 conn(0x7f9b780d2a60 0x7f9b780d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:39.136 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.135+0000 7f9b77e00640 1 -- 172.21.15.67:0/1436011804 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 129373076 0 0) 0x7f9b7815ee40 con 0x7f9b780cefb0 2024-05-30T16:43:39.136 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.135+0000 7f9b7d400640 1 -- 172.21.15.67:0/1436011804 >> v1:172.21.15.169:6789/0 conn(0x7f9b780d4150 legacy=0x7f9b780d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:39.136 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.135+0000 7f9b7d400640 1 -- 172.21.15.67:0/1436011804 >> v1:172.21.15.67:6789/0 conn(0x7f9b780cefb0 legacy=0x7f9b780cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:39.136 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.135+0000 7f9b7d400640 1 -- 172.21.15.67:0/1436011804 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7815f3c0 con 0x7f9b780d2a60 2024-05-30T16:43:39.137 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.136+0000 7f9b7d400640 1 --2- 172.21.15.67:0/1436011804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780d2a60 0x7f9b780d2e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b680097a0 tx=0x7f9b6802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6147fc35f516518 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:39.137 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b77e00640 1 -- 172.21.15.67:0/1436011804 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68037070 con 0x7f9b780d2a60 2024-05-30T16:43:39.137 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b77e00640 1 -- 172.21.15.67:0/1436011804 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b68032440 con 0x7f9b780d2a60 2024-05-30T16:43:39.137 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b77e00640 1 -- 172.21.15.67:0/1436011804 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68032810 con 0x7f9b780d2a60 2024-05-30T16:43:39.138 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b7e800640 1 -- 172.21.15.67:0/1436011804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780d2a60 msgr2=0x7f9b780d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:39.138 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b7e800640 1 --2- 172.21.15.67:0/1436011804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780d2a60 0x7f9b780d2e40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b680097a0 tx=0x7f9b6802a230 comp rx=0 tx=0).stop 2024-05-30T16:43:39.138 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b7e800640 1 -- 172.21.15.67:0/1436011804 shutdown_connections 2024-05-30T16:43:39.138 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b7e800640 1 --2- 172.21.15.67:0/1436011804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780d2a60 0x7f9b780d2e40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:39.138 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b7e800640 1 -- 172.21.15.67:0/1436011804 >> 172.21.15.67:0/1436011804 conn(0x7f9b7808bf60 msgr2=0x7f9b780d4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:39.139 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b7e800640 1 -- 172.21.15.67:0/1436011804 shutdown_connections 2024-05-30T16:43:39.139 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.137+0000 7f9b7e800640 1 -- 172.21.15.67:0/1436011804 wait complete. 2024-05-30T16:43:39.139 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.139+0000 7f9b7e800640 1 Processor -- start 2024-05-30T16:43:39.140 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.139+0000 7f9b7e800640 1 -- start start 2024-05-30T16:43:39.141 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.140+0000 7f9b7e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780cefb0 0x7f9b780cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:39.141 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.140+0000 7f9b7e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b780d2a60 0x7f9b781557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:39.141 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.140+0000 7f9b7e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b780d4150 0x7f9b780b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:39.141 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.140+0000 7f9b7e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7815e9e0 con 0x7f9b780cefb0 2024-05-30T16:43:39.141 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.140+0000 7f9b7e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9b780d2780 con 0x7f9b780d2a60 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.140+0000 7f9b7e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7815eb60 con 0x7f9b780d4150 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.140+0000 7f9b7d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780cefb0 0x7f9b780cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.140+0000 7f9b7d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780cefb0 0x7f9b780cde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37816/0 (socket says 172.21.15.67:37816) 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.140+0000 7f9b7d400640 1 -- 172.21.15.67:0/1176936251 learned_addr learned my addr 172.21.15.67:0/1176936251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b7d400640 1 -- 172.21.15.67:0/1176936251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b780d4150 msgr2=0x7f9b780b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b7d400640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b780d4150 0x7f9b780b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b7d400640 1 -- 172.21.15.67:0/1176936251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b780d2a60 msgr2=0x7f9b781557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b7d400640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b780d2a60 0x7f9b781557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b7d400640 1 -- 172.21.15.67:0/1176936251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b780b5f60 con 0x7f9b780cefb0 2024-05-30T16:43:39.142 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b7d400640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780cefb0 0x7f9b780cde90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b680098c0 tx=0x7f9b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:39.143 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b75600640 1 -- 172.21.15.67:0/1176936251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68037040 con 0x7f9b780cefb0 2024-05-30T16:43:39.143 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b75600640 1 -- 172.21.15.67:0/1176936251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b68042070 con 0x7f9b780cefb0 2024-05-30T16:43:39.143 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b75600640 1 -- 172.21.15.67:0/1176936251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68030a70 con 0x7f9b780cefb0 2024-05-30T16:43:39.143 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b780b6200 con 0x7f9b780cefb0 2024-05-30T16:43:39.143 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.141+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b780b67c0 con 0x7f9b780cefb0 2024-05-30T16:43:39.144 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.142+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b780a2440 con 0x7f9b780cefb0 2024-05-30T16:43:39.144 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.143+0000 7f9b75600640 1 -- 172.21.15.67:0/1176936251 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f9b68030c10 con 0x7f9b780cefb0 2024-05-30T16:43:39.145 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.144+0000 7f9b75600640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b5806c860 0x7f9b5806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:39.145 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.144+0000 7f9b75600640 1 -- 172.21.15.67:0/1176936251 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3071+0+0 (secure 0 0 0) 0x7f9b680abab0 con 0x7f9b780cefb0 2024-05-30T16:43:39.145 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.144+0000 7f9b7ca00640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b5806c860 0x7f9b5806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:39.145 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.145+0000 7f9b7ca00640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b5806c860 0x7f9b5806ed20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b64007fd0 tx=0x7f9b640079a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:39.150 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.150+0000 7f9b75600640 1 -- 172.21.15.67:0/1176936251 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b6807aa00 con 0x7f9b780cefb0 2024-05-30T16:43:39.347 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.346+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b78075e70 con 0x7f9b780cefb0 2024-05-30T16:43:39.348 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.348+0000 7f9b75600640 1 -- 172.21.15.67:0/1176936251 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8515 (secure 0 0 0) 0x7f9b6807a300 con 0x7f9b780cefb0 2024-05-30T16:43:39.349 INFO:teuthology.misc.health.smithi067.stdout: 2024-05-30T16:43:39.349 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":14,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:38.425436+0000","last_up_change":"2024-05-30T16:43:38.425436+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:43:39.352 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.350+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b5806c860 msgr2=0x7f9b5806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:39.352 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.350+0000 7f9b7e800640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b5806c860 0x7f9b5806ed20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b64007fd0 tx=0x7f9b640079a0 comp rx=0 tx=0).stop 2024-05-30T16:43:39.352 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780cefb0 msgr2=0x7f9b780cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:39.352 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780cefb0 0x7f9b780cde90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b680098c0 tx=0x7f9b680046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:39.352 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 shutdown_connections 2024-05-30T16:43:39.352 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b5806c860 0x7f9b5806ed20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:39.352 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b780d4150 0x7f9b780b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:39.352 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b780d2a60 0x7f9b781557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:39.352 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 --2- 172.21.15.67:0/1176936251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b780cefb0 0x7f9b780cde90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:39.353 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 >> 172.21.15.67:0/1176936251 conn(0x7f9b7808bf60 msgr2=0x7f9b78159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:39.353 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 shutdown_connections 2024-05-30T16:43:39.353 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:39.351+0000 7f9b7e800640 1 -- 172.21.15.67:0/1176936251 wait complete. 2024-05-30T16:43:39.365 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-05-30T16:43:46.171 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-30T16:43:46.364 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.363+0000 7f6f29e00640 1 Processor -- start 2024-05-30T16:43:46.364 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.364+0000 7f6f29e00640 1 -- start start 2024-05-30T16:43:46.366 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.365+0000 7f6f29e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f240cf7d0 0x7f6f240b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.366 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.365+0000 7f6f29e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f240d1df0 0x7f6f240b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.366 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.365+0000 7f6f29e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f240cfc60 con 0x7f6f2415e880 2024-05-30T16:43:46.366 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.365+0000 7f6f29e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f6f240b8c70 con 0x7f6f240cf7d0 2024-05-30T16:43:46.366 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.365+0000 7f6f29e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6f240b83d0 con 0x7f6f240d1df0 2024-05-30T16:43:46.367 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.365+0000 7f6f22000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f240cf7d0 0x7f6f240b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.367 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.366+0000 7f6f22000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f240cf7d0 0x7f6f240b9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52098/0 (socket says 172.21.15.67:52098) 2024-05-30T16:43:46.367 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.366+0000 7f6f22000640 1 -- 172.21.15.67:0/3403061158 learned_addr learned my addr 172.21.15.67:0/3403061158 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:46.367 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.366+0000 7f6f23400640 1 --2- 172.21.15.67:0/3403061158 >> v2:172.21.15.169:3300/0 conn(0x7f6f240d1df0 0x7f6f240b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.367 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.366+0000 7f6f22000640 1 -- 172.21.15.67:0/3403061158 >> v2:172.21.15.169:3300/0 conn(0x7f6f240d1df0 msgr2=0x7f6f240b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:46.367 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.366+0000 7f6f22000640 1 --2- 172.21.15.67:0/3403061158 >> v2:172.21.15.169:3300/0 conn(0x7f6f240d1df0 0x7f6f240b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.368 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.366+0000 7f6f22000640 1 -- 172.21.15.67:0/3403061158 >> v1:172.21.15.169:6789/0 conn(0x7f6f2415e880 legacy=0x7f6f240d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:46.368 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.366+0000 7f6f22000640 1 -- 172.21.15.67:0/3403061158 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f240b8620 con 0x7f6f240cf7d0 2024-05-30T16:43:46.370 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.367+0000 7f6f22000640 1 --2- 172.21.15.67:0/3403061158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240cf7d0 0x7f6f240b9100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f24163490 tx=0x7f6f0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c608a78fad7ff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:46.370 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.367+0000 7f6f21600640 1 -- 172.21.15.67:0/3403061158 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c037070 con 0x7f6f240cf7d0 2024-05-30T16:43:46.370 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.368+0000 7f6f21600640 1 -- 172.21.15.67:0/3403061158 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0c032440 con 0x7f6f240cf7d0 2024-05-30T16:43:46.370 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.368+0000 7f6f21600640 1 -- 172.21.15.67:0/3403061158 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c032810 con 0x7f6f240cf7d0 2024-05-30T16:43:46.370 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.368+0000 7f6f29e00640 1 -- 172.21.15.67:0/3403061158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240cf7d0 msgr2=0x7f6f240b9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.370 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.368+0000 7f6f29e00640 1 --2- 172.21.15.67:0/3403061158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240cf7d0 0x7f6f240b9100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f24163490 tx=0x7f6f0c02a1a0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.370 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.368+0000 7f6f29e00640 1 -- 172.21.15.67:0/3403061158 shutdown_connections 2024-05-30T16:43:46.370 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.368+0000 7f6f29e00640 1 --2- 172.21.15.67:0/3403061158 >> v2:172.21.15.169:3300/0 conn(0x7f6f240d1df0 0x7f6f240b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.370 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.368+0000 7f6f29e00640 1 --2- 172.21.15.67:0/3403061158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240cf7d0 0x7f6f240b9100 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.371 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.368+0000 7f6f29e00640 1 -- 172.21.15.67:0/3403061158 >> 172.21.15.67:0/3403061158 conn(0x7f6f24073460 msgr2=0x7f6f24073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:46.371 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.368+0000 7f6f29e00640 1 -- 172.21.15.67:0/3403061158 shutdown_connections 2024-05-30T16:43:46.371 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.369+0000 7f6f29e00640 1 -- 172.21.15.67:0/3403061158 wait complete. 2024-05-30T16:43:46.371 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.371+0000 7f6f29e00640 1 Processor -- start 2024-05-30T16:43:46.371 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.371+0000 7f6f29e00640 1 -- start start 2024-05-30T16:43:46.374 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.372+0000 7f6f29e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240d1df0 0x7f6f241f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.374 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.372+0000 7f6f29e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2415e880 0x7f6f24161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.374 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.372+0000 7f6f29e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f241616c0 0x7f6f2415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.374 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.372+0000 7f6f29e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6f240b39e0 con 0x7f6f240d1df0 2024-05-30T16:43:46.374 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.372+0000 7f6f29e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2415ec60 con 0x7f6f241616c0 2024-05-30T16:43:46.374 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.372+0000 7f6f29e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f240b3b60 con 0x7f6f2415e880 2024-05-30T16:43:46.374 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240d1df0 0x7f6f241f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.374 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240d1df0 0x7f6f241f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52102/0 (socket says 172.21.15.67:52102) 2024-05-30T16:43:46.374 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22a00640 1 -- 172.21.15.67:0/919840987 learned_addr learned my addr 172.21.15.67:0/919840987 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:46.375 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f23400640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f241616c0 0x7f6f2415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.375 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22a00640 1 -- 172.21.15.67:0/919840987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2415e880 msgr2=0x7f6f24161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.375 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22000640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2415e880 0x7f6f24161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.375 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22a00640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2415e880 0x7f6f24161180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.375 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22a00640 1 -- 172.21.15.67:0/919840987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f241616c0 msgr2=0x7f6f2415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.375 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22a00640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f241616c0 0x7f6f2415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.375 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22a00640 1 -- 172.21.15.67:0/919840987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2415fbc0 con 0x7f6f240d1df0 2024-05-30T16:43:46.375 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f23400640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f241616c0 0x7f6f2415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:46.375 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.373+0000 7f6f22a00640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240d1df0 0x7f6f241f7570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400ba50 tx=0x7f6f1400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:46.376 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.374+0000 7f6f1b400640 1 -- 172.21.15.67:0/919840987 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14019070 con 0x7f6f240d1df0 2024-05-30T16:43:46.376 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.374+0000 7f6f1b400640 1 -- 172.21.15.67:0/919840987 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f1400cc20 con 0x7f6f240d1df0 2024-05-30T16:43:46.376 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.374+0000 7f6f1b400640 1 -- 172.21.15.67:0/919840987 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14014440 con 0x7f6f240d1df0 2024-05-30T16:43:46.376 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.374+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2415fea0 con 0x7f6f240d1df0 2024-05-30T16:43:46.376 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.374+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f24160460 con 0x7f6f240d1df0 2024-05-30T16:43:46.376 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.375+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f24161ad0 con 0x7f6f240d1df0 2024-05-30T16:43:46.377 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.376+0000 7f6f1b400640 1 -- 172.21.15.67:0/919840987 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f1401c070 con 0x7f6f240d1df0 2024-05-30T16:43:46.378 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.377+0000 7f6f1b400640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f0006c8f0 0x7f6f0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.378 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.377+0000 7f6f1b400640 1 -- 172.21.15.67:0/919840987 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f6f1408b510 con 0x7f6f240d1df0 2024-05-30T16:43:46.378 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.378+0000 7f6f22000640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f0006c8f0 0x7f6f0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.379 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.378+0000 7f6f22000640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f0006c8f0 0x7f6f0006edb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c034000 tx=0x7f6f0c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:46.384 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.383+0000 7f6f1b400640 1 -- 172.21.15.67:0/919840987 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f24161ad0 con 0x7f6f240d1df0 2024-05-30T16:43:46.590 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.589+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f2404fce0 con 0x7f6f240d1df0 2024-05-30T16:43:46.592 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.591+0000 7f6f1b400640 1 -- 172.21.15.67:0/919840987 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12872 (secure 0 0 0) 0x7f6f140595a0 con 0x7f6f240d1df0 2024-05-30T16:43:46.594 INFO:teuthology.misc.health.smithi067.stdout: 2024-05-30T16:43:46.594 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":18,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:42.460064+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:43:46.595 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f0006c8f0 msgr2=0x7f6f0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.595 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f0006c8f0 0x7f6f0006edb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c034000 tx=0x7f6f0c02e000 comp rx=0 tx=0).stop 2024-05-30T16:43:46.595 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240d1df0 msgr2=0x7f6f241f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.595 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240d1df0 0x7f6f241f7570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400ba50 tx=0x7f6f1400bf10 comp rx=0 tx=0).stop 2024-05-30T16:43:46.595 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 shutdown_connections 2024-05-30T16:43:46.595 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f0006c8f0 0x7f6f0006edb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.595 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f241616c0 0x7f6f2415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.595 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f2415e880 0x7f6f24161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.596 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 --2- 172.21.15.67:0/919840987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f240d1df0 0x7f6f241f7570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.596 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 >> 172.21.15.67:0/919840987 conn(0x7f6f24073460 msgr2=0x7f6f240d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:46.597 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 shutdown_connections 2024-05-30T16:43:46.597 INFO:teuthology.misc.health.smithi067.stderr:2024-05-30T16:43:46.594+0000 7f6f29e00640 1 -- 172.21.15.67:0/919840987 wait complete. 2024-05-30T16:43:46.609 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-30T16:43:46.609 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-30T16:43:46.609 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-30T16:43:46.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-30T16:43:46.617 INFO:tasks.ceph.mds.a:Started 2024-05-30T16:43:46.617 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-30T16:43:46.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-30T16:43:46.624 INFO:tasks.ceph.mds.c:Started 2024-05-30T16:43:46.624 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-30T16:43:46.625 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-30T16:43:46.632 INFO:tasks.ceph.mds.b:Started 2024-05-30T16:43:46.632 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-30T16:43:46.632 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-30T16:43:46.639 INFO:tasks.ceph.mds.d:Started 2024-05-30T16:43:46.639 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-30T16:43:46.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:43:46.763 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T16:43:46.766 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T16:43:46.786 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T16:43:46.786 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T16:43:46.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.811+0000 7f1714a00640 1 Processor -- start 2024-05-30T16:43:46.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.811+0000 7f1714a00640 1 -- start start 2024-05-30T16:43:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.812+0000 7f1714a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f17100d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.812+0000 7f1714a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17100d36d0 con 0x7f171015ed20 2024-05-30T16:43:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.812+0000 7f1714a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1710163500 con 0x7f17100d2db0 2024-05-30T16:43:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.812+0000 7f1714a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f17100d5540 con 0x7f17100b3b50 2024-05-30T16:43:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.812+0000 7f170e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f171015ed20 0x7f171015f1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37890/0 (socket says 172.21.15.67:37890) 2024-05-30T16:43:46.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.812+0000 7f170e800640 1 -- 172.21.15.67:0/1203259724 learned_addr learned my addr 172.21.15.67:0/1203259724 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:46.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.812+0000 7f170d400640 1 --2- 172.21.15.67:0/1203259724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f17100d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.813+0000 7f170ca00640 1 -- 172.21.15.67:0/1203259724 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542852044 0 0) 0x7f17100d36d0 con 0x7f171015ed20 2024-05-30T16:43:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.813+0000 7f170ca00640 1 -- 172.21.15.67:0/1203259724 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16f4003520 con 0x7f171015ed20 2024-05-30T16:43:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.813+0000 7f170ca00640 1 -- 172.21.15.67:0/1203259724 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2626437455 0 0) 0x7f1710163500 con 0x7f17100d2db0 2024-05-30T16:43:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.813+0000 7f170ca00640 1 -- 172.21.15.67:0/1203259724 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17100d36d0 con 0x7f17100d2db0 2024-05-30T16:43:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.813+0000 7f170d400640 1 -- 172.21.15.67:0/1203259724 >> v1:172.21.15.169:6789/0 conn(0x7f17100d2db0 legacy=0x7f17100d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.813+0000 7f170d400640 1 -- 172.21.15.67:0/1203259724 >> v1:172.21.15.67:6789/0 conn(0x7f171015ed20 legacy=0x7f171015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.813+0000 7f170d400640 1 -- 172.21.15.67:0/1203259724 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1710162740 con 0x7f17100b3b50 2024-05-30T16:43:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.814+0000 7f170d400640 1 --2- 172.21.15.67:0/1203259724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f17100d3190 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0097d0 tx=0x7f16fc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4020c3934d8e09de server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:46.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.814+0000 7f170ca00640 1 -- 172.21.15.67:0/1203259724 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc037070 con 0x7f17100b3b50 2024-05-30T16:43:46.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.814+0000 7f170ca00640 1 -- 172.21.15.67:0/1203259724 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc032440 con 0x7f17100b3b50 2024-05-30T16:43:46.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.814+0000 7f170ca00640 1 -- 172.21.15.67:0/1203259724 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc032810 con 0x7f17100b3b50 2024-05-30T16:43:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.815+0000 7f1714a00640 1 -- 172.21.15.67:0/1203259724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 msgr2=0x7f17100d3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.815+0000 7f1714a00640 1 --2- 172.21.15.67:0/1203259724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f17100d3190 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0097d0 tx=0x7f16fc02a180 comp rx=0 tx=0).stop 2024-05-30T16:43:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.815+0000 7f1714a00640 1 -- 172.21.15.67:0/1203259724 shutdown_connections 2024-05-30T16:43:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.815+0000 7f1714a00640 1 --2- 172.21.15.67:0/1203259724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f17100d3190 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.815+0000 7f1714a00640 1 -- 172.21.15.67:0/1203259724 >> 172.21.15.67:0/1203259724 conn(0x7f171008bfc0 msgr2=0x7f17100d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.815+0000 7f1714a00640 1 -- 172.21.15.67:0/1203259724 shutdown_connections 2024-05-30T16:43:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.815+0000 7f1714a00640 1 -- 172.21.15.67:0/1203259724 wait complete. 2024-05-30T16:43:46.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.817+0000 7f1714a00640 1 Processor -- start 2024-05-30T16:43:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.817+0000 7f1714a00640 1 -- start start 2024-05-30T16:43:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f1714a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f1710157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f1714a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17100d2db0 0x7f17100cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f1714a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f171015ed20 0x7f171006dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f1714a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1710162fe0 con 0x7f17100d2db0 2024-05-30T16:43:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f1714a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1710162d00 con 0x7f171015ed20 2024-05-30T16:43:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f1714a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f17100d04a0 con 0x7f17100b3b50 2024-05-30T16:43:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17100d2db0 0x7f17100cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17100d2db0 0x7f17100cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52150/0 (socket says 172.21.15.67:52150) 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170d400640 1 -- 172.21.15.67:0/221721080 learned_addr learned my addr 172.21.15.67:0/221721080 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170e800640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f171015ed20 0x7f171006dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170de00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f1710157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170d400640 1 -- 172.21.15.67:0/221721080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 msgr2=0x7f1710157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170d400640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f1710157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170d400640 1 -- 172.21.15.67:0/221721080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f171015ed20 msgr2=0x7f171006dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170d400640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f171015ed20 0x7f171006dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170d400640 1 -- 172.21.15.67:0/221721080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17100cbc70 con 0x7f17100d2db0 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170de00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f1710157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.818+0000 7f170d400640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17100d2db0 0x7f17100cdc30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f16fc035040 tx=0x7f16fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.819+0000 7f1706000640 1 -- 172.21.15.67:0/221721080 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc041020 con 0x7f17100d2db0 2024-05-30T16:43:46.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.819+0000 7f1706000640 1 -- 172.21.15.67:0/221721080 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc03d720 con 0x7f17100d2db0 2024-05-30T16:43:46.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.819+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17100cbf50 con 0x7f17100d2db0 2024-05-30T16:43:46.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.819+0000 7f1706000640 1 -- 172.21.15.67:0/221721080 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc037040 con 0x7f17100d2db0 2024-05-30T16:43:46.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.819+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17100cc510 con 0x7f17100d2db0 2024-05-30T16:43:46.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.820+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17100689e0 con 0x7f17100d2db0 2024-05-30T16:43:46.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.821+0000 7f1706000640 1 -- 172.21.15.67:0/221721080 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16fc044020 con 0x7f17100d2db0 2024-05-30T16:43:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.822+0000 7f1706000640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16f006c8f0 0x7f16f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.822+0000 7f1706000640 1 -- 172.21.15.67:0/221721080 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f16fc0abf10 con 0x7f17100d2db0 2024-05-30T16:43:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.822+0000 7f170de00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16f006c8f0 0x7f16f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.823+0000 7f170de00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16f006c8f0 0x7f16f006edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17000011e0 tx=0x7f1700009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:46.826+0000 7f1706000640 1 -- 172.21.15.67:0/221721080 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16fc04b030 con 0x7f17100d2db0 2024-05-30T16:43:47.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.073+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17100d2af0 con 0x7f17100d2db0 2024-05-30T16:43:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.074+0000 7f1706000640 1 -- 172.21.15.67:0/221721080 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f16fc07a600 con 0x7f17100d2db0 2024-05-30T16:43:47.075 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:47.075 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-30T16:43:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 2 2024-05-30T16:43:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16f006c8f0 msgr2=0x7f16f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16f006c8f0 0x7f16f006edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17000011e0 tx=0x7f1700009040 comp rx=0 tx=0).stop 2024-05-30T16:43:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17100d2db0 msgr2=0x7f17100cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17100d2db0 0x7f17100cdc30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f16fc035040 tx=0x7f16fc0046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 shutdown_connections 2024-05-30T16:43:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16f006c8f0 0x7f16f006edb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f171015ed20 0x7f171006dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17100d2db0 0x7f17100cdc30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 --2- 172.21.15.67:0/221721080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17100b3b50 0x7f1710157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 >> 172.21.15.67:0/221721080 conn(0x7f171008bfc0 msgr2=0x7f171015c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 shutdown_connections 2024-05-30T16:43:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:47.077+0000 7f1714a00640 1 -- 172.21.15.67:0/221721080 wait complete. 2024-05-30T16:43:49.097 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:43:49.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.321+0000 7f6f09c00640 1 Processor -- start 2024-05-30T16:43:49.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.321+0000 7f6f09c00640 1 -- start start 2024-05-30T16:43:49.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.322+0000 7f6f09c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f0415e870 0x7f6f040d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.322+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f040d14d0 0x7f6f040cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.322+0000 7f6f09c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f040cfd00 con 0x7f6f040cf870 2024-05-30T16:43:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.322+0000 7f6f09c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f6f040d2150 con 0x7f6f0415e870 2024-05-30T16:43:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.322+0000 7f6f09c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0415ec50 con 0x7f6f040d14d0 2024-05-30T16:43:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.323+0000 7f6f03e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f0415e870 0x7f6f040d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.323+0000 7f6f03e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f0415e870 0x7f6f040d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52164/0 (socket says 172.21.15.67:52164) 2024-05-30T16:43:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.323+0000 7f6f03e00640 1 -- 172.21.15.67:0/3999069695 learned_addr learned my addr 172.21.15.67:0/3999069695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.323+0000 7f6f09200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6f040cf870 0x7f6f04162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37906/0 (socket says 172.21.15.67:37906) 2024-05-30T16:43:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.323+0000 7f6f03e00640 1 -- 172.21.15.67:0/3999069695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f040d14d0 msgr2=0x7f6f040cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.323+0000 7f6f03e00640 1 --2- 172.21.15.67:0/3999069695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f040d14d0 0x7f6f040cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.323+0000 7f6f03e00640 1 -- 172.21.15.67:0/3999069695 >> v1:172.21.15.67:6789/0 conn(0x7f6f040cf870 legacy=0x7f6f04162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.323+0000 7f6f03e00640 1 -- 172.21.15.67:0/3999069695 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f040cff80 con 0x7f6f0415e870 2024-05-30T16:43:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.323+0000 7f6f03400640 1 -- 172.21.15.67:0/3999069695 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318406932 0 0) 0x7f6efc001490 con 0x7f6f040cf870 2024-05-30T16:43:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.324+0000 7f6f03e00640 1 --2- 172.21.15.67:0/3999069695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 0x7f6f040d0f90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009870 tx=0x7f6ef402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841b1a800e677c0d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.325+0000 7f6f03400640 1 -- 172.21.15.67:0/3999069695 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4037070 con 0x7f6f0415e870 2024-05-30T16:43:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.325+0000 7f6f03400640 1 -- 172.21.15.67:0/3999069695 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef4032440 con 0x7f6f0415e870 2024-05-30T16:43:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.325+0000 7f6f03400640 1 -- 172.21.15.67:0/3999069695 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4032810 con 0x7f6f0415e870 2024-05-30T16:43:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.325+0000 7f6f09c00640 1 -- 172.21.15.67:0/3999069695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 msgr2=0x7f6f040d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.325+0000 7f6f09c00640 1 --2- 172.21.15.67:0/3999069695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 0x7f6f040d0f90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009870 tx=0x7f6ef402a270 comp rx=0 tx=0).stop 2024-05-30T16:43:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.325+0000 7f6f09c00640 1 -- 172.21.15.67:0/3999069695 shutdown_connections 2024-05-30T16:43:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.325+0000 7f6f09c00640 1 --2- 172.21.15.67:0/3999069695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f040d14d0 0x7f6f040cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.325+0000 7f6f09c00640 1 --2- 172.21.15.67:0/3999069695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 0x7f6f040d0f90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.325+0000 7f6f09c00640 1 -- 172.21.15.67:0/3999069695 >> 172.21.15.67:0/3999069695 conn(0x7f6f0408bfc0 msgr2=0x7f6f040d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.326+0000 7f6f09c00640 1 -- 172.21.15.67:0/3999069695 shutdown_connections 2024-05-30T16:43:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.326+0000 7f6f09c00640 1 -- 172.21.15.67:0/3999069695 wait complete. 2024-05-30T16:43:49.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.327+0000 7f6f09c00640 1 Processor -- start 2024-05-30T16:43:49.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.327+0000 7f6f09c00640 1 -- start start 2024-05-30T16:43:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.328+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f040cf870 0x7f6f0415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.328+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f040d14d0 0x7f6f041601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.328+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 0x7f6f041e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.328+0000 7f6f09c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6f040d0820 con 0x7f6f0415e870 2024-05-30T16:43:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.328+0000 7f6f09c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f040d0540 con 0x7f6f040cf870 2024-05-30T16:43:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.328+0000 7f6f09c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f040d4fb0 con 0x7f6f040d14d0 2024-05-30T16:43:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f09200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 0x7f6f041e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f09200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 0x7f6f041e96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52168/0 (socket says 172.21.15.67:52168) 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f09200640 1 -- 172.21.15.67:0/2783576435 learned_addr learned my addr 172.21.15.67:0/2783576435 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f03e00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f040cf870 0x7f6f0415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6efbe00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f040d14d0 0x7f6f041601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f09200640 1 -- 172.21.15.67:0/2783576435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f040d14d0 msgr2=0x7f6f041601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f09200640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f040d14d0 0x7f6f041601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f09200640 1 -- 172.21.15.67:0/2783576435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f040cf870 msgr2=0x7f6f0415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f09200640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f040cf870 0x7f6f0415a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f09200640 1 -- 172.21.15.67:0/2783576435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f04160d20 con 0x7f6f0415e870 2024-05-30T16:43:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.329+0000 7f6f09200640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 0x7f6f041e96d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c730 tx=0x7f6efc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.330+0000 7f6f00c00640 1 -- 172.21.15.67:0/2783576435 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc018070 con 0x7f6f0415e870 2024-05-30T16:43:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.330+0000 7f6f00c00640 1 -- 172.21.15.67:0/2783576435 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6efc00ce40 con 0x7f6f0415e870 2024-05-30T16:43:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.330+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f04161000 con 0x7f6f0415e870 2024-05-30T16:43:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.330+0000 7f6f00c00640 1 -- 172.21.15.67:0/2783576435 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc00a860 con 0x7f6f0415e870 2024-05-30T16:43:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.330+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0414ce60 con 0x7f6f0415e870 2024-05-30T16:43:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.331+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0414d0f0 con 0x7f6f0415e870 2024-05-30T16:43:49.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.332+0000 7f6f00c00640 1 -- 172.21.15.67:0/2783576435 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6efc01b070 con 0x7f6f0415e870 2024-05-30T16:43:49.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.333+0000 7f6f00c00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee006c8f0 0x7f6ee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.333+0000 7f6f00c00640 1 -- 172.21.15.67:0/2783576435 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f6efc08ae10 con 0x7f6f0415e870 2024-05-30T16:43:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.333+0000 7f6f03e00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee006c8f0 0x7f6ee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.334+0000 7f6f03e00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee006c8f0 0x7f6ee006edb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4034000 tx=0x7f6ef402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:49.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.337+0000 7f6f00c00640 1 -- 172.21.15.67:0/2783576435 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6efc08e030 con 0x7f6f0415e870 2024-05-30T16:43:49.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.578+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f040689e0 con 0x7f6f0415e870 2024-05-30T16:43:49.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.579+0000 7f6f00c00640 1 -- 172.21.15.67:0/2783576435 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f6f040cfd00 con 0x7f6f0415e870 2024-05-30T16:43:49.580 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:49.581 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2254297887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2254297887},{"type":"v1","addr":"172.21.15.67:6837","nonce":2254297887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/227511509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":227511509},{"type":"v1","addr":"172.21.15.169:6834","nonce":227511509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2028878495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2028878495},{"type":"v1","addr":"172.21.15.169:6835","nonce":2028878495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-30T16:43:49.581 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2024-05-30T16:43:49.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.582+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee006c8f0 msgr2=0x7f6ee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.582+0000 7f6f09c00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee006c8f0 0x7f6ee006edb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4034000 tx=0x7f6ef402e000 comp rx=0 tx=0).stop 2024-05-30T16:43:49.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 msgr2=0x7f6f041e96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 0x7f6f041e96d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c730 tx=0x7f6efc00cbf0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 shutdown_connections 2024-05-30T16:43:49.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee006c8f0 0x7f6ee006edb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0415e870 0x7f6f041e96d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f040d14d0 0x7f6f041601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 --2- 172.21.15.67:0/2783576435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f040cf870 0x7f6f0415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 >> 172.21.15.67:0/2783576435 conn(0x7f6f0408bfc0 msgr2=0x7f6f040b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:49.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 shutdown_connections 2024-05-30T16:43:49.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.583+0000 7f6f09c00640 1 -- 172.21.15.67:0/2783576435 wait complete. 2024-05-30T16:43:49.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-30T16:43:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.803+0000 7f34dca00640 1 Processor -- start 2024-05-30T16:43:49.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.804+0000 7f34dca00640 1 -- start start 2024-05-30T16:43:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.805+0000 7f34dca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f34d815e870 0x7f34d80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.805+0000 7f34dca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34d815ec50 con 0x7f34d80cfa40 2024-05-30T16:43:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.805+0000 7f34dca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34d80cff80 con 0x7f34d80d14d0 2024-05-30T16:43:49.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.805+0000 7f34dca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f34d80d2150 con 0x7f34d815e870 2024-05-30T16:43:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.806+0000 7f34d7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f34d80cfa40 0x7f34d80d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37918/0 (socket says 172.21.15.67:37918) 2024-05-30T16:43:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.806+0000 7f34d7e00640 1 -- 172.21.15.67:0/1823513733 learned_addr learned my addr 172.21.15.67:0/1823513733 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.806+0000 7f34d7400640 1 --2- 172.21.15.67:0/1823513733 >> v2:172.21.15.169:3300/0 conn(0x7f34d815e870 0x7f34d80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.806+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810809797 0 0) 0x7f34d815ec50 con 0x7f34d80cfa40 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.806+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34bc003520 con 0x7f34d80cfa40 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 577302883 0 0) 0x7f34bc003520 con 0x7f34d80cfa40 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 >> v2:172.21.15.169:3300/0 conn(0x7f34d815e870 msgr2=0x7f34d80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 --2- 172.21.15.67:0/1823513733 >> v2:172.21.15.169:3300/0 conn(0x7f34d815e870 0x7f34d80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 >> v1:172.21.15.169:6789/0 conn(0x7f34d80d14d0 legacy=0x7f34d80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d80d0260 con 0x7f34d80cfa40 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f34d0001670 con 0x7f34d80cfa40 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d7400640 1 --2- 172.21.15.67:0/1823513733 >> v2:172.21.15.169:3300/0 conn(0x7f34d815e870 0x7f34d80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 >> v1:172.21.15.67:6789/0 conn(0x7f34d80cfa40 legacy=0x7f34d80d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34bc0039b0 0x7f34bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34bc004800 0x7f34bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34bc0051c0 0x7f34bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34d80d2150 con 0x7f34bc0039b0 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc00c060 con 0x7f34bc004800 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc00c260 con 0x7f34bc0051c0 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d7400640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34bc0039b0 0x7f34bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d7e00640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34bc0051c0 0x7f34bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d7400640 1 -- 172.21.15.67:0/1823513733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34bc0051c0 msgr2=0x7f34bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d7400640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34bc0051c0 0x7f34bc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d7400640 1 -- 172.21.15.67:0/1823513733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34bc004800 msgr2=0x7f34bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d7400640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34bc004800 0x7f34bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d7e00640 1 -- 172.21.15.67:0/1823513733 reap_dead start 2024-05-30T16:43:49.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.807+0000 7f34d7400640 1 -- 172.21.15.67:0/1823513733 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc003520 con 0x7f34bc0039b0 2024-05-30T16:43:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.809+0000 7f34d7400640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34bc0039b0 0x7f34bc003dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009c90 tx=0x7f34bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6afa4012ff3047 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.809+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0036070 con 0x7f34bc0039b0 2024-05-30T16:43:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.809+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c0031400 con 0x7f34bc0039b0 2024-05-30T16:43:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.809+0000 7f34d6000640 1 -- 172.21.15.67:0/1823513733 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0009860 con 0x7f34bc0039b0 2024-05-30T16:43:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.809+0000 7f34dca00640 1 -- 172.21.15.67:0/1823513733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34bc0039b0 msgr2=0x7f34bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.809+0000 7f34dca00640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34bc0039b0 0x7f34bc003dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009c90 tx=0x7f34bc005830 comp rx=0 tx=0).stop 2024-05-30T16:43:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.810+0000 7f34dca00640 1 -- 172.21.15.67:0/1823513733 shutdown_connections 2024-05-30T16:43:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.810+0000 7f34dca00640 1 --2- 172.21.15.67:0/1823513733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34bc0039b0 0x7f34bc003dd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.810+0000 7f34dca00640 1 -- 172.21.15.67:0/1823513733 >> 172.21.15.67:0/1823513733 conn(0x7f34d808bfc0 msgr2=0x7f34d80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.810+0000 7f34dca00640 1 -- 172.21.15.67:0/1823513733 shutdown_connections 2024-05-30T16:43:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.810+0000 7f34dca00640 1 -- 172.21.15.67:0/1823513733 wait complete. 2024-05-30T16:43:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.811+0000 7f34dca00640 1 Processor -- start 2024-05-30T16:43:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.812+0000 7f34dca00640 1 -- start start 2024-05-30T16:43:49.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.812+0000 7f34dca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34bc0039b0 0x7f34d80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.812+0000 7f34dca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34d80ccea0 0x7f34d80cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.812+0000 7f34dca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d80cd320 0x7f34d80ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.812+0000 7f34dca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34d80d4fb0 con 0x7f34d80ccea0 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.812+0000 7f34dca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34d80d0820 con 0x7f34bc0039b0 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.812+0000 7f34dca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34d80d0f90 con 0x7f34d80cd320 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34d80ccea0 0x7f34d80cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34d80ccea0 0x7f34d80cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52186/0 (socket says 172.21.15.67:52186) 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d6a00640 1 -- 172.21.15.67:0/3138108288 learned_addr learned my addr 172.21.15.67:0/3138108288 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d7e00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d80cd320 0x7f34d80ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d7400640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34bc0039b0 0x7f34d80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d6a00640 1 -- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d80cd320 msgr2=0x7f34d80ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d6a00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d80cd320 0x7f34d80ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d6a00640 1 -- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34bc0039b0 msgr2=0x7f34d80cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d6a00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34bc0039b0 0x7f34d80cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d6a00640 1 -- 172.21.15.67:0/3138108288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d80cd880 con 0x7f34d80ccea0 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d7e00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d80cd320 0x7f34d80ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.813+0000 7f34d7400640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34bc0039b0 0x7f34d80cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.814+0000 7f34d6a00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34d80ccea0 0x7f34d80cdc40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f34c800c770 tx=0x7f34c800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.814+0000 7f34cf400640 1 -- 172.21.15.67:0/3138108288 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8016020 con 0x7f34d80ccea0 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.814+0000 7f34cf400640 1 -- 172.21.15.67:0/3138108288 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c8006de0 con 0x7f34d80ccea0 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.814+0000 7f34cf400640 1 -- 172.21.15.67:0/3138108288 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8003c00 con 0x7f34d80ccea0 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.814+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d80cb8a0 con 0x7f34d80ccea0 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.814+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d80cc8c0 con 0x7f34d80ccea0 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.815+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d80689e0 con 0x7f34d80ccea0 2024-05-30T16:43:49.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.816+0000 7f34cf400640 1 -- 172.21.15.67:0/3138108288 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34c800a850 con 0x7f34d80ccea0 2024-05-30T16:43:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.817+0000 7f34cf400640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34b406c8f0 0x7f34b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.818+0000 7f34cf400640 1 -- 172.21.15.67:0/3138108288 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f34c808b560 con 0x7f34d80ccea0 2024-05-30T16:43:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.818+0000 7f34d7400640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34b406c8f0 0x7f34b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.818+0000 7f34d7400640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34b406c8f0 0x7f34b406edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f34c0033000 tx=0x7f34c0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:49.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:49.822+0000 7f34cf400640 1 -- 172.21.15.67:0/3138108288 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34c8020020 con 0x7f34d80ccea0 2024-05-30T16:43:50.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.029+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f34d80b5f20 con 0x7f34d80ccea0 2024-05-30T16:43:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.030+0000 7f34cf400640 1 -- 172.21.15.67:0/3138108288 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f34c8019310 con 0x7f34d80ccea0 2024-05-30T16:43:50.031 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:43:50.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:43:50.031 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-05-30T16:43:50.031 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-05-30T16:43:50.031 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:43:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:43:50.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.032+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34b406c8f0 msgr2=0x7f34b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:50.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.032+0000 7f34dca00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34b406c8f0 0x7f34b406edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f34c0033000 tx=0x7f34c0008030 comp rx=0 tx=0).stop 2024-05-30T16:43:50.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.032+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34d80ccea0 msgr2=0x7f34d80cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.032+0000 7f34dca00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34d80ccea0 0x7f34d80cdc40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f34c800c770 tx=0x7f34c800cc30 comp rx=0 tx=0).stop 2024-05-30T16:43:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.033+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 shutdown_connections 2024-05-30T16:43:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.033+0000 7f34dca00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34b406c8f0 0x7f34b406edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.033+0000 7f34dca00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d80cd320 0x7f34d80ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.033+0000 7f34dca00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34d80ccea0 0x7f34d80cdc40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.033+0000 7f34dca00640 1 --2- 172.21.15.67:0/3138108288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34bc0039b0 0x7f34d80cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.033+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 >> 172.21.15.67:0/3138108288 conn(0x7f34d808bfc0 msgr2=0x7f34d80b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.033+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 shutdown_connections 2024-05-30T16:43:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.033+0000 7f34dca00640 1 -- 172.21.15.67:0/3138108288 wait complete. 2024-05-30T16:43:50.052 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-30T16:43:50.053 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-30T16:43:50.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.259+0000 7f5568a00640 1 Processor -- start 2024-05-30T16:43:50.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.259+0000 7f5568a00640 1 -- start start 2024-05-30T16:43:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.260+0000 7f5568a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55641621d0 0x7f55640d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.260+0000 7f5568a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f55641625b0 0x7f55640d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.260+0000 7f5568a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55640d0e50 con 0x7f556415ea10 2024-05-30T16:43:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.260+0000 7f5568a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f55640d1a60 con 0x7f55641625b0 2024-05-30T16:43:50.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.260+0000 7f5568a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f55640d1130 con 0x7f55641621d0 2024-05-30T16:43:50.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5561e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f55641625b0 0x7f55640d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:50.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5563200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f556415ea10 0x7f556415ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37920/0 (socket says 172.21.15.67:37920) 2024-05-30T16:43:50.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5561e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f55641625b0 0x7f55640d4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52196/0 (socket says 172.21.15.67:52196) 2024-05-30T16:43:50.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5563200640 1 -- 172.21.15.67:0/2263584820 learned_addr learned my addr 172.21.15.67:0/2263584820 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:50.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5562800640 1 --2- 172.21.15.67:0/2263584820 >> v2:172.21.15.169:3300/0 conn(0x7f55641621d0 0x7f55640d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:50.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5561e00640 1 -- 172.21.15.67:0/2263584820 >> v2:172.21.15.169:3300/0 conn(0x7f55641621d0 msgr2=0x7f55640d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:50.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5561e00640 1 --2- 172.21.15.67:0/2263584820 >> v2:172.21.15.169:3300/0 conn(0x7f55641621d0 0x7f55640d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5561e00640 1 -- 172.21.15.67:0/2263584820 >> v1:172.21.15.67:6789/0 conn(0x7f556415ea10 legacy=0x7f556415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:50.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5561e00640 1 -- 172.21.15.67:0/2263584820 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55640d1410 con 0x7f55641625b0 2024-05-30T16:43:50.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.261+0000 7f5561400640 1 -- 172.21.15.67:0/2263584820 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 813818258 0 0) 0x7f55640d0e50 con 0x7f556415ea10 2024-05-30T16:43:50.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.262+0000 7f5561e00640 1 --2- 172.21.15.67:0/2263584820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641625b0 0x7f55640d4f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55640b71d0 tx=0x7f555002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3491074668a5e719 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:50.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.263+0000 7f5561400640 1 -- 172.21.15.67:0/2263584820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550037070 con 0x7f55641625b0 2024-05-30T16:43:50.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.263+0000 7f5561400640 1 -- 172.21.15.67:0/2263584820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5550032440 con 0x7f55641625b0 2024-05-30T16:43:50.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.263+0000 7f5561400640 1 -- 172.21.15.67:0/2263584820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550032810 con 0x7f55641625b0 2024-05-30T16:43:50.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.263+0000 7f5568a00640 1 -- 172.21.15.67:0/2263584820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641625b0 msgr2=0x7f55640d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:50.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.263+0000 7f5568a00640 1 --2- 172.21.15.67:0/2263584820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641625b0 0x7f55640d4f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55640b71d0 tx=0x7f555002a1a0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.263+0000 7f5568a00640 1 -- 172.21.15.67:0/2263584820 shutdown_connections 2024-05-30T16:43:50.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.263+0000 7f5568a00640 1 --2- 172.21.15.67:0/2263584820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641625b0 0x7f55640d4f30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.263+0000 7f5568a00640 1 --2- 172.21.15.67:0/2263584820 >> v2:172.21.15.169:3300/0 conn(0x7f55641621d0 0x7f55640d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.263+0000 7f5568a00640 1 -- 172.21.15.67:0/2263584820 >> 172.21.15.67:0/2263584820 conn(0x7f556408bfc0 msgr2=0x7f55640d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:50.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.264+0000 7f5568a00640 1 -- 172.21.15.67:0/2263584820 shutdown_connections 2024-05-30T16:43:50.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.264+0000 7f5568a00640 1 -- 172.21.15.67:0/2263584820 wait complete. 2024-05-30T16:43:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.265+0000 7f5568a00640 1 Processor -- start 2024-05-30T16:43:50.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.265+0000 7f5568a00640 1 -- start start 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.266+0000 7f5568a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f556415ea10 0x7f55641f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.266+0000 7f5568a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641621d0 0x7f55641ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.266+0000 7f5568a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55641625b0 0x7f556415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.266+0000 7f5568a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55640d5470 con 0x7f55641621d0 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.266+0000 7f5568a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55640d1ef0 con 0x7f55641625b0 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.266+0000 7f5568a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5564162a40 con 0x7f556415ea10 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.267+0000 7f5561e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641621d0 0x7f55641ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.267+0000 7f5561e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641621d0 0x7f55641ea100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52212/0 (socket says 172.21.15.67:52212) 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.267+0000 7f5561e00640 1 -- 172.21.15.67:0/1928024693 learned_addr learned my addr 172.21.15.67:0/1928024693 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.267+0000 7f5563200640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55641625b0 0x7f556415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.267+0000 7f5561e00640 1 -- 172.21.15.67:0/1928024693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f556415ea10 msgr2=0x7f55641f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.267+0000 7f5561e00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f556415ea10 0x7f55641f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.267+0000 7f5561e00640 1 -- 172.21.15.67:0/1928024693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55641625b0 msgr2=0x7f556415c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.267+0000 7f5561e00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55641625b0 0x7f556415c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.267+0000 7f5561e00640 1 -- 172.21.15.67:0/1928024693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55640b9570 con 0x7f55641621d0 2024-05-30T16:43:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.268+0000 7f5561e00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641621d0 0x7f55641ea100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5550000c00 tx=0x7f55500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.268+0000 7f555aa00640 1 -- 172.21.15.67:0/1928024693 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550043070 con 0x7f55641621d0 2024-05-30T16:43:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.268+0000 7f555aa00640 1 -- 172.21.15.67:0/1928024693 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5550038040 con 0x7f55641621d0 2024-05-30T16:43:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.268+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55640b9850 con 0x7f55641621d0 2024-05-30T16:43:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.268+0000 7f555aa00640 1 -- 172.21.15.67:0/1928024693 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550037050 con 0x7f55641621d0 2024-05-30T16:43:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.268+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55640b9e10 con 0x7f55641621d0 2024-05-30T16:43:50.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.269+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55640689e0 con 0x7f55641621d0 2024-05-30T16:43:50.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.270+0000 7f555aa00640 1 -- 172.21.15.67:0/1928024693 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5550046070 con 0x7f55641621d0 2024-05-30T16:43:50.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.271+0000 7f555aa00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f554406c8f0 0x7f554406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:50.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.271+0000 7f555aa00640 1 -- 172.21.15.67:0/1928024693 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f55500ab950 con 0x7f55641621d0 2024-05-30T16:43:50.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.272+0000 7f5562800640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f554406c8f0 0x7f554406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:50.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.272+0000 7f5562800640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f554406c8f0 0x7f554406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5554006fd0 tx=0x7f5554008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:50.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.276+0000 7f555aa00640 1 -- 172.21.15.67:0/1928024693 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f555007a0c0 con 0x7f55641621d0 2024-05-30T16:43:50.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.479+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f55640d3000 con 0x7f55641621d0 2024-05-30T16:43:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.807+0000 7f555aa00640 1 -- 172.21.15.67:0/1928024693 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f55500799c0 con 0x7f55641621d0 2024-05-30T16:43:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-05-30T16:43:50.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f554406c8f0 msgr2=0x7f554406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:50.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f554406c8f0 0x7f554406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5554006fd0 tx=0x7f5554008040 comp rx=0 tx=0).stop 2024-05-30T16:43:50.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641621d0 msgr2=0x7f55641ea100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:50.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641621d0 0x7f55641ea100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5550000c00 tx=0x7f55500046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 shutdown_connections 2024-05-30T16:43:50.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f554406c8f0 0x7f554406edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55641625b0 0x7f556415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55641621d0 0x7f55641ea100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 --2- 172.21.15.67:0/1928024693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f556415ea10 0x7f55641f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:50.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.811+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 >> 172.21.15.67:0/1928024693 conn(0x7f556408bfc0 msgr2=0x7f556415dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:50.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.812+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 shutdown_connections 2024-05-30T16:43:50.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:50.812+0000 7f5568a00640 1 -- 172.21.15.67:0/1928024693 wait complete. 2024-05-30T16:43:50.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-30T16:43:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.033+0000 7f6b8f000640 1 Processor -- start 2024-05-30T16:43:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.033+0000 7f6b8f000640 1 -- start start 2024-05-30T16:43:51.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.035+0000 7f6b8f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6b880caa40 0x7f6b880ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:51.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.035+0000 7f6b8f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b880b3e80 con 0x7f6b880cae20 2024-05-30T16:43:51.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.035+0000 7f6b8f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b880cde80 con 0x7f6b881524c0 2024-05-30T16:43:51.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.035+0000 7f6b8f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6b880d0b30 con 0x7f6b880caa40 2024-05-30T16:43:51.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.035+0000 7f6b8d200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6b880cae20 0x7f6b88151db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35184/0 (socket says 172.21.15.67:35184) 2024-05-30T16:43:51.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.035+0000 7f6b8d200640 1 -- 172.21.15.67:0/2774859861 learned_addr learned my addr 172.21.15.67:0/2774859861 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b8dc00640 1 --2- 172.21.15.67:0/2774859861 >> v2:172.21.15.169:3300/0 conn(0x7f6b880caa40 0x7f6b880ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 684676319 0 0) 0x7f6b880b3e80 con 0x7f6b880cae20 2024-05-30T16:43:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b70003520 con 0x7f6b880cae20 2024-05-30T16:43:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2988464387 0 0) 0x7f6b70003520 con 0x7f6b880cae20 2024-05-30T16:43:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 >> v2:172.21.15.169:3300/0 conn(0x7f6b880caa40 msgr2=0x7f6b880ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 --2- 172.21.15.67:0/2774859861 >> v2:172.21.15.169:3300/0 conn(0x7f6b880caa40 0x7f6b880ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 >> v1:172.21.15.169:6789/0 conn(0x7f6b881524c0 legacy=0x7f6b880cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b880ce160 con 0x7f6b880cae20 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f6b74001670 con 0x7f6b880cae20 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 >> v1:172.21.15.67:6789/0 conn(0x7f6b880cae20 legacy=0x7f6b88151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b700039b0 0x7f6b70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b70004800 0x7f6b70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b700051c0 0x7f6b7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6b880d0b30 con 0x7f6b70004800 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7000c160 con 0x7f6b700051c0 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.036+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7000c360 con 0x7f6b700039b0 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.037+0000 7f6b8d200640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b70004800 0x7f6b70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.037+0000 7f6b8dc00640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b700039b0 0x7f6b70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.037+0000 7f6b8e600640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b700051c0 0x7f6b7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.037+0000 7f6b8d200640 1 -- 172.21.15.67:0/2774859861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b700039b0 msgr2=0x7f6b70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.037+0000 7f6b8d200640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b700039b0 0x7f6b70003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.037+0000 7f6b8d200640 1 -- 172.21.15.67:0/2774859861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b700051c0 msgr2=0x7f6b7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.037+0000 7f6b8d200640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b700051c0 0x7f6b7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.037+0000 7f6b8d200640 1 -- 172.21.15.67:0/2774859861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b70003520 con 0x7f6b70004800 2024-05-30T16:43:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.037+0000 7f6b8e600640 1 -- 172.21.15.67:0/2774859861 reap_dead start 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.038+0000 7f6b8d200640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b70004800 0x7f6b70004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c240 tx=0x7f6b70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8066ef414a31bb0a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.038+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74007cf0 con 0x7f6b70004800 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.039+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b740058d0 con 0x7f6b70004800 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.039+0000 7f6b87e00640 1 -- 172.21.15.67:0/2774859861 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74005d20 con 0x7f6b70004800 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.039+0000 7f6b8f000640 1 -- 172.21.15.67:0/2774859861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b70004800 msgr2=0x7f6b70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.039+0000 7f6b8f000640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b70004800 0x7f6b70004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c240 tx=0x7f6b70005830 comp rx=0 tx=0).stop 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.039+0000 7f6b8f000640 1 -- 172.21.15.67:0/2774859861 shutdown_connections 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.039+0000 7f6b8f000640 1 --2- 172.21.15.67:0/2774859861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b70004800 0x7f6b70004c80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.039+0000 7f6b8f000640 1 -- 172.21.15.67:0/2774859861 >> 172.21.15.67:0/2774859861 conn(0x7f6b8806b900 msgr2=0x7f6b8806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.039+0000 7f6b8f000640 1 -- 172.21.15.67:0/2774859861 shutdown_connections 2024-05-30T16:43:51.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.039+0000 7f6b8f000640 1 -- 172.21.15.67:0/2774859861 wait complete. 2024-05-30T16:43:51.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.041+0000 7f6b8f000640 1 Processor -- start 2024-05-30T16:43:51.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.041+0000 7f6b8f000640 1 -- start start 2024-05-30T16:43:51.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.042+0000 7f6b8f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b70004800 0x7f6b881542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:51.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.042+0000 7f6b8f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b881547f0 0x7f6b881f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.042+0000 7f6b8f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b8809a7f0 0x7f6b881f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.042+0000 7f6b8f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88151db0 con 0x7f6b8809a7f0 2024-05-30T16:43:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.042+0000 7f6b8f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b880b4160 con 0x7f6b881547f0 2024-05-30T16:43:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.042+0000 7f6b8f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b88151f30 con 0x7f6b70004800 2024-05-30T16:43:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b8809a7f0 0x7f6b881f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b8809a7f0 0x7f6b881f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43382/0 (socket says 172.21.15.67:43382) 2024-05-30T16:43:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8e600640 1 -- 172.21.15.67:0/3781540241 learned_addr learned my addr 172.21.15.67:0/3781540241 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8dc00640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b70004800 0x7f6b881542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8d200640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b881547f0 0x7f6b881f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:51.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8e600640 1 -- 172.21.15.67:0/3781540241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b70004800 msgr2=0x7f6b881542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:51.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8e600640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b70004800 0x7f6b881542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8e600640 1 -- 172.21.15.67:0/3781540241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b881547f0 msgr2=0x7f6b881f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:51.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8e600640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b881547f0 0x7f6b881f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.043+0000 7f6b8e600640 1 -- 172.21.15.67:0/3781540241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b88155230 con 0x7f6b8809a7f0 2024-05-30T16:43:51.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.044+0000 7f6b8e600640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b8809a7f0 0x7f6b881f4b60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b8806c8e0 tx=0x7f6b8000ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:51.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.044+0000 7f6b85600640 1 -- 172.21.15.67:0/3781540241 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80019070 con 0x7f6b8809a7f0 2024-05-30T16:43:51.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.044+0000 7f6b85600640 1 -- 172.21.15.67:0/3781540241 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b80006d60 con 0x7f6b8809a7f0 2024-05-30T16:43:51.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.044+0000 7f6b85600640 1 -- 172.21.15.67:0/3781540241 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80003b90 con 0x7f6b8809a7f0 2024-05-30T16:43:51.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.044+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b88154c70 con 0x7f6b8809a7f0 2024-05-30T16:43:51.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.044+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8809c150 con 0x7f6b8809a7f0 2024-05-30T16:43:51.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.045+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b880689e0 con 0x7f6b8809a7f0 2024-05-30T16:43:51.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.047+0000 7f6b85600640 1 -- 172.21.15.67:0/3781540241 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b8001c070 con 0x7f6b8809a7f0 2024-05-30T16:43:51.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.048+0000 7f6b85600640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6b6806c8f0 0x7f6b6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.048+0000 7f6b85600640 1 -- 172.21.15.67:0/3781540241 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f6b8008afc0 con 0x7f6b8809a7f0 2024-05-30T16:43:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.048+0000 7f6b8dc00640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6b6806c8f0 0x7f6b6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.049+0000 7f6b8dc00640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6b6806c8f0 0x7f6b6806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6b88051690 tx=0x7f6b7c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.054+0000 7f6b85600640 1 -- 172.21.15.67:0/3781540241 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b80059530 con 0x7f6b8809a7f0 2024-05-30T16:43:51.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.262+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6b8809bf20 con 0x7f6b8809a7f0 2024-05-30T16:43:51.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.817+0000 7f6b85600640 1 -- 172.21.15.67:0/3781540241 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f6b80058e30 con 0x7f6b8809a7f0 2024-05-30T16:43:51.818 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-05-30T16:43:51.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6b6806c8f0 msgr2=0x7f6b6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:51.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6b6806c8f0 0x7f6b6806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6b88051690 tx=0x7f6b7c008040 comp rx=0 tx=0).stop 2024-05-30T16:43:51.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b8809a7f0 msgr2=0x7f6b881f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:51.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b8809a7f0 0x7f6b881f4b60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b8806c8e0 tx=0x7f6b8000ea00 comp rx=0 tx=0).stop 2024-05-30T16:43:51.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 shutdown_connections 2024-05-30T16:43:51.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6b6806c8f0 0x7f6b6806edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b8809a7f0 0x7f6b881f4b60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b881547f0 0x7f6b881f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 --2- 172.21.15.67:0/3781540241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b70004800 0x7f6b881542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:51.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 >> 172.21.15.67:0/3781540241 conn(0x7f6b8806b900 msgr2=0x7f6b8815ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:51.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.822+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 shutdown_connections 2024-05-30T16:43:51.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:51.823+0000 7f6b8f000640 1 -- 172.21.15.67:0/3781540241 wait complete. 2024-05-30T16:43:51.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-30T16:43:52.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.046+0000 7f1c1ee00640 1 Processor -- start 2024-05-30T16:43:52.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.046+0000 7f1c1ee00640 1 -- start start 2024-05-30T16:43:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.047+0000 7f1c1ee00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1c1815d270 0x7f1c180b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.047+0000 7f1c1ee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1815d680 con 0x7f1c18153510 2024-05-30T16:43:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.047+0000 7f1c1ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1815d900 con 0x7f1c180cd2a0 2024-05-30T16:43:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.047+0000 7f1c1ee00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1c181538f0 con 0x7f1c1815d270 2024-05-30T16:43:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.047+0000 7f1c1da00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1c18153510 0x7f1c180ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35198/0 (socket says 172.21.15.67:35198) 2024-05-30T16:43:52.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.047+0000 7f1c1da00640 1 -- 172.21.15.67:0/2420512048 learned_addr learned my addr 172.21.15.67:0/2420512048 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c1e400640 1 --2- 172.21.15.67:0/2420512048 >> v2:172.21.15.169:3300/0 conn(0x7f1c1815d270 0x7f1c180b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122901736 0 0) 0x7f1c0c001110 con 0x7f1c18153510 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c00003520 con 0x7f1c18153510 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1627089464 0 0) 0x7f1c00003520 con 0x7f1c18153510 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 >> v2:172.21.15.169:3300/0 conn(0x7f1c1815d270 msgr2=0x7f1c180b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 --2- 172.21.15.67:0/2420512048 >> v2:172.21.15.169:3300/0 conn(0x7f1c1815d270 0x7f1c180b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 >> v1:172.21.15.169:6789/0 conn(0x7f1c180cd2a0 legacy=0x7f1c1815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c180b3f90 con 0x7f1c18153510 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f1c0c001840 con 0x7f1c18153510 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 >> v1:172.21.15.67:6789/0 conn(0x7f1c18153510 legacy=0x7f1c180ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:52.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00004800 0x7f1c00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.048+0000 7f1c17e00640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c000051c0 0x7f1c0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0000c130 con 0x7f1c000039b0 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0000c2e0 con 0x7f1c000051c0 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0000c4e0 con 0x7f1c00004800 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c1da00640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c1d000640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00004800 0x7f1c00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c1e400640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c000051c0 0x7f1c0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c1da00640 1 -- 172.21.15.67:0/2420512048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00004800 msgr2=0x7f1c00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c1da00640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00004800 0x7f1c00004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c1da00640 1 -- 172.21.15.67:0/2420512048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c000051c0 msgr2=0x7f1c0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c1da00640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c000051c0 0x7f1c0000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c1da00640 1 -- 172.21.15.67:0/2420512048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c180b3f90 con 0x7f1c000039b0 2024-05-30T16:43:52.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.049+0000 7f1c1e400640 1 -- 172.21.15.67:0/2420512048 reap_dead start 2024-05-30T16:43:52.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.050+0000 7f1c1da00640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c00003dd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00c4e0 tx=0x7f1c00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbe33ebc38e3c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.050+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c00c0e0 con 0x7f1c000039b0 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.050+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0c034070 con 0x7f1c000039b0 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.050+0000 7f1c17e00640 1 -- 172.21.15.67:0/2420512048 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c005d30 con 0x7f1c000039b0 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.051+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2420512048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 msgr2=0x7f1c00003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.051+0000 7f1c1ee00640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c00003dd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00c4e0 tx=0x7f1c00005830 comp rx=0 tx=0).stop 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.051+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2420512048 shutdown_connections 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.051+0000 7f1c1ee00640 1 --2- 172.21.15.67:0/2420512048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c00003dd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.051+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2420512048 >> 172.21.15.67:0/2420512048 conn(0x7f1c1806b900 msgr2=0x7f1c1806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.051+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2420512048 shutdown_connections 2024-05-30T16:43:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.051+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2420512048 wait complete. 2024-05-30T16:43:52.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.053+0000 7f1c1ee00640 1 Processor -- start 2024-05-30T16:43:52.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.053+0000 7f1c1ee00640 1 -- start start 2024-05-30T16:43:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.054+0000 7f1c1ee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c181f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.054+0000 7f1c1ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c180ce930 0x7f1c180cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.054+0000 7f1c1ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1815b410 0x7f1c1815b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.054+0000 7f1c1ee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1c180b4550 con 0x7f1c000039b0 2024-05-30T16:43:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.054+0000 7f1c1ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c180b48c0 con 0x7f1c1815b410 2024-05-30T16:43:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.054+0000 7f1c1ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c180b4d50 con 0x7f1c180ce930 2024-05-30T16:43:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.054+0000 7f1c1da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c181f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.054+0000 7f1c1da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c181f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43400/0 (socket says 172.21.15.67:43400) 2024-05-30T16:43:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.054+0000 7f1c1da00640 1 -- 172.21.15.67:0/2947137408 learned_addr learned my addr 172.21.15.67:0/2947137408 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.055+0000 7f1c1da00640 1 -- 172.21.15.67:0/2947137408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c180ce930 msgr2=0x7f1c180cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.055+0000 7f1c1da00640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c180ce930 0x7f1c180cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.055+0000 7f1c1da00640 1 -- 172.21.15.67:0/2947137408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1815b410 msgr2=0x7f1c1815b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.055+0000 7f1c1e400640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1815b410 0x7f1c1815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.055+0000 7f1c1da00640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1815b410 0x7f1c1815b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.055+0000 7f1c1da00640 1 -- 172.21.15.67:0/2947137408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c181f4270 con 0x7f1c000039b0 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.055+0000 7f1c1da00640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c181f3d30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c000c00 tx=0x7f1c0c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.055+0000 7f1c15600640 1 -- 172.21.15.67:0/2947137408 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c007270 con 0x7f1c000039b0 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.055+0000 7f1c15600640 1 -- 172.21.15.67:0/2947137408 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0c03f030 con 0x7f1c000039b0 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.056+0000 7f1c15600640 1 -- 172.21.15.67:0/2947137408 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c007630 con 0x7f1c000039b0 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.056+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c180cf3f0 con 0x7f1c000039b0 2024-05-30T16:43:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.056+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c180cf9b0 con 0x7f1c000039b0 2024-05-30T16:43:52.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.057+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c18160a30 con 0x7f1c000039b0 2024-05-30T16:43:52.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.058+0000 7f1c15600640 1 -- 172.21.15.67:0/2947137408 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c0c03b420 con 0x7f1c000039b0 2024-05-30T16:43:52.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.059+0000 7f1c15600640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bf806c8f0 0x7f1bf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:52.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.059+0000 7f1c15600640 1 -- 172.21.15.67:0/2947137408 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f1c0c0ac440 con 0x7f1c000039b0 2024-05-30T16:43:52.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.059+0000 7f1c1d000640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bf806c8f0 0x7f1bf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:52.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.060+0000 7f1c1d000640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bf806c8f0 0x7f1bf806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c04000f60 tx=0x7f1c04009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:52.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.065+0000 7f1c15600640 1 -- 172.21.15.67:0/2947137408 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c18160a30 con 0x7f1c000039b0 2024-05-30T16:43:52.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.307+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1c180681e0 con 0x7f1c000039b0 2024-05-30T16:43:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.860+0000 7f1c15600640 1 -- 172.21.15.67:0/2947137408 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f1c180681e0 con 0x7f1c000039b0 2024-05-30T16:43:52.865 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2024-05-30T16:43:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.868+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bf806c8f0 msgr2=0x7f1bf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.868+0000 7f1c1ee00640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bf806c8f0 0x7f1bf806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c04000f60 tx=0x7f1c04009040 comp rx=0 tx=0).stop 2024-05-30T16:43:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.868+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 msgr2=0x7f1c181f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.868+0000 7f1c1ee00640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c181f3d30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c000c00 tx=0x7f1c0c000fe0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.869+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 shutdown_connections 2024-05-30T16:43:52.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.869+0000 7f1c1ee00640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bf806c8f0 0x7f1bf806edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.869+0000 7f1c1ee00640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1815b410 0x7f1c1815b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.869+0000 7f1c1ee00640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c180ce930 0x7f1c180cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.869+0000 7f1c1ee00640 1 --2- 172.21.15.67:0/2947137408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c000039b0 0x7f1c181f3d30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:52.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.869+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 >> 172.21.15.67:0/2947137408 conn(0x7f1c1806b900 msgr2=0x7f1c180cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:52.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.870+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 shutdown_connections 2024-05-30T16:43:52.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:52.872+0000 7f1c1ee00640 1 -- 172.21.15.67:0/2947137408 wait complete. 2024-05-30T16:43:52.904 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:43:53.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.074+0000 7f8590400640 1 Processor -- start 2024-05-30T16:43:53.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.074+0000 7f8590400640 1 -- start start 2024-05-30T16:43:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.075+0000 7f8590400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85880d2db0 0x7f85880b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.075+0000 7f8590400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f85880d4d60 0x7f85880d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f8590400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85880d36d0 con 0x7f85880cf490 2024-05-30T16:43:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f8590400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f85880d5540 con 0x7f85880d2db0 2024-05-30T16:43:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f8590400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8588163500 con 0x7f85880d4d60 2024-05-30T16:43:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f858da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85880d2db0 0x7f85880b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f858e400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f85880cf490 0x7f85880cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35212/0 (socket says 172.21.15.67:35212) 2024-05-30T16:43:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f858e400640 1 -- 172.21.15.67:0/664421286 learned_addr learned my addr 172.21.15.67:0/664421286 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f858d000640 1 --2- 172.21.15.67:0/664421286 >> v2:172.21.15.169:3300/0 conn(0x7f85880d4d60 0x7f85880d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f858da00640 1 -- 172.21.15.67:0/664421286 >> v2:172.21.15.169:3300/0 conn(0x7f85880d4d60 msgr2=0x7f85880d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f858da00640 1 --2- 172.21.15.67:0/664421286 >> v2:172.21.15.169:3300/0 conn(0x7f85880d4d60 0x7f85880d3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f858da00640 1 -- 172.21.15.67:0/664421286 >> v1:172.21.15.67:6789/0 conn(0x7f85880cf490 legacy=0x7f85880cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f858da00640 1 -- 172.21.15.67:0/664421286 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8588162740 con 0x7f85880d2db0 2024-05-30T16:43:53.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.076+0000 7f8587e00640 1 -- 172.21.15.67:0/664421286 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3986130681 0 0) 0x7f85880d36d0 con 0x7f85880cf490 2024-05-30T16:43:53.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.078+0000 7f858da00640 1 --2- 172.21.15.67:0/664421286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880d2db0 0x7f85880b3b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85880b71b0 tx=0x7f857c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd517ba5ebf065e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.078+0000 7f8587e00640 1 -- 172.21.15.67:0/664421286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c037070 con 0x7f85880d2db0 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.078+0000 7f8587e00640 1 -- 172.21.15.67:0/664421286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c032440 con 0x7f85880d2db0 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.078+0000 7f8587e00640 1 -- 172.21.15.67:0/664421286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c02ae60 con 0x7f85880d2db0 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.078+0000 7f8590400640 1 -- 172.21.15.67:0/664421286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880d2db0 msgr2=0x7f85880b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.078+0000 7f8590400640 1 --2- 172.21.15.67:0/664421286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880d2db0 0x7f85880b3b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85880b71b0 tx=0x7f857c02a110 comp rx=0 tx=0).stop 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.079+0000 7f8590400640 1 -- 172.21.15.67:0/664421286 shutdown_connections 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.079+0000 7f8590400640 1 --2- 172.21.15.67:0/664421286 >> v2:172.21.15.169:3300/0 conn(0x7f85880d4d60 0x7f85880d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.079+0000 7f8590400640 1 --2- 172.21.15.67:0/664421286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880d2db0 0x7f85880b3b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.079+0000 7f8590400640 1 -- 172.21.15.67:0/664421286 >> 172.21.15.67:0/664421286 conn(0x7f858808bfc0 msgr2=0x7f85880d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.079+0000 7f8590400640 1 -- 172.21.15.67:0/664421286 shutdown_connections 2024-05-30T16:43:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.079+0000 7f8590400640 1 -- 172.21.15.67:0/664421286 wait complete. 2024-05-30T16:43:53.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.081+0000 7f8590400640 1 Processor -- start 2024-05-30T16:43:53.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.081+0000 7f8590400640 1 -- start start 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f8590400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880cf490 0x7f85880b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f8590400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85880d2db0 0x7f85880cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f8590400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85880d4d60 0x7f8588159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f8590400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8588162fe0 con 0x7f85880cf490 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f8590400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8588162d00 con 0x7f85880d4d60 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f8590400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f85880d51a0 con 0x7f85880d2db0 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f858da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880cf490 0x7f85880b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f858da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880cf490 0x7f85880b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43422/0 (socket says 172.21.15.67:43422) 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f858da00640 1 -- 172.21.15.67:0/2942176455 learned_addr learned my addr 172.21.15.67:0/2942176455 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f858da00640 1 -- 172.21.15.67:0/2942176455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85880d2db0 msgr2=0x7f85880cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f858da00640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85880d2db0 0x7f85880cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f858da00640 1 -- 172.21.15.67:0/2942176455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85880d4d60 msgr2=0x7f8588159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f858da00640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85880d4d60 0x7f8588159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.082+0000 7f858da00640 1 -- 172.21.15.67:0/2942176455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85880b2ca0 con 0x7f85880cf490 2024-05-30T16:43:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.083+0000 7f858da00640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880cf490 0x7f85880b25e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f857c000c00 tx=0x7f857c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.083+0000 7f8585600640 1 -- 172.21.15.67:0/2942176455 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c037050 con 0x7f85880cf490 2024-05-30T16:43:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.083+0000 7f8585600640 1 -- 172.21.15.67:0/2942176455 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c038040 con 0x7f85880cf490 2024-05-30T16:43:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.083+0000 7f8585600640 1 -- 172.21.15.67:0/2942176455 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c004450 con 0x7f85880cf490 2024-05-30T16:43:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.083+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85880b2f80 con 0x7f85880cf490 2024-05-30T16:43:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.083+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85880b1380 con 0x7f85880cf490 2024-05-30T16:43:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.084+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85880689e0 con 0x7f85880cf490 2024-05-30T16:43:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.085+0000 7f8585600640 1 -- 172.21.15.67:0/2942176455 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f857c030980 con 0x7f85880cf490 2024-05-30T16:43:53.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.086+0000 7f8585600640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f856806c8f0 0x7f856806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.086+0000 7f8585600640 1 -- 172.21.15.67:0/2942176455 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f857c0ac7d0 con 0x7f85880cf490 2024-05-30T16:43:53.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.086+0000 7f858d000640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f856806c8f0 0x7f856806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.087+0000 7f858d000640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f856806c8f0 0x7f856806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8574006fd0 tx=0x7f8574008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:53.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.091+0000 7f8585600640 1 -- 172.21.15.67:0/2942176455 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f857c07ab60 con 0x7f85880cf490 2024-05-30T16:43:53.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.295+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85880d2af0 con 0x7f85880cf490 2024-05-30T16:43:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.298+0000 7f8585600640 1 -- 172.21.15.67:0/2942176455 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16606 (secure 0 0 0) 0x7f857c07a460 con 0x7f85880cf490 2024-05-30T16:43:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:53.299 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:52.308492+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:43:50.496509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-30T16:43:51.276872+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:43:53.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f856806c8f0 msgr2=0x7f856806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f856806c8f0 0x7f856806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8574006fd0 tx=0x7f8574008040 comp rx=0 tx=0).stop 2024-05-30T16:43:53.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880cf490 msgr2=0x7f85880b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880cf490 0x7f85880b25e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f857c000c00 tx=0x7f857c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 shutdown_connections 2024-05-30T16:43:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f856806c8f0 0x7f856806edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85880d4d60 0x7f8588159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85880d2db0 0x7f85880cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 --2- 172.21.15.67:0/2942176455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85880cf490 0x7f85880b25e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.300+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 >> 172.21.15.67:0/2942176455 conn(0x7f858808bfc0 msgr2=0x7f85880d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.301+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 shutdown_connections 2024-05-30T16:43:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.301+0000 7f8590400640 1 -- 172.21.15.67:0/2942176455 wait complete. 2024-05-30T16:43:53.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:43:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.525+0000 7f5c5e200640 1 Processor -- start 2024-05-30T16:43:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.525+0000 7f5c5e200640 1 -- start start 2024-05-30T16:43:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.527+0000 7f5c5e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c58156b30 0x7f5c580af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.527+0000 7f5c5e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5c5806ee40 0x7f5c580ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.527+0000 7f5c5e200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c58156f10 con 0x7f5c5806f2c0 2024-05-30T16:43:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.527+0000 7f5c5e200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5806dda0 con 0x7f5c5806ee40 2024-05-30T16:43:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.527+0000 7f5c5e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5806e080 con 0x7f5c58156b30 2024-05-30T16:43:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.527+0000 7f5c5d800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5c5806f2c0 0x7f5c580b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35224/0 (socket says 172.21.15.67:35224) 2024-05-30T16:43:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.527+0000 7f5c5d800640 1 -- 172.21.15.67:0/2981830397 learned_addr learned my addr 172.21.15.67:0/2981830397 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57e00640 1 --2- 172.21.15.67:0/2981830397 >> v2:172.21.15.169:3300/0 conn(0x7f5c5806ee40 0x7f5c580ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c5ce00640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c58156b30 0x7f5c580af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615020213 0 0) 0x7f5c58156f10 con 0x7f5c5806f2c0 2024-05-30T16:43:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c40003520 con 0x7f5c5806f2c0 2024-05-30T16:43:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1094805623 0 0) 0x7f5c40003520 con 0x7f5c5806f2c0 2024-05-30T16:43:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c58156b30 msgr2=0x7f5c580af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c58156b30 0x7f5c580af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 >> v2:172.21.15.169:3300/0 conn(0x7f5c5806ee40 msgr2=0x7f5c580ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 --2- 172.21.15.67:0/2981830397 >> v2:172.21.15.169:3300/0 conn(0x7f5c5806ee40 0x7f5c580ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5806e360 con 0x7f5c5806f2c0 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5c500019f0 con 0x7f5c5806f2c0 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 >> v1:172.21.15.67:6789/0 conn(0x7f5c5806f2c0 legacy=0x7f5c580b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c58156b30 0x7f5c400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c40003f80 0x7f5c40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c40004d40 0x7f5c400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5806dda0 con 0x7f5c40004d40 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5806e080 con 0x7f5c40003f80 2024-05-30T16:43:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.528+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5c4000ba70 con 0x7f5c58156b30 2024-05-30T16:43:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.529+0000 7f5c5d800640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c40004d40 0x7f5c400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.529+0000 7f5c5ce00640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c58156b30 0x7f5c400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.529+0000 7f5c57e00640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c40003f80 0x7f5c40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.529+0000 7f5c5d800640 1 -- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c58156b30 msgr2=0x7f5c400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.529+0000 7f5c5d800640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c58156b30 0x7f5c400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.529+0000 7f5c5d800640 1 -- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c40003f80 msgr2=0x7f5c40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.529+0000 7f5c5d800640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c40003f80 0x7f5c40004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.529+0000 7f5c5d800640 1 -- 172.21.15.67:0/2981830397 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c40003520 con 0x7f5c40004d40 2024-05-30T16:43:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.530+0000 7f5c5d800640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c40004d40 0x7f5c400051c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c500013d0 tx=0x7f5c40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe0b2b33268f885 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.530+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000a410 con 0x7f5c40004d40 2024-05-30T16:43:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.530+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c5002f070 con 0x7f5c40004d40 2024-05-30T16:43:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.530+0000 7f5c57400640 1 -- 172.21.15.67:0/2981830397 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5002bc00 con 0x7f5c40004d40 2024-05-30T16:43:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.531+0000 7f5c5e200640 1 -- 172.21.15.67:0/2981830397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c40004d40 msgr2=0x7f5c400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.531+0000 7f5c5e200640 1 --2- 172.21.15.67:0/2981830397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c40004d40 0x7f5c400051c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c500013d0 tx=0x7f5c40005830 comp rx=0 tx=0).stop 2024-05-30T16:43:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.531+0000 7f5c5d800640 1 -- 172.21.15.67:0/2981830397 reap_dead start 2024-05-30T16:43:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.531+0000 7f5c5e200640 1 -- 172.21.15.67:0/2981830397 shutdown_connections 2024-05-30T16:43:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.531+0000 7f5c5e200640 1 -- 172.21.15.67:0/2981830397 >> 172.21.15.67:0/2981830397 conn(0x7f5c580b1fc0 msgr2=0x7f5c580b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.531+0000 7f5c5e200640 1 -- 172.21.15.67:0/2981830397 shutdown_connections 2024-05-30T16:43:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.531+0000 7f5c5e200640 1 -- 172.21.15.67:0/2981830397 wait complete. 2024-05-30T16:43:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.533+0000 7f5c5e200640 1 Processor -- start 2024-05-30T16:43:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.533+0000 7f5c5e200640 1 -- start start 2024-05-30T16:43:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.534+0000 7f5c5e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c40004d40 0x7f5c58153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.534+0000 7f5c5e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5815d310 0x7f5c5815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.534+0000 7f5c5e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c581f7770 0x7f5c581f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.534+0000 7f5c5e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5c580b2db0 con 0x7f5c581f7770 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.534+0000 7f5c5e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5c580b2c30 con 0x7f5c5815d310 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.534+0000 7f5c5e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5c580b39d0 con 0x7f5c40004d40 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.534+0000 7f5c5d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c581f7770 0x7f5c581f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.534+0000 7f5c5d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c581f7770 0x7f5c581f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43444/0 (socket says 172.21.15.67:43444) 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.534+0000 7f5c5d800640 1 -- 172.21.15.67:0/1217390408 learned_addr learned my addr 172.21.15.67:0/1217390408 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c57e00640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5815d310 0x7f5c5815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c5d800640 1 -- 172.21.15.67:0/1217390408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c40004d40 msgr2=0x7f5c58153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c5d800640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c40004d40 0x7f5c58153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c5d800640 1 -- 172.21.15.67:0/1217390408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5815d310 msgr2=0x7f5c5815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c5d800640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5815d310 0x7f5c5815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c5d800640 1 -- 172.21.15.67:0/1217390408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5815ddd0 con 0x7f5c581f7770 2024-05-30T16:43:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c57e00640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5815d310 0x7f5c5815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c5d800640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c581f7770 0x7f5c581f7c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c5002b470 tx=0x7f5c50001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c54c00640 1 -- 172.21.15.67:0/1217390408 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50001f70 con 0x7f5c581f7770 2024-05-30T16:43:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c54c00640 1 -- 172.21.15.67:0/1217390408 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c5000a3f0 con 0x7f5c581f7770 2024-05-30T16:43:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c54c00640 1 -- 172.21.15.67:0/1217390408 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5002cb00 con 0x7f5c581f7770 2024-05-30T16:43:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.535+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5815e0b0 con 0x7f5c581f7770 2024-05-30T16:43:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.536+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c581f53e0 con 0x7f5c581f7770 2024-05-30T16:43:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.537+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c580689e0 con 0x7f5c581f7770 2024-05-30T16:43:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.538+0000 7f5c54c00640 1 -- 172.21.15.67:0/1217390408 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c5002cca0 con 0x7f5c581f7770 2024-05-30T16:43:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.539+0000 7f5c54c00640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5c3806c970 0x7f5c3806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.539+0000 7f5c54c00640 1 -- 172.21.15.67:0/1217390408 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5c500abf00 con 0x7f5c581f7770 2024-05-30T16:43:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.540+0000 7f5c5ce00640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5c3806c970 0x7f5c3806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.540+0000 7f5c5ce00640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5c3806c970 0x7f5c3806ee30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009910 tx=0x7f5c4c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.544+0000 7f5c54c00640 1 -- 172.21.15.67:0/1217390408 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c5007a310 con 0x7f5c581f7770 2024-05-30T16:43:53.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.748+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c580cfce0 con 0x7f5c581f7770 2024-05-30T16:43:53.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.751+0000 7f5c54c00640 1 -- 172.21.15.67:0/1217390408 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16606 (secure 0 0 0) 0x7f5c50079c10 con 0x7f5c581f7770 2024-05-30T16:43:53.752 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:53.752 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:52.308492+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:43:50.496509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-30T16:43:51.276872+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:43:53.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.753+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5c3806c970 msgr2=0x7f5c3806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.753+0000 7f5c5e200640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5c3806c970 0x7f5c3806ee30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009910 tx=0x7f5c4c008040 comp rx=0 tx=0).stop 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.753+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c581f7770 msgr2=0x7f5c581f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.753+0000 7f5c5e200640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c581f7770 0x7f5c581f7c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c5002b470 tx=0x7f5c50001360 comp rx=0 tx=0).stop 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.754+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 shutdown_connections 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.754+0000 7f5c5e200640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5c3806c970 0x7f5c3806ee30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.754+0000 7f5c5e200640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c581f7770 0x7f5c581f7c20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.754+0000 7f5c5e200640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c5815d310 0x7f5c5815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.754+0000 7f5c5e200640 1 --2- 172.21.15.67:0/1217390408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c40004d40 0x7f5c58153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.754+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 >> 172.21.15.67:0/1217390408 conn(0x7f5c580b1fc0 msgr2=0x7f5c580b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.754+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 shutdown_connections 2024-05-30T16:43:53.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.754+0000 7f5c5e200640 1 -- 172.21.15.67:0/1217390408 wait complete. 2024-05-30T16:43:53.774 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-30T16:43:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.988+0000 7f76e5600640 1 Processor -- start 2024-05-30T16:43:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.988+0000 7f76e5600640 1 -- start start 2024-05-30T16:43:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.990+0000 7f76e5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e015d270 0x7f76e00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.990+0000 7f76e5600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e015d900 con 0x7f76e0153510 2024-05-30T16:43:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.990+0000 7f76e5600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e00b3f90 con 0x7f76e00cd2a0 2024-05-30T16:43:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.990+0000 7f76e5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76e01538f0 con 0x7f76e015d270 2024-05-30T16:43:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.990+0000 7f76dfe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f76e0153510 0x7f76e00ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35226/0 (socket says 172.21.15.67:35226) 2024-05-30T16:43:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.990+0000 7f76dfe00640 1 -- 172.21.15.67:0/1954472206 learned_addr learned my addr 172.21.15.67:0/1954472206 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.990+0000 7f76e4c00640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e015d270 0x7f76e00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.990+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2810116872 0 0) 0x7f76e015d900 con 0x7f76e0153510 2024-05-30T16:43:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.990+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76c4003520 con 0x7f76e0153510 2024-05-30T16:43:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3675274498 0 0) 0x7f76c4003520 con 0x7f76e0153510 2024-05-30T16:43:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e015d270 msgr2=0x7f76e00b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e015d270 0x7f76e00b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 >> v1:172.21.15.169:6789/0 conn(0x7f76e00cd2a0 legacy=0x7f76e015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e00b4270 con 0x7f76e0153510 2024-05-30T16:43:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f76d0001670 con 0x7f76e0153510 2024-05-30T16:43:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 >> v1:172.21.15.67:6789/0 conn(0x7f76e0153510 legacy=0x7f76e00ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40039e0 0x7f76c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76c4004800 0x7f76c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e015d270 0x7f76c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76c40054f0 con 0x7f76c4004800 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76c400bf00 con 0x7f76c40039e0 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76c400c080 con 0x7f76e015d270 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76df400640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76c4004800 0x7f76c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76e4c00640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e015d270 0x7f76c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76dfe00640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40039e0 0x7f76c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76df400640 1 -- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e015d270 msgr2=0x7f76c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76df400640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e015d270 0x7f76c400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.991+0000 7f76df400640 1 -- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40039e0 msgr2=0x7f76c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.992+0000 7f76df400640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40039e0 0x7f76c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.992+0000 7f76df400640 1 -- 172.21.15.67:0/1954472206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c4003520 con 0x7f76c4004800 2024-05-30T16:43:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.992+0000 7f76dfe00640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c40039e0 0x7f76c4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76df400640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76c4004800 0x7f76c4004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f76cc001750 tx=0x7f76c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283b684007858f11 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc001370 con 0x7f76c4004800 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc0052d0 con 0x7f76c4004800 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76dea00640 1 -- 172.21.15.67:0/1954472206 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc02abe0 con 0x7f76c4004800 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76e5600640 1 -- 172.21.15.67:0/1954472206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76c4004800 msgr2=0x7f76c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76e5600640 1 --2- 172.21.15.67:0/1954472206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76c4004800 0x7f76c4004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f76cc001750 tx=0x7f76c4005830 comp rx=0 tx=0).stop 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76e4c00640 1 -- 172.21.15.67:0/1954472206 reap_dead start 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76e5600640 1 -- 172.21.15.67:0/1954472206 shutdown_connections 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76e5600640 1 -- 172.21.15.67:0/1954472206 >> 172.21.15.67:0/1954472206 conn(0x7f76e006b900 msgr2=0x7f76e006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76e5600640 1 -- 172.21.15.67:0/1954472206 shutdown_connections 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.993+0000 7f76e5600640 1 -- 172.21.15.67:0/1954472206 wait complete. 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.995+0000 7f76e5600640 1 Processor -- start 2024-05-30T16:43:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.995+0000 7f76e5600640 1 -- start start 2024-05-30T16:43:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.996+0000 7f76e5600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c4004800 0x7f76e0150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.996+0000 7f76e5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e014c7c0 0x7f76e01e6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.996+0000 7f76e5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e00ae180 0x7f76e00ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.996+0000 7f76e5600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76e00b48c0 con 0x7f76e00ae180 2024-05-30T16:43:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.996+0000 7f76e5600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76e015d6c0 con 0x7f76c4004800 2024-05-30T16:43:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.996+0000 7f76e5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76e00b4d50 con 0x7f76e014c7c0 2024-05-30T16:43:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.996+0000 7f76e4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e00ae180 0x7f76e00ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.996+0000 7f76e4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e00ae180 0x7f76e00ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43460/0 (socket says 172.21.15.67:43460) 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.996+0000 7f76e4c00640 1 -- 172.21.15.67:0/767495300 learned_addr learned my addr 172.21.15.67:0/767495300 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76dfe00640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c4004800 0x7f76e0150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76df400640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e014c7c0 0x7f76e01e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76e4c00640 1 -- 172.21.15.67:0/767495300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e014c7c0 msgr2=0x7f76e01e6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76e4c00640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e014c7c0 0x7f76e01e6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76e4c00640 1 -- 172.21.15.67:0/767495300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c4004800 msgr2=0x7f76e0150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76e4c00640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c4004800 0x7f76e0150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76e4c00640 1 -- 172.21.15.67:0/767495300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e0151580 con 0x7f76e00ae180 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76dfe00640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c4004800 0x7f76e0150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76e4c00640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e00ae180 0x7f76e00ae600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f76e00b3c10 tx=0x7f76d8009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76d7e00640 1 -- 172.21.15.67:0/767495300 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8019070 con 0x7f76e00ae180 2024-05-30T16:43:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76d7e00640 1 -- 172.21.15.67:0/767495300 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d8007d10 con 0x7f76e00ae180 2024-05-30T16:43:54.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e0151860 con 0x7f76e00ae180 2024-05-30T16:43:54.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.997+0000 7f76d7e00640 1 -- 172.21.15.67:0/767495300 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d8014640 con 0x7f76e00ae180 2024-05-30T16:43:54.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.998+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e0151e20 con 0x7f76e00ae180 2024-05-30T16:43:54.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:53.999+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e00aeb40 con 0x7f76e00ae180 2024-05-30T16:43:54.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.000+0000 7f76d7e00640 1 -- 172.21.15.67:0/767495300 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76d801c070 con 0x7f76e00ae180 2024-05-30T16:43:54.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.000+0000 7f76d7e00640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76c006c920 0x7f76c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:54.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.001+0000 7f76d7e00640 1 -- 172.21.15.67:0/767495300 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f76d808b6b0 con 0x7f76e00ae180 2024-05-30T16:43:54.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.001+0000 7f76dfe00640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76c006c920 0x7f76c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:54.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.001+0000 7f76dfe00640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76c006c920 0x7f76c006ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76d00017e0 tx=0x7f76d0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:54.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.005+0000 7f76d7e00640 1 -- 172.21.15.67:0/767495300 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76d8059a40 con 0x7f76e00ae180 2024-05-30T16:43:54.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.253+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f76e00af380 con 0x7f76e00ae180 2024-05-30T16:43:54.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.879+0000 7f76d7e00640 1 -- 172.21.15.67:0/767495300 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f76d8059340 con 0x7f76e00ae180 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.882+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76c006c920 msgr2=0x7f76c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.882+0000 7f76e5600640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76c006c920 0x7f76c006ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76d00017e0 tx=0x7f76d0009040 comp rx=0 tx=0).stop 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.882+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e00ae180 msgr2=0x7f76e00ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.882+0000 7f76e5600640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e00ae180 0x7f76e00ae600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f76e00b3c10 tx=0x7f76d8009d50 comp rx=0 tx=0).stop 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.883+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 shutdown_connections 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.883+0000 7f76e5600640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76c006c920 0x7f76c006ede0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.883+0000 7f76e5600640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76e00ae180 0x7f76e00ae600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.883+0000 7f76e5600640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76e014c7c0 0x7f76e01e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.883+0000 7f76e5600640 1 --2- 172.21.15.67:0/767495300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76c4004800 0x7f76e0150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:54.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.883+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 >> 172.21.15.67:0/767495300 conn(0x7f76e006b900 msgr2=0x7f76e015c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:54.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.883+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 shutdown_connections 2024-05-30T16:43:54.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:54.883+0000 7f76e5600640 1 -- 172.21.15.67:0/767495300 wait complete. 2024-05-30T16:43:54.905 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-30T16:43:54.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-30T16:43:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.112+0000 7f1b05c00640 1 Processor -- start 2024-05-30T16:43:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.112+0000 7f1b05c00640 1 -- start start 2024-05-30T16:43:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.114+0000 7f1b05c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1b000cd2a0 0x7f1b000b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.114+0000 7f1b05c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1b0015cb60 0x7f1b0015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.114+0000 7f1b05c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b0015d900 con 0x7f1b00153510 2024-05-30T16:43:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.114+0000 7f1b05c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1b001538f0 con 0x7f1b000cd2a0 2024-05-30T16:43:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.114+0000 7f1b05c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1b000b48c0 con 0x7f1b0015cb60 2024-05-30T16:43:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.114+0000 7f1aff400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1b000cd2a0 0x7f1b000b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.114+0000 7f1affe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1b00153510 0x7f1b000ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35242/0 (socket says 172.21.15.67:35242) 2024-05-30T16:43:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.114+0000 7f1affe00640 1 -- 172.21.15.67:0/3608059373 learned_addr learned my addr 172.21.15.67:0/3608059373 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.114+0000 7f1b05200640 1 --2- 172.21.15.67:0/3608059373 >> v2:172.21.15.169:3300/0 conn(0x7f1b0015cb60 0x7f1b0015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.115+0000 7f1aff400640 1 -- 172.21.15.67:0/3608059373 >> v2:172.21.15.169:3300/0 conn(0x7f1b0015cb60 msgr2=0x7f1b0015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.115+0000 7f1aff400640 1 --2- 172.21.15.67:0/3608059373 >> v2:172.21.15.169:3300/0 conn(0x7f1b0015cb60 0x7f1b0015cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.115+0000 7f1aff400640 1 -- 172.21.15.67:0/3608059373 >> v1:172.21.15.67:6789/0 conn(0x7f1b00153510 legacy=0x7f1b000ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.115+0000 7f1afea00640 1 -- 172.21.15.67:0/3608059373 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2144545959 0 0) 0x7f1b0015d900 con 0x7f1b00153510 2024-05-30T16:43:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.115+0000 7f1aff400640 1 -- 172.21.15.67:0/3608059373 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b000b3f90 con 0x7f1b000cd2a0 2024-05-30T16:43:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.116+0000 7f1aff400640 1 --2- 172.21.15.67:0/3608059373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 0x7f1b000b4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b000b4a40 tx=0x7f1aec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bb7ba526793ab server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.116+0000 7f1afea00640 1 -- 172.21.15.67:0/3608059373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec037070 con 0x7f1b000cd2a0 2024-05-30T16:43:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.116+0000 7f1afea00640 1 -- 172.21.15.67:0/3608059373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aec032440 con 0x7f1b000cd2a0 2024-05-30T16:43:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.116+0000 7f1afea00640 1 -- 172.21.15.67:0/3608059373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec02ae60 con 0x7f1b000cd2a0 2024-05-30T16:43:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.117+0000 7f1b05c00640 1 -- 172.21.15.67:0/3608059373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 msgr2=0x7f1b000b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.117+0000 7f1b05c00640 1 --2- 172.21.15.67:0/3608059373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 0x7f1b000b4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b000b4a40 tx=0x7f1aec02a110 comp rx=0 tx=0).stop 2024-05-30T16:43:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.117+0000 7f1b05c00640 1 -- 172.21.15.67:0/3608059373 shutdown_connections 2024-05-30T16:43:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.117+0000 7f1b05c00640 1 --2- 172.21.15.67:0/3608059373 >> v2:172.21.15.169:3300/0 conn(0x7f1b0015cb60 0x7f1b0015cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.117+0000 7f1b05c00640 1 --2- 172.21.15.67:0/3608059373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 0x7f1b000b4d50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.117+0000 7f1b05c00640 1 -- 172.21.15.67:0/3608059373 >> 172.21.15.67:0/3608059373 conn(0x7f1b0006b900 msgr2=0x7f1b0006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.117+0000 7f1b05c00640 1 -- 172.21.15.67:0/3608059373 shutdown_connections 2024-05-30T16:43:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.117+0000 7f1b05c00640 1 -- 172.21.15.67:0/3608059373 wait complete. 2024-05-30T16:43:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.119+0000 7f1b05c00640 1 Processor -- start 2024-05-30T16:43:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.119+0000 7f1b05c00640 1 -- start start 2024-05-30T16:43:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.120+0000 7f1b05c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 0x7f1b0015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.120+0000 7f1b05c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b00153510 0x7f1b000ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.120+0000 7f1b05c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0015cb60 0x7f1b00072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.120+0000 7f1b05c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0015d4b0 con 0x7f1b000cd2a0 2024-05-30T16:43:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.120+0000 7f1b05c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1b000b4550 con 0x7f1b0015cb60 2024-05-30T16:43:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.120+0000 7f1b05c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0015d630 con 0x7f1b00153510 2024-05-30T16:43:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.120+0000 7f1affe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 0x7f1b0015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.120+0000 7f1affe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 0x7f1b0015db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43480/0 (socket says 172.21.15.67:43480) 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.120+0000 7f1affe00640 1 -- 172.21.15.67:0/3394709646 learned_addr learned my addr 172.21.15.67:0/3394709646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1b05200640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0015cb60 0x7f1b00072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1aff400640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b00153510 0x7f1b000ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1affe00640 1 -- 172.21.15.67:0/3394709646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b00153510 msgr2=0x7f1b000ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1affe00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b00153510 0x7f1b000ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1affe00640 1 -- 172.21.15.67:0/3394709646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0015cb60 msgr2=0x7f1b00072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1affe00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0015cb60 0x7f1b00072e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1affe00640 1 -- 172.21.15.67:0/3394709646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0015e290 con 0x7f1b000cd2a0 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1b05200640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0015cb60 0x7f1b00072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1affe00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 0x7f1b0015db20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1af000c4d0 tx=0x7f1af000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1af7e00640 1 -- 172.21.15.67:0/3394709646 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0018070 con 0x7f1b000cd2a0 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1af7e00640 1 -- 172.21.15.67:0/3394709646 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af0007500 con 0x7f1b000cd2a0 2024-05-30T16:43:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1af7e00640 1 -- 172.21.15.67:0/3394709646 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0004a20 con 0x7f1b000cd2a0 2024-05-30T16:43:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.121+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0015e570 con 0x7f1b000cd2a0 2024-05-30T16:43:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.122+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0015eb30 con 0x7f1b000cd2a0 2024-05-30T16:43:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.123+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b000689e0 con 0x7f1b000cd2a0 2024-05-30T16:43:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.124+0000 7f1af7e00640 1 -- 172.21.15.67:0/3394709646 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1af0004bc0 con 0x7f1b000cd2a0 2024-05-30T16:43:55.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.125+0000 7f1af7e00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae006c8f0 0x7f1ae006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:55.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.125+0000 7f1af7e00640 1 -- 172.21.15.67:0/3394709646 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1af008adb0 con 0x7f1b000cd2a0 2024-05-30T16:43:55.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.125+0000 7f1aff400640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae006c8f0 0x7f1ae006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:55.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.126+0000 7f1aff400640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae006c8f0 0x7f1ae006edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1aec034000 tx=0x7f1aec02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:55.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.129+0000 7f1af7e00640 1 -- 172.21.15.67:0/3394709646 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1af0059270 con 0x7f1b000cd2a0 2024-05-30T16:43:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.371+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1b000b3c40 con 0x7f1b000cd2a0 2024-05-30T16:43:55.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.882+0000 7f1af7e00640 1 -- 172.21.15.67:0/3394709646 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f1af0058b70 con 0x7f1b000cd2a0 2024-05-30T16:43:55.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.886+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae006c8f0 msgr2=0x7f1ae006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.886+0000 7f1b05c00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae006c8f0 0x7f1ae006edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1aec034000 tx=0x7f1aec02e000 comp rx=0 tx=0).stop 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.886+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 msgr2=0x7f1b0015db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.886+0000 7f1b05c00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 0x7f1b0015db20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1af000c4d0 tx=0x7f1af000c990 comp rx=0 tx=0).stop 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.887+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 shutdown_connections 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.887+0000 7f1b05c00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae006c8f0 0x7f1ae006edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.887+0000 7f1b05c00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0015cb60 0x7f1b00072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.887+0000 7f1b05c00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b00153510 0x7f1b000ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.887+0000 7f1b05c00640 1 --2- 172.21.15.67:0/3394709646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b000cd2a0 0x7f1b0015db20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.887+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 >> 172.21.15.67:0/3394709646 conn(0x7f1b0006b900 msgr2=0x7f1b000ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.887+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 shutdown_connections 2024-05-30T16:43:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:55.887+0000 7f1b05c00640 1 -- 172.21.15.67:0/3394709646 wait complete. 2024-05-30T16:43:55.908 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-30T16:43:55.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:43:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.119+0000 7f7fc1c00640 1 Processor -- start 2024-05-30T16:43:56.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.119+0000 7f7fc1c00640 1 -- start start 2024-05-30T16:43:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.121+0000 7f7fc1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06ee40 0x7f7fbc0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.121+0000 7f7fc1c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7fbc06f2c0 0x7f7fbc0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.121+0000 7f7fc1c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fbc156f10 con 0x7f7fbc156b30 2024-05-30T16:43:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.121+0000 7f7fc1c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc06dda0 con 0x7f7fbc06f2c0 2024-05-30T16:43:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.121+0000 7f7fc1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc06e080 con 0x7f7fbc06ee40 2024-05-30T16:43:56.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.122+0000 7f7fbbe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7fbc156b30 0x7f7fbc0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:53764/0 (socket says 172.21.15.67:53764) 2024-05-30T16:43:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.122+0000 7f7fbbe00640 1 -- 172.21.15.67:0/612665905 learned_addr learned my addr 172.21.15.67:0/612665905 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.122+0000 7f7fb3e00640 1 --2- 172.21.15.67:0/612665905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06ee40 0x7f7fbc0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.122+0000 7f7fc1200640 1 --2- 172.21.15.67:0/612665905 >> v2:172.21.15.169:3300/0 conn(0x7f7fbc06f2c0 0x7f7fbc0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.122+0000 7f7fb3e00640 1 -- 172.21.15.67:0/612665905 >> v2:172.21.15.169:3300/0 conn(0x7f7fbc06f2c0 msgr2=0x7f7fbc0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.122+0000 7f7fb3e00640 1 --2- 172.21.15.67:0/612665905 >> v2:172.21.15.169:3300/0 conn(0x7f7fbc06f2c0 0x7f7fbc0ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.122+0000 7f7fb3e00640 1 -- 172.21.15.67:0/612665905 >> v1:172.21.15.169:6789/0 conn(0x7f7fbc156b30 legacy=0x7f7fbc0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.122+0000 7f7fb3e00640 1 -- 172.21.15.67:0/612665905 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc06e360 con 0x7f7fbc06ee40 2024-05-30T16:43:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.122+0000 7f7fbb400640 1 -- 172.21.15.67:0/612665905 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1825780753 0 0) 0x7f7fbc156f10 con 0x7f7fbc156b30 2024-05-30T16:43:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.124+0000 7f7fb3e00640 1 --2- 172.21.15.67:0/612665905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06ee40 0x7f7fbc0ad230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40097a0 tx=0x7f7fa402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20c3dcf1b2d7e3f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.124+0000 7f7fbb400640 1 -- 172.21.15.67:0/612665905 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4037070 con 0x7f7fbc06ee40 2024-05-30T16:43:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.124+0000 7f7fbb400640 1 -- 172.21.15.67:0/612665905 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa402ae50 con 0x7f7fbc06ee40 2024-05-30T16:43:56.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.124+0000 7f7fbb400640 1 -- 172.21.15.67:0/612665905 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4032660 con 0x7f7fbc06ee40 2024-05-30T16:43:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.125+0000 7f7fc1c00640 1 -- 172.21.15.67:0/612665905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06ee40 msgr2=0x7f7fbc0ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.125+0000 7f7fc1c00640 1 --2- 172.21.15.67:0/612665905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06ee40 0x7f7fbc0ad230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40097a0 tx=0x7f7fa402a100 comp rx=0 tx=0).stop 2024-05-30T16:43:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.125+0000 7f7fc1c00640 1 -- 172.21.15.67:0/612665905 shutdown_connections 2024-05-30T16:43:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.125+0000 7f7fc1c00640 1 --2- 172.21.15.67:0/612665905 >> v2:172.21.15.169:3300/0 conn(0x7f7fbc06f2c0 0x7f7fbc0ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.125+0000 7f7fc1c00640 1 --2- 172.21.15.67:0/612665905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06ee40 0x7f7fbc0ad230 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.125+0000 7f7fc1c00640 1 -- 172.21.15.67:0/612665905 >> 172.21.15.67:0/612665905 conn(0x7f7fbc0b1fc0 msgr2=0x7f7fbc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:56.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.125+0000 7f7fc1c00640 1 -- 172.21.15.67:0/612665905 shutdown_connections 2024-05-30T16:43:56.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.125+0000 7f7fc1c00640 1 -- 172.21.15.67:0/612665905 wait complete. 2024-05-30T16:43:56.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.127+0000 7f7fc1c00640 1 Processor -- start 2024-05-30T16:43:56.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.127+0000 7f7fc1c00640 1 -- start start 2024-05-30T16:43:56.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.128+0000 7f7fc1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc06ee40 0x7f7fbc14c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.128+0000 7f7fc1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06f2c0 0x7f7fbc1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.128+0000 7f7fc1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fbc156b30 0x7f7fbc153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.128+0000 7f7fc1c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0b2db0 con 0x7f7fbc156b30 2024-05-30T16:43:56.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.128+0000 7f7fc1c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0b2c30 con 0x7f7fbc06ee40 2024-05-30T16:43:56.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.128+0000 7f7fc1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc0b32c0 con 0x7f7fbc06f2c0 2024-05-30T16:43:56.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fc1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fbc156b30 0x7f7fbc153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fc1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fbc156b30 0x7f7fbc153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43486/0 (socket says 172.21.15.67:43486) 2024-05-30T16:43:56.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fc1200640 1 -- 172.21.15.67:0/2172834018 learned_addr learned my addr 172.21.15.67:0/2172834018 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:56.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fb3e00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06f2c0 0x7f7fbc1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fbbe00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc06ee40 0x7f7fbc14c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fc1200640 1 -- 172.21.15.67:0/2172834018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06f2c0 msgr2=0x7f7fbc1515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fc1200640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06f2c0 0x7f7fbc1515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fc1200640 1 -- 172.21.15.67:0/2172834018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc06ee40 msgr2=0x7f7fbc14c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fc1200640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc06ee40 0x7f7fbc14c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fc1200640 1 -- 172.21.15.67:0/2172834018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc152470 con 0x7f7fbc156b30 2024-05-30T16:43:56.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.129+0000 7f7fb3e00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06f2c0 0x7f7fbc1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:43:56.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.130+0000 7f7fc1200640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fbc156b30 0x7f7fbc153230 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400efc0 tx=0x7f7fb400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:56.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.130+0000 7f7fb8c00640 1 -- 172.21.15.67:0/2172834018 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4018020 con 0x7f7fbc156b30 2024-05-30T16:43:56.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.130+0000 7f7fb8c00640 1 -- 172.21.15.67:0/2172834018 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb4009b10 con 0x7f7fbc156b30 2024-05-30T16:43:56.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.130+0000 7f7fb8c00640 1 -- 172.21.15.67:0/2172834018 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40146c0 con 0x7f7fbc156b30 2024-05-30T16:43:56.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.130+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc152750 con 0x7f7fbc156b30 2024-05-30T16:43:56.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.130+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc151af0 con 0x7f7fbc156b30 2024-05-30T16:43:56.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.131+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc0689e0 con 0x7f7fbc156b30 2024-05-30T16:43:56.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.132+0000 7f7fb8c00640 1 -- 172.21.15.67:0/2172834018 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fb4005d10 con 0x7f7fbc156b30 2024-05-30T16:43:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.133+0000 7f7fb8c00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f9806c8f0 0x7f7f9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.133+0000 7f7fb8c00640 1 -- 172.21.15.67:0/2172834018 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7fb408c930 con 0x7f7fbc156b30 2024-05-30T16:43:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.133+0000 7f7fbbe00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f9806c8f0 0x7f7f9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.134+0000 7f7fbbe00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f9806c8f0 0x7f7f9806edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fac000f60 tx=0x7f7fac007d00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:56.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.138+0000 7f7fb8c00640 1 -- 172.21.15.67:0/2172834018 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fb4022030 con 0x7f7fbc156b30 2024-05-30T16:43:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.383+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fbc0b52d0 con 0x7f7fbc156b30 2024-05-30T16:43:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.386+0000 7f7fb8c00640 1 -- 172.21.15.67:0/2172834018 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f7fb4010040 con 0x7f7fbc156b30 2024-05-30T16:43:56.387 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:56.387 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2254297887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2254297887},{"type":"v1","addr":"172.21.15.67:6837","nonce":2254297887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/227511509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":227511509},{"type":"v1","addr":"172.21.15.169:6834","nonce":227511509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:43:52.308425+0000","modified":"2024-05-30T16:43:55.372983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2028878495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2028878495},{"type":"v1","addr":"172.21.15.169:6835","nonce":2028878495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-30T16:43:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-05-30T16:43:56.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.388+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f9806c8f0 msgr2=0x7f7f9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.388+0000 7f7fc1c00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f9806c8f0 0x7f7f9806edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fac000f60 tx=0x7f7fac007d00 comp rx=0 tx=0).stop 2024-05-30T16:43:56.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.388+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fbc156b30 msgr2=0x7f7fbc153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.388+0000 7f7fc1c00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fbc156b30 0x7f7fbc153230 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400efc0 tx=0x7f7fb400c490 comp rx=0 tx=0).stop 2024-05-30T16:43:56.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.389+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 shutdown_connections 2024-05-30T16:43:56.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.389+0000 7f7fc1c00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f9806c8f0 0x7f7f9806edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.389+0000 7f7fc1c00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fbc156b30 0x7f7fbc153230 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.389+0000 7f7fc1c00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fbc06f2c0 0x7f7fbc1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.389+0000 7f7fc1c00640 1 --2- 172.21.15.67:0/2172834018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc06ee40 0x7f7fbc14c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.389+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 >> 172.21.15.67:0/2172834018 conn(0x7f7fbc0b1fc0 msgr2=0x7f7fbc15beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:56.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.389+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 shutdown_connections 2024-05-30T16:43:56.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.389+0000 7f7fc1c00640 1 -- 172.21.15.67:0/2172834018 wait complete. 2024-05-30T16:43:56.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:43:56.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.614+0000 7fbd76a00640 1 Processor -- start 2024-05-30T16:43:56.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.615+0000 7fbd76a00640 1 -- start start 2024-05-30T16:43:56.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.616+0000 7fbd76a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbd7006ee40 0x7fbd700ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.616+0000 7fbd76a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd70156f10 con 0x7fbd70156b30 2024-05-30T16:43:56.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.616+0000 7fbd76a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7006dda0 con 0x7fbd7006f2f0 2024-05-30T16:43:56.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.616+0000 7fbd76a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbd7006e080 con 0x7fbd7006ee40 2024-05-30T16:43:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.617+0000 7fbd75600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbd70156b30 0x7fbd700af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35244/0 (socket says 172.21.15.67:35244) 2024-05-30T16:43:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.617+0000 7fbd75600640 1 -- 172.21.15.67:0/51901017 learned_addr learned my addr 172.21.15.67:0/51901017 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.617+0000 7fbd74c00640 1 --2- 172.21.15.67:0/51901017 >> v2:172.21.15.169:3300/0 conn(0x7fbd7006ee40 0x7fbd700ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.617+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 530955609 0 0) 0x7fbd70156f10 con 0x7fbd70156b30 2024-05-30T16:43:56.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.617+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd58003520 con 0x7fbd70156b30 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3343200058 0 0) 0x7fbd58003520 con 0x7fbd70156b30 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 >> v2:172.21.15.169:3300/0 conn(0x7fbd7006ee40 msgr2=0x7fbd700ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 --2- 172.21.15.67:0/51901017 >> v2:172.21.15.169:3300/0 conn(0x7fbd7006ee40 0x7fbd700ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 >> v1:172.21.15.169:6789/0 conn(0x7fbd7006f2f0 legacy=0x7fbd700b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7006e360 con 0x7fbd70156b30 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fbd64001670 con 0x7fbd70156b30 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 >> v1:172.21.15.67:6789/0 conn(0x7fbd70156b30 legacy=0x7fbd700af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd580039b0 0x7fbd58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd58004800 0x7fbd58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd580051c0 0x7fbd5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7006e080 con 0x7fbd58004800 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5800c160 con 0x7fbd580039b0 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5800c360 con 0x7fbd580051c0 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd74c00640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd58004800 0x7fbd58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.618+0000 7fbd75600640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd580039b0 0x7fbd58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.619+0000 7fbd74c00640 1 -- 172.21.15.67:0/51901017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd580051c0 msgr2=0x7fbd5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.619+0000 7fbd76000640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd580051c0 0x7fbd5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.619+0000 7fbd74c00640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd580051c0 0x7fbd5800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.619+0000 7fbd74c00640 1 -- 172.21.15.67:0/51901017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd580039b0 msgr2=0x7fbd58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.619+0000 7fbd74c00640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd580039b0 0x7fbd58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.619+0000 7fbd74c00640 1 -- 172.21.15.67:0/51901017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd58003520 con 0x7fbd58004800 2024-05-30T16:43:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.619+0000 7fbd76000640 1 -- 172.21.15.67:0/51901017 reap_dead start 2024-05-30T16:43:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.620+0000 7fbd74c00640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd58004800 0x7fbd58004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009ac0 tx=0x7fbd58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4293afc0d1a19a08 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.620+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c036070 con 0x7fbd58004800 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.620+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd5c031400 con 0x7fbd58004800 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.620+0000 7fbd6fe00640 1 -- 172.21.15.67:0/51901017 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c009690 con 0x7fbd58004800 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.621+0000 7fbd76a00640 1 -- 172.21.15.67:0/51901017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd58004800 msgr2=0x7fbd58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.621+0000 7fbd76a00640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd58004800 0x7fbd58004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009ac0 tx=0x7fbd58005830 comp rx=0 tx=0).stop 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.621+0000 7fbd76a00640 1 -- 172.21.15.67:0/51901017 shutdown_connections 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.621+0000 7fbd76a00640 1 --2- 172.21.15.67:0/51901017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd58004800 0x7fbd58004c80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.621+0000 7fbd76a00640 1 -- 172.21.15.67:0/51901017 >> 172.21.15.67:0/51901017 conn(0x7fbd700b1fc0 msgr2=0x7fbd700b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.621+0000 7fbd76a00640 1 -- 172.21.15.67:0/51901017 shutdown_connections 2024-05-30T16:43:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.621+0000 7fbd76a00640 1 -- 172.21.15.67:0/51901017 wait complete. 2024-05-30T16:43:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.623+0000 7fbd76a00640 1 Processor -- start 2024-05-30T16:43:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.623+0000 7fbd76a00640 1 -- start start 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd58004800 0x7fbd700cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7015f190 0x7fbd70099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd701556f0 0x7fbd700b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd700b2db0 con 0x7fbd701556f0 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd700b2c30 con 0x7fbd7015f190 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd700af6e0 con 0x7fbd58004800 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd701556f0 0x7fbd700b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd701556f0 0x7fbd700b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43504/0 (socket says 172.21.15.67:43504) 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76000640 1 -- 172.21.15.67:0/2682998657 learned_addr learned my addr 172.21.15.67:0/2682998657 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd75600640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd58004800 0x7fbd700cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76000640 1 -- 172.21.15.67:0/2682998657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd58004800 msgr2=0x7fbd700cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76000640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd58004800 0x7fbd700cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd76000640 1 -- 172.21.15.67:0/2682998657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7015f190 msgr2=0x7fbd70099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.624+0000 7fbd74c00640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7015f190 0x7fbd70099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.625+0000 7fbd76000640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7015f190 0x7fbd70099a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.625+0000 7fbd76000640 1 -- 172.21.15.67:0/2682998657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd700b0910 con 0x7fbd701556f0 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.625+0000 7fbd75600640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd58004800 0x7fbd700cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.625+0000 7fbd76000640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd701556f0 0x7fbd700b46a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbd70076c10 tx=0x7fbd6800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.625+0000 7fbd6d600640 1 -- 172.21.15.67:0/2682998657 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68018070 con 0x7fbd701556f0 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.625+0000 7fbd6d600640 1 -- 172.21.15.67:0/2682998657 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd68003d80 con 0x7fbd701556f0 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.625+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd700b0bf0 con 0x7fbd701556f0 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.625+0000 7fbd6d600640 1 -- 172.21.15.67:0/2682998657 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68005270 con 0x7fbd701556f0 2024-05-30T16:43:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.625+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd700b11b0 con 0x7fbd701556f0 2024-05-30T16:43:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.626+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd700689e0 con 0x7fbd701556f0 2024-05-30T16:43:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.627+0000 7fbd6d600640 1 -- 172.21.15.67:0/2682998657 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd6801b070 con 0x7fbd701556f0 2024-05-30T16:43:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.628+0000 7fbd6d600640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5006c8f0 0x7fbd5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.628+0000 7fbd6d600640 1 -- 172.21.15.67:0/2682998657 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbd6808b080 con 0x7fbd701556f0 2024-05-30T16:43:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.629+0000 7fbd75600640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5006c8f0 0x7fbd5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.629+0000 7fbd75600640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5006c8f0 0x7fbd5006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd64001eb0 tx=0x7fbd64000ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.633+0000 7fbd6d600640 1 -- 172.21.15.67:0/2682998657 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd68058e60 con 0x7fbd701556f0 2024-05-30T16:43:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.836+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd7006f7f0 con 0x7fbd701556f0 2024-05-30T16:43:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.840+0000 7fbd6d600640 1 -- 172.21.15.67:0/2682998657 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7fbd6805d160 con 0x7fbd701556f0 2024-05-30T16:43:56.841 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:56.841 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:53.858729+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:43:50.496509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-30T16:43:51.276872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.842+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5006c8f0 msgr2=0x7fbd5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.842+0000 7fbd76a00640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5006c8f0 0x7fbd5006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd64001eb0 tx=0x7fbd64000ed0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.842+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd701556f0 msgr2=0x7fbd700b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.842+0000 7fbd76a00640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd701556f0 0x7fbd700b46a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbd70076c10 tx=0x7fbd6800d9d0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.843+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 shutdown_connections 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.843+0000 7fbd76a00640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5006c8f0 0x7fbd5006edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.843+0000 7fbd76a00640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd701556f0 0x7fbd700b46a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.843+0000 7fbd76a00640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7015f190 0x7fbd70099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.843+0000 7fbd76a00640 1 --2- 172.21.15.67:0/2682998657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd58004800 0x7fbd700cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.843+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 >> 172.21.15.67:0/2682998657 conn(0x7fbd700b1fc0 msgr2=0x7fbd7015b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.843+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 shutdown_connections 2024-05-30T16:43:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:56.843+0000 7fbd76a00640 1 -- 172.21.15.67:0/2682998657 wait complete. 2024-05-30T16:43:56.863 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T16:43:56.864 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:43:57.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.067+0000 7f34d1400640 1 Processor -- start 2024-05-30T16:43:57.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.068+0000 7f34d1400640 1 -- start start 2024-05-30T16:43:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.069+0000 7f34d1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34cc0d16a0 0x7f34cc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.069+0000 7f34d1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34cc0cf870 0x7f34cc0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.069+0000 7f34d1400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34cc15ec50 con 0x7f34cc15e870 2024-05-30T16:43:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.069+0000 7f34d1400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f34cc0d2150 con 0x7f34cc0d16a0 2024-05-30T16:43:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.069+0000 7f34d1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc0cfd20 con 0x7f34cc0cf870 2024-05-30T16:43:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.069+0000 7f34cbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f34cc15e870 0x7f34cc0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35256/0 (socket says 172.21.15.67:35256) 2024-05-30T16:43:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.069+0000 7f34cbe00640 1 -- 172.21.15.67:0/2286303450 learned_addr learned my addr 172.21.15.67:0/2286303450 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.069+0000 7f34cb400640 1 --2- 172.21.15.67:0/2286303450 >> v2:172.21.15.67:3300/0 conn(0x7f34cc0d16a0 0x7f34cc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.070+0000 7f34caa00640 1 -- 172.21.15.67:0/2286303450 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 917614999 0 0) 0x7f34cc15ec50 con 0x7f34cc15e870 2024-05-30T16:43:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.070+0000 7f34d0a00640 1 --2- 172.21.15.67:0/2286303450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34cc0cf870 0x7f34cc0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.070+0000 7f34caa00640 1 -- 172.21.15.67:0/2286303450 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34b0003520 con 0x7f34cc15e870 2024-05-30T16:43:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.070+0000 7f34cb400640 1 -- 172.21.15.67:0/2286303450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34cc0cf870 msgr2=0x7f34cc0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.070+0000 7f34cb400640 1 --2- 172.21.15.67:0/2286303450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34cc0cf870 0x7f34cc0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.070+0000 7f34cb400640 1 -- 172.21.15.67:0/2286303450 >> v1:172.21.15.67:6789/0 conn(0x7f34cc15e870 legacy=0x7f34cc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.070+0000 7f34cb400640 1 -- 172.21.15.67:0/2286303450 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc0cff80 con 0x7f34cc0d16a0 2024-05-30T16:43:57.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.071+0000 7f34cb400640 1 --2- 172.21.15.67:0/2286303450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0d16a0 0x7f34cc0cf330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009790 tx=0x7f34b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada1af266d6013ec server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.071+0000 7f34caa00640 1 -- 172.21.15.67:0/2286303450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8037070 con 0x7f34cc0d16a0 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.071+0000 7f34caa00640 1 -- 172.21.15.67:0/2286303450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b8032440 con 0x7f34cc0d16a0 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.071+0000 7f34caa00640 1 -- 172.21.15.67:0/2286303450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8032810 con 0x7f34cc0d16a0 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.072+0000 7f34d1400640 1 -- 172.21.15.67:0/2286303450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0d16a0 msgr2=0x7f34cc0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.072+0000 7f34d1400640 1 --2- 172.21.15.67:0/2286303450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0d16a0 0x7f34cc0cf330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009790 tx=0x7f34b802a180 comp rx=0 tx=0).stop 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.072+0000 7f34d1400640 1 -- 172.21.15.67:0/2286303450 shutdown_connections 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.072+0000 7f34d1400640 1 --2- 172.21.15.67:0/2286303450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34cc0cf870 0x7f34cc0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.072+0000 7f34d1400640 1 --2- 172.21.15.67:0/2286303450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0d16a0 0x7f34cc0cf330 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.072+0000 7f34d1400640 1 -- 172.21.15.67:0/2286303450 >> 172.21.15.67:0/2286303450 conn(0x7f34cc08bfc0 msgr2=0x7f34cc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.072+0000 7f34d1400640 1 -- 172.21.15.67:0/2286303450 shutdown_connections 2024-05-30T16:43:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.072+0000 7f34d1400640 1 -- 172.21.15.67:0/2286303450 wait complete. 2024-05-30T16:43:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.074+0000 7f34d1400640 1 Processor -- start 2024-05-30T16:43:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.074+0000 7f34d1400640 1 -- start start 2024-05-30T16:43:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.075+0000 7f34d1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0cf870 0x7f34cc1f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.075+0000 7f34d1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34cc0d16a0 0x7f34cc0cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.075+0000 7f34d1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc15e870 0x7f34cc0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.075+0000 7f34d1400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0d0820 con 0x7f34cc0cf870 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.075+0000 7f34d1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0d0540 con 0x7f34cc15e870 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.075+0000 7f34d1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc0d4fb0 con 0x7f34cc0d16a0 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.075+0000 7f34cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0cf870 0x7f34cc1f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.075+0000 7f34cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0cf870 0x7f34cc1f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43516/0 (socket says 172.21.15.67:43516) 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.075+0000 7f34cbe00640 1 -- 172.21.15.67:0/3042582364 learned_addr learned my addr 172.21.15.67:0/3042582364 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34cbe00640 1 -- 172.21.15.67:0/3042582364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34cc0d16a0 msgr2=0x7f34cc0cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34cbe00640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34cc0d16a0 0x7f34cc0cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34cbe00640 1 -- 172.21.15.67:0/3042582364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc15e870 msgr2=0x7f34cc0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34d0a00640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc15e870 0x7f34cc0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34cbe00640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc15e870 0x7f34cc0ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34cbe00640 1 -- 172.21.15.67:0/3042582364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc14b600 con 0x7f34cc0cf870 2024-05-30T16:43:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34cbe00640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0cf870 0x7f34cc1f42d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00c390 tx=0x7f34bc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34c3e00640 1 -- 172.21.15.67:0/3042582364 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc017070 con 0x7f34cc0cf870 2024-05-30T16:43:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34c3e00640 1 -- 172.21.15.67:0/3042582364 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc005ce0 con 0x7f34cc0cf870 2024-05-30T16:43:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34c3e00640 1 -- 172.21.15.67:0/3042582364 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0072f0 con 0x7f34cc0cf870 2024-05-30T16:43:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc1f4950 con 0x7f34cc0cf870 2024-05-30T16:43:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.076+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc1f4f10 con 0x7f34cc0cf870 2024-05-30T16:43:57.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.078+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc0689e0 con 0x7f34cc0cf870 2024-05-30T16:43:57.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.079+0000 7f34c3e00640 1 -- 172.21.15.67:0/3042582364 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34bc01a070 con 0x7f34cc0cf870 2024-05-30T16:43:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.080+0000 7f34c3e00640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34ac06c8f0 0x7f34ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.080+0000 7f34c3e00640 1 -- 172.21.15.67:0/3042582364 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f34bc08aff0 con 0x7f34cc0cf870 2024-05-30T16:43:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.080+0000 7f34cb400640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34ac06c8f0 0x7f34ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.081+0000 7f34cb400640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34ac06c8f0 0x7f34ac06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f34b8034000 tx=0x7f34b802c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:57.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.084+0000 7f34c3e00640 1 -- 172.21.15.67:0/3042582364 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34bc0593a0 con 0x7f34cc0cf870 2024-05-30T16:43:57.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.283+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34cc0ba500 con 0x7f34ac06c8f0 2024-05-30T16:43:57.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.297+0000 7f34c3e00640 1 -- 172.21.15.67:0/3042582364 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f34cc0ba500 con 0x7f34ac06c8f0 2024-05-30T16:43:57.301 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:57.303 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:43:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34ac06c8f0 msgr2=0x7f34ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34ac06c8f0 0x7f34ac06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f34b8034000 tx=0x7f34b802c000 comp rx=0 tx=0).stop 2024-05-30T16:43:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0cf870 msgr2=0x7f34cc1f42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0cf870 0x7f34cc1f42d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00c390 tx=0x7f34bc00c850 comp rx=0 tx=0).stop 2024-05-30T16:43:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 shutdown_connections 2024-05-30T16:43:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34ac06c8f0 0x7f34ac06edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc15e870 0x7f34cc0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34cc0d16a0 0x7f34cc0cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 --2- 172.21.15.67:0/3042582364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34cc0cf870 0x7f34cc1f42d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 >> 172.21.15.67:0/3042582364 conn(0x7f34cc08bfc0 msgr2=0x7f34cc0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 shutdown_connections 2024-05-30T16:43:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.302+0000 7f34d1400640 1 -- 172.21.15.67:0/3042582364 wait complete. 2024-05-30T16:43:57.322 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-30T16:43:55.975154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2892,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2961408,"data_stored":1771752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.164157"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.900236+0000","last_change":"2024-05-30T16:43:52.894258+0000","last_active":"2024-05-30T16:43:52.900236+0000","last_peered":"2024-05-30T16:43:52.900236+0000","last_clean":"2024-05-30T16:43:52.900236+0000","last_became_active":"2024-05-30T16:43:52.893919+0000","last_became_peered":"2024-05-30T16:43:52.893919+0000","last_unstale":"2024-05-30T16:43:52.900236+0000","last_undegraded":"2024-05-30T16:43:52.900236+0000","last_fullsized":"2024-05-30T16:43:52.900236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:50.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872293+0000","last_change":"2024-05-30T16:43:52.878535+0000","last_active":"2024-05-30T16:43:53.872293+0000","last_peered":"2024-05-30T16:43:53.872293+0000","last_clean":"2024-05-30T16:43:53.872293+0000","last_became_active":"2024-05-30T16:43:52.878243+0000","last_became_peered":"2024-05-30T16:43:52.878243+0000","last_unstale":"2024-05-30T16:43:53.872293+0000","last_undegraded":"2024-05-30T16:43:53.872293+0000","last_fullsized":"2024-05-30T16:43:53.872293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.881315+0000","last_change":"2024-05-30T16:43:52.874827+0000","last_active":"2024-05-30T16:43:52.881315+0000","last_peered":"2024-05-30T16:43:52.881315+0000","last_clean":"2024-05-30T16:43:52.881315+0000","last_became_active":"2024-05-30T16:43:52.874483+0000","last_became_peered":"2024-05-30T16:43:52.874483+0000","last_unstale":"2024-05-30T16:43:52.881315+0000","last_undegraded":"2024-05-30T16:43:52.881315+0000","last_fullsized":"2024-05-30T16:43:52.881315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:32:38.162415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875633+0000","last_change":"2024-05-30T16:43:52.884775+0000","last_active":"2024-05-30T16:43:53.875633+0000","last_peered":"2024-05-30T16:43:53.875633+0000","last_clean":"2024-05-30T16:43:53.875633+0000","last_became_active":"2024-05-30T16:43:52.884436+0000","last_became_peered":"2024-05-30T16:43:52.884436+0000","last_unstale":"2024-05-30T16:43:53.875633+0000","last_undegraded":"2024-05-30T16:43:53.875633+0000","last_fullsized":"2024-05-30T16:43:53.875633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:25.275803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.889535+0000","last_change":"2024-05-30T16:43:52.886555+0000","last_active":"2024-05-30T16:43:52.889535+0000","last_peered":"2024-05-30T16:43:52.889535+0000","last_clean":"2024-05-30T16:43:52.889535+0000","last_became_active":"2024-05-30T16:43:52.886203+0000","last_became_peered":"2024-05-30T16:43:52.886203+0000","last_unstale":"2024-05-30T16:43:52.889535+0000","last_undegraded":"2024-05-30T16:43:52.889535+0000","last_fullsized":"2024-05-30T16:43:52.889535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:05.404594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873858+0000","last_change":"2024-05-30T16:43:52.875329+0000","last_active":"2024-05-30T16:43:53.873858+0000","last_peered":"2024-05-30T16:43:53.873858+0000","last_clean":"2024-05-30T16:43:53.873858+0000","last_became_active":"2024-05-30T16:43:52.874980+0000","last_became_peered":"2024-05-30T16:43:52.874980+0000","last_unstale":"2024-05-30T16:43:53.873858+0000","last_undegraded":"2024-05-30T16:43:53.873858+0000","last_fullsized":"2024-05-30T16:43:53.873858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:48.436948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.900080+0000","last_change":"2024-05-30T16:43:52.894028+0000","last_active":"2024-05-30T16:43:52.900080+0000","last_peered":"2024-05-30T16:43:52.900080+0000","last_clean":"2024-05-30T16:43:52.900080+0000","last_became_active":"2024-05-30T16:43:52.889967+0000","last_became_peered":"2024-05-30T16:43:52.889967+0000","last_unstale":"2024-05-30T16:43:52.900080+0000","last_undegraded":"2024-05-30T16:43:52.900080+0000","last_fullsized":"2024-05-30T16:43:52.900080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:07.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.223951+0000","last_change":"2024-05-30T16:43:52.899916+0000","last_active":"2024-05-30T16:43:54.223951+0000","last_peered":"2024-05-30T16:43:54.223951+0000","last_clean":"2024-05-30T16:43:54.223951+0000","last_became_active":"2024-05-30T16:43:52.899619+0000","last_became_peered":"2024-05-30T16:43:52.899619+0000","last_unstale":"2024-05-30T16:43:54.223951+0000","last_undegraded":"2024-05-30T16:43:54.223951+0000","last_fullsized":"2024-05-30T16:43:54.223951+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:30.437464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.904415+0000","last_change":"2024-05-30T16:43:52.903027+0000","last_active":"2024-05-30T16:43:52.904415+0000","last_peered":"2024-05-30T16:43:52.904415+0000","last_clean":"2024-05-30T16:43:52.904415+0000","last_became_active":"2024-05-30T16:43:52.902799+0000","last_became_peered":"2024-05-30T16:43:52.902799+0000","last_unstale":"2024-05-30T16:43:52.904415+0000","last_undegraded":"2024-05-30T16:43:52.904415+0000","last_fullsized":"2024-05-30T16:43:52.904415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:08:57.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873608+0000","last_change":"2024-05-30T16:43:52.895565+0000","last_active":"2024-05-30T16:43:53.873608+0000","last_peered":"2024-05-30T16:43:53.873608+0000","last_clean":"2024-05-30T16:43:53.873608+0000","last_became_active":"2024-05-30T16:43:52.894829+0000","last_became_peered":"2024-05-30T16:43:52.894829+0000","last_unstale":"2024-05-30T16:43:53.873608+0000","last_undegraded":"2024-05-30T16:43:53.873608+0000","last_fullsized":"2024-05-30T16:43:53.873608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873611+0000","last_change":"2024-05-30T16:43:52.899464+0000","last_active":"2024-05-30T16:43:53.873611+0000","last_peered":"2024-05-30T16:43:53.873611+0000","last_clean":"2024-05-30T16:43:53.873611+0000","last_became_active":"2024-05-30T16:43:52.897925+0000","last_became_peered":"2024-05-30T16:43:52.897925+0000","last_unstale":"2024-05-30T16:43:53.873611+0000","last_undegraded":"2024-05-30T16:43:53.873611+0000","last_fullsized":"2024-05-30T16:43:53.873611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226199+0000","last_change":"2024-05-30T16:43:52.877782+0000","last_active":"2024-05-30T16:43:54.226199+0000","last_peered":"2024-05-30T16:43:54.226199+0000","last_clean":"2024-05-30T16:43:54.226199+0000","last_became_active":"2024-05-30T16:43:52.877466+0000","last_became_peered":"2024-05-30T16:43:52.877466+0000","last_unstale":"2024-05-30T16:43:54.226199+0000","last_undegraded":"2024-05-30T16:43:54.226199+0000","last_fullsized":"2024-05-30T16:43:54.226199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:33.066313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225046+0000","last_change":"2024-05-30T16:43:52.881578+0000","last_active":"2024-05-30T16:43:54.225046+0000","last_peered":"2024-05-30T16:43:54.225046+0000","last_clean":"2024-05-30T16:43:54.225046+0000","last_became_active":"2024-05-30T16:43:52.877597+0000","last_became_peered":"2024-05-30T16:43:52.877597+0000","last_unstale":"2024-05-30T16:43:54.225046+0000","last_undegraded":"2024-05-30T16:43:54.225046+0000","last_fullsized":"2024-05-30T16:43:54.225046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.879814+0000","last_change":"2024-05-30T16:43:52.874486+0000","last_active":"2024-05-30T16:43:52.879814+0000","last_peered":"2024-05-30T16:43:52.879814+0000","last_clean":"2024-05-30T16:43:52.879814+0000","last_became_active":"2024-05-30T16:43:52.872863+0000","last_became_peered":"2024-05-30T16:43:52.872863+0000","last_unstale":"2024-05-30T16:43:52.879814+0000","last_undegraded":"2024-05-30T16:43:52.879814+0000","last_fullsized":"2024-05-30T16:43:52.879814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:21:09.947345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872070+0000","last_change":"2024-05-30T16:43:52.877439+0000","last_active":"2024-05-30T16:43:53.872070+0000","last_peered":"2024-05-30T16:43:53.872070+0000","last_clean":"2024-05-30T16:43:53.872070+0000","last_became_active":"2024-05-30T16:43:52.877139+0000","last_became_peered":"2024-05-30T16:43:52.877139+0000","last_unstale":"2024-05-30T16:43:53.872070+0000","last_undegraded":"2024-05-30T16:43:53.872070+0000","last_fullsized":"2024-05-30T16:43:53.872070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873037+0000","last_change":"2024-05-30T16:43:52.872351+0000","last_active":"2024-05-30T16:43:53.873037+0000","last_peered":"2024-05-30T16:43:53.873037+0000","last_clean":"2024-05-30T16:43:53.873037+0000","last_became_active":"2024-05-30T16:43:52.872079+0000","last_became_peered":"2024-05-30T16:43:52.872079+0000","last_unstale":"2024-05-30T16:43:53.873037+0000","last_undegraded":"2024-05-30T16:43:53.873037+0000","last_fullsized":"2024-05-30T16:43:53.873037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:50.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874480+0000","last_change":"2024-05-30T16:43:52.897208+0000","last_active":"2024-05-30T16:43:53.874480+0000","last_peered":"2024-05-30T16:43:53.874480+0000","last_clean":"2024-05-30T16:43:53.874480+0000","last_became_active":"2024-05-30T16:43:52.895700+0000","last_became_peered":"2024-05-30T16:43:52.895700+0000","last_unstale":"2024-05-30T16:43:53.874480+0000","last_undegraded":"2024-05-30T16:43:53.874480+0000","last_fullsized":"2024-05-30T16:43:53.874480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:21.139158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086708+0000","last_change":"2024-05-30T16:43:52.870196+0000","last_active":"2024-05-30T16:43:54.086708+0000","last_peered":"2024-05-30T16:43:54.086708+0000","last_clean":"2024-05-30T16:43:54.086708+0000","last_became_active":"2024-05-30T16:43:52.869879+0000","last_became_peered":"2024-05-30T16:43:52.869879+0000","last_unstale":"2024-05-30T16:43:54.086708+0000","last_undegraded":"2024-05-30T16:43:54.086708+0000","last_fullsized":"2024-05-30T16:43:54.086708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:43.785583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.901088+0000","last_change":"2024-05-30T16:43:52.898388+0000","last_active":"2024-05-30T16:43:52.901088+0000","last_peered":"2024-05-30T16:43:52.901088+0000","last_clean":"2024-05-30T16:43:52.901088+0000","last_became_active":"2024-05-30T16:43:52.898159+0000","last_became_peered":"2024-05-30T16:43:52.898159+0000","last_unstale":"2024-05-30T16:43:52.901088+0000","last_undegraded":"2024-05-30T16:43:52.901088+0000","last_fullsized":"2024-05-30T16:43:52.901088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:31.485813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874369+0000","last_change":"2024-05-30T16:43:52.880715+0000","last_active":"2024-05-30T16:43:53.874369+0000","last_peered":"2024-05-30T16:43:53.874369+0000","last_clean":"2024-05-30T16:43:53.874369+0000","last_became_active":"2024-05-30T16:43:52.880406+0000","last_became_peered":"2024-05-30T16:43:52.880406+0000","last_unstale":"2024-05-30T16:43:53.874369+0000","last_undegraded":"2024-05-30T16:43:53.874369+0000","last_fullsized":"2024-05-30T16:43:53.874369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.898529+0000","last_change":"2024-05-30T16:43:52.895619+0000","last_active":"2024-05-30T16:43:52.898529+0000","last_peered":"2024-05-30T16:43:52.898529+0000","last_clean":"2024-05-30T16:43:52.898529+0000","last_became_active":"2024-05-30T16:43:52.891632+0000","last_became_peered":"2024-05-30T16:43:52.891632+0000","last_unstale":"2024-05-30T16:43:52.898529+0000","last_undegraded":"2024-05-30T16:43:52.898529+0000","last_fullsized":"2024-05-30T16:43:52.898529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:21.348915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873953+0000","last_change":"2024-05-30T16:43:52.872959+0000","last_active":"2024-05-30T16:43:53.873953+0000","last_peered":"2024-05-30T16:43:53.873953+0000","last_clean":"2024-05-30T16:43:53.873953+0000","last_became_active":"2024-05-30T16:43:52.872679+0000","last_became_peered":"2024-05-30T16:43:52.872679+0000","last_unstale":"2024-05-30T16:43:53.873953+0000","last_undegraded":"2024-05-30T16:43:53.873953+0000","last_fullsized":"2024-05-30T16:43:53.873953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:08:57.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225306+0000","last_change":"2024-05-30T16:43:52.885877+0000","last_active":"2024-05-30T16:43:54.225306+0000","last_peered":"2024-05-30T16:43:54.225306+0000","last_clean":"2024-05-30T16:43:54.225306+0000","last_became_active":"2024-05-30T16:43:52.885545+0000","last_became_peered":"2024-05-30T16:43:52.885545+0000","last_unstale":"2024-05-30T16:43:54.225306+0000","last_undegraded":"2024-05-30T16:43:54.225306+0000","last_fullsized":"2024-05-30T16:43:54.225306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:00.933568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873194+0000","last_change":"2024-05-30T16:43:52.895695+0000","last_active":"2024-05-30T16:43:53.873194+0000","last_peered":"2024-05-30T16:43:53.873194+0000","last_clean":"2024-05-30T16:43:53.873194+0000","last_became_active":"2024-05-30T16:43:52.895370+0000","last_became_peered":"2024-05-30T16:43:52.895370+0000","last_unstale":"2024-05-30T16:43:53.873194+0000","last_undegraded":"2024-05-30T16:43:53.873194+0000","last_fullsized":"2024-05-30T16:43:53.873194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:31.485813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224641+0000","last_change":"2024-05-30T16:43:52.883128+0000","last_active":"2024-05-30T16:43:54.224641+0000","last_peered":"2024-05-30T16:43:54.224641+0000","last_clean":"2024-05-30T16:43:54.224641+0000","last_became_active":"2024-05-30T16:43:52.881729+0000","last_became_peered":"2024-05-30T16:43:52.881729+0000","last_unstale":"2024-05-30T16:43:54.224641+0000","last_undegraded":"2024-05-30T16:43:54.224641+0000","last_fullsized":"2024-05-30T16:43:54.224641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873862+0000","last_change":"2024-05-30T16:43:52.876820+0000","last_active":"2024-05-30T16:43:53.873862+0000","last_peered":"2024-05-30T16:43:53.873862+0000","last_clean":"2024-05-30T16:43:53.873862+0000","last_became_active":"2024-05-30T16:43:52.875407+0000","last_became_peered":"2024-05-30T16:43:52.875407+0000","last_unstale":"2024-05-30T16:43:53.873862+0000","last_undegraded":"2024-05-30T16:43:53.873862+0000","last_fullsized":"2024-05-30T16:43:53.873862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871846+0000","last_change":"2024-05-30T16:43:52.883372+0000","last_active":"2024-05-30T16:43:53.871846+0000","last_peered":"2024-05-30T16:43:53.871846+0000","last_clean":"2024-05-30T16:43:53.871846+0000","last_became_active":"2024-05-30T16:43:52.882611+0000","last_became_peered":"2024-05-30T16:43:52.882611+0000","last_unstale":"2024-05-30T16:43:53.871846+0000","last_undegraded":"2024-05-30T16:43:53.871846+0000","last_fullsized":"2024-05-30T16:43:53.871846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224988+0000","last_change":"2024-05-30T16:43:52.882394+0000","last_active":"2024-05-30T16:43:54.224988+0000","last_peered":"2024-05-30T16:43:54.224988+0000","last_clean":"2024-05-30T16:43:54.224988+0000","last_became_active":"2024-05-30T16:43:52.882126+0000","last_became_peered":"2024-05-30T16:43:52.882126+0000","last_unstale":"2024-05-30T16:43:54.224988+0000","last_undegraded":"2024-05-30T16:43:54.224988+0000","last_fullsized":"2024-05-30T16:43:54.224988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.895448+0000","last_change":"2024-05-30T16:43:52.884885+0000","last_active":"2024-05-30T16:43:52.895448+0000","last_peered":"2024-05-30T16:43:52.895448+0000","last_clean":"2024-05-30T16:43:52.895448+0000","last_became_active":"2024-05-30T16:43:52.884414+0000","last_became_peered":"2024-05-30T16:43:52.884414+0000","last_unstale":"2024-05-30T16:43:52.895448+0000","last_undegraded":"2024-05-30T16:43:52.895448+0000","last_fullsized":"2024-05-30T16:43:52.895448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:25.394706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874039+0000","last_change":"2024-05-30T16:43:52.890624+0000","last_active":"2024-05-30T16:43:53.874039+0000","last_peered":"2024-05-30T16:43:53.874039+0000","last_clean":"2024-05-30T16:43:53.874039+0000","last_became_active":"2024-05-30T16:43:52.890269+0000","last_became_peered":"2024-05-30T16:43:52.890269+0000","last_unstale":"2024-05-30T16:43:53.874039+0000","last_undegraded":"2024-05-30T16:43:53.874039+0000","last_fullsized":"2024-05-30T16:43:53.874039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:48.878065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086162+0000","last_change":"2024-05-30T16:43:52.891560+0000","last_active":"2024-05-30T16:43:54.086162+0000","last_peered":"2024-05-30T16:43:54.086162+0000","last_clean":"2024-05-30T16:43:54.086162+0000","last_became_active":"2024-05-30T16:43:52.891194+0000","last_became_peered":"2024-05-30T16:43:52.891194+0000","last_unstale":"2024-05-30T16:43:54.086162+0000","last_undegraded":"2024-05-30T16:43:54.086162+0000","last_fullsized":"2024-05-30T16:43:54.086162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:33.066313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873436+0000","last_change":"2024-05-30T16:43:52.876082+0000","last_active":"2024-05-30T16:43:53.873436+0000","last_peered":"2024-05-30T16:43:53.873436+0000","last_clean":"2024-05-30T16:43:53.873436+0000","last_became_active":"2024-05-30T16:43:52.875792+0000","last_became_peered":"2024-05-30T16:43:52.875792+0000","last_unstale":"2024-05-30T16:43:53.873436+0000","last_undegraded":"2024-05-30T16:43:53.873436+0000","last_fullsized":"2024-05-30T16:43:53.873436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225172+0000","last_change":"2024-05-30T16:43:52.885848+0000","last_active":"2024-05-30T16:43:54.225172+0000","last_peered":"2024-05-30T16:43:54.225172+0000","last_clean":"2024-05-30T16:43:54.225172+0000","last_became_active":"2024-05-30T16:43:52.885532+0000","last_became_peered":"2024-05-30T16:43:52.885532+0000","last_unstale":"2024-05-30T16:43:54.225172+0000","last_undegraded":"2024-05-30T16:43:54.225172+0000","last_fullsized":"2024-05-30T16:43:54.225172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:04.698389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225258+0000","last_change":"2024-05-30T16:43:51.854344+0000","last_active":"2024-05-30T16:43:54.225258+0000","last_peered":"2024-05-30T16:43:54.225258+0000","last_clean":"2024-05-30T16:43:54.225258+0000","last_became_active":"2024-05-30T16:43:51.854081+0000","last_became_peered":"2024-05-30T16:43:51.854081+0000","last_unstale":"2024-05-30T16:43:54.225258+0000","last_undegraded":"2024-05-30T16:43:54.225258+0000","last_fullsized":"2024-05-30T16:43:54.225258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:50.620803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.895423+0000","last_change":"2024-05-30T16:43:52.881003+0000","last_active":"2024-05-30T16:43:52.895423+0000","last_peered":"2024-05-30T16:43:52.895423+0000","last_clean":"2024-05-30T16:43:52.895423+0000","last_became_active":"2024-05-30T16:43:52.880834+0000","last_became_peered":"2024-05-30T16:43:52.880834+0000","last_unstale":"2024-05-30T16:43:52.895423+0000","last_undegraded":"2024-05-30T16:43:52.895423+0000","last_fullsized":"2024-05-30T16:43:52.895423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:23.682583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873330+0000","last_change":"2024-05-30T16:43:51.863107+0000","last_active":"2024-05-30T16:43:53.873330+0000","last_peered":"2024-05-30T16:43:53.873330+0000","last_clean":"2024-05-30T16:43:53.873330+0000","last_became_active":"2024-05-30T16:43:51.855082+0000","last_became_peered":"2024-05-30T16:43:51.855082+0000","last_unstale":"2024-05-30T16:43:53.873330+0000","last_undegraded":"2024-05-30T16:43:53.873330+0000","last_fullsized":"2024-05-30T16:43:53.873330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224700+0000","last_change":"2024-05-30T16:43:51.856427+0000","last_active":"2024-05-30T16:43:54.224700+0000","last_peered":"2024-05-30T16:43:54.224700+0000","last_clean":"2024-05-30T16:43:54.224700+0000","last_became_active":"2024-05-30T16:43:51.856150+0000","last_became_peered":"2024-05-30T16:43:51.856150+0000","last_unstale":"2024-05-30T16:43:54.224700+0000","last_undegraded":"2024-05-30T16:43:54.224700+0000","last_fullsized":"2024-05-30T16:43:54.224700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:47.064682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224116+0000","last_change":"2024-05-30T16:43:52.875668+0000","last_active":"2024-05-30T16:43:54.224116+0000","last_peered":"2024-05-30T16:43:54.224116+0000","last_clean":"2024-05-30T16:43:54.224116+0000","last_became_active":"2024-05-30T16:43:52.875362+0000","last_became_peered":"2024-05-30T16:43:52.875362+0000","last_unstale":"2024-05-30T16:43:54.224116+0000","last_undegraded":"2024-05-30T16:43:54.224116+0000","last_fullsized":"2024-05-30T16:43:54.224116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873259+0000","last_change":"2024-05-30T16:43:51.854040+0000","last_active":"2024-05-30T16:43:53.873259+0000","last_peered":"2024-05-30T16:43:53.873259+0000","last_clean":"2024-05-30T16:43:53.873259+0000","last_became_active":"2024-05-30T16:43:51.852832+0000","last_became_peered":"2024-05-30T16:43:51.852832+0000","last_unstale":"2024-05-30T16:43:53.873259+0000","last_undegraded":"2024-05-30T16:43:53.873259+0000","last_fullsized":"2024-05-30T16:43:53.873259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:24.379706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871843+0000","last_change":"2024-05-30T16:43:52.880882+0000","last_active":"2024-05-30T16:43:53.871843+0000","last_peered":"2024-05-30T16:43:53.871843+0000","last_clean":"2024-05-30T16:43:53.871843+0000","last_became_active":"2024-05-30T16:43:52.880558+0000","last_became_peered":"2024-05-30T16:43:52.880558+0000","last_unstale":"2024-05-30T16:43:53.871843+0000","last_undegraded":"2024-05-30T16:43:53.871843+0000","last_fullsized":"2024-05-30T16:43:53.871843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872979+0000","last_change":"2024-05-30T16:43:51.864315+0000","last_active":"2024-05-30T16:43:53.872979+0000","last_peered":"2024-05-30T16:43:53.872979+0000","last_clean":"2024-05-30T16:43:53.872979+0000","last_became_active":"2024-05-30T16:43:51.855552+0000","last_became_peered":"2024-05-30T16:43:51.855552+0000","last_unstale":"2024-05-30T16:43:53.872979+0000","last_undegraded":"2024-05-30T16:43:53.872979+0000","last_fullsized":"2024-05-30T16:43:53.872979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:03.683389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875985+0000","last_change":"2024-05-30T16:43:52.876866+0000","last_active":"2024-05-30T16:43:53.875985+0000","last_peered":"2024-05-30T16:43:53.875985+0000","last_clean":"2024-05-30T16:43:53.875985+0000","last_became_active":"2024-05-30T16:43:52.876208+0000","last_became_peered":"2024-05-30T16:43:52.876208+0000","last_unstale":"2024-05-30T16:43:53.875985+0000","last_undegraded":"2024-05-30T16:43:53.875985+0000","last_fullsized":"2024-05-30T16:43:53.875985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:35.070892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.876134+0000","last_change":"2024-05-30T16:43:51.857532+0000","last_active":"2024-05-30T16:43:53.876134+0000","last_peered":"2024-05-30T16:43:53.876134+0000","last_clean":"2024-05-30T16:43:53.876134+0000","last_became_active":"2024-05-30T16:43:51.853042+0000","last_became_peered":"2024-05-30T16:43:51.853042+0000","last_unstale":"2024-05-30T16:43:53.876134+0000","last_undegraded":"2024-05-30T16:43:53.876134+0000","last_fullsized":"2024-05-30T16:43:53.876134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:19.738945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.904920+0000","last_change":"2024-05-30T16:43:52.901145+0000","last_active":"2024-05-30T16:43:52.904920+0000","last_peered":"2024-05-30T16:43:52.904920+0000","last_clean":"2024-05-30T16:43:52.904920+0000","last_became_active":"2024-05-30T16:43:52.900449+0000","last_became_peered":"2024-05-30T16:43:52.900449+0000","last_unstale":"2024-05-30T16:43:52.904920+0000","last_undegraded":"2024-05-30T16:43:52.904920+0000","last_fullsized":"2024-05-30T16:43:52.904920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:52.968401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871954+0000","last_change":"2024-05-30T16:43:51.857074+0000","last_active":"2024-05-30T16:43:53.871954+0000","last_peered":"2024-05-30T16:43:53.871954+0000","last_clean":"2024-05-30T16:43:53.871954+0000","last_became_active":"2024-05-30T16:43:51.856790+0000","last_became_peered":"2024-05-30T16:43:51.856790+0000","last_unstale":"2024-05-30T16:43:53.871954+0000","last_undegraded":"2024-05-30T16:43:53.871954+0000","last_fullsized":"2024-05-30T16:43:53.871954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226411+0000","last_change":"2024-05-30T16:43:52.883913+0000","last_active":"2024-05-30T16:43:54.226411+0000","last_peered":"2024-05-30T16:43:54.226411+0000","last_clean":"2024-05-30T16:43:54.226411+0000","last_became_active":"2024-05-30T16:43:52.883608+0000","last_became_peered":"2024-05-30T16:43:52.883608+0000","last_unstale":"2024-05-30T16:43:54.226411+0000","last_undegraded":"2024-05-30T16:43:54.226411+0000","last_fullsized":"2024-05-30T16:43:54.226411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:43.785583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874624+0000","last_change":"2024-05-30T16:43:51.859307+0000","last_active":"2024-05-30T16:43:53.874624+0000","last_peered":"2024-05-30T16:43:53.874624+0000","last_clean":"2024-05-30T16:43:53.874624+0000","last_became_active":"2024-05-30T16:43:51.858858+0000","last_became_peered":"2024-05-30T16:43:51.858858+0000","last_unstale":"2024-05-30T16:43:53.874624+0000","last_undegraded":"2024-05-30T16:43:53.874624+0000","last_fullsized":"2024-05-30T16:43:53.874624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.897427+0000","last_change":"2024-05-30T16:43:52.892760+0000","last_active":"2024-05-30T16:43:52.897427+0000","last_peered":"2024-05-30T16:43:52.897427+0000","last_clean":"2024-05-30T16:43:52.897427+0000","last_became_active":"2024-05-30T16:43:52.892483+0000","last_became_peered":"2024-05-30T16:43:52.892483+0000","last_unstale":"2024-05-30T16:43:52.897427+0000","last_undegraded":"2024-05-30T16:43:52.897427+0000","last_fullsized":"2024-05-30T16:43:52.897427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:57.617195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.898082+0000","last_change":"2024-05-30T16:43:51.849511+0000","last_active":"2024-05-30T16:43:52.898082+0000","last_peered":"2024-05-30T16:43:52.898082+0000","last_clean":"2024-05-30T16:43:52.898082+0000","last_became_active":"2024-05-30T16:43:51.842797+0000","last_became_peered":"2024-05-30T16:43:51.842797+0000","last_unstale":"2024-05-30T16:43:52.898082+0000","last_undegraded":"2024-05-30T16:43:52.898082+0000","last_fullsized":"2024-05-30T16:43:52.898082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:30:25.098938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.890789+0000","last_change":"2024-05-30T16:43:52.887806+0000","last_active":"2024-05-30T16:43:52.890789+0000","last_peered":"2024-05-30T16:43:52.890789+0000","last_clean":"2024-05-30T16:43:52.890789+0000","last_became_active":"2024-05-30T16:43:52.887486+0000","last_became_peered":"2024-05-30T16:43:52.887486+0000","last_unstale":"2024-05-30T16:43:52.890789+0000","last_undegraded":"2024-05-30T16:43:52.890789+0000","last_fullsized":"2024-05-30T16:43:52.890789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:41.040321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874098+0000","last_change":"2024-05-30T16:43:51.857906+0000","last_active":"2024-05-30T16:43:53.874098+0000","last_peered":"2024-05-30T16:43:53.874098+0000","last_clean":"2024-05-30T16:43:53.874098+0000","last_became_active":"2024-05-30T16:43:51.855395+0000","last_became_peered":"2024-05-30T16:43:51.855395+0000","last_unstale":"2024-05-30T16:43:53.874098+0000","last_undegraded":"2024-05-30T16:43:53.874098+0000","last_fullsized":"2024-05-30T16:43:53.874098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:32:37.147415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.892022+0000","last_change":"2024-05-30T16:43:52.888949+0000","last_active":"2024-05-30T16:43:52.892022+0000","last_peered":"2024-05-30T16:43:52.892022+0000","last_clean":"2024-05-30T16:43:52.892022+0000","last_became_active":"2024-05-30T16:43:52.887708+0000","last_became_peered":"2024-05-30T16:43:52.887708+0000","last_unstale":"2024-05-30T16:43:52.892022+0000","last_undegraded":"2024-05-30T16:43:52.892022+0000","last_fullsized":"2024-05-30T16:43:52.892022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:44.852718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873316+0000","last_change":"2024-05-30T16:43:51.856817+0000","last_active":"2024-05-30T16:43:53.873316+0000","last_peered":"2024-05-30T16:43:53.873316+0000","last_clean":"2024-05-30T16:43:53.873316+0000","last_became_active":"2024-05-30T16:43:51.856519+0000","last_became_peered":"2024-05-30T16:43:51.856519+0000","last_unstale":"2024-05-30T16:43:53.873316+0000","last_undegraded":"2024-05-30T16:43:53.873316+0000","last_fullsized":"2024-05-30T16:43:53.873316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873045+0000","last_change":"2024-05-30T16:43:52.874643+0000","last_active":"2024-05-30T16:43:53.873045+0000","last_peered":"2024-05-30T16:43:53.873045+0000","last_clean":"2024-05-30T16:43:53.873045+0000","last_became_active":"2024-05-30T16:43:52.874280+0000","last_became_peered":"2024-05-30T16:43:52.874280+0000","last_unstale":"2024-05-30T16:43:53.873045+0000","last_undegraded":"2024-05-30T16:43:53.873045+0000","last_fullsized":"2024-05-30T16:43:53.873045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086111+0000","last_change":"2024-05-30T16:43:51.851393+0000","last_active":"2024-05-30T16:43:54.086111+0000","last_peered":"2024-05-30T16:43:54.086111+0000","last_clean":"2024-05-30T16:43:54.086111+0000","last_became_active":"2024-05-30T16:43:51.851057+0000","last_became_peered":"2024-05-30T16:43:51.851057+0000","last_unstale":"2024-05-30T16:43:54.086111+0000","last_undegraded":"2024-05-30T16:43:54.086111+0000","last_fullsized":"2024-05-30T16:43:54.086111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:50.620803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.885359+0000","last_change":"2024-05-30T16:43:52.880958+0000","last_active":"2024-05-30T16:43:52.885359+0000","last_peered":"2024-05-30T16:43:52.885359+0000","last_clean":"2024-05-30T16:43:52.885359+0000","last_became_active":"2024-05-30T16:43:52.880536+0000","last_became_peered":"2024-05-30T16:43:52.880536+0000","last_unstale":"2024-05-30T16:43:52.885359+0000","last_undegraded":"2024-05-30T16:43:52.885359+0000","last_fullsized":"2024-05-30T16:43:52.885359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:58.783247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226477+0000","last_change":"2024-05-30T16:43:51.854467+0000","last_active":"2024-05-30T16:43:54.226477+0000","last_peered":"2024-05-30T16:43:54.226477+0000","last_clean":"2024-05-30T16:43:54.226477+0000","last_became_active":"2024-05-30T16:43:51.854104+0000","last_became_peered":"2024-05-30T16:43:51.854104+0000","last_unstale":"2024-05-30T16:43:54.226477+0000","last_undegraded":"2024-05-30T16:43:54.226477+0000","last_fullsized":"2024-05-30T16:43:54.226477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:26.393072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.891956+0000","last_change":"2024-05-30T16:43:52.888958+0000","last_active":"2024-05-30T16:43:52.891956+0000","last_peered":"2024-05-30T16:43:52.891956+0000","last_clean":"2024-05-30T16:43:52.891956+0000","last_became_active":"2024-05-30T16:43:52.886708+0000","last_became_peered":"2024-05-30T16:43:52.886708+0000","last_unstale":"2024-05-30T16:43:52.891956+0000","last_undegraded":"2024-05-30T16:43:52.891956+0000","last_fullsized":"2024-05-30T16:43:52.891956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:27.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086747+0000","last_change":"2024-05-30T16:43:51.859324+0000","last_active":"2024-05-30T16:43:54.086747+0000","last_peered":"2024-05-30T16:43:54.086747+0000","last_clean":"2024-05-30T16:43:54.086747+0000","last_became_active":"2024-05-30T16:43:51.858960+0000","last_became_peered":"2024-05-30T16:43:51.858960+0000","last_unstale":"2024-05-30T16:43:54.086747+0000","last_undegraded":"2024-05-30T16:43:54.086747+0000","last_fullsized":"2024-05-30T16:43:54.086747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.894490+0000","last_change":"2024-05-30T16:43:52.890768+0000","last_active":"2024-05-30T16:43:52.894490+0000","last_peered":"2024-05-30T16:43:52.894490+0000","last_clean":"2024-05-30T16:43:52.894490+0000","last_became_active":"2024-05-30T16:43:52.889014+0000","last_became_peered":"2024-05-30T16:43:52.889014+0000","last_unstale":"2024-05-30T16:43:52.894490+0000","last_undegraded":"2024-05-30T16:43:52.894490+0000","last_fullsized":"2024-05-30T16:43:52.894490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:33:20.915701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872029+0000","last_change":"2024-05-30T16:43:51.854719+0000","last_active":"2024-05-30T16:43:53.872029+0000","last_peered":"2024-05-30T16:43:53.872029+0000","last_clean":"2024-05-30T16:43:53.872029+0000","last_became_active":"2024-05-30T16:43:51.854430+0000","last_became_peered":"2024-05-30T16:43:51.854430+0000","last_unstale":"2024-05-30T16:43:53.872029+0000","last_undegraded":"2024-05-30T16:43:53.872029+0000","last_fullsized":"2024-05-30T16:43:53.872029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:03.683389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875608+0000","last_change":"2024-05-30T16:43:52.884624+0000","last_active":"2024-05-30T16:43:53.875608+0000","last_peered":"2024-05-30T16:43:53.875608+0000","last_clean":"2024-05-30T16:43:53.875608+0000","last_became_active":"2024-05-30T16:43:52.884348+0000","last_became_peered":"2024-05-30T16:43:52.884348+0000","last_unstale":"2024-05-30T16:43:53.875608+0000","last_undegraded":"2024-05-30T16:43:53.875608+0000","last_fullsized":"2024-05-30T16:43:53.875608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:32.373668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086493+0000","last_change":"2024-05-30T16:43:51.853597+0000","last_active":"2024-05-30T16:43:54.086493+0000","last_peered":"2024-05-30T16:43:54.086493+0000","last_clean":"2024-05-30T16:43:54.086493+0000","last_became_active":"2024-05-30T16:43:51.853235+0000","last_became_peered":"2024-05-30T16:43:51.853235+0000","last_unstale":"2024-05-30T16:43:54.086493+0000","last_undegraded":"2024-05-30T16:43:54.086493+0000","last_fullsized":"2024-05-30T16:43:54.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.903841+0000","last_change":"2024-05-30T16:43:52.902358+0000","last_active":"2024-05-30T16:43:52.903841+0000","last_peered":"2024-05-30T16:43:52.903841+0000","last_clean":"2024-05-30T16:43:52.903841+0000","last_became_active":"2024-05-30T16:43:52.902031+0000","last_became_peered":"2024-05-30T16:43:52.902031+0000","last_unstale":"2024-05-30T16:43:52.903841+0000","last_undegraded":"2024-05-30T16:43:52.903841+0000","last_fullsized":"2024-05-30T16:43:52.903841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:48.436948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.862895+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:43:52.862895+0000","last_peered":"2024-05-30T16:43:52.862895+0000","last_clean":"2024-05-30T16:43:52.862895+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:43:52.862895+0000","last_undegraded":"2024-05-30T16:43:52.862895+0000","last_fullsized":"2024-05-30T16:43:52.862895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875313+0000","last_change":"2024-05-30T16:43:51.849559+0000","last_active":"2024-05-30T16:43:53.875313+0000","last_peered":"2024-05-30T16:43:53.875313+0000","last_clean":"2024-05-30T16:43:53.875313+0000","last_became_active":"2024-05-30T16:43:51.848551+0000","last_became_peered":"2024-05-30T16:43:51.848551+0000","last_unstale":"2024-05-30T16:43:53.875313+0000","last_undegraded":"2024-05-30T16:43:53.875313+0000","last_fullsized":"2024-05-30T16:43:53.875313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:22.667583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225407+0000","last_change":"2024-05-30T16:43:52.900074+0000","last_active":"2024-05-30T16:43:54.225407+0000","last_peered":"2024-05-30T16:43:54.225407+0000","last_clean":"2024-05-30T16:43:54.225407+0000","last_became_active":"2024-05-30T16:43:52.899764+0000","last_became_peered":"2024-05-30T16:43:52.899764+0000","last_unstale":"2024-05-30T16:43:54.225407+0000","last_undegraded":"2024-05-30T16:43:54.225407+0000","last_fullsized":"2024-05-30T16:43:54.225407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:44.861208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.856729+0000","last_change":"2024-05-30T16:43:51.861550+0000","last_active":"2024-05-30T16:43:52.856729+0000","last_peered":"2024-05-30T16:43:52.856729+0000","last_clean":"2024-05-30T16:43:52.856729+0000","last_became_active":"2024-05-30T16:43:51.861157+0000","last_became_peered":"2024-05-30T16:43:51.861157+0000","last_unstale":"2024-05-30T16:43:52.856729+0000","last_undegraded":"2024-05-30T16:43:52.856729+0000","last_fullsized":"2024-05-30T16:43:52.856729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:20.333916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.883546+0000","last_change":"2024-05-30T16:43:52.878371+0000","last_active":"2024-05-30T16:43:52.883546+0000","last_peered":"2024-05-30T16:43:52.883546+0000","last_clean":"2024-05-30T16:43:52.883546+0000","last_became_active":"2024-05-30T16:43:52.877880+0000","last_became_peered":"2024-05-30T16:43:52.877880+0000","last_unstale":"2024-05-30T16:43:52.883546+0000","last_undegraded":"2024-05-30T16:43:52.883546+0000","last_fullsized":"2024-05-30T16:43:52.883546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:42.482122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086019+0000","last_change":"2024-05-30T16:43:51.858196+0000","last_active":"2024-05-30T16:43:54.086019+0000","last_peered":"2024-05-30T16:43:54.086019+0000","last_clean":"2024-05-30T16:43:54.086019+0000","last_became_active":"2024-05-30T16:43:51.857752+0000","last_became_peered":"2024-05-30T16:43:51.857752+0000","last_unstale":"2024-05-30T16:43:54.086019+0000","last_undegraded":"2024-05-30T16:43:54.086019+0000","last_fullsized":"2024-05-30T16:43:54.086019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871024+0000","last_change":"2024-05-30T16:43:52.863297+0000","last_active":"2024-05-30T16:43:53.871024+0000","last_peered":"2024-05-30T16:43:53.871024+0000","last_clean":"2024-05-30T16:43:53.871024+0000","last_became_active":"2024-05-30T16:43:52.863048+0000","last_became_peered":"2024-05-30T16:43:52.863048+0000","last_unstale":"2024-05-30T16:43:53.871024+0000","last_undegraded":"2024-05-30T16:43:53.871024+0000","last_fullsized":"2024-05-30T16:43:53.871024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874160+0000","last_change":"2024-05-30T16:43:51.854526+0000","last_active":"2024-05-30T16:43:53.874160+0000","last_peered":"2024-05-30T16:43:53.874160+0000","last_clean":"2024-05-30T16:43:53.874160+0000","last_became_active":"2024-05-30T16:43:51.853002+0000","last_became_peered":"2024-05-30T16:43:51.853002+0000","last_unstale":"2024-05-30T16:43:53.874160+0000","last_undegraded":"2024-05-30T16:43:53.874160+0000","last_fullsized":"2024-05-30T16:43:53.874160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:41.467122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225345+0000","last_change":"2024-05-30T16:43:52.876709+0000","last_active":"2024-05-30T16:43:54.225345+0000","last_peered":"2024-05-30T16:43:54.225345+0000","last_clean":"2024-05-30T16:43:54.225345+0000","last_became_active":"2024-05-30T16:43:52.876309+0000","last_became_peered":"2024-05-30T16:43:52.876309+0000","last_unstale":"2024-05-30T16:43:54.225345+0000","last_undegraded":"2024-05-30T16:43:54.225345+0000","last_fullsized":"2024-05-30T16:43:54.225345+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:35.643416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872135+0000","last_change":"2024-05-30T16:43:51.856730+0000","last_active":"2024-05-30T16:43:53.872135+0000","last_peered":"2024-05-30T16:43:53.872135+0000","last_clean":"2024-05-30T16:43:53.872135+0000","last_became_active":"2024-05-30T16:43:51.856409+0000","last_became_peered":"2024-05-30T16:43:51.856409+0000","last_unstale":"2024-05-30T16:43:53.872135+0000","last_undegraded":"2024-05-30T16:43:53.872135+0000","last_fullsized":"2024-05-30T16:43:53.872135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874822+0000","last_change":"2024-05-30T16:43:52.895760+0000","last_active":"2024-05-30T16:43:53.874822+0000","last_peered":"2024-05-30T16:43:53.874822+0000","last_clean":"2024-05-30T16:43:53.874822+0000","last_became_active":"2024-05-30T16:43:52.895305+0000","last_became_peered":"2024-05-30T16:43:52.895305+0000","last_unstale":"2024-05-30T16:43:53.874822+0000","last_undegraded":"2024-05-30T16:43:53.874822+0000","last_fullsized":"2024-05-30T16:43:53.874822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:03.535096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.881269+0000","last_change":"2024-05-30T16:43:51.864341+0000","last_active":"2024-05-30T16:43:52.881269+0000","last_peered":"2024-05-30T16:43:52.881269+0000","last_clean":"2024-05-30T16:43:52.881269+0000","last_became_active":"2024-05-30T16:43:51.864017+0000","last_became_peered":"2024-05-30T16:43:51.864017+0000","last_unstale":"2024-05-30T16:43:52.881269+0000","last_undegraded":"2024-05-30T16:43:52.881269+0000","last_fullsized":"2024-05-30T16:43:52.881269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:48.659115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871833+0000","last_change":"2024-05-30T16:43:52.865453+0000","last_active":"2024-05-30T16:43:53.871833+0000","last_peered":"2024-05-30T16:43:53.871833+0000","last_clean":"2024-05-30T16:43:53.871833+0000","last_became_active":"2024-05-30T16:43:52.864999+0000","last_became_peered":"2024-05-30T16:43:52.864999+0000","last_unstale":"2024-05-30T16:43:53.871833+0000","last_undegraded":"2024-05-30T16:43:53.871833+0000","last_fullsized":"2024-05-30T16:43:53.871833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.877105+0000","last_change":"2024-05-30T16:43:51.855757+0000","last_active":"2024-05-30T16:43:53.877105+0000","last_peered":"2024-05-30T16:43:53.877105+0000","last_clean":"2024-05-30T16:43:53.877105+0000","last_became_active":"2024-05-30T16:43:51.855051+0000","last_became_peered":"2024-05-30T16:43:51.855051+0000","last_unstale":"2024-05-30T16:43:53.877105+0000","last_undegraded":"2024-05-30T16:43:53.877105+0000","last_fullsized":"2024-05-30T16:43:53.877105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:34.628416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224825+0000","last_change":"2024-05-30T16:43:52.888339+0000","last_active":"2024-05-30T16:43:54.224825+0000","last_peered":"2024-05-30T16:43:54.224825+0000","last_clean":"2024-05-30T16:43:54.224825+0000","last_became_active":"2024-05-30T16:43:52.888104+0000","last_became_peered":"2024-05-30T16:43:52.888104+0000","last_unstale":"2024-05-30T16:43:54.224825+0000","last_undegraded":"2024-05-30T16:43:54.224825+0000","last_fullsized":"2024-05-30T16:43:54.224825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873115+0000","last_change":"2024-05-30T16:43:51.865200+0000","last_active":"2024-05-30T16:43:53.873115+0000","last_peered":"2024-05-30T16:43:53.873115+0000","last_clean":"2024-05-30T16:43:53.873115+0000","last_became_active":"2024-05-30T16:43:51.864864+0000","last_became_peered":"2024-05-30T16:43:51.864864+0000","last_unstale":"2024-05-30T16:43:53.873115+0000","last_undegraded":"2024-05-30T16:43:53.873115+0000","last_fullsized":"2024-05-30T16:43:53.873115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:34.628416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.881360+0000","last_change":"2024-05-30T16:43:52.875686+0000","last_active":"2024-05-30T16:43:52.881360+0000","last_peered":"2024-05-30T16:43:52.881360+0000","last_clean":"2024-05-30T16:43:52.881360+0000","last_became_active":"2024-05-30T16:43:52.875422+0000","last_became_peered":"2024-05-30T16:43:52.875422+0000","last_unstale":"2024-05-30T16:43:52.881360+0000","last_undegraded":"2024-05-30T16:43:52.881360+0000","last_fullsized":"2024-05-30T16:43:52.881360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:20.753945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.856196+0000","last_change":"2024-05-30T16:43:51.863391+0000","last_active":"2024-05-30T16:43:52.856196+0000","last_peered":"2024-05-30T16:43:52.856196+0000","last_clean":"2024-05-30T16:43:52.856196+0000","last_became_active":"2024-05-30T16:43:51.863097+0000","last_became_peered":"2024-05-30T16:43:51.863097+0000","last_unstale":"2024-05-30T16:43:52.856196+0000","last_undegraded":"2024-05-30T16:43:52.856196+0000","last_fullsized":"2024-05-30T16:43:52.856196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:40.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225485+0000","last_change":"2024-05-30T16:43:52.875177+0000","last_active":"2024-05-30T16:43:54.225485+0000","last_peered":"2024-05-30T16:43:54.225485+0000","last_clean":"2024-05-30T16:43:54.225485+0000","last_became_active":"2024-05-30T16:43:52.874601+0000","last_became_peered":"2024-05-30T16:43:52.874601+0000","last_unstale":"2024-05-30T16:43:54.225485+0000","last_undegraded":"2024-05-30T16:43:54.225485+0000","last_fullsized":"2024-05-30T16:43:54.225485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:50.920087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872070+0000","last_change":"2024-05-30T16:43:51.853127+0000","last_active":"2024-05-30T16:43:53.872070+0000","last_peered":"2024-05-30T16:43:53.872070+0000","last_clean":"2024-05-30T16:43:53.872070+0000","last_became_active":"2024-05-30T16:43:51.852860+0000","last_became_peered":"2024-05-30T16:43:51.852860+0000","last_unstale":"2024-05-30T16:43:53.872070+0000","last_undegraded":"2024-05-30T16:43:53.872070+0000","last_fullsized":"2024-05-30T16:43:53.872070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:49.905088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224426+0000","last_change":"2024-05-30T16:43:52.876127+0000","last_active":"2024-05-30T16:43:54.224426+0000","last_peered":"2024-05-30T16:43:54.224426+0000","last_clean":"2024-05-30T16:43:54.224426+0000","last_became_active":"2024-05-30T16:43:52.875763+0000","last_became_peered":"2024-05-30T16:43:52.875763+0000","last_unstale":"2024-05-30T16:43:54.224426+0000","last_undegraded":"2024-05-30T16:43:54.224426+0000","last_fullsized":"2024-05-30T16:43:54.224426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871499+0000","last_change":"2024-05-30T16:43:51.860285+0000","last_active":"2024-05-30T16:43:53.871499+0000","last_peered":"2024-05-30T16:43:53.871499+0000","last_clean":"2024-05-30T16:43:53.871499+0000","last_became_active":"2024-05-30T16:43:51.859565+0000","last_became_peered":"2024-05-30T16:43:51.859565+0000","last_unstale":"2024-05-30T16:43:53.871499+0000","last_undegraded":"2024-05-30T16:43:53.871499+0000","last_fullsized":"2024-05-30T16:43:53.871499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225236+0000","last_change":"2024-05-30T16:43:52.885852+0000","last_active":"2024-05-30T16:43:54.225236+0000","last_peered":"2024-05-30T16:43:54.225236+0000","last_clean":"2024-05-30T16:43:54.225236+0000","last_became_active":"2024-05-30T16:43:52.884228+0000","last_became_peered":"2024-05-30T16:43:52.884228+0000","last_unstale":"2024-05-30T16:43:54.225236+0000","last_undegraded":"2024-05-30T16:43:54.225236+0000","last_fullsized":"2024-05-30T16:43:54.225236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:12:06.956321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.889798+0000","last_change":"2024-05-30T16:43:51.863477+0000","last_active":"2024-05-30T16:43:52.889798+0000","last_peered":"2024-05-30T16:43:52.889798+0000","last_clean":"2024-05-30T16:43:52.889798+0000","last_became_active":"2024-05-30T16:43:51.863178+0000","last_became_peered":"2024-05-30T16:43:51.863178+0000","last_unstale":"2024-05-30T16:43:52.889798+0000","last_undegraded":"2024-05-30T16:43:52.889798+0000","last_fullsized":"2024-05-30T16:43:52.889798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:20:50.371105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873046+0000","last_change":"2024-05-30T16:43:52.880010+0000","last_active":"2024-05-30T16:43:53.873046+0000","last_peered":"2024-05-30T16:43:53.873046+0000","last_clean":"2024-05-30T16:43:53.873046+0000","last_became_active":"2024-05-30T16:43:52.879744+0000","last_became_peered":"2024-05-30T16:43:52.879744+0000","last_unstale":"2024-05-30T16:43:53.873046+0000","last_undegraded":"2024-05-30T16:43:53.873046+0000","last_fullsized":"2024-05-30T16:43:53.873046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:30.437464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874603+0000","last_change":"2024-05-30T16:43:52.878768+0000","last_active":"2024-05-30T16:43:53.874603+0000","last_peered":"2024-05-30T16:43:53.874603+0000","last_clean":"2024-05-30T16:43:53.874603+0000","last_became_active":"2024-05-30T16:43:52.878437+0000","last_became_peered":"2024-05-30T16:43:52.878437+0000","last_unstale":"2024-05-30T16:43:53.874603+0000","last_undegraded":"2024-05-30T16:43:53.874603+0000","last_fullsized":"2024-05-30T16:43:53.874603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:13:13.668843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.902941+0000","last_change":"2024-05-30T16:43:51.860527+0000","last_active":"2024-05-30T16:43:52.902941+0000","last_peered":"2024-05-30T16:43:52.902941+0000","last_clean":"2024-05-30T16:43:52.902941+0000","last_became_active":"2024-05-30T16:43:51.860201+0000","last_became_peered":"2024-05-30T16:43:51.860201+0000","last_unstale":"2024-05-30T16:43:52.902941+0000","last_undegraded":"2024-05-30T16:43:52.902941+0000","last_fullsized":"2024-05-30T16:43:52.902941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:44:14.302309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875458+0000","last_change":"2024-05-30T16:43:52.882046+0000","last_active":"2024-05-30T16:43:53.875458+0000","last_peered":"2024-05-30T16:43:53.875458+0000","last_clean":"2024-05-30T16:43:53.875458+0000","last_became_active":"2024-05-30T16:43:52.881777+0000","last_became_peered":"2024-05-30T16:43:52.881777+0000","last_unstale":"2024-05-30T16:43:53.875458+0000","last_undegraded":"2024-05-30T16:43:53.875458+0000","last_fullsized":"2024-05-30T16:43:53.875458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:05:00.764986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.901491+0000","last_change":"2024-05-30T16:43:51.865821+0000","last_active":"2024-05-30T16:43:52.901491+0000","last_peered":"2024-05-30T16:43:52.901491+0000","last_clean":"2024-05-30T16:43:52.901491+0000","last_became_active":"2024-05-30T16:43:51.865511+0000","last_became_peered":"2024-05-30T16:43:51.865511+0000","last_unstale":"2024-05-30T16:43:52.901491+0000","last_undegraded":"2024-05-30T16:43:52.901491+0000","last_fullsized":"2024-05-30T16:43:52.901491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:51.953401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.904940+0000","last_change":"2024-05-30T16:43:52.901336+0000","last_active":"2024-05-30T16:43:52.904940+0000","last_peered":"2024-05-30T16:43:52.904940+0000","last_clean":"2024-05-30T16:43:52.904940+0000","last_became_active":"2024-05-30T16:43:52.900847+0000","last_became_peered":"2024-05-30T16:43:52.900847+0000","last_unstale":"2024-05-30T16:43:52.904940+0000","last_undegraded":"2024-05-30T16:43:52.904940+0000","last_fullsized":"2024-05-30T16:43:52.904940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:44:15.317309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225470+0000","last_change":"2024-05-30T16:43:51.853049+0000","last_active":"2024-05-30T16:43:54.225470+0000","last_peered":"2024-05-30T16:43:54.225470+0000","last_clean":"2024-05-30T16:43:54.225470+0000","last_became_active":"2024-05-30T16:43:51.852719+0000","last_became_peered":"2024-05-30T16:43:51.852719+0000","last_unstale":"2024-05-30T16:43:54.225470+0000","last_undegraded":"2024-05-30T16:43:54.225470+0000","last_fullsized":"2024-05-30T16:43:54.225470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:15.475651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225448+0000","last_change":"2024-05-30T16:43:52.880495+0000","last_active":"2024-05-30T16:43:54.225448+0000","last_peered":"2024-05-30T16:43:54.225448+0000","last_clean":"2024-05-30T16:43:54.225448+0000","last_became_active":"2024-05-30T16:43:52.879641+0000","last_became_peered":"2024-05-30T16:43:52.879641+0000","last_unstale":"2024-05-30T16:43:54.225448+0000","last_undegraded":"2024-05-30T16:43:54.225448+0000","last_fullsized":"2024-05-30T16:43:54.225448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:36.892645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.902208+0000","last_change":"2024-05-30T16:43:51.859807+0000","last_active":"2024-05-30T16:43:52.902208+0000","last_peered":"2024-05-30T16:43:52.902208+0000","last_clean":"2024-05-30T16:43:52.902208+0000","last_became_active":"2024-05-30T16:43:51.859531+0000","last_became_peered":"2024-05-30T16:43:51.859531+0000","last_unstale":"2024-05-30T16:43:52.902208+0000","last_undegraded":"2024-05-30T16:43:52.902208+0000","last_fullsized":"2024-05-30T16:43:52.902208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:12.533740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":71038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":72524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":77212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":661862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":661918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:43:57.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:43:57.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.576+0000 7f18b1600640 1 Processor -- start 2024-05-30T16:43:57.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.577+0000 7f18b1600640 1 -- start start 2024-05-30T16:43:57.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.578+0000 7f18b1600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f18ac15e870 0x7f18ac0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.578+0000 7f18b1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 0x7f18ac0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.578+0000 7f18b1600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18ac15ec50 con 0x7f18ac0d14d0 2024-05-30T16:43:57.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.578+0000 7f18b1600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f18ac0d2150 con 0x7f18ac15e870 2024-05-30T16:43:57.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.578+0000 7f18b1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f18ac0cff80 con 0x7f18ac0cfa40 2024-05-30T16:43:57.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.579+0000 7f18abe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f18ac15e870 0x7f18ac0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.579+0000 7f18b0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 0x7f18ac0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.579+0000 7f18abe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f18ac15e870 0x7f18ac0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:34664/0 (socket says 172.21.15.67:34664) 2024-05-30T16:43:57.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.579+0000 7f18abe00640 1 -- 172.21.15.67:0/363001448 learned_addr learned my addr 172.21.15.67:0/363001448 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:57.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.579+0000 7f18ab400640 1 --1- 172.21.15.67:0/363001448 >> v1:172.21.15.169:6789/0 conn(0x7f18ac0d14d0 0x7f18ac0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:53770/0 (socket says 172.21.15.67:53770) 2024-05-30T16:43:57.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.580+0000 7f18abe00640 1 -- 172.21.15.67:0/363001448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 msgr2=0x7f18ac0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:57.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.580+0000 7f18abe00640 1 --2- 172.21.15.67:0/363001448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 0x7f18ac0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.580+0000 7f18abe00640 1 -- 172.21.15.67:0/363001448 >> v1:172.21.15.169:6789/0 conn(0x7f18ac0d14d0 legacy=0x7f18ac0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.580+0000 7f18abe00640 1 -- 172.21.15.67:0/363001448 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac0d0260 con 0x7f18ac15e870 2024-05-30T16:43:57.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.580+0000 7f18b0c00640 1 --2- 172.21.15.67:0/363001448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 0x7f18ac0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:57.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.581+0000 7f18abe00640 1 --2- 172.21.15.67:0/363001448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ac15e870 0x7f18ac0d0f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f189c009870 tx=0x7f189c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e05ea60f4a625f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:57.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.582+0000 7f18aaa00640 1 -- 172.21.15.67:0/363001448 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c037070 con 0x7f18ac15e870 2024-05-30T16:43:57.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.582+0000 7f18aaa00640 1 -- 172.21.15.67:0/363001448 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c032440 con 0x7f18ac15e870 2024-05-30T16:43:57.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.582+0000 7f18aaa00640 1 -- 172.21.15.67:0/363001448 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c032810 con 0x7f18ac15e870 2024-05-30T16:43:57.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.583+0000 7f18b1600640 1 -- 172.21.15.67:0/363001448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ac15e870 msgr2=0x7f18ac0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.583+0000 7f18b1600640 1 --2- 172.21.15.67:0/363001448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ac15e870 0x7f18ac0d0f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f189c009870 tx=0x7f189c02a250 comp rx=0 tx=0).stop 2024-05-30T16:43:57.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.583+0000 7f18b1600640 1 -- 172.21.15.67:0/363001448 shutdown_connections 2024-05-30T16:43:57.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.583+0000 7f18b1600640 1 --2- 172.21.15.67:0/363001448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 0x7f18ac0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.583+0000 7f18b1600640 1 --2- 172.21.15.67:0/363001448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ac15e870 0x7f18ac0d0f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.583+0000 7f18b1600640 1 -- 172.21.15.67:0/363001448 >> 172.21.15.67:0/363001448 conn(0x7f18ac08bfc0 msgr2=0x7f18ac0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:57.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.583+0000 7f18b1600640 1 -- 172.21.15.67:0/363001448 shutdown_connections 2024-05-30T16:43:57.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.583+0000 7f18b1600640 1 -- 172.21.15.67:0/363001448 wait complete. 2024-05-30T16:43:57.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.585+0000 7f18b1600640 1 Processor -- start 2024-05-30T16:43:57.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.585+0000 7f18b1600640 1 -- start start 2024-05-30T16:43:57.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.586+0000 7f18b1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 0x7f18ac1612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.586+0000 7f18b1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18ac0d14d0 0x7f18ac14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.586+0000 7f18b1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ac15e870 0x7f18ac0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.586+0000 7f18b1600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac0d4fb0 con 0x7f18ac0d14d0 2024-05-30T16:43:57.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.586+0000 7f18b1600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac0d0820 con 0x7f18ac15e870 2024-05-30T16:43:57.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.586+0000 7f18b1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f18ac0d2b20 con 0x7f18ac0cfa40 2024-05-30T16:43:57.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.586+0000 7f18ab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18ac0d14d0 0x7f18ac14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18ab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18ac0d14d0 0x7f18ac14f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43524/0 (socket says 172.21.15.67:43524) 2024-05-30T16:43:57.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18ab400640 1 -- 172.21.15.67:0/682251865 learned_addr learned my addr 172.21.15.67:0/682251865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:57.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18abe00640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 0x7f18ac1612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18b0c00640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ac15e870 0x7f18ac0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18ab400640 1 -- 172.21.15.67:0/682251865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 msgr2=0x7f18ac1612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18ab400640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 0x7f18ac1612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18ab400640 1 -- 172.21.15.67:0/682251865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ac15e870 msgr2=0x7f18ac0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18ab400640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ac15e870 0x7f18ac0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18ab400640 1 -- 172.21.15.67:0/682251865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac0cc030 con 0x7f18ac0d14d0 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.587+0000 7f18ab400640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18ac0d14d0 0x7f18ac14f960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f189400cbf0 tx=0x7f189400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.588+0000 7f18a3e00640 1 -- 172.21.15.67:0/682251865 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894017070 con 0x7f18ac0d14d0 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.588+0000 7f18a3e00640 1 -- 172.21.15.67:0/682251865 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1894005df0 con 0x7f18ac0d14d0 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.588+0000 7f18a3e00640 1 -- 172.21.15.67:0/682251865 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189400ad60 con 0x7f18ac0d14d0 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.588+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ac161810 con 0x7f18ac0d14d0 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.588+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ac0cc590 con 0x7f18ac0d14d0 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.589+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ac08c490 con 0x7f18ac0d14d0 2024-05-30T16:43:57.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.590+0000 7f18a3e00640 1 -- 172.21.15.67:0/682251865 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f189401a070 con 0x7f18ac0d14d0 2024-05-30T16:43:57.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.591+0000 7f18a3e00640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f188806c8f0 0x7f188806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:57.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.591+0000 7f18a3e00640 1 -- 172.21.15.67:0/682251865 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f189408b670 con 0x7f18ac0d14d0 2024-05-30T16:43:57.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.591+0000 7f18abe00640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f188806c8f0 0x7f188806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:57.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.592+0000 7f18abe00640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f188806c8f0 0x7f188806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f189c034000 tx=0x7f189c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:57.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.595+0000 7f18a3e00640 1 -- 172.21.15.67:0/682251865 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18ac08c490 con 0x7f18ac0d14d0 2024-05-30T16:43:57.794 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-05-30T16:43:57.792+0000 7ffb1d000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T16:43:57.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.799+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18ac0ba500 con 0x7f188806c8f0 2024-05-30T16:43:57.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.818+0000 7f18a3e00640 1 -- 172.21.15.67:0/682251865 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f18ac0ba500 con 0x7f188806c8f0 2024-05-30T16:43:57.820 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:57.823 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:43:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.824+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f188806c8f0 msgr2=0x7f188806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.824+0000 7f18b1600640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f188806c8f0 0x7f188806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f189c034000 tx=0x7f189c02e000 comp rx=0 tx=0).stop 2024-05-30T16:43:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.824+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18ac0d14d0 msgr2=0x7f18ac14f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.824+0000 7f18b1600640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18ac0d14d0 0x7f18ac14f960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f189400cbf0 tx=0x7f189400a430 comp rx=0 tx=0).stop 2024-05-30T16:43:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.825+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 shutdown_connections 2024-05-30T16:43:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.825+0000 7f18b1600640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f188806c8f0 0x7f188806edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.825+0000 7f18b1600640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18ac15e870 0x7f18ac0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.825+0000 7f18b1600640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18ac0d14d0 0x7f18ac14f960 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.825+0000 7f18b1600640 1 --2- 172.21.15.67:0/682251865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18ac0cfa40 0x7f18ac1612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.825+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 >> 172.21.15.67:0/682251865 conn(0x7f18ac08bfc0 msgr2=0x7f18ac0b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.825+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 shutdown_connections 2024-05-30T16:43:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:57.825+0000 7f18b1600640 1 -- 172.21.15.67:0/682251865 wait complete. 2024-05-30T16:43:57.844 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-30T16:43:55.975154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2892,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2961408,"data_stored":1771752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.164157"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.900236+0000","last_change":"2024-05-30T16:43:52.894258+0000","last_active":"2024-05-30T16:43:52.900236+0000","last_peered":"2024-05-30T16:43:52.900236+0000","last_clean":"2024-05-30T16:43:52.900236+0000","last_became_active":"2024-05-30T16:43:52.893919+0000","last_became_peered":"2024-05-30T16:43:52.893919+0000","last_unstale":"2024-05-30T16:43:52.900236+0000","last_undegraded":"2024-05-30T16:43:52.900236+0000","last_fullsized":"2024-05-30T16:43:52.900236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:50.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872293+0000","last_change":"2024-05-30T16:43:52.878535+0000","last_active":"2024-05-30T16:43:53.872293+0000","last_peered":"2024-05-30T16:43:53.872293+0000","last_clean":"2024-05-30T16:43:53.872293+0000","last_became_active":"2024-05-30T16:43:52.878243+0000","last_became_peered":"2024-05-30T16:43:52.878243+0000","last_unstale":"2024-05-30T16:43:53.872293+0000","last_undegraded":"2024-05-30T16:43:53.872293+0000","last_fullsized":"2024-05-30T16:43:53.872293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.881315+0000","last_change":"2024-05-30T16:43:52.874827+0000","last_active":"2024-05-30T16:43:52.881315+0000","last_peered":"2024-05-30T16:43:52.881315+0000","last_clean":"2024-05-30T16:43:52.881315+0000","last_became_active":"2024-05-30T16:43:52.874483+0000","last_became_peered":"2024-05-30T16:43:52.874483+0000","last_unstale":"2024-05-30T16:43:52.881315+0000","last_undegraded":"2024-05-30T16:43:52.881315+0000","last_fullsized":"2024-05-30T16:43:52.881315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:32:38.162415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875633+0000","last_change":"2024-05-30T16:43:52.884775+0000","last_active":"2024-05-30T16:43:53.875633+0000","last_peered":"2024-05-30T16:43:53.875633+0000","last_clean":"2024-05-30T16:43:53.875633+0000","last_became_active":"2024-05-30T16:43:52.884436+0000","last_became_peered":"2024-05-30T16:43:52.884436+0000","last_unstale":"2024-05-30T16:43:53.875633+0000","last_undegraded":"2024-05-30T16:43:53.875633+0000","last_fullsized":"2024-05-30T16:43:53.875633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:25.275803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.889535+0000","last_change":"2024-05-30T16:43:52.886555+0000","last_active":"2024-05-30T16:43:52.889535+0000","last_peered":"2024-05-30T16:43:52.889535+0000","last_clean":"2024-05-30T16:43:52.889535+0000","last_became_active":"2024-05-30T16:43:52.886203+0000","last_became_peered":"2024-05-30T16:43:52.886203+0000","last_unstale":"2024-05-30T16:43:52.889535+0000","last_undegraded":"2024-05-30T16:43:52.889535+0000","last_fullsized":"2024-05-30T16:43:52.889535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:05.404594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873858+0000","last_change":"2024-05-30T16:43:52.875329+0000","last_active":"2024-05-30T16:43:53.873858+0000","last_peered":"2024-05-30T16:43:53.873858+0000","last_clean":"2024-05-30T16:43:53.873858+0000","last_became_active":"2024-05-30T16:43:52.874980+0000","last_became_peered":"2024-05-30T16:43:52.874980+0000","last_unstale":"2024-05-30T16:43:53.873858+0000","last_undegraded":"2024-05-30T16:43:53.873858+0000","last_fullsized":"2024-05-30T16:43:53.873858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:48.436948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.900080+0000","last_change":"2024-05-30T16:43:52.894028+0000","last_active":"2024-05-30T16:43:52.900080+0000","last_peered":"2024-05-30T16:43:52.900080+0000","last_clean":"2024-05-30T16:43:52.900080+0000","last_became_active":"2024-05-30T16:43:52.889967+0000","last_became_peered":"2024-05-30T16:43:52.889967+0000","last_unstale":"2024-05-30T16:43:52.900080+0000","last_undegraded":"2024-05-30T16:43:52.900080+0000","last_fullsized":"2024-05-30T16:43:52.900080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:07.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.223951+0000","last_change":"2024-05-30T16:43:52.899916+0000","last_active":"2024-05-30T16:43:54.223951+0000","last_peered":"2024-05-30T16:43:54.223951+0000","last_clean":"2024-05-30T16:43:54.223951+0000","last_became_active":"2024-05-30T16:43:52.899619+0000","last_became_peered":"2024-05-30T16:43:52.899619+0000","last_unstale":"2024-05-30T16:43:54.223951+0000","last_undegraded":"2024-05-30T16:43:54.223951+0000","last_fullsized":"2024-05-30T16:43:54.223951+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:30.437464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.904415+0000","last_change":"2024-05-30T16:43:52.903027+0000","last_active":"2024-05-30T16:43:52.904415+0000","last_peered":"2024-05-30T16:43:52.904415+0000","last_clean":"2024-05-30T16:43:52.904415+0000","last_became_active":"2024-05-30T16:43:52.902799+0000","last_became_peered":"2024-05-30T16:43:52.902799+0000","last_unstale":"2024-05-30T16:43:52.904415+0000","last_undegraded":"2024-05-30T16:43:52.904415+0000","last_fullsized":"2024-05-30T16:43:52.904415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:08:57.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873608+0000","last_change":"2024-05-30T16:43:52.895565+0000","last_active":"2024-05-30T16:43:53.873608+0000","last_peered":"2024-05-30T16:43:53.873608+0000","last_clean":"2024-05-30T16:43:53.873608+0000","last_became_active":"2024-05-30T16:43:52.894829+0000","last_became_peered":"2024-05-30T16:43:52.894829+0000","last_unstale":"2024-05-30T16:43:53.873608+0000","last_undegraded":"2024-05-30T16:43:53.873608+0000","last_fullsized":"2024-05-30T16:43:53.873608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873611+0000","last_change":"2024-05-30T16:43:52.899464+0000","last_active":"2024-05-30T16:43:53.873611+0000","last_peered":"2024-05-30T16:43:53.873611+0000","last_clean":"2024-05-30T16:43:53.873611+0000","last_became_active":"2024-05-30T16:43:52.897925+0000","last_became_peered":"2024-05-30T16:43:52.897925+0000","last_unstale":"2024-05-30T16:43:53.873611+0000","last_undegraded":"2024-05-30T16:43:53.873611+0000","last_fullsized":"2024-05-30T16:43:53.873611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226199+0000","last_change":"2024-05-30T16:43:52.877782+0000","last_active":"2024-05-30T16:43:54.226199+0000","last_peered":"2024-05-30T16:43:54.226199+0000","last_clean":"2024-05-30T16:43:54.226199+0000","last_became_active":"2024-05-30T16:43:52.877466+0000","last_became_peered":"2024-05-30T16:43:52.877466+0000","last_unstale":"2024-05-30T16:43:54.226199+0000","last_undegraded":"2024-05-30T16:43:54.226199+0000","last_fullsized":"2024-05-30T16:43:54.226199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:33.066313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225046+0000","last_change":"2024-05-30T16:43:52.881578+0000","last_active":"2024-05-30T16:43:54.225046+0000","last_peered":"2024-05-30T16:43:54.225046+0000","last_clean":"2024-05-30T16:43:54.225046+0000","last_became_active":"2024-05-30T16:43:52.877597+0000","last_became_peered":"2024-05-30T16:43:52.877597+0000","last_unstale":"2024-05-30T16:43:54.225046+0000","last_undegraded":"2024-05-30T16:43:54.225046+0000","last_fullsized":"2024-05-30T16:43:54.225046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.879814+0000","last_change":"2024-05-30T16:43:52.874486+0000","last_active":"2024-05-30T16:43:52.879814+0000","last_peered":"2024-05-30T16:43:52.879814+0000","last_clean":"2024-05-30T16:43:52.879814+0000","last_became_active":"2024-05-30T16:43:52.872863+0000","last_became_peered":"2024-05-30T16:43:52.872863+0000","last_unstale":"2024-05-30T16:43:52.879814+0000","last_undegraded":"2024-05-30T16:43:52.879814+0000","last_fullsized":"2024-05-30T16:43:52.879814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:21:09.947345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872070+0000","last_change":"2024-05-30T16:43:52.877439+0000","last_active":"2024-05-30T16:43:53.872070+0000","last_peered":"2024-05-30T16:43:53.872070+0000","last_clean":"2024-05-30T16:43:53.872070+0000","last_became_active":"2024-05-30T16:43:52.877139+0000","last_became_peered":"2024-05-30T16:43:52.877139+0000","last_unstale":"2024-05-30T16:43:53.872070+0000","last_undegraded":"2024-05-30T16:43:53.872070+0000","last_fullsized":"2024-05-30T16:43:53.872070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873037+0000","last_change":"2024-05-30T16:43:52.872351+0000","last_active":"2024-05-30T16:43:53.873037+0000","last_peered":"2024-05-30T16:43:53.873037+0000","last_clean":"2024-05-30T16:43:53.873037+0000","last_became_active":"2024-05-30T16:43:52.872079+0000","last_became_peered":"2024-05-30T16:43:52.872079+0000","last_unstale":"2024-05-30T16:43:53.873037+0000","last_undegraded":"2024-05-30T16:43:53.873037+0000","last_fullsized":"2024-05-30T16:43:53.873037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:50.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874480+0000","last_change":"2024-05-30T16:43:52.897208+0000","last_active":"2024-05-30T16:43:53.874480+0000","last_peered":"2024-05-30T16:43:53.874480+0000","last_clean":"2024-05-30T16:43:53.874480+0000","last_became_active":"2024-05-30T16:43:52.895700+0000","last_became_peered":"2024-05-30T16:43:52.895700+0000","last_unstale":"2024-05-30T16:43:53.874480+0000","last_undegraded":"2024-05-30T16:43:53.874480+0000","last_fullsized":"2024-05-30T16:43:53.874480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:21.139158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086708+0000","last_change":"2024-05-30T16:43:52.870196+0000","last_active":"2024-05-30T16:43:54.086708+0000","last_peered":"2024-05-30T16:43:54.086708+0000","last_clean":"2024-05-30T16:43:54.086708+0000","last_became_active":"2024-05-30T16:43:52.869879+0000","last_became_peered":"2024-05-30T16:43:52.869879+0000","last_unstale":"2024-05-30T16:43:54.086708+0000","last_undegraded":"2024-05-30T16:43:54.086708+0000","last_fullsized":"2024-05-30T16:43:54.086708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:43.785583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.901088+0000","last_change":"2024-05-30T16:43:52.898388+0000","last_active":"2024-05-30T16:43:52.901088+0000","last_peered":"2024-05-30T16:43:52.901088+0000","last_clean":"2024-05-30T16:43:52.901088+0000","last_became_active":"2024-05-30T16:43:52.898159+0000","last_became_peered":"2024-05-30T16:43:52.898159+0000","last_unstale":"2024-05-30T16:43:52.901088+0000","last_undegraded":"2024-05-30T16:43:52.901088+0000","last_fullsized":"2024-05-30T16:43:52.901088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:31.485813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874369+0000","last_change":"2024-05-30T16:43:52.880715+0000","last_active":"2024-05-30T16:43:53.874369+0000","last_peered":"2024-05-30T16:43:53.874369+0000","last_clean":"2024-05-30T16:43:53.874369+0000","last_became_active":"2024-05-30T16:43:52.880406+0000","last_became_peered":"2024-05-30T16:43:52.880406+0000","last_unstale":"2024-05-30T16:43:53.874369+0000","last_undegraded":"2024-05-30T16:43:53.874369+0000","last_fullsized":"2024-05-30T16:43:53.874369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.898529+0000","last_change":"2024-05-30T16:43:52.895619+0000","last_active":"2024-05-30T16:43:52.898529+0000","last_peered":"2024-05-30T16:43:52.898529+0000","last_clean":"2024-05-30T16:43:52.898529+0000","last_became_active":"2024-05-30T16:43:52.891632+0000","last_became_peered":"2024-05-30T16:43:52.891632+0000","last_unstale":"2024-05-30T16:43:52.898529+0000","last_undegraded":"2024-05-30T16:43:52.898529+0000","last_fullsized":"2024-05-30T16:43:52.898529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:21.348915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873953+0000","last_change":"2024-05-30T16:43:52.872959+0000","last_active":"2024-05-30T16:43:53.873953+0000","last_peered":"2024-05-30T16:43:53.873953+0000","last_clean":"2024-05-30T16:43:53.873953+0000","last_became_active":"2024-05-30T16:43:52.872679+0000","last_became_peered":"2024-05-30T16:43:52.872679+0000","last_unstale":"2024-05-30T16:43:53.873953+0000","last_undegraded":"2024-05-30T16:43:53.873953+0000","last_fullsized":"2024-05-30T16:43:53.873953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:08:57.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225306+0000","last_change":"2024-05-30T16:43:52.885877+0000","last_active":"2024-05-30T16:43:54.225306+0000","last_peered":"2024-05-30T16:43:54.225306+0000","last_clean":"2024-05-30T16:43:54.225306+0000","last_became_active":"2024-05-30T16:43:52.885545+0000","last_became_peered":"2024-05-30T16:43:52.885545+0000","last_unstale":"2024-05-30T16:43:54.225306+0000","last_undegraded":"2024-05-30T16:43:54.225306+0000","last_fullsized":"2024-05-30T16:43:54.225306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:00.933568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873194+0000","last_change":"2024-05-30T16:43:52.895695+0000","last_active":"2024-05-30T16:43:53.873194+0000","last_peered":"2024-05-30T16:43:53.873194+0000","last_clean":"2024-05-30T16:43:53.873194+0000","last_became_active":"2024-05-30T16:43:52.895370+0000","last_became_peered":"2024-05-30T16:43:52.895370+0000","last_unstale":"2024-05-30T16:43:53.873194+0000","last_undegraded":"2024-05-30T16:43:53.873194+0000","last_fullsized":"2024-05-30T16:43:53.873194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:31.485813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224641+0000","last_change":"2024-05-30T16:43:52.883128+0000","last_active":"2024-05-30T16:43:54.224641+0000","last_peered":"2024-05-30T16:43:54.224641+0000","last_clean":"2024-05-30T16:43:54.224641+0000","last_became_active":"2024-05-30T16:43:52.881729+0000","last_became_peered":"2024-05-30T16:43:52.881729+0000","last_unstale":"2024-05-30T16:43:54.224641+0000","last_undegraded":"2024-05-30T16:43:54.224641+0000","last_fullsized":"2024-05-30T16:43:54.224641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873862+0000","last_change":"2024-05-30T16:43:52.876820+0000","last_active":"2024-05-30T16:43:53.873862+0000","last_peered":"2024-05-30T16:43:53.873862+0000","last_clean":"2024-05-30T16:43:53.873862+0000","last_became_active":"2024-05-30T16:43:52.875407+0000","last_became_peered":"2024-05-30T16:43:52.875407+0000","last_unstale":"2024-05-30T16:43:53.873862+0000","last_undegraded":"2024-05-30T16:43:53.873862+0000","last_fullsized":"2024-05-30T16:43:53.873862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871846+0000","last_change":"2024-05-30T16:43:52.883372+0000","last_active":"2024-05-30T16:43:53.871846+0000","last_peered":"2024-05-30T16:43:53.871846+0000","last_clean":"2024-05-30T16:43:53.871846+0000","last_became_active":"2024-05-30T16:43:52.882611+0000","last_became_peered":"2024-05-30T16:43:52.882611+0000","last_unstale":"2024-05-30T16:43:53.871846+0000","last_undegraded":"2024-05-30T16:43:53.871846+0000","last_fullsized":"2024-05-30T16:43:53.871846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224988+0000","last_change":"2024-05-30T16:43:52.882394+0000","last_active":"2024-05-30T16:43:54.224988+0000","last_peered":"2024-05-30T16:43:54.224988+0000","last_clean":"2024-05-30T16:43:54.224988+0000","last_became_active":"2024-05-30T16:43:52.882126+0000","last_became_peered":"2024-05-30T16:43:52.882126+0000","last_unstale":"2024-05-30T16:43:54.224988+0000","last_undegraded":"2024-05-30T16:43:54.224988+0000","last_fullsized":"2024-05-30T16:43:54.224988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.895448+0000","last_change":"2024-05-30T16:43:52.884885+0000","last_active":"2024-05-30T16:43:52.895448+0000","last_peered":"2024-05-30T16:43:52.895448+0000","last_clean":"2024-05-30T16:43:52.895448+0000","last_became_active":"2024-05-30T16:43:52.884414+0000","last_became_peered":"2024-05-30T16:43:52.884414+0000","last_unstale":"2024-05-30T16:43:52.895448+0000","last_undegraded":"2024-05-30T16:43:52.895448+0000","last_fullsized":"2024-05-30T16:43:52.895448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:25.394706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874039+0000","last_change":"2024-05-30T16:43:52.890624+0000","last_active":"2024-05-30T16:43:53.874039+0000","last_peered":"2024-05-30T16:43:53.874039+0000","last_clean":"2024-05-30T16:43:53.874039+0000","last_became_active":"2024-05-30T16:43:52.890269+0000","last_became_peered":"2024-05-30T16:43:52.890269+0000","last_unstale":"2024-05-30T16:43:53.874039+0000","last_undegraded":"2024-05-30T16:43:53.874039+0000","last_fullsized":"2024-05-30T16:43:53.874039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:48.878065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086162+0000","last_change":"2024-05-30T16:43:52.891560+0000","last_active":"2024-05-30T16:43:54.086162+0000","last_peered":"2024-05-30T16:43:54.086162+0000","last_clean":"2024-05-30T16:43:54.086162+0000","last_became_active":"2024-05-30T16:43:52.891194+0000","last_became_peered":"2024-05-30T16:43:52.891194+0000","last_unstale":"2024-05-30T16:43:54.086162+0000","last_undegraded":"2024-05-30T16:43:54.086162+0000","last_fullsized":"2024-05-30T16:43:54.086162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:33.066313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873436+0000","last_change":"2024-05-30T16:43:52.876082+0000","last_active":"2024-05-30T16:43:53.873436+0000","last_peered":"2024-05-30T16:43:53.873436+0000","last_clean":"2024-05-30T16:43:53.873436+0000","last_became_active":"2024-05-30T16:43:52.875792+0000","last_became_peered":"2024-05-30T16:43:52.875792+0000","last_unstale":"2024-05-30T16:43:53.873436+0000","last_undegraded":"2024-05-30T16:43:53.873436+0000","last_fullsized":"2024-05-30T16:43:53.873436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225172+0000","last_change":"2024-05-30T16:43:52.885848+0000","last_active":"2024-05-30T16:43:54.225172+0000","last_peered":"2024-05-30T16:43:54.225172+0000","last_clean":"2024-05-30T16:43:54.225172+0000","last_became_active":"2024-05-30T16:43:52.885532+0000","last_became_peered":"2024-05-30T16:43:52.885532+0000","last_unstale":"2024-05-30T16:43:54.225172+0000","last_undegraded":"2024-05-30T16:43:54.225172+0000","last_fullsized":"2024-05-30T16:43:54.225172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:04.698389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225258+0000","last_change":"2024-05-30T16:43:51.854344+0000","last_active":"2024-05-30T16:43:54.225258+0000","last_peered":"2024-05-30T16:43:54.225258+0000","last_clean":"2024-05-30T16:43:54.225258+0000","last_became_active":"2024-05-30T16:43:51.854081+0000","last_became_peered":"2024-05-30T16:43:51.854081+0000","last_unstale":"2024-05-30T16:43:54.225258+0000","last_undegraded":"2024-05-30T16:43:54.225258+0000","last_fullsized":"2024-05-30T16:43:54.225258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:50.620803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.895423+0000","last_change":"2024-05-30T16:43:52.881003+0000","last_active":"2024-05-30T16:43:52.895423+0000","last_peered":"2024-05-30T16:43:52.895423+0000","last_clean":"2024-05-30T16:43:52.895423+0000","last_became_active":"2024-05-30T16:43:52.880834+0000","last_became_peered":"2024-05-30T16:43:52.880834+0000","last_unstale":"2024-05-30T16:43:52.895423+0000","last_undegraded":"2024-05-30T16:43:52.895423+0000","last_fullsized":"2024-05-30T16:43:52.895423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:23.682583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873330+0000","last_change":"2024-05-30T16:43:51.863107+0000","last_active":"2024-05-30T16:43:53.873330+0000","last_peered":"2024-05-30T16:43:53.873330+0000","last_clean":"2024-05-30T16:43:53.873330+0000","last_became_active":"2024-05-30T16:43:51.855082+0000","last_became_peered":"2024-05-30T16:43:51.855082+0000","last_unstale":"2024-05-30T16:43:53.873330+0000","last_undegraded":"2024-05-30T16:43:53.873330+0000","last_fullsized":"2024-05-30T16:43:53.873330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224700+0000","last_change":"2024-05-30T16:43:51.856427+0000","last_active":"2024-05-30T16:43:54.224700+0000","last_peered":"2024-05-30T16:43:54.224700+0000","last_clean":"2024-05-30T16:43:54.224700+0000","last_became_active":"2024-05-30T16:43:51.856150+0000","last_became_peered":"2024-05-30T16:43:51.856150+0000","last_unstale":"2024-05-30T16:43:54.224700+0000","last_undegraded":"2024-05-30T16:43:54.224700+0000","last_fullsized":"2024-05-30T16:43:54.224700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:47.064682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224116+0000","last_change":"2024-05-30T16:43:52.875668+0000","last_active":"2024-05-30T16:43:54.224116+0000","last_peered":"2024-05-30T16:43:54.224116+0000","last_clean":"2024-05-30T16:43:54.224116+0000","last_became_active":"2024-05-30T16:43:52.875362+0000","last_became_peered":"2024-05-30T16:43:52.875362+0000","last_unstale":"2024-05-30T16:43:54.224116+0000","last_undegraded":"2024-05-30T16:43:54.224116+0000","last_fullsized":"2024-05-30T16:43:54.224116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873259+0000","last_change":"2024-05-30T16:43:51.854040+0000","last_active":"2024-05-30T16:43:53.873259+0000","last_peered":"2024-05-30T16:43:53.873259+0000","last_clean":"2024-05-30T16:43:53.873259+0000","last_became_active":"2024-05-30T16:43:51.852832+0000","last_became_peered":"2024-05-30T16:43:51.852832+0000","last_unstale":"2024-05-30T16:43:53.873259+0000","last_undegraded":"2024-05-30T16:43:53.873259+0000","last_fullsized":"2024-05-30T16:43:53.873259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:24.379706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871843+0000","last_change":"2024-05-30T16:43:52.880882+0000","last_active":"2024-05-30T16:43:53.871843+0000","last_peered":"2024-05-30T16:43:53.871843+0000","last_clean":"2024-05-30T16:43:53.871843+0000","last_became_active":"2024-05-30T16:43:52.880558+0000","last_became_peered":"2024-05-30T16:43:52.880558+0000","last_unstale":"2024-05-30T16:43:53.871843+0000","last_undegraded":"2024-05-30T16:43:53.871843+0000","last_fullsized":"2024-05-30T16:43:53.871843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872979+0000","last_change":"2024-05-30T16:43:51.864315+0000","last_active":"2024-05-30T16:43:53.872979+0000","last_peered":"2024-05-30T16:43:53.872979+0000","last_clean":"2024-05-30T16:43:53.872979+0000","last_became_active":"2024-05-30T16:43:51.855552+0000","last_became_peered":"2024-05-30T16:43:51.855552+0000","last_unstale":"2024-05-30T16:43:53.872979+0000","last_undegraded":"2024-05-30T16:43:53.872979+0000","last_fullsized":"2024-05-30T16:43:53.872979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:03.683389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875985+0000","last_change":"2024-05-30T16:43:52.876866+0000","last_active":"2024-05-30T16:43:53.875985+0000","last_peered":"2024-05-30T16:43:53.875985+0000","last_clean":"2024-05-30T16:43:53.875985+0000","last_became_active":"2024-05-30T16:43:52.876208+0000","last_became_peered":"2024-05-30T16:43:52.876208+0000","last_unstale":"2024-05-30T16:43:53.875985+0000","last_undegraded":"2024-05-30T16:43:53.875985+0000","last_fullsized":"2024-05-30T16:43:53.875985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:35.070892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.876134+0000","last_change":"2024-05-30T16:43:51.857532+0000","last_active":"2024-05-30T16:43:53.876134+0000","last_peered":"2024-05-30T16:43:53.876134+0000","last_clean":"2024-05-30T16:43:53.876134+0000","last_became_active":"2024-05-30T16:43:51.853042+0000","last_became_peered":"2024-05-30T16:43:51.853042+0000","last_unstale":"2024-05-30T16:43:53.876134+0000","last_undegraded":"2024-05-30T16:43:53.876134+0000","last_fullsized":"2024-05-30T16:43:53.876134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:19.738945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.904920+0000","last_change":"2024-05-30T16:43:52.901145+0000","last_active":"2024-05-30T16:43:52.904920+0000","last_peered":"2024-05-30T16:43:52.904920+0000","last_clean":"2024-05-30T16:43:52.904920+0000","last_became_active":"2024-05-30T16:43:52.900449+0000","last_became_peered":"2024-05-30T16:43:52.900449+0000","last_unstale":"2024-05-30T16:43:52.904920+0000","last_undegraded":"2024-05-30T16:43:52.904920+0000","last_fullsized":"2024-05-30T16:43:52.904920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:52.968401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871954+0000","last_change":"2024-05-30T16:43:51.857074+0000","last_active":"2024-05-30T16:43:53.871954+0000","last_peered":"2024-05-30T16:43:53.871954+0000","last_clean":"2024-05-30T16:43:53.871954+0000","last_became_active":"2024-05-30T16:43:51.856790+0000","last_became_peered":"2024-05-30T16:43:51.856790+0000","last_unstale":"2024-05-30T16:43:53.871954+0000","last_undegraded":"2024-05-30T16:43:53.871954+0000","last_fullsized":"2024-05-30T16:43:53.871954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226411+0000","last_change":"2024-05-30T16:43:52.883913+0000","last_active":"2024-05-30T16:43:54.226411+0000","last_peered":"2024-05-30T16:43:54.226411+0000","last_clean":"2024-05-30T16:43:54.226411+0000","last_became_active":"2024-05-30T16:43:52.883608+0000","last_became_peered":"2024-05-30T16:43:52.883608+0000","last_unstale":"2024-05-30T16:43:54.226411+0000","last_undegraded":"2024-05-30T16:43:54.226411+0000","last_fullsized":"2024-05-30T16:43:54.226411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:43.785583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874624+0000","last_change":"2024-05-30T16:43:51.859307+0000","last_active":"2024-05-30T16:43:53.874624+0000","last_peered":"2024-05-30T16:43:53.874624+0000","last_clean":"2024-05-30T16:43:53.874624+0000","last_became_active":"2024-05-30T16:43:51.858858+0000","last_became_peered":"2024-05-30T16:43:51.858858+0000","last_unstale":"2024-05-30T16:43:53.874624+0000","last_undegraded":"2024-05-30T16:43:53.874624+0000","last_fullsized":"2024-05-30T16:43:53.874624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.897427+0000","last_change":"2024-05-30T16:43:52.892760+0000","last_active":"2024-05-30T16:43:52.897427+0000","last_peered":"2024-05-30T16:43:52.897427+0000","last_clean":"2024-05-30T16:43:52.897427+0000","last_became_active":"2024-05-30T16:43:52.892483+0000","last_became_peered":"2024-05-30T16:43:52.892483+0000","last_unstale":"2024-05-30T16:43:52.897427+0000","last_undegraded":"2024-05-30T16:43:52.897427+0000","last_fullsized":"2024-05-30T16:43:52.897427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:57.617195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.898082+0000","last_change":"2024-05-30T16:43:51.849511+0000","last_active":"2024-05-30T16:43:52.898082+0000","last_peered":"2024-05-30T16:43:52.898082+0000","last_clean":"2024-05-30T16:43:52.898082+0000","last_became_active":"2024-05-30T16:43:51.842797+0000","last_became_peered":"2024-05-30T16:43:51.842797+0000","last_unstale":"2024-05-30T16:43:52.898082+0000","last_undegraded":"2024-05-30T16:43:52.898082+0000","last_fullsized":"2024-05-30T16:43:52.898082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:30:25.098938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.890789+0000","last_change":"2024-05-30T16:43:52.887806+0000","last_active":"2024-05-30T16:43:52.890789+0000","last_peered":"2024-05-30T16:43:52.890789+0000","last_clean":"2024-05-30T16:43:52.890789+0000","last_became_active":"2024-05-30T16:43:52.887486+0000","last_became_peered":"2024-05-30T16:43:52.887486+0000","last_unstale":"2024-05-30T16:43:52.890789+0000","last_undegraded":"2024-05-30T16:43:52.890789+0000","last_fullsized":"2024-05-30T16:43:52.890789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:41.040321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874098+0000","last_change":"2024-05-30T16:43:51.857906+0000","last_active":"2024-05-30T16:43:53.874098+0000","last_peered":"2024-05-30T16:43:53.874098+0000","last_clean":"2024-05-30T16:43:53.874098+0000","last_became_active":"2024-05-30T16:43:51.855395+0000","last_became_peered":"2024-05-30T16:43:51.855395+0000","last_unstale":"2024-05-30T16:43:53.874098+0000","last_undegraded":"2024-05-30T16:43:53.874098+0000","last_fullsized":"2024-05-30T16:43:53.874098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:32:37.147415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.892022+0000","last_change":"2024-05-30T16:43:52.888949+0000","last_active":"2024-05-30T16:43:52.892022+0000","last_peered":"2024-05-30T16:43:52.892022+0000","last_clean":"2024-05-30T16:43:52.892022+0000","last_became_active":"2024-05-30T16:43:52.887708+0000","last_became_peered":"2024-05-30T16:43:52.887708+0000","last_unstale":"2024-05-30T16:43:52.892022+0000","last_undegraded":"2024-05-30T16:43:52.892022+0000","last_fullsized":"2024-05-30T16:43:52.892022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:44.852718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873316+0000","last_change":"2024-05-30T16:43:51.856817+0000","last_active":"2024-05-30T16:43:53.873316+0000","last_peered":"2024-05-30T16:43:53.873316+0000","last_clean":"2024-05-30T16:43:53.873316+0000","last_became_active":"2024-05-30T16:43:51.856519+0000","last_became_peered":"2024-05-30T16:43:51.856519+0000","last_unstale":"2024-05-30T16:43:53.873316+0000","last_undegraded":"2024-05-30T16:43:53.873316+0000","last_fullsized":"2024-05-30T16:43:53.873316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873045+0000","last_change":"2024-05-30T16:43:52.874643+0000","last_active":"2024-05-30T16:43:53.873045+0000","last_peered":"2024-05-30T16:43:53.873045+0000","last_clean":"2024-05-30T16:43:53.873045+0000","last_became_active":"2024-05-30T16:43:52.874280+0000","last_became_peered":"2024-05-30T16:43:52.874280+0000","last_unstale":"2024-05-30T16:43:53.873045+0000","last_undegraded":"2024-05-30T16:43:53.873045+0000","last_fullsized":"2024-05-30T16:43:53.873045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086111+0000","last_change":"2024-05-30T16:43:51.851393+0000","last_active":"2024-05-30T16:43:54.086111+0000","last_peered":"2024-05-30T16:43:54.086111+0000","last_clean":"2024-05-30T16:43:54.086111+0000","last_became_active":"2024-05-30T16:43:51.851057+0000","last_became_peered":"2024-05-30T16:43:51.851057+0000","last_unstale":"2024-05-30T16:43:54.086111+0000","last_undegraded":"2024-05-30T16:43:54.086111+0000","last_fullsized":"2024-05-30T16:43:54.086111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:50.620803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.885359+0000","last_change":"2024-05-30T16:43:52.880958+0000","last_active":"2024-05-30T16:43:52.885359+0000","last_peered":"2024-05-30T16:43:52.885359+0000","last_clean":"2024-05-30T16:43:52.885359+0000","last_became_active":"2024-05-30T16:43:52.880536+0000","last_became_peered":"2024-05-30T16:43:52.880536+0000","last_unstale":"2024-05-30T16:43:52.885359+0000","last_undegraded":"2024-05-30T16:43:52.885359+0000","last_fullsized":"2024-05-30T16:43:52.885359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:58.783247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226477+0000","last_change":"2024-05-30T16:43:51.854467+0000","last_active":"2024-05-30T16:43:54.226477+0000","last_peered":"2024-05-30T16:43:54.226477+0000","last_clean":"2024-05-30T16:43:54.226477+0000","last_became_active":"2024-05-30T16:43:51.854104+0000","last_became_peered":"2024-05-30T16:43:51.854104+0000","last_unstale":"2024-05-30T16:43:54.226477+0000","last_undegraded":"2024-05-30T16:43:54.226477+0000","last_fullsized":"2024-05-30T16:43:54.226477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:26.393072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.891956+0000","last_change":"2024-05-30T16:43:52.888958+0000","last_active":"2024-05-30T16:43:52.891956+0000","last_peered":"2024-05-30T16:43:52.891956+0000","last_clean":"2024-05-30T16:43:52.891956+0000","last_became_active":"2024-05-30T16:43:52.886708+0000","last_became_peered":"2024-05-30T16:43:52.886708+0000","last_unstale":"2024-05-30T16:43:52.891956+0000","last_undegraded":"2024-05-30T16:43:52.891956+0000","last_fullsized":"2024-05-30T16:43:52.891956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:27.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086747+0000","last_change":"2024-05-30T16:43:51.859324+0000","last_active":"2024-05-30T16:43:54.086747+0000","last_peered":"2024-05-30T16:43:54.086747+0000","last_clean":"2024-05-30T16:43:54.086747+0000","last_became_active":"2024-05-30T16:43:51.858960+0000","last_became_peered":"2024-05-30T16:43:51.858960+0000","last_unstale":"2024-05-30T16:43:54.086747+0000","last_undegraded":"2024-05-30T16:43:54.086747+0000","last_fullsized":"2024-05-30T16:43:54.086747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.894490+0000","last_change":"2024-05-30T16:43:52.890768+0000","last_active":"2024-05-30T16:43:52.894490+0000","last_peered":"2024-05-30T16:43:52.894490+0000","last_clean":"2024-05-30T16:43:52.894490+0000","last_became_active":"2024-05-30T16:43:52.889014+0000","last_became_peered":"2024-05-30T16:43:52.889014+0000","last_unstale":"2024-05-30T16:43:52.894490+0000","last_undegraded":"2024-05-30T16:43:52.894490+0000","last_fullsized":"2024-05-30T16:43:52.894490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:33:20.915701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872029+0000","last_change":"2024-05-30T16:43:51.854719+0000","last_active":"2024-05-30T16:43:53.872029+0000","last_peered":"2024-05-30T16:43:53.872029+0000","last_clean":"2024-05-30T16:43:53.872029+0000","last_became_active":"2024-05-30T16:43:51.854430+0000","last_became_peered":"2024-05-30T16:43:51.854430+0000","last_unstale":"2024-05-30T16:43:53.872029+0000","last_undegraded":"2024-05-30T16:43:53.872029+0000","last_fullsized":"2024-05-30T16:43:53.872029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:03.683389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875608+0000","last_change":"2024-05-30T16:43:52.884624+0000","last_active":"2024-05-30T16:43:53.875608+0000","last_peered":"2024-05-30T16:43:53.875608+0000","last_clean":"2024-05-30T16:43:53.875608+0000","last_became_active":"2024-05-30T16:43:52.884348+0000","last_became_peered":"2024-05-30T16:43:52.884348+0000","last_unstale":"2024-05-30T16:43:53.875608+0000","last_undegraded":"2024-05-30T16:43:53.875608+0000","last_fullsized":"2024-05-30T16:43:53.875608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:32.373668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086493+0000","last_change":"2024-05-30T16:43:51.853597+0000","last_active":"2024-05-30T16:43:54.086493+0000","last_peered":"2024-05-30T16:43:54.086493+0000","last_clean":"2024-05-30T16:43:54.086493+0000","last_became_active":"2024-05-30T16:43:51.853235+0000","last_became_peered":"2024-05-30T16:43:51.853235+0000","last_unstale":"2024-05-30T16:43:54.086493+0000","last_undegraded":"2024-05-30T16:43:54.086493+0000","last_fullsized":"2024-05-30T16:43:54.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.903841+0000","last_change":"2024-05-30T16:43:52.902358+0000","last_active":"2024-05-30T16:43:52.903841+0000","last_peered":"2024-05-30T16:43:52.903841+0000","last_clean":"2024-05-30T16:43:52.903841+0000","last_became_active":"2024-05-30T16:43:52.902031+0000","last_became_peered":"2024-05-30T16:43:52.902031+0000","last_unstale":"2024-05-30T16:43:52.903841+0000","last_undegraded":"2024-05-30T16:43:52.903841+0000","last_fullsized":"2024-05-30T16:43:52.903841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:48.436948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":168,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.862895+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:43:52.862895+0000","last_peered":"2024-05-30T16:43:52.862895+0000","last_clean":"2024-05-30T16:43:52.862895+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:43:52.862895+0000","last_undegraded":"2024-05-30T16:43:52.862895+0000","last_fullsized":"2024-05-30T16:43:52.862895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875313+0000","last_change":"2024-05-30T16:43:51.849559+0000","last_active":"2024-05-30T16:43:53.875313+0000","last_peered":"2024-05-30T16:43:53.875313+0000","last_clean":"2024-05-30T16:43:53.875313+0000","last_became_active":"2024-05-30T16:43:51.848551+0000","last_became_peered":"2024-05-30T16:43:51.848551+0000","last_unstale":"2024-05-30T16:43:53.875313+0000","last_undegraded":"2024-05-30T16:43:53.875313+0000","last_fullsized":"2024-05-30T16:43:53.875313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:22.667583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225407+0000","last_change":"2024-05-30T16:43:52.900074+0000","last_active":"2024-05-30T16:43:54.225407+0000","last_peered":"2024-05-30T16:43:54.225407+0000","last_clean":"2024-05-30T16:43:54.225407+0000","last_became_active":"2024-05-30T16:43:52.899764+0000","last_became_peered":"2024-05-30T16:43:52.899764+0000","last_unstale":"2024-05-30T16:43:54.225407+0000","last_undegraded":"2024-05-30T16:43:54.225407+0000","last_fullsized":"2024-05-30T16:43:54.225407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:44.861208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.856729+0000","last_change":"2024-05-30T16:43:51.861550+0000","last_active":"2024-05-30T16:43:52.856729+0000","last_peered":"2024-05-30T16:43:52.856729+0000","last_clean":"2024-05-30T16:43:52.856729+0000","last_became_active":"2024-05-30T16:43:51.861157+0000","last_became_peered":"2024-05-30T16:43:51.861157+0000","last_unstale":"2024-05-30T16:43:52.856729+0000","last_undegraded":"2024-05-30T16:43:52.856729+0000","last_fullsized":"2024-05-30T16:43:52.856729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:20.333916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.883546+0000","last_change":"2024-05-30T16:43:52.878371+0000","last_active":"2024-05-30T16:43:52.883546+0000","last_peered":"2024-05-30T16:43:52.883546+0000","last_clean":"2024-05-30T16:43:52.883546+0000","last_became_active":"2024-05-30T16:43:52.877880+0000","last_became_peered":"2024-05-30T16:43:52.877880+0000","last_unstale":"2024-05-30T16:43:52.883546+0000","last_undegraded":"2024-05-30T16:43:52.883546+0000","last_fullsized":"2024-05-30T16:43:52.883546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:42.482122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086019+0000","last_change":"2024-05-30T16:43:51.858196+0000","last_active":"2024-05-30T16:43:54.086019+0000","last_peered":"2024-05-30T16:43:54.086019+0000","last_clean":"2024-05-30T16:43:54.086019+0000","last_became_active":"2024-05-30T16:43:51.857752+0000","last_became_peered":"2024-05-30T16:43:51.857752+0000","last_unstale":"2024-05-30T16:43:54.086019+0000","last_undegraded":"2024-05-30T16:43:54.086019+0000","last_fullsized":"2024-05-30T16:43:54.086019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871024+0000","last_change":"2024-05-30T16:43:52.863297+0000","last_active":"2024-05-30T16:43:53.871024+0000","last_peered":"2024-05-30T16:43:53.871024+0000","last_clean":"2024-05-30T16:43:53.871024+0000","last_became_active":"2024-05-30T16:43:52.863048+0000","last_became_peered":"2024-05-30T16:43:52.863048+0000","last_unstale":"2024-05-30T16:43:53.871024+0000","last_undegraded":"2024-05-30T16:43:53.871024+0000","last_fullsized":"2024-05-30T16:43:53.871024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874160+0000","last_change":"2024-05-30T16:43:51.854526+0000","last_active":"2024-05-30T16:43:53.874160+0000","last_peered":"2024-05-30T16:43:53.874160+0000","last_clean":"2024-05-30T16:43:53.874160+0000","last_became_active":"2024-05-30T16:43:51.853002+0000","last_became_peered":"2024-05-30T16:43:51.853002+0000","last_unstale":"2024-05-30T16:43:53.874160+0000","last_undegraded":"2024-05-30T16:43:53.874160+0000","last_fullsized":"2024-05-30T16:43:53.874160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:41.467122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225345+0000","last_change":"2024-05-30T16:43:52.876709+0000","last_active":"2024-05-30T16:43:54.225345+0000","last_peered":"2024-05-30T16:43:54.225345+0000","last_clean":"2024-05-30T16:43:54.225345+0000","last_became_active":"2024-05-30T16:43:52.876309+0000","last_became_peered":"2024-05-30T16:43:52.876309+0000","last_unstale":"2024-05-30T16:43:54.225345+0000","last_undegraded":"2024-05-30T16:43:54.225345+0000","last_fullsized":"2024-05-30T16:43:54.225345+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:35.643416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872135+0000","last_change":"2024-05-30T16:43:51.856730+0000","last_active":"2024-05-30T16:43:53.872135+0000","last_peered":"2024-05-30T16:43:53.872135+0000","last_clean":"2024-05-30T16:43:53.872135+0000","last_became_active":"2024-05-30T16:43:51.856409+0000","last_became_peered":"2024-05-30T16:43:51.856409+0000","last_unstale":"2024-05-30T16:43:53.872135+0000","last_undegraded":"2024-05-30T16:43:53.872135+0000","last_fullsized":"2024-05-30T16:43:53.872135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874822+0000","last_change":"2024-05-30T16:43:52.895760+0000","last_active":"2024-05-30T16:43:53.874822+0000","last_peered":"2024-05-30T16:43:53.874822+0000","last_clean":"2024-05-30T16:43:53.874822+0000","last_became_active":"2024-05-30T16:43:52.895305+0000","last_became_peered":"2024-05-30T16:43:52.895305+0000","last_unstale":"2024-05-30T16:43:53.874822+0000","last_undegraded":"2024-05-30T16:43:53.874822+0000","last_fullsized":"2024-05-30T16:43:53.874822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:03.535096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.881269+0000","last_change":"2024-05-30T16:43:51.864341+0000","last_active":"2024-05-30T16:43:52.881269+0000","last_peered":"2024-05-30T16:43:52.881269+0000","last_clean":"2024-05-30T16:43:52.881269+0000","last_became_active":"2024-05-30T16:43:51.864017+0000","last_became_peered":"2024-05-30T16:43:51.864017+0000","last_unstale":"2024-05-30T16:43:52.881269+0000","last_undegraded":"2024-05-30T16:43:52.881269+0000","last_fullsized":"2024-05-30T16:43:52.881269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:48.659115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871833+0000","last_change":"2024-05-30T16:43:52.865453+0000","last_active":"2024-05-30T16:43:53.871833+0000","last_peered":"2024-05-30T16:43:53.871833+0000","last_clean":"2024-05-30T16:43:53.871833+0000","last_became_active":"2024-05-30T16:43:52.864999+0000","last_became_peered":"2024-05-30T16:43:52.864999+0000","last_unstale":"2024-05-30T16:43:53.871833+0000","last_undegraded":"2024-05-30T16:43:53.871833+0000","last_fullsized":"2024-05-30T16:43:53.871833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.877105+0000","last_change":"2024-05-30T16:43:51.855757+0000","last_active":"2024-05-30T16:43:53.877105+0000","last_peered":"2024-05-30T16:43:53.877105+0000","last_clean":"2024-05-30T16:43:53.877105+0000","last_became_active":"2024-05-30T16:43:51.855051+0000","last_became_peered":"2024-05-30T16:43:51.855051+0000","last_unstale":"2024-05-30T16:43:53.877105+0000","last_undegraded":"2024-05-30T16:43:53.877105+0000","last_fullsized":"2024-05-30T16:43:53.877105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:34.628416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224825+0000","last_change":"2024-05-30T16:43:52.888339+0000","last_active":"2024-05-30T16:43:54.224825+0000","last_peered":"2024-05-30T16:43:54.224825+0000","last_clean":"2024-05-30T16:43:54.224825+0000","last_became_active":"2024-05-30T16:43:52.888104+0000","last_became_peered":"2024-05-30T16:43:52.888104+0000","last_unstale":"2024-05-30T16:43:54.224825+0000","last_undegraded":"2024-05-30T16:43:54.224825+0000","last_fullsized":"2024-05-30T16:43:54.224825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873115+0000","last_change":"2024-05-30T16:43:51.865200+0000","last_active":"2024-05-30T16:43:53.873115+0000","last_peered":"2024-05-30T16:43:53.873115+0000","last_clean":"2024-05-30T16:43:53.873115+0000","last_became_active":"2024-05-30T16:43:51.864864+0000","last_became_peered":"2024-05-30T16:43:51.864864+0000","last_unstale":"2024-05-30T16:43:53.873115+0000","last_undegraded":"2024-05-30T16:43:53.873115+0000","last_fullsized":"2024-05-30T16:43:53.873115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:34.628416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.881360+0000","last_change":"2024-05-30T16:43:52.875686+0000","last_active":"2024-05-30T16:43:52.881360+0000","last_peered":"2024-05-30T16:43:52.881360+0000","last_clean":"2024-05-30T16:43:52.881360+0000","last_became_active":"2024-05-30T16:43:52.875422+0000","last_became_peered":"2024-05-30T16:43:52.875422+0000","last_unstale":"2024-05-30T16:43:52.881360+0000","last_undegraded":"2024-05-30T16:43:52.881360+0000","last_fullsized":"2024-05-30T16:43:52.881360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:20.753945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.856196+0000","last_change":"2024-05-30T16:43:51.863391+0000","last_active":"2024-05-30T16:43:52.856196+0000","last_peered":"2024-05-30T16:43:52.856196+0000","last_clean":"2024-05-30T16:43:52.856196+0000","last_became_active":"2024-05-30T16:43:51.863097+0000","last_became_peered":"2024-05-30T16:43:51.863097+0000","last_unstale":"2024-05-30T16:43:52.856196+0000","last_undegraded":"2024-05-30T16:43:52.856196+0000","last_fullsized":"2024-05-30T16:43:52.856196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:40.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225485+0000","last_change":"2024-05-30T16:43:52.875177+0000","last_active":"2024-05-30T16:43:54.225485+0000","last_peered":"2024-05-30T16:43:54.225485+0000","last_clean":"2024-05-30T16:43:54.225485+0000","last_became_active":"2024-05-30T16:43:52.874601+0000","last_became_peered":"2024-05-30T16:43:52.874601+0000","last_unstale":"2024-05-30T16:43:54.225485+0000","last_undegraded":"2024-05-30T16:43:54.225485+0000","last_fullsized":"2024-05-30T16:43:54.225485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:50.920087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872070+0000","last_change":"2024-05-30T16:43:51.853127+0000","last_active":"2024-05-30T16:43:53.872070+0000","last_peered":"2024-05-30T16:43:53.872070+0000","last_clean":"2024-05-30T16:43:53.872070+0000","last_became_active":"2024-05-30T16:43:51.852860+0000","last_became_peered":"2024-05-30T16:43:51.852860+0000","last_unstale":"2024-05-30T16:43:53.872070+0000","last_undegraded":"2024-05-30T16:43:53.872070+0000","last_fullsized":"2024-05-30T16:43:53.872070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:49.905088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224426+0000","last_change":"2024-05-30T16:43:52.876127+0000","last_active":"2024-05-30T16:43:54.224426+0000","last_peered":"2024-05-30T16:43:54.224426+0000","last_clean":"2024-05-30T16:43:54.224426+0000","last_became_active":"2024-05-30T16:43:52.875763+0000","last_became_peered":"2024-05-30T16:43:52.875763+0000","last_unstale":"2024-05-30T16:43:54.224426+0000","last_undegraded":"2024-05-30T16:43:54.224426+0000","last_fullsized":"2024-05-30T16:43:54.224426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871499+0000","last_change":"2024-05-30T16:43:51.860285+0000","last_active":"2024-05-30T16:43:53.871499+0000","last_peered":"2024-05-30T16:43:53.871499+0000","last_clean":"2024-05-30T16:43:53.871499+0000","last_became_active":"2024-05-30T16:43:51.859565+0000","last_became_peered":"2024-05-30T16:43:51.859565+0000","last_unstale":"2024-05-30T16:43:53.871499+0000","last_undegraded":"2024-05-30T16:43:53.871499+0000","last_fullsized":"2024-05-30T16:43:53.871499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225236+0000","last_change":"2024-05-30T16:43:52.885852+0000","last_active":"2024-05-30T16:43:54.225236+0000","last_peered":"2024-05-30T16:43:54.225236+0000","last_clean":"2024-05-30T16:43:54.225236+0000","last_became_active":"2024-05-30T16:43:52.884228+0000","last_became_peered":"2024-05-30T16:43:52.884228+0000","last_unstale":"2024-05-30T16:43:54.225236+0000","last_undegraded":"2024-05-30T16:43:54.225236+0000","last_fullsized":"2024-05-30T16:43:54.225236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:12:06.956321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.889798+0000","last_change":"2024-05-30T16:43:51.863477+0000","last_active":"2024-05-30T16:43:52.889798+0000","last_peered":"2024-05-30T16:43:52.889798+0000","last_clean":"2024-05-30T16:43:52.889798+0000","last_became_active":"2024-05-30T16:43:51.863178+0000","last_became_peered":"2024-05-30T16:43:51.863178+0000","last_unstale":"2024-05-30T16:43:52.889798+0000","last_undegraded":"2024-05-30T16:43:52.889798+0000","last_fullsized":"2024-05-30T16:43:52.889798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:20:50.371105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873046+0000","last_change":"2024-05-30T16:43:52.880010+0000","last_active":"2024-05-30T16:43:53.873046+0000","last_peered":"2024-05-30T16:43:53.873046+0000","last_clean":"2024-05-30T16:43:53.873046+0000","last_became_active":"2024-05-30T16:43:52.879744+0000","last_became_peered":"2024-05-30T16:43:52.879744+0000","last_unstale":"2024-05-30T16:43:53.873046+0000","last_undegraded":"2024-05-30T16:43:53.873046+0000","last_fullsized":"2024-05-30T16:43:53.873046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:30.437464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874603+0000","last_change":"2024-05-30T16:43:52.878768+0000","last_active":"2024-05-30T16:43:53.874603+0000","last_peered":"2024-05-30T16:43:53.874603+0000","last_clean":"2024-05-30T16:43:53.874603+0000","last_became_active":"2024-05-30T16:43:52.878437+0000","last_became_peered":"2024-05-30T16:43:52.878437+0000","last_unstale":"2024-05-30T16:43:53.874603+0000","last_undegraded":"2024-05-30T16:43:53.874603+0000","last_fullsized":"2024-05-30T16:43:53.874603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:13:13.668843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.902941+0000","last_change":"2024-05-30T16:43:51.860527+0000","last_active":"2024-05-30T16:43:52.902941+0000","last_peered":"2024-05-30T16:43:52.902941+0000","last_clean":"2024-05-30T16:43:52.902941+0000","last_became_active":"2024-05-30T16:43:51.860201+0000","last_became_peered":"2024-05-30T16:43:51.860201+0000","last_unstale":"2024-05-30T16:43:52.902941+0000","last_undegraded":"2024-05-30T16:43:52.902941+0000","last_fullsized":"2024-05-30T16:43:52.902941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:44:14.302309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875458+0000","last_change":"2024-05-30T16:43:52.882046+0000","last_active":"2024-05-30T16:43:53.875458+0000","last_peered":"2024-05-30T16:43:53.875458+0000","last_clean":"2024-05-30T16:43:53.875458+0000","last_became_active":"2024-05-30T16:43:52.881777+0000","last_became_peered":"2024-05-30T16:43:52.881777+0000","last_unstale":"2024-05-30T16:43:53.875458+0000","last_undegraded":"2024-05-30T16:43:53.875458+0000","last_fullsized":"2024-05-30T16:43:53.875458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:05:00.764986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.901491+0000","last_change":"2024-05-30T16:43:51.865821+0000","last_active":"2024-05-30T16:43:52.901491+0000","last_peered":"2024-05-30T16:43:52.901491+0000","last_clean":"2024-05-30T16:43:52.901491+0000","last_became_active":"2024-05-30T16:43:51.865511+0000","last_became_peered":"2024-05-30T16:43:51.865511+0000","last_unstale":"2024-05-30T16:43:52.901491+0000","last_undegraded":"2024-05-30T16:43:52.901491+0000","last_fullsized":"2024-05-30T16:43:52.901491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:51.953401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.904940+0000","last_change":"2024-05-30T16:43:52.901336+0000","last_active":"2024-05-30T16:43:52.904940+0000","last_peered":"2024-05-30T16:43:52.904940+0000","last_clean":"2024-05-30T16:43:52.904940+0000","last_became_active":"2024-05-30T16:43:52.900847+0000","last_became_peered":"2024-05-30T16:43:52.900847+0000","last_unstale":"2024-05-30T16:43:52.904940+0000","last_undegraded":"2024-05-30T16:43:52.904940+0000","last_fullsized":"2024-05-30T16:43:52.904940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:44:15.317309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225470+0000","last_change":"2024-05-30T16:43:51.853049+0000","last_active":"2024-05-30T16:43:54.225470+0000","last_peered":"2024-05-30T16:43:54.225470+0000","last_clean":"2024-05-30T16:43:54.225470+0000","last_became_active":"2024-05-30T16:43:51.852719+0000","last_became_peered":"2024-05-30T16:43:51.852719+0000","last_unstale":"2024-05-30T16:43:54.225470+0000","last_undegraded":"2024-05-30T16:43:54.225470+0000","last_fullsized":"2024-05-30T16:43:54.225470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:15.475651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225448+0000","last_change":"2024-05-30T16:43:52.880495+0000","last_active":"2024-05-30T16:43:54.225448+0000","last_peered":"2024-05-30T16:43:54.225448+0000","last_clean":"2024-05-30T16:43:54.225448+0000","last_became_active":"2024-05-30T16:43:52.879641+0000","last_became_peered":"2024-05-30T16:43:52.879641+0000","last_unstale":"2024-05-30T16:43:54.225448+0000","last_undegraded":"2024-05-30T16:43:54.225448+0000","last_fullsized":"2024-05-30T16:43:54.225448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:36.892645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-30T16:43:52.902208+0000","last_change":"2024-05-30T16:43:51.859807+0000","last_active":"2024-05-30T16:43:52.902208+0000","last_peered":"2024-05-30T16:43:52.902208+0000","last_clean":"2024-05-30T16:43:52.902208+0000","last_became_active":"2024-05-30T16:43:51.859531+0000","last_became_peered":"2024-05-30T16:43:51.859531+0000","last_unstale":"2024-05-30T16:43:52.902208+0000","last_undegraded":"2024-05-30T16:43:52.902208+0000","last_fullsized":"2024-05-30T16:43:52.902208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:12.533740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":71038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":72524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":77212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":661862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":661918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:43:57.848 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T16:43:57.848 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-30T16:43:57.849 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-30T16:43:57.849 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-30T16:43:57.849 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-30T16:43:58.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.098+0000 7f10ed000640 1 Processor -- start 2024-05-30T16:43:58.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.098+0000 7f10ed000640 1 -- start start 2024-05-30T16:43:58.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.099+0000 7f10ed000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10e815b3a0 0x7f10e80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.099+0000 7f10ed000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e806dc20 0x7f10e815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.100+0000 7f10ed000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e80afd60 con 0x7f10e8072c00 2024-05-30T16:43:58.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.100+0000 7f10ed000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f10e806f980 con 0x7f10e815b3a0 2024-05-30T16:43:58.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.100+0000 7f10ed000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10e80ae0c0 con 0x7f10e806dc20 2024-05-30T16:43:58.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.100+0000 7f10e7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f10e8072c00 0x7f10e80ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35268/0 (socket says 172.21.15.67:35268) 2024-05-30T16:43:58.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.100+0000 7f10e7e00640 1 -- 172.21.15.67:0/734101388 learned_addr learned my addr 172.21.15.67:0/734101388 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:58.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.100+0000 7f10e7400640 1 --2- 172.21.15.67:0/734101388 >> v2:172.21.15.169:3300/0 conn(0x7f10e815b3a0 0x7f10e80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.100+0000 7f10e6a00640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e806dc20 0x7f10e815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.100+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4098314646 0 0) 0x7f10e80afd60 con 0x7f10e8072c00 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.100+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10cc005730 con 0x7f10e8072c00 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3251766001 0 0) 0x7f10cc005730 con 0x7f10e8072c00 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e806dc20 msgr2=0x7f10e815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e806dc20 0x7f10e815f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 >> v2:172.21.15.169:3300/0 conn(0x7f10e815b3a0 msgr2=0x7f10e80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 --2- 172.21.15.67:0/734101388 >> v2:172.21.15.169:3300/0 conn(0x7f10e815b3a0 0x7f10e80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e80af7a0 con 0x7f10e8072c00 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f10e00019f0 con 0x7f10e8072c00 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 >> v1:172.21.15.67:6789/0 conn(0x7f10e8072c00 legacy=0x7f10e80ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc005bc0 0x7f10cc005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc006a10 0x7f10cc006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e806dc20 0x7f10cc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f10e806f980 con 0x7f10cc005bc0 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10e80ae0c0 con 0x7f10cc006a10 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc00dbd0 con 0x7f10e806dc20 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.101+0000 7f10e7400640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc005bc0 0x7f10cc005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.102+0000 7f10e7400640 1 -- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e806dc20 msgr2=0x7f10cc0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.102+0000 7f10e6a00640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc006a10 0x7f10cc006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.102+0000 7f10e7400640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e806dc20 0x7f10cc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.102+0000 7f10e7400640 1 -- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc006a10 msgr2=0x7f10cc006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.102+0000 7f10e7400640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc006a10 0x7f10cc006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.102+0000 7f10e7400640 1 -- 172.21.15.67:0/734101388 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc005730 con 0x7f10cc005bc0 2024-05-30T16:43:58.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.103+0000 7f10e7400640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc005bc0 0x7f10cc005fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009ac0 tx=0x7f10cc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa93745fc32f3e43 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:58.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.103+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8036070 con 0x7f10cc005bc0 2024-05-30T16:43:58.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.103+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d8031400 con 0x7f10cc005bc0 2024-05-30T16:43:58.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.103+0000 7f10e6000640 1 -- 172.21.15.67:0/734101388 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8009690 con 0x7f10cc005bc0 2024-05-30T16:43:58.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.104+0000 7f10ed000640 1 -- 172.21.15.67:0/734101388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc005bc0 msgr2=0x7f10cc005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.104+0000 7f10ed000640 1 --2- 172.21.15.67:0/734101388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10cc005bc0 0x7f10cc005fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009ac0 tx=0x7f10cc007a40 comp rx=0 tx=0).stop 2024-05-30T16:43:58.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.104+0000 7f10e7e00640 1 -- 172.21.15.67:0/734101388 reap_dead start 2024-05-30T16:43:58.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.104+0000 7f10ed000640 1 -- 172.21.15.67:0/734101388 shutdown_connections 2024-05-30T16:43:58.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.104+0000 7f10ed000640 1 -- 172.21.15.67:0/734101388 >> 172.21.15.67:0/734101388 conn(0x7f10e80b1fc0 msgr2=0x7f10e80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:58.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.104+0000 7f10ed000640 1 -- 172.21.15.67:0/734101388 shutdown_connections 2024-05-30T16:43:58.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.104+0000 7f10ed000640 1 -- 172.21.15.67:0/734101388 wait complete. 2024-05-30T16:43:58.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.106+0000 7f10ed000640 1 Processor -- start 2024-05-30T16:43:58.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.106+0000 7f10ed000640 1 -- start start 2024-05-30T16:43:58.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10ed000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc005bc0 0x7f10e81430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10ed000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e81f3be0 0x7f10e814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10ed000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10e81f4040 0x7f10e814fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10ed000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8073050 con 0x7f10e81f4040 2024-05-30T16:43:58.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10ed000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10e815b780 con 0x7f10cc005bc0 2024-05-30T16:43:58.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10ed000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10e80731d0 con 0x7f10e81f3be0 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10e81f4040 0x7f10e814fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10e81f4040 0x7f10e814fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43538/0 (socket says 172.21.15.67:43538) 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10e7e00640 1 -- 172.21.15.67:0/6685553 learned_addr learned my addr 172.21.15.67:0/6685553 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10e7400640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc005bc0 0x7f10e81430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10e6a00640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e81f3be0 0x7f10e814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.107+0000 7f10e7e00640 1 -- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e81f3be0 msgr2=0x7f10e814f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10e7e00640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e81f3be0 0x7f10e814f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10e7e00640 1 -- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc005bc0 msgr2=0x7f10e81430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10e7e00640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc005bc0 0x7f10e81430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10e7e00640 1 -- 172.21.15.67:0/6685553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e80ca830 con 0x7f10e81f4040 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10e7400640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc005bc0 0x7f10e81430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10e6a00640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e81f3be0 0x7f10e814f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10e7e00640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10e81f4040 0x7f10e814fc40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f10e00013d0 tx=0x7f10e000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:58.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10df400640 1 -- 172.21.15.67:0/6685553 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e000a5b0 con 0x7f10e81f4040 2024-05-30T16:43:58.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10df400640 1 -- 172.21.15.67:0/6685553 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e000a750 con 0x7f10e81f4040 2024-05-30T16:43:58.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10df400640 1 -- 172.21.15.67:0/6685553 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e000ab40 con 0x7f10e81f4040 2024-05-30T16:43:58.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e8152800 con 0x7f10e81f4040 2024-05-30T16:43:58.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.108+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e80cadb0 con 0x7f10e81f4040 2024-05-30T16:43:58.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.109+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e80689e0 con 0x7f10e81f4040 2024-05-30T16:43:58.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.110+0000 7f10df400640 1 -- 172.21.15.67:0/6685553 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10e0011070 con 0x7f10e81f4040 2024-05-30T16:43:58.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.111+0000 7f10df400640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c806c920 0x7f10c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.111+0000 7f10df400640 1 -- 172.21.15.67:0/6685553 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f10e008b6f0 con 0x7f10e81f4040 2024-05-30T16:43:58.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.111+0000 7f10e7400640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c806c920 0x7f10c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.112+0000 7f10e7400640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c806c920 0x7f10c806ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f10d8033000 tx=0x7f10d8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:58.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.116+0000 7f10df400640 1 -- 172.21.15.67:0/6685553 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10e0059b10 con 0x7f10e81f4040 2024-05-30T16:43:58.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.377+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f10e80caf80 con 0x7f10e81f4040 2024-05-30T16:43:58.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.386+0000 7f10df400640 1 -- 172.21.15.67:0/6685553 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174131 (secure 0 0 0) 0x7f10e0059410 con 0x7f10e81f4040 2024-05-30T16:43:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c806c920 msgr2=0x7f10c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c806c920 0x7f10c806ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f10d8033000 tx=0x7f10d8008030 comp rx=0 tx=0).stop 2024-05-30T16:43:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10e81f4040 msgr2=0x7f10e814fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10e81f4040 0x7f10e814fc40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f10e00013d0 tx=0x7f10e000ccd0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 shutdown_connections 2024-05-30T16:43:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c806c920 0x7f10c806ede0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10e81f4040 0x7f10e814fc40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e81f3be0 0x7f10e814f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 --2- 172.21.15.67:0/6685553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10cc005bc0 0x7f10e81430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 >> 172.21.15.67:0/6685553 conn(0x7f10e80b1fc0 msgr2=0x7f10e80b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 shutdown_connections 2024-05-30T16:43:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.390+0000 7f10ed000640 1 -- 172.21.15.67:0/6685553 wait complete. 2024-05-30T16:43:58.409 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"flags":0,"active_gid":4098,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":46426377},{"type":"v1","addr":"172.21.15.67:6833","nonce":46426377}]},"active_addr":"172.21.15.67:6833/46426377","active_change":"2024-05-30T16:43:31.956838+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2101420957}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3516043286}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":1530119183}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":1839886696}]}]} 2024-05-30T16:43:58.412 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-30T16:43:58.412 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-30T16:43:58.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:43:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.611+0000 7f56e9600640 1 Processor -- start 2024-05-30T16:43:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.611+0000 7f56e9600640 1 -- start start 2024-05-30T16:43:58.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.613+0000 7f56e9600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f56e406f860 0x7f56e40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.613+0000 7f56e9600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56e406ed10 con 0x7f56e40b3690 2024-05-30T16:43:58.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.613+0000 7f56e9600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56e415cb00 con 0x7f56e40d0bb0 2024-05-30T16:43:58.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.613+0000 7f56e9600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f56e40b3cc0 con 0x7f56e406f860 2024-05-30T16:43:58.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.613+0000 7f56dbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f56e40b3690 0x7f56e40d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35278/0 (socket says 172.21.15.67:35278) 2024-05-30T16:43:58.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.613+0000 7f56dbe00640 1 -- 172.21.15.67:0/2186085974 learned_addr learned my addr 172.21.15.67:0/2186085974 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:58.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2a00640 1 --2- 172.21.15.67:0/2186085974 >> v2:172.21.15.169:3300/0 conn(0x7f56e406f860 0x7f56e40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876241346 0 0) 0x7f56cc001110 con 0x7f56e40b3690 2024-05-30T16:43:58.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56c8003520 con 0x7f56e40b3690 2024-05-30T16:43:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 455317462 0 0) 0x7f56c8003520 con 0x7f56e40b3690 2024-05-30T16:43:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 >> v2:172.21.15.169:3300/0 conn(0x7f56e406f860 msgr2=0x7f56e40b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2000640 1 --2- 172.21.15.67:0/2186085974 >> v2:172.21.15.169:3300/0 conn(0x7f56e406f860 0x7f56e40b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 >> v1:172.21.15.169:6789/0 conn(0x7f56e40d0bb0 legacy=0x7f56e40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e40b3fa0 con 0x7f56e40b3690 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f56cc001840 con 0x7f56e40b3690 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2a00640 1 --2- 172.21.15.67:0/2186085974 >> v2:172.21.15.169:3300/0 conn(0x7f56e406f860 0x7f56e40b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.614+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 >> v1:172.21.15.67:6789/0 conn(0x7f56e40b3690 legacy=0x7f56e40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e2000640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c80039b0 0x7f56c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e2000640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c8004800 0x7f56c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e2000640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56e40b3cc0 con 0x7f56c80051c0 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56c800c060 con 0x7f56c80039b0 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56c800c260 con 0x7f56c8004800 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e3400640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e2a00640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c80039b0 0x7f56c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56dbe00640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c8004800 0x7f56c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e3400640 1 -- 172.21.15.67:0/2186085974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c8004800 msgr2=0x7f56c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:58.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e3400640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c8004800 0x7f56c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e3400640 1 -- 172.21.15.67:0/2186085974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c80039b0 msgr2=0x7f56c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:58.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e3400640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56c80039b0 0x7f56c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56e3400640 1 -- 172.21.15.67:0/2186085974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c8003520 con 0x7f56c80051c0 2024-05-30T16:43:58.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.615+0000 7f56dbe00640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56c8004800 0x7f56c8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:58.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.616+0000 7f56e3400640 1 -- 172.21.15.67:0/2186085974 reap_dead start 2024-05-30T16:43:58.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.616+0000 7f56e3400640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56c800b9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f56dc001e50 tx=0x7f56c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a68bdd13b841b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:58.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc001a70 con 0x7f56c80051c0 2024-05-30T16:43:58.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56dc003970 con 0x7f56c80051c0 2024-05-30T16:43:58.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e2000640 1 -- 172.21.15.67:0/2186085974 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc003d60 con 0x7f56c80051c0 2024-05-30T16:43:58.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e9600640 1 -- 172.21.15.67:0/2186085974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 msgr2=0x7f56c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e9600640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56c800b9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f56dc001e50 tx=0x7f56c8005830 comp rx=0 tx=0).stop 2024-05-30T16:43:58.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e9600640 1 -- 172.21.15.67:0/2186085974 shutdown_connections 2024-05-30T16:43:58.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e9600640 1 --2- 172.21.15.67:0/2186085974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56c800b9c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e9600640 1 -- 172.21.15.67:0/2186085974 >> 172.21.15.67:0/2186085974 conn(0x7f56e406bae0 msgr2=0x7f56e406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:58.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e9600640 1 -- 172.21.15.67:0/2186085974 shutdown_connections 2024-05-30T16:43:58.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.617+0000 7f56e9600640 1 -- 172.21.15.67:0/2186085974 wait complete. 2024-05-30T16:43:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.619+0000 7f56e9600640 1 Processor -- start 2024-05-30T16:43:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.619+0000 7f56e9600640 1 -- start start 2024-05-30T16:43:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.620+0000 7f56e9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56e40c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.620+0000 7f56e9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e415c1e0 0x7f56e40ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.620+0000 7f56e9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e40ceec0 0x7f56e40c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.620+0000 7f56e9600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56e40b4840 con 0x7f56c80051c0 2024-05-30T16:43:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.620+0000 7f56e9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56e40b4560 con 0x7f56e415c1e0 2024-05-30T16:43:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.620+0000 7f56e9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56e40b3150 con 0x7f56e40ceec0 2024-05-30T16:43:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56e40c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56e40c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43552/0 (socket says 172.21.15.67:43552) 2024-05-30T16:43:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e2a00640 1 -- 172.21.15.67:0/947203020 learned_addr learned my addr 172.21.15.67:0/947203020 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e2a00640 1 -- 172.21.15.67:0/947203020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e40ceec0 msgr2=0x7f56e40c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e3400640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e40ceec0 0x7f56e40c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56dbe00640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e415c1e0 0x7f56e40ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e2a00640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e40ceec0 0x7f56e40c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e2a00640 1 -- 172.21.15.67:0/947203020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e415c1e0 msgr2=0x7f56e40ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e2a00640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e415c1e0 0x7f56e40ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e2a00640 1 -- 172.21.15.67:0/947203020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e41f4c30 con 0x7f56c80051c0 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.621+0000 7f56e2a00640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56e40c4710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f56d4011750 tx=0x7f56d4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.622+0000 7f56daa00640 1 -- 172.21.15.67:0/947203020 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d400fbd0 con 0x7f56c80051c0 2024-05-30T16:43:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.622+0000 7f56daa00640 1 -- 172.21.15.67:0/947203020 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d400fd70 con 0x7f56c80051c0 2024-05-30T16:43:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.622+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e41f4f10 con 0x7f56c80051c0 2024-05-30T16:43:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.622+0000 7f56daa00640 1 -- 172.21.15.67:0/947203020 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4015660 con 0x7f56c80051c0 2024-05-30T16:43:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.622+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e41f54d0 con 0x7f56c80051c0 2024-05-30T16:43:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.623+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e40689e0 con 0x7f56c80051c0 2024-05-30T16:43:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.624+0000 7f56daa00640 1 -- 172.21.15.67:0/947203020 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56d4015800 con 0x7f56c80051c0 2024-05-30T16:43:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.625+0000 7f56daa00640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f56c006c920 0x7f56c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.625+0000 7f56daa00640 1 -- 172.21.15.67:0/947203020 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f56d408c660 con 0x7f56c80051c0 2024-05-30T16:43:58.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.625+0000 7f56dbe00640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f56c006c920 0x7f56c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:58.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.626+0000 7f56dbe00640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f56c006c920 0x7f56c006ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0012b0 tx=0x7f56cc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:58.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.629+0000 7f56daa00640 1 -- 172.21.15.67:0/947203020 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56d4019030 con 0x7f56c80051c0 2024-05-30T16:43:58.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.836+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56e40cd480 con 0x7f56c80051c0 2024-05-30T16:43:58.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.838+0000 7f56daa00640 1 -- 172.21.15.67:0/947203020 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f56d405aa90 con 0x7f56c80051c0 2024-05-30T16:43:58.839 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:58.839 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:53.858729+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:43:50.496509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-30T16:43:51.276872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:43:58.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f56c006c920 msgr2=0x7f56c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f56c006c920 0x7f56c006ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0012b0 tx=0x7f56cc009040 comp rx=0 tx=0).stop 2024-05-30T16:43:58.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 msgr2=0x7f56e40c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:58.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56e40c4710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f56d4011750 tx=0x7f56d4011c10 comp rx=0 tx=0).stop 2024-05-30T16:43:58.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 shutdown_connections 2024-05-30T16:43:58.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f56c006c920 0x7f56c006ede0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56e40ceec0 0x7f56e40c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56e415c1e0 0x7f56e40ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 --2- 172.21.15.67:0/947203020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56c80051c0 0x7f56e40c4710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:58.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.841+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 >> 172.21.15.67:0/947203020 conn(0x7f56e406bae0 msgr2=0x7f56e40b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:58.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.842+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 shutdown_connections 2024-05-30T16:43:58.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:58.842+0000 7f56e9600640 1 -- 172.21.15.67:0/947203020 wait complete. 2024-05-30T16:43:58.861 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-30T16:43:58.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:43:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.072+0000 7f720e400640 1 Processor -- start 2024-05-30T16:43:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.073+0000 7f720e400640 1 -- start start 2024-05-30T16:43:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.074+0000 7f720e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72080cfa40 0x7f7208169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.074+0000 7f720e400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f720815ec50 con 0x7f720815e870 2024-05-30T16:43:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.074+0000 7f720e400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72080cff80 con 0x7f72080d16a0 2024-05-30T16:43:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.074+0000 7f720e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72080d2150 con 0x7f72080cfa40 2024-05-30T16:43:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.074+0000 7f720d000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f720815e870 0x7f72080d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35290/0 (socket says 172.21.15.67:35290) 2024-05-30T16:43:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f720d000640 1 -- 172.21.15.67:0/3285110562 learned_addr learned my addr 172.21.15.67:0/3285110562 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f720da00640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72080cfa40 0x7f7208169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1020656407 0 0) 0x7f720815ec50 con 0x7f720815e870 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71e8003520 con 0x7f720815e870 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1477037628 0 0) 0x7f71e8003520 con 0x7f720815e870 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72080cfa40 msgr2=0x7f7208169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f7207400640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72080cfa40 0x7f7208169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 >> v1:172.21.15.169:6789/0 conn(0x7f72080d16a0 legacy=0x7f72080cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72080d0260 con 0x7f720815e870 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f71f8001670 con 0x7f720815e870 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.075+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 >> v1:172.21.15.67:6789/0 conn(0x7f720815e870 legacy=0x7f72080d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f7207400640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72080cfa40 0x7f71e80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f7207400640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8003f60 0x7f71e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f7207400640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71e8004d40 0x7f71e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f72080d2150 con 0x7f71e8004d40 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71e800baa0 con 0x7f71e8003f60 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71e800bca0 con 0x7f72080cfa40 2024-05-30T16:43:59.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f720da00640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71e8004d40 0x7f71e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f7207e00640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8003f60 0x7f71e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f720da00640 1 -- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72080cfa40 msgr2=0x7f71e80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f720da00640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72080cfa40 0x7f71e80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f720da00640 1 -- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8003f60 msgr2=0x7f71e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f720da00640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8003f60 0x7f71e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.076+0000 7f720da00640 1 -- 172.21.15.67:0/3285110562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e8003520 con 0x7f71e8004d40 2024-05-30T16:43:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.077+0000 7f720da00640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71e8004d40 0x7f71e80051c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f720808caf0 tx=0x7f71e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97afc193c8df17eb server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.078+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200036070 con 0x7f71e8004d40 2024-05-30T16:43:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.078+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7200031400 con 0x7f71e8004d40 2024-05-30T16:43:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.078+0000 7f7207400640 1 -- 172.21.15.67:0/3285110562 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200009690 con 0x7f71e8004d40 2024-05-30T16:43:59.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.078+0000 7f720e400640 1 -- 172.21.15.67:0/3285110562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71e8004d40 msgr2=0x7f71e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.078+0000 7f720e400640 1 --2- 172.21.15.67:0/3285110562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71e8004d40 0x7f71e80051c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f720808caf0 tx=0x7f71e8005830 comp rx=0 tx=0).stop 2024-05-30T16:43:59.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.078+0000 7f720da00640 1 -- 172.21.15.67:0/3285110562 reap_dead start 2024-05-30T16:43:59.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.078+0000 7f720e400640 1 -- 172.21.15.67:0/3285110562 shutdown_connections 2024-05-30T16:43:59.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.078+0000 7f720e400640 1 -- 172.21.15.67:0/3285110562 >> 172.21.15.67:0/3285110562 conn(0x7f720808bfc0 msgr2=0x7f72080d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.078+0000 7f720e400640 1 -- 172.21.15.67:0/3285110562 shutdown_connections 2024-05-30T16:43:59.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.079+0000 7f720e400640 1 -- 172.21.15.67:0/3285110562 wait complete. 2024-05-30T16:43:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.080+0000 7f720e400640 1 Processor -- start 2024-05-30T16:43:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.081+0000 7f720e400640 1 -- start start 2024-05-30T16:43:59.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.081+0000 7f720e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8004d40 0x7f72081f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.081+0000 7f720e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72081f17b0 0x7f720815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f720e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815d220 0x7f72081f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f720e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f72080d4fb0 con 0x7f72081f17b0 2024-05-30T16:43:59.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f720e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72080d0820 con 0x7f71e8004d40 2024-05-30T16:43:59.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f720e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72080d35f0 con 0x7f720815d220 2024-05-30T16:43:59.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f7207e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72081f17b0 0x7f720815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f7207e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72081f17b0 0x7f720815cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43566/0 (socket says 172.21.15.67:43566) 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f7207e00640 1 -- 172.21.15.67:0/2389472310 learned_addr learned my addr 172.21.15.67:0/2389472310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f720d000640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8004d40 0x7f72081f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f720da00640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815d220 0x7f72081f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f7207e00640 1 -- 172.21.15.67:0/2389472310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815d220 msgr2=0x7f72081f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f7207e00640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815d220 0x7f72081f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f7207e00640 1 -- 172.21.15.67:0/2389472310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8004d40 msgr2=0x7f72081f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f7207e00640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8004d40 0x7f72081f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f7207e00640 1 -- 172.21.15.67:0/2389472310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72080cba10 con 0x7f72081f17b0 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.082+0000 7f720d000640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8004d40 0x7f72081f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:59.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.083+0000 7f7207e00640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72081f17b0 0x7f720815cce0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71f000e8f0 tx=0x7f71f000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.083+0000 7f7204c00640 1 -- 172.21.15.67:0/2389472310 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0003bd0 con 0x7f72081f17b0 2024-05-30T16:43:59.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.083+0000 7f7204c00640 1 -- 172.21.15.67:0/2389472310 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f0018070 con 0x7f72081f17b0 2024-05-30T16:43:59.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.083+0000 7f7204c00640 1 -- 172.21.15.67:0/2389472310 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f000c6c0 con 0x7f72081f17b0 2024-05-30T16:43:59.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.083+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72080cbcf0 con 0x7f72081f17b0 2024-05-30T16:43:59.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.083+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72080cc2b0 con 0x7f72081f17b0 2024-05-30T16:43:59.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.084+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720815d630 con 0x7f72081f17b0 2024-05-30T16:43:59.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.085+0000 7f7204c00640 1 -- 172.21.15.67:0/2389472310 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71f000c860 con 0x7f72081f17b0 2024-05-30T16:43:59.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.086+0000 7f7204c00640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406c8f0 0x7f71e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.086+0000 7f7204c00640 1 -- 172.21.15.67:0/2389472310 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f71f008bc70 con 0x7f72081f17b0 2024-05-30T16:43:59.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.086+0000 7f720d000640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406c8f0 0x7f71e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.087+0000 7f720d000640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406c8f0 0x7f71e406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f71f8001eb0 tx=0x7f71f8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.090+0000 7f7204c00640 1 -- 172.21.15.67:0/2389472310 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f720815d630 con 0x7f72081f17b0 2024-05-30T16:43:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.302+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72080689e0 con 0x7f72081f17b0 2024-05-30T16:43:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.304+0000 7f7204c00640 1 -- 172.21.15.67:0/2389472310 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f71f005a090 con 0x7f72081f17b0 2024-05-30T16:43:59.305 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:43:59.305 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:53.858729+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:43:50.496509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-30T16:43:51.276872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.306+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406c8f0 msgr2=0x7f71e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.306+0000 7f720e400640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406c8f0 0x7f71e406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f71f8001eb0 tx=0x7f71f8009040 comp rx=0 tx=0).stop 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.306+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72081f17b0 msgr2=0x7f720815cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.306+0000 7f720e400640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72081f17b0 0x7f720815cce0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71f000e8f0 tx=0x7f71f000edb0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.307+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 shutdown_connections 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.307+0000 7f720e400640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406c8f0 0x7f71e406edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.307+0000 7f720e400640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815d220 0x7f72081f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.307+0000 7f720e400640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72081f17b0 0x7f720815cce0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.307+0000 7f720e400640 1 --2- 172.21.15.67:0/2389472310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e8004d40 0x7f72081f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.307+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 >> 172.21.15.67:0/2389472310 conn(0x7f720808bfc0 msgr2=0x7f72080b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.307+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 shutdown_connections 2024-05-30T16:43:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.307+0000 7f720e400640 1 -- 172.21.15.67:0/2389472310 wait complete. 2024-05-30T16:43:59.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-30T16:43:59.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-30T16:43:59.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-30T16:43:59.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-30T16:43:59.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-30T16:43:59.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-30T16:43:59.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-30T16:43:59.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-30T16:43:59.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.625+0000 7f2d4ac00640 1 Processor -- start 2024-05-30T16:43:59.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.625+0000 7f2d4ac00640 1 -- start start 2024-05-30T16:43:59.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.625+0000 7fa29dc00640 1 Processor -- start 2024-05-30T16:43:59.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.625+0000 7fa29dc00640 1 -- start start 2024-05-30T16:43:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.627+0000 7f2d4ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406fdb0 0x7f2d440b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.627+0000 7f2d4ac00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d440b3960 con 0x7f2d4415a910 2024-05-30T16:43:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.627+0000 7f2d4ac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d440b3c40 con 0x7f2d4406eec0 2024-05-30T16:43:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.627+0000 7f2d4ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4415d010 con 0x7f2d4406fdb0 2024-05-30T16:43:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.628+0000 7f2d43e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2d4415a910 0x7f2d4406e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35306/0 (socket says 172.21.15.67:35306) 2024-05-30T16:43:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.627+0000 7fa29dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5d30 0x7fa2980c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.627+0000 7fa29dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa2980d1740 0x7fa2980b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.628+0000 7f2d43e00640 1 -- 172.21.15.67:0/2389494457 learned_addr learned my addr 172.21.15.67:0/2389494457 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.628+0000 7fa29dc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2980d3050 con 0x7fa2980b5950 2024-05-30T16:43:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.628+0000 7fa29dc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa2980cf480 con 0x7fa2980d1740 2024-05-30T16:43:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.628+0000 7fa29dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2980d3330 con 0x7fa2980b5d30 2024-05-30T16:43:59.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.628+0000 7f2d48c00640 1 --2- 172.21.15.67:0/2389494457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406fdb0 0x7f2d440b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7f2d42a00640 1 -- 172.21.15.67:0/2389494457 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3936202145 0 0) 0x7f2d440b3960 con 0x7f2d4415a910 2024-05-30T16:43:59.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7f2d42a00640 1 -- 172.21.15.67:0/2389494457 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d28003520 con 0x7f2d4415a910 2024-05-30T16:43:59.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7fa297400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa2980d1740 0x7fa2980b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7fa296000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5d30 0x7fa2980c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7fa296000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5d30 0x7fa2980c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45434/0 (socket says 172.21.15.67:45434) 2024-05-30T16:43:59.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7fa296000640 1 -- 172.21.15.67:0/397002240 learned_addr learned my addr 172.21.15.67:0/397002240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7f2d42a00640 1 -- 172.21.15.67:0/2389494457 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425373218 0 0) 0x7f2d440b3c40 con 0x7f2d4406eec0 2024-05-30T16:43:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7f2d42a00640 1 -- 172.21.15.67:0/2389494457 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d440b3960 con 0x7f2d4406eec0 2024-05-30T16:43:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7f2d48c00640 1 -- 172.21.15.67:0/2389494457 >> v1:172.21.15.169:6789/0 conn(0x7f2d4406eec0 legacy=0x7f2d44163460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7f2d48c00640 1 -- 172.21.15.67:0/2389494457 >> v1:172.21.15.67:6789/0 conn(0x7f2d4415a910 legacy=0x7f2d4406e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.629+0000 7f2d48c00640 1 -- 172.21.15.67:0/2389494457 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d440b3f20 con 0x7f2d4406fdb0 2024-05-30T16:43:59.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.630+0000 7fa297400640 1 -- 172.21.15.67:0/397002240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5d30 msgr2=0x7fa2980c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.630+0000 7fa297400640 1 --2- 172.21.15.67:0/397002240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5d30 0x7fa2980c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.630+0000 7fa297400640 1 -- 172.21.15.67:0/397002240 >> v1:172.21.15.169:6789/0 conn(0x7fa2980b5950 legacy=0x7fa2980d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.630+0000 7fa297400640 1 -- 172.21.15.67:0/397002240 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2980d3610 con 0x7fa2980d1740 2024-05-30T16:43:59.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.630+0000 7fa296000640 1 --2- 172.21.15.67:0/397002240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5d30 0x7fa2980c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:59.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.630+0000 7f3817400640 1 Processor -- start 2024-05-30T16:43:59.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.630+0000 7f3817400640 1 -- start start 2024-05-30T16:43:59.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.631+0000 7f2d48c00640 1 --2- 172.21.15.67:0/2389494457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406fdb0 0x7f2d440b3380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2d340097d0 tx=0x7f2d3402a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b883bc5abf7f721 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f966fe00640 1 Processor -- start 2024-05-30T16:43:59.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f3817400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d31f0 0x7f38100d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f2d42a00640 1 -- 172.21.15.67:0/2389494457 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34037070 con 0x7f2d4406fdb0 2024-05-30T16:43:59.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f2d42a00640 1 -- 172.21.15.67:0/2389494457 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d34032640 con 0x7f2d4406fdb0 2024-05-30T16:43:59.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f966fe00640 1 -- start start 2024-05-30T16:43:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f2d42a00640 1 -- 172.21.15.67:0/2389494457 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34032a50 con 0x7f2d4406fdb0 2024-05-30T16:43:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7fa297400640 1 --2- 172.21.15.67:0/397002240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2980d1740 0x7fa2980b7490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa290009d90 tx=0x7fa29002a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cffa7072c5f2c011 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f3817400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38100cf660 con 0x7f38100c7ae0 2024-05-30T16:43:59.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f3817400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38100cf940 con 0x7f38100d2e10 2024-05-30T16:43:59.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f3817400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38100d0270 con 0x7f38100d31f0 2024-05-30T16:43:59.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f3815400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f38100c7ae0 0x7f38100b4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35314/0 (socket says 172.21.15.67:35314) 2024-05-30T16:43:59.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.632+0000 7f3815400640 1 -- 172.21.15.67:0/2406413311 learned_addr learned my addr 172.21.15.67:0/2406413311 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:59.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.633+0000 7f2d4ac00640 1 -- 172.21.15.67:0/2389494457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406fdb0 msgr2=0x7f2d440b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.633+0000 7f2d4ac00640 1 --2- 172.21.15.67:0/2389494457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406fdb0 0x7f2d440b3380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2d340097d0 tx=0x7f2d3402a2c0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.633+0000 7fa295600640 1 -- 172.21.15.67:0/397002240 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290007740 con 0x7fa2980d1740 2024-05-30T16:43:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.633+0000 7fa295600640 1 -- 172.21.15.67:0/397002240 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2900078e0 con 0x7fa2980d1740 2024-05-30T16:43:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.633+0000 7f380fe00640 1 --2- 172.21.15.67:0/2406413311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d31f0 0x7f38100d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f966fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680512f0 0x7f96680516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f380f400640 1 -- 172.21.15.67:0/2406413311 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248744080 0 0) 0x7f38100cf940 con 0x7f38100d2e10 2024-05-30T16:43:59.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f380f400640 1 -- 172.21.15.67:0/2406413311 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37f8003520 con 0x7f38100d2e10 2024-05-30T16:43:59.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f380f400640 1 -- 172.21.15.67:0/2406413311 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4036872614 0 0) 0x7f38100cf660 con 0x7f38100c7ae0 2024-05-30T16:43:59.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f380f400640 1 -- 172.21.15.67:0/2406413311 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38100cf940 con 0x7f38100c7ae0 2024-05-30T16:43:59.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f966fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9668052510 con 0x7f96680e3a00 2024-05-30T16:43:59.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f966fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96680527f0 con 0x7f9668051c10 2024-05-30T16:43:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f2d4ac00640 1 -- 172.21.15.67:0/2389494457 shutdown_connections 2024-05-30T16:43:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f966fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9668053400 con 0x7f96680512f0 2024-05-30T16:43:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f2d4ac00640 1 --2- 172.21.15.67:0/2389494457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406fdb0 0x7f2d440b3380 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f2d4ac00640 1 -- 172.21.15.67:0/2389494457 >> 172.21.15.67:0/2389494457 conn(0x7f2d44067ce0 msgr2=0x7f2d4406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7fa29dc00640 1 -- 172.21.15.67:0/397002240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2980d1740 msgr2=0x7fa2980b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7fa29dc00640 1 --2- 172.21.15.67:0/397002240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2980d1740 0x7fa2980b7490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa290009d90 tx=0x7fa29002a570 comp rx=0 tx=0).stop 2024-05-30T16:43:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7fa29dc00640 1 -- 172.21.15.67:0/397002240 shutdown_connections 2024-05-30T16:43:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7fa29dc00640 1 --2- 172.21.15.67:0/397002240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2980d1740 0x7fa2980b7490 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7fa29dc00640 1 --2- 172.21.15.67:0/397002240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5d30 0x7fa2980c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7fa29dc00640 1 -- 172.21.15.67:0/397002240 >> 172.21.15.67:0/397002240 conn(0x7fa29808bfc0 msgr2=0x7fa2980d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f380fe00640 1 -- 172.21.15.67:0/2406413311 >> v1:172.21.15.169:6789/0 conn(0x7f38100d2e10 legacy=0x7f38100b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.634+0000 7f380fe00640 1 -- 172.21.15.67:0/2406413311 >> v1:172.21.15.67:6789/0 conn(0x7f38100c7ae0 legacy=0x7f38100b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7f380fe00640 1 -- 172.21.15.67:0/2406413311 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38100cfc20 con 0x7f38100d31f0 2024-05-30T16:43:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7f966e000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f96680e3a00 0x7f9668055960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35318/0 (socket says 172.21.15.67:35318) 2024-05-30T16:43:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7f966e000640 1 -- 172.21.15.67:0/186888421 learned_addr learned my addr 172.21.15.67:0/186888421 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7f966ea00640 1 --2- 172.21.15.67:0/186888421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680512f0 0x7f96680516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7f5b31c00640 1 Processor -- start 2024-05-30T16:43:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7fa29dc00640 1 -- 172.21.15.67:0/397002240 shutdown_connections 2024-05-30T16:43:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7fa29dc00640 1 -- 172.21.15.67:0/397002240 wait complete. 2024-05-30T16:43:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7f966d600640 1 -- 172.21.15.67:0/186888421 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1703766189 0 0) 0x7f9668052510 con 0x7f96680e3a00 2024-05-30T16:43:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7f966d600640 1 -- 172.21.15.67:0/186888421 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9654003520 con 0x7f96680e3a00 2024-05-30T16:43:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.635+0000 7f2d4ac00640 1 -- 172.21.15.67:0/2389494457 shutdown_connections 2024-05-30T16:43:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.636+0000 7f966ea00640 1 -- 172.21.15.67:0/186888421 >> v1:172.21.15.169:6789/0 conn(0x7f9668051c10 legacy=0x7f9668056070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.636+0000 7f966ea00640 1 -- 172.21.15.67:0/186888421 >> v1:172.21.15.67:6789/0 conn(0x7f96680e3a00 legacy=0x7f9668055960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.636+0000 7f966ea00640 1 -- 172.21.15.67:0/186888421 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9668052ad0 con 0x7f96680512f0 2024-05-30T16:43:59.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.636+0000 7f2d4ac00640 1 -- 172.21.15.67:0/2389494457 wait complete. 2024-05-30T16:43:59.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.637+0000 7f966ea00640 1 --2- 172.21.15.67:0/186888421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680512f0 0x7f96680516d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f96600097d0 tx=0x7f966002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8a62748e64d46d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.637+0000 7f380fe00640 1 --2- 172.21.15.67:0/2406413311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d31f0 0x7f38100d0700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009750 tx=0x7f37fc02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=243e13dc3e1bfa11 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.637+0000 7f5b31c00640 1 -- start start 2024-05-30T16:43:59.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.638+0000 7f966d600640 1 -- 172.21.15.67:0/186888421 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660037070 con 0x7f96680512f0 2024-05-30T16:43:59.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.638+0000 7f966d600640 1 -- 172.21.15.67:0/186888421 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9660032440 con 0x7f96680512f0 2024-05-30T16:43:59.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.638+0000 7f966d600640 1 -- 172.21.15.67:0/186888421 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96600327f0 con 0x7f96680512f0 2024-05-30T16:43:59.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.638+0000 7fa29dc00640 1 Processor -- start 2024-05-30T16:43:59.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f966fe00640 1 -- 172.21.15.67:0/186888421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680512f0 msgr2=0x7f96680516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f966fe00640 1 --2- 172.21.15.67:0/186888421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680512f0 0x7f96680516d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f96600097d0 tx=0x7f966002a1c0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7f2d4ac00640 1 Processor -- start 2024-05-30T16:43:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f380f400640 1 -- 172.21.15.67:0/2406413311 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc037070 con 0x7f38100d31f0 2024-05-30T16:43:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f380f400640 1 -- 172.21.15.67:0/2406413311 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37fc02ac40 con 0x7f38100d31f0 2024-05-30T16:43:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f380f400640 1 -- 172.21.15.67:0/2406413311 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804001670 con 0x7f38100d31f0 2024-05-30T16:43:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f966fe00640 1 -- 172.21.15.67:0/186888421 shutdown_connections 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f966fe00640 1 --2- 172.21.15.67:0/186888421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680512f0 0x7f96680516d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f966fe00640 1 -- 172.21.15.67:0/186888421 >> 172.21.15.67:0/186888421 conn(0x7f96680537b0 msgr2=0x7f9668053bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f966fe00640 1 -- 172.21.15.67:0/186888421 shutdown_connections 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f5b31c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c0b4870 0x7f5b2c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f5b31c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5b2c06e7e0 0x7f5b2c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f5b31c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b2c15bf00 con 0x7f5b2c0afee0 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f5b31c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5b2c0b02c0 con 0x7f5b2c06e7e0 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f5b31c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c06ef50 con 0x7f5b2c0b4870 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7f5b2aa00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5b2c0afee0 0x7f5b2c0b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:53830/0 (socket says 172.21.15.67:53830) 2024-05-30T16:43:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7f5b2aa00640 1 -- 172.21.15.67:0/2685604100 learned_addr learned my addr 172.21.15.67:0/2685604100 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:43:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f6186600640 1 Processor -- start 2024-05-30T16:43:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.641+0000 7f3817400640 1 -- 172.21.15.67:0/2406413311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d31f0 msgr2=0x7f38100d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.641+0000 7f3817400640 1 --2- 172.21.15.67:0/2406413311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d31f0 0x7f38100d0700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009750 tx=0x7f37fc02a130 comp rx=0 tx=0).stop 2024-05-30T16:43:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.638+0000 7fa29dc00640 1 -- start start 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7fa29dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5950 0x7fa298158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7fa29dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2980b5d30 0x7fa29815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7fa29dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2980d1740 0x7fa29815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7fa29dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2980c7c60 con 0x7fa2980b5d30 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7fa29dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2980d38f0 con 0x7fa2980d1740 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7fa29dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2980d5220 con 0x7fa2980b5950 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7fa296a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5950 0x7fa298158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7fa296a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5950 0x7fa298158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45464/0 (socket says 172.21.15.67:45464) 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7fa296a00640 1 -- 172.21.15.67:0/3627794165 learned_addr learned my addr 172.21.15.67:0/3627794165 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f6186600640 1 -- start start 2024-05-30T16:43:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7f6186600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6180162d80 0x7f618006f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7f6186600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6180163160 0x7f61800b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7f6186600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61801638c0 con 0x7f61800b4240 2024-05-30T16:43:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7f6186600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f61800b4670 con 0x7f6180163160 2024-05-30T16:43:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7f6186600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6180163ba0 con 0x7f6180162d80 2024-05-30T16:43:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.640+0000 7f5b23e00640 1 --2- 172.21.15.67:0/2685604100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c0b4870 0x7f5b2c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.641+0000 7f5b2a000640 1 -- 172.21.15.67:0/2685604100 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2837991670 0 0) 0x7f5b2c15bf00 con 0x7f5b2c0afee0 2024-05-30T16:43:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.641+0000 7f5b2a000640 1 -- 172.21.15.67:0/2685604100 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b0c003520 con 0x7f5b2c0afee0 2024-05-30T16:43:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.641+0000 7f617ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6180163160 0x7f61800b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.641+0000 7f617ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6180163160 0x7f61800b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43570/0 (socket says 172.21.15.67:43570) 2024-05-30T16:43:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.641+0000 7f617ea00640 1 -- 172.21.15.67:0/4139377284 learned_addr learned my addr 172.21.15.67:0/4139377284 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.641+0000 7f617f400640 1 --2- 172.21.15.67:0/4139377284 >> v2:172.21.15.169:3300/0 conn(0x7f6180162d80 0x7f618006f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.639+0000 7f966fe00640 1 -- 172.21.15.67:0/186888421 wait complete. 2024-05-30T16:43:59.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.641+0000 7f966fe00640 1 Processor -- start 2024-05-30T16:43:59.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7fa296000640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2980b5d30 0x7fa29815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f966fe00640 1 -- start start 2024-05-30T16:43:59.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f3817400640 1 -- 172.21.15.67:0/2406413311 shutdown_connections 2024-05-30T16:43:59.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f3817400640 1 --2- 172.21.15.67:0/2406413311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d31f0 0x7f38100d0700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f3817400640 1 -- 172.21.15.67:0/2406413311 >> 172.21.15.67:0/2406413311 conn(0x7f381008bfc0 msgr2=0x7f38100d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7fa297400640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2980d1740 0x7fa29815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7fa296a00640 1 -- 172.21.15.67:0/3627794165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2980d1740 msgr2=0x7fa29815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f5b23e00640 1 -- 172.21.15.67:0/2685604100 >> v2:172.21.15.169:3300/0 conn(0x7f5b2c06e7e0 msgr2=0x7f5b2c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f5b2b400640 1 --2- 172.21.15.67:0/2685604100 >> v2:172.21.15.169:3300/0 conn(0x7f5b2c06e7e0 0x7f5b2c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f5b23e00640 1 --2- 172.21.15.67:0/2685604100 >> v2:172.21.15.169:3300/0 conn(0x7f5b2c06e7e0 0x7f5b2c15c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f5b23e00640 1 -- 172.21.15.67:0/2685604100 >> v1:172.21.15.169:6789/0 conn(0x7f5b2c0afee0 legacy=0x7f5b2c0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f5b23e00640 1 -- 172.21.15.67:0/2685604100 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c06f230 con 0x7f5b2c0b4870 2024-05-30T16:43:59.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f3817400640 1 -- 172.21.15.67:0/2406413311 shutdown_connections 2024-05-30T16:43:59.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f2d4ac00640 1 -- start start 2024-05-30T16:43:59.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f617ea00640 1 -- 172.21.15.67:0/4139377284 >> v2:172.21.15.169:3300/0 conn(0x7f6180162d80 msgr2=0x7f618006f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7fab2c600640 1 Processor -- start 2024-05-30T16:43:59.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.643+0000 7fa296a00640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2980d1740 0x7fa29815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.643+0000 7fa296a00640 1 -- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2980b5d30 msgr2=0x7fa29815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.643+0000 7fa296a00640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2980b5d30 0x7fa29815f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.643+0000 7fa296a00640 1 -- 172.21.15.67:0/3627794165 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29815fe50 con 0x7fa2980b5950 2024-05-30T16:43:59.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.643+0000 7fa296000640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2980b5d30 0x7fa29815f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:59.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f617ea00640 1 --2- 172.21.15.67:0/4139377284 >> v2:172.21.15.169:3300/0 conn(0x7f6180162d80 0x7f618006f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7fa297400640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2980d1740 0x7fa29815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:59.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f5b23e00640 1 --2- 172.21.15.67:0/2685604100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c0b4870 0x7f5b2c15ac30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c15c0d0 tx=0x7f5b1402ea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b0bd449ff16e49 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.643+0000 7fab2c600640 1 -- start start 2024-05-30T16:43:59.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f617ea00640 1 -- 172.21.15.67:0/4139377284 >> v1:172.21.15.67:6789/0 conn(0x7f61800b4240 legacy=0x7f6180071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f617ea00640 1 -- 172.21.15.67:0/4139377284 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6180163e80 con 0x7f6180163160 2024-05-30T16:43:59.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f966fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680512f0 0x7f9668185650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f966fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9668051c10 0x7f966804cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f966fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680e3a00 0x7f966804d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f5b2a000640 1 -- 172.21.15.67:0/2685604100 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14032810 con 0x7f5b2c0b4870 2024-05-30T16:43:59.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f5b2a000640 1 -- 172.21.15.67:0/2685604100 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b140329b0 con 0x7f5b2c0b4870 2024-05-30T16:43:59.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f966fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f96680e4840 con 0x7f9668051c10 2024-05-30T16:43:59.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f966fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9668053090 con 0x7f96680512f0 2024-05-30T16:43:59.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.642+0000 7f3817400640 1 -- 172.21.15.67:0/2406413311 wait complete. 2024-05-30T16:43:59.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f966ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680512f0 0x7f9668185650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.644+0000 7f966e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9668051c10 0x7f966804cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f617ea00640 1 --2- 172.21.15.67:0/4139377284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6180163160 0x7f61800b3d00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6180072e80 tx=0x7f616c02a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790d307e3da2847b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f966e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9668051c10 0x7f966804cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43590/0 (socket says 172.21.15.67:43590) 2024-05-30T16:43:59.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f3817400640 1 Processor -- start 2024-05-30T16:43:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f966f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680e3a00 0x7f966804d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f966e000640 1 -- 172.21.15.67:0/2213183941 learned_addr learned my addr 172.21.15.67:0/2213183941 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f3817400640 1 -- start start 2024-05-30T16:43:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f5b31c00640 1 -- 172.21.15.67:0/2685604100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c0b4870 msgr2=0x7f5b2c15ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9668052070 con 0x7f96680e3a00 2024-05-30T16:43:59.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f5b31c00640 1 --2- 172.21.15.67:0/2685604100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c0b4870 0x7f5b2c15ac30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c15c0d0 tx=0x7f5b1402ea50 comp rx=0 tx=0).stop 2024-05-30T16:43:59.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f617e000640 1 -- 172.21.15.67:0/4139377284 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c037070 con 0x7f6180163160 2024-05-30T16:43:59.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f2d4ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406eec0 0x7f2d44144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f2d4ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d4406fdb0 0x7f2d440cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f2d4ac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d4415a910 0x7f2d4416ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f2d4ac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4406f500 con 0x7f2d4415a910 2024-05-30T16:43:59.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f2d4ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2d440b44e0 con 0x7f2d4406fdb0 2024-05-30T16:43:59.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f2d4ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d440b4a00 con 0x7f2d4406eec0 2024-05-30T16:43:59.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7fa296a00640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5950 0x7fa298158860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa288004640 tx=0x7fa28800c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f5b31c00640 1 -- 172.21.15.67:0/2685604100 shutdown_connections 2024-05-30T16:43:59.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f5b31c00640 1 --2- 172.21.15.67:0/2685604100 >> v2:172.21.15.169:3300/0 conn(0x7f5b2c06e7e0 0x7f5b2c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f5b31c00640 1 --2- 172.21.15.67:0/2685604100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c0b4870 0x7f5b2c15ac30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f5b31c00640 1 -- 172.21.15.67:0/2685604100 >> 172.21.15.67:0/2685604100 conn(0x7f5b2c06b900 msgr2=0x7f5b2c06bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f617e000640 1 -- 172.21.15.67:0/4139377284 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c032640 con 0x7f6180163160 2024-05-30T16:43:59.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.645+0000 7f617e000640 1 -- 172.21.15.67:0/4139377284 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c02ae60 con 0x7f6180163160 2024-05-30T16:43:59.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f2d43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406eec0 0x7f2d44144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f5b31c00640 1 -- 172.21.15.67:0/2685604100 shutdown_connections 2024-05-30T16:43:59.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f2d43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406eec0 0x7f2d44144170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45494/0 (socket says 172.21.15.67:45494) 2024-05-30T16:43:59.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f2d43e00640 1 -- 172.21.15.67:0/3239860807 learned_addr learned my addr 172.21.15.67:0/3239860807 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f2d48c00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d4415a910 0x7f2d4416ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7fa28ea00640 1 -- 172.21.15.67:0/3627794165 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288011790 con 0x7fa2980b5950 2024-05-30T16:43:59.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7fab2c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 0x7fab240b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f2d43400640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d4406fdb0 0x7f2d440cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f3817400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38100c7ae0 0x7f38100c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f3817400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d2e10 0x7f38100d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f3817400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38100d31f0 0x7f38100d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f3817400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f38100b3a60 con 0x7f38100d31f0 2024-05-30T16:43:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f3817400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f38100c7f30 con 0x7f38100c7ae0 2024-05-30T16:43:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f3817400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38100b3be0 con 0x7f38100d2e10 2024-05-30T16:43:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f380fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d2e10 0x7f38100d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f3815400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38100d31f0 0x7f38100d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f380fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d2e10 0x7f38100d4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45504/0 (socket says 172.21.15.67:45504) 2024-05-30T16:43:59.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f380fe00640 1 -- 172.21.15.67:0/2241909677 learned_addr learned my addr 172.21.15.67:0/2241909677 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7fa28ea00640 1 -- 172.21.15.67:0/3627794165 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa288017070 con 0x7fa2980b5950 2024-05-30T16:43:59.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7fa28ea00640 1 -- 172.21.15.67:0/3627794165 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288011bb0 con 0x7fa2980b5950 2024-05-30T16:43:59.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7fa29dc00640 1 -- 172.21.15.67:0/3627794165 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2980ce560 con 0x7fa2980b5950 2024-05-30T16:43:59.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7fa29dc00640 1 -- 172.21.15.67:0/3627794165 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2980c9c50 con 0x7fa2980b5950 2024-05-30T16:43:59.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f966f400640 1 -- 172.21.15.67:0/2213183941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680512f0 msgr2=0x7f9668185650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f966f400640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680512f0 0x7f9668185650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f966f400640 1 -- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9668051c10 msgr2=0x7f966804cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f966f400640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9668051c10 0x7f966804cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f966f400640 1 -- 172.21.15.67:0/2213183941 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9668189560 con 0x7f96680e3a00 2024-05-30T16:43:59.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f966e000640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9668051c10 0x7f966804cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:59.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f966f400640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680e3a00 0x7f966804d520 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f967006a0f0 tx=0x7f9670069600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7f5b31c00640 1 -- 172.21.15.67:0/2685604100 wait complete. 2024-05-30T16:43:59.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f6186600640 1 -- 172.21.15.67:0/4139377284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6180163160 msgr2=0x7f61800b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f6186600640 1 --2- 172.21.15.67:0/4139377284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6180163160 0x7f61800b3d00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6180072e80 tx=0x7f616c02a310 comp rx=0 tx=0).stop 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f6186600640 1 -- 172.21.15.67:0/4139377284 shutdown_connections 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f6186600640 1 --2- 172.21.15.67:0/4139377284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6180163160 0x7f61800b3d00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f6186600640 1 --2- 172.21.15.67:0/4139377284 >> v2:172.21.15.169:3300/0 conn(0x7f6180162d80 0x7f618006f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f6186600640 1 -- 172.21.15.67:0/4139377284 >> 172.21.15.67:0/4139377284 conn(0x7f6180067ce0 msgr2=0x7f618006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.646+0000 7fab2c600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab2415b850 0x7fab240720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7fab2c600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fab240b49e0 0x7fab240b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7fab2c600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fab2406e190 con 0x7fab2415b850 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7fab2c600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fab240725e0 con 0x7fab240b49e0 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7fab2c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab240b3a10 con 0x7fab24071cc0 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7fab29200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab2415b850 0x7fab240720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7fab29200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab2415b850 0x7fab240720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43604/0 (socket says 172.21.15.67:43604) 2024-05-30T16:43:59.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7fab29200640 1 -- 172.21.15.67:0/1363281475 learned_addr learned my addr 172.21.15.67:0/1363281475 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7fab2a600640 1 --2- 172.21.15.67:0/1363281475 >> v2:172.21.15.169:3300/0 conn(0x7fab240b49e0 0x7fab240b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.647+0000 7f7c5a000640 1 Processor -- start 2024-05-30T16:43:59.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f7c5a000640 1 -- start start 2024-05-30T16:43:59.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f9666a00640 1 -- 172.21.15.67:0/2213183941 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670071620 con 0x7f96680e3a00 2024-05-30T16:43:59.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f9666a00640 1 -- 172.21.15.67:0/2213183941 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96700717e0 con 0x7f96680e3a00 2024-05-30T16:43:59.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f6186600640 1 -- 172.21.15.67:0/4139377284 shutdown_connections 2024-05-30T16:43:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7fab29c00640 1 --2- 172.21.15.67:0/1363281475 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 0x7fab240b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f3814a00640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38100c7ae0 0x7f38100c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f6186600640 1 -- 172.21.15.67:0/4139377284 wait complete. 2024-05-30T16:43:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f2d48c00640 1 -- 172.21.15.67:0/3239860807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406eec0 msgr2=0x7f2d44144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f2d48c00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d4406eec0 0x7f2d44144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7fa29dc00640 1 -- 172.21.15.67:0/3627794165 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa2980689e0 con 0x7fa2980b5950 2024-05-30T16:43:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9668189840 con 0x7f96680e3a00 2024-05-30T16:43:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9668189e00 con 0x7f96680e3a00 2024-05-30T16:43:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7f9666a00640 1 -- 172.21.15.67:0/2213183941 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670071bd0 con 0x7f96680e3a00 2024-05-30T16:43:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.648+0000 7f2d48c00640 1 -- 172.21.15.67:0/3239860807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d4406fdb0 msgr2=0x7f2d440cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7f2d48c00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d4406fdb0 0x7f2d440cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f966804da60 con 0x7f96680e3a00 2024-05-30T16:43:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7fab2a600640 1 -- 172.21.15.67:0/1363281475 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 msgr2=0x7fab240b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7fab2a600640 1 --2- 172.21.15.67:0/1363281475 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 0x7fab240b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7fab2a600640 1 -- 172.21.15.67:0/1363281475 >> v2:172.21.15.67:3300/0 conn(0x7fab2415b850 msgr2=0x7fab240720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7fab2a600640 1 --2- 172.21.15.67:0/1363281475 >> v2:172.21.15.67:3300/0 conn(0x7fab2415b850 0x7fab240720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7fab2a600640 1 -- 172.21.15.67:0/1363281475 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab240b3cf0 con 0x7fab240b49e0 2024-05-30T16:43:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7f7c5a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7c5415c470 0x7f7c5406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7f7c5a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c540701e0 0x7f7c54071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f7c5a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c54072200 0x7f7c54162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f7c5a000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7c540adb40 con 0x7f7c5415c470 2024-05-30T16:43:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f7c5a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7c5415f0e0 con 0x7f7c54072200 2024-05-30T16:43:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f7c5a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5415f3c0 con 0x7f7c540701e0 2024-05-30T16:43:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f3815400640 1 -- 172.21.15.67:0/2241909677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d2e10 msgr2=0x7f38100d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.649+0000 7f2d48c00640 1 -- 172.21.15.67:0/3239860807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d441418e0 con 0x7f2d4415a910 2024-05-30T16:43:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f3815400640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38100d2e10 0x7f38100d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f3815400640 1 -- 172.21.15.67:0/2241909677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38100c7ae0 msgr2=0x7f38100c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f3815400640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38100c7ae0 0x7f38100c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f3815400640 1 -- 172.21.15.67:0/2241909677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38100c8ff0 con 0x7f38100d31f0 2024-05-30T16:43:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f7c53400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7c5415c470 0x7f7c5406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f7c53400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7c5415c470 0x7f7c5406fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43622/0 (socket says 172.21.15.67:43622) 2024-05-30T16:43:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f7c53400640 1 -- 172.21.15.67:0/1371599791 learned_addr learned my addr 172.21.15.67:0/1371599791 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f7c52a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c540701e0 0x7f7c54071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f7c53e00640 1 --2- 172.21.15.67:0/1371599791 >> v2:172.21.15.169:3300/0 conn(0x7f7c54072200 0x7f7c54162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f3815400640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38100d31f0 0x7f38100d4f40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f380800caa0 tx=0x7f380800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f7c53400640 1 -- 172.21.15.67:0/1371599791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c540701e0 msgr2=0x7f7c54071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f7c53400640 1 --2- 172.21.15.67:0/1371599791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c540701e0 0x7f7c54071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f7c53400640 1 -- 172.21.15.67:0/1371599791 >> v2:172.21.15.169:3300/0 conn(0x7f7c54072200 msgr2=0x7f7c54162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:43:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f7c53400640 1 --2- 172.21.15.67:0/1371599791 >> v2:172.21.15.169:3300/0 conn(0x7f7c54072200 0x7f7c54162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f7c53400640 1 -- 172.21.15.67:0/1371599791 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5415f6a0 con 0x7f7c5415c470 2024-05-30T16:43:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.650+0000 7f2d48c00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d4415a910 0x7f2d4416ccf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d4406c8e0 tx=0x7f2d340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f2d3be00640 1 -- 172.21.15.67:0/3239860807 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34040030 con 0x7f2d4415a910 2024-05-30T16:43:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4416a890 con 0x7f2d4415a910 2024-05-30T16:43:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f2d3be00640 1 -- 172.21.15.67:0/3239860807 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d34038040 con 0x7f2d4415a910 2024-05-30T16:43:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f5b31c00640 1 Processor -- start 2024-05-30T16:43:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f2d3be00640 1 -- 172.21.15.67:0/3239860807 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34037070 con 0x7f2d4415a910 2024-05-30T16:43:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f380cc00640 1 -- 172.21.15.67:0/2241909677 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808017070 con 0x7f38100d31f0 2024-05-30T16:43:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f380cc00640 1 -- 172.21.15.67:0/2241909677 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380800ab40 con 0x7f38100d31f0 2024-05-30T16:43:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.652+0000 7f380cc00640 1 -- 172.21.15.67:0/2241909677 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808017070 con 0x7f38100d31f0 2024-05-30T16:43:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.651+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4416ae50 con 0x7f2d4415a910 2024-05-30T16:43:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.652+0000 7fab2a600640 1 --2- 172.21.15.67:0/1363281475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab240b49e0 0x7fab240b4e70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab240cff20 tx=0x7fab1802e400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18cd6bf949a23db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.652+0000 7f5b31c00640 1 -- start start 2024-05-30T16:43:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.653+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38100cc1b0 con 0x7f38100d31f0 2024-05-30T16:43:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.653+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38100cc770 con 0x7f38100d31f0 2024-05-30T16:43:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.653+0000 7f7c53400640 1 --2- 172.21.15.67:0/1371599791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5415c470 0x7f7c5406fca0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7c540d0a60 tx=0x7f7c4402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bef810fc247f3f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.653+0000 7f6186600640 1 Processor -- start 2024-05-30T16:43:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.653+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d440689e0 con 0x7f2d4415a910 2024-05-30T16:43:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.653+0000 7fa28ea00640 1 -- 172.21.15.67:0/3627794165 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa288011d50 con 0x7fa2980b5950 2024-05-30T16:43:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7f6186600640 1 -- start start 2024-05-30T16:43:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7fab23e00640 1 -- 172.21.15.67:0/1363281475 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1803a070 con 0x7fab240b49e0 2024-05-30T16:43:59.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7f5b31c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c06e7e0 0x7f5b2c13e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7f5b31c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b2c0afee0 0x7f5b2c13edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7f5b31c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b2c0b4870 0x7f5b2c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7f5b31c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c06ec70 con 0x7f5b2c0afee0 2024-05-30T16:43:59.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7f5b31c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c06f7f0 con 0x7f5b2c0b4870 2024-05-30T16:43:59.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7fab23e00640 1 -- 172.21.15.67:0/1363281475 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab1800b650 con 0x7fab240b49e0 2024-05-30T16:43:59.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7fab23e00640 1 -- 172.21.15.67:0/1363281475 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1800ba20 con 0x7fab240b49e0 2024-05-30T16:43:59.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7f5b31c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c075220 con 0x7f5b2c06e7e0 2024-05-30T16:43:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.653+0000 7f9666a00640 1 -- 172.21.15.67:0/2213183941 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9670093400 con 0x7f96680e3a00 2024-05-30T16:43:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7f5b23e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b2c0afee0 0x7f5b2c13edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f5b23e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b2c0afee0 0x7f5b2c13edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43630/0 (socket says 172.21.15.67:43630) 2024-05-30T16:43:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f5b23e00640 1 -- 172.21.15.67:0/1162848088 learned_addr learned my addr 172.21.15.67:0/1162848088 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f5b2aa00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c06e7e0 0x7f5b2c13e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f5b2b400640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b2c0b4870 0x7f5b2c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f6186600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61800b4240 0x7f61801413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7fa28ea00640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa27806c920 0x7fa27806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f5b23e00640 1 -- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c06e7e0 msgr2=0x7f5b2c13e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7fa28ea00640 1 -- 172.21.15.67:0/3627794165 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa28808c050 con 0x7fa2980b5950 2024-05-30T16:43:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f6186600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6180162d80 0x7f618015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f6186600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6180163160 0x7f6180148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f6186600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6180163580 con 0x7f6180163160 2024-05-30T16:43:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f6186600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6180164440 con 0x7f6180162d80 2024-05-30T16:43:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f6186600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6180165970 con 0x7f61800b4240 2024-05-30T16:43:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f5b23e00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c06e7e0 0x7f5b2c13e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f5b23e00640 1 -- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b2c0b4870 msgr2=0x7f5b2c159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f5b23e00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b2c0b4870 0x7f5b2c159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f9666a00640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f965006c940 0x7f965006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f617f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61800b4240 0x7f61801413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f617fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6180163160 0x7f6180148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f617f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61800b4240 0x7f61801413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45534/0 (socket says 172.21.15.67:45534) 2024-05-30T16:43:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f617f400640 1 -- 172.21.15.67:0/588273955 learned_addr learned my addr 172.21.15.67:0/588273955 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7fa28ea00640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7fa278074ac0 0x7fa278076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7fa28ea00640 1 -- 172.21.15.67:0/3627794165 --> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa288025ea0 con 0x7fa278074ac0 2024-05-30T16:43:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7fa28ea00640 1 -- 172.21.15.67:0/3627794165 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa28808c4e0 con 0x7fa2980b5950 2024-05-30T16:43:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f5b23e00640 1 -- 172.21.15.67:0/1162848088 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c13f8c0 con 0x7f5b2c0afee0 2024-05-30T16:43:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f7c52000640 1 -- 172.21.15.67:0/1371599791 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44037070 con 0x7f7c5415c470 2024-05-30T16:43:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f7c52000640 1 -- 172.21.15.67:0/1371599791 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4402ae00 con 0x7f7c5415c470 2024-05-30T16:43:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.655+0000 7f7c52000640 1 -- 172.21.15.67:0/1371599791 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44032660 con 0x7f7c5415c470 2024-05-30T16:43:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f7c5a000640 1 -- 172.21.15.67:0/1371599791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5415c470 msgr2=0x7f7c5406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f7c5a000640 1 --2- 172.21.15.67:0/1371599791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5415c470 0x7f7c5406fca0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7c540d0a60 tx=0x7f7c4402a0b0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f5b2b400640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b2c0b4870 0x7f5b2c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:59.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f617ea00640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6180162d80 0x7f618015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7fa296000640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa27806c920 0x7fa27806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7fab2c600640 1 -- 172.21.15.67:0/1363281475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab240b49e0 msgr2=0x7fab240b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7fab2c600640 1 --2- 172.21.15.67:0/1363281475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab240b49e0 0x7fab240b4e70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fab240cff20 tx=0x7fab1802e400 comp rx=0 tx=0).stop 2024-05-30T16:43:59.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.654+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38100690f0 con 0x7f38100d31f0 2024-05-30T16:43:59.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f966ea00640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f965006c940 0x7f965006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f9666a00640 1 -- 172.21.15.67:0/2213183941 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9670107290 con 0x7f96680e3a00 2024-05-30T16:43:59.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f9666a00640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f9650074b40 0x7f9650076f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f9666a00640 1 -- 172.21.15.67:0/2213183941 --> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9670099e70 con 0x7f9650074b40 2024-05-30T16:43:59.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f9666a00640 1 -- 172.21.15.67:0/2213183941 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9670107720 con 0x7f96680e3a00 2024-05-30T16:43:59.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f5b2aa00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b2c06e7e0 0x7f5b2c13e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f617fe00640 1 -- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61800b4240 msgr2=0x7f61801413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.656+0000 7f5b23e00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b2c0afee0 0x7f5b2c13edc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b1402f040 tx=0x7f5b140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7fab2c600640 1 -- 172.21.15.67:0/1363281475 shutdown_connections 2024-05-30T16:43:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7fab2c600640 1 --2- 172.21.15.67:0/1363281475 >> v2:172.21.15.67:3300/0 conn(0x7fab2415b850 0x7fab240720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7fab2c600640 1 --2- 172.21.15.67:0/1363281475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab240b49e0 0x7fab240b4e70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7fab2c600640 1 --2- 172.21.15.67:0/1363281475 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 0x7fab240b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7fab2c600640 1 -- 172.21.15.67:0/1363281475 >> 172.21.15.67:0/1363281475 conn(0x7fab2406bae0 msgr2=0x7fab2406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f966e000640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f9650074b40 0x7f9650076f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f617fe00640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61800b4240 0x7f61801413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f617fe00640 1 -- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6180162d80 msgr2=0x7f618015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f617fe00640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6180162d80 0x7f618015de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f617fe00640 1 -- 172.21.15.67:0/588273955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6180148f10 con 0x7f6180163160 2024-05-30T16:43:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7fa297400640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7fa278074ac0 0x7fa278076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f617f400640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61800b4240 0x7f61801413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.658+0000 7f966ea00640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f965006c940 0x7f965006ee00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9660034000 tx=0x7f966002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.658+0000 7f617fe00640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6180163160 0x7f6180148450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f617800c730 tx=0x7f617800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f5b22a00640 1 -- 172.21.15.67:0/1162848088 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1404a070 con 0x7f5b2c0afee0 2024-05-30T16:43:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f5b22a00640 1 -- 172.21.15.67:0/1162848088 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b14004250 con 0x7f5b2c0afee0 2024-05-30T16:43:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f5b22a00640 1 -- 172.21.15.67:0/1162848088 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14003610 con 0x7f5b2c0afee0 2024-05-30T16:43:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.657+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2c13fba0 con 0x7f5b2c0afee0 2024-05-30T16:43:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.658+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2c140160 con 0x7f5b2c0afee0 2024-05-30T16:43:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.658+0000 7fa296000640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa27806c920 0x7fa27806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa284009630 tx=0x7fa284008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.658+0000 7f966e000640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f9650074b40 0x7f9650076f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7fa297400640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7fa278074ac0 0x7fa278076ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b2c140540 con 0x7f5b2c0afee0 2024-05-30T16:43:59.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f2d3be00640 1 -- 172.21.15.67:0/3239860807 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d34043070 con 0x7f2d4415a910 2024-05-30T16:43:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.660+0000 7f9666a00640 1 -- 172.21.15.67:0/2213183941 <== osd.3 v2:172.21.15.67:6814/2982751245 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9670099e70 con 0x7f9650074b40 2024-05-30T16:43:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f6177400640 1 -- 172.21.15.67:0/588273955 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178016020 con 0x7f6180163160 2024-05-30T16:43:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f6177400640 1 -- 172.21.15.67:0/588273955 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6178006d50 con 0x7f6180163160 2024-05-30T16:43:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f6177400640 1 -- 172.21.15.67:0/588273955 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178003bb0 con 0x7f6180163160 2024-05-30T16:43:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.660+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61801491f0 con 0x7f6180163160 2024-05-30T16:43:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.660+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618014c4f0 con 0x7f6180163160 2024-05-30T16:43:59.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f7c5a000640 1 -- 172.21.15.67:0/1371599791 shutdown_connections 2024-05-30T16:43:59.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f7c5a000640 1 --2- 172.21.15.67:0/1371599791 >> v2:172.21.15.169:3300/0 conn(0x7f7c54072200 0x7f7c54162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f7c5a000640 1 --2- 172.21.15.67:0/1371599791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c540701e0 0x7f7c54071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f7c5a000640 1 --2- 172.21.15.67:0/1371599791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5415c470 0x7f7c5406fca0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f7c5a000640 1 -- 172.21.15.67:0/1371599791 >> 172.21.15.67:0/1371599791 conn(0x7f7c5406bae0 msgr2=0x7f7c5406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.661+0000 7f5b22a00640 1 -- 172.21.15.67:0/1162848088 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b1404d070 con 0x7f5b2c0afee0 2024-05-30T16:43:59.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.660+0000 7fab2c600640 1 -- 172.21.15.67:0/1363281475 shutdown_connections 2024-05-30T16:43:59.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.661+0000 7fab2c600640 1 -- 172.21.15.67:0/1363281475 wait complete. 2024-05-30T16:43:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.660+0000 7fa28ea00640 1 -- 172.21.15.67:0/3627794165 <== osd.2 v2:172.21.15.67:6800/2549377238 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa288025ea0 con 0x7fa278074ac0 2024-05-30T16:43:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f7c5a000640 1 -- 172.21.15.67:0/1371599791 shutdown_connections 2024-05-30T16:43:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.659+0000 7f7c5a000640 1 -- 172.21.15.67:0/1371599791 wait complete. 2024-05-30T16:43:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.662+0000 7f7c5a000640 1 Processor -- start 2024-05-30T16:43:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.661+0000 7f2d3be00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d2406c8f0 0x7f2d2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.661+0000 7f2d3be00640 1 -- 172.21.15.67:0/3239860807 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2d340ab180 con 0x7f2d4415a910 2024-05-30T16:43:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.661+0000 7f2d3be00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f2d24074b70 0x7f2d24076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.661+0000 7f2d3be00640 1 -- 172.21.15.67:0/3239860807 --> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d340067f0 con 0x7f2d24074b70 2024-05-30T16:43:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.661+0000 7f2d3be00640 1 -- 172.21.15.67:0/3239860807 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2d340b0060 con 0x7f2d4415a910 2024-05-30T16:43:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.662+0000 7f2d43e00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d2406c8f0 0x7f2d2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.662+0000 7f7c5a000640 1 -- start start 2024-05-30T16:43:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.660+0000 7f380cc00640 1 -- 172.21.15.67:0/2241909677 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f380801a070 con 0x7f38100d31f0 2024-05-30T16:43:59.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.662+0000 7f2d43400640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f2d24074b70 0x7f2d24076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.662+0000 7f5b22a00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b0806c940 0x7f5b0806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f5b22a00640 1 -- 172.21.15.67:0/1162848088 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5b140b01a0 con 0x7f5b2c0afee0 2024-05-30T16:43:59.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f5b22a00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f5b08074ae0 0x7f5b08076f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f5b22a00640 1 -- 172.21.15.67:0/1162848088 --> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b140329b0 con 0x7f5b08074ae0 2024-05-30T16:43:59.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.661+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f61800689e0 con 0x7f6180163160 2024-05-30T16:43:59.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f7c5a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c540701e0 0x7f7c540b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f7c5a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c54072200 0x7f7c541e9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f7c5a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c5415c470 0x7f7c541e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f7c5a000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540b3380 con 0x7f7c540701e0 2024-05-30T16:43:59.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f7c5a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c541628c0 con 0x7f7c54072200 2024-05-30T16:43:59.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f7c5a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c540ce8e0 con 0x7f7c5415c470 2024-05-30T16:43:59.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f380cc00640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006caf0 0x7f37f006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f7c53400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c540701e0 0x7f7c540b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f7c53400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c540701e0 0x7f7c540b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43654/0 (socket says 172.21.15.67:43654) 2024-05-30T16:43:59.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f7c53400640 1 -- 172.21.15.67:0/3724551126 learned_addr learned my addr 172.21.15.67:0/3724551126 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f380cc00640 1 -- 172.21.15.67:0/2241909677 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f380808b2e0 con 0x7f38100d31f0 2024-05-30T16:43:59.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.663+0000 7f2d43e00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d2406c8f0 0x7f2d2406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0012e0 tx=0x7f2d3c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f380cc00640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f37f0074cf0 0x7f37f0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f380cc00640 1 -- 172.21.15.67:0/2241909677 --> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3808020e50 con 0x7f37f0074cf0 2024-05-30T16:43:59.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f2d43400640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f2d24074b70 0x7f2d24076f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f7c53e00640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c5415c470 0x7f7c541e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f7c53400640 1 -- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c5415c470 msgr2=0x7f7c541e9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f7c53400640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c5415c470 0x7f7c541e9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f7c53400640 1 -- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c54072200 msgr2=0x7f7c541e9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f7c53400640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c54072200 0x7f7c541e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f7c53400640 1 -- 172.21.15.67:0/3724551126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c540cebc0 con 0x7f7c540701e0 2024-05-30T16:43:59.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f7c53e00640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c5415c470 0x7f7c541e9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:43:59.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.664+0000 7f380fe00640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f37f0074cf0 0x7f37f0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.665+0000 7f380fe00640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f37f0074cf0 0x7f37f0077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.665+0000 7fab2c600640 1 Processor -- start 2024-05-30T16:43:59.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.666+0000 7fab2c600640 1 -- start start 2024-05-30T16:43:59.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.666+0000 7f2d3be00640 1 -- 172.21.15.67:0/3239860807 <== osd.0 v2:172.21.15.67:6823/274279214 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d340067f0 con 0x7f2d24074b70 2024-05-30T16:43:59.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.666+0000 7f380cc00640 1 -- 172.21.15.67:0/2241909677 <== osd.1 v2:172.21.15.67:6808/3260209820 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3808020e50 con 0x7f37f0074cf0 2024-05-30T16:43:59.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7f6177400640 1 -- 172.21.15.67:0/588273955 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61780050d0 con 0x7f6180163160 2024-05-30T16:43:59.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7f7c53400640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c540701e0 0x7f7c540b27d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c44000c00 tx=0x7f7c440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7fab2c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 0x7fab241ed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7fab2c600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab240b49e0 0x7fab2415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7f7c4b400640 1 -- 172.21.15.67:0/3724551126 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44042070 con 0x7f7c540701e0 2024-05-30T16:43:59.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7f7c4b400640 1 -- 172.21.15.67:0/3724551126 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4403e460 con 0x7f7c540701e0 2024-05-30T16:43:59.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7f7c4b400640 1 -- 172.21.15.67:0/3724551126 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44037040 con 0x7f7c540701e0 2024-05-30T16:43:59.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7f380cc00640 1 -- 172.21.15.67:0/2241909677 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3808092050 con 0x7f38100d31f0 2024-05-30T16:43:59.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7f3814a00640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006caf0 0x7f37f006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.667+0000 7fab29200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab240b49e0 0x7fab2415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7fab29c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 0x7fab241ed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7fab29200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab240b49e0 0x7fab2415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43666/0 (socket says 172.21.15.67:43666) 2024-05-30T16:43:59.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7fab29c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 0x7fab241ed0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45550/0 (socket says 172.21.15.67:45550) 2024-05-30T16:43:59.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7fab2c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab2415b850 0x7fab241f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7f6177400640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f616006c970 0x7f616006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7f5b2aa00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b0806c940 0x7f5b0806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7fab29200640 1 -- 172.21.15.67:0/868205773 learned_addr learned my addr 172.21.15.67:0/868205773 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:43:59.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7fab2c600640 1 -- 172.21.15.67:0/868205773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fab2415bcd0 con 0x7fab240b49e0 2024-05-30T16:43:59.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7fab2c600640 1 -- 172.21.15.67:0/868205773 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fab240b42b0 con 0x7fab2415b850 2024-05-30T16:43:59.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7fab2c600640 1 -- 172.21.15.67:0/868205773 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab241550d0 con 0x7fab24071cc0 2024-05-30T16:43:59.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7f5b2b400640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f5b08074ae0 0x7f5b08076f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.669+0000 7f5b22a00640 1 -- 172.21.15.67:0/1162848088 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5b140b7050 con 0x7f5b2c0afee0 2024-05-30T16:43:59.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.669+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c540ceea0 con 0x7f7c540701e0 2024-05-30T16:43:59.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7fab2a600640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab2415b850 0x7fab241f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.669+0000 7f3814a00640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006caf0 0x7f37f006efb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3804001eb0 tx=0x7f3804009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7f6177400640 1 -- 172.21.15.67:0/588273955 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f617808bf00 con 0x7f6180163160 2024-05-30T16:43:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7f6177400640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7f6160074b90 0x7f6160076fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7f6177400640 1 -- 172.21.15.67:0/588273955 --> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f617801ae50 con 0x7f6160074b90 2024-05-30T16:43:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.668+0000 7f6177400640 1 -- 172.21.15.67:0/588273955 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f617808c2b0 con 0x7f6180163160 2024-05-30T16:43:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.669+0000 7f617f400640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f616006c970 0x7f616006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.669+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c540cdb70 con 0x7f7c540701e0 2024-05-30T16:43:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.669+0000 7f5b2aa00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b0806c940 0x7f5b0806ee00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c007fd0 tx=0x7f5b1c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7fab2a600640 1 -- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 msgr2=0x7fab241ed0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7fab2a600640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 0x7fab241ed0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7fab2a600640 1 -- 172.21.15.67:0/868205773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab240b49e0 msgr2=0x7fab2415a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7fab2a600640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab240b49e0 0x7fab2415a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7fab2a600640 1 -- 172.21.15.67:0/868205773 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab24156310 con 0x7fab2415b850 2024-05-30T16:43:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7fab29200640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab240b49e0 0x7fab2415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7fab29c00640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab24071cc0 0x7fab241ed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:43:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7f617ea00640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7f6160074b90 0x7f6160076fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7fab2a600640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab2415b850 0x7fab241f3310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab18034000 tx=0x7fab180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7f5b2b400640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f5b08074ae0 0x7f5b08076f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.670+0000 7f617f400640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f616006c970 0x7f616006ee30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6170006fd0 tx=0x7f6170008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.671+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c540689e0 con 0x7f7c540701e0 2024-05-30T16:43:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.671+0000 7f617ea00640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7f6160074b90 0x7f6160076fb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.671+0000 7fab21600640 1 -- 172.21.15.67:0/868205773 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18045020 con 0x7fab2415b850 2024-05-30T16:43:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.671+0000 7fab21600640 1 -- 172.21.15.67:0/868205773 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab180357b0 con 0x7fab2415b850 2024-05-30T16:43:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.671+0000 7fab21600640 1 -- 172.21.15.67:0/868205773 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1803a040 con 0x7fab2415b850 2024-05-30T16:43:59.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.671+0000 7fab2c600640 1 -- 172.21.15.67:0/868205773 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab241565f0 con 0x7fab2415b850 2024-05-30T16:43:59.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.671+0000 7fab2c600640 1 -- 172.21.15.67:0/868205773 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab24156bb0 con 0x7fab2415b850 2024-05-30T16:43:59.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.672+0000 7fab16a00640 1 -- 172.21.15.67:0/868205773 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faaf8001030 con 0x7fab2415b850 2024-05-30T16:43:59.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.673+0000 7f5b22a00640 1 -- 172.21.15.67:0/1162848088 <== osd.4 v2:172.21.15.169:6812/432873034 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b140329b0 con 0x7f5b08074ae0 2024-05-30T16:43:59.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.673+0000 7f6177400640 1 -- 172.21.15.67:0/588273955 <== osd.5 v2:172.21.15.169:6824/3094236492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f617801ae50 con 0x7f6160074b90 2024-05-30T16:43:59.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.676+0000 7f7c4b400640 1 -- 172.21.15.67:0/3724551126 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c4403a040 con 0x7f7c540701e0 2024-05-30T16:43:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.678+0000 7fab21600640 1 -- 172.21.15.67:0/868205773 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab18048020 con 0x7fab2415b850 2024-05-30T16:43:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.678+0000 7fab21600640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab0406c9c0 0x7fab0406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.678+0000 7fab21600640 1 -- 172.21.15.67:0/868205773 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fab180b0c60 con 0x7fab2415b850 2024-05-30T16:43:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.679+0000 7fab21600640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7fab04074be0 0x7fab04077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.679+0000 7fab21600640 1 -- 172.21.15.67:0/868205773 --> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab040776b0 con 0x7fab04074be0 2024-05-30T16:43:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.679+0000 7fab21600640 1 -- 172.21.15.67:0/868205773 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fab180b1050 con 0x7fab2415b850 2024-05-30T16:43:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.679+0000 7fab29c00640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab0406c9c0 0x7fab0406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.679+0000 7f7c4b400640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c3406c940 0x7f7c3406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.680+0000 7fab29200640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7fab04074be0 0x7fab04077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.679+0000 7f7c4b400640 1 -- 172.21.15.67:0/3724551126 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7c440abc40 con 0x7f7c540701e0 2024-05-30T16:43:59.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.681+0000 7f7c52a00640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c3406c940 0x7f7c3406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.681+0000 7fab29c00640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab0406c9c0 0x7fab0406ee80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab1000b880 tx=0x7fab10008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.681+0000 7f7c4b400640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f7c34074b60 0x7f7c34076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:43:59.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.681+0000 7f7c4b400640 1 -- 172.21.15.67:0/3724551126 --> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c44033e50 con 0x7f7c34074b60 2024-05-30T16:43:59.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.681+0000 7f7c4b400640 1 -- 172.21.15.67:0/3724551126 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7c440ae390 con 0x7f7c540701e0 2024-05-30T16:43:59.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.682+0000 7fab29200640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7fab04074be0 0x7fab04077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.682+0000 7f7c53e00640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f7c34074b60 0x7f7c34076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:43:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.682+0000 7f7c52a00640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c3406c940 0x7f7c3406ee00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006b10 tx=0x7f7c40008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.683+0000 7f7c53e00640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f7c34074b60 0x7f7c34076f80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:43:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.684+0000 7fab21600640 1 -- 172.21.15.67:0/868205773 <== osd.6 v2:172.21.15.169:6800/2783087947 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab040776b0 con 0x7fab04074be0 2024-05-30T16:43:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.686+0000 7f7c4b400640 1 -- 172.21.15.67:0/3724551126 <== osd.7 v2:172.21.15.169:6808/4150327572 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c44033e50 con 0x7f7c34074b60 2024-05-30T16:43:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.694+0000 7fa29dc00640 1 -- 172.21.15.67:0/3627794165 --> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa2980d4f80 con 0x7fa278074ac0 2024-05-30T16:43:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.694+0000 7fa28ea00640 1 -- 172.21.15.67:0/3627794165 <== osd.2 v2:172.21.15.67:6800/2549377238 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa2980d4f80 con 0x7fa278074ac0 2024-05-30T16:43:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7fa283e00640 1 -- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7fa278074ac0 msgr2=0x7fa278076ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7fa283e00640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7fa278074ac0 0x7fa278076ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 --> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f96680057e0 con 0x7f9650074b40 2024-05-30T16:43:59.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7fa283e00640 1 -- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa27806c920 msgr2=0x7fa27806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7fa283e00640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa27806c920 0x7fa27806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa284009630 tx=0x7fa284008040 comp rx=0 tx=0).stop 2024-05-30T16:43:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7fa283e00640 1 -- 172.21.15.67:0/3627794165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5950 msgr2=0x7fa298158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7fa283e00640 1 --2- 172.21.15.67:0/3627794165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2980b5950 0x7fa298158860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa288004640 tx=0x7fa28800c8e0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7fa297400640 1 -- 172.21.15.67:0/3627794165 reap_dead start 2024-05-30T16:43:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7fa283e00640 1 -- 172.21.15.67:0/3627794165 shutdown_connections 2024-05-30T16:43:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.696+0000 7fa283e00640 1 -- 172.21.15.67:0/3627794165 >> 172.21.15.67:0/3627794165 conn(0x7fa29808bfc0 msgr2=0x7fa2980b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.698+0000 7fa283e00640 1 -- 172.21.15.67:0/3627794165 shutdown_connections 2024-05-30T16:43:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.698+0000 7fa283e00640 1 -- 172.21.15.67:0/3627794165 wait complete. 2024-05-30T16:43:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.698+0000 7f9666a00640 1 -- 172.21.15.67:0/2213183941 <== osd.3 v2:172.21.15.67:6814/2982751245 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f96680057e0 con 0x7f9650074b40 2024-05-30T16:43:59.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.699+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f9650074b40 msgr2=0x7f9650076f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.699+0000 7f966fe00640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f9650074b40 0x7f9650076f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.699+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f965006c940 msgr2=0x7f965006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.700+0000 7f966fe00640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f965006c940 0x7f965006ee00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9660034000 tx=0x7f966002e000 comp rx=0 tx=0).stop 2024-05-30T16:43:59.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.700+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680e3a00 msgr2=0x7f966804d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.700+0000 7f966fe00640 1 --2- 172.21.15.67:0/2213183941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96680e3a00 0x7f966804d520 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f967006a0f0 tx=0x7f9670069600 comp rx=0 tx=0).stop 2024-05-30T16:43:59.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.700+0000 7f966f400640 1 -- 172.21.15.67:0/2213183941 reap_dead start 2024-05-30T16:43:59.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.700+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 shutdown_connections 2024-05-30T16:43:59.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.700+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 --> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d440cc360 con 0x7f2d24074b70 2024-05-30T16:43:59.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.700+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 >> 172.21.15.67:0/2213183941 conn(0x7f96680537b0 msgr2=0x7f96680e5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.700+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 shutdown_connections 2024-05-30T16:43:59.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.700+0000 7f966fe00640 1 -- 172.21.15.67:0/2213183941 wait complete. 2024-05-30T16:43:59.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.701+0000 7f2d3be00640 1 -- 172.21.15.67:0/3239860807 <== osd.0 v2:172.21.15.67:6823/274279214 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d440cc360 con 0x7f2d24074b70 2024-05-30T16:43:59.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.702+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f2d24074b70 msgr2=0x7f2d24076f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.702+0000 7f2d4ac00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f2d24074b70 0x7f2d24076f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.701+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 --> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f38100ccbc0 con 0x7f37f0074cf0 2024-05-30T16:43:59.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.702+0000 7f380cc00640 1 -- 172.21.15.67:0/2241909677 <== osd.1 v2:172.21.15.67:6808/3260209820 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f38100ccbc0 con 0x7f37f0074cf0 2024-05-30T16:43:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.702+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d2406c8f0 msgr2=0x7f2d2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.702+0000 7f2d4ac00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d2406c8f0 0x7f2d2406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0012e0 tx=0x7f2d3c009040 comp rx=0 tx=0).stop 2024-05-30T16:43:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.702+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d4415a910 msgr2=0x7f2d4416ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.702+0000 7f2d4ac00640 1 --2- 172.21.15.67:0/3239860807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d4415a910 0x7f2d4416ccf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d4406c8e0 tx=0x7f2d340046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.702+0000 7f2d48c00640 1 -- 172.21.15.67:0/3239860807 reap_dead start 2024-05-30T16:43:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.703+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 shutdown_connections 2024-05-30T16:43:59.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.703+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 >> 172.21.15.67:0/3239860807 conn(0x7f2d44067ce0 msgr2=0x7f2d4406e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.703+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f37f0074cf0 msgr2=0x7f37f0077110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.703+0000 7f3817400640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f37f0074cf0 0x7f37f0077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.703+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 shutdown_connections 2024-05-30T16:43:59.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.704+0000 7f2d4ac00640 1 -- 172.21.15.67:0/3239860807 wait complete. 2024-05-30T16:43:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.704+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006caf0 msgr2=0x7f37f006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.704+0000 7f3817400640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006caf0 0x7f37f006efb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3804001eb0 tx=0x7f3804009040 comp rx=0 tx=0).stop 2024-05-30T16:43:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.704+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38100d31f0 msgr2=0x7f38100d4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.704+0000 7f3817400640 1 --2- 172.21.15.67:0/2241909677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38100d31f0 0x7f38100d4f40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f380800caa0 tx=0x7f380800cf60 comp rx=0 tx=0).stop 2024-05-30T16:43:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.704+0000 7f3815400640 1 -- 172.21.15.67:0/2241909677 reap_dead start 2024-05-30T16:43:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.705+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 shutdown_connections 2024-05-30T16:43:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.705+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 >> 172.21.15.67:0/2241909677 conn(0x7f381008bfc0 msgr2=0x7f38100b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.705+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 shutdown_connections 2024-05-30T16:43:59.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.705+0000 7f3817400640 1 -- 172.21.15.67:0/2241909677 wait complete. 2024-05-30T16:43:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.706+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 --> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b2c06bfa0 con 0x7f5b08074ae0 2024-05-30T16:43:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.707+0000 7f5b22a00640 1 -- 172.21.15.67:0/1162848088 <== osd.4 v2:172.21.15.169:6812/432873034 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5b2c06bfa0 con 0x7f5b08074ae0 2024-05-30T16:43:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.709+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f5b08074ae0 msgr2=0x7f5b08076f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.709+0000 7f5b31c00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f5b08074ae0 0x7f5b08076f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.711+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b0806c940 msgr2=0x7f5b0806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.711+0000 7f5b31c00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b0806c940 0x7f5b0806ee00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c007fd0 tx=0x7f5b1c009040 comp rx=0 tx=0).stop 2024-05-30T16:43:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.711+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b2c0afee0 msgr2=0x7f5b2c13edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.711+0000 7f5b31c00640 1 --2- 172.21.15.67:0/1162848088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b2c0afee0 0x7f5b2c13edc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b1402f040 tx=0x7f5b140046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.711+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 --> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f61800af4f0 con 0x7f6160074b90 2024-05-30T16:43:59.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.711+0000 7f5b2b400640 1 -- 172.21.15.67:0/1162848088 reap_dead start 2024-05-30T16:43:59.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.712+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 shutdown_connections 2024-05-30T16:43:59.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.712+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 >> 172.21.15.67:0/1162848088 conn(0x7f5b2c06b900 msgr2=0x7f5b2c074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.712+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 shutdown_connections 2024-05-30T16:43:59.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.712+0000 7f5b31c00640 1 -- 172.21.15.67:0/1162848088 wait complete. 2024-05-30T16:43:59.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.713+0000 7f6177400640 1 -- 172.21.15.67:0/588273955 <== osd.5 v2:172.21.15.169:6824/3094236492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f61800af4f0 con 0x7f6160074b90 2024-05-30T16:43:59.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.718+0000 7fab16a00640 1 -- 172.21.15.67:0/868205773 --> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faaf8002dd0 con 0x7fab04074be0 2024-05-30T16:43:59.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.716+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 --> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7c540b3780 con 0x7f7c34074b60 2024-05-30T16:43:59.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.719+0000 7fab21600640 1 -- 172.21.15.67:0/868205773 <== osd.6 v2:172.21.15.169:6800/2783087947 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faaf8002dd0 con 0x7fab04074be0 2024-05-30T16:43:59.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.719+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7f6160074b90 msgr2=0x7f6160076fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.719+0000 7f6186600640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7f6160074b90 0x7f6160076fb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.719+0000 7f7c4b400640 1 -- 172.21.15.67:0/3724551126 <== osd.7 v2:172.21.15.169:6808/4150327572 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7c540b3780 con 0x7f7c34074b60 2024-05-30T16:43:59.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f7c34074b60 msgr2=0x7f7c34076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f7c5a000640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f7c34074b60 0x7f7c34076f80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f616006c970 msgr2=0x7f616006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f6186600640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f616006c970 0x7f616006ee30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6170006fd0 tx=0x7f6170008040 comp rx=0 tx=0).stop 2024-05-30T16:43:59.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6180163160 msgr2=0x7f6180148450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f6186600640 1 --2- 172.21.15.67:0/588273955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6180163160 0x7f6180148450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f617800c730 tx=0x7f617800cbf0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f617fe00640 1 -- 172.21.15.67:0/588273955 reap_dead start 2024-05-30T16:43:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c3406c940 msgr2=0x7f7c3406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f7c5a000640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c3406c940 0x7f7c3406ee00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c40006b10 tx=0x7f7c40008040 comp rx=0 tx=0).stop 2024-05-30T16:43:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c540701e0 msgr2=0x7f7c540b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f7c5a000640 1 --2- 172.21.15.67:0/3724551126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c540701e0 0x7f7c540b27d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c44000c00 tx=0x7f7c440046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.720+0000 7f7c53e00640 1 -- 172.21.15.67:0/3724551126 reap_dead start 2024-05-30T16:43:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 shutdown_connections 2024-05-30T16:43:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 >> 172.21.15.67:0/3724551126 conn(0x7f7c5406bae0 msgr2=0x7f7c54162160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 shutdown_connections 2024-05-30T16:43:59.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 >> 172.21.15.67:0/588273955 conn(0x7f6180067ce0 msgr2=0x7f618006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7fab16a00640 1 -- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7fab04074be0 msgr2=0x7fab04077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7fab16a00640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7fab04074be0 0x7fab04077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 shutdown_connections 2024-05-30T16:43:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7f7c5a000640 1 -- 172.21.15.67:0/3724551126 wait complete. 2024-05-30T16:43:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 shutdown_connections 2024-05-30T16:43:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7f6186600640 1 -- 172.21.15.67:0/588273955 wait complete. 2024-05-30T16:43:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.721+0000 7fab16a00640 1 -- 172.21.15.67:0/868205773 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab0406c9c0 msgr2=0x7fab0406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.722+0000 7fab16a00640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab0406c9c0 0x7fab0406ee80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab1000b880 tx=0x7fab10008040 comp rx=0 tx=0).stop 2024-05-30T16:43:59.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.722+0000 7fab16a00640 1 -- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab2415b850 msgr2=0x7fab241f3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:43:59.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.722+0000 7fab16a00640 1 --2- 172.21.15.67:0/868205773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab2415b850 0x7fab241f3310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab18034000 tx=0x7fab180046c0 comp rx=0 tx=0).stop 2024-05-30T16:43:59.740 INFO:teuthology.orchestra.run.smithi067.stdout:64424509447 2024-05-30T16:43:59.740 INFO:teuthology.orchestra.run.smithi067.stdout:64424509447 2024-05-30T16:43:59.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.726+0000 7fab2a600640 1 -- 172.21.15.67:0/868205773 reap_dead start 2024-05-30T16:43:59.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.726+0000 7fab16a00640 1 -- 172.21.15.67:0/868205773 shutdown_connections 2024-05-30T16:43:59.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.726+0000 7fab16a00640 1 -- 172.21.15.67:0/868205773 >> 172.21.15.67:0/868205773 conn(0x7fab2406bae0 msgr2=0x7fab2415ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:43:59.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.726+0000 7fab16a00640 1 -- 172.21.15.67:0/868205773 shutdown_connections 2024-05-30T16:43:59.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:43:59.726+0000 7fab16a00640 1 -- 172.21.15.67:0/868205773 wait complete. 2024-05-30T16:43:59.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-30T16:43:59.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-30T16:43:59.743 INFO:teuthology.orchestra.run.smithi067.stdout:64424509446 2024-05-30T16:43:59.743 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-30T16:43:59.744 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-05-30T16:43:59.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-30T16:43:59.745 INFO:teuthology.orchestra.run.smithi067.stdout:64424509446 2024-05-30T16:43:59.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-30T16:43:59.746 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2024-05-30T16:43:59.747 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-30T16:43:59.747 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-05-30T16:43:59.747 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-30T16:43:59.748 INFO:teuthology.orchestra.run.smithi067.stdout:64424509446 2024-05-30T16:43:59.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-30T16:44:00.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.025+0000 7f2e50200640 1 Processor -- start 2024-05-30T16:44:00.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.025+0000 7f2e50200640 1 -- start start 2024-05-30T16:44:00.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.027+0000 7f2e50200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2e400e30b0 0x7f2e40057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.027+0000 7f2e50200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e4003e830 0x7f2e40053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.027+0000 7f2e50200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e400582c0 con 0x7f2e40054120 2024-05-30T16:44:00.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.027+0000 7f2e50200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2e40054550 con 0x7f2e400e30b0 2024-05-30T16:44:00.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.027+0000 7f2e50200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2e40058560 con 0x7f2e4003e830 2024-05-30T16:44:00.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.028+0000 7f2e4d800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2e400e30b0 0x7f2e40057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.031+0000 7f2481600640 1 Processor -- start 2024-05-30T16:44:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.031+0000 7f2481600640 1 -- start start 2024-05-30T16:44:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.028+0000 7f2e4d800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2e400e30b0 0x7f2e40057d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43676/0 (socket says 172.21.15.67:43676) 2024-05-30T16:44:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.028+0000 7f2e4d800640 1 -- 172.21.15.67:0/1464812958 learned_addr learned my addr 172.21.15.67:0/1464812958 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.029+0000 7f2e47e00640 1 -- 172.21.15.67:0/1464812958 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2042689542 0 0) 0x7f2e400582c0 con 0x7f2e40054120 2024-05-30T16:44:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.030+0000 7f2e47e00640 1 -- 172.21.15.67:0/1464812958 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e30003520 con 0x7f2e40054120 2024-05-30T16:44:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.030+0000 7f2e4d800640 1 -- 172.21.15.67:0/1464812958 >> v2:172.21.15.169:3300/0 conn(0x7f2e4003e830 msgr2=0x7f2e40053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.030+0000 7f2e4ce00640 1 --2- 172.21.15.67:0/1464812958 >> v2:172.21.15.169:3300/0 conn(0x7f2e4003e830 0x7f2e40053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.030+0000 7f2e47e00640 1 -- 172.21.15.67:0/1464812958 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1016124732 0 0) 0x7f2e30003520 con 0x7f2e40054120 2024-05-30T16:44:00.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.030+0000 7f2e4d800640 1 --2- 172.21.15.67:0/1464812958 >> v2:172.21.15.169:3300/0 conn(0x7f2e4003e830 0x7f2e40053be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.030+0000 7f2e4d800640 1 -- 172.21.15.67:0/1464812958 >> v1:172.21.15.67:6789/0 conn(0x7f2e40054120 legacy=0x7f2e400e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.030+0000 7f2e4d800640 1 -- 172.21.15.67:0/1464812958 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e40058840 con 0x7f2e400e30b0 2024-05-30T16:44:00.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.031+0000 7f2e4ce00640 1 --2- 172.21.15.67:0/1464812958 >> v2:172.21.15.169:3300/0 conn(0x7f2e4003e830 0x7f2e40053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:00.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.033+0000 7f2481600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f24740ead00 0x7f247404fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.033+0000 7f2481600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2474006ea0 0x7f2474007330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.033+0000 7f2481600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2474053720 con 0x7f247404ffd0 2024-05-30T16:44:00.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.033+0000 7f2481600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2474054330 con 0x7f24740ead00 2024-05-30T16:44:00.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.033+0000 7f2481600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2474053a00 con 0x7f2474006ea0 2024-05-30T16:44:00.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.033+0000 7f2e4d800640 1 --2- 172.21.15.67:0/1464812958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e400e30b0 0x7f2e40057d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e400e7cc0 tx=0x7f2e3402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855fb4f20744784e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.033+0000 7f2e47e00640 1 -- 172.21.15.67:0/1464812958 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34037070 con 0x7f2e400e30b0 2024-05-30T16:44:00.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.033+0000 7f2e47e00640 1 -- 172.21.15.67:0/1464812958 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3402ad30 con 0x7f2e400e30b0 2024-05-30T16:44:00.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.033+0000 7f2e47e00640 1 -- 172.21.15.67:0/1464812958 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34037070 con 0x7f2e400e30b0 2024-05-30T16:44:00.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.034+0000 7f247b400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f247404ffd0 0x7f2474037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:53836/0 (socket says 172.21.15.67:53836) 2024-05-30T16:44:00.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.034+0000 7f247b400640 1 -- 172.21.15.67:0/359868086 learned_addr learned my addr 172.21.15.67:0/359868086 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:00.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.034+0000 7f2480c00640 1 --2- 172.21.15.67:0/359868086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2474006ea0 0x7f2474007330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.034+0000 7f247be00640 1 --2- 172.21.15.67:0/359868086 >> v2:172.21.15.169:3300/0 conn(0x7f24740ead00 0x7f247404fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.035+0000 7f2e50200640 1 -- 172.21.15.67:0/1464812958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e400e30b0 msgr2=0x7f2e40057d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.035+0000 7f2e50200640 1 --2- 172.21.15.67:0/1464812958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e400e30b0 0x7f2e40057d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e400e7cc0 tx=0x7f2e3402a2a0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.035+0000 7f5b4ec00640 1 Processor -- start 2024-05-30T16:44:00.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.035+0000 7f247aa00640 1 -- 172.21.15.67:0/359868086 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849382599 0 0) 0x7f2474053720 con 0x7f247404ffd0 2024-05-30T16:44:00.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.035+0000 7f2e50200640 1 -- 172.21.15.67:0/1464812958 shutdown_connections 2024-05-30T16:44:00.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.035+0000 7f5b4ec00640 1 -- start start 2024-05-30T16:44:00.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.035+0000 7f2e50200640 1 --2- 172.21.15.67:0/1464812958 >> v2:172.21.15.169:3300/0 conn(0x7f2e4003e830 0x7f2e40053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.036+0000 7f247aa00640 1 -- 172.21.15.67:0/359868086 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2464003520 con 0x7f247404ffd0 2024-05-30T16:44:00.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.035+0000 7f2e50200640 1 --2- 172.21.15.67:0/1464812958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e400e30b0 0x7f2e40057d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.035+0000 7f2e50200640 1 -- 172.21.15.67:0/1464812958 >> 172.21.15.67:0/1464812958 conn(0x7f2e400559d0 msgr2=0x7f2e40055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.036+0000 7f2480c00640 1 -- 172.21.15.67:0/359868086 >> v2:172.21.15.169:3300/0 conn(0x7f24740ead00 msgr2=0x7f247404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.036+0000 7f2480c00640 1 --2- 172.21.15.67:0/359868086 >> v2:172.21.15.169:3300/0 conn(0x7f24740ead00 0x7f247404fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.036+0000 7f2480c00640 1 -- 172.21.15.67:0/359868086 >> v1:172.21.15.169:6789/0 conn(0x7f247404ffd0 legacy=0x7f2474037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.036+0000 7f2e50200640 1 -- 172.21.15.67:0/1464812958 shutdown_connections 2024-05-30T16:44:00.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.036+0000 7f2480c00640 1 -- 172.21.15.67:0/359868086 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2474053ce0 con 0x7f2474006ea0 2024-05-30T16:44:00.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.036+0000 7f2e50200640 1 -- 172.21.15.67:0/1464812958 wait complete. 2024-05-30T16:44:00.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.038+0000 7f5b4ec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5b4003e840 0x7f5b40053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.039+0000 7f2e50200640 1 Processor -- start 2024-05-30T16:44:00.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.039+0000 7f5b4ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 0x7f5b400068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.039+0000 7f2480c00640 1 --2- 172.21.15.67:0/359868086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2474006ea0 0x7f2474007330 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f247c02db50 tx=0x7f247c0a5b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8b666357b0a5abf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.039+0000 7f5b4d800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5b400e30c0 0x7f5b40057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:53846/0 (socket says 172.21.15.67:53846) 2024-05-30T16:44:00.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.039+0000 7f5b4d800640 1 -- 172.21.15.67:0/3297888947 learned_addr learned my addr 172.21.15.67:0/3297888947 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:00.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.039+0000 7f5b4ce00640 1 --2- 172.21.15.67:0/3297888947 >> v2:172.21.15.169:3300/0 conn(0x7f5b4003e840 0x7f5b40053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.039+0000 7f2e50200640 1 -- start start 2024-05-30T16:44:00.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.039+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3297888947 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b40058570 con 0x7f5b400e30c0 2024-05-30T16:44:00.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.039+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3297888947 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5b40058850 con 0x7f5b4003e840 2024-05-30T16:44:00.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.040+0000 7f247aa00640 1 -- 172.21.15.67:0/359868086 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0a7920 con 0x7f2474006ea0 2024-05-30T16:44:00.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.040+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3297888947 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b40058b30 con 0x7f5b40054130 2024-05-30T16:44:00.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.040+0000 7f5b4e200640 1 --2- 172.21.15.67:0/3297888947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 0x7f5b400068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.040+0000 7f8308a00640 1 Processor -- start 2024-05-30T16:44:00.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.040+0000 7f8308a00640 1 -- start start 2024-05-30T16:44:00.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.040+0000 7f2e50200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4003e830 0x7f2e400d7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2e50200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40054120 0x7f2e400db040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2e50200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e400e30b0 0x7f2e40051c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2e50200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4003bfb0 con 0x7f2e40054120 2024-05-30T16:44:00.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2e50200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e400063a0 con 0x7f2e4003e830 2024-05-30T16:44:00.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2e50200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2e400e3490 con 0x7f2e400e30b0 2024-05-30T16:44:00.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.040+0000 7f5b47e00640 1 -- 172.21.15.67:0/3297888947 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1961950820 0 0) 0x7f5b40058570 con 0x7f5b400e30c0 2024-05-30T16:44:00.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f5b47e00640 1 -- 172.21.15.67:0/3297888947 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b30003520 con 0x7f5b400e30c0 2024-05-30T16:44:00.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f5b4ce00640 1 -- 172.21.15.67:0/3297888947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 msgr2=0x7f5b400068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f5b4ce00640 1 --2- 172.21.15.67:0/3297888947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 0x7f5b400068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f5b4ce00640 1 -- 172.21.15.67:0/3297888947 >> v1:172.21.15.169:6789/0 conn(0x7f5b400e30c0 legacy=0x7f5b40057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f5b4ce00640 1 -- 172.21.15.67:0/3297888947 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b400063b0 con 0x7f5b4003e840 2024-05-30T16:44:00.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2e4d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4003e830 0x7f2e400d7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2e4d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4003e830 0x7f2e400d7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:34882/0 (socket says 172.21.15.67:34882) 2024-05-30T16:44:00.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2e4d800640 1 -- 172.21.15.67:0/1074743711 learned_addr learned my addr 172.21.15.67:0/1074743711 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f2e4ce00640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40054120 0x7f2e400db040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f2e4e200640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e400e30b0 0x7f2e40051c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.040+0000 7f247aa00640 1 -- 172.21.15.67:0/359868086 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c0a7ac0 con 0x7f2474006ea0 2024-05-30T16:44:00.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.040+0000 7f247aa00640 1 -- 172.21.15.67:0/359868086 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0a8430 con 0x7f2474006ea0 2024-05-30T16:44:00.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2481600640 1 -- 172.21.15.67:0/359868086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2474006ea0 msgr2=0x7f2474007330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.041+0000 7f2481600640 1 --2- 172.21.15.67:0/359868086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2474006ea0 0x7f2474007330 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f247c02db50 tx=0x7f247c0a5b00 comp rx=0 tx=0).stop 2024-05-30T16:44:00.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f5b4ce00640 1 --2- 172.21.15.67:0/3297888947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4003e840 0x7f5b40053bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b4802db50 tx=0x7f5b480a5be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5f358641e968753 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f2481600640 1 -- 172.21.15.67:0/359868086 shutdown_connections 2024-05-30T16:44:00.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f8308a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040d1140 0x7f83040b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f8308a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83040d0230 0x7f83040b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f8308a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83040cdcf0 con 0x7f83040b28c0 2024-05-30T16:44:00.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f8308a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f830409aa30 con 0x7f83040d0230 2024-05-30T16:44:00.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f8308a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f83040cdfd0 con 0x7f83040d1140 2024-05-30T16:44:00.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f2481600640 1 --2- 172.21.15.67:0/359868086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2474006ea0 0x7f2474007330 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.043+0000 7f2e4d800640 1 -- 172.21.15.67:0/1074743711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e400e30b0 msgr2=0x7f2e40051c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.043+0000 7f2e4d800640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e400e30b0 0x7f2e40051c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.043+0000 7f2e4d800640 1 -- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40054120 msgr2=0x7f2e400db040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.043+0000 7f2e4d800640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40054120 0x7f2e400db040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.043+0000 7f2e4d800640 1 -- 172.21.15.67:0/1074743711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4004ff90 con 0x7f2e4003e830 2024-05-30T16:44:00.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.043+0000 7f5b47e00640 1 -- 172.21.15.67:0/3297888947 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480a79c0 con 0x7f5b4003e840 2024-05-30T16:44:00.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.043+0000 7f5b47e00640 1 -- 172.21.15.67:0/3297888947 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b480a7b60 con 0x7f5b4003e840 2024-05-30T16:44:00.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f2481600640 1 --2- 172.21.15.67:0/359868086 >> v2:172.21.15.169:3300/0 conn(0x7f24740ead00 0x7f247404fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f2481600640 1 -- 172.21.15.67:0/359868086 >> 172.21.15.67:0/359868086 conn(0x7f2474009fb0 msgr2=0x7f247400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f2481600640 1 -- 172.21.15.67:0/359868086 shutdown_connections 2024-05-30T16:44:00.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.042+0000 7f2481600640 1 -- 172.21.15.67:0/359868086 wait complete. 2024-05-30T16:44:00.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.043+0000 7f8301400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040d1140 0x7f83040b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f8301400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040d1140 0x7f83040b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45584/0 (socket says 172.21.15.67:45584) 2024-05-30T16:44:00.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f8301400640 1 -- 172.21.15.67:0/2070784996 learned_addr learned my addr 172.21.15.67:0/2070784996 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.043+0000 7f5b47e00640 1 -- 172.21.15.67:0/3297888947 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480af420 con 0x7f5b4003e840 2024-05-30T16:44:00.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f2e4d800640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4003e830 0x7f2e400d7110 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e340090b0 tx=0x7f2e340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3297888947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4003e840 msgr2=0x7f5b40053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3297888947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4003e840 0x7f5b40053bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b4802db50 tx=0x7f5b480a5be0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3297888947 shutdown_connections 2024-05-30T16:44:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.045+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3297888947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 0x7f5b400068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.045+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3297888947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4003e840 0x7f5b40053bf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.045+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3297888947 >> 172.21.15.67:0/3297888947 conn(0x7f5b400559e0 msgr2=0x7f5b40055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.045+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3297888947 shutdown_connections 2024-05-30T16:44:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f8302800640 1 --2- 172.21.15.67:0/2070784996 >> v2:172.21.15.169:3300/0 conn(0x7f83040d0230 0x7f83040b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f8301400640 1 -- 172.21.15.67:0/2070784996 >> v2:172.21.15.169:3300/0 conn(0x7f83040d0230 msgr2=0x7f83040b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f8301400640 1 --2- 172.21.15.67:0/2070784996 >> v2:172.21.15.169:3300/0 conn(0x7f83040d0230 0x7f83040b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f8301400640 1 -- 172.21.15.67:0/2070784996 >> v1:172.21.15.67:6789/0 conn(0x7f83040b28c0 legacy=0x7f83040d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.044+0000 7f8301400640 1 -- 172.21.15.67:0/2070784996 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83040ce2b0 con 0x7f83040d1140 2024-05-30T16:44:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.045+0000 7f2e45600640 1 -- 172.21.15.67:0/1074743711 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34037040 con 0x7f2e4003e830 2024-05-30T16:44:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.045+0000 7f2e45600640 1 -- 172.21.15.67:0/1074743711 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e34042070 con 0x7f2e4003e830 2024-05-30T16:44:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.045+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e40050270 con 0x7f2e4003e830 2024-05-30T16:44:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.045+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e40050830 con 0x7f2e4003e830 2024-05-30T16:44:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.046+0000 7f2e45600640 1 -- 172.21.15.67:0/1074743711 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34037040 con 0x7f2e4003e830 2024-05-30T16:44:00.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.045+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3297888947 wait complete. 2024-05-30T16:44:00.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.047+0000 7f2481600640 1 Processor -- start 2024-05-30T16:44:00.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.047+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e400057d0 con 0x7f2e4003e830 2024-05-30T16:44:00.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.047+0000 7f8301400640 1 --2- 172.21.15.67:0/2070784996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040d1140 0x7f83040b2fe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009870 tx=0x7f82f002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d33455c4d6e0e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.047+0000 7f2481600640 1 -- start start 2024-05-30T16:44:00.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.048+0000 7f5b4ec00640 1 Processor -- start 2024-05-30T16:44:00.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.048+0000 7f8300a00640 1 -- 172.21.15.67:0/2070784996 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0037070 con 0x7f83040d1140 2024-05-30T16:44:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.048+0000 7f8300a00640 1 -- 172.21.15.67:0/2070784996 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f0032440 con 0x7f83040d1140 2024-05-30T16:44:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.048+0000 7f8300a00640 1 -- 172.21.15.67:0/2070784996 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0032810 con 0x7f83040d1140 2024-05-30T16:44:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.048+0000 7f5b4ec00640 1 -- start start 2024-05-30T16:44:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.048+0000 7fd44d600640 1 Processor -- start 2024-05-30T16:44:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.048+0000 7fd44d600640 1 -- start start 2024-05-30T16:44:00.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f2481600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2474006ea0 0x7f24740448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f2481600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f247404ffd0 0x7f2474046020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f2481600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24740ead00 0x7f247404dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f6f14800640 1 Processor -- start 2024-05-30T16:44:00.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f2481600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2474054940 con 0x7f2474006ea0 2024-05-30T16:44:00.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f2481600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f24740547c0 con 0x7f24740ead00 2024-05-30T16:44:00.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f2481600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f247404f550 con 0x7f247404ffd0 2024-05-30T16:44:00.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f8308a00640 1 -- 172.21.15.67:0/2070784996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040d1140 msgr2=0x7f83040b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f8308a00640 1 --2- 172.21.15.67:0/2070784996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040d1140 0x7f83040b2fe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009870 tx=0x7f82f002a250 comp rx=0 tx=0).stop 2024-05-30T16:44:00.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f8308a00640 1 -- 172.21.15.67:0/2070784996 shutdown_connections 2024-05-30T16:44:00.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f8308a00640 1 --2- 172.21.15.67:0/2070784996 >> v2:172.21.15.169:3300/0 conn(0x7f83040d0230 0x7f83040b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f8308a00640 1 --2- 172.21.15.67:0/2070784996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040d1140 0x7f83040b2fe0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f8308a00640 1 -- 172.21.15.67:0/2070784996 >> 172.21.15.67:0/2070784996 conn(0x7f83040b3f40 msgr2=0x7f83040b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.050+0000 7f8308a00640 1 -- 172.21.15.67:0/2070784996 shutdown_connections 2024-05-30T16:44:00.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.050+0000 7f8308a00640 1 -- 172.21.15.67:0/2070784996 wait complete. 2024-05-30T16:44:00.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.050+0000 7f2480c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24740ead00 0x7f247404dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.050+0000 7f2480c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24740ead00 0x7f247404dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:34904/0 (socket says 172.21.15.67:34904) 2024-05-30T16:44:00.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.050+0000 7f2480c00640 1 -- 172.21.15.67:0/3202468108 learned_addr learned my addr 172.21.15.67:0/3202468108 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.050+0000 7f2480c00640 1 -- 172.21.15.67:0/3202468108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f247404ffd0 msgr2=0x7f2474046020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-30T16:44:00.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.050+0000 7f5b4ec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b4003e840 0x7f5b400e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.050+0000 7f2e45600640 1 -- 172.21.15.67:0/1074743711 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e34047460 con 0x7f2e4003e830 2024-05-30T16:44:00.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7f5b4d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b4003e840 0x7f5b400e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7f5b4d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b4003e840 0x7f5b400e42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43704/0 (socket says 172.21.15.67:43704) 2024-05-30T16:44:00.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7fd44d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd4480d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7f5b4d800640 1 -- 172.21.15.67:0/3973195257 learned_addr learned my addr 172.21.15.67:0/3973195257 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7fd44d600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd4480ad130 0x7fd4480ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 0x7f5b400e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.049+0000 7f6f14800640 1 -- start start 2024-05-30T16:44:00.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7f6f14800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f0c0d04a0 0x7f6f0c0d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7f6f14800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c06f6e0 0x7f6f0c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7f6f14800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f0c0b4350 con 0x7f6f0c162e70 2024-05-30T16:44:00.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7f6f14800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0c0b3ec0 con 0x7f6f0c0d04a0 2024-05-30T16:44:00.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.051+0000 7f6f14800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c0b32d0 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7f8308a00640 1 Processor -- start 2024-05-30T16:44:00.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7fd44d600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd44806ed10 con 0x7fd44806f860 2024-05-30T16:44:00.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7fd446a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd44806f860 0x7fd4480b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35370/0 (socket says 172.21.15.67:35370) 2024-05-30T16:44:00.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7f2e45600640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e2806caf0 0x7f2e2806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7fd446a00640 1 -- 172.21.15.67:0/2996485131 learned_addr learned my addr 172.21.15.67:0/2996485131 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:00.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b400e30c0 0x7f5b4017ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7f2e45600640 1 -- 172.21.15.67:0/1074743711 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2e340ac790 con 0x7f2e4003e830 2024-05-30T16:44:00.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7fd44d600640 1 -- 172.21.15.67:0/2996485131 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd4480d0e40 con 0x7fd4480ad130 2024-05-30T16:44:00.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7fd44d600640 1 -- 172.21.15.67:0/2996485131 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd44815cb00 con 0x7fd4480d04a0 2024-05-30T16:44:00.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7f5b4ce00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 0x7f5b400e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7f6f11e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f0c0d04a0 0x7f6f0c0d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40006e10 con 0x7f5b4003e840 2024-05-30T16:44:00.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b400e34a0 con 0x7f5b400e30c0 2024-05-30T16:44:00.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f6f11e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f0c0d04a0 0x7f6f0c0d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43720/0 (socket says 172.21.15.67:43720) 2024-05-30T16:44:00.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f6f11e00640 1 -- 172.21.15.67:0/2923071700 learned_addr learned my addr 172.21.15.67:0/2923071700 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b40006f90 con 0x7f5b40054130 2024-05-30T16:44:00.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f247be00640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2474006ea0 0x7f24740448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f2480c00640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f247404ffd0 0x7f2474046020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f2480c00640 1 -- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2474006ea0 msgr2=0x7f24740448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3332168175 0 0) 0x7fd440001490 con 0x7fd44806f860 2024-05-30T16:44:00.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f5b4e200640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b400e30c0 0x7f5b4017ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f2480c00640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2474006ea0 0x7f24740448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f2480c00640 1 -- 172.21.15.67:0/3202468108 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247404e4f0 con 0x7f24740ead00 2024-05-30T16:44:00.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f247be00640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2474006ea0 0x7f24740448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:00.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd42c003520 con 0x7fd44806f860 2024-05-30T16:44:00.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f6f11400640 1 --2- 172.21.15.67:0/2923071700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c06f6e0 0x7f6f0c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.052+0000 7f8308a00640 1 -- start start 2024-05-30T16:44:00.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7f8308a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040b28c0 0x7f830409c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f2480c00640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24740ead00 0x7f247404dfb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f247c051690 tx=0x7f247c075b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f8308a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83040d0230 0x7f830409cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7fd447400640 1 --2- 172.21.15.67:0/2996485131 >> v2:172.21.15.169:3300/0 conn(0x7fd4480ad130 0x7fd4480ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.053+0000 7fd446000640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd4480d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f5b4d800640 1 -- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 msgr2=0x7f5b400e3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f6f11e00640 1 -- 172.21.15.67:0/2923071700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c06f6e0 msgr2=0x7f6f0c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f6f11e00640 1 --2- 172.21.15.67:0/2923071700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c06f6e0 0x7f6f0c0cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f6f11e00640 1 -- 172.21.15.67:0/2923071700 >> v1:172.21.15.169:6789/0 conn(0x7f6f0c162e70 legacy=0x7f6f0c06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f5b4d800640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 0x7f5b400e3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f6f11e00640 1 -- 172.21.15.67:0/2923071700 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c0b3590 con 0x7f6f0c0d04a0 2024-05-30T16:44:00.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f5b4d800640 1 -- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b400e30c0 msgr2=0x7f5b4017ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f5b4d800640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b400e30c0 0x7f5b4017ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f2e4ce00640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e2806caf0 0x7f2e2806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f5b4d800640 1 -- 172.21.15.67:0/3973195257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4017c710 con 0x7f5b4003e840 2024-05-30T16:44:00.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f5b4ce00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 0x7f5b400e3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:00.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f5b4e200640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b400e30c0 0x7f5b4017ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:00.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 480496251 0 0) 0x7fd42c003520 con 0x7fd44806f860 2024-05-30T16:44:00.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 msgr2=0x7fd4480d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd4480d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 >> v2:172.21.15.169:3300/0 conn(0x7fd4480ad130 msgr2=0x7fd4480ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 --2- 172.21.15.67:0/2996485131 >> v2:172.21.15.169:3300/0 conn(0x7fd4480ad130 0x7fd4480ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4480b3cc0 con 0x7fd44806f860 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fd440001bc0 con 0x7fd44806f860 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 >> v1:172.21.15.67:6789/0 conn(0x7fd44806f860 legacy=0x7fd4480b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42c0039b0 0x7fd42c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42c004800 0x7fd42c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd42c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd44815cb00 con 0x7fd42c0039b0 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c00ba20 con 0x7fd42c004800 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd42c00bc20 con 0x7fd4480d04a0 2024-05-30T16:44:00.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f6f11400640 1 --2- 172.21.15.67:0/2923071700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c06f6e0 0x7f6f0c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:00.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f5b4d800640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b4003e840 0x7f5b400e42f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00c410 tx=0x7f5b3c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f8308a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83040d1140 0x7f830409fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f8308a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f83040b2e20 con 0x7f83040d1140 2024-05-30T16:44:00.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f8308a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f83040b2ca0 con 0x7f83040d0230 2024-05-30T16:44:00.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.054+0000 7f8308a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f830409b1a0 con 0x7f83040b28c0 2024-05-30T16:44:00.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f8301e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040b28c0 0x7f830409c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f8301e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040b28c0 0x7f830409c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45628/0 (socket says 172.21.15.67:45628) 2024-05-30T16:44:00.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f8301e00640 1 -- 172.21.15.67:0/3265777350 learned_addr learned my addr 172.21.15.67:0/3265777350 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f8302800640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83040d1140 0x7f830409fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f5b45600640 1 -- 172.21.15.67:0/3973195257 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c017070 con 0x7f5b4003e840 2024-05-30T16:44:00.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f5b45600640 1 -- 172.21.15.67:0/3973195257 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c005ce0 con 0x7f5b4003e840 2024-05-30T16:44:00.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b400e3c40 con 0x7f5b4003e840 2024-05-30T16:44:00.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f2e4ce00640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e2806caf0 0x7f2e2806efb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2e40007540 tx=0x7f2e3c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b400e4bb0 con 0x7f5b4003e840 2024-05-30T16:44:00.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7f5b45600640 1 -- 172.21.15.67:0/3973195257 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0072f0 con 0x7f5b4003e840 2024-05-30T16:44:00.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7fd446000640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42c004800 0x7fd42c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7fd447400640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd42c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.055+0000 7fd446a00640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42c0039b0 0x7fd42c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.056+0000 7f8301400640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83040d0230 0x7f830409cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.056+0000 7f6f11e00640 1 --2- 172.21.15.67:0/2923071700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c0d04a0 0x7f6f0c0d0880 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009870 tx=0x7f6f0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64171aa2415047bd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.056+0000 7f8302800640 1 -- 172.21.15.67:0/3265777350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040b28c0 msgr2=0x7f830409c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.056+0000 7f8302800640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040b28c0 0x7f830409c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.056+0000 7f8302800640 1 -- 172.21.15.67:0/3265777350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83040d0230 msgr2=0x7f830409cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.056+0000 7f8302800640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83040d0230 0x7f830409cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.056+0000 7f8302800640 1 -- 172.21.15.67:0/3265777350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830409d5f0 con 0x7f83040d1140 2024-05-30T16:44:00.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.057+0000 7f8302800640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83040d1140 0x7f830409fae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82fc007d90 tx=0x7f82fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.057+0000 7fd447400640 1 -- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42c004800 msgr2=0x7fd42c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.057+0000 7fd447400640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42c004800 0x7fd42c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.057+0000 7fd447400640 1 -- 172.21.15.67:0/2996485131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42c0039b0 msgr2=0x7fd42c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.057+0000 7f6f10a00640 1 -- 172.21.15.67:0/2923071700 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00037070 con 0x7f6f0c0d04a0 2024-05-30T16:44:00.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.058+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b400057e0 con 0x7f5b4003e840 2024-05-30T16:44:00.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.058+0000 7f6f10a00640 1 -- 172.21.15.67:0/2923071700 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f00032440 con 0x7f6f0c0d04a0 2024-05-30T16:44:00.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.058+0000 7f6f10a00640 1 -- 172.21.15.67:0/2923071700 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00037070 con 0x7f6f0c0d04a0 2024-05-30T16:44:00.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.057+0000 7fd447400640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42c0039b0 0x7fd42c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.057+0000 7fd447400640 1 -- 172.21.15.67:0/2996485131 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4480b3cc0 con 0x7fd4480d04a0 2024-05-30T16:44:00.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.058+0000 7f6f14800640 1 -- 172.21.15.67:0/2923071700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c0d04a0 msgr2=0x7f6f0c0d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.057+0000 7fd446a00640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42c0039b0 0x7fd42c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:44:00.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f6f14800640 1 --2- 172.21.15.67:0/2923071700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c0d04a0 0x7f6f0c0d0880 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009870 tx=0x7f6f0002a270 comp rx=0 tx=0).stop 2024-05-30T16:44:00.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f5b45600640 1 -- 172.21.15.67:0/3973195257 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b3c01a070 con 0x7f5b4003e840 2024-05-30T16:44:00.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f82fa000640 1 -- 172.21.15.67:0/3265777350 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00edd0 con 0x7f83040d1140 2024-05-30T16:44:00.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f82fa000640 1 -- 172.21.15.67:0/3265777350 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82fc003990 con 0x7f83040d1140 2024-05-30T16:44:00.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f82fa000640 1 -- 172.21.15.67:0/3265777350 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc003d80 con 0x7f83040d1140 2024-05-30T16:44:00.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f6f14800640 1 -- 172.21.15.67:0/2923071700 shutdown_connections 2024-05-30T16:44:00.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f6f14800640 1 --2- 172.21.15.67:0/2923071700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c06f6e0 0x7f6f0c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f6f14800640 1 --2- 172.21.15.67:0/2923071700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c0d04a0 0x7f6f0c0d0880 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f6f14800640 1 -- 172.21.15.67:0/2923071700 >> 172.21.15.67:0/2923071700 conn(0x7f6f0c06bae0 msgr2=0x7f6f0c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f6f14800640 1 -- 172.21.15.67:0/2923071700 shutdown_connections 2024-05-30T16:44:00.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f6f14800640 1 -- 172.21.15.67:0/2923071700 wait complete. 2024-05-30T16:44:00.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7fd447400640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd42c0051c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd43800ba00 tx=0x7fd42c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433edf709c847398 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830409d8d0 con 0x7f83040d1140 2024-05-30T16:44:00.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.059+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830409f5c0 con 0x7f83040d1140 2024-05-30T16:44:00.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.060+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438030070 con 0x7fd4480d04a0 2024-05-30T16:44:00.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.060+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43802ade0 con 0x7fd4480d04a0 2024-05-30T16:44:00.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.060+0000 7fd445600640 1 -- 172.21.15.67:0/2996485131 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43800b600 con 0x7fd4480d04a0 2024-05-30T16:44:00.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.060+0000 7f5b45600640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b2806c940 0x7f5b2806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.060+0000 7f5b45600640 1 -- 172.21.15.67:0/3973195257 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5b3c08ae80 con 0x7f5b4003e840 2024-05-30T16:44:00.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7fd44d600640 1 -- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 msgr2=0x7fd42c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7fd44d600640 1 --2- 172.21.15.67:0/2996485131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd42c0051c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd43800ba00 tx=0x7fd42c005830 comp rx=0 tx=0).stop 2024-05-30T16:44:00.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7fd447400640 1 -- 172.21.15.67:0/2996485131 reap_dead start 2024-05-30T16:44:00.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7fd44d600640 1 -- 172.21.15.67:0/2996485131 shutdown_connections 2024-05-30T16:44:00.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7f5b4ce00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b2806c940 0x7f5b2806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83040689e0 con 0x7f83040d1140 2024-05-30T16:44:00.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7fd44d600640 1 -- 172.21.15.67:0/2996485131 >> 172.21.15.67:0/2996485131 conn(0x7fd44806bae0 msgr2=0x7fd44806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7fd44d600640 1 -- 172.21.15.67:0/2996485131 shutdown_connections 2024-05-30T16:44:00.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7f6f14800640 1 Processor -- start 2024-05-30T16:44:00.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.061+0000 7fd44d600640 1 -- 172.21.15.67:0/2996485131 wait complete. 2024-05-30T16:44:00.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.062+0000 7f5b4ce00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b2806c940 0x7f5b2806ee00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5b40008630 tx=0x7f5b4806e030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.062+0000 7f6f14800640 1 -- start start 2024-05-30T16:44:00.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.063+0000 7f6f14800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c06f6e0 0x7f6f0c165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.063+0000 7f6f14800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c0d04a0 0x7f6f0c165960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.063+0000 7f6f14800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c162e70 0x7f6f0c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.063+0000 7f6f14800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c0d0dc0 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.063+0000 7f6f14800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c0b3b50 con 0x7f6f0c162e70 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.063+0000 7f6f14800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c0cda30 con 0x7f6f0c0d04a0 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f6f11400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c0d04a0 0x7f6f0c165960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f6f11e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c06f6e0 0x7f6f0c165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f6f11400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c0d04a0 0x7f6f0c165960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45642/0 (socket says 172.21.15.67:45642) 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f6f11400640 1 -- 172.21.15.67:0/2512748412 learned_addr learned my addr 172.21.15.67:0/2512748412 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f6f11e00640 1 -- 172.21.15.67:0/2512748412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c0d04a0 msgr2=0x7f6f0c165960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f6f11e00640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c0d04a0 0x7f6f0c165960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f6f11e00640 1 -- 172.21.15.67:0/2512748412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c162e70 msgr2=0x7f6f0c152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f6f11e00640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c162e70 0x7f6f0c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f6f11e00640 1 -- 172.21.15.67:0/2512748412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c154f80 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7f82fa000640 1 -- 172.21.15.67:0/3265777350 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82fc01b020 con 0x7f83040d1140 2024-05-30T16:44:00.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.064+0000 7fd44d600640 1 Processor -- start 2024-05-30T16:44:00.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.065+0000 7fd44d600640 1 -- start start 2024-05-30T16:44:00.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.065+0000 7f9173600640 1 Processor -- start 2024-05-30T16:44:00.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.065+0000 7f6f11e00640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c06f6e0 0x7f6f0c165420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6f00031000 tx=0x7f6f000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.065+0000 7f9173600640 1 -- start start 2024-05-30T16:44:00.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.065+0000 7f82fa000640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82e406c9c0 0x7f82e406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.065+0000 7f82fa000640 1 -- 172.21.15.67:0/3265777350 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f82fc08bbf0 con 0x7f83040d1140 2024-05-30T16:44:00.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.066+0000 7f8301e00640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82e406c9c0 0x7f82e406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.066+0000 7fb770000640 1 Processor -- start 2024-05-30T16:44:00.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.066+0000 7fb770000640 1 -- start start 2024-05-30T16:44:00.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.066+0000 7fd44d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd44815c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.066+0000 7f9173600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f916c15eb40 0x7f916c0d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.066+0000 7f9173600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0d4bb0 0x7f916c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7fd44d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd44815f8b0 0x7fd4480c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7fd44d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd44815a910 0x7fd448072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.066+0000 7f9173600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f916c0cffb0 con 0x7f916c0d3040 2024-05-30T16:44:00.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7f9173600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f916c0b4530 con 0x7f916c15eb40 2024-05-30T16:44:00.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7f9173600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f916c0d0290 con 0x7f916c0d4bb0 2024-05-30T16:44:00.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7f9170c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f916c15eb40 0x7f916c0d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7f916be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f916c0d3040 0x7f916c0d44a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:53866/0 (socket says 172.21.15.67:53866) 2024-05-30T16:44:00.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7f9171600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0d4bb0 0x7f916c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7f916be00640 1 -- 172.21.15.67:0/3594165147 learned_addr learned my addr 172.21.15.67:0/3594165147 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:00.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7fd446a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd44815c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7fd44d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd4480b4560 con 0x7fd44815f8b0 2024-05-30T16:44:00.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7fd44d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4480b4280 con 0x7fd44815a910 2024-05-30T16:44:00.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7fd44d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4480b4840 con 0x7fd4480d04a0 2024-05-30T16:44:00.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7fd446a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd44815c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:45652/0 (socket says 172.21.15.67:45652) 2024-05-30T16:44:00.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.067+0000 7fd446a00640 1 -- 172.21.15.67:0/1229743833 learned_addr learned my addr 172.21.15.67:0/1229743833 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fd446000640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd44815f8b0 0x7fd4480c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fb770000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb76815ed00 0x7fb7680b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7f9170c00640 1 -- 172.21.15.67:0/3594165147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0d4bb0 msgr2=0x7f916c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7f9170c00640 1 --2- 172.21.15.67:0/3594165147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0d4bb0 0x7f916c0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7f9170c00640 1 -- 172.21.15.67:0/3594165147 >> v1:172.21.15.169:6789/0 conn(0x7f916c0d3040 legacy=0x7f916c0d44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7f9170c00640 1 -- 172.21.15.67:0/3594165147 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c0d0570 con 0x7f916c15eb40 2024-05-30T16:44:00.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7f8301e00640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82e406c9c0 0x7f82e406ee80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82f4007fd0 tx=0x7f82f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fb770000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb7680d46c0 0x7fb7680d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fb770000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7680b5070 con 0x7fb7680d0d60 2024-05-30T16:44:00.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fb770000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb7680d5060 con 0x7fb7680d46c0 2024-05-30T16:44:00.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fb770000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb7680b41a0 con 0x7fb76815ed00 2024-05-30T16:44:00.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fb76cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb7680d46c0 0x7fb7680d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fd446a00640 1 -- 172.21.15.67:0/1229743833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd44815a910 msgr2=0x7fd448072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fb76cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb7680d46c0 0x7fb7680d4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43798/0 (socket says 172.21.15.67:43798) 2024-05-30T16:44:00.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fb76cc00640 1 -- 172.21.15.67:0/2842140456 learned_addr learned my addr 172.21.15.67:0/2842140456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fd446a00640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd44815a910 0x7fd448072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7fd446a00640 1 -- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd44815f8b0 msgr2=0x7fd4480c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fd446a00640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd44815f8b0 0x7fd4480c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fd446a00640 1 -- 172.21.15.67:0/1229743833 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44815b820 con 0x7fd4480d04a0 2024-05-30T16:44:00.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fd446000640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd44815f8b0 0x7fd4480c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:44:00.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7f2e45600640 1 -- 172.21.15.67:0/1074743711 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e3407ab20 con 0x7f2e4003e830 2024-05-30T16:44:00.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fb76d600640 1 --2- 172.21.15.67:0/2842140456 >> v2:172.21.15.169:3300/0 conn(0x7fb76815ed00 0x7fb7680b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7f246fe00640 1 -- 172.21.15.67:0/3202468108 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0c1070 con 0x7f24740ead00 2024-05-30T16:44:00.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2474044f80 con 0x7f24740ead00 2024-05-30T16:44:00.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.068+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2474045540 con 0x7f24740ead00 2024-05-30T16:44:00.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fb76cc00640 1 -- 172.21.15.67:0/2842140456 >> v2:172.21.15.169:3300/0 conn(0x7fb76815ed00 msgr2=0x7fb7680b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:00.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fb76cc00640 1 --2- 172.21.15.67:0/2842140456 >> v2:172.21.15.169:3300/0 conn(0x7fb76815ed00 0x7fb7680b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7f246fe00640 1 -- 172.21.15.67:0/3202468108 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c0a7a50 con 0x7f24740ead00 2024-05-30T16:44:00.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7f246fe00640 1 -- 172.21.15.67:0/3202468108 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c067ac0 con 0x7f24740ead00 2024-05-30T16:44:00.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fd446a00640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd44815c400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd44000ae70 tx=0x7fd440007430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fb76cc00640 1 -- 172.21.15.67:0/2842140456 >> v1:172.21.15.169:6789/0 conn(0x7fb7680d0d60 legacy=0x7fb7680d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fb76cc00640 1 -- 172.21.15.67:0/2842140456 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7680d33b0 con 0x7fb7680d46c0 2024-05-30T16:44:00.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.069+0000 7fb76d600640 1 --2- 172.21.15.67:0/2842140456 >> v2:172.21.15.169:3300/0 conn(0x7fb76815ed00 0x7fb7680b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:00.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.070+0000 7f9170c00640 1 --2- 172.21.15.67:0/3594165147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c15eb40 0x7f916c0d2990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91600097a0 tx=0x7f916002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c86abd448f434e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.070+0000 7fd43ea00640 1 -- 172.21.15.67:0/1229743833 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440012560 con 0x7fd4480d04a0 2024-05-30T16:44:00.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.070+0000 7fd43ea00640 1 -- 172.21.15.67:0/1229743833 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440018070 con 0x7fd4480d04a0 2024-05-30T16:44:00.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.070+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44815ad90 con 0x7fd4480d04a0 2024-05-30T16:44:00.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.070+0000 7f5b45600640 1 -- 172.21.15.67:0/3973195257 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b3c0594e0 con 0x7f5b4003e840 2024-05-30T16:44:00.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.070+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2458005290 con 0x7f24740ead00 2024-05-30T16:44:00.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.070+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44815eaf0 con 0x7fd4480d04a0 2024-05-30T16:44:00.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.070+0000 7fd43ea00640 1 -- 172.21.15.67:0/1229743833 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440012920 con 0x7fd4480d04a0 2024-05-30T16:44:00.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.071+0000 7fb76cc00640 1 --2- 172.21.15.67:0/2842140456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d46c0 0x7fb7680d4b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb7680b7320 tx=0x7fb75402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea19f52873f85979 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.072+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd420005290 con 0x7fd4480d04a0 2024-05-30T16:44:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.074+0000 7fd43ea00640 1 -- 172.21.15.67:0/1229743833 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd440012ac0 con 0x7fd4480d04a0 2024-05-30T16:44:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.075+0000 7fd43ea00640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd42806c920 0x7fd42806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.075+0000 7fd43ea00640 1 -- 172.21.15.67:0/1229743833 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd44008c5f0 con 0x7fd4480d04a0 2024-05-30T16:44:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.074+0000 7f246fe00640 1 -- 172.21.15.67:0/3202468108 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f247c067c60 con 0x7f24740ead00 2024-05-30T16:44:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.075+0000 7f246fe00640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f245c06c920 0x7f245c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.075+0000 7f246fe00640 1 -- 172.21.15.67:0/3202468108 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f247c127450 con 0x7f24740ead00 2024-05-30T16:44:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.076+0000 7fd446000640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd42806c920 0x7fd42806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.076+0000 7f247be00640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f245c06c920 0x7f245c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7fd446000640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd42806c920 0x7fd42806ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4480d0c40 tx=0x7fd434008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7fb767e00640 1 -- 172.21.15.67:0/2842140456 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754037070 con 0x7fb7680d46c0 2024-05-30T16:44:00.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f6f06000640 1 -- 172.21.15.67:0/2512748412 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00037040 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f247be00640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f245c06c920 0x7f245c06ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2470009a20 tx=0x7f2470008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7fb767e00640 1 -- 172.21.15.67:0/2842140456 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb754032440 con 0x7fb7680d46c0 2024-05-30T16:44:00.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7fb767e00640 1 -- 172.21.15.67:0/2842140456 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754032810 con 0x7fb7680d46c0 2024-05-30T16:44:00.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f6f06000640 1 -- 172.21.15.67:0/2512748412 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f00042070 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f6f06000640 1 -- 172.21.15.67:0/2512748412 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00032810 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c153f20 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c151930 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f82fa000640 1 -- 172.21.15.67:0/3265777350 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82fc0599b0 con 0x7f83040d1140 2024-05-30T16:44:00.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f916b400640 1 -- 172.21.15.67:0/3594165147 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160037070 con 0x7f916c15eb40 2024-05-30T16:44:00.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f916b400640 1 -- 172.21.15.67:0/3594165147 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9160032440 con 0x7f916c15eb40 2024-05-30T16:44:00.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.077+0000 7f916b400640 1 -- 172.21.15.67:0/3594165147 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160032810 con 0x7f916c15eb40 2024-05-30T16:44:00.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.078+0000 7fb770000640 1 -- 172.21.15.67:0/2842140456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d46c0 msgr2=0x7fb7680d4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.078+0000 7fb770000640 1 --2- 172.21.15.67:0/2842140456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d46c0 0x7fb7680d4b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb7680b7320 tx=0x7fb75402a270 comp rx=0 tx=0).stop 2024-05-30T16:44:00.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.079+0000 7fb770000640 1 -- 172.21.15.67:0/2842140456 shutdown_connections 2024-05-30T16:44:00.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.079+0000 7fb770000640 1 --2- 172.21.15.67:0/2842140456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d46c0 0x7fb7680d4b20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.079+0000 7fb770000640 1 --2- 172.21.15.67:0/2842140456 >> v2:172.21.15.169:3300/0 conn(0x7fb76815ed00 0x7fb7680b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.079+0000 7fb770000640 1 -- 172.21.15.67:0/2842140456 >> 172.21.15.67:0/2842140456 conn(0x7fb76809da70 msgr2=0x7fb76808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.079+0000 7fb770000640 1 -- 172.21.15.67:0/2842140456 shutdown_connections 2024-05-30T16:44:00.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.079+0000 7fb770000640 1 -- 172.21.15.67:0/2842140456 wait complete. 2024-05-30T16:44:00.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.079+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0c0689e0 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.079+0000 7f9173600640 1 -- 172.21.15.67:0/3594165147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c15eb40 msgr2=0x7f916c0d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.079+0000 7f9173600640 1 --2- 172.21.15.67:0/3594165147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c15eb40 0x7f916c0d2990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f91600097a0 tx=0x7f916002a230 comp rx=0 tx=0).stop 2024-05-30T16:44:00.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.080+0000 7f9173600640 1 -- 172.21.15.67:0/3594165147 shutdown_connections 2024-05-30T16:44:00.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.080+0000 7f9173600640 1 --2- 172.21.15.67:0/3594165147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0d4bb0 0x7f916c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.080+0000 7f9173600640 1 --2- 172.21.15.67:0/3594165147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c15eb40 0x7f916c0d2990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.080+0000 7f9173600640 1 -- 172.21.15.67:0/3594165147 >> 172.21.15.67:0/3594165147 conn(0x7f916c09da70 msgr2=0x7f916c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.080+0000 7f9173600640 1 -- 172.21.15.67:0/3594165147 shutdown_connections 2024-05-30T16:44:00.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.080+0000 7f9173600640 1 -- 172.21.15.67:0/3594165147 wait complete. 2024-05-30T16:44:00.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.081+0000 7f6f06000640 1 -- 172.21.15.67:0/2512748412 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f000329b0 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.082+0000 7fb770000640 1 Processor -- start 2024-05-30T16:44:00.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.082+0000 7fb770000640 1 -- start start 2024-05-30T16:44:00.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.082+0000 7f6f06000640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ef006c9c0 0x7f6ef006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.082+0000 7f6f06000640 1 -- 172.21.15.67:0/2512748412 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6f000ac460 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.082+0000 7f9173600640 1 Processor -- start 2024-05-30T16:44:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.083+0000 7f9173600640 1 -- start start 2024-05-30T16:44:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7f6f11400640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ef006c9c0 0x7f6ef006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7fb770000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d0d60 0x7fb768159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7fb770000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7680d46c0 0x7fb7680b8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7fb770000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb76815ed00 0x7fb7681f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7fb770000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb7680b5610 con 0x7fb7680d0d60 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7fb770000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb7680b5330 con 0x7fb76815ed00 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7fb770000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb7680d3040 con 0x7fb7680d46c0 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7f9173600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f916c0d3040 0x7f916c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7f9173600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c0d4bb0 0x7f916c1600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7f9173600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0ce020 0x7f916c0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7f9173600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f916c0b4f00 con 0x7f916c0d4bb0 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7f9173600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f916c15ef20 con 0x7f916c0d3040 2024-05-30T16:44:00.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.084+0000 7f9173600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f916c0b5080 con 0x7f916c0ce020 2024-05-30T16:44:00.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.085+0000 7f6f11400640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ef006c9c0 0x7f6ef006ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c06de70 tx=0x7f6efc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.085+0000 7f916be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c0d4bb0 0x7f916c1600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.085+0000 7fb76d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d0d60 0x7fb768159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.085+0000 7fb76d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d0d60 0x7fb768159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43814/0 (socket says 172.21.15.67:43814) 2024-05-30T16:44:00.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.085+0000 7fb76d600640 1 -- 172.21.15.67:0/3629636924 learned_addr learned my addr 172.21.15.67:0/3629636924 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.085+0000 7fb76cc00640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7680d46c0 0x7fb7680b8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.085+0000 7f916be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c0d4bb0 0x7f916c1600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43816/0 (socket says 172.21.15.67:43816) 2024-05-30T16:44:00.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.085+0000 7f916be00640 1 -- 172.21.15.67:0/3094357359 learned_addr learned my addr 172.21.15.67:0/3094357359 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:00.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.085+0000 7f9171600640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0ce020 0x7f916c0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7f9170c00640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f916c0d3040 0x7f916c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7f246fe00640 1 -- 172.21.15.67:0/3202468108 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f247c0ad030 con 0x7f24740ead00 2024-05-30T16:44:00.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7fb76d600640 1 -- 172.21.15.67:0/3629636924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7680d46c0 msgr2=0x7fb7680b8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7fb76d600640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7680d46c0 0x7fb7680b8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7fb76d600640 1 -- 172.21.15.67:0/3629636924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb76815ed00 msgr2=0x7fb7681f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7fb76d600640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb76815ed00 0x7fb7681f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7fb76d600640 1 -- 172.21.15.67:0/3629636924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7680b7570 con 0x7fb7680d0d60 2024-05-30T16:44:00.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7f916be00640 1 -- 172.21.15.67:0/3094357359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0ce020 msgr2=0x7f916c0ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7f916be00640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0ce020 0x7f916c0ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7f916be00640 1 -- 172.21.15.67:0/3094357359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f916c0d3040 msgr2=0x7f916c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7f916be00640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f916c0d3040 0x7f916c15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7f916be00640 1 -- 172.21.15.67:0/3094357359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c0cda30 con 0x7f916c0d4bb0 2024-05-30T16:44:00.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.087+0000 7fb76d600640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d0d60 0x7fb768159060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00c7f0 tx=0x7fb75c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.086+0000 7f916be00640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c0d4bb0 0x7f916c1600f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f915800c4a0 tx=0x7f915800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.088+0000 7f9168c00640 1 -- 172.21.15.67:0/3094357359 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158017070 con 0x7f916c0d4bb0 2024-05-30T16:44:00.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.089+0000 7f9168c00640 1 -- 172.21.15.67:0/3094357359 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9158005ce0 con 0x7f916c0d4bb0 2024-05-30T16:44:00.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.089+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c0c8400 con 0x7f916c0d4bb0 2024-05-30T16:44:00.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.089+0000 7f6f06000640 1 -- 172.21.15.67:0/2512748412 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f0007a990 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.089+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c160630 con 0x7f916c0d4bb0 2024-05-30T16:44:00.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.089+0000 7f9168c00640 1 -- 172.21.15.67:0/3094357359 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158007270 con 0x7f916c0d4bb0 2024-05-30T16:44:00.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.090+0000 7fb765600640 1 -- 172.21.15.67:0/3629636924 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c015070 con 0x7fb7680d0d60 2024-05-30T16:44:00.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.090+0000 7fb765600640 1 -- 172.21.15.67:0/3629636924 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c007820 con 0x7fb7680d0d60 2024-05-30T16:44:00.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.090+0000 7fb765600640 1 -- 172.21.15.67:0/3629636924 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c007bd0 con 0x7fb7680d0d60 2024-05-30T16:44:00.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.090+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7680b7850 con 0x7fb7680d0d60 2024-05-30T16:44:00.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.090+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7680b7e10 con 0x7fb7680d0d60 2024-05-30T16:44:00.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.090+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9148005290 con 0x7f916c0d4bb0 2024-05-30T16:44:00.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.092+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7680689e0 con 0x7fb7680d0d60 2024-05-30T16:44:00.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.094+0000 7f9168c00640 1 -- 172.21.15.67:0/3094357359 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f915801a070 con 0x7f916c0d4bb0 2024-05-30T16:44:00.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.095+0000 7f9168c00640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f914c06c940 0x7f914c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.096+0000 7fb765600640 1 -- 172.21.15.67:0/3629636924 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb75c018070 con 0x7fb7680d0d60 2024-05-30T16:44:00.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.095+0000 7fd43ea00640 1 -- 172.21.15.67:0/1229743833 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd44815b820 con 0x7fd4480d04a0 2024-05-30T16:44:00.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.096+0000 7f9168c00640 1 -- 172.21.15.67:0/3094357359 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f915808add0 con 0x7f916c0d4bb0 2024-05-30T16:44:00.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.097+0000 7f9170c00640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f914c06c940 0x7f914c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.097+0000 7fb765600640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb74806c9c0 0x7fb74806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:00.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.097+0000 7fb765600640 1 -- 172.21.15.67:0/3629636924 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb75c08c230 con 0x7fb7680d0d60 2024-05-30T16:44:00.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.098+0000 7fb76cc00640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb74806c9c0 0x7fb74806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:00.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.099+0000 7f9170c00640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f914c06c940 0x7f914c06ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9160034000 tx=0x7f916002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.100+0000 7fb76cc00640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb74806c9c0 0x7fb74806ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb754034000 tx=0x7fb75402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:00.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.105+0000 7fb765600640 1 -- 172.21.15.67:0/3629636924 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb75c05a5b0 con 0x7fb7680d0d60 2024-05-30T16:44:00.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.114+0000 7f9168c00640 1 -- 172.21.15.67:0/3094357359 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9158058b90 con 0x7f916c0d4bb0 2024-05-30T16:44:00.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.513+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5b4004f830 con 0x7f5b4003e840 2024-05-30T16:44:00.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.514+0000 7f5b45600640 1 -- 172.21.15.67:0/3973195257 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b3c058de0 con 0x7f5b4003e840 2024-05-30T16:44:00.515 INFO:teuthology.orchestra.run.smithi067.stdout:64424509446 2024-05-30T16:44:00.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.518+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b2806c940 msgr2=0x7f5b2806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.518+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b2806c940 0x7f5b2806ee00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5b40008630 tx=0x7f5b4806e030 comp rx=0 tx=0).stop 2024-05-30T16:44:00.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.518+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b4003e840 msgr2=0x7f5b400e42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.518+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b4003e840 0x7f5b400e42f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00c410 tx=0x7f5b3c00c8d0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.519+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 shutdown_connections 2024-05-30T16:44:00.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.519+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b2806c940 0x7f5b2806ee00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.519+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b400e30c0 0x7f5b4017ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.519+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b40054130 0x7f5b400e3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.519+0000 7f5b4ec00640 1 --2- 172.21.15.67:0/3973195257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b4003e840 0x7f5b400e42f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.519+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 >> 172.21.15.67:0/3973195257 conn(0x7f5b400559e0 msgr2=0x7f5b40039e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.520+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 shutdown_connections 2024-05-30T16:44:00.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.520+0000 7f5b4ec00640 1 -- 172.21.15.67:0/3973195257 wait complete. 2024-05-30T16:44:00.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.522+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2e400d9910 con 0x7f2e4003e830 2024-05-30T16:44:00.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.524+0000 7f2e45600640 1 -- 172.21.15.67:0/1074743711 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e3407a420 con 0x7f2e4003e830 2024-05-30T16:44:00.525 INFO:teuthology.orchestra.run.smithi067.stdout:64424509446 2024-05-30T16:44:00.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.527+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e2806caf0 msgr2=0x7f2e2806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.527+0000 7f2e50200640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e2806caf0 0x7f2e2806efb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2e40007540 tx=0x7f2e3c008040 comp rx=0 tx=0).stop 2024-05-30T16:44:00.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.528+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4003e830 msgr2=0x7f2e400d7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.528+0000 7f2e50200640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4003e830 0x7f2e400d7110 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e340090b0 tx=0x7f2e340046c0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.528+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 shutdown_connections 2024-05-30T16:44:00.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.528+0000 7f2e50200640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e2806caf0 0x7f2e2806efb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.528+0000 7f2e50200640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e400e30b0 0x7f2e40051c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.528+0000 7f2e50200640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40054120 0x7f2e400db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.528+0000 7f2e50200640 1 --2- 172.21.15.67:0/1074743711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4003e830 0x7f2e400d7110 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.528+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 >> 172.21.15.67:0/1074743711 conn(0x7f2e400559d0 msgr2=0x7f2e400d9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.529+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 shutdown_connections 2024-05-30T16:44:00.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.530+0000 7f2e50200640 1 -- 172.21.15.67:0/1074743711 wait complete. 2024-05-30T16:44:00.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.532+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f83040b3200 con 0x7f83040d1140 2024-05-30T16:44:00.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.533+0000 7f82fa000640 1 -- 172.21.15.67:0/3265777350 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f82fc05dcb0 con 0x7f83040d1140 2024-05-30T16:44:00.537 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2024-05-30T16:44:00.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.536+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2458005850 con 0x7f24740ead00 2024-05-30T16:44:00.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.537+0000 7f246fe00640 1 -- 172.21.15.67:0/3202468108 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f247c0f5430 con 0x7f24740ead00 2024-05-30T16:44:00.541 INFO:teuthology.orchestra.run.smithi067.stdout:64424509445 2024-05-30T16:44:00.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82e406c9c0 msgr2=0x7f82e406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f8308a00640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82e406c9c0 0x7f82e406ee80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82f4007fd0 tx=0x7f82f4009040 comp rx=0 tx=0).stop 2024-05-30T16:44:00.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83040d1140 msgr2=0x7f830409fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f8308a00640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83040d1140 0x7f830409fae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f82fc007d90 tx=0x7f82fc00e4b0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f245c06c920 msgr2=0x7f245c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f245c06c920 0x7f245c06ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2470009a20 tx=0x7f2470008040 comp rx=0 tx=0).stop 2024-05-30T16:44:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24740ead00 msgr2=0x7f247404dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24740ead00 0x7f247404dfb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f247c051690 tx=0x7f247c075b40 comp rx=0 tx=0).stop 2024-05-30T16:44:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 shutdown_connections 2024-05-30T16:44:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f245c06c920 0x7f245c06ede0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24740ead00 0x7f247404dfb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f247404ffd0 0x7f2474046020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 --2- 172.21.15.67:0/3202468108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2474006ea0 0x7f24740448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.543+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 shutdown_connections 2024-05-30T16:44:00.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.543+0000 7f8308a00640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82e406c9c0 0x7f82e406ee80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.543+0000 7f8308a00640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83040d1140 0x7f830409fae0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.543+0000 7f8308a00640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83040d0230 0x7f830409cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.543+0000 7f8308a00640 1 --2- 172.21.15.67:0/3265777350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83040b28c0 0x7f830409c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.543+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 >> 172.21.15.67:0/3265777350 conn(0x7f83040b3f40 msgr2=0x7f830409b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.543+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 shutdown_connections 2024-05-30T16:44:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.543+0000 7f8308a00640 1 -- 172.21.15.67:0/3265777350 wait complete. 2024-05-30T16:44:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.541+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 >> 172.21.15.67:0/3202468108 conn(0x7f2474009fb0 msgr2=0x7f2474052c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.542+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 shutdown_connections 2024-05-30T16:44:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.542+0000 7f2481600640 1 -- 172.21.15.67:0/3202468108 wait complete. 2024-05-30T16:44:00.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.547+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6f0c0b0120 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-05-30T16:44:00.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.547+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd420005580 con 0x7fd4480d04a0 2024-05-30T16:44:00.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.548+0000 7f6f06000640 1 -- 172.21.15.67:0/2512748412 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6f0007a290 con 0x7f6f0c06f6e0 2024-05-30T16:44:00.549 INFO:teuthology.orchestra.run.smithi067.stdout:60129542148 2024-05-30T16:44:00.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.548+0000 7fd43ea00640 1 -- 172.21.15.67:0/1229743833 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd44806ed10 con 0x7fd4480d04a0 2024-05-30T16:44:00.552 INFO:teuthology.orchestra.run.smithi067.stdout:64424509445 2024-05-30T16:44:00.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ef006c9c0 msgr2=0x7f6ef006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ef006c9c0 0x7f6ef006ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c06de70 tx=0x7f6efc008040 comp rx=0 tx=0).stop 2024-05-30T16:44:00.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c06f6e0 msgr2=0x7f6f0c165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c06f6e0 0x7f6f0c165420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6f00031000 tx=0x7f6f000046c0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 shutdown_connections 2024-05-30T16:44:00.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ef006c9c0 0x7f6ef006ee80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c162e70 0x7f6f0c152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f0c0d04a0 0x7f6f0c165960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 --2- 172.21.15.67:0/2512748412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f0c06f6e0 0x7f6f0c165420 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6f00031000 tx=0x7f6f000046c0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 >> 172.21.15.67:0/2512748412 conn(0x7f6f0c06bae0 msgr2=0x7f6f0c0725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd42806c920 msgr2=0x7fd42806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7fd44d600640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd42806c920 0x7fd42806ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4480d0c40 tx=0x7fd434008040 comp rx=0 tx=0).stop 2024-05-30T16:44:00.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 msgr2=0x7fd44815c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.552+0000 7fd44d600640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd44815c400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd44000ae70 tx=0x7fd440007430 comp rx=0 tx=0).stop 2024-05-30T16:44:00.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.553+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 shutdown_connections 2024-05-30T16:44:00.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.553+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 shutdown_connections 2024-05-30T16:44:00.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.553+0000 7fd44d600640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd42806c920 0x7fd42806ede0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.553+0000 7fd44d600640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd44815a910 0x7fd448072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.553+0000 7fd44d600640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd44815f8b0 0x7fd4480c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.553+0000 7fd44d600640 1 --2- 172.21.15.67:0/1229743833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4480d04a0 0x7fd44815c400 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.553+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 >> 172.21.15.67:0/1229743833 conn(0x7fd44806bae0 msgr2=0x7fd4480b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.553+0000 7f6f14800640 1 -- 172.21.15.67:0/2512748412 wait complete. 2024-05-30T16:44:00.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.556+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb7680b3d50 con 0x7fb7680d0d60 2024-05-30T16:44:00.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.557+0000 7fb765600640 1 -- 172.21.15.67:0/3629636924 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb75c059eb0 con 0x7fb7680d0d60 2024-05-30T16:44:00.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.556+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 shutdown_connections 2024-05-30T16:44:00.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.557+0000 7fd44d600640 1 -- 172.21.15.67:0/1229743833 wait complete. 2024-05-30T16:44:00.559 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2024-05-30T16:44:00.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2024-05-30T16:44:00.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.559+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9148005580 con 0x7f916c0d4bb0 2024-05-30T16:44:00.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.560+0000 7f9168c00640 1 -- 172.21.15.67:0/3094357359 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f915805ce90 con 0x7f916c0d4bb0 2024-05-30T16:44:00.561 INFO:teuthology.orchestra.run.smithi067.stdout:64424509445 2024-05-30T16:44:00.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.561+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb74806c9c0 msgr2=0x7fb74806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.561+0000 7fb770000640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb74806c9c0 0x7fb74806ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb754034000 tx=0x7fb75402e000 comp rx=0 tx=0).stop 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.561+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d0d60 msgr2=0x7fb768159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.561+0000 7fb770000640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d0d60 0x7fb768159060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00c7f0 tx=0x7fb75c00ccb0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.562+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 shutdown_connections 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.562+0000 7fb770000640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb74806c9c0 0x7fb74806ee80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.562+0000 7fb770000640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb76815ed00 0x7fb7681f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.562+0000 7fb770000640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7680d46c0 0x7fb7680b8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.562+0000 7fb770000640 1 --2- 172.21.15.67:0/3629636924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7680d0d60 0x7fb768159060 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.562+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 >> 172.21.15.67:0/3629636924 conn(0x7fb76809da70 msgr2=0x7fb7680d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.562+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 shutdown_connections 2024-05-30T16:44:00.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.562+0000 7fb770000640 1 -- 172.21.15.67:0/3629636924 wait complete. 2024-05-30T16:44:00.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f914c06c940 msgr2=0x7f914c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f914c06c940 0x7f914c06ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9160034000 tx=0x7f916002e000 comp rx=0 tx=0).stop 2024-05-30T16:44:00.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c0d4bb0 msgr2=0x7f916c1600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c0d4bb0 0x7f916c1600f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f915800c4a0 tx=0x7f915800c960 comp rx=0 tx=0).stop 2024-05-30T16:44:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 shutdown_connections 2024-05-30T16:44:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f914c06c940 0x7f914c06ee00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f916c0ce020 0x7f916c0ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f916c0d4bb0 0x7f916c1600f0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f915800c4a0 tx=0x7f915800c960 comp rx=0 tx=0).stop 2024-05-30T16:44:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 --2- 172.21.15.67:0/3094357359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f916c0d3040 0x7f916c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.564+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 >> 172.21.15.67:0/3094357359 conn(0x7f916c09da70 msgr2=0x7f916c0b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.565+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 shutdown_connections 2024-05-30T16:44:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:00.565+0000 7f9173600640 1 -- 172.21.15.67:0/3094357359 wait complete. 2024-05-30T16:44:00.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.0 2024-05-30T16:44:00.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2024-05-30T16:44:00.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-05-30T16:44:00.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.5 2024-05-30T16:44:00.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.7 2024-05-30T16:44:00.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.6 2024-05-30T16:44:01.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-30T16:44:01.561 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-30T16:44:01.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-30T16:44:01.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-30T16:44:01.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-30T16:44:01.583 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-30T16:44:01.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-30T16:44:01.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-30T16:44:01.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.846+0000 7f40cca00640 1 Processor -- start 2024-05-30T16:44:01.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.847+0000 7f40cca00640 1 -- start start 2024-05-30T16:44:01.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.849+0000 7f40cca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00e3160 0x7f40c00e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.849+0000 7f40cca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f40c003c0a0 0x7f40c0058050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.849+0000 7f40cca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40c0058590 con 0x7f40c0053cb0 2024-05-30T16:44:01.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.849+0000 7f40cca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f40c0007b70 con 0x7f40c003c0a0 2024-05-30T16:44:01.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.849+0000 7f40cca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f40c0058810 con 0x7f40c00e3160 2024-05-30T16:44:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.851+0000 7f40c5e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f40c003c0a0 0x7f40c0058050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.851+0000 7f40c7200640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f40c0053cb0 0x7f40c0054120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:49684/0 (socket says 172.21.15.67:49684) 2024-05-30T16:44:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.851+0000 7f40c7200640 1 -- 172.21.15.67:0/2768901351 learned_addr learned my addr 172.21.15.67:0/2768901351 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.851+0000 7f40c6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00e3160 0x7f40c00e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.852+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447073631 0 0) 0x7f40c8050bd0 con 0x7f40c0053cb0 2024-05-30T16:44:01.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.852+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40ac003520 con 0x7f40c0053cb0 2024-05-30T16:44:01.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.853+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2719019327 0 0) 0x7f40ac003520 con 0x7f40c0053cb0 2024-05-30T16:44:01.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.853+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00e3160 msgr2=0x7f40c00e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:01.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.853+0000 7f40c5400640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00e3160 0x7f40c00e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.853+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 >> v2:172.21.15.169:3300/0 conn(0x7f40c003c0a0 msgr2=0x7f40c0058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:01.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.853+0000 7f40c5400640 1 --2- 172.21.15.67:0/2768901351 >> v2:172.21.15.169:3300/0 conn(0x7f40c003c0a0 0x7f40c0058050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.853+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c0058af0 con 0x7f40c0053cb0 2024-05-30T16:44:01.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.853+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f40c8075c50 con 0x7f40c0053cb0 2024-05-30T16:44:01.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.853+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 >> v1:172.21.15.169:6789/0 conn(0x7f40c0053cb0 legacy=0x7f40c0054120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.853+0000 7f40c5400640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac0039d0 0x7f40ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.854+0000 7f40c5400640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac004390 0x7f40ac004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.854+0000 7f40c6800640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac0039d0 0x7f40ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.855+0000 7f40c5400640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00e3160 0x7f40ac004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.855+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0058810 con 0x7f40ac0039d0 2024-05-30T16:44:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.855+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac00b9c0 con 0x7f40ac004390 2024-05-30T16:44:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.855+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f40ac00bb70 con 0x7f40c00e3160 2024-05-30T16:44:01.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.856+0000 7f40c5e00640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac004390 0x7f40ac004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.856+0000 7f40c6800640 1 -- 172.21.15.67:0/2768901351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00e3160 msgr2=0x7f40ac004fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:44:01.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.856+0000 7f40c6800640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00e3160 0x7f40ac004fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.856+0000 7f40c6800640 1 -- 172.21.15.67:0/2768901351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac004390 msgr2=0x7f40ac004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:01.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.856+0000 7f40c6800640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac004390 0x7f40ac004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.856+0000 7f40c6800640 1 -- 172.21.15.67:0/2768901351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac00be30 con 0x7f40ac0039d0 2024-05-30T16:44:01.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.859+0000 7f40c6800640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac0039d0 0x7f40ac003e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f40b400bc90 tx=0x7f40ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7429db527f565cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.860+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4038070 con 0x7f40ac0039d0 2024-05-30T16:44:01.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.860+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b4033410 con 0x7f40ac0039d0 2024-05-30T16:44:01.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.861+0000 7f40cca00640 1 -- 172.21.15.67:0/2768901351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac0039d0 msgr2=0x7f40ac003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.861+0000 7f40c5400640 1 -- 172.21.15.67:0/2768901351 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4038070 con 0x7f40ac0039d0 2024-05-30T16:44:01.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.861+0000 7f40cca00640 1 --2- 172.21.15.67:0/2768901351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac0039d0 0x7f40ac003e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f40b400bc90 tx=0x7f40ac005830 comp rx=0 tx=0).stop 2024-05-30T16:44:01.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.862+0000 7f40c7200640 1 -- 172.21.15.67:0/2768901351 reap_dead start 2024-05-30T16:44:01.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.862+0000 7f40cca00640 1 -- 172.21.15.67:0/2768901351 shutdown_connections 2024-05-30T16:44:01.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.863+0000 7f40cca00640 1 -- 172.21.15.67:0/2768901351 >> 172.21.15.67:0/2768901351 conn(0x7f40c0058ee0 msgr2=0x7f40c00592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:01.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.863+0000 7f40cca00640 1 -- 172.21.15.67:0/2768901351 shutdown_connections 2024-05-30T16:44:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.863+0000 7f40cca00640 1 -- 172.21.15.67:0/2768901351 wait complete. 2024-05-30T16:44:01.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.867+0000 7f40cca00640 1 Processor -- start 2024-05-30T16:44:01.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.868+0000 7f40cca00640 1 -- start start 2024-05-30T16:44:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.870+0000 7f40cca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac0039d0 0x7f40c004f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.870+0000 7f40cca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00388e0 0x7f40c00d81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.870+0000 7f40cca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c00e5d70 0x7f40c00e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.870+0000 7f40c6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac0039d0 0x7f40c004f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.870+0000 7f40c6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac0039d0 0x7f40c004f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36558/0 (socket says 172.21.15.67:36558) 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.871+0000 7f40c5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00388e0 0x7f40c00d81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.871+0000 7f40c5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00388e0 0x7f40c00d81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:50430/0 (socket says 172.21.15.67:50430) 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.872+0000 7f40c7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c00e5d70 0x7f40c00e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.872+0000 7f40c7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c00e5d70 0x7f40c00e6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53036/0 (socket says 172.21.15.67:53036) 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.872+0000 7f40c6800640 1 -- 172.21.15.67:0/1868618464 learned_addr learned my addr 172.21.15.67:0/1868618464 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.872+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f40c00e3540 con 0x7f40c00e5d70 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.872+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0006640 con 0x7f40ac0039d0 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.872+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f40c00e7780 con 0x7f40c00388e0 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.873+0000 7f40c7200640 1 -- 172.21.15.67:0/1868618464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00388e0 msgr2=0x7f40c00d81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.873+0000 7f40c7200640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00388e0 0x7f40c00d81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.873+0000 7f40c7200640 1 -- 172.21.15.67:0/1868618464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac0039d0 msgr2=0x7f40c004f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.873+0000 7f40c7200640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac0039d0 0x7f40c004f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.873+0000 7f40c7200640 1 -- 172.21.15.67:0/1868618464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c0038e50 con 0x7f40c00e5d70 2024-05-30T16:44:01.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.874+0000 7f40c7200640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c00e5d70 0x7f40c00e6220 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f40c8069090 tx=0x7f40c8068ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.875+0000 7f40c6800640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac0039d0 0x7f40c004f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:01.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.875+0000 7f40bea00640 1 -- 172.21.15.67:0/1868618464 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8071830 con 0x7f40c00e5d70 2024-05-30T16:44:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.876+0000 7f40bea00640 1 -- 172.21.15.67:0/1868618464 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c80719d0 con 0x7f40c00e5d70 2024-05-30T16:44:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.876+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c0039130 con 0x7f40c00e5d70 2024-05-30T16:44:01.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.876+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c00396f0 con 0x7f40c00e5d70 2024-05-30T16:44:01.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.877+0000 7f40bea00640 1 -- 172.21.15.67:0/1868618464 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8071dc0 con 0x7f40c00e5d70 2024-05-30T16:44:01.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.881+0000 7f40bea00640 1 -- 172.21.15.67:0/1868618464 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40c8095060 con 0x7f40c00e5d70 2024-05-30T16:44:01.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.883+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a0005290 con 0x7f40c00e5d70 2024-05-30T16:44:01.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.884+0000 7f40bea00640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f40a806c9c0 0x7f40a806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.884+0000 7f40bea00640 1 -- 172.21.15.67:0/1868618464 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f40c8105a90 con 0x7f40c00e5d70 2024-05-30T16:44:01.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.885+0000 7f40c6800640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f40a806c9c0 0x7f40a806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.888+0000 7ffae6800640 1 Processor -- start 2024-05-30T16:44:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.888+0000 7ffae6800640 1 -- start start 2024-05-30T16:44:01.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.890+0000 7f40c6800640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f40a806c9c0 0x7f40a806ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009090 tx=0x7f40b402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.890+0000 7ffae6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffad800aad0 0x7ffad80df510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.890+0000 7ffae6800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffad8053310 con 0x7ffad80df130 2024-05-30T16:44:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.890+0000 7ffae6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffad80535f0 con 0x7ffad8054690 2024-05-30T16:44:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.890+0000 7ffae6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ffad80e5e40 con 0x7ffad800aad0 2024-05-30T16:44:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.892+0000 7ffadf400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ffad80df130 0x7ffad804fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52312/0 (socket says 172.21.15.67:52312) 2024-05-30T16:44:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.892+0000 7ffadf400640 1 -- 172.21.15.67:0/3747321885 learned_addr learned my addr 172.21.15.67:0/3747321885 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.892+0000 7ffadea00640 1 --2- 172.21.15.67:0/3747321885 >> v2:172.21.15.169:3300/0 conn(0x7ffad800aad0 0x7ffad80df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.893+0000 7ffade000640 1 -- 172.21.15.67:0/3747321885 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1356815430 0 0) 0x7ffad80535f0 con 0x7ffad8054690 2024-05-30T16:44:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.893+0000 7ffade000640 1 -- 172.21.15.67:0/3747321885 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffac8003520 con 0x7ffad8054690 2024-05-30T16:44:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.893+0000 7ffadea00640 1 -- 172.21.15.67:0/3747321885 >> v1:172.21.15.169:6789/0 conn(0x7ffad8054690 legacy=0x7ffad80dfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.893+0000 7ffadea00640 1 -- 172.21.15.67:0/3747321885 >> v1:172.21.15.67:6789/0 conn(0x7ffad80df130 legacy=0x7ffad804fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.893+0000 7ffadea00640 1 -- 172.21.15.67:0/3747321885 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad80538d0 con 0x7ffad800aad0 2024-05-30T16:44:01.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.895+0000 7ffadea00640 1 --2- 172.21.15.67:0/3747321885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad800aad0 0x7ffad80df510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0097a0 tx=0x7ffacc02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c2def9e46e5ce49 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.896+0000 7ffade000640 1 -- 172.21.15.67:0/3747321885 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc037070 con 0x7ffad800aad0 2024-05-30T16:44:01.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.896+0000 7ffade000640 1 -- 172.21.15.67:0/3747321885 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffacc032640 con 0x7ffad800aad0 2024-05-30T16:44:01.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.896+0000 7ffade000640 1 -- 172.21.15.67:0/3747321885 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc032a10 con 0x7ffad800aad0 2024-05-30T16:44:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.897+0000 7ffae6800640 1 -- 172.21.15.67:0/3747321885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad800aad0 msgr2=0x7ffad80df510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.897+0000 7ffae6800640 1 --2- 172.21.15.67:0/3747321885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad800aad0 0x7ffad80df510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0097a0 tx=0x7ffacc02a2c0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.898+0000 7f40bea00640 1 -- 172.21.15.67:0/1868618464 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f40c0058590 con 0x7f40c00e5d70 2024-05-30T16:44:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.898+0000 7ffae6800640 1 -- 172.21.15.67:0/3747321885 shutdown_connections 2024-05-30T16:44:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.898+0000 7ffae6800640 1 --2- 172.21.15.67:0/3747321885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad800aad0 0x7ffad80df510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.898+0000 7ffae6800640 1 -- 172.21.15.67:0/3747321885 >> 172.21.15.67:0/3747321885 conn(0x7ffad8009fe0 msgr2=0x7ffad800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.898+0000 7ffae6800640 1 -- 172.21.15.67:0/3747321885 shutdown_connections 2024-05-30T16:44:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.898+0000 7ffae6800640 1 -- 172.21.15.67:0/3747321885 wait complete. 2024-05-30T16:44:01.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.900+0000 7ffae6800640 1 Processor -- start 2024-05-30T16:44:01.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.901+0000 7ffae6800640 1 -- start start 2024-05-30T16:44:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.901+0000 7fd6d3e00640 1 Processor -- start 2024-05-30T16:44:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.901+0000 7f222aa00640 1 Processor -- start 2024-05-30T16:44:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.901+0000 7f222aa00640 1 -- start start 2024-05-30T16:44:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.902+0000 7fd6d3e00640 1 -- start start 2024-05-30T16:44:01.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.903+0000 7ffae6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffad800aad0 0x7ffad80e5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.903+0000 7ffae6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad8054690 0x7ffad80ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.903+0000 7ffae6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad80df130 0x7ffad8048600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.903+0000 7ffae6800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8054170 con 0x7ffad800aad0 2024-05-30T16:44:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.903+0000 7ffadf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffad800aad0 0x7ffad80e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7ffadf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffad800aad0 0x7ffad80e5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53044/0 (socket says 172.21.15.67:53044) 2024-05-30T16:44:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7ffadf400640 1 -- 172.21.15.67:0/513423989 learned_addr learned my addr 172.21.15.67:0/513423989 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7ffadea00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad8054690 0x7ffad80ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7ffadfe00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad80df130 0x7ffad8048600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7fd6d3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d40cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7fd6d3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d40b18e0 con 0x7fd6d40b26d0 2024-05-30T16:44:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7fd6d3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d40b1bc0 con 0x7fd6d4073bc0 2024-05-30T16:44:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7fd6d3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d40ce480 con 0x7fd6d40b35d0 2024-05-30T16:44:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7ffae6800640 1 -- 172.21.15.67:0/513423989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8053e90 con 0x7ffad80df130 2024-05-30T16:44:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7f8782600640 1 Processor -- start 2024-05-30T16:44:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7ffae6800640 1 -- 172.21.15.67:0/513423989 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffad80e04e0 con 0x7ffad8054690 2024-05-30T16:44:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7fd6d2000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd6d40b26d0 0x7fd6d40ce910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52320/0 (socket says 172.21.15.67:52320) 2024-05-30T16:44:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7fd6d2000640 1 -- 172.21.15.67:0/3511065094 learned_addr learned my addr 172.21.15.67:0/3511065094 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.904+0000 7fd6d3400640 1 --2- 172.21.15.67:0/3511065094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d40cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.905+0000 7f8782600640 1 -- start start 2024-05-30T16:44:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.905+0000 7f222aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f2224169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.905+0000 7f222aa00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222415ec50 con 0x7f222415e870 2024-05-30T16:44:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.905+0000 7f222aa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22240cff80 con 0x7f22240d16a0 2024-05-30T16:44:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.905+0000 7f222aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22240d2150 con 0x7f22240cfa40 2024-05-30T16:44:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.905+0000 7fd6d3400640 1 -- 172.21.15.67:0/3511065094 >> v1:172.21.15.169:6789/0 conn(0x7fd6d4073bc0 legacy=0x7fd6d415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.905+0000 7fd6d3400640 1 -- 172.21.15.67:0/3511065094 >> v1:172.21.15.67:6789/0 conn(0x7fd6d40b26d0 legacy=0x7fd6d40ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.905+0000 7fd6d3400640 1 -- 172.21.15.67:0/3511065094 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d40cde30 con 0x7fd6d40b35d0 2024-05-30T16:44:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.906+0000 7ffadf400640 1 -- 172.21.15.67:0/513423989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad8054690 msgr2=0x7ffad80ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.906+0000 7ffadf400640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad8054690 0x7ffad80ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.906+0000 7ffadf400640 1 -- 172.21.15.67:0/513423989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad80df130 msgr2=0x7ffad8048600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.906+0000 7ffadf400640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad80df130 0x7ffad8048600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.906+0000 7f222a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f2224169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.906+0000 7ffadf400640 1 -- 172.21.15.67:0/513423989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad80cf7d0 con 0x7ffad800aad0 2024-05-30T16:44:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.906+0000 7ffadea00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad8054690 0x7ffad80ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.906+0000 7f2228c00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f22240d16a0 0x7f22240cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:49706/0 (socket says 172.21.15.67:49706) 2024-05-30T16:44:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.906+0000 7f2228c00640 1 -- 172.21.15.67:0/1694422024 learned_addr learned my addr 172.21.15.67:0/1694422024 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.907+0000 7f99e0c00640 1 Processor -- start 2024-05-30T16:44:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.907+0000 7f99e0c00640 1 -- start start 2024-05-30T16:44:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.907+0000 7ffadf400640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffad800aad0 0x7ffad80e5430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffad400c4b0 tx=0x7ffad400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.907+0000 7fd6d3400640 1 --2- 172.21.15.67:0/3511065094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d40cf020 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00bec0 tx=0x7fd6cc02c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=774bd41649782bc server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.907+0000 7ffad3400640 1 -- 172.21.15.67:0/513423989 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4017070 con 0x7ffad800aad0 2024-05-30T16:44:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.907+0000 7f222a000640 1 -- 172.21.15.67:0/1694422024 >> v1:172.21.15.169:6789/0 conn(0x7f22240d16a0 legacy=0x7f22240cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.907+0000 7f222a000640 1 -- 172.21.15.67:0/1694422024 >> v1:172.21.15.67:6789/0 conn(0x7f222415e870 legacy=0x7f22240d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.908+0000 7ffad3400640 1 -- 172.21.15.67:0/513423989 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad4005ce0 con 0x7ffad800aad0 2024-05-30T16:44:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.908+0000 7fd6d1600640 1 -- 172.21.15.67:0/3511065094 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc03b070 con 0x7fd6d40b35d0 2024-05-30T16:44:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.908+0000 7fd6d1600640 1 -- 172.21.15.67:0/3511065094 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc031070 con 0x7fd6d40b35d0 2024-05-30T16:44:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.908+0000 7fd6d1600640 1 -- 172.21.15.67:0/3511065094 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc033880 con 0x7fd6d40b35d0 2024-05-30T16:44:01.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.908+0000 7ffae6800640 1 -- 172.21.15.67:0/513423989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad80385c0 con 0x7ffad800aad0 2024-05-30T16:44:01.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.908+0000 7ffae6800640 1 -- 172.21.15.67:0/513423989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad8038b80 con 0x7ffad800aad0 2024-05-30T16:44:01.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.908+0000 7f8782600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0b3150 0x7f877c0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.908+0000 7f222a000640 1 -- 172.21.15.67:0/1694422024 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22240d0260 con 0x7f22240cfa40 2024-05-30T16:44:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7f99e0c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99dc0d0560 0x7f99dc162210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7f8782600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f877c0afc60 0x7f877c0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3511065094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 msgr2=0x7fd6d40cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7fd6d3e00640 1 --2- 172.21.15.67:0/3511065094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d40cf020 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00bec0 tx=0x7fd6cc02c830 comp rx=0 tx=0).stop 2024-05-30T16:44:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3511065094 shutdown_connections 2024-05-30T16:44:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7fd6d3e00640 1 --2- 172.21.15.67:0/3511065094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d40cf020 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3511065094 >> 172.21.15.67:0/3511065094 conn(0x7fd6d4072100 msgr2=0x7fd6d40b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f82e6800640 1 Processor -- start 2024-05-30T16:44:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7f99e0c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99dc0b3d30 con 0x7f99dc0d10a0 2024-05-30T16:44:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7f99e0c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99dc0d2930 con 0x7f99dc0d1480 2024-05-30T16:44:01.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f99e0c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f99dc0d2c10 con 0x7f99dc0d0560 2024-05-30T16:44:01.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f99d9400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f99dc0d1480 0x7f99dc161b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:49712/0 (socket says 172.21.15.67:49712) 2024-05-30T16:44:01.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f99d9400640 1 -- 172.21.15.67:0/1286430188 learned_addr learned my addr 172.21.15.67:0/1286430188 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:01.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3511065094 shutdown_connections 2024-05-30T16:44:01.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f82e6800640 1 -- start start 2024-05-30T16:44:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7ffad3400640 1 -- 172.21.15.67:0/513423989 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40072d0 con 0x7ffad800aad0 2024-05-30T16:44:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7f8782600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f877c0cca60 con 0x7f877c0b3a70 2024-05-30T16:44:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7f8782600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f877c0b1210 con 0x7f877c0afc60 2024-05-30T16:44:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7f8782600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f877c0b14f0 con 0x7f877c0b3150 2024-05-30T16:44:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.909+0000 7f877aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0b3150 0x7f877c0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f877aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0b3150 0x7f877c0b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:50454/0 (socket says 172.21.15.67:50454) 2024-05-30T16:44:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f877aa00640 1 -- 172.21.15.67:0/1405161938 learned_addr learned my addr 172.21.15.67:0/1405161938 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f877a000640 1 --2- 172.21.15.67:0/1405161938 >> v2:172.21.15.169:3300/0 conn(0x7f877c0afc60 0x7f877c0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f99da800640 1 --2- 172.21.15.67:0/1286430188 >> v2:172.21.15.67:3300/0 conn(0x7f99dc0d0560 0x7f99dc162210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 957568550 0 0) 0x7f99dc0d2930 con 0x7f99dc0d1480 2024-05-30T16:44:01.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99c0003520 con 0x7f99dc0d1480 2024-05-30T16:44:01.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f222a000640 1 --2- 172.21.15.67:0/1694422024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f2224169690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f22100097d0 tx=0x7f221002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dbf68452bb0252b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3643533010 0 0) 0x7f99c0003520 con 0x7f99dc0d1480 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 >> v2:172.21.15.67:3300/0 conn(0x7f99dc0d0560 msgr2=0x7f99dc162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 --2- 172.21.15.67:0/1286430188 >> v2:172.21.15.67:3300/0 conn(0x7f99dc0d0560 0x7f99dc162210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 >> v1:172.21.15.67:6789/0 conn(0x7f99dc0d10a0 legacy=0x7f99dc0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc0d2ef0 con 0x7f99dc0d1480 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f99c8001670 con 0x7f99dc0d1480 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 >> v1:172.21.15.169:6789/0 conn(0x7f99dc0d1480 legacy=0x7f99dc161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 --2- 172.21.15.67:0/1286430188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c00039b0 0x7f99c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 --2- 172.21.15.67:0/1286430188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99c0004800 0x7f99c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 --2- 172.21.15.67:0/1286430188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c00051c0 0x7f99c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc0d2c10 con 0x7f99c00051c0 2024-05-30T16:44:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f99c000c070 con 0x7f99c00039b0 2024-05-30T16:44:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.911+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f99c000c270 con 0x7f99c0004800 2024-05-30T16:44:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.910+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3511065094 wait complete. 2024-05-30T16:44:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f877aa00640 1 -- 172.21.15.67:0/1405161938 >> v2:172.21.15.169:3300/0 conn(0x7f877c0afc60 msgr2=0x7f877c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f877aa00640 1 --2- 172.21.15.67:0/1405161938 >> v2:172.21.15.169:3300/0 conn(0x7f877c0afc60 0x7f877c0b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f877aa00640 1 -- 172.21.15.67:0/1405161938 >> v1:172.21.15.67:6789/0 conn(0x7f877c0b3a70 legacy=0x7f877c0ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f877aa00640 1 -- 172.21.15.67:0/1405161938 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c0b17d0 con 0x7f877c0b3150 2024-05-30T16:44:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f99d9400640 1 --2- 172.21.15.67:0/1286430188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99c0004800 0x7f99c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f82e6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f82e00b3b70 0x7f82e00d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f82e6800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e00ced40 con 0x7f82e00b4b60 2024-05-30T16:44:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f82e6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e00b3f90 con 0x7f82e00d04a0 2024-05-30T16:44:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f82e6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f82e015fbb0 con 0x7f82e00b3b70 2024-05-30T16:44:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.912+0000 7f2223e00640 1 -- 172.21.15.67:0/1694422024 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210037070 con 0x7f22240cfa40 2024-05-30T16:44:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f2223e00640 1 -- 172.21.15.67:0/1694422024 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2210032440 con 0x7f22240cfa40 2024-05-30T16:44:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f2223e00640 1 -- 172.21.15.67:0/1694422024 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210032810 con 0x7f22240cfa40 2024-05-30T16:44:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f82dfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f82e00b3b70 0x7f82e00d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f82dfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f82e00b3b70 0x7f82e00d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36594/0 (socket says 172.21.15.67:36594) 2024-05-30T16:44:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f82dfe00640 1 -- 172.21.15.67:0/3064858439 learned_addr learned my addr 172.21.15.67:0/3064858439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f82dea00640 1 --1- 172.21.15.67:0/3064858439 >> v1:172.21.15.169:6789/0 conn(0x7f82e00d04a0 0x7f82e00b3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:49726/0 (socket says 172.21.15.67:49726) 2024-05-30T16:44:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f99d9400640 1 -- 172.21.15.67:0/1286430188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c00039b0 msgr2=0x7f99c0003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:44:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f99d9400640 1 --2- 172.21.15.67:0/1286430188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c00039b0 0x7f99c0003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f99d9400640 1 -- 172.21.15.67:0/1286430188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c00051c0 msgr2=0x7f99c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f99d9400640 1 --2- 172.21.15.67:0/1286430188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c00051c0 0x7f99c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f99d9400640 1 -- 172.21.15.67:0/1286430188 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c0003520 con 0x7f99c0004800 2024-05-30T16:44:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7fd6d3e00640 1 Processor -- start 2024-05-30T16:44:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7fd6d3e00640 1 -- start start 2024-05-30T16:44:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f82de000640 1 -- 172.21.15.67:0/3064858439 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1919943064 0 0) 0x7f82d0001110 con 0x7f82e00b4b60 2024-05-30T16:44:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.913+0000 7f82de000640 1 -- 172.21.15.67:0/3064858439 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82c4003520 con 0x7f82e00b4b60 2024-05-30T16:44:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.914+0000 7f99da800640 1 -- 172.21.15.67:0/1286430188 reap_dead start 2024-05-30T16:44:01.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.914+0000 7ffad0c00640 1 -- 172.21.15.67:0/513423989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab4005290 con 0x7ffad800aad0 2024-05-30T16:44:01.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.914+0000 7f99d9400640 1 --2- 172.21.15.67:0/1286430188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99c0004800 0x7f99c0004c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c240 tx=0x7f99c0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751317641c7eaf3d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.914+0000 7f82dfe00640 1 -- 172.21.15.67:0/3064858439 >> v1:172.21.15.169:6789/0 conn(0x7f82e00d04a0 legacy=0x7f82e00b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.914+0000 7f82dfe00640 1 -- 172.21.15.67:0/3064858439 >> v1:172.21.15.67:6789/0 conn(0x7f82e00b4b60 legacy=0x7f82e00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.914+0000 7f82dfe00640 1 -- 172.21.15.67:0/3064858439 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e00cefc0 con 0x7f82e00b3b70 2024-05-30T16:44:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7fd6d3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d4073bc0 0x7fd6d4153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f877aa00640 1 --2- 172.21.15.67:0/1405161938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0b3150 0x7f877c0b0b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f87700091e0 tx=0x7f877002b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b220a1ca303be64b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7fd6d3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d40b26d0 0x7fd6d4151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f222aa00640 1 -- 172.21.15.67:0/1694422024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 msgr2=0x7f2224169690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f222aa00640 1 --2- 172.21.15.67:0/1694422024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f2224169690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f22100097d0 tx=0x7f221002a210 comp rx=0 tx=0).stop 2024-05-30T16:44:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f222aa00640 1 -- 172.21.15.67:0/1694422024 shutdown_connections 2024-05-30T16:44:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f222aa00640 1 --2- 172.21.15.67:0/1694422024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f2224169690 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f222aa00640 1 -- 172.21.15.67:0/1694422024 >> 172.21.15.67:0/1694422024 conn(0x7f222408bfc0 msgr2=0x7f22240d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8007d10 con 0x7f99c0004800 2024-05-30T16:44:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c80058d0 con 0x7f99c0004800 2024-05-30T16:44:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f99d8a00640 1 -- 172.21.15.67:0/1286430188 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8005d80 con 0x7f99c0004800 2024-05-30T16:44:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7fd6d3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7fd6d3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d40cf560 con 0x7fd6d40b26d0 2024-05-30T16:44:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7fd6d3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d40b3a60 con 0x7fd6d4073bc0 2024-05-30T16:44:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7fd6d3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d40cf6e0 con 0x7fd6d40b35d0 2024-05-30T16:44:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f82dfe00640 1 --2- 172.21.15.67:0/3064858439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e00b3b70 0x7f82e00d0930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009e50 tx=0x7f82d802a7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea6cde6df94e5865 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f8781000640 1 -- 172.21.15.67:0/1405161938 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770037070 con 0x7f877c0b3150 2024-05-30T16:44:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f8781000640 1 -- 172.21.15.67:0/1405161938 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8770030710 con 0x7f877c0b3150 2024-05-30T16:44:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7f8781000640 1 -- 172.21.15.67:0/1405161938 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770030ae0 con 0x7f877c0b3150 2024-05-30T16:44:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7fd6d2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d4073bc0 0x7fd6d4153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.915+0000 7fd6d2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d40b26d0 0x7fd6d4151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7fd6d2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d4073bc0 0x7fd6d4153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36610/0 (socket says 172.21.15.67:36610) 2024-05-30T16:44:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7fd6d2a00640 1 -- 172.21.15.67:0/3338826073 learned_addr learned my addr 172.21.15.67:0/3338826073 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f82de000640 1 -- 172.21.15.67:0/3064858439 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8033610 con 0x7f82e00b3b70 2024-05-30T16:44:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f82de000640 1 -- 172.21.15.67:0/3064858439 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d802f070 con 0x7f82e00b3b70 2024-05-30T16:44:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f82de000640 1 -- 172.21.15.67:0/3064858439 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8033a10 con 0x7f82e00b3b70 2024-05-30T16:44:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7fd6d3400640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7fd6d2000640 1 -- 172.21.15.67:0/3338826073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 msgr2=0x7fd6d414d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f99e0c00640 1 -- 172.21.15.67:0/1286430188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99c0004800 msgr2=0x7f99c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f99e0c00640 1 --2- 172.21.15.67:0/1286430188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99c0004800 0x7f99c0004c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c240 tx=0x7f99c0005830 comp rx=0 tx=0).stop 2024-05-30T16:44:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f99e0c00640 1 -- 172.21.15.67:0/1286430188 shutdown_connections 2024-05-30T16:44:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7fd6d2000640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d414d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7fd6d2000640 1 -- 172.21.15.67:0/3338826073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d4073bc0 msgr2=0x7fd6d4153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7fd6d2000640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d4073bc0 0x7fd6d4153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7fd6d2000640 1 -- 172.21.15.67:0/3338826073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d41540c0 con 0x7fd6d40b26d0 2024-05-30T16:44:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f222aa00640 1 -- 172.21.15.67:0/1694422024 shutdown_connections 2024-05-30T16:44:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f222aa00640 1 -- 172.21.15.67:0/1694422024 wait complete. 2024-05-30T16:44:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f8782600640 1 -- 172.21.15.67:0/1405161938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0b3150 msgr2=0x7f877c0b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f8782600640 1 --2- 172.21.15.67:0/1405161938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0b3150 0x7f877c0b0b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f87700091e0 tx=0x7f877002b3f0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f99e0c00640 1 --2- 172.21.15.67:0/1286430188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99c0004800 0x7f99c0004c80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.916+0000 7f99e0c00640 1 -- 172.21.15.67:0/1286430188 >> 172.21.15.67:0/1286430188 conn(0x7f99dc08bfc0 msgr2=0x7f99dc0d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7f8782600640 1 -- 172.21.15.67:0/1405161938 shutdown_connections 2024-05-30T16:44:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7f8782600640 1 --2- 172.21.15.67:0/1405161938 >> v2:172.21.15.169:3300/0 conn(0x7f877c0afc60 0x7f877c0b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7f8782600640 1 --2- 172.21.15.67:0/1405161938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0b3150 0x7f877c0b0b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7f8782600640 1 -- 172.21.15.67:0/1405161938 >> 172.21.15.67:0/1405161938 conn(0x7f877c073010 msgr2=0x7f877c073420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7f8782600640 1 -- 172.21.15.67:0/1405161938 shutdown_connections 2024-05-30T16:44:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7f8782600640 1 -- 172.21.15.67:0/1405161938 wait complete. 2024-05-30T16:44:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7f99e0c00640 1 -- 172.21.15.67:0/1286430188 shutdown_connections 2024-05-30T16:44:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7f82e6800640 1 -- 172.21.15.67:0/3064858439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e00b3b70 msgr2=0x7f82e00d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7fd6d2000640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d40b26d0 0x7fd6d4151520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc000f60 tx=0x7fd6bc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.917+0000 7f82e6800640 1 --2- 172.21.15.67:0/3064858439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e00b3b70 0x7f82e00d0930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009e50 tx=0x7f82d802a7d0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7fd6cb400640 1 -- 172.21.15.67:0/3338826073 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc017070 con 0x7fd6d40b26d0 2024-05-30T16:44:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7f82e6800640 1 -- 172.21.15.67:0/3064858439 shutdown_connections 2024-05-30T16:44:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7f82e6800640 1 --2- 172.21.15.67:0/3064858439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e00b3b70 0x7f82e00d0930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7f82e6800640 1 -- 172.21.15.67:0/3064858439 >> 172.21.15.67:0/3064858439 conn(0x7f82e006bae0 msgr2=0x7f82e006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7fd6d3400640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7fd6cb400640 1 -- 172.21.15.67:0/3338826073 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6bc005ce0 con 0x7fd6d40b26d0 2024-05-30T16:44:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7fd6cb400640 1 -- 172.21.15.67:0/3338826073 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0072d0 con 0x7fd6d40b26d0 2024-05-30T16:44:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d41543a0 con 0x7fd6d40b26d0 2024-05-30T16:44:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d4154960 con 0x7fd6d40b26d0 2024-05-30T16:44:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7fd6d2a00640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d4073bc0 0x7fd6d4153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:01.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.918+0000 7f99e0c00640 1 -- 172.21.15.67:0/1286430188 wait complete. 2024-05-30T16:44:01.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.919+0000 7f222aa00640 1 Processor -- start 2024-05-30T16:44:01.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.919+0000 7f222aa00640 1 -- start start 2024-05-30T16:44:01.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.919+0000 7f8782600640 1 Processor -- start 2024-05-30T16:44:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.919+0000 7ffad3400640 1 -- 172.21.15.67:0/513423989 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffad401a070 con 0x7ffad800aad0 2024-05-30T16:44:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.919+0000 7f8782600640 1 -- start start 2024-05-30T16:44:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.920+0000 7f8782600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0afc60 0x7f877c1e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.920+0000 7f8782600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f877c0b3150 0x7f877c155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.920+0000 7f8782600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f877c0b3a70 0x7f877c1e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.920+0000 7f8782600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f877c0b3ea0 con 0x7f877c0b3a70 2024-05-30T16:44:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.921+0000 7f8782600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f877c0ad130 con 0x7f877c0b3150 2024-05-30T16:44:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.921+0000 7f8782600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f877c073b30 con 0x7f877c0afc60 2024-05-30T16:44:01.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.921+0000 7f99e0c00640 1 Processor -- start 2024-05-30T16:44:01.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.921+0000 7f222aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f22241f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.921+0000 7ffad3400640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffac006c920 0x7ffac006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.921+0000 7f99e0c00640 1 -- start start 2024-05-30T16:44:01.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.921+0000 7f222aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22240d16a0 0x7f22240cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.921+0000 7ffad3400640 1 -- 172.21.15.67:0/513423989 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ffad408ad30 con 0x7ffad800aad0 2024-05-30T16:44:01.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.921+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ac005290 con 0x7fd6d40b26d0 2024-05-30T16:44:01.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f877aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0afc60 0x7f877c1e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f877aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0afc60 0x7f877c1e79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:50480/0 (socket says 172.21.15.67:50480) 2024-05-30T16:44:01.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f877b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f877c0b3a70 0x7f877c1e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f877aa00640 1 -- 172.21.15.67:0/3456267675 learned_addr learned my addr 172.21.15.67:0/3456267675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f877a000640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f877c0b3150 0x7f877c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7ffadea00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffac006c920 0x7ffac006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f222aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f222415e870 0x7f22240cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f222aa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22240d4fb0 con 0x7f22240d16a0 2024-05-30T16:44:01.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f222aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f22240d0820 con 0x7f222415e870 2024-05-30T16:44:01.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f222aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22240d35f0 con 0x7f22240cfa40 2024-05-30T16:44:01.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f2229600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f22241f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f2229600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f22241f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:50490/0 (socket says 172.21.15.67:50490) 2024-05-30T16:44:01.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.922+0000 7f2229600640 1 -- 172.21.15.67:0/3866342047 learned_addr learned my addr 172.21.15.67:0/3866342047 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f7045800640 1 Processor -- start 2024-05-30T16:44:01.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f877b400640 1 -- 172.21.15.67:0/3456267675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0afc60 msgr2=0x7f877c1e79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f877b400640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0afc60 0x7f877c1e79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f877b400640 1 -- 172.21.15.67:0/3456267675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f877c0b3150 msgr2=0x7f877c155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f877b400640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f877c0b3150 0x7f877c155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f877b400640 1 -- 172.21.15.67:0/3456267675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c1fbeb0 con 0x7f877c0b3a70 2024-05-30T16:44:01.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f2228c00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22240d16a0 0x7f22240cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f877b400640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f877c0b3a70 0x7f877c1e8b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f876800c390 tx=0x7f876800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f99e0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c0004800 0x7f99dc14df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7fd6cb400640 1 -- 172.21.15.67:0/3338826073 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6bc01a070 con 0x7fd6d40b26d0 2024-05-30T16:44:01.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f222a000640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f222415e870 0x7f22240cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f99e0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99dc15c470 0x7f99dc15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7ffadea00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffac006c920 0x7ffac006ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffacc034000 tx=0x7ffacc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f82e6800640 1 -- 172.21.15.67:0/3064858439 shutdown_connections 2024-05-30T16:44:01.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f82e6800640 1 -- 172.21.15.67:0/3064858439 wait complete. 2024-05-30T16:44:01.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f2228c00640 1 -- 172.21.15.67:0/3866342047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 msgr2=0x7f22241f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f2228c00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f22241f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f2228c00640 1 -- 172.21.15.67:0/3866342047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f222415e870 msgr2=0x7f22240cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f2228c00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f222415e870 0x7f22240cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f2228c00640 1 -- 172.21.15.67:0/3866342047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22240b73f0 con 0x7f22240d16a0 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f99e0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99dc0cab40 0x7f99dc0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f99e0c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc0d2570 con 0x7f99dc15c470 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f99e0c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc0d34b0 con 0x7f99c0004800 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f99e0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f99dc0d26f0 con 0x7f99dc0cab40 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f99d9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c0004800 0x7f99dc14df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f99d9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99dc15c470 0x7f99dc15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f99d9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c0004800 0x7f99dc14df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36648/0 (socket says 172.21.15.67:36648) 2024-05-30T16:44:01.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f99d9e00640 1 -- 172.21.15.67:0/3131897450 learned_addr learned my addr 172.21.15.67:0/3131897450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f222a000640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f222415e870 0x7f22240cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:01.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f99da800640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99dc0cab40 0x7f99dc0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.924+0000 7f2228c00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22240d16a0 0x7f22240cd600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f221c000fb0 tx=0x7f221c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7fd6cb400640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd6b006c940 0x7fd6b006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f99d9400640 1 -- 172.21.15.67:0/3131897450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99dc0cab40 msgr2=0x7f99dc0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f99d9400640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99dc0cab40 0x7f99dc0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f99d9400640 1 -- 172.21.15.67:0/3131897450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c0004800 msgr2=0x7f99dc14df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f99d9400640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c0004800 0x7f99dc14df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f99d9400640 1 -- 172.21.15.67:0/3131897450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc0cb500 con 0x7f99dc15c470 2024-05-30T16:44:01.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f99d9e00640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c0004800 0x7f99dc14df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:44:01.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.923+0000 7f7045800640 1 -- start start 2024-05-30T16:44:01.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f7045800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70400d2db0 0x7f70400b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f7045800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7040163500 con 0x7f704015ed20 2024-05-30T16:44:01.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f7045800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7040162740 con 0x7f70400d4d60 2024-05-30T16:44:01.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f7045800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70400d5540 con 0x7f70400d2db0 2024-05-30T16:44:01.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7f99d9400640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99dc15c470 0x7f99dc15a710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f99c8030000 tx=0x7f99dc06ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.925+0000 7fd6cb400640 1 -- 172.21.15.67:0/3338826073 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd6bc08adc0 con 0x7fd6d40b26d0 2024-05-30T16:44:01.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.926+0000 7f703ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70400d2db0 0x7f70400b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.926+0000 7f703ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70400d2db0 0x7f70400b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:50516/0 (socket says 172.21.15.67:50516) 2024-05-30T16:44:01.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.926+0000 7fd6d2a00640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd6b006c940 0x7fd6b006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.926+0000 7f703f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f704015ed20 0x7f704015f1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52352/0 (socket says 172.21.15.67:52352) 2024-05-30T16:44:01.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.926+0000 7f703f400640 1 -- 172.21.15.67:0/2438403981 learned_addr learned my addr 172.21.15.67:0/2438403981 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:01.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.926+0000 7f703e000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f70400d4d60 0x7f70400d3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:49732/0 (socket says 172.21.15.67:49732) 2024-05-30T16:44:01.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.926+0000 7f82e6800640 1 Processor -- start 2024-05-30T16:44:01.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.927+0000 7f82e6800640 1 -- start start 2024-05-30T16:44:01.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.927+0000 7fd6d2a00640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd6b006c940 0x7fd6b006ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4003f70 tx=0x7fd6c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.927+0000 7f703d600640 1 -- 172.21.15.67:0/2438403981 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955811288 0 0) 0x7f7040162740 con 0x7f70400d4d60 2024-05-30T16:44:01.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f99d2000640 1 -- 172.21.15.67:0/3131897450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8040020 con 0x7f99dc15c470 2024-05-30T16:44:01.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f82e6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82e00b3b70 0x7f82e0157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f703ea00640 1 -- 172.21.15.67:0/2438403981 >> v1:172.21.15.169:6789/0 conn(0x7f70400d4d60 legacy=0x7f70400d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f703ea00640 1 -- 172.21.15.67:0/2438403981 >> v1:172.21.15.67:6789/0 conn(0x7f704015ed20 legacy=0x7f704015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f2221600640 1 -- 172.21.15.67:0/3866342047 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c017070 con 0x7f22240d16a0 2024-05-30T16:44:01.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f82e6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e00b4b60 0x7f82e00c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f82e6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e00d04a0 0x7f82e00c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f82e6800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f82e015ede0 con 0x7f82e00b3b70 2024-05-30T16:44:01.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f82e6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82e00cf580 con 0x7f82e00d04a0 2024-05-30T16:44:01.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f82e6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f82e015ef60 con 0x7f82e00b4b60 2024-05-30T16:44:01.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc15cd50 con 0x7f99dc15c470 2024-05-30T16:44:01.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22240b76d0 con 0x7f22240d16a0 2024-05-30T16:44:01.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.928+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22240b7c90 con 0x7f22240d16a0 2024-05-30T16:44:01.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.929+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc15d310 con 0x7f99dc15c470 2024-05-30T16:44:01.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.929+0000 7f703ea00640 1 -- 172.21.15.67:0/2438403981 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7040162a20 con 0x7f70400d2db0 2024-05-30T16:44:01.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.929+0000 7f82df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82e00b3b70 0x7f82e0157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.929+0000 7f82df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82e00b3b70 0x7f82e0157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53120/0 (socket says 172.21.15.67:53120) 2024-05-30T16:44:01.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.929+0000 7f82df400640 1 -- 172.21.15.67:0/462777848 learned_addr learned my addr 172.21.15.67:0/462777848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.929+0000 7f82dea00640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e00b4b60 0x7f82e00c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.929+0000 7f82dfe00640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e00d04a0 0x7f82e00c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.929+0000 7ffad3400640 1 -- 172.21.15.67:0/513423989 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffad4058b10 con 0x7ffad800aad0 2024-05-30T16:44:01.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.930+0000 7f876f400640 1 -- 172.21.15.67:0/3456267675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768017070 con 0x7f877c0b3a70 2024-05-30T16:44:01.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.930+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877c1e8070 con 0x7f877c0b3a70 2024-05-30T16:44:01.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.930+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877c1fc430 con 0x7f877c0b3a70 2024-05-30T16:44:01.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.930+0000 7f82df400640 1 -- 172.21.15.67:0/462777848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e00b4b60 msgr2=0x7f82e00c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.930+0000 7f82df400640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e00b4b60 0x7f82e00c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.930+0000 7f82df400640 1 -- 172.21.15.67:0/462777848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e00d04a0 msgr2=0x7f82e00c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.930+0000 7f82df400640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e00d04a0 0x7f82e00c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.930+0000 7f82df400640 1 -- 172.21.15.67:0/462777848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e0158490 con 0x7f82e00b3b70 2024-05-30T16:44:01.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.930+0000 7f82df400640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82e00b3b70 0x7f82e0157f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82d000c700 tx=0x7f82d000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.931+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f877c0689e0 con 0x7f877c0b3a70 2024-05-30T16:44:01.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.932+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99dc0690f0 con 0x7f99dc15c470 2024-05-30T16:44:01.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.932+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22240689e0 con 0x7f22240d16a0 2024-05-30T16:44:01.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.932+0000 7f703ea00640 1 --2- 172.21.15.67:0/2438403981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70400d2db0 0x7f70400b3b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f70300097d0 tx=0x7f703002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cfeef3ef6201d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.933+0000 7f703d600640 1 -- 172.21.15.67:0/2438403981 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030037070 con 0x7f70400d2db0 2024-05-30T16:44:01.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f2221600640 1 -- 172.21.15.67:0/3866342047 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c005d30 con 0x7f22240d16a0 2024-05-30T16:44:01.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f876f400640 1 -- 172.21.15.67:0/3456267675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87680052d0 con 0x7f877c0b3a70 2024-05-30T16:44:01.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f876f400640 1 -- 172.21.15.67:0/3456267675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768007270 con 0x7f877c0b3a70 2024-05-30T16:44:01.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f99d2000640 1 -- 172.21.15.67:0/3131897450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c8007dd0 con 0x7f99dc15c470 2024-05-30T16:44:01.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f99d2000640 1 -- 172.21.15.67:0/3131897450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8005890 con 0x7f99dc15c470 2024-05-30T16:44:01.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.933+0000 7f703d600640 1 -- 172.21.15.67:0/2438403981 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7030032440 con 0x7f70400d2db0 2024-05-30T16:44:01.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.933+0000 7f703d600640 1 -- 172.21.15.67:0/2438403981 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030032810 con 0x7f70400d2db0 2024-05-30T16:44:01.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f7045800640 1 -- 172.21.15.67:0/2438403981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70400d2db0 msgr2=0x7f70400b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f7045800640 1 --2- 172.21.15.67:0/2438403981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70400d2db0 0x7f70400b3b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f70300097d0 tx=0x7f703002a180 comp rx=0 tx=0).stop 2024-05-30T16:44:01.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f82d7400640 1 -- 172.21.15.67:0/462777848 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00050d0 con 0x7f82e00b3b70 2024-05-30T16:44:01.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e01609f0 con 0x7f82e00b3b70 2024-05-30T16:44:01.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f7045800640 1 -- 172.21.15.67:0/2438403981 shutdown_connections 2024-05-30T16:44:01.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f7045800640 1 --2- 172.21.15.67:0/2438403981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70400d2db0 0x7f70400b3b50 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f70300097d0 tx=0x7f703002a180 comp rx=0 tx=0).stop 2024-05-30T16:44:01.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f7045800640 1 -- 172.21.15.67:0/2438403981 >> 172.21.15.67:0/2438403981 conn(0x7f704008bfc0 msgr2=0x7f70400d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:01.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7fd6cb400640 1 -- 172.21.15.67:0/3338826073 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6bc058af0 con 0x7fd6d40b26d0 2024-05-30T16:44:01.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e0160fb0 con 0x7f82e00b3b70 2024-05-30T16:44:01.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.934+0000 7f2221600640 1 -- 172.21.15.67:0/3866342047 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c021060 con 0x7f22240d16a0 2024-05-30T16:44:01.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.935+0000 7f7045800640 1 -- 172.21.15.67:0/2438403981 shutdown_connections 2024-05-30T16:44:01.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.936+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e00689e0 con 0x7f82e00b3b70 2024-05-30T16:44:01.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.935+0000 7f7045800640 1 -- 172.21.15.67:0/2438403981 wait complete. 2024-05-30T16:44:01.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.938+0000 7f7045800640 1 Processor -- start 2024-05-30T16:44:01.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.938+0000 7f7045800640 1 -- start start 2024-05-30T16:44:01.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.937+0000 7f876f400640 1 -- 172.21.15.67:0/3456267675 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f876801a070 con 0x7f877c0b3a70 2024-05-30T16:44:01.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.939+0000 7f876f400640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f875c06c8f0 0x7f875c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.940+0000 7f876f400640 1 -- 172.21.15.67:0/3456267675 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8768056df0 con 0x7f877c0b3a70 2024-05-30T16:44:01.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.939+0000 7f82d7400640 1 -- 172.21.15.67:0/462777848 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d0005270 con 0x7f82e00b3b70 2024-05-30T16:44:01.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.939+0000 7f82d7400640 1 -- 172.21.15.67:0/462777848 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d000a840 con 0x7f82e00b3b70 2024-05-30T16:44:01.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.939+0000 7f82d7400640 1 -- 172.21.15.67:0/462777848 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82d000aaa0 con 0x7f82e00b3b70 2024-05-30T16:44:01.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.941+0000 7f7045800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70400d4d60 0x7f70400b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.941+0000 7f82d7400640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82c006c8f0 0x7f82c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.941+0000 7f7045800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f704015ed20 0x7f70402020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.942+0000 7f7045800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70400b9a00 0x7f7040159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.942+0000 7f703ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70400d4d60 0x7f70400b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.942+0000 7f82dea00640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82c006c8f0 0x7f82c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.942+0000 7f703ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70400d4d60 0x7f70400b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36664/0 (socket says 172.21.15.67:36664) 2024-05-30T16:44:01.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.942+0000 7f703e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f704015ed20 0x7f70402020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.942+0000 7f703e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f704015ed20 0x7f70402020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:50530/0 (socket says 172.21.15.67:50530) 2024-05-30T16:44:01.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.943+0000 7f703ea00640 1 -- 172.21.15.67:0/2992514462 learned_addr learned my addr 172.21.15.67:0/2992514462 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:01.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.940+0000 7f2221600640 1 -- 172.21.15.67:0/3866342047 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f221c01a070 con 0x7f22240d16a0 2024-05-30T16:44:01.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.941+0000 7f2221600640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.941+0000 7f2221600640 1 -- 172.21.15.67:0/3866342047 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f221c08b310 con 0x7f22240d16a0 2024-05-30T16:44:01.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.942+0000 7f2229600640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.943+0000 7f877aa00640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f875c06c8f0 0x7f875c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.943+0000 7f703f400640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70400b9a00 0x7f7040159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.943+0000 7f82dea00640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82c006c8f0 0x7f82c006edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82e00cdf80 tx=0x7f82cc000ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.943+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f70400d51a0 con 0x7f70400b9a00 2024-05-30T16:44:01.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.943+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7040162fe0 con 0x7f70400d4d60 2024-05-30T16:44:01.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.943+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70400d5320 con 0x7f704015ed20 2024-05-30T16:44:01.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.943+0000 7f2229600640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f220406c8f0 0x7f220406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2218001f90 tx=0x7f2218009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.944+0000 7f877aa00640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f875c06c8f0 0x7f875c06edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f877002b430 tx=0x7f8770008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.945+0000 7f703f400640 1 -- 172.21.15.67:0/2992514462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f704015ed20 msgr2=0x7f70402020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.945+0000 7f99d2000640 1 -- 172.21.15.67:0/3131897450 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99c8043020 con 0x7f99dc15c470 2024-05-30T16:44:01.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.945+0000 7f703f400640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f704015ed20 0x7f70402020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.945+0000 7f703f400640 1 -- 172.21.15.67:0/2992514462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70400d4d60 msgr2=0x7f70400b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:01.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.945+0000 7f703f400640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70400d4d60 0x7f70400b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:01.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.945+0000 7f703f400640 1 -- 172.21.15.67:0/2992514462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70400b17e0 con 0x7f70400b9a00 2024-05-30T16:44:01.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.945+0000 7f703e000640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f704015ed20 0x7f70402020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:44:01.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.945+0000 7f703ea00640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70400d4d60 0x7f70400b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:44:01.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.945+0000 7f703f400640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70400b9a00 0x7f7040159e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f703800ca70 tx=0x7f703800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.947+0000 7f876f400640 1 -- 172.21.15.67:0/3456267675 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8768021080 con 0x7f877c0b3a70 2024-05-30T16:44:01.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.947+0000 7f99d2000640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f99bc06c8f0 0x7f99bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.947+0000 7f99d2000640 1 -- 172.21.15.67:0/3131897450 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f99c80abcd0 con 0x7f99dc15c470 2024-05-30T16:44:01.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.948+0000 7f99d9e00640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f99bc06c8f0 0x7f99bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.949+0000 7f99d9e00640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f99bc06c8f0 0x7f99bc06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f99d4007fd0 tx=0x7f99d4003de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:01.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.952+0000 7f2221600640 1 -- 172.21.15.67:0/3866342047 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f221c0590e0 con 0x7f22240d16a0 2024-05-30T16:44:01.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.957+0000 7f99d2000640 1 -- 172.21.15.67:0/3131897450 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99c804b020 con 0x7f99dc15c470 2024-05-30T16:44:01.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.957+0000 7f82d7400640 1 -- 172.21.15.67:0/462777848 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f82d0058100 con 0x7f82e00b3b70 2024-05-30T16:44:01.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.959+0000 7f7036a00640 1 -- 172.21.15.67:0/2992514462 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038017070 con 0x7f70400b9a00 2024-05-30T16:44:01.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.959+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70400b1ac0 con 0x7f70400b9a00 2024-05-30T16:44:01.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.959+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70400b2080 con 0x7f70400b9a00 2024-05-30T16:44:01.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.960+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70400cb770 con 0x7f70400b9a00 2024-05-30T16:44:01.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.962+0000 7f7036a00640 1 -- 172.21.15.67:0/2992514462 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703800aa90 con 0x7f70400b9a00 2024-05-30T16:44:01.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.962+0000 7f7036a00640 1 -- 172.21.15.67:0/2992514462 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038021060 con 0x7f70400b9a00 2024-05-30T16:44:01.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.966+0000 7f82d7400640 1 -- 172.21.15.67:0/462777848 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82e00ced40 con 0x7f82e00b3b70 2024-05-30T16:44:01.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.967+0000 7f7036a00640 1 -- 172.21.15.67:0/2992514462 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f703801a070 con 0x7f70400b9a00 2024-05-30T16:44:01.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.971+0000 7f7036a00640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f702006caf0 0x7f702006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:01.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.971+0000 7f703ea00640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f702006caf0 0x7f702006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:01.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.972+0000 7f7036a00640 1 -- 172.21.15.67:0/2992514462 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f703808bb50 con 0x7f70400b9a00 2024-05-30T16:44:01.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.972+0000 7f7036a00640 1 -- 172.21.15.67:0/2992514462 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70380b9b80 con 0x7f70400b9a00 2024-05-30T16:44:01.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:01.977+0000 7f703ea00640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f702006caf0 0x7f702006efb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f703002a6f0 tx=0x7f703002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:02.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.364+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f40a0005580 con 0x7f40c00e5d70 2024-05-30T16:44:02.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.365+0000 7ffad0c00640 1 -- 172.21.15.67:0/513423989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ffab4005850 con 0x7ffad800aad0 2024-05-30T16:44:02.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.366+0000 7f40bea00640 1 -- 172.21.15.67:0/1868618464 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40c80e9020 con 0x7f40c00e5d70 2024-05-30T16:44:02.366 INFO:teuthology.orchestra.run.smithi067.stdout:64424509448 2024-05-30T16:44:02.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.367+0000 7ffad3400640 1 -- 172.21.15.67:0/513423989 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffad405ce10 con 0x7ffad800aad0 2024-05-30T16:44:02.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.369+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f40a806c9c0 msgr2=0x7f40a806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.369+0000 7f40cca00640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f40a806c9c0 0x7f40a806ee80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009090 tx=0x7f40b402e000 comp rx=0 tx=0).stop 2024-05-30T16:44:02.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.370+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c00e5d70 msgr2=0x7f40c00e6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.370+0000 7f40cca00640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c00e5d70 0x7f40c00e6220 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f40c8069090 tx=0x7f40c8068ef0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.370+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 shutdown_connections 2024-05-30T16:44:02.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.370+0000 7f40cca00640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f40a806c9c0 0x7f40a806ee80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.370+0000 7f40cca00640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c00e5d70 0x7f40c00e6220 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.370+0000 7f40cca00640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c00388e0 0x7f40c00d81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.370+0000 7f40cca00640 1 --2- 172.21.15.67:0/1868618464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac0039d0 0x7f40c004f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.370+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 >> 172.21.15.67:0/1868618464 conn(0x7f40c0058ee0 msgr2=0x7f40c0004a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.372 INFO:teuthology.orchestra.run.smithi067.stdout:64424509448 2024-05-30T16:44:02.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.371+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 shutdown_connections 2024-05-30T16:44:02.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.371+0000 7f40cca00640 1 -- 172.21.15.67:0/1868618464 wait complete. 2024-05-30T16:44:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.377+0000 7ffad0c00640 1 -- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffac006c920 msgr2=0x7ffac006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.377+0000 7ffad0c00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffac006c920 0x7ffac006ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffacc034000 tx=0x7ffacc02e000 comp rx=0 tx=0).stop 2024-05-30T16:44:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.377+0000 7ffad0c00640 1 -- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffad800aad0 msgr2=0x7ffad80e5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.377+0000 7ffad0c00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffad800aad0 0x7ffad80e5430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffad400c4b0 tx=0x7ffad400c8b0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.378+0000 7ffad0c00640 1 -- 172.21.15.67:0/513423989 shutdown_connections 2024-05-30T16:44:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.378+0000 7ffad0c00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffac006c920 0x7ffac006ede0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.378+0000 7ffad0c00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad80df130 0x7ffad8048600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.378+0000 7ffad0c00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad8054690 0x7ffad80ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.378+0000 7ffad0c00640 1 --2- 172.21.15.67:0/513423989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffad800aad0 0x7ffad80e5430 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.378+0000 7ffad0c00640 1 -- 172.21.15.67:0/513423989 >> 172.21.15.67:0/513423989 conn(0x7ffad8009fe0 msgr2=0x7ffad80e2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.379+0000 7ffad0c00640 1 -- 172.21.15.67:0/513423989 shutdown_connections 2024-05-30T16:44:02.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.379+0000 7ffad0c00640 1 -- 172.21.15.67:0/513423989 wait complete. 2024-05-30T16:44:02.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.383+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd6ac005580 con 0x7fd6d40b26d0 2024-05-30T16:44:02.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.384+0000 7fd6cb400640 1 -- 172.21.15.67:0/3338826073 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6bc05cdf0 con 0x7fd6d40b26d0 2024-05-30T16:44:02.385 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-05-30T16:44:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.387+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd6b006c940 msgr2=0x7fd6b006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.387+0000 7fd6d3e00640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd6b006c940 0x7fd6b006ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4003f70 tx=0x7fd6c4009040 comp rx=0 tx=0).stop 2024-05-30T16:44:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.387+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d40b26d0 msgr2=0x7fd6d4151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.387+0000 7fd6d3e00640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d40b26d0 0x7fd6d4151520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc000f60 tx=0x7fd6bc00c800 comp rx=0 tx=0).stop 2024-05-30T16:44:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.389+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 shutdown_connections 2024-05-30T16:44:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.389+0000 7fd6d3e00640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd6b006c940 0x7fd6b006ee00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.389+0000 7fd6d3e00640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6d40b35d0 0x7fd6d414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.389+0000 7fd6d3e00640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6d40b26d0 0x7fd6d4151520 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.389+0000 7fd6d3e00640 1 --2- 172.21.15.67:0/3338826073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d4073bc0 0x7fd6d4153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.389+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 >> 172.21.15.67:0/3338826073 conn(0x7fd6d4072100 msgr2=0x7fd6d406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.389+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 shutdown_connections 2024-05-30T16:44:02.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.389+0000 7fd6d3e00640 1 -- 172.21.15.67:0/3338826073 wait complete. 2024-05-30T16:44:02.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.2 2024-05-30T16:44:02.399 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:02.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.404+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f22240ba500 con 0x7f22240d16a0 2024-05-30T16:44:02.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.405+0000 7f2221600640 1 -- 172.21.15.67:0/3866342047 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f221c05d3e0 con 0x7f22240d16a0 2024-05-30T16:44:02.406 INFO:teuthology.orchestra.run.smithi067.stdout:64424509447 2024-05-30T16:44:02.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f877c09e290 con 0x7f877c0b3a70 2024-05-30T16:44:02.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f220406c8f0 msgr2=0x7f220406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f220406c8f0 0x7f220406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2218001f90 tx=0x7f2218009040 comp rx=0 tx=0).stop 2024-05-30T16:44:02.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22240d16a0 msgr2=0x7f22240cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22240d16a0 0x7f22240cd600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f221c000fb0 tx=0x7f221c00c800 comp rx=0 tx=0).stop 2024-05-30T16:44:02.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 shutdown_connections 2024-05-30T16:44:02.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f220406c8f0 0x7f220406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f222415e870 0x7f22240cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22240d16a0 0x7f22240cd600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 --2- 172.21.15.67:0/3866342047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22240cfa40 0x7f22241f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.407+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 >> 172.21.15.67:0/3866342047 conn(0x7f222408bfc0 msgr2=0x7f22240b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.408+0000 7f876f400640 1 -- 172.21.15.67:0/3456267675 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8768058d50 con 0x7f877c0b3a70 2024-05-30T16:44:02.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.408+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 shutdown_connections 2024-05-30T16:44:02.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.408+0000 7f222aa00640 1 -- 172.21.15.67:0/3866342047 wait complete. 2024-05-30T16:44:02.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.3 2024-05-30T16:44:02.410 INFO:teuthology.orchestra.run.smithi067.stdout:64424509447 2024-05-30T16:44:02.411 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:02.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.411+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f875c06c8f0 msgr2=0x7f875c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.411+0000 7f8782600640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f875c06c8f0 0x7f875c06edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f877002b430 tx=0x7f8770008030 comp rx=0 tx=0).stop 2024-05-30T16:44:02.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.411+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f877c0b3a70 msgr2=0x7f877c1e8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.411+0000 7f8782600640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f877c0b3a70 0x7f877c1e8b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f876800c390 tx=0x7f876800c850 comp rx=0 tx=0).stop 2024-05-30T16:44:02.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.412+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 shutdown_connections 2024-05-30T16:44:02.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.412+0000 7f8782600640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f875c06c8f0 0x7f875c06edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.412+0000 7f8782600640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f877c0b3a70 0x7f877c1e8b50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.412+0000 7f8782600640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f877c0b3150 0x7f877c155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.412+0000 7f8782600640 1 --2- 172.21.15.67:0/3456267675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f877c0afc60 0x7f877c1e79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.412+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 >> 172.21.15.67:0/3456267675 conn(0x7f877c073010 msgr2=0x7f877c159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.412+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 shutdown_connections 2024-05-30T16:44:02.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.412+0000 7f8782600640 1 -- 172.21.15.67:0/3456267675 wait complete. 2024-05-30T16:44:02.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-05-30T16:44:02.414 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:02.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.421+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f99dc0d5530 con 0x7f99dc15c470 2024-05-30T16:44:02.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.421+0000 7f99d2000640 1 -- 172.21.15.67:0/3131897450 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f99c8079b60 con 0x7f99dc15c470 2024-05-30T16:44:02.422 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2024-05-30T16:44:02.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f99bc06c8f0 msgr2=0x7f99bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f99bc06c8f0 0x7f99bc06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f99d4007fd0 tx=0x7f99d4003de0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99dc15c470 msgr2=0x7f99dc15a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99dc15c470 0x7f99dc15a710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f99c8030000 tx=0x7f99dc06ca70 comp rx=0 tx=0).stop 2024-05-30T16:44:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 shutdown_connections 2024-05-30T16:44:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f99bc06c8f0 0x7f99bc06edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99dc0cab40 0x7f99dc0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99dc15c470 0x7f99dc15a710 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f99c8030000 tx=0x7f99dc06ca70 comp rx=0 tx=0).stop 2024-05-30T16:44:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 --2- 172.21.15.67:0/3131897450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c0004800 0x7f99dc14df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.425+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 >> 172.21.15.67:0/3131897450 conn(0x7f99dc08bfc0 msgr2=0x7f99dc0d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.426+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 shutdown_connections 2024-05-30T16:44:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.426+0000 7f99e0c00640 1 -- 172.21.15.67:0/3131897450 wait complete. 2024-05-30T16:44:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.429+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f70400689e0 con 0x7f70400b9a00 2024-05-30T16:44:02.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.429+0000 7f7036a00640 1 -- 172.21.15.67:0/2992514462 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7038059910 con 0x7f70400b9a00 2024-05-30T16:44:02.430 INFO:teuthology.orchestra.run.smithi067.stdout:64424509447 2024-05-30T16:44:02.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.432+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f702006caf0 msgr2=0x7f702006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.432+0000 7f7045800640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f702006caf0 0x7f702006efb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f703002a6f0 tx=0x7f703002e000 comp rx=0 tx=0).stop 2024-05-30T16:44:02.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.432+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70400b9a00 msgr2=0x7f7040159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.432+0000 7f7045800640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70400b9a00 0x7f7040159e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f703800ca70 tx=0x7f703800cf30 comp rx=0 tx=0).stop 2024-05-30T16:44:02.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.433+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f82e00b1740 con 0x7f82e00b3b70 2024-05-30T16:44:02.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.433+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 shutdown_connections 2024-05-30T16:44:02.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.433+0000 7f7045800640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f702006caf0 0x7f702006efb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.433+0000 7f7045800640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70400b9a00 0x7f7040159e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.433+0000 7f7045800640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f704015ed20 0x7f70402020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.433+0000 7f7045800640 1 --2- 172.21.15.67:0/2992514462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70400d4d60 0x7f70400b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.433+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 >> 172.21.15.67:0/2992514462 conn(0x7f704008bfc0 msgr2=0x7f704015ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.1 2024-05-30T16:44:02.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.434+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 shutdown_connections 2024-05-30T16:44:02.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.434+0000 7f7045800640 1 -- 172.21.15.67:0/2992514462 wait complete. 2024-05-30T16:44:02.436 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:02.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.434+0000 7f82d7400640 1 -- 172.21.15.67:0/462777848 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f82d005a060 con 0x7f82e00b3b70 2024-05-30T16:44:02.437 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-05-30T16:44:02.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2024-05-30T16:44:02.437 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82c006c8f0 msgr2=0x7f82c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82c006c8f0 0x7f82c006edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82e00cdf80 tx=0x7f82cc000ed0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82e00b3b70 msgr2=0x7f82e0157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82e00b3b70 0x7f82e0157f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82d000c700 tx=0x7f82d000cbc0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 shutdown_connections 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82c006c8f0 0x7f82c006edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e00d04a0 0x7f82e00c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e00b4b60 0x7f82e00c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 --2- 172.21.15.67:0/462777848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82e00b3b70 0x7f82e0157f50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 >> 172.21.15.67:0/462777848 conn(0x7f82e006bae0 msgr2=0x7f82e00afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 shutdown_connections 2024-05-30T16:44:02.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.438+0000 7f82e6800640 1 -- 172.21.15.67:0/462777848 wait complete. 2024-05-30T16:44:02.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-05-30T16:44:02.448 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:02.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.6 2024-05-30T16:44:02.451 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:02.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.5 2024-05-30T16:44:02.457 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:02.457 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T16:44:02.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:44:02.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.697+0000 7f08c9400640 1 Processor -- start 2024-05-30T16:44:02.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.697+0000 7f08c9400640 1 -- start start 2024-05-30T16:44:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.698+0000 7f08c9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c406ee40 0x7f08c40ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.698+0000 7f08c9400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08c406f2c0 0x7f08c40ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.699+0000 7f08c9400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c4156f10 con 0x7f08c4156b30 2024-05-30T16:44:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.699+0000 7f08c9400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f08c406dda0 con 0x7f08c406f2c0 2024-05-30T16:44:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.699+0000 7f08c9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08c406e080 con 0x7f08c406ee40 2024-05-30T16:44:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.699+0000 7f08bb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c406ee40 0x7f08c40ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.699+0000 7f08c8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08c406f2c0 0x7f08c40ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.699+0000 7f08bb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c406ee40 0x7f08c40ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:50546/0 (socket says 172.21.15.67:50546) 2024-05-30T16:44:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.699+0000 7f08bb400640 1 -- 172.21.15.67:0/2307495690 learned_addr learned my addr 172.21.15.67:0/2307495690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:02.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.700+0000 7f08bb400640 1 -- 172.21.15.67:0/2307495690 >> v2:172.21.15.169:3300/0 conn(0x7f08c406f2c0 msgr2=0x7f08c40ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:02.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.700+0000 7f08bb400640 1 --2- 172.21.15.67:0/2307495690 >> v2:172.21.15.169:3300/0 conn(0x7f08c406f2c0 0x7f08c40ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.700+0000 7f08bb400640 1 -- 172.21.15.67:0/2307495690 >> v1:172.21.15.169:6789/0 conn(0x7f08c4156b30 legacy=0x7f08c40af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.700+0000 7f08bb400640 1 -- 172.21.15.67:0/2307495690 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c406e360 con 0x7f08c406ee40 2024-05-30T16:44:02.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.701+0000 7f08bb400640 1 --2- 172.21.15.67:0/2307495690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c406ee40 0x7f08c40ad230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009870 tx=0x7f08ac02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e1626d068454077 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:02.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.701+0000 7f08c3400640 1 -- 172.21.15.67:0/2307495690 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac037070 con 0x7f08c406ee40 2024-05-30T16:44:02.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c3400640 1 -- 172.21.15.67:0/2307495690 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ac032440 con 0x7f08c406ee40 2024-05-30T16:44:02.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c3400640 1 -- 172.21.15.67:0/2307495690 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac032810 con 0x7f08c406ee40 2024-05-30T16:44:02.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c9400640 1 -- 172.21.15.67:0/2307495690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c406ee40 msgr2=0x7f08c40ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c9400640 1 --2- 172.21.15.67:0/2307495690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c406ee40 0x7f08c40ad230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009870 tx=0x7f08ac02a1c0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c9400640 1 -- 172.21.15.67:0/2307495690 shutdown_connections 2024-05-30T16:44:02.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c9400640 1 --2- 172.21.15.67:0/2307495690 >> v2:172.21.15.169:3300/0 conn(0x7f08c406f2c0 0x7f08c40ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c9400640 1 --2- 172.21.15.67:0/2307495690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c406ee40 0x7f08c40ad230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c9400640 1 -- 172.21.15.67:0/2307495690 >> 172.21.15.67:0/2307495690 conn(0x7f08c40b1fc0 msgr2=0x7f08c40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c9400640 1 -- 172.21.15.67:0/2307495690 shutdown_connections 2024-05-30T16:44:02.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.702+0000 7f08c9400640 1 -- 172.21.15.67:0/2307495690 wait complete. 2024-05-30T16:44:02.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.704+0000 7f08c9400640 1 Processor -- start 2024-05-30T16:44:02.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.704+0000 7f08c9400640 1 -- start start 2024-05-30T16:44:02.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.705+0000 7f08c9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08c406ee40 0x7f08c415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.705+0000 7f08c9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c406f2c0 0x7f08c4159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.705+0000 7f08c9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c4156b30 0x7f08c4157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.705+0000 7f08c9400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f08c40b2db0 con 0x7f08c406ee40 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.705+0000 7f08c9400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08c40b2c30 con 0x7f08c406f2c0 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.705+0000 7f08c9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08c40b32c0 con 0x7f08c4156b30 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.705+0000 7f08c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08c406ee40 0x7f08c415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.705+0000 7f08c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08c406ee40 0x7f08c415b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53142/0 (socket says 172.21.15.67:53142) 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.705+0000 7f08c3e00640 1 -- 172.21.15.67:0/84362930 learned_addr learned my addr 172.21.15.67:0/84362930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c8a00640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c4156b30 0x7f08c4157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08bb400640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c406f2c0 0x7f08c4159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c3e00640 1 -- 172.21.15.67:0/84362930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c4156b30 msgr2=0x7f08c4157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c3e00640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c4156b30 0x7f08c4157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c3e00640 1 -- 172.21.15.67:0/84362930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c406f2c0 msgr2=0x7f08c4159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c3e00640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c406f2c0 0x7f08c4159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c3e00640 1 -- 172.21.15.67:0/84362930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c41f2c50 con 0x7f08c406ee40 2024-05-30T16:44:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c3e00640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08c406ee40 0x7f08c415b7c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c400 tx=0x7f08b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c0c00640 1 -- 172.21.15.67:0/84362930 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4017070 con 0x7f08c406ee40 2024-05-30T16:44:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c0c00640 1 -- 172.21.15.67:0/84362930 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b40052d0 con 0x7f08c406ee40 2024-05-30T16:44:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c41f3290 con 0x7f08c406ee40 2024-05-30T16:44:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.706+0000 7f08c0c00640 1 -- 172.21.15.67:0/84362930 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4003830 con 0x7f08c406ee40 2024-05-30T16:44:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.707+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c41f3850 con 0x7f08c406ee40 2024-05-30T16:44:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.707+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c40689e0 con 0x7f08c406ee40 2024-05-30T16:44:02.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.708+0000 7f08c0c00640 1 -- 172.21.15.67:0/84362930 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08b401a070 con 0x7f08c406ee40 2024-05-30T16:44:02.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.709+0000 7f08c0c00640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08a006c8f0 0x7f08a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:02.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.709+0000 7f08c0c00640 1 -- 172.21.15.67:0/84362930 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f08b408aec0 con 0x7f08c406ee40 2024-05-30T16:44:02.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.710+0000 7f08bb400640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08a006c8f0 0x7f08a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:02.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.710+0000 7f08bb400640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08a006c8f0 0x7f08a006edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08ac034000 tx=0x7f08ac02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:02.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.714+0000 7f08c0c00640 1 -- 172.21.15.67:0/84362930 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08b40592e0 con 0x7f08c406ee40 2024-05-30T16:44:02.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.905+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08c406f7f0 con 0x7f08a006c8f0 2024-05-30T16:44:02.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.923+0000 7f08c0c00640 1 -- 172.21.15.67:0/84362930 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f08c406f7f0 con 0x7f08a006c8f0 2024-05-30T16:44:02.926 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:44:02.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08a006c8f0 msgr2=0x7f08a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08a006c8f0 0x7f08a006edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08ac034000 tx=0x7f08ac02e000 comp rx=0 tx=0).stop 2024-05-30T16:44:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08c406ee40 msgr2=0x7f08c415b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08c406ee40 0x7f08c415b7c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c400 tx=0x7f08b400c8c0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 shutdown_connections 2024-05-30T16:44:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08a006c8f0 0x7f08a006edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08c4156b30 0x7f08c4157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c406f2c0 0x7f08c4159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 --2- 172.21.15.67:0/84362930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08c406ee40 0x7f08c415b7c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 >> 172.21.15.67:0/84362930 conn(0x7f08c40b1fc0 msgr2=0x7f08c4155bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 shutdown_connections 2024-05-30T16:44:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:02.930+0000 7f08c9400640 1 -- 172.21.15.67:0/84362930 wait complete. 2024-05-30T16:44:02.950 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-05-30T16:44:01.981954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219120,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755600,"statfs":{"total":773094113280,"available":772869734400,"internally_reserved":0,"allocated":3022848,"data_stored":1803252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.141352"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227428+0000","last_change":"2024-05-30T16:43:52.894258+0000","last_active":"2024-05-30T16:43:54.227428+0000","last_peered":"2024-05-30T16:43:54.227428+0000","last_clean":"2024-05-30T16:43:54.227428+0000","last_became_active":"2024-05-30T16:43:52.893919+0000","last_became_peered":"2024-05-30T16:43:52.893919+0000","last_unstale":"2024-05-30T16:43:54.227428+0000","last_undegraded":"2024-05-30T16:43:54.227428+0000","last_fullsized":"2024-05-30T16:43:54.227428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:32.373668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872293+0000","last_change":"2024-05-30T16:43:52.878535+0000","last_active":"2024-05-30T16:43:53.872293+0000","last_peered":"2024-05-30T16:43:53.872293+0000","last_clean":"2024-05-30T16:43:53.872293+0000","last_became_active":"2024-05-30T16:43:52.878243+0000","last_became_peered":"2024-05-30T16:43:52.878243+0000","last_unstale":"2024-05-30T16:43:53.872293+0000","last_undegraded":"2024-05-30T16:43:53.872293+0000","last_fullsized":"2024-05-30T16:43:53.872293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157387+0000","last_change":"2024-05-30T16:43:52.874827+0000","last_active":"2024-05-30T16:43:54.157387+0000","last_peered":"2024-05-30T16:43:54.157387+0000","last_clean":"2024-05-30T16:43:54.157387+0000","last_became_active":"2024-05-30T16:43:52.874483+0000","last_became_peered":"2024-05-30T16:43:52.874483+0000","last_unstale":"2024-05-30T16:43:54.157387+0000","last_undegraded":"2024-05-30T16:43:54.157387+0000","last_fullsized":"2024-05-30T16:43:54.157387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:27.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875633+0000","last_change":"2024-05-30T16:43:52.884775+0000","last_active":"2024-05-30T16:43:53.875633+0000","last_peered":"2024-05-30T16:43:53.875633+0000","last_clean":"2024-05-30T16:43:53.875633+0000","last_became_active":"2024-05-30T16:43:52.884436+0000","last_became_peered":"2024-05-30T16:43:52.884436+0000","last_unstale":"2024-05-30T16:43:53.875633+0000","last_undegraded":"2024-05-30T16:43:53.875633+0000","last_fullsized":"2024-05-30T16:43:53.875633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:25.275803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226885+0000","last_change":"2024-05-30T16:43:52.886555+0000","last_active":"2024-05-30T16:43:54.226885+0000","last_peered":"2024-05-30T16:43:54.226885+0000","last_clean":"2024-05-30T16:43:54.226885+0000","last_became_active":"2024-05-30T16:43:52.886203+0000","last_became_peered":"2024-05-30T16:43:52.886203+0000","last_unstale":"2024-05-30T16:43:54.226885+0000","last_undegraded":"2024-05-30T16:43:54.226885+0000","last_fullsized":"2024-05-30T16:43:54.226885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:03.535096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873858+0000","last_change":"2024-05-30T16:43:52.875329+0000","last_active":"2024-05-30T16:43:53.873858+0000","last_peered":"2024-05-30T16:43:53.873858+0000","last_clean":"2024-05-30T16:43:53.873858+0000","last_became_active":"2024-05-30T16:43:52.874980+0000","last_became_peered":"2024-05-30T16:43:52.874980+0000","last_unstale":"2024-05-30T16:43:53.873858+0000","last_undegraded":"2024-05-30T16:43:53.873858+0000","last_fullsized":"2024-05-30T16:43:53.873858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:48.436948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225680+0000","last_change":"2024-05-30T16:43:52.894028+0000","last_active":"2024-05-30T16:43:54.225680+0000","last_peered":"2024-05-30T16:43:54.225680+0000","last_clean":"2024-05-30T16:43:54.225680+0000","last_became_active":"2024-05-30T16:43:52.889967+0000","last_became_peered":"2024-05-30T16:43:52.889967+0000","last_unstale":"2024-05-30T16:43:54.225680+0000","last_undegraded":"2024-05-30T16:43:54.225680+0000","last_fullsized":"2024-05-30T16:43:54.225680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:21.139158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.223951+0000","last_change":"2024-05-30T16:43:52.899916+0000","last_active":"2024-05-30T16:43:54.223951+0000","last_peered":"2024-05-30T16:43:54.223951+0000","last_clean":"2024-05-30T16:43:54.223951+0000","last_became_active":"2024-05-30T16:43:52.899619+0000","last_became_peered":"2024-05-30T16:43:52.899619+0000","last_unstale":"2024-05-30T16:43:54.223951+0000","last_undegraded":"2024-05-30T16:43:54.223951+0000","last_fullsized":"2024-05-30T16:43:54.223951+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:30.437464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227567+0000","last_change":"2024-05-30T16:43:52.903027+0000","last_active":"2024-05-30T16:43:54.227567+0000","last_peered":"2024-05-30T16:43:54.227567+0000","last_clean":"2024-05-30T16:43:54.227567+0000","last_became_active":"2024-05-30T16:43:52.902799+0000","last_became_peered":"2024-05-30T16:43:52.902799+0000","last_unstale":"2024-05-30T16:43:54.227567+0000","last_undegraded":"2024-05-30T16:43:54.227567+0000","last_fullsized":"2024-05-30T16:43:54.227567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:35.070892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873608+0000","last_change":"2024-05-30T16:43:52.895565+0000","last_active":"2024-05-30T16:43:53.873608+0000","last_peered":"2024-05-30T16:43:53.873608+0000","last_clean":"2024-05-30T16:43:53.873608+0000","last_became_active":"2024-05-30T16:43:52.894829+0000","last_became_peered":"2024-05-30T16:43:52.894829+0000","last_unstale":"2024-05-30T16:43:53.873608+0000","last_undegraded":"2024-05-30T16:43:53.873608+0000","last_fullsized":"2024-05-30T16:43:53.873608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873611+0000","last_change":"2024-05-30T16:43:52.899464+0000","last_active":"2024-05-30T16:43:53.873611+0000","last_peered":"2024-05-30T16:43:53.873611+0000","last_clean":"2024-05-30T16:43:53.873611+0000","last_became_active":"2024-05-30T16:43:52.897925+0000","last_became_peered":"2024-05-30T16:43:52.897925+0000","last_unstale":"2024-05-30T16:43:53.873611+0000","last_undegraded":"2024-05-30T16:43:53.873611+0000","last_fullsized":"2024-05-30T16:43:53.873611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226199+0000","last_change":"2024-05-30T16:43:52.877782+0000","last_active":"2024-05-30T16:43:54.226199+0000","last_peered":"2024-05-30T16:43:54.226199+0000","last_clean":"2024-05-30T16:43:54.226199+0000","last_became_active":"2024-05-30T16:43:52.877466+0000","last_became_peered":"2024-05-30T16:43:52.877466+0000","last_unstale":"2024-05-30T16:43:54.226199+0000","last_undegraded":"2024-05-30T16:43:54.226199+0000","last_fullsized":"2024-05-30T16:43:54.226199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:33.066313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225046+0000","last_change":"2024-05-30T16:43:52.881578+0000","last_active":"2024-05-30T16:43:54.225046+0000","last_peered":"2024-05-30T16:43:54.225046+0000","last_clean":"2024-05-30T16:43:54.225046+0000","last_became_active":"2024-05-30T16:43:52.877597+0000","last_became_peered":"2024-05-30T16:43:52.877597+0000","last_unstale":"2024-05-30T16:43:54.225046+0000","last_undegraded":"2024-05-30T16:43:54.225046+0000","last_fullsized":"2024-05-30T16:43:54.225046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.156845+0000","last_change":"2024-05-30T16:43:52.874486+0000","last_active":"2024-05-30T16:43:54.156845+0000","last_peered":"2024-05-30T16:43:54.156845+0000","last_clean":"2024-05-30T16:43:54.156845+0000","last_became_active":"2024-05-30T16:43:52.872863+0000","last_became_peered":"2024-05-30T16:43:52.872863+0000","last_unstale":"2024-05-30T16:43:54.156845+0000","last_undegraded":"2024-05-30T16:43:54.156845+0000","last_fullsized":"2024-05-30T16:43:54.156845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:20:51.386105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872070+0000","last_change":"2024-05-30T16:43:52.877439+0000","last_active":"2024-05-30T16:43:53.872070+0000","last_peered":"2024-05-30T16:43:53.872070+0000","last_clean":"2024-05-30T16:43:53.872070+0000","last_became_active":"2024-05-30T16:43:52.877139+0000","last_became_peered":"2024-05-30T16:43:52.877139+0000","last_unstale":"2024-05-30T16:43:53.872070+0000","last_undegraded":"2024-05-30T16:43:53.872070+0000","last_fullsized":"2024-05-30T16:43:53.872070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873037+0000","last_change":"2024-05-30T16:43:52.872351+0000","last_active":"2024-05-30T16:43:53.873037+0000","last_peered":"2024-05-30T16:43:53.873037+0000","last_clean":"2024-05-30T16:43:53.873037+0000","last_became_active":"2024-05-30T16:43:52.872079+0000","last_became_peered":"2024-05-30T16:43:52.872079+0000","last_unstale":"2024-05-30T16:43:53.873037+0000","last_undegraded":"2024-05-30T16:43:53.873037+0000","last_fullsized":"2024-05-30T16:43:53.873037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:50.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874480+0000","last_change":"2024-05-30T16:43:52.897208+0000","last_active":"2024-05-30T16:43:53.874480+0000","last_peered":"2024-05-30T16:43:53.874480+0000","last_clean":"2024-05-30T16:43:53.874480+0000","last_became_active":"2024-05-30T16:43:52.895700+0000","last_became_peered":"2024-05-30T16:43:52.895700+0000","last_unstale":"2024-05-30T16:43:53.874480+0000","last_undegraded":"2024-05-30T16:43:53.874480+0000","last_fullsized":"2024-05-30T16:43:53.874480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:21.139158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086708+0000","last_change":"2024-05-30T16:43:52.870196+0000","last_active":"2024-05-30T16:43:54.086708+0000","last_peered":"2024-05-30T16:43:54.086708+0000","last_clean":"2024-05-30T16:43:54.086708+0000","last_became_active":"2024-05-30T16:43:52.869879+0000","last_became_peered":"2024-05-30T16:43:52.869879+0000","last_unstale":"2024-05-30T16:43:54.086708+0000","last_undegraded":"2024-05-30T16:43:54.086708+0000","last_fullsized":"2024-05-30T16:43:54.086708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:43.785583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.228432+0000","last_change":"2024-05-30T16:43:52.898388+0000","last_active":"2024-05-30T16:43:54.228432+0000","last_peered":"2024-05-30T16:43:54.228432+0000","last_clean":"2024-05-30T16:43:54.228432+0000","last_became_active":"2024-05-30T16:43:52.898159+0000","last_became_peered":"2024-05-30T16:43:52.898159+0000","last_unstale":"2024-05-30T16:43:54.228432+0000","last_undegraded":"2024-05-30T16:43:54.228432+0000","last_fullsized":"2024-05-30T16:43:54.228432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:41.187051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874369+0000","last_change":"2024-05-30T16:43:52.880715+0000","last_active":"2024-05-30T16:43:53.874369+0000","last_peered":"2024-05-30T16:43:53.874369+0000","last_clean":"2024-05-30T16:43:53.874369+0000","last_became_active":"2024-05-30T16:43:52.880406+0000","last_became_peered":"2024-05-30T16:43:52.880406+0000","last_unstale":"2024-05-30T16:43:53.874369+0000","last_undegraded":"2024-05-30T16:43:53.874369+0000","last_fullsized":"2024-05-30T16:43:53.874369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157707+0000","last_change":"2024-05-30T16:43:52.895619+0000","last_active":"2024-05-30T16:43:54.157707+0000","last_peered":"2024-05-30T16:43:54.157707+0000","last_clean":"2024-05-30T16:43:54.157707+0000","last_became_active":"2024-05-30T16:43:52.891632+0000","last_became_peered":"2024-05-30T16:43:52.891632+0000","last_unstale":"2024-05-30T16:43:54.157707+0000","last_undegraded":"2024-05-30T16:43:54.157707+0000","last_fullsized":"2024-05-30T16:43:54.157707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:07.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873953+0000","last_change":"2024-05-30T16:43:52.872959+0000","last_active":"2024-05-30T16:43:53.873953+0000","last_peered":"2024-05-30T16:43:53.873953+0000","last_clean":"2024-05-30T16:43:53.873953+0000","last_became_active":"2024-05-30T16:43:52.872679+0000","last_became_peered":"2024-05-30T16:43:52.872679+0000","last_unstale":"2024-05-30T16:43:53.873953+0000","last_undegraded":"2024-05-30T16:43:53.873953+0000","last_fullsized":"2024-05-30T16:43:53.873953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:08:57.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225306+0000","last_change":"2024-05-30T16:43:52.885877+0000","last_active":"2024-05-30T16:43:54.225306+0000","last_peered":"2024-05-30T16:43:54.225306+0000","last_clean":"2024-05-30T16:43:54.225306+0000","last_became_active":"2024-05-30T16:43:52.885545+0000","last_became_peered":"2024-05-30T16:43:52.885545+0000","last_unstale":"2024-05-30T16:43:54.225306+0000","last_undegraded":"2024-05-30T16:43:54.225306+0000","last_fullsized":"2024-05-30T16:43:54.225306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:00.933568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873194+0000","last_change":"2024-05-30T16:43:52.895695+0000","last_active":"2024-05-30T16:43:53.873194+0000","last_peered":"2024-05-30T16:43:53.873194+0000","last_clean":"2024-05-30T16:43:53.873194+0000","last_became_active":"2024-05-30T16:43:52.895370+0000","last_became_peered":"2024-05-30T16:43:52.895370+0000","last_unstale":"2024-05-30T16:43:53.873194+0000","last_undegraded":"2024-05-30T16:43:53.873194+0000","last_fullsized":"2024-05-30T16:43:53.873194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:31.485813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224641+0000","last_change":"2024-05-30T16:43:52.883128+0000","last_active":"2024-05-30T16:43:54.224641+0000","last_peered":"2024-05-30T16:43:54.224641+0000","last_clean":"2024-05-30T16:43:54.224641+0000","last_became_active":"2024-05-30T16:43:52.881729+0000","last_became_peered":"2024-05-30T16:43:52.881729+0000","last_unstale":"2024-05-30T16:43:54.224641+0000","last_undegraded":"2024-05-30T16:43:54.224641+0000","last_fullsized":"2024-05-30T16:43:54.224641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873862+0000","last_change":"2024-05-30T16:43:52.876820+0000","last_active":"2024-05-30T16:43:53.873862+0000","last_peered":"2024-05-30T16:43:53.873862+0000","last_clean":"2024-05-30T16:43:53.873862+0000","last_became_active":"2024-05-30T16:43:52.875407+0000","last_became_peered":"2024-05-30T16:43:52.875407+0000","last_unstale":"2024-05-30T16:43:53.873862+0000","last_undegraded":"2024-05-30T16:43:53.873862+0000","last_fullsized":"2024-05-30T16:43:53.873862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871846+0000","last_change":"2024-05-30T16:43:52.883372+0000","last_active":"2024-05-30T16:43:53.871846+0000","last_peered":"2024-05-30T16:43:53.871846+0000","last_clean":"2024-05-30T16:43:53.871846+0000","last_became_active":"2024-05-30T16:43:52.882611+0000","last_became_peered":"2024-05-30T16:43:52.882611+0000","last_unstale":"2024-05-30T16:43:53.871846+0000","last_undegraded":"2024-05-30T16:43:53.871846+0000","last_fullsized":"2024-05-30T16:43:53.871846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224988+0000","last_change":"2024-05-30T16:43:52.882394+0000","last_active":"2024-05-30T16:43:54.224988+0000","last_peered":"2024-05-30T16:43:54.224988+0000","last_clean":"2024-05-30T16:43:54.224988+0000","last_became_active":"2024-05-30T16:43:52.882126+0000","last_became_peered":"2024-05-30T16:43:52.882126+0000","last_unstale":"2024-05-30T16:43:54.224988+0000","last_undegraded":"2024-05-30T16:43:54.224988+0000","last_fullsized":"2024-05-30T16:43:54.224988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157761+0000","last_change":"2024-05-30T16:43:52.884885+0000","last_active":"2024-05-30T16:43:54.157761+0000","last_peered":"2024-05-30T16:43:54.157761+0000","last_clean":"2024-05-30T16:43:54.157761+0000","last_became_active":"2024-05-30T16:43:52.884414+0000","last_became_peered":"2024-05-30T16:43:52.884414+0000","last_unstale":"2024-05-30T16:43:54.157761+0000","last_undegraded":"2024-05-30T16:43:54.157761+0000","last_fullsized":"2024-05-30T16:43:54.157761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:50.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874039+0000","last_change":"2024-05-30T16:43:52.890624+0000","last_active":"2024-05-30T16:43:53.874039+0000","last_peered":"2024-05-30T16:43:53.874039+0000","last_clean":"2024-05-30T16:43:53.874039+0000","last_became_active":"2024-05-30T16:43:52.890269+0000","last_became_peered":"2024-05-30T16:43:52.890269+0000","last_unstale":"2024-05-30T16:43:53.874039+0000","last_undegraded":"2024-05-30T16:43:53.874039+0000","last_fullsized":"2024-05-30T16:43:53.874039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:48.878065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086162+0000","last_change":"2024-05-30T16:43:52.891560+0000","last_active":"2024-05-30T16:43:54.086162+0000","last_peered":"2024-05-30T16:43:54.086162+0000","last_clean":"2024-05-30T16:43:54.086162+0000","last_became_active":"2024-05-30T16:43:52.891194+0000","last_became_peered":"2024-05-30T16:43:52.891194+0000","last_unstale":"2024-05-30T16:43:54.086162+0000","last_undegraded":"2024-05-30T16:43:54.086162+0000","last_fullsized":"2024-05-30T16:43:54.086162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:33.066313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873436+0000","last_change":"2024-05-30T16:43:52.876082+0000","last_active":"2024-05-30T16:43:53.873436+0000","last_peered":"2024-05-30T16:43:53.873436+0000","last_clean":"2024-05-30T16:43:53.873436+0000","last_became_active":"2024-05-30T16:43:52.875792+0000","last_became_peered":"2024-05-30T16:43:52.875792+0000","last_unstale":"2024-05-30T16:43:53.873436+0000","last_undegraded":"2024-05-30T16:43:53.873436+0000","last_fullsized":"2024-05-30T16:43:53.873436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225172+0000","last_change":"2024-05-30T16:43:52.885848+0000","last_active":"2024-05-30T16:43:54.225172+0000","last_peered":"2024-05-30T16:43:54.225172+0000","last_clean":"2024-05-30T16:43:54.225172+0000","last_became_active":"2024-05-30T16:43:52.885532+0000","last_became_peered":"2024-05-30T16:43:52.885532+0000","last_unstale":"2024-05-30T16:43:54.225172+0000","last_undegraded":"2024-05-30T16:43:54.225172+0000","last_fullsized":"2024-05-30T16:43:54.225172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:04.698389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225258+0000","last_change":"2024-05-30T16:43:51.854344+0000","last_active":"2024-05-30T16:43:54.225258+0000","last_peered":"2024-05-30T16:43:54.225258+0000","last_clean":"2024-05-30T16:43:54.225258+0000","last_became_active":"2024-05-30T16:43:51.854081+0000","last_became_peered":"2024-05-30T16:43:51.854081+0000","last_unstale":"2024-05-30T16:43:54.225258+0000","last_undegraded":"2024-05-30T16:43:54.225258+0000","last_fullsized":"2024-05-30T16:43:54.225258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:50.620803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.158196+0000","last_change":"2024-05-30T16:43:52.881003+0000","last_active":"2024-05-30T16:43:54.158196+0000","last_peered":"2024-05-30T16:43:54.158196+0000","last_clean":"2024-05-30T16:43:54.158196+0000","last_became_active":"2024-05-30T16:43:52.880834+0000","last_became_peered":"2024-05-30T16:43:52.880834+0000","last_unstale":"2024-05-30T16:43:54.158196+0000","last_undegraded":"2024-05-30T16:43:54.158196+0000","last_fullsized":"2024-05-30T16:43:54.158196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:31.485813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873330+0000","last_change":"2024-05-30T16:43:51.863107+0000","last_active":"2024-05-30T16:43:53.873330+0000","last_peered":"2024-05-30T16:43:53.873330+0000","last_clean":"2024-05-30T16:43:53.873330+0000","last_became_active":"2024-05-30T16:43:51.855082+0000","last_became_peered":"2024-05-30T16:43:51.855082+0000","last_unstale":"2024-05-30T16:43:53.873330+0000","last_undegraded":"2024-05-30T16:43:53.873330+0000","last_fullsized":"2024-05-30T16:43:53.873330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224700+0000","last_change":"2024-05-30T16:43:51.856427+0000","last_active":"2024-05-30T16:43:54.224700+0000","last_peered":"2024-05-30T16:43:54.224700+0000","last_clean":"2024-05-30T16:43:54.224700+0000","last_became_active":"2024-05-30T16:43:51.856150+0000","last_became_peered":"2024-05-30T16:43:51.856150+0000","last_unstale":"2024-05-30T16:43:54.224700+0000","last_undegraded":"2024-05-30T16:43:54.224700+0000","last_fullsized":"2024-05-30T16:43:54.224700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:47.064682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224116+0000","last_change":"2024-05-30T16:43:52.875668+0000","last_active":"2024-05-30T16:43:54.224116+0000","last_peered":"2024-05-30T16:43:54.224116+0000","last_clean":"2024-05-30T16:43:54.224116+0000","last_became_active":"2024-05-30T16:43:52.875362+0000","last_became_peered":"2024-05-30T16:43:52.875362+0000","last_unstale":"2024-05-30T16:43:54.224116+0000","last_undegraded":"2024-05-30T16:43:54.224116+0000","last_fullsized":"2024-05-30T16:43:54.224116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873259+0000","last_change":"2024-05-30T16:43:51.854040+0000","last_active":"2024-05-30T16:43:53.873259+0000","last_peered":"2024-05-30T16:43:53.873259+0000","last_clean":"2024-05-30T16:43:53.873259+0000","last_became_active":"2024-05-30T16:43:51.852832+0000","last_became_peered":"2024-05-30T16:43:51.852832+0000","last_unstale":"2024-05-30T16:43:53.873259+0000","last_undegraded":"2024-05-30T16:43:53.873259+0000","last_fullsized":"2024-05-30T16:43:53.873259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:24.379706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871843+0000","last_change":"2024-05-30T16:43:52.880882+0000","last_active":"2024-05-30T16:43:53.871843+0000","last_peered":"2024-05-30T16:43:53.871843+0000","last_clean":"2024-05-30T16:43:53.871843+0000","last_became_active":"2024-05-30T16:43:52.880558+0000","last_became_peered":"2024-05-30T16:43:52.880558+0000","last_unstale":"2024-05-30T16:43:53.871843+0000","last_undegraded":"2024-05-30T16:43:53.871843+0000","last_fullsized":"2024-05-30T16:43:53.871843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872979+0000","last_change":"2024-05-30T16:43:51.864315+0000","last_active":"2024-05-30T16:43:53.872979+0000","last_peered":"2024-05-30T16:43:53.872979+0000","last_clean":"2024-05-30T16:43:53.872979+0000","last_became_active":"2024-05-30T16:43:51.855552+0000","last_became_peered":"2024-05-30T16:43:51.855552+0000","last_unstale":"2024-05-30T16:43:53.872979+0000","last_undegraded":"2024-05-30T16:43:53.872979+0000","last_fullsized":"2024-05-30T16:43:53.872979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:03.683389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875985+0000","last_change":"2024-05-30T16:43:52.876866+0000","last_active":"2024-05-30T16:43:53.875985+0000","last_peered":"2024-05-30T16:43:53.875985+0000","last_clean":"2024-05-30T16:43:53.875985+0000","last_became_active":"2024-05-30T16:43:52.876208+0000","last_became_peered":"2024-05-30T16:43:52.876208+0000","last_unstale":"2024-05-30T16:43:53.875985+0000","last_undegraded":"2024-05-30T16:43:53.875985+0000","last_fullsized":"2024-05-30T16:43:53.875985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:35.070892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.876134+0000","last_change":"2024-05-30T16:43:51.857532+0000","last_active":"2024-05-30T16:43:53.876134+0000","last_peered":"2024-05-30T16:43:53.876134+0000","last_clean":"2024-05-30T16:43:53.876134+0000","last_became_active":"2024-05-30T16:43:51.853042+0000","last_became_peered":"2024-05-30T16:43:51.853042+0000","last_unstale":"2024-05-30T16:43:53.876134+0000","last_undegraded":"2024-05-30T16:43:53.876134+0000","last_fullsized":"2024-05-30T16:43:53.876134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:19.738945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157472+0000","last_change":"2024-05-30T16:43:52.901145+0000","last_active":"2024-05-30T16:43:54.157472+0000","last_peered":"2024-05-30T16:43:54.157472+0000","last_clean":"2024-05-30T16:43:54.157472+0000","last_became_active":"2024-05-30T16:43:52.900449+0000","last_became_peered":"2024-05-30T16:43:52.900449+0000","last_unstale":"2024-05-30T16:43:54.157472+0000","last_undegraded":"2024-05-30T16:43:54.157472+0000","last_fullsized":"2024-05-30T16:43:54.157472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:33:20.915701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871954+0000","last_change":"2024-05-30T16:43:51.857074+0000","last_active":"2024-05-30T16:43:53.871954+0000","last_peered":"2024-05-30T16:43:53.871954+0000","last_clean":"2024-05-30T16:43:53.871954+0000","last_became_active":"2024-05-30T16:43:51.856790+0000","last_became_peered":"2024-05-30T16:43:51.856790+0000","last_unstale":"2024-05-30T16:43:53.871954+0000","last_undegraded":"2024-05-30T16:43:53.871954+0000","last_fullsized":"2024-05-30T16:43:53.871954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226411+0000","last_change":"2024-05-30T16:43:52.883913+0000","last_active":"2024-05-30T16:43:54.226411+0000","last_peered":"2024-05-30T16:43:54.226411+0000","last_clean":"2024-05-30T16:43:54.226411+0000","last_became_active":"2024-05-30T16:43:52.883608+0000","last_became_peered":"2024-05-30T16:43:52.883608+0000","last_unstale":"2024-05-30T16:43:54.226411+0000","last_undegraded":"2024-05-30T16:43:54.226411+0000","last_fullsized":"2024-05-30T16:43:54.226411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:43.785583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874624+0000","last_change":"2024-05-30T16:43:51.859307+0000","last_active":"2024-05-30T16:43:53.874624+0000","last_peered":"2024-05-30T16:43:53.874624+0000","last_clean":"2024-05-30T16:43:53.874624+0000","last_became_active":"2024-05-30T16:43:51.858858+0000","last_became_peered":"2024-05-30T16:43:51.858858+0000","last_unstale":"2024-05-30T16:43:53.874624+0000","last_undegraded":"2024-05-30T16:43:53.874624+0000","last_fullsized":"2024-05-30T16:43:53.874624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227963+0000","last_change":"2024-05-30T16:43:52.892760+0000","last_active":"2024-05-30T16:43:54.227963+0000","last_peered":"2024-05-30T16:43:54.227963+0000","last_clean":"2024-05-30T16:43:54.227963+0000","last_became_active":"2024-05-30T16:43:52.892483+0000","last_became_peered":"2024-05-30T16:43:52.892483+0000","last_unstale":"2024-05-30T16:43:54.227963+0000","last_undegraded":"2024-05-30T16:43:54.227963+0000","last_fullsized":"2024-05-30T16:43:54.227963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:11.630186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.158046+0000","last_change":"2024-05-30T16:43:51.849511+0000","last_active":"2024-05-30T16:43:54.158046+0000","last_peered":"2024-05-30T16:43:54.158046+0000","last_clean":"2024-05-30T16:43:54.158046+0000","last_became_active":"2024-05-30T16:43:51.842797+0000","last_became_peered":"2024-05-30T16:43:51.842797+0000","last_unstale":"2024-05-30T16:43:54.158046+0000","last_undegraded":"2024-05-30T16:43:54.158046+0000","last_fullsized":"2024-05-30T16:43:54.158046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:30:25.098938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157148+0000","last_change":"2024-05-30T16:43:52.887806+0000","last_active":"2024-05-30T16:43:54.157148+0000","last_peered":"2024-05-30T16:43:54.157148+0000","last_clean":"2024-05-30T16:43:54.157148+0000","last_became_active":"2024-05-30T16:43:52.887486+0000","last_became_peered":"2024-05-30T16:43:52.887486+0000","last_unstale":"2024-05-30T16:43:54.157148+0000","last_undegraded":"2024-05-30T16:43:54.157148+0000","last_fullsized":"2024-05-30T16:43:54.157148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:05.404594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874098+0000","last_change":"2024-05-30T16:43:51.857906+0000","last_active":"2024-05-30T16:43:53.874098+0000","last_peered":"2024-05-30T16:43:53.874098+0000","last_clean":"2024-05-30T16:43:53.874098+0000","last_became_active":"2024-05-30T16:43:51.855395+0000","last_became_peered":"2024-05-30T16:43:51.855395+0000","last_unstale":"2024-05-30T16:43:53.874098+0000","last_undegraded":"2024-05-30T16:43:53.874098+0000","last_fullsized":"2024-05-30T16:43:53.874098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:32:37.147415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226899+0000","last_change":"2024-05-30T16:43:52.888949+0000","last_active":"2024-05-30T16:43:54.226899+0000","last_peered":"2024-05-30T16:43:54.226899+0000","last_clean":"2024-05-30T16:43:54.226899+0000","last_became_active":"2024-05-30T16:43:52.887708+0000","last_became_peered":"2024-05-30T16:43:52.887708+0000","last_unstale":"2024-05-30T16:43:54.226899+0000","last_undegraded":"2024-05-30T16:43:54.226899+0000","last_fullsized":"2024-05-30T16:43:54.226899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:05:00.764986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873316+0000","last_change":"2024-05-30T16:43:51.856817+0000","last_active":"2024-05-30T16:43:53.873316+0000","last_peered":"2024-05-30T16:43:53.873316+0000","last_clean":"2024-05-30T16:43:53.873316+0000","last_became_active":"2024-05-30T16:43:51.856519+0000","last_became_peered":"2024-05-30T16:43:51.856519+0000","last_unstale":"2024-05-30T16:43:53.873316+0000","last_undegraded":"2024-05-30T16:43:53.873316+0000","last_fullsized":"2024-05-30T16:43:53.873316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873045+0000","last_change":"2024-05-30T16:43:52.874643+0000","last_active":"2024-05-30T16:43:53.873045+0000","last_peered":"2024-05-30T16:43:53.873045+0000","last_clean":"2024-05-30T16:43:53.873045+0000","last_became_active":"2024-05-30T16:43:52.874280+0000","last_became_peered":"2024-05-30T16:43:52.874280+0000","last_unstale":"2024-05-30T16:43:53.873045+0000","last_undegraded":"2024-05-30T16:43:53.873045+0000","last_fullsized":"2024-05-30T16:43:53.873045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086111+0000","last_change":"2024-05-30T16:43:51.851393+0000","last_active":"2024-05-30T16:43:54.086111+0000","last_peered":"2024-05-30T16:43:54.086111+0000","last_clean":"2024-05-30T16:43:54.086111+0000","last_became_active":"2024-05-30T16:43:51.851057+0000","last_became_peered":"2024-05-30T16:43:51.851057+0000","last_unstale":"2024-05-30T16:43:54.086111+0000","last_undegraded":"2024-05-30T16:43:54.086111+0000","last_fullsized":"2024-05-30T16:43:54.086111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:50.620803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225990+0000","last_change":"2024-05-30T16:43:52.880958+0000","last_active":"2024-05-30T16:43:54.225990+0000","last_peered":"2024-05-30T16:43:54.225990+0000","last_clean":"2024-05-30T16:43:54.225990+0000","last_became_active":"2024-05-30T16:43:52.880536+0000","last_became_peered":"2024-05-30T16:43:52.880536+0000","last_unstale":"2024-05-30T16:43:54.225990+0000","last_undegraded":"2024-05-30T16:43:54.225990+0000","last_fullsized":"2024-05-30T16:43:54.225990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:13:13.668843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226477+0000","last_change":"2024-05-30T16:43:51.854467+0000","last_active":"2024-05-30T16:43:54.226477+0000","last_peered":"2024-05-30T16:43:54.226477+0000","last_clean":"2024-05-30T16:43:54.226477+0000","last_became_active":"2024-05-30T16:43:51.854104+0000","last_became_peered":"2024-05-30T16:43:51.854104+0000","last_unstale":"2024-05-30T16:43:54.226477+0000","last_undegraded":"2024-05-30T16:43:54.226477+0000","last_fullsized":"2024-05-30T16:43:54.226477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:26.393072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227506+0000","last_change":"2024-05-30T16:43:52.888958+0000","last_active":"2024-05-30T16:43:54.227506+0000","last_peered":"2024-05-30T16:43:54.227506+0000","last_clean":"2024-05-30T16:43:54.227506+0000","last_became_active":"2024-05-30T16:43:52.886708+0000","last_became_peered":"2024-05-30T16:43:52.886708+0000","last_unstale":"2024-05-30T16:43:54.227506+0000","last_undegraded":"2024-05-30T16:43:54.227506+0000","last_fullsized":"2024-05-30T16:43:54.227506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:25.275803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086747+0000","last_change":"2024-05-30T16:43:51.859324+0000","last_active":"2024-05-30T16:43:54.086747+0000","last_peered":"2024-05-30T16:43:54.086747+0000","last_clean":"2024-05-30T16:43:54.086747+0000","last_became_active":"2024-05-30T16:43:51.858960+0000","last_became_peered":"2024-05-30T16:43:51.858960+0000","last_unstale":"2024-05-30T16:43:54.086747+0000","last_undegraded":"2024-05-30T16:43:54.086747+0000","last_fullsized":"2024-05-30T16:43:54.086747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227740+0000","last_change":"2024-05-30T16:43:52.890768+0000","last_active":"2024-05-30T16:43:54.227740+0000","last_peered":"2024-05-30T16:43:54.227740+0000","last_clean":"2024-05-30T16:43:54.227740+0000","last_became_active":"2024-05-30T16:43:52.889014+0000","last_became_peered":"2024-05-30T16:43:52.889014+0000","last_unstale":"2024-05-30T16:43:54.227740+0000","last_undegraded":"2024-05-30T16:43:54.227740+0000","last_fullsized":"2024-05-30T16:43:54.227740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:15:50.512981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872029+0000","last_change":"2024-05-30T16:43:51.854719+0000","last_active":"2024-05-30T16:43:53.872029+0000","last_peered":"2024-05-30T16:43:53.872029+0000","last_clean":"2024-05-30T16:43:53.872029+0000","last_became_active":"2024-05-30T16:43:51.854430+0000","last_became_peered":"2024-05-30T16:43:51.854430+0000","last_unstale":"2024-05-30T16:43:53.872029+0000","last_undegraded":"2024-05-30T16:43:53.872029+0000","last_fullsized":"2024-05-30T16:43:53.872029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:03.683389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875608+0000","last_change":"2024-05-30T16:43:52.884624+0000","last_active":"2024-05-30T16:43:53.875608+0000","last_peered":"2024-05-30T16:43:53.875608+0000","last_clean":"2024-05-30T16:43:53.875608+0000","last_became_active":"2024-05-30T16:43:52.884348+0000","last_became_peered":"2024-05-30T16:43:52.884348+0000","last_unstale":"2024-05-30T16:43:53.875608+0000","last_undegraded":"2024-05-30T16:43:53.875608+0000","last_fullsized":"2024-05-30T16:43:53.875608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:32.373668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086493+0000","last_change":"2024-05-30T16:43:51.853597+0000","last_active":"2024-05-30T16:43:54.086493+0000","last_peered":"2024-05-30T16:43:54.086493+0000","last_clean":"2024-05-30T16:43:54.086493+0000","last_became_active":"2024-05-30T16:43:51.853235+0000","last_became_peered":"2024-05-30T16:43:51.853235+0000","last_unstale":"2024-05-30T16:43:54.086493+0000","last_undegraded":"2024-05-30T16:43:54.086493+0000","last_fullsized":"2024-05-30T16:43:54.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.228053+0000","last_change":"2024-05-30T16:43:52.902358+0000","last_active":"2024-05-30T16:43:54.228053+0000","last_peered":"2024-05-30T16:43:54.228053+0000","last_clean":"2024-05-30T16:43:54.228053+0000","last_became_active":"2024-05-30T16:43:52.902031+0000","last_became_peered":"2024-05-30T16:43:52.902031+0000","last_unstale":"2024-05-30T16:43:54.228053+0000","last_undegraded":"2024-05-30T16:43:54.228053+0000","last_fullsized":"2024-05-30T16:43:54.228053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:38:04.016805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225364+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:43:54.225364+0000","last_peered":"2024-05-30T16:43:54.225364+0000","last_clean":"2024-05-30T16:43:54.225364+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:43:54.225364+0000","last_undegraded":"2024-05-30T16:43:54.225364+0000","last_fullsized":"2024-05-30T16:43:54.225364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875313+0000","last_change":"2024-05-30T16:43:51.849559+0000","last_active":"2024-05-30T16:43:53.875313+0000","last_peered":"2024-05-30T16:43:53.875313+0000","last_clean":"2024-05-30T16:43:53.875313+0000","last_became_active":"2024-05-30T16:43:51.848551+0000","last_became_peered":"2024-05-30T16:43:51.848551+0000","last_unstale":"2024-05-30T16:43:53.875313+0000","last_undegraded":"2024-05-30T16:43:53.875313+0000","last_fullsized":"2024-05-30T16:43:53.875313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:22.667583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225407+0000","last_change":"2024-05-30T16:43:52.900074+0000","last_active":"2024-05-30T16:43:54.225407+0000","last_peered":"2024-05-30T16:43:54.225407+0000","last_clean":"2024-05-30T16:43:54.225407+0000","last_became_active":"2024-05-30T16:43:52.899764+0000","last_became_peered":"2024-05-30T16:43:52.899764+0000","last_unstale":"2024-05-30T16:43:54.225407+0000","last_undegraded":"2024-05-30T16:43:54.225407+0000","last_fullsized":"2024-05-30T16:43:54.225407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:44.861208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225298+0000","last_change":"2024-05-30T16:43:51.861550+0000","last_active":"2024-05-30T16:43:54.225298+0000","last_peered":"2024-05-30T16:43:54.225298+0000","last_clean":"2024-05-30T16:43:54.225298+0000","last_became_active":"2024-05-30T16:43:51.861157+0000","last_became_peered":"2024-05-30T16:43:51.861157+0000","last_unstale":"2024-05-30T16:43:54.225298+0000","last_undegraded":"2024-05-30T16:43:54.225298+0000","last_fullsized":"2024-05-30T16:43:54.225298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:20.333916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.156940+0000","last_change":"2024-05-30T16:43:52.878371+0000","last_active":"2024-05-30T16:43:54.156940+0000","last_peered":"2024-05-30T16:43:54.156940+0000","last_clean":"2024-05-30T16:43:54.156940+0000","last_became_active":"2024-05-30T16:43:52.877880+0000","last_became_peered":"2024-05-30T16:43:52.877880+0000","last_unstale":"2024-05-30T16:43:54.156940+0000","last_undegraded":"2024-05-30T16:43:54.156940+0000","last_fullsized":"2024-05-30T16:43:54.156940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:58.783247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086019+0000","last_change":"2024-05-30T16:43:51.858196+0000","last_active":"2024-05-30T16:43:54.086019+0000","last_peered":"2024-05-30T16:43:54.086019+0000","last_clean":"2024-05-30T16:43:54.086019+0000","last_became_active":"2024-05-30T16:43:51.857752+0000","last_became_peered":"2024-05-30T16:43:51.857752+0000","last_unstale":"2024-05-30T16:43:54.086019+0000","last_undegraded":"2024-05-30T16:43:54.086019+0000","last_fullsized":"2024-05-30T16:43:54.086019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871024+0000","last_change":"2024-05-30T16:43:52.863297+0000","last_active":"2024-05-30T16:43:53.871024+0000","last_peered":"2024-05-30T16:43:53.871024+0000","last_clean":"2024-05-30T16:43:53.871024+0000","last_became_active":"2024-05-30T16:43:52.863048+0000","last_became_peered":"2024-05-30T16:43:52.863048+0000","last_unstale":"2024-05-30T16:43:53.871024+0000","last_undegraded":"2024-05-30T16:43:53.871024+0000","last_fullsized":"2024-05-30T16:43:53.871024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874160+0000","last_change":"2024-05-30T16:43:51.854526+0000","last_active":"2024-05-30T16:43:53.874160+0000","last_peered":"2024-05-30T16:43:53.874160+0000","last_clean":"2024-05-30T16:43:53.874160+0000","last_became_active":"2024-05-30T16:43:51.853002+0000","last_became_peered":"2024-05-30T16:43:51.853002+0000","last_unstale":"2024-05-30T16:43:53.874160+0000","last_undegraded":"2024-05-30T16:43:53.874160+0000","last_fullsized":"2024-05-30T16:43:53.874160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:41.467122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225345+0000","last_change":"2024-05-30T16:43:52.876709+0000","last_active":"2024-05-30T16:43:54.225345+0000","last_peered":"2024-05-30T16:43:54.225345+0000","last_clean":"2024-05-30T16:43:54.225345+0000","last_became_active":"2024-05-30T16:43:52.876309+0000","last_became_peered":"2024-05-30T16:43:52.876309+0000","last_unstale":"2024-05-30T16:43:54.225345+0000","last_undegraded":"2024-05-30T16:43:54.225345+0000","last_fullsized":"2024-05-30T16:43:54.225345+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:35.643416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872135+0000","last_change":"2024-05-30T16:43:51.856730+0000","last_active":"2024-05-30T16:43:53.872135+0000","last_peered":"2024-05-30T16:43:53.872135+0000","last_clean":"2024-05-30T16:43:53.872135+0000","last_became_active":"2024-05-30T16:43:51.856409+0000","last_became_peered":"2024-05-30T16:43:51.856409+0000","last_unstale":"2024-05-30T16:43:53.872135+0000","last_undegraded":"2024-05-30T16:43:53.872135+0000","last_fullsized":"2024-05-30T16:43:53.872135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874822+0000","last_change":"2024-05-30T16:43:52.895760+0000","last_active":"2024-05-30T16:43:53.874822+0000","last_peered":"2024-05-30T16:43:53.874822+0000","last_clean":"2024-05-30T16:43:53.874822+0000","last_became_active":"2024-05-30T16:43:52.895305+0000","last_became_peered":"2024-05-30T16:43:52.895305+0000","last_unstale":"2024-05-30T16:43:53.874822+0000","last_undegraded":"2024-05-30T16:43:53.874822+0000","last_fullsized":"2024-05-30T16:43:53.874822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:03.535096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.228313+0000","last_change":"2024-05-30T16:43:51.864341+0000","last_active":"2024-05-30T16:43:54.228313+0000","last_peered":"2024-05-30T16:43:54.228313+0000","last_clean":"2024-05-30T16:43:54.228313+0000","last_became_active":"2024-05-30T16:43:51.864017+0000","last_became_peered":"2024-05-30T16:43:51.864017+0000","last_unstale":"2024-05-30T16:43:54.228313+0000","last_undegraded":"2024-05-30T16:43:54.228313+0000","last_fullsized":"2024-05-30T16:43:54.228313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:48.659115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871833+0000","last_change":"2024-05-30T16:43:52.865453+0000","last_active":"2024-05-30T16:43:53.871833+0000","last_peered":"2024-05-30T16:43:53.871833+0000","last_clean":"2024-05-30T16:43:53.871833+0000","last_became_active":"2024-05-30T16:43:52.864999+0000","last_became_peered":"2024-05-30T16:43:52.864999+0000","last_unstale":"2024-05-30T16:43:53.871833+0000","last_undegraded":"2024-05-30T16:43:53.871833+0000","last_fullsized":"2024-05-30T16:43:53.871833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.877105+0000","last_change":"2024-05-30T16:43:51.855757+0000","last_active":"2024-05-30T16:43:53.877105+0000","last_peered":"2024-05-30T16:43:53.877105+0000","last_clean":"2024-05-30T16:43:53.877105+0000","last_became_active":"2024-05-30T16:43:51.855051+0000","last_became_peered":"2024-05-30T16:43:51.855051+0000","last_unstale":"2024-05-30T16:43:53.877105+0000","last_undegraded":"2024-05-30T16:43:53.877105+0000","last_fullsized":"2024-05-30T16:43:53.877105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:34.628416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224825+0000","last_change":"2024-05-30T16:43:52.888339+0000","last_active":"2024-05-30T16:43:54.224825+0000","last_peered":"2024-05-30T16:43:54.224825+0000","last_clean":"2024-05-30T16:43:54.224825+0000","last_became_active":"2024-05-30T16:43:52.888104+0000","last_became_peered":"2024-05-30T16:43:52.888104+0000","last_unstale":"2024-05-30T16:43:54.224825+0000","last_undegraded":"2024-05-30T16:43:54.224825+0000","last_fullsized":"2024-05-30T16:43:54.224825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873115+0000","last_change":"2024-05-30T16:43:51.865200+0000","last_active":"2024-05-30T16:43:53.873115+0000","last_peered":"2024-05-30T16:43:53.873115+0000","last_clean":"2024-05-30T16:43:53.873115+0000","last_became_active":"2024-05-30T16:43:51.864864+0000","last_became_peered":"2024-05-30T16:43:51.864864+0000","last_unstale":"2024-05-30T16:43:53.873115+0000","last_undegraded":"2024-05-30T16:43:53.873115+0000","last_fullsized":"2024-05-30T16:43:53.873115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:34.628416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157688+0000","last_change":"2024-05-30T16:43:52.875686+0000","last_active":"2024-05-30T16:43:54.157688+0000","last_peered":"2024-05-30T16:43:54.157688+0000","last_clean":"2024-05-30T16:43:54.157688+0000","last_became_active":"2024-05-30T16:43:52.875422+0000","last_became_peered":"2024-05-30T16:43:52.875422+0000","last_unstale":"2024-05-30T16:43:54.157688+0000","last_undegraded":"2024-05-30T16:43:54.157688+0000","last_fullsized":"2024-05-30T16:43:54.157688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:57.617195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224865+0000","last_change":"2024-05-30T16:43:51.863391+0000","last_active":"2024-05-30T16:43:54.224865+0000","last_peered":"2024-05-30T16:43:54.224865+0000","last_clean":"2024-05-30T16:43:54.224865+0000","last_became_active":"2024-05-30T16:43:51.863097+0000","last_became_peered":"2024-05-30T16:43:51.863097+0000","last_unstale":"2024-05-30T16:43:54.224865+0000","last_undegraded":"2024-05-30T16:43:54.224865+0000","last_fullsized":"2024-05-30T16:43:54.224865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:40.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225485+0000","last_change":"2024-05-30T16:43:52.875177+0000","last_active":"2024-05-30T16:43:54.225485+0000","last_peered":"2024-05-30T16:43:54.225485+0000","last_clean":"2024-05-30T16:43:54.225485+0000","last_became_active":"2024-05-30T16:43:52.874601+0000","last_became_peered":"2024-05-30T16:43:52.874601+0000","last_unstale":"2024-05-30T16:43:54.225485+0000","last_undegraded":"2024-05-30T16:43:54.225485+0000","last_fullsized":"2024-05-30T16:43:54.225485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:50.920087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872070+0000","last_change":"2024-05-30T16:43:51.853127+0000","last_active":"2024-05-30T16:43:53.872070+0000","last_peered":"2024-05-30T16:43:53.872070+0000","last_clean":"2024-05-30T16:43:53.872070+0000","last_became_active":"2024-05-30T16:43:51.852860+0000","last_became_peered":"2024-05-30T16:43:51.852860+0000","last_unstale":"2024-05-30T16:43:53.872070+0000","last_undegraded":"2024-05-30T16:43:53.872070+0000","last_fullsized":"2024-05-30T16:43:53.872070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:49.905088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224426+0000","last_change":"2024-05-30T16:43:52.876127+0000","last_active":"2024-05-30T16:43:54.224426+0000","last_peered":"2024-05-30T16:43:54.224426+0000","last_clean":"2024-05-30T16:43:54.224426+0000","last_became_active":"2024-05-30T16:43:52.875763+0000","last_became_peered":"2024-05-30T16:43:52.875763+0000","last_unstale":"2024-05-30T16:43:54.224426+0000","last_undegraded":"2024-05-30T16:43:54.224426+0000","last_fullsized":"2024-05-30T16:43:54.224426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871499+0000","last_change":"2024-05-30T16:43:51.860285+0000","last_active":"2024-05-30T16:43:53.871499+0000","last_peered":"2024-05-30T16:43:53.871499+0000","last_clean":"2024-05-30T16:43:53.871499+0000","last_became_active":"2024-05-30T16:43:51.859565+0000","last_became_peered":"2024-05-30T16:43:51.859565+0000","last_unstale":"2024-05-30T16:43:53.871499+0000","last_undegraded":"2024-05-30T16:43:53.871499+0000","last_fullsized":"2024-05-30T16:43:53.871499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225236+0000","last_change":"2024-05-30T16:43:52.885852+0000","last_active":"2024-05-30T16:43:54.225236+0000","last_peered":"2024-05-30T16:43:54.225236+0000","last_clean":"2024-05-30T16:43:54.225236+0000","last_became_active":"2024-05-30T16:43:52.884228+0000","last_became_peered":"2024-05-30T16:43:52.884228+0000","last_unstale":"2024-05-30T16:43:54.225236+0000","last_undegraded":"2024-05-30T16:43:54.225236+0000","last_fullsized":"2024-05-30T16:43:54.225236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:12:06.956321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227233+0000","last_change":"2024-05-30T16:43:51.863477+0000","last_active":"2024-05-30T16:43:54.227233+0000","last_peered":"2024-05-30T16:43:54.227233+0000","last_clean":"2024-05-30T16:43:54.227233+0000","last_became_active":"2024-05-30T16:43:51.863178+0000","last_became_peered":"2024-05-30T16:43:51.863178+0000","last_unstale":"2024-05-30T16:43:54.227233+0000","last_undegraded":"2024-05-30T16:43:54.227233+0000","last_fullsized":"2024-05-30T16:43:54.227233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:20:50.371105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873046+0000","last_change":"2024-05-30T16:43:52.880010+0000","last_active":"2024-05-30T16:43:53.873046+0000","last_peered":"2024-05-30T16:43:53.873046+0000","last_clean":"2024-05-30T16:43:53.873046+0000","last_became_active":"2024-05-30T16:43:52.879744+0000","last_became_peered":"2024-05-30T16:43:52.879744+0000","last_unstale":"2024-05-30T16:43:53.873046+0000","last_undegraded":"2024-05-30T16:43:53.873046+0000","last_fullsized":"2024-05-30T16:43:53.873046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:30.437464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874603+0000","last_change":"2024-05-30T16:43:52.878768+0000","last_active":"2024-05-30T16:43:53.874603+0000","last_peered":"2024-05-30T16:43:53.874603+0000","last_clean":"2024-05-30T16:43:53.874603+0000","last_became_active":"2024-05-30T16:43:52.878437+0000","last_became_peered":"2024-05-30T16:43:52.878437+0000","last_unstale":"2024-05-30T16:43:53.874603+0000","last_undegraded":"2024-05-30T16:43:53.874603+0000","last_fullsized":"2024-05-30T16:43:53.874603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:13:13.668843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226899+0000","last_change":"2024-05-30T16:43:51.860527+0000","last_active":"2024-05-30T16:43:54.226899+0000","last_peered":"2024-05-30T16:43:54.226899+0000","last_clean":"2024-05-30T16:43:54.226899+0000","last_became_active":"2024-05-30T16:43:51.860201+0000","last_became_peered":"2024-05-30T16:43:51.860201+0000","last_unstale":"2024-05-30T16:43:54.226899+0000","last_undegraded":"2024-05-30T16:43:54.226899+0000","last_fullsized":"2024-05-30T16:43:54.226899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:44:14.302309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875458+0000","last_change":"2024-05-30T16:43:52.882046+0000","last_active":"2024-05-30T16:43:53.875458+0000","last_peered":"2024-05-30T16:43:53.875458+0000","last_clean":"2024-05-30T16:43:53.875458+0000","last_became_active":"2024-05-30T16:43:52.881777+0000","last_became_peered":"2024-05-30T16:43:52.881777+0000","last_unstale":"2024-05-30T16:43:53.875458+0000","last_undegraded":"2024-05-30T16:43:53.875458+0000","last_fullsized":"2024-05-30T16:43:53.875458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:05:00.764986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.228175+0000","last_change":"2024-05-30T16:43:51.865821+0000","last_active":"2024-05-30T16:43:54.228175+0000","last_peered":"2024-05-30T16:43:54.228175+0000","last_clean":"2024-05-30T16:43:54.228175+0000","last_became_active":"2024-05-30T16:43:51.865511+0000","last_became_peered":"2024-05-30T16:43:51.865511+0000","last_unstale":"2024-05-30T16:43:54.228175+0000","last_undegraded":"2024-05-30T16:43:54.228175+0000","last_fullsized":"2024-05-30T16:43:54.228175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:51.953401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157276+0000","last_change":"2024-05-30T16:43:52.901336+0000","last_active":"2024-05-30T16:43:54.157276+0000","last_peered":"2024-05-30T16:43:54.157276+0000","last_clean":"2024-05-30T16:43:54.157276+0000","last_became_active":"2024-05-30T16:43:52.900847+0000","last_became_peered":"2024-05-30T16:43:52.900847+0000","last_unstale":"2024-05-30T16:43:54.157276+0000","last_undegraded":"2024-05-30T16:43:54.157276+0000","last_fullsized":"2024-05-30T16:43:54.157276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:44.852718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225470+0000","last_change":"2024-05-30T16:43:51.853049+0000","last_active":"2024-05-30T16:43:54.225470+0000","last_peered":"2024-05-30T16:43:54.225470+0000","last_clean":"2024-05-30T16:43:54.225470+0000","last_became_active":"2024-05-30T16:43:51.852719+0000","last_became_peered":"2024-05-30T16:43:51.852719+0000","last_unstale":"2024-05-30T16:43:54.225470+0000","last_undegraded":"2024-05-30T16:43:54.225470+0000","last_fullsized":"2024-05-30T16:43:54.225470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:15.475651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225448+0000","last_change":"2024-05-30T16:43:52.880495+0000","last_active":"2024-05-30T16:43:54.225448+0000","last_peered":"2024-05-30T16:43:54.225448+0000","last_clean":"2024-05-30T16:43:54.225448+0000","last_became_active":"2024-05-30T16:43:52.879641+0000","last_became_peered":"2024-05-30T16:43:52.879641+0000","last_unstale":"2024-05-30T16:43:54.225448+0000","last_undegraded":"2024-05-30T16:43:54.225448+0000","last_fullsized":"2024-05-30T16:43:54.225448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:36.892645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157601+0000","last_change":"2024-05-30T16:43:51.859807+0000","last_active":"2024-05-30T16:43:54.157601+0000","last_peered":"2024-05-30T16:43:54.157601+0000","last_clean":"2024-05-30T16:43:54.157601+0000","last_became_active":"2024-05-30T16:43:51.859531+0000","last_became_peered":"2024-05-30T16:43:51.859531+0000","last_unstale":"2024-05-30T16:43:54.157601+0000","last_undegraded":"2024-05-30T16:43:54.157601+0000","last_fullsized":"2024-05-30T16:43:54.157601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:12.533740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":77338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":77212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":668162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":668218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:44:02.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:44:03.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.186+0000 7f95fac00640 1 Processor -- start 2024-05-30T16:44:03.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.187+0000 7f95fac00640 1 -- start start 2024-05-30T16:44:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.188+0000 7f95fac00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f95f4071bc0 0x7f95f4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.188+0000 7f95fac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f95f40b32c0 0x7f95f40b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.188+0000 7f95fac00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f406e6e0 con 0x7f95f4158b80 2024-05-30T16:44:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.188+0000 7f95fac00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f95f406e9c0 con 0x7f95f4071bc0 2024-05-30T16:44:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.188+0000 7f95fac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f95f406eca0 con 0x7f95f40b32c0 2024-05-30T16:44:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95f9800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f95f4158b80 0x7f95f406dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52360/0 (socket says 172.21.15.67:52360) 2024-05-30T16:44:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95f9800640 1 -- 172.21.15.67:0/2237018836 learned_addr learned my addr 172.21.15.67:0/2237018836 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95f8e00640 1 --2- 172.21.15.67:0/2237018836 >> v2:172.21.15.67:3300/0 conn(0x7f95f4071bc0 0x7f95f4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95fa200640 1 --2- 172.21.15.67:0/2237018836 >> v2:172.21.15.169:3300/0 conn(0x7f95f40b32c0 0x7f95f40b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95f3e00640 1 -- 172.21.15.67:0/2237018836 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766287316 0 0) 0x7f95f406e6e0 con 0x7f95f4158b80 2024-05-30T16:44:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95f3e00640 1 -- 172.21.15.67:0/2237018836 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95dc003520 con 0x7f95f4158b80 2024-05-30T16:44:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95f8e00640 1 -- 172.21.15.67:0/2237018836 >> v2:172.21.15.169:3300/0 conn(0x7f95f40b32c0 msgr2=0x7f95f40b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95f8e00640 1 --2- 172.21.15.67:0/2237018836 >> v2:172.21.15.169:3300/0 conn(0x7f95f40b32c0 0x7f95f40b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95f8e00640 1 -- 172.21.15.67:0/2237018836 >> v1:172.21.15.67:6789/0 conn(0x7f95f4158b80 legacy=0x7f95f406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95f8e00640 1 -- 172.21.15.67:0/2237018836 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f406ef80 con 0x7f95f4071bc0 2024-05-30T16:44:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.189+0000 7f95fa200640 1 --2- 172.21.15.67:0/2237018836 >> v2:172.21.15.169:3300/0 conn(0x7f95f40b32c0 0x7f95f40b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.190+0000 7f95f8e00640 1 --2- 172.21.15.67:0/2237018836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 0x7f95f4072070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f95f40b3050 tx=0x7f95e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f91e321677fe38 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.190+0000 7f95f3e00640 1 -- 172.21.15.67:0/2237018836 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0037070 con 0x7f95f4071bc0 2024-05-30T16:44:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.190+0000 7f95f3e00640 1 -- 172.21.15.67:0/2237018836 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e0032440 con 0x7f95f4071bc0 2024-05-30T16:44:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.190+0000 7f95f3e00640 1 -- 172.21.15.67:0/2237018836 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e00327f0 con 0x7f95f4071bc0 2024-05-30T16:44:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.191+0000 7f95fac00640 1 -- 172.21.15.67:0/2237018836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 msgr2=0x7f95f4072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.191+0000 7f95fac00640 1 --2- 172.21.15.67:0/2237018836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 0x7f95f4072070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f95f40b3050 tx=0x7f95e002a1a0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.191+0000 7f95fac00640 1 -- 172.21.15.67:0/2237018836 shutdown_connections 2024-05-30T16:44:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.191+0000 7f95fac00640 1 --2- 172.21.15.67:0/2237018836 >> v2:172.21.15.169:3300/0 conn(0x7f95f40b32c0 0x7f95f40b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.191+0000 7f95fac00640 1 --2- 172.21.15.67:0/2237018836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 0x7f95f4072070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.191+0000 7f95fac00640 1 -- 172.21.15.67:0/2237018836 >> 172.21.15.67:0/2237018836 conn(0x7f95f40b1fc0 msgr2=0x7f95f40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.191+0000 7f95fac00640 1 -- 172.21.15.67:0/2237018836 shutdown_connections 2024-05-30T16:44:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.191+0000 7f95fac00640 1 -- 172.21.15.67:0/2237018836 wait complete. 2024-05-30T16:44:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.193+0000 7f95fac00640 1 Processor -- start 2024-05-30T16:44:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.193+0000 7f95fac00640 1 -- start start 2024-05-30T16:44:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95fac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 0x7f95f41517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95fac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95f40b32c0 0x7f95f41f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95fac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4158b80 0x7f95f4156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95fac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f95f406f780 con 0x7f95f4071bc0 2024-05-30T16:44:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95fac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f95f40b2d30 con 0x7f95f4158b80 2024-05-30T16:44:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95fac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f95f41f4fc0 con 0x7f95f40b32c0 2024-05-30T16:44:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95f9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 0x7f95f41517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95f9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 0x7f95f41517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53156/0 (socket says 172.21.15.67:53156) 2024-05-30T16:44:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95f9800640 1 -- 172.21.15.67:0/92129079 learned_addr learned my addr 172.21.15.67:0/92129079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:03.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95f9800640 1 -- 172.21.15.67:0/92129079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95f40b32c0 msgr2=0x7f95f41f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95f8e00640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95f40b32c0 0x7f95f41f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95fa200640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4158b80 0x7f95f4156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95f9800640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95f40b32c0 0x7f95f41f3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95f9800640 1 -- 172.21.15.67:0/92129079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4158b80 msgr2=0x7f95f4156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95f9800640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4158b80 0x7f95f4156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.194+0000 7f95f9800640 1 -- 172.21.15.67:0/92129079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f41f4680 con 0x7f95f4071bc0 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.195+0000 7f95f9800640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 0x7f95f41517e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c390 tx=0x7f95e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.195+0000 7f95f1600640 1 -- 172.21.15.67:0/92129079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8017070 con 0x7f95f4071bc0 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.195+0000 7f95f1600640 1 -- 172.21.15.67:0/92129079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e8005ce0 con 0x7f95f4071bc0 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.195+0000 7f95f1600640 1 -- 172.21.15.67:0/92129079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80072f0 con 0x7f95f4071bc0 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.195+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f41f4960 con 0x7f95f4071bc0 2024-05-30T16:44:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.195+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f4151d20 con 0x7f95f4071bc0 2024-05-30T16:44:03.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.196+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f40689e0 con 0x7f95f4071bc0 2024-05-30T16:44:03.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.197+0000 7f95f1600640 1 -- 172.21.15.67:0/92129079 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95e801a070 con 0x7f95f4071bc0 2024-05-30T16:44:03.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.198+0000 7f95f1600640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95d406c8f0 0x7f95d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.199+0000 7f95f1600640 1 -- 172.21.15.67:0/92129079 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f95e808af80 con 0x7f95f4071bc0 2024-05-30T16:44:03.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.199+0000 7f95f8e00640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95d406c8f0 0x7f95d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.199+0000 7f95f8e00640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95d406c8f0 0x7f95d406edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95e0034000 tx=0x7f95e002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:03.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.203+0000 7f95f1600640 1 -- 172.21.15.67:0/92129079 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95e80593a0 con 0x7f95f4071bc0 2024-05-30T16:44:03.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.398+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95f40b3eb0 con 0x7f95d406c8f0 2024-05-30T16:44:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.416+0000 7f95f1600640 1 -- 172.21.15.67:0/92129079 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f95f40b3eb0 con 0x7f95d406c8f0 2024-05-30T16:44:03.420 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:03.422 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:44:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95d406c8f0 msgr2=0x7f95d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95d406c8f0 0x7f95d406edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95e0034000 tx=0x7f95e002e000 comp rx=0 tx=0).stop 2024-05-30T16:44:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 msgr2=0x7f95f41517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 0x7f95f41517e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c390 tx=0x7f95e800c850 comp rx=0 tx=0).stop 2024-05-30T16:44:03.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 shutdown_connections 2024-05-30T16:44:03.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95d406c8f0 0x7f95d406edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4158b80 0x7f95f4156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95f40b32c0 0x7f95f41f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 --2- 172.21.15.67:0/92129079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95f4071bc0 0x7f95f41517e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 >> 172.21.15.67:0/92129079 conn(0x7f95f40b1fc0 msgr2=0x7f95f4157b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:03.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 shutdown_connections 2024-05-30T16:44:03.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.422+0000 7f95fac00640 1 -- 172.21.15.67:0/92129079 wait complete. 2024-05-30T16:44:03.440 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-05-30T16:44:01.981954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219120,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755600,"statfs":{"total":773094113280,"available":772869734400,"internally_reserved":0,"allocated":3022848,"data_stored":1803252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.141352"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227428+0000","last_change":"2024-05-30T16:43:52.894258+0000","last_active":"2024-05-30T16:43:54.227428+0000","last_peered":"2024-05-30T16:43:54.227428+0000","last_clean":"2024-05-30T16:43:54.227428+0000","last_became_active":"2024-05-30T16:43:52.893919+0000","last_became_peered":"2024-05-30T16:43:52.893919+0000","last_unstale":"2024-05-30T16:43:54.227428+0000","last_undegraded":"2024-05-30T16:43:54.227428+0000","last_fullsized":"2024-05-30T16:43:54.227428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:32.373668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872293+0000","last_change":"2024-05-30T16:43:52.878535+0000","last_active":"2024-05-30T16:43:53.872293+0000","last_peered":"2024-05-30T16:43:53.872293+0000","last_clean":"2024-05-30T16:43:53.872293+0000","last_became_active":"2024-05-30T16:43:52.878243+0000","last_became_peered":"2024-05-30T16:43:52.878243+0000","last_unstale":"2024-05-30T16:43:53.872293+0000","last_undegraded":"2024-05-30T16:43:53.872293+0000","last_fullsized":"2024-05-30T16:43:53.872293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157387+0000","last_change":"2024-05-30T16:43:52.874827+0000","last_active":"2024-05-30T16:43:54.157387+0000","last_peered":"2024-05-30T16:43:54.157387+0000","last_clean":"2024-05-30T16:43:54.157387+0000","last_became_active":"2024-05-30T16:43:52.874483+0000","last_became_peered":"2024-05-30T16:43:52.874483+0000","last_unstale":"2024-05-30T16:43:54.157387+0000","last_undegraded":"2024-05-30T16:43:54.157387+0000","last_fullsized":"2024-05-30T16:43:54.157387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:57:27.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875633+0000","last_change":"2024-05-30T16:43:52.884775+0000","last_active":"2024-05-30T16:43:53.875633+0000","last_peered":"2024-05-30T16:43:53.875633+0000","last_clean":"2024-05-30T16:43:53.875633+0000","last_became_active":"2024-05-30T16:43:52.884436+0000","last_became_peered":"2024-05-30T16:43:52.884436+0000","last_unstale":"2024-05-30T16:43:53.875633+0000","last_undegraded":"2024-05-30T16:43:53.875633+0000","last_fullsized":"2024-05-30T16:43:53.875633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:25.275803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226885+0000","last_change":"2024-05-30T16:43:52.886555+0000","last_active":"2024-05-30T16:43:54.226885+0000","last_peered":"2024-05-30T16:43:54.226885+0000","last_clean":"2024-05-30T16:43:54.226885+0000","last_became_active":"2024-05-30T16:43:52.886203+0000","last_became_peered":"2024-05-30T16:43:52.886203+0000","last_unstale":"2024-05-30T16:43:54.226885+0000","last_undegraded":"2024-05-30T16:43:54.226885+0000","last_fullsized":"2024-05-30T16:43:54.226885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:03.535096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873858+0000","last_change":"2024-05-30T16:43:52.875329+0000","last_active":"2024-05-30T16:43:53.873858+0000","last_peered":"2024-05-30T16:43:53.873858+0000","last_clean":"2024-05-30T16:43:53.873858+0000","last_became_active":"2024-05-30T16:43:52.874980+0000","last_became_peered":"2024-05-30T16:43:52.874980+0000","last_unstale":"2024-05-30T16:43:53.873858+0000","last_undegraded":"2024-05-30T16:43:53.873858+0000","last_fullsized":"2024-05-30T16:43:53.873858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:31:48.436948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225680+0000","last_change":"2024-05-30T16:43:52.894028+0000","last_active":"2024-05-30T16:43:54.225680+0000","last_peered":"2024-05-30T16:43:54.225680+0000","last_clean":"2024-05-30T16:43:54.225680+0000","last_became_active":"2024-05-30T16:43:52.889967+0000","last_became_peered":"2024-05-30T16:43:52.889967+0000","last_unstale":"2024-05-30T16:43:54.225680+0000","last_undegraded":"2024-05-30T16:43:54.225680+0000","last_fullsized":"2024-05-30T16:43:54.225680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:21.139158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.223951+0000","last_change":"2024-05-30T16:43:52.899916+0000","last_active":"2024-05-30T16:43:54.223951+0000","last_peered":"2024-05-30T16:43:54.223951+0000","last_clean":"2024-05-30T16:43:54.223951+0000","last_became_active":"2024-05-30T16:43:52.899619+0000","last_became_peered":"2024-05-30T16:43:52.899619+0000","last_unstale":"2024-05-30T16:43:54.223951+0000","last_undegraded":"2024-05-30T16:43:54.223951+0000","last_fullsized":"2024-05-30T16:43:54.223951+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:30.437464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227567+0000","last_change":"2024-05-30T16:43:52.903027+0000","last_active":"2024-05-30T16:43:54.227567+0000","last_peered":"2024-05-30T16:43:54.227567+0000","last_clean":"2024-05-30T16:43:54.227567+0000","last_became_active":"2024-05-30T16:43:52.902799+0000","last_became_peered":"2024-05-30T16:43:52.902799+0000","last_unstale":"2024-05-30T16:43:54.227567+0000","last_undegraded":"2024-05-30T16:43:54.227567+0000","last_fullsized":"2024-05-30T16:43:54.227567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:35.070892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873608+0000","last_change":"2024-05-30T16:43:52.895565+0000","last_active":"2024-05-30T16:43:53.873608+0000","last_peered":"2024-05-30T16:43:53.873608+0000","last_clean":"2024-05-30T16:43:53.873608+0000","last_became_active":"2024-05-30T16:43:52.894829+0000","last_became_peered":"2024-05-30T16:43:52.894829+0000","last_unstale":"2024-05-30T16:43:53.873608+0000","last_undegraded":"2024-05-30T16:43:53.873608+0000","last_fullsized":"2024-05-30T16:43:53.873608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873611+0000","last_change":"2024-05-30T16:43:52.899464+0000","last_active":"2024-05-30T16:43:53.873611+0000","last_peered":"2024-05-30T16:43:53.873611+0000","last_clean":"2024-05-30T16:43:53.873611+0000","last_became_active":"2024-05-30T16:43:52.897925+0000","last_became_peered":"2024-05-30T16:43:52.897925+0000","last_unstale":"2024-05-30T16:43:53.873611+0000","last_undegraded":"2024-05-30T16:43:53.873611+0000","last_fullsized":"2024-05-30T16:43:53.873611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226199+0000","last_change":"2024-05-30T16:43:52.877782+0000","last_active":"2024-05-30T16:43:54.226199+0000","last_peered":"2024-05-30T16:43:54.226199+0000","last_clean":"2024-05-30T16:43:54.226199+0000","last_became_active":"2024-05-30T16:43:52.877466+0000","last_became_peered":"2024-05-30T16:43:52.877466+0000","last_unstale":"2024-05-30T16:43:54.226199+0000","last_undegraded":"2024-05-30T16:43:54.226199+0000","last_fullsized":"2024-05-30T16:43:54.226199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:33.066313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225046+0000","last_change":"2024-05-30T16:43:52.881578+0000","last_active":"2024-05-30T16:43:54.225046+0000","last_peered":"2024-05-30T16:43:54.225046+0000","last_clean":"2024-05-30T16:43:54.225046+0000","last_became_active":"2024-05-30T16:43:52.877597+0000","last_became_peered":"2024-05-30T16:43:52.877597+0000","last_unstale":"2024-05-30T16:43:54.225046+0000","last_undegraded":"2024-05-30T16:43:54.225046+0000","last_fullsized":"2024-05-30T16:43:54.225046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.156845+0000","last_change":"2024-05-30T16:43:52.874486+0000","last_active":"2024-05-30T16:43:54.156845+0000","last_peered":"2024-05-30T16:43:54.156845+0000","last_clean":"2024-05-30T16:43:54.156845+0000","last_became_active":"2024-05-30T16:43:52.872863+0000","last_became_peered":"2024-05-30T16:43:52.872863+0000","last_unstale":"2024-05-30T16:43:54.156845+0000","last_undegraded":"2024-05-30T16:43:54.156845+0000","last_fullsized":"2024-05-30T16:43:54.156845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:20:51.386105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872070+0000","last_change":"2024-05-30T16:43:52.877439+0000","last_active":"2024-05-30T16:43:53.872070+0000","last_peered":"2024-05-30T16:43:53.872070+0000","last_clean":"2024-05-30T16:43:53.872070+0000","last_became_active":"2024-05-30T16:43:52.877139+0000","last_became_peered":"2024-05-30T16:43:52.877139+0000","last_unstale":"2024-05-30T16:43:53.872070+0000","last_undegraded":"2024-05-30T16:43:53.872070+0000","last_fullsized":"2024-05-30T16:43:53.872070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873037+0000","last_change":"2024-05-30T16:43:52.872351+0000","last_active":"2024-05-30T16:43:53.873037+0000","last_peered":"2024-05-30T16:43:53.873037+0000","last_clean":"2024-05-30T16:43:53.873037+0000","last_became_active":"2024-05-30T16:43:52.872079+0000","last_became_peered":"2024-05-30T16:43:52.872079+0000","last_unstale":"2024-05-30T16:43:53.873037+0000","last_undegraded":"2024-05-30T16:43:53.873037+0000","last_fullsized":"2024-05-30T16:43:53.873037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:50.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874480+0000","last_change":"2024-05-30T16:43:52.897208+0000","last_active":"2024-05-30T16:43:53.874480+0000","last_peered":"2024-05-30T16:43:53.874480+0000","last_clean":"2024-05-30T16:43:53.874480+0000","last_became_active":"2024-05-30T16:43:52.895700+0000","last_became_peered":"2024-05-30T16:43:52.895700+0000","last_unstale":"2024-05-30T16:43:53.874480+0000","last_undegraded":"2024-05-30T16:43:53.874480+0000","last_fullsized":"2024-05-30T16:43:53.874480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:08:21.139158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086708+0000","last_change":"2024-05-30T16:43:52.870196+0000","last_active":"2024-05-30T16:43:54.086708+0000","last_peered":"2024-05-30T16:43:54.086708+0000","last_clean":"2024-05-30T16:43:54.086708+0000","last_became_active":"2024-05-30T16:43:52.869879+0000","last_became_peered":"2024-05-30T16:43:52.869879+0000","last_unstale":"2024-05-30T16:43:54.086708+0000","last_undegraded":"2024-05-30T16:43:54.086708+0000","last_fullsized":"2024-05-30T16:43:54.086708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:43.785583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.228432+0000","last_change":"2024-05-30T16:43:52.898388+0000","last_active":"2024-05-30T16:43:54.228432+0000","last_peered":"2024-05-30T16:43:54.228432+0000","last_clean":"2024-05-30T16:43:54.228432+0000","last_became_active":"2024-05-30T16:43:52.898159+0000","last_became_peered":"2024-05-30T16:43:52.898159+0000","last_unstale":"2024-05-30T16:43:54.228432+0000","last_undegraded":"2024-05-30T16:43:54.228432+0000","last_fullsized":"2024-05-30T16:43:54.228432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:41.187051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874369+0000","last_change":"2024-05-30T16:43:52.880715+0000","last_active":"2024-05-30T16:43:53.874369+0000","last_peered":"2024-05-30T16:43:53.874369+0000","last_clean":"2024-05-30T16:43:53.874369+0000","last_became_active":"2024-05-30T16:43:52.880406+0000","last_became_peered":"2024-05-30T16:43:52.880406+0000","last_unstale":"2024-05-30T16:43:53.874369+0000","last_undegraded":"2024-05-30T16:43:53.874369+0000","last_fullsized":"2024-05-30T16:43:53.874369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157707+0000","last_change":"2024-05-30T16:43:52.895619+0000","last_active":"2024-05-30T16:43:54.157707+0000","last_peered":"2024-05-30T16:43:54.157707+0000","last_clean":"2024-05-30T16:43:54.157707+0000","last_became_active":"2024-05-30T16:43:52.891632+0000","last_became_peered":"2024-05-30T16:43:52.891632+0000","last_unstale":"2024-05-30T16:43:54.157707+0000","last_undegraded":"2024-05-30T16:43:54.157707+0000","last_fullsized":"2024-05-30T16:43:54.157707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:29:07.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873953+0000","last_change":"2024-05-30T16:43:52.872959+0000","last_active":"2024-05-30T16:43:53.873953+0000","last_peered":"2024-05-30T16:43:53.873953+0000","last_clean":"2024-05-30T16:43:53.873953+0000","last_became_active":"2024-05-30T16:43:52.872679+0000","last_became_peered":"2024-05-30T16:43:52.872679+0000","last_unstale":"2024-05-30T16:43:53.873953+0000","last_undegraded":"2024-05-30T16:43:53.873953+0000","last_fullsized":"2024-05-30T16:43:53.873953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:08:57.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225306+0000","last_change":"2024-05-30T16:43:52.885877+0000","last_active":"2024-05-30T16:43:54.225306+0000","last_peered":"2024-05-30T16:43:54.225306+0000","last_clean":"2024-05-30T16:43:54.225306+0000","last_became_active":"2024-05-30T16:43:52.885545+0000","last_became_peered":"2024-05-30T16:43:52.885545+0000","last_unstale":"2024-05-30T16:43:54.225306+0000","last_undegraded":"2024-05-30T16:43:54.225306+0000","last_fullsized":"2024-05-30T16:43:54.225306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:00.933568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873194+0000","last_change":"2024-05-30T16:43:52.895695+0000","last_active":"2024-05-30T16:43:53.873194+0000","last_peered":"2024-05-30T16:43:53.873194+0000","last_clean":"2024-05-30T16:43:53.873194+0000","last_became_active":"2024-05-30T16:43:52.895370+0000","last_became_peered":"2024-05-30T16:43:52.895370+0000","last_unstale":"2024-05-30T16:43:53.873194+0000","last_undegraded":"2024-05-30T16:43:53.873194+0000","last_fullsized":"2024-05-30T16:43:53.873194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:31.485813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224641+0000","last_change":"2024-05-30T16:43:52.883128+0000","last_active":"2024-05-30T16:43:54.224641+0000","last_peered":"2024-05-30T16:43:54.224641+0000","last_clean":"2024-05-30T16:43:54.224641+0000","last_became_active":"2024-05-30T16:43:52.881729+0000","last_became_peered":"2024-05-30T16:43:52.881729+0000","last_unstale":"2024-05-30T16:43:54.224641+0000","last_undegraded":"2024-05-30T16:43:54.224641+0000","last_fullsized":"2024-05-30T16:43:54.224641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:48:33.135201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873862+0000","last_change":"2024-05-30T16:43:52.876820+0000","last_active":"2024-05-30T16:43:53.873862+0000","last_peered":"2024-05-30T16:43:53.873862+0000","last_clean":"2024-05-30T16:43:53.873862+0000","last_became_active":"2024-05-30T16:43:52.875407+0000","last_became_peered":"2024-05-30T16:43:52.875407+0000","last_unstale":"2024-05-30T16:43:53.873862+0000","last_undegraded":"2024-05-30T16:43:53.873862+0000","last_fullsized":"2024-05-30T16:43:53.873862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871846+0000","last_change":"2024-05-30T16:43:52.883372+0000","last_active":"2024-05-30T16:43:53.871846+0000","last_peered":"2024-05-30T16:43:53.871846+0000","last_clean":"2024-05-30T16:43:53.871846+0000","last_became_active":"2024-05-30T16:43:52.882611+0000","last_became_peered":"2024-05-30T16:43:52.882611+0000","last_unstale":"2024-05-30T16:43:53.871846+0000","last_undegraded":"2024-05-30T16:43:53.871846+0000","last_fullsized":"2024-05-30T16:43:53.871846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:33.760386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224988+0000","last_change":"2024-05-30T16:43:52.882394+0000","last_active":"2024-05-30T16:43:54.224988+0000","last_peered":"2024-05-30T16:43:54.224988+0000","last_clean":"2024-05-30T16:43:54.224988+0000","last_became_active":"2024-05-30T16:43:52.882126+0000","last_became_peered":"2024-05-30T16:43:52.882126+0000","last_unstale":"2024-05-30T16:43:54.224988+0000","last_undegraded":"2024-05-30T16:43:54.224988+0000","last_fullsized":"2024-05-30T16:43:54.224988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157761+0000","last_change":"2024-05-30T16:43:52.884885+0000","last_active":"2024-05-30T16:43:54.157761+0000","last_peered":"2024-05-30T16:43:54.157761+0000","last_clean":"2024-05-30T16:43:54.157761+0000","last_became_active":"2024-05-30T16:43:52.884414+0000","last_became_peered":"2024-05-30T16:43:52.884414+0000","last_unstale":"2024-05-30T16:43:54.157761+0000","last_undegraded":"2024-05-30T16:43:54.157761+0000","last_fullsized":"2024-05-30T16:43:54.157761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:50.020862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874039+0000","last_change":"2024-05-30T16:43:52.890624+0000","last_active":"2024-05-30T16:43:53.874039+0000","last_peered":"2024-05-30T16:43:53.874039+0000","last_clean":"2024-05-30T16:43:53.874039+0000","last_became_active":"2024-05-30T16:43:52.890269+0000","last_became_peered":"2024-05-30T16:43:52.890269+0000","last_unstale":"2024-05-30T16:43:53.874039+0000","last_undegraded":"2024-05-30T16:43:53.874039+0000","last_fullsized":"2024-05-30T16:43:53.874039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:48.878065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086162+0000","last_change":"2024-05-30T16:43:52.891560+0000","last_active":"2024-05-30T16:43:54.086162+0000","last_peered":"2024-05-30T16:43:54.086162+0000","last_clean":"2024-05-30T16:43:54.086162+0000","last_became_active":"2024-05-30T16:43:52.891194+0000","last_became_peered":"2024-05-30T16:43:52.891194+0000","last_unstale":"2024-05-30T16:43:54.086162+0000","last_undegraded":"2024-05-30T16:43:54.086162+0000","last_fullsized":"2024-05-30T16:43:54.086162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:33.066313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873436+0000","last_change":"2024-05-30T16:43:52.876082+0000","last_active":"2024-05-30T16:43:53.873436+0000","last_peered":"2024-05-30T16:43:53.873436+0000","last_clean":"2024-05-30T16:43:53.873436+0000","last_became_active":"2024-05-30T16:43:52.875792+0000","last_became_peered":"2024-05-30T16:43:52.875792+0000","last_unstale":"2024-05-30T16:43:53.873436+0000","last_undegraded":"2024-05-30T16:43:53.873436+0000","last_fullsized":"2024-05-30T16:43:53.873436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225172+0000","last_change":"2024-05-30T16:43:52.885848+0000","last_active":"2024-05-30T16:43:54.225172+0000","last_peered":"2024-05-30T16:43:54.225172+0000","last_clean":"2024-05-30T16:43:54.225172+0000","last_became_active":"2024-05-30T16:43:52.885532+0000","last_became_peered":"2024-05-30T16:43:52.885532+0000","last_unstale":"2024-05-30T16:43:54.225172+0000","last_undegraded":"2024-05-30T16:43:54.225172+0000","last_fullsized":"2024-05-30T16:43:54.225172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:04.698389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225258+0000","last_change":"2024-05-30T16:43:51.854344+0000","last_active":"2024-05-30T16:43:54.225258+0000","last_peered":"2024-05-30T16:43:54.225258+0000","last_clean":"2024-05-30T16:43:54.225258+0000","last_became_active":"2024-05-30T16:43:51.854081+0000","last_became_peered":"2024-05-30T16:43:51.854081+0000","last_unstale":"2024-05-30T16:43:54.225258+0000","last_undegraded":"2024-05-30T16:43:54.225258+0000","last_fullsized":"2024-05-30T16:43:54.225258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:50.620803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.158196+0000","last_change":"2024-05-30T16:43:52.881003+0000","last_active":"2024-05-30T16:43:54.158196+0000","last_peered":"2024-05-30T16:43:54.158196+0000","last_clean":"2024-05-30T16:43:54.158196+0000","last_became_active":"2024-05-30T16:43:52.880834+0000","last_became_peered":"2024-05-30T16:43:52.880834+0000","last_unstale":"2024-05-30T16:43:54.158196+0000","last_undegraded":"2024-05-30T16:43:54.158196+0000","last_fullsized":"2024-05-30T16:43:54.158196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:04:31.485813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873330+0000","last_change":"2024-05-30T16:43:51.863107+0000","last_active":"2024-05-30T16:43:53.873330+0000","last_peered":"2024-05-30T16:43:53.873330+0000","last_clean":"2024-05-30T16:43:53.873330+0000","last_became_active":"2024-05-30T16:43:51.855082+0000","last_became_peered":"2024-05-30T16:43:51.855082+0000","last_unstale":"2024-05-30T16:43:53.873330+0000","last_undegraded":"2024-05-30T16:43:53.873330+0000","last_fullsized":"2024-05-30T16:43:53.873330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224700+0000","last_change":"2024-05-30T16:43:51.856427+0000","last_active":"2024-05-30T16:43:54.224700+0000","last_peered":"2024-05-30T16:43:54.224700+0000","last_clean":"2024-05-30T16:43:54.224700+0000","last_became_active":"2024-05-30T16:43:51.856150+0000","last_became_peered":"2024-05-30T16:43:51.856150+0000","last_unstale":"2024-05-30T16:43:54.224700+0000","last_undegraded":"2024-05-30T16:43:54.224700+0000","last_fullsized":"2024-05-30T16:43:54.224700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:47.064682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224116+0000","last_change":"2024-05-30T16:43:52.875668+0000","last_active":"2024-05-30T16:43:54.224116+0000","last_peered":"2024-05-30T16:43:54.224116+0000","last_clean":"2024-05-30T16:43:54.224116+0000","last_became_active":"2024-05-30T16:43:52.875362+0000","last_became_peered":"2024-05-30T16:43:52.875362+0000","last_unstale":"2024-05-30T16:43:54.224116+0000","last_undegraded":"2024-05-30T16:43:54.224116+0000","last_fullsized":"2024-05-30T16:43:54.224116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873259+0000","last_change":"2024-05-30T16:43:51.854040+0000","last_active":"2024-05-30T16:43:53.873259+0000","last_peered":"2024-05-30T16:43:53.873259+0000","last_clean":"2024-05-30T16:43:53.873259+0000","last_became_active":"2024-05-30T16:43:51.852832+0000","last_became_peered":"2024-05-30T16:43:51.852832+0000","last_unstale":"2024-05-30T16:43:53.873259+0000","last_undegraded":"2024-05-30T16:43:53.873259+0000","last_fullsized":"2024-05-30T16:43:53.873259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:16:24.379706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871843+0000","last_change":"2024-05-30T16:43:52.880882+0000","last_active":"2024-05-30T16:43:53.871843+0000","last_peered":"2024-05-30T16:43:53.871843+0000","last_clean":"2024-05-30T16:43:53.871843+0000","last_became_active":"2024-05-30T16:43:52.880558+0000","last_became_peered":"2024-05-30T16:43:52.880558+0000","last_unstale":"2024-05-30T16:43:53.871843+0000","last_undegraded":"2024-05-30T16:43:53.871843+0000","last_fullsized":"2024-05-30T16:43:53.871843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:16.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872979+0000","last_change":"2024-05-30T16:43:51.864315+0000","last_active":"2024-05-30T16:43:53.872979+0000","last_peered":"2024-05-30T16:43:53.872979+0000","last_clean":"2024-05-30T16:43:53.872979+0000","last_became_active":"2024-05-30T16:43:51.855552+0000","last_became_peered":"2024-05-30T16:43:51.855552+0000","last_unstale":"2024-05-30T16:43:53.872979+0000","last_undegraded":"2024-05-30T16:43:53.872979+0000","last_fullsized":"2024-05-30T16:43:53.872979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:03.683389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875985+0000","last_change":"2024-05-30T16:43:52.876866+0000","last_active":"2024-05-30T16:43:53.875985+0000","last_peered":"2024-05-30T16:43:53.875985+0000","last_clean":"2024-05-30T16:43:53.875985+0000","last_became_active":"2024-05-30T16:43:52.876208+0000","last_became_peered":"2024-05-30T16:43:52.876208+0000","last_unstale":"2024-05-30T16:43:53.875985+0000","last_undegraded":"2024-05-30T16:43:53.875985+0000","last_fullsized":"2024-05-30T16:43:53.875985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:35.070892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.876134+0000","last_change":"2024-05-30T16:43:51.857532+0000","last_active":"2024-05-30T16:43:53.876134+0000","last_peered":"2024-05-30T16:43:53.876134+0000","last_clean":"2024-05-30T16:43:53.876134+0000","last_became_active":"2024-05-30T16:43:51.853042+0000","last_became_peered":"2024-05-30T16:43:51.853042+0000","last_unstale":"2024-05-30T16:43:53.876134+0000","last_undegraded":"2024-05-30T16:43:53.876134+0000","last_fullsized":"2024-05-30T16:43:53.876134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:19.738945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157472+0000","last_change":"2024-05-30T16:43:52.901145+0000","last_active":"2024-05-30T16:43:54.157472+0000","last_peered":"2024-05-30T16:43:54.157472+0000","last_clean":"2024-05-30T16:43:54.157472+0000","last_became_active":"2024-05-30T16:43:52.900449+0000","last_became_peered":"2024-05-30T16:43:52.900449+0000","last_unstale":"2024-05-30T16:43:54.157472+0000","last_undegraded":"2024-05-30T16:43:54.157472+0000","last_fullsized":"2024-05-30T16:43:54.157472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:33:20.915701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871954+0000","last_change":"2024-05-30T16:43:51.857074+0000","last_active":"2024-05-30T16:43:53.871954+0000","last_peered":"2024-05-30T16:43:53.871954+0000","last_clean":"2024-05-30T16:43:53.871954+0000","last_became_active":"2024-05-30T16:43:51.856790+0000","last_became_peered":"2024-05-30T16:43:51.856790+0000","last_unstale":"2024-05-30T16:43:53.871954+0000","last_undegraded":"2024-05-30T16:43:53.871954+0000","last_fullsized":"2024-05-30T16:43:53.871954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226411+0000","last_change":"2024-05-30T16:43:52.883913+0000","last_active":"2024-05-30T16:43:54.226411+0000","last_peered":"2024-05-30T16:43:54.226411+0000","last_clean":"2024-05-30T16:43:54.226411+0000","last_became_active":"2024-05-30T16:43:52.883608+0000","last_became_peered":"2024-05-30T16:43:52.883608+0000","last_unstale":"2024-05-30T16:43:54.226411+0000","last_undegraded":"2024-05-30T16:43:54.226411+0000","last_fullsized":"2024-05-30T16:43:54.226411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:43.785583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874624+0000","last_change":"2024-05-30T16:43:51.859307+0000","last_active":"2024-05-30T16:43:53.874624+0000","last_peered":"2024-05-30T16:43:53.874624+0000","last_clean":"2024-05-30T16:43:53.874624+0000","last_became_active":"2024-05-30T16:43:51.858858+0000","last_became_peered":"2024-05-30T16:43:51.858858+0000","last_unstale":"2024-05-30T16:43:53.874624+0000","last_undegraded":"2024-05-30T16:43:53.874624+0000","last_fullsized":"2024-05-30T16:43:53.874624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227963+0000","last_change":"2024-05-30T16:43:52.892760+0000","last_active":"2024-05-30T16:43:54.227963+0000","last_peered":"2024-05-30T16:43:54.227963+0000","last_clean":"2024-05-30T16:43:54.227963+0000","last_became_active":"2024-05-30T16:43:52.892483+0000","last_became_peered":"2024-05-30T16:43:52.892483+0000","last_unstale":"2024-05-30T16:43:54.227963+0000","last_undegraded":"2024-05-30T16:43:54.227963+0000","last_fullsized":"2024-05-30T16:43:54.227963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:11.630186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.158046+0000","last_change":"2024-05-30T16:43:51.849511+0000","last_active":"2024-05-30T16:43:54.158046+0000","last_peered":"2024-05-30T16:43:54.158046+0000","last_clean":"2024-05-30T16:43:54.158046+0000","last_became_active":"2024-05-30T16:43:51.842797+0000","last_became_peered":"2024-05-30T16:43:51.842797+0000","last_unstale":"2024-05-30T16:43:54.158046+0000","last_undegraded":"2024-05-30T16:43:54.158046+0000","last_fullsized":"2024-05-30T16:43:54.158046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:30:25.098938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157148+0000","last_change":"2024-05-30T16:43:52.887806+0000","last_active":"2024-05-30T16:43:54.157148+0000","last_peered":"2024-05-30T16:43:54.157148+0000","last_clean":"2024-05-30T16:43:54.157148+0000","last_became_active":"2024-05-30T16:43:52.887486+0000","last_became_peered":"2024-05-30T16:43:52.887486+0000","last_unstale":"2024-05-30T16:43:54.157148+0000","last_undegraded":"2024-05-30T16:43:54.157148+0000","last_fullsized":"2024-05-30T16:43:54.157148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:05.404594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874098+0000","last_change":"2024-05-30T16:43:51.857906+0000","last_active":"2024-05-30T16:43:53.874098+0000","last_peered":"2024-05-30T16:43:53.874098+0000","last_clean":"2024-05-30T16:43:53.874098+0000","last_became_active":"2024-05-30T16:43:51.855395+0000","last_became_peered":"2024-05-30T16:43:51.855395+0000","last_unstale":"2024-05-30T16:43:53.874098+0000","last_undegraded":"2024-05-30T16:43:53.874098+0000","last_fullsized":"2024-05-30T16:43:53.874098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:32:37.147415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226899+0000","last_change":"2024-05-30T16:43:52.888949+0000","last_active":"2024-05-30T16:43:54.226899+0000","last_peered":"2024-05-30T16:43:54.226899+0000","last_clean":"2024-05-30T16:43:54.226899+0000","last_became_active":"2024-05-30T16:43:52.887708+0000","last_became_peered":"2024-05-30T16:43:52.887708+0000","last_unstale":"2024-05-30T16:43:54.226899+0000","last_undegraded":"2024-05-30T16:43:54.226899+0000","last_fullsized":"2024-05-30T16:43:54.226899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:05:00.764986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873316+0000","last_change":"2024-05-30T16:43:51.856817+0000","last_active":"2024-05-30T16:43:53.873316+0000","last_peered":"2024-05-30T16:43:53.873316+0000","last_clean":"2024-05-30T16:43:53.873316+0000","last_became_active":"2024-05-30T16:43:51.856519+0000","last_became_peered":"2024-05-30T16:43:51.856519+0000","last_unstale":"2024-05-30T16:43:53.873316+0000","last_undegraded":"2024-05-30T16:43:53.873316+0000","last_fullsized":"2024-05-30T16:43:53.873316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873045+0000","last_change":"2024-05-30T16:43:52.874643+0000","last_active":"2024-05-30T16:43:53.873045+0000","last_peered":"2024-05-30T16:43:53.873045+0000","last_clean":"2024-05-30T16:43:53.873045+0000","last_became_active":"2024-05-30T16:43:52.874280+0000","last_became_peered":"2024-05-30T16:43:52.874280+0000","last_unstale":"2024-05-30T16:43:53.873045+0000","last_undegraded":"2024-05-30T16:43:53.873045+0000","last_fullsized":"2024-05-30T16:43:53.873045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086111+0000","last_change":"2024-05-30T16:43:51.851393+0000","last_active":"2024-05-30T16:43:54.086111+0000","last_peered":"2024-05-30T16:43:54.086111+0000","last_clean":"2024-05-30T16:43:54.086111+0000","last_became_active":"2024-05-30T16:43:51.851057+0000","last_became_peered":"2024-05-30T16:43:51.851057+0000","last_unstale":"2024-05-30T16:43:54.086111+0000","last_undegraded":"2024-05-30T16:43:54.086111+0000","last_fullsized":"2024-05-30T16:43:54.086111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:15:50.620803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225990+0000","last_change":"2024-05-30T16:43:52.880958+0000","last_active":"2024-05-30T16:43:54.225990+0000","last_peered":"2024-05-30T16:43:54.225990+0000","last_clean":"2024-05-30T16:43:54.225990+0000","last_became_active":"2024-05-30T16:43:52.880536+0000","last_became_peered":"2024-05-30T16:43:52.880536+0000","last_unstale":"2024-05-30T16:43:54.225990+0000","last_undegraded":"2024-05-30T16:43:54.225990+0000","last_fullsized":"2024-05-30T16:43:54.225990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:13:13.668843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226477+0000","last_change":"2024-05-30T16:43:51.854467+0000","last_active":"2024-05-30T16:43:54.226477+0000","last_peered":"2024-05-30T16:43:54.226477+0000","last_clean":"2024-05-30T16:43:54.226477+0000","last_became_active":"2024-05-30T16:43:51.854104+0000","last_became_peered":"2024-05-30T16:43:51.854104+0000","last_unstale":"2024-05-30T16:43:54.226477+0000","last_undegraded":"2024-05-30T16:43:54.226477+0000","last_fullsized":"2024-05-30T16:43:54.226477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:26.393072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227506+0000","last_change":"2024-05-30T16:43:52.888958+0000","last_active":"2024-05-30T16:43:54.227506+0000","last_peered":"2024-05-30T16:43:54.227506+0000","last_clean":"2024-05-30T16:43:54.227506+0000","last_became_active":"2024-05-30T16:43:52.886708+0000","last_became_peered":"2024-05-30T16:43:52.886708+0000","last_unstale":"2024-05-30T16:43:54.227506+0000","last_undegraded":"2024-05-30T16:43:54.227506+0000","last_fullsized":"2024-05-30T16:43:54.227506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:25.275803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086747+0000","last_change":"2024-05-30T16:43:51.859324+0000","last_active":"2024-05-30T16:43:54.086747+0000","last_peered":"2024-05-30T16:43:54.086747+0000","last_clean":"2024-05-30T16:43:54.086747+0000","last_became_active":"2024-05-30T16:43:51.858960+0000","last_became_peered":"2024-05-30T16:43:51.858960+0000","last_unstale":"2024-05-30T16:43:54.086747+0000","last_undegraded":"2024-05-30T16:43:54.086747+0000","last_fullsized":"2024-05-30T16:43:54.086747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:35.508963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227740+0000","last_change":"2024-05-30T16:43:52.890768+0000","last_active":"2024-05-30T16:43:54.227740+0000","last_peered":"2024-05-30T16:43:54.227740+0000","last_clean":"2024-05-30T16:43:54.227740+0000","last_became_active":"2024-05-30T16:43:52.889014+0000","last_became_peered":"2024-05-30T16:43:52.889014+0000","last_unstale":"2024-05-30T16:43:54.227740+0000","last_undegraded":"2024-05-30T16:43:54.227740+0000","last_fullsized":"2024-05-30T16:43:54.227740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:15:50.512981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872029+0000","last_change":"2024-05-30T16:43:51.854719+0000","last_active":"2024-05-30T16:43:53.872029+0000","last_peered":"2024-05-30T16:43:53.872029+0000","last_clean":"2024-05-30T16:43:53.872029+0000","last_became_active":"2024-05-30T16:43:51.854430+0000","last_became_peered":"2024-05-30T16:43:51.854430+0000","last_unstale":"2024-05-30T16:43:53.872029+0000","last_undegraded":"2024-05-30T16:43:53.872029+0000","last_fullsized":"2024-05-30T16:43:53.872029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:50:03.683389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875608+0000","last_change":"2024-05-30T16:43:52.884624+0000","last_active":"2024-05-30T16:43:53.875608+0000","last_peered":"2024-05-30T16:43:53.875608+0000","last_clean":"2024-05-30T16:43:53.875608+0000","last_became_active":"2024-05-30T16:43:52.884348+0000","last_became_peered":"2024-05-30T16:43:52.884348+0000","last_unstale":"2024-05-30T16:43:53.875608+0000","last_undegraded":"2024-05-30T16:43:53.875608+0000","last_fullsized":"2024-05-30T16:43:53.875608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:32.373668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086493+0000","last_change":"2024-05-30T16:43:51.853597+0000","last_active":"2024-05-30T16:43:54.086493+0000","last_peered":"2024-05-30T16:43:54.086493+0000","last_clean":"2024-05-30T16:43:54.086493+0000","last_became_active":"2024-05-30T16:43:51.853235+0000","last_became_peered":"2024-05-30T16:43:51.853235+0000","last_unstale":"2024-05-30T16:43:54.086493+0000","last_undegraded":"2024-05-30T16:43:54.086493+0000","last_fullsized":"2024-05-30T16:43:54.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.228053+0000","last_change":"2024-05-30T16:43:52.902358+0000","last_active":"2024-05-30T16:43:54.228053+0000","last_peered":"2024-05-30T16:43:54.228053+0000","last_clean":"2024-05-30T16:43:54.228053+0000","last_became_active":"2024-05-30T16:43:52.902031+0000","last_became_peered":"2024-05-30T16:43:52.902031+0000","last_unstale":"2024-05-30T16:43:54.228053+0000","last_undegraded":"2024-05-30T16:43:54.228053+0000","last_fullsized":"2024-05-30T16:43:54.228053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:38:04.016805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225364+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:43:54.225364+0000","last_peered":"2024-05-30T16:43:54.225364+0000","last_clean":"2024-05-30T16:43:54.225364+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:43:54.225364+0000","last_undegraded":"2024-05-30T16:43:54.225364+0000","last_fullsized":"2024-05-30T16:43:54.225364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875313+0000","last_change":"2024-05-30T16:43:51.849559+0000","last_active":"2024-05-30T16:43:53.875313+0000","last_peered":"2024-05-30T16:43:53.875313+0000","last_clean":"2024-05-30T16:43:53.875313+0000","last_became_active":"2024-05-30T16:43:51.848551+0000","last_became_peered":"2024-05-30T16:43:51.848551+0000","last_unstale":"2024-05-30T16:43:53.875313+0000","last_undegraded":"2024-05-30T16:43:53.875313+0000","last_fullsized":"2024-05-30T16:43:53.875313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:22.667583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225407+0000","last_change":"2024-05-30T16:43:52.900074+0000","last_active":"2024-05-30T16:43:54.225407+0000","last_peered":"2024-05-30T16:43:54.225407+0000","last_clean":"2024-05-30T16:43:54.225407+0000","last_became_active":"2024-05-30T16:43:52.899764+0000","last_became_peered":"2024-05-30T16:43:52.899764+0000","last_unstale":"2024-05-30T16:43:54.225407+0000","last_undegraded":"2024-05-30T16:43:54.225407+0000","last_fullsized":"2024-05-30T16:43:54.225407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:44.861208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225298+0000","last_change":"2024-05-30T16:43:51.861550+0000","last_active":"2024-05-30T16:43:54.225298+0000","last_peered":"2024-05-30T16:43:54.225298+0000","last_clean":"2024-05-30T16:43:54.225298+0000","last_became_active":"2024-05-30T16:43:51.861157+0000","last_became_peered":"2024-05-30T16:43:51.861157+0000","last_unstale":"2024-05-30T16:43:54.225298+0000","last_undegraded":"2024-05-30T16:43:54.225298+0000","last_fullsized":"2024-05-30T16:43:54.225298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:20.333916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.156940+0000","last_change":"2024-05-30T16:43:52.878371+0000","last_active":"2024-05-30T16:43:54.156940+0000","last_peered":"2024-05-30T16:43:54.156940+0000","last_clean":"2024-05-30T16:43:54.156940+0000","last_became_active":"2024-05-30T16:43:52.877880+0000","last_became_peered":"2024-05-30T16:43:52.877880+0000","last_unstale":"2024-05-30T16:43:54.156940+0000","last_undegraded":"2024-05-30T16:43:54.156940+0000","last_fullsized":"2024-05-30T16:43:54.156940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:26:58.783247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.086019+0000","last_change":"2024-05-30T16:43:51.858196+0000","last_active":"2024-05-30T16:43:54.086019+0000","last_peered":"2024-05-30T16:43:54.086019+0000","last_clean":"2024-05-30T16:43:54.086019+0000","last_became_active":"2024-05-30T16:43:51.857752+0000","last_became_peered":"2024-05-30T16:43:51.857752+0000","last_unstale":"2024-05-30T16:43:54.086019+0000","last_undegraded":"2024-05-30T16:43:54.086019+0000","last_fullsized":"2024-05-30T16:43:54.086019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871024+0000","last_change":"2024-05-30T16:43:52.863297+0000","last_active":"2024-05-30T16:43:53.871024+0000","last_peered":"2024-05-30T16:43:53.871024+0000","last_clean":"2024-05-30T16:43:53.871024+0000","last_became_active":"2024-05-30T16:43:52.863048+0000","last_became_peered":"2024-05-30T16:43:52.863048+0000","last_unstale":"2024-05-30T16:43:53.871024+0000","last_undegraded":"2024-05-30T16:43:53.871024+0000","last_fullsized":"2024-05-30T16:43:53.871024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:41.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874160+0000","last_change":"2024-05-30T16:43:51.854526+0000","last_active":"2024-05-30T16:43:53.874160+0000","last_peered":"2024-05-30T16:43:53.874160+0000","last_clean":"2024-05-30T16:43:53.874160+0000","last_became_active":"2024-05-30T16:43:51.853002+0000","last_became_peered":"2024-05-30T16:43:51.853002+0000","last_unstale":"2024-05-30T16:43:53.874160+0000","last_undegraded":"2024-05-30T16:43:53.874160+0000","last_fullsized":"2024-05-30T16:43:53.874160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:41.467122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225345+0000","last_change":"2024-05-30T16:43:52.876709+0000","last_active":"2024-05-30T16:43:54.225345+0000","last_peered":"2024-05-30T16:43:54.225345+0000","last_clean":"2024-05-30T16:43:54.225345+0000","last_became_active":"2024-05-30T16:43:52.876309+0000","last_became_peered":"2024-05-30T16:43:52.876309+0000","last_unstale":"2024-05-30T16:43:54.225345+0000","last_undegraded":"2024-05-30T16:43:54.225345+0000","last_fullsized":"2024-05-30T16:43:54.225345+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:35.643416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872135+0000","last_change":"2024-05-30T16:43:51.856730+0000","last_active":"2024-05-30T16:43:53.872135+0000","last_peered":"2024-05-30T16:43:53.872135+0000","last_clean":"2024-05-30T16:43:53.872135+0000","last_became_active":"2024-05-30T16:43:51.856409+0000","last_became_peered":"2024-05-30T16:43:51.856409+0000","last_unstale":"2024-05-30T16:43:53.872135+0000","last_undegraded":"2024-05-30T16:43:53.872135+0000","last_fullsized":"2024-05-30T16:43:53.872135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:33:39.805551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874822+0000","last_change":"2024-05-30T16:43:52.895760+0000","last_active":"2024-05-30T16:43:53.874822+0000","last_peered":"2024-05-30T16:43:53.874822+0000","last_clean":"2024-05-30T16:43:53.874822+0000","last_became_active":"2024-05-30T16:43:52.895305+0000","last_became_peered":"2024-05-30T16:43:52.895305+0000","last_unstale":"2024-05-30T16:43:53.874822+0000","last_undegraded":"2024-05-30T16:43:53.874822+0000","last_fullsized":"2024-05-30T16:43:53.874822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:03.535096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.228313+0000","last_change":"2024-05-30T16:43:51.864341+0000","last_active":"2024-05-30T16:43:54.228313+0000","last_peered":"2024-05-30T16:43:54.228313+0000","last_clean":"2024-05-30T16:43:54.228313+0000","last_became_active":"2024-05-30T16:43:51.864017+0000","last_became_peered":"2024-05-30T16:43:51.864017+0000","last_unstale":"2024-05-30T16:43:54.228313+0000","last_undegraded":"2024-05-30T16:43:54.228313+0000","last_fullsized":"2024-05-30T16:43:54.228313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:48.659115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871833+0000","last_change":"2024-05-30T16:43:52.865453+0000","last_active":"2024-05-30T16:43:53.871833+0000","last_peered":"2024-05-30T16:43:53.871833+0000","last_clean":"2024-05-30T16:43:53.871833+0000","last_became_active":"2024-05-30T16:43:52.864999+0000","last_became_peered":"2024-05-30T16:43:52.864999+0000","last_unstale":"2024-05-30T16:43:53.871833+0000","last_undegraded":"2024-05-30T16:43:53.871833+0000","last_fullsized":"2024-05-30T16:43:53.871833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.877105+0000","last_change":"2024-05-30T16:43:51.855757+0000","last_active":"2024-05-30T16:43:53.877105+0000","last_peered":"2024-05-30T16:43:53.877105+0000","last_clean":"2024-05-30T16:43:53.877105+0000","last_became_active":"2024-05-30T16:43:51.855051+0000","last_became_peered":"2024-05-30T16:43:51.855051+0000","last_unstale":"2024-05-30T16:43:53.877105+0000","last_undegraded":"2024-05-30T16:43:53.877105+0000","last_fullsized":"2024-05-30T16:43:53.877105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:34.628416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224825+0000","last_change":"2024-05-30T16:43:52.888339+0000","last_active":"2024-05-30T16:43:54.224825+0000","last_peered":"2024-05-30T16:43:54.224825+0000","last_clean":"2024-05-30T16:43:54.224825+0000","last_became_active":"2024-05-30T16:43:52.888104+0000","last_became_peered":"2024-05-30T16:43:52.888104+0000","last_unstale":"2024-05-30T16:43:54.224825+0000","last_undegraded":"2024-05-30T16:43:54.224825+0000","last_fullsized":"2024-05-30T16:43:54.224825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:02.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873115+0000","last_change":"2024-05-30T16:43:51.865200+0000","last_active":"2024-05-30T16:43:53.873115+0000","last_peered":"2024-05-30T16:43:53.873115+0000","last_clean":"2024-05-30T16:43:53.873115+0000","last_became_active":"2024-05-30T16:43:51.864864+0000","last_became_peered":"2024-05-30T16:43:51.864864+0000","last_unstale":"2024-05-30T16:43:53.873115+0000","last_undegraded":"2024-05-30T16:43:53.873115+0000","last_fullsized":"2024-05-30T16:43:53.873115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:34.628416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157688+0000","last_change":"2024-05-30T16:43:52.875686+0000","last_active":"2024-05-30T16:43:54.157688+0000","last_peered":"2024-05-30T16:43:54.157688+0000","last_clean":"2024-05-30T16:43:54.157688+0000","last_became_active":"2024-05-30T16:43:52.875422+0000","last_became_peered":"2024-05-30T16:43:52.875422+0000","last_unstale":"2024-05-30T16:43:54.157688+0000","last_undegraded":"2024-05-30T16:43:54.157688+0000","last_fullsized":"2024-05-30T16:43:54.157688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:57.617195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224865+0000","last_change":"2024-05-30T16:43:51.863391+0000","last_active":"2024-05-30T16:43:54.224865+0000","last_peered":"2024-05-30T16:43:54.224865+0000","last_clean":"2024-05-30T16:43:54.224865+0000","last_became_active":"2024-05-30T16:43:51.863097+0000","last_became_peered":"2024-05-30T16:43:51.863097+0000","last_unstale":"2024-05-30T16:43:54.224865+0000","last_undegraded":"2024-05-30T16:43:54.224865+0000","last_fullsized":"2024-05-30T16:43:54.224865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:39:40.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225485+0000","last_change":"2024-05-30T16:43:52.875177+0000","last_active":"2024-05-30T16:43:54.225485+0000","last_peered":"2024-05-30T16:43:54.225485+0000","last_clean":"2024-05-30T16:43:54.225485+0000","last_became_active":"2024-05-30T16:43:52.874601+0000","last_became_peered":"2024-05-30T16:43:52.874601+0000","last_unstale":"2024-05-30T16:43:54.225485+0000","last_undegraded":"2024-05-30T16:43:54.225485+0000","last_fullsized":"2024-05-30T16:43:54.225485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:50.920087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.872070+0000","last_change":"2024-05-30T16:43:51.853127+0000","last_active":"2024-05-30T16:43:53.872070+0000","last_peered":"2024-05-30T16:43:53.872070+0000","last_clean":"2024-05-30T16:43:53.872070+0000","last_became_active":"2024-05-30T16:43:51.852860+0000","last_became_peered":"2024-05-30T16:43:51.852860+0000","last_unstale":"2024-05-30T16:43:53.872070+0000","last_undegraded":"2024-05-30T16:43:53.872070+0000","last_fullsized":"2024-05-30T16:43:53.872070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:49.905088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.224426+0000","last_change":"2024-05-30T16:43:52.876127+0000","last_active":"2024-05-30T16:43:54.224426+0000","last_peered":"2024-05-30T16:43:54.224426+0000","last_clean":"2024-05-30T16:43:54.224426+0000","last_became_active":"2024-05-30T16:43:52.875763+0000","last_became_peered":"2024-05-30T16:43:52.875763+0000","last_unstale":"2024-05-30T16:43:54.224426+0000","last_undegraded":"2024-05-30T16:43:54.224426+0000","last_fullsized":"2024-05-30T16:43:54.224426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:49.223539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.871499+0000","last_change":"2024-05-30T16:43:51.860285+0000","last_active":"2024-05-30T16:43:53.871499+0000","last_peered":"2024-05-30T16:43:53.871499+0000","last_clean":"2024-05-30T16:43:53.871499+0000","last_became_active":"2024-05-30T16:43:51.859565+0000","last_became_peered":"2024-05-30T16:43:51.859565+0000","last_unstale":"2024-05-30T16:43:53.871499+0000","last_undegraded":"2024-05-30T16:43:53.871499+0000","last_fullsized":"2024-05-30T16:43:53.871499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:32.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225236+0000","last_change":"2024-05-30T16:43:52.885852+0000","last_active":"2024-05-30T16:43:54.225236+0000","last_peered":"2024-05-30T16:43:54.225236+0000","last_clean":"2024-05-30T16:43:54.225236+0000","last_became_active":"2024-05-30T16:43:52.884228+0000","last_became_peered":"2024-05-30T16:43:52.884228+0000","last_unstale":"2024-05-30T16:43:54.225236+0000","last_undegraded":"2024-05-30T16:43:54.225236+0000","last_fullsized":"2024-05-30T16:43:54.225236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:12:06.956321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.227233+0000","last_change":"2024-05-30T16:43:51.863477+0000","last_active":"2024-05-30T16:43:54.227233+0000","last_peered":"2024-05-30T16:43:54.227233+0000","last_clean":"2024-05-30T16:43:54.227233+0000","last_became_active":"2024-05-30T16:43:51.863178+0000","last_became_peered":"2024-05-30T16:43:51.863178+0000","last_unstale":"2024-05-30T16:43:54.227233+0000","last_undegraded":"2024-05-30T16:43:54.227233+0000","last_fullsized":"2024-05-30T16:43:54.227233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:20:50.371105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.873046+0000","last_change":"2024-05-30T16:43:52.880010+0000","last_active":"2024-05-30T16:43:53.873046+0000","last_peered":"2024-05-30T16:43:53.873046+0000","last_clean":"2024-05-30T16:43:53.873046+0000","last_became_active":"2024-05-30T16:43:52.879744+0000","last_became_peered":"2024-05-30T16:43:52.879744+0000","last_unstale":"2024-05-30T16:43:53.873046+0000","last_undegraded":"2024-05-30T16:43:53.873046+0000","last_fullsized":"2024-05-30T16:43:53.873046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:30.437464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.874603+0000","last_change":"2024-05-30T16:43:52.878768+0000","last_active":"2024-05-30T16:43:53.874603+0000","last_peered":"2024-05-30T16:43:53.874603+0000","last_clean":"2024-05-30T16:43:53.874603+0000","last_became_active":"2024-05-30T16:43:52.878437+0000","last_became_peered":"2024-05-30T16:43:52.878437+0000","last_unstale":"2024-05-30T16:43:53.874603+0000","last_undegraded":"2024-05-30T16:43:53.874603+0000","last_fullsized":"2024-05-30T16:43:53.874603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:13:13.668843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.226899+0000","last_change":"2024-05-30T16:43:51.860527+0000","last_active":"2024-05-30T16:43:54.226899+0000","last_peered":"2024-05-30T16:43:54.226899+0000","last_clean":"2024-05-30T16:43:54.226899+0000","last_became_active":"2024-05-30T16:43:51.860201+0000","last_became_peered":"2024-05-30T16:43:51.860201+0000","last_unstale":"2024-05-30T16:43:54.226899+0000","last_undegraded":"2024-05-30T16:43:54.226899+0000","last_fullsized":"2024-05-30T16:43:54.226899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:44:14.302309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:53.875458+0000","last_change":"2024-05-30T16:43:52.882046+0000","last_active":"2024-05-30T16:43:53.875458+0000","last_peered":"2024-05-30T16:43:53.875458+0000","last_clean":"2024-05-30T16:43:53.875458+0000","last_became_active":"2024-05-30T16:43:52.881777+0000","last_became_peered":"2024-05-30T16:43:52.881777+0000","last_unstale":"2024-05-30T16:43:53.875458+0000","last_undegraded":"2024-05-30T16:43:53.875458+0000","last_fullsized":"2024-05-30T16:43:53.875458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:05:00.764986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.228175+0000","last_change":"2024-05-30T16:43:51.865821+0000","last_active":"2024-05-30T16:43:54.228175+0000","last_peered":"2024-05-30T16:43:54.228175+0000","last_clean":"2024-05-30T16:43:54.228175+0000","last_became_active":"2024-05-30T16:43:51.865511+0000","last_became_peered":"2024-05-30T16:43:51.865511+0000","last_unstale":"2024-05-30T16:43:54.228175+0000","last_undegraded":"2024-05-30T16:43:54.228175+0000","last_fullsized":"2024-05-30T16:43:54.228175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:51.953401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157276+0000","last_change":"2024-05-30T16:43:52.901336+0000","last_active":"2024-05-30T16:43:54.157276+0000","last_peered":"2024-05-30T16:43:54.157276+0000","last_clean":"2024-05-30T16:43:54.157276+0000","last_became_active":"2024-05-30T16:43:52.900847+0000","last_became_peered":"2024-05-30T16:43:52.900847+0000","last_unstale":"2024-05-30T16:43:54.157276+0000","last_undegraded":"2024-05-30T16:43:54.157276+0000","last_fullsized":"2024-05-30T16:43:54.157276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:44.852718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225470+0000","last_change":"2024-05-30T16:43:51.853049+0000","last_active":"2024-05-30T16:43:54.225470+0000","last_peered":"2024-05-30T16:43:54.225470+0000","last_clean":"2024-05-30T16:43:54.225470+0000","last_became_active":"2024-05-30T16:43:51.852719+0000","last_became_peered":"2024-05-30T16:43:51.852719+0000","last_unstale":"2024-05-30T16:43:54.225470+0000","last_undegraded":"2024-05-30T16:43:54.225470+0000","last_fullsized":"2024-05-30T16:43:54.225470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:15.475651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.225448+0000","last_change":"2024-05-30T16:43:52.880495+0000","last_active":"2024-05-30T16:43:54.225448+0000","last_peered":"2024-05-30T16:43:54.225448+0000","last_clean":"2024-05-30T16:43:54.225448+0000","last_became_active":"2024-05-30T16:43:52.879641+0000","last_became_peered":"2024-05-30T16:43:52.879641+0000","last_unstale":"2024-05-30T16:43:54.225448+0000","last_undegraded":"2024-05-30T16:43:54.225448+0000","last_fullsized":"2024-05-30T16:43:54.225448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:51.808032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:51.808032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:43:36.892645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-30T16:43:54.157601+0000","last_change":"2024-05-30T16:43:51.859807+0000","last_active":"2024-05-30T16:43:54.157601+0000","last_peered":"2024-05-30T16:43:54.157601+0000","last_clean":"2024-05-30T16:43:54.157601+0000","last_became_active":"2024-05-30T16:43:51.859531+0000","last_became_peered":"2024-05-30T16:43:51.859531+0000","last_unstale":"2024-05-30T16:43:54.157601+0000","last_undegraded":"2024-05-30T16:43:54.157601+0000","last_fullsized":"2024-05-30T16:43:54.157601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:50.793032+0000","last_clean_scrub_stamp":"2024-05-30T16:43:50.793032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:12.533740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":77338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":77212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":668162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":77302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":668218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:44:03.444 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T16:44:03.444 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-30T16:44:03.444 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-30T16:44:03.444 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-30T16:44:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.691+0000 7fbe53000640 1 Processor -- start 2024-05-30T16:44:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.691+0000 7fbe53000640 1 -- start start 2024-05-30T16:44:03.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.692+0000 7fbe53000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbe4c0cc2b0 0x7fbe4c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.692+0000 7fbe53000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0d09a0 0x7fbe4c0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.692+0000 7fbe53000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe4c1621d0 con 0x7fbe4c161df0 2024-05-30T16:44:03.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.692+0000 7fbe53000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbe4c0d0e10 con 0x7fbe4c0cc2b0 2024-05-30T16:44:03.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.692+0000 7fbe53000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe4c0cd840 con 0x7fbe4c0d09a0 2024-05-30T16:44:03.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe51c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbe4c161df0 0x7fbe4c099800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52376/0 (socket says 172.21.15.67:52376) 2024-05-30T16:44:03.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe51c00640 1 -- 172.21.15.67:0/3183109888 learned_addr learned my addr 172.21.15.67:0/3183109888 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe52600640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0d09a0 0x7fbe4c0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe51200640 1 --2- 172.21.15.67:0/3183109888 >> v2:172.21.15.169:3300/0 conn(0x7fbe4c0cc2b0 0x7fbe4c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020936823 0 0) 0x7fbe3c001110 con 0x7fbe4c161df0 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe34003520 con 0x7fbe4c161df0 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4104877042 0 0) 0x7fbe34003520 con 0x7fbe4c161df0 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0d09a0 msgr2=0x7fbe4c0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe4be00640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0d09a0 0x7fbe4c0cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 >> v2:172.21.15.169:3300/0 conn(0x7fbe4c0cc2b0 msgr2=0x7fbe4c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.693+0000 7fbe4be00640 1 --2- 172.21.15.67:0/3183109888 >> v2:172.21.15.169:3300/0 conn(0x7fbe4c0cc2b0 0x7fbe4c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4c0ccf10 con 0x7fbe4c161df0 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fbe3c001840 con 0x7fbe4c161df0 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 >> v1:172.21.15.67:6789/0 conn(0x7fbe4c161df0 legacy=0x7fbe4c099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe4be00640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe340039b0 0x7fbe34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe4be00640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe34004800 0x7fbe34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe4be00640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0d09a0 0x7fbe340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c0d0e10 con 0x7fbe340039b0 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3400b9c0 con 0x7fbe34004800 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3400bb40 con 0x7fbe4c0d09a0 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe51200640 1 --2- 172.21.15.67:0/3183109888 >> v2:172.21.15.169:3300/0 conn(0x7fbe4c0cc2b0 0x7fbe4c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe51c00640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe340039b0 0x7fbe34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe52600640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0d09a0 0x7fbe340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe51c00640 1 -- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0d09a0 msgr2=0x7fbe340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe51c00640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0d09a0 0x7fbe340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe51200640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe34004800 0x7fbe34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe51c00640 1 -- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe34004800 msgr2=0x7fbe34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe51c00640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe34004800 0x7fbe34004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe51c00640 1 -- 172.21.15.67:0/3183109888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe34003520 con 0x7fbe340039b0 2024-05-30T16:44:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.694+0000 7fbe52600640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0d09a0 0x7fbe340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.695+0000 7fbe51c00640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe340039b0 0x7fbe34003dd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00c3e0 tx=0x7fbe34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5816f267b4aeab05 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:03.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c032020 con 0x7fbe340039b0 2024-05-30T16:44:03.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe3c005ad0 con 0x7fbe340039b0 2024-05-30T16:44:03.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe4be00640 1 -- 172.21.15.67:0/3183109888 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00c0e0 con 0x7fbe340039b0 2024-05-30T16:44:03.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe53000640 1 -- 172.21.15.67:0/3183109888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe340039b0 msgr2=0x7fbe34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe53000640 1 --2- 172.21.15.67:0/3183109888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe340039b0 0x7fbe34003dd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00c3e0 tx=0x7fbe34005830 comp rx=0 tx=0).stop 2024-05-30T16:44:03.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe52600640 1 -- 172.21.15.67:0/3183109888 reap_dead start 2024-05-30T16:44:03.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe53000640 1 -- 172.21.15.67:0/3183109888 shutdown_connections 2024-05-30T16:44:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe53000640 1 -- 172.21.15.67:0/3183109888 >> 172.21.15.67:0/3183109888 conn(0x7fbe4c06bae0 msgr2=0x7fbe4c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe53000640 1 -- 172.21.15.67:0/3183109888 shutdown_connections 2024-05-30T16:44:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.696+0000 7fbe53000640 1 -- 172.21.15.67:0/3183109888 wait complete. 2024-05-30T16:44:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.698+0000 7fbe53000640 1 Processor -- start 2024-05-30T16:44:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.698+0000 7fbe53000640 1 -- start start 2024-05-30T16:44:03.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.699+0000 7fbe53000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe340039b0 0x7fbe4c161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.699+0000 7fbe53000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe4c0af8c0 0x7fbe4c1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.699+0000 7fbe53000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0c83c0 0x7fbe4c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.699+0000 7fbe53000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c0ccbc0 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.699+0000 7fbe53000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c0cd4d0 con 0x7fbe340039b0 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.699+0000 7fbe53000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe4c0ca870 con 0x7fbe4c0c83c0 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.699+0000 7fbe51200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe4c0af8c0 0x7fbe4c1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.699+0000 7fbe51200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe4c0af8c0 0x7fbe4c1544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53174/0 (socket says 172.21.15.67:53174) 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.699+0000 7fbe51200640 1 -- 172.21.15.67:0/462136066 learned_addr learned my addr 172.21.15.67:0/462136066 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe51c00640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe340039b0 0x7fbe4c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe52600640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0c83c0 0x7fbe4c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe51200640 1 -- 172.21.15.67:0/462136066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0c83c0 msgr2=0x7fbe4c0c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe51200640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0c83c0 0x7fbe4c0c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe51200640 1 -- 172.21.15.67:0/462136066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe340039b0 msgr2=0x7fbe4c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe51200640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe340039b0 0x7fbe4c161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe51200640 1 -- 172.21.15.67:0/462136066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4c0b0080 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe51c00640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe340039b0 0x7fbe4c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe51200640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe4c0af8c0 0x7fbe4c1544d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009fb0 tx=0x7fbe4000e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe49600640 1 -- 172.21.15.67:0/462136066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40015610 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe49600640 1 -- 172.21.15.67:0/462136066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4001a070 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe49600640 1 -- 172.21.15.67:0/462136066 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe400159f0 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4c0b0360 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.700+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe4c15ede0 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.702+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe4c0689e0 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.703+0000 7fbe49600640 1 -- 172.21.15.67:0/462136066 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe40012030 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.704+0000 7fbe49600640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe3006c920 0x7fbe3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.704+0000 7fbe49600640 1 -- 172.21.15.67:0/462136066 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbe4008c6f0 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.704+0000 7fbe51c00640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe3006c920 0x7fbe3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.705+0000 7fbe51c00640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe3006c920 0x7fbe3006ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00c3b0 tx=0x7fbe3c009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.707+0000 7fbe49600640 1 -- 172.21.15.67:0/462136066 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe40010040 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.949+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbe4c0af410 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.949+0000 7fbe49600640 1 -- 172.21.15.67:0/462136066 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbe4005ab10 con 0x7fbe4c0af8c0 2024-05-30T16:44:03.950 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:03.950 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe3006c920 msgr2=0x7fbe3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe3006c920 0x7fbe3006ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00c3b0 tx=0x7fbe3c009030 comp rx=0 tx=0).stop 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe4c0af8c0 msgr2=0x7fbe4c1544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe4c0af8c0 0x7fbe4c1544d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009fb0 tx=0x7fbe4000e5d0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 shutdown_connections 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe3006c920 0x7fbe3006ede0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe4c0c83c0 0x7fbe4c0c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe4c0af8c0 0x7fbe4c1544d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 --2- 172.21.15.67:0/462136066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe340039b0 0x7fbe4c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 >> 172.21.15.67:0/462136066 conn(0x7fbe4c06bae0 msgr2=0x7fbe4c0ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:03.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 shutdown_connections 2024-05-30T16:44:03.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:03.952+0000 7fbe53000640 1 -- 172.21.15.67:0/462136066 wait complete. 2024-05-30T16:44:03.969 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-30T16:44:03.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:44:04.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.174+0000 7fb89f200640 1 Processor -- start 2024-05-30T16:44:04.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.174+0000 7fb89f200640 1 -- start start 2024-05-30T16:44:04.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.175+0000 7fb89f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb8a00d16a0 0x7fb8a00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.175+0000 7fb89f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb8a00cf870 0x7fb8a00d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.175+0000 7fb89f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a00cfce0 con 0x7fb8a015e870 2024-05-30T16:44:04.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.175+0000 7fb89f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a00d2150 con 0x7fb8a00cf870 2024-05-30T16:44:04.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.175+0000 7fb89f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a015ec50 con 0x7fb8a00d16a0 2024-05-30T16:44:04.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.176+0000 7fb89e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb8a00cf870 0x7fb8a00d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.176+0000 7fb89e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb8a00cf870 0x7fb8a00d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53188/0 (socket says 172.21.15.67:53188) 2024-05-30T16:44:04.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.176+0000 7fb89e800640 1 -- 172.21.15.67:0/683236352 learned_addr learned my addr 172.21.15.67:0/683236352 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:04.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.176+0000 7fb89d400640 1 --2- 172.21.15.67:0/683236352 >> v2:172.21.15.169:3300/0 conn(0x7fb8a00d16a0 0x7fb8a00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.176+0000 7fb89e800640 1 -- 172.21.15.67:0/683236352 >> v2:172.21.15.169:3300/0 conn(0x7fb8a00d16a0 msgr2=0x7fb8a00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:04.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.176+0000 7fb89e800640 1 --2- 172.21.15.67:0/683236352 >> v2:172.21.15.169:3300/0 conn(0x7fb8a00d16a0 0x7fb8a00cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.176+0000 7fb89e800640 1 -- 172.21.15.67:0/683236352 >> v1:172.21.15.169:6789/0 conn(0x7fb8a015e870 legacy=0x7fb8a00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.176+0000 7fb89e800640 1 -- 172.21.15.67:0/683236352 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a00cff80 con 0x7fb8a00cf870 2024-05-30T16:44:04.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.178+0000 7fb89e800640 1 --2- 172.21.15.67:0/683236352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00cf870 0x7fb8a00d25e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8a008e520 tx=0x7fb89802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52bfcff0130f973 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.178+0000 7fb89ca00640 1 -- 172.21.15.67:0/683236352 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898036070 con 0x7fb8a00cf870 2024-05-30T16:44:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.178+0000 7fb89ca00640 1 -- 172.21.15.67:0/683236352 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb898031600 con 0x7fb8a00cf870 2024-05-30T16:44:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.178+0000 7fb89ca00640 1 -- 172.21.15.67:0/683236352 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8980319d0 con 0x7fb8a00cf870 2024-05-30T16:44:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.179+0000 7fb89f200640 1 -- 172.21.15.67:0/683236352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00cf870 msgr2=0x7fb8a00d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.179+0000 7fb89f200640 1 --2- 172.21.15.67:0/683236352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00cf870 0x7fb8a00d25e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8a008e520 tx=0x7fb89802a730 comp rx=0 tx=0).stop 2024-05-30T16:44:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.179+0000 7fb89f200640 1 -- 172.21.15.67:0/683236352 shutdown_connections 2024-05-30T16:44:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.179+0000 7fb89f200640 1 --2- 172.21.15.67:0/683236352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00cf870 0x7fb8a00d25e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.179+0000 7fb89f200640 1 --2- 172.21.15.67:0/683236352 >> v2:172.21.15.169:3300/0 conn(0x7fb8a00d16a0 0x7fb8a00cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.179+0000 7fb89f200640 1 -- 172.21.15.67:0/683236352 >> 172.21.15.67:0/683236352 conn(0x7fb8a008bfc0 msgr2=0x7fb8a00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.179+0000 7fb89f200640 1 -- 172.21.15.67:0/683236352 shutdown_connections 2024-05-30T16:44:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.179+0000 7fb89f200640 1 -- 172.21.15.67:0/683236352 wait complete. 2024-05-30T16:44:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.181+0000 7fb89f200640 1 Processor -- start 2024-05-30T16:44:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.181+0000 7fb89f200640 1 -- start start 2024-05-30T16:44:04.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.182+0000 7fb89f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00cf870 0x7fb8a0160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.182+0000 7fb89f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00d16a0 0x7fb8a015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.182+0000 7fb89f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a015e870 0x7fb8a015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.182+0000 7fb89f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a00d0820 con 0x7fb8a00d16a0 2024-05-30T16:44:04.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.182+0000 7fb89f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a00d0540 con 0x7fb8a00cf870 2024-05-30T16:44:04.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.182+0000 7fb89f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a00d4fb0 con 0x7fb8a015e870 2024-05-30T16:44:04.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.182+0000 7fb89d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00d16a0 0x7fb8a015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.182+0000 7fb89d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00d16a0 0x7fb8a015ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53200/0 (socket says 172.21.15.67:53200) 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.182+0000 7fb89d400640 1 -- 172.21.15.67:0/2909168615 learned_addr learned my addr 172.21.15.67:0/2909168615 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89e800640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a015e870 0x7fb8a015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89de00640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00cf870 0x7fb8a0160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89d400640 1 -- 172.21.15.67:0/2909168615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a015e870 msgr2=0x7fb8a015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89d400640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a015e870 0x7fb8a015f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89d400640 1 -- 172.21.15.67:0/2909168615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00cf870 msgr2=0x7fb8a0160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89d400640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00cf870 0x7fb8a0160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89d400640 1 -- 172.21.15.67:0/2909168615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a00b71a0 con 0x7fb8a00d16a0 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89de00640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00cf870 0x7fb8a0160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89d400640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00d16a0 0x7fb8a015ee60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb88800e9e0 tx=0x7fb88800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb896000640 1 -- 172.21.15.67:0/2909168615 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888017070 con 0x7fb8a00d16a0 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb896000640 1 -- 172.21.15.67:0/2909168615 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88800c9d0 con 0x7fb8a00d16a0 2024-05-30T16:44:04.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb896000640 1 -- 172.21.15.67:0/2909168615 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88800cda0 con 0x7fb8a00d16a0 2024-05-30T16:44:04.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.183+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a00b7480 con 0x7fb8a00d16a0 2024-05-30T16:44:04.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.184+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a00b7a40 con 0x7fb8a00d16a0 2024-05-30T16:44:04.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.185+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a00689e0 con 0x7fb8a00d16a0 2024-05-30T16:44:04.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.186+0000 7fb896000640 1 -- 172.21.15.67:0/2909168615 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb88801a070 con 0x7fb8a00d16a0 2024-05-30T16:44:04.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.187+0000 7fb896000640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb87c06c8f0 0x7fb87c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.187+0000 7fb896000640 1 -- 172.21.15.67:0/2909168615 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb88808bfe0 con 0x7fb8a00d16a0 2024-05-30T16:44:04.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.187+0000 7fb89de00640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb87c06c8f0 0x7fb87c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.188+0000 7fb89de00640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb87c06c8f0 0x7fb87c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb890001680 tx=0x7fb890009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:04.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.191+0000 7fb896000640 1 -- 172.21.15.67:0/2909168615 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb88805a390 con 0x7fb8a00d16a0 2024-05-30T16:44:04.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.430+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8a00d4830 con 0x7fb8a00d16a0 2024-05-30T16:44:04.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.431+0000 7fb896000640 1 -- 172.21.15.67:0/2909168615 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fb888059c90 con 0x7fb8a00d16a0 2024-05-30T16:44:04.432 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:04.432 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2254297887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2254297887},{"type":"v1","addr":"172.21.15.67:6837","nonce":2254297887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/227511509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":227511509},{"type":"v1","addr":"172.21.15.169:6834","nonce":227511509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:43:52.308425+0000","modified":"2024-05-30T16:43:55.372983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2028878495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2028878495},{"type":"v1","addr":"172.21.15.169:6835","nonce":2028878495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-30T16:44:04.432 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb87c06c8f0 msgr2=0x7fb87c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb87c06c8f0 0x7fb87c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb890001680 tx=0x7fb890009040 comp rx=0 tx=0).stop 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00d16a0 msgr2=0x7fb8a015ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00d16a0 0x7fb8a015ee60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb88800e9e0 tx=0x7fb88800eea0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 shutdown_connections 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb87c06c8f0 0x7fb87c06edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a015e870 0x7fb8a015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8a00d16a0 0x7fb8a015ee60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 --2- 172.21.15.67:0/2909168615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00cf870 0x7fb8a0160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 >> 172.21.15.67:0/2909168615 conn(0x7fb8a008bfc0 msgr2=0x7fb8a00b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:04.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 shutdown_connections 2024-05-30T16:44:04.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.434+0000 7fb89f200640 1 -- 172.21.15.67:0/2909168615 wait complete. 2024-05-30T16:44:04.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:44:04.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.655+0000 7fca9b200640 1 Processor -- start 2024-05-30T16:44:04.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.656+0000 7fca9b200640 1 -- start start 2024-05-30T16:44:04.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.657+0000 7fca9b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fca9c06ee40 0x7fca9c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.657+0000 7fca9b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca9c06f2f0 0x7fca9c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.657+0000 7fca9b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca9c156f10 con 0x7fca9c156b30 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.657+0000 7fca9b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fca9c06dda0 con 0x7fca9c06ee40 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.657+0000 7fca9b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fca9c06e080 con 0x7fca9c06f2f0 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.658+0000 7fca99e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fca9c156b30 0x7fca9c0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52390/0 (socket says 172.21.15.67:52390) 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.658+0000 7fca99e00640 1 -- 172.21.15.67:0/2525274004 learned_addr learned my addr 172.21.15.67:0/2525274004 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.658+0000 7fca99400640 1 --2- 172.21.15.67:0/2525274004 >> v2:172.21.15.67:3300/0 conn(0x7fca9c06ee40 0x7fca9c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.658+0000 7fca9a800640 1 --2- 172.21.15.67:0/2525274004 >> v2:172.21.15.169:3300/0 conn(0x7fca9c06f2f0 0x7fca9c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.658+0000 7fca99400640 1 -- 172.21.15.67:0/2525274004 >> v2:172.21.15.169:3300/0 conn(0x7fca9c06f2f0 msgr2=0x7fca9c0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.658+0000 7fca99400640 1 --2- 172.21.15.67:0/2525274004 >> v2:172.21.15.169:3300/0 conn(0x7fca9c06f2f0 0x7fca9c0ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.658+0000 7fca98a00640 1 -- 172.21.15.67:0/2525274004 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 232365344 0 0) 0x7fca9c156f10 con 0x7fca9c156b30 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.658+0000 7fca99400640 1 -- 172.21.15.67:0/2525274004 >> v1:172.21.15.67:6789/0 conn(0x7fca9c156b30 legacy=0x7fca9c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.658+0000 7fca99400640 1 -- 172.21.15.67:0/2525274004 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c06e360 con 0x7fca9c06ee40 2024-05-30T16:44:04.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.659+0000 7fca99400640 1 --2- 172.21.15.67:0/2525274004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 0x7fca9c0ad230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0d0180 tx=0x7fca8402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37a55847b224b17 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:04.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca98a00640 1 -- 172.21.15.67:0/2525274004 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84037070 con 0x7fca9c06ee40 2024-05-30T16:44:04.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca98a00640 1 -- 172.21.15.67:0/2525274004 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca84032440 con 0x7fca9c06ee40 2024-05-30T16:44:04.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca98a00640 1 -- 172.21.15.67:0/2525274004 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8402ae60 con 0x7fca9c06ee40 2024-05-30T16:44:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca9b200640 1 -- 172.21.15.67:0/2525274004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 msgr2=0x7fca9c0ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca9b200640 1 --2- 172.21.15.67:0/2525274004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 0x7fca9c0ad230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0d0180 tx=0x7fca8402a110 comp rx=0 tx=0).stop 2024-05-30T16:44:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca9b200640 1 -- 172.21.15.67:0/2525274004 shutdown_connections 2024-05-30T16:44:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca9b200640 1 --2- 172.21.15.67:0/2525274004 >> v2:172.21.15.169:3300/0 conn(0x7fca9c06f2f0 0x7fca9c0ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca9b200640 1 --2- 172.21.15.67:0/2525274004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 0x7fca9c0ad230 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca9b200640 1 -- 172.21.15.67:0/2525274004 >> 172.21.15.67:0/2525274004 conn(0x7fca9c0b1fc0 msgr2=0x7fca9c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca9b200640 1 -- 172.21.15.67:0/2525274004 shutdown_connections 2024-05-30T16:44:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.660+0000 7fca9b200640 1 -- 172.21.15.67:0/2525274004 wait complete. 2024-05-30T16:44:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.662+0000 7fca9b200640 1 Processor -- start 2024-05-30T16:44:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.662+0000 7fca9b200640 1 -- start start 2024-05-30T16:44:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.663+0000 7fca9b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 0x7fca9c14c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.663+0000 7fca9b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca9c06f2f0 0x7fca9c152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.663+0000 7fca9b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca9c156b30 0x7fca9c150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.663+0000 7fca9b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c0b2db0 con 0x7fca9c06ee40 2024-05-30T16:44:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.663+0000 7fca9b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c0b2c30 con 0x7fca9c156b30 2024-05-30T16:44:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.663+0000 7fca9b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fca9c0b32c0 con 0x7fca9c06f2f0 2024-05-30T16:44:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.663+0000 7fca99e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 0x7fca9c14c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.663+0000 7fca99e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 0x7fca9c14c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53208/0 (socket says 172.21.15.67:53208) 2024-05-30T16:44:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.663+0000 7fca99e00640 1 -- 172.21.15.67:0/1910250472 learned_addr learned my addr 172.21.15.67:0/1910250472 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca9a800640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca9c156b30 0x7fca9c150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca99e00640 1 -- 172.21.15.67:0/1910250472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca9c06f2f0 msgr2=0x7fca9c152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca99400640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca9c06f2f0 0x7fca9c152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca99e00640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca9c06f2f0 0x7fca9c152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca99e00640 1 -- 172.21.15.67:0/1910250472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca9c156b30 msgr2=0x7fca9c150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca99e00640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca9c156b30 0x7fca9c150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca99e00640 1 -- 172.21.15.67:0/1910250472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c1515f0 con 0x7fca9c06ee40 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca99e00640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 0x7fca9c14c0b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00c430 tx=0x7fca8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca92000640 1 -- 172.21.15.67:0/1910250472 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0074e0 con 0x7fca9c06ee40 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca92000640 1 -- 172.21.15.67:0/1910250472 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca8c0047e0 con 0x7fca9c06ee40 2024-05-30T16:44:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca92000640 1 -- 172.21.15.67:0/1910250472 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004bd0 con 0x7fca9c06ee40 2024-05-30T16:44:04.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.664+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c1518d0 con 0x7fca9c06ee40 2024-05-30T16:44:04.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.665+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c151e90 con 0x7fca9c06ee40 2024-05-30T16:44:04.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.666+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c0689e0 con 0x7fca9c06ee40 2024-05-30T16:44:04.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.667+0000 7fca92000640 1 -- 172.21.15.67:0/1910250472 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca8c011070 con 0x7fca9c06ee40 2024-05-30T16:44:04.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.668+0000 7fca92000640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fca7806c8f0 0x7fca7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:04.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.668+0000 7fca92000640 1 -- 172.21.15.67:0/1910250472 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fca8c08b830 con 0x7fca9c06ee40 2024-05-30T16:44:04.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.668+0000 7fca99400640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fca7806c8f0 0x7fca7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:04.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.669+0000 7fca99400640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fca7806c8f0 0x7fca7806edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca84034000 tx=0x7fca8402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:04.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.672+0000 7fca92000640 1 -- 172.21.15.67:0/1910250472 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca8c059c60 con 0x7fca9c06ee40 2024-05-30T16:44:04.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.913+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca9c0b4350 con 0x7fca9c06ee40 2024-05-30T16:44:04.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.914+0000 7fca92000640 1 -- 172.21.15.67:0/1910250472 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fca8c059560 con 0x7fca9c06ee40 2024-05-30T16:44:04.915 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:04.916 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2254297887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2254297887},{"type":"v1","addr":"172.21.15.67:6837","nonce":2254297887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/227511509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":227511509},{"type":"v1","addr":"172.21.15.169:6834","nonce":227511509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:43:52.308425+0000","modified":"2024-05-30T16:43:55.372983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2028878495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2028878495},{"type":"v1","addr":"172.21.15.169:6835","nonce":2028878495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-30T16:44:04.916 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-05-30T16:44:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fca7806c8f0 msgr2=0x7fca7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fca7806c8f0 0x7fca7806edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca84034000 tx=0x7fca8402e000 comp rx=0 tx=0).stop 2024-05-30T16:44:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 msgr2=0x7fca9c14c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 0x7fca9c14c0b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00c430 tx=0x7fca8c00c8f0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 shutdown_connections 2024-05-30T16:44:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fca7806c8f0 0x7fca7806edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca9c156b30 0x7fca9c150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca9c06f2f0 0x7fca9c152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 --2- 172.21.15.67:0/1910250472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca9c06ee40 0x7fca9c14c0b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 >> 172.21.15.67:0/1910250472 conn(0x7fca9c0b1fc0 msgr2=0x7fca9c0ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 shutdown_connections 2024-05-30T16:44:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:04.917+0000 7fca9b200640 1 -- 172.21.15.67:0/1910250472 wait complete. 2024-05-30T16:44:04.938 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:44:05.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.138+0000 7fee50e00640 1 Processor -- start 2024-05-30T16:44:05.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.138+0000 7fee50e00640 1 -- start start 2024-05-30T16:44:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.139+0000 7fee50e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fee4c0d0590 0x7fee4c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.139+0000 7fee50e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c099090 0x7fee4c0994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.139+0000 7fee50e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee4c0b51f0 con 0x7fee4c0b4e10 2024-05-30T16:44:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.139+0000 7fee50e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fee4c0aef60 con 0x7fee4c0d0590 2024-05-30T16:44:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.139+0000 7fee50e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fee4c0ad1d0 con 0x7fee4c099090 2024-05-30T16:44:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.140+0000 7fee49400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fee4c0d0590 0x7fee4c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.140+0000 7fee4a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c099090 0x7fee4c0994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.140+0000 7fee49e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fee4c0b4e10 0x7fee4c0cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:49756/0 (socket says 172.21.15.67:49756) 2024-05-30T16:44:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.140+0000 7fee49e00640 1 -- 172.21.15.67:0/3789261329 learned_addr learned my addr 172.21.15.67:0/3789261329 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.140+0000 7fee48a00640 1 -- 172.21.15.67:0/3789261329 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2646909404 0 0) 0x7fee4c0b51f0 con 0x7fee4c0b4e10 2024-05-30T16:44:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.140+0000 7fee48a00640 1 -- 172.21.15.67:0/3789261329 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee30005730 con 0x7fee4c0b4e10 2024-05-30T16:44:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.141+0000 7fee49400640 1 -- 172.21.15.67:0/3789261329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c099090 msgr2=0x7fee4c0994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.141+0000 7fee49400640 1 --2- 172.21.15.67:0/3789261329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c099090 0x7fee4c0994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.141+0000 7fee49400640 1 -- 172.21.15.67:0/3789261329 >> v1:172.21.15.169:6789/0 conn(0x7fee4c0b4e10 legacy=0x7fee4c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.141+0000 7fee49400640 1 -- 172.21.15.67:0/3789261329 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c0ad470 con 0x7fee4c0d0590 2024-05-30T16:44:05.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.142+0000 7fee49400640 1 --2- 172.21.15.67:0/3789261329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c0d0590 0x7fee4c0d0a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee340097a0 tx=0x7fee3402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d927b9de199bc368 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.142+0000 7fee48a00640 1 -- 172.21.15.67:0/3789261329 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34037070 con 0x7fee4c0d0590 2024-05-30T16:44:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee48a00640 1 -- 172.21.15.67:0/3789261329 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee3402ae50 con 0x7fee4c0d0590 2024-05-30T16:44:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee48a00640 1 -- 172.21.15.67:0/3789261329 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34032660 con 0x7fee4c0d0590 2024-05-30T16:44:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee50e00640 1 -- 172.21.15.67:0/3789261329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c0d0590 msgr2=0x7fee4c0d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee50e00640 1 --2- 172.21.15.67:0/3789261329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c0d0590 0x7fee4c0d0a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee340097a0 tx=0x7fee3402a100 comp rx=0 tx=0).stop 2024-05-30T16:44:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee50e00640 1 -- 172.21.15.67:0/3789261329 shutdown_connections 2024-05-30T16:44:05.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee50e00640 1 --2- 172.21.15.67:0/3789261329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c099090 0x7fee4c0994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee50e00640 1 --2- 172.21.15.67:0/3789261329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c0d0590 0x7fee4c0d0a20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee50e00640 1 -- 172.21.15.67:0/3789261329 >> 172.21.15.67:0/3789261329 conn(0x7fee4c06bae0 msgr2=0x7fee4c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:05.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee50e00640 1 -- 172.21.15.67:0/3789261329 shutdown_connections 2024-05-30T16:44:05.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.143+0000 7fee50e00640 1 -- 172.21.15.67:0/3789261329 wait complete. 2024-05-30T16:44:05.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.145+0000 7fee50e00640 1 Processor -- start 2024-05-30T16:44:05.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.145+0000 7fee50e00640 1 -- start start 2024-05-30T16:44:05.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.146+0000 7fee50e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c099090 0x7fee4c1f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.146+0000 7fee50e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee4c0b4e10 0x7fee4c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.146+0000 7fee50e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c0d0590 0x7fee4c0c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.146+0000 7fee50e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c0ce880 con 0x7fee4c0b4e10 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.146+0000 7fee50e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c0ce540 con 0x7fee4c099090 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.146+0000 7fee50e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fee4c0cea00 con 0x7fee4c0d0590 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.146+0000 7fee49400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee4c0b4e10 0x7fee4c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.146+0000 7fee49400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee4c0b4e10 0x7fee4c0c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53220/0 (socket says 172.21.15.67:53220) 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.146+0000 7fee49400640 1 -- 172.21.15.67:0/2819562148 learned_addr learned my addr 172.21.15.67:0/2819562148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee49e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c099090 0x7fee4c1f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee49400640 1 -- 172.21.15.67:0/2819562148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c0d0590 msgr2=0x7fee4c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee4a800640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c0d0590 0x7fee4c0c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee49400640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c0d0590 0x7fee4c0c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee49400640 1 -- 172.21.15.67:0/2819562148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c099090 msgr2=0x7fee4c1f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee49400640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c099090 0x7fee4c1f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee49400640 1 -- 172.21.15.67:0/2819562148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c15cc70 con 0x7fee4c0b4e10 2024-05-30T16:44:05.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee49e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c099090 0x7fee4c1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:05.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee49400640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee4c0b4e10 0x7fee4c0c8700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fee34035040 tx=0x7fee340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:05.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee42000640 1 -- 172.21.15.67:0/2819562148 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34041070 con 0x7fee4c0b4e10 2024-05-30T16:44:05.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee42000640 1 -- 172.21.15.67:0/2819562148 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee3403c460 con 0x7fee4c0b4e10 2024-05-30T16:44:05.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee42000640 1 -- 172.21.15.67:0/2819562148 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34037040 con 0x7fee4c0b4e10 2024-05-30T16:44:05.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.147+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c0c7f60 con 0x7fee4c0b4e10 2024-05-30T16:44:05.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.148+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c0c8240 con 0x7fee4c0b4e10 2024-05-30T16:44:05.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.149+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee4c06f210 con 0x7fee4c0b4e10 2024-05-30T16:44:05.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.150+0000 7fee42000640 1 -- 172.21.15.67:0/2819562148 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee3403a070 con 0x7fee4c0b4e10 2024-05-30T16:44:05.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.150+0000 7fee42000640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee2806c8f0 0x7fee2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.151+0000 7fee42000640 1 -- 172.21.15.67:0/2819562148 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fee340abd00 con 0x7fee4c0b4e10 2024-05-30T16:44:05.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.151+0000 7fee49e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee2806c8f0 0x7fee2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.152+0000 7fee49e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee2806c8f0 0x7fee2806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0011e0 tx=0x7fee3c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:05.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.155+0000 7fee42000640 1 -- 172.21.15.67:0/2819562148 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee4c06f210 con 0x7fee4c0b4e10 2024-05-30T16:44:05.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.362+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee4c0689e0 con 0x7fee4c0b4e10 2024-05-30T16:44:05.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.365+0000 7fee42000640 1 -- 172.21.15.67:0/2819562148 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7fee3407a120 con 0x7fee4c0b4e10 2024-05-30T16:44:05.366 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:05.367 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:53.858729+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:43:50.496509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-30T16:43:51.276872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:44:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee2806c8f0 msgr2=0x7fee2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee2806c8f0 0x7fee2806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0011e0 tx=0x7fee3c009040 comp rx=0 tx=0).stop 2024-05-30T16:44:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee4c0b4e10 msgr2=0x7fee4c0c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee4c0b4e10 0x7fee4c0c8700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fee34035040 tx=0x7fee340046c0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 shutdown_connections 2024-05-30T16:44:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee2806c8f0 0x7fee2806edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee4c0d0590 0x7fee4c0c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee4c0b4e10 0x7fee4c0c8700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 --2- 172.21.15.67:0/2819562148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee4c099090 0x7fee4c1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.368+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 >> 172.21.15.67:0/2819562148 conn(0x7fee4c06bae0 msgr2=0x7fee4c0cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.369+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 shutdown_connections 2024-05-30T16:44:05.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.369+0000 7fee50e00640 1 -- 172.21.15.67:0/2819562148 wait complete. 2024-05-30T16:44:05.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:44:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.590+0000 7f1302e00640 1 Processor -- start 2024-05-30T16:44:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.590+0000 7f1302e00640 1 -- start start 2024-05-30T16:44:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.592+0000 7f1302e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc156b30 0x7f12fc0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.592+0000 7f1302e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f12fc06ee40 0x7f12fc0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.592+0000 7f1302e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12fc06f2c0 0x7f12fc0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.592+0000 7f1302e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f12fc156f10 con 0x7f12fc06f2c0 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.592+0000 7f1302e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f12fc06dda0 con 0x7f12fc06ee40 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.592+0000 7f1302e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc06e080 con 0x7f12fc156b30 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.592+0000 7f1302400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12fc06f2c0 0x7f12fc0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.592+0000 7f1302400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12fc06f2c0 0x7f12fc0ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53222/0 (socket says 172.21.15.67:53222) 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.592+0000 7f1302400640 1 -- 172.21.15.67:0/1528374230 learned_addr learned my addr 172.21.15.67:0/1528374230 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.593+0000 7f1301000640 1 --2- 172.21.15.67:0/1528374230 >> v2:172.21.15.169:3300/0 conn(0x7f12fc06ee40 0x7f12fc0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.593+0000 7f1301a00640 1 --2- 172.21.15.67:0/1528374230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc156b30 0x7f12fc0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.593+0000 7f1302400640 1 -- 172.21.15.67:0/1528374230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc156b30 msgr2=0x7f12fc0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.593+0000 7f1302400640 1 --2- 172.21.15.67:0/1528374230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc156b30 0x7f12fc0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.593+0000 7f1302400640 1 -- 172.21.15.67:0/1528374230 >> v2:172.21.15.169:3300/0 conn(0x7f12fc06ee40 msgr2=0x7f12fc0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.593+0000 7f1302400640 1 --2- 172.21.15.67:0/1528374230 >> v2:172.21.15.169:3300/0 conn(0x7f12fc06ee40 0x7f12fc0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.593+0000 7f1302400640 1 -- 172.21.15.67:0/1528374230 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc06e360 con 0x7f12fc06f2c0 2024-05-30T16:44:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.593+0000 7f1301000640 1 --2- 172.21.15.67:0/1528374230 >> v2:172.21.15.169:3300/0 conn(0x7f12fc06ee40 0x7f12fc0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.594+0000 7f1302400640 1 --2- 172.21.15.67:0/1528374230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 0x7f12fc0ad770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f12fc06c860 tx=0x7f12f402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6542c7a4726a24 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.595+0000 7f12fbe00640 1 -- 172.21.15.67:0/1528374230 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4036070 con 0x7f12fc06f2c0 2024-05-30T16:44:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.595+0000 7f12fbe00640 1 -- 172.21.15.67:0/1528374230 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f4031600 con 0x7f12fc06f2c0 2024-05-30T16:44:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.595+0000 7f12fbe00640 1 -- 172.21.15.67:0/1528374230 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f40319d0 con 0x7f12fc06f2c0 2024-05-30T16:44:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.595+0000 7f1302e00640 1 -- 172.21.15.67:0/1528374230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 msgr2=0x7f12fc0ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.595+0000 7f1302e00640 1 --2- 172.21.15.67:0/1528374230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 0x7f12fc0ad770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f12fc06c860 tx=0x7f12f402a610 comp rx=0 tx=0).stop 2024-05-30T16:44:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.596+0000 7f1302e00640 1 -- 172.21.15.67:0/1528374230 shutdown_connections 2024-05-30T16:44:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.596+0000 7f1302e00640 1 --2- 172.21.15.67:0/1528374230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 0x7f12fc0ad770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.596+0000 7f1302e00640 1 --2- 172.21.15.67:0/1528374230 >> v2:172.21.15.169:3300/0 conn(0x7f12fc06ee40 0x7f12fc0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.596+0000 7f1302e00640 1 --2- 172.21.15.67:0/1528374230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc156b30 0x7f12fc0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.596+0000 7f1302e00640 1 -- 172.21.15.67:0/1528374230 >> 172.21.15.67:0/1528374230 conn(0x7f12fc0b1fc0 msgr2=0x7f12fc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.596+0000 7f1302e00640 1 -- 172.21.15.67:0/1528374230 shutdown_connections 2024-05-30T16:44:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.596+0000 7f1302e00640 1 -- 172.21.15.67:0/1528374230 wait complete. 2024-05-30T16:44:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.597+0000 7f1302e00640 1 Processor -- start 2024-05-30T16:44:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.598+0000 7f1302e00640 1 -- start start 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.598+0000 7f1302e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc06ee40 0x7f12fc147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1302e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 0x7f12fc157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1302e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12fc156b30 0x7f12fc146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1302e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc0afc20 con 0x7f12fc06f2c0 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1302e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc0b2c30 con 0x7f12fc156b30 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1302e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc158a50 con 0x7f12fc06ee40 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1301000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 0x7f12fc157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1301000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 0x7f12fc157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53236/0 (socket says 172.21.15.67:53236) 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1301000640 1 -- 172.21.15.67:0/2109956754 learned_addr learned my addr 172.21.15.67:0/2109956754 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1301a00640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc06ee40 0x7f12fc147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1302400640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12fc156b30 0x7f12fc146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1301000640 1 -- 172.21.15.67:0/2109956754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc06ee40 msgr2=0x7f12fc147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1301000640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc06ee40 0x7f12fc147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1301000640 1 -- 172.21.15.67:0/2109956754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12fc156b30 msgr2=0x7f12fc146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1301000640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12fc156b30 0x7f12fc146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1301000640 1 -- 172.21.15.67:0/2109956754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc1581f0 con 0x7f12fc06f2c0 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.599+0000 7f1302400640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12fc156b30 0x7f12fc146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.600+0000 7f1301000640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 0x7f12fc157aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12e800d800 tx=0x7f12e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.600+0000 7f12f9600640 1 -- 172.21.15.67:0/2109956754 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8018070 con 0x7f12fc06f2c0 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.600+0000 7f12f9600640 1 -- 172.21.15.67:0/2109956754 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e8004030 con 0x7f12fc06f2c0 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.600+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12fc158ca0 con 0x7f12fc06f2c0 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.600+0000 7f12f9600640 1 -- 172.21.15.67:0/2109956754 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8004400 con 0x7f12fc06f2c0 2024-05-30T16:44:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.600+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12fc159260 con 0x7f12fc06f2c0 2024-05-30T16:44:05.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.601+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12fc0689e0 con 0x7f12fc06f2c0 2024-05-30T16:44:05.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.602+0000 7f12f9600640 1 -- 172.21.15.67:0/2109956754 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12e8009640 con 0x7f12fc06f2c0 2024-05-30T16:44:05.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.603+0000 7f12f9600640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12dc06c8f0 0x7f12dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:05.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.603+0000 7f12f9600640 1 -- 172.21.15.67:0/2109956754 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f12e808bee0 con 0x7f12fc06f2c0 2024-05-30T16:44:05.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.603+0000 7f1301a00640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12dc06c8f0 0x7f12dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:05.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.604+0000 7f1301a00640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12dc06c8f0 0x7f12dc06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006fd0 tx=0x7f12f0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:05.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.607+0000 7f12f9600640 1 -- 172.21.15.67:0/2109956754 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12e805a300 con 0x7f12fc06f2c0 2024-05-30T16:44:05.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.849+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12fc073bb0 con 0x7f12fc06f2c0 2024-05-30T16:44:05.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.850+0000 7f12f9600640 1 -- 172.21.15.67:0/2109956754 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f12e8059c00 con 0x7f12fc06f2c0 2024-05-30T16:44:05.851 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:05.852 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2254297887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2254297887},{"type":"v1","addr":"172.21.15.67:6837","nonce":2254297887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/227511509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":227511509},{"type":"v1","addr":"172.21.15.169:6834","nonce":227511509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:43:52.308425+0000","modified":"2024-05-30T16:43:55.372983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2028878495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2028878495},{"type":"v1","addr":"172.21.15.169:6835","nonce":2028878495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-30T16:44:05.852 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-05-30T16:44:05.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12dc06c8f0 msgr2=0x7f12dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12dc06c8f0 0x7f12dc06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006fd0 tx=0x7f12f0008040 comp rx=0 tx=0).stop 2024-05-30T16:44:05.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 msgr2=0x7f12fc157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:05.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 0x7f12fc157aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12e800d800 tx=0x7f12e800dcc0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 shutdown_connections 2024-05-30T16:44:05.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12dc06c8f0 0x7f12dc06edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12fc156b30 0x7f12fc146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12fc06f2c0 0x7f12fc157aa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 --2- 172.21.15.67:0/2109956754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12fc06ee40 0x7f12fc147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:05.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 >> 172.21.15.67:0/2109956754 conn(0x7f12fc0b1fc0 msgr2=0x7f12fc0ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:05.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 shutdown_connections 2024-05-30T16:44:05.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:05.853+0000 7f1302e00640 1 -- 172.21.15.67:0/2109956754 wait complete. 2024-05-30T16:44:06.277 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-30T16:43:52.308425+0000', 'modified': '2024-05-30T16:43:55.372983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/2028878495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6833', 'nonce': 2028878495}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2028878495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4296, 'qdb_cluster': [4296]} 2024-05-30T16:44:06.277 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-30T16:44:06.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-30T16:44:06.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.499+0000 7f00be400640 1 Processor -- start 2024-05-30T16:44:06.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.499+0000 7f00be400640 1 -- start start 2024-05-30T16:44:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.501+0000 7f00be400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f00b8161fb0 0x7f00b809b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.501+0000 7f00be400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00b80b15e0 0x7f00b80d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.501+0000 7f00be400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b80d0630 con 0x7f00b809b980 2024-05-30T16:44:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.501+0000 7f00be400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f00b80d0400 con 0x7f00b8161fb0 2024-05-30T16:44:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.501+0000 7f00be400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f00b8162390 con 0x7f00b80b15e0 2024-05-30T16:44:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.501+0000 7f00bd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f00b8161fb0 0x7f00b809b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.501+0000 7f00bd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f00b8161fb0 0x7f00b809b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53244/0 (socket says 172.21.15.67:53244) 2024-05-30T16:44:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.501+0000 7f00bd000640 1 -- 172.21.15.67:0/3138917605 learned_addr learned my addr 172.21.15.67:0/3138917605 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.501+0000 7f00bda00640 1 --2- 172.21.15.67:0/3138917605 >> v2:172.21.15.169:3300/0 conn(0x7f00b80b15e0 0x7f00b80d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.502+0000 7f00bd000640 1 -- 172.21.15.67:0/3138917605 >> v2:172.21.15.169:3300/0 conn(0x7f00b80b15e0 msgr2=0x7f00b80d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.502+0000 7f00bd000640 1 --2- 172.21.15.67:0/3138917605 >> v2:172.21.15.169:3300/0 conn(0x7f00b80b15e0 0x7f00b80d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.502+0000 7f00bd000640 1 -- 172.21.15.67:0/3138917605 >> v1:172.21.15.169:6789/0 conn(0x7f00b809b980 legacy=0x7f00b80b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.502+0000 7f00bd000640 1 -- 172.21.15.67:0/3138917605 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b80d0910 con 0x7f00b8161fb0 2024-05-30T16:44:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.502+0000 7f00bda00640 1 --2- 172.21.15.67:0/3138917605 >> v2:172.21.15.169:3300/0 conn(0x7f00b80b15e0 0x7f00b80d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:06.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.503+0000 7f00bd000640 1 --2- 172.21.15.67:0/3138917605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b8161fb0 0x7f00b809b440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f00b8163640 tx=0x7f00ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57122b23f7554451 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:06.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.503+0000 7f00b7400640 1 -- 172.21.15.67:0/3138917605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac037070 con 0x7f00b8161fb0 2024-05-30T16:44:06.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.503+0000 7f00b7400640 1 -- 172.21.15.67:0/3138917605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ac032440 con 0x7f00b8161fb0 2024-05-30T16:44:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.503+0000 7f00b7400640 1 -- 172.21.15.67:0/3138917605 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac032810 con 0x7f00b8161fb0 2024-05-30T16:44:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.504+0000 7f00be400640 1 -- 172.21.15.67:0/3138917605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b8161fb0 msgr2=0x7f00b809b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.504+0000 7f00be400640 1 --2- 172.21.15.67:0/3138917605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b8161fb0 0x7f00b809b440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f00b8163640 tx=0x7f00ac02a230 comp rx=0 tx=0).stop 2024-05-30T16:44:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.504+0000 7f00be400640 1 -- 172.21.15.67:0/3138917605 shutdown_connections 2024-05-30T16:44:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.504+0000 7f00be400640 1 --2- 172.21.15.67:0/3138917605 >> v2:172.21.15.169:3300/0 conn(0x7f00b80b15e0 0x7f00b80d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.504+0000 7f00be400640 1 --2- 172.21.15.67:0/3138917605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b8161fb0 0x7f00b809b440 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.504+0000 7f00be400640 1 -- 172.21.15.67:0/3138917605 >> 172.21.15.67:0/3138917605 conn(0x7f00b80d23b0 msgr2=0x7f00b80d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.504+0000 7f00be400640 1 -- 172.21.15.67:0/3138917605 shutdown_connections 2024-05-30T16:44:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.504+0000 7f00be400640 1 -- 172.21.15.67:0/3138917605 wait complete. 2024-05-30T16:44:06.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.506+0000 7f00be400640 1 Processor -- start 2024-05-30T16:44:06.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.506+0000 7f00be400640 1 -- start start 2024-05-30T16:44:06.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00be400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00b809b980 0x7f00b814fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00be400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b80b15e0 0x7f00b80afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00be400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00b8161fb0 0x7f00b80b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00be400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00b80d0f60 con 0x7f00b80b15e0 2024-05-30T16:44:06.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00be400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00b809be20 con 0x7f00b8161fb0 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00be400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f00b80b2120 con 0x7f00b809b980 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b80b15e0 0x7f00b80afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b80b15e0 0x7f00b80afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53250/0 (socket says 172.21.15.67:53250) 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00b7e00640 1 -- 172.21.15.67:0/2057350532 learned_addr learned my addr 172.21.15.67:0/2057350532 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00bd000640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00b809b980 0x7f00b814fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00b7e00640 1 -- 172.21.15.67:0/2057350532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00b809b980 msgr2=0x7f00b814fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00b7e00640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00b809b980 0x7f00b814fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00b7e00640 1 -- 172.21.15.67:0/2057350532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00b8161fb0 msgr2=0x7f00b80b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.507+0000 7f00b7e00640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00b8161fb0 0x7f00b80b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.508+0000 7f00b7e00640 1 -- 172.21.15.67:0/2057350532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b81f81e0 con 0x7f00b80b15e0 2024-05-30T16:44:06.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.508+0000 7f00b7e00640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b80b15e0 0x7f00b80afd00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c430 tx=0x7f00a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:06.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.508+0000 7f00b4c00640 1 -- 172.21.15.67:0/2057350532 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4011790 con 0x7f00b80b15e0 2024-05-30T16:44:06.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.508+0000 7f00b4c00640 1 -- 172.21.15.67:0/2057350532 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a4017070 con 0x7f00b80b15e0 2024-05-30T16:44:06.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.508+0000 7f00b4c00640 1 -- 172.21.15.67:0/2057350532 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4011bb0 con 0x7f00b80b15e0 2024-05-30T16:44:06.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.508+0000 7f00be400640 1 -- 172.21.15.67:0/2057350532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b81f8480 con 0x7f00b80b15e0 2024-05-30T16:44:06.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.508+0000 7f00be400640 1 -- 172.21.15.67:0/2057350532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b81f8a40 con 0x7f00b80b15e0 2024-05-30T16:44:06.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.511+0000 7f00b4c00640 1 -- 172.21.15.67:0/2057350532 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00a401c460 con 0x7f00b80b15e0 2024-05-30T16:44:06.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.512+0000 7f00b4c00640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f009806c8f0 0x7f009806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.512+0000 7f00b4c00640 1 -- 172.21.15.67:0/2057350532 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f00a408ba80 con 0x7f00b80b15e0 2024-05-30T16:44:06.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.512+0000 7f00bd000640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f009806c8f0 0x7f009806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.513+0000 7f00c281e740 1 Processor -- start 2024-05-30T16:44:06.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.513+0000 7f00bd000640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f009806c8f0 0x7f009806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00ac034000 tx=0x7f00ac02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:06.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.513+0000 7f00c281e740 1 -- start start 2024-05-30T16:44:06.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.514+0000 7f00c281e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b91a490 0x55c39b91dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.514+0000 7f00c281e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b93a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.514+0000 7f00c281e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b93a5d0 0x55c39b93aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.514+0000 7f00c281e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c39b857b10 con 0x55c39b91e150 2024-05-30T16:44:06.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.514+0000 7f00c281e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c39b7fe2b0 con 0x55c39b91a490 2024-05-30T16:44:06.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.514+0000 7f00c281e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c39b853ad0 con 0x55c39b93a5d0 2024-05-30T16:44:06.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.514+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b93a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.514+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b93a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53254/0 (socket says 172.21.15.67:53254) 2024-05-30T16:44:06.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.514+0000 7f00b7e00640 1 -- 172.21.15.67:0/785957996 learned_addr learned my addr 172.21.15.67:0/785957996 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:06.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00bd000640 1 --2- 172.21.15.67:0/785957996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b91a490 0x55c39b91dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00b7e00640 1 -- 172.21.15.67:0/785957996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b93a5d0 msgr2=0x55c39b93aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00b7e00640 1 --2- 172.21.15.67:0/785957996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b93a5d0 0x55c39b93aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00b7e00640 1 -- 172.21.15.67:0/785957996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b91a490 msgr2=0x55c39b91dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00b7e00640 1 --2- 172.21.15.67:0/785957996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b91a490 0x55c39b91dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00b7e00640 1 -- 172.21.15.67:0/785957996 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39b56a2d0 con 0x55c39b91e150 2024-05-30T16:44:06.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00b7e00640 1 --2- 172.21.15.67:0/785957996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b93a090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f00a4055c00 tx=0x7f00a4059f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa847a4814e0ece9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00a9600640 1 -- 172.21.15.67:0/785957996 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4062380 con 0x55c39b91e150 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00a9600640 1 -- 172.21.15.67:0/785957996 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a4057aa0 con 0x55c39b91e150 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.515+0000 7f00a9600640 1 -- 172.21.15.67:0/785957996 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4055e10 con 0x55c39b91e150 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.516+0000 7f00c281e740 1 -- 172.21.15.67:0/785957996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 msgr2=0x55c39b93a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.516+0000 7f00c281e740 1 --2- 172.21.15.67:0/785957996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b93a090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f00a4055c00 tx=0x7f00a4059f60 comp rx=0 tx=0).stop 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.516+0000 7f00c281e740 1 -- 172.21.15.67:0/785957996 shutdown_connections 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.516+0000 7f00c281e740 1 --2- 172.21.15.67:0/785957996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b93a5d0 0x55c39b93aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.516+0000 7f00c281e740 1 --2- 172.21.15.67:0/785957996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b93a090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.516+0000 7f00c281e740 1 --2- 172.21.15.67:0/785957996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b91a490 0x55c39b91dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.516+0000 7f00c281e740 1 -- 172.21.15.67:0/785957996 >> 172.21.15.67:0/785957996 conn(0x55c39b8f0470 msgr2=0x55c39b8f1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.516+0000 7f00c281e740 1 -- 172.21.15.67:0/785957996 shutdown_connections 2024-05-30T16:44:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.516+0000 7f00c281e740 1 -- 172.21.15.67:0/785957996 wait complete. 2024-05-30T16:44:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.517+0000 7f00c281e740 1 -- start start 2024-05-30T16:44:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.517+0000 7f00c281e740 1 Processor -- start 2024-05-30T16:44:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00c281e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 0x55c39b9160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00c281e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 0x55c39b9208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00c281e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b9173b0 0x55c39b920de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00c281e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c39b8ee2a0 con 0x55c39b9173b0 2024-05-30T16:44:06.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00c281e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c39b811570 con 0x55c39b93a5d0 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00c281e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c39b8ee580 con 0x55c39b91e150 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00c281e740 10 client.0 _ll_register_callbacks cb 0x55c39b7dffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00bda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b9173b0 0x55c39b920de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00bda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b9173b0 0x55c39b920de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53264/0 (socket says 172.21.15.67:53264) 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.518+0000 7f00bda00640 1 -- 172.21.15.67:0/2145110114 learned_addr learned my addr 172.21.15.67:0/2145110114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f00bd000640 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 0x55c39b9160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f00b7e00640 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 0x55c39b9208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f00bda00640 1 -- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 msgr2=0x55c39b9160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f00bda00640 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 0x55c39b9160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f00bda00640 1 -- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 msgr2=0x55c39b9208a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f00bda00640 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 0x55c39b9208a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f00bda00640 1 -- 172.21.15.67:0/2145110114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39b6346b0 con 0x55c39b9173b0 2024-05-30T16:44:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f00bda00640 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b9173b0 0x55c39b920de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00b000cac0 tx=0x7f00b000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ee65aaf4f3a26b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f009e000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f009e000640 1 -- 172.21.15.67:0/2145110114 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b001a060 con 0x55c39b9173b0 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f009e000640 1 -- 172.21.15.67:0/2145110114 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f00b000adf0 con 0x55c39b9173b0 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.519+0000 7f00c281e740 1 -- 172.21.15.67:0/2145110114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a4057aa0 con 0x55c39b9173b0 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f009e000640 1 -- 172.21.15.67:0/2145110114 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f00b0015440 con 0x55c39b9173b0 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f00c281e740 10 client.4584 fetch_fsmap learned FSMap version 8 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f00c281e740 1 -- 172.21.15.67:0/2145110114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55c39b917860 con 0x55c39b9173b0 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f009e000640 1 -- 172.21.15.67:0/2145110114 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a4057aa0 con 0x55c39b9173b0 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f009e000640 1 -- 172.21.15.67:0/2145110114 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f00b0023020 con 0x55c39b9173b0 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f00c281e740 10 client.4584 fetch_fsmap finished waiting for FSMap version 8 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f00c281e740 10 client.4584 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f00c281e740 20 client.4584 populate_metadata read hostname 'smithi067' 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f00c281e740 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] conn(0x55c39b935d30 0x55c39b956ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f00c281e740 4 client.4584 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-30T16:44:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.520+0000 7f00c281e740 1 -- 172.21.15.67:0/2145110114 --> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c39b865530 con 0x55c39b935d30 2024-05-30T16:44:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.521+0000 7f00bd000640 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] conn(0x55c39b935d30 0x55c39b956ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.522+0000 7f00bd000640 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] conn(0x55c39b935d30 0x55c39b956ad0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d58ad1ddc39cb05 server_cookie=12dc28c79c82352a in_seq=0 out_seq=0 2024-05-30T16:44:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.522+0000 7f009e000640 10 client.4584 ms_handle_connect on v2:172.21.15.169:6833/2028878495 2024-05-30T16:44:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.524+0000 7f009e000640 1 -- 172.21.15.67:0/2145110114 <== mds.0 v2:172.21.15.169:6833/2028878495 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c39b865530 con 0x55c39b935d30 2024-05-30T16:44:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.524+0000 7f009e000640 10 client.4584 handle_command_reply: tid=0 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.524+0000 7f00c281e740 1 client.4584 shutdown 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 -- 172.21.15.67:0/2145110114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b9173b0 msgr2=0x55c39b920de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b9173b0 0x55c39b920de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00b000cac0 tx=0x7f00b000cf80 comp rx=0 tx=0).stop 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 -- 172.21.15.67:0/2145110114 shutdown_connections 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] conn(0x55c39b935d30 0x55c39b956ad0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b9173b0 0x55c39b920de0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 0x55c39b9208a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 --2- 172.21.15.67:0/2145110114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 0x55c39b9160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f009e000640 0 client.4584 ms_handle_reset on v2:172.21.15.169:6833/2028878495 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 -- 172.21.15.67:0/2145110114 >> 172.21.15.67:0/2145110114 conn(0x55c39b91a490 msgr2=0x55c39b8f3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 -- 172.21.15.67:0/2145110114 shutdown_connections 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 1 -- 172.21.15.67:0/2145110114 wait complete. 2024-05-30T16:44:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.525+0000 7f00c281e740 20 client.4584 trim_cache size 0 max 16384 2024-05-30T16:44:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.545+0000 7f00c281e740 1 Processor -- start 2024-05-30T16:44:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.545+0000 7f00c281e740 1 -- start start 2024-05-30T16:44:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.546+0000 7f00c281e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b9173b0 0x55c39b9213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.546+0000 7f00c281e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b921910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.546+0000 7f00c281e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.546+0000 7f00c281e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81ffbc0 con 0x55c39b91e150 2024-05-30T16:44:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.546+0000 7f00c281e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00b80c6b90 con 0x55c39b93a5d0 2024-05-30T16:44:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.546+0000 7f00c281e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f00b81ff570 con 0x55c39b9173b0 2024-05-30T16:44:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.546+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b921910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.546+0000 7f00b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b921910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53274/0 (socket says 172.21.15.67:53274) 2024-05-30T16:44:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.546+0000 7f00b7e00640 1 -- 172.21.15.67:0/690595991 learned_addr learned my addr 172.21.15.67:0/690595991 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00bda00640 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00bd000640 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b9173b0 0x55c39b9213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00b7e00640 1 -- 172.21.15.67:0/690595991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b9173b0 msgr2=0x55c39b9213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00b7e00640 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b9173b0 0x55c39b9213d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00b7e00640 1 -- 172.21.15.67:0/690595991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 msgr2=0x55c39b8ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00b7e00640 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00b7e00640 1 -- 172.21.15.67:0/690595991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39b93a400 con 0x55c39b91e150 2024-05-30T16:44:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00bd000640 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b9173b0 0x55c39b9213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00b7e00640 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b921910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f00a405a050 tx=0x7f00a401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797c23e2a4414d7a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00a8c00640 1 -- 172.21.15.67:0/690595991 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4057aa0 con 0x55c39b91e150 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00a8c00640 1 -- 172.21.15.67:0/690595991 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a4011d50 con 0x55c39b91e150 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.547+0000 7f00a8c00640 1 -- 172.21.15.67:0/690595991 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4061ef0 con 0x55c39b91e150 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.548+0000 7f00c281e740 1 -- 172.21.15.67:0/690595991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 msgr2=0x55c39b921910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.548+0000 7f00c281e740 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b921910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f00a405a050 tx=0x7f00a401e000 comp rx=0 tx=0).stop 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.548+0000 7f00c281e740 1 -- 172.21.15.67:0/690595991 shutdown_connections 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.548+0000 7f00c281e740 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.548+0000 7f00c281e740 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b91e150 0x55c39b921910 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.548+0000 7f00c281e740 1 --2- 172.21.15.67:0/690595991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b9173b0 0x55c39b9213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.548+0000 7f00c281e740 1 -- 172.21.15.67:0/690595991 >> 172.21.15.67:0/690595991 conn(0x55c39b91a490 msgr2=0x55c39b916bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.548+0000 7f00c281e740 1 -- 172.21.15.67:0/690595991 shutdown_connections 2024-05-30T16:44:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.548+0000 7f00c281e740 1 -- 172.21.15.67:0/690595991 wait complete. 2024-05-30T16:44:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.549+0000 7f00c281e740 1 -- start start 2024-05-30T16:44:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.549+0000 7f00c281e740 1 Processor -- start 2024-05-30T16:44:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.550+0000 7f00c281e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b9173b0 0x55c39b9b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.550+0000 7f00c281e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 0x55c39b8eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.550+0000 7f00c281e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.550+0000 7f00c281e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c39b9202f0 con 0x55c39b93a5d0 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.550+0000 7f00c281e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c39b91ff30 con 0x55c39b9173b0 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.550+0000 7f00c281e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c39b936180 con 0x55c39b91e150 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.550+0000 7f00c281e740 10 client.0 _ll_register_callbacks cb 0x7f00b000c5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53290/0 (socket says 172.21.15.67:53290) 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bda00640 1 -- 172.21.15.67:0/1519728571 learned_addr learned my addr 172.21.15.67:0/1519728571 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bd000640 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b9173b0 0x55c39b9b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00b7e00640 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 0x55c39b8eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bda00640 1 -- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 msgr2=0x55c39b8eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bda00640 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 0x55c39b8eefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bda00640 1 -- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b9173b0 msgr2=0x55c39b9b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bda00640 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b9173b0 0x55c39b9b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bda00640 1 -- 172.21.15.67:0/1519728571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39b91db50 con 0x55c39b93a5d0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bd000640 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b9173b0 0x55c39b9b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f00bda00640 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55c39b94bea0 tx=0x7f00b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7533550b255e3ef server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.551+0000 7f009e000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f009e000640 1 -- 172.21.15.67:0/1519728571 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0015b30 con 0x55c39b93a5d0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f009e000640 1 -- 172.21.15.67:0/1519728571 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f00b0015cd0 con 0x55c39b93a5d0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f00c281e740 1 -- 172.21.15.67:0/1519728571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a4011d50 con 0x55c39b93a5d0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f009e000640 1 -- 172.21.15.67:0/1519728571 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f00b0033050 con 0x55c39b93a5d0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f00c281e740 10 client.4590 fetch_fsmap learned FSMap version 8 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f00c281e740 1 -- 172.21.15.67:0/1519728571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55c39b950120 con 0x55c39b93a5d0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f009e000640 1 -- 172.21.15.67:0/1519728571 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a4011d50 con 0x55c39b93a5d0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f009e000640 1 -- 172.21.15.67:0/1519728571 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f00b0036020 con 0x55c39b93a5d0 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f00c281e740 10 client.4590 fetch_fsmap finished waiting for FSMap version 8 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f00c281e740 10 client.4590 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-30T16:44:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f00c281e740 20 client.4590 populate_metadata read hostname 'smithi067' 2024-05-30T16:44:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f00c281e740 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] conn(0x55c39b94fa90 0x55c39b95aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f00c281e740 4 client.4590 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-30T16:44:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.552+0000 7f00c281e740 1 -- 172.21.15.67:0/1519728571 --> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] -- command(tid 0: {"prefix": "status"}) -- 0x55c39b93b650 con 0x55c39b94fa90 2024-05-30T16:44:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.553+0000 7f00bd000640 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] conn(0x55c39b94fa90 0x55c39b95aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.554+0000 7f00bd000640 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] conn(0x55c39b94fa90 0x55c39b95aaf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa527fe29352d2e server_cookie=6d47f68869f6d54c in_seq=0 out_seq=0 2024-05-30T16:44:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.554+0000 7f009e000640 10 client.4590 ms_handle_connect on v2:172.21.15.169:6833/2028878495 2024-05-30T16:44:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f009e000640 1 -- 172.21.15.67:0/1519728571 <== mds.0 v2:172.21.15.169:6833/2028878495 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55c39b93b650 con 0x55c39b94fa90 2024-05-30T16:44:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f009e000640 10 client.4590 handle_command_reply: tid=0 2024-05-30T16:44:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 client.4590 shutdown 2024-05-30T16:44:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 -- 172.21.15.67:0/1519728571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b93a5d0 msgr2=0x55c39b8ef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef4f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55c39b94bea0 tx=0x7f00b0004a40 comp rx=0 tx=0).stop 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 -- 172.21.15.67:0/1519728571 shutdown_connections 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:6833/2028878495,v1:172.21.15.169:6835/2028878495] conn(0x55c39b94fa90 0x55c39b95aaf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c39b93a5d0 0x55c39b8ef4f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c39b91e150 0x55c39b8eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 --2- 172.21.15.67:0/1519728571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c39b9173b0 0x55c39b9b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f009e000640 0 client.4590 ms_handle_reset on v2:172.21.15.169:6833/2028878495 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 -- 172.21.15.67:0/1519728571 >> 172.21.15.67:0/1519728571 conn(0x55c39b91a490 msgr2=0x55c39b9325c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 -- 172.21.15.67:0/1519728571 shutdown_connections 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 1 -- 172.21.15.67:0/1519728571 wait complete. 2024-05-30T16:44:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.555+0000 7f00c281e740 20 client.4590 trim_cache size 0 max 16384 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4296, 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 13.691831306999999, 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 8, 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 22, 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 22, 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 19.764023684000001 2024-05-30T16:44:06.559 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:44:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 -- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f009806c8f0 msgr2=0x7f009806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f009806c8f0 0x7f009806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00ac034000 tx=0x7f00ac02e000 comp rx=0 tx=0).stop 2024-05-30T16:44:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 -- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b80b15e0 msgr2=0x7f00b80afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b80b15e0 0x7f00b80afd00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c430 tx=0x7f00a400c8f0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 -- 172.21.15.67:0/2057350532 shutdown_connections 2024-05-30T16:44:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f009806c8f0 0x7f009806edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00b8161fb0 0x7f00b80b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00b80b15e0 0x7f00b80afd00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 --2- 172.21.15.67:0/2057350532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00b809b980 0x7f00b814fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 -- 172.21.15.67:0/2057350532 >> 172.21.15.67:0/2057350532 conn(0x7f00b80d23b0 msgr2=0x7f00b815c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 -- 172.21.15.67:0/2057350532 shutdown_connections 2024-05-30T16:44:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:06.556+0000 7f00be400640 1 -- 172.21.15.67:0/2057350532 wait complete. 2024-05-30T16:44:06.574 INFO:teuthology.run_tasks:Running task sequential... 2024-05-30T16:44:06.587 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-30T16:44:06.587 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-30T16:44:06.588 INFO:teuthology.task.print:Enabling mgr modules 2024-05-30T16:44:06.588 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-30T16:44:06.688 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-30T16:44:06.689 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-30T16:44:06.689 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-30T16:44:06.689 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-30T16:44:06.689 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:06.689 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-30T16:44:06.689 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-30T16:44:06.689 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-30T16:44:06.689 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-30T16:44:06.689 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-30T16:44:06.689 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:06.690 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-30T16:44:06.690 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-30T16:44:06.690 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-30T16:44:06.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:06.690 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-05-30T16:44:06.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:06.750 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-05-30T16:44:06.840 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-05-30T16:44:06.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T16:44:06.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:06.844 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T16:44:06.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:06.906 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-05-30T16:44:06.999 INFO:teuthology.orchestra.run.smithi067.stderr:Cannot find device "ceph-brx" 2024-05-30T16:44:07.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T16:44:07.004 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-30T16:44:07.004 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T16:44:07.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.208+0000 7fb4a7e00640 1 Processor -- start 2024-05-30T16:44:07.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.208+0000 7fb4a7e00640 1 -- start start 2024-05-30T16:44:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.210+0000 7fb4a7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4a8156b30 0x7fb4a80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.210+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad940 0x7fb4a80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.210+0000 7fb4a7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a8156f10 con 0x7fb4a806ee40 2024-05-30T16:44:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.210+0000 7fb4a7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a806dda0 con 0x7fb4a8156b30 2024-05-30T16:44:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.210+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a806e080 con 0x7fb4a80ad940 2024-05-30T16:44:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.210+0000 7fb4a6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb4a806ee40 0x7fb4a80ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52392/0 (socket says 172.21.15.67:52392) 2024-05-30T16:44:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.210+0000 7fb4a6000640 1 -- 172.21.15.67:0/1913634176 learned_addr learned my addr 172.21.15.67:0/1913634176 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:07.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.210+0000 7fb4a7400640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad940 0x7fb4a80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:07.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.210+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3421425483 0 0) 0x7fb4a8156f10 con 0x7fb4a806ee40 2024-05-30T16:44:07.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb48c003520 con 0x7fb4a806ee40 2024-05-30T16:44:07.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a6a00640 1 --2- 172.21.15.67:0/1913634176 >> v2:172.21.15.169:3300/0 conn(0x7fb4a8156b30 0x7fb4a80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2240867344 0 0) 0x7fb48c003520 con 0x7fb4a806ee40 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad940 msgr2=0x7fb4a80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad940 0x7fb4a80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 >> v2:172.21.15.169:3300/0 conn(0x7fb4a8156b30 msgr2=0x7fb4a80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 --2- 172.21.15.67:0/1913634176 >> v2:172.21.15.169:3300/0 conn(0x7fb4a8156b30 0x7fb4a80af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a806e360 con 0x7fb4a806ee40 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fb494001670 con 0x7fb4a806ee40 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a7400640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad940 0x7fb4a80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 >> v1:172.21.15.67:6789/0 conn(0x7fb4a806ee40 legacy=0x7fb4a80ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad940 0x7fb48c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c003ff0 0x7fb48c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:07.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb48c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a806dda0 con 0x7fb48c004d40 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a806e080 con 0x7fb48c003ff0 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.211+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb48c00bb80 con 0x7fb4a80ad940 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.212+0000 7fb4a7400640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb48c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.212+0000 7fb4a6a00640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad940 0x7fb48c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.212+0000 7fb4a6000640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c003ff0 0x7fb48c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.212+0000 7fb4a7400640 1 -- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad940 msgr2=0x7fb48c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.212+0000 7fb4a7400640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad940 0x7fb48c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.212+0000 7fb4a7400640 1 -- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c003ff0 msgr2=0x7fb48c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.212+0000 7fb4a7400640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb48c003ff0 0x7fb48c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.212+0000 7fb4a7400640 1 -- 172.21.15.67:0/1913634176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c003520 con 0x7fb48c004d40 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.213+0000 7fb4a7400640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb48c0051c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00073b0 tx=0x7fb48c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdea41706536dfba server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.213+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0032070 con 0x7fb48c004d40 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.213+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a0003b90 con 0x7fb48c004d40 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.213+0000 7fb4a5600640 1 -- 172.21.15.67:0/1913634176 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0009c80 con 0x7fb48c004d40 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.214+0000 7fb4a7e00640 1 -- 172.21.15.67:0/1913634176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 msgr2=0x7fb48c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.214+0000 7fb4a7e00640 1 --2- 172.21.15.67:0/1913634176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb48c0051c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00073b0 tx=0x7fb48c005830 comp rx=0 tx=0).stop 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.214+0000 7fb4a7400640 1 -- 172.21.15.67:0/1913634176 reap_dead start 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.214+0000 7fb4a7e00640 1 -- 172.21.15.67:0/1913634176 shutdown_connections 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.214+0000 7fb4a7e00640 1 -- 172.21.15.67:0/1913634176 >> 172.21.15.67:0/1913634176 conn(0x7fb4a80b1fc0 msgr2=0x7fb4a80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.214+0000 7fb4a7e00640 1 -- 172.21.15.67:0/1913634176 shutdown_connections 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.214+0000 7fb4a7e00640 1 -- 172.21.15.67:0/1913634176 wait complete. 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.216+0000 7fb4a7e00640 1 Processor -- start 2024-05-30T16:44:07.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.216+0000 7fb4a7e00640 1 -- start start 2024-05-30T16:44:07.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.217+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb4a8158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:07.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.217+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad230 0x7fb4a81f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:07.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.217+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a815af50 0x7fb4a815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.217+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a80afc20 con 0x7fb48c004d40 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.217+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a80b2c30 con 0x7fb4a815af50 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.217+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a8158930 con 0x7fb4a80ad230 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.217+0000 7fb4a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb4a8158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.217+0000 7fb4a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb4a8158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53310/0 (socket says 172.21.15.67:53310) 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.217+0000 7fb4a6a00640 1 -- 172.21.15.67:0/3406499611 learned_addr learned my addr 172.21.15.67:0/3406499611 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb4a6000640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad230 0x7fb4a81f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb4a7400640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a815af50 0x7fb4a815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb4a6a00640 1 -- 172.21.15.67:0/3406499611 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad230 msgr2=0x7fb4a81f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb4a6a00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad230 0x7fb4a81f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb4a6a00640 1 -- 172.21.15.67:0/3406499611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a815af50 msgr2=0x7fb4a815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb4a6a00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a815af50 0x7fb4a815de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb4a6a00640 1 -- 172.21.15.67:0/3406499611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a815f620 con 0x7fb48c004d40 2024-05-30T16:44:07.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb4a6a00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb4a8158dc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb49800f600 tx=0x7fb49800fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:07.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb49ea00640 1 -- 172.21.15.67:0/3406499611 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498019020 con 0x7fb48c004d40 2024-05-30T16:44:07.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb49ea00640 1 -- 172.21.15.67:0/3406499611 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb498014870 con 0x7fb48c004d40 2024-05-30T16:44:07.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb49ea00640 1 -- 172.21.15.67:0/3406499611 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498014c20 con 0x7fb48c004d40 2024-05-30T16:44:07.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.218+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a815ea50 con 0x7fb48c004d40 2024-05-30T16:44:07.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.219+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a815f010 con 0x7fb48c004d40 2024-05-30T16:44:07.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.220+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a80689e0 con 0x7fb48c004d40 2024-05-30T16:44:07.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.221+0000 7fb49ea00640 1 -- 172.21.15.67:0/3406499611 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb49801c020 con 0x7fb48c004d40 2024-05-30T16:44:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.222+0000 7fb49ea00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb48806c920 0x7fb48806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.222+0000 7fb49ea00640 1 -- 172.21.15.67:0/3406499611 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb49808c450 con 0x7fb48c004d40 2024-05-30T16:44:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.222+0000 7fb4a6000640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb48806c920 0x7fb48806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.223+0000 7fb4a6000640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb48806c920 0x7fb48806ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb494000c00 tx=0x7fb494000fb0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:07.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.227+0000 7fb49ea00640 1 -- 172.21.15.67:0/3406499611 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb498023020 con 0x7fb48c004d40 2024-05-30T16:44:07.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.474+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb4a80ad6f0 con 0x7fb48c004d40 2024-05-30T16:44:07.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.475+0000 7fb49ea00640 1 -- 172.21.15.67:0/3406499611 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fb49805a7e0 con 0x7fb48c004d40 2024-05-30T16:44:07.476 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T16:44:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb48806c920 msgr2=0x7fb48806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb48806c920 0x7fb48806ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb494000c00 tx=0x7fb494000fb0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 msgr2=0x7fb4a8158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb4a8158dc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb49800f600 tx=0x7fb49800fac0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 shutdown_connections 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb48806c920 0x7fb48806ede0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a815af50 0x7fb4a815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a80ad230 0x7fb4a81f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 --2- 172.21.15.67:0/3406499611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48c004d40 0x7fb4a8158dc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.478+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 >> 172.21.15.67:0/3406499611 conn(0x7fb4a80b1fc0 msgr2=0x7fb4a80af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.479+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 shutdown_connections 2024-05-30T16:44:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07.479+0000 7fb4a7e00640 1 -- 172.21.15.67:0/3406499611 wait complete. 2024-05-30T16:44:07.498 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T16:44:07.498 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T16:44:07.498 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T16:44:07.498 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-30T16:44:07.499 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:07.499 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T16:44:07.499 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T16:44:07.499 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T16:44:07.499 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:44:07.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:07.499 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T16:44:07.551 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:44:07.551 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:07.552 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T16:44:07.553 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:07.553 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T16:44:07.553 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42524sec preferred_lft 42524sec 2024-05-30T16:44:07.553 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T16:44:07.553 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:07.554 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-30T16:44:07.554 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:07.554 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:07.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-05-30T16:44:07.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-05-30T16:44:07.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-05-30T16:44:07.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-30T16:44:07.555 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:07.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:08.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:08.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:08.066 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T16:44:08.139 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T16:44:08.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:08.142 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T16:44:08.197 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T16:44:08.197 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T16:44:08.197 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T16:44:08.198 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T16:44:08.198 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T16:44:08.198 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:08.198 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:08.198 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T16:44:08.198 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T16:44:08.198 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T16:44:08.199 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:08.263 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:44:08.268 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:44:08.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:08.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:08.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:08.996 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T16:44:09.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:09.050 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-05-30T16:44:09.107 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:09.107 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:09.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:09.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-30T16:44:09.107 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:09.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:09.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:09.303 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-30T16:44:09.303 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:09.304 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:09.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-30T16:44:09.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-30T16:44:09.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-30T16:44:09.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-30T16:44:09.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-30T16:44:09.304 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:09.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:09.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:09.599 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:09.599 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:09.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-05-30T16:44:09.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-05-30T16:44:09.599 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:09.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:09.808 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-30T16:44:09.809 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:09.809 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:09.856 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:44:09.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:09.857 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:09.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:44:10.027 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:44:10.079 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:44:10.080 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:44:10.081 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:44:10.081 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:44:10.081 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:44:10.081 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:44:10.081 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:44:10.081 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:10.081 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:10.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:10.082 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:44:10.134 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T16:44:10.135 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-30T16:44:10.140 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:44:10.210 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:10.255 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:44:10.255 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:44:10.255 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:10.256 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:10.257 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:10.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:10.258 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:44:10.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-05-30T16:44:10.327+0000 7f655ec0c480 -1 init, newargv = 0x55d1cc6ddc80 newargc=15 2024-05-30T16:44:10.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[44255]: starting ceph client 2024-05-30T16:44:10.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[44255]: starting fuse 2024-05-30T16:44:10.347 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T16:44:10.348 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-30T16:44:10.348 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-05-30T16:44:10.349 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T16:44:10.540 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.44255.asok 2024-05-30T16:44:10.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:10.558 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44255.asok 2024-05-30T16:44:10.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:10.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44255.asok status 2024-05-30T16:44:10.793 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:44:10.793 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T16:44:10.793 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T16:44:10.793 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T16:44:10.793 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-05-30T16:44:10.793 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T16:44:10.793 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-30T16:44:10.793 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "44255", 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4602, 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4602 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:44:10.794 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1457953212 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1457953212 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.4602 192.168.0.1:0/1457953212", 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/1457953212", 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 8, 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 22, 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T16:44:10.795 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:44:10.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T16:44:11.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.022+0000 7fe19f200640 1 Processor -- start 2024-05-30T16:44:11.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.022+0000 7fe19f200640 1 -- start start 2024-05-30T16:44:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.024+0000 7fe19f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe1a015e790 0x7fe1a00b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.024+0000 7fe19f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a00b5840 con 0x7fe1a00711c0 2024-05-30T16:44:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.024+0000 7fe19f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a00b5b20 con 0x7fe1a009b140 2024-05-30T16:44:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.024+0000 7fe19f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a015eb70 con 0x7fe1a015e790 2024-05-30T16:44:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.024+0000 7fe19de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe1a015e790 0x7fe1a00b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.024+0000 7fe19de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe1a015e790 0x7fe1a00b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37920/0 (socket says 172.21.15.67:37920) 2024-05-30T16:44:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.024+0000 7fe19de00640 1 -- 172.21.15.67:0/437489234 learned_addr learned my addr 172.21.15.67:0/437489234 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.025+0000 7fe19de00640 1 -- 172.21.15.67:0/437489234 >> v1:172.21.15.169:6789/0 conn(0x7fe1a009b140 legacy=0x7fe1a006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.025+0000 7fe19de00640 1 -- 172.21.15.67:0/437489234 >> v1:172.21.15.67:6789/0 conn(0x7fe1a00711c0 legacy=0x7fe1a009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.025+0000 7fe19de00640 1 -- 172.21.15.67:0/437489234 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a00b5e00 con 0x7fe1a015e790 2024-05-30T16:44:11.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.026+0000 7fe19de00640 1 --2- 172.21.15.67:0/437489234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a015e790 0x7fe1a00b68e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe190009870 tx=0x7fe19002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e42f954c4fac49 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:11.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.027+0000 7fe19ca00640 1 -- 172.21.15.67:0/437489234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190037070 con 0x7fe1a015e790 2024-05-30T16:44:11.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.027+0000 7fe19ca00640 1 -- 172.21.15.67:0/437489234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19002ae50 con 0x7fe1a015e790 2024-05-30T16:44:11.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.027+0000 7fe19ca00640 1 -- 172.21.15.67:0/437489234 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190032860 con 0x7fe1a015e790 2024-05-30T16:44:11.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.027+0000 7fe19f200640 1 -- 172.21.15.67:0/437489234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a015e790 msgr2=0x7fe1a00b68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.027+0000 7fe19f200640 1 --2- 172.21.15.67:0/437489234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a015e790 0x7fe1a00b68e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe190009870 tx=0x7fe19002a300 comp rx=0 tx=0).stop 2024-05-30T16:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.027+0000 7fe19f200640 1 -- 172.21.15.67:0/437489234 shutdown_connections 2024-05-30T16:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.027+0000 7fe19f200640 1 --2- 172.21.15.67:0/437489234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a015e790 0x7fe1a00b68e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.027+0000 7fe19f200640 1 -- 172.21.15.67:0/437489234 >> 172.21.15.67:0/437489234 conn(0x7fe1a00b3ec0 msgr2=0x7fe1a00b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.028+0000 7fe19f200640 1 -- 172.21.15.67:0/437489234 shutdown_connections 2024-05-30T16:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.028+0000 7fe19f200640 1 -- 172.21.15.67:0/437489234 wait complete. 2024-05-30T16:44:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.029+0000 7fe19f200640 1 Processor -- start 2024-05-30T16:44:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.029+0000 7fe19f200640 1 -- start start 2024-05-30T16:44:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.030+0000 7fe19f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00711c0 0x7fe1a0152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.030+0000 7fe19f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a009b140 0x7fe1a0160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.030+0000 7fe19f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a015e790 0x7fe1a01613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:11.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.030+0000 7fe19f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a00b54d0 con 0x7fe1a009b140 2024-05-30T16:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.030+0000 7fe19f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a00b63c0 con 0x7fe1a00711c0 2024-05-30T16:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.030+0000 7fe19f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a0071620 con 0x7fe1a015e790 2024-05-30T16:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a009b140 0x7fe1a0160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a009b140 0x7fe1a0160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37926/0 (socket says 172.21.15.67:37926) 2024-05-30T16:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19d400640 1 -- 172.21.15.67:0/2318845018 learned_addr learned my addr 172.21.15.67:0/2318845018 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19de00640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00711c0 0x7fe1a0152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19e800640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a015e790 0x7fe1a01613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:11.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19d400640 1 -- 172.21.15.67:0/2318845018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a015e790 msgr2=0x7fe1a01613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:11.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19d400640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a015e790 0x7fe1a01613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:11.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19d400640 1 -- 172.21.15.67:0/2318845018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00711c0 msgr2=0x7fe1a0152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:11.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19d400640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00711c0 0x7fe1a0152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:11.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.031+0000 7fe19d400640 1 -- 172.21.15.67:0/2318845018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a0160710 con 0x7fe1a009b140 2024-05-30T16:44:11.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.032+0000 7fe19d400640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a009b140 0x7fe1a0160e90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe18800c450 tx=0x7fe18800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:11.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.032+0000 7fe196000640 1 -- 172.21.15.67:0/2318845018 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1880052d0 con 0x7fe1a009b140 2024-05-30T16:44:11.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.032+0000 7fe196000640 1 -- 172.21.15.67:0/2318845018 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe188005470 con 0x7fe1a009b140 2024-05-30T16:44:11.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.032+0000 7fe196000640 1 -- 172.21.15.67:0/2318845018 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188012870 con 0x7fe1a009b140 2024-05-30T16:44:11.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.032+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a01537b0 con 0x7fe1a009b140 2024-05-30T16:44:11.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.032+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a0153d70 con 0x7fe1a009b140 2024-05-30T16:44:11.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.033+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a00690f0 con 0x7fe1a009b140 2024-05-30T16:44:11.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.034+0000 7fe196000640 1 -- 172.21.15.67:0/2318845018 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe188012a10 con 0x7fe1a009b140 2024-05-30T16:44:11.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.035+0000 7fe196000640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe17c06c8f0 0x7fe17c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:11.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.035+0000 7fe196000640 1 -- 172.21.15.67:0/2318845018 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe18808b820 con 0x7fe1a009b140 2024-05-30T16:44:11.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.036+0000 7fe19de00640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe17c06c8f0 0x7fe17c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:11.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.036+0000 7fe19de00640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe17c06c8f0 0x7fe17c06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe190034000 tx=0x7fe19002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:11.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.039+0000 7fe196000640 1 -- 172.21.15.67:0/2318845018 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe188059bb0 con 0x7fe1a009b140 2024-05-30T16:44:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.289+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe1a009b800 con 0x7fe1a009b140 2024-05-30T16:44:11.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.290+0000 7fe196000640 1 -- 172.21.15.67:0/2318845018 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe1880594b0 con 0x7fe1a009b140 2024-05-30T16:44:11.292 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T16:44:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe17c06c8f0 msgr2=0x7fe17c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe17c06c8f0 0x7fe17c06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe190034000 tx=0x7fe19002e000 comp rx=0 tx=0).stop 2024-05-30T16:44:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a009b140 msgr2=0x7fe1a0160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a009b140 0x7fe1a0160e90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe18800c450 tx=0x7fe18800c910 comp rx=0 tx=0).stop 2024-05-30T16:44:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 shutdown_connections 2024-05-30T16:44:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe17c06c8f0 0x7fe17c06edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a015e790 0x7fe1a01613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1a009b140 0x7fe1a0160e90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 --2- 172.21.15.67:0/2318845018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00711c0 0x7fe1a0152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:11.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 >> 172.21.15.67:0/2318845018 conn(0x7fe1a00b3ec0 msgr2=0x7fe1a009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:11.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 shutdown_connections 2024-05-30T16:44:11.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11.293+0000 7fe19f200640 1 -- 172.21.15.67:0/2318845018 wait complete. 2024-05-30T16:44:11.313 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T16:44:11.313 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T16:44:11.313 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T16:44:11.313 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-30T16:44:11.313 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-30T16:44:11.313 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T16:44:11.313 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T16:44:11.313 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T16:44:11.313 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-30T16:44:11.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:11.314 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T16:44:11.364 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:44:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:44:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42521sec preferred_lft 42521sec 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::74f5:cfff:fec0:693d/64 scope link 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::a8fe:aeff:fef7:f6a0/64 scope link tentative 2024-05-30T16:44:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:11.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:11.367 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T16:44:11.449 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T16:44:11.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:11.454 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T16:44:11.510 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T16:44:11.510 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T16:44:11.510 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T16:44:11.510 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T16:44:11.510 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T16:44:11.511 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:11.511 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:11.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T16:44:11.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T16:44:11.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T16:44:11.512 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:11.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:11.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:11.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:11.738 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T16:44:11.792 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T16:44:11.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:11.793 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-05-30T16:44:11.846 INFO:teuthology.orchestra.run.smithi067.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-30T16:44:11.847 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:11.847 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:11.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:11.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-30T16:44:11.848 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:11.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:12.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:12.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:12.044 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether b6:2b:7e:59:a9:e0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::b42b:7eff:fe59:a9e0/64 scope link 2024-05-30T16:44:12.121 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:12.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:12.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-30T16:44:12.207 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:44:12.207 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:44:12.207 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:44:12.207 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:12.207 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T16:44:12.207 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:12.207 INFO:teuthology.orchestra.run.smithi067.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:44:12.207 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether b6:2b:7e:59:a9:e0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-30T16:44:12.208 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-30T16:44:12.208 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:12.208 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::b42b:7eff:fe59:a9e0/64 scope link 2024-05-30T16:44:12.208 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:12.212 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-30T16:44:12.212 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:12.212 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:12.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-30T16:44:12.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-30T16:44:12.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-30T16:44:12.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-30T16:44:12.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-30T16:44:12.212 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:12.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:12.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:12.500 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:12.500 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:12.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.1 up 2024-05-30T16:44:12.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.1 master ceph-brx 2024-05-30T16:44:12.500 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:12.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:12.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:12.717 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-30T16:44:12.717 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:12.717 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-30T16:44:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-30T16:44:12.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:12.765 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-30T16:44:12.816 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:44:12.896 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:12.954 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:44:12.954 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:44:12.954 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:44:12.954 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:44:12.954 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:44:12.954 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:44:12.955 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:12.956 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:12.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:12.957 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:44:13.015 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T16:44:13.015 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-05-30T16:44:13.016 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-30T16:44:13.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:44:13.094 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:13.140 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:44:13.140 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:44:13.140 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:44:13.140 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:44:13.141 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:13.142 INFO:teuthology.orchestra.run.smithi067.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:13.143 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:13.143 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:13.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:13.143 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:44:13.187 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T16:44:13.195 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi067.stderr:2024-05-30T16:44:13.194+0000 7f66cd187480 -1 init, newargv = 0x55d531ed1a40 newargc=15 2024-05-30T16:44:13.195 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi067.stderr:ceph-fuse[44758]: starting ceph client 2024-05-30T16:44:13.206 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi067.stderr:ceph-fuse[44758]: starting fuse 2024-05-30T16:44:14.288 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:44:14.378 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:14.433 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:44:14.433 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:44:14.433 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:44:14.433 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:44:14.433 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:44:14.433 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:44:14.433 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:44:14.433 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:44:14.433 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:14.434 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:14.435 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:14.436 INFO:teuthology.orchestra.run.smithi067.stdout:/home/ubuntu/cephtest/mnt.2 2024-05-30T16:44:14.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:14.437 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:44:14.493 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T16:44:14.493 INFO:teuthology.orchestra.run.smithi067.stdout:43 2024-05-30T16:44:14.493 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-05-30T16:44:14.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T16:44:14.494 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.2")) 2024-05-30T16:44:14.495 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T16:44:14.672 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.2.44758.asok 2024-05-30T16:44:14.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:14.690 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.44758.asok 2024-05-30T16:44:14.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:14.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.44758.asok status 2024-05-30T16:44:14.927 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:44:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T16:44:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T16:44:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "2", 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "44758", 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4620, 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T16:44:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4620 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.2:0", 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 4039270450 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.2:0", 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 4039270450 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.4620 192.168.0.2:0/4039270450", 2024-05-30T16:44:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.2:0/4039270450", 2024-05-30T16:44:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T16:44:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 8, 2024-05-30T16:44:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 22, 2024-05-30T16:44:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T16:44:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T16:44:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T16:44:14.930 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:44:14.942 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T16:44:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.160+0000 7fb261800640 1 Processor -- start 2024-05-30T16:44:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.160+0000 7fb261800640 1 -- start start 2024-05-30T16:44:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.161+0000 7fb261800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb25c15b3a0 0x7fb25c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.161+0000 7fb261800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.161+0000 7fb261800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb25c0afd60 con 0x7fb25c072c00 2024-05-30T16:44:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.161+0000 7fb261800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb25c06f980 con 0x7fb25c15b3a0 2024-05-30T16:44:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.161+0000 7fb261800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb25c0ae0c0 con 0x7fb25c06dc20 2024-05-30T16:44:15.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.162+0000 7fb25b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:15.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.162+0000 7fb25be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb25c15b3a0 0x7fb25c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:15.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.162+0000 7fb25b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c15f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:52890/0 (socket says 172.21.15.67:52890) 2024-05-30T16:44:15.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.162+0000 7fb25b400640 1 -- 172.21.15.67:0/3485868101 learned_addr learned my addr 172.21.15.67:0/3485868101 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:15.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.163+0000 7fb25b400640 1 -- 172.21.15.67:0/3485868101 >> v2:172.21.15.169:3300/0 conn(0x7fb25c15b3a0 msgr2=0x7fb25c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:15.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.163+0000 7fb25b400640 1 --2- 172.21.15.67:0/3485868101 >> v2:172.21.15.169:3300/0 conn(0x7fb25c15b3a0 0x7fb25c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:15.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.163+0000 7fb25b400640 1 -- 172.21.15.67:0/3485868101 >> v1:172.21.15.169:6789/0 conn(0x7fb25c072c00 legacy=0x7fb25c0ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:15.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.163+0000 7fb25b400640 1 -- 172.21.15.67:0/3485868101 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c0af7a0 con 0x7fb25c06dc20 2024-05-30T16:44:15.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.164+0000 7fb25b400640 1 --2- 172.21.15.67:0/3485868101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c15f9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb244009e50 tx=0x7fb24402a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f715852171a8b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:15.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.165+0000 7fb25aa00640 1 -- 172.21.15.67:0/3485868101 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244036070 con 0x7fb25c06dc20 2024-05-30T16:44:15.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.165+0000 7fb25aa00640 1 -- 172.21.15.67:0/3485868101 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb244031600 con 0x7fb25c06dc20 2024-05-30T16:44:15.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.165+0000 7fb25aa00640 1 -- 172.21.15.67:0/3485868101 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2440319d0 con 0x7fb25c06dc20 2024-05-30T16:44:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.166+0000 7fb261800640 1 -- 172.21.15.67:0/3485868101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 msgr2=0x7fb25c15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.166+0000 7fb261800640 1 --2- 172.21.15.67:0/3485868101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c15f9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb244009e50 tx=0x7fb24402a730 comp rx=0 tx=0).stop 2024-05-30T16:44:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.166+0000 7fb261800640 1 -- 172.21.15.67:0/3485868101 shutdown_connections 2024-05-30T16:44:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.166+0000 7fb261800640 1 --2- 172.21.15.67:0/3485868101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c15f9a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.166+0000 7fb261800640 1 --2- 172.21.15.67:0/3485868101 >> v2:172.21.15.169:3300/0 conn(0x7fb25c15b3a0 0x7fb25c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.166+0000 7fb261800640 1 -- 172.21.15.67:0/3485868101 >> 172.21.15.67:0/3485868101 conn(0x7fb25c0b1fc0 msgr2=0x7fb25c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.166+0000 7fb261800640 1 -- 172.21.15.67:0/3485868101 shutdown_connections 2024-05-30T16:44:15.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.166+0000 7fb261800640 1 -- 172.21.15.67:0/3485868101 wait complete. 2024-05-30T16:44:15.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.168+0000 7fb261800640 1 Processor -- start 2024-05-30T16:44:15.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.168+0000 7fb261800640 1 -- start start 2024-05-30T16:44:15.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.169+0000 7fb261800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c14eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:15.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.169+0000 7fb261800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb25c072c00 0x7fb25c0c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.169+0000 7fb261800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb25c15b3a0 0x7fb25c1f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.169+0000 7fb261800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c15b780 con 0x7fb25c072c00 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.169+0000 7fb261800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c15fee0 con 0x7fb25c15b3a0 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.169+0000 7fb261800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb25c073050 con 0x7fb25c06dc20 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.169+0000 7fb25b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb25c072c00 0x7fb25c0c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.169+0000 7fb25b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb25c072c00 0x7fb25c0c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37928/0 (socket says 172.21.15.67:37928) 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.169+0000 7fb25b400640 1 -- 172.21.15.67:0/2307005149 learned_addr learned my addr 172.21.15.67:0/2307005149 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.170+0000 7fb25be00640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c14eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.170+0000 7fb260e00640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb25c15b3a0 0x7fb25c1f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.170+0000 7fb25b400640 1 -- 172.21.15.67:0/2307005149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 msgr2=0x7fb25c14eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.170+0000 7fb25b400640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c14eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.170+0000 7fb25b400640 1 -- 172.21.15.67:0/2307005149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb25c15b3a0 msgr2=0x7fb25c1f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:15.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.170+0000 7fb25b400640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb25c15b3a0 0x7fb25c1f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:15.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.170+0000 7fb25b400640 1 -- 172.21.15.67:0/2307005149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c0c9b80 con 0x7fb25c072c00 2024-05-30T16:44:15.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.170+0000 7fb25be00640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c14eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:15.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.170+0000 7fb25b400640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb25c072c00 0x7fb25c0c9640 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb244000f80 tx=0x7fb244004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:15.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.171+0000 7fb253e00640 1 -- 172.21.15.67:0/2307005149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244041020 con 0x7fb25c072c00 2024-05-30T16:44:15.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.171+0000 7fb253e00640 1 -- 172.21.15.67:0/2307005149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2440087b0 con 0x7fb25c072c00 2024-05-30T16:44:15.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.171+0000 7fb253e00640 1 -- 172.21.15.67:0/2307005149 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244036040 con 0x7fb25c072c00 2024-05-30T16:44:15.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.171+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25c0c9e20 con 0x7fb25c072c00 2024-05-30T16:44:15.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.171+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25c0ca3e0 con 0x7fb25c072c00 2024-05-30T16:44:15.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.172+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb25c0689e0 con 0x7fb25c072c00 2024-05-30T16:44:15.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.173+0000 7fb253e00640 1 -- 172.21.15.67:0/2307005149 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb244044020 con 0x7fb25c072c00 2024-05-30T16:44:15.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.174+0000 7fb253e00640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb23806c8f0 0x7fb23806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:15.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.174+0000 7fb253e00640 1 -- 172.21.15.67:0/2307005149 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb2440ac790 con 0x7fb25c072c00 2024-05-30T16:44:15.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.174+0000 7fb25be00640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb23806c8f0 0x7fb23806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:15.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.175+0000 7fb25be00640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb23806c8f0 0x7fb23806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0b1cc0 tx=0x7fb24c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:15.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.179+0000 7fb253e00640 1 -- 172.21.15.67:0/2307005149 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb24404b030 con 0x7fb25c072c00 2024-05-30T16:44:15.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.424+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb25c0b38a0 con 0x7fb25c072c00 2024-05-30T16:44:15.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.425+0000 7fb253e00640 1 -- 172.21.15.67:0/2307005149 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fb24407ab20 con 0x7fb25c072c00 2024-05-30T16:44:15.427 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T16:44:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb23806c8f0 msgr2=0x7fb23806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb23806c8f0 0x7fb23806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0b1cc0 tx=0x7fb24c008040 comp rx=0 tx=0).stop 2024-05-30T16:44:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb25c072c00 msgr2=0x7fb25c0c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb25c072c00 0x7fb25c0c9640 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb244000f80 tx=0x7fb244004a40 comp rx=0 tx=0).stop 2024-05-30T16:44:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 shutdown_connections 2024-05-30T16:44:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb23806c8f0 0x7fb23806edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb25c15b3a0 0x7fb25c1f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:15.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb25c072c00 0x7fb25c0c9640 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:15.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 --2- 172.21.15.67:0/2307005149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb25c06dc20 0x7fb25c14eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:15.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 >> 172.21.15.67:0/2307005149 conn(0x7fb25c0b1fc0 msgr2=0x7fb25c0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:15.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 shutdown_connections 2024-05-30T16:44:15.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:15.428+0000 7fb261800640 1 -- 172.21.15.67:0/2307005149 wait complete. 2024-05-30T16:44:15.447 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T16:44:15.447 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T16:44:15.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-05-30T16:44:15.447 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-30T16:44:15.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:15.447 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T16:44:15.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T16:44:15.447 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T16:44:15.447 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-30T16:44:15.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:15.448 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-05-30T16:44:15.505 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:15.506 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-05-30T16:44:15.507 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:15.507 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:15.507 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T16:44:15.507 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:15.507 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T16:44:15.507 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42523sec preferred_lft 42523sec 2024-05-30T16:44:15.507 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2024-05-30T16:44:15.507 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:15.508 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-30T16:44:15.508 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:15.508 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-05-30T16:44:15.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2024-05-30T16:44:15.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2024-05-30T16:44:15.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2024-05-30T16:44:15.509 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-30T16:44:15.509 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-05-30T16:44:15.617 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:15.995 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:16.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:16.004 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T16:44:16.067 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-05-30T16:44:16.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:16.070 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-05-30T16:44:16.125 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-05-30T16:44:16.125 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T16:44:16.125 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T16:44:16.125 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T16:44:16.125 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T16:44:16.126 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:16.126 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-05-30T16:44:16.126 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T16:44:16.126 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T16:44:16.126 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T16:44:16.126 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-05-30T16:44:16.230 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:16.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:16.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:16.892 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-05-30T16:44:16.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:16.942 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-05-30T16:44:16.993 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:16.993 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-05-30T16:44:16.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:16.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-30T16:44:16.993 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-05-30T16:44:17.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:17.168 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:17.177 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-30T16:44:17.178 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:17.178 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-05-30T16:44:17.178 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-30T16:44:17.178 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-30T16:44:17.178 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-30T16:44:17.178 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-30T16:44:17.178 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-30T16:44:17.178 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-05-30T16:44:17.278 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:17.452 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:17.461 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:17.461 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-05-30T16:44:17.461 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2024-05-30T16:44:17.461 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2024-05-30T16:44:17.461 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-05-30T16:44:17.549 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:17.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:17.658 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-30T16:44:17.659 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:17.659 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:17.699 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-30T16:44:17.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:17.699 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:17.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2024-05-30T16:44:17.864 DEBUG:teuthology.orchestra.run.smithi169:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:17.917 INFO:teuthology.orchestra.run.smithi169.stdout:/proc 2024-05-30T16:44:17.917 INFO:teuthology.orchestra.run.smithi169.stdout:/sys 2024-05-30T16:44:17.917 INFO:teuthology.orchestra.run.smithi169.stdout:/dev 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/security 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/shm 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/pts 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/run 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/cgroup 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/pstore 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/bpf 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/ 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/selinux 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/hugepages 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/mqueue 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/debug 2024-05-30T16:44:17.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/tracing 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/config 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/1000 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-4 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-5 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-6 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-7 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns 2024-05-30T16:44:17.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:17.920 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:17.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:17.921 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2024-05-30T16:44:17.969 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T16:44:17.969 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-30T16:44:17.974 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2024-05-30T16:44:18.043 DEBUG:teuthology.orchestra.run.smithi169:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:18.082 INFO:teuthology.orchestra.run.smithi169.stdout:/proc 2024-05-30T16:44:18.082 INFO:teuthology.orchestra.run.smithi169.stdout:/sys 2024-05-30T16:44:18.082 INFO:teuthology.orchestra.run.smithi169.stdout:/dev 2024-05-30T16:44:18.082 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/security 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/shm 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/pts 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/run 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/cgroup 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/pstore 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/bpf 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/ 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/selinux 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/hugepages 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/mqueue 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/debug 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/tracing 2024-05-30T16:44:18.083 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/config 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/1000 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-4 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-5 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-6 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-7 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:18.084 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:18.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:18.085 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2024-05-30T16:44:18.150 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:2024-05-30T16:44:18.150+0000 7fe0ad1ee480 -1 init, newargv = 0x557f9a8b3700 newargc=15 2024-05-30T16:44:18.151 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:ceph-fuse[41988]: starting ceph client 2024-05-30T16:44:18.161 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:ceph-fuse[41988]: starting fuse 2024-05-30T16:44:18.173 INFO:teuthology.orchestra.run.smithi169.stdout:42 2024-05-30T16:44:18.173 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-30T16:44:18.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo stdin-killer -- python3 -c ' 2024-05-30T16:44:18.173 DEBUG:teuthology.orchestra.run.smithi169:> import glob 2024-05-30T16:44:18.173 DEBUG:teuthology.orchestra.run.smithi169:> import re 2024-05-30T16:44:18.173 DEBUG:teuthology.orchestra.run.smithi169:> import os 2024-05-30T16:44:18.173 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> def _find_admin_socket(client_name): 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> files = glob.glob(asok_path) 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> # Given a non-glob path, it better be there 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> if "*" not in asok_path: 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> assert(len(files) == 1) 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> return files[0] 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> for f in files: 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T16:44:18.174 DEBUG:teuthology.orchestra.run.smithi169:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T16:44:18.175 DEBUG:teuthology.orchestra.run.smithi169:> contents = proc_f.read() 2024-05-30T16:44:18.175 DEBUG:teuthology.orchestra.run.smithi169:> if mountpoint in contents: 2024-05-30T16:44:18.175 DEBUG:teuthology.orchestra.run.smithi169:> return f 2024-05-30T16:44:18.175 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T16:44:18.175 DEBUG:teuthology.orchestra.run.smithi169:> 2024-05-30T16:44:18.175 DEBUG:teuthology.orchestra.run.smithi169:> print(_find_admin_socket("client.1")) 2024-05-30T16:44:18.175 DEBUG:teuthology.orchestra.run.smithi169:> ' 2024-05-30T16:44:18.372 INFO:teuthology.orchestra.run.smithi169.stdout:/var/run/ceph/ceph-client.1.41988.asok 2024-05-30T16:44:18.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:18.389 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.41988.asok 2024-05-30T16:44:18.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:18.389 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.41988.asok status 2024-05-30T16:44:18.614 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-30T16:44:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": { 2024-05-30T16:44:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "entity_id": "1", 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "hostname": "smithi169", 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "pid": "41988", 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "root": "/" 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_count": 0, 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_pinned_count": 0, 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 4415, 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "inst": { 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "name": { 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "client", 2024-05-30T16:44:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "num": 4415 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1886540673 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1886540673 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "inst_str": "client.4415 192.168.0.1:0/1886540673", 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "addr_str": "192.168.0.1:0/1886540673", 2024-05-30T16:44:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "inode_count": 1, 2024-05-30T16:44:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_epoch": 8, 2024-05-30T16:44:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch": 22, 2024-05-30T16:44:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch_barrier": 0, 2024-05-30T16:44:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklisted": false, 2024-05-30T16:44:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs" 2024-05-30T16:44:18.617 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-30T16:44:18.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T16:44:18.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.849+0000 7fc4fa200640 1 Processor -- start 2024-05-30T16:44:18.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.849+0000 7fc4fa200640 1 -- start start 2024-05-30T16:44:18.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.850+0000 7fc4fa200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4f40b4e10 0x7fc4f40cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:18.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.850+0000 7fc4fa200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4f40d0ca0 0x7fc4f4099090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.850+0000 7fc4fa200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f40b51f0 con 0x7fc4f40ce370 2024-05-30T16:44:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.850+0000 7fc4fa200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f40aef60 con 0x7fc4f40b4e10 2024-05-30T16:44:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.850+0000 7fc4fa200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f40ad1d0 con 0x7fc4f40d0ca0 2024-05-30T16:44:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.851+0000 7fc4f3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4f40b4e10 0x7fc4f40cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.851+0000 7fc4f3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4f40b4e10 0x7fc4f40cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37938/0 (socket says 172.21.15.67:37938) 2024-05-30T16:44:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.851+0000 7fc4f3400640 1 -- 172.21.15.67:0/2303844808 learned_addr learned my addr 172.21.15.67:0/2303844808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.851+0000 7fc4f3400640 1 -- 172.21.15.67:0/2303844808 >> v2:172.21.15.169:3300/0 conn(0x7fc4f40d0ca0 msgr2=0x7fc4f4099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.851+0000 7fc4f3400640 1 --2- 172.21.15.67:0/2303844808 >> v2:172.21.15.169:3300/0 conn(0x7fc4f40d0ca0 0x7fc4f4099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.851+0000 7fc4f3400640 1 -- 172.21.15.67:0/2303844808 >> v1:172.21.15.67:6789/0 conn(0x7fc4f40ce370 legacy=0x7fc4f40d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:18.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.851+0000 7fc4f3400640 1 -- 172.21.15.67:0/2303844808 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f40ad470 con 0x7fc4f40b4e10 2024-05-30T16:44:18.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.853+0000 7fc4f3400640 1 --2- 172.21.15.67:0/2303844808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40b4e10 0x7fc4f40cde30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40afd70 tx=0x7fc4e402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9081f8ee482bb72 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:18.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.853+0000 7fc4f2000640 1 -- 172.21.15.67:0/2303844808 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4033640 con 0x7fc4f40b4e10 2024-05-30T16:44:18.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.853+0000 7fc4f2000640 1 -- 172.21.15.67:0/2303844808 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e4038070 con 0x7fc4f40b4e10 2024-05-30T16:44:18.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.853+0000 7fc4f2000640 1 -- 172.21.15.67:0/2303844808 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4033a00 con 0x7fc4f40b4e10 2024-05-30T16:44:18.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.853+0000 7fc4fa200640 1 -- 172.21.15.67:0/2303844808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40b4e10 msgr2=0x7fc4f40cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:18.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.853+0000 7fc4fa200640 1 --2- 172.21.15.67:0/2303844808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40b4e10 0x7fc4f40cde30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40afd70 tx=0x7fc4e402c8b0 comp rx=0 tx=0).stop 2024-05-30T16:44:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.854+0000 7fc4fa200640 1 -- 172.21.15.67:0/2303844808 shutdown_connections 2024-05-30T16:44:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.854+0000 7fc4fa200640 1 --2- 172.21.15.67:0/2303844808 >> v2:172.21.15.169:3300/0 conn(0x7fc4f40d0ca0 0x7fc4f4099090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.854+0000 7fc4fa200640 1 --2- 172.21.15.67:0/2303844808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40b4e10 0x7fc4f40cde30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.854+0000 7fc4fa200640 1 -- 172.21.15.67:0/2303844808 >> 172.21.15.67:0/2303844808 conn(0x7fc4f406bae0 msgr2=0x7fc4f406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.854+0000 7fc4fa200640 1 -- 172.21.15.67:0/2303844808 shutdown_connections 2024-05-30T16:44:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.854+0000 7fc4fa200640 1 -- 172.21.15.67:0/2303844808 wait complete. 2024-05-30T16:44:18.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.856+0000 7fc4fa200640 1 Processor -- start 2024-05-30T16:44:18.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.856+0000 7fc4fa200640 1 -- start start 2024-05-30T16:44:18.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.857+0000 7fc4fa200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f40b4e10 0x7fc4f41f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:18.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.857+0000 7fc4fa200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40ce370 0x7fc4f4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:18.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.857+0000 7fc4fa200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4f40d0ca0 0x7fc4f415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:18.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.857+0000 7fc4fa200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f40cea00 con 0x7fc4f40ce370 2024-05-30T16:44:18.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.857+0000 7fc4fa200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f40ce880 con 0x7fc4f40b4e10 2024-05-30T16:44:18.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.857+0000 7fc4fa200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f40b3150 con 0x7fc4f40d0ca0 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40ce370 0x7fc4f4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40ce370 0x7fc4f4157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37948/0 (socket says 172.21.15.67:37948) 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f2a00640 1 -- 172.21.15.67:0/1841939575 learned_addr learned my addr 172.21.15.67:0/1841939575 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f3400640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f40b4e10 0x7fc4f41f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f3e00640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4f40d0ca0 0x7fc4f415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f2a00640 1 -- 172.21.15.67:0/1841939575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4f40d0ca0 msgr2=0x7fc4f415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f2a00640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4f40d0ca0 0x7fc4f415dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f2a00640 1 -- 172.21.15.67:0/1841939575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f40b4e10 msgr2=0x7fc4f41f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f2a00640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f40b4e10 0x7fc4f41f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.858+0000 7fc4f2a00640 1 -- 172.21.15.67:0/1841939575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f40cdf00 con 0x7fc4f40ce370 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.859+0000 7fc4f2a00640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40ce370 0x7fc4f4157a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c430 tx=0x7fc4e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.859+0000 7fc4eb400640 1 -- 172.21.15.67:0/1841939575 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00052d0 con 0x7fc4f40ce370 2024-05-30T16:44:18.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.859+0000 7fc4eb400640 1 -- 172.21.15.67:0/1841939575 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e0005490 con 0x7fc4f40ce370 2024-05-30T16:44:18.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.859+0000 7fc4eb400640 1 -- 172.21.15.67:0/1841939575 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0007260 con 0x7fc4f40ce370 2024-05-30T16:44:18.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.859+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f40b2800 con 0x7fc4f40ce370 2024-05-30T16:44:18.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.859+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f415d7f0 con 0x7fc4f40ce370 2024-05-30T16:44:18.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.860+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f415d5e0 con 0x7fc4f40ce370 2024-05-30T16:44:18.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.861+0000 7fc4eb400640 1 -- 172.21.15.67:0/1841939575 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4e0007400 con 0x7fc4f40ce370 2024-05-30T16:44:18.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.862+0000 7fc4eb400640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc4d406c8f0 0x7fc4d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:18.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.862+0000 7fc4eb400640 1 -- 172.21.15.67:0/1841939575 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc4e008b150 con 0x7fc4f40ce370 2024-05-30T16:44:18.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.863+0000 7fc4f3400640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc4d406c8f0 0x7fc4d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:18.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.863+0000 7fc4f3400640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc4d406c8f0 0x7fc4d406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4037000 tx=0x7fc4e4032000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:18.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:18.869+0000 7fc4eb400640 1 -- 172.21.15.67:0/1841939575 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4f40cdf00 con 0x7fc4f40ce370 2024-05-30T16:44:19.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.114+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc4f40689e0 con 0x7fc4f40ce370 2024-05-30T16:44:19.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.115+0000 7fc4eb400640 1 -- 172.21.15.67:0/1841939575 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fc4e00594e0 con 0x7fc4f40ce370 2024-05-30T16:44:19.115 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc4d406c8f0 msgr2=0x7fc4d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc4d406c8f0 0x7fc4d406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4037000 tx=0x7fc4e4032000 comp rx=0 tx=0).stop 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40ce370 msgr2=0x7fc4f4157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40ce370 0x7fc4f4157a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c430 tx=0x7fc4e000c8f0 comp rx=0 tx=0).stop 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 shutdown_connections 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc4d406c8f0 0x7fc4d406edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4f40d0ca0 0x7fc4f415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f40ce370 0x7fc4f4157a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 --2- 172.21.15.67:0/1841939575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4f40b4e10 0x7fc4f41f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 >> 172.21.15.67:0/1841939575 conn(0x7fc4f406bae0 msgr2=0x7fc4f40cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 shutdown_connections 2024-05-30T16:44:19.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:19.118+0000 7fc4fa200640 1 -- 172.21.15.67:0/1841939575 wait complete. 2024-05-30T16:44:19.136 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T16:44:19.136 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T16:44:19.137 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-05-30T16:44:19.137 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-30T16:44:19.137 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-30T16:44:19.137 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T16:44:19.137 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T16:44:19.137 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T16:44:19.137 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-30T16:44:19.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:19.137 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-05-30T16:44:19.193 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:44:19.193 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:44:19.193 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:44:19.193 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T16:44:19.194 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42519sec preferred_lft 42519sec 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether f2:f0:c9:62:00:9c brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::e82d:94ff:fe7e:eb0/64 scope link 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether f2:f0:c9:62:00:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::f0f0:c9ff:fe62:9c/64 scope link 2024-05-30T16:44:19.195 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:19.196 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T16:44:19.275 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-05-30T16:44:19.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:19.279 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-05-30T16:44:19.332 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-05-30T16:44:19.332 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T16:44:19.332 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T16:44:19.332 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T16:44:19.332 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T16:44:19.332 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:19.332 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-05-30T16:44:19.332 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T16:44:19.333 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T16:44:19.333 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T16:44:19.333 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-05-30T16:44:19.440 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:19.535 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:19.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:19.544 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-05-30T16:44:19.590 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-30T16:44:19.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:19.591 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-05-30T16:44:19.683 INFO:teuthology.orchestra.run.smithi169.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-30T16:44:19.684 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:19.684 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-05-30T16:44:19.684 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-30T16:44:19.685 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-30T16:44:19.685 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-05-30T16:44:19.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:19.854 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:19.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:19.864 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-30T16:44:19.936 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:44:19.936 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:44:19.936 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:44:19.936 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.936 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-05-30T16:44:19.937 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.937 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:44:19.937 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 66:ee:3a:94:0a:7c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-30T16:44:19.937 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-30T16:44:19.937 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.937 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::64ee:3aff:fe94:a7c/64 scope link 2024-05-30T16:44:19.937 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:19.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:19.940 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-30T16:44:20.021 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:44:20.021 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 66:ee:3a:94:0a:7c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::64ee:3aff:fe94:a7c/64 scope link 2024-05-30T16:44:20.022 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:20.026 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-30T16:44:20.026 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:20.026 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-05-30T16:44:20.026 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-30T16:44:20.026 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-30T16:44:20.026 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-30T16:44:20.026 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-30T16:44:20.026 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-30T16:44:20.026 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-05-30T16:44:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:20.296 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:20.307 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:20.307 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-05-30T16:44:20.307 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.1 up 2024-05-30T16:44:20.307 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.1 master ceph-brx 2024-05-30T16:44:20.307 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-05-30T16:44:20.407 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:20.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:20.517 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-30T16:44:20.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:20.517 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-30T16:44:20.567 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-30T16:44:20.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:20.568 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-30T16:44:20.615 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2024-05-30T16:44:20.698 DEBUG:teuthology.orchestra.run.smithi169:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/proc 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/sys 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/dev 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/security 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/shm 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/pts 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/run 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/cgroup 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/pstore 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/bpf 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/ 2024-05-30T16:44:20.751 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/selinux 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/hugepages 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/mqueue 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/debug 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/tracing 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/config 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/1000 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph 2024-05-30T16:44:20.752 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-4 2024-05-30T16:44:20.753 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-5 2024-05-30T16:44:20.753 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-6 2024-05-30T16:44:20.753 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-7 2024-05-30T16:44:20.753 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns 2024-05-30T16:44:20.753 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:20.753 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:20.753 INFO:teuthology.orchestra.run.smithi169.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:20.753 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-30T16:44:20.753 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-30T16:44:20.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:20.754 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2024-05-30T16:44:20.806 INFO:teuthology.orchestra.run.smithi169.stdout:42 2024-05-30T16:44:20.806 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-05-30T16:44:20.806 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-30T16:44:20.811 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2024-05-30T16:44:20.877 DEBUG:teuthology.orchestra.run.smithi169:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/proc 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/sys 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/dev 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/security 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/shm 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/pts 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/run 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/cgroup 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/pstore 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/bpf 2024-05-30T16:44:20.917 INFO:teuthology.orchestra.run.smithi169.stdout:/ 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/selinux 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/hugepages 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/mqueue 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/debug 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/tracing 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/config 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/1000 2024-05-30T16:44:20.918 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-4 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-5 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-6 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph/osd/ceph-7 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-30T16:44:20.919 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-30T16:44:20.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:20.920 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2024-05-30T16:44:20.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi169.stderr:2024-05-30T16:44:20.974+0000 7f4542302480 -1 init, newargv = 0x562a1f0c2cb0 newargc=15 2024-05-30T16:44:20.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi169.stderr:ceph-fuse[42444]: starting ceph client 2024-05-30T16:44:20.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi169.stderr:ceph-fuse[42444]: starting fuse 2024-05-30T16:44:21.002 INFO:teuthology.orchestra.run.smithi169.stdout:42 2024-05-30T16:44:21.002 INFO:teuthology.orchestra.run.smithi169.stdout:43 2024-05-30T16:44:21.003 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> sudo stdin-killer -- python3 -c ' 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> import glob 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> import re 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> import os 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> def _find_admin_socket(client_name): 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> files = glob.glob(asok_path) 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-30T16:44:21.003 DEBUG:teuthology.orchestra.run.smithi169:> 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> # Given a non-glob path, it better be there 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> if "*" not in asok_path: 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> assert(len(files) == 1) 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> return files[0] 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> for f in files: 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> contents = proc_f.read() 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> if mountpoint in contents: 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> return f 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> print(_find_admin_socket("client.3")) 2024-05-30T16:44:21.004 DEBUG:teuthology.orchestra.run.smithi169:> ' 2024-05-30T16:44:21.198 INFO:teuthology.orchestra.run.smithi169.stdout:/var/run/ceph/ceph-client.3.42444.asok 2024-05-30T16:44:21.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-05-30T16:44:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:21.215 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.42444.asok 2024-05-30T16:44:21.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:21.215 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.42444.asok status 2024-05-30T16:44:21.445 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-05-30T16:44:21.445 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": { 2024-05-30T16:44:21.445 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T16:44:21.445 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T16:44:21.445 INFO:teuthology.orchestra.run.smithi169.stdout: "entity_id": "3", 2024-05-30T16:44:21.445 INFO:teuthology.orchestra.run.smithi169.stdout: "hostname": "smithi169", 2024-05-30T16:44:21.445 INFO:teuthology.orchestra.run.smithi169.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-30T16:44:21.445 INFO:teuthology.orchestra.run.smithi169.stdout: "pid": "42444", 2024-05-30T16:44:21.445 INFO:teuthology.orchestra.run.smithi169.stdout: "root": "/" 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_count": 0, 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_pinned_count": 0, 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 4564, 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "inst": { 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "name": { 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "client", 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "num": 4564 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.2:0", 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1594737212 2024-05-30T16:44:21.446 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.2:0", 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1594737212 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "inst_str": "client.4564 192.168.0.2:0/1594737212", 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "addr_str": "192.168.0.2:0/1594737212", 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "inode_count": 1, 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_epoch": 8, 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch": 22, 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch_barrier": 0, 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklisted": false, 2024-05-30T16:44:21.447 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs" 2024-05-30T16:44:21.448 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-05-30T16:44:21.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:21.460 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T16:44:21.460 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:21.521 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T16:44:21.521 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:21.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:21.522 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:21.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:21.616 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T16:44:21.616 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-30T16:44:21.674 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T16:44:21.674 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-30T16:44:21.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:21.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-30T16:44:21.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:21.775 DEBUG:teuthology.orchestra.run.smithi169:> stat --file-system '--printf=%T 2024-05-30T16:44:21.775 DEBUG:teuthology.orchestra.run.smithi169:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:21.835 INFO:teuthology.orchestra.run.smithi169.stdout:fuseblk 2024-05-30T16:44:21.835 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:21.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:21.836 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:21.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:21.920 DEBUG:teuthology.orchestra.run.smithi169:> stat --file-system '--printf=%T 2024-05-30T16:44:21.920 DEBUG:teuthology.orchestra.run.smithi169:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-30T16:44:21.975 INFO:teuthology.orchestra.run.smithi169.stdout:fuseblk 2024-05-30T16:44:21.976 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-30T16:44:21.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:21.976 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-30T16:44:22.061 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-30T16:44:22.150 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that when one clone is in progress, one progress bar is printed 2024-05-30T16:44:22.150 INFO:tasks.cephfs_test_runner:test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T16:44:22.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:44:22.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.368+0000 7f33ab000640 1 Processor -- start 2024-05-30T16:44:22.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.368+0000 7f33ab000640 1 -- start start 2024-05-30T16:44:22.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.370+0000 7f33ab000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a406bb70 0x7f33a409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.370+0000 7f33ab000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a40ae220 con 0x7f33a4073230 2024-05-30T16:44:22.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.370+0000 7f33ab000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a40cfef0 con 0x7f33a406bf50 2024-05-30T16:44:22.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.370+0000 7f33ab000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33a40d01d0 con 0x7f33a406bb70 2024-05-30T16:44:22.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.370+0000 7f33a9200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f33a4073230 0x7f33a415c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56708/0 (socket says 172.21.15.67:56708) 2024-05-30T16:44:22.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.370+0000 7f33a9200640 1 -- 172.21.15.67:0/827491335 learned_addr learned my addr 172.21.15.67:0/827491335 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.370+0000 7f33a9c00640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a406bb70 0x7f33a409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.370+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3063275114 0 0) 0x7f3390001110 con 0x7f33a4073230 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f338c003520 con 0x7f33a4073230 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3433860319 0 0) 0x7f338c003520 con 0x7f33a4073230 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a406bb70 msgr2=0x7f33a409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a406bb70 0x7f33a409aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 >> v1:172.21.15.169:6789/0 conn(0x7f33a406bf50 legacy=0x7f33a40b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a40d04b0 con 0x7f33a4073230 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f3390001840 con 0x7f33a4073230 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 >> v1:172.21.15.67:6789/0 conn(0x7f33a4073230 legacy=0x7f33a415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f338c0039e0 0x7f338c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f338c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a406bb70 0x7f338c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40d01d0 con 0x7f338c004800 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f338c00baa0 con 0x7f338c0039e0 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f338c00bca0 con 0x7f33a406bb70 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.371+0000 7f33a9200640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f338c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.372+0000 7f33a9c00640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f338c0039e0 0x7f338c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.372+0000 7f33aa600640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a406bb70 0x7f338c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.372+0000 7f33a9200640 1 -- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a406bb70 msgr2=0x7f338c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.372+0000 7f33a9200640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a406bb70 0x7f338c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.372+0000 7f33a9200640 1 -- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f338c0039e0 msgr2=0x7f338c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.372+0000 7f33a9200640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f338c0039e0 0x7f338c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.372+0000 7f33a9200640 1 -- 172.21.15.67:0/827491335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c003520 con 0x7f338c004800 2024-05-30T16:44:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.372+0000 7f33a9c00640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f338c0039e0 0x7f338c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:22.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.372+0000 7f33aa600640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a406bb70 0x7f338c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.373+0000 7f33a9200640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f338c004c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3390001eb0 tx=0x7f338c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53da30c764098cc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.373+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390036070 con 0x7f338c004800 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.373+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3390005af0 con 0x7f338c004800 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.373+0000 7f33a3e00640 1 -- 172.21.15.67:0/827491335 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390007e30 con 0x7f338c004800 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.374+0000 7f33ab000640 1 -- 172.21.15.67:0/827491335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 msgr2=0x7f338c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.374+0000 7f33ab000640 1 --2- 172.21.15.67:0/827491335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f338c004c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3390001eb0 tx=0x7f338c005830 comp rx=0 tx=0).stop 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.374+0000 7f33aa600640 1 -- 172.21.15.67:0/827491335 reap_dead start 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.374+0000 7f33ab000640 1 -- 172.21.15.67:0/827491335 shutdown_connections 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.374+0000 7f33ab000640 1 -- 172.21.15.67:0/827491335 >> 172.21.15.67:0/827491335 conn(0x7f33a40b6980 msgr2=0x7f33a40b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.374+0000 7f33ab000640 1 -- 172.21.15.67:0/827491335 shutdown_connections 2024-05-30T16:44:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.374+0000 7f33ab000640 1 -- 172.21.15.67:0/827491335 wait complete. 2024-05-30T16:44:22.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.376+0000 7f33ab000640 1 Processor -- start 2024-05-30T16:44:22.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.376+0000 7f33ab000640 1 -- start start 2024-05-30T16:44:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.377+0000 7f33ab000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f33a41514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.377+0000 7f33ab000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33a4153d70 0x7f33a4155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.377+0000 7f33ab000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a41552d0 0x7f33a414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.377+0000 7f33ab000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40b4dd0 con 0x7f338c004800 2024-05-30T16:44:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.377+0000 7f33ab000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40b4c50 con 0x7f33a4153d70 2024-05-30T16:44:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.377+0000 7f33ab000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33a415c7e0 con 0x7f33a41552d0 2024-05-30T16:44:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.377+0000 7f33a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f33a41514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.377+0000 7f33a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f33a41514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49132/0 (socket says 172.21.15.67:49132) 2024-05-30T16:44:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.377+0000 7f33a9c00640 1 -- 172.21.15.67:0/2615202732 learned_addr learned my addr 172.21.15.67:0/2615202732 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33a9200640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33a4153d70 0x7f33a4155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33aa600640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a41552d0 0x7f33a414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33a9c00640 1 -- 172.21.15.67:0/2615202732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a41552d0 msgr2=0x7f33a414f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33a9c00640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a41552d0 0x7f33a414f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33a9c00640 1 -- 172.21.15.67:0/2615202732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33a4153d70 msgr2=0x7f33a4155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33a9c00640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33a4153d70 0x7f33a4155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33a9c00640 1 -- 172.21.15.67:0/2615202732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a4154b10 con 0x7f338c004800 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33a9200640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33a4153d70 0x7f33a4155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33a9c00640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f33a41514f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f33980108b0 tx=0x7f3398010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.378+0000 7f33a1600640 1 -- 172.21.15.67:0/2615202732 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339800f040 con 0x7f338c004800 2024-05-30T16:44:22.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.379+0000 7f33a1600640 1 -- 172.21.15.67:0/2615202732 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339800eb00 con 0x7f338c004800 2024-05-30T16:44:22.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.379+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a41567b0 con 0x7f338c004800 2024-05-30T16:44:22.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.379+0000 7f33a1600640 1 -- 172.21.15.67:0/2615202732 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339801c400 con 0x7f338c004800 2024-05-30T16:44:22.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.379+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a4154510 con 0x7f338c004800 2024-05-30T16:44:22.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.380+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a40690f0 con 0x7f338c004800 2024-05-30T16:44:22.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.381+0000 7f33a1600640 1 -- 172.21.15.67:0/2615202732 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f339801c5a0 con 0x7f338c004800 2024-05-30T16:44:22.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.382+0000 7f33a1600640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338406c8f0 0x7f338406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.383+0000 7f33a1600640 1 -- 172.21.15.67:0/2615202732 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f339808c1f0 con 0x7f338c004800 2024-05-30T16:44:22.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.383+0000 7f33a9200640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338406c8f0 0x7f338406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.383+0000 7f33a9200640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338406c8f0 0x7f338406edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3390033000 tx=0x7f339002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:22.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.387+0000 7f33a1600640 1 -- 172.21.15.67:0/2615202732 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3398018030 con 0x7f338c004800 2024-05-30T16:44:22.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.593+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33a40ced90 con 0x7f338c004800 2024-05-30T16:44:22.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.596+0000 7f33a1600640 1 -- 172.21.15.67:0/2615202732 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16593 (secure 0 0 0) 0x7f339805a620 con 0x7f338c004800 2024-05-30T16:44:22.597 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:22.597 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:43:53.858729+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:43:50.496509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-30T16:43:51.276872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:44:22.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338406c8f0 msgr2=0x7f338406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338406c8f0 0x7f338406edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3390033000 tx=0x7f339002e000 comp rx=0 tx=0).stop 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 msgr2=0x7f33a41514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f33a41514f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f33980108b0 tx=0x7f3398010d70 comp rx=0 tx=0).stop 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 shutdown_connections 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338406c8f0 0x7f338406edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33a41552d0 0x7f33a414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33a4153d70 0x7f33a4155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 --2- 172.21.15.67:0/2615202732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f338c004800 0x7f33a41514f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 >> 172.21.15.67:0/2615202732 conn(0x7f33a40b6980 msgr2=0x7f33a40b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 shutdown_connections 2024-05-30T16:44:22.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.599+0000 7f33ab000640 1 -- 172.21.15.67:0/2615202732 wait complete. 2024-05-30T16:44:22.620 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-30T16:43:40.009725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-30T16:43:50.496509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-30T16:43:51.276872+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-30T16:44:22.621 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-30T16:44:22.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.827+0000 7f844be00640 1 Processor -- start 2024-05-30T16:44:22.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.828+0000 7f844be00640 1 -- start start 2024-05-30T16:44:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.829+0000 7f844be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f844c15e870 0x7f844c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.829+0000 7f844be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f844c0d14d0 0x7f844c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.829+0000 7f844be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 0x7f844c162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.829+0000 7f844be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f844c0d2150 con 0x7f844c15e870 2024-05-30T16:44:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.829+0000 7f844be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f844c0cfd00 con 0x7f844c0d14d0 2024-05-30T16:44:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.829+0000 7f844be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f844c15ec50 con 0x7f844c0cf870 2024-05-30T16:44:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f844c15e870 0x7f844c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f844c15e870 0x7f844c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49146/0 (socket says 172.21.15.67:49146) 2024-05-30T16:44:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844aa00640 1 -- 172.21.15.67:0/3361023063 learned_addr learned my addr 172.21.15.67:0/3361023063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844b400640 1 --2- 172.21.15.67:0/3361023063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 0x7f844c162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844a000640 1 --2- 172.21.15.67:0/3361023063 >> v2:172.21.15.169:3300/0 conn(0x7f844c0d14d0 0x7f844c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844aa00640 1 -- 172.21.15.67:0/3361023063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 msgr2=0x7f844c162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844aa00640 1 --2- 172.21.15.67:0/3361023063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 0x7f844c162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844aa00640 1 -- 172.21.15.67:0/3361023063 >> v2:172.21.15.169:3300/0 conn(0x7f844c0d14d0 msgr2=0x7f844c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844aa00640 1 --2- 172.21.15.67:0/3361023063 >> v2:172.21.15.169:3300/0 conn(0x7f844c0d14d0 0x7f844c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.830+0000 7f844aa00640 1 -- 172.21.15.67:0/3361023063 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c0cff80 con 0x7f844c15e870 2024-05-30T16:44:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.832+0000 7f844aa00640 1 --2- 172.21.15.67:0/3361023063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 0x7f844c0d0f90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f844c08e520 tx=0x7f843c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688ce9df546a6af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.832+0000 7f8449600640 1 -- 172.21.15.67:0/3361023063 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c037070 con 0x7f844c15e870 2024-05-30T16:44:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.832+0000 7f8449600640 1 -- 172.21.15.67:0/3361023063 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f843c032440 con 0x7f844c15e870 2024-05-30T16:44:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.832+0000 7f8449600640 1 -- 172.21.15.67:0/3361023063 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c032810 con 0x7f844c15e870 2024-05-30T16:44:22.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.833+0000 7f844be00640 1 -- 172.21.15.67:0/3361023063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 msgr2=0x7f844c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.833+0000 7f844be00640 1 --2- 172.21.15.67:0/3361023063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 0x7f844c0d0f90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f844c08e520 tx=0x7f843c02a1e0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.833+0000 7f844be00640 1 -- 172.21.15.67:0/3361023063 shutdown_connections 2024-05-30T16:44:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.833+0000 7f844be00640 1 --2- 172.21.15.67:0/3361023063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 0x7f844c162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.833+0000 7f844be00640 1 --2- 172.21.15.67:0/3361023063 >> v2:172.21.15.169:3300/0 conn(0x7f844c0d14d0 0x7f844c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.833+0000 7f844be00640 1 --2- 172.21.15.67:0/3361023063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 0x7f844c0d0f90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.833+0000 7f844be00640 1 -- 172.21.15.67:0/3361023063 >> 172.21.15.67:0/3361023063 conn(0x7f844c08bfc0 msgr2=0x7f844c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.833+0000 7f844be00640 1 -- 172.21.15.67:0/3361023063 shutdown_connections 2024-05-30T16:44:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.833+0000 7f844be00640 1 -- 172.21.15.67:0/3361023063 wait complete. 2024-05-30T16:44:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.835+0000 7f844be00640 1 Processor -- start 2024-05-30T16:44:22.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.835+0000 7f844be00640 1 -- start start 2024-05-30T16:44:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.836+0000 7f844be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 0x7f844c0b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.836+0000 7f844be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f844c0d14d0 0x7f844c0cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.836+0000 7f844be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 0x7f844c0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.836+0000 7f844be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f844c0d0820 con 0x7f844c15e870 2024-05-30T16:44:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.836+0000 7f844be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f844c0d0540 con 0x7f844c0d14d0 2024-05-30T16:44:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.836+0000 7f844be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f844c0d4fb0 con 0x7f844c0cf870 2024-05-30T16:44:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.836+0000 7f844b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 0x7f844c0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.836+0000 7f844b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 0x7f844c0cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49148/0 (socket says 172.21.15.67:49148) 2024-05-30T16:44:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.836+0000 7f844b400640 1 -- 172.21.15.67:0/1910016044 learned_addr learned my addr 172.21.15.67:0/1910016044 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844aa00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 0x7f844c0b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844a000640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f844c0d14d0 0x7f844c0cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844b400640 1 -- 172.21.15.67:0/1910016044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 msgr2=0x7f844c0b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844b400640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 0x7f844c0b7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844b400640 1 -- 172.21.15.67:0/1910016044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f844c0d14d0 msgr2=0x7f844c0cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844b400640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f844c0d14d0 0x7f844c0cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844b400640 1 -- 172.21.15.67:0/1910016044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c0b8600 con 0x7f844c15e870 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844a000640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f844c0d14d0 0x7f844c0cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844b400640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 0x7f844c0cc2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8444007e60 tx=0x7f844400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f8442a00640 1 -- 172.21.15.67:0/1910016044 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444014420 con 0x7f844c15e870 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f8442a00640 1 -- 172.21.15.67:0/1910016044 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8444019070 con 0x7f844c15e870 2024-05-30T16:44:22.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f8442a00640 1 -- 172.21.15.67:0/1910016044 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444014820 con 0x7f844c15e870 2024-05-30T16:44:22.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.837+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c0b88e0 con 0x7f844c15e870 2024-05-30T16:44:22.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.838+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c0b8ea0 con 0x7f844c15e870 2024-05-30T16:44:22.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.839+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844c0689e0 con 0x7f844c15e870 2024-05-30T16:44:22.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.840+0000 7f8442a00640 1 -- 172.21.15.67:0/1910016044 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84440149c0 con 0x7f844c15e870 2024-05-30T16:44:22.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.840+0000 7f8442a00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f842c06c8f0 0x7f842c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:22.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.841+0000 7f8442a00640 1 -- 172.21.15.67:0/1910016044 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f844408c470 con 0x7f844c15e870 2024-05-30T16:44:22.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.841+0000 7f844aa00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f842c06c8f0 0x7f842c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:22.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.841+0000 7f844aa00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f842c06c8f0 0x7f842c06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f843c034000 tx=0x7f843c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:22.847+0000 7f8442a00640 1 -- 172.21.15.67:0/1910016044 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f844405a800 con 0x7f844c15e870 2024-05-30T16:44:23.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.051+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f844c0ba500 con 0x7f844c15e870 2024-05-30T16:44:23.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.052+0000 7f8442a00640 1 -- 172.21.15.67:0/1910016044 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f844405a100 con 0x7f844c15e870 2024-05-30T16:44:23.052 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-05-30T16:44:23.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f842c06c8f0 msgr2=0x7f842c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f842c06c8f0 0x7f842c06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f843c034000 tx=0x7f843c02e000 comp rx=0 tx=0).stop 2024-05-30T16:44:23.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 msgr2=0x7f844c0cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 0x7f844c0cc2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8444007e60 tx=0x7f844400e4b0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 shutdown_connections 2024-05-30T16:44:23.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f842c06c8f0 0x7f842c06edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844c15e870 0x7f844c0cc2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f844c0d14d0 0x7f844c0cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 --2- 172.21.15.67:0/1910016044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f844c0cf870 0x7f844c0b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 >> 172.21.15.67:0/1910016044 conn(0x7f844c08bfc0 msgr2=0x7f844c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:23.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 shutdown_connections 2024-05-30T16:44:23.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.055+0000 7f844be00640 1 -- 172.21.15.67:0/1910016044 wait complete. 2024-05-30T16:44:23.076 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-30T16:44:23.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.299+0000 7f50f2c00640 1 Processor -- start 2024-05-30T16:44:23.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.299+0000 7f50f2c00640 1 -- start start 2024-05-30T16:44:23.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.301+0000 7f50f2c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f50ec0b4dd0 0x7f50ec0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.301+0000 7f50f2c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50ec0b3580 con 0x7f50ec0ce1f0 2024-05-30T16:44:23.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.301+0000 7f50f2c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50ec15f490 con 0x7f50ec0ce5d0 2024-05-30T16:44:23.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.301+0000 7f50f2c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f50ec15f770 con 0x7f50ec0b4dd0 2024-05-30T16:44:23.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.301+0000 7f50f1800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f50ec0ce1f0 0x7f50ec15ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56722/0 (socket says 172.21.15.67:56722) 2024-05-30T16:44:23.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.301+0000 7f50f1800640 1 -- 172.21.15.67:0/1565387273 learned_addr learned my addr 172.21.15.67:0/1565387273 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:23.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.301+0000 7f50f2200640 1 --2- 172.21.15.67:0/1565387273 >> v2:172.21.15.169:3300/0 conn(0x7f50ec0b4dd0 0x7f50ec0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.301+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415629549 0 0) 0x7f50ec0b3580 con 0x7f50ec0ce1f0 2024-05-30T16:44:23.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50d4003520 con 0x7f50ec0ce1f0 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 160795903 0 0) 0x7f50d4003520 con 0x7f50ec0ce1f0 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 >> v2:172.21.15.169:3300/0 conn(0x7f50ec0b4dd0 msgr2=0x7f50ec0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 --2- 172.21.15.67:0/1565387273 >> v2:172.21.15.169:3300/0 conn(0x7f50ec0b4dd0 0x7f50ec0ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 >> v1:172.21.15.169:6789/0 conn(0x7f50ec0ce5d0 legacy=0x7f50ec0b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec15fa50 con 0x7f50ec0ce1f0 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f50e0001670 con 0x7f50ec0ce1f0 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 >> v1:172.21.15.67:6789/0 conn(0x7f50ec0ce1f0 legacy=0x7f50ec15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50d4003a00 0x7f50d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50d4004800 0x7f50d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d40053c0 0x7f50d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec15f770 con 0x7f50d4004800 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50d400c2a0 con 0x7f50d4003a00 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50d400c4a0 con 0x7f50d40053c0 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.302+0000 7f50f0e00640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50d4004800 0x7f50d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.303+0000 7f50f1800640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50d4003a00 0x7f50d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.303+0000 7f50f2200640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d40053c0 0x7f50d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.303+0000 7f50f0e00640 1 -- 172.21.15.67:0/1565387273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d40053c0 msgr2=0x7f50d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:23.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.303+0000 7f50f0e00640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d40053c0 0x7f50d400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.303+0000 7f50f0e00640 1 -- 172.21.15.67:0/1565387273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50d4003a00 msgr2=0x7f50d4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:23.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.303+0000 7f50f0e00640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50d4003a00 0x7f50d4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.303+0000 7f50f0e00640 1 -- 172.21.15.67:0/1565387273 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d4003520 con 0x7f50d4004800 2024-05-30T16:44:23.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.303+0000 7f50f2200640 1 -- 172.21.15.67:0/1565387273 reap_dead start 2024-05-30T16:44:23.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.304+0000 7f50f0e00640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50d4004800 0x7f50d4004c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f50d8001750 tx=0x7f50d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5810e98e1cac42 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:23.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.304+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8001370 con 0x7f50d4004800 2024-05-30T16:44:23.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.304+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d80052d0 con 0x7f50d4004800 2024-05-30T16:44:23.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.304+0000 7f50ebe00640 1 -- 172.21.15.67:0/1565387273 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d802abe0 con 0x7f50d4004800 2024-05-30T16:44:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.305+0000 7f50f2c00640 1 -- 172.21.15.67:0/1565387273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50d4004800 msgr2=0x7f50d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.305+0000 7f50f2c00640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50d4004800 0x7f50d4004c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f50d8001750 tx=0x7f50d4005830 comp rx=0 tx=0).stop 2024-05-30T16:44:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.305+0000 7f50f2c00640 1 -- 172.21.15.67:0/1565387273 shutdown_connections 2024-05-30T16:44:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.305+0000 7f50f2c00640 1 --2- 172.21.15.67:0/1565387273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50d4004800 0x7f50d4004c80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.305+0000 7f50f2c00640 1 -- 172.21.15.67:0/1565387273 >> 172.21.15.67:0/1565387273 conn(0x7f50ec06b900 msgr2=0x7f50ec06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.305+0000 7f50f2c00640 1 -- 172.21.15.67:0/1565387273 shutdown_connections 2024-05-30T16:44:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.305+0000 7f50f2c00640 1 -- 172.21.15.67:0/1565387273 wait complete. 2024-05-30T16:44:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.307+0000 7f50f2c00640 1 Processor -- start 2024-05-30T16:44:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.307+0000 7f50f2c00640 1 -- start start 2024-05-30T16:44:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.308+0000 7f50f2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d4004800 0x7f50ec144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.308+0000 7f50f2c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ec1f5030 0x7f50ec1f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.308+0000 7f50f2c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ec0af530 0x7f50ec0af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.308+0000 7f50f2c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec15edc0 con 0x7f50ec0af530 2024-05-30T16:44:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.308+0000 7f50f2c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec15ec40 con 0x7f50ec1f5030 2024-05-30T16:44:23.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.308+0000 7f50f2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50ec15ef40 con 0x7f50d4004800 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.308+0000 7f50f2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ec0af530 0x7f50ec0af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.308+0000 7f50f2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ec0af530 0x7f50ec0af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49174/0 (socket says 172.21.15.67:49174) 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.308+0000 7f50f2200640 1 -- 172.21.15.67:0/947232723 learned_addr learned my addr 172.21.15.67:0/947232723 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f0e00640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ec1f5030 0x7f50ec1f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f1800640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d4004800 0x7f50ec144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f2200640 1 -- 172.21.15.67:0/947232723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d4004800 msgr2=0x7f50ec144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f2200640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d4004800 0x7f50ec144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f2200640 1 -- 172.21.15.67:0/947232723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ec1f5030 msgr2=0x7f50ec1f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f2200640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ec1f5030 0x7f50ec1f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f2200640 1 -- 172.21.15.67:0/947232723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec0b1750 con 0x7f50ec0af530 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f0e00640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ec1f5030 0x7f50ec1f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f2200640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ec0af530 0x7f50ec0af9b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0cf5b0 tx=0x7f50e4009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50e9600640 1 -- 172.21.15.67:0/947232723 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4019070 con 0x7f50ec0af530 2024-05-30T16:44:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50e9600640 1 -- 172.21.15.67:0/947232723 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e4007d10 con 0x7f50ec0af530 2024-05-30T16:44:23.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50e9600640 1 -- 172.21.15.67:0/947232723 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4014640 con 0x7f50ec0af530 2024-05-30T16:44:23.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.309+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec0b19f0 con 0x7f50ec0af530 2024-05-30T16:44:23.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.310+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec0b0060 con 0x7f50ec0af530 2024-05-30T16:44:23.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.311+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ec0689e0 con 0x7f50ec0af530 2024-05-30T16:44:23.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.312+0000 7f50e9600640 1 -- 172.21.15.67:0/947232723 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50e401c070 con 0x7f50ec0af530 2024-05-30T16:44:23.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.313+0000 7f50e9600640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f50cc06c920 0x7f50cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.313+0000 7f50e9600640 1 -- 172.21.15.67:0/947232723 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f50e408b750 con 0x7f50ec0af530 2024-05-30T16:44:23.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.313+0000 7f50f1800640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f50cc06c920 0x7f50cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.314+0000 7f50f1800640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f50cc06c920 0x7f50cc06ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f50e0000c00 tx=0x7f50e0001c80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:23.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.317+0000 7f50e9600640 1 -- 172.21.15.67:0/947232723 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50e4059ae0 con 0x7f50ec0af530 2024-05-30T16:44:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.519+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f50ec15e350 con 0x7f50ec0af530 2024-05-30T16:44:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.520+0000 7f50e9600640 1 -- 172.21.15.67:0/947232723 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f50e40593e0 con 0x7f50ec0af530 2024-05-30T16:44:23.521 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-05-30T16:44:23.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f50cc06c920 msgr2=0x7f50cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f50cc06c920 0x7f50cc06ede0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f50e0000c00 tx=0x7f50e0001c80 comp rx=0 tx=0).stop 2024-05-30T16:44:23.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ec0af530 msgr2=0x7f50ec0af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ec0af530 0x7f50ec0af9b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0cf5b0 tx=0x7f50e4009d40 comp rx=0 tx=0).stop 2024-05-30T16:44:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 shutdown_connections 2024-05-30T16:44:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f50cc06c920 0x7f50cc06ede0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50ec0af530 0x7f50ec0af9b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ec1f5030 0x7f50ec1f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 --2- 172.21.15.67:0/947232723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d4004800 0x7f50ec144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.522+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 >> 172.21.15.67:0/947232723 conn(0x7f50ec06b900 msgr2=0x7f50ec0adbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.523+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 shutdown_connections 2024-05-30T16:44:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.523+0000 7f50f2c00640 1 -- 172.21.15.67:0/947232723 wait complete. 2024-05-30T16:44:23.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-30T16:44:23.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.758+0000 7f768bc00640 1 Processor -- start 2024-05-30T16:44:23.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.758+0000 7f768bc00640 1 -- start start 2024-05-30T16:44:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.759+0000 7f768bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 0x7f768406f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.759+0000 7f768bc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f768406f8c0 0x7f76840b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f768bc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f76840b4b10 0x7f76840725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f768bc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f768406e2e0 con 0x7f768406f8c0 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f768bc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f768406e5c0 con 0x7f76840b4b10 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f768bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f768406e8a0 con 0x7f768415abe0 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7683e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f768406f8c0 0x7f76840b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7683e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f768406f8c0 0x7f76840b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49188/0 (socket says 172.21.15.67:49188) 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7683e00640 1 -- 172.21.15.67:0/3665905573 learned_addr learned my addr 172.21.15.67:0/3665905573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7689200640 1 --2- 172.21.15.67:0/3665905573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 0x7f768406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7683e00640 1 -- 172.21.15.67:0/3665905573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 msgr2=0x7f768406f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7689c00640 1 --2- 172.21.15.67:0/3665905573 >> v2:172.21.15.169:3300/0 conn(0x7f76840b4b10 0x7f76840725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7683e00640 1 --2- 172.21.15.67:0/3665905573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 0x7f768406f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7683e00640 1 -- 172.21.15.67:0/3665905573 >> v2:172.21.15.169:3300/0 conn(0x7f76840b4b10 msgr2=0x7f76840725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7683e00640 1 --2- 172.21.15.67:0/3665905573 >> v2:172.21.15.169:3300/0 conn(0x7f76840b4b10 0x7f76840725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.760+0000 7f7683e00640 1 -- 172.21.15.67:0/3665905573 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768406eb80 con 0x7f768406f8c0 2024-05-30T16:44:23.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.761+0000 7f7683e00640 1 --2- 172.21.15.67:0/3665905573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f768406f8c0 0x7f76840b45d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76840b5370 tx=0x7f766c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502ddd41257b90a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.762+0000 7f7683400640 1 -- 172.21.15.67:0/3665905573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c037070 con 0x7f768406f8c0 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.762+0000 7f7683400640 1 -- 172.21.15.67:0/3665905573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c032440 con 0x7f768406f8c0 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.762+0000 7f7683400640 1 -- 172.21.15.67:0/3665905573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c032810 con 0x7f768406f8c0 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.763+0000 7f768bc00640 1 -- 172.21.15.67:0/3665905573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f768406f8c0 msgr2=0x7f76840b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.763+0000 7f768bc00640 1 --2- 172.21.15.67:0/3665905573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f768406f8c0 0x7f76840b45d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76840b5370 tx=0x7f766c02a210 comp rx=0 tx=0).stop 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.763+0000 7f768bc00640 1 -- 172.21.15.67:0/3665905573 shutdown_connections 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.763+0000 7f768bc00640 1 --2- 172.21.15.67:0/3665905573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f768406f8c0 0x7f76840b45d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.763+0000 7f768bc00640 1 --2- 172.21.15.67:0/3665905573 >> v2:172.21.15.169:3300/0 conn(0x7f76840b4b10 0x7f76840725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.763+0000 7f768bc00640 1 --2- 172.21.15.67:0/3665905573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 0x7f768406f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.763+0000 7f768bc00640 1 -- 172.21.15.67:0/3665905573 >> 172.21.15.67:0/3665905573 conn(0x7f768406bd20 msgr2=0x7f768406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.763+0000 7f768bc00640 1 -- 172.21.15.67:0/3665905573 shutdown_connections 2024-05-30T16:44:23.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.763+0000 7f768bc00640 1 -- 172.21.15.67:0/3665905573 wait complete. 2024-05-30T16:44:23.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.764+0000 7f768bc00640 1 Processor -- start 2024-05-30T16:44:23.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.765+0000 7f768bc00640 1 -- start start 2024-05-30T16:44:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f768bc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f768406f8c0 0x7f76840b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f768bc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76840b4b10 0x7f76840989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f768bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 0x7f7684151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f768bc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f768406df60 con 0x7f76840b4b10 2024-05-30T16:44:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f768bc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f768415afc0 con 0x7f768406f8c0 2024-05-30T16:44:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f768bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7684072b00 con 0x7f768415abe0 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7683e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76840b4b10 0x7f76840989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7683e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76840b4b10 0x7f76840989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49190/0 (socket says 172.21.15.67:49190) 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7683e00640 1 -- 172.21.15.67:0/2816584492 learned_addr learned my addr 172.21.15.67:0/2816584492 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7689200640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f768406f8c0 0x7f76840b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7689c00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 0x7f7684151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7683e00640 1 -- 172.21.15.67:0/2816584492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 msgr2=0x7f7684151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7683e00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 0x7f7684151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7683e00640 1 -- 172.21.15.67:0/2816584492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f768406f8c0 msgr2=0x7f76840b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7683e00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f768406f8c0 0x7f76840b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7683e00640 1 -- 172.21.15.67:0/2816584492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7684095d20 con 0x7f76840b4b10 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.766+0000 7f7689200640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f768406f8c0 0x7f76840b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.767+0000 7f7683e00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76840b4b10 0x7f76840989d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f766c02a250 tx=0x7f766c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:23.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.767+0000 7f7680c00640 1 -- 172.21.15.67:0/2816584492 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c041020 con 0x7f76840b4b10 2024-05-30T16:44:23.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.767+0000 7f7680c00640 1 -- 172.21.15.67:0/2816584492 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c0306c0 con 0x7f76840b4b10 2024-05-30T16:44:23.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.767+0000 7f7680c00640 1 -- 172.21.15.67:0/2816584492 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c037040 con 0x7f76840b4b10 2024-05-30T16:44:23.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.767+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7684099b00 con 0x7f76840b4b10 2024-05-30T16:44:23.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.767+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7684142fd0 con 0x7f76840b4b10 2024-05-30T16:44:23.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.769+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76840689e0 con 0x7f76840b4b10 2024-05-30T16:44:23.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.769+0000 7f7680c00640 1 -- 172.21.15.67:0/2816584492 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f766c044020 con 0x7f76840b4b10 2024-05-30T16:44:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.770+0000 7f7680c00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f766006c8f0 0x7f766006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.771+0000 7f7680c00640 1 -- 172.21.15.67:0/2816584492 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f766c0ac2d0 con 0x7f76840b4b10 2024-05-30T16:44:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.771+0000 7f7689200640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f766006c8f0 0x7f766006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:23.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.771+0000 7f7689200640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f766006c8f0 0x7f766006edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7674006fd0 tx=0x7f7674008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:23.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.775+0000 7f7680c00640 1 -- 172.21.15.67:0/2816584492 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f766c04b030 con 0x7f76840b4b10 2024-05-30T16:44:23.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.981+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f768406dd40 con 0x7f76840b4b10 2024-05-30T16:44:23.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.982+0000 7f7680c00640 1 -- 172.21.15.67:0/2816584492 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f766c07a790 con 0x7f76840b4b10 2024-05-30T16:44:23.983 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-05-30T16:44:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f766006c8f0 msgr2=0x7f766006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f766006c8f0 0x7f766006edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7674006fd0 tx=0x7f7674008040 comp rx=0 tx=0).stop 2024-05-30T16:44:23.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76840b4b10 msgr2=0x7f76840989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76840b4b10 0x7f76840989d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f766c02a250 tx=0x7f766c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 shutdown_connections 2024-05-30T16:44:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f766006c8f0 0x7f766006edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f768415abe0 0x7f7684151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76840b4b10 0x7f76840989d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 --2- 172.21.15.67:0/2816584492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f768406f8c0 0x7f76840b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 >> 172.21.15.67:0/2816584492 conn(0x7f768406bd20 msgr2=0x7f76840b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 shutdown_connections 2024-05-30T16:44:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:23.985+0000 7f768bc00640 1 -- 172.21.15.67:0/2816584492 wait complete. 2024-05-30T16:44:24.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone' 2024-05-30T16:44:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.214+0000 7f8322400640 1 Processor -- start 2024-05-30T16:44:24.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.215+0000 7f8322400640 1 -- start start 2024-05-30T16:44:24.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.216+0000 7f8322400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f831c15e870 0x7f831c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:24.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.216+0000 7f8322400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cfa40 0x7f831c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:24.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.216+0000 7f8322400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f831c15ec50 con 0x7f831c0d14d0 2024-05-30T16:44:24.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.216+0000 7f8322400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f831c0d2150 con 0x7f831c15e870 2024-05-30T16:44:24.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.216+0000 7f8322400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f831c0cff80 con 0x7f831c0cfa40 2024-05-30T16:44:24.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.217+0000 7f831be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f831c0d14d0 0x7f831c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56732/0 (socket says 172.21.15.67:56732) 2024-05-30T16:44:24.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.217+0000 7f831be00640 1 -- 172.21.15.67:0/2849760863 learned_addr learned my addr 172.21.15.67:0/2849760863 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.217+0000 7f8321000640 1 --2- 172.21.15.67:0/2849760863 >> v2:172.21.15.169:3300/0 conn(0x7f831c15e870 0x7f831c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.217+0000 7f8321a00640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cfa40 0x7f831c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.217+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 872296178 0 0) 0x7f831c15ec50 con 0x7f831c0d14d0 2024-05-30T16:44:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.217+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8300003520 con 0x7f831c0d14d0 2024-05-30T16:44:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2505667484 0 0) 0x7f8300003520 con 0x7f831c0d14d0 2024-05-30T16:44:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cfa40 msgr2=0x7f831c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cfa40 0x7f831c0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 >> v2:172.21.15.169:3300/0 conn(0x7f831c15e870 msgr2=0x7f831c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 --2- 172.21.15.67:0/2849760863 >> v2:172.21.15.169:3300/0 conn(0x7f831c15e870 0x7f831c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c0d0260 con 0x7f831c0d14d0 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f83080017f0 con 0x7f831c0d14d0 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 >> v1:172.21.15.67:6789/0 conn(0x7f831c0d14d0 legacy=0x7f831c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f8300003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8300004800 0x7f8300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cfa40 0x7f83000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0d2150 con 0x7f83000039b0 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0cff80 con 0x7f8300004800 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f830000ba10 con 0x7f831c0cfa40 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f8321000640 1 --2- 172.21.15.67:0/2849760863 >> v2:172.21.15.169:3300/0 conn(0x7f831c15e870 0x7f831c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f8321000640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f8300003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.218+0000 7f831be00640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8300004800 0x7f8300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.219+0000 7f8321a00640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cfa40 0x7f83000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.219+0000 7f8321000640 1 -- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cfa40 msgr2=0x7f83000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.219+0000 7f8321000640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cfa40 0x7f83000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.219+0000 7f8321000640 1 -- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8300004800 msgr2=0x7f8300004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.219+0000 7f8321000640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8300004800 0x7f8300004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.219+0000 7f8321000640 1 -- 172.21.15.67:0/2849760863 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8300003520 con 0x7f83000039b0 2024-05-30T16:44:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.219+0000 7f8321a00640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cfa40 0x7f83000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:24.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.220+0000 7f8321000640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f8300003dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098a0 tx=0x7f8300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b531e51b68ac1380 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:24.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.220+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c037070 con 0x7f83000039b0 2024-05-30T16:44:24.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.220+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c02ad20 con 0x7f83000039b0 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.220+0000 7f831b400640 1 -- 172.21.15.67:0/2849760863 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c00f040 con 0x7f83000039b0 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.221+0000 7f8322400640 1 -- 172.21.15.67:0/2849760863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 msgr2=0x7f8300003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.221+0000 7f8322400640 1 --2- 172.21.15.67:0/2849760863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f8300003dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098a0 tx=0x7f8300005830 comp rx=0 tx=0).stop 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.221+0000 7f8321a00640 1 -- 172.21.15.67:0/2849760863 reap_dead start 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.221+0000 7f8322400640 1 -- 172.21.15.67:0/2849760863 shutdown_connections 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.221+0000 7f8322400640 1 -- 172.21.15.67:0/2849760863 >> 172.21.15.67:0/2849760863 conn(0x7f831c08bfc0 msgr2=0x7f831c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.221+0000 7f8322400640 1 -- 172.21.15.67:0/2849760863 shutdown_connections 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.221+0000 7f8322400640 1 -- 172.21.15.67:0/2849760863 wait complete. 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.223+0000 7f8322400640 1 Processor -- start 2024-05-30T16:44:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.223+0000 7f8322400640 1 -- start start 2024-05-30T16:44:24.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.224+0000 7f8322400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f831c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:24.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.224+0000 7f8322400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f831c0cb650 0x7f831c0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:24.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.224+0000 7f8322400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cc010 0x7f831c0b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:24.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.224+0000 7f8322400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0d4fb0 con 0x7f83000039b0 2024-05-30T16:44:24.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.224+0000 7f8322400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0d0820 con 0x7f831c0cb650 2024-05-30T16:44:24.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.224+0000 7f8322400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f831c0d25e0 con 0x7f831c0cc010 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.224+0000 7f8321000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f831c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.224+0000 7f8321000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f831c159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49208/0 (socket says 172.21.15.67:49208) 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.224+0000 7f8321000640 1 -- 172.21.15.67:0/45243981 learned_addr learned my addr 172.21.15.67:0/45243981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8321000640 1 -- 172.21.15.67:0/45243981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cc010 msgr2=0x7f831c0b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8321000640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cc010 0x7f831c0b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8321000640 1 -- 172.21.15.67:0/45243981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f831c0cb650 msgr2=0x7f831c0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f831be00640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f831c0cb650 0x7f831c0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8321000640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f831c0cb650 0x7f831c0cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8321000640 1 -- 172.21.15.67:0/45243981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c0b7340 con 0x7f83000039b0 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8321000640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f831c159620 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f830c02a810 tx=0x7f830c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8318c00640 1 -- 172.21.15.67:0/45243981 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c040020 con 0x7f83000039b0 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8318c00640 1 -- 172.21.15.67:0/45243981 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c038040 con 0x7f83000039b0 2024-05-30T16:44:24.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8318c00640 1 -- 172.21.15.67:0/45243981 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c037070 con 0x7f83000039b0 2024-05-30T16:44:24.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.225+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c0b75e0 con 0x7f83000039b0 2024-05-30T16:44:24.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.226+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c0b7ba0 con 0x7f83000039b0 2024-05-30T16:44:24.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.227+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c0cc460 con 0x7f83000039b0 2024-05-30T16:44:24.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.228+0000 7f8318c00640 1 -- 172.21.15.67:0/45243981 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f830c043020 con 0x7f83000039b0 2024-05-30T16:44:24.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.228+0000 7f8318c00640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82fc06c8f0 0x7f82fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:24.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.229+0000 7f8318c00640 1 -- 172.21.15.67:0/45243981 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f830c0abb10 con 0x7f83000039b0 2024-05-30T16:44:24.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.229+0000 7f831be00640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82fc06c8f0 0x7f82fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:24.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.229+0000 7f831be00640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82fc06c8f0 0x7f82fc06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f831c0a2440 tx=0x7f8308009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:24.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.233+0000 7f8318c00640 1 -- 172.21.15.67:0/45243981 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f830c04a030 con 0x7f83000039b0 2024-05-30T16:44:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:24.500+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]} v 0) -- 0x7f831c0689e0 con 0x7f83000039b0 2024-05-30T16:44:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.154+0000 7f8318c00640 1 -- 172.21.15.67:0/45243981 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]}]=0 v52) ==== 163+0+0 (secure 0 0 0) 0x7f830c00f000 con 0x7f83000039b0 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82fc06c8f0 msgr2=0x7f82fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82fc06c8f0 0x7f82fc06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f831c0a2440 tx=0x7f8308009040 comp rx=0 tx=0).stop 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 msgr2=0x7f831c159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f831c159620 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f830c02a810 tx=0x7f830c004730 comp rx=0 tx=0).stop 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 shutdown_connections 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f82fc06c8f0 0x7f82fc06edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831c0cc010 0x7f831c0b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f831c0cb650 0x7f831c0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 --2- 172.21.15.67:0/45243981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83000039b0 0x7f831c159620 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 >> 172.21.15.67:0/45243981 conn(0x7f831c08bfc0 msgr2=0x7f831c0b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 shutdown_connections 2024-05-30T16:44:25.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.159+0000 7f8322400640 1 -- 172.21.15.67:0/45243981 wait complete. 2024-05-30T16:44:25.182 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:44:25.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.401+0000 7f265b000640 1 Processor -- start 2024-05-30T16:44:25.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.401+0000 7f265b000640 1 -- start start 2024-05-30T16:44:25.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.402+0000 7f265b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540d16a0 0x7f26540cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:25.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.403+0000 7f265b000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265415ec50 con 0x7f26540cf870 2024-05-30T16:44:25.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.403+0000 7f265b000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26540cff80 con 0x7f265415e870 2024-05-30T16:44:25.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.403+0000 7f265b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26540d2150 con 0x7f26540d16a0 2024-05-30T16:44:25.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.403+0000 7f265a600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f26540cf870 0x7f26540d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56744/0 (socket says 172.21.15.67:56744) 2024-05-30T16:44:25.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.403+0000 7f265a600640 1 -- 172.21.15.67:0/4172990542 learned_addr learned my addr 172.21.15.67:0/4172990542 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.403+0000 7f2659200640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540d16a0 0x7f26540cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4214124040 0 0) 0x7f265415ec50 con 0x7f26540cf870 2024-05-30T16:44:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f263c003520 con 0x7f26540cf870 2024-05-30T16:44:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 497178071 0 0) 0x7f26540cff80 con 0x7f265415e870 2024-05-30T16:44:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f265415ec50 con 0x7f265415e870 2024-05-30T16:44:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3665011703 0 0) 0x7f263c003520 con 0x7f26540cf870 2024-05-30T16:44:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540d16a0 msgr2=0x7f26540cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540d16a0 0x7f26540cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 >> v1:172.21.15.169:6789/0 conn(0x7f265415e870 legacy=0x7f26540d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26540d0260 con 0x7f26540cf870 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f264c0019f0 con 0x7f26540cf870 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 >> v1:172.21.15.67:6789/0 conn(0x7f26540cf870 legacy=0x7f26540d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f263c003b70 0x7f263c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540d16a0 0x7f263c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f263c004d40 0x7f263c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26540d2150 con 0x7f263c004d40 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f263c00ba70 con 0x7f263c003b70 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.404+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f263c00bc70 con 0x7f26540d16a0 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.405+0000 7f265a600640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f263c004d40 0x7f263c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.405+0000 7f2659c00640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f263c003b70 0x7f263c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.405+0000 7f2659200640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540d16a0 0x7f263c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.405+0000 7f265a600640 1 -- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540d16a0 msgr2=0x7f263c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:25.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.405+0000 7f265a600640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540d16a0 0x7f263c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.405+0000 7f265a600640 1 -- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f263c003b70 msgr2=0x7f263c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:25.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.405+0000 7f265a600640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f263c003b70 0x7f263c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.405+0000 7f265a600640 1 -- 172.21.15.67:0/4172990542 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c003520 con 0x7f263c004d40 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.407+0000 7f265a600640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f263c004d40 0x7f263c0051c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f264c00a720 tx=0x7f263c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719059ed31c0023d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.407+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c00a370 con 0x7f263c004d40 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.407+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c02f070 con 0x7f263c004d40 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.407+0000 7f2653e00640 1 -- 172.21.15.67:0/4172990542 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c02bb90 con 0x7f263c004d40 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.408+0000 7f265b000640 1 -- 172.21.15.67:0/4172990542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f263c004d40 msgr2=0x7f263c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.408+0000 7f265b000640 1 --2- 172.21.15.67:0/4172990542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f263c004d40 0x7f263c0051c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f264c00a720 tx=0x7f263c005830 comp rx=0 tx=0).stop 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.408+0000 7f265a600640 1 -- 172.21.15.67:0/4172990542 reap_dead start 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.408+0000 7f265b000640 1 -- 172.21.15.67:0/4172990542 shutdown_connections 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.408+0000 7f265b000640 1 -- 172.21.15.67:0/4172990542 >> 172.21.15.67:0/4172990542 conn(0x7f265408bfc0 msgr2=0x7f26540d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.408+0000 7f265b000640 1 -- 172.21.15.67:0/4172990542 shutdown_connections 2024-05-30T16:44:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.408+0000 7f265b000640 1 -- 172.21.15.67:0/4172990542 wait complete. 2024-05-30T16:44:25.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.410+0000 7f265b000640 1 Processor -- start 2024-05-30T16:44:25.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.410+0000 7f265b000640 1 -- start start 2024-05-30T16:44:25.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.412+0000 7f265b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26541593f0 0x7f26541f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:25.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.412+0000 7f265b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26541597d0 0x7f26540ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:25.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.412+0000 7f265b000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2654160c60 0x7f26541610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:25.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.412+0000 7f265b000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26540d0820 con 0x7f2654160c60 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.412+0000 7f265b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26540cfd20 con 0x7f26541597d0 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.412+0000 7f265b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26540d4fb0 con 0x7f26541593f0 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.412+0000 7f265a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2654160c60 0x7f26541610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.412+0000 7f265a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2654160c60 0x7f26541610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49230/0 (socket says 172.21.15.67:49230) 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.412+0000 7f265a600640 1 -- 172.21.15.67:0/1536522582 learned_addr learned my addr 172.21.15.67:0/1536522582 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f2659c00640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26541593f0 0x7f26541f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f2659200640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26541597d0 0x7f26540ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f265a600640 1 -- 172.21.15.67:0/1536522582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26541593f0 msgr2=0x7f26541f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f265a600640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26541593f0 0x7f26541f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f265a600640 1 -- 172.21.15.67:0/1536522582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26541597d0 msgr2=0x7f26540ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f265a600640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26541597d0 0x7f26540ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f265a600640 1 -- 172.21.15.67:0/1536522582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26540caea0 con 0x7f2654160c60 2024-05-30T16:44:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f2659200640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26541597d0 0x7f26540ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:25.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f265a600640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2654160c60 0x7f26541610c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f264c00a810 tx=0x7f264c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:25.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f2651600640 1 -- 172.21.15.67:0/1536522582 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c00a370 con 0x7f2654160c60 2024-05-30T16:44:25.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f2651600640 1 -- 172.21.15.67:0/1536522582 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c034470 con 0x7f2654160c60 2024-05-30T16:44:25.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.413+0000 7f2651600640 1 -- 172.21.15.67:0/1536522582 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0011c0 con 0x7f2654160c60 2024-05-30T16:44:25.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.414+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26540c9bc0 con 0x7f2654160c60 2024-05-30T16:44:25.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.414+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26540ca180 con 0x7f2654160c60 2024-05-30T16:44:25.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.415+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26540689e0 con 0x7f2654160c60 2024-05-30T16:44:25.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.416+0000 7f2651600640 1 -- 172.21.15.67:0/1536522582 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f264c034860 con 0x7f2654160c60 2024-05-30T16:44:25.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.417+0000 7f2651600640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263406c8f0 0x7f263406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:25.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.417+0000 7f2651600640 1 -- 172.21.15.67:0/1536522582 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f264c0ab9e0 con 0x7f2654160c60 2024-05-30T16:44:25.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.417+0000 7f2659c00640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263406c8f0 0x7f263406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:25.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.418+0000 7f2659c00640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263406c8f0 0x7f263406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f26480055b0 tx=0x7f2648009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:25.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.422+0000 7f2651600640 1 -- 172.21.15.67:0/1536522582 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f264c079e00 con 0x7f2654160c60 2024-05-30T16:44:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.631+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f26540cb0e0 con 0x7f2654160c60 2024-05-30T16:44:25.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.637+0000 7f2651600640 1 -- 172.21.15.67:0/1536522582 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f264c079700 con 0x7f2654160c60 2024-05-30T16:44:25.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.640+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263406c8f0 msgr2=0x7f263406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:25.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.640+0000 7f265b000640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263406c8f0 0x7f263406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f26480055b0 tx=0x7f2648009040 comp rx=0 tx=0).stop 2024-05-30T16:44:25.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.640+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2654160c60 msgr2=0x7f26541610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.640+0000 7f265b000640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2654160c60 0x7f26541610c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f264c00a810 tx=0x7f264c02d070 comp rx=0 tx=0).stop 2024-05-30T16:44:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.640+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 shutdown_connections 2024-05-30T16:44:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.640+0000 7f265b000640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263406c8f0 0x7f263406edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.640+0000 7f265b000640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2654160c60 0x7f26541610c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.640+0000 7f265b000640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26541597d0 0x7f26540ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.640+0000 7f265b000640 1 --2- 172.21.15.67:0/1536522582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26541593f0 0x7f26541f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.641+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 >> 172.21.15.67:0/1536522582 conn(0x7f265408bfc0 msgr2=0x7f265406e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.641+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 shutdown_connections 2024-05-30T16:44:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:25.641+0000 7f265b000640 1 -- 172.21.15.67:0/1536522582 wait complete. 2024-05-30T16:44:25.662 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T16:44:25.662 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:44:25.772 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:44:25.878 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T16:44:25.878 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:44:25.878 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:25.940 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:44:25.940 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:25.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:44:26.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.205+0000 7f33b1800640 1 Processor -- start 2024-05-30T16:44:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.206+0000 7f33b1800640 1 -- start start 2024-05-30T16:44:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.208+0000 7f33b1800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f33ac158b80 0x7f33ac06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.208+0000 7f33b1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac06e160 0x7f33ac071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.208+0000 7f33b1800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33ac06e6e0 con 0x7f33ac072100 2024-05-30T16:44:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.208+0000 7f33b1800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f33ac0b42d0 con 0x7f33ac158b80 2024-05-30T16:44:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.208+0000 7f33b1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac06e9c0 con 0x7f33ac06e160 2024-05-30T16:44:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.209+0000 7f33a3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac06e160 0x7f33ac071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.209+0000 7f33abe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f33ac158b80 0x7f33ac06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.209+0000 7f33a3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac06e160 0x7f33ac071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:57846/0 (socket says 172.21.15.67:57846) 2024-05-30T16:44:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.209+0000 7f33a3e00640 1 -- 172.21.15.67:0/2843323930 learned_addr learned my addr 172.21.15.67:0/2843323930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.210+0000 7f33a3e00640 1 -- 172.21.15.67:0/2843323930 >> v2:172.21.15.169:3300/0 conn(0x7f33ac158b80 msgr2=0x7f33ac06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.210+0000 7f33a3e00640 1 --2- 172.21.15.67:0/2843323930 >> v2:172.21.15.169:3300/0 conn(0x7f33ac158b80 0x7f33ac06dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.210+0000 7f33a3e00640 1 -- 172.21.15.67:0/2843323930 >> v1:172.21.15.169:6789/0 conn(0x7f33ac072100 legacy=0x7f33ac1621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.210+0000 7f33a3e00640 1 -- 172.21.15.67:0/2843323930 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac06eca0 con 0x7f33ac06e160 2024-05-30T16:44:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.211+0000 7f33a3e00640 1 --2- 172.21.15.67:0/2843323930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac06e160 0x7f33ac071bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3394009870 tx=0x7f339402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ef047d3263eb4a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:26.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.211+0000 7f33ab400640 1 -- 172.21.15.67:0/2843323930 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394037070 con 0x7f33ac06e160 2024-05-30T16:44:26.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.211+0000 7f33ab400640 1 -- 172.21.15.67:0/2843323930 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3394032440 con 0x7f33ac06e160 2024-05-30T16:44:26.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.212+0000 7f33ab400640 1 -- 172.21.15.67:0/2843323930 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394032810 con 0x7f33ac06e160 2024-05-30T16:44:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.212+0000 7f33b1800640 1 -- 172.21.15.67:0/2843323930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac06e160 msgr2=0x7f33ac071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.212+0000 7f33b1800640 1 --2- 172.21.15.67:0/2843323930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac06e160 0x7f33ac071bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3394009870 tx=0x7f339402a270 comp rx=0 tx=0).stop 2024-05-30T16:44:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.212+0000 7f33b1800640 1 -- 172.21.15.67:0/2843323930 shutdown_connections 2024-05-30T16:44:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.212+0000 7f33b1800640 1 --2- 172.21.15.67:0/2843323930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac06e160 0x7f33ac071bc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.212+0000 7f33b1800640 1 --2- 172.21.15.67:0/2843323930 >> v2:172.21.15.169:3300/0 conn(0x7f33ac158b80 0x7f33ac06dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.212+0000 7f33b1800640 1 -- 172.21.15.67:0/2843323930 >> 172.21.15.67:0/2843323930 conn(0x7f33ac0b1fc0 msgr2=0x7f33ac0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.212+0000 7f33b1800640 1 -- 172.21.15.67:0/2843323930 shutdown_connections 2024-05-30T16:44:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.212+0000 7f33b1800640 1 -- 172.21.15.67:0/2843323930 wait complete. 2024-05-30T16:44:26.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.214+0000 7f33b1800640 1 Processor -- start 2024-05-30T16:44:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.214+0000 7f33b1800640 1 -- start start 2024-05-30T16:44:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.215+0000 7f33b1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33ac06e160 0x7f33ac150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.215+0000 7f33b1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac072100 0x7f33ac157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.215+0000 7f33b1800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac158b80 0x7f33ac157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.215+0000 7f33b1800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac0b2d30 con 0x7f33ac06e160 2024-05-30T16:44:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.215+0000 7f33b1800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac06f260 con 0x7f33ac158b80 2024-05-30T16:44:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.215+0000 7f33b1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac06f780 con 0x7f33ac072100 2024-05-30T16:44:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.215+0000 7f33abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33ac06e160 0x7f33ac150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.215+0000 7f33abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33ac06e160 0x7f33ac150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49246/0 (socket says 172.21.15.67:49246) 2024-05-30T16:44:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.215+0000 7f33abe00640 1 -- 172.21.15.67:0/2025637254 learned_addr learned my addr 172.21.15.67:0/2025637254 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.216+0000 7f33a3e00640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac072100 0x7f33ac157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.216+0000 7f33abe00640 1 -- 172.21.15.67:0/2025637254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac072100 msgr2=0x7f33ac157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.216+0000 7f33b0e00640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac158b80 0x7f33ac157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.216+0000 7f33abe00640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac072100 0x7f33ac157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.216+0000 7f33abe00640 1 -- 172.21.15.67:0/2025637254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac158b80 msgr2=0x7f33ac157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.216+0000 7f33abe00640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac158b80 0x7f33ac157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.216+0000 7f33abe00640 1 -- 172.21.15.67:0/2025637254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac0ad480 con 0x7f33ac06e160 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.216+0000 7f33abe00640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33ac06e160 0x7f33ac150cb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f339c00e9b0 tx=0x7f339c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.217+0000 7f33a8c00640 1 -- 172.21.15.67:0/2025637254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c017070 con 0x7f33ac06e160 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.217+0000 7f33a8c00640 1 -- 172.21.15.67:0/2025637254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c00c9a0 con 0x7f33ac06e160 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.217+0000 7f33a8c00640 1 -- 172.21.15.67:0/2025637254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c00cd90 con 0x7f33ac06e160 2024-05-30T16:44:26.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.217+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac0ad760 con 0x7f33ac06e160 2024-05-30T16:44:26.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.217+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac0add20 con 0x7f33ac06e160 2024-05-30T16:44:26.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.218+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ac0689e0 con 0x7f33ac06e160 2024-05-30T16:44:26.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.219+0000 7f33a8c00640 1 -- 172.21.15.67:0/2025637254 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f339c01a070 con 0x7f33ac06e160 2024-05-30T16:44:26.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.220+0000 7f33a8c00640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338806c8f0 0x7f338806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.220+0000 7f33a8c00640 1 -- 172.21.15.67:0/2025637254 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f339c08bff0 con 0x7f33ac06e160 2024-05-30T16:44:26.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.220+0000 7f33a3e00640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338806c8f0 0x7f338806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.221+0000 7f33a3e00640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338806c8f0 0x7f338806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3394034000 tx=0x7f339402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:26.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.224+0000 7f33a8c00640 1 -- 172.21.15.67:0/2025637254 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f339c05a390 con 0x7f33ac06e160 2024-05-30T16:44:26.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.442+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f33ac072b40 con 0x7f33ac06e160 2024-05-30T16:44:26.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.443+0000 7f33a8c00640 1 -- 172.21.15.67:0/2025637254 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f339c059c90 con 0x7f33ac06e160 2024-05-30T16:44:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.445+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338806c8f0 msgr2=0x7f338806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.445+0000 7f33b1800640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338806c8f0 0x7f338806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3394034000 tx=0x7f339402e000 comp rx=0 tx=0).stop 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.445+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33ac06e160 msgr2=0x7f33ac150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.445+0000 7f33b1800640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33ac06e160 0x7f33ac150cb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f339c00e9b0 tx=0x7f339c00ee70 comp rx=0 tx=0).stop 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.446+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 shutdown_connections 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.446+0000 7f33b1800640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f338806c8f0 0x7f338806edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.446+0000 7f33b1800640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33ac158b80 0x7f33ac157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.446+0000 7f33b1800640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33ac072100 0x7f33ac157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.446+0000 7f33b1800640 1 --2- 172.21.15.67:0/2025637254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33ac06e160 0x7f33ac150cb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.446+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 >> 172.21.15.67:0/2025637254 conn(0x7f33ac0b1fc0 msgr2=0x7f33ac156df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.446+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 shutdown_connections 2024-05-30T16:44:26.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.446+0000 7f33b1800640 1 -- 172.21.15.67:0/2025637254 wait complete. 2024-05-30T16:44:26.466 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:44:26.466 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:26.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:44:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.736+0000 7f5b83200640 1 Processor -- start 2024-05-30T16:44:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.736+0000 7f5b83200640 1 -- start start 2024-05-30T16:44:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.738+0000 7f5b83200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b7c162940 0x7f5b7c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.738+0000 7f5b83200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7c06f590 con 0x7f5b7c0722d0 2024-05-30T16:44:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.738+0000 7f5b83200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7c06f870 con 0x7f5b7c162d20 2024-05-30T16:44:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.738+0000 7f5b83200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5b7c0ad3d0 con 0x7f5b7c162940 2024-05-30T16:44:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.738+0000 7f5b81e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b7c162940 0x7f5b7c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.738+0000 7f5b82800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5b7c0722d0 0x7f5b7c0adbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56760/0 (socket says 172.21.15.67:56760) 2024-05-30T16:44:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.738+0000 7f5b82800640 1 -- 172.21.15.67:0/2853507386 learned_addr learned my addr 172.21.15.67:0/2853507386 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.738+0000 7f5b81e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b7c162940 0x7f5b7c06eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49256/0 (socket says 172.21.15.67:49256) 2024-05-30T16:44:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.739+0000 7f5b81e00640 1 -- 172.21.15.67:0/2853507386 >> v1:172.21.15.169:6789/0 conn(0x7f5b7c162d20 legacy=0x7f5b7c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.739+0000 7f5b81e00640 1 -- 172.21.15.67:0/2853507386 >> v1:172.21.15.67:6789/0 conn(0x7f5b7c0722d0 legacy=0x7f5b7c0adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.739+0000 7f5b81e00640 1 -- 172.21.15.67:0/2853507386 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c06dde0 con 0x7f5b7c162940 2024-05-30T16:44:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.739+0000 7f5b80a00640 1 -- 172.21.15.67:0/2853507386 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3610638778 0 0) 0x7f5b7c06f590 con 0x7f5b7c0722d0 2024-05-30T16:44:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.740+0000 7f5b81e00640 1 --2- 172.21.15.67:0/2853507386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162940 0x7f5b7c06eb00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b680097a0 tx=0x7f5b6802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9126b58ec456459 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b80a00640 1 -- 172.21.15.67:0/2853507386 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68037070 con 0x7f5b7c162940 2024-05-30T16:44:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b80a00640 1 -- 172.21.15.67:0/2853507386 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6802adc0 con 0x7f5b7c162940 2024-05-30T16:44:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b80a00640 1 -- 172.21.15.67:0/2853507386 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68032660 con 0x7f5b7c162940 2024-05-30T16:44:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b83200640 1 -- 172.21.15.67:0/2853507386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162940 msgr2=0x7f5b7c06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b83200640 1 --2- 172.21.15.67:0/2853507386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162940 0x7f5b7c06eb00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b680097a0 tx=0x7f5b6802a0b0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b83200640 1 -- 172.21.15.67:0/2853507386 shutdown_connections 2024-05-30T16:44:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b83200640 1 --2- 172.21.15.67:0/2853507386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162940 0x7f5b7c06eb00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b83200640 1 -- 172.21.15.67:0/2853507386 >> 172.21.15.67:0/2853507386 conn(0x7f5b7c0b1fc0 msgr2=0x7f5b7c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:26.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b83200640 1 -- 172.21.15.67:0/2853507386 shutdown_connections 2024-05-30T16:44:26.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.741+0000 7f5b83200640 1 -- 172.21.15.67:0/2853507386 wait complete. 2024-05-30T16:44:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.743+0000 7f5b83200640 1 Processor -- start 2024-05-30T16:44:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.743+0000 7f5b83200640 1 -- start start 2024-05-30T16:44:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.744+0000 7f5b83200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7c0722d0 0x7f5b7c1f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.744+0000 7f5b83200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7c162940 0x7f5b7c09a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.744+0000 7f5b83200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162d20 0x7f5b7c09c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.744+0000 7f5b83200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c0ae2c0 con 0x7f5b7c162d20 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.744+0000 7f5b83200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c06f250 con 0x7f5b7c162940 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.744+0000 7f5b83200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7c0ae440 con 0x7f5b7c0722d0 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162d20 0x7f5b7c09c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162d20 0x7f5b7c09c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49260/0 (socket says 172.21.15.67:49260) 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b82800640 1 -- 172.21.15.67:0/2637934182 learned_addr learned my addr 172.21.15.67:0/2637934182 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b81400640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7c162940 0x7f5b7c09a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b81e00640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7c0722d0 0x7f5b7c1f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b82800640 1 -- 172.21.15.67:0/2637934182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7c0722d0 msgr2=0x7f5b7c1f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b82800640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7c0722d0 0x7f5b7c1f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b82800640 1 -- 172.21.15.67:0/2637934182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7c162940 msgr2=0x7f5b7c09a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b82800640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7c162940 0x7f5b7c09a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b82800640 1 -- 172.21.15.67:0/2637934182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c1f8100 con 0x7f5b7c162d20 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.745+0000 7f5b81e00640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7c0722d0 0x7f5b7c1f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.746+0000 7f5b82800640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162d20 0x7f5b7c09c2b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c800 tx=0x7f5b7800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.746+0000 7f5b76000640 1 -- 172.21.15.67:0/2637934182 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78018070 con 0x7f5b7c162d20 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.746+0000 7f5b76000640 1 -- 172.21.15.67:0/2637934182 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78004d80 con 0x7f5b7c162d20 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.746+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c09ae70 con 0x7f5b7c162d20 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.746+0000 7f5b76000640 1 -- 172.21.15.67:0/2637934182 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800a880 con 0x7f5b7c162d20 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.746+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c09b430 con 0x7f5b7c162d20 2024-05-30T16:44:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.747+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7c0689e0 con 0x7f5b7c162d20 2024-05-30T16:44:26.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.749+0000 7f5b76000640 1 -- 172.21.15.67:0/2637934182 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b7801b070 con 0x7f5b7c162d20 2024-05-30T16:44:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.750+0000 7f5b76000640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5c06c8f0 0x7f5b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.750+0000 7f5b76000640 1 -- 172.21.15.67:0/2637934182 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5b7808b0b0 con 0x7f5b7c162d20 2024-05-30T16:44:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.750+0000 7f5b81e00640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5c06c8f0 0x7f5b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:26.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.751+0000 7f5b81e00640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5c06c8f0 0x7f5b5c06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b68034000 tx=0x7f5b6802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:26.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.755+0000 7f5b76000640 1 -- 172.21.15.67:0/2637934182 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b7c1f8100 con 0x7f5b7c162d20 2024-05-30T16:44:26.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.972+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b7c165960 con 0x7f5b7c162d20 2024-05-30T16:44:26.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.973+0000 7f5b76000640 1 -- 172.21.15.67:0/2637934182 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f5b78059590 con 0x7f5b7c162d20 2024-05-30T16:44:26.974 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:44:26.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.975+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5c06c8f0 msgr2=0x7f5b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.975+0000 7f5b83200640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5c06c8f0 0x7f5b5c06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b68034000 tx=0x7f5b6802e000 comp rx=0 tx=0).stop 2024-05-30T16:44:26.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.975+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162d20 msgr2=0x7f5b7c09c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:26.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.975+0000 7f5b83200640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162d20 0x7f5b7c09c2b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c800 tx=0x7f5b7800ccc0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.976+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 shutdown_connections 2024-05-30T16:44:26.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.976+0000 7f5b83200640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5c06c8f0 0x7f5b5c06edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.976+0000 7f5b83200640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7c162d20 0x7f5b7c09c2b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.976+0000 7f5b83200640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7c162940 0x7f5b7c09a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.976+0000 7f5b83200640 1 --2- 172.21.15.67:0/2637934182 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7c0722d0 0x7f5b7c1f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:26.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.976+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 >> 172.21.15.67:0/2637934182 conn(0x7f5b7c0b1fc0 msgr2=0x7f5b7c0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:26.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.976+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 shutdown_connections 2024-05-30T16:44:26.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:26.976+0000 7f5b83200640 1 -- 172.21.15.67:0/2637934182 wait complete. 2024-05-30T16:44:26.997 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T16:44:26.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:26.997 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:27.084 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:44:27.084 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:44:27.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:27.084 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-30T16:44:27.164 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:44:27.164 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:27.221 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:44:27.222 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:27.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:44:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.499+0000 7f80ea800640 1 Processor -- start 2024-05-30T16:44:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.499+0000 7f80ea800640 1 -- start start 2024-05-30T16:44:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.500+0000 7f80ea800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80e406f860 0x7f80e40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.500+0000 7f80ea800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e406ed10 con 0x7f80e40b3690 2024-05-30T16:44:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.500+0000 7f80ea800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e415cb00 con 0x7f80e40d0bb0 2024-05-30T16:44:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.500+0000 7f80ea800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f80e40b3cc0 con 0x7f80e406f860 2024-05-30T16:44:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.501+0000 7f80e3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80e406f860 0x7f80e40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.501+0000 7f80e2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f80e40b3690 0x7f80e40d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56764/0 (socket says 172.21.15.67:56764) 2024-05-30T16:44:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.501+0000 7f80e2a00640 1 -- 172.21.15.67:0/1394256137 learned_addr learned my addr 172.21.15.67:0/1394256137 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.501+0000 7f80e3400640 1 -- 172.21.15.67:0/1394256137 >> v1:172.21.15.169:6789/0 conn(0x7f80e40d0bb0 legacy=0x7f80e40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.501+0000 7f80e3400640 1 -- 172.21.15.67:0/1394256137 >> v1:172.21.15.67:6789/0 conn(0x7f80e40b3690 legacy=0x7f80e40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.501+0000 7f80e3400640 1 -- 172.21.15.67:0/1394256137 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e40b3fa0 con 0x7f80e406f860 2024-05-30T16:44:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.501+0000 7f80e2000640 1 -- 172.21.15.67:0/1394256137 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1017956497 0 0) 0x7f80cc001490 con 0x7f80e40b3690 2024-05-30T16:44:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.503+0000 7f80e3400640 1 --2- 172.21.15.67:0/1394256137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e406f860 0x7f80e40b3150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80d40097a0 tx=0x7f80d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ed82faacb1655f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.503+0000 7f80e2000640 1 -- 172.21.15.67:0/1394256137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4037070 con 0x7f80e406f860 2024-05-30T16:44:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.503+0000 7f80e2000640 1 -- 172.21.15.67:0/1394256137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d4032440 con 0x7f80e406f860 2024-05-30T16:44:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.503+0000 7f80e2000640 1 -- 172.21.15.67:0/1394256137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4032810 con 0x7f80e406f860 2024-05-30T16:44:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.504+0000 7f80ea800640 1 -- 172.21.15.67:0/1394256137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e406f860 msgr2=0x7f80e40b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.504+0000 7f80ea800640 1 --2- 172.21.15.67:0/1394256137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e406f860 0x7f80e40b3150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80d40097a0 tx=0x7f80d402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:44:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.504+0000 7f80ea800640 1 -- 172.21.15.67:0/1394256137 shutdown_connections 2024-05-30T16:44:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.504+0000 7f80ea800640 1 --2- 172.21.15.67:0/1394256137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e406f860 0x7f80e40b3150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.504+0000 7f80ea800640 1 -- 172.21.15.67:0/1394256137 >> 172.21.15.67:0/1394256137 conn(0x7f80e406bae0 msgr2=0x7f80e406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.504+0000 7f80ea800640 1 -- 172.21.15.67:0/1394256137 shutdown_connections 2024-05-30T16:44:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.504+0000 7f80ea800640 1 -- 172.21.15.67:0/1394256137 wait complete. 2024-05-30T16:44:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.506+0000 7f80ea800640 1 Processor -- start 2024-05-30T16:44:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.506+0000 7f80ea800640 1 -- start start 2024-05-30T16:44:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.507+0000 7f80ea800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80e406f860 0x7f80e4099530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.507+0000 7f80ea800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e40b3690 0x7f80e40cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80ea800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e40d0bb0 0x7f80e40ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80ea800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f80e40b4840 con 0x7f80e40d0bb0 2024-05-30T16:44:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80ea800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f80e40b4560 con 0x7f80e40b3690 2024-05-30T16:44:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80ea800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f80e40ad840 con 0x7f80e406f860 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e40d0bb0 0x7f80e40ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e40d0bb0 0x7f80e40ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49278/0 (socket says 172.21.15.67:49278) 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e3e00640 1 -- 172.21.15.67:0/3630571822 learned_addr learned my addr 172.21.15.67:0/3630571822 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e3400640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80e406f860 0x7f80e4099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e2a00640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e40b3690 0x7f80e40cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e3e00640 1 -- 172.21.15.67:0/3630571822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80e406f860 msgr2=0x7f80e4099530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e3e00640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80e406f860 0x7f80e4099530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e3e00640 1 -- 172.21.15.67:0/3630571822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e40b3690 msgr2=0x7f80e40cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e3e00640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e40b3690 0x7f80e40cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.508+0000 7f80e3e00640 1 -- 172.21.15.67:0/3630571822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e40c9950 con 0x7f80e40d0bb0 2024-05-30T16:44:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.509+0000 7f80e3e00640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e40d0bb0 0x7f80e40ccc50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00b440 tx=0x7f80dc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.509+0000 7f80db400640 1 -- 172.21.15.67:0/3630571822 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc015020 con 0x7f80e40d0bb0 2024-05-30T16:44:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.509+0000 7f80db400640 1 -- 172.21.15.67:0/3630571822 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80dc011830 con 0x7f80e40d0bb0 2024-05-30T16:44:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.509+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e40c9c30 con 0x7f80e40d0bb0 2024-05-30T16:44:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.509+0000 7f80db400640 1 -- 172.21.15.67:0/3630571822 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc011c20 con 0x7f80e40d0bb0 2024-05-30T16:44:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.509+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e40c9580 con 0x7f80e40d0bb0 2024-05-30T16:44:27.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.511+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e40689e0 con 0x7f80e40d0bb0 2024-05-30T16:44:27.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.512+0000 7f80db400640 1 -- 172.21.15.67:0/3630571822 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80dc018020 con 0x7f80e40d0bb0 2024-05-30T16:44:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.513+0000 7f80db400640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f80c006c8f0 0x7f80c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.513+0000 7f80db400640 1 -- 172.21.15.67:0/3630571822 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f80dc08b6f0 con 0x7f80e40d0bb0 2024-05-30T16:44:27.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.513+0000 7f80e3400640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f80c006c8f0 0x7f80c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:27.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.514+0000 7f80e3400640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f80c006c8f0 0x7f80c006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f80d4034000 tx=0x7f80d402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.518+0000 7f80db400640 1 -- 172.21.15.67:0/3630571822 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80dc01f030 con 0x7f80e40d0bb0 2024-05-30T16:44:27.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.736+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f80e40ca480 con 0x7f80e40d0bb0 2024-05-30T16:44:27.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.736+0000 7f80db400640 1 -- 172.21.15.67:0/3630571822 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f80dc0172c0 con 0x7f80e40d0bb0 2024-05-30T16:44:27.737 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.739+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f80c006c8f0 msgr2=0x7f80c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.739+0000 7f80ea800640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f80c006c8f0 0x7f80c006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f80d4034000 tx=0x7f80d402e000 comp rx=0 tx=0).stop 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.739+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e40d0bb0 msgr2=0x7f80e40ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.739+0000 7f80ea800640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e40d0bb0 0x7f80e40ccc50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00b440 tx=0x7f80dc00b900 comp rx=0 tx=0).stop 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.740+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 shutdown_connections 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.740+0000 7f80ea800640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f80c006c8f0 0x7f80c006edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.740+0000 7f80ea800640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80e40d0bb0 0x7f80e40ccc50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.740+0000 7f80ea800640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e40b3690 0x7f80e40cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.740+0000 7f80ea800640 1 --2- 172.21.15.67:0/3630571822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80e406f860 0x7f80e4099530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.740+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 >> 172.21.15.67:0/3630571822 conn(0x7f80e406bae0 msgr2=0x7f80e40afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.740+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 shutdown_connections 2024-05-30T16:44:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:27.740+0000 7f80ea800640 1 -- 172.21.15.67:0/3630571822 wait complete. 2024-05-30T16:44:27.759 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:44:27.759 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:27.816 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:44:28.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.014+0000 7f7b78000640 1 Processor -- start 2024-05-30T16:44:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.015+0000 7f7b78000640 1 -- start start 2024-05-30T16:44:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.016+0000 7f7b78000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7b700d2db0 0x7f7b700b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.016+0000 7f7b78000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 0x7f7b700d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.016+0000 7f7b78000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b700d36d0 con 0x7f7b700cf490 2024-05-30T16:44:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.016+0000 7f7b78000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7b700d5540 con 0x7f7b700d2db0 2024-05-30T16:44:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.016+0000 7f7b78000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7b70163500 con 0x7f7b700d4d60 2024-05-30T16:44:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.016+0000 7f7b75600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7b700d2db0 0x7f7b700b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.016+0000 7f7b76000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7b700cf490 0x7f7b700cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56780/0 (socket says 172.21.15.67:56780) 2024-05-30T16:44:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.016+0000 7f7b76000640 1 -- 172.21.15.67:0/3764657016 learned_addr learned my addr 172.21.15.67:0/3764657016 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:28.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.017+0000 7f7b74c00640 1 --2- 172.21.15.67:0/3764657016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 0x7f7b700d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.017+0000 7f7b75600640 1 -- 172.21.15.67:0/3764657016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 msgr2=0x7f7b700d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.017+0000 7f7b75600640 1 --2- 172.21.15.67:0/3764657016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 0x7f7b700d3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.017+0000 7f7b75600640 1 -- 172.21.15.67:0/3764657016 >> v1:172.21.15.67:6789/0 conn(0x7f7b700cf490 legacy=0x7f7b700cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.017+0000 7f7b75600640 1 -- 172.21.15.67:0/3764657016 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b70162740 con 0x7f7b700d2db0 2024-05-30T16:44:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.018+0000 7f7b75600640 1 --2- 172.21.15.67:0/3764657016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700d2db0 0x7f7b700b3b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b640097a0 tx=0x7f7b6402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5705836efa4035 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.018+0000 7f7b6fe00640 1 -- 172.21.15.67:0/3764657016 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64037070 con 0x7f7b700d2db0 2024-05-30T16:44:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.018+0000 7f7b6fe00640 1 -- 172.21.15.67:0/3764657016 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6402ae50 con 0x7f7b700d2db0 2024-05-30T16:44:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.019+0000 7f7b6fe00640 1 -- 172.21.15.67:0/3764657016 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64032660 con 0x7f7b700d2db0 2024-05-30T16:44:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.019+0000 7f7b78000640 1 -- 172.21.15.67:0/3764657016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700d2db0 msgr2=0x7f7b700b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:28.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.019+0000 7f7b78000640 1 --2- 172.21.15.67:0/3764657016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700d2db0 0x7f7b700b3b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b640097a0 tx=0x7f7b6402a100 comp rx=0 tx=0).stop 2024-05-30T16:44:28.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.019+0000 7f7b78000640 1 -- 172.21.15.67:0/3764657016 shutdown_connections 2024-05-30T16:44:28.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.019+0000 7f7b78000640 1 --2- 172.21.15.67:0/3764657016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 0x7f7b700d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.019+0000 7f7b78000640 1 --2- 172.21.15.67:0/3764657016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700d2db0 0x7f7b700b3b50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.019+0000 7f7b78000640 1 -- 172.21.15.67:0/3764657016 >> 172.21.15.67:0/3764657016 conn(0x7f7b7008bfc0 msgr2=0x7f7b700d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:28.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.020+0000 7f7b78000640 1 -- 172.21.15.67:0/3764657016 shutdown_connections 2024-05-30T16:44:28.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.020+0000 7f7b78000640 1 -- 172.21.15.67:0/3764657016 wait complete. 2024-05-30T16:44:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.021+0000 7f7b78000640 1 Processor -- start 2024-05-30T16:44:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.021+0000 7f7b78000640 1 -- start start 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.022+0000 7f7b78000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700cf490 0x7f7b7014ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.022+0000 7f7b78000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b700d2db0 0x7f7b70153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.022+0000 7f7b78000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 0x7f7b7006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.022+0000 7f7b78000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70162fe0 con 0x7f7b700cf490 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.022+0000 7f7b78000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70162d00 con 0x7f7b700d2db0 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.022+0000 7f7b78000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7b700d51a0 con 0x7f7b700d4d60 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b75600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700cf490 0x7f7b7014ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b75600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700cf490 0x7f7b7014ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49290/0 (socket says 172.21.15.67:49290) 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b75600640 1 -- 172.21.15.67:0/3089772002 learned_addr learned my addr 172.21.15.67:0/3089772002 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:28.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b74c00640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b700d2db0 0x7f7b70153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b76000640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 0x7f7b7006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b75600640 1 -- 172.21.15.67:0/3089772002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 msgr2=0x7f7b7006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b75600640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 0x7f7b7006dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b75600640 1 -- 172.21.15.67:0/3089772002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b700d2db0 msgr2=0x7f7b70153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b75600640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b700d2db0 0x7f7b70153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.023+0000 7f7b75600640 1 -- 172.21.15.67:0/3089772002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b70158c90 con 0x7f7b700cf490 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.024+0000 7f7b75600640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700cf490 0x7f7b7014ba60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b640098c0 tx=0x7f7b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.024+0000 7f7b6d600640 1 -- 172.21.15.67:0/3089772002 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64041030 con 0x7f7b700cf490 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.024+0000 7f7b6d600640 1 -- 172.21.15.67:0/3089772002 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6402ac50 con 0x7f7b700cf490 2024-05-30T16:44:28.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.024+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b70158f70 con 0x7f7b700cf490 2024-05-30T16:44:28.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.024+0000 7f7b6d600640 1 -- 172.21.15.67:0/3089772002 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64037040 con 0x7f7b700cf490 2024-05-30T16:44:28.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.024+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b70159810 con 0x7f7b700cf490 2024-05-30T16:44:28.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.025+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b700689e0 con 0x7f7b700cf490 2024-05-30T16:44:28.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.026+0000 7f7b6d600640 1 -- 172.21.15.67:0/3089772002 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b64044070 con 0x7f7b700cf490 2024-05-30T16:44:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.027+0000 7f7b6d600640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7b5006c8f0 0x7f7b5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.027+0000 7f7b6d600640 1 -- 172.21.15.67:0/3089772002 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7b6402ae00 con 0x7f7b700cf490 2024-05-30T16:44:28.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.028+0000 7f7b74c00640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7b5006c8f0 0x7f7b5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:28.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.028+0000 7f7b74c00640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7b5006c8f0 0x7f7b5006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b700d2920 tx=0x7f7b5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:28.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.032+0000 7f7b6d600640 1 -- 172.21.15.67:0/3089772002 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b6404b070 con 0x7f7b700cf490 2024-05-30T16:44:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.249+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7b700c7ea0 con 0x7f7b700cf490 2024-05-30T16:44:28.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.250+0000 7f7b6d600640 1 -- 172.21.15.67:0/3089772002 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f7b64044e40 con 0x7f7b700cf490 2024-05-30T16:44:28.251 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:44:28.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7b5006c8f0 msgr2=0x7f7b5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:28.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7b5006c8f0 0x7f7b5006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b700d2920 tx=0x7f7b5c008040 comp rx=0 tx=0).stop 2024-05-30T16:44:28.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700cf490 msgr2=0x7f7b7014ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:28.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700cf490 0x7f7b7014ba60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b640098c0 tx=0x7f7b640046c0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 shutdown_connections 2024-05-30T16:44:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7b5006c8f0 0x7f7b5006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b700d4d60 0x7f7b7006dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b700d2db0 0x7f7b70153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 --2- 172.21.15.67:0/3089772002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b700cf490 0x7f7b7014ba60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 >> 172.21.15.67:0/3089772002 conn(0x7f7b7008bfc0 msgr2=0x7f7b700d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 shutdown_connections 2024-05-30T16:44:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:28.253+0000 7f7b78000640 1 -- 172.21.15.67:0/3089772002 wait complete. 2024-05-30T16:44:28.273 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi169.front.sepia.ceph.com... 2024-05-30T16:44:28.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:28.273 DEBUG:teuthology.orchestra.run.smithi169:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-30T16:44:28.364 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:44:28.364 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T16:44:28.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:28.364 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-30T16:44:28.676 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:44:28.676 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:28.730 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:44:28.731 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:28.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:44:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.007+0000 7fd8df600640 1 Processor -- start 2024-05-30T16:44:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.007+0000 7fd8df600640 1 -- start start 2024-05-30T16:44:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.008+0000 7fd8df600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d806f860 0x7fd8d80b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.008+0000 7fd8df600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd8d80d0bb0 0x7fd8d80ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.008+0000 7fd8df600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8d806ed10 con 0x7fd8d80b3690 2024-05-30T16:44:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.008+0000 7fd8df600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd8d815cb00 con 0x7fd8d80d0bb0 2024-05-30T16:44:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.008+0000 7fd8df600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d80b3cc0 con 0x7fd8d806f860 2024-05-30T16:44:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.009+0000 7fd8d7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd8d80b3690 0x7fd8d80d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56786/0 (socket says 172.21.15.67:56786) 2024-05-30T16:44:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.009+0000 7fd8d7e00640 1 -- 172.21.15.67:0/16406286 learned_addr learned my addr 172.21.15.67:0/16406286 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.009+0000 7fd8dd600640 1 --2- 172.21.15.67:0/16406286 >> v2:172.21.15.169:3300/0 conn(0x7fd8d80d0bb0 0x7fd8d80ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.009+0000 7fd8dcc00640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d806f860 0x7fd8d80b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.009+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 918862673 0 0) 0x7fd8c4001110 con 0x7fd8d80b3690 2024-05-30T16:44:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.009+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8c0003520 con 0x7fd8d80b3690 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2263525637 0 0) 0x7fd8c0003520 con 0x7fd8d80b3690 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d806f860 msgr2=0x7fd8d80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d806f860 0x7fd8d80b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 >> v2:172.21.15.169:3300/0 conn(0x7fd8d80d0bb0 msgr2=0x7fd8d80ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 --2- 172.21.15.67:0/16406286 >> v2:172.21.15.169:3300/0 conn(0x7fd8d80d0bb0 0x7fd8d80ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d80b3fa0 con 0x7fd8d80b3690 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fd8c4001840 con 0x7fd8d80b3690 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 >> v1:172.21.15.67:6789/0 conn(0x7fd8d80b3690 legacy=0x7fd8d80d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8c00039b0 0x7fd8c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d806f860 0x7fd8c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c0004d40 0x7fd8c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d815cb00 con 0x7fd8c00039b0 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d80b3cc0 con 0x7fd8c0004d40 2024-05-30T16:44:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c000b9c0 con 0x7fd8d806f860 2024-05-30T16:44:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8dcc00640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8c00039b0 0x7fd8c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8d7e00640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d806f860 0x7fd8c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.010+0000 7fd8dd600640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c0004d40 0x7fd8c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.011+0000 7fd8dcc00640 1 -- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d806f860 msgr2=0x7fd8c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.011+0000 7fd8dcc00640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d806f860 0x7fd8c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.011+0000 7fd8dcc00640 1 -- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c0004d40 msgr2=0x7fd8c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.011+0000 7fd8dcc00640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c0004d40 0x7fd8c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.011+0000 7fd8dcc00640 1 -- 172.21.15.67:0/16406286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c0003520 con 0x7fd8c00039b0 2024-05-30T16:44:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.011+0000 7fd8dd600640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c0004d40 0x7fd8c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.012+0000 7fd8dcc00640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8c00039b0 0x7fd8c0003dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009ac0 tx=0x7fd8c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93cddd0711d3f17 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.012+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc036070 con 0x7fd8c00039b0 2024-05-30T16:44:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.012+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8cc031400 con 0x7fd8c00039b0 2024-05-30T16:44:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.012+0000 7fd8d7400640 1 -- 172.21.15.67:0/16406286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc009690 con 0x7fd8c00039b0 2024-05-30T16:44:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.013+0000 7fd8df600640 1 -- 172.21.15.67:0/16406286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8c00039b0 msgr2=0x7fd8c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.013+0000 7fd8df600640 1 --2- 172.21.15.67:0/16406286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8c00039b0 0x7fd8c0003dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009ac0 tx=0x7fd8c0005830 comp rx=0 tx=0).stop 2024-05-30T16:44:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.013+0000 7fd8dd600640 1 -- 172.21.15.67:0/16406286 reap_dead start 2024-05-30T16:44:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.013+0000 7fd8df600640 1 -- 172.21.15.67:0/16406286 shutdown_connections 2024-05-30T16:44:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.013+0000 7fd8df600640 1 -- 172.21.15.67:0/16406286 >> 172.21.15.67:0/16406286 conn(0x7fd8d806bae0 msgr2=0x7fd8d806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.013+0000 7fd8df600640 1 -- 172.21.15.67:0/16406286 shutdown_connections 2024-05-30T16:44:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.013+0000 7fd8df600640 1 -- 172.21.15.67:0/16406286 wait complete. 2024-05-30T16:44:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.015+0000 7fd8df600640 1 Processor -- start 2024-05-30T16:44:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.015+0000 7fd8df600640 1 -- start start 2024-05-30T16:44:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.016+0000 7fd8df600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c00039b0 0x7fd8d80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.016+0000 7fd8df600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8d80c60e0 0x7fd8d815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.016+0000 7fd8df600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d815d250 0x7fd8d80af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.016+0000 7fd8df600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d80b4840 con 0x7fd8d80c60e0 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.016+0000 7fd8df600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d80b4560 con 0x7fd8c00039b0 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.016+0000 7fd8df600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d80b3150 con 0x7fd8d815d250 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.016+0000 7fd8d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8d80c60e0 0x7fd8d815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.016+0000 7fd8d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8d80c60e0 0x7fd8d815cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49314/0 (socket says 172.21.15.67:49314) 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.016+0000 7fd8d7e00640 1 -- 172.21.15.67:0/1090585009 learned_addr learned my addr 172.21.15.67:0/1090585009 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.017+0000 7fd8dd600640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d815d250 0x7fd8d80af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.017+0000 7fd8dcc00640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c00039b0 0x7fd8d80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.017+0000 7fd8d7e00640 1 -- 172.21.15.67:0/1090585009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d815d250 msgr2=0x7fd8d80af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.017+0000 7fd8d7e00640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d815d250 0x7fd8d80af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.017+0000 7fd8d7e00640 1 -- 172.21.15.67:0/1090585009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c00039b0 msgr2=0x7fd8d80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.017+0000 7fd8d7e00640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c00039b0 0x7fd8d80b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.017+0000 7fd8d7e00640 1 -- 172.21.15.67:0/1090585009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d80b1af0 con 0x7fd8d80c60e0 2024-05-30T16:44:29.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.017+0000 7fd8d7e00640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8d80c60e0 0x7fd8d815cd10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400e780 tx=0x7fd8c400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.018+0000 7fd8d4c00640 1 -- 172.21.15.67:0/1090585009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400ee30 con 0x7fd8d80c60e0 2024-05-30T16:44:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.018+0000 7fd8d4c00640 1 -- 172.21.15.67:0/1090585009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c4018070 con 0x7fd8d80c60e0 2024-05-30T16:44:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.018+0000 7fd8d4c00640 1 -- 172.21.15.67:0/1090585009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400cb10 con 0x7fd8d80c60e0 2024-05-30T16:44:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.018+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d815e580 con 0x7fd8d80c60e0 2024-05-30T16:44:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.018+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d815eb40 con 0x7fd8d80c60e0 2024-05-30T16:44:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.019+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d80689e0 con 0x7fd8d80c60e0 2024-05-30T16:44:29.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.020+0000 7fd8d4c00640 1 -- 172.21.15.67:0/1090585009 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8c400ccb0 con 0x7fd8d80c60e0 2024-05-30T16:44:29.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.021+0000 7fd8d4c00640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd8b806c940 0x7fd8b806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.021+0000 7fd8d4c00640 1 -- 172.21.15.67:0/1090585009 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd8c408b990 con 0x7fd8d80c60e0 2024-05-30T16:44:29.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.021+0000 7fd8dcc00640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd8b806c940 0x7fd8b806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.022+0000 7fd8dcc00640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd8b806c940 0x7fd8b806ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc033000 tx=0x7fd8cc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:29.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.026+0000 7fd8d4c00640 1 -- 172.21.15.67:0/1090585009 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8d806ed10 con 0x7fd8d80c60e0 2024-05-30T16:44:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.241+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd8d806e2f0 con 0x7fd8d80c60e0 2024-05-30T16:44:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.242+0000 7fd8d4c00640 1 -- 172.21.15.67:0/1090585009 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fd8c4059ea0 con 0x7fd8d80c60e0 2024-05-30T16:44:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:44:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd8b806c940 msgr2=0x7fd8b806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd8b806c940 0x7fd8b806ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc033000 tx=0x7fd8cc008030 comp rx=0 tx=0).stop 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8d80c60e0 msgr2=0x7fd8d815cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8d80c60e0 0x7fd8d815cd10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400e780 tx=0x7fd8c400ec40 comp rx=0 tx=0).stop 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 shutdown_connections 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd8b806c940 0x7fd8b806ee00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8d815d250 0x7fd8d80af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8d80c60e0 0x7fd8d815cd10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 --2- 172.21.15.67:0/1090585009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8c00039b0 0x7fd8d80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 >> 172.21.15.67:0/1090585009 conn(0x7fd8d806bae0 msgr2=0x7fd8d80cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 shutdown_connections 2024-05-30T16:44:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.245+0000 7fd8df600640 1 -- 172.21.15.67:0/1090585009 wait complete. 2024-05-30T16:44:29.265 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:44:29.265 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:29.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:44:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.534+0000 7fdad8a00640 1 Processor -- start 2024-05-30T16:44:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.534+0000 7fdad8a00640 1 -- start start 2024-05-30T16:44:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.536+0000 7fdad8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdad4162d20 0x7fdad4071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.536+0000 7fdad8a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad406f590 con 0x7fdad4072100 2024-05-30T16:44:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.536+0000 7fdad8a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad406f870 con 0x7fdad4162940 2024-05-30T16:44:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.536+0000 7fdad8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdad4072530 con 0x7fdad4162d20 2024-05-30T16:44:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.536+0000 7fdad3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdad4072100 0x7fdad40adbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56802/0 (socket says 172.21.15.67:56802) 2024-05-30T16:44:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.536+0000 7fdad3e00640 1 -- 172.21.15.67:0/3164070951 learned_addr learned my addr 172.21.15.67:0/3164070951 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2a00640 1 --2- 172.21.15.67:0/3164070951 >> v2:172.21.15.169:3300/0 conn(0x7fdad4162d20 0x7fdad4071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 496875873 0 0) 0x7fdad406f590 con 0x7fdad4072100 2024-05-30T16:44:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdab8003520 con 0x7fdad4072100 2024-05-30T16:44:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2987676227 0 0) 0x7fdab8003520 con 0x7fdad4072100 2024-05-30T16:44:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 >> v2:172.21.15.169:3300/0 conn(0x7fdad4162d20 msgr2=0x7fdad4071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 --2- 172.21.15.67:0/3164070951 >> v2:172.21.15.169:3300/0 conn(0x7fdad4162d20 0x7fdad4071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 >> v1:172.21.15.169:6789/0 conn(0x7fdad4162940 legacy=0x7fdad406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad40ad3d0 con 0x7fdad4072100 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fdacc001670 con 0x7fdad4072100 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 >> v1:172.21.15.67:6789/0 conn(0x7fdad4072100 legacy=0x7fdad40adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab80039b0 0x7fdab8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab8004800 0x7fdab8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdab800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4072530 con 0x7fdab80052e0 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdab800c1b0 con 0x7fdab8004800 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.537+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdab800c3b0 con 0x7fdab80039b0 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.538+0000 7fdad3e00640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdab800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.538+0000 7fdad3400640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab80039b0 0x7fdab8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.538+0000 7fdad2a00640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab8004800 0x7fdab8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.538+0000 7fdad3e00640 1 -- 172.21.15.67:0/3164070951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab80039b0 msgr2=0x7fdab8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:29.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.538+0000 7fdad3e00640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdab80039b0 0x7fdab8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.538+0000 7fdad3e00640 1 -- 172.21.15.67:0/3164070951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab8004800 msgr2=0x7fdab8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:29.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.538+0000 7fdad3e00640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab8004800 0x7fdab8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.538+0000 7fdad3e00640 1 -- 172.21.15.67:0/3164070951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab8003520 con 0x7fdab80052e0 2024-05-30T16:44:29.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.538+0000 7fdad2a00640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab8004800 0x7fdab8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:29.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.539+0000 7fdad3e00640 1 -- 172.21.15.67:0/3164070951 reap_dead start 2024-05-30T16:44:29.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.539+0000 7fdad3e00640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdab800b9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c210 tx=0x7fdab8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfd8768dd94a136 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:29.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.539+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc007cf0 con 0x7fdab80052e0 2024-05-30T16:44:29.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.540+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc005af0 con 0x7fdab80052e0 2024-05-30T16:44:29.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.540+0000 7fdad2000640 1 -- 172.21.15.67:0/3164070951 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc007070 con 0x7fdab80052e0 2024-05-30T16:44:29.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.540+0000 7fdad8a00640 1 -- 172.21.15.67:0/3164070951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 msgr2=0x7fdab800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.540+0000 7fdad8a00640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdab800b9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c210 tx=0x7fdab8005830 comp rx=0 tx=0).stop 2024-05-30T16:44:29.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.540+0000 7fdad8a00640 1 -- 172.21.15.67:0/3164070951 shutdown_connections 2024-05-30T16:44:29.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.540+0000 7fdad8a00640 1 --2- 172.21.15.67:0/3164070951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdab800b9c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.540+0000 7fdad8a00640 1 -- 172.21.15.67:0/3164070951 >> 172.21.15.67:0/3164070951 conn(0x7fdad40b1fc0 msgr2=0x7fdad40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:29.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.540+0000 7fdad8a00640 1 -- 172.21.15.67:0/3164070951 shutdown_connections 2024-05-30T16:44:29.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.540+0000 7fdad8a00640 1 -- 172.21.15.67:0/3164070951 wait complete. 2024-05-30T16:44:29.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.542+0000 7fdad8a00640 1 Processor -- start 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.542+0000 7fdad8a00640 1 -- start start 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.543+0000 7fdad8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdad41536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.543+0000 7fdad8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdad40ccca0 0x7fdad41f8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.543+0000 7fdad8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdad41f8b90 0x7fdad409ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.543+0000 7fdad8a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4071d40 con 0x7fdab80052e0 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.543+0000 7fdad8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4071bc0 con 0x7fdad41f8b90 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.543+0000 7fdad8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdad4071ec0 con 0x7fdad40ccca0 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.543+0000 7fdad3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdad41536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.543+0000 7fdad3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdad41536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49330/0 (socket says 172.21.15.67:49330) 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.543+0000 7fdad3400640 1 -- 172.21.15.67:0/2338744847 learned_addr learned my addr 172.21.15.67:0/2338744847 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.544+0000 7fdad2a00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdad40ccca0 0x7fdad41f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.544+0000 7fdad3e00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdad41f8b90 0x7fdad409ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.544+0000 7fdad3400640 1 -- 172.21.15.67:0/2338744847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdad40ccca0 msgr2=0x7fdad41f8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.544+0000 7fdad3400640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdad40ccca0 0x7fdad41f8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.544+0000 7fdad3400640 1 -- 172.21.15.67:0/2338744847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdad41f8b90 msgr2=0x7fdad409ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.544+0000 7fdad3400640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdad41f8b90 0x7fdad409ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.544+0000 7fdad3400640 1 -- 172.21.15.67:0/2338744847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad4153c20 con 0x7fdab80052e0 2024-05-30T16:44:29.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.544+0000 7fdad3400640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdad41536e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00e4d0 tx=0x7fdabc00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:29.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.544+0000 7fdacb400640 1 -- 172.21.15.67:0/2338744847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0052e0 con 0x7fdab80052e0 2024-05-30T16:44:29.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.545+0000 7fdacb400640 1 -- 172.21.15.67:0/2338744847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdabc018070 con 0x7fdab80052e0 2024-05-30T16:44:29.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.545+0000 7fdacb400640 1 -- 172.21.15.67:0/2338744847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc003810 con 0x7fdab80052e0 2024-05-30T16:44:29.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.545+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41f9110 con 0x7fdab80052e0 2024-05-30T16:44:29.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.545+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad409a570 con 0x7fdab80052e0 2024-05-30T16:44:29.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.546+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad40689e0 con 0x7fdab80052e0 2024-05-30T16:44:29.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.547+0000 7fdacb400640 1 -- 172.21.15.67:0/2338744847 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdabc0039b0 con 0x7fdab80052e0 2024-05-30T16:44:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.548+0000 7fdacb400640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdab006c920 0x7fdab006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.548+0000 7fdacb400640 1 -- 172.21.15.67:0/2338744847 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdabc08b830 con 0x7fdab80052e0 2024-05-30T16:44:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.548+0000 7fdad2a00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdab006c920 0x7fdab006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.548+0000 7fdad2a00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdab006c920 0x7fdab006ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdad4069230 tx=0x7fdac400d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:29.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.552+0000 7fdacb400640 1 -- 172.21.15.67:0/2338744847 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdabc059c50 con 0x7fdab80052e0 2024-05-30T16:44:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.763+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdad40b2c30 con 0x7fdab80052e0 2024-05-30T16:44:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.764+0000 7fdacb400640 1 -- 172.21.15.67:0/2338744847 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fdabc059550 con 0x7fdab80052e0 2024-05-30T16:44:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:44:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdab006c920 msgr2=0x7fdab006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdab006c920 0x7fdab006ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdad4069230 tx=0x7fdac400d040 comp rx=0 tx=0).stop 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 msgr2=0x7fdad41536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdad41536e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00e4d0 tx=0x7fdabc00e990 comp rx=0 tx=0).stop 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 shutdown_connections 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdab006c920 0x7fdab006ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdad41f8b90 0x7fdad409ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdad40ccca0 0x7fdad41f8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 --2- 172.21.15.67:0/2338744847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdab80052e0 0x7fdad41536e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 >> 172.21.15.67:0/2338744847 conn(0x7fdad40b1fc0 msgr2=0x7fdad40b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 shutdown_connections 2024-05-30T16:44:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:29.767+0000 7fdad8a00640 1 -- 172.21.15.67:0/2338744847 wait complete. 2024-05-30T16:44:29.787 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T16:44:29.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:29.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-30T16:44:29.870 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:44:29.870 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:44:29.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:29.870 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-30T16:44:29.923 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:44:29.923 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:29.980 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:44:29.980 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:30.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:44:30.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.256+0000 7f7e8ea00640 1 Processor -- start 2024-05-30T16:44:30.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.256+0000 7f7e8ea00640 1 -- start start 2024-05-30T16:44:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.258+0000 7f7e8ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e880d0930 0x7f7e880ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.258+0000 7f7e8ea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8806ed10 con 0x7f7e8806f860 2024-05-30T16:44:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.258+0000 7f7e8ea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8815cb00 con 0x7f7e880d04a0 2024-05-30T16:44:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.258+0000 7f7e8ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7e880b3cc0 con 0x7f7e880d0930 2024-05-30T16:44:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.258+0000 7f7e87e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7e8806f860 0x7f7e880b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56804/0 (socket says 172.21.15.67:56804) 2024-05-30T16:44:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.258+0000 7f7e87e00640 1 -- 172.21.15.67:0/1747762588 learned_addr learned my addr 172.21.15.67:0/1747762588 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.258+0000 7f7e8ca00640 1 --2- 172.21.15.67:0/1747762588 >> v2:172.21.15.169:3300/0 conn(0x7f7e880d0930 0x7f7e880ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.258+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3117915856 0 0) 0x7f7e78001110 con 0x7f7e8806f860 2024-05-30T16:44:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.258+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e6c003520 con 0x7f7e8806f860 2024-05-30T16:44:30.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3433301276 0 0) 0x7f7e6c003520 con 0x7f7e8806f860 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 >> v2:172.21.15.169:3300/0 conn(0x7f7e880d0930 msgr2=0x7f7e880ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 --2- 172.21.15.67:0/1747762588 >> v2:172.21.15.169:3300/0 conn(0x7f7e880d0930 0x7f7e880ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 >> v1:172.21.15.169:6789/0 conn(0x7f7e880d04a0 legacy=0x7f7e880ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e880b3fa0 con 0x7f7e8806f860 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7e78001840 con 0x7f7e8806f860 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 >> v1:172.21.15.67:6789/0 conn(0x7f7e8806f860 legacy=0x7f7e880b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6c004800 0x7f7e6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e6c0051c0 0x7f7e6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7e880b3cc0 con 0x7f7e6c0039b0 2024-05-30T16:44:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c00c060 con 0x7f7e6c004800 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6c00c260 con 0x7f7e6c0051c0 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.259+0000 7f7e87e00640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.260+0000 7f7e87e00640 1 -- 172.21.15.67:0/1747762588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e6c0051c0 msgr2=0x7f7e6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.260+0000 7f7e87e00640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e6c0051c0 0x7f7e6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.260+0000 7f7e7f400640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6c004800 0x7f7e6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.260+0000 7f7e87e00640 1 -- 172.21.15.67:0/1747762588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6c004800 msgr2=0x7f7e6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.260+0000 7f7e87e00640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6c004800 0x7f7e6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.260+0000 7f7e87e00640 1 -- 172.21.15.67:0/1747762588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c003520 con 0x7f7e6c0039b0 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.260+0000 7f7e8ca00640 1 -- 172.21.15.67:0/1747762588 reap_dead start 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.260+0000 7f7e7f400640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6c004800 0x7f7e6c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.261+0000 7f7e87e00640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e6c003dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800c4c0 tx=0x7f7e6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b61163fb8739ca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.261+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800c0e0 con 0x7f7e6c0039b0 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.261+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78034070 con 0x7f7e6c0039b0 2024-05-30T16:44:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.261+0000 7f7e87400640 1 -- 172.21.15.67:0/1747762588 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78005d70 con 0x7f7e6c0039b0 2024-05-30T16:44:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.262+0000 7f7e8ea00640 1 -- 172.21.15.67:0/1747762588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 msgr2=0x7f7e6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.262+0000 7f7e8ea00640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e6c003dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800c4c0 tx=0x7f7e6c005830 comp rx=0 tx=0).stop 2024-05-30T16:44:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.262+0000 7f7e8ea00640 1 -- 172.21.15.67:0/1747762588 shutdown_connections 2024-05-30T16:44:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.262+0000 7f7e8ea00640 1 --2- 172.21.15.67:0/1747762588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e6c003dd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.262+0000 7f7e8ea00640 1 -- 172.21.15.67:0/1747762588 >> 172.21.15.67:0/1747762588 conn(0x7f7e8806bae0 msgr2=0x7f7e8806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.262+0000 7f7e8ea00640 1 -- 172.21.15.67:0/1747762588 shutdown_connections 2024-05-30T16:44:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.262+0000 7f7e8ea00640 1 -- 172.21.15.67:0/1747762588 wait complete. 2024-05-30T16:44:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.264+0000 7f7e8ea00640 1 Processor -- start 2024-05-30T16:44:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.264+0000 7f7e8ea00640 1 -- start start 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e8ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e880ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e8ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e8815ade0 0x7f7e880c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e8ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8815d760 0x7f7e8815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e8ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7e880b4840 con 0x7f7e6c0039b0 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e8ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e880b4560 con 0x7f7e8815ade0 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e8ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e880d0d40 con 0x7f7e8815d760 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e87e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e880ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e87e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e880ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49352/0 (socket says 172.21.15.67:49352) 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e87e00640 1 -- 172.21.15.67:0/3967885358 learned_addr learned my addr 172.21.15.67:0/3967885358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e87e00640 1 -- 172.21.15.67:0/3967885358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8815d760 msgr2=0x7f7e8815dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e87e00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8815d760 0x7f7e8815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e87e00640 1 -- 172.21.15.67:0/3967885358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e8815ade0 msgr2=0x7f7e880c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e87e00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e8815ade0 0x7f7e880c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.265+0000 7f7e87e00640 1 -- 172.21.15.67:0/3967885358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e880c3a50 con 0x7f7e6c0039b0 2024-05-30T16:44:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.266+0000 7f7e87e00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e880ccff0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800c4c0 tx=0x7f7e880b4170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.266+0000 7f7e84c00640 1 -- 172.21.15.67:0/3967885358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78000ed0 con 0x7f7e6c0039b0 2024-05-30T16:44:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.266+0000 7f7e84c00640 1 -- 172.21.15.67:0/3967885358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7803f070 con 0x7f7e6c0039b0 2024-05-30T16:44:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.266+0000 7f7e84c00640 1 -- 172.21.15.67:0/3967885358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78007290 con 0x7f7e6c0039b0 2024-05-30T16:44:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.266+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e880c6970 con 0x7f7e6c0039b0 2024-05-30T16:44:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.266+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881e5330 con 0x7f7e6c0039b0 2024-05-30T16:44:30.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.267+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e880689e0 con 0x7f7e6c0039b0 2024-05-30T16:44:30.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.268+0000 7f7e84c00640 1 -- 172.21.15.67:0/3967885358 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e78007430 con 0x7f7e6c0039b0 2024-05-30T16:44:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.269+0000 7f7e84c00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6406cac0 0x7f7e6406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.269+0000 7f7e84c00640 1 -- 172.21.15.67:0/3967885358 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7e780acdb0 con 0x7f7e6c0039b0 2024-05-30T16:44:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.269+0000 7f7e7f400640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6406cac0 0x7f7e6406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.270+0000 7f7e7f400640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6406cac0 0x7f7e6406ef80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e88063510 tx=0x7f7e70009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.274+0000 7f7e84c00640 1 -- 172.21.15.67:0/3967885358 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e780b1070 con 0x7f7e6c0039b0 2024-05-30T16:44:30.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.490+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7e8806e2f0 con 0x7f7e6c0039b0 2024-05-30T16:44:30.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.491+0000 7f7e84c00640 1 -- 172.21.15.67:0/3967885358 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f7e78049060 con 0x7f7e6c0039b0 2024-05-30T16:44:30.491 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6406cac0 msgr2=0x7f7e6406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6406cac0 0x7f7e6406ef80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e88063510 tx=0x7f7e70009040 comp rx=0 tx=0).stop 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 msgr2=0x7f7e880ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e880ccff0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800c4c0 tx=0x7f7e880b4170 comp rx=0 tx=0).stop 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 shutdown_connections 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6406cac0 0x7f7e6406ef80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8815d760 0x7f7e8815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e8815ade0 0x7f7e880c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 --2- 172.21.15.67:0/3967885358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e6c0039b0 0x7f7e880ccff0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 >> 172.21.15.67:0/3967885358 conn(0x7f7e8806bae0 msgr2=0x7f7e880b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:30.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 shutdown_connections 2024-05-30T16:44:30.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.495+0000 7f7e8ea00640 1 -- 172.21.15.67:0/3967885358 wait complete. 2024-05-30T16:44:30.515 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:44:30.515 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:44:30.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:44:30.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.797+0000 7f8aaf200640 1 Processor -- start 2024-05-30T16:44:30.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.797+0000 7f8aaf200640 1 -- start start 2024-05-30T16:44:30.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.798+0000 7f8aaf200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8aa815e870 0x7f8aa80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.798+0000 7f8aaf200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa815ec50 con 0x7f8aa80cfa40 2024-05-30T16:44:30.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.798+0000 7f8aaf200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa80cff80 con 0x7f8aa80d14d0 2024-05-30T16:44:30.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.798+0000 7f8aaf200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa80d2150 con 0x7f8aa815e870 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.799+0000 7f8aae800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8aa80cfa40 0x7f8aa80d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49706/0 (socket says 172.21.15.67:49706) 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.799+0000 7f8aae800640 1 -- 172.21.15.67:0/3144940555 learned_addr learned my addr 172.21.15.67:0/3144940555 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.799+0000 7f8aade00640 1 --2- 172.21.15.67:0/3144940555 >> v2:172.21.15.169:3300/0 conn(0x7f8aa815e870 0x7f8aa80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.799+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1834239435 0 0) 0x7f8aa815ec50 con 0x7f8aa80cfa40 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.799+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a90003520 con 0x7f8aa80cfa40 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1425663735 0 0) 0x7f8a90003520 con 0x7f8aa80cfa40 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 >> v2:172.21.15.169:3300/0 conn(0x7f8aa815e870 msgr2=0x7f8aa80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 --2- 172.21.15.67:0/3144940555 >> v2:172.21.15.169:3300/0 conn(0x7f8aa815e870 0x7f8aa80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 >> v1:172.21.15.169:6789/0 conn(0x7f8aa80d14d0 legacy=0x7f8aa80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa80d0260 con 0x7f8aa80cfa40 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f8aa40019f0 con 0x7f8aa80cfa40 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 >> v1:172.21.15.67:6789/0 conn(0x7f8aa80cfa40 legacy=0x7f8aa80d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a90003a00 0x7f8a90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a90004800 0x7f8a90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8a9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa80d2150 con 0x7f8a900051c0 2024-05-30T16:44:30.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9000c0c0 con 0x7f8a90003a00 2024-05-30T16:44:30.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9000c2c0 con 0x7f8a90004800 2024-05-30T16:44:30.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.800+0000 7f8aae800640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8a9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.801+0000 7f8aad400640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a90004800 0x7f8a90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.801+0000 7f8aade00640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a90003a00 0x7f8a90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.801+0000 7f8aae800640 1 -- 172.21.15.67:0/3144940555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a90004800 msgr2=0x7f8a90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.801+0000 7f8aae800640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a90004800 0x7f8a90004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.801+0000 7f8aae800640 1 -- 172.21.15.67:0/3144940555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a90003a00 msgr2=0x7f8a90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.801+0000 7f8aae800640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a90003a00 0x7f8a90003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.801+0000 7f8aae800640 1 -- 172.21.15.67:0/3144940555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a90003520 con 0x7f8a900051c0 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.802+0000 7f8aae800640 1 -- 172.21.15.67:0/3144940555 reap_dead start 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.802+0000 7f8aae800640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8a9000b9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400a780 tx=0x7f8a90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce403ee47ecf033 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.802+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400a3d0 con 0x7f8a900051c0 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.802+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa402f070 con 0x7f8a900051c0 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.802+0000 7f8aaca00640 1 -- 172.21.15.67:0/3144940555 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa402bbe0 con 0x7f8a900051c0 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.803+0000 7f8aaf200640 1 -- 172.21.15.67:0/3144940555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 msgr2=0x7f8a9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.803+0000 7f8aaf200640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8a9000b9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400a780 tx=0x7f8a90005830 comp rx=0 tx=0).stop 2024-05-30T16:44:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.803+0000 7f8aaf200640 1 -- 172.21.15.67:0/3144940555 shutdown_connections 2024-05-30T16:44:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.803+0000 7f8aaf200640 1 --2- 172.21.15.67:0/3144940555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8a9000b9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.803+0000 7f8aaf200640 1 -- 172.21.15.67:0/3144940555 >> 172.21.15.67:0/3144940555 conn(0x7f8aa808bfc0 msgr2=0x7f8aa80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.803+0000 7f8aaf200640 1 -- 172.21.15.67:0/3144940555 shutdown_connections 2024-05-30T16:44:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.803+0000 7f8aaf200640 1 -- 172.21.15.67:0/3144940555 wait complete. 2024-05-30T16:44:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.805+0000 7f8aaf200640 1 Processor -- start 2024-05-30T16:44:30.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.805+0000 7f8aaf200640 1 -- start start 2024-05-30T16:44:30.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aaf200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8aa81509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aaf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8aa80b93a0 0x7f8aa80b9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aaf200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aa80b7110 0x7f8aa80b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aaf200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa80d4fb0 con 0x7f8a900051c0 2024-05-30T16:44:30.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aaf200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa80d0820 con 0x7f8aa80b7110 2024-05-30T16:44:30.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aaf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa80d0f90 con 0x7f8aa80b93a0 2024-05-30T16:44:30.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aade00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8aa81509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aade00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8aa81509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36270/0 (socket says 172.21.15.67:36270) 2024-05-30T16:44:30.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aade00640 1 -- 172.21.15.67:0/4202414694 learned_addr learned my addr 172.21.15.67:0/4202414694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:30.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aad400640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8aa80b93a0 0x7f8aa80b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aae800640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aa80b7110 0x7f8aa80b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aade00640 1 -- 172.21.15.67:0/4202414694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8aa80b93a0 msgr2=0x7f8aa80b9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aade00640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8aa80b93a0 0x7f8aa80b9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aade00640 1 -- 172.21.15.67:0/4202414694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aa80b7110 msgr2=0x7f8aa80b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:30.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aade00640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aa80b7110 0x7f8aa80b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:30.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.806+0000 7f8aade00640 1 -- 172.21.15.67:0/4202414694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa80b8e30 con 0x7f8a900051c0 2024-05-30T16:44:30.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.807+0000 7f8aade00640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8aa81509e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c010890 tx=0x7f8a9c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:30.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.807+0000 7f8aa2000640 1 -- 172.21.15.67:0/4202414694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00f040 con 0x7f8a900051c0 2024-05-30T16:44:30.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.807+0000 7f8aa2000640 1 -- 172.21.15.67:0/4202414694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9c00eb00 con 0x7f8a900051c0 2024-05-30T16:44:30.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.807+0000 7f8aa2000640 1 -- 172.21.15.67:0/4202414694 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c01c400 con 0x7f8a900051c0 2024-05-30T16:44:30.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.807+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa8150f20 con 0x7f8a900051c0 2024-05-30T16:44:30.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.807+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa80b7d90 con 0x7f8a900051c0 2024-05-30T16:44:30.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.808+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa80689e0 con 0x7f8a900051c0 2024-05-30T16:44:30.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.809+0000 7f8aa2000640 1 -- 172.21.15.67:0/4202414694 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a9c01c5a0 con 0x7f8a900051c0 2024-05-30T16:44:30.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.810+0000 7f8aa2000640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:30.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.810+0000 7f8aa2000640 1 -- 172.21.15.67:0/4202414694 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8a9c08c1c0 con 0x7f8a900051c0 2024-05-30T16:44:30.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.811+0000 7f8aad400640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:30.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.811+0000 7f8aad400640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80b9230 tx=0x7f8a98009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:30.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:30.815+0000 7f8aa2000640 1 -- 172.21.15.67:0/4202414694 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8aa8150f20 con 0x7f8a900051c0 2024-05-30T16:44:31.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.022+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8aa80ba500 con 0x7f8a900051c0 2024-05-30T16:44:31.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.023+0000 7f8aa2000640 1 -- 172.21.15.67:0/4202414694 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f8aa80b8e30 con 0x7f8a900051c0 2024-05-30T16:44:31.023 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:44:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a8c06c8f0 msgr2=0x7f8a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80b9230 tx=0x7f8a98009040 comp rx=0 tx=0).stop 2024-05-30T16:44:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 msgr2=0x7f8aa81509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8aa81509e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c010890 tx=0x7f8a9c010d50 comp rx=0 tx=0).stop 2024-05-30T16:44:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 shutdown_connections 2024-05-30T16:44:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aa80b7110 0x7f8aa80b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8aa80b93a0 0x7f8aa80b9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 --2- 172.21.15.67:0/4202414694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a900051c0 0x7f8aa81509e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 >> 172.21.15.67:0/4202414694 conn(0x7f8aa808bfc0 msgr2=0x7f8aa80b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 shutdown_connections 2024-05-30T16:44:31.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.025+0000 7f8aaf200640 1 -- 172.21.15.67:0/4202414694 wait complete. 2024-05-30T16:44:31.045 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi169.front.sepia.ceph.com... 2024-05-30T16:44:31.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:31.045 DEBUG:teuthology.orchestra.run.smithi169:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-30T16:44:31.131 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:44:31.131 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T16:44:31.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:31.131 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-30T16:44:31.182 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-30T16:44:31.183 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:44:31.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.397+0000 7f359ca00640 1 Processor -- start 2024-05-30T16:44:31.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.397+0000 7f359ca00640 1 -- start start 2024-05-30T16:44:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.398+0000 7f359ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3598073310 0x7f35980b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.398+0000 7f359ca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35980afd60 con 0x7f359815b3a0 2024-05-30T16:44:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.398+0000 7f359ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35980af7a0 con 0x7f359806dc20 2024-05-30T16:44:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.398+0000 7f359ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f359806f980 con 0x7f3598073310 2024-05-30T16:44:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.399+0000 7f3596800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f359815b3a0 0x7f3598072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49710/0 (socket says 172.21.15.67:49710) 2024-05-30T16:44:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.399+0000 7f3596800640 1 -- 172.21.15.67:0/2978894940 learned_addr learned my addr 172.21.15.67:0/2978894940 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:31.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.399+0000 7f3595e00640 1 --2- 172.21.15.67:0/2978894940 >> v2:172.21.15.169:3300/0 conn(0x7f3598073310 0x7f35980b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.399+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188239544 0 0) 0x7f35980afd60 con 0x7f359815b3a0 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.399+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f357c003520 con 0x7f359815b3a0 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1441343967 0 0) 0x7f357c003520 con 0x7f359815b3a0 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 >> v2:172.21.15.169:3300/0 conn(0x7f3598073310 msgr2=0x7f35980b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 --2- 172.21.15.67:0/2978894940 >> v2:172.21.15.169:3300/0 conn(0x7f3598073310 0x7f35980b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 >> v1:172.21.15.169:6789/0 conn(0x7f359806dc20 legacy=0x7f35980ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35980afa80 con 0x7f359815b3a0 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f3588001670 con 0x7f359815b3a0 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 >> v1:172.21.15.67:6789/0 conn(0x7f359815b3a0 legacy=0x7f3598072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c0039b0 0x7f357c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f357c004800 0x7f357c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357c0051c0 0x7f357c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f359806f980 con 0x7f357c004800 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f357c00e270 con 0x7f357c0039b0 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f357c00e470 con 0x7f357c0051c0 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3595e00640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f357c004800 0x7f357c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.400+0000 7f3596800640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c0039b0 0x7f357c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.401+0000 7f3595e00640 1 -- 172.21.15.67:0/2978894940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357c0051c0 msgr2=0x7f357c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.401+0000 7f3595e00640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357c0051c0 0x7f357c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.401+0000 7f3595e00640 1 -- 172.21.15.67:0/2978894940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c0039b0 msgr2=0x7f357c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.401+0000 7f3595e00640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c0039b0 0x7f357c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.401+0000 7f3595e00640 1 -- 172.21.15.67:0/2978894940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357c003520 con 0x7f357c004800 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.401+0000 7f3596800640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f357c0039b0 0x7f357c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.401+0000 7f3597200640 1 -- 172.21.15.67:0/2978894940 reap_dead start 2024-05-30T16:44:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.402+0000 7f3595e00640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f357c004800 0x7f357c004c80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3584009ac0 tx=0x7f357c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80f0135c1f7738a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.402+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584036070 con 0x7f357c004800 2024-05-30T16:44:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.402+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3584031400 con 0x7f357c004800 2024-05-30T16:44:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.402+0000 7f3595400640 1 -- 172.21.15.67:0/2978894940 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584009690 con 0x7f357c004800 2024-05-30T16:44:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.402+0000 7f359ca00640 1 -- 172.21.15.67:0/2978894940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f357c004800 msgr2=0x7f357c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.402+0000 7f359ca00640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f357c004800 0x7f357c004c80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3584009ac0 tx=0x7f357c005830 comp rx=0 tx=0).stop 2024-05-30T16:44:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.403+0000 7f359ca00640 1 -- 172.21.15.67:0/2978894940 shutdown_connections 2024-05-30T16:44:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.403+0000 7f359ca00640 1 --2- 172.21.15.67:0/2978894940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f357c004800 0x7f357c004c80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.403+0000 7f359ca00640 1 -- 172.21.15.67:0/2978894940 >> 172.21.15.67:0/2978894940 conn(0x7f35980b1fc0 msgr2=0x7f35980b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.403+0000 7f359ca00640 1 -- 172.21.15.67:0/2978894940 shutdown_connections 2024-05-30T16:44:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.403+0000 7f359ca00640 1 -- 172.21.15.67:0/2978894940 wait complete. 2024-05-30T16:44:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.404+0000 7f359ca00640 1 Processor -- start 2024-05-30T16:44:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.405+0000 7f359ca00640 1 -- start start 2024-05-30T16:44:31.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f359ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357c004800 0x7f35980cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f359ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359814dc30 0x7f35981561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f359ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f359814d610 0x7f35981566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f359ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f35980b32c0 con 0x7f359814d610 2024-05-30T16:44:31.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f359ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35980ae0c0 con 0x7f359814dc30 2024-05-30T16:44:31.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f359ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35980b3440 con 0x7f357c004800 2024-05-30T16:44:31.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3597200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f359814d610 0x7f35981566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3597200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f359814d610 0x7f35981566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36286/0 (socket says 172.21.15.67:36286) 2024-05-30T16:44:31.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3597200640 1 -- 172.21.15.67:0/1362829460 learned_addr learned my addr 172.21.15.67:0/1362829460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:31.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3596800640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357c004800 0x7f35980cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3595e00640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359814dc30 0x7f35981561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3597200640 1 -- 172.21.15.67:0/1362829460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357c004800 msgr2=0x7f35980cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3597200640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357c004800 0x7f35980cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3597200640 1 -- 172.21.15.67:0/1362829460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359814dc30 msgr2=0x7f35981561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3597200640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359814dc30 0x7f35981561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.406+0000 7f3597200640 1 -- 172.21.15.67:0/1362829460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359815b990 con 0x7f359814d610 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.407+0000 7f3595e00640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359814dc30 0x7f35981561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.407+0000 7f3597200640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f359814d610 0x7f35981566e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3598074120 tx=0x7f359000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.407+0000 7f358ea00640 1 -- 172.21.15.67:0/1362829460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590007070 con 0x7f359814d610 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.407+0000 7f358ea00640 1 -- 172.21.15.67:0/1362829460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3590007230 con 0x7f359814d610 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.407+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3598155100 con 0x7f359814d610 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.407+0000 7f358ea00640 1 -- 172.21.15.67:0/1362829460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590007620 con 0x7f359814d610 2024-05-30T16:44:31.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.407+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35981556c0 con 0x7f359814d610 2024-05-30T16:44:31.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.408+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35980689e0 con 0x7f359814d610 2024-05-30T16:44:31.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.409+0000 7f358ea00640 1 -- 172.21.15.67:0/1362829460 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3590011070 con 0x7f359814d610 2024-05-30T16:44:31.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.410+0000 7f358ea00640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f357806c8f0 0x7f357806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.410+0000 7f358ea00640 1 -- 172.21.15.67:0/1362829460 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f359008ab90 con 0x7f359814d610 2024-05-30T16:44:31.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.411+0000 7f3596800640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f357806c8f0 0x7f357806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.411+0000 7f3596800640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f357806c8f0 0x7f357806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f35880017e0 tx=0x7f3588009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:31.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.415+0000 7f358ea00640 1 -- 172.21.15.67:0/1362829460 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3590019060 con 0x7f359814d610 2024-05-30T16:44:31.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.660+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35980cc2a0 con 0x7f359814d610 2024-05-30T16:44:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.661+0000 7f358ea00640 1 -- 172.21.15.67:0/1362829460 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f3590058f20 con 0x7f359814d610 2024-05-30T16:44:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:31.662 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2254297887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2254297887},{"type":"v1","addr":"172.21.15.67:6837","nonce":2254297887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/227511509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":227511509},{"type":"v1","addr":"172.21.15.169:6834","nonce":227511509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:43:52.308425+0000","modified":"2024-05-30T16:43:55.372983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2028878495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2028878495},{"type":"v1","addr":"172.21.15.169:6835","nonce":2028878495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-30T16:44:31.662 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.663+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f357806c8f0 msgr2=0x7f357806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.663+0000 7f359ca00640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f357806c8f0 0x7f357806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f35880017e0 tx=0x7f3588009040 comp rx=0 tx=0).stop 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.663+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f359814d610 msgr2=0x7f35981566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.663+0000 7f359ca00640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f359814d610 0x7f35981566e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3598074120 tx=0x7f359000c820 comp rx=0 tx=0).stop 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.664+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 shutdown_connections 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.664+0000 7f359ca00640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f357806c8f0 0x7f357806edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.664+0000 7f359ca00640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f359814d610 0x7f35981566e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.664+0000 7f359ca00640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f359814dc30 0x7f35981561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.664+0000 7f359ca00640 1 --2- 172.21.15.67:0/1362829460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f357c004800 0x7f35980cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.664+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 >> 172.21.15.67:0/1362829460 conn(0x7f35980b1fc0 msgr2=0x7f35980b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.664+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 shutdown_connections 2024-05-30T16:44:31.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.664+0000 7f359ca00640 1 -- 172.21.15.67:0/1362829460 wait complete. 2024-05-30T16:44:31.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:44:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.898+0000 7f2c43200640 1 Processor -- start 2024-05-30T16:44:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.898+0000 7f2c43200640 1 -- start start 2024-05-30T16:44:31.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.899+0000 7f2c43200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2c44156b30 0x7f2c440af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.899+0000 7f2c43200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c4406ee40 0x7f2c440ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.899+0000 7f2c43200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c44156f10 con 0x7f2c4406f2f0 2024-05-30T16:44:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.899+0000 7f2c43200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4406dda0 con 0x7f2c4406ee40 2024-05-30T16:44:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.899+0000 7f2c43200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4406e080 con 0x7f2c44156b30 2024-05-30T16:44:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c41400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c4406ee40 0x7f2c440ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c41400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c4406ee40 0x7f2c440ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36298/0 (socket says 172.21.15.67:36298) 2024-05-30T16:44:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c41400640 1 -- 172.21.15.67:0/580704071 learned_addr learned my addr 172.21.15.67:0/580704071 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c42800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2c4406f2f0 0x7f2c440b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49714/0 (socket says 172.21.15.67:49714) 2024-05-30T16:44:31.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c41e00640 1 --2- 172.21.15.67:0/580704071 >> v2:172.21.15.169:3300/0 conn(0x7f2c44156b30 0x7f2c440af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c41400640 1 -- 172.21.15.67:0/580704071 >> v2:172.21.15.169:3300/0 conn(0x7f2c44156b30 msgr2=0x7f2c440af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:31.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c41400640 1 --2- 172.21.15.67:0/580704071 >> v2:172.21.15.169:3300/0 conn(0x7f2c44156b30 0x7f2c440af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c41400640 1 -- 172.21.15.67:0/580704071 >> v1:172.21.15.67:6789/0 conn(0x7f2c4406f2f0 legacy=0x7f2c440b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c40a00640 1 -- 172.21.15.67:0/580704071 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003211852 0 0) 0x7f2c44156f10 con 0x7f2c4406f2f0 2024-05-30T16:44:31.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c41400640 1 -- 172.21.15.67:0/580704071 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4406e360 con 0x7f2c4406ee40 2024-05-30T16:44:31.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.900+0000 7f2c41e00640 1 --2- 172.21.15.67:0/580704071 >> v2:172.21.15.169:3300/0 conn(0x7f2c44156b30 0x7f2c440af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:31.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.902+0000 7f2c41400640 1 --2- 172.21.15.67:0/580704071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 0x7f2c440ad230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c4406c8b0 tx=0x7f2c3002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61df2ed1ff7aab57 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.902+0000 7f2c40a00640 1 -- 172.21.15.67:0/580704071 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30037070 con 0x7f2c4406ee40 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.902+0000 7f2c40a00640 1 -- 172.21.15.67:0/580704071 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c30032440 con 0x7f2c4406ee40 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.902+0000 7f2c40a00640 1 -- 172.21.15.67:0/580704071 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30032810 con 0x7f2c4406ee40 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.902+0000 7f2c43200640 1 -- 172.21.15.67:0/580704071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 msgr2=0x7f2c440ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.902+0000 7f2c43200640 1 --2- 172.21.15.67:0/580704071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 0x7f2c440ad230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c4406c8b0 tx=0x7f2c3002a1e0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.903+0000 7f2c43200640 1 -- 172.21.15.67:0/580704071 shutdown_connections 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.903+0000 7f2c43200640 1 --2- 172.21.15.67:0/580704071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 0x7f2c440ad230 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.903+0000 7f2c43200640 1 --2- 172.21.15.67:0/580704071 >> v2:172.21.15.169:3300/0 conn(0x7f2c44156b30 0x7f2c440af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.903+0000 7f2c43200640 1 -- 172.21.15.67:0/580704071 >> 172.21.15.67:0/580704071 conn(0x7f2c440b1fc0 msgr2=0x7f2c440b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.903+0000 7f2c43200640 1 -- 172.21.15.67:0/580704071 shutdown_connections 2024-05-30T16:44:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.903+0000 7f2c43200640 1 -- 172.21.15.67:0/580704071 wait complete. 2024-05-30T16:44:31.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.904+0000 7f2c43200640 1 Processor -- start 2024-05-30T16:44:31.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.905+0000 7f2c43200640 1 -- start start 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.905+0000 7f2c43200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 0x7f2c4415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c43200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c4406f2f0 0x7f2c441f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c43200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c44156b30 0x7f2c441f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c43200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2c440afc20 con 0x7f2c4406ee40 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c43200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2c440b2c30 con 0x7f2c44156b30 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c43200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4414dff0 con 0x7f2c4406f2f0 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c41e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 0x7f2c4415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c41e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 0x7f2c4415ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36300/0 (socket says 172.21.15.67:36300) 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c41e00640 1 -- 172.21.15.67:0/2484019692 learned_addr learned my addr 172.21.15.67:0/2484019692 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c41e00640 1 -- 172.21.15.67:0/2484019692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c4406f2f0 msgr2=0x7f2c441f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c41e00640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c4406f2f0 0x7f2c441f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c41e00640 1 -- 172.21.15.67:0/2484019692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c44156b30 msgr2=0x7f2c441f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:31.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c41e00640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c44156b30 0x7f2c441f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:31.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.906+0000 7f2c41e00640 1 -- 172.21.15.67:0/2484019692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4414d6c0 con 0x7f2c4406ee40 2024-05-30T16:44:31.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.907+0000 7f2c41e00640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 0x7f2c4415ba60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c820 tx=0x7f2c3400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:31.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.907+0000 7f2c3a000640 1 -- 172.21.15.67:0/2484019692 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34015070 con 0x7f2c4406ee40 2024-05-30T16:44:31.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.907+0000 7f2c3a000640 1 -- 172.21.15.67:0/2484019692 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c34007a30 con 0x7f2c4406ee40 2024-05-30T16:44:31.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.907+0000 7f2c3a000640 1 -- 172.21.15.67:0/2484019692 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34007de0 con 0x7f2c4406ee40 2024-05-30T16:44:31.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.907+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4414d9a0 con 0x7f2c4406ee40 2024-05-30T16:44:31.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.907+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c44072540 con 0x7f2c4406ee40 2024-05-30T16:44:31.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.908+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c440689e0 con 0x7f2c4406ee40 2024-05-30T16:44:31.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.909+0000 7f2c3a000640 1 -- 172.21.15.67:0/2484019692 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c34018070 con 0x7f2c4406ee40 2024-05-30T16:44:31.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.910+0000 7f2c3a000640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c2406c920 0x7f2c2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:31.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.911+0000 7f2c3a000640 1 -- 172.21.15.67:0/2484019692 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2c3408ba90 con 0x7f2c4406ee40 2024-05-30T16:44:31.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.911+0000 7f2c41400640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c2406c920 0x7f2c2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:31.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.912+0000 7f2c41400640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c2406c920 0x7f2c2406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2c30034000 tx=0x7f2c3002c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:31.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:31.915+0000 7f2c3a000640 1 -- 172.21.15.67:0/2484019692 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c34059e20 con 0x7f2c4406ee40 2024-05-30T16:44:32.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.169+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c4406f7f0 con 0x7f2c4406ee40 2024-05-30T16:44:32.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.170+0000 7f2c3a000640 1 -- 172.21.15.67:0/2484019692 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f2c34059720 con 0x7f2c4406ee40 2024-05-30T16:44:32.171 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:32.171 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2254297887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2254297887},{"type":"v1","addr":"172.21.15.67:6837","nonce":2254297887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/227511509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":227511509},{"type":"v1","addr":"172.21.15.169:6834","nonce":227511509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:43:52.308425+0000","modified":"2024-05-30T16:43:55.372983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2028878495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2028878495},{"type":"v1","addr":"172.21.15.169:6835","nonce":2028878495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-30T16:44:32.171 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-05-30T16:44:32.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.172+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c2406c920 msgr2=0x7f2c2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.172+0000 7f2c43200640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c2406c920 0x7f2c2406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2c30034000 tx=0x7f2c3002c000 comp rx=0 tx=0).stop 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.172+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 msgr2=0x7f2c4415ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.172+0000 7f2c43200640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 0x7f2c4415ba60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400c820 tx=0x7f2c3400cce0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.173+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 shutdown_connections 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.173+0000 7f2c43200640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c2406c920 0x7f2c2406ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.173+0000 7f2c43200640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c44156b30 0x7f2c441f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.173+0000 7f2c43200640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c4406f2f0 0x7f2c441f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.173+0000 7f2c43200640 1 --2- 172.21.15.67:0/2484019692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4406ee40 0x7f2c4415ba60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.173+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 >> 172.21.15.67:0/2484019692 conn(0x7f2c440b1fc0 msgr2=0x7f2c44073560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.173+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 shutdown_connections 2024-05-30T16:44:32.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.173+0000 7f2c43200640 1 -- 172.21.15.67:0/2484019692 wait complete. 2024-05-30T16:44:32.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:44:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.376+0000 7fea05000640 1 Processor -- start 2024-05-30T16:44:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.376+0000 7fea05000640 1 -- start start 2024-05-30T16:44:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.378+0000 7fea05000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fea0015b3a0 0x7fea000b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.378+0000 7fea05000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0006dc20 0x7fea0015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.378+0000 7fea05000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea000afd60 con 0x7fea00072c00 2024-05-30T16:44:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.378+0000 7fea05000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fea0006f980 con 0x7fea0015b3a0 2024-05-30T16:44:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.378+0000 7fea05000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fea000ae0c0 con 0x7fea0006dc20 2024-05-30T16:44:32.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9ffe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fea00072c00 0x7fea000ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49728/0 (socket says 172.21.15.67:49728) 2024-05-30T16:44:32.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9ffe00640 1 -- 172.21.15.67:0/1580598231 learned_addr learned my addr 172.21.15.67:0/1580598231 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:32.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9ff400640 1 --2- 172.21.15.67:0/1580598231 >> v2:172.21.15.169:3300/0 conn(0x7fea0015b3a0 0x7fea000b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9f7e00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0006dc20 0x7fea0015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100101233 0 0) 0x7fea000afd60 con 0x7fea00072c00 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9e4005730 con 0x7fea00072c00 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1517070483 0 0) 0x7fe9e4005730 con 0x7fea00072c00 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0006dc20 msgr2=0x7fea0015f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9fea00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0006dc20 0x7fea0015f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 >> v2:172.21.15.169:3300/0 conn(0x7fea0015b3a0 msgr2=0x7fea000b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9fea00640 1 --2- 172.21.15.67:0/1580598231 >> v2:172.21.15.169:3300/0 conn(0x7fea0015b3a0 0x7fea000b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea000af7a0 con 0x7fea00072c00 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.379+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fe9f80019f0 con 0x7fea00072c00 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 >> v1:172.21.15.67:6789/0 conn(0x7fea00072c00 legacy=0x7fea000ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9fea00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9e4005bc0 0x7fe9e4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9fea00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0006dc20 0x7fe9e4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9fea00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9e4006f50 0x7fe9e40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fea0006f980 con 0x7fe9e4006f50 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea000ae0c0 con 0x7fe9e4005bc0 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e400dbd0 con 0x7fea0006dc20 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9ffe00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9e4006f50 0x7fe9e40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9f7e00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0006dc20 0x7fe9e4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9ff400640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9e4005bc0 0x7fe9e4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9ffe00640 1 -- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0006dc20 msgr2=0x7fe9e4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9ffe00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0006dc20 0x7fe9e4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9ffe00640 1 -- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9e4005bc0 msgr2=0x7fe9e4005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9ffe00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9e4005bc0 0x7fe9e4005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.380+0000 7fe9ffe00640 1 -- 172.21.15.67:0/1580598231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e4005730 con 0x7fe9e4006f50 2024-05-30T16:44:32.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.381+0000 7fe9ffe00640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9e4006f50 0x7fe9e40073d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800a7d0 tx=0x7fe9e4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00fe33f45e494c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:32.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.382+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800a3f0 con 0x7fe9e4006f50 2024-05-30T16:44:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.382+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f802f070 con 0x7fe9e4006f50 2024-05-30T16:44:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.382+0000 7fe9fea00640 1 -- 172.21.15.67:0/1580598231 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f802bcd0 con 0x7fe9e4006f50 2024-05-30T16:44:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.382+0000 7fea05000640 1 -- 172.21.15.67:0/1580598231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9e4006f50 msgr2=0x7fe9e40073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.382+0000 7fea05000640 1 --2- 172.21.15.67:0/1580598231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9e4006f50 0x7fe9e40073d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800a7d0 tx=0x7fe9e4007a40 comp rx=0 tx=0).stop 2024-05-30T16:44:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.382+0000 7fe9ffe00640 1 -- 172.21.15.67:0/1580598231 reap_dead start 2024-05-30T16:44:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.382+0000 7fea05000640 1 -- 172.21.15.67:0/1580598231 shutdown_connections 2024-05-30T16:44:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.382+0000 7fea05000640 1 -- 172.21.15.67:0/1580598231 >> 172.21.15.67:0/1580598231 conn(0x7fea000b1fc0 msgr2=0x7fea000b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.382+0000 7fea05000640 1 -- 172.21.15.67:0/1580598231 shutdown_connections 2024-05-30T16:44:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.383+0000 7fea05000640 1 -- 172.21.15.67:0/1580598231 wait complete. 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.384+0000 7fea05000640 1 Processor -- start 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.384+0000 7fea05000640 1 -- start start 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.385+0000 7fea05000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9e4006f50 0x7fea00147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.385+0000 7fea05000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea00146490 0x7fea00155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.385+0000 7fea05000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea00146910 0x7fea00153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.385+0000 7fea05000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fea00073050 con 0x7fea00146910 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.385+0000 7fea05000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea0015b780 con 0x7fe9e4006f50 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.385+0000 7fea05000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fea000731d0 con 0x7fea00146490 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9ffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea00146910 0x7fea00153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9ffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea00146910 0x7fea00153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36328/0 (socket says 172.21.15.67:36328) 2024-05-30T16:44:32.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9ffe00640 1 -- 172.21.15.67:0/36893868 learned_addr learned my addr 172.21.15.67:0/36893868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:32.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9f7e00640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea00146490 0x7fea00155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9ff400640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9e4006f50 0x7fea00147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9ffe00640 1 -- 172.21.15.67:0/36893868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea00146490 msgr2=0x7fea00155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9ffe00640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea00146490 0x7fea00155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9ffe00640 1 -- 172.21.15.67:0/36893868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9e4006f50 msgr2=0x7fea00147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9ffe00640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9e4006f50 0x7fea00147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.386+0000 7fe9ffe00640 1 -- 172.21.15.67:0/36893868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea000ce230 con 0x7fea00146910 2024-05-30T16:44:32.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.387+0000 7fe9ffe00640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea00146910 0x7fea00153f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800a7d0 tx=0x7fe9f8001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:32.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.387+0000 7fe9f7400640 1 -- 172.21.15.67:0/36893868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800a3f0 con 0x7fea00146910 2024-05-30T16:44:32.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.387+0000 7fe9f7400640 1 -- 172.21.15.67:0/36893868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f802bc30 con 0x7fea00146910 2024-05-30T16:44:32.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.387+0000 7fe9f7400640 1 -- 172.21.15.67:0/36893868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f802be00 con 0x7fea00146910 2024-05-30T16:44:32.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.387+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea000ce510 con 0x7fea00146910 2024-05-30T16:44:32.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.387+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea000cead0 con 0x7fea00146910 2024-05-30T16:44:32.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.388+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea000689e0 con 0x7fea00146910 2024-05-30T16:44:32.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.389+0000 7fe9f7400640 1 -- 172.21.15.67:0/36893868 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9f8046020 con 0x7fea00146910 2024-05-30T16:44:32.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.391+0000 7fe9f7400640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe9dc06c920 0x7fe9dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.391+0000 7fe9f7400640 1 -- 172.21.15.67:0/36893868 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe9f80abd30 con 0x7fea00146910 2024-05-30T16:44:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.391+0000 7fe9ff400640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe9dc06c920 0x7fe9dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.392+0000 7fe9ff400640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe9dc06c920 0x7fe9dc06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009a10 tx=0x7fe9f0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.395+0000 7fe9f7400640 1 -- 172.21.15.67:0/36893868 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9f804d020 con 0x7fea00146910 2024-05-30T16:44:32.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.601+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea00153830 con 0x7fea00146910 2024-05-30T16:44:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.603+0000 7fe9f7400640 1 -- 172.21.15.67:0/36893868 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16843 (secure 0 0 0) 0x7fe9f807a0e0 con 0x7fea00146910 2024-05-30T16:44:32.604 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:32.604 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":23,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:44:32.188060+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:43:50.496509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-30T16:43:51.276872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:44:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.605+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe9dc06c920 msgr2=0x7fe9dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.605+0000 7fea05000640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe9dc06c920 0x7fe9dc06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009a10 tx=0x7fe9f0008040 comp rx=0 tx=0).stop 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.605+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea00146910 msgr2=0x7fea00153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.605+0000 7fea05000640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea00146910 0x7fea00153f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800a7d0 tx=0x7fe9f8001cf0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.605+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 shutdown_connections 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.605+0000 7fea05000640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe9dc06c920 0x7fe9dc06ede0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.606+0000 7fea05000640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea00146910 0x7fea00153f10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.606+0000 7fea05000640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea00146490 0x7fea00155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.606+0000 7fea05000640 1 --2- 172.21.15.67:0/36893868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9e4006f50 0x7fea00147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.606+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 >> 172.21.15.67:0/36893868 conn(0x7fea000b1fc0 msgr2=0x7fea000afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.606+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 shutdown_connections 2024-05-30T16:44:32.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.606+0000 7fea05000640 1 -- 172.21.15.67:0/36893868 wait complete. 2024-05-30T16:44:32.625 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-30T16:44:32.625 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:44:32.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.835+0000 7fbe73c00640 1 Processor -- start 2024-05-30T16:44:32.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.836+0000 7fbe73c00640 1 -- start start 2024-05-30T16:44:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.837+0000 7fbe73c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.837+0000 7fbe73c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbe6c0b3690 0x7fbe6c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.837+0000 7fbe73c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe6c06ed10 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.837+0000 7fbe73c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbe6c0d0e50 con 0x7fbe6c0b3690 2024-05-30T16:44:32.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.837+0000 7fbe73c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6c15cb00 con 0x7fbe6c06f860 2024-05-30T16:44:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.838+0000 7fbe71200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.838+0000 7fbe6be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbe6c0b3690 0x7fbe6c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.838+0000 7fbe71200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c0b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:53596/0 (socket says 172.21.15.67:53596) 2024-05-30T16:44:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.838+0000 7fbe71200640 1 -- 172.21.15.67:0/3003758686 learned_addr learned my addr 172.21.15.67:0/3003758686 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:32.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.839+0000 7fbe6be00640 1 -- 172.21.15.67:0/3003758686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 msgr2=0x7fbe6c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:32.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.839+0000 7fbe6be00640 1 --2- 172.21.15.67:0/3003758686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.839+0000 7fbe6be00640 1 -- 172.21.15.67:0/3003758686 >> v1:172.21.15.169:6789/0 conn(0x7fbe6c0d09e0 legacy=0x7fbe6c160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.839+0000 7fbe6be00640 1 -- 172.21.15.67:0/3003758686 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c0b3cc0 con 0x7fbe6c0b3690 2024-05-30T16:44:32.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.839+0000 7fbe71200640 1 --2- 172.21.15.67:0/3003758686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:44:32.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.839+0000 7fbe6b400640 1 -- 172.21.15.67:0/3003758686 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1616731778 0 0) 0x7fbe64001110 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.840+0000 7fbe6be00640 1 --2- 172.21.15.67:0/3003758686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 0x7fbe6c0d04a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009d90 tx=0x7fbe5802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8af23fb220d35e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.841+0000 7fbe6b400640 1 -- 172.21.15.67:0/3003758686 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58007740 con 0x7fbe6c0b3690 2024-05-30T16:44:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.841+0000 7fbe6b400640 1 -- 172.21.15.67:0/3003758686 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe580078e0 con 0x7fbe6c0b3690 2024-05-30T16:44:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.841+0000 7fbe6b400640 1 -- 172.21.15.67:0/3003758686 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58007cb0 con 0x7fbe6c0b3690 2024-05-30T16:44:32.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.842+0000 7fbe73c00640 1 -- 172.21.15.67:0/3003758686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 msgr2=0x7fbe6c0d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.842+0000 7fbe73c00640 1 --2- 172.21.15.67:0/3003758686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 0x7fbe6c0d04a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009d90 tx=0x7fbe5802a660 comp rx=0 tx=0).stop 2024-05-30T16:44:32.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.842+0000 7fbe73c00640 1 -- 172.21.15.67:0/3003758686 shutdown_connections 2024-05-30T16:44:32.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.842+0000 7fbe73c00640 1 --2- 172.21.15.67:0/3003758686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 0x7fbe6c0d04a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.842+0000 7fbe73c00640 1 --2- 172.21.15.67:0/3003758686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.842+0000 7fbe73c00640 1 -- 172.21.15.67:0/3003758686 >> 172.21.15.67:0/3003758686 conn(0x7fbe6c06bae0 msgr2=0x7fbe6c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:32.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.842+0000 7fbe73c00640 1 -- 172.21.15.67:0/3003758686 shutdown_connections 2024-05-30T16:44:32.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.842+0000 7fbe73c00640 1 -- 172.21.15.67:0/3003758686 wait complete. 2024-05-30T16:44:32.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.845+0000 7fbe73c00640 1 Processor -- start 2024-05-30T16:44:32.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.845+0000 7fbe73c00640 1 -- start start 2024-05-30T16:44:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.846+0000 7fbe73c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.846+0000 7fbe73c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 0x7fbe6c0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.846+0000 7fbe73c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe6c0d09e0 0x7fbe6c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.846+0000 7fbe73c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c0b4560 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.846+0000 7fbe73c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c0b4280 con 0x7fbe6c0b3690 2024-05-30T16:44:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.846+0000 7fbe73c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6c0b4840 con 0x7fbe6c06f860 2024-05-30T16:44:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe71c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe6c0d09e0 0x7fbe6c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe71c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe6c0d09e0 0x7fbe6c15b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36344/0 (socket says 172.21.15.67:36344) 2024-05-30T16:44:32.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe71c00640 1 -- 172.21.15.67:0/3525039624 learned_addr learned my addr 172.21.15.67:0/3525039624 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:32.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe6be00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 0x7fbe6c0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe71200640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe71c00640 1 -- 172.21.15.67:0/3525039624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 msgr2=0x7fbe6c15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe71c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe71c00640 1 -- 172.21.15.67:0/3525039624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 msgr2=0x7fbe6c0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:32.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe71c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 0x7fbe6c0b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:32.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe71c00640 1 -- 172.21.15.67:0/3525039624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c15f4c0 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.847+0000 7fbe6be00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 0x7fbe6c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:32.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.848+0000 7fbe71c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe6c0d09e0 0x7fbe6c15b340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400c410 tx=0x7fbe6400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:32.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.848+0000 7fbe68c00640 1 -- 172.21.15.67:0/3525039624 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64017070 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.848+0000 7fbe68c00640 1 -- 172.21.15.67:0/3525039624 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe640035f0 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.848+0000 7fbe68c00640 1 -- 172.21.15.67:0/3525039624 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe640039c0 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.848+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c15f7a0 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.848+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c15da40 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.850+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6c0689e0 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.850+0000 7fbe68c00640 1 -- 172.21.15.67:0/3525039624 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe6401a070 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.851+0000 7fbe68c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe4c06c920 0x7fbe4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.851+0000 7fbe68c00640 1 -- 172.21.15.67:0/3525039624 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbe6408ae30 con 0x7fbe6c0d09e0 2024-05-30T16:44:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.852+0000 7fbe71200640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe4c06c920 0x7fbe4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:32.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.853+0000 7fbe71200640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe4c06c920 0x7fbe4c06ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009b70 tx=0x7fbe5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:32.856+0000 7fbe68c00640 1 -- 172.21.15.67:0/3525039624 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe6c06ed10 con 0x7fbe6c0d09e0 2024-05-30T16:44:33.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.110+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe6c06e2f0 con 0x7fbe6c0d09e0 2024-05-30T16:44:33.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.111+0000 7fbe68c00640 1 -- 172.21.15.67:0/3525039624 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fbe64059270 con 0x7fbe6c0d09e0 2024-05-30T16:44:33.111 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:33.112 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2254297887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2254297887},{"type":"v1","addr":"172.21.15.67:6837","nonce":2254297887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/227511509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":227511509},{"type":"v1","addr":"172.21.15.169:6834","nonce":227511509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:43:52.308425+0000","modified":"2024-05-30T16:43:55.372983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2028878495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2028878495},{"type":"v1","addr":"172.21.15.169:6835","nonce":2028878495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-30T16:44:33.112 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-05-30T16:44:33.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.113+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe4c06c920 msgr2=0x7fbe4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.113+0000 7fbe73c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe4c06c920 0x7fbe4c06ede0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009b70 tx=0x7fbe5c008040 comp rx=0 tx=0).stop 2024-05-30T16:44:33.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.113+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe6c0d09e0 msgr2=0x7fbe6c15b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.113+0000 7fbe73c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe6c0d09e0 0x7fbe6c15b340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400c410 tx=0x7fbe6400c8d0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.114+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 shutdown_connections 2024-05-30T16:44:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.114+0000 7fbe73c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe4c06c920 0x7fbe4c06ede0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.114+0000 7fbe73c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe6c0d09e0 0x7fbe6c15b340 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.114+0000 7fbe73c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe6c0b3690 0x7fbe6c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.114+0000 7fbe73c00640 1 --2- 172.21.15.67:0/3525039624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe6c06f860 0x7fbe6c15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.114+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 >> 172.21.15.67:0/3525039624 conn(0x7fbe6c06bae0 msgr2=0x7fbe6c0725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.114+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 shutdown_connections 2024-05-30T16:44:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.114+0000 7fbe73c00640 1 -- 172.21.15.67:0/3525039624 wait complete. 2024-05-30T16:44:33.132 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:44:33.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.341+0000 7f0dc9800640 1 Processor -- start 2024-05-30T16:44:33.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.341+0000 7f0dc9800640 1 -- start start 2024-05-30T16:44:33.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.343+0000 7f0dc9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dc4072070 0x7f0dc40b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.343+0000 7f0dc9800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc406e6e0 con 0x7f0dc4071bc0 2024-05-30T16:44:33.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.343+0000 7f0dc9800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc406e9c0 con 0x7f0dc4158b80 2024-05-30T16:44:33.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.343+0000 7f0dc9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc406eca0 con 0x7f0dc4072070 2024-05-30T16:44:33.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.343+0000 7f0dc3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0dc4071bc0 0x7f0dc40b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49744/0 (socket says 172.21.15.67:49744) 2024-05-30T16:44:33.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.343+0000 7f0dc3400640 1 -- 172.21.15.67:0/1187652747 learned_addr learned my addr 172.21.15.67:0/1187652747 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:33.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.343+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329507385 0 0) 0x7f0dc406e6e0 con 0x7f0dc4071bc0 2024-05-30T16:44:33.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.343+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dc4072070 0x7f0dc40b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0da8003520 con 0x7f0dc4071bc0 2024-05-30T16:44:33.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1199387858 0 0) 0x7f0da8003520 con 0x7f0dc4071bc0 2024-05-30T16:44:33.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dc4072070 msgr2=0x7f0dc40b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:33.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dc4072070 0x7f0dc40b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 >> v1:172.21.15.169:6789/0 conn(0x7f0dc4158b80 legacy=0x7f0dc406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc406ef80 con 0x7f0dc4071bc0 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f0db0001670 con 0x7f0dc4071bc0 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 >> v1:172.21.15.67:6789/0 conn(0x7f0dc4071bc0 legacy=0x7f0dc40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da80039e0 0x7f0da8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dc4072070 0x7f0da8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da8004d40 0x7f0da80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc406eca0 con 0x7f0da8004d40 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0da800baa0 con 0x7f0da80039e0 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0da800bca0 con 0x7f0dc4072070 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.344+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da8004d40 0x7f0da80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.345+0000 7f0dc3e00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da80039e0 0x7f0da8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.345+0000 7f0dc3400640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dc4072070 0x7f0da8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.345+0000 7f0dc8e00640 1 -- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dc4072070 msgr2=0x7f0da8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.345+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dc4072070 0x7f0da8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.345+0000 7f0dc8e00640 1 -- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da80039e0 msgr2=0x7f0da8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.345+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0da80039e0 0x7f0da8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.345+0000 7f0dc8e00640 1 -- 172.21.15.67:0/1187652747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da8003520 con 0x7f0da8004d40 2024-05-30T16:44:33.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.345+0000 7f0dc3400640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0dc4072070 0x7f0da8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.346+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da8004d40 0x7f0da80051c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009f90 tx=0x7f0da8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3387c4255e9f608 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.346+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc009c10 con 0x7f0da8004d40 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.346+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc036070 con 0x7f0da8004d40 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.346+0000 7f0dc2a00640 1 -- 172.21.15.67:0/1187652747 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc031850 con 0x7f0da8004d40 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.347+0000 7f0dc9800640 1 -- 172.21.15.67:0/1187652747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da8004d40 msgr2=0x7f0da80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.347+0000 7f0dc9800640 1 --2- 172.21.15.67:0/1187652747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da8004d40 0x7f0da80051c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009f90 tx=0x7f0da8005830 comp rx=0 tx=0).stop 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.347+0000 7f0dc8e00640 1 -- 172.21.15.67:0/1187652747 reap_dead start 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.347+0000 7f0dc9800640 1 -- 172.21.15.67:0/1187652747 shutdown_connections 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.347+0000 7f0dc9800640 1 -- 172.21.15.67:0/1187652747 >> 172.21.15.67:0/1187652747 conn(0x7f0dc40b1fc0 msgr2=0x7f0dc40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.347+0000 7f0dc9800640 1 -- 172.21.15.67:0/1187652747 shutdown_connections 2024-05-30T16:44:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.347+0000 7f0dc9800640 1 -- 172.21.15.67:0/1187652747 wait complete. 2024-05-30T16:44:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.349+0000 7f0dc9800640 1 Processor -- start 2024-05-30T16:44:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.349+0000 7f0dc9800640 1 -- start start 2024-05-30T16:44:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.350+0000 7f0dc9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da8004d40 0x7f0dc4155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.350+0000 7f0dc9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc409a620 0x7f0dc41f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.350+0000 7f0dc9800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dc409aaa0 0x7f0dc41f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.350+0000 7f0dc9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4072480 con 0x7f0dc409a620 2024-05-30T16:44:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.350+0000 7f0dc9800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc40b2d30 con 0x7f0dc409aaa0 2024-05-30T16:44:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.350+0000 7f0dc9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc406f780 con 0x7f0da8004d40 2024-05-30T16:44:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.350+0000 7f0dc3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc409a620 0x7f0dc41f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.350+0000 7f0dc3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc409a620 0x7f0dc41f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36364/0 (socket says 172.21.15.67:36364) 2024-05-30T16:44:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.350+0000 7f0dc3400640 1 -- 172.21.15.67:0/2665226361 learned_addr learned my addr 172.21.15.67:0/2665226361 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc3e00640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da8004d40 0x7f0dc4155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dc409aaa0 0x7f0dc41f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc3400640 1 -- 172.21.15.67:0/2665226361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da8004d40 msgr2=0x7f0dc4155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc3400640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da8004d40 0x7f0dc4155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc3400640 1 -- 172.21.15.67:0/2665226361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dc409aaa0 msgr2=0x7f0dc41f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc3400640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dc409aaa0 0x7f0dc41f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc3400640 1 -- 172.21.15.67:0/2665226361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc409c7a0 con 0x7f0dc409a620 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dc409aaa0 0x7f0dc41f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc3400640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc409a620 0x7f0dc41f6b40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0db000d890 tx=0x7f0db000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dbbe00640 1 -- 172.21.15.67:0/2665226361 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0019070 con 0x7f0dc409a620 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dbbe00640 1 -- 172.21.15.67:0/2665226361 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db00074e0 con 0x7f0dc409a620 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dbbe00640 1 -- 172.21.15.67:0/2665226361 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00049e0 con 0x7f0dc409a620 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.351+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc409ca80 con 0x7f0dc409a620 2024-05-30T16:44:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.352+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc409c450 con 0x7f0dc409a620 2024-05-30T16:44:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.353+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc40689e0 con 0x7f0dc409a620 2024-05-30T16:44:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.354+0000 7f0dbbe00640 1 -- 172.21.15.67:0/2665226361 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0db001c070 con 0x7f0dc409a620 2024-05-30T16:44:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.355+0000 7f0dbbe00640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0da406c8f0 0x7f0da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.355+0000 7f0dbbe00640 1 -- 172.21.15.67:0/2665226361 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f0db008b800 con 0x7f0dc409a620 2024-05-30T16:44:33.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.355+0000 7f0dc3e00640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0da406c8f0 0x7f0da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.356+0000 7f0dc3e00640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0da406c8f0 0x7f0da406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0db4007fd0 tx=0x7f0db4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.360+0000 7f0dbbe00640 1 -- 172.21.15.67:0/2665226361 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0db0059b10 con 0x7f0dc409a620 2024-05-30T16:44:33.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.607+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dc4153ec0 con 0x7f0dc409a620 2024-05-30T16:44:33.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.608+0000 7f0dbbe00640 1 -- 172.21.15.67:0/2665226361 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f0db0059410 con 0x7f0dc409a620 2024-05-30T16:44:33.609 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:33.609 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2254297887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2254297887},{"type":"v1","addr":"172.21.15.67:6837","nonce":2254297887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3558938170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3558938170},{"type":"v1","addr":"172.21.15.67:6836","nonce":3558938170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/227511509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":227511509},{"type":"v1","addr":"172.21.15.169:6834","nonce":227511509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:43:52.308425+0000","modified":"2024-05-30T16:43:55.372983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2028878495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2028878495},{"type":"v1","addr":"172.21.15.169:6835","nonce":2028878495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-30T16:44:33.609 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-05-30T16:44:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0da406c8f0 msgr2=0x7f0da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0da406c8f0 0x7f0da406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0db4007fd0 tx=0x7f0db4009040 comp rx=0 tx=0).stop 2024-05-30T16:44:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc409a620 msgr2=0x7f0dc41f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc409a620 0x7f0dc41f6b40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0db000d890 tx=0x7f0db000dd50 comp rx=0 tx=0).stop 2024-05-30T16:44:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 shutdown_connections 2024-05-30T16:44:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0da406c8f0 0x7f0da406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dc409aaa0 0x7f0dc41f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc409a620 0x7f0dc41f6b40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 --2- 172.21.15.67:0/2665226361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0da8004d40 0x7f0dc4155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 >> 172.21.15.67:0/2665226361 conn(0x7f0dc40b1fc0 msgr2=0x7f0dc40b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 shutdown_connections 2024-05-30T16:44:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.611+0000 7f0dc9800640 1 -- 172.21.15.67:0/2665226361 wait complete. 2024-05-30T16:44:33.632 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:44:33.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.847+0000 7ff13b600640 1 Processor -- start 2024-05-30T16:44:33.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.847+0000 7ff13b600640 1 -- start start 2024-05-30T16:44:33.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.848+0000 7ff13b600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff13406f860 0x7ff1340b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.848+0000 7ff13b600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13406ed10 con 0x7ff1340d0bb0 2024-05-30T16:44:33.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.848+0000 7ff13b600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13415cb00 con 0x7ff1340b3690 2024-05-30T16:44:33.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.848+0000 7ff13b600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff1340b3cc0 con 0x7ff13406f860 2024-05-30T16:44:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.848+0000 7ff139600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff1340d0bb0 0x7ff1340ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49756/0 (socket says 172.21.15.67:49756) 2024-05-30T16:44:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.848+0000 7ff139600640 1 -- 172.21.15.67:0/2603321630 learned_addr learned my addr 172.21.15.67:0/2603321630 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff138c00640 1 --2- 172.21.15.67:0/2603321630 >> v2:172.21.15.169:3300/0 conn(0x7ff13406f860 0x7ff1340b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 75770800 0 0) 0x7ff12c001110 con 0x7ff1340d0bb0 2024-05-30T16:44:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff118003520 con 0x7ff1340d0bb0 2024-05-30T16:44:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4125876632 0 0) 0x7ff118003520 con 0x7ff1340d0bb0 2024-05-30T16:44:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 >> v2:172.21.15.169:3300/0 conn(0x7ff13406f860 msgr2=0x7ff1340b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:33.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 --2- 172.21.15.67:0/2603321630 >> v2:172.21.15.169:3300/0 conn(0x7ff13406f860 0x7ff1340b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 >> v1:172.21.15.169:6789/0 conn(0x7ff1340b3690 legacy=0x7ff1340d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1340b3fa0 con 0x7ff1340d0bb0 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff12c001840 con 0x7ff1340d0bb0 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 >> v1:172.21.15.67:6789/0 conn(0x7ff1340d0bb0 legacy=0x7ff1340ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180039b0 0x7ff118003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118004800 0x7ff118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180051c0 0x7ff11800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340b3cc0 con 0x7ff1180051c0 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff11800c060 con 0x7ff118004800 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.849+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff11800c260 con 0x7ff1180039b0 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff139600640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180051c0 0x7ff11800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff12be00640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118004800 0x7ff118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff138c00640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180039b0 0x7ff118003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff139600640 1 -- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180039b0 msgr2=0x7ff118003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:33.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff139600640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180039b0 0x7ff118003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff139600640 1 -- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118004800 msgr2=0x7ff118004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:33.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff139600640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118004800 0x7ff118004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff139600640 1 -- 172.21.15.67:0/2603321630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff118003520 con 0x7ff1180051c0 2024-05-30T16:44:33.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff138c00640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180039b0 0x7ff118003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:33.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.850+0000 7ff12be00640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff118004800 0x7ff118004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:33.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.851+0000 7ff139600640 1 -- 172.21.15.67:0/2603321630 reap_dead start 2024-05-30T16:44:33.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.851+0000 7ff139600640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180051c0 0x7ff11800b9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00c4c0 tx=0x7ff118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af9b8f555d5d742 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:33.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00c0e0 con 0x7ff1180051c0 2024-05-30T16:44:33.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c0058d0 con 0x7ff1180051c0 2024-05-30T16:44:33.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff133e00640 1 -- 172.21.15.67:0/2603321630 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c005ce0 con 0x7ff1180051c0 2024-05-30T16:44:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff13b600640 1 -- 172.21.15.67:0/2603321630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180051c0 msgr2=0x7ff11800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff13b600640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180051c0 0x7ff11800b9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00c4c0 tx=0x7ff118005830 comp rx=0 tx=0).stop 2024-05-30T16:44:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff13b600640 1 -- 172.21.15.67:0/2603321630 shutdown_connections 2024-05-30T16:44:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff13b600640 1 --2- 172.21.15.67:0/2603321630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180051c0 0x7ff11800b9c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff13b600640 1 -- 172.21.15.67:0/2603321630 >> 172.21.15.67:0/2603321630 conn(0x7ff13406bae0 msgr2=0x7ff13406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff13b600640 1 -- 172.21.15.67:0/2603321630 shutdown_connections 2024-05-30T16:44:33.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.852+0000 7ff13b600640 1 -- 172.21.15.67:0/2603321630 wait complete. 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.854+0000 7ff13b600640 1 Processor -- start 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.854+0000 7ff13b600640 1 -- start start 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.855+0000 7ff13b600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180051c0 0x7ff13415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.855+0000 7ff13b600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1341e4d80 0x7ff1341e5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.855+0000 7ff13b600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1341e6ae0 0x7ff1341e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.855+0000 7ff13b600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340b4840 con 0x7ff1341e4d80 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.855+0000 7ff13b600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340b4560 con 0x7ff1341e6ae0 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.855+0000 7ff13b600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1340b3150 con 0x7ff1180051c0 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff12be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1341e4d80 0x7ff1341e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff12be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1341e4d80 0x7ff1341e5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36380/0 (socket says 172.21.15.67:36380) 2024-05-30T16:44:33.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff12be00640 1 -- 172.21.15.67:0/1555233055 learned_addr learned my addr 172.21.15.67:0/1555233055 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:33.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff138c00640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180051c0 0x7ff13415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff139600640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1341e6ae0 0x7ff1341e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff12be00640 1 -- 172.21.15.67:0/1555233055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180051c0 msgr2=0x7ff13415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff12be00640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180051c0 0x7ff13415f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff12be00640 1 -- 172.21.15.67:0/1555233055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1341e6ae0 msgr2=0x7ff1341e6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:33.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff12be00640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1341e6ae0 0x7ff1341e6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:33.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff12be00640 1 -- 172.21.15.67:0/1555233055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1341e5840 con 0x7ff1341e4d80 2024-05-30T16:44:33.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.856+0000 7ff12be00640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1341e4d80 0x7ff1341e5200 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff11c000c00 tx=0x7ff11c00e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:33.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.857+0000 7ff131600640 1 -- 172.21.15.67:0/1555233055 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c019070 con 0x7ff1341e4d80 2024-05-30T16:44:33.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.857+0000 7ff131600640 1 -- 172.21.15.67:0/1555233055 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c0052e0 con 0x7ff1341e4d80 2024-05-30T16:44:33.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.857+0000 7ff131600640 1 -- 172.21.15.67:0/1555233055 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c003830 con 0x7ff1341e4d80 2024-05-30T16:44:33.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.857+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1341e5b20 con 0x7ff1341e4d80 2024-05-30T16:44:33.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.857+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1341e60e0 con 0x7ff1341e4d80 2024-05-30T16:44:33.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.858+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1340689e0 con 0x7ff1341e4d80 2024-05-30T16:44:33.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.859+0000 7ff131600640 1 -- 172.21.15.67:0/1555233055 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff11c01c070 con 0x7ff1341e4d80 2024-05-30T16:44:33.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.860+0000 7ff131600640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff11006c920 0x7ff11006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:33.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.860+0000 7ff131600640 1 -- 172.21.15.67:0/1555233055 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7ff11c08b110 con 0x7ff1341e4d80 2024-05-30T16:44:33.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.860+0000 7ff138c00640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff11006c920 0x7ff11006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:33.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.861+0000 7ff138c00640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff11006c920 0x7ff11006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff124007440 tx=0x7ff12400d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:33.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:33.864+0000 7ff131600640 1 -- 172.21.15.67:0/1555233055 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff11c058e70 con 0x7ff1341e4d80 2024-05-30T16:44:34.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.069+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff13406e2f0 con 0x7ff1341e4d80 2024-05-30T16:44:34.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.072+0000 7ff131600640 1 -- 172.21.15.67:0/1555233055 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16843 (secure 0 0 0) 0x7ff11c05d170 con 0x7ff1341e4d80 2024-05-30T16:44:34.073 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:34.073 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:44:33.196881+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:43:50.496509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-30T16:43:51.276872+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:44:34.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff11006c920 msgr2=0x7ff11006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff11006c920 0x7ff11006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff124007440 tx=0x7ff12400d040 comp rx=0 tx=0).stop 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1341e4d80 msgr2=0x7ff1341e5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1341e4d80 0x7ff1341e5200 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff11c000c00 tx=0x7ff11c00e8f0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 shutdown_connections 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff11006c920 0x7ff11006ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1341e6ae0 0x7ff1341e6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1341e4d80 0x7ff1341e5200 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 --2- 172.21.15.67:0/1555233055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180051c0 0x7ff13415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 >> 172.21.15.67:0/1555233055 conn(0x7ff13406bae0 msgr2=0x7ff1340ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 shutdown_connections 2024-05-30T16:44:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.075+0000 7ff13b600640 1 -- 172.21.15.67:0/1555233055 wait complete. 2024-05-30T16:44:34.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-30T16:44:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.302+0000 7f5d09800640 1 Processor -- start 2024-05-30T16:44:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.303+0000 7f5d09800640 1 -- start start 2024-05-30T16:44:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.304+0000 7f5d09800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5d0415e870 0x7f5d040d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.304+0000 7f5d09800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d040cfa40 0x7f5d040d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.304+0000 7f5d09800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d0415ec50 con 0x7f5d040d14d0 2024-05-30T16:44:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.304+0000 7f5d09800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5d040d2150 con 0x7f5d0415e870 2024-05-30T16:44:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.304+0000 7f5d09800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d040cff80 con 0x7f5d040cfa40 2024-05-30T16:44:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.304+0000 7f5d03e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5d0415e870 0x7f5d040d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.304+0000 7f5d03400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5d040d14d0 0x7f5d040cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49764/0 (socket says 172.21.15.67:49764) 2024-05-30T16:44:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.304+0000 7f5d03400640 1 -- 172.21.15.67:0/4176661623 learned_addr learned my addr 172.21.15.67:0/4176661623 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:34.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.304+0000 7f5d03e00640 1 --2- 172.21.15.67:0/4176661623 >> v2:172.21.15.67:3300/0 conn(0x7f5d0415e870 0x7f5d040d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36394/0 (socket says 172.21.15.67:36394) 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.305+0000 7f5d08e00640 1 --2- 172.21.15.67:0/4176661623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d040cfa40 0x7f5d040d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.305+0000 7f5d03e00640 1 -- 172.21.15.67:0/4176661623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d040cfa40 msgr2=0x7f5d040d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.305+0000 7f5d03e00640 1 --2- 172.21.15.67:0/4176661623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d040cfa40 0x7f5d040d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.305+0000 7f5d03e00640 1 -- 172.21.15.67:0/4176661623 >> v1:172.21.15.67:6789/0 conn(0x7f5d040d14d0 legacy=0x7f5d040cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.305+0000 7f5d03e00640 1 -- 172.21.15.67:0/4176661623 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d040d0260 con 0x7f5d0415e870 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.305+0000 7f5d02a00640 1 -- 172.21.15.67:0/4176661623 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2461464838 0 0) 0x7f5d0415ec50 con 0x7f5d040d14d0 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.306+0000 7f5d03e00640 1 --2- 172.21.15.67:0/4176661623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0415e870 0x7f5d040d0f90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40097a0 tx=0x7f5cf402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c4d51ea1f84946 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.306+0000 7f5d02a00640 1 -- 172.21.15.67:0/4176661623 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4037070 con 0x7f5d0415e870 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.306+0000 7f5d02a00640 1 -- 172.21.15.67:0/4176661623 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf4032440 con 0x7f5d0415e870 2024-05-30T16:44:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.306+0000 7f5d02a00640 1 -- 172.21.15.67:0/4176661623 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4032810 con 0x7f5d0415e870 2024-05-30T16:44:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.307+0000 7f5d09800640 1 -- 172.21.15.67:0/4176661623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0415e870 msgr2=0x7f5d040d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.307+0000 7f5d09800640 1 --2- 172.21.15.67:0/4176661623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0415e870 0x7f5d040d0f90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40097a0 tx=0x7f5cf402a180 comp rx=0 tx=0).stop 2024-05-30T16:44:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.307+0000 7f5d09800640 1 -- 172.21.15.67:0/4176661623 shutdown_connections 2024-05-30T16:44:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.307+0000 7f5d09800640 1 --2- 172.21.15.67:0/4176661623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d040cfa40 0x7f5d040d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.307+0000 7f5d09800640 1 --2- 172.21.15.67:0/4176661623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0415e870 0x7f5d040d0f90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.307+0000 7f5d09800640 1 -- 172.21.15.67:0/4176661623 >> 172.21.15.67:0/4176661623 conn(0x7f5d0408bfc0 msgr2=0x7f5d040d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:34.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.307+0000 7f5d09800640 1 -- 172.21.15.67:0/4176661623 shutdown_connections 2024-05-30T16:44:34.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.307+0000 7f5d09800640 1 -- 172.21.15.67:0/4176661623 wait complete. 2024-05-30T16:44:34.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.309+0000 7f5d09800640 1 Processor -- start 2024-05-30T16:44:34.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.309+0000 7f5d09800640 1 -- start start 2024-05-30T16:44:34.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.310+0000 7f5d09800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d040cfa40 0x7f5d040b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.310+0000 7f5d09800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d040d14d0 0x7f5d04159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.310+0000 7f5d09800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d0415e870 0x7f5d0415ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.310+0000 7f5d09800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d040d4fb0 con 0x7f5d040d14d0 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.310+0000 7f5d09800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d040d0820 con 0x7f5d040cfa40 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.310+0000 7f5d09800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d040d2b20 con 0x7f5d0415e870 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.310+0000 7f5d03400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d040d14d0 0x7f5d04159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.310+0000 7f5d03400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d040d14d0 0x7f5d04159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36408/0 (socket says 172.21.15.67:36408) 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.310+0000 7f5d03400640 1 -- 172.21.15.67:0/4223802309 learned_addr learned my addr 172.21.15.67:0/4223802309 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d03e00640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d040cfa40 0x7f5d040b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d08e00640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d0415e870 0x7f5d0415ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d03400640 1 -- 172.21.15.67:0/4223802309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d0415e870 msgr2=0x7f5d0415ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d03400640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d0415e870 0x7f5d0415ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d03400640 1 -- 172.21.15.67:0/4223802309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d040cfa40 msgr2=0x7f5d040b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d03400640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d040cfa40 0x7f5d040b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d03400640 1 -- 172.21.15.67:0/4223802309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0415fd10 con 0x7f5d040d14d0 2024-05-30T16:44:34.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d03e00640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d040cfa40 0x7f5d040b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:34.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d03400640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d040d14d0 0x7f5d04159b70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c4d0 tx=0x7f5cf000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:34.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5cfbe00640 1 -- 172.21.15.67:0/4223802309 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0007480 con 0x7f5d040d14d0 2024-05-30T16:44:34.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5cfbe00640 1 -- 172.21.15.67:0/4223802309 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf00057a0 con 0x7f5d040d14d0 2024-05-30T16:44:34.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5cfbe00640 1 -- 172.21.15.67:0/4223802309 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0007620 con 0x7f5d040d14d0 2024-05-30T16:44:34.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.311+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0415fff0 con 0x7f5d040d14d0 2024-05-30T16:44:34.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.312+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d040b76d0 con 0x7f5d040d14d0 2024-05-30T16:44:34.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.313+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d040b78d0 con 0x7f5d040d14d0 2024-05-30T16:44:34.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.314+0000 7f5cfbe00640 1 -- 172.21.15.67:0/4223802309 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5cf0005b50 con 0x7f5d040d14d0 2024-05-30T16:44:34.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.315+0000 7f5cfbe00640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5ce406c8f0 0x7f5ce406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.315+0000 7f5cfbe00640 1 -- 172.21.15.67:0/4223802309 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f5cf008b870 con 0x7f5d040d14d0 2024-05-30T16:44:34.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.315+0000 7f5d03e00640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5ce406c8f0 0x7f5ce406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.316+0000 7f5d03e00640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5ce406c8f0 0x7f5ce406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4034000 tx=0x7f5cf402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.319+0000 7f5cfbe00640 1 -- 172.21.15.67:0/4223802309 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d040b78d0 con 0x7f5d040d14d0 2024-05-30T16:44:34.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.556+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5d040689e0 con 0x7f5d040d14d0 2024-05-30T16:44:34.558 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-05-30T16:44:34.557+0000 7f4982000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-30T16:44:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.560+0000 7f5cfbe00640 1 -- 172.21.15.67:0/4223802309 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f5cf00182b0 con 0x7f5d040d14d0 2024-05-30T16:44:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-30T16:44:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5ce406c8f0 msgr2=0x7f5ce406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5ce406c8f0 0x7f5ce406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4034000 tx=0x7f5cf402e000 comp rx=0 tx=0).stop 2024-05-30T16:44:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d040d14d0 msgr2=0x7f5d04159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d040d14d0 0x7f5d04159b70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c4d0 tx=0x7f5cf000c990 comp rx=0 tx=0).stop 2024-05-30T16:44:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 shutdown_connections 2024-05-30T16:44:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5ce406c8f0 0x7f5ce406edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d0415e870 0x7f5d0415ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d040d14d0 0x7f5d04159b70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 --2- 172.21.15.67:0/4223802309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d040cfa40 0x7f5d040b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 >> 172.21.15.67:0/4223802309 conn(0x7f5d0408bfc0 msgr2=0x7f5d040b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:34.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 shutdown_connections 2024-05-30T16:44:34.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.563+0000 7f5d09800640 1 -- 172.21.15.67:0/4223802309 wait complete. 2024-05-30T16:44:34.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-30T16:44:34.587 INFO:tasks.ceph.mds.b.smithi169.stderr: -2618> 2024-05-30T16:43:57.792+0000 7ffb1d000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T16:44:34.666 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T16:44:34.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.790+0000 7f9eff200640 1 Processor -- start 2024-05-30T16:44:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.791+0000 7f9eff200640 1 -- start start 2024-05-30T16:44:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.792+0000 7f9eff200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9f000cfa40 0x7f9f00169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.792+0000 7f9eff200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f0015ec50 con 0x7f9f0015e870 2024-05-30T16:44:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.792+0000 7f9eff200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f000cff80 con 0x7f9f000d16a0 2024-05-30T16:44:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.792+0000 7f9eff200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9f000d2150 con 0x7f9f000cfa40 2024-05-30T16:44:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.792+0000 7f9efde00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9f0015e870 0x7f9f000d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49768/0 (socket says 172.21.15.67:49768) 2024-05-30T16:44:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.792+0000 7f9efde00640 1 -- 172.21.15.67:0/145873404 learned_addr learned my addr 172.21.15.67:0/145873404 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 907676683 0 0) 0x7f9f0015ec50 con 0x7f9f0015e870 2024-05-30T16:44:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ee4003520 con 0x7f9f0015e870 2024-05-30T16:44:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efe800640 1 --2- 172.21.15.67:0/145873404 >> v2:172.21.15.169:3300/0 conn(0x7f9f000cfa40 0x7f9f00169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 322631945 0 0) 0x7f9ee4003520 con 0x7f9f0015e870 2024-05-30T16:44:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 >> v2:172.21.15.169:3300/0 conn(0x7f9f000cfa40 msgr2=0x7f9f00169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 --2- 172.21.15.67:0/145873404 >> v2:172.21.15.169:3300/0 conn(0x7f9f000cfa40 0x7f9f00169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 >> v1:172.21.15.169:6789/0 conn(0x7f9f000d16a0 legacy=0x7f9f000cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f000d0260 con 0x7f9f0015e870 2024-05-30T16:44:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9ef0001670 con 0x7f9f0015e870 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 >> v1:172.21.15.67:6789/0 conn(0x7f9f0015e870 legacy=0x7f9f000d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 --2- 172.21.15.67:0/145873404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee40039b0 0x7f9ee4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 --2- 172.21.15.67:0/145873404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee4004800 0x7f9ee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 --2- 172.21.15.67:0/145873404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ee40051c0 0x7f9ee400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9f000d2150 con 0x7f9ee40039b0 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee400c110 con 0x7f9ee4004800 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee400c310 con 0x7f9ee40051c0 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.793+0000 7f9efde00640 1 --2- 172.21.15.67:0/145873404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee40039b0 0x7f9ee4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.794+0000 7f9efde00640 1 -- 172.21.15.67:0/145873404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ee40051c0 msgr2=0x7f9ee400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.794+0000 7f9efde00640 1 --2- 172.21.15.67:0/145873404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ee40051c0 0x7f9ee400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.794+0000 7f9efde00640 1 -- 172.21.15.67:0/145873404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee4004800 msgr2=0x7f9ee4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.794+0000 7f9efde00640 1 --2- 172.21.15.67:0/145873404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ee4004800 0x7f9ee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.794+0000 7f9efde00640 1 -- 172.21.15.67:0/145873404 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee4003520 con 0x7f9ee40039b0 2024-05-30T16:44:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.794+0000 7f9efe800640 1 -- 172.21.15.67:0/145873404 reap_dead start 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.795+0000 7f9efde00640 1 --2- 172.21.15.67:0/145873404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee40039b0 0x7f9ee4003dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000c210 tx=0x7f9ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b053ee89ed106e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.795+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0007d30 con 0x7f9ee40039b0 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.795+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef0005ad0 con 0x7f9ee40039b0 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.795+0000 7f9efca00640 1 -- 172.21.15.67:0/145873404 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0007070 con 0x7f9ee40039b0 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.796+0000 7f9eff200640 1 -- 172.21.15.67:0/145873404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee40039b0 msgr2=0x7f9ee4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.796+0000 7f9eff200640 1 --2- 172.21.15.67:0/145873404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee40039b0 0x7f9ee4003dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000c210 tx=0x7f9ee4005830 comp rx=0 tx=0).stop 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.796+0000 7f9eff200640 1 -- 172.21.15.67:0/145873404 shutdown_connections 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.796+0000 7f9eff200640 1 --2- 172.21.15.67:0/145873404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee40039b0 0x7f9ee4003dd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.796+0000 7f9eff200640 1 -- 172.21.15.67:0/145873404 >> 172.21.15.67:0/145873404 conn(0x7f9f0008bfc0 msgr2=0x7f9f000d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.796+0000 7f9eff200640 1 -- 172.21.15.67:0/145873404 shutdown_connections 2024-05-30T16:44:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.796+0000 7f9eff200640 1 -- 172.21.15.67:0/145873404 wait complete. 2024-05-30T16:44:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.798+0000 7f9eff200640 1 Processor -- start 2024-05-30T16:44:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.798+0000 7f9eff200640 1 -- start start 2024-05-30T16:44:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.799+0000 7f9eff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ee40039b0 0x7f9f0015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.799+0000 7f9eff200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f000ce2b0 0x7f9f001f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.799+0000 7f9eff200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f000b9280 0x7f9f000b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.799+0000 7f9eff200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9f000d4fb0 con 0x7f9f000ce2b0 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.799+0000 7f9eff200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f000d0820 con 0x7f9f000b9280 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.799+0000 7f9eff200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9f000cf330 con 0x7f9ee40039b0 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.799+0000 7f9efd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f000ce2b0 0x7f9f001f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.799+0000 7f9efd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f000ce2b0 0x7f9f001f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36426/0 (socket says 172.21.15.67:36426) 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.799+0000 7f9efd400640 1 -- 172.21.15.67:0/4118649458 learned_addr learned my addr 172.21.15.67:0/4118649458 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9efde00640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ee40039b0 0x7f9f0015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9efe800640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f000b9280 0x7f9f000b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9efd400640 1 -- 172.21.15.67:0/4118649458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ee40039b0 msgr2=0x7f9f0015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9efd400640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ee40039b0 0x7f9f0015b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9efd400640 1 -- 172.21.15.67:0/4118649458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f000b9280 msgr2=0x7f9f000b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9efd400640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f000b9280 0x7f9f000b9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9efd400640 1 -- 172.21.15.67:0/4118649458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0015d7a0 con 0x7f9f000ce2b0 2024-05-30T16:44:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9efd400640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f000ce2b0 0x7f9f001f3a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00c390 tx=0x7f9eec00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9ef6000640 1 -- 172.21.15.67:0/4118649458 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0052d0 con 0x7f9f000ce2b0 2024-05-30T16:44:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9ef6000640 1 -- 172.21.15.67:0/4118649458 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec005470 con 0x7f9f000ce2b0 2024-05-30T16:44:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0015bd50 con 0x7f9f000ce2b0 2024-05-30T16:44:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9ef6000640 1 -- 172.21.15.67:0/4118649458 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec006a60 con 0x7f9f000ce2b0 2024-05-30T16:44:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.800+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0015c310 con 0x7f9f000ce2b0 2024-05-30T16:44:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.802+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f000689e0 con 0x7f9f000ce2b0 2024-05-30T16:44:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.802+0000 7f9ef6000640 1 -- 172.21.15.67:0/4118649458 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9eec003640 con 0x7f9f000ce2b0 2024-05-30T16:44:34.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.804+0000 7f9ef6000640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ee006c920 0x7f9ee006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:34.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.805+0000 7f9ef6000640 1 -- 172.21.15.67:0/4118649458 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5911+0+0 (secure 0 0 0) 0x7f9eec08b1b0 con 0x7f9f000ce2b0 2024-05-30T16:44:34.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.805+0000 7f9efde00640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ee006c920 0x7f9ee006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:34.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.805+0000 7f9efde00640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ee006c920 0x7f9ee006ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0033000 tx=0x7f9ef002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:34.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:34.809+0000 7f9ef6000640 1 -- 172.21.15.67:0/4118649458 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f0015d7a0 con 0x7f9f000ce2b0 2024-05-30T16:44:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.056+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9f000ba500 con 0x7f9f000ce2b0 2024-05-30T16:44:35.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.564+0000 7f9ef6000640 1 -- 172.21.15.67:0/4118649458 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f9eec059470 con 0x7f9f000ce2b0 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ee006c920 msgr2=0x7f9ee006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ee006c920 0x7f9ee006ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0033000 tx=0x7f9ef002e000 comp rx=0 tx=0).stop 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f000ce2b0 msgr2=0x7f9f001f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f000ce2b0 0x7f9f001f3a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00c390 tx=0x7f9eec00c850 comp rx=0 tx=0).stop 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 shutdown_connections 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ee006c920 0x7f9ee006ede0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f000b9280 0x7f9f000b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f000ce2b0 0x7f9f001f3a50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 --2- 172.21.15.67:0/4118649458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ee40039b0 0x7f9f0015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:35.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 >> 172.21.15.67:0/4118649458 conn(0x7f9f0008bfc0 msgr2=0x7f9f000b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:35.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 shutdown_connections 2024-05-30T16:44:35.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.568+0000 7f9eff200640 1 -- 172.21.15.67:0/4118649458 wait complete. 2024-05-30T16:44:35.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-30T16:44:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.818+0000 7fc3d2200640 1 Processor -- start 2024-05-30T16:44:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.819+0000 7fc3d2200640 1 -- start start 2024-05-30T16:44:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.820+0000 7fc3d2200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3cc15e830 0x7fc3cc06dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.820+0000 7fc3d2200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3cc06e190 con 0x7fc3cc072310 2024-05-30T16:44:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.820+0000 7fc3d2200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3cc06e470 con 0x7fc3cc0b3510 2024-05-30T16:44:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.820+0000 7fc3d2200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3cc06e750 con 0x7fc3cc15e830 2024-05-30T16:44:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.821+0000 7fc3cbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc3cc072310 0x7fc3cc0ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49782/0 (socket says 172.21.15.67:49782) 2024-05-30T16:44:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.821+0000 7fc3cbe00640 1 -- 172.21.15.67:0/3537711172 learned_addr learned my addr 172.21.15.67:0/3537711172 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.821+0000 7fc3cb400640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3cc15e830 0x7fc3cc06dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.821+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3689599666 0 0) 0x7fc3cc06e190 con 0x7fc3cc072310 2024-05-30T16:44:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.821+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3b0003520 con 0x7fc3cc072310 2024-05-30T16:44:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 385169231 0 0) 0x7fc3b0003520 con 0x7fc3cc072310 2024-05-30T16:44:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3cc15e830 msgr2=0x7fc3cc06dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3cc15e830 0x7fc3cc06dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 >> v1:172.21.15.169:6789/0 conn(0x7fc3cc0b3510 legacy=0x7fc3cc071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc06ea30 con 0x7fc3cc072310 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fc3c40019f0 con 0x7fc3cc072310 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 >> v1:172.21.15.67:6789/0 conn(0x7fc3cc072310 legacy=0x7fc3cc0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b00039e0 0x7fc3b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3cc15e830 0x7fc3b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b0004d40 0x7fc3b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc06e750 con 0x7fc3b0004d40 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b000ba70 con 0x7fc3b00039e0 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b000bc70 con 0x7fc3cc15e830 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.822+0000 7fc3cbe00640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b0004d40 0x7fc3b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.823+0000 7fc3cb400640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b00039e0 0x7fc3b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.823+0000 7fc3c3e00640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3cc15e830 0x7fc3b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.823+0000 7fc3cbe00640 1 -- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3cc15e830 msgr2=0x7fc3b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.823+0000 7fc3cbe00640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3cc15e830 0x7fc3b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.823+0000 7fc3cbe00640 1 -- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b00039e0 msgr2=0x7fc3b0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:35.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.823+0000 7fc3cbe00640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b00039e0 0x7fc3b0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:35.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.823+0000 7fc3cbe00640 1 -- 172.21.15.67:0/3537711172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b0003520 con 0x7fc3b0004d40 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.824+0000 7fc3cbe00640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b0004d40 0x7fc3b00051c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc163540 tx=0x7fc3b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3897eb63195e70f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.824+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400a460 con 0x7fc3b0004d40 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.824+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c402f070 con 0x7fc3b0004d40 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.824+0000 7fc3caa00640 1 -- 172.21.15.67:0/3537711172 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c402bc50 con 0x7fc3b0004d40 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.825+0000 7fc3d2200640 1 -- 172.21.15.67:0/3537711172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b0004d40 msgr2=0x7fc3b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.825+0000 7fc3d2200640 1 --2- 172.21.15.67:0/3537711172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b0004d40 0x7fc3b00051c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc163540 tx=0x7fc3b0005830 comp rx=0 tx=0).stop 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.825+0000 7fc3cbe00640 1 -- 172.21.15.67:0/3537711172 reap_dead start 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.825+0000 7fc3d2200640 1 -- 172.21.15.67:0/3537711172 shutdown_connections 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.825+0000 7fc3d2200640 1 -- 172.21.15.67:0/3537711172 >> 172.21.15.67:0/3537711172 conn(0x7fc3cc0b1e50 msgr2=0x7fc3cc0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.825+0000 7fc3d2200640 1 -- 172.21.15.67:0/3537711172 shutdown_connections 2024-05-30T16:44:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.825+0000 7fc3d2200640 1 -- 172.21.15.67:0/3537711172 wait complete. 2024-05-30T16:44:35.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.827+0000 7fc3d2200640 1 Processor -- start 2024-05-30T16:44:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.827+0000 7fc3d2200640 1 -- start start 2024-05-30T16:44:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.828+0000 7fc3d2200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b0004d40 0x7fc3cc09cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.828+0000 7fc3d2200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3cc09db80 0x7fc3cc09b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.828+0000 7fc3d2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3cc1456a0 0x7fc3cc160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.828+0000 7fc3d2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc0728c0 con 0x7fc3cc1456a0 2024-05-30T16:44:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.828+0000 7fc3d2200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc15ec10 con 0x7fc3cc09db80 2024-05-30T16:44:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.828+0000 7fc3d2200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3cc072a40 con 0x7fc3b0004d40 2024-05-30T16:44:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.828+0000 7fc3cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3cc1456a0 0x7fc3cc160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.828+0000 7fc3cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3cc1456a0 0x7fc3cc160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36444/0 (socket says 172.21.15.67:36444) 2024-05-30T16:44:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.828+0000 7fc3cbe00640 1 -- 172.21.15.67:0/3474518557 learned_addr learned my addr 172.21.15.67:0/3474518557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3cb400640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b0004d40 0x7fc3cc09cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3c3e00640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3cc09db80 0x7fc3cc09b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3cbe00640 1 -- 172.21.15.67:0/3474518557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b0004d40 msgr2=0x7fc3cc09cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3cbe00640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b0004d40 0x7fc3cc09cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3cbe00640 1 -- 172.21.15.67:0/3474518557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3cc09db80 msgr2=0x7fc3cc09b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3cbe00640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3cc09db80 0x7fc3cc09b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3cbe00640 1 -- 172.21.15.67:0/3474518557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc09ba40 con 0x7fc3cc1456a0 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3cbe00640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3cc1456a0 0x7fc3cc160ff0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400a8d0 tx=0x7fc3cc06ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3c3400640 1 -- 172.21.15.67:0/3474518557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4044070 con 0x7fc3cc1456a0 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3c3400640 1 -- 172.21.15.67:0/3474518557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c4034470 con 0x7fc3cc1456a0 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.829+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc09bcc0 con 0x7fc3cc1456a0 2024-05-30T16:44:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.830+0000 7fc3c3400640 1 -- 172.21.15.67:0/3474518557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400a460 con 0x7fc3cc1456a0 2024-05-30T16:44:35.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.830+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc09c280 con 0x7fc3cc1456a0 2024-05-30T16:44:35.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.831+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3cc0689e0 con 0x7fc3cc1456a0 2024-05-30T16:44:35.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.832+0000 7fc3c3400640 1 -- 172.21.15.67:0/3474518557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3c4047070 con 0x7fc3cc1456a0 2024-05-30T16:44:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.833+0000 7fc3c3400640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc3a806c8f0 0x7fc3a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.833+0000 7fc3c3400640 1 -- 172.21.15.67:0/3474518557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5911+0+0 (secure 0 0 0) 0x7fc3c40abdb0 con 0x7fc3cc1456a0 2024-05-30T16:44:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.833+0000 7fc3cb400640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc3a806c8f0 0x7fc3a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.834+0000 7fc3cb400640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc3a806c8f0 0x7fc3a806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009950 tx=0x7fc3bc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:35.838+0000 7fc3c3400640 1 -- 172.21.15.67:0/3474518557 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3c40b0070 con 0x7fc3cc1456a0 2024-05-30T16:44:36.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.040+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc3cc0b1410 con 0x7fc3cc1456a0 2024-05-30T16:44:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.568+0000 7fc3c3400640 1 -- 172.21.15.67:0/3474518557 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7fc3cc09ba40 con 0x7fc3cc1456a0 2024-05-30T16:44:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-05-30T16:44:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.572+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc3a806c8f0 msgr2=0x7fc3a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.572+0000 7fc3d2200640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc3a806c8f0 0x7fc3a806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009950 tx=0x7fc3bc008040 comp rx=0 tx=0).stop 2024-05-30T16:44:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.573+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3cc1456a0 msgr2=0x7fc3cc160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.573+0000 7fc3d2200640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3cc1456a0 0x7fc3cc160ff0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400a8d0 tx=0x7fc3cc06ec00 comp rx=0 tx=0).stop 2024-05-30T16:44:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.573+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 shutdown_connections 2024-05-30T16:44:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.573+0000 7fc3d2200640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc3a806c8f0 0x7fc3a806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.573+0000 7fc3d2200640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3cc1456a0 0x7fc3cc160ff0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.573+0000 7fc3d2200640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3cc09db80 0x7fc3cc09b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.573+0000 7fc3d2200640 1 --2- 172.21.15.67:0/3474518557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3b0004d40 0x7fc3cc09cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.573+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 >> 172.21.15.67:0/3474518557 conn(0x7fc3cc0b1e50 msgr2=0x7fc3cc0b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.574+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 shutdown_connections 2024-05-30T16:44:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.574+0000 7fc3d2200640 1 -- 172.21.15.67:0/3474518557 wait complete. 2024-05-30T16:44:36.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-30T16:44:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.814+0000 7f776e400640 1 Processor -- start 2024-05-30T16:44:36.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.814+0000 7f776e400640 1 -- start start 2024-05-30T16:44:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.815+0000 7f776e400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f776809b7b0 0x7f77680d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.815+0000 7f776e400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776809bc40 con 0x7f776809aa30 2024-05-30T16:44:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.815+0000 7f776e400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776815ef20 con 0x7f776806cc20 2024-05-30T16:44:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.815+0000 7f776e400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f77680b1bf0 con 0x7f776809b7b0 2024-05-30T16:44:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.815+0000 7f776d000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f776809aa30 0x7f776809ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49786/0 (socket says 172.21.15.67:49786) 2024-05-30T16:44:36.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.816+0000 7f776d000640 1 -- 172.21.15.67:0/1942977428 learned_addr learned my addr 172.21.15.67:0/1942977428 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.816+0000 7f776da00640 1 --2- 172.21.15.67:0/1942977428 >> v2:172.21.15.169:3300/0 conn(0x7f776809b7b0 0x7f77680d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.816+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2303068428 0 0) 0x7f776809bc40 con 0x7f776809aa30 2024-05-30T16:44:36.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.816+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f774c003520 con 0x7f776809aa30 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.816+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2323659253 0 0) 0x7f774c003520 con 0x7f776809aa30 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.816+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 >> v2:172.21.15.169:3300/0 conn(0x7f776809b7b0 msgr2=0x7f77680d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.816+0000 7f7767400640 1 --2- 172.21.15.67:0/1942977428 >> v2:172.21.15.169:3300/0 conn(0x7f776809b7b0 0x7f77680d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.816+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 >> v1:172.21.15.169:6789/0 conn(0x7f776806cc20 legacy=0x7f77680d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.816+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776815f200 con 0x7f776809aa30 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f77580019f0 con 0x7f776809aa30 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 >> v1:172.21.15.67:6789/0 conn(0x7f776809aa30 legacy=0x7f776809ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f7767400640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f774c0039b0 0x7f774c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f7767400640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f774c004800 0x7f774c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f7767400640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f774c0051c0 0x7f774c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f77680b1bf0 con 0x7f774c0051c0 2024-05-30T16:44:36.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f774c00c110 con 0x7f774c0039b0 2024-05-30T16:44:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f774c00c310 con 0x7f774c004800 2024-05-30T16:44:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f776da00640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f774c0051c0 0x7f774c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f776d000640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f774c0039b0 0x7f774c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.817+0000 7f7767e00640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f774c004800 0x7f774c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.818+0000 7f776da00640 1 -- 172.21.15.67:0/1942977428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f774c004800 msgr2=0x7f774c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.818+0000 7f776da00640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f774c004800 0x7f774c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.818+0000 7f776da00640 1 -- 172.21.15.67:0/1942977428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f774c0039b0 msgr2=0x7f774c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.818+0000 7f776da00640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f774c0039b0 0x7f774c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.818+0000 7f776da00640 1 -- 172.21.15.67:0/1942977428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c003520 con 0x7f774c0051c0 2024-05-30T16:44:36.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.818+0000 7f776da00640 1 -- 172.21.15.67:0/1942977428 reap_dead start 2024-05-30T16:44:36.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.819+0000 7f776da00640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f774c0051c0 0x7f774c00b9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f776000ba30 tx=0x7f774c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a1a056ddc7a9c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:36.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.819+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760030070 con 0x7f774c0051c0 2024-05-30T16:44:36.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.819+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776002ae10 con 0x7f774c0051c0 2024-05-30T16:44:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.819+0000 7f7767400640 1 -- 172.21.15.67:0/1942977428 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776000b600 con 0x7f774c0051c0 2024-05-30T16:44:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.819+0000 7f776e400640 1 -- 172.21.15.67:0/1942977428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f774c0051c0 msgr2=0x7f774c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.819+0000 7f776e400640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f774c0051c0 0x7f774c00b9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f776000ba30 tx=0x7f774c005830 comp rx=0 tx=0).stop 2024-05-30T16:44:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.820+0000 7f776e400640 1 -- 172.21.15.67:0/1942977428 shutdown_connections 2024-05-30T16:44:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.820+0000 7f776e400640 1 --2- 172.21.15.67:0/1942977428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f774c0051c0 0x7f774c00b9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.820+0000 7f776e400640 1 -- 172.21.15.67:0/1942977428 >> 172.21.15.67:0/1942977428 conn(0x7f77680b3e20 msgr2=0x7f77680b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.820+0000 7f776e400640 1 -- 172.21.15.67:0/1942977428 shutdown_connections 2024-05-30T16:44:36.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.820+0000 7f776e400640 1 -- 172.21.15.67:0/1942977428 wait complete. 2024-05-30T16:44:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.822+0000 7f776e400640 1 Processor -- start 2024-05-30T16:44:36.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.822+0000 7f776e400640 1 -- start start 2024-05-30T16:44:36.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f776e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f774c0051c0 0x7f77680b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:36.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f776e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77680c8600 0x7f77680b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:36.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f776e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77681523a0 0x7f7768152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:36.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f776e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f776815faa0 con 0x7f77680c8600 2024-05-30T16:44:36.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f776e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f776815f7c0 con 0x7f77681523a0 2024-05-30T16:44:36.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f776e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f776815ebb0 con 0x7f774c0051c0 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f7767e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77680c8600 0x7f77680b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f7767e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77680c8600 0x7f77680b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36470/0 (socket says 172.21.15.67:36470) 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f7767e00640 1 -- 172.21.15.67:0/585265537 learned_addr learned my addr 172.21.15.67:0/585265537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f776d000640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f774c0051c0 0x7f77680b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f776da00640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77681523a0 0x7f7768152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f7767e00640 1 -- 172.21.15.67:0/585265537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f774c0051c0 msgr2=0x7f77680b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f7767e00640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f774c0051c0 0x7f77680b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f7767e00640 1 -- 172.21.15.67:0/585265537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77681523a0 msgr2=0x7f7768152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f7767e00640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77681523a0 0x7f7768152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.823+0000 7f7767e00640 1 -- 172.21.15.67:0/585265537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77680c8a80 con 0x7f77680c8600 2024-05-30T16:44:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.824+0000 7f7767e00640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77680c8600 0x7f77680b2c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f775400d570 tx=0x7f775400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:36.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.824+0000 7f7764c00640 1 -- 172.21.15.67:0/585265537 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754003610 con 0x7f77680c8600 2024-05-30T16:44:36.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.824+0000 7f7764c00640 1 -- 172.21.15.67:0/585265537 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7754017070 con 0x7f77680c8600 2024-05-30T16:44:36.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.824+0000 7f7764c00640 1 -- 172.21.15.67:0/585265537 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77540039f0 con 0x7f77680c8600 2024-05-30T16:44:36.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.824+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77680c8d00 con 0x7f77680c8600 2024-05-30T16:44:36.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.824+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77681f7c80 con 0x7f77680c8600 2024-05-30T16:44:36.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.825+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77680690f0 con 0x7f77680c8600 2024-05-30T16:44:36.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.826+0000 7f7764c00640 1 -- 172.21.15.67:0/585265537 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7754003b90 con 0x7f77680c8600 2024-05-30T16:44:36.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.827+0000 7f7764c00640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f774806c920 0x7f774806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:36.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.827+0000 7f7764c00640 1 -- 172.21.15.67:0/585265537 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5473+0+0 (secure 0 0 0) 0x7f775408b8f0 con 0x7f77680c8600 2024-05-30T16:44:36.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.828+0000 7f776d000640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f774806c920 0x7f774806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.828+0000 7f776d000640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f774806c920 0x7f774806ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7758001580 tx=0x7f7758001250 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:36.832+0000 7f7764c00640 1 -- 172.21.15.67:0/585265537 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77680c8a80 con 0x7f77680c8600 2024-05-30T16:44:37.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.040+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f77680b4d30 con 0x7f77680c8600 2024-05-30T16:44:37.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.588+0000 7f7764c00640 1 -- 172.21.15.67:0/585265537 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7f7754059eb0 con 0x7f77680c8600 2024-05-30T16:44:37.589 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-05-30T16:44:37.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.591+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f774806c920 msgr2=0x7f774806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.591+0000 7f776e400640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f774806c920 0x7f774806ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7758001580 tx=0x7f7758001250 comp rx=0 tx=0).stop 2024-05-30T16:44:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.591+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77680c8600 msgr2=0x7f77680b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.591+0000 7f776e400640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77680c8600 0x7f77680b2c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f775400d570 tx=0x7f775400da30 comp rx=0 tx=0).stop 2024-05-30T16:44:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.592+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 shutdown_connections 2024-05-30T16:44:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.592+0000 7f776e400640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f774806c920 0x7f774806ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.592+0000 7f776e400640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77681523a0 0x7f7768152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.592+0000 7f776e400640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77680c8600 0x7f77680b2c10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.592+0000 7f776e400640 1 --2- 172.21.15.67:0/585265537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f774c0051c0 0x7f77680b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.592+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 >> 172.21.15.67:0/585265537 conn(0x7f77680b3e20 msgr2=0x7f776815b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.592+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 shutdown_connections 2024-05-30T16:44:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.592+0000 7f776e400640 1 -- 172.21.15.67:0/585265537 wait complete. 2024-05-30T16:44:37.624 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-30T16:44:37.624 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-30T16:44:37.625 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-30T16:44:37.625 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:44:37.625 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-30T16:44:37.625 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-30T16:44:37.625 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-30T16:44:37.625 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:44:37.626 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-30T16:44:37.626 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-30T16:44:37.626 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-30T16:44:37.626 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:44:37.626 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-30T16:44:37.626 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-30T16:44:37.627 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-30T16:44:37.627 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:44:37.642 INFO:tasks.ceph.mds.a:Stopped 2024-05-30T16:44:37.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-30T16:44:37.649 INFO:tasks.ceph.mds.c:Stopped 2024-05-30T16:44:37.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-30T16:44:37.651 INFO:tasks.ceph.mds.b:Stopped 2024-05-30T16:44:37.651 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-30T16:44:37.652 INFO:tasks.ceph.mds.d:Stopped 2024-05-30T16:44:37.652 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-30T16:44:37.697 INFO:tasks.ceph.mds.a:Started 2024-05-30T16:44:37.697 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:37.698 INFO:tasks.ceph.mds.b:Started 2024-05-30T16:44:37.698 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:37.702 INFO:tasks.ceph.mds.d:Started 2024-05-30T16:44:37.703 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:37.706 INFO:tasks.ceph.mds.c:Started 2024-05-30T16:44:37.706 DEBUG:teuthology.parallel:result is None 2024-05-30T16:44:37.706 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-30T16:44:37.852 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T16:44:37.856 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T16:44:37.862 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T16:44:37.862 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T16:44:37.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.894+0000 7f682dc00640 1 Processor -- start 2024-05-30T16:44:37.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.894+0000 7f682dc00640 1 -- start start 2024-05-30T16:44:37.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.895+0000 7f682dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f682815ed00 0x7f68280b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:37.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.895+0000 7f682dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68280d46c0 0x7f68280d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:37.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f682dc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68280b5070 con 0x7f68280d0d60 2024-05-30T16:44:37.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f682dc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f68280d5060 con 0x7f682815ed00 2024-05-30T16:44:37.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f682dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68280b41a0 con 0x7f68280d46c0 2024-05-30T16:44:37.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f6826a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f682815ed00 0x7f68280b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:37.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f6827400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f68280d0d60 0x7f68280d11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49804/0 (socket says 172.21.15.67:49804) 2024-05-30T16:44:37.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f6827400640 1 -- 172.21.15.67:0/103826000 learned_addr learned my addr 172.21.15.67:0/103826000 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:37.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f681fe00640 1 --2- 172.21.15.67:0/103826000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68280d46c0 0x7f68280d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:37.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f6826a00640 1 -- 172.21.15.67:0/103826000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68280d46c0 msgr2=0x7f68280d4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:37.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f6826000640 1 -- 172.21.15.67:0/103826000 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2450976416 0 0) 0x7f68280b5070 con 0x7f68280d0d60 2024-05-30T16:44:37.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f6826a00640 1 --2- 172.21.15.67:0/103826000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68280d46c0 0x7f68280d4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:37.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f6826a00640 1 -- 172.21.15.67:0/103826000 >> v1:172.21.15.67:6789/0 conn(0x7f68280d0d60 legacy=0x7f68280d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:37.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.896+0000 7f6826a00640 1 -- 172.21.15.67:0/103826000 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68280d33b0 con 0x7f682815ed00 2024-05-30T16:44:37.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.897+0000 7f6826a00640 1 --2- 172.21.15.67:0/103826000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f682815ed00 0x7f68280b3c60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f68180097a0 tx=0x7f681802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8031af8b7b332e37 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:37.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.898+0000 7f6826000640 1 -- 172.21.15.67:0/103826000 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818037070 con 0x7f682815ed00 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.898+0000 7f6826000640 1 -- 172.21.15.67:0/103826000 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6818032440 con 0x7f682815ed00 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.898+0000 7f6826000640 1 -- 172.21.15.67:0/103826000 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818032810 con 0x7f682815ed00 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.898+0000 7f682dc00640 1 -- 172.21.15.67:0/103826000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f682815ed00 msgr2=0x7f68280b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.898+0000 7f682dc00640 1 --2- 172.21.15.67:0/103826000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f682815ed00 0x7f68280b3c60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f68180097a0 tx=0x7f681802a180 comp rx=0 tx=0).stop 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.898+0000 7f682dc00640 1 -- 172.21.15.67:0/103826000 shutdown_connections 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.898+0000 7f682dc00640 1 --2- 172.21.15.67:0/103826000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68280d46c0 0x7f68280d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.898+0000 7f682dc00640 1 --2- 172.21.15.67:0/103826000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f682815ed00 0x7f68280b3c60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.898+0000 7f682dc00640 1 -- 172.21.15.67:0/103826000 >> 172.21.15.67:0/103826000 conn(0x7f682809da70 msgr2=0x7f682808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.899+0000 7f682dc00640 1 -- 172.21.15.67:0/103826000 shutdown_connections 2024-05-30T16:44:37.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.899+0000 7f682dc00640 1 -- 172.21.15.67:0/103826000 wait complete. 2024-05-30T16:44:37.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.900+0000 7f682dc00640 1 Processor -- start 2024-05-30T16:44:37.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.900+0000 7f682dc00640 1 -- start start 2024-05-30T16:44:37.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.901+0000 7f682dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68280d0d60 0x7f682815f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:37.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.901+0000 7f682dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68280d46c0 0x7f682815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:37.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.901+0000 7f682dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f682815ed00 0x7f68280cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:37.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.901+0000 7f682dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68280b5610 con 0x7f68280d0d60 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.901+0000 7f682dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f68280b5330 con 0x7f68280d46c0 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.901+0000 7f682dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68280d3040 con 0x7f682815ed00 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6826a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68280d0d60 0x7f682815f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6826a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68280d0d60 0x7f682815f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36526/0 (socket says 172.21.15.67:36526) 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6826a00640 1 -- 172.21.15.67:0/3950141646 learned_addr learned my addr 172.21.15.67:0/3950141646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6827400640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f682815ed00 0x7f68280cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f681fe00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68280d46c0 0x7f682815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6826a00640 1 -- 172.21.15.67:0/3950141646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f682815ed00 msgr2=0x7f68280cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6826a00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f682815ed00 0x7f68280cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6826a00640 1 -- 172.21.15.67:0/3950141646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68280d46c0 msgr2=0x7f682815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6826a00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68280d46c0 0x7f682815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6826a00640 1 -- 172.21.15.67:0/3950141646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682815f880 con 0x7f68280d0d60 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.902+0000 7f6826a00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68280d0d60 0x7f682815f0e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6818035040 tx=0x7f68180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.903+0000 7f681ea00640 1 -- 172.21.15.67:0/3950141646 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818037040 con 0x7f68280d0d60 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.903+0000 7f681ea00640 1 -- 172.21.15.67:0/3950141646 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6818030500 con 0x7f68280d0d60 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.903+0000 7f681ea00640 1 -- 172.21.15.67:0/3950141646 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818030910 con 0x7f68280d0d60 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.903+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682815fb60 con 0x7f68280d0d60 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.903+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6828160120 con 0x7f68280d0d60 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.904+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68280689e0 con 0x7f68280d0d60 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.905+0000 7f681ea00640 1 -- 172.21.15.67:0/3950141646 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6818044020 con 0x7f68280d0d60 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.906+0000 7f681ea00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f680406c920 0x7f680406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.906+0000 7f681ea00640 1 -- 172.21.15.67:0/3950141646 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4943+0+0 (secure 0 0 0) 0x7f68180aba50 con 0x7f68280d0d60 2024-05-30T16:44:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.906+0000 7f681fe00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f680406c920 0x7f680406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:37.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.907+0000 7f681fe00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f680406c920 0x7f680406ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f68280d07e0 tx=0x7f6810008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:37.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:37.910+0000 7f681ea00640 1 -- 172.21.15.67:0/3950141646 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f681807a180 con 0x7f68280d0d60 2024-05-30T16:44:38.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.131+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f68280b3ee0 con 0x7f68280d0d60 2024-05-30T16:44:38.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:44:38.399 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:44:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.609+0000 7f681ea00640 1 -- 172.21.15.67:0/3950141646 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) ==== 97+0+0 (secure 0 0 0) 0x7f6818079a80 con 0x7f68280d0d60 2024-05-30T16:44:38.610 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-05-30T16:44:38.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.612+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f680406c920 msgr2=0x7f680406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.612+0000 7f682dc00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f680406c920 0x7f680406ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f68280d07e0 tx=0x7f6810008040 comp rx=0 tx=0).stop 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.612+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68280d0d60 msgr2=0x7f682815f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.612+0000 7f682dc00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68280d0d60 0x7f682815f0e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6818035040 tx=0x7f68180046c0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.613+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 shutdown_connections 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.613+0000 7f682dc00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f680406c920 0x7f680406ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.613+0000 7f682dc00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f682815ed00 0x7f68280cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.613+0000 7f682dc00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68280d46c0 0x7f682815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.613+0000 7f682dc00640 1 --2- 172.21.15.67:0/3950141646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68280d0d60 0x7f682815f0e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.613+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 >> 172.21.15.67:0/3950141646 conn(0x7f682809da70 msgr2=0x7f682806e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.613+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 shutdown_connections 2024-05-30T16:44:38.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.613+0000 7f682dc00640 1 -- 172.21.15.67:0/3950141646 wait complete. 2024-05-30T16:44:38.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-30T16:44:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.839+0000 7fd865c00640 1 Processor -- start 2024-05-30T16:44:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.840+0000 7fd865c00640 1 -- start start 2024-05-30T16:44:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.841+0000 7fd865c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd860162940 0x7fd86006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.842+0000 7fd865c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd8600722d0 0x7fd8600adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.842+0000 7fd865c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd86006f590 con 0x7fd860162d20 2024-05-30T16:44:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.842+0000 7fd865c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd86006f870 con 0x7fd8600722d0 2024-05-30T16:44:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.842+0000 7fd865c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd8600ad3d0 con 0x7fd860162940 2024-05-30T16:44:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.842+0000 7fd857e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd860162d20 0x7fd860071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49810/0 (socket says 172.21.15.67:49810) 2024-05-30T16:44:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.842+0000 7fd857e00640 1 -- 172.21.15.67:0/4118743770 learned_addr learned my addr 172.21.15.67:0/4118743770 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.842+0000 7fd865200640 1 --2- 172.21.15.67:0/4118743770 >> v2:172.21.15.67:3300/0 conn(0x7fd8600722d0 0x7fd8600adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.842+0000 7fd85fe00640 1 --2- 172.21.15.67:0/4118743770 >> v2:172.21.15.169:3300/0 conn(0x7fd860162940 0x7fd86006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.843+0000 7fd85f400640 1 -- 172.21.15.67:0/4118743770 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681535309 0 0) 0x7fd86006f590 con 0x7fd860162d20 2024-05-30T16:44:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.843+0000 7fd85f400640 1 -- 172.21.15.67:0/4118743770 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd844003520 con 0x7fd860162d20 2024-05-30T16:44:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.843+0000 7fd865200640 1 -- 172.21.15.67:0/4118743770 >> v2:172.21.15.169:3300/0 conn(0x7fd860162940 msgr2=0x7fd86006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.843+0000 7fd865200640 1 --2- 172.21.15.67:0/4118743770 >> v2:172.21.15.169:3300/0 conn(0x7fd860162940 0x7fd86006eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.843+0000 7fd865200640 1 -- 172.21.15.67:0/4118743770 >> v1:172.21.15.67:6789/0 conn(0x7fd860162d20 legacy=0x7fd860071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.843+0000 7fd865200640 1 -- 172.21.15.67:0/4118743770 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86006dde0 con 0x7fd8600722d0 2024-05-30T16:44:38.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.844+0000 7fd865200640 1 --2- 172.21.15.67:0/4118743770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600722d0 0x7fd8600adbb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd860071100 tx=0x7fd85802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d38a7aaf02e5ea6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:38.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.844+0000 7fd85f400640 1 -- 172.21.15.67:0/4118743770 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858007740 con 0x7fd8600722d0 2024-05-30T16:44:38.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.844+0000 7fd85f400640 1 -- 172.21.15.67:0/4118743770 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8580078e0 con 0x7fd8600722d0 2024-05-30T16:44:38.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.844+0000 7fd85f400640 1 -- 172.21.15.67:0/4118743770 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858007c90 con 0x7fd8600722d0 2024-05-30T16:44:38.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.845+0000 7fd865c00640 1 -- 172.21.15.67:0/4118743770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600722d0 msgr2=0x7fd8600adbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:38.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.845+0000 7fd865c00640 1 --2- 172.21.15.67:0/4118743770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600722d0 0x7fd8600adbb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd860071100 tx=0x7fd85802a560 comp rx=0 tx=0).stop 2024-05-30T16:44:38.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.845+0000 7fd865c00640 1 -- 172.21.15.67:0/4118743770 shutdown_connections 2024-05-30T16:44:38.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.845+0000 7fd865c00640 1 --2- 172.21.15.67:0/4118743770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600722d0 0x7fd8600adbb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.845+0000 7fd865c00640 1 --2- 172.21.15.67:0/4118743770 >> v2:172.21.15.169:3300/0 conn(0x7fd860162940 0x7fd86006eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.845+0000 7fd865c00640 1 -- 172.21.15.67:0/4118743770 >> 172.21.15.67:0/4118743770 conn(0x7fd8600b1fc0 msgr2=0x7fd8600b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:38.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.845+0000 7fd865c00640 1 -- 172.21.15.67:0/4118743770 shutdown_connections 2024-05-30T16:44:38.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.845+0000 7fd865c00640 1 -- 172.21.15.67:0/4118743770 wait complete. 2024-05-30T16:44:38.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.847+0000 7fd865c00640 1 Processor -- start 2024-05-30T16:44:38.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.847+0000 7fd865c00640 1 -- start start 2024-05-30T16:44:38.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.848+0000 7fd865c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8600722d0 0x7fd86015b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:38.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.848+0000 7fd865c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd860162940 0x7fd86015fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:38.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.848+0000 7fd865c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd860162d20 0x7fd86015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:38.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.848+0000 7fd865c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8600ae0f0 con 0x7fd860162940 2024-05-30T16:44:38.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.848+0000 7fd865c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd86006f250 con 0x7fd860162d20 2024-05-30T16:44:38.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.848+0000 7fd865c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd8600ae270 con 0x7fd8600722d0 2024-05-30T16:44:38.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.848+0000 7fd857e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd860162940 0x7fd86015fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:38.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.848+0000 7fd857e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd860162940 0x7fd86015fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36546/0 (socket says 172.21.15.67:36546) 2024-05-30T16:44:38.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.848+0000 7fd857e00640 1 -- 172.21.15.67:0/1198888698 learned_addr learned my addr 172.21.15.67:0/1198888698 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd865200640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd860162d20 0x7fd86015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd857e00640 1 -- 172.21.15.67:0/1198888698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8600722d0 msgr2=0x7fd86015b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd857e00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8600722d0 0x7fd86015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd857e00640 1 -- 172.21.15.67:0/1198888698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd860162d20 msgr2=0x7fd86015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd857e00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd860162d20 0x7fd86015ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd857e00640 1 -- 172.21.15.67:0/1198888698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8601f83a0 con 0x7fd860162940 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd865200640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd860162d20 0x7fd86015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd857e00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd860162940 0x7fd86015fa50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd84800c3c0 tx=0x7fd84800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd85cc00640 1 -- 172.21.15.67:0/1198888698 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848017070 con 0x7fd860162940 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd85cc00640 1 -- 172.21.15.67:0/1198888698 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd848005ce0 con 0x7fd860162940 2024-05-30T16:44:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.849+0000 7fd85cc00640 1 -- 172.21.15.67:0/1198888698 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848007310 con 0x7fd860162940 2024-05-30T16:44:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.850+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8601f8680 con 0x7fd860162940 2024-05-30T16:44:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.850+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8601f8c40 con 0x7fd860162940 2024-05-30T16:44:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.851+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8600689e0 con 0x7fd860162940 2024-05-30T16:44:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.852+0000 7fd85cc00640 1 -- 172.21.15.67:0/1198888698 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd84801a070 con 0x7fd860162940 2024-05-30T16:44:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.853+0000 7fd85cc00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd83c06c8f0 0x7fd83c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.853+0000 7fd85cc00640 1 -- 172.21.15.67:0/1198888698 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fd84808ab00 con 0x7fd860162940 2024-05-30T16:44:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.853+0000 7fd85fe00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd83c06c8f0 0x7fd83c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:38.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.854+0000 7fd85fe00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd83c06c8f0 0x7fd83c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd850006a90 tx=0x7fd850006a20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:38.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:38.858+0000 7fd85cc00640 1 -- 172.21.15.67:0/1198888698 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd848059310 con 0x7fd860162940 2024-05-30T16:44:39.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.137+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd86006cbc0 con 0x7fd860162940 2024-05-30T16:44:39.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.138+0000 7fd85cc00640 1 -- 172.21.15.67:0/1198888698 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fd848058c10 con 0x7fd860162940 2024-05-30T16:44:39.139 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:39.139 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:44:39.139 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA==", 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw==", 2024-05-30T16:44:39.140 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw==", 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:44:39.141 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg==", 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng==", 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg==", 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:44:39.143 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w==", 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-05-30T16:44:39.144 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ==", 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A==", 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ==", 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w==", 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:44:39.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A==", 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-05-30T16:44:39.147 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg==", 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ==", 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:44:39.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.2", 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ==", 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.3", 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A==", 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCArFhmF5mhNhAA4H3cgcH6YdD53fANDW4seA==", 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-mds", 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCurFhmTERZIxAAsri+Jq24X/7YiRNWMt7+sg==", 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-mds" 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-mgr", 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCurFhmaWJZIxAABSAwmq782yVqiIMmbZvXqA==", 2024-05-30T16:44:39.151 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-osd", 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCurFhmqH9ZIxAAlKFfjs24/oeWXu3KE1yIxg==", 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-osd" 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-rbd", 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCurFhmfZlZIxAAm40rQ2Qxvt193C3rHcsULA==", 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.152 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCurFhm/7JZIxAAFfvEzixfZRJ7yvp59KDPjA==", 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-rgw", 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCurFhmZM1ZIxAA4spQ4rfV2eps0ulhEOtl4A==", 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-30T16:44:39.153 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ==", 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw==", 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:44:39.154 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:44:39.155 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:44:39.155 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:44:39.155 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.155 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:39.155 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-05-30T16:44:39.155 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:44:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd83c06c8f0 msgr2=0x7fd83c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd83c06c8f0 0x7fd83c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd850006a90 tx=0x7fd850006a20 comp rx=0 tx=0).stop 2024-05-30T16:44:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd860162940 msgr2=0x7fd86015fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd860162940 0x7fd86015fa50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd84800c3c0 tx=0x7fd84800c880 comp rx=0 tx=0).stop 2024-05-30T16:44:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 shutdown_connections 2024-05-30T16:44:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd83c06c8f0 0x7fd83c06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd860162d20 0x7fd86015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd860162940 0x7fd86015fa50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 --2- 172.21.15.67:0/1198888698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8600722d0 0x7fd86015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 >> 172.21.15.67:0/1198888698 conn(0x7fd8600b1fc0 msgr2=0x7fd8600b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:39.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 shutdown_connections 2024-05-30T16:44:39.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.141+0000 7fd865c00640 1 -- 172.21.15.67:0/1198888698 wait complete. 2024-05-30T16:44:39.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-30T16:44:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.375+0000 7f853dc00640 1 Processor -- start 2024-05-30T16:44:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.375+0000 7f853dc00640 1 -- start start 2024-05-30T16:44:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.377+0000 7f853dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8538156b30 0x7f85380af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.377+0000 7f853dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f853806ee40 0x7f85380ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.377+0000 7f853dc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8538156f10 con 0x7f853806f2c0 2024-05-30T16:44:39.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.377+0000 7f853dc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f853806dda0 con 0x7f853806ee40 2024-05-30T16:44:39.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.377+0000 7f853dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f853806e080 con 0x7f8538156b30 2024-05-30T16:44:39.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.378+0000 7f8537e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8538156b30 0x7f85380af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.378+0000 7f852fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f853806ee40 0x7f85380ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.378+0000 7f8537e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8538156b30 0x7f85380af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:53812/0 (socket says 172.21.15.67:53812) 2024-05-30T16:44:39.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.378+0000 7f8537e00640 1 -- 172.21.15.67:0/3419210209 learned_addr learned my addr 172.21.15.67:0/3419210209 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:39.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.379+0000 7f8537e00640 1 -- 172.21.15.67:0/3419210209 >> v2:172.21.15.169:3300/0 conn(0x7f853806ee40 msgr2=0x7f85380ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.379+0000 7f8537e00640 1 --2- 172.21.15.67:0/3419210209 >> v2:172.21.15.169:3300/0 conn(0x7f853806ee40 0x7f85380ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.379+0000 7f8537e00640 1 -- 172.21.15.67:0/3419210209 >> v1:172.21.15.169:6789/0 conn(0x7f853806f2c0 legacy=0x7f85380b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.379+0000 7f8537e00640 1 -- 172.21.15.67:0/3419210209 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853806e360 con 0x7f8538156b30 2024-05-30T16:44:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.379+0000 7f852fe00640 1 --2- 172.21.15.67:0/3419210209 >> v2:172.21.15.169:3300/0 conn(0x7f853806ee40 0x7f85380ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:44:39.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.381+0000 7f8537e00640 1 --2- 172.21.15.67:0/3419210209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8538156b30 0x7f85380af6e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8528009870 tx=0x7f852802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5aa1f06311064fa server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.381+0000 7f8537400640 1 -- 172.21.15.67:0/3419210209 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528037070 con 0x7f8538156b30 2024-05-30T16:44:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.381+0000 7f8537400640 1 -- 172.21.15.67:0/3419210209 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8528032440 con 0x7f8538156b30 2024-05-30T16:44:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.381+0000 7f8537400640 1 -- 172.21.15.67:0/3419210209 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528032810 con 0x7f8538156b30 2024-05-30T16:44:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.382+0000 7f853dc00640 1 -- 172.21.15.67:0/3419210209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8538156b30 msgr2=0x7f85380af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.382+0000 7f853dc00640 1 --2- 172.21.15.67:0/3419210209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8538156b30 0x7f85380af6e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8528009870 tx=0x7f852802a1c0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.382+0000 7f853dc00640 1 -- 172.21.15.67:0/3419210209 shutdown_connections 2024-05-30T16:44:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.382+0000 7f853dc00640 1 --2- 172.21.15.67:0/3419210209 >> v2:172.21.15.169:3300/0 conn(0x7f853806ee40 0x7f85380ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.382+0000 7f853dc00640 1 --2- 172.21.15.67:0/3419210209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8538156b30 0x7f85380af6e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.382+0000 7f853dc00640 1 -- 172.21.15.67:0/3419210209 >> 172.21.15.67:0/3419210209 conn(0x7f85380b1fc0 msgr2=0x7f85380b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.382+0000 7f853dc00640 1 -- 172.21.15.67:0/3419210209 shutdown_connections 2024-05-30T16:44:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.382+0000 7f853dc00640 1 -- 172.21.15.67:0/3419210209 wait complete. 2024-05-30T16:44:39.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.384+0000 7f853dc00640 1 Processor -- start 2024-05-30T16:44:39.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.384+0000 7f853dc00640 1 -- start start 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.385+0000 7f853dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f853806ee40 0x7f85381f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.385+0000 7f853dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853806f2c0 0x7f85380b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.385+0000 7f853dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8538156b30 0x7f85380b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.385+0000 7f853dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85380afc20 con 0x7f853806f2c0 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.385+0000 7f853dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85380b2c30 con 0x7f8538156b30 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.385+0000 7f853dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8538159b30 con 0x7f853806ee40 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f852fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853806f2c0 0x7f85380b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f852fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853806f2c0 0x7f85380b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36552/0 (socket says 172.21.15.67:36552) 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f852fe00640 1 -- 172.21.15.67:0/2293380848 learned_addr learned my addr 172.21.15.67:0/2293380848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f853d200640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8538156b30 0x7f85380b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f8537e00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f853806ee40 0x7f85381f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f852fe00640 1 -- 172.21.15.67:0/2293380848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f853806ee40 msgr2=0x7f85381f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f852fe00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f853806ee40 0x7f85381f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f852fe00640 1 -- 172.21.15.67:0/2293380848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8538156b30 msgr2=0x7f85380b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f852fe00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8538156b30 0x7f85380b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f852fe00640 1 -- 172.21.15.67:0/2293380848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8538158f20 con 0x7f853806f2c0 2024-05-30T16:44:39.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.386+0000 7f852fe00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853806f2c0 0x7f85380b47d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f852000e960 tx=0x7f852000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:39.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.387+0000 7f8534c00640 1 -- 172.21.15.67:0/2293380848 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520017070 con 0x7f853806f2c0 2024-05-30T16:44:39.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.387+0000 7f8534c00640 1 -- 172.21.15.67:0/2293380848 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852000ca00 con 0x7f853806f2c0 2024-05-30T16:44:39.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.387+0000 7f8534c00640 1 -- 172.21.15.67:0/2293380848 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852000cdd0 con 0x7f853806f2c0 2024-05-30T16:44:39.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.387+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8538159200 con 0x7f853806f2c0 2024-05-30T16:44:39.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.387+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85381597c0 con 0x7f853806f2c0 2024-05-30T16:44:39.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.388+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85380689e0 con 0x7f853806f2c0 2024-05-30T16:44:39.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.389+0000 7f8534c00640 1 -- 172.21.15.67:0/2293380848 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f852001a070 con 0x7f853806f2c0 2024-05-30T16:44:39.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.390+0000 7f8534c00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f851406c8f0 0x7f851406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.390+0000 7f8534c00640 1 -- 172.21.15.67:0/2293380848 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f852008bb10 con 0x7f853806f2c0 2024-05-30T16:44:39.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.390+0000 7f8537e00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f851406c8f0 0x7f851406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.391+0000 7f8537e00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f851406c8f0 0x7f851406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8528034000 tx=0x7f852802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.394+0000 7f8534c00640 1 -- 172.21.15.67:0/2293380848 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f852005a1f0 con 0x7f853806f2c0 2024-05-30T16:44:39.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.679+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f853806f7f0 con 0x7f853806f2c0 2024-05-30T16:44:39.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.684+0000 7f8534c00640 1 -- 172.21.15.67:0/2293380848 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f8520059af0 con 0x7f853806f2c0 2024-05-30T16:44:39.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f851406c8f0 msgr2=0x7f851406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f851406c8f0 0x7f851406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8528034000 tx=0x7f852802e000 comp rx=0 tx=0).stop 2024-05-30T16:44:39.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853806f2c0 msgr2=0x7f85380b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853806f2c0 0x7f85380b47d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f852000e960 tx=0x7f852000ee20 comp rx=0 tx=0).stop 2024-05-30T16:44:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 shutdown_connections 2024-05-30T16:44:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f851406c8f0 0x7f851406edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8538156b30 0x7f85380b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853806f2c0 0x7f85380b47d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 --2- 172.21.15.67:0/2293380848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f853806ee40 0x7f85381f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.687+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 >> 172.21.15.67:0/2293380848 conn(0x7f85380b1fc0 msgr2=0x7f853815b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.688+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 shutdown_connections 2024-05-30T16:44:39.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.688+0000 7f853dc00640 1 -- 172.21.15.67:0/2293380848 wait complete. 2024-05-30T16:44:39.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-30T16:44:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.920+0000 7f6fe1800640 1 Processor -- start 2024-05-30T16:44:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.920+0000 7f6fe1800640 1 -- start start 2024-05-30T16:44:39.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.921+0000 7f6fe1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc15e870 0x7f6fdc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.921+0000 7f6fe1800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fdc0cfa40 0x7f6fdc0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.921+0000 7f6fe1800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fdc15ec50 con 0x7f6fdc0d14d0 2024-05-30T16:44:39.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.921+0000 7f6fe1800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f6fdc0d2150 con 0x7f6fdc0cfa40 2024-05-30T16:44:39.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.921+0000 7f6fe1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc0cff80 con 0x7f6fdc15e870 2024-05-30T16:44:39.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.922+0000 7f6fe0e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fdc0cfa40 0x7f6fdc0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.922+0000 7f6fe0e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fdc0cfa40 0x7f6fdc0d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36556/0 (socket says 172.21.15.67:36556) 2024-05-30T16:44:39.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.922+0000 7f6fe0e00640 1 -- 172.21.15.67:0/1674814545 learned_addr learned my addr 172.21.15.67:0/1674814545 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:39.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.922+0000 7f6fdbe00640 1 --2- 172.21.15.67:0/1674814545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc15e870 0x7f6fdc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.922+0000 7f6fe0e00640 1 -- 172.21.15.67:0/1674814545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc15e870 msgr2=0x7f6fdc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:39.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.922+0000 7f6fe0e00640 1 --2- 172.21.15.67:0/1674814545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc15e870 0x7f6fdc0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.922+0000 7f6fe0e00640 1 -- 172.21.15.67:0/1674814545 >> v1:172.21.15.169:6789/0 conn(0x7f6fdc0d14d0 legacy=0x7f6fdc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:39.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.922+0000 7f6fe0e00640 1 -- 172.21.15.67:0/1674814545 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc0d0260 con 0x7f6fdc0cfa40 2024-05-30T16:44:39.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.924+0000 7f6fe0e00640 1 --2- 172.21.15.67:0/1674814545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc0cfa40 0x7f6fdc0d25e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009e50 tx=0x7f6fd402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654e9d7c8351eba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:39.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.924+0000 7f6fdaa00640 1 -- 172.21.15.67:0/1674814545 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4036070 con 0x7f6fdc0cfa40 2024-05-30T16:44:39.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.924+0000 7f6fdaa00640 1 -- 172.21.15.67:0/1674814545 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd4031600 con 0x7f6fdc0cfa40 2024-05-30T16:44:39.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.924+0000 7f6fdaa00640 1 -- 172.21.15.67:0/1674814545 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40319d0 con 0x7f6fdc0cfa40 2024-05-30T16:44:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.925+0000 7f6fe1800640 1 -- 172.21.15.67:0/1674814545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc0cfa40 msgr2=0x7f6fdc0d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.925+0000 7f6fe1800640 1 --2- 172.21.15.67:0/1674814545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc0cfa40 0x7f6fdc0d25e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009e50 tx=0x7f6fd402a6c0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.925+0000 7f6fe1800640 1 -- 172.21.15.67:0/1674814545 shutdown_connections 2024-05-30T16:44:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.925+0000 7f6fe1800640 1 --2- 172.21.15.67:0/1674814545 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc0cfa40 0x7f6fdc0d25e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.925+0000 7f6fe1800640 1 --2- 172.21.15.67:0/1674814545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc15e870 0x7f6fdc0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.925+0000 7f6fe1800640 1 -- 172.21.15.67:0/1674814545 >> 172.21.15.67:0/1674814545 conn(0x7f6fdc08bfc0 msgr2=0x7f6fdc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.925+0000 7f6fe1800640 1 -- 172.21.15.67:0/1674814545 shutdown_connections 2024-05-30T16:44:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.925+0000 7f6fe1800640 1 -- 172.21.15.67:0/1674814545 wait complete. 2024-05-30T16:44:39.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.927+0000 7f6fe1800640 1 Processor -- start 2024-05-30T16:44:39.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.927+0000 7f6fe1800640 1 -- start start 2024-05-30T16:44:39.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.928+0000 7f6fe1800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fdc0cfa40 0x7f6fdc14c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.928+0000 7f6fe1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc0d14d0 0x7f6fdc160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.928+0000 7f6fe1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc15e870 0x7f6fdc0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.928+0000 7f6fe1800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc0d4fb0 con 0x7f6fdc15e870 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.928+0000 7f6fe1800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc0d0820 con 0x7f6fdc0cfa40 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.928+0000 7f6fe1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc0d2b20 con 0x7f6fdc0d14d0 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.928+0000 7f6fe0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc15e870 0x7f6fdc0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.928+0000 7f6fe0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc15e870 0x7f6fdc0b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36570/0 (socket says 172.21.15.67:36570) 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.928+0000 7f6fe0e00640 1 -- 172.21.15.67:0/672545641 learned_addr learned my addr 172.21.15.67:0/672545641 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fdbe00640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fdc0cfa40 0x7f6fdc14c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fdb400640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc0d14d0 0x7f6fdc160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fe0e00640 1 -- 172.21.15.67:0/672545641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc0d14d0 msgr2=0x7f6fdc160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fe0e00640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc0d14d0 0x7f6fdc160970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fe0e00640 1 -- 172.21.15.67:0/672545641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fdc0cfa40 msgr2=0x7f6fdc14c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fe0e00640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fdc0cfa40 0x7f6fdc14c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fe0e00640 1 -- 172.21.15.67:0/672545641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc14cee0 con 0x7f6fdc15e870 2024-05-30T16:44:39.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fe0e00640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc15e870 0x7f6fdc0b7a40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4034040 tx=0x7f6fd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:39.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fd3e00640 1 -- 172.21.15.67:0/672545641 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4041020 con 0x7f6fdc15e870 2024-05-30T16:44:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fd3e00640 1 -- 172.21.15.67:0/672545641 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd40077a0 con 0x7f6fdc15e870 2024-05-30T16:44:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.929+0000 7f6fd3e00640 1 -- 172.21.15.67:0/672545641 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4036040 con 0x7f6fdc15e870 2024-05-30T16:44:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.930+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc161470 con 0x7f6fdc15e870 2024-05-30T16:44:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.930+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc161a30 con 0x7f6fdc15e870 2024-05-30T16:44:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.931+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc0689e0 con 0x7f6fdc15e870 2024-05-30T16:44:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.932+0000 7f6fd3e00640 1 -- 172.21.15.67:0/672545641 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fd4044020 con 0x7f6fdc15e870 2024-05-30T16:44:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.933+0000 7f6fd3e00640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.933+0000 7f6fd3e00640 1 -- 172.21.15.67:0/672545641 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f6fd40ac790 con 0x7f6fdc15e870 2024-05-30T16:44:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.933+0000 7f6fdbe00640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:39.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.934+0000 7f6fdbe00640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00b7c0 tx=0x7f6fcc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:39.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:39.937+0000 7f6fd3e00640 1 -- 172.21.15.67:0/672545641 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fd404b030 con 0x7f6fdc15e870 2024-05-30T16:44:40.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.219+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f6fdc0ba500 con 0x7f6fdc15e870 2024-05-30T16:44:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.224+0000 7f6fd3e00640 1 -- 172.21.15.67:0/672545641 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f6fdc14cee0 con 0x7f6fdc15e870 2024-05-30T16:44:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fbc06c8f0 msgr2=0x7f6fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00b7c0 tx=0x7f6fcc008040 comp rx=0 tx=0).stop 2024-05-30T16:44:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc15e870 msgr2=0x7f6fdc0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc15e870 0x7f6fdc0b7a40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4034040 tx=0x7f6fd4004a40 comp rx=0 tx=0).stop 2024-05-30T16:44:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 shutdown_connections 2024-05-30T16:44:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fbc06c8f0 0x7f6fbc06edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fdc15e870 0x7f6fdc0b7a40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fdc0d14d0 0x7f6fdc160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 --2- 172.21.15.67:0/672545641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fdc0cfa40 0x7f6fdc14c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 >> 172.21.15.67:0/672545641 conn(0x7f6fdc08bfc0 msgr2=0x7f6fdc0b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:40.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 shutdown_connections 2024-05-30T16:44:40.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.227+0000 7f6fe1800640 1 -- 172.21.15.67:0/672545641 wait complete. 2024-05-30T16:44:40.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-30T16:44:40.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.460+0000 7f9cd0a00640 1 Processor -- start 2024-05-30T16:44:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.460+0000 7f9cd0a00640 1 -- start start 2024-05-30T16:44:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.461+0000 7f9cd0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc158b80 0x7f9ccc06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9cd0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ccc0722d0 0x7f9ccc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9cd0a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc06e6e0 con 0x7f9ccc06e160 2024-05-30T16:44:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9cd0a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9ccc06e9c0 con 0x7f9ccc0722d0 2024-05-30T16:44:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9cd0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ccc06eca0 con 0x7f9ccc158b80 2024-05-30T16:44:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9ccb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ccc0722d0 0x7f9ccc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9ccb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ccc0722d0 0x7f9ccc0b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36574/0 (socket says 172.21.15.67:36574) 2024-05-30T16:44:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9ccb200640 1 -- 172.21.15.67:0/3203279877 learned_addr learned my addr 172.21.15.67:0/3203279877 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9cca800640 1 --2- 172.21.15.67:0/3203279877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc158b80 0x7f9ccc06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9ccb200640 1 -- 172.21.15.67:0/3203279877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc158b80 msgr2=0x7f9ccc06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9ccb200640 1 --2- 172.21.15.67:0/3203279877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc158b80 0x7f9ccc06dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9ccb200640 1 -- 172.21.15.67:0/3203279877 >> v1:172.21.15.169:6789/0 conn(0x7f9ccc06e160 legacy=0x7f9ccc071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.462+0000 7f9ccb200640 1 -- 172.21.15.67:0/3203279877 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc06ef80 con 0x7f9ccc0722d0 2024-05-30T16:44:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.463+0000 7f9ccb200640 1 --2- 172.21.15.67:0/3203279877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc0722d0 0x7f9ccc0b32c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009e50 tx=0x7f9cc402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab6504743c88ab1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.464+0000 7f9cc9e00640 1 -- 172.21.15.67:0/3203279877 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4036070 con 0x7f9ccc0722d0 2024-05-30T16:44:40.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.464+0000 7f9cc9e00640 1 -- 172.21.15.67:0/3203279877 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc4031600 con 0x7f9ccc0722d0 2024-05-30T16:44:40.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.464+0000 7f9cc9e00640 1 -- 172.21.15.67:0/3203279877 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40319d0 con 0x7f9ccc0722d0 2024-05-30T16:44:40.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.464+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3203279877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc0722d0 msgr2=0x7f9ccc0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:40.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.464+0000 7f9cd0a00640 1 --2- 172.21.15.67:0/3203279877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc0722d0 0x7f9ccc0b32c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009e50 tx=0x7f9cc402a6c0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.465+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3203279877 shutdown_connections 2024-05-30T16:44:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.465+0000 7f9cd0a00640 1 --2- 172.21.15.67:0/3203279877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc0722d0 0x7f9ccc0b32c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.465+0000 7f9cd0a00640 1 --2- 172.21.15.67:0/3203279877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc158b80 0x7f9ccc06dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.465+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3203279877 >> 172.21.15.67:0/3203279877 conn(0x7f9ccc0b1fc0 msgr2=0x7f9ccc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.465+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3203279877 shutdown_connections 2024-05-30T16:44:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.465+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3203279877 wait complete. 2024-05-30T16:44:40.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.466+0000 7f9cd0a00640 1 Processor -- start 2024-05-30T16:44:40.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.467+0000 7f9cd0a00640 1 -- start start 2024-05-30T16:44:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9cd0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ccc06e160 0x7f9ccc0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9cd0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc0722d0 0x7f9ccc13e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9cd0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc158b80 0x7f9ccc153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9cd0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc06f780 con 0x7f9ccc158b80 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9cd0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc0b2d30 con 0x7f9ccc06e160 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9cd0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ccc13f110 con 0x7f9ccc0722d0 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9ccb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc158b80 0x7f9ccc153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9ccb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc158b80 0x7f9ccc153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36576/0 (socket says 172.21.15.67:36576) 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9ccb200640 1 -- 172.21.15.67:0/3262092347 learned_addr learned my addr 172.21.15.67:0/3262092347 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9cca800640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ccc06e160 0x7f9ccc0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9cc3e00640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc0722d0 0x7f9ccc13e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9ccb200640 1 -- 172.21.15.67:0/3262092347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc0722d0 msgr2=0x7f9ccc13e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9ccb200640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc0722d0 0x7f9ccc13e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9ccb200640 1 -- 172.21.15.67:0/3262092347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ccc06e160 msgr2=0x7f9ccc0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9ccb200640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ccc06e160 0x7f9ccc0b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.468+0000 7f9ccb200640 1 -- 172.21.15.67:0/3262092347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc13f3f0 con 0x7f9ccc158b80 2024-05-30T16:44:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.469+0000 7f9ccb200640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc158b80 0x7f9ccc153370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009e50 tx=0x7f9cc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.469+0000 7f9cc2a00640 1 -- 172.21.15.67:0/3262092347 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4036050 con 0x7f9ccc158b80 2024-05-30T16:44:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.469+0000 7f9cc2a00640 1 -- 172.21.15.67:0/3262092347 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc4042070 con 0x7f9ccc158b80 2024-05-30T16:44:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.469+0000 7f9cc2a00640 1 -- 172.21.15.67:0/3262092347 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4009510 con 0x7f9ccc158b80 2024-05-30T16:44:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.469+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc13f6d0 con 0x7f9ccc158b80 2024-05-30T16:44:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.469+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc13fc90 con 0x7f9ccc158b80 2024-05-30T16:44:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.470+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc0689e0 con 0x7f9ccc158b80 2024-05-30T16:44:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.471+0000 7f9cc2a00640 1 -- 172.21.15.67:0/3262092347 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cc4007bf0 con 0x7f9ccc158b80 2024-05-30T16:44:40.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.473+0000 7f9cc2a00640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ca806c8f0 0x7f9ca806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:40.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.473+0000 7f9cc2a00640 1 -- 172.21.15.67:0/3262092347 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f9cc40ac7c0 con 0x7f9ccc158b80 2024-05-30T16:44:40.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.473+0000 7f9cca800640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ca806c8f0 0x7f9ca806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:40.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.474+0000 7f9cca800640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ca806c8f0 0x7f9ca806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006fd0 tx=0x7f9cbc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:40.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.477+0000 7f9cc2a00640 1 -- 172.21.15.67:0/3262092347 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9cc4045360 con 0x7f9ccc158b80 2024-05-30T16:44:40.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.759+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9ccc153f50 con 0x7f9ccc158b80 2024-05-30T16:44:40.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.764+0000 7f9cc2a00640 1 -- 172.21.15.67:0/3262092347 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f9cc407aea0 con 0x7f9ccc158b80 2024-05-30T16:44:40.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ca806c8f0 msgr2=0x7f9ca806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:40.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ca806c8f0 0x7f9ca806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006fd0 tx=0x7f9cbc008040 comp rx=0 tx=0).stop 2024-05-30T16:44:40.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc158b80 msgr2=0x7f9ccc153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:40.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc158b80 0x7f9ccc153370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009e50 tx=0x7f9cc4004a40 comp rx=0 tx=0).stop 2024-05-30T16:44:40.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 shutdown_connections 2024-05-30T16:44:40.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ca806c8f0 0x7f9ca806edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ccc158b80 0x7f9ccc153370 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ccc0722d0 0x7f9ccc13e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 --2- 172.21.15.67:0/3262092347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ccc06e160 0x7f9ccc0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:40.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 >> 172.21.15.67:0/3262092347 conn(0x7f9ccc0b1fc0 msgr2=0x7f9ccc1523e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:40.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 shutdown_connections 2024-05-30T16:44:40.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.767+0000 7f9cd0a00640 1 -- 172.21.15.67:0/3262092347 wait complete. 2024-05-30T16:44:40.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-30T16:44:40.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.995+0000 7efccf000640 1 Processor -- start 2024-05-30T16:44:40.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.996+0000 7efccf000640 1 -- start start 2024-05-30T16:44:40.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.997+0000 7efccf000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efcc80b6570 0x7efcc81697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:40.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.997+0000 7efccf000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc80d3300 con 0x7efcc8160380 2024-05-30T16:44:40.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.997+0000 7efccf000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc80d35e0 con 0x7efcc815fae0 2024-05-30T16:44:40.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.997+0000 7efccf000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efcc80ced60 con 0x7efcc80b6570 2024-05-30T16:44:40.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.997+0000 7efccd200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efcc8160380 0x7efcc80b5e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34594/0 (socket says 172.21.15.67:34594) 2024-05-30T16:44:40.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efccd200640 1 -- 172.21.15.67:0/1007610680 learned_addr learned my addr 172.21.15.67:0/1007610680 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:41.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efcce600640 1 --2- 172.21.15.67:0/1007610680 >> v2:172.21.15.169:3300/0 conn(0x7efcc80b6570 0x7efcc81697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972056992 0 0) 0x7efcc80d3300 con 0x7efcc8160380 2024-05-30T16:44:41.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcb0003520 con 0x7efcc8160380 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 881435781 0 0) 0x7efcb0003520 con 0x7efcc8160380 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 >> v2:172.21.15.169:3300/0 conn(0x7efcc80b6570 msgr2=0x7efcc81697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efcc7e00640 1 --2- 172.21.15.67:0/1007610680 >> v2:172.21.15.169:3300/0 conn(0x7efcc80b6570 0x7efcc81697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 >> v1:172.21.15.169:6789/0 conn(0x7efcc815fae0 legacy=0x7efcc80d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc80d38c0 con 0x7efcc8160380 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.998+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7efcb40017f0 con 0x7efcc8160380 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 >> v1:172.21.15.67:6789/0 conn(0x7efcc8160380 legacy=0x7efcc80b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efcc7e00640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb00039b0 0x7efcb0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efcc7e00640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efcc7e00640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb00051c0 0x7efcb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efcc80ced60 con 0x7efcb0004800 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efcb000c090 con 0x7efcb00051c0 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efcb000c290 con 0x7efcb00039b0 2024-05-30T16:44:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efccd200640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efccdc00640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb00039b0 0x7efcb0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efcce600640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb00051c0 0x7efcb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efccd200640 1 -- 172.21.15.67:0/1007610680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb00039b0 msgr2=0x7efcb0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efccd200640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb00039b0 0x7efcb0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efccd200640 1 -- 172.21.15.67:0/1007610680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb00051c0 msgr2=0x7efcb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efccd200640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb00051c0 0x7efcb000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efcce600640 1 -- 172.21.15.67:0/1007610680 reap_dead start 2024-05-30T16:44:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:40.999+0000 7efccd200640 1 -- 172.21.15.67:0/1007610680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb0003520 con 0x7efcb0004800 2024-05-30T16:44:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.000+0000 7efccdc00640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb00039b0 0x7efcb0003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:41.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.000+0000 7efccd200640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcb0004c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c210 tx=0x7efcb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779f4ce118676de server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:41.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.001+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4007ce0 con 0x7efcb0004800 2024-05-30T16:44:41.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.001+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb4007270 con 0x7efcb0004800 2024-05-30T16:44:41.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.001+0000 7efcc7e00640 1 -- 172.21.15.67:0/1007610680 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40057a0 con 0x7efcb0004800 2024-05-30T16:44:41.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.001+0000 7efccf000640 1 -- 172.21.15.67:0/1007610680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 msgr2=0x7efcb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.001+0000 7efccf000640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcb0004c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c210 tx=0x7efcb0005830 comp rx=0 tx=0).stop 2024-05-30T16:44:41.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.001+0000 7efccf000640 1 -- 172.21.15.67:0/1007610680 shutdown_connections 2024-05-30T16:44:41.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.001+0000 7efccf000640 1 --2- 172.21.15.67:0/1007610680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcb0004c80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.001+0000 7efccf000640 1 -- 172.21.15.67:0/1007610680 >> 172.21.15.67:0/1007610680 conn(0x7efcc808bfc0 msgr2=0x7efcc80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:41.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.002+0000 7efccf000640 1 -- 172.21.15.67:0/1007610680 shutdown_connections 2024-05-30T16:44:41.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.002+0000 7efccf000640 1 -- 172.21.15.67:0/1007610680 wait complete. 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.003+0000 7efccf000640 1 Processor -- start 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.003+0000 7efccf000640 1 -- start start 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.004+0000 7efccf000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcc81f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.004+0000 7efccf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcc806dcb0 0x7efcc806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.004+0000 7efccf000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcc81f47a0 0x7efcc81f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.004+0000 7efccf000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efcc80b55b0 con 0x7efcb0004800 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.004+0000 7efccf000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efcc80b5430 con 0x7efcc81f47a0 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.004+0000 7efccf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efcc80d4e20 con 0x7efcc806dcb0 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcc81f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcc81f1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50944/0 (socket says 172.21.15.67:50944) 2024-05-30T16:44:41.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccdc00640 1 -- 172.21.15.67:0/374685845 learned_addr learned my addr 172.21.15.67:0/374685845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:41.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccd200640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcc806dcb0 0x7efcc806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efcce600640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcc81f47a0 0x7efcc81f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccdc00640 1 -- 172.21.15.67:0/374685845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcc806dcb0 msgr2=0x7efcc806e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccdc00640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcc806dcb0 0x7efcc806e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccdc00640 1 -- 172.21.15.67:0/374685845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcc81f47a0 msgr2=0x7efcc81f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccdc00640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcc81f47a0 0x7efcc81f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccdc00640 1 -- 172.21.15.67:0/374685845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc80cd630 con 0x7efcb0004800 2024-05-30T16:44:41.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.005+0000 7efccdc00640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcc81f1a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00d4b0 tx=0x7efcbc00d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:41.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.006+0000 7efcc5600640 1 -- 172.21.15.67:0/374685845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc005480 con 0x7efcb0004800 2024-05-30T16:44:41.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.006+0000 7efcc5600640 1 -- 172.21.15.67:0/374685845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcbc017070 con 0x7efcb0004800 2024-05-30T16:44:41.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.006+0000 7efcc5600640 1 -- 172.21.15.67:0/374685845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc003830 con 0x7efcb0004800 2024-05-30T16:44:41.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.006+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc80cc870 con 0x7efcb0004800 2024-05-30T16:44:41.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.006+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc80cce30 con 0x7efcb0004800 2024-05-30T16:44:41.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.007+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc806e670 con 0x7efcb0004800 2024-05-30T16:44:41.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.008+0000 7efcc5600640 1 -- 172.21.15.67:0/374685845 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcbc0039d0 con 0x7efcb0004800 2024-05-30T16:44:41.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.009+0000 7efcc5600640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efca806c920 0x7efca806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.009+0000 7efcc5600640 1 -- 172.21.15.67:0/374685845 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7efcbc08b630 con 0x7efcb0004800 2024-05-30T16:44:41.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.009+0000 7efccd200640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efca806c920 0x7efca806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.010+0000 7efccd200640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efca806c920 0x7efca806ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c210 tx=0x7efcb4034040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:41.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.013+0000 7efcc5600640 1 -- 172.21.15.67:0/374685845 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcc80cd630 con 0x7efcb0004800 2024-05-30T16:44:41.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.294+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7efcc80689e0 con 0x7efcb0004800 2024-05-30T16:44:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.300+0000 7efcc5600640 1 -- 172.21.15.67:0/374685845 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7efcbc059dd0 con 0x7efcb0004800 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.302+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efca806c920 msgr2=0x7efca806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.302+0000 7efccf000640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efca806c920 0x7efca806ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c210 tx=0x7efcb4034040 comp rx=0 tx=0).stop 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.302+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 msgr2=0x7efcc81f1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.302+0000 7efccf000640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcc81f1a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00d4b0 tx=0x7efcbc00d970 comp rx=0 tx=0).stop 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.303+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 shutdown_connections 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.303+0000 7efccf000640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efca806c920 0x7efca806ede0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.303+0000 7efccf000640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcc81f47a0 0x7efcc81f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.303+0000 7efccf000640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcc806dcb0 0x7efcc806e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.303+0000 7efccf000640 1 --2- 172.21.15.67:0/374685845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb0004800 0x7efcc81f1a50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.303+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 >> 172.21.15.67:0/374685845 conn(0x7efcc808bfc0 msgr2=0x7efcc80d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.303+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 shutdown_connections 2024-05-30T16:44:41.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.303+0000 7efccf000640 1 -- 172.21.15.67:0/374685845 wait complete. 2024-05-30T16:44:41.323 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-30T16:44:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.532+0000 7fd580600640 1 Processor -- start 2024-05-30T16:44:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.533+0000 7fd580600640 1 -- start start 2024-05-30T16:44:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.534+0000 7fd580600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5780d04a0 0x7fd5780d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.534+0000 7fd580600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57806ed10 con 0x7fd57806f860 2024-05-30T16:44:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.534+0000 7fd580600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57815cb00 con 0x7fd5780ad130 2024-05-30T16:44:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.534+0000 7fd580600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd5780b3cc0 con 0x7fd5780d04a0 2024-05-30T16:44:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.534+0000 7fd57d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5780d04a0 0x7fd5780d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.534+0000 7fd57dc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd57806f860 0x7fd5780b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34606/0 (socket says 172.21.15.67:34606) 2024-05-30T16:44:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.534+0000 7fd57dc00640 1 -- 172.21.15.67:0/2015782582 learned_addr learned my addr 172.21.15.67:0/2015782582 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.535+0000 7fd57d200640 1 -- 172.21.15.67:0/2015782582 >> v1:172.21.15.169:6789/0 conn(0x7fd5780ad130 legacy=0x7fd5780ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.535+0000 7fd57d200640 1 -- 172.21.15.67:0/2015782582 >> v1:172.21.15.67:6789/0 conn(0x7fd57806f860 legacy=0x7fd5780b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.535+0000 7fd57d200640 1 -- 172.21.15.67:0/2015782582 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5780b3fa0 con 0x7fd5780d04a0 2024-05-30T16:44:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.536+0000 7fd57d200640 1 --2- 172.21.15.67:0/2015782582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780d04a0 0x7fd5780d0930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd568009d90 tx=0x7fd56802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c31cfcda3de722 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.536+0000 7fd577e00640 1 -- 172.21.15.67:0/2015782582 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568007740 con 0x7fd5780d04a0 2024-05-30T16:44:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.536+0000 7fd577e00640 1 -- 172.21.15.67:0/2015782582 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5680078e0 con 0x7fd5780d04a0 2024-05-30T16:44:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.536+0000 7fd577e00640 1 -- 172.21.15.67:0/2015782582 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568007cb0 con 0x7fd5780d04a0 2024-05-30T16:44:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.537+0000 7fd580600640 1 -- 172.21.15.67:0/2015782582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780d04a0 msgr2=0x7fd5780d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.537+0000 7fd580600640 1 --2- 172.21.15.67:0/2015782582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780d04a0 0x7fd5780d0930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd568009d90 tx=0x7fd56802a6a0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.537+0000 7fd580600640 1 -- 172.21.15.67:0/2015782582 shutdown_connections 2024-05-30T16:44:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.537+0000 7fd580600640 1 --2- 172.21.15.67:0/2015782582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780d04a0 0x7fd5780d0930 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.537+0000 7fd580600640 1 -- 172.21.15.67:0/2015782582 >> 172.21.15.67:0/2015782582 conn(0x7fd57806bae0 msgr2=0x7fd57806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.537+0000 7fd580600640 1 -- 172.21.15.67:0/2015782582 shutdown_connections 2024-05-30T16:44:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.537+0000 7fd580600640 1 -- 172.21.15.67:0/2015782582 wait complete. 2024-05-30T16:44:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.539+0000 7fd580600640 1 Processor -- start 2024-05-30T16:44:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.539+0000 7fd580600640 1 -- start start 2024-05-30T16:44:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.540+0000 7fd580600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57806f860 0x7fd578157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.540+0000 7fd580600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780ad130 0x7fd57815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.540+0000 7fd580600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780d04a0 0x7fd5780cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.540+0000 7fd580600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780b4840 con 0x7fd5780ad130 2024-05-30T16:44:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.540+0000 7fd580600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780b4560 con 0x7fd5780d04a0 2024-05-30T16:44:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.540+0000 7fd580600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5780b3860 con 0x7fd57806f860 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780ad130 0x7fd57815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780ad130 0x7fd57815d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50962/0 (socket says 172.21.15.67:50962) 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57d200640 1 -- 172.21.15.67:0/3798407830 learned_addr learned my addr 172.21.15.67:0/3798407830 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57e600640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780d04a0 0x7fd5780cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57dc00640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57806f860 0x7fd578157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57d200640 1 -- 172.21.15.67:0/3798407830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57806f860 msgr2=0x7fd578157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57d200640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57806f860 0x7fd578157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57d200640 1 -- 172.21.15.67:0/3798407830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780d04a0 msgr2=0x7fd5780cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57d200640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780d04a0 0x7fd5780cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.541+0000 7fd57d200640 1 -- 172.21.15.67:0/3798407830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57815e1c0 con 0x7fd5780ad130 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.542+0000 7fd57d200640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780ad130 0x7fd57815d6d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd568009d90 tx=0x7fd568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.542+0000 7fd575600640 1 -- 172.21.15.67:0/3798407830 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56803f070 con 0x7fd5780ad130 2024-05-30T16:44:41.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.542+0000 7fd575600640 1 -- 172.21.15.67:0/3798407830 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd568031d00 con 0x7fd5780ad130 2024-05-30T16:44:41.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.542+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57815e4a0 con 0x7fd5780ad130 2024-05-30T16:44:41.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.542+0000 7fd575600640 1 -- 172.21.15.67:0/3798407830 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568044660 con 0x7fd5780ad130 2024-05-30T16:44:41.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.542+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57815ea60 con 0x7fd5780ad130 2024-05-30T16:44:41.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.543+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5780cb060 con 0x7fd5780ad130 2024-05-30T16:44:41.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.544+0000 7fd575600640 1 -- 172.21.15.67:0/3798407830 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd56803b410 con 0x7fd5780ad130 2024-05-30T16:44:41.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.546+0000 7fd575600640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd55c06c920 0x7fd55c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:41.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.546+0000 7fd575600640 1 -- 172.21.15.67:0/3798407830 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fd5680ad200 con 0x7fd5780ad130 2024-05-30T16:44:41.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.546+0000 7fd57dc00640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd55c06c920 0x7fd55c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:41.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.547+0000 7fd57dc00640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd55c06c920 0x7fd55c06ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd56c000f60 tx=0x7fd56c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:41.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.551+0000 7fd575600640 1 -- 172.21.15.67:0/3798407830 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd5780cb060 con 0x7fd5780ad130 2024-05-30T16:44:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.830+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fd5780689e0 con 0x7fd5780ad130 2024-05-30T16:44:41.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.835+0000 7fd575600640 1 -- 172.21.15.67:0/3798407830 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fd56807b290 con 0x7fd5780ad130 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.837+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd55c06c920 msgr2=0x7fd55c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd55c06c920 0x7fd55c06ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd56c000f60 tx=0x7fd56c009040 comp rx=0 tx=0).stop 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780ad130 msgr2=0x7fd57815d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780ad130 0x7fd57815d6d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd568009d90 tx=0x7fd568004a40 comp rx=0 tx=0).stop 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 shutdown_connections 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd55c06c920 0x7fd55c06ede0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780d04a0 0x7fd5780cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5780ad130 0x7fd57815d6d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 --2- 172.21.15.67:0/3798407830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57806f860 0x7fd578157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 >> 172.21.15.67:0/3798407830 conn(0x7fd57806bae0 msgr2=0x7fd578073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 shutdown_connections 2024-05-30T16:44:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:41.838+0000 7fd580600640 1 -- 172.21.15.67:0/3798407830 wait complete. 2024-05-30T16:44:41.855 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-30T16:44:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.056+0000 7f46bfe00640 1 Processor -- start 2024-05-30T16:44:42.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.056+0000 7f46bfe00640 1 -- start start 2024-05-30T16:44:42.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.058+0000 7f46bfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f46c0156b30 0x7f46c00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.058+0000 7f46bfe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c0156f10 con 0x7f46c00ad940 2024-05-30T16:44:42.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.058+0000 7f46bfe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c006dda0 con 0x7f46c006ee40 2024-05-30T16:44:42.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.058+0000 7f46bfe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f46c006e080 con 0x7f46c0156b30 2024-05-30T16:44:42.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.058+0000 7f46bf400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f46c00ad940 0x7f46c00b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34614/0 (socket says 172.21.15.67:34614) 2024-05-30T16:44:42.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.058+0000 7f46bf400640 1 -- 172.21.15.67:0/3294753719 learned_addr learned my addr 172.21.15.67:0/3294753719 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bea00640 1 --2- 172.21.15.67:0/3294753719 >> v2:172.21.15.169:3300/0 conn(0x7f46c0156b30 0x7f46c00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531253761 0 0) 0x7f46c0156f10 con 0x7f46c00ad940 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46a4003520 con 0x7f46c00ad940 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 788517860 0 0) 0x7f46a4003520 con 0x7f46c00ad940 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 >> v2:172.21.15.169:3300/0 conn(0x7f46c0156b30 msgr2=0x7f46c00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 --2- 172.21.15.67:0/3294753719 >> v2:172.21.15.169:3300/0 conn(0x7f46c0156b30 0x7f46c00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 >> v1:172.21.15.169:6789/0 conn(0x7f46c006ee40 legacy=0x7f46c00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c006e360 con 0x7f46c00ad940 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f46b80019f0 con 0x7f46c00ad940 2024-05-30T16:44:42.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 >> v1:172.21.15.67:6789/0 conn(0x7f46c00ad940 legacy=0x7f46c00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 --2- 172.21.15.67:0/3294753719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a40039b0 0x7f46a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 --2- 172.21.15.67:0/3294753719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46a4004800 0x7f46a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 --2- 172.21.15.67:0/3294753719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a40051c0 0x7f46a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46c006e080 con 0x7f46a40039b0 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46a400c160 con 0x7f46a40051c0 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.059+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f46a400c360 con 0x7f46a4004800 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.060+0000 7f46bea00640 1 --2- 172.21.15.67:0/3294753719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a40039b0 0x7f46a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.060+0000 7f46bea00640 1 -- 172.21.15.67:0/3294753719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46a4004800 msgr2=0x7f46a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.060+0000 7f46bea00640 1 --2- 172.21.15.67:0/3294753719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46a4004800 0x7f46a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.060+0000 7f46bea00640 1 -- 172.21.15.67:0/3294753719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a40051c0 msgr2=0x7f46a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.060+0000 7f46bea00640 1 --2- 172.21.15.67:0/3294753719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a40051c0 0x7f46a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.060+0000 7f46bea00640 1 -- 172.21.15.67:0/3294753719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a4003520 con 0x7f46a40039b0 2024-05-30T16:44:42.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.060+0000 7f46bf400640 1 -- 172.21.15.67:0/3294753719 reap_dead start 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.061+0000 7f46bea00640 1 --2- 172.21.15.67:0/3294753719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a40039b0 0x7f46a4003dd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009ac0 tx=0x7f46a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14153d7c2c9a9018 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.062+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0036070 con 0x7f46a40039b0 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.062+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b0031400 con 0x7f46a40039b0 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.062+0000 7f46bd600640 1 -- 172.21.15.67:0/3294753719 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0009690 con 0x7f46a40039b0 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.062+0000 7f46bfe00640 1 -- 172.21.15.67:0/3294753719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a40039b0 msgr2=0x7f46a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.062+0000 7f46bfe00640 1 --2- 172.21.15.67:0/3294753719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a40039b0 0x7f46a4003dd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009ac0 tx=0x7f46a4005830 comp rx=0 tx=0).stop 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.062+0000 7f46bfe00640 1 -- 172.21.15.67:0/3294753719 shutdown_connections 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.062+0000 7f46bfe00640 1 --2- 172.21.15.67:0/3294753719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a40039b0 0x7f46a4003dd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.062+0000 7f46bfe00640 1 -- 172.21.15.67:0/3294753719 >> 172.21.15.67:0/3294753719 conn(0x7f46c00b1fc0 msgr2=0x7f46c00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.063+0000 7f46bfe00640 1 -- 172.21.15.67:0/3294753719 shutdown_connections 2024-05-30T16:44:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.063+0000 7f46bfe00640 1 -- 172.21.15.67:0/3294753719 wait complete. 2024-05-30T16:44:42.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.064+0000 7f46bfe00640 1 Processor -- start 2024-05-30T16:44:42.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.064+0000 7f46bfe00640 1 -- start start 2024-05-30T16:44:42.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.065+0000 7f46bfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a40039b0 0x7f46c00b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.065+0000 7f46bfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46c01f4440 0x7f46c0073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.065+0000 7f46bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c014a690 0x7f46c014ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.065+0000 7f46bfe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46c00b2db0 con 0x7f46c014a690 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.065+0000 7f46bfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46c00b2c30 con 0x7f46a40039b0 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.065+0000 7f46bfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f46c006f2f0 con 0x7f46c01f4440 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c014a690 0x7f46c014ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c014a690 0x7f46c014ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50984/0 (socket says 172.21.15.67:50984) 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46bf400640 1 -- 172.21.15.67:0/282314656 learned_addr learned my addr 172.21.15.67:0/282314656 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46be000640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46c01f4440 0x7f46c0073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46bea00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a40039b0 0x7f46c00b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46bf400640 1 -- 172.21.15.67:0/282314656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46c01f4440 msgr2=0x7f46c0073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46bf400640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46c01f4440 0x7f46c0073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46bf400640 1 -- 172.21.15.67:0/282314656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a40039b0 msgr2=0x7f46c00b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46bf400640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a40039b0 0x7f46c00b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.066+0000 7f46bf400640 1 -- 172.21.15.67:0/282314656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c0072770 con 0x7f46c014a690 2024-05-30T16:44:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.067+0000 7f46bf400640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c014a690 0x7f46c014ab10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f46b80012e0 tx=0x7f46b800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.067+0000 7f46b6a00640 1 -- 172.21.15.67:0/282314656 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800ce50 con 0x7f46c014a690 2024-05-30T16:44:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.067+0000 7f46b6a00640 1 -- 172.21.15.67:0/282314656 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b800a5b0 con 0x7f46c014a690 2024-05-30T16:44:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.067+0000 7f46b6a00640 1 -- 172.21.15.67:0/282314656 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800a980 con 0x7f46c014a690 2024-05-30T16:44:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.067+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c0072a50 con 0x7f46c014a690 2024-05-30T16:44:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.067+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c0073010 con 0x7f46c014a690 2024-05-30T16:44:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.068+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c0073210 con 0x7f46c014a690 2024-05-30T16:44:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.069+0000 7f46b6a00640 1 -- 172.21.15.67:0/282314656 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46b8011070 con 0x7f46c014a690 2024-05-30T16:44:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.070+0000 7f46b6a00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46a006c920 0x7f46a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.070+0000 7f46b6a00640 1 -- 172.21.15.67:0/282314656 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f46b808add0 con 0x7f46c014a690 2024-05-30T16:44:42.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.070+0000 7f46bea00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46a006c920 0x7f46a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.071+0000 7f46bea00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46a006c920 0x7f46a006ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f46b0033000 tx=0x7f46b0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:42.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.075+0000 7f46b6a00640 1 -- 172.21.15.67:0/282314656 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46c0073210 con 0x7f46c014a690 2024-05-30T16:44:42.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.357+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f46c00689e0 con 0x7f46c014a690 2024-05-30T16:44:42.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.360+0000 7f46b6a00640 1 -- 172.21.15.67:0/282314656 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f46b80594b0 con 0x7f46c014a690 2024-05-30T16:44:42.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46a006c920 msgr2=0x7f46a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46a006c920 0x7f46a006ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f46b0033000 tx=0x7f46b0008030 comp rx=0 tx=0).stop 2024-05-30T16:44:42.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c014a690 msgr2=0x7f46c014ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c014a690 0x7f46c014ab10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f46b80012e0 tx=0x7f46b800cc00 comp rx=0 tx=0).stop 2024-05-30T16:44:42.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 shutdown_connections 2024-05-30T16:44:42.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46a006c920 0x7f46a006ede0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46c014a690 0x7f46c014ab10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46c01f4440 0x7f46c0073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 --2- 172.21.15.67:0/282314656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46a40039b0 0x7f46c00b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 >> 172.21.15.67:0/282314656 conn(0x7f46c00b1fc0 msgr2=0x7f46c00b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:42.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 shutdown_connections 2024-05-30T16:44:42.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.363+0000 7f46bfe00640 1 -- 172.21.15.67:0/282314656 wait complete. 2024-05-30T16:44:42.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-30T16:44:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.587+0000 7fea97000640 1 Processor -- start 2024-05-30T16:44:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.587+0000 7fea97000640 1 -- start start 2024-05-30T16:44:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.589+0000 7fea97000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea90073310 0x7fea900b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.589+0000 7fea97000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fea9006dc20 0x7fea9015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.589+0000 7fea97000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea900afd60 con 0x7fea9015b3a0 2024-05-30T16:44:42.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.589+0000 7fea97000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fea9006f980 con 0x7fea9006dc20 2024-05-30T16:44:42.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.589+0000 7fea97000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fea900ae0c0 con 0x7fea90073310 2024-05-30T16:44:42.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.590+0000 7fea96600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fea9006dc20 0x7fea9015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.590+0000 7fea96600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fea9006dc20 0x7fea9015f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:35614/0 (socket says 172.21.15.67:35614) 2024-05-30T16:44:42.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.590+0000 7fea95200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea90073310 0x7fea900b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.590+0000 7fea96600640 1 -- 172.21.15.67:0/164448856 learned_addr learned my addr 172.21.15.67:0/164448856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:42.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.591+0000 7fea96600640 1 -- 172.21.15.67:0/164448856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea90073310 msgr2=0x7fea900b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:42.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.591+0000 7fea96600640 1 --2- 172.21.15.67:0/164448856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea90073310 0x7fea900b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.591+0000 7fea96600640 1 -- 172.21.15.67:0/164448856 >> v1:172.21.15.169:6789/0 conn(0x7fea9015b3a0 legacy=0x7fea90072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.591+0000 7fea96600640 1 -- 172.21.15.67:0/164448856 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea900af7a0 con 0x7fea9006dc20 2024-05-30T16:44:42.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.592+0000 7fea96600640 1 --2- 172.21.15.67:0/164448856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea9006dc20 0x7fea9015f9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea88009e50 tx=0x7fea8802c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf4961c078624a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:42.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea8fe00640 1 -- 172.21.15.67:0/164448856 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88032610 con 0x7fea9006dc20 2024-05-30T16:44:42.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea8fe00640 1 -- 172.21.15.67:0/164448856 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea88037070 con 0x7fea9006dc20 2024-05-30T16:44:42.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea8fe00640 1 -- 172.21.15.67:0/164448856 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880329d0 con 0x7fea9006dc20 2024-05-30T16:44:42.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea97000640 1 -- 172.21.15.67:0/164448856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea9006dc20 msgr2=0x7fea9015f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea97000640 1 --2- 172.21.15.67:0/164448856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea9006dc20 0x7fea9015f9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea88009e50 tx=0x7fea8802c920 comp rx=0 tx=0).stop 2024-05-30T16:44:42.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea97000640 1 -- 172.21.15.67:0/164448856 shutdown_connections 2024-05-30T16:44:42.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea97000640 1 --2- 172.21.15.67:0/164448856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea9006dc20 0x7fea9015f9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea97000640 1 --2- 172.21.15.67:0/164448856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea90073310 0x7fea900b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea97000640 1 -- 172.21.15.67:0/164448856 >> 172.21.15.67:0/164448856 conn(0x7fea900b1fc0 msgr2=0x7fea900b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:42.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.593+0000 7fea97000640 1 -- 172.21.15.67:0/164448856 shutdown_connections 2024-05-30T16:44:42.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.594+0000 7fea97000640 1 -- 172.21.15.67:0/164448856 wait complete. 2024-05-30T16:44:42.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.595+0000 7fea97000640 1 Processor -- start 2024-05-30T16:44:42.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.595+0000 7fea97000640 1 -- start start 2024-05-30T16:44:42.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.596+0000 7fea97000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006dc20 0x7fea901574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.596+0000 7fea97000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90073310 0x7fea9014b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.596+0000 7fea97000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea9015b3a0 0x7fea900cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.596+0000 7fea97000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fea9015b780 con 0x7fea9015b3a0 2024-05-30T16:44:42.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.596+0000 7fea97000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea9015fee0 con 0x7fea90073310 2024-05-30T16:44:42.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.596+0000 7fea97000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fea900ae550 con 0x7fea9006dc20 2024-05-30T16:44:42.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea96600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea9015b3a0 0x7fea900cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea96600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea9015b3a0 0x7fea900cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50996/0 (socket says 172.21.15.67:50996) 2024-05-30T16:44:42.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea96600640 1 -- 172.21.15.67:0/2291105858 learned_addr learned my addr 172.21.15.67:0/2291105858 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:42.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea95200640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90073310 0x7fea9014b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea95c00640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006dc20 0x7fea901574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea96600640 1 -- 172.21.15.67:0/2291105858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006dc20 msgr2=0x7fea901574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea96600640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006dc20 0x7fea901574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea96600640 1 -- 172.21.15.67:0/2291105858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90073310 msgr2=0x7fea9014b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea96600640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90073310 0x7fea9014b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.597+0000 7fea96600640 1 -- 172.21.15.67:0/2291105858 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea90157ab0 con 0x7fea9015b3a0 2024-05-30T16:44:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.598+0000 7fea96600640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea9015b3a0 0x7fea900cb650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fea88009e50 tx=0x7fea88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.598+0000 7fea8d600640 1 -- 172.21.15.67:0/2291105858 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88041070 con 0x7fea9015b3a0 2024-05-30T16:44:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.598+0000 7fea8d600640 1 -- 172.21.15.67:0/2291105858 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8803b620 con 0x7fea9015b3a0 2024-05-30T16:44:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.598+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea90157d90 con 0x7fea9015b3a0 2024-05-30T16:44:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.598+0000 7fea8d600640 1 -- 172.21.15.67:0/2291105858 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88009510 con 0x7fea9015b3a0 2024-05-30T16:44:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.598+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea90158350 con 0x7fea9015b3a0 2024-05-30T16:44:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.600+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea900689e0 con 0x7fea9015b3a0 2024-05-30T16:44:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.601+0000 7fea8d600640 1 -- 172.21.15.67:0/2291105858 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea88005d10 con 0x7fea9015b3a0 2024-05-30T16:44:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.602+0000 7fea8d600640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea7406c8f0 0x7fea7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.602+0000 7fea8d600640 1 -- 172.21.15.67:0/2291105858 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fea880abda0 con 0x7fea9015b3a0 2024-05-30T16:44:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.602+0000 7fea95c00640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea7406c8f0 0x7fea7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.603+0000 7fea95c00640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea7406c8f0 0x7fea7406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fea84001500 tx=0x7fea84009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.606+0000 7fea8d600640 1 -- 172.21.15.67:0/2291105858 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea8804b070 con 0x7fea9015b3a0 2024-05-30T16:44:42.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.813+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fea900b0c60 con 0x7fea9015b3a0 2024-05-30T16:44:42.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.814+0000 7fea8d600640 1 -- 172.21.15.67:0/2291105858 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7fea8802f070 con 0x7fea9015b3a0 2024-05-30T16:44:42.815 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:42.815 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:44:42.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:44:42.815 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-05-30T16:44:42.815 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-05-30T16:44:42.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:42.815 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:44:42.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea7406c8f0 msgr2=0x7fea7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea7406c8f0 0x7fea7406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fea84001500 tx=0x7fea84009040 comp rx=0 tx=0).stop 2024-05-30T16:44:42.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea9015b3a0 msgr2=0x7fea900cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:42.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea9015b3a0 0x7fea900cb650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fea88009e50 tx=0x7fea88004a40 comp rx=0 tx=0).stop 2024-05-30T16:44:42.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 shutdown_connections 2024-05-30T16:44:42.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea7406c8f0 0x7fea7406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea9015b3a0 0x7fea900cb650 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90073310 0x7fea9014b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 --2- 172.21.15.67:0/2291105858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006dc20 0x7fea901574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 >> 172.21.15.67:0/2291105858 conn(0x7fea900b1fc0 msgr2=0x7fea900b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 shutdown_connections 2024-05-30T16:44:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:42.817+0000 7fea97000640 1 -- 172.21.15.67:0/2291105858 wait complete. 2024-05-30T16:44:42.837 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-30T16:44:42.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-30T16:44:43.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.047+0000 7f9437200640 1 Processor -- start 2024-05-30T16:44:43.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.047+0000 7f9437200640 1 -- start start 2024-05-30T16:44:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.049+0000 7f9437200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94381621d0 0x7f94380d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.049+0000 7f9437200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94380cf180 0x7f94380cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.049+0000 7f9437200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94380d0e50 con 0x7f94381625b0 2024-05-30T16:44:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.049+0000 7f9437200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f94380d1a60 con 0x7f94381621d0 2024-05-30T16:44:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.049+0000 7f9437200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f94380d1130 con 0x7f94380cf180 2024-05-30T16:44:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.049+0000 7f9435e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94381621d0 0x7f94380d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.049+0000 7f9435e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94381621d0 0x7f94380d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51002/0 (socket says 172.21.15.67:51002) 2024-05-30T16:44:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.049+0000 7f9435e00640 1 -- 172.21.15.67:0/1511544341 learned_addr learned my addr 172.21.15.67:0/1511544341 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.050+0000 7f9435e00640 1 -- 172.21.15.67:0/1511544341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94380cf180 msgr2=0x7f94380cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.050+0000 7f9435e00640 1 --2- 172.21.15.67:0/1511544341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94380cf180 0x7f94380cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.050+0000 7f9435e00640 1 -- 172.21.15.67:0/1511544341 >> v1:172.21.15.169:6789/0 conn(0x7f94381625b0 legacy=0x7f94380d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.050+0000 7f9435e00640 1 -- 172.21.15.67:0/1511544341 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94380d1410 con 0x7f94381621d0 2024-05-30T16:44:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.051+0000 7f9435e00640 1 --2- 172.21.15.67:0/1511544341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 0x7f94380d3220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9428009870 tx=0x7f942802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bada8165d38853c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:43.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.051+0000 7f9434a00640 1 -- 172.21.15.67:0/1511544341 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428037070 con 0x7f94381621d0 2024-05-30T16:44:43.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.051+0000 7f9434a00640 1 -- 172.21.15.67:0/1511544341 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9428032440 con 0x7f94381621d0 2024-05-30T16:44:43.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.051+0000 7f9434a00640 1 -- 172.21.15.67:0/1511544341 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428032810 con 0x7f94381621d0 2024-05-30T16:44:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.052+0000 7f9437200640 1 -- 172.21.15.67:0/1511544341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 msgr2=0x7f94380d3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.052+0000 7f9437200640 1 --2- 172.21.15.67:0/1511544341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 0x7f94380d3220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9428009870 tx=0x7f942802a210 comp rx=0 tx=0).stop 2024-05-30T16:44:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.052+0000 7f9437200640 1 -- 172.21.15.67:0/1511544341 shutdown_connections 2024-05-30T16:44:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.052+0000 7f9437200640 1 --2- 172.21.15.67:0/1511544341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94380cf180 0x7f94380cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.052+0000 7f9437200640 1 --2- 172.21.15.67:0/1511544341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 0x7f94380d3220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.052+0000 7f9437200640 1 -- 172.21.15.67:0/1511544341 >> 172.21.15.67:0/1511544341 conn(0x7f943808bfc0 msgr2=0x7f94380d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.052+0000 7f9437200640 1 -- 172.21.15.67:0/1511544341 shutdown_connections 2024-05-30T16:44:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.053+0000 7f9437200640 1 -- 172.21.15.67:0/1511544341 wait complete. 2024-05-30T16:44:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.055+0000 7f9437200640 1 Processor -- start 2024-05-30T16:44:43.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.055+0000 7f9437200640 1 -- start start 2024-05-30T16:44:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.056+0000 7f9437200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94380cf180 0x7f94381f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.056+0000 7f9437200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 0x7f94381f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.056+0000 7f9437200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94381625b0 0x7f94380b9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.056+0000 7f9437200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9438162a40 con 0x7f94381621d0 2024-05-30T16:44:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.056+0000 7f9437200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94380d1ef0 con 0x7f94381625b0 2024-05-30T16:44:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.056+0000 7f9437200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f943815fa20 con 0x7f94380cf180 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9435400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 0x7f94381f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9435400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 0x7f94381f4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51006/0 (socket says 172.21.15.67:51006) 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9435400640 1 -- 172.21.15.67:0/509664347 learned_addr learned my addr 172.21.15.67:0/509664347 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9435e00640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94380cf180 0x7f94381f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9436800640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94381625b0 0x7f94380b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9435400640 1 -- 172.21.15.67:0/509664347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94380cf180 msgr2=0x7f94381f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9435400640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94380cf180 0x7f94381f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9435400640 1 -- 172.21.15.67:0/509664347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94381625b0 msgr2=0x7f94380b9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9435400640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94381625b0 0x7f94380b9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9435400640 1 -- 172.21.15.67:0/509664347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94381f8dc0 con 0x7f94381621d0 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.057+0000 7f9436800640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94381625b0 0x7f94380b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.058+0000 7f9435400640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 0x7f94381f4990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f942400b3f0 tx=0x7f942400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.058+0000 7f942e000640 1 -- 172.21.15.67:0/509664347 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424015030 con 0x7f94381621d0 2024-05-30T16:44:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.058+0000 7f942e000640 1 -- 172.21.15.67:0/509664347 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9424005920 con 0x7f94381621d0 2024-05-30T16:44:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.058+0000 7f942e000640 1 -- 172.21.15.67:0/509664347 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424005cf0 con 0x7f94381621d0 2024-05-30T16:44:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.058+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94380b72a0 con 0x7f94381621d0 2024-05-30T16:44:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.058+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94380b7860 con 0x7f94381621d0 2024-05-30T16:44:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.059+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94380689e0 con 0x7f94381621d0 2024-05-30T16:44:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.060+0000 7f942e000640 1 -- 172.21.15.67:0/509664347 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9424018070 con 0x7f94381621d0 2024-05-30T16:44:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.062+0000 7f942e000640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f941806c8f0 0x7f941806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.062+0000 7f942e000640 1 -- 172.21.15.67:0/509664347 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f942408b420 con 0x7f94381621d0 2024-05-30T16:44:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.062+0000 7f9435e00640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f941806c8f0 0x7f941806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.063+0000 7f9435e00640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f941806c8f0 0x7f941806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9428034000 tx=0x7f942802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.066+0000 7f942e000640 1 -- 172.21.15.67:0/509664347 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9424059b00 con 0x7f94381621d0 2024-05-30T16:44:43.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.271+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f94380b5ac0 con 0x7f94381621d0 2024-05-30T16:44:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.843+0000 7f942e000640 1 -- 172.21.15.67:0/509664347 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7f9424059400 con 0x7f94381621d0 2024-05-30T16:44:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-05-30T16:44:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f941806c8f0 msgr2=0x7f941806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f941806c8f0 0x7f941806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9428034000 tx=0x7f942802e000 comp rx=0 tx=0).stop 2024-05-30T16:44:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 msgr2=0x7f94381f4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 0x7f94381f4990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f942400b3f0 tx=0x7f942400b8b0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 shutdown_connections 2024-05-30T16:44:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f941806c8f0 0x7f941806edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94381625b0 0x7f94380b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94381621d0 0x7f94381f4990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 --2- 172.21.15.67:0/509664347 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94380cf180 0x7f94381f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.847+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 >> 172.21.15.67:0/509664347 conn(0x7f943808bfc0 msgr2=0x7f94380cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.848+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 shutdown_connections 2024-05-30T16:44:43.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:43.849+0000 7f9437200640 1 -- 172.21.15.67:0/509664347 wait complete. 2024-05-30T16:44:43.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-30T16:44:44.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.079+0000 7ff88c600640 1 Processor -- start 2024-05-30T16:44:44.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.080+0000 7ff88c600640 1 -- start start 2024-05-30T16:44:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.081+0000 7ff88c600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff88415d5d0 0x7ff8840cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.082+0000 7ff88c600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88415cb10 con 0x7ff8840b32c0 2024-05-30T16:44:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.082+0000 7ff88c600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8840ad5f0 con 0x7ff8840d0640 2024-05-30T16:44:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.082+0000 7ff88c600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff8840adbb0 con 0x7ff88415d5d0 2024-05-30T16:44:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.082+0000 7ff88a600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff8840b32c0 0x7ff8840b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34618/0 (socket says 172.21.15.67:34618) 2024-05-30T16:44:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.082+0000 7ff88a600640 1 -- 172.21.15.67:0/975751408 learned_addr learned my addr 172.21.15.67:0/975751408 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.082+0000 7ff889200640 1 --2- 172.21.15.67:0/975751408 >> v2:172.21.15.169:3300/0 conn(0x7ff88415d5d0 0x7ff8840cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2261503937 0 0) 0x7ff88415cb10 con 0x7ff8840b32c0 2024-05-30T16:44:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff868003520 con 0x7ff8840b32c0 2024-05-30T16:44:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 981502357 0 0) 0x7ff868003520 con 0x7ff8840b32c0 2024-05-30T16:44:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 >> v2:172.21.15.169:3300/0 conn(0x7ff88415d5d0 msgr2=0x7ff8840cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 --2- 172.21.15.67:0/975751408 >> v2:172.21.15.169:3300/0 conn(0x7ff88415d5d0 0x7ff8840cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 >> v1:172.21.15.169:6789/0 conn(0x7ff8840d0640 legacy=0x7ff8840d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8840ad8d0 con 0x7ff8840b32c0 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff87c0019f0 con 0x7ff8840b32c0 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 >> v1:172.21.15.67:6789/0 conn(0x7ff8840b32c0 legacy=0x7ff8840b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8680039b0 0x7ff868003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff868004800 0x7ff868004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8680051c0 0x7ff86800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff86800c130 con 0x7ff8680039b0 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff86800c2e0 con 0x7ff8680051c0 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.083+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8680108f0 con 0x7ff868004800 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff889c00640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8680039b0 0x7ff868003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff88a600640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8680051c0 0x7ff86800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff889200640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff868004800 0x7ff868004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff889c00640 1 -- 172.21.15.67:0/975751408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff868004800 msgr2=0x7ff868004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff889c00640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff868004800 0x7ff868004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff889c00640 1 -- 172.21.15.67:0/975751408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8680051c0 msgr2=0x7ff86800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff889c00640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8680051c0 0x7ff86800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff889c00640 1 -- 172.21.15.67:0/975751408 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff868003520 con 0x7ff8680039b0 2024-05-30T16:44:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff88a600640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8680051c0 0x7ff86800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.084+0000 7ff88a600640 1 -- 172.21.15.67:0/975751408 reap_dead start 2024-05-30T16:44:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.085+0000 7ff889c00640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8680039b0 0x7ff868003dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff874001750 tx=0x7ff868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8001e235f74e8f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.085+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874001370 con 0x7ff8680039b0 2024-05-30T16:44:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.085+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8740052d0 con 0x7ff8680039b0 2024-05-30T16:44:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.086+0000 7ff883e00640 1 -- 172.21.15.67:0/975751408 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87402abc0 con 0x7ff8680039b0 2024-05-30T16:44:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.086+0000 7ff88c600640 1 -- 172.21.15.67:0/975751408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8680039b0 msgr2=0x7ff868003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.086+0000 7ff88c600640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8680039b0 0x7ff868003dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff874001750 tx=0x7ff868005830 comp rx=0 tx=0).stop 2024-05-30T16:44:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.086+0000 7ff88c600640 1 -- 172.21.15.67:0/975751408 shutdown_connections 2024-05-30T16:44:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.086+0000 7ff88c600640 1 --2- 172.21.15.67:0/975751408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8680039b0 0x7ff868003dd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.086+0000 7ff88c600640 1 -- 172.21.15.67:0/975751408 >> 172.21.15.67:0/975751408 conn(0x7ff88406bd20 msgr2=0x7ff88406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.086+0000 7ff88c600640 1 -- 172.21.15.67:0/975751408 shutdown_connections 2024-05-30T16:44:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.086+0000 7ff88c600640 1 -- 172.21.15.67:0/975751408 wait complete. 2024-05-30T16:44:44.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.088+0000 7ff88c600640 1 Processor -- start 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.088+0000 7ff88c600640 1 -- start start 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.089+0000 7ff88c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8680039b0 0x7ff88414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.089+0000 7ff88c600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff884145520 0x7ff8840b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.089+0000 7ff88c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff884099910 0x7ff884137c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.089+0000 7ff88c600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff88415da50 con 0x7ff884145520 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.089+0000 7ff88c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff88415d0b0 con 0x7ff884099910 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.089+0000 7ff88c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8840ad2a0 con 0x7ff8680039b0 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.089+0000 7ff889200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff884145520 0x7ff8840b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff889200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff884145520 0x7ff8840b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51028/0 (socket says 172.21.15.67:51028) 2024-05-30T16:44:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff889200640 1 -- 172.21.15.67:0/1101415766 learned_addr learned my addr 172.21.15.67:0/1101415766 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff88a600640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff884099910 0x7ff884137c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff889c00640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8680039b0 0x7ff88414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff889200640 1 -- 172.21.15.67:0/1101415766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8680039b0 msgr2=0x7ff88414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff889200640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8680039b0 0x7ff88414e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff889200640 1 -- 172.21.15.67:0/1101415766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff884099910 msgr2=0x7ff884137c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff889200640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff884099910 0x7ff884137c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff889200640 1 -- 172.21.15.67:0/1101415766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88414dc10 con 0x7ff884145520 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff88a600640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff884099910 0x7ff884137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.090+0000 7ff889200640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff884145520 0x7ff8840b26d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff870010910 tx=0x7ff870010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.091+0000 7ff881600640 1 -- 172.21.15.67:0/1101415766 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87000f040 con 0x7ff884145520 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.091+0000 7ff881600640 1 -- 172.21.15.67:0/1101415766 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87000ec20 con 0x7ff884145520 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.091+0000 7ff881600640 1 -- 172.21.15.67:0/1101415766 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87001c400 con 0x7ff884145520 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.091+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88414def0 con 0x7ff884145520 2024-05-30T16:44:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.091+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88414e4b0 con 0x7ff884145520 2024-05-30T16:44:44.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.092+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8840689e0 con 0x7ff884145520 2024-05-30T16:44:44.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.093+0000 7ff881600640 1 -- 172.21.15.67:0/1101415766 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff87001c5a0 con 0x7ff884145520 2024-05-30T16:44:44.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.094+0000 7ff881600640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff86406c8f0 0x7ff86406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:44.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.094+0000 7ff881600640 1 -- 172.21.15.67:0/1101415766 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7ff87008c090 con 0x7ff884145520 2024-05-30T16:44:44.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.095+0000 7ff889c00640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff86406c8f0 0x7ff86406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:44.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.095+0000 7ff889c00640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff86406c8f0 0x7ff86406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff874032000 tx=0x7ff87402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:44.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.100+0000 7ff881600640 1 -- 172.21.15.67:0/1101415766 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff870018030 con 0x7ff884145520 2024-05-30T16:44:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.304+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff884071ef0 con 0x7ff884145520 2024-05-30T16:44:44.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.850+0000 7ff881600640 1 -- 172.21.15.67:0/1101415766 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7ff87005a600 con 0x7ff884145520 2024-05-30T16:44:44.851 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-05-30T16:44:44.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff86406c8f0 msgr2=0x7ff86406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:44.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff86406c8f0 0x7ff86406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff874032000 tx=0x7ff87402e000 comp rx=0 tx=0).stop 2024-05-30T16:44:44.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff884145520 msgr2=0x7ff8840b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:44.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff884145520 0x7ff8840b26d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff870010910 tx=0x7ff870010dd0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 shutdown_connections 2024-05-30T16:44:44.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff86406c8f0 0x7ff86406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff884099910 0x7ff884137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff884145520 0x7ff8840b26d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 --2- 172.21.15.67:0/1101415766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8680039b0 0x7ff88414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:44.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.854+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 >> 172.21.15.67:0/1101415766 conn(0x7ff88406bd20 msgr2=0x7ff88415e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:44.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.855+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 shutdown_connections 2024-05-30T16:44:44.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:44.855+0000 7ff88c600640 1 -- 172.21.15.67:0/1101415766 wait complete. 2024-05-30T16:44:44.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-30T16:44:45.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.074+0000 7fd93b200640 1 Processor -- start 2024-05-30T16:44:45.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.074+0000 7fd93b200640 1 -- start start 2024-05-30T16:44:45.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.076+0000 7fd93b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd9340ce5d0 0x7fd9340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:45.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.076+0000 7fd93b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 0x7fd9340b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:45.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.076+0000 7fd93b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9340b3580 con 0x7fd9340ce1f0 2024-05-30T16:44:45.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.076+0000 7fd93b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd93415f490 con 0x7fd9340ce5d0 2024-05-30T16:44:45.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.076+0000 7fd93b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd93415f770 con 0x7fd9340b3d60 2024-05-30T16:44:45.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.076+0000 7fd939400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd9340ce5d0 0x7fd9340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:45.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.076+0000 7fd939400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd9340ce5d0 0x7fd9340b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51032/0 (socket says 172.21.15.67:51032) 2024-05-30T16:44:45.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.076+0000 7fd939400640 1 -- 172.21.15.67:0/2148545607 learned_addr learned my addr 172.21.15.67:0/2148545607 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.076+0000 7fd93a800640 1 --2- 172.21.15.67:0/2148545607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 0x7fd9340b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.077+0000 7fd939400640 1 -- 172.21.15.67:0/2148545607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 msgr2=0x7fd9340b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.077+0000 7fd939400640 1 --2- 172.21.15.67:0/2148545607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 0x7fd9340b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.077+0000 7fd939400640 1 -- 172.21.15.67:0/2148545607 >> v1:172.21.15.169:6789/0 conn(0x7fd9340ce1f0 legacy=0x7fd93415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.077+0000 7fd939400640 1 -- 172.21.15.67:0/2148545607 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93415fa50 con 0x7fd9340ce5d0 2024-05-30T16:44:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.078+0000 7fd939400640 1 --2- 172.21.15.67:0/2148545607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 0x7fd9340b4dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd924009870 tx=0x7fd92402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b28ece62b4518c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.078+0000 7fd938a00640 1 -- 172.21.15.67:0/2148545607 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924037070 con 0x7fd9340ce5d0 2024-05-30T16:44:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.078+0000 7fd938a00640 1 -- 172.21.15.67:0/2148545607 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92402ae10 con 0x7fd9340ce5d0 2024-05-30T16:44:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.078+0000 7fd938a00640 1 -- 172.21.15.67:0/2148545607 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924032660 con 0x7fd9340ce5d0 2024-05-30T16:44:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.079+0000 7fd93b200640 1 -- 172.21.15.67:0/2148545607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 msgr2=0x7fd9340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.079+0000 7fd93b200640 1 --2- 172.21.15.67:0/2148545607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 0x7fd9340b4dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd924009870 tx=0x7fd92402a100 comp rx=0 tx=0).stop 2024-05-30T16:44:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.079+0000 7fd93b200640 1 -- 172.21.15.67:0/2148545607 shutdown_connections 2024-05-30T16:44:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.079+0000 7fd93b200640 1 --2- 172.21.15.67:0/2148545607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 0x7fd9340b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.079+0000 7fd93b200640 1 --2- 172.21.15.67:0/2148545607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 0x7fd9340b4dd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.079+0000 7fd93b200640 1 -- 172.21.15.67:0/2148545607 >> 172.21.15.67:0/2148545607 conn(0x7fd93406b900 msgr2=0x7fd93406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.079+0000 7fd93b200640 1 -- 172.21.15.67:0/2148545607 shutdown_connections 2024-05-30T16:44:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.079+0000 7fd93b200640 1 -- 172.21.15.67:0/2148545607 wait complete. 2024-05-30T16:44:45.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.081+0000 7fd93b200640 1 Processor -- start 2024-05-30T16:44:45.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.081+0000 7fd93b200640 1 -- start start 2024-05-30T16:44:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.082+0000 7fd93b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 0x7fd93415a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.082+0000 7fd93b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9340ce1f0 0x7fd934148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.082+0000 7fd93b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 0x7fd93406ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.082+0000 7fd93b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd9340ccd10 con 0x7fd9340ce5d0 2024-05-30T16:44:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.082+0000 7fd93b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd9340ccb90 con 0x7fd9340ce1f0 2024-05-30T16:44:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.082+0000 7fd93b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd9340cce90 con 0x7fd9340b3d60 2024-05-30T16:44:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.082+0000 7fd93a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 0x7fd93406ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.082+0000 7fd93a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 0x7fd93406ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51040/0 (socket says 172.21.15.67:51040) 2024-05-30T16:44:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.082+0000 7fd93a800640 1 -- 172.21.15.67:0/1231051517 learned_addr learned my addr 172.21.15.67:0/1231051517 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.083+0000 7fd939400640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9340ce1f0 0x7fd934148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.083+0000 7fd939e00640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 0x7fd93415a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:45.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.083+0000 7fd93a800640 1 -- 172.21.15.67:0/1231051517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 msgr2=0x7fd93415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:45.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.083+0000 7fd93a800640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 0x7fd93415a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.083+0000 7fd93a800640 1 -- 172.21.15.67:0/1231051517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9340ce1f0 msgr2=0x7fd934148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:45.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.083+0000 7fd93a800640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9340ce1f0 0x7fd934148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.083+0000 7fd93a800640 1 -- 172.21.15.67:0/1231051517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd934159480 con 0x7fd9340ce5d0 2024-05-30T16:44:45.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.083+0000 7fd93a800640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 0x7fd93406ec30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd930007e40 tx=0x7fd93000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:45.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.084+0000 7fd92e000640 1 -- 172.21.15.67:0/1231051517 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930014620 con 0x7fd9340ce5d0 2024-05-30T16:44:45.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.084+0000 7fd92e000640 1 -- 172.21.15.67:0/1231051517 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd930019070 con 0x7fd9340ce5d0 2024-05-30T16:44:45.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.084+0000 7fd92e000640 1 -- 172.21.15.67:0/1231051517 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930014a20 con 0x7fd9340ce5d0 2024-05-30T16:44:45.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.084+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd934159760 con 0x7fd9340ce5d0 2024-05-30T16:44:45.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.084+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd934159d20 con 0x7fd9340ce5d0 2024-05-30T16:44:45.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.085+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9340689e0 con 0x7fd9340ce5d0 2024-05-30T16:44:45.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.086+0000 7fd92e000640 1 -- 172.21.15.67:0/1231051517 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd930014bc0 con 0x7fd9340ce5d0 2024-05-30T16:44:45.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.087+0000 7fd92e000640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd91806c8f0 0x7fd91806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:45.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.087+0000 7fd92e000640 1 -- 172.21.15.67:0/1231051517 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7fd93008c410 con 0x7fd9340ce5d0 2024-05-30T16:44:45.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.087+0000 7fd939e00640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd91806c8f0 0x7fd91806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:45.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.088+0000 7fd939e00640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd91806c8f0 0x7fd91806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd928004ea0 tx=0x7fd928009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:45.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.094+0000 7fd92e000640 1 -- 172.21.15.67:0/1231051517 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd93005a890 con 0x7fd9340ce5d0 2024-05-30T16:44:45.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.337+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd9340ca070 con 0x7fd9340ce5d0 2024-05-30T16:44:45.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.898+0000 7fd92e000640 1 -- 172.21.15.67:0/1231051517 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fd93005a190 con 0x7fd9340ce5d0 2024-05-30T16:44:45.901 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 4 and data pool 5 2024-05-30T16:44:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.907+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd91806c8f0 msgr2=0x7fd91806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.907+0000 7fd93b200640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd91806c8f0 0x7fd91806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd928004ea0 tx=0x7fd928009040 comp rx=0 tx=0).stop 2024-05-30T16:44:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.907+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 msgr2=0x7fd93406ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.907+0000 7fd93b200640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 0x7fd93406ec30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd930007e40 tx=0x7fd93000e4b0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.908+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 shutdown_connections 2024-05-30T16:44:45.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.908+0000 7fd93b200640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd91806c8f0 0x7fd91806edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.908+0000 7fd93b200640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9340ce5d0 0x7fd93406ec30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.908+0000 7fd93b200640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9340ce1f0 0x7fd934148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.908+0000 7fd93b200640 1 --2- 172.21.15.67:0/1231051517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9340b3d60 0x7fd93415a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:45.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.908+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 >> 172.21.15.67:0/1231051517 conn(0x7fd93406b900 msgr2=0x7fd93415c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.909+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 shutdown_connections 2024-05-30T16:44:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:45.909+0000 7fd93b200640 1 -- 172.21.15.67:0/1231051517 wait complete. 2024-05-30T16:44:45.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:44:46.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.119+0000 7f7d82800640 1 Processor -- start 2024-05-30T16:44:46.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.119+0000 7f7d82800640 1 -- start start 2024-05-30T16:44:46.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.120+0000 7f7d82800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d7c15b3a0 0x7f7d7c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.120+0000 7f7d82800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7d7c06dc20 0x7f7d7c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.120+0000 7f7d82800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d7c072c00 0x7f7d7c073050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.120+0000 7f7d82800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7c06f980 con 0x7f7d7c15b3a0 2024-05-30T16:44:46.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.120+0000 7f7d82800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7c073590 con 0x7f7d7c06dc20 2024-05-30T16:44:46.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.120+0000 7f7d82800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7c0ae0c0 con 0x7f7d7c072c00 2024-05-30T16:44:46.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.120+0000 7f7d81400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d7c15b3a0 0x7f7d7c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.120+0000 7f7d81400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d7c15b3a0 0x7f7d7c0b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51054/0 (socket says 172.21.15.67:51054) 2024-05-30T16:44:46.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.120+0000 7f7d81400640 1 -- 172.21.15.67:0/3160590925 learned_addr learned my addr 172.21.15.67:0/3160590925 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:46.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.121+0000 7f7d81400640 1 -- 172.21.15.67:0/3160590925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d7c072c00 msgr2=0x7f7d7c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:46.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.121+0000 7f7d81400640 1 --2- 172.21.15.67:0/3160590925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d7c072c00 0x7f7d7c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.121+0000 7f7d81400640 1 -- 172.21.15.67:0/3160590925 >> v2:172.21.15.169:3300/0 conn(0x7f7d7c06dc20 msgr2=0x7f7d7c15f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:46.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.121+0000 7f7d81400640 1 --2- 172.21.15.67:0/3160590925 >> v2:172.21.15.169:3300/0 conn(0x7f7d7c06dc20 0x7f7d7c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.121+0000 7f7d81400640 1 -- 172.21.15.67:0/3160590925 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c0afd60 con 0x7f7d7c15b3a0 2024-05-30T16:44:46.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.122+0000 7f7d81400640 1 --2- 172.21.15.67:0/3160590925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c15b3a0 0x7f7d7c0b32c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0711f0 tx=0x7f7d7002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100a71b17b3303ea server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.122+0000 7f7d7be00640 1 -- 172.21.15.67:0/3160590925 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70032790 con 0x7f7d7c15b3a0 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.122+0000 7f7d7be00640 1 -- 172.21.15.67:0/3160590925 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d70038070 con 0x7f7d7c15b3a0 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.122+0000 7f7d7be00640 1 -- 172.21.15.67:0/3160590925 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70032b50 con 0x7f7d7c15b3a0 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.123+0000 7f7d82800640 1 -- 172.21.15.67:0/3160590925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c15b3a0 msgr2=0x7f7d7c0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.123+0000 7f7d82800640 1 --2- 172.21.15.67:0/3160590925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c15b3a0 0x7f7d7c0b32c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0711f0 tx=0x7f7d7002c890 comp rx=0 tx=0).stop 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.123+0000 7f7d82800640 1 -- 172.21.15.67:0/3160590925 shutdown_connections 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.123+0000 7f7d82800640 1 --2- 172.21.15.67:0/3160590925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d7c072c00 0x7f7d7c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.123+0000 7f7d82800640 1 --2- 172.21.15.67:0/3160590925 >> v2:172.21.15.169:3300/0 conn(0x7f7d7c06dc20 0x7f7d7c15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.123+0000 7f7d82800640 1 --2- 172.21.15.67:0/3160590925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c15b3a0 0x7f7d7c0b32c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.123+0000 7f7d82800640 1 -- 172.21.15.67:0/3160590925 >> 172.21.15.67:0/3160590925 conn(0x7f7d7c0b1fc0 msgr2=0x7f7d7c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.123+0000 7f7d82800640 1 -- 172.21.15.67:0/3160590925 shutdown_connections 2024-05-30T16:44:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.123+0000 7f7d82800640 1 -- 172.21.15.67:0/3160590925 wait complete. 2024-05-30T16:44:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.125+0000 7f7d82800640 1 Processor -- start 2024-05-30T16:44:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.125+0000 7f7d82800640 1 -- start start 2024-05-30T16:44:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.126+0000 7f7d82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c06dc20 0x7f7d7c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.126+0000 7f7d82800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d7c072c00 0x7f7d7c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.126+0000 7f7d82800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d7c15b3a0 0x7f7d7c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.126+0000 7f7d82800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c0afa80 con 0x7f7d7c06dc20 2024-05-30T16:44:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.126+0000 7f7d82800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c15fee0 con 0x7f7d7c072c00 2024-05-30T16:44:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.126+0000 7f7d82800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7c15b780 con 0x7f7d7c15b3a0 2024-05-30T16:44:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.126+0000 7f7d81400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c06dc20 0x7f7d7c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.126+0000 7f7d81400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c06dc20 0x7f7d7c0cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51062/0 (socket says 172.21.15.67:51062) 2024-05-30T16:44:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.126+0000 7f7d81400640 1 -- 172.21.15.67:0/4031349228 learned_addr learned my addr 172.21.15.67:0/4031349228 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d81400640 1 -- 172.21.15.67:0/4031349228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d7c15b3a0 msgr2=0x7f7d7c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d81400640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d7c15b3a0 0x7f7d7c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d81400640 1 -- 172.21.15.67:0/4031349228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d7c072c00 msgr2=0x7f7d7c0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d81400640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d7c072c00 0x7f7d7c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d81400640 1 -- 172.21.15.67:0/4031349228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c1f7da0 con 0x7f7d7c06dc20 2024-05-30T16:44:46.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d81400640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c06dc20 0x7f7d7c0cba50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d70000c00 tx=0x7f7d70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:46.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d79600640 1 -- 172.21.15.67:0/4031349228 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70043070 con 0x7f7d7c06dc20 2024-05-30T16:44:46.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d79600640 1 -- 172.21.15.67:0/4031349228 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7003c640 con 0x7f7d7c06dc20 2024-05-30T16:44:46.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d79600640 1 -- 172.21.15.67:0/4031349228 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7003ca30 con 0x7f7d7c06dc20 2024-05-30T16:44:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.127+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c0ced40 con 0x7f7d7c06dc20 2024-05-30T16:44:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.128+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c0cf300 con 0x7f7d7c06dc20 2024-05-30T16:44:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.129+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c1f7f70 con 0x7f7d7c06dc20 2024-05-30T16:44:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.130+0000 7f7d79600640 1 -- 172.21.15.67:0/4031349228 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d70046070 con 0x7f7d7c06dc20 2024-05-30T16:44:46.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.130+0000 7f7d79600640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.131+0000 7f7d79600640 1 -- 172.21.15.67:0/4031349228 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7d700ac1c0 con 0x7f7d7c06dc20 2024-05-30T16:44:46.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.131+0000 7f7d80a00640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.131+0000 7f7d80a00640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c1f7c60 tx=0x7f7d68006bf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.135+0000 7f7d79600640 1 -- 172.21.15.67:0/4031349228 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d7007a680 con 0x7f7d7c06dc20 2024-05-30T16:44:46.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.337+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d7c06e640 con 0x7f7d7c06dc20 2024-05-30T16:44:46.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.340+0000 7f7d79600640 1 -- 172.21.15.67:0/4031349228 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16670 (secure 0 0 0) 0x7f7d70079f80 con 0x7f7d7c06dc20 2024-05-30T16:44:46.342 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:46.342 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":34,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:44:45.339051+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:44:43.288139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-30T16:44:44.321242+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:44:46.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.343+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d5c06c8f0 msgr2=0x7f7d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.343+0000 7f7d82800640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c1f7c60 tx=0x7f7d68006bf0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.343+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c06dc20 msgr2=0x7f7d7c0cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.343+0000 7f7d82800640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c06dc20 0x7f7d7c0cba50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d70000c00 tx=0x7f7d70004730 comp rx=0 tx=0).stop 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.344+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 shutdown_connections 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.344+0000 7f7d82800640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.344+0000 7f7d82800640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d7c15b3a0 0x7f7d7c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.344+0000 7f7d82800640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d7c072c00 0x7f7d7c0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.344+0000 7f7d82800640 1 --2- 172.21.15.67:0/4031349228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d7c06dc20 0x7f7d7c0cba50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.344+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 >> 172.21.15.67:0/4031349228 conn(0x7f7d7c0b1fc0 msgr2=0x7f7d7c06eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.344+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 shutdown_connections 2024-05-30T16:44:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.344+0000 7f7d82800640 1 -- 172.21.15.67:0/4031349228 wait complete. 2024-05-30T16:44:46.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:44:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.572+0000 7fdb07000640 1 Processor -- start 2024-05-30T16:44:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.572+0000 7fdb07000640 1 -- start start 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.574+0000 7fdb07000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdb0006f860 0x7fdb000b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.574+0000 7fdb07000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb000d0bb0 0x7fdb000ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.574+0000 7fdb07000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb0006ed10 con 0x7fdb000b3690 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.574+0000 7fdb07000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdb0015cb00 con 0x7fdb0006f860 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.574+0000 7fdb07000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb000b3cc0 con 0x7fdb000d0bb0 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.574+0000 7fdaf7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdb000b3690 0x7fdb000d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34634/0 (socket says 172.21.15.67:34634) 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.574+0000 7fdaf7e00640 1 -- 172.21.15.67:0/1049798767 learned_addr learned my addr 172.21.15.67:0/1049798767 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdb05000640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb000d0bb0 0x7fdb000ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaffe00640 1 --2- 172.21.15.67:0/1049798767 >> v2:172.21.15.169:3300/0 conn(0x7fdb0006f860 0x7fdb000b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 727070224 0 0) 0x7fdae8001110 con 0x7fdb000b3690 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdae4003520 con 0x7fdb000b3690 2024-05-30T16:44:46.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2885480030 0 0) 0x7fdae4003520 con 0x7fdb000b3690 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb000d0bb0 msgr2=0x7fdb000ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb000d0bb0 0x7fdb000ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 >> v2:172.21.15.169:3300/0 conn(0x7fdb0006f860 msgr2=0x7fdb000b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 --2- 172.21.15.67:0/1049798767 >> v2:172.21.15.169:3300/0 conn(0x7fdb0006f860 0x7fdb000b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb000b3fa0 con 0x7fdb000b3690 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fdae8001840 con 0x7fdb000b3690 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 >> v1:172.21.15.67:6789/0 conn(0x7fdb000b3690 legacy=0x7fdb000d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb000d0bb0 0x7fdae40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdae4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdae4004d40 0x7fdae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0015cb00 con 0x7fdae4003f80 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb000b3cc0 con 0x7fdae4004d40 2024-05-30T16:44:46.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.575+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdae400b9c0 con 0x7fdb000d0bb0 2024-05-30T16:44:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.576+0000 7fdaf7e00640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdae4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.576+0000 7fdaffe00640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb000d0bb0 0x7fdae40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.576+0000 7fdb05000640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdae4004d40 0x7fdae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.576+0000 7fdaf7e00640 1 -- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb000d0bb0 msgr2=0x7fdae40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.576+0000 7fdaf7e00640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb000d0bb0 0x7fdae40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.576+0000 7fdaf7e00640 1 -- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdae4004d40 msgr2=0x7fdae40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.576+0000 7fdaf7e00640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdae4004d40 0x7fdae40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.576+0000 7fdaf7e00640 1 -- 172.21.15.67:0/1049798767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae4003520 con 0x7fdae4003f80 2024-05-30T16:44:46.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.577+0000 7fdaf7e00640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdae4004800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdae800c350 tx=0x7fdae4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6e4c82930caae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:46.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.577+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8035070 con 0x7fdae4003f80 2024-05-30T16:44:46.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.577+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae8007e50 con 0x7fdae4003f80 2024-05-30T16:44:46.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.577+0000 7fdaff400640 1 -- 172.21.15.67:0/1049798767 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8005d70 con 0x7fdae4003f80 2024-05-30T16:44:46.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.578+0000 7fdb07000640 1 -- 172.21.15.67:0/1049798767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 msgr2=0x7fdae4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.578+0000 7fdb07000640 1 --2- 172.21.15.67:0/1049798767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdae4004800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdae800c350 tx=0x7fdae4005830 comp rx=0 tx=0).stop 2024-05-30T16:44:46.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.578+0000 7fdb05000640 1 -- 172.21.15.67:0/1049798767 reap_dead start 2024-05-30T16:44:46.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.578+0000 7fdb07000640 1 -- 172.21.15.67:0/1049798767 shutdown_connections 2024-05-30T16:44:46.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.578+0000 7fdb07000640 1 -- 172.21.15.67:0/1049798767 >> 172.21.15.67:0/1049798767 conn(0x7fdb0006bae0 msgr2=0x7fdb0006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:46.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.578+0000 7fdb07000640 1 -- 172.21.15.67:0/1049798767 shutdown_connections 2024-05-30T16:44:46.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.578+0000 7fdb07000640 1 -- 172.21.15.67:0/1049798767 wait complete. 2024-05-30T16:44:46.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.580+0000 7fdb07000640 1 Processor -- start 2024-05-30T16:44:46.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.580+0000 7fdb07000640 1 -- start start 2024-05-30T16:44:46.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.581+0000 7fdb07000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdb001f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.581+0000 7fdb07000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb000c7330 0x7fdb000c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.581+0000 7fdb07000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb00071b30 0x7fdb00071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.581+0000 7fdb07000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdb000b4840 con 0x7fdae4003f80 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.581+0000 7fdb07000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb000b4560 con 0x7fdb000c7330 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.581+0000 7fdb07000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb000ad130 con 0x7fdb00071b30 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.581+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdb001f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.581+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdb001f3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51074/0 (socket says 172.21.15.67:51074) 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.581+0000 7fdaffe00640 1 -- 172.21.15.67:0/3819376369 learned_addr learned my addr 172.21.15.67:0/3819376369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdaf7e00640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb000c7330 0x7fdb000c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdb05000640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb00071b30 0x7fdb00071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdaffe00640 1 -- 172.21.15.67:0/3819376369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb00071b30 msgr2=0x7fdb00071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdaffe00640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb00071b30 0x7fdb00071fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdaffe00640 1 -- 172.21.15.67:0/3819376369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb000c7330 msgr2=0x7fdb000c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdaffe00640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb000c7330 0x7fdb000c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdaffe00640 1 -- 172.21.15.67:0/3819376369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb000c7d90 con 0x7fdae4003f80 2024-05-30T16:44:46.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdaf7e00640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb000c7330 0x7fdb000c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:46.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdaffe00640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdb001f3e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0011600 tx=0x7fdaf0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:46.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.582+0000 7fdafcc00640 1 -- 172.21.15.67:0/3819376369 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000f920 con 0x7fdae4003f80 2024-05-30T16:44:46.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.583+0000 7fdafcc00640 1 -- 172.21.15.67:0/3819376369 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf000fac0 con 0x7fdae4003f80 2024-05-30T16:44:46.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.583+0000 7fdafcc00640 1 -- 172.21.15.67:0/3819376369 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000fe70 con 0x7fdae4003f80 2024-05-30T16:44:46.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.583+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb000c8070 con 0x7fdae4003f80 2024-05-30T16:44:46.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.583+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb000c8630 con 0x7fdae4003f80 2024-05-30T16:44:46.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.584+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb000689e0 con 0x7fdae4003f80 2024-05-30T16:44:46.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.585+0000 7fdafcc00640 1 -- 172.21.15.67:0/3819376369 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdaf0015460 con 0x7fdae4003f80 2024-05-30T16:44:46.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.586+0000 7fdafcc00640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdadc06c940 0x7fdadc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:46.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.586+0000 7fdafcc00640 1 -- 172.21.15.67:0/3819376369 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fdaf008c590 con 0x7fdae4003f80 2024-05-30T16:44:46.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.586+0000 7fdaf7e00640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdadc06c940 0x7fdadc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:46.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.587+0000 7fdaf7e00640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdadc06c940 0x7fdadc06ee00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdae8032000 tx=0x7fdae8009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.591+0000 7fdafcc00640 1 -- 172.21.15.67:0/3819376369 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdaf0019030 con 0x7fdae4003f80 2024-05-30T16:44:46.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.797+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb000c46f0 con 0x7fdae4003f80 2024-05-30T16:44:46.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.800+0000 7fdafcc00640 1 -- 172.21.15.67:0/3819376369 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16670 (secure 0 0 0) 0x7fdaf005a9a0 con 0x7fdae4003f80 2024-05-30T16:44:46.801 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:46.802 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":34,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:44:45.339051+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:44:43.288139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-30T16:44:44.321242+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:44:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdadc06c940 msgr2=0x7fdadc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdadc06c940 0x7fdadc06ee00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdae8032000 tx=0x7fdae8009030 comp rx=0 tx=0).stop 2024-05-30T16:44:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 msgr2=0x7fdb001f3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdb001f3e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0011600 tx=0x7fdaf0011ac0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 shutdown_connections 2024-05-30T16:44:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdadc06c940 0x7fdadc06ee00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb00071b30 0x7fdb00071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb000c7330 0x7fdb000c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 --2- 172.21.15.67:0/3819376369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdae4003f80 0x7fdb001f3e60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 >> 172.21.15.67:0/3819376369 conn(0x7fdb0006bae0 msgr2=0x7fdb0006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 shutdown_connections 2024-05-30T16:44:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:46.803+0000 7fdb07000640 1 -- 172.21.15.67:0/3819376369 wait complete. 2024-05-30T16:44:46.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-30T16:44:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.040+0000 7faf05400640 1 Processor -- start 2024-05-30T16:44:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.041+0000 7faf05400640 1 -- start start 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.042+0000 7faf05400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf000d0640 0x7faf0015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.042+0000 7faf05400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faf000d0a20 0x7faf000cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.042+0000 7faf05400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 0x7faf000b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.042+0000 7faf05400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7faf000adbb0 con 0x7faf000d0640 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.042+0000 7faf05400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faf0015db10 con 0x7faf000d0a20 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.042+0000 7faf05400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faf0015cb10 con 0x7faf000b32c0 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faefe800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf000d0640 0x7faf0015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faefe800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf000d0640 0x7faf0015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51084/0 (socket says 172.21.15.67:51084) 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faefe800640 1 -- 172.21.15.67:0/765962690 learned_addr learned my addr 172.21.15.67:0/765962690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faeff200640 1 --2- 172.21.15.67:0/765962690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 0x7faf000b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faefe800640 1 -- 172.21.15.67:0/765962690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 msgr2=0x7faf000b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faef7e00640 1 --2- 172.21.15.67:0/765962690 >> v2:172.21.15.169:3300/0 conn(0x7faf000d0a20 0x7faf000cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faefe800640 1 --2- 172.21.15.67:0/765962690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 0x7faf000b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faefe800640 1 -- 172.21.15.67:0/765962690 >> v2:172.21.15.169:3300/0 conn(0x7faf000d0a20 msgr2=0x7faf000cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faefe800640 1 --2- 172.21.15.67:0/765962690 >> v2:172.21.15.169:3300/0 conn(0x7faf000d0a20 0x7faf000cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faefe800640 1 -- 172.21.15.67:0/765962690 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf000ad5f0 con 0x7faf000d0640 2024-05-30T16:44:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.043+0000 7faef7e00640 1 --2- 172.21.15.67:0/765962690 >> v2:172.21.15.169:3300/0 conn(0x7faf000d0a20 0x7faf000cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.044+0000 7faefe800640 1 --2- 172.21.15.67:0/765962690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 0x7faf0015d5d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf00074120 tx=0x7faef002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2faec08035306ae server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faefde00640 1 -- 172.21.15.67:0/765962690 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0037070 con 0x7faf000d0640 2024-05-30T16:44:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faefde00640 1 -- 172.21.15.67:0/765962690 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef0032440 con 0x7faf000d0640 2024-05-30T16:44:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faefde00640 1 -- 172.21.15.67:0/765962690 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0032810 con 0x7faf000d0640 2024-05-30T16:44:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faf05400640 1 -- 172.21.15.67:0/765962690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 msgr2=0x7faf0015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faf05400640 1 --2- 172.21.15.67:0/765962690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 0x7faf0015d5d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf00074120 tx=0x7faef002a1e0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faf05400640 1 -- 172.21.15.67:0/765962690 shutdown_connections 2024-05-30T16:44:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faf05400640 1 --2- 172.21.15.67:0/765962690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 0x7faf000b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faf05400640 1 --2- 172.21.15.67:0/765962690 >> v2:172.21.15.169:3300/0 conn(0x7faf000d0a20 0x7faf000cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faf05400640 1 --2- 172.21.15.67:0/765962690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 0x7faf0015d5d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faf05400640 1 -- 172.21.15.67:0/765962690 >> 172.21.15.67:0/765962690 conn(0x7faf0006bd20 msgr2=0x7faf0006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faf05400640 1 -- 172.21.15.67:0/765962690 shutdown_connections 2024-05-30T16:44:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.045+0000 7faf05400640 1 -- 172.21.15.67:0/765962690 wait complete. 2024-05-30T16:44:47.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.047+0000 7faf05400640 1 Processor -- start 2024-05-30T16:44:47.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.047+0000 7faf05400640 1 -- start start 2024-05-30T16:44:47.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.048+0000 7faf05400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 0x7faf00145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.048+0000 7faf05400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 0x7faf00072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.048+0000 7faf05400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000d0a20 0x7faf00072650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.048+0000 7faf05400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faf0015d0b0 con 0x7faf000d0640 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.048+0000 7faf05400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faf0015cdd0 con 0x7faf000d0a20 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.048+0000 7faf05400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faf000ad2a0 con 0x7faf000b32c0 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faef7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 0x7faf00072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faef7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 0x7faf00072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51094/0 (socket says 172.21.15.67:51094) 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faef7e00640 1 -- 172.21.15.67:0/1407919726 learned_addr learned my addr 172.21.15.67:0/1407919726 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faeff200640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000d0a20 0x7faf00072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faefe800640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 0x7faf00145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faef7e00640 1 -- 172.21.15.67:0/1407919726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 msgr2=0x7faf00145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faef7e00640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 0x7faf00145290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faef7e00640 1 -- 172.21.15.67:0/1407919726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000d0a20 msgr2=0x7faf00072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faef7e00640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000d0a20 0x7faf00072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faef7e00640 1 -- 172.21.15.67:0/1407919726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf001433d0 con 0x7faf000d0640 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.049+0000 7faeff200640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000d0a20 0x7faf00072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.050+0000 7faef7e00640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 0x7faf00072110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faee800c820 tx=0x7faee800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.050+0000 7faef6a00640 1 -- 172.21.15.67:0/1407919726 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8015070 con 0x7faf000d0640 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.050+0000 7faef6a00640 1 -- 172.21.15.67:0/1407919726 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee8007a30 con 0x7faf000d0640 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.050+0000 7faef6a00640 1 -- 172.21.15.67:0/1407919726 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8007de0 con 0x7faf000d0640 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.050+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf001436b0 con 0x7faf000d0640 2024-05-30T16:44:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.050+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf00143c70 con 0x7faf000d0640 2024-05-30T16:44:47.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.051+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf000689e0 con 0x7faf000d0640 2024-05-30T16:44:47.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.052+0000 7faef6a00640 1 -- 172.21.15.67:0/1407919726 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faee8018070 con 0x7faf000d0640 2024-05-30T16:44:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.053+0000 7faef6a00640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faedc06c8f0 0x7faedc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.053+0000 7faef6a00640 1 -- 172.21.15.67:0/1407919726 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7faee808b9b0 con 0x7faf000d0640 2024-05-30T16:44:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.054+0000 7faefe800640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faedc06c8f0 0x7faedc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.054+0000 7faefe800640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faedc06c8f0 0x7faedc06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faef0034000 tx=0x7faef002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.058+0000 7faef6a00640 1 -- 172.21.15.67:0/1407919726 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faee8059e70 con 0x7faf000d0640 2024-05-30T16:44:47.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.302+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faf00071ef0 con 0x7faf000d0640 2024-05-30T16:44:47.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.909+0000 7faef6a00640 1 -- 172.21.15.67:0/1407919726 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7faee8059770 con 0x7faf000d0640 2024-05-30T16:44:47.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.912+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faedc06c8f0 msgr2=0x7faedc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:47.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.912+0000 7faf05400640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faedc06c8f0 0x7faedc06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faef0034000 tx=0x7faef002e000 comp rx=0 tx=0).stop 2024-05-30T16:44:47.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.912+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 msgr2=0x7faf00072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:47.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.912+0000 7faf05400640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 0x7faf00072110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faee800c820 tx=0x7faee800cce0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.913+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 shutdown_connections 2024-05-30T16:44:47.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.913+0000 7faf05400640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faedc06c8f0 0x7faedc06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.913+0000 7faf05400640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000d0a20 0x7faf00072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.913+0000 7faf05400640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf000d0640 0x7faf00072110 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.913+0000 7faf05400640 1 --2- 172.21.15.67:0/1407919726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b32c0 0x7faf00145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:47.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.913+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 >> 172.21.15.67:0/1407919726 conn(0x7faf0006bd20 msgr2=0x7faf00073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:47.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.913+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 shutdown_connections 2024-05-30T16:44:47.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:47.913+0000 7faf05400640 1 -- 172.21.15.67:0/1407919726 wait complete. 2024-05-30T16:44:47.939 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-30T16:44:47.940 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-30T16:44:48.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.169+0000 7f9a5ae00640 1 Processor -- start 2024-05-30T16:44:48.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.169+0000 7f9a5ae00640 1 -- start start 2024-05-30T16:44:48.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.171+0000 7f9a5ae00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a540cd2a0 0x7f9a540b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:48.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.171+0000 7f9a5ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 0x7f9a5415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:48.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.171+0000 7f9a5ae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a5415d900 con 0x7f9a54153510 2024-05-30T16:44:48.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.171+0000 7f9a5ae00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9a541538f0 con 0x7f9a540cd2a0 2024-05-30T16:44:48.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.171+0000 7f9a5ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a540b48c0 con 0x7f9a5415cb60 2024-05-30T16:44:48.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.172+0000 7f9a59000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a540cd2a0 0x7f9a540b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:48.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.172+0000 7f9a59000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a540cd2a0 0x7f9a540b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51100/0 (socket says 172.21.15.67:51100) 2024-05-30T16:44:48.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.172+0000 7f9a59000640 1 -- 172.21.15.67:0/779143184 learned_addr learned my addr 172.21.15.67:0/779143184 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:48.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.172+0000 7f9a5a400640 1 --2- 172.21.15.67:0/779143184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 0x7f9a5415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:48.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.172+0000 7f9a59000640 1 -- 172.21.15.67:0/779143184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 msgr2=0x7f9a5415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:48.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.172+0000 7f9a59000640 1 --2- 172.21.15.67:0/779143184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 0x7f9a5415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.172+0000 7f9a59000640 1 -- 172.21.15.67:0/779143184 >> v1:172.21.15.169:6789/0 conn(0x7f9a54153510 legacy=0x7f9a540ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:48.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.172+0000 7f9a59000640 1 -- 172.21.15.67:0/779143184 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a540b3f90 con 0x7f9a540cd2a0 2024-05-30T16:44:48.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.172+0000 7f9a5a400640 1 --2- 172.21.15.67:0/779143184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 0x7f9a5415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:48.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.173+0000 7f9a59000640 1 --2- 172.21.15.67:0/779143184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 0x7f9a540b4d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009870 tx=0x7f9a4002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f92d3abe300f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:48.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.174+0000 7f9a53e00640 1 -- 172.21.15.67:0/779143184 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40037070 con 0x7f9a540cd2a0 2024-05-30T16:44:48.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.174+0000 7f9a53e00640 1 -- 172.21.15.67:0/779143184 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a40032440 con 0x7f9a540cd2a0 2024-05-30T16:44:48.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.174+0000 7f9a53e00640 1 -- 172.21.15.67:0/779143184 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40032810 con 0x7f9a540cd2a0 2024-05-30T16:44:48.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.175+0000 7f9a5ae00640 1 -- 172.21.15.67:0/779143184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 msgr2=0x7f9a540b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:48.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.175+0000 7f9a5ae00640 1 --2- 172.21.15.67:0/779143184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 0x7f9a540b4d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009870 tx=0x7f9a4002a180 comp rx=0 tx=0).stop 2024-05-30T16:44:48.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.175+0000 7f9a5ae00640 1 -- 172.21.15.67:0/779143184 shutdown_connections 2024-05-30T16:44:48.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.175+0000 7f9a5ae00640 1 --2- 172.21.15.67:0/779143184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 0x7f9a5415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.175+0000 7f9a5ae00640 1 --2- 172.21.15.67:0/779143184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 0x7f9a540b4d50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.175+0000 7f9a5ae00640 1 -- 172.21.15.67:0/779143184 >> 172.21.15.67:0/779143184 conn(0x7f9a5406b900 msgr2=0x7f9a5406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:48.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.175+0000 7f9a5ae00640 1 -- 172.21.15.67:0/779143184 shutdown_connections 2024-05-30T16:44:48.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.175+0000 7f9a5ae00640 1 -- 172.21.15.67:0/779143184 wait complete. 2024-05-30T16:44:48.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.177+0000 7f9a5ae00640 1 Processor -- start 2024-05-30T16:44:48.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.177+0000 7f9a5ae00640 1 -- start start 2024-05-30T16:44:48.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.178+0000 7f9a5ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 0x7f9a540ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:48.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.178+0000 7f9a5ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a54153510 0x7f9a540cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:48.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.178+0000 7f9a5ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 0x7f9a54153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:48.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.178+0000 7f9a5ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5415d4f0 con 0x7f9a540cd2a0 2024-05-30T16:44:48.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.178+0000 7f9a5ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a540b4550 con 0x7f9a54153510 2024-05-30T16:44:48.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.178+0000 7f9a5ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5415d670 con 0x7f9a5415cb60 2024-05-30T16:44:48.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.178+0000 7f9a59a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 0x7f9a540ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:48.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.178+0000 7f9a59a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 0x7f9a540ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51116/0 (socket says 172.21.15.67:51116) 2024-05-30T16:44:48.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.178+0000 7f9a59a00640 1 -- 172.21.15.67:0/56545052 learned_addr learned my addr 172.21.15.67:0/56545052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a5a400640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 0x7f9a54153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a59000640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a54153510 0x7f9a540cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a59a00640 1 -- 172.21.15.67:0/56545052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 msgr2=0x7f9a54153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a59a00640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 0x7f9a54153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a59a00640 1 -- 172.21.15.67:0/56545052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a54153510 msgr2=0x7f9a540cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a59a00640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a54153510 0x7f9a540cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a59a00640 1 -- 172.21.15.67:0/56545052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a54153fc0 con 0x7f9a540cd2a0 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a59a00640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 0x7f9a540ad2c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800c430 tx=0x7f9a4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a51600640 1 -- 172.21.15.67:0/56545052 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48011790 con 0x7f9a540cd2a0 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a51600640 1 -- 172.21.15.67:0/56545052 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a48017070 con 0x7f9a540cd2a0 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a51600640 1 -- 172.21.15.67:0/56545052 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48011bb0 con 0x7f9a540cd2a0 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.179+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a540aeee0 con 0x7f9a540cd2a0 2024-05-30T16:44:48.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.180+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a540aebb0 con 0x7f9a540cd2a0 2024-05-30T16:44:48.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.181+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a540689e0 con 0x7f9a540cd2a0 2024-05-30T16:44:48.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.182+0000 7f9a51600640 1 -- 172.21.15.67:0/56545052 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a4801c460 con 0x7f9a540cd2a0 2024-05-30T16:44:48.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.182+0000 7f9a51600640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a3406c920 0x7f9a3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:48.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.183+0000 7f9a51600640 1 -- 172.21.15.67:0/56545052 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9a4808b960 con 0x7f9a540cd2a0 2024-05-30T16:44:48.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.183+0000 7f9a59000640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a3406c920 0x7f9a3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:48.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.184+0000 7f9a59000640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a3406c920 0x7f9a3406ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a40034000 tx=0x7f9a4002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:48.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.187+0000 7f9a51600640 1 -- 172.21.15.67:0/56545052 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a48059e20 con 0x7f9a540cd2a0 2024-05-30T16:44:48.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.431+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9a540b3c40 con 0x7f9a540cd2a0 2024-05-30T16:44:48.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.920+0000 7f9a51600640 1 -- 172.21.15.67:0/56545052 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f9a48059720 con 0x7f9a540cd2a0 2024-05-30T16:44:48.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.923+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a3406c920 msgr2=0x7f9a3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:48.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.923+0000 7f9a5ae00640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a3406c920 0x7f9a3406ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a40034000 tx=0x7f9a4002e000 comp rx=0 tx=0).stop 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.923+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 msgr2=0x7f9a540ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.923+0000 7f9a5ae00640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 0x7f9a540ad2c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800c430 tx=0x7f9a4800c8f0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.923+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 shutdown_connections 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.923+0000 7f9a5ae00640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a3406c920 0x7f9a3406ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.924+0000 7f9a5ae00640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a5415cb60 0x7f9a54153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.924+0000 7f9a5ae00640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a54153510 0x7f9a540cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.924+0000 7f9a5ae00640 1 --2- 172.21.15.67:0/56545052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a540cd2a0 0x7f9a540ad2c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.924+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 >> 172.21.15.67:0/56545052 conn(0x7f9a5406b900 msgr2=0x7f9a5406f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.924+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 shutdown_connections 2024-05-30T16:44:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:48.924+0000 7f9a5ae00640 1 -- 172.21.15.67:0/56545052 wait complete. 2024-05-30T16:44:48.946 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-30T16:44:48.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:44:49.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.149+0000 7fa38fe00640 1 Processor -- start 2024-05-30T16:44:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.150+0000 7fa38fe00640 1 -- start start 2024-05-30T16:44:49.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.151+0000 7fa38fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa390073310 0x7fa3900b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.151+0000 7fa38fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3900afd60 con 0x7fa39006dc20 2024-05-30T16:44:49.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.151+0000 7fa38fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3900af7a0 con 0x7fa39015b3a0 2024-05-30T16:44:49.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.151+0000 7fa38fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa39006f980 con 0x7fa390073310 2024-05-30T16:44:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.152+0000 7fa38ea00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa39015b3a0 0x7fa390072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:46830/0 (socket says 172.21.15.67:46830) 2024-05-30T16:44:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.152+0000 7fa38ea00640 1 -- 172.21.15.67:0/598982097 learned_addr learned my addr 172.21.15.67:0/598982097 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.152+0000 7fa38e000640 1 --2- 172.21.15.67:0/598982097 >> v2:172.21.15.67:3300/0 conn(0x7fa390073310 0x7fa3900b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.153+0000 7fa38d600640 1 -- 172.21.15.67:0/598982097 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1951575872 0 0) 0x7fa3900afd60 con 0x7fa39006dc20 2024-05-30T16:44:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.153+0000 7fa38e000640 1 -- 172.21.15.67:0/598982097 >> v1:172.21.15.169:6789/0 conn(0x7fa39015b3a0 legacy=0x7fa390072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.153+0000 7fa38e000640 1 -- 172.21.15.67:0/598982097 >> v1:172.21.15.67:6789/0 conn(0x7fa39006dc20 legacy=0x7fa3900ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.153+0000 7fa38e000640 1 -- 172.21.15.67:0/598982097 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3900afa80 con 0x7fa390073310 2024-05-30T16:44:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.154+0000 7fa38e000640 1 --2- 172.21.15.67:0/598982097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa390073310 0x7fa3900b32c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0097a0 tx=0x7fa37c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0af714cb9d98ba server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.154+0000 7fa38d600640 1 -- 172.21.15.67:0/598982097 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c037070 con 0x7fa390073310 2024-05-30T16:44:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.154+0000 7fa38d600640 1 -- 172.21.15.67:0/598982097 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37c032440 con 0x7fa390073310 2024-05-30T16:44:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.154+0000 7fa38d600640 1 -- 172.21.15.67:0/598982097 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c032810 con 0x7fa390073310 2024-05-30T16:44:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.155+0000 7fa38fe00640 1 -- 172.21.15.67:0/598982097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa390073310 msgr2=0x7fa3900b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.155+0000 7fa38fe00640 1 --2- 172.21.15.67:0/598982097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa390073310 0x7fa3900b32c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0097a0 tx=0x7fa37c02a230 comp rx=0 tx=0).stop 2024-05-30T16:44:49.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.155+0000 7fa38fe00640 1 -- 172.21.15.67:0/598982097 shutdown_connections 2024-05-30T16:44:49.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.155+0000 7fa38fe00640 1 --2- 172.21.15.67:0/598982097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa390073310 0x7fa3900b32c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.155+0000 7fa38fe00640 1 -- 172.21.15.67:0/598982097 >> 172.21.15.67:0/598982097 conn(0x7fa3900b1fc0 msgr2=0x7fa3900b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:49.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.155+0000 7fa38fe00640 1 -- 172.21.15.67:0/598982097 shutdown_connections 2024-05-30T16:44:49.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.155+0000 7fa38fe00640 1 -- 172.21.15.67:0/598982097 wait complete. 2024-05-30T16:44:49.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.157+0000 7fa38fe00640 1 Processor -- start 2024-05-30T16:44:49.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.157+0000 7fa38fe00640 1 -- start start 2024-05-30T16:44:49.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.158+0000 7fa38fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa39006dc20 0x7fa3901f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.158+0000 7fa38fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa390073310 0x7fa390159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.158+0000 7fa38fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa39015b3a0 0x7fa39014bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.158+0000 7fa38fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa39015f9a0 con 0x7fa39015b3a0 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.158+0000 7fa38fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa3900ae0c0 con 0x7fa390073310 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.158+0000 7fa38fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa39015fb20 con 0x7fa39006dc20 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.158+0000 7fa38f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa39015b3a0 0x7fa39014bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.158+0000 7fa38f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa39015b3a0 0x7fa39014bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51128/0 (socket says 172.21.15.67:51128) 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.158+0000 7fa38f400640 1 -- 172.21.15.67:0/125975285 learned_addr learned my addr 172.21.15.67:0/125975285 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38e000640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa390073310 0x7fa390159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38ea00640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa39006dc20 0x7fa3901f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38f400640 1 -- 172.21.15.67:0/125975285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa39006dc20 msgr2=0x7fa3901f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38f400640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa39006dc20 0x7fa3901f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38f400640 1 -- 172.21.15.67:0/125975285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa390073310 msgr2=0x7fa390159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38f400640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa390073310 0x7fa390159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38f400640 1 -- 172.21.15.67:0/125975285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39014cf70 con 0x7fa39015b3a0 2024-05-30T16:44:49.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38e000640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa390073310 0x7fa390159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:49.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38f400640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa39015b3a0 0x7fa39014bf50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa38800c780 tx=0x7fa38800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:49.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa387400640 1 -- 172.21.15.67:0/125975285 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388003da0 con 0x7fa39015b3a0 2024-05-30T16:44:49.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa387400640 1 -- 172.21.15.67:0/125975285 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38800ce30 con 0x7fa39015b3a0 2024-05-30T16:44:49.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa387400640 1 -- 172.21.15.67:0/125975285 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388005290 con 0x7fa39015b3a0 2024-05-30T16:44:49.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39014ca10 con 0x7fa39015b3a0 2024-05-30T16:44:49.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.159+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa390159930 con 0x7fa39015b3a0 2024-05-30T16:44:49.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.161+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3900689e0 con 0x7fa39015b3a0 2024-05-30T16:44:49.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.162+0000 7fa387400640 1 -- 172.21.15.67:0/125975285 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa388005430 con 0x7fa39015b3a0 2024-05-30T16:44:49.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.163+0000 7fa387400640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa37006c8f0 0x7fa37006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.163+0000 7fa387400640 1 -- 172.21.15.67:0/125975285 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa38808ba10 con 0x7fa39015b3a0 2024-05-30T16:44:49.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.163+0000 7fa38ea00640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa37006c8f0 0x7fa37006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.164+0000 7fa38ea00640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa37006c8f0 0x7fa37006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa380007fd0 tx=0x7fa380009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:49.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.167+0000 7fa387400640 1 -- 172.21.15.67:0/125975285 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa39014ca10 con 0x7fa39015b3a0 2024-05-30T16:44:49.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.412+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3900b2e90 con 0x7fa39015b3a0 2024-05-30T16:44:49.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.413+0000 7fa387400640 1 -- 172.21.15.67:0/125975285 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5181 (secure 0 0 0) 0x7fa388059e30 con 0x7fa39015b3a0 2024-05-30T16:44:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:49.414 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/475904074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":475904074},{"type":"v1","addr":"172.21.15.169:6834","nonce":475904074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4812,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3464775083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3464775083},{"type":"v1","addr":"172.21.15.67:6835","nonce":3464775083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/650843675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":650843675},{"type":"v1","addr":"172.21.15.67:6837","nonce":650843675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:44:45.338997+0000","modified":"2024-05-30T16:44:48.433470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4821":{"gid":4821,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/2481507396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2481507396},{"type":"v1","addr":"172.21.15.169:6835","nonce":2481507396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4821,"qdb_cluster":[4821]},"id":2}]} 2024-05-30T16:44:49.415 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 16 2024-05-30T16:44:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa37006c8f0 msgr2=0x7fa37006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa37006c8f0 0x7fa37006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa380007fd0 tx=0x7fa380009040 comp rx=0 tx=0).stop 2024-05-30T16:44:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa39015b3a0 msgr2=0x7fa39014bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa39015b3a0 0x7fa39014bf50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa38800c780 tx=0x7fa38800cc40 comp rx=0 tx=0).stop 2024-05-30T16:44:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 shutdown_connections 2024-05-30T16:44:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa37006c8f0 0x7fa37006edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa39015b3a0 0x7fa39014bf50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa390073310 0x7fa390159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 --2- 172.21.15.67:0/125975285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa39006dc20 0x7fa3901f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 >> 172.21.15.67:0/125975285 conn(0x7fa3900b1fc0 msgr2=0x7fa3900cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:49.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 shutdown_connections 2024-05-30T16:44:49.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.416+0000 7fa38fe00640 1 -- 172.21.15.67:0/125975285 wait complete. 2024-05-30T16:44:49.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:44:49.625 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-05-30T16:44:49.623+0000 7f7d58c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T16:44:49.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.649+0000 7fb509600640 1 Processor -- start 2024-05-30T16:44:49.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.649+0000 7fb509600640 1 -- start start 2024-05-30T16:44:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.650+0000 7fb509600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5040cd8b0 0x7fb50415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.650+0000 7fb509600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb50415e6f0 0x7fb5040cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.650+0000 7fb509600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5040ce020 con 0x7fb50415db30 2024-05-30T16:44:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.650+0000 7fb509600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb5040d11b0 con 0x7fb50415e6f0 2024-05-30T16:44:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.650+0000 7fb509600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb50409aa30 con 0x7fb5040cd8b0 2024-05-30T16:44:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.650+0000 7fb508c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb50415e6f0 0x7fb5040cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.650+0000 7fb503e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb50415db30 0x7fb50409b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34656/0 (socket says 172.21.15.67:34656) 2024-05-30T16:44:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.651+0000 7fb503e00640 1 -- 172.21.15.67:0/536873548 learned_addr learned my addr 172.21.15.67:0/536873548 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.651+0000 7fb503400640 1 --2- 172.21.15.67:0/536873548 >> v2:172.21.15.169:3300/0 conn(0x7fb5040cd8b0 0x7fb50415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.651+0000 7fb508c00640 1 -- 172.21.15.67:0/536873548 >> v2:172.21.15.169:3300/0 conn(0x7fb5040cd8b0 msgr2=0x7fb50415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.651+0000 7fb508c00640 1 --2- 172.21.15.67:0/536873548 >> v2:172.21.15.169:3300/0 conn(0x7fb5040cd8b0 0x7fb50415e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.651+0000 7fb508c00640 1 -- 172.21.15.67:0/536873548 >> v1:172.21.15.67:6789/0 conn(0x7fb50415db30 legacy=0x7fb50409b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.651+0000 7fb508c00640 1 -- 172.21.15.67:0/536873548 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50409ac80 con 0x7fb50415e6f0 2024-05-30T16:44:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.651+0000 7fb502a00640 1 -- 172.21.15.67:0/536873548 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 169141206 0 0) 0x7fb5040ce020 con 0x7fb50415db30 2024-05-30T16:44:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.651+0000 7fb503400640 1 --2- 172.21.15.67:0/536873548 >> v2:172.21.15.169:3300/0 conn(0x7fb5040cd8b0 0x7fb50415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.652+0000 7fb508c00640 1 --2- 172.21.15.67:0/536873548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415e6f0 0x7fb5040cff00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb5040734a0 tx=0x7fb4fc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6960caab7db20c69 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.652+0000 7fb502a00640 1 -- 172.21.15.67:0/536873548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc037070 con 0x7fb50415e6f0 2024-05-30T16:44:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.652+0000 7fb502a00640 1 -- 172.21.15.67:0/536873548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc032440 con 0x7fb50415e6f0 2024-05-30T16:44:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.652+0000 7fb502a00640 1 -- 172.21.15.67:0/536873548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc02ae60 con 0x7fb50415e6f0 2024-05-30T16:44:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.653+0000 7fb509600640 1 -- 172.21.15.67:0/536873548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415e6f0 msgr2=0x7fb5040cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.653+0000 7fb509600640 1 --2- 172.21.15.67:0/536873548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415e6f0 0x7fb5040cff00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb5040734a0 tx=0x7fb4fc02a110 comp rx=0 tx=0).stop 2024-05-30T16:44:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.653+0000 7fb509600640 1 -- 172.21.15.67:0/536873548 shutdown_connections 2024-05-30T16:44:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.653+0000 7fb509600640 1 --2- 172.21.15.67:0/536873548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415e6f0 0x7fb5040cff00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.653+0000 7fb509600640 1 --2- 172.21.15.67:0/536873548 >> v2:172.21.15.169:3300/0 conn(0x7fb5040cd8b0 0x7fb50415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.653+0000 7fb509600640 1 -- 172.21.15.67:0/536873548 >> 172.21.15.67:0/536873548 conn(0x7fb5040d2330 msgr2=0x7fb5040d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:49.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.653+0000 7fb509600640 1 -- 172.21.15.67:0/536873548 shutdown_connections 2024-05-30T16:44:49.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.653+0000 7fb509600640 1 -- 172.21.15.67:0/536873548 wait complete. 2024-05-30T16:44:49.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.655+0000 7fb509600640 1 Processor -- start 2024-05-30T16:44:49.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.655+0000 7fb509600640 1 -- start start 2024-05-30T16:44:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.656+0000 7fb509600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5040cd8b0 0x7fb5040cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.656+0000 7fb509600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415db30 0x7fb50415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.656+0000 7fb509600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb50415e6f0 0x7fb5041f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.656+0000 7fb509600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb50409b240 con 0x7fb50415db30 2024-05-30T16:44:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.656+0000 7fb509600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb50409be70 con 0x7fb5040cd8b0 2024-05-30T16:44:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.656+0000 7fb509600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb5040cdd40 con 0x7fb50415e6f0 2024-05-30T16:44:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.656+0000 7fb503400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415db30 0x7fb50415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb503400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415db30 0x7fb50415c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51138/0 (socket says 172.21.15.67:51138) 2024-05-30T16:44:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb503400640 1 -- 172.21.15.67:0/841086564 learned_addr learned my addr 172.21.15.67:0/841086564 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb503e00640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5040cd8b0 0x7fb5040cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb508c00640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb50415e6f0 0x7fb5041f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb503400640 1 -- 172.21.15.67:0/841086564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb50415e6f0 msgr2=0x7fb5041f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb503400640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb50415e6f0 0x7fb5041f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb503400640 1 -- 172.21.15.67:0/841086564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5040cd8b0 msgr2=0x7fb5040cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb503400640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5040cd8b0 0x7fb5040cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb503400640 1 -- 172.21.15.67:0/841086564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5040cba60 con 0x7fb50415db30 2024-05-30T16:44:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb508c00640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb50415e6f0 0x7fb5041f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.657+0000 7fb503400640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415db30 0x7fb50415c360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c7f0 tx=0x7fb4ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:49.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.658+0000 7fb4fbe00640 1 -- 172.21.15.67:0/841086564 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec015070 con 0x7fb50415db30 2024-05-30T16:44:49.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.658+0000 7fb4fbe00640 1 -- 172.21.15.67:0/841086564 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4ec007820 con 0x7fb50415db30 2024-05-30T16:44:49.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.658+0000 7fb4fbe00640 1 -- 172.21.15.67:0/841086564 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec007bd0 con 0x7fb50415db30 2024-05-30T16:44:49.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.658+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5040cbe30 con 0x7fb50415db30 2024-05-30T16:44:49.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.658+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb504152810 con 0x7fb50415db30 2024-05-30T16:44:49.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.659+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5040cc350 con 0x7fb50415db30 2024-05-30T16:44:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.660+0000 7fb4fbe00640 1 -- 172.21.15.67:0/841086564 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4ec018070 con 0x7fb50415db30 2024-05-30T16:44:49.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.661+0000 7fb4fbe00640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4e006c8f0 0x7fb4e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:49.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.661+0000 7fb4fbe00640 1 -- 172.21.15.67:0/841086564 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb4ec08bd00 con 0x7fb50415db30 2024-05-30T16:44:49.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.661+0000 7fb503e00640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4e006c8f0 0x7fb4e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:49.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.662+0000 7fb503e00640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4e006c8f0 0x7fb4e006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40012e0 tx=0x7fb4f4007ef0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:49.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.666+0000 7fb4fbe00640 1 -- 172.21.15.67:0/841086564 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5040cc350 con 0x7fb50415db30 2024-05-30T16:44:49.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.869+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5040690f0 con 0x7fb50415db30 2024-05-30T16:44:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.872+0000 7fb4fbe00640 1 -- 172.21.15.67:0/841086564 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16657 (secure 0 0 0) 0x7fb5040cba60 con 0x7fb50415db30 2024-05-30T16:44:49.874 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:49.874 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:44:46.898376+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:44:43.288139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-30T16:44:44.321242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:44:49.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4e006c8f0 msgr2=0x7fb4e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4e006c8f0 0x7fb4e006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40012e0 tx=0x7fb4f4007ef0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415db30 msgr2=0x7fb50415c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:49.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415db30 0x7fb50415c360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c7f0 tx=0x7fb4ec00ccb0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 shutdown_connections 2024-05-30T16:44:49.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4e006c8f0 0x7fb4e006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb50415e6f0 0x7fb5041f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50415db30 0x7fb50415c360 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 --2- 172.21.15.67:0/841086564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5040cd8b0 0x7fb5040cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:49.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 >> 172.21.15.67:0/841086564 conn(0x7fb5040d2330 msgr2=0x7fb5040cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:49.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 shutdown_connections 2024-05-30T16:44:49.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:49.876+0000 7fb509600640 1 -- 172.21.15.67:0/841086564 wait complete. 2024-05-30T16:44:49.897 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T16:44:49.897 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:44:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.105+0000 7f34e1200640 1 Processor -- start 2024-05-30T16:44:50.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.105+0000 7f34e1200640 1 -- start start 2024-05-30T16:44:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.106+0000 7f34e1200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc073310 0x7f34dc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.106+0000 7f34e1200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f34dc06dc20 0x7f34dc15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.106+0000 7f34e1200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34dc0afd60 con 0x7f34dc15b3a0 2024-05-30T16:44:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.106+0000 7f34e1200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f34dc06f980 con 0x7f34dc06dc20 2024-05-30T16:44:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.106+0000 7f34e1200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc0ae0c0 con 0x7f34dc073310 2024-05-30T16:44:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.107+0000 7f34daa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc073310 0x7f34dc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.107+0000 7f34dbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f34dc06dc20 0x7f34dc15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.107+0000 7f34daa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc073310 0x7f34dc0b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:53874/0 (socket says 172.21.15.67:53874) 2024-05-30T16:44:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.107+0000 7f34daa00640 1 -- 172.21.15.67:0/3535203656 learned_addr learned my addr 172.21.15.67:0/3535203656 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.108+0000 7f34dbe00640 1 -- 172.21.15.67:0/3535203656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc073310 msgr2=0x7f34dc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.108+0000 7f34dbe00640 1 --2- 172.21.15.67:0/3535203656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc073310 0x7f34dc0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.108+0000 7f34dbe00640 1 -- 172.21.15.67:0/3535203656 >> v1:172.21.15.169:6789/0 conn(0x7f34dc15b3a0 legacy=0x7f34dc072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.108+0000 7f34dbe00640 1 -- 172.21.15.67:0/3535203656 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc0af7a0 con 0x7f34dc06dc20 2024-05-30T16:44:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.108+0000 7f34daa00640 1 --2- 172.21.15.67:0/3535203656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc073310 0x7f34dc0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:44:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.109+0000 7f34dbe00640 1 --2- 172.21.15.67:0/3535203656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34dc06dc20 0x7f34dc15f9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009d90 tx=0x7f34d402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cdd00d4ef51a904 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:50.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.109+0000 7f34da000640 1 -- 172.21.15.67:0/3535203656 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4007740 con 0x7f34dc06dc20 2024-05-30T16:44:50.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.109+0000 7f34da000640 1 -- 172.21.15.67:0/3535203656 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d40078e0 con 0x7f34dc06dc20 2024-05-30T16:44:50.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.109+0000 7f34da000640 1 -- 172.21.15.67:0/3535203656 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4007cb0 con 0x7f34dc06dc20 2024-05-30T16:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.110+0000 7f34e1200640 1 -- 172.21.15.67:0/3535203656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34dc06dc20 msgr2=0x7f34dc15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.110+0000 7f34e1200640 1 --2- 172.21.15.67:0/3535203656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34dc06dc20 0x7f34dc15f9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009d90 tx=0x7f34d402a660 comp rx=0 tx=0).stop 2024-05-30T16:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.110+0000 7f34e1200640 1 -- 172.21.15.67:0/3535203656 shutdown_connections 2024-05-30T16:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.110+0000 7f34e1200640 1 --2- 172.21.15.67:0/3535203656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34dc06dc20 0x7f34dc15f9a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.110+0000 7f34e1200640 1 --2- 172.21.15.67:0/3535203656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc073310 0x7f34dc0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.110+0000 7f34e1200640 1 -- 172.21.15.67:0/3535203656 >> 172.21.15.67:0/3535203656 conn(0x7f34dc0b1fc0 msgr2=0x7f34dc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.110+0000 7f34e1200640 1 -- 172.21.15.67:0/3535203656 shutdown_connections 2024-05-30T16:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.110+0000 7f34e1200640 1 -- 172.21.15.67:0/3535203656 wait complete. 2024-05-30T16:44:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.112+0000 7f34e1200640 1 Processor -- start 2024-05-30T16:44:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.112+0000 7f34e1200640 1 -- start start 2024-05-30T16:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.113+0000 7f34e1200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34dc06dc20 0x7f34dc153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.113+0000 7f34e1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34dc073310 0x7f34dc1f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.113+0000 7f34e1200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc15b3a0 0x7f34dc154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.113+0000 7f34e1200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc15b780 con 0x7f34dc073310 2024-05-30T16:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.113+0000 7f34e1200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc15fee0 con 0x7f34dc06dc20 2024-05-30T16:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.113+0000 7f34e1200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc0ae550 con 0x7f34dc15b3a0 2024-05-30T16:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.113+0000 7f34daa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34dc073310 0x7f34dc1f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34daa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34dc073310 0x7f34dc1f76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51142/0 (socket says 172.21.15.67:51142) 2024-05-30T16:44:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34daa00640 1 -- 172.21.15.67:0/3113154525 learned_addr learned my addr 172.21.15.67:0/3113154525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34dbe00640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc15b3a0 0x7f34dc154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34db400640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34dc06dc20 0x7f34dc153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34daa00640 1 -- 172.21.15.67:0/3113154525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc15b3a0 msgr2=0x7f34dc154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34daa00640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc15b3a0 0x7f34dc154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34daa00640 1 -- 172.21.15.67:0/3113154525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34dc06dc20 msgr2=0x7f34dc153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34daa00640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34dc06dc20 0x7f34dc153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34daa00640 1 -- 172.21.15.67:0/3113154525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc14dc90 con 0x7f34dc073310 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.114+0000 7f34daa00640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34dc073310 0x7f34dc1f76f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f34c800e9d0 tx=0x7f34c800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.115+0000 7f34d3400640 1 -- 172.21.15.67:0/3113154525 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8017070 con 0x7f34dc073310 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.115+0000 7f34d3400640 1 -- 172.21.15.67:0/3113154525 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c800ca60 con 0x7f34dc073310 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.115+0000 7f34d3400640 1 -- 172.21.15.67:0/3113154525 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c800ce10 con 0x7f34dc073310 2024-05-30T16:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.115+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc14df70 con 0x7f34dc073310 2024-05-30T16:44:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.115+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc14e530 con 0x7f34dc073310 2024-05-30T16:44:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.116+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34dc0b1270 con 0x7f34dc073310 2024-05-30T16:44:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.117+0000 7f34d3400640 1 -- 172.21.15.67:0/3113154525 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34c801a070 con 0x7f34dc073310 2024-05-30T16:44:50.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.118+0000 7f34d3400640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34bc06c8f0 0x7f34bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.118+0000 7f34d3400640 1 -- 172.21.15.67:0/3113154525 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f34c808ba70 con 0x7f34dc073310 2024-05-30T16:44:50.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.118+0000 7f34db400640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34bc06c8f0 0x7f34bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.119+0000 7f34db400640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34bc06c8f0 0x7f34bc06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34cc007fd0 tx=0x7f34cc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:50.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.122+0000 7f34d3400640 1 -- 172.21.15.67:0/3113154525 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34dc0b1270 con 0x7f34dc073310 2024-05-30T16:44:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.322+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34dc0689e0 con 0x7f34bc06c8f0 2024-05-30T16:44:50.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.338+0000 7f34d3400640 1 -- 172.21.15.67:0/3113154525 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248689 (secure 0 0 0) 0x7f34dc0689e0 con 0x7f34bc06c8f0 2024-05-30T16:44:50.343 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:50.344 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:44:50.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.344+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34bc06c8f0 msgr2=0x7f34bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.344+0000 7f34e1200640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34bc06c8f0 0x7f34bc06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34cc007fd0 tx=0x7f34cc009040 comp rx=0 tx=0).stop 2024-05-30T16:44:50.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.344+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34dc073310 msgr2=0x7f34dc1f76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.344+0000 7f34e1200640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34dc073310 0x7f34dc1f76f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f34c800e9d0 tx=0x7f34c800ee90 comp rx=0 tx=0).stop 2024-05-30T16:44:50.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.345+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 shutdown_connections 2024-05-30T16:44:50.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.345+0000 7f34e1200640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34bc06c8f0 0x7f34bc06edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.345+0000 7f34e1200640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34dc15b3a0 0x7f34dc154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.345+0000 7f34e1200640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34dc073310 0x7f34dc1f76f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.345+0000 7f34e1200640 1 --2- 172.21.15.67:0/3113154525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34dc06dc20 0x7f34dc153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.345+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 >> 172.21.15.67:0/3113154525 conn(0x7f34dc0b1fc0 msgr2=0x7f34dc071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:50.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.345+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 shutdown_connections 2024-05-30T16:44:50.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.345+0000 7f34e1200640 1 -- 172.21.15.67:0/3113154525 wait complete. 2024-05-30T16:44:50.366 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-05-30T16:44:49.996577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":165,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220236,"kb_used_data":4056,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754484,"statfs":{"total":773094113280,"available":772868591616,"internally_reserved":0,"allocated":4153344,"data_stored":2320410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.871893+0000","last_change":"2024-05-30T16:44:44.871893+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.871893+0000","last_undegraded":"2024-05-30T16:44:44.871893+0000","last_fullsized":"2024-05-30T16:44:44.871893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870925+0000","last_change":"2024-05-30T16:44:44.870925+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870925+0000","last_undegraded":"2024-05-30T16:44:44.870925+0000","last_fullsized":"2024-05-30T16:44:44.870925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915144+0000","last_change":"2024-05-30T16:44:45.915063+0000","last_active":"2024-05-30T16:44:46.915144+0000","last_peered":"2024-05-30T16:44:46.915144+0000","last_clean":"2024-05-30T16:44:46.915144+0000","last_became_active":"2024-05-30T16:44:45.914534+0000","last_became_peered":"2024-05-30T16:44:45.914534+0000","last_unstale":"2024-05-30T16:44:46.915144+0000","last_undegraded":"2024-05-30T16:44:46.915144+0000","last_fullsized":"2024-05-30T16:44:46.915144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.871890+0000","last_change":"2024-05-30T16:44:44.871890+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.871890+0000","last_undegraded":"2024-05-30T16:44:44.871890+0000","last_fullsized":"2024-05-30T16:44:44.871890+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.869063+0000","last_change":"2024-05-30T16:44:44.869063+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.869063+0000","last_undegraded":"2024-05-30T16:44:44.869063+0000","last_fullsized":"2024-05-30T16:44:44.869063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.872128+0000","last_change":"2024-05-30T16:44:44.872128+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.872128+0000","last_undegraded":"2024-05-30T16:44:44.872128+0000","last_fullsized":"2024-05-30T16:44:44.872128+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914905+0000","last_change":"2024-05-30T16:44:45.922189+0000","last_active":"2024-05-30T16:44:46.914905+0000","last_peered":"2024-05-30T16:44:46.914905+0000","last_clean":"2024-05-30T16:44:46.914905+0000","last_became_active":"2024-05-30T16:44:45.921920+0000","last_became_peered":"2024-05-30T16:44:45.921920+0000","last_unstale":"2024-05-30T16:44:46.914905+0000","last_undegraded":"2024-05-30T16:44:46.914905+0000","last_fullsized":"2024-05-30T16:44:46.914905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:32:30.873632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912732+0000","last_change":"2024-05-30T16:44:45.918239+0000","last_active":"2024-05-30T16:44:46.912732+0000","last_peered":"2024-05-30T16:44:46.912732+0000","last_clean":"2024-05-30T16:44:46.912732+0000","last_became_active":"2024-05-30T16:44:45.917936+0000","last_became_peered":"2024-05-30T16:44:45.917936+0000","last_unstale":"2024-05-30T16:44:46.912732+0000","last_undegraded":"2024-05-30T16:44:46.912732+0000","last_fullsized":"2024-05-30T16:44:46.912732+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866406+0000","last_change":"2024-05-30T16:44:44.866406+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866406+0000","last_undegraded":"2024-05-30T16:44:44.866406+0000","last_fullsized":"2024-05-30T16:44:44.866406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.872840+0000","last_change":"2024-05-30T16:44:44.872840+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.872840+0000","last_undegraded":"2024-05-30T16:44:44.872840+0000","last_fullsized":"2024-05-30T16:44:44.872840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915320+0000","last_change":"2024-05-30T16:44:45.927692+0000","last_active":"2024-05-30T16:44:46.915320+0000","last_peered":"2024-05-30T16:44:46.915320+0000","last_clean":"2024-05-30T16:44:46.915320+0000","last_became_active":"2024-05-30T16:44:45.927381+0000","last_became_peered":"2024-05-30T16:44:45.927381+0000","last_unstale":"2024-05-30T16:44:46.915320+0000","last_undegraded":"2024-05-30T16:44:46.915320+0000","last_fullsized":"2024-05-30T16:44:46.915320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:20:16.734760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914592+0000","last_change":"2024-05-30T16:44:45.922504+0000","last_active":"2024-05-30T16:44:46.914592+0000","last_peered":"2024-05-30T16:44:46.914592+0000","last_clean":"2024-05-30T16:44:46.914592+0000","last_became_active":"2024-05-30T16:44:45.922230+0000","last_became_peered":"2024-05-30T16:44:45.922230+0000","last_unstale":"2024-05-30T16:44:46.914592+0000","last_undegraded":"2024-05-30T16:44:46.914592+0000","last_fullsized":"2024-05-30T16:44:46.914592+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867394+0000","last_change":"2024-05-30T16:44:44.867394+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867394+0000","last_undegraded":"2024-05-30T16:44:44.867394+0000","last_fullsized":"2024-05-30T16:44:44.867394+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870665+0000","last_change":"2024-05-30T16:44:44.870665+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870665+0000","last_undegraded":"2024-05-30T16:44:44.870665+0000","last_fullsized":"2024-05-30T16:44:44.870665+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914323+0000","last_change":"2024-05-30T16:44:45.912535+0000","last_active":"2024-05-30T16:44:46.914323+0000","last_peered":"2024-05-30T16:44:46.914323+0000","last_clean":"2024-05-30T16:44:46.914323+0000","last_became_active":"2024-05-30T16:44:45.911217+0000","last_became_peered":"2024-05-30T16:44:45.911217+0000","last_unstale":"2024-05-30T16:44:46.914323+0000","last_undegraded":"2024-05-30T16:44:46.914323+0000","last_fullsized":"2024-05-30T16:44:46.914323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:10:41.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914560+0000","last_change":"2024-05-30T16:44:45.925677+0000","last_active":"2024-05-30T16:44:46.914560+0000","last_peered":"2024-05-30T16:44:46.914560+0000","last_clean":"2024-05-30T16:44:46.914560+0000","last_became_active":"2024-05-30T16:44:45.925335+0000","last_became_peered":"2024-05-30T16:44:45.925335+0000","last_unstale":"2024-05-30T16:44:46.914560+0000","last_undegraded":"2024-05-30T16:44:46.914560+0000","last_fullsized":"2024-05-30T16:44:46.914560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:24:35.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915605+0000","last_change":"2024-05-30T16:44:45.921499+0000","last_active":"2024-05-30T16:44:46.915605+0000","last_peered":"2024-05-30T16:44:46.915605+0000","last_clean":"2024-05-30T16:44:46.915605+0000","last_became_active":"2024-05-30T16:44:45.921212+0000","last_became_peered":"2024-05-30T16:44:45.921212+0000","last_unstale":"2024-05-30T16:44:46.915605+0000","last_undegraded":"2024-05-30T16:44:46.915605+0000","last_fullsized":"2024-05-30T16:44:46.915605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:14.743364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.868154+0000","last_change":"2024-05-30T16:44:44.868154+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.868154+0000","last_undegraded":"2024-05-30T16:44:44.868154+0000","last_fullsized":"2024-05-30T16:44:44.868154+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.869345+0000","last_change":"2024-05-30T16:44:44.869345+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.869345+0000","last_undegraded":"2024-05-30T16:44:44.869345+0000","last_fullsized":"2024-05-30T16:44:44.869345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.875019+0000","last_change":"2024-05-30T16:44:44.875019+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.875019+0000","last_undegraded":"2024-05-30T16:44:44.875019+0000","last_fullsized":"2024-05-30T16:44:44.875019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866701+0000","last_change":"2024-05-30T16:44:44.866701+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866701+0000","last_undegraded":"2024-05-30T16:44:44.866701+0000","last_fullsized":"2024-05-30T16:44:44.866701+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.861135+0000","last_change":"2024-05-30T16:44:44.861135+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.861135+0000","last_undegraded":"2024-05-30T16:44:44.861135+0000","last_fullsized":"2024-05-30T16:44:44.861135+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932990+0000","last_change":"2024-05-30T16:44:45.920307+0000","last_active":"2024-05-30T16:44:46.932990+0000","last_peered":"2024-05-30T16:44:46.932990+0000","last_clean":"2024-05-30T16:44:46.932990+0000","last_became_active":"2024-05-30T16:44:45.918862+0000","last_became_peered":"2024-05-30T16:44:45.918862+0000","last_unstale":"2024-05-30T16:44:46.932990+0000","last_undegraded":"2024-05-30T16:44:46.932990+0000","last_fullsized":"2024-05-30T16:44:46.932990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:52:44.864855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912985+0000","last_change":"2024-05-30T16:44:45.909508+0000","last_active":"2024-05-30T16:44:46.912985+0000","last_peered":"2024-05-30T16:44:46.912985+0000","last_clean":"2024-05-30T16:44:46.912985+0000","last_became_active":"2024-05-30T16:44:45.904007+0000","last_became_peered":"2024-05-30T16:44:45.904007+0000","last_unstale":"2024-05-30T16:44:46.912985+0000","last_undegraded":"2024-05-30T16:44:46.912985+0000","last_fullsized":"2024-05-30T16:44:46.912985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:11.996704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.873189+0000","last_change":"2024-05-30T16:44:44.873189+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.873189+0000","last_undegraded":"2024-05-30T16:44:44.873189+0000","last_fullsized":"2024-05-30T16:44:44.873189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867724+0000","last_change":"2024-05-30T16:44:44.867724+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867724+0000","last_undegraded":"2024-05-30T16:44:44.867724+0000","last_fullsized":"2024-05-30T16:44:44.867724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870720+0000","last_change":"2024-05-30T16:44:44.870720+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870720+0000","last_undegraded":"2024-05-30T16:44:44.870720+0000","last_fullsized":"2024-05-30T16:44:44.870720+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915334+0000","last_change":"2024-05-30T16:44:45.908143+0000","last_active":"2024-05-30T16:44:46.915334+0000","last_peered":"2024-05-30T16:44:46.915334+0000","last_clean":"2024-05-30T16:44:46.915334+0000","last_became_active":"2024-05-30T16:44:45.907830+0000","last_became_peered":"2024-05-30T16:44:45.907830+0000","last_unstale":"2024-05-30T16:44:46.915334+0000","last_undegraded":"2024-05-30T16:44:46.915334+0000","last_fullsized":"2024-05-30T16:44:46.915334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.865560+0000","last_change":"2024-05-30T16:44:44.865560+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.865560+0000","last_undegraded":"2024-05-30T16:44:44.865560+0000","last_fullsized":"2024-05-30T16:44:44.865560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913062+0000","last_change":"2024-05-30T16:44:45.926086+0000","last_active":"2024-05-30T16:44:46.913062+0000","last_peered":"2024-05-30T16:44:46.913062+0000","last_clean":"2024-05-30T16:44:46.913062+0000","last_became_active":"2024-05-30T16:44:45.921839+0000","last_became_peered":"2024-05-30T16:44:45.921839+0000","last_unstale":"2024-05-30T16:44:46.913062+0000","last_undegraded":"2024-05-30T16:44:46.913062+0000","last_fullsized":"2024-05-30T16:44:46.913062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.872445+0000","last_change":"2024-05-30T16:44:44.872445+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.872445+0000","last_undegraded":"2024-05-30T16:44:44.872445+0000","last_fullsized":"2024-05-30T16:44:44.872445+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912803+0000","last_change":"2024-05-30T16:44:45.931329+0000","last_active":"2024-05-30T16:44:46.912803+0000","last_peered":"2024-05-30T16:44:46.912803+0000","last_clean":"2024-05-30T16:44:46.912803+0000","last_became_active":"2024-05-30T16:44:45.929807+0000","last_became_peered":"2024-05-30T16:44:45.929807+0000","last_unstale":"2024-05-30T16:44:46.912803+0000","last_undegraded":"2024-05-30T16:44:46.912803+0000","last_fullsized":"2024-05-30T16:44:46.912803+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933144+0000","last_change":"2024-05-30T16:44:45.918314+0000","last_active":"2024-05-30T16:44:46.933144+0000","last_peered":"2024-05-30T16:44:46.933144+0000","last_clean":"2024-05-30T16:44:46.933144+0000","last_became_active":"2024-05-30T16:44:45.917998+0000","last_became_peered":"2024-05-30T16:44:45.917998+0000","last_unstale":"2024-05-30T16:44:46.933144+0000","last_undegraded":"2024-05-30T16:44:46.933144+0000","last_fullsized":"2024-05-30T16:44:46.933144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:09.909703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.897452+0000","last_change":"2024-05-30T16:44:44.892355+0000","last_active":"2024-05-30T16:44:44.897452+0000","last_peered":"2024-05-30T16:44:44.897452+0000","last_clean":"2024-05-30T16:44:44.897452+0000","last_became_active":"2024-05-30T16:44:44.891889+0000","last_became_peered":"2024-05-30T16:44:44.891889+0000","last_unstale":"2024-05-30T16:44:44.897452+0000","last_undegraded":"2024-05-30T16:44:44.897452+0000","last_fullsized":"2024-05-30T16:44:44.897452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:54.736980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870784+0000","last_change":"2024-05-30T16:44:44.870784+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870784+0000","last_undegraded":"2024-05-30T16:44:44.870784+0000","last_fullsized":"2024-05-30T16:44:44.870784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932836+0000","last_change":"2024-05-30T16:44:44.889925+0000","last_active":"2024-05-30T16:44:46.932836+0000","last_peered":"2024-05-30T16:44:46.932836+0000","last_clean":"2024-05-30T16:44:46.932836+0000","last_became_active":"2024-05-30T16:44:44.889564+0000","last_became_peered":"2024-05-30T16:44:44.889564+0000","last_unstale":"2024-05-30T16:44:46.932836+0000","last_undegraded":"2024-05-30T16:44:46.932836+0000","last_fullsized":"2024-05-30T16:44:46.932836+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:45.960008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.861455+0000","last_change":"2024-05-30T16:44:44.861455+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.861455+0000","last_undegraded":"2024-05-30T16:44:44.861455+0000","last_fullsized":"2024-05-30T16:44:44.861455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.885660+0000","last_change":"2024-05-30T16:44:44.883317+0000","last_active":"2024-05-30T16:44:44.885660+0000","last_peered":"2024-05-30T16:44:44.885660+0000","last_clean":"2024-05-30T16:44:44.885660+0000","last_became_active":"2024-05-30T16:44:44.882994+0000","last_became_peered":"2024-05-30T16:44:44.882994+0000","last_unstale":"2024-05-30T16:44:44.885660+0000","last_undegraded":"2024-05-30T16:44:44.885660+0000","last_fullsized":"2024-05-30T16:44:44.885660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:02:28.009022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870120+0000","last_change":"2024-05-30T16:44:44.870120+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870120+0000","last_undegraded":"2024-05-30T16:44:44.870120+0000","last_fullsized":"2024-05-30T16:44:44.870120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.890431+0000","last_change":"2024-05-30T16:44:44.886536+0000","last_active":"2024-05-30T16:44:44.890431+0000","last_peered":"2024-05-30T16:44:44.890431+0000","last_clean":"2024-05-30T16:44:44.890431+0000","last_became_active":"2024-05-30T16:44:44.886261+0000","last_became_peered":"2024-05-30T16:44:44.886261+0000","last_unstale":"2024-05-30T16:44:44.890431+0000","last_undegraded":"2024-05-30T16:44:44.890431+0000","last_fullsized":"2024-05-30T16:44:44.890431+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:07.538799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913886+0000","last_change":"2024-05-30T16:44:45.936295+0000","last_active":"2024-05-30T16:44:46.913886+0000","last_peered":"2024-05-30T16:44:46.913886+0000","last_clean":"2024-05-30T16:44:46.913886+0000","last_became_active":"2024-05-30T16:44:45.935416+0000","last_became_peered":"2024-05-30T16:44:45.935416+0000","last_unstale":"2024-05-30T16:44:46.913886+0000","last_undegraded":"2024-05-30T16:44:46.913886+0000","last_fullsized":"2024-05-30T16:44:46.913886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894858+0000","last_change":"2024-05-30T16:44:44.886562+0000","last_active":"2024-05-30T16:44:44.894858+0000","last_peered":"2024-05-30T16:44:44.894858+0000","last_clean":"2024-05-30T16:44:44.894858+0000","last_became_active":"2024-05-30T16:44:44.886254+0000","last_became_peered":"2024-05-30T16:44:44.886254+0000","last_unstale":"2024-05-30T16:44:44.894858+0000","last_undegraded":"2024-05-30T16:44:44.894858+0000","last_fullsized":"2024-05-30T16:44:44.894858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:53.956014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913996+0000","last_change":"2024-05-30T16:44:45.905732+0000","last_active":"2024-05-30T16:44:46.913996+0000","last_peered":"2024-05-30T16:44:46.913996+0000","last_clean":"2024-05-30T16:44:46.913996+0000","last_became_active":"2024-05-30T16:44:45.905117+0000","last_became_peered":"2024-05-30T16:44:45.905117+0000","last_unstale":"2024-05-30T16:44:46.913996+0000","last_undegraded":"2024-05-30T16:44:46.913996+0000","last_fullsized":"2024-05-30T16:44:46.913996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:55:15.879718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.891719+0000","last_change":"2024-05-30T16:44:44.879442+0000","last_active":"2024-05-30T16:44:44.891719+0000","last_peered":"2024-05-30T16:44:44.891719+0000","last_clean":"2024-05-30T16:44:44.891719+0000","last_became_active":"2024-05-30T16:44:44.877334+0000","last_became_peered":"2024-05-30T16:44:44.877334+0000","last_unstale":"2024-05-30T16:44:44.891719+0000","last_undegraded":"2024-05-30T16:44:44.891719+0000","last_fullsized":"2024-05-30T16:44:44.891719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:18.355954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.860816+0000","last_change":"2024-05-30T16:44:44.860816+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.860816+0000","last_undegraded":"2024-05-30T16:44:44.860816+0000","last_fullsized":"2024-05-30T16:44:44.860816+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.879443+0000","last_change":"2024-05-30T16:44:44.876950+0000","last_active":"2024-05-30T16:44:44.879443+0000","last_peered":"2024-05-30T16:44:44.879443+0000","last_clean":"2024-05-30T16:44:44.879443+0000","last_became_active":"2024-05-30T16:44:44.876666+0000","last_became_peered":"2024-05-30T16:44:44.876666+0000","last_unstale":"2024-05-30T16:44:44.879443+0000","last_undegraded":"2024-05-30T16:44:44.879443+0000","last_fullsized":"2024-05-30T16:44:44.879443+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:44.589105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867195+0000","last_change":"2024-05-30T16:44:44.867195+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867195+0000","last_undegraded":"2024-05-30T16:44:44.867195+0000","last_fullsized":"2024-05-30T16:44:44.867195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894960+0000","last_change":"2024-05-30T16:44:44.887604+0000","last_active":"2024-05-30T16:44:44.894960+0000","last_peered":"2024-05-30T16:44:44.894960+0000","last_clean":"2024-05-30T16:44:44.894960+0000","last_became_active":"2024-05-30T16:44:44.887259+0000","last_became_peered":"2024-05-30T16:44:44.887259+0000","last_unstale":"2024-05-30T16:44:44.894960+0000","last_undegraded":"2024-05-30T16:44:44.894960+0000","last_fullsized":"2024-05-30T16:44:44.894960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:34:29.911545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915928+0000","last_change":"2024-05-30T16:44:45.918490+0000","last_active":"2024-05-30T16:44:46.915928+0000","last_peered":"2024-05-30T16:44:46.915928+0000","last_clean":"2024-05-30T16:44:46.915928+0000","last_became_active":"2024-05-30T16:44:45.917451+0000","last_became_peered":"2024-05-30T16:44:45.917451+0000","last_unstale":"2024-05-30T16:44:46.915928+0000","last_undegraded":"2024-05-30T16:44:46.915928+0000","last_fullsized":"2024-05-30T16:44:46.915928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893908+0000","last_change":"2024-05-30T16:44:44.889369+0000","last_active":"2024-05-30T16:44:44.893908+0000","last_peered":"2024-05-30T16:44:44.893908+0000","last_clean":"2024-05-30T16:44:44.893908+0000","last_became_active":"2024-05-30T16:44:44.889051+0000","last_became_peered":"2024-05-30T16:44:44.889051+0000","last_unstale":"2024-05-30T16:44:44.893908+0000","last_undegraded":"2024-05-30T16:44:44.893908+0000","last_fullsized":"2024-05-30T16:44:44.893908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:25.424530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866906+0000","last_change":"2024-05-30T16:44:44.866906+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866906+0000","last_undegraded":"2024-05-30T16:44:44.866906+0000","last_fullsized":"2024-05-30T16:44:44.866906+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932945+0000","last_change":"2024-05-30T16:44:44.885815+0000","last_active":"2024-05-30T16:44:46.932945+0000","last_peered":"2024-05-30T16:44:46.932945+0000","last_clean":"2024-05-30T16:44:46.932945+0000","last_became_active":"2024-05-30T16:44:44.885007+0000","last_became_peered":"2024-05-30T16:44:44.885007+0000","last_unstale":"2024-05-30T16:44:46.932945+0000","last_undegraded":"2024-05-30T16:44:46.932945+0000","last_fullsized":"2024-05-30T16:44:46.932945+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:24:02.943052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866137+0000","last_change":"2024-05-30T16:44:44.866137+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866137+0000","last_undegraded":"2024-05-30T16:44:44.866137+0000","last_fullsized":"2024-05-30T16:44:44.866137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915318+0000","last_change":"2024-05-30T16:44:44.891929+0000","last_active":"2024-05-30T16:44:46.915318+0000","last_peered":"2024-05-30T16:44:46.915318+0000","last_clean":"2024-05-30T16:44:46.915318+0000","last_became_active":"2024-05-30T16:44:44.891611+0000","last_became_peered":"2024-05-30T16:44:44.891611+0000","last_unstale":"2024-05-30T16:44:46.915318+0000","last_undegraded":"2024-05-30T16:44:46.915318+0000","last_fullsized":"2024-05-30T16:44:46.915318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:41.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914621+0000","last_change":"2024-05-30T16:44:45.915178+0000","last_active":"2024-05-30T16:44:46.914621+0000","last_peered":"2024-05-30T16:44:46.914621+0000","last_clean":"2024-05-30T16:44:46.914621+0000","last_became_active":"2024-05-30T16:44:45.914901+0000","last_became_peered":"2024-05-30T16:44:45.914901+0000","last_unstale":"2024-05-30T16:44:46.914621+0000","last_undegraded":"2024-05-30T16:44:46.914621+0000","last_fullsized":"2024-05-30T16:44:46.914621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.886164+0000","last_change":"2024-05-30T16:44:44.884173+0000","last_active":"2024-05-30T16:44:44.886164+0000","last_peered":"2024-05-30T16:44:44.886164+0000","last_clean":"2024-05-30T16:44:44.886164+0000","last_became_active":"2024-05-30T16:44:44.883842+0000","last_became_peered":"2024-05-30T16:44:44.883842+0000","last_unstale":"2024-05-30T16:44:44.886164+0000","last_undegraded":"2024-05-30T16:44:44.886164+0000","last_fullsized":"2024-05-30T16:44:44.886164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:52.965427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866920+0000","last_change":"2024-05-30T16:44:44.866920+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866920+0000","last_undegraded":"2024-05-30T16:44:44.866920+0000","last_fullsized":"2024-05-30T16:44:44.866920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.895228+0000","last_change":"2024-05-30T16:44:44.888087+0000","last_active":"2024-05-30T16:44:44.895228+0000","last_peered":"2024-05-30T16:44:44.895228+0000","last_clean":"2024-05-30T16:44:44.895228+0000","last_became_active":"2024-05-30T16:44:44.887807+0000","last_became_peered":"2024-05-30T16:44:44.887807+0000","last_unstale":"2024-05-30T16:44:44.895228+0000","last_undegraded":"2024-05-30T16:44:44.895228+0000","last_fullsized":"2024-05-30T16:44:44.895228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:53.783765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.864963+0000","last_change":"2024-05-30T16:44:44.864963+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.864963+0000","last_undegraded":"2024-05-30T16:44:44.864963+0000","last_fullsized":"2024-05-30T16:44:44.864963+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894088+0000","last_change":"2024-05-30T16:44:44.891918+0000","last_active":"2024-05-30T16:44:44.894088+0000","last_peered":"2024-05-30T16:44:44.894088+0000","last_clean":"2024-05-30T16:44:44.894088+0000","last_became_active":"2024-05-30T16:44:44.891134+0000","last_became_peered":"2024-05-30T16:44:44.891134+0000","last_unstale":"2024-05-30T16:44:44.894088+0000","last_undegraded":"2024-05-30T16:44:44.894088+0000","last_fullsized":"2024-05-30T16:44:44.894088+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:14.335974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913160+0000","last_change":"2024-05-30T16:44:45.931105+0000","last_active":"2024-05-30T16:44:46.913160+0000","last_peered":"2024-05-30T16:44:46.913160+0000","last_clean":"2024-05-30T16:44:46.913160+0000","last_became_active":"2024-05-30T16:44:45.929921+0000","last_became_peered":"2024-05-30T16:44:45.929921+0000","last_unstale":"2024-05-30T16:44:46.913160+0000","last_undegraded":"2024-05-30T16:44:46.913160+0000","last_fullsized":"2024-05-30T16:44:46.913160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893925+0000","last_change":"2024-05-30T16:44:44.889587+0000","last_active":"2024-05-30T16:44:44.893925+0000","last_peered":"2024-05-30T16:44:44.893925+0000","last_clean":"2024-05-30T16:44:44.893925+0000","last_became_active":"2024-05-30T16:44:44.889226+0000","last_became_peered":"2024-05-30T16:44:44.889226+0000","last_unstale":"2024-05-30T16:44:44.893925+0000","last_undegraded":"2024-05-30T16:44:44.893925+0000","last_fullsized":"2024-05-30T16:44:44.893925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:34.292596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870707+0000","last_change":"2024-05-30T16:44:44.870707+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870707+0000","last_undegraded":"2024-05-30T16:44:44.870707+0000","last_fullsized":"2024-05-30T16:44:44.870707+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.879106+0000","last_change":"2024-05-30T16:44:44.871442+0000","last_active":"2024-05-30T16:44:44.879106+0000","last_peered":"2024-05-30T16:44:44.879106+0000","last_clean":"2024-05-30T16:44:44.879106+0000","last_became_active":"2024-05-30T16:44:44.871162+0000","last_became_peered":"2024-05-30T16:44:44.871162+0000","last_unstale":"2024-05-30T16:44:44.879106+0000","last_undegraded":"2024-05-30T16:44:44.879106+0000","last_fullsized":"2024-05-30T16:44:44.879106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:14.335974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.863239+0000","last_change":"2024-05-30T16:44:44.863239+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.863239+0000","last_undegraded":"2024-05-30T16:44:44.863239+0000","last_fullsized":"2024-05-30T16:44:44.863239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.891914+0000","last_change":"2024-05-30T16:44:44.886316+0000","last_active":"2024-05-30T16:44:44.891914+0000","last_peered":"2024-05-30T16:44:44.891914+0000","last_clean":"2024-05-30T16:44:44.891914+0000","last_became_active":"2024-05-30T16:44:44.885953+0000","last_became_peered":"2024-05-30T16:44:44.885953+0000","last_unstale":"2024-05-30T16:44:44.891914+0000","last_undegraded":"2024-05-30T16:44:44.891914+0000","last_fullsized":"2024-05-30T16:44:44.891914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:37.611751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915844+0000","last_change":"2024-05-30T16:44:45.910082+0000","last_active":"2024-05-30T16:44:46.915844+0000","last_peered":"2024-05-30T16:44:46.915844+0000","last_clean":"2024-05-30T16:44:46.915844+0000","last_became_active":"2024-05-30T16:44:45.909541+0000","last_became_peered":"2024-05-30T16:44:45.909541+0000","last_unstale":"2024-05-30T16:44:46.915844+0000","last_undegraded":"2024-05-30T16:44:46.915844+0000","last_fullsized":"2024-05-30T16:44:46.915844+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.895123+0000","last_change":"2024-05-30T16:44:44.888958+0000","last_active":"2024-05-30T16:44:44.895123+0000","last_peered":"2024-05-30T16:44:44.895123+0000","last_clean":"2024-05-30T16:44:44.895123+0000","last_became_active":"2024-05-30T16:44:44.888629+0000","last_became_peered":"2024-05-30T16:44:44.888629+0000","last_unstale":"2024-05-30T16:44:44.895123+0000","last_undegraded":"2024-05-30T16:44:44.895123+0000","last_fullsized":"2024-05-30T16:44:44.895123+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:23:49.576943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.871260+0000","last_change":"2024-05-30T16:44:44.871260+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.871260+0000","last_undegraded":"2024-05-30T16:44:44.871260+0000","last_fullsized":"2024-05-30T16:44:44.871260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.895035+0000","last_change":"2024-05-30T16:44:44.887204+0000","last_active":"2024-05-30T16:44:44.895035+0000","last_peered":"2024-05-30T16:44:44.895035+0000","last_clean":"2024-05-30T16:44:44.895035+0000","last_became_active":"2024-05-30T16:44:44.886702+0000","last_became_peered":"2024-05-30T16:44:44.886702+0000","last_unstale":"2024-05-30T16:44:44.895035+0000","last_undegraded":"2024-05-30T16:44:44.895035+0000","last_fullsized":"2024-05-30T16:44:44.895035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:07.538799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867090+0000","last_change":"2024-05-30T16:44:44.867090+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867090+0000","last_undegraded":"2024-05-30T16:44:44.867090+0000","last_fullsized":"2024-05-30T16:44:44.867090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.878752+0000","last_change":"2024-05-30T16:44:44.874074+0000","last_active":"2024-05-30T16:44:44.878752+0000","last_peered":"2024-05-30T16:44:44.878752+0000","last_clean":"2024-05-30T16:44:44.878752+0000","last_became_active":"2024-05-30T16:44:44.872396+0000","last_became_peered":"2024-05-30T16:44:44.872396+0000","last_unstale":"2024-05-30T16:44:44.878752+0000","last_undegraded":"2024-05-30T16:44:44.878752+0000","last_fullsized":"2024-05-30T16:44:44.878752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:28:38.953023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.868645+0000","last_change":"2024-05-30T16:44:44.868645+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.868645+0000","last_undegraded":"2024-05-30T16:44:44.868645+0000","last_fullsized":"2024-05-30T16:44:44.868645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932742+0000","last_change":"2024-05-30T16:44:44.881588+0000","last_active":"2024-05-30T16:44:46.932742+0000","last_peered":"2024-05-30T16:44:46.932742+0000","last_clean":"2024-05-30T16:44:46.932742+0000","last_became_active":"2024-05-30T16:44:44.881282+0000","last_became_peered":"2024-05-30T16:44:44.881282+0000","last_unstale":"2024-05-30T16:44:46.932742+0000","last_undegraded":"2024-05-30T16:44:46.932742+0000","last_fullsized":"2024-05-30T16:44:46.932742+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:27:30.795217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.865956+0000","last_change":"2024-05-30T16:44:44.865956+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.865956+0000","last_undegraded":"2024-05-30T16:44:44.865956+0000","last_fullsized":"2024-05-30T16:44:44.865956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893051+0000","last_change":"2024-05-30T16:44:44.890541+0000","last_active":"2024-05-30T16:44:44.893051+0000","last_peered":"2024-05-30T16:44:44.893051+0000","last_clean":"2024-05-30T16:44:44.893051+0000","last_became_active":"2024-05-30T16:44:44.890280+0000","last_became_peered":"2024-05-30T16:44:44.890280+0000","last_unstale":"2024-05-30T16:44:44.893051+0000","last_undegraded":"2024-05-30T16:44:44.893051+0000","last_fullsized":"2024-05-30T16:44:44.893051+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:43.725581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867121+0000","last_change":"2024-05-30T16:44:44.867121+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867121+0000","last_undegraded":"2024-05-30T16:44:44.867121+0000","last_fullsized":"2024-05-30T16:44:44.867121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894788+0000","last_change":"2024-05-30T16:44:44.885235+0000","last_active":"2024-05-30T16:44:44.894788+0000","last_peered":"2024-05-30T16:44:44.894788+0000","last_clean":"2024-05-30T16:44:44.894788+0000","last_became_active":"2024-05-30T16:44:44.884907+0000","last_became_peered":"2024-05-30T16:44:44.884907+0000","last_unstale":"2024-05-30T16:44:44.894788+0000","last_undegraded":"2024-05-30T16:44:44.894788+0000","last_fullsized":"2024-05-30T16:44:44.894788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:05.228927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913450+0000","last_change":"2024-05-30T16:44:45.919445+0000","last_active":"2024-05-30T16:44:46.913450+0000","last_peered":"2024-05-30T16:44:46.913450+0000","last_clean":"2024-05-30T16:44:46.913450+0000","last_became_active":"2024-05-30T16:44:45.919149+0000","last_became_peered":"2024-05-30T16:44:45.919149+0000","last_unstale":"2024-05-30T16:44:46.913450+0000","last_undegraded":"2024-05-30T16:44:46.913450+0000","last_fullsized":"2024-05-30T16:44:46.913450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913128+0000","last_change":"2024-05-30T16:44:44.887506+0000","last_active":"2024-05-30T16:44:46.913128+0000","last_peered":"2024-05-30T16:44:46.913128+0000","last_clean":"2024-05-30T16:44:46.913128+0000","last_became_active":"2024-05-30T16:44:44.887180+0000","last_became_peered":"2024-05-30T16:44:44.887180+0000","last_unstale":"2024-05-30T16:44:46.913128+0000","last_undegraded":"2024-05-30T16:44:46.913128+0000","last_fullsized":"2024-05-30T16:44:46.913128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:08.904901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.868406+0000","last_change":"2024-05-30T16:44:44.868406+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.868406+0000","last_undegraded":"2024-05-30T16:44:44.868406+0000","last_fullsized":"2024-05-30T16:44:44.868406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894070+0000","last_change":"2024-05-30T16:44:44.891749+0000","last_active":"2024-05-30T16:44:44.894070+0000","last_peered":"2024-05-30T16:44:44.894070+0000","last_clean":"2024-05-30T16:44:44.894070+0000","last_became_active":"2024-05-30T16:44:44.891440+0000","last_became_peered":"2024-05-30T16:44:44.891440+0000","last_unstale":"2024-05-30T16:44:44.894070+0000","last_undegraded":"2024-05-30T16:44:44.894070+0000","last_fullsized":"2024-05-30T16:44:44.894070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:37.347555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.916134+0000","last_change":"2024-05-30T16:44:45.911902+0000","last_active":"2024-05-30T16:44:46.916134+0000","last_peered":"2024-05-30T16:44:46.916134+0000","last_clean":"2024-05-30T16:44:46.916134+0000","last_became_active":"2024-05-30T16:44:45.911282+0000","last_became_peered":"2024-05-30T16:44:45.911282+0000","last_unstale":"2024-05-30T16:44:46.916134+0000","last_undegraded":"2024-05-30T16:44:46.916134+0000","last_fullsized":"2024-05-30T16:44:46.916134+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933055+0000","last_change":"2024-05-30T16:44:44.885333+0000","last_active":"2024-05-30T16:44:46.933055+0000","last_peered":"2024-05-30T16:44:46.933055+0000","last_clean":"2024-05-30T16:44:46.933055+0000","last_became_active":"2024-05-30T16:44:44.883685+0000","last_became_peered":"2024-05-30T16:44:44.883685+0000","last_unstale":"2024-05-30T16:44:46.933055+0000","last_undegraded":"2024-05-30T16:44:46.933055+0000","last_fullsized":"2024-05-30T16:44:46.933055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:52.935243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932751+0000","last_change":"2024-05-30T16:44:45.915538+0000","last_active":"2024-05-30T16:44:46.932751+0000","last_peered":"2024-05-30T16:44:46.932751+0000","last_clean":"2024-05-30T16:44:46.932751+0000","last_became_active":"2024-05-30T16:44:45.914763+0000","last_became_peered":"2024-05-30T16:44:45.914763+0000","last_unstale":"2024-05-30T16:44:46.932751+0000","last_undegraded":"2024-05-30T16:44:46.932751+0000","last_fullsized":"2024-05-30T16:44:46.932751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:30.231921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893559+0000","last_change":"2024-05-30T16:44:44.888789+0000","last_active":"2024-05-30T16:44:44.893559+0000","last_peered":"2024-05-30T16:44:44.893559+0000","last_clean":"2024-05-30T16:44:44.893559+0000","last_became_active":"2024-05-30T16:44:44.888193+0000","last_became_peered":"2024-05-30T16:44:44.888193+0000","last_unstale":"2024-05-30T16:44:44.893559+0000","last_undegraded":"2024-05-30T16:44:44.893559+0000","last_fullsized":"2024-05-30T16:44:44.893559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:54.736980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.876070+0000","last_change":"2024-05-30T16:44:44.876070+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.876070+0000","last_undegraded":"2024-05-30T16:44:44.876070+0000","last_fullsized":"2024-05-30T16:44:44.876070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933045+0000","last_change":"2024-05-30T16:44:44.892955+0000","last_active":"2024-05-30T16:44:46.933045+0000","last_peered":"2024-05-30T16:44:46.933045+0000","last_clean":"2024-05-30T16:44:46.933045+0000","last_became_active":"2024-05-30T16:44:44.892513+0000","last_became_peered":"2024-05-30T16:44:44.892513+0000","last_unstale":"2024-05-30T16:44:46.933045+0000","last_undegraded":"2024-05-30T16:44:46.933045+0000","last_fullsized":"2024-05-30T16:44:46.933045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:04:30.373290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932947+0000","last_change":"2024-05-30T16:44:45.910492+0000","last_active":"2024-05-30T16:44:46.932947+0000","last_peered":"2024-05-30T16:44:46.932947+0000","last_clean":"2024-05-30T16:44:46.932947+0000","last_became_active":"2024-05-30T16:44:45.909824+0000","last_became_peered":"2024-05-30T16:44:45.909824+0000","last_unstale":"2024-05-30T16:44:46.932947+0000","last_undegraded":"2024-05-30T16:44:46.932947+0000","last_fullsized":"2024-05-30T16:44:46.932947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:45.491068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.892063+0000","last_change":"2024-05-30T16:44:44.887201+0000","last_active":"2024-05-30T16:44:44.892063+0000","last_peered":"2024-05-30T16:44:44.892063+0000","last_clean":"2024-05-30T16:44:44.892063+0000","last_became_active":"2024-05-30T16:44:44.886748+0000","last_became_peered":"2024-05-30T16:44:44.886748+0000","last_unstale":"2024-05-30T16:44:44.892063+0000","last_undegraded":"2024-05-30T16:44:44.892063+0000","last_fullsized":"2024-05-30T16:44:44.892063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:43.725581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867588+0000","last_change":"2024-05-30T16:44:44.867588+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867588+0000","last_undegraded":"2024-05-30T16:44:44.867588+0000","last_fullsized":"2024-05-30T16:44:44.867588+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.892502+0000","last_change":"2024-05-30T16:44:44.882578+0000","last_active":"2024-05-30T16:44:44.892502+0000","last_peered":"2024-05-30T16:44:44.892502+0000","last_clean":"2024-05-30T16:44:44.892502+0000","last_became_active":"2024-05-30T16:44:44.882197+0000","last_became_peered":"2024-05-30T16:44:44.882197+0000","last_unstale":"2024-05-30T16:44:44.892502+0000","last_undegraded":"2024-05-30T16:44:44.892502+0000","last_fullsized":"2024-05-30T16:44:44.892502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:38:11.267149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.861724+0000","last_change":"2024-05-30T16:44:44.861724+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.861724+0000","last_undegraded":"2024-05-30T16:44:44.861724+0000","last_fullsized":"2024-05-30T16:44:44.861724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893846+0000","last_change":"2024-05-30T16:44:44.888466+0000","last_active":"2024-05-30T16:44:44.893846+0000","last_peered":"2024-05-30T16:44:44.893846+0000","last_clean":"2024-05-30T16:44:44.893846+0000","last_became_active":"2024-05-30T16:44:44.888164+0000","last_became_peered":"2024-05-30T16:44:44.888164+0000","last_unstale":"2024-05-30T16:44:44.893846+0000","last_undegraded":"2024-05-30T16:44:44.893846+0000","last_fullsized":"2024-05-30T16:44:44.893846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:33.985709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914547+0000","last_change":"2024-05-30T16:44:45.919696+0000","last_active":"2024-05-30T16:44:46.914547+0000","last_peered":"2024-05-30T16:44:46.914547+0000","last_clean":"2024-05-30T16:44:46.914547+0000","last_became_active":"2024-05-30T16:44:45.918248+0000","last_became_peered":"2024-05-30T16:44:45.918248+0000","last_unstale":"2024-05-30T16:44:46.914547+0000","last_undegraded":"2024-05-30T16:44:46.914547+0000","last_fullsized":"2024-05-30T16:44:46.914547+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.897520+0000","last_change":"2024-05-30T16:44:44.893954+0000","last_active":"2024-05-30T16:44:44.897520+0000","last_peered":"2024-05-30T16:44:44.897520+0000","last_clean":"2024-05-30T16:44:44.897520+0000","last_became_active":"2024-05-30T16:44:44.893629+0000","last_became_peered":"2024-05-30T16:44:44.893629+0000","last_unstale":"2024-05-30T16:44:44.897520+0000","last_undegraded":"2024-05-30T16:44:44.897520+0000","last_fullsized":"2024-05-30T16:44:44.897520+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:37.347555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913336+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:44:46.913336+0000","last_peered":"2024-05-30T16:44:46.913336+0000","last_clean":"2024-05-30T16:44:46.913336+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:44:46.913336+0000","last_undegraded":"2024-05-30T16:44:46.913336+0000","last_fullsized":"2024-05-30T16:44:46.913336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":133544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":133544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":139811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":139811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":133544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":743344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":153558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":743254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:44:50.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:44:50.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.625+0000 7f6da3e00640 1 Processor -- start 2024-05-30T16:44:50.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.626+0000 7f6da3e00640 1 -- start start 2024-05-30T16:44:50.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.627+0000 7f6da3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6da415e530 0x7f6da406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.627+0000 7f6da3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da40afff0 0x7f6da40b5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.627+0000 7f6da3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6da406e500 con 0x7f6da4070700 2024-05-30T16:44:50.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.627+0000 7f6da3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6da406e7e0 con 0x7f6da415e530 2024-05-30T16:44:50.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.627+0000 7f6da3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6da406eac0 con 0x7f6da40afff0 2024-05-30T16:44:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.628+0000 7f6da2a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6da415e530 0x7f6da406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.628+0000 7f6da3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da40afff0 0x7f6da40b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.628+0000 7f6da2a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6da415e530 0x7f6da406f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:35804/0 (socket says 172.21.15.67:35804) 2024-05-30T16:44:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.628+0000 7f6da2a00640 1 -- 172.21.15.67:0/841279632 learned_addr learned my addr 172.21.15.67:0/841279632 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.629+0000 7f6da2a00640 1 -- 172.21.15.67:0/841279632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da40afff0 msgr2=0x7f6da40b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.629+0000 7f6da2a00640 1 --2- 172.21.15.67:0/841279632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da40afff0 0x7f6da40b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.629+0000 7f6da2a00640 1 -- 172.21.15.67:0/841279632 >> v1:172.21.15.169:6789/0 conn(0x7f6da4070700 legacy=0x7f6da40af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.629+0000 7f6da2a00640 1 -- 172.21.15.67:0/841279632 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da406eda0 con 0x7f6da415e530 2024-05-30T16:44:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.630+0000 7f6da2a00640 1 --2- 172.21.15.67:0/841279632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da415e530 0x7f6da406f5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009870 tx=0x7f6d9402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b246579788d8a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da1600640 1 -- 172.21.15.67:0/841279632 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94037070 con 0x7f6da415e530 2024-05-30T16:44:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da1600640 1 -- 172.21.15.67:0/841279632 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d94032440 con 0x7f6da415e530 2024-05-30T16:44:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da1600640 1 -- 172.21.15.67:0/841279632 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94032810 con 0x7f6da415e530 2024-05-30T16:44:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da3e00640 1 -- 172.21.15.67:0/841279632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da415e530 msgr2=0x7f6da406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da3e00640 1 --2- 172.21.15.67:0/841279632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da415e530 0x7f6da406f5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009870 tx=0x7f6d9402a1c0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da3e00640 1 -- 172.21.15.67:0/841279632 shutdown_connections 2024-05-30T16:44:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da3e00640 1 --2- 172.21.15.67:0/841279632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da40afff0 0x7f6da40b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da3e00640 1 --2- 172.21.15.67:0/841279632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da415e530 0x7f6da406f5a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da3e00640 1 -- 172.21.15.67:0/841279632 >> 172.21.15.67:0/841279632 conn(0x7f6da406b900 msgr2=0x7f6da406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da3e00640 1 -- 172.21.15.67:0/841279632 shutdown_connections 2024-05-30T16:44:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.631+0000 7f6da3e00640 1 -- 172.21.15.67:0/841279632 wait complete. 2024-05-30T16:44:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.633+0000 7f6da3e00640 1 Processor -- start 2024-05-30T16:44:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.633+0000 7f6da3e00640 1 -- start start 2024-05-30T16:44:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.634+0000 7f6da3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da4070700 0x7f6da4151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.634+0000 7f6da3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da40afff0 0x7f6da4159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.634+0000 7f6da3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da415e530 0x7f6da415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.634+0000 7f6da3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6da40af570 con 0x7f6da40afff0 2024-05-30T16:44:50.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.634+0000 7f6da3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6da415e910 con 0x7f6da4070700 2024-05-30T16:44:50.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.634+0000 7f6da3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6da40af6f0 con 0x7f6da415e530 2024-05-30T16:44:50.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.634+0000 7f6da2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da40afff0 0x7f6da4159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da40afff0 0x7f6da4159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51144/0 (socket says 172.21.15.67:51144) 2024-05-30T16:44:50.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da2000640 1 -- 172.21.15.67:0/3897692440 learned_addr learned my addr 172.21.15.67:0/3897692440 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da2a00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da4070700 0x7f6da4151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da3400640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da415e530 0x7f6da415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da2000640 1 -- 172.21.15.67:0/3897692440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da415e530 msgr2=0x7f6da415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da2000640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da415e530 0x7f6da415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da2000640 1 -- 172.21.15.67:0/3897692440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da4070700 msgr2=0x7f6da4151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da2000640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da4070700 0x7f6da4151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da2000640 1 -- 172.21.15.67:0/3897692440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da415f660 con 0x7f6da40afff0 2024-05-30T16:44:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da3400640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da415e530 0x7f6da415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.635+0000 7f6da2000640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da40afff0 0x7f6da4159810 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c340 tx=0x7f6d9000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.636+0000 7f6d9aa00640 1 -- 172.21.15.67:0/3897692440 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90017070 con 0x7f6da40afff0 2024-05-30T16:44:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.636+0000 7f6d9aa00640 1 -- 172.21.15.67:0/3897692440 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d900052d0 con 0x7f6da40afff0 2024-05-30T16:44:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.636+0000 7f6d9aa00640 1 -- 172.21.15.67:0/3897692440 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90003850 con 0x7f6da40afff0 2024-05-30T16:44:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.636+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da415f900 con 0x7f6da40afff0 2024-05-30T16:44:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.636+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da415fec0 con 0x7f6da40afff0 2024-05-30T16:44:50.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.637+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da40689e0 con 0x7f6da40afff0 2024-05-30T16:44:50.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.638+0000 7f6d9aa00640 1 -- 172.21.15.67:0/3897692440 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d9001a070 con 0x7f6da40afff0 2024-05-30T16:44:50.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.639+0000 7f6d9aa00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d8406c8f0 0x7f6d8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:50.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.639+0000 7f6d9aa00640 1 -- 172.21.15.67:0/3897692440 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6d9008ae60 con 0x7f6da40afff0 2024-05-30T16:44:50.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.640+0000 7f6da2a00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d8406c8f0 0x7f6d8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.640+0000 7f6da2a00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d8406c8f0 0x7f6d8406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d94034000 tx=0x7f6d9402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:50.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.644+0000 7f6d9aa00640 1 -- 172.21.15.67:0/3897692440 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d90059280 con 0x7f6da40afff0 2024-05-30T16:44:50.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.847+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6da4070080 con 0x7f6d8406c8f0 2024-05-30T16:44:50.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.865+0000 7f6d9aa00640 1 -- 172.21.15.67:0/3897692440 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248689 (secure 0 0 0) 0x7f6da4070080 con 0x7f6d8406c8f0 2024-05-30T16:44:50.869 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:50.871 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:44:50.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d8406c8f0 msgr2=0x7f6d8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d8406c8f0 0x7f6d8406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d94034000 tx=0x7f6d9402e000 comp rx=0 tx=0).stop 2024-05-30T16:44:50.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da40afff0 msgr2=0x7f6da4159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:50.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da40afff0 0x7f6da4159810 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c340 tx=0x7f6d9000c800 comp rx=0 tx=0).stop 2024-05-30T16:44:50.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 shutdown_connections 2024-05-30T16:44:50.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d8406c8f0 0x7f6d8406edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6da415e530 0x7f6da415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da40afff0 0x7f6da4159810 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 --2- 172.21.15.67:0/3897692440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6da4070700 0x7f6da4151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:50.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.871+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 >> 172.21.15.67:0/3897692440 conn(0x7f6da406b900 msgr2=0x7f6da40b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:50.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.872+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 shutdown_connections 2024-05-30T16:44:50.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:50.872+0000 7f6da3e00640 1 -- 172.21.15.67:0/3897692440 wait complete. 2024-05-30T16:44:50.893 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-05-30T16:44:49.996577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":165,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220236,"kb_used_data":4056,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754484,"statfs":{"total":773094113280,"available":772868591616,"internally_reserved":0,"allocated":4153344,"data_stored":2320410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.871893+0000","last_change":"2024-05-30T16:44:44.871893+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.871893+0000","last_undegraded":"2024-05-30T16:44:44.871893+0000","last_fullsized":"2024-05-30T16:44:44.871893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870925+0000","last_change":"2024-05-30T16:44:44.870925+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870925+0000","last_undegraded":"2024-05-30T16:44:44.870925+0000","last_fullsized":"2024-05-30T16:44:44.870925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915144+0000","last_change":"2024-05-30T16:44:45.915063+0000","last_active":"2024-05-30T16:44:46.915144+0000","last_peered":"2024-05-30T16:44:46.915144+0000","last_clean":"2024-05-30T16:44:46.915144+0000","last_became_active":"2024-05-30T16:44:45.914534+0000","last_became_peered":"2024-05-30T16:44:45.914534+0000","last_unstale":"2024-05-30T16:44:46.915144+0000","last_undegraded":"2024-05-30T16:44:46.915144+0000","last_fullsized":"2024-05-30T16:44:46.915144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.871890+0000","last_change":"2024-05-30T16:44:44.871890+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.871890+0000","last_undegraded":"2024-05-30T16:44:44.871890+0000","last_fullsized":"2024-05-30T16:44:44.871890+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.869063+0000","last_change":"2024-05-30T16:44:44.869063+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.869063+0000","last_undegraded":"2024-05-30T16:44:44.869063+0000","last_fullsized":"2024-05-30T16:44:44.869063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.872128+0000","last_change":"2024-05-30T16:44:44.872128+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.872128+0000","last_undegraded":"2024-05-30T16:44:44.872128+0000","last_fullsized":"2024-05-30T16:44:44.872128+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914905+0000","last_change":"2024-05-30T16:44:45.922189+0000","last_active":"2024-05-30T16:44:46.914905+0000","last_peered":"2024-05-30T16:44:46.914905+0000","last_clean":"2024-05-30T16:44:46.914905+0000","last_became_active":"2024-05-30T16:44:45.921920+0000","last_became_peered":"2024-05-30T16:44:45.921920+0000","last_unstale":"2024-05-30T16:44:46.914905+0000","last_undegraded":"2024-05-30T16:44:46.914905+0000","last_fullsized":"2024-05-30T16:44:46.914905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:32:30.873632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912732+0000","last_change":"2024-05-30T16:44:45.918239+0000","last_active":"2024-05-30T16:44:46.912732+0000","last_peered":"2024-05-30T16:44:46.912732+0000","last_clean":"2024-05-30T16:44:46.912732+0000","last_became_active":"2024-05-30T16:44:45.917936+0000","last_became_peered":"2024-05-30T16:44:45.917936+0000","last_unstale":"2024-05-30T16:44:46.912732+0000","last_undegraded":"2024-05-30T16:44:46.912732+0000","last_fullsized":"2024-05-30T16:44:46.912732+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866406+0000","last_change":"2024-05-30T16:44:44.866406+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866406+0000","last_undegraded":"2024-05-30T16:44:44.866406+0000","last_fullsized":"2024-05-30T16:44:44.866406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.872840+0000","last_change":"2024-05-30T16:44:44.872840+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.872840+0000","last_undegraded":"2024-05-30T16:44:44.872840+0000","last_fullsized":"2024-05-30T16:44:44.872840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915320+0000","last_change":"2024-05-30T16:44:45.927692+0000","last_active":"2024-05-30T16:44:46.915320+0000","last_peered":"2024-05-30T16:44:46.915320+0000","last_clean":"2024-05-30T16:44:46.915320+0000","last_became_active":"2024-05-30T16:44:45.927381+0000","last_became_peered":"2024-05-30T16:44:45.927381+0000","last_unstale":"2024-05-30T16:44:46.915320+0000","last_undegraded":"2024-05-30T16:44:46.915320+0000","last_fullsized":"2024-05-30T16:44:46.915320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:20:16.734760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914592+0000","last_change":"2024-05-30T16:44:45.922504+0000","last_active":"2024-05-30T16:44:46.914592+0000","last_peered":"2024-05-30T16:44:46.914592+0000","last_clean":"2024-05-30T16:44:46.914592+0000","last_became_active":"2024-05-30T16:44:45.922230+0000","last_became_peered":"2024-05-30T16:44:45.922230+0000","last_unstale":"2024-05-30T16:44:46.914592+0000","last_undegraded":"2024-05-30T16:44:46.914592+0000","last_fullsized":"2024-05-30T16:44:46.914592+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867394+0000","last_change":"2024-05-30T16:44:44.867394+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867394+0000","last_undegraded":"2024-05-30T16:44:44.867394+0000","last_fullsized":"2024-05-30T16:44:44.867394+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870665+0000","last_change":"2024-05-30T16:44:44.870665+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870665+0000","last_undegraded":"2024-05-30T16:44:44.870665+0000","last_fullsized":"2024-05-30T16:44:44.870665+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914323+0000","last_change":"2024-05-30T16:44:45.912535+0000","last_active":"2024-05-30T16:44:46.914323+0000","last_peered":"2024-05-30T16:44:46.914323+0000","last_clean":"2024-05-30T16:44:46.914323+0000","last_became_active":"2024-05-30T16:44:45.911217+0000","last_became_peered":"2024-05-30T16:44:45.911217+0000","last_unstale":"2024-05-30T16:44:46.914323+0000","last_undegraded":"2024-05-30T16:44:46.914323+0000","last_fullsized":"2024-05-30T16:44:46.914323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:10:41.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914560+0000","last_change":"2024-05-30T16:44:45.925677+0000","last_active":"2024-05-30T16:44:46.914560+0000","last_peered":"2024-05-30T16:44:46.914560+0000","last_clean":"2024-05-30T16:44:46.914560+0000","last_became_active":"2024-05-30T16:44:45.925335+0000","last_became_peered":"2024-05-30T16:44:45.925335+0000","last_unstale":"2024-05-30T16:44:46.914560+0000","last_undegraded":"2024-05-30T16:44:46.914560+0000","last_fullsized":"2024-05-30T16:44:46.914560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:24:35.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915605+0000","last_change":"2024-05-30T16:44:45.921499+0000","last_active":"2024-05-30T16:44:46.915605+0000","last_peered":"2024-05-30T16:44:46.915605+0000","last_clean":"2024-05-30T16:44:46.915605+0000","last_became_active":"2024-05-30T16:44:45.921212+0000","last_became_peered":"2024-05-30T16:44:45.921212+0000","last_unstale":"2024-05-30T16:44:46.915605+0000","last_undegraded":"2024-05-30T16:44:46.915605+0000","last_fullsized":"2024-05-30T16:44:46.915605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:14.743364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.868154+0000","last_change":"2024-05-30T16:44:44.868154+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.868154+0000","last_undegraded":"2024-05-30T16:44:44.868154+0000","last_fullsized":"2024-05-30T16:44:44.868154+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.869345+0000","last_change":"2024-05-30T16:44:44.869345+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.869345+0000","last_undegraded":"2024-05-30T16:44:44.869345+0000","last_fullsized":"2024-05-30T16:44:44.869345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.875019+0000","last_change":"2024-05-30T16:44:44.875019+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.875019+0000","last_undegraded":"2024-05-30T16:44:44.875019+0000","last_fullsized":"2024-05-30T16:44:44.875019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866701+0000","last_change":"2024-05-30T16:44:44.866701+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866701+0000","last_undegraded":"2024-05-30T16:44:44.866701+0000","last_fullsized":"2024-05-30T16:44:44.866701+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.861135+0000","last_change":"2024-05-30T16:44:44.861135+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.861135+0000","last_undegraded":"2024-05-30T16:44:44.861135+0000","last_fullsized":"2024-05-30T16:44:44.861135+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932990+0000","last_change":"2024-05-30T16:44:45.920307+0000","last_active":"2024-05-30T16:44:46.932990+0000","last_peered":"2024-05-30T16:44:46.932990+0000","last_clean":"2024-05-30T16:44:46.932990+0000","last_became_active":"2024-05-30T16:44:45.918862+0000","last_became_peered":"2024-05-30T16:44:45.918862+0000","last_unstale":"2024-05-30T16:44:46.932990+0000","last_undegraded":"2024-05-30T16:44:46.932990+0000","last_fullsized":"2024-05-30T16:44:46.932990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:52:44.864855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912985+0000","last_change":"2024-05-30T16:44:45.909508+0000","last_active":"2024-05-30T16:44:46.912985+0000","last_peered":"2024-05-30T16:44:46.912985+0000","last_clean":"2024-05-30T16:44:46.912985+0000","last_became_active":"2024-05-30T16:44:45.904007+0000","last_became_peered":"2024-05-30T16:44:45.904007+0000","last_unstale":"2024-05-30T16:44:46.912985+0000","last_undegraded":"2024-05-30T16:44:46.912985+0000","last_fullsized":"2024-05-30T16:44:46.912985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:11.996704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.873189+0000","last_change":"2024-05-30T16:44:44.873189+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.873189+0000","last_undegraded":"2024-05-30T16:44:44.873189+0000","last_fullsized":"2024-05-30T16:44:44.873189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867724+0000","last_change":"2024-05-30T16:44:44.867724+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867724+0000","last_undegraded":"2024-05-30T16:44:44.867724+0000","last_fullsized":"2024-05-30T16:44:44.867724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870720+0000","last_change":"2024-05-30T16:44:44.870720+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870720+0000","last_undegraded":"2024-05-30T16:44:44.870720+0000","last_fullsized":"2024-05-30T16:44:44.870720+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915334+0000","last_change":"2024-05-30T16:44:45.908143+0000","last_active":"2024-05-30T16:44:46.915334+0000","last_peered":"2024-05-30T16:44:46.915334+0000","last_clean":"2024-05-30T16:44:46.915334+0000","last_became_active":"2024-05-30T16:44:45.907830+0000","last_became_peered":"2024-05-30T16:44:45.907830+0000","last_unstale":"2024-05-30T16:44:46.915334+0000","last_undegraded":"2024-05-30T16:44:46.915334+0000","last_fullsized":"2024-05-30T16:44:46.915334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.865560+0000","last_change":"2024-05-30T16:44:44.865560+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.865560+0000","last_undegraded":"2024-05-30T16:44:44.865560+0000","last_fullsized":"2024-05-30T16:44:44.865560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913062+0000","last_change":"2024-05-30T16:44:45.926086+0000","last_active":"2024-05-30T16:44:46.913062+0000","last_peered":"2024-05-30T16:44:46.913062+0000","last_clean":"2024-05-30T16:44:46.913062+0000","last_became_active":"2024-05-30T16:44:45.921839+0000","last_became_peered":"2024-05-30T16:44:45.921839+0000","last_unstale":"2024-05-30T16:44:46.913062+0000","last_undegraded":"2024-05-30T16:44:46.913062+0000","last_fullsized":"2024-05-30T16:44:46.913062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.872445+0000","last_change":"2024-05-30T16:44:44.872445+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.872445+0000","last_undegraded":"2024-05-30T16:44:44.872445+0000","last_fullsized":"2024-05-30T16:44:44.872445+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912803+0000","last_change":"2024-05-30T16:44:45.931329+0000","last_active":"2024-05-30T16:44:46.912803+0000","last_peered":"2024-05-30T16:44:46.912803+0000","last_clean":"2024-05-30T16:44:46.912803+0000","last_became_active":"2024-05-30T16:44:45.929807+0000","last_became_peered":"2024-05-30T16:44:45.929807+0000","last_unstale":"2024-05-30T16:44:46.912803+0000","last_undegraded":"2024-05-30T16:44:46.912803+0000","last_fullsized":"2024-05-30T16:44:46.912803+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933144+0000","last_change":"2024-05-30T16:44:45.918314+0000","last_active":"2024-05-30T16:44:46.933144+0000","last_peered":"2024-05-30T16:44:46.933144+0000","last_clean":"2024-05-30T16:44:46.933144+0000","last_became_active":"2024-05-30T16:44:45.917998+0000","last_became_peered":"2024-05-30T16:44:45.917998+0000","last_unstale":"2024-05-30T16:44:46.933144+0000","last_undegraded":"2024-05-30T16:44:46.933144+0000","last_fullsized":"2024-05-30T16:44:46.933144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:09.909703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.897452+0000","last_change":"2024-05-30T16:44:44.892355+0000","last_active":"2024-05-30T16:44:44.897452+0000","last_peered":"2024-05-30T16:44:44.897452+0000","last_clean":"2024-05-30T16:44:44.897452+0000","last_became_active":"2024-05-30T16:44:44.891889+0000","last_became_peered":"2024-05-30T16:44:44.891889+0000","last_unstale":"2024-05-30T16:44:44.897452+0000","last_undegraded":"2024-05-30T16:44:44.897452+0000","last_fullsized":"2024-05-30T16:44:44.897452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:54.736980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870784+0000","last_change":"2024-05-30T16:44:44.870784+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870784+0000","last_undegraded":"2024-05-30T16:44:44.870784+0000","last_fullsized":"2024-05-30T16:44:44.870784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932836+0000","last_change":"2024-05-30T16:44:44.889925+0000","last_active":"2024-05-30T16:44:46.932836+0000","last_peered":"2024-05-30T16:44:46.932836+0000","last_clean":"2024-05-30T16:44:46.932836+0000","last_became_active":"2024-05-30T16:44:44.889564+0000","last_became_peered":"2024-05-30T16:44:44.889564+0000","last_unstale":"2024-05-30T16:44:46.932836+0000","last_undegraded":"2024-05-30T16:44:46.932836+0000","last_fullsized":"2024-05-30T16:44:46.932836+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:45.960008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.861455+0000","last_change":"2024-05-30T16:44:44.861455+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.861455+0000","last_undegraded":"2024-05-30T16:44:44.861455+0000","last_fullsized":"2024-05-30T16:44:44.861455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.885660+0000","last_change":"2024-05-30T16:44:44.883317+0000","last_active":"2024-05-30T16:44:44.885660+0000","last_peered":"2024-05-30T16:44:44.885660+0000","last_clean":"2024-05-30T16:44:44.885660+0000","last_became_active":"2024-05-30T16:44:44.882994+0000","last_became_peered":"2024-05-30T16:44:44.882994+0000","last_unstale":"2024-05-30T16:44:44.885660+0000","last_undegraded":"2024-05-30T16:44:44.885660+0000","last_fullsized":"2024-05-30T16:44:44.885660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:02:28.009022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870120+0000","last_change":"2024-05-30T16:44:44.870120+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870120+0000","last_undegraded":"2024-05-30T16:44:44.870120+0000","last_fullsized":"2024-05-30T16:44:44.870120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.890431+0000","last_change":"2024-05-30T16:44:44.886536+0000","last_active":"2024-05-30T16:44:44.890431+0000","last_peered":"2024-05-30T16:44:44.890431+0000","last_clean":"2024-05-30T16:44:44.890431+0000","last_became_active":"2024-05-30T16:44:44.886261+0000","last_became_peered":"2024-05-30T16:44:44.886261+0000","last_unstale":"2024-05-30T16:44:44.890431+0000","last_undegraded":"2024-05-30T16:44:44.890431+0000","last_fullsized":"2024-05-30T16:44:44.890431+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:07.538799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913886+0000","last_change":"2024-05-30T16:44:45.936295+0000","last_active":"2024-05-30T16:44:46.913886+0000","last_peered":"2024-05-30T16:44:46.913886+0000","last_clean":"2024-05-30T16:44:46.913886+0000","last_became_active":"2024-05-30T16:44:45.935416+0000","last_became_peered":"2024-05-30T16:44:45.935416+0000","last_unstale":"2024-05-30T16:44:46.913886+0000","last_undegraded":"2024-05-30T16:44:46.913886+0000","last_fullsized":"2024-05-30T16:44:46.913886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894858+0000","last_change":"2024-05-30T16:44:44.886562+0000","last_active":"2024-05-30T16:44:44.894858+0000","last_peered":"2024-05-30T16:44:44.894858+0000","last_clean":"2024-05-30T16:44:44.894858+0000","last_became_active":"2024-05-30T16:44:44.886254+0000","last_became_peered":"2024-05-30T16:44:44.886254+0000","last_unstale":"2024-05-30T16:44:44.894858+0000","last_undegraded":"2024-05-30T16:44:44.894858+0000","last_fullsized":"2024-05-30T16:44:44.894858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:53.956014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913996+0000","last_change":"2024-05-30T16:44:45.905732+0000","last_active":"2024-05-30T16:44:46.913996+0000","last_peered":"2024-05-30T16:44:46.913996+0000","last_clean":"2024-05-30T16:44:46.913996+0000","last_became_active":"2024-05-30T16:44:45.905117+0000","last_became_peered":"2024-05-30T16:44:45.905117+0000","last_unstale":"2024-05-30T16:44:46.913996+0000","last_undegraded":"2024-05-30T16:44:46.913996+0000","last_fullsized":"2024-05-30T16:44:46.913996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:55:15.879718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.891719+0000","last_change":"2024-05-30T16:44:44.879442+0000","last_active":"2024-05-30T16:44:44.891719+0000","last_peered":"2024-05-30T16:44:44.891719+0000","last_clean":"2024-05-30T16:44:44.891719+0000","last_became_active":"2024-05-30T16:44:44.877334+0000","last_became_peered":"2024-05-30T16:44:44.877334+0000","last_unstale":"2024-05-30T16:44:44.891719+0000","last_undegraded":"2024-05-30T16:44:44.891719+0000","last_fullsized":"2024-05-30T16:44:44.891719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:18.355954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.860816+0000","last_change":"2024-05-30T16:44:44.860816+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.860816+0000","last_undegraded":"2024-05-30T16:44:44.860816+0000","last_fullsized":"2024-05-30T16:44:44.860816+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.879443+0000","last_change":"2024-05-30T16:44:44.876950+0000","last_active":"2024-05-30T16:44:44.879443+0000","last_peered":"2024-05-30T16:44:44.879443+0000","last_clean":"2024-05-30T16:44:44.879443+0000","last_became_active":"2024-05-30T16:44:44.876666+0000","last_became_peered":"2024-05-30T16:44:44.876666+0000","last_unstale":"2024-05-30T16:44:44.879443+0000","last_undegraded":"2024-05-30T16:44:44.879443+0000","last_fullsized":"2024-05-30T16:44:44.879443+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:44.589105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867195+0000","last_change":"2024-05-30T16:44:44.867195+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867195+0000","last_undegraded":"2024-05-30T16:44:44.867195+0000","last_fullsized":"2024-05-30T16:44:44.867195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894960+0000","last_change":"2024-05-30T16:44:44.887604+0000","last_active":"2024-05-30T16:44:44.894960+0000","last_peered":"2024-05-30T16:44:44.894960+0000","last_clean":"2024-05-30T16:44:44.894960+0000","last_became_active":"2024-05-30T16:44:44.887259+0000","last_became_peered":"2024-05-30T16:44:44.887259+0000","last_unstale":"2024-05-30T16:44:44.894960+0000","last_undegraded":"2024-05-30T16:44:44.894960+0000","last_fullsized":"2024-05-30T16:44:44.894960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:34:29.911545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915928+0000","last_change":"2024-05-30T16:44:45.918490+0000","last_active":"2024-05-30T16:44:46.915928+0000","last_peered":"2024-05-30T16:44:46.915928+0000","last_clean":"2024-05-30T16:44:46.915928+0000","last_became_active":"2024-05-30T16:44:45.917451+0000","last_became_peered":"2024-05-30T16:44:45.917451+0000","last_unstale":"2024-05-30T16:44:46.915928+0000","last_undegraded":"2024-05-30T16:44:46.915928+0000","last_fullsized":"2024-05-30T16:44:46.915928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893908+0000","last_change":"2024-05-30T16:44:44.889369+0000","last_active":"2024-05-30T16:44:44.893908+0000","last_peered":"2024-05-30T16:44:44.893908+0000","last_clean":"2024-05-30T16:44:44.893908+0000","last_became_active":"2024-05-30T16:44:44.889051+0000","last_became_peered":"2024-05-30T16:44:44.889051+0000","last_unstale":"2024-05-30T16:44:44.893908+0000","last_undegraded":"2024-05-30T16:44:44.893908+0000","last_fullsized":"2024-05-30T16:44:44.893908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:25.424530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866906+0000","last_change":"2024-05-30T16:44:44.866906+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866906+0000","last_undegraded":"2024-05-30T16:44:44.866906+0000","last_fullsized":"2024-05-30T16:44:44.866906+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932945+0000","last_change":"2024-05-30T16:44:44.885815+0000","last_active":"2024-05-30T16:44:46.932945+0000","last_peered":"2024-05-30T16:44:46.932945+0000","last_clean":"2024-05-30T16:44:46.932945+0000","last_became_active":"2024-05-30T16:44:44.885007+0000","last_became_peered":"2024-05-30T16:44:44.885007+0000","last_unstale":"2024-05-30T16:44:46.932945+0000","last_undegraded":"2024-05-30T16:44:46.932945+0000","last_fullsized":"2024-05-30T16:44:46.932945+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:24:02.943052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866137+0000","last_change":"2024-05-30T16:44:44.866137+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866137+0000","last_undegraded":"2024-05-30T16:44:44.866137+0000","last_fullsized":"2024-05-30T16:44:44.866137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915318+0000","last_change":"2024-05-30T16:44:44.891929+0000","last_active":"2024-05-30T16:44:46.915318+0000","last_peered":"2024-05-30T16:44:46.915318+0000","last_clean":"2024-05-30T16:44:46.915318+0000","last_became_active":"2024-05-30T16:44:44.891611+0000","last_became_peered":"2024-05-30T16:44:44.891611+0000","last_unstale":"2024-05-30T16:44:46.915318+0000","last_undegraded":"2024-05-30T16:44:46.915318+0000","last_fullsized":"2024-05-30T16:44:46.915318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:41.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914621+0000","last_change":"2024-05-30T16:44:45.915178+0000","last_active":"2024-05-30T16:44:46.914621+0000","last_peered":"2024-05-30T16:44:46.914621+0000","last_clean":"2024-05-30T16:44:46.914621+0000","last_became_active":"2024-05-30T16:44:45.914901+0000","last_became_peered":"2024-05-30T16:44:45.914901+0000","last_unstale":"2024-05-30T16:44:46.914621+0000","last_undegraded":"2024-05-30T16:44:46.914621+0000","last_fullsized":"2024-05-30T16:44:46.914621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.886164+0000","last_change":"2024-05-30T16:44:44.884173+0000","last_active":"2024-05-30T16:44:44.886164+0000","last_peered":"2024-05-30T16:44:44.886164+0000","last_clean":"2024-05-30T16:44:44.886164+0000","last_became_active":"2024-05-30T16:44:44.883842+0000","last_became_peered":"2024-05-30T16:44:44.883842+0000","last_unstale":"2024-05-30T16:44:44.886164+0000","last_undegraded":"2024-05-30T16:44:44.886164+0000","last_fullsized":"2024-05-30T16:44:44.886164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:52.965427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866920+0000","last_change":"2024-05-30T16:44:44.866920+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866920+0000","last_undegraded":"2024-05-30T16:44:44.866920+0000","last_fullsized":"2024-05-30T16:44:44.866920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.895228+0000","last_change":"2024-05-30T16:44:44.888087+0000","last_active":"2024-05-30T16:44:44.895228+0000","last_peered":"2024-05-30T16:44:44.895228+0000","last_clean":"2024-05-30T16:44:44.895228+0000","last_became_active":"2024-05-30T16:44:44.887807+0000","last_became_peered":"2024-05-30T16:44:44.887807+0000","last_unstale":"2024-05-30T16:44:44.895228+0000","last_undegraded":"2024-05-30T16:44:44.895228+0000","last_fullsized":"2024-05-30T16:44:44.895228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:53.783765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.864963+0000","last_change":"2024-05-30T16:44:44.864963+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.864963+0000","last_undegraded":"2024-05-30T16:44:44.864963+0000","last_fullsized":"2024-05-30T16:44:44.864963+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894088+0000","last_change":"2024-05-30T16:44:44.891918+0000","last_active":"2024-05-30T16:44:44.894088+0000","last_peered":"2024-05-30T16:44:44.894088+0000","last_clean":"2024-05-30T16:44:44.894088+0000","last_became_active":"2024-05-30T16:44:44.891134+0000","last_became_peered":"2024-05-30T16:44:44.891134+0000","last_unstale":"2024-05-30T16:44:44.894088+0000","last_undegraded":"2024-05-30T16:44:44.894088+0000","last_fullsized":"2024-05-30T16:44:44.894088+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:14.335974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913160+0000","last_change":"2024-05-30T16:44:45.931105+0000","last_active":"2024-05-30T16:44:46.913160+0000","last_peered":"2024-05-30T16:44:46.913160+0000","last_clean":"2024-05-30T16:44:46.913160+0000","last_became_active":"2024-05-30T16:44:45.929921+0000","last_became_peered":"2024-05-30T16:44:45.929921+0000","last_unstale":"2024-05-30T16:44:46.913160+0000","last_undegraded":"2024-05-30T16:44:46.913160+0000","last_fullsized":"2024-05-30T16:44:46.913160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893925+0000","last_change":"2024-05-30T16:44:44.889587+0000","last_active":"2024-05-30T16:44:44.893925+0000","last_peered":"2024-05-30T16:44:44.893925+0000","last_clean":"2024-05-30T16:44:44.893925+0000","last_became_active":"2024-05-30T16:44:44.889226+0000","last_became_peered":"2024-05-30T16:44:44.889226+0000","last_unstale":"2024-05-30T16:44:44.893925+0000","last_undegraded":"2024-05-30T16:44:44.893925+0000","last_fullsized":"2024-05-30T16:44:44.893925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:34.292596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870707+0000","last_change":"2024-05-30T16:44:44.870707+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870707+0000","last_undegraded":"2024-05-30T16:44:44.870707+0000","last_fullsized":"2024-05-30T16:44:44.870707+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.879106+0000","last_change":"2024-05-30T16:44:44.871442+0000","last_active":"2024-05-30T16:44:44.879106+0000","last_peered":"2024-05-30T16:44:44.879106+0000","last_clean":"2024-05-30T16:44:44.879106+0000","last_became_active":"2024-05-30T16:44:44.871162+0000","last_became_peered":"2024-05-30T16:44:44.871162+0000","last_unstale":"2024-05-30T16:44:44.879106+0000","last_undegraded":"2024-05-30T16:44:44.879106+0000","last_fullsized":"2024-05-30T16:44:44.879106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:14.335974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.863239+0000","last_change":"2024-05-30T16:44:44.863239+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.863239+0000","last_undegraded":"2024-05-30T16:44:44.863239+0000","last_fullsized":"2024-05-30T16:44:44.863239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.891914+0000","last_change":"2024-05-30T16:44:44.886316+0000","last_active":"2024-05-30T16:44:44.891914+0000","last_peered":"2024-05-30T16:44:44.891914+0000","last_clean":"2024-05-30T16:44:44.891914+0000","last_became_active":"2024-05-30T16:44:44.885953+0000","last_became_peered":"2024-05-30T16:44:44.885953+0000","last_unstale":"2024-05-30T16:44:44.891914+0000","last_undegraded":"2024-05-30T16:44:44.891914+0000","last_fullsized":"2024-05-30T16:44:44.891914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:37.611751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915844+0000","last_change":"2024-05-30T16:44:45.910082+0000","last_active":"2024-05-30T16:44:46.915844+0000","last_peered":"2024-05-30T16:44:46.915844+0000","last_clean":"2024-05-30T16:44:46.915844+0000","last_became_active":"2024-05-30T16:44:45.909541+0000","last_became_peered":"2024-05-30T16:44:45.909541+0000","last_unstale":"2024-05-30T16:44:46.915844+0000","last_undegraded":"2024-05-30T16:44:46.915844+0000","last_fullsized":"2024-05-30T16:44:46.915844+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.895123+0000","last_change":"2024-05-30T16:44:44.888958+0000","last_active":"2024-05-30T16:44:44.895123+0000","last_peered":"2024-05-30T16:44:44.895123+0000","last_clean":"2024-05-30T16:44:44.895123+0000","last_became_active":"2024-05-30T16:44:44.888629+0000","last_became_peered":"2024-05-30T16:44:44.888629+0000","last_unstale":"2024-05-30T16:44:44.895123+0000","last_undegraded":"2024-05-30T16:44:44.895123+0000","last_fullsized":"2024-05-30T16:44:44.895123+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:23:49.576943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.871260+0000","last_change":"2024-05-30T16:44:44.871260+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.871260+0000","last_undegraded":"2024-05-30T16:44:44.871260+0000","last_fullsized":"2024-05-30T16:44:44.871260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.895035+0000","last_change":"2024-05-30T16:44:44.887204+0000","last_active":"2024-05-30T16:44:44.895035+0000","last_peered":"2024-05-30T16:44:44.895035+0000","last_clean":"2024-05-30T16:44:44.895035+0000","last_became_active":"2024-05-30T16:44:44.886702+0000","last_became_peered":"2024-05-30T16:44:44.886702+0000","last_unstale":"2024-05-30T16:44:44.895035+0000","last_undegraded":"2024-05-30T16:44:44.895035+0000","last_fullsized":"2024-05-30T16:44:44.895035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:07.538799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867090+0000","last_change":"2024-05-30T16:44:44.867090+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867090+0000","last_undegraded":"2024-05-30T16:44:44.867090+0000","last_fullsized":"2024-05-30T16:44:44.867090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.878752+0000","last_change":"2024-05-30T16:44:44.874074+0000","last_active":"2024-05-30T16:44:44.878752+0000","last_peered":"2024-05-30T16:44:44.878752+0000","last_clean":"2024-05-30T16:44:44.878752+0000","last_became_active":"2024-05-30T16:44:44.872396+0000","last_became_peered":"2024-05-30T16:44:44.872396+0000","last_unstale":"2024-05-30T16:44:44.878752+0000","last_undegraded":"2024-05-30T16:44:44.878752+0000","last_fullsized":"2024-05-30T16:44:44.878752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:28:38.953023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.868645+0000","last_change":"2024-05-30T16:44:44.868645+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.868645+0000","last_undegraded":"2024-05-30T16:44:44.868645+0000","last_fullsized":"2024-05-30T16:44:44.868645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932742+0000","last_change":"2024-05-30T16:44:44.881588+0000","last_active":"2024-05-30T16:44:46.932742+0000","last_peered":"2024-05-30T16:44:46.932742+0000","last_clean":"2024-05-30T16:44:46.932742+0000","last_became_active":"2024-05-30T16:44:44.881282+0000","last_became_peered":"2024-05-30T16:44:44.881282+0000","last_unstale":"2024-05-30T16:44:46.932742+0000","last_undegraded":"2024-05-30T16:44:46.932742+0000","last_fullsized":"2024-05-30T16:44:46.932742+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:27:30.795217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.865956+0000","last_change":"2024-05-30T16:44:44.865956+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.865956+0000","last_undegraded":"2024-05-30T16:44:44.865956+0000","last_fullsized":"2024-05-30T16:44:44.865956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893051+0000","last_change":"2024-05-30T16:44:44.890541+0000","last_active":"2024-05-30T16:44:44.893051+0000","last_peered":"2024-05-30T16:44:44.893051+0000","last_clean":"2024-05-30T16:44:44.893051+0000","last_became_active":"2024-05-30T16:44:44.890280+0000","last_became_peered":"2024-05-30T16:44:44.890280+0000","last_unstale":"2024-05-30T16:44:44.893051+0000","last_undegraded":"2024-05-30T16:44:44.893051+0000","last_fullsized":"2024-05-30T16:44:44.893051+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:43.725581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867121+0000","last_change":"2024-05-30T16:44:44.867121+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867121+0000","last_undegraded":"2024-05-30T16:44:44.867121+0000","last_fullsized":"2024-05-30T16:44:44.867121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894788+0000","last_change":"2024-05-30T16:44:44.885235+0000","last_active":"2024-05-30T16:44:44.894788+0000","last_peered":"2024-05-30T16:44:44.894788+0000","last_clean":"2024-05-30T16:44:44.894788+0000","last_became_active":"2024-05-30T16:44:44.884907+0000","last_became_peered":"2024-05-30T16:44:44.884907+0000","last_unstale":"2024-05-30T16:44:44.894788+0000","last_undegraded":"2024-05-30T16:44:44.894788+0000","last_fullsized":"2024-05-30T16:44:44.894788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:05.228927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913450+0000","last_change":"2024-05-30T16:44:45.919445+0000","last_active":"2024-05-30T16:44:46.913450+0000","last_peered":"2024-05-30T16:44:46.913450+0000","last_clean":"2024-05-30T16:44:46.913450+0000","last_became_active":"2024-05-30T16:44:45.919149+0000","last_became_peered":"2024-05-30T16:44:45.919149+0000","last_unstale":"2024-05-30T16:44:46.913450+0000","last_undegraded":"2024-05-30T16:44:46.913450+0000","last_fullsized":"2024-05-30T16:44:46.913450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913128+0000","last_change":"2024-05-30T16:44:44.887506+0000","last_active":"2024-05-30T16:44:46.913128+0000","last_peered":"2024-05-30T16:44:46.913128+0000","last_clean":"2024-05-30T16:44:46.913128+0000","last_became_active":"2024-05-30T16:44:44.887180+0000","last_became_peered":"2024-05-30T16:44:44.887180+0000","last_unstale":"2024-05-30T16:44:46.913128+0000","last_undegraded":"2024-05-30T16:44:46.913128+0000","last_fullsized":"2024-05-30T16:44:46.913128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:08.904901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.868406+0000","last_change":"2024-05-30T16:44:44.868406+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.868406+0000","last_undegraded":"2024-05-30T16:44:44.868406+0000","last_fullsized":"2024-05-30T16:44:44.868406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894070+0000","last_change":"2024-05-30T16:44:44.891749+0000","last_active":"2024-05-30T16:44:44.894070+0000","last_peered":"2024-05-30T16:44:44.894070+0000","last_clean":"2024-05-30T16:44:44.894070+0000","last_became_active":"2024-05-30T16:44:44.891440+0000","last_became_peered":"2024-05-30T16:44:44.891440+0000","last_unstale":"2024-05-30T16:44:44.894070+0000","last_undegraded":"2024-05-30T16:44:44.894070+0000","last_fullsized":"2024-05-30T16:44:44.894070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:37.347555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.916134+0000","last_change":"2024-05-30T16:44:45.911902+0000","last_active":"2024-05-30T16:44:46.916134+0000","last_peered":"2024-05-30T16:44:46.916134+0000","last_clean":"2024-05-30T16:44:46.916134+0000","last_became_active":"2024-05-30T16:44:45.911282+0000","last_became_peered":"2024-05-30T16:44:45.911282+0000","last_unstale":"2024-05-30T16:44:46.916134+0000","last_undegraded":"2024-05-30T16:44:46.916134+0000","last_fullsized":"2024-05-30T16:44:46.916134+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933055+0000","last_change":"2024-05-30T16:44:44.885333+0000","last_active":"2024-05-30T16:44:46.933055+0000","last_peered":"2024-05-30T16:44:46.933055+0000","last_clean":"2024-05-30T16:44:46.933055+0000","last_became_active":"2024-05-30T16:44:44.883685+0000","last_became_peered":"2024-05-30T16:44:44.883685+0000","last_unstale":"2024-05-30T16:44:46.933055+0000","last_undegraded":"2024-05-30T16:44:46.933055+0000","last_fullsized":"2024-05-30T16:44:46.933055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:52.935243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932751+0000","last_change":"2024-05-30T16:44:45.915538+0000","last_active":"2024-05-30T16:44:46.932751+0000","last_peered":"2024-05-30T16:44:46.932751+0000","last_clean":"2024-05-30T16:44:46.932751+0000","last_became_active":"2024-05-30T16:44:45.914763+0000","last_became_peered":"2024-05-30T16:44:45.914763+0000","last_unstale":"2024-05-30T16:44:46.932751+0000","last_undegraded":"2024-05-30T16:44:46.932751+0000","last_fullsized":"2024-05-30T16:44:46.932751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:30.231921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893559+0000","last_change":"2024-05-30T16:44:44.888789+0000","last_active":"2024-05-30T16:44:44.893559+0000","last_peered":"2024-05-30T16:44:44.893559+0000","last_clean":"2024-05-30T16:44:44.893559+0000","last_became_active":"2024-05-30T16:44:44.888193+0000","last_became_peered":"2024-05-30T16:44:44.888193+0000","last_unstale":"2024-05-30T16:44:44.893559+0000","last_undegraded":"2024-05-30T16:44:44.893559+0000","last_fullsized":"2024-05-30T16:44:44.893559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:54.736980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.876070+0000","last_change":"2024-05-30T16:44:44.876070+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.876070+0000","last_undegraded":"2024-05-30T16:44:44.876070+0000","last_fullsized":"2024-05-30T16:44:44.876070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933045+0000","last_change":"2024-05-30T16:44:44.892955+0000","last_active":"2024-05-30T16:44:46.933045+0000","last_peered":"2024-05-30T16:44:46.933045+0000","last_clean":"2024-05-30T16:44:46.933045+0000","last_became_active":"2024-05-30T16:44:44.892513+0000","last_became_peered":"2024-05-30T16:44:44.892513+0000","last_unstale":"2024-05-30T16:44:46.933045+0000","last_undegraded":"2024-05-30T16:44:46.933045+0000","last_fullsized":"2024-05-30T16:44:46.933045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:04:30.373290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932947+0000","last_change":"2024-05-30T16:44:45.910492+0000","last_active":"2024-05-30T16:44:46.932947+0000","last_peered":"2024-05-30T16:44:46.932947+0000","last_clean":"2024-05-30T16:44:46.932947+0000","last_became_active":"2024-05-30T16:44:45.909824+0000","last_became_peered":"2024-05-30T16:44:45.909824+0000","last_unstale":"2024-05-30T16:44:46.932947+0000","last_undegraded":"2024-05-30T16:44:46.932947+0000","last_fullsized":"2024-05-30T16:44:46.932947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:45.491068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.892063+0000","last_change":"2024-05-30T16:44:44.887201+0000","last_active":"2024-05-30T16:44:44.892063+0000","last_peered":"2024-05-30T16:44:44.892063+0000","last_clean":"2024-05-30T16:44:44.892063+0000","last_became_active":"2024-05-30T16:44:44.886748+0000","last_became_peered":"2024-05-30T16:44:44.886748+0000","last_unstale":"2024-05-30T16:44:44.892063+0000","last_undegraded":"2024-05-30T16:44:44.892063+0000","last_fullsized":"2024-05-30T16:44:44.892063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:43.725581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867588+0000","last_change":"2024-05-30T16:44:44.867588+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867588+0000","last_undegraded":"2024-05-30T16:44:44.867588+0000","last_fullsized":"2024-05-30T16:44:44.867588+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.892502+0000","last_change":"2024-05-30T16:44:44.882578+0000","last_active":"2024-05-30T16:44:44.892502+0000","last_peered":"2024-05-30T16:44:44.892502+0000","last_clean":"2024-05-30T16:44:44.892502+0000","last_became_active":"2024-05-30T16:44:44.882197+0000","last_became_peered":"2024-05-30T16:44:44.882197+0000","last_unstale":"2024-05-30T16:44:44.892502+0000","last_undegraded":"2024-05-30T16:44:44.892502+0000","last_fullsized":"2024-05-30T16:44:44.892502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:38:11.267149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.861724+0000","last_change":"2024-05-30T16:44:44.861724+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.861724+0000","last_undegraded":"2024-05-30T16:44:44.861724+0000","last_fullsized":"2024-05-30T16:44:44.861724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893846+0000","last_change":"2024-05-30T16:44:44.888466+0000","last_active":"2024-05-30T16:44:44.893846+0000","last_peered":"2024-05-30T16:44:44.893846+0000","last_clean":"2024-05-30T16:44:44.893846+0000","last_became_active":"2024-05-30T16:44:44.888164+0000","last_became_peered":"2024-05-30T16:44:44.888164+0000","last_unstale":"2024-05-30T16:44:44.893846+0000","last_undegraded":"2024-05-30T16:44:44.893846+0000","last_fullsized":"2024-05-30T16:44:44.893846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:33.985709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914547+0000","last_change":"2024-05-30T16:44:45.919696+0000","last_active":"2024-05-30T16:44:46.914547+0000","last_peered":"2024-05-30T16:44:46.914547+0000","last_clean":"2024-05-30T16:44:46.914547+0000","last_became_active":"2024-05-30T16:44:45.918248+0000","last_became_peered":"2024-05-30T16:44:45.918248+0000","last_unstale":"2024-05-30T16:44:46.914547+0000","last_undegraded":"2024-05-30T16:44:46.914547+0000","last_fullsized":"2024-05-30T16:44:46.914547+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.897520+0000","last_change":"2024-05-30T16:44:44.893954+0000","last_active":"2024-05-30T16:44:44.897520+0000","last_peered":"2024-05-30T16:44:44.897520+0000","last_clean":"2024-05-30T16:44:44.897520+0000","last_became_active":"2024-05-30T16:44:44.893629+0000","last_became_peered":"2024-05-30T16:44:44.893629+0000","last_unstale":"2024-05-30T16:44:44.897520+0000","last_undegraded":"2024-05-30T16:44:44.897520+0000","last_fullsized":"2024-05-30T16:44:44.897520+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:37.347555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913336+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:44:46.913336+0000","last_peered":"2024-05-30T16:44:46.913336+0000","last_clean":"2024-05-30T16:44:46.913336+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:44:46.913336+0000","last_undegraded":"2024-05-30T16:44:46.913336+0000","last_fullsized":"2024-05-30T16:44:46.913336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":133544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":133544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":139811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":139811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":133544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":743344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":153558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":743254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:44:50.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:44:51.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.146+0000 7fbe9fe00640 1 Processor -- start 2024-05-30T16:44:51.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.146+0000 7fbe9fe00640 1 -- start start 2024-05-30T16:44:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.148+0000 7fbe9fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbea0156b30 0x7fbea00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.148+0000 7fbe9fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbea006ee40 0x7fbea00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.148+0000 7fbe9fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbea0156f10 con 0x7fbea006f2f0 2024-05-30T16:44:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.148+0000 7fbe9fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fbea006dda0 con 0x7fbea006ee40 2024-05-30T16:44:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.148+0000 7fbe9fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbea006e080 con 0x7fbea0156b30 2024-05-30T16:44:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.148+0000 7fbe9e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbea006ee40 0x7fbea00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.148+0000 7fbe9f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbea006f2f0 0x7fbea00b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54824/0 (socket says 172.21.15.67:54824) 2024-05-30T16:44:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.148+0000 7fbe9f400640 1 -- 172.21.15.67:0/3806602064 learned_addr learned my addr 172.21.15.67:0/3806602064 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.148+0000 7fbe9ea00640 1 --2- 172.21.15.67:0/3806602064 >> v2:172.21.15.169:3300/0 conn(0x7fbea0156b30 0x7fbea00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.149+0000 7fbe9e000640 1 -- 172.21.15.67:0/3806602064 >> v2:172.21.15.169:3300/0 conn(0x7fbea0156b30 msgr2=0x7fbea00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.149+0000 7fbe9e000640 1 --2- 172.21.15.67:0/3806602064 >> v2:172.21.15.169:3300/0 conn(0x7fbea0156b30 0x7fbea00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.149+0000 7fbe9e000640 1 -- 172.21.15.67:0/3806602064 >> v1:172.21.15.67:6789/0 conn(0x7fbea006f2f0 legacy=0x7fbea00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.149+0000 7fbe9e000640 1 -- 172.21.15.67:0/3806602064 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea006e360 con 0x7fbea006ee40 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.149+0000 7fbe9ea00640 1 --2- 172.21.15.67:0/3806602064 >> v2:172.21.15.169:3300/0 conn(0x7fbea0156b30 0x7fbea00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.149+0000 7fbe9d600640 1 -- 172.21.15.67:0/3806602064 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992037444 0 0) 0x7fbea0156f10 con 0x7fbea006f2f0 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.150+0000 7fbe9e000640 1 --2- 172.21.15.67:0/3806602064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea006ee40 0x7fbea00ad230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbea006c8b0 tx=0x7fbe8802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7bdd28a7faab38 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.150+0000 7fbe9d600640 1 -- 172.21.15.67:0/3806602064 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88037070 con 0x7fbea006ee40 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.150+0000 7fbe9d600640 1 -- 172.21.15.67:0/3806602064 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe88032440 con 0x7fbea006ee40 2024-05-30T16:44:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.150+0000 7fbe9d600640 1 -- 172.21.15.67:0/3806602064 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8802ae60 con 0x7fbea006ee40 2024-05-30T16:44:51.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.151+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3806602064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea006ee40 msgr2=0x7fbea00ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.151+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3806602064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea006ee40 0x7fbea00ad230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbea006c8b0 tx=0x7fbe8802a110 comp rx=0 tx=0).stop 2024-05-30T16:44:51.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.151+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3806602064 shutdown_connections 2024-05-30T16:44:51.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.151+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3806602064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea006ee40 0x7fbea00ad230 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.151+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3806602064 >> v2:172.21.15.169:3300/0 conn(0x7fbea0156b30 0x7fbea00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.151+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3806602064 >> 172.21.15.67:0/3806602064 conn(0x7fbea00b1fc0 msgr2=0x7fbea00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:51.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.151+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3806602064 shutdown_connections 2024-05-30T16:44:51.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.151+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3806602064 wait complete. 2024-05-30T16:44:51.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.153+0000 7fbe9fe00640 1 Processor -- start 2024-05-30T16:44:51.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.153+0000 7fbe9fe00640 1 -- start start 2024-05-30T16:44:51.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbea006ee40 0x7fbea00b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbea006f2f0 0x7fbea0156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea0156b30 0x7fbea0155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbea00afc20 con 0x7fbea0156b30 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbea00b2c30 con 0x7fbea006f2f0 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbea0148d60 con 0x7fbea006ee40 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea0156b30 0x7fbea0155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea0156b30 0x7fbea0155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46626/0 (socket says 172.21.15.67:46626) 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9f400640 1 -- 172.21.15.67:0/3180305766 learned_addr learned my addr 172.21.15.67:0/3180305766 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9f400640 1 -- 172.21.15.67:0/3180305766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbea006ee40 msgr2=0x7fbea00b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9f400640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbea006ee40 0x7fbea00b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9f400640 1 -- 172.21.15.67:0/3180305766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbea006f2f0 msgr2=0x7fbea0156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9e000640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbea006f2f0 0x7fbea0156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9f400640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbea006f2f0 0x7fbea0156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.154+0000 7fbe9f400640 1 -- 172.21.15.67:0/3180305766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea0149320 con 0x7fbea0156b30 2024-05-30T16:44:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.155+0000 7fbe9f400640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea0156b30 0x7fbea0155b10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800c800 tx=0x7fbe9800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.155+0000 7fbe96a00640 1 -- 172.21.15.67:0/3180305766 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98018070 con 0x7fbea0156b30 2024-05-30T16:44:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.155+0000 7fbe96a00640 1 -- 172.21.15.67:0/3180305766 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe98004d80 con 0x7fbea0156b30 2024-05-30T16:44:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.155+0000 7fbe96a00640 1 -- 172.21.15.67:0/3180305766 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9800a880 con 0x7fbea0156b30 2024-05-30T16:44:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.155+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea00b4e50 con 0x7fbea0156b30 2024-05-30T16:44:51.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.155+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea0148fb0 con 0x7fbea0156b30 2024-05-30T16:44:51.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.156+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea00689e0 con 0x7fbea0156b30 2024-05-30T16:44:51.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.157+0000 7fbe96a00640 1 -- 172.21.15.67:0/3180305766 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe9801b070 con 0x7fbea0156b30 2024-05-30T16:44:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.158+0000 7fbe96a00640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7c06c8f0 0x7fbe7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.158+0000 7fbe96a00640 1 -- 172.21.15.67:0/3180305766 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fbe9808b140 con 0x7fbea0156b30 2024-05-30T16:44:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.159+0000 7fbe9ea00640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7c06c8f0 0x7fbe7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.159+0000 7fbe9ea00640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7c06c8f0 0x7fbe7c06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe90006fd0 tx=0x7fbe90008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.163+0000 7fbe96a00640 1 -- 172.21.15.67:0/3180305766 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe98059590 con 0x7fbea0156b30 2024-05-30T16:44:51.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.366+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbea00b3030 con 0x7fbe7c06c8f0 2024-05-30T16:44:51.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.384+0000 7fbe96a00640 1 -- 172.21.15.67:0/3180305766 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248689 (secure 0 0 0) 0x7fbea00b3030 con 0x7fbe7c06c8f0 2024-05-30T16:44:51.389 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:51.391 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:44:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7c06c8f0 msgr2=0x7fbe7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7c06c8f0 0x7fbe7c06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe90006fd0 tx=0x7fbe90008040 comp rx=0 tx=0).stop 2024-05-30T16:44:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea0156b30 msgr2=0x7fbea0155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea0156b30 0x7fbea0155b10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800c800 tx=0x7fbe9800ccc0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 shutdown_connections 2024-05-30T16:44:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7c06c8f0 0x7fbe7c06edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea0156b30 0x7fbea0155b10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbea006f2f0 0x7fbea0156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3180305766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbea006ee40 0x7fbea00b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 >> 172.21.15.67:0/3180305766 conn(0x7fbea00b1fc0 msgr2=0x7fbea00b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.391+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 shutdown_connections 2024-05-30T16:44:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.392+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3180305766 wait complete. 2024-05-30T16:44:51.412 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-05-30T16:44:49.996577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":165,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220236,"kb_used_data":4056,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754484,"statfs":{"total":773094113280,"available":772868591616,"internally_reserved":0,"allocated":4153344,"data_stored":2320410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.871893+0000","last_change":"2024-05-30T16:44:44.871893+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.871893+0000","last_undegraded":"2024-05-30T16:44:44.871893+0000","last_fullsized":"2024-05-30T16:44:44.871893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870925+0000","last_change":"2024-05-30T16:44:44.870925+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870925+0000","last_undegraded":"2024-05-30T16:44:44.870925+0000","last_fullsized":"2024-05-30T16:44:44.870925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915144+0000","last_change":"2024-05-30T16:44:45.915063+0000","last_active":"2024-05-30T16:44:46.915144+0000","last_peered":"2024-05-30T16:44:46.915144+0000","last_clean":"2024-05-30T16:44:46.915144+0000","last_became_active":"2024-05-30T16:44:45.914534+0000","last_became_peered":"2024-05-30T16:44:45.914534+0000","last_unstale":"2024-05-30T16:44:46.915144+0000","last_undegraded":"2024-05-30T16:44:46.915144+0000","last_fullsized":"2024-05-30T16:44:46.915144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.871890+0000","last_change":"2024-05-30T16:44:44.871890+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.871890+0000","last_undegraded":"2024-05-30T16:44:44.871890+0000","last_fullsized":"2024-05-30T16:44:44.871890+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.869063+0000","last_change":"2024-05-30T16:44:44.869063+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.869063+0000","last_undegraded":"2024-05-30T16:44:44.869063+0000","last_fullsized":"2024-05-30T16:44:44.869063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.872128+0000","last_change":"2024-05-30T16:44:44.872128+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.872128+0000","last_undegraded":"2024-05-30T16:44:44.872128+0000","last_fullsized":"2024-05-30T16:44:44.872128+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914905+0000","last_change":"2024-05-30T16:44:45.922189+0000","last_active":"2024-05-30T16:44:46.914905+0000","last_peered":"2024-05-30T16:44:46.914905+0000","last_clean":"2024-05-30T16:44:46.914905+0000","last_became_active":"2024-05-30T16:44:45.921920+0000","last_became_peered":"2024-05-30T16:44:45.921920+0000","last_unstale":"2024-05-30T16:44:46.914905+0000","last_undegraded":"2024-05-30T16:44:46.914905+0000","last_fullsized":"2024-05-30T16:44:46.914905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:32:30.873632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912732+0000","last_change":"2024-05-30T16:44:45.918239+0000","last_active":"2024-05-30T16:44:46.912732+0000","last_peered":"2024-05-30T16:44:46.912732+0000","last_clean":"2024-05-30T16:44:46.912732+0000","last_became_active":"2024-05-30T16:44:45.917936+0000","last_became_peered":"2024-05-30T16:44:45.917936+0000","last_unstale":"2024-05-30T16:44:46.912732+0000","last_undegraded":"2024-05-30T16:44:46.912732+0000","last_fullsized":"2024-05-30T16:44:46.912732+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866406+0000","last_change":"2024-05-30T16:44:44.866406+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866406+0000","last_undegraded":"2024-05-30T16:44:44.866406+0000","last_fullsized":"2024-05-30T16:44:44.866406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.872840+0000","last_change":"2024-05-30T16:44:44.872840+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.872840+0000","last_undegraded":"2024-05-30T16:44:44.872840+0000","last_fullsized":"2024-05-30T16:44:44.872840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915320+0000","last_change":"2024-05-30T16:44:45.927692+0000","last_active":"2024-05-30T16:44:46.915320+0000","last_peered":"2024-05-30T16:44:46.915320+0000","last_clean":"2024-05-30T16:44:46.915320+0000","last_became_active":"2024-05-30T16:44:45.927381+0000","last_became_peered":"2024-05-30T16:44:45.927381+0000","last_unstale":"2024-05-30T16:44:46.915320+0000","last_undegraded":"2024-05-30T16:44:46.915320+0000","last_fullsized":"2024-05-30T16:44:46.915320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:20:16.734760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914592+0000","last_change":"2024-05-30T16:44:45.922504+0000","last_active":"2024-05-30T16:44:46.914592+0000","last_peered":"2024-05-30T16:44:46.914592+0000","last_clean":"2024-05-30T16:44:46.914592+0000","last_became_active":"2024-05-30T16:44:45.922230+0000","last_became_peered":"2024-05-30T16:44:45.922230+0000","last_unstale":"2024-05-30T16:44:46.914592+0000","last_undegraded":"2024-05-30T16:44:46.914592+0000","last_fullsized":"2024-05-30T16:44:46.914592+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867394+0000","last_change":"2024-05-30T16:44:44.867394+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867394+0000","last_undegraded":"2024-05-30T16:44:44.867394+0000","last_fullsized":"2024-05-30T16:44:44.867394+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870665+0000","last_change":"2024-05-30T16:44:44.870665+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870665+0000","last_undegraded":"2024-05-30T16:44:44.870665+0000","last_fullsized":"2024-05-30T16:44:44.870665+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914323+0000","last_change":"2024-05-30T16:44:45.912535+0000","last_active":"2024-05-30T16:44:46.914323+0000","last_peered":"2024-05-30T16:44:46.914323+0000","last_clean":"2024-05-30T16:44:46.914323+0000","last_became_active":"2024-05-30T16:44:45.911217+0000","last_became_peered":"2024-05-30T16:44:45.911217+0000","last_unstale":"2024-05-30T16:44:46.914323+0000","last_undegraded":"2024-05-30T16:44:46.914323+0000","last_fullsized":"2024-05-30T16:44:46.914323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:10:41.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914560+0000","last_change":"2024-05-30T16:44:45.925677+0000","last_active":"2024-05-30T16:44:46.914560+0000","last_peered":"2024-05-30T16:44:46.914560+0000","last_clean":"2024-05-30T16:44:46.914560+0000","last_became_active":"2024-05-30T16:44:45.925335+0000","last_became_peered":"2024-05-30T16:44:45.925335+0000","last_unstale":"2024-05-30T16:44:46.914560+0000","last_undegraded":"2024-05-30T16:44:46.914560+0000","last_fullsized":"2024-05-30T16:44:46.914560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:24:35.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915605+0000","last_change":"2024-05-30T16:44:45.921499+0000","last_active":"2024-05-30T16:44:46.915605+0000","last_peered":"2024-05-30T16:44:46.915605+0000","last_clean":"2024-05-30T16:44:46.915605+0000","last_became_active":"2024-05-30T16:44:45.921212+0000","last_became_peered":"2024-05-30T16:44:45.921212+0000","last_unstale":"2024-05-30T16:44:46.915605+0000","last_undegraded":"2024-05-30T16:44:46.915605+0000","last_fullsized":"2024-05-30T16:44:46.915605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:14.743364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.868154+0000","last_change":"2024-05-30T16:44:44.868154+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.868154+0000","last_undegraded":"2024-05-30T16:44:44.868154+0000","last_fullsized":"2024-05-30T16:44:44.868154+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.869345+0000","last_change":"2024-05-30T16:44:44.869345+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.869345+0000","last_undegraded":"2024-05-30T16:44:44.869345+0000","last_fullsized":"2024-05-30T16:44:44.869345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.875019+0000","last_change":"2024-05-30T16:44:44.875019+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.875019+0000","last_undegraded":"2024-05-30T16:44:44.875019+0000","last_fullsized":"2024-05-30T16:44:44.875019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866701+0000","last_change":"2024-05-30T16:44:44.866701+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866701+0000","last_undegraded":"2024-05-30T16:44:44.866701+0000","last_fullsized":"2024-05-30T16:44:44.866701+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.861135+0000","last_change":"2024-05-30T16:44:44.861135+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.861135+0000","last_undegraded":"2024-05-30T16:44:44.861135+0000","last_fullsized":"2024-05-30T16:44:44.861135+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932990+0000","last_change":"2024-05-30T16:44:45.920307+0000","last_active":"2024-05-30T16:44:46.932990+0000","last_peered":"2024-05-30T16:44:46.932990+0000","last_clean":"2024-05-30T16:44:46.932990+0000","last_became_active":"2024-05-30T16:44:45.918862+0000","last_became_peered":"2024-05-30T16:44:45.918862+0000","last_unstale":"2024-05-30T16:44:46.932990+0000","last_undegraded":"2024-05-30T16:44:46.932990+0000","last_fullsized":"2024-05-30T16:44:46.932990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:52:44.864855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912985+0000","last_change":"2024-05-30T16:44:45.909508+0000","last_active":"2024-05-30T16:44:46.912985+0000","last_peered":"2024-05-30T16:44:46.912985+0000","last_clean":"2024-05-30T16:44:46.912985+0000","last_became_active":"2024-05-30T16:44:45.904007+0000","last_became_peered":"2024-05-30T16:44:45.904007+0000","last_unstale":"2024-05-30T16:44:46.912985+0000","last_undegraded":"2024-05-30T16:44:46.912985+0000","last_fullsized":"2024-05-30T16:44:46.912985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:11.996704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.873189+0000","last_change":"2024-05-30T16:44:44.873189+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.873189+0000","last_undegraded":"2024-05-30T16:44:44.873189+0000","last_fullsized":"2024-05-30T16:44:44.873189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867724+0000","last_change":"2024-05-30T16:44:44.867724+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867724+0000","last_undegraded":"2024-05-30T16:44:44.867724+0000","last_fullsized":"2024-05-30T16:44:44.867724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870720+0000","last_change":"2024-05-30T16:44:44.870720+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870720+0000","last_undegraded":"2024-05-30T16:44:44.870720+0000","last_fullsized":"2024-05-30T16:44:44.870720+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915334+0000","last_change":"2024-05-30T16:44:45.908143+0000","last_active":"2024-05-30T16:44:46.915334+0000","last_peered":"2024-05-30T16:44:46.915334+0000","last_clean":"2024-05-30T16:44:46.915334+0000","last_became_active":"2024-05-30T16:44:45.907830+0000","last_became_peered":"2024-05-30T16:44:45.907830+0000","last_unstale":"2024-05-30T16:44:46.915334+0000","last_undegraded":"2024-05-30T16:44:46.915334+0000","last_fullsized":"2024-05-30T16:44:46.915334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.865560+0000","last_change":"2024-05-30T16:44:44.865560+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.865560+0000","last_undegraded":"2024-05-30T16:44:44.865560+0000","last_fullsized":"2024-05-30T16:44:44.865560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913062+0000","last_change":"2024-05-30T16:44:45.926086+0000","last_active":"2024-05-30T16:44:46.913062+0000","last_peered":"2024-05-30T16:44:46.913062+0000","last_clean":"2024-05-30T16:44:46.913062+0000","last_became_active":"2024-05-30T16:44:45.921839+0000","last_became_peered":"2024-05-30T16:44:45.921839+0000","last_unstale":"2024-05-30T16:44:46.913062+0000","last_undegraded":"2024-05-30T16:44:46.913062+0000","last_fullsized":"2024-05-30T16:44:46.913062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.872445+0000","last_change":"2024-05-30T16:44:44.872445+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.872445+0000","last_undegraded":"2024-05-30T16:44:44.872445+0000","last_fullsized":"2024-05-30T16:44:44.872445+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912803+0000","last_change":"2024-05-30T16:44:45.931329+0000","last_active":"2024-05-30T16:44:46.912803+0000","last_peered":"2024-05-30T16:44:46.912803+0000","last_clean":"2024-05-30T16:44:46.912803+0000","last_became_active":"2024-05-30T16:44:45.929807+0000","last_became_peered":"2024-05-30T16:44:45.929807+0000","last_unstale":"2024-05-30T16:44:46.912803+0000","last_undegraded":"2024-05-30T16:44:46.912803+0000","last_fullsized":"2024-05-30T16:44:46.912803+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933144+0000","last_change":"2024-05-30T16:44:45.918314+0000","last_active":"2024-05-30T16:44:46.933144+0000","last_peered":"2024-05-30T16:44:46.933144+0000","last_clean":"2024-05-30T16:44:46.933144+0000","last_became_active":"2024-05-30T16:44:45.917998+0000","last_became_peered":"2024-05-30T16:44:45.917998+0000","last_unstale":"2024-05-30T16:44:46.933144+0000","last_undegraded":"2024-05-30T16:44:46.933144+0000","last_fullsized":"2024-05-30T16:44:46.933144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:09.909703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.897452+0000","last_change":"2024-05-30T16:44:44.892355+0000","last_active":"2024-05-30T16:44:44.897452+0000","last_peered":"2024-05-30T16:44:44.897452+0000","last_clean":"2024-05-30T16:44:44.897452+0000","last_became_active":"2024-05-30T16:44:44.891889+0000","last_became_peered":"2024-05-30T16:44:44.891889+0000","last_unstale":"2024-05-30T16:44:44.897452+0000","last_undegraded":"2024-05-30T16:44:44.897452+0000","last_fullsized":"2024-05-30T16:44:44.897452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:54.736980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870784+0000","last_change":"2024-05-30T16:44:44.870784+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870784+0000","last_undegraded":"2024-05-30T16:44:44.870784+0000","last_fullsized":"2024-05-30T16:44:44.870784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932836+0000","last_change":"2024-05-30T16:44:44.889925+0000","last_active":"2024-05-30T16:44:46.932836+0000","last_peered":"2024-05-30T16:44:46.932836+0000","last_clean":"2024-05-30T16:44:46.932836+0000","last_became_active":"2024-05-30T16:44:44.889564+0000","last_became_peered":"2024-05-30T16:44:44.889564+0000","last_unstale":"2024-05-30T16:44:46.932836+0000","last_undegraded":"2024-05-30T16:44:46.932836+0000","last_fullsized":"2024-05-30T16:44:46.932836+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:45.960008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.861455+0000","last_change":"2024-05-30T16:44:44.861455+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.861455+0000","last_undegraded":"2024-05-30T16:44:44.861455+0000","last_fullsized":"2024-05-30T16:44:44.861455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.885660+0000","last_change":"2024-05-30T16:44:44.883317+0000","last_active":"2024-05-30T16:44:44.885660+0000","last_peered":"2024-05-30T16:44:44.885660+0000","last_clean":"2024-05-30T16:44:44.885660+0000","last_became_active":"2024-05-30T16:44:44.882994+0000","last_became_peered":"2024-05-30T16:44:44.882994+0000","last_unstale":"2024-05-30T16:44:44.885660+0000","last_undegraded":"2024-05-30T16:44:44.885660+0000","last_fullsized":"2024-05-30T16:44:44.885660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:02:28.009022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870120+0000","last_change":"2024-05-30T16:44:44.870120+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870120+0000","last_undegraded":"2024-05-30T16:44:44.870120+0000","last_fullsized":"2024-05-30T16:44:44.870120+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.890431+0000","last_change":"2024-05-30T16:44:44.886536+0000","last_active":"2024-05-30T16:44:44.890431+0000","last_peered":"2024-05-30T16:44:44.890431+0000","last_clean":"2024-05-30T16:44:44.890431+0000","last_became_active":"2024-05-30T16:44:44.886261+0000","last_became_peered":"2024-05-30T16:44:44.886261+0000","last_unstale":"2024-05-30T16:44:44.890431+0000","last_undegraded":"2024-05-30T16:44:44.890431+0000","last_fullsized":"2024-05-30T16:44:44.890431+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:07.538799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913886+0000","last_change":"2024-05-30T16:44:45.936295+0000","last_active":"2024-05-30T16:44:46.913886+0000","last_peered":"2024-05-30T16:44:46.913886+0000","last_clean":"2024-05-30T16:44:46.913886+0000","last_became_active":"2024-05-30T16:44:45.935416+0000","last_became_peered":"2024-05-30T16:44:45.935416+0000","last_unstale":"2024-05-30T16:44:46.913886+0000","last_undegraded":"2024-05-30T16:44:46.913886+0000","last_fullsized":"2024-05-30T16:44:46.913886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894858+0000","last_change":"2024-05-30T16:44:44.886562+0000","last_active":"2024-05-30T16:44:44.894858+0000","last_peered":"2024-05-30T16:44:44.894858+0000","last_clean":"2024-05-30T16:44:44.894858+0000","last_became_active":"2024-05-30T16:44:44.886254+0000","last_became_peered":"2024-05-30T16:44:44.886254+0000","last_unstale":"2024-05-30T16:44:44.894858+0000","last_undegraded":"2024-05-30T16:44:44.894858+0000","last_fullsized":"2024-05-30T16:44:44.894858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:53.956014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913996+0000","last_change":"2024-05-30T16:44:45.905732+0000","last_active":"2024-05-30T16:44:46.913996+0000","last_peered":"2024-05-30T16:44:46.913996+0000","last_clean":"2024-05-30T16:44:46.913996+0000","last_became_active":"2024-05-30T16:44:45.905117+0000","last_became_peered":"2024-05-30T16:44:45.905117+0000","last_unstale":"2024-05-30T16:44:46.913996+0000","last_undegraded":"2024-05-30T16:44:46.913996+0000","last_fullsized":"2024-05-30T16:44:46.913996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:55:15.879718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.891719+0000","last_change":"2024-05-30T16:44:44.879442+0000","last_active":"2024-05-30T16:44:44.891719+0000","last_peered":"2024-05-30T16:44:44.891719+0000","last_clean":"2024-05-30T16:44:44.891719+0000","last_became_active":"2024-05-30T16:44:44.877334+0000","last_became_peered":"2024-05-30T16:44:44.877334+0000","last_unstale":"2024-05-30T16:44:44.891719+0000","last_undegraded":"2024-05-30T16:44:44.891719+0000","last_fullsized":"2024-05-30T16:44:44.891719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:18.355954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.860816+0000","last_change":"2024-05-30T16:44:44.860816+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.860816+0000","last_undegraded":"2024-05-30T16:44:44.860816+0000","last_fullsized":"2024-05-30T16:44:44.860816+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.879443+0000","last_change":"2024-05-30T16:44:44.876950+0000","last_active":"2024-05-30T16:44:44.879443+0000","last_peered":"2024-05-30T16:44:44.879443+0000","last_clean":"2024-05-30T16:44:44.879443+0000","last_became_active":"2024-05-30T16:44:44.876666+0000","last_became_peered":"2024-05-30T16:44:44.876666+0000","last_unstale":"2024-05-30T16:44:44.879443+0000","last_undegraded":"2024-05-30T16:44:44.879443+0000","last_fullsized":"2024-05-30T16:44:44.879443+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:44.589105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867195+0000","last_change":"2024-05-30T16:44:44.867195+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867195+0000","last_undegraded":"2024-05-30T16:44:44.867195+0000","last_fullsized":"2024-05-30T16:44:44.867195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894960+0000","last_change":"2024-05-30T16:44:44.887604+0000","last_active":"2024-05-30T16:44:44.894960+0000","last_peered":"2024-05-30T16:44:44.894960+0000","last_clean":"2024-05-30T16:44:44.894960+0000","last_became_active":"2024-05-30T16:44:44.887259+0000","last_became_peered":"2024-05-30T16:44:44.887259+0000","last_unstale":"2024-05-30T16:44:44.894960+0000","last_undegraded":"2024-05-30T16:44:44.894960+0000","last_fullsized":"2024-05-30T16:44:44.894960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:34:29.911545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915928+0000","last_change":"2024-05-30T16:44:45.918490+0000","last_active":"2024-05-30T16:44:46.915928+0000","last_peered":"2024-05-30T16:44:46.915928+0000","last_clean":"2024-05-30T16:44:46.915928+0000","last_became_active":"2024-05-30T16:44:45.917451+0000","last_became_peered":"2024-05-30T16:44:45.917451+0000","last_unstale":"2024-05-30T16:44:46.915928+0000","last_undegraded":"2024-05-30T16:44:46.915928+0000","last_fullsized":"2024-05-30T16:44:46.915928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893908+0000","last_change":"2024-05-30T16:44:44.889369+0000","last_active":"2024-05-30T16:44:44.893908+0000","last_peered":"2024-05-30T16:44:44.893908+0000","last_clean":"2024-05-30T16:44:44.893908+0000","last_became_active":"2024-05-30T16:44:44.889051+0000","last_became_peered":"2024-05-30T16:44:44.889051+0000","last_unstale":"2024-05-30T16:44:44.893908+0000","last_undegraded":"2024-05-30T16:44:44.893908+0000","last_fullsized":"2024-05-30T16:44:44.893908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:25.424530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866906+0000","last_change":"2024-05-30T16:44:44.866906+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866906+0000","last_undegraded":"2024-05-30T16:44:44.866906+0000","last_fullsized":"2024-05-30T16:44:44.866906+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932945+0000","last_change":"2024-05-30T16:44:44.885815+0000","last_active":"2024-05-30T16:44:46.932945+0000","last_peered":"2024-05-30T16:44:46.932945+0000","last_clean":"2024-05-30T16:44:46.932945+0000","last_became_active":"2024-05-30T16:44:44.885007+0000","last_became_peered":"2024-05-30T16:44:44.885007+0000","last_unstale":"2024-05-30T16:44:46.932945+0000","last_undegraded":"2024-05-30T16:44:46.932945+0000","last_fullsized":"2024-05-30T16:44:46.932945+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:24:02.943052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866137+0000","last_change":"2024-05-30T16:44:44.866137+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866137+0000","last_undegraded":"2024-05-30T16:44:44.866137+0000","last_fullsized":"2024-05-30T16:44:44.866137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915318+0000","last_change":"2024-05-30T16:44:44.891929+0000","last_active":"2024-05-30T16:44:46.915318+0000","last_peered":"2024-05-30T16:44:46.915318+0000","last_clean":"2024-05-30T16:44:46.915318+0000","last_became_active":"2024-05-30T16:44:44.891611+0000","last_became_peered":"2024-05-30T16:44:44.891611+0000","last_unstale":"2024-05-30T16:44:46.915318+0000","last_undegraded":"2024-05-30T16:44:46.915318+0000","last_fullsized":"2024-05-30T16:44:46.915318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:41.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914621+0000","last_change":"2024-05-30T16:44:45.915178+0000","last_active":"2024-05-30T16:44:46.914621+0000","last_peered":"2024-05-30T16:44:46.914621+0000","last_clean":"2024-05-30T16:44:46.914621+0000","last_became_active":"2024-05-30T16:44:45.914901+0000","last_became_peered":"2024-05-30T16:44:45.914901+0000","last_unstale":"2024-05-30T16:44:46.914621+0000","last_undegraded":"2024-05-30T16:44:46.914621+0000","last_fullsized":"2024-05-30T16:44:46.914621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.886164+0000","last_change":"2024-05-30T16:44:44.884173+0000","last_active":"2024-05-30T16:44:44.886164+0000","last_peered":"2024-05-30T16:44:44.886164+0000","last_clean":"2024-05-30T16:44:44.886164+0000","last_became_active":"2024-05-30T16:44:44.883842+0000","last_became_peered":"2024-05-30T16:44:44.883842+0000","last_unstale":"2024-05-30T16:44:44.886164+0000","last_undegraded":"2024-05-30T16:44:44.886164+0000","last_fullsized":"2024-05-30T16:44:44.886164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:52.965427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.866920+0000","last_change":"2024-05-30T16:44:44.866920+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.866920+0000","last_undegraded":"2024-05-30T16:44:44.866920+0000","last_fullsized":"2024-05-30T16:44:44.866920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.895228+0000","last_change":"2024-05-30T16:44:44.888087+0000","last_active":"2024-05-30T16:44:44.895228+0000","last_peered":"2024-05-30T16:44:44.895228+0000","last_clean":"2024-05-30T16:44:44.895228+0000","last_became_active":"2024-05-30T16:44:44.887807+0000","last_became_peered":"2024-05-30T16:44:44.887807+0000","last_unstale":"2024-05-30T16:44:44.895228+0000","last_undegraded":"2024-05-30T16:44:44.895228+0000","last_fullsized":"2024-05-30T16:44:44.895228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:53.783765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.864963+0000","last_change":"2024-05-30T16:44:44.864963+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.864963+0000","last_undegraded":"2024-05-30T16:44:44.864963+0000","last_fullsized":"2024-05-30T16:44:44.864963+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894088+0000","last_change":"2024-05-30T16:44:44.891918+0000","last_active":"2024-05-30T16:44:44.894088+0000","last_peered":"2024-05-30T16:44:44.894088+0000","last_clean":"2024-05-30T16:44:44.894088+0000","last_became_active":"2024-05-30T16:44:44.891134+0000","last_became_peered":"2024-05-30T16:44:44.891134+0000","last_unstale":"2024-05-30T16:44:44.894088+0000","last_undegraded":"2024-05-30T16:44:44.894088+0000","last_fullsized":"2024-05-30T16:44:44.894088+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:14.335974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913160+0000","last_change":"2024-05-30T16:44:45.931105+0000","last_active":"2024-05-30T16:44:46.913160+0000","last_peered":"2024-05-30T16:44:46.913160+0000","last_clean":"2024-05-30T16:44:46.913160+0000","last_became_active":"2024-05-30T16:44:45.929921+0000","last_became_peered":"2024-05-30T16:44:45.929921+0000","last_unstale":"2024-05-30T16:44:46.913160+0000","last_undegraded":"2024-05-30T16:44:46.913160+0000","last_fullsized":"2024-05-30T16:44:46.913160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893925+0000","last_change":"2024-05-30T16:44:44.889587+0000","last_active":"2024-05-30T16:44:44.893925+0000","last_peered":"2024-05-30T16:44:44.893925+0000","last_clean":"2024-05-30T16:44:44.893925+0000","last_became_active":"2024-05-30T16:44:44.889226+0000","last_became_peered":"2024-05-30T16:44:44.889226+0000","last_unstale":"2024-05-30T16:44:44.893925+0000","last_undegraded":"2024-05-30T16:44:44.893925+0000","last_fullsized":"2024-05-30T16:44:44.893925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:34.292596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.870707+0000","last_change":"2024-05-30T16:44:44.870707+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.870707+0000","last_undegraded":"2024-05-30T16:44:44.870707+0000","last_fullsized":"2024-05-30T16:44:44.870707+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.879106+0000","last_change":"2024-05-30T16:44:44.871442+0000","last_active":"2024-05-30T16:44:44.879106+0000","last_peered":"2024-05-30T16:44:44.879106+0000","last_clean":"2024-05-30T16:44:44.879106+0000","last_became_active":"2024-05-30T16:44:44.871162+0000","last_became_peered":"2024-05-30T16:44:44.871162+0000","last_unstale":"2024-05-30T16:44:44.879106+0000","last_undegraded":"2024-05-30T16:44:44.879106+0000","last_fullsized":"2024-05-30T16:44:44.879106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:14.335974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.863239+0000","last_change":"2024-05-30T16:44:44.863239+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.863239+0000","last_undegraded":"2024-05-30T16:44:44.863239+0000","last_fullsized":"2024-05-30T16:44:44.863239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.891914+0000","last_change":"2024-05-30T16:44:44.886316+0000","last_active":"2024-05-30T16:44:44.891914+0000","last_peered":"2024-05-30T16:44:44.891914+0000","last_clean":"2024-05-30T16:44:44.891914+0000","last_became_active":"2024-05-30T16:44:44.885953+0000","last_became_peered":"2024-05-30T16:44:44.885953+0000","last_unstale":"2024-05-30T16:44:44.891914+0000","last_undegraded":"2024-05-30T16:44:44.891914+0000","last_fullsized":"2024-05-30T16:44:44.891914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:37.611751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915844+0000","last_change":"2024-05-30T16:44:45.910082+0000","last_active":"2024-05-30T16:44:46.915844+0000","last_peered":"2024-05-30T16:44:46.915844+0000","last_clean":"2024-05-30T16:44:46.915844+0000","last_became_active":"2024-05-30T16:44:45.909541+0000","last_became_peered":"2024-05-30T16:44:45.909541+0000","last_unstale":"2024-05-30T16:44:46.915844+0000","last_undegraded":"2024-05-30T16:44:46.915844+0000","last_fullsized":"2024-05-30T16:44:46.915844+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.895123+0000","last_change":"2024-05-30T16:44:44.888958+0000","last_active":"2024-05-30T16:44:44.895123+0000","last_peered":"2024-05-30T16:44:44.895123+0000","last_clean":"2024-05-30T16:44:44.895123+0000","last_became_active":"2024-05-30T16:44:44.888629+0000","last_became_peered":"2024-05-30T16:44:44.888629+0000","last_unstale":"2024-05-30T16:44:44.895123+0000","last_undegraded":"2024-05-30T16:44:44.895123+0000","last_fullsized":"2024-05-30T16:44:44.895123+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:23:49.576943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.871260+0000","last_change":"2024-05-30T16:44:44.871260+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.871260+0000","last_undegraded":"2024-05-30T16:44:44.871260+0000","last_fullsized":"2024-05-30T16:44:44.871260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.895035+0000","last_change":"2024-05-30T16:44:44.887204+0000","last_active":"2024-05-30T16:44:44.895035+0000","last_peered":"2024-05-30T16:44:44.895035+0000","last_clean":"2024-05-30T16:44:44.895035+0000","last_became_active":"2024-05-30T16:44:44.886702+0000","last_became_peered":"2024-05-30T16:44:44.886702+0000","last_unstale":"2024-05-30T16:44:44.895035+0000","last_undegraded":"2024-05-30T16:44:44.895035+0000","last_fullsized":"2024-05-30T16:44:44.895035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:07.538799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867090+0000","last_change":"2024-05-30T16:44:44.867090+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867090+0000","last_undegraded":"2024-05-30T16:44:44.867090+0000","last_fullsized":"2024-05-30T16:44:44.867090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.878752+0000","last_change":"2024-05-30T16:44:44.874074+0000","last_active":"2024-05-30T16:44:44.878752+0000","last_peered":"2024-05-30T16:44:44.878752+0000","last_clean":"2024-05-30T16:44:44.878752+0000","last_became_active":"2024-05-30T16:44:44.872396+0000","last_became_peered":"2024-05-30T16:44:44.872396+0000","last_unstale":"2024-05-30T16:44:44.878752+0000","last_undegraded":"2024-05-30T16:44:44.878752+0000","last_fullsized":"2024-05-30T16:44:44.878752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:28:38.953023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.868645+0000","last_change":"2024-05-30T16:44:44.868645+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.868645+0000","last_undegraded":"2024-05-30T16:44:44.868645+0000","last_fullsized":"2024-05-30T16:44:44.868645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932742+0000","last_change":"2024-05-30T16:44:44.881588+0000","last_active":"2024-05-30T16:44:46.932742+0000","last_peered":"2024-05-30T16:44:46.932742+0000","last_clean":"2024-05-30T16:44:46.932742+0000","last_became_active":"2024-05-30T16:44:44.881282+0000","last_became_peered":"2024-05-30T16:44:44.881282+0000","last_unstale":"2024-05-30T16:44:46.932742+0000","last_undegraded":"2024-05-30T16:44:46.932742+0000","last_fullsized":"2024-05-30T16:44:46.932742+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:27:30.795217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.865956+0000","last_change":"2024-05-30T16:44:44.865956+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.865956+0000","last_undegraded":"2024-05-30T16:44:44.865956+0000","last_fullsized":"2024-05-30T16:44:44.865956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893051+0000","last_change":"2024-05-30T16:44:44.890541+0000","last_active":"2024-05-30T16:44:44.893051+0000","last_peered":"2024-05-30T16:44:44.893051+0000","last_clean":"2024-05-30T16:44:44.893051+0000","last_became_active":"2024-05-30T16:44:44.890280+0000","last_became_peered":"2024-05-30T16:44:44.890280+0000","last_unstale":"2024-05-30T16:44:44.893051+0000","last_undegraded":"2024-05-30T16:44:44.893051+0000","last_fullsized":"2024-05-30T16:44:44.893051+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:43.725581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867121+0000","last_change":"2024-05-30T16:44:44.867121+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867121+0000","last_undegraded":"2024-05-30T16:44:44.867121+0000","last_fullsized":"2024-05-30T16:44:44.867121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894788+0000","last_change":"2024-05-30T16:44:44.885235+0000","last_active":"2024-05-30T16:44:44.894788+0000","last_peered":"2024-05-30T16:44:44.894788+0000","last_clean":"2024-05-30T16:44:44.894788+0000","last_became_active":"2024-05-30T16:44:44.884907+0000","last_became_peered":"2024-05-30T16:44:44.884907+0000","last_unstale":"2024-05-30T16:44:44.894788+0000","last_undegraded":"2024-05-30T16:44:44.894788+0000","last_fullsized":"2024-05-30T16:44:44.894788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:30:05.228927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913450+0000","last_change":"2024-05-30T16:44:45.919445+0000","last_active":"2024-05-30T16:44:46.913450+0000","last_peered":"2024-05-30T16:44:46.913450+0000","last_clean":"2024-05-30T16:44:46.913450+0000","last_became_active":"2024-05-30T16:44:45.919149+0000","last_became_peered":"2024-05-30T16:44:45.919149+0000","last_unstale":"2024-05-30T16:44:46.913450+0000","last_undegraded":"2024-05-30T16:44:46.913450+0000","last_fullsized":"2024-05-30T16:44:46.913450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913128+0000","last_change":"2024-05-30T16:44:44.887506+0000","last_active":"2024-05-30T16:44:46.913128+0000","last_peered":"2024-05-30T16:44:46.913128+0000","last_clean":"2024-05-30T16:44:46.913128+0000","last_became_active":"2024-05-30T16:44:44.887180+0000","last_became_peered":"2024-05-30T16:44:44.887180+0000","last_unstale":"2024-05-30T16:44:46.913128+0000","last_undegraded":"2024-05-30T16:44:46.913128+0000","last_fullsized":"2024-05-30T16:44:46.913128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:08.904901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.868406+0000","last_change":"2024-05-30T16:44:44.868406+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.868406+0000","last_undegraded":"2024-05-30T16:44:44.868406+0000","last_fullsized":"2024-05-30T16:44:44.868406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.894070+0000","last_change":"2024-05-30T16:44:44.891749+0000","last_active":"2024-05-30T16:44:44.894070+0000","last_peered":"2024-05-30T16:44:44.894070+0000","last_clean":"2024-05-30T16:44:44.894070+0000","last_became_active":"2024-05-30T16:44:44.891440+0000","last_became_peered":"2024-05-30T16:44:44.891440+0000","last_unstale":"2024-05-30T16:44:44.894070+0000","last_undegraded":"2024-05-30T16:44:44.894070+0000","last_fullsized":"2024-05-30T16:44:44.894070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:37.347555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.916134+0000","last_change":"2024-05-30T16:44:45.911902+0000","last_active":"2024-05-30T16:44:46.916134+0000","last_peered":"2024-05-30T16:44:46.916134+0000","last_clean":"2024-05-30T16:44:46.916134+0000","last_became_active":"2024-05-30T16:44:45.911282+0000","last_became_peered":"2024-05-30T16:44:45.911282+0000","last_unstale":"2024-05-30T16:44:46.916134+0000","last_undegraded":"2024-05-30T16:44:46.916134+0000","last_fullsized":"2024-05-30T16:44:46.916134+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933055+0000","last_change":"2024-05-30T16:44:44.885333+0000","last_active":"2024-05-30T16:44:46.933055+0000","last_peered":"2024-05-30T16:44:46.933055+0000","last_clean":"2024-05-30T16:44:46.933055+0000","last_became_active":"2024-05-30T16:44:44.883685+0000","last_became_peered":"2024-05-30T16:44:44.883685+0000","last_unstale":"2024-05-30T16:44:46.933055+0000","last_undegraded":"2024-05-30T16:44:46.933055+0000","last_fullsized":"2024-05-30T16:44:46.933055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:52.935243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932751+0000","last_change":"2024-05-30T16:44:45.915538+0000","last_active":"2024-05-30T16:44:46.932751+0000","last_peered":"2024-05-30T16:44:46.932751+0000","last_clean":"2024-05-30T16:44:46.932751+0000","last_became_active":"2024-05-30T16:44:45.914763+0000","last_became_peered":"2024-05-30T16:44:45.914763+0000","last_unstale":"2024-05-30T16:44:46.932751+0000","last_undegraded":"2024-05-30T16:44:46.932751+0000","last_fullsized":"2024-05-30T16:44:46.932751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:30.231921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893559+0000","last_change":"2024-05-30T16:44:44.888789+0000","last_active":"2024-05-30T16:44:44.893559+0000","last_peered":"2024-05-30T16:44:44.893559+0000","last_clean":"2024-05-30T16:44:44.893559+0000","last_became_active":"2024-05-30T16:44:44.888193+0000","last_became_peered":"2024-05-30T16:44:44.888193+0000","last_unstale":"2024-05-30T16:44:44.893559+0000","last_undegraded":"2024-05-30T16:44:44.893559+0000","last_fullsized":"2024-05-30T16:44:44.893559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:54.736980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.876070+0000","last_change":"2024-05-30T16:44:44.876070+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.876070+0000","last_undegraded":"2024-05-30T16:44:44.876070+0000","last_fullsized":"2024-05-30T16:44:44.876070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933045+0000","last_change":"2024-05-30T16:44:44.892955+0000","last_active":"2024-05-30T16:44:46.933045+0000","last_peered":"2024-05-30T16:44:46.933045+0000","last_clean":"2024-05-30T16:44:46.933045+0000","last_became_active":"2024-05-30T16:44:44.892513+0000","last_became_peered":"2024-05-30T16:44:44.892513+0000","last_unstale":"2024-05-30T16:44:46.933045+0000","last_undegraded":"2024-05-30T16:44:46.933045+0000","last_fullsized":"2024-05-30T16:44:46.933045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:04:30.373290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932947+0000","last_change":"2024-05-30T16:44:45.910492+0000","last_active":"2024-05-30T16:44:46.932947+0000","last_peered":"2024-05-30T16:44:46.932947+0000","last_clean":"2024-05-30T16:44:46.932947+0000","last_became_active":"2024-05-30T16:44:45.909824+0000","last_became_peered":"2024-05-30T16:44:45.909824+0000","last_unstale":"2024-05-30T16:44:46.932947+0000","last_undegraded":"2024-05-30T16:44:46.932947+0000","last_fullsized":"2024-05-30T16:44:46.932947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:45.491068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.892063+0000","last_change":"2024-05-30T16:44:44.887201+0000","last_active":"2024-05-30T16:44:44.892063+0000","last_peered":"2024-05-30T16:44:44.892063+0000","last_clean":"2024-05-30T16:44:44.892063+0000","last_became_active":"2024-05-30T16:44:44.886748+0000","last_became_peered":"2024-05-30T16:44:44.886748+0000","last_unstale":"2024-05-30T16:44:44.892063+0000","last_undegraded":"2024-05-30T16:44:44.892063+0000","last_fullsized":"2024-05-30T16:44:44.892063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:43.725581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.867588+0000","last_change":"2024-05-30T16:44:44.867588+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.867588+0000","last_undegraded":"2024-05-30T16:44:44.867588+0000","last_fullsized":"2024-05-30T16:44:44.867588+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.892502+0000","last_change":"2024-05-30T16:44:44.882578+0000","last_active":"2024-05-30T16:44:44.892502+0000","last_peered":"2024-05-30T16:44:44.892502+0000","last_clean":"2024-05-30T16:44:44.892502+0000","last_became_active":"2024-05-30T16:44:44.882197+0000","last_became_peered":"2024-05-30T16:44:44.882197+0000","last_unstale":"2024-05-30T16:44:44.892502+0000","last_undegraded":"2024-05-30T16:44:44.892502+0000","last_fullsized":"2024-05-30T16:44:44.892502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:38:11.267149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-30T16:44:44.861724+0000","last_change":"2024-05-30T16:44:44.861724+0000","last_active":"2024-05-30T16:44:44.844223+0000","last_peered":"2024-05-30T16:44:44.844223+0000","last_clean":"2024-05-30T16:44:44.844223+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:44:44.861724+0000","last_undegraded":"2024-05-30T16:44:44.861724+0000","last_fullsized":"2024-05-30T16:44:44.861724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.893846+0000","last_change":"2024-05-30T16:44:44.888466+0000","last_active":"2024-05-30T16:44:44.893846+0000","last_peered":"2024-05-30T16:44:44.893846+0000","last_clean":"2024-05-30T16:44:44.893846+0000","last_became_active":"2024-05-30T16:44:44.888164+0000","last_became_peered":"2024-05-30T16:44:44.888164+0000","last_unstale":"2024-05-30T16:44:44.893846+0000","last_undegraded":"2024-05-30T16:44:44.893846+0000","last_fullsized":"2024-05-30T16:44:44.893846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:33.985709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914547+0000","last_change":"2024-05-30T16:44:45.919696+0000","last_active":"2024-05-30T16:44:46.914547+0000","last_peered":"2024-05-30T16:44:46.914547+0000","last_clean":"2024-05-30T16:44:46.914547+0000","last_became_active":"2024-05-30T16:44:45.918248+0000","last_became_peered":"2024-05-30T16:44:45.918248+0000","last_unstale":"2024-05-30T16:44:46.914547+0000","last_undegraded":"2024-05-30T16:44:46.914547+0000","last_fullsized":"2024-05-30T16:44:46.914547+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-30T16:44:44.897520+0000","last_change":"2024-05-30T16:44:44.893954+0000","last_active":"2024-05-30T16:44:44.897520+0000","last_peered":"2024-05-30T16:44:44.897520+0000","last_clean":"2024-05-30T16:44:44.897520+0000","last_became_active":"2024-05-30T16:44:44.893629+0000","last_became_peered":"2024-05-30T16:44:44.893629+0000","last_unstale":"2024-05-30T16:44:44.897520+0000","last_undegraded":"2024-05-30T16:44:44.897520+0000","last_fullsized":"2024-05-30T16:44:44.897520+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:37.347555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913336+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:44:46.913336+0000","last_peered":"2024-05-30T16:44:46.913336+0000","last_clean":"2024-05-30T16:44:46.913336+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:44:46.913336+0000","last_undegraded":"2024-05-30T16:44:46.913336+0000","last_fullsized":"2024-05-30T16:44:46.913336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":133544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":133544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":139811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":139811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":133544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":743344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":153558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":743254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:44:51.416 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3f is not active+clean 2024-05-30T16:44:51.416 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.871893+0000', 'last_change': '2024-05-30T16:44:44.871893+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.871893+0000', 'last_undegraded': '2024-05-30T16:44:44.871893+0000', 'last_fullsized': '2024-05-30T16:44:44.871893+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:44:51.416 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3e is not active+clean 2024-05-30T16:44:51.416 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.870925+0000', 'last_change': '2024-05-30T16:44:44.870925+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.870925+0000', 'last_undegraded': '2024-05-30T16:44:44.870925+0000', 'last_fullsized': '2024-05-30T16:44:44.870925+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.416 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3c is not active+clean 2024-05-30T16:44:51.416 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.871890+0000', 'last_change': '2024-05-30T16:44:44.871890+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.871890+0000', 'last_undegraded': '2024-05-30T16:44:44.871890+0000', 'last_fullsized': '2024-05-30T16:44:44.871890+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:44:51.416 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3b is not active+clean 2024-05-30T16:44:51.416 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.869063+0000', 'last_change': '2024-05-30T16:44:44.869063+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.869063+0000', 'last_undegraded': '2024-05-30T16:44:44.869063+0000', 'last_fullsized': '2024-05-30T16:44:44.869063+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3a is not active+clean 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.872128+0000', 'last_change': '2024-05-30T16:44:44.872128+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.872128+0000', 'last_undegraded': '2024-05-30T16:44:44.872128+0000', 'last_fullsized': '2024-05-30T16:44:44.872128+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.866406+0000', 'last_change': '2024-05-30T16:44:44.866406+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.866406+0000', 'last_undegraded': '2024-05-30T16:44:44.866406+0000', 'last_fullsized': '2024-05-30T16:44:44.866406+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:PG 5.36 is not active+clean 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.872840+0000', 'last_change': '2024-05-30T16:44:44.872840+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.872840+0000', 'last_undegraded': '2024-05-30T16:44:44.872840+0000', 'last_fullsized': '2024-05-30T16:44:44.872840+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:PG 5.33 is not active+clean 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.867394+0000', 'last_change': '2024-05-30T16:44:44.867394+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.867394+0000', 'last_undegraded': '2024-05-30T16:44:44.867394+0000', 'last_fullsized': '2024-05-30T16:44:44.867394+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:PG 5.32 is not active+clean 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.870665+0000', 'last_change': '2024-05-30T16:44:44.870665+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.870665+0000', 'last_undegraded': '2024-05-30T16:44:44.870665+0000', 'last_fullsized': '2024-05-30T16:44:44.870665+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.417 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2e is not active+clean 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.868154+0000', 'last_change': '2024-05-30T16:44:44.868154+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.868154+0000', 'last_undegraded': '2024-05-30T16:44:44.868154+0000', 'last_fullsized': '2024-05-30T16:44:44.868154+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2d is not active+clean 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.869345+0000', 'last_change': '2024-05-30T16:44:44.869345+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.869345+0000', 'last_undegraded': '2024-05-30T16:44:44.869345+0000', 'last_fullsized': '2024-05-30T16:44:44.869345+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2c is not active+clean 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.875019+0000', 'last_change': '2024-05-30T16:44:44.875019+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.875019+0000', 'last_undegraded': '2024-05-30T16:44:44.875019+0000', 'last_fullsized': '2024-05-30T16:44:44.875019+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2b is not active+clean 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.866701+0000', 'last_change': '2024-05-30T16:44:44.866701+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.866701+0000', 'last_undegraded': '2024-05-30T16:44:44.866701+0000', 'last_fullsized': '2024-05-30T16:44:44.866701+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2a is not active+clean 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.861135+0000', 'last_change': '2024-05-30T16:44:44.861135+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.861135+0000', 'last_undegraded': '2024-05-30T16:44:44.861135+0000', 'last_fullsized': '2024-05-30T16:44:44.861135+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:PG 5.27 is not active+clean 2024-05-30T16:44:51.418 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.873189+0000', 'last_change': '2024-05-30T16:44:44.873189+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.873189+0000', 'last_undegraded': '2024-05-30T16:44:44.873189+0000', 'last_fullsized': '2024-05-30T16:44:44.873189+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:PG 5.26 is not active+clean 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.867724+0000', 'last_change': '2024-05-30T16:44:44.867724+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.867724+0000', 'last_undegraded': '2024-05-30T16:44:44.867724+0000', 'last_fullsized': '2024-05-30T16:44:44.867724+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:PG 5.25 is not active+clean 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.870720+0000', 'last_change': '2024-05-30T16:44:44.870720+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.870720+0000', 'last_undegraded': '2024-05-30T16:44:44.870720+0000', 'last_fullsized': '2024-05-30T16:44:44.870720+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:PG 5.23 is not active+clean 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.865560+0000', 'last_change': '2024-05-30T16:44:44.865560+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.865560+0000', 'last_undegraded': '2024-05-30T16:44:44.865560+0000', 'last_fullsized': '2024-05-30T16:44:44.865560+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:PG 5.21 is not active+clean 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.872445+0000', 'last_change': '2024-05-30T16:44:44.872445+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.872445+0000', 'last_undegraded': '2024-05-30T16:44:44.872445+0000', 'last_fullsized': '2024-05-30T16:44:44.872445+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.870784+0000', 'last_change': '2024-05-30T16:44:44.870784+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.870784+0000', 'last_undegraded': '2024-05-30T16:44:44.870784+0000', 'last_fullsized': '2024-05-30T16:44:44.870784+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:44:51.419 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.861455+0000', 'last_change': '2024-05-30T16:44:44.861455+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.861455+0000', 'last_undegraded': '2024-05-30T16:44:44.861455+0000', 'last_fullsized': '2024-05-30T16:44:44.861455+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1d is not active+clean 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.870120+0000', 'last_change': '2024-05-30T16:44:44.870120+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.870120+0000', 'last_undegraded': '2024-05-30T16:44:44.870120+0000', 'last_fullsized': '2024-05-30T16:44:44.870120+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.860816+0000', 'last_change': '2024-05-30T16:44:44.860816+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.860816+0000', 'last_undegraded': '2024-05-30T16:44:44.860816+0000', 'last_fullsized': '2024-05-30T16:44:44.860816+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:PG 5.19 is not active+clean 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.867195+0000', 'last_change': '2024-05-30T16:44:44.867195+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.867195+0000', 'last_undegraded': '2024-05-30T16:44:44.867195+0000', 'last_fullsized': '2024-05-30T16:44:44.867195+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:PG 5.17 is not active+clean 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.866906+0000', 'last_change': '2024-05-30T16:44:44.866906+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.866906+0000', 'last_undegraded': '2024-05-30T16:44:44.866906+0000', 'last_fullsized': '2024-05-30T16:44:44.866906+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:PG 5.14 is not active+clean 2024-05-30T16:44:51.420 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.866137+0000', 'last_change': '2024-05-30T16:44:44.866137+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.866137+0000', 'last_undegraded': '2024-05-30T16:44:44.866137+0000', 'last_fullsized': '2024-05-30T16:44:44.866137+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:PG 5.12 is not active+clean 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.866920+0000', 'last_change': '2024-05-30T16:44:44.866920+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.866920+0000', 'last_undegraded': '2024-05-30T16:44:44.866920+0000', 'last_fullsized': '2024-05-30T16:44:44.866920+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:PG 5.13 is not active+clean 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.864963+0000', 'last_change': '2024-05-30T16:44:44.864963+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.864963+0000', 'last_undegraded': '2024-05-30T16:44:44.864963+0000', 'last_fullsized': '2024-05-30T16:44:44.864963+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.870707+0000', 'last_change': '2024-05-30T16:44:44.870707+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.870707+0000', 'last_undegraded': '2024-05-30T16:44:44.870707+0000', 'last_fullsized': '2024-05-30T16:44:44.870707+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.863239+0000', 'last_change': '2024-05-30T16:44:44.863239+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.863239+0000', 'last_undegraded': '2024-05-30T16:44:44.863239+0000', 'last_fullsized': '2024-05-30T16:44:44.863239+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:PG 5.c is not active+clean 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.871260+0000', 'last_change': '2024-05-30T16:44:44.871260+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.871260+0000', 'last_undegraded': '2024-05-30T16:44:44.871260+0000', 'last_fullsized': '2024-05-30T16:44:44.871260+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-05-30T16:44:51.421 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.867090+0000', 'last_change': '2024-05-30T16:44:44.867090+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.867090+0000', 'last_undegraded': '2024-05-30T16:44:44.867090+0000', 'last_fullsized': '2024-05-30T16:44:44.867090+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.868645+0000', 'last_change': '2024-05-30T16:44:44.868645+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.868645+0000', 'last_undegraded': '2024-05-30T16:44:44.868645+0000', 'last_fullsized': '2024-05-30T16:44:44.868645+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.865956+0000', 'last_change': '2024-05-30T16:44:44.865956+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.865956+0000', 'last_undegraded': '2024-05-30T16:44:44.865956+0000', 'last_fullsized': '2024-05-30T16:44:44.865956+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.867121+0000', 'last_change': '2024-05-30T16:44:44.867121+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.867121+0000', 'last_undegraded': '2024-05-30T16:44:44.867121+0000', 'last_fullsized': '2024-05-30T16:44:44.867121+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.868406+0000', 'last_change': '2024-05-30T16:44:44.868406+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.868406+0000', 'last_undegraded': '2024-05-30T16:44:44.868406+0000', 'last_fullsized': '2024-05-30T16:44:44.868406+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.876070+0000', 'last_change': '2024-05-30T16:44:44.876070+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.876070+0000', 'last_undegraded': '2024-05-30T16:44:44.876070+0000', 'last_fullsized': '2024-05-30T16:44:44.876070+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:PG 5.0 is not active+clean 2024-05-30T16:44:51.422 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.867588+0000', 'last_change': '2024-05-30T16:44:44.867588+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.867588+0000', 'last_undegraded': '2024-05-30T16:44:44.867588+0000', 'last_fullsized': '2024-05-30T16:44:44.867588+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:44:51.423 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-05-30T16:44:51.423 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:44:44.861724+0000', 'last_change': '2024-05-30T16:44:44.861724+0000', 'last_active': '2024-05-30T16:44:44.844223+0000', 'last_peered': '2024-05-30T16:44:44.844223+0000', 'last_clean': '2024-05-30T16:44:44.844223+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:44:44.861724+0000', 'last_undegraded': '2024-05-30T16:44:44.861724+0000', 'last_fullsized': '2024-05-30T16:44:44.861724+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'last_clean_scrub_stamp': '2024-05-30T16:44:44.844223+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:44:51.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-30T16:44:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.670+0000 7f9a3d000640 1 Processor -- start 2024-05-30T16:44:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.670+0000 7f9a3d000640 1 -- start start 2024-05-30T16:44:51.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.672+0000 7f9a3d000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a3815ac80 0x7f9a3815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.672+0000 7f9a3d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a380cd310 0x7f9a3815b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.672+0000 7f9a3d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a380b4be0 con 0x7f9a380ca870 2024-05-30T16:44:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.672+0000 7f9a3d000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9a3815f630 con 0x7f9a3815ac80 2024-05-30T16:44:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.672+0000 7f9a3d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a380cf8a0 con 0x7f9a380cd310 2024-05-30T16:44:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.672+0000 7f9a36a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9a380ca870 0x7f9a380ccc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54830/0 (socket says 172.21.15.67:54830) 2024-05-30T16:44:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.672+0000 7f9a36a00640 1 -- 172.21.15.67:0/3409042878 learned_addr learned my addr 172.21.15.67:0/3409042878 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a37400640 1 --2- 172.21.15.67:0/3409042878 >> v2:172.21.15.169:3300/0 conn(0x7f9a3815ac80 0x7f9a3815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a37e00640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a380cd310 0x7f9a3815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 672920498 0 0) 0x7f9a380b4be0 con 0x7f9a380ca870 2024-05-30T16:44:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a1c003520 con 0x7f9a380ca870 2024-05-30T16:44:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2405891112 0 0) 0x7f9a1c003520 con 0x7f9a380ca870 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a380cd310 msgr2=0x7f9a3815b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a380cd310 0x7f9a3815b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 >> v2:172.21.15.169:3300/0 conn(0x7f9a3815ac80 msgr2=0x7f9a3815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 --2- 172.21.15.67:0/3409042878 >> v2:172.21.15.169:3300/0 conn(0x7f9a3815ac80 0x7f9a3815f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a380cde90 con 0x7f9a380ca870 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9a24001690 con 0x7f9a380ca870 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 >> v1:172.21.15.67:6789/0 conn(0x7f9a380ca870 legacy=0x7f9a380ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a380cd310 0x7f9a1c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c003ef0 0x7f9a1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a1c004370 0x7f9a1c004d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3815f630 con 0x7f9a1c004370 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a380cf8a0 con 0x7f9a1c003ef0 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.673+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c0055b0 con 0x7f9a380cd310 2024-05-30T16:44:51.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a37e00640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a1c004370 0x7f9a1c004d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a37400640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a380cd310 0x7f9a1c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a36a00640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c003ef0 0x7f9a1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a37e00640 1 -- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a380cd310 msgr2=0x7f9a1c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a37e00640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a380cd310 0x7f9a1c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a37e00640 1 -- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c003ef0 msgr2=0x7f9a1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a37e00640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c003ef0 0x7f9a1c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a37e00640 1 -- 172.21.15.67:0/3409042878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c003520 con 0x7f9a1c004370 2024-05-30T16:44:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a37400640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a380cd310 0x7f9a1c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.674+0000 7f9a36a00640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c003ef0 0x7f9a1c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.675+0000 7f9a37e00640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a1c004370 0x7f9a1c004d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000aa40 tx=0x7f9a1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d27e6721875f6a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.675+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30036070 con 0x7f9a1c004370 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.675+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a30031600 con 0x7f9a1c004370 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.675+0000 7f9a36000640 1 -- 172.21.15.67:0/3409042878 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3000b040 con 0x7f9a1c004370 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.676+0000 7f9a3d000640 1 -- 172.21.15.67:0/3409042878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a1c004370 msgr2=0x7f9a1c004d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.676+0000 7f9a3d000640 1 --2- 172.21.15.67:0/3409042878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a1c004370 0x7f9a1c004d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000aa40 tx=0x7f9a1c005830 comp rx=0 tx=0).stop 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.676+0000 7f9a37e00640 1 -- 172.21.15.67:0/3409042878 reap_dead start 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.676+0000 7f9a3d000640 1 -- 172.21.15.67:0/3409042878 shutdown_connections 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.676+0000 7f9a3d000640 1 -- 172.21.15.67:0/3409042878 >> 172.21.15.67:0/3409042878 conn(0x7f9a3806bae0 msgr2=0x7f9a3806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.676+0000 7f9a3d000640 1 -- 172.21.15.67:0/3409042878 shutdown_connections 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.676+0000 7f9a3d000640 1 -- 172.21.15.67:0/3409042878 wait complete. 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.678+0000 7f9a3d000640 1 Processor -- start 2024-05-30T16:44:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.678+0000 7f9a3d000640 1 -- start start 2024-05-30T16:44:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.679+0000 7f9a3d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c004370 0x7f9a38155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.679+0000 7f9a3d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38154530 0x7f9a381f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.679+0000 7f9a3d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a38072540 0x7f9a381f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.679+0000 7f9a3d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a380cfe20 con 0x7f9a38154530 2024-05-30T16:44:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.679+0000 7f9a3d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a380cfb40 con 0x7f9a1c004370 2024-05-30T16:44:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.679+0000 7f9a3d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a380af720 con 0x7f9a38072540 2024-05-30T16:44:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38154530 0x7f9a381f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38154530 0x7f9a381f3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46646/0 (socket says 172.21.15.67:46646) 2024-05-30T16:44:51.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a36a00640 1 -- 172.21.15.67:0/2545738926 learned_addr learned my addr 172.21.15.67:0/2545738926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a37e00640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a38072540 0x7f9a381f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a37400640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c004370 0x7f9a38155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a36a00640 1 -- 172.21.15.67:0/2545738926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a38072540 msgr2=0x7f9a381f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a36a00640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a38072540 0x7f9a381f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a36a00640 1 -- 172.21.15.67:0/2545738926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c004370 msgr2=0x7f9a38155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a36a00640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c004370 0x7f9a38155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.680+0000 7f9a36a00640 1 -- 172.21.15.67:0/2545738926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a38072ae0 con 0x7f9a38154530 2024-05-30T16:44:51.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.681+0000 7f9a36a00640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38154530 0x7f9a381f3860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400d8d0 tx=0x7f9a2400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:51.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.681+0000 7f9a2f400640 1 -- 172.21.15.67:0/2545738926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24019070 con 0x7f9a38154530 2024-05-30T16:44:51.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.681+0000 7f9a2f400640 1 -- 172.21.15.67:0/2545738926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a240074e0 con 0x7f9a38154530 2024-05-30T16:44:51.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.681+0000 7f9a2f400640 1 -- 172.21.15.67:0/2545738926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a240047e0 con 0x7f9a38154530 2024-05-30T16:44:51.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.681+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a38072dc0 con 0x7f9a38154530 2024-05-30T16:44:51.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.681+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a38073380 con 0x7f9a38154530 2024-05-30T16:44:51.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.682+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a380689e0 con 0x7f9a38154530 2024-05-30T16:44:51.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.683+0000 7f9a2f400640 1 -- 172.21.15.67:0/2545738926 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a2401c070 con 0x7f9a38154530 2024-05-30T16:44:51.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.684+0000 7f9a2f400640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1806c970 0x7f9a1806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:51.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.684+0000 7f9a2f400640 1 -- 172.21.15.67:0/2545738926 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9a2408b500 con 0x7f9a38154530 2024-05-30T16:44:51.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.685+0000 7f9a37400640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1806c970 0x7f9a1806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:51.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.685+0000 7f9a37400640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1806c970 0x7f9a1806ee30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a28003be0 tx=0x7f9a2800b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:51.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.690+0000 7f9a2f400640 1 -- 172.21.15.67:0/2545738926 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a24059350 con 0x7f9a38154530 2024-05-30T16:44:51.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.968+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9a3806e0f0 con 0x7f9a38154530 2024-05-30T16:44:51.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.968+0000 7f9a2f400640 1 -- 172.21.15.67:0/2545738926 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1644 (secure 0 0 0) 0x7f9a2405d650 con 0x7f9a38154530 2024-05-30T16:44:51.969 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:51.969 INFO:teuthology.orchestra.run.smithi067.stdout:{"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":85,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1717087419,"num_in_osds":8,"osd_in_since":1717087410,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":58},{"state_name":"creating+peering","count":39}],"num_pgs":97,"num_pools":3,"num_objects":6,"data_bytes":590458,"bytes_used":225521664,"bytes_avail":772868591616,"bytes_total":773094113280,"inactive_pgs_ratio":0.40206184983253479,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":4821}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-05-30T16:44:41.994704+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-30T16:44:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1806c970 msgr2=0x7f9a1806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1806c970 0x7f9a1806ee30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a28003be0 tx=0x7f9a2800b040 comp rx=0 tx=0).stop 2024-05-30T16:44:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38154530 msgr2=0x7f9a381f3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38154530 0x7f9a381f3860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400d8d0 tx=0x7f9a2400dd90 comp rx=0 tx=0).stop 2024-05-30T16:44:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 shutdown_connections 2024-05-30T16:44:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1806c970 0x7f9a1806ee30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a38072540 0x7f9a381f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a38154530 0x7f9a381f3860 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 --2- 172.21.15.67:0/2545738926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1c004370 0x7f9a38155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 >> 172.21.15.67:0/2545738926 conn(0x7f9a3806bae0 msgr2=0x7f9a38157720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 shutdown_connections 2024-05-30T16:44:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:51.971+0000 7f9a3d000640 1 -- 172.21.15.67:0/2545738926 wait complete. 2024-05-30T16:44:51.992 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-30T16:44:51.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:44:52.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.205+0000 7fe552600640 1 Processor -- start 2024-05-30T16:44:52.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.206+0000 7fe552600640 1 -- start start 2024-05-30T16:44:52.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.207+0000 7fe552600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe54c0d16a0 0x7fe54c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:52.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.207+0000 7fe552600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe54c0cfce0 con 0x7fe54c0cf870 2024-05-30T16:44:52.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.207+0000 7fe552600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe54c15ec50 con 0x7fe54c15e870 2024-05-30T16:44:52.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.207+0000 7fe552600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe54c0d2150 con 0x7fe54c0d16a0 2024-05-30T16:44:52.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.208+0000 7fe551200640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe54c15e870 0x7fe54c0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:53264/0 (socket says 172.21.15.67:53264) 2024-05-30T16:44:52.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.208+0000 7fe551200640 1 -- 172.21.15.67:0/994542165 learned_addr learned my addr 172.21.15.67:0/994542165 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:52.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.208+0000 7fe551c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe54c0cf870 0x7fe54c0d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54834/0 (socket says 172.21.15.67:54834) 2024-05-30T16:44:52.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.208+0000 7fe54be00640 1 --2- 172.21.15.67:0/994542165 >> v2:172.21.15.67:3300/0 conn(0x7fe54c0d16a0 0x7fe54c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:52.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.209+0000 7fe54be00640 1 -- 172.21.15.67:0/994542165 >> v1:172.21.15.169:6789/0 conn(0x7fe54c15e870 legacy=0x7fe54c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:52.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.209+0000 7fe54be00640 1 -- 172.21.15.67:0/994542165 >> v1:172.21.15.67:6789/0 conn(0x7fe54c0cf870 legacy=0x7fe54c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:52.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.209+0000 7fe54be00640 1 -- 172.21.15.67:0/994542165 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c0cff80 con 0x7fe54c0d16a0 2024-05-30T16:44:52.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.210+0000 7fe54be00640 1 --2- 172.21.15.67:0/994542165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 0x7fe54c0cf330 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe538009790 tx=0x7fe53802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18c05c1f96eb363 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:52.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.210+0000 7fe54b400640 1 -- 172.21.15.67:0/994542165 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538037070 con 0x7fe54c0d16a0 2024-05-30T16:44:52.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.210+0000 7fe54b400640 1 -- 172.21.15.67:0/994542165 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe538032440 con 0x7fe54c0d16a0 2024-05-30T16:44:52.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.210+0000 7fe54b400640 1 -- 172.21.15.67:0/994542165 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538032810 con 0x7fe54c0d16a0 2024-05-30T16:44:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.211+0000 7fe552600640 1 -- 172.21.15.67:0/994542165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 msgr2=0x7fe54c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.211+0000 7fe552600640 1 --2- 172.21.15.67:0/994542165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 0x7fe54c0cf330 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe538009790 tx=0x7fe53802a210 comp rx=0 tx=0).stop 2024-05-30T16:44:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.211+0000 7fe552600640 1 -- 172.21.15.67:0/994542165 shutdown_connections 2024-05-30T16:44:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.211+0000 7fe552600640 1 --2- 172.21.15.67:0/994542165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 0x7fe54c0cf330 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.211+0000 7fe552600640 1 -- 172.21.15.67:0/994542165 >> 172.21.15.67:0/994542165 conn(0x7fe54c08bfc0 msgr2=0x7fe54c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.211+0000 7fe552600640 1 -- 172.21.15.67:0/994542165 shutdown_connections 2024-05-30T16:44:52.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.211+0000 7fe552600640 1 -- 172.21.15.67:0/994542165 wait complete. 2024-05-30T16:44:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.213+0000 7fe552600640 1 Processor -- start 2024-05-30T16:44:52.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.213+0000 7fe552600640 1 -- start start 2024-05-30T16:44:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe552600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe54c0cf870 0x7fe54c14c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe552600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 0x7fe54c0b9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe552600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe54c15e870 0x7fe54c0b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe552600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c0d0820 con 0x7fe54c0d16a0 2024-05-30T16:44:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe552600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c0d0540 con 0x7fe54c0cf870 2024-05-30T16:44:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe552600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe54c0d4fb0 con 0x7fe54c15e870 2024-05-30T16:44:52.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe54be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 0x7fe54c0b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:52.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe54be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 0x7fe54c0b9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46664/0 (socket says 172.21.15.67:46664) 2024-05-30T16:44:52.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe54be00640 1 -- 172.21.15.67:0/1047411462 learned_addr learned my addr 172.21.15.67:0/1047411462 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:52.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe551200640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe54c0cf870 0x7fe54c14c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:52.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe54be00640 1 -- 172.21.15.67:0/1047411462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe54c15e870 msgr2=0x7fe54c0b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:52.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe54be00640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe54c15e870 0x7fe54c0b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:52.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe54be00640 1 -- 172.21.15.67:0/1047411462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe54c0cf870 msgr2=0x7fe54c14c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:52.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe54be00640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe54c0cf870 0x7fe54c14c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:52.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.214+0000 7fe54be00640 1 -- 172.21.15.67:0/1047411462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c1e8fb0 con 0x7fe54c0d16a0 2024-05-30T16:44:52.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.215+0000 7fe54be00640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 0x7fe54c0b9370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe5380096e0 tx=0x7fe5380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:52.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.215+0000 7fe548c00640 1 -- 172.21.15.67:0/1047411462 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538037040 con 0x7fe54c0d16a0 2024-05-30T16:44:52.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.215+0000 7fe548c00640 1 -- 172.21.15.67:0/1047411462 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5380307b0 con 0x7fe54c0d16a0 2024-05-30T16:44:52.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.215+0000 7fe548c00640 1 -- 172.21.15.67:0/1047411462 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538030b80 con 0x7fe54c0d16a0 2024-05-30T16:44:52.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.215+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c1e9290 con 0x7fe54c0d16a0 2024-05-30T16:44:52.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.215+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1e9850 con 0x7fe54c0d16a0 2024-05-30T16:44:52.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.217+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54c0689e0 con 0x7fe54c0d16a0 2024-05-30T16:44:52.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.218+0000 7fe548c00640 1 -- 172.21.15.67:0/1047411462 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe538044020 con 0x7fe54c0d16a0 2024-05-30T16:44:52.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.219+0000 7fe548c00640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe52c06c8f0 0x7fe52c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:52.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.219+0000 7fe548c00640 1 -- 172.21.15.67:0/1047411462 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe5380abe40 con 0x7fe54c0d16a0 2024-05-30T16:44:52.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.219+0000 7fe551200640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe52c06c8f0 0x7fe52c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:52.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.220+0000 7fe551200640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe52c06c8f0 0x7fe52c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe53c005880 tx=0x7fe53c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:52.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.223+0000 7fe548c00640 1 -- 172.21.15.67:0/1047411462 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe53804b020 con 0x7fe54c0d16a0 2024-05-30T16:44:52.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.424+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe54c0ba500 con 0x7fe52c06c8f0 2024-05-30T16:44:52.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.435+0000 7fe548c00640 1 -- 172.21.15.67:0/1047411462 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7fe54c0ba500 con 0x7fe52c06c8f0 2024-05-30T16:44:52.439 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:52.440 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:44:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe52c06c8f0 msgr2=0x7fe52c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe52c06c8f0 0x7fe52c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe53c005880 tx=0x7fe53c009040 comp rx=0 tx=0).stop 2024-05-30T16:44:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 msgr2=0x7fe54c0b9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 0x7fe54c0b9370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe5380096e0 tx=0x7fe5380046c0 comp rx=0 tx=0).stop 2024-05-30T16:44:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 shutdown_connections 2024-05-30T16:44:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe52c06c8f0 0x7fe52c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe54c15e870 0x7fe54c0b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe54c0d16a0 0x7fe54c0b9370 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:52.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 --2- 172.21.15.67:0/1047411462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe54c0cf870 0x7fe54c14c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:52.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 >> 172.21.15.67:0/1047411462 conn(0x7fe54c08bfc0 msgr2=0x7fe54c0b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:52.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 shutdown_connections 2024-05-30T16:44:52.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:52.440+0000 7fe552600640 1 -- 172.21.15.67:0/1047411462 wait complete. 2024-05-30T16:44:52.461 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-05-30T16:44:51.997097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220484,"kb_used_data":4264,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754236,"statfs":{"total":773094113280,"available":772868337664,"internally_reserved":0,"allocated":4366336,"data_stored":2408644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.144416"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948948+0000","last_change":"2024-05-30T16:44:45.920620+0000","last_active":"2024-05-30T16:44:46.948948+0000","last_peered":"2024-05-30T16:44:46.948948+0000","last_clean":"2024-05-30T16:44:46.948948+0000","last_became_active":"2024-05-30T16:44:45.920342+0000","last_became_peered":"2024-05-30T16:44:45.920342+0000","last_unstale":"2024-05-30T16:44:46.948948+0000","last_undegraded":"2024-05-30T16:44:46.948948+0000","last_fullsized":"2024-05-30T16:44:46.948948+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:00.032212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912434+0000","last_change":"2024-05-30T16:44:45.919659+0000","last_active":"2024-05-30T16:44:46.912434+0000","last_peered":"2024-05-30T16:44:46.912434+0000","last_clean":"2024-05-30T16:44:46.912434+0000","last_became_active":"2024-05-30T16:44:45.919335+0000","last_became_peered":"2024-05-30T16:44:45.919335+0000","last_unstale":"2024-05-30T16:44:46.912434+0000","last_undegraded":"2024-05-30T16:44:46.912434+0000","last_fullsized":"2024-05-30T16:44:46.912434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:08.259292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915144+0000","last_change":"2024-05-30T16:44:45.915063+0000","last_active":"2024-05-30T16:44:46.915144+0000","last_peered":"2024-05-30T16:44:46.915144+0000","last_clean":"2024-05-30T16:44:46.915144+0000","last_became_active":"2024-05-30T16:44:45.914534+0000","last_became_peered":"2024-05-30T16:44:45.914534+0000","last_unstale":"2024-05-30T16:44:46.915144+0000","last_undegraded":"2024-05-30T16:44:46.915144+0000","last_fullsized":"2024-05-30T16:44:46.915144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938680+0000","last_change":"2024-05-30T16:44:45.934496+0000","last_active":"2024-05-30T16:44:46.938680+0000","last_peered":"2024-05-30T16:44:46.938680+0000","last_clean":"2024-05-30T16:44:46.938680+0000","last_became_active":"2024-05-30T16:44:45.934191+0000","last_became_peered":"2024-05-30T16:44:45.934191+0000","last_unstale":"2024-05-30T16:44:46.938680+0000","last_undegraded":"2024-05-30T16:44:46.938680+0000","last_fullsized":"2024-05-30T16:44:46.938680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:16.535318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.945968+0000","last_change":"2024-05-30T16:44:45.923509+0000","last_active":"2024-05-30T16:44:46.945968+0000","last_peered":"2024-05-30T16:44:46.945968+0000","last_clean":"2024-05-30T16:44:46.945968+0000","last_became_active":"2024-05-30T16:44:45.923188+0000","last_became_peered":"2024-05-30T16:44:45.923188+0000","last_unstale":"2024-05-30T16:44:46.945968+0000","last_undegraded":"2024-05-30T16:44:46.945968+0000","last_fullsized":"2024-05-30T16:44:46.945968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:15.340776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911312+0000","last_change":"2024-05-30T16:44:45.922357+0000","last_active":"2024-05-30T16:44:46.911312+0000","last_peered":"2024-05-30T16:44:46.911312+0000","last_clean":"2024-05-30T16:44:46.911312+0000","last_became_active":"2024-05-30T16:44:45.922054+0000","last_became_peered":"2024-05-30T16:44:45.922054+0000","last_unstale":"2024-05-30T16:44:46.911312+0000","last_undegraded":"2024-05-30T16:44:46.911312+0000","last_fullsized":"2024-05-30T16:44:46.911312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:50:53.312917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914905+0000","last_change":"2024-05-30T16:44:45.922189+0000","last_active":"2024-05-30T16:44:46.914905+0000","last_peered":"2024-05-30T16:44:46.914905+0000","last_clean":"2024-05-30T16:44:46.914905+0000","last_became_active":"2024-05-30T16:44:45.921920+0000","last_became_peered":"2024-05-30T16:44:45.921920+0000","last_unstale":"2024-05-30T16:44:46.914905+0000","last_undegraded":"2024-05-30T16:44:46.914905+0000","last_fullsized":"2024-05-30T16:44:46.914905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:32:30.873632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912732+0000","last_change":"2024-05-30T16:44:45.918239+0000","last_active":"2024-05-30T16:44:46.912732+0000","last_peered":"2024-05-30T16:44:46.912732+0000","last_clean":"2024-05-30T16:44:46.912732+0000","last_became_active":"2024-05-30T16:44:45.917936+0000","last_became_peered":"2024-05-30T16:44:45.917936+0000","last_unstale":"2024-05-30T16:44:46.912732+0000","last_undegraded":"2024-05-30T16:44:46.912732+0000","last_fullsized":"2024-05-30T16:44:46.912732+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948199+0000","last_change":"2024-05-30T16:44:45.913561+0000","last_active":"2024-05-30T16:44:46.948199+0000","last_peered":"2024-05-30T16:44:46.948199+0000","last_clean":"2024-05-30T16:44:46.948199+0000","last_became_active":"2024-05-30T16:44:45.913310+0000","last_became_peered":"2024-05-30T16:44:45.913310+0000","last_unstale":"2024-05-30T16:44:46.948199+0000","last_undegraded":"2024-05-30T16:44:46.948199+0000","last_fullsized":"2024-05-30T16:44:46.948199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:44.819439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939202+0000","last_change":"2024-05-30T16:44:45.929035+0000","last_active":"2024-05-30T16:44:46.939202+0000","last_peered":"2024-05-30T16:44:46.939202+0000","last_clean":"2024-05-30T16:44:46.939202+0000","last_became_active":"2024-05-30T16:44:45.928481+0000","last_became_peered":"2024-05-30T16:44:45.928481+0000","last_unstale":"2024-05-30T16:44:46.939202+0000","last_undegraded":"2024-05-30T16:44:46.939202+0000","last_fullsized":"2024-05-30T16:44:46.939202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:36.738446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915320+0000","last_change":"2024-05-30T16:44:45.927692+0000","last_active":"2024-05-30T16:44:46.915320+0000","last_peered":"2024-05-30T16:44:46.915320+0000","last_clean":"2024-05-30T16:44:46.915320+0000","last_became_active":"2024-05-30T16:44:45.927381+0000","last_became_peered":"2024-05-30T16:44:45.927381+0000","last_unstale":"2024-05-30T16:44:46.915320+0000","last_undegraded":"2024-05-30T16:44:46.915320+0000","last_fullsized":"2024-05-30T16:44:46.915320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:20:16.734760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914592+0000","last_change":"2024-05-30T16:44:45.922504+0000","last_active":"2024-05-30T16:44:46.914592+0000","last_peered":"2024-05-30T16:44:46.914592+0000","last_clean":"2024-05-30T16:44:46.914592+0000","last_became_active":"2024-05-30T16:44:45.922230+0000","last_became_peered":"2024-05-30T16:44:45.922230+0000","last_unstale":"2024-05-30T16:44:46.914592+0000","last_undegraded":"2024-05-30T16:44:46.914592+0000","last_fullsized":"2024-05-30T16:44:46.914592+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946285+0000","last_change":"2024-05-30T16:44:45.915229+0000","last_active":"2024-05-30T16:44:46.946285+0000","last_peered":"2024-05-30T16:44:46.946285+0000","last_clean":"2024-05-30T16:44:46.946285+0000","last_became_active":"2024-05-30T16:44:45.914899+0000","last_became_peered":"2024-05-30T16:44:45.914899+0000","last_unstale":"2024-05-30T16:44:46.946285+0000","last_undegraded":"2024-05-30T16:44:46.946285+0000","last_fullsized":"2024-05-30T16:44:46.946285+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:45.593907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911678+0000","last_change":"2024-05-30T16:44:45.921686+0000","last_active":"2024-05-30T16:44:46.911678+0000","last_peered":"2024-05-30T16:44:46.911678+0000","last_clean":"2024-05-30T16:44:46.911678+0000","last_became_active":"2024-05-30T16:44:45.920353+0000","last_became_peered":"2024-05-30T16:44:45.920353+0000","last_unstale":"2024-05-30T16:44:46.911678+0000","last_undegraded":"2024-05-30T16:44:46.911678+0000","last_fullsized":"2024-05-30T16:44:46.911678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:00.989943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914323+0000","last_change":"2024-05-30T16:44:45.912535+0000","last_active":"2024-05-30T16:44:46.914323+0000","last_peered":"2024-05-30T16:44:46.914323+0000","last_clean":"2024-05-30T16:44:46.914323+0000","last_became_active":"2024-05-30T16:44:45.911217+0000","last_became_peered":"2024-05-30T16:44:45.911217+0000","last_unstale":"2024-05-30T16:44:46.914323+0000","last_undegraded":"2024-05-30T16:44:46.914323+0000","last_fullsized":"2024-05-30T16:44:46.914323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:10:41.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914560+0000","last_change":"2024-05-30T16:44:45.925677+0000","last_active":"2024-05-30T16:44:46.914560+0000","last_peered":"2024-05-30T16:44:46.914560+0000","last_clean":"2024-05-30T16:44:46.914560+0000","last_became_active":"2024-05-30T16:44:45.925335+0000","last_became_peered":"2024-05-30T16:44:45.925335+0000","last_unstale":"2024-05-30T16:44:46.914560+0000","last_undegraded":"2024-05-30T16:44:46.914560+0000","last_fullsized":"2024-05-30T16:44:46.914560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:24:35.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915605+0000","last_change":"2024-05-30T16:44:45.921499+0000","last_active":"2024-05-30T16:44:46.915605+0000","last_peered":"2024-05-30T16:44:46.915605+0000","last_clean":"2024-05-30T16:44:46.915605+0000","last_became_active":"2024-05-30T16:44:45.921212+0000","last_became_peered":"2024-05-30T16:44:45.921212+0000","last_unstale":"2024-05-30T16:44:46.915605+0000","last_undegraded":"2024-05-30T16:44:46.915605+0000","last_fullsized":"2024-05-30T16:44:46.915605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:14.743364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912765+0000","last_change":"2024-05-30T16:44:45.926506+0000","last_active":"2024-05-30T16:44:46.912765+0000","last_peered":"2024-05-30T16:44:46.912765+0000","last_clean":"2024-05-30T16:44:46.912765+0000","last_became_active":"2024-05-30T16:44:45.926206+0000","last_became_peered":"2024-05-30T16:44:45.926206+0000","last_unstale":"2024-05-30T16:44:46.912765+0000","last_undegraded":"2024-05-30T16:44:46.912765+0000","last_fullsized":"2024-05-30T16:44:46.912765+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:30.990650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912373+0000","last_change":"2024-05-30T16:44:45.927031+0000","last_active":"2024-05-30T16:44:46.912373+0000","last_peered":"2024-05-30T16:44:46.912373+0000","last_clean":"2024-05-30T16:44:46.912373+0000","last_became_active":"2024-05-30T16:44:45.926753+0000","last_became_peered":"2024-05-30T16:44:45.926753+0000","last_unstale":"2024-05-30T16:44:46.912373+0000","last_undegraded":"2024-05-30T16:44:46.912373+0000","last_fullsized":"2024-05-30T16:44:46.912373+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:08:31.486476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946549+0000","last_change":"2024-05-30T16:44:45.923222+0000","last_active":"2024-05-30T16:44:46.946549+0000","last_peered":"2024-05-30T16:44:46.946549+0000","last_clean":"2024-05-30T16:44:46.946549+0000","last_became_active":"2024-05-30T16:44:45.922944+0000","last_became_peered":"2024-05-30T16:44:45.922944+0000","last_unstale":"2024-05-30T16:44:46.946549+0000","last_undegraded":"2024-05-30T16:44:46.946549+0000","last_fullsized":"2024-05-30T16:44:46.946549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:53.970228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947387+0000","last_change":"2024-05-30T16:44:45.916346+0000","last_active":"2024-05-30T16:44:46.947387+0000","last_peered":"2024-05-30T16:44:46.947387+0000","last_clean":"2024-05-30T16:44:46.947387+0000","last_became_active":"2024-05-30T16:44:45.914039+0000","last_became_peered":"2024-05-30T16:44:45.914039+0000","last_unstale":"2024-05-30T16:44:46.947387+0000","last_undegraded":"2024-05-30T16:44:46.947387+0000","last_fullsized":"2024-05-30T16:44:46.947387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:45.611361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947069+0000","last_change":"2024-05-30T16:44:45.932304+0000","last_active":"2024-05-30T16:44:46.947069+0000","last_peered":"2024-05-30T16:44:46.947069+0000","last_clean":"2024-05-30T16:44:46.947069+0000","last_became_active":"2024-05-30T16:44:45.932040+0000","last_became_peered":"2024-05-30T16:44:45.932040+0000","last_unstale":"2024-05-30T16:44:46.947069+0000","last_undegraded":"2024-05-30T16:44:46.947069+0000","last_fullsized":"2024-05-30T16:44:46.947069+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:10.302887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932990+0000","last_change":"2024-05-30T16:44:45.920307+0000","last_active":"2024-05-30T16:44:46.932990+0000","last_peered":"2024-05-30T16:44:46.932990+0000","last_clean":"2024-05-30T16:44:46.932990+0000","last_became_active":"2024-05-30T16:44:45.918862+0000","last_became_peered":"2024-05-30T16:44:45.918862+0000","last_unstale":"2024-05-30T16:44:46.932990+0000","last_undegraded":"2024-05-30T16:44:46.932990+0000","last_fullsized":"2024-05-30T16:44:46.932990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:52:44.864855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912985+0000","last_change":"2024-05-30T16:44:45.909508+0000","last_active":"2024-05-30T16:44:46.912985+0000","last_peered":"2024-05-30T16:44:46.912985+0000","last_clean":"2024-05-30T16:44:46.912985+0000","last_became_active":"2024-05-30T16:44:45.904007+0000","last_became_peered":"2024-05-30T16:44:45.904007+0000","last_unstale":"2024-05-30T16:44:46.912985+0000","last_undegraded":"2024-05-30T16:44:46.912985+0000","last_fullsized":"2024-05-30T16:44:46.912985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:11.996704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939458+0000","last_change":"2024-05-30T16:44:45.926434+0000","last_active":"2024-05-30T16:44:46.939458+0000","last_peered":"2024-05-30T16:44:46.939458+0000","last_clean":"2024-05-30T16:44:46.939458+0000","last_became_active":"2024-05-30T16:44:45.925699+0000","last_became_peered":"2024-05-30T16:44:45.925699+0000","last_unstale":"2024-05-30T16:44:46.939458+0000","last_undegraded":"2024-05-30T16:44:46.939458+0000","last_fullsized":"2024-05-30T16:44:46.939458+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:58:04.880354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913006+0000","last_change":"2024-05-30T16:44:45.921752+0000","last_active":"2024-05-30T16:44:46.913006+0000","last_peered":"2024-05-30T16:44:46.913006+0000","last_clean":"2024-05-30T16:44:46.913006+0000","last_became_active":"2024-05-30T16:44:45.921371+0000","last_became_peered":"2024-05-30T16:44:45.921371+0000","last_unstale":"2024-05-30T16:44:46.913006+0000","last_undegraded":"2024-05-30T16:44:46.913006+0000","last_fullsized":"2024-05-30T16:44:46.913006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:58.405600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939331+0000","last_change":"2024-05-30T16:44:45.934317+0000","last_active":"2024-05-30T16:44:46.939331+0000","last_peered":"2024-05-30T16:44:46.939331+0000","last_clean":"2024-05-30T16:44:46.939331+0000","last_became_active":"2024-05-30T16:44:45.934006+0000","last_became_peered":"2024-05-30T16:44:45.934006+0000","last_unstale":"2024-05-30T16:44:46.939331+0000","last_undegraded":"2024-05-30T16:44:46.939331+0000","last_fullsized":"2024-05-30T16:44:46.939331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:05:02.607422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915334+0000","last_change":"2024-05-30T16:44:45.908143+0000","last_active":"2024-05-30T16:44:46.915334+0000","last_peered":"2024-05-30T16:44:46.915334+0000","last_clean":"2024-05-30T16:44:46.915334+0000","last_became_active":"2024-05-30T16:44:45.907830+0000","last_became_peered":"2024-05-30T16:44:45.907830+0000","last_unstale":"2024-05-30T16:44:46.915334+0000","last_undegraded":"2024-05-30T16:44:46.915334+0000","last_fullsized":"2024-05-30T16:44:46.915334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948751+0000","last_change":"2024-05-30T16:44:45.928573+0000","last_active":"2024-05-30T16:44:46.948751+0000","last_peered":"2024-05-30T16:44:46.948751+0000","last_clean":"2024-05-30T16:44:46.948751+0000","last_became_active":"2024-05-30T16:44:45.928354+0000","last_became_peered":"2024-05-30T16:44:45.928354+0000","last_unstale":"2024-05-30T16:44:46.948751+0000","last_undegraded":"2024-05-30T16:44:46.948751+0000","last_fullsized":"2024-05-30T16:44:46.948751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:13.039069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913062+0000","last_change":"2024-05-30T16:44:45.926086+0000","last_active":"2024-05-30T16:44:46.913062+0000","last_peered":"2024-05-30T16:44:46.913062+0000","last_clean":"2024-05-30T16:44:46.913062+0000","last_became_active":"2024-05-30T16:44:45.921839+0000","last_became_peered":"2024-05-30T16:44:45.921839+0000","last_unstale":"2024-05-30T16:44:46.913062+0000","last_undegraded":"2024-05-30T16:44:46.913062+0000","last_fullsized":"2024-05-30T16:44:46.913062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948177+0000","last_change":"2024-05-30T16:44:45.908086+0000","last_active":"2024-05-30T16:44:46.948177+0000","last_peered":"2024-05-30T16:44:46.948177+0000","last_clean":"2024-05-30T16:44:46.948177+0000","last_became_active":"2024-05-30T16:44:45.907793+0000","last_became_peered":"2024-05-30T16:44:45.907793+0000","last_unstale":"2024-05-30T16:44:46.948177+0000","last_undegraded":"2024-05-30T16:44:46.948177+0000","last_fullsized":"2024-05-30T16:44:46.948177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:17.249731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912803+0000","last_change":"2024-05-30T16:44:45.931329+0000","last_active":"2024-05-30T16:44:46.912803+0000","last_peered":"2024-05-30T16:44:46.912803+0000","last_clean":"2024-05-30T16:44:46.912803+0000","last_became_active":"2024-05-30T16:44:45.929807+0000","last_became_peered":"2024-05-30T16:44:45.929807+0000","last_unstale":"2024-05-30T16:44:46.912803+0000","last_undegraded":"2024-05-30T16:44:46.912803+0000","last_fullsized":"2024-05-30T16:44:46.912803+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933144+0000","last_change":"2024-05-30T16:44:45.918314+0000","last_active":"2024-05-30T16:44:46.933144+0000","last_peered":"2024-05-30T16:44:46.933144+0000","last_clean":"2024-05-30T16:44:46.933144+0000","last_became_active":"2024-05-30T16:44:45.917998+0000","last_became_peered":"2024-05-30T16:44:45.917998+0000","last_unstale":"2024-05-30T16:44:46.933144+0000","last_undegraded":"2024-05-30T16:44:46.933144+0000","last_fullsized":"2024-05-30T16:44:46.933144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:09.909703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.949118+0000","last_change":"2024-05-30T16:44:44.892355+0000","last_active":"2024-05-30T16:44:46.949118+0000","last_peered":"2024-05-30T16:44:46.949118+0000","last_clean":"2024-05-30T16:44:46.949118+0000","last_became_active":"2024-05-30T16:44:44.891889+0000","last_became_peered":"2024-05-30T16:44:44.891889+0000","last_unstale":"2024-05-30T16:44:46.949118+0000","last_undegraded":"2024-05-30T16:44:46.949118+0000","last_fullsized":"2024-05-30T16:44:46.949118+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:44.589105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.949261+0000","last_change":"2024-05-30T16:44:45.934290+0000","last_active":"2024-05-30T16:44:46.949261+0000","last_peered":"2024-05-30T16:44:46.949261+0000","last_clean":"2024-05-30T16:44:46.949261+0000","last_became_active":"2024-05-30T16:44:45.933465+0000","last_became_peered":"2024-05-30T16:44:45.933465+0000","last_unstale":"2024-05-30T16:44:46.949261+0000","last_undegraded":"2024-05-30T16:44:46.949261+0000","last_fullsized":"2024-05-30T16:44:46.949261+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:10.302887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932836+0000","last_change":"2024-05-30T16:44:44.889925+0000","last_active":"2024-05-30T16:44:46.932836+0000","last_peered":"2024-05-30T16:44:46.932836+0000","last_clean":"2024-05-30T16:44:46.932836+0000","last_became_active":"2024-05-30T16:44:44.889564+0000","last_became_peered":"2024-05-30T16:44:44.889564+0000","last_unstale":"2024-05-30T16:44:46.932836+0000","last_undegraded":"2024-05-30T16:44:46.932836+0000","last_fullsized":"2024-05-30T16:44:46.932836+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:45.960008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.949413+0000","last_change":"2024-05-30T16:44:45.931771+0000","last_active":"2024-05-30T16:44:46.949413+0000","last_peered":"2024-05-30T16:44:46.949413+0000","last_clean":"2024-05-30T16:44:46.949413+0000","last_became_active":"2024-05-30T16:44:45.931320+0000","last_became_peered":"2024-05-30T16:44:45.931320+0000","last_unstale":"2024-05-30T16:44:46.949413+0000","last_undegraded":"2024-05-30T16:44:46.949413+0000","last_fullsized":"2024-05-30T16:44:46.949413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:16.535318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912059+0000","last_change":"2024-05-30T16:44:44.883317+0000","last_active":"2024-05-30T16:44:46.912059+0000","last_peered":"2024-05-30T16:44:46.912059+0000","last_clean":"2024-05-30T16:44:46.912059+0000","last_became_active":"2024-05-30T16:44:44.882994+0000","last_became_peered":"2024-05-30T16:44:44.882994+0000","last_unstale":"2024-05-30T16:44:46.912059+0000","last_undegraded":"2024-05-30T16:44:46.912059+0000","last_fullsized":"2024-05-30T16:44:46.912059+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:29:54.140992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939544+0000","last_change":"2024-05-30T16:44:45.923316+0000","last_active":"2024-05-30T16:44:46.939544+0000","last_peered":"2024-05-30T16:44:46.939544+0000","last_clean":"2024-05-30T16:44:46.939544+0000","last_became_active":"2024-05-30T16:44:45.922728+0000","last_became_peered":"2024-05-30T16:44:45.922728+0000","last_unstale":"2024-05-30T16:44:46.939544+0000","last_undegraded":"2024-05-30T16:44:46.939544+0000","last_fullsized":"2024-05-30T16:44:46.939544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:42.475947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946479+0000","last_change":"2024-05-30T16:44:44.886536+0000","last_active":"2024-05-30T16:44:46.946479+0000","last_peered":"2024-05-30T16:44:46.946479+0000","last_clean":"2024-05-30T16:44:46.946479+0000","last_became_active":"2024-05-30T16:44:44.886261+0000","last_became_peered":"2024-05-30T16:44:44.886261+0000","last_unstale":"2024-05-30T16:44:46.946479+0000","last_undegraded":"2024-05-30T16:44:46.946479+0000","last_fullsized":"2024-05-30T16:44:46.946479+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:53.783765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913886+0000","last_change":"2024-05-30T16:44:45.936295+0000","last_active":"2024-05-30T16:44:46.913886+0000","last_peered":"2024-05-30T16:44:46.913886+0000","last_clean":"2024-05-30T16:44:46.913886+0000","last_became_active":"2024-05-30T16:44:45.935416+0000","last_became_peered":"2024-05-30T16:44:45.935416+0000","last_unstale":"2024-05-30T16:44:46.913886+0000","last_undegraded":"2024-05-30T16:44:46.913886+0000","last_fullsized":"2024-05-30T16:44:46.913886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947689+0000","last_change":"2024-05-30T16:44:44.886562+0000","last_active":"2024-05-30T16:44:46.947689+0000","last_peered":"2024-05-30T16:44:46.947689+0000","last_clean":"2024-05-30T16:44:46.947689+0000","last_became_active":"2024-05-30T16:44:44.886254+0000","last_became_peered":"2024-05-30T16:44:44.886254+0000","last_unstale":"2024-05-30T16:44:46.947689+0000","last_undegraded":"2024-05-30T16:44:46.947689+0000","last_fullsized":"2024-05-30T16:44:46.947689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:37.347555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913996+0000","last_change":"2024-05-30T16:44:45.905732+0000","last_active":"2024-05-30T16:44:46.913996+0000","last_peered":"2024-05-30T16:44:46.913996+0000","last_clean":"2024-05-30T16:44:46.913996+0000","last_became_active":"2024-05-30T16:44:45.905117+0000","last_became_peered":"2024-05-30T16:44:45.905117+0000","last_unstale":"2024-05-30T16:44:46.913996+0000","last_undegraded":"2024-05-30T16:44:46.913996+0000","last_fullsized":"2024-05-30T16:44:46.913996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:55:15.879718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947140+0000","last_change":"2024-05-30T16:44:44.879442+0000","last_active":"2024-05-30T16:44:46.947140+0000","last_peered":"2024-05-30T16:44:46.947140+0000","last_clean":"2024-05-30T16:44:46.947140+0000","last_became_active":"2024-05-30T16:44:44.877334+0000","last_became_peered":"2024-05-30T16:44:44.877334+0000","last_unstale":"2024-05-30T16:44:46.947140+0000","last_undegraded":"2024-05-30T16:44:46.947140+0000","last_fullsized":"2024-05-30T16:44:46.947140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:44.589105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946788+0000","last_change":"2024-05-30T16:44:45.916967+0000","last_active":"2024-05-30T16:44:46.946788+0000","last_peered":"2024-05-30T16:44:46.946788+0000","last_clean":"2024-05-30T16:44:46.946788+0000","last_became_active":"2024-05-30T16:44:45.916682+0000","last_became_peered":"2024-05-30T16:44:45.916682+0000","last_unstale":"2024-05-30T16:44:46.946788+0000","last_undegraded":"2024-05-30T16:44:46.946788+0000","last_fullsized":"2024-05-30T16:44:46.946788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:00.032212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911056+0000","last_change":"2024-05-30T16:44:44.876950+0000","last_active":"2024-05-30T16:44:46.911056+0000","last_peered":"2024-05-30T16:44:46.911056+0000","last_clean":"2024-05-30T16:44:46.911056+0000","last_became_active":"2024-05-30T16:44:44.876666+0000","last_became_peered":"2024-05-30T16:44:44.876666+0000","last_unstale":"2024-05-30T16:44:46.911056+0000","last_undegraded":"2024-05-30T16:44:46.911056+0000","last_fullsized":"2024-05-30T16:44:46.911056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:16.244929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938130+0000","last_change":"2024-05-30T16:44:45.930629+0000","last_active":"2024-05-30T16:44:46.938130+0000","last_peered":"2024-05-30T16:44:46.938130+0000","last_clean":"2024-05-30T16:44:46.938130+0000","last_became_active":"2024-05-30T16:44:45.921414+0000","last_became_peered":"2024-05-30T16:44:45.921414+0000","last_unstale":"2024-05-30T16:44:46.938130+0000","last_undegraded":"2024-05-30T16:44:46.938130+0000","last_fullsized":"2024-05-30T16:44:46.938130+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:44.819439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946576+0000","last_change":"2024-05-30T16:44:44.887604+0000","last_active":"2024-05-30T16:44:46.946576+0000","last_peered":"2024-05-30T16:44:46.946576+0000","last_clean":"2024-05-30T16:44:46.946576+0000","last_became_active":"2024-05-30T16:44:44.887259+0000","last_became_peered":"2024-05-30T16:44:44.887259+0000","last_unstale":"2024-05-30T16:44:46.946576+0000","last_undegraded":"2024-05-30T16:44:46.946576+0000","last_fullsized":"2024-05-30T16:44:46.946576+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:15:02.759428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915928+0000","last_change":"2024-05-30T16:44:45.918490+0000","last_active":"2024-05-30T16:44:46.915928+0000","last_peered":"2024-05-30T16:44:46.915928+0000","last_clean":"2024-05-30T16:44:46.915928+0000","last_became_active":"2024-05-30T16:44:45.917451+0000","last_became_peered":"2024-05-30T16:44:45.917451+0000","last_unstale":"2024-05-30T16:44:46.915928+0000","last_undegraded":"2024-05-30T16:44:46.915928+0000","last_fullsized":"2024-05-30T16:44:46.915928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912835+0000","last_change":"2024-05-30T16:44:44.889369+0000","last_active":"2024-05-30T16:44:46.912835+0000","last_peered":"2024-05-30T16:44:46.912835+0000","last_clean":"2024-05-30T16:44:46.912835+0000","last_became_active":"2024-05-30T16:44:44.889051+0000","last_became_peered":"2024-05-30T16:44:44.889051+0000","last_unstale":"2024-05-30T16:44:46.912835+0000","last_undegraded":"2024-05-30T16:44:46.912835+0000","last_fullsized":"2024-05-30T16:44:46.912835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:44.606560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912841+0000","last_change":"2024-05-30T16:44:45.924478+0000","last_active":"2024-05-30T16:44:46.912841+0000","last_peered":"2024-05-30T16:44:46.912841+0000","last_clean":"2024-05-30T16:44:46.912841+0000","last_became_active":"2024-05-30T16:44:45.922542+0000","last_became_peered":"2024-05-30T16:44:45.922542+0000","last_unstale":"2024-05-30T16:44:46.912841+0000","last_undegraded":"2024-05-30T16:44:46.912841+0000","last_fullsized":"2024-05-30T16:44:46.912841+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:51:40.612481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932945+0000","last_change":"2024-05-30T16:44:44.885815+0000","last_active":"2024-05-30T16:44:46.932945+0000","last_peered":"2024-05-30T16:44:46.932945+0000","last_clean":"2024-05-30T16:44:46.932945+0000","last_became_active":"2024-05-30T16:44:44.885007+0000","last_became_peered":"2024-05-30T16:44:44.885007+0000","last_unstale":"2024-05-30T16:44:46.932945+0000","last_undegraded":"2024-05-30T16:44:46.932945+0000","last_fullsized":"2024-05-30T16:44:46.932945+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:24:02.943052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912059+0000","last_change":"2024-05-30T16:44:45.921702+0000","last_active":"2024-05-30T16:44:46.912059+0000","last_peered":"2024-05-30T16:44:46.912059+0000","last_clean":"2024-05-30T16:44:46.912059+0000","last_became_active":"2024-05-30T16:44:45.921427+0000","last_became_peered":"2024-05-30T16:44:45.921427+0000","last_unstale":"2024-05-30T16:44:46.912059+0000","last_undegraded":"2024-05-30T16:44:46.912059+0000","last_fullsized":"2024-05-30T16:44:46.912059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:07.509629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915318+0000","last_change":"2024-05-30T16:44:44.891929+0000","last_active":"2024-05-30T16:44:46.915318+0000","last_peered":"2024-05-30T16:44:46.915318+0000","last_clean":"2024-05-30T16:44:46.915318+0000","last_became_active":"2024-05-30T16:44:44.891611+0000","last_became_peered":"2024-05-30T16:44:44.891611+0000","last_unstale":"2024-05-30T16:44:46.915318+0000","last_undegraded":"2024-05-30T16:44:46.915318+0000","last_fullsized":"2024-05-30T16:44:46.915318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:41.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914621+0000","last_change":"2024-05-30T16:44:45.915178+0000","last_active":"2024-05-30T16:44:46.914621+0000","last_peered":"2024-05-30T16:44:46.914621+0000","last_clean":"2024-05-30T16:44:46.914621+0000","last_became_active":"2024-05-30T16:44:45.914901+0000","last_became_peered":"2024-05-30T16:44:45.914901+0000","last_unstale":"2024-05-30T16:44:46.914621+0000","last_undegraded":"2024-05-30T16:44:46.914621+0000","last_fullsized":"2024-05-30T16:44:46.914621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912180+0000","last_change":"2024-05-30T16:44:44.884173+0000","last_active":"2024-05-30T16:44:46.912180+0000","last_peered":"2024-05-30T16:44:46.912180+0000","last_clean":"2024-05-30T16:44:46.912180+0000","last_became_active":"2024-05-30T16:44:44.883842+0000","last_became_peered":"2024-05-30T16:44:44.883842+0000","last_unstale":"2024-05-30T16:44:46.912180+0000","last_undegraded":"2024-05-30T16:44:46.912180+0000","last_fullsized":"2024-05-30T16:44:46.912180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:43.814638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938290+0000","last_change":"2024-05-30T16:44:45.922734+0000","last_active":"2024-05-30T16:44:46.938290+0000","last_peered":"2024-05-30T16:44:46.938290+0000","last_clean":"2024-05-30T16:44:46.938290+0000","last_became_active":"2024-05-30T16:44:45.921882+0000","last_became_peered":"2024-05-30T16:44:45.921882+0000","last_unstale":"2024-05-30T16:44:46.938290+0000","last_undegraded":"2024-05-30T16:44:46.938290+0000","last_fullsized":"2024-05-30T16:44:46.938290+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:13.039069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947991+0000","last_change":"2024-05-30T16:44:44.888087+0000","last_active":"2024-05-30T16:44:46.947991+0000","last_peered":"2024-05-30T16:44:46.947991+0000","last_clean":"2024-05-30T16:44:46.947991+0000","last_became_active":"2024-05-30T16:44:44.887807+0000","last_became_peered":"2024-05-30T16:44:44.887807+0000","last_unstale":"2024-05-30T16:44:46.947991+0000","last_undegraded":"2024-05-30T16:44:46.947991+0000","last_fullsized":"2024-05-30T16:44:46.947991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:02:28.009022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912092+0000","last_change":"2024-05-30T16:44:45.915303+0000","last_active":"2024-05-30T16:44:46.912092+0000","last_peered":"2024-05-30T16:44:46.912092+0000","last_clean":"2024-05-30T16:44:46.912092+0000","last_became_active":"2024-05-30T16:44:45.915028+0000","last_became_peered":"2024-05-30T16:44:45.915028+0000","last_unstale":"2024-05-30T16:44:46.912092+0000","last_undegraded":"2024-05-30T16:44:46.912092+0000","last_fullsized":"2024-05-30T16:44:46.912092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:46.821070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938260+0000","last_change":"2024-05-30T16:44:44.891918+0000","last_active":"2024-05-30T16:44:46.938260+0000","last_peered":"2024-05-30T16:44:46.938260+0000","last_clean":"2024-05-30T16:44:46.938260+0000","last_became_active":"2024-05-30T16:44:44.891134+0000","last_became_peered":"2024-05-30T16:44:44.891134+0000","last_unstale":"2024-05-30T16:44:46.938260+0000","last_undegraded":"2024-05-30T16:44:46.938260+0000","last_fullsized":"2024-05-30T16:44:46.938260+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:02:28.009022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913160+0000","last_change":"2024-05-30T16:44:45.931105+0000","last_active":"2024-05-30T16:44:46.913160+0000","last_peered":"2024-05-30T16:44:46.913160+0000","last_clean":"2024-05-30T16:44:46.913160+0000","last_became_active":"2024-05-30T16:44:45.929921+0000","last_became_peered":"2024-05-30T16:44:45.929921+0000","last_unstale":"2024-05-30T16:44:46.913160+0000","last_undegraded":"2024-05-30T16:44:46.913160+0000","last_fullsized":"2024-05-30T16:44:46.913160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911264+0000","last_change":"2024-05-30T16:44:44.889587+0000","last_active":"2024-05-30T16:44:46.911264+0000","last_peered":"2024-05-30T16:44:46.911264+0000","last_clean":"2024-05-30T16:44:46.911264+0000","last_became_active":"2024-05-30T16:44:44.889226+0000","last_became_peered":"2024-05-30T16:44:44.889226+0000","last_unstale":"2024-05-30T16:44:46.911264+0000","last_undegraded":"2024-05-30T16:44:46.911264+0000","last_fullsized":"2024-05-30T16:44:46.911264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:59.521947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948591+0000","last_change":"2024-05-30T16:44:45.935844+0000","last_active":"2024-05-30T16:44:46.948591+0000","last_peered":"2024-05-30T16:44:46.948591+0000","last_clean":"2024-05-30T16:44:46.948591+0000","last_became_active":"2024-05-30T16:44:45.935320+0000","last_became_peered":"2024-05-30T16:44:45.935320+0000","last_unstale":"2024-05-30T16:44:46.948591+0000","last_undegraded":"2024-05-30T16:44:46.948591+0000","last_fullsized":"2024-05-30T16:44:46.948591+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:00.526748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911046+0000","last_change":"2024-05-30T16:44:44.871442+0000","last_active":"2024-05-30T16:44:46.911046+0000","last_peered":"2024-05-30T16:44:46.911046+0000","last_clean":"2024-05-30T16:44:46.911046+0000","last_became_active":"2024-05-30T16:44:44.871162+0000","last_became_peered":"2024-05-30T16:44:44.871162+0000","last_unstale":"2024-05-30T16:44:46.911046+0000","last_undegraded":"2024-05-30T16:44:46.911046+0000","last_fullsized":"2024-05-30T16:44:46.911046+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:09.298085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947619+0000","last_change":"2024-05-30T16:44:45.921757+0000","last_active":"2024-05-30T16:44:46.947619+0000","last_peered":"2024-05-30T16:44:46.947619+0000","last_clean":"2024-05-30T16:44:46.947619+0000","last_became_active":"2024-05-30T16:44:45.921468+0000","last_became_peered":"2024-05-30T16:44:45.921468+0000","last_unstale":"2024-05-30T16:44:46.947619+0000","last_undegraded":"2024-05-30T16:44:46.947619+0000","last_fullsized":"2024-05-30T16:44:46.947619+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:29:55.145793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939525+0000","last_change":"2024-05-30T16:44:44.886316+0000","last_active":"2024-05-30T16:44:46.939525+0000","last_peered":"2024-05-30T16:44:46.939525+0000","last_clean":"2024-05-30T16:44:46.939525+0000","last_became_active":"2024-05-30T16:44:44.885953+0000","last_became_peered":"2024-05-30T16:44:44.885953+0000","last_unstale":"2024-05-30T16:44:46.939525+0000","last_undegraded":"2024-05-30T16:44:46.939525+0000","last_fullsized":"2024-05-30T16:44:46.939525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:34.292596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915844+0000","last_change":"2024-05-30T16:44:45.910082+0000","last_active":"2024-05-30T16:44:46.915844+0000","last_peered":"2024-05-30T16:44:46.915844+0000","last_clean":"2024-05-30T16:44:46.915844+0000","last_became_active":"2024-05-30T16:44:45.909541+0000","last_became_peered":"2024-05-30T16:44:45.909541+0000","last_unstale":"2024-05-30T16:44:46.915844+0000","last_undegraded":"2024-05-30T16:44:46.915844+0000","last_fullsized":"2024-05-30T16:44:46.915844+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947925+0000","last_change":"2024-05-30T16:44:44.888958+0000","last_active":"2024-05-30T16:44:46.947925+0000","last_peered":"2024-05-30T16:44:46.947925+0000","last_clean":"2024-05-30T16:44:46.947925+0000","last_became_active":"2024-05-30T16:44:44.888629+0000","last_became_peered":"2024-05-30T16:44:44.888629+0000","last_unstale":"2024-05-30T16:44:46.947925+0000","last_undegraded":"2024-05-30T16:44:46.947925+0000","last_fullsized":"2024-05-30T16:44:46.947925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:16.244929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939191+0000","last_change":"2024-05-30T16:44:45.911071+0000","last_active":"2024-05-30T16:44:46.939191+0000","last_peered":"2024-05-30T16:44:46.939191+0000","last_clean":"2024-05-30T16:44:46.939191+0000","last_became_active":"2024-05-30T16:44:45.910768+0000","last_became_peered":"2024-05-30T16:44:45.910768+0000","last_unstale":"2024-05-30T16:44:46.939191+0000","last_undegraded":"2024-05-30T16:44:46.939191+0000","last_fullsized":"2024-05-30T16:44:46.939191+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:53:54.936011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948560+0000","last_change":"2024-05-30T16:44:44.887204+0000","last_active":"2024-05-30T16:44:46.948560+0000","last_peered":"2024-05-30T16:44:46.948560+0000","last_clean":"2024-05-30T16:44:46.948560+0000","last_became_active":"2024-05-30T16:44:44.886702+0000","last_became_peered":"2024-05-30T16:44:44.886702+0000","last_unstale":"2024-05-30T16:44:46.948560+0000","last_undegraded":"2024-05-30T16:44:46.948560+0000","last_fullsized":"2024-05-30T16:44:46.948560+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:33.985709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946548+0000","last_change":"2024-05-30T16:44:45.916951+0000","last_active":"2024-05-30T16:44:46.946548+0000","last_peered":"2024-05-30T16:44:46.946548+0000","last_clean":"2024-05-30T16:44:46.946548+0000","last_became_active":"2024-05-30T16:44:45.916566+0000","last_became_peered":"2024-05-30T16:44:45.916566+0000","last_unstale":"2024-05-30T16:44:46.946548+0000","last_undegraded":"2024-05-30T16:44:46.946548+0000","last_fullsized":"2024-05-30T16:44:46.946548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:34.990511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.949462+0000","last_change":"2024-05-30T16:44:44.874074+0000","last_active":"2024-05-30T16:44:46.949462+0000","last_peered":"2024-05-30T16:44:46.949462+0000","last_clean":"2024-05-30T16:44:46.949462+0000","last_became_active":"2024-05-30T16:44:44.872396+0000","last_became_peered":"2024-05-30T16:44:44.872396+0000","last_unstale":"2024-05-30T16:44:46.949462+0000","last_undegraded":"2024-05-30T16:44:46.949462+0000","last_fullsized":"2024-05-30T16:44:46.949462+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:52.965427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946316+0000","last_change":"2024-05-30T16:44:45.922182+0000","last_active":"2024-05-30T16:44:46.946316+0000","last_peered":"2024-05-30T16:44:46.946316+0000","last_clean":"2024-05-30T16:44:46.946316+0000","last_became_active":"2024-05-30T16:44:45.921770+0000","last_became_peered":"2024-05-30T16:44:45.921770+0000","last_unstale":"2024-05-30T16:44:46.946316+0000","last_undegraded":"2024-05-30T16:44:46.946316+0000","last_fullsized":"2024-05-30T16:44:46.946316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:02:29.013824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932742+0000","last_change":"2024-05-30T16:44:44.881588+0000","last_active":"2024-05-30T16:44:46.932742+0000","last_peered":"2024-05-30T16:44:46.932742+0000","last_clean":"2024-05-30T16:44:46.932742+0000","last_became_active":"2024-05-30T16:44:44.881282+0000","last_became_peered":"2024-05-30T16:44:44.881282+0000","last_unstale":"2024-05-30T16:44:46.932742+0000","last_undegraded":"2024-05-30T16:44:46.932742+0000","last_fullsized":"2024-05-30T16:44:46.932742+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:27:30.795217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946324+0000","last_change":"2024-05-30T16:44:45.910190+0000","last_active":"2024-05-30T16:44:46.946324+0000","last_peered":"2024-05-30T16:44:46.946324+0000","last_clean":"2024-05-30T16:44:46.946324+0000","last_became_active":"2024-05-30T16:44:45.909349+0000","last_became_peered":"2024-05-30T16:44:45.909349+0000","last_unstale":"2024-05-30T16:44:46.946324+0000","last_undegraded":"2024-05-30T16:44:46.946324+0000","last_fullsized":"2024-05-30T16:44:46.946324+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:35.297397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947768+0000","last_change":"2024-05-30T16:44:44.890541+0000","last_active":"2024-05-30T16:44:46.947768+0000","last_peered":"2024-05-30T16:44:46.947768+0000","last_clean":"2024-05-30T16:44:46.947768+0000","last_became_active":"2024-05-30T16:44:44.890280+0000","last_became_peered":"2024-05-30T16:44:44.890280+0000","last_unstale":"2024-05-30T16:44:46.947768+0000","last_undegraded":"2024-05-30T16:44:46.947768+0000","last_fullsized":"2024-05-30T16:44:46.947768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:14.335974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.945922+0000","last_change":"2024-05-30T16:44:45.909453+0000","last_active":"2024-05-30T16:44:46.945922+0000","last_peered":"2024-05-30T16:44:46.945922+0000","last_clean":"2024-05-30T16:44:46.945922+0000","last_became_active":"2024-05-30T16:44:45.909136+0000","last_became_peered":"2024-05-30T16:44:45.909136+0000","last_unstale":"2024-05-30T16:44:46.945922+0000","last_undegraded":"2024-05-30T16:44:46.945922+0000","last_fullsized":"2024-05-30T16:44:46.945922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:38.352356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946514+0000","last_change":"2024-05-30T16:44:44.885235+0000","last_active":"2024-05-30T16:44:46.946514+0000","last_peered":"2024-05-30T16:44:46.946514+0000","last_clean":"2024-05-30T16:44:46.946514+0000","last_became_active":"2024-05-30T16:44:44.884907+0000","last_became_peered":"2024-05-30T16:44:44.884907+0000","last_unstale":"2024-05-30T16:44:46.946514+0000","last_undegraded":"2024-05-30T16:44:46.946514+0000","last_fullsized":"2024-05-30T16:44:46.946514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:37.611751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913450+0000","last_change":"2024-05-30T16:44:45.919445+0000","last_active":"2024-05-30T16:44:46.913450+0000","last_peered":"2024-05-30T16:44:46.913450+0000","last_clean":"2024-05-30T16:44:46.913450+0000","last_became_active":"2024-05-30T16:44:45.919149+0000","last_became_peered":"2024-05-30T16:44:45.919149+0000","last_unstale":"2024-05-30T16:44:46.913450+0000","last_undegraded":"2024-05-30T16:44:46.913450+0000","last_fullsized":"2024-05-30T16:44:46.913450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913128+0000","last_change":"2024-05-30T16:44:44.887506+0000","last_active":"2024-05-30T16:44:46.913128+0000","last_peered":"2024-05-30T16:44:46.913128+0000","last_clean":"2024-05-30T16:44:46.913128+0000","last_became_active":"2024-05-30T16:44:44.887180+0000","last_became_peered":"2024-05-30T16:44:44.887180+0000","last_unstale":"2024-05-30T16:44:46.913128+0000","last_undegraded":"2024-05-30T16:44:46.913128+0000","last_fullsized":"2024-05-30T16:44:46.913128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:08.904901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946568+0000","last_change":"2024-05-30T16:44:45.917932+0000","last_active":"2024-05-30T16:44:46.946568+0000","last_peered":"2024-05-30T16:44:46.946568+0000","last_clean":"2024-05-30T16:44:46.946568+0000","last_became_active":"2024-05-30T16:44:45.917604+0000","last_became_peered":"2024-05-30T16:44:45.917604+0000","last_unstale":"2024-05-30T16:44:46.946568+0000","last_undegraded":"2024-05-30T16:44:46.946568+0000","last_fullsized":"2024-05-30T16:44:46.946568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:26.429331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939719+0000","last_change":"2024-05-30T16:44:44.891749+0000","last_active":"2024-05-30T16:44:46.939719+0000","last_peered":"2024-05-30T16:44:46.939719+0000","last_clean":"2024-05-30T16:44:46.939719+0000","last_became_active":"2024-05-30T16:44:44.891440+0000","last_became_peered":"2024-05-30T16:44:44.891440+0000","last_unstale":"2024-05-30T16:44:46.939719+0000","last_undegraded":"2024-05-30T16:44:46.939719+0000","last_fullsized":"2024-05-30T16:44:46.939719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:25.424530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.916134+0000","last_change":"2024-05-30T16:44:45.911902+0000","last_active":"2024-05-30T16:44:46.916134+0000","last_peered":"2024-05-30T16:44:46.916134+0000","last_clean":"2024-05-30T16:44:46.916134+0000","last_became_active":"2024-05-30T16:44:45.911282+0000","last_became_peered":"2024-05-30T16:44:45.911282+0000","last_unstale":"2024-05-30T16:44:46.916134+0000","last_undegraded":"2024-05-30T16:44:46.916134+0000","last_fullsized":"2024-05-30T16:44:46.916134+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933055+0000","last_change":"2024-05-30T16:44:44.885333+0000","last_active":"2024-05-30T16:44:46.933055+0000","last_peered":"2024-05-30T16:44:46.933055+0000","last_clean":"2024-05-30T16:44:46.933055+0000","last_became_active":"2024-05-30T16:44:44.883685+0000","last_became_peered":"2024-05-30T16:44:44.883685+0000","last_unstale":"2024-05-30T16:44:46.933055+0000","last_undegraded":"2024-05-30T16:44:46.933055+0000","last_fullsized":"2024-05-30T16:44:46.933055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:52.935243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932751+0000","last_change":"2024-05-30T16:44:45.915538+0000","last_active":"2024-05-30T16:44:46.932751+0000","last_peered":"2024-05-30T16:44:46.932751+0000","last_clean":"2024-05-30T16:44:46.932751+0000","last_became_active":"2024-05-30T16:44:45.914763+0000","last_became_peered":"2024-05-30T16:44:45.914763+0000","last_unstale":"2024-05-30T16:44:46.932751+0000","last_undegraded":"2024-05-30T16:44:46.932751+0000","last_fullsized":"2024-05-30T16:44:46.932751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:30.231921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939112+0000","last_change":"2024-05-30T16:44:44.888789+0000","last_active":"2024-05-30T16:44:46.939112+0000","last_peered":"2024-05-30T16:44:46.939112+0000","last_clean":"2024-05-30T16:44:46.939112+0000","last_became_active":"2024-05-30T16:44:44.888193+0000","last_became_peered":"2024-05-30T16:44:44.888193+0000","last_unstale":"2024-05-30T16:44:46.939112+0000","last_undegraded":"2024-05-30T16:44:46.939112+0000","last_fullsized":"2024-05-30T16:44:46.939112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:52.965427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947816+0000","last_change":"2024-05-30T16:44:45.918877+0000","last_active":"2024-05-30T16:44:46.947816+0000","last_peered":"2024-05-30T16:44:46.947816+0000","last_clean":"2024-05-30T16:44:46.947816+0000","last_became_active":"2024-05-30T16:44:45.915980+0000","last_became_peered":"2024-05-30T16:44:45.915980+0000","last_unstale":"2024-05-30T16:44:46.947816+0000","last_undegraded":"2024-05-30T16:44:46.947816+0000","last_fullsized":"2024-05-30T16:44:46.947816+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:50.906656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933045+0000","last_change":"2024-05-30T16:44:44.892955+0000","last_active":"2024-05-30T16:44:46.933045+0000","last_peered":"2024-05-30T16:44:46.933045+0000","last_clean":"2024-05-30T16:44:46.933045+0000","last_became_active":"2024-05-30T16:44:44.892513+0000","last_became_peered":"2024-05-30T16:44:44.892513+0000","last_unstale":"2024-05-30T16:44:46.933045+0000","last_undegraded":"2024-05-30T16:44:46.933045+0000","last_fullsized":"2024-05-30T16:44:46.933045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:04:30.373290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932947+0000","last_change":"2024-05-30T16:44:45.910492+0000","last_active":"2024-05-30T16:44:46.932947+0000","last_peered":"2024-05-30T16:44:46.932947+0000","last_clean":"2024-05-30T16:44:46.932947+0000","last_became_active":"2024-05-30T16:44:45.909824+0000","last_became_peered":"2024-05-30T16:44:45.909824+0000","last_unstale":"2024-05-30T16:44:46.932947+0000","last_undegraded":"2024-05-30T16:44:46.932947+0000","last_fullsized":"2024-05-30T16:44:46.932947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:45.491068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938449+0000","last_change":"2024-05-30T16:44:44.887201+0000","last_active":"2024-05-30T16:44:46.938449+0000","last_peered":"2024-05-30T16:44:46.938449+0000","last_clean":"2024-05-30T16:44:46.938449+0000","last_became_active":"2024-05-30T16:44:44.886748+0000","last_became_peered":"2024-05-30T16:44:44.886748+0000","last_unstale":"2024-05-30T16:44:46.938449+0000","last_undegraded":"2024-05-30T16:44:46.938449+0000","last_fullsized":"2024-05-30T16:44:46.938449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:44.589105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.910921+0000","last_change":"2024-05-30T16:44:45.929143+0000","last_active":"2024-05-30T16:44:46.910921+0000","last_peered":"2024-05-30T16:44:46.910921+0000","last_clean":"2024-05-30T16:44:46.910921+0000","last_became_active":"2024-05-30T16:44:45.928818+0000","last_became_peered":"2024-05-30T16:44:45.928818+0000","last_unstale":"2024-05-30T16:44:46.910921+0000","last_undegraded":"2024-05-30T16:44:46.910921+0000","last_fullsized":"2024-05-30T16:44:46.910921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:14:56.612071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946809+0000","last_change":"2024-05-30T16:44:44.882578+0000","last_active":"2024-05-30T16:44:46.946809+0000","last_peered":"2024-05-30T16:44:46.946809+0000","last_clean":"2024-05-30T16:44:46.946809+0000","last_became_active":"2024-05-30T16:44:44.882197+0000","last_became_peered":"2024-05-30T16:44:44.882197+0000","last_unstale":"2024-05-30T16:44:46.946809+0000","last_undegraded":"2024-05-30T16:44:46.946809+0000","last_fullsized":"2024-05-30T16:44:46.946809+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:43.725581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946528+0000","last_change":"2024-05-30T16:44:45.928687+0000","last_active":"2024-05-30T16:44:46.946528+0000","last_peered":"2024-05-30T16:44:46.946528+0000","last_clean":"2024-05-30T16:44:46.946528+0000","last_became_active":"2024-05-30T16:44:45.928347+0000","last_became_peered":"2024-05-30T16:44:45.928347+0000","last_unstale":"2024-05-30T16:44:46.946528+0000","last_undegraded":"2024-05-30T16:44:46.946528+0000","last_fullsized":"2024-05-30T16:44:46.946528+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:00.526748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.910954+0000","last_change":"2024-05-30T16:44:44.888466+0000","last_active":"2024-05-30T16:44:46.910954+0000","last_peered":"2024-05-30T16:44:46.910954+0000","last_clean":"2024-05-30T16:44:46.910954+0000","last_became_active":"2024-05-30T16:44:44.888164+0000","last_became_peered":"2024-05-30T16:44:44.888164+0000","last_unstale":"2024-05-30T16:44:46.910954+0000","last_undegraded":"2024-05-30T16:44:46.910954+0000","last_fullsized":"2024-05-30T16:44:46.910954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:45:59.027410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914547+0000","last_change":"2024-05-30T16:44:45.919696+0000","last_active":"2024-05-30T16:44:46.914547+0000","last_peered":"2024-05-30T16:44:46.914547+0000","last_clean":"2024-05-30T16:44:46.914547+0000","last_became_active":"2024-05-30T16:44:45.918248+0000","last_became_peered":"2024-05-30T16:44:45.918248+0000","last_unstale":"2024-05-30T16:44:46.914547+0000","last_undegraded":"2024-05-30T16:44:46.914547+0000","last_fullsized":"2024-05-30T16:44:46.914547+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948615+0000","last_change":"2024-05-30T16:44:44.893954+0000","last_active":"2024-05-30T16:44:46.948615+0000","last_peered":"2024-05-30T16:44:46.948615+0000","last_clean":"2024-05-30T16:44:46.948615+0000","last_became_active":"2024-05-30T16:44:44.893629+0000","last_became_peered":"2024-05-30T16:44:44.893629+0000","last_unstale":"2024-05-30T16:44:46.948615+0000","last_undegraded":"2024-05-30T16:44:46.948615+0000","last_fullsized":"2024-05-30T16:44:46.948615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:52.965427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913336+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:44:46.913336+0000","last_peered":"2024-05-30T16:44:46.913336+0000","last_clean":"2024-05-30T16:44:46.913336+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:44:46.913336+0000","last_undegraded":"2024-05-30T16:44:46.913336+0000","last_fullsized":"2024-05-30T16:44:46.913336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":153502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":152990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":154520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":152956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":154520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":743344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":153558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":743254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:44:55.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:44:55.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.683+0000 7f0f0ae00640 1 Processor -- start 2024-05-30T16:44:55.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.684+0000 7f0f0ae00640 1 -- start start 2024-05-30T16:44:55.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.685+0000 7f0f0ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f040d16a0 0x7f0f040cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:55.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.685+0000 7f0f0ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0f040cf870 0x7f0f040d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:55.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.685+0000 7f0f0ae00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f0415ec50 con 0x7f0f0415e870 2024-05-30T16:44:55.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.685+0000 7f0f0ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0f040d2150 con 0x7f0f040cf870 2024-05-30T16:44:55.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.685+0000 7f0f0ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f040cfd20 con 0x7f0f040d16a0 2024-05-30T16:44:55.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.686+0000 7f0f09a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0f0415e870 0x7f0f040d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54850/0 (socket says 172.21.15.67:54850) 2024-05-30T16:44:55.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.686+0000 7f0f09a00640 1 -- 172.21.15.67:0/1650506756 learned_addr learned my addr 172.21.15.67:0/1650506756 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:55.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.686+0000 7f0f0a400640 1 --2- 172.21.15.67:0/1650506756 >> v2:172.21.15.169:3300/0 conn(0x7f0f040cf870 0x7f0f040d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.686+0000 7f0f09000640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f040d16a0 0x7f0f040cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.686+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1321693998 0 0) 0x7f0f0415ec50 con 0x7f0f0415e870 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.686+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0eec003520 con 0x7f0f0415e870 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1073645164 0 0) 0x7f0eec003520 con 0x7f0f0415e870 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f040d16a0 msgr2=0x7f0f040cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f040d16a0 0x7f0f040cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 >> v2:172.21.15.169:3300/0 conn(0x7f0f040cf870 msgr2=0x7f0f040d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 --2- 172.21.15.67:0/1650506756 >> v2:172.21.15.169:3300/0 conn(0x7f0f040cf870 0x7f0f040d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f040cff80 con 0x7f0f0415e870 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f0ef8001670 con 0x7f0f0415e870 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 >> v1:172.21.15.67:6789/0 conn(0x7f0f0415e870 legacy=0x7f0f040d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:55.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eec0039b0 0x7f0eec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eec004800 0x7f0eec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f040d16a0 0x7f0eec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0f040d2150 con 0x7f0eec004800 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f040cfd20 con 0x7f0eec0039b0 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0eec00b9c0 con 0x7f0f040d16a0 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.687+0000 7f0f09000640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eec004800 0x7f0eec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.688+0000 7f0f09000640 1 -- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f040d16a0 msgr2=0x7f0eec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.688+0000 7f0f09a00640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eec0039b0 0x7f0eec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.688+0000 7f0f0a400640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f040d16a0 0x7f0eec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.688+0000 7f0f09000640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f040d16a0 0x7f0eec0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.688+0000 7f0f09000640 1 -- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eec0039b0 msgr2=0x7f0eec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.688+0000 7f0f09000640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eec0039b0 0x7f0eec003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.688+0000 7f0f09000640 1 -- 172.21.15.67:0/1650506756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eec003520 con 0x7f0eec004800 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.689+0000 7f0f09000640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eec004800 0x7f0eec004c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009ab0 tx=0x7f0eec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5defe64aea70689 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.689+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4036070 con 0x7f0eec004800 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.689+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef4005990 con 0x7f0eec004800 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.689+0000 7f0f03e00640 1 -- 172.21.15.67:0/1650506756 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4009680 con 0x7f0eec004800 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.690+0000 7f0f0ae00640 1 -- 172.21.15.67:0/1650506756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eec004800 msgr2=0x7f0eec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.690+0000 7f0f0ae00640 1 --2- 172.21.15.67:0/1650506756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eec004800 0x7f0eec004c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009ab0 tx=0x7f0eec005830 comp rx=0 tx=0).stop 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.690+0000 7f0f0a400640 1 -- 172.21.15.67:0/1650506756 reap_dead start 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.690+0000 7f0f0ae00640 1 -- 172.21.15.67:0/1650506756 shutdown_connections 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.690+0000 7f0f0ae00640 1 -- 172.21.15.67:0/1650506756 >> 172.21.15.67:0/1650506756 conn(0x7f0f0408bfc0 msgr2=0x7f0f040d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.690+0000 7f0f0ae00640 1 -- 172.21.15.67:0/1650506756 shutdown_connections 2024-05-30T16:44:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.690+0000 7f0f0ae00640 1 -- 172.21.15.67:0/1650506756 wait complete. 2024-05-30T16:44:55.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.692+0000 7f0f0ae00640 1 Processor -- start 2024-05-30T16:44:55.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.692+0000 7f0f0ae00640 1 -- start start 2024-05-30T16:44:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.693+0000 7f0f0ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eec004800 0x7f0f041f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.693+0000 7f0f0ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f040cb190 0x7f0f040cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.693+0000 7f0f0ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f041f75f0 0x7f0f041f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.693+0000 7f0f0ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0f040d0820 con 0x7f0f040cb190 2024-05-30T16:44:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.693+0000 7f0f0ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f040d0540 con 0x7f0f041f75f0 2024-05-30T16:44:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.693+0000 7f0f0ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f040d4fb0 con 0x7f0eec004800 2024-05-30T16:44:55.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f040cb190 0x7f0f040cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:55.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f040cb190 0x7f0f040cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46672/0 (socket says 172.21.15.67:46672) 2024-05-30T16:44:55.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09000640 1 -- 172.21.15.67:0/2136468345 learned_addr learned my addr 172.21.15.67:0/2136468345 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:55.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f0a400640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f041f75f0 0x7f0f041f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:55.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09a00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eec004800 0x7f0f041f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:55.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09000640 1 -- 172.21.15.67:0/2136468345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eec004800 msgr2=0x7f0f041f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:55.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09000640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eec004800 0x7f0f041f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09000640 1 -- 172.21.15.67:0/2136468345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f041f75f0 msgr2=0x7f0f041f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09000640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f041f75f0 0x7f0f041f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09000640 1 -- 172.21.15.67:0/2136468345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f040cc960 con 0x7f0f040cb190 2024-05-30T16:44:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.694+0000 7f0f09a00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eec004800 0x7f0f041f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.695+0000 7f0f09000640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f040cb190 0x7f0f040cc030 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009ba0 tx=0x7f0ef4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.695+0000 7f0f01600640 1 -- 172.21.15.67:0/2136468345 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4040030 con 0x7f0f040cb190 2024-05-30T16:44:55.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.695+0000 7f0f01600640 1 -- 172.21.15.67:0/2136468345 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef4037040 con 0x7f0f040cb190 2024-05-30T16:44:55.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.695+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f040ccc40 con 0x7f0f040cb190 2024-05-30T16:44:55.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.695+0000 7f0f01600640 1 -- 172.21.15.67:0/2136468345 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4036030 con 0x7f0f040cb190 2024-05-30T16:44:55.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.695+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f040cc570 con 0x7f0f040cb190 2024-05-30T16:44:55.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.696+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f040689e0 con 0x7f0f040cb190 2024-05-30T16:44:55.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.697+0000 7f0f01600640 1 -- 172.21.15.67:0/2136468345 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ef4043070 con 0x7f0f040cb190 2024-05-30T16:44:55.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.698+0000 7f0f01600640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ee806c8f0 0x7f0ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:55.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.698+0000 7f0f01600640 1 -- 172.21.15.67:0/2136468345 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0ef40ab2f0 con 0x7f0f040cb190 2024-05-30T16:44:55.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.698+0000 7f0f09a00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ee806c8f0 0x7f0ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:55.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.699+0000 7f0f09a00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ee806c8f0 0x7f0ee806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8001eb0 tx=0x7f0ef8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:55.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.703+0000 7f0f01600640 1 -- 172.21.15.67:0/2136468345 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ef40790c0 con 0x7f0f040cb190 2024-05-30T16:44:55.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.901+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f041f3f40 con 0x7f0ee806c8f0 2024-05-30T16:44:55.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.909+0000 7f0f01600640 1 -- 172.21.15.67:0/2136468345 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7f0f041f3f40 con 0x7f0ee806c8f0 2024-05-30T16:44:55.912 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:55.915 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:44:55.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ee806c8f0 msgr2=0x7f0ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:55.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ee806c8f0 0x7f0ee806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8001eb0 tx=0x7f0ef8009040 comp rx=0 tx=0).stop 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f040cb190 msgr2=0x7f0f040cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f040cb190 0x7f0f040cc030 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009ba0 tx=0x7f0ef4004730 comp rx=0 tx=0).stop 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 shutdown_connections 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ee806c8f0 0x7f0ee806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f041f75f0 0x7f0f041f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f040cb190 0x7f0f040cc030 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 --2- 172.21.15.67:0/2136468345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eec004800 0x7f0f041f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.915+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 >> 172.21.15.67:0/2136468345 conn(0x7f0f0408bfc0 msgr2=0x7f0f040b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.916+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 shutdown_connections 2024-05-30T16:44:55.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:55.916+0000 7f0f0ae00640 1 -- 172.21.15.67:0/2136468345 wait complete. 2024-05-30T16:44:55.938 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-05-30T16:44:53.997906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220484,"kb_used_data":4264,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754236,"statfs":{"total":773094113280,"available":772868337664,"internally_reserved":0,"allocated":4366336,"data_stored":2408644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.117223"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948948+0000","last_change":"2024-05-30T16:44:45.920620+0000","last_active":"2024-05-30T16:44:46.948948+0000","last_peered":"2024-05-30T16:44:46.948948+0000","last_clean":"2024-05-30T16:44:46.948948+0000","last_became_active":"2024-05-30T16:44:45.920342+0000","last_became_peered":"2024-05-30T16:44:45.920342+0000","last_unstale":"2024-05-30T16:44:46.948948+0000","last_undegraded":"2024-05-30T16:44:46.948948+0000","last_fullsized":"2024-05-30T16:44:46.948948+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:00.032212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912434+0000","last_change":"2024-05-30T16:44:45.919659+0000","last_active":"2024-05-30T16:44:46.912434+0000","last_peered":"2024-05-30T16:44:46.912434+0000","last_clean":"2024-05-30T16:44:46.912434+0000","last_became_active":"2024-05-30T16:44:45.919335+0000","last_became_peered":"2024-05-30T16:44:45.919335+0000","last_unstale":"2024-05-30T16:44:46.912434+0000","last_undegraded":"2024-05-30T16:44:46.912434+0000","last_fullsized":"2024-05-30T16:44:46.912434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:08.259292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915144+0000","last_change":"2024-05-30T16:44:45.915063+0000","last_active":"2024-05-30T16:44:46.915144+0000","last_peered":"2024-05-30T16:44:46.915144+0000","last_clean":"2024-05-30T16:44:46.915144+0000","last_became_active":"2024-05-30T16:44:45.914534+0000","last_became_peered":"2024-05-30T16:44:45.914534+0000","last_unstale":"2024-05-30T16:44:46.915144+0000","last_undegraded":"2024-05-30T16:44:46.915144+0000","last_fullsized":"2024-05-30T16:44:46.915144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938680+0000","last_change":"2024-05-30T16:44:45.934496+0000","last_active":"2024-05-30T16:44:46.938680+0000","last_peered":"2024-05-30T16:44:46.938680+0000","last_clean":"2024-05-30T16:44:46.938680+0000","last_became_active":"2024-05-30T16:44:45.934191+0000","last_became_peered":"2024-05-30T16:44:45.934191+0000","last_unstale":"2024-05-30T16:44:46.938680+0000","last_undegraded":"2024-05-30T16:44:46.938680+0000","last_fullsized":"2024-05-30T16:44:46.938680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:16.535318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.945968+0000","last_change":"2024-05-30T16:44:45.923509+0000","last_active":"2024-05-30T16:44:46.945968+0000","last_peered":"2024-05-30T16:44:46.945968+0000","last_clean":"2024-05-30T16:44:46.945968+0000","last_became_active":"2024-05-30T16:44:45.923188+0000","last_became_peered":"2024-05-30T16:44:45.923188+0000","last_unstale":"2024-05-30T16:44:46.945968+0000","last_undegraded":"2024-05-30T16:44:46.945968+0000","last_fullsized":"2024-05-30T16:44:46.945968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:15.340776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911312+0000","last_change":"2024-05-30T16:44:45.922357+0000","last_active":"2024-05-30T16:44:46.911312+0000","last_peered":"2024-05-30T16:44:46.911312+0000","last_clean":"2024-05-30T16:44:46.911312+0000","last_became_active":"2024-05-30T16:44:45.922054+0000","last_became_peered":"2024-05-30T16:44:45.922054+0000","last_unstale":"2024-05-30T16:44:46.911312+0000","last_undegraded":"2024-05-30T16:44:46.911312+0000","last_fullsized":"2024-05-30T16:44:46.911312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:50:53.312917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914905+0000","last_change":"2024-05-30T16:44:45.922189+0000","last_active":"2024-05-30T16:44:46.914905+0000","last_peered":"2024-05-30T16:44:46.914905+0000","last_clean":"2024-05-30T16:44:46.914905+0000","last_became_active":"2024-05-30T16:44:45.921920+0000","last_became_peered":"2024-05-30T16:44:45.921920+0000","last_unstale":"2024-05-30T16:44:46.914905+0000","last_undegraded":"2024-05-30T16:44:46.914905+0000","last_fullsized":"2024-05-30T16:44:46.914905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:32:30.873632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912732+0000","last_change":"2024-05-30T16:44:45.918239+0000","last_active":"2024-05-30T16:44:46.912732+0000","last_peered":"2024-05-30T16:44:46.912732+0000","last_clean":"2024-05-30T16:44:46.912732+0000","last_became_active":"2024-05-30T16:44:45.917936+0000","last_became_peered":"2024-05-30T16:44:45.917936+0000","last_unstale":"2024-05-30T16:44:46.912732+0000","last_undegraded":"2024-05-30T16:44:46.912732+0000","last_fullsized":"2024-05-30T16:44:46.912732+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948199+0000","last_change":"2024-05-30T16:44:45.913561+0000","last_active":"2024-05-30T16:44:46.948199+0000","last_peered":"2024-05-30T16:44:46.948199+0000","last_clean":"2024-05-30T16:44:46.948199+0000","last_became_active":"2024-05-30T16:44:45.913310+0000","last_became_peered":"2024-05-30T16:44:45.913310+0000","last_unstale":"2024-05-30T16:44:46.948199+0000","last_undegraded":"2024-05-30T16:44:46.948199+0000","last_fullsized":"2024-05-30T16:44:46.948199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:44.819439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939202+0000","last_change":"2024-05-30T16:44:45.929035+0000","last_active":"2024-05-30T16:44:46.939202+0000","last_peered":"2024-05-30T16:44:46.939202+0000","last_clean":"2024-05-30T16:44:46.939202+0000","last_became_active":"2024-05-30T16:44:45.928481+0000","last_became_peered":"2024-05-30T16:44:45.928481+0000","last_unstale":"2024-05-30T16:44:46.939202+0000","last_undegraded":"2024-05-30T16:44:46.939202+0000","last_fullsized":"2024-05-30T16:44:46.939202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:47:36.738446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915320+0000","last_change":"2024-05-30T16:44:45.927692+0000","last_active":"2024-05-30T16:44:46.915320+0000","last_peered":"2024-05-30T16:44:46.915320+0000","last_clean":"2024-05-30T16:44:46.915320+0000","last_became_active":"2024-05-30T16:44:45.927381+0000","last_became_peered":"2024-05-30T16:44:45.927381+0000","last_unstale":"2024-05-30T16:44:46.915320+0000","last_undegraded":"2024-05-30T16:44:46.915320+0000","last_fullsized":"2024-05-30T16:44:46.915320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:20:16.734760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914592+0000","last_change":"2024-05-30T16:44:45.922504+0000","last_active":"2024-05-30T16:44:46.914592+0000","last_peered":"2024-05-30T16:44:46.914592+0000","last_clean":"2024-05-30T16:44:46.914592+0000","last_became_active":"2024-05-30T16:44:45.922230+0000","last_became_peered":"2024-05-30T16:44:45.922230+0000","last_unstale":"2024-05-30T16:44:46.914592+0000","last_undegraded":"2024-05-30T16:44:46.914592+0000","last_fullsized":"2024-05-30T16:44:46.914592+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:03.693389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946285+0000","last_change":"2024-05-30T16:44:45.915229+0000","last_active":"2024-05-30T16:44:46.946285+0000","last_peered":"2024-05-30T16:44:46.946285+0000","last_clean":"2024-05-30T16:44:46.946285+0000","last_became_active":"2024-05-30T16:44:45.914899+0000","last_became_peered":"2024-05-30T16:44:45.914899+0000","last_unstale":"2024-05-30T16:44:46.946285+0000","last_undegraded":"2024-05-30T16:44:46.946285+0000","last_fullsized":"2024-05-30T16:44:46.946285+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:45.593907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911678+0000","last_change":"2024-05-30T16:44:45.921686+0000","last_active":"2024-05-30T16:44:46.911678+0000","last_peered":"2024-05-30T16:44:46.911678+0000","last_clean":"2024-05-30T16:44:46.911678+0000","last_became_active":"2024-05-30T16:44:45.920353+0000","last_became_peered":"2024-05-30T16:44:45.920353+0000","last_unstale":"2024-05-30T16:44:46.911678+0000","last_undegraded":"2024-05-30T16:44:46.911678+0000","last_fullsized":"2024-05-30T16:44:46.911678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:27:00.989943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914323+0000","last_change":"2024-05-30T16:44:45.912535+0000","last_active":"2024-05-30T16:44:46.914323+0000","last_peered":"2024-05-30T16:44:46.914323+0000","last_clean":"2024-05-30T16:44:46.914323+0000","last_became_active":"2024-05-30T16:44:45.911217+0000","last_became_peered":"2024-05-30T16:44:45.911217+0000","last_unstale":"2024-05-30T16:44:46.914323+0000","last_undegraded":"2024-05-30T16:44:46.914323+0000","last_fullsized":"2024-05-30T16:44:46.914323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:10:41.045895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914560+0000","last_change":"2024-05-30T16:44:45.925677+0000","last_active":"2024-05-30T16:44:46.914560+0000","last_peered":"2024-05-30T16:44:46.914560+0000","last_clean":"2024-05-30T16:44:46.914560+0000","last_became_active":"2024-05-30T16:44:45.925335+0000","last_became_peered":"2024-05-30T16:44:45.925335+0000","last_unstale":"2024-05-30T16:44:46.914560+0000","last_undegraded":"2024-05-30T16:44:46.914560+0000","last_fullsized":"2024-05-30T16:44:46.914560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:24:35.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915605+0000","last_change":"2024-05-30T16:44:45.921499+0000","last_active":"2024-05-30T16:44:46.915605+0000","last_peered":"2024-05-30T16:44:46.915605+0000","last_clean":"2024-05-30T16:44:46.915605+0000","last_became_active":"2024-05-30T16:44:45.921212+0000","last_became_peered":"2024-05-30T16:44:45.921212+0000","last_unstale":"2024-05-30T16:44:46.915605+0000","last_undegraded":"2024-05-30T16:44:46.915605+0000","last_fullsized":"2024-05-30T16:44:46.915605+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:14.743364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912765+0000","last_change":"2024-05-30T16:44:45.926506+0000","last_active":"2024-05-30T16:44:46.912765+0000","last_peered":"2024-05-30T16:44:46.912765+0000","last_clean":"2024-05-30T16:44:46.912765+0000","last_became_active":"2024-05-30T16:44:45.926206+0000","last_became_peered":"2024-05-30T16:44:45.926206+0000","last_unstale":"2024-05-30T16:44:46.912765+0000","last_undegraded":"2024-05-30T16:44:46.912765+0000","last_fullsized":"2024-05-30T16:44:46.912765+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:30.990650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912373+0000","last_change":"2024-05-30T16:44:45.927031+0000","last_active":"2024-05-30T16:44:46.912373+0000","last_peered":"2024-05-30T16:44:46.912373+0000","last_clean":"2024-05-30T16:44:46.912373+0000","last_became_active":"2024-05-30T16:44:45.926753+0000","last_became_peered":"2024-05-30T16:44:45.926753+0000","last_unstale":"2024-05-30T16:44:46.912373+0000","last_undegraded":"2024-05-30T16:44:46.912373+0000","last_fullsized":"2024-05-30T16:44:46.912373+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:08:31.486476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946549+0000","last_change":"2024-05-30T16:44:45.923222+0000","last_active":"2024-05-30T16:44:46.946549+0000","last_peered":"2024-05-30T16:44:46.946549+0000","last_clean":"2024-05-30T16:44:46.946549+0000","last_became_active":"2024-05-30T16:44:45.922944+0000","last_became_peered":"2024-05-30T16:44:45.922944+0000","last_unstale":"2024-05-30T16:44:46.946549+0000","last_undegraded":"2024-05-30T16:44:46.946549+0000","last_fullsized":"2024-05-30T16:44:46.946549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:53.970228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947387+0000","last_change":"2024-05-30T16:44:45.916346+0000","last_active":"2024-05-30T16:44:46.947387+0000","last_peered":"2024-05-30T16:44:46.947387+0000","last_clean":"2024-05-30T16:44:46.947387+0000","last_became_active":"2024-05-30T16:44:45.914039+0000","last_became_peered":"2024-05-30T16:44:45.914039+0000","last_unstale":"2024-05-30T16:44:46.947387+0000","last_undegraded":"2024-05-30T16:44:46.947387+0000","last_fullsized":"2024-05-30T16:44:46.947387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:45.611361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947069+0000","last_change":"2024-05-30T16:44:45.932304+0000","last_active":"2024-05-30T16:44:46.947069+0000","last_peered":"2024-05-30T16:44:46.947069+0000","last_clean":"2024-05-30T16:44:46.947069+0000","last_became_active":"2024-05-30T16:44:45.932040+0000","last_became_peered":"2024-05-30T16:44:45.932040+0000","last_unstale":"2024-05-30T16:44:46.947069+0000","last_undegraded":"2024-05-30T16:44:46.947069+0000","last_fullsized":"2024-05-30T16:44:46.947069+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:10.302887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932990+0000","last_change":"2024-05-30T16:44:45.920307+0000","last_active":"2024-05-30T16:44:46.932990+0000","last_peered":"2024-05-30T16:44:46.932990+0000","last_clean":"2024-05-30T16:44:46.932990+0000","last_became_active":"2024-05-30T16:44:45.918862+0000","last_became_peered":"2024-05-30T16:44:45.918862+0000","last_unstale":"2024-05-30T16:44:46.932990+0000","last_undegraded":"2024-05-30T16:44:46.932990+0000","last_fullsized":"2024-05-30T16:44:46.932990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:52:44.864855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912985+0000","last_change":"2024-05-30T16:44:45.909508+0000","last_active":"2024-05-30T16:44:46.912985+0000","last_peered":"2024-05-30T16:44:46.912985+0000","last_clean":"2024-05-30T16:44:46.912985+0000","last_became_active":"2024-05-30T16:44:45.904007+0000","last_became_peered":"2024-05-30T16:44:45.904007+0000","last_unstale":"2024-05-30T16:44:46.912985+0000","last_undegraded":"2024-05-30T16:44:46.912985+0000","last_fullsized":"2024-05-30T16:44:46.912985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:11.996704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939458+0000","last_change":"2024-05-30T16:44:45.926434+0000","last_active":"2024-05-30T16:44:46.939458+0000","last_peered":"2024-05-30T16:44:46.939458+0000","last_clean":"2024-05-30T16:44:46.939458+0000","last_became_active":"2024-05-30T16:44:45.925699+0000","last_became_peered":"2024-05-30T16:44:45.925699+0000","last_unstale":"2024-05-30T16:44:46.939458+0000","last_undegraded":"2024-05-30T16:44:46.939458+0000","last_fullsized":"2024-05-30T16:44:46.939458+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:58:04.880354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913006+0000","last_change":"2024-05-30T16:44:45.921752+0000","last_active":"2024-05-30T16:44:46.913006+0000","last_peered":"2024-05-30T16:44:46.913006+0000","last_clean":"2024-05-30T16:44:46.913006+0000","last_became_active":"2024-05-30T16:44:45.921371+0000","last_became_peered":"2024-05-30T16:44:45.921371+0000","last_unstale":"2024-05-30T16:44:46.913006+0000","last_undegraded":"2024-05-30T16:44:46.913006+0000","last_fullsized":"2024-05-30T16:44:46.913006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:58.405600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939331+0000","last_change":"2024-05-30T16:44:45.934317+0000","last_active":"2024-05-30T16:44:46.939331+0000","last_peered":"2024-05-30T16:44:46.939331+0000","last_clean":"2024-05-30T16:44:46.939331+0000","last_became_active":"2024-05-30T16:44:45.934006+0000","last_became_peered":"2024-05-30T16:44:45.934006+0000","last_unstale":"2024-05-30T16:44:46.939331+0000","last_undegraded":"2024-05-30T16:44:46.939331+0000","last_fullsized":"2024-05-30T16:44:46.939331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:05:02.607422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915334+0000","last_change":"2024-05-30T16:44:45.908143+0000","last_active":"2024-05-30T16:44:46.915334+0000","last_peered":"2024-05-30T16:44:46.915334+0000","last_clean":"2024-05-30T16:44:46.915334+0000","last_became_active":"2024-05-30T16:44:45.907830+0000","last_became_peered":"2024-05-30T16:44:45.907830+0000","last_unstale":"2024-05-30T16:44:46.915334+0000","last_undegraded":"2024-05-30T16:44:46.915334+0000","last_fullsized":"2024-05-30T16:44:46.915334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948751+0000","last_change":"2024-05-30T16:44:45.928573+0000","last_active":"2024-05-30T16:44:46.948751+0000","last_peered":"2024-05-30T16:44:46.948751+0000","last_clean":"2024-05-30T16:44:46.948751+0000","last_became_active":"2024-05-30T16:44:45.928354+0000","last_became_peered":"2024-05-30T16:44:45.928354+0000","last_unstale":"2024-05-30T16:44:46.948751+0000","last_undegraded":"2024-05-30T16:44:46.948751+0000","last_fullsized":"2024-05-30T16:44:46.948751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:13.039069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913062+0000","last_change":"2024-05-30T16:44:45.926086+0000","last_active":"2024-05-30T16:44:46.913062+0000","last_peered":"2024-05-30T16:44:46.913062+0000","last_clean":"2024-05-30T16:44:46.913062+0000","last_became_active":"2024-05-30T16:44:45.921839+0000","last_became_peered":"2024-05-30T16:44:45.921839+0000","last_unstale":"2024-05-30T16:44:46.913062+0000","last_undegraded":"2024-05-30T16:44:46.913062+0000","last_fullsized":"2024-05-30T16:44:46.913062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:19.657204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948177+0000","last_change":"2024-05-30T16:44:45.908086+0000","last_active":"2024-05-30T16:44:46.948177+0000","last_peered":"2024-05-30T16:44:46.948177+0000","last_clean":"2024-05-30T16:44:46.948177+0000","last_became_active":"2024-05-30T16:44:45.907793+0000","last_became_peered":"2024-05-30T16:44:45.907793+0000","last_unstale":"2024-05-30T16:44:46.948177+0000","last_undegraded":"2024-05-30T16:44:46.948177+0000","last_fullsized":"2024-05-30T16:44:46.948177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:17.249731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912803+0000","last_change":"2024-05-30T16:44:45.931329+0000","last_active":"2024-05-30T16:44:46.912803+0000","last_peered":"2024-05-30T16:44:46.912803+0000","last_clean":"2024-05-30T16:44:46.912803+0000","last_became_active":"2024-05-30T16:44:45.929807+0000","last_became_peered":"2024-05-30T16:44:45.929807+0000","last_unstale":"2024-05-30T16:44:46.912803+0000","last_undegraded":"2024-05-30T16:44:46.912803+0000","last_fullsized":"2024-05-30T16:44:46.912803+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933144+0000","last_change":"2024-05-30T16:44:45.918314+0000","last_active":"2024-05-30T16:44:46.933144+0000","last_peered":"2024-05-30T16:44:46.933144+0000","last_clean":"2024-05-30T16:44:46.933144+0000","last_became_active":"2024-05-30T16:44:45.917998+0000","last_became_peered":"2024-05-30T16:44:45.917998+0000","last_unstale":"2024-05-30T16:44:46.933144+0000","last_undegraded":"2024-05-30T16:44:46.933144+0000","last_fullsized":"2024-05-30T16:44:46.933144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:09.909703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.949118+0000","last_change":"2024-05-30T16:44:44.892355+0000","last_active":"2024-05-30T16:44:46.949118+0000","last_peered":"2024-05-30T16:44:46.949118+0000","last_clean":"2024-05-30T16:44:46.949118+0000","last_became_active":"2024-05-30T16:44:44.891889+0000","last_became_peered":"2024-05-30T16:44:44.891889+0000","last_unstale":"2024-05-30T16:44:46.949118+0000","last_undegraded":"2024-05-30T16:44:46.949118+0000","last_fullsized":"2024-05-30T16:44:46.949118+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:44.589105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.949261+0000","last_change":"2024-05-30T16:44:45.934290+0000","last_active":"2024-05-30T16:44:46.949261+0000","last_peered":"2024-05-30T16:44:46.949261+0000","last_clean":"2024-05-30T16:44:46.949261+0000","last_became_active":"2024-05-30T16:44:45.933465+0000","last_became_peered":"2024-05-30T16:44:45.933465+0000","last_unstale":"2024-05-30T16:44:46.949261+0000","last_undegraded":"2024-05-30T16:44:46.949261+0000","last_fullsized":"2024-05-30T16:44:46.949261+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:10.302887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932836+0000","last_change":"2024-05-30T16:44:44.889925+0000","last_active":"2024-05-30T16:44:46.932836+0000","last_peered":"2024-05-30T16:44:46.932836+0000","last_clean":"2024-05-30T16:44:46.932836+0000","last_became_active":"2024-05-30T16:44:44.889564+0000","last_became_peered":"2024-05-30T16:44:44.889564+0000","last_unstale":"2024-05-30T16:44:46.932836+0000","last_undegraded":"2024-05-30T16:44:46.932836+0000","last_fullsized":"2024-05-30T16:44:46.932836+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:45.960008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.949413+0000","last_change":"2024-05-30T16:44:45.931771+0000","last_active":"2024-05-30T16:44:46.949413+0000","last_peered":"2024-05-30T16:44:46.949413+0000","last_clean":"2024-05-30T16:44:46.949413+0000","last_became_active":"2024-05-30T16:44:45.931320+0000","last_became_peered":"2024-05-30T16:44:45.931320+0000","last_unstale":"2024-05-30T16:44:46.949413+0000","last_undegraded":"2024-05-30T16:44:46.949413+0000","last_fullsized":"2024-05-30T16:44:46.949413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:16.535318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912059+0000","last_change":"2024-05-30T16:44:44.883317+0000","last_active":"2024-05-30T16:44:46.912059+0000","last_peered":"2024-05-30T16:44:46.912059+0000","last_clean":"2024-05-30T16:44:46.912059+0000","last_became_active":"2024-05-30T16:44:44.882994+0000","last_became_peered":"2024-05-30T16:44:44.882994+0000","last_unstale":"2024-05-30T16:44:46.912059+0000","last_undegraded":"2024-05-30T16:44:46.912059+0000","last_fullsized":"2024-05-30T16:44:46.912059+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:29:54.140992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939544+0000","last_change":"2024-05-30T16:44:45.923316+0000","last_active":"2024-05-30T16:44:46.939544+0000","last_peered":"2024-05-30T16:44:46.939544+0000","last_clean":"2024-05-30T16:44:46.939544+0000","last_became_active":"2024-05-30T16:44:45.922728+0000","last_became_peered":"2024-05-30T16:44:45.922728+0000","last_unstale":"2024-05-30T16:44:46.939544+0000","last_undegraded":"2024-05-30T16:44:46.939544+0000","last_fullsized":"2024-05-30T16:44:46.939544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:42.475947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946479+0000","last_change":"2024-05-30T16:44:44.886536+0000","last_active":"2024-05-30T16:44:46.946479+0000","last_peered":"2024-05-30T16:44:46.946479+0000","last_clean":"2024-05-30T16:44:46.946479+0000","last_became_active":"2024-05-30T16:44:44.886261+0000","last_became_peered":"2024-05-30T16:44:44.886261+0000","last_unstale":"2024-05-30T16:44:46.946479+0000","last_undegraded":"2024-05-30T16:44:46.946479+0000","last_fullsized":"2024-05-30T16:44:46.946479+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:48:53.783765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913886+0000","last_change":"2024-05-30T16:44:45.936295+0000","last_active":"2024-05-30T16:44:46.913886+0000","last_peered":"2024-05-30T16:44:46.913886+0000","last_clean":"2024-05-30T16:44:46.913886+0000","last_became_active":"2024-05-30T16:44:45.935416+0000","last_became_peered":"2024-05-30T16:44:45.935416+0000","last_unstale":"2024-05-30T16:44:46.913886+0000","last_undegraded":"2024-05-30T16:44:46.913886+0000","last_fullsized":"2024-05-30T16:44:46.913886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947689+0000","last_change":"2024-05-30T16:44:44.886562+0000","last_active":"2024-05-30T16:44:46.947689+0000","last_peered":"2024-05-30T16:44:46.947689+0000","last_clean":"2024-05-30T16:44:46.947689+0000","last_became_active":"2024-05-30T16:44:44.886254+0000","last_became_peered":"2024-05-30T16:44:44.886254+0000","last_unstale":"2024-05-30T16:44:46.947689+0000","last_undegraded":"2024-05-30T16:44:46.947689+0000","last_fullsized":"2024-05-30T16:44:46.947689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:37.347555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913996+0000","last_change":"2024-05-30T16:44:45.905732+0000","last_active":"2024-05-30T16:44:46.913996+0000","last_peered":"2024-05-30T16:44:46.913996+0000","last_clean":"2024-05-30T16:44:46.913996+0000","last_became_active":"2024-05-30T16:44:45.905117+0000","last_became_peered":"2024-05-30T16:44:45.905117+0000","last_unstale":"2024-05-30T16:44:46.913996+0000","last_undegraded":"2024-05-30T16:44:46.913996+0000","last_fullsized":"2024-05-30T16:44:46.913996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:55:15.879718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947140+0000","last_change":"2024-05-30T16:44:44.879442+0000","last_active":"2024-05-30T16:44:46.947140+0000","last_peered":"2024-05-30T16:44:46.947140+0000","last_clean":"2024-05-30T16:44:46.947140+0000","last_became_active":"2024-05-30T16:44:44.877334+0000","last_became_peered":"2024-05-30T16:44:44.877334+0000","last_unstale":"2024-05-30T16:44:46.947140+0000","last_undegraded":"2024-05-30T16:44:46.947140+0000","last_fullsized":"2024-05-30T16:44:46.947140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:44.589105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946788+0000","last_change":"2024-05-30T16:44:45.916967+0000","last_active":"2024-05-30T16:44:46.946788+0000","last_peered":"2024-05-30T16:44:46.946788+0000","last_clean":"2024-05-30T16:44:46.946788+0000","last_became_active":"2024-05-30T16:44:45.916682+0000","last_became_peered":"2024-05-30T16:44:45.916682+0000","last_unstale":"2024-05-30T16:44:46.946788+0000","last_undegraded":"2024-05-30T16:44:46.946788+0000","last_fullsized":"2024-05-30T16:44:46.946788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:00.032212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911056+0000","last_change":"2024-05-30T16:44:44.876950+0000","last_active":"2024-05-30T16:44:46.911056+0000","last_peered":"2024-05-30T16:44:46.911056+0000","last_clean":"2024-05-30T16:44:46.911056+0000","last_became_active":"2024-05-30T16:44:44.876666+0000","last_became_peered":"2024-05-30T16:44:44.876666+0000","last_unstale":"2024-05-30T16:44:46.911056+0000","last_undegraded":"2024-05-30T16:44:46.911056+0000","last_fullsized":"2024-05-30T16:44:46.911056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:16.244929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938130+0000","last_change":"2024-05-30T16:44:45.930629+0000","last_active":"2024-05-30T16:44:46.938130+0000","last_peered":"2024-05-30T16:44:46.938130+0000","last_clean":"2024-05-30T16:44:46.938130+0000","last_became_active":"2024-05-30T16:44:45.921414+0000","last_became_peered":"2024-05-30T16:44:45.921414+0000","last_unstale":"2024-05-30T16:44:46.938130+0000","last_undegraded":"2024-05-30T16:44:46.938130+0000","last_fullsized":"2024-05-30T16:44:46.938130+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:44.819439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946576+0000","last_change":"2024-05-30T16:44:44.887604+0000","last_active":"2024-05-30T16:44:46.946576+0000","last_peered":"2024-05-30T16:44:46.946576+0000","last_clean":"2024-05-30T16:44:46.946576+0000","last_became_active":"2024-05-30T16:44:44.887259+0000","last_became_peered":"2024-05-30T16:44:44.887259+0000","last_unstale":"2024-05-30T16:44:46.946576+0000","last_undegraded":"2024-05-30T16:44:46.946576+0000","last_fullsized":"2024-05-30T16:44:46.946576+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:15:02.759428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915928+0000","last_change":"2024-05-30T16:44:45.918490+0000","last_active":"2024-05-30T16:44:46.915928+0000","last_peered":"2024-05-30T16:44:46.915928+0000","last_clean":"2024-05-30T16:44:46.915928+0000","last_became_active":"2024-05-30T16:44:45.917451+0000","last_became_peered":"2024-05-30T16:44:45.917451+0000","last_unstale":"2024-05-30T16:44:46.915928+0000","last_undegraded":"2024-05-30T16:44:46.915928+0000","last_fullsized":"2024-05-30T16:44:46.915928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:21.777810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912835+0000","last_change":"2024-05-30T16:44:44.889369+0000","last_active":"2024-05-30T16:44:46.912835+0000","last_peered":"2024-05-30T16:44:46.912835+0000","last_clean":"2024-05-30T16:44:46.912835+0000","last_became_active":"2024-05-30T16:44:44.889051+0000","last_became_peered":"2024-05-30T16:44:44.889051+0000","last_unstale":"2024-05-30T16:44:46.912835+0000","last_undegraded":"2024-05-30T16:44:46.912835+0000","last_fullsized":"2024-05-30T16:44:46.912835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:31:44.606560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912841+0000","last_change":"2024-05-30T16:44:45.924478+0000","last_active":"2024-05-30T16:44:46.912841+0000","last_peered":"2024-05-30T16:44:46.912841+0000","last_clean":"2024-05-30T16:44:46.912841+0000","last_became_active":"2024-05-30T16:44:45.922542+0000","last_became_peered":"2024-05-30T16:44:45.922542+0000","last_unstale":"2024-05-30T16:44:46.912841+0000","last_undegraded":"2024-05-30T16:44:46.912841+0000","last_fullsized":"2024-05-30T16:44:46.912841+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:51:40.612481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932945+0000","last_change":"2024-05-30T16:44:44.885815+0000","last_active":"2024-05-30T16:44:46.932945+0000","last_peered":"2024-05-30T16:44:46.932945+0000","last_clean":"2024-05-30T16:44:46.932945+0000","last_became_active":"2024-05-30T16:44:44.885007+0000","last_became_peered":"2024-05-30T16:44:44.885007+0000","last_unstale":"2024-05-30T16:44:46.932945+0000","last_undegraded":"2024-05-30T16:44:46.932945+0000","last_fullsized":"2024-05-30T16:44:46.932945+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:24:02.943052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912059+0000","last_change":"2024-05-30T16:44:45.921702+0000","last_active":"2024-05-30T16:44:46.912059+0000","last_peered":"2024-05-30T16:44:46.912059+0000","last_clean":"2024-05-30T16:44:46.912059+0000","last_became_active":"2024-05-30T16:44:45.921427+0000","last_became_peered":"2024-05-30T16:44:45.921427+0000","last_unstale":"2024-05-30T16:44:46.912059+0000","last_undegraded":"2024-05-30T16:44:46.912059+0000","last_fullsized":"2024-05-30T16:44:46.912059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:07.509629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915318+0000","last_change":"2024-05-30T16:44:44.891929+0000","last_active":"2024-05-30T16:44:46.915318+0000","last_peered":"2024-05-30T16:44:46.915318+0000","last_clean":"2024-05-30T16:44:46.915318+0000","last_became_active":"2024-05-30T16:44:44.891611+0000","last_became_peered":"2024-05-30T16:44:44.891611+0000","last_unstale":"2024-05-30T16:44:46.915318+0000","last_undegraded":"2024-05-30T16:44:46.915318+0000","last_fullsized":"2024-05-30T16:44:46.915318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:41.115203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914621+0000","last_change":"2024-05-30T16:44:45.915178+0000","last_active":"2024-05-30T16:44:46.914621+0000","last_peered":"2024-05-30T16:44:46.914621+0000","last_clean":"2024-05-30T16:44:46.914621+0000","last_became_active":"2024-05-30T16:44:45.914901+0000","last_became_peered":"2024-05-30T16:44:45.914901+0000","last_unstale":"2024-05-30T16:44:46.914621+0000","last_undegraded":"2024-05-30T16:44:46.914621+0000","last_fullsized":"2024-05-30T16:44:46.914621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:22:01.675145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912180+0000","last_change":"2024-05-30T16:44:44.884173+0000","last_active":"2024-05-30T16:44:46.912180+0000","last_peered":"2024-05-30T16:44:46.912180+0000","last_clean":"2024-05-30T16:44:46.912180+0000","last_became_active":"2024-05-30T16:44:44.883842+0000","last_became_peered":"2024-05-30T16:44:44.883842+0000","last_unstale":"2024-05-30T16:44:46.912180+0000","last_undegraded":"2024-05-30T16:44:46.912180+0000","last_fullsized":"2024-05-30T16:44:46.912180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:43.814638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938290+0000","last_change":"2024-05-30T16:44:45.922734+0000","last_active":"2024-05-30T16:44:46.938290+0000","last_peered":"2024-05-30T16:44:46.938290+0000","last_clean":"2024-05-30T16:44:46.938290+0000","last_became_active":"2024-05-30T16:44:45.921882+0000","last_became_peered":"2024-05-30T16:44:45.921882+0000","last_unstale":"2024-05-30T16:44:46.938290+0000","last_undegraded":"2024-05-30T16:44:46.938290+0000","last_fullsized":"2024-05-30T16:44:46.938290+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:13.039069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947991+0000","last_change":"2024-05-30T16:44:44.888087+0000","last_active":"2024-05-30T16:44:46.947991+0000","last_peered":"2024-05-30T16:44:46.947991+0000","last_clean":"2024-05-30T16:44:46.947991+0000","last_became_active":"2024-05-30T16:44:44.887807+0000","last_became_peered":"2024-05-30T16:44:44.887807+0000","last_unstale":"2024-05-30T16:44:46.947991+0000","last_undegraded":"2024-05-30T16:44:46.947991+0000","last_fullsized":"2024-05-30T16:44:46.947991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:02:28.009022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.912092+0000","last_change":"2024-05-30T16:44:45.915303+0000","last_active":"2024-05-30T16:44:46.912092+0000","last_peered":"2024-05-30T16:44:46.912092+0000","last_clean":"2024-05-30T16:44:46.912092+0000","last_became_active":"2024-05-30T16:44:45.915028+0000","last_became_peered":"2024-05-30T16:44:45.915028+0000","last_unstale":"2024-05-30T16:44:46.912092+0000","last_undegraded":"2024-05-30T16:44:46.912092+0000","last_fullsized":"2024-05-30T16:44:46.912092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:46.821070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938260+0000","last_change":"2024-05-30T16:44:44.891918+0000","last_active":"2024-05-30T16:44:46.938260+0000","last_peered":"2024-05-30T16:44:46.938260+0000","last_clean":"2024-05-30T16:44:46.938260+0000","last_became_active":"2024-05-30T16:44:44.891134+0000","last_became_peered":"2024-05-30T16:44:44.891134+0000","last_unstale":"2024-05-30T16:44:46.938260+0000","last_undegraded":"2024-05-30T16:44:46.938260+0000","last_fullsized":"2024-05-30T16:44:46.938260+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:02:28.009022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913160+0000","last_change":"2024-05-30T16:44:45.931105+0000","last_active":"2024-05-30T16:44:46.913160+0000","last_peered":"2024-05-30T16:44:46.913160+0000","last_clean":"2024-05-30T16:44:46.913160+0000","last_became_active":"2024-05-30T16:44:45.929921+0000","last_became_peered":"2024-05-30T16:44:45.929921+0000","last_unstale":"2024-05-30T16:44:46.913160+0000","last_undegraded":"2024-05-30T16:44:46.913160+0000","last_fullsized":"2024-05-30T16:44:46.913160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:24.930957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911264+0000","last_change":"2024-05-30T16:44:44.889587+0000","last_active":"2024-05-30T16:44:46.911264+0000","last_peered":"2024-05-30T16:44:46.911264+0000","last_clean":"2024-05-30T16:44:46.911264+0000","last_became_active":"2024-05-30T16:44:44.889226+0000","last_became_peered":"2024-05-30T16:44:44.889226+0000","last_unstale":"2024-05-30T16:44:46.911264+0000","last_undegraded":"2024-05-30T16:44:46.911264+0000","last_fullsized":"2024-05-30T16:44:46.911264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:59.521947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948591+0000","last_change":"2024-05-30T16:44:45.935844+0000","last_active":"2024-05-30T16:44:46.948591+0000","last_peered":"2024-05-30T16:44:46.948591+0000","last_clean":"2024-05-30T16:44:46.948591+0000","last_became_active":"2024-05-30T16:44:45.935320+0000","last_became_peered":"2024-05-30T16:44:45.935320+0000","last_unstale":"2024-05-30T16:44:46.948591+0000","last_undegraded":"2024-05-30T16:44:46.948591+0000","last_fullsized":"2024-05-30T16:44:46.948591+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:00.526748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.911046+0000","last_change":"2024-05-30T16:44:44.871442+0000","last_active":"2024-05-30T16:44:46.911046+0000","last_peered":"2024-05-30T16:44:46.911046+0000","last_clean":"2024-05-30T16:44:46.911046+0000","last_became_active":"2024-05-30T16:44:44.871162+0000","last_became_peered":"2024-05-30T16:44:44.871162+0000","last_unstale":"2024-05-30T16:44:46.911046+0000","last_undegraded":"2024-05-30T16:44:46.911046+0000","last_fullsized":"2024-05-30T16:44:46.911046+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:09.298085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947619+0000","last_change":"2024-05-30T16:44:45.921757+0000","last_active":"2024-05-30T16:44:46.947619+0000","last_peered":"2024-05-30T16:44:46.947619+0000","last_clean":"2024-05-30T16:44:46.947619+0000","last_became_active":"2024-05-30T16:44:45.921468+0000","last_became_peered":"2024-05-30T16:44:45.921468+0000","last_unstale":"2024-05-30T16:44:46.947619+0000","last_undegraded":"2024-05-30T16:44:46.947619+0000","last_fullsized":"2024-05-30T16:44:46.947619+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:29:55.145793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939525+0000","last_change":"2024-05-30T16:44:44.886316+0000","last_active":"2024-05-30T16:44:46.939525+0000","last_peered":"2024-05-30T16:44:46.939525+0000","last_clean":"2024-05-30T16:44:46.939525+0000","last_became_active":"2024-05-30T16:44:44.885953+0000","last_became_peered":"2024-05-30T16:44:44.885953+0000","last_unstale":"2024-05-30T16:44:46.939525+0000","last_undegraded":"2024-05-30T16:44:46.939525+0000","last_fullsized":"2024-05-30T16:44:46.939525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:34.292596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.915844+0000","last_change":"2024-05-30T16:44:45.910082+0000","last_active":"2024-05-30T16:44:46.915844+0000","last_peered":"2024-05-30T16:44:46.915844+0000","last_clean":"2024-05-30T16:44:46.915844+0000","last_became_active":"2024-05-30T16:44:45.909541+0000","last_became_peered":"2024-05-30T16:44:45.909541+0000","last_unstale":"2024-05-30T16:44:46.915844+0000","last_undegraded":"2024-05-30T16:44:46.915844+0000","last_fullsized":"2024-05-30T16:44:46.915844+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947925+0000","last_change":"2024-05-30T16:44:44.888958+0000","last_active":"2024-05-30T16:44:46.947925+0000","last_peered":"2024-05-30T16:44:46.947925+0000","last_clean":"2024-05-30T16:44:46.947925+0000","last_became_active":"2024-05-30T16:44:44.888629+0000","last_became_peered":"2024-05-30T16:44:44.888629+0000","last_unstale":"2024-05-30T16:44:46.947925+0000","last_undegraded":"2024-05-30T16:44:46.947925+0000","last_fullsized":"2024-05-30T16:44:46.947925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:16.244929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939191+0000","last_change":"2024-05-30T16:44:45.911071+0000","last_active":"2024-05-30T16:44:46.939191+0000","last_peered":"2024-05-30T16:44:46.939191+0000","last_clean":"2024-05-30T16:44:46.939191+0000","last_became_active":"2024-05-30T16:44:45.910768+0000","last_became_peered":"2024-05-30T16:44:45.910768+0000","last_unstale":"2024-05-30T16:44:46.939191+0000","last_undegraded":"2024-05-30T16:44:46.939191+0000","last_fullsized":"2024-05-30T16:44:46.939191+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:53:54.936011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948560+0000","last_change":"2024-05-30T16:44:44.887204+0000","last_active":"2024-05-30T16:44:46.948560+0000","last_peered":"2024-05-30T16:44:46.948560+0000","last_clean":"2024-05-30T16:44:46.948560+0000","last_became_active":"2024-05-30T16:44:44.886702+0000","last_became_peered":"2024-05-30T16:44:44.886702+0000","last_unstale":"2024-05-30T16:44:46.948560+0000","last_undegraded":"2024-05-30T16:44:46.948560+0000","last_fullsized":"2024-05-30T16:44:46.948560+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:33.985709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946548+0000","last_change":"2024-05-30T16:44:45.916951+0000","last_active":"2024-05-30T16:44:46.946548+0000","last_peered":"2024-05-30T16:44:46.946548+0000","last_clean":"2024-05-30T16:44:46.946548+0000","last_became_active":"2024-05-30T16:44:45.916566+0000","last_became_peered":"2024-05-30T16:44:45.916566+0000","last_unstale":"2024-05-30T16:44:46.946548+0000","last_undegraded":"2024-05-30T16:44:46.946548+0000","last_fullsized":"2024-05-30T16:44:46.946548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:34.990511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.949462+0000","last_change":"2024-05-30T16:44:44.874074+0000","last_active":"2024-05-30T16:44:46.949462+0000","last_peered":"2024-05-30T16:44:46.949462+0000","last_clean":"2024-05-30T16:44:46.949462+0000","last_became_active":"2024-05-30T16:44:44.872396+0000","last_became_peered":"2024-05-30T16:44:44.872396+0000","last_unstale":"2024-05-30T16:44:46.949462+0000","last_undegraded":"2024-05-30T16:44:46.949462+0000","last_fullsized":"2024-05-30T16:44:46.949462+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:52.965427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946316+0000","last_change":"2024-05-30T16:44:45.922182+0000","last_active":"2024-05-30T16:44:46.946316+0000","last_peered":"2024-05-30T16:44:46.946316+0000","last_clean":"2024-05-30T16:44:46.946316+0000","last_became_active":"2024-05-30T16:44:45.921770+0000","last_became_peered":"2024-05-30T16:44:45.921770+0000","last_unstale":"2024-05-30T16:44:46.946316+0000","last_undegraded":"2024-05-30T16:44:46.946316+0000","last_fullsized":"2024-05-30T16:44:46.946316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:02:29.013824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932742+0000","last_change":"2024-05-30T16:44:44.881588+0000","last_active":"2024-05-30T16:44:46.932742+0000","last_peered":"2024-05-30T16:44:46.932742+0000","last_clean":"2024-05-30T16:44:46.932742+0000","last_became_active":"2024-05-30T16:44:44.881282+0000","last_became_peered":"2024-05-30T16:44:44.881282+0000","last_unstale":"2024-05-30T16:44:46.932742+0000","last_undegraded":"2024-05-30T16:44:46.932742+0000","last_fullsized":"2024-05-30T16:44:46.932742+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:27:30.795217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946324+0000","last_change":"2024-05-30T16:44:45.910190+0000","last_active":"2024-05-30T16:44:46.946324+0000","last_peered":"2024-05-30T16:44:46.946324+0000","last_clean":"2024-05-30T16:44:46.946324+0000","last_became_active":"2024-05-30T16:44:45.909349+0000","last_became_peered":"2024-05-30T16:44:45.909349+0000","last_unstale":"2024-05-30T16:44:46.946324+0000","last_undegraded":"2024-05-30T16:44:46.946324+0000","last_fullsized":"2024-05-30T16:44:46.946324+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:35.297397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947768+0000","last_change":"2024-05-30T16:44:44.890541+0000","last_active":"2024-05-30T16:44:46.947768+0000","last_peered":"2024-05-30T16:44:46.947768+0000","last_clean":"2024-05-30T16:44:46.947768+0000","last_became_active":"2024-05-30T16:44:44.890280+0000","last_became_peered":"2024-05-30T16:44:44.890280+0000","last_unstale":"2024-05-30T16:44:46.947768+0000","last_undegraded":"2024-05-30T16:44:46.947768+0000","last_fullsized":"2024-05-30T16:44:46.947768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:26:14.335974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.945922+0000","last_change":"2024-05-30T16:44:45.909453+0000","last_active":"2024-05-30T16:44:46.945922+0000","last_peered":"2024-05-30T16:44:46.945922+0000","last_clean":"2024-05-30T16:44:46.945922+0000","last_became_active":"2024-05-30T16:44:45.909136+0000","last_became_peered":"2024-05-30T16:44:45.909136+0000","last_unstale":"2024-05-30T16:44:46.945922+0000","last_undegraded":"2024-05-30T16:44:46.945922+0000","last_fullsized":"2024-05-30T16:44:46.945922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:38.352356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946514+0000","last_change":"2024-05-30T16:44:44.885235+0000","last_active":"2024-05-30T16:44:46.946514+0000","last_peered":"2024-05-30T16:44:46.946514+0000","last_clean":"2024-05-30T16:44:46.946514+0000","last_became_active":"2024-05-30T16:44:44.884907+0000","last_became_peered":"2024-05-30T16:44:44.884907+0000","last_unstale":"2024-05-30T16:44:46.946514+0000","last_undegraded":"2024-05-30T16:44:46.946514+0000","last_fullsized":"2024-05-30T16:44:46.946514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:22:37.611751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913450+0000","last_change":"2024-05-30T16:44:45.919445+0000","last_active":"2024-05-30T16:44:46.913450+0000","last_peered":"2024-05-30T16:44:46.913450+0000","last_clean":"2024-05-30T16:44:46.913450+0000","last_became_active":"2024-05-30T16:44:45.919149+0000","last_became_peered":"2024-05-30T16:44:45.919149+0000","last_unstale":"2024-05-30T16:44:46.913450+0000","last_undegraded":"2024-05-30T16:44:46.913450+0000","last_fullsized":"2024-05-30T16:44:46.913450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:04:48.630940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913128+0000","last_change":"2024-05-30T16:44:44.887506+0000","last_active":"2024-05-30T16:44:46.913128+0000","last_peered":"2024-05-30T16:44:46.913128+0000","last_clean":"2024-05-30T16:44:46.913128+0000","last_became_active":"2024-05-30T16:44:44.887180+0000","last_became_peered":"2024-05-30T16:44:44.887180+0000","last_unstale":"2024-05-30T16:44:46.913128+0000","last_undegraded":"2024-05-30T16:44:46.913128+0000","last_fullsized":"2024-05-30T16:44:46.913128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:08.904901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946568+0000","last_change":"2024-05-30T16:44:45.917932+0000","last_active":"2024-05-30T16:44:46.946568+0000","last_peered":"2024-05-30T16:44:46.946568+0000","last_clean":"2024-05-30T16:44:46.946568+0000","last_became_active":"2024-05-30T16:44:45.917604+0000","last_became_peered":"2024-05-30T16:44:45.917604+0000","last_unstale":"2024-05-30T16:44:46.946568+0000","last_undegraded":"2024-05-30T16:44:46.946568+0000","last_fullsized":"2024-05-30T16:44:46.946568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:26.429331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939719+0000","last_change":"2024-05-30T16:44:44.891749+0000","last_active":"2024-05-30T16:44:46.939719+0000","last_peered":"2024-05-30T16:44:46.939719+0000","last_clean":"2024-05-30T16:44:46.939719+0000","last_became_active":"2024-05-30T16:44:44.891440+0000","last_became_peered":"2024-05-30T16:44:44.891440+0000","last_unstale":"2024-05-30T16:44:46.939719+0000","last_undegraded":"2024-05-30T16:44:46.939719+0000","last_fullsized":"2024-05-30T16:44:46.939719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:25.424530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.916134+0000","last_change":"2024-05-30T16:44:45.911902+0000","last_active":"2024-05-30T16:44:46.916134+0000","last_peered":"2024-05-30T16:44:46.916134+0000","last_clean":"2024-05-30T16:44:46.916134+0000","last_became_active":"2024-05-30T16:44:45.911282+0000","last_became_peered":"2024-05-30T16:44:45.911282+0000","last_unstale":"2024-05-30T16:44:46.916134+0000","last_undegraded":"2024-05-30T16:44:46.916134+0000","last_fullsized":"2024-05-30T16:44:46.916134+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933055+0000","last_change":"2024-05-30T16:44:44.885333+0000","last_active":"2024-05-30T16:44:46.933055+0000","last_peered":"2024-05-30T16:44:46.933055+0000","last_clean":"2024-05-30T16:44:46.933055+0000","last_became_active":"2024-05-30T16:44:44.883685+0000","last_became_peered":"2024-05-30T16:44:44.883685+0000","last_unstale":"2024-05-30T16:44:46.933055+0000","last_undegraded":"2024-05-30T16:44:46.933055+0000","last_fullsized":"2024-05-30T16:44:46.933055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:52.935243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932751+0000","last_change":"2024-05-30T16:44:45.915538+0000","last_active":"2024-05-30T16:44:46.932751+0000","last_peered":"2024-05-30T16:44:46.932751+0000","last_clean":"2024-05-30T16:44:46.932751+0000","last_became_active":"2024-05-30T16:44:45.914763+0000","last_became_peered":"2024-05-30T16:44:45.914763+0000","last_unstale":"2024-05-30T16:44:46.932751+0000","last_undegraded":"2024-05-30T16:44:46.932751+0000","last_fullsized":"2024-05-30T16:44:46.932751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:30.231921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.939112+0000","last_change":"2024-05-30T16:44:44.888789+0000","last_active":"2024-05-30T16:44:46.939112+0000","last_peered":"2024-05-30T16:44:46.939112+0000","last_clean":"2024-05-30T16:44:46.939112+0000","last_became_active":"2024-05-30T16:44:44.888193+0000","last_became_peered":"2024-05-30T16:44:44.888193+0000","last_unstale":"2024-05-30T16:44:46.939112+0000","last_undegraded":"2024-05-30T16:44:46.939112+0000","last_fullsized":"2024-05-30T16:44:46.939112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:52.965427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.947816+0000","last_change":"2024-05-30T16:44:45.918877+0000","last_active":"2024-05-30T16:44:46.947816+0000","last_peered":"2024-05-30T16:44:46.947816+0000","last_clean":"2024-05-30T16:44:46.947816+0000","last_became_active":"2024-05-30T16:44:45.915980+0000","last_became_peered":"2024-05-30T16:44:45.915980+0000","last_unstale":"2024-05-30T16:44:46.947816+0000","last_undegraded":"2024-05-30T16:44:46.947816+0000","last_fullsized":"2024-05-30T16:44:46.947816+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:50.906656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.933045+0000","last_change":"2024-05-30T16:44:44.892955+0000","last_active":"2024-05-30T16:44:46.933045+0000","last_peered":"2024-05-30T16:44:46.933045+0000","last_clean":"2024-05-30T16:44:46.933045+0000","last_became_active":"2024-05-30T16:44:44.892513+0000","last_became_peered":"2024-05-30T16:44:44.892513+0000","last_unstale":"2024-05-30T16:44:46.933045+0000","last_undegraded":"2024-05-30T16:44:46.933045+0000","last_fullsized":"2024-05-30T16:44:46.933045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:04:30.373290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.932947+0000","last_change":"2024-05-30T16:44:45.910492+0000","last_active":"2024-05-30T16:44:46.932947+0000","last_peered":"2024-05-30T16:44:46.932947+0000","last_clean":"2024-05-30T16:44:46.932947+0000","last_became_active":"2024-05-30T16:44:45.909824+0000","last_became_peered":"2024-05-30T16:44:45.909824+0000","last_unstale":"2024-05-30T16:44:46.932947+0000","last_undegraded":"2024-05-30T16:44:46.932947+0000","last_fullsized":"2024-05-30T16:44:46.932947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:45.491068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.938449+0000","last_change":"2024-05-30T16:44:44.887201+0000","last_active":"2024-05-30T16:44:46.938449+0000","last_peered":"2024-05-30T16:44:46.938449+0000","last_clean":"2024-05-30T16:44:46.938449+0000","last_became_active":"2024-05-30T16:44:44.886748+0000","last_became_peered":"2024-05-30T16:44:44.886748+0000","last_unstale":"2024-05-30T16:44:46.938449+0000","last_undegraded":"2024-05-30T16:44:46.938449+0000","last_fullsized":"2024-05-30T16:44:46.938449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:23:44.589105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.910921+0000","last_change":"2024-05-30T16:44:45.929143+0000","last_active":"2024-05-30T16:44:46.910921+0000","last_peered":"2024-05-30T16:44:46.910921+0000","last_clean":"2024-05-30T16:44:46.910921+0000","last_became_active":"2024-05-30T16:44:45.928818+0000","last_became_peered":"2024-05-30T16:44:45.928818+0000","last_unstale":"2024-05-30T16:44:46.910921+0000","last_undegraded":"2024-05-30T16:44:46.910921+0000","last_fullsized":"2024-05-30T16:44:46.910921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:14:56.612071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946809+0000","last_change":"2024-05-30T16:44:44.882578+0000","last_active":"2024-05-30T16:44:46.946809+0000","last_peered":"2024-05-30T16:44:46.946809+0000","last_clean":"2024-05-30T16:44:46.946809+0000","last_became_active":"2024-05-30T16:44:44.882197+0000","last_became_peered":"2024-05-30T16:44:44.882197+0000","last_unstale":"2024-05-30T16:44:46.946809+0000","last_undegraded":"2024-05-30T16:44:46.946809+0000","last_fullsized":"2024-05-30T16:44:46.946809+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:43.725581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.946528+0000","last_change":"2024-05-30T16:44:45.928687+0000","last_active":"2024-05-30T16:44:46.946528+0000","last_peered":"2024-05-30T16:44:46.946528+0000","last_clean":"2024-05-30T16:44:46.946528+0000","last_became_active":"2024-05-30T16:44:45.928347+0000","last_became_peered":"2024-05-30T16:44:45.928347+0000","last_unstale":"2024-05-30T16:44:46.946528+0000","last_undegraded":"2024-05-30T16:44:46.946528+0000","last_fullsized":"2024-05-30T16:44:46.946528+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:00.526748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.910954+0000","last_change":"2024-05-30T16:44:44.888466+0000","last_active":"2024-05-30T16:44:46.910954+0000","last_peered":"2024-05-30T16:44:46.910954+0000","last_clean":"2024-05-30T16:44:46.910954+0000","last_became_active":"2024-05-30T16:44:44.888164+0000","last_became_peered":"2024-05-30T16:44:44.888164+0000","last_unstale":"2024-05-30T16:44:46.910954+0000","last_undegraded":"2024-05-30T16:44:46.910954+0000","last_fullsized":"2024-05-30T16:44:46.910954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:45:59.027410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.914547+0000","last_change":"2024-05-30T16:44:45.919696+0000","last_active":"2024-05-30T16:44:46.914547+0000","last_peered":"2024-05-30T16:44:46.914547+0000","last_clean":"2024-05-30T16:44:46.914547+0000","last_became_active":"2024-05-30T16:44:45.918248+0000","last_became_peered":"2024-05-30T16:44:45.918248+0000","last_unstale":"2024-05-30T16:44:46.914547+0000","last_undegraded":"2024-05-30T16:44:46.914547+0000","last_fullsized":"2024-05-30T16:44:46.914547+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:44.844223+0000","last_clean_scrub_stamp":"2024-05-30T16:44:44.844223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:44.034568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.948615+0000","last_change":"2024-05-30T16:44:44.893954+0000","last_active":"2024-05-30T16:44:46.948615+0000","last_peered":"2024-05-30T16:44:46.948615+0000","last_clean":"2024-05-30T16:44:46.948615+0000","last_became_active":"2024-05-30T16:44:44.893629+0000","last_became_peered":"2024-05-30T16:44:44.893629+0000","last_unstale":"2024-05-30T16:44:46.948615+0000","last_undegraded":"2024-05-30T16:44:46.948615+0000","last_fullsized":"2024-05-30T16:44:46.948615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:44:43.839422+0000","last_clean_scrub_stamp":"2024-05-30T16:44:43.839422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:52.965427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-30T16:44:46.913336+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:44:46.913336+0000","last_peered":"2024-05-30T16:44:46.913336+0000","last_clean":"2024-05-30T16:44:46.913336+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:44:46.913336+0000","last_undegraded":"2024-05-30T16:44:46.913336+0000","last_fullsized":"2024-05-30T16:44:46.913336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":153502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":152990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":154520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":152956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":154520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":743344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":153558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":743254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:44:55.942 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T16:44:55.943 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-30T16:44:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.201+0000 7f10ece00640 1 Processor -- start 2024-05-30T16:44:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.201+0000 7f10ece00640 1 -- start start 2024-05-30T16:44:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.202+0000 7f10ece00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10e815d5d0 0x7f10e80cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.202+0000 7f10ece00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e815cb10 con 0x7f10e80b32c0 2024-05-30T16:44:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.202+0000 7f10ece00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e80ad5f0 con 0x7f10e80d0640 2024-05-30T16:44:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.202+0000 7f10ece00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f10e80adbb0 con 0x7f10e815d5d0 2024-05-30T16:44:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.203+0000 7f10e6800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f10e80b32c0 0x7f10e80b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54854/0 (socket says 172.21.15.67:54854) 2024-05-30T16:44:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.203+0000 7f10e6800640 1 -- 172.21.15.67:0/1975480538 learned_addr learned my addr 172.21.15.67:0/1975480538 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.203+0000 7f10e5400640 1 --2- 172.21.15.67:0/1975480538 >> v2:172.21.15.169:3300/0 conn(0x7f10e815d5d0 0x7f10e80cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3334616463 0 0) 0x7f10e815cb10 con 0x7f10e80b32c0 2024-05-30T16:44:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10c8003520 con 0x7f10e80b32c0 2024-05-30T16:44:56.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2496577309 0 0) 0x7f10c8003520 con 0x7f10e80b32c0 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 >> v2:172.21.15.169:3300/0 conn(0x7f10e815d5d0 msgr2=0x7f10e80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 --2- 172.21.15.67:0/1975480538 >> v2:172.21.15.169:3300/0 conn(0x7f10e815d5d0 0x7f10e80cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 >> v1:172.21.15.169:6789/0 conn(0x7f10e80d0640 legacy=0x7f10e80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e80ad8d0 con 0x7f10e80b32c0 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f10e0001670 con 0x7f10e80b32c0 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e5400640 1 --2- 172.21.15.67:0/1975480538 >> v2:172.21.15.169:3300/0 conn(0x7f10e815d5d0 0x7f10e80cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 >> v1:172.21.15.67:6789/0 conn(0x7f10e80b32c0 legacy=0x7f10e80b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c80039b0 0x7f10c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c8004800 0x7f10c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f10c800c130 con 0x7f10c80051c0 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10c800c2e0 con 0x7f10c8004800 2024-05-30T16:44:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.204+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10c800c4e0 con 0x7f10c80039b0 2024-05-30T16:44:56.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.205+0000 7f10e6800640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.205+0000 7f10e5400640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c8004800 0x7f10c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.205+0000 7f10e5e00640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c80039b0 0x7f10c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.205+0000 7f10e6800640 1 -- 172.21.15.67:0/1975480538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c80039b0 msgr2=0x7f10c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.205+0000 7f10e6800640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c80039b0 0x7f10c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.205+0000 7f10e6800640 1 -- 172.21.15.67:0/1975480538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c8004800 msgr2=0x7f10c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.205+0000 7f10e6800640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c8004800 0x7f10c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.205+0000 7f10e6800640 1 -- 172.21.15.67:0/1975480538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e80ad8d0 con 0x7f10c80051c0 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.205+0000 7f10e5e00640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c80039b0 0x7f10c8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.207+0000 7f10e6800640 1 -- 172.21.15.67:0/1975480538 reap_dead start 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.207+0000 7f10e6800640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10c800b9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f10e0001050 tx=0x7f10c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6089013c0374dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.207+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0007d10 con 0x7f10c80051c0 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.207+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e0005af0 con 0x7f10c80051c0 2024-05-30T16:44:56.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.207+0000 7f10e4a00640 1 -- 172.21.15.67:0/1975480538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0007070 con 0x7f10c80051c0 2024-05-30T16:44:56.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.208+0000 7f10ece00640 1 -- 172.21.15.67:0/1975480538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 msgr2=0x7f10c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.208+0000 7f10ece00640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10c800b9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f10e0001050 tx=0x7f10c8005830 comp rx=0 tx=0).stop 2024-05-30T16:44:56.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.208+0000 7f10ece00640 1 -- 172.21.15.67:0/1975480538 shutdown_connections 2024-05-30T16:44:56.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.208+0000 7f10ece00640 1 --2- 172.21.15.67:0/1975480538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10c800b9c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.208+0000 7f10ece00640 1 -- 172.21.15.67:0/1975480538 >> 172.21.15.67:0/1975480538 conn(0x7f10e806bd20 msgr2=0x7f10e806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:56.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.208+0000 7f10ece00640 1 -- 172.21.15.67:0/1975480538 shutdown_connections 2024-05-30T16:44:56.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.208+0000 7f10ece00640 1 -- 172.21.15.67:0/1975480538 wait complete. 2024-05-30T16:44:56.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.210+0000 7f10ece00640 1 Processor -- start 2024-05-30T16:44:56.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.210+0000 7f10ece00640 1 -- start start 2024-05-30T16:44:56.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.211+0000 7f10ece00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10e8148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.211+0000 7f10ece00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e8152ca0 0x7f10e80b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.211+0000 7f10ece00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e814bae0 0x7f10e81f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.211+0000 7f10ece00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f10e815da50 con 0x7f10c80051c0 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.211+0000 7f10ece00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10e815d0b0 con 0x7f10e8152ca0 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.211+0000 7f10ece00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10e80ad2a0 con 0x7f10e814bae0 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.211+0000 7f10e5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10e8148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.211+0000 7f10e5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10e8148f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46688/0 (socket says 172.21.15.67:46688) 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.211+0000 7f10e5e00640 1 -- 172.21.15.67:0/201539284 learned_addr learned my addr 172.21.15.67:0/201539284 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10e5400640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e8152ca0 0x7f10e80b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10e5e00640 1 -- 172.21.15.67:0/201539284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e814bae0 msgr2=0x7f10e81f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10e5e00640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e814bae0 0x7f10e81f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10e5e00640 1 -- 172.21.15.67:0/201539284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e8152ca0 msgr2=0x7f10e80b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10e5e00640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e8152ca0 0x7f10e80b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10e5e00640 1 -- 172.21.15.67:0/201539284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e814ea80 con 0x7f10c80051c0 2024-05-30T16:44:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10e5400640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e8152ca0 0x7f10e80b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10e5e00640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10e8148f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f10d800e460 tx=0x7f10d800e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10de000640 1 -- 172.21.15.67:0/201539284 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d80052e0 con 0x7f10c80051c0 2024-05-30T16:44:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10de000640 1 -- 172.21.15.67:0/201539284 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d8018070 con 0x7f10c80051c0 2024-05-30T16:44:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10de000640 1 -- 172.21.15.67:0/201539284 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8003830 con 0x7f10c80051c0 2024-05-30T16:44:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81494b0 con 0x7f10c80051c0 2024-05-30T16:44:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.212+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e814c2f0 con 0x7f10c80051c0 2024-05-30T16:44:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.214+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e80689e0 con 0x7f10c80051c0 2024-05-30T16:44:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.215+0000 7f10de000640 1 -- 172.21.15.67:0/201539284 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10d80039d0 con 0x7f10c80051c0 2024-05-30T16:44:56.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.216+0000 7f10de000640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c406c8f0 0x7f10c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.216+0000 7f10de000640 1 -- 172.21.15.67:0/201539284 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f10d808b770 con 0x7f10c80051c0 2024-05-30T16:44:56.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.216+0000 7f10e5400640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c406c8f0 0x7f10c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.217+0000 7f10e5400640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c406c8f0 0x7f10c406edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10e8147e50 tx=0x7f10d000b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:56.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.220+0000 7f10de000640 1 -- 172.21.15.67:0/201539284 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10d8059b90 con 0x7f10c80051c0 2024-05-30T16:44:56.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.496+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f10e8073070 con 0x7f10c80051c0 2024-05-30T16:44:56.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.501+0000 7f10de000640 1 -- 172.21.15.67:0/201539284 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f10d8059490 con 0x7f10c80051c0 2024-05-30T16:44:56.503 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-05-30T16:44:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T16:44:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-05-30T16:44:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-05-30T16:44:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-30T16:44:56.504 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-05-30T16:44:56.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c406c8f0 msgr2=0x7f10c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c406c8f0 0x7f10c406edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10e8147e50 tx=0x7f10d000b040 comp rx=0 tx=0).stop 2024-05-30T16:44:56.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 msgr2=0x7f10e8148f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10e8148f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f10d800e460 tx=0x7f10d800e920 comp rx=0 tx=0).stop 2024-05-30T16:44:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 shutdown_connections 2024-05-30T16:44:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f10c406c8f0 0x7f10c406edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10e814bae0 0x7f10e81f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10e8152ca0 0x7f10e80b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 --2- 172.21.15.67:0/201539284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10c80051c0 0x7f10e8148f70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 >> 172.21.15.67:0/201539284 conn(0x7f10e806bd20 msgr2=0x7f10e8151e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.504+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 shutdown_connections 2024-05-30T16:44:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.505+0000 7f10ece00640 1 -- 172.21.15.67:0/201539284 wait complete. 2024-05-30T16:44:56.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:44:56.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.724+0000 7f2a89400640 1 Processor -- start 2024-05-30T16:44:56.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.724+0000 7f2a89400640 1 -- start start 2024-05-30T16:44:56.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.726+0000 7f2a89400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2a840d04a0 0x7f2a840d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.726+0000 7f2a89400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2a840ad130 0x7f2a840ad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.726+0000 7f2a89400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a8406ed10 con 0x7f2a8406f860 2024-05-30T16:44:56.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.726+0000 7f2a89400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2a8415cb00 con 0x7f2a840ad130 2024-05-30T16:44:56.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.726+0000 7f2a89400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2a840b3cc0 con 0x7f2a840d04a0 2024-05-30T16:44:56.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.726+0000 7f2a82800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2a8406f860 0x7f2a840b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54866/0 (socket says 172.21.15.67:54866) 2024-05-30T16:44:56.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.726+0000 7f2a82800640 1 -- 172.21.15.67:0/3526253017 learned_addr learned my addr 172.21.15.67:0/3526253017 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:56.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.726+0000 7f2a83200640 1 --2- 172.21.15.67:0/3526253017 >> v2:172.21.15.67:3300/0 conn(0x7f2a840ad130 0x7f2a840ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.727+0000 7f2a81e00640 1 --2- 172.21.15.67:0/3526253017 >> v2:172.21.15.169:3300/0 conn(0x7f2a840d04a0 0x7f2a840d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.727+0000 7f2a81400640 1 -- 172.21.15.67:0/3526253017 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 618757954 0 0) 0x7f2a74001110 con 0x7f2a8406f860 2024-05-30T16:44:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.727+0000 7f2a81400640 1 -- 172.21.15.67:0/3526253017 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a68003520 con 0x7f2a8406f860 2024-05-30T16:44:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.727+0000 7f2a83200640 1 -- 172.21.15.67:0/3526253017 >> v2:172.21.15.169:3300/0 conn(0x7f2a840d04a0 msgr2=0x7f2a840d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.727+0000 7f2a83200640 1 --2- 172.21.15.67:0/3526253017 >> v2:172.21.15.169:3300/0 conn(0x7f2a840d04a0 0x7f2a840d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.727+0000 7f2a83200640 1 -- 172.21.15.67:0/3526253017 >> v1:172.21.15.67:6789/0 conn(0x7f2a8406f860 legacy=0x7f2a840b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.727+0000 7f2a83200640 1 -- 172.21.15.67:0/3526253017 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a840b3fa0 con 0x7f2a840ad130 2024-05-30T16:44:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.728+0000 7f2a83200640 1 --2- 172.21.15.67:0/3526253017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 0x7f2a840ad540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2a84070120 tx=0x7f2a7c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb612c1f63b6651 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.728+0000 7f2a81400640 1 -- 172.21.15.67:0/3526253017 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c007740 con 0x7f2a840ad130 2024-05-30T16:44:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.728+0000 7f2a81400640 1 -- 172.21.15.67:0/3526253017 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a7c0078e0 con 0x7f2a840ad130 2024-05-30T16:44:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.728+0000 7f2a81400640 1 -- 172.21.15.67:0/3526253017 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c007c90 con 0x7f2a840ad130 2024-05-30T16:44:56.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.729+0000 7f2a89400640 1 -- 172.21.15.67:0/3526253017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 msgr2=0x7f2a840ad540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.729+0000 7f2a89400640 1 --2- 172.21.15.67:0/3526253017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 0x7f2a840ad540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2a84070120 tx=0x7f2a7c02a680 comp rx=0 tx=0).stop 2024-05-30T16:44:56.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.729+0000 7f2a89400640 1 -- 172.21.15.67:0/3526253017 shutdown_connections 2024-05-30T16:44:56.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.729+0000 7f2a89400640 1 --2- 172.21.15.67:0/3526253017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 0x7f2a840ad540 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.729+0000 7f2a89400640 1 --2- 172.21.15.67:0/3526253017 >> v2:172.21.15.169:3300/0 conn(0x7f2a840d04a0 0x7f2a840d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.729+0000 7f2a89400640 1 -- 172.21.15.67:0/3526253017 >> 172.21.15.67:0/3526253017 conn(0x7f2a8406bae0 msgr2=0x7f2a8406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.729+0000 7f2a89400640 1 -- 172.21.15.67:0/3526253017 shutdown_connections 2024-05-30T16:44:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.729+0000 7f2a89400640 1 -- 172.21.15.67:0/3526253017 wait complete. 2024-05-30T16:44:56.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.731+0000 7f2a89400640 1 Processor -- start 2024-05-30T16:44:56.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.731+0000 7f2a89400640 1 -- start start 2024-05-30T16:44:56.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.732+0000 7f2a89400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a8406f860 0x7f2a8415c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.732+0000 7f2a89400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 0x7f2a840c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.732+0000 7f2a89400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a840d04a0 0x7f2a840c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.732+0000 7f2a89400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2a840b4840 con 0x7f2a840ad130 2024-05-30T16:44:56.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.732+0000 7f2a89400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2a840b4560 con 0x7f2a840d04a0 2024-05-30T16:44:56.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.732+0000 7f2a89400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a840b3860 con 0x7f2a8406f860 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a81e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 0x7f2a840c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a81e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 0x7f2a840c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46704/0 (socket says 172.21.15.67:46704) 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a81e00640 1 -- 172.21.15.67:0/327164928 learned_addr learned my addr 172.21.15.67:0/327164928 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a82800640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a8406f860 0x7f2a8415c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a83200640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a840d04a0 0x7f2a840c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a81e00640 1 -- 172.21.15.67:0/327164928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a8406f860 msgr2=0x7f2a8415c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a81e00640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a8406f860 0x7f2a8415c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a81e00640 1 -- 172.21.15.67:0/327164928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a840d04a0 msgr2=0x7f2a840c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a81e00640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a840d04a0 0x7f2a840c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a81e00640 1 -- 172.21.15.67:0/327164928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8415f340 con 0x7f2a840ad130 2024-05-30T16:44:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.733+0000 7f2a83200640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a840d04a0 0x7f2a840c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.734+0000 7f2a81e00640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 0x7f2a840c6030 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00d620 tx=0x7f2a6c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.734+0000 7f2a7aa00640 1 -- 172.21.15.67:0/327164928 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c00b620 con 0x7f2a840ad130 2024-05-30T16:44:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.734+0000 7f2a7aa00640 1 -- 172.21.15.67:0/327164928 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c019070 con 0x7f2a840ad130 2024-05-30T16:44:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.734+0000 7f2a7aa00640 1 -- 172.21.15.67:0/327164928 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c00ba00 con 0x7f2a840ad130 2024-05-30T16:44:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.734+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a840c85d0 con 0x7f2a840ad130 2024-05-30T16:44:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.734+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a840c8b90 con 0x7f2a840ad130 2024-05-30T16:44:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.735+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a840689e0 con 0x7f2a840ad130 2024-05-30T16:44:56.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.737+0000 7f2a7aa00640 1 -- 172.21.15.67:0/327164928 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a6c00bba0 con 0x7f2a840ad130 2024-05-30T16:44:56.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.738+0000 7f2a7aa00640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a6006c8f0 0x7f2a6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:56.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.739+0000 7f2a7aa00640 1 -- 172.21.15.67:0/327164928 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2a6c08bec0 con 0x7f2a840ad130 2024-05-30T16:44:56.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.739+0000 7f2a82800640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a6006c8f0 0x7f2a6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:56.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.739+0000 7f2a82800640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a6006c8f0 0x7f2a6006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a74005880 tx=0x7f2a74009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:56.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.744+0000 7f2a7aa00640 1 -- 172.21.15.67:0/327164928 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a6c05a380 con 0x7f2a840ad130 2024-05-30T16:44:56.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.980+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a8406e2f0 con 0x7f2a840ad130 2024-05-30T16:44:56.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.981+0000 7f2a7aa00640 1 -- 172.21.15.67:0/327164928 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5181 (secure 0 0 0) 0x7f2a6c059c80 con 0x7f2a840ad130 2024-05-30T16:44:56.983 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:44:56.983 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/475904074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":475904074},{"type":"v1","addr":"172.21.15.169:6834","nonce":475904074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4812,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3464775083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3464775083},{"type":"v1","addr":"172.21.15.67:6835","nonce":3464775083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/650843675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":650843675},{"type":"v1","addr":"172.21.15.67:6837","nonce":650843675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:44:45.338997+0000","modified":"2024-05-30T16:44:48.433470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4821":{"gid":4821,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/2481507396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2481507396},{"type":"v1","addr":"172.21.15.169:6835","nonce":2481507396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4821,"qdb_cluster":[4821]},"id":2}]} 2024-05-30T16:44:56.983 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 16 2024-05-30T16:44:56.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a6006c8f0 msgr2=0x7f2a6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a6006c8f0 0x7f2a6006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a74005880 tx=0x7f2a74009040 comp rx=0 tx=0).stop 2024-05-30T16:44:56.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 msgr2=0x7f2a840c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:56.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 0x7f2a840c6030 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00d620 tx=0x7f2a6c00dae0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 shutdown_connections 2024-05-30T16:44:56.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a6006c8f0 0x7f2a6006edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a840d04a0 0x7f2a840c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a840ad130 0x7f2a840c6030 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 --2- 172.21.15.67:0/327164928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a8406f860 0x7f2a8415c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:56.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.983+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 >> 172.21.15.67:0/327164928 conn(0x7f2a8406bae0 msgr2=0x7f2a84073960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:56.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.984+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 shutdown_connections 2024-05-30T16:44:56.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:56.984+0000 7f2a89400640 1 -- 172.21.15.67:0/327164928 wait complete. 2024-05-30T16:44:57.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-30T16:44:45.338997+0000', 'modified': '2024-05-30T16:44:48.433470+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4821}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4821': {'gid': 4821, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.169:6835/2481507396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6833', 'nonce': 2481507396}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2481507396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4821, 'qdb_cluster': [4821]} 2024-05-30T16:44:57.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-30T16:44:57.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-30T16:44:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.623+0000 7f0581400640 1 Processor -- start 2024-05-30T16:44:57.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.624+0000 7f0581400640 1 -- start start 2024-05-30T16:44:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.625+0000 7f0581400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f057c06f860 0x7f057c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.625+0000 7f0581400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f057c0d0bb0 0x7f057c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.625+0000 7f0581400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057c06ed10 con 0x7f057c0b3690 2024-05-30T16:44:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.625+0000 7f0581400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f057c15cb00 con 0x7f057c0d0bb0 2024-05-30T16:44:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.625+0000 7f0581400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f057c0b3cc0 con 0x7f057c06f860 2024-05-30T16:44:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.626+0000 7f057b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f057c0d0bb0 0x7f057c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.626+0000 7f0579e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f057c0b3690 0x7f057c0d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54868/0 (socket says 172.21.15.67:54868) 2024-05-30T16:44:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.626+0000 7f0579e00640 1 -- 172.21.15.67:0/2731843821 learned_addr learned my addr 172.21.15.67:0/2731843821 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.626+0000 7f057a800640 1 --2- 172.21.15.67:0/2731843821 >> v2:172.21.15.169:3300/0 conn(0x7f057c06f860 0x7f057c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.626+0000 7f057b200640 1 -- 172.21.15.67:0/2731843821 >> v2:172.21.15.169:3300/0 conn(0x7f057c06f860 msgr2=0x7f057c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.626+0000 7f057b200640 1 --2- 172.21.15.67:0/2731843821 >> v2:172.21.15.169:3300/0 conn(0x7f057c06f860 0x7f057c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.626+0000 7f0579400640 1 -- 172.21.15.67:0/2731843821 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 906492760 0 0) 0x7f0568001110 con 0x7f057c0b3690 2024-05-30T16:44:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.626+0000 7f057b200640 1 -- 172.21.15.67:0/2731843821 >> v1:172.21.15.67:6789/0 conn(0x7f057c0b3690 legacy=0x7f057c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.626+0000 7f057b200640 1 -- 172.21.15.67:0/2731843821 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c0b3fa0 con 0x7f057c0d0bb0 2024-05-30T16:44:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.627+0000 7f057b200640 1 --2- 172.21.15.67:0/2731843821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0d0bb0 0x7f057c0ad130 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f057c06c8d0 tx=0x7f057402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74049297e057e730 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.627+0000 7f0579400640 1 -- 172.21.15.67:0/2731843821 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574037070 con 0x7f057c0d0bb0 2024-05-30T16:44:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.627+0000 7f0579400640 1 -- 172.21.15.67:0/2731843821 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0574032440 con 0x7f057c0d0bb0 2024-05-30T16:44:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.627+0000 7f0579400640 1 -- 172.21.15.67:0/2731843821 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574032810 con 0x7f057c0d0bb0 2024-05-30T16:44:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.628+0000 7f0581400640 1 -- 172.21.15.67:0/2731843821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0d0bb0 msgr2=0x7f057c0ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.628+0000 7f0581400640 1 --2- 172.21.15.67:0/2731843821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0d0bb0 0x7f057c0ad130 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f057c06c8d0 tx=0x7f057402a1a0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.628+0000 7f0581400640 1 -- 172.21.15.67:0/2731843821 shutdown_connections 2024-05-30T16:44:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.628+0000 7f0581400640 1 --2- 172.21.15.67:0/2731843821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0d0bb0 0x7f057c0ad130 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.628+0000 7f0581400640 1 --2- 172.21.15.67:0/2731843821 >> v2:172.21.15.169:3300/0 conn(0x7f057c06f860 0x7f057c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.628+0000 7f0581400640 1 -- 172.21.15.67:0/2731843821 >> 172.21.15.67:0/2731843821 conn(0x7f057c06bae0 msgr2=0x7f057c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.628+0000 7f0581400640 1 -- 172.21.15.67:0/2731843821 shutdown_connections 2024-05-30T16:44:57.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.628+0000 7f0581400640 1 -- 172.21.15.67:0/2731843821 wait complete. 2024-05-30T16:44:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.630+0000 7f0581400640 1 Processor -- start 2024-05-30T16:44:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.630+0000 7f0581400640 1 -- start start 2024-05-30T16:44:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.631+0000 7f0581400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f057c06f860 0x7f057c099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.631+0000 7f0581400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0b3690 0x7f057c15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.631+0000 7f0581400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f057c0d0bb0 0x7f057c15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.631+0000 7f0581400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f057c0b4840 con 0x7f057c0b3690 2024-05-30T16:44:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.631+0000 7f0581400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f057c0b4560 con 0x7f057c06f860 2024-05-30T16:44:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.631+0000 7f0581400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f057c0ad670 con 0x7f057c0d0bb0 2024-05-30T16:44:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.631+0000 7f0579e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0b3690 0x7f057c15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.631+0000 7f0579e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0b3690 0x7f057c15f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46722/0 (socket says 172.21.15.67:46722) 2024-05-30T16:44:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.631+0000 7f0579e00640 1 -- 172.21.15.67:0/2093240725 learned_addr learned my addr 172.21.15.67:0/2093240725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0579e00640 1 -- 172.21.15.67:0/2093240725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f057c0d0bb0 msgr2=0x7f057c15dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0579e00640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f057c0d0bb0 0x7f057c15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0579e00640 1 -- 172.21.15.67:0/2093240725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f057c06f860 msgr2=0x7f057c099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0579e00640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f057c06f860 0x7f057c099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0579e00640 1 -- 172.21.15.67:0/2093240725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c099ec0 con 0x7f057c0b3690 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0579e00640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0b3690 0x7f057c15f5e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f056800c780 tx=0x7f056800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0572a00640 1 -- 172.21.15.67:0/2093240725 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568018070 con 0x7f057c0b3690 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0572a00640 1 -- 172.21.15.67:0/2093240725 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05680072d0 con 0x7f057c0b3690 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0572a00640 1 -- 172.21.15.67:0/2093240725 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05680047e0 con 0x7f057c0b3690 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.632+0000 7f0581400640 1 -- 172.21.15.67:0/2093240725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057c15fb20 con 0x7f057c0b3690 2024-05-30T16:44:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.633+0000 7f0581400640 1 -- 172.21.15.67:0/2093240725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057c15e820 con 0x7f057c0b3690 2024-05-30T16:44:57.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.635+0000 7f0572a00640 1 -- 172.21.15.67:0/2093240725 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f056801b070 con 0x7f057c0b3690 2024-05-30T16:44:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.636+0000 7f0572a00640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f055c06c8f0 0x7f055c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.636+0000 7f0572a00640 1 -- 172.21.15.67:0/2093240725 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f056808ae00 con 0x7f057c0b3690 2024-05-30T16:44:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.636+0000 7f057a800640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f055c06c8f0 0x7f055c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.636+0000 7f057a800640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f055c06c8f0 0x7f055c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f056c0070b0 tx=0x7f056c007040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.637+0000 7f0584224740 1 Processor -- start 2024-05-30T16:44:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.637+0000 7f0584224740 1 -- start start 2024-05-30T16:44:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f0584224740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea532a0 0x55f44ea56a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f0584224740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 0x55f44ea72ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f0584224740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea73840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f0584224740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f44e990b10 con 0x55f44ea733e0 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f0584224740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f44e9372b0 con 0x55f44ea56f60 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f0584224740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f44e98cad0 con 0x55f44ea532a0 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f057b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea73840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f057b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea73840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46734/0 (socket says 172.21.15.67:46734) 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f057b200640 1 -- 172.21.15.67:0/405243022 learned_addr learned my addr 172.21.15.67:0/405243022 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f057b200640 1 -- 172.21.15.67:0/405243022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea532a0 msgr2=0x55f44ea56a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f057b200640 1 --2- 172.21.15.67:0/405243022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea532a0 0x55f44ea56a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f057b200640 1 -- 172.21.15.67:0/405243022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 msgr2=0x55f44ea72ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f057b200640 1 --2- 172.21.15.67:0/405243022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 0x55f44ea72ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.638+0000 7f057b200640 1 -- 172.21.15.67:0/405243022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44e6a32d0 con 0x55f44ea733e0 2024-05-30T16:44:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.639+0000 7f057b200640 1 --2- 172.21.15.67:0/405243022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea73840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0574009890 tx=0x7f057402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377050487f8d0a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.639+0000 7f0567000640 1 -- 172.21.15.67:0/405243022 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574037040 con 0x55f44ea733e0 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.639+0000 7f0567000640 1 -- 172.21.15.67:0/405243022 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0574042070 con 0x55f44ea733e0 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.639+0000 7f0567000640 1 -- 172.21.15.67:0/405243022 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574006e10 con 0x55f44ea733e0 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.640+0000 7f0584224740 1 -- 172.21.15.67:0/405243022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 msgr2=0x55f44ea73840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.640+0000 7f0584224740 1 --2- 172.21.15.67:0/405243022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea73840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0574009890 tx=0x7f057402c070 comp rx=0 tx=0).stop 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.640+0000 7f0584224740 1 -- 172.21.15.67:0/405243022 shutdown_connections 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.640+0000 7f0584224740 1 --2- 172.21.15.67:0/405243022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea73840 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.640+0000 7f0584224740 1 --2- 172.21.15.67:0/405243022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 0x55f44ea72ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.640+0000 7f0584224740 1 --2- 172.21.15.67:0/405243022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea532a0 0x55f44ea56a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.640+0000 7f0584224740 1 -- 172.21.15.67:0/405243022 >> 172.21.15.67:0/405243022 conn(0x55f44ea29280 msgr2=0x55f44ea2a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.640+0000 7f0584224740 1 -- 172.21.15.67:0/405243022 shutdown_connections 2024-05-30T16:44:57.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.640+0000 7f0584224740 1 -- 172.21.15.67:0/405243022 wait complete. 2024-05-30T16:44:57.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.641+0000 7f0584224740 1 -- start start 2024-05-30T16:44:57.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.641+0000 7f0584224740 1 Processor -- start 2024-05-30T16:44:57.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f0584224740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f0584224740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea733e0 0x55f44ea71a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f0584224740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea4f100 0x55f44ea71fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f0584224740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f44ea270b0 con 0x55f44ea56f60 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f0584224740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f44e94a570 con 0x55f44ea4f100 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f0584224740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f44ea27390 con 0x55f44ea733e0 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f0584224740 10 client.0 _ll_register_callbacks cb 0x55f44e918fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f057a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f057a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea857b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46748/0 (socket says 172.21.15.67:46748) 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f057a800640 1 -- 172.21.15.67:0/1919722393 learned_addr learned my addr 172.21.15.67:0/1919722393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f057a800640 1 -- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea733e0 msgr2=0x55f44ea71a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.642+0000 7f057a800640 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea733e0 0x55f44ea71a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.643+0000 7f057a800640 1 -- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea4f100 msgr2=0x55f44ea71fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.643+0000 7f057b200640 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea4f100 0x55f44ea71fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.643+0000 7f057a800640 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea4f100 0x55f44ea71fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.643+0000 7f057a800640 1 -- 172.21.15.67:0/1919722393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f44e76d6b0 con 0x55f44ea56f60 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.643+0000 7f057a800640 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea857b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f056c017e90 tx=0x7f056c015780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60f1eb55d198f7f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.643+0000 7f055b400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.643+0000 7f055b400640 1 -- 172.21.15.67:0/1919722393 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c0096d0 con 0x55f44ea56f60 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.643+0000 7f0584224740 1 -- 172.21.15.67:0/1919722393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574042070 con 0x55f44ea56f60 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.643+0000 7f055b400640 1 -- 172.21.15.67:0/1919722393 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f056c0203f0 con 0x55f44ea56f60 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f055b400640 1 -- 172.21.15.67:0/1919722393 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f056c0205f0 con 0x55f44ea56f60 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f0584224740 10 client.5004 fetch_fsmap learned FSMap version 16 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f0584224740 1 -- 172.21.15.67:0/1919722393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55f44ea5a220 con 0x55f44ea56f60 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f055b400640 1 -- 172.21.15.67:0/1919722393 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0574042070 con 0x55f44ea56f60 2024-05-30T16:44:57.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f055b400640 1 -- 172.21.15.67:0/1919722393 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f056c028030 con 0x55f44ea56f60 2024-05-30T16:44:57.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f0584224740 10 client.5004 fetch_fsmap finished waiting for FSMap version 16 2024-05-30T16:44:57.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f0584224740 10 client.5004 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-30T16:44:57.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f0584224740 20 client.5004 populate_metadata read hostname 'smithi067' 2024-05-30T16:44:57.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f0584224740 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] conn(0x55f44ea51410 0x55f44ea58440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f0584224740 4 client.5004 mds_command: new command op to 4821 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-30T16:44:57.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.644+0000 7f0584224740 1 -- 172.21.15.67:0/1919722393 --> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f44e99e530 con 0x55f44ea51410 2024-05-30T16:44:57.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.645+0000 7f0579e00640 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] conn(0x55f44ea51410 0x55f44ea58440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.646+0000 7f0579e00640 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] conn(0x55f44ea51410 0x55f44ea58440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464fb0ed1a93ba3a server_cookie=c35ac427eaa20e6 in_seq=0 out_seq=0 2024-05-30T16:44:57.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.646+0000 7f055b400640 10 client.5004 ms_handle_connect on v2:172.21.15.169:6833/2481507396 2024-05-30T16:44:57.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f055b400640 1 -- 172.21.15.67:0/1919722393 <== mds.0 v2:172.21.15.169:6833/2481507396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f44e99e530 con 0x55f44ea51410 2024-05-30T16:44:57.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f055b400640 10 client.5004 handle_command_reply: tid=0 2024-05-30T16:44:57.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 client.5004 shutdown 2024-05-30T16:44:57.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 -- 172.21.15.67:0/1919722393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 msgr2=0x55f44ea857b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea857b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f056c017e90 tx=0x7f056c015780 comp rx=0 tx=0).stop 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 -- 172.21.15.67:0/1919722393 shutdown_connections 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] conn(0x55f44ea51410 0x55f44ea58440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea4f100 0x55f44ea71fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea733e0 0x55f44ea71a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 --2- 172.21.15.67:0/1919722393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea857b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f055b400640 0 client.5004 ms_handle_reset on v2:172.21.15.169:6833/2481507396 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 -- 172.21.15.67:0/1919722393 >> 172.21.15.67:0/1919722393 conn(0x55f44ea532a0 msgr2=0x55f44ea2cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 -- 172.21.15.67:0/1919722393 shutdown_connections 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 1 -- 172.21.15.67:0/1919722393 wait complete. 2024-05-30T16:44:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.649+0000 7f0584224740 20 client.5004 trim_cache size 0 max 16384 2024-05-30T16:44:57.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.669+0000 7f0584224740 1 Processor -- start 2024-05-30T16:44:57.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.669+0000 7f0584224740 1 -- start start 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.670+0000 7f0584224740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 0x55f44ea2d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.670+0000 7f0584224740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.670+0000 7f0584224740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea733e0 0x55f44ea52990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.670+0000 7f0584224740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f057c1fb960 con 0x55f44ea56f60 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.670+0000 7f0584224740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f057c0c80d0 con 0x55f44ea733e0 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.670+0000 7f0584224740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f057c0c4330 con 0x55f44ea4f100 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f0579e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f0579e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea737c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46754/0 (socket says 172.21.15.67:46754) 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f0579e00640 1 -- 172.21.15.67:0/2048066685 learned_addr learned my addr 172.21.15.67:0/2048066685 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f057b200640 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea733e0 0x55f44ea52990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f057a800640 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 0x55f44ea2d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f0579e00640 1 -- 172.21.15.67:0/2048066685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 msgr2=0x55f44ea2d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f0579e00640 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 0x55f44ea2d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f0579e00640 1 -- 172.21.15.67:0/2048066685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea733e0 msgr2=0x55f44ea52990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f0579e00640 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea733e0 0x55f44ea52990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f0579e00640 1 -- 172.21.15.67:0/2048066685 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44ea597b0 con 0x55f44ea56f60 2024-05-30T16:44:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.671+0000 7f057a800640 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 0x55f44ea2d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.672+0000 7f0579e00640 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea737c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0568054f90 tx=0x7f056801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ecf34f23037dc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.672+0000 7f0566600640 1 -- 172.21.15.67:0/2048066685 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568087070 con 0x55f44ea56f60 2024-05-30T16:44:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.672+0000 7f0566600640 1 -- 172.21.15.67:0/2048066685 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0568056bd0 con 0x55f44ea56f60 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.672+0000 7f0566600640 1 -- 172.21.15.67:0/2048066685 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568059e60 con 0x55f44ea56f60 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.672+0000 7f0584224740 1 -- 172.21.15.67:0/2048066685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 msgr2=0x55f44ea737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.672+0000 7f0584224740 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea737c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0568054f90 tx=0x7f056801f040 comp rx=0 tx=0).stop 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.672+0000 7f0584224740 1 -- 172.21.15.67:0/2048066685 shutdown_connections 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.672+0000 7f0584224740 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea733e0 0x55f44ea52990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.673+0000 7f0584224740 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea56f60 0x55f44ea737c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.673+0000 7f0584224740 1 --2- 172.21.15.67:0/2048066685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 0x55f44ea2d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.673+0000 7f0584224740 1 -- 172.21.15.67:0/2048066685 >> 172.21.15.67:0/2048066685 conn(0x55f44ea532a0 msgr2=0x55f44ea573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.673+0000 7f0584224740 1 -- 172.21.15.67:0/2048066685 shutdown_connections 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.673+0000 7f0584224740 1 -- 172.21.15.67:0/2048066685 wait complete. 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.673+0000 7f0584224740 1 -- start start 2024-05-30T16:44:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.673+0000 7f0584224740 1 Processor -- start 2024-05-30T16:44:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.674+0000 7f0584224740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 0x55f44eaef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.674+0000 7f0584224740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 0x55f44eaeac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.674+0000 7f0584224740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea55690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.674+0000 7f0584224740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f44ea51290 con 0x55f44ea733e0 2024-05-30T16:44:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.674+0000 7f0584224740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f44ea5a5c0 con 0x55f44ea56f60 2024-05-30T16:44:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.674+0000 7f0584224740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f44ea51880 con 0x55f44ea4f100 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.674+0000 7f0584224740 10 client.0 _ll_register_callbacks cb 0x7f056c017ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f057b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea55690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f057b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea55690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46758/0 (socket says 172.21.15.67:46758) 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f057b200640 1 -- 172.21.15.67:0/1852648515 learned_addr learned my addr 172.21.15.67:0/1852648515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f0579e00640 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 0x55f44eaeac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f057a800640 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 0x55f44eaef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f057b200640 1 -- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 msgr2=0x55f44eaef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f057b200640 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 0x55f44eaef280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f057b200640 1 -- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 msgr2=0x55f44eaeac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f057b200640 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 0x55f44eaeac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f057b200640 1 -- 172.21.15.67:0/1852648515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f44ea949a0 con 0x55f44ea733e0 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.675+0000 7f0579e00640 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 0x55f44eaeac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:44:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.676+0000 7f057b200640 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea55690 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f057403f040 tx=0x7f0574004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf9d9fa90c7f619 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.676+0000 7f055b400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.676+0000 7f055b400640 1 -- 172.21.15.67:0/1852648515 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574037040 con 0x55f44ea733e0 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.676+0000 7f055b400640 1 -- 172.21.15.67:0/1852648515 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0574003810 con 0x55f44ea733e0 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.676+0000 7f0584224740 1 -- 172.21.15.67:0/1852648515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0568056bd0 con 0x55f44ea733e0 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.676+0000 7f055b400640 1 -- 172.21.15.67:0/1852648515 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f0574004490 con 0x55f44ea733e0 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.676+0000 7f0584224740 10 client.5010 fetch_fsmap learned FSMap version 16 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.676+0000 7f0584224740 1 -- 172.21.15.67:0/1852648515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55f44ea94390 con 0x55f44ea733e0 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.676+0000 7f055b400640 1 -- 172.21.15.67:0/1852648515 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0568056bd0 con 0x55f44ea733e0 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.677+0000 7f055b400640 1 -- 172.21.15.67:0/1852648515 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f057403a030 con 0x55f44ea733e0 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.677+0000 7f0584224740 10 client.5010 fetch_fsmap finished waiting for FSMap version 16 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.677+0000 7f0584224740 10 client.5010 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.677+0000 7f0584224740 20 client.5010 populate_metadata read hostname 'smithi067' 2024-05-30T16:44:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.677+0000 7f0584224740 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] conn(0x55f44ea28430 0x55f44ea83900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.677+0000 7f0584224740 4 client.5010 mds_command: new command op to 4821 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-30T16:44:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.677+0000 7f0584224740 1 -- 172.21.15.67:0/1852648515 --> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] -- command(tid 0: {"prefix": "status"}) -- 0x55f44ea6d5b0 con 0x55f44ea28430 2024-05-30T16:44:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.677+0000 7f057a800640 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] conn(0x55f44ea28430 0x55f44ea83900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.678+0000 7f057a800640 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] conn(0x55f44ea28430 0x55f44ea83900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d101d7185baecf4 server_cookie=765acf32ae48bf64 in_seq=0 out_seq=0 2024-05-30T16:44:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.678+0000 7f055b400640 10 client.5010 ms_handle_connect on v2:172.21.15.169:6833/2481507396 2024-05-30T16:44:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f055b400640 1 -- 172.21.15.67:0/1852648515 <== mds.0 v2:172.21.15.169:6833/2481507396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55f44ea6d5b0 con 0x55f44ea28430 2024-05-30T16:44:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f055b400640 10 client.5010 handle_command_reply: tid=0 2024-05-30T16:44:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f0584224740 1 client.5010 shutdown 2024-05-30T16:44:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f0584224740 1 -- 172.21.15.67:0/1852648515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 msgr2=0x55f44ea55690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f0584224740 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea55690 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f057403f040 tx=0x7f0574004730 comp rx=0 tx=0).stop 2024-05-30T16:44:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f0584224740 1 -- 172.21.15.67:0/1852648515 shutdown_connections 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f0584224740 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:6833/2481507396,v1:172.21.15.169:6835/2481507396] conn(0x55f44ea28430 0x55f44ea83900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f0584224740 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f44ea733e0 0x55f44ea55690 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f0584224740 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f44ea56f60 0x55f44eaeac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f0584224740 1 --2- 172.21.15.67:0/1852648515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f44ea4f100 0x55f44eaef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f055b400640 0 client.5010 ms_handle_reset on v2:172.21.15.169:6833/2481507396 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.679+0000 7f0584224740 1 -- 172.21.15.67:0/1852648515 >> 172.21.15.67:0/1852648515 conn(0x55f44ea532a0 msgr2=0x55f44ea2cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0584224740 1 -- 172.21.15.67:0/1852648515 shutdown_connections 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0584224740 1 -- 172.21.15.67:0/1852648515 wait complete. 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0584224740 20 client.5010 trim_cache size 0 max 16384 2024-05-30T16:44:57.684 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4821, 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 11.77733293, 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 16, 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 35, 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 35, 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 19.814189997 2024-05-30T16:44:57.685 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:44:57.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 -- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f055c06c8f0 msgr2=0x7f055c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f055c06c8f0 0x7f055c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f056c0070b0 tx=0x7f056c007040 comp rx=0 tx=0).stop 2024-05-30T16:44:57.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 -- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0b3690 msgr2=0x7f057c15f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0b3690 0x7f057c15f5e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f056800c780 tx=0x7f056800cc40 comp rx=0 tx=0).stop 2024-05-30T16:44:57.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 -- 172.21.15.67:0/2093240725 shutdown_connections 2024-05-30T16:44:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f055c06c8f0 0x7f055c06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f057c0d0bb0 0x7f057c15dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f057c0b3690 0x7f057c15f5e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 --2- 172.21.15.67:0/2093240725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f057c06f860 0x7f057c099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.680+0000 7f0581400640 1 -- 172.21.15.67:0/2093240725 >> 172.21.15.67:0/2093240725 conn(0x7f057c06bae0 msgr2=0x7f057c0d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.681+0000 7f0581400640 1 -- 172.21.15.67:0/2093240725 shutdown_connections 2024-05-30T16:44:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.681+0000 7f0581400640 1 -- 172.21.15.67:0/2093240725 wait complete. 2024-05-30T16:44:57.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T16:44:57.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.901+0000 7fce0a200640 1 Processor -- start 2024-05-30T16:44:57.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.902+0000 7fce0a200640 1 -- start start 2024-05-30T16:44:57.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.903+0000 7fce0a200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fce04071bc0 0x7fce04072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.903+0000 7fce0a200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0406e6e0 con 0x7fce04158b80 2024-05-30T16:44:57.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.903+0000 7fce0a200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0406e9c0 con 0x7fce040b32c0 2024-05-30T16:44:57.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.903+0000 7fce0a200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fce0406eca0 con 0x7fce04071bc0 2024-05-30T16:44:57.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.903+0000 7fce08e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fce04158b80 0x7fce0406dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54876/0 (socket says 172.21.15.67:54876) 2024-05-30T16:44:57.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.903+0000 7fce08e00640 1 -- 172.21.15.67:0/98830436 learned_addr learned my addr 172.21.15.67:0/98830436 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03e00640 1 --2- 172.21.15.67:0/98830436 >> v2:172.21.15.169:3300/0 conn(0x7fce04071bc0 0x7fce04072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3784372456 0 0) 0x7fce0406e6e0 con 0x7fce04158b80 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdec003520 con 0x7fce04158b80 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 206889300 0 0) 0x7fcdec003520 con 0x7fce04158b80 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 >> v2:172.21.15.169:3300/0 conn(0x7fce04071bc0 msgr2=0x7fce04072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03400640 1 --2- 172.21.15.67:0/98830436 >> v2:172.21.15.169:3300/0 conn(0x7fce04071bc0 0x7fce04072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 >> v1:172.21.15.169:6789/0 conn(0x7fce040b32c0 legacy=0x7fce040b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0406ef80 con 0x7fce04158b80 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fcdf8001670 con 0x7fce04158b80 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.904+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 >> v1:172.21.15.67:6789/0 conn(0x7fce04158b80 legacy=0x7fce0406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03400640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec003a00 0x7fcdec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03400640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec004800 0x7fcdec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03400640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0051c0 0x7fcdec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fce0406eca0 con 0x7fcdec004800 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec00c0c0 con 0x7fcdec0051c0 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec00c2c0 con 0x7fcdec003a00 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03e00640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec004800 0x7fcdec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce08e00640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec003a00 0x7fcdec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce09800640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0051c0 0x7fcdec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03e00640 1 -- 172.21.15.67:0/98830436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec003a00 msgr2=0x7fcdec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03e00640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdec003a00 0x7fcdec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03e00640 1 -- 172.21.15.67:0/98830436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0051c0 msgr2=0x7fcdec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03e00640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec0051c0 0x7fcdec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce03e00640 1 -- 172.21.15.67:0/98830436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec003520 con 0x7fcdec004800 2024-05-30T16:44:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.905+0000 7fce09800640 1 -- 172.21.15.67:0/98830436 reap_dead start 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.907+0000 7fce03e00640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec004800 0x7fcdec004c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009ac0 tx=0x7fcdec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dae1ce92d213eb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.907+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0036070 con 0x7fcdec004800 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.907+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf0031400 con 0x7fcdec004800 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.907+0000 7fce03400640 1 -- 172.21.15.67:0/98830436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0009690 con 0x7fcdec004800 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.908+0000 7fce0a200640 1 -- 172.21.15.67:0/98830436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec004800 msgr2=0x7fcdec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.908+0000 7fce0a200640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec004800 0x7fcdec004c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009ac0 tx=0x7fcdec005830 comp rx=0 tx=0).stop 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.908+0000 7fce0a200640 1 -- 172.21.15.67:0/98830436 shutdown_connections 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.908+0000 7fce0a200640 1 --2- 172.21.15.67:0/98830436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec004800 0x7fcdec004c80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.908+0000 7fce0a200640 1 -- 172.21.15.67:0/98830436 >> 172.21.15.67:0/98830436 conn(0x7fce040b1fc0 msgr2=0x7fce040b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.908+0000 7fce0a200640 1 -- 172.21.15.67:0/98830436 shutdown_connections 2024-05-30T16:44:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.908+0000 7fce0a200640 1 -- 172.21.15.67:0/98830436 wait complete. 2024-05-30T16:44:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.910+0000 7fce0a200640 1 Processor -- start 2024-05-30T16:44:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.910+0000 7fce0a200640 1 -- start start 2024-05-30T16:44:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce0a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec004800 0x7fce0409d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce0a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce04160b10 0x7fce041f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce0a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce040d03f0 0x7fce0409a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce0a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fce0406f780 con 0x7fce04160b10 2024-05-30T16:44:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce0a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce040b2d30 con 0x7fcdec004800 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce0a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fce041f4760 con 0x7fce040d03f0 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce03e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce04160b10 0x7fce041f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce03e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce04160b10 0x7fce041f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46766/0 (socket says 172.21.15.67:46766) 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce03e00640 1 -- 172.21.15.67:0/1317666823 learned_addr learned my addr 172.21.15.67:0/1317666823 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce08e00640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec004800 0x7fce0409d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce03e00640 1 -- 172.21.15.67:0/1317666823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce040d03f0 msgr2=0x7fce0409a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce03e00640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce040d03f0 0x7fce0409a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce03e00640 1 -- 172.21.15.67:0/1317666823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec004800 msgr2=0x7fce0409d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce03e00640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec004800 0x7fce0409d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.911+0000 7fce03e00640 1 -- 172.21.15.67:0/1317666823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0409dc80 con 0x7fce04160b10 2024-05-30T16:44:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.912+0000 7fce03e00640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce04160b10 0x7fce041f4bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0034040 tx=0x7fcdf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.912+0000 7fce00c00640 1 -- 172.21.15.67:0/1317666823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf003f020 con 0x7fce04160b10 2024-05-30T16:44:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.912+0000 7fce00c00640 1 -- 172.21.15.67:0/1317666823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf0037040 con 0x7fce04160b10 2024-05-30T16:44:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.912+0000 7fce00c00640 1 -- 172.21.15.67:0/1317666823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0036070 con 0x7fce04160b10 2024-05-30T16:44:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.912+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0409df60 con 0x7fce04160b10 2024-05-30T16:44:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.912+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce04160f70 con 0x7fce04160b10 2024-05-30T16:44:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.913+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce040689e0 con 0x7fce04160b10 2024-05-30T16:44:57.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.914+0000 7fce00c00640 1 -- 172.21.15.67:0/1317666823 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdf0042020 con 0x7fce04160b10 2024-05-30T16:44:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.915+0000 7fce00c00640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcde406c8f0 0x7fcde406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:44:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.915+0000 7fce00c00640 1 -- 172.21.15.67:0/1317666823 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fcdf00aba10 con 0x7fce04160b10 2024-05-30T16:44:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.916+0000 7fce08e00640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcde406c8f0 0x7fcde406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:44:57.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.916+0000 7fce08e00640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcde406c8f0 0x7fcde406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8001050 tx=0x7fcdf8001b60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:44:57.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:57.920+0000 7fce00c00640 1 -- 172.21.15.67:0/1317666823 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcdf0049070 con 0x7fce04160b10 2024-05-30T16:44:58.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.164+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fce040b0c50 con 0x7fce04160b10 2024-05-30T16:44:58.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.165+0000 7fce00c00640 1 -- 172.21.15.67:0/1317666823 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fcdf0079da0 con 0x7fce04160b10 2024-05-30T16:44:58.167 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T16:44:58.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.167+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcde406c8f0 msgr2=0x7fcde406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:58.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.167+0000 7fce0a200640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcde406c8f0 0x7fcde406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8001050 tx=0x7fcdf8001b60 comp rx=0 tx=0).stop 2024-05-30T16:44:58.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.167+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce04160b10 msgr2=0x7fce041f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:44:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.167+0000 7fce0a200640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce04160b10 0x7fce041f4bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0034040 tx=0x7fcdf0004730 comp rx=0 tx=0).stop 2024-05-30T16:44:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.168+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 shutdown_connections 2024-05-30T16:44:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.168+0000 7fce0a200640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcde406c8f0 0x7fcde406edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.168+0000 7fce0a200640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce040d03f0 0x7fce0409a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.168+0000 7fce0a200640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce04160b10 0x7fce041f4bf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.168+0000 7fce0a200640 1 --2- 172.21.15.67:0/1317666823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdec004800 0x7fce0409d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:44:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.168+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 >> 172.21.15.67:0/1317666823 conn(0x7fce040b1fc0 msgr2=0x7fce04162160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:44:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.168+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 shutdown_connections 2024-05-30T16:44:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58.168+0000 7fce0a200640 1 -- 172.21.15.67:0/1317666823 wait complete. 2024-05-30T16:44:58.189 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T16:44:58.189 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T16:44:58.189 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T16:44:58.189 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-30T16:44:58.189 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:58.189 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T16:44:58.189 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T16:44:58.189 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T16:44:58.189 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:44:58.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:58.190 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T16:44:58.245 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:44:58.245 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:44:58.245 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:44:58.245 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:58.245 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T16:44:58.245 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42474sec preferred_lft 42474sec 2024-05-30T16:44:58.246 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::74f5:cfff:fec0:693d/64 scope link 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::a8fe:aeff:fef7:f6a0/64 scope link 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:58.247 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::185e:a4ff:fec6:7d8c/64 scope link 2024-05-30T16:44:58.248 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:44:58.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:58.249 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T16:44:58.333 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T16:44:58.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:58.337 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T16:44:58.392 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T16:44:58.393 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T16:44:58.393 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T16:44:58.393 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T16:44:58.393 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T16:44:58.394 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:44:58.394 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:44:58.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T16:44:58.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T16:44:58.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T16:44:58.394 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:44:58.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:44:58.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:58.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:58.620 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T16:44:58.674 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T16:44:58.675 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-30T16:44:58.676 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:44:58.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:58.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-05-30T16:44:58.766 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-30T16:44:58.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:58.766 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:58.824 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:44:58.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:44:58.825 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:58.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:44:58.968 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:59.028 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:44:59.028 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:44:59.029 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:44:59.030 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:44:59.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:44:59.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:59.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:59.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:59.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:59.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:59.032 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:44:59.087 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T16:44:59.088 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-30T16:44:59.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:44:59.167 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:44:59.213 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:44:59.213 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:44:59.214 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:44:59.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:44:59.216 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:44:59.216 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:44:59.216 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:59.216 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:44:59.216 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:59.216 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:44:59.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:59.217 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:44:59.279 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-05-30T16:44:59.277+0000 7f1f4e722480 -1 init, newargv = 0x563cbf24cb80 newargc=15 2024-05-30T16:44:59.279 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[47805]: starting ceph client 2024-05-30T16:44:59.290 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[47805]: starting fuse 2024-05-30T16:44:59.304 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T16:44:59.304 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T16:44:59.305 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-05-30T16:44:59.306 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T16:44:59.493 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.47805.asok 2024-05-30T16:44:59.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:44:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:44:59.509 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47805.asok 2024-05-30T16:44:59.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:59.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47805.asok status 2024-05-30T16:44:59.751 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:44:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T16:44:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T16:44:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T16:44:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-05-30T16:44:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T16:44:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-30T16:44:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "47805", 2024-05-30T16:44:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T16:44:59.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5025, 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5025 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1647879332 2024-05-30T16:44:59.753 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1647879332 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.5025 192.168.0.1:0/1647879332", 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/1647879332", 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 16, 2024-05-30T16:44:59.754 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 35, 2024-05-30T16:44:59.755 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T16:44:59.755 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T16:44:59.755 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T16:44:59.755 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:44:59.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:59.767 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T16:44:59.767 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:59.822 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T16:44:59.822 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:59.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:44:59.823 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:44:59.910 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-30T16:45:00.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.122+0000 7fd748c00640 1 Processor -- start 2024-05-30T16:45:00.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.122+0000 7fd748c00640 1 -- start start 2024-05-30T16:45:00.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.124+0000 7fd748c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7440af6f0 0x7fd7441623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:00.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.124+0000 7fd748c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7441428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:00.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.124+0000 7fd748c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd744162cb0 con 0x7fd7441480a0 2024-05-30T16:45:00.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.124+0000 7fd748c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd744162f90 con 0x7fd7440af6f0 2024-05-30T16:45:00.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.124+0000 7fd748c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd744163270 con 0x7fd7440d0b40 2024-05-30T16:45:00.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.125+0000 7fd743400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7440af6f0 0x7fd7441623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:00.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.125+0000 7fd743e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7441428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:00.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.125+0000 7fd743400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7440af6f0 0x7fd7441623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:46020/0 (socket says 172.21.15.67:46020) 2024-05-30T16:45:00.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.125+0000 7fd743400640 1 -- 172.21.15.67:0/2641170219 learned_addr learned my addr 172.21.15.67:0/2641170219 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:45:00.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.126+0000 7fd743e00640 1 -- 172.21.15.67:0/2641170219 >> v2:172.21.15.169:3300/0 conn(0x7fd7440af6f0 msgr2=0x7fd7441623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:00.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.126+0000 7fd743e00640 1 --2- 172.21.15.67:0/2641170219 >> v2:172.21.15.169:3300/0 conn(0x7fd7440af6f0 0x7fd7441623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:00.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.126+0000 7fd743e00640 1 -- 172.21.15.67:0/2641170219 >> v1:172.21.15.169:6789/0 conn(0x7fd7441480a0 legacy=0x7fd7440d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:00.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.126+0000 7fd743e00640 1 -- 172.21.15.67:0/2641170219 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7441420d0 con 0x7fd7440d0b40 2024-05-30T16:45:00.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.127+0000 7fd743e00640 1 --2- 172.21.15.67:0/2641170219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7441428d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009d90 tx=0x7fd73c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=242e700db8c3f4d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:00.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.127+0000 7fd742000640 1 -- 172.21.15.67:0/2641170219 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c007740 con 0x7fd7440d0b40 2024-05-30T16:45:00.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.127+0000 7fd742000640 1 -- 172.21.15.67:0/2641170219 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c0078e0 con 0x7fd7440d0b40 2024-05-30T16:45:00.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.127+0000 7fd742000640 1 -- 172.21.15.67:0/2641170219 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c007cb0 con 0x7fd7440d0b40 2024-05-30T16:45:00.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.128+0000 7fd748c00640 1 -- 172.21.15.67:0/2641170219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 msgr2=0x7fd7441428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:00.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.128+0000 7fd748c00640 1 --2- 172.21.15.67:0/2641170219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7441428d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009d90 tx=0x7fd73c02a660 comp rx=0 tx=0).stop 2024-05-30T16:45:00.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.128+0000 7fd748c00640 1 -- 172.21.15.67:0/2641170219 shutdown_connections 2024-05-30T16:45:00.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.128+0000 7fd748c00640 1 --2- 172.21.15.67:0/2641170219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7441428d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:00.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.128+0000 7fd748c00640 1 --2- 172.21.15.67:0/2641170219 >> v2:172.21.15.169:3300/0 conn(0x7fd7440af6f0 0x7fd7441623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:00.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.128+0000 7fd748c00640 1 -- 172.21.15.67:0/2641170219 >> 172.21.15.67:0/2641170219 conn(0x7fd74406bae0 msgr2=0x7fd74406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:00.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.128+0000 7fd748c00640 1 -- 172.21.15.67:0/2641170219 shutdown_connections 2024-05-30T16:45:00.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.128+0000 7fd748c00640 1 -- 172.21.15.67:0/2641170219 wait complete. 2024-05-30T16:45:00.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.130+0000 7fd748c00640 1 Processor -- start 2024-05-30T16:45:00.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.130+0000 7fd748c00640 1 -- start start 2024-05-30T16:45:00.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.131+0000 7fd748c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7440af6f0 0x7fd744142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:00.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.131+0000 7fd748c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7440998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:00.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.131+0000 7fd748c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7441480a0 0x7fd744073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:00.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.131+0000 7fd748c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd744162a70 con 0x7fd7441480a0 2024-05-30T16:45:00.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.131+0000 7fd748c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441628f0 con 0x7fd7440af6f0 2024-05-30T16:45:00.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.131+0000 7fd748c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd7440afad0 con 0x7fd7440d0b40 2024-05-30T16:45:00.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.131+0000 7fd743e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7441480a0 0x7fd744073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.131+0000 7fd743e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7441480a0 0x7fd744073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46774/0 (socket says 172.21.15.67:46774) 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd743e00640 1 -- 172.21.15.67:0/821526983 learned_addr learned my addr 172.21.15.67:0/821526983 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd743400640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7440af6f0 0x7fd744142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd742a00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7440998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd743e00640 1 -- 172.21.15.67:0/821526983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 msgr2=0x7fd7440998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd743e00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7440998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd743e00640 1 -- 172.21.15.67:0/821526983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7440af6f0 msgr2=0x7fd744142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd743e00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7440af6f0 0x7fd744142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd743e00640 1 -- 172.21.15.67:0/821526983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd744144bc0 con 0x7fd7441480a0 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd742a00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7440998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.132+0000 7fd743e00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7441480a0 0x7fd744073160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009eb0 tx=0x7fd73c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.133+0000 7fd73b400640 1 -- 172.21.15.67:0/821526983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c040070 con 0x7fd7441480a0 2024-05-30T16:45:00.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.133+0000 7fd73b400640 1 -- 172.21.15.67:0/821526983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c031b40 con 0x7fd7441480a0 2024-05-30T16:45:00.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.133+0000 7fd73b400640 1 -- 172.21.15.67:0/821526983 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c045460 con 0x7fd7441480a0 2024-05-30T16:45:00.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.133+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd744143b20 con 0x7fd7441480a0 2024-05-30T16:45:00.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.133+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7441440e0 con 0x7fd7441480a0 2024-05-30T16:45:00.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.134+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7440689e0 con 0x7fd7441480a0 2024-05-30T16:45:00.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.135+0000 7fd73b400640 1 -- 172.21.15.67:0/821526983 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd73c039040 con 0x7fd7441480a0 2024-05-30T16:45:00.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.136+0000 7fd73b400640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd72406c940 0x7fd72406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:00.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.136+0000 7fd73b400640 1 -- 172.21.15.67:0/821526983 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd73c0ad360 con 0x7fd7441480a0 2024-05-30T16:45:00.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.136+0000 7fd743400640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd72406c940 0x7fd72406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:00.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.137+0000 7fd743400640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd72406c940 0x7fd72406ee00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd734009ac0 tx=0x7fd734008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:00.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.141+0000 7fd73b400640 1 -- 172.21.15.67:0/821526983 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd73c07b800 con 0x7fd7441480a0 2024-05-30T16:45:00.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:00.386+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd7440c9e50 con 0x7fd7441480a0 2024-05-30T16:45:01.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.083+0000 7fd73b400640 1 -- 172.21.15.67:0/821526983 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) ==== 136+0+0 (secure 0 0 0) 0x7fd73c07b100 con 0x7fd7441480a0 2024-05-30T16:45:01.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.087+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd72406c940 msgr2=0x7fd72406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.087+0000 7fd748c00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd72406c940 0x7fd72406ee00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd734009ac0 tx=0x7fd734008040 comp rx=0 tx=0).stop 2024-05-30T16:45:01.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.087+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7441480a0 msgr2=0x7fd744073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.087+0000 7fd748c00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7441480a0 0x7fd744073160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009eb0 tx=0x7fd73c004a40 comp rx=0 tx=0).stop 2024-05-30T16:45:01.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.088+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 shutdown_connections 2024-05-30T16:45:01.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.088+0000 7fd748c00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd72406c940 0x7fd72406ee00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.088+0000 7fd748c00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7441480a0 0x7fd744073160 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.088+0000 7fd748c00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7440d0b40 0x7fd7440998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.088+0000 7fd748c00640 1 --2- 172.21.15.67:0/821526983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7440af6f0 0x7fd744142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.088+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 >> 172.21.15.67:0/821526983 conn(0x7fd74406bae0 msgr2=0x7fd7440afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:01.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.088+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 shutdown_connections 2024-05-30T16:45:01.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.088+0000 7fd748c00640 1 -- 172.21.15.67:0/821526983 wait complete. 2024-05-30T16:45:01.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:45:01.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.330+0000 7f08d0a00640 1 Processor -- start 2024-05-30T16:45:01.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.332+0000 7f08d0a00640 1 -- start start 2024-05-30T16:45:01.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.332+0000 7f08d0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0b5e20 0x7f08cc0b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.332+0000 7f08d0a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08cc0b4b90 con 0x7f08cc06d020 2024-05-30T16:45:01.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.332+0000 7f08d0a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08cc0b5060 con 0x7f08cc158d20 2024-05-30T16:45:01.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.332+0000 7f08d0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc0b5340 con 0x7f08cc0b5e20 2024-05-30T16:45:01.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.333+0000 7f08c9400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f08cc06d020 0x7f08cc0b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47778/0 (socket says 172.21.15.67:47778) 2024-05-30T16:45:01.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.333+0000 7f08c9400640 1 -- 172.21.15.67:0/2920400973 learned_addr learned my addr 172.21.15.67:0/2920400973 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:45:01.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.333+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006730575 0 0) 0x7f08cc0b4b90 con 0x7f08cc06d020 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.333+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08b0003520 con 0x7f08cc06d020 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.333+0000 7f08ca800640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0b5e20 0x7f08cc0b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2318714510 0 0) 0x7f08b0003520 con 0x7f08cc06d020 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0b5e20 msgr2=0x7f08cc0b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0b5e20 0x7f08cc0b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 >> v1:172.21.15.169:6789/0 conn(0x7f08cc158d20 legacy=0x7f08cc06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc0b5620 con 0x7f08cc06d020 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f08b8001670 con 0x7f08cc06d020 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 >> v1:172.21.15.67:6789/0 conn(0x7f08cc06d020 legacy=0x7f08cc0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b00039e0 0x7f08b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08b0004800 0x7f08b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08b00051c0 0x7f08b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc0b5340 con 0x7f08b00051c0 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08b000c090 con 0x7f08b00039e0 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08b000c260 con 0x7f08b0004800 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.334+0000 7f08ca800640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08b00051c0 0x7f08b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.335+0000 7f08c9e00640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b00039e0 0x7f08b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.335+0000 7f08c9400640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08b0004800 0x7f08b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.335+0000 7f08ca800640 1 -- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08b0004800 msgr2=0x7f08b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.335+0000 7f08ca800640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08b0004800 0x7f08b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.335+0000 7f08ca800640 1 -- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b00039e0 msgr2=0x7f08b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.335+0000 7f08ca800640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b00039e0 0x7f08b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.335+0000 7f08ca800640 1 -- 172.21.15.67:0/2920400973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b0003520 con 0x7f08b00051c0 2024-05-30T16:45:01.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.336+0000 7f08ca800640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08b00051c0 0x7f08b000b9c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f08cc15ed40 tx=0x7f08b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adf9bf4f1723382 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.336+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4030070 con 0x7f08b00051c0 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.336+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c402acc0 con 0x7f08b00051c0 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.336+0000 7f08c8a00640 1 -- 172.21.15.67:0/2920400973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c400b540 con 0x7f08b00051c0 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.337+0000 7f08d0a00640 1 -- 172.21.15.67:0/2920400973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08b00051c0 msgr2=0x7f08b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.337+0000 7f08d0a00640 1 --2- 172.21.15.67:0/2920400973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08b00051c0 0x7f08b000b9c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f08cc15ed40 tx=0x7f08b0005830 comp rx=0 tx=0).stop 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.337+0000 7f08ca800640 1 -- 172.21.15.67:0/2920400973 reap_dead start 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.337+0000 7f08d0a00640 1 -- 172.21.15.67:0/2920400973 shutdown_connections 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.337+0000 7f08d0a00640 1 -- 172.21.15.67:0/2920400973 >> 172.21.15.67:0/2920400973 conn(0x7f08cc0724f0 msgr2=0x7f08cc0728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.337+0000 7f08d0a00640 1 -- 172.21.15.67:0/2920400973 shutdown_connections 2024-05-30T16:45:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.337+0000 7f08d0a00640 1 -- 172.21.15.67:0/2920400973 wait complete. 2024-05-30T16:45:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.339+0000 7f08d0a00640 1 Processor -- start 2024-05-30T16:45:01.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.339+0000 7f08d0a00640 1 -- start start 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.340+0000 7f08d0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b00051c0 0x7f08cc15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.340+0000 7f08d0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc06e550 0x7f08cc06e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.340+0000 7f08d0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08cc06eef0 0x7f08cc0b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.340+0000 7f08d0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc0b4530 con 0x7f08cc06eef0 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.340+0000 7f08d0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc159100 con 0x7f08b00051c0 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.340+0000 7f08d0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc0b46b0 con 0x7f08cc06e550 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08ca800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08cc06eef0 0x7f08cc0b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08ca800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08cc06eef0 0x7f08cc0b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34092/0 (socket says 172.21.15.67:34092) 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08ca800640 1 -- 172.21.15.67:0/3358404631 learned_addr learned my addr 172.21.15.67:0/3358404631 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08c9e00640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b00051c0 0x7f08cc15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08c9400640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc06e550 0x7f08cc06e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08ca800640 1 -- 172.21.15.67:0/3358404631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc06e550 msgr2=0x7f08cc06e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08ca800640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc06e550 0x7f08cc06e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08ca800640 1 -- 172.21.15.67:0/3358404631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b00051c0 msgr2=0x7f08cc15dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08ca800640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b00051c0 0x7f08cc15dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.341+0000 7f08ca800640 1 -- 172.21.15.67:0/3358404631 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc152fa0 con 0x7f08cc06eef0 2024-05-30T16:45:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.342+0000 7f08ca800640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08cc06eef0 0x7f08cc0b06e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b9b0 tx=0x7f08c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.342+0000 7f08c2000640 1 -- 172.21.15.67:0/3358404631 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c402aac0 con 0x7f08cc06eef0 2024-05-30T16:45:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.342+0000 7f08c2000640 1 -- 172.21.15.67:0/3358404631 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c403d650 con 0x7f08cc06eef0 2024-05-30T16:45:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.342+0000 7f08c2000640 1 -- 172.21.15.67:0/3358404631 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c403da40 con 0x7f08cc06eef0 2024-05-30T16:45:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.342+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc153210 con 0x7f08cc06eef0 2024-05-30T16:45:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.342+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc1537d0 con 0x7f08cc06eef0 2024-05-30T16:45:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.343+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc0689e0 con 0x7f08cc06eef0 2024-05-30T16:45:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.344+0000 7f08c2000640 1 -- 172.21.15.67:0/3358404631 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08c4046020 con 0x7f08cc06eef0 2024-05-30T16:45:01.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.346+0000 7f08c2000640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08ac06c920 0x7f08ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.346+0000 7f08c2000640 1 -- 172.21.15.67:0/3358404631 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f08c40abc80 con 0x7f08cc06eef0 2024-05-30T16:45:01.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.346+0000 7f08c9e00640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08ac06c920 0x7f08ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.347+0000 7f08c9e00640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08ac06c920 0x7f08ac06ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08bc005540 tx=0x7f08bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:01.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.350+0000 7f08c2000640 1 -- 172.21.15.67:0/3358404631 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08c404d020 con 0x7f08cc06eef0 2024-05-30T16:45:01.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.588+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f08cc070e80 con 0x7f08ac06c920 2024-05-30T16:45:01.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.590+0000 7f08c2000640 1 -- 172.21.15.67:0/3358404631 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f08cc070e80 con 0x7f08ac06c920 2024-05-30T16:45:01.592 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:45:01.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:45:01.592 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:45:01.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:45:01.592 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:45:01.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08ac06c920 msgr2=0x7f08ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08ac06c920 0x7f08ac06ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08bc005540 tx=0x7f08bc009040 comp rx=0 tx=0).stop 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08cc06eef0 msgr2=0x7f08cc0b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08cc06eef0 0x7f08cc0b06e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b9b0 tx=0x7f08c4004730 comp rx=0 tx=0).stop 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 shutdown_connections 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f08ac06c920 0x7f08ac06ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08cc06eef0 0x7f08cc0b06e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc06e550 0x7f08cc06e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 --2- 172.21.15.67:0/3358404631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b00051c0 0x7f08cc15dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 >> 172.21.15.67:0/3358404631 conn(0x7f08cc0724f0 msgr2=0x7f08cc06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 shutdown_connections 2024-05-30T16:45:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.593+0000 7f08d0a00640 1 -- 172.21.15.67:0/3358404631 wait complete. 2024-05-30T16:45:01.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:45:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.829+0000 7f925b200640 1 Processor -- start 2024-05-30T16:45:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.829+0000 7f925b200640 1 -- start start 2024-05-30T16:45:01.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.830+0000 7f925b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0ce1f0 0x7f925c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.831+0000 7f925b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c0b3580 con 0x7f925c0ce5d0 2024-05-30T16:45:01.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.831+0000 7f925b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c15f490 con 0x7f925c0b3d60 2024-05-30T16:45:01.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.831+0000 7f925b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f925c15f770 con 0x7f925c0ce1f0 2024-05-30T16:45:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.831+0000 7f9259400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f925c0ce5d0 0x7f925c15ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47788/0 (socket says 172.21.15.67:47788) 2024-05-30T16:45:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.831+0000 7f9259400640 1 -- 172.21.15.67:0/3974382246 learned_addr learned my addr 172.21.15.67:0/3974382246 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.831+0000 7f9259e00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0ce1f0 0x7f925c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.831+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2217375814 0 0) 0x7f925c0b3580 con 0x7f925c0ce5d0 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9240003520 con 0x7f925c0ce5d0 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1182809827 0 0) 0x7f9240003520 con 0x7f925c0ce5d0 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0ce1f0 msgr2=0x7f925c0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0ce1f0 0x7f925c0b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 >> v1:172.21.15.169:6789/0 conn(0x7f925c0b3d60 legacy=0x7f925c0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c15fa50 con 0x7f925c0ce5d0 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9244001670 con 0x7f925c0ce5d0 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 >> v1:172.21.15.67:6789/0 conn(0x7f925c0ce5d0 legacy=0x7f925c15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92400039e0 0x7f9240003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9240004800 0x7f9240004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92400052e0 0x7f924000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f925c15f770 con 0x7f92400039e0 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f924000c1b0 con 0x7f9240004800 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f924000c3b0 con 0x7f92400052e0 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.832+0000 7f9259e00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0ce1f0 0x7f925c0b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.833+0000 7f9259e00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92400039e0 0x7f9240003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.833+0000 7f925a800640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92400052e0 0x7f924000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.833+0000 7f9259400640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9240004800 0x7f9240004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.833+0000 7f9259e00640 1 -- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92400052e0 msgr2=0x7f924000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.833+0000 7f9259e00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92400052e0 0x7f924000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.833+0000 7f9259e00640 1 -- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9240004800 msgr2=0x7f9240004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.833+0000 7f9259e00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9240004800 0x7f9240004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.833+0000 7f9259e00640 1 -- 172.21.15.67:0/3974382246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9240003520 con 0x7f92400039e0 2024-05-30T16:45:01.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.834+0000 7f9259e00640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92400039e0 0x7f9240003e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b6410 tx=0x7f9240005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3242e674c34533ad server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:01.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.834+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c030050 con 0x7f92400039e0 2024-05-30T16:45:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.834+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924c003b90 con 0x7f92400039e0 2024-05-30T16:45:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.834+0000 7f9258a00640 1 -- 172.21.15.67:0/3974382246 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c009b50 con 0x7f92400039e0 2024-05-30T16:45:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.835+0000 7f925b200640 1 -- 172.21.15.67:0/3974382246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92400039e0 msgr2=0x7f9240003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.835+0000 7f925b200640 1 --2- 172.21.15.67:0/3974382246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92400039e0 0x7f9240003e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b6410 tx=0x7f9240005830 comp rx=0 tx=0).stop 2024-05-30T16:45:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.835+0000 7f925a800640 1 -- 172.21.15.67:0/3974382246 reap_dead start 2024-05-30T16:45:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.835+0000 7f925b200640 1 -- 172.21.15.67:0/3974382246 shutdown_connections 2024-05-30T16:45:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.835+0000 7f925b200640 1 -- 172.21.15.67:0/3974382246 >> 172.21.15.67:0/3974382246 conn(0x7f925c06b900 msgr2=0x7f925c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.835+0000 7f925b200640 1 -- 172.21.15.67:0/3974382246 shutdown_connections 2024-05-30T16:45:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.835+0000 7f925b200640 1 -- 172.21.15.67:0/3974382246 wait complete. 2024-05-30T16:45:01.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.837+0000 7f925b200640 1 Processor -- start 2024-05-30T16:45:01.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.837+0000 7f925b200640 1 -- start start 2024-05-30T16:45:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.838+0000 7f925b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92400039e0 0x7f925c0999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.838+0000 7f925b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0afb60 0x7f925c0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.838+0000 7f925b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925c0affe0 0x7f925c0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.838+0000 7f925b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0cd420 con 0x7f925c0affe0 2024-05-30T16:45:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.838+0000 7f925b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0cd2a0 con 0x7f92400039e0 2024-05-30T16:45:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.838+0000 7f925b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f925c0b4dd0 con 0x7f925c0afb60 2024-05-30T16:45:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f925a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925c0affe0 0x7f925c0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f925a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925c0affe0 0x7f925c0ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34114/0 (socket says 172.21.15.67:34114) 2024-05-30T16:45:01.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f925a800640 1 -- 172.21.15.67:0/388630062 learned_addr learned my addr 172.21.15.67:0/388630062 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:45:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f9259e00640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92400039e0 0x7f925c0999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f9259400640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0afb60 0x7f925c0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f925a800640 1 -- 172.21.15.67:0/388630062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0afb60 msgr2=0x7f925c0ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f925a800640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0afb60 0x7f925c0ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f925a800640 1 -- 172.21.15.67:0/388630062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92400039e0 msgr2=0x7f925c0999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f925a800640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92400039e0 0x7f925c0999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f925a800640 1 -- 172.21.15.67:0/388630062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c1442c0 con 0x7f925c0affe0 2024-05-30T16:45:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.839+0000 7f925a800640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925c0affe0 0x7f925c0ae6c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b3340 tx=0x7f925400e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.840+0000 7f9252000640 1 -- 172.21.15.67:0/388630062 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254019070 con 0x7f925c0affe0 2024-05-30T16:45:01.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.840+0000 7f9252000640 1 -- 172.21.15.67:0/388630062 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f925400ee70 con 0x7f925c0affe0 2024-05-30T16:45:01.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.840+0000 7f9252000640 1 -- 172.21.15.67:0/388630062 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540035f0 con 0x7f925c0affe0 2024-05-30T16:45:01.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.840+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c1445a0 con 0x7f925c0affe0 2024-05-30T16:45:01.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.840+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c144b60 con 0x7f925c0affe0 2024-05-30T16:45:01.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.841+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925c072de0 con 0x7f925c0affe0 2024-05-30T16:45:01.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.842+0000 7f9252000640 1 -- 172.21.15.67:0/388630062 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f925401c070 con 0x7f925c0affe0 2024-05-30T16:45:01.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.843+0000 7f9252000640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f923806c8f0 0x7f923806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:01.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.843+0000 7f9252000640 1 -- 172.21.15.67:0/388630062 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f925408ae90 con 0x7f925c0affe0 2024-05-30T16:45:01.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.843+0000 7f9259e00640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f923806c8f0 0x7f923806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:01.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.844+0000 7f9259e00640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f923806c8f0 0x7f923806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f924c035000 tx=0x7f924c00d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:01.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:01.848+0000 7f9252000640 1 -- 172.21.15.67:0/388630062 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f925c072de0 con 0x7f925c0affe0 2024-05-30T16:45:02.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.065+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f925c0689e0 con 0x7f925c0affe0 2024-05-30T16:45:02.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.066+0000 7f9252000640 1 -- 172.21.15.67:0/388630062 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f925408c020 con 0x7f925c0affe0 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f923806c8f0 msgr2=0x7f923806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f923806c8f0 0x7f923806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f924c035000 tx=0x7f924c00d040 comp rx=0 tx=0).stop 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925c0affe0 msgr2=0x7f925c0ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925c0affe0 0x7f925c0ae6c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b3340 tx=0x7f925400e670 comp rx=0 tx=0).stop 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 shutdown_connections 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f923806c8f0 0x7f923806edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f925c0affe0 0x7f925c0ae6c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f925c0afb60 0x7f925c0ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 --2- 172.21.15.67:0/388630062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92400039e0 0x7f925c0999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 >> 172.21.15.67:0/388630062 conn(0x7f925c06b900 msgr2=0x7f925c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:02.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 shutdown_connections 2024-05-30T16:45:02.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.069+0000 7f925b200640 1 -- 172.21.15.67:0/388630062 wait complete. 2024-05-30T16:45:02.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-30T16:45:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.303+0000 7f3c22800640 1 Processor -- start 2024-05-30T16:45:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.304+0000 7f3c22800640 1 -- start start 2024-05-30T16:45:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.305+0000 7f3c22800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 0x7f3c1c1608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.305+0000 7f3c22800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c1c06ef00 0x7f3c1c0d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.305+0000 7f3c22800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3c1c0b4790 0x7f3c1c0d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.305+0000 7f3c22800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f3c1c15c7d0 con 0x7f3c1c06ef00 2024-05-30T16:45:02.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.305+0000 7f3c22800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3c1c0ad9a0 con 0x7f3c1c0b4790 2024-05-30T16:45:02.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.305+0000 7f3c22800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c160e10 con 0x7f3c1c15c3f0 2024-05-30T16:45:02.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.306+0000 7f3c20a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c1c06ef00 0x7f3c1c0d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.306+0000 7f3c20a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c1c06ef00 0x7f3c1c0d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34124/0 (socket says 172.21.15.67:34124) 2024-05-30T16:45:02.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.306+0000 7f3c20a00640 1 -- 172.21.15.67:0/3214307256 learned_addr learned my addr 172.21.15.67:0/3214307256 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:45:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.306+0000 7f3c21400640 1 --2- 172.21.15.67:0/3214307256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 0x7f3c1c1608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.306+0000 7f3c20a00640 1 -- 172.21.15.67:0/3214307256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 msgr2=0x7f3c1c1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.306+0000 7f3c20a00640 1 --2- 172.21.15.67:0/3214307256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 0x7f3c1c1608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.306+0000 7f3c20a00640 1 -- 172.21.15.67:0/3214307256 >> v2:172.21.15.169:3300/0 conn(0x7f3c1c0b4790 msgr2=0x7f3c1c0d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.306+0000 7f3c20a00640 1 --2- 172.21.15.67:0/3214307256 >> v2:172.21.15.169:3300/0 conn(0x7f3c1c0b4790 0x7f3c1c0d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.306+0000 7f3c20a00640 1 -- 172.21.15.67:0/3214307256 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c0ad3e0 con 0x7f3c1c06ef00 2024-05-30T16:45:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.307+0000 7f3c20a00640 1 --2- 172.21.15.67:0/3214307256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c06ef00 0x7f3c1c0d0320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0aebf0 tx=0x7f3c0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3806ea23bbd0d72d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.308+0000 7f3c1be00640 1 -- 172.21.15.67:0/3214307256 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08037070 con 0x7f3c1c06ef00 2024-05-30T16:45:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.308+0000 7f3c1be00640 1 -- 172.21.15.67:0/3214307256 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c08032440 con 0x7f3c1c06ef00 2024-05-30T16:45:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.308+0000 7f3c1be00640 1 -- 172.21.15.67:0/3214307256 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08032810 con 0x7f3c1c06ef00 2024-05-30T16:45:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.309+0000 7f3c22800640 1 -- 172.21.15.67:0/3214307256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c06ef00 msgr2=0x7f3c1c0d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.309+0000 7f3c22800640 1 --2- 172.21.15.67:0/3214307256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c06ef00 0x7f3c1c0d0320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0aebf0 tx=0x7f3c0802a1e0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.309+0000 7f3c22800640 1 -- 172.21.15.67:0/3214307256 shutdown_connections 2024-05-30T16:45:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.309+0000 7f3c22800640 1 --2- 172.21.15.67:0/3214307256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c06ef00 0x7f3c1c0d0320 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.309+0000 7f3c22800640 1 --2- 172.21.15.67:0/3214307256 >> v2:172.21.15.169:3300/0 conn(0x7f3c1c0b4790 0x7f3c1c0d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.309+0000 7f3c22800640 1 --2- 172.21.15.67:0/3214307256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 0x7f3c1c1608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.309+0000 7f3c22800640 1 -- 172.21.15.67:0/3214307256 >> 172.21.15.67:0/3214307256 conn(0x7f3c1c06bae0 msgr2=0x7f3c1c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.309+0000 7f3c22800640 1 -- 172.21.15.67:0/3214307256 shutdown_connections 2024-05-30T16:45:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.309+0000 7f3c22800640 1 -- 172.21.15.67:0/3214307256 wait complete. 2024-05-30T16:45:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.310+0000 7f3c22800640 1 Processor -- start 2024-05-30T16:45:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.311+0000 7f3c22800640 1 -- start start 2024-05-30T16:45:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c22800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c1c06ef00 0x7f3c1c15f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c22800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c0b4790 0x7f3c1c13c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c22800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 0x7f3c1c13cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c22800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1603b0 con 0x7f3c1c0b4790 2024-05-30T16:45:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c22800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1600d0 con 0x7f3c1c06ef00 2024-05-30T16:45:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c22800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c0ad090 con 0x7f3c1c15c3f0 2024-05-30T16:45:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c20a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c0b4790 0x7f3c1c13c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c20a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c0b4790 0x7f3c1c13c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34138/0 (socket says 172.21.15.67:34138) 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c20a00640 1 -- 172.21.15.67:0/3680847095 learned_addr learned my addr 172.21.15.67:0/3680847095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c21400640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c1c06ef00 0x7f3c1c15f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c21e00640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 0x7f3c1c13cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c20a00640 1 -- 172.21.15.67:0/3680847095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 msgr2=0x7f3c1c13cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c20a00640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 0x7f3c1c13cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c20a00640 1 -- 172.21.15.67:0/3680847095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c1c06ef00 msgr2=0x7f3c1c15f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c20a00640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c1c06ef00 0x7f3c1c15f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c20a00640 1 -- 172.21.15.67:0/3680847095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c1516b0 con 0x7f3c1c0b4790 2024-05-30T16:45:02.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.312+0000 7f3c21400640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c1c06ef00 0x7f3c1c15f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:45:02.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.313+0000 7f3c20a00640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c0b4790 0x7f3c1c13c660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c08000c00 tx=0x7f3c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:02.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.313+0000 7f3c19600640 1 -- 172.21.15.67:0/3680847095 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08041020 con 0x7f3c1c0b4790 2024-05-30T16:45:02.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.313+0000 7f3c19600640 1 -- 172.21.15.67:0/3680847095 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c08030970 con 0x7f3c1c0b4790 2024-05-30T16:45:02.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.313+0000 7f3c19600640 1 -- 172.21.15.67:0/3680847095 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08037040 con 0x7f3c1c0b4790 2024-05-30T16:45:02.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.313+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c151990 con 0x7f3c1c0b4790 2024-05-30T16:45:02.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.313+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c151f50 con 0x7f3c1c0b4790 2024-05-30T16:45:02.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.315+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c0689e0 con 0x7f3c1c0b4790 2024-05-30T16:45:02.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.316+0000 7f3c19600640 1 -- 172.21.15.67:0/3680847095 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c08044020 con 0x7f3c1c0b4790 2024-05-30T16:45:02.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.317+0000 7f3c19600640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.317+0000 7f3c19600640 1 -- 172.21.15.67:0/3680847095 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3c080ac660 con 0x7f3c1c0b4790 2024-05-30T16:45:02.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.317+0000 7f3c21400640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.318+0000 7f3c21400640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000b820 tx=0x7f3c10008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:02.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.321+0000 7f3c19600640 1 -- 172.21.15.67:0/3680847095 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c0804b030 con 0x7f3c1c0b4790 2024-05-30T16:45:02.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.552+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f3c1c166b80 con 0x7f3bfc06c8f0 2024-05-30T16:45:02.562 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:45:02.561+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:45:02.562 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:45:02.561+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:45:02.562 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:45:02.561+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:45:02.562 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:45:02.561+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:45:02.563 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:45:02.561+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:45:02.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.651+0000 7f3c19600640 1 -- 172.21.15.67:0/3680847095 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3c1c166b80 con 0x7f3bfc06c8f0 2024-05-30T16:45:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bfc06c8f0 msgr2=0x7f3bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000b820 tx=0x7f3c10008040 comp rx=0 tx=0).stop 2024-05-30T16:45:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c0b4790 msgr2=0x7f3c1c13c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c0b4790 0x7f3c1c13c660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c08000c00 tx=0x7f3c080046c0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 shutdown_connections 2024-05-30T16:45:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1c15c3f0 0x7f3c1c13cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1c0b4790 0x7f3c1c13c660 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 --2- 172.21.15.67:0/3680847095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c1c06ef00 0x7f3c1c15f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 >> 172.21.15.67:0/3680847095 conn(0x7f3c1c06bae0 msgr2=0x7f3c1c073030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:02.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 shutdown_connections 2024-05-30T16:45:02.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.654+0000 7f3c22800640 1 -- 172.21.15.67:0/3680847095 wait complete. 2024-05-30T16:45:02.675 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:45:02.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.876+0000 7fba36a00640 1 Processor -- start 2024-05-30T16:45:02.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.876+0000 7fba36a00640 1 -- start start 2024-05-30T16:45:02.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.878+0000 7fba36a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fba300cd2a0 0x7fba300b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.878+0000 7fba36a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba3015cb60 0x7fba3015cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.878+0000 7fba36a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba3015d900 con 0x7fba30153510 2024-05-30T16:45:02.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.878+0000 7fba36a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fba301538f0 con 0x7fba300cd2a0 2024-05-30T16:45:02.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.878+0000 7fba36a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fba300b48c0 con 0x7fba3015cb60 2024-05-30T16:45:02.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.878+0000 7fba34c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fba300cd2a0 0x7fba300b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.878+0000 7fba34c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fba300cd2a0 0x7fba300b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34168/0 (socket says 172.21.15.67:34168) 2024-05-30T16:45:02.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.878+0000 7fba34c00640 1 -- 172.21.15.67:0/2853795781 learned_addr learned my addr 172.21.15.67:0/2853795781 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:45:02.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.878+0000 7fba36000640 1 --2- 172.21.15.67:0/2853795781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba3015cb60 0x7fba3015cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.879+0000 7fba34c00640 1 -- 172.21.15.67:0/2853795781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba3015cb60 msgr2=0x7fba3015cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:02.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.879+0000 7fba34c00640 1 --2- 172.21.15.67:0/2853795781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba3015cb60 0x7fba3015cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.879+0000 7fba34c00640 1 -- 172.21.15.67:0/2853795781 >> v1:172.21.15.169:6789/0 conn(0x7fba30153510 legacy=0x7fba300ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:45:02.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.879+0000 7fba34c00640 1 -- 172.21.15.67:0/2853795781 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba300b3f90 con 0x7fba300cd2a0 2024-05-30T16:45:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.880+0000 7fba34c00640 1 --2- 172.21.15.67:0/2853795781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 0x7fba300b4d50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009870 tx=0x7fba1c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdccd076abe687d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.880+0000 7fba2fe00640 1 -- 172.21.15.67:0/2853795781 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c037070 con 0x7fba300cd2a0 2024-05-30T16:45:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.880+0000 7fba2fe00640 1 -- 172.21.15.67:0/2853795781 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1c032440 con 0x7fba300cd2a0 2024-05-30T16:45:02.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.880+0000 7fba2fe00640 1 -- 172.21.15.67:0/2853795781 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c032810 con 0x7fba300cd2a0 2024-05-30T16:45:02.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.881+0000 7fba36a00640 1 -- 172.21.15.67:0/2853795781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 msgr2=0x7fba300b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.881+0000 7fba36a00640 1 --2- 172.21.15.67:0/2853795781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 0x7fba300b4d50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009870 tx=0x7fba1c02a180 comp rx=0 tx=0).stop 2024-05-30T16:45:02.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.881+0000 7fba36a00640 1 -- 172.21.15.67:0/2853795781 shutdown_connections 2024-05-30T16:45:02.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.881+0000 7fba36a00640 1 --2- 172.21.15.67:0/2853795781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba3015cb60 0x7fba3015cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.881+0000 7fba36a00640 1 --2- 172.21.15.67:0/2853795781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 0x7fba300b4d50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.881+0000 7fba36a00640 1 -- 172.21.15.67:0/2853795781 >> 172.21.15.67:0/2853795781 conn(0x7fba3006b900 msgr2=0x7fba3006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:02.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.881+0000 7fba36a00640 1 -- 172.21.15.67:0/2853795781 shutdown_connections 2024-05-30T16:45:02.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.881+0000 7fba36a00640 1 -- 172.21.15.67:0/2853795781 wait complete. 2024-05-30T16:45:02.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.883+0000 7fba36a00640 1 Processor -- start 2024-05-30T16:45:02.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.883+0000 7fba36a00640 1 -- start start 2024-05-30T16:45:02.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.884+0000 7fba36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 0x7fba301512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.884+0000 7fba36a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba30153510 0x7fba30072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.884+0000 7fba36a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba3015cb60 0x7fba30154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.884+0000 7fba36a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fba3015d4f0 con 0x7fba300cd2a0 2024-05-30T16:45:02.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.884+0000 7fba36a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba300b4550 con 0x7fba30153510 2024-05-30T16:45:02.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.884+0000 7fba36a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fba3015d670 con 0x7fba3015cb60 2024-05-30T16:45:02.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.884+0000 7fba35600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 0x7fba301512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.884+0000 7fba35600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 0x7fba301512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34176/0 (socket says 172.21.15.67:34176) 2024-05-30T16:45:02.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.884+0000 7fba35600640 1 -- 172.21.15.67:0/2326768969 learned_addr learned my addr 172.21.15.67:0/2326768969 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba35600640 1 -- 172.21.15.67:0/2326768969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba3015cb60 msgr2=0x7fba30154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba35600640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba3015cb60 0x7fba30154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba34c00640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba30153510 0x7fba30072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba35600640 1 -- 172.21.15.67:0/2326768969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba30153510 msgr2=0x7fba30072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba35600640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba30153510 0x7fba30072350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba35600640 1 -- 172.21.15.67:0/2326768969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba30151810 con 0x7fba300cd2a0 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba35600640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 0x7fba301512d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c430 tx=0x7fba2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba2d600640 1 -- 172.21.15.67:0/2326768969 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24011790 con 0x7fba300cd2a0 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba2d600640 1 -- 172.21.15.67:0/2326768969 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba24017070 con 0x7fba300cd2a0 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba300cea40 con 0x7fba300cd2a0 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.885+0000 7fba2d600640 1 -- 172.21.15.67:0/2326768969 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24011bb0 con 0x7fba300cd2a0 2024-05-30T16:45:02.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.886+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba300cf000 con 0x7fba300cd2a0 2024-05-30T16:45:02.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.887+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba300ccb10 con 0x7fba300cd2a0 2024-05-30T16:45:02.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.888+0000 7fba2d600640 1 -- 172.21.15.67:0/2326768969 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba2401c460 con 0x7fba300cd2a0 2024-05-30T16:45:02.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.888+0000 7fba2d600640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fba1006c920 0x7fba1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:45:02.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.889+0000 7fba2d600640 1 -- 172.21.15.67:0/2326768969 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fba2408b990 con 0x7fba300cd2a0 2024-05-30T16:45:02.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.889+0000 7fba34c00640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fba1006c920 0x7fba1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:45:02.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.889+0000 7fba34c00640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fba1006c920 0x7fba1006ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fba1c034000 tx=0x7fba1c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:45:02.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:02.893+0000 7fba2d600640 1 -- 172.21.15.67:0/2326768969 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba24059db0 con 0x7fba300cd2a0 2024-05-30T16:45:03.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.129+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fba300689e0 con 0x7fba1006c920 2024-05-30T16:45:03.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.140+0000 7fba2d600640 1 -- 172.21.15.67:0/2326768969 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fba300689e0 con 0x7fba1006c920 2024-05-30T16:45:03.141 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb 2024-05-30T16:45:03.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fba1006c920 msgr2=0x7fba1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:03.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fba1006c920 0x7fba1006ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fba1c034000 tx=0x7fba1c02e000 comp rx=0 tx=0).stop 2024-05-30T16:45:03.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 msgr2=0x7fba301512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:45:03.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 0x7fba301512d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c430 tx=0x7fba2400c8f0 comp rx=0 tx=0).stop 2024-05-30T16:45:03.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 shutdown_connections 2024-05-30T16:45:03.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fba1006c920 0x7fba1006ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:03.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba3015cb60 0x7fba30154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:03.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba30153510 0x7fba30072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:03.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 --2- 172.21.15.67:0/2326768969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba300cd2a0 0x7fba301512d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:45:03.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 >> 172.21.15.67:0/2326768969 conn(0x7fba3006b900 msgr2=0x7fba3006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:45:03.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.142+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 shutdown_connections 2024-05-30T16:45:03.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:45:03.143+0000 7fba36a00640 1 -- 172.21.15.67:0/2326768969 wait complete. 2024-05-30T16:45:03.163 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb 2024-05-30T16:45:03.164 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:45:03.164 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:45:03.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:45:03.213 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:45:08.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:45:08.515 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:45:19.819 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:45:19.819 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:45:19.819 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.4757 s, 65.2 MB/s 2024-05-30T16:45:19.820 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:45:19.820 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:45:19.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:45:19.876 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:45:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:45:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:45:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.9193 s, 72.0 MB/s 2024-05-30T16:45:34.860 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:45:34.860 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:45:34.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:45:34.914 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:45:38.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:45:38.941 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:45:48.527 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:45:48.528 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:45:48.528 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.5503 s, 79.2 MB/s 2024-05-30T16:45:48.529 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:45:48.530 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:45:48.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:45:48.585 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:46:02.977 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:46:02.978 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:46:02.978 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.3251 s, 75.0 MB/s 2024-05-30T16:46:02.980 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:46:02.980 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:46:03.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:46:03.037 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:46:09.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:46:09.075 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:46:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:46:17.092 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:46:17.092 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.9866 s, 76.8 MB/s 2024-05-30T16:46:17.094 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:46:17.094 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:46:17.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:46:17.151 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:46:31.415 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:46:31.415 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:46:31.415 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.1971 s, 75.6 MB/s 2024-05-30T16:46:31.417 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:46:31.418 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:46:31.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:46:31.472 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:46:39.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:46:39.209 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:46:45.154 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:46:45.154 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:46:45.155 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.6156 s, 78.9 MB/s 2024-05-30T16:46:45.156 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:46:45.156 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:46:45.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:46:45.213 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:46:59.376 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:46:59.376 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:46:59.376 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.0996 s, 76.2 MB/s 2024-05-30T16:46:59.378 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:46:59.378 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:46:59.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:46:59.439 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:47:09.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:47:09.347 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:47:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:47:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:47:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.7494 s, 78.1 MB/s 2024-05-30T16:47:13.255 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:47:13.255 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:47:13.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:47:13.314 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:47:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:47:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:47:27.009 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.6251 s, 78.8 MB/s 2024-05-30T16:47:27.010 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-30T16:47:27.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.228+0000 7fda40c00640 1 Processor -- start 2024-05-30T16:47:27.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.228+0000 7fda40c00640 1 -- start start 2024-05-30T16:47:27.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.229+0000 7fda40c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c15e5a0 0x7fda3c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.229+0000 7fda40c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fda3c0cfb40 0x7fda3c0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.230+0000 7fda40c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda3c0d3500 con 0x7fda3c0cffa0 2024-05-30T16:47:27.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.230+0000 7fda40c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fda3c0d2b30 con 0x7fda3c0cfb40 2024-05-30T16:47:27.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.230+0000 7fda40c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c0d2200 con 0x7fda3c15e5a0 2024-05-30T16:47:27.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.230+0000 7fda3aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fda3c0cfb40 0x7fda3c0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.230+0000 7fda3aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fda3c0cfb40 0x7fda3c0d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50482/0 (socket says 172.21.15.67:50482) 2024-05-30T16:47:27.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.230+0000 7fda3aa00640 1 -- 172.21.15.67:0/2655496298 learned_addr learned my addr 172.21.15.67:0/2655496298 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:27.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.230+0000 7fda3be00640 1 --1- 172.21.15.67:0/2655496298 >> v1:172.21.15.67:6789/0 conn(0x7fda3c0cffa0 0x7fda3c06c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57872/0 (socket says 172.21.15.67:57872) 2024-05-30T16:47:27.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.230+0000 7fda3b400640 1 --2- 172.21.15.67:0/2655496298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c15e5a0 0x7fda3c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.231+0000 7fda3aa00640 1 -- 172.21.15.67:0/2655496298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c15e5a0 msgr2=0x7fda3c0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:27.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.231+0000 7fda3aa00640 1 --2- 172.21.15.67:0/2655496298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c15e5a0 0x7fda3c0d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.231+0000 7fda3aa00640 1 -- 172.21.15.67:0/2655496298 >> v1:172.21.15.67:6789/0 conn(0x7fda3c0cffa0 legacy=0x7fda3c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.231+0000 7fda3aa00640 1 -- 172.21.15.67:0/2655496298 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c0d24e0 con 0x7fda3c0cfb40 2024-05-30T16:47:27.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.231+0000 7fda3a000640 1 -- 172.21.15.67:0/2655496298 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3354166964 0 0) 0x7fda3c0d3500 con 0x7fda3c0cffa0 2024-05-30T16:47:27.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.232+0000 7fda3aa00640 1 --2- 172.21.15.67:0/2655496298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 0x7fda3c0d0d10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda28009870 tx=0x7fda2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bf8c7924433c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.232+0000 7fda3a000640 1 -- 172.21.15.67:0/2655496298 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28037070 con 0x7fda3c0cfb40 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.232+0000 7fda3a000640 1 -- 172.21.15.67:0/2655496298 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda28032440 con 0x7fda3c0cfb40 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.232+0000 7fda3a000640 1 -- 172.21.15.67:0/2655496298 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28032810 con 0x7fda3c0cfb40 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.233+0000 7fda40c00640 1 -- 172.21.15.67:0/2655496298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 msgr2=0x7fda3c0d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.233+0000 7fda40c00640 1 --2- 172.21.15.67:0/2655496298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 0x7fda3c0d0d10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda28009870 tx=0x7fda2802a270 comp rx=0 tx=0).stop 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.233+0000 7fda40c00640 1 -- 172.21.15.67:0/2655496298 shutdown_connections 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.233+0000 7fda40c00640 1 --2- 172.21.15.67:0/2655496298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 0x7fda3c0d0d10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.233+0000 7fda40c00640 1 --2- 172.21.15.67:0/2655496298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c15e5a0 0x7fda3c0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.233+0000 7fda40c00640 1 -- 172.21.15.67:0/2655496298 >> 172.21.15.67:0/2655496298 conn(0x7fda3c08bfc0 msgr2=0x7fda3c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.233+0000 7fda40c00640 1 -- 172.21.15.67:0/2655496298 shutdown_connections 2024-05-30T16:47:27.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.233+0000 7fda40c00640 1 -- 172.21.15.67:0/2655496298 wait complete. 2024-05-30T16:47:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.235+0000 7fda40c00640 1 Processor -- start 2024-05-30T16:47:27.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.235+0000 7fda40c00640 1 -- start start 2024-05-30T16:47:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda40c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 0x7fda3c154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda40c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0cffa0 0x7fda3c0c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda40c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c15e5a0 0x7fda3c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda40c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c15eb00 con 0x7fda3c0cfb40 2024-05-30T16:47:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda40c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c15e980 con 0x7fda3c15e5a0 2024-05-30T16:47:27.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda40c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c0d5220 con 0x7fda3c0cffa0 2024-05-30T16:47:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda3b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 0x7fda3c154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda3b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 0x7fda3c154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50488/0 (socket says 172.21.15.67:50488) 2024-05-30T16:47:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda3b400640 1 -- 172.21.15.67:0/1082177436 learned_addr learned my addr 172.21.15.67:0/1082177436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda3b400640 1 -- 172.21.15.67:0/1082177436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0cffa0 msgr2=0x7fda3c0c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda3b400640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0cffa0 0x7fda3c0c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda3b400640 1 -- 172.21.15.67:0/1082177436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c15e5a0 msgr2=0x7fda3c0ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda3b400640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c15e5a0 0x7fda3c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.236+0000 7fda3b400640 1 -- 172.21.15.67:0/1082177436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c0b42b0 con 0x7fda3c0cfb40 2024-05-30T16:47:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.237+0000 7fda3b400640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 0x7fda3c154ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00d5f0 tx=0x7fda2c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.237+0000 7fda33400640 1 -- 172.21.15.67:0/1082177436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0125f0 con 0x7fda3c0cfb40 2024-05-30T16:47:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.237+0000 7fda33400640 1 -- 172.21.15.67:0/1082177436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c019070 con 0x7fda3c0cfb40 2024-05-30T16:47:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.237+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c0b4590 con 0x7fda3c0cfb40 2024-05-30T16:47:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.237+0000 7fda33400640 1 -- 172.21.15.67:0/1082177436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0129d0 con 0x7fda3c0cfb40 2024-05-30T16:47:27.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.237+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c0b4b50 con 0x7fda3c0cfb40 2024-05-30T16:47:27.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.238+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c0689e0 con 0x7fda3c0cfb40 2024-05-30T16:47:27.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.239+0000 7fda33400640 1 -- 172.21.15.67:0/1082177436 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 7) ==== 91040+0+0 (secure 0 0 0) 0x7fda2c012b70 con 0x7fda3c0cfb40 2024-05-30T16:47:27.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.240+0000 7fda33400640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda1c06c8f0 0x7fda1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.240+0000 7fda33400640 1 -- 172.21.15.67:0/1082177436 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fda2c08bff0 con 0x7fda3c0cfb40 2024-05-30T16:47:27.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.240+0000 7fda3aa00640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda1c06c8f0 0x7fda1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.241+0000 7fda3aa00640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda1c06c8f0 0x7fda1c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda28034000 tx=0x7fda2802c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:27.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.247+0000 7fda33400640 1 -- 172.21.15.67:0/1082177436 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda2c05a380 con 0x7fda3c0cfb40 2024-05-30T16:47:27.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.483+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fda3c0d1d30 con 0x7fda1c06c8f0 2024-05-30T16:47:27.493 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:27.493+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:27.493 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:27.493+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:27.493 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:27.493+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:27.494 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:27.493+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:27.494 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:27.493+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.541+0000 7fda33400640 1 -- 172.21.15.67:0/1082177436 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda3c0d1d30 con 0x7fda1c06c8f0 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda1c06c8f0 msgr2=0x7fda1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda1c06c8f0 0x7fda1c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda28034000 tx=0x7fda2802c000 comp rx=0 tx=0).stop 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 msgr2=0x7fda3c154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 0x7fda3c154ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00d5f0 tx=0x7fda2c00dab0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 shutdown_connections 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda1c06c8f0 0x7fda1c06edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c15e5a0 0x7fda3c0ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0cffa0 0x7fda3c0c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 --2- 172.21.15.67:0/1082177436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda3c0cfb40 0x7fda3c154ce0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 >> 172.21.15.67:0/1082177436 conn(0x7fda3c08bfc0 msgr2=0x7fda3c0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 shutdown_connections 2024-05-30T16:47:27.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.544+0000 7fda40c00640 1 -- 172.21.15.67:0/1082177436 wait complete. 2024-05-30T16:47:27.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:47:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.762+0000 7f1ecd800640 1 Processor -- start 2024-05-30T16:47:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.762+0000 7f1ecd800640 1 -- start start 2024-05-30T16:47:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.764+0000 7f1ecd800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1ec80ce5d0 0x7f1ec80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.764+0000 7f1ecd800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ec80b3d60 0x7f1ec80b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.764+0000 7f1ecd800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec80b3580 con 0x7f1ec80ce1f0 2024-05-30T16:47:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.764+0000 7f1ecd800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec815f490 con 0x7f1ec80b3d60 2024-05-30T16:47:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.764+0000 7f1ecd800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec815f770 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.764+0000 7f1ec7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1ec80ce1f0 0x7f1ec815ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57874/0 (socket says 172.21.15.67:57874) 2024-05-30T16:47:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.764+0000 7f1ec7e00640 1 -- 172.21.15.67:0/1030424343 learned_addr learned my addr 172.21.15.67:0/1030424343 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:47:27.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.764+0000 7f1ecce00640 1 --2- 172.21.15.67:0/1030424343 >> v2:172.21.15.67:3300/0 conn(0x7f1ec80b3d60 0x7f1ec80b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.764+0000 7f1ebfe00640 1 --2- 172.21.15.67:0/1030424343 >> v2:172.21.15.169:3300/0 conn(0x7f1ec80ce5d0 0x7f1ec80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.765+0000 7f1ec7400640 1 -- 172.21.15.67:0/1030424343 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358263318 0 0) 0x7f1ec80b3580 con 0x7f1ec80ce1f0 2024-05-30T16:47:27.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.765+0000 7f1ec7400640 1 -- 172.21.15.67:0/1030424343 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ea8003520 con 0x7f1ec80ce1f0 2024-05-30T16:47:27.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.765+0000 7f1ecce00640 1 -- 172.21.15.67:0/1030424343 >> v2:172.21.15.169:3300/0 conn(0x7f1ec80ce5d0 msgr2=0x7f1ec80b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:27.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.765+0000 7f1ecce00640 1 --2- 172.21.15.67:0/1030424343 >> v2:172.21.15.169:3300/0 conn(0x7f1ec80ce5d0 0x7f1ec80b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.765+0000 7f1ecce00640 1 -- 172.21.15.67:0/1030424343 >> v1:172.21.15.67:6789/0 conn(0x7f1ec80ce1f0 legacy=0x7f1ec815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.765+0000 7f1ecce00640 1 -- 172.21.15.67:0/1030424343 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec815fa50 con 0x7f1ec80b3d60 2024-05-30T16:47:27.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.766+0000 7f1ecce00640 1 --2- 172.21.15.67:0/1030424343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80b3d60 0x7f1ec80b4210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8070060 tx=0x7f1ec002a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4725c820e69000b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:27.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.766+0000 7f1ec7400640 1 -- 172.21.15.67:0/1030424343 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0007740 con 0x7f1ec80b3d60 2024-05-30T16:47:27.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.766+0000 7f1ec7400640 1 -- 172.21.15.67:0/1030424343 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec00078e0 con 0x7f1ec80b3d60 2024-05-30T16:47:27.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.766+0000 7f1ec7400640 1 -- 172.21.15.67:0/1030424343 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0007c90 con 0x7f1ec80b3d60 2024-05-30T16:47:27.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.767+0000 7f1ecd800640 1 -- 172.21.15.67:0/1030424343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80b3d60 msgr2=0x7f1ec80b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.767+0000 7f1ecd800640 1 --2- 172.21.15.67:0/1030424343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80b3d60 0x7f1ec80b4210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8070060 tx=0x7f1ec002a550 comp rx=0 tx=0).stop 2024-05-30T16:47:27.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.767+0000 7f1ecd800640 1 -- 172.21.15.67:0/1030424343 shutdown_connections 2024-05-30T16:47:27.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.767+0000 7f1ecd800640 1 --2- 172.21.15.67:0/1030424343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80b3d60 0x7f1ec80b4210 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.767+0000 7f1ecd800640 1 --2- 172.21.15.67:0/1030424343 >> v2:172.21.15.169:3300/0 conn(0x7f1ec80ce5d0 0x7f1ec80b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.767+0000 7f1ecd800640 1 -- 172.21.15.67:0/1030424343 >> 172.21.15.67:0/1030424343 conn(0x7f1ec806b900 msgr2=0x7f1ec806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.767+0000 7f1ecd800640 1 -- 172.21.15.67:0/1030424343 shutdown_connections 2024-05-30T16:47:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.767+0000 7f1ecd800640 1 -- 172.21.15.67:0/1030424343 wait complete. 2024-05-30T16:47:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.769+0000 7f1ecd800640 1 Processor -- start 2024-05-30T16:47:27.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.769+0000 7f1ecd800640 1 -- start start 2024-05-30T16:47:27.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.770+0000 7f1ecd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec80b3d60 0x7f1ec81543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.770+0000 7f1ecd800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec80ce1f0 0x7f1ec806f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.770+0000 7f1ecd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80ce5d0 0x7f1ec8161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.770+0000 7f1ecd800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80ccd10 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.770+0000 7f1ecd800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80ccb90 con 0x7f1ec80ce1f0 2024-05-30T16:47:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.770+0000 7f1ecd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec80cce90 con 0x7f1ec80b3d60 2024-05-30T16:47:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.770+0000 7f1ecce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80ce5d0 0x7f1ec8161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.770+0000 7f1ecce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80ce5d0 0x7f1ec8161800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50536/0 (socket says 172.21.15.67:50536) 2024-05-30T16:47:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.770+0000 7f1ecce00640 1 -- 172.21.15.67:0/287907154 learned_addr learned my addr 172.21.15.67:0/287907154 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.771+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec80b3d60 0x7f1ec81543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.771+0000 7f1ebfe00640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec80ce1f0 0x7f1ec806f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.771+0000 7f1ecce00640 1 -- 172.21.15.67:0/287907154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec80b3d60 msgr2=0x7f1ec81543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.771+0000 7f1ecce00640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec80b3d60 0x7f1ec81543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.771+0000 7f1ecce00640 1 -- 172.21.15.67:0/287907154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec80ce1f0 msgr2=0x7f1ec806f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.771+0000 7f1ecce00640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec80ce1f0 0x7f1ec806f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.771+0000 7f1ecce00640 1 -- 172.21.15.67:0/287907154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec806fe60 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.771+0000 7f1ecce00640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80ce5d0 0x7f1ec8161800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009e80 tx=0x7f1ec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.771+0000 7f1ec4c00640 1 -- 172.21.15.67:0/287907154 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0009450 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.772+0000 7f1ec4c00640 1 -- 172.21.15.67:0/287907154 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec0004240 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.772+0000 7f1ec4c00640 1 -- 172.21.15.67:0/287907154 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0004630 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.772+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81552e0 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.772+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec8155000 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.773+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec80689e0 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.774+0000 7f1ec4c00640 1 -- 172.21.15.67:0/287907154 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 7) ==== 91040+0+0 (secure 0 0 0) 0x7f1ec0039040 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.775+0000 7f1ec4c00640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ea406c920 0x7f1ea406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.775+0000 7f1ec4c00640 1 -- 172.21.15.67:0/287907154 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1ec00acc00 con 0x7f1ec80ce5d0 2024-05-30T16:47:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.775+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ea406c920 0x7f1ea406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.776+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ea406c920 0x7f1ea406ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8005880 tx=0x7f1eb8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:27.779+0000 7f1ec4c00640 1 -- 172.21.15.67:0/287907154 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ec0040030 con 0x7f1ec80ce5d0 2024-05-30T16:47:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.016+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f1ec80ca070 con 0x7f1ea406c920 2024-05-30T16:47:28.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.029+0000 7f1ec4c00640 1 -- 172.21.15.67:0/287907154 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f1ec80ca070 con 0x7f1ea406c920 2024-05-30T16:47:28.030 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb 2024-05-30T16:47:28.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ea406c920 msgr2=0x7f1ea406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:28.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ea406c920 0x7f1ea406ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8005880 tx=0x7f1eb8009040 comp rx=0 tx=0).stop 2024-05-30T16:47:28.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80ce5d0 msgr2=0x7f1ec8161800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:28.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80ce5d0 0x7f1ec8161800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009e80 tx=0x7f1ec0004a40 comp rx=0 tx=0).stop 2024-05-30T16:47:28.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 shutdown_connections 2024-05-30T16:47:28.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ea406c920 0x7f1ea406ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80ce5d0 0x7f1ec8161800 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec80ce1f0 0x7f1ec806f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 --2- 172.21.15.67:0/287907154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec80b3d60 0x7f1ec81543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 >> 172.21.15.67:0/287907154 conn(0x7f1ec806b900 msgr2=0x7f1ec806bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:28.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 shutdown_connections 2024-05-30T16:47:28.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.032+0000 7f1ecd800640 1 -- 172.21.15.67:0/287907154 wait complete. 2024-05-30T16:47:28.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:47:28.052 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb 2024-05-30T16:47:28.052 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:47:28.111 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/f1076073-1193-40ab-82de-02730bed6efb 2024-05-30T16:47:28.111 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="10737418240" 2024-05-30T16:47:28.112 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:47:28.113 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-05-30T16:47:28.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-30T16:47:28.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.329+0000 7f7faf600640 1 Processor -- start 2024-05-30T16:47:28.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.329+0000 7f7faf600640 1 -- start start 2024-05-30T16:47:28.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.331+0000 7f7faf600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7fa815b6f0 0x7f7fa806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:28.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.331+0000 7f7faf600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa80ade90 con 0x7f7fa80d0640 2024-05-30T16:47:28.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.331+0000 7f7faf600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa8071e70 con 0x7f7fa80725e0 2024-05-30T16:47:28.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.331+0000 7f7faf600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa8071c40 con 0x7f7fa815b6f0 2024-05-30T16:47:28.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.331+0000 7f7fa7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7fa815b6f0 0x7f7fa806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.331+0000 7f7fa7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7fa815b6f0 0x7f7fa806e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50548/0 (socket says 172.21.15.67:50548) 2024-05-30T16:47:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.331+0000 7f7fa7e00640 1 -- 172.21.15.67:0/2765061108 learned_addr learned my addr 172.21.15.67:0/2765061108 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.331+0000 7f7facc00640 1 --1- 172.21.15.67:0/2765061108 >> v1:172.21.15.67:6789/0 conn(0x7f7fa80d0640 0x7f7fa80d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57880/0 (socket says 172.21.15.67:57880) 2024-05-30T16:47:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.332+0000 7f7fa7e00640 1 -- 172.21.15.67:0/2765061108 >> v1:172.21.15.169:6789/0 conn(0x7f7fa80725e0 legacy=0x7f7fa80ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.332+0000 7f7fa7e00640 1 -- 172.21.15.67:0/2765061108 >> v1:172.21.15.67:6789/0 conn(0x7f7fa80d0640 legacy=0x7f7fa80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.332+0000 7f7fa7e00640 1 -- 172.21.15.67:0/2765061108 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa8072130 con 0x7f7fa815b6f0 2024-05-30T16:47:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.333+0000 7f7fa7e00640 1 --2- 172.21.15.67:0/2765061108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa815b6f0 0x7f7fa806e350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009870 tx=0x7f7f9402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bff5606c0f1486 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.333+0000 7f7fa7400640 1 -- 172.21.15.67:0/2765061108 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94037070 con 0x7f7fa815b6f0 2024-05-30T16:47:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.333+0000 7f7fa7400640 1 -- 172.21.15.67:0/2765061108 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f94032440 con 0x7f7fa815b6f0 2024-05-30T16:47:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.333+0000 7f7fa7400640 1 -- 172.21.15.67:0/2765061108 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94032810 con 0x7f7fa815b6f0 2024-05-30T16:47:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.334+0000 7f7faf600640 1 -- 172.21.15.67:0/2765061108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa815b6f0 msgr2=0x7f7fa806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.334+0000 7f7faf600640 1 --2- 172.21.15.67:0/2765061108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa815b6f0 0x7f7fa806e350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009870 tx=0x7f7f9402a2a0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.334+0000 7f7faf600640 1 -- 172.21.15.67:0/2765061108 shutdown_connections 2024-05-30T16:47:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.334+0000 7f7faf600640 1 --2- 172.21.15.67:0/2765061108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa815b6f0 0x7f7fa806e350 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.334+0000 7f7faf600640 1 -- 172.21.15.67:0/2765061108 >> 172.21.15.67:0/2765061108 conn(0x7f7fa806bd20 msgr2=0x7f7fa806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.334+0000 7f7faf600640 1 -- 172.21.15.67:0/2765061108 shutdown_connections 2024-05-30T16:47:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.334+0000 7f7faf600640 1 -- 172.21.15.67:0/2765061108 wait complete. 2024-05-30T16:47:28.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.336+0000 7f7faf600640 1 Processor -- start 2024-05-30T16:47:28.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.337+0000 7f7faf600640 1 -- start start 2024-05-30T16:47:28.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.337+0000 7f7faf600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa80725e0 0x7f7fa8099830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:28.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.337+0000 7f7faf600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa80d0640 0x7f7fa8152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:28.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.337+0000 7f7faf600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815b6f0 0x7f7fa8152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.337+0000 7f7faf600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa80aed60 con 0x7f7fa80d0640 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.337+0000 7f7faf600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa806e890 con 0x7f7fa80725e0 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.337+0000 7f7faf600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa80aeee0 con 0x7f7fa815b6f0 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa80d0640 0x7f7fa8152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa80d0640 0x7f7fa8152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50554/0 (socket says 172.21.15.67:50554) 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fa7e00640 1 -- 172.21.15.67:0/140049737 learned_addr learned my addr 172.21.15.67:0/140049737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fad600640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815b6f0 0x7f7fa8152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7facc00640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa80725e0 0x7f7fa8099830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fa7e00640 1 -- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815b6f0 msgr2=0x7f7fa8152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fa7e00640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815b6f0 0x7f7fa8152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fa7e00640 1 -- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa80725e0 msgr2=0x7f7fa8099830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fa7e00640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa80725e0 0x7f7fa8099830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fa7e00640 1 -- 172.21.15.67:0/140049737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa8156910 con 0x7f7fa80d0640 2024-05-30T16:47:28.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7fad600640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815b6f0 0x7f7fa8152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:47:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.338+0000 7f7facc00640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa80725e0 0x7f7fa8099830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:47:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.339+0000 7f7fa7e00640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa80d0640 0x7f7fa8152570 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7f9402a2e0 tx=0x7f7f940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.339+0000 7f7fa4c00640 1 -- 172.21.15.67:0/140049737 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94037040 con 0x7f7fa80d0640 2024-05-30T16:47:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.339+0000 7f7fa4c00640 1 -- 172.21.15.67:0/140049737 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f94042070 con 0x7f7fa80d0640 2024-05-30T16:47:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.339+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa8156bf0 con 0x7f7fa80d0640 2024-05-30T16:47:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.339+0000 7f7fa4c00640 1 -- 172.21.15.67:0/140049737 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94030bb0 con 0x7f7fa80d0640 2024-05-30T16:47:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.339+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa81571b0 con 0x7f7fa80d0640 2024-05-30T16:47:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.340+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa80689e0 con 0x7f7fa80d0640 2024-05-30T16:47:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.341+0000 7f7fa4c00640 1 -- 172.21.15.67:0/140049737 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 7) ==== 91040+0+0 (secure 0 0 0) 0x7f7f94047460 con 0x7f7fa80d0640 2024-05-30T16:47:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.343+0000 7f7fa4c00640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f8806c8f0 0x7f7f8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.343+0000 7f7fa4c00640 1 -- 172.21.15.67:0/140049737 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f7f940ac760 con 0x7f7fa80d0640 2024-05-30T16:47:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.343+0000 7f7facc00640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f8806c8f0 0x7f7f8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.344+0000 7f7facc00640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f8806c8f0 0x7f7f8806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c000f60 tx=0x7f7f9c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:28.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.347+0000 7f7fa4c00640 1 -- 172.21.15.67:0/140049737 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f9407ab80 con 0x7f7fa80d0640 2024-05-30T16:47:28.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.588+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f7fa806cc20 con 0x7f7f8806c8f0 2024-05-30T16:47:28.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.628+0000 7f7fa4c00640 1 -- 172.21.15.67:0/140049737 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 8) ==== 91091+0+0 (secure 0 0 0) 0x7f7f94076af0 con 0x7f7fa80d0640 2024-05-30T16:47:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.791+0000 7f7fa4c00640 1 -- 172.21.15.67:0/140049737 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7fa806cc20 con 0x7f7f8806c8f0 2024-05-30T16:47:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f8806c8f0 msgr2=0x7f7f8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f8806c8f0 0x7f7f8806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c000f60 tx=0x7f7f9c009040 comp rx=0 tx=0).stop 2024-05-30T16:47:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa80d0640 msgr2=0x7f7fa8152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa80d0640 0x7f7fa8152570 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7f9402a2e0 tx=0x7f7f940046c0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 shutdown_connections 2024-05-30T16:47:28.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7f8806c8f0 0x7f7f8806edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815b6f0 0x7f7fa8152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fa80d0640 0x7f7fa8152570 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 --2- 172.21.15.67:0/140049737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa80725e0 0x7f7fa8099830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:28.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 >> 172.21.15.67:0/140049737 conn(0x7f7fa806bd20 msgr2=0x7f7fa8074b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:28.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 shutdown_connections 2024-05-30T16:47:28.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:28.796+0000 7f7faf600640 1 -- 172.21.15.67:0/140049737 wait complete. 2024-05-30T16:47:28.799 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.798+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:28.799 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.798+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:28.799 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.798+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:28.799 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.798+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:28.799 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.798+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:28.822 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:47:28.834 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.833+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:28.834 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.833+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:28.834 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.833+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:28.834 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.833+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:28.834 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:47:28.833+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:47:29.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.006+0000 7fdbdec00640 1 Processor -- start 2024-05-30T16:47:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.006+0000 7fdbdec00640 1 -- start start 2024-05-30T16:47:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.007+0000 7fdbdec00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdbd80ccc30 0x7fdbd806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.007+0000 7fdbdec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdbd80d0460 0x7fdbd80d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.007+0000 7fdbdec00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd80cd1a0 con 0x7fdbd815c6b0 2024-05-30T16:47:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.007+0000 7fdbdec00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd80d0df0 con 0x7fdbd80ccc30 2024-05-30T16:47:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.007+0000 7fdbdec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd80cd480 con 0x7fdbd80d0460 2024-05-30T16:47:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbdce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdbd80ccc30 0x7fdbd806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbdce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdbd80ccc30 0x7fdbd806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50596/0 (socket says 172.21.15.67:50596) 2024-05-30T16:47:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbdd800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdbd815c6b0 0x7fdbd80b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57896/0 (socket says 172.21.15.67:57896) 2024-05-30T16:47:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbdce00640 1 -- 172.21.15.67:0/606622940 learned_addr learned my addr 172.21.15.67:0/606622940 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbde200640 1 --2- 172.21.15.67:0/606622940 >> v2:172.21.15.169:3300/0 conn(0x7fdbd80d0460 0x7fdbd80d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:29.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbdce00640 1 -- 172.21.15.67:0/606622940 >> v2:172.21.15.169:3300/0 conn(0x7fdbd80d0460 msgr2=0x7fdbd80d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:29.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbdce00640 1 --2- 172.21.15.67:0/606622940 >> v2:172.21.15.169:3300/0 conn(0x7fdbd80d0460 0x7fdbd80d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbdce00640 1 -- 172.21.15.67:0/606622940 >> v1:172.21.15.67:6789/0 conn(0x7fdbd815c6b0 legacy=0x7fdbd80b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:29.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbdce00640 1 -- 172.21.15.67:0/606622940 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd80cd760 con 0x7fdbd80ccc30 2024-05-30T16:47:29.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.008+0000 7fdbd7e00640 1 -- 172.21.15.67:0/606622940 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1139473667 0 0) 0x7fdbd80cd1a0 con 0x7fdbd815c6b0 2024-05-30T16:47:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.009+0000 7fdbdce00640 1 --2- 172.21.15.67:0/606622940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd80ccc30 0x7fdbd806dc90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009870 tx=0x7fdbc802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c90cc879a867f55 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbd7e00640 1 -- 172.21.15.67:0/606622940 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8037070 con 0x7fdbd80ccc30 2024-05-30T16:47:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbd7e00640 1 -- 172.21.15.67:0/606622940 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc8032440 con 0x7fdbd80ccc30 2024-05-30T16:47:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbd7e00640 1 -- 172.21.15.67:0/606622940 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8032810 con 0x7fdbd80ccc30 2024-05-30T16:47:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbdec00640 1 -- 172.21.15.67:0/606622940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd80ccc30 msgr2=0x7fdbd806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:29.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbdec00640 1 --2- 172.21.15.67:0/606622940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd80ccc30 0x7fdbd806dc90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009870 tx=0x7fdbc802a2a0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbdec00640 1 -- 172.21.15.67:0/606622940 shutdown_connections 2024-05-30T16:47:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbdec00640 1 --2- 172.21.15.67:0/606622940 >> v2:172.21.15.169:3300/0 conn(0x7fdbd80d0460 0x7fdbd80d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbdec00640 1 --2- 172.21.15.67:0/606622940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd80ccc30 0x7fdbd806dc90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbdec00640 1 -- 172.21.15.67:0/606622940 >> 172.21.15.67:0/606622940 conn(0x7fdbd806bae0 msgr2=0x7fdbd806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbdec00640 1 -- 172.21.15.67:0/606622940 shutdown_connections 2024-05-30T16:47:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.010+0000 7fdbdec00640 1 -- 172.21.15.67:0/606622940 wait complete. 2024-05-30T16:47:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.012+0000 7fdbdec00640 1 Processor -- start 2024-05-30T16:47:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.012+0000 7fdbdec00640 1 -- start start 2024-05-30T16:47:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.013+0000 7fdbdec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd80ccc30 0x7fdbd80b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.013+0000 7fdbdec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd80d0460 0x7fdbd80c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.013+0000 7fdbdec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd815c6b0 0x7fdbd81f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.013+0000 7fdbdec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd806e1d0 con 0x7fdbd815c6b0 2024-05-30T16:47:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.013+0000 7fdbdec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd80cba10 con 0x7fdbd80d0460 2024-05-30T16:47:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.013+0000 7fdbdec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd806e350 con 0x7fdbd80ccc30 2024-05-30T16:47:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbde200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd815c6b0 0x7fdbd81f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbde200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd815c6b0 0x7fdbd81f6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50608/0 (socket says 172.21.15.67:50608) 2024-05-30T16:47:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbde200640 1 -- 172.21.15.67:0/3212563340 learned_addr learned my addr 172.21.15.67:0/3212563340 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbdd800640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd80ccc30 0x7fdbd80b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbdce00640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd80d0460 0x7fdbd80c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbde200640 1 -- 172.21.15.67:0/3212563340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd80ccc30 msgr2=0x7fdbd80b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbde200640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd80ccc30 0x7fdbd80b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbde200640 1 -- 172.21.15.67:0/3212563340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd80d0460 msgr2=0x7fdbd80c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbde200640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd80d0460 0x7fdbd80c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbde200640 1 -- 172.21.15.67:0/3212563340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd80b0e20 con 0x7fdbd815c6b0 2024-05-30T16:47:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.014+0000 7fdbdd800640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd80ccc30 0x7fdbd80b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:47:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.015+0000 7fdbde200640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd815c6b0 0x7fdbd81f6c20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000cd70 tx=0x7fdbd000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.015+0000 7fdbd5600640 1 -- 172.21.15.67:0/3212563340 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0003b90 con 0x7fdbd815c6b0 2024-05-30T16:47:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.015+0000 7fdbd5600640 1 -- 172.21.15.67:0/3212563340 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd0003d30 con 0x7fdbd815c6b0 2024-05-30T16:47:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.015+0000 7fdbd5600640 1 -- 172.21.15.67:0/3212563340 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0004240 con 0x7fdbd815c6b0 2024-05-30T16:47:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.015+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd80b1100 con 0x7fdbd815c6b0 2024-05-30T16:47:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.015+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd80b16c0 con 0x7fdbd815c6b0 2024-05-30T16:47:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.016+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd80afb40 con 0x7fdbd815c6b0 2024-05-30T16:47:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.017+0000 7fdbd5600640 1 -- 172.21.15.67:0/3212563340 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) ==== 91091+0+0 (secure 0 0 0) 0x7fdbd00043e0 con 0x7fdbd815c6b0 2024-05-30T16:47:29.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.018+0000 7fdbd5600640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbbc06c9b0 0x7fdbbc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:29.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.018+0000 7fdbd5600640 1 -- 172.21.15.67:0/3212563340 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fdbd008c840 con 0x7fdbd815c6b0 2024-05-30T16:47:29.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.018+0000 7fdbdd800640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbbc06c9b0 0x7fdbbc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:29.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.019+0000 7fdbdd800640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbbc06c9b0 0x7fdbbc06ee70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc000f60 tx=0x7fdbcc007ca0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:29.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.023+0000 7fdbd5600640 1 -- 172.21.15.67:0/3212563340 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbd80afb40 con 0x7fdbd815c6b0 2024-05-30T16:47:29.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.292+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdbd80d09e0 con 0x7fdbd815c6b0 2024-05-30T16:47:29.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.293+0000 7fdbd5600640 1 -- 172.21.15.67:0/3212563340 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1829 (secure 0 0 0) 0x7fdbd005aa70 con 0x7fdbd815c6b0 2024-05-30T16:47:29.294 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:47:29.294 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:47:29.294 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:47:29.294 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:47:29.294 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:47:29.294 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:47:29.295 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 242, 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:47:29.296 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2534, 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 10524717306, 2024-05-30T16:47:29.297 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 21245100032, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 751849013248, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 64288842, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 16 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:47:29.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:47:29.299 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": {} 2024-05-30T16:47:29.300 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:47:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.295+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbbc06c9b0 msgr2=0x7fdbbc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.295+0000 7fdbdec00640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbbc06c9b0 0x7fdbbc06ee70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc000f60 tx=0x7fdbcc007ca0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.295+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd815c6b0 msgr2=0x7fdbd81f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:29.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.295+0000 7fdbdec00640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd815c6b0 0x7fdbd81f6c20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000cd70 tx=0x7fdbd000a660 comp rx=0 tx=0).stop 2024-05-30T16:47:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.295+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 shutdown_connections 2024-05-30T16:47:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.295+0000 7fdbdec00640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbbc06c9b0 0x7fdbbc06ee70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.295+0000 7fdbdec00640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd815c6b0 0x7fdbd81f6c20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.295+0000 7fdbdec00640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd80d0460 0x7fdbd80c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.295+0000 7fdbdec00640 1 --2- 172.21.15.67:0/3212563340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd80ccc30 0x7fdbd80b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.296+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 >> 172.21.15.67:0/3212563340 conn(0x7fdbd806bae0 msgr2=0x7fdbd8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.296+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 shutdown_connections 2024-05-30T16:47:29.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:29.296+0000 7fdbdec00640 1 -- 172.21.15.67:0/3212563340 wait complete. 2024-05-30T16:47:31.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:47:31.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.537+0000 7fe97f200640 1 Processor -- start 2024-05-30T16:47:31.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.538+0000 7fe97f200640 1 -- start start 2024-05-30T16:47:31.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.539+0000 7fe97f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9800b4dc0 0x7fe980162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:31.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.539+0000 7fe97f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9800b5250 con 0x7fe9800b35b0 2024-05-30T16:47:31.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.539+0000 7fe97f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9800d2c70 con 0x7fe9800b3990 2024-05-30T16:47:31.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.539+0000 7fe97f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe980162390 con 0x7fe9800b4dc0 2024-05-30T16:47:31.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.539+0000 7fe97de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe9800b35b0 0x7fe980162590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40020/0 (socket says 172.21.15.67:40020) 2024-05-30T16:47:31.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.539+0000 7fe97de00640 1 -- 172.21.15.67:0/1398151935 learned_addr learned my addr 172.21.15.67:0/1398151935 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:47:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe975400640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9800b4dc0 0x7fe980162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 610701102 0 0) 0x7fe9800b5250 con 0x7fe9800b35b0 2024-05-30T16:47:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe964003520 con 0x7fe9800b35b0 2024-05-30T16:47:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 698702689 0 0) 0x7fe964003520 con 0x7fe9800b35b0 2024-05-30T16:47:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9800b4dc0 msgr2=0x7fe980162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9800b4dc0 0x7fe980162ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 >> v1:172.21.15.169:6789/0 conn(0x7fe9800b3990 legacy=0x7fe980160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9800d2f50 con 0x7fe9800b35b0 2024-05-30T16:47:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fe9700017f0 con 0x7fe9800b35b0 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 >> v1:172.21.15.67:6789/0 conn(0x7fe9800b35b0 legacy=0x7fe980162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9640039e0 0x7fe964003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe964004800 0x7fe964004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe964005180 0x7fe96400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe96400bf00 con 0x7fe9640039e0 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe96400c0b0 con 0x7fe964005180 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.540+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe96400c2b0 con 0x7fe964004800 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.541+0000 7fe97de00640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9640039e0 0x7fe964003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.541+0000 7fe975400640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe964004800 0x7fe964004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.541+0000 7fe97e800640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe964005180 0x7fe96400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.541+0000 7fe97de00640 1 -- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe964004800 msgr2=0x7fe964004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.541+0000 7fe97de00640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe964004800 0x7fe964004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.541+0000 7fe97de00640 1 -- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe964005180 msgr2=0x7fe96400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.541+0000 7fe97de00640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe964005180 0x7fe96400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.541+0000 7fe97de00640 1 -- 172.21.15.67:0/1398151935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe964003520 con 0x7fe9640039e0 2024-05-30T16:47:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.541+0000 7fe97e800640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe964005180 0x7fe96400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.542+0000 7fe97de00640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9640039e0 0x7fe964003e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe970007fd0 tx=0x7fe964005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a363efc5607864 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970007c20 con 0x7fe9640039e0 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe970007270 con 0x7fe9640039e0 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97d400640 1 -- 172.21.15.67:0/1398151935 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9700057a0 con 0x7fe9640039e0 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97f200640 1 -- 172.21.15.67:0/1398151935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9640039e0 msgr2=0x7fe964003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97f200640 1 --2- 172.21.15.67:0/1398151935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9640039e0 0x7fe964003e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe970007fd0 tx=0x7fe964005830 comp rx=0 tx=0).stop 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97e800640 1 -- 172.21.15.67:0/1398151935 reap_dead start 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97f200640 1 -- 172.21.15.67:0/1398151935 shutdown_connections 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97f200640 1 -- 172.21.15.67:0/1398151935 >> 172.21.15.67:0/1398151935 conn(0x7fe98008bfc0 msgr2=0x7fe9800d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97f200640 1 -- 172.21.15.67:0/1398151935 shutdown_connections 2024-05-30T16:47:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.543+0000 7fe97f200640 1 -- 172.21.15.67:0/1398151935 wait complete. 2024-05-30T16:47:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.545+0000 7fe97f200640 1 Processor -- start 2024-05-30T16:47:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.545+0000 7fe97f200640 1 -- start start 2024-05-30T16:47:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.546+0000 7fe97f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9640039e0 0x7fe98015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.546+0000 7fe97f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9800b7c70 0x7fe9800b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.546+0000 7fe97f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9800b94e0 0x7fe9800b9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.546+0000 7fe97f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe980162ca0 con 0x7fe9800b7c70 2024-05-30T16:47:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.546+0000 7fe97f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe9800b5620 con 0x7fe9800b94e0 2024-05-30T16:47:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.546+0000 7fe97f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe980162e20 con 0x7fe9640039e0 2024-05-30T16:47:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe975400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9800b7c70 0x7fe9800b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe975400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9800b7c70 0x7fe9800b80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57760/0 (socket says 172.21.15.67:57760) 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe975400640 1 -- 172.21.15.67:0/3222116605 learned_addr learned my addr 172.21.15.67:0/3222116605 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe97de00640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9640039e0 0x7fe98015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe97e800640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9800b94e0 0x7fe9800b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe975400640 1 -- 172.21.15.67:0/3222116605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9640039e0 msgr2=0x7fe98015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe975400640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9640039e0 0x7fe98015aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe975400640 1 -- 172.21.15.67:0/3222116605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9800b94e0 msgr2=0x7fe9800b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe975400640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9800b94e0 0x7fe9800b9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe975400640 1 -- 172.21.15.67:0/3222116605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98015b3e0 con 0x7fe9800b7c70 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe97de00640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9640039e0 0x7fe98015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:47:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.547+0000 7fe975400640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9800b7c70 0x7fe9800b80f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe9680115a0 tx=0x7fe968011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.548+0000 7fe976a00640 1 -- 172.21.15.67:0/3222116605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968015870 con 0x7fe9800b7c70 2024-05-30T16:47:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.548+0000 7fe976a00640 1 -- 172.21.15.67:0/3222116605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe968015a10 con 0x7fe9800b7c70 2024-05-30T16:47:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.548+0000 7fe976a00640 1 -- 172.21.15.67:0/3222116605 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968015e00 con 0x7fe9800b7c70 2024-05-30T16:47:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.548+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9800b87c0 con 0x7fe9800b7c70 2024-05-30T16:47:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.548+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9800b8d80 con 0x7fe9800b7c70 2024-05-30T16:47:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.549+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9800689e0 con 0x7fe9800b7c70 2024-05-30T16:47:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.550+0000 7fe976a00640 1 -- 172.21.15.67:0/3222116605 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fe96801d400 con 0x7fe9800b7c70 2024-05-30T16:47:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.551+0000 7fe976a00640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95c06cad0 0x7fe95c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.551+0000 7fe976a00640 1 -- 172.21.15.67:0/3222116605 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe96808c6c0 con 0x7fe9800b7c70 2024-05-30T16:47:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.551+0000 7fe97de00640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95c06cad0 0x7fe95c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.552+0000 7fe97de00640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95c06cad0 0x7fe95c06ef90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe970032000 tx=0x7fe970009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:31.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.556+0000 7fe976a00640 1 -- 172.21.15.67:0/3222116605 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe968019030 con 0x7fe9800b7c70 2024-05-30T16:47:31.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.832+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe9800ba500 con 0x7fe9800b7c70 2024-05-30T16:47:31.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.833+0000 7fe976a00640 1 -- 172.21.15.67:0/3222116605 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1859 (secure 0 0 0) 0x7fe96805a7d0 con 0x7fe9800b7c70 2024-05-30T16:47:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:47:31.834 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:47:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:47:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:47:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:47:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:47:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:47:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 245, 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:47:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:47:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2578, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 10705149427, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 21610094592, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 751484018688, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 85, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 79322190, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:47:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 21 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:47:31.838 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:47:31.839 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:31.840 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:31.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:31.840 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:47:31.840 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:47:31.840 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:47:31.840 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:31.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:31.840 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": {} 2024-05-30T16:47:31.840 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:47:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95c06cad0 msgr2=0x7fe95c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95c06cad0 0x7fe95c06ef90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe970032000 tx=0x7fe970009030 comp rx=0 tx=0).stop 2024-05-30T16:47:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9800b7c70 msgr2=0x7fe9800b80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9800b7c70 0x7fe9800b80f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe9680115a0 tx=0x7fe968011a60 comp rx=0 tx=0).stop 2024-05-30T16:47:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 shutdown_connections 2024-05-30T16:47:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95c06cad0 0x7fe95c06ef90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9800b94e0 0x7fe9800b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9800b7c70 0x7fe9800b80f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:31.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 --2- 172.21.15.67:0/3222116605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9640039e0 0x7fe98015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:31.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 >> 172.21.15.67:0/3222116605 conn(0x7fe98008bfc0 msgr2=0x7fe980160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:31.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 shutdown_connections 2024-05-30T16:47:31.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:31.836+0000 7fe97f200640 1 -- 172.21.15.67:0/3222116605 wait complete. 2024-05-30T16:47:33.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:47:34.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.196+0000 7f8001200640 1 Processor -- start 2024-05-30T16:47:34.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.197+0000 7f8001200640 1 -- start start 2024-05-30T16:47:34.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.199+0000 7f8001200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ffc06fcc0 0x7f7ffc070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:34.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.199+0000 7f8001200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ffc15efa0 con 0x7f7ffc162380 2024-05-30T16:47:34.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.199+0000 7f8001200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ffc15fd20 con 0x7f7ffc15c370 2024-05-30T16:47:34.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.199+0000 7f8001200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc072830 con 0x7f7ffc06fcc0 2024-05-30T16:47:34.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.200+0000 7f7ff9e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7ffc15c370 0x7f7ffc071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:40476/0 (socket says 172.21.15.67:40476) 2024-05-30T16:47:34.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.200+0000 7f7ff9e00640 1 -- 172.21.15.67:0/2958972114 learned_addr learned my addr 172.21.15.67:0/2958972114 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:47:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.201+0000 7f7ff9400640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ffc06fcc0 0x7f7ffc070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:34.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.202+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1731746622 0 0) 0x7f7ffc15efa0 con 0x7f7ffc162380 2024-05-30T16:47:34.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.202+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fe0003520 con 0x7f7ffc162380 2024-05-30T16:47:34.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.203+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 120605515 0 0) 0x7f7ffc15fd20 con 0x7f7ffc15c370 2024-05-30T16:47:34.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.203+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ffc15efa0 con 0x7f7ffc15c370 2024-05-30T16:47:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.203+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 103672923 0 0) 0x7f7fe0003520 con 0x7f7ffc162380 2024-05-30T16:47:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.203+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ffc06fcc0 msgr2=0x7f7ffc070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.203+0000 7f7ff8a00640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ffc06fcc0 0x7f7ffc070140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.203+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 >> v1:172.21.15.169:6789/0 conn(0x7f7ffc15c370 legacy=0x7f7ffc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.203+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc15f240 con 0x7f7ffc162380 2024-05-30T16:47:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.204+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7ff4001690 con 0x7f7ffc162380 2024-05-30T16:47:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.204+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 >> v1:172.21.15.67:6789/0 conn(0x7f7ffc162380 legacy=0x7f7ffc1627a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.204+0000 7f7ff8a00640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ffc06fcc0 0x7f7fe0003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.204+0000 7f7ff8a00640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe00040e0 0x7f7fe0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.204+0000 7f7ff8a00640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fe0004d40 0x7f7fe00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.204+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc072830 con 0x7f7fe0004d40 2024-05-30T16:47:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.205+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe00143c0 con 0x7f7fe00040e0 2024-05-30T16:47:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.205+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe00145c0 con 0x7f7ffc06fcc0 2024-05-30T16:47:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.205+0000 7f7ff9400640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe00040e0 0x7f7fe0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.205+0000 7f7ffa800640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fe0004d40 0x7f7fe00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.207+0000 7f7ffa800640 1 -- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ffc06fcc0 msgr2=0x7f7fe0003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.207+0000 7f7ffa800640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ffc06fcc0 0x7f7fe0003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.207+0000 7f7ffa800640 1 -- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe00040e0 msgr2=0x7f7fe0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.207+0000 7f7ffa800640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe00040e0 0x7f7fe0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.207+0000 7f7ffa800640 1 -- 172.21.15.67:0/2958972114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe0003520 con 0x7f7fe0004d40 2024-05-30T16:47:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.211+0000 7f7ffa800640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fe0004d40 0x7f7fe00051c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4007f90 tx=0x7f7fe0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b21e91d9229384 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.212+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4036070 con 0x7f7fe0004d40 2024-05-30T16:47:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.212+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff4031600 con 0x7f7fe0004d40 2024-05-30T16:47:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.212+0000 7f7ff8a00640 1 -- 172.21.15.67:0/2958972114 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4007be0 con 0x7f7fe0004d40 2024-05-30T16:47:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.213+0000 7f8001200640 1 -- 172.21.15.67:0/2958972114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fe0004d40 msgr2=0x7f7fe00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.213+0000 7f8001200640 1 --2- 172.21.15.67:0/2958972114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fe0004d40 0x7f7fe00051c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4007f90 tx=0x7f7fe0005830 comp rx=0 tx=0).stop 2024-05-30T16:47:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.213+0000 7f7ffa800640 1 -- 172.21.15.67:0/2958972114 reap_dead start 2024-05-30T16:47:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.214+0000 7f8001200640 1 -- 172.21.15.67:0/2958972114 shutdown_connections 2024-05-30T16:47:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.214+0000 7f8001200640 1 -- 172.21.15.67:0/2958972114 >> 172.21.15.67:0/2958972114 conn(0x7f7ffc06bae0 msgr2=0x7f7ffc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.214+0000 7f8001200640 1 -- 172.21.15.67:0/2958972114 shutdown_connections 2024-05-30T16:47:34.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.214+0000 7f8001200640 1 -- 172.21.15.67:0/2958972114 wait complete. 2024-05-30T16:47:34.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.219+0000 7f8001200640 1 Processor -- start 2024-05-30T16:47:34.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.220+0000 7f8001200640 1 -- start start 2024-05-30T16:47:34.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.223+0000 7f8001200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0004d40 0x7f7ffc1f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:34.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.223+0000 7f8001200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ffc0a9490 0x7f7ffc1ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:34.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.223+0000 7f8001200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ffc156700 0x7f7ffc156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:34.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.223+0000 7f8001200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc0af000 con 0x7f7ffc156700 2024-05-30T16:47:34.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.223+0000 7f8001200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc15f800 con 0x7f7ffc0a9490 2024-05-30T16:47:34.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.223+0000 7f8001200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc15c750 con 0x7f7fe0004d40 2024-05-30T16:47:34.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.224+0000 7f7ff9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ffc0a9490 0x7f7ffc1ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:34.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.224+0000 7f7ff9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ffc0a9490 0x7f7ffc1ef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:40198/0 (socket says 172.21.15.67:40198) 2024-05-30T16:47:34.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.225+0000 7f7ff9400640 1 -- 172.21.15.67:0/640535591 learned_addr learned my addr 172.21.15.67:0/640535591 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:34.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.225+0000 7f7ffa800640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ffc156700 0x7f7ffc156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:34.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.225+0000 7f7ff9e00640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0004d40 0x7f7ffc1f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:34.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.226+0000 7f7ffa800640 1 -- 172.21.15.67:0/640535591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0004d40 msgr2=0x7f7ffc1f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:34.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.226+0000 7f7ffa800640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0004d40 0x7f7ffc1f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:34.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.226+0000 7f7ffa800640 1 -- 172.21.15.67:0/640535591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ffc0a9490 msgr2=0x7f7ffc1ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:34.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.226+0000 7f7ffa800640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ffc0a9490 0x7f7ffc1ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:34.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.226+0000 7f7ffa800640 1 -- 172.21.15.67:0/640535591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc1f4530 con 0x7f7ffc156700 2024-05-30T16:47:34.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.226+0000 7f7ff9400640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ffc0a9490 0x7f7ffc1ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:47:34.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.227+0000 7f7ffa800640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ffc156700 0x7f7ffc156b80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4031000 tx=0x7f7ffc15f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:34.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.228+0000 7f7ff2000640 1 -- 172.21.15.67:0/640535591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4036070 con 0x7f7ffc156700 2024-05-30T16:47:34.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.228+0000 7f7ff2000640 1 -- 172.21.15.67:0/640535591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff4043070 con 0x7f7ffc156700 2024-05-30T16:47:34.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.228+0000 7f7ff2000640 1 -- 172.21.15.67:0/640535591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4007be0 con 0x7f7ffc156700 2024-05-30T16:47:34.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.228+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc0a99c0 con 0x7f7ffc156700 2024-05-30T16:47:34.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.229+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc157fd0 con 0x7f7ffc156700 2024-05-30T16:47:34.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.230+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd4005290 con 0x7f7ffc156700 2024-05-30T16:47:34.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.232+0000 7f7ff2000640 1 -- 172.21.15.67:0/640535591 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f7ff40035f0 con 0x7f7ffc156700 2024-05-30T16:47:34.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.233+0000 7f7ff2000640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fdc06caa0 0x7f7fdc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:34.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.233+0000 7f7ff2000640 1 -- 172.21.15.67:0/640535591 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f7ff40abf50 con 0x7f7ffc156700 2024-05-30T16:47:34.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.234+0000 7f7ff9e00640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fdc06caa0 0x7f7fdc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:34.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.235+0000 7f7ff9e00640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fdc06caa0 0x7f7fdc06ef60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7fec007d90 tx=0x7f7fec007d20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:34.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.240+0000 7f7ff2000640 1 -- 172.21.15.67:0/640535591 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ff404d020 con 0x7f7ffc156700 2024-05-30T16:47:34.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.774+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7fd4005850 con 0x7f7ffc156700 2024-05-30T16:47:34.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.775+0000 7f7ff2000640 1 -- 172.21.15.67:0/640535591 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2119 (secure 0 0 0) 0x7f7ffc1f4530 con 0x7f7ffc156700 2024-05-30T16:47:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:47:34.776 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:47:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:47:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:47:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:47:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:47:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:47:34.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 248, 2024-05-30T16:47:34.777 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:47:34.778 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2588, 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 10738704219, 2024-05-30T16:47:34.779 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 21736013824, 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 751358099456, 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 170, 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 50667039, 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 14 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:47:34.780 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:34.781 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:34.782 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "e5bd7e10-6021-4b29-bc64-f4d35d2e63df": { 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "1 ongoing clones; Avg progress is 3.125% (1s)\n [............................] (remaining: 33s)", 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.03125, 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:34.783 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:47:34.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fdc06caa0 msgr2=0x7f7fdc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fdc06caa0 0x7f7fdc06ef60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7fec007d90 tx=0x7f7fec007d20 comp rx=0 tx=0).stop 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ffc156700 msgr2=0x7f7ffc156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ffc156700 0x7f7ffc156b80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4031000 tx=0x7f7ffc15f410 comp rx=0 tx=0).stop 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 shutdown_connections 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fdc06caa0 0x7f7fdc06ef60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ffc156700 0x7f7ffc156b80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ffc0a9490 0x7f7ffc1ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 --2- 172.21.15.67:0/640535591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fe0004d40 0x7f7ffc1f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 >> 172.21.15.67:0/640535591 conn(0x7f7ffc06bae0 msgr2=0x7f7ffc06eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 shutdown_connections 2024-05-30T16:47:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:34.778+0000 7f8001200640 1 -- 172.21.15.67:0/640535591 wait complete. 2024-05-30T16:47:34.797 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-05-30T16:47:35.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.101+0000 7f52ac600640 1 Processor -- start 2024-05-30T16:47:35.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.101+0000 7f52ac600640 1 -- start start 2024-05-30T16:47:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.103+0000 7f52ac600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52a415e940 0x7f52a4073030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.103+0000 7f52ac600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a40afdf0 con 0x7f52a40b2ae0 2024-05-30T16:47:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.103+0000 7f52ac600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a40b0060 con 0x7f52a40b32c0 2024-05-30T16:47:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.103+0000 7f52ac600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f52a40b0340 con 0x7f52a415e940 2024-05-30T16:47:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.104+0000 7f52a9c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52a415e940 0x7f52a4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.104+0000 7f52a9c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52a415e940 0x7f52a4073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:40214/0 (socket says 172.21.15.67:40214) 2024-05-30T16:47:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.104+0000 7f52a9c00640 1 -- 172.21.15.67:0/210931640 learned_addr learned my addr 172.21.15.67:0/210931640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:35.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.105+0000 7f52a9c00640 1 -- 172.21.15.67:0/210931640 >> v1:172.21.15.169:6789/0 conn(0x7f52a40b32c0 legacy=0x7f52a4073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.105+0000 7f52a9c00640 1 -- 172.21.15.67:0/210931640 >> v1:172.21.15.67:6789/0 conn(0x7f52a40b2ae0 legacy=0x7f52a40b0e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:47:35.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.105+0000 7f52a9c00640 1 -- 172.21.15.67:0/210931640 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a40b0620 con 0x7f52a415e940 2024-05-30T16:47:35.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.106+0000 7f52a9c00640 1 --2- 172.21.15.67:0/210931640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a415e940 0x7f52a4073030 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5298009870 tx=0x7f529802a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e27e7d4d32b259b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:35.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.107+0000 7f52a3e00640 1 -- 172.21.15.67:0/210931640 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298037070 con 0x7f52a415e940 2024-05-30T16:47:35.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.107+0000 7f52a3e00640 1 -- 172.21.15.67:0/210931640 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5298032440 con 0x7f52a415e940 2024-05-30T16:47:35.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.107+0000 7f52a3e00640 1 -- 172.21.15.67:0/210931640 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298032810 con 0x7f52a415e940 2024-05-30T16:47:35.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.108+0000 7f52ac600640 1 -- 172.21.15.67:0/210931640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a415e940 msgr2=0x7f52a4073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.108+0000 7f52ac600640 1 --2- 172.21.15.67:0/210931640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a415e940 0x7f52a4073030 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5298009870 tx=0x7f529802a2a0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.109+0000 7f52ac600640 1 -- 172.21.15.67:0/210931640 shutdown_connections 2024-05-30T16:47:35.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.109+0000 7f52ac600640 1 --2- 172.21.15.67:0/210931640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a415e940 0x7f52a4073030 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.109+0000 7f52ac600640 1 -- 172.21.15.67:0/210931640 >> 172.21.15.67:0/210931640 conn(0x7f52a40b1fc0 msgr2=0x7f52a40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:35.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.109+0000 7f52ac600640 1 -- 172.21.15.67:0/210931640 shutdown_connections 2024-05-30T16:47:35.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.109+0000 7f52ac600640 1 -- 172.21.15.67:0/210931640 wait complete. 2024-05-30T16:47:35.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.111+0000 7f52ac600640 1 Processor -- start 2024-05-30T16:47:35.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.112+0000 7f52ac600640 1 -- start start 2024-05-30T16:47:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.113+0000 7f52ac600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52a40b2ae0 0x7f52a41617a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.113+0000 7f52ac600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a415e940 0x7f52a40ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.113+0000 7f52ac600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a4160d10 0x7f52a40cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.113+0000 7f52ac600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f52a40ad420 con 0x7f52a40b2ae0 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.113+0000 7f52ac600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f52a40ad2a0 con 0x7f52a4160d10 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.113+0000 7f52ac600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f52a4073c80 con 0x7f52a415e940 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.114+0000 7f52a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52a40b2ae0 0x7f52a41617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.115+0000 7f52aa600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a4160d10 0x7f52a40cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.115+0000 7f52a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52a40b2ae0 0x7f52a41617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57794/0 (socket says 172.21.15.67:57794) 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.115+0000 7f52a9c00640 1 -- 172.21.15.67:0/531015885 learned_addr learned my addr 172.21.15.67:0/531015885 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.115+0000 7f52a9200640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a415e940 0x7f52a40ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.117+0000 7f52a9c00640 1 -- 172.21.15.67:0/531015885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a415e940 msgr2=0x7f52a40ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.117+0000 7f52a9c00640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a415e940 0x7f52a40ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.117+0000 7f52a9c00640 1 -- 172.21.15.67:0/531015885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a4160d10 msgr2=0x7f52a40cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.117+0000 7f52a9c00640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a4160d10 0x7f52a40cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.117+0000 7f52a9c00640 1 -- 172.21.15.67:0/531015885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a41fc9c0 con 0x7f52a40b2ae0 2024-05-30T16:47:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.118+0000 7f52a9c00640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52a40b2ae0 0x7f52a41617a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5298009870 tx=0x7f52980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:35.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.118+0000 7f52a1600640 1 -- 172.21.15.67:0/531015885 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298041070 con 0x7f52a40b2ae0 2024-05-30T16:47:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.118+0000 7f52a1600640 1 -- 172.21.15.67:0/531015885 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f529803c930 con 0x7f52a40b2ae0 2024-05-30T16:47:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.119+0000 7f52a1600640 1 -- 172.21.15.67:0/531015885 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298041070 con 0x7f52a40b2ae0 2024-05-30T16:47:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.119+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a41fcca0 con 0x7f52a40b2ae0 2024-05-30T16:47:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.119+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a414ee20 con 0x7f52a40b2ae0 2024-05-30T16:47:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.120+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a40689e0 con 0x7f52a40b2ae0 2024-05-30T16:47:35.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.123+0000 7f52a1600640 1 -- 172.21.15.67:0/531015885 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f5298044070 con 0x7f52a40b2ae0 2024-05-30T16:47:35.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.124+0000 7f52a1600640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f528406cea0 0x7f528406f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.125+0000 7f52a1600640 1 -- 172.21.15.67:0/531015885 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f529803c730 con 0x7f52a40b2ae0 2024-05-30T16:47:35.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.127+0000 7f52a9200640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f528406cea0 0x7f528406f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.130+0000 7f52a9200640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f528406cea0 0x7f528406f360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52a4069160 tx=0x7f5290009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:35.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.135+0000 7f52a1600640 1 -- 172.21.15.67:0/531015885 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f529807a120 con 0x7f52a40b2ae0 2024-05-30T16:47:35.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.497+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f52a415edb0 con 0x7f528406cea0 2024-05-30T16:47:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.698+0000 7f52a1600640 1 -- 172.21.15.67:0/531015885 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f52a415edb0 con 0x7f528406cea0 2024-05-30T16:47:35.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.701+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f528406cea0 msgr2=0x7f528406f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.701+0000 7f52ac600640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f528406cea0 0x7f528406f360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52a4069160 tx=0x7f5290009040 comp rx=0 tx=0).stop 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.701+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52a40b2ae0 msgr2=0x7f52a41617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.701+0000 7f52ac600640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52a40b2ae0 0x7f52a41617a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5298009870 tx=0x7f52980046c0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.702+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 shutdown_connections 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.702+0000 7f52ac600640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f528406cea0 0x7f528406f360 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.702+0000 7f52ac600640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a4160d10 0x7f52a40cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.702+0000 7f52ac600640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a415e940 0x7f52a40ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.702+0000 7f52ac600640 1 --2- 172.21.15.67:0/531015885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52a40b2ae0 0x7f52a41617a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.702+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 >> 172.21.15.67:0/531015885 conn(0x7f52a40b1fc0 msgr2=0x7f52a406f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.702+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 shutdown_connections 2024-05-30T16:47:35.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.702+0000 7f52ac600640 1 -- 172.21.15.67:0/531015885 wait complete. 2024-05-30T16:47:35.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:47:35.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.924+0000 7f9c46a00640 1 Processor -- start 2024-05-30T16:47:35.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.925+0000 7f9c46a00640 1 -- start start 2024-05-30T16:47:35.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.926+0000 7f9c46a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c4015e870 0x7f9c400d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.926+0000 7f9c46a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c400cfa40 0x7f9c400d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.926+0000 7f9c46a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c4015ec50 con 0x7f9c400d14d0 2024-05-30T16:47:35.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.926+0000 7f9c46a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9c400d2150 con 0x7f9c400cfa40 2024-05-30T16:47:35.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.926+0000 7f9c46a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c400cff80 con 0x7f9c4015e870 2024-05-30T16:47:35.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.927+0000 7f9c45600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c4015e870 0x7f9c400d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.927+0000 7f9c46000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c400cfa40 0x7f9c400d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.927+0000 7f9c45600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c4015e870 0x7f9c400d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:46170/0 (socket says 172.21.15.67:46170) 2024-05-30T16:47:35.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.927+0000 7f9c45600640 1 -- 172.21.15.67:0/2338805607 learned_addr learned my addr 172.21.15.67:0/2338805607 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:35.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.928+0000 7f9c46000640 1 -- 172.21.15.67:0/2338805607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c4015e870 msgr2=0x7f9c400d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:35.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.928+0000 7f9c46000640 1 --2- 172.21.15.67:0/2338805607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c4015e870 0x7f9c400d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.928+0000 7f9c46000640 1 -- 172.21.15.67:0/2338805607 >> v1:172.21.15.169:6789/0 conn(0x7f9c400d14d0 legacy=0x7f9c400cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.928+0000 7f9c46000640 1 -- 172.21.15.67:0/2338805607 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c400d0260 con 0x7f9c400cfa40 2024-05-30T16:47:35.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.928+0000 7f9c45600640 1 --2- 172.21.15.67:0/2338805607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c4015e870 0x7f9c400d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:47:35.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.929+0000 7f9c46000640 1 --2- 172.21.15.67:0/2338805607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400cfa40 0x7f9c400d25e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c380097a0 tx=0x7f9c3802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8760f45d3f14904a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:35.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.930+0000 7f9c3fe00640 1 -- 172.21.15.67:0/2338805607 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38037070 con 0x7f9c400cfa40 2024-05-30T16:47:35.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.930+0000 7f9c3fe00640 1 -- 172.21.15.67:0/2338805607 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c38032440 con 0x7f9c400cfa40 2024-05-30T16:47:35.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.930+0000 7f9c3fe00640 1 -- 172.21.15.67:0/2338805607 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38032810 con 0x7f9c400cfa40 2024-05-30T16:47:35.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.930+0000 7f9c46a00640 1 -- 172.21.15.67:0/2338805607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400cfa40 msgr2=0x7f9c400d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.930+0000 7f9c46a00640 1 --2- 172.21.15.67:0/2338805607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400cfa40 0x7f9c400d25e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c380097a0 tx=0x7f9c3802a180 comp rx=0 tx=0).stop 2024-05-30T16:47:35.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.930+0000 7f9c46a00640 1 -- 172.21.15.67:0/2338805607 shutdown_connections 2024-05-30T16:47:35.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.930+0000 7f9c46a00640 1 --2- 172.21.15.67:0/2338805607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400cfa40 0x7f9c400d25e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.930+0000 7f9c46a00640 1 --2- 172.21.15.67:0/2338805607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c4015e870 0x7f9c400d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.930+0000 7f9c46a00640 1 -- 172.21.15.67:0/2338805607 >> 172.21.15.67:0/2338805607 conn(0x7f9c4008bfc0 msgr2=0x7f9c400d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:35.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.931+0000 7f9c46a00640 1 -- 172.21.15.67:0/2338805607 shutdown_connections 2024-05-30T16:47:35.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.931+0000 7f9c46a00640 1 -- 172.21.15.67:0/2338805607 wait complete. 2024-05-30T16:47:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.932+0000 7f9c46a00640 1 Processor -- start 2024-05-30T16:47:35.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.932+0000 7f9c46a00640 1 -- start start 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.933+0000 7f9c46a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c400cfa40 0x7f9c4014f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.933+0000 7f9c46a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c400d14d0 0x7f9c400cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.933+0000 7f9c46a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c4015e870 0x7f9c400cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.933+0000 7f9c46a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400d4fb0 con 0x7f9c400cfa40 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.933+0000 7f9c46a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400d0820 con 0x7f9c4015e870 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.933+0000 7f9c46a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c400d2b20 con 0x7f9c400d14d0 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c45600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c400cfa40 0x7f9c4014f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c45600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c400cfa40 0x7f9c4014f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57806/0 (socket says 172.21.15.67:57806) 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c45600640 1 -- 172.21.15.67:0/4165147636 learned_addr learned my addr 172.21.15.67:0/4165147636 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:35.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c46000640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c4015e870 0x7f9c400cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c44c00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c400d14d0 0x7f9c400cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c45600640 1 -- 172.21.15.67:0/4165147636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c400d14d0 msgr2=0x7f9c400cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c45600640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c400d14d0 0x7f9c400cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c45600640 1 -- 172.21.15.67:0/4165147636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c4015e870 msgr2=0x7f9c400cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:35.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c45600640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c4015e870 0x7f9c400cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:35.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c45600640 1 -- 172.21.15.67:0/4165147636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c400b7320 con 0x7f9c400cfa40 2024-05-30T16:47:35.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c44c00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c400d14d0 0x7f9c400cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:47:35.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.934+0000 7f9c45600640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c400cfa40 0x7f9c4014f810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000efc0 tx=0x7f9c3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:35.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.935+0000 7f9c3d600640 1 -- 172.21.15.67:0/4165147636 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000ce60 con 0x7f9c400cfa40 2024-05-30T16:47:35.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.935+0000 7f9c3d600640 1 -- 172.21.15.67:0/4165147636 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c30019070 con 0x7f9c400cfa40 2024-05-30T16:47:35.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.935+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c400b7600 con 0x7f9c400cfa40 2024-05-30T16:47:35.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.935+0000 7f9c3d600640 1 -- 172.21.15.67:0/4165147636 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30014680 con 0x7f9c400cfa40 2024-05-30T16:47:35.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.935+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c400b7bc0 con 0x7f9c400cfa40 2024-05-30T16:47:35.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.936+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c400689e0 con 0x7f9c400cfa40 2024-05-30T16:47:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.937+0000 7f9c3d600640 1 -- 172.21.15.67:0/4165147636 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f9c30005d10 con 0x7f9c400cfa40 2024-05-30T16:47:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.938+0000 7f9c3d600640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9c2006caa0 0x7f9c2006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.938+0000 7f9c3d600640 1 -- 172.21.15.67:0/4165147636 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9c3008c420 con 0x7f9c400cfa40 2024-05-30T16:47:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.938+0000 7f9c44c00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9c2006caa0 0x7f9c2006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:35.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.938+0000 7f9c44c00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9c2006caa0 0x7f9c2006ef60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9c400b86f0 tx=0x7f9c2c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:35.942+0000 7f9c3d600640 1 -- 172.21.15.67:0/4165147636 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c30010040 con 0x7f9c400cfa40 2024-05-30T16:47:36.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.166+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9c400b41c0 con 0x7f9c2006caa0 2024-05-30T16:47:36.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.168+0000 7f9c3d600640 1 -- 172.21.15.67:0/4165147636 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9c400b41c0 con 0x7f9c2006caa0 2024-05-30T16:47:36.169 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:47:36.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:36.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:47:36.169 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:36.169 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.170+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9c2006caa0 msgr2=0x7f9c2006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.170+0000 7f9c46a00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9c2006caa0 0x7f9c2006ef60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9c400b86f0 tx=0x7f9c2c009040 comp rx=0 tx=0).stop 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c400cfa40 msgr2=0x7f9c4014f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c400cfa40 0x7f9c4014f810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000efc0 tx=0x7f9c3000c490 comp rx=0 tx=0).stop 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 shutdown_connections 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9c2006caa0 0x7f9c2006ef60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c4015e870 0x7f9c400cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c400d14d0 0x7f9c400cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 --2- 172.21.15.67:0/4165147636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c400cfa40 0x7f9c4014f810 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 >> 172.21.15.67:0/4165147636 conn(0x7f9c4008bfc0 msgr2=0x7f9c400b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:36.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 shutdown_connections 2024-05-30T16:47:36.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.171+0000 7f9c46a00640 1 -- 172.21.15.67:0/4165147636 wait complete. 2024-05-30T16:47:36.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:47:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.371+0000 7f954b200640 1 Processor -- start 2024-05-30T16:47:36.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.372+0000 7f954b200640 1 -- start start 2024-05-30T16:47:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.373+0000 7f954b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c0cc2b0 0x7f954c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.373+0000 7f954b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f954c0d09a0 0x7f954c0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.373+0000 7f954b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f954c1621d0 con 0x7f954c161df0 2024-05-30T16:47:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.373+0000 7f954b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f954c0cd840 con 0x7f954c0d09a0 2024-05-30T16:47:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.373+0000 7f954b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f954c0ccf10 con 0x7f954c0cc2b0 2024-05-30T16:47:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.374+0000 7f9541400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c0cc2b0 0x7f954c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.374+0000 7f954a800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f954c0d09a0 0x7f954c0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.374+0000 7f9541400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c0cc2b0 0x7f954c0d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:46188/0 (socket says 172.21.15.67:46188) 2024-05-30T16:47:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.374+0000 7f9541400640 1 -- 172.21.15.67:0/2844236805 learned_addr learned my addr 172.21.15.67:0/2844236805 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:36.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.375+0000 7f9541400640 1 -- 172.21.15.67:0/2844236805 >> v2:172.21.15.169:3300/0 conn(0x7f954c0d09a0 msgr2=0x7f954c0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:36.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.375+0000 7f9541400640 1 --2- 172.21.15.67:0/2844236805 >> v2:172.21.15.169:3300/0 conn(0x7f954c0d09a0 0x7f954c0cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.375+0000 7f9541400640 1 -- 172.21.15.67:0/2844236805 >> v1:172.21.15.169:6789/0 conn(0x7f954c161df0 legacy=0x7f954c099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:36.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.375+0000 7f9541400640 1 -- 172.21.15.67:0/2844236805 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c0cd1f0 con 0x7f954c0cc2b0 2024-05-30T16:47:36.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.375+0000 7f954a800640 1 --2- 172.21.15.67:0/2844236805 >> v2:172.21.15.169:3300/0 conn(0x7f954c0d09a0 0x7f954c0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:47:36.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.376+0000 7f9541400640 1 --2- 172.21.15.67:0/2844236805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c0cc2b0 0x7f954c0d0460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9534009870 tx=0x7f953402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349d124b95c6713 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:36.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f9549400640 1 -- 172.21.15.67:0/2844236805 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534037070 con 0x7f954c0cc2b0 2024-05-30T16:47:36.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f9549400640 1 -- 172.21.15.67:0/2844236805 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9534032440 con 0x7f954c0cc2b0 2024-05-30T16:47:36.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f9549400640 1 -- 172.21.15.67:0/2844236805 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534032810 con 0x7f954c0cc2b0 2024-05-30T16:47:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f954b200640 1 -- 172.21.15.67:0/2844236805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c0cc2b0 msgr2=0x7f954c0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f954b200640 1 --2- 172.21.15.67:0/2844236805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c0cc2b0 0x7f954c0d0460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9534009870 tx=0x7f953402a250 comp rx=0 tx=0).stop 2024-05-30T16:47:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f954b200640 1 -- 172.21.15.67:0/2844236805 shutdown_connections 2024-05-30T16:47:36.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f954b200640 1 --2- 172.21.15.67:0/2844236805 >> v2:172.21.15.169:3300/0 conn(0x7f954c0d09a0 0x7f954c0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f954b200640 1 --2- 172.21.15.67:0/2844236805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c0cc2b0 0x7f954c0d0460 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f954b200640 1 -- 172.21.15.67:0/2844236805 >> 172.21.15.67:0/2844236805 conn(0x7f954c06bae0 msgr2=0x7f954c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f954b200640 1 -- 172.21.15.67:0/2844236805 shutdown_connections 2024-05-30T16:47:36.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.377+0000 7f954b200640 1 -- 172.21.15.67:0/2844236805 wait complete. 2024-05-30T16:47:36.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.379+0000 7f954b200640 1 Processor -- start 2024-05-30T16:47:36.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.379+0000 7f954b200640 1 -- start start 2024-05-30T16:47:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.380+0000 7f954b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0cc2b0 0x7f954c0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.380+0000 7f954b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c0d09a0 0x7f954c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.380+0000 7f954b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c161df0 0x7f954c1610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.380+0000 7f954b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0ca870 con 0x7f954c0d09a0 2024-05-30T16:47:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.380+0000 7f954b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0ccbc0 con 0x7f954c0cc2b0 2024-05-30T16:47:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.380+0000 7f954b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f954c0ca9f0 con 0x7f954c161df0 2024-05-30T16:47:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.380+0000 7f9541400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c0d09a0 0x7f954c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.380+0000 7f9541400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c0d09a0 0x7f954c1587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57812/0 (socket says 172.21.15.67:57812) 2024-05-30T16:47:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.380+0000 7f9541400640 1 -- 172.21.15.67:0/1948551415 learned_addr learned my addr 172.21.15.67:0/1948551415 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.381+0000 7f9549e00640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0cc2b0 0x7f954c0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.381+0000 7f954a800640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c161df0 0x7f954c1610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.381+0000 7f9541400640 1 -- 172.21.15.67:0/1948551415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c161df0 msgr2=0x7f954c1610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.381+0000 7f9541400640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c161df0 0x7f954c1610a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.381+0000 7f9541400640 1 -- 172.21.15.67:0/1948551415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0cc2b0 msgr2=0x7f954c0af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.381+0000 7f9541400640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0cc2b0 0x7f954c0af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.381+0000 7f9541400640 1 -- 172.21.15.67:0/1948551415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c158ce0 con 0x7f954c0d09a0 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.381+0000 7f9541400640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c0d09a0 0x7f954c1587a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f953402a290 tx=0x7f95340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.382+0000 7f9542a00640 1 -- 172.21.15.67:0/1948551415 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534037040 con 0x7f954c0d09a0 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.382+0000 7f9542a00640 1 -- 172.21.15.67:0/1948551415 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9534042070 con 0x7f954c0d09a0 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.382+0000 7f9542a00640 1 -- 172.21.15.67:0/1948551415 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534030880 con 0x7f954c0d09a0 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.382+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c0afea0 con 0x7f954c0d09a0 2024-05-30T16:47:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.382+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c0b0460 con 0x7f954c0d09a0 2024-05-30T16:47:36.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.383+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954c0689e0 con 0x7f954c0d09a0 2024-05-30T16:47:36.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.384+0000 7f9542a00640 1 -- 172.21.15.67:0/1948551415 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f9534030a20 con 0x7f954c0d09a0 2024-05-30T16:47:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.385+0000 7f9542a00640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f952806cad0 0x7f952806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.385+0000 7f9542a00640 1 -- 172.21.15.67:0/1948551415 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f95340ac6c0 con 0x7f954c0d09a0 2024-05-30T16:47:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.385+0000 7f9549e00640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f952806cad0 0x7f952806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.385+0000 7f9549e00640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f952806cad0 0x7f952806ef90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f953c007fd0 tx=0x7f953c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.389+0000 7f9542a00640 1 -- 172.21.15.67:0/1948551415 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f953407a870 con 0x7f954c0d09a0 2024-05-30T16:47:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.667+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f954c0aed20 con 0x7f954c0d09a0 2024-05-30T16:47:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.667+0000 7f9542a00640 1 -- 172.21.15.67:0/1948551415 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2119 (secure 0 0 0) 0x7f953407a170 con 0x7f954c0d09a0 2024-05-30T16:47:36.668 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:47:36.668 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:36.669 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 250, 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:47:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:36.671 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2588, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 10738704219, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 21736013824, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 751358099456, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 170, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 50667039, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 14 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:47:36.672 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:36.673 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:47:36.674 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:36.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:36.675 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:47:36.675 INFO:teuthology.orchestra.run.smithi067.stdout: "e5bd7e10-6021-4b29-bc64-f4d35d2e63df": { 2024-05-30T16:47:36.675 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "1 ongoing clones; Avg progress is 3.125% (1s)\n [............................] (remaining: 33s)", 2024-05-30T16:47:36.675 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.03125, 2024-05-30T16:47:36.675 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:47:36.675 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:36.675 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:36.675 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f952806cad0 msgr2=0x7f952806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f952806cad0 0x7f952806ef90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f953c007fd0 tx=0x7f953c009040 comp rx=0 tx=0).stop 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c0d09a0 msgr2=0x7f954c1587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c0d09a0 0x7f954c1587a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f953402a290 tx=0x7f95340046c0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 shutdown_connections 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f952806cad0 0x7f952806ef90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f954c161df0 0x7f954c1610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954c0d09a0 0x7f954c1587a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 --2- 172.21.15.67:0/1948551415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f954c0cc2b0 0x7f954c0af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:36.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 >> 172.21.15.67:0/1948551415 conn(0x7f954c06bae0 msgr2=0x7f954c0b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:36.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 shutdown_connections 2024-05-30T16:47:36.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:36.670+0000 7f954b200640 1 -- 172.21.15.67:0/1948551415 wait complete. 2024-05-30T16:47:39.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:47:39.480 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:47:41.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:47:41.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.909+0000 7f5fdde00640 1 Processor -- start 2024-05-30T16:47:41.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.909+0000 7f5fdde00640 1 -- start start 2024-05-30T16:47:41.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.910+0000 7f5fdde00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5fd80ccc30 0x7f5fd806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:41.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.910+0000 7f5fdde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5fd80d0460 0x7f5fd80d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:41.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.910+0000 7f5fdde00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fd80cd1a0 con 0x7f5fd815c6b0 2024-05-30T16:47:41.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.910+0000 7f5fdde00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd80d0df0 con 0x7f5fd80ccc30 2024-05-30T16:47:41.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.910+0000 7f5fdde00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd80cd480 con 0x7f5fd80d0460 2024-05-30T16:47:41.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fd7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5fd80ccc30 0x7f5fd806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:41.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fd7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5fd80ccc30 0x7f5fd806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58104/0 (socket says 172.21.15.67:58104) 2024-05-30T16:47:41.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fd7e00640 1 -- 172.21.15.67:0/2226942094 learned_addr learned my addr 172.21.15.67:0/2226942094 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:41.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fdca00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5fd815c6b0 0x7f5fd80b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48020/0 (socket says 172.21.15.67:48020) 2024-05-30T16:47:41.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fd7e00640 1 -- 172.21.15.67:0/2226942094 >> v2:172.21.15.169:3300/0 conn(0x7f5fd80d0460 msgr2=0x7f5fd80d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:41.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fdd400640 1 --2- 172.21.15.67:0/2226942094 >> v2:172.21.15.169:3300/0 conn(0x7f5fd80d0460 0x7f5fd80d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:41.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fd7e00640 1 --2- 172.21.15.67:0/2226942094 >> v2:172.21.15.169:3300/0 conn(0x7f5fd80d0460 0x7f5fd80d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:41.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fd7e00640 1 -- 172.21.15.67:0/2226942094 >> v1:172.21.15.67:6789/0 conn(0x7f5fd815c6b0 legacy=0x7f5fd80b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:41.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fd7400640 1 -- 172.21.15.67:0/2226942094 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147568217 0 0) 0x7f5fd80cd1a0 con 0x7f5fd815c6b0 2024-05-30T16:47:41.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.911+0000 7f5fd7e00640 1 -- 172.21.15.67:0/2226942094 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd80cd760 con 0x7f5fd80ccc30 2024-05-30T16:47:41.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.912+0000 7f5fd7e00640 1 --2- 172.21.15.67:0/2226942094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd80ccc30 0x7f5fd806dc90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009870 tx=0x7f5fc402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb3b97bc1e0d35 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:41.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.913+0000 7f5fd7400640 1 -- 172.21.15.67:0/2226942094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4037070 con 0x7f5fd80ccc30 2024-05-30T16:47:41.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.913+0000 7f5fd7400640 1 -- 172.21.15.67:0/2226942094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc4032440 con 0x7f5fd80ccc30 2024-05-30T16:47:41.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.913+0000 7f5fd7400640 1 -- 172.21.15.67:0/2226942094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4032810 con 0x7f5fd80ccc30 2024-05-30T16:47:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.913+0000 7f5fdde00640 1 -- 172.21.15.67:0/2226942094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd80ccc30 msgr2=0x7f5fd806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.913+0000 7f5fdde00640 1 --2- 172.21.15.67:0/2226942094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd80ccc30 0x7f5fd806dc90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009870 tx=0x7f5fc402a2a0 comp rx=0 tx=0).stop 2024-05-30T16:47:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.914+0000 7f5fdde00640 1 -- 172.21.15.67:0/2226942094 shutdown_connections 2024-05-30T16:47:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.914+0000 7f5fdde00640 1 --2- 172.21.15.67:0/2226942094 >> v2:172.21.15.169:3300/0 conn(0x7f5fd80d0460 0x7f5fd80d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.914+0000 7f5fdde00640 1 --2- 172.21.15.67:0/2226942094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd80ccc30 0x7f5fd806dc90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.914+0000 7f5fdde00640 1 -- 172.21.15.67:0/2226942094 >> 172.21.15.67:0/2226942094 conn(0x7f5fd806bae0 msgr2=0x7f5fd806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.914+0000 7f5fdde00640 1 -- 172.21.15.67:0/2226942094 shutdown_connections 2024-05-30T16:47:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.914+0000 7f5fdde00640 1 -- 172.21.15.67:0/2226942094 wait complete. 2024-05-30T16:47:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.916+0000 7f5fdde00640 1 Processor -- start 2024-05-30T16:47:41.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.916+0000 7f5fdde00640 1 -- start start 2024-05-30T16:47:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.917+0000 7f5fdde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fd80ccc30 0x7f5fd814b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.917+0000 7f5fdde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fd80d0460 0x7f5fd81562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.917+0000 7f5fdde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd815c6b0 0x7f5fd80cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.917+0000 7f5fdde00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd806e1d0 con 0x7f5fd815c6b0 2024-05-30T16:47:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.917+0000 7f5fdde00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd80cba10 con 0x7f5fd80ccc30 2024-05-30T16:47:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.917+0000 7f5fdde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd806e350 con 0x7f5fd80d0460 2024-05-30T16:47:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.917+0000 7f5fdd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd815c6b0 0x7f5fd80cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:41.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.917+0000 7f5fdd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd815c6b0 0x7f5fd80cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58108/0 (socket says 172.21.15.67:58108) 2024-05-30T16:47:41.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.917+0000 7f5fdd400640 1 -- 172.21.15.67:0/2804061119 learned_addr learned my addr 172.21.15.67:0/2804061119 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.918+0000 7f5fdd400640 1 -- 172.21.15.67:0/2804061119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fd80d0460 msgr2=0x7f5fd81562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.918+0000 7f5fdca00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fd80ccc30 0x7f5fd814b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.918+0000 7f5fdd400640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fd80d0460 0x7f5fd81562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.918+0000 7f5fdd400640 1 -- 172.21.15.67:0/2804061119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fd80ccc30 msgr2=0x7f5fd814b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.918+0000 7f5fdd400640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fd80ccc30 0x7f5fd814b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.918+0000 7f5fdd400640 1 -- 172.21.15.67:0/2804061119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd8156c80 con 0x7f5fd815c6b0 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.918+0000 7f5fdd400640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd815c6b0 0x7f5fd80cbd80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000cd70 tx=0x7f5fd000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.918+0000 7f5fd4c00640 1 -- 172.21.15.67:0/2804061119 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0003b90 con 0x7f5fd815c6b0 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.918+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd8156f60 con 0x7f5fd815c6b0 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.919+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd8157520 con 0x7f5fd815c6b0 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.919+0000 7f5fd4c00640 1 -- 172.21.15.67:0/2804061119 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd0003d30 con 0x7f5fd815c6b0 2024-05-30T16:47:41.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.919+0000 7f5fd4c00640 1 -- 172.21.15.67:0/2804061119 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0004220 con 0x7f5fd815c6b0 2024-05-30T16:47:41.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.920+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb4005290 con 0x7f5fd815c6b0 2024-05-30T16:47:41.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.921+0000 7f5fd4c00640 1 -- 172.21.15.67:0/2804061119 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f5fd00043c0 con 0x7f5fd815c6b0 2024-05-30T16:47:41.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.922+0000 7f5fd4c00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fb806caa0 0x7f5fb806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:41.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.922+0000 7f5fd4c00640 1 -- 172.21.15.67:0/2804061119 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5fd008cc20 con 0x7f5fd815c6b0 2024-05-30T16:47:41.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.922+0000 7f5fdca00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fb806caa0 0x7f5fb806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:41.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.923+0000 7f5fdca00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fb806caa0 0x7f5fb806ef60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000f60 tx=0x7f5fcc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:41.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:41.928+0000 7f5fd4c00640 1 -- 172.21.15.67:0/2804061119 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fd005acb0 con 0x7f5fd815c6b0 2024-05-30T16:47:42.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.197+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5fb4005850 con 0x7f5fd815c6b0 2024-05-30T16:47:42.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.198+0000 7f5fd4c00640 1 -- 172.21.15.67:0/2804061119 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2120 (secure 0 0 0) 0x7f5fd005a5b0 con 0x7f5fd815c6b0 2024-05-30T16:47:42.198 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:47:42.198 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:47:42.199 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 255, 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:47:42.200 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:47:42.201 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2860, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 11871288713, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 24005451776, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 749088661504, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 96429147, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 97139418, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 23, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 28 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:47:42.202 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:47:42.203 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "e5bd7e10-6021-4b29-bc64-f4d35d2e63df": { 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 25.0 seconds before exiting (8s)\n [============================] ", 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:47:42.204 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:47:42.205 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:42.205 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:42.205 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:47:42.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fb806caa0 msgr2=0x7f5fb806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:42.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fb806caa0 0x7f5fb806ef60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000f60 tx=0x7f5fcc009040 comp rx=0 tx=0).stop 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd815c6b0 msgr2=0x7f5fd80cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd815c6b0 0x7f5fd80cbd80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000cd70 tx=0x7f5fd000a660 comp rx=0 tx=0).stop 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 shutdown_connections 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fb806caa0 0x7f5fb806ef60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd815c6b0 0x7f5fd80cbd80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fd80d0460 0x7f5fd81562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 --2- 172.21.15.67:0/2804061119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fd80ccc30 0x7f5fd814b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 >> 172.21.15.67:0/2804061119 conn(0x7f5fd806bae0 msgr2=0x7f5fd806bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 shutdown_connections 2024-05-30T16:47:42.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:42.200+0000 7f5fdde00640 1 -- 172.21.15.67:0/2804061119 wait complete. 2024-05-30T16:47:47.218 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:47:47.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.437+0000 7f5f9ae00640 1 Processor -- start 2024-05-30T16:47:47.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.437+0000 7f5f9ae00640 1 -- start start 2024-05-30T16:47:47.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.439+0000 7f5f9ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f9415b380 0x7f5f9415ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:47.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.439+0000 7f5f9ae00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f9415fe50 con 0x7f5f940d0460 2024-05-30T16:47:47.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.439+0000 7f5f9ae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f940cec60 con 0x7f5f940ce800 2024-05-30T16:47:47.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.439+0000 7f5f9ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f940b3310 con 0x7f5f9415b380 2024-05-30T16:47:47.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.439+0000 7f5f9a400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5f940d0460 0x7f5f940b37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48022/0 (socket says 172.21.15.67:48022) 2024-05-30T16:47:47.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.439+0000 7f5f9a400640 1 -- 172.21.15.67:0/3505714469 learned_addr learned my addr 172.21.15.67:0/3505714469 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:47:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f99a00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f9415b380 0x7f5f9415ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 849465290 0 0) 0x7f5f9415fe50 con 0x7f5f940d0460 2024-05-30T16:47:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f7c005730 con 0x7f5f940d0460 2024-05-30T16:47:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2002274864 0 0) 0x7f5f7c005730 con 0x7f5f940d0460 2024-05-30T16:47:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f9415b380 msgr2=0x7f5f9415ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f9415b380 0x7f5f9415ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 >> v1:172.21.15.169:6789/0 conn(0x7f5f940ce800 legacy=0x7f5f940ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9415f5b0 con 0x7f5f940d0460 2024-05-30T16:47:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5f8c001670 con 0x7f5f940d0460 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 >> v1:172.21.15.67:6789/0 conn(0x7f5f940d0460 legacy=0x7f5f940b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f7c005bf0 0x7f5f7c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f9415b380 0x7f5f7c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f7c006f50 0x7f5f7c0073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c00ddb0 con 0x7f5f7c005bf0 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c00df60 con 0x7f5f7c006f50 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.440+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7c00e160 con 0x7f5f9415b380 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.441+0000 7f5f99a00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f7c005bf0 0x7f5f7c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.441+0000 7f5f99000640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f9415b380 0x7f5f7c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.441+0000 7f5f99a00640 1 -- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f9415b380 msgr2=0x7f5f7c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.441+0000 7f5f99a00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f9415b380 0x7f5f7c006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.441+0000 7f5f99a00640 1 -- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f7c006f50 msgr2=0x7f5f7c0073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.441+0000 7f5f99a00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f7c006f50 0x7f5f7c0073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.441+0000 7f5f99a00640 1 -- 172.21.15.67:0/3505714469 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c005730 con 0x7f5f7c005bf0 2024-05-30T16:47:47.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.442+0000 7f5f99a00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f7c005bf0 0x7f5f7c006010 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009fc0 tx=0x7f5f7c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f1c3ab0305676b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.442+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88009c10 con 0x7f5f7c005bf0 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.443+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f88036070 con 0x7f5f7c005bf0 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.443+0000 7f5f93e00640 1 -- 172.21.15.67:0/3505714469 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88031880 con 0x7f5f7c005bf0 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.443+0000 7f5f9ae00640 1 -- 172.21.15.67:0/3505714469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f7c005bf0 msgr2=0x7f5f7c006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.443+0000 7f5f9ae00640 1 --2- 172.21.15.67:0/3505714469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f7c005bf0 0x7f5f7c006010 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009fc0 tx=0x7f5f7c007a40 comp rx=0 tx=0).stop 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.443+0000 7f5f9a400640 1 -- 172.21.15.67:0/3505714469 reap_dead start 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.443+0000 7f5f9ae00640 1 -- 172.21.15.67:0/3505714469 shutdown_connections 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.443+0000 7f5f9ae00640 1 -- 172.21.15.67:0/3505714469 >> 172.21.15.67:0/3505714469 conn(0x7f5f9406bae0 msgr2=0x7f5f9406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.443+0000 7f5f9ae00640 1 -- 172.21.15.67:0/3505714469 shutdown_connections 2024-05-30T16:47:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.443+0000 7f5f9ae00640 1 -- 172.21.15.67:0/3505714469 wait complete. 2024-05-30T16:47:47.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.445+0000 7f5f9ae00640 1 Processor -- start 2024-05-30T16:47:47.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.445+0000 7f5f9ae00640 1 -- start start 2024-05-30T16:47:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.446+0000 7f5f9ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f7c005bf0 0x7f5f9414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.446+0000 7f5f9ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f94071c70 0x7f5f940ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.446+0000 7f5f9ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f940720f0 0x7f5f940ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.446+0000 7f5f9ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9415b760 con 0x7f5f940720f0 2024-05-30T16:47:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.446+0000 7f5f9ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9415fb70 con 0x7f5f94071c70 2024-05-30T16:47:47.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.446+0000 7f5f9ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9415b8e0 con 0x7f5f7c005bf0 2024-05-30T16:47:47.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.446+0000 7f5f9a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f940720f0 0x7f5f940ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:47.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f9a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f940720f0 0x7f5f940ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58118/0 (socket says 172.21.15.67:58118) 2024-05-30T16:47:47.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f9a400640 1 -- 172.21.15.67:0/1252622000 learned_addr learned my addr 172.21.15.67:0/1252622000 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:47.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f99a00640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f7c005bf0 0x7f5f9414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:47.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f99000640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f94071c70 0x7f5f940ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f9a400640 1 -- 172.21.15.67:0/1252622000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f7c005bf0 msgr2=0x7f5f9414d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f9a400640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f7c005bf0 0x7f5f9414d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f9a400640 1 -- 172.21.15.67:0/1252622000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f94071c70 msgr2=0x7f5f940ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f9a400640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f94071c70 0x7f5f940ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f9a400640 1 -- 172.21.15.67:0/1252622000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f940728f0 con 0x7f5f940720f0 2024-05-30T16:47:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f99000640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f94071c70 0x7f5f940ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:47:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.447+0000 7f5f9a400640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f940720f0 0x7f5f940ae7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0010e0 tx=0x7f5f8c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.448+0000 7f5f91600640 1 -- 172.21.15.67:0/1252622000 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0074e0 con 0x7f5f940720f0 2024-05-30T16:47:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.448+0000 7f5f91600640 1 -- 172.21.15.67:0/1252622000 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8c0047e0 con 0x7f5f940720f0 2024-05-30T16:47:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.448+0000 7f5f91600640 1 -- 172.21.15.67:0/1252622000 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c004bd0 con 0x7f5f940720f0 2024-05-30T16:47:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.448+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f94072bd0 con 0x7f5f940720f0 2024-05-30T16:47:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.448+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f94073190 con 0x7f5f940720f0 2024-05-30T16:47:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.449+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f940689e0 con 0x7f5f940720f0 2024-05-30T16:47:47.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.450+0000 7f5f91600640 1 -- 172.21.15.67:0/1252622000 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f5f8c014460 con 0x7f5f940720f0 2024-05-30T16:47:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.451+0000 7f5f91600640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f7406caa0 0x7f5f7406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.451+0000 7f5f91600640 1 -- 172.21.15.67:0/1252622000 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5f8c08b1a0 con 0x7f5f940720f0 2024-05-30T16:47:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.451+0000 7f5f99a00640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f7406caa0 0x7f5f7406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.452+0000 7f5f99a00640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f7406caa0 0x7f5f7406ef60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5f88035000 tx=0x7f5f8802f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:47.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.456+0000 7f5f91600640 1 -- 172.21.15.67:0/1252622000 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f8c01b030 con 0x7f5f940720f0 2024-05-30T16:47:47.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.734+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5f940b4fe0 con 0x7f5f940720f0 2024-05-30T16:47:47.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.735+0000 7f5f91600640 1 -- 172.21.15.67:0/1252622000 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2121 (secure 0 0 0) 0x7f5f8c059340 con 0x7f5f940720f0 2024-05-30T16:47:47.736 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:47:47.736 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:47:47.736 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:47:47.736 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:47:47.736 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:47:47.736 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:47:47.737 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 261, 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:47:47.738 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:47:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2859, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 11871310549, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 24005484544, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 749088628736, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 96435307, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 94352469, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 23, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 28 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:47:47.740 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:47.741 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:47:47.742 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout: "e5bd7e10-6021-4b29-bc64-f4d35d2e63df": { 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (12s)\n [============================] ", 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:47.743 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:47:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.737+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f7406caa0 msgr2=0x7f5f7406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.737+0000 7f5f9ae00640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f7406caa0 0x7f5f7406ef60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5f88035000 tx=0x7f5f8802f000 comp rx=0 tx=0).stop 2024-05-30T16:47:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.737+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f940720f0 msgr2=0x7f5f940ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.737+0000 7f5f9ae00640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f940720f0 0x7f5f940ae7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0010e0 tx=0x7f5f8c00c8e0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.738+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 shutdown_connections 2024-05-30T16:47:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.738+0000 7f5f9ae00640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f7406caa0 0x7f5f7406ef60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.738+0000 7f5f9ae00640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f940720f0 0x7f5f940ae7b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.738+0000 7f5f9ae00640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f94071c70 0x7f5f940ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.738+0000 7f5f9ae00640 1 --2- 172.21.15.67:0/1252622000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f7c005bf0 0x7f5f9414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.738+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 >> 172.21.15.67:0/1252622000 conn(0x7f5f9406bae0 msgr2=0x7f5f940d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.738+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 shutdown_connections 2024-05-30T16:47:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:47.738+0000 7f5f9ae00640 1 -- 172.21.15.67:0/1252622000 wait complete. 2024-05-30T16:47:52.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:47:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.976+0000 7f5d79400640 1 Processor -- start 2024-05-30T16:47:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.976+0000 7f5d79400640 1 -- start start 2024-05-30T16:47:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.978+0000 7f5d79400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5d740ccc30 0x7f5d7406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.978+0000 7f5d79400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d740cd1a0 con 0x7f5d7415c6b0 2024-05-30T16:47:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.978+0000 7f5d79400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d740cd480 con 0x7f5d740d0460 2024-05-30T16:47:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.978+0000 7f5d79400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5d740cd760 con 0x7f5d740ccc30 2024-05-30T16:47:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.978+0000 7f5d73e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5d7415c6b0 0x7f5d740b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45514/0 (socket says 172.21.15.67:45514) 2024-05-30T16:47:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.978+0000 7f5d73e00640 1 -- 172.21.15.67:0/2351988532 learned_addr learned my addr 172.21.15.67:0/2351988532 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:47:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.978+0000 7f5d73400640 1 --2- 172.21.15.67:0/2351988532 >> v2:172.21.15.67:3300/0 conn(0x7f5d740ccc30 0x7f5d7406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.979+0000 7f5d72a00640 1 -- 172.21.15.67:0/2351988532 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209485486 0 0) 0x7f5d740cd1a0 con 0x7f5d7415c6b0 2024-05-30T16:47:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.979+0000 7f5d72a00640 1 -- 172.21.15.67:0/2351988532 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d58003520 con 0x7f5d7415c6b0 2024-05-30T16:47:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.979+0000 7f5d73400640 1 -- 172.21.15.67:0/2351988532 >> v1:172.21.15.169:6789/0 conn(0x7f5d740d0460 legacy=0x7f5d740cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.979+0000 7f5d73400640 1 -- 172.21.15.67:0/2351988532 >> v1:172.21.15.67:6789/0 conn(0x7f5d7415c6b0 legacy=0x7f5d740b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:52.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.979+0000 7f5d73400640 1 -- 172.21.15.67:0/2351988532 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d740cda40 con 0x7f5d740ccc30 2024-05-30T16:47:52.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.980+0000 7f5d73400640 1 --2- 172.21.15.67:0/2351988532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740ccc30 0x7f5d7406dc90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5d600097d0 tx=0x7f5d6002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9774da7b9396c119 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:52.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.980+0000 7f5d72a00640 1 -- 172.21.15.67:0/2351988532 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60037070 con 0x7f5d740ccc30 2024-05-30T16:47:52.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.980+0000 7f5d72a00640 1 -- 172.21.15.67:0/2351988532 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d60032440 con 0x7f5d740ccc30 2024-05-30T16:47:52.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.980+0000 7f5d72a00640 1 -- 172.21.15.67:0/2351988532 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d600327f0 con 0x7f5d740ccc30 2024-05-30T16:47:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.981+0000 7f5d79400640 1 -- 172.21.15.67:0/2351988532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740ccc30 msgr2=0x7f5d7406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.981+0000 7f5d79400640 1 --2- 172.21.15.67:0/2351988532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740ccc30 0x7f5d7406dc90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5d600097d0 tx=0x7f5d6002a210 comp rx=0 tx=0).stop 2024-05-30T16:47:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.981+0000 7f5d79400640 1 -- 172.21.15.67:0/2351988532 shutdown_connections 2024-05-30T16:47:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.981+0000 7f5d79400640 1 --2- 172.21.15.67:0/2351988532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740ccc30 0x7f5d7406dc90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.981+0000 7f5d79400640 1 -- 172.21.15.67:0/2351988532 >> 172.21.15.67:0/2351988532 conn(0x7f5d7406bae0 msgr2=0x7f5d7406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.981+0000 7f5d79400640 1 -- 172.21.15.67:0/2351988532 shutdown_connections 2024-05-30T16:47:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.981+0000 7f5d79400640 1 -- 172.21.15.67:0/2351988532 wait complete. 2024-05-30T16:47:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.983+0000 7f5d79400640 1 Processor -- start 2024-05-30T16:47:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.983+0000 7f5d79400640 1 -- start start 2024-05-30T16:47:52.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.984+0000 7f5d79400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d740ccc30 0x7f5d74099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:52.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.984+0000 7f5d79400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740d0460 0x7f5d740ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:52.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.984+0000 7f5d79400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d7415c6b0 0x7f5d74154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.984+0000 7f5d79400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7406e350 con 0x7f5d740d0460 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.984+0000 7f5d79400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7406e1d0 con 0x7f5d7415c6b0 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.984+0000 7f5d79400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d740cc640 con 0x7f5d740ccc30 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.984+0000 7f5d73400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740d0460 0x7f5d740ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.984+0000 7f5d73400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740d0460 0x7f5d740ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36422/0 (socket says 172.21.15.67:36422) 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.984+0000 7f5d73400640 1 -- 172.21.15.67:0/2548640869 learned_addr learned my addr 172.21.15.67:0/2548640869 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.985+0000 7f5d73e00640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d740ccc30 0x7f5d74099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.985+0000 7f5d73400640 1 -- 172.21.15.67:0/2548640869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d740ccc30 msgr2=0x7f5d74099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.985+0000 7f5d73400640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d740ccc30 0x7f5d74099900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.985+0000 7f5d73400640 1 -- 172.21.15.67:0/2548640869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d7415c6b0 msgr2=0x7f5d74154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.985+0000 7f5d73400640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d7415c6b0 0x7f5d74154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.985+0000 7f5d73400640 1 -- 172.21.15.67:0/2548640869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d740aebb0 con 0x7f5d740d0460 2024-05-30T16:47:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.985+0000 7f5d73400640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740d0460 0x7f5d740ad130 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5d60035040 tx=0x7f5d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.986+0000 7f5d6be00640 1 -- 172.21.15.67:0/2548640869 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60041070 con 0x7f5d740d0460 2024-05-30T16:47:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.986+0000 7f5d6be00640 1 -- 172.21.15.67:0/2548640869 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d60004220 con 0x7f5d740d0460 2024-05-30T16:47:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.986+0000 7f5d6be00640 1 -- 172.21.15.67:0/2548640869 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60037040 con 0x7f5d740d0460 2024-05-30T16:47:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.986+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d740adb10 con 0x7f5d740d0460 2024-05-30T16:47:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.986+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d740ae0d0 con 0x7f5d740d0460 2024-05-30T16:47:52.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.987+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d740689e0 con 0x7f5d740d0460 2024-05-30T16:47:52.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.988+0000 7f5d6be00640 1 -- 172.21.15.67:0/2548640869 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f5d60003610 con 0x7f5d740d0460 2024-05-30T16:47:52.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.989+0000 7f5d6be00640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d5406caf0 0x7f5d5406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:52.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.989+0000 7f5d6be00640 1 -- 172.21.15.67:0/2548640869 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5d600ac660 con 0x7f5d740d0460 2024-05-30T16:47:52.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.990+0000 7f5d73e00640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d5406caf0 0x7f5d5406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:52.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.990+0000 7f5d73e00640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d5406caf0 0x7f5d5406efb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d64000f60 tx=0x7f5d64009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:52.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:52.994+0000 7f5d6be00640 1 -- 172.21.15.67:0/2548640869 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d6007a7f0 con 0x7f5d740d0460 2024-05-30T16:47:53.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.272+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5d7406cc30 con 0x7f5d740d0460 2024-05-30T16:47:53.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.272+0000 7f5d6be00640 1 -- 172.21.15.67:0/2548640869 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2116 (secure 0 0 0) 0x7f5d6007a0f0 con 0x7f5d740d0460 2024-05-30T16:47:53.274 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:47:53.275 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 266, 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:53.276 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:53.277 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2859, 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 11871310549, 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 24005484544, 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 749088628736, 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 1747041, 2024-05-30T16:47:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 700965, 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 1 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:47:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:47:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "e5bd7e10-6021-4b29-bc64-f4d35d2e63df": { 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (18s)\n [============================] ", 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:47:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:47:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:53.282 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:47:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.275+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d5406caf0 msgr2=0x7f5d5406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.275+0000 7f5d79400640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d5406caf0 0x7f5d5406efb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5d64000f60 tx=0x7f5d64009040 comp rx=0 tx=0).stop 2024-05-30T16:47:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.275+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740d0460 msgr2=0x7f5d740ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.275+0000 7f5d79400640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740d0460 0x7f5d740ad130 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5d60035040 tx=0x7f5d600046c0 comp rx=0 tx=0).stop 2024-05-30T16:47:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.275+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 shutdown_connections 2024-05-30T16:47:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.275+0000 7f5d79400640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d5406caf0 0x7f5d5406efb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:53.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.275+0000 7f5d79400640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d7415c6b0 0x7f5d74154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:53.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.275+0000 7f5d79400640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d740d0460 0x7f5d740ad130 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:53.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.275+0000 7f5d79400640 1 --2- 172.21.15.67:0/2548640869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d740ccc30 0x7f5d74099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:53.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.276+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 >> 172.21.15.67:0/2548640869 conn(0x7f5d7406bae0 msgr2=0x7f5d7406bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:53.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.276+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 shutdown_connections 2024-05-30T16:47:53.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:53.276+0000 7f5d79400640 1 -- 172.21.15.67:0/2548640869 wait complete. 2024-05-30T16:47:58.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:47:58.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.518+0000 7f9103000640 1 Processor -- start 2024-05-30T16:47:58.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.518+0000 7f9103000640 1 -- start start 2024-05-30T16:47:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.519+0000 7f9103000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f90fc15ef60 0x7f90fc0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.519+0000 7f9103000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc0d0460 0x7f90fc0d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.519+0000 7f9103000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90fc15fe50 con 0x7f90fc15b380 2024-05-30T16:47:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.519+0000 7f9103000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f90fc0b3310 con 0x7f90fc15ef60 2024-05-30T16:47:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.519+0000 7f9103000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc15f5b0 con 0x7f90fc0d0460 2024-05-30T16:47:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.519+0000 7f9101200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f90fc15ef60 0x7f90fc0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.519+0000 7f9101c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f90fc15b380 0x7f90fc0ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45518/0 (socket says 172.21.15.67:45518) 2024-05-30T16:47:58.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.519+0000 7f9101c00640 1 -- 172.21.15.67:0/3639603508 learned_addr learned my addr 172.21.15.67:0/3639603508 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:47:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.520+0000 7f9102600640 1 --2- 172.21.15.67:0/3639603508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc0d0460 0x7f90fc0d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.520+0000 7f9101200640 1 -- 172.21.15.67:0/3639603508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc0d0460 msgr2=0x7f90fc0d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:47:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.520+0000 7f9101200640 1 --2- 172.21.15.67:0/3639603508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc0d0460 0x7f90fc0d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.520+0000 7f9101200640 1 -- 172.21.15.67:0/3639603508 >> v1:172.21.15.67:6789/0 conn(0x7f90fc15b380 legacy=0x7f90fc0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.520+0000 7f9101200640 1 -- 172.21.15.67:0/3639603508 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc15f890 con 0x7f90fc15ef60 2024-05-30T16:47:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.520+0000 7f90fbe00640 1 -- 172.21.15.67:0/3639603508 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1974192120 0 0) 0x7f90fc15fe50 con 0x7f90fc15b380 2024-05-30T16:47:58.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.520+0000 7f9102600640 1 --2- 172.21.15.67:0/3639603508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc0d0460 0x7f90fc0d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:47:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.521+0000 7f9101200640 1 --2- 172.21.15.67:0/3639603508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 0x7f90fc0ce800 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90e80097a0 tx=0x7f90e802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2a2ebbd01dcefd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.521+0000 7f90fbe00640 1 -- 172.21.15.67:0/3639603508 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8037070 con 0x7f90fc15ef60 2024-05-30T16:47:58.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.521+0000 7f90fbe00640 1 -- 172.21.15.67:0/3639603508 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e802adb0 con 0x7f90fc15ef60 2024-05-30T16:47:58.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.521+0000 7f90fbe00640 1 -- 172.21.15.67:0/3639603508 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8032660 con 0x7f90fc15ef60 2024-05-30T16:47:58.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.522+0000 7f9103000640 1 -- 172.21.15.67:0/3639603508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 msgr2=0x7f90fc0ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:58.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.522+0000 7f9103000640 1 --2- 172.21.15.67:0/3639603508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 0x7f90fc0ce800 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90e80097a0 tx=0x7f90e802a060 comp rx=0 tx=0).stop 2024-05-30T16:47:58.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.522+0000 7f9103000640 1 -- 172.21.15.67:0/3639603508 shutdown_connections 2024-05-30T16:47:58.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.522+0000 7f9103000640 1 --2- 172.21.15.67:0/3639603508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc0d0460 0x7f90fc0d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:58.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.522+0000 7f9103000640 1 --2- 172.21.15.67:0/3639603508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 0x7f90fc0ce800 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:58.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.522+0000 7f9103000640 1 -- 172.21.15.67:0/3639603508 >> 172.21.15.67:0/3639603508 conn(0x7f90fc06bae0 msgr2=0x7f90fc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:58.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.522+0000 7f9103000640 1 -- 172.21.15.67:0/3639603508 shutdown_connections 2024-05-30T16:47:58.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.522+0000 7f9103000640 1 -- 172.21.15.67:0/3639603508 wait complete. 2024-05-30T16:47:58.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.524+0000 7f9103000640 1 Processor -- start 2024-05-30T16:47:58.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.524+0000 7f9103000640 1 -- start start 2024-05-30T16:47:58.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.525+0000 7f9103000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90fc0d0460 0x7f90fc157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:58.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.525+0000 7f9103000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc15b380 0x7f90fc0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:58.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.525+0000 7f9103000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 0x7f90fc0af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:58.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.525+0000 7f9103000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc15b8e0 con 0x7f90fc15ef60 2024-05-30T16:47:58.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.525+0000 7f9103000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc15b760 con 0x7f90fc0d0460 2024-05-30T16:47:58.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.525+0000 7f9103000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc0b37a0 con 0x7f90fc15b380 2024-05-30T16:47:58.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.525+0000 7f9102600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 0x7f90fc0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:58.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.525+0000 7f9102600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 0x7f90fc0af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36440/0 (socket says 172.21.15.67:36440) 2024-05-30T16:47:58.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.525+0000 7f9102600640 1 -- 172.21.15.67:0/504289374 learned_addr learned my addr 172.21.15.67:0/504289374 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.526+0000 7f9101200640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc15b380 0x7f90fc0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.526+0000 7f9101c00640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90fc0d0460 0x7f90fc157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.526+0000 7f9102600640 1 -- 172.21.15.67:0/504289374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc15b380 msgr2=0x7f90fc0aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.526+0000 7f9102600640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc15b380 0x7f90fc0aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.526+0000 7f9102600640 1 -- 172.21.15.67:0/504289374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90fc0d0460 msgr2=0x7f90fc157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.526+0000 7f9102600640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90fc0d0460 0x7f90fc157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.526+0000 7f9102600640 1 -- 172.21.15.67:0/504289374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc15c220 con 0x7f90fc15ef60 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.526+0000 7f9101c00640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90fc0d0460 0x7f90fc157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.526+0000 7f9102600640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 0x7f90fc0af250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f90f400cef0 tx=0x7f90f400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.527+0000 7f90f9600640 1 -- 172.21.15.67:0/504289374 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f401a070 con 0x7f90fc15ef60 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.527+0000 7f90f9600640 1 -- 172.21.15.67:0/504289374 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f400d860 con 0x7f90fc15ef60 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.527+0000 7f90f9600640 1 -- 172.21.15.67:0/504289374 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f400dc50 con 0x7f90fc15ef60 2024-05-30T16:47:58.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.527+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc15bf40 con 0x7f90fc15ef60 2024-05-30T16:47:58.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.527+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc06e4e0 con 0x7f90fc15ef60 2024-05-30T16:47:58.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.528+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc0689e0 con 0x7f90fc15ef60 2024-05-30T16:47:58.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.529+0000 7f90f9600640 1 -- 172.21.15.67:0/504289374 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f90f401d070 con 0x7f90fc15ef60 2024-05-30T16:47:58.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.530+0000 7f90f9600640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f90dc06caa0 0x7f90dc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:47:58.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.531+0000 7f90f9600640 1 -- 172.21.15.67:0/504289374 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f90f408c770 con 0x7f90fc15ef60 2024-05-30T16:47:58.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.531+0000 7f9101c00640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f90dc06caa0 0x7f90dc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:47:58.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.532+0000 7f9101c00640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f90dc06caa0 0x7f90dc06ef60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000f60 tx=0x7f90f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:47:58.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.536+0000 7f90f9600640 1 -- 172.21.15.67:0/504289374 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90f405a9d0 con 0x7f90fc15ef60 2024-05-30T16:47:58.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.813+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f90fc073f10 con 0x7f90fc15ef60 2024-05-30T16:47:58.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.814+0000 7f90f9600640 1 -- 172.21.15.67:0/504289374 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2077 (secure 0 0 0) 0x7f90f405a2d0 con 0x7f90fc15ef60 2024-05-30T16:47:58.818 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:47:58.819 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 272, 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:47:58.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:47:58.821 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2859, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 11871312164, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 24005484544, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 749088628736, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 255, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:58.822 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:58.823 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:47:58.824 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout: "e5bd7e10-6021-4b29-bc64-f4d35d2e63df": { 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 9.0 seconds before exiting (24s)\n [============================] ", 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:47:58.825 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f90dc06caa0 msgr2=0x7f90dc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f90dc06caa0 0x7f90dc06ef60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000f60 tx=0x7f90f0009040 comp rx=0 tx=0).stop 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 msgr2=0x7f90fc0af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 0x7f90fc0af250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f90f400cef0 tx=0x7f90f400a710 comp rx=0 tx=0).stop 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 shutdown_connections 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f90dc06caa0 0x7f90dc06ef60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90fc15ef60 0x7f90fc0af250 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90fc15b380 0x7f90fc0aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 --2- 172.21.15.67:0/504289374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90fc0d0460 0x7f90fc157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 >> 172.21.15.67:0/504289374 conn(0x7f90fc06bae0 msgr2=0x7f90fc0adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:47:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 shutdown_connections 2024-05-30T16:47:58.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:47:58.817+0000 7f9103000640 1 -- 172.21.15.67:0/504289374 wait complete. 2024-05-30T16:48:03.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:48:04.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.060+0000 7f0597e00640 1 Processor -- start 2024-05-30T16:48:04.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.060+0000 7f0597e00640 1 -- start start 2024-05-30T16:48:04.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.061+0000 7f0597e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f059815c6b0 0x7f05980b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:04.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.061+0000 7f0597e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 0x7f059806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:04.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.061+0000 7f0597e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05980cd1a0 con 0x7f05980d0460 2024-05-30T16:48:04.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0597e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f05980cd480 con 0x7f059815c6b0 2024-05-30T16:48:04.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0597e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f05980cd760 con 0x7f05980ccc30 2024-05-30T16:48:04.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0596a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f059815c6b0 0x7f05980b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:04.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0596a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f059815c6b0 0x7f05980b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40694/0 (socket says 172.21.15.67:40694) 2024-05-30T16:48:04.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0596a00640 1 -- 172.21.15.67:0/2490242057 learned_addr learned my addr 172.21.15.67:0/2490242057 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:04.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0596a00640 1 -- 172.21.15.67:0/2490242057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 msgr2=0x7f059806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:04.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0596000640 1 --2- 172.21.15.67:0/2490242057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 0x7f059806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:04.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0596a00640 1 --2- 172.21.15.67:0/2490242057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 0x7f059806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:04.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0596a00640 1 -- 172.21.15.67:0/2490242057 >> v1:172.21.15.169:6789/0 conn(0x7f05980d0460 legacy=0x7f05980cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:04.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.062+0000 7f0596a00640 1 -- 172.21.15.67:0/2490242057 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05980cda40 con 0x7f059815c6b0 2024-05-30T16:48:04.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.063+0000 7f0596a00640 1 --2- 172.21.15.67:0/2490242057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 0x7f05980b3db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0588009870 tx=0x7f058802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec6eada4faf6c89 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:04.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.064+0000 7f0595600640 1 -- 172.21.15.67:0/2490242057 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588037070 con 0x7f059815c6b0 2024-05-30T16:48:04.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.064+0000 7f0595600640 1 -- 172.21.15.67:0/2490242057 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0588032440 con 0x7f059815c6b0 2024-05-30T16:48:04.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.064+0000 7f0595600640 1 -- 172.21.15.67:0/2490242057 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588032810 con 0x7f059815c6b0 2024-05-30T16:48:04.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.064+0000 7f0597e00640 1 -- 172.21.15.67:0/2490242057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 msgr2=0x7f05980b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:04.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.064+0000 7f0597e00640 1 --2- 172.21.15.67:0/2490242057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 0x7f05980b3db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0588009870 tx=0x7f058802a210 comp rx=0 tx=0).stop 2024-05-30T16:48:04.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.065+0000 7f0597e00640 1 -- 172.21.15.67:0/2490242057 shutdown_connections 2024-05-30T16:48:04.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.065+0000 7f0597e00640 1 --2- 172.21.15.67:0/2490242057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 0x7f059806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:04.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.065+0000 7f0597e00640 1 --2- 172.21.15.67:0/2490242057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 0x7f05980b3db0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:04.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.065+0000 7f0597e00640 1 -- 172.21.15.67:0/2490242057 >> 172.21.15.67:0/2490242057 conn(0x7f059806bae0 msgr2=0x7f059806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:04.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.065+0000 7f0597e00640 1 -- 172.21.15.67:0/2490242057 shutdown_connections 2024-05-30T16:48:04.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.065+0000 7f0597e00640 1 -- 172.21.15.67:0/2490242057 wait complete. 2024-05-30T16:48:04.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.066+0000 7f0597e00640 1 Processor -- start 2024-05-30T16:48:04.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.067+0000 7f0597e00640 1 -- start start 2024-05-30T16:48:04.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 0x7f059814d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:04.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05980d0460 0x7f05980c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:04.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 0x7f059806f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:04.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05980b4470 con 0x7f059815c6b0 2024-05-30T16:48:04.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f05980b42f0 con 0x7f05980d0460 2024-05-30T16:48:04.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f059806e1d0 con 0x7f05980ccc30 2024-05-30T16:48:04.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 0x7f059806f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 0x7f059806f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40704/0 (socket says 172.21.15.67:40704) 2024-05-30T16:48:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597400640 1 -- 172.21.15.67:0/3197662145 learned_addr learned my addr 172.21.15.67:0/3197662145 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0596a00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 0x7f059814d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0596000640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05980d0460 0x7f05980c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597400640 1 -- 172.21.15.67:0/3197662145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 msgr2=0x7f059814d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597400640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 0x7f059814d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597400640 1 -- 172.21.15.67:0/3197662145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05980d0460 msgr2=0x7f05980c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597400640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05980d0460 0x7f05980c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.068+0000 7f0597400640 1 -- 172.21.15.67:0/3197662145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981f64a0 con 0x7f059815c6b0 2024-05-30T16:48:04.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.069+0000 7f0597400640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 0x7f059806f2d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f059000c780 tx=0x7f059000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:04.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.069+0000 7f058ea00640 1 -- 172.21.15.67:0/3197662145 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590004b60 con 0x7f059815c6b0 2024-05-30T16:48:04.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.069+0000 7f058ea00640 1 -- 172.21.15.67:0/3197662145 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0590004d00 con 0x7f059815c6b0 2024-05-30T16:48:04.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.069+0000 7f058ea00640 1 -- 172.21.15.67:0/3197662145 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059000aa90 con 0x7f059815c6b0 2024-05-30T16:48:04.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.069+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059814dd30 con 0x7f059815c6b0 2024-05-30T16:48:04.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.069+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981f5ee0 con 0x7f059815c6b0 2024-05-30T16:48:04.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.070+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05980689e0 con 0x7f059815c6b0 2024-05-30T16:48:04.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.071+0000 7f058ea00640 1 -- 172.21.15.67:0/3197662145 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f059000ac30 con 0x7f059815c6b0 2024-05-30T16:48:04.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.073+0000 7f058ea00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f057806cad0 0x7f057806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:04.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.073+0000 7f058ea00640 1 -- 172.21.15.67:0/3197662145 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f059008c120 con 0x7f059815c6b0 2024-05-30T16:48:04.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.073+0000 7f0596a00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f057806cad0 0x7f057806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:04.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.074+0000 7f0596a00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f057806cad0 0x7f057806ef90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0588034000 tx=0x7f058802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:04.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.078+0000 7f058ea00640 1 -- 172.21.15.67:0/3197662145 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0590020070 con 0x7f059815c6b0 2024-05-30T16:48:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.359+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f05980ca870 con 0x7f059815c6b0 2024-05-30T16:48:04.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.359+0000 7f058ea00640 1 -- 172.21.15.67:0/3197662145 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2077 (secure 0 0 0) 0x7f059814dd30 con 0x7f059815c6b0 2024-05-30T16:48:04.360 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:48:04.361 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 277, 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:48:04.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:04.363 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2859, 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 11871312164, 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 24005484544, 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 749088628736, 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 255, 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T16:48:04.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:04.365 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:04.366 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: "e5bd7e10-6021-4b29-bc64-f4d35d2e63df": { 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 3.0 seconds before exiting (30s)\n [============================] ", 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:04.367 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:48:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f057806cad0 msgr2=0x7f057806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f057806cad0 0x7f057806ef90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0588034000 tx=0x7f058802e000 comp rx=0 tx=0).stop 2024-05-30T16:48:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 msgr2=0x7f059806f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 0x7f059806f2d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f059000c780 tx=0x7f059000cc40 comp rx=0 tx=0).stop 2024-05-30T16:48:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 shutdown_connections 2024-05-30T16:48:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f057806cad0 0x7f057806ef90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f059815c6b0 0x7f059806f2d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05980d0460 0x7f05980c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 --2- 172.21.15.67:0/3197662145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05980ccc30 0x7f059814d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 >> 172.21.15.67:0/3197662145 conn(0x7f059806bae0 msgr2=0x7f05980cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 shutdown_connections 2024-05-30T16:48:04.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:04.362+0000 7f0597e00640 1 -- 172.21.15.67:0/3197662145 wait complete. 2024-05-30T16:48:09.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:48:09.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.602+0000 7f2db2a00640 1 Processor -- start 2024-05-30T16:48:09.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.603+0000 7f2db2a00640 1 -- start start 2024-05-30T16:48:09.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.604+0000 7f2db2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2dac0ccc30 0x7f2dac06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:09.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2db2a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dac0cd1a0 con 0x7f2dac15c6b0 2024-05-30T16:48:09.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2db2a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dac0cd480 con 0x7f2dac0d0460 2024-05-30T16:48:09.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2db2a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2dac0cd760 con 0x7f2dac0ccc30 2024-05-30T16:48:09.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2db1600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2dac15c6b0 0x7f2dac0b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59866/0 (socket says 172.21.15.67:59866) 2024-05-30T16:48:09.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2db1600640 1 -- 172.21.15.67:0/3096177647 learned_addr learned my addr 172.21.15.67:0/3096177647 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:09.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2db0c00640 1 --2- 172.21.15.67:0/3096177647 >> v2:172.21.15.67:3300/0 conn(0x7f2dac0ccc30 0x7f2dac06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:09.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2db0c00640 1 -- 172.21.15.67:0/3096177647 >> v1:172.21.15.169:6789/0 conn(0x7f2dac0d0460 legacy=0x7f2dac0cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:09.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2dabe00640 1 -- 172.21.15.67:0/3096177647 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2610979707 0 0) 0x7f2dac0cd1a0 con 0x7f2dac15c6b0 2024-05-30T16:48:09.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2db0c00640 1 -- 172.21.15.67:0/3096177647 >> v1:172.21.15.67:6789/0 conn(0x7f2dac15c6b0 legacy=0x7f2dac0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:09.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.605+0000 7f2db0c00640 1 -- 172.21.15.67:0/3096177647 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac0cda40 con 0x7f2dac0ccc30 2024-05-30T16:48:09.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.607+0000 7f2db0c00640 1 --2- 172.21.15.67:0/3096177647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 0x7f2dac06dc90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2d980097d0 tx=0x7f2d9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59879d7446c7839c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:09.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.607+0000 7f2dabe00640 1 -- 172.21.15.67:0/3096177647 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98037070 con 0x7f2dac0ccc30 2024-05-30T16:48:09.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.607+0000 7f2dabe00640 1 -- 172.21.15.67:0/3096177647 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d98032440 con 0x7f2dac0ccc30 2024-05-30T16:48:09.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.607+0000 7f2dabe00640 1 -- 172.21.15.67:0/3096177647 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98032810 con 0x7f2dac0ccc30 2024-05-30T16:48:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.608+0000 7f2db2a00640 1 -- 172.21.15.67:0/3096177647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 msgr2=0x7f2dac06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.608+0000 7f2db2a00640 1 --2- 172.21.15.67:0/3096177647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 0x7f2dac06dc90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2d980097d0 tx=0x7f2d9802a210 comp rx=0 tx=0).stop 2024-05-30T16:48:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.608+0000 7f2db2a00640 1 -- 172.21.15.67:0/3096177647 shutdown_connections 2024-05-30T16:48:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.608+0000 7f2db2a00640 1 --2- 172.21.15.67:0/3096177647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 0x7f2dac06dc90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.608+0000 7f2db2a00640 1 -- 172.21.15.67:0/3096177647 >> 172.21.15.67:0/3096177647 conn(0x7f2dac06bae0 msgr2=0x7f2dac06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.608+0000 7f2db2a00640 1 -- 172.21.15.67:0/3096177647 shutdown_connections 2024-05-30T16:48:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.608+0000 7f2db2a00640 1 -- 172.21.15.67:0/3096177647 wait complete. 2024-05-30T16:48:09.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:48:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.610+0000 7f2db2a00640 1 Processor -- start 2024-05-30T16:48:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.610+0000 7f2db2a00640 1 -- start start 2024-05-30T16:48:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 0x7f2dac0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dac0d0460 0x7f2dac1f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dac15c6b0 0x7f2dac0b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac06e350 con 0x7f2dac0ccc30 2024-05-30T16:48:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac06e1d0 con 0x7f2dac15c6b0 2024-05-30T16:48:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2dac0cc640 con 0x7f2dac0d0460 2024-05-30T16:48:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 0x7f2dac0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 0x7f2dac0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40728/0 (socket says 172.21.15.67:40728) 2024-05-30T16:48:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db1600640 1 -- 172.21.15.67:0/3102705206 learned_addr learned my addr 172.21.15.67:0/3102705206 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:09.615 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:48:09.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db1600640 1 -- 172.21.15.67:0/3102705206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dac0d0460 msgr2=0x7f2dac1f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.611+0000 7f2db1600640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dac0d0460 0x7f2dac1f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.612+0000 7f2db1600640 1 -- 172.21.15.67:0/3102705206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dac15c6b0 msgr2=0x7f2dac0b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.612+0000 7f2db1600640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dac15c6b0 0x7f2dac0b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.612+0000 7f2db1600640 1 -- 172.21.15.67:0/3102705206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac0c8c50 con 0x7f2dac0ccc30 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.612+0000 7f2db1600640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 0x7f2dac0c9cf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c430 tx=0x7f2da000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.612+0000 7f2da9600640 1 -- 172.21.15.67:0/3102705206 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da00074e0 con 0x7f2dac0ccc30 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.612+0000 7f2da9600640 1 -- 172.21.15.67:0/3102705206 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da00047e0 con 0x7f2dac0ccc30 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.612+0000 7f2da9600640 1 -- 172.21.15.67:0/3102705206 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0004bd0 con 0x7f2dac0ccc30 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.612+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dac0c8f30 con 0x7f2dac0ccc30 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.612+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dac0c94f0 con 0x7f2dac0ccc30 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.614+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dac0689e0 con 0x7f2dac0ccc30 2024-05-30T16:48:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.614+0000 7f2da9600640 1 -- 172.21.15.67:0/3102705206 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f2da0011070 con 0x7f2dac0ccc30 2024-05-30T16:48:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.616+0000 7f2da9600640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d8c06caa0 0x7f2d8c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.616+0000 7f2da9600640 1 -- 172.21.15.67:0/3102705206 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2da008bc80 con 0x7f2dac0ccc30 2024-05-30T16:48:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.616+0000 7f2db0c00640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d8c06caa0 0x7f2d8c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.617+0000 7f2db0c00640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d8c06caa0 0x7f2d8c06ef60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d98034000 tx=0x7f2d9802c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.621+0000 7f2da9600640 1 -- 172.21.15.67:0/3102705206 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2da0059d00 con 0x7f2dac0ccc30 2024-05-30T16:48:09.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.904+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2dac0afa00 con 0x7f2dac0ccc30 2024-05-30T16:48:09.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.905+0000 7f2da9600640 1 -- 172.21.15.67:0/3102705206 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1730 (secure 0 0 0) 0x7f2da0059600 con 0x7f2dac0ccc30 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:48:09.906 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 283, 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:09.907 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 35, 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:09.908 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2859, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 11871312164, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 24005484544, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 749088628736, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 2, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:48:09.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 2, 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 4821 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:48:09.910 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 6, 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:45:50.020907+0000", 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": {} 2024-05-30T16:48:09.911 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:48:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d8c06caa0 msgr2=0x7f2d8c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d8c06caa0 0x7f2d8c06ef60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d98034000 tx=0x7f2d9802c000 comp rx=0 tx=0).stop 2024-05-30T16:48:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 msgr2=0x7f2dac0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 0x7f2dac0c9cf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c430 tx=0x7f2da000c8f0 comp rx=0 tx=0).stop 2024-05-30T16:48:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 shutdown_connections 2024-05-30T16:48:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d8c06caa0 0x7f2d8c06ef60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dac15c6b0 0x7f2dac0b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dac0d0460 0x7f2dac1f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 --2- 172.21.15.67:0/3102705206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac0ccc30 0x7f2dac0c9cf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 >> 172.21.15.67:0/3102705206 conn(0x7f2dac06bae0 msgr2=0x7f2dac06bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.907+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 shutdown_connections 2024-05-30T16:48:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:09.908+0000 7f2db2a00640 1 -- 172.21.15.67:0/3102705206 wait complete. 2024-05-30T16:48:09.928 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-30T16:48:10.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.144+0000 7f7564a00640 1 Processor -- start 2024-05-30T16:48:10.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.144+0000 7f7564a00640 1 -- start start 2024-05-30T16:48:10.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.145+0000 7f7564a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75600cfa40 0x7f7560169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.145+0000 7f7564a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f756015ec50 con 0x7f756015e870 2024-05-30T16:48:10.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.145+0000 7f7564a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75600cff80 con 0x7f75600d16a0 2024-05-30T16:48:10.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.145+0000 7f7564a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f75600d2150 con 0x7f75600cfa40 2024-05-30T16:48:10.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.146+0000 7f755f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f756015e870 0x7f75600d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59876/0 (socket says 172.21.15.67:59876) 2024-05-30T16:48:10.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.146+0000 7f755f400640 1 -- 172.21.15.67:0/2700799155 learned_addr learned my addr 172.21.15.67:0/2700799155 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:10.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.146+0000 7f755fe00640 1 --2- 172.21.15.67:0/2700799155 >> v2:172.21.15.169:3300/0 conn(0x7f75600cfa40 0x7f7560169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.146+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 985034522 0 0) 0x7f756015ec50 con 0x7f756015e870 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.146+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7540003520 con 0x7f756015e870 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 449766637 0 0) 0x7f7540003520 con 0x7f756015e870 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 >> v2:172.21.15.169:3300/0 conn(0x7f75600cfa40 msgr2=0x7f7560169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 --2- 172.21.15.67:0/2700799155 >> v2:172.21.15.169:3300/0 conn(0x7f75600cfa40 0x7f7560169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 >> v1:172.21.15.169:6789/0 conn(0x7f75600d16a0 legacy=0x7f75600cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75600d0260 con 0x7f756015e870 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7550001670 con 0x7f756015e870 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 >> v1:172.21.15.67:6789/0 conn(0x7f756015e870 legacy=0x7f75600d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 --2- 172.21.15.67:0/2700799155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f7540003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 --2- 172.21.15.67:0/2700799155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7540004800 0x7f7540004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 --2- 172.21.15.67:0/2700799155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75400051c0 0x7f754000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f75600d2150 con 0x7f75400039b0 2024-05-30T16:48:10.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f754000c060 con 0x7f75400051c0 2024-05-30T16:48:10.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f754000c260 con 0x7f7540004800 2024-05-30T16:48:10.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.147+0000 7f755f400640 1 --2- 172.21.15.67:0/2700799155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f7540003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.148+0000 7f755f400640 1 -- 172.21.15.67:0/2700799155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7540004800 msgr2=0x7f7540004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:10.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.148+0000 7f755f400640 1 --2- 172.21.15.67:0/2700799155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7540004800 0x7f7540004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.148+0000 7f755f400640 1 -- 172.21.15.67:0/2700799155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75400051c0 msgr2=0x7f754000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:10.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.148+0000 7f755f400640 1 --2- 172.21.15.67:0/2700799155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75400051c0 0x7f754000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.148+0000 7f755f400640 1 -- 172.21.15.67:0/2700799155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7540003520 con 0x7f75400039b0 2024-05-30T16:48:10.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.148+0000 7f755fe00640 1 -- 172.21.15.67:0/2700799155 reap_dead start 2024-05-30T16:48:10.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.149+0000 7f755f400640 1 --2- 172.21.15.67:0/2700799155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f7540003dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f755000c210 tx=0x7f7540005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed8450e95dc0b39 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:10.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.149+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550007d30 con 0x7f75400039b0 2024-05-30T16:48:10.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.149+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7550005af0 con 0x7f75400039b0 2024-05-30T16:48:10.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.149+0000 7f755e000640 1 -- 172.21.15.67:0/2700799155 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550007070 con 0x7f75400039b0 2024-05-30T16:48:10.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.149+0000 7f7564a00640 1 -- 172.21.15.67:0/2700799155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 msgr2=0x7f7540003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.149+0000 7f7564a00640 1 --2- 172.21.15.67:0/2700799155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f7540003dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f755000c210 tx=0x7f7540005830 comp rx=0 tx=0).stop 2024-05-30T16:48:10.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.150+0000 7f7564a00640 1 -- 172.21.15.67:0/2700799155 shutdown_connections 2024-05-30T16:48:10.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.150+0000 7f7564a00640 1 --2- 172.21.15.67:0/2700799155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f7540003dd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.150+0000 7f7564a00640 1 -- 172.21.15.67:0/2700799155 >> 172.21.15.67:0/2700799155 conn(0x7f756008bfc0 msgr2=0x7f75600d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:10.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.150+0000 7f7564a00640 1 -- 172.21.15.67:0/2700799155 shutdown_connections 2024-05-30T16:48:10.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.150+0000 7f7564a00640 1 -- 172.21.15.67:0/2700799155 wait complete. 2024-05-30T16:48:10.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.151+0000 7f7564a00640 1 Processor -- start 2024-05-30T16:48:10.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.152+0000 7f7564a00640 1 -- start start 2024-05-30T16:48:10.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.152+0000 7f7564a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f75600b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.152+0000 7f7564a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600c9af0 0x7f75600c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f7564a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f756014c590 0x7f756014ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f7564a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f75600d4fb0 con 0x7f75400039b0 2024-05-30T16:48:10.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f7564a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75600d0820 con 0x7f75600c9af0 2024-05-30T16:48:10.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f7564a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f75600cf330 con 0x7f756014c590 2024-05-30T16:48:10.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f75600b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f75600b7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40748/0 (socket says 172.21.15.67:40748) 2024-05-30T16:48:10.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755f400640 1 -- 172.21.15.67:0/2341176411 learned_addr learned my addr 172.21.15.67:0/2341176411 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:10.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755ea00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600c9af0 0x7f75600c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755fe00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f756014c590 0x7f756014ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755f400640 1 -- 172.21.15.67:0/2341176411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f756014c590 msgr2=0x7f756014ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755f400640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f756014c590 0x7f756014ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755f400640 1 -- 172.21.15.67:0/2341176411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600c9af0 msgr2=0x7f75600c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755f400640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600c9af0 0x7f75600c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.153+0000 7f755f400640 1 -- 172.21.15.67:0/2341176411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75600cbf00 con 0x7f75400039b0 2024-05-30T16:48:10.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.154+0000 7f755f400640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f75600b7410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f755000c210 tx=0x7f7550001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:10.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.154+0000 7f7557400640 1 -- 172.21.15.67:0/2341176411 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550038040 con 0x7f75400039b0 2024-05-30T16:48:10.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.154+0000 7f7557400640 1 -- 172.21.15.67:0/2341176411 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7550042070 con 0x7f75400039b0 2024-05-30T16:48:10.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.154+0000 7f7557400640 1 -- 172.21.15.67:0/2341176411 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550007270 con 0x7f75400039b0 2024-05-30T16:48:10.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.154+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75600cc1e0 con 0x7f75400039b0 2024-05-30T16:48:10.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.154+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75600cc7a0 con 0x7f75400039b0 2024-05-30T16:48:10.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.155+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75600689e0 con 0x7f75400039b0 2024-05-30T16:48:10.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.156+0000 7f7557400640 1 -- 172.21.15.67:0/2341176411 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f7550007410 con 0x7f75400039b0 2024-05-30T16:48:10.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.157+0000 7f7557400640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f753c06cad0 0x7f753c06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.157+0000 7f7557400640 1 -- 172.21.15.67:0/2341176411 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f75500ac290 con 0x7f75400039b0 2024-05-30T16:48:10.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.157+0000 7f755ea00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f753c06cad0 0x7f753c06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.158+0000 7f755ea00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f753c06cad0 0x7f753c06ef90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f75600ba4d0 tx=0x7f7548009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:10.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.161+0000 7f7557400640 1 -- 172.21.15.67:0/2341176411 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f755007a430 con 0x7f75400039b0 2024-05-30T16:48:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.395+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75600cbcb0 con 0x7f753c06cad0 2024-05-30T16:48:10.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.406+0000 7f7557400640 1 -- 172.21.15.67:0/2341176411 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f75600cbcb0 con 0x7f753c06cad0 2024-05-30T16:48:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:10.407 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:48:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sv1" 2024-05-30T16:48:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1clone1" 2024-05-30T16:48:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:10.408 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:48:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f753c06cad0 msgr2=0x7f753c06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f753c06cad0 0x7f753c06ef90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f75600ba4d0 tx=0x7f7548009040 comp rx=0 tx=0).stop 2024-05-30T16:48:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 msgr2=0x7f75600b7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f75600b7410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f755000c210 tx=0x7f7550001a40 comp rx=0 tx=0).stop 2024-05-30T16:48:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 shutdown_connections 2024-05-30T16:48:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f753c06cad0 0x7f753c06ef90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f756014c590 0x7f756014ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75600c9af0 0x7f75600c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 --2- 172.21.15.67:0/2341176411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75400039b0 0x7f75600b7410 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.409+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 >> 172.21.15.67:0/2341176411 conn(0x7f756008bfc0 msgr2=0x7f75600b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.410+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 shutdown_connections 2024-05-30T16:48:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.410+0000 7f7564a00640 1 -- 172.21.15.67:0/2341176411 wait complete. 2024-05-30T16:48:10.428 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-30T16:48:10.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.637+0000 7fee31e00640 1 Processor -- start 2024-05-30T16:48:10.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.637+0000 7fee31e00640 1 -- start start 2024-05-30T16:48:10.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.638+0000 7fee31e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fee2c15fa00 0x7fee2c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.638+0000 7fee31e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c1600f0 0x7fee2c0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.638+0000 7fee31e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee2c0d0d90 con 0x7fee2c0d0970 2024-05-30T16:48:10.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.638+0000 7fee31e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fee2c06fa00 con 0x7fee2c15fa00 2024-05-30T16:48:10.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.638+0000 7fee31e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c06eeb0 con 0x7fee2c1600f0 2024-05-30T16:48:10.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.639+0000 7fee31400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fee2c0d0970 0x7fee2c0b3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59880/0 (socket says 172.21.15.67:59880) 2024-05-30T16:48:10.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.639+0000 7fee31400640 1 -- 172.21.15.67:0/4059037910 learned_addr learned my addr 172.21.15.67:0/4059037910 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:10.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.639+0000 7fee23e00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c1600f0 0x7fee2c0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.639+0000 7fee30a00640 1 --2- 172.21.15.67:0/4059037910 >> v2:172.21.15.169:3300/0 conn(0x7fee2c15fa00 0x7fee2c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.639+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4171381019 0 0) 0x7fee2c0d0d90 con 0x7fee2c0d0970 2024-05-30T16:48:10.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.639+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee10003520 con 0x7fee2c0d0970 2024-05-30T16:48:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2692013278 0 0) 0x7fee10003520 con 0x7fee2c0d0970 2024-05-30T16:48:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c1600f0 msgr2=0x7fee2c0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c1600f0 0x7fee2c0d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 >> v2:172.21.15.169:3300/0 conn(0x7fee2c15fa00 msgr2=0x7fee2c06e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 --2- 172.21.15.67:0/4059037910 >> v2:172.21.15.169:3300/0 conn(0x7fee2c15fa00 0x7fee2c06e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c06f160 con 0x7fee2c0d0970 2024-05-30T16:48:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fee24001670 con 0x7fee2c0d0970 2024-05-30T16:48:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 >> v1:172.21.15.67:6789/0 conn(0x7fee2c0d0970 legacy=0x7fee2c0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee100039b0 0x7fee10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c1600f0 0x7fee10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c06fa00 con 0x7fee10004d40 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c06eeb0 con 0x7fee100039b0 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fee1000ba40 con 0x7fee2c1600f0 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.640+0000 7fee31400640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.641+0000 7fee23e00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c1600f0 0x7fee10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.641+0000 7fee30a00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee100039b0 0x7fee10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.641+0000 7fee31400640 1 -- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c1600f0 msgr2=0x7fee10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.641+0000 7fee31400640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c1600f0 0x7fee10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.641+0000 7fee31400640 1 -- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee100039b0 msgr2=0x7fee10003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:10.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.641+0000 7fee31400640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee100039b0 0x7fee10003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.641+0000 7fee31400640 1 -- 172.21.15.67:0/4059037910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee10003520 con 0x7fee10004d40 2024-05-30T16:48:10.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.641+0000 7fee23e00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c1600f0 0x7fee10004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:10.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.641+0000 7fee30a00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee100039b0 0x7fee10003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:10.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.642+0000 7fee31400640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee100051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c190 tx=0x7fee10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65453823b1be14e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:10.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.642+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24007c80 con 0x7fee10004d40 2024-05-30T16:48:10.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.642+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee24005ad0 con 0x7fee10004d40 2024-05-30T16:48:10.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.642+0000 7fee2be00640 1 -- 172.21.15.67:0/4059037910 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24007070 con 0x7fee10004d40 2024-05-30T16:48:10.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.643+0000 7fee31e00640 1 -- 172.21.15.67:0/4059037910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 msgr2=0x7fee100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.643+0000 7fee31e00640 1 --2- 172.21.15.67:0/4059037910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee100051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c190 tx=0x7fee10005830 comp rx=0 tx=0).stop 2024-05-30T16:48:10.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.643+0000 7fee31400640 1 -- 172.21.15.67:0/4059037910 reap_dead start 2024-05-30T16:48:10.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.643+0000 7fee31e00640 1 -- 172.21.15.67:0/4059037910 shutdown_connections 2024-05-30T16:48:10.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.643+0000 7fee31e00640 1 -- 172.21.15.67:0/4059037910 >> 172.21.15.67:0/4059037910 conn(0x7fee2c06bae0 msgr2=0x7fee2c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:10.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.643+0000 7fee31e00640 1 -- 172.21.15.67:0/4059037910 shutdown_connections 2024-05-30T16:48:10.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.643+0000 7fee31e00640 1 -- 172.21.15.67:0/4059037910 wait complete. 2024-05-30T16:48:10.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.645+0000 7fee31e00640 1 Processor -- start 2024-05-30T16:48:10.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.645+0000 7fee31e00640 1 -- start start 2024-05-30T16:48:10.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.646+0000 7fee31e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee2c1f9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.646+0000 7fee31e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c154e30 0x7fee2c1fc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.646+0000 7fee31e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c1fc5b0 0x7fee2c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.646+0000 7fee31e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c071c70 con 0x7fee10004d40 2024-05-30T16:48:10.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.646+0000 7fee31e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c06f720 con 0x7fee2c1fc5b0 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.646+0000 7fee31e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c15fde0 con 0x7fee2c154e30 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.646+0000 7fee30a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee2c1f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.646+0000 7fee30a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee2c1f9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40768/0 (socket says 172.21.15.67:40768) 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.646+0000 7fee30a00640 1 -- 172.21.15.67:0/284744530 learned_addr learned my addr 172.21.15.67:0/284744530 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee30a00640 1 -- 172.21.15.67:0/284744530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c154e30 msgr2=0x7fee2c1fc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee30a00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c154e30 0x7fee2c1fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee30a00640 1 -- 172.21.15.67:0/284744530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c1fc5b0 msgr2=0x7fee2c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee30a00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c1fc5b0 0x7fee2c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee30a00640 1 -- 172.21.15.67:0/284744530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c1f8a80 con 0x7fee10004d40 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee30a00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee2c1f9540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00f400 tx=0x7fee1c00f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee29600640 1 -- 172.21.15.67:0/284744530 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c003990 con 0x7fee10004d40 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee29600640 1 -- 172.21.15.67:0/284744530 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee1c003b30 con 0x7fee10004d40 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2c1fcaf0 con 0x7fee10004d40 2024-05-30T16:48:10.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee29600640 1 -- 172.21.15.67:0/284744530 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c004020 con 0x7fee10004d40 2024-05-30T16:48:10.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.647+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2c1f8d40 con 0x7fee10004d40 2024-05-30T16:48:10.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.648+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2c0689e0 con 0x7fee10004d40 2024-05-30T16:48:10.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.649+0000 7fee29600640 1 -- 172.21.15.67:0/284744530 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fee1c01d030 con 0x7fee10004d40 2024-05-30T16:48:10.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.650+0000 7fee29600640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee0806cad0 0x7fee0806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:10.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.650+0000 7fee29600640 1 -- 172.21.15.67:0/284744530 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fee1c08c9d0 con 0x7fee10004d40 2024-05-30T16:48:10.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.651+0000 7fee23e00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee0806cad0 0x7fee0806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:10.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.651+0000 7fee23e00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee0806cad0 0x7fee0806ef90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0d0700 tx=0x7fee14008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:10.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.655+0000 7fee29600640 1 -- 172.21.15.67:0/284744530 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee1c024020 con 0x7fee10004d40 2024-05-30T16:48:10.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.891+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee2c0cda60 con 0x7fee0806cad0 2024-05-30T16:48:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.923+0000 7fee29600640 1 -- 172.21.15.67:0/284744530 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fee2c0cda60 con 0x7fee0806cad0 2024-05-30T16:48:10.924 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:10.924 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:48:10.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:10.924 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1" 2024-05-30T16:48:10.925 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:10.925 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:48:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee0806cad0 msgr2=0x7fee0806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee0806cad0 0x7fee0806ef90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0d0700 tx=0x7fee14008040 comp rx=0 tx=0).stop 2024-05-30T16:48:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 msgr2=0x7fee2c1f9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee2c1f9540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00f400 tx=0x7fee1c00f8c0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 shutdown_connections 2024-05-30T16:48:10.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee0806cad0 0x7fee0806ef90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c1fc5b0 0x7fee2c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee2c154e30 0x7fee2c1fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 --2- 172.21.15.67:0/284744530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee10004d40 0x7fee2c1f9540 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:10.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.926+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 >> 172.21.15.67:0/284744530 conn(0x7fee2c06bae0 msgr2=0x7fee2c1604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:10.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.927+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 shutdown_connections 2024-05-30T16:48:10.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:10.927+0000 7fee31e00640 1 -- 172.21.15.67:0/284744530 wait complete. 2024-05-30T16:48:10.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-30T16:48:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.160+0000 7f1842a00640 1 Processor -- start 2024-05-30T16:48:11.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.160+0000 7f1842a00640 1 -- start start 2024-05-30T16:48:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.162+0000 7f1842a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f183c0cae20 0x7f183c151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.162+0000 7f1842a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f183c1522f0 0x7f183c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.162+0000 7f1842a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183c0b3e80 con 0x7f183c0caa40 2024-05-30T16:48:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.162+0000 7f1842a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f183c0d0b30 con 0x7f183c0cae20 2024-05-30T16:48:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.162+0000 7f1842a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f183c0b4160 con 0x7f183c1522f0 2024-05-30T16:48:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.162+0000 7f1840c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f183c0cae20 0x7f183c151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.162+0000 7f1840c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f183c0cae20 0x7f183c151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38860/0 (socket says 172.21.15.67:38860) 2024-05-30T16:48:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.162+0000 7f1840c00640 1 -- 172.21.15.67:0/2351830073 learned_addr learned my addr 172.21.15.67:0/2351830073 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.162+0000 7f1842000640 1 --2- 172.21.15.67:0/2351830073 >> v2:172.21.15.169:3300/0 conn(0x7f183c1522f0 0x7f183c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.163+0000 7f1840c00640 1 -- 172.21.15.67:0/2351830073 >> v2:172.21.15.169:3300/0 conn(0x7f183c1522f0 msgr2=0x7f183c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.163+0000 7f1840c00640 1 --2- 172.21.15.67:0/2351830073 >> v2:172.21.15.169:3300/0 conn(0x7f183c1522f0 0x7f183c0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.163+0000 7f1840c00640 1 -- 172.21.15.67:0/2351830073 >> v1:172.21.15.169:6789/0 conn(0x7f183c0caa40 legacy=0x7f183c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.163+0000 7f1840c00640 1 -- 172.21.15.67:0/2351830073 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c0cde80 con 0x7f183c0cae20 2024-05-30T16:48:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.164+0000 7f1840c00640 1 --2- 172.21.15.67:0/2351830073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0cae20 0x7f183c151db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f183c06c8b0 tx=0x7f182802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4985afbaa9928d6c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.165+0000 7f183be00640 1 -- 172.21.15.67:0/2351830073 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828037070 con 0x7f183c0cae20 2024-05-30T16:48:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.165+0000 7f183be00640 1 -- 172.21.15.67:0/2351830073 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1828032440 con 0x7f183c0cae20 2024-05-30T16:48:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.165+0000 7f183be00640 1 -- 172.21.15.67:0/2351830073 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828032810 con 0x7f183c0cae20 2024-05-30T16:48:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.165+0000 7f1842a00640 1 -- 172.21.15.67:0/2351830073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0cae20 msgr2=0x7f183c151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.166+0000 7f1842a00640 1 --2- 172.21.15.67:0/2351830073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0cae20 0x7f183c151db0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f183c06c8b0 tx=0x7f182802a1a0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.166+0000 7f1842a00640 1 -- 172.21.15.67:0/2351830073 shutdown_connections 2024-05-30T16:48:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.166+0000 7f1842a00640 1 --2- 172.21.15.67:0/2351830073 >> v2:172.21.15.169:3300/0 conn(0x7f183c1522f0 0x7f183c0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.166+0000 7f1842a00640 1 --2- 172.21.15.67:0/2351830073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0cae20 0x7f183c151db0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.166+0000 7f1842a00640 1 -- 172.21.15.67:0/2351830073 >> 172.21.15.67:0/2351830073 conn(0x7f183c06b900 msgr2=0x7f183c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.166+0000 7f1842a00640 1 -- 172.21.15.67:0/2351830073 shutdown_connections 2024-05-30T16:48:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.166+0000 7f1842a00640 1 -- 172.21.15.67:0/2351830073 wait complete. 2024-05-30T16:48:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.168+0000 7f1842a00640 1 Processor -- start 2024-05-30T16:48:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.168+0000 7f1842a00640 1 -- start start 2024-05-30T16:48:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.169+0000 7f1842a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0caa40 0x7f183c0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.169+0000 7f1842a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0cae20 0x7f183c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.169+0000 7f1842a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f183c1522f0 0x7f183c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.169+0000 7f1842a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0ce440 con 0x7f183c0caa40 2024-05-30T16:48:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.169+0000 7f1842a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0cf070 con 0x7f183c0cae20 2024-05-30T16:48:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.169+0000 7f1842a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f183c152710 con 0x7f183c1522f0 2024-05-30T16:48:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.169+0000 7f1841600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0caa40 0x7f183c0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.169+0000 7f1841600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0caa40 0x7f183c0b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38870/0 (socket says 172.21.15.67:38870) 2024-05-30T16:48:11.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.169+0000 7f1841600640 1 -- 172.21.15.67:0/3175396034 learned_addr learned my addr 172.21.15.67:0/3175396034 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:11.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1840c00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0cae20 0x7f183c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1842000640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f183c1522f0 0x7f183c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1841600640 1 -- 172.21.15.67:0/3175396034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f183c1522f0 msgr2=0x7f183c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1841600640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f183c1522f0 0x7f183c0af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1841600640 1 -- 172.21.15.67:0/3175396034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0cae20 msgr2=0x7f183c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1841600640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0cae20 0x7f183c0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1841600640 1 -- 172.21.15.67:0/3175396034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c0b0530 con 0x7f183c0caa40 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1840c00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0cae20 0x7f183c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1841600640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0caa40 0x7f183c0b15d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f183000c430 tx=0x7f183000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1839600640 1 -- 172.21.15.67:0/3175396034 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830011790 con 0x7f183c0caa40 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1839600640 1 -- 172.21.15.67:0/3175396034 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1830017070 con 0x7f183c0caa40 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1839600640 1 -- 172.21.15.67:0/3175396034 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830011bb0 con 0x7f183c0caa40 2024-05-30T16:48:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.170+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c0b0810 con 0x7f183c0caa40 2024-05-30T16:48:11.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.171+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c0b0dd0 con 0x7f183c0caa40 2024-05-30T16:48:11.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.172+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c0689e0 con 0x7f183c0caa40 2024-05-30T16:48:11.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.173+0000 7f1839600640 1 -- 172.21.15.67:0/3175396034 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f183001c460 con 0x7f183c0caa40 2024-05-30T16:48:11.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.174+0000 7f1839600640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f181c06caa0 0x7f181c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.174+0000 7f1839600640 1 -- 172.21.15.67:0/3175396034 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f183008bd90 con 0x7f183c0caa40 2024-05-30T16:48:11.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.174+0000 7f1840c00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f181c06caa0 0x7f181c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.175+0000 7f1840c00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f181c06caa0 0x7f181c06ef60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1828034000 tx=0x7f182802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:11.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.178+0000 7f1839600640 1 -- 172.21.15.67:0/3175396034 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1830059f30 con 0x7f183c0caa40 2024-05-30T16:48:11.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.420+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f183c0b4ee0 con 0x7f181c06caa0 2024-05-30T16:48:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.490+0000 7f1839600640 1 -- 172.21.15.67:0/3175396034 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f183c0b4ee0 con 0x7f181c06caa0 2024-05-30T16:48:11.490 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:11.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f181c06caa0 msgr2=0x7f181c06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:11.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f181c06caa0 0x7f181c06ef60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1828034000 tx=0x7f182802e000 comp rx=0 tx=0).stop 2024-05-30T16:48:11.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0caa40 msgr2=0x7f183c0b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:11.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0caa40 0x7f183c0b15d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f183000c430 tx=0x7f183000c8f0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 shutdown_connections 2024-05-30T16:48:11.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f181c06caa0 0x7f181c06ef60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f183c1522f0 0x7f183c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0cae20 0x7f183c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 --2- 172.21.15.67:0/3175396034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c0caa40 0x7f183c0b15d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 >> 172.21.15.67:0/3175396034 conn(0x7f183c06b900 msgr2=0x7f183c15a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 shutdown_connections 2024-05-30T16:48:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.493+0000 7f1842a00640 1 -- 172.21.15.67:0/3175396034 wait complete. 2024-05-30T16:48:11.515 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-30T16:48:11.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.731+0000 7f3c57e00640 1 Processor -- start 2024-05-30T16:48:11.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.732+0000 7f3c57e00640 1 -- start start 2024-05-30T16:48:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.733+0000 7f3c57e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c5815e870 0x7f3c580d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.733+0000 7f3c57e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3c580d14d0 0x7f3c580cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.733+0000 7f3c57e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c580cf870 0x7f3c58162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.733+0000 7f3c57e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f3c580d2150 con 0x7f3c5815e870 2024-05-30T16:48:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.733+0000 7f3c57e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3c580cfd00 con 0x7f3c580d14d0 2024-05-30T16:48:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.733+0000 7f3c57e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5815ec50 con 0x7f3c580cf870 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c5815e870 0x7f3c580d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c5815e870 0x7f3c580d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38874/0 (socket says 172.21.15.67:38874) 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56a00640 1 -- 172.21.15.67:0/3863592238 learned_addr learned my addr 172.21.15.67:0/3863592238 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56a00640 1 -- 172.21.15.67:0/3863592238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c580cf870 msgr2=0x7f3c58162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56a00640 1 --2- 172.21.15.67:0/3863592238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c580cf870 0x7f3c58162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56000640 1 --2- 172.21.15.67:0/3863592238 >> v2:172.21.15.169:3300/0 conn(0x7f3c580d14d0 0x7f3c580cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56a00640 1 -- 172.21.15.67:0/3863592238 >> v2:172.21.15.169:3300/0 conn(0x7f3c580d14d0 msgr2=0x7f3c580cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56a00640 1 --2- 172.21.15.67:0/3863592238 >> v2:172.21.15.169:3300/0 conn(0x7f3c580d14d0 0x7f3c580cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56a00640 1 -- 172.21.15.67:0/3863592238 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c580cff80 con 0x7f3c5815e870 2024-05-30T16:48:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.734+0000 7f3c56000640 1 --2- 172.21.15.67:0/3863592238 >> v2:172.21.15.169:3300/0 conn(0x7f3c580d14d0 0x7f3c580cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.735+0000 7f3c56a00640 1 --2- 172.21.15.67:0/3863592238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c5815e870 0x7f3c580d0f90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3c5808e520 tx=0x7f3c4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6b987bf022dddb server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.736+0000 7f3c55600640 1 -- 172.21.15.67:0/3863592238 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48037070 con 0x7f3c5815e870 2024-05-30T16:48:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.736+0000 7f3c55600640 1 -- 172.21.15.67:0/3863592238 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c48032440 con 0x7f3c5815e870 2024-05-30T16:48:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.736+0000 7f3c55600640 1 -- 172.21.15.67:0/3863592238 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48032810 con 0x7f3c5815e870 2024-05-30T16:48:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.737+0000 7f3c57e00640 1 -- 172.21.15.67:0/3863592238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c5815e870 msgr2=0x7f3c580d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.737+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3863592238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c5815e870 0x7f3c580d0f90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3c5808e520 tx=0x7f3c4802a1e0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.737+0000 7f3c57e00640 1 -- 172.21.15.67:0/3863592238 shutdown_connections 2024-05-30T16:48:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.737+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3863592238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c580cf870 0x7f3c58162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.737+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3863592238 >> v2:172.21.15.169:3300/0 conn(0x7f3c580d14d0 0x7f3c580cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.737+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3863592238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c5815e870 0x7f3c580d0f90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.737+0000 7f3c57e00640 1 -- 172.21.15.67:0/3863592238 >> 172.21.15.67:0/3863592238 conn(0x7f3c5808bfc0 msgr2=0x7f3c580d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.737+0000 7f3c57e00640 1 -- 172.21.15.67:0/3863592238 shutdown_connections 2024-05-30T16:48:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.737+0000 7f3c57e00640 1 -- 172.21.15.67:0/3863592238 wait complete. 2024-05-30T16:48:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.739+0000 7f3c57e00640 1 Processor -- start 2024-05-30T16:48:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.740+0000 7f3c57e00640 1 -- start start 2024-05-30T16:48:11.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.741+0000 7f3c57e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c580cf870 0x7f3c5814f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.741+0000 7f3c57e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c580d14d0 0x7f3c580b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.741+0000 7f3c57e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c5815e870 0x7f3c580b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.741+0000 7f3c57e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3c580d0820 con 0x7f3c580cf870 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.741+0000 7f3c57e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c580d0540 con 0x7f3c5815e870 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.741+0000 7f3c57e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3c580d4fb0 con 0x7f3c580d14d0 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.741+0000 7f3c56a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c580cf870 0x7f3c5814f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.741+0000 7f3c56a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c580cf870 0x7f3c5814f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38884/0 (socket says 172.21.15.67:38884) 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.741+0000 7f3c56a00640 1 -- 172.21.15.67:0/3200050572 learned_addr learned my addr 172.21.15.67:0/3200050572 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.742+0000 7f3c57400640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c5815e870 0x7f3c580b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.742+0000 7f3c56000640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c580d14d0 0x7f3c580b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.742+0000 7f3c56a00640 1 -- 172.21.15.67:0/3200050572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c580d14d0 msgr2=0x7f3c580b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.742+0000 7f3c56a00640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c580d14d0 0x7f3c580b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.742+0000 7f3c56a00640 1 -- 172.21.15.67:0/3200050572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c5815e870 msgr2=0x7f3c580b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.742+0000 7f3c56a00640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c5815e870 0x7f3c580b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.742+0000 7f3c56a00640 1 -- 172.21.15.67:0/3200050572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5814ff10 con 0x7f3c580cf870 2024-05-30T16:48:11.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.742+0000 7f3c56a00640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c580cf870 0x7f3c5814f9d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3c480097c0 tx=0x7f3c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:11.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.743+0000 7f3c4f400640 1 -- 172.21.15.67:0/3200050572 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48037040 con 0x7f3c580cf870 2024-05-30T16:48:11.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.743+0000 7f3c4f400640 1 -- 172.21.15.67:0/3200050572 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c48042070 con 0x7f3c580cf870 2024-05-30T16:48:11.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.743+0000 7f3c4f400640 1 -- 172.21.15.67:0/3200050572 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48030bb0 con 0x7f3c580cf870 2024-05-30T16:48:11.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.743+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c580b98e0 con 0x7f3c580cf870 2024-05-30T16:48:11.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.743+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5815f140 con 0x7f3c580cf870 2024-05-30T16:48:11.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.745+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c580689e0 con 0x7f3c580cf870 2024-05-30T16:48:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.746+0000 7f3c4f400640 1 -- 172.21.15.67:0/3200050572 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f3c48047460 con 0x7f3c580cf870 2024-05-30T16:48:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.747+0000 7f3c4f400640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3c3806caa0 0x7f3c3806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.747+0000 7f3c4f400640 1 -- 172.21.15.67:0/3200050572 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3c480acaa0 con 0x7f3c580cf870 2024-05-30T16:48:11.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.747+0000 7f3c56000640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3c3806caa0 0x7f3c3806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:11.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.748+0000 7f3c56000640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3c3806caa0 0x7f3c3806ef60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c44007560 tx=0x7f3c440074f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:11.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.752+0000 7f3c4f400640 1 -- 172.21.15.67:0/3200050572 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c5814ff10 con 0x7f3c580cf870 2024-05-30T16:48:11.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:11.986+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f3c5806e070 con 0x7f3c3806caa0 2024-05-30T16:48:12.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.030+0000 7f3c4f400640 1 -- 172.21.15.67:0/3200050572 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3c5806e070 con 0x7f3c3806caa0 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3c3806caa0 msgr2=0x7f3c3806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3c3806caa0 0x7f3c3806ef60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c44007560 tx=0x7f3c440074f0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c580cf870 msgr2=0x7f3c5814f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c580cf870 0x7f3c5814f9d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3c480097c0 tx=0x7f3c480046c0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 shutdown_connections 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3c3806caa0 0x7f3c3806ef60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c5815e870 0x7f3c580b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c580d14d0 0x7f3c580b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 --2- 172.21.15.67:0/3200050572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c580cf870 0x7f3c5814f9d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 >> 172.21.15.67:0/3200050572 conn(0x7f3c5808bfc0 msgr2=0x7f3c580b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:12.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 shutdown_connections 2024-05-30T16:48:12.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.034+0000 7f3c57e00640 1 -- 172.21.15.67:0/3200050572 wait complete. 2024-05-30T16:48:12.039 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.038+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.039 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.038+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.039 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.038+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.039 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.038+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.039 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.038+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-30T16:48:12.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.069+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.069+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.069+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.069+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:12.069+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.250+0000 7f25fb200640 1 Processor -- start 2024-05-30T16:48:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.250+0000 7f25fb200640 1 -- start start 2024-05-30T16:48:12.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.252+0000 7f25fb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25fc15e5a0 0x7f25fc0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.252+0000 7f25fb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f25fc0cfb40 0x7f25fc0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.253+0000 7f25fb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25fc0d3500 con 0x7f25fc0cffd0 2024-05-30T16:48:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.253+0000 7f25fb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f25fc0d2b30 con 0x7f25fc15e5a0 2024-05-30T16:48:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.253+0000 7f25fb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f25fc0d2200 con 0x7f25fc0cfb40 2024-05-30T16:48:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.254+0000 7f25f9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25fc15e5a0 0x7f25fc0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.254+0000 7f25f9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25fc15e5a0 0x7f25fc0d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38924/0 (socket says 172.21.15.67:38924) 2024-05-30T16:48:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.254+0000 7f25f9e00640 1 -- 172.21.15.67:0/3856147867 learned_addr learned my addr 172.21.15.67:0/3856147867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.254+0000 7f25f9400640 1 --2- 172.21.15.67:0/3856147867 >> v2:172.21.15.169:3300/0 conn(0x7f25fc0cfb40 0x7f25fc0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.254+0000 7f25f9e00640 1 -- 172.21.15.67:0/3856147867 >> v2:172.21.15.169:3300/0 conn(0x7f25fc0cfb40 msgr2=0x7f25fc0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.255+0000 7f25f9e00640 1 --2- 172.21.15.67:0/3856147867 >> v2:172.21.15.169:3300/0 conn(0x7f25fc0cfb40 0x7f25fc0d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.255+0000 7f25f9e00640 1 -- 172.21.15.67:0/3856147867 >> v1:172.21.15.67:6789/0 conn(0x7f25fc0cffd0 legacy=0x7f25fc06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.255+0000 7f25f9e00640 1 -- 172.21.15.67:0/3856147867 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc0d24e0 con 0x7f25fc15e5a0 2024-05-30T16:48:12.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.257+0000 7f25f9e00640 1 --2- 172.21.15.67:0/3856147867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc15e5a0 0x7f25fc0d2fc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f25fc073fa0 tx=0x7f25ec02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16f7d7d3774f682 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:12.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.258+0000 7f25f8a00640 1 -- 172.21.15.67:0/3856147867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec037070 con 0x7f25fc15e5a0 2024-05-30T16:48:12.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.259+0000 7f25f8a00640 1 -- 172.21.15.67:0/3856147867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ec032440 con 0x7f25fc15e5a0 2024-05-30T16:48:12.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.259+0000 7f25f8a00640 1 -- 172.21.15.67:0/3856147867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec032850 con 0x7f25fc15e5a0 2024-05-30T16:48:12.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.259+0000 7f25fb200640 1 -- 172.21.15.67:0/3856147867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc15e5a0 msgr2=0x7f25fc0d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:12.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.259+0000 7f25fb200640 1 --2- 172.21.15.67:0/3856147867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc15e5a0 0x7f25fc0d2fc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f25fc073fa0 tx=0x7f25ec02a250 comp rx=0 tx=0).stop 2024-05-30T16:48:12.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.259+0000 7f25fb200640 1 -- 172.21.15.67:0/3856147867 shutdown_connections 2024-05-30T16:48:12.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.260+0000 7f25fb200640 1 --2- 172.21.15.67:0/3856147867 >> v2:172.21.15.169:3300/0 conn(0x7f25fc0cfb40 0x7f25fc0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.260+0000 7f25fb200640 1 --2- 172.21.15.67:0/3856147867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc15e5a0 0x7f25fc0d2fc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.260+0000 7f25fb200640 1 -- 172.21.15.67:0/3856147867 >> 172.21.15.67:0/3856147867 conn(0x7f25fc08bfc0 msgr2=0x7f25fc0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.260+0000 7f25fb200640 1 -- 172.21.15.67:0/3856147867 shutdown_connections 2024-05-30T16:48:12.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.260+0000 7f25fb200640 1 -- 172.21.15.67:0/3856147867 wait complete. 2024-05-30T16:48:12.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.261+0000 7f25fb200640 1 Processor -- start 2024-05-30T16:48:12.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.262+0000 7f25fb200640 1 -- start start 2024-05-30T16:48:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.263+0000 7f25fb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc0cfb40 0x7f25fc1f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.263+0000 7f25fb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc0cffd0 0x7f25fc0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.263+0000 7f25fb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc15e5a0 0x7f25fc0cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.263+0000 7f25fb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc15eb00 con 0x7f25fc0cffd0 2024-05-30T16:48:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.263+0000 7f25fb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc15e980 con 0x7f25fc0cfb40 2024-05-30T16:48:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.263+0000 7f25fb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25fc0d5220 con 0x7f25fc15e5a0 2024-05-30T16:48:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.264+0000 7f25f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc0cffd0 0x7f25fc0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.264+0000 7f25f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc0cffd0 0x7f25fc0cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38930/0 (socket says 172.21.15.67:38930) 2024-05-30T16:48:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.264+0000 7f25f9400640 1 -- 172.21.15.67:0/401553427 learned_addr learned my addr 172.21.15.67:0/401553427 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.264+0000 7f25f9400640 1 -- 172.21.15.67:0/401553427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc15e5a0 msgr2=0x7f25fc0cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.264+0000 7f25f9e00640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc0cfb40 0x7f25fc1f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.264+0000 7f25f9400640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc15e5a0 0x7f25fc0cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.264+0000 7f25f9400640 1 -- 172.21.15.67:0/401553427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc0cfb40 msgr2=0x7f25fc1f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.264+0000 7f25f9400640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc0cfb40 0x7f25fc1f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.264+0000 7f25f9400640 1 -- 172.21.15.67:0/401553427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc0cde30 con 0x7f25fc0cffd0 2024-05-30T16:48:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.265+0000 7f25f9400640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc0cffd0 0x7f25fc0cb580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f25e800d800 tx=0x7f25e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.265+0000 7f25f2000640 1 -- 172.21.15.67:0/401553427 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e800b6c0 con 0x7f25fc0cffd0 2024-05-30T16:48:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.265+0000 7f25f2000640 1 -- 172.21.15.67:0/401553427 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e8019070 con 0x7f25fc0cffd0 2024-05-30T16:48:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.265+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc0ce110 con 0x7f25fc0cffd0 2024-05-30T16:48:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.265+0000 7f25f2000640 1 -- 172.21.15.67:0/401553427 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e800ba80 con 0x7f25fc0cffd0 2024-05-30T16:48:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.265+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc0ce6d0 con 0x7f25fc0cffd0 2024-05-30T16:48:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.266+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc0689e0 con 0x7f25fc0cffd0 2024-05-30T16:48:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.267+0000 7f25f2000640 1 -- 172.21.15.67:0/401553427 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f25e800bc20 con 0x7f25fc0cffd0 2024-05-30T16:48:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.268+0000 7f25f2000640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25dc06cbc0 0x7f25dc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.268+0000 7f25f2000640 1 -- 172.21.15.67:0/401553427 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f25e808c4a0 con 0x7f25fc0cffd0 2024-05-30T16:48:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.269+0000 7f25f9e00640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25dc06cbc0 0x7f25dc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:12.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.269+0000 7f25f9e00640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25dc06cbc0 0x7f25dc06f080 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25ec034000 tx=0x7f25ec02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:12.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.273+0000 7f25f2000640 1 -- 172.21.15.67:0/401553427 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25e805a580 con 0x7f25fc0cffd0 2024-05-30T16:48:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.507+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f25fc0b1220 con 0x7f25dc06cbc0 2024-05-30T16:48:12.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.550+0000 7f25f2000640 1 -- 172.21.15.67:0/401553427 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25fc0b1220 con 0x7f25dc06cbc0 2024-05-30T16:48:12.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25dc06cbc0 msgr2=0x7f25dc06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:12.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25dc06cbc0 0x7f25dc06f080 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25ec034000 tx=0x7f25ec02e000 comp rx=0 tx=0).stop 2024-05-30T16:48:12.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc0cffd0 msgr2=0x7f25fc0cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:12.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc0cffd0 0x7f25fc0cb580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f25e800d800 tx=0x7f25e800dcc0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 shutdown_connections 2024-05-30T16:48:12.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25dc06cbc0 0x7f25dc06f080 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25fc15e5a0 0x7f25fc0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25fc0cffd0 0x7f25fc0cb580 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 --2- 172.21.15.67:0/401553427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25fc0cfb40 0x7f25fc1f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:12.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.553+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 >> 172.21.15.67:0/401553427 conn(0x7f25fc08bfc0 msgr2=0x7f25fc06de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:12.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.554+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 shutdown_connections 2024-05-30T16:48:12.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12.554+0000 7f25fb200640 1 -- 172.21.15.67:0/401553427 wait complete. 2024-05-30T16:48:12.575 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-30T16:48:12.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-30T16:48:12.729 INFO:teuthology.orchestra.run.smithi067.stdout:2 2024-05-30T16:48:12.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:48:12.739 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T16:48:12.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:48:12.849 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:48:12.959 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T16:48:12.959 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:48:12.959 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:48:12.959 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:13.014 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:48:13.014 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:13.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:48:13.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.304+0000 7f5d2f200640 1 Processor -- start 2024-05-30T16:48:13.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.305+0000 7f5d2f200640 1 -- start start 2024-05-30T16:48:13.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.306+0000 7f5d2f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5d3006ee40 0x7f5d300ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:13.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.306+0000 7f5d2f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d3006f2f0 0x7f5d300ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:13.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.306+0000 7f5d2f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d30156f10 con 0x7f5d30156b30 2024-05-30T16:48:13.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.306+0000 7f5d2f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5d3006dda0 con 0x7f5d3006ee40 2024-05-30T16:48:13.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.306+0000 7f5d2f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5d3006e080 con 0x7f5d3006f2f0 2024-05-30T16:48:13.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.306+0000 7f5d25400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5d3006ee40 0x7f5d300ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:13.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.306+0000 7f5d25400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5d3006ee40 0x7f5d300ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38946/0 (socket says 172.21.15.67:38946) 2024-05-30T16:48:13.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.306+0000 7f5d25400640 1 -- 172.21.15.67:0/901165846 learned_addr learned my addr 172.21.15.67:0/901165846 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:13.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.307+0000 7f5d2e800640 1 --2- 172.21.15.67:0/901165846 >> v2:172.21.15.169:3300/0 conn(0x7f5d3006f2f0 0x7f5d300ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:13.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.307+0000 7f5d25400640 1 -- 172.21.15.67:0/901165846 >> v2:172.21.15.169:3300/0 conn(0x7f5d3006f2f0 msgr2=0x7f5d300ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:13.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.307+0000 7f5d25400640 1 --2- 172.21.15.67:0/901165846 >> v2:172.21.15.169:3300/0 conn(0x7f5d3006f2f0 0x7f5d300ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.307+0000 7f5d25400640 1 -- 172.21.15.67:0/901165846 >> v1:172.21.15.169:6789/0 conn(0x7f5d30156b30 legacy=0x7f5d300af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:13.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.307+0000 7f5d25400640 1 -- 172.21.15.67:0/901165846 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3006e360 con 0x7f5d3006ee40 2024-05-30T16:48:13.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.308+0000 7f5d25400640 1 --2- 172.21.15.67:0/901165846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006ee40 0x7f5d300ad230 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5d300d0180 tx=0x7f5d1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c62b919597d7b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:13.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.308+0000 7f5d2d400640 1 -- 172.21.15.67:0/901165846 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18037070 con 0x7f5d3006ee40 2024-05-30T16:48:13.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.308+0000 7f5d2d400640 1 -- 172.21.15.67:0/901165846 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d18032440 con 0x7f5d3006ee40 2024-05-30T16:48:13.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.308+0000 7f5d2d400640 1 -- 172.21.15.67:0/901165846 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18032810 con 0x7f5d3006ee40 2024-05-30T16:48:13.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.309+0000 7f5d2f200640 1 -- 172.21.15.67:0/901165846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006ee40 msgr2=0x7f5d300ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:13.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.309+0000 7f5d2f200640 1 --2- 172.21.15.67:0/901165846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006ee40 0x7f5d300ad230 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5d300d0180 tx=0x7f5d1802a1a0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.309+0000 7f5d2f200640 1 -- 172.21.15.67:0/901165846 shutdown_connections 2024-05-30T16:48:13.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.309+0000 7f5d2f200640 1 --2- 172.21.15.67:0/901165846 >> v2:172.21.15.169:3300/0 conn(0x7f5d3006f2f0 0x7f5d300ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.309+0000 7f5d2f200640 1 --2- 172.21.15.67:0/901165846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006ee40 0x7f5d300ad230 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.309+0000 7f5d2f200640 1 -- 172.21.15.67:0/901165846 >> 172.21.15.67:0/901165846 conn(0x7f5d300b1fc0 msgr2=0x7f5d300b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:13.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.309+0000 7f5d2f200640 1 -- 172.21.15.67:0/901165846 shutdown_connections 2024-05-30T16:48:13.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.309+0000 7f5d2f200640 1 -- 172.21.15.67:0/901165846 wait complete. 2024-05-30T16:48:13.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.311+0000 7f5d2f200640 1 Processor -- start 2024-05-30T16:48:13.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.311+0000 7f5d2f200640 1 -- start start 2024-05-30T16:48:13.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.312+0000 7f5d2f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d3006ee40 0x7f5d301476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:13.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.312+0000 7f5d2f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006f2f0 0x7f5d30158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:13.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.312+0000 7f5d2f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d30156b30 0x7f5d30159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:13.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.312+0000 7f5d2f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d300b2db0 con 0x7f5d3006f2f0 2024-05-30T16:48:13.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.312+0000 7f5d2f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d300b2c30 con 0x7f5d30156b30 2024-05-30T16:48:13.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.312+0000 7f5d2f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d300b32c0 con 0x7f5d3006ee40 2024-05-30T16:48:13.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.312+0000 7f5d25400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006f2f0 0x7f5d30158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:13.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.312+0000 7f5d25400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006f2f0 0x7f5d30158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38948/0 (socket says 172.21.15.67:38948) 2024-05-30T16:48:13.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.312+0000 7f5d25400640 1 -- 172.21.15.67:0/3950328788 learned_addr learned my addr 172.21.15.67:0/3950328788 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:13.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d25400640 1 -- 172.21.15.67:0/3950328788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d3006ee40 msgr2=0x7f5d301476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d25400640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d3006ee40 0x7f5d301476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d25400640 1 -- 172.21.15.67:0/3950328788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d30156b30 msgr2=0x7f5d30159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d25400640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d30156b30 0x7f5d30159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d25400640 1 -- 172.21.15.67:0/3950328788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d30153030 con 0x7f5d3006f2f0 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d25400640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006f2f0 0x7f5d30158d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5d18000c00 tx=0x7f5d180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d26a00640 1 -- 172.21.15.67:0/3950328788 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18041020 con 0x7f5d3006f2f0 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d26a00640 1 -- 172.21.15.67:0/3950328788 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d180306c0 con 0x7f5d3006f2f0 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d26a00640 1 -- 172.21.15.67:0/3950328788 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18037040 con 0x7f5d3006f2f0 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d30153310 con 0x7f5d3006f2f0 2024-05-30T16:48:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.313+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d301538d0 con 0x7f5d3006f2f0 2024-05-30T16:48:13.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.314+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d300689e0 con 0x7f5d3006f2f0 2024-05-30T16:48:13.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.316+0000 7f5d26a00640 1 -- 172.21.15.67:0/3950328788 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f5d18044020 con 0x7f5d3006f2f0 2024-05-30T16:48:13.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.317+0000 7f5d26a00640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d0c06cc40 0x7f5d0c06f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:13.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.317+0000 7f5d26a00640 1 -- 172.21.15.67:0/3950328788 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5765+0+0 (secure 0 0 0) 0x7f5d180ac810 con 0x7f5d3006f2f0 2024-05-30T16:48:13.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.317+0000 7f5d2de00640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d0c06cc40 0x7f5d0c06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.318+0000 7f5d2de00640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d0c06cc40 0x7f5d0c06f100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006fd0 tx=0x7f5d20006bc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.321+0000 7f5d26a00640 1 -- 172.21.15.67:0/3950328788 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d1804b030 con 0x7f5d3006f2f0 2024-05-30T16:48:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.545+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5d3006f7f0 con 0x7f5d3006f2f0 2024-05-30T16:48:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.546+0000 7f5d26a00640 1 -- 172.21.15.67:0/3950328788 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f5d1807a960 con 0x7f5d3006f2f0 2024-05-30T16:48:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:48:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d0c06cc40 msgr2=0x7f5d0c06f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d0c06cc40 0x7f5d0c06f100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006fd0 tx=0x7f5d20006bc0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006f2f0 msgr2=0x7f5d30158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006f2f0 0x7f5d30158d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5d18000c00 tx=0x7f5d180046c0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 shutdown_connections 2024-05-30T16:48:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d0c06cc40 0x7f5d0c06f100 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d30156b30 0x7f5d30159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d3006f2f0 0x7f5d30158d40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 --2- 172.21.15.67:0/3950328788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d3006ee40 0x7f5d301476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 >> 172.21.15.67:0/3950328788 conn(0x7f5d300b1fc0 msgr2=0x7f5d3014c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 shutdown_connections 2024-05-30T16:48:13.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:13.549+0000 7f5d2f200640 1 -- 172.21.15.67:0/3950328788 wait complete. 2024-05-30T16:48:13.569 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T16:48:13.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:13.570 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-30T16:48:13.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[47805]: fuse finished with error 0 and tester_r 0 2024-05-30T16:48:14.049 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:48:14.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:14.050 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-30T16:48:14.101 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:48:14.101 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:48:14.101 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:14.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:48:14.161 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:48:14.161 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:48:14.161 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:14.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:48:14.215 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:48:14.215 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:48:14.215 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:14.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:48:14.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:48:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.540+0000 7f1e1f000640 1 Processor -- start 2024-05-30T16:48:14.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.540+0000 7f1e1f000640 1 -- start start 2024-05-30T16:48:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.542+0000 7f1e1f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1e180d3350 0x7f1e180cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.542+0000 7f1e1f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e180cf850 0x7f1e180b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.542+0000 7f1e1f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e18162580 con 0x7f1e181621a0 2024-05-30T16:48:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.542+0000 7f1e1f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1e180b39c0 con 0x7f1e180cf850 2024-05-30T16:48:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.542+0000 7f1e1f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1e180d0b70 con 0x7f1e180d3350 2024-05-30T16:48:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.542+0000 7f1e1d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e180cf850 0x7f1e180b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.542+0000 7f1e17400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1e180d3350 0x7f1e180cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.542+0000 7f1e17400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1e180d3350 0x7f1e180cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:51126/0 (socket says 172.21.15.67:51126) 2024-05-30T16:48:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.542+0000 7f1e17400640 1 -- 172.21.15.67:0/143898272 learned_addr learned my addr 172.21.15.67:0/143898272 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.543+0000 7f1e17400640 1 -- 172.21.15.67:0/143898272 >> v2:172.21.15.67:3300/0 conn(0x7f1e180cf850 msgr2=0x7f1e180b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.543+0000 7f1e17400640 1 --2- 172.21.15.67:0/143898272 >> v2:172.21.15.67:3300/0 conn(0x7f1e180cf850 0x7f1e180b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.543+0000 7f1e17400640 1 -- 172.21.15.67:0/143898272 >> v1:172.21.15.169:6789/0 conn(0x7f1e181621a0 legacy=0x7f1e180d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.543+0000 7f1e17400640 1 -- 172.21.15.67:0/143898272 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e180cfff0 con 0x7f1e180d3350 2024-05-30T16:48:14.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.545+0000 7f1e17400640 1 --2- 172.21.15.67:0/143898272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180d3350 0x7f1e180cf310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e180d45b0 tx=0x7f1e0402a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc53014d29ac7b64 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.545+0000 7f1e16a00640 1 -- 172.21.15.67:0/143898272 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04037070 con 0x7f1e180d3350 2024-05-30T16:48:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.545+0000 7f1e16a00640 1 -- 172.21.15.67:0/143898272 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e04032440 con 0x7f1e180d3350 2024-05-30T16:48:14.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.546+0000 7f1e16a00640 1 -- 172.21.15.67:0/143898272 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04032810 con 0x7f1e180d3350 2024-05-30T16:48:14.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.547+0000 7f1e1f000640 1 -- 172.21.15.67:0/143898272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180d3350 msgr2=0x7f1e180cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:14.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.547+0000 7f1e1f000640 1 --2- 172.21.15.67:0/143898272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180d3350 0x7f1e180cf310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e180d45b0 tx=0x7f1e0402a1f0 comp rx=0 tx=0).stop 2024-05-30T16:48:14.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.547+0000 7f1e1f000640 1 -- 172.21.15.67:0/143898272 shutdown_connections 2024-05-30T16:48:14.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.547+0000 7f1e1f000640 1 --2- 172.21.15.67:0/143898272 >> v2:172.21.15.67:3300/0 conn(0x7f1e180cf850 0x7f1e180b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:14.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.547+0000 7f1e1f000640 1 --2- 172.21.15.67:0/143898272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180d3350 0x7f1e180cf310 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:14.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.547+0000 7f1e1f000640 1 -- 172.21.15.67:0/143898272 >> 172.21.15.67:0/143898272 conn(0x7f1e1808bfc0 msgr2=0x7f1e180d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:14.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.548+0000 7f1e1f000640 1 -- 172.21.15.67:0/143898272 shutdown_connections 2024-05-30T16:48:14.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.548+0000 7f1e1f000640 1 -- 172.21.15.67:0/143898272 wait complete. 2024-05-30T16:48:14.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.550+0000 7f1e1f000640 1 Processor -- start 2024-05-30T16:48:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.551+0000 7f1e1f000640 1 -- start start 2024-05-30T16:48:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.553+0000 7f1e1f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180cf850 0x7f1e180cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.553+0000 7f1e1f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e180d3350 0x7f1e1815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.553+0000 7f1e1f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e181621a0 0x7f1e1809e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.553+0000 7f1e1f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1e180d0890 con 0x7f1e181621a0 2024-05-30T16:48:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.553+0000 7f1e1f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e180d05b0 con 0x7f1e180cf850 2024-05-30T16:48:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.553+0000 7f1e1f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1e180cfc60 con 0x7f1e180d3350 2024-05-30T16:48:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.554+0000 7f1e17e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180cf850 0x7f1e180cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.554+0000 7f1e1d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e181621a0 0x7f1e1809e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.554+0000 7f1e17e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180cf850 0x7f1e180cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:51132/0 (socket says 172.21.15.67:51132) 2024-05-30T16:48:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.554+0000 7f1e17e00640 1 -- 172.21.15.67:0/3006137637 learned_addr learned my addr 172.21.15.67:0/3006137637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.554+0000 7f1e17400640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e180d3350 0x7f1e1815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.554+0000 7f1e1d000640 1 -- 172.21.15.67:0/3006137637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e180d3350 msgr2=0x7f1e1815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:14.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.555+0000 7f1e1d000640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e180d3350 0x7f1e1815e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:14.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.555+0000 7f1e1d000640 1 -- 172.21.15.67:0/3006137637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180cf850 msgr2=0x7f1e180cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:14.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.555+0000 7f1e1d000640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180cf850 0x7f1e180cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:14.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.555+0000 7f1e1d000640 1 -- 172.21.15.67:0/3006137637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e180cd0e0 con 0x7f1e181621a0 2024-05-30T16:48:14.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.555+0000 7f1e17e00640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180cf850 0x7f1e180cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:48:14.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.555+0000 7f1e17400640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e180d3350 0x7f1e1815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:14.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.555+0000 7f1e1d000640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e181621a0 0x7f1e1809e380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1e10007cf0 tx=0x7f1e1000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:14.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.556+0000 7f1e0fe00640 1 -- 172.21.15.67:0/3006137637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000edd0 con 0x7f1e181621a0 2024-05-30T16:48:14.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.556+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e180cd3c0 con 0x7f1e181621a0 2024-05-30T16:48:14.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.556+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e180cd980 con 0x7f1e181621a0 2024-05-30T16:48:14.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.556+0000 7f1e0fe00640 1 -- 172.21.15.67:0/3006137637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e10003990 con 0x7f1e181621a0 2024-05-30T16:48:14.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.556+0000 7f1e0fe00640 1 -- 172.21.15.67:0/3006137637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10003d80 con 0x7f1e181621a0 2024-05-30T16:48:14.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.558+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df0005290 con 0x7f1e181621a0 2024-05-30T16:48:14.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.560+0000 7f1e0fe00640 1 -- 172.21.15.67:0/3006137637 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f1e1001b020 con 0x7f1e181621a0 2024-05-30T16:48:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.562+0000 7f1e0fe00640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1df806cbc0 0x7f1df806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.562+0000 7f1e0fe00640 1 -- 172.21.15.67:0/3006137637 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5765+0+0 (secure 0 0 0) 0x7f1e1008cfb0 con 0x7f1e181621a0 2024-05-30T16:48:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.562+0000 7f1e17e00640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1df806cbc0 0x7f1df806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.563+0000 7f1e17e00640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1df806cbc0 0x7f1df806f080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e180ba690 tx=0x7f1e08009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.567+0000 7f1e0fe00640 1 -- 172.21.15.67:0/3006137637 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e1005b050 con 0x7f1e181621a0 2024-05-30T16:48:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.995+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1df0005580 con 0x7f1e181621a0 2024-05-30T16:48:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:14.996+0000 7f1e0fe00640 1 -- 172.21.15.67:0/3006137637 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5181 (secure 0 0 0) 0x7f1e1005a950 con 0x7f1e181621a0 2024-05-30T16:48:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:14.998 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/475904074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":475904074},{"type":"v1","addr":"172.21.15.169:6834","nonce":475904074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4812,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3464775083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3464775083},{"type":"v1","addr":"172.21.15.67:6835","nonce":3464775083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/650843675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":650843675},{"type":"v1","addr":"172.21.15.67:6837","nonce":650843675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:44:45.338997+0000","modified":"2024-05-30T16:44:48.433470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4821":{"gid":4821,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/2481507396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2481507396},{"type":"v1","addr":"172.21.15.169:6835","nonce":2481507396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4821,"qdb_cluster":[4821]},"id":2}]} 2024-05-30T16:48:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-05-30T16:48:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.000+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1df806cbc0 msgr2=0x7f1df806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.000+0000 7f1e1f000640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1df806cbc0 0x7f1df806f080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e180ba690 tx=0x7f1e08009040 comp rx=0 tx=0).stop 2024-05-30T16:48:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.000+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e181621a0 msgr2=0x7f1e1809e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.000+0000 7f1e1f000640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e181621a0 0x7f1e1809e380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1e10007cf0 tx=0x7f1e1000e4b0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.001+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 shutdown_connections 2024-05-30T16:48:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.001+0000 7f1e1f000640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1df806cbc0 0x7f1df806f080 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.001+0000 7f1e1f000640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e181621a0 0x7f1e1809e380 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.001+0000 7f1e1f000640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e180d3350 0x7f1e1815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.001+0000 7f1e1f000640 1 --2- 172.21.15.67:0/3006137637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e180cf850 0x7f1e180cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.001+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 >> 172.21.15.67:0/3006137637 conn(0x7f1e1808bfc0 msgr2=0x7f1e180d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.002+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 shutdown_connections 2024-05-30T16:48:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.002+0000 7f1e1f000640 1 -- 172.21.15.67:0/3006137637 wait complete. 2024-05-30T16:48:15.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:48:15.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.270+0000 7fdffbe00640 1 Processor -- start 2024-05-30T16:48:15.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.270+0000 7fdffbe00640 1 -- start start 2024-05-30T16:48:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.272+0000 7fdffbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdff40b0a00 0x7fdff40cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.272+0000 7fdffbe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff40b1770 con 0x7fdff4073730 2024-05-30T16:48:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.272+0000 7fdffbe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff40b1a50 con 0x7fdff4161fe0 2024-05-30T16:48:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.272+0000 7fdffbe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdff40b0ed0 con 0x7fdff40b0a00 2024-05-30T16:48:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.273+0000 7fdff9e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdff40b0a00 0x7fdff40cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.273+0000 7fdff9e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdff40b0a00 0x7fdff40cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:51140/0 (socket says 172.21.15.67:51140) 2024-05-30T16:48:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.273+0000 7fdff9e00640 1 -- 172.21.15.67:0/549368617 learned_addr learned my addr 172.21.15.67:0/549368617 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:15.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.274+0000 7fdff9e00640 1 -- 172.21.15.67:0/549368617 >> v1:172.21.15.169:6789/0 conn(0x7fdff4161fe0 legacy=0x7fdff4073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:15.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.274+0000 7fdff9e00640 1 -- 172.21.15.67:0/549368617 >> v1:172.21.15.67:6789/0 conn(0x7fdff4073730 legacy=0x7fdff40caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.274+0000 7fdff9e00640 1 -- 172.21.15.67:0/549368617 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff41623c0 con 0x7fdff40b0a00 2024-05-30T16:48:15.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.275+0000 7fdff9e00640 1 --2- 172.21.15.67:0/549368617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff40b0a00 0x7fdff40cb660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009e50 tx=0x7fdfec02a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efa568d06a76c5b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:15.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.279+0000 7fdff3e00640 1 -- 172.21.15.67:0/549368617 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec036070 con 0x7fdff40b0a00 2024-05-30T16:48:15.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.279+0000 7fdff3e00640 1 -- 172.21.15.67:0/549368617 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfec031600 con 0x7fdff40b0a00 2024-05-30T16:48:15.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.279+0000 7fdff3e00640 1 -- 172.21.15.67:0/549368617 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0319d0 con 0x7fdff40b0a00 2024-05-30T16:48:15.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.280+0000 7fdffbe00640 1 -- 172.21.15.67:0/549368617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff40b0a00 msgr2=0x7fdff40cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.280+0000 7fdffbe00640 1 --2- 172.21.15.67:0/549368617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff40b0a00 0x7fdff40cb660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009e50 tx=0x7fdfec02a6d0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.281+0000 7fdffbe00640 1 -- 172.21.15.67:0/549368617 shutdown_connections 2024-05-30T16:48:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.281+0000 7fdffbe00640 1 --2- 172.21.15.67:0/549368617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff40b0a00 0x7fdff40cb660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.281+0000 7fdffbe00640 1 -- 172.21.15.67:0/549368617 >> 172.21.15.67:0/549368617 conn(0x7fdff40d0260 msgr2=0x7fdff40d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:15.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.282+0000 7fdffbe00640 1 -- 172.21.15.67:0/549368617 shutdown_connections 2024-05-30T16:48:15.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.283+0000 7fdffbe00640 1 -- 172.21.15.67:0/549368617 wait complete. 2024-05-30T16:48:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.286+0000 7fdffbe00640 1 Processor -- start 2024-05-30T16:48:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.286+0000 7fdffbe00640 1 -- start start 2024-05-30T16:48:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.287+0000 7fdffbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff4073730 0x7fdff40c24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.287+0000 7fdffbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdff40b0a00 0x7fdff409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.287+0000 7fdffbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdff4161fe0 0x7fdff40ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.287+0000 7fdffbe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40caac0 con 0x7fdff4161fe0 2024-05-30T16:48:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.287+0000 7fdffbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40d0030 con 0x7fdff4073730 2024-05-30T16:48:15.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.287+0000 7fdffbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdff40b10f0 con 0x7fdff40b0a00 2024-05-30T16:48:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.288+0000 7fdff9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff4073730 0x7fdff40c24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.288+0000 7fdff9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdff4161fe0 0x7fdff40ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.288+0000 7fdff9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdff4161fe0 0x7fdff40ada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38976/0 (socket says 172.21.15.67:38976) 2024-05-30T16:48:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.288+0000 7fdff9e00640 1 -- 172.21.15.67:0/683958252 learned_addr learned my addr 172.21.15.67:0/683958252 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.289+0000 7fdff9e00640 1 -- 172.21.15.67:0/683958252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdff40b0a00 msgr2=0x7fdff409db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.289+0000 7fdff9e00640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdff40b0a00 0x7fdff409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.289+0000 7fdff9e00640 1 -- 172.21.15.67:0/683958252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff4073730 msgr2=0x7fdff40c24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.289+0000 7fdff9e00640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff4073730 0x7fdff40c24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.289+0000 7fdff9e00640 1 -- 172.21.15.67:0/683958252 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff40afd20 con 0x7fdff4161fe0 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.290+0000 7fdff9e00640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdff4161fe0 0x7fdff40ada40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009e50 tx=0x7fdfec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.290+0000 7fdff1600640 1 -- 172.21.15.67:0/683958252 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec041020 con 0x7fdff4161fe0 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.290+0000 7fdff1600640 1 -- 172.21.15.67:0/683958252 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfec008690 con 0x7fdff4161fe0 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.290+0000 7fdff1600640 1 -- 172.21.15.67:0/683958252 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec036040 con 0x7fdff4161fe0 2024-05-30T16:48:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.291+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff40b0000 con 0x7fdff4161fe0 2024-05-30T16:48:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.291+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff40b05c0 con 0x7fdff4161fe0 2024-05-30T16:48:15.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.294+0000 7fdff1600640 1 -- 172.21.15.67:0/683958252 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fdfec044020 con 0x7fdff4161fe0 2024-05-30T16:48:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.295+0000 7fdff1600640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfd406cbc0 0x7fdfd406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.295+0000 7fdff1600640 1 -- 172.21.15.67:0/683958252 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5765+0+0 (secure 0 0 0) 0x7fdfec0ad040 con 0x7fdff4161fe0 2024-05-30T16:48:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.296+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff40689e0 con 0x7fdff4161fe0 2024-05-30T16:48:15.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.297+0000 7fdff9400640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfd406cbc0 0x7fdfd406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:15.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.298+0000 7fdff9400640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfd406cbc0 0x7fdfd406f080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000a750 tx=0x7fdfe0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:15.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.305+0000 7fdff1600640 1 -- 172.21.15.67:0/683958252 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdfec04b070 con 0x7fdff4161fe0 2024-05-30T16:48:15.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.681+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdff4069280 con 0x7fdff4161fe0 2024-05-30T16:48:15.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.682+0000 7fdff1600640 1 -- 172.21.15.67:0/683958252 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5181 (secure 0 0 0) 0x7fdfec07aeb0 con 0x7fdff4161fe0 2024-05-30T16:48:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:15.683 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/475904074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":475904074},{"type":"v1","addr":"172.21.15.169:6834","nonce":475904074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4812,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3464775083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3464775083},{"type":"v1","addr":"172.21.15.67:6835","nonce":3464775083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/650843675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":650843675},{"type":"v1","addr":"172.21.15.67:6837","nonce":650843675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:44:45.338997+0000","modified":"2024-05-30T16:44:48.433470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4821":{"gid":4821,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/2481507396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2481507396},{"type":"v1","addr":"172.21.15.169:6835","nonce":2481507396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4821,"qdb_cluster":[4821]},"id":2}]} 2024-05-30T16:48:15.683 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-05-30T16:48:15.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfd406cbc0 msgr2=0x7fdfd406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfd406cbc0 0x7fdfd406f080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000a750 tx=0x7fdfe0008040 comp rx=0 tx=0).stop 2024-05-30T16:48:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdff4161fe0 msgr2=0x7fdff40ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdff4161fe0 0x7fdff40ada40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009e50 tx=0x7fdfec004a40 comp rx=0 tx=0).stop 2024-05-30T16:48:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 shutdown_connections 2024-05-30T16:48:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfd406cbc0 0x7fdfd406f080 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdff4161fe0 0x7fdff40ada40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdff40b0a00 0x7fdff409db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 --2- 172.21.15.67:0/683958252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff4073730 0x7fdff40c24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.685+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 >> 172.21.15.67:0/683958252 conn(0x7fdff40d0260 msgr2=0x7fdff41f7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.686+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 shutdown_connections 2024-05-30T16:48:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.686+0000 7fdffbe00640 1 -- 172.21.15.67:0/683958252 wait complete. 2024-05-30T16:48:15.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:48:15.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.953+0000 7fbc6b600640 1 Processor -- start 2024-05-30T16:48:15.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.954+0000 7fbc6b600640 1 -- start start 2024-05-30T16:48:15.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.955+0000 7fbc6b600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbc640b0a00 0x7fbc640cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.955+0000 7fbc6b600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc640b1770 con 0x7fbc64073730 2024-05-30T16:48:15.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.955+0000 7fbc6b600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc640b1a50 con 0x7fbc64161fe0 2024-05-30T16:48:15.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.955+0000 7fbc6b600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbc640b0ed0 con 0x7fbc640b0a00 2024-05-30T16:48:15.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.956+0000 7fbc68c00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fbc64161fe0 0x7fbc64073020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:58400/0 (socket says 172.21.15.67:58400) 2024-05-30T16:48:15.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.956+0000 7fbc68c00640 1 -- 172.21.15.67:0/3050286613 learned_addr learned my addr 172.21.15.67:0/3050286613 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.956+0000 7fbc69600640 1 --2- 172.21.15.67:0/3050286613 >> v2:172.21.15.169:3300/0 conn(0x7fbc640b0a00 0x7fbc640cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.957+0000 7fbc63400640 1 -- 172.21.15.67:0/3050286613 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1813036782 0 0) 0x7fbc640b1a50 con 0x7fbc64161fe0 2024-05-30T16:48:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.957+0000 7fbc63400640 1 -- 172.21.15.67:0/3050286613 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc4c003520 con 0x7fbc64161fe0 2024-05-30T16:48:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.957+0000 7fbc63400640 1 -- 172.21.15.67:0/3050286613 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900617461 0 0) 0x7fbc640b1770 con 0x7fbc64073730 2024-05-30T16:48:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.957+0000 7fbc63400640 1 -- 172.21.15.67:0/3050286613 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc640b1a50 con 0x7fbc64073730 2024-05-30T16:48:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.957+0000 7fbc69600640 1 -- 172.21.15.67:0/3050286613 >> v1:172.21.15.169:6789/0 conn(0x7fbc64161fe0 legacy=0x7fbc64073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.957+0000 7fbc69600640 1 -- 172.21.15.67:0/3050286613 >> v1:172.21.15.67:6789/0 conn(0x7fbc64073730 legacy=0x7fbc640caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.957+0000 7fbc69600640 1 -- 172.21.15.67:0/3050286613 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc641623c0 con 0x7fbc640b0a00 2024-05-30T16:48:15.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.958+0000 7fbc69600640 1 --2- 172.21.15.67:0/3050286613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc640b0a00 0x7fbc640cb660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009d90 tx=0x7fbc5c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8802752c96adb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:15.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.958+0000 7fbc63400640 1 -- 172.21.15.67:0/3050286613 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c007740 con 0x7fbc640b0a00 2024-05-30T16:48:15.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.958+0000 7fbc63400640 1 -- 172.21.15.67:0/3050286613 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5c0078e0 con 0x7fbc640b0a00 2024-05-30T16:48:15.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.958+0000 7fbc63400640 1 -- 172.21.15.67:0/3050286613 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c007cb0 con 0x7fbc640b0a00 2024-05-30T16:48:15.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.959+0000 7fbc6b600640 1 -- 172.21.15.67:0/3050286613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc640b0a00 msgr2=0x7fbc640cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.959+0000 7fbc6b600640 1 --2- 172.21.15.67:0/3050286613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc640b0a00 0x7fbc640cb660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009d90 tx=0x7fbc5c02a660 comp rx=0 tx=0).stop 2024-05-30T16:48:15.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.959+0000 7fbc6b600640 1 -- 172.21.15.67:0/3050286613 shutdown_connections 2024-05-30T16:48:15.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.959+0000 7fbc6b600640 1 --2- 172.21.15.67:0/3050286613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc640b0a00 0x7fbc640cb660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.959+0000 7fbc6b600640 1 -- 172.21.15.67:0/3050286613 >> 172.21.15.67:0/3050286613 conn(0x7fbc640d0260 msgr2=0x7fbc640d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:15.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.960+0000 7fbc6b600640 1 -- 172.21.15.67:0/3050286613 shutdown_connections 2024-05-30T16:48:15.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.960+0000 7fbc6b600640 1 -- 172.21.15.67:0/3050286613 wait complete. 2024-05-30T16:48:15.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.964+0000 7fbc6b600640 1 Processor -- start 2024-05-30T16:48:15.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.964+0000 7fbc6b600640 1 -- start start 2024-05-30T16:48:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc6b600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc64073730 0x7fbc640b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc6b600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc640b0a00 0x7fbc640994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc6b600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc64161fe0 0x7fbc640999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc6b600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbc640caac0 con 0x7fbc640b0a00 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc6b600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbc640d0030 con 0x7fbc64073730 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc6b600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbc640b10f0 con 0x7fbc64161fe0 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc640b0a00 0x7fbc640994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc640b0a00 0x7fbc640994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38988/0 (socket says 172.21.15.67:38988) 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc63e00640 1 -- 172.21.15.67:0/3632260477 learned_addr learned my addr 172.21.15.67:0/3632260477 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.966+0000 7fbc68c00640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc64073730 0x7fbc640b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.967+0000 7fbc69600640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc64161fe0 0x7fbc640999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.967+0000 7fbc63e00640 1 -- 172.21.15.67:0/3632260477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc64161fe0 msgr2=0x7fbc640999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.967+0000 7fbc63e00640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc64161fe0 0x7fbc640999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.967+0000 7fbc63e00640 1 -- 172.21.15.67:0/3632260477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc64073730 msgr2=0x7fbc640b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:15.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.967+0000 7fbc63e00640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc64073730 0x7fbc640b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:15.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.967+0000 7fbc63e00640 1 -- 172.21.15.67:0/3632260477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc640b5f90 con 0x7fbc640b0a00 2024-05-30T16:48:15.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.967+0000 7fbc63e00640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc640b0a00 0x7fbc640994a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000c480 tx=0x7fbc5000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.967+0000 7fbc60c00640 1 -- 172.21.15.67:0/3632260477 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50017070 con 0x7fbc640b0a00 2024-05-30T16:48:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.967+0000 7fbc60c00640 1 -- 172.21.15.67:0/3632260477 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc500052d0 con 0x7fbc640b0a00 2024-05-30T16:48:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.968+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc640b6270 con 0x7fbc640b0a00 2024-05-30T16:48:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.968+0000 7fbc60c00640 1 -- 172.21.15.67:0/3632260477 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50012870 con 0x7fbc640b0a00 2024-05-30T16:48:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.968+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc640ae600 con 0x7fbc640b0a00 2024-05-30T16:48:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.969+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc640689e0 con 0x7fbc640b0a00 2024-05-30T16:48:15.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.970+0000 7fbc60c00640 1 -- 172.21.15.67:0/3632260477 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fbc5001a070 con 0x7fbc640b0a00 2024-05-30T16:48:15.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.971+0000 7fbc60c00640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbc4406cc10 0x7fbc4406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.971+0000 7fbc60c00640 1 -- 172.21.15.67:0/3632260477 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5765+0+0 (secure 0 0 0) 0x7fbc5008b2a0 con 0x7fbc640b0a00 2024-05-30T16:48:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.971+0000 7fbc68c00640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbc4406cc10 0x7fbc4406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.972+0000 7fbc68c00640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbc4406cc10 0x7fbc4406f0d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc580011e0 tx=0x7fbc58007d30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:15.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:15.975+0000 7fbc60c00640 1 -- 172.21.15.67:0/3632260477 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc50058dc0 con 0x7fbc640b0a00 2024-05-30T16:48:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.177+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc641e76a0 con 0x7fbc640b0a00 2024-05-30T16:48:16.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.181+0000 7fbc60c00640 1 -- 172.21.15.67:0/3632260477 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16765 (secure 0 0 0) 0x7fbc5005d0c0 con 0x7fbc640b0a00 2024-05-30T16:48:16.182 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:16.182 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":36,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:48:13.250521+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:44:43.288139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-30T16:44:44.321242+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:48:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbc4406cc10 msgr2=0x7fbc4406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbc4406cc10 0x7fbc4406f0d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc580011e0 tx=0x7fbc58007d30 comp rx=0 tx=0).stop 2024-05-30T16:48:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc640b0a00 msgr2=0x7fbc640994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc640b0a00 0x7fbc640994a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000c480 tx=0x7fbc5000c940 comp rx=0 tx=0).stop 2024-05-30T16:48:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 shutdown_connections 2024-05-30T16:48:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbc4406cc10 0x7fbc4406f0d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc64161fe0 0x7fbc640999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc640b0a00 0x7fbc640994a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 --2- 172.21.15.67:0/3632260477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc64073730 0x7fbc640b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 >> 172.21.15.67:0/3632260477 conn(0x7fbc640d0260 msgr2=0x7fbc641611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.183+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 shutdown_connections 2024-05-30T16:48:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.184+0000 7fbc6b600640 1 -- 172.21.15.67:0/3632260477 wait complete. 2024-05-30T16:48:16.202 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-30T16:48:16.203 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:48:16.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.405+0000 7f0b4b200640 1 Processor -- start 2024-05-30T16:48:16.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.406+0000 7f0b4b200640 1 -- start start 2024-05-30T16:48:16.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.408+0000 7f0b4b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c15e9c0 0x7f0b4c0b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.408+0000 7f0b4b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0b4c0b1630 0x7f0b4c0b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.408+0000 7f0b4b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b4c0ae450 con 0x7f0b4c15eda0 2024-05-30T16:48:16.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.408+0000 7f0b4b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0b4c0ceea0 con 0x7f0b4c0b1630 2024-05-30T16:48:16.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.408+0000 7f0b4b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4c0ae730 con 0x7f0b4c15e9c0 2024-05-30T16:48:16.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.409+0000 7f0b49e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c15e9c0 0x7f0b4c0b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:16.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.409+0000 7f0b49400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f0b4c15eda0 0x7f0b4c0ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:58404/0 (socket says 172.21.15.67:58404) 2024-05-30T16:48:16.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.409+0000 7f0b49400640 1 -- 172.21.15.67:0/3011144157 learned_addr learned my addr 172.21.15.67:0/3011144157 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:16.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.410+0000 7f0b4a800640 1 --2- 172.21.15.67:0/3011144157 >> v2:172.21.15.169:3300/0 conn(0x7f0b4c0b1630 0x7f0b4c0b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:16.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.411+0000 7f0b4a800640 1 -- 172.21.15.67:0/3011144157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c15e9c0 msgr2=0x7f0b4c0b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:16.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.411+0000 7f0b4a800640 1 --2- 172.21.15.67:0/3011144157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c15e9c0 0x7f0b4c0b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.411+0000 7f0b4a800640 1 -- 172.21.15.67:0/3011144157 >> v1:172.21.15.169:6789/0 conn(0x7f0b4c15eda0 legacy=0x7f0b4c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.411+0000 7f0b4a800640 1 -- 172.21.15.67:0/3011144157 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c0aea10 con 0x7f0b4c0b1630 2024-05-30T16:48:16.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.412+0000 7f0b4a800640 1 --2- 172.21.15.67:0/3011144157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b4c0b1630 0x7f0b4c0b1ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009d90 tx=0x7f0b4402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=908ddf5de2520afe server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:16.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.414+0000 7f0b48a00640 1 -- 172.21.15.67:0/3011144157 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44031600 con 0x7f0b4c0b1630 2024-05-30T16:48:16.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.414+0000 7f0b48a00640 1 -- 172.21.15.67:0/3011144157 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b440317a0 con 0x7f0b4c0b1630 2024-05-30T16:48:16.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.414+0000 7f0b48a00640 1 -- 172.21.15.67:0/3011144157 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44031b70 con 0x7f0b4c0b1630 2024-05-30T16:48:16.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.415+0000 7f0b4b200640 1 -- 172.21.15.67:0/3011144157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b4c0b1630 msgr2=0x7f0b4c0b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.415+0000 7f0b4b200640 1 --2- 172.21.15.67:0/3011144157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b4c0b1630 0x7f0b4c0b1ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009d90 tx=0x7f0b4402a5d0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.415+0000 7f0b4b200640 1 -- 172.21.15.67:0/3011144157 shutdown_connections 2024-05-30T16:48:16.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.415+0000 7f0b4b200640 1 --2- 172.21.15.67:0/3011144157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b4c0b1630 0x7f0b4c0b1ae0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.415+0000 7f0b4b200640 1 --2- 172.21.15.67:0/3011144157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c15e9c0 0x7f0b4c0b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.415+0000 7f0b4b200640 1 -- 172.21.15.67:0/3011144157 >> 172.21.15.67:0/3011144157 conn(0x7f0b4c0b6860 msgr2=0x7f0b4c0b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:16.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.415+0000 7f0b4b200640 1 -- 172.21.15.67:0/3011144157 shutdown_connections 2024-05-30T16:48:16.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.415+0000 7f0b4b200640 1 -- 172.21.15.67:0/3011144157 wait complete. 2024-05-30T16:48:16.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.417+0000 7f0b4b200640 1 Processor -- start 2024-05-30T16:48:16.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.417+0000 7f0b4b200640 1 -- start start 2024-05-30T16:48:16.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.418+0000 7f0b4b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c0b1630 0x7f0b4c15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.418+0000 7f0b4b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b4c15e9c0 0x7f0b4c151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.418+0000 7f0b4b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4c15eda0 0x7f0b4c1612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.418+0000 7f0b4b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c0b44c0 con 0x7f0b4c15eda0 2024-05-30T16:48:16.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.418+0000 7f0b4b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c0b4340 con 0x7f0b4c15e9c0 2024-05-30T16:48:16.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.418+0000 7f0b4b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4c0af210 con 0x7f0b4c0b1630 2024-05-30T16:48:16.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.418+0000 7f0b4a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4c15eda0 0x7f0b4c1612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.418+0000 7f0b4a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4c15eda0 0x7f0b4c1612d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39004/0 (socket says 172.21.15.67:39004) 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.418+0000 7f0b4a800640 1 -- 172.21.15.67:0/4099869868 learned_addr learned my addr 172.21.15.67:0/4099869868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b49e00640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c0b1630 0x7f0b4c15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b4a800640 1 -- 172.21.15.67:0/4099869868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c0b1630 msgr2=0x7f0b4c15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b4a800640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c0b1630 0x7f0b4c15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b4a800640 1 -- 172.21.15.67:0/4099869868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b4c15e9c0 msgr2=0x7f0b4c151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b4a800640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b4c15e9c0 0x7f0b4c151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b4a800640 1 -- 172.21.15.67:0/4099869868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c161b00 con 0x7f0b4c15eda0 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b4a800640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4c15eda0 0x7f0b4c1612d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009eb0 tx=0x7f0b44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b42000640 1 -- 172.21.15.67:0/4099869868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44040070 con 0x7f0b4c15eda0 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b42000640 1 -- 172.21.15.67:0/4099869868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b44030b60 con 0x7f0b4c15eda0 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b42000640 1 -- 172.21.15.67:0/4099869868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44009450 con 0x7f0b4c15eda0 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.419+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4c161de0 con 0x7f0b4c15eda0 2024-05-30T16:48:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.420+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b4c0cabf0 con 0x7f0b4c15eda0 2024-05-30T16:48:16.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.421+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b4c0cadf0 con 0x7f0b4c15eda0 2024-05-30T16:48:16.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.421+0000 7f0b42000640 1 -- 172.21.15.67:0/4099869868 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f0b44039040 con 0x7f0b4c15eda0 2024-05-30T16:48:16.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.422+0000 7f0b42000640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b2c06cbf0 0x7f0b2c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.422+0000 7f0b42000640 1 -- 172.21.15.67:0/4099869868 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5737+0+0 (secure 0 0 0) 0x7f0b440ad7d0 con 0x7f0b4c15eda0 2024-05-30T16:48:16.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.423+0000 7f0b49e00640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b2c06cbf0 0x7f0b2c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:16.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.423+0000 7f0b49e00640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b2c06cbf0 0x7f0b2c06f0b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009ad0 tx=0x7f0b3c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:16.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.427+0000 7f0b42000640 1 -- 172.21.15.67:0/4099869868 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b4c0cadf0 con 0x7f0b4c15eda0 2024-05-30T16:48:16.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.664+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b4c0689e0 con 0x7f0b4c15eda0 2024-05-30T16:48:16.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.665+0000 7f0b42000640 1 -- 172.21.15.67:0/4099869868 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5181 (secure 0 0 0) 0x7f0b4407b740 con 0x7f0b4c15eda0 2024-05-30T16:48:16.666 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:16.666 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/475904074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":475904074},{"type":"v1","addr":"172.21.15.169:6834","nonce":475904074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4812,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3464775083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3464775083},{"type":"v1","addr":"172.21.15.67:6835","nonce":3464775083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/650843675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":650843675},{"type":"v1","addr":"172.21.15.67:6837","nonce":650843675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:44:45.338997+0000","modified":"2024-05-30T16:44:48.433470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4821":{"gid":4821,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/2481507396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2481507396},{"type":"v1","addr":"172.21.15.169:6835","nonce":2481507396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4821,"qdb_cluster":[4821]},"id":2}]} 2024-05-30T16:48:16.666 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-05-30T16:48:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.667+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b2c06cbf0 msgr2=0x7f0b2c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.667+0000 7f0b4b200640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b2c06cbf0 0x7f0b2c06f0b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009ad0 tx=0x7f0b3c008040 comp rx=0 tx=0).stop 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.667+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4c15eda0 msgr2=0x7f0b4c1612d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.667+0000 7f0b4b200640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4c15eda0 0x7f0b4c1612d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009eb0 tx=0x7f0b44004a40 comp rx=0 tx=0).stop 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.668+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 shutdown_connections 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.668+0000 7f0b4b200640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b2c06cbf0 0x7f0b2c06f0b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.668+0000 7f0b4b200640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b4c15eda0 0x7f0b4c1612d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.668+0000 7f0b4b200640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b4c15e9c0 0x7f0b4c151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.668+0000 7f0b4b200640 1 --2- 172.21.15.67:0/4099869868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b4c0b1630 0x7f0b4c15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.668+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 >> 172.21.15.67:0/4099869868 conn(0x7f0b4c0b6860 msgr2=0x7f0b4c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.668+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 shutdown_connections 2024-05-30T16:48:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.668+0000 7f0b4b200640 1 -- 172.21.15.67:0/4099869868 wait complete. 2024-05-30T16:48:16.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:48:16.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.893+0000 7f7be6c00640 1 Processor -- start 2024-05-30T16:48:16.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.893+0000 7f7be6c00640 1 -- start start 2024-05-30T16:48:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.894+0000 7f7be6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0158b80 0x7f7be006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.894+0000 7f7be6c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7be006e160 0x7f7be0071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.894+0000 7f7be6c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be006e6e0 con 0x7f7be0072100 2024-05-30T16:48:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.894+0000 7f7be6c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7be00b42d0 con 0x7f7be006e160 2024-05-30T16:48:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.894+0000 7f7be6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7be0072530 con 0x7f7be0158b80 2024-05-30T16:48:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.895+0000 7f7be4e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7be006e160 0x7f7be0071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.895+0000 7f7be6200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7be0072100 0x7f7be01621f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55854/0 (socket says 172.21.15.67:55854) 2024-05-30T16:48:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.895+0000 7f7be6200640 1 -- 172.21.15.67:0/1508250932 learned_addr learned my addr 172.21.15.67:0/1508250932 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.895+0000 7f7be5800640 1 --2- 172.21.15.67:0/1508250932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0158b80 0x7f7be006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.895+0000 7f7be4e00640 1 -- 172.21.15.67:0/1508250932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0158b80 msgr2=0x7f7be006dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:16.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.895+0000 7f7be4e00640 1 --2- 172.21.15.67:0/1508250932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0158b80 0x7f7be006dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.895+0000 7f7be4e00640 1 -- 172.21.15.67:0/1508250932 >> v1:172.21.15.67:6789/0 conn(0x7f7be0072100 legacy=0x7f7be01621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.895+0000 7f7be4e00640 1 -- 172.21.15.67:0/1508250932 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be006e9c0 con 0x7f7be006e160 2024-05-30T16:48:16.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.895+0000 7f7be5800640 1 --2- 172.21.15.67:0/1508250932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0158b80 0x7f7be006dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:16.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.896+0000 7f7be4e00640 1 --2- 172.21.15.67:0/1508250932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 0x7f7be0071bc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0097a0 tx=0x7f7bcc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8ffef72719be90 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.896+0000 7f7bdfe00640 1 -- 172.21.15.67:0/1508250932 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc037070 con 0x7f7be006e160 2024-05-30T16:48:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.896+0000 7f7bdfe00640 1 -- 172.21.15.67:0/1508250932 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bcc032440 con 0x7f7be006e160 2024-05-30T16:48:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.896+0000 7f7bdfe00640 1 -- 172.21.15.67:0/1508250932 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc032810 con 0x7f7be006e160 2024-05-30T16:48:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.897+0000 7f7be6c00640 1 -- 172.21.15.67:0/1508250932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 msgr2=0x7f7be0071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.897+0000 7f7be6c00640 1 --2- 172.21.15.67:0/1508250932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 0x7f7be0071bc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0097a0 tx=0x7f7bcc02a180 comp rx=0 tx=0).stop 2024-05-30T16:48:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.897+0000 7f7be6c00640 1 -- 172.21.15.67:0/1508250932 shutdown_connections 2024-05-30T16:48:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.897+0000 7f7be6c00640 1 --2- 172.21.15.67:0/1508250932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 0x7f7be0071bc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.897+0000 7f7be6c00640 1 --2- 172.21.15.67:0/1508250932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0158b80 0x7f7be006dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.897+0000 7f7be6c00640 1 -- 172.21.15.67:0/1508250932 >> 172.21.15.67:0/1508250932 conn(0x7f7be00b1fc0 msgr2=0x7f7be00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:16.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.897+0000 7f7be6c00640 1 -- 172.21.15.67:0/1508250932 shutdown_connections 2024-05-30T16:48:16.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.897+0000 7f7be6c00640 1 -- 172.21.15.67:0/1508250932 wait complete. 2024-05-30T16:48:16.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.899+0000 7f7be6c00640 1 Processor -- start 2024-05-30T16:48:16.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.899+0000 7f7be6c00640 1 -- start start 2024-05-30T16:48:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be6c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 0x7f7be00999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0072100 0x7f7be009c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7be0158b80 0x7f7be009cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be6c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7be006f260 con 0x7f7be006e160 2024-05-30T16:48:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7be006ef80 con 0x7f7be0158b80 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7be00b2d30 con 0x7f7be0072100 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 0x7f7be00999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 0x7f7be00999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39014/0 (socket says 172.21.15.67:39014) 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be5800640 1 -- 172.21.15.67:0/3233100807 learned_addr learned my addr 172.21.15.67:0/3233100807 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be5800640 1 -- 172.21.15.67:0/3233100807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0072100 msgr2=0x7f7be009c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be5800640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0072100 0x7f7be009c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be5800640 1 -- 172.21.15.67:0/3233100807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7be0158b80 msgr2=0x7f7be009cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be5800640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7be0158b80 0x7f7be009cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.900+0000 7f7be5800640 1 -- 172.21.15.67:0/3233100807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be009b270 con 0x7f7be006e160 2024-05-30T16:48:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.901+0000 7f7be5800640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 0x7f7be00999f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400c820 tx=0x7f7bd400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.901+0000 7f7bdd600640 1 -- 172.21.15.67:0/3233100807 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4015070 con 0x7f7be006e160 2024-05-30T16:48:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.901+0000 7f7bdd600640 1 -- 172.21.15.67:0/3233100807 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd4007a30 con 0x7f7be006e160 2024-05-30T16:48:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.901+0000 7f7bdd600640 1 -- 172.21.15.67:0/3233100807 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4007de0 con 0x7f7be006e160 2024-05-30T16:48:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.901+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be009b550 con 0x7f7be006e160 2024-05-30T16:48:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.901+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be009bb10 con 0x7f7be006e160 2024-05-30T16:48:16.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.902+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be00681e0 con 0x7f7be006e160 2024-05-30T16:48:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.903+0000 7f7bdd600640 1 -- 172.21.15.67:0/3233100807 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f7bd4018070 con 0x7f7be006e160 2024-05-30T16:48:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.904+0000 7f7bdd600640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bc006cbc0 0x7f7bc006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.904+0000 7f7bdd600640 1 -- 172.21.15.67:0/3233100807 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5737+0+0 (secure 0 0 0) 0x7f7bd408bfd0 con 0x7f7be006e160 2024-05-30T16:48:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.904+0000 7f7be4e00640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bc006cbc0 0x7f7bc006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:16.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.905+0000 7f7be4e00640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bc006cbc0 0x7f7bc006f080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc034000 tx=0x7f7bcc02c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:16.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:16.909+0000 7f7bdd600640 1 -- 172.21.15.67:0/3233100807 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7be00681e0 con 0x7f7be006e160 2024-05-30T16:48:17.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.141+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7be00689e0 con 0x7f7be006e160 2024-05-30T16:48:17.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.141+0000 7f7bdd600640 1 -- 172.21.15.67:0/3233100807 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5181 (secure 0 0 0) 0x7f7bd4059f40 con 0x7f7be006e160 2024-05-30T16:48:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:17.143 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/475904074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":475904074},{"type":"v1","addr":"172.21.15.169:6834","nonce":475904074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4812,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3464775083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3464775083},{"type":"v1","addr":"172.21.15.67:6835","nonce":3464775083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/650843675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":650843675},{"type":"v1","addr":"172.21.15.67:6837","nonce":650843675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:44:45.338997+0000","modified":"2024-05-30T16:44:48.433470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4821":{"gid":4821,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/2481507396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2481507396},{"type":"v1","addr":"172.21.15.169:6835","nonce":2481507396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4821,"qdb_cluster":[4821]},"id":2}]} 2024-05-30T16:48:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bc006cbc0 msgr2=0x7f7bc006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bc006cbc0 0x7f7bc006f080 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc034000 tx=0x7f7bcc02c000 comp rx=0 tx=0).stop 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 msgr2=0x7f7be00999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 0x7f7be00999f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400c820 tx=0x7f7bd400cce0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 shutdown_connections 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bc006cbc0 0x7f7bc006f080 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7be0158b80 0x7f7be009cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7be0072100 0x7f7be009c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 --2- 172.21.15.67:0/3233100807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7be006e160 0x7f7be00999f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 >> 172.21.15.67:0/3233100807 conn(0x7f7be00b1fc0 msgr2=0x7f7be00b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:17.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 shutdown_connections 2024-05-30T16:48:17.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.144+0000 7f7be6c00640 1 -- 172.21.15.67:0/3233100807 wait complete. 2024-05-30T16:48:17.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:48:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.370+0000 7f6397c00640 1 Processor -- start 2024-05-30T16:48:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.370+0000 7f6397c00640 1 -- start start 2024-05-30T16:48:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.371+0000 7f6397c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f639006f860 0x7f63900b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.371+0000 7f6397c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f63900b3690 0x7f63900d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6397c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900d09e0 0x7f6390160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6397c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f63900d0e30 con 0x7f639006f860 2024-05-30T16:48:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6397c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f639006ed10 con 0x7f63900b3690 2024-05-30T16:48:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6397c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f639015cb00 con 0x7f63900d09e0 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6395200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f639006f860 0x7f63900b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6395200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f639006f860 0x7f63900b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39028/0 (socket says 172.21.15.67:39028) 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6395200640 1 -- 172.21.15.67:0/3811710074 learned_addr learned my addr 172.21.15.67:0/3811710074 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6395c00640 1 --2- 172.21.15.67:0/3811710074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900d09e0 0x7f6390160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f638fe00640 1 --2- 172.21.15.67:0/3811710074 >> v2:172.21.15.169:3300/0 conn(0x7f63900b3690 0x7f63900d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6395200640 1 -- 172.21.15.67:0/3811710074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900d09e0 msgr2=0x7f6390160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6395200640 1 --2- 172.21.15.67:0/3811710074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900d09e0 0x7f6390160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6395200640 1 -- 172.21.15.67:0/3811710074 >> v2:172.21.15.169:3300/0 conn(0x7f63900b3690 msgr2=0x7f63900d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6395200640 1 --2- 172.21.15.67:0/3811710074 >> v2:172.21.15.169:3300/0 conn(0x7f63900b3690 0x7f63900d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.372+0000 7f6395200640 1 -- 172.21.15.67:0/3811710074 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63900b3cc0 con 0x7f639006f860 2024-05-30T16:48:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.373+0000 7f6395c00640 1 --2- 172.21.15.67:0/3811710074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900d09e0 0x7f6390160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.374+0000 7f6395200640 1 --2- 172.21.15.67:0/3811710074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 0x7f63900b3150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6390070120 tx=0x7f638402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edee6b3247c958a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.374+0000 7f638f400640 1 -- 172.21.15.67:0/3811710074 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384037070 con 0x7f639006f860 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.374+0000 7f638f400640 1 -- 172.21.15.67:0/3811710074 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6384032440 con 0x7f639006f860 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.374+0000 7f638f400640 1 -- 172.21.15.67:0/3811710074 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384032810 con 0x7f639006f860 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.375+0000 7f6397c00640 1 -- 172.21.15.67:0/3811710074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 msgr2=0x7f63900b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.375+0000 7f6397c00640 1 --2- 172.21.15.67:0/3811710074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 0x7f63900b3150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6390070120 tx=0x7f638402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.375+0000 7f6397c00640 1 -- 172.21.15.67:0/3811710074 shutdown_connections 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.375+0000 7f6397c00640 1 --2- 172.21.15.67:0/3811710074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900d09e0 0x7f6390160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.375+0000 7f6397c00640 1 --2- 172.21.15.67:0/3811710074 >> v2:172.21.15.169:3300/0 conn(0x7f63900b3690 0x7f63900d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.375+0000 7f6397c00640 1 --2- 172.21.15.67:0/3811710074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 0x7f63900b3150 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.375+0000 7f6397c00640 1 -- 172.21.15.67:0/3811710074 >> 172.21.15.67:0/3811710074 conn(0x7f639006bae0 msgr2=0x7f639006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.375+0000 7f6397c00640 1 -- 172.21.15.67:0/3811710074 shutdown_connections 2024-05-30T16:48:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.375+0000 7f6397c00640 1 -- 172.21.15.67:0/3811710074 wait complete. 2024-05-30T16:48:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.377+0000 7f6397c00640 1 Processor -- start 2024-05-30T16:48:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.377+0000 7f6397c00640 1 -- start start 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.378+0000 7f6397c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 0x7f63900c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.378+0000 7f6397c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900b3690 0x7f63901f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.378+0000 7f6397c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63900d09e0 0x7f63901f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.378+0000 7f6397c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f63900b4560 con 0x7f639006f860 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.378+0000 7f6397c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63900b4280 con 0x7f63900d09e0 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.378+0000 7f6397c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f63900b4840 con 0x7f63900b3690 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.378+0000 7f6395200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 0x7f63900c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.378+0000 7f6395200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 0x7f63900c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39044/0 (socket says 172.21.15.67:39044) 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.378+0000 7f6395200640 1 -- 172.21.15.67:0/3294199904 learned_addr learned my addr 172.21.15.67:0/3294199904 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f6395c00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63900d09e0 0x7f63901f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f638fe00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900b3690 0x7f63901f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f6395200640 1 -- 172.21.15.67:0/3294199904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900b3690 msgr2=0x7f63901f7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f6395200640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900b3690 0x7f63901f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f6395200640 1 -- 172.21.15.67:0/3294199904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63900d09e0 msgr2=0x7f63901f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f6395200640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63900d09e0 0x7f63901f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f6395200640 1 -- 172.21.15.67:0/3294199904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63901f42c0 con 0x7f639006f860 2024-05-30T16:48:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f6395200640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 0x7f63900c8d60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f638402a220 tx=0x7f63840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f638cc00640 1 -- 172.21.15.67:0/3294199904 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384037040 con 0x7f639006f860 2024-05-30T16:48:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f638cc00640 1 -- 172.21.15.67:0/3294199904 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6384042070 con 0x7f639006f860 2024-05-30T16:48:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f638cc00640 1 -- 172.21.15.67:0/3294199904 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384030bb0 con 0x7f639006f860 2024-05-30T16:48:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.379+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63900c93b0 con 0x7f639006f860 2024-05-30T16:48:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.380+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63900c74d0 con 0x7f639006f860 2024-05-30T16:48:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.381+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63900689e0 con 0x7f639006f860 2024-05-30T16:48:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.382+0000 7f638cc00640 1 -- 172.21.15.67:0/3294199904 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f6384047460 con 0x7f639006f860 2024-05-30T16:48:17.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.383+0000 7f638cc00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f637006cbc0 0x7f637006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.383+0000 7f638cc00640 1 -- 172.21.15.67:0/3294199904 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5737+0+0 (secure 0 0 0) 0x7f63840acc70 con 0x7f639006f860 2024-05-30T16:48:17.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.383+0000 7f638fe00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f637006cbc0 0x7f637006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.384+0000 7f638fe00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f637006cbc0 0x7f637006f080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f637c006fd0 tx=0x7f637c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:17.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.387+0000 7f638cc00640 1 -- 172.21.15.67:0/3294199904 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6384045360 con 0x7f639006f860 2024-05-30T16:48:17.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.593+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f639006ef70 con 0x7f639006f860 2024-05-30T16:48:17.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.596+0000 7f638cc00640 1 -- 172.21.15.67:0/3294199904 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16722 (secure 0 0 0) 0x7f638407acf0 con 0x7f639006f860 2024-05-30T16:48:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:17.597 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:48:16.403979+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:44:43.288139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-30T16:44:44.321242+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:48:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f637006cbc0 msgr2=0x7f637006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f637006cbc0 0x7f637006f080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f637c006fd0 tx=0x7f637c008040 comp rx=0 tx=0).stop 2024-05-30T16:48:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 msgr2=0x7f63900c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 0x7f63900c8d60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f638402a220 tx=0x7f63840046c0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 shutdown_connections 2024-05-30T16:48:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f637006cbc0 0x7f637006f080 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63900d09e0 0x7f63901f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63900b3690 0x7f63901f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 --2- 172.21.15.67:0/3294199904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639006f860 0x7f63900c8d60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 >> 172.21.15.67:0/3294199904 conn(0x7f639006bae0 msgr2=0x7f6390071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.598+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 shutdown_connections 2024-05-30T16:48:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.599+0000 7f6397c00640 1 -- 172.21.15.67:0/3294199904 wait complete. 2024-05-30T16:48:17.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-30T16:48:17.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.831+0000 7fddeb200640 1 Processor -- start 2024-05-30T16:48:17.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.831+0000 7fddeb200640 1 -- start start 2024-05-30T16:48:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.833+0000 7fddeb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde415b3a0 0x7fdde40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.833+0000 7fddeb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdde4073310 0x7fdde415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.833+0000 7fddeb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdde40afd60 con 0x7fdde406dc20 2024-05-30T16:48:17.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.833+0000 7fddeb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fdde406f980 con 0x7fdde4073310 2024-05-30T16:48:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.833+0000 7fddeb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdde40ae0c0 con 0x7fdde415b3a0 2024-05-30T16:48:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.833+0000 7fddea800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdde4073310 0x7fdde415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.833+0000 7fddea800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdde4073310 0x7fdde415f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39054/0 (socket says 172.21.15.67:39054) 2024-05-30T16:48:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.833+0000 7fddea800640 1 -- 172.21.15.67:0/351425925 learned_addr learned my addr 172.21.15.67:0/351425925 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.834+0000 7fdde9e00640 1 --2- 172.21.15.67:0/351425925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde415b3a0 0x7fdde40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.834+0000 7fddea800640 1 -- 172.21.15.67:0/351425925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde415b3a0 msgr2=0x7fdde40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:17.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.834+0000 7fddea800640 1 --2- 172.21.15.67:0/351425925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde415b3a0 0x7fdde40b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.834+0000 7fddea800640 1 -- 172.21.15.67:0/351425925 >> v1:172.21.15.169:6789/0 conn(0x7fdde406dc20 legacy=0x7fdde4072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.834+0000 7fddea800640 1 -- 172.21.15.67:0/351425925 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde40af7a0 con 0x7fdde4073310 2024-05-30T16:48:17.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.834+0000 7fdde9e00640 1 --2- 172.21.15.67:0/351425925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde415b3a0 0x7fdde40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:48:17.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.836+0000 7fddea800640 1 --2- 172.21.15.67:0/351425925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde4073310 0x7fdde415f9a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d150 tx=0x7fdde002c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d918812a4dcc85 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.836+0000 7fdde8a00640 1 -- 172.21.15.67:0/351425925 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0009600 con 0x7fdde4073310 2024-05-30T16:48:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.836+0000 7fdde8a00640 1 -- 172.21.15.67:0/351425925 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde0037070 con 0x7fdde4073310 2024-05-30T16:48:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.836+0000 7fdde8a00640 1 -- 172.21.15.67:0/351425925 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde00099c0 con 0x7fdde4073310 2024-05-30T16:48:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.837+0000 7fddeb200640 1 -- 172.21.15.67:0/351425925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde4073310 msgr2=0x7fdde415f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.837+0000 7fddeb200640 1 --2- 172.21.15.67:0/351425925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde4073310 0x7fdde415f9a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d150 tx=0x7fdde002c930 comp rx=0 tx=0).stop 2024-05-30T16:48:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.837+0000 7fddeb200640 1 -- 172.21.15.67:0/351425925 shutdown_connections 2024-05-30T16:48:17.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.837+0000 7fddeb200640 1 --2- 172.21.15.67:0/351425925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde4073310 0x7fdde415f9a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.837+0000 7fddeb200640 1 --2- 172.21.15.67:0/351425925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde415b3a0 0x7fdde40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.837+0000 7fddeb200640 1 -- 172.21.15.67:0/351425925 >> 172.21.15.67:0/351425925 conn(0x7fdde40b1fc0 msgr2=0x7fdde40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:17.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.837+0000 7fddeb200640 1 -- 172.21.15.67:0/351425925 shutdown_connections 2024-05-30T16:48:17.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.837+0000 7fddeb200640 1 -- 172.21.15.67:0/351425925 wait complete. 2024-05-30T16:48:17.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.839+0000 7fddeb200640 1 Processor -- start 2024-05-30T16:48:17.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.839+0000 7fddeb200640 1 -- start start 2024-05-30T16:48:17.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.840+0000 7fddeb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde406dc20 0x7fdde40cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.840+0000 7fddeb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde4073310 0x7fdde40b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.840+0000 7fddeb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdde415b3a0 0x7fdde41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.840+0000 7fddeb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdde415b780 con 0x7fdde406dc20 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.840+0000 7fddeb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdde415fee0 con 0x7fdde415b3a0 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.840+0000 7fddeb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdde40ae550 con 0x7fdde4073310 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.840+0000 7fdde9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde406dc20 0x7fdde40cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.840+0000 7fdde9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde406dc20 0x7fdde40cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39064/0 (socket says 172.21.15.67:39064) 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.840+0000 7fdde9e00640 1 -- 172.21.15.67:0/1910388154 learned_addr learned my addr 172.21.15.67:0/1910388154 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.841+0000 7fddea800640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdde415b3a0 0x7fdde41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.841+0000 7fdde9400640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde4073310 0x7fdde40b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.841+0000 7fdde9e00640 1 -- 172.21.15.67:0/1910388154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde4073310 msgr2=0x7fdde40b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.841+0000 7fdde9e00640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde4073310 0x7fdde40b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.841+0000 7fdde9e00640 1 -- 172.21.15.67:0/1910388154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdde415b3a0 msgr2=0x7fdde41582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.841+0000 7fdde9e00640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdde415b3a0 0x7fdde41582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.841+0000 7fdde9e00640 1 -- 172.21.15.67:0/1910388154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde40ccdd0 con 0x7fdde406dc20 2024-05-30T16:48:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.841+0000 7fdde9e00640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde406dc20 0x7fdde40cd870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fddd800d7d0 tx=0x7fddd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.842+0000 7fddde000640 1 -- 172.21.15.67:0/1910388154 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8004030 con 0x7fdde406dc20 2024-05-30T16:48:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.842+0000 7fddde000640 1 -- 172.21.15.67:0/1910388154 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd80041d0 con 0x7fdde406dc20 2024-05-30T16:48:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.842+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde4158c90 con 0x7fdde406dc20 2024-05-30T16:48:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.842+0000 7fddde000640 1 -- 172.21.15.67:0/1910388154 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd800b660 con 0x7fdde406dc20 2024-05-30T16:48:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.842+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde40cd350 con 0x7fdde406dc20 2024-05-30T16:48:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.843+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde4159030 con 0x7fdde406dc20 2024-05-30T16:48:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.844+0000 7fddde000640 1 -- 172.21.15.67:0/1910388154 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fddd800b800 con 0x7fdde406dc20 2024-05-30T16:48:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.845+0000 7fddde000640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fddc806cbc0 0x7fddc806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.845+0000 7fddde000640 1 -- 172.21.15.67:0/1910388154 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5737+0+0 (secure 0 0 0) 0x7fddd808c1c0 con 0x7fdde406dc20 2024-05-30T16:48:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.845+0000 7fdde9400640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fddc806cbc0 0x7fddc806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.846+0000 7fdde9400640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fddc806cbc0 0x7fddc806f080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdde415adb0 tx=0x7fddd4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:17.849+0000 7fddde000640 1 -- 172.21.15.67:0/1910388154 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fddd805a2d0 con 0x7fdde406dc20 2024-05-30T16:48:18.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.105+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdde40689e0 con 0x7fdde406dc20 2024-05-30T16:48:18.107 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-05-30T16:48:18.106+0000 7f4982000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-30T16:48:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.110+0000 7fddde000640 1 -- 172.21.15.67:0/1910388154 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fddd8059bd0 con 0x7fdde406dc20 2024-05-30T16:48:18.111 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-30T16:48:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fddc806cbc0 msgr2=0x7fddc806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fddc806cbc0 0x7fddc806f080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdde415adb0 tx=0x7fddd4009040 comp rx=0 tx=0).stop 2024-05-30T16:48:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde406dc20 msgr2=0x7fdde40cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde406dc20 0x7fdde40cd870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fddd800d7d0 tx=0x7fddd800dc90 comp rx=0 tx=0).stop 2024-05-30T16:48:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 shutdown_connections 2024-05-30T16:48:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fddc806cbc0 0x7fddc806f080 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdde415b3a0 0x7fdde41582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdde4073310 0x7fdde40b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 --2- 172.21.15.67:0/1910388154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde406dc20 0x7fdde40cd870 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.113+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 >> 172.21.15.67:0/1910388154 conn(0x7fdde40b1fc0 msgr2=0x7fdde40b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.114+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 shutdown_connections 2024-05-30T16:48:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.114+0000 7fddeb200640 1 -- 172.21.15.67:0/1910388154 wait complete. 2024-05-30T16:48:18.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-30T16:48:18.248 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T16:48:18.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.345+0000 7f898a800640 1 Processor -- start 2024-05-30T16:48:18.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.345+0000 7f898a800640 1 -- start start 2024-05-30T16:48:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.347+0000 7f898a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f89841639b0 0x7f89840ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.347+0000 7f898a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 0x7f89840cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.347+0000 7f898a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89841670c0 con 0x7f8984163d90 2024-05-30T16:48:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.347+0000 7f898a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f89840cf890 con 0x7f89841639b0 2024-05-30T16:48:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.347+0000 7f898a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f89840ceca0 con 0x7f89840ceee0 2024-05-30T16:48:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.347+0000 7f8989400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f89841639b0 0x7f89840ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.347+0000 7f8988a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8984163d90 0x7f8984166670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55856/0 (socket says 172.21.15.67:55856) 2024-05-30T16:48:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.347+0000 7f8989400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f89841639b0 0x7f89840ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39076/0 (socket says 172.21.15.67:39076) 2024-05-30T16:48:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.347+0000 7f8988a00640 1 -- 172.21.15.67:0/2868153021 learned_addr learned my addr 172.21.15.67:0/2868153021 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.348+0000 7f8989e00640 1 --2- 172.21.15.67:0/2868153021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 0x7f89840cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.348+0000 7f8989400640 1 -- 172.21.15.67:0/2868153021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 msgr2=0x7f89840cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.348+0000 7f8989400640 1 --2- 172.21.15.67:0/2868153021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 0x7f89840cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.348+0000 7f8989400640 1 -- 172.21.15.67:0/2868153021 >> v1:172.21.15.67:6789/0 conn(0x7f8984163d90 legacy=0x7f8984166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.348+0000 7f8989400640 1 -- 172.21.15.67:0/2868153021 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89841673a0 con 0x7f89841639b0 2024-05-30T16:48:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.348+0000 7f8983e00640 1 -- 172.21.15.67:0/2868153021 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2708087016 0 0) 0x7f89841670c0 con 0x7f8984163d90 2024-05-30T16:48:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.348+0000 7f8989e00640 1 --2- 172.21.15.67:0/2868153021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 0x7f89840cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.349+0000 7f8989400640 1 --2- 172.21.15.67:0/2868153021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89841639b0 0x7f89840ce760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f89780097d0 tx=0x7f897802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7d9640b43436e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.349+0000 7f8983e00640 1 -- 172.21.15.67:0/2868153021 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978037070 con 0x7f89841639b0 2024-05-30T16:48:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.349+0000 7f8983e00640 1 -- 172.21.15.67:0/2868153021 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8978032440 con 0x7f89841639b0 2024-05-30T16:48:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.349+0000 7f8983e00640 1 -- 172.21.15.67:0/2868153021 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978032810 con 0x7f89841639b0 2024-05-30T16:48:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.350+0000 7f898a800640 1 -- 172.21.15.67:0/2868153021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89841639b0 msgr2=0x7f89840ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.350+0000 7f898a800640 1 --2- 172.21.15.67:0/2868153021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89841639b0 0x7f89840ce760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f89780097d0 tx=0x7f897802a1c0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.350+0000 7f898a800640 1 -- 172.21.15.67:0/2868153021 shutdown_connections 2024-05-30T16:48:18.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.350+0000 7f898a800640 1 --2- 172.21.15.67:0/2868153021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 0x7f89840cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.350+0000 7f898a800640 1 --2- 172.21.15.67:0/2868153021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89841639b0 0x7f89840ce760 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.350+0000 7f898a800640 1 -- 172.21.15.67:0/2868153021 >> 172.21.15.67:0/2868153021 conn(0x7f898406b900 msgr2=0x7f898406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:18.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.350+0000 7f898a800640 1 -- 172.21.15.67:0/2868153021 shutdown_connections 2024-05-30T16:48:18.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.350+0000 7f898a800640 1 -- 172.21.15.67:0/2868153021 wait complete. 2024-05-30T16:48:18.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.352+0000 7f898a800640 1 Processor -- start 2024-05-30T16:48:18.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.352+0000 7f898a800640 1 -- start start 2024-05-30T16:48:18.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.353+0000 7f898a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 0x7f89840ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:18.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.353+0000 7f898a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89841639b0 0x7f89840b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:18.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.353+0000 7f898a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8984163d90 0x7f8984148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:18.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.353+0000 7f898a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8984166d80 con 0x7f8984163d90 2024-05-30T16:48:18.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.353+0000 7f898a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f89840ce240 con 0x7f89841639b0 2024-05-30T16:48:18.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.353+0000 7f898a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f89840afa70 con 0x7f89840ceee0 2024-05-30T16:48:18.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.353+0000 7f8989e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8984163d90 0x7f8984148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:18.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.353+0000 7f8989e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8984163d90 0x7f8984148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39092/0 (socket says 172.21.15.67:39092) 2024-05-30T16:48:18.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.353+0000 7f8989e00640 1 -- 172.21.15.67:0/2584030728 learned_addr learned my addr 172.21.15.67:0/2584030728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:18.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f8989e00640 1 -- 172.21.15.67:0/2584030728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 msgr2=0x7f89840ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:18.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f8989e00640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 0x7f89840ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f8989e00640 1 -- 172.21.15.67:0/2584030728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89841639b0 msgr2=0x7f89840b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:18.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f8989e00640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89841639b0 0x7f89840b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:18.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f8989e00640 1 -- 172.21.15.67:0/2584030728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89840b2400 con 0x7f8984163d90 2024-05-30T16:48:18.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f8989e00640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8984163d90 0x7f8984148850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f897c00cef0 tx=0x7f897c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:18.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f8981600640 1 -- 172.21.15.67:0/2584030728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c01a070 con 0x7f8984163d90 2024-05-30T16:48:18.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f8981600640 1 -- 172.21.15.67:0/2584030728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897c00d860 con 0x7f8984163d90 2024-05-30T16:48:18.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f8981600640 1 -- 172.21.15.67:0/2584030728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c00dc50 con 0x7f8984163d90 2024-05-30T16:48:18.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89840b26e0 con 0x7f8984163d90 2024-05-30T16:48:18.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.354+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8984149660 con 0x7f8984163d90 2024-05-30T16:48:18.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.356+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89840689e0 con 0x7f8984163d90 2024-05-30T16:48:18.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.357+0000 7f8981600640 1 -- 172.21.15.67:0/2584030728 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f897c01d070 con 0x7f8984163d90 2024-05-30T16:48:18.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.357+0000 7f8981600640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f896406cbc0 0x7f896406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:18.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.358+0000 7f8981600640 1 -- 172.21.15.67:0/2584030728 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5795+0+0 (secure 0 0 0) 0x7f897c08c980 con 0x7f8984163d90 2024-05-30T16:48:18.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.358+0000 7f8989400640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f896406cbc0 0x7f896406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:18.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.359+0000 7f8989400640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f896406cbc0 0x7f896406f080 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8978034000 tx=0x7f897802c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:18.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.362+0000 7f8981600640 1 -- 172.21.15.67:0/2584030728 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f897c05aa30 con 0x7f8984163d90 2024-05-30T16:48:18.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:18.608+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f89840ca110 con 0x7f8984163d90 2024-05-30T16:48:19.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.114+0000 7f8981600640 1 -- 172.21.15.67:0/2584030728 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f897c05a330 con 0x7f8984163d90 2024-05-30T16:48:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.117+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f896406cbc0 msgr2=0x7f896406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.117+0000 7f898a800640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f896406cbc0 0x7f896406f080 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8978034000 tx=0x7f897802c000 comp rx=0 tx=0).stop 2024-05-30T16:48:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.117+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8984163d90 msgr2=0x7f8984148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.117+0000 7f898a800640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8984163d90 0x7f8984148850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f897c00cef0 tx=0x7f897c00a710 comp rx=0 tx=0).stop 2024-05-30T16:48:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.118+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 shutdown_connections 2024-05-30T16:48:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.118+0000 7f898a800640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f896406cbc0 0x7f896406f080 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.118+0000 7f898a800640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8984163d90 0x7f8984148850 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.118+0000 7f898a800640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89841639b0 0x7f89840b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.118+0000 7f898a800640 1 --2- 172.21.15.67:0/2584030728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89840ceee0 0x7f89840ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.118+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 >> 172.21.15.67:0/2584030728 conn(0x7f898406b900 msgr2=0x7f89840cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.118+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 shutdown_connections 2024-05-30T16:48:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.118+0000 7f898a800640 1 -- 172.21.15.67:0/2584030728 wait complete. 2024-05-30T16:48:19.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-30T16:48:19.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.354+0000 7f7261e00640 1 Processor -- start 2024-05-30T16:48:19.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.355+0000 7f7261e00640 1 -- start start 2024-05-30T16:48:19.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.356+0000 7f7261e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f725c15e5a0 0x7f725c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:19.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.356+0000 7f7261e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f725c0cfb40 0x7f725c0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:19.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.356+0000 7f7261e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f725c0d3500 con 0x7f725c0cffd0 2024-05-30T16:48:19.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.356+0000 7f7261e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f725c0d2b30 con 0x7f725c15e5a0 2024-05-30T16:48:19.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.356+0000 7f7261e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f725c0d2200 con 0x7f725c0cfb40 2024-05-30T16:48:19.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.356+0000 7f7260a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f725c15e5a0 0x7f725c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:19.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.356+0000 7f7260a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f725c15e5a0 0x7f725c0d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39108/0 (socket says 172.21.15.67:39108) 2024-05-30T16:48:19.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.357+0000 7f7260a00640 1 -- 172.21.15.67:0/2519099448 learned_addr learned my addr 172.21.15.67:0/2519099448 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:19.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.357+0000 7f725be00640 1 --2- 172.21.15.67:0/2519099448 >> v2:172.21.15.169:3300/0 conn(0x7f725c0cfb40 0x7f725c0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:19.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.357+0000 7f7260a00640 1 -- 172.21.15.67:0/2519099448 >> v2:172.21.15.169:3300/0 conn(0x7f725c0cfb40 msgr2=0x7f725c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:19.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.357+0000 7f7260a00640 1 --2- 172.21.15.67:0/2519099448 >> v2:172.21.15.169:3300/0 conn(0x7f725c0cfb40 0x7f725c0d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:19.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.357+0000 7f7260a00640 1 -- 172.21.15.67:0/2519099448 >> v1:172.21.15.169:6789/0 conn(0x7f725c0cffd0 legacy=0x7f725c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:19.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.357+0000 7f7260a00640 1 -- 172.21.15.67:0/2519099448 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c0d24e0 con 0x7f725c15e5a0 2024-05-30T16:48:19.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.357+0000 7f725be00640 1 --2- 172.21.15.67:0/2519099448 >> v2:172.21.15.169:3300/0 conn(0x7f725c0cfb40 0x7f725c0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.358+0000 7f7260a00640 1 --2- 172.21.15.67:0/2519099448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 0x7f725c0d2fc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f725c073fa0 tx=0x7f725002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a207ba26ca8a365 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.358+0000 7f725b400640 1 -- 172.21.15.67:0/2519099448 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250037070 con 0x7f725c15e5a0 2024-05-30T16:48:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.358+0000 7f725b400640 1 -- 172.21.15.67:0/2519099448 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7250032440 con 0x7f725c15e5a0 2024-05-30T16:48:19.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.358+0000 7f725b400640 1 -- 172.21.15.67:0/2519099448 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250032810 con 0x7f725c15e5a0 2024-05-30T16:48:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.359+0000 7f7261e00640 1 -- 172.21.15.67:0/2519099448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 msgr2=0x7f725c0d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.359+0000 7f7261e00640 1 --2- 172.21.15.67:0/2519099448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 0x7f725c0d2fc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f725c073fa0 tx=0x7f725002a230 comp rx=0 tx=0).stop 2024-05-30T16:48:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.359+0000 7f7261e00640 1 -- 172.21.15.67:0/2519099448 shutdown_connections 2024-05-30T16:48:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.359+0000 7f7261e00640 1 --2- 172.21.15.67:0/2519099448 >> v2:172.21.15.169:3300/0 conn(0x7f725c0cfb40 0x7f725c0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.359+0000 7f7261e00640 1 --2- 172.21.15.67:0/2519099448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 0x7f725c0d2fc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.359+0000 7f7261e00640 1 -- 172.21.15.67:0/2519099448 >> 172.21.15.67:0/2519099448 conn(0x7f725c08bfc0 msgr2=0x7f725c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:19.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.359+0000 7f7261e00640 1 -- 172.21.15.67:0/2519099448 shutdown_connections 2024-05-30T16:48:19.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.359+0000 7f7261e00640 1 -- 172.21.15.67:0/2519099448 wait complete. 2024-05-30T16:48:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.361+0000 7f7261e00640 1 Processor -- start 2024-05-30T16:48:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.361+0000 7f7261e00640 1 -- start start 2024-05-30T16:48:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.362+0000 7f7261e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c0cfb40 0x7f725c14b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.362+0000 7f7261e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0cffd0 0x7f725c14b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.362+0000 7f7261e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 0x7f725c14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.362+0000 7f7261e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f725c15eb00 con 0x7f725c15e5a0 2024-05-30T16:48:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.362+0000 7f7261e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f725c15e980 con 0x7f725c0cfb40 2024-05-30T16:48:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.362+0000 7f7261e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f725c0d5220 con 0x7f725c0cffd0 2024-05-30T16:48:19.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.362+0000 7f7261400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 0x7f725c14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:19.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.362+0000 7f7261400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 0x7f725c14d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39118/0 (socket says 172.21.15.67:39118) 2024-05-30T16:48:19.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.363+0000 7f7261400640 1 -- 172.21.15.67:0/1628078428 learned_addr learned my addr 172.21.15.67:0/1628078428 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.363+0000 7f725be00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0cffd0 0x7f725c14b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.363+0000 7f7260a00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c0cfb40 0x7f725c14b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.363+0000 7f7261400640 1 -- 172.21.15.67:0/1628078428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0cffd0 msgr2=0x7f725c14b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.363+0000 7f7261400640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0cffd0 0x7f725c14b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.363+0000 7f7261400640 1 -- 172.21.15.67:0/1628078428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c0cfb40 msgr2=0x7f725c14b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.363+0000 7f7261400640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c0cfb40 0x7f725c14b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.363+0000 7f7261400640 1 -- 172.21.15.67:0/1628078428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c0b7530 con 0x7f725c15e5a0 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.363+0000 7f7261400640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 0x7f725c14d9d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f725400c7b0 tx=0x7f725400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.364+0000 7f7258c00640 1 -- 172.21.15.67:0/1628078428 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254017070 con 0x7f725c15e5a0 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.364+0000 7f7258c00640 1 -- 172.21.15.67:0/1628078428 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7254003da0 con 0x7f725c15e5a0 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.364+0000 7f7258c00640 1 -- 172.21.15.67:0/1628078428 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72540052b0 con 0x7f725c15e5a0 2024-05-30T16:48:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.364+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c0b7810 con 0x7f725c15e5a0 2024-05-30T16:48:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.364+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c0b53e0 con 0x7f725c15e5a0 2024-05-30T16:48:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.365+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725c0689e0 con 0x7f725c15e5a0 2024-05-30T16:48:19.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.366+0000 7f7258c00640 1 -- 172.21.15.67:0/1628078428 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f7254005450 con 0x7f725c15e5a0 2024-05-30T16:48:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.367+0000 7f7258c00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723c06cbc0 0x7f723c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.368+0000 7f7258c00640 1 -- 172.21.15.67:0/1628078428 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5795+0+0 (secure 0 0 0) 0x7f725408b610 con 0x7f725c15e5a0 2024-05-30T16:48:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.368+0000 7f7260a00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723c06cbc0 0x7f723c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.369+0000 7f7260a00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723c06cbc0 0x7f723c06f080 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7250034000 tx=0x7f725002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.372+0000 7f7258c00640 1 -- 172.21.15.67:0/1628078428 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7254059650 con 0x7f725c15e5a0 2024-05-30T16:48:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:19.576+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f725c0d1d30 con 0x7f725c15e5a0 2024-05-30T16:48:20.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.437+0000 7f7258c00640 1 -- 172.21.15.67:0/1628078428 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) ==== 181+0+0 (secure 0 0 0) 0x7f7254058f50 con 0x7f725c15e5a0 2024-05-30T16:48:20.438 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-05-30T16:48:20.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.441+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723c06cbc0 msgr2=0x7f723c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:20.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.441+0000 7f7261e00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723c06cbc0 0x7f723c06f080 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7250034000 tx=0x7f725002e000 comp rx=0 tx=0).stop 2024-05-30T16:48:20.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.441+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 msgr2=0x7f725c14d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:20.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.441+0000 7f7261e00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 0x7f725c14d9d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f725400c7b0 tx=0x7f725400cc70 comp rx=0 tx=0).stop 2024-05-30T16:48:20.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.442+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 shutdown_connections 2024-05-30T16:48:20.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.442+0000 7f7261e00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723c06cbc0 0x7f723c06f080 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.442+0000 7f7261e00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c15e5a0 0x7f725c14d9d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.442+0000 7f7261e00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0cffd0 0x7f725c14b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.442+0000 7f7261e00640 1 --2- 172.21.15.67:0/1628078428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c0cfb40 0x7f725c14b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.442+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 >> 172.21.15.67:0/1628078428 conn(0x7f725c08bfc0 msgr2=0x7f725c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:20.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.444+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 shutdown_connections 2024-05-30T16:48:20.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.444+0000 7f7261e00640 1 -- 172.21.15.67:0/1628078428 wait complete. 2024-05-30T16:48:20.469 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-30T16:48:20.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.676+0000 7fe96d600640 1 Processor -- start 2024-05-30T16:48:20.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.676+0000 7fe96d600640 1 -- start start 2024-05-30T16:48:20.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.677+0000 7fe96d600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9681600f0 0x7fe9680d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:20.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.677+0000 7fe96d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9680d0970 0x7fe968072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:20.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.677+0000 7fe96d600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9680d0d90 con 0x7fe96815fa00 2024-05-30T16:48:20.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.677+0000 7fe96d600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe96806fa00 con 0x7fe9680d0970 2024-05-30T16:48:20.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.677+0000 7fe96d600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe96806f160 con 0x7fe9681600f0 2024-05-30T16:48:20.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.678+0000 7fe96cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9680d0970 0x7fe968072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:20.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.678+0000 7fe96cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9680d0970 0x7fe968072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39126/0 (socket says 172.21.15.67:39126) 2024-05-30T16:48:20.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.678+0000 7fe96cc00640 1 -- 172.21.15.67:0/2312313269 learned_addr learned my addr 172.21.15.67:0/2312313269 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:20.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.678+0000 7fe95fe00640 1 --2- 172.21.15.67:0/2312313269 >> v2:172.21.15.169:3300/0 conn(0x7fe9681600f0 0x7fe9680d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:20.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.678+0000 7fe96cc00640 1 -- 172.21.15.67:0/2312313269 >> v2:172.21.15.169:3300/0 conn(0x7fe9681600f0 msgr2=0x7fe9680d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:20.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.678+0000 7fe96cc00640 1 --2- 172.21.15.67:0/2312313269 >> v2:172.21.15.169:3300/0 conn(0x7fe9681600f0 0x7fe9680d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.678+0000 7fe96cc00640 1 -- 172.21.15.67:0/2312313269 >> v1:172.21.15.169:6789/0 conn(0x7fe96815fa00 legacy=0x7fe96806e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:20.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.678+0000 7fe96cc00640 1 -- 172.21.15.67:0/2312313269 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96806f440 con 0x7fe9680d0970 2024-05-30T16:48:20.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.678+0000 7fe95fe00640 1 --2- 172.21.15.67:0/2312313269 >> v2:172.21.15.169:3300/0 conn(0x7fe9681600f0 0x7fe9680d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.679+0000 7fe96cc00640 1 --2- 172.21.15.67:0/2312313269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 0x7fe968072190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe96806d950 tx=0x7fe96002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6938b6ac71d4d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.680+0000 7fe967400640 1 -- 172.21.15.67:0/2312313269 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960036070 con 0x7fe9680d0970 2024-05-30T16:48:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.680+0000 7fe967400640 1 -- 172.21.15.67:0/2312313269 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe960031600 con 0x7fe9680d0970 2024-05-30T16:48:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.680+0000 7fe967400640 1 -- 172.21.15.67:0/2312313269 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9600319d0 con 0x7fe9680d0970 2024-05-30T16:48:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.680+0000 7fe96d600640 1 -- 172.21.15.67:0/2312313269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 msgr2=0x7fe968072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.680+0000 7fe96d600640 1 --2- 172.21.15.67:0/2312313269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 0x7fe968072190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe96806d950 tx=0x7fe96002a5d0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.681+0000 7fe96d600640 1 -- 172.21.15.67:0/2312313269 shutdown_connections 2024-05-30T16:48:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.681+0000 7fe96d600640 1 --2- 172.21.15.67:0/2312313269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 0x7fe968072190 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.681+0000 7fe96d600640 1 --2- 172.21.15.67:0/2312313269 >> v2:172.21.15.169:3300/0 conn(0x7fe9681600f0 0x7fe9680d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.681+0000 7fe96d600640 1 -- 172.21.15.67:0/2312313269 >> 172.21.15.67:0/2312313269 conn(0x7fe96806bae0 msgr2=0x7fe96806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.681+0000 7fe96d600640 1 -- 172.21.15.67:0/2312313269 shutdown_connections 2024-05-30T16:48:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.681+0000 7fe96d600640 1 -- 172.21.15.67:0/2312313269 wait complete. 2024-05-30T16:48:20.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.682+0000 7fe96d600640 1 Processor -- start 2024-05-30T16:48:20.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.683+0000 7fe96d600640 1 -- start start 2024-05-30T16:48:20.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.683+0000 7fe96d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 0x7fe968150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:20.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.683+0000 7fe96d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96815fa00 0x7fe96809acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:20.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.683+0000 7fe96d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9681600f0 0x7fe96815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:20.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.683+0000 7fe96d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe96815fde0 con 0x7fe9680d0970 2024-05-30T16:48:20.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.683+0000 7fe96d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe968071c70 con 0x7fe96815fa00 2024-05-30T16:48:20.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.683+0000 7fe96d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9680726d0 con 0x7fe9681600f0 2024-05-30T16:48:20.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe967e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 0x7fe968150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:20.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe967e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 0x7fe968150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39134/0 (socket says 172.21.15.67:39134) 2024-05-30T16:48:20.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe967e00640 1 -- 172.21.15.67:0/177258277 learned_addr learned my addr 172.21.15.67:0/177258277 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:20.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe95fe00640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96815fa00 0x7fe96809acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:20.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe967e00640 1 -- 172.21.15.67:0/177258277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9681600f0 msgr2=0x7fe96815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:20.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe967e00640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9681600f0 0x7fe96815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe967e00640 1 -- 172.21.15.67:0/177258277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96815fa00 msgr2=0x7fe96809acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:20.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe967e00640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96815fa00 0x7fe96809acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:20.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe967e00640 1 -- 172.21.15.67:0/177258277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9680b06e0 con 0x7fe9680d0970 2024-05-30T16:48:20.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.684+0000 7fe967e00640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 0x7fe968150530 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c460 tx=0x7fe95800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:20.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.685+0000 7fe964c00640 1 -- 172.21.15.67:0/177258277 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958016020 con 0x7fe9680d0970 2024-05-30T16:48:20.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.685+0000 7fe964c00640 1 -- 172.21.15.67:0/177258277 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe958012870 con 0x7fe9680d0970 2024-05-30T16:48:20.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.685+0000 7fe964c00640 1 -- 172.21.15.67:0/177258277 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958012c60 con 0x7fe9680d0970 2024-05-30T16:48:20.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.685+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9680b09c0 con 0x7fe9680d0970 2024-05-30T16:48:20.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.685+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9680b0f80 con 0x7fe9680d0970 2024-05-30T16:48:20.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.686+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9680689e0 con 0x7fe9680d0970 2024-05-30T16:48:20.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.687+0000 7fe964c00640 1 -- 172.21.15.67:0/177258277 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fe958019020 con 0x7fe9680d0970 2024-05-30T16:48:20.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.688+0000 7fe964c00640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe94406cbc0 0x7fe94406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:20.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.688+0000 7fe964c00640 1 -- 172.21.15.67:0/177258277 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5357+0+0 (secure 0 0 0) 0x7fe95808bcc0 con 0x7fe9680d0970 2024-05-30T16:48:20.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.688+0000 7fe95fe00640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe94406cbc0 0x7fe94406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:20.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.689+0000 7fe95fe00640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe94406cbc0 0x7fe94406f080 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe968072610 tx=0x7fe950008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:20.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.692+0000 7fe964c00640 1 -- 172.21.15.67:0/177258277 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe958092080 con 0x7fe9680d0970 2024-05-30T16:48:20.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:20.909+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe9680b1c30 con 0x7fe9680d0970 2024-05-30T16:48:21.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.441+0000 7fe964c00640 1 -- 172.21.15.67:0/177258277 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) ==== 169+0+0 (secure 0 0 0) 0x7fe958020030 con 0x7fe9680d0970 2024-05-30T16:48:21.442 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-05-30T16:48:21.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.444+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe94406cbc0 msgr2=0x7fe94406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:21.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.444+0000 7fe96d600640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe94406cbc0 0x7fe94406f080 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe968072610 tx=0x7fe950008040 comp rx=0 tx=0).stop 2024-05-30T16:48:21.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.444+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 msgr2=0x7fe968150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:21.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.444+0000 7fe96d600640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 0x7fe968150530 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c460 tx=0x7fe95800c920 comp rx=0 tx=0).stop 2024-05-30T16:48:21.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.446+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 shutdown_connections 2024-05-30T16:48:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.446+0000 7fe96d600640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe94406cbc0 0x7fe94406f080 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.446+0000 7fe96d600640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9681600f0 0x7fe96815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.446+0000 7fe96d600640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96815fa00 0x7fe96809acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.446+0000 7fe96d600640 1 --2- 172.21.15.67:0/177258277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9680d0970 0x7fe968150530 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.446+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 >> 172.21.15.67:0/177258277 conn(0x7fe96806bae0 msgr2=0x7fe968072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:21.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.447+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 shutdown_connections 2024-05-30T16:48:21.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.448+0000 7fe96d600640 1 -- 172.21.15.67:0/177258277 wait complete. 2024-05-30T16:48:21.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-30T16:48:21.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.667+0000 7fcfe7800640 1 Processor -- start 2024-05-30T16:48:21.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.667+0000 7fcfe7800640 1 -- start start 2024-05-30T16:48:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.669+0000 7fcfe7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe0161e70 0x7fcfe00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.669+0000 7fcfe7800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcfe00b3800 0x7fcfe00ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.669+0000 7fcfe7800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe00b3d40 con 0x7fcfe0161a90 2024-05-30T16:48:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.669+0000 7fcfe7800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe00b4020 con 0x7fcfe00b3800 2024-05-30T16:48:21.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.669+0000 7fcfe7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe00b4300 con 0x7fcfe0161e70 2024-05-30T16:48:21.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.669+0000 7fcfe4e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcfe0161a90 0x7fcfe006e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45910/0 (socket says 172.21.15.67:45910) 2024-05-30T16:48:21.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.669+0000 7fcfe4e00640 1 -- 172.21.15.67:0/3742209792 learned_addr learned my addr 172.21.15.67:0/3742209792 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdfe00640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe0161e70 0x7fcfe00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfe5800640 1 --2- 172.21.15.67:0/3742209792 >> v2:172.21.15.169:3300/0 conn(0x7fcfe00b3800 0x7fcfe00ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2881804735 0 0) 0x7fcfe00b3d40 con 0x7fcfe0161a90 2024-05-30T16:48:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfc4003520 con 0x7fcfe0161a90 2024-05-30T16:48:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3998365939 0 0) 0x7fcfc4003520 con 0x7fcfe0161a90 2024-05-30T16:48:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe0161e70 msgr2=0x7fcfe00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe0161e70 0x7fcfe00b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 >> v2:172.21.15.169:3300/0 conn(0x7fcfe00b3800 msgr2=0x7fcfe00ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 --2- 172.21.15.67:0/3742209792 >> v2:172.21.15.169:3300/0 conn(0x7fcfe00b3800 0x7fcfe00ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe006dc70 con 0x7fcfe0161a90 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fcfd0001670 con 0x7fcfe0161a90 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 >> v1:172.21.15.67:6789/0 conn(0x7fcfe0161a90 legacy=0x7fcfe006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe0161e70 0x7fcfc40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc4004d40 0x7fcfc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe00b4020 con 0x7fcfc4003f80 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe00b4300 con 0x7fcfc4004d40 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.670+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc400ba70 con 0x7fcfe0161e70 2024-05-30T16:48:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.671+0000 7fcfdfe00640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.671+0000 7fcfe4e00640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe0161e70 0x7fcfc40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.671+0000 7fcfe5800640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc4004d40 0x7fcfc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.671+0000 7fcfdfe00640 1 -- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe0161e70 msgr2=0x7fcfc40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.671+0000 7fcfdfe00640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe0161e70 0x7fcfc40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.671+0000 7fcfdfe00640 1 -- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc4004d40 msgr2=0x7fcfc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.671+0000 7fcfdfe00640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc4004d40 0x7fcfc40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.671+0000 7fcfdfe00640 1 -- 172.21.15.67:0/3742209792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc4003520 con 0x7fcfc4003f80 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.671+0000 7fcfe5800640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfc4004d40 0x7fcfc40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.672+0000 7fcfdfe00640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfc4004800 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00ba00 tx=0x7fcfc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123875a8a887908c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc030070 con 0x7fcfc4003f80 2024-05-30T16:48:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfcc02adb0 con 0x7fcfc4003f80 2024-05-30T16:48:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfdf400640 1 -- 172.21.15.67:0/3742209792 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00b600 con 0x7fcfc4003f80 2024-05-30T16:48:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfe7800640 1 -- 172.21.15.67:0/3742209792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 msgr2=0x7fcfc4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfe7800640 1 --2- 172.21.15.67:0/3742209792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfc4004800 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00ba00 tx=0x7fcfc4005830 comp rx=0 tx=0).stop 2024-05-30T16:48:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfe5800640 1 -- 172.21.15.67:0/3742209792 reap_dead start 2024-05-30T16:48:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfe7800640 1 -- 172.21.15.67:0/3742209792 shutdown_connections 2024-05-30T16:48:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfe7800640 1 -- 172.21.15.67:0/3742209792 >> 172.21.15.67:0/3742209792 conn(0x7fcfe006bd20 msgr2=0x7fcfe006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfe7800640 1 -- 172.21.15.67:0/3742209792 shutdown_connections 2024-05-30T16:48:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.673+0000 7fcfe7800640 1 -- 172.21.15.67:0/3742209792 wait complete. 2024-05-30T16:48:21.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.675+0000 7fcfe7800640 1 Processor -- start 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.675+0000 7fcfe7800640 1 -- start start 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.676+0000 7fcfe7800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfe0155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.676+0000 7fcfe7800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe01593c0 0x7fcfe0159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.676+0000 7fcfe7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe00d0610 0x7fcfe00d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.676+0000 7fcfe7800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe006eb80 con 0x7fcfc4003f80 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.676+0000 7fcfe7800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe00b5020 con 0x7fcfe01593c0 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.676+0000 7fcfe7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe00b32c0 con 0x7fcfe00d0610 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.676+0000 7fcfe4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfe0155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.677+0000 7fcfe4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfe0155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42648/0 (socket says 172.21.15.67:42648) 2024-05-30T16:48:21.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.677+0000 7fcfe4e00640 1 -- 172.21.15.67:0/3951732390 learned_addr learned my addr 172.21.15.67:0/3951732390 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.677+0000 7fcfe4e00640 1 -- 172.21.15.67:0/3951732390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe00d0610 msgr2=0x7fcfe00d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.677+0000 7fcfe4e00640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe00d0610 0x7fcfe00d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.677+0000 7fcfe4e00640 1 -- 172.21.15.67:0/3951732390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe01593c0 msgr2=0x7fcfe0159840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.677+0000 7fcfe4e00640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe01593c0 0x7fcfe0159840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.677+0000 7fcfe4e00640 1 -- 172.21.15.67:0/3951732390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe01f4d10 con 0x7fcfc4003f80 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.677+0000 7fcfe4e00640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfe0155d20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0000c00 tx=0x7fcfd000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.678+0000 7fcfdcc00640 1 -- 172.21.15.67:0/3951732390 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000ee00 con 0x7fcfc4003f80 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.678+0000 7fcfdcc00640 1 -- 172.21.15.67:0/3951732390 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd0018070 con 0x7fcfc4003f80 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.678+0000 7fcfdcc00640 1 -- 172.21.15.67:0/3951732390 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000ca60 con 0x7fcfc4003f80 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.678+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe01f4ff0 con 0x7fcfc4003f80 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.678+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe01f55b0 con 0x7fcfc4003f80 2024-05-30T16:48:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.679+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe00689e0 con 0x7fcfc4003f80 2024-05-30T16:48:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.680+0000 7fcfdcc00640 1 -- 172.21.15.67:0/3951732390 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fcfd000cc00 con 0x7fcfc4003f80 2024-05-30T16:48:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.681+0000 7fcfdcc00640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfc006cbc0 0x7fcfc006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.681+0000 7fcfdcc00640 1 -- 172.21.15.67:0/3951732390 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7fcfd008bb00 con 0x7fcfc4003f80 2024-05-30T16:48:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.681+0000 7fcfdfe00640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfc006cbc0 0x7fcfc006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:21.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.682+0000 7fcfdfe00640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfc006cbc0 0x7fcfc006f080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0162270 tx=0x7fcfcc030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:21.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.686+0000 7fcfdcc00640 1 -- 172.21.15.67:0/3951732390 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfd0059ee0 con 0x7fcfc4003f80 2024-05-30T16:48:21.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.889+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fcfe006e930 con 0x7fcfc4003f80 2024-05-30T16:48:21.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.896+0000 7fcfdcc00640 1 -- 172.21.15.67:0/3951732390 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fcfd00597e0 con 0x7fcfc4003f80 2024-05-30T16:48:21.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfc006cbc0 msgr2=0x7fcfc006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:21.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfc006cbc0 0x7fcfc006f080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0162270 tx=0x7fcfcc030000 comp rx=0 tx=0).stop 2024-05-30T16:48:21.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 msgr2=0x7fcfe0155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:21.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfe0155d20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0000c00 tx=0x7fcfd000ec10 comp rx=0 tx=0).stop 2024-05-30T16:48:21.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 shutdown_connections 2024-05-30T16:48:21.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfc006cbc0 0x7fcfc006f080 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfe00d0610 0x7fcfe00d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe01593c0 0x7fcfe0159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 --2- 172.21.15.67:0/3951732390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfc4003f80 0x7fcfe0155d20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:21.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 >> 172.21.15.67:0/3951732390 conn(0x7fcfe006bd20 msgr2=0x7fcfe006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:21.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 shutdown_connections 2024-05-30T16:48:21.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:21.899+0000 7fcfe7800640 1 -- 172.21.15.67:0/3951732390 wait complete. 2024-05-30T16:48:21.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone' 2024-05-30T16:48:22.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.116+0000 7f16c2600640 1 Processor -- start 2024-05-30T16:48:22.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.117+0000 7f16c2600640 1 -- start start 2024-05-30T16:48:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.118+0000 7f16c2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f16bc15e870 0x7f16bc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.118+0000 7f16c2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 0x7f16bc0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.118+0000 7f16c2600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16bc15ec50 con 0x7f16bc0d14d0 2024-05-30T16:48:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.118+0000 7f16c2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f16bc0d2150 con 0x7f16bc15e870 2024-05-30T16:48:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.118+0000 7f16c2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc0cff80 con 0x7f16bc0cfa40 2024-05-30T16:48:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.118+0000 7f16c1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f16bc15e870 0x7f16bc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.118+0000 7f16bbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f16bc0d14d0 0x7f16bc0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45920/0 (socket says 172.21.15.67:45920) 2024-05-30T16:48:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.118+0000 7f16bbe00640 1 -- 172.21.15.67:0/2082784233 learned_addr learned my addr 172.21.15.67:0/2082784233 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.118+0000 7f16c1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f16bc15e870 0x7f16bc0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42652/0 (socket says 172.21.15.67:42652) 2024-05-30T16:48:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.119+0000 7f16c1c00640 1 --2- 172.21.15.67:0/2082784233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 0x7f16bc0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.119+0000 7f16c1200640 1 -- 172.21.15.67:0/2082784233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 msgr2=0x7f16bc0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.119+0000 7f16c1200640 1 --2- 172.21.15.67:0/2082784233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 0x7f16bc0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.119+0000 7f16c1200640 1 -- 172.21.15.67:0/2082784233 >> v1:172.21.15.67:6789/0 conn(0x7f16bc0d14d0 legacy=0x7f16bc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.119+0000 7f16bb400640 1 -- 172.21.15.67:0/2082784233 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064016812 0 0) 0x7f16bc15ec50 con 0x7f16bc0d14d0 2024-05-30T16:48:22.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.119+0000 7f16c1200640 1 -- 172.21.15.67:0/2082784233 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc0d0260 con 0x7f16bc15e870 2024-05-30T16:48:22.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.120+0000 7f16c1200640 1 --2- 172.21.15.67:0/2082784233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc15e870 0x7f16bc0d0f90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0097a0 tx=0x7f16ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cfe0b6bc17beee server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:22.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.120+0000 7f16bb400640 1 -- 172.21.15.67:0/2082784233 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac037070 con 0x7f16bc15e870 2024-05-30T16:48:22.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.120+0000 7f16bb400640 1 -- 172.21.15.67:0/2082784233 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ac032440 con 0x7f16bc15e870 2024-05-30T16:48:22.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.120+0000 7f16bb400640 1 -- 172.21.15.67:0/2082784233 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac032810 con 0x7f16bc15e870 2024-05-30T16:48:22.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.121+0000 7f16c2600640 1 -- 172.21.15.67:0/2082784233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc15e870 msgr2=0x7f16bc0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.121+0000 7f16c2600640 1 --2- 172.21.15.67:0/2082784233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc15e870 0x7f16bc0d0f90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0097a0 tx=0x7f16ac02a180 comp rx=0 tx=0).stop 2024-05-30T16:48:22.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.121+0000 7f16c2600640 1 -- 172.21.15.67:0/2082784233 shutdown_connections 2024-05-30T16:48:22.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.121+0000 7f16c2600640 1 --2- 172.21.15.67:0/2082784233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 0x7f16bc0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.121+0000 7f16c2600640 1 --2- 172.21.15.67:0/2082784233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc15e870 0x7f16bc0d0f90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.121+0000 7f16c2600640 1 -- 172.21.15.67:0/2082784233 >> 172.21.15.67:0/2082784233 conn(0x7f16bc08bfc0 msgr2=0x7f16bc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:22.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.121+0000 7f16c2600640 1 -- 172.21.15.67:0/2082784233 shutdown_connections 2024-05-30T16:48:22.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.121+0000 7f16c2600640 1 -- 172.21.15.67:0/2082784233 wait complete. 2024-05-30T16:48:22.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.123+0000 7f16c2600640 1 Processor -- start 2024-05-30T16:48:22.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.123+0000 7f16c2600640 1 -- start start 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.124+0000 7f16c2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 0x7f16bc0b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.124+0000 7f16c2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc0d14d0 0x7f16bc159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.124+0000 7f16c2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16bc15e870 0x7f16bc15ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.124+0000 7f16c2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc0d4fb0 con 0x7f16bc0d14d0 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.124+0000 7f16c2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc0d0820 con 0x7f16bc15e870 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.124+0000 7f16c2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc0d2b20 con 0x7f16bc0cfa40 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.124+0000 7f16bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc0d14d0 0x7f16bc159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.124+0000 7f16bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc0d14d0 0x7f16bc159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42656/0 (socket says 172.21.15.67:42656) 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.124+0000 7f16bbe00640 1 -- 172.21.15.67:0/4186573524 learned_addr learned my addr 172.21.15.67:0/4186573524 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:22.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.125+0000 7f16c1200640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 0x7f16bc0b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.125+0000 7f16c1c00640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16bc15e870 0x7f16bc15ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.125+0000 7f16bbe00640 1 -- 172.21.15.67:0/4186573524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 msgr2=0x7f16bc0b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.125+0000 7f16bbe00640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 0x7f16bc0b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.125+0000 7f16bbe00640 1 -- 172.21.15.67:0/4186573524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16bc15e870 msgr2=0x7f16bc15ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.125+0000 7f16bbe00640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16bc15e870 0x7f16bc15ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.125+0000 7f16bbe00640 1 -- 172.21.15.67:0/4186573524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc15fd10 con 0x7f16bc0d14d0 2024-05-30T16:48:22.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.125+0000 7f16c1200640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 0x7f16bc0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:22.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.125+0000 7f16bbe00640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc0d14d0 0x7f16bc159b70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f16a800c840 tx=0x7f16a800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:22.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.126+0000 7f16b8c00640 1 -- 172.21.15.67:0/4186573524 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800a640 con 0x7f16bc0d14d0 2024-05-30T16:48:22.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.126+0000 7f16b8c00640 1 -- 172.21.15.67:0/4186573524 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a800a7e0 con 0x7f16bc0d14d0 2024-05-30T16:48:22.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.126+0000 7f16b8c00640 1 -- 172.21.15.67:0/4186573524 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800abd0 con 0x7f16bc0d14d0 2024-05-30T16:48:22.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.126+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc15fff0 con 0x7f16bc0d14d0 2024-05-30T16:48:22.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.126+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc0b76d0 con 0x7f16bc0d14d0 2024-05-30T16:48:22.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.127+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc0b78d0 con 0x7f16bc0d14d0 2024-05-30T16:48:22.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.128+0000 7f16b8c00640 1 -- 172.21.15.67:0/4186573524 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f16a8007070 con 0x7f16bc0d14d0 2024-05-30T16:48:22.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.129+0000 7f16b8c00640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169c06cbc0 0x7f169c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.129+0000 7f16b8c00640 1 -- 172.21.15.67:0/4186573524 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f16a808bcc0 con 0x7f16bc0d14d0 2024-05-30T16:48:22.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.129+0000 7f16c1200640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169c06cbc0 0x7f169c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.130+0000 7f16c1200640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169c06cbc0 0x7f169c06f080 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16ac034000 tx=0x7f16ac02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:22.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.137+0000 7f16b8c00640 1 -- 172.21.15.67:0/4186573524 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16bc0b78d0 con 0x7f16bc0d14d0 2024-05-30T16:48:22.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.426+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]} v 0) -- 0x7f16bc0689e0 con 0x7f16bc0d14d0 2024-05-30T16:48:22.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.458+0000 7f16b8c00640 1 -- 172.21.15.67:0/4186573524 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]}]=0 v197) ==== 160+0+0 (secure 0 0 0) 0x7f16a8092080 con 0x7f16bc0d14d0 2024-05-30T16:48:22.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.460+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169c06cbc0 msgr2=0x7f169c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.460+0000 7f16c2600640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169c06cbc0 0x7f169c06f080 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16ac034000 tx=0x7f16ac02e000 comp rx=0 tx=0).stop 2024-05-30T16:48:22.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.460+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc0d14d0 msgr2=0x7f16bc159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.460+0000 7f16c2600640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc0d14d0 0x7f16bc159b70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f16a800c840 tx=0x7f16a800cd00 comp rx=0 tx=0).stop 2024-05-30T16:48:22.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.461+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 shutdown_connections 2024-05-30T16:48:22.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.461+0000 7f16c2600640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169c06cbc0 0x7f169c06f080 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.461+0000 7f16c2600640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16bc15e870 0x7f16bc15ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.461+0000 7f16c2600640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16bc0d14d0 0x7f16bc159b70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.461+0000 7f16c2600640 1 --2- 172.21.15.67:0/4186573524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16bc0cfa40 0x7f16bc0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.461+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 >> 172.21.15.67:0/4186573524 conn(0x7f16bc08bfc0 msgr2=0x7f16bc0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:22.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.461+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 shutdown_connections 2024-05-30T16:48:22.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.461+0000 7f16c2600640 1 -- 172.21.15.67:0/4186573524 wait complete. 2024-05-30T16:48:22.482 INFO:tasks.cephfs_test_runner:Test that when one clone is in progress, one progress bar is printed ... ok 2024-05-30T16:48:22.482 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bars_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed when number of clone jobs is 2024-05-30T16:48:22.482 INFO:tasks.cephfs_test_runner:test_progress_bars_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T16:48:22.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:48:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.694+0000 7f2a5e600640 1 Processor -- start 2024-05-30T16:48:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.695+0000 7f2a5e600640 1 -- start start 2024-05-30T16:48:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.696+0000 7f2a5e600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2a5815e870 0x7f2a580d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.696+0000 7f2a5e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2a580d14d0 0x7f2a580cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.696+0000 7f2a5e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 0x7f2a58162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.696+0000 7f2a5e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2a580d2150 con 0x7f2a580d14d0 2024-05-30T16:48:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.696+0000 7f2a5e600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2a580cfd00 con 0x7f2a5815e870 2024-05-30T16:48:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.696+0000 7f2a5e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5815ec50 con 0x7f2a580cf870 2024-05-30T16:48:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.696+0000 7f2a57e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2a580d14d0 0x7f2a580cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a57e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2a580d14d0 0x7f2a580cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42672/0 (socket says 172.21.15.67:42672) 2024-05-30T16:48:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a57e00640 1 -- 172.21.15.67:0/1773261450 learned_addr learned my addr 172.21.15.67:0/1773261450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a5d200640 1 --2- 172.21.15.67:0/1773261450 >> v2:172.21.15.169:3300/0 conn(0x7f2a5815e870 0x7f2a580d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a5dc00640 1 --2- 172.21.15.67:0/1773261450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 0x7f2a58162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a57e00640 1 -- 172.21.15.67:0/1773261450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 msgr2=0x7f2a58162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a57e00640 1 --2- 172.21.15.67:0/1773261450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 0x7f2a58162360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a57e00640 1 -- 172.21.15.67:0/1773261450 >> v2:172.21.15.169:3300/0 conn(0x7f2a5815e870 msgr2=0x7f2a580d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a57e00640 1 --2- 172.21.15.67:0/1773261450 >> v2:172.21.15.169:3300/0 conn(0x7f2a5815e870 0x7f2a580d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a57e00640 1 -- 172.21.15.67:0/1773261450 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a580cff80 con 0x7f2a580d14d0 2024-05-30T16:48:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.697+0000 7f2a5d200640 1 --2- 172.21.15.67:0/1773261450 >> v2:172.21.15.169:3300/0 conn(0x7f2a5815e870 0x7f2a580d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.698+0000 7f2a57e00640 1 --2- 172.21.15.67:0/1773261450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 0x7f2a580cf330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2a5808e720 tx=0x7f2a4402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f831f645cf7ad2f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.699+0000 7f2a57400640 1 -- 172.21.15.67:0/1773261450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44004030 con 0x7f2a580d14d0 2024-05-30T16:48:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.699+0000 7f2a57400640 1 -- 172.21.15.67:0/1773261450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a440041d0 con 0x7f2a580d14d0 2024-05-30T16:48:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.699+0000 7f2a57400640 1 -- 172.21.15.67:0/1773261450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44003610 con 0x7f2a580d14d0 2024-05-30T16:48:22.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.699+0000 7f2a5e600640 1 -- 172.21.15.67:0/1773261450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 msgr2=0x7f2a580cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.699+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1773261450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 0x7f2a580cf330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2a5808e720 tx=0x7f2a4402a1c0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.700+0000 7f2a5e600640 1 -- 172.21.15.67:0/1773261450 shutdown_connections 2024-05-30T16:48:22.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.700+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1773261450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 0x7f2a580cf330 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.700+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1773261450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 0x7f2a58162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.700+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1773261450 >> v2:172.21.15.169:3300/0 conn(0x7f2a5815e870 0x7f2a580d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.700+0000 7f2a5e600640 1 -- 172.21.15.67:0/1773261450 >> 172.21.15.67:0/1773261450 conn(0x7f2a5808bfc0 msgr2=0x7f2a580d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:22.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.700+0000 7f2a5e600640 1 -- 172.21.15.67:0/1773261450 shutdown_connections 2024-05-30T16:48:22.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.700+0000 7f2a5e600640 1 -- 172.21.15.67:0/1773261450 wait complete. 2024-05-30T16:48:22.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.702+0000 7f2a5e600640 1 Processor -- start 2024-05-30T16:48:22.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.702+0000 7f2a5e600640 1 -- start start 2024-05-30T16:48:22.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a5e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 0x7f2a580cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a5e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 0x7f2a581f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a5e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5815e870 0x7f2a581f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a5e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2a580d0820 con 0x7f2a580d14d0 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a5e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2a580d0540 con 0x7f2a5815e870 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a5e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a580d4fb0 con 0x7f2a580cf870 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a57e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 0x7f2a581f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a57e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 0x7f2a581f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42678/0 (socket says 172.21.15.67:42678) 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a57e00640 1 -- 172.21.15.67:0/1566911166 learned_addr learned my addr 172.21.15.67:0/1566911166 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a5d200640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 0x7f2a580cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.703+0000 7f2a57e00640 1 -- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 msgr2=0x7f2a580cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a5dc00640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5815e870 0x7f2a581f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a57e00640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 0x7f2a580cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a57e00640 1 -- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5815e870 msgr2=0x7f2a581f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a57e00640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5815e870 0x7f2a581f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a57e00640 1 -- 172.21.15.67:0/1566911166 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a580cc300 con 0x7f2a580d14d0 2024-05-30T16:48:22.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a5d200640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 0x7f2a580cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:22.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a5dc00640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5815e870 0x7f2a581f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:22.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a57e00640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 0x7f2a581f4400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2a44034040 tx=0x7f2a440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:22.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a54c00640 1 -- 172.21.15.67:0/1566911166 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44041030 con 0x7f2a580d14d0 2024-05-30T16:48:22.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a54c00640 1 -- 172.21.15.67:0/1566911166 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a44003c50 con 0x7f2a580d14d0 2024-05-30T16:48:22.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a54c00640 1 -- 172.21.15.67:0/1566911166 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44004030 con 0x7f2a580d14d0 2024-05-30T16:48:22.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.704+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a580cc5e0 con 0x7f2a580d14d0 2024-05-30T16:48:22.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.705+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a580ccba0 con 0x7f2a580d14d0 2024-05-30T16:48:22.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.706+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a580689e0 con 0x7f2a580d14d0 2024-05-30T16:48:22.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.707+0000 7f2a54c00640 1 -- 172.21.15.67:0/1566911166 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f2a44044070 con 0x7f2a580d14d0 2024-05-30T16:48:22.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.708+0000 7f2a54c00640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a3806cbc0 0x7f2a3806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:22.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.708+0000 7f2a54c00640 1 -- 172.21.15.67:0/1566911166 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f2a440ac980 con 0x7f2a580d14d0 2024-05-30T16:48:22.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.708+0000 7f2a5d200640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a3806cbc0 0x7f2a3806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:22.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.709+0000 7f2a5d200640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a3806cbc0 0x7f2a3806f080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0088b0 tx=0x7f2a4c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:22.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.713+0000 7f2a54c00640 1 -- 172.21.15.67:0/1566911166 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a4403b3b0 con 0x7f2a580d14d0 2024-05-30T16:48:22.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.921+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a5806cf30 con 0x7f2a580d14d0 2024-05-30T16:48:22.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.923+0000 7f2a54c00640 1 -- 172.21.15.67:0/1566911166 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+13010 (secure 0 0 0) 0x7f2a44003e20 con 0x7f2a580d14d0 2024-05-30T16:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:22.924 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":40,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:48:21.437328+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/2481507396":"2024-05-31T16:48:18.106890+0000","172.21.15.169:6833/2481507396":"2024-05-31T16:48:18.106890+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:48:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a3806cbc0 msgr2=0x7f2a3806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a3806cbc0 0x7f2a3806f080 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0088b0 tx=0x7f2a4c008040 comp rx=0 tx=0).stop 2024-05-30T16:48:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 msgr2=0x7f2a581f4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 0x7f2a581f4400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2a44034040 tx=0x7f2a440046c0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 shutdown_connections 2024-05-30T16:48:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a3806cbc0 0x7f2a3806f080 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5815e870 0x7f2a581f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a580d14d0 0x7f2a581f4400 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 --2- 172.21.15.67:0/1566911166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a580cf870 0x7f2a580cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 >> 172.21.15.67:0/1566911166 conn(0x7f2a5808bfc0 msgr2=0x7f2a580b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.926+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 shutdown_connections 2024-05-30T16:48:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:22.927+0000 7f2a5e600640 1 -- 172.21.15.67:0/1566911166 wait complete. 2024-05-30T16:48:22.946 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-30T16:43:40.009725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-30T16:48:22.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-30T16:48:23.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.160+0000 7feba9000640 1 Processor -- start 2024-05-30T16:48:23.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.160+0000 7feba9000640 1 -- start start 2024-05-30T16:48:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.161+0000 7feba9000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feba415e870 0x7feba40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.161+0000 7feba9000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feba40d14d0 0x7feba40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.161+0000 7feba9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba40cf870 0x7feba4162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.161+0000 7feba9000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7feba40d2150 con 0x7feba40d14d0 2024-05-30T16:48:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.161+0000 7feba9000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7feba40cfd00 con 0x7feba415e870 2024-05-30T16:48:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.161+0000 7feba9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feba415ec50 con 0x7feba40cf870 2024-05-30T16:48:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feba40d14d0 0x7feba40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feba40d14d0 0x7feba40cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42686/0 (socket says 172.21.15.67:42686) 2024-05-30T16:48:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba2a00640 1 -- 172.21.15.67:0/218726622 learned_addr learned my addr 172.21.15.67:0/218726622 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba3400640 1 --2- 172.21.15.67:0/218726622 >> v2:172.21.15.169:3300/0 conn(0x7feba415e870 0x7feba40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba3e00640 1 --2- 172.21.15.67:0/218726622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba40cf870 0x7feba4162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba2a00640 1 -- 172.21.15.67:0/218726622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba40cf870 msgr2=0x7feba4162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba2a00640 1 --2- 172.21.15.67:0/218726622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba40cf870 0x7feba4162360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba2a00640 1 -- 172.21.15.67:0/218726622 >> v2:172.21.15.169:3300/0 conn(0x7feba415e870 msgr2=0x7feba40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba2a00640 1 --2- 172.21.15.67:0/218726622 >> v2:172.21.15.169:3300/0 conn(0x7feba415e870 0x7feba40d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.162+0000 7feba2a00640 1 -- 172.21.15.67:0/218726622 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba40cff80 con 0x7feba40d14d0 2024-05-30T16:48:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.163+0000 7feba2a00640 1 --2- 172.21.15.67:0/218726622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40d14d0 0x7feba40cf330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feba408e720 tx=0x7feb9002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32103a2bc6760b5f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.164+0000 7feba2000640 1 -- 172.21.15.67:0/218726622 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90004030 con 0x7feba40d14d0 2024-05-30T16:48:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.164+0000 7feba2000640 1 -- 172.21.15.67:0/218726622 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb900041d0 con 0x7feba40d14d0 2024-05-30T16:48:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.164+0000 7feba2000640 1 -- 172.21.15.67:0/218726622 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90003610 con 0x7feba40d14d0 2024-05-30T16:48:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.164+0000 7feba9000640 1 -- 172.21.15.67:0/218726622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40d14d0 msgr2=0x7feba40cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.164+0000 7feba9000640 1 --2- 172.21.15.67:0/218726622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40d14d0 0x7feba40cf330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feba408e720 tx=0x7feb9002a1c0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.165+0000 7feba9000640 1 -- 172.21.15.67:0/218726622 shutdown_connections 2024-05-30T16:48:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.165+0000 7feba9000640 1 --2- 172.21.15.67:0/218726622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40d14d0 0x7feba40cf330 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.165+0000 7feba9000640 1 --2- 172.21.15.67:0/218726622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba40cf870 0x7feba4162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.165+0000 7feba9000640 1 --2- 172.21.15.67:0/218726622 >> v2:172.21.15.169:3300/0 conn(0x7feba415e870 0x7feba40d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.165+0000 7feba9000640 1 -- 172.21.15.67:0/218726622 >> 172.21.15.67:0/218726622 conn(0x7feba408bfc0 msgr2=0x7feba40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.165+0000 7feba9000640 1 -- 172.21.15.67:0/218726622 shutdown_connections 2024-05-30T16:48:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.165+0000 7feba9000640 1 -- 172.21.15.67:0/218726622 wait complete. 2024-05-30T16:48:23.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.166+0000 7feba9000640 1 Processor -- start 2024-05-30T16:48:23.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.167+0000 7feba9000640 1 -- start start 2024-05-30T16:48:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40cf870 0x7feba415f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba40d14d0 0x7feba40cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba9000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feba415e870 0x7feba41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba9000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feba40d0820 con 0x7feba40cf870 2024-05-30T16:48:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba9000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feba40d0540 con 0x7feba415e870 2024-05-30T16:48:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feba40d4fb0 con 0x7feba40d14d0 2024-05-30T16:48:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40cf870 0x7feba415f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40cf870 0x7feba415f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42692/0 (socket says 172.21.15.67:42692) 2024-05-30T16:48:23.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba3400640 1 -- 172.21.15.67:0/2432858513 learned_addr learned my addr 172.21.15.67:0/2432858513 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:23.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba3400640 1 -- 172.21.15.67:0/2432858513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba40d14d0 msgr2=0x7feba40cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:23.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba3400640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba40d14d0 0x7feba40cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba3400640 1 -- 172.21.15.67:0/2432858513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feba415e870 msgr2=0x7feba41599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:23.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba3400640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feba415e870 0x7feba41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.168+0000 7feba3400640 1 -- 172.21.15.67:0/2432858513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba415fe50 con 0x7feba40cf870 2024-05-30T16:48:23.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.169+0000 7feba3400640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40cf870 0x7feba415f840 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7feb94007f20 tx=0x7feb9400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:23.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.169+0000 7feb9b400640 1 -- 172.21.15.67:0/2432858513 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94014620 con 0x7feba40cf870 2024-05-30T16:48:23.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.169+0000 7feb9b400640 1 -- 172.21.15.67:0/2432858513 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94019070 con 0x7feba40cf870 2024-05-30T16:48:23.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.169+0000 7feb9b400640 1 -- 172.21.15.67:0/2432858513 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94014a00 con 0x7feba40cf870 2024-05-30T16:48:23.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.169+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba4160130 con 0x7feba40cf870 2024-05-30T16:48:23.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.169+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba41606f0 con 0x7feba40cf870 2024-05-30T16:48:23.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.170+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba40cdb80 con 0x7feba40cf870 2024-05-30T16:48:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.171+0000 7feb9b400640 1 -- 172.21.15.67:0/2432858513 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7feb94014ba0 con 0x7feba40cf870 2024-05-30T16:48:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.172+0000 7feb9b400640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8406cbc0 0x7feb8406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.172+0000 7feb9b400640 1 -- 172.21.15.67:0/2432858513 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7feb9408c5b0 con 0x7feba40cf870 2024-05-30T16:48:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.172+0000 7feba2a00640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8406cbc0 0x7feb8406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.173+0000 7feba2a00640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8406cbc0 0x7feb8406f080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb90033000 tx=0x7feb9002c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:23.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.176+0000 7feb9b400640 1 -- 172.21.15.67:0/2432858513 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feb9405a950 con 0x7feba40cf870 2024-05-30T16:48:23.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.379+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feba40689e0 con 0x7feba40cf870 2024-05-30T16:48:23.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.380+0000 7feb9b400640 1 -- 172.21.15.67:0/2432858513 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7feb9405a250 con 0x7feba40cf870 2024-05-30T16:48:23.381 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-05-30T16:48:23.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8406cbc0 msgr2=0x7feb8406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:23.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8406cbc0 0x7feb8406f080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb90033000 tx=0x7feb9002c000 comp rx=0 tx=0).stop 2024-05-30T16:48:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40cf870 msgr2=0x7feba415f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40cf870 0x7feba415f840 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7feb94007f20 tx=0x7feb9400e570 comp rx=0 tx=0).stop 2024-05-30T16:48:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 shutdown_connections 2024-05-30T16:48:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8406cbc0 0x7feb8406f080 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feba415e870 0x7feba41599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba40d14d0 0x7feba40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 --2- 172.21.15.67:0/2432858513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba40cf870 0x7feba415f840 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 >> 172.21.15.67:0/2432858513 conn(0x7feba408bfc0 msgr2=0x7feba40b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:23.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 shutdown_connections 2024-05-30T16:48:23.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.383+0000 7feba9000640 1 -- 172.21.15.67:0/2432858513 wait complete. 2024-05-30T16:48:23.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones' 2024-05-30T16:48:23.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.614+0000 7f1b30a00640 1 Processor -- start 2024-05-30T16:48:23.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.614+0000 7f1b30a00640 1 -- start start 2024-05-30T16:48:23.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.616+0000 7f1b30a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 0x7f1b2c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.616+0000 7f1b30a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1b2c0b3690 0x7f1b2c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.616+0000 7f1b30a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b2c06ed10 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.616+0000 7f1b30a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2c0d0e50 con 0x7f1b2c0b3690 2024-05-30T16:48:23.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.616+0000 7f1b30a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c15cb00 con 0x7f1b2c06f860 2024-05-30T16:48:23.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.616+0000 7f1b21400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1b2c0b3690 0x7f1b2c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.616+0000 7f1b21400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1b2c0b3690 0x7f1b2c0d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42696/0 (socket says 172.21.15.67:42696) 2024-05-30T16:48:23.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.616+0000 7f1b21400640 1 -- 172.21.15.67:0/1451298736 learned_addr learned my addr 172.21.15.67:0/1451298736 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.616+0000 7f1b29e00640 1 --2- 172.21.15.67:0/1451298736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 0x7f1b2c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.617+0000 7f1b21400640 1 -- 172.21.15.67:0/1451298736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 msgr2=0x7f1b2c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.617+0000 7f1b21400640 1 --2- 172.21.15.67:0/1451298736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 0x7f1b2c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.617+0000 7f1b21400640 1 -- 172.21.15.67:0/1451298736 >> v1:172.21.15.67:6789/0 conn(0x7f1b2c0d09e0 legacy=0x7f1b2c160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.617+0000 7f1b21400640 1 -- 172.21.15.67:0/1451298736 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c0b3cc0 con 0x7f1b2c0b3690 2024-05-30T16:48:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.617+0000 7f1b29400640 1 -- 172.21.15.67:0/1451298736 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560957825 0 0) 0x7f1b24001110 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.618+0000 7f1b21400640 1 --2- 172.21.15.67:0/1451298736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0b3690 0x7f1b2c0d04a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009870 tx=0x7f1b1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b101c0f3904123a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.618+0000 7f1b29400640 1 -- 172.21.15.67:0/1451298736 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14037070 con 0x7f1b2c0b3690 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.618+0000 7f1b29400640 1 -- 172.21.15.67:0/1451298736 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b14032440 con 0x7f1b2c0b3690 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.618+0000 7f1b29400640 1 -- 172.21.15.67:0/1451298736 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14032810 con 0x7f1b2c0b3690 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.619+0000 7f1b30a00640 1 -- 172.21.15.67:0/1451298736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0b3690 msgr2=0x7f1b2c0d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.619+0000 7f1b30a00640 1 --2- 172.21.15.67:0/1451298736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0b3690 0x7f1b2c0d04a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009870 tx=0x7f1b1402a270 comp rx=0 tx=0).stop 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.619+0000 7f1b30a00640 1 -- 172.21.15.67:0/1451298736 shutdown_connections 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.619+0000 7f1b30a00640 1 --2- 172.21.15.67:0/1451298736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0b3690 0x7f1b2c0d04a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.619+0000 7f1b30a00640 1 --2- 172.21.15.67:0/1451298736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 0x7f1b2c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.619+0000 7f1b30a00640 1 -- 172.21.15.67:0/1451298736 >> 172.21.15.67:0/1451298736 conn(0x7f1b2c06bae0 msgr2=0x7f1b2c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.619+0000 7f1b30a00640 1 -- 172.21.15.67:0/1451298736 shutdown_connections 2024-05-30T16:48:23.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.619+0000 7f1b30a00640 1 -- 172.21.15.67:0/1451298736 wait complete. 2024-05-30T16:48:23.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.620+0000 7f1b30a00640 1 Processor -- start 2024-05-30T16:48:23.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.621+0000 7f1b30a00640 1 -- start start 2024-05-30T16:48:23.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b30a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 0x7f1b2c0c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b30a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0b3690 0x7f1b2c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b30a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0d09e0 0x7f1b2c15d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b30a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c0b4560 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b30a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c0b4280 con 0x7f1b2c0b3690 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b30a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c0b4840 con 0x7f1b2c06f860 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b2a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0d09e0 0x7f1b2c15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b2a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0d09e0 0x7f1b2c15d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42698/0 (socket says 172.21.15.67:42698) 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b2a800640 1 -- 172.21.15.67:0/1674504192 learned_addr learned my addr 172.21.15.67:0/1674504192 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b29e00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 0x7f1b2c0c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b21400640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0b3690 0x7f1b2c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b2a800640 1 -- 172.21.15.67:0/1674504192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 msgr2=0x7f1b2c0c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b2a800640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 0x7f1b2c0c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b2a800640 1 -- 172.21.15.67:0/1674504192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0b3690 msgr2=0x7f1b2c15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b2a800640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0b3690 0x7f1b2c15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.622+0000 7f1b2a800640 1 -- 172.21.15.67:0/1674504192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c0ce3a0 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.623+0000 7f1b21400640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0b3690 0x7f1b2c15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.623+0000 7f1b2a800640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0d09e0 0x7f1b2c15d250 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c3e0 tx=0x7f1b2400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:23.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.623+0000 7f1b22a00640 1 -- 172.21.15.67:0/1674504192 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24017070 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.623+0000 7f1b22a00640 1 -- 172.21.15.67:0/1674504192 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b240035f0 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.623+0000 7f1b22a00640 1 -- 172.21.15.67:0/1674504192 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24003a20 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.623+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c0ce680 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.623+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c0cef20 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.624+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c0689e0 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.625+0000 7f1b22a00640 1 -- 172.21.15.67:0/1674504192 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f1b2401a070 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.626+0000 7f1b22a00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1b0806cbf0 0x7f1b0806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:23.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.626+0000 7f1b22a00640 1 -- 172.21.15.67:0/1674504192 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f1b2408b070 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.627+0000 7f1b29e00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1b0806cbf0 0x7f1b0806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:23.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.627+0000 7f1b29e00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1b0806cbf0 0x7f1b0806f0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0099c0 tx=0x7f1b1c003990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.631+0000 7f1b22a00640 1 -- 172.21.15.67:0/1674504192 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b2c06ed10 con 0x7f1b2c0d09e0 2024-05-30T16:48:23.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:23.912+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]} v 0) -- 0x7f1b2c0cea10 con 0x7f1b2c0d09e0 2024-05-30T16:48:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.471+0000 7f1b22a00640 1 -- 172.21.15.67:0/1674504192 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]}]=0 v199) ==== 165+0+0 (secure 0 0 0) 0x7f1b240594a0 con 0x7f1b2c0d09e0 2024-05-30T16:48:24.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1b0806cbf0 msgr2=0x7f1b0806f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:24.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1b0806cbf0 0x7f1b0806f0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0099c0 tx=0x7f1b1c003990 comp rx=0 tx=0).stop 2024-05-30T16:48:24.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0d09e0 msgr2=0x7f1b2c15d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:24.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0d09e0 0x7f1b2c15d250 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c3e0 tx=0x7f1b2400c8a0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 shutdown_connections 2024-05-30T16:48:24.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1b0806cbf0 0x7f1b0806f0b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b2c0d09e0 0x7f1b2c15d250 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b2c0b3690 0x7f1b2c15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 --2- 172.21.15.67:0/1674504192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b2c06f860 0x7f1b2c0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 >> 172.21.15.67:0/1674504192 conn(0x7f1b2c06bae0 msgr2=0x7f1b2c0730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.474+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 shutdown_connections 2024-05-30T16:48:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.475+0000 7f1b30a00640 1 -- 172.21.15.67:0/1674504192 wait complete. 2024-05-30T16:48:24.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:48:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.719+0000 7ff496e00640 1 Processor -- start 2024-05-30T16:48:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.719+0000 7ff496e00640 1 -- start start 2024-05-30T16:48:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.721+0000 7ff496e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff490161e70 0x7ff4900b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.721+0000 7ff496e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b3800 0x7ff4900ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.721+0000 7ff496e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900b3d40 con 0x7ff490161a90 2024-05-30T16:48:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.721+0000 7ff496e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff4900b4020 con 0x7ff490161e70 2024-05-30T16:48:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.721+0000 7ff496e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900b4300 con 0x7ff4900b3800 2024-05-30T16:48:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.722+0000 7ff487400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff490161e70 0x7ff4900b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.722+0000 7ff494e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b3800 0x7ff4900ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.722+0000 7ff487400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff490161e70 0x7ff4900b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36156/0 (socket says 172.21.15.67:36156) 2024-05-30T16:48:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.722+0000 7ff487400640 1 -- 172.21.15.67:0/3109140931 learned_addr learned my addr 172.21.15.67:0/3109140931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.723+0000 7ff487400640 1 -- 172.21.15.67:0/3109140931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b3800 msgr2=0x7ff4900ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.723+0000 7ff487400640 1 --2- 172.21.15.67:0/3109140931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b3800 0x7ff4900ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.723+0000 7ff487400640 1 -- 172.21.15.67:0/3109140931 >> v1:172.21.15.169:6789/0 conn(0x7ff490161a90 legacy=0x7ff49006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:24.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.723+0000 7ff487400640 1 -- 172.21.15.67:0/3109140931 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49006dc70 con 0x7ff490161e70 2024-05-30T16:48:24.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.724+0000 7ff487400640 1 --2- 172.21.15.67:0/3109140931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490161e70 0x7ff4900b32c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff478009870 tx=0x7ff47802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5593e209696b91a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.724+0000 7ff48f400640 1 -- 172.21.15.67:0/3109140931 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478037070 con 0x7ff490161e70 2024-05-30T16:48:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.725+0000 7ff48f400640 1 -- 172.21.15.67:0/3109140931 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff478032440 con 0x7ff490161e70 2024-05-30T16:48:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.725+0000 7ff48f400640 1 -- 172.21.15.67:0/3109140931 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478032810 con 0x7ff490161e70 2024-05-30T16:48:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.725+0000 7ff496e00640 1 -- 172.21.15.67:0/3109140931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490161e70 msgr2=0x7ff4900b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.725+0000 7ff496e00640 1 --2- 172.21.15.67:0/3109140931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490161e70 0x7ff4900b32c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff478009870 tx=0x7ff47802a270 comp rx=0 tx=0).stop 2024-05-30T16:48:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.725+0000 7ff496e00640 1 -- 172.21.15.67:0/3109140931 shutdown_connections 2024-05-30T16:48:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.725+0000 7ff496e00640 1 --2- 172.21.15.67:0/3109140931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b3800 0x7ff4900ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.725+0000 7ff496e00640 1 --2- 172.21.15.67:0/3109140931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490161e70 0x7ff4900b32c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.725+0000 7ff496e00640 1 -- 172.21.15.67:0/3109140931 >> 172.21.15.67:0/3109140931 conn(0x7ff49006bd20 msgr2=0x7ff49006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.726+0000 7ff496e00640 1 -- 172.21.15.67:0/3109140931 shutdown_connections 2024-05-30T16:48:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.726+0000 7ff496e00640 1 -- 172.21.15.67:0/3109140931 wait complete. 2024-05-30T16:48:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.728+0000 7ff496e00640 1 Processor -- start 2024-05-30T16:48:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.728+0000 7ff496e00640 1 -- start start 2024-05-30T16:48:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.729+0000 7ff496e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4900b3800 0x7ff49014b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.729+0000 7ff496e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490161a90 0x7ff4900b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.729+0000 7ff496e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff490161e70 0x7ff49015a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.729+0000 7ff496e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff49006eb80 con 0x7ff4900b3800 2024-05-30T16:48:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.729+0000 7ff496e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900b5020 con 0x7ff490161a90 2024-05-30T16:48:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.729+0000 7ff496e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900ad7e0 con 0x7ff490161e70 2024-05-30T16:48:24.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4900b3800 0x7ff49014b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4900b3800 0x7ff49014b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42706/0 (socket says 172.21.15.67:42706) 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff48fe00640 1 -- 172.21.15.67:0/3449636197 learned_addr learned my addr 172.21.15.67:0/3449636197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff487400640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490161a90 0x7ff4900b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff494e00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff490161e70 0x7ff49015a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff48fe00640 1 -- 172.21.15.67:0/3449636197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff490161e70 msgr2=0x7ff49015a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff48fe00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff490161e70 0x7ff49015a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff48fe00640 1 -- 172.21.15.67:0/3449636197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490161a90 msgr2=0x7ff4900b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff48fe00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490161a90 0x7ff4900b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.730+0000 7ff48fe00640 1 -- 172.21.15.67:0/3449636197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4901592c0 con 0x7ff4900b3800 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.731+0000 7ff48fe00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4900b3800 0x7ff49014b6a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c7f0 tx=0x7ff48000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.731+0000 7ff48cc00640 1 -- 172.21.15.67:0/3449636197 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480003b70 con 0x7ff4900b3800 2024-05-30T16:48:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.731+0000 7ff48cc00640 1 -- 172.21.15.67:0/3449636197 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff480003d10 con 0x7ff4900b3800 2024-05-30T16:48:24.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.731+0000 7ff48cc00640 1 -- 172.21.15.67:0/3449636197 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48000aa90 con 0x7ff4900b3800 2024-05-30T16:48:24.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.731+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4901595a0 con 0x7ff4900b3800 2024-05-30T16:48:24.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.731+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff490156340 con 0x7ff4900b3800 2024-05-30T16:48:24.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.732+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4900689e0 con 0x7ff4900b3800 2024-05-30T16:48:24.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.733+0000 7ff48cc00640 1 -- 172.21.15.67:0/3449636197 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7ff48000ac30 con 0x7ff4900b3800 2024-05-30T16:48:24.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.734+0000 7ff48cc00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff46c06cbc0 0x7ff46c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:24.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.734+0000 7ff48cc00640 1 -- 172.21.15.67:0/3449636197 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7ff48008bd40 con 0x7ff4900b3800 2024-05-30T16:48:24.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.735+0000 7ff487400640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff46c06cbc0 0x7ff46c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:24.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.735+0000 7ff487400640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff46c06cbc0 0x7ff46c06f080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff478034000 tx=0x7ff47802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:24.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.739+0000 7ff48cc00640 1 -- 172.21.15.67:0/3449636197 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff48005a0e0 con 0x7ff4900b3800 2024-05-30T16:48:24.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.947+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff490075000 con 0x7ff4900b3800 2024-05-30T16:48:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.954+0000 7ff48cc00640 1 -- 172.21.15.67:0/3449636197 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7ff4800599e0 con 0x7ff4900b3800 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.956+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff46c06cbc0 msgr2=0x7ff46c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.956+0000 7ff496e00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff46c06cbc0 0x7ff46c06f080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff478034000 tx=0x7ff47802e000 comp rx=0 tx=0).stop 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4900b3800 msgr2=0x7ff49014b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4900b3800 0x7ff49014b6a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c7f0 tx=0x7ff48000ccb0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 shutdown_connections 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff46c06cbc0 0x7ff46c06f080 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff490161e70 0x7ff49015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490161a90 0x7ff4900b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 --2- 172.21.15.67:0/3449636197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4900b3800 0x7ff49014b6a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 >> 172.21.15.67:0/3449636197 conn(0x7ff49006bd20 msgr2=0x7ff490072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 shutdown_connections 2024-05-30T16:48:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:24.957+0000 7ff496e00640 1 -- 172.21.15.67:0/3449636197 wait complete. 2024-05-30T16:48:24.978 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T16:48:24.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:48:25.077 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:48:25.193 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T16:48:25.194 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:48:25.194 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:25.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:48:25.253 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:48:25.253 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:25.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:48:25.310 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:48:25.310 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:25.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:48:25.370 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:48:25.370 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:25.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:48:25.428 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-30T16:48:25.428 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:48:25.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.641+0000 7f4ca0a00640 1 Processor -- start 2024-05-30T16:48:25.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.641+0000 7f4ca0a00640 1 -- start start 2024-05-30T16:48:25.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.642+0000 7f4ca0a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c9c06c010 0x7f4c9c145810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:25.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.642+0000 7f4ca0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c9c145d50 0x7f4c9c15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:25.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.642+0000 7f4ca0a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c9c146200 con 0x7f4c9c145430 2024-05-30T16:48:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.642+0000 7f4ca0a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4c9c0b4320 con 0x7f4c9c145d50 2024-05-30T16:48:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.642+0000 7f4ca0a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4c9c0b4600 con 0x7f4c9c06c010 2024-05-30T16:48:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.643+0000 7f4c9a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c9c145d50 0x7f4c9c15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.643+0000 7f4c9a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c9c145d50 0x7f4c9c15cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42716/0 (socket says 172.21.15.67:42716) 2024-05-30T16:48:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.643+0000 7f4c9a800640 1 -- 172.21.15.67:0/3430398438 learned_addr learned my addr 172.21.15.67:0/3430398438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.643+0000 7f4c99e00640 1 --1- 172.21.15.67:0/3430398438 >> v1:172.21.15.67:6789/0 conn(0x7f4c9c145430 0x7f4c9c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45938/0 (socket says 172.21.15.67:45938) 2024-05-30T16:48:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.643+0000 7f4c9a800640 1 -- 172.21.15.67:0/3430398438 >> v2:172.21.15.169:3300/0 conn(0x7f4c9c06c010 msgr2=0x7f4c9c145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.643+0000 7f4c9a800640 1 --2- 172.21.15.67:0/3430398438 >> v2:172.21.15.169:3300/0 conn(0x7f4c9c06c010 0x7f4c9c145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.643+0000 7f4c9a800640 1 -- 172.21.15.67:0/3430398438 >> v1:172.21.15.67:6789/0 conn(0x7f4c9c145430 legacy=0x7f4c9c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.643+0000 7f4c9a800640 1 -- 172.21.15.67:0/3430398438 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c0b48e0 con 0x7f4c9c145d50 2024-05-30T16:48:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.644+0000 7f4c9a800640 1 --2- 172.21.15.67:0/3430398438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c145d50 0x7f4c9c15cc90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c06c600 tx=0x7f4c9402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e801a295e6c3407 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.645+0000 7f4c98a00640 1 -- 172.21.15.67:0/3430398438 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94036070 con 0x7f4c9c145d50 2024-05-30T16:48:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.645+0000 7f4c98a00640 1 -- 172.21.15.67:0/3430398438 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c94031600 con 0x7f4c9c145d50 2024-05-30T16:48:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.645+0000 7f4c98a00640 1 -- 172.21.15.67:0/3430398438 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c940319d0 con 0x7f4c9c145d50 2024-05-30T16:48:25.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.645+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3430398438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c145d50 msgr2=0x7f4c9c15cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:25.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.645+0000 7f4ca0a00640 1 --2- 172.21.15.67:0/3430398438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c145d50 0x7f4c9c15cc90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c06c600 tx=0x7f4c9402a730 comp rx=0 tx=0).stop 2024-05-30T16:48:25.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.645+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3430398438 shutdown_connections 2024-05-30T16:48:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.645+0000 7f4ca0a00640 1 --2- 172.21.15.67:0/3430398438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c145d50 0x7f4c9c15cc90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.645+0000 7f4ca0a00640 1 --2- 172.21.15.67:0/3430398438 >> v2:172.21.15.169:3300/0 conn(0x7f4c9c06c010 0x7f4c9c145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.645+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3430398438 >> 172.21.15.67:0/3430398438 conn(0x7f4c9c09aa30 msgr2=0x7f4c9c0b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.646+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3430398438 shutdown_connections 2024-05-30T16:48:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.646+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3430398438 wait complete. 2024-05-30T16:48:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.647+0000 7f4ca0a00640 1 Processor -- start 2024-05-30T16:48:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.647+0000 7f4ca0a00640 1 -- start start 2024-05-30T16:48:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.648+0000 7f4ca0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c06c010 0x7f4c9c136810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.648+0000 7f4ca0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c9c145430 0x7f4c9c0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.648+0000 7f4ca0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c9c145d50 0x7f4c9c1f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.648+0000 7f4ca0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c0ae520 con 0x7f4c9c06c010 2024-05-30T16:48:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.648+0000 7f4ca0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c0b4ea0 con 0x7f4c9c145d50 2024-05-30T16:48:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.648+0000 7f4ca0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9c0b53a0 con 0x7f4c9c145430 2024-05-30T16:48:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c06c010 0x7f4c9c136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c06c010 0x7f4c9c136810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42720/0 (socket says 172.21.15.67:42720) 2024-05-30T16:48:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99e00640 1 -- 172.21.15.67:0/3678835900 learned_addr learned my addr 172.21.15.67:0/3678835900 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99e00640 1 -- 172.21.15.67:0/3678835900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c9c145430 msgr2=0x7f4c9c0ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99400640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c9c145430 0x7f4c9c0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99e00640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c9c145430 0x7f4c9c0ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99e00640 1 -- 172.21.15.67:0/3678835900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c9c145d50 msgr2=0x7f4c9c1f8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99e00640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c9c145d50 0x7f4c9c1f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99e00640 1 -- 172.21.15.67:0/3678835900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c159040 con 0x7f4c9c06c010 2024-05-30T16:48:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.649+0000 7f4c99e00640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c06c010 0x7f4c9c136810 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00c400 tx=0x7f4c8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.650+0000 7f4c92000640 1 -- 172.21.15.67:0/3678835900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0074e0 con 0x7f4c9c06c010 2024-05-30T16:48:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.650+0000 7f4c92000640 1 -- 172.21.15.67:0/3678835900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8c007680 con 0x7f4c9c06c010 2024-05-30T16:48:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.650+0000 7f4c92000640 1 -- 172.21.15.67:0/3678835900 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c004a00 con 0x7f4c9c06c010 2024-05-30T16:48:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.650+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9c136df0 con 0x7f4c9c06c010 2024-05-30T16:48:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.650+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9c1373b0 con 0x7f4c9c06c010 2024-05-30T16:48:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.651+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9c06be00 con 0x7f4c9c06c010 2024-05-30T16:48:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.652+0000 7f4c92000640 1 -- 172.21.15.67:0/3678835900 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f4c8c01a020 con 0x7f4c9c06c010 2024-05-30T16:48:25.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.653+0000 7f4c92000640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c7c06cbf0 0x7f4c7c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:25.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.653+0000 7f4c92000640 1 -- 172.21.15.67:0/3678835900 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f4c8c08b5c0 con 0x7f4c9c06c010 2024-05-30T16:48:25.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.653+0000 7f4c99400640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c7c06cbf0 0x7f4c7c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.654+0000 7f4c99400640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c7c06cbf0 0x7f4c7c06f0b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c141e00 tx=0x7f4c88008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:25.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.658+0000 7f4c92000640 1 -- 172.21.15.67:0/3678835900 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c9c06be00 con 0x7f4c9c06c010 2024-05-30T16:48:25.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.905+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c9c0681e0 con 0x7f4c9c06c010 2024-05-30T16:48:25.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.906+0000 7f4c92000640 1 -- 172.21.15.67:0/3678835900 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3869 (secure 0 0 0) 0x7f4c9c0681e0 con 0x7f4c9c06c010 2024-05-30T16:48:25.907 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:25.907 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/475904074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":475904074},{"type":"v1","addr":"172.21.15.169:6834","nonce":475904074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4812,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3464775083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3464775083},{"type":"v1","addr":"172.21.15.67:6835","nonce":3464775083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/650843675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":650843675},{"type":"v1","addr":"172.21.15.67:6837","nonce":650843675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4933,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3193538429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":3193538429},{"type":"v1","addr":"172.21.15.169:6835","nonce":3193538429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-05-30T16:48:25.907 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 19 2024-05-30T16:48:25.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c7c06cbf0 msgr2=0x7f4c7c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c7c06cbf0 0x7f4c7c06f0b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c141e00 tx=0x7f4c88008040 comp rx=0 tx=0).stop 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c06c010 msgr2=0x7f4c9c136810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c06c010 0x7f4c9c136810 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00c400 tx=0x7f4c8c00c8c0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 shutdown_connections 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c7c06cbf0 0x7f4c7c06f0b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c9c145d50 0x7f4c9c1f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c9c145430 0x7f4c9c0ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 --2- 172.21.15.67:0/3678835900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9c06c010 0x7f4c9c136810 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 >> 172.21.15.67:0/3678835900 conn(0x7f4c9c09aa30 msgr2=0x7f4c9c0b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 shutdown_connections 2024-05-30T16:48:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:25.909+0000 7f4ca0a00640 1 -- 172.21.15.67:0/3678835900 wait complete. 2024-05-30T16:48:25.931 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-30T16:48:25.931 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-30T16:48:25.932 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-30T16:48:25.932 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:48:25.932 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-30T16:48:25.932 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-30T16:48:25.932 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-30T16:48:25.932 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:48:25.932 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-30T16:48:25.932 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-30T16:48:25.933 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-30T16:48:25.933 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:48:25.933 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-30T16:48:25.933 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-30T16:48:25.933 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-30T16:48:25.933 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:48:25.951 INFO:tasks.ceph.mds.a:Stopped 2024-05-30T16:48:25.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-30T16:48:25.952 INFO:tasks.ceph.mds.c:Stopped 2024-05-30T16:48:25.952 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-30T16:48:25.966 INFO:tasks.ceph.mds.d:Stopped 2024-05-30T16:48:25.966 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-30T16:48:25.967 INFO:tasks.ceph.mds.b:Stopped 2024-05-30T16:48:25.967 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-30T16:48:26.000 INFO:tasks.ceph.mds.a:Started 2024-05-30T16:48:26.001 DEBUG:teuthology.parallel:result is None 2024-05-30T16:48:26.006 INFO:tasks.ceph.mds.c:Started 2024-05-30T16:48:26.006 DEBUG:teuthology.parallel:result is None 2024-05-30T16:48:26.011 INFO:tasks.ceph.mds.d:Started 2024-05-30T16:48:26.011 DEBUG:teuthology.parallel:result is None 2024-05-30T16:48:26.016 INFO:tasks.ceph.mds.b:Started 2024-05-30T16:48:26.017 DEBUG:teuthology.parallel:result is None 2024-05-30T16:48:26.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-30T16:48:26.154 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T16:48:26.157 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T16:48:26.166 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T16:48:26.166 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T16:48:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.201+0000 7faad2c00640 1 Processor -- start 2024-05-30T16:48:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.201+0000 7faad2c00640 1 -- start start 2024-05-30T16:48:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.202+0000 7faad2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc0707d0 0x7faacc06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.202+0000 7faad2c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faacc15fbc0 con 0x7faacc06c310 2024-05-30T16:48:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.202+0000 7faad2c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faacc070d50 con 0x7faacc0b01e0 2024-05-30T16:48:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.202+0000 7faad2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faacc0b6140 con 0x7faacc0707d0 2024-05-30T16:48:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.203+0000 7faad0c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faacc06c310 0x7faacc0d1470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45956/0 (socket says 172.21.15.67:45956) 2024-05-30T16:48:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.203+0000 7faad0c00640 1 -- 172.21.15.67:0/1779531543 learned_addr learned my addr 172.21.15.67:0/1779531543 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.203+0000 7faacb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc0707d0 0x7faacc06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.203+0000 7faacbe00640 1 --1- 172.21.15.67:0/1779531543 >> v1:172.21.15.169:6789/0 conn(0x7faacc0b01e0 0x7faacc0b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:60392/0 (socket says 172.21.15.67:60392) 2024-05-30T16:48:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.203+0000 7faacb400640 1 -- 172.21.15.67:0/1779531543 >> v1:172.21.15.169:6789/0 conn(0x7faacc0b01e0 legacy=0x7faacc0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.203+0000 7faacb400640 1 -- 172.21.15.67:0/1779531543 >> v1:172.21.15.67:6789/0 conn(0x7faacc06c310 legacy=0x7faacc0d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.203+0000 7faacaa00640 1 -- 172.21.15.67:0/1779531543 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3901649996 0 0) 0x7faacc15fbc0 con 0x7faacc06c310 2024-05-30T16:48:26.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.203+0000 7faacb400640 1 -- 172.21.15.67:0/1779531543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc0b6420 con 0x7faacc0707d0 2024-05-30T16:48:26.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.204+0000 7faacb400640 1 --2- 172.21.15.67:0/1779531543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc0707d0 0x7faacc06ee90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faab80097d0 tx=0x7faab802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6fafc0dd3a4b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:26.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.204+0000 7faacaa00640 1 -- 172.21.15.67:0/1779531543 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8037070 con 0x7faacc0707d0 2024-05-30T16:48:26.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.205+0000 7faacaa00640 1 -- 172.21.15.67:0/1779531543 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab8032440 con 0x7faacc0707d0 2024-05-30T16:48:26.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.205+0000 7faacaa00640 1 -- 172.21.15.67:0/1779531543 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8032810 con 0x7faacc0707d0 2024-05-30T16:48:26.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.205+0000 7faad2c00640 1 -- 172.21.15.67:0/1779531543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc0707d0 msgr2=0x7faacc06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.205+0000 7faad2c00640 1 --2- 172.21.15.67:0/1779531543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc0707d0 0x7faacc06ee90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faab80097d0 tx=0x7faab802a210 comp rx=0 tx=0).stop 2024-05-30T16:48:26.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.205+0000 7faad2c00640 1 -- 172.21.15.67:0/1779531543 shutdown_connections 2024-05-30T16:48:26.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.205+0000 7faad2c00640 1 --2- 172.21.15.67:0/1779531543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc0707d0 0x7faacc06ee90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.205+0000 7faad2c00640 1 -- 172.21.15.67:0/1779531543 >> 172.21.15.67:0/1779531543 conn(0x7faacc09aa30 msgr2=0x7faacc0b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:26.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.205+0000 7faad2c00640 1 -- 172.21.15.67:0/1779531543 shutdown_connections 2024-05-30T16:48:26.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.205+0000 7faad2c00640 1 -- 172.21.15.67:0/1779531543 wait complete. 2024-05-30T16:48:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.207+0000 7faad2c00640 1 Processor -- start 2024-05-30T16:48:26.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.207+0000 7faad2c00640 1 -- start start 2024-05-30T16:48:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faad2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc06c310 0x7faacc143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faad2c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faacc0707d0 0x7faacc156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faad2c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc0b01e0 0x7faacc153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faad2c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faacc0d1b80 con 0x7faacc0b01e0 2024-05-30T16:48:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faad2c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faacc0b5df0 con 0x7faacc0707d0 2024-05-30T16:48:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faad2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faacc0d1d00 con 0x7faacc06c310 2024-05-30T16:48:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faad0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc0b01e0 0x7faacc153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faad0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc0b01e0 0x7faacc153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42772/0 (socket says 172.21.15.67:42772) 2024-05-30T16:48:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faad0c00640 1 -- 172.21.15.67:0/2323299079 learned_addr learned my addr 172.21.15.67:0/2323299079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faacbe00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc06c310 0x7faacc143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.208+0000 7faacb400640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faacc0707d0 0x7faacc156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faad0c00640 1 -- 172.21.15.67:0/2323299079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc06c310 msgr2=0x7faacc143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faad0c00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc06c310 0x7faacc143c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faad0c00640 1 -- 172.21.15.67:0/2323299079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faacc0707d0 msgr2=0x7faacc156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faad0c00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faacc0707d0 0x7faacc156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faad0c00640 1 -- 172.21.15.67:0/2323299079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc156ae0 con 0x7faacc0b01e0 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faad0c00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc0b01e0 0x7faacc153b80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faac400c730 tx=0x7faac400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faac3e00640 1 -- 172.21.15.67:0/2323299079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4018070 con 0x7faacc0b01e0 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faac3e00640 1 -- 172.21.15.67:0/2323299079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac400ce40 con 0x7faacc0b01e0 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faacc15ffe0 con 0x7faacc0b01e0 2024-05-30T16:48:26.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.209+0000 7faac3e00640 1 -- 172.21.15.67:0/2323299079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac400a860 con 0x7faacc0b01e0 2024-05-30T16:48:26.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.210+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faacc1567d0 con 0x7faacc0b01e0 2024-05-30T16:48:26.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.211+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faacc0689e0 con 0x7faacc0b01e0 2024-05-30T16:48:26.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.212+0000 7faac3e00640 1 -- 172.21.15.67:0/2323299079 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7faac401b070 con 0x7faacc0b01e0 2024-05-30T16:48:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.212+0000 7faac3e00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faaac06cbc0 0x7faaac06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.213+0000 7faac3e00640 1 -- 172.21.15.67:0/2323299079 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7faac408b2a0 con 0x7faacc0b01e0 2024-05-30T16:48:26.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.213+0000 7faacbe00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faaac06cbc0 0x7faaac06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.213+0000 7faacbe00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faaac06cbc0 0x7faaac06f080 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faabc000f60 tx=0x7faabc000ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:26.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.217+0000 7faac3e00640 1 -- 172.21.15.67:0/2323299079 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faac4058ff0 con 0x7faacc0b01e0 2024-05-30T16:48:26.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.434+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7faacc06bdc0 con 0x7faacc0b01e0 2024-05-30T16:48:26.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.497+0000 7faac3e00640 1 -- 172.21.15.67:0/2323299079 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) ==== 97+0+0 (secure 0 0 0) 0x7faac405d2f0 con 0x7faacc0b01e0 2024-05-30T16:48:26.498 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-05-30T16:48:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.500+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faaac06cbc0 msgr2=0x7faaac06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.500+0000 7faad2c00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faaac06cbc0 0x7faaac06f080 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faabc000f60 tx=0x7faabc000ed0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.500+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc0b01e0 msgr2=0x7faacc153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.500+0000 7faad2c00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc0b01e0 0x7faacc153b80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faac400c730 tx=0x7faac400cbf0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.501+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 shutdown_connections 2024-05-30T16:48:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.501+0000 7faad2c00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faaac06cbc0 0x7faaac06f080 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.501+0000 7faad2c00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faacc0b01e0 0x7faacc153b80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.501+0000 7faad2c00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faacc0707d0 0x7faacc156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.501+0000 7faad2c00640 1 --2- 172.21.15.67:0/2323299079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faacc06c310 0x7faacc143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.501+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 >> 172.21.15.67:0/2323299079 conn(0x7faacc09aa30 msgr2=0x7faacc074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.501+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 shutdown_connections 2024-05-30T16:48:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.501+0000 7faad2c00640 1 -- 172.21.15.67:0/2323299079 wait complete. 2024-05-30T16:48:26.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-30T16:48:26.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.719+0000 7ff273200640 1 Processor -- start 2024-05-30T16:48:26.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.719+0000 7ff273200640 1 -- start start 2024-05-30T16:48:26.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.721+0000 7ff273200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27415b3a0 0x7ff2740b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.721+0000 7ff273200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff27406dc20 0x7ff27415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.721+0000 7ff273200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff274072c00 0x7ff274073050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.721+0000 7ff273200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff27406f980 con 0x7ff274072c00 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.721+0000 7ff273200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff274073590 con 0x7ff27406dc20 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.721+0000 7ff273200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff2740ae0c0 con 0x7ff27415b3a0 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.721+0000 7ff272800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff274072c00 0x7ff274073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.721+0000 7ff272800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff274072c00 0x7ff274073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42782/0 (socket says 172.21.15.67:42782) 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.721+0000 7ff272800640 1 -- 172.21.15.67:0/411609562 learned_addr learned my addr 172.21.15.67:0/411609562 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.722+0000 7ff271e00640 1 --2- 172.21.15.67:0/411609562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27415b3a0 0x7ff2740b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.722+0000 7ff26be00640 1 --2- 172.21.15.67:0/411609562 >> v2:172.21.15.169:3300/0 conn(0x7ff27406dc20 0x7ff27415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.722+0000 7ff272800640 1 -- 172.21.15.67:0/411609562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27415b3a0 msgr2=0x7ff2740b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.722+0000 7ff272800640 1 --2- 172.21.15.67:0/411609562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27415b3a0 0x7ff2740b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.722+0000 7ff272800640 1 -- 172.21.15.67:0/411609562 >> v2:172.21.15.169:3300/0 conn(0x7ff27406dc20 msgr2=0x7ff27415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:26.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.722+0000 7ff272800640 1 --2- 172.21.15.67:0/411609562 >> v2:172.21.15.169:3300/0 conn(0x7ff27406dc20 0x7ff27415f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.722+0000 7ff272800640 1 -- 172.21.15.67:0/411609562 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2740afd60 con 0x7ff274072c00 2024-05-30T16:48:26.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.723+0000 7ff272800640 1 --2- 172.21.15.67:0/411609562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff274072c00 0x7ff274073050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff27406c860 tx=0x7ff26c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c7509b9fde8f11 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:26.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.723+0000 7ff271400640 1 -- 172.21.15.67:0/411609562 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c036070 con 0x7ff274072c00 2024-05-30T16:48:26.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.723+0000 7ff271400640 1 -- 172.21.15.67:0/411609562 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26c031600 con 0x7ff274072c00 2024-05-30T16:48:26.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.723+0000 7ff271400640 1 -- 172.21.15.67:0/411609562 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0319d0 con 0x7ff274072c00 2024-05-30T16:48:26.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.724+0000 7ff273200640 1 -- 172.21.15.67:0/411609562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff274072c00 msgr2=0x7ff274073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.724+0000 7ff273200640 1 --2- 172.21.15.67:0/411609562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff274072c00 0x7ff274073050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff27406c860 tx=0x7ff26c02a6a0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.724+0000 7ff273200640 1 -- 172.21.15.67:0/411609562 shutdown_connections 2024-05-30T16:48:26.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.724+0000 7ff273200640 1 --2- 172.21.15.67:0/411609562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff274072c00 0x7ff274073050 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.724+0000 7ff273200640 1 --2- 172.21.15.67:0/411609562 >> v2:172.21.15.169:3300/0 conn(0x7ff27406dc20 0x7ff27415f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.724+0000 7ff273200640 1 --2- 172.21.15.67:0/411609562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff27415b3a0 0x7ff2740b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.724+0000 7ff273200640 1 -- 172.21.15.67:0/411609562 >> 172.21.15.67:0/411609562 conn(0x7ff2740b1fc0 msgr2=0x7ff2740b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:26.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.724+0000 7ff273200640 1 -- 172.21.15.67:0/411609562 shutdown_connections 2024-05-30T16:48:26.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.724+0000 7ff273200640 1 -- 172.21.15.67:0/411609562 wait complete. 2024-05-30T16:48:26.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.726+0000 7ff273200640 1 Processor -- start 2024-05-30T16:48:26.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.726+0000 7ff273200640 1 -- start start 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.727+0000 7ff273200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27406dc20 0x7ff27414e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.727+0000 7ff273200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274072c00 0x7ff2740cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.727+0000 7ff273200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff27415b3a0 0x7ff2741561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.727+0000 7ff273200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff2740afa80 con 0x7ff27415b3a0 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.727+0000 7ff273200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff27415fee0 con 0x7ff27406dc20 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.727+0000 7ff273200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff27415b780 con 0x7ff274072c00 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff272800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff27415b3a0 0x7ff2741561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff272800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff27415b3a0 0x7ff2741561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42796/0 (socket says 172.21.15.67:42796) 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff272800640 1 -- 172.21.15.67:0/3383010021 learned_addr learned my addr 172.21.15.67:0/3383010021 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff271e00640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27406dc20 0x7ff27414e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff26be00640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274072c00 0x7ff2740cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff272800640 1 -- 172.21.15.67:0/3383010021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274072c00 msgr2=0x7ff2740cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff272800640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274072c00 0x7ff2740cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff272800640 1 -- 172.21.15.67:0/3383010021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27406dc20 msgr2=0x7ff27414e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:26.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff272800640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27406dc20 0x7ff27414e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:26.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff272800640 1 -- 172.21.15.67:0/3383010021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2740c9640 con 0x7ff27415b3a0 2024-05-30T16:48:26.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.728+0000 7ff271e00640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27406dc20 0x7ff27414e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:48:26.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.729+0000 7ff272800640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff27415b3a0 0x7ff2741561c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009f40 tx=0x7ff26c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:26.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.729+0000 7ff26a000640 1 -- 172.21.15.67:0/3383010021 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c041020 con 0x7ff27415b3a0 2024-05-30T16:48:26.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.729+0000 7ff26a000640 1 -- 172.21.15.67:0/3383010021 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26c007720 con 0x7ff27415b3a0 2024-05-30T16:48:26.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.729+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2740c9920 con 0x7ff27415b3a0 2024-05-30T16:48:26.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.729+0000 7ff26a000640 1 -- 172.21.15.67:0/3383010021 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c036040 con 0x7ff27415b3a0 2024-05-30T16:48:26.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.729+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2740c9ee0 con 0x7ff27415b3a0 2024-05-30T16:48:26.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.730+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2740689e0 con 0x7ff27415b3a0 2024-05-30T16:48:26.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.731+0000 7ff26a000640 1 -- 172.21.15.67:0/3383010021 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7ff26c044020 con 0x7ff27415b3a0 2024-05-30T16:48:26.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.732+0000 7ff26a000640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff25006cbc0 0x7ff25006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:26.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.732+0000 7ff26a000640 1 -- 172.21.15.67:0/3383010021 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7ff26c0acab0 con 0x7ff27415b3a0 2024-05-30T16:48:26.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.732+0000 7ff271e00640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff25006cbc0 0x7ff25006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:26.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.733+0000 7ff271e00640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff25006cbc0 0x7ff25006f080 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff2741f7c60 tx=0x7ff264008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:26.737+0000 7ff26a000640 1 -- 172.21.15.67:0/3383010021 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff26c04b030 con 0x7ff27415b3a0 2024-05-30T16:48:27.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.015+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff2741522a0 con 0x7ff27415b3a0 2024-05-30T16:48:27.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.016+0000 7ff26a000640 1 -- 172.21.15.67:0/3383010021 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7ff26c07ad90 con 0x7ff27415b3a0 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA==", 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw==", 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.018 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw==", 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg==", 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:48:27.019 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng==", 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.020 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg==", 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w==", 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ==", 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-05-30T16:48:27.022 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A==", 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ==", 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.023 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w==", 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A==", 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.024 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg==", 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.025 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ==", 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.2", 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ==", 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.026 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.3", 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A==", 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCArFhmF5mhNhAA4H3cgcH6YdD53fANDW4seA==", 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-05-30T16:48:27.028 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ==", 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw==", 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:48:27.029 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:48:27.030 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.030 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.030 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-05-30T16:48:27.030 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:48:27.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff25006cbc0 msgr2=0x7ff25006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff25006cbc0 0x7ff25006f080 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff2741f7c60 tx=0x7ff264008040 comp rx=0 tx=0).stop 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff27415b3a0 msgr2=0x7ff2741561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff27415b3a0 0x7ff2741561c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009f40 tx=0x7ff26c004a40 comp rx=0 tx=0).stop 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 shutdown_connections 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff25006cbc0 0x7ff25006f080 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff27415b3a0 0x7ff2741561c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274072c00 0x7ff2740cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 --2- 172.21.15.67:0/3383010021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff27406dc20 0x7ff27414e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.018+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 >> 172.21.15.67:0/3383010021 conn(0x7ff2740b1fc0 msgr2=0x7ff2740b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.019+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 shutdown_connections 2024-05-30T16:48:27.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.019+0000 7ff273200640 1 -- 172.21.15.67:0/3383010021 wait complete. 2024-05-30T16:48:27.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-30T16:48:27.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.251+0000 7f5435e00640 1 Processor -- start 2024-05-30T16:48:27.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.251+0000 7f5435e00640 1 -- start start 2024-05-30T16:48:27.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.252+0000 7f5435e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5430162e70 0x7f543016d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.252+0000 7f5435e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54300b4350 con 0x7f54300d04a0 2024-05-30T16:48:27.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.252+0000 7f5435e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54300b32d0 con 0x7f543006f6e0 2024-05-30T16:48:27.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.252+0000 7f5435e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f54300b3ec0 con 0x7f5430162e70 2024-05-30T16:48:27.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.253+0000 7f542ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f54300d04a0 0x7f54300d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45964/0 (socket says 172.21.15.67:45964) 2024-05-30T16:48:27.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.253+0000 7f542ea00640 1 -- 172.21.15.67:0/1755504260 learned_addr learned my addr 172.21.15.67:0/1755504260 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:27.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.253+0000 7f542f400640 1 --2- 172.21.15.67:0/1755504260 >> v2:172.21.15.169:3300/0 conn(0x7f5430162e70 0x7f543016d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.253+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3091061686 0 0) 0x7f54300b4350 con 0x7f54300d04a0 2024-05-30T16:48:27.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.253+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5414003520 con 0x7f54300d04a0 2024-05-30T16:48:27.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 661273012 0 0) 0x7f5414003520 con 0x7f54300d04a0 2024-05-30T16:48:27.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 >> v2:172.21.15.169:3300/0 conn(0x7f5430162e70 msgr2=0x7f543016d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 --2- 172.21.15.67:0/1755504260 >> v2:172.21.15.169:3300/0 conn(0x7f5430162e70 0x7f543016d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 >> v1:172.21.15.169:6789/0 conn(0x7f543006f6e0 legacy=0x7f54300cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54300b3590 con 0x7f54300d04a0 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5420001670 con 0x7f54300d04a0 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 >> v1:172.21.15.67:6789/0 conn(0x7f54300d04a0 legacy=0x7f54300d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54140039b0 0x7f5414003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5414004800 0x7f5414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54140051c0 0x7f541400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f54300b3ec0 con 0x7f54140039b0 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f541400c160 con 0x7f54140051c0 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f541400c360 con 0x7f5414004800 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.254+0000 7f542ea00640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54140039b0 0x7f5414003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.255+0000 7f542f400640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54140051c0 0x7f541400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.255+0000 7f5427e00640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5414004800 0x7f5414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.255+0000 7f542ea00640 1 -- 172.21.15.67:0/1755504260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5414004800 msgr2=0x7f5414004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.255+0000 7f542ea00640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5414004800 0x7f5414004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.255+0000 7f542ea00640 1 -- 172.21.15.67:0/1755504260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54140051c0 msgr2=0x7f541400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.255+0000 7f542ea00640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54140051c0 0x7f541400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.255+0000 7f542ea00640 1 -- 172.21.15.67:0/1755504260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5414003520 con 0x7f54140039b0 2024-05-30T16:48:27.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.255+0000 7f542f400640 1 -- 172.21.15.67:0/1755504260 reap_dead start 2024-05-30T16:48:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.256+0000 7f542ea00640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54140039b0 0x7f5414003dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f542000c210 tx=0x7f5414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716877543d447880 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.256+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420007d30 con 0x7f54140039b0 2024-05-30T16:48:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.256+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5420005ad0 con 0x7f54140039b0 2024-05-30T16:48:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.256+0000 7f542e000640 1 -- 172.21.15.67:0/1755504260 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420007070 con 0x7f54140039b0 2024-05-30T16:48:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.257+0000 7f5435e00640 1 -- 172.21.15.67:0/1755504260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54140039b0 msgr2=0x7f5414003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.257+0000 7f5435e00640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54140039b0 0x7f5414003dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f542000c210 tx=0x7f5414005830 comp rx=0 tx=0).stop 2024-05-30T16:48:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.257+0000 7f5435e00640 1 -- 172.21.15.67:0/1755504260 shutdown_connections 2024-05-30T16:48:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.257+0000 7f5435e00640 1 --2- 172.21.15.67:0/1755504260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54140039b0 0x7f5414003dd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.257+0000 7f5435e00640 1 -- 172.21.15.67:0/1755504260 >> 172.21.15.67:0/1755504260 conn(0x7f543006bae0 msgr2=0x7f543006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.257+0000 7f5435e00640 1 -- 172.21.15.67:0/1755504260 shutdown_connections 2024-05-30T16:48:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.257+0000 7f5435e00640 1 -- 172.21.15.67:0/1755504260 wait complete. 2024-05-30T16:48:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.259+0000 7f5435e00640 1 Processor -- start 2024-05-30T16:48:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.259+0000 7f5435e00640 1 -- start start 2024-05-30T16:48:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.260+0000 7f5435e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54140039b0 0x7f5430163250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.260+0000 7f5435e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5430163790 0x7f54300c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.260+0000 7f5435e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54300c6850 0x7f543014d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.260+0000 7f5435e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f54300d0880 con 0x7f5430163790 2024-05-30T16:48:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.260+0000 7f5435e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f54300b3b50 con 0x7f54300c6850 2024-05-30T16:48:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.260+0000 7f5435e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f54300d0a00 con 0x7f54140039b0 2024-05-30T16:48:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.260+0000 7f5427e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5430163790 0x7f54300c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.260+0000 7f5427e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5430163790 0x7f54300c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42822/0 (socket says 172.21.15.67:42822) 2024-05-30T16:48:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.260+0000 7f5427e00640 1 -- 172.21.15.67:0/3656769800 learned_addr learned my addr 172.21.15.67:0/3656769800 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:27.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f542ea00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54140039b0 0x7f5430163250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f542f400640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54300c6850 0x7f543014d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5427e00640 1 -- 172.21.15.67:0/3656769800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54140039b0 msgr2=0x7f5430163250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5427e00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54140039b0 0x7f5430163250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5427e00640 1 -- 172.21.15.67:0/3656769800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54300c6850 msgr2=0x7f543014d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5427e00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54300c6850 0x7f543014d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5427e00640 1 -- 172.21.15.67:0/3656769800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5430160890 con 0x7f5430163790 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f542f400640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54300c6850 0x7f543014d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5427e00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5430163790 0x7f54300c6310 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5418001eb0 tx=0x7f541800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5427400640 1 -- 172.21.15.67:0/3656769800 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418018070 con 0x7f5430163790 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5427400640 1 -- 172.21.15.67:0/3656769800 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5418003610 con 0x7f5430163790 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5427400640 1 -- 172.21.15.67:0/3656769800 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54180039e0 con 0x7f5430163790 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.261+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5430160b70 con 0x7f5430163790 2024-05-30T16:48:27.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.262+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5430161130 con 0x7f5430163790 2024-05-30T16:48:27.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.263+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54300689e0 con 0x7f5430163790 2024-05-30T16:48:27.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.264+0000 7f5427400640 1 -- 172.21.15.67:0/3656769800 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f541801b070 con 0x7f5430163790 2024-05-30T16:48:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.265+0000 7f5427400640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f540c06cbf0 0x7f540c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.265+0000 7f5427400640 1 -- 172.21.15.67:0/3656769800 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f541808b380 con 0x7f5430163790 2024-05-30T16:48:27.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.265+0000 7f542ea00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f540c06cbf0 0x7f540c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.266+0000 7f542ea00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f540c06cbf0 0x7f540c06f0b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5420033000 tx=0x7f542002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:27.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.270+0000 7f5427400640 1 -- 172.21.15.67:0/3656769800 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5418059010 con 0x7f5430163790 2024-05-30T16:48:27.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.476+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f54300afc80 con 0x7f5430163790 2024-05-30T16:48:27.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.477+0000 7f5427400640 1 -- 172.21.15.67:0/3656769800 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) ==== 84+0+65 (secure 0 0 0) 0x7f541805d310 con 0x7f5430163790 2024-05-30T16:48:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:27.478 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:48:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-05-30T16:48:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-05-30T16:48:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:27.478 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:48:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.479+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f540c06cbf0 msgr2=0x7f540c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.479+0000 7f5435e00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f540c06cbf0 0x7f540c06f0b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5420033000 tx=0x7f542002e000 comp rx=0 tx=0).stop 2024-05-30T16:48:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.479+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5430163790 msgr2=0x7f54300c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.479+0000 7f5435e00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5430163790 0x7f54300c6310 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5418001eb0 tx=0x7f541800d890 comp rx=0 tx=0).stop 2024-05-30T16:48:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.479+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 shutdown_connections 2024-05-30T16:48:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.479+0000 7f5435e00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f540c06cbf0 0x7f540c06f0b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.479+0000 7f5435e00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54300c6850 0x7f543014d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.479+0000 7f5435e00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5430163790 0x7f54300c6310 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.479+0000 7f5435e00640 1 --2- 172.21.15.67:0/3656769800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54140039b0 0x7f5430163250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.480+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 >> 172.21.15.67:0/3656769800 conn(0x7f543006bae0 msgr2=0x7f543015e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.480+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 shutdown_connections 2024-05-30T16:48:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.480+0000 7f5435e00640 1 -- 172.21.15.67:0/3656769800 wait complete. 2024-05-30T16:48:27.502 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-30T16:48:27.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-30T16:48:27.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.700+0000 7f316d600640 1 Processor -- start 2024-05-30T16:48:27.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.700+0000 7f316d600640 1 -- start start 2024-05-30T16:48:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.701+0000 7f316d600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f316815c830 0x7f31680ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.701+0000 7f316d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31680ce770 0x7f31680b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.701+0000 7f316d600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f316815cc10 con 0x7f31680b45f0 2024-05-30T16:48:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.701+0000 7f316d600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f31680b4b90 con 0x7f316815c830 2024-05-30T16:48:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.701+0000 7f316d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f31680b4e70 con 0x7f31680ce770 2024-05-30T16:48:27.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.702+0000 7f3167400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f31680b45f0 0x7f31680d08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45966/0 (socket says 172.21.15.67:45966) 2024-05-30T16:48:27.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.702+0000 7f3167400640 1 -- 172.21.15.67:0/1332025588 learned_addr learned my addr 172.21.15.67:0/1332025588 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:27.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.702+0000 7f3166000640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31680ce770 0x7f31680b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.702+0000 7f3166a00640 1 --2- 172.21.15.67:0/1332025588 >> v2:172.21.15.169:3300/0 conn(0x7f316815c830 0x7f31680ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.702+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3868904222 0 0) 0x7f316815cc10 con 0x7f31680b45f0 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.702+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f314c003520 con 0x7f31680b45f0 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1605864120 0 0) 0x7f314c003520 con 0x7f31680b45f0 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31680ce770 msgr2=0x7f31680b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31680ce770 0x7f31680b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 >> v2:172.21.15.169:3300/0 conn(0x7f316815c830 msgr2=0x7f31680ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 --2- 172.21.15.67:0/1332025588 >> v2:172.21.15.169:3300/0 conn(0x7f316815c830 0x7f31680ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31680b5150 con 0x7f31680b45f0 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f31600019f0 con 0x7f31680b45f0 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 >> v1:172.21.15.67:6789/0 conn(0x7f31680b45f0 legacy=0x7f31680d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f314c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31680ce770 0x7f314c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f314c004d40 0x7f314c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f31680b4b90 con 0x7f314c0039b0 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f31680b4e70 con 0x7f314c004d40 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f314c00b9c0 con 0x7f31680ce770 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.703+0000 7f3166a00640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f314c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.704+0000 7f3166000640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31680ce770 0x7f314c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.704+0000 7f3166a00640 1 -- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31680ce770 msgr2=0x7f314c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.704+0000 7f3166a00640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31680ce770 0x7f314c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.704+0000 7f3166a00640 1 -- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f314c004d40 msgr2=0x7f314c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.704+0000 7f3166a00640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f314c004d40 0x7f314c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.704+0000 7f3166a00640 1 -- 172.21.15.67:0/1332025588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c003520 con 0x7f314c0039b0 2024-05-30T16:48:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.705+0000 7f3166a00640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f314c003dd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3158009af0 tx=0x7f314c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd46f44b4588723 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.705+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158036070 con 0x7f314c0039b0 2024-05-30T16:48:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.705+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3158031400 con 0x7f314c0039b0 2024-05-30T16:48:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.705+0000 7f3165600640 1 -- 172.21.15.67:0/1332025588 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158009690 con 0x7f314c0039b0 2024-05-30T16:48:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.706+0000 7f316d600640 1 -- 172.21.15.67:0/1332025588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 msgr2=0x7f314c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.706+0000 7f316d600640 1 --2- 172.21.15.67:0/1332025588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f314c003dd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3158009af0 tx=0x7f314c005830 comp rx=0 tx=0).stop 2024-05-30T16:48:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.706+0000 7f3167400640 1 -- 172.21.15.67:0/1332025588 reap_dead start 2024-05-30T16:48:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.706+0000 7f316d600640 1 -- 172.21.15.67:0/1332025588 shutdown_connections 2024-05-30T16:48:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.706+0000 7f316d600640 1 -- 172.21.15.67:0/1332025588 >> 172.21.15.67:0/1332025588 conn(0x7f31680b20a0 msgr2=0x7f31680b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.706+0000 7f316d600640 1 -- 172.21.15.67:0/1332025588 shutdown_connections 2024-05-30T16:48:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.706+0000 7f316d600640 1 -- 172.21.15.67:0/1332025588 wait complete. 2024-05-30T16:48:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.708+0000 7f316d600640 1 Processor -- start 2024-05-30T16:48:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.708+0000 7f316d600640 1 -- start start 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.709+0000 7f316d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f31681581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.709+0000 7f316d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31681589e0 0x7f316806e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.709+0000 7f316d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3168158e60 0x7f316806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.709+0000 7f316d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f31680d0160 con 0x7f314c0039b0 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.709+0000 7f316d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f31680cffe0 con 0x7f3168158e60 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.709+0000 7f316d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3168160480 con 0x7f31681589e0 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.709+0000 7f3166a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f31681581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.709+0000 7f3166a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f31681581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42832/0 (socket says 172.21.15.67:42832) 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.709+0000 7f3166a00640 1 -- 172.21.15.67:0/2178179940 learned_addr learned my addr 172.21.15.67:0/2178179940 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.710+0000 7f3167400640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3168158e60 0x7f316806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.710+0000 7f3166000640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31681589e0 0x7f316806e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.710+0000 7f3166a00640 1 -- 172.21.15.67:0/2178179940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31681589e0 msgr2=0x7f316806e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.710+0000 7f3166a00640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31681589e0 0x7f316806e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.710+0000 7f3166a00640 1 -- 172.21.15.67:0/2178179940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3168158e60 msgr2=0x7f316806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.710+0000 7f3166a00640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3168158e60 0x7f316806eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.710+0000 7f3166a00640 1 -- 172.21.15.67:0/2178179940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31681586e0 con 0x7f314c0039b0 2024-05-30T16:48:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.710+0000 7f3166000640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31681589e0 0x7f316806e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.710+0000 7f3166a00640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f31681581a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3158009af0 tx=0x7f3158004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.711+0000 7f315ea00640 1 -- 172.21.15.67:0/2178179940 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158041020 con 0x7f314c0039b0 2024-05-30T16:48:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.711+0000 7f315ea00640 1 -- 172.21.15.67:0/2178179940 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f315802ade0 con 0x7f314c0039b0 2024-05-30T16:48:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.711+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31680caea0 con 0x7f314c0039b0 2024-05-30T16:48:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.711+0000 7f315ea00640 1 -- 172.21.15.67:0/2178179940 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158036040 con 0x7f314c0039b0 2024-05-30T16:48:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.711+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31680cb460 con 0x7f314c0039b0 2024-05-30T16:48:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.712+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31680757d0 con 0x7f314c0039b0 2024-05-30T16:48:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.713+0000 7f315ea00640 1 -- 172.21.15.67:0/2178179940 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f3158044020 con 0x7f314c0039b0 2024-05-30T16:48:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.714+0000 7f315ea00640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f314806cbc0 0x7f314806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.714+0000 7f315ea00640 1 -- 172.21.15.67:0/2178179940 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f31580ac180 con 0x7f314c0039b0 2024-05-30T16:48:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.714+0000 7f3166000640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f314806cbc0 0x7f314806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.715+0000 7f3166000640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f314806cbc0 0x7f314806f080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31540099e0 tx=0x7f3154008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.718+0000 7f315ea00640 1 -- 172.21.15.67:0/2178179940 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f315804b030 con 0x7f314c0039b0 2024-05-30T16:48:27.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:27.922+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3168160790 con 0x7f314c0039b0 2024-05-30T16:48:28.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.512+0000 7f315ea00640 1 -- 172.21.15.67:0/2178179940 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) ==== 138+0+0 (secure 0 0 0) 0x7f31681586e0 con 0x7f314c0039b0 2024-05-30T16:48:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-05-30T16:48:28.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f314806cbc0 msgr2=0x7f314806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:28.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f314806cbc0 0x7f314806f080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31540099e0 tx=0x7f3154008040 comp rx=0 tx=0).stop 2024-05-30T16:48:28.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 msgr2=0x7f31681581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:28.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f31681581a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3158009af0 tx=0x7f3158004730 comp rx=0 tx=0).stop 2024-05-30T16:48:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 shutdown_connections 2024-05-30T16:48:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f314806cbc0 0x7f314806f080 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3168158e60 0x7f316806eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31681589e0 0x7f316806e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 --2- 172.21.15.67:0/2178179940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f314c0039b0 0x7f31681581a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.516+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 >> 172.21.15.67:0/2178179940 conn(0x7f31680b20a0 msgr2=0x7f31680b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.517+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 shutdown_connections 2024-05-30T16:48:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.517+0000 7f316d600640 1 -- 172.21.15.67:0/2178179940 wait complete. 2024-05-30T16:48:28.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-30T16:48:28.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.761+0000 7fab96600640 1 Processor -- start 2024-05-30T16:48:28.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.761+0000 7fab96600640 1 -- start start 2024-05-30T16:48:28.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.762+0000 7fab96600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 0x7fab900ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:28.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.762+0000 7fab96600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab901524c0 0x7fab900cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:28.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.762+0000 7fab96600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab900b3e80 con 0x7fab900cae20 2024-05-30T16:48:28.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.762+0000 7fab96600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fab900d0b30 con 0x7fab901524c0 2024-05-30T16:48:28.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.762+0000 7fab96600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab900b4160 con 0x7fab900caa40 2024-05-30T16:48:28.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.763+0000 7fab95c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab901524c0 0x7fab900cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:28.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.763+0000 7fab95c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab901524c0 0x7fab900cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42840/0 (socket says 172.21.15.67:42840) 2024-05-30T16:48:28.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.763+0000 7fab95c00640 1 -- 172.21.15.67:0/81262884 learned_addr learned my addr 172.21.15.67:0/81262884 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:28.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.763+0000 7fab95200640 1 --2- 172.21.15.67:0/81262884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 0x7fab900ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:28.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.763+0000 7fab95c00640 1 -- 172.21.15.67:0/81262884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 msgr2=0x7fab900ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:28.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.763+0000 7fab95c00640 1 --2- 172.21.15.67:0/81262884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 0x7fab900ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:28.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.763+0000 7fab95c00640 1 -- 172.21.15.67:0/81262884 >> v1:172.21.15.67:6789/0 conn(0x7fab900cae20 legacy=0x7fab90151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:28.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.763+0000 7fab95c00640 1 -- 172.21.15.67:0/81262884 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab900cde80 con 0x7fab901524c0 2024-05-30T16:48:28.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.763+0000 7fab8f400640 1 -- 172.21.15.67:0/81262884 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656475143 0 0) 0x7fab900b3e80 con 0x7fab900cae20 2024-05-30T16:48:28.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.765+0000 7fab95c00640 1 --2- 172.21.15.67:0/81262884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 0x7fab900cf710 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fab88009e50 tx=0x7fab8802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fde5f2080f5c4e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:28.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.765+0000 7fab8f400640 1 -- 172.21.15.67:0/81262884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88036070 con 0x7fab901524c0 2024-05-30T16:48:28.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.765+0000 7fab8f400640 1 -- 172.21.15.67:0/81262884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab88031600 con 0x7fab901524c0 2024-05-30T16:48:28.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.765+0000 7fab8f400640 1 -- 172.21.15.67:0/81262884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab880319d0 con 0x7fab901524c0 2024-05-30T16:48:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.765+0000 7fab96600640 1 -- 172.21.15.67:0/81262884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 msgr2=0x7fab900cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.765+0000 7fab96600640 1 --2- 172.21.15.67:0/81262884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 0x7fab900cf710 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fab88009e50 tx=0x7fab8802a680 comp rx=0 tx=0).stop 2024-05-30T16:48:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.766+0000 7fab96600640 1 -- 172.21.15.67:0/81262884 shutdown_connections 2024-05-30T16:48:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.766+0000 7fab96600640 1 --2- 172.21.15.67:0/81262884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 0x7fab900cf710 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.766+0000 7fab96600640 1 --2- 172.21.15.67:0/81262884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 0x7fab900ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.766+0000 7fab96600640 1 -- 172.21.15.67:0/81262884 >> 172.21.15.67:0/81262884 conn(0x7fab9006b900 msgr2=0x7fab9006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.766+0000 7fab96600640 1 -- 172.21.15.67:0/81262884 shutdown_connections 2024-05-30T16:48:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.766+0000 7fab96600640 1 -- 172.21.15.67:0/81262884 wait complete. 2024-05-30T16:48:28.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.768+0000 7fab96600640 1 Processor -- start 2024-05-30T16:48:28.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.768+0000 7fab96600640 1 -- start start 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab96600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 0x7fab9009a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab96600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab900cae20 0x7fab901474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab96600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 0x7fab90150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab96600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fab900ceea0 con 0x7fab901524c0 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab96600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fab900ce440 con 0x7fab900cae20 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab96600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab900cf020 con 0x7fab900caa40 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab95c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 0x7fab90150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab95c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 0x7fab90150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42856/0 (socket says 172.21.15.67:42856) 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab95c00640 1 -- 172.21.15.67:0/3917973157 learned_addr learned my addr 172.21.15.67:0/3917973157 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:28.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.769+0000 7fab95200640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 0x7fab9009a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab8fe00640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab900cae20 0x7fab901474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab95c00640 1 -- 172.21.15.67:0/3917973157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 msgr2=0x7fab9009a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab95c00640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 0x7fab9009a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab95c00640 1 -- 172.21.15.67:0/3917973157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab900cae20 msgr2=0x7fab901474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab95c00640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab900cae20 0x7fab901474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab95c00640 1 -- 172.21.15.67:0/3917973157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab900b1600 con 0x7fab901524c0 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab95c00640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 0x7fab90150210 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fab88034040 tx=0x7fab88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab8cc00640 1 -- 172.21.15.67:0/3917973157 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88041020 con 0x7fab901524c0 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab8cc00640 1 -- 172.21.15.67:0/3917973157 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab88037040 con 0x7fab901524c0 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab8cc00640 1 -- 172.21.15.67:0/3917973157 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88036050 con 0x7fab901524c0 2024-05-30T16:48:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.770+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab900b1870 con 0x7fab901524c0 2024-05-30T16:48:28.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.771+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9014fcf0 con 0x7fab901524c0 2024-05-30T16:48:28.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.772+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab900689e0 con 0x7fab901524c0 2024-05-30T16:48:28.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.773+0000 7fab8cc00640 1 -- 172.21.15.67:0/3917973157 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fab88007c50 con 0x7fab901524c0 2024-05-30T16:48:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.774+0000 7fab8cc00640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab7006cbc0 0x7fab7006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.774+0000 7fab8cc00640 1 -- 172.21.15.67:0/3917973157 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5227+0+0 (secure 0 0 0) 0x7fab880ac610 con 0x7fab901524c0 2024-05-30T16:48:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.774+0000 7fab95200640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab7006cbc0 0x7fab7006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.775+0000 7fab95200640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab7006cbc0 0x7fab7006f080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab80009ad0 tx=0x7fab80008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:28.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.779+0000 7fab8cc00640 1 -- 172.21.15.67:0/3917973157 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab8804b060 con 0x7fab901524c0 2024-05-30T16:48:28.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:28.986+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fab9009a7f0 con 0x7fab901524c0 2024-05-30T16:48:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.529+0000 7fab8cc00640 1 -- 172.21.15.67:0/3917973157 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) ==== 170+0+0 (secure 0 0 0) 0x7fab8807a810 con 0x7fab901524c0 2024-05-30T16:48:29.530 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-05-30T16:48:29.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.533+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab7006cbc0 msgr2=0x7fab7006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.533+0000 7fab96600640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab7006cbc0 0x7fab7006f080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab80009ad0 tx=0x7fab80008040 comp rx=0 tx=0).stop 2024-05-30T16:48:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.533+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 msgr2=0x7fab90150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:29.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.533+0000 7fab96600640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 0x7fab90150210 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fab88034040 tx=0x7fab88004a40 comp rx=0 tx=0).stop 2024-05-30T16:48:29.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.535+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 shutdown_connections 2024-05-30T16:48:29.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.535+0000 7fab96600640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab7006cbc0 0x7fab7006f080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:29.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.535+0000 7fab96600640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab901524c0 0x7fab90150210 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:29.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.535+0000 7fab96600640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab900cae20 0x7fab901474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.535+0000 7fab96600640 1 --2- 172.21.15.67:0/3917973157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab900caa40 0x7fab9009a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.535+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 >> 172.21.15.67:0/3917973157 conn(0x7fab9006b900 msgr2=0x7fab9006e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.536+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 shutdown_connections 2024-05-30T16:48:29.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.536+0000 7fab96600640 1 -- 172.21.15.67:0/3917973157 wait complete. 2024-05-30T16:48:29.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-30T16:48:29.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.752+0000 7f208f200640 1 Processor -- start 2024-05-30T16:48:29.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.753+0000 7f208f200640 1 -- start start 2024-05-30T16:48:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.754+0000 7f208f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20900cae20 0x7f2090151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.754+0000 7f208f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2090152780 con 0x7f20901522f0 2024-05-30T16:48:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.754+0000 7f208f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20900b3e80 con 0x7f20900caa40 2024-05-30T16:48:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.754+0000 7f208f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20900d0b30 con 0x7f20900cae20 2024-05-30T16:48:29.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.755+0000 7f208e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f20901522f0 0x7f20900cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45976/0 (socket says 172.21.15.67:45976) 2024-05-30T16:48:29.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.755+0000 7f208e800640 1 -- 172.21.15.67:0/1351636493 learned_addr learned my addr 172.21.15.67:0/1351636493 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:29.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.755+0000 7f2085400640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20900cae20 0x7f2090151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:29.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.755+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4294897696 0 0) 0x7f2090152780 con 0x7f20901522f0 2024-05-30T16:48:29.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.755+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2074005730 con 0x7f20901522f0 2024-05-30T16:48:29.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2574029013 0 0) 0x7f2074005730 con 0x7f20901522f0 2024-05-30T16:48:29.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20900cae20 msgr2=0x7f2090151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:29.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20900cae20 0x7f2090151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:29.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 >> v1:172.21.15.169:6789/0 conn(0x7f20900caa40 legacy=0x7f20900ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:29.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20900cde80 con 0x7f20901522f0 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f20880019f0 con 0x7f20901522f0 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 >> v1:172.21.15.67:6789/0 conn(0x7f20901522f0 legacy=0x7f20900cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2074005bf0 0x7f2074006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20900cae20 0x7f2074006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2074006f50 0x7f20740073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20900d0b30 con 0x7f2074006f50 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f207400dd00 con 0x7f2074005bf0 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f207400df00 con 0x7f20900cae20 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.756+0000 7f208e800640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2074006f50 0x7f20740073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.757+0000 7f208de00640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2074005bf0 0x7f2074006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.757+0000 7f2085400640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20900cae20 0x7f2074006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.757+0000 7f208e800640 1 -- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20900cae20 msgr2=0x7f2074006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.757+0000 7f208e800640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20900cae20 0x7f2074006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.757+0000 7f208e800640 1 -- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2074005bf0 msgr2=0x7f2074006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.757+0000 7f208e800640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2074005bf0 0x7f2074006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.757+0000 7f208e800640 1 -- 172.21.15.67:0/1351636493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2074005730 con 0x7f2074006f50 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.758+0000 7f208e800640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2074006f50 0x7f20740073d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f208800a840 tx=0x7f2074007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72640a61423ca244 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.758+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208800a460 con 0x7f2074006f50 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.758+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f208802f070 con 0x7f2074006f50 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.758+0000 7f208d400640 1 -- 172.21.15.67:0/1351636493 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208802bcb0 con 0x7f2074006f50 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.759+0000 7f208f200640 1 -- 172.21.15.67:0/1351636493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2074006f50 msgr2=0x7f20740073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.759+0000 7f208f200640 1 --2- 172.21.15.67:0/1351636493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2074006f50 0x7f20740073d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f208800a840 tx=0x7f2074007a40 comp rx=0 tx=0).stop 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.759+0000 7f208e800640 1 -- 172.21.15.67:0/1351636493 reap_dead start 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.759+0000 7f208f200640 1 -- 172.21.15.67:0/1351636493 shutdown_connections 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.759+0000 7f208f200640 1 -- 172.21.15.67:0/1351636493 >> 172.21.15.67:0/1351636493 conn(0x7f209006b900 msgr2=0x7f209006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.759+0000 7f208f200640 1 -- 172.21.15.67:0/1351636493 shutdown_connections 2024-05-30T16:48:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.759+0000 7f208f200640 1 -- 172.21.15.67:0/1351636493 wait complete. 2024-05-30T16:48:29.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.761+0000 7f208f200640 1 Processor -- start 2024-05-30T16:48:29.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.761+0000 7f208f200640 1 -- start start 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.762+0000 7f208f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2074006f50 0x7f2090141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.762+0000 7f208f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2090151320 0x7f20901439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.762+0000 7f208f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2090158660 0x7f2090158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.762+0000 7f208f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20900cf070 con 0x7f2090158660 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.762+0000 7f208f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20900b4160 con 0x7f2074006f50 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.762+0000 7f208f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20900ce440 con 0x7f2090151320 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f208e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2090158660 0x7f2090158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f208e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2090158660 0x7f2090158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42872/0 (socket says 172.21.15.67:42872) 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f208e800640 1 -- 172.21.15.67:0/2684779428 learned_addr learned my addr 172.21.15.67:0/2684779428 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f208de00640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2074006f50 0x7f2090141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f2085400640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2090151320 0x7f20901439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f208e800640 1 -- 172.21.15.67:0/2684779428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2090151320 msgr2=0x7f20901439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f208e800640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2090151320 0x7f20901439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f208e800640 1 -- 172.21.15.67:0/2684779428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2074006f50 msgr2=0x7f2090141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f208e800640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2074006f50 0x7f2090141240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.763+0000 7f208e800640 1 -- 172.21.15.67:0/2684779428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901f4250 con 0x7f2090158660 2024-05-30T16:48:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.764+0000 7f208e800640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2090158660 0x7f2090158ae0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2088000f80 tx=0x7f2090155ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.764+0000 7f2086a00640 1 -- 172.21.15.67:0/2684779428 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208800a460 con 0x7f2090158660 2024-05-30T16:48:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.764+0000 7f2086a00640 1 -- 172.21.15.67:0/2684779428 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2088034470 con 0x7f2090158660 2024-05-30T16:48:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.764+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2090144900 con 0x7f2090158660 2024-05-30T16:48:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.764+0000 7f2086a00640 1 -- 172.21.15.67:0/2684779428 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088001fb0 con 0x7f2090158660 2024-05-30T16:48:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.764+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2090144620 con 0x7f2090158660 2024-05-30T16:48:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.765+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20900689e0 con 0x7f2090158660 2024-05-30T16:48:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.767+0000 7f2086a00640 1 -- 172.21.15.67:0/2684779428 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f2088034820 con 0x7f2090158660 2024-05-30T16:48:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.768+0000 7f2086a00640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f206c06cbc0 0x7f206c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.768+0000 7f2086a00640 1 -- 172.21.15.67:0/2684779428 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5609+0+0 (secure 0 0 0) 0x7f20880ac360 con 0x7f2090158660 2024-05-30T16:48:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.768+0000 7f208de00640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f206c06cbc0 0x7f206c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.769+0000 7f208de00640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f206c06cbc0 0x7f206c06f080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2080007fd0 tx=0x7f2080009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:29.772+0000 7f2086a00640 1 -- 172.21.15.67:0/2684779428 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2088046060 con 0x7f2090158660 2024-05-30T16:48:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.018+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f209006c060 con 0x7f2090158660 2024-05-30T16:48:30.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.533+0000 7f2086a00640 1 -- 172.21.15.67:0/2684779428 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7f208807a450 con 0x7f2090158660 2024-05-30T16:48:30.535 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 6 and data pool 7 2024-05-30T16:48:30.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.538+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f206c06cbc0 msgr2=0x7f206c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:30.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.538+0000 7f208f200640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f206c06cbc0 0x7f206c06f080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2080007fd0 tx=0x7f2080009040 comp rx=0 tx=0).stop 2024-05-30T16:48:30.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.538+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2090158660 msgr2=0x7f2090158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:30.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.538+0000 7f208f200640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2090158660 0x7f2090158ae0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2088000f80 tx=0x7f2090155ec0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.540+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 shutdown_connections 2024-05-30T16:48:30.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.540+0000 7f208f200640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f206c06cbc0 0x7f206c06f080 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.540+0000 7f208f200640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2090158660 0x7f2090158ae0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.540+0000 7f208f200640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2090151320 0x7f20901439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.540+0000 7f208f200640 1 --2- 172.21.15.67:0/2684779428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2074006f50 0x7f2090141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.540+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 >> 172.21.15.67:0/2684779428 conn(0x7f209006b900 msgr2=0x7f209006e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:30.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.541+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 shutdown_connections 2024-05-30T16:48:30.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.541+0000 7f208f200640 1 -- 172.21.15.67:0/2684779428 wait complete. 2024-05-30T16:48:30.575 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:48:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.755+0000 7fc84ca00640 1 Processor -- start 2024-05-30T16:48:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.756+0000 7fc84ca00640 1 -- start start 2024-05-30T16:48:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.757+0000 7fc84ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc848072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.757+0000 7fc84ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc8480b32c0 0x7fc8480b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.757+0000 7fc84ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc84806e6e0 con 0x7fc848158b80 2024-05-30T16:48:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.757+0000 7fc84ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc84806e9c0 con 0x7fc8480b32c0 2024-05-30T16:48:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.757+0000 7fc84ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc84806eca0 con 0x7fc848071bc0 2024-05-30T16:48:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.758+0000 7fc83de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc848072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:30.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.758+0000 7fc847200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc8480b32c0 0x7fc8480b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:30.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.758+0000 7fc83de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc848072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:41328/0 (socket says 172.21.15.67:41328) 2024-05-30T16:48:30.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.758+0000 7fc83de00640 1 -- 172.21.15.67:0/3858152761 learned_addr learned my addr 172.21.15.67:0/3858152761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:30.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.758+0000 7fc847200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc8480b32c0 0x7fc8480b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36358/0 (socket says 172.21.15.67:36358) 2024-05-30T16:48:30.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.759+0000 7fc83de00640 1 -- 172.21.15.67:0/3858152761 >> v2:172.21.15.169:3300/0 conn(0x7fc8480b32c0 msgr2=0x7fc8480b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:30.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.759+0000 7fc83de00640 1 --2- 172.21.15.67:0/3858152761 >> v2:172.21.15.169:3300/0 conn(0x7fc8480b32c0 0x7fc8480b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.759+0000 7fc83de00640 1 -- 172.21.15.67:0/3858152761 >> v1:172.21.15.169:6789/0 conn(0x7fc848158b80 legacy=0x7fc84806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:30.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.759+0000 7fc83de00640 1 -- 172.21.15.67:0/3858152761 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84806ef80 con 0x7fc848071bc0 2024-05-30T16:48:30.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.759+0000 7fc847200640 1 --2- 172.21.15.67:0/3858152761 >> v2:172.21.15.169:3300/0 conn(0x7fc8480b32c0 0x7fc8480b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:48:30.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.760+0000 7fc83de00640 1 --2- 172.21.15.67:0/3858152761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc848072070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc830009870 tx=0x7fc83002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c34b058057836fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:30.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.760+0000 7fc845e00640 1 -- 172.21.15.67:0/3858152761 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830037070 con 0x7fc848071bc0 2024-05-30T16:48:30.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.760+0000 7fc845e00640 1 -- 172.21.15.67:0/3858152761 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc830032440 con 0x7fc848071bc0 2024-05-30T16:48:30.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.760+0000 7fc845e00640 1 -- 172.21.15.67:0/3858152761 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830032810 con 0x7fc848071bc0 2024-05-30T16:48:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.761+0000 7fc84ca00640 1 -- 172.21.15.67:0/3858152761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 msgr2=0x7fc848072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.761+0000 7fc84ca00640 1 --2- 172.21.15.67:0/3858152761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc848072070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc830009870 tx=0x7fc83002a250 comp rx=0 tx=0).stop 2024-05-30T16:48:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.761+0000 7fc84ca00640 1 -- 172.21.15.67:0/3858152761 shutdown_connections 2024-05-30T16:48:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.761+0000 7fc84ca00640 1 --2- 172.21.15.67:0/3858152761 >> v2:172.21.15.169:3300/0 conn(0x7fc8480b32c0 0x7fc8480b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.761+0000 7fc84ca00640 1 --2- 172.21.15.67:0/3858152761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc848072070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.761+0000 7fc84ca00640 1 -- 172.21.15.67:0/3858152761 >> 172.21.15.67:0/3858152761 conn(0x7fc8480b1fc0 msgr2=0x7fc8480b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.761+0000 7fc84ca00640 1 -- 172.21.15.67:0/3858152761 shutdown_connections 2024-05-30T16:48:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.761+0000 7fc84ca00640 1 -- 172.21.15.67:0/3858152761 wait complete. 2024-05-30T16:48:30.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.763+0000 7fc84ca00640 1 Processor -- start 2024-05-30T16:48:30.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.763+0000 7fc84ca00640 1 -- start start 2024-05-30T16:48:30.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.764+0000 7fc84ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc8481513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:30.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.764+0000 7fc84ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8480b32c0 0x7fc8481f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:30.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.764+0000 7fc84ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc848158b80 0x7fc8481f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:30.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.764+0000 7fc84ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc84806f780 con 0x7fc8480b32c0 2024-05-30T16:48:30.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.764+0000 7fc84ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8480b2d30 con 0x7fc848158b80 2024-05-30T16:48:30.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.764+0000 7fc84ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc848156020 con 0x7fc848071bc0 2024-05-30T16:48:30.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.764+0000 7fc83de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8480b32c0 0x7fc8481f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:30.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.764+0000 7fc83de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8480b32c0 0x7fc8481f7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42876/0 (socket says 172.21.15.67:42876) 2024-05-30T16:48:30.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.764+0000 7fc83de00640 1 -- 172.21.15.67:0/3835934213 learned_addr learned my addr 172.21.15.67:0/3835934213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc846800640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc8481513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc847200640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc848158b80 0x7fc8481f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc83de00640 1 -- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 msgr2=0x7fc8481513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc83de00640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc8481513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc83de00640 1 -- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc848158b80 msgr2=0x7fc8481f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc83de00640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc848158b80 0x7fc8481f83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc83de00640 1 -- 172.21.15.67:0/3835934213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8481518f0 con 0x7fc8480b32c0 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc846800640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc8481513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc847200640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc848158b80 0x7fc8481f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:30.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc83de00640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8480b32c0 0x7fc8481f7eb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc830000c00 tx=0x7fc8300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:30.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.765+0000 7fc83f400640 1 -- 172.21.15.67:0/3835934213 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830037040 con 0x7fc8480b32c0 2024-05-30T16:48:30.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.766+0000 7fc83f400640 1 -- 172.21.15.67:0/3835934213 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc830042070 con 0x7fc8480b32c0 2024-05-30T16:48:30.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.766+0000 7fc83f400640 1 -- 172.21.15.67:0/3835934213 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830030880 con 0x7fc8480b32c0 2024-05-30T16:48:30.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.766+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8481e95d0 con 0x7fc8480b32c0 2024-05-30T16:48:30.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.766+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481e9b90 con 0x7fc8480b32c0 2024-05-30T16:48:30.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.767+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8480689e0 con 0x7fc8480b32c0 2024-05-30T16:48:30.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.768+0000 7fc83f400640 1 -- 172.21.15.67:0/3835934213 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fc830030a20 con 0x7fc8480b32c0 2024-05-30T16:48:30.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.769+0000 7fc83f400640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc82406cbf0 0x7fc82406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:30.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.769+0000 7fc83f400640 1 -- 172.21.15.67:0/3835934213 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fc8300ac7f0 con 0x7fc8480b32c0 2024-05-30T16:48:30.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.769+0000 7fc846800640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc82406cbf0 0x7fc82406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:30.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.770+0000 7fc846800640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc82406cbf0 0x7fc82406f0b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc8380015e0 tx=0x7fc838009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.774+0000 7fc83f400640 1 -- 172.21.15.67:0/3835934213 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc83007a920 con 0x7fc8480b32c0 2024-05-30T16:48:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.979+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8480af740 con 0x7fc8480b32c0 2024-05-30T16:48:30.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.982+0000 7fc83f400640 1 -- 172.21.15.67:0/3835934213 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16614 (secure 0 0 0) 0x7fc83007a220 con 0x7fc8480b32c0 2024-05-30T16:48:30.983 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:30.983 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":47,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:48:30.019991+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:48:27.936635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-30T16:48:29.004136+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc82406cbf0 msgr2=0x7fc82406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc82406cbf0 0x7fc82406f0b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc8380015e0 tx=0x7fc838009040 comp rx=0 tx=0).stop 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8480b32c0 msgr2=0x7fc8481f7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8480b32c0 0x7fc8481f7eb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc830000c00 tx=0x7fc8300046c0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 shutdown_connections 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc82406cbf0 0x7fc82406f0b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc848158b80 0x7fc8481f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8480b32c0 0x7fc8481f7eb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 --2- 172.21.15.67:0/3835934213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc848071bc0 0x7fc8481513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 >> 172.21.15.67:0/3835934213 conn(0x7fc8480b1fc0 msgr2=0x7fc8480d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 shutdown_connections 2024-05-30T16:48:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:30.985+0000 7fc84ca00640 1 -- 172.21.15.67:0/3835934213 wait complete. 2024-05-30T16:48:31.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:48:31.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.191+0000 7f5be9e00640 1 Processor -- start 2024-05-30T16:48:31.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.191+0000 7f5be9e00640 1 -- start start 2024-05-30T16:48:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.193+0000 7f5be9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5be415e870 0x7f5be40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.193+0000 7f5be9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40d14d0 0x7f5be40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.193+0000 7f5be9e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be40cfd00 con 0x7f5be40cf870 2024-05-30T16:48:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.193+0000 7f5be9e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5be40d2150 con 0x7f5be415e870 2024-05-30T16:48:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.193+0000 7f5be9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5be415ec50 con 0x7f5be40d14d0 2024-05-30T16:48:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.193+0000 7f5be8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5be415e870 0x7f5be40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.194+0000 7f5be8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5be415e870 0x7f5be40d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57942/0 (socket says 172.21.15.67:57942) 2024-05-30T16:48:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.194+0000 7f5be8a00640 1 -- 172.21.15.67:0/2523988553 learned_addr learned my addr 172.21.15.67:0/2523988553 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.194+0000 7f5be3e00640 1 --2- 172.21.15.67:0/2523988553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40d14d0 0x7f5be40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.194+0000 7f5be8a00640 1 -- 172.21.15.67:0/2523988553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40d14d0 msgr2=0x7f5be40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.194+0000 7f5be8a00640 1 --2- 172.21.15.67:0/2523988553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40d14d0 0x7f5be40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.194+0000 7f5be8a00640 1 -- 172.21.15.67:0/2523988553 >> v1:172.21.15.67:6789/0 conn(0x7f5be40cf870 legacy=0x7f5be4162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.194+0000 7f5be8a00640 1 -- 172.21.15.67:0/2523988553 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be40cff80 con 0x7f5be415e870 2024-05-30T16:48:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.194+0000 7f5be3400640 1 -- 172.21.15.67:0/2523988553 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3173438627 0 0) 0x7f5bdc001490 con 0x7f5be40cf870 2024-05-30T16:48:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.195+0000 7f5be8a00640 1 --2- 172.21.15.67:0/2523988553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 0x7f5be40d0f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009870 tx=0x7f5bd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6646298b77e347 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be3400640 1 -- 172.21.15.67:0/2523988553 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8037070 con 0x7f5be415e870 2024-05-30T16:48:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be3400640 1 -- 172.21.15.67:0/2523988553 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd8032440 con 0x7f5be415e870 2024-05-30T16:48:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be3400640 1 -- 172.21.15.67:0/2523988553 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8032810 con 0x7f5be415e870 2024-05-30T16:48:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be9e00640 1 -- 172.21.15.67:0/2523988553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 msgr2=0x7f5be40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be9e00640 1 --2- 172.21.15.67:0/2523988553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 0x7f5be40d0f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009870 tx=0x7f5bd802a270 comp rx=0 tx=0).stop 2024-05-30T16:48:31.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be9e00640 1 -- 172.21.15.67:0/2523988553 shutdown_connections 2024-05-30T16:48:31.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be9e00640 1 --2- 172.21.15.67:0/2523988553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40d14d0 0x7f5be40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be9e00640 1 --2- 172.21.15.67:0/2523988553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 0x7f5be40d0f90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be9e00640 1 -- 172.21.15.67:0/2523988553 >> 172.21.15.67:0/2523988553 conn(0x7f5be408bfc0 msgr2=0x7f5be40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:31.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be9e00640 1 -- 172.21.15.67:0/2523988553 shutdown_connections 2024-05-30T16:48:31.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.196+0000 7f5be9e00640 1 -- 172.21.15.67:0/2523988553 wait complete. 2024-05-30T16:48:31.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.198+0000 7f5be9e00640 1 Processor -- start 2024-05-30T16:48:31.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.198+0000 7f5be9e00640 1 -- start start 2024-05-30T16:48:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.199+0000 7f5be9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40cf870 0x7f5be415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.199+0000 7f5be9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be40d14d0 0x7f5be414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.199+0000 7f5be9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 0x7f5be415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.199+0000 7f5be9e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5be40d0820 con 0x7f5be415e870 2024-05-30T16:48:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.199+0000 7f5be9e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5be40d0540 con 0x7f5be40d14d0 2024-05-30T16:48:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.199+0000 7f5be9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5be40d4fb0 con 0x7f5be40cf870 2024-05-30T16:48:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.199+0000 7f5be9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 0x7f5be415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.199+0000 7f5be9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 0x7f5be415bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57958/0 (socket says 172.21.15.67:57958) 2024-05-30T16:48:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.199+0000 7f5be9400640 1 -- 172.21.15.67:0/1677847972 learned_addr learned my addr 172.21.15.67:0/1677847972 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be8a00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40cf870 0x7f5be415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be9400640 1 -- 172.21.15.67:0/1677847972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40cf870 msgr2=0x7f5be415a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be9400640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40cf870 0x7f5be415a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be3e00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be40d14d0 0x7f5be414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be9400640 1 -- 172.21.15.67:0/1677847972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be40d14d0 msgr2=0x7f5be414fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be9400640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be40d14d0 0x7f5be414fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be9400640 1 -- 172.21.15.67:0/1677847972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be415c100 con 0x7f5be415e870 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be9400640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 0x7f5be415bbc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00c7b0 tx=0x7f5bdc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be0c00640 1 -- 172.21.15.67:0/1677847972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc003ba0 con 0x7f5be415e870 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be0c00640 1 -- 172.21.15.67:0/1677847972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bdc003d40 con 0x7f5be415e870 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be0c00640 1 -- 172.21.15.67:0/1677847972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc005270 con 0x7f5be415e870 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.200+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be40b5900 con 0x7f5be415e870 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.201+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be40b5ec0 con 0x7f5be415e870 2024-05-30T16:48:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.202+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be40689e0 con 0x7f5be415e870 2024-05-30T16:48:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.203+0000 7f5be0c00640 1 -- 172.21.15.67:0/1677847972 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f5bdc005410 con 0x7f5be415e870 2024-05-30T16:48:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.204+0000 7f5be0c00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5bc406cbc0 0x7f5bc406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.204+0000 7f5be0c00640 1 -- 172.21.15.67:0/1677847972 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f5bdc08be70 con 0x7f5be415e870 2024-05-30T16:48:31.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.204+0000 7f5be8a00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5bc406cbc0 0x7f5bc406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.205+0000 7f5be8a00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5bc406cbc0 0x7f5bc406f080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8034000 tx=0x7f5bd802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:31.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.208+0000 7f5be0c00640 1 -- 172.21.15.67:0/1677847972 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5be40cfd00 con 0x7f5be415e870 2024-05-30T16:48:31.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.412+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5be40ba500 con 0x7f5be415e870 2024-05-30T16:48:31.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.415+0000 7f5be0c00640 1 -- 172.21.15.67:0/1677847972 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16614 (secure 0 0 0) 0x7f5bdc05ee20 con 0x7f5be415e870 2024-05-30T16:48:31.416 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:31.416 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":47,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:48:30.019991+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:48:27.936635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-30T16:48:29.004136+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:48:31.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.417+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5bc406cbc0 msgr2=0x7f5bc406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.417+0000 7f5be9e00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5bc406cbc0 0x7f5bc406f080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8034000 tx=0x7f5bd802e000 comp rx=0 tx=0).stop 2024-05-30T16:48:31.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.417+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 msgr2=0x7f5be415bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.417+0000 7f5be9e00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 0x7f5be415bbc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00c7b0 tx=0x7f5bdc00cbb0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.418+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 shutdown_connections 2024-05-30T16:48:31.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.418+0000 7f5be9e00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5bc406cbc0 0x7f5bc406f080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.418+0000 7f5be9e00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be415e870 0x7f5be415bbc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.418+0000 7f5be9e00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5be40d14d0 0x7f5be414fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.418+0000 7f5be9e00640 1 --2- 172.21.15.67:0/1677847972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5be40cf870 0x7f5be415a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.418+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 >> 172.21.15.67:0/1677847972 conn(0x7f5be408bfc0 msgr2=0x7f5be40b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:31.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.418+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 shutdown_connections 2024-05-30T16:48:31.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.418+0000 7f5be9e00640 1 -- 172.21.15.67:0/1677847972 wait complete. 2024-05-30T16:48:31.438 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-30T16:48:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.636+0000 7f1c8aa00640 1 Processor -- start 2024-05-30T16:48:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.636+0000 7f1c8aa00640 1 -- start start 2024-05-30T16:48:31.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.637+0000 7f1c8aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1c84153510 0x7f1c840b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.637+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c840ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.637+0000 7f1c8aa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c8415d900 con 0x7f1c840ccb90 2024-05-30T16:48:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.637+0000 7f1c8aa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1c8415d6e0 con 0x7f1c84153510 2024-05-30T16:48:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.637+0000 7f1c8aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c841538f0 con 0x7f1c8415d270 2024-05-30T16:48:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.638+0000 7f1c89600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1c84153510 0x7f1c840b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.638+0000 7f1c8a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c840ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.638+0000 7f1c8a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c840ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:40796/0 (socket says 172.21.15.67:40796) 2024-05-30T16:48:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.638+0000 7f1c89600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1c84153510 0x7f1c840b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:47140/0 (socket says 172.21.15.67:47140) 2024-05-30T16:48:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.638+0000 7f1c8a000640 1 -- 172.21.15.67:0/4225421129 learned_addr learned my addr 172.21.15.67:0/4225421129 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.639+0000 7f1c8a000640 1 -- 172.21.15.67:0/4225421129 >> v2:172.21.15.169:3300/0 conn(0x7f1c84153510 msgr2=0x7f1c840b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.639+0000 7f1c8a000640 1 --2- 172.21.15.67:0/4225421129 >> v2:172.21.15.169:3300/0 conn(0x7f1c84153510 0x7f1c840b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.639+0000 7f1c8a000640 1 -- 172.21.15.67:0/4225421129 >> v1:172.21.15.169:6789/0 conn(0x7f1c840ccb90 legacy=0x7f1c8415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.639+0000 7f1c8a000640 1 -- 172.21.15.67:0/4225421129 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c840b3f90 con 0x7f1c8415d270 2024-05-30T16:48:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.640+0000 7f1c8a000640 1 --2- 172.21.15.67:0/4225421129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c840ccff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009e50 tx=0x7f1c7c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f101cccc16556051 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.641+0000 7f1c83e00640 1 -- 172.21.15.67:0/4225421129 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c036070 con 0x7f1c8415d270 2024-05-30T16:48:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c83e00640 1 -- 172.21.15.67:0/4225421129 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c7c031600 con 0x7f1c8415d270 2024-05-30T16:48:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c83e00640 1 -- 172.21.15.67:0/4225421129 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0319d0 con 0x7f1c8415d270 2024-05-30T16:48:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c8aa00640 1 -- 172.21.15.67:0/4225421129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 msgr2=0x7f1c840ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c8aa00640 1 --2- 172.21.15.67:0/4225421129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c840ccff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009e50 tx=0x7f1c7c02a680 comp rx=0 tx=0).stop 2024-05-30T16:48:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c8aa00640 1 -- 172.21.15.67:0/4225421129 shutdown_connections 2024-05-30T16:48:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c8aa00640 1 --2- 172.21.15.67:0/4225421129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c840ccff0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c8aa00640 1 --2- 172.21.15.67:0/4225421129 >> v2:172.21.15.169:3300/0 conn(0x7f1c84153510 0x7f1c840b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c8aa00640 1 -- 172.21.15.67:0/4225421129 >> 172.21.15.67:0/4225421129 conn(0x7f1c8406b900 msgr2=0x7f1c8406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c8aa00640 1 -- 172.21.15.67:0/4225421129 shutdown_connections 2024-05-30T16:48:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.642+0000 7f1c8aa00640 1 -- 172.21.15.67:0/4225421129 wait complete. 2024-05-30T16:48:31.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.644+0000 7f1c8aa00640 1 Processor -- start 2024-05-30T16:48:31.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.645+0000 7f1c8aa00640 1 -- start start 2024-05-30T16:48:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c840ccb90 0x7f1c8414db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c84153510 0x7f1c84072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c8aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c84073010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c8aa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1c840b48c0 con 0x7f1c84153510 2024-05-30T16:48:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c8aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c840b4550 con 0x7f1c840ccb90 2024-05-30T16:48:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c8aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8406e940 con 0x7f1c8415d270 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c88c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c84153510 0x7f1c84072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c88c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c84153510 0x7f1c84072ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57966/0 (socket says 172.21.15.67:57966) 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c88c00640 1 -- 172.21.15.67:0/2788526432 learned_addr learned my addr 172.21.15.67:0/2788526432 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c8a000640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c84073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c89600640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c840ccb90 0x7f1c8414db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c88c00640 1 -- 172.21.15.67:0/2788526432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 msgr2=0x7f1c84073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.646+0000 7f1c88c00640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c84073010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.647+0000 7f1c88c00640 1 -- 172.21.15.67:0/2788526432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c840ccb90 msgr2=0x7f1c8414db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.647+0000 7f1c88c00640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c840ccb90 0x7f1c8414db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.647+0000 7f1c88c00640 1 -- 172.21.15.67:0/2788526432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c841558b0 con 0x7f1c84153510 2024-05-30T16:48:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.647+0000 7f1c88c00640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c84153510 0x7f1c84072ad0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000c400 tx=0x7f1c7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.647+0000 7f1c81600640 1 -- 172.21.15.67:0/2788526432 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c700052d0 con 0x7f1c84153510 2024-05-30T16:48:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.647+0000 7f1c81600640 1 -- 172.21.15.67:0/2788526432 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c70005490 con 0x7f1c84153510 2024-05-30T16:48:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.647+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c84155b90 con 0x7f1c84153510 2024-05-30T16:48:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.647+0000 7f1c81600640 1 -- 172.21.15.67:0/2788526432 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70007260 con 0x7f1c84153510 2024-05-30T16:48:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.647+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c84156150 con 0x7f1c84153510 2024-05-30T16:48:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.649+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c840689e0 con 0x7f1c84153510 2024-05-30T16:48:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.649+0000 7f1c81600640 1 -- 172.21.15.67:0/2788526432 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f1c70007400 con 0x7f1c84153510 2024-05-30T16:48:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.650+0000 7f1c81600640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c6406cc40 0x7f1c6406f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.650+0000 7f1c81600640 1 -- 172.21.15.67:0/2788526432 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f1c7008b340 con 0x7f1c84153510 2024-05-30T16:48:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.651+0000 7f1c89600640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c6406cc40 0x7f1c6406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.651+0000 7f1c89600640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c6406cc40 0x7f1c6406f100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009ad0 tx=0x7f1c78008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.655+0000 7f1c81600640 1 -- 172.21.15.67:0/2788526432 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c700595b0 con 0x7f1c84153510 2024-05-30T16:48:31.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:31.891+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1c840b3c40 con 0x7f1c84153510 2024-05-30T16:48:32.508 INFO:tasks.ceph.mds.a.smithi067.stderr:2024-05-30T16:48:32.506+0000 7f295d400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T16:48:32.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.569+0000 7f1c81600640 1 -- 172.21.15.67:0/2788526432 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f1c70058eb0 con 0x7f1c84153510 2024-05-30T16:48:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.573+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c6406cc40 msgr2=0x7f1c6406f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.573+0000 7f1c8aa00640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c6406cc40 0x7f1c6406f100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009ad0 tx=0x7f1c78008040 comp rx=0 tx=0).stop 2024-05-30T16:48:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c84153510 msgr2=0x7f1c84072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c84153510 0x7f1c84072ad0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000c400 tx=0x7f1c7000c8c0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 shutdown_connections 2024-05-30T16:48:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c6406cc40 0x7f1c6406f100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c8415d270 0x7f1c84073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c84153510 0x7f1c84072ad0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 --2- 172.21.15.67:0/2788526432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c840ccb90 0x7f1c8414db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 >> 172.21.15.67:0/2788526432 conn(0x7f1c8406b900 msgr2=0x7f1c8406f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 shutdown_connections 2024-05-30T16:48:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.574+0000 7f1c8aa00640 1 -- 172.21.15.67:0/2788526432 wait complete. 2024-05-30T16:48:32.596 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-30T16:48:32.596 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-30T16:48:32.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.807+0000 7f82a9c00640 1 Processor -- start 2024-05-30T16:48:32.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.808+0000 7f82a9c00640 1 -- start start 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.809+0000 7f82a9c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f82a40ce1f0 0x7f82a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.809+0000 7f82a9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82a40b3d60 0x7f82a40b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.809+0000 7f82a9c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a40b3580 con 0x7f82a40ce5d0 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.809+0000 7f82a9c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f82a415f490 con 0x7f82a40ce1f0 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.809+0000 7f82a9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f82a415f770 con 0x7f82a40b3d60 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.810+0000 7f82a3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f82a40ce1f0 0x7f82a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.810+0000 7f82a3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f82a40ce1f0 0x7f82a40b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57976/0 (socket says 172.21.15.67:57976) 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.810+0000 7f82a3e00640 1 -- 172.21.15.67:0/410960864 learned_addr learned my addr 172.21.15.67:0/410960864 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.810+0000 7f82a3e00640 1 -- 172.21.15.67:0/410960864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82a40b3d60 msgr2=0x7f82a40b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.810+0000 7f82a3e00640 1 --2- 172.21.15.67:0/410960864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82a40b3d60 0x7f82a40b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.810+0000 7f82a3e00640 1 -- 172.21.15.67:0/410960864 >> v1:172.21.15.67:6789/0 conn(0x7f82a40ce5d0 legacy=0x7f82a415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:32.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.810+0000 7f82a3e00640 1 -- 172.21.15.67:0/410960864 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a415fa50 con 0x7f82a40ce1f0 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.811+0000 7f82a3e00640 1 --2- 172.21.15.67:0/410960864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40ce1f0 0x7f82a40b4dd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8294009870 tx=0x7f829402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e714b520654372 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.812+0000 7f82a3400640 1 -- 172.21.15.67:0/410960864 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294037070 con 0x7f82a40ce1f0 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.812+0000 7f82a3400640 1 -- 172.21.15.67:0/410960864 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8294032440 con 0x7f82a40ce1f0 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.812+0000 7f82a3400640 1 -- 172.21.15.67:0/410960864 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294032810 con 0x7f82a40ce1f0 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.812+0000 7f82a9c00640 1 -- 172.21.15.67:0/410960864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40ce1f0 msgr2=0x7f82a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.812+0000 7f82a9c00640 1 --2- 172.21.15.67:0/410960864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40ce1f0 0x7f82a40b4dd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8294009870 tx=0x7f829402a130 comp rx=0 tx=0).stop 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.812+0000 7f82a9c00640 1 -- 172.21.15.67:0/410960864 shutdown_connections 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.812+0000 7f82a9c00640 1 --2- 172.21.15.67:0/410960864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82a40b3d60 0x7f82a40b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.812+0000 7f82a9c00640 1 --2- 172.21.15.67:0/410960864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40ce1f0 0x7f82a40b4dd0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.812+0000 7f82a9c00640 1 -- 172.21.15.67:0/410960864 >> 172.21.15.67:0/410960864 conn(0x7f82a406b900 msgr2=0x7f82a406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.813+0000 7f82a9c00640 1 -- 172.21.15.67:0/410960864 shutdown_connections 2024-05-30T16:48:32.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.813+0000 7f82a9c00640 1 -- 172.21.15.67:0/410960864 wait complete. 2024-05-30T16:48:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.814+0000 7f82a9c00640 1 Processor -- start 2024-05-30T16:48:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.815+0000 7f82a9c00640 1 -- start start 2024-05-30T16:48:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.815+0000 7f82a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40b3d60 0x7f82a415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a9c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40ce1f0 0x7f82a414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82a40ce5d0 0x7f82a41e9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a9c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f82a40ccd10 con 0x7f82a40b3d60 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a9c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82a40ccb90 con 0x7f82a40ce1f0 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f82a40cce90 con 0x7f82a40ce5d0 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40b3d60 0x7f82a415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40b3d60 0x7f82a415fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57978/0 (socket says 172.21.15.67:57978) 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a3e00640 1 -- 172.21.15.67:0/3834231701 learned_addr learned my addr 172.21.15.67:0/3834231701 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a3e00640 1 -- 172.21.15.67:0/3834231701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82a40ce5d0 msgr2=0x7f82a41e9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a3e00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82a40ce5d0 0x7f82a41e9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a3e00640 1 -- 172.21.15.67:0/3834231701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40ce1f0 msgr2=0x7f82a414d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f829be00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40ce1f0 0x7f82a414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a3e00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40ce1f0 0x7f82a414d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f82a3e00640 1 -- 172.21.15.67:0/3834231701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a40b0ed0 con 0x7f82a40b3d60 2024-05-30T16:48:32.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.816+0000 7f829be00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40ce1f0 0x7f82a414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:32.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.817+0000 7f82a3e00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40b3d60 0x7f82a415fd70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f82940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:32.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.817+0000 7f82a0c00640 1 -- 172.21.15.67:0/3834231701 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294037040 con 0x7f82a40b3d60 2024-05-30T16:48:32.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.817+0000 7f82a0c00640 1 -- 172.21.15.67:0/3834231701 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8294042070 con 0x7f82a40b3d60 2024-05-30T16:48:32.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.817+0000 7f82a0c00640 1 -- 172.21.15.67:0/3834231701 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294030bd0 con 0x7f82a40b3d60 2024-05-30T16:48:32.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.817+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a41e8e40 con 0x7f82a40b3d60 2024-05-30T16:48:32.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.817+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a40b1150 con 0x7f82a40b3d60 2024-05-30T16:48:32.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.818+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a41e9360 con 0x7f82a40b3d60 2024-05-30T16:48:32.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.819+0000 7f82a0c00640 1 -- 172.21.15.67:0/3834231701 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f8294047460 con 0x7f82a40b3d60 2024-05-30T16:48:32.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.821+0000 7f82a0c00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f828006cbc0 0x7f828006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:32.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.821+0000 7f82a0c00640 1 -- 172.21.15.67:0/3834231701 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f82940acbc0 con 0x7f82a40b3d60 2024-05-30T16:48:32.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.821+0000 7f829be00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f828006cbc0 0x7f828006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:32.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.822+0000 7f829be00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f828006cbc0 0x7f828006f080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82a41f7f30 tx=0x7f828c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:32.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:32.826+0000 7f82a0c00640 1 -- 172.21.15.67:0/3834231701 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82a40b0ed0 con 0x7f82a40b3d60 2024-05-30T16:48:33.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.068+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f82a40689e0 con 0x7f82a40b3d60 2024-05-30T16:48:33.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.572+0000 7f82a0c00640 1 -- 172.21.15.67:0/3834231701 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7f8294045360 con 0x7f82a40b3d60 2024-05-30T16:48:33.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f828006cbc0 msgr2=0x7f828006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:33.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f828006cbc0 0x7f828006f080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f82a41f7f30 tx=0x7f828c009040 comp rx=0 tx=0).stop 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40b3d60 msgr2=0x7f82a415fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40b3d60 0x7f82a415fd70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f82940046c0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 shutdown_connections 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f828006cbc0 0x7f828006f080 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82a40ce5d0 0x7f82a41e9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82a40ce1f0 0x7f82a414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 --2- 172.21.15.67:0/3834231701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82a40b3d60 0x7f82a415fd70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.576+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 >> 172.21.15.67:0/3834231701 conn(0x7f82a406b900 msgr2=0x7f82a40ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.577+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 shutdown_connections 2024-05-30T16:48:33.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.577+0000 7f82a9c00640 1 -- 172.21.15.67:0/3834231701 wait complete. 2024-05-30T16:48:33.605 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-30T16:48:33.605 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:48:33.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.817+0000 7f8aa1c00640 1 Processor -- start 2024-05-30T16:48:33.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.817+0000 7f8aa1c00640 1 -- start start 2024-05-30T16:48:33.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.819+0000 7f8aa1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c073310 0x7f8a9c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:33.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.819+0000 7f8aa1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a9c06dc20 0x7f8a9c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:33.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.819+0000 7f8aa1c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a9c0afd60 con 0x7f8a9c15b3a0 2024-05-30T16:48:33.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.819+0000 7f8aa1c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9c06f980 con 0x7f8a9c06dc20 2024-05-30T16:48:33.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.819+0000 7f8aa1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9c0ae0c0 con 0x7f8a9c073310 2024-05-30T16:48:33.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.819+0000 7f8aa1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a9c06dc20 0x7f8a9c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:33.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.820+0000 7f8aa1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a9c06dc20 0x7f8a9c15f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57980/0 (socket says 172.21.15.67:57980) 2024-05-30T16:48:33.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.820+0000 7f8aa1200640 1 -- 172.21.15.67:0/1099925046 learned_addr learned my addr 172.21.15.67:0/1099925046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:33.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.820+0000 7f8a9b400640 1 --2- 172.21.15.67:0/1099925046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c073310 0x7f8a9c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:33.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.820+0000 7f8aa1200640 1 -- 172.21.15.67:0/1099925046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c073310 msgr2=0x7f8a9c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:33.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.820+0000 7f8aa1200640 1 --2- 172.21.15.67:0/1099925046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c073310 0x7f8a9c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.820+0000 7f8aa1200640 1 -- 172.21.15.67:0/1099925046 >> v1:172.21.15.169:6789/0 conn(0x7f8a9c15b3a0 legacy=0x7f8a9c072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:33.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.820+0000 7f8aa1200640 1 -- 172.21.15.67:0/1099925046 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c0af7a0 con 0x7f8a9c06dc20 2024-05-30T16:48:33.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.821+0000 7f8aa1200640 1 --2- 172.21.15.67:0/1099925046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 0x7f8a9c15f9a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009e50 tx=0x7f8a9402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fc8ff540ac99c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:33.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.822+0000 7f8a9aa00640 1 -- 172.21.15.67:0/1099925046 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94036070 con 0x7f8a9c06dc20 2024-05-30T16:48:33.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.822+0000 7f8a9aa00640 1 -- 172.21.15.67:0/1099925046 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a94031600 con 0x7f8a9c06dc20 2024-05-30T16:48:33.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.822+0000 7f8a9aa00640 1 -- 172.21.15.67:0/1099925046 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a940319d0 con 0x7f8a9c06dc20 2024-05-30T16:48:33.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.822+0000 7f8aa1c00640 1 -- 172.21.15.67:0/1099925046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 msgr2=0x7f8a9c15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:33.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.822+0000 7f8aa1c00640 1 --2- 172.21.15.67:0/1099925046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 0x7f8a9c15f9a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009e50 tx=0x7f8a9402a710 comp rx=0 tx=0).stop 2024-05-30T16:48:33.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.823+0000 7f8aa1c00640 1 -- 172.21.15.67:0/1099925046 shutdown_connections 2024-05-30T16:48:33.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.823+0000 7f8aa1c00640 1 --2- 172.21.15.67:0/1099925046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 0x7f8a9c15f9a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.823+0000 7f8aa1c00640 1 --2- 172.21.15.67:0/1099925046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c073310 0x7f8a9c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.823+0000 7f8aa1c00640 1 -- 172.21.15.67:0/1099925046 >> 172.21.15.67:0/1099925046 conn(0x7f8a9c0b1fc0 msgr2=0x7f8a9c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:33.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.823+0000 7f8aa1c00640 1 -- 172.21.15.67:0/1099925046 shutdown_connections 2024-05-30T16:48:33.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.823+0000 7f8aa1c00640 1 -- 172.21.15.67:0/1099925046 wait complete. 2024-05-30T16:48:33.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.825+0000 7f8aa1c00640 1 Processor -- start 2024-05-30T16:48:33.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.825+0000 7f8aa1c00640 1 -- start start 2024-05-30T16:48:33.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.826+0000 7f8aa1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 0x7f8a9c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:33.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.826+0000 7f8aa1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9c073310 0x7f8a9c150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:33.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.826+0000 7f8aa1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c15b3a0 0x7f8a9c153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:33.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.826+0000 7f8aa1c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c15b780 con 0x7f8a9c06dc20 2024-05-30T16:48:33.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.826+0000 7f8aa1c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c15fee0 con 0x7f8a9c073310 2024-05-30T16:48:33.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.826+0000 7f8aa1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9c0ae550 con 0x7f8a9c15b3a0 2024-05-30T16:48:33.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.826+0000 7f8a9be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 0x7f8a9c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:33.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 0x7f8a9c0cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57996/0 (socket says 172.21.15.67:57996) 2024-05-30T16:48:33.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9be00640 1 -- 172.21.15.67:0/3562958470 learned_addr learned my addr 172.21.15.67:0/3562958470 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9b400640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9c073310 0x7f8a9c150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8aa1200640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c15b3a0 0x7f8a9c153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9be00640 1 -- 172.21.15.67:0/3562958470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c15b3a0 msgr2=0x7f8a9c153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9be00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c15b3a0 0x7f8a9c153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9be00640 1 -- 172.21.15.67:0/3562958470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9c073310 msgr2=0x7f8a9c150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9be00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9c073310 0x7f8a9c150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9be00640 1 -- 172.21.15.67:0/3562958470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c153f50 con 0x7f8a9c06dc20 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9b400640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9c073310 0x7f8a9c150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.827+0000 7f8a9be00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 0x7f8a9c0cd740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c400 tx=0x7f8a8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.828+0000 7f8a93e00640 1 -- 172.21.15.67:0/3562958470 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c017070 con 0x7f8a9c06dc20 2024-05-30T16:48:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.828+0000 7f8a93e00640 1 -- 172.21.15.67:0/3562958470 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8c0052d0 con 0x7f8a9c06dc20 2024-05-30T16:48:33.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.828+0000 7f8a93e00640 1 -- 172.21.15.67:0/3562958470 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c003830 con 0x7f8a9c06dc20 2024-05-30T16:48:33.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.828+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9c154230 con 0x7f8a9c06dc20 2024-05-30T16:48:33.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.828+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9c1f8d40 con 0x7f8a9c06dc20 2024-05-30T16:48:33.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.829+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9c0689e0 con 0x7f8a9c06dc20 2024-05-30T16:48:33.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.830+0000 7f8a93e00640 1 -- 172.21.15.67:0/3562958470 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f8a8c01a070 con 0x7f8a9c06dc20 2024-05-30T16:48:33.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.831+0000 7f8a93e00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a7c06cbf0 0x7f8a7c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:33.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.832+0000 7f8a93e00640 1 -- 172.21.15.67:0/3562958470 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f8a8c08b360 con 0x7f8a9c06dc20 2024-05-30T16:48:33.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.832+0000 7f8a9b400640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a7c06cbf0 0x7f8a7c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:33.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.832+0000 7f8a9b400640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a7c06cbf0 0x7f8a7c06f0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0b4fe0 tx=0x7f8a88008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:33.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:33.836+0000 7f8a93e00640 1 -- 172.21.15.67:0/3562958470 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a8c0593b0 con 0x7f8a9c06dc20 2024-05-30T16:48:34.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.087+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a9c0b2e90 con 0x7f8a9c06dc20 2024-05-30T16:48:34.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.087+0000 7f8a93e00640 1 -- 172.21.15.67:0/3562958470 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5184 (secure 0 0 0) 0x7f8a8c058cb0 con 0x7f8a9c06dc20 2024-05-30T16:48:34.090 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:34.090 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2139720967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2139720967},{"type":"v1","addr":"172.21.15.169:6834","nonce":2139720967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4981,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2317664228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2317664228},{"type":"v1","addr":"172.21.15.169:6835","nonce":2317664228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/350920637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":350920637},{"type":"v1","addr":"172.21.15.67:6835","nonce":350920637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:48:30.019937+0000","modified":"2024-05-30T16:48:33.071363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/3595714006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3595714006},{"type":"v1","addr":"172.21.15.67:6837","nonce":3595714006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5361,"qdb_cluster":[5361]},"id":3}]} 2024-05-30T16:48:34.090 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 25 2024-05-30T16:48:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.090+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a7c06cbf0 msgr2=0x7f8a7c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.090+0000 7f8aa1c00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a7c06cbf0 0x7f8a7c06f0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0b4fe0 tx=0x7f8a88008040 comp rx=0 tx=0).stop 2024-05-30T16:48:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.090+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 msgr2=0x7f8a9c0cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.090+0000 7f8aa1c00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 0x7f8a9c0cd740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c400 tx=0x7f8a8c00c8c0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.090+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 shutdown_connections 2024-05-30T16:48:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.090+0000 7f8aa1c00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a7c06cbf0 0x7f8a7c06f0b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.090+0000 7f8aa1c00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9c15b3a0 0x7f8a9c153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.090+0000 7f8aa1c00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9c073310 0x7f8a9c150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.090+0000 7f8aa1c00640 1 --2- 172.21.15.67:0/3562958470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a9c06dc20 0x7f8a9c0cd740 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.091+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 >> 172.21.15.67:0/3562958470 conn(0x7f8a9c0b1fc0 msgr2=0x7f8a9c0ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.091+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 shutdown_connections 2024-05-30T16:48:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.091+0000 7f8aa1c00640 1 -- 172.21.15.67:0/3562958470 wait complete. 2024-05-30T16:48:34.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:48:34.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.327+0000 7f8c57e00640 1 Processor -- start 2024-05-30T16:48:34.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.328+0000 7f8c57e00640 1 -- start start 2024-05-30T16:48:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.329+0000 7f8c57e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8c5815e870 0x7f8c580d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.329+0000 7f8c57e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8c580d14d0 0x7f8c580cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.329+0000 7f8c57e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cf870 0x7f8c58162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.329+0000 7f8c57e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f8c580d2150 con 0x7f8c580d14d0 2024-05-30T16:48:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.329+0000 7f8c57e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8c580cfd00 con 0x7f8c5815e870 2024-05-30T16:48:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.329+0000 7f8c57e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5815ec50 con 0x7f8c580cf870 2024-05-30T16:48:34.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.329+0000 7f8c56000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8c580d14d0 0x7f8c580cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.330+0000 7f8c56000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8c580d14d0 0x7f8c580cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57998/0 (socket says 172.21.15.67:57998) 2024-05-30T16:48:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.330+0000 7f8c56000640 1 -- 172.21.15.67:0/231178465 learned_addr learned my addr 172.21.15.67:0/231178465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.330+0000 7f8c56a00640 1 --2- 172.21.15.67:0/231178465 >> v2:172.21.15.169:3300/0 conn(0x7f8c5815e870 0x7f8c580d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.330+0000 7f8c57400640 1 --2- 172.21.15.67:0/231178465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cf870 0x7f8c58162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.330+0000 7f8c56000640 1 -- 172.21.15.67:0/231178465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cf870 msgr2=0x7f8c58162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.330+0000 7f8c56000640 1 --2- 172.21.15.67:0/231178465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cf870 0x7f8c58162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.330+0000 7f8c56000640 1 -- 172.21.15.67:0/231178465 >> v2:172.21.15.169:3300/0 conn(0x7f8c5815e870 msgr2=0x7f8c580d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.330+0000 7f8c56000640 1 --2- 172.21.15.67:0/231178465 >> v2:172.21.15.169:3300/0 conn(0x7f8c5815e870 0x7f8c580d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.330+0000 7f8c56000640 1 -- 172.21.15.67:0/231178465 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c580cff80 con 0x7f8c580d14d0 2024-05-30T16:48:34.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.331+0000 7f8c56000640 1 --2- 172.21.15.67:0/231178465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 0x7f8c580cf330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8c5808e720 tx=0x7f8c4002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ee93e07aa60732 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c55600640 1 -- 172.21.15.67:0/231178465 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40036070 con 0x7f8c580d14d0 2024-05-30T16:48:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c55600640 1 -- 172.21.15.67:0/231178465 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c40031600 con 0x7f8c580d14d0 2024-05-30T16:48:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c55600640 1 -- 172.21.15.67:0/231178465 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c400319d0 con 0x7f8c580d14d0 2024-05-30T16:48:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c57e00640 1 -- 172.21.15.67:0/231178465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 msgr2=0x7f8c580cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c57e00640 1 --2- 172.21.15.67:0/231178465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 0x7f8c580cf330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8c5808e720 tx=0x7f8c4002a6f0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c57e00640 1 -- 172.21.15.67:0/231178465 shutdown_connections 2024-05-30T16:48:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c57e00640 1 --2- 172.21.15.67:0/231178465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 0x7f8c580cf330 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c57e00640 1 --2- 172.21.15.67:0/231178465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cf870 0x7f8c58162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c57e00640 1 --2- 172.21.15.67:0/231178465 >> v2:172.21.15.169:3300/0 conn(0x7f8c5815e870 0x7f8c580d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c57e00640 1 -- 172.21.15.67:0/231178465 >> 172.21.15.67:0/231178465 conn(0x7f8c5808bfc0 msgr2=0x7f8c580d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c57e00640 1 -- 172.21.15.67:0/231178465 shutdown_connections 2024-05-30T16:48:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.332+0000 7f8c57e00640 1 -- 172.21.15.67:0/231178465 wait complete. 2024-05-30T16:48:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.334+0000 7f8c57e00640 1 Processor -- start 2024-05-30T16:48:34.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.334+0000 7f8c57e00640 1 -- start start 2024-05-30T16:48:34.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.335+0000 7f8c57e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c580cf870 0x7f8c581509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.335+0000 7f8c57e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 0x7f8c5815f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.335+0000 7f8c57e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c5815e870 0x7f8c580b9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.335+0000 7f8c57e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8c580d0820 con 0x7f8c580d14d0 2024-05-30T16:48:34.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.335+0000 7f8c57e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c580d0540 con 0x7f8c580cf870 2024-05-30T16:48:34.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.335+0000 7f8c57e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8c580d4fb0 con 0x7f8c5815e870 2024-05-30T16:48:34.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c56000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 0x7f8c5815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c56000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 0x7f8c5815f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58008/0 (socket says 172.21.15.67:58008) 2024-05-30T16:48:34.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c56000640 1 -- 172.21.15.67:0/2904108101 learned_addr learned my addr 172.21.15.67:0/2904108101 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:34.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c56a00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c580cf870 0x7f8c581509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c57400640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c5815e870 0x7f8c580b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c56000640 1 -- 172.21.15.67:0/2904108101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c5815e870 msgr2=0x7f8c580b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c56000640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c5815e870 0x7f8c580b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c56000640 1 -- 172.21.15.67:0/2904108101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c580cf870 msgr2=0x7f8c581509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c56000640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c580cf870 0x7f8c581509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.336+0000 7f8c56000640 1 -- 172.21.15.67:0/2904108101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c58150f20 con 0x7f8c580d14d0 2024-05-30T16:48:34.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.337+0000 7f8c56000640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 0x7f8c5815f870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009e70 tx=0x7f8c40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:34.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.337+0000 7f8c4ea00640 1 -- 172.21.15.67:0/2904108101 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40036050 con 0x7f8c580d14d0 2024-05-30T16:48:34.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.337+0000 7f8c4ea00640 1 -- 172.21.15.67:0/2904108101 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c40042070 con 0x7f8c580d14d0 2024-05-30T16:48:34.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.337+0000 7f8c4ea00640 1 -- 172.21.15.67:0/2904108101 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40009510 con 0x7f8c580d14d0 2024-05-30T16:48:34.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.337+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c58160170 con 0x7f8c580d14d0 2024-05-30T16:48:34.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.337+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c58160730 con 0x7f8c580d14d0 2024-05-30T16:48:34.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.338+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c580689e0 con 0x7f8c580d14d0 2024-05-30T16:48:34.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.340+0000 7f8c4ea00640 1 -- 172.21.15.67:0/2904108101 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f8c40007c70 con 0x7f8c580d14d0 2024-05-30T16:48:34.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.340+0000 7f8c4ea00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406cbc0 0x7f8c3406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.341+0000 7f8c4ea00640 1 -- 172.21.15.67:0/2904108101 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f8c400acf30 con 0x7f8c580d14d0 2024-05-30T16:48:34.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.341+0000 7f8c56a00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406cbc0 0x7f8c3406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.342+0000 7f8c56a00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406cbc0 0x7f8c3406f080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006fd0 tx=0x7f8c48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:34.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.345+0000 7f8c4ea00640 1 -- 172.21.15.67:0/2904108101 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c40045360 con 0x7f8c580d14d0 2024-05-30T16:48:34.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.555+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c580b24c0 con 0x7f8c580d14d0 2024-05-30T16:48:34.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.558+0000 7f8c4ea00640 1 -- 172.21.15.67:0/2904108101 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16601 (secure 0 0 0) 0x7f8c4007af50 con 0x7f8c580d14d0 2024-05-30T16:48:34.560 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:34.560 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":48,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:48:31.548557+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:48:27.936635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-30T16:48:29.004136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:48:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.560+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406cbc0 msgr2=0x7f8c3406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.560+0000 7f8c57e00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406cbc0 0x7f8c3406f080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006fd0 tx=0x7f8c48008040 comp rx=0 tx=0).stop 2024-05-30T16:48:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.560+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 msgr2=0x7f8c5815f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.560+0000 7f8c57e00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 0x7f8c5815f870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009e70 tx=0x7f8c40004a40 comp rx=0 tx=0).stop 2024-05-30T16:48:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.561+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 shutdown_connections 2024-05-30T16:48:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.561+0000 7f8c57e00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406cbc0 0x7f8c3406f080 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.561+0000 7f8c57e00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c5815e870 0x7f8c580b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.561+0000 7f8c57e00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580d14d0 0x7f8c5815f870 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.561+0000 7f8c57e00640 1 --2- 172.21.15.67:0/2904108101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c580cf870 0x7f8c581509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.561+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 >> 172.21.15.67:0/2904108101 conn(0x7f8c5808bfc0 msgr2=0x7f8c580b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.561+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 shutdown_connections 2024-05-30T16:48:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.561+0000 7f8c57e00640 1 -- 172.21.15.67:0/2904108101 wait complete. 2024-05-30T16:48:34.581 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T16:48:34.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:48:34.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.772+0000 7f9e95a00640 1 Processor -- start 2024-05-30T16:48:34.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.773+0000 7f9e95a00640 1 -- start start 2024-05-30T16:48:34.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.775+0000 7f9e95a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e900cfa40 0x7f9e90169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.775+0000 7f9e95a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e9015ec50 con 0x7f9e900d16a0 2024-05-30T16:48:34.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.775+0000 7f9e95a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e900cff80 con 0x7f9e9015e870 2024-05-30T16:48:34.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.775+0000 7f9e95a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9e900d2150 con 0x7f9e900cfa40 2024-05-30T16:48:34.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.775+0000 7f9e87e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9e900d16a0 0x7f9e900cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52158/0 (socket says 172.21.15.67:52158) 2024-05-30T16:48:34.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.775+0000 7f9e87e00640 1 -- 172.21.15.67:0/2779743343 learned_addr learned my addr 172.21.15.67:0/2779743343 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.775+0000 7f9e95000640 1 --2- 172.21.15.67:0/2779743343 >> v2:172.21.15.169:3300/0 conn(0x7f9e900cfa40 0x7f9e90169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.775+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561646692 0 0) 0x7f9e9015ec50 con 0x7f9e900d16a0 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.775+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e74003520 con 0x7f9e900d16a0 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3800322087 0 0) 0x7f9e74003520 con 0x7f9e900d16a0 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 >> v2:172.21.15.169:3300/0 conn(0x7f9e900cfa40 msgr2=0x7f9e90169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 --2- 172.21.15.67:0/2779743343 >> v2:172.21.15.169:3300/0 conn(0x7f9e900cfa40 0x7f9e90169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 >> v1:172.21.15.169:6789/0 conn(0x7f9e9015e870 legacy=0x7f9e900d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e900d0260 con 0x7f9e900d16a0 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9e780017f0 con 0x7f9e900d16a0 2024-05-30T16:48:34.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 >> v1:172.21.15.67:6789/0 conn(0x7f9e900d16a0 legacy=0x7f9e900cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740039b0 0x7f9e74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e740051c0 0x7f9e7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e900d2150 con 0x7f9e74004800 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7400c060 con 0x7f9e740051c0 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7400c260 con 0x7f9e740039b0 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e95000640 1 --2- 172.21.15.67:0/2779743343 >> v2:172.21.15.169:3300/0 conn(0x7f9e900cfa40 0x7f9e90169690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.776+0000 7f9e87e00640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.777+0000 7f9e8fe00640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740039b0 0x7f9e74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.777+0000 7f9e87e00640 1 -- 172.21.15.67:0/2779743343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740039b0 msgr2=0x7f9e74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.777+0000 7f9e95000640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e740051c0 0x7f9e7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.777+0000 7f9e87e00640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740039b0 0x7f9e74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.777+0000 7f9e87e00640 1 -- 172.21.15.67:0/2779743343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e740051c0 msgr2=0x7f9e7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.777+0000 7f9e87e00640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e740051c0 0x7f9e7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.777+0000 7f9e87e00640 1 -- 172.21.15.67:0/2779743343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e74003520 con 0x7f9e74004800 2024-05-30T16:48:34.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.777+0000 7f9e95000640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e740051c0 0x7f9e7400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:34.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.777+0000 7f9e95000640 1 -- 172.21.15.67:0/2779743343 reap_dead start 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.778+0000 7f9e87e00640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e74004c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800c210 tx=0x7f9e74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfffc81a6358394b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.778+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78007d20 con 0x7f9e74004800 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.778+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e78007270 con 0x7f9e74004800 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.778+0000 7f9e8f400640 1 -- 172.21.15.67:0/2779743343 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e780057a0 con 0x7f9e74004800 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.779+0000 7f9e95a00640 1 -- 172.21.15.67:0/2779743343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 msgr2=0x7f9e74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.779+0000 7f9e95a00640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e74004c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800c210 tx=0x7f9e74005830 comp rx=0 tx=0).stop 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.779+0000 7f9e95a00640 1 -- 172.21.15.67:0/2779743343 shutdown_connections 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.779+0000 7f9e95a00640 1 --2- 172.21.15.67:0/2779743343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e74004c80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.779+0000 7f9e95a00640 1 -- 172.21.15.67:0/2779743343 >> 172.21.15.67:0/2779743343 conn(0x7f9e9008bfc0 msgr2=0x7f9e900d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.779+0000 7f9e95a00640 1 -- 172.21.15.67:0/2779743343 shutdown_connections 2024-05-30T16:48:34.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.779+0000 7f9e95a00640 1 -- 172.21.15.67:0/2779743343 wait complete. 2024-05-30T16:48:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.781+0000 7f9e95a00640 1 Processor -- start 2024-05-30T16:48:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.781+0000 7f9e95a00640 1 -- start start 2024-05-30T16:48:34.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.782+0000 7f9e95a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e900cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.782+0000 7f9e95a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e900cccd0 0x7f9e901514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.782+0000 7f9e95a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e901519e0 0x7f9e900b9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.782+0000 7f9e95a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e900d4fb0 con 0x7f9e74004800 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.782+0000 7f9e95a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e900d0820 con 0x7f9e900cccd0 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.782+0000 7f9e95a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e900cf330 con 0x7f9e901519e0 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.782+0000 7f9e8fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e900cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.782+0000 7f9e8fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e900cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58032/0 (socket says 172.21.15.67:58032) 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.782+0000 7f9e8fe00640 1 -- 172.21.15.67:0/3220030329 learned_addr learned my addr 172.21.15.67:0/3220030329 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e87e00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e900cccd0 0x7f9e901514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e8fe00640 1 -- 172.21.15.67:0/3220030329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e901519e0 msgr2=0x7f9e900b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e8fe00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e901519e0 0x7f9e900b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e8fe00640 1 -- 172.21.15.67:0/3220030329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e900cccd0 msgr2=0x7f9e901514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e8fe00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e900cccd0 0x7f9e901514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e8fe00640 1 -- 172.21.15.67:0/3220030329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e901512b0 con 0x7f9e74004800 2024-05-30T16:48:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e87e00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e900cccd0 0x7f9e901514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e8fe00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e900cc790 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000e450 tx=0x7f9e8000e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e8cc00640 1 -- 172.21.15.67:0/3220030329 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e800052e0 con 0x7f9e74004800 2024-05-30T16:48:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e8cc00640 1 -- 172.21.15.67:0/3220030329 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e80018070 con 0x7f9e74004800 2024-05-30T16:48:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e8cc00640 1 -- 172.21.15.67:0/3220030329 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80003810 con 0x7f9e74004800 2024-05-30T16:48:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.783+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e900b74a0 con 0x7f9e74004800 2024-05-30T16:48:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.784+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e900cb9d0 con 0x7f9e74004800 2024-05-30T16:48:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.785+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e900689e0 con 0x7f9e74004800 2024-05-30T16:48:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.786+0000 7f9e8cc00640 1 -- 172.21.15.67:0/3220030329 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f9e800039b0 con 0x7f9e74004800 2024-05-30T16:48:34.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.787+0000 7f9e8cc00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e6c06cbc0 0x7f9e6c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:34.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.787+0000 7f9e8cc00640 1 -- 172.21.15.67:0/3220030329 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f9e8008bb60 con 0x7f9e74004800 2024-05-30T16:48:34.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.787+0000 7f9e87e00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e6c06cbc0 0x7f9e6c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:34.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.788+0000 7f9e87e00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e6c06cbc0 0x7f9e6c06f080 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9e78032000 tx=0x7f9e7802d000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.791+0000 7f9e8cc00640 1 -- 172.21.15.67:0/3220030329 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e80059cd0 con 0x7f9e74004800 2024-05-30T16:48:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:34.993+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e900b2350 con 0x7f9e6c06cbc0 2024-05-30T16:48:35.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.010+0000 7f9e8cc00640 1 -- 172.21.15.67:0/3220030329 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247387 (secure 0 0 0) 0x7f9e900b2350 con 0x7f9e6c06cbc0 2024-05-30T16:48:35.014 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:35.017 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.016+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e6c06cbc0 msgr2=0x7f9e6c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.016+0000 7f9e95a00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e6c06cbc0 0x7f9e6c06f080 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9e78032000 tx=0x7f9e7802d000 comp rx=0 tx=0).stop 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 msgr2=0x7f9e900cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e900cc790 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000e450 tx=0x7f9e8000e910 comp rx=0 tx=0).stop 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 shutdown_connections 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e6c06cbc0 0x7f9e6c06f080 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e901519e0 0x7f9e900b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e900cccd0 0x7f9e901514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 --2- 172.21.15.67:0/3220030329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e74004800 0x7f9e900cc790 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 >> 172.21.15.67:0/3220030329 conn(0x7f9e9008bfc0 msgr2=0x7f9e900b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 shutdown_connections 2024-05-30T16:48:35.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.017+0000 7f9e95a00640 1 -- 172.21.15.67:0/3220030329 wait complete. 2024-05-30T16:48:35.039 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-05-30T16:48:34.086725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279736,"kb_used_data":5316,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754694984,"statfs":{"total":773094113280,"available":772807663616,"internally_reserved":0,"allocated":5443584,"data_stored":2898471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.572212"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570382+0000","last_change":"2024-05-30T16:48:30.584563+0000","last_active":"2024-05-30T16:48:31.570382+0000","last_peered":"2024-05-30T16:48:31.570382+0000","last_clean":"2024-05-30T16:48:31.570382+0000","last_became_active":"2024-05-30T16:48:30.584262+0000","last_became_peered":"2024-05-30T16:48:30.584262+0000","last_unstale":"2024-05-30T16:48:31.570382+0000","last_undegraded":"2024-05-30T16:48:31.570382+0000","last_fullsized":"2024-05-30T16:48:31.570382+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.553111+0000","last_change":"2024-05-30T16:48:29.553111+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.553111+0000","last_undegraded":"2024-05-30T16:48:29.553111+0000","last_fullsized":"2024-05-30T16:48:29.553111+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603004+0000","last_change":"2024-05-30T16:48:30.564821+0000","last_active":"2024-05-30T16:48:31.603004+0000","last_peered":"2024-05-30T16:48:31.603004+0000","last_clean":"2024-05-30T16:48:31.603004+0000","last_became_active":"2024-05-30T16:48:30.564537+0000","last_became_peered":"2024-05-30T16:48:30.564537+0000","last_unstale":"2024-05-30T16:48:31.603004+0000","last_undegraded":"2024-05-30T16:48:31.603004+0000","last_fullsized":"2024-05-30T16:48:31.603004+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:00:29.368856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570614+0000","last_change":"2024-05-30T16:48:30.565013+0000","last_active":"2024-05-30T16:48:31.570614+0000","last_peered":"2024-05-30T16:48:31.570614+0000","last_clean":"2024-05-30T16:48:31.570614+0000","last_became_active":"2024-05-30T16:48:30.564344+0000","last_became_peered":"2024-05-30T16:48:30.564344+0000","last_unstale":"2024-05-30T16:48:31.570614+0000","last_undegraded":"2024-05-30T16:48:31.570614+0000","last_fullsized":"2024-05-30T16:48:31.570614+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:38.774999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.542049+0000","last_change":"2024-05-30T16:48:29.542049+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.542049+0000","last_undegraded":"2024-05-30T16:48:29.542049+0000","last_fullsized":"2024-05-30T16:48:29.542049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600493+0000","last_change":"2024-05-30T16:48:30.561287+0000","last_active":"2024-05-30T16:48:31.600493+0000","last_peered":"2024-05-30T16:48:31.600493+0000","last_clean":"2024-05-30T16:48:31.600493+0000","last_became_active":"2024-05-30T16:48:30.560737+0000","last_became_peered":"2024-05-30T16:48:30.560737+0000","last_unstale":"2024-05-30T16:48:31.600493+0000","last_undegraded":"2024-05-30T16:48:31.600493+0000","last_fullsized":"2024-05-30T16:48:31.600493+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:50:31.061295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603257+0000","last_change":"2024-05-30T16:48:30.587362+0000","last_active":"2024-05-30T16:48:31.603257+0000","last_peered":"2024-05-30T16:48:31.603257+0000","last_clean":"2024-05-30T16:48:31.603257+0000","last_became_active":"2024-05-30T16:48:30.587100+0000","last_became_peered":"2024-05-30T16:48:30.587100+0000","last_unstale":"2024-05-30T16:48:31.603257+0000","last_undegraded":"2024-05-30T16:48:31.603257+0000","last_fullsized":"2024-05-30T16:48:31.603257+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:00.811234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603466+0000","last_change":"2024-05-30T16:48:30.582474+0000","last_active":"2024-05-30T16:48:31.603466+0000","last_peered":"2024-05-30T16:48:31.603466+0000","last_clean":"2024-05-30T16:48:31.603466+0000","last_became_active":"2024-05-30T16:48:30.582148+0000","last_became_peered":"2024-05-30T16:48:30.582148+0000","last_unstale":"2024-05-30T16:48:31.603466+0000","last_undegraded":"2024-05-30T16:48:31.603466+0000","last_fullsized":"2024-05-30T16:48:31.603466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602911+0000","last_change":"2024-05-30T16:48:30.577384+0000","last_active":"2024-05-30T16:48:31.602911+0000","last_peered":"2024-05-30T16:48:31.602911+0000","last_clean":"2024-05-30T16:48:31.602911+0000","last_became_active":"2024-05-30T16:48:30.570714+0000","last_became_peered":"2024-05-30T16:48:30.570714+0000","last_unstale":"2024-05-30T16:48:31.602911+0000","last_undegraded":"2024-05-30T16:48:31.602911+0000","last_fullsized":"2024-05-30T16:48:31.602911+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:05.025187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570620+0000","last_change":"2024-05-30T16:48:30.582165+0000","last_active":"2024-05-30T16:48:31.570620+0000","last_peered":"2024-05-30T16:48:31.570620+0000","last_clean":"2024-05-30T16:48:31.570620+0000","last_became_active":"2024-05-30T16:48:30.581896+0000","last_became_peered":"2024-05-30T16:48:30.581896+0000","last_unstale":"2024-05-30T16:48:31.570620+0000","last_undegraded":"2024-05-30T16:48:31.570620+0000","last_fullsized":"2024-05-30T16:48:31.570620+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.538253+0000","last_change":"2024-05-30T16:48:29.538253+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.538253+0000","last_undegraded":"2024-05-30T16:48:29.538253+0000","last_fullsized":"2024-05-30T16:48:29.538253+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601441+0000","last_change":"2024-05-30T16:48:30.555321+0000","last_active":"2024-05-30T16:48:31.601441+0000","last_peered":"2024-05-30T16:48:31.601441+0000","last_clean":"2024-05-30T16:48:31.601441+0000","last_became_active":"2024-05-30T16:48:30.555012+0000","last_became_peered":"2024-05-30T16:48:30.555012+0000","last_unstale":"2024-05-30T16:48:31.601441+0000","last_undegraded":"2024-05-30T16:48:31.601441+0000","last_fullsized":"2024-05-30T16:48:31.601441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:41.388387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.543234+0000","last_change":"2024-05-30T16:48:29.543234+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.543234+0000","last_undegraded":"2024-05-30T16:48:29.543234+0000","last_fullsized":"2024-05-30T16:48:29.543234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.560080+0000","last_change":"2024-05-30T16:48:29.560080+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.560080+0000","last_undegraded":"2024-05-30T16:48:29.560080+0000","last_fullsized":"2024-05-30T16:48:29.560080+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.552921+0000","last_change":"2024-05-30T16:48:29.552921+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.552921+0000","last_undegraded":"2024-05-30T16:48:29.552921+0000","last_fullsized":"2024-05-30T16:48:29.552921+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.561708+0000","last_change":"2024-05-30T16:48:29.561708+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.561708+0000","last_undegraded":"2024-05-30T16:48:29.561708+0000","last_fullsized":"2024-05-30T16:48:29.561708+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.555816+0000","last_change":"2024-05-30T16:48:29.555816+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.555816+0000","last_undegraded":"2024-05-30T16:48:29.555816+0000","last_fullsized":"2024-05-30T16:48:29.555816+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.540205+0000","last_change":"2024-05-30T16:48:29.540205+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.540205+0000","last_undegraded":"2024-05-30T16:48:29.540205+0000","last_fullsized":"2024-05-30T16:48:29.540205+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.541235+0000","last_change":"2024-05-30T16:48:29.541235+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.541235+0000","last_undegraded":"2024-05-30T16:48:29.541235+0000","last_fullsized":"2024-05-30T16:48:29.541235+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601819+0000","last_change":"2024-05-30T16:48:30.563868+0000","last_active":"2024-05-30T16:48:31.601819+0000","last_peered":"2024-05-30T16:48:31.601819+0000","last_clean":"2024-05-30T16:48:31.601819+0000","last_became_active":"2024-05-30T16:48:30.563554+0000","last_became_peered":"2024-05-30T16:48:30.563554+0000","last_unstale":"2024-05-30T16:48:31.601819+0000","last_undegraded":"2024-05-30T16:48:31.601819+0000","last_fullsized":"2024-05-30T16:48:31.601819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:47.906916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.547433+0000","last_change":"2024-05-30T16:48:29.547433+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.547433+0000","last_undegraded":"2024-05-30T16:48:29.547433+0000","last_fullsized":"2024-05-30T16:48:29.547433+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.548367+0000","last_change":"2024-05-30T16:48:29.548367+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.548367+0000","last_undegraded":"2024-05-30T16:48:29.548367+0000","last_fullsized":"2024-05-30T16:48:29.548367+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.565566+0000","last_change":"2024-05-30T16:48:29.565566+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.565566+0000","last_undegraded":"2024-05-30T16:48:29.565566+0000","last_fullsized":"2024-05-30T16:48:29.565566+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.541213+0000","last_change":"2024-05-30T16:48:29.541213+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.541213+0000","last_undegraded":"2024-05-30T16:48:29.541213+0000","last_fullsized":"2024-05-30T16:48:29.541213+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.552035+0000","last_change":"2024-05-30T16:48:29.552035+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.552035+0000","last_undegraded":"2024-05-30T16:48:29.552035+0000","last_fullsized":"2024-05-30T16:48:29.552035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571075+0000","last_change":"2024-05-30T16:48:30.582495+0000","last_active":"2024-05-30T16:48:31.571075+0000","last_peered":"2024-05-30T16:48:31.571075+0000","last_clean":"2024-05-30T16:48:31.571075+0000","last_became_active":"2024-05-30T16:48:30.582199+0000","last_became_peered":"2024-05-30T16:48:30.582199+0000","last_unstale":"2024-05-30T16:48:31.571075+0000","last_undegraded":"2024-05-30T16:48:31.571075+0000","last_fullsized":"2024-05-30T16:48:31.571075+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:18.447770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.556706+0000","last_change":"2024-05-30T16:48:29.556706+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.556706+0000","last_undegraded":"2024-05-30T16:48:29.556706+0000","last_fullsized":"2024-05-30T16:48:29.556706+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.560620+0000","last_change":"2024-05-30T16:48:29.560620+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.560620+0000","last_undegraded":"2024-05-30T16:48:29.560620+0000","last_fullsized":"2024-05-30T16:48:29.560620+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.542690+0000","last_change":"2024-05-30T16:48:29.542690+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.542690+0000","last_undegraded":"2024-05-30T16:48:29.542690+0000","last_fullsized":"2024-05-30T16:48:29.542690+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.539188+0000","last_change":"2024-05-30T16:48:29.539188+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.539188+0000","last_undegraded":"2024-05-30T16:48:29.539188+0000","last_fullsized":"2024-05-30T16:48:29.539188+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.555599+0000","last_change":"2024-05-30T16:48:29.555599+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.555599+0000","last_undegraded":"2024-05-30T16:48:29.555599+0000","last_fullsized":"2024-05-30T16:48:29.555599+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.573884+0000","last_change":"2024-05-30T16:48:29.572429+0000","last_active":"2024-05-30T16:48:29.573884+0000","last_peered":"2024-05-30T16:48:29.573884+0000","last_clean":"2024-05-30T16:48:29.573884+0000","last_became_active":"2024-05-30T16:48:29.572105+0000","last_became_peered":"2024-05-30T16:48:29.572105+0000","last_unstale":"2024-05-30T16:48:29.573884+0000","last_undegraded":"2024-05-30T16:48:29.573884+0000","last_fullsized":"2024-05-30T16:48:29.573884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:53.214944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.560040+0000","last_change":"2024-05-30T16:48:29.560040+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.560040+0000","last_undegraded":"2024-05-30T16:48:29.560040+0000","last_fullsized":"2024-05-30T16:48:29.560040+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.573974+0000","last_change":"2024-05-30T16:48:29.572447+0000","last_active":"2024-05-30T16:48:29.573974+0000","last_peered":"2024-05-30T16:48:29.573974+0000","last_clean":"2024-05-30T16:48:29.573974+0000","last_became_active":"2024-05-30T16:48:29.572102+0000","last_became_peered":"2024-05-30T16:48:29.572102+0000","last_unstale":"2024-05-30T16:48:29.573974+0000","last_undegraded":"2024-05-30T16:48:29.573974+0000","last_fullsized":"2024-05-30T16:48:29.573974+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:49.145998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571330+0000","last_change":"2024-05-30T16:48:30.570556+0000","last_active":"2024-05-30T16:48:31.571330+0000","last_peered":"2024-05-30T16:48:31.571330+0000","last_clean":"2024-05-30T16:48:31.571330+0000","last_became_active":"2024-05-30T16:48:30.569419+0000","last_became_peered":"2024-05-30T16:48:30.569419+0000","last_unstale":"2024-05-30T16:48:31.571330+0000","last_undegraded":"2024-05-30T16:48:31.571330+0000","last_fullsized":"2024-05-30T16:48:31.571330+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:33.900129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568268+0000","last_change":"2024-05-30T16:48:29.563603+0000","last_active":"2024-05-30T16:48:29.568268+0000","last_peered":"2024-05-30T16:48:29.568268+0000","last_clean":"2024-05-30T16:48:29.568268+0000","last_became_active":"2024-05-30T16:48:29.563260+0000","last_became_peered":"2024-05-30T16:48:29.563260+0000","last_unstale":"2024-05-30T16:48:29.568268+0000","last_undegraded":"2024-05-30T16:48:29.568268+0000","last_fullsized":"2024-05-30T16:48:29.568268+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:52:26.866201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.549866+0000","last_change":"2024-05-30T16:48:29.549866+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.549866+0000","last_undegraded":"2024-05-30T16:48:29.549866+0000","last_fullsized":"2024-05-30T16:48:29.549866+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.531350+0000","last_change":"2024-05-30T16:48:28.531350+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.531350+0000","last_undegraded":"2024-05-30T16:48:28.531350+0000","last_fullsized":"2024-05-30T16:48:28.531350+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.558514+0000","last_change":"2024-05-30T16:48:29.558514+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.558514+0000","last_undegraded":"2024-05-30T16:48:29.558514+0000","last_fullsized":"2024-05-30T16:48:29.558514+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568452+0000","last_change":"2024-05-30T16:48:29.561932+0000","last_active":"2024-05-30T16:48:29.568452+0000","last_peered":"2024-05-30T16:48:29.568452+0000","last_clean":"2024-05-30T16:48:29.568452+0000","last_became_active":"2024-05-30T16:48:29.561609+0000","last_became_peered":"2024-05-30T16:48:29.561609+0000","last_unstale":"2024-05-30T16:48:29.568452+0000","last_undegraded":"2024-05-30T16:48:29.568452+0000","last_fullsized":"2024-05-30T16:48:29.568452+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:54:21.183349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602281+0000","last_change":"2024-05-30T16:48:30.570083+0000","last_active":"2024-05-30T16:48:31.602281+0000","last_peered":"2024-05-30T16:48:31.602281+0000","last_clean":"2024-05-30T16:48:31.602281+0000","last_became_active":"2024-05-30T16:48:30.568578+0000","last_became_peered":"2024-05-30T16:48:30.568578+0000","last_unstale":"2024-05-30T16:48:31.602281+0000","last_undegraded":"2024-05-30T16:48:31.602281+0000","last_fullsized":"2024-05-30T16:48:31.602281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:51:20.993001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.569697+0000","last_change":"2024-05-30T16:48:29.568131+0000","last_active":"2024-05-30T16:48:29.569697+0000","last_peered":"2024-05-30T16:48:29.569697+0000","last_clean":"2024-05-30T16:48:29.569697+0000","last_became_active":"2024-05-30T16:48:29.567819+0000","last_became_peered":"2024-05-30T16:48:29.567819+0000","last_unstale":"2024-05-30T16:48:29.569697+0000","last_undegraded":"2024-05-30T16:48:29.569697+0000","last_fullsized":"2024-05-30T16:48:29.569697+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:46.064182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600660+0000","last_change":"2024-05-30T16:48:30.561051+0000","last_active":"2024-05-30T16:48:31.600660+0000","last_peered":"2024-05-30T16:48:31.600660+0000","last_clean":"2024-05-30T16:48:31.600660+0000","last_became_active":"2024-05-30T16:48:30.560423+0000","last_became_peered":"2024-05-30T16:48:30.560423+0000","last_unstale":"2024-05-30T16:48:31.600660+0000","last_undegraded":"2024-05-30T16:48:31.600660+0000","last_fullsized":"2024-05-30T16:48:31.600660+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:02.774077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569744+0000","last_change":"2024-05-30T16:48:29.566803+0000","last_active":"2024-05-30T16:48:31.569744+0000","last_peered":"2024-05-30T16:48:31.569744+0000","last_clean":"2024-05-30T16:48:31.569744+0000","last_became_active":"2024-05-30T16:48:29.563938+0000","last_became_peered":"2024-05-30T16:48:29.563938+0000","last_unstale":"2024-05-30T16:48:31.569744+0000","last_undegraded":"2024-05-30T16:48:31.569744+0000","last_fullsized":"2024-05-30T16:48:31.569744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:46.892080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.548745+0000","last_change":"2024-05-30T16:48:29.548745+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.548745+0000","last_undegraded":"2024-05-30T16:48:29.548745+0000","last_fullsized":"2024-05-30T16:48:29.548745+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600710+0000","last_change":"2024-05-30T16:48:29.557714+0000","last_active":"2024-05-30T16:48:31.600710+0000","last_peered":"2024-05-30T16:48:31.600710+0000","last_clean":"2024-05-30T16:48:31.600710+0000","last_became_active":"2024-05-30T16:48:29.557329+0000","last_became_peered":"2024-05-30T16:48:29.557329+0000","last_unstale":"2024-05-30T16:48:31.600710+0000","last_undegraded":"2024-05-30T16:48:31.600710+0000","last_fullsized":"2024-05-30T16:48:31.600710+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:55:42.557260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.559496+0000","last_change":"2024-05-30T16:48:29.559496+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.559496+0000","last_undegraded":"2024-05-30T16:48:29.559496+0000","last_fullsized":"2024-05-30T16:48:29.559496+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.567480+0000","last_change":"2024-05-30T16:48:29.562616+0000","last_active":"2024-05-30T16:48:29.567480+0000","last_peered":"2024-05-30T16:48:29.567480+0000","last_clean":"2024-05-30T16:48:29.567480+0000","last_became_active":"2024-05-30T16:48:29.562279+0000","last_became_peered":"2024-05-30T16:48:29.562279+0000","last_unstale":"2024-05-30T16:48:29.567480+0000","last_undegraded":"2024-05-30T16:48:29.567480+0000","last_fullsized":"2024-05-30T16:48:29.567480+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:09:58.486024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603628+0000","last_change":"2024-05-30T16:48:29.565569+0000","last_active":"2024-05-30T16:48:31.603628+0000","last_peered":"2024-05-30T16:48:31.603628+0000","last_clean":"2024-05-30T16:48:31.603628+0000","last_became_active":"2024-05-30T16:48:29.565276+0000","last_became_peered":"2024-05-30T16:48:29.565276+0000","last_unstale":"2024-05-30T16:48:31.603628+0000","last_undegraded":"2024-05-30T16:48:31.603628+0000","last_fullsized":"2024-05-30T16:48:31.603628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:02.208362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.541170+0000","last_change":"2024-05-30T16:48:29.541170+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.541170+0000","last_undegraded":"2024-05-30T16:48:29.541170+0000","last_fullsized":"2024-05-30T16:48:29.541170+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.534129+0000","last_change":"2024-05-30T16:48:28.534129+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.534129+0000","last_undegraded":"2024-05-30T16:48:28.534129+0000","last_fullsized":"2024-05-30T16:48:28.534129+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.567260+0000","last_change":"2024-05-30T16:48:29.564330+0000","last_active":"2024-05-30T16:48:29.567260+0000","last_peered":"2024-05-30T16:48:29.567260+0000","last_clean":"2024-05-30T16:48:29.567260+0000","last_became_active":"2024-05-30T16:48:29.564050+0000","last_became_peered":"2024-05-30T16:48:29.564050+0000","last_unstale":"2024-05-30T16:48:29.567260+0000","last_undegraded":"2024-05-30T16:48:29.567260+0000","last_fullsized":"2024-05-30T16:48:29.567260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:45.064827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.554849+0000","last_change":"2024-05-30T16:48:29.554849+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.554849+0000","last_undegraded":"2024-05-30T16:48:29.554849+0000","last_fullsized":"2024-05-30T16:48:29.554849+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.560225+0000","last_change":"2024-05-30T16:48:29.556834+0000","last_active":"2024-05-30T16:48:29.560225+0000","last_peered":"2024-05-30T16:48:29.560225+0000","last_clean":"2024-05-30T16:48:29.560225+0000","last_became_active":"2024-05-30T16:48:29.556546+0000","last_became_peered":"2024-05-30T16:48:29.556546+0000","last_unstale":"2024-05-30T16:48:29.560225+0000","last_undegraded":"2024-05-30T16:48:29.560225+0000","last_fullsized":"2024-05-30T16:48:29.560225+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:01.668836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602679+0000","last_change":"2024-05-30T16:48:30.574973+0000","last_active":"2024-05-30T16:48:31.602679+0000","last_peered":"2024-05-30T16:48:31.602679+0000","last_clean":"2024-05-30T16:48:31.602679+0000","last_became_active":"2024-05-30T16:48:30.574679+0000","last_became_peered":"2024-05-30T16:48:30.574679+0000","last_unstale":"2024-05-30T16:48:31.602679+0000","last_undegraded":"2024-05-30T16:48:31.602679+0000","last_fullsized":"2024-05-30T16:48:31.602679+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:02:00.674835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.528356+0000","last_change":"2024-05-30T16:48:28.528356+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.528356+0000","last_undegraded":"2024-05-30T16:48:28.528356+0000","last_fullsized":"2024-05-30T16:48:28.528356+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.543580+0000","last_change":"2024-05-30T16:48:29.543580+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.543580+0000","last_undegraded":"2024-05-30T16:48:29.543580+0000","last_fullsized":"2024-05-30T16:48:29.543580+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.560395+0000","last_change":"2024-05-30T16:48:29.557641+0000","last_active":"2024-05-30T16:48:29.560395+0000","last_peered":"2024-05-30T16:48:29.560395+0000","last_clean":"2024-05-30T16:48:29.560395+0000","last_became_active":"2024-05-30T16:48:29.556424+0000","last_became_peered":"2024-05-30T16:48:29.556424+0000","last_unstale":"2024-05-30T16:48:29.560395+0000","last_undegraded":"2024-05-30T16:48:29.560395+0000","last_fullsized":"2024-05-30T16:48:29.560395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:44.700516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":22,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600828+0000","last_change":"2024-05-30T16:48:29.568303+0000","last_active":"2024-05-30T16:48:31.600828+0000","last_peered":"2024-05-30T16:48:31.600828+0000","last_clean":"2024-05-30T16:48:31.600828+0000","last_became_active":"2024-05-30T16:48:29.567991+0000","last_became_peered":"2024-05-30T16:48:29.567991+0000","last_unstale":"2024-05-30T16:48:31.600828+0000","last_undegraded":"2024-05-30T16:48:31.600828+0000","last_fullsized":"2024-05-30T16:48:31.600828+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:31.450890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.540700+0000","last_change":"2024-05-30T16:48:29.540700+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.540700+0000","last_undegraded":"2024-05-30T16:48:29.540700+0000","last_fullsized":"2024-05-30T16:48:29.540700+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.553284+0000","last_change":"2024-05-30T16:48:29.547415+0000","last_active":"2024-05-30T16:48:29.553284+0000","last_peered":"2024-05-30T16:48:29.553284+0000","last_clean":"2024-05-30T16:48:29.553284+0000","last_became_active":"2024-05-30T16:48:29.547133+0000","last_became_peered":"2024-05-30T16:48:29.547133+0000","last_unstale":"2024-05-30T16:48:29.553284+0000","last_undegraded":"2024-05-30T16:48:29.553284+0000","last_fullsized":"2024-05-30T16:48:29.553284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:13.553689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603817+0000","last_change":"2024-05-30T16:48:30.588261+0000","last_active":"2024-05-30T16:48:31.603817+0000","last_peered":"2024-05-30T16:48:31.603817+0000","last_clean":"2024-05-30T16:48:31.603817+0000","last_became_active":"2024-05-30T16:48:30.587964+0000","last_became_peered":"2024-05-30T16:48:30.587964+0000","last_unstale":"2024-05-30T16:48:31.603817+0000","last_undegraded":"2024-05-30T16:48:31.603817+0000","last_fullsized":"2024-05-30T16:48:31.603817+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.571432+0000","last_change":"2024-05-30T16:48:29.569871+0000","last_active":"2024-05-30T16:48:29.571432+0000","last_peered":"2024-05-30T16:48:29.571432+0000","last_clean":"2024-05-30T16:48:29.571432+0000","last_became_active":"2024-05-30T16:48:29.569572+0000","last_became_peered":"2024-05-30T16:48:29.569572+0000","last_unstale":"2024-05-30T16:48:29.571432+0000","last_undegraded":"2024-05-30T16:48:29.571432+0000","last_fullsized":"2024-05-30T16:48:29.571432+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:23.963030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.557004+0000","last_change":"2024-05-30T16:48:29.557004+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.557004+0000","last_undegraded":"2024-05-30T16:48:29.557004+0000","last_fullsized":"2024-05-30T16:48:29.557004+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570987+0000","last_change":"2024-05-30T16:48:29.556896+0000","last_active":"2024-05-30T16:48:31.570987+0000","last_peered":"2024-05-30T16:48:31.570987+0000","last_clean":"2024-05-30T16:48:31.570987+0000","last_became_active":"2024-05-30T16:48:29.556569+0000","last_became_peered":"2024-05-30T16:48:29.556569+0000","last_unstale":"2024-05-30T16:48:31.570987+0000","last_undegraded":"2024-05-30T16:48:31.570987+0000","last_fullsized":"2024-05-30T16:48:31.570987+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:27.482686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.558489+0000","last_change":"2024-05-30T16:48:29.558489+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.558489+0000","last_undegraded":"2024-05-30T16:48:29.558489+0000","last_fullsized":"2024-05-30T16:48:29.558489+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.569070+0000","last_change":"2024-05-30T16:48:29.565727+0000","last_active":"2024-05-30T16:48:29.569070+0000","last_peered":"2024-05-30T16:48:29.569070+0000","last_clean":"2024-05-30T16:48:29.569070+0000","last_became_active":"2024-05-30T16:48:29.565445+0000","last_became_peered":"2024-05-30T16:48:29.565445+0000","last_unstale":"2024-05-30T16:48:29.569070+0000","last_undegraded":"2024-05-30T16:48:29.569070+0000","last_fullsized":"2024-05-30T16:48:29.569070+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:38:06.238035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.540461+0000","last_change":"2024-05-30T16:48:29.540461+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.540461+0000","last_undegraded":"2024-05-30T16:48:29.540461+0000","last_fullsized":"2024-05-30T16:48:29.540461+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568475+0000","last_change":"2024-05-30T16:48:29.562648+0000","last_active":"2024-05-30T16:48:29.568475+0000","last_peered":"2024-05-30T16:48:29.568475+0000","last_clean":"2024-05-30T16:48:29.568475+0000","last_became_active":"2024-05-30T16:48:29.562380+0000","last_became_peered":"2024-05-30T16:48:29.562380+0000","last_unstale":"2024-05-30T16:48:29.568475+0000","last_undegraded":"2024-05-30T16:48:29.568475+0000","last_fullsized":"2024-05-30T16:48:29.568475+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:36.649081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.551258+0000","last_change":"2024-05-30T16:48:29.551258+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.551258+0000","last_undegraded":"2024-05-30T16:48:29.551258+0000","last_fullsized":"2024-05-30T16:48:29.551258+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569935+0000","last_change":"2024-05-30T16:48:29.564622+0000","last_active":"2024-05-30T16:48:31.569935+0000","last_peered":"2024-05-30T16:48:31.569935+0000","last_clean":"2024-05-30T16:48:31.569935+0000","last_became_active":"2024-05-30T16:48:29.564294+0000","last_became_peered":"2024-05-30T16:48:29.564294+0000","last_unstale":"2024-05-30T16:48:31.569935+0000","last_undegraded":"2024-05-30T16:48:31.569935+0000","last_fullsized":"2024-05-30T16:48:31.569935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:01.759242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.538084+0000","last_change":"2024-05-30T16:48:29.538084+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.538084+0000","last_undegraded":"2024-05-30T16:48:29.538084+0000","last_fullsized":"2024-05-30T16:48:29.538084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601305+0000","last_change":"2024-05-30T16:48:29.565513+0000","last_active":"2024-05-30T16:48:31.601305+0000","last_peered":"2024-05-30T16:48:31.601305+0000","last_clean":"2024-05-30T16:48:31.601305+0000","last_became_active":"2024-05-30T16:48:29.565019+0000","last_became_peered":"2024-05-30T16:48:29.565019+0000","last_unstale":"2024-05-30T16:48:31.601305+0000","last_undegraded":"2024-05-30T16:48:31.601305+0000","last_fullsized":"2024-05-30T16:48:31.601305+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:43.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601161+0000","last_change":"2024-05-30T16:48:30.554107+0000","last_active":"2024-05-30T16:48:31.601161+0000","last_peered":"2024-05-30T16:48:31.601161+0000","last_clean":"2024-05-30T16:48:31.601161+0000","last_became_active":"2024-05-30T16:48:30.553499+0000","last_became_peered":"2024-05-30T16:48:30.553499+0000","last_unstale":"2024-05-30T16:48:31.601161+0000","last_undegraded":"2024-05-30T16:48:31.601161+0000","last_fullsized":"2024-05-30T16:48:31.601161+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:28.497521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.528249+0000","last_change":"2024-05-30T16:48:28.528249+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.528249+0000","last_undegraded":"2024-05-30T16:48:28.528249+0000","last_fullsized":"2024-05-30T16:48:28.528249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.559430+0000","last_change":"2024-05-30T16:48:29.559430+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.559430+0000","last_undegraded":"2024-05-30T16:48:29.559430+0000","last_fullsized":"2024-05-30T16:48:29.559430+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568589+0000","last_change":"2024-05-30T16:48:29.563607+0000","last_active":"2024-05-30T16:48:29.568589+0000","last_peered":"2024-05-30T16:48:29.568589+0000","last_clean":"2024-05-30T16:48:29.568589+0000","last_became_active":"2024-05-30T16:48:29.563281+0000","last_became_peered":"2024-05-30T16:48:29.563281+0000","last_unstale":"2024-05-30T16:48:29.568589+0000","last_undegraded":"2024-05-30T16:48:29.568589+0000","last_fullsized":"2024-05-30T16:48:29.568589+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:29.118183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603410+0000","last_change":"2024-05-30T16:48:30.577542+0000","last_active":"2024-05-30T16:48:31.603410+0000","last_peered":"2024-05-30T16:48:31.603410+0000","last_clean":"2024-05-30T16:48:31.603410+0000","last_became_active":"2024-05-30T16:48:30.576718+0000","last_became_peered":"2024-05-30T16:48:30.576718+0000","last_unstale":"2024-05-30T16:48:31.603410+0000","last_undegraded":"2024-05-30T16:48:31.603410+0000","last_fullsized":"2024-05-30T16:48:31.603410+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:24.028012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.533009+0000","last_change":"2024-05-30T16:48:28.533009+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.533009+0000","last_undegraded":"2024-05-30T16:48:28.533009+0000","last_fullsized":"2024-05-30T16:48:28.533009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568163+0000","last_change":"2024-05-30T16:48:29.563508+0000","last_active":"2024-05-30T16:48:29.568163+0000","last_peered":"2024-05-30T16:48:29.568163+0000","last_clean":"2024-05-30T16:48:29.568163+0000","last_became_active":"2024-05-30T16:48:29.563206+0000","last_became_peered":"2024-05-30T16:48:29.563206+0000","last_unstale":"2024-05-30T16:48:29.568163+0000","last_undegraded":"2024-05-30T16:48:29.568163+0000","last_fullsized":"2024-05-30T16:48:29.568163+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:49.145998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.539746+0000","last_change":"2024-05-30T16:48:29.539746+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.539746+0000","last_undegraded":"2024-05-30T16:48:29.539746+0000","last_fullsized":"2024-05-30T16:48:29.539746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.528519+0000","last_change":"2024-05-30T16:48:28.528519+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.528519+0000","last_undegraded":"2024-05-30T16:48:28.528519+0000","last_fullsized":"2024-05-30T16:48:28.528519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.539795+0000","last_change":"2024-05-30T16:48:29.539795+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.539795+0000","last_undegraded":"2024-05-30T16:48:29.539795+0000","last_fullsized":"2024-05-30T16:48:29.539795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.571255+0000","last_change":"2024-05-30T16:48:29.569525+0000","last_active":"2024-05-30T16:48:29.571255+0000","last_peered":"2024-05-30T16:48:29.571255+0000","last_clean":"2024-05-30T16:48:29.571255+0000","last_became_active":"2024-05-30T16:48:29.568636+0000","last_became_peered":"2024-05-30T16:48:29.568636+0000","last_unstale":"2024-05-30T16:48:29.571255+0000","last_undegraded":"2024-05-30T16:48:29.571255+0000","last_fullsized":"2024-05-30T16:48:29.571255+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:33.870945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569918+0000","last_change":"2024-05-30T16:48:30.553665+0000","last_active":"2024-05-30T16:48:31.569918+0000","last_peered":"2024-05-30T16:48:31.569918+0000","last_clean":"2024-05-30T16:48:31.569918+0000","last_became_active":"2024-05-30T16:48:30.552512+0000","last_became_peered":"2024-05-30T16:48:30.552512+0000","last_unstale":"2024-05-30T16:48:31.569918+0000","last_undegraded":"2024-05-30T16:48:31.569918+0000","last_fullsized":"2024-05-30T16:48:31.569918+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.573789+0000","last_change":"2024-05-30T16:48:29.572007+0000","last_active":"2024-05-30T16:48:29.573789+0000","last_peered":"2024-05-30T16:48:29.573789+0000","last_clean":"2024-05-30T16:48:29.573789+0000","last_became_active":"2024-05-30T16:48:29.571652+0000","last_became_peered":"2024-05-30T16:48:29.571652+0000","last_unstale":"2024-05-30T16:48:29.573789+0000","last_undegraded":"2024-05-30T16:48:29.573789+0000","last_fullsized":"2024-05-30T16:48:29.573789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:17.171552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.553680+0000","last_change":"2024-05-30T16:48:29.553680+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.553680+0000","last_undegraded":"2024-05-30T16:48:29.553680+0000","last_fullsized":"2024-05-30T16:48:29.553680+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600934+0000","last_change":"2024-05-30T16:48:29.554465+0000","last_active":"2024-05-30T16:48:31.600934+0000","last_peered":"2024-05-30T16:48:31.600934+0000","last_clean":"2024-05-30T16:48:31.600934+0000","last_became_active":"2024-05-30T16:48:29.554117+0000","last_became_peered":"2024-05-30T16:48:29.554117+0000","last_unstale":"2024-05-30T16:48:31.600934+0000","last_undegraded":"2024-05-30T16:48:31.600934+0000","last_fullsized":"2024-05-30T16:48:31.600934+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:30.281975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603615+0000","last_change":"2024-05-30T16:48:30.585362+0000","last_active":"2024-05-30T16:48:31.603615+0000","last_peered":"2024-05-30T16:48:31.603615+0000","last_clean":"2024-05-30T16:48:31.603615+0000","last_became_active":"2024-05-30T16:48:30.583719+0000","last_became_peered":"2024-05-30T16:48:30.583719+0000","last_unstale":"2024-05-30T16:48:31.603615+0000","last_undegraded":"2024-05-30T16:48:31.603615+0000","last_fullsized":"2024-05-30T16:48:31.603615+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570531+0000","last_change":"2024-05-30T16:48:29.558681+0000","last_active":"2024-05-30T16:48:31.570531+0000","last_peered":"2024-05-30T16:48:31.570531+0000","last_clean":"2024-05-30T16:48:31.570531+0000","last_became_active":"2024-05-30T16:48:29.558149+0000","last_became_peered":"2024-05-30T16:48:29.558149+0000","last_unstale":"2024-05-30T16:48:31.570531+0000","last_undegraded":"2024-05-30T16:48:31.570531+0000","last_fullsized":"2024-05-30T16:48:31.570531+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:40.373552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":215,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-30T16:48:28.535382+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:48:28.535382+0000","last_peered":"2024-05-30T16:48:28.535382+0000","last_clean":"2024-05-30T16:48:28.535382+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:48:28.535382+0000","last_undegraded":"2024-05-30T16:48:28.535382+0000","last_fullsized":"2024-05-30T16:48:28.535382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":532480,"data_stored":215352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335412,"statfs":{"total":96636764160,"available":96599461888,"internally_reserved":0,"allocated":1114112,"data_stored":799453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35896,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335944,"statfs":{"total":96636764160,"available":96600006656,"internally_reserved":0,"allocated":561152,"data_stored":228497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36472,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335368,"statfs":{"total":96636764160,"available":96599416832,"internally_reserved":0,"allocated":1155072,"data_stored":818829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:48:35.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:48:35.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.295+0000 7f5fc3200640 1 Processor -- start 2024-05-30T16:48:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.295+0000 7f5fc3200640 1 -- start start 2024-05-30T16:48:35.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.296+0000 7f5fc3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 0x7f5fc406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.296+0000 7f5fc3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5fc409b140 0x7f5fc40cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.296+0000 7f5fc3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc40d04c0 con 0x7f5fc406d960 2024-05-30T16:48:35.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.296+0000 7f5fc3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc40d1350 con 0x7f5fc409b140 2024-05-30T16:48:35.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.296+0000 7f5fc3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc40ce780 con 0x7f5fc40ce3a0 2024-05-30T16:48:35.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.297+0000 7f5fc2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5fc409b140 0x7f5fc40cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.297+0000 7f5fc1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 0x7f5fc406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.297+0000 7f5fc2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5fc409b140 0x7f5fc40cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:47240/0 (socket says 172.21.15.67:47240) 2024-05-30T16:48:35.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.297+0000 7f5fc2800640 1 -- 172.21.15.67:0/4229710407 learned_addr learned my addr 172.21.15.67:0/4229710407 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:35.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.297+0000 7f5fc1400640 1 --1- 172.21.15.67:0/4229710407 >> v1:172.21.15.169:6789/0 conn(0x7f5fc406d960 0x7f5fc409aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:41106/0 (socket says 172.21.15.67:41106) 2024-05-30T16:48:35.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.298+0000 7f5fc2800640 1 -- 172.21.15.67:0/4229710407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 msgr2=0x7f5fc406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:35.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.298+0000 7f5fc2800640 1 --2- 172.21.15.67:0/4229710407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 0x7f5fc406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.298+0000 7f5fc2800640 1 -- 172.21.15.67:0/4229710407 >> v1:172.21.15.169:6789/0 conn(0x7f5fc406d960 legacy=0x7f5fc409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.298+0000 7f5fc2800640 1 -- 172.21.15.67:0/4229710407 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc40d0740 con 0x7f5fc409b140 2024-05-30T16:48:35.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.298+0000 7f5fc0a00640 1 -- 172.21.15.67:0/4229710407 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769655871 0 0) 0x7f5fb0001110 con 0x7f5fc406d960 2024-05-30T16:48:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.300+0000 7f5fc2800640 1 --2- 172.21.15.67:0/4229710407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc409b140 0x7f5fc40cc980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009870 tx=0x7f5fbc02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad425aa50997ff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.300+0000 7f5fc0a00640 1 -- 172.21.15.67:0/4229710407 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc037070 con 0x7f5fc409b140 2024-05-30T16:48:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.300+0000 7f5fc0a00640 1 -- 172.21.15.67:0/4229710407 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fbc02ae50 con 0x7f5fc409b140 2024-05-30T16:48:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.300+0000 7f5fc0a00640 1 -- 172.21.15.67:0/4229710407 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc032860 con 0x7f5fc409b140 2024-05-30T16:48:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.301+0000 7f5fc3200640 1 -- 172.21.15.67:0/4229710407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc409b140 msgr2=0x7f5fc40cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.301+0000 7f5fc3200640 1 --2- 172.21.15.67:0/4229710407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc409b140 0x7f5fc40cc980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009870 tx=0x7f5fbc02a300 comp rx=0 tx=0).stop 2024-05-30T16:48:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.301+0000 7f5fc3200640 1 -- 172.21.15.67:0/4229710407 shutdown_connections 2024-05-30T16:48:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.301+0000 7f5fc3200640 1 --2- 172.21.15.67:0/4229710407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc409b140 0x7f5fc40cc980 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.301+0000 7f5fc3200640 1 --2- 172.21.15.67:0/4229710407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 0x7f5fc406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.301+0000 7f5fc3200640 1 -- 172.21.15.67:0/4229710407 >> 172.21.15.67:0/4229710407 conn(0x7f5fc40d2330 msgr2=0x7f5fc40d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.301+0000 7f5fc3200640 1 -- 172.21.15.67:0/4229710407 shutdown_connections 2024-05-30T16:48:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.301+0000 7f5fc3200640 1 -- 172.21.15.67:0/4229710407 wait complete. 2024-05-30T16:48:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.303+0000 7f5fc3200640 1 Processor -- start 2024-05-30T16:48:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.303+0000 7f5fc3200640 1 -- start start 2024-05-30T16:48:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc406d960 0x7f5fc4154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fc409b140 0x7f5fc409d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 0x7f5fc41f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc40d0fe0 con 0x7f5fc409b140 2024-05-30T16:48:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc40d0d00 con 0x7f5fc406d960 2024-05-30T16:48:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc40d17e0 con 0x7f5fc40ce3a0 2024-05-30T16:48:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fc409b140 0x7f5fc409d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fc409b140 0x7f5fc409d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58046/0 (socket says 172.21.15.67:58046) 2024-05-30T16:48:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc1400640 1 -- 172.21.15.67:0/1157150725 learned_addr learned my addr 172.21.15.67:0/1157150725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.304+0000 7f5fc1e00640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc406d960 0x7f5fc4154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fc2800640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 0x7f5fc41f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fc1400640 1 -- 172.21.15.67:0/1157150725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 msgr2=0x7f5fc41f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fc1400640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 0x7f5fc41f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fc1400640 1 -- 172.21.15.67:0/1157150725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc406d960 msgr2=0x7f5fc4154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fc1400640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc406d960 0x7f5fc4154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fc1400640 1 -- 172.21.15.67:0/1157150725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc4154e50 con 0x7f5fc409b140 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fc2800640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 0x7f5fc41f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fc1400640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fc409b140 0x7f5fc409d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000c780 tx=0x7f5fb000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fba000640 1 -- 172.21.15.67:0/1157150725 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0018070 con 0x7f5fc409b140 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fba000640 1 -- 172.21.15.67:0/1157150725 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb00072d0 con 0x7f5fc409b140 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fba000640 1 -- 172.21.15.67:0/1157150725 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00047e0 con 0x7f5fc409b140 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.305+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc41f86a0 con 0x7f5fc409b140 2024-05-30T16:48:35.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.306+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc4154110 con 0x7f5fc409b140 2024-05-30T16:48:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.307+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc40690f0 con 0x7f5fc409b140 2024-05-30T16:48:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.308+0000 7f5fba000640 1 -- 172.21.15.67:0/1157150725 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f5fb001b070 con 0x7f5fc409b140 2024-05-30T16:48:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.309+0000 7f5fba000640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fa406cbf0 0x7f5fa406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.309+0000 7f5fba000640 1 -- 172.21.15.67:0/1157150725 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f5fb008b1f0 con 0x7f5fc409b140 2024-05-30T16:48:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.309+0000 7f5fc1e00640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fa406cbf0 0x7f5fa406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.310+0000 7f5fc1e00640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fa406cbf0 0x7f5fa406f0b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40073f0 tx=0x7f5fb400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.314+0000 7f5fba000640 1 -- 172.21.15.67:0/1157150725 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fc40d04c0 con 0x7f5fc409b140 2024-05-30T16:48:35.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.514+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5fc40d1dc0 con 0x7f5fa406cbf0 2024-05-30T16:48:35.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.532+0000 7f5fba000640 1 -- 172.21.15.67:0/1157150725 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247387 (secure 0 0 0) 0x7f5fc40d1dc0 con 0x7f5fa406cbf0 2024-05-30T16:48:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:35.538 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:48:35.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fa406cbf0 msgr2=0x7f5fa406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fa406cbf0 0x7f5fa406f0b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40073f0 tx=0x7f5fb400b040 comp rx=0 tx=0).stop 2024-05-30T16:48:35.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fc409b140 msgr2=0x7f5fc409d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fc409b140 0x7f5fc409d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000c780 tx=0x7f5fb000cc40 comp rx=0 tx=0).stop 2024-05-30T16:48:35.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 shutdown_connections 2024-05-30T16:48:35.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5fa406cbf0 0x7f5fa406f0b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fc40ce3a0 0x7f5fc41f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fc409b140 0x7f5fc409d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 --2- 172.21.15.67:0/1157150725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fc406d960 0x7f5fc4154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 >> 172.21.15.67:0/1157150725 conn(0x7f5fc40d2330 msgr2=0x7f5fc415f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:35.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 shutdown_connections 2024-05-30T16:48:35.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.538+0000 7f5fc3200640 1 -- 172.21.15.67:0/1157150725 wait complete. 2024-05-30T16:48:35.558 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-05-30T16:48:34.086725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279736,"kb_used_data":5316,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754694984,"statfs":{"total":773094113280,"available":772807663616,"internally_reserved":0,"allocated":5443584,"data_stored":2898471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.572212"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570382+0000","last_change":"2024-05-30T16:48:30.584563+0000","last_active":"2024-05-30T16:48:31.570382+0000","last_peered":"2024-05-30T16:48:31.570382+0000","last_clean":"2024-05-30T16:48:31.570382+0000","last_became_active":"2024-05-30T16:48:30.584262+0000","last_became_peered":"2024-05-30T16:48:30.584262+0000","last_unstale":"2024-05-30T16:48:31.570382+0000","last_undegraded":"2024-05-30T16:48:31.570382+0000","last_fullsized":"2024-05-30T16:48:31.570382+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.553111+0000","last_change":"2024-05-30T16:48:29.553111+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.553111+0000","last_undegraded":"2024-05-30T16:48:29.553111+0000","last_fullsized":"2024-05-30T16:48:29.553111+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603004+0000","last_change":"2024-05-30T16:48:30.564821+0000","last_active":"2024-05-30T16:48:31.603004+0000","last_peered":"2024-05-30T16:48:31.603004+0000","last_clean":"2024-05-30T16:48:31.603004+0000","last_became_active":"2024-05-30T16:48:30.564537+0000","last_became_peered":"2024-05-30T16:48:30.564537+0000","last_unstale":"2024-05-30T16:48:31.603004+0000","last_undegraded":"2024-05-30T16:48:31.603004+0000","last_fullsized":"2024-05-30T16:48:31.603004+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:00:29.368856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570614+0000","last_change":"2024-05-30T16:48:30.565013+0000","last_active":"2024-05-30T16:48:31.570614+0000","last_peered":"2024-05-30T16:48:31.570614+0000","last_clean":"2024-05-30T16:48:31.570614+0000","last_became_active":"2024-05-30T16:48:30.564344+0000","last_became_peered":"2024-05-30T16:48:30.564344+0000","last_unstale":"2024-05-30T16:48:31.570614+0000","last_undegraded":"2024-05-30T16:48:31.570614+0000","last_fullsized":"2024-05-30T16:48:31.570614+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:38.774999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.542049+0000","last_change":"2024-05-30T16:48:29.542049+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.542049+0000","last_undegraded":"2024-05-30T16:48:29.542049+0000","last_fullsized":"2024-05-30T16:48:29.542049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600493+0000","last_change":"2024-05-30T16:48:30.561287+0000","last_active":"2024-05-30T16:48:31.600493+0000","last_peered":"2024-05-30T16:48:31.600493+0000","last_clean":"2024-05-30T16:48:31.600493+0000","last_became_active":"2024-05-30T16:48:30.560737+0000","last_became_peered":"2024-05-30T16:48:30.560737+0000","last_unstale":"2024-05-30T16:48:31.600493+0000","last_undegraded":"2024-05-30T16:48:31.600493+0000","last_fullsized":"2024-05-30T16:48:31.600493+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:50:31.061295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603257+0000","last_change":"2024-05-30T16:48:30.587362+0000","last_active":"2024-05-30T16:48:31.603257+0000","last_peered":"2024-05-30T16:48:31.603257+0000","last_clean":"2024-05-30T16:48:31.603257+0000","last_became_active":"2024-05-30T16:48:30.587100+0000","last_became_peered":"2024-05-30T16:48:30.587100+0000","last_unstale":"2024-05-30T16:48:31.603257+0000","last_undegraded":"2024-05-30T16:48:31.603257+0000","last_fullsized":"2024-05-30T16:48:31.603257+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:00.811234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603466+0000","last_change":"2024-05-30T16:48:30.582474+0000","last_active":"2024-05-30T16:48:31.603466+0000","last_peered":"2024-05-30T16:48:31.603466+0000","last_clean":"2024-05-30T16:48:31.603466+0000","last_became_active":"2024-05-30T16:48:30.582148+0000","last_became_peered":"2024-05-30T16:48:30.582148+0000","last_unstale":"2024-05-30T16:48:31.603466+0000","last_undegraded":"2024-05-30T16:48:31.603466+0000","last_fullsized":"2024-05-30T16:48:31.603466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602911+0000","last_change":"2024-05-30T16:48:30.577384+0000","last_active":"2024-05-30T16:48:31.602911+0000","last_peered":"2024-05-30T16:48:31.602911+0000","last_clean":"2024-05-30T16:48:31.602911+0000","last_became_active":"2024-05-30T16:48:30.570714+0000","last_became_peered":"2024-05-30T16:48:30.570714+0000","last_unstale":"2024-05-30T16:48:31.602911+0000","last_undegraded":"2024-05-30T16:48:31.602911+0000","last_fullsized":"2024-05-30T16:48:31.602911+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:05.025187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570620+0000","last_change":"2024-05-30T16:48:30.582165+0000","last_active":"2024-05-30T16:48:31.570620+0000","last_peered":"2024-05-30T16:48:31.570620+0000","last_clean":"2024-05-30T16:48:31.570620+0000","last_became_active":"2024-05-30T16:48:30.581896+0000","last_became_peered":"2024-05-30T16:48:30.581896+0000","last_unstale":"2024-05-30T16:48:31.570620+0000","last_undegraded":"2024-05-30T16:48:31.570620+0000","last_fullsized":"2024-05-30T16:48:31.570620+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.538253+0000","last_change":"2024-05-30T16:48:29.538253+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.538253+0000","last_undegraded":"2024-05-30T16:48:29.538253+0000","last_fullsized":"2024-05-30T16:48:29.538253+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601441+0000","last_change":"2024-05-30T16:48:30.555321+0000","last_active":"2024-05-30T16:48:31.601441+0000","last_peered":"2024-05-30T16:48:31.601441+0000","last_clean":"2024-05-30T16:48:31.601441+0000","last_became_active":"2024-05-30T16:48:30.555012+0000","last_became_peered":"2024-05-30T16:48:30.555012+0000","last_unstale":"2024-05-30T16:48:31.601441+0000","last_undegraded":"2024-05-30T16:48:31.601441+0000","last_fullsized":"2024-05-30T16:48:31.601441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:41.388387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.543234+0000","last_change":"2024-05-30T16:48:29.543234+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.543234+0000","last_undegraded":"2024-05-30T16:48:29.543234+0000","last_fullsized":"2024-05-30T16:48:29.543234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.560080+0000","last_change":"2024-05-30T16:48:29.560080+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.560080+0000","last_undegraded":"2024-05-30T16:48:29.560080+0000","last_fullsized":"2024-05-30T16:48:29.560080+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.552921+0000","last_change":"2024-05-30T16:48:29.552921+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.552921+0000","last_undegraded":"2024-05-30T16:48:29.552921+0000","last_fullsized":"2024-05-30T16:48:29.552921+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.561708+0000","last_change":"2024-05-30T16:48:29.561708+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.561708+0000","last_undegraded":"2024-05-30T16:48:29.561708+0000","last_fullsized":"2024-05-30T16:48:29.561708+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.555816+0000","last_change":"2024-05-30T16:48:29.555816+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.555816+0000","last_undegraded":"2024-05-30T16:48:29.555816+0000","last_fullsized":"2024-05-30T16:48:29.555816+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.540205+0000","last_change":"2024-05-30T16:48:29.540205+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.540205+0000","last_undegraded":"2024-05-30T16:48:29.540205+0000","last_fullsized":"2024-05-30T16:48:29.540205+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.541235+0000","last_change":"2024-05-30T16:48:29.541235+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.541235+0000","last_undegraded":"2024-05-30T16:48:29.541235+0000","last_fullsized":"2024-05-30T16:48:29.541235+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601819+0000","last_change":"2024-05-30T16:48:30.563868+0000","last_active":"2024-05-30T16:48:31.601819+0000","last_peered":"2024-05-30T16:48:31.601819+0000","last_clean":"2024-05-30T16:48:31.601819+0000","last_became_active":"2024-05-30T16:48:30.563554+0000","last_became_peered":"2024-05-30T16:48:30.563554+0000","last_unstale":"2024-05-30T16:48:31.601819+0000","last_undegraded":"2024-05-30T16:48:31.601819+0000","last_fullsized":"2024-05-30T16:48:31.601819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:47.906916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.547433+0000","last_change":"2024-05-30T16:48:29.547433+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.547433+0000","last_undegraded":"2024-05-30T16:48:29.547433+0000","last_fullsized":"2024-05-30T16:48:29.547433+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.548367+0000","last_change":"2024-05-30T16:48:29.548367+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.548367+0000","last_undegraded":"2024-05-30T16:48:29.548367+0000","last_fullsized":"2024-05-30T16:48:29.548367+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.565566+0000","last_change":"2024-05-30T16:48:29.565566+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.565566+0000","last_undegraded":"2024-05-30T16:48:29.565566+0000","last_fullsized":"2024-05-30T16:48:29.565566+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.541213+0000","last_change":"2024-05-30T16:48:29.541213+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.541213+0000","last_undegraded":"2024-05-30T16:48:29.541213+0000","last_fullsized":"2024-05-30T16:48:29.541213+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.552035+0000","last_change":"2024-05-30T16:48:29.552035+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.552035+0000","last_undegraded":"2024-05-30T16:48:29.552035+0000","last_fullsized":"2024-05-30T16:48:29.552035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571075+0000","last_change":"2024-05-30T16:48:30.582495+0000","last_active":"2024-05-30T16:48:31.571075+0000","last_peered":"2024-05-30T16:48:31.571075+0000","last_clean":"2024-05-30T16:48:31.571075+0000","last_became_active":"2024-05-30T16:48:30.582199+0000","last_became_peered":"2024-05-30T16:48:30.582199+0000","last_unstale":"2024-05-30T16:48:31.571075+0000","last_undegraded":"2024-05-30T16:48:31.571075+0000","last_fullsized":"2024-05-30T16:48:31.571075+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:18.447770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.556706+0000","last_change":"2024-05-30T16:48:29.556706+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.556706+0000","last_undegraded":"2024-05-30T16:48:29.556706+0000","last_fullsized":"2024-05-30T16:48:29.556706+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.560620+0000","last_change":"2024-05-30T16:48:29.560620+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.560620+0000","last_undegraded":"2024-05-30T16:48:29.560620+0000","last_fullsized":"2024-05-30T16:48:29.560620+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.542690+0000","last_change":"2024-05-30T16:48:29.542690+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.542690+0000","last_undegraded":"2024-05-30T16:48:29.542690+0000","last_fullsized":"2024-05-30T16:48:29.542690+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.539188+0000","last_change":"2024-05-30T16:48:29.539188+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.539188+0000","last_undegraded":"2024-05-30T16:48:29.539188+0000","last_fullsized":"2024-05-30T16:48:29.539188+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.555599+0000","last_change":"2024-05-30T16:48:29.555599+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.555599+0000","last_undegraded":"2024-05-30T16:48:29.555599+0000","last_fullsized":"2024-05-30T16:48:29.555599+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.573884+0000","last_change":"2024-05-30T16:48:29.572429+0000","last_active":"2024-05-30T16:48:29.573884+0000","last_peered":"2024-05-30T16:48:29.573884+0000","last_clean":"2024-05-30T16:48:29.573884+0000","last_became_active":"2024-05-30T16:48:29.572105+0000","last_became_peered":"2024-05-30T16:48:29.572105+0000","last_unstale":"2024-05-30T16:48:29.573884+0000","last_undegraded":"2024-05-30T16:48:29.573884+0000","last_fullsized":"2024-05-30T16:48:29.573884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:53.214944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.560040+0000","last_change":"2024-05-30T16:48:29.560040+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.560040+0000","last_undegraded":"2024-05-30T16:48:29.560040+0000","last_fullsized":"2024-05-30T16:48:29.560040+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.573974+0000","last_change":"2024-05-30T16:48:29.572447+0000","last_active":"2024-05-30T16:48:29.573974+0000","last_peered":"2024-05-30T16:48:29.573974+0000","last_clean":"2024-05-30T16:48:29.573974+0000","last_became_active":"2024-05-30T16:48:29.572102+0000","last_became_peered":"2024-05-30T16:48:29.572102+0000","last_unstale":"2024-05-30T16:48:29.573974+0000","last_undegraded":"2024-05-30T16:48:29.573974+0000","last_fullsized":"2024-05-30T16:48:29.573974+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:49.145998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571330+0000","last_change":"2024-05-30T16:48:30.570556+0000","last_active":"2024-05-30T16:48:31.571330+0000","last_peered":"2024-05-30T16:48:31.571330+0000","last_clean":"2024-05-30T16:48:31.571330+0000","last_became_active":"2024-05-30T16:48:30.569419+0000","last_became_peered":"2024-05-30T16:48:30.569419+0000","last_unstale":"2024-05-30T16:48:31.571330+0000","last_undegraded":"2024-05-30T16:48:31.571330+0000","last_fullsized":"2024-05-30T16:48:31.571330+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:33.900129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568268+0000","last_change":"2024-05-30T16:48:29.563603+0000","last_active":"2024-05-30T16:48:29.568268+0000","last_peered":"2024-05-30T16:48:29.568268+0000","last_clean":"2024-05-30T16:48:29.568268+0000","last_became_active":"2024-05-30T16:48:29.563260+0000","last_became_peered":"2024-05-30T16:48:29.563260+0000","last_unstale":"2024-05-30T16:48:29.568268+0000","last_undegraded":"2024-05-30T16:48:29.568268+0000","last_fullsized":"2024-05-30T16:48:29.568268+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:52:26.866201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.549866+0000","last_change":"2024-05-30T16:48:29.549866+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.549866+0000","last_undegraded":"2024-05-30T16:48:29.549866+0000","last_fullsized":"2024-05-30T16:48:29.549866+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.531350+0000","last_change":"2024-05-30T16:48:28.531350+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.531350+0000","last_undegraded":"2024-05-30T16:48:28.531350+0000","last_fullsized":"2024-05-30T16:48:28.531350+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.558514+0000","last_change":"2024-05-30T16:48:29.558514+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.558514+0000","last_undegraded":"2024-05-30T16:48:29.558514+0000","last_fullsized":"2024-05-30T16:48:29.558514+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568452+0000","last_change":"2024-05-30T16:48:29.561932+0000","last_active":"2024-05-30T16:48:29.568452+0000","last_peered":"2024-05-30T16:48:29.568452+0000","last_clean":"2024-05-30T16:48:29.568452+0000","last_became_active":"2024-05-30T16:48:29.561609+0000","last_became_peered":"2024-05-30T16:48:29.561609+0000","last_unstale":"2024-05-30T16:48:29.568452+0000","last_undegraded":"2024-05-30T16:48:29.568452+0000","last_fullsized":"2024-05-30T16:48:29.568452+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:54:21.183349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602281+0000","last_change":"2024-05-30T16:48:30.570083+0000","last_active":"2024-05-30T16:48:31.602281+0000","last_peered":"2024-05-30T16:48:31.602281+0000","last_clean":"2024-05-30T16:48:31.602281+0000","last_became_active":"2024-05-30T16:48:30.568578+0000","last_became_peered":"2024-05-30T16:48:30.568578+0000","last_unstale":"2024-05-30T16:48:31.602281+0000","last_undegraded":"2024-05-30T16:48:31.602281+0000","last_fullsized":"2024-05-30T16:48:31.602281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:51:20.993001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.569697+0000","last_change":"2024-05-30T16:48:29.568131+0000","last_active":"2024-05-30T16:48:29.569697+0000","last_peered":"2024-05-30T16:48:29.569697+0000","last_clean":"2024-05-30T16:48:29.569697+0000","last_became_active":"2024-05-30T16:48:29.567819+0000","last_became_peered":"2024-05-30T16:48:29.567819+0000","last_unstale":"2024-05-30T16:48:29.569697+0000","last_undegraded":"2024-05-30T16:48:29.569697+0000","last_fullsized":"2024-05-30T16:48:29.569697+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:46.064182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600660+0000","last_change":"2024-05-30T16:48:30.561051+0000","last_active":"2024-05-30T16:48:31.600660+0000","last_peered":"2024-05-30T16:48:31.600660+0000","last_clean":"2024-05-30T16:48:31.600660+0000","last_became_active":"2024-05-30T16:48:30.560423+0000","last_became_peered":"2024-05-30T16:48:30.560423+0000","last_unstale":"2024-05-30T16:48:31.600660+0000","last_undegraded":"2024-05-30T16:48:31.600660+0000","last_fullsized":"2024-05-30T16:48:31.600660+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:02.774077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569744+0000","last_change":"2024-05-30T16:48:29.566803+0000","last_active":"2024-05-30T16:48:31.569744+0000","last_peered":"2024-05-30T16:48:31.569744+0000","last_clean":"2024-05-30T16:48:31.569744+0000","last_became_active":"2024-05-30T16:48:29.563938+0000","last_became_peered":"2024-05-30T16:48:29.563938+0000","last_unstale":"2024-05-30T16:48:31.569744+0000","last_undegraded":"2024-05-30T16:48:31.569744+0000","last_fullsized":"2024-05-30T16:48:31.569744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:46.892080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.548745+0000","last_change":"2024-05-30T16:48:29.548745+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.548745+0000","last_undegraded":"2024-05-30T16:48:29.548745+0000","last_fullsized":"2024-05-30T16:48:29.548745+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600710+0000","last_change":"2024-05-30T16:48:29.557714+0000","last_active":"2024-05-30T16:48:31.600710+0000","last_peered":"2024-05-30T16:48:31.600710+0000","last_clean":"2024-05-30T16:48:31.600710+0000","last_became_active":"2024-05-30T16:48:29.557329+0000","last_became_peered":"2024-05-30T16:48:29.557329+0000","last_unstale":"2024-05-30T16:48:31.600710+0000","last_undegraded":"2024-05-30T16:48:31.600710+0000","last_fullsized":"2024-05-30T16:48:31.600710+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:55:42.557260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.559496+0000","last_change":"2024-05-30T16:48:29.559496+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.559496+0000","last_undegraded":"2024-05-30T16:48:29.559496+0000","last_fullsized":"2024-05-30T16:48:29.559496+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.567480+0000","last_change":"2024-05-30T16:48:29.562616+0000","last_active":"2024-05-30T16:48:29.567480+0000","last_peered":"2024-05-30T16:48:29.567480+0000","last_clean":"2024-05-30T16:48:29.567480+0000","last_became_active":"2024-05-30T16:48:29.562279+0000","last_became_peered":"2024-05-30T16:48:29.562279+0000","last_unstale":"2024-05-30T16:48:29.567480+0000","last_undegraded":"2024-05-30T16:48:29.567480+0000","last_fullsized":"2024-05-30T16:48:29.567480+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:09:58.486024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603628+0000","last_change":"2024-05-30T16:48:29.565569+0000","last_active":"2024-05-30T16:48:31.603628+0000","last_peered":"2024-05-30T16:48:31.603628+0000","last_clean":"2024-05-30T16:48:31.603628+0000","last_became_active":"2024-05-30T16:48:29.565276+0000","last_became_peered":"2024-05-30T16:48:29.565276+0000","last_unstale":"2024-05-30T16:48:31.603628+0000","last_undegraded":"2024-05-30T16:48:31.603628+0000","last_fullsized":"2024-05-30T16:48:31.603628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:02.208362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.541170+0000","last_change":"2024-05-30T16:48:29.541170+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.541170+0000","last_undegraded":"2024-05-30T16:48:29.541170+0000","last_fullsized":"2024-05-30T16:48:29.541170+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.534129+0000","last_change":"2024-05-30T16:48:28.534129+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.534129+0000","last_undegraded":"2024-05-30T16:48:28.534129+0000","last_fullsized":"2024-05-30T16:48:28.534129+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.567260+0000","last_change":"2024-05-30T16:48:29.564330+0000","last_active":"2024-05-30T16:48:29.567260+0000","last_peered":"2024-05-30T16:48:29.567260+0000","last_clean":"2024-05-30T16:48:29.567260+0000","last_became_active":"2024-05-30T16:48:29.564050+0000","last_became_peered":"2024-05-30T16:48:29.564050+0000","last_unstale":"2024-05-30T16:48:29.567260+0000","last_undegraded":"2024-05-30T16:48:29.567260+0000","last_fullsized":"2024-05-30T16:48:29.567260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:45.064827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.554849+0000","last_change":"2024-05-30T16:48:29.554849+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.554849+0000","last_undegraded":"2024-05-30T16:48:29.554849+0000","last_fullsized":"2024-05-30T16:48:29.554849+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.560225+0000","last_change":"2024-05-30T16:48:29.556834+0000","last_active":"2024-05-30T16:48:29.560225+0000","last_peered":"2024-05-30T16:48:29.560225+0000","last_clean":"2024-05-30T16:48:29.560225+0000","last_became_active":"2024-05-30T16:48:29.556546+0000","last_became_peered":"2024-05-30T16:48:29.556546+0000","last_unstale":"2024-05-30T16:48:29.560225+0000","last_undegraded":"2024-05-30T16:48:29.560225+0000","last_fullsized":"2024-05-30T16:48:29.560225+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:01.668836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602679+0000","last_change":"2024-05-30T16:48:30.574973+0000","last_active":"2024-05-30T16:48:31.602679+0000","last_peered":"2024-05-30T16:48:31.602679+0000","last_clean":"2024-05-30T16:48:31.602679+0000","last_became_active":"2024-05-30T16:48:30.574679+0000","last_became_peered":"2024-05-30T16:48:30.574679+0000","last_unstale":"2024-05-30T16:48:31.602679+0000","last_undegraded":"2024-05-30T16:48:31.602679+0000","last_fullsized":"2024-05-30T16:48:31.602679+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:02:00.674835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.528356+0000","last_change":"2024-05-30T16:48:28.528356+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.528356+0000","last_undegraded":"2024-05-30T16:48:28.528356+0000","last_fullsized":"2024-05-30T16:48:28.528356+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.543580+0000","last_change":"2024-05-30T16:48:29.543580+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.543580+0000","last_undegraded":"2024-05-30T16:48:29.543580+0000","last_fullsized":"2024-05-30T16:48:29.543580+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.560395+0000","last_change":"2024-05-30T16:48:29.557641+0000","last_active":"2024-05-30T16:48:29.560395+0000","last_peered":"2024-05-30T16:48:29.560395+0000","last_clean":"2024-05-30T16:48:29.560395+0000","last_became_active":"2024-05-30T16:48:29.556424+0000","last_became_peered":"2024-05-30T16:48:29.556424+0000","last_unstale":"2024-05-30T16:48:29.560395+0000","last_undegraded":"2024-05-30T16:48:29.560395+0000","last_fullsized":"2024-05-30T16:48:29.560395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:44.700516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":22,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600828+0000","last_change":"2024-05-30T16:48:29.568303+0000","last_active":"2024-05-30T16:48:31.600828+0000","last_peered":"2024-05-30T16:48:31.600828+0000","last_clean":"2024-05-30T16:48:31.600828+0000","last_became_active":"2024-05-30T16:48:29.567991+0000","last_became_peered":"2024-05-30T16:48:29.567991+0000","last_unstale":"2024-05-30T16:48:31.600828+0000","last_undegraded":"2024-05-30T16:48:31.600828+0000","last_fullsized":"2024-05-30T16:48:31.600828+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:31.450890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.540700+0000","last_change":"2024-05-30T16:48:29.540700+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.540700+0000","last_undegraded":"2024-05-30T16:48:29.540700+0000","last_fullsized":"2024-05-30T16:48:29.540700+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.553284+0000","last_change":"2024-05-30T16:48:29.547415+0000","last_active":"2024-05-30T16:48:29.553284+0000","last_peered":"2024-05-30T16:48:29.553284+0000","last_clean":"2024-05-30T16:48:29.553284+0000","last_became_active":"2024-05-30T16:48:29.547133+0000","last_became_peered":"2024-05-30T16:48:29.547133+0000","last_unstale":"2024-05-30T16:48:29.553284+0000","last_undegraded":"2024-05-30T16:48:29.553284+0000","last_fullsized":"2024-05-30T16:48:29.553284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:13.553689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603817+0000","last_change":"2024-05-30T16:48:30.588261+0000","last_active":"2024-05-30T16:48:31.603817+0000","last_peered":"2024-05-30T16:48:31.603817+0000","last_clean":"2024-05-30T16:48:31.603817+0000","last_became_active":"2024-05-30T16:48:30.587964+0000","last_became_peered":"2024-05-30T16:48:30.587964+0000","last_unstale":"2024-05-30T16:48:31.603817+0000","last_undegraded":"2024-05-30T16:48:31.603817+0000","last_fullsized":"2024-05-30T16:48:31.603817+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.571432+0000","last_change":"2024-05-30T16:48:29.569871+0000","last_active":"2024-05-30T16:48:29.571432+0000","last_peered":"2024-05-30T16:48:29.571432+0000","last_clean":"2024-05-30T16:48:29.571432+0000","last_became_active":"2024-05-30T16:48:29.569572+0000","last_became_peered":"2024-05-30T16:48:29.569572+0000","last_unstale":"2024-05-30T16:48:29.571432+0000","last_undegraded":"2024-05-30T16:48:29.571432+0000","last_fullsized":"2024-05-30T16:48:29.571432+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:23.963030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.557004+0000","last_change":"2024-05-30T16:48:29.557004+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.557004+0000","last_undegraded":"2024-05-30T16:48:29.557004+0000","last_fullsized":"2024-05-30T16:48:29.557004+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570987+0000","last_change":"2024-05-30T16:48:29.556896+0000","last_active":"2024-05-30T16:48:31.570987+0000","last_peered":"2024-05-30T16:48:31.570987+0000","last_clean":"2024-05-30T16:48:31.570987+0000","last_became_active":"2024-05-30T16:48:29.556569+0000","last_became_peered":"2024-05-30T16:48:29.556569+0000","last_unstale":"2024-05-30T16:48:31.570987+0000","last_undegraded":"2024-05-30T16:48:31.570987+0000","last_fullsized":"2024-05-30T16:48:31.570987+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:27.482686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.558489+0000","last_change":"2024-05-30T16:48:29.558489+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.558489+0000","last_undegraded":"2024-05-30T16:48:29.558489+0000","last_fullsized":"2024-05-30T16:48:29.558489+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.569070+0000","last_change":"2024-05-30T16:48:29.565727+0000","last_active":"2024-05-30T16:48:29.569070+0000","last_peered":"2024-05-30T16:48:29.569070+0000","last_clean":"2024-05-30T16:48:29.569070+0000","last_became_active":"2024-05-30T16:48:29.565445+0000","last_became_peered":"2024-05-30T16:48:29.565445+0000","last_unstale":"2024-05-30T16:48:29.569070+0000","last_undegraded":"2024-05-30T16:48:29.569070+0000","last_fullsized":"2024-05-30T16:48:29.569070+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:38:06.238035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.540461+0000","last_change":"2024-05-30T16:48:29.540461+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.540461+0000","last_undegraded":"2024-05-30T16:48:29.540461+0000","last_fullsized":"2024-05-30T16:48:29.540461+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568475+0000","last_change":"2024-05-30T16:48:29.562648+0000","last_active":"2024-05-30T16:48:29.568475+0000","last_peered":"2024-05-30T16:48:29.568475+0000","last_clean":"2024-05-30T16:48:29.568475+0000","last_became_active":"2024-05-30T16:48:29.562380+0000","last_became_peered":"2024-05-30T16:48:29.562380+0000","last_unstale":"2024-05-30T16:48:29.568475+0000","last_undegraded":"2024-05-30T16:48:29.568475+0000","last_fullsized":"2024-05-30T16:48:29.568475+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:36.649081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.551258+0000","last_change":"2024-05-30T16:48:29.551258+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.551258+0000","last_undegraded":"2024-05-30T16:48:29.551258+0000","last_fullsized":"2024-05-30T16:48:29.551258+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569935+0000","last_change":"2024-05-30T16:48:29.564622+0000","last_active":"2024-05-30T16:48:31.569935+0000","last_peered":"2024-05-30T16:48:31.569935+0000","last_clean":"2024-05-30T16:48:31.569935+0000","last_became_active":"2024-05-30T16:48:29.564294+0000","last_became_peered":"2024-05-30T16:48:29.564294+0000","last_unstale":"2024-05-30T16:48:31.569935+0000","last_undegraded":"2024-05-30T16:48:31.569935+0000","last_fullsized":"2024-05-30T16:48:31.569935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:01.759242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.538084+0000","last_change":"2024-05-30T16:48:29.538084+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.538084+0000","last_undegraded":"2024-05-30T16:48:29.538084+0000","last_fullsized":"2024-05-30T16:48:29.538084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601305+0000","last_change":"2024-05-30T16:48:29.565513+0000","last_active":"2024-05-30T16:48:31.601305+0000","last_peered":"2024-05-30T16:48:31.601305+0000","last_clean":"2024-05-30T16:48:31.601305+0000","last_became_active":"2024-05-30T16:48:29.565019+0000","last_became_peered":"2024-05-30T16:48:29.565019+0000","last_unstale":"2024-05-30T16:48:31.601305+0000","last_undegraded":"2024-05-30T16:48:31.601305+0000","last_fullsized":"2024-05-30T16:48:31.601305+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:43.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601161+0000","last_change":"2024-05-30T16:48:30.554107+0000","last_active":"2024-05-30T16:48:31.601161+0000","last_peered":"2024-05-30T16:48:31.601161+0000","last_clean":"2024-05-30T16:48:31.601161+0000","last_became_active":"2024-05-30T16:48:30.553499+0000","last_became_peered":"2024-05-30T16:48:30.553499+0000","last_unstale":"2024-05-30T16:48:31.601161+0000","last_undegraded":"2024-05-30T16:48:31.601161+0000","last_fullsized":"2024-05-30T16:48:31.601161+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:28.497521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.528249+0000","last_change":"2024-05-30T16:48:28.528249+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.528249+0000","last_undegraded":"2024-05-30T16:48:28.528249+0000","last_fullsized":"2024-05-30T16:48:28.528249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.559430+0000","last_change":"2024-05-30T16:48:29.559430+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.559430+0000","last_undegraded":"2024-05-30T16:48:29.559430+0000","last_fullsized":"2024-05-30T16:48:29.559430+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568589+0000","last_change":"2024-05-30T16:48:29.563607+0000","last_active":"2024-05-30T16:48:29.568589+0000","last_peered":"2024-05-30T16:48:29.568589+0000","last_clean":"2024-05-30T16:48:29.568589+0000","last_became_active":"2024-05-30T16:48:29.563281+0000","last_became_peered":"2024-05-30T16:48:29.563281+0000","last_unstale":"2024-05-30T16:48:29.568589+0000","last_undegraded":"2024-05-30T16:48:29.568589+0000","last_fullsized":"2024-05-30T16:48:29.568589+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:29.118183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603410+0000","last_change":"2024-05-30T16:48:30.577542+0000","last_active":"2024-05-30T16:48:31.603410+0000","last_peered":"2024-05-30T16:48:31.603410+0000","last_clean":"2024-05-30T16:48:31.603410+0000","last_became_active":"2024-05-30T16:48:30.576718+0000","last_became_peered":"2024-05-30T16:48:30.576718+0000","last_unstale":"2024-05-30T16:48:31.603410+0000","last_undegraded":"2024-05-30T16:48:31.603410+0000","last_fullsized":"2024-05-30T16:48:31.603410+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:24.028012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.533009+0000","last_change":"2024-05-30T16:48:28.533009+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.533009+0000","last_undegraded":"2024-05-30T16:48:28.533009+0000","last_fullsized":"2024-05-30T16:48:28.533009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568163+0000","last_change":"2024-05-30T16:48:29.563508+0000","last_active":"2024-05-30T16:48:29.568163+0000","last_peered":"2024-05-30T16:48:29.568163+0000","last_clean":"2024-05-30T16:48:29.568163+0000","last_became_active":"2024-05-30T16:48:29.563206+0000","last_became_peered":"2024-05-30T16:48:29.563206+0000","last_unstale":"2024-05-30T16:48:29.568163+0000","last_undegraded":"2024-05-30T16:48:29.568163+0000","last_fullsized":"2024-05-30T16:48:29.568163+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:49.145998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.539746+0000","last_change":"2024-05-30T16:48:29.539746+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.539746+0000","last_undegraded":"2024-05-30T16:48:29.539746+0000","last_fullsized":"2024-05-30T16:48:29.539746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.528519+0000","last_change":"2024-05-30T16:48:28.528519+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.528519+0000","last_undegraded":"2024-05-30T16:48:28.528519+0000","last_fullsized":"2024-05-30T16:48:28.528519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.539795+0000","last_change":"2024-05-30T16:48:29.539795+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.539795+0000","last_undegraded":"2024-05-30T16:48:29.539795+0000","last_fullsized":"2024-05-30T16:48:29.539795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.571255+0000","last_change":"2024-05-30T16:48:29.569525+0000","last_active":"2024-05-30T16:48:29.571255+0000","last_peered":"2024-05-30T16:48:29.571255+0000","last_clean":"2024-05-30T16:48:29.571255+0000","last_became_active":"2024-05-30T16:48:29.568636+0000","last_became_peered":"2024-05-30T16:48:29.568636+0000","last_unstale":"2024-05-30T16:48:29.571255+0000","last_undegraded":"2024-05-30T16:48:29.571255+0000","last_fullsized":"2024-05-30T16:48:29.571255+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:33.870945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569918+0000","last_change":"2024-05-30T16:48:30.553665+0000","last_active":"2024-05-30T16:48:31.569918+0000","last_peered":"2024-05-30T16:48:31.569918+0000","last_clean":"2024-05-30T16:48:31.569918+0000","last_became_active":"2024-05-30T16:48:30.552512+0000","last_became_peered":"2024-05-30T16:48:30.552512+0000","last_unstale":"2024-05-30T16:48:31.569918+0000","last_undegraded":"2024-05-30T16:48:31.569918+0000","last_fullsized":"2024-05-30T16:48:31.569918+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.573789+0000","last_change":"2024-05-30T16:48:29.572007+0000","last_active":"2024-05-30T16:48:29.573789+0000","last_peered":"2024-05-30T16:48:29.573789+0000","last_clean":"2024-05-30T16:48:29.573789+0000","last_became_active":"2024-05-30T16:48:29.571652+0000","last_became_peered":"2024-05-30T16:48:29.571652+0000","last_unstale":"2024-05-30T16:48:29.573789+0000","last_undegraded":"2024-05-30T16:48:29.573789+0000","last_fullsized":"2024-05-30T16:48:29.573789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:17.171552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.553680+0000","last_change":"2024-05-30T16:48:29.553680+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.553680+0000","last_undegraded":"2024-05-30T16:48:29.553680+0000","last_fullsized":"2024-05-30T16:48:29.553680+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600934+0000","last_change":"2024-05-30T16:48:29.554465+0000","last_active":"2024-05-30T16:48:31.600934+0000","last_peered":"2024-05-30T16:48:31.600934+0000","last_clean":"2024-05-30T16:48:31.600934+0000","last_became_active":"2024-05-30T16:48:29.554117+0000","last_became_peered":"2024-05-30T16:48:29.554117+0000","last_unstale":"2024-05-30T16:48:31.600934+0000","last_undegraded":"2024-05-30T16:48:31.600934+0000","last_fullsized":"2024-05-30T16:48:31.600934+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:30.281975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603615+0000","last_change":"2024-05-30T16:48:30.585362+0000","last_active":"2024-05-30T16:48:31.603615+0000","last_peered":"2024-05-30T16:48:31.603615+0000","last_clean":"2024-05-30T16:48:31.603615+0000","last_became_active":"2024-05-30T16:48:30.583719+0000","last_became_peered":"2024-05-30T16:48:30.583719+0000","last_unstale":"2024-05-30T16:48:31.603615+0000","last_undegraded":"2024-05-30T16:48:31.603615+0000","last_fullsized":"2024-05-30T16:48:31.603615+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570531+0000","last_change":"2024-05-30T16:48:29.558681+0000","last_active":"2024-05-30T16:48:31.570531+0000","last_peered":"2024-05-30T16:48:31.570531+0000","last_clean":"2024-05-30T16:48:31.570531+0000","last_became_active":"2024-05-30T16:48:29.558149+0000","last_became_peered":"2024-05-30T16:48:29.558149+0000","last_unstale":"2024-05-30T16:48:31.570531+0000","last_undegraded":"2024-05-30T16:48:31.570531+0000","last_fullsized":"2024-05-30T16:48:31.570531+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:40.373552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":215,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-30T16:48:28.535382+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:48:28.535382+0000","last_peered":"2024-05-30T16:48:28.535382+0000","last_clean":"2024-05-30T16:48:28.535382+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:48:28.535382+0000","last_undegraded":"2024-05-30T16:48:28.535382+0000","last_fullsized":"2024-05-30T16:48:28.535382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":532480,"data_stored":215352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335412,"statfs":{"total":96636764160,"available":96599461888,"internally_reserved":0,"allocated":1114112,"data_stored":799453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35896,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335944,"statfs":{"total":96636764160,"available":96600006656,"internally_reserved":0,"allocated":561152,"data_stored":228497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36472,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335368,"statfs":{"total":96636764160,"available":96599416832,"internally_reserved":0,"allocated":1155072,"data_stored":818829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:48:35.562 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:48:35.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.814+0000 7f79a9c00640 1 Processor -- start 2024-05-30T16:48:35.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.814+0000 7f79a9c00640 1 -- start start 2024-05-30T16:48:35.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.815+0000 7f79a9c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79a40d16a0 0x7f79a40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.815+0000 7f79a9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a40cf870 0x7f79a40d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.815+0000 7f79a9c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a415ec50 con 0x7f79a415e870 2024-05-30T16:48:35.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.815+0000 7f79a9c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f79a40d2150 con 0x7f79a40d16a0 2024-05-30T16:48:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.815+0000 7f79a9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79a40cfd20 con 0x7f79a40cf870 2024-05-30T16:48:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.816+0000 7f79a3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f79a415e870 0x7f79a40d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52162/0 (socket says 172.21.15.67:52162) 2024-05-30T16:48:35.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.816+0000 7f79a3e00640 1 -- 172.21.15.67:0/825587312 learned_addr learned my addr 172.21.15.67:0/825587312 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.816+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1796941059 0 0) 0x7f79a415ec50 con 0x7f79a415e870 2024-05-30T16:48:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.816+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7984003520 con 0x7f79a415e870 2024-05-30T16:48:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.816+0000 7f79a9200640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a40cf870 0x7f79a40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.816+0000 7f79a3400640 1 --2- 172.21.15.67:0/825587312 >> v2:172.21.15.169:3300/0 conn(0x7f79a40d16a0 0x7f79a40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1500972667 0 0) 0x7f7984003520 con 0x7f79a415e870 2024-05-30T16:48:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a40cf870 msgr2=0x7f79a40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a40cf870 0x7f79a40d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 >> v2:172.21.15.169:3300/0 conn(0x7f79a40d16a0 msgr2=0x7f79a40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 --2- 172.21.15.67:0/825587312 >> v2:172.21.15.169:3300/0 conn(0x7f79a40d16a0 0x7f79a40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a40cff80 con 0x7f79a415e870 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7994001670 con 0x7f79a415e870 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a3400640 1 --2- 172.21.15.67:0/825587312 >> v2:172.21.15.169:3300/0 conn(0x7f79a40d16a0 0x7f79a40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 >> v1:172.21.15.67:6789/0 conn(0x7f79a415e870 legacy=0x7f79a40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79840039b0 0x7f7984003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a40cf870 0x7f7984004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7984004d40 0x7f79840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40d2150 con 0x7f7984004d40 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40cfd20 con 0x7f79840039b0 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f798400ba70 con 0x7f79a40cf870 2024-05-30T16:48:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a9200640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7984004d40 0x7f79840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.817+0000 7f79a3e00640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79840039b0 0x7f7984003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.818+0000 7f79a3400640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a40cf870 0x7f7984004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.818+0000 7f79a9200640 1 -- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a40cf870 msgr2=0x7f7984004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.818+0000 7f79a9200640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a40cf870 0x7f7984004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.818+0000 7f79a9200640 1 -- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79840039b0 msgr2=0x7f7984003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.818+0000 7f79a9200640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79840039b0 0x7f7984003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.818+0000 7f79a9200640 1 -- 172.21.15.67:0/825587312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7984003520 con 0x7f7984004d40 2024-05-30T16:48:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.819+0000 7f79a9200640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7984004d40 0x7f79840051c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f799c00ba60 tx=0x7f7984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14867574bcf92987 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:35.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.820+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c030070 con 0x7f7984004d40 2024-05-30T16:48:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.820+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c02ae10 con 0x7f7984004d40 2024-05-30T16:48:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.820+0000 7f79a2a00640 1 -- 172.21.15.67:0/825587312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c00b600 con 0x7f7984004d40 2024-05-30T16:48:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.821+0000 7f79a9c00640 1 -- 172.21.15.67:0/825587312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7984004d40 msgr2=0x7f79840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.821+0000 7f79a9c00640 1 --2- 172.21.15.67:0/825587312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7984004d40 0x7f79840051c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f799c00ba60 tx=0x7f7984005830 comp rx=0 tx=0).stop 2024-05-30T16:48:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.821+0000 7f79a9200640 1 -- 172.21.15.67:0/825587312 reap_dead start 2024-05-30T16:48:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.821+0000 7f79a9c00640 1 -- 172.21.15.67:0/825587312 shutdown_connections 2024-05-30T16:48:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.821+0000 7f79a9c00640 1 -- 172.21.15.67:0/825587312 >> 172.21.15.67:0/825587312 conn(0x7f79a408bfc0 msgr2=0x7f79a40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.821+0000 7f79a9c00640 1 -- 172.21.15.67:0/825587312 shutdown_connections 2024-05-30T16:48:35.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.821+0000 7f79a9c00640 1 -- 172.21.15.67:0/825587312 wait complete. 2024-05-30T16:48:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.823+0000 7f79a9c00640 1 Processor -- start 2024-05-30T16:48:35.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.823+0000 7f79a9c00640 1 -- start start 2024-05-30T16:48:35.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a4152540 0x7f79a41560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a4152920 0x7f79a406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a406d700 0x7f79a41f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40d0820 con 0x7f79a406d700 2024-05-30T16:48:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40d0540 con 0x7f79a4152920 2024-05-30T16:48:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79a40d4fb0 con 0x7f79a4152540 2024-05-30T16:48:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a406d700 0x7f79a41f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a406d700 0x7f79a41f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58072/0 (socket says 172.21.15.67:58072) 2024-05-30T16:48:35.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9200640 1 -- 172.21.15.67:0/800325880 learned_addr learned my addr 172.21.15.67:0/800325880 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:35.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a3400640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a4152920 0x7f79a406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a3e00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a4152540 0x7f79a41560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9200640 1 -- 172.21.15.67:0/800325880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a4152540 msgr2=0x7f79a41560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.824+0000 7f79a9200640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a4152540 0x7f79a41560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.825+0000 7f79a9200640 1 -- 172.21.15.67:0/800325880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a4152920 msgr2=0x7f79a406d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.825+0000 7f79a9200640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a4152920 0x7f79a406d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.825+0000 7f79a9200640 1 -- 172.21.15.67:0/800325880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a40c80b0 con 0x7f79a406d700 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.825+0000 7f79a9200640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a406d700 0x7f79a41f7fb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f799c02bfd0 tx=0x7f799c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.825+0000 7f799be00640 1 -- 172.21.15.67:0/800325880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c00b600 con 0x7f79a406d700 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.825+0000 7f799be00640 1 -- 172.21.15.67:0/800325880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c02ac10 con 0x7f79a406d700 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.825+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a40cb780 con 0x7f79a406d700 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.825+0000 7f799be00640 1 -- 172.21.15.67:0/800325880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c004260 con 0x7f79a406d700 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.825+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a40cbd40 con 0x7f79a406d700 2024-05-30T16:48:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.827+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a40689e0 con 0x7f79a406d700 2024-05-30T16:48:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.827+0000 7f799be00640 1 -- 172.21.15.67:0/800325880 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f799c045030 con 0x7f79a406d700 2024-05-30T16:48:35.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.829+0000 7f799be00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f798006cbc0 0x7f798006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:35.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.829+0000 7f799be00640 1 -- 172.21.15.67:0/800325880 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f799c0abfb0 con 0x7f79a406d700 2024-05-30T16:48:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.829+0000 7f79a3e00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f798006cbc0 0x7f798006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.830+0000 7f79a3e00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f798006cbc0 0x7f798006f080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7994007f90 tx=0x7f7994009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:35.833+0000 7f799be00640 1 -- 172.21.15.67:0/800325880 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f799c04c060 con 0x7f79a406d700 2024-05-30T16:48:36.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.033+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79a40ba500 con 0x7f798006cbc0 2024-05-30T16:48:36.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.050+0000 7f799be00640 1 -- 172.21.15.67:0/800325880 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247387 (secure 0 0 0) 0x7f79a40ba500 con 0x7f798006cbc0 2024-05-30T16:48:36.056 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:36.057 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:48:36.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f798006cbc0 msgr2=0x7f798006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f798006cbc0 0x7f798006f080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7994007f90 tx=0x7f7994009040 comp rx=0 tx=0).stop 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a406d700 msgr2=0x7f79a41f7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a406d700 0x7f79a41f7fb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f799c02bfd0 tx=0x7f799c004730 comp rx=0 tx=0).stop 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 shutdown_connections 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f798006cbc0 0x7f798006f080 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79a406d700 0x7f79a41f7fb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79a4152920 0x7f79a406d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 --2- 172.21.15.67:0/800325880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79a4152540 0x7f79a41560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.056+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 >> 172.21.15.67:0/800325880 conn(0x7f79a408bfc0 msgr2=0x7f79a40b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.057+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 shutdown_connections 2024-05-30T16:48:36.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.057+0000 7f79a9c00640 1 -- 172.21.15.67:0/800325880 wait complete. 2024-05-30T16:48:36.079 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-05-30T16:48:34.086725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279736,"kb_used_data":5316,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754694984,"statfs":{"total":773094113280,"available":772807663616,"internally_reserved":0,"allocated":5443584,"data_stored":2898471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.572212"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570382+0000","last_change":"2024-05-30T16:48:30.584563+0000","last_active":"2024-05-30T16:48:31.570382+0000","last_peered":"2024-05-30T16:48:31.570382+0000","last_clean":"2024-05-30T16:48:31.570382+0000","last_became_active":"2024-05-30T16:48:30.584262+0000","last_became_peered":"2024-05-30T16:48:30.584262+0000","last_unstale":"2024-05-30T16:48:31.570382+0000","last_undegraded":"2024-05-30T16:48:31.570382+0000","last_fullsized":"2024-05-30T16:48:31.570382+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.553111+0000","last_change":"2024-05-30T16:48:29.553111+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.553111+0000","last_undegraded":"2024-05-30T16:48:29.553111+0000","last_fullsized":"2024-05-30T16:48:29.553111+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603004+0000","last_change":"2024-05-30T16:48:30.564821+0000","last_active":"2024-05-30T16:48:31.603004+0000","last_peered":"2024-05-30T16:48:31.603004+0000","last_clean":"2024-05-30T16:48:31.603004+0000","last_became_active":"2024-05-30T16:48:30.564537+0000","last_became_peered":"2024-05-30T16:48:30.564537+0000","last_unstale":"2024-05-30T16:48:31.603004+0000","last_undegraded":"2024-05-30T16:48:31.603004+0000","last_fullsized":"2024-05-30T16:48:31.603004+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:00:29.368856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570614+0000","last_change":"2024-05-30T16:48:30.565013+0000","last_active":"2024-05-30T16:48:31.570614+0000","last_peered":"2024-05-30T16:48:31.570614+0000","last_clean":"2024-05-30T16:48:31.570614+0000","last_became_active":"2024-05-30T16:48:30.564344+0000","last_became_peered":"2024-05-30T16:48:30.564344+0000","last_unstale":"2024-05-30T16:48:31.570614+0000","last_undegraded":"2024-05-30T16:48:31.570614+0000","last_fullsized":"2024-05-30T16:48:31.570614+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:38.774999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.542049+0000","last_change":"2024-05-30T16:48:29.542049+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.542049+0000","last_undegraded":"2024-05-30T16:48:29.542049+0000","last_fullsized":"2024-05-30T16:48:29.542049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600493+0000","last_change":"2024-05-30T16:48:30.561287+0000","last_active":"2024-05-30T16:48:31.600493+0000","last_peered":"2024-05-30T16:48:31.600493+0000","last_clean":"2024-05-30T16:48:31.600493+0000","last_became_active":"2024-05-30T16:48:30.560737+0000","last_became_peered":"2024-05-30T16:48:30.560737+0000","last_unstale":"2024-05-30T16:48:31.600493+0000","last_undegraded":"2024-05-30T16:48:31.600493+0000","last_fullsized":"2024-05-30T16:48:31.600493+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:50:31.061295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603257+0000","last_change":"2024-05-30T16:48:30.587362+0000","last_active":"2024-05-30T16:48:31.603257+0000","last_peered":"2024-05-30T16:48:31.603257+0000","last_clean":"2024-05-30T16:48:31.603257+0000","last_became_active":"2024-05-30T16:48:30.587100+0000","last_became_peered":"2024-05-30T16:48:30.587100+0000","last_unstale":"2024-05-30T16:48:31.603257+0000","last_undegraded":"2024-05-30T16:48:31.603257+0000","last_fullsized":"2024-05-30T16:48:31.603257+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:00.811234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603466+0000","last_change":"2024-05-30T16:48:30.582474+0000","last_active":"2024-05-30T16:48:31.603466+0000","last_peered":"2024-05-30T16:48:31.603466+0000","last_clean":"2024-05-30T16:48:31.603466+0000","last_became_active":"2024-05-30T16:48:30.582148+0000","last_became_peered":"2024-05-30T16:48:30.582148+0000","last_unstale":"2024-05-30T16:48:31.603466+0000","last_undegraded":"2024-05-30T16:48:31.603466+0000","last_fullsized":"2024-05-30T16:48:31.603466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602911+0000","last_change":"2024-05-30T16:48:30.577384+0000","last_active":"2024-05-30T16:48:31.602911+0000","last_peered":"2024-05-30T16:48:31.602911+0000","last_clean":"2024-05-30T16:48:31.602911+0000","last_became_active":"2024-05-30T16:48:30.570714+0000","last_became_peered":"2024-05-30T16:48:30.570714+0000","last_unstale":"2024-05-30T16:48:31.602911+0000","last_undegraded":"2024-05-30T16:48:31.602911+0000","last_fullsized":"2024-05-30T16:48:31.602911+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:05.025187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570620+0000","last_change":"2024-05-30T16:48:30.582165+0000","last_active":"2024-05-30T16:48:31.570620+0000","last_peered":"2024-05-30T16:48:31.570620+0000","last_clean":"2024-05-30T16:48:31.570620+0000","last_became_active":"2024-05-30T16:48:30.581896+0000","last_became_peered":"2024-05-30T16:48:30.581896+0000","last_unstale":"2024-05-30T16:48:31.570620+0000","last_undegraded":"2024-05-30T16:48:31.570620+0000","last_fullsized":"2024-05-30T16:48:31.570620+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.538253+0000","last_change":"2024-05-30T16:48:29.538253+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.538253+0000","last_undegraded":"2024-05-30T16:48:29.538253+0000","last_fullsized":"2024-05-30T16:48:29.538253+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601441+0000","last_change":"2024-05-30T16:48:30.555321+0000","last_active":"2024-05-30T16:48:31.601441+0000","last_peered":"2024-05-30T16:48:31.601441+0000","last_clean":"2024-05-30T16:48:31.601441+0000","last_became_active":"2024-05-30T16:48:30.555012+0000","last_became_peered":"2024-05-30T16:48:30.555012+0000","last_unstale":"2024-05-30T16:48:31.601441+0000","last_undegraded":"2024-05-30T16:48:31.601441+0000","last_fullsized":"2024-05-30T16:48:31.601441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:41.388387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.543234+0000","last_change":"2024-05-30T16:48:29.543234+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.543234+0000","last_undegraded":"2024-05-30T16:48:29.543234+0000","last_fullsized":"2024-05-30T16:48:29.543234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.560080+0000","last_change":"2024-05-30T16:48:29.560080+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.560080+0000","last_undegraded":"2024-05-30T16:48:29.560080+0000","last_fullsized":"2024-05-30T16:48:29.560080+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.552921+0000","last_change":"2024-05-30T16:48:29.552921+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.552921+0000","last_undegraded":"2024-05-30T16:48:29.552921+0000","last_fullsized":"2024-05-30T16:48:29.552921+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.561708+0000","last_change":"2024-05-30T16:48:29.561708+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.561708+0000","last_undegraded":"2024-05-30T16:48:29.561708+0000","last_fullsized":"2024-05-30T16:48:29.561708+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.555816+0000","last_change":"2024-05-30T16:48:29.555816+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.555816+0000","last_undegraded":"2024-05-30T16:48:29.555816+0000","last_fullsized":"2024-05-30T16:48:29.555816+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.540205+0000","last_change":"2024-05-30T16:48:29.540205+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.540205+0000","last_undegraded":"2024-05-30T16:48:29.540205+0000","last_fullsized":"2024-05-30T16:48:29.540205+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.541235+0000","last_change":"2024-05-30T16:48:29.541235+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.541235+0000","last_undegraded":"2024-05-30T16:48:29.541235+0000","last_fullsized":"2024-05-30T16:48:29.541235+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601819+0000","last_change":"2024-05-30T16:48:30.563868+0000","last_active":"2024-05-30T16:48:31.601819+0000","last_peered":"2024-05-30T16:48:31.601819+0000","last_clean":"2024-05-30T16:48:31.601819+0000","last_became_active":"2024-05-30T16:48:30.563554+0000","last_became_peered":"2024-05-30T16:48:30.563554+0000","last_unstale":"2024-05-30T16:48:31.601819+0000","last_undegraded":"2024-05-30T16:48:31.601819+0000","last_fullsized":"2024-05-30T16:48:31.601819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:47.906916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.547433+0000","last_change":"2024-05-30T16:48:29.547433+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.547433+0000","last_undegraded":"2024-05-30T16:48:29.547433+0000","last_fullsized":"2024-05-30T16:48:29.547433+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.548367+0000","last_change":"2024-05-30T16:48:29.548367+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.548367+0000","last_undegraded":"2024-05-30T16:48:29.548367+0000","last_fullsized":"2024-05-30T16:48:29.548367+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.565566+0000","last_change":"2024-05-30T16:48:29.565566+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.565566+0000","last_undegraded":"2024-05-30T16:48:29.565566+0000","last_fullsized":"2024-05-30T16:48:29.565566+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.541213+0000","last_change":"2024-05-30T16:48:29.541213+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.541213+0000","last_undegraded":"2024-05-30T16:48:29.541213+0000","last_fullsized":"2024-05-30T16:48:29.541213+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.552035+0000","last_change":"2024-05-30T16:48:29.552035+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.552035+0000","last_undegraded":"2024-05-30T16:48:29.552035+0000","last_fullsized":"2024-05-30T16:48:29.552035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571075+0000","last_change":"2024-05-30T16:48:30.582495+0000","last_active":"2024-05-30T16:48:31.571075+0000","last_peered":"2024-05-30T16:48:31.571075+0000","last_clean":"2024-05-30T16:48:31.571075+0000","last_became_active":"2024-05-30T16:48:30.582199+0000","last_became_peered":"2024-05-30T16:48:30.582199+0000","last_unstale":"2024-05-30T16:48:31.571075+0000","last_undegraded":"2024-05-30T16:48:31.571075+0000","last_fullsized":"2024-05-30T16:48:31.571075+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:18.447770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.556706+0000","last_change":"2024-05-30T16:48:29.556706+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.556706+0000","last_undegraded":"2024-05-30T16:48:29.556706+0000","last_fullsized":"2024-05-30T16:48:29.556706+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.560620+0000","last_change":"2024-05-30T16:48:29.560620+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.560620+0000","last_undegraded":"2024-05-30T16:48:29.560620+0000","last_fullsized":"2024-05-30T16:48:29.560620+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.542690+0000","last_change":"2024-05-30T16:48:29.542690+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.542690+0000","last_undegraded":"2024-05-30T16:48:29.542690+0000","last_fullsized":"2024-05-30T16:48:29.542690+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.539188+0000","last_change":"2024-05-30T16:48:29.539188+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.539188+0000","last_undegraded":"2024-05-30T16:48:29.539188+0000","last_fullsized":"2024-05-30T16:48:29.539188+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.555599+0000","last_change":"2024-05-30T16:48:29.555599+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.555599+0000","last_undegraded":"2024-05-30T16:48:29.555599+0000","last_fullsized":"2024-05-30T16:48:29.555599+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.573884+0000","last_change":"2024-05-30T16:48:29.572429+0000","last_active":"2024-05-30T16:48:29.573884+0000","last_peered":"2024-05-30T16:48:29.573884+0000","last_clean":"2024-05-30T16:48:29.573884+0000","last_became_active":"2024-05-30T16:48:29.572105+0000","last_became_peered":"2024-05-30T16:48:29.572105+0000","last_unstale":"2024-05-30T16:48:29.573884+0000","last_undegraded":"2024-05-30T16:48:29.573884+0000","last_fullsized":"2024-05-30T16:48:29.573884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:53.214944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.560040+0000","last_change":"2024-05-30T16:48:29.560040+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.560040+0000","last_undegraded":"2024-05-30T16:48:29.560040+0000","last_fullsized":"2024-05-30T16:48:29.560040+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.573974+0000","last_change":"2024-05-30T16:48:29.572447+0000","last_active":"2024-05-30T16:48:29.573974+0000","last_peered":"2024-05-30T16:48:29.573974+0000","last_clean":"2024-05-30T16:48:29.573974+0000","last_became_active":"2024-05-30T16:48:29.572102+0000","last_became_peered":"2024-05-30T16:48:29.572102+0000","last_unstale":"2024-05-30T16:48:29.573974+0000","last_undegraded":"2024-05-30T16:48:29.573974+0000","last_fullsized":"2024-05-30T16:48:29.573974+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:49.145998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571330+0000","last_change":"2024-05-30T16:48:30.570556+0000","last_active":"2024-05-30T16:48:31.571330+0000","last_peered":"2024-05-30T16:48:31.571330+0000","last_clean":"2024-05-30T16:48:31.571330+0000","last_became_active":"2024-05-30T16:48:30.569419+0000","last_became_peered":"2024-05-30T16:48:30.569419+0000","last_unstale":"2024-05-30T16:48:31.571330+0000","last_undegraded":"2024-05-30T16:48:31.571330+0000","last_fullsized":"2024-05-30T16:48:31.571330+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:33.900129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568268+0000","last_change":"2024-05-30T16:48:29.563603+0000","last_active":"2024-05-30T16:48:29.568268+0000","last_peered":"2024-05-30T16:48:29.568268+0000","last_clean":"2024-05-30T16:48:29.568268+0000","last_became_active":"2024-05-30T16:48:29.563260+0000","last_became_peered":"2024-05-30T16:48:29.563260+0000","last_unstale":"2024-05-30T16:48:29.568268+0000","last_undegraded":"2024-05-30T16:48:29.568268+0000","last_fullsized":"2024-05-30T16:48:29.568268+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:52:26.866201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.549866+0000","last_change":"2024-05-30T16:48:29.549866+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.549866+0000","last_undegraded":"2024-05-30T16:48:29.549866+0000","last_fullsized":"2024-05-30T16:48:29.549866+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.531350+0000","last_change":"2024-05-30T16:48:28.531350+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.531350+0000","last_undegraded":"2024-05-30T16:48:28.531350+0000","last_fullsized":"2024-05-30T16:48:28.531350+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.558514+0000","last_change":"2024-05-30T16:48:29.558514+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.558514+0000","last_undegraded":"2024-05-30T16:48:29.558514+0000","last_fullsized":"2024-05-30T16:48:29.558514+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568452+0000","last_change":"2024-05-30T16:48:29.561932+0000","last_active":"2024-05-30T16:48:29.568452+0000","last_peered":"2024-05-30T16:48:29.568452+0000","last_clean":"2024-05-30T16:48:29.568452+0000","last_became_active":"2024-05-30T16:48:29.561609+0000","last_became_peered":"2024-05-30T16:48:29.561609+0000","last_unstale":"2024-05-30T16:48:29.568452+0000","last_undegraded":"2024-05-30T16:48:29.568452+0000","last_fullsized":"2024-05-30T16:48:29.568452+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:54:21.183349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602281+0000","last_change":"2024-05-30T16:48:30.570083+0000","last_active":"2024-05-30T16:48:31.602281+0000","last_peered":"2024-05-30T16:48:31.602281+0000","last_clean":"2024-05-30T16:48:31.602281+0000","last_became_active":"2024-05-30T16:48:30.568578+0000","last_became_peered":"2024-05-30T16:48:30.568578+0000","last_unstale":"2024-05-30T16:48:31.602281+0000","last_undegraded":"2024-05-30T16:48:31.602281+0000","last_fullsized":"2024-05-30T16:48:31.602281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:51:20.993001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.569697+0000","last_change":"2024-05-30T16:48:29.568131+0000","last_active":"2024-05-30T16:48:29.569697+0000","last_peered":"2024-05-30T16:48:29.569697+0000","last_clean":"2024-05-30T16:48:29.569697+0000","last_became_active":"2024-05-30T16:48:29.567819+0000","last_became_peered":"2024-05-30T16:48:29.567819+0000","last_unstale":"2024-05-30T16:48:29.569697+0000","last_undegraded":"2024-05-30T16:48:29.569697+0000","last_fullsized":"2024-05-30T16:48:29.569697+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:46.064182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600660+0000","last_change":"2024-05-30T16:48:30.561051+0000","last_active":"2024-05-30T16:48:31.600660+0000","last_peered":"2024-05-30T16:48:31.600660+0000","last_clean":"2024-05-30T16:48:31.600660+0000","last_became_active":"2024-05-30T16:48:30.560423+0000","last_became_peered":"2024-05-30T16:48:30.560423+0000","last_unstale":"2024-05-30T16:48:31.600660+0000","last_undegraded":"2024-05-30T16:48:31.600660+0000","last_fullsized":"2024-05-30T16:48:31.600660+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:02.774077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569744+0000","last_change":"2024-05-30T16:48:29.566803+0000","last_active":"2024-05-30T16:48:31.569744+0000","last_peered":"2024-05-30T16:48:31.569744+0000","last_clean":"2024-05-30T16:48:31.569744+0000","last_became_active":"2024-05-30T16:48:29.563938+0000","last_became_peered":"2024-05-30T16:48:29.563938+0000","last_unstale":"2024-05-30T16:48:31.569744+0000","last_undegraded":"2024-05-30T16:48:31.569744+0000","last_fullsized":"2024-05-30T16:48:31.569744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:46.892080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.548745+0000","last_change":"2024-05-30T16:48:29.548745+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.548745+0000","last_undegraded":"2024-05-30T16:48:29.548745+0000","last_fullsized":"2024-05-30T16:48:29.548745+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600710+0000","last_change":"2024-05-30T16:48:29.557714+0000","last_active":"2024-05-30T16:48:31.600710+0000","last_peered":"2024-05-30T16:48:31.600710+0000","last_clean":"2024-05-30T16:48:31.600710+0000","last_became_active":"2024-05-30T16:48:29.557329+0000","last_became_peered":"2024-05-30T16:48:29.557329+0000","last_unstale":"2024-05-30T16:48:31.600710+0000","last_undegraded":"2024-05-30T16:48:31.600710+0000","last_fullsized":"2024-05-30T16:48:31.600710+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:55:42.557260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.559496+0000","last_change":"2024-05-30T16:48:29.559496+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.559496+0000","last_undegraded":"2024-05-30T16:48:29.559496+0000","last_fullsized":"2024-05-30T16:48:29.559496+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.567480+0000","last_change":"2024-05-30T16:48:29.562616+0000","last_active":"2024-05-30T16:48:29.567480+0000","last_peered":"2024-05-30T16:48:29.567480+0000","last_clean":"2024-05-30T16:48:29.567480+0000","last_became_active":"2024-05-30T16:48:29.562279+0000","last_became_peered":"2024-05-30T16:48:29.562279+0000","last_unstale":"2024-05-30T16:48:29.567480+0000","last_undegraded":"2024-05-30T16:48:29.567480+0000","last_fullsized":"2024-05-30T16:48:29.567480+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:09:58.486024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603628+0000","last_change":"2024-05-30T16:48:29.565569+0000","last_active":"2024-05-30T16:48:31.603628+0000","last_peered":"2024-05-30T16:48:31.603628+0000","last_clean":"2024-05-30T16:48:31.603628+0000","last_became_active":"2024-05-30T16:48:29.565276+0000","last_became_peered":"2024-05-30T16:48:29.565276+0000","last_unstale":"2024-05-30T16:48:31.603628+0000","last_undegraded":"2024-05-30T16:48:31.603628+0000","last_fullsized":"2024-05-30T16:48:31.603628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:02.208362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.541170+0000","last_change":"2024-05-30T16:48:29.541170+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.541170+0000","last_undegraded":"2024-05-30T16:48:29.541170+0000","last_fullsized":"2024-05-30T16:48:29.541170+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.534129+0000","last_change":"2024-05-30T16:48:28.534129+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.534129+0000","last_undegraded":"2024-05-30T16:48:28.534129+0000","last_fullsized":"2024-05-30T16:48:28.534129+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.567260+0000","last_change":"2024-05-30T16:48:29.564330+0000","last_active":"2024-05-30T16:48:29.567260+0000","last_peered":"2024-05-30T16:48:29.567260+0000","last_clean":"2024-05-30T16:48:29.567260+0000","last_became_active":"2024-05-30T16:48:29.564050+0000","last_became_peered":"2024-05-30T16:48:29.564050+0000","last_unstale":"2024-05-30T16:48:29.567260+0000","last_undegraded":"2024-05-30T16:48:29.567260+0000","last_fullsized":"2024-05-30T16:48:29.567260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:17:45.064827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.554849+0000","last_change":"2024-05-30T16:48:29.554849+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.554849+0000","last_undegraded":"2024-05-30T16:48:29.554849+0000","last_fullsized":"2024-05-30T16:48:29.554849+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.560225+0000","last_change":"2024-05-30T16:48:29.556834+0000","last_active":"2024-05-30T16:48:29.560225+0000","last_peered":"2024-05-30T16:48:29.560225+0000","last_clean":"2024-05-30T16:48:29.560225+0000","last_became_active":"2024-05-30T16:48:29.556546+0000","last_became_peered":"2024-05-30T16:48:29.556546+0000","last_unstale":"2024-05-30T16:48:29.560225+0000","last_undegraded":"2024-05-30T16:48:29.560225+0000","last_fullsized":"2024-05-30T16:48:29.560225+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:01.668836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602679+0000","last_change":"2024-05-30T16:48:30.574973+0000","last_active":"2024-05-30T16:48:31.602679+0000","last_peered":"2024-05-30T16:48:31.602679+0000","last_clean":"2024-05-30T16:48:31.602679+0000","last_became_active":"2024-05-30T16:48:30.574679+0000","last_became_peered":"2024-05-30T16:48:30.574679+0000","last_unstale":"2024-05-30T16:48:31.602679+0000","last_undegraded":"2024-05-30T16:48:31.602679+0000","last_fullsized":"2024-05-30T16:48:31.602679+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:02:00.674835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.528356+0000","last_change":"2024-05-30T16:48:28.528356+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.528356+0000","last_undegraded":"2024-05-30T16:48:28.528356+0000","last_fullsized":"2024-05-30T16:48:28.528356+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.543580+0000","last_change":"2024-05-30T16:48:29.543580+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.543580+0000","last_undegraded":"2024-05-30T16:48:29.543580+0000","last_fullsized":"2024-05-30T16:48:29.543580+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.560395+0000","last_change":"2024-05-30T16:48:29.557641+0000","last_active":"2024-05-30T16:48:29.560395+0000","last_peered":"2024-05-30T16:48:29.560395+0000","last_clean":"2024-05-30T16:48:29.560395+0000","last_became_active":"2024-05-30T16:48:29.556424+0000","last_became_peered":"2024-05-30T16:48:29.556424+0000","last_unstale":"2024-05-30T16:48:29.560395+0000","last_undegraded":"2024-05-30T16:48:29.560395+0000","last_fullsized":"2024-05-30T16:48:29.560395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:44.700516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":22,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600828+0000","last_change":"2024-05-30T16:48:29.568303+0000","last_active":"2024-05-30T16:48:31.600828+0000","last_peered":"2024-05-30T16:48:31.600828+0000","last_clean":"2024-05-30T16:48:31.600828+0000","last_became_active":"2024-05-30T16:48:29.567991+0000","last_became_peered":"2024-05-30T16:48:29.567991+0000","last_unstale":"2024-05-30T16:48:31.600828+0000","last_undegraded":"2024-05-30T16:48:31.600828+0000","last_fullsized":"2024-05-30T16:48:31.600828+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:31.450890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.540700+0000","last_change":"2024-05-30T16:48:29.540700+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.540700+0000","last_undegraded":"2024-05-30T16:48:29.540700+0000","last_fullsized":"2024-05-30T16:48:29.540700+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.553284+0000","last_change":"2024-05-30T16:48:29.547415+0000","last_active":"2024-05-30T16:48:29.553284+0000","last_peered":"2024-05-30T16:48:29.553284+0000","last_clean":"2024-05-30T16:48:29.553284+0000","last_became_active":"2024-05-30T16:48:29.547133+0000","last_became_peered":"2024-05-30T16:48:29.547133+0000","last_unstale":"2024-05-30T16:48:29.553284+0000","last_undegraded":"2024-05-30T16:48:29.553284+0000","last_fullsized":"2024-05-30T16:48:29.553284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:13.553689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603817+0000","last_change":"2024-05-30T16:48:30.588261+0000","last_active":"2024-05-30T16:48:31.603817+0000","last_peered":"2024-05-30T16:48:31.603817+0000","last_clean":"2024-05-30T16:48:31.603817+0000","last_became_active":"2024-05-30T16:48:30.587964+0000","last_became_peered":"2024-05-30T16:48:30.587964+0000","last_unstale":"2024-05-30T16:48:31.603817+0000","last_undegraded":"2024-05-30T16:48:31.603817+0000","last_fullsized":"2024-05-30T16:48:31.603817+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.571432+0000","last_change":"2024-05-30T16:48:29.569871+0000","last_active":"2024-05-30T16:48:29.571432+0000","last_peered":"2024-05-30T16:48:29.571432+0000","last_clean":"2024-05-30T16:48:29.571432+0000","last_became_active":"2024-05-30T16:48:29.569572+0000","last_became_peered":"2024-05-30T16:48:29.569572+0000","last_unstale":"2024-05-30T16:48:29.571432+0000","last_undegraded":"2024-05-30T16:48:29.571432+0000","last_fullsized":"2024-05-30T16:48:29.571432+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:23.963030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.557004+0000","last_change":"2024-05-30T16:48:29.557004+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.557004+0000","last_undegraded":"2024-05-30T16:48:29.557004+0000","last_fullsized":"2024-05-30T16:48:29.557004+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570987+0000","last_change":"2024-05-30T16:48:29.556896+0000","last_active":"2024-05-30T16:48:31.570987+0000","last_peered":"2024-05-30T16:48:31.570987+0000","last_clean":"2024-05-30T16:48:31.570987+0000","last_became_active":"2024-05-30T16:48:29.556569+0000","last_became_peered":"2024-05-30T16:48:29.556569+0000","last_unstale":"2024-05-30T16:48:31.570987+0000","last_undegraded":"2024-05-30T16:48:31.570987+0000","last_fullsized":"2024-05-30T16:48:31.570987+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:27.482686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.558489+0000","last_change":"2024-05-30T16:48:29.558489+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.558489+0000","last_undegraded":"2024-05-30T16:48:29.558489+0000","last_fullsized":"2024-05-30T16:48:29.558489+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.569070+0000","last_change":"2024-05-30T16:48:29.565727+0000","last_active":"2024-05-30T16:48:29.569070+0000","last_peered":"2024-05-30T16:48:29.569070+0000","last_clean":"2024-05-30T16:48:29.569070+0000","last_became_active":"2024-05-30T16:48:29.565445+0000","last_became_peered":"2024-05-30T16:48:29.565445+0000","last_unstale":"2024-05-30T16:48:29.569070+0000","last_undegraded":"2024-05-30T16:48:29.569070+0000","last_fullsized":"2024-05-30T16:48:29.569070+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:38:06.238035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.540461+0000","last_change":"2024-05-30T16:48:29.540461+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.540461+0000","last_undegraded":"2024-05-30T16:48:29.540461+0000","last_fullsized":"2024-05-30T16:48:29.540461+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568475+0000","last_change":"2024-05-30T16:48:29.562648+0000","last_active":"2024-05-30T16:48:29.568475+0000","last_peered":"2024-05-30T16:48:29.568475+0000","last_clean":"2024-05-30T16:48:29.568475+0000","last_became_active":"2024-05-30T16:48:29.562380+0000","last_became_peered":"2024-05-30T16:48:29.562380+0000","last_unstale":"2024-05-30T16:48:29.568475+0000","last_undegraded":"2024-05-30T16:48:29.568475+0000","last_fullsized":"2024-05-30T16:48:29.568475+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:36.649081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.551258+0000","last_change":"2024-05-30T16:48:29.551258+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.551258+0000","last_undegraded":"2024-05-30T16:48:29.551258+0000","last_fullsized":"2024-05-30T16:48:29.551258+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569935+0000","last_change":"2024-05-30T16:48:29.564622+0000","last_active":"2024-05-30T16:48:31.569935+0000","last_peered":"2024-05-30T16:48:31.569935+0000","last_clean":"2024-05-30T16:48:31.569935+0000","last_became_active":"2024-05-30T16:48:29.564294+0000","last_became_peered":"2024-05-30T16:48:29.564294+0000","last_unstale":"2024-05-30T16:48:31.569935+0000","last_undegraded":"2024-05-30T16:48:31.569935+0000","last_fullsized":"2024-05-30T16:48:31.569935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:01.759242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.538084+0000","last_change":"2024-05-30T16:48:29.538084+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.538084+0000","last_undegraded":"2024-05-30T16:48:29.538084+0000","last_fullsized":"2024-05-30T16:48:29.538084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601305+0000","last_change":"2024-05-30T16:48:29.565513+0000","last_active":"2024-05-30T16:48:31.601305+0000","last_peered":"2024-05-30T16:48:31.601305+0000","last_clean":"2024-05-30T16:48:31.601305+0000","last_became_active":"2024-05-30T16:48:29.565019+0000","last_became_peered":"2024-05-30T16:48:29.565019+0000","last_unstale":"2024-05-30T16:48:31.601305+0000","last_undegraded":"2024-05-30T16:48:31.601305+0000","last_fullsized":"2024-05-30T16:48:31.601305+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:43.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601161+0000","last_change":"2024-05-30T16:48:30.554107+0000","last_active":"2024-05-30T16:48:31.601161+0000","last_peered":"2024-05-30T16:48:31.601161+0000","last_clean":"2024-05-30T16:48:31.601161+0000","last_became_active":"2024-05-30T16:48:30.553499+0000","last_became_peered":"2024-05-30T16:48:30.553499+0000","last_unstale":"2024-05-30T16:48:31.601161+0000","last_undegraded":"2024-05-30T16:48:31.601161+0000","last_fullsized":"2024-05-30T16:48:31.601161+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:28.497521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.528249+0000","last_change":"2024-05-30T16:48:28.528249+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.528249+0000","last_undegraded":"2024-05-30T16:48:28.528249+0000","last_fullsized":"2024-05-30T16:48:28.528249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.559430+0000","last_change":"2024-05-30T16:48:29.559430+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.559430+0000","last_undegraded":"2024-05-30T16:48:29.559430+0000","last_fullsized":"2024-05-30T16:48:29.559430+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568589+0000","last_change":"2024-05-30T16:48:29.563607+0000","last_active":"2024-05-30T16:48:29.568589+0000","last_peered":"2024-05-30T16:48:29.568589+0000","last_clean":"2024-05-30T16:48:29.568589+0000","last_became_active":"2024-05-30T16:48:29.563281+0000","last_became_peered":"2024-05-30T16:48:29.563281+0000","last_unstale":"2024-05-30T16:48:29.568589+0000","last_undegraded":"2024-05-30T16:48:29.568589+0000","last_fullsized":"2024-05-30T16:48:29.568589+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:29.118183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603410+0000","last_change":"2024-05-30T16:48:30.577542+0000","last_active":"2024-05-30T16:48:31.603410+0000","last_peered":"2024-05-30T16:48:31.603410+0000","last_clean":"2024-05-30T16:48:31.603410+0000","last_became_active":"2024-05-30T16:48:30.576718+0000","last_became_peered":"2024-05-30T16:48:30.576718+0000","last_unstale":"2024-05-30T16:48:31.603410+0000","last_undegraded":"2024-05-30T16:48:31.603410+0000","last_fullsized":"2024-05-30T16:48:31.603410+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:24.028012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.533009+0000","last_change":"2024-05-30T16:48:28.533009+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.533009+0000","last_undegraded":"2024-05-30T16:48:28.533009+0000","last_fullsized":"2024-05-30T16:48:28.533009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T16:48:29.513818+0000","last_change":"2024-05-30T16:48:29.513818+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.513818+0000","last_undegraded":"2024-05-30T16:48:29.513818+0000","last_fullsized":"2024-05-30T16:48:29.513818+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.568163+0000","last_change":"2024-05-30T16:48:29.563508+0000","last_active":"2024-05-30T16:48:29.568163+0000","last_peered":"2024-05-30T16:48:29.568163+0000","last_clean":"2024-05-30T16:48:29.568163+0000","last_became_active":"2024-05-30T16:48:29.563206+0000","last_became_peered":"2024-05-30T16:48:29.563206+0000","last_unstale":"2024-05-30T16:48:29.568163+0000","last_undegraded":"2024-05-30T16:48:29.568163+0000","last_fullsized":"2024-05-30T16:48:29.568163+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:49.145998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.539746+0000","last_change":"2024-05-30T16:48:29.539746+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.539746+0000","last_undegraded":"2024-05-30T16:48:29.539746+0000","last_fullsized":"2024-05-30T16:48:29.539746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-30T16:48:28.528519+0000","last_change":"2024-05-30T16:48:28.528519+0000","last_active":"2024-05-30T16:48:28.498983+0000","last_peered":"2024-05-30T16:48:28.498983+0000","last_clean":"2024-05-30T16:48:28.498983+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:28.528519+0000","last_undegraded":"2024-05-30T16:48:28.528519+0000","last_fullsized":"2024-05-30T16:48:28.528519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.539795+0000","last_change":"2024-05-30T16:48:29.539795+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.539795+0000","last_undegraded":"2024-05-30T16:48:29.539795+0000","last_fullsized":"2024-05-30T16:48:29.539795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.571255+0000","last_change":"2024-05-30T16:48:29.569525+0000","last_active":"2024-05-30T16:48:29.571255+0000","last_peered":"2024-05-30T16:48:29.571255+0000","last_clean":"2024-05-30T16:48:29.571255+0000","last_became_active":"2024-05-30T16:48:29.568636+0000","last_became_peered":"2024-05-30T16:48:29.568636+0000","last_unstale":"2024-05-30T16:48:29.571255+0000","last_undegraded":"2024-05-30T16:48:29.571255+0000","last_fullsized":"2024-05-30T16:48:29.571255+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:33.870945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569918+0000","last_change":"2024-05-30T16:48:30.553665+0000","last_active":"2024-05-30T16:48:31.569918+0000","last_peered":"2024-05-30T16:48:31.569918+0000","last_clean":"2024-05-30T16:48:31.569918+0000","last_became_active":"2024-05-30T16:48:30.552512+0000","last_became_peered":"2024-05-30T16:48:30.552512+0000","last_unstale":"2024-05-30T16:48:31.569918+0000","last_undegraded":"2024-05-30T16:48:31.569918+0000","last_fullsized":"2024-05-30T16:48:31.569918+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-30T16:48:29.573789+0000","last_change":"2024-05-30T16:48:29.572007+0000","last_active":"2024-05-30T16:48:29.573789+0000","last_peered":"2024-05-30T16:48:29.573789+0000","last_clean":"2024-05-30T16:48:29.573789+0000","last_became_active":"2024-05-30T16:48:29.571652+0000","last_became_peered":"2024-05-30T16:48:29.571652+0000","last_unstale":"2024-05-30T16:48:29.573789+0000","last_undegraded":"2024-05-30T16:48:29.573789+0000","last_fullsized":"2024-05-30T16:48:29.573789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:17.171552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-05-30T16:48:29.553680+0000","last_change":"2024-05-30T16:48:29.553680+0000","last_active":"2024-05-30T16:48:29.513818+0000","last_peered":"2024-05-30T16:48:29.513818+0000","last_clean":"2024-05-30T16:48:29.513818+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T16:48:29.553680+0000","last_undegraded":"2024-05-30T16:48:29.553680+0000","last_fullsized":"2024-05-30T16:48:29.553680+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600934+0000","last_change":"2024-05-30T16:48:29.554465+0000","last_active":"2024-05-30T16:48:31.600934+0000","last_peered":"2024-05-30T16:48:31.600934+0000","last_clean":"2024-05-30T16:48:31.600934+0000","last_became_active":"2024-05-30T16:48:29.554117+0000","last_became_peered":"2024-05-30T16:48:29.554117+0000","last_unstale":"2024-05-30T16:48:31.600934+0000","last_undegraded":"2024-05-30T16:48:31.600934+0000","last_fullsized":"2024-05-30T16:48:31.600934+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:30.281975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603615+0000","last_change":"2024-05-30T16:48:30.585362+0000","last_active":"2024-05-30T16:48:31.603615+0000","last_peered":"2024-05-30T16:48:31.603615+0000","last_clean":"2024-05-30T16:48:31.603615+0000","last_became_active":"2024-05-30T16:48:30.583719+0000","last_became_peered":"2024-05-30T16:48:30.583719+0000","last_unstale":"2024-05-30T16:48:31.603615+0000","last_undegraded":"2024-05-30T16:48:31.603615+0000","last_fullsized":"2024-05-30T16:48:31.603615+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570531+0000","last_change":"2024-05-30T16:48:29.558681+0000","last_active":"2024-05-30T16:48:31.570531+0000","last_peered":"2024-05-30T16:48:31.570531+0000","last_clean":"2024-05-30T16:48:31.570531+0000","last_became_active":"2024-05-30T16:48:29.558149+0000","last_became_peered":"2024-05-30T16:48:29.558149+0000","last_unstale":"2024-05-30T16:48:31.570531+0000","last_undegraded":"2024-05-30T16:48:31.570531+0000","last_fullsized":"2024-05-30T16:48:31.570531+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:40.373552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":215,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-30T16:48:28.535382+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:48:28.535382+0000","last_peered":"2024-05-30T16:48:28.535382+0000","last_clean":"2024-05-30T16:48:28.535382+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:48:28.535382+0000","last_undegraded":"2024-05-30T16:48:28.535382+0000","last_fullsized":"2024-05-30T16:48:28.535382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":209085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":532480,"data_stored":215352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335412,"statfs":{"total":96636764160,"available":96599461888,"internally_reserved":0,"allocated":1114112,"data_stored":799453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35896,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335944,"statfs":{"total":96636764160,"available":96600006656,"internally_reserved":0,"allocated":561152,"data_stored":228497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36472,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335368,"statfs":{"total":96636764160,"available":96599416832,"internally_reserved":0,"allocated":1155072,"data_stored":818829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:48:36.083 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3f is not active+clean 2024-05-30T16:48:36.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T16:48:29.513818+0000', 'last_change': '2024-05-30T16:48:29.513818+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.513818+0000', 'last_undegraded': '2024-05-30T16:48:29.513818+0000', 'last_fullsized': '2024-05-30T16:48:29.513818+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T16:48:36.083 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3d is not active+clean 2024-05-30T16:48:36.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.553111+0000', 'last_change': '2024-05-30T16:48:29.553111+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.553111+0000', 'last_undegraded': '2024-05-30T16:48:29.553111+0000', 'last_fullsized': '2024-05-30T16:48:29.553111+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:48:36.083 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3a is not active+clean 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.542049+0000', 'last_change': '2024-05-30T16:48:29.542049+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.542049+0000', 'last_undegraded': '2024-05-30T16:48:29.542049+0000', 'last_fullsized': '2024-05-30T16:48:29.542049+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:PG 7.39 is not active+clean 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T16:48:29.513818+0000', 'last_change': '2024-05-30T16:48:29.513818+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.513818+0000', 'last_undegraded': '2024-05-30T16:48:29.513818+0000', 'last_fullsized': '2024-05-30T16:48:29.513818+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:PG 7.33 is not active+clean 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.538253+0000', 'last_change': '2024-05-30T16:48:29.538253+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.538253+0000', 'last_undegraded': '2024-05-30T16:48:29.538253+0000', 'last_fullsized': '2024-05-30T16:48:29.538253+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:PG 7.31 is not active+clean 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.543234+0000', 'last_change': '2024-05-30T16:48:29.543234+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.543234+0000', 'last_undegraded': '2024-05-30T16:48:29.543234+0000', 'last_fullsized': '2024-05-30T16:48:29.543234+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:PG 7.30 is not active+clean 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.560080+0000', 'last_change': '2024-05-30T16:48:29.560080+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.560080+0000', 'last_undegraded': '2024-05-30T16:48:29.560080+0000', 'last_fullsized': '2024-05-30T16:48:29.560080+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2f is not active+clean 2024-05-30T16:48:36.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.552921+0000', 'last_change': '2024-05-30T16:48:29.552921+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.552921+0000', 'last_undegraded': '2024-05-30T16:48:29.552921+0000', 'last_fullsized': '2024-05-30T16:48:29.552921+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2e is not active+clean 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.561708+0000', 'last_change': '2024-05-30T16:48:29.561708+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.561708+0000', 'last_undegraded': '2024-05-30T16:48:29.561708+0000', 'last_fullsized': '2024-05-30T16:48:29.561708+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2d is not active+clean 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.555816+0000', 'last_change': '2024-05-30T16:48:29.555816+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.555816+0000', 'last_undegraded': '2024-05-30T16:48:29.555816+0000', 'last_fullsized': '2024-05-30T16:48:29.555816+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2c is not active+clean 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.540205+0000', 'last_change': '2024-05-30T16:48:29.540205+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.540205+0000', 'last_undegraded': '2024-05-30T16:48:29.540205+0000', 'last_fullsized': '2024-05-30T16:48:29.540205+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2b is not active+clean 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.541235+0000', 'last_change': '2024-05-30T16:48:29.541235+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.541235+0000', 'last_undegraded': '2024-05-30T16:48:29.541235+0000', 'last_fullsized': '2024-05-30T16:48:29.541235+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.29 is not active+clean 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.547433+0000', 'last_change': '2024-05-30T16:48:29.547433+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.547433+0000', 'last_undegraded': '2024-05-30T16:48:29.547433+0000', 'last_fullsized': '2024-05-30T16:48:29.547433+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.28 is not active+clean 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.548367+0000', 'last_change': '2024-05-30T16:48:29.548367+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.548367+0000', 'last_undegraded': '2024-05-30T16:48:29.548367+0000', 'last_fullsized': '2024-05-30T16:48:29.548367+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.27 is not active+clean 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.565566+0000', 'last_change': '2024-05-30T16:48:29.565566+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.565566+0000', 'last_undegraded': '2024-05-30T16:48:29.565566+0000', 'last_fullsized': '2024-05-30T16:48:29.565566+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:PG 7.26 is not active+clean 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.541213+0000', 'last_change': '2024-05-30T16:48:29.541213+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.541213+0000', 'last_undegraded': '2024-05-30T16:48:29.541213+0000', 'last_fullsized': '2024-05-30T16:48:29.541213+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:PG 7.25 is not active+clean 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.552035+0000', 'last_change': '2024-05-30T16:48:29.552035+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.552035+0000', 'last_undegraded': '2024-05-30T16:48:29.552035+0000', 'last_fullsized': '2024-05-30T16:48:29.552035+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:PG 7.23 is not active+clean 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.556706+0000', 'last_change': '2024-05-30T16:48:29.556706+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.556706+0000', 'last_undegraded': '2024-05-30T16:48:29.556706+0000', 'last_fullsized': '2024-05-30T16:48:29.556706+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:PG 7.22 is not active+clean 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.560620+0000', 'last_change': '2024-05-30T16:48:29.560620+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.560620+0000', 'last_undegraded': '2024-05-30T16:48:29.560620+0000', 'last_fullsized': '2024-05-30T16:48:29.560620+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:PG 7.21 is not active+clean 2024-05-30T16:48:36.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.542690+0000', 'last_change': '2024-05-30T16:48:29.542690+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.542690+0000', 'last_undegraded': '2024-05-30T16:48:29.542690+0000', 'last_fullsized': '2024-05-30T16:48:29.542690+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:PG 7.20 is not active+clean 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.539188+0000', 'last_change': '2024-05-30T16:48:29.539188+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.539188+0000', 'last_undegraded': '2024-05-30T16:48:29.539188+0000', 'last_fullsized': '2024-05-30T16:48:29.539188+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1e is not active+clean 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.555599+0000', 'last_change': '2024-05-30T16:48:29.555599+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.555599+0000', 'last_undegraded': '2024-05-30T16:48:29.555599+0000', 'last_fullsized': '2024-05-30T16:48:29.555599+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1f is not active+clean 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.560040+0000', 'last_change': '2024-05-30T16:48:29.560040+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.560040+0000', 'last_undegraded': '2024-05-30T16:48:29.560040+0000', 'last_fullsized': '2024-05-30T16:48:29.560040+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1d is not active+clean 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.549866+0000', 'last_change': '2024-05-30T16:48:29.549866+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.549866+0000', 'last_undegraded': '2024-05-30T16:48:29.549866+0000', 'last_fullsized': '2024-05-30T16:48:29.549866+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:28.531350+0000', 'last_change': '2024-05-30T16:48:28.531350+0000', 'last_active': '2024-05-30T16:48:28.498983+0000', 'last_peered': '2024-05-30T16:48:28.498983+0000', 'last_clean': '2024-05-30T16:48:28.498983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:28.531350+0000', 'last_undegraded': '2024-05-30T16:48:28.531350+0000', 'last_fullsized': '2024-05-30T16:48:28.531350+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1a is not active+clean 2024-05-30T16:48:36.087 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.558514+0000', 'last_change': '2024-05-30T16:48:29.558514+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.558514+0000', 'last_undegraded': '2024-05-30T16:48:29.558514+0000', 'last_fullsized': '2024-05-30T16:48:29.558514+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:PG 7.19 is not active+clean 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.548745+0000', 'last_change': '2024-05-30T16:48:29.548745+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.548745+0000', 'last_undegraded': '2024-05-30T16:48:29.548745+0000', 'last_fullsized': '2024-05-30T16:48:29.548745+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:PG 7.16 is not active+clean 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.559496+0000', 'last_change': '2024-05-30T16:48:29.559496+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.559496+0000', 'last_undegraded': '2024-05-30T16:48:29.559496+0000', 'last_fullsized': '2024-05-30T16:48:29.559496+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:PG 7.17 is not active+clean 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T16:48:29.513818+0000', 'last_change': '2024-05-30T16:48:29.513818+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.513818+0000', 'last_undegraded': '2024-05-30T16:48:29.513818+0000', 'last_fullsized': '2024-05-30T16:48:29.513818+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:PG 7.14 is not active+clean 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.541170+0000', 'last_change': '2024-05-30T16:48:29.541170+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.541170+0000', 'last_undegraded': '2024-05-30T16:48:29.541170+0000', 'last_fullsized': '2024-05-30T16:48:29.541170+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:28.534129+0000', 'last_change': '2024-05-30T16:48:28.534129+0000', 'last_active': '2024-05-30T16:48:28.498983+0000', 'last_peered': '2024-05-30T16:48:28.498983+0000', 'last_clean': '2024-05-30T16:48:28.498983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:28.534129+0000', 'last_undegraded': '2024-05-30T16:48:28.534129+0000', 'last_fullsized': '2024-05-30T16:48:28.534129+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-30T16:48:36.088 INFO:tasks.ceph.ceph_manager.ceph:PG 7.15 is not active+clean 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T16:48:29.513818+0000', 'last_change': '2024-05-30T16:48:29.513818+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.513818+0000', 'last_undegraded': '2024-05-30T16:48:29.513818+0000', 'last_fullsized': '2024-05-30T16:48:29.513818+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:PG 7.12 is not active+clean 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.554849+0000', 'last_change': '2024-05-30T16:48:29.554849+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.554849+0000', 'last_undegraded': '2024-05-30T16:48:29.554849+0000', 'last_fullsized': '2024-05-30T16:48:29.554849+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:28.528356+0000', 'last_change': '2024-05-30T16:48:28.528356+0000', 'last_active': '2024-05-30T16:48:28.498983+0000', 'last_peered': '2024-05-30T16:48:28.498983+0000', 'last_clean': '2024-05-30T16:48:28.498983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:28.528356+0000', 'last_undegraded': '2024-05-30T16:48:28.528356+0000', 'last_fullsized': '2024-05-30T16:48:28.528356+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.543580+0000', 'last_change': '2024-05-30T16:48:29.543580+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.543580+0000', 'last_undegraded': '2024-05-30T16:48:29.543580+0000', 'last_fullsized': '2024-05-30T16:48:29.543580+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:PG 7.11 is not active+clean 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T16:48:29.513818+0000', 'last_change': '2024-05-30T16:48:29.513818+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.513818+0000', 'last_undegraded': '2024-05-30T16:48:29.513818+0000', 'last_fullsized': '2024-05-30T16:48:29.513818+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:PG 7.e is not active+clean 2024-05-30T16:48:36.089 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.540700+0000', 'last_change': '2024-05-30T16:48:29.540700+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.540700+0000', 'last_undegraded': '2024-05-30T16:48:29.540700+0000', 'last_fullsized': '2024-05-30T16:48:29.540700+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:PG 7.c is not active+clean 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.557004+0000', 'last_change': '2024-05-30T16:48:29.557004+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.557004+0000', 'last_undegraded': '2024-05-30T16:48:29.557004+0000', 'last_fullsized': '2024-05-30T16:48:29.557004+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:PG 7.d is not active+clean 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.558489+0000', 'last_change': '2024-05-30T16:48:29.558489+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.558489+0000', 'last_undegraded': '2024-05-30T16:48:29.558489+0000', 'last_fullsized': '2024-05-30T16:48:29.558489+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:PG 7.a is not active+clean 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.540461+0000', 'last_change': '2024-05-30T16:48:29.540461+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.540461+0000', 'last_undegraded': '2024-05-30T16:48:29.540461+0000', 'last_fullsized': '2024-05-30T16:48:29.540461+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:PG 7.b is not active+clean 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.551258+0000', 'last_change': '2024-05-30T16:48:29.551258+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.551258+0000', 'last_undegraded': '2024-05-30T16:48:29.551258+0000', 'last_fullsized': '2024-05-30T16:48:29.551258+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:PG 7.8 is not active+clean 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.538084+0000', 'last_change': '2024-05-30T16:48:29.538084+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.538084+0000', 'last_undegraded': '2024-05-30T16:48:29.538084+0000', 'last_fullsized': '2024-05-30T16:48:29.538084+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-05-30T16:48:36.090 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:28.528249+0000', 'last_change': '2024-05-30T16:48:28.528249+0000', 'last_active': '2024-05-30T16:48:28.498983+0000', 'last_peered': '2024-05-30T16:48:28.498983+0000', 'last_clean': '2024-05-30T16:48:28.498983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:28.528249+0000', 'last_undegraded': '2024-05-30T16:48:28.528249+0000', 'last_fullsized': '2024-05-30T16:48:28.528249+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:PG 7.7 is not active+clean 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.559430+0000', 'last_change': '2024-05-30T16:48:29.559430+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.559430+0000', 'last_undegraded': '2024-05-30T16:48:29.559430+0000', 'last_fullsized': '2024-05-30T16:48:29.559430+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:28.533009+0000', 'last_change': '2024-05-30T16:48:28.533009+0000', 'last_active': '2024-05-30T16:48:28.498983+0000', 'last_peered': '2024-05-30T16:48:28.498983+0000', 'last_clean': '2024-05-30T16:48:28.498983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:28.533009+0000', 'last_undegraded': '2024-05-30T16:48:28.533009+0000', 'last_fullsized': '2024-05-30T16:48:28.533009+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:PG 7.5 is not active+clean 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T16:48:29.513818+0000', 'last_change': '2024-05-30T16:48:29.513818+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.513818+0000', 'last_undegraded': '2024-05-30T16:48:29.513818+0000', 'last_fullsized': '2024-05-30T16:48:29.513818+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2 is not active+clean 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.539746+0000', 'last_change': '2024-05-30T16:48:29.539746+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.539746+0000', 'last_undegraded': '2024-05-30T16:48:29.539746+0000', 'last_fullsized': '2024-05-30T16:48:29.539746+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:28.528519+0000', 'last_change': '2024-05-30T16:48:28.528519+0000', 'last_active': '2024-05-30T16:48:28.498983+0000', 'last_peered': '2024-05-30T16:48:28.498983+0000', 'last_clean': '2024-05-30T16:48:28.498983+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:28.528519+0000', 'last_undegraded': '2024-05-30T16:48:28.528519+0000', 'last_fullsized': '2024-05-30T16:48:28.528519+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:28.498983+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-30T16:48:36.091 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3 is not active+clean 2024-05-30T16:48:36.092 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.539795+0000', 'last_change': '2024-05-30T16:48:29.539795+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.539795+0000', 'last_undegraded': '2024-05-30T16:48:29.539795+0000', 'last_fullsized': '2024-05-30T16:48:29.539795+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T16:48:36.092 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1 is not active+clean 2024-05-30T16:48:36.092 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-05-30T16:48:29.553680+0000', 'last_change': '2024-05-30T16:48:29.553680+0000', 'last_active': '2024-05-30T16:48:29.513818+0000', 'last_peered': '2024-05-30T16:48:29.513818+0000', 'last_clean': '2024-05-30T16:48:29.513818+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T16:48:29.553680+0000', 'last_undegraded': '2024-05-30T16:48:29.553680+0000', 'last_fullsized': '2024-05-30T16:48:29.553680+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'last_clean_scrub_stamp': '2024-05-30T16:48:29.513818+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T16:48:36.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-30T16:48:36.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.332+0000 7fa6ab000640 1 Processor -- start 2024-05-30T16:48:36.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.333+0000 7fa6ab000640 1 -- start start 2024-05-30T16:48:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.334+0000 7fa6ab000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6a40d5150 0x7fa6a40d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.334+0000 7fa6ab000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a40d2200 con 0x7fa6a415fc70 2024-05-30T16:48:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.334+0000 7fa6ab000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a40d2470 con 0x7fa6a40d3490 2024-05-30T16:48:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.334+0000 7fa6ab000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a4160050 con 0x7fa6a40d5150 2024-05-30T16:48:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.334+0000 7fa6a9c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa6a415fc70 0x7fa6a40cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52170/0 (socket says 172.21.15.67:52170) 2024-05-30T16:48:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.334+0000 7fa6a9c00640 1 -- 172.21.15.67:0/2339767965 learned_addr learned my addr 172.21.15.67:0/2339767965 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.335+0000 7fa6a9200640 1 --2- 172.21.15.67:0/2339767965 >> v2:172.21.15.67:3300/0 conn(0x7fa6a40d5150 0x7fa6a40d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.335+0000 7fa6a9200640 1 -- 172.21.15.67:0/2339767965 >> v1:172.21.15.169:6789/0 conn(0x7fa6a40d3490 legacy=0x7fa6a40d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.335+0000 7fa6a9200640 1 -- 172.21.15.67:0/2339767965 >> v1:172.21.15.67:6789/0 conn(0x7fa6a415fc70 legacy=0x7fa6a40cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.335+0000 7fa6a9200640 1 -- 172.21.15.67:0/2339767965 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a40d2750 con 0x7fa6a40d5150 2024-05-30T16:48:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.335+0000 7fa6a3e00640 1 -- 172.21.15.67:0/2339767965 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217807659 0 0) 0x7fa6a40d2200 con 0x7fa6a415fc70 2024-05-30T16:48:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.336+0000 7fa6a9200640 1 --2- 172.21.15.67:0/2339767965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a40d5150 0x7fa6a40d2f50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa690009790 tx=0x7fa69002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dc2f7145f04bae server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.337+0000 7fa6a3e00640 1 -- 172.21.15.67:0/2339767965 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690037070 con 0x7fa6a40d5150 2024-05-30T16:48:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.337+0000 7fa6a3e00640 1 -- 172.21.15.67:0/2339767965 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa690032440 con 0x7fa6a40d5150 2024-05-30T16:48:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.337+0000 7fa6a3e00640 1 -- 172.21.15.67:0/2339767965 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690032810 con 0x7fa6a40d5150 2024-05-30T16:48:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.337+0000 7fa6ab000640 1 -- 172.21.15.67:0/2339767965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a40d5150 msgr2=0x7fa6a40d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.337+0000 7fa6ab000640 1 --2- 172.21.15.67:0/2339767965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a40d5150 0x7fa6a40d2f50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa690009790 tx=0x7fa69002a1c0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.337+0000 7fa6ab000640 1 -- 172.21.15.67:0/2339767965 shutdown_connections 2024-05-30T16:48:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.337+0000 7fa6ab000640 1 --2- 172.21.15.67:0/2339767965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a40d5150 0x7fa6a40d2f50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.337+0000 7fa6ab000640 1 -- 172.21.15.67:0/2339767965 >> 172.21.15.67:0/2339767965 conn(0x7fa6a408bfc0 msgr2=0x7fa6a40d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.337+0000 7fa6ab000640 1 -- 172.21.15.67:0/2339767965 shutdown_connections 2024-05-30T16:48:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.338+0000 7fa6ab000640 1 -- 172.21.15.67:0/2339767965 wait complete. 2024-05-30T16:48:36.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.339+0000 7fa6ab000640 1 Processor -- start 2024-05-30T16:48:36.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.339+0000 7fa6ab000640 1 -- start start 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.340+0000 7fa6ab000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a40d3490 0x7fa6a40b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.340+0000 7fa6ab000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a40d5150 0x7fa6a415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.340+0000 7fa6ab000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a415fc70 0x7fa6a40a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.340+0000 7fa6ab000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a40d1620 con 0x7fa6a415fc70 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.340+0000 7fa6ab000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a40d2ac0 con 0x7fa6a40d3490 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.340+0000 7fa6ab000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a40d17a0 con 0x7fa6a40d5150 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6aa600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a415fc70 0x7fa6a40a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6aa600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a415fc70 0x7fa6a40a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58090/0 (socket says 172.21.15.67:58090) 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6aa600640 1 -- 172.21.15.67:0/3696597595 learned_addr learned my addr 172.21.15.67:0/3696597595 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6a9c00640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a40d3490 0x7fa6a40b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6a9200640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a40d5150 0x7fa6a415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6aa600640 1 -- 172.21.15.67:0/3696597595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a40d5150 msgr2=0x7fa6a415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6aa600640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a40d5150 0x7fa6a415e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6aa600640 1 -- 172.21.15.67:0/3696597595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a40d3490 msgr2=0x7fa6a40b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6aa600640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a40d3490 0x7fa6a40b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.341+0000 7fa6aa600640 1 -- 172.21.15.67:0/3696597595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a40b1260 con 0x7fa6a415fc70 2024-05-30T16:48:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.342+0000 7fa6aa600640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a415fc70 0x7fa6a40a0e30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00be60 tx=0x7fa69c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.342+0000 7fa6a1600640 1 -- 172.21.15.67:0/3696597595 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c019070 con 0x7fa6a415fc70 2024-05-30T16:48:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.342+0000 7fa6a1600640 1 -- 172.21.15.67:0/3696597595 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69c014440 con 0x7fa6a415fc70 2024-05-30T16:48:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.342+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a40b1540 con 0x7fa6a415fc70 2024-05-30T16:48:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.342+0000 7fa6a1600640 1 -- 172.21.15.67:0/3696597595 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c014870 con 0x7fa6a415fc70 2024-05-30T16:48:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.342+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a40b1b00 con 0x7fa6a415fc70 2024-05-30T16:48:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.343+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a40689e0 con 0x7fa6a415fc70 2024-05-30T16:48:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.345+0000 7fa6a1600640 1 -- 172.21.15.67:0/3696597595 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fa69c01c070 con 0x7fa6a415fc70 2024-05-30T16:48:36.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.346+0000 7fa6a1600640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa68406cbc0 0x7fa68406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.346+0000 7fa6a1600640 1 -- 172.21.15.67:0/3696597595 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fa69c08bef0 con 0x7fa6a415fc70 2024-05-30T16:48:36.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.346+0000 7fa6a9c00640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa68406cbc0 0x7fa68406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.347+0000 7fa6a9c00640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa68406cbc0 0x7fa68406f080 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa698004c00 tx=0x7fa698006050 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.350+0000 7fa6a1600640 1 -- 172.21.15.67:0/3696597595 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa69c05a060 con 0x7fa6a415fc70 2024-05-30T16:48:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.631+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa6a40d1d60 con 0x7fa6a415fc70 2024-05-30T16:48:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.632+0000 7fa6a1600640 1 -- 172.21.15.67:0/3696597595 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1013 (secure 0 0 0) 0x7fa69c059960 con 0x7fa6a415fc70 2024-05-30T16:48:36.634 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:36.634 INFO:teuthology.orchestra.run.smithi067.stdout:{"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":310,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1717087419,"num_in_osds":8,"osd_in_since":1717087410,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":295239680,"bytes_avail":772798873600,"bytes_total":773094113280,"write_bytes_sec":2186,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5361}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-30T16:45:50.020907+0000","services":{}},"progress_events":{}} 2024-05-30T16:48:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa68406cbc0 msgr2=0x7fa68406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa68406cbc0 0x7fa68406f080 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa698004c00 tx=0x7fa698006050 comp rx=0 tx=0).stop 2024-05-30T16:48:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a415fc70 msgr2=0x7fa6a40a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a415fc70 0x7fa6a40a0e30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00be60 tx=0x7fa69c00bf50 comp rx=0 tx=0).stop 2024-05-30T16:48:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 shutdown_connections 2024-05-30T16:48:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa68406cbc0 0x7fa68406f080 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6a415fc70 0x7fa6a40a0e30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6a40d5150 0x7fa6a415e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 --2- 172.21.15.67:0/3696597595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6a40d3490 0x7fa6a40b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 >> 172.21.15.67:0/3696597595 conn(0x7fa6a408bfc0 msgr2=0x7fa6a40cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 shutdown_connections 2024-05-30T16:48:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.635+0000 7fa6ab000640 1 -- 172.21.15.67:0/3696597595 wait complete. 2024-05-30T16:48:36.654 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-30T16:48:36.654 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:48:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.850+0000 7f68d7e00640 1 Processor -- start 2024-05-30T16:48:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.850+0000 7f68d7e00640 1 -- start start 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.851+0000 7f68d7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f68d806ee40 0x7f68d80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.851+0000 7f68d7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d8156f10 con 0x7f68d806f2f0 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.851+0000 7f68d7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d806dda0 con 0x7f68d8156b30 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.851+0000 7f68d7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f68d806e080 con 0x7f68d806ee40 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.852+0000 7f68d6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f68d806ee40 0x7f68d80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.852+0000 7f68d7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f68d806f2f0 0x7f68d80b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52172/0 (socket says 172.21.15.67:52172) 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.852+0000 7f68d7400640 1 -- 172.21.15.67:0/3424944766 learned_addr learned my addr 172.21.15.67:0/3424944766 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.852+0000 7f68d6000640 1 -- 172.21.15.67:0/3424944766 >> v1:172.21.15.169:6789/0 conn(0x7f68d8156b30 legacy=0x7f68d80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.852+0000 7f68d6000640 1 -- 172.21.15.67:0/3424944766 >> v1:172.21.15.67:6789/0 conn(0x7f68d806f2f0 legacy=0x7f68d80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.852+0000 7f68d6000640 1 -- 172.21.15.67:0/3424944766 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d806e360 con 0x7f68d806ee40 2024-05-30T16:48:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.852+0000 7f68d5600640 1 -- 172.21.15.67:0/3424944766 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 380255148 0 0) 0x7f68d8156f10 con 0x7f68d806f2f0 2024-05-30T16:48:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.853+0000 7f68d6000640 1 --2- 172.21.15.67:0/3424944766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d806ee40 0x7f68d80ad230 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f68c80097a0 tx=0x7f68c802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ee39474a2e98dd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.854+0000 7f68d5600640 1 -- 172.21.15.67:0/3424944766 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8037070 con 0x7f68d806ee40 2024-05-30T16:48:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.854+0000 7f68d5600640 1 -- 172.21.15.67:0/3424944766 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c802ae50 con 0x7f68d806ee40 2024-05-30T16:48:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.854+0000 7f68d5600640 1 -- 172.21.15.67:0/3424944766 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8032660 con 0x7f68d806ee40 2024-05-30T16:48:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.854+0000 7f68d7e00640 1 -- 172.21.15.67:0/3424944766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d806ee40 msgr2=0x7f68d80ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.854+0000 7f68d7e00640 1 --2- 172.21.15.67:0/3424944766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d806ee40 0x7f68d80ad230 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f68c80097a0 tx=0x7f68c802a140 comp rx=0 tx=0).stop 2024-05-30T16:48:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.854+0000 7f68d7e00640 1 -- 172.21.15.67:0/3424944766 shutdown_connections 2024-05-30T16:48:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.854+0000 7f68d7e00640 1 --2- 172.21.15.67:0/3424944766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d806ee40 0x7f68d80ad230 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.854+0000 7f68d7e00640 1 -- 172.21.15.67:0/3424944766 >> 172.21.15.67:0/3424944766 conn(0x7f68d80b1fc0 msgr2=0x7f68d80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.855+0000 7f68d7e00640 1 -- 172.21.15.67:0/3424944766 shutdown_connections 2024-05-30T16:48:36.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.855+0000 7f68d7e00640 1 -- 172.21.15.67:0/3424944766 wait complete. 2024-05-30T16:48:36.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.856+0000 7f68d7e00640 1 Processor -- start 2024-05-30T16:48:36.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.857+0000 7f68d7e00640 1 -- start start 2024-05-30T16:48:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d806ee40 0x7f68d814bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d806f2f0 0x7f68d8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d8156b30 0x7f68d8146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68d80b2db0 con 0x7f68d8156b30 2024-05-30T16:48:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f68d80b2c30 con 0x7f68d806f2f0 2024-05-30T16:48:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68d80b39d0 con 0x7f68d806ee40 2024-05-30T16:48:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d8156b30 0x7f68d8146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d8156b30 0x7f68d8146330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58112/0 (socket says 172.21.15.67:58112) 2024-05-30T16:48:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7400640 1 -- 172.21.15.67:0/881861468 learned_addr learned my addr 172.21.15.67:0/881861468 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d6a00640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d806ee40 0x7f68d814bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d6000640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d806f2f0 0x7f68d8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7400640 1 -- 172.21.15.67:0/881861468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d806ee40 msgr2=0x7f68d814bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7400640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d806ee40 0x7f68d814bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7400640 1 -- 172.21.15.67:0/881861468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d806f2f0 msgr2=0x7f68d8157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7400640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d806f2f0 0x7f68d8157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.858+0000 7f68d7400640 1 -- 172.21.15.67:0/881861468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d81596a0 con 0x7f68d8156b30 2024-05-30T16:48:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.859+0000 7f68d7400640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d8156b30 0x7f68d8146330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f68d000c800 tx=0x7f68d000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.859+0000 7f68cf400640 1 -- 172.21.15.67:0/881861468 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0018070 con 0x7f68d8156b30 2024-05-30T16:48:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.859+0000 7f68cf400640 1 -- 172.21.15.67:0/881861468 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d0004d80 con 0x7f68d8156b30 2024-05-30T16:48:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.859+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d8159980 con 0x7f68d8156b30 2024-05-30T16:48:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.859+0000 7f68cf400640 1 -- 172.21.15.67:0/881861468 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d000a880 con 0x7f68d8156b30 2024-05-30T16:48:36.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.859+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d8150cd0 con 0x7f68d8156b30 2024-05-30T16:48:36.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.860+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d80689e0 con 0x7f68d8156b30 2024-05-30T16:48:36.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.861+0000 7f68cf400640 1 -- 172.21.15.67:0/881861468 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f68d001b070 con 0x7f68d8156b30 2024-05-30T16:48:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.862+0000 7f68cf400640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68b406cbc0 0x7f68b406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.862+0000 7f68cf400640 1 -- 172.21.15.67:0/881861468 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f68d008b500 con 0x7f68d8156b30 2024-05-30T16:48:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.862+0000 7f68d6a00640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68b406cbc0 0x7f68b406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:36.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.863+0000 7f68d6a00640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68b406cbc0 0x7f68b406f080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68c0006fd0 tx=0x7f68c0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:36.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:36.867+0000 7f68cf400640 1 -- 172.21.15.67:0/881861468 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68d00596f0 con 0x7f68d8156b30 2024-05-30T16:48:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.074+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68d8159ea0 con 0x7f68b406cbc0 2024-05-30T16:48:37.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.092+0000 7f68cf400640 1 -- 172.21.15.67:0/881861468 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251958 (secure 0 0 0) 0x7f68d8159ea0 con 0x7f68b406cbc0 2024-05-30T16:48:37.098 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:37.099 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.097+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68b406cbc0 msgr2=0x7f68b406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.097+0000 7f68d7e00640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68b406cbc0 0x7f68b406f080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68c0006fd0 tx=0x7f68c0008040 comp rx=0 tx=0).stop 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.097+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d8156b30 msgr2=0x7f68d8146330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.097+0000 7f68d7e00640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d8156b30 0x7f68d8146330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f68d000c800 tx=0x7f68d000ccc0 comp rx=0 tx=0).stop 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.098+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 shutdown_connections 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.098+0000 7f68d7e00640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68b406cbc0 0x7f68b406f080 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.098+0000 7f68d7e00640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68d8156b30 0x7f68d8146330 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.098+0000 7f68d7e00640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d806f2f0 0x7f68d8157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.098+0000 7f68d7e00640 1 --2- 172.21.15.67:0/881861468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d806ee40 0x7f68d814bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.098+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 >> 172.21.15.67:0/881861468 conn(0x7f68d80b1fc0 msgr2=0x7f68d815aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.098+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 shutdown_connections 2024-05-30T16:48:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:37.098+0000 7f68d7e00640 1 -- 172.21.15.67:0/881861468 wait complete. 2024-05-30T16:48:37.119 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-05-30T16:48:36.087438+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":288320,"kb_used_data":5576,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754686400,"statfs":{"total":773094113280,"available":772798873600,"internally_reserved":0,"allocated":5709824,"data_stored":3012972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.555916"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569722+0000","last_change":"2024-05-30T16:48:30.576499+0000","last_active":"2024-05-30T16:48:31.569722+0000","last_peered":"2024-05-30T16:48:31.569722+0000","last_clean":"2024-05-30T16:48:31.569722+0000","last_became_active":"2024-05-30T16:48:30.576126+0000","last_became_peered":"2024-05-30T16:48:30.576126+0000","last_unstale":"2024-05-30T16:48:31.569722+0000","last_undegraded":"2024-05-30T16:48:31.569722+0000","last_fullsized":"2024-05-30T16:48:31.569722+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:20:17.680707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570382+0000","last_change":"2024-05-30T16:48:30.584563+0000","last_active":"2024-05-30T16:48:31.570382+0000","last_peered":"2024-05-30T16:48:31.570382+0000","last_clean":"2024-05-30T16:48:31.570382+0000","last_became_active":"2024-05-30T16:48:30.584262+0000","last_became_peered":"2024-05-30T16:48:30.584262+0000","last_unstale":"2024-05-30T16:48:31.570382+0000","last_undegraded":"2024-05-30T16:48:31.570382+0000","last_fullsized":"2024-05-30T16:48:31.570382+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601162+0000","last_change":"2024-05-30T16:48:30.577269+0000","last_active":"2024-05-30T16:48:31.601162+0000","last_peered":"2024-05-30T16:48:31.601162+0000","last_clean":"2024-05-30T16:48:31.601162+0000","last_became_active":"2024-05-30T16:48:30.576986+0000","last_became_peered":"2024-05-30T16:48:30.576986+0000","last_unstale":"2024-05-30T16:48:31.601162+0000","last_undegraded":"2024-05-30T16:48:31.601162+0000","last_fullsized":"2024-05-30T16:48:31.601162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:37.663916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603004+0000","last_change":"2024-05-30T16:48:30.564821+0000","last_active":"2024-05-30T16:48:31.603004+0000","last_peered":"2024-05-30T16:48:31.603004+0000","last_clean":"2024-05-30T16:48:31.603004+0000","last_became_active":"2024-05-30T16:48:30.564537+0000","last_became_peered":"2024-05-30T16:48:30.564537+0000","last_unstale":"2024-05-30T16:48:31.603004+0000","last_undegraded":"2024-05-30T16:48:31.603004+0000","last_fullsized":"2024-05-30T16:48:31.603004+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:00:29.368856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570614+0000","last_change":"2024-05-30T16:48:30.565013+0000","last_active":"2024-05-30T16:48:31.570614+0000","last_peered":"2024-05-30T16:48:31.570614+0000","last_clean":"2024-05-30T16:48:31.570614+0000","last_became_active":"2024-05-30T16:48:30.564344+0000","last_became_peered":"2024-05-30T16:48:30.564344+0000","last_unstale":"2024-05-30T16:48:31.570614+0000","last_undegraded":"2024-05-30T16:48:31.570614+0000","last_fullsized":"2024-05-30T16:48:31.570614+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:38.774999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567205+0000","last_change":"2024-05-30T16:48:30.573684+0000","last_active":"2024-05-30T16:48:31.567205+0000","last_peered":"2024-05-30T16:48:31.567205+0000","last_clean":"2024-05-30T16:48:31.567205+0000","last_became_active":"2024-05-30T16:48:30.573404+0000","last_became_peered":"2024-05-30T16:48:30.573404+0000","last_unstale":"2024-05-30T16:48:31.567205+0000","last_undegraded":"2024-05-30T16:48:31.567205+0000","last_fullsized":"2024-05-30T16:48:31.567205+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:49:08.996106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568975+0000","last_change":"2024-05-30T16:48:30.576603+0000","last_active":"2024-05-30T16:48:31.568975+0000","last_peered":"2024-05-30T16:48:31.568975+0000","last_clean":"2024-05-30T16:48:31.568975+0000","last_became_active":"2024-05-30T16:48:30.576320+0000","last_became_peered":"2024-05-30T16:48:30.576320+0000","last_unstale":"2024-05-30T16:48:31.568975+0000","last_undegraded":"2024-05-30T16:48:31.568975+0000","last_fullsized":"2024-05-30T16:48:31.568975+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:25.686849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600493+0000","last_change":"2024-05-30T16:48:30.561287+0000","last_active":"2024-05-30T16:48:31.600493+0000","last_peered":"2024-05-30T16:48:31.600493+0000","last_clean":"2024-05-30T16:48:31.600493+0000","last_became_active":"2024-05-30T16:48:30.560737+0000","last_became_peered":"2024-05-30T16:48:30.560737+0000","last_unstale":"2024-05-30T16:48:31.600493+0000","last_undegraded":"2024-05-30T16:48:31.600493+0000","last_fullsized":"2024-05-30T16:48:31.600493+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:50:31.061295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603257+0000","last_change":"2024-05-30T16:48:30.587362+0000","last_active":"2024-05-30T16:48:31.603257+0000","last_peered":"2024-05-30T16:48:31.603257+0000","last_clean":"2024-05-30T16:48:31.603257+0000","last_became_active":"2024-05-30T16:48:30.587100+0000","last_became_peered":"2024-05-30T16:48:30.587100+0000","last_unstale":"2024-05-30T16:48:31.603257+0000","last_undegraded":"2024-05-30T16:48:31.603257+0000","last_fullsized":"2024-05-30T16:48:31.603257+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:00.811234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603466+0000","last_change":"2024-05-30T16:48:30.582474+0000","last_active":"2024-05-30T16:48:31.603466+0000","last_peered":"2024-05-30T16:48:31.603466+0000","last_clean":"2024-05-30T16:48:31.603466+0000","last_became_active":"2024-05-30T16:48:30.582148+0000","last_became_peered":"2024-05-30T16:48:30.582148+0000","last_unstale":"2024-05-30T16:48:31.603466+0000","last_undegraded":"2024-05-30T16:48:31.603466+0000","last_fullsized":"2024-05-30T16:48:31.603466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602911+0000","last_change":"2024-05-30T16:48:30.577384+0000","last_active":"2024-05-30T16:48:31.602911+0000","last_peered":"2024-05-30T16:48:31.602911+0000","last_clean":"2024-05-30T16:48:31.602911+0000","last_became_active":"2024-05-30T16:48:30.570714+0000","last_became_peered":"2024-05-30T16:48:30.570714+0000","last_unstale":"2024-05-30T16:48:31.602911+0000","last_undegraded":"2024-05-30T16:48:31.602911+0000","last_fullsized":"2024-05-30T16:48:31.602911+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:05.025187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570620+0000","last_change":"2024-05-30T16:48:30.582165+0000","last_active":"2024-05-30T16:48:31.570620+0000","last_peered":"2024-05-30T16:48:31.570620+0000","last_clean":"2024-05-30T16:48:31.570620+0000","last_became_active":"2024-05-30T16:48:30.581896+0000","last_became_peered":"2024-05-30T16:48:30.581896+0000","last_unstale":"2024-05-30T16:48:31.570620+0000","last_undegraded":"2024-05-30T16:48:31.570620+0000","last_fullsized":"2024-05-30T16:48:31.570620+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602235+0000","last_change":"2024-05-30T16:48:30.585005+0000","last_active":"2024-05-30T16:48:31.602235+0000","last_peered":"2024-05-30T16:48:31.602235+0000","last_clean":"2024-05-30T16:48:31.602235+0000","last_became_active":"2024-05-30T16:48:30.584718+0000","last_became_peered":"2024-05-30T16:48:30.584718+0000","last_unstale":"2024-05-30T16:48:31.602235+0000","last_undegraded":"2024-05-30T16:48:31.602235+0000","last_fullsized":"2024-05-30T16:48:31.602235+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:34.971904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601441+0000","last_change":"2024-05-30T16:48:30.555321+0000","last_active":"2024-05-30T16:48:31.601441+0000","last_peered":"2024-05-30T16:48:31.601441+0000","last_clean":"2024-05-30T16:48:31.601441+0000","last_became_active":"2024-05-30T16:48:30.555012+0000","last_became_peered":"2024-05-30T16:48:30.555012+0000","last_unstale":"2024-05-30T16:48:31.601441+0000","last_undegraded":"2024-05-30T16:48:31.601441+0000","last_fullsized":"2024-05-30T16:48:31.601441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:41.388387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567200+0000","last_change":"2024-05-30T16:48:30.574629+0000","last_active":"2024-05-30T16:48:31.567200+0000","last_peered":"2024-05-30T16:48:31.567200+0000","last_clean":"2024-05-30T16:48:31.567200+0000","last_became_active":"2024-05-30T16:48:30.574307+0000","last_became_peered":"2024-05-30T16:48:30.574307+0000","last_unstale":"2024-05-30T16:48:31.567200+0000","last_undegraded":"2024-05-30T16:48:31.567200+0000","last_fullsized":"2024-05-30T16:48:31.567200+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:13.127547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602095+0000","last_change":"2024-05-30T16:48:30.572742+0000","last_active":"2024-05-30T16:48:31.602095+0000","last_peered":"2024-05-30T16:48:31.602095+0000","last_clean":"2024-05-30T16:48:31.602095+0000","last_became_active":"2024-05-30T16:48:30.572396+0000","last_became_peered":"2024-05-30T16:48:30.572396+0000","last_unstale":"2024-05-30T16:48:31.602095+0000","last_undegraded":"2024-05-30T16:48:31.602095+0000","last_fullsized":"2024-05-30T16:48:31.602095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:39.464041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601466+0000","last_change":"2024-05-30T16:48:30.574852+0000","last_active":"2024-05-30T16:48:31.601466+0000","last_peered":"2024-05-30T16:48:31.601466+0000","last_clean":"2024-05-30T16:48:31.601466+0000","last_became_active":"2024-05-30T16:48:30.574568+0000","last_became_peered":"2024-05-30T16:48:30.574568+0000","last_unstale":"2024-05-30T16:48:31.601466+0000","last_undegraded":"2024-05-30T16:48:31.601466+0000","last_fullsized":"2024-05-30T16:48:31.601466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:38:07.252870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602414+0000","last_change":"2024-05-30T16:48:30.589719+0000","last_active":"2024-05-30T16:48:31.602414+0000","last_peered":"2024-05-30T16:48:31.602414+0000","last_clean":"2024-05-30T16:48:31.602414+0000","last_became_active":"2024-05-30T16:48:30.589375+0000","last_became_peered":"2024-05-30T16:48:30.589375+0000","last_unstale":"2024-05-30T16:48:31.602414+0000","last_undegraded":"2024-05-30T16:48:31.602414+0000","last_fullsized":"2024-05-30T16:48:31.602414+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:54.229780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603555+0000","last_change":"2024-05-30T16:48:30.587045+0000","last_active":"2024-05-30T16:48:31.603555+0000","last_peered":"2024-05-30T16:48:31.603555+0000","last_clean":"2024-05-30T16:48:31.603555+0000","last_became_active":"2024-05-30T16:48:30.586703+0000","last_became_peered":"2024-05-30T16:48:30.586703+0000","last_unstale":"2024-05-30T16:48:31.603555+0000","last_undegraded":"2024-05-30T16:48:31.603555+0000","last_fullsized":"2024-05-30T16:48:31.603555+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:15.486340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601462+0000","last_change":"2024-05-30T16:48:30.589187+0000","last_active":"2024-05-30T16:48:31.601462+0000","last_peered":"2024-05-30T16:48:31.601462+0000","last_clean":"2024-05-30T16:48:31.601462+0000","last_became_active":"2024-05-30T16:48:30.588913+0000","last_became_peered":"2024-05-30T16:48:30.588913+0000","last_unstale":"2024-05-30T16:48:31.601462+0000","last_undegraded":"2024-05-30T16:48:31.601462+0000","last_fullsized":"2024-05-30T16:48:31.601462+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:50.160833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567419+0000","last_change":"2024-05-30T16:48:30.572439+0000","last_active":"2024-05-30T16:48:31.567419+0000","last_peered":"2024-05-30T16:48:31.567419+0000","last_clean":"2024-05-30T16:48:31.567419+0000","last_became_active":"2024-05-30T16:48:30.571643+0000","last_became_peered":"2024-05-30T16:48:30.571643+0000","last_unstale":"2024-05-30T16:48:31.567419+0000","last_undegraded":"2024-05-30T16:48:31.567419+0000","last_fullsized":"2024-05-30T16:48:31.567419+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:11.352814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601819+0000","last_change":"2024-05-30T16:48:30.563868+0000","last_active":"2024-05-30T16:48:31.601819+0000","last_peered":"2024-05-30T16:48:31.601819+0000","last_clean":"2024-05-30T16:48:31.601819+0000","last_became_active":"2024-05-30T16:48:30.563554+0000","last_became_peered":"2024-05-30T16:48:30.563554+0000","last_unstale":"2024-05-30T16:48:31.601819+0000","last_undegraded":"2024-05-30T16:48:31.601819+0000","last_fullsized":"2024-05-30T16:48:31.601819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:47.906916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602616+0000","last_change":"2024-05-30T16:48:30.574382+0000","last_active":"2024-05-30T16:48:31.602616+0000","last_peered":"2024-05-30T16:48:31.602616+0000","last_clean":"2024-05-30T16:48:31.602616+0000","last_became_active":"2024-05-30T16:48:30.574033+0000","last_became_peered":"2024-05-30T16:48:30.574033+0000","last_unstale":"2024-05-30T16:48:31.602616+0000","last_undegraded":"2024-05-30T16:48:31.602616+0000","last_fullsized":"2024-05-30T16:48:31.602616+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:45.188875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602234+0000","last_change":"2024-05-30T16:48:30.571696+0000","last_active":"2024-05-30T16:48:31.602234+0000","last_peered":"2024-05-30T16:48:31.602234+0000","last_clean":"2024-05-30T16:48:31.602234+0000","last_became_active":"2024-05-30T16:48:30.570676+0000","last_became_peered":"2024-05-30T16:48:30.570676+0000","last_unstale":"2024-05-30T16:48:31.602234+0000","last_undegraded":"2024-05-30T16:48:31.602234+0000","last_fullsized":"2024-05-30T16:48:31.602234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:34.971904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602930+0000","last_change":"2024-05-30T16:48:30.565038+0000","last_active":"2024-05-30T16:48:31.602930+0000","last_peered":"2024-05-30T16:48:31.602930+0000","last_clean":"2024-05-30T16:48:31.602930+0000","last_became_active":"2024-05-30T16:48:30.563510+0000","last_became_peered":"2024-05-30T16:48:30.563510+0000","last_unstale":"2024-05-30T16:48:31.602930+0000","last_undegraded":"2024-05-30T16:48:31.602930+0000","last_fullsized":"2024-05-30T16:48:31.602930+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:01.173277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602171+0000","last_change":"2024-05-30T16:48:30.585362+0000","last_active":"2024-05-30T16:48:31.602171+0000","last_peered":"2024-05-30T16:48:31.602171+0000","last_clean":"2024-05-30T16:48:31.602171+0000","last_became_active":"2024-05-30T16:48:30.584610+0000","last_became_peered":"2024-05-30T16:48:30.584610+0000","last_unstale":"2024-05-30T16:48:31.602171+0000","last_undegraded":"2024-05-30T16:48:31.602171+0000","last_fullsized":"2024-05-30T16:48:31.602171+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:39.464041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601623+0000","last_change":"2024-05-30T16:48:30.575524+0000","last_active":"2024-05-30T16:48:31.601623+0000","last_peered":"2024-05-30T16:48:31.601623+0000","last_clean":"2024-05-30T16:48:31.601623+0000","last_became_active":"2024-05-30T16:48:30.573685+0000","last_became_peered":"2024-05-30T16:48:30.573685+0000","last_unstale":"2024-05-30T16:48:31.601623+0000","last_undegraded":"2024-05-30T16:48:31.601623+0000","last_fullsized":"2024-05-30T16:48:31.601623+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:43.435766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571075+0000","last_change":"2024-05-30T16:48:30.582495+0000","last_active":"2024-05-30T16:48:31.571075+0000","last_peered":"2024-05-30T16:48:31.571075+0000","last_clean":"2024-05-30T16:48:31.571075+0000","last_became_active":"2024-05-30T16:48:30.582199+0000","last_became_peered":"2024-05-30T16:48:30.582199+0000","last_unstale":"2024-05-30T16:48:31.571075+0000","last_undegraded":"2024-05-30T16:48:31.571075+0000","last_fullsized":"2024-05-30T16:48:31.571075+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:18.447770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603245+0000","last_change":"2024-05-30T16:48:30.557574+0000","last_active":"2024-05-30T16:48:31.603245+0000","last_peered":"2024-05-30T16:48:31.603245+0000","last_clean":"2024-05-30T16:48:31.603245+0000","last_became_active":"2024-05-30T16:48:30.557169+0000","last_became_peered":"2024-05-30T16:48:30.557169+0000","last_unstale":"2024-05-30T16:48:31.603245+0000","last_undegraded":"2024-05-30T16:48:31.603245+0000","last_fullsized":"2024-05-30T16:48:31.603245+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:19.255146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600833+0000","last_change":"2024-05-30T16:48:30.572687+0000","last_active":"2024-05-30T16:48:31.600833+0000","last_peered":"2024-05-30T16:48:31.600833+0000","last_clean":"2024-05-30T16:48:31.600833+0000","last_became_active":"2024-05-30T16:48:30.571850+0000","last_became_peered":"2024-05-30T16:48:30.571850+0000","last_unstale":"2024-05-30T16:48:31.600833+0000","last_undegraded":"2024-05-30T16:48:31.600833+0000","last_fullsized":"2024-05-30T16:48:31.600833+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:54:22.198184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567341+0000","last_change":"2024-05-30T16:48:30.574622+0000","last_active":"2024-05-30T16:48:31.567341+0000","last_peered":"2024-05-30T16:48:31.567341+0000","last_clean":"2024-05-30T16:48:31.567341+0000","last_became_active":"2024-05-30T16:48:30.573902+0000","last_became_peered":"2024-05-30T16:48:30.573902+0000","last_unstale":"2024-05-30T16:48:31.567341+0000","last_undegraded":"2024-05-30T16:48:31.567341+0000","last_fullsized":"2024-05-30T16:48:31.567341+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:35.550338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601139+0000","last_change":"2024-05-30T16:48:30.570675+0000","last_active":"2024-05-30T16:48:31.601139+0000","last_peered":"2024-05-30T16:48:31.601139+0000","last_clean":"2024-05-30T16:48:31.601139+0000","last_became_active":"2024-05-30T16:48:30.569596+0000","last_became_peered":"2024-05-30T16:48:30.569596+0000","last_unstale":"2024-05-30T16:48:31.601139+0000","last_undegraded":"2024-05-30T16:48:31.601139+0000","last_fullsized":"2024-05-30T16:48:31.601139+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:24.977866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604125+0000","last_change":"2024-05-30T16:48:30.563191+0000","last_active":"2024-05-30T16:48:31.604125+0000","last_peered":"2024-05-30T16:48:31.604125+0000","last_clean":"2024-05-30T16:48:31.604125+0000","last_became_active":"2024-05-30T16:48:30.562912+0000","last_became_peered":"2024-05-30T16:48:30.562912+0000","last_unstale":"2024-05-30T16:48:31.604125+0000","last_undegraded":"2024-05-30T16:48:31.604125+0000","last_fullsized":"2024-05-30T16:48:31.604125+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:19.255146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568728+0000","last_change":"2024-05-30T16:48:29.572429+0000","last_active":"2024-05-30T16:48:31.568728+0000","last_peered":"2024-05-30T16:48:31.568728+0000","last_clean":"2024-05-30T16:48:31.568728+0000","last_became_active":"2024-05-30T16:48:29.572105+0000","last_became_peered":"2024-05-30T16:48:29.572105+0000","last_unstale":"2024-05-30T16:48:31.568728+0000","last_undegraded":"2024-05-30T16:48:31.568728+0000","last_fullsized":"2024-05-30T16:48:31.568728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:55.818348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603707+0000","last_change":"2024-05-30T16:48:30.567278+0000","last_active":"2024-05-30T16:48:31.603707+0000","last_peered":"2024-05-30T16:48:31.603707+0000","last_clean":"2024-05-30T16:48:31.603707+0000","last_became_active":"2024-05-30T16:48:30.566562+0000","last_became_peered":"2024-05-30T16:48:30.566562+0000","last_unstale":"2024-05-30T16:48:31.603707+0000","last_undegraded":"2024-05-30T16:48:31.603707+0000","last_fullsized":"2024-05-30T16:48:31.603707+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:29:25.409504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601689+0000","last_change":"2024-05-30T16:48:29.572447+0000","last_active":"2024-05-30T16:48:31.601689+0000","last_peered":"2024-05-30T16:48:31.601689+0000","last_clean":"2024-05-30T16:48:31.601689+0000","last_became_active":"2024-05-30T16:48:29.572102+0000","last_became_peered":"2024-05-30T16:48:29.572102+0000","last_unstale":"2024-05-30T16:48:31.601689+0000","last_undegraded":"2024-05-30T16:48:31.601689+0000","last_fullsized":"2024-05-30T16:48:31.601689+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:38.449206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571330+0000","last_change":"2024-05-30T16:48:30.570556+0000","last_active":"2024-05-30T16:48:31.571330+0000","last_peered":"2024-05-30T16:48:31.571330+0000","last_clean":"2024-05-30T16:48:31.571330+0000","last_became_active":"2024-05-30T16:48:30.569419+0000","last_became_peered":"2024-05-30T16:48:30.569419+0000","last_unstale":"2024-05-30T16:48:31.571330+0000","last_undegraded":"2024-05-30T16:48:31.571330+0000","last_fullsized":"2024-05-30T16:48:31.571330+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:33.900129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604848+0000","last_change":"2024-05-30T16:48:29.563603+0000","last_active":"2024-05-30T16:48:31.604848+0000","last_peered":"2024-05-30T16:48:31.604848+0000","last_clean":"2024-05-30T16:48:31.604848+0000","last_became_active":"2024-05-30T16:48:29.563260+0000","last_became_peered":"2024-05-30T16:48:29.563260+0000","last_unstale":"2024-05-30T16:48:31.604848+0000","last_undegraded":"2024-05-30T16:48:31.604848+0000","last_fullsized":"2024-05-30T16:48:31.604848+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:33.957069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604644+0000","last_change":"2024-05-30T16:48:30.563803+0000","last_active":"2024-05-30T16:48:31.604644+0000","last_peered":"2024-05-30T16:48:31.604644+0000","last_clean":"2024-05-30T16:48:31.604644+0000","last_became_active":"2024-05-30T16:48:30.563522+0000","last_became_peered":"2024-05-30T16:48:30.563522+0000","last_unstale":"2024-05-30T16:48:31.604644+0000","last_undegraded":"2024-05-30T16:48:31.604644+0000","last_fullsized":"2024-05-30T16:48:31.604644+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:15.486340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569290+0000","last_change":"2024-05-30T16:48:29.568315+0000","last_active":"2024-05-30T16:48:31.569290+0000","last_peered":"2024-05-30T16:48:31.569290+0000","last_clean":"2024-05-30T16:48:31.569290+0000","last_became_active":"2024-05-30T16:48:29.567996+0000","last_became_peered":"2024-05-30T16:48:29.567996+0000","last_unstale":"2024-05-30T16:48:31.569290+0000","last_undegraded":"2024-05-30T16:48:31.569290+0000","last_fullsized":"2024-05-30T16:48:31.569290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:23.701866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600792+0000","last_change":"2024-05-30T16:48:30.570026+0000","last_active":"2024-05-30T16:48:31.600792+0000","last_peered":"2024-05-30T16:48:31.600792+0000","last_clean":"2024-05-30T16:48:31.600792+0000","last_became_active":"2024-05-30T16:48:30.567668+0000","last_became_peered":"2024-05-30T16:48:30.567668+0000","last_unstale":"2024-05-30T16:48:31.600792+0000","last_undegraded":"2024-05-30T16:48:31.600792+0000","last_fullsized":"2024-05-30T16:48:31.600792+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:45.715351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567673+0000","last_change":"2024-05-30T16:48:29.561932+0000","last_active":"2024-05-30T16:48:31.567673+0000","last_peered":"2024-05-30T16:48:31.567673+0000","last_clean":"2024-05-30T16:48:31.567673+0000","last_became_active":"2024-05-30T16:48:29.561609+0000","last_became_peered":"2024-05-30T16:48:29.561609+0000","last_unstale":"2024-05-30T16:48:31.567673+0000","last_undegraded":"2024-05-30T16:48:31.567673+0000","last_fullsized":"2024-05-30T16:48:31.567673+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:55:03.134214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602281+0000","last_change":"2024-05-30T16:48:30.570083+0000","last_active":"2024-05-30T16:48:31.602281+0000","last_peered":"2024-05-30T16:48:31.602281+0000","last_clean":"2024-05-30T16:48:31.602281+0000","last_became_active":"2024-05-30T16:48:30.568578+0000","last_became_peered":"2024-05-30T16:48:30.568578+0000","last_unstale":"2024-05-30T16:48:31.602281+0000","last_undegraded":"2024-05-30T16:48:31.602281+0000","last_fullsized":"2024-05-30T16:48:31.602281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:51:20.993001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601117+0000","last_change":"2024-05-30T16:48:29.568131+0000","last_active":"2024-05-30T16:48:31.601117+0000","last_peered":"2024-05-30T16:48:31.601117+0000","last_clean":"2024-05-30T16:48:31.601117+0000","last_became_active":"2024-05-30T16:48:29.567819+0000","last_became_peered":"2024-05-30T16:48:29.567819+0000","last_unstale":"2024-05-30T16:48:31.601117+0000","last_undegraded":"2024-05-30T16:48:31.601117+0000","last_fullsized":"2024-05-30T16:48:31.601117+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:52:26.866201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600660+0000","last_change":"2024-05-30T16:48:30.561051+0000","last_active":"2024-05-30T16:48:31.600660+0000","last_peered":"2024-05-30T16:48:31.600660+0000","last_clean":"2024-05-30T16:48:31.600660+0000","last_became_active":"2024-05-30T16:48:30.560423+0000","last_became_peered":"2024-05-30T16:48:30.560423+0000","last_unstale":"2024-05-30T16:48:31.600660+0000","last_undegraded":"2024-05-30T16:48:31.600660+0000","last_fullsized":"2024-05-30T16:48:31.600660+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:02.774077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569744+0000","last_change":"2024-05-30T16:48:29.566803+0000","last_active":"2024-05-30T16:48:31.569744+0000","last_peered":"2024-05-30T16:48:31.569744+0000","last_clean":"2024-05-30T16:48:31.569744+0000","last_became_active":"2024-05-30T16:48:29.563938+0000","last_became_peered":"2024-05-30T16:48:29.563938+0000","last_unstale":"2024-05-30T16:48:31.569744+0000","last_undegraded":"2024-05-30T16:48:31.569744+0000","last_fullsized":"2024-05-30T16:48:31.569744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:46.892080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602039+0000","last_change":"2024-05-30T16:48:30.573009+0000","last_active":"2024-05-30T16:48:31.602039+0000","last_peered":"2024-05-30T16:48:31.602039+0000","last_clean":"2024-05-30T16:48:31.602039+0000","last_became_active":"2024-05-30T16:48:30.571970+0000","last_became_peered":"2024-05-30T16:48:30.571970+0000","last_unstale":"2024-05-30T16:48:31.602039+0000","last_undegraded":"2024-05-30T16:48:31.602039+0000","last_fullsized":"2024-05-30T16:48:31.602039+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:18.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600710+0000","last_change":"2024-05-30T16:48:29.557714+0000","last_active":"2024-05-30T16:48:31.600710+0000","last_peered":"2024-05-30T16:48:31.600710+0000","last_clean":"2024-05-30T16:48:31.600710+0000","last_became_active":"2024-05-30T16:48:29.557329+0000","last_became_peered":"2024-05-30T16:48:29.557329+0000","last_unstale":"2024-05-30T16:48:31.600710+0000","last_undegraded":"2024-05-30T16:48:31.600710+0000","last_fullsized":"2024-05-30T16:48:31.600710+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:55:42.557260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601207+0000","last_change":"2024-05-30T16:48:30.589751+0000","last_active":"2024-05-30T16:48:31.601207+0000","last_peered":"2024-05-30T16:48:31.601207+0000","last_clean":"2024-05-30T16:48:31.601207+0000","last_became_active":"2024-05-30T16:48:30.589452+0000","last_became_peered":"2024-05-30T16:48:30.589452+0000","last_unstale":"2024-05-30T16:48:31.601207+0000","last_undegraded":"2024-05-30T16:48:31.601207+0000","last_fullsized":"2024-05-30T16:48:31.601207+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:30.133018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601963+0000","last_change":"2024-05-30T16:48:29.562616+0000","last_active":"2024-05-30T16:48:31.601963+0000","last_peered":"2024-05-30T16:48:31.601963+0000","last_clean":"2024-05-30T16:48:31.601963+0000","last_became_active":"2024-05-30T16:48:29.562279+0000","last_became_peered":"2024-05-30T16:48:29.562279+0000","last_unstale":"2024-05-30T16:48:31.601963+0000","last_undegraded":"2024-05-30T16:48:31.601963+0000","last_fullsized":"2024-05-30T16:48:31.601963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:44.174040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570265+0000","last_change":"2024-05-30T16:48:30.577259+0000","last_active":"2024-05-30T16:48:31.570265+0000","last_peered":"2024-05-30T16:48:31.570265+0000","last_clean":"2024-05-30T16:48:31.570265+0000","last_became_active":"2024-05-30T16:48:30.576185+0000","last_became_peered":"2024-05-30T16:48:30.576185+0000","last_unstale":"2024-05-30T16:48:31.570265+0000","last_undegraded":"2024-05-30T16:48:31.570265+0000","last_fullsized":"2024-05-30T16:48:31.570265+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:25.346604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603628+0000","last_change":"2024-05-30T16:48:29.565569+0000","last_active":"2024-05-30T16:48:31.603628+0000","last_peered":"2024-05-30T16:48:31.603628+0000","last_clean":"2024-05-30T16:48:31.603628+0000","last_became_active":"2024-05-30T16:48:29.565276+0000","last_became_peered":"2024-05-30T16:48:29.565276+0000","last_unstale":"2024-05-30T16:48:31.603628+0000","last_undegraded":"2024-05-30T16:48:31.603628+0000","last_fullsized":"2024-05-30T16:48:31.603628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:02.208362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568433+0000","last_change":"2024-05-30T16:48:30.581656+0000","last_active":"2024-05-30T16:48:31.568433+0000","last_peered":"2024-05-30T16:48:31.568433+0000","last_clean":"2024-05-30T16:48:31.568433+0000","last_became_active":"2024-05-30T16:48:30.580423+0000","last_became_peered":"2024-05-30T16:48:30.580423+0000","last_unstale":"2024-05-30T16:48:31.568433+0000","last_undegraded":"2024-05-30T16:48:31.568433+0000","last_fullsized":"2024-05-30T16:48:31.568433+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:06:55.647538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570858+0000","last_change":"2024-05-30T16:48:29.573492+0000","last_active":"2024-05-30T16:48:31.570858+0000","last_peered":"2024-05-30T16:48:31.570858+0000","last_clean":"2024-05-30T16:48:31.570858+0000","last_became_active":"2024-05-30T16:48:29.573157+0000","last_became_peered":"2024-05-30T16:48:29.573157+0000","last_unstale":"2024-05-30T16:48:31.570858+0000","last_undegraded":"2024-05-30T16:48:31.570858+0000","last_fullsized":"2024-05-30T16:48:31.570858+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:14.481047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570294+0000","last_change":"2024-05-30T16:48:30.580712+0000","last_active":"2024-05-30T16:48:31.570294+0000","last_peered":"2024-05-30T16:48:31.570294+0000","last_clean":"2024-05-30T16:48:31.570294+0000","last_became_active":"2024-05-30T16:48:30.580126+0000","last_became_peered":"2024-05-30T16:48:30.580126+0000","last_unstale":"2024-05-30T16:48:31.570294+0000","last_undegraded":"2024-05-30T16:48:31.570294+0000","last_fullsized":"2024-05-30T16:48:31.570294+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:58:49.335864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":20,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601327+0000","last_change":"2024-05-30T16:48:29.564330+0000","last_active":"2024-05-30T16:48:31.601327+0000","last_peered":"2024-05-30T16:48:31.601327+0000","last_clean":"2024-05-30T16:48:31.601327+0000","last_became_active":"2024-05-30T16:48:29.564050+0000","last_became_peered":"2024-05-30T16:48:29.564050+0000","last_unstale":"2024-05-30T16:48:31.601327+0000","last_undegraded":"2024-05-30T16:48:31.601327+0000","last_fullsized":"2024-05-30T16:48:31.601327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:38.418273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602724+0000","last_change":"2024-05-30T16:48:30.587402+0000","last_active":"2024-05-30T16:48:31.602724+0000","last_peered":"2024-05-30T16:48:31.602724+0000","last_clean":"2024-05-30T16:48:31.602724+0000","last_became_active":"2024-05-30T16:48:30.586819+0000","last_became_peered":"2024-05-30T16:48:30.586819+0000","last_unstale":"2024-05-30T16:48:31.602724+0000","last_undegraded":"2024-05-30T16:48:31.602724+0000","last_fullsized":"2024-05-30T16:48:31.602724+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:10:47.394167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567770+0000","last_change":"2024-05-30T16:48:29.556834+0000","last_active":"2024-05-30T16:48:31.567770+0000","last_peered":"2024-05-30T16:48:31.567770+0000","last_clean":"2024-05-30T16:48:31.567770+0000","last_became_active":"2024-05-30T16:48:29.556546+0000","last_became_peered":"2024-05-30T16:48:29.556546+0000","last_unstale":"2024-05-30T16:48:31.567770+0000","last_undegraded":"2024-05-30T16:48:31.567770+0000","last_fullsized":"2024-05-30T16:48:31.567770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:07:33.077371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602679+0000","last_change":"2024-05-30T16:48:30.574973+0000","last_active":"2024-05-30T16:48:31.602679+0000","last_peered":"2024-05-30T16:48:31.602679+0000","last_clean":"2024-05-30T16:48:31.602679+0000","last_became_active":"2024-05-30T16:48:30.574679+0000","last_became_peered":"2024-05-30T16:48:30.574679+0000","last_unstale":"2024-05-30T16:48:31.602679+0000","last_undegraded":"2024-05-30T16:48:31.602679+0000","last_fullsized":"2024-05-30T16:48:31.602679+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:02:00.674835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569040+0000","last_change":"2024-05-30T16:48:29.568023+0000","last_active":"2024-05-30T16:48:31.569040+0000","last_peered":"2024-05-30T16:48:31.569040+0000","last_clean":"2024-05-30T16:48:31.569040+0000","last_became_active":"2024-05-30T16:48:29.567174+0000","last_became_peered":"2024-05-30T16:48:29.567174+0000","last_unstale":"2024-05-30T16:48:31.569040+0000","last_undegraded":"2024-05-30T16:48:31.569040+0000","last_fullsized":"2024-05-30T16:48:31.569040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:59.796399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567307+0000","last_change":"2024-05-30T16:48:30.569413+0000","last_active":"2024-05-30T16:48:31.567307+0000","last_peered":"2024-05-30T16:48:31.567307+0000","last_clean":"2024-05-30T16:48:31.567307+0000","last_became_active":"2024-05-30T16:48:30.569144+0000","last_became_peered":"2024-05-30T16:48:30.569144+0000","last_unstale":"2024-05-30T16:48:31.567307+0000","last_undegraded":"2024-05-30T16:48:31.567307+0000","last_fullsized":"2024-05-30T16:48:31.567307+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:19:21.611466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567474+0000","last_change":"2024-05-30T16:48:29.557641+0000","last_active":"2024-05-30T16:48:31.567474+0000","last_peered":"2024-05-30T16:48:31.567474+0000","last_clean":"2024-05-30T16:48:31.567474+0000","last_became_active":"2024-05-30T16:48:29.556424+0000","last_became_peered":"2024-05-30T16:48:29.556424+0000","last_unstale":"2024-05-30T16:48:31.567474+0000","last_undegraded":"2024-05-30T16:48:31.567474+0000","last_fullsized":"2024-05-30T16:48:31.567474+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:42.420930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569023+0000","last_change":"2024-05-30T16:48:30.575675+0000","last_active":"2024-05-30T16:48:31.569023+0000","last_peered":"2024-05-30T16:48:31.569023+0000","last_clean":"2024-05-30T16:48:31.569023+0000","last_became_active":"2024-05-30T16:48:30.575404+0000","last_became_peered":"2024-05-30T16:48:30.575404+0000","last_unstale":"2024-05-30T16:48:31.569023+0000","last_undegraded":"2024-05-30T16:48:31.569023+0000","last_fullsized":"2024-05-30T16:48:31.569023+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:39:54.346548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":22,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600828+0000","last_change":"2024-05-30T16:48:29.568303+0000","last_active":"2024-05-30T16:48:31.600828+0000","last_peered":"2024-05-30T16:48:31.600828+0000","last_clean":"2024-05-30T16:48:31.600828+0000","last_became_active":"2024-05-30T16:48:29.567991+0000","last_became_peered":"2024-05-30T16:48:29.567991+0000","last_unstale":"2024-05-30T16:48:31.600828+0000","last_undegraded":"2024-05-30T16:48:31.600828+0000","last_fullsized":"2024-05-30T16:48:31.600828+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:31.450890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602073+0000","last_change":"2024-05-30T16:48:30.573320+0000","last_active":"2024-05-30T16:48:31.602073+0000","last_peered":"2024-05-30T16:48:31.602073+0000","last_clean":"2024-05-30T16:48:31.602073+0000","last_became_active":"2024-05-30T16:48:30.572981+0000","last_became_peered":"2024-05-30T16:48:30.572981+0000","last_unstale":"2024-05-30T16:48:31.602073+0000","last_undegraded":"2024-05-30T16:48:31.602073+0000","last_fullsized":"2024-05-30T16:48:31.602073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:18.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601181+0000","last_change":"2024-05-30T16:48:29.547415+0000","last_active":"2024-05-30T16:48:31.601181+0000","last_peered":"2024-05-30T16:48:31.601181+0000","last_clean":"2024-05-30T16:48:31.601181+0000","last_became_active":"2024-05-30T16:48:29.547133+0000","last_became_peered":"2024-05-30T16:48:29.547133+0000","last_unstale":"2024-05-30T16:48:31.601181+0000","last_undegraded":"2024-05-30T16:48:31.601181+0000","last_fullsized":"2024-05-30T16:48:31.601181+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:18.079276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603817+0000","last_change":"2024-05-30T16:48:30.588261+0000","last_active":"2024-05-30T16:48:31.603817+0000","last_peered":"2024-05-30T16:48:31.603817+0000","last_clean":"2024-05-30T16:48:31.603817+0000","last_became_active":"2024-05-30T16:48:30.587964+0000","last_became_peered":"2024-05-30T16:48:30.587964+0000","last_unstale":"2024-05-30T16:48:31.603817+0000","last_undegraded":"2024-05-30T16:48:31.603817+0000","last_fullsized":"2024-05-30T16:48:31.603817+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602712+0000","last_change":"2024-05-30T16:48:29.569871+0000","last_active":"2024-05-30T16:48:31.602712+0000","last_peered":"2024-05-30T16:48:31.602712+0000","last_clean":"2024-05-30T16:48:31.602712+0000","last_became_active":"2024-05-30T16:48:29.569572+0000","last_became_peered":"2024-05-30T16:48:29.569572+0000","last_unstale":"2024-05-30T16:48:31.602712+0000","last_undegraded":"2024-05-30T16:48:31.602712+0000","last_fullsized":"2024-05-30T16:48:31.602712+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:33.957069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602844+0000","last_change":"2024-05-30T16:48:30.580358+0000","last_active":"2024-05-30T16:48:31.602844+0000","last_peered":"2024-05-30T16:48:31.602844+0000","last_clean":"2024-05-30T16:48:31.602844+0000","last_became_active":"2024-05-30T16:48:30.580087+0000","last_became_peered":"2024-05-30T16:48:30.580087+0000","last_unstale":"2024-05-30T16:48:31.602844+0000","last_undegraded":"2024-05-30T16:48:31.602844+0000","last_fullsized":"2024-05-30T16:48:31.602844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:11.840908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570987+0000","last_change":"2024-05-30T16:48:29.556896+0000","last_active":"2024-05-30T16:48:31.570987+0000","last_peered":"2024-05-30T16:48:31.570987+0000","last_clean":"2024-05-30T16:48:31.570987+0000","last_became_active":"2024-05-30T16:48:29.556569+0000","last_became_peered":"2024-05-30T16:48:29.556569+0000","last_unstale":"2024-05-30T16:48:31.570987+0000","last_undegraded":"2024-05-30T16:48:31.570987+0000","last_fullsized":"2024-05-30T16:48:31.570987+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:27.482686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602981+0000","last_change":"2024-05-30T16:48:30.569176+0000","last_active":"2024-05-30T16:48:31.602981+0000","last_peered":"2024-05-30T16:48:31.602981+0000","last_clean":"2024-05-30T16:48:31.602981+0000","last_became_active":"2024-05-30T16:48:30.568847+0000","last_became_peered":"2024-05-30T16:48:30.568847+0000","last_unstale":"2024-05-30T16:48:31.602981+0000","last_undegraded":"2024-05-30T16:48:31.602981+0000","last_fullsized":"2024-05-30T16:48:31.602981+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:39.409875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568330+0000","last_change":"2024-05-30T16:48:29.565727+0000","last_active":"2024-05-30T16:48:31.568330+0000","last_peered":"2024-05-30T16:48:31.568330+0000","last_clean":"2024-05-30T16:48:31.568330+0000","last_became_active":"2024-05-30T16:48:29.565445+0000","last_became_peered":"2024-05-30T16:48:29.565445+0000","last_unstale":"2024-05-30T16:48:31.568330+0000","last_undegraded":"2024-05-30T16:48:31.568330+0000","last_fullsized":"2024-05-30T16:48:31.568330+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:29:03.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567327+0000","last_change":"2024-05-30T16:48:30.573197+0000","last_active":"2024-05-30T16:48:31.567327+0000","last_peered":"2024-05-30T16:48:31.567327+0000","last_clean":"2024-05-30T16:48:31.567327+0000","last_became_active":"2024-05-30T16:48:30.572912+0000","last_became_peered":"2024-05-30T16:48:30.572912+0000","last_unstale":"2024-05-30T16:48:31.567327+0000","last_undegraded":"2024-05-30T16:48:31.567327+0000","last_fullsized":"2024-05-30T16:48:31.567327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:02:23.023604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568144+0000","last_change":"2024-05-30T16:48:29.562648+0000","last_active":"2024-05-30T16:48:31.568144+0000","last_peered":"2024-05-30T16:48:31.568144+0000","last_clean":"2024-05-30T16:48:31.568144+0000","last_became_active":"2024-05-30T16:48:29.562380+0000","last_became_peered":"2024-05-30T16:48:29.562380+0000","last_unstale":"2024-05-30T16:48:31.568144+0000","last_undegraded":"2024-05-30T16:48:31.568144+0000","last_fullsized":"2024-05-30T16:48:31.568144+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:06.069947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604190+0000","last_change":"2024-05-30T16:48:30.552462+0000","last_active":"2024-05-30T16:48:31.604190+0000","last_peered":"2024-05-30T16:48:31.604190+0000","last_clean":"2024-05-30T16:48:31.604190+0000","last_became_active":"2024-05-30T16:48:30.552175+0000","last_became_peered":"2024-05-30T16:48:30.552175+0000","last_unstale":"2024-05-30T16:48:31.604190+0000","last_undegraded":"2024-05-30T16:48:31.604190+0000","last_fullsized":"2024-05-30T16:48:31.604190+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:37:16.441185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569935+0000","last_change":"2024-05-30T16:48:29.564622+0000","last_active":"2024-05-30T16:48:31.569935+0000","last_peered":"2024-05-30T16:48:31.569935+0000","last_clean":"2024-05-30T16:48:31.569935+0000","last_became_active":"2024-05-30T16:48:29.564294+0000","last_became_peered":"2024-05-30T16:48:29.564294+0000","last_unstale":"2024-05-30T16:48:31.569935+0000","last_undegraded":"2024-05-30T16:48:31.569935+0000","last_fullsized":"2024-05-30T16:48:31.569935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:01.759242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601535+0000","last_change":"2024-05-30T16:48:30.572550+0000","last_active":"2024-05-30T16:48:31.601535+0000","last_peered":"2024-05-30T16:48:31.601535+0000","last_clean":"2024-05-30T16:48:31.601535+0000","last_became_active":"2024-05-30T16:48:30.572162+0000","last_became_peered":"2024-05-30T16:48:30.572162+0000","last_unstale":"2024-05-30T16:48:31.601535+0000","last_undegraded":"2024-05-30T16:48:31.601535+0000","last_fullsized":"2024-05-30T16:48:31.601535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:52:27.881036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601305+0000","last_change":"2024-05-30T16:48:29.565513+0000","last_active":"2024-05-30T16:48:31.601305+0000","last_peered":"2024-05-30T16:48:31.601305+0000","last_clean":"2024-05-30T16:48:31.601305+0000","last_became_active":"2024-05-30T16:48:29.565019+0000","last_became_peered":"2024-05-30T16:48:29.565019+0000","last_unstale":"2024-05-30T16:48:31.601305+0000","last_undegraded":"2024-05-30T16:48:31.601305+0000","last_fullsized":"2024-05-30T16:48:31.601305+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:43.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601161+0000","last_change":"2024-05-30T16:48:30.554107+0000","last_active":"2024-05-30T16:48:31.601161+0000","last_peered":"2024-05-30T16:48:31.601161+0000","last_clean":"2024-05-30T16:48:31.601161+0000","last_became_active":"2024-05-30T16:48:30.553499+0000","last_became_peered":"2024-05-30T16:48:30.553499+0000","last_unstale":"2024-05-30T16:48:31.601161+0000","last_undegraded":"2024-05-30T16:48:31.601161+0000","last_fullsized":"2024-05-30T16:48:31.601161+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:28.497521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568109+0000","last_change":"2024-05-30T16:48:29.565658+0000","last_active":"2024-05-30T16:48:31.568109+0000","last_peered":"2024-05-30T16:48:31.568109+0000","last_clean":"2024-05-30T16:48:31.568109+0000","last_became_active":"2024-05-30T16:48:29.565325+0000","last_became_peered":"2024-05-30T16:48:29.565325+0000","last_unstale":"2024-05-30T16:48:31.568109+0000","last_undegraded":"2024-05-30T16:48:31.568109+0000","last_fullsized":"2024-05-30T16:48:31.568109+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:00:28.354021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603385+0000","last_change":"2024-05-30T16:48:30.589674+0000","last_active":"2024-05-30T16:48:31.603385+0000","last_peered":"2024-05-30T16:48:31.603385+0000","last_clean":"2024-05-30T16:48:31.603385+0000","last_became_active":"2024-05-30T16:48:30.589308+0000","last_became_peered":"2024-05-30T16:48:30.589308+0000","last_unstale":"2024-05-30T16:48:31.603385+0000","last_undegraded":"2024-05-30T16:48:31.603385+0000","last_fullsized":"2024-05-30T16:48:31.603385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:37:16.441185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568417+0000","last_change":"2024-05-30T16:48:29.563607+0000","last_active":"2024-05-30T16:48:31.568417+0000","last_peered":"2024-05-30T16:48:31.568417+0000","last_clean":"2024-05-30T16:48:31.568417+0000","last_became_active":"2024-05-30T16:48:29.563281+0000","last_became_peered":"2024-05-30T16:48:29.563281+0000","last_unstale":"2024-05-30T16:48:31.568417+0000","last_undegraded":"2024-05-30T16:48:31.568417+0000","last_fullsized":"2024-05-30T16:48:31.568417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:29.072276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603410+0000","last_change":"2024-05-30T16:48:30.577542+0000","last_active":"2024-05-30T16:48:31.603410+0000","last_peered":"2024-05-30T16:48:31.603410+0000","last_clean":"2024-05-30T16:48:31.603410+0000","last_became_active":"2024-05-30T16:48:30.576718+0000","last_became_peered":"2024-05-30T16:48:30.576718+0000","last_unstale":"2024-05-30T16:48:31.603410+0000","last_undegraded":"2024-05-30T16:48:31.603410+0000","last_fullsized":"2024-05-30T16:48:31.603410+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:24.028012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570863+0000","last_change":"2024-05-30T16:48:29.572620+0000","last_active":"2024-05-30T16:48:31.570863+0000","last_peered":"2024-05-30T16:48:31.570863+0000","last_clean":"2024-05-30T16:48:31.570863+0000","last_became_active":"2024-05-30T16:48:29.572329+0000","last_became_peered":"2024-05-30T16:48:29.572329+0000","last_unstale":"2024-05-30T16:48:31.570863+0000","last_undegraded":"2024-05-30T16:48:31.570863+0000","last_fullsized":"2024-05-30T16:48:31.570863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:42:59.187391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570734+0000","last_change":"2024-05-30T16:48:30.578757+0000","last_active":"2024-05-30T16:48:31.570734+0000","last_peered":"2024-05-30T16:48:31.570734+0000","last_clean":"2024-05-30T16:48:31.570734+0000","last_became_active":"2024-05-30T16:48:30.578484+0000","last_became_peered":"2024-05-30T16:48:30.578484+0000","last_unstale":"2024-05-30T16:48:31.570734+0000","last_undegraded":"2024-05-30T16:48:31.570734+0000","last_fullsized":"2024-05-30T16:48:31.570734+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:47.794458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604674+0000","last_change":"2024-05-30T16:48:29.563508+0000","last_active":"2024-05-30T16:48:31.604674+0000","last_peered":"2024-05-30T16:48:31.604674+0000","last_clean":"2024-05-30T16:48:31.604674+0000","last_became_active":"2024-05-30T16:48:29.563206+0000","last_became_peered":"2024-05-30T16:48:29.563206+0000","last_unstale":"2024-05-30T16:48:31.604674+0000","last_undegraded":"2024-05-30T16:48:31.604674+0000","last_fullsized":"2024-05-30T16:48:31.604674+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:44.174040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567764+0000","last_change":"2024-05-30T16:48:30.558867+0000","last_active":"2024-05-30T16:48:31.567764+0000","last_peered":"2024-05-30T16:48:31.567764+0000","last_clean":"2024-05-30T16:48:31.567764+0000","last_became_active":"2024-05-30T16:48:30.558563+0000","last_became_peered":"2024-05-30T16:48:30.558563+0000","last_unstale":"2024-05-30T16:48:31.567764+0000","last_undegraded":"2024-05-30T16:48:31.567764+0000","last_fullsized":"2024-05-30T16:48:31.567764+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:07.248500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569370+0000","last_change":"2024-05-30T16:48:29.573875+0000","last_active":"2024-05-30T16:48:31.569370+0000","last_peered":"2024-05-30T16:48:31.569370+0000","last_clean":"2024-05-30T16:48:31.569370+0000","last_became_active":"2024-05-30T16:48:29.572773+0000","last_became_peered":"2024-05-30T16:48:29.572773+0000","last_unstale":"2024-05-30T16:48:31.569370+0000","last_undegraded":"2024-05-30T16:48:31.569370+0000","last_fullsized":"2024-05-30T16:48:31.569370+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:23.013176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567552+0000","last_change":"2024-05-30T16:48:30.573312+0000","last_active":"2024-05-30T16:48:31.567552+0000","last_peered":"2024-05-30T16:48:31.567552+0000","last_clean":"2024-05-30T16:48:31.567552+0000","last_became_active":"2024-05-30T16:48:30.572980+0000","last_became_peered":"2024-05-30T16:48:30.572980+0000","last_unstale":"2024-05-30T16:48:31.567552+0000","last_undegraded":"2024-05-30T16:48:31.567552+0000","last_fullsized":"2024-05-30T16:48:31.567552+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:22.477685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601134+0000","last_change":"2024-05-30T16:48:29.569525+0000","last_active":"2024-05-30T16:48:31.601134+0000","last_peered":"2024-05-30T16:48:31.601134+0000","last_clean":"2024-05-30T16:48:31.601134+0000","last_became_active":"2024-05-30T16:48:29.568636+0000","last_became_peered":"2024-05-30T16:48:29.568636+0000","last_unstale":"2024-05-30T16:48:31.601134+0000","last_undegraded":"2024-05-30T16:48:31.601134+0000","last_fullsized":"2024-05-30T16:48:31.601134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:17.171552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569918+0000","last_change":"2024-05-30T16:48:30.553665+0000","last_active":"2024-05-30T16:48:31.569918+0000","last_peered":"2024-05-30T16:48:31.569918+0000","last_clean":"2024-05-30T16:48:31.569918+0000","last_became_active":"2024-05-30T16:48:30.552512+0000","last_became_peered":"2024-05-30T16:48:30.552512+0000","last_unstale":"2024-05-30T16:48:31.569918+0000","last_undegraded":"2024-05-30T16:48:31.569918+0000","last_fullsized":"2024-05-30T16:48:31.569918+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604236+0000","last_change":"2024-05-30T16:48:29.572007+0000","last_active":"2024-05-30T16:48:31.604236+0000","last_peered":"2024-05-30T16:48:31.604236+0000","last_clean":"2024-05-30T16:48:31.604236+0000","last_became_active":"2024-05-30T16:48:29.571652+0000","last_became_peered":"2024-05-30T16:48:29.571652+0000","last_unstale":"2024-05-30T16:48:31.604236+0000","last_undegraded":"2024-05-30T16:48:31.604236+0000","last_fullsized":"2024-05-30T16:48:31.604236+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:10:46.379332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600726+0000","last_change":"2024-05-30T16:48:30.583592+0000","last_active":"2024-05-30T16:48:31.600726+0000","last_peered":"2024-05-30T16:48:31.600726+0000","last_clean":"2024-05-30T16:48:31.600726+0000","last_became_active":"2024-05-30T16:48:30.583350+0000","last_became_peered":"2024-05-30T16:48:30.583350+0000","last_unstale":"2024-05-30T16:48:31.600726+0000","last_undegraded":"2024-05-30T16:48:31.600726+0000","last_fullsized":"2024-05-30T16:48:31.600726+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:02.683671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600934+0000","last_change":"2024-05-30T16:48:29.554465+0000","last_active":"2024-05-30T16:48:31.600934+0000","last_peered":"2024-05-30T16:48:31.600934+0000","last_clean":"2024-05-30T16:48:31.600934+0000","last_became_active":"2024-05-30T16:48:29.554117+0000","last_became_peered":"2024-05-30T16:48:29.554117+0000","last_unstale":"2024-05-30T16:48:31.600934+0000","last_undegraded":"2024-05-30T16:48:31.600934+0000","last_fullsized":"2024-05-30T16:48:31.600934+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:30.281975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603615+0000","last_change":"2024-05-30T16:48:30.585362+0000","last_active":"2024-05-30T16:48:31.603615+0000","last_peered":"2024-05-30T16:48:31.603615+0000","last_clean":"2024-05-30T16:48:31.603615+0000","last_became_active":"2024-05-30T16:48:30.583719+0000","last_became_peered":"2024-05-30T16:48:30.583719+0000","last_unstale":"2024-05-30T16:48:31.603615+0000","last_undegraded":"2024-05-30T16:48:31.603615+0000","last_fullsized":"2024-05-30T16:48:31.603615+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570531+0000","last_change":"2024-05-30T16:48:29.558681+0000","last_active":"2024-05-30T16:48:31.570531+0000","last_peered":"2024-05-30T16:48:31.570531+0000","last_clean":"2024-05-30T16:48:31.570531+0000","last_became_active":"2024-05-30T16:48:29.558149+0000","last_became_peered":"2024-05-30T16:48:29.558149+0000","last_unstale":"2024-05-30T16:48:31.570531+0000","last_undegraded":"2024-05-30T16:48:31.570531+0000","last_fullsized":"2024-05-30T16:48:31.570531+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:40.373552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":221,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568175+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:48:31.568175+0000","last_peered":"2024-05-30T16:48:31.568175+0000","last_clean":"2024-05-30T16:48:31.568175+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:48:31.568175+0000","last_undegraded":"2024-05-30T16:48:31.568175+0000","last_fullsized":"2024-05-30T16:48:31.568175+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":565248,"data_stored":229099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35896,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335944,"statfs":{"total":96636764160,"available":96600006656,"internally_reserved":0,"allocated":569344,"data_stored":230039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509503,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":569344,"data_stored":229065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35904,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335936,"statfs":{"total":96636764160,"available":96599998464,"internally_reserved":0,"allocated":573440,"data_stored":230061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":561152,"data_stored":228497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36468,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335372,"statfs":{"total":96636764160,"available":96599420928,"internally_reserved":0,"allocated":1155072,"data_stored":818885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35896,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335944,"statfs":{"total":96636764160,"available":96600006656,"internally_reserved":0,"allocated":561152,"data_stored":228497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36472,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335368,"statfs":{"total":96636764160,"available":96599416832,"internally_reserved":0,"allocated":1155072,"data_stored":818829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:48:39.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:48:39.924 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:48:40.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:48:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.342+0000 7f7d87200640 1 Processor -- start 2024-05-30T16:48:40.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.342+0000 7f7d87200640 1 -- start start 2024-05-30T16:48:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.344+0000 7f7d87200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d8806ee40 0x7f7d880ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.344+0000 7f7d87200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7d8806f2c0 0x7f7d880ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.344+0000 7f7d87200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d88156f10 con 0x7f7d88156b30 2024-05-30T16:48:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.344+0000 7f7d87200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8806dda0 con 0x7f7d8806f2c0 2024-05-30T16:48:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.344+0000 7f7d87200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8806e080 con 0x7f7d8806ee40 2024-05-30T16:48:40.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.344+0000 7f7d85e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7d88156b30 0x7f7d880af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52174/0 (socket says 172.21.15.67:52174) 2024-05-30T16:48:40.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.344+0000 7f7d85e00640 1 -- 172.21.15.67:0/3270660512 learned_addr learned my addr 172.21.15.67:0/3270660512 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:40.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d86800640 1 --2- 172.21.15.67:0/3270660512 >> v2:172.21.15.169:3300/0 conn(0x7f7d8806f2c0 0x7f7d880ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d7fe00640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d8806ee40 0x7f7d880ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1403739774 0 0) 0x7f7d88156f10 con 0x7f7d88156b30 2024-05-30T16:48:40.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d6c003520 con 0x7f7d88156b30 2024-05-30T16:48:40.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1814937621 0 0) 0x7f7d6c003520 con 0x7f7d88156b30 2024-05-30T16:48:40.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d8806ee40 msgr2=0x7f7d880ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:40.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d8806ee40 0x7f7d880ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 >> v2:172.21.15.169:3300/0 conn(0x7f7d8806f2c0 msgr2=0x7f7d880ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:40.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 --2- 172.21.15.67:0/3270660512 >> v2:172.21.15.169:3300/0 conn(0x7f7d8806f2c0 0x7f7d880ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8806e360 con 0x7f7d88156b30 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7d78001670 con 0x7f7d88156b30 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 >> v1:172.21.15.67:6789/0 conn(0x7f7d88156b30 legacy=0x7f7d880af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.345+0000 7f7d85400640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d8806ee40 0x7f7d6c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d85400640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c003f30 0x7f7d6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d85400640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c004d40 0x7f7d6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8806dda0 con 0x7f7d6c004d40 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8806e080 con 0x7f7d6c003f30 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c00ba10 con 0x7f7d8806ee40 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d86800640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c004d40 0x7f7d6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d85e00640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d8806ee40 0x7f7d6c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d7fe00640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c003f30 0x7f7d6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d86800640 1 -- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d8806ee40 msgr2=0x7f7d6c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d86800640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d8806ee40 0x7f7d6c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d86800640 1 -- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c003f30 msgr2=0x7f7d6c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d86800640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6c003f30 0x7f7d6c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.346+0000 7f7d86800640 1 -- 172.21.15.67:0/3270660512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c003520 con 0x7f7d6c004d40 2024-05-30T16:48:40.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.348+0000 7f7d86800640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c004d40 0x7f7d6c0051c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000e170 tx=0x7f7d6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f88090e5a32847 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:40.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.348+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80034020 con 0x7f7d6c004d40 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.348+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d80030430 con 0x7f7d6c004d40 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.348+0000 7f7d85400640 1 -- 172.21.15.67:0/3270660512 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80009c10 con 0x7f7d6c004d40 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.348+0000 7f7d87200640 1 -- 172.21.15.67:0/3270660512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c004d40 msgr2=0x7f7d6c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.349+0000 7f7d87200640 1 --2- 172.21.15.67:0/3270660512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d6c004d40 0x7f7d6c0051c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000e170 tx=0x7f7d6c005830 comp rx=0 tx=0).stop 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.349+0000 7f7d86800640 1 -- 172.21.15.67:0/3270660512 reap_dead start 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.349+0000 7f7d87200640 1 -- 172.21.15.67:0/3270660512 shutdown_connections 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.349+0000 7f7d87200640 1 -- 172.21.15.67:0/3270660512 >> 172.21.15.67:0/3270660512 conn(0x7f7d880b1fc0 msgr2=0x7f7d880b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.349+0000 7f7d87200640 1 -- 172.21.15.67:0/3270660512 shutdown_connections 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.349+0000 7f7d87200640 1 -- 172.21.15.67:0/3270660512 wait complete. 2024-05-30T16:48:40.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.351+0000 7f7d87200640 1 Processor -- start 2024-05-30T16:48:40.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.351+0000 7f7d87200640 1 -- start start 2024-05-30T16:48:40.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d87200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c004d40 0x7f7d88148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d87200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d88157d70 0x7f7d881581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d87200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8815e960 0x7f7d8815ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d87200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7d880b2db0 con 0x7f7d8815e960 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d87200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d880b2c30 con 0x7f7d88157d70 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d87200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d880ad230 con 0x7f7d6c004d40 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d86800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8815e960 0x7f7d8815ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d86800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8815e960 0x7f7d8815ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58132/0 (socket says 172.21.15.67:58132) 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d86800640 1 -- 172.21.15.67:0/916332148 learned_addr learned my addr 172.21.15.67:0/916332148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d85e00640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c004d40 0x7f7d88148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d7fe00640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d88157d70 0x7f7d881581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d86800640 1 -- 172.21.15.67:0/916332148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c004d40 msgr2=0x7f7d88148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d86800640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c004d40 0x7f7d88148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d86800640 1 -- 172.21.15.67:0/916332148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d88157d70 msgr2=0x7f7d881581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d86800640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d88157d70 0x7f7d881581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d86800640 1 -- 172.21.15.67:0/916332148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d880b19a0 con 0x7f7d8815e960 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.352+0000 7f7d85e00640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c004d40 0x7f7d88148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.353+0000 7f7d86800640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8815e960 0x7f7d8815ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009f90 tx=0x7f7d80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.353+0000 7f7d7e000640 1 -- 172.21.15.67:0/916332148 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80034040 con 0x7f7d8815e960 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.353+0000 7f7d7e000640 1 -- 172.21.15.67:0/916332148 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d8003e070 con 0x7f7d8815e960 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.353+0000 7f7d7e000640 1 -- 172.21.15.67:0/916332148 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80009c10 con 0x7f7d8815e960 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.353+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d88148fb0 con 0x7f7d8815e960 2024-05-30T16:48:40.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.353+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d880af300 con 0x7f7d8815e960 2024-05-30T16:48:40.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.355+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d880689e0 con 0x7f7d8815e960 2024-05-30T16:48:40.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.356+0000 7f7d7e000640 1 -- 172.21.15.67:0/916332148 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f7d80005d10 con 0x7f7d8815e960 2024-05-30T16:48:40.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.356+0000 7f7d7e000640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d6406cbf0 0x7f7d6406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.357+0000 7f7d7e000640 1 -- 172.21.15.67:0/916332148 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f7d800ac930 con 0x7f7d8815e960 2024-05-30T16:48:40.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.357+0000 7f7d85e00640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d6406cbf0 0x7f7d6406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.358+0000 7f7d85e00640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d6406cbf0 0x7f7d6406f0b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7d780017e0 tx=0x7f7d78001db0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:40.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.361+0000 7f7d7e000640 1 -- 172.21.15.67:0/916332148 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d880b19a0 con 0x7f7d8815e960 2024-05-30T16:48:40.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.561+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d8806f7f0 con 0x7f7d6406cbf0 2024-05-30T16:48:40.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.571+0000 7f7d7e000640 1 -- 172.21.15.67:0/916332148 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7f7d8806f7f0 con 0x7f7d6406cbf0 2024-05-30T16:48:40.576 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:40.578 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:48:40.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d6406cbf0 msgr2=0x7f7d6406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d6406cbf0 0x7f7d6406f0b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7d780017e0 tx=0x7f7d78001db0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8815e960 msgr2=0x7f7d8815ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8815e960 0x7f7d8815ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009f90 tx=0x7f7d80009750 comp rx=0 tx=0).stop 2024-05-30T16:48:40.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 shutdown_connections 2024-05-30T16:48:40.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d6406cbf0 0x7f7d6406f0b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d8815e960 0x7f7d8815ede0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d88157d70 0x7f7d881581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 --2- 172.21.15.67:0/916332148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6c004d40 0x7f7d88148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 >> 172.21.15.67:0/916332148 conn(0x7f7d880b1fc0 msgr2=0x7f7d8814ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:40.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 shutdown_connections 2024-05-30T16:48:40.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.576+0000 7f7d87200640 1 -- 172.21.15.67:0/916332148 wait complete. 2024-05-30T16:48:40.598 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-05-30T16:48:40.088974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":288320,"kb_used_data":5576,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754686400,"statfs":{"total":773094113280,"available":772798873600,"internally_reserved":0,"allocated":5709824,"data_stored":3012972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.552532"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569722+0000","last_change":"2024-05-30T16:48:30.576499+0000","last_active":"2024-05-30T16:48:31.569722+0000","last_peered":"2024-05-30T16:48:31.569722+0000","last_clean":"2024-05-30T16:48:31.569722+0000","last_became_active":"2024-05-30T16:48:30.576126+0000","last_became_peered":"2024-05-30T16:48:30.576126+0000","last_unstale":"2024-05-30T16:48:31.569722+0000","last_undegraded":"2024-05-30T16:48:31.569722+0000","last_fullsized":"2024-05-30T16:48:31.569722+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:20:17.680707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570382+0000","last_change":"2024-05-30T16:48:30.584563+0000","last_active":"2024-05-30T16:48:31.570382+0000","last_peered":"2024-05-30T16:48:31.570382+0000","last_clean":"2024-05-30T16:48:31.570382+0000","last_became_active":"2024-05-30T16:48:30.584262+0000","last_became_peered":"2024-05-30T16:48:30.584262+0000","last_unstale":"2024-05-30T16:48:31.570382+0000","last_undegraded":"2024-05-30T16:48:31.570382+0000","last_fullsized":"2024-05-30T16:48:31.570382+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601162+0000","last_change":"2024-05-30T16:48:30.577269+0000","last_active":"2024-05-30T16:48:31.601162+0000","last_peered":"2024-05-30T16:48:31.601162+0000","last_clean":"2024-05-30T16:48:31.601162+0000","last_became_active":"2024-05-30T16:48:30.576986+0000","last_became_peered":"2024-05-30T16:48:30.576986+0000","last_unstale":"2024-05-30T16:48:31.601162+0000","last_undegraded":"2024-05-30T16:48:31.601162+0000","last_fullsized":"2024-05-30T16:48:31.601162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:37.663916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603004+0000","last_change":"2024-05-30T16:48:30.564821+0000","last_active":"2024-05-30T16:48:31.603004+0000","last_peered":"2024-05-30T16:48:31.603004+0000","last_clean":"2024-05-30T16:48:31.603004+0000","last_became_active":"2024-05-30T16:48:30.564537+0000","last_became_peered":"2024-05-30T16:48:30.564537+0000","last_unstale":"2024-05-30T16:48:31.603004+0000","last_undegraded":"2024-05-30T16:48:31.603004+0000","last_fullsized":"2024-05-30T16:48:31.603004+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:00:29.368856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570614+0000","last_change":"2024-05-30T16:48:30.565013+0000","last_active":"2024-05-30T16:48:31.570614+0000","last_peered":"2024-05-30T16:48:31.570614+0000","last_clean":"2024-05-30T16:48:31.570614+0000","last_became_active":"2024-05-30T16:48:30.564344+0000","last_became_peered":"2024-05-30T16:48:30.564344+0000","last_unstale":"2024-05-30T16:48:31.570614+0000","last_undegraded":"2024-05-30T16:48:31.570614+0000","last_fullsized":"2024-05-30T16:48:31.570614+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:38.774999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567205+0000","last_change":"2024-05-30T16:48:30.573684+0000","last_active":"2024-05-30T16:48:31.567205+0000","last_peered":"2024-05-30T16:48:31.567205+0000","last_clean":"2024-05-30T16:48:31.567205+0000","last_became_active":"2024-05-30T16:48:30.573404+0000","last_became_peered":"2024-05-30T16:48:30.573404+0000","last_unstale":"2024-05-30T16:48:31.567205+0000","last_undegraded":"2024-05-30T16:48:31.567205+0000","last_fullsized":"2024-05-30T16:48:31.567205+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:49:08.996106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568975+0000","last_change":"2024-05-30T16:48:30.576603+0000","last_active":"2024-05-30T16:48:31.568975+0000","last_peered":"2024-05-30T16:48:31.568975+0000","last_clean":"2024-05-30T16:48:31.568975+0000","last_became_active":"2024-05-30T16:48:30.576320+0000","last_became_peered":"2024-05-30T16:48:30.576320+0000","last_unstale":"2024-05-30T16:48:31.568975+0000","last_undegraded":"2024-05-30T16:48:31.568975+0000","last_fullsized":"2024-05-30T16:48:31.568975+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:25.686849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600493+0000","last_change":"2024-05-30T16:48:30.561287+0000","last_active":"2024-05-30T16:48:31.600493+0000","last_peered":"2024-05-30T16:48:31.600493+0000","last_clean":"2024-05-30T16:48:31.600493+0000","last_became_active":"2024-05-30T16:48:30.560737+0000","last_became_peered":"2024-05-30T16:48:30.560737+0000","last_unstale":"2024-05-30T16:48:31.600493+0000","last_undegraded":"2024-05-30T16:48:31.600493+0000","last_fullsized":"2024-05-30T16:48:31.600493+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:50:31.061295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603257+0000","last_change":"2024-05-30T16:48:30.587362+0000","last_active":"2024-05-30T16:48:31.603257+0000","last_peered":"2024-05-30T16:48:31.603257+0000","last_clean":"2024-05-30T16:48:31.603257+0000","last_became_active":"2024-05-30T16:48:30.587100+0000","last_became_peered":"2024-05-30T16:48:30.587100+0000","last_unstale":"2024-05-30T16:48:31.603257+0000","last_undegraded":"2024-05-30T16:48:31.603257+0000","last_fullsized":"2024-05-30T16:48:31.603257+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:51:00.811234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603466+0000","last_change":"2024-05-30T16:48:30.582474+0000","last_active":"2024-05-30T16:48:31.603466+0000","last_peered":"2024-05-30T16:48:31.603466+0000","last_clean":"2024-05-30T16:48:31.603466+0000","last_became_active":"2024-05-30T16:48:30.582148+0000","last_became_peered":"2024-05-30T16:48:30.582148+0000","last_unstale":"2024-05-30T16:48:31.603466+0000","last_undegraded":"2024-05-30T16:48:31.603466+0000","last_fullsized":"2024-05-30T16:48:31.603466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602911+0000","last_change":"2024-05-30T16:48:30.577384+0000","last_active":"2024-05-30T16:48:31.602911+0000","last_peered":"2024-05-30T16:48:31.602911+0000","last_clean":"2024-05-30T16:48:31.602911+0000","last_became_active":"2024-05-30T16:48:30.570714+0000","last_became_peered":"2024-05-30T16:48:30.570714+0000","last_unstale":"2024-05-30T16:48:31.602911+0000","last_undegraded":"2024-05-30T16:48:31.602911+0000","last_fullsized":"2024-05-30T16:48:31.602911+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:05.025187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570620+0000","last_change":"2024-05-30T16:48:30.582165+0000","last_active":"2024-05-30T16:48:31.570620+0000","last_peered":"2024-05-30T16:48:31.570620+0000","last_clean":"2024-05-30T16:48:31.570620+0000","last_became_active":"2024-05-30T16:48:30.581896+0000","last_became_peered":"2024-05-30T16:48:30.581896+0000","last_unstale":"2024-05-30T16:48:31.570620+0000","last_undegraded":"2024-05-30T16:48:31.570620+0000","last_fullsized":"2024-05-30T16:48:31.570620+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602235+0000","last_change":"2024-05-30T16:48:30.585005+0000","last_active":"2024-05-30T16:48:31.602235+0000","last_peered":"2024-05-30T16:48:31.602235+0000","last_clean":"2024-05-30T16:48:31.602235+0000","last_became_active":"2024-05-30T16:48:30.584718+0000","last_became_peered":"2024-05-30T16:48:30.584718+0000","last_unstale":"2024-05-30T16:48:31.602235+0000","last_undegraded":"2024-05-30T16:48:31.602235+0000","last_fullsized":"2024-05-30T16:48:31.602235+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:34.971904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601441+0000","last_change":"2024-05-30T16:48:30.555321+0000","last_active":"2024-05-30T16:48:31.601441+0000","last_peered":"2024-05-30T16:48:31.601441+0000","last_clean":"2024-05-30T16:48:31.601441+0000","last_became_active":"2024-05-30T16:48:30.555012+0000","last_became_peered":"2024-05-30T16:48:30.555012+0000","last_unstale":"2024-05-30T16:48:31.601441+0000","last_undegraded":"2024-05-30T16:48:31.601441+0000","last_fullsized":"2024-05-30T16:48:31.601441+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:41.388387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567200+0000","last_change":"2024-05-30T16:48:30.574629+0000","last_active":"2024-05-30T16:48:31.567200+0000","last_peered":"2024-05-30T16:48:31.567200+0000","last_clean":"2024-05-30T16:48:31.567200+0000","last_became_active":"2024-05-30T16:48:30.574307+0000","last_became_peered":"2024-05-30T16:48:30.574307+0000","last_unstale":"2024-05-30T16:48:31.567200+0000","last_undegraded":"2024-05-30T16:48:31.567200+0000","last_fullsized":"2024-05-30T16:48:31.567200+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:13.127547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602095+0000","last_change":"2024-05-30T16:48:30.572742+0000","last_active":"2024-05-30T16:48:31.602095+0000","last_peered":"2024-05-30T16:48:31.602095+0000","last_clean":"2024-05-30T16:48:31.602095+0000","last_became_active":"2024-05-30T16:48:30.572396+0000","last_became_peered":"2024-05-30T16:48:30.572396+0000","last_unstale":"2024-05-30T16:48:31.602095+0000","last_undegraded":"2024-05-30T16:48:31.602095+0000","last_fullsized":"2024-05-30T16:48:31.602095+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:39.464041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601466+0000","last_change":"2024-05-30T16:48:30.574852+0000","last_active":"2024-05-30T16:48:31.601466+0000","last_peered":"2024-05-30T16:48:31.601466+0000","last_clean":"2024-05-30T16:48:31.601466+0000","last_became_active":"2024-05-30T16:48:30.574568+0000","last_became_peered":"2024-05-30T16:48:30.574568+0000","last_unstale":"2024-05-30T16:48:31.601466+0000","last_undegraded":"2024-05-30T16:48:31.601466+0000","last_fullsized":"2024-05-30T16:48:31.601466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:38:07.252870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602414+0000","last_change":"2024-05-30T16:48:30.589719+0000","last_active":"2024-05-30T16:48:31.602414+0000","last_peered":"2024-05-30T16:48:31.602414+0000","last_clean":"2024-05-30T16:48:31.602414+0000","last_became_active":"2024-05-30T16:48:30.589375+0000","last_became_peered":"2024-05-30T16:48:30.589375+0000","last_unstale":"2024-05-30T16:48:31.602414+0000","last_undegraded":"2024-05-30T16:48:31.602414+0000","last_fullsized":"2024-05-30T16:48:31.602414+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:54.229780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603555+0000","last_change":"2024-05-30T16:48:30.587045+0000","last_active":"2024-05-30T16:48:31.603555+0000","last_peered":"2024-05-30T16:48:31.603555+0000","last_clean":"2024-05-30T16:48:31.603555+0000","last_became_active":"2024-05-30T16:48:30.586703+0000","last_became_peered":"2024-05-30T16:48:30.586703+0000","last_unstale":"2024-05-30T16:48:31.603555+0000","last_undegraded":"2024-05-30T16:48:31.603555+0000","last_fullsized":"2024-05-30T16:48:31.603555+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:15.486340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601462+0000","last_change":"2024-05-30T16:48:30.589187+0000","last_active":"2024-05-30T16:48:31.601462+0000","last_peered":"2024-05-30T16:48:31.601462+0000","last_clean":"2024-05-30T16:48:31.601462+0000","last_became_active":"2024-05-30T16:48:30.588913+0000","last_became_peered":"2024-05-30T16:48:30.588913+0000","last_unstale":"2024-05-30T16:48:31.601462+0000","last_undegraded":"2024-05-30T16:48:31.601462+0000","last_fullsized":"2024-05-30T16:48:31.601462+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:37:50.160833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567419+0000","last_change":"2024-05-30T16:48:30.572439+0000","last_active":"2024-05-30T16:48:31.567419+0000","last_peered":"2024-05-30T16:48:31.567419+0000","last_clean":"2024-05-30T16:48:31.567419+0000","last_became_active":"2024-05-30T16:48:30.571643+0000","last_became_peered":"2024-05-30T16:48:30.571643+0000","last_unstale":"2024-05-30T16:48:31.567419+0000","last_undegraded":"2024-05-30T16:48:31.567419+0000","last_fullsized":"2024-05-30T16:48:31.567419+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:11.352814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601819+0000","last_change":"2024-05-30T16:48:30.563868+0000","last_active":"2024-05-30T16:48:31.601819+0000","last_peered":"2024-05-30T16:48:31.601819+0000","last_clean":"2024-05-30T16:48:31.601819+0000","last_became_active":"2024-05-30T16:48:30.563554+0000","last_became_peered":"2024-05-30T16:48:30.563554+0000","last_unstale":"2024-05-30T16:48:31.601819+0000","last_undegraded":"2024-05-30T16:48:31.601819+0000","last_fullsized":"2024-05-30T16:48:31.601819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:47.906916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602616+0000","last_change":"2024-05-30T16:48:30.574382+0000","last_active":"2024-05-30T16:48:31.602616+0000","last_peered":"2024-05-30T16:48:31.602616+0000","last_clean":"2024-05-30T16:48:31.602616+0000","last_became_active":"2024-05-30T16:48:30.574033+0000","last_became_peered":"2024-05-30T16:48:30.574033+0000","last_unstale":"2024-05-30T16:48:31.602616+0000","last_undegraded":"2024-05-30T16:48:31.602616+0000","last_fullsized":"2024-05-30T16:48:31.602616+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:45.188875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602234+0000","last_change":"2024-05-30T16:48:30.571696+0000","last_active":"2024-05-30T16:48:31.602234+0000","last_peered":"2024-05-30T16:48:31.602234+0000","last_clean":"2024-05-30T16:48:31.602234+0000","last_became_active":"2024-05-30T16:48:30.570676+0000","last_became_peered":"2024-05-30T16:48:30.570676+0000","last_unstale":"2024-05-30T16:48:31.602234+0000","last_undegraded":"2024-05-30T16:48:31.602234+0000","last_fullsized":"2024-05-30T16:48:31.602234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:34.971904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602930+0000","last_change":"2024-05-30T16:48:30.565038+0000","last_active":"2024-05-30T16:48:31.602930+0000","last_peered":"2024-05-30T16:48:31.602930+0000","last_clean":"2024-05-30T16:48:31.602930+0000","last_became_active":"2024-05-30T16:48:30.563510+0000","last_became_peered":"2024-05-30T16:48:30.563510+0000","last_unstale":"2024-05-30T16:48:31.602930+0000","last_undegraded":"2024-05-30T16:48:31.602930+0000","last_fullsized":"2024-05-30T16:48:31.602930+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:01.173277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602171+0000","last_change":"2024-05-30T16:48:30.585362+0000","last_active":"2024-05-30T16:48:31.602171+0000","last_peered":"2024-05-30T16:48:31.602171+0000","last_clean":"2024-05-30T16:48:31.602171+0000","last_became_active":"2024-05-30T16:48:30.584610+0000","last_became_peered":"2024-05-30T16:48:30.584610+0000","last_unstale":"2024-05-30T16:48:31.602171+0000","last_undegraded":"2024-05-30T16:48:31.602171+0000","last_fullsized":"2024-05-30T16:48:31.602171+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:39.464041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601623+0000","last_change":"2024-05-30T16:48:30.575524+0000","last_active":"2024-05-30T16:48:31.601623+0000","last_peered":"2024-05-30T16:48:31.601623+0000","last_clean":"2024-05-30T16:48:31.601623+0000","last_became_active":"2024-05-30T16:48:30.573685+0000","last_became_peered":"2024-05-30T16:48:30.573685+0000","last_unstale":"2024-05-30T16:48:31.601623+0000","last_undegraded":"2024-05-30T16:48:31.601623+0000","last_fullsized":"2024-05-30T16:48:31.601623+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:43.435766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571075+0000","last_change":"2024-05-30T16:48:30.582495+0000","last_active":"2024-05-30T16:48:31.571075+0000","last_peered":"2024-05-30T16:48:31.571075+0000","last_clean":"2024-05-30T16:48:31.571075+0000","last_became_active":"2024-05-30T16:48:30.582199+0000","last_became_peered":"2024-05-30T16:48:30.582199+0000","last_unstale":"2024-05-30T16:48:31.571075+0000","last_undegraded":"2024-05-30T16:48:31.571075+0000","last_fullsized":"2024-05-30T16:48:31.571075+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:18.447770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603245+0000","last_change":"2024-05-30T16:48:30.557574+0000","last_active":"2024-05-30T16:48:31.603245+0000","last_peered":"2024-05-30T16:48:31.603245+0000","last_clean":"2024-05-30T16:48:31.603245+0000","last_became_active":"2024-05-30T16:48:30.557169+0000","last_became_peered":"2024-05-30T16:48:30.557169+0000","last_unstale":"2024-05-30T16:48:31.603245+0000","last_undegraded":"2024-05-30T16:48:31.603245+0000","last_fullsized":"2024-05-30T16:48:31.603245+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:19.255146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600833+0000","last_change":"2024-05-30T16:48:30.572687+0000","last_active":"2024-05-30T16:48:31.600833+0000","last_peered":"2024-05-30T16:48:31.600833+0000","last_clean":"2024-05-30T16:48:31.600833+0000","last_became_active":"2024-05-30T16:48:30.571850+0000","last_became_peered":"2024-05-30T16:48:30.571850+0000","last_unstale":"2024-05-30T16:48:31.600833+0000","last_undegraded":"2024-05-30T16:48:31.600833+0000","last_fullsized":"2024-05-30T16:48:31.600833+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:54:22.198184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567341+0000","last_change":"2024-05-30T16:48:30.574622+0000","last_active":"2024-05-30T16:48:31.567341+0000","last_peered":"2024-05-30T16:48:31.567341+0000","last_clean":"2024-05-30T16:48:31.567341+0000","last_became_active":"2024-05-30T16:48:30.573902+0000","last_became_peered":"2024-05-30T16:48:30.573902+0000","last_unstale":"2024-05-30T16:48:31.567341+0000","last_undegraded":"2024-05-30T16:48:31.567341+0000","last_fullsized":"2024-05-30T16:48:31.567341+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:35.550338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601139+0000","last_change":"2024-05-30T16:48:30.570675+0000","last_active":"2024-05-30T16:48:31.601139+0000","last_peered":"2024-05-30T16:48:31.601139+0000","last_clean":"2024-05-30T16:48:31.601139+0000","last_became_active":"2024-05-30T16:48:30.569596+0000","last_became_peered":"2024-05-30T16:48:30.569596+0000","last_unstale":"2024-05-30T16:48:31.601139+0000","last_undegraded":"2024-05-30T16:48:31.601139+0000","last_fullsized":"2024-05-30T16:48:31.601139+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:24.977866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604125+0000","last_change":"2024-05-30T16:48:30.563191+0000","last_active":"2024-05-30T16:48:31.604125+0000","last_peered":"2024-05-30T16:48:31.604125+0000","last_clean":"2024-05-30T16:48:31.604125+0000","last_became_active":"2024-05-30T16:48:30.562912+0000","last_became_peered":"2024-05-30T16:48:30.562912+0000","last_unstale":"2024-05-30T16:48:31.604125+0000","last_undegraded":"2024-05-30T16:48:31.604125+0000","last_fullsized":"2024-05-30T16:48:31.604125+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:19.255146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568728+0000","last_change":"2024-05-30T16:48:29.572429+0000","last_active":"2024-05-30T16:48:31.568728+0000","last_peered":"2024-05-30T16:48:31.568728+0000","last_clean":"2024-05-30T16:48:31.568728+0000","last_became_active":"2024-05-30T16:48:29.572105+0000","last_became_peered":"2024-05-30T16:48:29.572105+0000","last_unstale":"2024-05-30T16:48:31.568728+0000","last_undegraded":"2024-05-30T16:48:31.568728+0000","last_fullsized":"2024-05-30T16:48:31.568728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:07:55.818348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603707+0000","last_change":"2024-05-30T16:48:30.567278+0000","last_active":"2024-05-30T16:48:31.603707+0000","last_peered":"2024-05-30T16:48:31.603707+0000","last_clean":"2024-05-30T16:48:31.603707+0000","last_became_active":"2024-05-30T16:48:30.566562+0000","last_became_peered":"2024-05-30T16:48:30.566562+0000","last_unstale":"2024-05-30T16:48:31.603707+0000","last_undegraded":"2024-05-30T16:48:31.603707+0000","last_fullsized":"2024-05-30T16:48:31.603707+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:29:25.409504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601689+0000","last_change":"2024-05-30T16:48:29.572447+0000","last_active":"2024-05-30T16:48:31.601689+0000","last_peered":"2024-05-30T16:48:31.601689+0000","last_clean":"2024-05-30T16:48:31.601689+0000","last_became_active":"2024-05-30T16:48:29.572102+0000","last_became_peered":"2024-05-30T16:48:29.572102+0000","last_unstale":"2024-05-30T16:48:31.601689+0000","last_undegraded":"2024-05-30T16:48:31.601689+0000","last_fullsized":"2024-05-30T16:48:31.601689+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:31:38.449206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.571330+0000","last_change":"2024-05-30T16:48:30.570556+0000","last_active":"2024-05-30T16:48:31.571330+0000","last_peered":"2024-05-30T16:48:31.571330+0000","last_clean":"2024-05-30T16:48:31.571330+0000","last_became_active":"2024-05-30T16:48:30.569419+0000","last_became_peered":"2024-05-30T16:48:30.569419+0000","last_unstale":"2024-05-30T16:48:31.571330+0000","last_undegraded":"2024-05-30T16:48:31.571330+0000","last_fullsized":"2024-05-30T16:48:31.571330+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:10:33.900129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604848+0000","last_change":"2024-05-30T16:48:29.563603+0000","last_active":"2024-05-30T16:48:31.604848+0000","last_peered":"2024-05-30T16:48:31.604848+0000","last_clean":"2024-05-30T16:48:31.604848+0000","last_became_active":"2024-05-30T16:48:29.563260+0000","last_became_peered":"2024-05-30T16:48:29.563260+0000","last_unstale":"2024-05-30T16:48:31.604848+0000","last_undegraded":"2024-05-30T16:48:31.604848+0000","last_fullsized":"2024-05-30T16:48:31.604848+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:33.957069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604644+0000","last_change":"2024-05-30T16:48:30.563803+0000","last_active":"2024-05-30T16:48:31.604644+0000","last_peered":"2024-05-30T16:48:31.604644+0000","last_clean":"2024-05-30T16:48:31.604644+0000","last_became_active":"2024-05-30T16:48:30.563522+0000","last_became_peered":"2024-05-30T16:48:30.563522+0000","last_unstale":"2024-05-30T16:48:31.604644+0000","last_undegraded":"2024-05-30T16:48:31.604644+0000","last_fullsized":"2024-05-30T16:48:31.604644+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:15.486340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569290+0000","last_change":"2024-05-30T16:48:29.568315+0000","last_active":"2024-05-30T16:48:31.569290+0000","last_peered":"2024-05-30T16:48:31.569290+0000","last_clean":"2024-05-30T16:48:31.569290+0000","last_became_active":"2024-05-30T16:48:29.567996+0000","last_became_peered":"2024-05-30T16:48:29.567996+0000","last_unstale":"2024-05-30T16:48:31.569290+0000","last_undegraded":"2024-05-30T16:48:31.569290+0000","last_fullsized":"2024-05-30T16:48:31.569290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:23.701866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600792+0000","last_change":"2024-05-30T16:48:30.570026+0000","last_active":"2024-05-30T16:48:31.600792+0000","last_peered":"2024-05-30T16:48:31.600792+0000","last_clean":"2024-05-30T16:48:31.600792+0000","last_became_active":"2024-05-30T16:48:30.567668+0000","last_became_peered":"2024-05-30T16:48:30.567668+0000","last_unstale":"2024-05-30T16:48:31.600792+0000","last_undegraded":"2024-05-30T16:48:31.600792+0000","last_fullsized":"2024-05-30T16:48:31.600792+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:45.715351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567673+0000","last_change":"2024-05-30T16:48:29.561932+0000","last_active":"2024-05-30T16:48:31.567673+0000","last_peered":"2024-05-30T16:48:31.567673+0000","last_clean":"2024-05-30T16:48:31.567673+0000","last_became_active":"2024-05-30T16:48:29.561609+0000","last_became_peered":"2024-05-30T16:48:29.561609+0000","last_unstale":"2024-05-30T16:48:31.567673+0000","last_undegraded":"2024-05-30T16:48:31.567673+0000","last_fullsized":"2024-05-30T16:48:31.567673+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:55:03.134214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602281+0000","last_change":"2024-05-30T16:48:30.570083+0000","last_active":"2024-05-30T16:48:31.602281+0000","last_peered":"2024-05-30T16:48:31.602281+0000","last_clean":"2024-05-30T16:48:31.602281+0000","last_became_active":"2024-05-30T16:48:30.568578+0000","last_became_peered":"2024-05-30T16:48:30.568578+0000","last_unstale":"2024-05-30T16:48:31.602281+0000","last_undegraded":"2024-05-30T16:48:31.602281+0000","last_fullsized":"2024-05-30T16:48:31.602281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:51:20.993001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601117+0000","last_change":"2024-05-30T16:48:29.568131+0000","last_active":"2024-05-30T16:48:31.601117+0000","last_peered":"2024-05-30T16:48:31.601117+0000","last_clean":"2024-05-30T16:48:31.601117+0000","last_became_active":"2024-05-30T16:48:29.567819+0000","last_became_peered":"2024-05-30T16:48:29.567819+0000","last_unstale":"2024-05-30T16:48:31.601117+0000","last_undegraded":"2024-05-30T16:48:31.601117+0000","last_fullsized":"2024-05-30T16:48:31.601117+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:52:26.866201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600660+0000","last_change":"2024-05-30T16:48:30.561051+0000","last_active":"2024-05-30T16:48:31.600660+0000","last_peered":"2024-05-30T16:48:31.600660+0000","last_clean":"2024-05-30T16:48:31.600660+0000","last_became_active":"2024-05-30T16:48:30.560423+0000","last_became_peered":"2024-05-30T16:48:30.560423+0000","last_unstale":"2024-05-30T16:48:31.600660+0000","last_undegraded":"2024-05-30T16:48:31.600660+0000","last_fullsized":"2024-05-30T16:48:31.600660+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:02.774077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569744+0000","last_change":"2024-05-30T16:48:29.566803+0000","last_active":"2024-05-30T16:48:31.569744+0000","last_peered":"2024-05-30T16:48:31.569744+0000","last_clean":"2024-05-30T16:48:31.569744+0000","last_became_active":"2024-05-30T16:48:29.563938+0000","last_became_peered":"2024-05-30T16:48:29.563938+0000","last_unstale":"2024-05-30T16:48:31.569744+0000","last_undegraded":"2024-05-30T16:48:31.569744+0000","last_fullsized":"2024-05-30T16:48:31.569744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:49:46.892080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602039+0000","last_change":"2024-05-30T16:48:30.573009+0000","last_active":"2024-05-30T16:48:31.602039+0000","last_peered":"2024-05-30T16:48:31.602039+0000","last_clean":"2024-05-30T16:48:31.602039+0000","last_became_active":"2024-05-30T16:48:30.571970+0000","last_became_peered":"2024-05-30T16:48:30.571970+0000","last_unstale":"2024-05-30T16:48:31.602039+0000","last_undegraded":"2024-05-30T16:48:31.602039+0000","last_fullsized":"2024-05-30T16:48:31.602039+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:18.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600710+0000","last_change":"2024-05-30T16:48:29.557714+0000","last_active":"2024-05-30T16:48:31.600710+0000","last_peered":"2024-05-30T16:48:31.600710+0000","last_clean":"2024-05-30T16:48:31.600710+0000","last_became_active":"2024-05-30T16:48:29.557329+0000","last_became_peered":"2024-05-30T16:48:29.557329+0000","last_unstale":"2024-05-30T16:48:31.600710+0000","last_undegraded":"2024-05-30T16:48:31.600710+0000","last_fullsized":"2024-05-30T16:48:31.600710+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:55:42.557260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601207+0000","last_change":"2024-05-30T16:48:30.589751+0000","last_active":"2024-05-30T16:48:31.601207+0000","last_peered":"2024-05-30T16:48:31.601207+0000","last_clean":"2024-05-30T16:48:31.601207+0000","last_became_active":"2024-05-30T16:48:30.589452+0000","last_became_peered":"2024-05-30T16:48:30.589452+0000","last_unstale":"2024-05-30T16:48:31.601207+0000","last_undegraded":"2024-05-30T16:48:31.601207+0000","last_fullsized":"2024-05-30T16:48:31.601207+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:47:30.133018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601963+0000","last_change":"2024-05-30T16:48:29.562616+0000","last_active":"2024-05-30T16:48:31.601963+0000","last_peered":"2024-05-30T16:48:31.601963+0000","last_clean":"2024-05-30T16:48:31.601963+0000","last_became_active":"2024-05-30T16:48:29.562279+0000","last_became_peered":"2024-05-30T16:48:29.562279+0000","last_unstale":"2024-05-30T16:48:31.601963+0000","last_undegraded":"2024-05-30T16:48:31.601963+0000","last_fullsized":"2024-05-30T16:48:31.601963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:44.174040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570265+0000","last_change":"2024-05-30T16:48:30.577259+0000","last_active":"2024-05-30T16:48:31.570265+0000","last_peered":"2024-05-30T16:48:31.570265+0000","last_clean":"2024-05-30T16:48:31.570265+0000","last_became_active":"2024-05-30T16:48:30.576185+0000","last_became_peered":"2024-05-30T16:48:30.576185+0000","last_unstale":"2024-05-30T16:48:31.570265+0000","last_undegraded":"2024-05-30T16:48:31.570265+0000","last_fullsized":"2024-05-30T16:48:31.570265+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:25.346604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603628+0000","last_change":"2024-05-30T16:48:29.565569+0000","last_active":"2024-05-30T16:48:31.603628+0000","last_peered":"2024-05-30T16:48:31.603628+0000","last_clean":"2024-05-30T16:48:31.603628+0000","last_became_active":"2024-05-30T16:48:29.565276+0000","last_became_peered":"2024-05-30T16:48:29.565276+0000","last_unstale":"2024-05-30T16:48:31.603628+0000","last_undegraded":"2024-05-30T16:48:31.603628+0000","last_fullsized":"2024-05-30T16:48:31.603628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:02.208362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568433+0000","last_change":"2024-05-30T16:48:30.581656+0000","last_active":"2024-05-30T16:48:31.568433+0000","last_peered":"2024-05-30T16:48:31.568433+0000","last_clean":"2024-05-30T16:48:31.568433+0000","last_became_active":"2024-05-30T16:48:30.580423+0000","last_became_peered":"2024-05-30T16:48:30.580423+0000","last_unstale":"2024-05-30T16:48:31.568433+0000","last_undegraded":"2024-05-30T16:48:31.568433+0000","last_fullsized":"2024-05-30T16:48:31.568433+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:06:55.647538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570858+0000","last_change":"2024-05-30T16:48:29.573492+0000","last_active":"2024-05-30T16:48:31.570858+0000","last_peered":"2024-05-30T16:48:31.570858+0000","last_clean":"2024-05-30T16:48:31.570858+0000","last_became_active":"2024-05-30T16:48:29.573157+0000","last_became_peered":"2024-05-30T16:48:29.573157+0000","last_unstale":"2024-05-30T16:48:31.570858+0000","last_undegraded":"2024-05-30T16:48:31.570858+0000","last_fullsized":"2024-05-30T16:48:31.570858+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:14.481047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570294+0000","last_change":"2024-05-30T16:48:30.580712+0000","last_active":"2024-05-30T16:48:31.570294+0000","last_peered":"2024-05-30T16:48:31.570294+0000","last_clean":"2024-05-30T16:48:31.570294+0000","last_became_active":"2024-05-30T16:48:30.580126+0000","last_became_peered":"2024-05-30T16:48:30.580126+0000","last_unstale":"2024-05-30T16:48:31.570294+0000","last_undegraded":"2024-05-30T16:48:31.570294+0000","last_fullsized":"2024-05-30T16:48:31.570294+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T16:58:49.335864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":20,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601327+0000","last_change":"2024-05-30T16:48:29.564330+0000","last_active":"2024-05-30T16:48:31.601327+0000","last_peered":"2024-05-30T16:48:31.601327+0000","last_clean":"2024-05-30T16:48:31.601327+0000","last_became_active":"2024-05-30T16:48:29.564050+0000","last_became_peered":"2024-05-30T16:48:29.564050+0000","last_unstale":"2024-05-30T16:48:31.601327+0000","last_undegraded":"2024-05-30T16:48:31.601327+0000","last_fullsized":"2024-05-30T16:48:31.601327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:38.418273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602724+0000","last_change":"2024-05-30T16:48:30.587402+0000","last_active":"2024-05-30T16:48:31.602724+0000","last_peered":"2024-05-30T16:48:31.602724+0000","last_clean":"2024-05-30T16:48:31.602724+0000","last_became_active":"2024-05-30T16:48:30.586819+0000","last_became_peered":"2024-05-30T16:48:30.586819+0000","last_unstale":"2024-05-30T16:48:31.602724+0000","last_undegraded":"2024-05-30T16:48:31.602724+0000","last_fullsized":"2024-05-30T16:48:31.602724+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:10:47.394167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567770+0000","last_change":"2024-05-30T16:48:29.556834+0000","last_active":"2024-05-30T16:48:31.567770+0000","last_peered":"2024-05-30T16:48:31.567770+0000","last_clean":"2024-05-30T16:48:31.567770+0000","last_became_active":"2024-05-30T16:48:29.556546+0000","last_became_peered":"2024-05-30T16:48:29.556546+0000","last_unstale":"2024-05-30T16:48:31.567770+0000","last_undegraded":"2024-05-30T16:48:31.567770+0000","last_fullsized":"2024-05-30T16:48:31.567770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:07:33.077371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602679+0000","last_change":"2024-05-30T16:48:30.574973+0000","last_active":"2024-05-30T16:48:31.602679+0000","last_peered":"2024-05-30T16:48:31.602679+0000","last_clean":"2024-05-30T16:48:31.602679+0000","last_became_active":"2024-05-30T16:48:30.574679+0000","last_became_peered":"2024-05-30T16:48:30.574679+0000","last_unstale":"2024-05-30T16:48:31.602679+0000","last_undegraded":"2024-05-30T16:48:31.602679+0000","last_fullsized":"2024-05-30T16:48:31.602679+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:02:00.674835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569040+0000","last_change":"2024-05-30T16:48:29.568023+0000","last_active":"2024-05-30T16:48:31.569040+0000","last_peered":"2024-05-30T16:48:31.569040+0000","last_clean":"2024-05-30T16:48:31.569040+0000","last_became_active":"2024-05-30T16:48:29.567174+0000","last_became_peered":"2024-05-30T16:48:29.567174+0000","last_unstale":"2024-05-30T16:48:31.569040+0000","last_undegraded":"2024-05-30T16:48:31.569040+0000","last_fullsized":"2024-05-30T16:48:31.569040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:59.796399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567307+0000","last_change":"2024-05-30T16:48:30.569413+0000","last_active":"2024-05-30T16:48:31.567307+0000","last_peered":"2024-05-30T16:48:31.567307+0000","last_clean":"2024-05-30T16:48:31.567307+0000","last_became_active":"2024-05-30T16:48:30.569144+0000","last_became_peered":"2024-05-30T16:48:30.569144+0000","last_unstale":"2024-05-30T16:48:31.567307+0000","last_undegraded":"2024-05-30T16:48:31.567307+0000","last_fullsized":"2024-05-30T16:48:31.567307+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:19:21.611466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567474+0000","last_change":"2024-05-30T16:48:29.557641+0000","last_active":"2024-05-30T16:48:31.567474+0000","last_peered":"2024-05-30T16:48:31.567474+0000","last_clean":"2024-05-30T16:48:31.567474+0000","last_became_active":"2024-05-30T16:48:29.556424+0000","last_became_peered":"2024-05-30T16:48:29.556424+0000","last_unstale":"2024-05-30T16:48:31.567474+0000","last_undegraded":"2024-05-30T16:48:31.567474+0000","last_fullsized":"2024-05-30T16:48:31.567474+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:42.420930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569023+0000","last_change":"2024-05-30T16:48:30.575675+0000","last_active":"2024-05-30T16:48:31.569023+0000","last_peered":"2024-05-30T16:48:31.569023+0000","last_clean":"2024-05-30T16:48:31.569023+0000","last_became_active":"2024-05-30T16:48:30.575404+0000","last_became_peered":"2024-05-30T16:48:30.575404+0000","last_unstale":"2024-05-30T16:48:31.569023+0000","last_undegraded":"2024-05-30T16:48:31.569023+0000","last_fullsized":"2024-05-30T16:48:31.569023+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:39:54.346548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":22,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600828+0000","last_change":"2024-05-30T16:48:29.568303+0000","last_active":"2024-05-30T16:48:31.600828+0000","last_peered":"2024-05-30T16:48:31.600828+0000","last_clean":"2024-05-30T16:48:31.600828+0000","last_became_active":"2024-05-30T16:48:29.567991+0000","last_became_peered":"2024-05-30T16:48:29.567991+0000","last_unstale":"2024-05-30T16:48:31.600828+0000","last_undegraded":"2024-05-30T16:48:31.600828+0000","last_fullsized":"2024-05-30T16:48:31.600828+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:11:31.450890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602073+0000","last_change":"2024-05-30T16:48:30.573320+0000","last_active":"2024-05-30T16:48:31.602073+0000","last_peered":"2024-05-30T16:48:31.602073+0000","last_clean":"2024-05-30T16:48:31.602073+0000","last_became_active":"2024-05-30T16:48:30.572981+0000","last_became_peered":"2024-05-30T16:48:30.572981+0000","last_unstale":"2024-05-30T16:48:31.602073+0000","last_undegraded":"2024-05-30T16:48:31.602073+0000","last_fullsized":"2024-05-30T16:48:31.602073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:18.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601181+0000","last_change":"2024-05-30T16:48:29.547415+0000","last_active":"2024-05-30T16:48:31.601181+0000","last_peered":"2024-05-30T16:48:31.601181+0000","last_clean":"2024-05-30T16:48:31.601181+0000","last_became_active":"2024-05-30T16:48:29.547133+0000","last_became_peered":"2024-05-30T16:48:29.547133+0000","last_unstale":"2024-05-30T16:48:31.601181+0000","last_undegraded":"2024-05-30T16:48:31.601181+0000","last_fullsized":"2024-05-30T16:48:31.601181+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:18:18.079276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603817+0000","last_change":"2024-05-30T16:48:30.588261+0000","last_active":"2024-05-30T16:48:31.603817+0000","last_peered":"2024-05-30T16:48:31.603817+0000","last_clean":"2024-05-30T16:48:31.603817+0000","last_became_active":"2024-05-30T16:48:30.587964+0000","last_became_peered":"2024-05-30T16:48:30.587964+0000","last_unstale":"2024-05-30T16:48:31.603817+0000","last_undegraded":"2024-05-30T16:48:31.603817+0000","last_fullsized":"2024-05-30T16:48:31.603817+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:00.202227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602712+0000","last_change":"2024-05-30T16:48:29.569871+0000","last_active":"2024-05-30T16:48:31.602712+0000","last_peered":"2024-05-30T16:48:31.602712+0000","last_clean":"2024-05-30T16:48:31.602712+0000","last_became_active":"2024-05-30T16:48:29.569572+0000","last_became_peered":"2024-05-30T16:48:29.569572+0000","last_unstale":"2024-05-30T16:48:31.602712+0000","last_undegraded":"2024-05-30T16:48:31.602712+0000","last_fullsized":"2024-05-30T16:48:31.602712+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:06:33.957069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602844+0000","last_change":"2024-05-30T16:48:30.580358+0000","last_active":"2024-05-30T16:48:31.602844+0000","last_peered":"2024-05-30T16:48:31.602844+0000","last_clean":"2024-05-30T16:48:31.602844+0000","last_became_active":"2024-05-30T16:48:30.580087+0000","last_became_peered":"2024-05-30T16:48:30.580087+0000","last_unstale":"2024-05-30T16:48:31.602844+0000","last_undegraded":"2024-05-30T16:48:31.602844+0000","last_fullsized":"2024-05-30T16:48:31.602844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:11.840908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570987+0000","last_change":"2024-05-30T16:48:29.556896+0000","last_active":"2024-05-30T16:48:31.570987+0000","last_peered":"2024-05-30T16:48:31.570987+0000","last_clean":"2024-05-30T16:48:31.570987+0000","last_became_active":"2024-05-30T16:48:29.556569+0000","last_became_peered":"2024-05-30T16:48:29.556569+0000","last_unstale":"2024-05-30T16:48:31.570987+0000","last_undegraded":"2024-05-30T16:48:31.570987+0000","last_fullsized":"2024-05-30T16:48:31.570987+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:27.482686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.602981+0000","last_change":"2024-05-30T16:48:30.569176+0000","last_active":"2024-05-30T16:48:31.602981+0000","last_peered":"2024-05-30T16:48:31.602981+0000","last_clean":"2024-05-30T16:48:31.602981+0000","last_became_active":"2024-05-30T16:48:30.568847+0000","last_became_peered":"2024-05-30T16:48:30.568847+0000","last_unstale":"2024-05-30T16:48:31.602981+0000","last_undegraded":"2024-05-30T16:48:31.602981+0000","last_fullsized":"2024-05-30T16:48:31.602981+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:59:39.409875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568330+0000","last_change":"2024-05-30T16:48:29.565727+0000","last_active":"2024-05-30T16:48:31.568330+0000","last_peered":"2024-05-30T16:48:31.568330+0000","last_clean":"2024-05-30T16:48:31.568330+0000","last_became_active":"2024-05-30T16:48:29.565445+0000","last_became_peered":"2024-05-30T16:48:29.565445+0000","last_unstale":"2024-05-30T16:48:31.568330+0000","last_undegraded":"2024-05-30T16:48:31.568330+0000","last_fullsized":"2024-05-30T16:48:31.568330+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:29:03.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567327+0000","last_change":"2024-05-30T16:48:30.573197+0000","last_active":"2024-05-30T16:48:31.567327+0000","last_peered":"2024-05-30T16:48:31.567327+0000","last_clean":"2024-05-30T16:48:31.567327+0000","last_became_active":"2024-05-30T16:48:30.572912+0000","last_became_peered":"2024-05-30T16:48:30.572912+0000","last_unstale":"2024-05-30T16:48:31.567327+0000","last_undegraded":"2024-05-30T16:48:31.567327+0000","last_fullsized":"2024-05-30T16:48:31.567327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:02:23.023604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568144+0000","last_change":"2024-05-30T16:48:29.562648+0000","last_active":"2024-05-30T16:48:31.568144+0000","last_peered":"2024-05-30T16:48:31.568144+0000","last_clean":"2024-05-30T16:48:31.568144+0000","last_became_active":"2024-05-30T16:48:29.562380+0000","last_became_peered":"2024-05-30T16:48:29.562380+0000","last_unstale":"2024-05-30T16:48:31.568144+0000","last_undegraded":"2024-05-30T16:48:31.568144+0000","last_fullsized":"2024-05-30T16:48:31.568144+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:06.069947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604190+0000","last_change":"2024-05-30T16:48:30.552462+0000","last_active":"2024-05-30T16:48:31.604190+0000","last_peered":"2024-05-30T16:48:31.604190+0000","last_clean":"2024-05-30T16:48:31.604190+0000","last_became_active":"2024-05-30T16:48:30.552175+0000","last_became_peered":"2024-05-30T16:48:30.552175+0000","last_unstale":"2024-05-30T16:48:31.604190+0000","last_undegraded":"2024-05-30T16:48:31.604190+0000","last_fullsized":"2024-05-30T16:48:31.604190+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:37:16.441185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569935+0000","last_change":"2024-05-30T16:48:29.564622+0000","last_active":"2024-05-30T16:48:31.569935+0000","last_peered":"2024-05-30T16:48:31.569935+0000","last_clean":"2024-05-30T16:48:31.569935+0000","last_became_active":"2024-05-30T16:48:29.564294+0000","last_became_peered":"2024-05-30T16:48:29.564294+0000","last_unstale":"2024-05-30T16:48:31.569935+0000","last_undegraded":"2024-05-30T16:48:31.569935+0000","last_fullsized":"2024-05-30T16:48:31.569935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:01.759242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601535+0000","last_change":"2024-05-30T16:48:30.572550+0000","last_active":"2024-05-30T16:48:31.601535+0000","last_peered":"2024-05-30T16:48:31.601535+0000","last_clean":"2024-05-30T16:48:31.601535+0000","last_became_active":"2024-05-30T16:48:30.572162+0000","last_became_peered":"2024-05-30T16:48:30.572162+0000","last_unstale":"2024-05-30T16:48:31.601535+0000","last_undegraded":"2024-05-30T16:48:31.601535+0000","last_fullsized":"2024-05-30T16:48:31.601535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:52:27.881036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601305+0000","last_change":"2024-05-30T16:48:29.565513+0000","last_active":"2024-05-30T16:48:31.601305+0000","last_peered":"2024-05-30T16:48:31.601305+0000","last_clean":"2024-05-30T16:48:31.601305+0000","last_became_active":"2024-05-30T16:48:29.565019+0000","last_became_peered":"2024-05-30T16:48:29.565019+0000","last_unstale":"2024-05-30T16:48:31.601305+0000","last_undegraded":"2024-05-30T16:48:31.601305+0000","last_fullsized":"2024-05-30T16:48:31.601305+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:43.183632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601161+0000","last_change":"2024-05-30T16:48:30.554107+0000","last_active":"2024-05-30T16:48:31.601161+0000","last_peered":"2024-05-30T16:48:31.601161+0000","last_clean":"2024-05-30T16:48:31.601161+0000","last_became_active":"2024-05-30T16:48:30.553499+0000","last_became_peered":"2024-05-30T16:48:30.553499+0000","last_unstale":"2024-05-30T16:48:31.601161+0000","last_undegraded":"2024-05-30T16:48:31.601161+0000","last_fullsized":"2024-05-30T16:48:31.601161+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:28.497521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568109+0000","last_change":"2024-05-30T16:48:29.565658+0000","last_active":"2024-05-30T16:48:31.568109+0000","last_peered":"2024-05-30T16:48:31.568109+0000","last_clean":"2024-05-30T16:48:31.568109+0000","last_became_active":"2024-05-30T16:48:29.565325+0000","last_became_peered":"2024-05-30T16:48:29.565325+0000","last_unstale":"2024-05-30T16:48:31.568109+0000","last_undegraded":"2024-05-30T16:48:31.568109+0000","last_fullsized":"2024-05-30T16:48:31.568109+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:00:28.354021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603385+0000","last_change":"2024-05-30T16:48:30.589674+0000","last_active":"2024-05-30T16:48:31.603385+0000","last_peered":"2024-05-30T16:48:31.603385+0000","last_clean":"2024-05-30T16:48:31.603385+0000","last_became_active":"2024-05-30T16:48:30.589308+0000","last_became_peered":"2024-05-30T16:48:30.589308+0000","last_unstale":"2024-05-30T16:48:31.603385+0000","last_undegraded":"2024-05-30T16:48:31.603385+0000","last_fullsized":"2024-05-30T16:48:31.603385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:37:16.441185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568417+0000","last_change":"2024-05-30T16:48:29.563607+0000","last_active":"2024-05-30T16:48:31.568417+0000","last_peered":"2024-05-30T16:48:31.568417+0000","last_clean":"2024-05-30T16:48:31.568417+0000","last_became_active":"2024-05-30T16:48:29.563281+0000","last_became_peered":"2024-05-30T16:48:29.563281+0000","last_unstale":"2024-05-30T16:48:31.568417+0000","last_undegraded":"2024-05-30T16:48:31.568417+0000","last_fullsized":"2024-05-30T16:48:31.568417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:29.072276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603410+0000","last_change":"2024-05-30T16:48:30.577542+0000","last_active":"2024-05-30T16:48:31.603410+0000","last_peered":"2024-05-30T16:48:31.603410+0000","last_clean":"2024-05-30T16:48:31.603410+0000","last_became_active":"2024-05-30T16:48:30.576718+0000","last_became_peered":"2024-05-30T16:48:30.576718+0000","last_unstale":"2024-05-30T16:48:31.603410+0000","last_undegraded":"2024-05-30T16:48:31.603410+0000","last_fullsized":"2024-05-30T16:48:31.603410+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:24.028012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570863+0000","last_change":"2024-05-30T16:48:29.572620+0000","last_active":"2024-05-30T16:48:31.570863+0000","last_peered":"2024-05-30T16:48:31.570863+0000","last_clean":"2024-05-30T16:48:31.570863+0000","last_became_active":"2024-05-30T16:48:29.572329+0000","last_became_peered":"2024-05-30T16:48:29.572329+0000","last_unstale":"2024-05-30T16:48:31.570863+0000","last_undegraded":"2024-05-30T16:48:31.570863+0000","last_fullsized":"2024-05-30T16:48:31.570863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:42:59.187391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570734+0000","last_change":"2024-05-30T16:48:30.578757+0000","last_active":"2024-05-30T16:48:31.570734+0000","last_peered":"2024-05-30T16:48:31.570734+0000","last_clean":"2024-05-30T16:48:31.570734+0000","last_became_active":"2024-05-30T16:48:30.578484+0000","last_became_peered":"2024-05-30T16:48:30.578484+0000","last_unstale":"2024-05-30T16:48:31.570734+0000","last_undegraded":"2024-05-30T16:48:31.570734+0000","last_fullsized":"2024-05-30T16:48:31.570734+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:47.794458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604674+0000","last_change":"2024-05-30T16:48:29.563508+0000","last_active":"2024-05-30T16:48:31.604674+0000","last_peered":"2024-05-30T16:48:31.604674+0000","last_clean":"2024-05-30T16:48:31.604674+0000","last_became_active":"2024-05-30T16:48:29.563206+0000","last_became_peered":"2024-05-30T16:48:29.563206+0000","last_unstale":"2024-05-30T16:48:31.604674+0000","last_undegraded":"2024-05-30T16:48:31.604674+0000","last_fullsized":"2024-05-30T16:48:31.604674+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:10:44.174040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567764+0000","last_change":"2024-05-30T16:48:30.558867+0000","last_active":"2024-05-30T16:48:31.567764+0000","last_peered":"2024-05-30T16:48:31.567764+0000","last_clean":"2024-05-30T16:48:31.567764+0000","last_became_active":"2024-05-30T16:48:30.558563+0000","last_became_peered":"2024-05-30T16:48:30.558563+0000","last_unstale":"2024-05-30T16:48:31.567764+0000","last_undegraded":"2024-05-30T16:48:31.567764+0000","last_fullsized":"2024-05-30T16:48:31.567764+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:07.248500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569370+0000","last_change":"2024-05-30T16:48:29.573875+0000","last_active":"2024-05-30T16:48:31.569370+0000","last_peered":"2024-05-30T16:48:31.569370+0000","last_clean":"2024-05-30T16:48:31.569370+0000","last_became_active":"2024-05-30T16:48:29.572773+0000","last_became_peered":"2024-05-30T16:48:29.572773+0000","last_unstale":"2024-05-30T16:48:31.569370+0000","last_undegraded":"2024-05-30T16:48:31.569370+0000","last_fullsized":"2024-05-30T16:48:31.569370+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:15:23.013176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.567552+0000","last_change":"2024-05-30T16:48:30.573312+0000","last_active":"2024-05-30T16:48:31.567552+0000","last_peered":"2024-05-30T16:48:31.567552+0000","last_clean":"2024-05-30T16:48:31.567552+0000","last_became_active":"2024-05-30T16:48:30.572980+0000","last_became_peered":"2024-05-30T16:48:30.572980+0000","last_unstale":"2024-05-30T16:48:31.567552+0000","last_undegraded":"2024-05-30T16:48:31.567552+0000","last_fullsized":"2024-05-30T16:48:31.567552+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:58:22.477685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.601134+0000","last_change":"2024-05-30T16:48:29.569525+0000","last_active":"2024-05-30T16:48:31.601134+0000","last_peered":"2024-05-30T16:48:31.601134+0000","last_clean":"2024-05-30T16:48:31.601134+0000","last_became_active":"2024-05-30T16:48:29.568636+0000","last_became_peered":"2024-05-30T16:48:29.568636+0000","last_unstale":"2024-05-30T16:48:31.601134+0000","last_undegraded":"2024-05-30T16:48:31.601134+0000","last_fullsized":"2024-05-30T16:48:31.601134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:17.171552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.569918+0000","last_change":"2024-05-30T16:48:30.553665+0000","last_active":"2024-05-30T16:48:31.569918+0000","last_peered":"2024-05-30T16:48:31.569918+0000","last_clean":"2024-05-30T16:48:31.569918+0000","last_became_active":"2024-05-30T16:48:30.552512+0000","last_became_peered":"2024-05-30T16:48:30.552512+0000","last_unstale":"2024-05-30T16:48:31.569918+0000","last_undegraded":"2024-05-30T16:48:31.569918+0000","last_fullsized":"2024-05-30T16:48:31.569918+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:24.716701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.604236+0000","last_change":"2024-05-30T16:48:29.572007+0000","last_active":"2024-05-30T16:48:31.604236+0000","last_peered":"2024-05-30T16:48:31.604236+0000","last_clean":"2024-05-30T16:48:31.604236+0000","last_became_active":"2024-05-30T16:48:29.571652+0000","last_became_peered":"2024-05-30T16:48:29.571652+0000","last_unstale":"2024-05-30T16:48:31.604236+0000","last_undegraded":"2024-05-30T16:48:31.604236+0000","last_fullsized":"2024-05-30T16:48:31.604236+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:10:46.379332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600726+0000","last_change":"2024-05-30T16:48:30.583592+0000","last_active":"2024-05-30T16:48:31.600726+0000","last_peered":"2024-05-30T16:48:31.600726+0000","last_clean":"2024-05-30T16:48:31.600726+0000","last_became_active":"2024-05-30T16:48:30.583350+0000","last_became_peered":"2024-05-30T16:48:30.583350+0000","last_unstale":"2024-05-30T16:48:31.600726+0000","last_undegraded":"2024-05-30T16:48:31.600726+0000","last_fullsized":"2024-05-30T16:48:31.600726+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:02.683671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.600934+0000","last_change":"2024-05-30T16:48:29.554465+0000","last_active":"2024-05-30T16:48:31.600934+0000","last_peered":"2024-05-30T16:48:31.600934+0000","last_clean":"2024-05-30T16:48:31.600934+0000","last_became_active":"2024-05-30T16:48:29.554117+0000","last_became_peered":"2024-05-30T16:48:29.554117+0000","last_unstale":"2024-05-30T16:48:31.600934+0000","last_undegraded":"2024-05-30T16:48:31.600934+0000","last_fullsized":"2024-05-30T16:48:31.600934+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:30.281975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.603615+0000","last_change":"2024-05-30T16:48:30.585362+0000","last_active":"2024-05-30T16:48:31.603615+0000","last_peered":"2024-05-30T16:48:31.603615+0000","last_clean":"2024-05-30T16:48:31.603615+0000","last_became_active":"2024-05-30T16:48:30.583719+0000","last_became_peered":"2024-05-30T16:48:30.583719+0000","last_unstale":"2024-05-30T16:48:31.603615+0000","last_undegraded":"2024-05-30T16:48:31.603615+0000","last_fullsized":"2024-05-30T16:48:31.603615+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:29.513818+0000","last_clean_scrub_stamp":"2024-05-30T16:48:29.513818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:42:15.495883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.570531+0000","last_change":"2024-05-30T16:48:29.558681+0000","last_active":"2024-05-30T16:48:31.570531+0000","last_peered":"2024-05-30T16:48:31.570531+0000","last_clean":"2024-05-30T16:48:31.570531+0000","last_became_active":"2024-05-30T16:48:29.558149+0000","last_became_peered":"2024-05-30T16:48:29.558149+0000","last_unstale":"2024-05-30T16:48:31.570531+0000","last_undegraded":"2024-05-30T16:48:31.570531+0000","last_fullsized":"2024-05-30T16:48:31.570531+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:48:28.498983+0000","last_clean_scrub_stamp":"2024-05-30T16:48:28.498983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:40.373552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":221,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-30T16:48:31.568175+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:48:31.568175+0000","last_peered":"2024-05-30T16:48:31.568175+0000","last_clean":"2024-05-30T16:48:31.568175+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:48:31.568175+0000","last_undegraded":"2024-05-30T16:48:31.568175+0000","last_fullsized":"2024-05-30T16:48:31.568175+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":565248,"data_stored":229099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35896,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335944,"statfs":{"total":96636764160,"available":96600006656,"internally_reserved":0,"allocated":569344,"data_stored":230039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509504,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":569344,"data_stored":229065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509504,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35904,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335936,"statfs":{"total":96636764160,"available":96599998464,"internally_reserved":0,"allocated":573440,"data_stored":230061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":561152,"data_stored":228497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542206,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36468,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335372,"statfs":{"total":96636764160,"available":96599420928,"internally_reserved":0,"allocated":1155072,"data_stored":818885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35896,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335944,"statfs":{"total":96636764160,"available":96600006656,"internally_reserved":0,"allocated":561152,"data_stored":228497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542207,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36472,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335368,"statfs":{"total":96636764160,"available":96599416832,"internally_reserved":0,"allocated":1155072,"data_stored":818829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:48:40.602 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T16:48:40.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-30T16:48:40.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.861+0000 7f6eafe00640 1 Processor -- start 2024-05-30T16:48:40.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.861+0000 7f6eafe00640 1 -- start start 2024-05-30T16:48:40.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.862+0000 7f6eafe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6eb015e870 0x7f6eb00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.862+0000 7f6eafe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00d14d0 0x7f6eb00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.862+0000 7f6eafe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb00cfd00 con 0x7f6eb00cf870 2024-05-30T16:48:40.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.862+0000 7f6eafe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb00d2150 con 0x7f6eb015e870 2024-05-30T16:48:40.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.862+0000 7f6eafe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb015ec50 con 0x7f6eb00d14d0 2024-05-30T16:48:40.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.863+0000 7f6eaea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6eb015e870 0x7f6eb00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.863+0000 7f6eae000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00d14d0 0x7f6eb00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.863+0000 7f6eaea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6eb015e870 0x7f6eb00d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:59990/0 (socket says 172.21.15.67:59990) 2024-05-30T16:48:40.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.863+0000 7f6eaea00640 1 -- 172.21.15.67:0/4268730265 learned_addr learned my addr 172.21.15.67:0/4268730265 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:40.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.864+0000 7f6eae000640 1 -- 172.21.15.67:0/4268730265 >> v2:172.21.15.169:3300/0 conn(0x7f6eb015e870 msgr2=0x7f6eb00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:40.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.864+0000 7f6eae000640 1 --2- 172.21.15.67:0/4268730265 >> v2:172.21.15.169:3300/0 conn(0x7f6eb015e870 0x7f6eb00d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.864+0000 7f6eae000640 1 -- 172.21.15.67:0/4268730265 >> v1:172.21.15.169:6789/0 conn(0x7f6eb00cf870 legacy=0x7f6eb0162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.864+0000 7f6eae000640 1 -- 172.21.15.67:0/4268730265 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb00cff80 con 0x7f6eb00d14d0 2024-05-30T16:48:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.865+0000 7f6eae000640 1 --2- 172.21.15.67:0/4268730265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00d14d0 0x7f6eb00cf330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009790 tx=0x7f6e9c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=874ea7e7dcaa55c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:40.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.866+0000 7f6ead600640 1 -- 172.21.15.67:0/4268730265 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c037070 con 0x7f6eb00d14d0 2024-05-30T16:48:40.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.866+0000 7f6ead600640 1 -- 172.21.15.67:0/4268730265 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c032440 con 0x7f6eb00d14d0 2024-05-30T16:48:40.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.866+0000 7f6ead600640 1 -- 172.21.15.67:0/4268730265 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c032810 con 0x7f6eb00d14d0 2024-05-30T16:48:40.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.866+0000 7f6eafe00640 1 -- 172.21.15.67:0/4268730265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00d14d0 msgr2=0x7f6eb00cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.866+0000 7f6eafe00640 1 --2- 172.21.15.67:0/4268730265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00d14d0 0x7f6eb00cf330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009790 tx=0x7f6e9c02a180 comp rx=0 tx=0).stop 2024-05-30T16:48:40.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.866+0000 7f6eafe00640 1 -- 172.21.15.67:0/4268730265 shutdown_connections 2024-05-30T16:48:40.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.866+0000 7f6eafe00640 1 --2- 172.21.15.67:0/4268730265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00d14d0 0x7f6eb00cf330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.866+0000 7f6eafe00640 1 --2- 172.21.15.67:0/4268730265 >> v2:172.21.15.169:3300/0 conn(0x7f6eb015e870 0x7f6eb00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.866+0000 7f6eafe00640 1 -- 172.21.15.67:0/4268730265 >> 172.21.15.67:0/4268730265 conn(0x7f6eb008bfc0 msgr2=0x7f6eb00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:40.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.867+0000 7f6eafe00640 1 -- 172.21.15.67:0/4268730265 shutdown_connections 2024-05-30T16:48:40.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.867+0000 7f6eafe00640 1 -- 172.21.15.67:0/4268730265 wait complete. 2024-05-30T16:48:40.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.868+0000 7f6eafe00640 1 Processor -- start 2024-05-30T16:48:40.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.868+0000 7f6eafe00640 1 -- start start 2024-05-30T16:48:40.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.869+0000 7f6eafe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb00cf870 0x7f6eb015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.869+0000 7f6eafe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb00d14d0 0x7f6eb00b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.869+0000 7f6eafe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb015e870 0x7f6eb00cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.869+0000 7f6eafe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00d0820 con 0x7f6eb00cf870 2024-05-30T16:48:40.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.869+0000 7f6eafe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00d0540 con 0x7f6eb00d14d0 2024-05-30T16:48:40.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.869+0000 7f6eafe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb00d4fb0 con 0x7f6eb015e870 2024-05-30T16:48:40.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb00cf870 0x7f6eb015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb00cf870 0x7f6eb015ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49316/0 (socket says 172.21.15.67:49316) 2024-05-30T16:48:40.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaea00640 1 -- 172.21.15.67:0/4208589738 learned_addr learned my addr 172.21.15.67:0/4208589738 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:40.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaf400640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb015e870 0x7f6eb00cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eae000640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb00d14d0 0x7f6eb00b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaea00640 1 -- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb015e870 msgr2=0x7f6eb00cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaea00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb015e870 0x7f6eb00cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaea00640 1 -- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb00d14d0 msgr2=0x7f6eb00b95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaea00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb00d14d0 0x7f6eb00b95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaea00640 1 -- 172.21.15.67:0/4208589738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb01f7d40 con 0x7f6eb00cf870 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eaf400640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb015e870 0x7f6eb00cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.870+0000 7f6eae000640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb00d14d0 0x7f6eb00b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.871+0000 7f6eaea00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb00cf870 0x7f6eb015ee60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009fd0 tx=0x7f6ea000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.871+0000 7f6ea6a00640 1 -- 172.21.15.67:0/4208589738 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0017070 con 0x7f6eb00cf870 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.871+0000 7f6ea6a00640 1 -- 172.21.15.67:0/4208589738 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea000c940 con 0x7f6eb00cf870 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.871+0000 7f6ea6a00640 1 -- 172.21.15.67:0/4208589738 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000cd10 con 0x7f6eb00cf870 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.871+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb01f8020 con 0x7f6eb00cf870 2024-05-30T16:48:40.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.871+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb01f85e0 con 0x7f6eb00cf870 2024-05-30T16:48:40.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.872+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb01f43f0 con 0x7f6eb00cf870 2024-05-30T16:48:40.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.873+0000 7f6ea6a00640 1 -- 172.21.15.67:0/4208589738 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f6ea001a070 con 0x7f6eb00cf870 2024-05-30T16:48:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.874+0000 7f6ea6a00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6e9006cbc0 0x7f6e9006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.874+0000 7f6ea6a00640 1 -- 172.21.15.67:0/4208589738 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f6ea008c3f0 con 0x7f6eb00cf870 2024-05-30T16:48:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.875+0000 7f6eae000640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6e9006cbc0 0x7f6e9006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:40.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.875+0000 7f6eae000640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6e9006cbc0 0x7f6e9006f080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c034000 tx=0x7f6e9c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:40.878+0000 7f6ea6a00640 1 -- 172.21.15.67:0/4208589738 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ea005a3b0 con 0x7f6eb00cf870 2024-05-30T16:48:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.160+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6eb00689e0 con 0x7f6eb00cf870 2024-05-30T16:48:41.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.166+0000 7f6ea6a00640 1 -- 172.21.15.67:0/4208589738 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f6ea0059cb0 con 0x7f6eb00cf870 2024-05-30T16:48:41.167 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-05-30T16:48:41.168 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T16:48:41.168 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-05-30T16:48:41.168 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-05-30T16:48:41.168 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-30T16:48:41.168 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-05-30T16:48:41.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6e9006cbc0 msgr2=0x7f6e9006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:41.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6e9006cbc0 0x7f6e9006f080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c034000 tx=0x7f6e9c02e000 comp rx=0 tx=0).stop 2024-05-30T16:48:41.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb00cf870 msgr2=0x7f6eb015ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:41.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb00cf870 0x7f6eb015ee60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009fd0 tx=0x7f6ea000ed40 comp rx=0 tx=0).stop 2024-05-30T16:48:41.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 shutdown_connections 2024-05-30T16:48:41.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6e9006cbc0 0x7f6e9006f080 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb015e870 0x7f6eb00cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb00d14d0 0x7f6eb00b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 --2- 172.21.15.67:0/4208589738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb00cf870 0x7f6eb015ee60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 >> 172.21.15.67:0/4208589738 conn(0x7f6eb008bfc0 msgr2=0x7f6eb00b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:41.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 shutdown_connections 2024-05-30T16:48:41.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.169+0000 7f6eafe00640 1 -- 172.21.15.67:0/4208589738 wait complete. 2024-05-30T16:48:41.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:48:41.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.402+0000 7f535a600640 1 Processor -- start 2024-05-30T16:48:41.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.403+0000 7f535a600640 1 -- start start 2024-05-30T16:48:41.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.404+0000 7f535a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5354072070 0x7f53540b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:41.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.404+0000 7f535a600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f535406e6e0 con 0x7f5354071bc0 2024-05-30T16:48:41.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.404+0000 7f535a600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f535406e9c0 con 0x7f5354158b80 2024-05-30T16:48:41.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.404+0000 7f535a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f535406eca0 con 0x7f5354072070 2024-05-30T16:48:41.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.405+0000 7f5353e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5354071bc0 0x7f53540b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56580/0 (socket says 172.21.15.67:56580) 2024-05-30T16:48:41.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.405+0000 7f5353e00640 1 -- 172.21.15.67:0/1896450878 learned_addr learned my addr 172.21.15.67:0/1896450878 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:41.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.405+0000 7f5359c00640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5354072070 0x7f53540b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:41.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.405+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 147146536 0 0) 0x7f535406e6e0 con 0x7f5354071bc0 2024-05-30T16:48:41.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.405+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f533c003520 con 0x7f5354071bc0 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 683277277 0 0) 0x7f533c003520 con 0x7f5354071bc0 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5354072070 msgr2=0x7f53540b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5354072070 0x7f53540b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 >> v1:172.21.15.169:6789/0 conn(0x7f5354158b80 legacy=0x7f535406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535406ef80 con 0x7f5354071bc0 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5340001670 con 0x7f5354071bc0 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 >> v1:172.21.15.67:6789/0 conn(0x7f5354071bc0 legacy=0x7f53540b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5354072070 0x7f533c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533c003f60 0x7f533c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f533c004d40 0x7f533c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f535406eca0 con 0x7f533c004d40 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f533c00baa0 con 0x7f533c003f60 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f533c00bca0 con 0x7f5354072070 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.406+0000 7f5359c00640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f533c004d40 0x7f533c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.407+0000 7f5359200640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5354072070 0x7f533c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.407+0000 7f5353e00640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533c003f60 0x7f533c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.407+0000 7f5359c00640 1 -- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5354072070 msgr2=0x7f533c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.407+0000 7f5359c00640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5354072070 0x7f533c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.407+0000 7f5359c00640 1 -- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533c003f60 msgr2=0x7f533c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.407+0000 7f5359c00640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533c003f60 0x7f533c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.407+0000 7f5359c00640 1 -- 172.21.15.67:0/1896450878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c003520 con 0x7f533c004d40 2024-05-30T16:48:41.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.408+0000 7f5359c00640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f533c004d40 0x7f533c0051c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f534c009fc0 tx=0x7f533c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af72a36be0c64260 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:41.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.408+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c034020 con 0x7f533c004d40 2024-05-30T16:48:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.408+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c0305f0 con 0x7f533c004d40 2024-05-30T16:48:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.408+0000 7f5353400640 1 -- 172.21.15.67:0/1896450878 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c009c10 con 0x7f533c004d40 2024-05-30T16:48:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.409+0000 7f535a600640 1 -- 172.21.15.67:0/1896450878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f533c004d40 msgr2=0x7f533c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.409+0000 7f535a600640 1 --2- 172.21.15.67:0/1896450878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f533c004d40 0x7f533c0051c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f534c009fc0 tx=0x7f533c005830 comp rx=0 tx=0).stop 2024-05-30T16:48:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.409+0000 7f5359c00640 1 -- 172.21.15.67:0/1896450878 reap_dead start 2024-05-30T16:48:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.409+0000 7f535a600640 1 -- 172.21.15.67:0/1896450878 shutdown_connections 2024-05-30T16:48:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.409+0000 7f535a600640 1 -- 172.21.15.67:0/1896450878 >> 172.21.15.67:0/1896450878 conn(0x7f53540b1fc0 msgr2=0x7f53540b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.409+0000 7f535a600640 1 -- 172.21.15.67:0/1896450878 shutdown_connections 2024-05-30T16:48:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.409+0000 7f535a600640 1 -- 172.21.15.67:0/1896450878 wait complete. 2024-05-30T16:48:41.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.411+0000 7f535a600640 1 Processor -- start 2024-05-30T16:48:41.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.411+0000 7f535a600640 1 -- start start 2024-05-30T16:48:41.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.412+0000 7f535a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533c004d40 0x7f535414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:41.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.412+0000 7f535a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354151b60 0x7f5354151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.412+0000 7f535a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53541604e0 0x7f5354160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.412+0000 7f535a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5354072480 con 0x7f53541604e0 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.412+0000 7f535a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53540b2d30 con 0x7f5354151b60 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.412+0000 7f535a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f535406f780 con 0x7f533c004d40 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.412+0000 7f5359c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53541604e0 0x7f5354160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.412+0000 7f5359c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53541604e0 0x7f5354160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49332/0 (socket says 172.21.15.67:49332) 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.412+0000 7f5359c00640 1 -- 172.21.15.67:0/830185209 learned_addr learned my addr 172.21.15.67:0/830185209 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.413+0000 7f5353e00640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354151b60 0x7f5354151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.413+0000 7f5359200640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533c004d40 0x7f535414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.413+0000 7f5359c00640 1 -- 172.21.15.67:0/830185209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533c004d40 msgr2=0x7f535414b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.413+0000 7f5359c00640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533c004d40 0x7f535414b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.413+0000 7f5359c00640 1 -- 172.21.15.67:0/830185209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354151b60 msgr2=0x7f5354151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:41.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.413+0000 7f5359c00640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354151b60 0x7f5354151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.413+0000 7f5359c00640 1 -- 172.21.15.67:0/830185209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53541f4ba0 con 0x7f53541604e0 2024-05-30T16:48:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.413+0000 7f5359c00640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53541604e0 0x7f5354160960 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f534c00e200 tx=0x7f534c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.413+0000 7f5350c00640 1 -- 172.21.15.67:0/830185209 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c034030 con 0x7f53541604e0 2024-05-30T16:48:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.414+0000 7f5350c00640 1 -- 172.21.15.67:0/830185209 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c003990 con 0x7f53541604e0 2024-05-30T16:48:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.414+0000 7f5350c00640 1 -- 172.21.15.67:0/830185209 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c009c10 con 0x7f53541604e0 2024-05-30T16:48:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.414+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53541f4e80 con 0x7f53541604e0 2024-05-30T16:48:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.414+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535414ae10 con 0x7f53541604e0 2024-05-30T16:48:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.415+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53540689e0 con 0x7f53541604e0 2024-05-30T16:48:41.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.416+0000 7f5350c00640 1 -- 172.21.15.67:0/830185209 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f534c037020 con 0x7f53541604e0 2024-05-30T16:48:41.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.417+0000 7f5350c00640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f533406cbc0 0x7f533406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:41.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.418+0000 7f5350c00640 1 -- 172.21.15.67:0/830185209 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f534c0ac670 con 0x7f53541604e0 2024-05-30T16:48:41.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.418+0000 7f5359200640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f533406cbc0 0x7f533406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:41.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.418+0000 7f5359200640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f533406cbc0 0x7f533406f080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5348007fd0 tx=0x7f5348009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:41.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.423+0000 7f5350c00640 1 -- 172.21.15.67:0/830185209 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f534c07a600 con 0x7f53541604e0 2024-05-30T16:48:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.674+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53540b4340 con 0x7f53541604e0 2024-05-30T16:48:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.675+0000 7f5350c00640 1 -- 172.21.15.67:0/830185209 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5184 (secure 0 0 0) 0x7f534c079f00 con 0x7f53541604e0 2024-05-30T16:48:41.677 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:48:41.677 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2139720967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2139720967},{"type":"v1","addr":"172.21.15.169:6834","nonce":2139720967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4981,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2317664228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2317664228},{"type":"v1","addr":"172.21.15.169:6835","nonce":2317664228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/350920637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":350920637},{"type":"v1","addr":"172.21.15.67:6835","nonce":350920637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:48:30.019937+0000","modified":"2024-05-30T16:48:33.071363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/3595714006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3595714006},{"type":"v1","addr":"172.21.15.67:6837","nonce":3595714006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5361,"qdb_cluster":[5361]},"id":3}]} 2024-05-30T16:48:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 25 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.677+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f533406cbc0 msgr2=0x7f533406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.677+0000 7f535a600640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f533406cbc0 0x7f533406f080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5348007fd0 tx=0x7f5348009040 comp rx=0 tx=0).stop 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.677+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53541604e0 msgr2=0x7f5354160960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.677+0000 7f535a600640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53541604e0 0x7f5354160960 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f534c00e200 tx=0x7f534c009750 comp rx=0 tx=0).stop 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.678+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 shutdown_connections 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.678+0000 7f535a600640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f533406cbc0 0x7f533406f080 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.678+0000 7f535a600640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53541604e0 0x7f5354160960 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.678+0000 7f535a600640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5354151b60 0x7f5354151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.678+0000 7f535a600640 1 --2- 172.21.15.67:0/830185209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533c004d40 0x7f535414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.678+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 >> 172.21.15.67:0/830185209 conn(0x7f53540b1fc0 msgr2=0x7f53540b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.678+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 shutdown_connections 2024-05-30T16:48:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:41.678+0000 7f535a600640 1 -- 172.21.15.67:0/830185209 wait complete. 2024-05-30T16:48:42.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-30T16:48:30.019937+0000', 'modified': '2024-05-30T16:48:33.071363+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5361}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5361': {'gid': 5361, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.67:6837/3595714006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6836', 'nonce': 3595714006}, {'type': 'v1', 'addr': '172.21.15.67:6837', 'nonce': 3595714006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5361, 'qdb_cluster': [5361]} 2024-05-30T16:48:42.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-30T16:48:42.103 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-30T16:48:42.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.323+0000 7f4f63e00640 1 Processor -- start 2024-05-30T16:48:42.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.323+0000 7f4f63e00640 1 -- start start 2024-05-30T16:48:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.325+0000 7f4f63e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f64156b30 0x7f4f640af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.325+0000 7f4f63e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f6406ee40 0x7f4f640ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.325+0000 7f4f63e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f6406f2c0 0x7f4f640ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.325+0000 7f4f63e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4f64156f10 con 0x7f4f6406f2c0 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.325+0000 7f4f63e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4f6406dda0 con 0x7f4f64156b30 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.325+0000 7f4f63e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6406e080 con 0x7f4f6406ee40 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.325+0000 7f4f63400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f6406f2c0 0x7f4f640ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.325+0000 7f4f63400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f6406f2c0 0x7f4f640ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49338/0 (socket says 172.21.15.67:49338) 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.325+0000 7f4f63400640 1 -- 172.21.15.67:0/3359764800 learned_addr learned my addr 172.21.15.67:0/3359764800 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.326+0000 7f4f5be00640 1 --2- 172.21.15.67:0/3359764800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f6406ee40 0x7f4f640ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.326+0000 7f4f62a00640 1 --2- 172.21.15.67:0/3359764800 >> v2:172.21.15.169:3300/0 conn(0x7f4f64156b30 0x7f4f640af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.326+0000 7f4f63400640 1 -- 172.21.15.67:0/3359764800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f6406ee40 msgr2=0x7f4f640ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.326+0000 7f4f63400640 1 --2- 172.21.15.67:0/3359764800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f6406ee40 0x7f4f640ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.326+0000 7f4f63400640 1 -- 172.21.15.67:0/3359764800 >> v2:172.21.15.169:3300/0 conn(0x7f4f64156b30 msgr2=0x7f4f640af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.326+0000 7f4f63400640 1 --2- 172.21.15.67:0/3359764800 >> v2:172.21.15.169:3300/0 conn(0x7f4f64156b30 0x7f4f640af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.326+0000 7f4f63400640 1 -- 172.21.15.67:0/3359764800 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6406e360 con 0x7f4f6406f2c0 2024-05-30T16:48:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.326+0000 7f4f62a00640 1 --2- 172.21.15.67:0/3359764800 >> v2:172.21.15.169:3300/0 conn(0x7f4f64156b30 0x7f4f640af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.327+0000 7f4f63400640 1 --2- 172.21.15.67:0/3359764800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406f2c0 0x7f4f640ad770 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f6406c8b0 tx=0x7f4f5c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f7edcb6e5c43ac server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.327+0000 7f4f62000640 1 -- 172.21.15.67:0/3359764800 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c036070 con 0x7f4f6406f2c0 2024-05-30T16:48:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.328+0000 7f4f62000640 1 -- 172.21.15.67:0/3359764800 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5c031600 con 0x7f4f6406f2c0 2024-05-30T16:48:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.328+0000 7f4f62000640 1 -- 172.21.15.67:0/3359764800 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0319d0 con 0x7f4f6406f2c0 2024-05-30T16:48:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.328+0000 7f4f63e00640 1 -- 172.21.15.67:0/3359764800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406f2c0 msgr2=0x7f4f640ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.328+0000 7f4f63e00640 1 --2- 172.21.15.67:0/3359764800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406f2c0 0x7f4f640ad770 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f6406c8b0 tx=0x7f4f5c02a610 comp rx=0 tx=0).stop 2024-05-30T16:48:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.328+0000 7f4f63e00640 1 -- 172.21.15.67:0/3359764800 shutdown_connections 2024-05-30T16:48:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.328+0000 7f4f63e00640 1 --2- 172.21.15.67:0/3359764800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406f2c0 0x7f4f640ad770 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.328+0000 7f4f63e00640 1 --2- 172.21.15.67:0/3359764800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f6406ee40 0x7f4f640ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.328+0000 7f4f63e00640 1 --2- 172.21.15.67:0/3359764800 >> v2:172.21.15.169:3300/0 conn(0x7f4f64156b30 0x7f4f640af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.328+0000 7f4f63e00640 1 -- 172.21.15.67:0/3359764800 >> 172.21.15.67:0/3359764800 conn(0x7f4f640b1fc0 msgr2=0x7f4f640b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.329+0000 7f4f63e00640 1 -- 172.21.15.67:0/3359764800 shutdown_connections 2024-05-30T16:48:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.329+0000 7f4f63e00640 1 -- 172.21.15.67:0/3359764800 wait complete. 2024-05-30T16:48:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.330+0000 7f4f63e00640 1 Processor -- start 2024-05-30T16:48:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.330+0000 7f4f63e00640 1 -- start start 2024-05-30T16:48:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.331+0000 7f4f63e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406ee40 0x7f4f640ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.331+0000 7f4f63e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f6406f2c0 0x7f4f64157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.331+0000 7f4f63e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f64156b30 0x7f4f641575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.331+0000 7f4f63e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4f640afc20 con 0x7f4f6406ee40 2024-05-30T16:48:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.331+0000 7f4f63e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f640b2c30 con 0x7f4f6406f2c0 2024-05-30T16:48:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.331+0000 7f4f63e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f641494c0 con 0x7f4f64156b30 2024-05-30T16:48:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406ee40 0x7f4f640ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406ee40 0x7f4f640ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49346/0 (socket says 172.21.15.67:49346) 2024-05-30T16:48:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f62a00640 1 -- 172.21.15.67:0/2011581021 learned_addr learned my addr 172.21.15.67:0/2011581021 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f5be00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f6406f2c0 0x7f4f64157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f63400640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f64156b30 0x7f4f641575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f62a00640 1 -- 172.21.15.67:0/2011581021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f64156b30 msgr2=0x7f4f641575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f62a00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f64156b30 0x7f4f641575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f62a00640 1 -- 172.21.15.67:0/2011581021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f6406f2c0 msgr2=0x7f4f64157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f62a00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f6406f2c0 0x7f4f64157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.332+0000 7f4f62a00640 1 -- 172.21.15.67:0/2011581021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f64154950 con 0x7f4f6406ee40 2024-05-30T16:48:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.333+0000 7f4f62a00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406ee40 0x7f4f640ceb30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c820 tx=0x7f4f5400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.333+0000 7f4f5aa00640 1 -- 172.21.15.67:0/2011581021 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54015070 con 0x7f4f6406ee40 2024-05-30T16:48:42.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.333+0000 7f4f5aa00640 1 -- 172.21.15.67:0/2011581021 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f54007a30 con 0x7f4f6406ee40 2024-05-30T16:48:42.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.333+0000 7f4f63e00640 1 -- 172.21.15.67:0/2011581021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f64159080 con 0x7f4f6406ee40 2024-05-30T16:48:42.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.333+0000 7f4f5aa00640 1 -- 172.21.15.67:0/2011581021 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54007de0 con 0x7f4f6406ee40 2024-05-30T16:48:42.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.333+0000 7f4f63e00640 1 -- 172.21.15.67:0/2011581021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f64149710 con 0x7f4f6406ee40 2024-05-30T16:48:42.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.335+0000 7f4f5aa00640 1 -- 172.21.15.67:0/2011581021 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f4f54018070 con 0x7f4f6406ee40 2024-05-30T16:48:42.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.336+0000 7f4f5aa00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f4006cbc0 0x7f4f4006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.336+0000 7f4f5aa00640 1 -- 172.21.15.67:0/2011581021 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f4f5408bf10 con 0x7f4f6406ee40 2024-05-30T16:48:42.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.337+0000 7f4f5be00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f4006cbc0 0x7f4f4006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.337+0000 7f4f6c649740 1 Processor -- start 2024-05-30T16:48:42.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.337+0000 7f4f6c649740 1 -- start start 2024-05-30T16:48:42.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.337+0000 7f4f5be00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f4006cbc0 0x7f4f4006f080 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f6414bf40 tx=0x7f4f4c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.338+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd2490 0x56486acd5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.338+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acd6150 0x56486acf2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.338+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acf25d0 0x56486acf2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.338+0000 7f4f6c649740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56486ac0fb10 con 0x56486acd2490 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.338+0000 7f4f6c649740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56486abb62b0 con 0x56486acf25d0 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.338+0000 7f4f6c649740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56486ac0bad0 con 0x56486acd6150 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.338+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd2490 0x56486acd5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.338+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd2490 0x56486acd5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49362/0 (socket says 172.21.15.67:49362) 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.338+0000 7f4f62a00640 1 -- 172.21.15.67:0/1703814634 learned_addr learned my addr 172.21.15.67:0/1703814634 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.339+0000 7f4f62a00640 1 -- 172.21.15.67:0/1703814634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acd6150 msgr2=0x56486acf2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.339+0000 7f4f62a00640 1 --2- 172.21.15.67:0/1703814634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acd6150 0x56486acf2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.339+0000 7f4f62a00640 1 -- 172.21.15.67:0/1703814634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acf25d0 msgr2=0x56486acf2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.339+0000 7f4f62a00640 1 --2- 172.21.15.67:0/1703814634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acf25d0 0x56486acf2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.339+0000 7f4f62a00640 1 -- 172.21.15.67:0/1703814634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56486a9222d0 con 0x56486acd2490 2024-05-30T16:48:42.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.339+0000 7f4f62a00640 1 --2- 172.21.15.67:0/1703814634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd2490 0x56486acd5c10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4f54059530 tx=0x7f4f54059970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f1c9e4ca3593ca server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.339+0000 7f4f53400640 1 -- 172.21.15.67:0/1703814634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5401f040 con 0x56486acd2490 2024-05-30T16:48:42.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.339+0000 7f4f53400640 1 -- 172.21.15.67:0/1703814634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5400ce10 con 0x56486acd2490 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.339+0000 7f4f53400640 1 -- 172.21.15.67:0/1703814634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54055ea0 con 0x56486acd2490 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.340+0000 7f4f6c649740 1 -- 172.21.15.67:0/1703814634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd2490 msgr2=0x56486acd5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.340+0000 7f4f6c649740 1 --2- 172.21.15.67:0/1703814634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd2490 0x56486acd5c10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4f54059530 tx=0x7f4f54059970 comp rx=0 tx=0).stop 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.340+0000 7f4f6c649740 1 -- 172.21.15.67:0/1703814634 shutdown_connections 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.340+0000 7f4f6c649740 1 --2- 172.21.15.67:0/1703814634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acf25d0 0x56486acf2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.340+0000 7f4f6c649740 1 --2- 172.21.15.67:0/1703814634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acd6150 0x56486acf2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.340+0000 7f4f6c649740 1 --2- 172.21.15.67:0/1703814634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd2490 0x56486acd5c10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.340+0000 7f4f6c649740 1 -- 172.21.15.67:0/1703814634 >> 172.21.15.67:0/1703814634 conn(0x56486aca8470 msgr2=0x56486aca9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.340+0000 7f4f6c649740 1 -- 172.21.15.67:0/1703814634 shutdown_connections 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.340+0000 7f4f6c649740 1 -- 172.21.15.67:0/1703814634 wait complete. 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.341+0000 7f4f6c649740 1 -- start start 2024-05-30T16:48:42.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.341+0000 7f4f6c649740 1 Processor -- start 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486acd99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acf25d0 0x56486acd8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acca740 0x56486acd8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f6c649740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56486aca62a0 con 0x56486acd6150 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f6c649740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56486abc9570 con 0x56486acf25d0 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f6c649740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56486aca6580 con 0x56486acca740 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f6c649740 10 client.0 _ll_register_callbacks cb 0x56486ab97fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486acd99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486acd99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49366/0 (socket says 172.21.15.67:49366) 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.342+0000 7f4f62a00640 1 -- 172.21.15.67:0/3494165600 learned_addr learned my addr 172.21.15.67:0/3494165600 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f62a00640 1 -- 172.21.15.67:0/3494165600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acca740 msgr2=0x56486acd8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f62a00640 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acca740 0x56486acd8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f62a00640 1 -- 172.21.15.67:0/3494165600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acf25d0 msgr2=0x56486acd8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f62a00640 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acf25d0 0x56486acd8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f62a00640 1 -- 172.21.15.67:0/3494165600 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56486a9ec6b0 con 0x56486acd6150 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f62a00640 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486acd99c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4f540597c0 tx=0x7f4f5401c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aab714790880ad4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f47e00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f47e00640 1 -- 172.21.15.67:0/3494165600 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5401f040 con 0x56486acd6150 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f47e00640 1 -- 172.21.15.67:0/3494165600 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f4f5405ab60 con 0x56486acd6150 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f6c649740 1 -- 172.21.15.67:0/3494165600 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f5400ce10 con 0x56486acd6150 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f47e00640 1 -- 172.21.15.67:0/3494165600 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f54080050 con 0x56486acd6150 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f6c649740 10 client.5502 fetch_fsmap learned FSMap version 25 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.343+0000 7f4f6c649740 1 -- 172.21.15.67:0/3494165600 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x56486acd1b80 con 0x56486acd6150 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.344+0000 7f4f47e00640 1 -- 172.21.15.67:0/3494165600 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5400ce10 con 0x56486acd6150 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.344+0000 7f4f47e00640 1 -- 172.21.15.67:0/3494165600 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7f4f54083020 con 0x56486acd6150 2024-05-30T16:48:42.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.344+0000 7f4f6c649740 10 client.5502 fetch_fsmap finished waiting for FSMap version 25 2024-05-30T16:48:42.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.344+0000 7f4f6c649740 10 client.5502 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-30T16:48:42.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.344+0000 7f4f6c649740 20 client.5502 populate_metadata read hostname 'smithi067' 2024-05-30T16:48:42.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.344+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] conn(0x56486ad010e0 0x56486acd74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.344+0000 7f4f6c649740 4 client.5502 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-30T16:48:42.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.344+0000 7f4f6c649740 1 -- 172.21.15.67:0/3494165600 --> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56486ac1d530 con 0x56486ad010e0 2024-05-30T16:48:42.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.344+0000 7f4f5be00640 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] conn(0x56486ad010e0 0x56486acd74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.345+0000 7f4f5be00640 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] conn(0x56486ad010e0 0x56486acd74f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f9655710f69a7d server_cookie=6cbbf4571c5edded in_seq=0 out_seq=0 2024-05-30T16:48:42.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.345+0000 7f4f47e00640 10 client.5502 ms_handle_connect on v2:172.21.15.67:6836/3595714006 2024-05-30T16:48:42.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f47e00640 1 -- 172.21.15.67:0/3494165600 <== mds.0 v2:172.21.15.67:6836/3595714006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56486ac1d530 con 0x56486ad010e0 2024-05-30T16:48:42.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f47e00640 10 client.5502 handle_command_reply: tid=0 2024-05-30T16:48:42.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f6c649740 1 client.5502 shutdown 2024-05-30T16:48:42.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f6c649740 1 -- 172.21.15.67:0/3494165600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 msgr2=0x56486acd99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486acd99c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4f540597c0 tx=0x7f4f5401c070 comp rx=0 tx=0).stop 2024-05-30T16:48:42.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f6c649740 1 -- 172.21.15.67:0/3494165600 shutdown_connections 2024-05-30T16:48:42.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] conn(0x56486ad010e0 0x56486acd74f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acca740 0x56486acd8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acf25d0 0x56486acd8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3494165600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486acd99c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f6c649740 1 -- 172.21.15.67:0/3494165600 >> 172.21.15.67:0/3494165600 conn(0x56486acd2490 msgr2=0x56486aca51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.346+0000 7f4f47e00640 0 client.5502 ms_handle_reset on v2:172.21.15.67:6836/3595714006 2024-05-30T16:48:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.347+0000 7f4f6c649740 1 -- 172.21.15.67:0/3494165600 shutdown_connections 2024-05-30T16:48:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.347+0000 7f4f6c649740 1 -- 172.21.15.67:0/3494165600 wait complete. 2024-05-30T16:48:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.347+0000 7f4f6c649740 20 client.5502 trim_cache size 0 max 16384 2024-05-30T16:48:42.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.367+0000 7f4f6c649740 1 Processor -- start 2024-05-30T16:48:42.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.367+0000 7f4f6c649740 1 -- start start 2024-05-30T16:48:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.368+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acca740 0x56486acab580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.368+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486aca7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.368+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 0x56486acacb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.368+0000 7f4f6c649740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641ffd30 con 0x56486acd6150 2024-05-30T16:48:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.368+0000 7f4f6c649740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641ff0c0 con 0x56486acca740 2024-05-30T16:48:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.368+0000 7f4f6c649740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f641ff6e0 con 0x56486acf25d0 2024-05-30T16:48:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.368+0000 7f4f5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486aca7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.368+0000 7f4f5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486aca7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49374/0 (socket says 172.21.15.67:49374) 2024-05-30T16:48:42.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.368+0000 7f4f5be00640 1 -- 172.21.15.67:0/2962298780 learned_addr learned my addr 172.21.15.67:0/2962298780 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f63400640 1 --2- 172.21.15.67:0/2962298780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 0x56486acacb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f5be00640 1 -- 172.21.15.67:0/2962298780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 msgr2=0x56486acacb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f62a00640 1 --2- 172.21.15.67:0/2962298780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acca740 0x56486acab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f5be00640 1 --2- 172.21.15.67:0/2962298780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 0x56486acacb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f5be00640 1 -- 172.21.15.67:0/2962298780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acca740 msgr2=0x56486acab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f5be00640 1 --2- 172.21.15.67:0/2962298780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acca740 0x56486acab580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f5be00640 1 -- 172.21.15.67:0/2962298780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56486acf2400 con 0x56486acd6150 2024-05-30T16:48:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f5be00640 1 --2- 172.21.15.67:0/2962298780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486aca7680 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c006fd0 tx=0x7f4f4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4e78b36a752c50 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f52a00640 1 -- 172.21.15.67:0/2962298780 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c017930 con 0x56486acd6150 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f52a00640 1 -- 172.21.15.67:0/2962298780 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c017ad0 con 0x56486acd6150 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.369+0000 7f4f52a00640 1 -- 172.21.15.67:0/2962298780 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c017e80 con 0x56486acd6150 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.370+0000 7f4f6c649740 1 -- 172.21.15.67:0/2962298780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 msgr2=0x56486aca7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.370+0000 7f4f6c649740 1 --2- 172.21.15.67:0/2962298780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486aca7680 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c006fd0 tx=0x7f4f4c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.370+0000 7f4f6c649740 1 -- 172.21.15.67:0/2962298780 shutdown_connections 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.370+0000 7f4f6c649740 1 --2- 172.21.15.67:0/2962298780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 0x56486acacb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.370+0000 7f4f6c649740 1 --2- 172.21.15.67:0/2962298780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acd6150 0x56486aca7680 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.370+0000 7f4f6c649740 1 --2- 172.21.15.67:0/2962298780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acca740 0x56486acab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.370+0000 7f4f6c649740 1 -- 172.21.15.67:0/2962298780 >> 172.21.15.67:0/2962298780 conn(0x56486acd2490 msgr2=0x56486acabe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.370+0000 7f4f6c649740 1 -- 172.21.15.67:0/2962298780 shutdown_connections 2024-05-30T16:48:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.370+0000 7f4f6c649740 1 -- 172.21.15.67:0/2962298780 wait complete. 2024-05-30T16:48:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.371+0000 7f4f6c649740 1 -- start start 2024-05-30T16:48:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.371+0000 7f4f6c649740 1 Processor -- start 2024-05-30T16:48:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acca740 0x56486acacf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acd6150 0x56486ad69a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f6c649740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 0x56486ad69f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f6c649740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56486aca62a0 con 0x56486acca740 2024-05-30T16:48:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f6c649740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56486aca6580 con 0x56486acd6150 2024-05-30T16:48:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f6c649740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56486abc9570 con 0x56486acf25d0 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f6c649740 10 client.0 _ll_register_callbacks cb 0x7f4f5400a450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acca740 0x56486acacf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acca740 0x56486acacf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49384/0 (socket says 172.21.15.67:49384) 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.372+0000 7f4f62a00640 1 -- 172.21.15.67:0/3411585317 learned_addr learned my addr 172.21.15.67:0/3411585317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f63400640 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 0x56486ad69f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f5be00640 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acd6150 0x56486ad69a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f62a00640 1 -- 172.21.15.67:0/3411585317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 msgr2=0x56486ad69f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f62a00640 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 0x56486ad69f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f62a00640 1 -- 172.21.15.67:0/3411585317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acd6150 msgr2=0x56486ad69a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f62a00640 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acd6150 0x56486ad69a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f62a00640 1 -- 172.21.15.67:0/3411585317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56486acad490 con 0x56486acca740 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f62a00640 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acca740 0x56486acacf50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4f5405dd70 tx=0x7f4f54055c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b80bcf2f3e7a72a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f47e00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f47e00640 1 -- 172.21.15.67:0/3411585317 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5401f040 con 0x56486acca740 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f47e00640 1 -- 172.21.15.67:0/3411585317 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f4f5408a240 con 0x56486acca740 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.373+0000 7f4f6c649740 1 -- 172.21.15.67:0/3411585317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4c017ad0 con 0x56486acca740 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f47e00640 1 -- 172.21.15.67:0/3411585317 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f5406b050 con 0x56486acca740 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f6c649740 10 client.5508 fetch_fsmap learned FSMap version 25 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f6c649740 1 -- 172.21.15.67:0/3411585317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x56486acd09a0 con 0x56486acca740 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f47e00640 1 -- 172.21.15.67:0/3411585317 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c017ad0 con 0x56486acca740 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f47e00640 1 -- 172.21.15.67:0/3411585317 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7f4f54083040 con 0x56486acca740 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f6c649740 10 client.5508 fetch_fsmap finished waiting for FSMap version 25 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f6c649740 10 client.5508 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f6c649740 20 client.5508 populate_metadata read hostname 'smithi067' 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] conn(0x56486aced900 0x56486acccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f6c649740 4 client.5508 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f6c649740 1 -- 172.21.15.67:0/3411585317 --> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] -- command(tid 0: {"prefix": "status"}) -- 0x56486acd0160 con 0x56486aced900 2024-05-30T16:48:42.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.374+0000 7f4f5be00640 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] conn(0x56486aced900 0x56486acccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.375+0000 7f4f5be00640 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] conn(0x56486aced900 0x56486acccb30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b45110f39ce6ff server_cookie=81a1c0a93e5a7c5d in_seq=0 out_seq=0 2024-05-30T16:48:42.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.375+0000 7f4f47e00640 10 client.5508 ms_handle_connect on v2:172.21.15.67:6836/3595714006 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.375+0000 7f4f47e00640 1 -- 172.21.15.67:0/3411585317 <== mds.0 v2:172.21.15.67:6836/3595714006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56486acd0160 con 0x56486aced900 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.375+0000 7f4f47e00640 10 client.5508 handle_command_reply: tid=0 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.375+0000 7f4f6c649740 1 client.5508 shutdown 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.375+0000 7f4f6c649740 1 -- 172.21.15.67:0/3411585317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acca740 msgr2=0x56486acacf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.375+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acca740 0x56486acacf50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4f5405dd70 tx=0x7f4f54055c00 comp rx=0 tx=0).stop 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f6c649740 1 -- 172.21.15.67:0/3411585317 shutdown_connections 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.67:6836/3595714006,v1:172.21.15.67:6837/3595714006] conn(0x56486aced900 0x56486acccb30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56486acf25d0 0x56486ad69f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56486acd6150 0x56486ad69a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f6c649740 1 --2- 172.21.15.67:0/3411585317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56486acca740 0x56486acacf50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f47e00640 0 client.5508 ms_handle_reset on v2:172.21.15.67:6836/3595714006 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f6c649740 1 -- 172.21.15.67:0/3411585317 >> 172.21.15.67:0/3411585317 conn(0x56486acd2490 msgr2=0x56486ad001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f6c649740 1 -- 172.21.15.67:0/3411585317 shutdown_connections 2024-05-30T16:48:42.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f6c649740 1 -- 172.21.15.67:0/3411585317 wait complete. 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.376+0000 7f4f6c649740 20 client.5508 trim_cache size 0 max 16384 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5361, 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 11.840259158, 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 25, 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 48, 2024-05-30T16:48:42.380 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 48, 2024-05-30T16:48:42.381 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 16.212303134999999 2024-05-30T16:48:42.381 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:48:42.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 -- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f4006cbc0 msgr2=0x7f4f4006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f4006cbc0 0x7f4f4006f080 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f6414bf40 tx=0x7f4f4c008040 comp rx=0 tx=0).stop 2024-05-30T16:48:42.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 -- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406ee40 msgr2=0x7f4f640ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406ee40 0x7f4f640ceb30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c820 tx=0x7f4f5400cce0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 -- 172.21.15.67:0/2011581021 shutdown_connections 2024-05-30T16:48:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f4006cbc0 0x7f4f4006f080 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f64156b30 0x7f4f641575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f6406f2c0 0x7f4f64157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 --2- 172.21.15.67:0/2011581021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f6406ee40 0x7f4f640ceb30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 -- 172.21.15.67:0/2011581021 >> 172.21.15.67:0/2011581021 conn(0x7f4f640b1fc0 msgr2=0x7f4f640aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 -- 172.21.15.67:0/2011581021 shutdown_connections 2024-05-30T16:48:42.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.377+0000 7f4f63e00640 1 -- 172.21.15.67:0/2011581021 wait complete. 2024-05-30T16:48:42.397 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T16:48:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.606+0000 7f7fd1a00640 1 Processor -- start 2024-05-30T16:48:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.606+0000 7f7fd1a00640 1 -- start start 2024-05-30T16:48:42.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.608+0000 7f7fd1a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7fcc158b80 0x7f7fcc06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.608+0000 7f7fd1a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fcc06e6e0 con 0x7f7fcc06e160 2024-05-30T16:48:42.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.608+0000 7f7fd1a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fcc06e9c0 con 0x7f7fcc0722d0 2024-05-30T16:48:42.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.608+0000 7f7fd1a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7fcc06eca0 con 0x7f7fcc158b80 2024-05-30T16:48:42.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.608+0000 7f7fcbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7fcc158b80 0x7f7fcc06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.608+0000 7f7fcbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7fcc158b80 0x7f7fcc06dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49386/0 (socket says 172.21.15.67:49386) 2024-05-30T16:48:42.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.608+0000 7f7fcbe00640 1 -- 172.21.15.67:0/3925316002 learned_addr learned my addr 172.21.15.67:0/3925316002 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:42.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.609+0000 7f7fcbe00640 1 -- 172.21.15.67:0/3925316002 >> v1:172.21.15.169:6789/0 conn(0x7f7fcc0722d0 legacy=0x7f7fcc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:42.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.609+0000 7f7fcbe00640 1 -- 172.21.15.67:0/3925316002 >> v1:172.21.15.67:6789/0 conn(0x7f7fcc06e160 legacy=0x7f7fcc071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.609+0000 7f7fcbe00640 1 -- 172.21.15.67:0/3925316002 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc06ef80 con 0x7f7fcc158b80 2024-05-30T16:48:42.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.610+0000 7f7fcbe00640 1 --2- 172.21.15.67:0/3925316002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc158b80 0x7f7fcc06dc20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009870 tx=0x7f7fbc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c474b03c817c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.611+0000 7f7fcaa00640 1 -- 172.21.15.67:0/3925316002 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc037070 con 0x7f7fcc158b80 2024-05-30T16:48:42.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.611+0000 7f7fcaa00640 1 -- 172.21.15.67:0/3925316002 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc032440 con 0x7f7fcc158b80 2024-05-30T16:48:42.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.611+0000 7f7fcaa00640 1 -- 172.21.15.67:0/3925316002 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc032810 con 0x7f7fcc158b80 2024-05-30T16:48:42.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.611+0000 7f7fd1a00640 1 -- 172.21.15.67:0/3925316002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc158b80 msgr2=0x7f7fcc06dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.611+0000 7f7fd1a00640 1 --2- 172.21.15.67:0/3925316002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc158b80 0x7f7fcc06dc20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009870 tx=0x7f7fbc02a2a0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.611+0000 7f7fd1a00640 1 -- 172.21.15.67:0/3925316002 shutdown_connections 2024-05-30T16:48:42.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.611+0000 7f7fd1a00640 1 --2- 172.21.15.67:0/3925316002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc158b80 0x7f7fcc06dc20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.611+0000 7f7fd1a00640 1 -- 172.21.15.67:0/3925316002 >> 172.21.15.67:0/3925316002 conn(0x7f7fcc0b1fc0 msgr2=0x7f7fcc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:42.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.612+0000 7f7fd1a00640 1 -- 172.21.15.67:0/3925316002 shutdown_connections 2024-05-30T16:48:42.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.612+0000 7f7fd1a00640 1 -- 172.21.15.67:0/3925316002 wait complete. 2024-05-30T16:48:42.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.613+0000 7f7fd1a00640 1 Processor -- start 2024-05-30T16:48:42.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.613+0000 7f7fd1a00640 1 -- start start 2024-05-30T16:48:42.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.614+0000 7f7fd1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc06e160 0x7f7fcc09a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.614+0000 7f7fd1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fcc0722d0 0x7f7fcc0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.614+0000 7f7fd1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fcc158b80 0x7f7fcc14f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.614+0000 7f7fd1a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc06f780 con 0x7f7fcc06e160 2024-05-30T16:48:42.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.614+0000 7f7fd1a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc0b2d30 con 0x7f7fcc0722d0 2024-05-30T16:48:42.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.614+0000 7f7fd1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7fcc157620 con 0x7f7fcc158b80 2024-05-30T16:48:42.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.615+0000 7f7fcbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc06e160 0x7f7fcc09a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.615+0000 7f7fcbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc06e160 0x7f7fcc09a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49400/0 (socket says 172.21.15.67:49400) 2024-05-30T16:48:42.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.615+0000 7f7fcbe00640 1 -- 172.21.15.67:0/481577429 learned_addr learned my addr 172.21.15.67:0/481577429 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.615+0000 7f7fcb400640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fcc0722d0 0x7f7fcc0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.615+0000 7f7fcbe00640 1 -- 172.21.15.67:0/481577429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fcc158b80 msgr2=0x7f7fcc14f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.615+0000 7f7fcbe00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fcc158b80 0x7f7fcc14f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.615+0000 7f7fcbe00640 1 -- 172.21.15.67:0/481577429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fcc0722d0 msgr2=0x7f7fcc0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.615+0000 7f7fcbe00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fcc0722d0 0x7f7fcc0b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.615+0000 7f7fcbe00640 1 -- 172.21.15.67:0/481577429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc09a8f0 con 0x7f7fcc06e160 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.616+0000 7f7fcbe00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc06e160 0x7f7fcc09a3b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009870 tx=0x7f7fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.616+0000 7f7fc3e00640 1 -- 172.21.15.67:0/481577429 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc041070 con 0x7f7fcc06e160 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.616+0000 7f7fc3e00640 1 -- 172.21.15.67:0/481577429 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc03c870 con 0x7f7fcc06e160 2024-05-30T16:48:42.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.616+0000 7f7fc3e00640 1 -- 172.21.15.67:0/481577429 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc037040 con 0x7f7fcc06e160 2024-05-30T16:48:42.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.616+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc0b4c80 con 0x7f7fcc06e160 2024-05-30T16:48:42.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.616+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc13d810 con 0x7f7fcc06e160 2024-05-30T16:48:42.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.617+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fcc0689e0 con 0x7f7fcc06e160 2024-05-30T16:48:42.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.618+0000 7f7fc3e00640 1 -- 172.21.15.67:0/481577429 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f7fbc044070 con 0x7f7fcc06e160 2024-05-30T16:48:42.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.619+0000 7f7fc3e00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fac06cbf0 0x7f7fac06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:42.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.619+0000 7f7fc3e00640 1 -- 172.21.15.67:0/481577429 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f7fbc0accb0 con 0x7f7fcc06e160 2024-05-30T16:48:42.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.619+0000 7f7fcb400640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fac06cbf0 0x7f7fac06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:42.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.620+0000 7f7fcb400640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fac06cbf0 0x7f7fac06f0b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0b5250 tx=0x7f7fb8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.624+0000 7f7fc3e00640 1 -- 172.21.15.67:0/481577429 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fbc07aeb0 con 0x7f7fcc06e160 2024-05-30T16:48:42.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.876+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7fcc1611a0 con 0x7f7fcc06e160 2024-05-30T16:48:42.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.877+0000 7f7fc3e00640 1 -- 172.21.15.67:0/481577429 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f7fbc07a7b0 con 0x7f7fcc06e160 2024-05-30T16:48:42.878 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T16:48:42.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.879+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fac06cbf0 msgr2=0x7f7fac06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.879+0000 7f7fd1a00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fac06cbf0 0x7f7fac06f0b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0b5250 tx=0x7f7fb8009040 comp rx=0 tx=0).stop 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.879+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc06e160 msgr2=0x7f7fcc09a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.879+0000 7f7fd1a00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc06e160 0x7f7fcc09a3b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009870 tx=0x7f7fbc0046c0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.879+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 shutdown_connections 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.879+0000 7f7fd1a00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7fac06cbf0 0x7f7fac06f0b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.879+0000 7f7fd1a00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fcc158b80 0x7f7fcc14f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.879+0000 7f7fd1a00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fcc0722d0 0x7f7fcc0b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.879+0000 7f7fd1a00640 1 --2- 172.21.15.67:0/481577429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fcc06e160 0x7f7fcc09a3b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.880+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 >> 172.21.15.67:0/481577429 conn(0x7f7fcc0b1fc0 msgr2=0x7f7fcc1616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.880+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 shutdown_connections 2024-05-30T16:48:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:42.880+0000 7f7fd1a00640 1 -- 172.21.15.67:0/481577429 wait complete. 2024-05-30T16:48:42.900 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T16:48:42.900 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T16:48:42.900 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T16:48:42.900 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-30T16:48:42.900 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T16:48:42.900 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T16:48:42.900 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T16:48:42.900 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T16:48:42.900 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:48:42.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:42.900 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T16:48:42.953 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:48:42.953 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:48:42.953 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T16:48:42.954 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42249sec preferred_lft 42249sec 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::74f5:cfff:fec0:693d/64 scope link 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::a8fe:aeff:fef7:f6a0/64 scope link 2024-05-30T16:48:42.955 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:48:42.956 INFO:teuthology.orchestra.run.smithi067.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:48:42.956 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:48:42.956 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::185e:a4ff:fec6:7d8c/64 scope link 2024-05-30T16:48:42.956 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:48:42.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:42.957 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T16:48:43.038 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T16:48:43.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:43.042 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T16:48:43.101 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T16:48:43.101 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T16:48:43.101 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T16:48:43.101 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T16:48:43.101 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T16:48:43.102 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:48:43.102 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:48:43.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T16:48:43.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T16:48:43.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T16:48:43.102 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:48:43.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:48:43.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:48:43.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:43.328 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T16:48:43.377 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T16:48:43.378 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-30T16:48:43.378 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:48:43.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:43.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-05-30T16:48:43.464 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-30T16:48:43.464 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:48:43.464 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-30T16:48:43.523 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:48:43.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:48:43.524 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:48:43.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:48:43.663 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:48:43.720 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:48:43.720 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:48:43.720 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:48:43.720 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:48:43.720 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:48:43.720 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:48:43.721 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:48:43.722 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:48:43.723 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:48:43.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:43.723 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:48:43.775 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T16:48:43.776 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-30T16:48:43.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:48:43.847 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:48:43.887 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:48:43.887 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:48:43.887 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:48:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:48:43.889 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:48:43.890 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:48:43.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:43.890 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:48:43.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-05-30T16:48:43.950+0000 7f939a0f1480 -1 init, newargv = 0x560a6ba41e30 newargc=15 2024-05-30T16:48:43.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[51789]: starting ceph client 2024-05-30T16:48:43.960 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[51789]: starting fuse 2024-05-30T16:48:43.973 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T16:48:43.973 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-30T16:48:43.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T16:48:43.973 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T16:48:43.973 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T16:48:43.973 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T16:48:43.973 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T16:48:43.973 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T16:48:43.974 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T16:48:43.975 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T16:48:43.975 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T16:48:43.975 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T16:48:43.975 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:48:43.975 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-05-30T16:48:43.975 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T16:48:44.165 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.51789.asok 2024-05-30T16:48:44.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:48:44.182 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51789.asok 2024-05-30T16:48:44.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:44.182 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51789.asok status 2024-05-30T16:48:44.423 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:48:44.423 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T16:48:44.423 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T16:48:44.423 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T16:48:44.423 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-05-30T16:48:44.423 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T16:48:44.423 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "51789", 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5526, 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5526 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:48:44.424 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2234149451 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2234149451 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.5526 192.168.0.1:0/2234149451", 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/2234149451", 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 25, 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 48, 2024-05-30T16:48:44.425 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T16:48:44.426 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T16:48:44.426 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T16:48:44.426 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:48:44.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:44.439 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T16:48:44.439 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:48:44.495 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T16:48:44.495 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-30T16:48:44.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:44.496 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:48:44.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-30T16:48:44.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.805+0000 7f6e1ac00640 1 Processor -- start 2024-05-30T16:48:44.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.805+0000 7f6e1ac00640 1 -- start start 2024-05-30T16:48:44.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.807+0000 7f6e1ac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6e140af6f0 0x7f6e141623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:44.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.807+0000 7f6e1ac00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6e141480a0 0x7f6e140d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:44.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.807+0000 7f6e1ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140d0970 0x7f6e14144060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:44.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.807+0000 7f6e1ac00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f6e140d0dc0 con 0x7f6e141480a0 2024-05-30T16:48:44.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.807+0000 7f6e1ac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6e14162cb0 con 0x7f6e140af6f0 2024-05-30T16:48:44.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.807+0000 7f6e1ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6e14162f90 con 0x7f6e140d0970 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.807+0000 7f6e18e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6e141480a0 0x7f6e140d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.807+0000 7f6e18e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6e141480a0 0x7f6e140d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49412/0 (socket says 172.21.15.67:49412) 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.807+0000 7f6e18e00640 1 -- 172.21.15.67:0/3124644268 learned_addr learned my addr 172.21.15.67:0/3124644268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.808+0000 7f6e1a200640 1 --2- 172.21.15.67:0/3124644268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140d0970 0x7f6e14144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.808+0000 7f6e19800640 1 --2- 172.21.15.67:0/3124644268 >> v2:172.21.15.169:3300/0 conn(0x7f6e140af6f0 0x7f6e141623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.808+0000 7f6e18e00640 1 -- 172.21.15.67:0/3124644268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140d0970 msgr2=0x7f6e14144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.808+0000 7f6e18e00640 1 --2- 172.21.15.67:0/3124644268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140d0970 0x7f6e14144060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.808+0000 7f6e18e00640 1 -- 172.21.15.67:0/3124644268 >> v2:172.21.15.169:3300/0 conn(0x7f6e140af6f0 msgr2=0x7f6e141623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.808+0000 7f6e18e00640 1 --2- 172.21.15.67:0/3124644268 >> v2:172.21.15.169:3300/0 conn(0x7f6e140af6f0 0x7f6e141623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.808+0000 7f6e18e00640 1 -- 172.21.15.67:0/3124644268 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e14163270 con 0x7f6e141480a0 2024-05-30T16:48:44.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.808+0000 7f6e19800640 1 --2- 172.21.15.67:0/3124644268 >> v2:172.21.15.169:3300/0 conn(0x7f6e140af6f0 0x7f6e141623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:44.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.808+0000 7f6e1a200640 1 --2- 172.21.15.67:0/3124644268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140d0970 0x7f6e14144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:44.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.809+0000 7f6e18e00640 1 --2- 172.21.15.67:0/3124644268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 0x7f6e140d0430 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6e140d10c0 tx=0x7f6e0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8002a69c5c4512a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:44.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.810+0000 7f6e13e00640 1 -- 172.21.15.67:0/3124644268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00037070 con 0x7f6e141480a0 2024-05-30T16:48:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.810+0000 7f6e13e00640 1 -- 172.21.15.67:0/3124644268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e00032440 con 0x7f6e141480a0 2024-05-30T16:48:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.810+0000 7f6e13e00640 1 -- 172.21.15.67:0/3124644268 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00032810 con 0x7f6e141480a0 2024-05-30T16:48:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.811+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3124644268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 msgr2=0x7f6e140d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.811+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3124644268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 0x7f6e140d0430 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6e140d10c0 tx=0x7f6e0002a1e0 comp rx=0 tx=0).stop 2024-05-30T16:48:44.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.811+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3124644268 shutdown_connections 2024-05-30T16:48:44.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.811+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3124644268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 0x7f6e140d0430 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:44.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.811+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3124644268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140d0970 0x7f6e14144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:44.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.811+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3124644268 >> v2:172.21.15.169:3300/0 conn(0x7f6e140af6f0 0x7f6e141623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:44.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.811+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3124644268 >> 172.21.15.67:0/3124644268 conn(0x7f6e1406bae0 msgr2=0x7f6e1406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:44.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.811+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3124644268 shutdown_connections 2024-05-30T16:48:44.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.811+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3124644268 wait complete. 2024-05-30T16:48:44.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.813+0000 7f6e1ac00640 1 Processor -- start 2024-05-30T16:48:44.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.814+0000 7f6e1ac00640 1 -- start start 2024-05-30T16:48:44.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.815+0000 7f6e1ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140af6f0 0x7f6e14159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:44.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.815+0000 7f6e1ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e140d0970 0x7f6e1415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:44.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.815+0000 7f6e1ac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 0x7f6e140cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:44.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.815+0000 7f6e1ac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141628f0 con 0x7f6e141480a0 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.815+0000 7f6e1ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141423b0 con 0x7f6e140d0970 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.815+0000 7f6e1ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6e14162a70 con 0x7f6e140af6f0 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.815+0000 7f6e1a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 0x7f6e140cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.815+0000 7f6e1a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 0x7f6e140cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49414/0 (socket says 172.21.15.67:49414) 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.815+0000 7f6e1a200640 1 -- 172.21.15.67:0/3388227761 learned_addr learned my addr 172.21.15.67:0/3388227761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e1a200640 1 -- 172.21.15.67:0/3388227761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140af6f0 msgr2=0x7f6e14159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e1a200640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140af6f0 0x7f6e14159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e1a200640 1 -- 172.21.15.67:0/3388227761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e140d0970 msgr2=0x7f6e1415f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e18e00640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e140d0970 0x7f6e1415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e1a200640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e140d0970 0x7f6e1415f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e1a200640 1 -- 172.21.15.67:0/3388227761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1415a500 con 0x7f6e141480a0 2024-05-30T16:48:44.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e1a200640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 0x7f6e140cc7a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00cd80 tx=0x7f6e0c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:44.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e11600640 1 -- 172.21.15.67:0/3388227761 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00f680 con 0x7f6e141480a0 2024-05-30T16:48:44.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e11600640 1 -- 172.21.15.67:0/3388227761 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0c00f840 con 0x7f6e141480a0 2024-05-30T16:48:44.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e11600640 1 -- 172.21.15.67:0/3388227761 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00fc30 con 0x7f6e141480a0 2024-05-30T16:48:44.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.816+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1415ad10 con 0x7f6e141480a0 2024-05-30T16:48:44.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.817+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1415b2d0 con 0x7f6e141480a0 2024-05-30T16:48:44.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.818+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e140689e0 con 0x7f6e141480a0 2024-05-30T16:48:44.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.819+0000 7f6e11600640 1 -- 172.21.15.67:0/3388227761 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f6e0c0213f0 con 0x7f6e141480a0 2024-05-30T16:48:44.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.820+0000 7f6e11600640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6df406cbc0 0x7f6df406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:44.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.820+0000 7f6e11600640 1 -- 172.21.15.67:0/3388227761 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f6e0c08ceb0 con 0x7f6e141480a0 2024-05-30T16:48:44.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.820+0000 7f6e19800640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6df406cbc0 0x7f6df406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:44.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.821+0000 7f6e19800640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6df406cbc0 0x7f6df406f080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6e08006b10 tx=0x7f6e08008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:44.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:44.825+0000 7f6e11600640 1 -- 172.21.15.67:0/3388227761 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e0c014030 con 0x7f6e141480a0 2024-05-30T16:48:45.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.071+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f6e140c9e50 con 0x7f6e141480a0 2024-05-30T16:48:45.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.657+0000 7f6e11600640 1 -- 172.21.15.67:0/3388227761 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) ==== 136+0+0 (secure 0 0 0) 0x7f6e0c05adf0 con 0x7f6e141480a0 2024-05-30T16:48:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6df406cbc0 msgr2=0x7f6df406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6df406cbc0 0x7f6df406f080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6e08006b10 tx=0x7f6e08008040 comp rx=0 tx=0).stop 2024-05-30T16:48:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 msgr2=0x7f6e140cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 0x7f6e140cc7a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00cd80 tx=0x7f6e0c00a650 comp rx=0 tx=0).stop 2024-05-30T16:48:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 shutdown_connections 2024-05-30T16:48:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6df406cbc0 0x7f6df406f080 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e141480a0 0x7f6e140cc7a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e140d0970 0x7f6e1415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:45.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 --2- 172.21.15.67:0/3388227761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e140af6f0 0x7f6e14159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:45.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 >> 172.21.15.67:0/3388227761 conn(0x7f6e1406bae0 msgr2=0x7f6e1406ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:45.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 shutdown_connections 2024-05-30T16:48:45.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.662+0000 7f6e1ac00640 1 -- 172.21.15.67:0/3388227761 wait complete. 2024-05-30T16:48:45.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:48:45.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.897+0000 7f7a46400640 1 Processor -- start 2024-05-30T16:48:45.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.898+0000 7f7a46400640 1 -- start start 2024-05-30T16:48:45.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.899+0000 7f7a46400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a40156b30 0x7f7a400af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:45.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.899+0000 7f7a46400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7a4006ee40 0x7f7a400ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:45.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.899+0000 7f7a46400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a40156f10 con 0x7f7a4006f2c0 2024-05-30T16:48:45.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.899+0000 7f7a46400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7a4006dda0 con 0x7f7a4006ee40 2024-05-30T16:48:45.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.899+0000 7f7a46400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4006e080 con 0x7f7a40156b30 2024-05-30T16:48:45.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.900+0000 7f7a3fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7a4006ee40 0x7f7a400ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:45.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.900+0000 7f7a3fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7a4006ee40 0x7f7a400ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49416/0 (socket says 172.21.15.67:49416) 2024-05-30T16:48:45.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.900+0000 7f7a3fe00640 1 -- 172.21.15.67:0/4031495227 learned_addr learned my addr 172.21.15.67:0/4031495227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:45.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.900+0000 7f7a45000640 1 --2- 172.21.15.67:0/4031495227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a40156b30 0x7f7a400af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:45.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.900+0000 7f7a3fe00640 1 -- 172.21.15.67:0/4031495227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a40156b30 msgr2=0x7f7a400af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:45.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.900+0000 7f7a3fe00640 1 --2- 172.21.15.67:0/4031495227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a40156b30 0x7f7a400af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:45.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.900+0000 7f7a3fe00640 1 -- 172.21.15.67:0/4031495227 >> v1:172.21.15.169:6789/0 conn(0x7f7a4006f2c0 legacy=0x7f7a400b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:45.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.900+0000 7f7a3fe00640 1 -- 172.21.15.67:0/4031495227 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4006e360 con 0x7f7a4006ee40 2024-05-30T16:48:45.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.901+0000 7f7a3fe00640 1 --2- 172.21.15.67:0/4031495227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006ee40 0x7f7a400ad230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009870 tx=0x7f7a2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3450923b5a1f70a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:45.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.902+0000 7f7a3f400640 1 -- 172.21.15.67:0/4031495227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c037070 con 0x7f7a4006ee40 2024-05-30T16:48:45.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.902+0000 7f7a3f400640 1 -- 172.21.15.67:0/4031495227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2c032440 con 0x7f7a4006ee40 2024-05-30T16:48:45.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.902+0000 7f7a3f400640 1 -- 172.21.15.67:0/4031495227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c032810 con 0x7f7a4006ee40 2024-05-30T16:48:45.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.902+0000 7f7a46400640 1 -- 172.21.15.67:0/4031495227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006ee40 msgr2=0x7f7a400ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:45.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.902+0000 7f7a46400640 1 --2- 172.21.15.67:0/4031495227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006ee40 0x7f7a400ad230 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009870 tx=0x7f7a2c02a180 comp rx=0 tx=0).stop 2024-05-30T16:48:45.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.903+0000 7f7a46400640 1 -- 172.21.15.67:0/4031495227 shutdown_connections 2024-05-30T16:48:45.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.903+0000 7f7a46400640 1 --2- 172.21.15.67:0/4031495227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006ee40 0x7f7a400ad230 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:45.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.903+0000 7f7a46400640 1 --2- 172.21.15.67:0/4031495227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a40156b30 0x7f7a400af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:45.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.903+0000 7f7a46400640 1 -- 172.21.15.67:0/4031495227 >> 172.21.15.67:0/4031495227 conn(0x7f7a400b1fc0 msgr2=0x7f7a400b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:45.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.903+0000 7f7a46400640 1 -- 172.21.15.67:0/4031495227 shutdown_connections 2024-05-30T16:48:45.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.903+0000 7f7a46400640 1 -- 172.21.15.67:0/4031495227 wait complete. 2024-05-30T16:48:45.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.904+0000 7f7a46400640 1 Processor -- start 2024-05-30T16:48:45.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.905+0000 7f7a46400640 1 -- start start 2024-05-30T16:48:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.905+0000 7f7a46400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a4006ee40 0x7f7a401f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.905+0000 7f7a46400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006f2c0 0x7f7a401f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.905+0000 7f7a46400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a40156b30 0x7f7a401f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a46400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400afc20 con 0x7f7a4006f2c0 2024-05-30T16:48:45.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a46400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400b2c30 con 0x7f7a40156b30 2024-05-30T16:48:45.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a46400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7a40156270 con 0x7f7a4006ee40 2024-05-30T16:48:45.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006f2c0 0x7f7a401f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:45.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006f2c0 0x7f7a401f5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49420/0 (socket says 172.21.15.67:49420) 2024-05-30T16:48:45.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a3fe00640 1 -- 172.21.15.67:0/1842132430 learned_addr learned my addr 172.21.15.67:0/1842132430 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:45.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a3fe00640 1 -- 172.21.15.67:0/1842132430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a4006ee40 msgr2=0x7f7a401f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:45.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a3fe00640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a4006ee40 0x7f7a401f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a3fe00640 1 -- 172.21.15.67:0/1842132430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a40156b30 msgr2=0x7f7a401f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a45a00640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a40156b30 0x7f7a401f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a3fe00640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a40156b30 0x7f7a401f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a3fe00640 1 -- 172.21.15.67:0/1842132430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a40159f60 con 0x7f7a4006f2c0 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.906+0000 7f7a45a00640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a40156b30 0x7f7a401f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.907+0000 7f7a3fe00640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006f2c0 0x7f7a401f5fe0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c000c00 tx=0x7f7a2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.907+0000 7f7a3cc00640 1 -- 172.21.15.67:0/1842132430 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c041020 con 0x7f7a4006f2c0 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.907+0000 7f7a3cc00640 1 -- 172.21.15.67:0/1842132430 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2c0306c0 con 0x7f7a4006f2c0 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.907+0000 7f7a3cc00640 1 -- 172.21.15.67:0/1842132430 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c037040 con 0x7f7a4006f2c0 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.907+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a40159c80 con 0x7f7a4006f2c0 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.907+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a40155f00 con 0x7f7a4006f2c0 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.908+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a400689e0 con 0x7f7a4006f2c0 2024-05-30T16:48:45.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.910+0000 7f7a3cc00640 1 -- 172.21.15.67:0/1842132430 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f7a2c044020 con 0x7f7a4006f2c0 2024-05-30T16:48:45.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.910+0000 7f7a3cc00640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7a2006cbf0 0x7f7a2006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:45.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.910+0000 7f7a3cc00640 1 -- 172.21.15.67:0/1842132430 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f7a2c0ac880 con 0x7f7a4006f2c0 2024-05-30T16:48:45.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.911+0000 7f7a45000640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7a2006cbf0 0x7f7a2006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:45.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.912+0000 7f7a45000640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7a2006cbf0 0x7f7a2006f0b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a34006fd0 tx=0x7f7a34008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:45.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:45.915+0000 7f7a3cc00640 1 -- 172.21.15.67:0/1842132430 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a2c04b030 con 0x7f7a4006f2c0 2024-05-30T16:48:46.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.155+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7a40158400 con 0x7f7a2006cbf0 2024-05-30T16:48:46.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.157+0000 7f7a3cc00640 1 -- 172.21.15.67:0/1842132430 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7a3400be20 con 0x7f7a2006cbf0 2024-05-30T16:48:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:48:46.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:48:46.159 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:48:46.159 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:48:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:48:46.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.160+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7a2006cbf0 msgr2=0x7f7a2006f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7a2006cbf0 0x7f7a2006f0b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a34006fd0 tx=0x7f7a34008040 comp rx=0 tx=0).stop 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006f2c0 msgr2=0x7f7a401f5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006f2c0 0x7f7a401f5fe0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c000c00 tx=0x7f7a2c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 shutdown_connections 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7a2006cbf0 0x7f7a2006f0b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a40156b30 0x7f7a401f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4006f2c0 0x7f7a401f5fe0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 --2- 172.21.15.67:0/1842132430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a4006ee40 0x7f7a401f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 >> 172.21.15.67:0/1842132430 conn(0x7f7a400b1fc0 msgr2=0x7f7a400b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 shutdown_connections 2024-05-30T16:48:46.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.161+0000 7f7a46400640 1 -- 172.21.15.67:0/1842132430 wait complete. 2024-05-30T16:48:46.181 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:48:46.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.397+0000 7f0068800640 1 Processor -- start 2024-05-30T16:48:46.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.398+0000 7f0068800640 1 -- start start 2024-05-30T16:48:46.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.399+0000 7f0068800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00600d0640 0x7f006015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.399+0000 7f0068800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f00600b32c0 0x7f00600b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.399+0000 7f0068800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00600d0060 con 0x7f00600d0a20 2024-05-30T16:48:46.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.399+0000 7f0068800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f00600adbb0 con 0x7f00600b32c0 2024-05-30T16:48:46.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.399+0000 7f0068800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f006015db10 con 0x7f00600d0640 2024-05-30T16:48:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.399+0000 7f0066800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f00600b32c0 0x7f00600b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.399+0000 7f0066800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f00600b32c0 0x7f00600b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49426/0 (socket says 172.21.15.67:49426) 2024-05-30T16:48:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.399+0000 7f0066800640 1 -- 172.21.15.67:0/3139346898 learned_addr learned my addr 172.21.15.67:0/3139346898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.400+0000 7f0065e00640 1 --2- 172.21.15.67:0/3139346898 >> v2:172.21.15.169:3300/0 conn(0x7f00600d0640 0x7f006015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.400+0000 7f0066800640 1 -- 172.21.15.67:0/3139346898 >> v2:172.21.15.169:3300/0 conn(0x7f00600d0640 msgr2=0x7f006015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.400+0000 7f0066800640 1 --2- 172.21.15.67:0/3139346898 >> v2:172.21.15.169:3300/0 conn(0x7f00600d0640 0x7f006015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.400+0000 7f0066800640 1 -- 172.21.15.67:0/3139346898 >> v1:172.21.15.169:6789/0 conn(0x7f00600d0a20 legacy=0x7f00600cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.400+0000 7f0066800640 1 -- 172.21.15.67:0/3139346898 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006015cb10 con 0x7f00600b32c0 2024-05-30T16:48:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.401+0000 7f0066800640 1 --2- 172.21.15.67:0/3139346898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600b32c0 0x7f00600b3730 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0060074150 tx=0x7f005c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a159e023a1afa775 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.401+0000 7f0064a00640 1 -- 172.21.15.67:0/3139346898 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c036070 con 0x7f00600b32c0 2024-05-30T16:48:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0064a00640 1 -- 172.21.15.67:0/3139346898 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005c031600 con 0x7f00600b32c0 2024-05-30T16:48:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0064a00640 1 -- 172.21.15.67:0/3139346898 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c02ae60 con 0x7f00600b32c0 2024-05-30T16:48:46.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0068800640 1 -- 172.21.15.67:0/3139346898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600b32c0 msgr2=0x7f00600b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0068800640 1 --2- 172.21.15.67:0/3139346898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600b32c0 0x7f00600b3730 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0060074150 tx=0x7f005c02a680 comp rx=0 tx=0).stop 2024-05-30T16:48:46.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0068800640 1 -- 172.21.15.67:0/3139346898 shutdown_connections 2024-05-30T16:48:46.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0068800640 1 --2- 172.21.15.67:0/3139346898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600b32c0 0x7f00600b3730 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0068800640 1 --2- 172.21.15.67:0/3139346898 >> v2:172.21.15.169:3300/0 conn(0x7f00600d0640 0x7f006015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0068800640 1 -- 172.21.15.67:0/3139346898 >> 172.21.15.67:0/3139346898 conn(0x7f006006bd20 msgr2=0x7f006006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:46.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0068800640 1 -- 172.21.15.67:0/3139346898 shutdown_connections 2024-05-30T16:48:46.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.402+0000 7f0068800640 1 -- 172.21.15.67:0/3139346898 wait complete. 2024-05-30T16:48:46.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.404+0000 7f0068800640 1 Processor -- start 2024-05-30T16:48:46.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.404+0000 7f0068800640 1 -- start start 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.405+0000 7f0068800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00600b32c0 0x7f0060150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.405+0000 7f0068800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00600d0640 0x7f006014b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.405+0000 7f0068800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600d0a20 0x7f0060147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.405+0000 7f0068800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f006015cdd0 con 0x7f00600d0a20 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.405+0000 7f0068800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00600ad8d0 con 0x7f00600b32c0 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.405+0000 7f0068800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f006015d0b0 con 0x7f00600d0640 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.405+0000 7f0066800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600d0a20 0x7f0060147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.405+0000 7f0066800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600d0a20 0x7f0060147bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49430/0 (socket says 172.21.15.67:49430) 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.405+0000 7f0066800640 1 -- 172.21.15.67:0/72176000 learned_addr learned my addr 172.21.15.67:0/72176000 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.406+0000 7f0065400640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00600d0640 0x7f006014b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.406+0000 7f0065e00640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00600b32c0 0x7f0060150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.406+0000 7f0066800640 1 -- 172.21.15.67:0/72176000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00600d0640 msgr2=0x7f006014b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.406+0000 7f0066800640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00600d0640 0x7f006014b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.406+0000 7f0066800640 1 -- 172.21.15.67:0/72176000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00600b32c0 msgr2=0x7f0060150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.406+0000 7f0066800640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00600b32c0 0x7f0060150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.406+0000 7f0066800640 1 -- 172.21.15.67:0/72176000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00601480f0 con 0x7f00600d0a20 2024-05-30T16:48:46.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.406+0000 7f0065400640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00600d0640 0x7f006014b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:48:46.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.406+0000 7f0066800640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600d0a20 0x7f0060147bb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f005c009e50 tx=0x7f005c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:46.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.407+0000 7f005a000640 1 -- 172.21.15.67:0/72176000 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c036050 con 0x7f00600d0a20 2024-05-30T16:48:46.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.407+0000 7f005a000640 1 -- 172.21.15.67:0/72176000 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005c042070 con 0x7f00600d0a20 2024-05-30T16:48:46.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.407+0000 7f005a000640 1 -- 172.21.15.67:0/72176000 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c009510 con 0x7f00600d0a20 2024-05-30T16:48:46.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.407+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006014cb70 con 0x7f00600d0a20 2024-05-30T16:48:46.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.407+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006014c2d0 con 0x7f00600d0a20 2024-05-30T16:48:46.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.408+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00600689e0 con 0x7f00600d0a20 2024-05-30T16:48:46.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.409+0000 7f005a000640 1 -- 172.21.15.67:0/72176000 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f005c007910 con 0x7f00600d0a20 2024-05-30T16:48:46.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.410+0000 7f005a000640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f004406cbc0 0x7f004406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.411+0000 7f005a000640 1 -- 172.21.15.67:0/72176000 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f005c0acdb0 con 0x7f00600d0a20 2024-05-30T16:48:46.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.411+0000 7f0065e00640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f004406cbc0 0x7f004406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.411+0000 7f0065e00640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f004406cbc0 0x7f004406f080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0054006fd0 tx=0x7f0054008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:46.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.415+0000 7f005a000640 1 -- 172.21.15.67:0/72176000 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f005c0b1070 con 0x7f00600d0a20 2024-05-30T16:48:46.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.628+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f006006c230 con 0x7f00600d0a20 2024-05-30T16:48:46.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.629+0000 7f005a000640 1 -- 172.21.15.67:0/72176000 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f005c07ad40 con 0x7f00600d0a20 2024-05-30T16:48:46.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f004406cbc0 msgr2=0x7f004406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f004406cbc0 0x7f004406f080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0054006fd0 tx=0x7f0054008040 comp rx=0 tx=0).stop 2024-05-30T16:48:46.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600d0a20 msgr2=0x7f0060147bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600d0a20 0x7f0060147bb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f005c009e50 tx=0x7f005c004a40 comp rx=0 tx=0).stop 2024-05-30T16:48:46.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 shutdown_connections 2024-05-30T16:48:46.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f004406cbc0 0x7f004406f080 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00600d0a20 0x7f0060147bb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00600d0640 0x7f006014b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 --2- 172.21.15.67:0/72176000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00600b32c0 0x7f0060150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 >> 172.21.15.67:0/72176000 conn(0x7f006006bd20 msgr2=0x7f006015c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:46.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 shutdown_connections 2024-05-30T16:48:46.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.632+0000 7f0068800640 1 -- 172.21.15.67:0/72176000 wait complete. 2024-05-30T16:48:46.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-30T16:48:46.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.864+0000 7f0bc3400640 1 Processor -- start 2024-05-30T16:48:46.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.865+0000 7f0bc3400640 1 -- start start 2024-05-30T16:48:46.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.866+0000 7f0bc3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0bbc09aeb0 0x7f0bbc15e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.866+0000 7f0bc3400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bbc09b360 con 0x7f0bbc09aa30 2024-05-30T16:48:46.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.866+0000 7f0bc3400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bbc06cc60 con 0x7f0bbc0b3990 2024-05-30T16:48:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.866+0000 7f0bc3400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f0bbc06ca10 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.867+0000 7f0bbbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0bbc09aa30 0x7f0bbc15e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56610/0 (socket says 172.21.15.67:56610) 2024-05-30T16:48:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.867+0000 7f0bbbe00640 1 -- 172.21.15.67:0/319527682 learned_addr learned my addr 172.21.15.67:0/319527682 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:48:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.867+0000 7f0bc1400640 1 --2- 172.21.15.67:0/319527682 >> v2:172.21.15.67:3300/0 conn(0x7f0bbc09aeb0 0x7f0bbc15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.867+0000 7f0bbb400640 1 -- 172.21.15.67:0/319527682 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4183055159 0 0) 0x7f0bbc09b360 con 0x7f0bbc09aa30 2024-05-30T16:48:46.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.867+0000 7f0bbb400640 1 -- 172.21.15.67:0/319527682 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ba4003520 con 0x7f0bbc09aa30 2024-05-30T16:48:46.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.867+0000 7f0bc1400640 1 -- 172.21.15.67:0/319527682 >> v1:172.21.15.169:6789/0 conn(0x7f0bbc0b3990 legacy=0x7f0bbc0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:46.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.867+0000 7f0bc1400640 1 -- 172.21.15.67:0/319527682 >> v1:172.21.15.67:6789/0 conn(0x7f0bbc09aa30 legacy=0x7f0bbc15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.867+0000 7f0bc1400640 1 -- 172.21.15.67:0/319527682 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc06cf40 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.868+0000 7f0bc1400640 1 --2- 172.21.15.67:0/319527682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 0x7f0bbc15e860 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009d90 tx=0x7f0bb402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59996d2569351fd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:46.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.869+0000 7f0bbb400640 1 -- 172.21.15.67:0/319527682 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4031600 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.869+0000 7f0bbb400640 1 -- 172.21.15.67:0/319527682 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb40317a0 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.869+0000 7f0bbb400640 1 -- 172.21.15.67:0/319527682 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4031b70 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.869+0000 7f0bc3400640 1 -- 172.21.15.67:0/319527682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 msgr2=0x7f0bbc15e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.869+0000 7f0bc3400640 1 --2- 172.21.15.67:0/319527682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 0x7f0bbc15e860 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009d90 tx=0x7f0bb402a610 comp rx=0 tx=0).stop 2024-05-30T16:48:46.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.869+0000 7f0bc3400640 1 -- 172.21.15.67:0/319527682 shutdown_connections 2024-05-30T16:48:46.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.869+0000 7f0bc3400640 1 --2- 172.21.15.67:0/319527682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 0x7f0bbc15e860 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.869+0000 7f0bc3400640 1 -- 172.21.15.67:0/319527682 >> 172.21.15.67:0/319527682 conn(0x7f0bbc0b3fd0 msgr2=0x7f0bbc0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:46.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.870+0000 7f0bc3400640 1 -- 172.21.15.67:0/319527682 shutdown_connections 2024-05-30T16:48:46.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.870+0000 7f0bc3400640 1 -- 172.21.15.67:0/319527682 wait complete. 2024-05-30T16:48:46.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.871+0000 7f0bc3400640 1 Processor -- start 2024-05-30T16:48:46.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.872+0000 7f0bc3400640 1 -- start start 2024-05-30T16:48:46.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.872+0000 7f0bc3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bbc09aa30 0x7f0bbc1f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bc3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 0x7f0bbc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bc3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc0b3990 0x7f0bbc158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bc3400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc06d500 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bc3400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc06d870 con 0x7f0bbc0b3990 2024-05-30T16:48:46.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bc3400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0bbc0b2e50 con 0x7f0bbc09aa30 2024-05-30T16:48:46.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bbbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 0x7f0bbc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bbbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 0x7f0bbc1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49446/0 (socket says 172.21.15.67:49446) 2024-05-30T16:48:46.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bbbe00640 1 -- 172.21.15.67:0/1233106994 learned_addr learned my addr 172.21.15.67:0/1233106994 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:46.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bc0a00640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bbc09aa30 0x7f0bbc1f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bc1400640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc0b3990 0x7f0bbc158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bbbe00640 1 -- 172.21.15.67:0/1233106994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bbc09aa30 msgr2=0x7f0bbc1f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bbbe00640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bbc09aa30 0x7f0bbc1f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bbbe00640 1 -- 172.21.15.67:0/1233106994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc0b3990 msgr2=0x7f0bbc158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:46.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bbbe00640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc0b3990 0x7f0bbc158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:46.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.873+0000 7f0bbbe00640 1 -- 172.21.15.67:0/1233106994 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc1f5830 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.874+0000 7f0bbbe00640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 0x7f0bbc1587d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8001eb0 tx=0x7f0ba800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:46.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.874+0000 7f0bb8c00640 1 -- 172.21.15.67:0/1233106994 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8017070 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.874+0000 7f0bb8c00640 1 -- 172.21.15.67:0/1233106994 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba8005d00 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.874+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc15b500 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.874+0000 7f0bb8c00640 1 -- 172.21.15.67:0/1233106994 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8007310 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.874+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc0c68c0 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.875+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc0690f0 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.877+0000 7f0bb8c00640 1 -- 172.21.15.67:0/1233106994 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7f0ba801a070 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.878+0000 7f0bb8c00640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9c06cbc0 0x7f0b9c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:46.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.878+0000 7f0bb8c00640 1 -- 172.21.15.67:0/1233106994 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f0ba808b3a0 con 0x7f0bbc09aeb0 2024-05-30T16:48:46.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.878+0000 7f0bc0a00640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9c06cbc0 0x7f0b9c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:46.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.879+0000 7f0bc0a00640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9c06cbc0 0x7f0b9c06f080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006fd0 tx=0x7f0bb0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:46.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:46.883+0000 7f0bb8c00640 1 -- 172.21.15.67:0/1233106994 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ba80594d0 con 0x7f0bbc09aeb0 2024-05-30T16:48:47.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.118+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f0bbc0b1380 con 0x7f0b9c06cbc0 2024-05-30T16:48:47.126 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:47.126+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:47.127 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:47.126+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:47.127 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:47.126+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:47.127 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:47.126+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:47.127 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:48:47.126+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:48:47.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.173+0000 7f0bb8c00640 1 -- 172.21.15.67:0/1233106994 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0bbc0b1380 con 0x7f0b9c06cbc0 2024-05-30T16:48:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9c06cbc0 msgr2=0x7f0b9c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9c06cbc0 0x7f0b9c06f080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006fd0 tx=0x7f0bb0008040 comp rx=0 tx=0).stop 2024-05-30T16:48:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 msgr2=0x7f0bbc1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 0x7f0bbc1587d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8001eb0 tx=0x7f0ba800c820 comp rx=0 tx=0).stop 2024-05-30T16:48:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 shutdown_connections 2024-05-30T16:48:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9c06cbc0 0x7f0b9c06f080 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc0b3990 0x7f0bbc158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bbc09aeb0 0x7f0bbc1587d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 --2- 172.21.15.67:0/1233106994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bbc09aa30 0x7f0bbc1f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 >> 172.21.15.67:0/1233106994 conn(0x7f0bbc0b3fd0 msgr2=0x7f0bbc0b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 shutdown_connections 2024-05-30T16:48:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.177+0000 7f0bc3400640 1 -- 172.21.15.67:0/1233106994 wait complete. 2024-05-30T16:48:47.199 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:48:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.394+0000 7fad3be00640 1 Processor -- start 2024-05-30T16:48:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.394+0000 7fad3be00640 1 -- start start 2024-05-30T16:48:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.396+0000 7fad3be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c15e870 0x7fad3c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.396+0000 7fad3be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad3c0cfa40 0x7fad3c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.396+0000 7fad3be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3c15ec50 con 0x7fad3c0d14d0 2024-05-30T16:48:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.396+0000 7fad3be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fad3c0d2150 con 0x7fad3c0cfa40 2024-05-30T16:48:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.396+0000 7fad3be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad3c0cff80 con 0x7fad3c15e870 2024-05-30T16:48:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.397+0000 7fad3b400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad3c0cfa40 0x7fad3c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.397+0000 7fad3aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c15e870 0x7fad3c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.397+0000 7fad3b400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad3c0cfa40 0x7fad3c0d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:60172/0 (socket says 172.21.15.67:60172) 2024-05-30T16:48:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.397+0000 7fad3aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c15e870 0x7fad3c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:37558/0 (socket says 172.21.15.67:37558) 2024-05-30T16:48:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.397+0000 7fad3b400640 1 -- 172.21.15.67:0/4290240918 learned_addr learned my addr 172.21.15.67:0/4290240918 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.398+0000 7fad3aa00640 1 -- 172.21.15.67:0/4290240918 >> v2:172.21.15.169:3300/0 conn(0x7fad3c0cfa40 msgr2=0x7fad3c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:48:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.398+0000 7fad3aa00640 1 --2- 172.21.15.67:0/4290240918 >> v2:172.21.15.169:3300/0 conn(0x7fad3c0cfa40 0x7fad3c0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.398+0000 7fad3aa00640 1 -- 172.21.15.67:0/4290240918 >> v1:172.21.15.169:6789/0 conn(0x7fad3c0d14d0 legacy=0x7fad3c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.398+0000 7fad3aa00640 1 -- 172.21.15.67:0/4290240918 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c0d0260 con 0x7fad3c15e870 2024-05-30T16:48:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.399+0000 7fad3aa00640 1 --2- 172.21.15.67:0/4290240918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c15e870 0x7fad3c0d0f90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0097a0 tx=0x7fad2c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe1c0ac54da20461 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:47.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.399+0000 7fad39600640 1 -- 172.21.15.67:0/4290240918 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c037070 con 0x7fad3c15e870 2024-05-30T16:48:47.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.399+0000 7fad39600640 1 -- 172.21.15.67:0/4290240918 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2c032440 con 0x7fad3c15e870 2024-05-30T16:48:47.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.399+0000 7fad39600640 1 -- 172.21.15.67:0/4290240918 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c032810 con 0x7fad3c15e870 2024-05-30T16:48:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.400+0000 7fad3be00640 1 -- 172.21.15.67:0/4290240918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c15e870 msgr2=0x7fad3c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.400+0000 7fad3be00640 1 --2- 172.21.15.67:0/4290240918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c15e870 0x7fad3c0d0f90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0097a0 tx=0x7fad2c02a180 comp rx=0 tx=0).stop 2024-05-30T16:48:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.400+0000 7fad3be00640 1 -- 172.21.15.67:0/4290240918 shutdown_connections 2024-05-30T16:48:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.400+0000 7fad3be00640 1 --2- 172.21.15.67:0/4290240918 >> v2:172.21.15.169:3300/0 conn(0x7fad3c0cfa40 0x7fad3c0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.400+0000 7fad3be00640 1 --2- 172.21.15.67:0/4290240918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c15e870 0x7fad3c0d0f90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.400+0000 7fad3be00640 1 -- 172.21.15.67:0/4290240918 >> 172.21.15.67:0/4290240918 conn(0x7fad3c08bfc0 msgr2=0x7fad3c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.400+0000 7fad3be00640 1 -- 172.21.15.67:0/4290240918 shutdown_connections 2024-05-30T16:48:47.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.400+0000 7fad3be00640 1 -- 172.21.15.67:0/4290240918 wait complete. 2024-05-30T16:48:47.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.402+0000 7fad3be00640 1 Processor -- start 2024-05-30T16:48:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.402+0000 7fad3be00640 1 -- start start 2024-05-30T16:48:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.403+0000 7fad3be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c0cfa40 0x7fad3c156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.403+0000 7fad3be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3c0d14d0 0x7fad3c1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.403+0000 7fad3be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad3c15e870 0x7fad3c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.403+0000 7fad3be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c0d4fb0 con 0x7fad3c15e870 2024-05-30T16:48:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.403+0000 7fad3be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c0d0820 con 0x7fad3c0d14d0 2024-05-30T16:48:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.403+0000 7fad3be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad3c0d2b20 con 0x7fad3c0cfa40 2024-05-30T16:48:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad3c15e870 0x7fad3c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad3c15e870 0x7fad3c0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49474/0 (socket says 172.21.15.67:49474) 2024-05-30T16:48:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3b400640 1 -- 172.21.15.67:0/1255075081 learned_addr learned my addr 172.21.15.67:0/1255075081 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:48:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3a000640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3c0d14d0 0x7fad3c1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3aa00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c0cfa40 0x7fad3c156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3b400640 1 -- 172.21.15.67:0/1255075081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c0cfa40 msgr2=0x7fad3c156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3b400640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c0cfa40 0x7fad3c156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3b400640 1 -- 172.21.15.67:0/1255075081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3c0d14d0 msgr2=0x7fad3c1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3b400640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3c0d14d0 0x7fad3c1600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.404+0000 7fad3b400640 1 -- 172.21.15.67:0/1255075081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c1f7ad0 con 0x7fad3c15e870 2024-05-30T16:48:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.405+0000 7fad3b400640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad3c15e870 0x7fad3c0cd820 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fad3400efc0 tx=0x7fad3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.405+0000 7fad32a00640 1 -- 172.21.15.67:0/1255075081 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3400cd20 con 0x7fad3c15e870 2024-05-30T16:48:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.405+0000 7fad32a00640 1 -- 172.21.15.67:0/1255075081 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad34003990 con 0x7fad3c15e870 2024-05-30T16:48:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.405+0000 7fad32a00640 1 -- 172.21.15.67:0/1255075081 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34003d80 con 0x7fad3c15e870 2024-05-30T16:48:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.405+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3c1f7d70 con 0x7fad3c15e870 2024-05-30T16:48:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.405+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3c1f8330 con 0x7fad3c15e870 2024-05-30T16:48:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.406+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad3c0689e0 con 0x7fad3c15e870 2024-05-30T16:48:47.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.407+0000 7fad32a00640 1 -- 172.21.15.67:0/1255075081 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fad34005d10 con 0x7fad3c15e870 2024-05-30T16:48:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.408+0000 7fad32a00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad1c06cbc0 0x7fad1c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:48:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.408+0000 7fad32a00640 1 -- 172.21.15.67:0/1255075081 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fad3408c5f0 con 0x7fad3c15e870 2024-05-30T16:48:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.409+0000 7fad3aa00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad1c06cbc0 0x7fad1c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:48:47.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.409+0000 7fad3aa00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad1c06cbc0 0x7fad1c06f080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad2c034000 tx=0x7fad2c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:48:47.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.413+0000 7fad32a00640 1 -- 172.21.15.67:0/1255075081 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad34022020 con 0x7fad3c15e870 2024-05-30T16:48:47.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.648+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fad3c0d3f00 con 0x7fad1c06cbc0 2024-05-30T16:48:47.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.658+0000 7fad32a00640 1 -- 172.21.15.67:0/1255075081 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fad3c0d3f00 con 0x7fad1c06cbc0 2024-05-30T16:48:47.659 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded 2024-05-30T16:48:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad1c06cbc0 msgr2=0x7fad1c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad1c06cbc0 0x7fad1c06f080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad2c034000 tx=0x7fad2c02e000 comp rx=0 tx=0).stop 2024-05-30T16:48:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad3c15e870 msgr2=0x7fad3c0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:48:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad3c15e870 0x7fad3c0cd820 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fad3400efc0 tx=0x7fad3400c490 comp rx=0 tx=0).stop 2024-05-30T16:48:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 shutdown_connections 2024-05-30T16:48:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad1c06cbc0 0x7fad1c06f080 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad3c15e870 0x7fad3c0cd820 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3c0d14d0 0x7fad3c1600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 --2- 172.21.15.67:0/1255075081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3c0cfa40 0x7fad3c156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:48:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.661+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 >> 172.21.15.67:0/1255075081 conn(0x7fad3c08bfc0 msgr2=0x7fad3c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:48:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.662+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 shutdown_connections 2024-05-30T16:48:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:48:47.662+0000 7fad3be00640 1 -- 172.21.15.67:0/1255075081 wait complete. 2024-05-30T16:48:47.683 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded 2024-05-30T16:48:47.683 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:48:47.683 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:48:47.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:48:47.736 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:49:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:49:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:49:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 17.3948 s, 61.7 MB/s 2024-05-30T16:49:05.210 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:49:05.210 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:49:05.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:49:05.265 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:49:10.063 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:49:10.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:49:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:49:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:49:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.6569 s, 78.6 MB/s 2024-05-30T16:49:18.981 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:49:18.981 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:49:19.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:49:19.039 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:49:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:49:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:49:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.6326 s, 73.4 MB/s 2024-05-30T16:49:33.738 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:49:33.739 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:49:33.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:49:33.793 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:49:40.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:49:40.203 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:49:48.224 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:49:48.224 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:49:48.225 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.3739 s, 74.7 MB/s 2024-05-30T16:49:48.226 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:49:48.227 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:49:48.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:49:48.283 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:50:02.467 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:50:02.467 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:50:02.467 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.1205 s, 76.0 MB/s 2024-05-30T16:50:02.468 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:50:02.468 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:50:02.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:50:02.523 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:50:10.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:50:10.320 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:50:17.162 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:50:17.162 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:50:17.162 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.5856 s, 73.6 MB/s 2024-05-30T16:50:17.164 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:50:17.164 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:50:17.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:50:17.216 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:50:31.676 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:50:31.676 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:50:31.676 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.3546 s, 74.8 MB/s 2024-05-30T16:50:31.678 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:50:31.678 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:50:31.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:50:31.735 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:50:40.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:50:40.450 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:50:45.631 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:50:45.631 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:50:45.632 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.8374 s, 77.6 MB/s 2024-05-30T16:50:45.633 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:50:45.634 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:50:45.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:50:45.690 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:50:59.875 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:50:59.875 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:50:59.875 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.1258 s, 76.0 MB/s 2024-05-30T16:50:59.877 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:50:59.877 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:50:59.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:50:59.935 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:51:10.918 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:51:10.927 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:51:14.007 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:51:14.007 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:51:14.007 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.012 s, 76.6 MB/s 2024-05-30T16:51:14.009 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-30T16:51:14.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.221+0000 7f0885e00640 1 Processor -- start 2024-05-30T16:51:14.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.221+0000 7f0885e00640 1 -- start start 2024-05-30T16:51:14.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.223+0000 7f0885e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f08800af6f0 0x7f08801623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.223+0000 7f0885e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08800d0b40 0x7f08801428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.223+0000 7f0885e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0880162cb0 con 0x7f08801480a0 2024-05-30T16:51:14.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.223+0000 7f0885e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f0880162f90 con 0x7f08800af6f0 2024-05-30T16:51:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.223+0000 7f0885e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0880163270 con 0x7f08800d0b40 2024-05-30T16:51:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.223+0000 7f0884a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f08800af6f0 0x7f08801623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.223+0000 7f0884a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f08800af6f0 0x7f08801623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52120/0 (socket says 172.21.15.67:52120) 2024-05-30T16:51:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.223+0000 7f0884a00640 1 -- 172.21.15.67:0/1244885810 learned_addr learned my addr 172.21.15.67:0/1244885810 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.224+0000 7f0884a00640 1 -- 172.21.15.67:0/1244885810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08800d0b40 msgr2=0x7f08801428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.224+0000 7f0884a00640 1 --2- 172.21.15.67:0/1244885810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08800d0b40 0x7f08801428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.224+0000 7f0884a00640 1 -- 172.21.15.67:0/1244885810 >> v1:172.21.15.169:6789/0 conn(0x7f08801480a0 legacy=0x7f08800d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.224+0000 7f0884a00640 1 -- 172.21.15.67:0/1244885810 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08801420d0 con 0x7f08800af6f0 2024-05-30T16:51:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.225+0000 7f0884a00640 1 --2- 172.21.15.67:0/1244885810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08800af6f0 0x7f08801623b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f08800d10c0 tx=0x7f087402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4d6f04a23269a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.225+0000 7f087f400640 1 -- 172.21.15.67:0/1244885810 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874037070 con 0x7f08800af6f0 2024-05-30T16:51:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.225+0000 7f087f400640 1 -- 172.21.15.67:0/1244885810 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0874032440 con 0x7f08800af6f0 2024-05-30T16:51:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.225+0000 7f087f400640 1 -- 172.21.15.67:0/1244885810 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874032810 con 0x7f08800af6f0 2024-05-30T16:51:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.226+0000 7f0885e00640 1 -- 172.21.15.67:0/1244885810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08800af6f0 msgr2=0x7f08801623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.226+0000 7f0885e00640 1 --2- 172.21.15.67:0/1244885810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08800af6f0 0x7f08801623b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f08800d10c0 tx=0x7f087402a200 comp rx=0 tx=0).stop 2024-05-30T16:51:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.226+0000 7f0885e00640 1 -- 172.21.15.67:0/1244885810 shutdown_connections 2024-05-30T16:51:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.226+0000 7f0885e00640 1 --2- 172.21.15.67:0/1244885810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08800d0b40 0x7f08801428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.226+0000 7f0885e00640 1 --2- 172.21.15.67:0/1244885810 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08800af6f0 0x7f08801623b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.226+0000 7f0885e00640 1 -- 172.21.15.67:0/1244885810 >> 172.21.15.67:0/1244885810 conn(0x7f088006bae0 msgr2=0x7f088006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.226+0000 7f0885e00640 1 -- 172.21.15.67:0/1244885810 shutdown_connections 2024-05-30T16:51:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.226+0000 7f0885e00640 1 -- 172.21.15.67:0/1244885810 wait complete. 2024-05-30T16:51:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.228+0000 7f0885e00640 1 Processor -- start 2024-05-30T16:51:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.228+0000 7f0885e00640 1 -- start start 2024-05-30T16:51:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.229+0000 7f0885e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08800af6f0 0x7f08801f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.229+0000 7f0885e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08800d0b40 0x7f08801e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.229+0000 7f0885e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08801480a0 0x7f08801e8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.229+0000 7f0885e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0880162a70 con 0x7f08801480a0 2024-05-30T16:51:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.229+0000 7f0885e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08801628f0 con 0x7f08800af6f0 2024-05-30T16:51:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.229+0000 7f0885e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08800afad0 con 0x7f08800d0b40 2024-05-30T16:51:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.229+0000 7f0885400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08801480a0 0x7f08801e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.229+0000 7f0885400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08801480a0 0x7f08801e8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52122/0 (socket says 172.21.15.67:52122) 2024-05-30T16:51:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.229+0000 7f0885400640 1 -- 172.21.15.67:0/535617592 learned_addr learned my addr 172.21.15.67:0/535617592 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.230+0000 7f0884a00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08800af6f0 0x7f08801f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.230+0000 7f0885400640 1 -- 172.21.15.67:0/535617592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08800d0b40 msgr2=0x7f08801e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.230+0000 7f0885400640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08800d0b40 0x7f08801e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.230+0000 7f0885400640 1 -- 172.21.15.67:0/535617592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08800af6f0 msgr2=0x7f08801f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.230+0000 7f0885400640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08800af6f0 0x7f08801f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.230+0000 7f0885400640 1 -- 172.21.15.67:0/535617592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0880072af0 con 0x7f08801480a0 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.230+0000 7f0884a00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08800af6f0 0x7f08801f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.230+0000 7f0885400640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08801480a0 0x7f08801e8700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f087800b9e0 tx=0x7f087800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.230+0000 7f087cc00640 1 -- 172.21.15.67:0/535617592 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878019070 con 0x7f08801480a0 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.231+0000 7f087cc00640 1 -- 172.21.15.67:0/535617592 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087800cb40 con 0x7f08801480a0 2024-05-30T16:51:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.231+0000 7f087cc00640 1 -- 172.21.15.67:0/535617592 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878014440 con 0x7f08801480a0 2024-05-30T16:51:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.231+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0880072dd0 con 0x7f08801480a0 2024-05-30T16:51:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.231+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0880073390 con 0x7f08801480a0 2024-05-30T16:51:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.232+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08800689e0 con 0x7f08801480a0 2024-05-30T16:51:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.233+0000 7f087cc00640 1 -- 172.21.15.67:0/535617592 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) ==== 91295+0+0 (secure 0 0 0) 0x7f087801c070 con 0x7f08801480a0 2024-05-30T16:51:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.234+0000 7f087cc00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086006cbc0 0x7f086006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.234+0000 7f087cc00640 1 -- 172.21.15.67:0/535617592 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f087808be70 con 0x7f08801480a0 2024-05-30T16:51:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.235+0000 7f0884a00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086006cbc0 0x7f086006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.235+0000 7f0884a00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086006cbc0 0x7f086006f080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0874034000 tx=0x7f087402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:14.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.239+0000 7f087cc00640 1 -- 172.21.15.67:0/535617592 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0878059df0 con 0x7f08801480a0 2024-05-30T16:51:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.474+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f088006cb40 con 0x7f086006cbc0 2024-05-30T16:51:14.485 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:14.484+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:14.485 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:14.484+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:14.485 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:14.484+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:14.485 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:14.484+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:14.485 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:14.484+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.528+0000 7f087cc00640 1 -- 172.21.15.67:0/535617592 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f088006cb40 con 0x7f086006cbc0 2024-05-30T16:51:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086006cbc0 msgr2=0x7f086006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086006cbc0 0x7f086006f080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0874034000 tx=0x7f087402e000 comp rx=0 tx=0).stop 2024-05-30T16:51:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08801480a0 msgr2=0x7f08801e8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08801480a0 0x7f08801e8700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f087800b9e0 tx=0x7f087800bea0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 shutdown_connections 2024-05-30T16:51:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086006cbc0 0x7f086006f080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08801480a0 0x7f08801e8700 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08800d0b40 0x7f08801e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 --2- 172.21.15.67:0/535617592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08800af6f0 0x7f08801f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 >> 172.21.15.67:0/535617592 conn(0x7f088006bae0 msgr2=0x7f08800d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 shutdown_connections 2024-05-30T16:51:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.531+0000 7f0885e00640 1 -- 172.21.15.67:0/535617592 wait complete. 2024-05-30T16:51:14.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:51:14.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.757+0000 7f15d3000640 1 Processor -- start 2024-05-30T16:51:14.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.758+0000 7f15d3000640 1 -- start start 2024-05-30T16:51:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.759+0000 7f15d3000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f15cc0d16a0 0x7f15cc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.759+0000 7f15d3000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15cc0cf870 0x7f15cc0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.759+0000 7f15d3000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15cc0cfce0 con 0x7f15cc15e870 2024-05-30T16:51:14.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.759+0000 7f15d3000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f15cc0d2150 con 0x7f15cc0d16a0 2024-05-30T16:51:14.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.759+0000 7f15d3000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f15cc15ec50 con 0x7f15cc0cf870 2024-05-30T16:51:14.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.760+0000 7f15d1c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f15cc15e870 0x7f15cc0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58064/0 (socket says 172.21.15.67:58064) 2024-05-30T16:51:14.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.760+0000 7f15d1c00640 1 -- 172.21.15.67:0/1354261293 learned_addr learned my addr 172.21.15.67:0/1354261293 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:51:14.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.760+0000 7f15d1200640 1 --2- 172.21.15.67:0/1354261293 >> v2:172.21.15.67:3300/0 conn(0x7f15cc0d16a0 0x7f15cc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.760+0000 7f15d2600640 1 --2- 172.21.15.67:0/1354261293 >> v2:172.21.15.169:3300/0 conn(0x7f15cc0cf870 0x7f15cc0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.760+0000 7f15d1200640 1 -- 172.21.15.67:0/1354261293 >> v2:172.21.15.169:3300/0 conn(0x7f15cc0cf870 msgr2=0x7f15cc0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:14.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.760+0000 7f15d1200640 1 --2- 172.21.15.67:0/1354261293 >> v2:172.21.15.169:3300/0 conn(0x7f15cc0cf870 0x7f15cc0d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.760+0000 7f15d1200640 1 -- 172.21.15.67:0/1354261293 >> v1:172.21.15.67:6789/0 conn(0x7f15cc15e870 legacy=0x7f15cc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:14.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.760+0000 7f15cbe00640 1 -- 172.21.15.67:0/1354261293 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576589278 0 0) 0x7f15cc0cfce0 con 0x7f15cc15e870 2024-05-30T16:51:14.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.760+0000 7f15d1200640 1 -- 172.21.15.67:0/1354261293 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc0cff80 con 0x7f15cc0d16a0 2024-05-30T16:51:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.761+0000 7f15d1200640 1 --2- 172.21.15.67:0/1354261293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 0x7f15cc0cf330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f15cc163400 tx=0x7f15b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5577ef49941f2603 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.762+0000 7f15cbe00640 1 -- 172.21.15.67:0/1354261293 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8037070 con 0x7f15cc0d16a0 2024-05-30T16:51:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.762+0000 7f15cbe00640 1 -- 172.21.15.67:0/1354261293 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b8032440 con 0x7f15cc0d16a0 2024-05-30T16:51:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.762+0000 7f15cbe00640 1 -- 172.21.15.67:0/1354261293 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8032810 con 0x7f15cc0d16a0 2024-05-30T16:51:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.762+0000 7f15d3000640 1 -- 172.21.15.67:0/1354261293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 msgr2=0x7f15cc0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.762+0000 7f15d3000640 1 --2- 172.21.15.67:0/1354261293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 0x7f15cc0cf330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f15cc163400 tx=0x7f15b802a1a0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.763+0000 7f15d3000640 1 -- 172.21.15.67:0/1354261293 shutdown_connections 2024-05-30T16:51:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.763+0000 7f15d3000640 1 --2- 172.21.15.67:0/1354261293 >> v2:172.21.15.169:3300/0 conn(0x7f15cc0cf870 0x7f15cc0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.763+0000 7f15d3000640 1 --2- 172.21.15.67:0/1354261293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 0x7f15cc0cf330 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.763+0000 7f15d3000640 1 -- 172.21.15.67:0/1354261293 >> 172.21.15.67:0/1354261293 conn(0x7f15cc08bfc0 msgr2=0x7f15cc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.763+0000 7f15d3000640 1 -- 172.21.15.67:0/1354261293 shutdown_connections 2024-05-30T16:51:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.763+0000 7f15d3000640 1 -- 172.21.15.67:0/1354261293 wait complete. 2024-05-30T16:51:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.764+0000 7f15d3000640 1 Processor -- start 2024-05-30T16:51:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.765+0000 7f15d3000640 1 -- start start 2024-05-30T16:51:14.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.765+0000 7f15d3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15cc0cf870 0x7f15cc159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.765+0000 7f15d3000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 0x7f15cc15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.765+0000 7f15d3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15cc15e870 0x7f15cc0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.765+0000 7f15d3000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc0d0820 con 0x7f15cc0d16a0 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.765+0000 7f15d3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc0d0540 con 0x7f15cc0cf870 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.765+0000 7f15d3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15cc0d4fb0 con 0x7f15cc15e870 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 0x7f15cc15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 0x7f15cc15cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52150/0 (socket says 172.21.15.67:52150) 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d1200640 1 -- 172.21.15.67:0/1748768313 learned_addr learned my addr 172.21.15.67:0/1748768313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d1c00640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15cc0cf870 0x7f15cc159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d2600640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15cc15e870 0x7f15cc0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d1200640 1 -- 172.21.15.67:0/1748768313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15cc15e870 msgr2=0x7f15cc0c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d1200640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15cc15e870 0x7f15cc0c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d1200640 1 -- 172.21.15.67:0/1748768313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15cc0cf870 msgr2=0x7f15cc159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d1200640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15cc0cf870 0x7f15cc159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d1200640 1 -- 172.21.15.67:0/1748768313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc15a690 con 0x7f15cc0d16a0 2024-05-30T16:51:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.766+0000 7f15d2600640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15cc15e870 0x7f15cc0c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:51:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.767+0000 7f15d1200640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 0x7f15cc15cbc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.767+0000 7f15c9600640 1 -- 172.21.15.67:0/1748768313 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8037040 con 0x7f15cc0d16a0 2024-05-30T16:51:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.767+0000 7f15c9600640 1 -- 172.21.15.67:0/1748768313 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b8042070 con 0x7f15cc0d16a0 2024-05-30T16:51:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.767+0000 7f15c9600640 1 -- 172.21.15.67:0/1748768313 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8030a50 con 0x7f15cc0d16a0 2024-05-30T16:51:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.767+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc15a970 con 0x7f15cc0d16a0 2024-05-30T16:51:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.767+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc15af30 con 0x7f15cc0d16a0 2024-05-30T16:51:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.769+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15cc0689e0 con 0x7f15cc0d16a0 2024-05-30T16:51:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.770+0000 7f15c9600640 1 -- 172.21.15.67:0/1748768313 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) ==== 91295+0+0 (secure 0 0 0) 0x7f15b8030bf0 con 0x7f15cc0d16a0 2024-05-30T16:51:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.770+0000 7f15c9600640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15ac06cbc0 0x7f15ac06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.771+0000 7f15c9600640 1 -- 172.21.15.67:0/1748768313 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f15b80ac9e0 con 0x7f15cc0d16a0 2024-05-30T16:51:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.771+0000 7f15d1c00640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15ac06cbc0 0x7f15ac06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.772+0000 7f15d1c00640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15ac06cbc0 0x7f15ac06f080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15c0000f60 tx=0x7f15c0007b50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:14.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:14.776+0000 7f15c9600640 1 -- 172.21.15.67:0/1748768313 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15b807ab10 con 0x7f15cc0d16a0 2024-05-30T16:51:15.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.011+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f15cc15a440 con 0x7f15ac06cbc0 2024-05-30T16:51:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.023+0000 7f15c9600640 1 -- 172.21.15.67:0/1748768313 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f15cc15a440 con 0x7f15ac06cbc0 2024-05-30T16:51:15.024 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15ac06cbc0 msgr2=0x7f15ac06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15ac06cbc0 0x7f15ac06f080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15c0000f60 tx=0x7f15c0007b50 comp rx=0 tx=0).stop 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 msgr2=0x7f15cc15cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 0x7f15cc15cbc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b80046c0 comp rx=0 tx=0).stop 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 shutdown_connections 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15ac06cbc0 0x7f15ac06f080 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15cc15e870 0x7f15cc0c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15cc0d16a0 0x7f15cc15cbc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 --2- 172.21.15.67:0/1748768313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15cc0cf870 0x7f15cc159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 >> 172.21.15.67:0/1748768313 conn(0x7f15cc08bfc0 msgr2=0x7f15cc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 shutdown_connections 2024-05-30T16:51:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:15.026+0000 7f15d3000640 1 -- 172.21.15.67:0/1748768313 wait complete. 2024-05-30T16:51:15.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:51:15.046 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded 2024-05-30T16:51:15.046 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:51:15.109 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded 2024-05-30T16:51:15.110 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="10654449664" 2024-05-30T16:51:15.110 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:51:15.111 INFO:tasks.cephfs.test_volumes:r_size = 10654449664 exp_size = 10737418240 2024-05-30T16:51:17.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:51:17.112 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded 2024-05-30T16:51:17.112 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:51:17.183 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/96a41f23-e8eb-4ecd-81b5-4dd3e51b0ded 2024-05-30T16:51:17.183 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="10737418240" 2024-05-30T16:51:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:51:17.185 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-05-30T16:51:17.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone75831 2024-05-30T16:51:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.403+0000 7fd2a7e00640 1 Processor -- start 2024-05-30T16:51:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.403+0000 7fd2a7e00640 1 -- start start 2024-05-30T16:51:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.405+0000 7fd2a7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd2a8161d00 0x7fd2a80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.405+0000 7fd2a7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2a80d1550 0x7fd2a80d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.405+0000 7fd2a7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a80cfb40 con 0x7fd2a80d4d90 2024-05-30T16:51:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.405+0000 7fd2a7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a80d2040 con 0x7fd2a80d1550 2024-05-30T16:51:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.405+0000 7fd2a7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a80cfdc0 con 0x7fd2a8161d00 2024-05-30T16:51:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.405+0000 7fd2a7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2a80d1550 0x7fd2a80d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.406+0000 7fd2a7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2a80d1550 0x7fd2a80d27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52164/0 (socket says 172.21.15.67:52164) 2024-05-30T16:51:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.406+0000 7fd2a7400640 1 -- 172.21.15.67:0/335833013 learned_addr learned my addr 172.21.15.67:0/335833013 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.406+0000 7fd2a6a00640 1 --2- 172.21.15.67:0/335833013 >> v2:172.21.15.169:3300/0 conn(0x7fd2a8161d00 0x7fd2a80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.406+0000 7fd2a7400640 1 -- 172.21.15.67:0/335833013 >> v2:172.21.15.169:3300/0 conn(0x7fd2a8161d00 msgr2=0x7fd2a80b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.406+0000 7fd2a7400640 1 --2- 172.21.15.67:0/335833013 >> v2:172.21.15.169:3300/0 conn(0x7fd2a8161d00 0x7fd2a80b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.406+0000 7fd2a7400640 1 -- 172.21.15.67:0/335833013 >> v1:172.21.15.169:6789/0 conn(0x7fd2a80d4d90 legacy=0x7fd2a80d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.406+0000 7fd2a7400640 1 -- 172.21.15.67:0/335833013 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a80d00a0 con 0x7fd2a80d1550 2024-05-30T16:51:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.407+0000 7fd2a7400640 1 --2- 172.21.15.67:0/335833013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 0x7fd2a80d27b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80b7030 tx=0x7fd2a002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3dca401e5b83adb server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.407+0000 7fd2a5600640 1 -- 172.21.15.67:0/335833013 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0036070 con 0x7fd2a80d1550 2024-05-30T16:51:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.407+0000 7fd2a5600640 1 -- 172.21.15.67:0/335833013 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a0031600 con 0x7fd2a80d1550 2024-05-30T16:51:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.407+0000 7fd2a5600640 1 -- 172.21.15.67:0/335833013 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00319d0 con 0x7fd2a80d1550 2024-05-30T16:51:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.408+0000 7fd2a7e00640 1 -- 172.21.15.67:0/335833013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 msgr2=0x7fd2a80d27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.408+0000 7fd2a7e00640 1 --2- 172.21.15.67:0/335833013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 0x7fd2a80d27b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80b7030 tx=0x7fd2a002a6f0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.408+0000 7fd2a7e00640 1 -- 172.21.15.67:0/335833013 shutdown_connections 2024-05-30T16:51:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.408+0000 7fd2a7e00640 1 --2- 172.21.15.67:0/335833013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 0x7fd2a80d27b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.408+0000 7fd2a7e00640 1 --2- 172.21.15.67:0/335833013 >> v2:172.21.15.169:3300/0 conn(0x7fd2a8161d00 0x7fd2a80b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.408+0000 7fd2a7e00640 1 -- 172.21.15.67:0/335833013 >> 172.21.15.67:0/335833013 conn(0x7fd2a808bfc0 msgr2=0x7fd2a80d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.408+0000 7fd2a7e00640 1 -- 172.21.15.67:0/335833013 shutdown_connections 2024-05-30T16:51:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.408+0000 7fd2a7e00640 1 -- 172.21.15.67:0/335833013 wait complete. 2024-05-30T16:51:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.410+0000 7fd2a7e00640 1 Processor -- start 2024-05-30T16:51:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.410+0000 7fd2a7e00640 1 -- start start 2024-05-30T16:51:17.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.411+0000 7fd2a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 0x7fd2a81487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:17.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.411+0000 7fd2a7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2a80d4d90 0x7fd2a80cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:17.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.411+0000 7fd2a7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8161d00 0x7fd2a80cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:17.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.411+0000 7fd2a7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a80b5560 con 0x7fd2a80d1550 2024-05-30T16:51:17.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.411+0000 7fd2a7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a80d2290 con 0x7fd2a8161d00 2024-05-30T16:51:17.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.411+0000 7fd2a7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a80b56e0 con 0x7fd2a80d4d90 2024-05-30T16:51:17.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.411+0000 7fd2a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 0x7fd2a81487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 0x7fd2a81487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52178/0 (socket says 172.21.15.67:52178) 2024-05-30T16:51:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a6a00640 1 -- 172.21.15.67:0/1508299167 learned_addr learned my addr 172.21.15.67:0/1508299167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a7400640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8161d00 0x7fd2a80cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a6000640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2a80d4d90 0x7fd2a80cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a6a00640 1 -- 172.21.15.67:0/1508299167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2a80d4d90 msgr2=0x7fd2a80cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a6a00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2a80d4d90 0x7fd2a80cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a6a00640 1 -- 172.21.15.67:0/1508299167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8161d00 msgr2=0x7fd2a80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a6a00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8161d00 0x7fd2a80cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a6a00640 1 -- 172.21.15.67:0/1508299167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a80b73f0 con 0x7fd2a80d1550 2024-05-30T16:51:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.412+0000 7fd2a6a00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 0x7fd2a81487e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd29800d620 tx=0x7fd29800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.413+0000 7fd29ea00640 1 -- 172.21.15.67:0/1508299167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29800b620 con 0x7fd2a80d1550 2024-05-30T16:51:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.413+0000 7fd29ea00640 1 -- 172.21.15.67:0/1508299167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd298019070 con 0x7fd2a80d1550 2024-05-30T16:51:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.413+0000 7fd29ea00640 1 -- 172.21.15.67:0/1508299167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29800ba00 con 0x7fd2a80d1550 2024-05-30T16:51:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.413+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a80b76d0 con 0x7fd2a80d1550 2024-05-30T16:51:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.413+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a80b7c90 con 0x7fd2a80d1550 2024-05-30T16:51:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.414+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a80689e0 con 0x7fd2a80d1550 2024-05-30T16:51:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.415+0000 7fd29ea00640 1 -- 172.21.15.67:0/1508299167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) ==== 91346+0+0 (secure 0 0 0) 0x7fd29800bba0 con 0x7fd2a80d1550 2024-05-30T16:51:17.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.416+0000 7fd29ea00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd28406cc50 0x7fd28406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:17.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.417+0000 7fd29ea00640 1 -- 172.21.15.67:0/1508299167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fd29808c510 con 0x7fd2a80d1550 2024-05-30T16:51:17.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.417+0000 7fd2a6000640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd28406cc50 0x7fd28406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:17.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.418+0000 7fd2a6000640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd28406cc50 0x7fd28406f110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd2a815d130 tx=0x7fd290009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.421+0000 7fd29ea00640 1 -- 172.21.15.67:0/1508299167 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd29805a4b0 con 0x7fd2a80d1550 2024-05-30T16:51:17.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.668+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone75831", "target": ["mon-mgr", ""]}) -- 0x7fd2a80b7ed0 con 0x7fd28406cc50 2024-05-30T16:51:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.844+0000 7fd29ea00640 1 -- 172.21.15.67:0/1508299167 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd2a80b7ed0 con 0x7fd28406cc50 2024-05-30T16:51:17.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd28406cc50 msgr2=0x7fd28406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:17.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd28406cc50 0x7fd28406f110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd2a815d130 tx=0x7fd290009040 comp rx=0 tx=0).stop 2024-05-30T16:51:17.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 msgr2=0x7fd2a81487e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 0x7fd2a81487e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd29800d620 tx=0x7fd29800dae0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 shutdown_connections 2024-05-30T16:51:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd28406cc50 0x7fd28406f110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2a8161d00 0x7fd2a80cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2a80d4d90 0x7fd2a80cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 --2- 172.21.15.67:0/1508299167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a80d1550 0x7fd2a81487e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 >> 172.21.15.67:0/1508299167 conn(0x7fd2a808bfc0 msgr2=0x7fd2a815c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 shutdown_connections 2024-05-30T16:51:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:17.848+0000 7fd2a7e00640 1 -- 172.21.15.67:0/1508299167 wait complete. 2024-05-30T16:51:17.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.851+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:17.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.851+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:17.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.851+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:17.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.851+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:17.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.851+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:17.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone75832 2024-05-30T16:51:17.877 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.875+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:17.878 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.875+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:17.878 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.875+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:17.878 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.875+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:17.878 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:17.875+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:18.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.065+0000 7ff0d2600640 1 Processor -- start 2024-05-30T16:51:18.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.066+0000 7ff0d2600640 1 -- start start 2024-05-30T16:51:18.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.067+0000 7ff0d2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff0cc1525d0 0x7ff0cc073910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:18.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.067+0000 7ff0d2600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff0cc0af230 0x7ff0cc0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:18.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.067+0000 7ff0d2600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0cc1529b0 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.067+0000 7ff0d2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff0cc15f050 con 0x7ff0cc1525d0 2024-05-30T16:51:18.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.067+0000 7ff0d2600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff0cc0d1050 con 0x7ff0cc0af230 2024-05-30T16:51:18.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.068+0000 7ff0cb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff0cc1525d0 0x7ff0cc073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:18.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.068+0000 7ff0cb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff0cc1525d0 0x7ff0cc073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52236/0 (socket says 172.21.15.67:52236) 2024-05-30T16:51:18.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.068+0000 7ff0cb400640 1 -- 172.21.15.67:0/469279452 learned_addr learned my addr 172.21.15.67:0/469279452 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:18.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.068+0000 7ff0cbe00640 1 --2- 172.21.15.67:0/469279452 >> v2:172.21.15.169:3300/0 conn(0x7ff0cc0af230 0x7ff0cc0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:18.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.068+0000 7ff0cb400640 1 -- 172.21.15.67:0/469279452 >> v2:172.21.15.169:3300/0 conn(0x7ff0cc0af230 msgr2=0x7ff0cc0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:18.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.068+0000 7ff0cb400640 1 --2- 172.21.15.67:0/469279452 >> v2:172.21.15.169:3300/0 conn(0x7ff0cc0af230 0x7ff0cc0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:18.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.068+0000 7ff0cb400640 1 -- 172.21.15.67:0/469279452 >> v1:172.21.15.169:6789/0 conn(0x7ff0cc0d06d0 legacy=0x7ff0cc0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:18.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.068+0000 7ff0cb400640 1 -- 172.21.15.67:0/469279452 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc15f2a0 con 0x7ff0cc1525d0 2024-05-30T16:51:18.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.069+0000 7ff0cb400640 1 --2- 172.21.15.67:0/469279452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc1525d0 0x7ff0cc073910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0adcd0 tx=0x7ff0bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370875213c62bbef server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:18.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.070+0000 7ff0caa00640 1 -- 172.21.15.67:0/469279452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc037070 con 0x7ff0cc1525d0 2024-05-30T16:51:18.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.070+0000 7ff0caa00640 1 -- 172.21.15.67:0/469279452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0bc032440 con 0x7ff0cc1525d0 2024-05-30T16:51:18.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.070+0000 7ff0caa00640 1 -- 172.21.15.67:0/469279452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc032810 con 0x7ff0cc1525d0 2024-05-30T16:51:18.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.070+0000 7ff0d2600640 1 -- 172.21.15.67:0/469279452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc1525d0 msgr2=0x7ff0cc073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:18.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.070+0000 7ff0d2600640 1 --2- 172.21.15.67:0/469279452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc1525d0 0x7ff0cc073910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0adcd0 tx=0x7ff0bc02a230 comp rx=0 tx=0).stop 2024-05-30T16:51:18.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.071+0000 7ff0d2600640 1 -- 172.21.15.67:0/469279452 shutdown_connections 2024-05-30T16:51:18.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.071+0000 7ff0d2600640 1 --2- 172.21.15.67:0/469279452 >> v2:172.21.15.169:3300/0 conn(0x7ff0cc0af230 0x7ff0cc0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:18.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.071+0000 7ff0d2600640 1 --2- 172.21.15.67:0/469279452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc1525d0 0x7ff0cc073910 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:18.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.071+0000 7ff0d2600640 1 -- 172.21.15.67:0/469279452 >> 172.21.15.67:0/469279452 conn(0x7ff0cc06bd20 msgr2=0x7ff0cc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:18.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.071+0000 7ff0d2600640 1 -- 172.21.15.67:0/469279452 shutdown_connections 2024-05-30T16:51:18.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.071+0000 7ff0d2600640 1 -- 172.21.15.67:0/469279452 wait complete. 2024-05-30T16:51:18.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.073+0000 7ff0d2600640 1 Processor -- start 2024-05-30T16:51:18.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.073+0000 7ff0d2600640 1 -- start start 2024-05-30T16:51:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.074+0000 7ff0d2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0cc0af230 0x7ff0cc1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.074+0000 7ff0d2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc0d06d0 0x7ff0cc143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.074+0000 7ff0d2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0cc1525d0 0x7ff0cc144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.074+0000 7ff0d2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc15fb40 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.074+0000 7ff0d2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc15f860 con 0x7ff0cc1525d0 2024-05-30T16:51:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.074+0000 7ff0d2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc15fe20 con 0x7ff0cc0af230 2024-05-30T16:51:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.074+0000 7ff0c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc0d06d0 0x7ff0cc143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.074+0000 7ff0c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc0d06d0 0x7ff0cc143ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52238/0 (socket says 172.21.15.67:52238) 2024-05-30T16:51:18.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.074+0000 7ff0c3e00640 1 -- 172.21.15.67:0/1610488235 learned_addr learned my addr 172.21.15.67:0/1610488235 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:18.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0c3e00640 1 -- 172.21.15.67:0/1610488235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0cc0af230 msgr2=0x7ff0cc1f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:18.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0c3e00640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0cc0af230 0x7ff0cc1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:18.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0c3e00640 1 -- 172.21.15.67:0/1610488235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0cc1525d0 msgr2=0x7ff0cc144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:18.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0c3e00640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0cc1525d0 0x7ff0cc144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:18.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0c3e00640 1 -- 172.21.15.67:0/1610488235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc0cd920 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0c3e00640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc0d06d0 0x7ff0cc143ff0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400c410 tx=0x7ff0b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:18.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0c3400640 1 -- 172.21.15.67:0/1610488235 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4017070 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0c3400640 1 -- 172.21.15.67:0/1610488235 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b40035f0 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc0cdc00 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0c3400640 1 -- 172.21.15.67:0/1610488235 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4003a20 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.075+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc0ce1c0 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.077+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0cc0689e0 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.078+0000 7ff0c3400640 1 -- 172.21.15.67:0/1610488235 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) ==== 91346+0+0 (secure 0 0 0) 0x7ff0b401a070 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.078+0000 7ff0c3400640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0a806cc50 0x7ff0a806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:18.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.079+0000 7ff0c3400640 1 -- 172.21.15.67:0/1610488235 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7ff0b408b5e0 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.079+0000 7ff0cb400640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0a806cc50 0x7ff0a806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:18.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.080+0000 7ff0cb400640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0a806cc50 0x7ff0a806f110 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc034000 tx=0x7ff0bc02c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:18.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.083+0000 7ff0c3400640 1 -- 172.21.15.67:0/1610488235 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0b40595c0 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.331+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone75832", "target": ["mon-mgr", ""]}) -- 0x7ff0cc0cd6d0 con 0x7ff0a806cc50 2024-05-30T16:51:18.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:18.747+0000 7ff0c3400640 1 -- 172.21.15.67:0/1610488235 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 14) ==== 91448+0+0 (secure 0 0 0) 0x7ff0b4055530 con 0x7ff0cc0d06d0 2024-05-30T16:51:18.853 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:18.852+0000 7f0c77200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:18.853 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:18.852+0000 7f0c77200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:18.853 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:18.852+0000 7f0c77200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:18.853 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:18.852+0000 7f0c77200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:18.853 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:18.852+0000 7f0c77200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:19.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.093+0000 7ff0c3400640 1 -- 172.21.15.67:0/1610488235 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff0cc0cd6d0 con 0x7ff0a806cc50 2024-05-30T16:51:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.098+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0a806cc50 msgr2=0x7ff0a806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.098+0000 7ff0d2600640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0a806cc50 0x7ff0a806f110 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc034000 tx=0x7ff0bc02c000 comp rx=0 tx=0).stop 2024-05-30T16:51:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.099+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc0d06d0 msgr2=0x7ff0cc143ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.099+0000 7ff0d2600640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc0d06d0 0x7ff0cc143ff0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400c410 tx=0x7ff0b400c8d0 comp rx=0 tx=0).stop 2024-05-30T16:51:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.099+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 shutdown_connections 2024-05-30T16:51:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.099+0000 7ff0d2600640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0a806cc50 0x7ff0a806f110 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc034000 tx=0x7ff0bc02c000 comp rx=0 tx=0).stop 2024-05-30T16:51:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.099+0000 7ff0d2600640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0cc1525d0 0x7ff0cc144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.099+0000 7ff0d2600640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0cc0d06d0 0x7ff0cc143ff0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.099+0000 7ff0d2600640 1 --2- 172.21.15.67:0/1610488235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0cc0af230 0x7ff0cc1f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.099+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 >> 172.21.15.67:0/1610488235 conn(0x7ff0cc06bd20 msgr2=0x7ff0cc06de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.100+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 shutdown_connections 2024-05-30T16:51:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.101+0000 7ff0d2600640 1 -- 172.21.15.67:0/1610488235 wait complete. 2024-05-30T16:51:19.125 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:19.124+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:19.125 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:19.124+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:19.125 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:19.124+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:19.125 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:19.124+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:19.125 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:19.124+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:19.142 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone75833 2024-05-30T16:51:19.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.477+0000 7f2a74800640 1 Processor -- start 2024-05-30T16:51:19.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.477+0000 7f2a74800640 1 -- start start 2024-05-30T16:51:19.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.481+0000 7f2a74800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2a6c162c20 0x7f2a6c072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.481+0000 7f2a74800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c06e450 0x7f2a6c0d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:19.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.483+0000 7f2a71e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2a6c162c20 0x7f2a6c072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.483+0000 7f2a71e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2a6c162c20 0x7f2a6c072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52278/0 (socket says 172.21.15.67:52278) 2024-05-30T16:51:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.483+0000 7f2a71e00640 1 -- 172.21.15.67:0/372232322 learned_addr learned my addr 172.21.15.67:0/372232322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.481+0000 7f2a74800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a6c072db0 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.484+0000 7f2a74800640 1 -- 172.21.15.67:0/372232322 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2a6c072b80 con 0x7f2a6c162c20 2024-05-30T16:51:19.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.484+0000 7f2a74800640 1 -- 172.21.15.67:0/372232322 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6c163000 con 0x7f2a6c06e450 2024-05-30T16:51:19.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.484+0000 7f2a71400640 1 --2- 172.21.15.67:0/372232322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c06e450 0x7f2a6c0d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:19.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.485+0000 7f2a71e00640 1 -- 172.21.15.67:0/372232322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c06e450 msgr2=0x7f2a6c0d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:19.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.485+0000 7f2a71e00640 1 --2- 172.21.15.67:0/372232322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c06e450 0x7f2a6c0d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:19.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.485+0000 7f2a71e00640 1 -- 172.21.15.67:0/372232322 >> v1:172.21.15.169:6789/0 conn(0x7f2a6c0d0c10 legacy=0x7f2a6c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:19.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.485+0000 7f2a71e00640 1 -- 172.21.15.67:0/372232322 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c073090 con 0x7f2a6c162c20 2024-05-30T16:51:19.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.487+0000 7f2a71e00640 1 --2- 172.21.15.67:0/372232322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c162c20 0x7f2a6c072640 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009e50 tx=0x7f2a6802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cca10a6591b121 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:19.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.487+0000 7f2a70a00640 1 -- 172.21.15.67:0/372232322 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68036070 con 0x7f2a6c162c20 2024-05-30T16:51:19.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.487+0000 7f2a70a00640 1 -- 172.21.15.67:0/372232322 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a68031600 con 0x7f2a6c162c20 2024-05-30T16:51:19.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.487+0000 7f2a70a00640 1 -- 172.21.15.67:0/372232322 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680319d0 con 0x7f2a6c162c20 2024-05-30T16:51:19.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.489+0000 7f2a74800640 1 -- 172.21.15.67:0/372232322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c162c20 msgr2=0x7f2a6c072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:19.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.489+0000 7f2a74800640 1 --2- 172.21.15.67:0/372232322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c162c20 0x7f2a6c072640 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009e50 tx=0x7f2a6802a680 comp rx=0 tx=0).stop 2024-05-30T16:51:19.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.490+0000 7f2a74800640 1 -- 172.21.15.67:0/372232322 shutdown_connections 2024-05-30T16:51:19.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.490+0000 7f2a74800640 1 --2- 172.21.15.67:0/372232322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c06e450 0x7f2a6c0d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:19.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.490+0000 7f2a74800640 1 --2- 172.21.15.67:0/372232322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c162c20 0x7f2a6c072640 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:19.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.490+0000 7f2a74800640 1 -- 172.21.15.67:0/372232322 >> 172.21.15.67:0/372232322 conn(0x7f2a6c06bd20 msgr2=0x7f2a6c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:19.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.492+0000 7f2a74800640 1 -- 172.21.15.67:0/372232322 shutdown_connections 2024-05-30T16:51:19.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.492+0000 7f2a74800640 1 -- 172.21.15.67:0/372232322 wait complete. 2024-05-30T16:51:19.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.501+0000 7f2a74800640 1 Processor -- start 2024-05-30T16:51:19.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.504+0000 7f2a74800640 1 -- start start 2024-05-30T16:51:19.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.506+0000 7f2a74800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a6c06e450 0x7f2a6c09a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:19.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.506+0000 7f2a74800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c0d0c10 0x7f2a6c09cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:19.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.506+0000 7f2a74800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c162c20 0x7f2a6c0b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:19.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.506+0000 7f2a74800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c0d1040 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.506+0000 7f2a74800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c0739c0 con 0x7f2a6c06e450 2024-05-30T16:51:19.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.506+0000 7f2a74800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6c073650 con 0x7f2a6c162c20 2024-05-30T16:51:19.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.508+0000 7f2a71400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c0d0c10 0x7f2a6c09cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:19.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.508+0000 7f2a71400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c0d0c10 0x7f2a6c09cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52282/0 (socket says 172.21.15.67:52282) 2024-05-30T16:51:19.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.508+0000 7f2a71400640 1 -- 172.21.15.67:0/3904530609 learned_addr learned my addr 172.21.15.67:0/3904530609 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:19.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.508+0000 7f2a71e00640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a6c06e450 0x7f2a6c09a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:19.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.509+0000 7f2a72800640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c162c20 0x7f2a6c0b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:19.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.511+0000 7f2a71400640 1 -- 172.21.15.67:0/3904530609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c162c20 msgr2=0x7f2a6c0b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:19.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.511+0000 7f2a71400640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c162c20 0x7f2a6c0b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:19.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.511+0000 7f2a71400640 1 -- 172.21.15.67:0/3904530609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a6c06e450 msgr2=0x7f2a6c09a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:19.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.511+0000 7f2a71400640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a6c06e450 0x7f2a6c09a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:19.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.511+0000 7f2a71400640 1 -- 172.21.15.67:0/3904530609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c09d880 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.513+0000 7f2a71400640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c0d0c10 0x7f2a6c09cfc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000d920 tx=0x7f2a6000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:19.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.514+0000 7f2a66000640 1 -- 172.21.15.67:0/3904530609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000b780 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.514+0000 7f2a66000640 1 -- 172.21.15.67:0/3904530609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a60019070 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.514+0000 7f2a66000640 1 -- 172.21.15.67:0/3904530609 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000bb40 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.516+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c09db60 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.516+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c09d500 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.519+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6c202750 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.520+0000 7f2a66000640 1 -- 172.21.15.67:0/3904530609 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 14) ==== 91448+0+0 (secure 0 0 0) 0x7f2a6000bce0 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.522+0000 7f2a66000640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a5006cdc0 0x7f2a5006f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:19.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.522+0000 7f2a66000640 1 -- 172.21.15.67:0/3904530609 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f2a6008c6e0 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.529+0000 7f2a66000640 1 -- 172.21.15.67:0/3904530609 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a6005a690 con 0x7f2a6c0d0c10 2024-05-30T16:51:19.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.559+0000 7f2a71e00640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a5006cdc0 0x7f2a5006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:19.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:19.570+0000 7f2a71e00640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a5006cdc0 0x7f2a5006f280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2a68033000 tx=0x7f2a68009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:20.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.064+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone75833", "target": ["mon-mgr", ""]}) -- 0x7f2a6c0cfd80 con 0x7f2a5006cdc0 2024-05-30T16:51:20.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.722+0000 7f2a66000640 1 -- 172.21.15.67:0/3904530609 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2a6c0cfd80 con 0x7f2a5006cdc0 2024-05-30T16:51:20.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.730+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a5006cdc0 msgr2=0x7f2a5006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:20.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.730+0000 7f2a74800640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a5006cdc0 0x7f2a5006f280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2a68033000 tx=0x7f2a68009000 comp rx=0 tx=0).stop 2024-05-30T16:51:20.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.730+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c0d0c10 msgr2=0x7f2a6c09cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:20.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.730+0000 7f2a74800640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c0d0c10 0x7f2a6c09cfc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000d920 tx=0x7f2a6000dde0 comp rx=0 tx=0).stop 2024-05-30T16:51:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.735+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 shutdown_connections 2024-05-30T16:51:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.735+0000 7f2a74800640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a5006cdc0 0x7f2a5006f280 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.735+0000 7f2a74800640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a6c162c20 0x7f2a6c0b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.735+0000 7f2a74800640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a6c0d0c10 0x7f2a6c09cfc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.735+0000 7f2a74800640 1 --2- 172.21.15.67:0/3904530609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a6c06e450 0x7f2a6c09a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:20.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.735+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 >> 172.21.15.67:0/3904530609 conn(0x7f2a6c06bd20 msgr2=0x7f2a6c160c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:20.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.735+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 shutdown_connections 2024-05-30T16:51:20.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:20.735+0000 7f2a74800640 1 -- 172.21.15.67:0/3904530609 wait complete. 2024-05-30T16:51:20.765 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:20.764+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:20.765 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:20.764+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:20.765 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:20.764+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:20.765 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:20.764+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:20.765 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:20.764+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:20.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone75834 2024-05-30T16:51:21.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.193+0000 7fcd13e00640 1 Processor -- start 2024-05-30T16:51:21.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.193+0000 7fcd13e00640 1 -- start start 2024-05-30T16:51:21.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.195+0000 7fcd13e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcd0c155000 0x7fcd0c15b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:21.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.195+0000 7fcd13e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd0c0b3c30 0x7fcd0c0d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:21.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.195+0000 7fcd13e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd0c15bdd0 con 0x7fcd0c0d0c40 2024-05-30T16:51:21.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.195+0000 7fcd13e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fcd0c0b3510 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.195+0000 7fcd13e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcd0c06e530 con 0x7fcd0c155000 2024-05-30T16:51:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.195+0000 7fcd10a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd0c0b3c30 0x7fcd0c0d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.195+0000 7fcd10a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd0c0b3c30 0x7fcd0c0d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33952/0 (socket says 172.21.15.67:33952) 2024-05-30T16:51:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.195+0000 7fcd10a00640 1 -- 172.21.15.67:0/640036104 learned_addr learned my addr 172.21.15.67:0/640036104 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.196+0000 7fcd11400640 1 --2- 172.21.15.67:0/640036104 >> v2:172.21.15.169:3300/0 conn(0x7fcd0c155000 0x7fcd0c15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.196+0000 7fcd10a00640 1 -- 172.21.15.67:0/640036104 >> v2:172.21.15.169:3300/0 conn(0x7fcd0c155000 msgr2=0x7fcd0c15b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.196+0000 7fcd10a00640 1 --2- 172.21.15.67:0/640036104 >> v2:172.21.15.169:3300/0 conn(0x7fcd0c155000 0x7fcd0c15b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.196+0000 7fcd10a00640 1 -- 172.21.15.67:0/640036104 >> v1:172.21.15.67:6789/0 conn(0x7fcd0c0d0c40 legacy=0x7fcd0c073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.196+0000 7fcd10a00640 1 -- 172.21.15.67:0/640036104 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c0b40d0 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.197+0000 7fcd10a00640 1 --2- 172.21.15.67:0/640036104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0b3c30 0x7fcd0c0d0700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c071590 tx=0x7fccf802c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411db72c875401c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:21.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.197+0000 7fcd0be00640 1 -- 172.21.15.67:0/640036104 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf803a070 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.198+0000 7fcd0be00640 1 -- 172.21.15.67:0/640036104 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf8032480 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.198+0000 7fcd0be00640 1 -- 172.21.15.67:0/640036104 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8032850 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.199+0000 7fcd13e00640 1 -- 172.21.15.67:0/640036104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0b3c30 msgr2=0x7fcd0c0d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.199+0000 7fcd13e00640 1 --2- 172.21.15.67:0/640036104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0b3c30 0x7fcd0c0d0700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c071590 tx=0x7fccf802c730 comp rx=0 tx=0).stop 2024-05-30T16:51:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.199+0000 7fcd13e00640 1 -- 172.21.15.67:0/640036104 shutdown_connections 2024-05-30T16:51:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.199+0000 7fcd13e00640 1 --2- 172.21.15.67:0/640036104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0b3c30 0x7fcd0c0d0700 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.199+0000 7fcd13e00640 1 --2- 172.21.15.67:0/640036104 >> v2:172.21.15.169:3300/0 conn(0x7fcd0c155000 0x7fcd0c15b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.199+0000 7fcd13e00640 1 -- 172.21.15.67:0/640036104 >> 172.21.15.67:0/640036104 conn(0x7fcd0c0b56e0 msgr2=0x7fcd0c0b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:21.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.202+0000 7fcd13e00640 1 -- 172.21.15.67:0/640036104 shutdown_connections 2024-05-30T16:51:21.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.202+0000 7fcd13e00640 1 -- 172.21.15.67:0/640036104 wait complete. 2024-05-30T16:51:21.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.205+0000 7fcd13e00640 1 Processor -- start 2024-05-30T16:51:21.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.205+0000 7fcd13e00640 1 -- start start 2024-05-30T16:51:21.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.207+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c0b3c30 0x7fcd0c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:21.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.207+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0d0c40 0x7fcd0c1f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:21.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.207+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd0c155000 0x7fcd0c0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:21.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.207+0000 7fcd13e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c06e1c0 con 0x7fcd0c0d0c40 2024-05-30T16:51:21.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.207+0000 7fcd13e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c0d1070 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.207+0000 7fcd13e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0c0d02b0 con 0x7fcd0c155000 2024-05-30T16:51:21.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.208+0000 7fcd11400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c0b3c30 0x7fcd0c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:21.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.209+0000 7fcd11400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c0b3c30 0x7fcd0c0afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:41588/0 (socket says 172.21.15.67:41588) 2024-05-30T16:51:21.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.209+0000 7fcd11400640 1 -- 172.21.15.67:0/50080558 learned_addr learned my addr 172.21.15.67:0/50080558 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:21.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.209+0000 7fcd10a00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0d0c40 0x7fcd0c1f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:21.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.209+0000 7fcd11e00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd0c155000 0x7fcd0c0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:21.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.210+0000 7fcd11400640 1 -- 172.21.15.67:0/50080558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd0c155000 msgr2=0x7fcd0c0ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:21.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.210+0000 7fcd11400640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd0c155000 0x7fcd0c0ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:21.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.210+0000 7fcd11400640 1 -- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0d0c40 msgr2=0x7fcd0c1f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:21.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.213+0000 7fcd11400640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0d0c40 0x7fcd0c1f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:21.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.213+0000 7fcd11400640 1 -- 172.21.15.67:0/50080558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c0cf390 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.213+0000 7fcd11e00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd0c155000 0x7fcd0c0ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:51:21.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.214+0000 7fcd10a00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0d0c40 0x7fcd0c1f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:51:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.216+0000 7fcd11400640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c0b3c30 0x7fcd0c0afec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000efc0 tx=0x7fcd0000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:21.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.218+0000 7fcd09600640 1 -- 172.21.15.67:0/50080558 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00018020 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.218+0000 7fcd09600640 1 -- 172.21.15.67:0/50080558 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd00014480 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.218+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0c0cf0b0 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.218+0000 7fcd09600640 1 -- 172.21.15.67:0/50080558 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00014850 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.218+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0c0b1c10 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.224+0000 7fcd09600640 1 -- 172.21.15.67:0/50080558 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 91550+0+0 (secure 0 0 0) 0x7fcd00005d10 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.224+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd0c0689e0 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.225+0000 7fcd09600640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fccec06ce90 0x7fccec06f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.226+0000 7fcd09600640 1 -- 172.21.15.67:0/50080558 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fcd0008cc50 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.236+0000 7fcd09600640 1 -- 172.21.15.67:0/50080558 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd00010040 con 0x7fcd0c0b3c30 2024-05-30T16:51:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.254+0000 7fcd10a00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fccec06ce90 0x7fccec06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.257+0000 7fcd10a00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fccec06ce90 0x7fccec06f350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fccf8037000 tx=0x7fccf8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:21.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:21.818+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone75834", "target": ["mon-mgr", ""]}) -- 0x7fcd0c152460 con 0x7fccec06ce90 2024-05-30T16:51:22.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.326+0000 7fcd09600640 1 -- 172.21.15.67:0/50080558 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mgrmap(e 16) ==== 91601+0+0 (secure 0 0 0) 0x7fcd0001b020 con 0x7fcd0c0b3c30 2024-05-30T16:51:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.965+0000 7fcd09600640 1 -- 172.21.15.67:0/50080558 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd0c152460 con 0x7fccec06ce90 2024-05-30T16:51:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fccec06ce90 msgr2=0x7fccec06f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fccec06ce90 0x7fccec06f350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fccf8037000 tx=0x7fccf8008040 comp rx=0 tx=0).stop 2024-05-30T16:51:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c0b3c30 msgr2=0x7fcd0c0afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c0b3c30 0x7fcd0c0afec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000efc0 tx=0x7fcd0000c490 comp rx=0 tx=0).stop 2024-05-30T16:51:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 shutdown_connections 2024-05-30T16:51:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fccec06ce90 0x7fccec06f350 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd0c155000 0x7fcd0c0ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:22.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd0c0d0c40 0x7fcd0c1f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:22.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 --2- 172.21.15.67:0/50080558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd0c0b3c30 0x7fcd0c0afec0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:22.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 >> 172.21.15.67:0/50080558 conn(0x7fcd0c0b56e0 msgr2=0x7fcd0c06e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:22.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 shutdown_connections 2024-05-30T16:51:22.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:22.977+0000 7fcd13e00640 1 -- 172.21.15.67:0/50080558 wait complete. 2024-05-30T16:51:23.031 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:51:23.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.459+0000 7ff41b200640 1 Processor -- start 2024-05-30T16:51:23.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.459+0000 7ff41b200640 1 -- start start 2024-05-30T16:51:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.462+0000 7ff41b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff414054200 0x7ff414050740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.462+0000 7ff41b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff414006720 0x7ff414006b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.462+0000 7ff41b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4140526a0 con 0x7ff4140df800 2024-05-30T16:51:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.462+0000 7ff41b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff4140dfbe0 con 0x7ff414054200 2024-05-30T16:51:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.462+0000 7ff41b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff4140070b0 con 0x7ff414006720 2024-05-30T16:51:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.462+0000 7ff419e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ff4140df800 0x7ff414053140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:52928/0 (socket says 172.21.15.67:52928) 2024-05-30T16:51:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.462+0000 7ff419e00640 1 -- 172.21.15.67:0/1037679115 learned_addr learned my addr 172.21.15.67:0/1037679115 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:51:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.463+0000 7ff419400640 1 --2- 172.21.15.67:0/1037679115 >> v2:172.21.15.67:3300/0 conn(0x7ff414054200 0x7ff414050740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.463+0000 7ff41a800640 1 --2- 172.21.15.67:0/1037679115 >> v2:172.21.15.169:3300/0 conn(0x7ff414006720 0x7ff414006b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.465+0000 7ff418a00640 1 -- 172.21.15.67:0/1037679115 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2392765211 0 0) 0x7ff4140526a0 con 0x7ff4140df800 2024-05-30T16:51:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.465+0000 7ff418a00640 1 -- 172.21.15.67:0/1037679115 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff400003520 con 0x7ff4140df800 2024-05-30T16:51:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.466+0000 7ff419400640 1 -- 172.21.15.67:0/1037679115 >> v2:172.21.15.169:3300/0 conn(0x7ff414006720 msgr2=0x7ff414006b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.466+0000 7ff419400640 1 --2- 172.21.15.67:0/1037679115 >> v2:172.21.15.169:3300/0 conn(0x7ff414006720 0x7ff414006b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.466+0000 7ff419400640 1 -- 172.21.15.67:0/1037679115 >> v1:172.21.15.169:6789/0 conn(0x7ff4140df800 legacy=0x7ff414053140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.466+0000 7ff419400640 1 -- 172.21.15.67:0/1037679115 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff414053850 con 0x7ff414054200 2024-05-30T16:51:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.468+0000 7ff419400640 1 --2- 172.21.15.67:0/1037679115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414054200 0x7ff414050740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff4140e4300 tx=0x7ff40402a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50c65f74dc63902 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.469+0000 7ff418a00640 1 -- 172.21.15.67:0/1037679115 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404037070 con 0x7ff414054200 2024-05-30T16:51:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.469+0000 7ff418a00640 1 -- 172.21.15.67:0/1037679115 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40402ac60 con 0x7ff414054200 2024-05-30T16:51:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.470+0000 7ff41b200640 1 -- 172.21.15.67:0/1037679115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414054200 msgr2=0x7ff414050740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.470+0000 7ff418a00640 1 -- 172.21.15.67:0/1037679115 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404037070 con 0x7ff414054200 2024-05-30T16:51:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.470+0000 7ff41b200640 1 --2- 172.21.15.67:0/1037679115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414054200 0x7ff414050740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff4140e4300 tx=0x7ff40402a3e0 comp rx=0 tx=0).stop 2024-05-30T16:51:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.472+0000 7ff41b200640 1 -- 172.21.15.67:0/1037679115 shutdown_connections 2024-05-30T16:51:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.472+0000 7ff41b200640 1 --2- 172.21.15.67:0/1037679115 >> v2:172.21.15.169:3300/0 conn(0x7ff414006720 0x7ff414006b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.472+0000 7ff41b200640 1 --2- 172.21.15.67:0/1037679115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414054200 0x7ff414050740 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.472+0000 7ff41b200640 1 -- 172.21.15.67:0/1037679115 >> 172.21.15.67:0/1037679115 conn(0x7ff414009fb0 msgr2=0x7ff41400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.472+0000 7ff41b200640 1 -- 172.21.15.67:0/1037679115 shutdown_connections 2024-05-30T16:51:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.472+0000 7ff41b200640 1 -- 172.21.15.67:0/1037679115 wait complete. 2024-05-30T16:51:23.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.477+0000 7ff41b200640 1 Processor -- start 2024-05-30T16:51:23.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.477+0000 7ff41b200640 1 -- start start 2024-05-30T16:51:23.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.479+0000 7ff41b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414006720 0x7ff4140dfd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:23.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.480+0000 7ff41b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff414054200 0x7ff41404afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:23.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.480+0000 7ff419e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414006720 0x7ff4140dfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:23.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.480+0000 7ff419e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414006720 0x7ff4140dfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33982/0 (socket says 172.21.15.67:33982) 2024-05-30T16:51:23.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.481+0000 7ff419e00640 1 -- 172.21.15.67:0/1102563831 learned_addr learned my addr 172.21.15.67:0/1102563831 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:23.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.481+0000 7ff41b200640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4140df800 0x7ff414177fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:23.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.482+0000 7ff419400640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff414054200 0x7ff41404afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:23.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.482+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff414052c20 con 0x7ff414006720 2024-05-30T16:51:23.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.483+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff414052940 con 0x7ff414054200 2024-05-30T16:51:23.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.483+0000 7ff41a800640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4140df800 0x7ff414177fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:23.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.483+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff414054850 con 0x7ff4140df800 2024-05-30T16:51:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.485+0000 7ff41a800640 1 -- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff414054200 msgr2=0x7ff41404afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.486+0000 7ff41a800640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff414054200 0x7ff41404afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.486+0000 7ff41a800640 1 -- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414006720 msgr2=0x7ff4140dfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.486+0000 7ff41a800640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414006720 0x7ff4140dfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.486+0000 7ff41a800640 1 -- 172.21.15.67:0/1102563831 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4140d8930 con 0x7ff4140df800 2024-05-30T16:51:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.486+0000 7ff419400640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff414054200 0x7ff41404afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:51:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.486+0000 7ff419e00640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414006720 0x7ff4140dfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:51:23.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.488+0000 7ff41a800640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4140df800 0x7ff414177fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000b60 tx=0x7ff41c06bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:23.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.489+0000 7ff412000640 1 -- 172.21.15.67:0/1102563831 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c073810 con 0x7ff4140df800 2024-05-30T16:51:23.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.490+0000 7ff412000640 1 -- 172.21.15.67:0/1102563831 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41c08f070 con 0x7ff4140df800 2024-05-30T16:51:23.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.490+0000 7ff412000640 1 -- 172.21.15.67:0/1102563831 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c073c10 con 0x7ff4140df800 2024-05-30T16:51:23.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.490+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4140d7b70 con 0x7ff4140df800 2024-05-30T16:51:23.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.490+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4140d8130 con 0x7ff4140df800 2024-05-30T16:51:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.494+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4140385a0 con 0x7ff4140df800 2024-05-30T16:51:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.499+0000 7ff412000640 1 -- 172.21.15.67:0/1102563831 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 16) ==== 91601+0+0 (secure 0 0 0) 0x7ff41c0b0890 con 0x7ff4140df800 2024-05-30T16:51:23.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.501+0000 7ff412000640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff3f806cff0 0x7ff3f806f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:23.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.502+0000 7ff412000640 1 -- 172.21.15.67:0/1102563831 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7ff41c107960 con 0x7ff4140df800 2024-05-30T16:51:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.519+0000 7ff412000640 1 -- 172.21.15.67:0/1102563831 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4140385a0 con 0x7ff4140df800 2024-05-30T16:51:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.558+0000 7ff419e00640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff3f806cff0 0x7ff3f806f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:23.562+0000 7ff419e00640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff3f806cff0 0x7ff3f806f4b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff40c007fd0 tx=0x7ff40c007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:24.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.245+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff414005390 con 0x7ff4140df800 2024-05-30T16:51:24.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.247+0000 7ff412000640 1 -- 172.21.15.67:0/1102563831 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2137 (secure 0 0 0) 0x7ff41c0d4e90 con 0x7ff4140df800 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:51:24.248 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 477, 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:24.249 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 50, 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:51:24.250 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2652, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 10994649307, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 22061363200, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 751032750080, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 29345873, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 50672447, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 7, 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 16 2024-05-30T16:51:24.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 26, 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 3, 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 3, 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5361 2024-05-30T16:51:24.252 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:51:24.253 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 8, 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:49:38.113647+0000", 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: "de99db33-4236-4531-bbc4-b658dcb5478d": { 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "2 ongoing clones; Avg progress is 1.562% (1s)\n [............................] (remaining: 88s)", 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.015619999729096889, 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:24.254 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff3f806cff0 msgr2=0x7ff3f806f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff3f806cff0 0x7ff3f806f4b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff40c007fd0 tx=0x7ff40c007de0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4140df800 msgr2=0x7ff414177fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4140df800 0x7ff414177fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000b60 tx=0x7ff41c06bcd0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 shutdown_connections 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff3f806cff0 0x7ff3f806f4b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4140df800 0x7ff414177fb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff414054200 0x7ff41404afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 --2- 172.21.15.67:0/1102563831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff414006720 0x7ff4140dfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.252+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 >> 172.21.15.67:0/1102563831 conn(0x7ff414009fb0 msgr2=0x7ff41400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.254+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 shutdown_connections 2024-05-30T16:51:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.254+0000 7ff41b200640 1 -- 172.21.15.67:0/1102563831 wait complete. 2024-05-30T16:51:24.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone75831 2024-05-30T16:51:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.708+0000 7f088e200640 1 Processor -- start 2024-05-30T16:51:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.708+0000 7f088e200640 1 -- start start 2024-05-30T16:51:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.713+0000 7f088e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f088815e870 0x7f08880d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.713+0000 7f088e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880cfa40 0x7f08880d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.713+0000 7f088e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f088815ec50 con 0x7f08880d14d0 2024-05-30T16:51:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.713+0000 7f088e200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f08880d2150 con 0x7f088815e870 2024-05-30T16:51:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.713+0000 7f088e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08880cff80 con 0x7f08880cfa40 2024-05-30T16:51:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.714+0000 7f088ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f088815e870 0x7f08880d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.714+0000 7f088ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f088815e870 0x7f08880d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:41608/0 (socket says 172.21.15.67:41608) 2024-05-30T16:51:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.714+0000 7f088ce00640 1 -- 172.21.15.67:0/2707764114 learned_addr learned my addr 172.21.15.67:0/2707764114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.714+0000 7f0887e00640 1 --1- 172.21.15.67:0/2707764114 >> v1:172.21.15.169:6789/0 conn(0x7f08880d14d0 0x7f08880cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:52938/0 (socket says 172.21.15.67:52938) 2024-05-30T16:51:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.715+0000 7f088d800640 1 --2- 172.21.15.67:0/2707764114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880cfa40 0x7f08880d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.716+0000 7f088d800640 1 -- 172.21.15.67:0/2707764114 >> v2:172.21.15.169:3300/0 conn(0x7f088815e870 msgr2=0x7f08880d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.716+0000 7f088d800640 1 --2- 172.21.15.67:0/2707764114 >> v2:172.21.15.169:3300/0 conn(0x7f088815e870 0x7f08880d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.716+0000 7f088d800640 1 -- 172.21.15.67:0/2707764114 >> v1:172.21.15.169:6789/0 conn(0x7f08880d14d0 legacy=0x7f08880cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.716+0000 7f088d800640 1 -- 172.21.15.67:0/2707764114 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08880d0260 con 0x7f08880cfa40 2024-05-30T16:51:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.719+0000 7f088d800640 1 --2- 172.21.15.67:0/2707764114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880cfa40 0x7f08880d25e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0880009d90 tx=0x7f088002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81358d1fc0ddc4c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.720+0000 7f0887400640 1 -- 172.21.15.67:0/2707764114 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880007740 con 0x7f08880cfa40 2024-05-30T16:51:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.720+0000 7f0887400640 1 -- 172.21.15.67:0/2707764114 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08800078e0 con 0x7f08880cfa40 2024-05-30T16:51:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.720+0000 7f0887400640 1 -- 172.21.15.67:0/2707764114 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880007cf0 con 0x7f08880cfa40 2024-05-30T16:51:24.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.721+0000 7f088e200640 1 -- 172.21.15.67:0/2707764114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880cfa40 msgr2=0x7f08880d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.721+0000 7f088e200640 1 --2- 172.21.15.67:0/2707764114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880cfa40 0x7f08880d25e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0880009d90 tx=0x7f088002a680 comp rx=0 tx=0).stop 2024-05-30T16:51:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.722+0000 7f088e200640 1 -- 172.21.15.67:0/2707764114 shutdown_connections 2024-05-30T16:51:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.722+0000 7f088e200640 1 --2- 172.21.15.67:0/2707764114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880cfa40 0x7f08880d25e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.722+0000 7f088e200640 1 --2- 172.21.15.67:0/2707764114 >> v2:172.21.15.169:3300/0 conn(0x7f088815e870 0x7f08880d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.722+0000 7f088e200640 1 -- 172.21.15.67:0/2707764114 >> 172.21.15.67:0/2707764114 conn(0x7f088808bfc0 msgr2=0x7f08880d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.724+0000 7f088e200640 1 -- 172.21.15.67:0/2707764114 shutdown_connections 2024-05-30T16:51:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.724+0000 7f088e200640 1 -- 172.21.15.67:0/2707764114 wait complete. 2024-05-30T16:51:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.727+0000 7f088e200640 1 Processor -- start 2024-05-30T16:51:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.727+0000 7f088e200640 1 -- start start 2024-05-30T16:51:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.728+0000 7f088e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08880cfa40 0x7f08880b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:24.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.728+0000 7f088e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880d14d0 0x7f088815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:24.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.728+0000 7f088e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f088815e870 0x7f088815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:24.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.728+0000 7f088e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f08880d4fb0 con 0x7f08880cfa40 2024-05-30T16:51:24.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.728+0000 7f088e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08880d0820 con 0x7f088815e870 2024-05-30T16:51:24.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.728+0000 7f088e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08880d2b20 con 0x7f08880d14d0 2024-05-30T16:51:24.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.729+0000 7f088ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08880cfa40 0x7f08880b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:24.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.729+0000 7f088ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08880cfa40 0x7f08880b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33986/0 (socket says 172.21.15.67:33986) 2024-05-30T16:51:24.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.729+0000 7f088ce00640 1 -- 172.21.15.67:0/1117341586 learned_addr learned my addr 172.21.15.67:0/1117341586 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:24.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.729+0000 7f088d800640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f088815e870 0x7f088815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:24.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.729+0000 7f0887e00640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880d14d0 0x7f088815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:24.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.730+0000 7f088ce00640 1 -- 172.21.15.67:0/1117341586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880d14d0 msgr2=0x7f088815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:24.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.730+0000 7f088ce00640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880d14d0 0x7f088815ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.730+0000 7f088ce00640 1 -- 172.21.15.67:0/1117341586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f088815e870 msgr2=0x7f088815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.730+0000 7f088ce00640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f088815e870 0x7f088815f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.730+0000 7f088ce00640 1 -- 172.21.15.67:0/1117341586 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08880cbd60 con 0x7f08880cfa40 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.730+0000 7f088ce00640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08880cfa40 0x7f08880b2c80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f087c00e840 tx=0x7f087c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.731+0000 7f0884c00640 1 -- 172.21.15.67:0/1117341586 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c017070 con 0x7f08880cfa40 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.731+0000 7f0884c00640 1 -- 172.21.15.67:0/1117341586 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087c00c8e0 con 0x7f08880cfa40 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.731+0000 7f0884c00640 1 -- 172.21.15.67:0/1117341586 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c00cc90 con 0x7f08880cfa40 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.731+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08880b34d0 con 0x7f08880cfa40 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.732+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08880b3a90 con 0x7f08880cfa40 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.733+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08880689e0 con 0x7f08880cfa40 2024-05-30T16:51:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.734+0000 7f0884c00640 1 -- 172.21.15.67:0/1117341586 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 91601+0+0 (secure 0 0 0) 0x7f087c01a070 con 0x7f08880cfa40 2024-05-30T16:51:24.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.736+0000 7f0884c00640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086806cf70 0x7f086806f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:24.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.736+0000 7f0884c00640 1 -- 172.21.15.67:0/1117341586 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f087c00ce30 con 0x7f08880cfa40 2024-05-30T16:51:24.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.747+0000 7f0884c00640 1 -- 172.21.15.67:0/1117341586 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f087c05a0c0 con 0x7f08880cfa40 2024-05-30T16:51:24.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.780+0000 7f0887e00640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086806cf70 0x7f086806f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:24.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:24.793+0000 7f0887e00640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086806cf70 0x7f086806f430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f08881532f0 tx=0x7f0874007c40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:25.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:25.330+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone75831", "target": ["mon-mgr", ""]}) -- 0x7f08880ba500 con 0x7f086806cf70 2024-05-30T16:51:25.487 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:25.486+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:25.487 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:25.486+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:25.487 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:25.486+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:25.487 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:25.486+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:25.487 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:51:25.486+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:51:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:26.423+0000 7f0884c00640 1 -- 172.21.15.67:0/1117341586 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f087c056030 con 0x7f08880cfa40 2024-05-30T16:51:27.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.024+0000 7f0884c00640 1 -- 172.21.15.67:0/1117341586 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f08880ba500 con 0x7f086806cf70 2024-05-30T16:51:27.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.038+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086806cf70 msgr2=0x7f086806f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:27.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.038+0000 7f088e200640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086806cf70 0x7f086806f430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f08881532f0 tx=0x7f0874007c40 comp rx=0 tx=0).stop 2024-05-30T16:51:27.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.038+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08880cfa40 msgr2=0x7f08880b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:27.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.038+0000 7f088e200640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08880cfa40 0x7f08880b2c80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f087c00e840 tx=0x7f087c00ed00 comp rx=0 tx=0).stop 2024-05-30T16:51:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.039+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 shutdown_connections 2024-05-30T16:51:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.039+0000 7f088e200640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f086806cf70 0x7f086806f430 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.039+0000 7f088e200640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f088815e870 0x7f088815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.039+0000 7f088e200640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08880d14d0 0x7f088815ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.039+0000 7f088e200640 1 --2- 172.21.15.67:0/1117341586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08880cfa40 0x7f08880b2c80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.039+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 >> 172.21.15.67:0/1117341586 conn(0x7f088808bfc0 msgr2=0x7f08880b71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:27.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.039+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 shutdown_connections 2024-05-30T16:51:27.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.040+0000 7f088e200640 1 -- 172.21.15.67:0/1117341586 wait complete. 2024-05-30T16:51:27.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone75832 2024-05-30T16:51:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.495+0000 7fe797a00640 1 Processor -- start 2024-05-30T16:51:27.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.495+0000 7fe797a00640 1 -- start start 2024-05-30T16:51:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.498+0000 7fe797a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe79015e850 0x7fe7900b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.499+0000 7fe797a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7900b8000 con 0x7fe7900d1d50 2024-05-30T16:51:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.499+0000 7fe797a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7900b82e0 con 0x7fe7900cef30 2024-05-30T16:51:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.499+0000 7fe797a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe79015ec30 con 0x7fe79015e850 2024-05-30T16:51:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.499+0000 7fe795a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe7900d1d50 0x7fe7900b90a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34394/0 (socket says 172.21.15.67:34394) 2024-05-30T16:51:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.499+0000 7fe795a00640 1 -- 172.21.15.67:0/2970096136 learned_addr learned my addr 172.21.15.67:0/2970096136 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:51:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.500+0000 7fe795000640 1 --2- 172.21.15.67:0/2970096136 >> v2:172.21.15.169:3300/0 conn(0x7fe79015e850 0x7fe7900b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.501+0000 7fe78f400640 1 -- 172.21.15.67:0/2970096136 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1251318374 0 0) 0x7fe7900b8000 con 0x7fe7900d1d50 2024-05-30T16:51:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.501+0000 7fe78f400640 1 -- 172.21.15.67:0/2970096136 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe778003520 con 0x7fe7900d1d50 2024-05-30T16:51:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.502+0000 7fe78f400640 1 -- 172.21.15.67:0/2970096136 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2377682614 0 0) 0x7fe7900b82e0 con 0x7fe7900cef30 2024-05-30T16:51:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.502+0000 7fe795000640 1 -- 172.21.15.67:0/2970096136 >> v1:172.21.15.169:6789/0 conn(0x7fe7900cef30 legacy=0x7fe7900d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.502+0000 7fe795000640 1 -- 172.21.15.67:0/2970096136 >> v1:172.21.15.67:6789/0 conn(0x7fe7900d1d50 legacy=0x7fe7900b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.502+0000 7fe795000640 1 -- 172.21.15.67:0/2970096136 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7900b85c0 con 0x7fe79015e850 2024-05-30T16:51:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.507+0000 7fe795000640 1 --2- 172.21.15.67:0/2970096136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79015e850 0x7fe7900b77b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097a0 tx=0x7fe78402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=743b83510d6827e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.508+0000 7fe78f400640 1 -- 172.21.15.67:0/2970096136 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784037070 con 0x7fe79015e850 2024-05-30T16:51:27.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.508+0000 7fe78f400640 1 -- 172.21.15.67:0/2970096136 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78402ab90 con 0x7fe79015e850 2024-05-30T16:51:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.508+0000 7fe78f400640 1 -- 172.21.15.67:0/2970096136 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7880019f0 con 0x7fe79015e850 2024-05-30T16:51:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.510+0000 7fe797a00640 1 -- 172.21.15.67:0/2970096136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79015e850 msgr2=0x7fe7900b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.510+0000 7fe797a00640 1 --2- 172.21.15.67:0/2970096136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79015e850 0x7fe7900b77b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097a0 tx=0x7fe78402a100 comp rx=0 tx=0).stop 2024-05-30T16:51:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.519+0000 7fe797a00640 1 -- 172.21.15.67:0/2970096136 shutdown_connections 2024-05-30T16:51:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.519+0000 7fe797a00640 1 --2- 172.21.15.67:0/2970096136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79015e850 0x7fe7900b77b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.519+0000 7fe797a00640 1 -- 172.21.15.67:0/2970096136 >> 172.21.15.67:0/2970096136 conn(0x7fe790067aa0 msgr2=0x7fe790067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.520+0000 7fe797a00640 1 -- 172.21.15.67:0/2970096136 shutdown_connections 2024-05-30T16:51:27.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.524+0000 7fe797a00640 1 -- 172.21.15.67:0/2970096136 wait complete. 2024-05-30T16:51:27.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.534+0000 7fe797a00640 1 Processor -- start 2024-05-30T16:51:27.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.535+0000 7fe797a00640 1 -- start start 2024-05-30T16:51:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.540+0000 7fe797a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7900cef30 0x7fe7901561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.540+0000 7fe797a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7900d1d50 0x7fe790159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.540+0000 7fe797a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79015e850 0x7fe790159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.541+0000 7fe797a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe7900b97b0 con 0x7fe7900cef30 2024-05-30T16:51:27.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.541+0000 7fe797a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7900b7cf0 con 0x7fe79015e850 2024-05-30T16:51:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.541+0000 7fe797a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7900b9930 con 0x7fe7900d1d50 2024-05-30T16:51:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.541+0000 7fe795000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7900cef30 0x7fe7901561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.541+0000 7fe795000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7900cef30 0x7fe7901561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33998/0 (socket says 172.21.15.67:33998) 2024-05-30T16:51:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.541+0000 7fe795000640 1 -- 172.21.15.67:0/604071229 learned_addr learned my addr 172.21.15.67:0/604071229 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.542+0000 7fe795a00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79015e850 0x7fe790159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.542+0000 7fe78fe00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7900d1d50 0x7fe790159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.543+0000 7fe795000640 1 -- 172.21.15.67:0/604071229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7900d1d50 msgr2=0x7fe790159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.543+0000 7fe795000640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7900d1d50 0x7fe790159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.543+0000 7fe795000640 1 -- 172.21.15.67:0/604071229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79015e850 msgr2=0x7fe790159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.543+0000 7fe795000640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79015e850 0x7fe790159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.543+0000 7fe795000640 1 -- 172.21.15.67:0/604071229 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79015c720 con 0x7fe7900cef30 2024-05-30T16:51:27.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.544+0000 7fe795000640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7900cef30 0x7fe7901561d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe784000c00 tx=0x7fe7840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.544+0000 7fe78cc00640 1 -- 172.21.15.67:0/604071229 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784037040 con 0x7fe7900cef30 2024-05-30T16:51:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.544+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7900cc7c0 con 0x7fe7900cef30 2024-05-30T16:51:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.544+0000 7fe78cc00640 1 -- 172.21.15.67:0/604071229 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe784004030 con 0x7fe7900cef30 2024-05-30T16:51:27.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.549+0000 7fe78cc00640 1 -- 172.21.15.67:0/604071229 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784004400 con 0x7fe7900cef30 2024-05-30T16:51:27.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.550+0000 7fe78cc00640 1 -- 172.21.15.67:0/604071229 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7fe78403a040 con 0x7fe7900cef30 2024-05-30T16:51:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.551+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79015c9e0 con 0x7fe7900cef30 2024-05-30T16:51:27.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.554+0000 7fe78cc00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77006cf90 0x7fe77006f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:27.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.560+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7900689e0 con 0x7fe7900cef30 2024-05-30T16:51:27.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.561+0000 7fe78fe00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77006cf90 0x7fe77006f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:27.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.570+0000 7fe78cc00640 1 -- 172.21.15.67:0/604071229 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fe7840778a0 con 0x7fe7900cef30 2024-05-30T16:51:27.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.589+0000 7fe78cc00640 1 -- 172.21.15.67:0/604071229 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe784042020 con 0x7fe7900cef30 2024-05-30T16:51:27.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:27.610+0000 7fe78fe00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77006cf90 0x7fe77006f450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe7900cc0a0 tx=0x7fe77c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:28.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.172+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone75832", "target": ["mon-mgr", ""]}) -- 0x7fe790156710 con 0x7fe77006cf90 2024-05-30T16:51:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.992+0000 7fe78cc00640 1 -- 172.21.15.67:0/604071229 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe790156710 con 0x7fe77006cf90 2024-05-30T16:51:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77006cf90 msgr2=0x7fe77006f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77006cf90 0x7fe77006f450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe7900cc0a0 tx=0x7fe77c009040 comp rx=0 tx=0).stop 2024-05-30T16:51:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7900cef30 msgr2=0x7fe7901561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7900cef30 0x7fe7901561d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe784000c00 tx=0x7fe7840046c0 comp rx=0 tx=0).stop 2024-05-30T16:51:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 shutdown_connections 2024-05-30T16:51:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77006cf90 0x7fe77006f450 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79015e850 0x7fe790159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7900d1d50 0x7fe790159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 --2- 172.21.15.67:0/604071229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7900cef30 0x7fe7901561d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:28.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.995+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 >> 172.21.15.67:0/604071229 conn(0x7fe790067aa0 msgr2=0x7fe790162190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:28.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.996+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 shutdown_connections 2024-05-30T16:51:28.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:28.996+0000 7fe797a00640 1 -- 172.21.15.67:0/604071229 wait complete. 2024-05-30T16:51:29.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone75833 2024-05-30T16:51:29.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.350+0000 7f107ea00640 1 Processor -- start 2024-05-30T16:51:29.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.350+0000 7f107ea00640 1 -- start start 2024-05-30T16:51:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.353+0000 7f107ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f107815e940 0x7f1078073030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.354+0000 7f107ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10780b2ae0 0x7f10780b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.355+0000 7f1077e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f107815e940 0x7f1078073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:29.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.355+0000 7f1077e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f107815e940 0x7f1078073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:41658/0 (socket says 172.21.15.67:41658) 2024-05-30T16:51:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.357+0000 7f1077400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f10780b32c0 0x7f1078073570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34396/0 (socket says 172.21.15.67:34396) 2024-05-30T16:51:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.357+0000 7f1077e00640 1 -- 172.21.15.67:0/4024890046 learned_addr learned my addr 172.21.15.67:0/4024890046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.357+0000 7f107ea00640 1 -- 172.21.15.67:0/4024890046 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10780afdf0 con 0x7f10780b32c0 2024-05-30T16:51:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.357+0000 7f107ea00640 1 -- 172.21.15.67:0/4024890046 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f10780b0060 con 0x7f107815e940 2024-05-30T16:51:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.357+0000 7f107ea00640 1 -- 172.21.15.67:0/4024890046 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10780b0340 con 0x7f10780b2ae0 2024-05-30T16:51:29.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.357+0000 7f107ca00640 1 --2- 172.21.15.67:0/4024890046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10780b2ae0 0x7f10780b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:29.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.358+0000 7f1076a00640 1 -- 172.21.15.67:0/4024890046 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 73416153 0 0) 0x7f10780afdf0 con 0x7f10780b32c0 2024-05-30T16:51:29.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.359+0000 7f1076a00640 1 -- 172.21.15.67:0/4024890046 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f105c003520 con 0x7f10780b32c0 2024-05-30T16:51:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.359+0000 7f1077e00640 1 -- 172.21.15.67:0/4024890046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10780b2ae0 msgr2=0x7f10780b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.359+0000 7f1077e00640 1 --2- 172.21.15.67:0/4024890046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10780b2ae0 0x7f10780b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.360+0000 7f1077e00640 1 -- 172.21.15.67:0/4024890046 >> v1:172.21.15.67:6789/0 conn(0x7f10780b32c0 legacy=0x7f1078073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:29.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.360+0000 7f1077e00640 1 -- 172.21.15.67:0/4024890046 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780b0620 con 0x7f107815e940 2024-05-30T16:51:29.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.363+0000 7f1077e00640 1 --2- 172.21.15.67:0/4024890046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107815e940 0x7f1078073030 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1064013db0 tx=0x7f10640328c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130f0e9d36d5273b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:29.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.364+0000 7f1076a00640 1 -- 172.21.15.67:0/4024890046 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106403f070 con 0x7f107815e940 2024-05-30T16:51:29.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.366+0000 7f1076a00640 1 -- 172.21.15.67:0/4024890046 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1064038500 con 0x7f107815e940 2024-05-30T16:51:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.368+0000 7f107ea00640 1 -- 172.21.15.67:0/4024890046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107815e940 msgr2=0x7f1078073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:29.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.368+0000 7f107ea00640 1 --2- 172.21.15.67:0/4024890046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107815e940 0x7f1078073030 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1064013db0 tx=0x7f10640328c0 comp rx=0 tx=0).stop 2024-05-30T16:51:29.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.369+0000 7f107ea00640 1 -- 172.21.15.67:0/4024890046 shutdown_connections 2024-05-30T16:51:29.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.369+0000 7f107ea00640 1 --2- 172.21.15.67:0/4024890046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10780b2ae0 0x7f10780b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:29.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.369+0000 7f107ea00640 1 --2- 172.21.15.67:0/4024890046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107815e940 0x7f1078073030 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:29.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.369+0000 7f107ea00640 1 -- 172.21.15.67:0/4024890046 >> 172.21.15.67:0/4024890046 conn(0x7f10780b1fc0 msgr2=0x7f10780b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:29.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.369+0000 7f107ea00640 1 -- 172.21.15.67:0/4024890046 shutdown_connections 2024-05-30T16:51:29.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.369+0000 7f107ea00640 1 -- 172.21.15.67:0/4024890046 wait complete. 2024-05-30T16:51:29.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.374+0000 7f107ea00640 1 Processor -- start 2024-05-30T16:51:29.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.375+0000 7f107ea00640 1 -- start start 2024-05-30T16:51:29.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.376+0000 7f107ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10780b2ae0 0x7f107814dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.376+0000 7f107ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107815e940 0x7f107809b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.376+0000 7f107ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107809b730 0x7f1078152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.376+0000 7f107ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f10780ad420 con 0x7f10780b2ae0 2024-05-30T16:51:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.376+0000 7f107ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10780ad2a0 con 0x7f107809b730 2024-05-30T16:51:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.376+0000 7f107ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1078073c80 con 0x7f107815e940 2024-05-30T16:51:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.377+0000 7f1077e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10780b2ae0 0x7f107814dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.377+0000 7f1077e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10780b2ae0 0x7f107814dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34002/0 (socket says 172.21.15.67:34002) 2024-05-30T16:51:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.377+0000 7f1077e00640 1 -- 172.21.15.67:0/3737901750 learned_addr learned my addr 172.21.15.67:0/3737901750 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.377+0000 7f1077400640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107815e940 0x7f107809b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.377+0000 7f107ca00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107809b730 0x7f1078152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.378+0000 7f1077e00640 1 -- 172.21.15.67:0/3737901750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107815e940 msgr2=0x7f107809b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.378+0000 7f1077e00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107815e940 0x7f107809b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.378+0000 7f1077e00640 1 -- 172.21.15.67:0/3737901750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107809b730 msgr2=0x7f1078152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.378+0000 7f1077e00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107809b730 0x7f1078152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.378+0000 7f1077e00640 1 -- 172.21.15.67:0/3737901750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107809d240 con 0x7f10780b2ae0 2024-05-30T16:51:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.379+0000 7f1077e00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10780b2ae0 0x7f107814dae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1064013db0 tx=0x7f10640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:29.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.379+0000 7f106fe00640 1 -- 172.21.15.67:0/3737901750 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064049070 con 0x7f10780b2ae0 2024-05-30T16:51:29.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.379+0000 7f106fe00640 1 -- 172.21.15.67:0/3737901750 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106403a960 con 0x7f10780b2ae0 2024-05-30T16:51:29.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.380+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107809d520 con 0x7f10780b2ae0 2024-05-30T16:51:29.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.380+0000 7f106fe00640 1 -- 172.21.15.67:0/3737901750 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106403f040 con 0x7f10780b2ae0 2024-05-30T16:51:29.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.380+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107809dae0 con 0x7f10780b2ae0 2024-05-30T16:51:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.385+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10780689e0 con 0x7f10780b2ae0 2024-05-30T16:51:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.389+0000 7f106fe00640 1 -- 172.21.15.67:0/3737901750 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f106404c070 con 0x7f10780b2ae0 2024-05-30T16:51:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.391+0000 7f106fe00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f105806d000 0x7f105806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:29.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.391+0000 7f106fe00640 1 -- 172.21.15.67:0/3737901750 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f10640b5220 con 0x7f10780b2ae0 2024-05-30T16:51:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.398+0000 7f1077400640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f105806d000 0x7f105806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:29.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.400+0000 7f106fe00640 1 -- 172.21.15.67:0/3737901750 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1064037070 con 0x7f10780b2ae0 2024-05-30T16:51:29.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.402+0000 7f1077400640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f105806d000 0x7f105806f4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1068001600 tx=0x7f1068001570 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:29.872+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone75833", "target": ["mon-mgr", ""]}) -- 0x7f10780b0ee0 con 0x7f105806d000 2024-05-30T16:51:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.369+0000 7f106fe00640 1 -- 172.21.15.67:0/3737901750 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10780b0ee0 con 0x7f105806d000 2024-05-30T16:51:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.372+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f105806d000 msgr2=0x7f105806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.372+0000 7f107ea00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f105806d000 0x7f105806f4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1068001600 tx=0x7f1068001570 comp rx=0 tx=0).stop 2024-05-30T16:51:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.372+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10780b2ae0 msgr2=0x7f107814dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.372+0000 7f107ea00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10780b2ae0 0x7f107814dae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1064013db0 tx=0x7f10640046c0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.373+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 shutdown_connections 2024-05-30T16:51:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.373+0000 7f107ea00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f105806d000 0x7f105806f4c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.373+0000 7f107ea00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f107809b730 0x7f1078152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.373+0000 7f107ea00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f107815e940 0x7f107809b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.373+0000 7f107ea00640 1 --2- 172.21.15.67:0/3737901750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10780b2ae0 0x7f107814dae0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.373+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 >> 172.21.15.67:0/3737901750 conn(0x7f10780b1fc0 msgr2=0x7f10780b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.374+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 shutdown_connections 2024-05-30T16:51:30.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.374+0000 7f107ea00640 1 -- 172.21.15.67:0/3737901750 wait complete. 2024-05-30T16:51:30.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone75834 2024-05-30T16:51:30.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.701+0000 7f87f7e00640 1 Processor -- start 2024-05-30T16:51:30.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.701+0000 7f87f7e00640 1 -- start start 2024-05-30T16:51:30.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.703+0000 7f87f7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f87f015e850 0x7f87f00b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:30.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.704+0000 7f87f7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87f00d1d50 0x7f87f00b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:30.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.704+0000 7f87f7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f00b8000 con 0x7f87f00cef30 2024-05-30T16:51:30.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.704+0000 7f87f7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f87f015ec30 con 0x7f87f00d1d50 2024-05-30T16:51:30.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.704+0000 7f87f7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f87f00b82e0 con 0x7f87f015e850 2024-05-30T16:51:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.704+0000 7f87f5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87f00d1d50 0x7f87f00b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.704+0000 7f87f5400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f87f015e850 0x7f87f00b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.704+0000 7f87f5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87f00d1d50 0x7f87f00b90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34006/0 (socket says 172.21.15.67:34006) 2024-05-30T16:51:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.704+0000 7f87f5e00640 1 -- 172.21.15.67:0/1064444427 learned_addr learned my addr 172.21.15.67:0/1064444427 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.705+0000 7f87f5e00640 1 -- 172.21.15.67:0/1064444427 >> v2:172.21.15.169:3300/0 conn(0x7f87f015e850 msgr2=0x7f87f00b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.705+0000 7f87f5e00640 1 --2- 172.21.15.67:0/1064444427 >> v2:172.21.15.169:3300/0 conn(0x7f87f015e850 0x7f87f00b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.705+0000 7f87f5e00640 1 -- 172.21.15.67:0/1064444427 >> v1:172.21.15.169:6789/0 conn(0x7f87f00cef30 legacy=0x7f87f00d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:30.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.705+0000 7f87f5e00640 1 -- 172.21.15.67:0/1064444427 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f00b85c0 con 0x7f87f00d1d50 2024-05-30T16:51:30.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.707+0000 7f87f5e00640 1 --2- 172.21.15.67:0/1064444427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f00d1d50 0x7f87f00b90a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f87f006dba0 tx=0x7f87e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f403e4b1872abc3e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:30.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.707+0000 7f87efe00640 1 -- 172.21.15.67:0/1064444427 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8036070 con 0x7f87f00d1d50 2024-05-30T16:51:30.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.707+0000 7f87efe00640 1 -- 172.21.15.67:0/1064444427 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e8031600 con 0x7f87f00d1d50 2024-05-30T16:51:30.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.707+0000 7f87efe00640 1 -- 172.21.15.67:0/1064444427 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80319b0 con 0x7f87f00d1d50 2024-05-30T16:51:30.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.708+0000 7f87f7e00640 1 -- 172.21.15.67:0/1064444427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f00d1d50 msgr2=0x7f87f00b90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:30.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.708+0000 7f87f7e00640 1 --2- 172.21.15.67:0/1064444427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f00d1d50 0x7f87f00b90a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f87f006dba0 tx=0x7f87e802a6f0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.709+0000 7f87f7e00640 1 -- 172.21.15.67:0/1064444427 shutdown_connections 2024-05-30T16:51:30.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.709+0000 7f87f7e00640 1 --2- 172.21.15.67:0/1064444427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f00d1d50 0x7f87f00b90a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.709+0000 7f87f7e00640 1 --2- 172.21.15.67:0/1064444427 >> v2:172.21.15.169:3300/0 conn(0x7f87f015e850 0x7f87f00b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.709+0000 7f87f7e00640 1 -- 172.21.15.67:0/1064444427 >> 172.21.15.67:0/1064444427 conn(0x7f87f0067aa0 msgr2=0x7f87f0067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:30.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.709+0000 7f87f7e00640 1 -- 172.21.15.67:0/1064444427 shutdown_connections 2024-05-30T16:51:30.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.709+0000 7f87f7e00640 1 -- 172.21.15.67:0/1064444427 wait complete. 2024-05-30T16:51:30.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.712+0000 7f87f7e00640 1 Processor -- start 2024-05-30T16:51:30.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.712+0000 7f87f7e00640 1 -- start start 2024-05-30T16:51:30.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.714+0000 7f87f7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87f00cef30 0x7f87f014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:30.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.714+0000 7f87f7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00d1d50 0x7f87f00cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:30.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.714+0000 7f87f7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f015e850 0x7f87f00cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:30.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.714+0000 7f87f7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87f00b95e0 con 0x7f87f015e850 2024-05-30T16:51:30.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.714+0000 7f87f7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f87f00b7cf0 con 0x7f87f00d1d50 2024-05-30T16:51:30.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.714+0000 7f87f7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f87f00b9760 con 0x7f87f00cef30 2024-05-30T16:51:30.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.715+0000 7f87f5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87f00cef30 0x7f87f014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:30.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.715+0000 7f87f5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87f00cef30 0x7f87f014cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:44034/0 (socket says 172.21.15.67:44034) 2024-05-30T16:51:30.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.715+0000 7f87f5400640 1 -- 172.21.15.67:0/1906885365 learned_addr learned my addr 172.21.15.67:0/1906885365 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:30.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.715+0000 7f87f5400640 1 -- 172.21.15.67:0/1906885365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00d1d50 msgr2=0x7f87f00cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:30.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.715+0000 7f87f5400640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00d1d50 0x7f87f00cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.715+0000 7f87f5400640 1 -- 172.21.15.67:0/1906885365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f015e850 msgr2=0x7f87f00cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:30.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.716+0000 7f87f5400640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f015e850 0x7f87f00cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:30.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.716+0000 7f87f5400640 1 -- 172.21.15.67:0/1906885365 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f00ce350 con 0x7f87f00cef30 2024-05-30T16:51:30.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.717+0000 7f87f5400640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87f00cef30 0x7f87f014cd10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f87e400e870 tx=0x7f87e400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:30.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.718+0000 7f87ed600640 1 -- 172.21.15.67:0/1906885365 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4017070 con 0x7f87f00cef30 2024-05-30T16:51:30.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.718+0000 7f87ed600640 1 -- 172.21.15.67:0/1906885365 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e400c940 con 0x7f87f00cef30 2024-05-30T16:51:30.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.719+0000 7f87ed600640 1 -- 172.21.15.67:0/1906885365 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4017070 con 0x7f87f00cef30 2024-05-30T16:51:30.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.719+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f00ce630 con 0x7f87f00cef30 2024-05-30T16:51:30.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.721+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f00cebf0 con 0x7f87f00cef30 2024-05-30T16:51:30.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.722+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f00689e0 con 0x7f87f00cef30 2024-05-30T16:51:30.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.723+0000 7f87ed600640 1 -- 172.21.15.67:0/1906885365 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f87e401a070 con 0x7f87f00cef30 2024-05-30T16:51:30.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.725+0000 7f87ed600640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f87d006d1b0 0x7f87d006f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:30.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.725+0000 7f87ed600640 1 -- 172.21.15.67:0/1906885365 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f87e408bd10 con 0x7f87f00cef30 2024-05-30T16:51:30.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.729+0000 7f87f4a00640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f87d006d1b0 0x7f87d006f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:30.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.731+0000 7f87f4a00640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f87d006d1b0 0x7f87d006f670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000f60 tx=0x7f87dc0079b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:30.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:30.744+0000 7f87ed600640 1 -- 172.21.15.67:0/1906885365 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f87e4059770 con 0x7f87f00cef30 2024-05-30T16:51:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.095+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone75834", "target": ["mon-mgr", ""]}) -- 0x7f87f014d980 con 0x7f87d006d1b0 2024-05-30T16:51:31.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.287+0000 7f87ed600640 1 -- 172.21.15.67:0/1906885365 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f87f014d980 con 0x7f87d006d1b0 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f87d006d1b0 msgr2=0x7f87d006f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f87d006d1b0 0x7f87d006f670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000f60 tx=0x7f87dc0079b0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87f00cef30 msgr2=0x7f87f014cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87f00cef30 0x7f87f014cd10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f87e400e870 tx=0x7f87e400ed30 comp rx=0 tx=0).stop 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 shutdown_connections 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f87d006d1b0 0x7f87d006f670 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87f015e850 0x7f87f00cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00d1d50 0x7f87f00cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 --2- 172.21.15.67:0/1906885365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87f00cef30 0x7f87f014cd10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 >> 172.21.15.67:0/1906885365 conn(0x7f87f0067aa0 msgr2=0x7f87f00c8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 shutdown_connections 2024-05-30T16:51:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.290+0000 7f87f7e00640 1 -- 172.21.15.67:0/1906885365 wait complete. 2024-05-30T16:51:31.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:51:31.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.504+0000 7f7ad2a00640 1 Processor -- start 2024-05-30T16:51:31.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.505+0000 7f7ad2a00640 1 -- start start 2024-05-30T16:51:31.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.506+0000 7f7ad2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7acc158b80 0x7f7acc06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.506+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc06e160 0x7f7acc071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.506+0000 7f7ad2a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7acc06e6e0 con 0x7f7acc072100 2024-05-30T16:51:31.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.506+0000 7f7ad2a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7acc0b42d0 con 0x7f7acc158b80 2024-05-30T16:51:31.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.506+0000 7f7ad2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc06e9c0 con 0x7f7acc06e160 2024-05-30T16:51:31.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.506+0000 7f7ad1600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7acc158b80 0x7f7acc06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.506+0000 7f7ad1600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7acc158b80 0x7f7acc06dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51302/0 (socket says 172.21.15.67:51302) 2024-05-30T16:51:31.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.507+0000 7f7ad1600640 1 -- 172.21.15.67:0/1132737619 learned_addr learned my addr 172.21.15.67:0/1132737619 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:31.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.507+0000 7f7ad1600640 1 -- 172.21.15.67:0/1132737619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc06e160 msgr2=0x7f7acc071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:31.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.507+0000 7f7ad1600640 1 --2- 172.21.15.67:0/1132737619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc06e160 0x7f7acc071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.507+0000 7f7ad1600640 1 -- 172.21.15.67:0/1132737619 >> v1:172.21.15.169:6789/0 conn(0x7f7acc072100 legacy=0x7f7acc1621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:31.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.507+0000 7f7ad1600640 1 -- 172.21.15.67:0/1132737619 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc06eca0 con 0x7f7acc158b80 2024-05-30T16:51:31.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.508+0000 7f7ad1600640 1 --2- 172.21.15.67:0/1132737619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 0x7f7acc06dc20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009870 tx=0x7f7ac002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2233ceea65dec49 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:31.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.508+0000 7f7acbe00640 1 -- 172.21.15.67:0/1132737619 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0037070 con 0x7f7acc158b80 2024-05-30T16:51:31.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.508+0000 7f7acbe00640 1 -- 172.21.15.67:0/1132737619 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac0032440 con 0x7f7acc158b80 2024-05-30T16:51:31.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.508+0000 7f7acbe00640 1 -- 172.21.15.67:0/1132737619 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0032810 con 0x7f7acc158b80 2024-05-30T16:51:31.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.509+0000 7f7ad2a00640 1 -- 172.21.15.67:0/1132737619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 msgr2=0x7f7acc06dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.509+0000 7f7ad2a00640 1 --2- 172.21.15.67:0/1132737619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 0x7f7acc06dc20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009870 tx=0x7f7ac002a210 comp rx=0 tx=0).stop 2024-05-30T16:51:31.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.509+0000 7f7ad2a00640 1 -- 172.21.15.67:0/1132737619 shutdown_connections 2024-05-30T16:51:31.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.509+0000 7f7ad2a00640 1 --2- 172.21.15.67:0/1132737619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc06e160 0x7f7acc071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.509+0000 7f7ad2a00640 1 --2- 172.21.15.67:0/1132737619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 0x7f7acc06dc20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.509+0000 7f7ad2a00640 1 -- 172.21.15.67:0/1132737619 >> 172.21.15.67:0/1132737619 conn(0x7f7acc0b1fc0 msgr2=0x7f7acc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:31.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.509+0000 7f7ad2a00640 1 -- 172.21.15.67:0/1132737619 shutdown_connections 2024-05-30T16:51:31.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.509+0000 7f7ad2a00640 1 -- 172.21.15.67:0/1132737619 wait complete. 2024-05-30T16:51:31.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.511+0000 7f7ad2a00640 1 Processor -- start 2024-05-30T16:51:31.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.511+0000 7f7ad2a00640 1 -- start start 2024-05-30T16:51:31.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.512+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc06e160 0x7f7acc09c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.512+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc072100 0x7f7acc1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.512+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 0x7f7acc146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.512+0000 7f7ad2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc0b2d30 con 0x7f7acc158b80 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.512+0000 7f7ad2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc06f260 con 0x7f7acc06e160 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.512+0000 7f7ad2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc06f780 con 0x7f7acc072100 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.512+0000 7f7ad2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 0x7f7acc146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.512+0000 7f7ad2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 0x7f7acc146470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51316/0 (socket says 172.21.15.67:51316) 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.512+0000 7f7ad2000640 1 -- 172.21.15.67:0/500041479 learned_addr learned my addr 172.21.15.67:0/500041479 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ad1600640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc06e160 0x7f7acc09c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ad0c00640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc072100 0x7f7acc1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ad2000640 1 -- 172.21.15.67:0/500041479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc072100 msgr2=0x7f7acc1f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ad2000640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc072100 0x7f7acc1f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ad2000640 1 -- 172.21.15.67:0/500041479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc06e160 msgr2=0x7f7acc09c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ad2000640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc06e160 0x7f7acc09c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ad2000640 1 -- 172.21.15.67:0/500041479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc09a450 con 0x7f7acc158b80 2024-05-30T16:51:31.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ad2000640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 0x7f7acc146470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c780 tx=0x7f7ac400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:31.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ac9600640 1 -- 172.21.15.67:0/500041479 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4004b10 con 0x7f7acc158b80 2024-05-30T16:51:31.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ac9600640 1 -- 172.21.15.67:0/500041479 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac4004cb0 con 0x7f7acc158b80 2024-05-30T16:51:31.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ac9600640 1 -- 172.21.15.67:0/500041479 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400aa90 con 0x7f7acc158b80 2024-05-30T16:51:31.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.513+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7acc09a730 con 0x7f7acc158b80 2024-05-30T16:51:31.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.514+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7acc1469b0 con 0x7f7acc158b80 2024-05-30T16:51:31.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.515+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7acc0689e0 con 0x7f7acc158b80 2024-05-30T16:51:31.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.516+0000 7f7ac9600640 1 -- 172.21.15.67:0/500041479 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f7ac400ac30 con 0x7f7acc158b80 2024-05-30T16:51:31.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.517+0000 7f7ac9600640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ab006cfb0 0x7f7ab006f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.517+0000 7f7ac9600640 1 -- 172.21.15.67:0/500041479 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f7ac408c8b0 con 0x7f7acc158b80 2024-05-30T16:51:31.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.517+0000 7f7ad1600640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ab006cfb0 0x7f7ab006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.518+0000 7f7ad1600640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ab006cfb0 0x7f7ab006f470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0034000 tx=0x7f7ac002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:31.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.522+0000 7f7ac9600640 1 -- 172.21.15.67:0/500041479 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ac4020070 con 0x7f7acc158b80 2024-05-30T16:51:31.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.745+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7acc09c9c0 con 0x7f7ab006cfb0 2024-05-30T16:51:31.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.748+0000 7f7ac9600640 1 -- 172.21.15.67:0/500041479 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7acc09c9c0 con 0x7f7ab006cfb0 2024-05-30T16:51:31.750 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:51:31.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:31.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:51:31.751 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:31.751 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.750+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ab006cfb0 msgr2=0x7f7ab006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.750+0000 7f7ad2a00640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ab006cfb0 0x7f7ab006f470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0034000 tx=0x7f7ac002e000 comp rx=0 tx=0).stop 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.750+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 msgr2=0x7f7acc146470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.750+0000 7f7ad2a00640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 0x7f7acc146470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c780 tx=0x7f7ac400cc40 comp rx=0 tx=0).stop 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.751+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 shutdown_connections 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.751+0000 7f7ad2a00640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ab006cfb0 0x7f7ab006f470 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.751+0000 7f7ad2a00640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7acc158b80 0x7f7acc146470 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.751+0000 7f7ad2a00640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc072100 0x7f7acc1f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.751+0000 7f7ad2a00640 1 --2- 172.21.15.67:0/500041479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc06e160 0x7f7acc09c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.751+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 >> 172.21.15.67:0/500041479 conn(0x7f7acc0b1fc0 msgr2=0x7f7acc1532d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.751+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 shutdown_connections 2024-05-30T16:51:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.751+0000 7f7ad2a00640 1 -- 172.21.15.67:0/500041479 wait complete. 2024-05-30T16:51:31.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:51:31.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.976+0000 7fa50be00640 1 Processor -- start 2024-05-30T16:51:31.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.976+0000 7fa50be00640 1 -- start start 2024-05-30T16:51:31.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.978+0000 7fa50be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa50c162bb0 0x7fa50c0b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.978+0000 7fa50be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa50c0b6130 con 0x7fa50c162f90 2024-05-30T16:51:31.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.978+0000 7fa50be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa50c0aebf0 con 0x7fa50c0af3f0 2024-05-30T16:51:31.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.978+0000 7fa50be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa50c0aeed0 con 0x7fa50c162bb0 2024-05-30T16:51:31.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.979+0000 7fa503e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa50c162f90 0x7fa50c0b41d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42794/0 (socket says 172.21.15.67:42794) 2024-05-30T16:51:31.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.979+0000 7fa503e00640 1 -- 172.21.15.67:0/1093668872 learned_addr learned my addr 172.21.15.67:0/1093668872 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:51:31.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.979+0000 7fa50aa00640 1 --2- 172.21.15.67:0/1093668872 >> v2:172.21.15.67:3300/0 conn(0x7fa50c162bb0 0x7fa50c0b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.979+0000 7fa50a000640 1 -- 172.21.15.67:0/1093668872 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322057086 0 0) 0x7fa50c0b6130 con 0x7fa50c162f90 2024-05-30T16:51:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.979+0000 7fa50a000640 1 -- 172.21.15.67:0/1093668872 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4f0003520 con 0x7fa50c162f90 2024-05-30T16:51:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.979+0000 7fa50aa00640 1 -- 172.21.15.67:0/1093668872 >> v1:172.21.15.169:6789/0 conn(0x7fa50c0af3f0 legacy=0x7fa50c0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.979+0000 7fa50aa00640 1 -- 172.21.15.67:0/1093668872 >> v1:172.21.15.67:6789/0 conn(0x7fa50c162f90 legacy=0x7fa50c0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.979+0000 7fa50aa00640 1 -- 172.21.15.67:0/1093668872 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c0b6410 con 0x7fa50c162bb0 2024-05-30T16:51:31.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.981+0000 7fa50aa00640 1 --2- 172.21.15.67:0/1093668872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c162bb0 0x7fa50c0b0070 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0097a0 tx=0x7fa4fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae95cd2c9f4dd4a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:31.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.981+0000 7fa50a000640 1 -- 172.21.15.67:0/1093668872 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc037070 con 0x7fa50c162bb0 2024-05-30T16:51:31.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.981+0000 7fa50a000640 1 -- 172.21.15.67:0/1093668872 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4fc032440 con 0x7fa50c162bb0 2024-05-30T16:51:31.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.981+0000 7fa50a000640 1 -- 172.21.15.67:0/1093668872 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc032810 con 0x7fa50c162bb0 2024-05-30T16:51:31.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.982+0000 7fa50be00640 1 -- 172.21.15.67:0/1093668872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c162bb0 msgr2=0x7fa50c0b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.982+0000 7fa50be00640 1 --2- 172.21.15.67:0/1093668872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c162bb0 0x7fa50c0b0070 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0097a0 tx=0x7fa4fc02a230 comp rx=0 tx=0).stop 2024-05-30T16:51:31.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.982+0000 7fa50be00640 1 -- 172.21.15.67:0/1093668872 shutdown_connections 2024-05-30T16:51:31.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.982+0000 7fa50be00640 1 --2- 172.21.15.67:0/1093668872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c162bb0 0x7fa50c0b0070 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.982+0000 7fa50be00640 1 -- 172.21.15.67:0/1093668872 >> 172.21.15.67:0/1093668872 conn(0x7fa50c0b66f0 msgr2=0x7fa50c0b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:31.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.982+0000 7fa50be00640 1 -- 172.21.15.67:0/1093668872 shutdown_connections 2024-05-30T16:51:31.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.982+0000 7fa50be00640 1 -- 172.21.15.67:0/1093668872 wait complete. 2024-05-30T16:51:31.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.983+0000 7fa50be00640 1 Processor -- start 2024-05-30T16:51:31.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.983+0000 7fa50be00640 1 -- start start 2024-05-30T16:51:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.984+0000 7fa50be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c0af3f0 0x7fa50c14d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.984+0000 7fa50be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa50c162bb0 0x7fa50c15d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.984+0000 7fa50be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c162f90 0x7fa50c1614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.984+0000 7fa50be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0b1410 con 0x7fa50c0af3f0 2024-05-30T16:51:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.984+0000 7fa50be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0b1290 con 0x7fa50c162f90 2024-05-30T16:51:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.984+0000 7fa50be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa50c0af840 con 0x7fa50c162bb0 2024-05-30T16:51:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c0af3f0 0x7fa50c14d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c0af3f0 0x7fa50c14d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51346/0 (socket says 172.21.15.67:51346) 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50aa00640 1 -- 172.21.15.67:0/242668442 learned_addr learned my addr 172.21.15.67:0/242668442 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa503e00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa50c162bb0 0x7fa50c15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50b400640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c162f90 0x7fa50c1614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50aa00640 1 -- 172.21.15.67:0/242668442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa50c162bb0 msgr2=0x7fa50c15d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50aa00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa50c162bb0 0x7fa50c15d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50aa00640 1 -- 172.21.15.67:0/242668442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c162f90 msgr2=0x7fa50c1614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50aa00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c162f90 0x7fa50c1614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50aa00640 1 -- 172.21.15.67:0/242668442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c1601b0 con 0x7fa50c0af3f0 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.985+0000 7fa50b400640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c162f90 0x7fa50c1614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:51:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.986+0000 7fa50aa00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c0af3f0 0x7fa50c14d5d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc000c00 tx=0x7fa4fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.986+0000 7fa502a00640 1 -- 172.21.15.67:0/242668442 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc037040 con 0x7fa50c0af3f0 2024-05-30T16:51:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.986+0000 7fa502a00640 1 -- 172.21.15.67:0/242668442 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4fc042070 con 0x7fa50c0af3f0 2024-05-30T16:51:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.986+0000 7fa502a00640 1 -- 172.21.15.67:0/242668442 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc030940 con 0x7fa50c0af3f0 2024-05-30T16:51:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.986+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c14db10 con 0x7fa50c0af3f0 2024-05-30T16:51:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.986+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c160710 con 0x7fa50c0af3f0 2024-05-30T16:51:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.987+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50c0690f0 con 0x7fa50c0af3f0 2024-05-30T16:51:31.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.988+0000 7fa502a00640 1 -- 172.21.15.67:0/242668442 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7fa4fc030ae0 con 0x7fa50c0af3f0 2024-05-30T16:51:31.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.989+0000 7fa502a00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa4e806cfe0 0x7fa4e806f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.989+0000 7fa502a00640 1 -- 172.21.15.67:0/242668442 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fa4fc0ad050 con 0x7fa50c0af3f0 2024-05-30T16:51:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.989+0000 7fa503e00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa4e806cfe0 0x7fa4e806f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.990+0000 7fa503e00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa4e806cfe0 0x7fa4e806f4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa50c15a6a0 tx=0x7fa4f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:31.993+0000 7fa502a00640 1 -- 172.21.15.67:0/242668442 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa50c1601b0 con 0x7fa50c0af3f0 2024-05-30T16:51:32.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.272+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa50c072e70 con 0x7fa50c0af3f0 2024-05-30T16:51:32.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.273+0000 7fa502a00640 1 -- 172.21.15.67:0/242668442 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2142 (secure 0 0 0) 0x7fa4fc045360 con 0x7fa50c0af3f0 2024-05-30T16:51:32.275 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:51:32.275 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:51:32.275 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:51:32.275 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:51:32.276 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 485, 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 50, 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:51:32.277 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:51:32.278 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4086, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 16992954564, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 33703489536, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 739390623744, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 529924156, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 520897826, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 129, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 142 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 26, 2024-05-30T16:51:32.279 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 3, 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 3, 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5361 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:32.280 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:51:32.281 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 8, 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:49:38.113647+0000", 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: "de99db33-4236-4531-bbc4-b658dcb5478d": { 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "3 ongoing clones; Avg progress is 11.849% (9s)\n [===.........................] (remaining: 74s)", 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.11849000304937363, 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:32.282 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:51:32.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa4e806cfe0 msgr2=0x7fa4e806f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:32.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa4e806cfe0 0x7fa4e806f4a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa50c15a6a0 tx=0x7fa4f4009040 comp rx=0 tx=0).stop 2024-05-30T16:51:32.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c0af3f0 msgr2=0x7fa50c14d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:32.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c0af3f0 0x7fa50c14d5d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc000c00 tx=0x7fa4fc0046c0 comp rx=0 tx=0).stop 2024-05-30T16:51:32.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 shutdown_connections 2024-05-30T16:51:32.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa4e806cfe0 0x7fa4e806f4a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:32.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa50c162f90 0x7fa50c1614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:32.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa50c162bb0 0x7fa50c15d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:32.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 --2- 172.21.15.67:0/242668442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50c0af3f0 0x7fa50c14d5d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:32.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 >> 172.21.15.67:0/242668442 conn(0x7fa50c0b66f0 msgr2=0x7fa50c0b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:32.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 shutdown_connections 2024-05-30T16:51:32.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:32.277+0000 7fa50be00640 1 -- 172.21.15.67:0/242668442 wait complete. 2024-05-30T16:51:37.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:51:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.520+0000 7f73b8400640 1 Processor -- start 2024-05-30T16:51:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.520+0000 7f73b8400640 1 -- start start 2024-05-30T16:51:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.521+0000 7f73b8400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b0145320 0x7f73b0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.521+0000 7f73b8400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f73b00b43b0 0x7f73b00cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.522+0000 7f73b8400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b00cf0c0 con 0x7f73b0072200 2024-05-30T16:51:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.522+0000 7f73b8400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f73b0072660 con 0x7f73b00b43b0 2024-05-30T16:51:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.522+0000 7f73b8400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73b00b4a20 con 0x7f73b0145320 2024-05-30T16:51:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.522+0000 7f73b5000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f73b0072200 0x7f73b00b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42800/0 (socket says 172.21.15.67:42800) 2024-05-30T16:51:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.522+0000 7f73b5000640 1 -- 172.21.15.67:0/2941974945 learned_addr learned my addr 172.21.15.67:0/2941974945 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:51:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.522+0000 7f73b5a00640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b0145320 0x7f73b0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.522+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3516098672 0 0) 0x7f73b00cf0c0 con 0x7f73b0072200 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.522+0000 7f73b6400640 1 --2- 172.21.15.67:0/2941974945 >> v2:172.21.15.169:3300/0 conn(0x7f73b00b43b0 0x7f73b00cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.522+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7398003520 con 0x7f73b0072200 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 406248146 0 0) 0x7f7398003520 con 0x7f73b0072200 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b0145320 msgr2=0x7f73b0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b0145320 0x7f73b0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 >> v2:172.21.15.169:3300/0 conn(0x7f73b00b43b0 msgr2=0x7f73b00cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 --2- 172.21.15.67:0/2941974945 >> v2:172.21.15.169:3300/0 conn(0x7f73b00b43b0 0x7f73b00cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b00b4d00 con 0x7f73b0072200 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f739c001670 con 0x7f73b0072200 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73b6400640 1 --2- 172.21.15.67:0/2941974945 >> v2:172.21.15.169:3300/0 conn(0x7f73b00b43b0 0x7f73b00cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:51:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 >> v1:172.21.15.67:6789/0 conn(0x7f73b0072200 legacy=0x7f73b00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73980039b0 0x7f7398003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b0145320 0x7f7398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f73b00b4a20 con 0x7f7398004d40 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f739800b9c0 con 0x7f73980039b0 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f739800bbc0 con 0x7f73b0145320 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.523+0000 7f73b6400640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.524+0000 7f73b6400640 1 -- 172.21.15.67:0/2941974945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b0145320 msgr2=0x7f7398004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.524+0000 7f73b6400640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b0145320 0x7f7398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.524+0000 7f73b6400640 1 -- 172.21.15.67:0/2941974945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73980039b0 msgr2=0x7f7398003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.524+0000 7f73b6400640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73980039b0 0x7f7398003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.524+0000 7f73b6400640 1 -- 172.21.15.67:0/2941974945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7398003520 con 0x7f7398004d40 2024-05-30T16:51:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.525+0000 7f73b6400640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73980051c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f73a8010dc0 tx=0x7f7398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c5efbfd2e9aa49 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.525+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8010a40 con 0x7f7398004d40 2024-05-30T16:51:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.525+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a80102d0 con 0x7f7398004d40 2024-05-30T16:51:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.525+0000 7f73afe00640 1 -- 172.21.15.67:0/2941974945 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8004270 con 0x7f7398004d40 2024-05-30T16:51:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.526+0000 7f73b8400640 1 -- 172.21.15.67:0/2941974945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 msgr2=0x7f73980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.526+0000 7f73b8400640 1 --2- 172.21.15.67:0/2941974945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73980051c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f73a8010dc0 tx=0x7f7398005830 comp rx=0 tx=0).stop 2024-05-30T16:51:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.526+0000 7f73b6400640 1 -- 172.21.15.67:0/2941974945 reap_dead start 2024-05-30T16:51:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.526+0000 7f73b8400640 1 -- 172.21.15.67:0/2941974945 shutdown_connections 2024-05-30T16:51:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.526+0000 7f73b8400640 1 -- 172.21.15.67:0/2941974945 >> 172.21.15.67:0/2941974945 conn(0x7f73b0067ce0 msgr2=0x7f73b006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.526+0000 7f73b8400640 1 -- 172.21.15.67:0/2941974945 shutdown_connections 2024-05-30T16:51:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.526+0000 7f73b8400640 1 -- 172.21.15.67:0/2941974945 wait complete. 2024-05-30T16:51:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.528+0000 7f73b8400640 1 Processor -- start 2024-05-30T16:51:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.528+0000 7f73b8400640 1 -- start start 2024-05-30T16:51:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.529+0000 7f73b8400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73b013ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.529+0000 7f73b8400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b015cce0 0x7f73b015f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.529+0000 7f73b8400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b006e550 0x7f73b006ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.529+0000 7f73b8400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f73b0145700 con 0x7f7398004d40 2024-05-30T16:51:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.529+0000 7f73b8400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73b00cf360 con 0x7f73b015cce0 2024-05-30T16:51:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.529+0000 7f73b8400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73b0071cc0 con 0x7f73b006e550 2024-05-30T16:51:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.529+0000 7f73b5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73b013ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.529+0000 7f73b5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73b013ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51366/0 (socket says 172.21.15.67:51366) 2024-05-30T16:51:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.529+0000 7f73b5a00640 1 -- 172.21.15.67:0/3592328317 learned_addr learned my addr 172.21.15.67:0/3592328317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73b5000640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b015cce0 0x7f73b015f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73b5a00640 1 -- 172.21.15.67:0/3592328317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b006e550 msgr2=0x7f73b006ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73b5a00640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b006e550 0x7f73b006ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73b5a00640 1 -- 172.21.15.67:0/3592328317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b015cce0 msgr2=0x7f73b015f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73b5a00640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b015cce0 0x7f73b015f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73b5a00640 1 -- 172.21.15.67:0/3592328317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b015d2a0 con 0x7f7398004d40 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73b5000640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b015cce0 0x7f73b015f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73b5a00640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73b013ccc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f73a400ef40 tx=0x7f73a400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73ad600640 1 -- 172.21.15.67:0/3592328317 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400be40 con 0x7f7398004d40 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73ad600640 1 -- 172.21.15.67:0/3592328317 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a4019070 con 0x7f7398004d40 2024-05-30T16:51:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73ad600640 1 -- 172.21.15.67:0/3592328317 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4003bb0 con 0x7f7398004d40 2024-05-30T16:51:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.530+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b015d580 con 0x7f7398004d40 2024-05-30T16:51:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.531+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b015db40 con 0x7f7398004d40 2024-05-30T16:51:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.532+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b00689e0 con 0x7f7398004d40 2024-05-30T16:51:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.533+0000 7f73ad600640 1 -- 172.21.15.67:0/3592328317 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f73a4005d10 con 0x7f7398004d40 2024-05-30T16:51:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.534+0000 7f73ad600640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f739006cfb0 0x7f739006f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.534+0000 7f73ad600640 1 -- 172.21.15.67:0/3592328317 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f73a408cab0 con 0x7f7398004d40 2024-05-30T16:51:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.534+0000 7f73b5000640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f739006cfb0 0x7f739006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.535+0000 7f73b5000640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f739006cfb0 0x7f739006f470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f73b00cf680 tx=0x7f739c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.539+0000 7f73ad600640 1 -- 172.21.15.67:0/3592328317 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73a405a4c0 con 0x7f7398004d40 2024-05-30T16:51:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.817+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f73b00b34e0 con 0x7f7398004d40 2024-05-30T16:51:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.819+0000 7f73ad600640 1 -- 172.21.15.67:0/3592328317 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2125 (secure 0 0 0) 0x7f73a4059dc0 con 0x7f7398004d40 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:51:37.821 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 491, 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:51:37.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 50, 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:37.823 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4259, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 17727067974, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 35837911040, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 737256202240, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 541137018, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 538742957, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 132, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 147 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:37.824 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 26, 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 3, 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 3, 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5361 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:51:37.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 8, 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:49:38.113647+0000", 2024-05-30T16:51:37.826 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:51:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "de99db33-4236-4531-bbc4-b658dcb5478d": { 2024-05-30T16:51:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (16s)\n [============================] ", 2024-05-30T16:51:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:51:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:51:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:37.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:37.827 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:51:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f739006cfb0 msgr2=0x7f739006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f739006cfb0 0x7f739006f470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f73b00cf680 tx=0x7f739c009040 comp rx=0 tx=0).stop 2024-05-30T16:51:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 msgr2=0x7f73b013ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73b013ccc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f73a400ef40 tx=0x7f73a400b480 comp rx=0 tx=0).stop 2024-05-30T16:51:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 shutdown_connections 2024-05-30T16:51:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f739006cfb0 0x7f739006f470 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b006e550 0x7f73b006ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b015cce0 0x7f73b015f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 --2- 172.21.15.67:0/3592328317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7398004d40 0x7f73b013ccc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 >> 172.21.15.67:0/3592328317 conn(0x7f73b0067ce0 msgr2=0x7f73b006dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 shutdown_connections 2024-05-30T16:51:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:37.822+0000 7f73b8400640 1 -- 172.21.15.67:0/3592328317 wait complete. 2024-05-30T16:51:41.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:51:41.058 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:51:42.842 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:51:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.060+0000 7f4955600640 1 Processor -- start 2024-05-30T16:51:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.060+0000 7f4955600640 1 -- start start 2024-05-30T16:51:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.062+0000 7f4955600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f49500d0b70 0x7f4950169460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.062+0000 7f4955600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49501621d0 con 0x7f4950161df0 2024-05-30T16:51:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.062+0000 7f4955600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49500ccf10 con 0x7f49500cc2b0 2024-05-30T16:51:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.062+0000 7f4955600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f49500cd840 con 0x7f49500d0b70 2024-05-30T16:51:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.062+0000 7f494fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4950161df0 0x7f4950099800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47462/0 (socket says 172.21.15.67:47462) 2024-05-30T16:51:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.062+0000 7f494fe00640 1 -- 172.21.15.67:0/2283073507 learned_addr learned my addr 172.21.15.67:0/2283073507 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:51:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123270409 0 0) 0x7f4938001110 con 0x7f4950161df0 2024-05-30T16:51:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4934003520 con 0x7f4950161df0 2024-05-30T16:51:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f4954c00640 1 --2- 172.21.15.67:0/2283073507 >> v2:172.21.15.169:3300/0 conn(0x7f49500d0b70 0x7f4950169460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1920523994 0 0) 0x7f4934003520 con 0x7f4950161df0 2024-05-30T16:51:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 >> v2:172.21.15.169:3300/0 conn(0x7f49500d0b70 msgr2=0x7f4950169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 --2- 172.21.15.67:0/2283073507 >> v2:172.21.15.169:3300/0 conn(0x7f49500d0b70 0x7f4950169460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 >> v1:172.21.15.169:6789/0 conn(0x7f49500cc2b0 legacy=0x7f49500d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49500cd1f0 con 0x7f4950161df0 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f4938001840 con 0x7f4950161df0 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 >> v1:172.21.15.67:6789/0 conn(0x7f4950161df0 legacy=0x7f4950099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4934003a00 0x7f4934003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4934004800 0x7f4934004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f493400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f493400c110 con 0x7f4934005210 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f493400c2c0 con 0x7f4934004800 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.063+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f493400c4c0 con 0x7f4934003a00 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.064+0000 7f4954c00640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f493400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.064+0000 7f494f400640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4934004800 0x7f4934004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.064+0000 7f494fe00640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4934003a00 0x7f4934003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.064+0000 7f4954c00640 1 -- 172.21.15.67:0/2283073507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4934003a00 msgr2=0x7f4934003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.064+0000 7f4954c00640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4934003a00 0x7f4934003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.064+0000 7f4954c00640 1 -- 172.21.15.67:0/2283073507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4934004800 msgr2=0x7f4934004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.064+0000 7f4954c00640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4934004800 0x7f4934004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.064+0000 7f4954c00640 1 -- 172.21.15.67:0/2283073507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4934003520 con 0x7f4934005210 2024-05-30T16:51:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.065+0000 7f4954c00640 1 -- 172.21.15.67:0/2283073507 reap_dead start 2024-05-30T16:51:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.065+0000 7f4954c00640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f493400b9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f49500995c0 tx=0x7f4934005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad18a489a57cedd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.065+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948033070 con 0x7f4934005210 2024-05-30T16:51:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.065+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494802f650 con 0x7f4934005210 2024-05-30T16:51:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.065+0000 7f494ea00640 1 -- 172.21.15.67:0/2283073507 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948009be0 con 0x7f4934005210 2024-05-30T16:51:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.066+0000 7f4955600640 1 -- 172.21.15.67:0/2283073507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 msgr2=0x7f493400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.066+0000 7f4955600640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f493400b9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f49500995c0 tx=0x7f4934005830 comp rx=0 tx=0).stop 2024-05-30T16:51:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.066+0000 7f4955600640 1 -- 172.21.15.67:0/2283073507 shutdown_connections 2024-05-30T16:51:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.066+0000 7f4955600640 1 --2- 172.21.15.67:0/2283073507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f493400b9c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.066+0000 7f4955600640 1 -- 172.21.15.67:0/2283073507 >> 172.21.15.67:0/2283073507 conn(0x7f495006bae0 msgr2=0x7f495006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.066+0000 7f4955600640 1 -- 172.21.15.67:0/2283073507 shutdown_connections 2024-05-30T16:51:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.066+0000 7f4955600640 1 -- 172.21.15.67:0/2283073507 wait complete. 2024-05-30T16:51:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.068+0000 7f4955600640 1 Processor -- start 2024-05-30T16:51:43.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.068+0000 7f4955600640 1 -- start start 2024-05-30T16:51:43.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.069+0000 7f4955600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f49500b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:43.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.069+0000 7f4955600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49500c72f0 0x7f49500c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:43.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.069+0000 7f4955600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49500c9320 0x7f49500c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:43.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.069+0000 7f4955600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f49500ca870 con 0x7f4934005210 2024-05-30T16:51:43.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.069+0000 7f4955600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f49500ccbc0 con 0x7f49500c72f0 2024-05-30T16:51:43.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.069+0000 7f4955600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f49500ca9f0 con 0x7f49500c9320 2024-05-30T16:51:43.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f494fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f49500b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:43.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f494fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f49500b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56980/0 (socket says 172.21.15.67:56980) 2024-05-30T16:51:43.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f494fe00640 1 -- 172.21.15.67:0/586703137 learned_addr learned my addr 172.21.15.67:0/586703137 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:43.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f494f400640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49500c72f0 0x7f49500c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:43.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f4954c00640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49500c9320 0x7f49500c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:43.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f494fe00640 1 -- 172.21.15.67:0/586703137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49500c9320 msgr2=0x7f49500c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:43.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f494fe00640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49500c9320 0x7f49500c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f494fe00640 1 -- 172.21.15.67:0/586703137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49500c72f0 msgr2=0x7f49500c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f494fe00640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49500c72f0 0x7f49500c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.070+0000 7f494fe00640 1 -- 172.21.15.67:0/586703137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49500c7d40 con 0x7f4934005210 2024-05-30T16:51:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.071+0000 7f494fe00640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f49500b1010 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f493800e700 tx=0x7f493800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.071+0000 7f4947e00640 1 -- 172.21.15.67:0/586703137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493800c6c0 con 0x7f4934005210 2024-05-30T16:51:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.071+0000 7f4947e00640 1 -- 172.21.15.67:0/586703137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4938018070 con 0x7f4934005210 2024-05-30T16:51:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.071+0000 7f4947e00640 1 -- 172.21.15.67:0/586703137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493800ca80 con 0x7f4934005210 2024-05-30T16:51:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.071+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49500c8020 con 0x7f4934005210 2024-05-30T16:51:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.071+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49500c85e0 con 0x7f4934005210 2024-05-30T16:51:43.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.072+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49500689e0 con 0x7f4934005210 2024-05-30T16:51:43.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.073+0000 7f4947e00640 1 -- 172.21.15.67:0/586703137 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f493800cc20 con 0x7f4934005210 2024-05-30T16:51:43.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.074+0000 7f4947e00640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f492c06cfb0 0x7f492c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:43.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.074+0000 7f4947e00640 1 -- 172.21.15.67:0/586703137 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f493808c530 con 0x7f4934005210 2024-05-30T16:51:43.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.074+0000 7f494f400640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f492c06cfb0 0x7f492c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:43.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.075+0000 7f494f400640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f492c06cfb0 0x7f492c06f470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f49500c6b70 tx=0x7f4940009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:43.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.079+0000 7f4947e00640 1 -- 172.21.15.67:0/586703137 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49501621d0 con 0x7f4934005210 2024-05-30T16:51:43.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.359+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f49500af750 con 0x7f4934005210 2024-05-30T16:51:43.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.361+0000 7f4947e00640 1 -- 172.21.15.67:0/586703137 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2121 (secure 0 0 0) 0x7f493805a050 con 0x7f4934005210 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:51:43.362 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 496, 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:51:43.363 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 50, 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:43.364 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4258, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 17727122152, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 35838013440, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 737256099840, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 59746470, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 61159287, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 15, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 20 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 26, 2024-05-30T16:51:43.365 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 3, 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 3, 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5361 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:43.366 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 8, 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:49:38.113647+0000", 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:43.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:43.368 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:51:43.368 INFO:teuthology.orchestra.run.smithi067.stdout: "de99db33-4236-4531-bbc4-b658dcb5478d": { 2024-05-30T16:51:43.368 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (22s)\n [============================] ", 2024-05-30T16:51:43.368 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:51:43.368 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:51:43.368 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:43.368 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:43.368 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:51:43.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f492c06cfb0 msgr2=0x7f492c06f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:43.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f492c06cfb0 0x7f492c06f470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f49500c6b70 tx=0x7f4940009040 comp rx=0 tx=0).stop 2024-05-30T16:51:43.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 msgr2=0x7f49500b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:43.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f49500b1010 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f493800e700 tx=0x7f493800ebc0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 shutdown_connections 2024-05-30T16:51:43.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f492c06cfb0 0x7f492c06f470 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49500c9320 0x7f49500c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49500c72f0 0x7f49500c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 --2- 172.21.15.67:0/586703137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4934005210 0x7f49500b1010 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:43.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.363+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 >> 172.21.15.67:0/586703137 conn(0x7f495006bae0 msgr2=0x7f49500ae900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:43.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.364+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 shutdown_connections 2024-05-30T16:51:43.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:43.364+0000 7f4955600640 1 -- 172.21.15.67:0/586703137 wait complete. 2024-05-30T16:51:48.385 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:51:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.602+0000 7f02a3e00640 1 Processor -- start 2024-05-30T16:51:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.602+0000 7f02a3e00640 1 -- start start 2024-05-30T16:51:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.604+0000 7f02a3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f02a415c6b0 0x7f02a40b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.604+0000 7f02a3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a40ccc30 0x7f02a406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.604+0000 7f02a3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f02a40d0460 0x7f02a40d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.604+0000 7f02a3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f02a40cd1a0 con 0x7f02a40d0460 2024-05-30T16:51:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.604+0000 7f02a3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f02a40cd480 con 0x7f02a415c6b0 2024-05-30T16:51:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.604+0000 7f02a3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02a40cd760 con 0x7f02a40ccc30 2024-05-30T16:51:48.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.604+0000 7f02a3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f02a40d0460 0x7f02a40d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:48.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.605+0000 7f02a3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f02a40d0460 0x7f02a40d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56986/0 (socket says 172.21.15.67:56986) 2024-05-30T16:51:48.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.605+0000 7f02a3400640 1 -- 172.21.15.67:0/2951662911 learned_addr learned my addr 172.21.15.67:0/2951662911 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:48.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.605+0000 7f02a2a00640 1 --2- 172.21.15.67:0/2951662911 >> v2:172.21.15.169:3300/0 conn(0x7f02a415c6b0 0x7f02a40b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:48.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.605+0000 7f02a3400640 1 -- 172.21.15.67:0/2951662911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a40ccc30 msgr2=0x7f02a406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:48.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.605+0000 7f02a3400640 1 --2- 172.21.15.67:0/2951662911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a40ccc30 0x7f02a406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.605+0000 7f02a3400640 1 -- 172.21.15.67:0/2951662911 >> v2:172.21.15.169:3300/0 conn(0x7f02a415c6b0 msgr2=0x7f02a40b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:48.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.605+0000 7f02a3400640 1 --2- 172.21.15.67:0/2951662911 >> v2:172.21.15.169:3300/0 conn(0x7f02a415c6b0 0x7f02a40b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.605+0000 7f02a3400640 1 -- 172.21.15.67:0/2951662911 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a40cda40 con 0x7f02a40d0460 2024-05-30T16:51:48.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.607+0000 7f02a3400640 1 --2- 172.21.15.67:0/2951662911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 0x7f02a40d08f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f02a40b3770 tx=0x7f029c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda6951730754115 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:48.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.607+0000 7f02a2000640 1 -- 172.21.15.67:0/2951662911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c036070 con 0x7f02a40d0460 2024-05-30T16:51:48.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.607+0000 7f02a2000640 1 -- 172.21.15.67:0/2951662911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c031600 con 0x7f02a40d0460 2024-05-30T16:51:48.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.607+0000 7f02a2000640 1 -- 172.21.15.67:0/2951662911 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c0319d0 con 0x7f02a40d0460 2024-05-30T16:51:48.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.608+0000 7f02a3e00640 1 -- 172.21.15.67:0/2951662911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 msgr2=0x7f02a40d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:48.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.608+0000 7f02a3e00640 1 --2- 172.21.15.67:0/2951662911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 0x7f02a40d08f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f02a40b3770 tx=0x7f029c02a6a0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.608+0000 7f02a3e00640 1 -- 172.21.15.67:0/2951662911 shutdown_connections 2024-05-30T16:51:48.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.608+0000 7f02a3e00640 1 --2- 172.21.15.67:0/2951662911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 0x7f02a40d08f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.608+0000 7f02a3e00640 1 --2- 172.21.15.67:0/2951662911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a40ccc30 0x7f02a406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.608+0000 7f02a3e00640 1 --2- 172.21.15.67:0/2951662911 >> v2:172.21.15.169:3300/0 conn(0x7f02a415c6b0 0x7f02a40b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.608+0000 7f02a3e00640 1 -- 172.21.15.67:0/2951662911 >> 172.21.15.67:0/2951662911 conn(0x7f02a406bae0 msgr2=0x7f02a406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:48.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.608+0000 7f02a3e00640 1 -- 172.21.15.67:0/2951662911 shutdown_connections 2024-05-30T16:51:48.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.608+0000 7f02a3e00640 1 -- 172.21.15.67:0/2951662911 wait complete. 2024-05-30T16:51:48.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.610+0000 7f02a3e00640 1 Processor -- start 2024-05-30T16:51:48.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.610+0000 7f02a3e00640 1 -- start start 2024-05-30T16:51:48.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.611+0000 7f02a3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a40ccc30 0x7f02a41541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.611+0000 7f02a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 0x7f02a40c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.611+0000 7f02a3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a415c6b0 0x7f02a40b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.611+0000 7f02a3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f02a40b4470 con 0x7f02a40d0460 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.611+0000 7f02a3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02a40b42f0 con 0x7f02a415c6b0 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.611+0000 7f02a3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02a406e1d0 con 0x7f02a40ccc30 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.611+0000 7f029a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 0x7f02a40c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.611+0000 7f029a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 0x7f02a40c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56994/0 (socket says 172.21.15.67:56994) 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.611+0000 7f029a000640 1 -- 172.21.15.67:0/3886990121 learned_addr learned my addr 172.21.15.67:0/3886990121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.612+0000 7f02a2a00640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a40ccc30 0x7f02a41541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.612+0000 7f029a000640 1 -- 172.21.15.67:0/3886990121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a40ccc30 msgr2=0x7f02a41541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.612+0000 7f029a000640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a40ccc30 0x7f02a41541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.612+0000 7f02a3400640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a415c6b0 0x7f02a40b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.612+0000 7f029a000640 1 -- 172.21.15.67:0/3886990121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a415c6b0 msgr2=0x7f02a40b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:48.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.612+0000 7f029a000640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a415c6b0 0x7f02a40b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.612+0000 7f029a000640 1 -- 172.21.15.67:0/3886990121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a40c9ce0 con 0x7f02a40d0460 2024-05-30T16:51:48.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.612+0000 7f029a000640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 0x7f02a40c9400 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f028c00b410 tx=0x7f028c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.613+0000 7f029b400640 1 -- 172.21.15.67:0/3886990121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c015020 con 0x7f02a40d0460 2024-05-30T16:51:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.613+0000 7f029b400640 1 -- 172.21.15.67:0/3886990121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c011850 con 0x7f02a40d0460 2024-05-30T16:51:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.613+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a40c9fc0 con 0x7f02a40d0460 2024-05-30T16:51:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.613+0000 7f029b400640 1 -- 172.21.15.67:0/3886990121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c011c40 con 0x7f02a40d0460 2024-05-30T16:51:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.613+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a41546e0 con 0x7f02a40d0460 2024-05-30T16:51:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.614+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a40ca1e0 con 0x7f02a40d0460 2024-05-30T16:51:48.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.615+0000 7f029b400640 1 -- 172.21.15.67:0/3886990121 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f028c018020 con 0x7f02a40d0460 2024-05-30T16:51:48.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.616+0000 7f029b400640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f028006cfb0 0x7f028006f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:48.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.616+0000 7f029b400640 1 -- 172.21.15.67:0/3886990121 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f028c08c6d0 con 0x7f02a40d0460 2024-05-30T16:51:48.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.616+0000 7f02a2a00640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f028006cfb0 0x7f028006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:48.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.617+0000 7f02a2a00640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f028006cfb0 0x7f028006f470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0294009ad0 tx=0x7f0294008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:48.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.621+0000 7f029b400640 1 -- 172.21.15.67:0/3886990121 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f028c01f020 con 0x7f02a40d0460 2024-05-30T16:51:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.902+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f02a40689e0 con 0x7f02a40d0460 2024-05-30T16:51:48.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.903+0000 7f029b400640 1 -- 172.21.15.67:0/3886990121 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2109 (secure 0 0 0) 0x7f028c0172c0 con 0x7f02a40d0460 2024-05-30T16:51:48.904 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:51:48.904 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:51:48.904 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:51:48.904 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:51:48.905 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:51:48.905 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:51:48.905 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:51:48.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:48.905 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:51:48.905 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:51:48.905 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:51:48.905 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 502, 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:51:48.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 50, 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:48.907 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4258, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 17727132655, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 35838025728, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 737256087552, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 85, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 6056, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 1 2024-05-30T16:51:48.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 26, 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 3, 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 3, 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:51:48.909 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5361 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:48.910 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 8, 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:49:38.113647+0000", 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "de99db33-4236-4531-bbc4-b658dcb5478d": { 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (28s)\n [============================] ", 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:48.911 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f028006cfb0 msgr2=0x7f028006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f028006cfb0 0x7f028006f470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0294009ad0 tx=0x7f0294008040 comp rx=0 tx=0).stop 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 msgr2=0x7f02a40c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 0x7f02a40c9400 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f028c00b410 tx=0x7f028c00b8d0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 shutdown_connections 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f028006cfb0 0x7f028006f470 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a415c6b0 0x7f02a40b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02a40d0460 0x7f02a40c9400 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 --2- 172.21.15.67:0/3886990121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a40ccc30 0x7f02a41541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:48.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 >> 172.21.15.67:0/3886990121 conn(0x7f02a406bae0 msgr2=0x7f02a4073910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:48.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 shutdown_connections 2024-05-30T16:51:48.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:48.906+0000 7f02a3e00640 1 -- 172.21.15.67:0/3886990121 wait complete. 2024-05-30T16:51:53.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:51:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.149+0000 7fbd7b200640 1 Processor -- start 2024-05-30T16:51:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.150+0000 7fbd7b200640 1 -- start start 2024-05-30T16:51:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.151+0000 7fbd7b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbd7c0ca870 0x7fbd7c0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.151+0000 7fbd7b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbd7c0af720 0x7fbd7c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.151+0000 7fbd7b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7c0b4be0 con 0x7fbd7c15ac80 2024-05-30T16:51:54.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.151+0000 7fbd7b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fbd7c0cf8a0 con 0x7fbd7c0ca870 2024-05-30T16:51:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.151+0000 7fbd7b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbd7c0cde90 con 0x7fbd7c0af720 2024-05-30T16:51:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.152+0000 7fbd79e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbd7c15ac80 0x7fbd7c15f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34248/0 (socket says 172.21.15.67:34248) 2024-05-30T16:51:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.152+0000 7fbd79e00640 1 -- 172.21.15.67:0/1670163761 learned_addr learned my addr 172.21.15.67:0/1670163761 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:51:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.152+0000 7fbd79400640 1 --2- 172.21.15.67:0/1670163761 >> v2:172.21.15.67:3300/0 conn(0x7fbd7c0ca870 0x7fbd7c0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.152+0000 7fbd78a00640 1 -- 172.21.15.67:0/1670163761 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001785455 0 0) 0x7fbd7c0b4be0 con 0x7fbd7c15ac80 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.152+0000 7fbd78a00640 1 -- 172.21.15.67:0/1670163761 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd60003520 con 0x7fbd7c15ac80 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.152+0000 7fbd7a800640 1 --2- 172.21.15.67:0/1670163761 >> v2:172.21.15.169:3300/0 conn(0x7fbd7c0af720 0x7fbd7c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.152+0000 7fbd79400640 1 -- 172.21.15.67:0/1670163761 >> v2:172.21.15.169:3300/0 conn(0x7fbd7c0af720 msgr2=0x7fbd7c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.153+0000 7fbd78a00640 1 -- 172.21.15.67:0/1670163761 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3750945207 0 0) 0x7fbd60003520 con 0x7fbd7c15ac80 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.153+0000 7fbd79400640 1 --2- 172.21.15.67:0/1670163761 >> v2:172.21.15.169:3300/0 conn(0x7fbd7c0af720 0x7fbd7c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.153+0000 7fbd79400640 1 -- 172.21.15.67:0/1670163761 >> v1:172.21.15.67:6789/0 conn(0x7fbd7c15ac80 legacy=0x7fbd7c15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.153+0000 7fbd79400640 1 -- 172.21.15.67:0/1670163761 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c0ce170 con 0x7fbd7c0ca870 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.153+0000 7fbd7a800640 1 --2- 172.21.15.67:0/1670163761 >> v2:172.21.15.169:3300/0 conn(0x7fbd7c0af720 0x7fbd7c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.154+0000 7fbd79400640 1 --2- 172.21.15.67:0/1670163761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0ca870 0x7fbd7c0ccc00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0b4430 tx=0x7fbd6c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160599babb49794c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:54.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.154+0000 7fbd78a00640 1 -- 172.21.15.67:0/1670163761 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64001670 con 0x7fbd7c0ca870 2024-05-30T16:51:54.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.154+0000 7fbd78a00640 1 -- 172.21.15.67:0/1670163761 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c007540 con 0x7fbd7c0ca870 2024-05-30T16:51:54.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.154+0000 7fbd78a00640 1 -- 172.21.15.67:0/1670163761 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0078f0 con 0x7fbd7c0ca870 2024-05-30T16:51:54.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.155+0000 7fbd7b200640 1 -- 172.21.15.67:0/1670163761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0ca870 msgr2=0x7fbd7c0ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.155+0000 7fbd7b200640 1 --2- 172.21.15.67:0/1670163761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0ca870 0x7fbd7c0ccc00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0b4430 tx=0x7fbd6c02a610 comp rx=0 tx=0).stop 2024-05-30T16:51:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.155+0000 7fbd7b200640 1 -- 172.21.15.67:0/1670163761 shutdown_connections 2024-05-30T16:51:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.155+0000 7fbd7b200640 1 --2- 172.21.15.67:0/1670163761 >> v2:172.21.15.169:3300/0 conn(0x7fbd7c0af720 0x7fbd7c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.155+0000 7fbd7b200640 1 --2- 172.21.15.67:0/1670163761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0ca870 0x7fbd7c0ccc00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.155+0000 7fbd7b200640 1 -- 172.21.15.67:0/1670163761 >> 172.21.15.67:0/1670163761 conn(0x7fbd7c06bae0 msgr2=0x7fbd7c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.155+0000 7fbd7b200640 1 -- 172.21.15.67:0/1670163761 shutdown_connections 2024-05-30T16:51:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.155+0000 7fbd7b200640 1 -- 172.21.15.67:0/1670163761 wait complete. 2024-05-30T16:51:54.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.157+0000 7fbd7b200640 1 Processor -- start 2024-05-30T16:51:54.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.157+0000 7fbd7b200640 1 -- start start 2024-05-30T16:51:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.158+0000 7fbd7b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0af720 0x7fbd7c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.158+0000 7fbd7b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c0ca870 0x7fbd7c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.158+0000 7fbd7b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c15ac80 0x7fbd7c0aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.158+0000 7fbd7b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c0d0460 con 0x7fbd7c0af720 2024-05-30T16:51:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.158+0000 7fbd7b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c0cfe20 con 0x7fbd7c0ca870 2024-05-30T16:51:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.158+0000 7fbd7b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7c0d05e0 con 0x7fbd7c15ac80 2024-05-30T16:51:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.158+0000 7fbd79e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0af720 0x7fbd7c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.158+0000 7fbd79e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0af720 0x7fbd7c0ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56998/0 (socket says 172.21.15.67:56998) 2024-05-30T16:51:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.158+0000 7fbd79e00640 1 -- 172.21.15.67:0/2561561988 learned_addr learned my addr 172.21.15.67:0/2561561988 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd79400640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c0ca870 0x7fbd7c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd7a800640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c15ac80 0x7fbd7c0aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd79e00640 1 -- 172.21.15.67:0/2561561988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c15ac80 msgr2=0x7fbd7c0aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd79e00640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c15ac80 0x7fbd7c0aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd79e00640 1 -- 172.21.15.67:0/2561561988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c0ca870 msgr2=0x7fbd7c155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd79e00640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c0ca870 0x7fbd7c155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd79e00640 1 -- 172.21.15.67:0/2561561988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c0ad980 con 0x7fbd7c0af720 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd79400640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c0ca870 0x7fbd7c155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd79e00640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0af720 0x7fbd7c0ad130 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbd64001eb0 tx=0x7fbd6400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd72000640 1 -- 172.21.15.67:0/2561561988 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64017070 con 0x7fbd7c0af720 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd72000640 1 -- 172.21.15.67:0/2561561988 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd64005ce0 con 0x7fbd7c0af720 2024-05-30T16:51:54.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7c0adc60 con 0x7fbd7c0af720 2024-05-30T16:51:54.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.159+0000 7fbd72000640 1 -- 172.21.15.67:0/2561561988 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64007290 con 0x7fbd7c0af720 2024-05-30T16:51:54.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.160+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7c0ae220 con 0x7fbd7c0af720 2024-05-30T16:51:54.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.161+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd7c0689e0 con 0x7fbd7c0af720 2024-05-30T16:51:54.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.162+0000 7fbd72000640 1 -- 172.21.15.67:0/2561561988 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7fbd6401a070 con 0x7fbd7c0af720 2024-05-30T16:51:54.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.163+0000 7fbd72000640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5806cfb0 0x7fbd5806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.163+0000 7fbd72000640 1 -- 172.21.15.67:0/2561561988 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fbd6408bcf0 con 0x7fbd7c0af720 2024-05-30T16:51:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.163+0000 7fbd79400640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5806cfb0 0x7fbd5806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.164+0000 7fbd79400640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5806cfb0 0x7fbd5806f470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c034000 tx=0x7fbd6c02f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.168+0000 7fbd72000640 1 -- 172.21.15.67:0/2561561988 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd64059680 con 0x7fbd7c0af720 2024-05-30T16:51:54.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.445+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbd7c0b4790 con 0x7fbd7c0af720 2024-05-30T16:51:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.446+0000 7fbd72000640 1 -- 172.21.15.67:0/2561561988 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2079 (secure 0 0 0) 0x7fbd64058f80 con 0x7fbd7c0af720 2024-05-30T16:51:54.447 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:51:54.447 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:51:54.447 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:51:54.447 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:51:54.447 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:51:54.447 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:51:54.448 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 507, 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 50, 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:51:54.449 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4258, 2024-05-30T16:51:54.450 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 17727134270, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 35838033920, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 737256079360, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 1279, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 26, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 3, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:51:54.451 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 3, 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5361 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:54.452 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 8, 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:49:38.113647+0000", 2024-05-30T16:51:54.453 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:54.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:54.454 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:51:54.454 INFO:teuthology.orchestra.run.smithi067.stdout: "de99db33-4236-4531-bbc4-b658dcb5478d": { 2024-05-30T16:51:54.454 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 11.0 seconds before exiting (32s)\n [============================] ", 2024-05-30T16:51:54.454 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:51:54.454 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:51:54.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:54.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:54.454 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:51:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5806cfb0 msgr2=0x7fbd5806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5806cfb0 0x7fbd5806f470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c034000 tx=0x7fbd6c02f000 comp rx=0 tx=0).stop 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0af720 msgr2=0x7fbd7c0ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0af720 0x7fbd7c0ad130 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbd64001eb0 tx=0x7fbd6400c9b0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 shutdown_connections 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd5806cfb0 0x7fbd5806f470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd7c15ac80 0x7fbd7c0aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c0ca870 0x7fbd7c155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 --2- 172.21.15.67:0/2561561988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd7c0af720 0x7fbd7c0ad130 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 >> 172.21.15.67:0/2561561988 conn(0x7fbd7c06bae0 msgr2=0x7fbd7c0ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 shutdown_connections 2024-05-30T16:51:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:54.449+0000 7fbd7b200640 1 -- 172.21.15.67:0/2561561988 wait complete. 2024-05-30T16:51:59.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:51:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.687+0000 7f1d0de00640 1 Processor -- start 2024-05-30T16:51:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.688+0000 7f1d0de00640 1 -- start start 2024-05-30T16:51:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.689+0000 7f1d0de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d080d0460 0x7f1d080d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.689+0000 7f1d0de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d080cd1a0 con 0x7f1d0815c6b0 2024-05-30T16:51:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.689+0000 7f1d0de00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d080cd480 con 0x7f1d080ccc30 2024-05-30T16:51:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.689+0000 7f1d0de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0806e3a0 con 0x7f1d080d0460 2024-05-30T16:51:59.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.690+0000 7f1d0ca00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1d0815c6b0 0x7f1d080b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34252/0 (socket says 172.21.15.67:34252) 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.690+0000 7f1d0ca00640 1 -- 172.21.15.67:0/3112588861 learned_addr learned my addr 172.21.15.67:0/3112588861 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.690+0000 7f1d0d400640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d080d0460 0x7f1d080d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.690+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1679569878 0 0) 0x7f1d080cd1a0 con 0x7f1d0815c6b0 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.690+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cf0003520 con 0x7f1d0815c6b0 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1671725046 0 0) 0x7f1cf0003520 con 0x7f1d0815c6b0 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d080d0460 msgr2=0x7f1d080d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d080d0460 0x7f1d080d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 >> v1:172.21.15.169:6789/0 conn(0x7f1d080ccc30 legacy=0x7f1d0806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d080cd760 con 0x7f1d0815c6b0 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f1cfc0019f0 con 0x7f1d0815c6b0 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 >> v1:172.21.15.67:6789/0 conn(0x7f1d0815c6b0 legacy=0x7f1d080b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00039e0 0x7f1cf0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cf0004800 0x7f1cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d080d0460 0x7f1cf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0806e3a0 con 0x7f1cf00039e0 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf000fe80 con 0x7f1cf0004800 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf0010080 con 0x7f1d080d0460 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d0ca00640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00039e0 0x7f1cf0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d07e00640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cf0004800 0x7f1cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.691+0000 7f1d0d400640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d080d0460 0x7f1cf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.692+0000 7f1d0ca00640 1 -- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d080d0460 msgr2=0x7f1cf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.692+0000 7f1d0ca00640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d080d0460 0x7f1cf00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.692+0000 7f1d0ca00640 1 -- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cf0004800 msgr2=0x7f1cf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.692+0000 7f1d0ca00640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cf0004800 0x7f1cf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.692+0000 7f1d0ca00640 1 -- 172.21.15.67:0/3112588861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf0003520 con 0x7f1cf00039e0 2024-05-30T16:51:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.693+0000 7f1d0ca00640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00039e0 0x7f1cf0003e00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00a830 tx=0x7f1cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c0b98070a4b97d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.693+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00a480 con 0x7f1cf00039e0 2024-05-30T16:51:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.693+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cfc02f070 con 0x7f1cf00039e0 2024-05-30T16:51:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.693+0000 7f1d07400640 1 -- 172.21.15.67:0/3112588861 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc02bc90 con 0x7f1cf00039e0 2024-05-30T16:51:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.694+0000 7f1d0de00640 1 -- 172.21.15.67:0/3112588861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00039e0 msgr2=0x7f1cf0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.694+0000 7f1d0de00640 1 --2- 172.21.15.67:0/3112588861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00039e0 0x7f1cf0003e00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00a830 tx=0x7f1cf0005830 comp rx=0 tx=0).stop 2024-05-30T16:51:59.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.694+0000 7f1d0d400640 1 -- 172.21.15.67:0/3112588861 reap_dead start 2024-05-30T16:51:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.694+0000 7f1d0de00640 1 -- 172.21.15.67:0/3112588861 shutdown_connections 2024-05-30T16:51:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.694+0000 7f1d0de00640 1 -- 172.21.15.67:0/3112588861 >> 172.21.15.67:0/3112588861 conn(0x7f1d0806bae0 msgr2=0x7f1d0806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.694+0000 7f1d0de00640 1 -- 172.21.15.67:0/3112588861 shutdown_connections 2024-05-30T16:51:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.694+0000 7f1d0de00640 1 -- 172.21.15.67:0/3112588861 wait complete. 2024-05-30T16:51:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.696+0000 7f1d0de00640 1 Processor -- start 2024-05-30T16:51:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.696+0000 7f1d0de00640 1 -- start start 2024-05-30T16:51:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.697+0000 7f1d0de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00039e0 0x7f1d080c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.697+0000 7f1d0de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080c72f0 0x7f1d080b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.697+0000 7f1d0de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d080ae600 0x7f1d080aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.697+0000 7f1d0de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1d080d08f0 con 0x7f1d080ae600 2024-05-30T16:51:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.697+0000 7f1d0de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1d080cba10 con 0x7f1d080c72f0 2024-05-30T16:51:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.697+0000 7f1d0de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1d080d0a70 con 0x7f1cf00039e0 2024-05-30T16:51:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.697+0000 7f1d0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d080ae600 0x7f1d080aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.697+0000 7f1d0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d080ae600 0x7f1d080aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57020/0 (socket says 172.21.15.67:57020) 2024-05-30T16:51:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.697+0000 7f1d0d400640 1 -- 172.21.15.67:0/3614970510 learned_addr learned my addr 172.21.15.67:0/3614970510 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:51:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0ca00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00039e0 0x7f1d080c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d07e00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080c72f0 0x7f1d080b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0d400640 1 -- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00039e0 msgr2=0x7f1d080c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0d400640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00039e0 0x7f1d080c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0d400640 1 -- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080c72f0 msgr2=0x7f1d080b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0d400640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080c72f0 0x7f1d080b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0d400640 1 -- 172.21.15.67:0/3614970510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d080b1a10 con 0x7f1d080ae600 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d07e00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080c72f0 0x7f1d080b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0ca00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00039e0 0x7f1d080c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0d400640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d080ae600 0x7f1d080aea80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1d080cb660 tx=0x7f1d00011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d04c00640 1 -- 172.21.15.67:0/3614970510 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00019020 con 0x7f1d080ae600 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d04c00640 1 -- 172.21.15.67:0/3614970510 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d00003610 con 0x7f1d080ae600 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d081f0390 con 0x7f1d080ae600 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d04c00640 1 -- 172.21.15.67:0/3614970510 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00003a40 con 0x7f1d080ae600 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.698+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d081f0950 con 0x7f1d080ae600 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.700+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d080689e0 con 0x7f1d080ae600 2024-05-30T16:51:59.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.701+0000 7f1d04c00640 1 -- 172.21.15.67:0/3614970510 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f1d0001c020 con 0x7f1d080ae600 2024-05-30T16:51:59.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.702+0000 7f1d04c00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ce806cfb0 0x7f1ce806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:51:59.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.702+0000 7f1d04c00640 1 -- 172.21.15.67:0/3614970510 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f1d0008c9b0 con 0x7f1d080ae600 2024-05-30T16:51:59.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.702+0000 7f1d0ca00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ce806cfb0 0x7f1ce806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:51:59.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.703+0000 7f1d0ca00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ce806cfb0 0x7f1ce806f470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc033000 tx=0x7f1cfc009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:51:59.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.707+0000 7f1d04c00640 1 -- 172.21.15.67:0/3614970510 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d00023030 con 0x7f1d080ae600 2024-05-30T16:51:59.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.988+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1d080c70b0 con 0x7f1d080ae600 2024-05-30T16:51:59.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.989+0000 7f1d04c00640 1 -- 172.21.15.67:0/3614970510 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2077 (secure 0 0 0) 0x7f1d0005a3d0 con 0x7f1d080ae600 2024-05-30T16:51:59.990 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:51:59.990 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:51:59.990 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:51:59.990 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:51:59.990 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:51:59.990 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 513, 2024-05-30T16:51:59.991 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 50, 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:51:59.992 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4258, 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 17727134270, 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 35838033920, 2024-05-30T16:51:59.993 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 737256079360, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 170, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 26, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 3, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:51:59.994 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 3, 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5361 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:51:59.995 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 8, 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:49:38.113647+0000", 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "de99db33-4236-4531-bbc4-b658dcb5478d": { 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 5.0 seconds before exiting (38s)\n [============================] ", 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:51:59.996 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:59.997 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:51:59.997 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:51:59.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ce806cfb0 msgr2=0x7f1ce806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ce806cfb0 0x7f1ce806f470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc033000 tx=0x7f1cfc009030 comp rx=0 tx=0).stop 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d080ae600 msgr2=0x7f1d080aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d080ae600 0x7f1d080aea80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1d080cb660 tx=0x7f1d00011920 comp rx=0 tx=0).stop 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 shutdown_connections 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ce806cfb0 0x7f1ce806f470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d080ae600 0x7f1d080aea80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d080c72f0 0x7f1d080b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 --2- 172.21.15.67:0/3614970510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00039e0 0x7f1d080c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.991+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 >> 172.21.15.67:0/3614970510 conn(0x7f1d0806bae0 msgr2=0x7f1d080cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.992+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 shutdown_connections 2024-05-30T16:51:59.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:51:59.992+0000 7f1d0de00640 1 -- 172.21.15.67:0/3614970510 wait complete. 2024-05-30T16:52:05.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:52:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.238+0000 7f5290a00640 1 Processor -- start 2024-05-30T16:52:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.238+0000 7f5290a00640 1 -- start start 2024-05-30T16:52:05.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.240+0000 7f5290a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c163dc0 0x7f528c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.240+0000 7f5290a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f528c15d550 con 0x7f528c15d170 2024-05-30T16:52:05.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.240+0000 7f5290a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f528c0ad1d0 con 0x7f528c0d09a0 2024-05-30T16:52:05.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.240+0000 7f5290a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f528c0c57d0 con 0x7f528c163dc0 2024-05-30T16:52:05.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.240+0000 7f528b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f528c15d170 0x7f528c0c50c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45138/0 (socket says 172.21.15.67:45138) 2024-05-30T16:52:05.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.240+0000 7f528b400640 1 -- 172.21.15.67:0/2255114689 learned_addr learned my addr 172.21.15.67:0/2255114689 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:05.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f5282a00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c163dc0 0x7f528c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741845008 0 0) 0x7f528c15d550 con 0x7f528c15d170 2024-05-30T16:52:05.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f526c003520 con 0x7f528c15d170 2024-05-30T16:52:05.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 241145651 0 0) 0x7f526c003520 con 0x7f528c15d170 2024-05-30T16:52:05.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c163dc0 msgr2=0x7f528c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:05.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c163dc0 0x7f528c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 >> v1:172.21.15.169:6789/0 conn(0x7f528c0d09a0 legacy=0x7f528c0b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c0ad4b0 con 0x7f528c15d170 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f527c001670 con 0x7f528c15d170 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 >> v1:172.21.15.67:6789/0 conn(0x7f528c15d170 legacy=0x7f528c0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.241+0000 7f528aa00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f526c0039e0 0x7f526c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f528aa00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f526c004800 0x7f526c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f528aa00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c163dc0 0x7f526c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f528c0c57d0 con 0x7f526c004800 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f526c00baa0 con 0x7f526c0039e0 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f526c00bca0 con 0x7f528c163dc0 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f5282a00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f526c004800 0x7f526c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f528b400640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f526c0039e0 0x7f526c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f5282a00640 1 -- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c163dc0 msgr2=0x7f526c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:05.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f5282a00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c163dc0 0x7f526c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f5282a00640 1 -- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f526c0039e0 msgr2=0x7f526c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:05.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f5282a00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f526c0039e0 0x7f526c003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.242+0000 7f5282a00640 1 -- 172.21.15.67:0/2255114689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c003520 con 0x7f526c004800 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.243+0000 7f5282a00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f526c004800 0x7f526c004c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5274009a90 tx=0x7f526c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d94e79cbfa9070 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.244+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274036070 con 0x7f526c004800 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.244+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5274031400 con 0x7f526c004800 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.244+0000 7f528aa00640 1 -- 172.21.15.67:0/2255114689 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274009690 con 0x7f526c004800 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.244+0000 7f5290a00640 1 -- 172.21.15.67:0/2255114689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f526c004800 msgr2=0x7f526c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.244+0000 7f5290a00640 1 --2- 172.21.15.67:0/2255114689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f526c004800 0x7f526c004c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5274009a90 tx=0x7f526c005830 comp rx=0 tx=0).stop 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.244+0000 7f528be00640 1 -- 172.21.15.67:0/2255114689 reap_dead start 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.244+0000 7f5290a00640 1 -- 172.21.15.67:0/2255114689 shutdown_connections 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.244+0000 7f5290a00640 1 -- 172.21.15.67:0/2255114689 >> 172.21.15.67:0/2255114689 conn(0x7f528c06bae0 msgr2=0x7f528c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:05.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.245+0000 7f5290a00640 1 -- 172.21.15.67:0/2255114689 shutdown_connections 2024-05-30T16:52:05.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.245+0000 7f5290a00640 1 -- 172.21.15.67:0/2255114689 wait complete. 2024-05-30T16:52:05.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.246+0000 7f5290a00640 1 Processor -- start 2024-05-30T16:52:05.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.247+0000 7f5290a00640 1 -- start start 2024-05-30T16:52:05.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f5290a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f526c004800 0x7f528c0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f5290a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c156600 0x7f528c155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f5290a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f528c0c7e20 0x7f528c0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f5290a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f528c0d0e30 con 0x7f528c0c7e20 2024-05-30T16:52:05.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f5290a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f528c0adb00 con 0x7f526c004800 2024-05-30T16:52:05.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f5290a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f528c0aefa0 con 0x7f528c156600 2024-05-30T16:52:05.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f528be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f528c0c7e20 0x7f528c0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f528be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f528c0c7e20 0x7f528c0c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52008/0 (socket says 172.21.15.67:52008) 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f528be00640 1 -- 172.21.15.67:0/1273906795 learned_addr learned my addr 172.21.15.67:0/1273906795 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f528b400640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f526c004800 0x7f528c0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f5282a00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c156600 0x7f528c155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f528be00640 1 -- 172.21.15.67:0/1273906795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c156600 msgr2=0x7f528c155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f528be00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c156600 0x7f528c155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f528be00640 1 -- 172.21.15.67:0/1273906795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f526c004800 msgr2=0x7f528c0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.248+0000 7f528be00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f526c004800 0x7f528c0c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.249+0000 7f528be00640 1 -- 172.21.15.67:0/1273906795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c0cf160 con 0x7f528c0c7e20 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.249+0000 7f528be00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f528c0c7e20 0x7f528c0c82a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f528c0cdc80 tx=0x7f528400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.249+0000 7f5283e00640 1 -- 172.21.15.67:0/1273906795 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284017070 con 0x7f528c0c7e20 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.249+0000 7f5283e00640 1 -- 172.21.15.67:0/1273906795 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5284005d60 con 0x7f528c0c7e20 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.249+0000 7f5283e00640 1 -- 172.21.15.67:0/1273906795 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528400a850 con 0x7f528c0c7e20 2024-05-30T16:52:05.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.249+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528c0cf440 con 0x7f528c0c7e20 2024-05-30T16:52:05.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.249+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528c0725c0 con 0x7f528c0c7e20 2024-05-30T16:52:05.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.250+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528c0689e0 con 0x7f528c0c7e20 2024-05-30T16:52:05.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.252+0000 7f5283e00640 1 -- 172.21.15.67:0/1273906795 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f528400a9f0 con 0x7f528c0c7e20 2024-05-30T16:52:05.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.252+0000 7f5283e00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f526806cfb0 0x7f526806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.253+0000 7f5283e00640 1 -- 172.21.15.67:0/1273906795 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f528408bb00 con 0x7f528c0c7e20 2024-05-30T16:52:05.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.253+0000 7f528b400640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f526806cfb0 0x7f526806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.253+0000 7f528b400640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f526806cfb0 0x7f526806f470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f527c005a90 tx=0x7f527c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:05.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.257+0000 7f5283e00640 1 -- 172.21.15.67:0/1273906795 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5284059490 con 0x7f528c0c7e20 2024-05-30T16:52:05.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.540+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f528c099810 con 0x7f528c0c7e20 2024-05-30T16:52:05.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.541+0000 7f5283e00640 1 -- 172.21.15.67:0/1273906795 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1730 (secure 0 0 0) 0x7f5284058d90 con 0x7f528c0c7e20 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:52:05.542 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 518, 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 50, 2024-05-30T16:52:05.543 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:52:05.544 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4258, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 17727134270, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 35838033920, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 737256079360, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 26, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 3, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:52:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 3, 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5361 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:52:05.546 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 8, 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:49:38.113647+0000", 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": {} 2024-05-30T16:52:05.547 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:52:05.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f526806cfb0 msgr2=0x7f526806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f526806cfb0 0x7f526806f470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f527c005a90 tx=0x7f527c009040 comp rx=0 tx=0).stop 2024-05-30T16:52:05.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f528c0c7e20 msgr2=0x7f528c0c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f528c0c7e20 0x7f528c0c82a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f528c0cdc80 tx=0x7f528400cb90 comp rx=0 tx=0).stop 2024-05-30T16:52:05.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 shutdown_connections 2024-05-30T16:52:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f526806cfb0 0x7f526806f470 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f528c0c7e20 0x7f528c0c82a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f528c156600 0x7f528c155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 --2- 172.21.15.67:0/1273906795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f526c004800 0x7f528c0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 >> 172.21.15.67:0/1273906795 conn(0x7f528c06bae0 msgr2=0x7f528c0b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 shutdown_connections 2024-05-30T16:52:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.544+0000 7f5290a00640 1 -- 172.21.15.67:0/1273906795 wait complete. 2024-05-30T16:52:05.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-30T16:52:05.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.780+0000 7faecdc00640 1 Processor -- start 2024-05-30T16:52:05.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.780+0000 7faecdc00640 1 -- start start 2024-05-30T16:52:05.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.782+0000 7faecdc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faec8153510 0x7faec80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.782+0000 7faecdc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faec815d270 0x7faec80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.782+0000 7faecdc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faec815d900 con 0x7faec80ccb90 2024-05-30T16:52:05.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.782+0000 7faecdc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7faec81538f0 con 0x7faec815d270 2024-05-30T16:52:05.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.782+0000 7faecdc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faec80cd530 con 0x7faec8153510 2024-05-30T16:52:05.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.782+0000 7faecd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faec815d270 0x7faec80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.782+0000 7faec7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faec80ccb90 0x7faec815cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45152/0 (socket says 172.21.15.67:45152) 2024-05-30T16:52:05.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.782+0000 7faecd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faec815d270 0x7faec80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52012/0 (socket says 172.21.15.67:52012) 2024-05-30T16:52:05.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.782+0000 7faec7400640 1 -- 172.21.15.67:0/4047644958 learned_addr learned my addr 172.21.15.67:0/4047644958 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:05.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.783+0000 7faec7e00640 1 --2- 172.21.15.67:0/4047644958 >> v2:172.21.15.169:3300/0 conn(0x7faec8153510 0x7faec80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.783+0000 7faecd200640 1 -- 172.21.15.67:0/4047644958 >> v2:172.21.15.169:3300/0 conn(0x7faec8153510 msgr2=0x7faec80b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:05.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.783+0000 7faecd200640 1 --2- 172.21.15.67:0/4047644958 >> v2:172.21.15.169:3300/0 conn(0x7faec8153510 0x7faec80b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.783+0000 7faecd200640 1 -- 172.21.15.67:0/4047644958 >> v1:172.21.15.67:6789/0 conn(0x7faec80ccb90 legacy=0x7faec815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.783+0000 7faecd200640 1 -- 172.21.15.67:0/4047644958 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec80b3f90 con 0x7faec815d270 2024-05-30T16:52:05.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.784+0000 7faecd200640 1 --2- 172.21.15.67:0/4047644958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec815d270 0x7faec80ccff0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7faec80b3970 tx=0x7faec002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37af1f64b5ee0a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:05.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.784+0000 7faec6a00640 1 -- 172.21.15.67:0/4047644958 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0007740 con 0x7faec815d270 2024-05-30T16:52:05.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.784+0000 7faec6a00640 1 -- 172.21.15.67:0/4047644958 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec00078e0 con 0x7faec815d270 2024-05-30T16:52:05.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.784+0000 7faec6a00640 1 -- 172.21.15.67:0/4047644958 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0007cb0 con 0x7faec815d270 2024-05-30T16:52:05.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.785+0000 7faecdc00640 1 -- 172.21.15.67:0/4047644958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec815d270 msgr2=0x7faec80ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.785+0000 7faecdc00640 1 --2- 172.21.15.67:0/4047644958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec815d270 0x7faec80ccff0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7faec80b3970 tx=0x7faec002a5f0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.785+0000 7faecdc00640 1 -- 172.21.15.67:0/4047644958 shutdown_connections 2024-05-30T16:52:05.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.785+0000 7faecdc00640 1 --2- 172.21.15.67:0/4047644958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec815d270 0x7faec80ccff0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.785+0000 7faecdc00640 1 --2- 172.21.15.67:0/4047644958 >> v2:172.21.15.169:3300/0 conn(0x7faec8153510 0x7faec80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.785+0000 7faecdc00640 1 -- 172.21.15.67:0/4047644958 >> 172.21.15.67:0/4047644958 conn(0x7faec806b900 msgr2=0x7faec806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:05.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.785+0000 7faecdc00640 1 -- 172.21.15.67:0/4047644958 shutdown_connections 2024-05-30T16:52:05.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.785+0000 7faecdc00640 1 -- 172.21.15.67:0/4047644958 wait complete. 2024-05-30T16:52:05.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.788+0000 7faecdc00640 1 Processor -- start 2024-05-30T16:52:05.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.788+0000 7faecdc00640 1 -- start start 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.789+0000 7faecdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec80ccb90 0x7faec8099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.789+0000 7faecdc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec8153510 0x7faec8150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.789+0000 7faecdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faec815d270 0x7faec81e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.789+0000 7faecdc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faec815d6a0 con 0x7faec80ccb90 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.789+0000 7faecdc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faec80b48c0 con 0x7faec8153510 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.789+0000 7faecdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faec80b4550 con 0x7faec815d270 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.789+0000 7faec7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec80ccb90 0x7faec8099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faec7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec80ccb90 0x7faec8099910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52024/0 (socket says 172.21.15.67:52024) 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faec7e00640 1 -- 172.21.15.67:0/1945080886 learned_addr learned my addr 172.21.15.67:0/1945080886 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faecd200640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faec815d270 0x7faec81e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faec7400640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec8153510 0x7faec8150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faec7e00640 1 -- 172.21.15.67:0/1945080886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faec815d270 msgr2=0x7faec81e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faec7e00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faec815d270 0x7faec81e83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faec7e00640 1 -- 172.21.15.67:0/1945080886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec8153510 msgr2=0x7faec8150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:05.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faec7e00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec8153510 0x7faec8150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:05.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faec7e00640 1 -- 172.21.15.67:0/1945080886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec81e5ef0 con 0x7faec80ccb90 2024-05-30T16:52:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.790+0000 7faec7e00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec80ccb90 0x7faec8099910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7faeb800d800 tx=0x7faeb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.791+0000 7faebfe00640 1 -- 172.21.15.67:0/1945080886 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb800b6c0 con 0x7faec80ccb90 2024-05-30T16:52:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.791+0000 7faebfe00640 1 -- 172.21.15.67:0/1945080886 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeb8019070 con 0x7faec80ccb90 2024-05-30T16:52:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.791+0000 7faebfe00640 1 -- 172.21.15.67:0/1945080886 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb800ba80 con 0x7faec80ccb90 2024-05-30T16:52:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.791+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec81e61d0 con 0x7faec80ccb90 2024-05-30T16:52:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.791+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec81e7ed0 con 0x7faec80ccb90 2024-05-30T16:52:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.792+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec80689e0 con 0x7faec80ccb90 2024-05-30T16:52:05.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.793+0000 7faebfe00640 1 -- 172.21.15.67:0/1945080886 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7faeb800bc20 con 0x7faec80ccb90 2024-05-30T16:52:05.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.794+0000 7faebfe00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faea806cfe0 0x7faea806f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:05.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.794+0000 7faebfe00640 1 -- 172.21.15.67:0/1945080886 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7faeb808cb80 con 0x7faec80ccb90 2024-05-30T16:52:05.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.795+0000 7faec7400640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faea806cfe0 0x7faea806f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:05.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.795+0000 7faec7400640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faea806cfe0 0x7faea806f4a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faec81f6ab0 tx=0x7faeb4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:05.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:05.800+0000 7faebfe00640 1 -- 172.21.15.67:0/1945080886 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faeb801c360 con 0x7faec80ccb90 2024-05-30T16:52:06.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.030+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faec80b3c40 con 0x7faea806cfe0 2024-05-30T16:52:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.041+0000 7faebfe00640 1 -- 172.21.15.67:0/1945080886 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+234 (secure 0 0 0) 0x7faec80b3c40 con 0x7faea806cfe0 2024-05-30T16:52:06.042 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:06.042 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:52:06.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:06.042 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone75833" 2024-05-30T16:52:06.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:06.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:06.042 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sv1" 2024-05-30T16:52:06.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone75831" 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone75834" 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone75832" 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:06.043 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:52:06.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.043+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faea806cfe0 msgr2=0x7faea806f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.043+0000 7faecdc00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faea806cfe0 0x7faea806f4a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faec81f6ab0 tx=0x7faeb4009040 comp rx=0 tx=0).stop 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.043+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec80ccb90 msgr2=0x7faec8099910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.043+0000 7faecdc00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec80ccb90 0x7faec8099910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7faeb800d800 tx=0x7faeb800dcc0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.044+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 shutdown_connections 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.044+0000 7faecdc00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faea806cfe0 0x7faea806f4a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.044+0000 7faecdc00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faec815d270 0x7faec81e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.044+0000 7faecdc00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec8153510 0x7faec8150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.044+0000 7faecdc00640 1 --2- 172.21.15.67:0/1945080886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faec80ccb90 0x7faec8099910 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.044+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 >> 172.21.15.67:0/1945080886 conn(0x7faec806b900 msgr2=0x7faec806f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.044+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 shutdown_connections 2024-05-30T16:52:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.044+0000 7faecdc00640 1 -- 172.21.15.67:0/1945080886 wait complete. 2024-05-30T16:52:06.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone75833 2024-05-30T16:52:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.279+0000 7fdab1000640 1 Processor -- start 2024-05-30T16:52:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.279+0000 7fdab1000640 1 -- start start 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.281+0000 7fdab1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaac0d03f0 0x7fdaac0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.281+0000 7fdab1000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdaac0cce20 0x7fdaac0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.281+0000 7fdab1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdaac0cdc30 0x7fdaac0ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.281+0000 7fdab1000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fdaac0d0d10 con 0x7fdaac0cce20 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.281+0000 7fdab1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdaac06b900 con 0x7fdaac0cdc30 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.281+0000 7fdab1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdaac06bbc0 con 0x7fdaac0d03f0 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.281+0000 7fdaa3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdaac0cce20 0x7fdaac0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.281+0000 7fdaa3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdaac0cce20 0x7fdaac0cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52036/0 (socket says 172.21.15.67:52036) 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.281+0000 7fdaa3e00640 1 -- 172.21.15.67:0/1462579456 learned_addr learned my addr 172.21.15.67:0/1462579456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.282+0000 7fdaab400640 1 --2- 172.21.15.67:0/1462579456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaac0d03f0 0x7fdaac0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.282+0000 7fdaabe00640 1 --2- 172.21.15.67:0/1462579456 >> v2:172.21.15.169:3300/0 conn(0x7fdaac0cdc30 0x7fdaac0ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.282+0000 7fdaa3e00640 1 -- 172.21.15.67:0/1462579456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaac0d03f0 msgr2=0x7fdaac0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:06.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.282+0000 7fdaa3e00640 1 --2- 172.21.15.67:0/1462579456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaac0d03f0 0x7fdaac0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.282+0000 7fdaa3e00640 1 -- 172.21.15.67:0/1462579456 >> v2:172.21.15.169:3300/0 conn(0x7fdaac0cdc30 msgr2=0x7fdaac0ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:06.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.282+0000 7fdaa3e00640 1 --2- 172.21.15.67:0/1462579456 >> v2:172.21.15.169:3300/0 conn(0x7fdaac0cdc30 0x7fdaac0ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.282+0000 7fdaa3e00640 1 -- 172.21.15.67:0/1462579456 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac06bea0 con 0x7fdaac0cce20 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.283+0000 7fdaa3e00640 1 --2- 172.21.15.67:0/1462579456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 0x7fdaac0cd260 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0b3f90 tx=0x7fda9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9082dc363c85a962 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.283+0000 7fdaaaa00640 1 -- 172.21.15.67:0/1462579456 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94037070 con 0x7fdaac0cce20 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.283+0000 7fdaaaa00640 1 -- 172.21.15.67:0/1462579456 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda94032440 con 0x7fdaac0cce20 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.283+0000 7fdaaaa00640 1 -- 172.21.15.67:0/1462579456 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94032810 con 0x7fdaac0cce20 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.284+0000 7fdab1000640 1 -- 172.21.15.67:0/1462579456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 msgr2=0x7fdaac0cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.284+0000 7fdab1000640 1 --2- 172.21.15.67:0/1462579456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 0x7fdaac0cd260 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0b3f90 tx=0x7fda9402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.284+0000 7fdab1000640 1 -- 172.21.15.67:0/1462579456 shutdown_connections 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.284+0000 7fdab1000640 1 --2- 172.21.15.67:0/1462579456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 0x7fdaac0cd260 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.284+0000 7fdab1000640 1 --2- 172.21.15.67:0/1462579456 >> v2:172.21.15.169:3300/0 conn(0x7fdaac0cdc30 0x7fdaac0ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.284+0000 7fdab1000640 1 --2- 172.21.15.67:0/1462579456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaac0d03f0 0x7fdaac0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.284+0000 7fdab1000640 1 -- 172.21.15.67:0/1462579456 >> 172.21.15.67:0/1462579456 conn(0x7fdaac0b2ed0 msgr2=0x7fdaac0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.284+0000 7fdab1000640 1 -- 172.21.15.67:0/1462579456 shutdown_connections 2024-05-30T16:52:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.284+0000 7fdab1000640 1 -- 172.21.15.67:0/1462579456 wait complete. 2024-05-30T16:52:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.287+0000 7fdab1000640 1 Processor -- start 2024-05-30T16:52:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.287+0000 7fdab1000640 1 -- start start 2024-05-30T16:52:06.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.289+0000 7fdab1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 0x7fdaac1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.289+0000 7fdab1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaac0cdc30 0x7fdaac09ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.289+0000 7fdab1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaac0d03f0 0x7fdaac13dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.289+0000 7fdab1000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac0cd7a0 con 0x7fdaac0cce20 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.289+0000 7fdab1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1455e0 con 0x7fdaac0cdc30 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.289+0000 7fdab1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdaac0cd920 con 0x7fdaac0d03f0 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.289+0000 7fdaab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 0x7fdaac1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.289+0000 7fdaab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 0x7fdaac1587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52050/0 (socket says 172.21.15.67:52050) 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.289+0000 7fdaab400640 1 -- 172.21.15.67:0/1616274776 learned_addr learned my addr 172.21.15.67:0/1616274776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdaab400640 1 -- 172.21.15.67:0/1616274776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaac0d03f0 msgr2=0x7fdaac13dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdaab400640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaac0d03f0 0x7fdaac13dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdaab400640 1 -- 172.21.15.67:0/1616274776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaac0cdc30 msgr2=0x7fdaac09ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdaab400640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaac0cdc30 0x7fdaac09ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdaab400640 1 -- 172.21.15.67:0/1616274776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac09b720 con 0x7fdaac0cce20 2024-05-30T16:52:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdaab400640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 0x7fdaac1587b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00d800 tx=0x7fda9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdaa3400640 1 -- 172.21.15.67:0/1616274776 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00b6c0 con 0x7fdaac0cce20 2024-05-30T16:52:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdaa3400640 1 -- 172.21.15.67:0/1616274776 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9c019070 con 0x7fdaac0cce20 2024-05-30T16:52:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdaa3400640 1 -- 172.21.15.67:0/1616274776 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00ba60 con 0x7fdaac0cce20 2024-05-30T16:52:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac09ba00 con 0x7fdaac0cce20 2024-05-30T16:52:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.290+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac09bfc0 con 0x7fdaac0cce20 2024-05-30T16:52:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.292+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaac0690f0 con 0x7fdaac0cce20 2024-05-30T16:52:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.293+0000 7fdaa3400640 1 -- 172.21.15.67:0/1616274776 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7fda9c00bc00 con 0x7fdaac0cce20 2024-05-30T16:52:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.295+0000 7fdaa3400640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda8806cfb0 0x7fda8806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.295+0000 7fdaa3400640 1 -- 172.21.15.67:0/1616274776 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fda9c08cbe0 con 0x7fdaac0cce20 2024-05-30T16:52:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.295+0000 7fdaa3e00640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda8806cfb0 0x7fda8806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.296+0000 7fdaa3e00640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda8806cfb0 0x7fda8806f470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fda94034000 tx=0x7fda9402c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:06.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.299+0000 7fdaa3400640 1 -- 172.21.15.67:0/1616274776 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda9c011340 con 0x7fdaac0cce20 2024-05-30T16:52:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.523+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone75833", "target": ["mon-mgr", ""]}) -- 0x7fdaac0affc0 con 0x7fda8806cfb0 2024-05-30T16:52:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.563+0000 7fdaa3400640 1 -- 172.21.15.67:0/1616274776 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdaac0affc0 con 0x7fda8806cfb0 2024-05-30T16:52:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda8806cfb0 msgr2=0x7fda8806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda8806cfb0 0x7fda8806f470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fda94034000 tx=0x7fda9402c000 comp rx=0 tx=0).stop 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 msgr2=0x7fdaac1587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 0x7fdaac1587b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00d800 tx=0x7fda9c00dcc0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 shutdown_connections 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda8806cfb0 0x7fda8806f470 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdaac0d03f0 0x7fdaac13dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaac0cdc30 0x7fdaac09ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 --2- 172.21.15.67:0/1616274776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaac0cce20 0x7fdaac1587b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.565+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 >> 172.21.15.67:0/1616274776 conn(0x7fdaac0b2ed0 msgr2=0x7fdaac0b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.566+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 shutdown_connections 2024-05-30T16:52:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.566+0000 7fdab1000640 1 -- 172.21.15.67:0/1616274776 wait complete. 2024-05-30T16:52:06.570 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.569+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.570 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.569+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.570 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.569+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.570 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.569+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.570 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.569+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-30T16:52:06.593 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.590+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.593 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.590+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.593 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.590+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.593 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.590+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.593 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:06.590+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:06.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.773+0000 7f3664200640 1 Processor -- start 2024-05-30T16:52:06.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.774+0000 7f3664200640 1 -- start start 2024-05-30T16:52:06.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.776+0000 7f3664200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f365c073080 0x7f365c06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.776+0000 7f3664200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f365c15f240 con 0x7f365c06c910 2024-05-30T16:52:06.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.776+0000 7f3664200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f365c15f520 con 0x7f365c15ed60 2024-05-30T16:52:06.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.776+0000 7f3664200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f365c0734d0 con 0x7f365c073080 2024-05-30T16:52:06.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.776+0000 7f3661800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f365c06c910 0x7f365c06ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45160/0 (socket says 172.21.15.67:45160) 2024-05-30T16:52:06.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.776+0000 7f3661800640 1 -- 172.21.15.67:0/880262742 learned_addr learned my addr 172.21.15.67:0/880262742 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:06.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.776+0000 7f3662200640 1 --2- 172.21.15.67:0/880262742 >> v2:172.21.15.169:3300/0 conn(0x7f365c073080 0x7f365c06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042886296 0 0) 0x7f365c15f240 con 0x7f365c06c910 2024-05-30T16:52:06.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3644003520 con 0x7f365c06c910 2024-05-30T16:52:06.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 155734852 0 0) 0x7f3644003520 con 0x7f365c06c910 2024-05-30T16:52:06.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 >> v2:172.21.15.169:3300/0 conn(0x7f365c073080 msgr2=0x7f365c06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:06.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 --2- 172.21.15.67:0/880262742 >> v2:172.21.15.169:3300/0 conn(0x7f365c073080 0x7f365c06ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 >> v1:172.21.15.169:6789/0 conn(0x7f365c15ed60 legacy=0x7f365c06e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c15f800 con 0x7f365c06c910 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f3650001670 con 0x7f365c06c910 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 >> v1:172.21.15.67:6789/0 conn(0x7f365c06c910 legacy=0x7f365c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36440039b0 0x7f3644003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3644004800 0x7f3644004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36440051c0 0x7f364400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f364400c0e0 con 0x7f36440051c0 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f364400c290 con 0x7f3644004800 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f364400c490 con 0x7f36440039b0 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.777+0000 7f3662200640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36440051c0 0x7f364400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.778+0000 7f3660e00640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3644004800 0x7f3644004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.778+0000 7f3661800640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36440039b0 0x7f3644003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.778+0000 7f3662200640 1 -- 172.21.15.67:0/880262742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36440039b0 msgr2=0x7f3644003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:06.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.778+0000 7f3662200640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36440039b0 0x7f3644003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.778+0000 7f3662200640 1 -- 172.21.15.67:0/880262742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3644004800 msgr2=0x7f3644004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:06.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.778+0000 7f3662200640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3644004800 0x7f3644004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.778+0000 7f3662200640 1 -- 172.21.15.67:0/880262742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3644003520 con 0x7f36440051c0 2024-05-30T16:52:06.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.779+0000 7f3662200640 1 -- 172.21.15.67:0/880262742 reap_dead start 2024-05-30T16:52:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.779+0000 7f3662200640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36440051c0 0x7f364400b9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3654007360 tx=0x7f3644005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ec70c9be1061f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.779+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654009be0 con 0x7f36440051c0 2024-05-30T16:52:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.779+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3654003bd0 con 0x7f36440051c0 2024-05-30T16:52:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.779+0000 7f365be00640 1 -- 172.21.15.67:0/880262742 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654004020 con 0x7f36440051c0 2024-05-30T16:52:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.780+0000 7f3664200640 1 -- 172.21.15.67:0/880262742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36440051c0 msgr2=0x7f364400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.780+0000 7f3664200640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36440051c0 0x7f364400b9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3654007360 tx=0x7f3644005830 comp rx=0 tx=0).stop 2024-05-30T16:52:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.780+0000 7f3664200640 1 -- 172.21.15.67:0/880262742 shutdown_connections 2024-05-30T16:52:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.780+0000 7f3664200640 1 --2- 172.21.15.67:0/880262742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36440051c0 0x7f364400b9c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.780+0000 7f3664200640 1 -- 172.21.15.67:0/880262742 >> 172.21.15.67:0/880262742 conn(0x7f365c072560 msgr2=0x7f365c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.780+0000 7f3664200640 1 -- 172.21.15.67:0/880262742 shutdown_connections 2024-05-30T16:52:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.780+0000 7f3664200640 1 -- 172.21.15.67:0/880262742 wait complete. 2024-05-30T16:52:06.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.782+0000 7f3664200640 1 Processor -- start 2024-05-30T16:52:06.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.782+0000 7f3664200640 1 -- start start 2024-05-30T16:52:06.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3664200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36440051c0 0x7f365c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3664200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f365c14c9b0 0x7f365c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3664200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c159dd0 0x7f365c15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3664200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f365c15c670 con 0x7f365c14c9b0 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3664200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f365c067870 con 0x7f365c159dd0 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3664200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f365c1590f0 con 0x7f36440051c0 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3660e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f365c14c9b0 0x7f365c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3660e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f365c14c9b0 0x7f365c158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52098/0 (socket says 172.21.15.67:52098) 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3660e00640 1 -- 172.21.15.67:0/3212541381 learned_addr learned my addr 172.21.15.67:0/3212541381 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.783+0000 7f3661800640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36440051c0 0x7f365c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3662200640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c159dd0 0x7f365c15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3660e00640 1 -- 172.21.15.67:0/3212541381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36440051c0 msgr2=0x7f365c156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3660e00640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36440051c0 0x7f365c156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3660e00640 1 -- 172.21.15.67:0/3212541381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c159dd0 msgr2=0x7f365c15a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:06.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3660e00640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c159dd0 0x7f365c15a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:06.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3660e00640 1 -- 172.21.15.67:0/3212541381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c1571a0 con 0x7f365c14c9b0 2024-05-30T16:52:06.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3660e00640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f365c14c9b0 0x7f365c158bb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f364800e500 tx=0x7f364800e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:06.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3659600640 1 -- 172.21.15.67:0/3212541381 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36480052e0 con 0x7f365c14c9b0 2024-05-30T16:52:06.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3659600640 1 -- 172.21.15.67:0/3212541381 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3648018070 con 0x7f365c14c9b0 2024-05-30T16:52:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3659600640 1 -- 172.21.15.67:0/3212541381 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648003810 con 0x7f365c14c9b0 2024-05-30T16:52:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c157480 con 0x7f365c14c9b0 2024-05-30T16:52:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.784+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c1540e0 con 0x7f365c14c9b0 2024-05-30T16:52:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.785+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365c15a790 con 0x7f365c14c9b0 2024-05-30T16:52:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.786+0000 7f3659600640 1 -- 172.21.15.67:0/3212541381 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f36480039b0 con 0x7f365c14c9b0 2024-05-30T16:52:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.787+0000 7f3659600640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f363c06cfb0 0x7f363c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.787+0000 7f3659600640 1 -- 172.21.15.67:0/3212541381 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f364808c3b0 con 0x7f365c14c9b0 2024-05-30T16:52:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.788+0000 7f3661800640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f363c06cfb0 0x7f363c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.788+0000 7f3661800640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f363c06cfb0 0x7f363c06f470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f36500017b0 tx=0x7f3650000f10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:06.792+0000 7f3659600640 1 -- 172.21.15.67:0/3212541381 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3648059ee0 con 0x7f365c14c9b0 2024-05-30T16:52:07.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.027+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f365c0d06f0 con 0x7f363c06cfb0 2024-05-30T16:52:07.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.051+0000 7f3659600640 1 -- 172.21.15.67:0/3212541381 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f365c0d06f0 con 0x7f363c06cfb0 2024-05-30T16:52:07.052 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:07.052 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:52:07.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:07.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1" 2024-05-30T16:52:07.052 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:07.052 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f363c06cfb0 msgr2=0x7f363c06f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f363c06cfb0 0x7f363c06f470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f36500017b0 tx=0x7f3650000f10 comp rx=0 tx=0).stop 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f365c14c9b0 msgr2=0x7f365c158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f365c14c9b0 0x7f365c158bb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f364800e500 tx=0x7f364800e9c0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 shutdown_connections 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f363c06cfb0 0x7f363c06f470 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f365c159dd0 0x7f365c15a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f365c14c9b0 0x7f365c158bb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 --2- 172.21.15.67:0/3212541381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36440051c0 0x7f365c156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 >> 172.21.15.67:0/3212541381 conn(0x7f365c072560 msgr2=0x7f365c15b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:07.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 shutdown_connections 2024-05-30T16:52:07.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.054+0000 7f3664200640 1 -- 172.21.15.67:0/3212541381 wait complete. 2024-05-30T16:52:07.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-30T16:52:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.284+0000 7f30d6800640 1 Processor -- start 2024-05-30T16:52:07.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.285+0000 7f30d6800640 1 -- start start 2024-05-30T16:52:07.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.286+0000 7f30d6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f30d015abe0 0x7f30d006f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.286+0000 7f30d6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d006f8c0 0x7f30d00b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.286+0000 7f30d6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30d00b4b10 0x7f30d00725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.286+0000 7f30d6800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f30d006e2e0 con 0x7f30d015abe0 2024-05-30T16:52:07.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.286+0000 7f30d6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f30d006e5c0 con 0x7f30d00b4b10 2024-05-30T16:52:07.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.286+0000 7f30d6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30d006e8a0 con 0x7f30d006f8c0 2024-05-30T16:52:07.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.286+0000 7f30cf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f30d015abe0 0x7f30d006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:07.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.287+0000 7f30cf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f30d015abe0 0x7f30d006f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52106/0 (socket says 172.21.15.67:52106) 2024-05-30T16:52:07.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.287+0000 7f30cf400640 1 -- 172.21.15.67:0/1734978387 learned_addr learned my addr 172.21.15.67:0/1734978387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:07.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.287+0000 7f30cf400640 1 -- 172.21.15.67:0/1734978387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d006f8c0 msgr2=0x7f30d00b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:07.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.287+0000 7f30cf400640 1 --2- 172.21.15.67:0/1734978387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d006f8c0 0x7f30d00b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.287+0000 7f30cf400640 1 -- 172.21.15.67:0/1734978387 >> v2:172.21.15.169:3300/0 conn(0x7f30d00b4b10 msgr2=0x7f30d00725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:07.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.287+0000 7f30cf400640 1 --2- 172.21.15.67:0/1734978387 >> v2:172.21.15.169:3300/0 conn(0x7f30d00b4b10 0x7f30d00725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.287+0000 7f30cf400640 1 -- 172.21.15.67:0/1734978387 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d006eb80 con 0x7f30d015abe0 2024-05-30T16:52:07.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.288+0000 7f30cf400640 1 --2- 172.21.15.67:0/1734978387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d015abe0 0x7f30d006f380 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f30d00b5370 tx=0x7f30c002b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3db9197590f20f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.288+0000 7f30ce000640 1 -- 172.21.15.67:0/1734978387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0038070 con 0x7f30d015abe0 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.288+0000 7f30ce000640 1 -- 172.21.15.67:0/1734978387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c0033640 con 0x7f30d015abe0 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.288+0000 7f30ce000640 1 -- 172.21.15.67:0/1734978387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0033a10 con 0x7f30d015abe0 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.289+0000 7f30d6800640 1 -- 172.21.15.67:0/1734978387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d015abe0 msgr2=0x7f30d006f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.289+0000 7f30d6800640 1 --2- 172.21.15.67:0/1734978387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d015abe0 0x7f30d006f380 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f30d00b5370 tx=0x7f30c002b2f0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.289+0000 7f30d6800640 1 -- 172.21.15.67:0/1734978387 shutdown_connections 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.289+0000 7f30d6800640 1 --2- 172.21.15.67:0/1734978387 >> v2:172.21.15.169:3300/0 conn(0x7f30d00b4b10 0x7f30d00725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.289+0000 7f30d6800640 1 --2- 172.21.15.67:0/1734978387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d006f8c0 0x7f30d00b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.289+0000 7f30d6800640 1 --2- 172.21.15.67:0/1734978387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d015abe0 0x7f30d006f380 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.289+0000 7f30d6800640 1 -- 172.21.15.67:0/1734978387 >> 172.21.15.67:0/1734978387 conn(0x7f30d006bd20 msgr2=0x7f30d006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.289+0000 7f30d6800640 1 -- 172.21.15.67:0/1734978387 shutdown_connections 2024-05-30T16:52:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.289+0000 7f30d6800640 1 -- 172.21.15.67:0/1734978387 wait complete. 2024-05-30T16:52:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.291+0000 7f30d6800640 1 Processor -- start 2024-05-30T16:52:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.291+0000 7f30d6800640 1 -- start start 2024-05-30T16:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.292+0000 7f30d6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d006f8c0 0x7f30d0159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.292+0000 7f30d6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d00b4b10 0x7f30d0156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.292+0000 7f30d6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d015abe0 0x7f30d00b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.292+0000 7f30d6800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f30d006df60 con 0x7f30d006f8c0 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.292+0000 7f30d6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30d015afc0 con 0x7f30d00b4b10 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.292+0000 7f30d6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30d0072b00 con 0x7f30d015abe0 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.292+0000 7f30cf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d006f8c0 0x7f30d0159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.292+0000 7f30cf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d006f8c0 0x7f30d0159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52128/0 (socket says 172.21.15.67:52128) 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.292+0000 7f30cf400640 1 -- 172.21.15.67:0/3338265106 learned_addr learned my addr 172.21.15.67:0/3338265106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30cea00640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d00b4b10 0x7f30d0156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30cf400640 1 -- 172.21.15.67:0/3338265106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d015abe0 msgr2=0x7f30d00b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30cf400640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d015abe0 0x7f30d00b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30cf400640 1 -- 172.21.15.67:0/3338265106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d00b4b10 msgr2=0x7f30d0156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30cf400640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d00b4b10 0x7f30d0156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30cf400640 1 -- 172.21.15.67:0/3338265106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d01577e0 con 0x7f30d006f8c0 2024-05-30T16:52:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30cea00640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d00b4b10 0x7f30d0156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30cf400640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d006f8c0 0x7f30d0159860 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f30c0000c00 tx=0x7f30c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30c7400640 1 -- 172.21.15.67:0/3338265106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0042020 con 0x7f30d006f8c0 2024-05-30T16:52:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30c7400640 1 -- 172.21.15.67:0/3338265106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c0031770 con 0x7f30d006f8c0 2024-05-30T16:52:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30c7400640 1 -- 172.21.15.67:0/3338265106 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0038040 con 0x7f30d006f8c0 2024-05-30T16:52:07.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.293+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d0157ac0 con 0x7f30d006f8c0 2024-05-30T16:52:07.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.294+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d0159060 con 0x7f30d006f8c0 2024-05-30T16:52:07.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.295+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d00689e0 con 0x7f30d006f8c0 2024-05-30T16:52:07.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.296+0000 7f30c7400640 1 -- 172.21.15.67:0/3338265106 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f30c0045020 con 0x7f30d006f8c0 2024-05-30T16:52:07.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.296+0000 7f30c7400640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f30b006cfb0 0x7f30b006f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.297+0000 7f30c7400640 1 -- 172.21.15.67:0/3338265106 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f30c00ad120 con 0x7f30d006f8c0 2024-05-30T16:52:07.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.297+0000 7f30cea00640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f30b006cfb0 0x7f30b006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:07.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.298+0000 7f30cea00640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f30b006cfb0 0x7f30b006f470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f30bc006fd0 tx=0x7f30bc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:07.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.301+0000 7f30c7400640 1 -- 172.21.15.67:0/3338265106 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30c004c030 con 0x7f30d006f8c0 2024-05-30T16:52:07.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.535+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f30d009ba40 con 0x7f30b006cfb0 2024-05-30T16:52:07.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.592+0000 7f30c7400640 1 -- 172.21.15.67:0/3338265106 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f30d009ba40 con 0x7f30b006cfb0 2024-05-30T16:52:07.594 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:07.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.595+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f30b006cfb0 msgr2=0x7f30b006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f30b006cfb0 0x7f30b006f470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f30bc006fd0 tx=0x7f30bc008040 comp rx=0 tx=0).stop 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d006f8c0 msgr2=0x7f30d0159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d006f8c0 0x7f30d0159860 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f30c0000c00 tx=0x7f30c00046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 shutdown_connections 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f30b006cfb0 0x7f30b006f470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30d015abe0 0x7f30d00b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d00b4b10 0x7f30d0156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 --2- 172.21.15.67:0/3338265106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d006f8c0 0x7f30d0159860 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 >> 172.21.15.67:0/3338265106 conn(0x7f30d006bd20 msgr2=0x7f30d0071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 shutdown_connections 2024-05-30T16:52:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.596+0000 7f30d6800640 1 -- 172.21.15.67:0/3338265106 wait complete. 2024-05-30T16:52:07.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-30T16:52:07.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.794+0000 7fde5b200640 1 Processor -- start 2024-05-30T16:52:07.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.795+0000 7fde5b200640 1 -- start start 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.796+0000 7fde5b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fde5c0caa40 0x7fde5c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.796+0000 7fde5b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fde5c1524c0 0x7fde5c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.796+0000 7fde5b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde5c0b3e80 con 0x7fde5c0cae20 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.796+0000 7fde5b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fde5c0d0b30 con 0x7fde5c1524c0 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.796+0000 7fde5b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fde5c0b4160 con 0x7fde5c0caa40 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.797+0000 7fde59400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fde5c0cae20 0x7fde5c151db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45172/0 (socket says 172.21.15.67:45172) 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.797+0000 7fde59400640 1 -- 172.21.15.67:0/2320105270 learned_addr learned my addr 172.21.15.67:0/2320105270 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.797+0000 7fde5a800640 1 --2- 172.21.15.67:0/2320105270 >> v2:172.21.15.67:3300/0 conn(0x7fde5c1524c0 0x7fde5c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.797+0000 7fde59e00640 1 --2- 172.21.15.67:0/2320105270 >> v2:172.21.15.169:3300/0 conn(0x7fde5c0caa40 0x7fde5c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.797+0000 7fde5a800640 1 -- 172.21.15.67:0/2320105270 >> v2:172.21.15.169:3300/0 conn(0x7fde5c0caa40 msgr2=0x7fde5c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.797+0000 7fde5a800640 1 --2- 172.21.15.67:0/2320105270 >> v2:172.21.15.169:3300/0 conn(0x7fde5c0caa40 0x7fde5c0ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.797+0000 7fde5a800640 1 -- 172.21.15.67:0/2320105270 >> v1:172.21.15.67:6789/0 conn(0x7fde5c0cae20 legacy=0x7fde5c151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.797+0000 7fde5a800640 1 -- 172.21.15.67:0/2320105270 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5c0cde80 con 0x7fde5c1524c0 2024-05-30T16:52:07.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.797+0000 7fde58a00640 1 -- 172.21.15.67:0/2320105270 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377653606 0 0) 0x7fde5c0b3e80 con 0x7fde5c0cae20 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.798+0000 7fde5a800640 1 --2- 172.21.15.67:0/2320105270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c1524c0 0x7fde5c0cf710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0b4c10 tx=0x7fde5402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb060d06c86fc70 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.798+0000 7fde58a00640 1 -- 172.21.15.67:0/2320105270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54037070 con 0x7fde5c1524c0 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde58a00640 1 -- 172.21.15.67:0/2320105270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde54032440 con 0x7fde5c1524c0 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde58a00640 1 -- 172.21.15.67:0/2320105270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5402ae60 con 0x7fde5c1524c0 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde5b200640 1 -- 172.21.15.67:0/2320105270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c1524c0 msgr2=0x7fde5c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde5b200640 1 --2- 172.21.15.67:0/2320105270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c1524c0 0x7fde5c0cf710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0b4c10 tx=0x7fde5402a110 comp rx=0 tx=0).stop 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde5b200640 1 -- 172.21.15.67:0/2320105270 shutdown_connections 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde5b200640 1 --2- 172.21.15.67:0/2320105270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c1524c0 0x7fde5c0cf710 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde5b200640 1 --2- 172.21.15.67:0/2320105270 >> v2:172.21.15.169:3300/0 conn(0x7fde5c0caa40 0x7fde5c0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde5b200640 1 -- 172.21.15.67:0/2320105270 >> 172.21.15.67:0/2320105270 conn(0x7fde5c06b900 msgr2=0x7fde5c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde5b200640 1 -- 172.21.15.67:0/2320105270 shutdown_connections 2024-05-30T16:52:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.799+0000 7fde5b200640 1 -- 172.21.15.67:0/2320105270 wait complete. 2024-05-30T16:52:07.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.801+0000 7fde5b200640 1 Processor -- start 2024-05-30T16:52:07.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.801+0000 7fde5b200640 1 -- start start 2024-05-30T16:52:07.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.802+0000 7fde5b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c0caa40 0x7fde5c150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.802+0000 7fde5b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde5c0cae20 0x7fde5c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.802+0000 7fde5b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde5c1524c0 0x7fde5c154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.802+0000 7fde5b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c0ceea0 con 0x7fde5c0caa40 2024-05-30T16:52:07.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.802+0000 7fde5b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c0ce440 con 0x7fde5c1524c0 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.802+0000 7fde5b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fde5c0cf020 con 0x7fde5c0cae20 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.802+0000 7fde59e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c0caa40 0x7fde5c150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.802+0000 7fde59e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c0caa40 0x7fde5c150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52134/0 (socket says 172.21.15.67:52134) 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.802+0000 7fde59e00640 1 -- 172.21.15.67:0/1044256985 learned_addr learned my addr 172.21.15.67:0/1044256985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde5a800640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde5c1524c0 0x7fde5c154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde59e00640 1 -- 172.21.15.67:0/1044256985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde5c0cae20 msgr2=0x7fde5c1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde59e00640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde5c0cae20 0x7fde5c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde59e00640 1 -- 172.21.15.67:0/1044256985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde5c1524c0 msgr2=0x7fde5c154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde59e00640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde5c1524c0 0x7fde5c154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde59e00640 1 -- 172.21.15.67:0/1044256985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5c155210 con 0x7fde5c0caa40 2024-05-30T16:52:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde59e00640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c0caa40 0x7fde5c150b40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00ce10 tx=0x7fde4c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde52000640 1 -- 172.21.15.67:0/1044256985 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c003b90 con 0x7fde5c0caa40 2024-05-30T16:52:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde52000640 1 -- 172.21.15.67:0/1044256985 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4c003d30 con 0x7fde5c0caa40 2024-05-30T16:52:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde52000640 1 -- 172.21.15.67:0/1044256985 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c004240 con 0x7fde5c0caa40 2024-05-30T16:52:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5c071c20 con 0x7fde5c0caa40 2024-05-30T16:52:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.803+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde5c0721e0 con 0x7fde5c0caa40 2024-05-30T16:52:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.804+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde5c0689e0 con 0x7fde5c0caa40 2024-05-30T16:52:07.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.805+0000 7fde52000640 1 -- 172.21.15.67:0/1044256985 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7fde4c0043e0 con 0x7fde5c0caa40 2024-05-30T16:52:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.806+0000 7fde52000640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fde3806cfb0 0x7fde3806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.806+0000 7fde52000640 1 -- 172.21.15.67:0/1044256985 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fde4c08d2e0 con 0x7fde5c0caa40 2024-05-30T16:52:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.807+0000 7fde59400640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fde3806cfb0 0x7fde3806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:07.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.807+0000 7fde59400640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fde3806cfb0 0x7fde3806f470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde44007fd0 tx=0x7fde44007920 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:07.811+0000 7fde52000640 1 -- 172.21.15.67:0/1044256985 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde4c01d340 con 0x7fde5c0caa40 2024-05-30T16:52:08.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.040+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fde5c0b4ee0 con 0x7fde3806cfb0 2024-05-30T16:52:08.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.064+0000 7fde52000640 1 -- 172.21.15.67:0/1044256985 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde5c0b4ee0 con 0x7fde3806cfb0 2024-05-30T16:52:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fde3806cfb0 msgr2=0x7fde3806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fde3806cfb0 0x7fde3806f470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde44007fd0 tx=0x7fde44007920 comp rx=0 tx=0).stop 2024-05-30T16:52:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c0caa40 msgr2=0x7fde5c150b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c0caa40 0x7fde5c150b40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00ce10 tx=0x7fde4c00a660 comp rx=0 tx=0).stop 2024-05-30T16:52:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 shutdown_connections 2024-05-30T16:52:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fde3806cfb0 0x7fde3806f470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde5c1524c0 0x7fde5c154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde5c0cae20 0x7fde5c1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 --2- 172.21.15.67:0/1044256985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde5c0caa40 0x7fde5c150b40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 >> 172.21.15.67:0/1044256985 conn(0x7fde5c06b900 msgr2=0x7fde5c159b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:08.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.067+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 shutdown_connections 2024-05-30T16:52:08.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.068+0000 7fde5b200640 1 -- 172.21.15.67:0/1044256985 wait complete. 2024-05-30T16:52:08.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone75831 2024-05-30T16:52:08.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.293+0000 7f21cfe00640 1 Processor -- start 2024-05-30T16:52:08.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.293+0000 7f21cfe00640 1 -- start start 2024-05-30T16:52:08.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.295+0000 7f21cfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21d015e800 0x7f21d00aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.295+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 0x7f21d00b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.295+0000 7f21cfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21d015ee50 0x7f21d015f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.295+0000 7f21cfe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f21d00b5780 con 0x7f21d015ee50 2024-05-30T16:52:08.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.295+0000 7f21cfe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f21d015ebe0 con 0x7f21d015e800 2024-05-30T16:52:08.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.295+0000 7f21cfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21d00cfa10 con 0x7f21d00b4de0 2024-05-30T16:52:08.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.295+0000 7f21cf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21d015ee50 0x7f21d015f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.296+0000 7f21cf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21d015ee50 0x7f21d015f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52140/0 (socket says 172.21.15.67:52140) 2024-05-30T16:52:08.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.296+0000 7f21cf400640 1 -- 172.21.15.67:0/2704278840 learned_addr learned my addr 172.21.15.67:0/2704278840 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:08.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.296+0000 7f21ce000640 1 --2- 172.21.15.67:0/2704278840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 0x7f21d00b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.296+0000 7f21cea00640 1 --2- 172.21.15.67:0/2704278840 >> v2:172.21.15.169:3300/0 conn(0x7f21d015e800 0x7f21d00aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.296+0000 7f21cf400640 1 -- 172.21.15.67:0/2704278840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 msgr2=0x7f21d00b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:08.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.296+0000 7f21cf400640 1 --2- 172.21.15.67:0/2704278840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 0x7f21d00b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.296+0000 7f21cf400640 1 -- 172.21.15.67:0/2704278840 >> v2:172.21.15.169:3300/0 conn(0x7f21d015e800 msgr2=0x7f21d00aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:08.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.296+0000 7f21cf400640 1 --2- 172.21.15.67:0/2704278840 >> v2:172.21.15.169:3300/0 conn(0x7f21d015e800 0x7f21d00aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.296+0000 7f21cf400640 1 -- 172.21.15.67:0/2704278840 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d00cfcf0 con 0x7f21d015ee50 2024-05-30T16:52:08.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.297+0000 7f21cf400640 1 --2- 172.21.15.67:0/2704278840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015ee50 0x7f21d015f2e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f21d00756c0 tx=0x7f21c802c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee95d3d1a2d36b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:08.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.297+0000 7f21cd600640 1 -- 172.21.15.67:0/2704278840 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8039070 con 0x7f21d015ee50 2024-05-30T16:52:08.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cd600640 1 -- 172.21.15.67:0/2704278840 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c8034410 con 0x7f21d015ee50 2024-05-30T16:52:08.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cd600640 1 -- 172.21.15.67:0/2704278840 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c80347e0 con 0x7f21d015ee50 2024-05-30T16:52:08.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cfe00640 1 -- 172.21.15.67:0/2704278840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015ee50 msgr2=0x7f21d015f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cfe00640 1 --2- 172.21.15.67:0/2704278840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015ee50 0x7f21d015f2e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f21d00756c0 tx=0x7f21c802c820 comp rx=0 tx=0).stop 2024-05-30T16:52:08.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cfe00640 1 -- 172.21.15.67:0/2704278840 shutdown_connections 2024-05-30T16:52:08.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cfe00640 1 --2- 172.21.15.67:0/2704278840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015ee50 0x7f21d015f2e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cfe00640 1 --2- 172.21.15.67:0/2704278840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 0x7f21d00b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cfe00640 1 --2- 172.21.15.67:0/2704278840 >> v2:172.21.15.169:3300/0 conn(0x7f21d015e800 0x7f21d00aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cfe00640 1 -- 172.21.15.67:0/2704278840 >> 172.21.15.67:0/2704278840 conn(0x7f21d0073230 msgr2=0x7f21d006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:08.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cfe00640 1 -- 172.21.15.67:0/2704278840 shutdown_connections 2024-05-30T16:52:08.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.298+0000 7f21cfe00640 1 -- 172.21.15.67:0/2704278840 wait complete. 2024-05-30T16:52:08.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.300+0000 7f21cfe00640 1 Processor -- start 2024-05-30T16:52:08.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.300+0000 7f21cfe00640 1 -- start start 2024-05-30T16:52:08.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.301+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 0x7f21d01424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.301+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015e800 0x7f21d01506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.301+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d015ee50 0x7f21d0150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.301+0000 7f21cfe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21d00d0590 con 0x7f21d015e800 2024-05-30T16:52:08.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.301+0000 7f21cfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21d00d02b0 con 0x7f21d015ee50 2024-05-30T16:52:08.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.301+0000 7f21cfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21d00b04c0 con 0x7f21d00b4de0 2024-05-30T16:52:08.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21ce000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015e800 0x7f21d01506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21ce000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015e800 0x7f21d01506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52144/0 (socket says 172.21.15.67:52144) 2024-05-30T16:52:08.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21ce000640 1 -- 172.21.15.67:0/1915023203 learned_addr learned my addr 172.21.15.67:0/1915023203 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21cea00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 0x7f21d01424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21cf400640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d015ee50 0x7f21d0150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21ce000640 1 -- 172.21.15.67:0/1915023203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 msgr2=0x7f21d01424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21ce000640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 0x7f21d01424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21ce000640 1 -- 172.21.15.67:0/1915023203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d015ee50 msgr2=0x7f21d0150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21ce000640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d015ee50 0x7f21d0150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21ce000640 1 -- 172.21.15.67:0/1915023203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d014ec60 con 0x7f21d015e800 2024-05-30T16:52:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.302+0000 7f21cf400640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d015ee50 0x7f21d0150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.303+0000 7f21ce000640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015e800 0x7f21d01506a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00d800 tx=0x7f21bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.303+0000 7f21c6a00640 1 -- 172.21.15.67:0/1915023203 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc018020 con 0x7f21d015e800 2024-05-30T16:52:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.303+0000 7f21c6a00640 1 -- 172.21.15.67:0/1915023203 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc0096a0 con 0x7f21d015e800 2024-05-30T16:52:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.303+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d014ef40 con 0x7f21d015e800 2024-05-30T16:52:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.303+0000 7f21c6a00640 1 -- 172.21.15.67:0/1915023203 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc009a70 con 0x7f21d015e800 2024-05-30T16:52:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.303+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d014f500 con 0x7f21d015e800 2024-05-30T16:52:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.304+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d00690f0 con 0x7f21d015e800 2024-05-30T16:52:08.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.305+0000 7f21c6a00640 1 -- 172.21.15.67:0/1915023203 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 91652+0+0 (secure 0 0 0) 0x7f21bc01b020 con 0x7f21d015e800 2024-05-30T16:52:08.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.307+0000 7f21c6a00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21b006cfb0 0x7f21b006f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.307+0000 7f21c6a00640 1 -- 172.21.15.67:0/1915023203 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f21bc08c970 con 0x7f21d015e800 2024-05-30T16:52:08.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.307+0000 7f21cea00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21b006cfb0 0x7f21b006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.308+0000 7f21cea00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21b006cfb0 0x7f21b006f470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:08.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.312+0000 7f21c6a00640 1 -- 172.21.15.67:0/1915023203 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21bc022030 con 0x7f21d015e800 2024-05-30T16:52:08.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.547+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone75831", "target": ["mon-mgr", ""]}) -- 0x7f21d00b1950 con 0x7f21b006cfb0 2024-05-30T16:52:08.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.579+0000 7f21c6a00640 1 -- 172.21.15.67:0/1915023203 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f21d00b1950 con 0x7f21b006cfb0 2024-05-30T16:52:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.581+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21b006cfb0 msgr2=0x7f21b006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.581+0000 7f21cfe00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21b006cfb0 0x7f21b006f470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0008040 comp rx=0 tx=0).stop 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.581+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015e800 msgr2=0x7f21d01506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.581+0000 7f21cfe00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015e800 0x7f21d01506a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00d800 tx=0x7f21bc00dcc0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.582+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 shutdown_connections 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.582+0000 7f21cfe00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21b006cfb0 0x7f21b006f470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.582+0000 7f21cfe00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d015ee50 0x7f21d0150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.582+0000 7f21cfe00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d015e800 0x7f21d01506a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.582+0000 7f21cfe00640 1 --2- 172.21.15.67:0/1915023203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21d00b4de0 0x7f21d01424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.582+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 >> 172.21.15.67:0/1915023203 conn(0x7f21d0073230 msgr2=0x7f21d00b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.582+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 shutdown_connections 2024-05-30T16:52:08.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.582+0000 7f21cfe00640 1 -- 172.21.15.67:0/1915023203 wait complete. 2024-05-30T16:52:08.605 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone75834 2024-05-30T16:52:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.793+0000 7f4c07000640 1 Processor -- start 2024-05-30T16:52:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.793+0000 7f4c07000640 1 -- start start 2024-05-30T16:52:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.795+0000 7f4c07000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c00155a60 0x7f4c0015f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.795+0000 7f4c07000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c000b4240 0x7f4c0006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.795+0000 7f4c07000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 0x7f4c0006be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.795+0000 7f4c07000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4c000b2c10 con 0x7f4c00155a60 2024-05-30T16:52:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.795+0000 7f4c07000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4c000b2650 con 0x7f4c000b4240 2024-05-30T16:52:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.795+0000 7f4c07000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c000b2930 con 0x7f4c000b46d0 2024-05-30T16:52:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.795+0000 7f4bffe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c00155a60 0x7f4c0015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.795+0000 7f4bffe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c00155a60 0x7f4c0015f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52156/0 (socket says 172.21.15.67:52156) 2024-05-30T16:52:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.795+0000 7f4bffe00640 1 -- 172.21.15.67:0/1812999822 learned_addr learned my addr 172.21.15.67:0/1812999822 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.796+0000 7f4bff400640 1 --2- 172.21.15.67:0/1812999822 >> v2:172.21.15.169:3300/0 conn(0x7f4c000b4240 0x7f4c0006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.796+0000 7f4c05000640 1 --2- 172.21.15.67:0/1812999822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 0x7f4c0006be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.796+0000 7f4bffe00640 1 -- 172.21.15.67:0/1812999822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 msgr2=0x7f4c0006be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.796+0000 7f4bffe00640 1 --2- 172.21.15.67:0/1812999822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 0x7f4c0006be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.796+0000 7f4bffe00640 1 -- 172.21.15.67:0/1812999822 >> v2:172.21.15.169:3300/0 conn(0x7f4c000b4240 msgr2=0x7f4c0006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.796+0000 7f4bffe00640 1 --2- 172.21.15.67:0/1812999822 >> v2:172.21.15.169:3300/0 conn(0x7f4c000b4240 0x7f4c0006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.796+0000 7f4bffe00640 1 -- 172.21.15.67:0/1812999822 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c000b5ef0 con 0x7f4c00155a60 2024-05-30T16:52:08.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.796+0000 7f4c05000640 1 --2- 172.21.15.67:0/1812999822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 0x7f4c0006be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:08.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.796+0000 7f4bff400640 1 --2- 172.21.15.67:0/1812999822 >> v2:172.21.15.169:3300/0 conn(0x7f4c000b4240 0x7f4c0006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.797+0000 7f4bffe00640 1 --2- 172.21.15.67:0/1812999822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 0x7f4c0015f970 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4c000b74e0 tx=0x7f4bf002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249c6a389f9246af server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.797+0000 7f4bfea00640 1 -- 172.21.15.67:0/1812999822 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0037070 con 0x7f4c00155a60 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.797+0000 7f4bfea00640 1 -- 172.21.15.67:0/1812999822 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf0032440 con 0x7f4c00155a60 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.797+0000 7f4bfea00640 1 -- 172.21.15.67:0/1812999822 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0032810 con 0x7f4c00155a60 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.798+0000 7f4c07000640 1 -- 172.21.15.67:0/1812999822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 msgr2=0x7f4c0015f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.798+0000 7f4c07000640 1 --2- 172.21.15.67:0/1812999822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 0x7f4c0015f970 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4c000b74e0 tx=0x7f4bf002a1d0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.798+0000 7f4c07000640 1 -- 172.21.15.67:0/1812999822 shutdown_connections 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.798+0000 7f4c07000640 1 --2- 172.21.15.67:0/1812999822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 0x7f4c0006be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.798+0000 7f4c07000640 1 --2- 172.21.15.67:0/1812999822 >> v2:172.21.15.169:3300/0 conn(0x7f4c000b4240 0x7f4c0006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.798+0000 7f4c07000640 1 --2- 172.21.15.67:0/1812999822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 0x7f4c0015f970 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.798+0000 7f4c07000640 1 -- 172.21.15.67:0/1812999822 >> 172.21.15.67:0/1812999822 conn(0x7f4c000b6760 msgr2=0x7f4c000b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:08.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.798+0000 7f4c07000640 1 -- 172.21.15.67:0/1812999822 shutdown_connections 2024-05-30T16:52:08.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.798+0000 7f4c07000640 1 -- 172.21.15.67:0/1812999822 wait complete. 2024-05-30T16:52:08.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.800+0000 7f4c07000640 1 Processor -- start 2024-05-30T16:52:08.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.800+0000 7f4c07000640 1 -- start start 2024-05-30T16:52:08.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.801+0000 7f4c07000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c000b4240 0x7f4c00161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.801+0000 7f4c07000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 0x7f4c001e8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.801+0000 7f4c07000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 0x7f4c001f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.801+0000 7f4c07000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4c00162540 con 0x7f4c00155a60 2024-05-30T16:52:08.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.801+0000 7f4c07000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c001623c0 con 0x7f4c000b4240 2024-05-30T16:52:08.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.801+0000 7f4c07000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c001626c0 con 0x7f4c000b46d0 2024-05-30T16:52:08.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.801+0000 7f4c05000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 0x7f4c001f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.801+0000 7f4c05000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 0x7f4c001f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52168/0 (socket says 172.21.15.67:52168) 2024-05-30T16:52:08.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.801+0000 7f4c05000640 1 -- 172.21.15.67:0/2283576954 learned_addr learned my addr 172.21.15.67:0/2283576954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:08.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.802+0000 7f4bffe00640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c000b4240 0x7f4c00161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.802+0000 7f4bff400640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 0x7f4c001e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.802+0000 7f4c05000640 1 -- 172.21.15.67:0/2283576954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 msgr2=0x7f4c001e8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.802+0000 7f4c05000640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 0x7f4c001e8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.802+0000 7f4c05000640 1 -- 172.21.15.67:0/2283576954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c000b4240 msgr2=0x7f4c00161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.802+0000 7f4c05000640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c000b4240 0x7f4c00161ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.802+0000 7f4c05000640 1 -- 172.21.15.67:0/2283576954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c001564b0 con 0x7f4c00155a60 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.802+0000 7f4c05000640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 0x7f4c001f7c50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800cef0 tx=0x7f4bf800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.803+0000 7f4bf7e00640 1 -- 172.21.15.67:0/2283576954 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf801a070 con 0x7f4c00155a60 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.803+0000 7f4bf7e00640 1 -- 172.21.15.67:0/2283576954 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf800d860 con 0x7f4c00155a60 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.803+0000 7f4bf7e00640 1 -- 172.21.15.67:0/2283576954 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800dc50 con 0x7f4c00155a60 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.803+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c001e8790 con 0x7f4c00155a60 2024-05-30T16:52:08.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.803+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c001f87a0 con 0x7f4c00155a60 2024-05-30T16:52:08.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.804+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0015ec60 con 0x7f4c00155a60 2024-05-30T16:52:08.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.805+0000 7f4bf7e00640 1 -- 172.21.15.67:0/2283576954 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f4bf801d070 con 0x7f4c00155a60 2024-05-30T16:52:08.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.806+0000 7f4bf7e00640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be006d100 0x7f4be006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:08.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.806+0000 7f4bf7e00640 1 -- 172.21.15.67:0/2283576954 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f4bf808d5a0 con 0x7f4c00155a60 2024-05-30T16:52:08.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.807+0000 7f4bffe00640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be006d100 0x7f4be006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:08.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.807+0000 7f4bffe00640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be006d100 0x7f4be006f5c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0034000 tx=0x7f4bf002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:08.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:08.810+0000 7f4bf7e00640 1 -- 172.21.15.67:0/2283576954 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c0015ec60 con 0x7f4c00155a60 2024-05-30T16:52:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.154+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone75834", "target": ["mon-mgr", ""]}) -- 0x7f4c000689e0 con 0x7f4be006d100 2024-05-30T16:52:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.188+0000 7f4bf7e00640 1 -- 172.21.15.67:0/2283576954 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c000689e0 con 0x7f4be006d100 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be006d100 msgr2=0x7f4be006f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be006d100 0x7f4be006f5c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0034000 tx=0x7f4bf002e000 comp rx=0 tx=0).stop 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 msgr2=0x7f4c001f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 0x7f4c001f7c50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800cef0 tx=0x7f4bf800a710 comp rx=0 tx=0).stop 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 shutdown_connections 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be006d100 0x7f4be006f5c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c00155a60 0x7f4c001f7c50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c000b46d0 0x7f4c001e8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 --2- 172.21.15.67:0/2283576954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c000b4240 0x7f4c00161ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 >> 172.21.15.67:0/2283576954 conn(0x7f4c000b6760 msgr2=0x7f4c0006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:09.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 shutdown_connections 2024-05-30T16:52:09.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.191+0000 7f4c07000640 1 -- 172.21.15.67:0/2283576954 wait complete. 2024-05-30T16:52:09.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone75832 2024-05-30T16:52:09.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.382+0000 7f64bd200640 1 Processor -- start 2024-05-30T16:52:09.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.382+0000 7f64bd200640 1 -- start start 2024-05-30T16:52:09.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.384+0000 7f64bd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b8140500 0x7f64b80ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:09.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.384+0000 7f64bd200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64b80d0020 con 0x7f64b806edf0 2024-05-30T16:52:09.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.384+0000 7f64bd200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64b806dd50 con 0x7f64b80cf2f0 2024-05-30T16:52:09.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.384+0000 7f64bd200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64b806e030 con 0x7f64b8140500 2024-05-30T16:52:09.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.384+0000 7f64ad400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f64b806edf0 0x7f64b813fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45176/0 (socket says 172.21.15.67:45176) 2024-05-30T16:52:09.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.384+0000 7f64ad400640 1 -- 172.21.15.67:0/220496350 learned_addr learned my addr 172.21.15.67:0/220496350 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:09.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.384+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2210490153 0 0) 0x7f64b80d0020 con 0x7f64b806edf0 2024-05-30T16:52:09.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.384+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6498003520 con 0x7f64b806edf0 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.384+0000 7f64b6800640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b8140500 0x7f64b80ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 718455599 0 0) 0x7f6498003520 con 0x7f64b806edf0 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b8140500 msgr2=0x7f64b80ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b8140500 0x7f64b80ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 >> v1:172.21.15.169:6789/0 conn(0x7f64b80cf2f0 legacy=0x7f64b80cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b806e310 con 0x7f64b806edf0 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f64a0001670 con 0x7f64b806edf0 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 >> v1:172.21.15.67:6789/0 conn(0x7f64b806edf0 legacy=0x7f64b813fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980039e0 0x7f6498003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498004800 0x7f6498004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64980051c0 0x7f649800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64b806e030 con 0x7f64980051c0 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f649800bfe0 con 0x7f6498004800 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f649800c1e0 con 0x7f64980039e0 2024-05-30T16:52:09.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.385+0000 7f64b6800640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64980051c0 0x7f649800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:09.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.386+0000 7f64ad400640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498004800 0x7f6498004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:09.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.386+0000 7f64b6800640 1 -- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980039e0 msgr2=0x7f6498003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:09.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.386+0000 7f64b5e00640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980039e0 0x7f6498003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:09.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.386+0000 7f64b6800640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980039e0 0x7f6498003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.386+0000 7f64b6800640 1 -- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498004800 msgr2=0x7f6498004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:09.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.386+0000 7f64b6800640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498004800 0x7f6498004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.386+0000 7f64b6800640 1 -- 172.21.15.67:0/220496350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6498003520 con 0x7f64980051c0 2024-05-30T16:52:09.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.386+0000 7f64b5e00640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64980039e0 0x7f6498003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:09.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.387+0000 7f64b6800640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64980051c0 0x7f649800b9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f64b8142f40 tx=0x7f6498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf402ef060c9a3e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:09.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.387+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0033070 con 0x7f64980051c0 2024-05-30T16:52:09.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.387+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b002f630 con 0x7f64980051c0 2024-05-30T16:52:09.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.387+0000 7f64b5400640 1 -- 172.21.15.67:0/220496350 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0009be0 con 0x7f64980051c0 2024-05-30T16:52:09.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.388+0000 7f64bd200640 1 -- 172.21.15.67:0/220496350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64980051c0 msgr2=0x7f649800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.388+0000 7f64bd200640 1 --2- 172.21.15.67:0/220496350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64980051c0 0x7f649800b9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f64b8142f40 tx=0x7f6498005830 comp rx=0 tx=0).stop 2024-05-30T16:52:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.388+0000 7f64b6800640 1 -- 172.21.15.67:0/220496350 reap_dead start 2024-05-30T16:52:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.388+0000 7f64bd200640 1 -- 172.21.15.67:0/220496350 shutdown_connections 2024-05-30T16:52:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.388+0000 7f64bd200640 1 -- 172.21.15.67:0/220496350 >> 172.21.15.67:0/220496350 conn(0x7f64b80b1e50 msgr2=0x7f64b80b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.388+0000 7f64bd200640 1 -- 172.21.15.67:0/220496350 shutdown_connections 2024-05-30T16:52:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.388+0000 7f64bd200640 1 -- 172.21.15.67:0/220496350 wait complete. 2024-05-30T16:52:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.390+0000 7f64bd200640 1 Processor -- start 2024-05-30T16:52:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.390+0000 7f64bd200640 1 -- start start 2024-05-30T16:52:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.391+0000 7f64bd200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b81f78f0 0x7f64b80cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.391+0000 7f64bd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b80ca650 0x7f64b80b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.391+0000 7f64bd200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b80b3bf0 0x7f64b8072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.391+0000 7f64bd200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8146020 con 0x7f64b81f78f0 2024-05-30T16:52:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.391+0000 7f64bd200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64b806e8d0 con 0x7f64b80b3bf0 2024-05-30T16:52:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.391+0000 7f64bd200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64b81461a0 con 0x7f64b80ca650 2024-05-30T16:52:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.391+0000 7f64b5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b81f78f0 0x7f64b80cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.391+0000 7f64b5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b81f78f0 0x7f64b80cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52180/0 (socket says 172.21.15.67:52180) 2024-05-30T16:52:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.391+0000 7f64b5e00640 1 -- 172.21.15.67:0/109407602 learned_addr learned my addr 172.21.15.67:0/109407602 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:09.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64b6800640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b80b3bf0 0x7f64b8072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64ad400640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b80ca650 0x7f64b80b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64b5e00640 1 -- 172.21.15.67:0/109407602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b80ca650 msgr2=0x7f64b80b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64b5e00640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b80ca650 0x7f64b80b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64b5e00640 1 -- 172.21.15.67:0/109407602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b80b3bf0 msgr2=0x7f64b8072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64b5e00640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b80b3bf0 0x7f64b8072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64b5e00640 1 -- 172.21.15.67:0/109407602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b815ce80 con 0x7f64b81f78f0 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64b6800640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b80b3bf0 0x7f64b8072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64b5e00640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b81f78f0 0x7f64b80cb090 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f64a800d7d0 tx=0x7f64a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64aea00640 1 -- 172.21.15.67:0/109407602 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8005070 con 0x7f64b81f78f0 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64aea00640 1 -- 172.21.15.67:0/109407602 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a8005230 con 0x7f64b81f78f0 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64aea00640 1 -- 172.21.15.67:0/109407602 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8007030 con 0x7f64b81f78f0 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.392+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b815d160 con 0x7f64b81f78f0 2024-05-30T16:52:09.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.393+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b815d720 con 0x7f64b81f78f0 2024-05-30T16:52:09.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.394+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b80689e0 con 0x7f64b81f78f0 2024-05-30T16:52:09.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.395+0000 7f64aea00640 1 -- 172.21.15.67:0/109407602 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f64a801a020 con 0x7f64b81f78f0 2024-05-30T16:52:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.395+0000 7f64aea00640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649406d100 0x7f649406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.396+0000 7f64aea00640 1 -- 172.21.15.67:0/109407602 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f64a808c6c0 con 0x7f64b81f78f0 2024-05-30T16:52:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.396+0000 7f64ad400640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649406d100 0x7f649406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:09.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.396+0000 7f64ad400640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649406d100 0x7f649406f5c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f64a0001050 tx=0x7f64a0001190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:09.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.404+0000 7f64aea00640 1 -- 172.21.15.67:0/109407602 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64a8059ad0 con 0x7f64b81f78f0 2024-05-30T16:52:09.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.631+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone75832", "target": ["mon-mgr", ""]}) -- 0x7f64b8073f30 con 0x7f649406d100 2024-05-30T16:52:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.662+0000 7f64aea00640 1 -- 172.21.15.67:0/109407602 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f64b8073f30 con 0x7f649406d100 2024-05-30T16:52:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.665+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649406d100 msgr2=0x7f649406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.665+0000 7f64bd200640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649406d100 0x7f649406f5c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f64a0001050 tx=0x7f64a0001190 comp rx=0 tx=0).stop 2024-05-30T16:52:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.665+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b81f78f0 msgr2=0x7f64b80cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.665+0000 7f64bd200640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b81f78f0 0x7f64b80cb090 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f64a800d7d0 tx=0x7f64a800dc90 comp rx=0 tx=0).stop 2024-05-30T16:52:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.665+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 shutdown_connections 2024-05-30T16:52:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.665+0000 7f64bd200640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649406d100 0x7f649406f5c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.666+0000 7f64bd200640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b80b3bf0 0x7f64b8072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.666+0000 7f64bd200640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b80ca650 0x7f64b80b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.666+0000 7f64bd200640 1 --2- 172.21.15.67:0/109407602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b81f78f0 0x7f64b80cb090 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.666+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 >> 172.21.15.67:0/109407602 conn(0x7f64b80b1e50 msgr2=0x7f64b80cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.666+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 shutdown_connections 2024-05-30T16:52:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09.666+0000 7f64bd200640 1 -- 172.21.15.67:0/109407602 wait complete. 2024-05-30T16:52:09.689 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-30T16:52:09.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-30T16:52:09.789 INFO:teuthology.orchestra.run.smithi067.stdout:2 2024-05-30T16:52:09.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:52:09.798 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T16:52:09.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:52:09.882 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:52:10.002 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T16:52:10.002 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:52:10.002 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:52:10.002 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:10.053 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:52:10.053 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:10.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:52:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.382+0000 7f6503200640 1 Processor -- start 2024-05-30T16:52:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.382+0000 7f6503200640 1 -- start start 2024-05-30T16:52:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.384+0000 7f6503200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0b37a0 0x7f64fc0d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.384+0000 7f6503200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f64fc06d950 0x7f64fc09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.384+0000 7f6503200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc0d0ef0 con 0x7f64fc09af70 2024-05-30T16:52:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.384+0000 7f6503200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f64fc09b3e0 con 0x7f64fc06d950 2024-05-30T16:52:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.384+0000 7f6503200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc0d0cc0 con 0x7f64fc0b37a0 2024-05-30T16:52:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.384+0000 7f6501200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f64fc09af70 0x7f64fc162610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45182/0 (socket says 172.21.15.67:45182) 2024-05-30T16:52:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.384+0000 7f6501200640 1 -- 172.21.15.67:0/543719635 learned_addr learned my addr 172.21.15.67:0/543719635 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fbe00640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0b37a0 0x7f64fc0d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64f3400640 1 --2- 172.21.15.67:0/543719635 >> v2:172.21.15.169:3300/0 conn(0x7f64fc06d950 0x7f64fc09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344769586 0 0) 0x7f64fc0d0ef0 con 0x7f64fc09af70 2024-05-30T16:52:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e0003520 con 0x7f64fc09af70 2024-05-30T16:52:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2735436419 0 0) 0x7f64e0003520 con 0x7f64fc09af70 2024-05-30T16:52:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0b37a0 msgr2=0x7f64fc0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0b37a0 0x7f64fc0d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 >> v2:172.21.15.169:3300/0 conn(0x7f64fc06d950 msgr2=0x7f64fc09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 --2- 172.21.15.67:0/543719635 >> v2:172.21.15.169:3300/0 conn(0x7f64fc06d950 0x7f64fc09aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0d11d0 con 0x7f64fc09af70 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f64f40019f0 con 0x7f64fc09af70 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 >> v1:172.21.15.67:6789/0 conn(0x7f64fc09af70 legacy=0x7f64fc162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e00039b0 0x7f64e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64e0004800 0x7f64e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.385+0000 7f64fb400640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0b37a0 0x7f64e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc09b3e0 con 0x7f64e0004800 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64e000b9c0 con 0x7f64e00039b0 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64e000bb40 con 0x7f64fc0b37a0 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64f3400640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64e0004800 0x7f64e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64fbe00640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e00039b0 0x7f64e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64f3400640 1 -- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0b37a0 msgr2=0x7f64e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64f3400640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc0b37a0 0x7f64e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64f3400640 1 -- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e00039b0 msgr2=0x7f64e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64f3400640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e00039b0 0x7f64e0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.386+0000 7f64f3400640 1 -- 172.21.15.67:0/543719635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e0003520 con 0x7f64e0004800 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.387+0000 7f64f3400640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64e0004800 0x7f64e0004c80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009ac0 tx=0x7f64e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745f83c5840be734 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.387+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4036070 con 0x7f64e0004800 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.387+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e4031400 con 0x7f64e0004800 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.387+0000 7f64fb400640 1 -- 172.21.15.67:0/543719635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4009690 con 0x7f64e0004800 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.388+0000 7f6503200640 1 -- 172.21.15.67:0/543719635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64e0004800 msgr2=0x7f64e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.388+0000 7f6503200640 1 --2- 172.21.15.67:0/543719635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64e0004800 0x7f64e0004c80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009ac0 tx=0x7f64e0005830 comp rx=0 tx=0).stop 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.388+0000 7f6501200640 1 -- 172.21.15.67:0/543719635 reap_dead start 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.388+0000 7f6503200640 1 -- 172.21.15.67:0/543719635 shutdown_connections 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.388+0000 7f6503200640 1 -- 172.21.15.67:0/543719635 >> 172.21.15.67:0/543719635 conn(0x7f64fc0b3f40 msgr2=0x7f64fc0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.388+0000 7f6503200640 1 -- 172.21.15.67:0/543719635 shutdown_connections 2024-05-30T16:52:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.388+0000 7f6503200640 1 -- 172.21.15.67:0/543719635 wait complete. 2024-05-30T16:52:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.390+0000 7f6503200640 1 Processor -- start 2024-05-30T16:52:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.390+0000 7f6503200640 1 -- start start 2024-05-30T16:52:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.391+0000 7f6503200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e0004800 0x7f64fc0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.391+0000 7f6503200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc1e4050 0x7f64fc0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.391+0000 7f6503200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64fc1f53e0 0x7f64fc1e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.391+0000 7f6503200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc0d1a70 con 0x7f64fc1f53e0 2024-05-30T16:52:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.391+0000 7f6503200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc0d1790 con 0x7f64e0004800 2024-05-30T16:52:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.391+0000 7f6503200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc06dd90 con 0x7f64fc1e4050 2024-05-30T16:52:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.391+0000 7f6501200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64fc1f53e0 0x7f64fc1e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f6501200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64fc1f53e0 0x7f64fc1e69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52202/0 (socket says 172.21.15.67:52202) 2024-05-30T16:52:10.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f6501200640 1 -- 172.21.15.67:0/3026805950 learned_addr learned my addr 172.21.15.67:0/3026805950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f64fbe00640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e0004800 0x7f64fc0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f6501200640 1 -- 172.21.15.67:0/3026805950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc1e4050 msgr2=0x7f64fc0c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f6501200640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc1e4050 0x7f64fc0c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f6501200640 1 -- 172.21.15.67:0/3026805950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e0004800 msgr2=0x7f64fc0c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f6501200640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e0004800 0x7f64fc0c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f6501200640 1 -- 172.21.15.67:0/3026805950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0c64a0 con 0x7f64fc1f53e0 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f64fbe00640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e0004800 0x7f64fc0c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f6501200640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64fc1f53e0 0x7f64fc1e69e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0725a0 tx=0x7f64f400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.392+0000 7f64f8c00640 1 -- 172.21.15.67:0/3026805950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f400a610 con 0x7f64fc1f53e0 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.393+0000 7f64f8c00640 1 -- 172.21.15.67:0/3026805950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f400a7b0 con 0x7f64fc1f53e0 2024-05-30T16:52:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.393+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc0c6770 con 0x7f64fc1f53e0 2024-05-30T16:52:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.393+0000 7f64f8c00640 1 -- 172.21.15.67:0/3026805950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f400aba0 con 0x7f64fc1f53e0 2024-05-30T16:52:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.393+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc0c6d30 con 0x7f64fc1f53e0 2024-05-30T16:52:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.394+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc074bb0 con 0x7f64fc1f53e0 2024-05-30T16:52:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.395+0000 7f64f8c00640 1 -- 172.21.15.67:0/3026805950 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f64f4011070 con 0x7f64fc1f53e0 2024-05-30T16:52:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.396+0000 7f64f8c00640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f64d806d100 0x7f64d806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.396+0000 7f64f8c00640 1 -- 172.21.15.67:0/3026805950 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f64f408a0a0 con 0x7f64fc1f53e0 2024-05-30T16:52:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.396+0000 7f64fbe00640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f64d806d100 0x7f64d806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:10.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.397+0000 7f64fbe00640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f64d806d100 0x7f64d806f5c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009910 tx=0x7f64ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.400+0000 7f64f8c00640 1 -- 172.21.15.67:0/3026805950 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64fc074bb0 con 0x7f64fc1f53e0 2024-05-30T16:52:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.607+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f64fc0689e0 con 0x7f64fc1f53e0 2024-05-30T16:52:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.608+0000 7f64f8c00640 1 -- 172.21.15.67:0/3026805950 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) ==== 80+0+0 (secure 0 0 0) 0x7f64f401a060 con 0x7f64fc1f53e0 2024-05-30T16:52:10.609 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:52:10.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.610+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f64d806d100 msgr2=0x7f64d806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.610+0000 7f6503200640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f64d806d100 0x7f64d806f5c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009910 tx=0x7f64ec008040 comp rx=0 tx=0).stop 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.610+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64fc1f53e0 msgr2=0x7f64fc1e69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.610+0000 7f6503200640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64fc1f53e0 0x7f64fc1e69e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0725a0 tx=0x7f64f400cf00 comp rx=0 tx=0).stop 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.610+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 shutdown_connections 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.610+0000 7f6503200640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f64d806d100 0x7f64d806f5c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.611+0000 7f6503200640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64fc1f53e0 0x7f64fc1e69e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.611+0000 7f6503200640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64fc1e4050 0x7f64fc0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.611+0000 7f6503200640 1 --2- 172.21.15.67:0/3026805950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64e0004800 0x7f64fc0c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.611+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 >> 172.21.15.67:0/3026805950 conn(0x7f64fc0b3f40 msgr2=0x7f64fc1e5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.611+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 shutdown_connections 2024-05-30T16:52:10.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:10.611+0000 7f6503200640 1 -- 172.21.15.67:0/3026805950 wait complete. 2024-05-30T16:52:10.629 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T16:52:10.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:10.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-30T16:52:10.706 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:52:10.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:10.706 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-30T16:52:10.711 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[51789]: fuse finished with error 0 and tester_r 0 2024-05-30T16:52:10.867 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:52:10.867 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:52:10.867 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:10.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:52:10.914 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:52:10.914 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:52:10.914 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:10.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:52:10.976 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:52:10.976 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:52:10.976 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:11.036 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:52:11.036 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:52:11.218 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:52:11.225 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:52:11.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.315+0000 7f92fc000640 1 Processor -- start 2024-05-30T16:52:11.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.315+0000 7f92fc000640 1 -- start start 2024-05-30T16:52:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.317+0000 7f92fc000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f92f415ed00 0x7f92f40b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.317+0000 7f92fc000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f92f40d0350 0x7f92f40d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.317+0000 7f92fc000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f40b5070 con 0x7f92f40d46c0 2024-05-30T16:52:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.317+0000 7f92fc000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f92f40d5060 con 0x7f92f415ed00 2024-05-30T16:52:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.317+0000 7f92fc000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f92f40b41a0 con 0x7f92f40d0350 2024-05-30T16:52:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.318+0000 7f92f9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f92f415ed00 0x7f92f40b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.318+0000 7f92f9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f92f415ed00 0x7f92f40b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38322/0 (socket says 172.21.15.67:38322) 2024-05-30T16:52:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.318+0000 7f92f9600640 1 -- 172.21.15.67:0/2990969048 learned_addr learned my addr 172.21.15.67:0/2990969048 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.318+0000 7f92f9600640 1 -- 172.21.15.67:0/2990969048 >> v2:172.21.15.169:3300/0 conn(0x7f92f40d0350 msgr2=0x7f92f40d4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.318+0000 7f92f9600640 1 --2- 172.21.15.67:0/2990969048 >> v2:172.21.15.169:3300/0 conn(0x7f92f40d0350 0x7f92f40d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.318+0000 7f92f9600640 1 -- 172.21.15.67:0/2990969048 >> v1:172.21.15.67:6789/0 conn(0x7f92f40d46c0 legacy=0x7f92f40d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.319+0000 7f92f9600640 1 -- 172.21.15.67:0/2990969048 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f40d33b0 con 0x7f92f415ed00 2024-05-30T16:52:11.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.321+0000 7f92f9600640 1 --2- 172.21.15.67:0/2990969048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f415ed00 0x7f92f40b3c60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f92f40d5380 tx=0x7f92e802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5e7bce76ae3acf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.326+0000 7f92f3e00640 1 -- 172.21.15.67:0/2990969048 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8037070 con 0x7f92f415ed00 2024-05-30T16:52:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.326+0000 7f92f3e00640 1 -- 172.21.15.67:0/2990969048 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e8032440 con 0x7f92f415ed00 2024-05-30T16:52:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.326+0000 7f92f3e00640 1 -- 172.21.15.67:0/2990969048 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8032830 con 0x7f92f415ed00 2024-05-30T16:52:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.328+0000 7f92fc000640 1 -- 172.21.15.67:0/2990969048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f415ed00 msgr2=0x7f92f40b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.328+0000 7f92fc000640 1 --2- 172.21.15.67:0/2990969048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f415ed00 0x7f92f40b3c60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f92f40d5380 tx=0x7f92e802a290 comp rx=0 tx=0).stop 2024-05-30T16:52:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.328+0000 7f92fc000640 1 -- 172.21.15.67:0/2990969048 shutdown_connections 2024-05-30T16:52:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.328+0000 7f92fc000640 1 --2- 172.21.15.67:0/2990969048 >> v2:172.21.15.169:3300/0 conn(0x7f92f40d0350 0x7f92f40d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.328+0000 7f92fc000640 1 --2- 172.21.15.67:0/2990969048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f415ed00 0x7f92f40b3c60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.328+0000 7f92fc000640 1 -- 172.21.15.67:0/2990969048 >> 172.21.15.67:0/2990969048 conn(0x7f92f409da70 msgr2=0x7f92f408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:11.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.329+0000 7f92fc000640 1 -- 172.21.15.67:0/2990969048 shutdown_connections 2024-05-30T16:52:11.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.329+0000 7f92fc000640 1 -- 172.21.15.67:0/2990969048 wait complete. 2024-05-30T16:52:11.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.331+0000 7f92fc000640 1 Processor -- start 2024-05-30T16:52:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.332+0000 7f92fc000640 1 -- start start 2024-05-30T16:52:11.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.334+0000 7f92fc000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f40d0350 0x7f92f415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.335+0000 7f92fc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f40d46c0 0x7f92f40cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.335+0000 7f92fc000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f415ed00 0x7f92f42048f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.335+0000 7f92fc000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f92f40b5610 con 0x7f92f415ed00 2024-05-30T16:52:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.335+0000 7f92fc000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f92f40b5330 con 0x7f92f40d0350 2024-05-30T16:52:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.335+0000 7f92fc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92f40d3040 con 0x7f92f40d46c0 2024-05-30T16:52:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.336+0000 7f92f9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f40d0350 0x7f92f415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.336+0000 7f92f9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f40d0350 0x7f92f415b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:50002/0 (socket says 172.21.15.67:50002) 2024-05-30T16:52:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.336+0000 7f92f9600640 1 -- 172.21.15.67:0/3555760206 learned_addr learned my addr 172.21.15.67:0/3555760206 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.336+0000 7f92f8c00640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f40d46c0 0x7f92f40cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.336+0000 7f92fa000640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f415ed00 0x7f92f42048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.337+0000 7f92f8c00640 1 -- 172.21.15.67:0/3555760206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f40d0350 msgr2=0x7f92f415b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.337+0000 7f92f8c00640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f40d0350 0x7f92f415b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.337+0000 7f92f8c00640 1 -- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f415ed00 msgr2=0x7f92f42048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.337+0000 7f92f8c00640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f415ed00 0x7f92f42048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.337+0000 7f92f8c00640 1 -- 172.21.15.67:0/3555760206 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f40b8410 con 0x7f92f40d46c0 2024-05-30T16:52:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.337+0000 7f92f9600640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f40d0350 0x7f92f415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:52:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.337+0000 7f92f8c00640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f40d46c0 0x7f92f40cb800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92e000c370 tx=0x7f92e000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.338+0000 7f92f1600640 1 -- 172.21.15.67:0/3555760206 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e00052d0 con 0x7f92f40d46c0 2024-05-30T16:52:11.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.338+0000 7f92f1600640 1 -- 172.21.15.67:0/3555760206 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e0005490 con 0x7f92f40d46c0 2024-05-30T16:52:11.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.338+0000 7f92f1600640 1 -- 172.21.15.67:0/3555760206 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0007260 con 0x7f92f40d46c0 2024-05-30T16:52:11.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.338+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f40b86f0 con 0x7f92f40d46c0 2024-05-30T16:52:11.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.338+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f415be70 con 0x7f92f40d46c0 2024-05-30T16:52:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.339+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f40689e0 con 0x7f92f40d46c0 2024-05-30T16:52:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.344+0000 7f92f1600640 1 -- 172.21.15.67:0/3555760206 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f92e0007400 con 0x7f92f40d46c0 2024-05-30T16:52:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.345+0000 7f92f1600640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f92d406d100 0x7f92d406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.345+0000 7f92f1600640 1 -- 172.21.15.67:0/3555760206 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f92e008c290 con 0x7f92f40d46c0 2024-05-30T16:52:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.346+0000 7f92f9600640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f92d406d100 0x7f92d406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:11.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.347+0000 7f92f9600640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f92d406d100 0x7f92d406f5c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f92e80325c0 tx=0x7f92e802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:11.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.353+0000 7f92f1600640 1 -- 172.21.15.67:0/3555760206 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92e0059050 con 0x7f92f40d46c0 2024-05-30T16:52:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.750+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92f406dcb0 con 0x7f92f40d46c0 2024-05-30T16:52:11.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.751+0000 7f92f1600640 1 -- 172.21.15.67:0/3555760206 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7f92e005d350 con 0x7f92f40d46c0 2024-05-30T16:52:11.755 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:11.755 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2139720967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2139720967},{"type":"v1","addr":"172.21.15.169:6834","nonce":2139720967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4981,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2317664228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2317664228},{"type":"v1","addr":"172.21.15.169:6835","nonce":2317664228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/350920637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":350920637},{"type":"v1","addr":"172.21.15.67:6835","nonce":350920637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:48:30.019937+0000","modified":"2024-05-30T16:48:33.071363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/3595714006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3595714006},{"type":"v1","addr":"172.21.15.67:6837","nonce":3595714006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5361,"qdb_cluster":[5361]},"id":3}]} 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f92d406d100 msgr2=0x7f92d406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f92d406d100 0x7f92d406f5c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f92e80325c0 tx=0x7f92e802e000 comp rx=0 tx=0).stop 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f40d46c0 msgr2=0x7f92f40cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f40d46c0 0x7f92f40cb800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92e000c370 tx=0x7f92e000c830 comp rx=0 tx=0).stop 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 shutdown_connections 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f92d406d100 0x7f92d406f5c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f415ed00 0x7f92f42048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92f40d46c0 0x7f92f40cb800 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 --2- 172.21.15.67:0/3555760206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92f40d0350 0x7f92f415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:11.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.755+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 >> 172.21.15.67:0/3555760206 conn(0x7f92f409da70 msgr2=0x7f92f40ca250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:11.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.756+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 shutdown_connections 2024-05-30T16:52:11.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:11.756+0000 7f92fc000640 1 -- 172.21.15.67:0/3555760206 wait complete. 2024-05-30T16:52:11.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:52:12.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.055+0000 7f3c17c00640 1 Processor -- start 2024-05-30T16:52:12.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.055+0000 7f3c17c00640 1 -- start start 2024-05-30T16:52:12.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.056+0000 7f3c17c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c1006f6e0 0x7f3c100cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.056+0000 7f3c17c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3c10162e70 0x7f3c100ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c17c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c100b4350 con 0x7f3c100d04a0 2024-05-30T16:52:12.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c17c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f3c100b3ec0 con 0x7f3c1006f6e0 2024-05-30T16:52:12.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c17c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3c100b32d0 con 0x7f3c10162e70 2024-05-30T16:52:12.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c0fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c1006f6e0 0x7f3c100cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c0fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c1006f6e0 0x7f3c100cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38332/0 (socket says 172.21.15.67:38332) 2024-05-30T16:52:12.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c0fe00640 1 -- 172.21.15.67:0/2215559124 learned_addr learned my addr 172.21.15.67:0/2215559124 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:12.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c0fe00640 1 -- 172.21.15.67:0/2215559124 >> v2:172.21.15.169:3300/0 conn(0x7f3c10162e70 msgr2=0x7f3c100ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:12.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c0fe00640 1 --2- 172.21.15.67:0/2215559124 >> v2:172.21.15.169:3300/0 conn(0x7f3c10162e70 0x7f3c100ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c0fe00640 1 -- 172.21.15.67:0/2215559124 >> v1:172.21.15.169:6789/0 conn(0x7f3c100d04a0 legacy=0x7f3c100d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:12.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.057+0000 7f3c0fe00640 1 -- 172.21.15.67:0/2215559124 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c100b3590 con 0x7f3c1006f6e0 2024-05-30T16:52:12.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.058+0000 7f3c0fe00640 1 --2- 172.21.15.67:0/2215559124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1006f6e0 0x7f3c100cd4f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3c100ba9b0 tx=0x7f3bfc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32861f11bcabe854 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:12.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.059+0000 7f3c0f400640 1 -- 172.21.15.67:0/2215559124 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc037070 con 0x7f3c1006f6e0 2024-05-30T16:52:12.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.059+0000 7f3c0f400640 1 -- 172.21.15.67:0/2215559124 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc032440 con 0x7f3c1006f6e0 2024-05-30T16:52:12.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.059+0000 7f3c0f400640 1 -- 172.21.15.67:0/2215559124 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc032810 con 0x7f3c1006f6e0 2024-05-30T16:52:12.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.060+0000 7f3c17c00640 1 -- 172.21.15.67:0/2215559124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1006f6e0 msgr2=0x7f3c100cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:12.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.060+0000 7f3c17c00640 1 --2- 172.21.15.67:0/2215559124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1006f6e0 0x7f3c100cd4f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3c100ba9b0 tx=0x7f3bfc02a1a0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.062+0000 7f3c17c00640 1 -- 172.21.15.67:0/2215559124 shutdown_connections 2024-05-30T16:52:12.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.062+0000 7f3c17c00640 1 --2- 172.21.15.67:0/2215559124 >> v2:172.21.15.169:3300/0 conn(0x7f3c10162e70 0x7f3c100ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.062+0000 7f3c17c00640 1 --2- 172.21.15.67:0/2215559124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c1006f6e0 0x7f3c100cd4f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.062+0000 7f3c17c00640 1 -- 172.21.15.67:0/2215559124 >> 172.21.15.67:0/2215559124 conn(0x7f3c1006bae0 msgr2=0x7f3c1006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.062+0000 7f3c17c00640 1 -- 172.21.15.67:0/2215559124 shutdown_connections 2024-05-30T16:52:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.062+0000 7f3c17c00640 1 -- 172.21.15.67:0/2215559124 wait complete. 2024-05-30T16:52:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.065+0000 7f3c17c00640 1 Processor -- start 2024-05-30T16:52:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.065+0000 7f3c17c00640 1 -- start start 2024-05-30T16:52:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.068+0000 7f3c17c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1006f6e0 0x7f3c10160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.068+0000 7f3c17c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c100d04a0 0x7f3c10157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.069+0000 7f3c17c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c10162e70 0x7f3c101647b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.069+0000 7f3c17c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3c100cda30 con 0x7f3c10162e70 2024-05-30T16:52:12.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.069+0000 7f3c17c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c100b3b50 con 0x7f3c100d04a0 2024-05-30T16:52:12.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.069+0000 7f3c17c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3c100ce770 con 0x7f3c1006f6e0 2024-05-30T16:52:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.069+0000 7f3c15200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1006f6e0 0x7f3c10160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.069+0000 7f3c15200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1006f6e0 0x7f3c10160190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:35884/0 (socket says 172.21.15.67:35884) 2024-05-30T16:52:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.069+0000 7f3c15200640 1 -- 172.21.15.67:0/426347756 learned_addr learned my addr 172.21.15.67:0/426347756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.069+0000 7f3c0fe00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c100d04a0 0x7f3c10157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.070+0000 7f3c15200640 1 -- 172.21.15.67:0/426347756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c100d04a0 msgr2=0x7f3c10157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.070+0000 7f3c15200640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c100d04a0 0x7f3c10157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.070+0000 7f3c15200640 1 -- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c10162e70 msgr2=0x7f3c101647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.071+0000 7f3c15c00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c10162e70 0x7f3c101647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.071+0000 7f3c15200640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c10162e70 0x7f3c101647b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.071+0000 7f3c15200640 1 -- 172.21.15.67:0/426347756 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c10165530 con 0x7f3c1006f6e0 2024-05-30T16:52:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.071+0000 7f3c15200640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1006f6e0 0x7f3c10160190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c410 tx=0x7f3c0400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.071+0000 7f3c15c00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c10162e70 0x7f3c101647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.071+0000 7f3c0cc00640 1 -- 172.21.15.67:0/426347756 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04017070 con 0x7f3c1006f6e0 2024-05-30T16:52:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.072+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c10165810 con 0x7f3c1006f6e0 2024-05-30T16:52:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.072+0000 7f3c0cc00640 1 -- 172.21.15.67:0/426347756 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c040035f0 con 0x7f3c1006f6e0 2024-05-30T16:52:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.072+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c10158b10 con 0x7f3c1006f6e0 2024-05-30T16:52:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.072+0000 7f3c0cc00640 1 -- 172.21.15.67:0/426347756 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04003a00 con 0x7f3c1006f6e0 2024-05-30T16:52:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.074+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bec005290 con 0x7f3c1006f6e0 2024-05-30T16:52:12.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.075+0000 7f3c0cc00640 1 -- 172.21.15.67:0/426347756 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f3c04003ba0 con 0x7f3c1006f6e0 2024-05-30T16:52:12.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.075+0000 7f3c0cc00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bf006d100 0x7f3bf006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.076+0000 7f3c0cc00640 1 -- 172.21.15.67:0/426347756 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 5823+0+0 (secure 0 0 0) 0x7f3c0408c0e0 con 0x7f3c1006f6e0 2024-05-30T16:52:12.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.076+0000 7f3c0fe00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bf006d100 0x7f3bf006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.077+0000 7f3c0fe00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bf006d100 0x7f3bf006f5c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc034000 tx=0x7f3bfc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.083+0000 7f3c0cc00640 1 -- 172.21.15.67:0/426347756 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c04058e30 con 0x7f3c1006f6e0 2024-05-30T16:52:12.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.488+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bec005850 con 0x7f3c1006f6e0 2024-05-30T16:52:12.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.490+0000 7f3c0cc00640 1 -- 172.21.15.67:0/426347756 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7f3c0405d130 con 0x7f3c1006f6e0 2024-05-30T16:52:12.492 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:12.492 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2139720967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2139720967},{"type":"v1","addr":"172.21.15.169:6834","nonce":2139720967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4981,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2317664228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2317664228},{"type":"v1","addr":"172.21.15.169:6835","nonce":2317664228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/350920637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":350920637},{"type":"v1","addr":"172.21.15.67:6835","nonce":350920637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:48:30.019937+0000","modified":"2024-05-30T16:48:33.071363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/3595714006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3595714006},{"type":"v1","addr":"172.21.15.67:6837","nonce":3595714006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5361,"qdb_cluster":[5361]},"id":3}]} 2024-05-30T16:52:12.492 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.492+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bf006d100 msgr2=0x7f3bf006f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.492+0000 7f3c17c00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bf006d100 0x7f3bf006f5c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc034000 tx=0x7f3bfc02e000 comp rx=0 tx=0).stop 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1006f6e0 msgr2=0x7f3c10160190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1006f6e0 0x7f3c10160190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c410 tx=0x7f3c0400c8d0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 shutdown_connections 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3bf006d100 0x7f3bf006f5c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c10162e70 0x7f3c101647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c100d04a0 0x7f3c10157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 --2- 172.21.15.67:0/426347756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1006f6e0 0x7f3c10160190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 >> 172.21.15.67:0/426347756 conn(0x7f3c1006bae0 msgr2=0x7f3c100b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 shutdown_connections 2024-05-30T16:52:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.493+0000 7f3c17c00640 1 -- 172.21.15.67:0/426347756 wait complete. 2024-05-30T16:52:12.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:52:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.792+0000 7f2025400640 1 Processor -- start 2024-05-30T16:52:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.792+0000 7f2025400640 1 -- start start 2024-05-30T16:52:12.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.794+0000 7f2025400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f202006f530 0x7f20200cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.794+0000 7f2025400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20200af890 0x7f20200d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.795+0000 7f2025400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20200cfb30 con 0x7f20200b4e80 2024-05-30T16:52:12.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.795+0000 7f201e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f202006f530 0x7f20200cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.795+0000 7f2025400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f20200cfe10 con 0x7f202006f530 2024-05-30T16:52:12.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.795+0000 7f201e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f202006f530 0x7f20200cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38344/0 (socket says 172.21.15.67:38344) 2024-05-30T16:52:12.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.795+0000 7f201e800640 1 -- 172.21.15.67:0/2902519002 learned_addr learned my addr 172.21.15.67:0/2902519002 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:12.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.795+0000 7f2025400640 1 -- 172.21.15.67:0/2902519002 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20200b4680 con 0x7f20200af890 2024-05-30T16:52:12.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.795+0000 7f201de00640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20200af890 0x7f20200d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.796+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 639458190 0 0) 0x7f20200cfb30 con 0x7f20200b4e80 2024-05-30T16:52:12.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.796+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2004003520 con 0x7f20200b4e80 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.797+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3214395535 0 0) 0x7f2004003520 con 0x7f20200b4e80 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.797+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20200af890 msgr2=0x7f20200d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.797+0000 7f201d400640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20200af890 0x7f20200d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.797+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 >> v2:172.21.15.67:3300/0 conn(0x7f202006f530 msgr2=0x7f20200cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.797+0000 7f201d400640 1 --2- 172.21.15.67:0/2902519002 >> v2:172.21.15.67:3300/0 conn(0x7f202006f530 0x7f20200cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.797+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20200b4960 con 0x7f20200b4e80 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.797+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f20180019f0 con 0x7f20200b4e80 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.797+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 >> v1:172.21.15.169:6789/0 conn(0x7f20200b4e80 legacy=0x7f20200cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.797+0000 7f201d400640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2004003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.798+0000 7f201d400640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2004004800 0x7f2004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.798+0000 7f201d400640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20200af890 0x7f20040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.798+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20200cfe10 con 0x7f20040039b0 2024-05-30T16:52:12.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.798+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20200b4680 con 0x7f2004004800 2024-05-30T16:52:12.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.798+0000 7f201e800640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2004003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.798+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f200400dbd0 con 0x7f20200af890 2024-05-30T16:52:12.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.798+0000 7f201de00640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2004004800 0x7f2004004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.798+0000 7f201f200640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20200af890 0x7f20040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.799+0000 7f201e800640 1 -- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20200af890 msgr2=0x7f20040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:12.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.799+0000 7f201e800640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20200af890 0x7f20040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.799+0000 7f201e800640 1 -- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2004004800 msgr2=0x7f2004004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:12.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.799+0000 7f201e800640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2004004800 0x7f2004004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.799+0000 7f201e800640 1 -- 172.21.15.67:0/2902519002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20200b4960 con 0x7f20040039b0 2024-05-30T16:52:12.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.800+0000 7f201e800640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2004003dd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2010005bd0 tx=0x7f2004005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b824f5652444a9eb server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:12.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.801+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201003c070 con 0x7f20040039b0 2024-05-30T16:52:12.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.801+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201002f8b0 con 0x7f20040039b0 2024-05-30T16:52:12.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.801+0000 7f201d400640 1 -- 172.21.15.67:0/2902519002 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010007790 con 0x7f20040039b0 2024-05-30T16:52:12.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.802+0000 7f2025400640 1 -- 172.21.15.67:0/2902519002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 msgr2=0x7f2004003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:12.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.802+0000 7f2025400640 1 --2- 172.21.15.67:0/2902519002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2004003dd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2010005bd0 tx=0x7f2004005830 comp rx=0 tx=0).stop 2024-05-30T16:52:12.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.802+0000 7f201f200640 1 -- 172.21.15.67:0/2902519002 reap_dead start 2024-05-30T16:52:12.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.802+0000 7f2025400640 1 -- 172.21.15.67:0/2902519002 shutdown_connections 2024-05-30T16:52:12.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.803+0000 7f2025400640 1 -- 172.21.15.67:0/2902519002 >> 172.21.15.67:0/2902519002 conn(0x7f202006bae0 msgr2=0x7f202006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:12.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.804+0000 7f2025400640 1 -- 172.21.15.67:0/2902519002 shutdown_connections 2024-05-30T16:52:12.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.805+0000 7f2025400640 1 -- 172.21.15.67:0/2902519002 wait complete. 2024-05-30T16:52:12.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.807+0000 7f2025400640 1 Processor -- start 2024-05-30T16:52:12.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.808+0000 7f2025400640 1 -- start start 2024-05-30T16:52:12.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.810+0000 7f2025400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2020099730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.810+0000 7f2025400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202015c900 0x7f20201f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.811+0000 7f2025400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202015cd80 0x7f20201f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.811+0000 7f2025400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20200cf8a0 con 0x7f20040039b0 2024-05-30T16:52:12.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.811+0000 7f2025400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20200cf720 con 0x7f202015c900 2024-05-30T16:52:12.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.811+0000 7f2025400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f202006fd20 con 0x7f202015cd80 2024-05-30T16:52:12.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.811+0000 7f201e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2020099730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.811+0000 7f201e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2020099730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38364/0 (socket says 172.21.15.67:38364) 2024-05-30T16:52:12.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.811+0000 7f201e800640 1 -- 172.21.15.67:0/225360769 learned_addr learned my addr 172.21.15.67:0/225360769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:12.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.811+0000 7f201de00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202015c900 0x7f20201f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.811+0000 7f201f200640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202015cd80 0x7f20201f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.812+0000 7f201e800640 1 -- 172.21.15.67:0/225360769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202015cd80 msgr2=0x7f20201f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:12.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.812+0000 7f201e800640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202015cd80 0x7f20201f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.812+0000 7f201e800640 1 -- 172.21.15.67:0/225360769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202015c900 msgr2=0x7f20201f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:12.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.812+0000 7f201e800640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202015c900 0x7f20201f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:12.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.812+0000 7f201e800640 1 -- 172.21.15.67:0/225360769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20200c5ab0 con 0x7f20040039b0 2024-05-30T16:52:12.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.813+0000 7f201e800640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2020099730 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2010007c30 tx=0x7f2010005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:12.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.814+0000 7f2016a00640 1 -- 172.21.15.67:0/225360769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010045020 con 0x7f20040039b0 2024-05-30T16:52:12.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.814+0000 7f2016a00640 1 -- 172.21.15.67:0/225360769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2010038a50 con 0x7f20040039b0 2024-05-30T16:52:12.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.814+0000 7f2016a00640 1 -- 172.21.15.67:0/225360769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201003c040 con 0x7f20040039b0 2024-05-30T16:52:12.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.814+0000 7f2025400640 1 -- 172.21.15.67:0/225360769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202015bb40 con 0x7f20040039b0 2024-05-30T16:52:12.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.814+0000 7f2025400640 1 -- 172.21.15.67:0/225360769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202015c100 con 0x7f20040039b0 2024-05-30T16:52:12.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.817+0000 7f200be00640 1 -- 172.21.15.67:0/225360769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20200689e0 con 0x7f20040039b0 2024-05-30T16:52:12.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.819+0000 7f2016a00640 1 -- 172.21.15.67:0/225360769 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f2010048020 con 0x7f20040039b0 2024-05-30T16:52:12.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.820+0000 7f2016a00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f200006d0d0 0x7f200006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:12.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.820+0000 7f2016a00640 1 -- 172.21.15.67:0/225360769 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5823+0+0 (secure 0 0 0) 0x7f20100b19b0 con 0x7f20040039b0 2024-05-30T16:52:12.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.825+0000 7f201de00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f200006d0d0 0x7f200006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.827+0000 7f201de00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f200006d0d0 0x7f200006f590 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f202015a360 tx=0x7f200c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:12.828+0000 7f2016a00640 1 -- 172.21.15.67:0/225360769 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f201004f030 con 0x7f20040039b0 2024-05-30T16:52:13.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.171+0000 7f200be00640 1 -- 172.21.15.67:0/225360769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f202006e2e0 con 0x7f20040039b0 2024-05-30T16:52:13.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.173+0000 7f2016a00640 1 -- 172.21.15.67:0/225360769 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16805 (secure 0 0 0) 0x7f20200c5ab0 con 0x7f20040039b0 2024-05-30T16:52:13.175 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:13.175 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":51,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:52:11.647751+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:48:27.936635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-30T16:48:29.004136+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2e","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:52:13.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.176+0000 7f200be00640 1 -- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f200006d0d0 msgr2=0x7f200006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.176+0000 7f200be00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f200006d0d0 0x7f200006f590 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f202015a360 tx=0x7f200c008040 comp rx=0 tx=0).stop 2024-05-30T16:52:13.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.176+0000 7f200be00640 1 -- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 msgr2=0x7f2020099730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.176+0000 7f200be00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2020099730 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2010007c30 tx=0x7f2010005e70 comp rx=0 tx=0).stop 2024-05-30T16:52:13.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.178+0000 7f200be00640 1 -- 172.21.15.67:0/225360769 shutdown_connections 2024-05-30T16:52:13.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.178+0000 7f200be00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f200006d0d0 0x7f200006f590 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.178+0000 7f200be00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f202015cd80 0x7f20201f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.178+0000 7f200be00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202015c900 0x7f20201f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.178+0000 7f200be00640 1 --2- 172.21.15.67:0/225360769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20040039b0 0x7f2020099730 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.178+0000 7f200be00640 1 -- 172.21.15.67:0/225360769 >> 172.21.15.67:0/225360769 conn(0x7f202006bae0 msgr2=0x7f20200b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:13.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.178+0000 7f200be00640 1 -- 172.21.15.67:0/225360769 shutdown_connections 2024-05-30T16:52:13.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.178+0000 7f200be00640 1 -- 172.21.15.67:0/225360769 wait complete. 2024-05-30T16:52:13.207 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-30T16:52:13.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:52:13.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.413+0000 7fefaac00640 1 Processor -- start 2024-05-30T16:52:13.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.414+0000 7fefaac00640 1 -- start start 2024-05-30T16:52:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.415+0000 7fefaac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fefa40722d0 0x7fefa416d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.415+0000 7fefaac00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa406f590 con 0x7fefa4162940 2024-05-30T16:52:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.415+0000 7fefaac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa406f870 con 0x7fefa4162d20 2024-05-30T16:52:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.415+0000 7fefaac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fefa40ad3d0 con 0x7fefa40722d0 2024-05-30T16:52:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.415+0000 7fefa9800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fefa4162940 0x7fefa406eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48412/0 (socket says 172.21.15.67:48412) 2024-05-30T16:52:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.415+0000 7fefa9800640 1 -- 172.21.15.67:0/1531569498 learned_addr learned my addr 172.21.15.67:0/1531569498 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefaa200640 1 --2- 172.21.15.67:0/1531569498 >> v2:172.21.15.169:3300/0 conn(0x7fefa40722d0 0x7fefa416d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384654913 0 0) 0x7fefa406f590 con 0x7fefa4162940 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef8c003520 con 0x7fefa4162940 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3867613632 0 0) 0x7fef8c003520 con 0x7fefa4162940 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 >> v2:172.21.15.169:3300/0 conn(0x7fefa40722d0 msgr2=0x7fefa416d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 --2- 172.21.15.67:0/1531569498 >> v2:172.21.15.169:3300/0 conn(0x7fefa40722d0 0x7fefa416d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 >> v1:172.21.15.169:6789/0 conn(0x7fefa4162d20 legacy=0x7fefa4071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa406f210 con 0x7fefa4162940 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fef980019f0 con 0x7fefa4162940 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 >> v1:172.21.15.67:6789/0 conn(0x7fefa4162940 legacy=0x7fefa406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c0039b0 0x7fef8c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c004800 0x7fef8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c005290 0x7fef8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fefa40ad3d0 con 0x7fef8c0039b0 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c00c1e0 con 0x7fef8c004800 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.416+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fef8c00c3e0 con 0x7fef8c005290 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.417+0000 7fefa9800640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c0039b0 0x7fef8c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.417+0000 7fefa8e00640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c004800 0x7fef8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.417+0000 7fefaa200640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c005290 0x7fef8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.417+0000 7fefa9800640 1 -- 172.21.15.67:0/1531569498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c005290 msgr2=0x7fef8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.417+0000 7fefa9800640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef8c005290 0x7fef8c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.417+0000 7fefa9800640 1 -- 172.21.15.67:0/1531569498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c004800 msgr2=0x7fef8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.417+0000 7fefa9800640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c004800 0x7fef8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.417+0000 7fefaa200640 1 -- 172.21.15.67:0/1531569498 reap_dead start 2024-05-30T16:52:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.417+0000 7fefa9800640 1 -- 172.21.15.67:0/1531569498 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa406f210 con 0x7fef8c0039b0 2024-05-30T16:52:13.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.418+0000 7fefa9800640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c0039b0 0x7fef8c003db0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fef9800a9c0 tx=0x7fef8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70047efe2e4a6a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:13.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.418+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9800a590 con 0x7fef8c0039b0 2024-05-30T16:52:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.418+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9802f070 con 0x7fef8c0039b0 2024-05-30T16:52:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.419+0000 7fefa3e00640 1 -- 172.21.15.67:0/1531569498 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9802bdb0 con 0x7fef8c0039b0 2024-05-30T16:52:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.419+0000 7fefaac00640 1 -- 172.21.15.67:0/1531569498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c0039b0 msgr2=0x7fef8c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.419+0000 7fefaac00640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c0039b0 0x7fef8c003db0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fef9800a9c0 tx=0x7fef8c005830 comp rx=0 tx=0).stop 2024-05-30T16:52:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.419+0000 7fefaac00640 1 -- 172.21.15.67:0/1531569498 shutdown_connections 2024-05-30T16:52:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.419+0000 7fefaac00640 1 --2- 172.21.15.67:0/1531569498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c0039b0 0x7fef8c003db0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.419+0000 7fefaac00640 1 -- 172.21.15.67:0/1531569498 >> 172.21.15.67:0/1531569498 conn(0x7fefa40b1fc0 msgr2=0x7fefa40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.419+0000 7fefaac00640 1 -- 172.21.15.67:0/1531569498 shutdown_connections 2024-05-30T16:52:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.419+0000 7fefaac00640 1 -- 172.21.15.67:0/1531569498 wait complete. 2024-05-30T16:52:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.421+0000 7fefaac00640 1 Processor -- start 2024-05-30T16:52:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.421+0000 7fefaac00640 1 -- start start 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0039b0 0x7fefa415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefa41f8900 0x7fefa4165d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa41524c0 0x7fefa40b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fefa406ec80 con 0x7fefa41524c0 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fefa406eb00 con 0x7fef8c0039b0 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fefa406ee00 con 0x7fefa41f8900 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaa200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa41524c0 0x7fefa40b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaa200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa41524c0 0x7fefa40b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38372/0 (socket says 172.21.15.67:38372) 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaa200640 1 -- 172.21.15.67:0/2576824467 learned_addr learned my addr 172.21.15.67:0/2576824467 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefa9800640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0039b0 0x7fefa415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefa8e00640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefa41f8900 0x7fefa4165d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.422+0000 7fefaa200640 1 -- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefa41f8900 msgr2=0x7fefa4165d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefaa200640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefa41f8900 0x7fefa4165d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefaa200640 1 -- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0039b0 msgr2=0x7fefa415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefaa200640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0039b0 0x7fefa415cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefaa200640 1 -- 172.21.15.67:0/2576824467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa41663a0 con 0x7fefa41524c0 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefa9800640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0039b0 0x7fefa415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefa8e00640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefa41f8900 0x7fefa4165d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefaa200640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa41524c0 0x7fefa40b40e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fef9c006fd0 tx=0x7fef9c00f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefa1600640 1 -- 172.21.15.67:0/2576824467 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c017020 con 0x7fefa41524c0 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefa1600640 1 -- 172.21.15.67:0/2576824467 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9c004460 con 0x7fefa41524c0 2024-05-30T16:52:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefa1600640 1 -- 172.21.15.67:0/2576824467 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c003850 con 0x7fefa41524c0 2024-05-30T16:52:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa40cdf80 con 0x7fefa41524c0 2024-05-30T16:52:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.423+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa40ce540 con 0x7fefa41524c0 2024-05-30T16:52:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.424+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa40689e0 con 0x7fefa41524c0 2024-05-30T16:52:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.425+0000 7fefa1600640 1 -- 172.21.15.67:0/2576824467 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7fef9c01a020 con 0x7fefa41524c0 2024-05-30T16:52:13.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.426+0000 7fefa1600640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef8406d100 0x7fef8406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.426+0000 7fefa1600640 1 -- 172.21.15.67:0/2576824467 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5823+0+0 (secure 0 0 0) 0x7fef9c08cbf0 con 0x7fefa41524c0 2024-05-30T16:52:13.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.426+0000 7fefa9800640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef8406d100 0x7fef8406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.427+0000 7fefa9800640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef8406d100 0x7fef8406f5c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef98033000 tx=0x7fef98038040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.430+0000 7fefa1600640 1 -- 172.21.15.67:0/2576824467 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef9c021020 con 0x7fefa41524c0 2024-05-30T16:52:13.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.665+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefa40ae000 con 0x7fefa41524c0 2024-05-30T16:52:13.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.666+0000 7fefa1600640 1 -- 172.21.15.67:0/2576824467 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7fef9c05a0e0 con 0x7fefa41524c0 2024-05-30T16:52:13.668 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:13.668 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2139720967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2139720967},{"type":"v1","addr":"172.21.15.169:6834","nonce":2139720967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4981,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2317664228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2317664228},{"type":"v1","addr":"172.21.15.169:6835","nonce":2317664228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/350920637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":350920637},{"type":"v1","addr":"172.21.15.67:6835","nonce":350920637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:48:30.019937+0000","modified":"2024-05-30T16:48:33.071363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/3595714006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3595714006},{"type":"v1","addr":"172.21.15.67:6837","nonce":3595714006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5361,"qdb_cluster":[5361]},"id":3}]} 2024-05-30T16:52:13.668 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-05-30T16:52:13.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef8406d100 msgr2=0x7fef8406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef8406d100 0x7fef8406f5c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef98033000 tx=0x7fef98038040 comp rx=0 tx=0).stop 2024-05-30T16:52:13.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa41524c0 msgr2=0x7fefa40b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa41524c0 0x7fefa40b40e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fef9c006fd0 tx=0x7fef9c00f920 comp rx=0 tx=0).stop 2024-05-30T16:52:13.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 shutdown_connections 2024-05-30T16:52:13.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef8406d100 0x7fef8406f5c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa41524c0 0x7fefa40b40e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fefa41f8900 0x7fefa4165d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 --2- 172.21.15.67:0/2576824467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef8c0039b0 0x7fefa415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 >> 172.21.15.67:0/2576824467 conn(0x7fefa40b1fc0 msgr2=0x7fefa40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:13.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 shutdown_connections 2024-05-30T16:52:13.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.669+0000 7fefaac00640 1 -- 172.21.15.67:0/2576824467 wait complete. 2024-05-30T16:52:13.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:52:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.877+0000 7f26da800640 1 Processor -- start 2024-05-30T16:52:13.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.878+0000 7f26da800640 1 -- start start 2024-05-30T16:52:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.879+0000 7f26da800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cfa40 0x7f26d4169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.879+0000 7f26da800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d415ec50 con 0x7f26d40d16a0 2024-05-30T16:52:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.879+0000 7f26da800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d40cff80 con 0x7f26d415e870 2024-05-30T16:52:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.879+0000 7f26da800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26d40d2150 con 0x7f26d40cfa40 2024-05-30T16:52:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.879+0000 7f26d8a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f26d40d16a0 0x7f26d40cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48418/0 (socket says 172.21.15.67:48418) 2024-05-30T16:52:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.879+0000 7f26d8a00640 1 -- 172.21.15.67:0/4032089136 learned_addr learned my addr 172.21.15.67:0/4032089136 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d9e00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cfa40 0x7f26d4169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042176722 0 0) 0x7f26d415ec50 con 0x7f26d40d16a0 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26bc003520 con 0x7f26d40d16a0 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1708433062 0 0) 0x7f26bc003520 con 0x7f26d40d16a0 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cfa40 msgr2=0x7f26d4169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cfa40 0x7f26d4169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 >> v1:172.21.15.169:6789/0 conn(0x7f26d415e870 legacy=0x7f26d40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d40d0260 con 0x7f26d40d16a0 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f26c40017f0 con 0x7f26d40d16a0 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 >> v1:172.21.15.67:6789/0 conn(0x7f26d40d16a0 legacy=0x7f26d40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc0039e0 0x7f26bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc0043d0 0x7f26bc0048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26bc0050a0 0x7f26bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40d2150 con 0x7f26bc0050a0 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc00bfb0 con 0x7f26bc0043d0 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.880+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26bc00c1b0 con 0x7f26bc0039e0 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.881+0000 7f26d9e00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26bc0050a0 0x7f26bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.881+0000 7f26d9e00640 1 -- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc0039e0 msgr2=0x7f26bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.881+0000 7f26d8a00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc0043d0 0x7f26bc0048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.881+0000 7f26d9e00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26bc0039e0 0x7f26bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.881+0000 7f26d9e00640 1 -- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc0043d0 msgr2=0x7f26bc0048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.881+0000 7f26d9e00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc0043d0 0x7f26bc0048a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.881+0000 7f26d9e00640 1 -- 172.21.15.67:0/4032089136 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc003520 con 0x7f26bc0050a0 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.882+0000 7f26d9e00640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26bc0050a0 0x7f26bc00b9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26d40cfe20 tx=0x7f26bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eada8341b458bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.882+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc036070 con 0x7f26bc0050a0 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.882+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26cc031600 con 0x7f26bc0050a0 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.882+0000 7f26d3e00640 1 -- 172.21.15.67:0/4032089136 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc009bd0 con 0x7f26bc0050a0 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.883+0000 7f26da800640 1 -- 172.21.15.67:0/4032089136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26bc0050a0 msgr2=0x7f26bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.883+0000 7f26da800640 1 --2- 172.21.15.67:0/4032089136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26bc0050a0 0x7f26bc00b9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26d40cfe20 tx=0x7f26bc005830 comp rx=0 tx=0).stop 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.883+0000 7f26d9e00640 1 -- 172.21.15.67:0/4032089136 reap_dead start 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.883+0000 7f26da800640 1 -- 172.21.15.67:0/4032089136 shutdown_connections 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.883+0000 7f26da800640 1 -- 172.21.15.67:0/4032089136 >> 172.21.15.67:0/4032089136 conn(0x7f26d408bfc0 msgr2=0x7f26d40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.883+0000 7f26da800640 1 -- 172.21.15.67:0/4032089136 shutdown_connections 2024-05-30T16:52:13.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.883+0000 7f26da800640 1 -- 172.21.15.67:0/4032089136 wait complete. 2024-05-30T16:52:13.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.885+0000 7f26da800640 1 Processor -- start 2024-05-30T16:52:13.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.886+0000 7f26da800640 1 -- start start 2024-05-30T16:52:13.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26da800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc0050a0 0x7f26d41598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26da800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cd820 0x7f26d40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26da800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26d4160f40 0x7f26d41613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26da800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40d4fb0 con 0x7f26d4160f40 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26da800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40d0820 con 0x7f26bc0050a0 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26da800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26d40d35f0 con 0x7f26d40cd820 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26d9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26d4160f40 0x7f26d41613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26d9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26d4160f40 0x7f26d41613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38386/0 (socket says 172.21.15.67:38386) 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26d9e00640 1 -- 172.21.15.67:0/1463575766 learned_addr learned my addr 172.21.15.67:0/1463575766 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26d8a00640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cd820 0x7f26d40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.887+0000 7f26d9e00640 1 -- 172.21.15.67:0/1463575766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cd820 msgr2=0x7f26d40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d9e00640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cd820 0x7f26d40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d9400640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc0050a0 0x7f26d41598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d9e00640 1 -- 172.21.15.67:0/1463575766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc0050a0 msgr2=0x7f26d41598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:13.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d9e00640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc0050a0 0x7f26d41598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:13.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d9e00640 1 -- 172.21.15.67:0/1463575766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d4161c70 con 0x7f26d4160f40 2024-05-30T16:52:13.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d8a00640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cd820 0x7f26d40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:13.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d9e00640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26d4160f40 0x7f26d41613c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f26cc034040 tx=0x7f26cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:13.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d1600640 1 -- 172.21.15.67:0/1463575766 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc036040 con 0x7f26d4160f40 2024-05-30T16:52:13.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d1600640 1 -- 172.21.15.67:0/1463575766 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26cc043070 con 0x7f26d4160f40 2024-05-30T16:52:13.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26d1600640 1 -- 172.21.15.67:0/1463575766 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc009bd0 con 0x7f26d4160f40 2024-05-30T16:52:13.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.888+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d40c8250 con 0x7f26d4160f40 2024-05-30T16:52:13.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.889+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d40c8810 con 0x7f26d4160f40 2024-05-30T16:52:13.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.890+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d40ce1e0 con 0x7f26d4160f40 2024-05-30T16:52:13.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.890+0000 7f26d1600640 1 -- 172.21.15.67:0/1463575766 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f26cc0078e0 con 0x7f26d4160f40 2024-05-30T16:52:13.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.891+0000 7f26d1600640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f26b806d100 0x7f26b806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:13.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.891+0000 7f26d1600640 1 -- 172.21.15.67:0/1463575766 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5823+0+0 (secure 0 0 0) 0x7f26cc0adc00 con 0x7f26d4160f40 2024-05-30T16:52:13.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.892+0000 7f26d9400640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f26b806d100 0x7f26b806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:13.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.892+0000 7f26d9400640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f26b806d100 0x7f26b806f5c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f26c80015d0 tx=0x7f26c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:13.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:13.896+0000 7f26d1600640 1 -- 172.21.15.67:0/1463575766 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26cc0afe20 con 0x7f26d4160f40 2024-05-30T16:52:14.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.132+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26d40c8a10 con 0x7f26d4160f40 2024-05-30T16:52:14.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.135+0000 7f26d1600640 1 -- 172.21.15.67:0/1463575766 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7f26d40c8a10 con 0x7f26d4160f40 2024-05-30T16:52:14.137 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:14.138 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2139720967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2139720967},{"type":"v1","addr":"172.21.15.169:6834","nonce":2139720967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4981,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2317664228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2317664228},{"type":"v1","addr":"172.21.15.169:6835","nonce":2317664228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/350920637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":350920637},{"type":"v1","addr":"172.21.15.67:6835","nonce":350920637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:48:30.019937+0000","modified":"2024-05-30T16:48:33.071363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/3595714006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3595714006},{"type":"v1","addr":"172.21.15.67:6837","nonce":3595714006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5361,"qdb_cluster":[5361]},"id":3}]} 2024-05-30T16:52:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-05-30T16:52:14.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f26b806d100 msgr2=0x7f26b806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f26b806d100 0x7f26b806f5c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f26c80015d0 tx=0x7f26c8009040 comp rx=0 tx=0).stop 2024-05-30T16:52:14.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26d4160f40 msgr2=0x7f26d41613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26d4160f40 0x7f26d41613c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f26cc034040 tx=0x7f26cc009750 comp rx=0 tx=0).stop 2024-05-30T16:52:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 shutdown_connections 2024-05-30T16:52:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f26b806d100 0x7f26b806f5c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26d4160f40 0x7f26d41613c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26d40cd820 0x7f26d40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 --2- 172.21.15.67:0/1463575766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26bc0050a0 0x7f26d41598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 >> 172.21.15.67:0/1463575766 conn(0x7f26d408bfc0 msgr2=0x7f26d40b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 shutdown_connections 2024-05-30T16:52:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.138+0000 7f26da800640 1 -- 172.21.15.67:0/1463575766 wait complete. 2024-05-30T16:52:14.159 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:52:14.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.354+0000 7f7086a00640 1 Processor -- start 2024-05-30T16:52:14.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.354+0000 7f7086a00640 1 -- start start 2024-05-30T16:52:14.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.356+0000 7f7086a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7080156b30 0x7f70800af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.356+0000 7f7086a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70800ad940 0x7f70800b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.356+0000 7f7086a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7080156f10 con 0x7f708006ee40 2024-05-30T16:52:14.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.356+0000 7f7086a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f708006dda0 con 0x7f7080156b30 2024-05-30T16:52:14.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.356+0000 7f7086a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f708006e080 con 0x7f70800ad940 2024-05-30T16:52:14.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.356+0000 7f7084c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f708006ee40 0x7f70800ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48424/0 (socket says 172.21.15.67:48424) 2024-05-30T16:52:14.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.356+0000 7f7084c00640 1 -- 172.21.15.67:0/2568828247 learned_addr learned my addr 172.21.15.67:0/2568828247 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:14.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.356+0000 7f7085600640 1 --2- 172.21.15.67:0/2568828247 >> v2:172.21.15.67:3300/0 conn(0x7f7080156b30 0x7f70800af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.357+0000 7f7086000640 1 --2- 172.21.15.67:0/2568828247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70800ad940 0x7f70800b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.357+0000 7f707fe00640 1 -- 172.21.15.67:0/2568828247 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1696432126 0 0) 0x7f7080156f10 con 0x7f708006ee40 2024-05-30T16:52:14.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.357+0000 7f707fe00640 1 -- 172.21.15.67:0/2568828247 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7068003520 con 0x7f708006ee40 2024-05-30T16:52:14.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.357+0000 7f7085600640 1 -- 172.21.15.67:0/2568828247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70800ad940 msgr2=0x7f70800b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:14.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.357+0000 7f7085600640 1 --2- 172.21.15.67:0/2568828247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70800ad940 0x7f70800b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.357+0000 7f7085600640 1 -- 172.21.15.67:0/2568828247 >> v1:172.21.15.67:6789/0 conn(0x7f708006ee40 legacy=0x7f70800ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.357+0000 7f7085600640 1 -- 172.21.15.67:0/2568828247 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708006e360 con 0x7f7080156b30 2024-05-30T16:52:14.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.357+0000 7f7086000640 1 --2- 172.21.15.67:0/2568828247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70800ad940 0x7f70800b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:52:14.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.358+0000 7f7085600640 1 --2- 172.21.15.67:0/2568828247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7080156b30 0x7f70800af6e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f70740097a0 tx=0x7f707402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc444a2c9dc02c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:14.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.359+0000 7f707fe00640 1 -- 172.21.15.67:0/2568828247 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074037070 con 0x7f7080156b30 2024-05-30T16:52:14.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.359+0000 7f707fe00640 1 -- 172.21.15.67:0/2568828247 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707402ae50 con 0x7f7080156b30 2024-05-30T16:52:14.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.359+0000 7f707fe00640 1 -- 172.21.15.67:0/2568828247 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074032640 con 0x7f7080156b30 2024-05-30T16:52:14.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.359+0000 7f7086a00640 1 -- 172.21.15.67:0/2568828247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7080156b30 msgr2=0x7f70800af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.359+0000 7f7086a00640 1 --2- 172.21.15.67:0/2568828247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7080156b30 0x7f70800af6e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f70740097a0 tx=0x7f707402a100 comp rx=0 tx=0).stop 2024-05-30T16:52:14.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.359+0000 7f7086a00640 1 -- 172.21.15.67:0/2568828247 shutdown_connections 2024-05-30T16:52:14.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.359+0000 7f7086a00640 1 --2- 172.21.15.67:0/2568828247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70800ad940 0x7f70800b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.359+0000 7f7086a00640 1 --2- 172.21.15.67:0/2568828247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7080156b30 0x7f70800af6e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.359+0000 7f7086a00640 1 -- 172.21.15.67:0/2568828247 >> 172.21.15.67:0/2568828247 conn(0x7f70800b1fc0 msgr2=0x7f70800b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:14.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.360+0000 7f7086a00640 1 -- 172.21.15.67:0/2568828247 shutdown_connections 2024-05-30T16:52:14.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.360+0000 7f7086a00640 1 -- 172.21.15.67:0/2568828247 wait complete. 2024-05-30T16:52:14.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.361+0000 7f7086a00640 1 Processor -- start 2024-05-30T16:52:14.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.361+0000 7f7086a00640 1 -- start start 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.362+0000 7f7086a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f708006ee40 0x7f708015f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.362+0000 7f7086a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70800ad940 0x7f7080158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.362+0000 7f7086a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7080156b30 0x7f7080149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.362+0000 7f7086a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f70800afc20 con 0x7f70800ad940 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.362+0000 7f7086a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f70800b2c30 con 0x7f7080156b30 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.362+0000 7f7086a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f708015e5c0 con 0x7f708006ee40 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.362+0000 7f7084c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70800ad940 0x7f7080158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7084c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70800ad940 0x7f7080158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38408/0 (socket says 172.21.15.67:38408) 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7084c00640 1 -- 172.21.15.67:0/1512913969 learned_addr learned my addr 172.21.15.67:0/1512913969 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7084c00640 1 -- 172.21.15.67:0/1512913969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f708006ee40 msgr2=0x7f708015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7085600640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f708006ee40 0x7f708015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7084c00640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f708006ee40 0x7f708015f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7086000640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7080156b30 0x7f7080149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7084c00640 1 -- 172.21.15.67:0/1512913969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7080156b30 msgr2=0x7f7080149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7084c00640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7080156b30 0x7f7080149f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7084c00640 1 -- 172.21.15.67:0/1512913969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708015e8a0 con 0x7f70800ad940 2024-05-30T16:52:14.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f7084c00640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70800ad940 0x7f7080158410 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c390 tx=0x7f706c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.363+0000 7f707d600640 1 -- 172.21.15.67:0/1512913969 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c017070 con 0x7f70800ad940 2024-05-30T16:52:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.364+0000 7f707d600640 1 -- 172.21.15.67:0/1512913969 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f706c005ce0 con 0x7f70800ad940 2024-05-30T16:52:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.364+0000 7f707d600640 1 -- 172.21.15.67:0/1512913969 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c0072f0 con 0x7f70800ad940 2024-05-30T16:52:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.364+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708015eb80 con 0x7f70800ad940 2024-05-30T16:52:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.364+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708015f140 con 0x7f70800ad940 2024-05-30T16:52:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.365+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708015f340 con 0x7f70800ad940 2024-05-30T16:52:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.366+0000 7f707d600640 1 -- 172.21.15.67:0/1512913969 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f706c01a070 con 0x7f70800ad940 2024-05-30T16:52:14.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.367+0000 7f707d600640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706006d100 0x7f706006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.367+0000 7f707d600640 1 -- 172.21.15.67:0/1512913969 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5823+0+0 (secure 0 0 0) 0x7f706c089920 con 0x7f70800ad940 2024-05-30T16:52:14.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.367+0000 7f7085600640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706006d100 0x7f706006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.367+0000 7f7085600640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706006d100 0x7f706006f5c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7074034000 tx=0x7f707402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:14.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.371+0000 7f707d600640 1 -- 172.21.15.67:0/1512913969 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f708015f340 con 0x7f70800ad940 2024-05-30T16:52:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.567+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70800689e0 con 0x7f70800ad940 2024-05-30T16:52:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.569+0000 7f707d600640 1 -- 172.21.15.67:0/1512913969 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16805 (secure 0 0 0) 0x7f706c059640 con 0x7f70800ad940 2024-05-30T16:52:14.571 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:14.571 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":51,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:52:11.647751+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:48:27.936635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-30T16:48:29.004136+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2e","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706006d100 msgr2=0x7f706006f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706006d100 0x7f706006f5c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7074034000 tx=0x7f707402e000 comp rx=0 tx=0).stop 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70800ad940 msgr2=0x7f7080158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70800ad940 0x7f7080158410 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c390 tx=0x7f706c00c850 comp rx=0 tx=0).stop 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 shutdown_connections 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706006d100 0x7f706006f5c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7080156b30 0x7f7080149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70800ad940 0x7f7080158410 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 --2- 172.21.15.67:0/1512913969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f708006ee40 0x7f708015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 >> 172.21.15.67:0/1512913969 conn(0x7f70800b1fc0 msgr2=0x7f708015b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 shutdown_connections 2024-05-30T16:52:14.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.572+0000 7f7086a00640 1 -- 172.21.15.67:0/1512913969 wait complete. 2024-05-30T16:52:14.594 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-30T16:52:14.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.802+0000 7f2cc2800640 1 Processor -- start 2024-05-30T16:52:14.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.802+0000 7f2cc2800640 1 -- start start 2024-05-30T16:52:14.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.804+0000 7f2cc2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc15e870 0x7f2cbc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.804+0000 7f2cc2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2cbc0d14d0 0x7f2cbc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.804+0000 7f2cc2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cbc0cfd00 con 0x7f2cbc0cf870 2024-05-30T16:52:14.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.804+0000 7f2cc2800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2cbc0d2150 con 0x7f2cbc0d14d0 2024-05-30T16:52:14.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.804+0000 7f2cc2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2cbc15ec50 con 0x7f2cbc15e870 2024-05-30T16:52:14.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.804+0000 7f2cc0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2cbc0d14d0 0x7f2cbc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.804+0000 7f2cc0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2cbc0d14d0 0x7f2cbc0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38410/0 (socket says 172.21.15.67:38410) 2024-05-30T16:52:14.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.804+0000 7f2cc0a00640 1 -- 172.21.15.67:0/644068740 learned_addr learned my addr 172.21.15.67:0/644068740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.805+0000 7f2cc0a00640 1 -- 172.21.15.67:0/644068740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc15e870 msgr2=0x7f2cbc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.805+0000 7f2cc0a00640 1 --2- 172.21.15.67:0/644068740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc15e870 0x7f2cbc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.805+0000 7f2cc0a00640 1 -- 172.21.15.67:0/644068740 >> v1:172.21.15.169:6789/0 conn(0x7f2cbc0cf870 legacy=0x7f2cbc162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.805+0000 7f2cc0a00640 1 -- 172.21.15.67:0/644068740 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc0cff80 con 0x7f2cbc0d14d0 2024-05-30T16:52:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.806+0000 7f2cc0a00640 1 --2- 172.21.15.67:0/644068740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0d14d0 0x7f2cbc0cf330 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009ea0 tx=0x7f2ca802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9aaf246058abdf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.806+0000 7f2cbbe00640 1 -- 172.21.15.67:0/644068740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8036070 con 0x7f2cbc0d14d0 2024-05-30T16:52:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.806+0000 7f2cbbe00640 1 -- 172.21.15.67:0/644068740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca8031600 con 0x7f2cbc0d14d0 2024-05-30T16:52:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.806+0000 7f2cbbe00640 1 -- 172.21.15.67:0/644068740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80319d0 con 0x7f2cbc0d14d0 2024-05-30T16:52:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.807+0000 7f2cc2800640 1 -- 172.21.15.67:0/644068740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0d14d0 msgr2=0x7f2cbc0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.807+0000 7f2cc2800640 1 --2- 172.21.15.67:0/644068740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0d14d0 0x7f2cbc0cf330 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009ea0 tx=0x7f2ca802a760 comp rx=0 tx=0).stop 2024-05-30T16:52:14.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.807+0000 7f2cc2800640 1 -- 172.21.15.67:0/644068740 shutdown_connections 2024-05-30T16:52:14.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.807+0000 7f2cc2800640 1 --2- 172.21.15.67:0/644068740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0d14d0 0x7f2cbc0cf330 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.807+0000 7f2cc2800640 1 --2- 172.21.15.67:0/644068740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc15e870 0x7f2cbc0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.807+0000 7f2cc2800640 1 -- 172.21.15.67:0/644068740 >> 172.21.15.67:0/644068740 conn(0x7f2cbc08bfc0 msgr2=0x7f2cbc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:14.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.807+0000 7f2cc2800640 1 -- 172.21.15.67:0/644068740 shutdown_connections 2024-05-30T16:52:14.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.807+0000 7f2cc2800640 1 -- 172.21.15.67:0/644068740 wait complete. 2024-05-30T16:52:14.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.809+0000 7f2cc2800640 1 Processor -- start 2024-05-30T16:52:14.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.809+0000 7f2cc2800640 1 -- start start 2024-05-30T16:52:14.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.810+0000 7f2cc2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0cf870 0x7f2cbc1f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.810+0000 7f2cc2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0d14d0 0x7f2cbc1f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.810+0000 7f2cc2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc15e870 0x7f2cbc1e8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.810+0000 7f2cc2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc0d0820 con 0x7f2cbc0cf870 2024-05-30T16:52:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.810+0000 7f2cc2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc0d0540 con 0x7f2cbc15e870 2024-05-30T16:52:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.810+0000 7f2cc2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2cbc0d4fb0 con 0x7f2cbc0d14d0 2024-05-30T16:52:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.810+0000 7f2cc1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0cf870 0x7f2cbc1f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.810+0000 7f2cc1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0cf870 0x7f2cbc1f9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38416/0 (socket says 172.21.15.67:38416) 2024-05-30T16:52:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.810+0000 7f2cc1400640 1 -- 172.21.15.67:0/54594360 learned_addr learned my addr 172.21.15.67:0/54594360 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:14.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc1e00640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc15e870 0x7f2cbc1e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc0a00640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0d14d0 0x7f2cbc1f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc1400640 1 -- 172.21.15.67:0/54594360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0d14d0 msgr2=0x7f2cbc1f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc1400640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0d14d0 0x7f2cbc1f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc1400640 1 -- 172.21.15.67:0/54594360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc15e870 msgr2=0x7f2cbc1e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc1400640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc15e870 0x7f2cbc1e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc1400640 1 -- 172.21.15.67:0/54594360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc1e88d0 con 0x7f2cbc0cf870 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc1e00640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc15e870 0x7f2cbc1e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc1400640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0cf870 0x7f2cbc1f9d90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00b4d0 tx=0x7f2cac00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cb9600640 1 -- 172.21.15.67:0/54594360 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac014070 con 0x7f2cbc0cf870 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cb9600640 1 -- 172.21.15.67:0/54594360 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cac010870 con 0x7f2cbc0cf870 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cb9600640 1 -- 172.21.15.67:0/54594360 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac010c60 con 0x7f2cbc0cf870 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc1e8bb0 con 0x7f2cbc0cf870 2024-05-30T16:52:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.811+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1e9170 con 0x7f2cbc0cf870 2024-05-30T16:52:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.813+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc0689e0 con 0x7f2cbc0cf870 2024-05-30T16:52:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.814+0000 7f2cb9600640 1 -- 172.21.15.67:0/54594360 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f2cac017070 con 0x7f2cbc0cf870 2024-05-30T16:52:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.814+0000 7f2cb9600640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c9c06d0d0 0x7f2c9c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.815+0000 7f2cb9600640 1 -- 172.21.15.67:0/54594360 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5823+0+0 (secure 0 0 0) 0x7f2cac08ccb0 con 0x7f2cbc0cf870 2024-05-30T16:52:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.815+0000 7f2cc0a00640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c9c06d0d0 0x7f2c9c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:14.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.816+0000 7f2cc0a00640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c9c06d0d0 0x7f2c9c06f590 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8033000 tx=0x7f2ca8009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:14.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:14.819+0000 7f2cb9600640 1 -- 172.21.15.67:0/54594360 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2cac05a050 con 0x7f2cbc0cf870 2024-05-30T16:52:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.064+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2cbc0ba500 con 0x7f2cbc0cf870 2024-05-30T16:52:15.066 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-05-30T16:52:15.065+0000 7f4982000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-30T16:52:15.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.068+0000 7f2cb9600640 1 -- 172.21.15.67:0/54594360 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f2cac059950 con 0x7f2cbc0cf870 2024-05-30T16:52:15.070 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-30T16:52:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c9c06d0d0 msgr2=0x7f2c9c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c9c06d0d0 0x7f2c9c06f590 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8033000 tx=0x7f2ca8009000 comp rx=0 tx=0).stop 2024-05-30T16:52:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0cf870 msgr2=0x7f2cbc1f9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0cf870 0x7f2cbc1f9d90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00b4d0 tx=0x7f2cac00b990 comp rx=0 tx=0).stop 2024-05-30T16:52:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 shutdown_connections 2024-05-30T16:52:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2c9c06d0d0 0x7f2c9c06f590 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc15e870 0x7f2cbc1e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:15.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cbc0d14d0 0x7f2cbc1f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 --2- 172.21.15.67:0/54594360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0cf870 0x7f2cbc1f9d90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 >> 172.21.15.67:0/54594360 conn(0x7f2cbc08bfc0 msgr2=0x7f2cbc06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 shutdown_connections 2024-05-30T16:52:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.072+0000 7f2cc2800640 1 -- 172.21.15.67:0/54594360 wait complete. 2024-05-30T16:52:15.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-30T16:52:15.157 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T16:52:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.286+0000 7f04ae800640 1 Processor -- start 2024-05-30T16:52:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.287+0000 7f04ae800640 1 -- start start 2024-05-30T16:52:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.288+0000 7f04ae800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f04a80cd2a0 0x7f04a80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.288+0000 7f04ae800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a815cb60 0x7f04a815cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.288+0000 7f04ae800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a815d900 con 0x7f04a8153510 2024-05-30T16:52:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.288+0000 7f04ae800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f04a81538f0 con 0x7f04a80cd2a0 2024-05-30T16:52:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.288+0000 7f04ae800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f04a80b48c0 con 0x7f04a815cb60 2024-05-30T16:52:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.288+0000 7f04ad400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f04a8153510 0x7f04a80ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48428/0 (socket says 172.21.15.67:48428) 2024-05-30T16:52:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.288+0000 7f04ad400640 1 -- 172.21.15.67:0/2813360178 learned_addr learned my addr 172.21.15.67:0/2813360178 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.288+0000 7f04aca00640 1 --2- 172.21.15.67:0/2813360178 >> v2:172.21.15.67:3300/0 conn(0x7f04a80cd2a0 0x7f04a80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.289+0000 7f04a7e00640 1 -- 172.21.15.67:0/2813360178 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 80588225 0 0) 0x7f04a815d900 con 0x7f04a8153510 2024-05-30T16:52:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.289+0000 7f04aca00640 1 -- 172.21.15.67:0/2813360178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a815cb60 msgr2=0x7f04a815cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.289+0000 7f04ade00640 1 --2- 172.21.15.67:0/2813360178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a815cb60 0x7f04a815cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.289+0000 7f04aca00640 1 --2- 172.21.15.67:0/2813360178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a815cb60 0x7f04a815cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.289+0000 7f04aca00640 1 -- 172.21.15.67:0/2813360178 >> v1:172.21.15.67:6789/0 conn(0x7f04a8153510 legacy=0x7f04a80ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.289+0000 7f04aca00640 1 -- 172.21.15.67:0/2813360178 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a80b3f90 con 0x7f04a80cd2a0 2024-05-30T16:52:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.289+0000 7f04ade00640 1 --2- 172.21.15.67:0/2813360178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a815cb60 0x7f04a815cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.290+0000 7f04aca00640 1 --2- 172.21.15.67:0/2813360178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a80cd2a0 0x7f04a80b4d50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f04980097a0 tx=0x7f049802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd4691dd1332a1c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.290+0000 7f04a7e00640 1 -- 172.21.15.67:0/2813360178 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498037070 con 0x7f04a80cd2a0 2024-05-30T16:52:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.290+0000 7f04a7e00640 1 -- 172.21.15.67:0/2813360178 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049802ae50 con 0x7f04a80cd2a0 2024-05-30T16:52:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.290+0000 7f04a7e00640 1 -- 172.21.15.67:0/2813360178 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498032660 con 0x7f04a80cd2a0 2024-05-30T16:52:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.291+0000 7f04ae800640 1 -- 172.21.15.67:0/2813360178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a80cd2a0 msgr2=0x7f04a80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.291+0000 7f04ae800640 1 --2- 172.21.15.67:0/2813360178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a80cd2a0 0x7f04a80b4d50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f04980097a0 tx=0x7f049802a100 comp rx=0 tx=0).stop 2024-05-30T16:52:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.291+0000 7f04ae800640 1 -- 172.21.15.67:0/2813360178 shutdown_connections 2024-05-30T16:52:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.291+0000 7f04ae800640 1 --2- 172.21.15.67:0/2813360178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a815cb60 0x7f04a815cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.291+0000 7f04ae800640 1 --2- 172.21.15.67:0/2813360178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a80cd2a0 0x7f04a80b4d50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:15.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.291+0000 7f04ae800640 1 -- 172.21.15.67:0/2813360178 >> 172.21.15.67:0/2813360178 conn(0x7f04a806b900 msgr2=0x7f04a806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:15.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.291+0000 7f04ae800640 1 -- 172.21.15.67:0/2813360178 shutdown_connections 2024-05-30T16:52:15.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.291+0000 7f04ae800640 1 -- 172.21.15.67:0/2813360178 wait complete. 2024-05-30T16:52:15.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.293+0000 7f04ae800640 1 Processor -- start 2024-05-30T16:52:15.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.293+0000 7f04ae800640 1 -- start start 2024-05-30T16:52:15.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04ae800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a80cd2a0 0x7f04a80727e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04ae800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a8153510 0x7f04a80cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04ae800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a815cb60 0x7f04a815edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04ae800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f04a815d4f0 con 0x7f04a8153510 2024-05-30T16:52:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04ae800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04a80b4550 con 0x7f04a815cb60 2024-05-30T16:52:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04ae800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f04a815d670 con 0x7f04a80cd2a0 2024-05-30T16:52:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04aca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a8153510 0x7f04a80cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04aca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a8153510 0x7f04a80cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38454/0 (socket says 172.21.15.67:38454) 2024-05-30T16:52:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04aca00640 1 -- 172.21.15.67:0/2608621625 learned_addr learned my addr 172.21.15.67:0/2608621625 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:15.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04aca00640 1 -- 172.21.15.67:0/2608621625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a80cd2a0 msgr2=0x7f04a80727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:15.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04aca00640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a80cd2a0 0x7f04a80727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:15.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04aca00640 1 -- 172.21.15.67:0/2608621625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a815cb60 msgr2=0x7f04a815edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:15.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04aca00640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a815cb60 0x7f04a815edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:15.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.294+0000 7f04aca00640 1 -- 172.21.15.67:0/2608621625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a8151900 con 0x7f04a8153510 2024-05-30T16:52:15.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.295+0000 7f04aca00640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a8153510 0x7f04a80cdd50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f04980097a0 tx=0x7f04980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:15.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.295+0000 7f04a5600640 1 -- 172.21.15.67:0/2608621625 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498037040 con 0x7f04a8153510 2024-05-30T16:52:15.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.295+0000 7f04a5600640 1 -- 172.21.15.67:0/2608621625 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0498004250 con 0x7f04a8153510 2024-05-30T16:52:15.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.295+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a80cebe0 con 0x7f04a8153510 2024-05-30T16:52:15.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.295+0000 7f04a5600640 1 -- 172.21.15.67:0/2608621625 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498003610 con 0x7f04a8153510 2024-05-30T16:52:15.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.295+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a8151620 con 0x7f04a8153510 2024-05-30T16:52:15.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.296+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a80689e0 con 0x7f04a8153510 2024-05-30T16:52:15.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.297+0000 7f04a5600640 1 -- 172.21.15.67:0/2608621625 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f049802ad10 con 0x7f04a8153510 2024-05-30T16:52:15.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.299+0000 7f04a5600640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f048c06d0d0 0x7f048c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:15.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.299+0000 7f04a5600640 1 -- 172.21.15.67:0/2608621625 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5881+0+0 (secure 0 0 0) 0x7f04980adb00 con 0x7f04a8153510 2024-05-30T16:52:15.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.299+0000 7f04ad400640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f048c06d0d0 0x7f048c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:15.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.300+0000 7f04ad400640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f048c06d0d0 0x7f048c06f590 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f049c007fd0 tx=0x7f049c007920 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:15.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.303+0000 7f04a5600640 1 -- 172.21.15.67:0/2608621625 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f049804c070 con 0x7f04a8153510 2024-05-30T16:52:15.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:15.545+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f04a80af150 con 0x7f04a8153510 2024-05-30T16:52:16.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.072+0000 7f04a5600640 1 -- 172.21.15.67:0/2608621625 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f049807aef0 con 0x7f04a8153510 2024-05-30T16:52:16.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.075+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f048c06d0d0 msgr2=0x7f048c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:16.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.075+0000 7f04ae800640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f048c06d0d0 0x7f048c06f590 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f049c007fd0 tx=0x7f049c007920 comp rx=0 tx=0).stop 2024-05-30T16:52:16.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.075+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a8153510 msgr2=0x7f04a80cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:16.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.075+0000 7f04ae800640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a8153510 0x7f04a80cdd50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f04980097a0 tx=0x7f04980046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.076+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 shutdown_connections 2024-05-30T16:52:16.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.076+0000 7f04ae800640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f048c06d0d0 0x7f048c06f590 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.076+0000 7f04ae800640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a815cb60 0x7f04a815edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.076+0000 7f04ae800640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a8153510 0x7f04a80cdd50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.076+0000 7f04ae800640 1 --2- 172.21.15.67:0/2608621625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04a80cd2a0 0x7f04a80727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.076+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 >> 172.21.15.67:0/2608621625 conn(0x7f04a806b900 msgr2=0x7f04a806bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:16.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.076+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 shutdown_connections 2024-05-30T16:52:16.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.076+0000 7f04ae800640 1 -- 172.21.15.67:0/2608621625 wait complete. 2024-05-30T16:52:16.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-30T16:52:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.318+0000 7f1a74200640 1 Processor -- start 2024-05-30T16:52:16.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.318+0000 7f1a74200640 1 -- start start 2024-05-30T16:52:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.320+0000 7f1a74200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06edf0 0x7f1a6c13fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.320+0000 7f1a74200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1a6c06f250 0x7f1a6c140330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:16.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.320+0000 7f1a74200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a6c0d0020 con 0x7f1a6c0cf2f0 2024-05-30T16:52:16.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.320+0000 7f1a74200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1a6c06dd50 con 0x7f1a6c06f250 2024-05-30T16:52:16.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.320+0000 7f1a74200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6c06e030 con 0x7f1a6c06edf0 2024-05-30T16:52:16.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.320+0000 7f1a72200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1a6c06f250 0x7f1a6c140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.320+0000 7f1a72200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1a6c06f250 0x7f1a6c140330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38462/0 (socket says 172.21.15.67:38462) 2024-05-30T16:52:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.320+0000 7f1a72200640 1 -- 172.21.15.67:0/2919295601 learned_addr learned my addr 172.21.15.67:0/2919295601 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.321+0000 7f1a70e00640 1 --2- 172.21.15.67:0/2919295601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06edf0 0x7f1a6c13fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.321+0000 7f1a72200640 1 -- 172.21.15.67:0/2919295601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06edf0 msgr2=0x7f1a6c13fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.321+0000 7f1a72200640 1 --2- 172.21.15.67:0/2919295601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06edf0 0x7f1a6c13fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.321+0000 7f1a72200640 1 -- 172.21.15.67:0/2919295601 >> v1:172.21.15.169:6789/0 conn(0x7f1a6c0cf2f0 legacy=0x7f1a6c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.321+0000 7f1a72200640 1 -- 172.21.15.67:0/2919295601 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c06e310 con 0x7f1a6c06f250 2024-05-30T16:52:16.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.321+0000 7f1a70e00640 1 --2- 172.21.15.67:0/2919295601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06edf0 0x7f1a6c13fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:16.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.322+0000 7f1a72200640 1 --2- 172.21.15.67:0/2919295601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06f250 0x7f1a6c140330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009870 tx=0x7f1a6402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb312f5a277ceaa server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:16.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.322+0000 7f1a6be00640 1 -- 172.21.15.67:0/2919295601 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64037070 con 0x7f1a6c06f250 2024-05-30T16:52:16.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.322+0000 7f1a6be00640 1 -- 172.21.15.67:0/2919295601 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a64032440 con 0x7f1a6c06f250 2024-05-30T16:52:16.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.322+0000 7f1a6be00640 1 -- 172.21.15.67:0/2919295601 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64032810 con 0x7f1a6c06f250 2024-05-30T16:52:16.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.323+0000 7f1a74200640 1 -- 172.21.15.67:0/2919295601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06f250 msgr2=0x7f1a6c140330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:16.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.323+0000 7f1a74200640 1 --2- 172.21.15.67:0/2919295601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06f250 0x7f1a6c140330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009870 tx=0x7f1a6402a180 comp rx=0 tx=0).stop 2024-05-30T16:52:16.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.323+0000 7f1a74200640 1 -- 172.21.15.67:0/2919295601 shutdown_connections 2024-05-30T16:52:16.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.323+0000 7f1a74200640 1 --2- 172.21.15.67:0/2919295601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06f250 0x7f1a6c140330 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.323+0000 7f1a74200640 1 --2- 172.21.15.67:0/2919295601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06edf0 0x7f1a6c13fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.323+0000 7f1a74200640 1 -- 172.21.15.67:0/2919295601 >> 172.21.15.67:0/2919295601 conn(0x7f1a6c0b1e50 msgr2=0x7f1a6c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:16.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.323+0000 7f1a74200640 1 -- 172.21.15.67:0/2919295601 shutdown_connections 2024-05-30T16:52:16.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.323+0000 7f1a74200640 1 -- 172.21.15.67:0/2919295601 wait complete. 2024-05-30T16:52:16.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.325+0000 7f1a74200640 1 Processor -- start 2024-05-30T16:52:16.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.325+0000 7f1a74200640 1 -- start start 2024-05-30T16:52:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.326+0000 7f1a74200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06edf0 0x7f1a6c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.326+0000 7f1a74200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06f250 0x7f1a6c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.326+0000 7f1a74200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0cf2f0 0x7f1a6c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.326+0000 7f1a74200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c146020 con 0x7f1a6c06edf0 2024-05-30T16:52:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.326+0000 7f1a74200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c06e8d0 con 0x7f1a6c0cf2f0 2024-05-30T16:52:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.326+0000 7f1a74200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6c1461a0 con 0x7f1a6c06f250 2024-05-30T16:52:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.326+0000 7f1a71800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06edf0 0x7f1a6c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.326+0000 7f1a71800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06edf0 0x7f1a6c0cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38464/0 (socket says 172.21.15.67:38464) 2024-05-30T16:52:16.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.326+0000 7f1a71800640 1 -- 172.21.15.67:0/162192801 learned_addr learned my addr 172.21.15.67:0/162192801 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.327+0000 7f1a72200640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0cf2f0 0x7f1a6c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.327+0000 7f1a70e00640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06f250 0x7f1a6c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.327+0000 7f1a71800640 1 -- 172.21.15.67:0/162192801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06f250 msgr2=0x7f1a6c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.327+0000 7f1a71800640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06f250 0x7f1a6c15cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.327+0000 7f1a71800640 1 -- 172.21.15.67:0/162192801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0cf2f0 msgr2=0x7f1a6c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.327+0000 7f1a71800640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0cf2f0 0x7f1a6c0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.327+0000 7f1a71800640 1 -- 172.21.15.67:0/162192801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c0caf10 con 0x7f1a6c06edf0 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.327+0000 7f1a72200640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0cf2f0 0x7f1a6c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.327+0000 7f1a71800640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06edf0 0x7f1a6c0cbf40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c7c0 tx=0x7f1a6000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.328+0000 7f1a69600640 1 -- 172.21.15.67:0/162192801 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60016020 con 0x7f1a6c06edf0 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.328+0000 7f1a69600640 1 -- 172.21.15.67:0/162192801 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a6000aa50 con 0x7f1a6c06edf0 2024-05-30T16:52:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.328+0000 7f1a69600640 1 -- 172.21.15.67:0/162192801 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60012420 con 0x7f1a6c06edf0 2024-05-30T16:52:16.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.328+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6c0cb180 con 0x7f1a6c06edf0 2024-05-30T16:52:16.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.328+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6c0cb740 con 0x7f1a6c06edf0 2024-05-30T16:52:16.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.329+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6c145dd0 con 0x7f1a6c06edf0 2024-05-30T16:52:16.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.330+0000 7f1a69600640 1 -- 172.21.15.67:0/162192801 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f1a60019020 con 0x7f1a6c06edf0 2024-05-30T16:52:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.331+0000 7f1a69600640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4c06d0d0 0x7f1a4c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.331+0000 7f1a69600640 1 -- 172.21.15.67:0/162192801 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5881+0+0 (secure 0 0 0) 0x7f1a6008cc40 con 0x7f1a6c06edf0 2024-05-30T16:52:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.332+0000 7f1a70e00640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4c06d0d0 0x7f1a4c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.332+0000 7f1a70e00640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4c06d0d0 0x7f1a4c06f590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c15ea60 tx=0x7f1a58008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.336+0000 7f1a69600640 1 -- 172.21.15.67:0/162192801 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a6c145dd0 con 0x7f1a6c06edf0 2024-05-30T16:52:16.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:16.539+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1a6c0689e0 con 0x7f1a6c06edf0 2024-05-30T16:52:17.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.076+0000 7f1a69600640 1 -- 172.21.15.67:0/162192801 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) ==== 181+0+0 (secure 0 0 0) 0x7f1a60020070 con 0x7f1a6c06edf0 2024-05-30T16:52:17.078 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-05-30T16:52:17.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.079+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4c06d0d0 msgr2=0x7f1a4c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:17.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.079+0000 7f1a74200640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4c06d0d0 0x7f1a4c06f590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c15ea60 tx=0x7f1a58008040 comp rx=0 tx=0).stop 2024-05-30T16:52:17.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.079+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06edf0 msgr2=0x7f1a6c0cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:17.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.079+0000 7f1a74200640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06edf0 0x7f1a6c0cbf40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c7c0 tx=0x7f1a6000cc80 comp rx=0 tx=0).stop 2024-05-30T16:52:17.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.080+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 shutdown_connections 2024-05-30T16:52:17.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.080+0000 7f1a74200640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4c06d0d0 0x7f1a4c06f590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.080+0000 7f1a74200640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a6c0cf2f0 0x7f1a6c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.080+0000 7f1a74200640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6c06f250 0x7f1a6c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.080+0000 7f1a74200640 1 --2- 172.21.15.67:0/162192801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6c06edf0 0x7f1a6c0cbf40 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.080+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 >> 172.21.15.67:0/162192801 conn(0x7f1a6c0b1e50 msgr2=0x7f1a6c0ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:17.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.080+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 shutdown_connections 2024-05-30T16:52:17.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.080+0000 7f1a74200640 1 -- 172.21.15.67:0/162192801 wait complete. 2024-05-30T16:52:17.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-30T16:52:17.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.322+0000 7f2278200640 1 Processor -- start 2024-05-30T16:52:17.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.323+0000 7f2278200640 1 -- start start 2024-05-30T16:52:17.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.324+0000 7f2278200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22700cf2f0 0x7f22700cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:17.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.324+0000 7f2278200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2270140500 0x7f22700ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:17.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.324+0000 7f2278200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f227006dd50 con 0x7f227006edf0 2024-05-30T16:52:17.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.324+0000 7f2278200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f227006e030 con 0x7f22700cf2f0 2024-05-30T16:52:17.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.324+0000 7f2278200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f227006e310 con 0x7f2270140500 2024-05-30T16:52:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.325+0000 7f2275800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22700cf2f0 0x7f22700cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.325+0000 7f2275800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22700cf2f0 0x7f22700cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38480/0 (socket says 172.21.15.67:38480) 2024-05-30T16:52:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.325+0000 7f2275800640 1 -- 172.21.15.67:0/545138848 learned_addr learned my addr 172.21.15.67:0/545138848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.325+0000 7f2275800640 1 -- 172.21.15.67:0/545138848 >> v2:172.21.15.169:3300/0 conn(0x7f2270140500 msgr2=0x7f22700ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.325+0000 7f2275800640 1 --2- 172.21.15.67:0/545138848 >> v2:172.21.15.169:3300/0 conn(0x7f2270140500 0x7f22700ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.325+0000 7f2275800640 1 -- 172.21.15.67:0/545138848 >> v1:172.21.15.169:6789/0 conn(0x7f227006edf0 legacy=0x7f227013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.325+0000 7f2275800640 1 -- 172.21.15.67:0/545138848 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227006e5f0 con 0x7f22700cf2f0 2024-05-30T16:52:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.326+0000 7f2275800640 1 --2- 172.21.15.67:0/545138848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 0x7f22700cf910 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f22700afd50 tx=0x7f226402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51342c9d2bea9646 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.327+0000 7f226fe00640 1 -- 172.21.15.67:0/545138848 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264037070 con 0x7f22700cf2f0 2024-05-30T16:52:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.327+0000 7f226fe00640 1 -- 172.21.15.67:0/545138848 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2264032440 con 0x7f22700cf2f0 2024-05-30T16:52:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.327+0000 7f226fe00640 1 -- 172.21.15.67:0/545138848 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264032810 con 0x7f22700cf2f0 2024-05-30T16:52:17.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.327+0000 7f2278200640 1 -- 172.21.15.67:0/545138848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 msgr2=0x7f22700cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:17.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.327+0000 7f2278200640 1 --2- 172.21.15.67:0/545138848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 0x7f22700cf910 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f22700afd50 tx=0x7f226402a1a0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.328+0000 7f2278200640 1 -- 172.21.15.67:0/545138848 shutdown_connections 2024-05-30T16:52:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.328+0000 7f2278200640 1 --2- 172.21.15.67:0/545138848 >> v2:172.21.15.169:3300/0 conn(0x7f2270140500 0x7f22700ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.328+0000 7f2278200640 1 --2- 172.21.15.67:0/545138848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 0x7f22700cf910 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.328+0000 7f2278200640 1 -- 172.21.15.67:0/545138848 >> 172.21.15.67:0/545138848 conn(0x7f22700b1e50 msgr2=0x7f22700b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.328+0000 7f2278200640 1 -- 172.21.15.67:0/545138848 shutdown_connections 2024-05-30T16:52:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.328+0000 7f2278200640 1 -- 172.21.15.67:0/545138848 wait complete. 2024-05-30T16:52:17.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.330+0000 7f2278200640 1 Processor -- start 2024-05-30T16:52:17.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.330+0000 7f2278200640 1 -- start start 2024-05-30T16:52:17.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2278200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f227006edf0 0x7f2270099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:17.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2278200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 0x7f22701f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2278200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227015f530 0x7f22701f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2278200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22701461a0 con 0x7f22700cf2f0 2024-05-30T16:52:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2278200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2270146020 con 0x7f227015f530 2024-05-30T16:52:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2278200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22700cfe50 con 0x7f227006edf0 2024-05-30T16:52:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2274e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 0x7f22701f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2274e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 0x7f22701f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38488/0 (socket says 172.21.15.67:38488) 2024-05-30T16:52:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2274e00640 1 -- 172.21.15.67:0/3709434538 learned_addr learned my addr 172.21.15.67:0/3709434538 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:17.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2275800640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f227006edf0 0x7f2270099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:17.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2276200640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227015f530 0x7f22701f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:17.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2274e00640 1 -- 172.21.15.67:0/3709434538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f227006edf0 msgr2=0x7f2270099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2274e00640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f227006edf0 0x7f2270099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2274e00640 1 -- 172.21.15.67:0/3709434538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227015f530 msgr2=0x7f22701f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.332+0000 7f2274e00640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227015f530 0x7f22701f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.333+0000 7f2274e00640 1 -- 172.21.15.67:0/3709434538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22701f3c40 con 0x7f22700cf2f0 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.333+0000 7f2276200640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227015f530 0x7f22701f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.333+0000 7f2274e00640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 0x7f22701f2de0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f225c00b410 tx=0x7f225c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.333+0000 7f226d600640 1 -- 172.21.15.67:0/3709434538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c015020 con 0x7f22700cf2f0 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.333+0000 7f226d600640 1 -- 172.21.15.67:0/3709434538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225c011830 con 0x7f22700cf2f0 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.333+0000 7f226d600640 1 -- 172.21.15.67:0/3709434538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c011c20 con 0x7f22700cf2f0 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.333+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2270140a20 con 0x7f22700cf2f0 2024-05-30T16:52:17.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.333+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2270140fe0 con 0x7f22700cf2f0 2024-05-30T16:52:17.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.335+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22700689e0 con 0x7f22700cf2f0 2024-05-30T16:52:17.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.336+0000 7f226d600640 1 -- 172.21.15.67:0/3709434538 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f225c018020 con 0x7f22700cf2f0 2024-05-30T16:52:17.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.336+0000 7f226d600640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f225006d0d0 0x7f225006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:17.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.337+0000 7f226d600640 1 -- 172.21.15.67:0/3709434538 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5443+0+0 (secure 0 0 0) 0x7f225c08ccc0 con 0x7f22700cf2f0 2024-05-30T16:52:17.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.337+0000 7f2275800640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f225006d0d0 0x7f225006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:17.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.338+0000 7f2275800640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f225006d0d0 0x7f225006f590 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2264034000 tx=0x7f226402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:17.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.343+0000 7f226d600640 1 -- 172.21.15.67:0/3709434538 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f225c01f020 con 0x7f22700cf2f0 2024-05-30T16:52:17.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:17.554+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22701411b0 con 0x7f22700cf2f0 2024-05-30T16:52:18.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.094+0000 7f226d600640 1 -- 172.21.15.67:0/3709434538 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) ==== 169+0+0 (secure 0 0 0) 0x7f225c0172c0 con 0x7f22700cf2f0 2024-05-30T16:52:18.096 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f225006d0d0 msgr2=0x7f225006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f225006d0d0 0x7f225006f590 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2264034000 tx=0x7f226402e000 comp rx=0 tx=0).stop 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 msgr2=0x7f22701f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 0x7f22701f2de0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f225c00b410 tx=0x7f225c00b8d0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 shutdown_connections 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f225006d0d0 0x7f225006f590 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227015f530 0x7f22701f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22700cf2f0 0x7f22701f2de0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 --2- 172.21.15.67:0/3709434538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f227006edf0 0x7f2270099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 >> 172.21.15.67:0/3709434538 conn(0x7f22700b1e50 msgr2=0x7f227015bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 shutdown_connections 2024-05-30T16:52:18.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.098+0000 7f2278200640 1 -- 172.21.15.67:0/3709434538 wait complete. 2024-05-30T16:52:18.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-30T16:52:18.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.339+0000 7f325f000640 1 Processor -- start 2024-05-30T16:52:18.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.339+0000 7f325f000640 1 -- start start 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.340+0000 7f325f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f325815d270 0x7f32580b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.340+0000 7f325f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f325815d680 con 0x7f32580cd2a0 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.340+0000 7f325f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f325815d900 con 0x7f3258153510 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.340+0000 7f325f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f32581538f0 con 0x7f325815d270 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.340+0000 7f325d200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f32580cd2a0 0x7f325815cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48430/0 (socket says 172.21.15.67:48430) 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.340+0000 7f325d200640 1 -- 172.21.15.67:0/1074107404 learned_addr learned my addr 172.21.15.67:0/1074107404 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.341+0000 7f325e600640 1 --2- 172.21.15.67:0/1074107404 >> v2:172.21.15.67:3300/0 conn(0x7f325815d270 0x7f32580b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.341+0000 7f3257e00640 1 -- 172.21.15.67:0/1074107404 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2526567481 0 0) 0x7f3244001110 con 0x7f32580cd2a0 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.341+0000 7f3257e00640 1 -- 172.21.15.67:0/1074107404 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f323c003520 con 0x7f32580cd2a0 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.341+0000 7f325e600640 1 -- 172.21.15.67:0/1074107404 >> v1:172.21.15.169:6789/0 conn(0x7f3258153510 legacy=0x7f32580ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.341+0000 7f325e600640 1 -- 172.21.15.67:0/1074107404 >> v1:172.21.15.67:6789/0 conn(0x7f32580cd2a0 legacy=0x7f325815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.341+0000 7f325e600640 1 -- 172.21.15.67:0/1074107404 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32580b3f90 con 0x7f325815d270 2024-05-30T16:52:18.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.342+0000 7f325e600640 1 --2- 172.21.15.67:0/1074107404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f325815d270 0x7f32580b4d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f32500097a0 tx=0x7f325002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f0d10952971110 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:18.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.343+0000 7f3257e00640 1 -- 172.21.15.67:0/1074107404 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250037070 con 0x7f325815d270 2024-05-30T16:52:18.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.343+0000 7f3257e00640 1 -- 172.21.15.67:0/1074107404 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3250032440 con 0x7f325815d270 2024-05-30T16:52:18.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.343+0000 7f3257e00640 1 -- 172.21.15.67:0/1074107404 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32500327f0 con 0x7f325815d270 2024-05-30T16:52:18.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.343+0000 7f325f000640 1 -- 172.21.15.67:0/1074107404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f325815d270 msgr2=0x7f32580b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.343+0000 7f325f000640 1 --2- 172.21.15.67:0/1074107404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f325815d270 0x7f32580b4d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f32500097a0 tx=0x7f325002a1a0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.343+0000 7f325f000640 1 -- 172.21.15.67:0/1074107404 shutdown_connections 2024-05-30T16:52:18.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.343+0000 7f325f000640 1 --2- 172.21.15.67:0/1074107404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f325815d270 0x7f32580b4d50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.343+0000 7f325f000640 1 -- 172.21.15.67:0/1074107404 >> 172.21.15.67:0/1074107404 conn(0x7f325806b900 msgr2=0x7f325806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:18.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.344+0000 7f325f000640 1 -- 172.21.15.67:0/1074107404 shutdown_connections 2024-05-30T16:52:18.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.344+0000 7f325f000640 1 -- 172.21.15.67:0/1074107404 wait complete. 2024-05-30T16:52:18.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.345+0000 7f325f000640 1 Processor -- start 2024-05-30T16:52:18.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.345+0000 7f325f000640 1 -- start start 2024-05-30T16:52:18.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.346+0000 7f325f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32580cd2a0 0x7f32580cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.346+0000 7f325f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3258153510 0x7f32581f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.346+0000 7f325f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f325815d270 0x7f325815db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f32580b4550 con 0x7f3258153510 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32580b48c0 con 0x7f325815d270 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f325806e940 con 0x7f32580cd2a0 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3258153510 0x7f32581f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3258153510 0x7f32581f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38502/0 (socket says 172.21.15.67:38502) 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325d200640 1 -- 172.21.15.67:0/4156869879 learned_addr learned my addr 172.21.15.67:0/4156869879 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325dc00640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32580cd2a0 0x7f32580cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325e600640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f325815d270 0x7f325815db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325d200640 1 -- 172.21.15.67:0/4156869879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32580cd2a0 msgr2=0x7f32580cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325d200640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32580cd2a0 0x7f32580cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325d200640 1 -- 172.21.15.67:0/4156869879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f325815d270 msgr2=0x7f325815db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325d200640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f325815d270 0x7f325815db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.347+0000 7f325d200640 1 -- 172.21.15.67:0/4156869879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32581f7da0 con 0x7f3258153510 2024-05-30T16:52:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.348+0000 7f325d200640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3258153510 0x7f32581f42f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f324400c730 tx=0x7f324400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.348+0000 7f3255600640 1 -- 172.21.15.67:0/4156869879 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244017070 con 0x7f3258153510 2024-05-30T16:52:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.348+0000 7f3255600640 1 -- 172.21.15.67:0/4156869879 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3244005ab0 con 0x7f3258153510 2024-05-30T16:52:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.348+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32580ce330 con 0x7f3258153510 2024-05-30T16:52:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.348+0000 7f3255600640 1 -- 172.21.15.67:0/4156869879 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244007070 con 0x7f3258153510 2024-05-30T16:52:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.348+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32581f8340 con 0x7f3258153510 2024-05-30T16:52:18.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.349+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32580689e0 con 0x7f3258153510 2024-05-30T16:52:18.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.350+0000 7f3255600640 1 -- 172.21.15.67:0/4156869879 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f324401a070 con 0x7f3258153510 2024-05-30T16:52:18.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.351+0000 7f3255600640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f323806d100 0x7f323806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.351+0000 7f3255600640 1 -- 172.21.15.67:0/4156869879 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f324408be90 con 0x7f3258153510 2024-05-30T16:52:18.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.352+0000 7f325dc00640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f323806d100 0x7f323806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.352+0000 7f325dc00640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f323806d100 0x7f323806f5c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3248001d40 tx=0x7f3248009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:18.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.355+0000 7f3255600640 1 -- 172.21.15.67:0/4156869879 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3244091020 con 0x7f3258153510 2024-05-30T16:52:18.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.565+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f32580af380 con 0x7f3258153510 2024-05-30T16:52:18.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.572+0000 7f3255600640 1 -- 172.21.15.67:0/4156869879 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f325815d680 con 0x7f3258153510 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f323806d100 msgr2=0x7f323806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f323806d100 0x7f323806f5c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3248001d40 tx=0x7f3248009040 comp rx=0 tx=0).stop 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3258153510 msgr2=0x7f32581f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3258153510 0x7f32581f42f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f324400c730 tx=0x7f324400cbf0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 shutdown_connections 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f323806d100 0x7f323806f5c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f325815d270 0x7f325815db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3258153510 0x7f32581f42f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 --2- 172.21.15.67:0/4156869879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32580cd2a0 0x7f32580cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 >> 172.21.15.67:0/4156869879 conn(0x7f325806b900 msgr2=0x7f3258151180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:18.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 shutdown_connections 2024-05-30T16:52:18.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.575+0000 7f325f000640 1 -- 172.21.15.67:0/4156869879 wait complete. 2024-05-30T16:52:18.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones' 2024-05-30T16:52:18.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.792+0000 7f45e0c00640 1 Processor -- start 2024-05-30T16:52:18.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.792+0000 7f45e0c00640 1 -- start start 2024-05-30T16:52:18.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.794+0000 7f45e0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 0x7f45dc071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.794+0000 7f45e0c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f45dc072100 0x7f45dc0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.794+0000 7f45e0c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45dc06f590 con 0x7f45dc162940 2024-05-30T16:52:18.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.794+0000 7f45e0c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f45dc072550 con 0x7f45dc072100 2024-05-30T16:52:18.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.794+0000 7f45e0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc06f870 con 0x7f45dc162d20 2024-05-30T16:52:18.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.794+0000 7f45dbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f45dc072100 0x7f45dc0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.794+0000 7f45dbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f45dc072100 0x7f45dc0adbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38514/0 (socket says 172.21.15.67:38514) 2024-05-30T16:52:18.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.794+0000 7f45dbe00640 1 -- 172.21.15.67:0/2548057413 learned_addr learned my addr 172.21.15.67:0/2548057413 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:18.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.794+0000 7f45daa00640 1 --2- 172.21.15.67:0/2548057413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 0x7f45dc071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.795+0000 7f45dbe00640 1 -- 172.21.15.67:0/2548057413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 msgr2=0x7f45dc071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:18.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.795+0000 7f45dbe00640 1 --2- 172.21.15.67:0/2548057413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 0x7f45dc071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.795+0000 7f45dbe00640 1 -- 172.21.15.67:0/2548057413 >> v1:172.21.15.169:6789/0 conn(0x7f45dc162940 legacy=0x7f45dc06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:18.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.795+0000 7f45dbe00640 1 -- 172.21.15.67:0/2548057413 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc0ad3d0 con 0x7f45dc072100 2024-05-30T16:52:18.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.796+0000 7f45dbe00640 1 --2- 172.21.15.67:0/2548057413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc072100 0x7f45dc0adbb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009870 tx=0x7f45d402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dfa70a87246363 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:18.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.796+0000 7f45da000640 1 -- 172.21.15.67:0/2548057413 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4037070 con 0x7f45dc072100 2024-05-30T16:52:18.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.796+0000 7f45da000640 1 -- 172.21.15.67:0/2548057413 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d402ae10 con 0x7f45dc072100 2024-05-30T16:52:18.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.796+0000 7f45da000640 1 -- 172.21.15.67:0/2548057413 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4032660 con 0x7f45dc072100 2024-05-30T16:52:18.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.797+0000 7f45e0c00640 1 -- 172.21.15.67:0/2548057413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc072100 msgr2=0x7f45dc0adbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.797+0000 7f45e0c00640 1 --2- 172.21.15.67:0/2548057413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc072100 0x7f45dc0adbb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009870 tx=0x7f45d402a100 comp rx=0 tx=0).stop 2024-05-30T16:52:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.797+0000 7f45e0c00640 1 -- 172.21.15.67:0/2548057413 shutdown_connections 2024-05-30T16:52:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.797+0000 7f45e0c00640 1 --2- 172.21.15.67:0/2548057413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc072100 0x7f45dc0adbb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.797+0000 7f45e0c00640 1 --2- 172.21.15.67:0/2548057413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 0x7f45dc071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.797+0000 7f45e0c00640 1 -- 172.21.15.67:0/2548057413 >> 172.21.15.67:0/2548057413 conn(0x7f45dc0b1fc0 msgr2=0x7f45dc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.797+0000 7f45e0c00640 1 -- 172.21.15.67:0/2548057413 shutdown_connections 2024-05-30T16:52:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.797+0000 7f45e0c00640 1 -- 172.21.15.67:0/2548057413 wait complete. 2024-05-30T16:52:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.799+0000 7f45e0c00640 1 Processor -- start 2024-05-30T16:52:18.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.799+0000 7f45e0c00640 1 -- start start 2024-05-30T16:52:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45e0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45dc072100 0x7f45dc0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45e0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc162940 0x7f45dc09a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45e0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 0x7f45dc09a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45e0c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc06df60 con 0x7f45dc162940 2024-05-30T16:52:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45e0c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc06dde0 con 0x7f45dc072100 2024-05-30T16:52:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45e0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc0ae0f0 con 0x7f45dc162d20 2024-05-30T16:52:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45daa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc162940 0x7f45dc09a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45daa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc162940 0x7f45dc09a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38516/0 (socket says 172.21.15.67:38516) 2024-05-30T16:52:18.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45daa00640 1 -- 172.21.15.67:0/4268501526 learned_addr learned my addr 172.21.15.67:0/4268501526 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:18.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45dbe00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 0x7f45dc09a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45daa00640 1 -- 172.21.15.67:0/4268501526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 msgr2=0x7f45dc09a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.800+0000 7f45db400640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45dc072100 0x7f45dc0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45daa00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 0x7f45dc09a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45daa00640 1 -- 172.21.15.67:0/4268501526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45dc072100 msgr2=0x7f45dc0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45daa00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45dc072100 0x7f45dc0cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45daa00640 1 -- 172.21.15.67:0/4268501526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc0cd4c0 con 0x7f45dc162940 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45db400640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45dc072100 0x7f45dc0cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45daa00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc162940 0x7f45dc09a3b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f45cc007e30 tx=0x7f45cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45d3400640 1 -- 172.21.15.67:0/4268501526 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc014620 con 0x7f45dc162940 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45d3400640 1 -- 172.21.15.67:0/4268501526 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc019070 con 0x7f45dc162940 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45d3400640 1 -- 172.21.15.67:0/4268501526 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc014a00 con 0x7f45dc162940 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc1650f0 con 0x7f45dc162940 2024-05-30T16:52:18.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.801+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc1656b0 con 0x7f45dc162940 2024-05-30T16:52:18.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.803+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45dc0689e0 con 0x7f45dc162940 2024-05-30T16:52:18.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.803+0000 7f45d3400640 1 -- 172.21.15.67:0/4268501526 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f45cc014ba0 con 0x7f45dc162940 2024-05-30T16:52:18.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.804+0000 7f45d3400640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f45b806d120 0x7f45b806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:18.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.804+0000 7f45d3400640 1 -- 172.21.15.67:0/4268501526 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f45cc08d380 con 0x7f45dc162940 2024-05-30T16:52:18.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.805+0000 7f45db400640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f45b806d120 0x7f45b806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:18.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.806+0000 7f45db400640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f45b806d120 0x7f45b806f5e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f45c4003eb0 tx=0x7f45c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:18.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:18.809+0000 7f45d3400640 1 -- 172.21.15.67:0/4268501526 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45cc05aaf0 con 0x7f45dc162940 2024-05-30T16:52:19.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:19.094+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]} v 0) -- 0x7f45dc165a50 con 0x7f45dc162940 2024-05-30T16:52:20.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.099+0000 7f45d3400640 1 -- 172.21.15.67:0/4268501526 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]}]=0 v344) ==== 162+0+0 (secure 0 0 0) 0x7f45cc05a3f0 con 0x7f45dc162940 2024-05-30T16:52:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f45b806d120 msgr2=0x7f45b806f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f45b806d120 0x7f45b806f5e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f45c4003eb0 tx=0x7f45c4009040 comp rx=0 tx=0).stop 2024-05-30T16:52:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc162940 msgr2=0x7f45dc09a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc162940 0x7f45dc09a3b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f45cc007e30 tx=0x7f45cc00e4b0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 shutdown_connections 2024-05-30T16:52:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f45b806d120 0x7f45b806f5e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45dc162d20 0x7f45dc09a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45dc162940 0x7f45dc09a3b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 --2- 172.21.15.67:0/4268501526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45dc072100 0x7f45dc0cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 >> 172.21.15.67:0/4268501526 conn(0x7f45dc0b1fc0 msgr2=0x7f45dc0b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:20.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 shutdown_connections 2024-05-30T16:52:20.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.103+0000 7f45e0c00640 1 -- 172.21.15.67:0/4268501526 wait complete. 2024-05-30T16:52:20.126 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed when number of clone jobs is ... ok 2024-05-30T16:52:20.127 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed when number of clone jobs is 2024-05-30T16:52:20.127 INFO:tasks.cephfs_test_runner:test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T16:52:20.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:52:20.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.336+0000 7fc2e8c00640 1 Processor -- start 2024-05-30T16:52:20.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.337+0000 7fc2e8c00640 1 -- start start 2024-05-30T16:52:20.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.338+0000 7fc2e8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc2e406ee90 0x7fc2e406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.338+0000 7fc2e8c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2e415fbc0 con 0x7fc2e40b01e0 2024-05-30T16:52:20.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.338+0000 7fc2e8c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2e4070d50 con 0x7fc2e40707d0 2024-05-30T16:52:20.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.338+0000 7fc2e8c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc2e406c850 con 0x7fc2e406ee90 2024-05-30T16:52:20.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.339+0000 7fc2e2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc2e406ee90 0x7fc2e406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:20.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.339+0000 7fc2e1e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc2e40b01e0 0x7fc2e40b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48444/0 (socket says 172.21.15.67:48444) 2024-05-30T16:52:20.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.339+0000 7fc2e1e00640 1 -- 172.21.15.67:0/2827029833 learned_addr learned my addr 172.21.15.67:0/2827029833 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:20.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.339+0000 7fc2e2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc2e406ee90 0x7fc2e406c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38520/0 (socket says 172.21.15.67:38520) 2024-05-30T16:52:20.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.339+0000 7fc2e2800640 1 -- 172.21.15.67:0/2827029833 >> v1:172.21.15.169:6789/0 conn(0x7fc2e40707d0 legacy=0x7fc2e40d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:20.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.339+0000 7fc2e2800640 1 -- 172.21.15.67:0/2827029833 >> v1:172.21.15.67:6789/0 conn(0x7fc2e40b01e0 legacy=0x7fc2e40b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.339+0000 7fc2e2800640 1 -- 172.21.15.67:0/2827029833 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e40b6140 con 0x7fc2e406ee90 2024-05-30T16:52:20.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.339+0000 7fc2e0a00640 1 -- 172.21.15.67:0/2827029833 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3401952668 0 0) 0x7fc2e415fbc0 con 0x7fc2e40b01e0 2024-05-30T16:52:20.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.340+0000 7fc2e2800640 1 --2- 172.21.15.67:0/2827029833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 0x7fc2e406c310 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009d90 tx=0x7fc2dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2c542693e41d14 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:20.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.341+0000 7fc2e0a00640 1 -- 172.21.15.67:0/2827029833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc007740 con 0x7fc2e406ee90 2024-05-30T16:52:20.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.341+0000 7fc2e0a00640 1 -- 172.21.15.67:0/2827029833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc0078e0 con 0x7fc2e406ee90 2024-05-30T16:52:20.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.341+0000 7fc2e0a00640 1 -- 172.21.15.67:0/2827029833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc007cb0 con 0x7fc2e406ee90 2024-05-30T16:52:20.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.341+0000 7fc2e8c00640 1 -- 172.21.15.67:0/2827029833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 msgr2=0x7fc2e406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.341+0000 7fc2e8c00640 1 --2- 172.21.15.67:0/2827029833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 0x7fc2e406c310 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009d90 tx=0x7fc2dc02a6a0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.342+0000 7fc2e8c00640 1 -- 172.21.15.67:0/2827029833 shutdown_connections 2024-05-30T16:52:20.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.342+0000 7fc2e8c00640 1 --2- 172.21.15.67:0/2827029833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 0x7fc2e406c310 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.342+0000 7fc2e8c00640 1 -- 172.21.15.67:0/2827029833 >> 172.21.15.67:0/2827029833 conn(0x7fc2e409aa30 msgr2=0x7fc2e40b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:20.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.342+0000 7fc2e8c00640 1 -- 172.21.15.67:0/2827029833 shutdown_connections 2024-05-30T16:52:20.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.342+0000 7fc2e8c00640 1 -- 172.21.15.67:0/2827029833 wait complete. 2024-05-30T16:52:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.343+0000 7fc2e8c00640 1 Processor -- start 2024-05-30T16:52:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.344+0000 7fc2e8c00640 1 -- start start 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.344+0000 7fc2e8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 0x7fc2e414cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2e40707d0 0x7fc2e409b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2e40b01e0 0x7fc2e409cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e8c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e40d1b80 con 0x7fc2e406ee90 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e40b5df0 con 0x7fc2e40707d0 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e40d1d00 con 0x7fc2e40b01e0 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 0x7fc2e414cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 0x7fc2e414cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38532/0 (socket says 172.21.15.67:38532) 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e1e00640 1 -- 172.21.15.67:0/564536420 learned_addr learned my addr 172.21.15.67:0/564536420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e1e00640 1 -- 172.21.15.67:0/564536420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2e40b01e0 msgr2=0x7fc2e409cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e1e00640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2e40b01e0 0x7fc2e409cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e1e00640 1 -- 172.21.15.67:0/564536420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2e40707d0 msgr2=0x7fc2e409b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e1e00640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2e40707d0 0x7fc2e409b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.345+0000 7fc2e1e00640 1 -- 172.21.15.67:0/564536420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e409bed0 con 0x7fc2e406ee90 2024-05-30T16:52:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.346+0000 7fc2e1e00640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 0x7fc2e414cf30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400c430 tx=0x7fc2d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.346+0000 7fc2da000640 1 -- 172.21.15.67:0/564536420 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40074e0 con 0x7fc2e406ee90 2024-05-30T16:52:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.346+0000 7fc2da000640 1 -- 172.21.15.67:0/564536420 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d40047e0 con 0x7fc2e406ee90 2024-05-30T16:52:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.346+0000 7fc2da000640 1 -- 172.21.15.67:0/564536420 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4004bd0 con 0x7fc2e406ee90 2024-05-30T16:52:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.346+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e409c1b0 con 0x7fc2e406ee90 2024-05-30T16:52:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.346+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e409c770 con 0x7fc2e406ee90 2024-05-30T16:52:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.347+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e40689e0 con 0x7fc2e406ee90 2024-05-30T16:52:20.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.348+0000 7fc2da000640 1 -- 172.21.15.67:0/564536420 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7fc2d4011070 con 0x7fc2e406ee90 2024-05-30T16:52:20.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.349+0000 7fc2da000640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc2c006d0d0 0x7fc2c006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.349+0000 7fc2da000640 1 -- 172.21.15.67:0/564536420 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7fc2d408c7e0 con 0x7fc2e406ee90 2024-05-30T16:52:20.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.349+0000 7fc2e1400640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc2c006d0d0 0x7fc2c006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:20.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.350+0000 7fc2e1400640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc2c006d0d0 0x7fc2c006f590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc2e415d830 tx=0x7fc2cc0079b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:20.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.354+0000 7fc2da000640 1 -- 172.21.15.67:0/564536420 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2d400e280 con 0x7fc2e406ee90 2024-05-30T16:52:20.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.559+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2e4157770 con 0x7fc2e406ee90 2024-05-30T16:52:20.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.561+0000 7fc2da000640 1 -- 172.21.15.67:0/564536420 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+13051 (secure 0 0 0) 0x7fc2d4059ed0 con 0x7fc2e406ee90 2024-05-30T16:52:20.563 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:20.563 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":54,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:52:18.091704+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6837/3595714006":"2024-05-31T16:52:15.066970+0000","172.21.15.67:6836/3595714006":"2024-05-31T16:52:15.066970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:52:20.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc2c006d0d0 msgr2=0x7fc2c006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc2c006d0d0 0x7fc2c006f590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc2e415d830 tx=0x7fc2cc0079b0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 msgr2=0x7fc2e414cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 0x7fc2e414cf30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400c430 tx=0x7fc2d400c8f0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 shutdown_connections 2024-05-30T16:52:20.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc2c006d0d0 0x7fc2c006f590 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2e40b01e0 0x7fc2e409cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2e40707d0 0x7fc2e409b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 --2- 172.21.15.67:0/564536420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2e406ee90 0x7fc2e414cf30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.563+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 >> 172.21.15.67:0/564536420 conn(0x7fc2e409aa30 msgr2=0x7fc2e40aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:20.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.564+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 shutdown_connections 2024-05-30T16:52:20.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.564+0000 7fc2e8c00640 1 -- 172.21.15.67:0/564536420 wait complete. 2024-05-30T16:52:20.585 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-30T16:43:40.009725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-30T16:52:20.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-30T16:52:20.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.790+0000 7f0aca600640 1 Processor -- start 2024-05-30T16:52:20.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.790+0000 7f0aca600640 1 -- start start 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.791+0000 7f0aca600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0ac40d0640 0x7f0ac415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.791+0000 7f0aca600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0ac40d0a20 0x7f0ac40cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.791+0000 7f0aca600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac415cb10 con 0x7f0ac40b32c0 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.791+0000 7f0aca600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac40adbb0 con 0x7f0ac40d0a20 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.791+0000 7f0aca600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac415db10 con 0x7f0ac40d0640 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.791+0000 7f0ac2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0ac40d0a20 0x7f0ac40cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.791+0000 7f0ac2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0ac40d0a20 0x7f0ac40cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39580/0 (socket says 172.21.15.67:39580) 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.791+0000 7f0ac3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0ac40b32c0 0x7f0ac40b3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36514/0 (socket says 172.21.15.67:36514) 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.791+0000 7f0ac2a00640 1 -- 172.21.15.67:0/1108904841 learned_addr learned my addr 172.21.15.67:0/1108904841 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:20.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.792+0000 7f0ac3400640 1 --2- 172.21.15.67:0/1108904841 >> v2:172.21.15.169:3300/0 conn(0x7f0ac40d0640 0x7f0ac415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:20.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.792+0000 7f0ac2a00640 1 -- 172.21.15.67:0/1108904841 >> v2:172.21.15.169:3300/0 conn(0x7f0ac40d0640 msgr2=0x7f0ac415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:20.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.792+0000 7f0ac2a00640 1 --2- 172.21.15.67:0/1108904841 >> v2:172.21.15.169:3300/0 conn(0x7f0ac40d0640 0x7f0ac415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.792+0000 7f0ac2a00640 1 -- 172.21.15.67:0/1108904841 >> v1:172.21.15.67:6789/0 conn(0x7f0ac40b32c0 legacy=0x7f0ac40b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.792+0000 7f0ac2a00640 1 -- 172.21.15.67:0/1108904841 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac40ad5f0 con 0x7f0ac40d0a20 2024-05-30T16:52:20.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.792+0000 7f0ac2000640 1 -- 172.21.15.67:0/1108904841 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1543201483 0 0) 0x7f0ac415cb10 con 0x7f0ac40b32c0 2024-05-30T16:52:20.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.793+0000 7f0ac2a00640 1 --2- 172.21.15.67:0/1108904841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40d0a20 0x7f0ac40cf950 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4074150 tx=0x7f0ab002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd33244b1c73dc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:20.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.793+0000 7f0ac2000640 1 -- 172.21.15.67:0/1108904841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0036070 con 0x7f0ac40d0a20 2024-05-30T16:52:20.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.793+0000 7f0ac2000640 1 -- 172.21.15.67:0/1108904841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab0031600 con 0x7f0ac40d0a20 2024-05-30T16:52:20.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.793+0000 7f0ac2000640 1 -- 172.21.15.67:0/1108904841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00319d0 con 0x7f0ac40d0a20 2024-05-30T16:52:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.794+0000 7f0aca600640 1 -- 172.21.15.67:0/1108904841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40d0a20 msgr2=0x7f0ac40cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.794+0000 7f0aca600640 1 --2- 172.21.15.67:0/1108904841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40d0a20 0x7f0ac40cf950 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4074150 tx=0x7f0ab002a6d0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.794+0000 7f0aca600640 1 -- 172.21.15.67:0/1108904841 shutdown_connections 2024-05-30T16:52:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.794+0000 7f0aca600640 1 --2- 172.21.15.67:0/1108904841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40d0a20 0x7f0ac40cf950 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.794+0000 7f0aca600640 1 --2- 172.21.15.67:0/1108904841 >> v2:172.21.15.169:3300/0 conn(0x7f0ac40d0640 0x7f0ac415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.794+0000 7f0aca600640 1 -- 172.21.15.67:0/1108904841 >> 172.21.15.67:0/1108904841 conn(0x7f0ac406bd20 msgr2=0x7f0ac406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.794+0000 7f0aca600640 1 -- 172.21.15.67:0/1108904841 shutdown_connections 2024-05-30T16:52:20.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.794+0000 7f0aca600640 1 -- 172.21.15.67:0/1108904841 wait complete. 2024-05-30T16:52:20.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.796+0000 7f0aca600640 1 Processor -- start 2024-05-30T16:52:20.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.796+0000 7f0aca600640 1 -- start start 2024-05-30T16:52:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.797+0000 7f0aca600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40b32c0 0x7f0ac414d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.797+0000 7f0aca600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac40d0640 0x7f0ac4155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.797+0000 7f0aca600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac40d0a20 0x7f0ac4156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.797+0000 7f0aca600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac415d0b0 con 0x7f0ac40b32c0 2024-05-30T16:52:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.797+0000 7f0aca600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac415cdd0 con 0x7f0ac40d0a20 2024-05-30T16:52:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.797+0000 7f0aca600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac40b3e40 con 0x7f0ac40d0640 2024-05-30T16:52:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.797+0000 7f0ac3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40b32c0 0x7f0ac414d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.797+0000 7f0ac3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40b32c0 0x7f0ac414d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39584/0 (socket says 172.21.15.67:39584) 2024-05-30T16:52:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.797+0000 7f0ac3400640 1 -- 172.21.15.67:0/3212522449 learned_addr learned my addr 172.21.15.67:0/3212522449 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac3e00640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac40d0a20 0x7f0ac4156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac2a00640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac40d0640 0x7f0ac4155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac3400640 1 -- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac40d0640 msgr2=0x7f0ac4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac3400640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac40d0640 0x7f0ac4155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac3400640 1 -- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac40d0a20 msgr2=0x7f0ac4156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac3400640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac40d0a20 0x7f0ac4156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac3400640 1 -- 172.21.15.67:0/3212522449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac41f7ac0 con 0x7f0ac40b32c0 2024-05-30T16:52:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac3e00640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac40d0a20 0x7f0ac4156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac2a00640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac40d0640 0x7f0ac4155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:20.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0ac3400640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40b32c0 0x7f0ac414d850 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400d830 tx=0x7f0ab400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:20.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.798+0000 7f0abb400640 1 -- 172.21.15.67:0/3212522449 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400b6c0 con 0x7f0ac40b32c0 2024-05-30T16:52:20.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.799+0000 7f0abb400640 1 -- 172.21.15.67:0/3212522449 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab4019070 con 0x7f0ac40b32c0 2024-05-30T16:52:20.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.799+0000 7f0abb400640 1 -- 172.21.15.67:0/3212522449 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400ba80 con 0x7f0ac40b32c0 2024-05-30T16:52:20.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.799+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac41f7da0 con 0x7f0ac40b32c0 2024-05-30T16:52:20.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.799+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac41f8360 con 0x7f0ac40b32c0 2024-05-30T16:52:20.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.800+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac40689e0 con 0x7f0ac40b32c0 2024-05-30T16:52:20.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.801+0000 7f0abb400640 1 -- 172.21.15.67:0/3212522449 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f0ab401e460 con 0x7f0ac40b32c0 2024-05-30T16:52:20.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.802+0000 7f0abb400640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0aa406d100 0x7f0aa406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:20.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.802+0000 7f0abb400640 1 -- 172.21.15.67:0/3212522449 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f0ab408abe0 con 0x7f0ac40b32c0 2024-05-30T16:52:20.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.802+0000 7f0ac2a00640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0aa406d100 0x7f0aa406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:20.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.803+0000 7f0ac2a00640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0aa406d100 0x7f0aa406f5c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0033000 tx=0x7f0ab0009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:20.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:20.806+0000 7f0abb400640 1 -- 172.21.15.67:0/3212522449 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ab405a900 con 0x7f0ac40b32c0 2024-05-30T16:52:21.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.009+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ac40b3960 con 0x7f0ac40b32c0 2024-05-30T16:52:21.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.010+0000 7f0abb400640 1 -- 172.21.15.67:0/3212522449 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f0ab405a200 con 0x7f0ac40b32c0 2024-05-30T16:52:21.012 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0aa406d100 msgr2=0x7f0aa406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0aa406d100 0x7f0aa406f5c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0033000 tx=0x7f0ab0009000 comp rx=0 tx=0).stop 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40b32c0 msgr2=0x7f0ac414d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40b32c0 0x7f0ac414d850 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400d830 tx=0x7f0ab400dcf0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 shutdown_connections 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0aa406d100 0x7f0aa406f5c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac40d0a20 0x7f0ac4156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac40d0640 0x7f0ac4155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 --2- 172.21.15.67:0/3212522449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac40b32c0 0x7f0ac414d850 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 >> 172.21.15.67:0/3212522449 conn(0x7f0ac406bd20 msgr2=0x7f0ac4153c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 shutdown_connections 2024-05-30T16:52:21.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.013+0000 7f0aca600640 1 -- 172.21.15.67:0/3212522449 wait complete. 2024-05-30T16:52:21.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones' 2024-05-30T16:52:21.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.224+0000 7f1f27200640 1 Processor -- start 2024-05-30T16:52:21.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.224+0000 7f1f27200640 1 -- start start 2024-05-30T16:52:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.225+0000 7f1f27200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1f2015ed00 0x7f1f200b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.225+0000 7f1f27200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f200d1b80 con 0x7f1f200d0350 2024-05-30T16:52:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.225+0000 7f1f27200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f200b5070 con 0x7f1f200d46c0 2024-05-30T16:52:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.225+0000 7f1f27200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1f200b41a0 con 0x7f1f2015ed00 2024-05-30T16:52:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.225+0000 7f1f25200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1f200d0350 0x7f1f200d1470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36530/0 (socket says 172.21.15.67:36530) 2024-05-30T16:52:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.225+0000 7f1f25200640 1 -- 172.21.15.67:0/811877059 learned_addr learned my addr 172.21.15.67:0/811877059 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:21.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1fe00640 1 --2- 172.21.15.67:0/811877059 >> v2:172.21.15.169:3300/0 conn(0x7f1f2015ed00 0x7f1f200b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:21.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2626294271 0 0) 0x7f1f200d1b80 con 0x7f1f200d0350 2024-05-30T16:52:21.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f04003520 con 0x7f1f200d0350 2024-05-30T16:52:21.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2207488339 0 0) 0x7f1f04003520 con 0x7f1f200d0350 2024-05-30T16:52:21.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 >> v2:172.21.15.169:3300/0 conn(0x7f1f2015ed00 msgr2=0x7f1f200b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:21.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 --2- 172.21.15.67:0/811877059 >> v2:172.21.15.169:3300/0 conn(0x7f1f2015ed00 0x7f1f200b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 >> v1:172.21.15.169:6789/0 conn(0x7f1f200d46c0 legacy=0x7f1f200d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:21.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f200d33b0 con 0x7f1f200d0350 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f1f180019f0 con 0x7f1f200d0350 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 >> v1:172.21.15.67:6789/0 conn(0x7f1f200d0350 legacy=0x7f1f200d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f04003a00 0x7f1f04003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.226+0000 7f1f1f400640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f04004800 0x7f1f04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f1f400640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f040051c0 0x7f1f0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1f200b41a0 con 0x7f1f040051c0 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0400c0c0 con 0x7f1f04004800 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0400c2c0 con 0x7f1f04003a00 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f25200640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f040051c0 0x7f1f0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f1fe00640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f04003a00 0x7f1f04003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f17e00640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f04004800 0x7f1f04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f25200640 1 -- 172.21.15.67:0/811877059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f04003a00 msgr2=0x7f1f04003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:21.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f25200640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f04003a00 0x7f1f04003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f25200640 1 -- 172.21.15.67:0/811877059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f04004800 msgr2=0x7f1f04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:21.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f25200640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f04004800 0x7f1f04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.227+0000 7f1f25200640 1 -- 172.21.15.67:0/811877059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f04003520 con 0x7f1f040051c0 2024-05-30T16:52:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.228+0000 7f1f25200640 1 -- 172.21.15.67:0/811877059 reap_dead start 2024-05-30T16:52:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.228+0000 7f1f25200640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f040051c0 0x7f1f0400b9c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800a7b0 tx=0x7f1f04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d97e9834a6a37b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1800a3d0 con 0x7f1f040051c0 2024-05-30T16:52:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f1802f070 con 0x7f1f040051c0 2024-05-30T16:52:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f1f400640 1 -- 172.21.15.67:0/811877059 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1802bc20 con 0x7f1f040051c0 2024-05-30T16:52:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f27200640 1 -- 172.21.15.67:0/811877059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f040051c0 msgr2=0x7f1f0400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f27200640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f040051c0 0x7f1f0400b9c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800a7b0 tx=0x7f1f04005830 comp rx=0 tx=0).stop 2024-05-30T16:52:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f27200640 1 -- 172.21.15.67:0/811877059 shutdown_connections 2024-05-30T16:52:21.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f27200640 1 --2- 172.21.15.67:0/811877059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f040051c0 0x7f1f0400b9c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f27200640 1 -- 172.21.15.67:0/811877059 >> 172.21.15.67:0/811877059 conn(0x7f1f2009da70 msgr2=0x7f1f2008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:21.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f27200640 1 -- 172.21.15.67:0/811877059 shutdown_connections 2024-05-30T16:52:21.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.229+0000 7f1f27200640 1 -- 172.21.15.67:0/811877059 wait complete. 2024-05-30T16:52:21.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.231+0000 7f1f27200640 1 Processor -- start 2024-05-30T16:52:21.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.231+0000 7f1f27200640 1 -- start start 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.232+0000 7f1f27200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040051c0 0x7f1f2015c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.232+0000 7f1f27200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f200cd190 0x7f1f200cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.232+0000 7f1f27200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f200b7350 0x7f1f200b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.232+0000 7f1f27200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1f200b5610 con 0x7f1f200cd190 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.232+0000 7f1f27200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f200b5330 con 0x7f1f040051c0 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.232+0000 7f1f27200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1f200d3040 con 0x7f1f200b7350 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f200cd190 0x7f1f200cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f17e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f200cd190 0x7f1f200cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39594/0 (socket says 172.21.15.67:39594) 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f17e00640 1 -- 172.21.15.67:0/509469796 learned_addr learned my addr 172.21.15.67:0/509469796 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:21.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f25200640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f200b7350 0x7f1f200b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:21.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f1fe00640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040051c0 0x7f1f2015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:21.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f17e00640 1 -- 172.21.15.67:0/509469796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f200b7350 msgr2=0x7f1f200b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:21.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f17e00640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f200b7350 0x7f1f200b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f17e00640 1 -- 172.21.15.67:0/509469796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040051c0 msgr2=0x7f1f2015c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:21.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f17e00640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040051c0 0x7f1f2015c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:21.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f17e00640 1 -- 172.21.15.67:0/509469796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2015cd40 con 0x7f1f200cd190 2024-05-30T16:52:21.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f1fe00640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040051c0 0x7f1f2015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:21.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.233+0000 7f1f17e00640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f200cd190 0x7f1f200cd5f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800e410 tx=0x7f1f0800e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:21.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.234+0000 7f1f1cc00640 1 -- 172.21.15.67:0/509469796 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08019070 con 0x7f1f200cd190 2024-05-30T16:52:21.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.234+0000 7f1f1cc00640 1 -- 172.21.15.67:0/509469796 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f080052e0 con 0x7f1f200cd190 2024-05-30T16:52:21.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.234+0000 7f1f1cc00640 1 -- 172.21.15.67:0/509469796 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08003810 con 0x7f1f200cd190 2024-05-30T16:52:21.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.234+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f200b7f60 con 0x7f1f200cd190 2024-05-30T16:52:21.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.234+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f200b8520 con 0x7f1f200cd190 2024-05-30T16:52:21.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.235+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f201f7ab0 con 0x7f1f200cd190 2024-05-30T16:52:21.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.236+0000 7f1f1cc00640 1 -- 172.21.15.67:0/509469796 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f1f0801c070 con 0x7f1f200cd190 2024-05-30T16:52:21.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.237+0000 7f1f1cc00640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1efc06d100 0x7f1efc06f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:21.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.238+0000 7f1f1cc00640 1 -- 172.21.15.67:0/509469796 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f1f0808bd60 con 0x7f1f200cd190 2024-05-30T16:52:21.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.238+0000 7f1f1fe00640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1efc06d100 0x7f1efc06f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:21.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.239+0000 7f1f1fe00640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1efc06d100 0x7f1efc06f5c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f100099e0 tx=0x7f1f10008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:21.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.242+0000 7f1f1cc00640 1 -- 172.21.15.67:0/509469796 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f201f7ab0 con 0x7f1f200cd190 2024-05-30T16:52:21.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:21.513+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]} v 0) -- 0x7f1f200689e0 con 0x7f1f200cd190 2024-05-30T16:52:22.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.114+0000 7f1f1cc00640 1 -- 172.21.15.67:0/509469796 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]}]=0 v346) ==== 165+0+0 (secure 0 0 0) 0x7f1f08059180 con 0x7f1f200cd190 2024-05-30T16:52:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1efc06d100 msgr2=0x7f1efc06f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1efc06d100 0x7f1efc06f5c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f100099e0 tx=0x7f1f10008040 comp rx=0 tx=0).stop 2024-05-30T16:52:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f200cd190 msgr2=0x7f1f200cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f200cd190 0x7f1f200cd5f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800e410 tx=0x7f1f0800e8d0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 shutdown_connections 2024-05-30T16:52:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1efc06d100 0x7f1efc06f5c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f200b7350 0x7f1f200b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f200cd190 0x7f1f200cd5f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 --2- 172.21.15.67:0/509469796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f040051c0 0x7f1f2015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.118+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 >> 172.21.15.67:0/509469796 conn(0x7f1f2009da70 msgr2=0x7f1f200b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.119+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 shutdown_connections 2024-05-30T16:52:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.119+0000 7f1f27200640 1 -- 172.21.15.67:0/509469796 wait complete. 2024-05-30T16:52:22.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:52:22.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.350+0000 7f177fc00640 1 Processor -- start 2024-05-30T16:52:22.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.350+0000 7f177fc00640 1 -- start start 2024-05-30T16:52:22.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.352+0000 7f177fc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1778162ac0 0x7f17780d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:22.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.352+0000 7f177fc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1778163ac0 0x7f17780d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:22.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.352+0000 7f177fc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17781640e0 con 0x7f17780b4120 2024-05-30T16:52:22.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.352+0000 7f177fc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f17780b3c90 con 0x7f1778162ac0 2024-05-30T16:52:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.352+0000 7f177fc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f17780b3360 con 0x7f1778163ac0 2024-05-30T16:52:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.352+0000 7f177d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1778162ac0 0x7f17780d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.352+0000 7f177d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1778162ac0 0x7f17780d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39596/0 (socket says 172.21.15.67:39596) 2024-05-30T16:52:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.352+0000 7f177d200640 1 -- 172.21.15.67:0/1241924058 learned_addr learned my addr 172.21.15.67:0/1241924058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.352+0000 7f176fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f17780b4120 0x7f17781633b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36546/0 (socket says 172.21.15.67:36546) 2024-05-30T16:52:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.353+0000 7f177d200640 1 -- 172.21.15.67:0/1241924058 >> v2:172.21.15.169:3300/0 conn(0x7f1778163ac0 msgr2=0x7f17780d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.353+0000 7f177d200640 1 --2- 172.21.15.67:0/1241924058 >> v2:172.21.15.169:3300/0 conn(0x7f1778163ac0 0x7f17780d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.353+0000 7f177d200640 1 -- 172.21.15.67:0/1241924058 >> v1:172.21.15.67:6789/0 conn(0x7f17780b4120 legacy=0x7f17781633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.353+0000 7f177d200640 1 -- 172.21.15.67:0/1241924058 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17780b3640 con 0x7f1778162ac0 2024-05-30T16:52:22.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.354+0000 7f177d200640 1 --2- 172.21.15.67:0/1241924058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1778162ac0 0x7f17780d0640 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f17780740c0 tx=0x7f176802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35d033d718f340b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.354+0000 7f1777e00640 1 -- 172.21.15.67:0/1241924058 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768036070 con 0x7f1778162ac0 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.354+0000 7f1777e00640 1 -- 172.21.15.67:0/1241924058 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1768031600 con 0x7f1778162ac0 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.354+0000 7f1777e00640 1 -- 172.21.15.67:0/1241924058 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17680319d0 con 0x7f1778162ac0 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.355+0000 7f177fc00640 1 -- 172.21.15.67:0/1241924058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1778162ac0 msgr2=0x7f17780d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.355+0000 7f177fc00640 1 --2- 172.21.15.67:0/1241924058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1778162ac0 0x7f17780d0640 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f17780740c0 tx=0x7f176802a730 comp rx=0 tx=0).stop 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.355+0000 7f177fc00640 1 -- 172.21.15.67:0/1241924058 shutdown_connections 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.355+0000 7f177fc00640 1 --2- 172.21.15.67:0/1241924058 >> v2:172.21.15.169:3300/0 conn(0x7f1778163ac0 0x7f17780d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.355+0000 7f177fc00640 1 --2- 172.21.15.67:0/1241924058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1778162ac0 0x7f17780d0640 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.355+0000 7f177fc00640 1 -- 172.21.15.67:0/1241924058 >> 172.21.15.67:0/1241924058 conn(0x7f177806bd20 msgr2=0x7f177806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.355+0000 7f177fc00640 1 -- 172.21.15.67:0/1241924058 shutdown_connections 2024-05-30T16:52:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.355+0000 7f177fc00640 1 -- 172.21.15.67:0/1241924058 wait complete. 2024-05-30T16:52:22.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.357+0000 7f177fc00640 1 Processor -- start 2024-05-30T16:52:22.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.357+0000 7f177fc00640 1 -- start start 2024-05-30T16:52:22.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.358+0000 7f177fc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17780b4120 0x7f17781fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:22.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.358+0000 7f177fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1778163ac0 0x7f17780cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:22.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.358+0000 7f177fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781648b0 0x7f17780ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:22.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.358+0000 7f177fc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1778163020 con 0x7f17780b4120 2024-05-30T16:52:22.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.358+0000 7f177fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1778162ea0 con 0x7f17781648b0 2024-05-30T16:52:22.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.358+0000 7f177fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f177806dbd0 con 0x7f1778163ac0 2024-05-30T16:52:22.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.358+0000 7f177d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17780b4120 0x7f17781fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:22.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.358+0000 7f177d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17780b4120 0x7f17781fbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39610/0 (socket says 172.21.15.67:39610) 2024-05-30T16:52:22.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.358+0000 7f177d200640 1 -- 172.21.15.67:0/1498118695 learned_addr learned my addr 172.21.15.67:0/1498118695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:22.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f177d200640 1 -- 172.21.15.67:0/1498118695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1778163ac0 msgr2=0x7f17780cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f177d200640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1778163ac0 0x7f17780cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f177d200640 1 -- 172.21.15.67:0/1498118695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781648b0 msgr2=0x7f17780ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f177d200640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781648b0 0x7f17780ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f177d200640 1 -- 172.21.15.67:0/1498118695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1778164e50 con 0x7f17780b4120 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f177d200640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17780b4120 0x7f17781fbc00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f176802aca0 tx=0x7f1768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f1775600640 1 -- 172.21.15.67:0/1498118695 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768041020 con 0x7f17780b4120 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f1775600640 1 -- 172.21.15.67:0/1498118695 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1768007820 con 0x7f17780b4120 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f1775600640 1 -- 172.21.15.67:0/1498118695 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768036040 con 0x7f17780b4120 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.359+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1778165130 con 0x7f17780b4120 2024-05-30T16:52:22.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.360+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17781656f0 con 0x7f17780b4120 2024-05-30T16:52:22.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.361+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17780689e0 con 0x7f17780b4120 2024-05-30T16:52:22.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.362+0000 7f1775600640 1 -- 172.21.15.67:0/1498118695 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f1768044020 con 0x7f17780b4120 2024-05-30T16:52:22.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.363+0000 7f1775600640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f175406d100 0x7f175406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:22.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.363+0000 7f1775600640 1 -- 172.21.15.67:0/1498118695 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f17680ab1e0 con 0x7f17780b4120 2024-05-30T16:52:22.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.363+0000 7f176fe00640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f175406d100 0x7f175406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:22.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.363+0000 7f176fe00640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f175406d100 0x7f175406f5c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1760000f60 tx=0x7f1760007a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:22.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.368+0000 7f1775600640 1 -- 172.21.15.67:0/1498118695 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f176804b030 con 0x7f17780b4120 2024-05-30T16:52:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.563+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f17780af8a0 con 0x7f17780b4120 2024-05-30T16:52:22.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.568+0000 7f1775600640 1 -- 172.21.15.67:0/1498118695 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f176807af00 con 0x7f17780b4120 2024-05-30T16:52:22.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f175406d100 msgr2=0x7f175406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:22.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f175406d100 0x7f175406f5c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1760000f60 tx=0x7f1760007a00 comp rx=0 tx=0).stop 2024-05-30T16:52:22.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17780b4120 msgr2=0x7f17781fbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:22.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17780b4120 0x7f17781fbc00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f176802aca0 tx=0x7f1768004a40 comp rx=0 tx=0).stop 2024-05-30T16:52:22.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 shutdown_connections 2024-05-30T16:52:22.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f175406d100 0x7f175406f5c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17781648b0 0x7f17780ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1778163ac0 0x7f17780cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 --2- 172.21.15.67:0/1498118695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17780b4120 0x7f17781fbc00 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:22.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 >> 172.21.15.67:0/1498118695 conn(0x7f177806bd20 msgr2=0x7f17780b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:22.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 shutdown_connections 2024-05-30T16:52:22.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:22.571+0000 7f177fc00640 1 -- 172.21.15.67:0/1498118695 wait complete. 2024-05-30T16:52:22.589 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T16:52:22.590 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:52:22.685 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:52:22.803 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T16:52:22.803 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:52:22.803 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:22.863 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:52:22.864 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:52:22.864 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:22.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:52:22.921 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:52:22.921 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:22.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:52:22.980 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:52:22.980 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:23.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:52:23.041 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-30T16:52:23.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:52:23.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.262+0000 7ff74a800640 1 Processor -- start 2024-05-30T16:52:23.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.262+0000 7ff74a800640 1 -- start start 2024-05-30T16:52:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.264+0000 7ff74a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440ad940 0x7ff7440b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.264+0000 7ff74a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff744156f10 con 0x7ff744156b30 2024-05-30T16:52:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.264+0000 7ff74a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff74406dda0 con 0x7ff74406ee40 2024-05-30T16:52:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.264+0000 7ff74a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff74406e080 con 0x7ff7440ad940 2024-05-30T16:52:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.264+0000 7ff749400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff744156b30 0x7ff7440af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36552/0 (socket says 172.21.15.67:36552) 2024-05-30T16:52:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.264+0000 7ff749400640 1 -- 172.21.15.67:0/2639454458 learned_addr learned my addr 172.21.15.67:0/2639454458 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff749e00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440ad940 0x7ff7440b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781370120 0 0) 0x7ff744156f10 con 0x7ff744156b30 2024-05-30T16:52:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff72c003520 con 0x7ff744156b30 2024-05-30T16:52:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1584603703 0 0) 0x7ff72c003520 con 0x7ff744156b30 2024-05-30T16:52:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440ad940 msgr2=0x7ff7440b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440ad940 0x7ff7440b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 >> v1:172.21.15.169:6789/0 conn(0x7ff74406ee40 legacy=0x7ff7440ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74406e360 con 0x7ff744156b30 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff738001670 con 0x7ff744156b30 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 >> v1:172.21.15.67:6789/0 conn(0x7ff744156b30 legacy=0x7ff7440af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.265+0000 7ff743e00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440ad940 0x7ff72c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff743e00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c003f60 0x7ff72c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff743e00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff72c004d40 0x7ff72c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff74406e080 con 0x7ff72c004d40 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c00bb50 con 0x7ff72c003f60 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff72c00bd50 con 0x7ff7440ad940 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff749e00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff72c004d40 0x7ff72c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff748a00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c003f60 0x7ff72c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff749e00640 1 -- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440ad940 msgr2=0x7ff72c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff749400640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440ad940 0x7ff72c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff749e00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440ad940 0x7ff72c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff749e00640 1 -- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c003f60 msgr2=0x7ff72c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff749e00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c003f60 0x7ff72c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff749e00640 1 -- 172.21.15.67:0/2639454458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c003520 con 0x7ff72c004d40 2024-05-30T16:52:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff748a00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c003f60 0x7ff72c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.266+0000 7ff749400640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440ad940 0x7ff72c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.267+0000 7ff749e00640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff72c004d40 0x7ff72c0051c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009f90 tx=0x7ff72c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3de5e5dc8952d75 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.268+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c009c10 con 0x7ff72c004d40 2024-05-30T16:52:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.268+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c036070 con 0x7ff72c004d40 2024-05-30T16:52:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.268+0000 7ff743e00640 1 -- 172.21.15.67:0/2639454458 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0317f0 con 0x7ff72c004d40 2024-05-30T16:52:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.268+0000 7ff74a800640 1 -- 172.21.15.67:0/2639454458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff72c004d40 msgr2=0x7ff72c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.268+0000 7ff74a800640 1 --2- 172.21.15.67:0/2639454458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff72c004d40 0x7ff72c0051c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009f90 tx=0x7ff72c005830 comp rx=0 tx=0).stop 2024-05-30T16:52:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.268+0000 7ff749e00640 1 -- 172.21.15.67:0/2639454458 reap_dead start 2024-05-30T16:52:23.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.268+0000 7ff74a800640 1 -- 172.21.15.67:0/2639454458 shutdown_connections 2024-05-30T16:52:23.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.268+0000 7ff74a800640 1 -- 172.21.15.67:0/2639454458 >> 172.21.15.67:0/2639454458 conn(0x7ff7440b1fc0 msgr2=0x7ff7440b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:23.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.269+0000 7ff74a800640 1 -- 172.21.15.67:0/2639454458 shutdown_connections 2024-05-30T16:52:23.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.269+0000 7ff74a800640 1 -- 172.21.15.67:0/2639454458 wait complete. 2024-05-30T16:52:23.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.270+0000 7ff74a800640 1 Processor -- start 2024-05-30T16:52:23.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.270+0000 7ff74a800640 1 -- start start 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.271+0000 7ff74a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c004d40 0x7ff74414d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.271+0000 7ff74a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff74414a580 0x7ff7441f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.271+0000 7ff74a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440b07c0 0x7ff744152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.271+0000 7ff74a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff7440b2db0 con 0x7ff74414a580 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.271+0000 7ff74a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff7440b2c30 con 0x7ff72c004d40 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.271+0000 7ff74a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff74406f2f0 con 0x7ff7440b07c0 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff748a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff74414a580 0x7ff7441f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff748a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff74414a580 0x7ff7441f5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39626/0 (socket says 172.21.15.67:39626) 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff748a00640 1 -- 172.21.15.67:0/3484452990 learned_addr learned my addr 172.21.15.67:0/3484452990 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff749400640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c004d40 0x7ff74414d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff749e00640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440b07c0 0x7ff744152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff748a00640 1 -- 172.21.15.67:0/3484452990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440b07c0 msgr2=0x7ff744152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff748a00640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440b07c0 0x7ff744152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff748a00640 1 -- 172.21.15.67:0/3484452990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c004d40 msgr2=0x7ff74414d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff748a00640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c004d40 0x7ff74414d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.272+0000 7ff748a00640 1 -- 172.21.15.67:0/3484452990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74414aa10 con 0x7ff74414a580 2024-05-30T16:52:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.273+0000 7ff748a00640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff74414a580 0x7ff7441f5100 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff73000d7a0 tx=0x7ff73000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.273+0000 7ff741600640 1 -- 172.21.15.67:0/3484452990 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730005070 con 0x7ff74414a580 2024-05-30T16:52:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.273+0000 7ff741600640 1 -- 172.21.15.67:0/3484452990 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff730005210 con 0x7ff74414a580 2024-05-30T16:52:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.273+0000 7ff741600640 1 -- 172.21.15.67:0/3484452990 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730007030 con 0x7ff74414a580 2024-05-30T16:52:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.273+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74414acf0 con 0x7ff74414a580 2024-05-30T16:52:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.273+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74414b2b0 con 0x7ff74414a580 2024-05-30T16:52:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.274+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7440689e0 con 0x7ff74414a580 2024-05-30T16:52:23.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.275+0000 7ff741600640 1 -- 172.21.15.67:0/3484452990 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7ff73001a020 con 0x7ff74414a580 2024-05-30T16:52:23.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.276+0000 7ff741600640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff72406d0d0 0x7ff72406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.276+0000 7ff741600640 1 -- 172.21.15.67:0/3484452990 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7ff73008c540 con 0x7ff74414a580 2024-05-30T16:52:23.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.276+0000 7ff749400640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff72406d0d0 0x7ff72406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.277+0000 7ff749400640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff72406d0d0 0x7ff72406f590 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff738005b50 tx=0x7ff7380015b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:23.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.280+0000 7ff741600640 1 -- 172.21.15.67:0/3484452990 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff730059b20 con 0x7ff74414a580 2024-05-30T16:52:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.513+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff74415b0b0 con 0x7ff74414a580 2024-05-30T16:52:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.514+0000 7ff741600640 1 -- 172.21.15.67:0/3484452990 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3872 (secure 0 0 0) 0x7ff730059420 con 0x7ff74414a580 2024-05-30T16:52:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:23.515 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2139720967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2139720967},{"type":"v1","addr":"172.21.15.169:6834","nonce":2139720967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4981,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2317664228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2317664228},{"type":"v1","addr":"172.21.15.169:6835","nonce":2317664228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/350920637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":350920637},{"type":"v1","addr":"172.21.15.67:6835","nonce":350920637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5838,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1919795360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":1919795360},{"type":"v1","addr":"172.21.15.67:6837","nonce":1919795360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-05-30T16:52:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2024-05-30T16:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff72406d0d0 msgr2=0x7ff72406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff72406d0d0 0x7ff72406f590 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff738005b50 tx=0x7ff7380015b0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff74414a580 msgr2=0x7ff7441f5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff74414a580 0x7ff7441f5100 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff73000d7a0 tx=0x7ff73000dc60 comp rx=0 tx=0).stop 2024-05-30T16:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 shutdown_connections 2024-05-30T16:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff72406d0d0 0x7ff72406f590 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7440b07c0 0x7ff744152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff74414a580 0x7ff7441f5100 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 --2- 172.21.15.67:0/3484452990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff72c004d40 0x7ff74414d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.516+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 >> 172.21.15.67:0/3484452990 conn(0x7ff7440b1fc0 msgr2=0x7ff74415dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.517+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 shutdown_connections 2024-05-30T16:52:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.517+0000 7ff74a800640 1 -- 172.21.15.67:0/3484452990 wait complete. 2024-05-30T16:52:23.537 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-30T16:52:23.537 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-30T16:52:23.537 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-30T16:52:23.537 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:52:23.537 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-30T16:52:23.537 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-30T16:52:23.538 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-30T16:52:23.538 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:52:23.538 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-30T16:52:23.538 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-30T16:52:23.538 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-30T16:52:23.538 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:52:23.538 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-30T16:52:23.539 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-30T16:52:23.539 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-30T16:52:23.539 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:52:23.562 INFO:tasks.ceph.mds.b:Stopped 2024-05-30T16:52:23.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-30T16:52:23.563 INFO:tasks.ceph.mds.a:Stopped 2024-05-30T16:52:23.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-30T16:52:23.564 INFO:tasks.ceph.mds.c:Stopped 2024-05-30T16:52:23.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-30T16:52:23.573 INFO:tasks.ceph.mds.d:Stopped 2024-05-30T16:52:23.573 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-30T16:52:23.611 INFO:tasks.ceph.mds.a:Started 2024-05-30T16:52:23.612 DEBUG:teuthology.parallel:result is None 2024-05-30T16:52:23.616 INFO:tasks.ceph.mds.c:Started 2024-05-30T16:52:23.616 DEBUG:teuthology.parallel:result is None 2024-05-30T16:52:23.621 INFO:tasks.ceph.mds.b:Started 2024-05-30T16:52:23.621 DEBUG:teuthology.parallel:result is None 2024-05-30T16:52:23.627 INFO:tasks.ceph.mds.d:Started 2024-05-30T16:52:23.627 DEBUG:teuthology.parallel:result is None 2024-05-30T16:52:23.627 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-30T16:52:23.754 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T16:52:23.757 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T16:52:23.782 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T16:52:23.782 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T16:52:23.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.799+0000 7f3813e00640 1 Processor -- start 2024-05-30T16:52:23.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.799+0000 7f3813e00640 1 -- start start 2024-05-30T16:52:23.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.800+0000 7f3813e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 0x7f38140d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.800+0000 7f3813e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f381415ec50 con 0x7f38140cfa40 2024-05-30T16:52:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.800+0000 7f3813e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38140cff80 con 0x7f38140d14d0 2024-05-30T16:52:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.800+0000 7f3813e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38140d2150 con 0x7f381415e870 2024-05-30T16:52:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.801+0000 7f3813400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f38140cfa40 0x7f38140d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36570/0 (socket says 172.21.15.67:36570) 2024-05-30T16:52:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.801+0000 7f3813400640 1 -- 172.21.15.67:0/783062379 learned_addr learned my addr 172.21.15.67:0/783062379 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.801+0000 7f3812a00640 1 --2- 172.21.15.67:0/783062379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 0x7f38140d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.801+0000 7f3812000640 1 -- 172.21.15.67:0/783062379 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 781539501 0 0) 0x7f381415ec50 con 0x7f38140cfa40 2024-05-30T16:52:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.801+0000 7f3812000640 1 -- 172.21.15.67:0/783062379 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37f8003520 con 0x7f38140cfa40 2024-05-30T16:52:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.801+0000 7f3812a00640 1 -- 172.21.15.67:0/783062379 >> v1:172.21.15.169:6789/0 conn(0x7f38140d14d0 legacy=0x7f38140cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.801+0000 7f3812a00640 1 -- 172.21.15.67:0/783062379 >> v1:172.21.15.67:6789/0 conn(0x7f38140cfa40 legacy=0x7f38140d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.801+0000 7f3812000640 1 -- 172.21.15.67:0/783062379 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 373729087 0 0) 0x7f37f8003520 con 0x7f38140cfa40 2024-05-30T16:52:23.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.801+0000 7f3812a00640 1 -- 172.21.15.67:0/783062379 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38140d0260 con 0x7f381415e870 2024-05-30T16:52:23.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.802+0000 7f3812a00640 1 --2- 172.21.15.67:0/783062379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 0x7f38140d0f90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38040097d0 tx=0x7f380402a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f92976f03e2827 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:23.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.803+0000 7f3812000640 1 -- 172.21.15.67:0/783062379 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804037070 con 0x7f381415e870 2024-05-30T16:52:23.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.803+0000 7f3812000640 1 -- 172.21.15.67:0/783062379 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380402ab40 con 0x7f381415e870 2024-05-30T16:52:23.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.803+0000 7f3812000640 1 -- 172.21.15.67:0/783062379 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c001970 con 0x7f381415e870 2024-05-30T16:52:23.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.803+0000 7f3813e00640 1 -- 172.21.15.67:0/783062379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 msgr2=0x7f38140d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.803+0000 7f3813e00640 1 --2- 172.21.15.67:0/783062379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 0x7f38140d0f90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38040097d0 tx=0x7f380402a0b0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.804+0000 7f3813e00640 1 -- 172.21.15.67:0/783062379 shutdown_connections 2024-05-30T16:52:23.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.804+0000 7f3813e00640 1 --2- 172.21.15.67:0/783062379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 0x7f38140d0f90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.804+0000 7f3813e00640 1 -- 172.21.15.67:0/783062379 >> 172.21.15.67:0/783062379 conn(0x7f381408bfc0 msgr2=0x7f38140d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:23.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.804+0000 7f3813e00640 1 -- 172.21.15.67:0/783062379 shutdown_connections 2024-05-30T16:52:23.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.804+0000 7f3813e00640 1 -- 172.21.15.67:0/783062379 wait complete. 2024-05-30T16:52:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.805+0000 7f3813e00640 1 Processor -- start 2024-05-30T16:52:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.805+0000 7f3813e00640 1 -- start start 2024-05-30T16:52:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.806+0000 7f3813e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38140cfa40 0x7f381415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.806+0000 7f3813e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38140d14d0 0x7f38140cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.806+0000 7f3813e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 0x7f381414f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.806+0000 7f3813e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f38140d4fb0 con 0x7f38140cfa40 2024-05-30T16:52:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.806+0000 7f3813e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f38140d0820 con 0x7f38140d14d0 2024-05-30T16:52:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.806+0000 7f3813e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38140d2cf0 con 0x7f381415e870 2024-05-30T16:52:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.806+0000 7f3812a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38140cfa40 0x7f381415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.806+0000 7f3812a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38140cfa40 0x7f381415b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39652/0 (socket says 172.21.15.67:39652) 2024-05-30T16:52:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.806+0000 7f3812a00640 1 -- 172.21.15.67:0/1442880477 learned_addr learned my addr 172.21.15.67:0/1442880477 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.807+0000 7f380be00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38140d14d0 0x7f38140cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.807+0000 7f3812a00640 1 -- 172.21.15.67:0/1442880477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 msgr2=0x7f381414f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.807+0000 7f3812a00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 0x7f381414f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.807+0000 7f3812a00640 1 -- 172.21.15.67:0/1442880477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38140d14d0 msgr2=0x7f38140cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.807+0000 7f3812a00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38140d14d0 0x7f38140cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.807+0000 7f3812a00640 1 -- 172.21.15.67:0/1442880477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381415b640 con 0x7f38140cfa40 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.807+0000 7f380be00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38140d14d0 0x7f38140cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.807+0000 7f3812a00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38140cfa40 0x7f381415b060 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3804000c00 tx=0x7f38040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.807+0000 7f380aa00640 1 -- 172.21.15.67:0/1442880477 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804037040 con 0x7f38140cfa40 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.808+0000 7f380aa00640 1 -- 172.21.15.67:0/1442880477 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3804004030 con 0x7f38140cfa40 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.808+0000 7f380aa00640 1 -- 172.21.15.67:0/1442880477 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804004420 con 0x7f38140cfa40 2024-05-30T16:52:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.808+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381415b920 con 0x7f38140cfa40 2024-05-30T16:52:23.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.808+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38140cc570 con 0x7f38140cfa40 2024-05-30T16:52:23.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.809+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38140689e0 con 0x7f38140cfa40 2024-05-30T16:52:23.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.810+0000 7f380aa00640 1 -- 172.21.15.67:0/1442880477 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f3804003610 con 0x7f38140cfa40 2024-05-30T16:52:23.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.810+0000 7f380aa00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006d0d0 0x7f37f006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:23.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.811+0000 7f380aa00640 1 -- 172.21.15.67:0/1442880477 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f38040acc10 con 0x7f38140cfa40 2024-05-30T16:52:23.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.811+0000 7f380be00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006d0d0 0x7f37f006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:23.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.811+0000 7f380be00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006d0d0 0x7f37f006f590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3814161020 tx=0x7f37fc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:23.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:23.815+0000 7f380aa00640 1 -- 172.21.15.67:0/1442880477 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3804042020 con 0x7f38140cfa40 2024-05-30T16:52:24.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.032+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f381415baf0 con 0x7f38140cfa40 2024-05-30T16:52:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.151+0000 7f380aa00640 1 -- 172.21.15.67:0/1442880477 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) ==== 97+0+0 (secure 0 0 0) 0x7f380407a7a0 con 0x7f38140cfa40 2024-05-30T16:52:24.152 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-05-30T16:52:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006d0d0 msgr2=0x7f37f006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006d0d0 0x7f37f006f590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3814161020 tx=0x7f37fc009040 comp rx=0 tx=0).stop 2024-05-30T16:52:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38140cfa40 msgr2=0x7f381415b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38140cfa40 0x7f381415b060 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3804000c00 tx=0x7f38040046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 shutdown_connections 2024-05-30T16:52:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f37f006d0d0 0x7f37f006f590 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f381415e870 0x7f381414f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38140d14d0 0x7f38140cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 --2- 172.21.15.67:0/1442880477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38140cfa40 0x7f381415b060 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.154+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 >> 172.21.15.67:0/1442880477 conn(0x7f381408bfc0 msgr2=0x7f38140b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.155+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 shutdown_connections 2024-05-30T16:52:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.155+0000 7f3813e00640 1 -- 172.21.15.67:0/1442880477 wait complete. 2024-05-30T16:52:24.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-30T16:52:24.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.386+0000 7f3f9d600640 1 Processor -- start 2024-05-30T16:52:24.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.386+0000 7f3f9d600640 1 -- start start 2024-05-30T16:52:24.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.388+0000 7f3f9d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f980b4e10 0x7f3f980cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.388+0000 7f3f9d600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f980b51f0 con 0x7f3f980ce370 2024-05-30T16:52:24.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.388+0000 7f3f9d600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f980aef60 con 0x7f3f980d0ca0 2024-05-30T16:52:24.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.388+0000 7f3f9d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3f980ad1d0 con 0x7f3f980b4e10 2024-05-30T16:52:24.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.388+0000 7f3f8e000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3f980ce370 0x7f3f980d0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36576/0 (socket says 172.21.15.67:36576) 2024-05-30T16:52:24.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.388+0000 7f3f8e000640 1 -- 172.21.15.67:0/3978144869 learned_addr learned my addr 172.21.15.67:0/3978144869 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:24.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.388+0000 7f3f96a00640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f980b4e10 0x7f3f980cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2444251221 0 0) 0x7f3f980b51f0 con 0x7f3f980ce370 2024-05-30T16:52:24.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f7c003520 con 0x7f3f980ce370 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 17845600 0 0) 0x7f3f7c003520 con 0x7f3f980ce370 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f980b4e10 msgr2=0x7f3f980cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f980b4e10 0x7f3f980cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 >> v1:172.21.15.169:6789/0 conn(0x7f3f980d0ca0 legacy=0x7f3f98099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f980ad470 con 0x7f3f980ce370 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f3f80001670 con 0x7f3f980ce370 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96a00640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f980b4e10 0x7f3f980cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 >> v1:172.21.15.67:6789/0 conn(0x7f3f980ce370 legacy=0x7f3f980d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f7c0039e0 0x7f3f7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f7c004800 0x7f3f7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f980b4e10 0x7f3f7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3f980ad1d0 con 0x7f3f7c0039e0 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c0054e0 con 0x7f3f7c004800 2024-05-30T16:52:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7c005660 con 0x7f3f980b4e10 2024-05-30T16:52:24.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.389+0000 7f3f96a00640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f7c0039e0 0x7f3f7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.390+0000 7f3f96a00640 1 -- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f980b4e10 msgr2=0x7f3f7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:24.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.390+0000 7f3f96a00640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f980b4e10 0x7f3f7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.390+0000 7f3f96a00640 1 -- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f7c004800 msgr2=0x7f3f7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:24.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.390+0000 7f3f96a00640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f7c004800 0x7f3f7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.390+0000 7f3f96a00640 1 -- 172.21.15.67:0/3978144869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c003520 con 0x7f3f7c0039e0 2024-05-30T16:52:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.391+0000 7f3f96a00640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f7c0039e0 0x7f3f7c003e00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3f880098a0 tx=0x7f3f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a55b25037057be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.391+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800f040 con 0x7f3f7c0039e0 2024-05-30T16:52:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.391+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8802ad00 con 0x7f3f7c0039e0 2024-05-30T16:52:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.391+0000 7f3f96000640 1 -- 172.21.15.67:0/3978144869 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88004250 con 0x7f3f7c0039e0 2024-05-30T16:52:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.392+0000 7f3f9d600640 1 -- 172.21.15.67:0/3978144869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f7c0039e0 msgr2=0x7f3f7c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.392+0000 7f3f9d600640 1 --2- 172.21.15.67:0/3978144869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f7c0039e0 0x7f3f7c003e00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3f880098a0 tx=0x7f3f7c005830 comp rx=0 tx=0).stop 2024-05-30T16:52:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.392+0000 7f3f97400640 1 -- 172.21.15.67:0/3978144869 reap_dead start 2024-05-30T16:52:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.392+0000 7f3f9d600640 1 -- 172.21.15.67:0/3978144869 shutdown_connections 2024-05-30T16:52:24.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.392+0000 7f3f9d600640 1 -- 172.21.15.67:0/3978144869 >> 172.21.15.67:0/3978144869 conn(0x7f3f9806bae0 msgr2=0x7f3f9806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:24.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.392+0000 7f3f9d600640 1 -- 172.21.15.67:0/3978144869 shutdown_connections 2024-05-30T16:52:24.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.392+0000 7f3f9d600640 1 -- 172.21.15.67:0/3978144869 wait complete. 2024-05-30T16:52:24.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.394+0000 7f3f9d600640 1 Processor -- start 2024-05-30T16:52:24.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.394+0000 7f3f9d600640 1 -- start start 2024-05-30T16:52:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f9d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f7c0039e0 0x7f3f9815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f9d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f980c3d90 0x7f3f981f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f9d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f980afdc0 0x7f3f980c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f9d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3f980cea00 con 0x7f3f980afdc0 2024-05-30T16:52:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f9d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3f980ce880 con 0x7f3f980c3d90 2024-05-30T16:52:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f9d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3f980b3150 con 0x7f3f7c0039e0 2024-05-30T16:52:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f980afdc0 0x7f3f980c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f980afdc0 0x7f3f980c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39660/0 (socket says 172.21.15.67:39660) 2024-05-30T16:52:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f97400640 1 -- 172.21.15.67:0/2984096247 learned_addr learned my addr 172.21.15.67:0/2984096247 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.395+0000 7f3f96a00640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f7c0039e0 0x7f3f9815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f97400640 1 -- 172.21.15.67:0/2984096247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f7c0039e0 msgr2=0x7f3f9815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f97400640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f7c0039e0 0x7f3f9815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f97400640 1 -- 172.21.15.67:0/2984096247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f980c3d90 msgr2=0x7f3f981f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f97400640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f980c3d90 0x7f3f981f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f97400640 1 -- 172.21.15.67:0/2984096247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f980b1530 con 0x7f3f980afdc0 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f97400640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f980afdc0 0x7f3f980c96b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3f980af870 tx=0x7f3f9000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f8f400640 1 -- 172.21.15.67:0/2984096247 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90017070 con 0x7f3f980afdc0 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f8f400640 1 -- 172.21.15.67:0/2984096247 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f90005d40 con 0x7f3f980afdc0 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f8f400640 1 -- 172.21.15.67:0/2984096247 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9000a880 con 0x7f3f980afdc0 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f980c73d0 con 0x7f3f980afdc0 2024-05-30T16:52:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.396+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f980c7990 con 0x7f3f980afdc0 2024-05-30T16:52:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.397+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f980689e0 con 0x7f3f980afdc0 2024-05-30T16:52:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.399+0000 7f3f8f400640 1 -- 172.21.15.67:0/2984096247 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f3f9000aa20 con 0x7f3f980afdc0 2024-05-30T16:52:24.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.399+0000 7f3f8f400640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3f7406d0d0 0x7f3f7406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.400+0000 7f3f8f400640 1 -- 172.21.15.67:0/2984096247 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4885+0+0 (secure 0 0 0) 0x7f3f9008bbb0 con 0x7f3f980afdc0 2024-05-30T16:52:24.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.400+0000 7f3f96a00640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3f7406d0d0 0x7f3f7406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.400+0000 7f3f96a00640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3f7406d0d0 0x7f3f7406f590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3f88033000 tx=0x7f3f8802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:24.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.404+0000 7f3f8f400640 1 -- 172.21.15.67:0/2984096247 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f90059380 con 0x7f3f980afdc0 2024-05-30T16:52:24.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.681+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3f980cc4f0 con 0x7f3f980afdc0 2024-05-30T16:52:24.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.682+0000 7f3f8f400640 1 -- 172.21.15.67:0/2984096247 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f3f90058c80 con 0x7f3f980afdc0 2024-05-30T16:52:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:24.683 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:52:24.683 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA==", 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw==", 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.684 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw==", 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.685 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg==", 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.686 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng==", 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg==", 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.687 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w==", 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ==", 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:52:24.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A==", 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-05-30T16:52:24.689 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ==", 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w==", 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A==", 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg==", 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.691 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ==", 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.2", 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ==", 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.3", 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A==", 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:52:24.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCArFhmF5mhNhAA4H3cgcH6YdD53fANDW4seA==", 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ==", 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw==", 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:52:24.695 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:52:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:52:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:24.696 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-05-30T16:52:24.696 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3f7406d0d0 msgr2=0x7f3f7406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3f7406d0d0 0x7f3f7406f590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3f88033000 tx=0x7f3f8802e000 comp rx=0 tx=0).stop 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f980afdc0 msgr2=0x7f3f980c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f980afdc0 0x7f3f980c96b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3f980af870 tx=0x7f3f9000cb90 comp rx=0 tx=0).stop 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 shutdown_connections 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3f7406d0d0 0x7f3f7406f590 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f980afdc0 0x7f3f980c96b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f980c3d90 0x7f3f981f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 --2- 172.21.15.67:0/2984096247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f7c0039e0 0x7f3f9815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 >> 172.21.15.67:0/2984096247 conn(0x7f3f9806bae0 msgr2=0x7f3f980cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:24.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 shutdown_connections 2024-05-30T16:52:24.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.685+0000 7f3f9d600640 1 -- 172.21.15.67:0/2984096247 wait complete. 2024-05-30T16:52:24.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-30T16:52:24.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.919+0000 7efda9600640 1 Processor -- start 2024-05-30T16:52:24.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.919+0000 7efda9600640 1 -- start start 2024-05-30T16:52:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efda40d0590 0x7efda40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda40994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda9600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda40b51f0 con 0x7efda40b4e10 2024-05-30T16:52:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda9600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7efda40aef60 con 0x7efda40d0590 2024-05-30T16:52:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efda40ad1d0 con 0x7efda4099090 2024-05-30T16:52:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efda40d0590 0x7efda40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efda40b4e10 0x7efda40cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36592/0 (socket says 172.21.15.67:36592) 2024-05-30T16:52:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efda40d0590 0x7efda40d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39676/0 (socket says 172.21.15.67:39676) 2024-05-30T16:52:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda2a00640 1 -- 172.21.15.67:0/3859852635 learned_addr learned my addr 172.21.15.67:0/3859852635 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda3400640 1 --2- 172.21.15.67:0/3859852635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda40994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda2000640 1 -- 172.21.15.67:0/3859852635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 msgr2=0x7efda40994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda2000640 1 --2- 172.21.15.67:0/3859852635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda40994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.921+0000 7efda2000640 1 -- 172.21.15.67:0/3859852635 >> v1:172.21.15.67:6789/0 conn(0x7efda40b4e10 legacy=0x7efda40cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.922+0000 7efda2000640 1 -- 172.21.15.67:0/3859852635 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda40ad470 con 0x7efda40d0590 2024-05-30T16:52:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.922+0000 7efda1600640 1 -- 172.21.15.67:0/3859852635 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365325551 0 0) 0x7efda40b51f0 con 0x7efda40b4e10 2024-05-30T16:52:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.922+0000 7efda3400640 1 --2- 172.21.15.67:0/3859852635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda40994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:24.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.923+0000 7efda2000640 1 --2- 172.21.15.67:0/3859852635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 0x7efda40d0a20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efd900097a0 tx=0x7efd9002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a612652b6c491b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:24.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.923+0000 7efda1600640 1 -- 172.21.15.67:0/3859852635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90037070 con 0x7efda40d0590 2024-05-30T16:52:24.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.923+0000 7efda1600640 1 -- 172.21.15.67:0/3859852635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9002ae50 con 0x7efda40d0590 2024-05-30T16:52:24.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.923+0000 7efda1600640 1 -- 172.21.15.67:0/3859852635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90032660 con 0x7efda40d0590 2024-05-30T16:52:24.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.923+0000 7efda9600640 1 -- 172.21.15.67:0/3859852635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 msgr2=0x7efda40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.923+0000 7efda9600640 1 --2- 172.21.15.67:0/3859852635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 0x7efda40d0a20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efd900097a0 tx=0x7efd9002a100 comp rx=0 tx=0).stop 2024-05-30T16:52:24.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.924+0000 7efda9600640 1 -- 172.21.15.67:0/3859852635 shutdown_connections 2024-05-30T16:52:24.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.924+0000 7efda9600640 1 --2- 172.21.15.67:0/3859852635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda40994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.924+0000 7efda9600640 1 --2- 172.21.15.67:0/3859852635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 0x7efda40d0a20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.924+0000 7efda9600640 1 -- 172.21.15.67:0/3859852635 >> 172.21.15.67:0/3859852635 conn(0x7efda406bae0 msgr2=0x7efda406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:24.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.924+0000 7efda9600640 1 -- 172.21.15.67:0/3859852635 shutdown_connections 2024-05-30T16:52:24.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.924+0000 7efda9600640 1 -- 172.21.15.67:0/3859852635 wait complete. 2024-05-30T16:52:24.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.925+0000 7efda9600640 1 Processor -- start 2024-05-30T16:52:24.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.926+0000 7efda9600640 1 -- start start 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda41f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda40b4e10 0x7efda40c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 0x7efda415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda9600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efda40ce880 con 0x7efda40d0590 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efda40ce540 con 0x7efda40b4e10 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efda40cea00 con 0x7efda4099090 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 0x7efda415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 0x7efda415d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39680/0 (socket says 172.21.15.67:39680) 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda3400640 1 -- 172.21.15.67:0/4020307784 learned_addr learned my addr 172.21.15.67:0/4020307784 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda2a00640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda41f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda3400640 1 -- 172.21.15.67:0/4020307784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 msgr2=0x7efda41f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda3400640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda41f4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda3400640 1 -- 172.21.15.67:0/4020307784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda40b4e10 msgr2=0x7efda40c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda2000640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda40b4e10 0x7efda40c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda3400640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda40b4e10 0x7efda40c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda3400640 1 -- 172.21.15.67:0/4020307784 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda40c4d10 con 0x7efda40d0590 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.927+0000 7efda2a00640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda41f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.928+0000 7efda3400640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 0x7efda415d720 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00cd50 tx=0x7efd9c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.928+0000 7efd9aa00640 1 -- 172.21.15.67:0/4020307784 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00f680 con 0x7efda40d0590 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.928+0000 7efd9aa00640 1 -- 172.21.15.67:0/4020307784 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd9c00f840 con 0x7efda40d0590 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.928+0000 7efd9aa00640 1 -- 172.21.15.67:0/4020307784 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00fc30 con 0x7efda40d0590 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.928+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda40c5740 con 0x7efda40d0590 2024-05-30T16:52:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.928+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda40c5d00 con 0x7efda40d0590 2024-05-30T16:52:24.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.929+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda40689e0 con 0x7efda40d0590 2024-05-30T16:52:24.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.930+0000 7efd9aa00640 1 -- 172.21.15.67:0/4020307784 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7efd9c00e3e0 con 0x7efda40d0590 2024-05-30T16:52:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.931+0000 7efd9aa00640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406d0d0 0x7efd8406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.931+0000 7efd9aa00640 1 -- 172.21.15.67:0/4020307784 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4885+0+0 (secure 0 0 0) 0x7efd9c08d580 con 0x7efda40d0590 2024-05-30T16:52:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.932+0000 7efda2a00640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406d0d0 0x7efd8406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.932+0000 7efda2a00640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406d0d0 0x7efd8406f590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efd94000f60 tx=0x7efd94009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:24.936+0000 7efd9aa00640 1 -- 172.21.15.67:0/4020307784 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd9c014030 con 0x7efda40d0590 2024-05-30T16:52:25.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.144+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efda40cf6c0 con 0x7efda40d0590 2024-05-30T16:52:25.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.151+0000 7efd9aa00640 1 -- 172.21.15.67:0/4020307784 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) ==== 84+0+65 (secure 0 0 0) 0x7efd9c05b050 con 0x7efda40d0590 2024-05-30T16:52:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:25.152 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:52:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-05-30T16:52:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-05-30T16:52:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:25.152 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406d0d0 msgr2=0x7efd8406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406d0d0 0x7efd8406f590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efd94000f60 tx=0x7efd94009040 comp rx=0 tx=0).stop 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 msgr2=0x7efda415d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 0x7efda415d720 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00cd50 tx=0x7efd9c00a650 comp rx=0 tx=0).stop 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 shutdown_connections 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406d0d0 0x7efd8406f590 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda40d0590 0x7efda415d720 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda40b4e10 0x7efda40c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 --2- 172.21.15.67:0/4020307784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda4099090 0x7efda41f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 >> 172.21.15.67:0/4020307784 conn(0x7efda406bae0 msgr2=0x7efda40b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 shutdown_connections 2024-05-30T16:52:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.154+0000 7efda9600640 1 -- 172.21.15.67:0/4020307784 wait complete. 2024-05-30T16:52:25.175 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-30T16:52:25.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-30T16:52:25.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.372+0000 7f55c6c00640 1 Processor -- start 2024-05-30T16:52:25.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.372+0000 7f55c6c00640 1 -- start start 2024-05-30T16:52:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.374+0000 7f55c6c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55c006ef00 0x7f55c00d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.374+0000 7f55c6c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c015c7d0 con 0x7f55c015c3f0 2024-05-30T16:52:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.374+0000 7f55c6c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c00ad9a0 con 0x7f55c00b4790 2024-05-30T16:52:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.374+0000 7f55c6c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f55c00ad3e0 con 0x7f55c006ef00 2024-05-30T16:52:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.374+0000 7f55c5800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f55c015c3f0 0x7f55c01608d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36604/0 (socket says 172.21.15.67:36604) 2024-05-30T16:52:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.374+0000 7f55c5800640 1 -- 172.21.15.67:0/199596167 learned_addr learned my addr 172.21.15.67:0/199596167 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55c4e00640 1 --2- 172.21.15.67:0/199596167 >> v2:172.21.15.169:3300/0 conn(0x7f55c006ef00 0x7f55c00d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 946930791 0 0) 0x7f55c015c7d0 con 0x7f55c015c3f0 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55a8003520 con 0x7f55c015c3f0 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3628444694 0 0) 0x7f55a8003520 con 0x7f55c015c3f0 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 >> v2:172.21.15.169:3300/0 conn(0x7f55c006ef00 msgr2=0x7f55c00d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 --2- 172.21.15.67:0/199596167 >> v2:172.21.15.169:3300/0 conn(0x7f55c006ef00 0x7f55c00d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 >> v1:172.21.15.169:6789/0 conn(0x7f55c00b4790 legacy=0x7f55c00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c00ad6c0 con 0x7f55c015c3f0 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f55b4001670 con 0x7f55c015c3f0 2024-05-30T16:52:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 >> v1:172.21.15.67:6789/0 conn(0x7f55c015c3f0 legacy=0x7f55c01608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55a80039b0 0x7f55a8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55a80051c0 0x7f55a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55c00ad3e0 con 0x7f55a8004800 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.375+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55a800c090 con 0x7f55a80039b0 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.376+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f55a800c290 con 0x7f55a80051c0 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.376+0000 7f55c4e00640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.376+0000 7f55c5800640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55a80039b0 0x7f55a8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.376+0000 7f55c4e00640 1 -- 172.21.15.67:0/199596167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55a80051c0 msgr2=0x7f55a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.376+0000 7f55c4e00640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55a80051c0 0x7f55a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.376+0000 7f55c4e00640 1 -- 172.21.15.67:0/199596167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55a80039b0 msgr2=0x7f55a8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.376+0000 7f55c4e00640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55a80039b0 0x7f55a8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.376+0000 7f55c4e00640 1 -- 172.21.15.67:0/199596167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a8003520 con 0x7f55a8004800 2024-05-30T16:52:25.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.376+0000 7f55c6200640 1 -- 172.21.15.67:0/199596167 reap_dead start 2024-05-30T16:52:25.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.377+0000 7f55c4e00640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55a8004c80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009a90 tx=0x7f55a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d16d025151dc805 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac036070 con 0x7f55a8004800 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac031400 con 0x7f55a8004800 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55bfe00640 1 -- 172.21.15.67:0/199596167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac009690 con 0x7f55a8004800 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55c6c00640 1 -- 172.21.15.67:0/199596167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 msgr2=0x7f55a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55c6c00640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55a8004c80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009a90 tx=0x7f55a8005830 comp rx=0 tx=0).stop 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55c6c00640 1 -- 172.21.15.67:0/199596167 shutdown_connections 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55c6c00640 1 --2- 172.21.15.67:0/199596167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55a8004c80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55c6c00640 1 -- 172.21.15.67:0/199596167 >> 172.21.15.67:0/199596167 conn(0x7f55c006bae0 msgr2=0x7f55c006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55c6c00640 1 -- 172.21.15.67:0/199596167 shutdown_connections 2024-05-30T16:52:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.378+0000 7f55c6c00640 1 -- 172.21.15.67:0/199596167 wait complete. 2024-05-30T16:52:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.380+0000 7f55c6c00640 1 Processor -- start 2024-05-30T16:52:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.380+0000 7f55c6c00640 1 -- start start 2024-05-30T16:52:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.381+0000 7f55c6c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55c015f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.381+0000 7f55c6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55c015b3f0 0x7f55c0147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.381+0000 7f55c6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c0159d90 0x7f55c015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.381+0000 7f55c6c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55c00ad090 con 0x7f55a8004800 2024-05-30T16:52:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.381+0000 7f55c6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01603b0 con 0x7f55c0159d90 2024-05-30T16:52:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.381+0000 7f55c6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f55c01608d0 con 0x7f55c015b3f0 2024-05-30T16:52:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55c015f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55c015f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39706/0 (socket says 172.21.15.67:39706) 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c5800640 1 -- 172.21.15.67:0/2833225292 learned_addr learned my addr 172.21.15.67:0/2833225292 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c4e00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55c015b3f0 0x7f55c0147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c6200640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c0159d90 0x7f55c015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c5800640 1 -- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55c015b3f0 msgr2=0x7f55c0147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c5800640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55c015b3f0 0x7f55c0147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c5800640 1 -- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c0159d90 msgr2=0x7f55c015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c5800640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c0159d90 0x7f55c015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c5800640 1 -- 172.21.15.67:0/2833225292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c015a350 con 0x7f55a8004800 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c4e00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55c015b3f0 0x7f55c0147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.382+0000 7f55c6200640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c0159d90 0x7f55c015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.383+0000 7f55c5800640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55c015f4f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f55b400e8f0 tx=0x7f55b400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.383+0000 7f55bd600640 1 -- 172.21.15.67:0/2833225292 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b400c8d0 con 0x7f55a8004800 2024-05-30T16:52:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.383+0000 7f55bd600640 1 -- 172.21.15.67:0/2833225292 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b400ca90 con 0x7f55a8004800 2024-05-30T16:52:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.383+0000 7f55bd600640 1 -- 172.21.15.67:0/2833225292 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b400cc80 con 0x7f55a8004800 2024-05-30T16:52:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.383+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c015a630 con 0x7f55a8004800 2024-05-30T16:52:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.383+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c015abf0 con 0x7f55a8004800 2024-05-30T16:52:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.384+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c015fdf0 con 0x7f55a8004800 2024-05-30T16:52:25.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.385+0000 7f55bd600640 1 -- 172.21.15.67:0/2833225292 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f55b401a020 con 0x7f55a8004800 2024-05-30T16:52:25.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.386+0000 7f55bd600640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f55a006d0d0 0x7f55a006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:25.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.386+0000 7f55bd600640 1 -- 172.21.15.67:0/2833225292 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4885+0+0 (secure 0 0 0) 0x7f55b408c230 con 0x7f55a8004800 2024-05-30T16:52:25.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.387+0000 7f55c4e00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f55a006d0d0 0x7f55a006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:25.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.387+0000 7f55c4e00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f55a006d0d0 0x7f55a006f590 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009a90 tx=0x7f55ac008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:25.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.391+0000 7f55bd600640 1 -- 172.21.15.67:0/2833225292 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55c015fdf0 con 0x7f55a8004800 2024-05-30T16:52:25.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:25.586+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f55c00689e0 con 0x7f55a8004800 2024-05-30T16:52:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.176+0000 7f55bd600640 1 -- 172.21.15.67:0/2833225292 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) ==== 138+0+0 (secure 0 0 0) 0x7f55b4021020 con 0x7f55a8004800 2024-05-30T16:52:26.178 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f55a006d0d0 msgr2=0x7f55a006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f55a006d0d0 0x7f55a006f590 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009a90 tx=0x7f55ac008030 comp rx=0 tx=0).stop 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 msgr2=0x7f55c015f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55c015f4f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f55b400e8f0 tx=0x7f55b400edb0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 shutdown_connections 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f55a006d0d0 0x7f55a006f590 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c0159d90 0x7f55c015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55c015b3f0 0x7f55c0147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 --2- 172.21.15.67:0/2833225292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55a8004800 0x7f55c015f4f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.180+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 >> 172.21.15.67:0/2833225292 conn(0x7f55c006bae0 msgr2=0x7f55c006dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.181+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 shutdown_connections 2024-05-30T16:52:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.181+0000 7f55c6c00640 1 -- 172.21.15.67:0/2833225292 wait complete. 2024-05-30T16:52:26.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-30T16:52:26.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.416+0000 7fb6c2800640 1 Processor -- start 2024-05-30T16:52:26.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.417+0000 7fb6c2800640 1 -- start start 2024-05-30T16:52:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.418+0000 7fb6c2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc0ce1f0 0x7fb6bc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.418+0000 7fb6c2800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6bc0b3580 con 0x7fb6bc0b3d60 2024-05-30T16:52:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.418+0000 7fb6c2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6bc15f490 con 0x7fb6bc0ce5d0 2024-05-30T16:52:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.418+0000 7fb6c2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6bc15f770 con 0x7fb6bc0ce1f0 2024-05-30T16:52:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.419+0000 7fb6c1e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb6bc0b3d60 0x7fb6bc0ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36618/0 (socket says 172.21.15.67:36618) 2024-05-30T16:52:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.419+0000 7fb6c1e00640 1 -- 172.21.15.67:0/875945306 learned_addr learned my addr 172.21.15.67:0/875945306 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.419+0000 7fb6c1400640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc0ce1f0 0x7fb6bc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.419+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 275806667 0 0) 0x7fb6bc0b3580 con 0x7fb6bc0b3d60 2024-05-30T16:52:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.419+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6a4003520 con 0x7fb6bc0b3d60 2024-05-30T16:52:26.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1403528611 0 0) 0x7fb6a4003520 con 0x7fb6bc0b3d60 2024-05-30T16:52:26.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc0ce1f0 msgr2=0x7fb6bc0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:26.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc0ce1f0 0x7fb6bc0b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 >> v1:172.21.15.169:6789/0 conn(0x7fb6bc0ce5d0 legacy=0x7fb6bc15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:26.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc15fa50 con 0x7fb6bc0b3d60 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fb6b40019f0 con 0x7fb6bc0b3d60 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 >> v1:172.21.15.67:6789/0 conn(0x7fb6bc0b3d60 legacy=0x7fb6bc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc0ce1f0 0x7fb6a40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4003f60 0x7fb6a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a4004d40 0x7fb6a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc15f770 con 0x7fb6a4004d40 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a400bc00 con 0x7fb6a4003f60 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a400be00 con 0x7fb6bc0ce1f0 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6c1e00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a4004d40 0x7fb6a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.420+0000 7fb6c1400640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc0ce1f0 0x7fb6a40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.421+0000 7fb6c0a00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4003f60 0x7fb6a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.421+0000 7fb6c1e00640 1 -- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc0ce1f0 msgr2=0x7fb6a40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.421+0000 7fb6c1e00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc0ce1f0 0x7fb6a40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.421+0000 7fb6c1e00640 1 -- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4003f60 msgr2=0x7fb6a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.421+0000 7fb6c1e00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4003f60 0x7fb6a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.421+0000 7fb6c1e00640 1 -- 172.21.15.67:0/875945306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a4003520 con 0x7fb6a4004d40 2024-05-30T16:52:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.421+0000 7fb6c0a00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4003f60 0x7fb6a4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.422+0000 7fb6c1e00640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a4004d40 0x7fb6a40051c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400a860 tx=0x7fb6a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2805906a6abc9d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.422+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b400a480 con 0x7fb6a4004d40 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.422+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b402f070 con 0x7fb6a4004d40 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.422+0000 7fb6bbe00640 1 -- 172.21.15.67:0/875945306 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b402bd20 con 0x7fb6a4004d40 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.423+0000 7fb6c2800640 1 -- 172.21.15.67:0/875945306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a4004d40 msgr2=0x7fb6a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.423+0000 7fb6c2800640 1 --2- 172.21.15.67:0/875945306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a4004d40 0x7fb6a40051c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400a860 tx=0x7fb6a4005830 comp rx=0 tx=0).stop 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.423+0000 7fb6c1e00640 1 -- 172.21.15.67:0/875945306 reap_dead start 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.423+0000 7fb6c2800640 1 -- 172.21.15.67:0/875945306 shutdown_connections 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.423+0000 7fb6c2800640 1 -- 172.21.15.67:0/875945306 >> 172.21.15.67:0/875945306 conn(0x7fb6bc06b900 msgr2=0x7fb6bc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.423+0000 7fb6c2800640 1 -- 172.21.15.67:0/875945306 shutdown_connections 2024-05-30T16:52:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.423+0000 7fb6c2800640 1 -- 172.21.15.67:0/875945306 wait complete. 2024-05-30T16:52:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.425+0000 7fb6c2800640 1 Processor -- start 2024-05-30T16:52:26.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.425+0000 7fb6c2800640 1 -- start start 2024-05-30T16:52:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.426+0000 7fb6c2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4004d40 0x7fb6bc14a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.426+0000 7fb6c2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc14da90 0x7fb6bc161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.426+0000 7fb6c2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc162150 0x7fb6bc09a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.426+0000 7fb6c2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc0cd420 con 0x7fb6bc14da90 2024-05-30T16:52:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.426+0000 7fb6c2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc0cd2a0 con 0x7fb6a4004d40 2024-05-30T16:52:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.426+0000 7fb6c2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6bc0b4dd0 con 0x7fb6bc162150 2024-05-30T16:52:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.426+0000 7fb6c0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc14da90 0x7fb6bc161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.426+0000 7fb6c0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc14da90 0x7fb6bc161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39720/0 (socket says 172.21.15.67:39720) 2024-05-30T16:52:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.426+0000 7fb6c0a00640 1 -- 172.21.15.67:0/3552252190 learned_addr learned my addr 172.21.15.67:0/3552252190 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:26.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.427+0000 7fb6c1400640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4004d40 0x7fb6bc14a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.427+0000 7fb6c1e00640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc162150 0x7fb6bc09a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.427+0000 7fb6c0a00640 1 -- 172.21.15.67:0/3552252190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc162150 msgr2=0x7fb6bc09a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.427+0000 7fb6c0a00640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc162150 0x7fb6bc09a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.427+0000 7fb6c0a00640 1 -- 172.21.15.67:0/3552252190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4004d40 msgr2=0x7fb6bc14a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.427+0000 7fb6c0a00640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4004d40 0x7fb6bc14a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.427+0000 7fb6c0a00640 1 -- 172.21.15.67:0/3552252190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc09c560 con 0x7fb6bc14da90 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.427+0000 7fb6c0a00640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc14da90 0x7fb6bc161c10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800d7d0 tx=0x7fb6a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.428+0000 7fb6b9600640 1 -- 172.21.15.67:0/3552252190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8005070 con 0x7fb6bc14da90 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.428+0000 7fb6b9600640 1 -- 172.21.15.67:0/3552252190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a8005230 con 0x7fb6bc14da90 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.428+0000 7fb6b9600640 1 -- 172.21.15.67:0/3552252190 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8007030 con 0x7fb6bc14da90 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.428+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6bc1625f0 con 0x7fb6bc14da90 2024-05-30T16:52:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.428+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6bc162bb0 con 0x7fb6bc14da90 2024-05-30T16:52:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.429+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6bc0689e0 con 0x7fb6bc14da90 2024-05-30T16:52:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.430+0000 7fb6b9600640 1 -- 172.21.15.67:0/3552252190 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7fb6a801a020 con 0x7fb6bc14da90 2024-05-30T16:52:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.431+0000 7fb6b9600640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69c06d0d0 0x7fb69c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.431+0000 7fb6b9600640 1 -- 172.21.15.67:0/3552252190 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5255+0+0 (secure 0 0 0) 0x7fb6a8089dd0 con 0x7fb6bc14da90 2024-05-30T16:52:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.431+0000 7fb6c1400640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69c06d0d0 0x7fb69c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.432+0000 7fb6c1400640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69c06d0d0 0x7fb69c06f590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009a10 tx=0x7fb6b0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:26.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.436+0000 7fb6b9600640 1 -- 172.21.15.67:0/3552252190 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6a8059af0 con 0x7fb6bc14da90 2024-05-30T16:52:26.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:26.638+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb6bc0734d0 con 0x7fb6bc14da90 2024-05-30T16:52:27.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.193+0000 7fb6b9600640 1 -- 172.21.15.67:0/3552252190 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) ==== 170+0+0 (secure 0 0 0) 0x7fb6a80593f0 con 0x7fb6bc14da90 2024-05-30T16:52:27.195 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-05-30T16:52:27.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69c06d0d0 msgr2=0x7fb69c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:27.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69c06d0d0 0x7fb69c06f590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009a10 tx=0x7fb6b0008040 comp rx=0 tx=0).stop 2024-05-30T16:52:27.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc14da90 msgr2=0x7fb6bc161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:27.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc14da90 0x7fb6bc161c10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800d7d0 tx=0x7fb6a800dc90 comp rx=0 tx=0).stop 2024-05-30T16:52:27.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 shutdown_connections 2024-05-30T16:52:27.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69c06d0d0 0x7fb69c06f590 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:27.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6bc162150 0x7fb6bc09a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:27.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc14da90 0x7fb6bc161c10 secure :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800d7d0 tx=0x7fb6a800dc90 comp rx=0 tx=0).stop 2024-05-30T16:52:27.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 --2- 172.21.15.67:0/3552252190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a4004d40 0x7fb6bc14a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:27.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.198+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 >> 172.21.15.67:0/3552252190 conn(0x7fb6bc06b900 msgr2=0x7fb6bc0ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:27.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.199+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 shutdown_connections 2024-05-30T16:52:27.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.199+0000 7fb6c2800640 1 -- 172.21.15.67:0/3552252190 wait complete. 2024-05-30T16:52:27.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-30T16:52:27.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.414+0000 7fe5c4e00640 1 Processor -- start 2024-05-30T16:52:27.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.415+0000 7fe5c4e00640 1 -- start start 2024-05-30T16:52:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.416+0000 7fe5c4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c00ce1f0 0x7fe5c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.416+0000 7fe5c4e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5c00ce5d0 0x7fe5c015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.416+0000 7fe5c4e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c00b3580 con 0x7fe5c00b3d60 2024-05-30T16:52:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.416+0000 7fe5c4e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c015f490 con 0x7fe5c00ce5d0 2024-05-30T16:52:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.416+0000 7fe5c4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c015f770 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.417+0000 7fe5b7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5c00ce5d0 0x7fe5c015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:27.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.417+0000 7fe5bf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c00ce1f0 0x7fe5c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:27.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.417+0000 7fe5b7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5c00ce5d0 0x7fe5c015ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:42356/0 (socket says 172.21.15.67:42356) 2024-05-30T16:52:27.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.417+0000 7fe5b7e00640 1 -- 172.21.15.67:0/1689257151 learned_addr learned my addr 172.21.15.67:0/1689257151 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.418+0000 7fe5b7e00640 1 -- 172.21.15.67:0/1689257151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c00ce1f0 msgr2=0x7fe5c00b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.418+0000 7fe5b7e00640 1 --2- 172.21.15.67:0/1689257151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c00ce1f0 0x7fe5c00b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.418+0000 7fe5b7e00640 1 -- 172.21.15.67:0/1689257151 >> v1:172.21.15.169:6789/0 conn(0x7fe5c00b3d60 legacy=0x7fe5c00ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:27.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.418+0000 7fe5b7e00640 1 -- 172.21.15.67:0/1689257151 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c015fa50 con 0x7fe5c00ce5d0 2024-05-30T16:52:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.420+0000 7fe5b7e00640 1 --2- 172.21.15.67:0/1689257151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c00ce5d0 0x7fe5c015ec40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009870 tx=0x7fe5a802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ab1728604e816f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:27.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.420+0000 7fe5bea00640 1 -- 172.21.15.67:0/1689257151 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8037070 con 0x7fe5c00ce5d0 2024-05-30T16:52:27.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.421+0000 7fe5bea00640 1 -- 172.21.15.67:0/1689257151 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a8032440 con 0x7fe5c00ce5d0 2024-05-30T16:52:27.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.421+0000 7fe5bea00640 1 -- 172.21.15.67:0/1689257151 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8032810 con 0x7fe5c00ce5d0 2024-05-30T16:52:27.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.421+0000 7fe5c4e00640 1 -- 172.21.15.67:0/1689257151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c00ce5d0 msgr2=0x7fe5c015ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:27.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.421+0000 7fe5c4e00640 1 --2- 172.21.15.67:0/1689257151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c00ce5d0 0x7fe5c015ec40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009870 tx=0x7fe5a802a130 comp rx=0 tx=0).stop 2024-05-30T16:52:27.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.422+0000 7fe5c4e00640 1 -- 172.21.15.67:0/1689257151 shutdown_connections 2024-05-30T16:52:27.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.422+0000 7fe5c4e00640 1 --2- 172.21.15.67:0/1689257151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c00ce5d0 0x7fe5c015ec40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:27.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.422+0000 7fe5c4e00640 1 --2- 172.21.15.67:0/1689257151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c00ce1f0 0x7fe5c00b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:27.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.422+0000 7fe5c4e00640 1 -- 172.21.15.67:0/1689257151 >> 172.21.15.67:0/1689257151 conn(0x7fe5c006b900 msgr2=0x7fe5c006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:27.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.422+0000 7fe5c4e00640 1 -- 172.21.15.67:0/1689257151 shutdown_connections 2024-05-30T16:52:27.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.422+0000 7fe5c4e00640 1 -- 172.21.15.67:0/1689257151 wait complete. 2024-05-30T16:52:27.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.423+0000 7fe5c4e00640 1 Processor -- start 2024-05-30T16:52:27.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.424+0000 7fe5c4e00640 1 -- start start 2024-05-30T16:52:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5c4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c00b3d60 0x7fe5c0158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5c4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c00ce1f0 0x7fe5c00af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5c4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c00ce5d0 0x7fe5c00b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5c4e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c00cd2a0 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5c4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c015f180 con 0x7fe5c00b3d60 2024-05-30T16:52:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5c4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c00cd420 con 0x7fe5c00ce5d0 2024-05-30T16:52:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c00ce1f0 0x7fe5c00af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:27.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c00ce1f0 0x7fe5c00af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39732/0 (socket says 172.21.15.67:39732) 2024-05-30T16:52:27.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5b7e00640 1 -- 172.21.15.67:0/2734056762 learned_addr learned my addr 172.21.15.67:0/2734056762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:27.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5bf400640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c00b3d60 0x7fe5c0158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:27.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5b7e00640 1 -- 172.21.15.67:0/2734056762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c00ce5d0 msgr2=0x7fe5c00b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:27.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5b7e00640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c00ce5d0 0x7fe5c00b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:27.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5b7e00640 1 -- 172.21.15.67:0/2734056762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c00b3d60 msgr2=0x7fe5c0158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:27.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.425+0000 7fe5b7e00640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c00b3d60 0x7fe5c0158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:27.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.426+0000 7fe5b7e00640 1 -- 172.21.15.67:0/2734056762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c01597e0 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.426+0000 7fe5b7e00640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c00ce1f0 0x7fe5c00af030 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80090b0 tx=0x7fe5a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:27.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.426+0000 7fe5b7400640 1 -- 172.21.15.67:0/2734056762 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8043070 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.426+0000 7fe5b7400640 1 -- 172.21.15.67:0/2734056762 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a8038040 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.426+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c0159ac0 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.426+0000 7fe5b7400640 1 -- 172.21.15.67:0/2734056762 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8037050 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.426+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c015a080 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.428+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c00cd0c0 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.429+0000 7fe5b7400640 1 -- 172.21.15.67:0/2734056762 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7fe5a803cb90 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.430+0000 7fe5b7400640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe59c06d0d0 0x7fe59c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:27.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.430+0000 7fe5b7400640 1 -- 172.21.15.67:0/2734056762 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5637+0+0 (secure 0 0 0) 0x7fe5a80ad650 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.430+0000 7fe5bf400640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe59c06d0d0 0x7fe59c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:27.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.431+0000 7fe5bf400640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe59c06d0d0 0x7fe59c06f590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0162f40 tx=0x7fe5b0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:27.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.435+0000 7fe5b7400640 1 -- 172.21.15.67:0/2734056762 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5c00cd0c0 con 0x7fe5c00ce1f0 2024-05-30T16:52:27.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:27.679+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe5c00689e0 con 0x7fe5c00ce1f0 2024-05-30T16:52:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.198+0000 7fe5b7400640 1 -- 172.21.15.67:0/2734056762 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v31) ==== 172+0+0 (secure 0 0 0) 0x7fe5a807a600 con 0x7fe5c00ce1f0 2024-05-30T16:52:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 8 and data pool 9 2024-05-30T16:52:28.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.203+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe59c06d0d0 msgr2=0x7fe59c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.203+0000 7fe5c4e00640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe59c06d0d0 0x7fe59c06f590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0162f40 tx=0x7fe5b0008040 comp rx=0 tx=0).stop 2024-05-30T16:52:28.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.203+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c00ce1f0 msgr2=0x7fe5c00af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.203+0000 7fe5c4e00640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c00ce1f0 0x7fe5c00af030 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80090b0 tx=0x7fe5a80046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.204+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 shutdown_connections 2024-05-30T16:52:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.204+0000 7fe5c4e00640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe59c06d0d0 0x7fe59c06f590 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.204+0000 7fe5c4e00640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c00ce5d0 0x7fe5c00b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.204+0000 7fe5c4e00640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c00ce1f0 0x7fe5c00af030 secure :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80090b0 tx=0x7fe5a80046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.204+0000 7fe5c4e00640 1 --2- 172.21.15.67:0/2734056762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c00b3d60 0x7fe5c0158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.204+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 >> 172.21.15.67:0/2734056762 conn(0x7fe5c006b900 msgr2=0x7fe5c015e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:28.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.207+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 shutdown_connections 2024-05-30T16:52:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.208+0000 7fe5c4e00640 1 -- 172.21.15.67:0/2734056762 wait complete. 2024-05-30T16:52:28.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:52:28.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.421+0000 7f7d02a00640 1 Processor -- start 2024-05-30T16:52:28.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.422+0000 7f7d02a00640 1 -- start start 2024-05-30T16:52:28.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.423+0000 7f7d02a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cfc15e870 0x7f7cfc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.423+0000 7f7d02a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7cfc0d14d0 0x7f7cfc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7cfc0cf870 0x7f7cfc162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7cfc0d2150 con 0x7f7cfc0cf870 2024-05-30T16:52:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7cfc0cfd00 con 0x7f7cfc0d14d0 2024-05-30T16:52:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7cfc15ec50 con 0x7f7cfc15e870 2024-05-30T16:52:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7cfc0cf870 0x7f7cfc162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7cfc0cf870 0x7f7cfc162360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39734/0 (socket says 172.21.15.67:39734) 2024-05-30T16:52:28.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02000640 1 -- 172.21.15.67:0/3589344418 learned_addr learned my addr 172.21.15.67:0/3589344418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:28.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02000640 1 -- 172.21.15.67:0/3589344418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cfc15e870 msgr2=0x7f7cfc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:28.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02000640 1 --2- 172.21.15.67:0/3589344418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cfc15e870 0x7f7cfc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02000640 1 -- 172.21.15.67:0/3589344418 >> v2:172.21.15.169:3300/0 conn(0x7f7cfc0d14d0 msgr2=0x7f7cfc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:28.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02000640 1 --2- 172.21.15.67:0/3589344418 >> v2:172.21.15.169:3300/0 conn(0x7f7cfc0d14d0 0x7f7cfc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.424+0000 7f7d02000640 1 -- 172.21.15.67:0/3589344418 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc0cff80 con 0x7f7cfc0cf870 2024-05-30T16:52:28.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.425+0000 7f7d02000640 1 --2- 172.21.15.67:0/3589344418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 0x7f7cfc162360 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0b7170 tx=0x7f7cf402a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691bcf571d252835 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:28.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7cfbe00640 1 -- 172.21.15.67:0/3589344418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4036070 con 0x7f7cfc0cf870 2024-05-30T16:52:28.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7cfbe00640 1 -- 172.21.15.67:0/3589344418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf4031600 con 0x7f7cfc0cf870 2024-05-30T16:52:28.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7cfbe00640 1 -- 172.21.15.67:0/3589344418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40319d0 con 0x7f7cfc0cf870 2024-05-30T16:52:28.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7d02a00640 1 -- 172.21.15.67:0/3589344418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 msgr2=0x7f7cfc162360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7d02a00640 1 --2- 172.21.15.67:0/3589344418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 0x7f7cfc162360 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0b7170 tx=0x7f7cf402a690 comp rx=0 tx=0).stop 2024-05-30T16:52:28.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7d02a00640 1 -- 172.21.15.67:0/3589344418 shutdown_connections 2024-05-30T16:52:28.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7d02a00640 1 --2- 172.21.15.67:0/3589344418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 0x7f7cfc162360 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7d02a00640 1 --2- 172.21.15.67:0/3589344418 >> v2:172.21.15.169:3300/0 conn(0x7f7cfc0d14d0 0x7f7cfc0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7d02a00640 1 --2- 172.21.15.67:0/3589344418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cfc15e870 0x7f7cfc0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.426+0000 7f7d02a00640 1 -- 172.21.15.67:0/3589344418 >> 172.21.15.67:0/3589344418 conn(0x7f7cfc08bfc0 msgr2=0x7f7cfc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:28.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.427+0000 7f7d02a00640 1 -- 172.21.15.67:0/3589344418 shutdown_connections 2024-05-30T16:52:28.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.427+0000 7f7d02a00640 1 -- 172.21.15.67:0/3589344418 wait complete. 2024-05-30T16:52:28.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.428+0000 7f7d02a00640 1 Processor -- start 2024-05-30T16:52:28.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.428+0000 7f7d02a00640 1 -- start start 2024-05-30T16:52:28.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.429+0000 7f7d02a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 0x7f7cfc1604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.429+0000 7f7d02a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cfc0d14d0 0x7f7cfc0b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.429+0000 7f7d02a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cfc15e870 0x7f7cfc0b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.429+0000 7f7d02a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc0d0820 con 0x7f7cfc0cf870 2024-05-30T16:52:28.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.429+0000 7f7d02a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc0d0540 con 0x7f7cfc0d14d0 2024-05-30T16:52:28.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.429+0000 7f7d02a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7cfc0d4fb0 con 0x7f7cfc15e870 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d01600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 0x7f7cfc1604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d01600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 0x7f7cfc1604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39746/0 (socket says 172.21.15.67:39746) 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d01600640 1 -- 172.21.15.67:0/2405558529 learned_addr learned my addr 172.21.15.67:0/2405558529 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d00c00640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cfc0d14d0 0x7f7cfc0b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d01600640 1 -- 172.21.15.67:0/2405558529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cfc15e870 msgr2=0x7f7cfc0b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d01600640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cfc15e870 0x7f7cfc0b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d01600640 1 -- 172.21.15.67:0/2405558529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cfc0d14d0 msgr2=0x7f7cfc0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d01600640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cfc0d14d0 0x7f7cfc0b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d01600640 1 -- 172.21.15.67:0/2405558529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc0b9290 con 0x7f7cfc0cf870 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.430+0000 7f7d01600640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 0x7f7cfc1604f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000b4d0 tx=0x7f7cf000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.431+0000 7f7cf9600640 1 -- 172.21.15.67:0/2405558529 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0014070 con 0x7f7cfc0cf870 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.431+0000 7f7cf9600640 1 -- 172.21.15.67:0/2405558529 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf0010870 con 0x7f7cfc0cf870 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.431+0000 7f7cf9600640 1 -- 172.21.15.67:0/2405558529 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0010c60 con 0x7f7cfc0cf870 2024-05-30T16:52:28.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.431+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cfc15fa10 con 0x7f7cfc0cf870 2024-05-30T16:52:28.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.431+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc15ffd0 con 0x7f7cfc0cf870 2024-05-30T16:52:28.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.432+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cfc0689e0 con 0x7f7cfc0cf870 2024-05-30T16:52:28.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.433+0000 7f7cf9600640 1 -- 172.21.15.67:0/2405558529 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f7cf0017070 con 0x7f7cfc0cf870 2024-05-30T16:52:28.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.434+0000 7f7cf9600640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cdc06d0d0 0x7f7cdc06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.434+0000 7f7cf9600640 1 -- 172.21.15.67:0/2405558529 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f7cf008cc10 con 0x7f7cfc0cf870 2024-05-30T16:52:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.434+0000 7f7d00c00640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cdc06d0d0 0x7f7cdc06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.435+0000 7f7d00c00640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cdc06d0d0 0x7f7cdc06f590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc15a1d0 tx=0x7f7ce8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.439+0000 7f7cf9600640 1 -- 172.21.15.67:0/2405558529 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7cf005a090 con 0x7f7cfc0cf870 2024-05-30T16:52:28.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.636+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7cfc0ce920 con 0x7f7cfc0cf870 2024-05-30T16:52:28.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.639+0000 7f7cf9600640 1 -- 172.21.15.67:0/2405558529 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16709 (secure 0 0 0) 0x7f7cf0059990 con 0x7f7cfc0cf870 2024-05-30T16:52:28.640 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:28.640 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":61,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:52:27.681519+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:52:25.604028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-30T16:52:26.652938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:52:28.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cdc06d0d0 msgr2=0x7f7cdc06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cdc06d0d0 0x7f7cdc06f590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc15a1d0 tx=0x7f7ce8008040 comp rx=0 tx=0).stop 2024-05-30T16:52:28.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 msgr2=0x7f7cfc1604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 0x7f7cfc1604f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000b4d0 tx=0x7f7cf000b990 comp rx=0 tx=0).stop 2024-05-30T16:52:28.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 shutdown_connections 2024-05-30T16:52:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cdc06d0d0 0x7f7cdc06f590 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cfc15e870 0x7f7cfc0b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cfc0d14d0 0x7f7cfc0b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 --2- 172.21.15.67:0/2405558529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7cfc0cf870 0x7f7cfc1604f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.641+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 >> 172.21.15.67:0/2405558529 conn(0x7f7cfc08bfc0 msgr2=0x7f7cfc0cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.642+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 shutdown_connections 2024-05-30T16:52:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.642+0000 7f7d02a00640 1 -- 172.21.15.67:0/2405558529 wait complete. 2024-05-30T16:52:28.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:52:28.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.859+0000 7f4e6a200640 1 Processor -- start 2024-05-30T16:52:28.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.859+0000 7f4e6a200640 1 -- start start 2024-05-30T16:52:28.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.860+0000 7f4e6a200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4e64156b30 0x7f4e640af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e6a200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e6406ee40 0x7f4e640ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e6a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406f2f0 0x7f4e640ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e6a200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4e64156f10 con 0x7f4e6406ee40 2024-05-30T16:52:28.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e6a200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4e6406dda0 con 0x7f4e64156b30 2024-05-30T16:52:28.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e6a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6406e080 con 0x7f4e6406f2f0 2024-05-30T16:52:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e63e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e6406ee40 0x7f4e640ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e63e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e6406ee40 0x7f4e640ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39756/0 (socket says 172.21.15.67:39756) 2024-05-30T16:52:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e63e00640 1 -- 172.21.15.67:0/2387789345 learned_addr learned my addr 172.21.15.67:0/2387789345 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e68e00640 1 --2- 172.21.15.67:0/2387789345 >> v2:172.21.15.169:3300/0 conn(0x7f4e64156b30 0x7f4e640af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.861+0000 7f4e69800640 1 --2- 172.21.15.67:0/2387789345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406f2f0 0x7f4e640ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.862+0000 7f4e63e00640 1 -- 172.21.15.67:0/2387789345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406f2f0 msgr2=0x7f4e640ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.862+0000 7f4e63e00640 1 --2- 172.21.15.67:0/2387789345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406f2f0 0x7f4e640ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.862+0000 7f4e63e00640 1 -- 172.21.15.67:0/2387789345 >> v2:172.21.15.169:3300/0 conn(0x7f4e64156b30 msgr2=0x7f4e640af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.862+0000 7f4e63e00640 1 --2- 172.21.15.67:0/2387789345 >> v2:172.21.15.169:3300/0 conn(0x7f4e64156b30 0x7f4e640af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.862+0000 7f4e63e00640 1 -- 172.21.15.67:0/2387789345 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6406e360 con 0x7f4e6406ee40 2024-05-30T16:52:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.863+0000 7f4e63e00640 1 --2- 172.21.15.67:0/2387789345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406ee40 0x7f4e640ad230 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4e6406c8b0 tx=0x7f4e5002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b70e5a3d43d535 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.863+0000 7f4e63400640 1 -- 172.21.15.67:0/2387789345 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50037070 con 0x7f4e6406ee40 2024-05-30T16:52:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.863+0000 7f4e63400640 1 -- 172.21.15.67:0/2387789345 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e50032440 con 0x7f4e6406ee40 2024-05-30T16:52:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.863+0000 7f4e63400640 1 -- 172.21.15.67:0/2387789345 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50032810 con 0x7f4e6406ee40 2024-05-30T16:52:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.864+0000 7f4e6a200640 1 -- 172.21.15.67:0/2387789345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406ee40 msgr2=0x7f4e640ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.864+0000 7f4e6a200640 1 --2- 172.21.15.67:0/2387789345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406ee40 0x7f4e640ad230 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4e6406c8b0 tx=0x7f4e5002a140 comp rx=0 tx=0).stop 2024-05-30T16:52:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.864+0000 7f4e6a200640 1 -- 172.21.15.67:0/2387789345 shutdown_connections 2024-05-30T16:52:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.864+0000 7f4e6a200640 1 --2- 172.21.15.67:0/2387789345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406ee40 0x7f4e640ad230 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.864+0000 7f4e6a200640 1 --2- 172.21.15.67:0/2387789345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406f2f0 0x7f4e640ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.864+0000 7f4e6a200640 1 --2- 172.21.15.67:0/2387789345 >> v2:172.21.15.169:3300/0 conn(0x7f4e64156b30 0x7f4e640af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.864+0000 7f4e6a200640 1 -- 172.21.15.67:0/2387789345 >> 172.21.15.67:0/2387789345 conn(0x7f4e640b1fc0 msgr2=0x7f4e640b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.864+0000 7f4e6a200640 1 -- 172.21.15.67:0/2387789345 shutdown_connections 2024-05-30T16:52:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.864+0000 7f4e6a200640 1 -- 172.21.15.67:0/2387789345 wait complete. 2024-05-30T16:52:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.866+0000 7f4e6a200640 1 Processor -- start 2024-05-30T16:52:28.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.866+0000 7f4e6a200640 1 -- start start 2024-05-30T16:52:28.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.866+0000 7f4e6a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406ee40 0x7f4e6414a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e6a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406f2f0 0x7f4e6415c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e6a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e64156b30 0x7f4e6415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e6a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640afc20 con 0x7f4e6406f2f0 2024-05-30T16:52:28.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e6a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640b2c30 con 0x7f4e64156b30 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e6a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4e64073aa0 con 0x7f4e6406ee40 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406f2f0 0x7f4e6415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406f2f0 0x7f4e6415c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39770/0 (socket says 172.21.15.67:39770) 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e63e00640 1 -- 172.21.15.67:0/3975811957 learned_addr learned my addr 172.21.15.67:0/3975811957 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e68e00640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406ee40 0x7f4e6414a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e63e00640 1 -- 172.21.15.67:0/3975811957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406ee40 msgr2=0x7f4e6414a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e69800640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e64156b30 0x7f4e6415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e63e00640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406ee40 0x7f4e6414a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e63e00640 1 -- 172.21.15.67:0/3975811957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e64156b30 msgr2=0x7f4e6415ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e63e00640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e64156b30 0x7f4e6415ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.867+0000 7f4e63e00640 1 -- 172.21.15.67:0/3975811957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e64148e90 con 0x7f4e6406f2f0 2024-05-30T16:52:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.868+0000 7f4e63e00640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406f2f0 0x7f4e6415c920 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4e5002a180 tx=0x7f4e500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.868+0000 7f4e60c00640 1 -- 172.21.15.67:0/3975811957 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50041020 con 0x7f4e6406f2f0 2024-05-30T16:52:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.868+0000 7f4e60c00640 1 -- 172.21.15.67:0/3975811957 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e500306c0 con 0x7f4e6406f2f0 2024-05-30T16:52:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.868+0000 7f4e60c00640 1 -- 172.21.15.67:0/3975811957 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50037040 con 0x7f4e6406f2f0 2024-05-30T16:52:28.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.868+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e641f8050 con 0x7f4e6406f2f0 2024-05-30T16:52:28.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.868+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e641f8610 con 0x7f4e6406f2f0 2024-05-30T16:52:28.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.869+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e640689e0 con 0x7f4e6406f2f0 2024-05-30T16:52:28.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.870+0000 7f4e60c00640 1 -- 172.21.15.67:0/3975811957 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f4e50044020 con 0x7f4e6406f2f0 2024-05-30T16:52:28.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.871+0000 7f4e60c00640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e4406d0d0 0x7f4e4406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:28.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.871+0000 7f4e60c00640 1 -- 172.21.15.67:0/3975811957 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7f4e500ad5e0 con 0x7f4e6406f2f0 2024-05-30T16:52:28.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.871+0000 7f4e68e00640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e4406d0d0 0x7f4e4406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:28.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.872+0000 7f4e68e00640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e4406d0d0 0x7f4e4406f590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4e58006fd0 tx=0x7f4e58008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:28.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:28.876+0000 7f4e60c00640 1 -- 172.21.15.67:0/3975811957 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e5004b030 con 0x7f4e6406f2f0 2024-05-30T16:52:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.075+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e6406f7f0 con 0x7f4e6406f2f0 2024-05-30T16:52:29.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.077+0000 7f4e60c00640 1 -- 172.21.15.67:0/3975811957 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16709 (secure 0 0 0) 0x7f4e5007aa60 con 0x7f4e6406f2f0 2024-05-30T16:52:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:29.079 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":61,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:52:27.681519+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:52:25.604028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-30T16:52:26.652938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:52:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e4406d0d0 msgr2=0x7f4e4406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e4406d0d0 0x7f4e4406f590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4e58006fd0 tx=0x7f4e58008040 comp rx=0 tx=0).stop 2024-05-30T16:52:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406f2f0 msgr2=0x7f4e6415c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406f2f0 0x7f4e6415c920 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4e5002a180 tx=0x7f4e500046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 shutdown_connections 2024-05-30T16:52:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e4406d0d0 0x7f4e4406f590 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e64156b30 0x7f4e6415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6406f2f0 0x7f4e6415c920 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 --2- 172.21.15.67:0/3975811957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406ee40 0x7f4e6414a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.080+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 >> 172.21.15.67:0/3975811957 conn(0x7f4e640b1fc0 msgr2=0x7f4e640b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.081+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 shutdown_connections 2024-05-30T16:52:29.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.081+0000 7f4e6a200640 1 -- 172.21.15.67:0/3975811957 wait complete. 2024-05-30T16:52:29.100 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-30T16:52:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.313+0000 7fe60fe00640 1 Processor -- start 2024-05-30T16:52:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.314+0000 7fe60fe00640 1 -- start start 2024-05-30T16:52:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.315+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 0x7fe610070380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.315+0000 7fe60fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe61015e980 0x7fe61015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.315+0000 7fe60fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6100af480 con 0x7fe61015e5a0 2024-05-30T16:52:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.315+0000 7fe60fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe6100708c0 con 0x7fe61015e980 2024-05-30T16:52:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.315+0000 7fe60fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe6100b4be0 con 0x7fe61006ff00 2024-05-30T16:52:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.315+0000 7fe60f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe61015e980 0x7fe61015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.315+0000 7fe60f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe61015e980 0x7fe61015eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39780/0 (socket says 172.21.15.67:39780) 2024-05-30T16:52:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.315+0000 7fe60f400640 1 -- 172.21.15.67:0/3598132033 learned_addr learned my addr 172.21.15.67:0/3598132033 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.316+0000 7fe60e000640 1 --2- 172.21.15.67:0/3598132033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 0x7fe610070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.316+0000 7fe60f400640 1 -- 172.21.15.67:0/3598132033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 msgr2=0x7fe610070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.316+0000 7fe60f400640 1 --2- 172.21.15.67:0/3598132033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 0x7fe610070380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.316+0000 7fe60f400640 1 -- 172.21.15.67:0/3598132033 >> v1:172.21.15.169:6789/0 conn(0x7fe61015e5a0 legacy=0x7fe6100ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.316+0000 7fe60f400640 1 -- 172.21.15.67:0/3598132033 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6100af760 con 0x7fe61015e980 2024-05-30T16:52:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.317+0000 7fe60f400640 1 --2- 172.21.15.67:0/3598132033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 0x7fe61015eda0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd30bb671ffef1ac server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.317+0000 7fe60d600640 1 -- 172.21.15.67:0/3598132033 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600037070 con 0x7fe61015e980 2024-05-30T16:52:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.317+0000 7fe60d600640 1 -- 172.21.15.67:0/3598132033 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe600032440 con 0x7fe61015e980 2024-05-30T16:52:29.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.317+0000 7fe60d600640 1 -- 172.21.15.67:0/3598132033 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600032810 con 0x7fe61015e980 2024-05-30T16:52:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.318+0000 7fe60fe00640 1 -- 172.21.15.67:0/3598132033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 msgr2=0x7fe61015eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.318+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3598132033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 0x7fe61015eda0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60002a210 comp rx=0 tx=0).stop 2024-05-30T16:52:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.318+0000 7fe60fe00640 1 -- 172.21.15.67:0/3598132033 shutdown_connections 2024-05-30T16:52:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.318+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3598132033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 0x7fe61015eda0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.318+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3598132033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 0x7fe610070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.318+0000 7fe60fe00640 1 -- 172.21.15.67:0/3598132033 >> 172.21.15.67:0/3598132033 conn(0x7fe6100732f0 msgr2=0x7fe61006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.318+0000 7fe60fe00640 1 -- 172.21.15.67:0/3598132033 shutdown_connections 2024-05-30T16:52:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.318+0000 7fe60fe00640 1 -- 172.21.15.67:0/3598132033 wait complete. 2024-05-30T16:52:29.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.319+0000 7fe60fe00640 1 Processor -- start 2024-05-30T16:52:29.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.320+0000 7fe60fe00640 1 -- start start 2024-05-30T16:52:29.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.320+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 0x7fe6101500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:29.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.320+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe61015e5a0 0x7fe6101f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:29.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.320+0000 7fe60fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 0x7fe6101f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:29.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.320+0000 7fe60fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe6100b0000 con 0x7fe61015e980 2024-05-30T16:52:29.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe6100afda0 con 0x7fe61015e5a0 2024-05-30T16:52:29.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe6100b0520 con 0x7fe61006ff00 2024-05-30T16:52:29.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 0x7fe6101f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:29.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 0x7fe6101f7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39794/0 (socket says 172.21.15.67:39794) 2024-05-30T16:52:29.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60f400640 1 -- 172.21.15.67:0/3353516898 learned_addr learned my addr 172.21.15.67:0/3353516898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:29.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60ea00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 0x7fe6101500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:29.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60e000640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe61015e5a0 0x7fe6101f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:29.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60f400640 1 -- 172.21.15.67:0/3353516898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 msgr2=0x7fe6101500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:29.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60f400640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 0x7fe6101500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60f400640 1 -- 172.21.15.67:0/3353516898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe61015e5a0 msgr2=0x7fe6101f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:29.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60f400640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe61015e5a0 0x7fe6101f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:29.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60f400640 1 -- 172.21.15.67:0/3353516898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61015f770 con 0x7fe61015e980 2024-05-30T16:52:29.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.321+0000 7fe60e000640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe61015e5a0 0x7fe6101f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.322+0000 7fe60f400640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 0x7fe6101f7ed0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe6000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.322+0000 7fe606a00640 1 -- 172.21.15.67:0/3353516898 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600041030 con 0x7fe61015e980 2024-05-30T16:52:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.322+0000 7fe606a00640 1 -- 172.21.15.67:0/3353516898 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe600030600 con 0x7fe61015e980 2024-05-30T16:52:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.322+0000 7fe606a00640 1 -- 172.21.15.67:0/3353516898 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600037040 con 0x7fe61015e980 2024-05-30T16:52:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.322+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61015fa10 con 0x7fe61015e980 2024-05-30T16:52:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.322+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61015ffd0 con 0x7fe61015e980 2024-05-30T16:52:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.323+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6100690f0 con 0x7fe61015e980 2024-05-30T16:52:29.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.324+0000 7fe606a00640 1 -- 172.21.15.67:0/3353516898 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7fe600044070 con 0x7fe61015e980 2024-05-30T16:52:29.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.325+0000 7fe606a00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5ec06d0d0 0x7fe5ec06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:29.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.325+0000 7fe606a00640 1 -- 172.21.15.67:0/3353516898 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fe6000ad650 con 0x7fe61015e980 2024-05-30T16:52:29.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.325+0000 7fe60ea00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5ec06d0d0 0x7fe5ec06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:29.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.326+0000 7fe60ea00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5ec06d0d0 0x7fe5ec06f590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe608007fd0 tx=0x7fe608009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:29.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.329+0000 7fe606a00640 1 -- 172.21.15.67:0/3353516898 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe60007aa40 con 0x7fe61015e980 2024-05-30T16:52:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:29.573+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe61006f830 con 0x7fe61015e980 2024-05-30T16:52:30.168 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-05-30T16:52:30.167+0000 7f56ff000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T16:52:30.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.236+0000 7fe606a00640 1 -- 172.21.15.67:0/3353516898 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7fe60007a340 con 0x7fe61015e980 2024-05-30T16:52:30.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.241+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5ec06d0d0 msgr2=0x7fe5ec06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:30.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.241+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5ec06d0d0 0x7fe5ec06f590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe608007fd0 tx=0x7fe608009040 comp rx=0 tx=0).stop 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.241+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 msgr2=0x7fe6101f7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.241+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 0x7fe6101f7ed0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe6000046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.242+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 shutdown_connections 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.242+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5ec06d0d0 0x7fe5ec06f590 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.242+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe61015e980 0x7fe6101f7ed0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.242+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe61015e5a0 0x7fe6101f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.242+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3353516898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe61006ff00 0x7fe6101500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.242+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 >> 172.21.15.67:0/3353516898 conn(0x7fe6100732f0 msgr2=0x7fe61006bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.242+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 shutdown_connections 2024-05-30T16:52:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.242+0000 7fe60fe00640 1 -- 172.21.15.67:0/3353516898 wait complete. 2024-05-30T16:52:30.267 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-30T16:52:30.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-30T16:52:30.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.486+0000 7fbb62800640 1 Processor -- start 2024-05-30T16:52:30.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.486+0000 7fbb62800640 1 -- start start 2024-05-30T16:52:30.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb62800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb5c15e870 0x7fbb5c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:30.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb62800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0d14d0 0x7fbb5c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:30.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb62800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb5c0cf870 0x7fbb5c162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:30.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb62800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5c0d2150 con 0x7fbb5c15e870 2024-05-30T16:52:30.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb62800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5c0cfd00 con 0x7fbb5c0cf870 2024-05-30T16:52:30.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb62800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c15ec50 con 0x7fbb5c0d14d0 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb61400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb5c15e870 0x7fbb5c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb61400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb5c15e870 0x7fbb5c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39806/0 (socket says 172.21.15.67:39806) 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb61400640 1 -- 172.21.15.67:0/1933717152 learned_addr learned my addr 172.21.15.67:0/1933717152 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb61e00640 1 --2- 172.21.15.67:0/1933717152 >> v2:172.21.15.169:3300/0 conn(0x7fbb5c0cf870 0x7fbb5c162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.488+0000 7fbb60a00640 1 --2- 172.21.15.67:0/1933717152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0d14d0 0x7fbb5c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.489+0000 7fbb61400640 1 -- 172.21.15.67:0/1933717152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0d14d0 msgr2=0x7fbb5c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.489+0000 7fbb61400640 1 --2- 172.21.15.67:0/1933717152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0d14d0 0x7fbb5c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.489+0000 7fbb61400640 1 -- 172.21.15.67:0/1933717152 >> v2:172.21.15.169:3300/0 conn(0x7fbb5c0cf870 msgr2=0x7fbb5c162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.489+0000 7fbb61400640 1 --2- 172.21.15.67:0/1933717152 >> v2:172.21.15.169:3300/0 conn(0x7fbb5c0cf870 0x7fbb5c162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.489+0000 7fbb61400640 1 -- 172.21.15.67:0/1933717152 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c0cff80 con 0x7fbb5c15e870 2024-05-30T16:52:30.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.490+0000 7fbb61400640 1 --2- 172.21.15.67:0/1933717152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15e870 0x7fbb5c0d0f90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b3f60 tx=0x7fbb5002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ad1b58e509df91 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:30.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.490+0000 7fbb5be00640 1 -- 172.21.15.67:0/1933717152 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50036070 con 0x7fbb5c15e870 2024-05-30T16:52:30.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.490+0000 7fbb5be00640 1 -- 172.21.15.67:0/1933717152 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb50031600 con 0x7fbb5c15e870 2024-05-30T16:52:30.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.490+0000 7fbb5be00640 1 -- 172.21.15.67:0/1933717152 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb500319d0 con 0x7fbb5c15e870 2024-05-30T16:52:30.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.491+0000 7fbb62800640 1 -- 172.21.15.67:0/1933717152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15e870 msgr2=0x7fbb5c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:30.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.491+0000 7fbb62800640 1 --2- 172.21.15.67:0/1933717152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15e870 0x7fbb5c0d0f90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b3f60 tx=0x7fbb5002a6a0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.491+0000 7fbb62800640 1 -- 172.21.15.67:0/1933717152 shutdown_connections 2024-05-30T16:52:30.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.491+0000 7fbb62800640 1 --2- 172.21.15.67:0/1933717152 >> v2:172.21.15.169:3300/0 conn(0x7fbb5c0cf870 0x7fbb5c162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.491+0000 7fbb62800640 1 --2- 172.21.15.67:0/1933717152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0d14d0 0x7fbb5c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.491+0000 7fbb62800640 1 --2- 172.21.15.67:0/1933717152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15e870 0x7fbb5c0d0f90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.491+0000 7fbb62800640 1 -- 172.21.15.67:0/1933717152 >> 172.21.15.67:0/1933717152 conn(0x7fbb5c08bfc0 msgr2=0x7fbb5c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:30.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.491+0000 7fbb62800640 1 -- 172.21.15.67:0/1933717152 shutdown_connections 2024-05-30T16:52:30.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.491+0000 7fbb62800640 1 -- 172.21.15.67:0/1933717152 wait complete. 2024-05-30T16:52:30.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.493+0000 7fbb62800640 1 Processor -- start 2024-05-30T16:52:30.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.493+0000 7fbb62800640 1 -- start start 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.494+0000 7fbb62800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c0cf870 0x7fbb5c14c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.494+0000 7fbb62800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb5c0d14d0 0x7fbb5c0cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.494+0000 7fbb62800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c15e870 0x7fbb5c0b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.494+0000 7fbb62800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0d0820 con 0x7fbb5c0cf870 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.494+0000 7fbb62800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0d0540 con 0x7fbb5c0d14d0 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.494+0000 7fbb62800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0d4fb0 con 0x7fbb5c15e870 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.494+0000 7fbb61400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c0cf870 0x7fbb5c14c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.494+0000 7fbb61400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c0cf870 0x7fbb5c14c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39812/0 (socket says 172.21.15.67:39812) 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.494+0000 7fbb61400640 1 -- 172.21.15.67:0/2871061543 learned_addr learned my addr 172.21.15.67:0/2871061543 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.495+0000 7fbb61400640 1 -- 172.21.15.67:0/2871061543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c15e870 msgr2=0x7fbb5c0b98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.495+0000 7fbb61400640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c15e870 0x7fbb5c0b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.495+0000 7fbb60a00640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb5c0d14d0 0x7fbb5c0cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.495+0000 7fbb61400640 1 -- 172.21.15.67:0/2871061543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb5c0d14d0 msgr2=0x7fbb5c0cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.495+0000 7fbb61400640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb5c0d14d0 0x7fbb5c0cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.495+0000 7fbb61400640 1 -- 172.21.15.67:0/2871061543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c0cd570 con 0x7fbb5c0cf870 2024-05-30T16:52:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.495+0000 7fbb61400640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c0cf870 0x7fbb5c14c4c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbb50000f80 tx=0x7fbb50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.495+0000 7fbb59600640 1 -- 172.21.15.67:0/2871061543 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50041020 con 0x7fbb5c0cf870 2024-05-30T16:52:30.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.496+0000 7fbb59600640 1 -- 172.21.15.67:0/2871061543 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb50007820 con 0x7fbb5c0cf870 2024-05-30T16:52:30.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.496+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c0ba120 con 0x7fbb5c0cf870 2024-05-30T16:52:30.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.496+0000 7fbb59600640 1 -- 172.21.15.67:0/2871061543 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50036040 con 0x7fbb5c0cf870 2024-05-30T16:52:30.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.496+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c149d10 con 0x7fbb5c0cf870 2024-05-30T16:52:30.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.497+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5c0689e0 con 0x7fbb5c0cf870 2024-05-30T16:52:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.498+0000 7fbb59600640 1 -- 172.21.15.67:0/2871061543 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7fbb50044020 con 0x7fbb5c0cf870 2024-05-30T16:52:30.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.499+0000 7fbb59600640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3c06d0d0 0x7fbb3c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:30.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.499+0000 7fbb59600640 1 -- 172.21.15.67:0/2871061543 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fbb500add90 con 0x7fbb5c0cf870 2024-05-30T16:52:30.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.499+0000 7fbb60a00640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3c06d0d0 0x7fbb3c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:30.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.500+0000 7fbb60a00640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3c06d0d0 0x7fbb3c06f590 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b13d0 tx=0x7fbb48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:30.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.504+0000 7fbb59600640 1 -- 172.21.15.67:0/2871061543 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb5004b030 con 0x7fbb5c0cf870 2024-05-30T16:52:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:30.747+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbb5c0b4570 con 0x7fbb5c0cf870 2024-05-30T16:52:31.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.247+0000 7fbb59600640 1 -- 172.21.15.67:0/2871061543 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fbb5007afe0 con 0x7fbb5c0cf870 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.251+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3c06d0d0 msgr2=0x7fbb3c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.251+0000 7fbb62800640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3c06d0d0 0x7fbb3c06f590 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b13d0 tx=0x7fbb48008040 comp rx=0 tx=0).stop 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.251+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c0cf870 msgr2=0x7fbb5c14c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.251+0000 7fbb62800640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c0cf870 0x7fbb5c14c4c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbb50000f80 tx=0x7fbb50004a40 comp rx=0 tx=0).stop 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.252+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 shutdown_connections 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.252+0000 7fbb62800640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3c06d0d0 0x7fbb3c06f590 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.252+0000 7fbb62800640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c15e870 0x7fbb5c0b98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.252+0000 7fbb62800640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb5c0d14d0 0x7fbb5c0cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.252+0000 7fbb62800640 1 --2- 172.21.15.67:0/2871061543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c0cf870 0x7fbb5c14c4c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.252+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 >> 172.21.15.67:0/2871061543 conn(0x7fbb5c08bfc0 msgr2=0x7fbb5c06df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.252+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 shutdown_connections 2024-05-30T16:52:31.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.252+0000 7fbb62800640 1 -- 172.21.15.67:0/2871061543 wait complete. 2024-05-30T16:52:31.275 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-30T16:52:31.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:52:31.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.485+0000 7faa7dc00640 1 Processor -- start 2024-05-30T16:52:31.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.486+0000 7faa7dc00640 1 -- start start 2024-05-30T16:52:31.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.487+0000 7faa7dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa780d16a0 0x7faa780cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.487+0000 7faa7dc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa780cfce0 con 0x7faa780cf870 2024-05-30T16:52:31.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.487+0000 7faa7dc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa7815ec50 con 0x7faa7815e870 2024-05-30T16:52:31.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.487+0000 7faa7dc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7faa780d2150 con 0x7faa780d16a0 2024-05-30T16:52:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.488+0000 7faa77400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa780d16a0 0x7faa780cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.488+0000 7faa77400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa780d16a0 0x7faa780cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41302/0 (socket says 172.21.15.67:41302) 2024-05-30T16:52:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.488+0000 7faa77400640 1 -- 172.21.15.67:0/2189489395 learned_addr learned my addr 172.21.15.67:0/2189489395 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.488+0000 7faa7d200640 1 --1- 172.21.15.67:0/2189489395 >> v1:172.21.15.67:6789/0 conn(0x7faa780cf870 0x7faa780d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40518/0 (socket says 172.21.15.67:40518) 2024-05-30T16:52:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.488+0000 7faa77400640 1 -- 172.21.15.67:0/2189489395 >> v1:172.21.15.169:6789/0 conn(0x7faa7815e870 legacy=0x7faa780d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.488+0000 7faa77400640 1 -- 172.21.15.67:0/2189489395 >> v1:172.21.15.67:6789/0 conn(0x7faa780cf870 legacy=0x7faa780d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.488+0000 7faa77400640 1 -- 172.21.15.67:0/2189489395 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa780cff80 con 0x7faa780d16a0 2024-05-30T16:52:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.488+0000 7faa76a00640 1 -- 172.21.15.67:0/2189489395 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1340855659 0 0) 0x7faa780cfce0 con 0x7faa780cf870 2024-05-30T16:52:31.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.489+0000 7faa77400640 1 --2- 172.21.15.67:0/2189489395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 0x7faa780cf330 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7faa60009ea0 tx=0x7faa6002a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0c8103d7a513cd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:31.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.490+0000 7faa76a00640 1 -- 172.21.15.67:0/2189489395 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60036070 con 0x7faa780d16a0 2024-05-30T16:52:31.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.490+0000 7faa76a00640 1 -- 172.21.15.67:0/2189489395 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa60031600 con 0x7faa780d16a0 2024-05-30T16:52:31.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.490+0000 7faa76a00640 1 -- 172.21.15.67:0/2189489395 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600319d0 con 0x7faa780d16a0 2024-05-30T16:52:31.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.490+0000 7faa7dc00640 1 -- 172.21.15.67:0/2189489395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 msgr2=0x7faa780cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.490+0000 7faa7dc00640 1 --2- 172.21.15.67:0/2189489395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 0x7faa780cf330 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7faa60009ea0 tx=0x7faa6002a7f0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.490+0000 7faa7dc00640 1 -- 172.21.15.67:0/2189489395 shutdown_connections 2024-05-30T16:52:31.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.490+0000 7faa7dc00640 1 --2- 172.21.15.67:0/2189489395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 0x7faa780cf330 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.490+0000 7faa7dc00640 1 -- 172.21.15.67:0/2189489395 >> 172.21.15.67:0/2189489395 conn(0x7faa7808bfc0 msgr2=0x7faa780d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:31.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.490+0000 7faa7dc00640 1 -- 172.21.15.67:0/2189489395 shutdown_connections 2024-05-30T16:52:31.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.491+0000 7faa7dc00640 1 -- 172.21.15.67:0/2189489395 wait complete. 2024-05-30T16:52:31.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.492+0000 7faa7dc00640 1 Processor -- start 2024-05-30T16:52:31.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.492+0000 7faa7dc00640 1 -- start start 2024-05-30T16:52:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.493+0000 7faa7dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa780cf870 0x7faa780cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.493+0000 7faa7dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 0x7faa780b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.493+0000 7faa7dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7815e870 0x7faa78160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.493+0000 7faa7dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa780d0820 con 0x7faa780d16a0 2024-05-30T16:52:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.493+0000 7faa7dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa780d0540 con 0x7faa7815e870 2024-05-30T16:52:31.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.493+0000 7faa7dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faa780d4fb0 con 0x7faa780cf870 2024-05-30T16:52:31.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa77400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 0x7faa780b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa77400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 0x7faa780b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41312/0 (socket says 172.21.15.67:41312) 2024-05-30T16:52:31.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa77400640 1 -- 172.21.15.67:0/3044065686 learned_addr learned my addr 172.21.15.67:0/3044065686 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:31.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa7d200640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7815e870 0x7faa78160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa77e00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa780cf870 0x7faa780cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa77400640 1 -- 172.21.15.67:0/3044065686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa780cf870 msgr2=0x7faa780cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa77400640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa780cf870 0x7faa780cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa77400640 1 -- 172.21.15.67:0/3044065686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7815e870 msgr2=0x7faa78160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa77400640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7815e870 0x7faa78160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.494+0000 7faa77400640 1 -- 172.21.15.67:0/3044065686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa780b71a0 con 0x7faa780d16a0 2024-05-30T16:52:31.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.495+0000 7faa77400640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 0x7faa780b2a90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faa6002a3d0 tx=0x7faa60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:31.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.495+0000 7faa6fe00640 1 -- 172.21.15.67:0/3044065686 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6003f070 con 0x7faa780d16a0 2024-05-30T16:52:31.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.495+0000 7faa6fe00640 1 -- 172.21.15.67:0/3044065686 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6003b690 con 0x7faa780d16a0 2024-05-30T16:52:31.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.495+0000 7faa6fe00640 1 -- 172.21.15.67:0/3044065686 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60036040 con 0x7faa780d16a0 2024-05-30T16:52:31.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.495+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa780b7460 con 0x7faa780d16a0 2024-05-30T16:52:31.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.495+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa780b7a20 con 0x7faa780d16a0 2024-05-30T16:52:31.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.496+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa780689e0 con 0x7faa780d16a0 2024-05-30T16:52:31.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.497+0000 7faa6fe00640 1 -- 172.21.15.67:0/3044065686 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7faa60039070 con 0x7faa780d16a0 2024-05-30T16:52:31.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.499+0000 7faa6fe00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa5406d0d0 0x7faa5406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.499+0000 7faa6fe00640 1 -- 172.21.15.67:0/3044065686 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7faa600ad0b0 con 0x7faa780d16a0 2024-05-30T16:52:31.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.499+0000 7faa77e00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa5406d0d0 0x7faa5406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.500+0000 7faa77e00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa5406d0d0 0x7faa5406f590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faa68006fd0 tx=0x7faa68008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:31.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.504+0000 7faa6fe00640 1 -- 172.21.15.67:0/3044065686 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa6007a830 con 0x7faa780d16a0 2024-05-30T16:52:31.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.751+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa7815a220 con 0x7faa780d16a0 2024-05-30T16:52:31.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.753+0000 7faa6fe00640 1 -- 172.21.15.67:0/3044065686 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5181 (secure 0 0 0) 0x7faa6007a130 con 0x7faa780d16a0 2024-05-30T16:52:31.754 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:31.754 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2932035652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2932035652},{"type":"v1","addr":"172.21.15.67:6837","nonce":2932035652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/327493818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":327493818},{"type":"v1","addr":"172.21.15.67:6836","nonce":327493818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1437754767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1437754767},{"type":"v1","addr":"172.21.15.169:6834","nonce":1437754767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:52:27.681449+0000","modified":"2024-05-30T16:52:30.749556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/898694760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":898694760},{"type":"v1","addr":"172.21.15.169:6835","nonce":898694760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5937,"qdb_cluster":[5937]},"id":4}]} 2024-05-30T16:52:31.754 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.755+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa5406d0d0 msgr2=0x7faa5406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.755+0000 7faa7dc00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa5406d0d0 0x7faa5406f590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faa68006fd0 tx=0x7faa68008040 comp rx=0 tx=0).stop 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.755+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 msgr2=0x7faa780b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.755+0000 7faa7dc00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 0x7faa780b2a90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faa6002a3d0 tx=0x7faa60004a40 comp rx=0 tx=0).stop 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.756+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 shutdown_connections 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.756+0000 7faa7dc00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa5406d0d0 0x7faa5406f590 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.756+0000 7faa7dc00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7815e870 0x7faa78160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.756+0000 7faa7dc00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa780d16a0 0x7faa780b2a90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.756+0000 7faa7dc00640 1 --2- 172.21.15.67:0/3044065686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa780cf870 0x7faa780cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.756+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 >> 172.21.15.67:0/3044065686 conn(0x7faa7808bfc0 msgr2=0x7faa780d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.756+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 shutdown_connections 2024-05-30T16:52:31.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.756+0000 7faa7dc00640 1 -- 172.21.15.67:0/3044065686 wait complete. 2024-05-30T16:52:31.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:52:31.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.985+0000 7f722b200640 1 Processor -- start 2024-05-30T16:52:31.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.985+0000 7f722b200640 1 -- start start 2024-05-30T16:52:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.986+0000 7f722b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f722c06dc20 0x7f722c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.986+0000 7f722b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f722c0afd60 con 0x7f722c15b3a0 2024-05-30T16:52:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.986+0000 7f722b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f722c0af7a0 con 0x7f722c073310 2024-05-30T16:52:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.986+0000 7f722b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f722c06f980 con 0x7f722c06dc20 2024-05-30T16:52:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.987+0000 7f7229e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f722c15b3a0 0x7f722c072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40530/0 (socket says 172.21.15.67:40530) 2024-05-30T16:52:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.987+0000 7f7229e00640 1 -- 172.21.15.67:0/1571971891 learned_addr learned my addr 172.21.15.67:0/1571971891 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:31.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.987+0000 7f722a800640 1 --2- 172.21.15.67:0/1571971891 >> v2:172.21.15.67:3300/0 conn(0x7f722c06dc20 0x7f722c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.987+0000 7f7228a00640 1 -- 172.21.15.67:0/1571971891 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652169758 0 0) 0x7f722c0afd60 con 0x7f722c15b3a0 2024-05-30T16:52:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.987+0000 7f7228a00640 1 -- 172.21.15.67:0/1571971891 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7210003520 con 0x7f722c15b3a0 2024-05-30T16:52:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.987+0000 7f722a800640 1 -- 172.21.15.67:0/1571971891 >> v1:172.21.15.169:6789/0 conn(0x7f722c073310 legacy=0x7f722c0ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.987+0000 7f722a800640 1 -- 172.21.15.67:0/1571971891 >> v1:172.21.15.67:6789/0 conn(0x7f722c15b3a0 legacy=0x7f722c072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.988+0000 7f722a800640 1 -- 172.21.15.67:0/1571971891 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c0afa80 con 0x7f722c06dc20 2024-05-30T16:52:31.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.989+0000 7f722a800640 1 --2- 172.21.15.67:0/1571971891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c06dc20 0x7f722c0b32c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7224009d90 tx=0x7f722402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fc099b53c8423 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:31.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.989+0000 7f7228a00640 1 -- 172.21.15.67:0/1571971891 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224007740 con 0x7f722c06dc20 2024-05-30T16:52:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.989+0000 7f7228a00640 1 -- 172.21.15.67:0/1571971891 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72240078e0 con 0x7f722c06dc20 2024-05-30T16:52:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.989+0000 7f7228a00640 1 -- 172.21.15.67:0/1571971891 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224007cb0 con 0x7f722c06dc20 2024-05-30T16:52:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.989+0000 7f722b200640 1 -- 172.21.15.67:0/1571971891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c06dc20 msgr2=0x7f722c0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.989+0000 7f722b200640 1 --2- 172.21.15.67:0/1571971891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c06dc20 0x7f722c0b32c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7224009d90 tx=0x7f722402a6f0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.990+0000 7f722b200640 1 -- 172.21.15.67:0/1571971891 shutdown_connections 2024-05-30T16:52:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.990+0000 7f722b200640 1 --2- 172.21.15.67:0/1571971891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c06dc20 0x7f722c0b32c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.990+0000 7f722b200640 1 -- 172.21.15.67:0/1571971891 >> 172.21.15.67:0/1571971891 conn(0x7f722c0b1fc0 msgr2=0x7f722c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.990+0000 7f722b200640 1 -- 172.21.15.67:0/1571971891 shutdown_connections 2024-05-30T16:52:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.990+0000 7f722b200640 1 -- 172.21.15.67:0/1571971891 wait complete. 2024-05-30T16:52:31.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.991+0000 7f722b200640 1 Processor -- start 2024-05-30T16:52:31.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.991+0000 7f722b200640 1 -- start start 2024-05-30T16:52:31.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.992+0000 7f722b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c06dc20 0x7f722c14f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.992+0000 7f722b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f722c073310 0x7f722c14df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.992+0000 7f722b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c15b3a0 0x7f722c099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.992+0000 7f722b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f722c15f9a0 con 0x7f722c15b3a0 2024-05-30T16:52:31.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.992+0000 7f722b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f722c0ae0c0 con 0x7f722c06dc20 2024-05-30T16:52:31.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.992+0000 7f722b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f722c15fb20 con 0x7f722c073310 2024-05-30T16:52:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f722a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c15b3a0 0x7f722c099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f722a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c15b3a0 0x7f722c099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41324/0 (socket says 172.21.15.67:41324) 2024-05-30T16:52:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f722a800640 1 -- 172.21.15.67:0/990097713 learned_addr learned my addr 172.21.15.67:0/990097713 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f7229e00640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c06dc20 0x7f722c14f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f7229400640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f722c073310 0x7f722c14df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f722a800640 1 -- 172.21.15.67:0/990097713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f722c073310 msgr2=0x7f722c14df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f722a800640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f722c073310 0x7f722c14df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f722a800640 1 -- 172.21.15.67:0/990097713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c06dc20 msgr2=0x7f722c14f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f722a800640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c06dc20 0x7f722c14f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f722a800640 1 -- 172.21.15.67:0/990097713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c14e770 con 0x7f722c15b3a0 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.993+0000 7f7229400640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f722c073310 0x7f722c14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.994+0000 7f722a800640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c15b3a0 0x7f722c099850 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f722402ace0 tx=0x7f7224004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.994+0000 7f7222000640 1 -- 172.21.15.67:0/990097713 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224040070 con 0x7f722c15b3a0 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.994+0000 7f7222000640 1 -- 172.21.15.67:0/990097713 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7224031bc0 con 0x7f722c15b3a0 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.994+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722c14ea50 con 0x7f722c15b3a0 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.994+0000 7f7222000640 1 -- 172.21.15.67:0/990097713 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224009450 con 0x7f722c15b3a0 2024-05-30T16:52:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.994+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f722c14f010 con 0x7f722c15b3a0 2024-05-30T16:52:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.995+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722c0689e0 con 0x7f722c15b3a0 2024-05-30T16:52:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.996+0000 7f7222000640 1 -- 172.21.15.67:0/990097713 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f7224039040 con 0x7f722c15b3a0 2024-05-30T16:52:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.997+0000 7f7222000640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f720c06d0d0 0x7f720c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.997+0000 7f7222000640 1 -- 172.21.15.67:0/990097713 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f72240ae680 con 0x7f722c15b3a0 2024-05-30T16:52:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.998+0000 7f7229e00640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f720c06d0d0 0x7f720c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:31.998+0000 7f7229e00640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f720c06d0d0 0x7f720c06f590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f721c0011e0 tx=0x7f721c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.002+0000 7f7222000640 1 -- 172.21.15.67:0/990097713 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7224043360 con 0x7f722c15b3a0 2024-05-30T16:52:32.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.207+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f722c072c00 con 0x7f722c15b3a0 2024-05-30T16:52:32.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.210+0000 7f7222000640 1 -- 172.21.15.67:0/990097713 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16696 (secure 0 0 0) 0x7f722407bbd0 con 0x7f722c15b3a0 2024-05-30T16:52:32.212 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:32.212 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":62,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:52:29.212591+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:52:25.604028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-30T16:52:26.652938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:52:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.213+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f720c06d0d0 msgr2=0x7f720c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.213+0000 7f722b200640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f720c06d0d0 0x7f720c06f590 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f721c0011e0 tx=0x7f721c009040 comp rx=0 tx=0).stop 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.213+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c15b3a0 msgr2=0x7f722c099850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.213+0000 7f722b200640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c15b3a0 0x7f722c099850 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f722402ace0 tx=0x7f7224004a40 comp rx=0 tx=0).stop 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.213+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 shutdown_connections 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.214+0000 7f722b200640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f720c06d0d0 0x7f720c06f590 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.214+0000 7f722b200640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f722c15b3a0 0x7f722c099850 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.214+0000 7f722b200640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f722c073310 0x7f722c14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.214+0000 7f722b200640 1 --2- 172.21.15.67:0/990097713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f722c06dc20 0x7f722c14f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.214+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 >> 172.21.15.67:0/990097713 conn(0x7f722c0b1fc0 msgr2=0x7f722c0b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.214+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 shutdown_connections 2024-05-30T16:52:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.214+0000 7f722b200640 1 -- 172.21.15.67:0/990097713 wait complete. 2024-05-30T16:52:32.235 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T16:52:32.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:52:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.432+0000 7fe252800640 1 Processor -- start 2024-05-30T16:52:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.432+0000 7fe252800640 1 -- start start 2024-05-30T16:52:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.434+0000 7fe252800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe24c158b80 0x7fe24c06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.434+0000 7fe252800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe24c0722d0 0x7fe24c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.434+0000 7fe252800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe24c06e6e0 con 0x7fe24c06e160 2024-05-30T16:52:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.434+0000 7fe252800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe24c06e9c0 con 0x7fe24c158b80 2024-05-30T16:52:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.434+0000 7fe252800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe24c06eca0 con 0x7fe24c0722d0 2024-05-30T16:52:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.434+0000 7fe250a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe24c06e160 0x7fe24c071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40534/0 (socket says 172.21.15.67:40534) 2024-05-30T16:52:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.434+0000 7fe250a00640 1 -- 172.21.15.67:0/3720610221 learned_addr learned my addr 172.21.15.67:0/3720610221 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.434+0000 7fe251400640 1 --2- 172.21.15.67:0/3720610221 >> v2:172.21.15.67:3300/0 conn(0x7fe24c158b80 0x7fe24c06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.435+0000 7fe251e00640 1 --2- 172.21.15.67:0/3720610221 >> v2:172.21.15.169:3300/0 conn(0x7fe24c0722d0 0x7fe24c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.435+0000 7fe24be00640 1 -- 172.21.15.67:0/3720610221 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1923673201 0 0) 0x7fe24c06e6e0 con 0x7fe24c06e160 2024-05-30T16:52:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.435+0000 7fe24be00640 1 -- 172.21.15.67:0/3720610221 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe234003520 con 0x7fe24c06e160 2024-05-30T16:52:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.435+0000 7fe251400640 1 -- 172.21.15.67:0/3720610221 >> v2:172.21.15.169:3300/0 conn(0x7fe24c0722d0 msgr2=0x7fe24c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.435+0000 7fe251400640 1 --2- 172.21.15.67:0/3720610221 >> v2:172.21.15.169:3300/0 conn(0x7fe24c0722d0 0x7fe24c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.435+0000 7fe24be00640 1 -- 172.21.15.67:0/3720610221 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4102544229 0 0) 0x7fe234003520 con 0x7fe24c06e160 2024-05-30T16:52:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.435+0000 7fe251400640 1 -- 172.21.15.67:0/3720610221 >> v1:172.21.15.67:6789/0 conn(0x7fe24c06e160 legacy=0x7fe24c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.435+0000 7fe251400640 1 -- 172.21.15.67:0/3720610221 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c06ef80 con 0x7fe24c158b80 2024-05-30T16:52:32.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.437+0000 7fe251400640 1 --2- 172.21.15.67:0/3720610221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c158b80 0x7fe24c06dc20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe24c06c890 tx=0x7fe24002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16af23a4adde159e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.437+0000 7fe24be00640 1 -- 172.21.15.67:0/3720610221 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240037070 con 0x7fe24c158b80 2024-05-30T16:52:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.437+0000 7fe24be00640 1 -- 172.21.15.67:0/3720610221 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24002abc0 con 0x7fe24c158b80 2024-05-30T16:52:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.437+0000 7fe24be00640 1 -- 172.21.15.67:0/3720610221 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238001670 con 0x7fe24c158b80 2024-05-30T16:52:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.438+0000 7fe252800640 1 -- 172.21.15.67:0/3720610221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c158b80 msgr2=0x7fe24c06dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.438+0000 7fe252800640 1 --2- 172.21.15.67:0/3720610221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c158b80 0x7fe24c06dc20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe24c06c890 tx=0x7fe24002a130 comp rx=0 tx=0).stop 2024-05-30T16:52:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.438+0000 7fe252800640 1 -- 172.21.15.67:0/3720610221 shutdown_connections 2024-05-30T16:52:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.438+0000 7fe252800640 1 --2- 172.21.15.67:0/3720610221 >> v2:172.21.15.169:3300/0 conn(0x7fe24c0722d0 0x7fe24c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.438+0000 7fe252800640 1 --2- 172.21.15.67:0/3720610221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c158b80 0x7fe24c06dc20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.438+0000 7fe252800640 1 -- 172.21.15.67:0/3720610221 >> 172.21.15.67:0/3720610221 conn(0x7fe24c0b1fc0 msgr2=0x7fe24c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.438+0000 7fe252800640 1 -- 172.21.15.67:0/3720610221 shutdown_connections 2024-05-30T16:52:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.438+0000 7fe252800640 1 -- 172.21.15.67:0/3720610221 wait complete. 2024-05-30T16:52:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.440+0000 7fe252800640 1 Processor -- start 2024-05-30T16:52:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.440+0000 7fe252800640 1 -- start start 2024-05-30T16:52:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe252800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c06e160 0x7fe24c09a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe252800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe24c0722d0 0x7fe24c09de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe252800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24c158b80 0x7fe24c14efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe252800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c06f780 con 0x7fe24c06e160 2024-05-30T16:52:32.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe252800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c0b2d30 con 0x7fe24c158b80 2024-05-30T16:52:32.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe252800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe24c09b630 con 0x7fe24c0722d0 2024-05-30T16:52:32.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe251400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c06e160 0x7fe24c09a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe251400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c06e160 0x7fe24c09a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41332/0 (socket says 172.21.15.67:41332) 2024-05-30T16:52:32.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe251400640 1 -- 172.21.15.67:0/1002357208 learned_addr learned my addr 172.21.15.67:0/1002357208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:32.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe251400640 1 -- 172.21.15.67:0/1002357208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe24c0722d0 msgr2=0x7fe24c09de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe251400640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe24c0722d0 0x7fe24c09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe251e00640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24c158b80 0x7fe24c14efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe251400640 1 -- 172.21.15.67:0/1002357208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24c158b80 msgr2=0x7fe24c14efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.441+0000 7fe251400640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24c158b80 0x7fe24c14efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.442+0000 7fe251400640 1 -- 172.21.15.67:0/1002357208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c09e390 con 0x7fe24c06e160 2024-05-30T16:52:32.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.442+0000 7fe251400640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c06e160 0x7fe24c09a3b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe240035040 tx=0x7fe2400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.442+0000 7fe249600640 1 -- 172.21.15.67:0/1002357208 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240041020 con 0x7fe24c06e160 2024-05-30T16:52:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.442+0000 7fe249600640 1 -- 172.21.15.67:0/1002357208 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24003d620 con 0x7fe24c06e160 2024-05-30T16:52:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.442+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24c09aa20 con 0x7fe24c06e160 2024-05-30T16:52:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.442+0000 7fe249600640 1 -- 172.21.15.67:0/1002357208 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240037040 con 0x7fe24c06e160 2024-05-30T16:52:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.442+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24c09afe0 con 0x7fe24c06e160 2024-05-30T16:52:32.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.444+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24c0689e0 con 0x7fe24c06e160 2024-05-30T16:52:32.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.445+0000 7fe249600640 1 -- 172.21.15.67:0/1002357208 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7fe240044020 con 0x7fe24c06e160 2024-05-30T16:52:32.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.445+0000 7fe249600640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe22c06d0d0 0x7fe22c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.446+0000 7fe249600640 1 -- 172.21.15.67:0/1002357208 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fe2400ad360 con 0x7fe24c06e160 2024-05-30T16:52:32.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.446+0000 7fe250a00640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe22c06d0d0 0x7fe22c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.446+0000 7fe250a00640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe22c06d0d0 0x7fe22c06f590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe24c152e50 tx=0x7fe2380011a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:32.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.450+0000 7fe249600640 1 -- 172.21.15.67:0/1002357208 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe24004b030 con 0x7fe24c06e160 2024-05-30T16:52:32.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.649+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe24c0b1d10 con 0x7fe22c06d0d0 2024-05-30T16:52:32.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.660+0000 7fe249600640 1 -- 172.21.15.67:0/1002357208 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251922 (secure 0 0 0) 0x7fe24c0b1d10 con 0x7fe22c06d0d0 2024-05-30T16:52:32.665 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:32.667 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:52:32.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe22c06d0d0 msgr2=0x7fe22c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe22c06d0d0 0x7fe22c06f590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe24c152e50 tx=0x7fe2380011a0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c06e160 msgr2=0x7fe24c09a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c06e160 0x7fe24c09a3b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe240035040 tx=0x7fe2400046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 shutdown_connections 2024-05-30T16:52:32.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe22c06d0d0 0x7fe22c06f590 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe24c158b80 0x7fe24c14efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe24c0722d0 0x7fe24c09de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 --2- 172.21.15.67:0/1002357208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe24c06e160 0x7fe24c09a3b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 >> 172.21.15.67:0/1002357208 conn(0x7fe24c0b1fc0 msgr2=0x7fe24c15f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:32.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.667+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 shutdown_connections 2024-05-30T16:52:32.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.668+0000 7fe252800640 1 -- 172.21.15.67:0/1002357208 wait complete. 2024-05-30T16:52:32.688 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-05-30T16:52:32.187012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342476,"kb_used_data":6936,"kb_used_omap":12,"kb_used_meta":335411,"kb_avail":754632244,"statfs":{"total":773094113280,"available":772743417856,"internally_reserved":0,"allocated":7102464,"data_stored":3644460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":343461424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002150"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231339+0000","last_change":"2024-05-30T16:52:28.236659+0000","last_active":"2024-05-30T16:52:29.231339+0000","last_peered":"2024-05-30T16:52:29.231339+0000","last_clean":"2024-05-30T16:52:29.231339+0000","last_became_active":"2024-05-30T16:52:28.236345+0000","last_became_peered":"2024-05-30T16:52:28.236345+0000","last_unstale":"2024-05-30T16:52:29.231339+0000","last_undegraded":"2024-05-30T16:52:29.231339+0000","last_fullsized":"2024-05-30T16:52:29.231339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:32:55.357340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233664+0000","last_change":"2024-05-30T16:52:28.229747+0000","last_active":"2024-05-30T16:52:29.233664+0000","last_peered":"2024-05-30T16:52:29.233664+0000","last_clean":"2024-05-30T16:52:29.233664+0000","last_became_active":"2024-05-30T16:52:28.229445+0000","last_became_peered":"2024-05-30T16:52:28.229445+0000","last_unstale":"2024-05-30T16:52:29.233664+0000","last_undegraded":"2024-05-30T16:52:29.233664+0000","last_fullsized":"2024-05-30T16:52:29.233664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:41.007887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.228099+0000","last_change":"2024-05-30T16:52:28.225267+0000","last_active":"2024-05-30T16:52:28.228099+0000","last_peered":"2024-05-30T16:52:28.228099+0000","last_clean":"2024-05-30T16:52:28.228099+0000","last_became_active":"2024-05-30T16:52:28.224952+0000","last_became_peered":"2024-05-30T16:52:28.224952+0000","last_unstale":"2024-05-30T16:52:28.228099+0000","last_undegraded":"2024-05-30T16:52:28.228099+0000","last_fullsized":"2024-05-30T16:52:28.228099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:04.705529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.234168+0000","last_change":"2024-05-30T16:52:28.232700+0000","last_active":"2024-05-30T16:52:28.234168+0000","last_peered":"2024-05-30T16:52:28.234168+0000","last_clean":"2024-05-30T16:52:28.234168+0000","last_became_active":"2024-05-30T16:52:28.230687+0000","last_became_peered":"2024-05-30T16:52:28.230687+0000","last_unstale":"2024-05-30T16:52:28.234168+0000","last_undegraded":"2024-05-30T16:52:28.234168+0000","last_fullsized":"2024-05-30T16:52:28.234168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:30.079671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230689+0000","last_change":"2024-05-30T16:52:28.228914+0000","last_active":"2024-05-30T16:52:29.230689+0000","last_peered":"2024-05-30T16:52:29.230689+0000","last_clean":"2024-05-30T16:52:29.230689+0000","last_became_active":"2024-05-30T16:52:28.228598+0000","last_became_peered":"2024-05-30T16:52:28.228598+0000","last_unstale":"2024-05-30T16:52:29.230689+0000","last_undegraded":"2024-05-30T16:52:29.230689+0000","last_fullsized":"2024-05-30T16:52:29.230689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:22.042582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.245237+0000","last_change":"2024-05-30T16:52:28.243282+0000","last_active":"2024-05-30T16:52:28.245237+0000","last_peered":"2024-05-30T16:52:28.245237+0000","last_clean":"2024-05-30T16:52:28.245237+0000","last_became_active":"2024-05-30T16:52:28.243031+0000","last_became_peered":"2024-05-30T16:52:28.243031+0000","last_unstale":"2024-05-30T16:52:28.245237+0000","last_undegraded":"2024-05-30T16:52:28.245237+0000","last_fullsized":"2024-05-30T16:52:28.245237+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:15.325955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.666762+0000","last_change":"2024-05-30T16:52:28.217357+0000","last_active":"2024-05-30T16:52:29.666762+0000","last_peered":"2024-05-30T16:52:29.666762+0000","last_clean":"2024-05-30T16:52:29.666762+0000","last_became_active":"2024-05-30T16:52:28.217064+0000","last_became_peered":"2024-05-30T16:52:28.217064+0000","last_unstale":"2024-05-30T16:52:29.666762+0000","last_undegraded":"2024-05-30T16:52:29.666762+0000","last_fullsized":"2024-05-30T16:52:29.666762+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:36:55.582160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.229662+0000","last_change":"2024-05-30T16:52:28.222400+0000","last_active":"2024-05-30T16:52:29.229662+0000","last_peered":"2024-05-30T16:52:29.229662+0000","last_clean":"2024-05-30T16:52:29.229662+0000","last_became_active":"2024-05-30T16:52:28.222104+0000","last_became_peered":"2024-05-30T16:52:28.222104+0000","last_unstale":"2024-05-30T16:52:29.229662+0000","last_undegraded":"2024-05-30T16:52:29.229662+0000","last_fullsized":"2024-05-30T16:52:29.229662+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:44:46.486607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668033+0000","last_change":"2024-05-30T16:52:28.226294+0000","last_active":"2024-05-30T16:52:29.668033+0000","last_peered":"2024-05-30T16:52:29.668033+0000","last_clean":"2024-05-30T16:52:29.668033+0000","last_became_active":"2024-05-30T16:52:28.225954+0000","last_became_peered":"2024-05-30T16:52:28.225954+0000","last_unstale":"2024-05-30T16:52:29.668033+0000","last_undegraded":"2024-05-30T16:52:29.668033+0000","last_fullsized":"2024-05-30T16:52:29.668033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:18.248572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670544+0000","last_change":"2024-05-30T16:52:28.233808+0000","last_active":"2024-05-30T16:52:29.670544+0000","last_peered":"2024-05-30T16:52:29.670544+0000","last_clean":"2024-05-30T16:52:29.670544+0000","last_became_active":"2024-05-30T16:52:28.233511+0000","last_became_peered":"2024-05-30T16:52:28.233511+0000","last_unstale":"2024-05-30T16:52:29.670544+0000","last_undegraded":"2024-05-30T16:52:29.670544+0000","last_fullsized":"2024-05-30T16:52:29.670544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:46.000242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670508+0000","last_change":"2024-05-30T16:52:28.233868+0000","last_active":"2024-05-30T16:52:29.670508+0000","last_peered":"2024-05-30T16:52:29.670508+0000","last_clean":"2024-05-30T16:52:29.670508+0000","last_became_active":"2024-05-30T16:52:28.233286+0000","last_became_peered":"2024-05-30T16:52:28.233286+0000","last_unstale":"2024-05-30T16:52:29.670508+0000","last_undegraded":"2024-05-30T16:52:29.670508+0000","last_fullsized":"2024-05-30T16:52:29.670508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:08:28.772918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232236+0000","last_change":"2024-05-30T16:52:28.224121+0000","last_active":"2024-05-30T16:52:28.232236+0000","last_peered":"2024-05-30T16:52:28.232236+0000","last_clean":"2024-05-30T16:52:28.232236+0000","last_became_active":"2024-05-30T16:52:28.219408+0000","last_became_peered":"2024-05-30T16:52:28.219408+0000","last_unstale":"2024-05-30T16:52:28.232236+0000","last_undegraded":"2024-05-30T16:52:28.232236+0000","last_fullsized":"2024-05-30T16:52:28.232236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:18.702444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670853+0000","last_change":"2024-05-30T16:52:28.226515+0000","last_active":"2024-05-30T16:52:29.670853+0000","last_peered":"2024-05-30T16:52:29.670853+0000","last_clean":"2024-05-30T16:52:29.670853+0000","last_became_active":"2024-05-30T16:52:28.226070+0000","last_became_peered":"2024-05-30T16:52:28.226070+0000","last_unstale":"2024-05-30T16:52:29.670853+0000","last_undegraded":"2024-05-30T16:52:29.670853+0000","last_fullsized":"2024-05-30T16:52:29.670853+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:25:23.505977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667382+0000","last_change":"2024-05-30T16:52:28.229072+0000","last_active":"2024-05-30T16:52:29.667382+0000","last_peered":"2024-05-30T16:52:29.667382+0000","last_clean":"2024-05-30T16:52:29.667382+0000","last_became_active":"2024-05-30T16:52:28.228799+0000","last_became_peered":"2024-05-30T16:52:28.228799+0000","last_unstale":"2024-05-30T16:52:29.667382+0000","last_undegraded":"2024-05-30T16:52:29.667382+0000","last_fullsized":"2024-05-30T16:52:29.667382+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:40:43.349838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667032+0000","last_change":"2024-05-30T16:52:28.220747+0000","last_active":"2024-05-30T16:52:29.667032+0000","last_peered":"2024-05-30T16:52:29.667032+0000","last_clean":"2024-05-30T16:52:29.667032+0000","last_became_active":"2024-05-30T16:52:28.220458+0000","last_became_peered":"2024-05-30T16:52:28.220458+0000","last_unstale":"2024-05-30T16:52:29.667032+0000","last_undegraded":"2024-05-30T16:52:29.667032+0000","last_fullsized":"2024-05-30T16:52:29.667032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:20.196221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230186+0000","last_change":"2024-05-30T16:52:28.225802+0000","last_active":"2024-05-30T16:52:28.230186+0000","last_peered":"2024-05-30T16:52:28.230186+0000","last_clean":"2024-05-30T16:52:28.230186+0000","last_became_active":"2024-05-30T16:52:28.225366+0000","last_became_peered":"2024-05-30T16:52:28.225366+0000","last_unstale":"2024-05-30T16:52:28.230186+0000","last_undegraded":"2024-05-30T16:52:28.230186+0000","last_fullsized":"2024-05-30T16:52:28.230186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:11:24.392435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.234506+0000","last_change":"2024-05-30T16:52:28.223194+0000","last_active":"2024-05-30T16:52:29.234506+0000","last_peered":"2024-05-30T16:52:29.234506+0000","last_clean":"2024-05-30T16:52:29.234506+0000","last_became_active":"2024-05-30T16:52:28.221888+0000","last_became_peered":"2024-05-30T16:52:28.221888+0000","last_unstale":"2024-05-30T16:52:29.234506+0000","last_undegraded":"2024-05-30T16:52:29.234506+0000","last_fullsized":"2024-05-30T16:52:29.234506+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:26.623648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235333+0000","last_change":"2024-05-30T16:52:28.232643+0000","last_active":"2024-05-30T16:52:28.235333+0000","last_peered":"2024-05-30T16:52:28.235333+0000","last_clean":"2024-05-30T16:52:28.235333+0000","last_became_active":"2024-05-30T16:52:28.231215+0000","last_became_peered":"2024-05-30T16:52:28.231215+0000","last_unstale":"2024-05-30T16:52:28.235333+0000","last_undegraded":"2024-05-30T16:52:28.235333+0000","last_fullsized":"2024-05-30T16:52:28.235333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:15.325955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231247+0000","last_change":"2024-05-30T16:52:28.228070+0000","last_active":"2024-05-30T16:52:29.231247+0000","last_peered":"2024-05-30T16:52:29.231247+0000","last_clean":"2024-05-30T16:52:29.231247+0000","last_became_active":"2024-05-30T16:52:28.227771+0000","last_became_peered":"2024-05-30T16:52:28.227771+0000","last_unstale":"2024-05-30T16:52:29.231247+0000","last_undegraded":"2024-05-30T16:52:29.231247+0000","last_fullsized":"2024-05-30T16:52:29.231247+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:49.844872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233536+0000","last_change":"2024-05-30T16:52:28.227305+0000","last_active":"2024-05-30T16:52:29.233536+0000","last_peered":"2024-05-30T16:52:29.233536+0000","last_clean":"2024-05-30T16:52:29.233536+0000","last_became_active":"2024-05-30T16:52:28.226993+0000","last_became_peered":"2024-05-30T16:52:28.226993+0000","last_unstale":"2024-05-30T16:52:29.233536+0000","last_undegraded":"2024-05-30T16:52:29.233536+0000","last_fullsized":"2024-05-30T16:52:29.233536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:48:49.281817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235274+0000","last_change":"2024-05-30T16:52:28.232418+0000","last_active":"2024-05-30T16:52:28.235274+0000","last_peered":"2024-05-30T16:52:28.235274+0000","last_clean":"2024-05-30T16:52:28.235274+0000","last_became_active":"2024-05-30T16:52:28.231934+0000","last_became_peered":"2024-05-30T16:52:28.231934+0000","last_unstale":"2024-05-30T16:52:28.235274+0000","last_undegraded":"2024-05-30T16:52:28.235274+0000","last_fullsized":"2024-05-30T16:52:28.235274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:13.568129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.229893+0000","last_change":"2024-05-30T16:52:28.232229+0000","last_active":"2024-05-30T16:52:29.229893+0000","last_peered":"2024-05-30T16:52:29.229893+0000","last_clean":"2024-05-30T16:52:29.229893+0000","last_became_active":"2024-05-30T16:52:28.229698+0000","last_became_peered":"2024-05-30T16:52:28.229698+0000","last_unstale":"2024-05-30T16:52:29.229893+0000","last_undegraded":"2024-05-30T16:52:29.229893+0000","last_fullsized":"2024-05-30T16:52:29.229893+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:01.564145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230781+0000","last_change":"2024-05-30T16:52:28.227789+0000","last_active":"2024-05-30T16:52:28.230781+0000","last_peered":"2024-05-30T16:52:28.230781+0000","last_clean":"2024-05-30T16:52:28.230781+0000","last_became_active":"2024-05-30T16:52:28.227492+0000","last_became_peered":"2024-05-30T16:52:28.227492+0000","last_unstale":"2024-05-30T16:52:28.230781+0000","last_undegraded":"2024-05-30T16:52:28.230781+0000","last_fullsized":"2024-05-30T16:52:28.230781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:36.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668578+0000","last_change":"2024-05-30T16:52:28.232008+0000","last_active":"2024-05-30T16:52:29.668578+0000","last_peered":"2024-05-30T16:52:29.668578+0000","last_clean":"2024-05-30T16:52:29.668578+0000","last_became_active":"2024-05-30T16:52:28.230283+0000","last_became_peered":"2024-05-30T16:52:28.230283+0000","last_unstale":"2024-05-30T16:52:29.668578+0000","last_undegraded":"2024-05-30T16:52:29.668578+0000","last_fullsized":"2024-05-30T16:52:29.668578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:47:00.613414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231828+0000","last_change":"2024-05-30T16:52:28.228832+0000","last_active":"2024-05-30T16:52:29.231828+0000","last_peered":"2024-05-30T16:52:29.231828+0000","last_clean":"2024-05-30T16:52:29.231828+0000","last_became_active":"2024-05-30T16:52:28.228498+0000","last_became_peered":"2024-05-30T16:52:28.228498+0000","last_unstale":"2024-05-30T16:52:29.231828+0000","last_undegraded":"2024-05-30T16:52:29.231828+0000","last_fullsized":"2024-05-30T16:52:29.231828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:57:08.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.224326+0000","last_change":"2024-05-30T16:52:28.216836+0000","last_active":"2024-05-30T16:52:28.224326+0000","last_peered":"2024-05-30T16:52:28.224326+0000","last_clean":"2024-05-30T16:52:28.224326+0000","last_became_active":"2024-05-30T16:52:28.216551+0000","last_became_peered":"2024-05-30T16:52:28.216551+0000","last_unstale":"2024-05-30T16:52:28.224326+0000","last_undegraded":"2024-05-30T16:52:28.224326+0000","last_fullsized":"2024-05-30T16:52:28.224326+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:33.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235599+0000","last_change":"2024-05-30T16:52:28.230069+0000","last_active":"2024-05-30T16:52:28.235599+0000","last_peered":"2024-05-30T16:52:28.235599+0000","last_clean":"2024-05-30T16:52:28.235599+0000","last_became_active":"2024-05-30T16:52:28.229721+0000","last_became_peered":"2024-05-30T16:52:28.229721+0000","last_unstale":"2024-05-30T16:52:28.235599+0000","last_undegraded":"2024-05-30T16:52:28.235599+0000","last_fullsized":"2024-05-30T16:52:28.235599+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:36.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231342+0000","last_change":"2024-05-30T16:52:28.235586+0000","last_active":"2024-05-30T16:52:29.231342+0000","last_peered":"2024-05-30T16:52:29.231342+0000","last_clean":"2024-05-30T16:52:29.231342+0000","last_became_active":"2024-05-30T16:52:28.235317+0000","last_became_peered":"2024-05-30T16:52:28.235317+0000","last_unstale":"2024-05-30T16:52:29.231342+0000","last_undegraded":"2024-05-30T16:52:29.231342+0000","last_fullsized":"2024-05-30T16:52:29.231342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:51.107093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233228+0000","last_change":"2024-05-30T16:52:28.220267+0000","last_active":"2024-05-30T16:52:29.233228+0000","last_peered":"2024-05-30T16:52:29.233228+0000","last_clean":"2024-05-30T16:52:29.233228+0000","last_became_active":"2024-05-30T16:52:28.219886+0000","last_became_peered":"2024-05-30T16:52:28.219886+0000","last_unstale":"2024-05-30T16:52:29.233228+0000","last_undegraded":"2024-05-30T16:52:29.233228+0000","last_fullsized":"2024-05-30T16:52:29.233228+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:19:03.977651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.226746+0000","last_change":"2024-05-30T16:52:28.222902+0000","last_active":"2024-05-30T16:52:28.226746+0000","last_peered":"2024-05-30T16:52:28.226746+0000","last_clean":"2024-05-30T16:52:28.226746+0000","last_became_active":"2024-05-30T16:52:28.222572+0000","last_became_peered":"2024-05-30T16:52:28.222572+0000","last_unstale":"2024-05-30T16:52:28.226746+0000","last_undegraded":"2024-05-30T16:52:28.226746+0000","last_fullsized":"2024-05-30T16:52:28.226746+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:18.702444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230696+0000","last_change":"2024-05-30T16:52:28.223983+0000","last_active":"2024-05-30T16:52:29.230696+0000","last_peered":"2024-05-30T16:52:29.230696+0000","last_clean":"2024-05-30T16:52:29.230696+0000","last_became_active":"2024-05-30T16:52:28.223668+0000","last_became_peered":"2024-05-30T16:52:28.223668+0000","last_unstale":"2024-05-30T16:52:29.230696+0000","last_undegraded":"2024-05-30T16:52:29.230696+0000","last_fullsized":"2024-05-30T16:52:29.230696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:38:04.772837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.247401+0000","last_change":"2024-05-30T16:52:28.245829+0000","last_active":"2024-05-30T16:52:28.247401+0000","last_peered":"2024-05-30T16:52:28.247401+0000","last_clean":"2024-05-30T16:52:28.247401+0000","last_became_active":"2024-05-30T16:52:28.245500+0000","last_became_peered":"2024-05-30T16:52:28.245500+0000","last_unstale":"2024-05-30T16:52:28.247401+0000","last_undegraded":"2024-05-30T16:52:28.247401+0000","last_fullsized":"2024-05-30T16:52:28.247401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:28.718841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.245115+0000","last_change":"2024-05-30T16:52:28.240515+0000","last_active":"2024-05-30T16:52:28.245115+0000","last_peered":"2024-05-30T16:52:28.245115+0000","last_clean":"2024-05-30T16:52:28.245115+0000","last_became_active":"2024-05-30T16:52:28.240030+0000","last_became_peered":"2024-05-30T16:52:28.240030+0000","last_unstale":"2024-05-30T16:52:28.245115+0000","last_undegraded":"2024-05-30T16:52:28.245115+0000","last_fullsized":"2024-05-30T16:52:28.245115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:13.568129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.241653+0000","last_change":"2024-05-30T16:52:27.223522+0000","last_active":"2024-05-30T16:52:28.241653+0000","last_peered":"2024-05-30T16:52:28.241653+0000","last_clean":"2024-05-30T16:52:28.241653+0000","last_became_active":"2024-05-30T16:52:27.223234+0000","last_became_peered":"2024-05-30T16:52:27.223234+0000","last_unstale":"2024-05-30T16:52:28.241653+0000","last_undegraded":"2024-05-30T16:52:28.241653+0000","last_fullsized":"2024-05-30T16:52:28.241653+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:29.073369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670863+0000","last_change":"2024-05-30T16:52:28.238004+0000","last_active":"2024-05-30T16:52:29.670863+0000","last_peered":"2024-05-30T16:52:29.670863+0000","last_clean":"2024-05-30T16:52:29.670863+0000","last_became_active":"2024-05-30T16:52:28.237617+0000","last_became_peered":"2024-05-30T16:52:28.237617+0000","last_unstale":"2024-05-30T16:52:29.670863+0000","last_undegraded":"2024-05-30T16:52:29.670863+0000","last_fullsized":"2024-05-30T16:52:29.670863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:20.237839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233991+0000","last_change":"2024-05-30T16:52:27.232664+0000","last_active":"2024-05-30T16:52:29.233991+0000","last_peered":"2024-05-30T16:52:29.233991+0000","last_clean":"2024-05-30T16:52:29.233991+0000","last_became_active":"2024-05-30T16:52:27.232414+0000","last_became_peered":"2024-05-30T16:52:27.232414+0000","last_unstale":"2024-05-30T16:52:29.233991+0000","last_undegraded":"2024-05-30T16:52:29.233991+0000","last_fullsized":"2024-05-30T16:52:29.233991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:30.720551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230177+0000","last_change":"2024-05-30T16:52:28.225173+0000","last_active":"2024-05-30T16:52:28.230177+0000","last_peered":"2024-05-30T16:52:28.230177+0000","last_clean":"2024-05-30T16:52:28.230177+0000","last_became_active":"2024-05-30T16:52:28.224753+0000","last_became_peered":"2024-05-30T16:52:28.224753+0000","last_unstale":"2024-05-30T16:52:28.230177+0000","last_undegraded":"2024-05-30T16:52:28.230177+0000","last_fullsized":"2024-05-30T16:52:28.230177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:48.871632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.247799+0000","last_change":"2024-05-30T16:52:27.224465+0000","last_active":"2024-05-30T16:52:28.247799+0000","last_peered":"2024-05-30T16:52:28.247799+0000","last_clean":"2024-05-30T16:52:28.247799+0000","last_became_active":"2024-05-30T16:52:27.224176+0000","last_became_peered":"2024-05-30T16:52:27.224176+0000","last_unstale":"2024-05-30T16:52:28.247799+0000","last_undegraded":"2024-05-30T16:52:28.247799+0000","last_fullsized":"2024-05-30T16:52:28.247799+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:13:38.253232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.239687+0000","last_change":"2024-05-30T16:52:28.236088+0000","last_active":"2024-05-30T16:52:28.239687+0000","last_peered":"2024-05-30T16:52:28.239687+0000","last_clean":"2024-05-30T16:52:28.239687+0000","last_became_active":"2024-05-30T16:52:28.235840+0000","last_became_peered":"2024-05-30T16:52:28.235840+0000","last_unstale":"2024-05-30T16:52:28.239687+0000","last_undegraded":"2024-05-30T16:52:28.239687+0000","last_fullsized":"2024-05-30T16:52:28.239687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:39:06.774775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235824+0000","last_change":"2024-05-30T16:52:27.232144+0000","last_active":"2024-05-30T16:52:28.235824+0000","last_peered":"2024-05-30T16:52:28.235824+0000","last_clean":"2024-05-30T16:52:28.235824+0000","last_became_active":"2024-05-30T16:52:27.231860+0000","last_became_peered":"2024-05-30T16:52:27.231860+0000","last_unstale":"2024-05-30T16:52:28.235824+0000","last_undegraded":"2024-05-30T16:52:28.235824+0000","last_fullsized":"2024-05-30T16:52:28.235824+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:10.878114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231684+0000","last_change":"2024-05-30T16:52:28.229395+0000","last_active":"2024-05-30T16:52:29.231684+0000","last_peered":"2024-05-30T16:52:29.231684+0000","last_clean":"2024-05-30T16:52:29.231684+0000","last_became_active":"2024-05-30T16:52:28.228882+0000","last_became_peered":"2024-05-30T16:52:28.228882+0000","last_unstale":"2024-05-30T16:52:29.231684+0000","last_undegraded":"2024-05-30T16:52:29.231684+0000","last_fullsized":"2024-05-30T16:52:29.231684+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:05:33.766019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667298+0000","last_change":"2024-05-30T16:52:27.221692+0000","last_active":"2024-05-30T16:52:29.667298+0000","last_peered":"2024-05-30T16:52:29.667298+0000","last_clean":"2024-05-30T16:52:29.667298+0000","last_became_active":"2024-05-30T16:52:27.221222+0000","last_became_peered":"2024-05-30T16:52:27.221222+0000","last_unstale":"2024-05-30T16:52:29.667298+0000","last_undegraded":"2024-05-30T16:52:29.667298+0000","last_fullsized":"2024-05-30T16:52:29.667298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:15:03.761475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233069+0000","last_change":"2024-05-30T16:52:28.227194+0000","last_active":"2024-05-30T16:52:29.233069+0000","last_peered":"2024-05-30T16:52:29.233069+0000","last_clean":"2024-05-30T16:52:29.233069+0000","last_became_active":"2024-05-30T16:52:28.226829+0000","last_became_peered":"2024-05-30T16:52:28.226829+0000","last_unstale":"2024-05-30T16:52:29.233069+0000","last_undegraded":"2024-05-30T16:52:29.233069+0000","last_fullsized":"2024-05-30T16:52:29.233069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:20.196845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.234660+0000","last_change":"2024-05-30T16:52:27.222100+0000","last_active":"2024-05-30T16:52:28.234660+0000","last_peered":"2024-05-30T16:52:28.234660+0000","last_clean":"2024-05-30T16:52:28.234660+0000","last_became_active":"2024-05-30T16:52:27.221718+0000","last_became_peered":"2024-05-30T16:52:27.221718+0000","last_unstale":"2024-05-30T16:52:28.234660+0000","last_undegraded":"2024-05-30T16:52:28.234660+0000","last_fullsized":"2024-05-30T16:52:28.234660+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:32.296488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668635+0000","last_change":"2024-05-30T16:52:28.220825+0000","last_active":"2024-05-30T16:52:29.668635+0000","last_peered":"2024-05-30T16:52:29.668635+0000","last_clean":"2024-05-30T16:52:29.668635+0000","last_became_active":"2024-05-30T16:52:28.220547+0000","last_became_peered":"2024-05-30T16:52:28.220547+0000","last_unstale":"2024-05-30T16:52:29.668635+0000","last_undegraded":"2024-05-30T16:52:29.668635+0000","last_fullsized":"2024-05-30T16:52:29.668635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:03:38.973622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.209872+0000","last_change":"2024-05-30T16:52:27.219695+0000","last_active":"2024-05-30T16:52:28.209872+0000","last_peered":"2024-05-30T16:52:28.209872+0000","last_clean":"2024-05-30T16:52:28.209872+0000","last_became_active":"2024-05-30T16:52:27.219366+0000","last_became_peered":"2024-05-30T16:52:27.219366+0000","last_unstale":"2024-05-30T16:52:28.209872+0000","last_undegraded":"2024-05-30T16:52:28.209872+0000","last_fullsized":"2024-05-30T16:52:28.209872+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:47.865330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230102+0000","last_change":"2024-05-30T16:52:28.216745+0000","last_active":"2024-05-30T16:52:29.230102+0000","last_peered":"2024-05-30T16:52:29.230102+0000","last_clean":"2024-05-30T16:52:29.230102+0000","last_became_active":"2024-05-30T16:52:28.216245+0000","last_became_peered":"2024-05-30T16:52:28.216245+0000","last_unstale":"2024-05-30T16:52:29.230102+0000","last_undegraded":"2024-05-30T16:52:29.230102+0000","last_fullsized":"2024-05-30T16:52:29.230102+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:40:29.196239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.666727+0000","last_change":"2024-05-30T16:52:27.227619+0000","last_active":"2024-05-30T16:52:29.666727+0000","last_peered":"2024-05-30T16:52:29.666727+0000","last_clean":"2024-05-30T16:52:29.666727+0000","last_became_active":"2024-05-30T16:52:27.227326+0000","last_became_peered":"2024-05-30T16:52:27.227326+0000","last_unstale":"2024-05-30T16:52:29.666727+0000","last_undegraded":"2024-05-30T16:52:29.666727+0000","last_fullsized":"2024-05-30T16:52:29.666727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:03:37.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668259+0000","last_change":"2024-05-30T16:52:28.224110+0000","last_active":"2024-05-30T16:52:29.668259+0000","last_peered":"2024-05-30T16:52:29.668259+0000","last_clean":"2024-05-30T16:52:29.668259+0000","last_became_active":"2024-05-30T16:52:28.223620+0000","last_became_peered":"2024-05-30T16:52:28.223620+0000","last_unstale":"2024-05-30T16:52:29.668259+0000","last_undegraded":"2024-05-30T16:52:29.668259+0000","last_fullsized":"2024-05-30T16:52:29.668259+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:00:48.375080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.217209+0000","last_change":"2024-05-30T16:52:27.229397+0000","last_active":"2024-05-30T16:52:28.217209+0000","last_peered":"2024-05-30T16:52:28.217209+0000","last_clean":"2024-05-30T16:52:28.217209+0000","last_became_active":"2024-05-30T16:52:27.229136+0000","last_became_peered":"2024-05-30T16:52:27.229136+0000","last_unstale":"2024-05-30T16:52:28.217209+0000","last_undegraded":"2024-05-30T16:52:28.217209+0000","last_fullsized":"2024-05-30T16:52:28.217209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:32.296488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232494+0000","last_change":"2024-05-30T16:52:28.230499+0000","last_active":"2024-05-30T16:52:28.232494+0000","last_peered":"2024-05-30T16:52:28.232494+0000","last_clean":"2024-05-30T16:52:28.232494+0000","last_became_active":"2024-05-30T16:52:28.230165+0000","last_became_peered":"2024-05-30T16:52:28.230165+0000","last_unstale":"2024-05-30T16:52:28.232494+0000","last_undegraded":"2024-05-30T16:52:28.232494+0000","last_fullsized":"2024-05-30T16:52:28.232494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:19:36.748624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.234149+0000","last_change":"2024-05-30T16:52:27.231492+0000","last_active":"2024-05-30T16:52:29.234149+0000","last_peered":"2024-05-30T16:52:29.234149+0000","last_clean":"2024-05-30T16:52:29.234149+0000","last_became_active":"2024-05-30T16:52:27.231145+0000","last_became_peered":"2024-05-30T16:52:27.231145+0000","last_unstale":"2024-05-30T16:52:29.234149+0000","last_undegraded":"2024-05-30T16:52:29.234149+0000","last_fullsized":"2024-05-30T16:52:29.234149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:03.513990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668714+0000","last_change":"2024-05-30T16:52:28.228720+0000","last_active":"2024-05-30T16:52:29.668714+0000","last_peered":"2024-05-30T16:52:29.668714+0000","last_clean":"2024-05-30T16:52:29.668714+0000","last_became_active":"2024-05-30T16:52:28.222768+0000","last_became_peered":"2024-05-30T16:52:28.222768+0000","last_unstale":"2024-05-30T16:52:29.668714+0000","last_undegraded":"2024-05-30T16:52:29.668714+0000","last_fullsized":"2024-05-30T16:52:29.668714+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:01.515201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.242729+0000","last_change":"2024-05-30T16:52:27.235155+0000","last_active":"2024-05-30T16:52:28.242729+0000","last_peered":"2024-05-30T16:52:28.242729+0000","last_clean":"2024-05-30T16:52:28.242729+0000","last_became_active":"2024-05-30T16:52:27.234883+0000","last_became_peered":"2024-05-30T16:52:27.234883+0000","last_unstale":"2024-05-30T16:52:28.242729+0000","last_undegraded":"2024-05-30T16:52:28.242729+0000","last_fullsized":"2024-05-30T16:52:28.242729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:11:23.386133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.239634+0000","last_change":"2024-05-30T16:52:28.238075+0000","last_active":"2024-05-30T16:52:28.239634+0000","last_peered":"2024-05-30T16:52:28.239634+0000","last_clean":"2024-05-30T16:52:28.239634+0000","last_became_active":"2024-05-30T16:52:28.237775+0000","last_became_peered":"2024-05-30T16:52:28.237775+0000","last_unstale":"2024-05-30T16:52:28.239634+0000","last_undegraded":"2024-05-30T16:52:28.239634+0000","last_fullsized":"2024-05-30T16:52:28.239634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:28.718841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667941+0000","last_change":"2024-05-30T16:52:27.218529+0000","last_active":"2024-05-30T16:52:29.667941+0000","last_peered":"2024-05-30T16:52:29.667941+0000","last_clean":"2024-05-30T16:52:29.667941+0000","last_became_active":"2024-05-30T16:52:27.218063+0000","last_became_peered":"2024-05-30T16:52:27.218063+0000","last_unstale":"2024-05-30T16:52:29.667941+0000","last_undegraded":"2024-05-30T16:52:29.667941+0000","last_fullsized":"2024-05-30T16:52:29.667941+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:59.607113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.229690+0000","last_change":"2024-05-30T16:52:28.223595+0000","last_active":"2024-05-30T16:52:28.229690+0000","last_peered":"2024-05-30T16:52:28.229690+0000","last_clean":"2024-05-30T16:52:28.229690+0000","last_became_active":"2024-05-30T16:52:28.223293+0000","last_became_peered":"2024-05-30T16:52:28.223293+0000","last_unstale":"2024-05-30T16:52:28.229690+0000","last_undegraded":"2024-05-30T16:52:28.229690+0000","last_fullsized":"2024-05-30T16:52:28.229690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:11:24.392435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233161+0000","last_change":"2024-05-30T16:52:27.229883+0000","last_active":"2024-05-30T16:52:29.233161+0000","last_peered":"2024-05-30T16:52:29.233161+0000","last_clean":"2024-05-30T16:52:29.233161+0000","last_became_active":"2024-05-30T16:52:27.228306+0000","last_became_peered":"2024-05-30T16:52:27.228306+0000","last_unstale":"2024-05-30T16:52:29.233161+0000","last_undegraded":"2024-05-30T16:52:29.233161+0000","last_fullsized":"2024-05-30T16:52:29.233161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:49.542333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232555+0000","last_change":"2024-05-30T16:52:28.228631+0000","last_active":"2024-05-30T16:52:28.232555+0000","last_peered":"2024-05-30T16:52:28.232555+0000","last_clean":"2024-05-30T16:52:28.232555+0000","last_became_active":"2024-05-30T16:52:28.225966+0000","last_became_peered":"2024-05-30T16:52:28.225966+0000","last_unstale":"2024-05-30T16:52:28.232555+0000","last_undegraded":"2024-05-30T16:52:28.232555+0000","last_fullsized":"2024-05-30T16:52:28.232555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:30.079671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669125+0000","last_change":"2024-05-30T16:52:27.233165+0000","last_active":"2024-05-30T16:52:29.669125+0000","last_peered":"2024-05-30T16:52:29.669125+0000","last_clean":"2024-05-30T16:52:29.669125+0000","last_became_active":"2024-05-30T16:52:27.232819+0000","last_became_peered":"2024-05-30T16:52:27.232819+0000","last_unstale":"2024-05-30T16:52:29.669125+0000","last_undegraded":"2024-05-30T16:52:29.669125+0000","last_fullsized":"2024-05-30T16:52:29.669125+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:19.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.666727+0000","last_change":"2024-05-30T16:52:28.223161+0000","last_active":"2024-05-30T16:52:29.666727+0000","last_peered":"2024-05-30T16:52:29.666727+0000","last_clean":"2024-05-30T16:52:29.666727+0000","last_became_active":"2024-05-30T16:52:28.222599+0000","last_became_peered":"2024-05-30T16:52:28.222599+0000","last_unstale":"2024-05-30T16:52:29.666727+0000","last_undegraded":"2024-05-30T16:52:29.666727+0000","last_fullsized":"2024-05-30T16:52:29.666727+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:55:05.755482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.208909+0000","last_change":"2024-05-30T16:52:27.229295+0000","last_active":"2024-05-30T16:52:28.208909+0000","last_peered":"2024-05-30T16:52:28.208909+0000","last_clean":"2024-05-30T16:52:28.208909+0000","last_became_active":"2024-05-30T16:52:27.228977+0000","last_became_peered":"2024-05-30T16:52:27.228977+0000","last_unstale":"2024-05-30T16:52:28.208909+0000","last_undegraded":"2024-05-30T16:52:28.208909+0000","last_fullsized":"2024-05-30T16:52:28.208909+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:21.449739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232534+0000","last_change":"2024-05-30T16:52:28.230635+0000","last_active":"2024-05-30T16:52:28.232534+0000","last_peered":"2024-05-30T16:52:28.232534+0000","last_clean":"2024-05-30T16:52:28.232534+0000","last_became_active":"2024-05-30T16:52:28.229433+0000","last_became_peered":"2024-05-30T16:52:28.229433+0000","last_unstale":"2024-05-30T16:52:28.232534+0000","last_undegraded":"2024-05-30T16:52:28.232534+0000","last_fullsized":"2024-05-30T16:52:28.232534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:11.006848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.240815+0000","last_change":"2024-05-30T16:52:27.230985+0000","last_active":"2024-05-30T16:52:28.240815+0000","last_peered":"2024-05-30T16:52:28.240815+0000","last_clean":"2024-05-30T16:52:28.240815+0000","last_became_active":"2024-05-30T16:52:27.230551+0000","last_became_peered":"2024-05-30T16:52:27.230551+0000","last_unstale":"2024-05-30T16:52:28.240815+0000","last_undegraded":"2024-05-30T16:52:28.240815+0000","last_fullsized":"2024-05-30T16:52:28.240815+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:47.865330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232771+0000","last_change":"2024-05-30T16:52:28.228841+0000","last_active":"2024-05-30T16:52:28.232771+0000","last_peered":"2024-05-30T16:52:28.232771+0000","last_clean":"2024-05-30T16:52:28.232771+0000","last_became_active":"2024-05-30T16:52:28.228518+0000","last_became_peered":"2024-05-30T16:52:28.228518+0000","last_unstale":"2024-05-30T16:52:28.232771+0000","last_undegraded":"2024-05-30T16:52:28.232771+0000","last_fullsized":"2024-05-30T16:52:28.232771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:34:59.657491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.234529+0000","last_change":"2024-05-30T16:52:27.229987+0000","last_active":"2024-05-30T16:52:29.234529+0000","last_peered":"2024-05-30T16:52:29.234529+0000","last_clean":"2024-05-30T16:52:29.234529+0000","last_became_active":"2024-05-30T16:52:27.227660+0000","last_became_peered":"2024-05-30T16:52:27.227660+0000","last_unstale":"2024-05-30T16:52:29.234529+0000","last_undegraded":"2024-05-30T16:52:29.234529+0000","last_fullsized":"2024-05-30T16:52:29.234529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:48.577658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.229846+0000","last_change":"2024-05-30T16:52:28.225425+0000","last_active":"2024-05-30T16:52:28.229846+0000","last_peered":"2024-05-30T16:52:28.229846+0000","last_clean":"2024-05-30T16:52:28.229846+0000","last_became_active":"2024-05-30T16:52:28.225152+0000","last_became_peered":"2024-05-30T16:52:28.225152+0000","last_unstale":"2024-05-30T16:52:28.229846+0000","last_undegraded":"2024-05-30T16:52:28.229846+0000","last_fullsized":"2024-05-30T16:52:28.229846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:04.705529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.247944+0000","last_change":"2024-05-30T16:52:27.227849+0000","last_active":"2024-05-30T16:52:28.247944+0000","last_peered":"2024-05-30T16:52:28.247944+0000","last_clean":"2024-05-30T16:52:28.247944+0000","last_became_active":"2024-05-30T16:52:27.227465+0000","last_became_peered":"2024-05-30T16:52:27.227465+0000","last_unstale":"2024-05-30T16:52:28.247944+0000","last_undegraded":"2024-05-30T16:52:28.247944+0000","last_fullsized":"2024-05-30T16:52:28.247944+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:34:58.651189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.234488+0000","last_change":"2024-05-30T16:52:28.220537+0000","last_active":"2024-05-30T16:52:29.234488+0000","last_peered":"2024-05-30T16:52:29.234488+0000","last_clean":"2024-05-30T16:52:29.234488+0000","last_became_active":"2024-05-30T16:52:28.220232+0000","last_became_peered":"2024-05-30T16:52:28.220232+0000","last_unstale":"2024-05-30T16:52:29.234488+0000","last_undegraded":"2024-05-30T16:52:29.234488+0000","last_fullsized":"2024-05-30T16:52:29.234488+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:08.526023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.237004+0000","last_change":"2024-05-30T16:52:27.232326+0000","last_active":"2024-05-30T16:52:28.237004+0000","last_peered":"2024-05-30T16:52:28.237004+0000","last_clean":"2024-05-30T16:52:28.237004+0000","last_became_active":"2024-05-30T16:52:27.232034+0000","last_became_peered":"2024-05-30T16:52:27.232034+0000","last_unstale":"2024-05-30T16:52:28.237004+0000","last_undegraded":"2024-05-30T16:52:28.237004+0000","last_fullsized":"2024-05-30T16:52:28.237004+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:48:01.498972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235444+0000","last_change":"2024-05-30T16:52:28.232285+0000","last_active":"2024-05-30T16:52:28.235444+0000","last_peered":"2024-05-30T16:52:28.235444+0000","last_clean":"2024-05-30T16:52:28.235444+0000","last_became_active":"2024-05-30T16:52:28.231386+0000","last_became_peered":"2024-05-30T16:52:28.231386+0000","last_unstale":"2024-05-30T16:52:28.235444+0000","last_undegraded":"2024-05-30T16:52:28.235444+0000","last_fullsized":"2024-05-30T16:52:28.235444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:39:06.774775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.231462+0000","last_change":"2024-05-30T16:52:27.218134+0000","last_active":"2024-05-30T16:52:28.231462+0000","last_peered":"2024-05-30T16:52:28.231462+0000","last_clean":"2024-05-30T16:52:28.231462+0000","last_became_active":"2024-05-30T16:52:27.217838+0000","last_became_peered":"2024-05-30T16:52:27.217838+0000","last_unstale":"2024-05-30T16:52:28.231462+0000","last_undegraded":"2024-05-30T16:52:28.231462+0000","last_fullsized":"2024-05-30T16:52:28.231462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:10.000547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.231569+0000","last_change":"2024-05-30T16:52:28.227266+0000","last_active":"2024-05-30T16:52:28.231569+0000","last_peered":"2024-05-30T16:52:28.231569+0000","last_clean":"2024-05-30T16:52:28.231569+0000","last_became_active":"2024-05-30T16:52:28.223692+0000","last_became_peered":"2024-05-30T16:52:28.223692+0000","last_unstale":"2024-05-30T16:52:28.231569+0000","last_undegraded":"2024-05-30T16:52:28.231569+0000","last_fullsized":"2024-05-30T16:52:28.231569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:36.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233330+0000","last_change":"2024-05-30T16:52:27.230550+0000","last_active":"2024-05-30T16:52:29.233330+0000","last_peered":"2024-05-30T16:52:29.233330+0000","last_clean":"2024-05-30T16:52:29.233330+0000","last_became_active":"2024-05-30T16:52:27.230236+0000","last_became_peered":"2024-05-30T16:52:27.230236+0000","last_unstale":"2024-05-30T16:52:29.233330+0000","last_undegraded":"2024-05-30T16:52:29.233330+0000","last_fullsized":"2024-05-30T16:52:29.233330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:11:56.573368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.223370+0000","last_change":"2024-05-30T16:52:28.222454+0000","last_active":"2024-05-30T16:52:28.223370+0000","last_peered":"2024-05-30T16:52:28.223370+0000","last_clean":"2024-05-30T16:52:28.223370+0000","last_became_active":"2024-05-30T16:52:28.221376+0000","last_became_peered":"2024-05-30T16:52:28.221376+0000","last_unstale":"2024-05-30T16:52:28.223370+0000","last_undegraded":"2024-05-30T16:52:28.223370+0000","last_fullsized":"2024-05-30T16:52:28.223370+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:15:26.192897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669368+0000","last_change":"2024-05-30T16:52:27.229665+0000","last_active":"2024-05-30T16:52:29.669368+0000","last_peered":"2024-05-30T16:52:29.669368+0000","last_clean":"2024-05-30T16:52:29.669368+0000","last_became_active":"2024-05-30T16:52:27.229334+0000","last_became_peered":"2024-05-30T16:52:27.229334+0000","last_unstale":"2024-05-30T16:52:29.669368+0000","last_undegraded":"2024-05-30T16:52:29.669368+0000","last_fullsized":"2024-05-30T16:52:29.669368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:32:54.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230312+0000","last_change":"2024-05-30T16:52:28.226084+0000","last_active":"2024-05-30T16:52:28.230312+0000","last_peered":"2024-05-30T16:52:28.230312+0000","last_clean":"2024-05-30T16:52:28.230312+0000","last_became_active":"2024-05-30T16:52:28.225511+0000","last_became_peered":"2024-05-30T16:52:28.225511+0000","last_unstale":"2024-05-30T16:52:28.230312+0000","last_undegraded":"2024-05-30T16:52:28.230312+0000","last_fullsized":"2024-05-30T16:52:28.230312+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:18.702444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.207835+0000","last_change":"2024-05-30T16:52:27.221456+0000","last_active":"2024-05-30T16:52:28.207835+0000","last_peered":"2024-05-30T16:52:28.207835+0000","last_clean":"2024-05-30T16:52:28.207835+0000","last_became_active":"2024-05-30T16:52:27.220980+0000","last_became_peered":"2024-05-30T16:52:27.220980+0000","last_unstale":"2024-05-30T16:52:28.207835+0000","last_undegraded":"2024-05-30T16:52:28.207835+0000","last_fullsized":"2024-05-30T16:52:28.207835+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:39:35.473873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230642+0000","last_change":"2024-05-30T16:52:28.228414+0000","last_active":"2024-05-30T16:52:28.230642+0000","last_peered":"2024-05-30T16:52:28.230642+0000","last_clean":"2024-05-30T16:52:28.230642+0000","last_became_active":"2024-05-30T16:52:28.228160+0000","last_became_peered":"2024-05-30T16:52:28.228160+0000","last_unstale":"2024-05-30T16:52:28.230642+0000","last_undegraded":"2024-05-30T16:52:28.230642+0000","last_fullsized":"2024-05-30T16:52:28.230642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:19:36.748624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670928+0000","last_change":"2024-05-30T16:52:27.227842+0000","last_active":"2024-05-30T16:52:29.670928+0000","last_peered":"2024-05-30T16:52:29.670928+0000","last_clean":"2024-05-30T16:52:29.670928+0000","last_became_active":"2024-05-30T16:52:27.227553+0000","last_became_peered":"2024-05-30T16:52:27.227553+0000","last_unstale":"2024-05-30T16:52:29.670928+0000","last_undegraded":"2024-05-30T16:52:29.670928+0000","last_fullsized":"2024-05-30T16:52:29.670928+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:57:07.352356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230645+0000","last_change":"2024-05-30T16:52:28.227492+0000","last_active":"2024-05-30T16:52:28.230645+0000","last_peered":"2024-05-30T16:52:28.230645+0000","last_clean":"2024-05-30T16:52:28.230645+0000","last_became_active":"2024-05-30T16:52:28.226966+0000","last_became_peered":"2024-05-30T16:52:28.226966+0000","last_unstale":"2024-05-30T16:52:28.230645+0000","last_undegraded":"2024-05-30T16:52:28.230645+0000","last_fullsized":"2024-05-30T16:52:28.230645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:34:59.657491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230760+0000","last_change":"2024-05-30T16:52:27.222452+0000","last_active":"2024-05-30T16:52:28.230760+0000","last_peered":"2024-05-30T16:52:28.230760+0000","last_clean":"2024-05-30T16:52:28.230760+0000","last_became_active":"2024-05-30T16:52:27.221459+0000","last_became_peered":"2024-05-30T16:52:27.221459+0000","last_unstale":"2024-05-30T16:52:28.230760+0000","last_undegraded":"2024-05-30T16:52:28.230760+0000","last_fullsized":"2024-05-30T16:52:28.230760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:19:35.742322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669161+0000","last_change":"2024-05-30T16:52:28.221634+0000","last_active":"2024-05-30T16:52:29.669161+0000","last_peered":"2024-05-30T16:52:29.669161+0000","last_clean":"2024-05-30T16:52:29.669161+0000","last_became_active":"2024-05-30T16:52:28.221348+0000","last_became_peered":"2024-05-30T16:52:28.221348+0000","last_unstale":"2024-05-30T16:52:29.669161+0000","last_undegraded":"2024-05-30T16:52:29.669161+0000","last_fullsized":"2024-05-30T16:52:29.669161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:09.342892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669827+0000","last_change":"2024-05-30T16:52:27.225833+0000","last_active":"2024-05-30T16:52:29.669827+0000","last_peered":"2024-05-30T16:52:29.669827+0000","last_clean":"2024-05-30T16:52:29.669827+0000","last_became_active":"2024-05-30T16:52:27.225533+0000","last_became_peered":"2024-05-30T16:52:27.225533+0000","last_unstale":"2024-05-30T16:52:29.669827+0000","last_undegraded":"2024-05-30T16:52:29.669827+0000","last_fullsized":"2024-05-30T16:52:29.669827+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:08:27.766617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235579+0000","last_change":"2024-05-30T16:52:28.227720+0000","last_active":"2024-05-30T16:52:28.235579+0000","last_peered":"2024-05-30T16:52:28.235579+0000","last_clean":"2024-05-30T16:52:28.235579+0000","last_became_active":"2024-05-30T16:52:28.227323+0000","last_became_peered":"2024-05-30T16:52:28.227323+0000","last_unstale":"2024-05-30T16:52:28.235579+0000","last_undegraded":"2024-05-30T16:52:28.235579+0000","last_fullsized":"2024-05-30T16:52:28.235579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:04.705529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.209057+0000","last_change":"2024-05-30T16:52:27.220057+0000","last_active":"2024-05-30T16:52:28.209057+0000","last_peered":"2024-05-30T16:52:28.209057+0000","last_clean":"2024-05-30T16:52:28.209057+0000","last_became_active":"2024-05-30T16:52:27.219483+0000","last_became_peered":"2024-05-30T16:52:27.219483+0000","last_unstale":"2024-05-30T16:52:28.209057+0000","last_undegraded":"2024-05-30T16:52:28.209057+0000","last_fullsized":"2024-05-30T16:52:28.209057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:15:25.186595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.232263+0000","last_change":"2024-05-30T16:52:28.229162+0000","last_active":"2024-05-30T16:52:29.232263+0000","last_peered":"2024-05-30T16:52:29.232263+0000","last_clean":"2024-05-30T16:52:29.232263+0000","last_became_active":"2024-05-30T16:52:28.227949+0000","last_became_peered":"2024-05-30T16:52:28.227949+0000","last_unstale":"2024-05-30T16:52:29.232263+0000","last_undegraded":"2024-05-30T16:52:29.232263+0000","last_fullsized":"2024-05-30T16:52:29.232263+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:54.104397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668739+0000","last_change":"2024-05-30T16:52:27.224539+0000","last_active":"2024-05-30T16:52:29.668739+0000","last_peered":"2024-05-30T16:52:29.668739+0000","last_clean":"2024-05-30T16:52:29.668739+0000","last_became_active":"2024-05-30T16:52:27.224228+0000","last_became_peered":"2024-05-30T16:52:27.224228+0000","last_unstale":"2024-05-30T16:52:29.668739+0000","last_undegraded":"2024-05-30T16:52:29.668739+0000","last_fullsized":"2024-05-30T16:52:29.668739+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:50.100791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.233490+0000","last_change":"2024-05-30T16:52:28.231009+0000","last_active":"2024-05-30T16:52:28.233490+0000","last_peered":"2024-05-30T16:52:28.233490+0000","last_clean":"2024-05-30T16:52:28.233490+0000","last_became_active":"2024-05-30T16:52:28.229859+0000","last_became_peered":"2024-05-30T16:52:28.229859+0000","last_unstale":"2024-05-30T16:52:28.233490+0000","last_undegraded":"2024-05-30T16:52:28.233490+0000","last_fullsized":"2024-05-30T16:52:28.233490+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:13:39.259533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667484+0000","last_change":"2024-05-30T16:52:27.214583+0000","last_active":"2024-05-30T16:52:29.667484+0000","last_peered":"2024-05-30T16:52:29.667484+0000","last_clean":"2024-05-30T16:52:29.667484+0000","last_became_active":"2024-05-30T16:52:27.214182+0000","last_became_peered":"2024-05-30T16:52:27.214182+0000","last_unstale":"2024-05-30T16:52:29.667484+0000","last_undegraded":"2024-05-30T16:52:29.667484+0000","last_fullsized":"2024-05-30T16:52:29.667484+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:39:04.956426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667689+0000","last_change":"2024-05-30T16:52:28.233315+0000","last_active":"2024-05-30T16:52:29.667689+0000","last_peered":"2024-05-30T16:52:29.667689+0000","last_clean":"2024-05-30T16:52:29.667689+0000","last_became_active":"2024-05-30T16:52:28.233041+0000","last_became_peered":"2024-05-30T16:52:28.233041+0000","last_unstale":"2024-05-30T16:52:29.667689+0000","last_undegraded":"2024-05-30T16:52:29.667689+0000","last_fullsized":"2024-05-30T16:52:29.667689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:51:30.479286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668431+0000","last_change":"2024-05-30T16:52:27.227072+0000","last_active":"2024-05-30T16:52:29.668431+0000","last_peered":"2024-05-30T16:52:29.668431+0000","last_clean":"2024-05-30T16:52:29.668431+0000","last_became_active":"2024-05-30T16:52:27.226804+0000","last_became_peered":"2024-05-30T16:52:27.226804+0000","last_unstale":"2024-05-30T16:52:29.668431+0000","last_undegraded":"2024-05-30T16:52:29.668431+0000","last_fullsized":"2024-05-30T16:52:29.668431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:48.838570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230543+0000","last_change":"2024-05-30T16:52:28.216511+0000","last_active":"2024-05-30T16:52:29.230543+0000","last_peered":"2024-05-30T16:52:29.230543+0000","last_clean":"2024-05-30T16:52:29.230543+0000","last_became_active":"2024-05-30T16:52:28.215895+0000","last_became_peered":"2024-05-30T16:52:28.215895+0000","last_unstale":"2024-05-30T16:52:29.230543+0000","last_undegraded":"2024-05-30T16:52:29.230543+0000","last_fullsized":"2024-05-30T16:52:29.230543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:13:02.686629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230079+0000","last_change":"2024-05-30T16:52:27.224814+0000","last_active":"2024-05-30T16:52:29.230079+0000","last_peered":"2024-05-30T16:52:29.230079+0000","last_clean":"2024-05-30T16:52:29.230079+0000","last_became_active":"2024-05-30T16:52:27.224354+0000","last_became_peered":"2024-05-30T16:52:27.224354+0000","last_unstale":"2024-05-30T16:52:29.230079+0000","last_undegraded":"2024-05-30T16:52:29.230079+0000","last_fullsized":"2024-05-30T16:52:29.230079+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:43:58.690696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.229631+0000","last_change":"2024-05-30T16:52:28.238951+0000","last_active":"2024-05-30T16:52:29.229631+0000","last_peered":"2024-05-30T16:52:29.229631+0000","last_clean":"2024-05-30T16:52:29.229631+0000","last_became_active":"2024-05-30T16:52:28.234929+0000","last_became_peered":"2024-05-30T16:52:28.234929+0000","last_unstale":"2024-05-30T16:52:29.229631+0000","last_undegraded":"2024-05-30T16:52:29.229631+0000","last_fullsized":"2024-05-30T16:52:29.229631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:56.945930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669009+0000","last_change":"2024-05-30T16:52:27.223659+0000","last_active":"2024-05-30T16:52:29.669009+0000","last_peered":"2024-05-30T16:52:29.669009+0000","last_clean":"2024-05-30T16:52:29.669009+0000","last_became_active":"2024-05-30T16:52:27.223321+0000","last_became_peered":"2024-05-30T16:52:27.223321+0000","last_unstale":"2024-05-30T16:52:29.669009+0000","last_undegraded":"2024-05-30T16:52:29.669009+0000","last_fullsized":"2024-05-30T16:52:29.669009+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:44.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":247,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.214126+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:52:28.214126+0000","last_peered":"2024-05-30T16:52:28.214126+0000","last_clean":"2024-05-30T16:52:28.214126+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:52:28.214126+0000","last_undegraded":"2024-05-30T16:52:28.214126+0000","last_fullsized":"2024-05-30T16:52:28.214126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509549,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45788,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326052,"statfs":{"total":96636764160,"available":96589877248,"internally_reserved":0,"allocated":737280,"data_stored":310527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45788,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326052,"statfs":{"total":96636764160,"available":96589877248,"internally_reserved":0,"allocated":737280,"data_stored":305811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509550,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45796,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326044,"statfs":{"total":96636764160,"available":96589869056,"internally_reserved":0,"allocated":745472,"data_stored":310583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":310583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509549,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45796,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326044,"statfs":{"total":96636764160,"available":96589869056,"internally_reserved":0,"allocated":741376,"data_stored":310617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542252,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44960,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326880,"statfs":{"total":96636764160,"available":96590725120,"internally_reserved":0,"allocated":1327104,"data_stored":894727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542253,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40216,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331624,"statfs":{"total":96636764160,"available":96595582976,"internally_reserved":0,"allocated":733184,"data_stored":304851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542253,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46380,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325460,"statfs":{"total":96636764160,"available":96589271040,"internally_reserved":0,"allocated":1335296,"data_stored":896761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:52:32.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:52:32.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.942+0000 7f606ee00640 1 Processor -- start 2024-05-30T16:52:32.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.943+0000 7f606ee00640 1 -- start start 2024-05-30T16:52:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.944+0000 7f606ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60680d16a0 0x7f60680cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.944+0000 7f606ee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606815ec50 con 0x7f606815e870 2024-05-30T16:52:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.944+0000 7f606ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60680cff80 con 0x7f60680cf870 2024-05-30T16:52:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.944+0000 7f606ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f60680d2150 con 0x7f60680d16a0 2024-05-30T16:52:32.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f606da00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f606815e870 0x7f60680d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40548/0 (socket says 172.21.15.67:40548) 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f606da00640 1 -- 172.21.15.67:0/117524217 learned_addr learned my addr 172.21.15.67:0/117524217 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f606d000640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60680d16a0 0x7f60680cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2288921708 0 0) 0x7f606815ec50 con 0x7f606815e870 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6050003520 con 0x7f606815e870 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1254450121 0 0) 0x7f6050003520 con 0x7f606815e870 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60680d16a0 msgr2=0x7f60680cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f6067e00640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60680d16a0 0x7f60680cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 >> v1:172.21.15.169:6789/0 conn(0x7f60680cf870 legacy=0x7f60680d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.945+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60680d0260 con 0x7f606815e870 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f605c001670 con 0x7f606815e870 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 >> v1:172.21.15.67:6789/0 conn(0x7f606815e870 legacy=0x7f60680d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f6067e00640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60500039e0 0x7f6050003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f6067e00640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60680d16a0 0x7f6050004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f6067e00640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6050004d40 0x7f60500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f60680d2150 con 0x7f6050004d40 2024-05-30T16:52:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f605000ba70 con 0x7f60500039e0 2024-05-30T16:52:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f605000bc70 con 0x7f60680d16a0 2024-05-30T16:52:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f606e400640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6050004d40 0x7f60500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f606da00640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60500039e0 0x7f6050003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f606d000640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60680d16a0 0x7f6050004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f606e400640 1 -- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60680d16a0 msgr2=0x7f6050004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:32.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.946+0000 7f606e400640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60680d16a0 0x7f6050004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.947+0000 7f606e400640 1 -- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60500039e0 msgr2=0x7f6050003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:32.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.947+0000 7f606e400640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60500039e0 0x7f6050003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.947+0000 7f606e400640 1 -- 172.21.15.67:0/117524217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6050003520 con 0x7f6050004d40 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.948+0000 7f606e400640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6050004d40 0x7f60500051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6060001e20 tx=0x7f6050005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399111005fa2e9fe server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.948+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060001a70 con 0x7f6050004d40 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.948+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606002ae40 con 0x7f6050004d40 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.948+0000 7f6067e00640 1 -- 172.21.15.67:0/117524217 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060003bb0 con 0x7f6050004d40 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.949+0000 7f606ee00640 1 -- 172.21.15.67:0/117524217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6050004d40 msgr2=0x7f60500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.949+0000 7f606ee00640 1 --2- 172.21.15.67:0/117524217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6050004d40 0x7f60500051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6060001e20 tx=0x7f6050005830 comp rx=0 tx=0).stop 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.949+0000 7f606e400640 1 -- 172.21.15.67:0/117524217 reap_dead start 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.949+0000 7f606ee00640 1 -- 172.21.15.67:0/117524217 shutdown_connections 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.949+0000 7f606ee00640 1 -- 172.21.15.67:0/117524217 >> 172.21.15.67:0/117524217 conn(0x7f606808bfc0 msgr2=0x7f60680d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.949+0000 7f606ee00640 1 -- 172.21.15.67:0/117524217 shutdown_connections 2024-05-30T16:52:32.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.949+0000 7f606ee00640 1 -- 172.21.15.67:0/117524217 wait complete. 2024-05-30T16:52:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.951+0000 7f606ee00640 1 Processor -- start 2024-05-30T16:52:32.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.951+0000 7f606ee00640 1 -- start start 2024-05-30T16:52:32.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6050004d40 0x7f606815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606815b9d0 0x7f60680ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606ee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60680cdd00 0x7f60680ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606ee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f60680d0820 con 0x7f60680cdd00 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60680cfd20 con 0x7f6050004d40 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f60680d4fb0 con 0x7f606815b9d0 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60680cdd00 0x7f60680ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60680cdd00 0x7f60680ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41344/0 (socket says 172.21.15.67:41344) 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606e400640 1 -- 172.21.15.67:0/2734883762 learned_addr learned my addr 172.21.15.67:0/2734883762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606da00640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6050004d40 0x7f606815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.952+0000 7f606d000640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606815b9d0 0x7f60680ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f606e400640 1 -- 172.21.15.67:0/2734883762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606815b9d0 msgr2=0x7f60680ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f606e400640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606815b9d0 0x7f60680ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f606e400640 1 -- 172.21.15.67:0/2734883762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6050004d40 msgr2=0x7f606815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f606e400640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6050004d40 0x7f606815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f606e400640 1 -- 172.21.15.67:0/2734883762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60680ccca0 con 0x7f60680cdd00 2024-05-30T16:52:32.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f606d000640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606815b9d0 0x7f60680ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:32.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f606e400640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60680cdd00 0x7f60680ce180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6060001e20 tx=0x7f60600014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:32.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f6065600640 1 -- 172.21.15.67:0/2734883762 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060001a70 con 0x7f60680cdd00 2024-05-30T16:52:32.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f6065600640 1 -- 172.21.15.67:0/2734883762 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6060043070 con 0x7f60680cdd00 2024-05-30T16:52:32.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606815be30 con 0x7f60680cdd00 2024-05-30T16:52:32.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.953+0000 7f6065600640 1 -- 172.21.15.67:0/2734883762 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060007db0 con 0x7f60680cdd00 2024-05-30T16:52:32.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.954+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60680cd220 con 0x7f60680cdd00 2024-05-30T16:52:32.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.955+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6068163180 con 0x7f60680cdd00 2024-05-30T16:52:32.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.956+0000 7f6065600640 1 -- 172.21.15.67:0/2734883762 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f6060048460 con 0x7f60680cdd00 2024-05-30T16:52:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.957+0000 7f6065600640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604806d100 0x7f604806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.957+0000 7f6065600640 1 -- 172.21.15.67:0/2734883762 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f60600aab20 con 0x7f60680cdd00 2024-05-30T16:52:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.957+0000 7f606da00640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604806d100 0x7f604806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:32.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.958+0000 7f606da00640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604806d100 0x7f604806f5c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f605c001cd0 tx=0x7f605c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:32.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:32.961+0000 7f6065600640 1 -- 172.21.15.67:0/2734883762 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6068163180 con 0x7f60680cdd00 2024-05-30T16:52:33.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.162+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60680ba500 con 0x7f604806d100 2024-05-30T16:52:33.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.170+0000 7f6065600640 1 -- 172.21.15.67:0/2734883762 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251922 (secure 0 0 0) 0x7f60680ba500 con 0x7f604806d100 2024-05-30T16:52:33.175 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:33.176 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:52:33.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604806d100 msgr2=0x7f604806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604806d100 0x7f604806f5c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f605c001cd0 tx=0x7f605c009040 comp rx=0 tx=0).stop 2024-05-30T16:52:33.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60680cdd00 msgr2=0x7f60680ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60680cdd00 0x7f60680ce180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6060001e20 tx=0x7f60600014a0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 shutdown_connections 2024-05-30T16:52:33.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604806d100 0x7f604806f5c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60680cdd00 0x7f60680ce180 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606815b9d0 0x7f60680ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 --2- 172.21.15.67:0/2734883762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6050004d40 0x7f606815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 >> 172.21.15.67:0/2734883762 conn(0x7f606808bfc0 msgr2=0x7f606815d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:33.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 shutdown_connections 2024-05-30T16:52:33.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.176+0000 7f606ee00640 1 -- 172.21.15.67:0/2734883762 wait complete. 2024-05-30T16:52:33.197 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-05-30T16:52:32.187012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342476,"kb_used_data":6936,"kb_used_omap":12,"kb_used_meta":335411,"kb_avail":754632244,"statfs":{"total":773094113280,"available":772743417856,"internally_reserved":0,"allocated":7102464,"data_stored":3644460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":343461424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002150"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231339+0000","last_change":"2024-05-30T16:52:28.236659+0000","last_active":"2024-05-30T16:52:29.231339+0000","last_peered":"2024-05-30T16:52:29.231339+0000","last_clean":"2024-05-30T16:52:29.231339+0000","last_became_active":"2024-05-30T16:52:28.236345+0000","last_became_peered":"2024-05-30T16:52:28.236345+0000","last_unstale":"2024-05-30T16:52:29.231339+0000","last_undegraded":"2024-05-30T16:52:29.231339+0000","last_fullsized":"2024-05-30T16:52:29.231339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:32:55.357340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233664+0000","last_change":"2024-05-30T16:52:28.229747+0000","last_active":"2024-05-30T16:52:29.233664+0000","last_peered":"2024-05-30T16:52:29.233664+0000","last_clean":"2024-05-30T16:52:29.233664+0000","last_became_active":"2024-05-30T16:52:28.229445+0000","last_became_peered":"2024-05-30T16:52:28.229445+0000","last_unstale":"2024-05-30T16:52:29.233664+0000","last_undegraded":"2024-05-30T16:52:29.233664+0000","last_fullsized":"2024-05-30T16:52:29.233664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:41.007887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.228099+0000","last_change":"2024-05-30T16:52:28.225267+0000","last_active":"2024-05-30T16:52:28.228099+0000","last_peered":"2024-05-30T16:52:28.228099+0000","last_clean":"2024-05-30T16:52:28.228099+0000","last_became_active":"2024-05-30T16:52:28.224952+0000","last_became_peered":"2024-05-30T16:52:28.224952+0000","last_unstale":"2024-05-30T16:52:28.228099+0000","last_undegraded":"2024-05-30T16:52:28.228099+0000","last_fullsized":"2024-05-30T16:52:28.228099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:04.705529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.234168+0000","last_change":"2024-05-30T16:52:28.232700+0000","last_active":"2024-05-30T16:52:28.234168+0000","last_peered":"2024-05-30T16:52:28.234168+0000","last_clean":"2024-05-30T16:52:28.234168+0000","last_became_active":"2024-05-30T16:52:28.230687+0000","last_became_peered":"2024-05-30T16:52:28.230687+0000","last_unstale":"2024-05-30T16:52:28.234168+0000","last_undegraded":"2024-05-30T16:52:28.234168+0000","last_fullsized":"2024-05-30T16:52:28.234168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:30.079671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230689+0000","last_change":"2024-05-30T16:52:28.228914+0000","last_active":"2024-05-30T16:52:29.230689+0000","last_peered":"2024-05-30T16:52:29.230689+0000","last_clean":"2024-05-30T16:52:29.230689+0000","last_became_active":"2024-05-30T16:52:28.228598+0000","last_became_peered":"2024-05-30T16:52:28.228598+0000","last_unstale":"2024-05-30T16:52:29.230689+0000","last_undegraded":"2024-05-30T16:52:29.230689+0000","last_fullsized":"2024-05-30T16:52:29.230689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:07:22.042582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.245237+0000","last_change":"2024-05-30T16:52:28.243282+0000","last_active":"2024-05-30T16:52:28.245237+0000","last_peered":"2024-05-30T16:52:28.245237+0000","last_clean":"2024-05-30T16:52:28.245237+0000","last_became_active":"2024-05-30T16:52:28.243031+0000","last_became_peered":"2024-05-30T16:52:28.243031+0000","last_unstale":"2024-05-30T16:52:28.245237+0000","last_undegraded":"2024-05-30T16:52:28.245237+0000","last_fullsized":"2024-05-30T16:52:28.245237+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:15.325955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.666762+0000","last_change":"2024-05-30T16:52:28.217357+0000","last_active":"2024-05-30T16:52:29.666762+0000","last_peered":"2024-05-30T16:52:29.666762+0000","last_clean":"2024-05-30T16:52:29.666762+0000","last_became_active":"2024-05-30T16:52:28.217064+0000","last_became_peered":"2024-05-30T16:52:28.217064+0000","last_unstale":"2024-05-30T16:52:29.666762+0000","last_undegraded":"2024-05-30T16:52:29.666762+0000","last_fullsized":"2024-05-30T16:52:29.666762+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:36:55.582160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.229662+0000","last_change":"2024-05-30T16:52:28.222400+0000","last_active":"2024-05-30T16:52:29.229662+0000","last_peered":"2024-05-30T16:52:29.229662+0000","last_clean":"2024-05-30T16:52:29.229662+0000","last_became_active":"2024-05-30T16:52:28.222104+0000","last_became_peered":"2024-05-30T16:52:28.222104+0000","last_unstale":"2024-05-30T16:52:29.229662+0000","last_undegraded":"2024-05-30T16:52:29.229662+0000","last_fullsized":"2024-05-30T16:52:29.229662+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:44:46.486607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668033+0000","last_change":"2024-05-30T16:52:28.226294+0000","last_active":"2024-05-30T16:52:29.668033+0000","last_peered":"2024-05-30T16:52:29.668033+0000","last_clean":"2024-05-30T16:52:29.668033+0000","last_became_active":"2024-05-30T16:52:28.225954+0000","last_became_peered":"2024-05-30T16:52:28.225954+0000","last_unstale":"2024-05-30T16:52:29.668033+0000","last_undegraded":"2024-05-30T16:52:29.668033+0000","last_fullsized":"2024-05-30T16:52:29.668033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:18.248572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670544+0000","last_change":"2024-05-30T16:52:28.233808+0000","last_active":"2024-05-30T16:52:29.670544+0000","last_peered":"2024-05-30T16:52:29.670544+0000","last_clean":"2024-05-30T16:52:29.670544+0000","last_became_active":"2024-05-30T16:52:28.233511+0000","last_became_peered":"2024-05-30T16:52:28.233511+0000","last_unstale":"2024-05-30T16:52:29.670544+0000","last_undegraded":"2024-05-30T16:52:29.670544+0000","last_fullsized":"2024-05-30T16:52:29.670544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:46.000242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670508+0000","last_change":"2024-05-30T16:52:28.233868+0000","last_active":"2024-05-30T16:52:29.670508+0000","last_peered":"2024-05-30T16:52:29.670508+0000","last_clean":"2024-05-30T16:52:29.670508+0000","last_became_active":"2024-05-30T16:52:28.233286+0000","last_became_peered":"2024-05-30T16:52:28.233286+0000","last_unstale":"2024-05-30T16:52:29.670508+0000","last_undegraded":"2024-05-30T16:52:29.670508+0000","last_fullsized":"2024-05-30T16:52:29.670508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:08:28.772918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232236+0000","last_change":"2024-05-30T16:52:28.224121+0000","last_active":"2024-05-30T16:52:28.232236+0000","last_peered":"2024-05-30T16:52:28.232236+0000","last_clean":"2024-05-30T16:52:28.232236+0000","last_became_active":"2024-05-30T16:52:28.219408+0000","last_became_peered":"2024-05-30T16:52:28.219408+0000","last_unstale":"2024-05-30T16:52:28.232236+0000","last_undegraded":"2024-05-30T16:52:28.232236+0000","last_fullsized":"2024-05-30T16:52:28.232236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:18.702444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670853+0000","last_change":"2024-05-30T16:52:28.226515+0000","last_active":"2024-05-30T16:52:29.670853+0000","last_peered":"2024-05-30T16:52:29.670853+0000","last_clean":"2024-05-30T16:52:29.670853+0000","last_became_active":"2024-05-30T16:52:28.226070+0000","last_became_peered":"2024-05-30T16:52:28.226070+0000","last_unstale":"2024-05-30T16:52:29.670853+0000","last_undegraded":"2024-05-30T16:52:29.670853+0000","last_fullsized":"2024-05-30T16:52:29.670853+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:25:23.505977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667382+0000","last_change":"2024-05-30T16:52:28.229072+0000","last_active":"2024-05-30T16:52:29.667382+0000","last_peered":"2024-05-30T16:52:29.667382+0000","last_clean":"2024-05-30T16:52:29.667382+0000","last_became_active":"2024-05-30T16:52:28.228799+0000","last_became_peered":"2024-05-30T16:52:28.228799+0000","last_unstale":"2024-05-30T16:52:29.667382+0000","last_undegraded":"2024-05-30T16:52:29.667382+0000","last_fullsized":"2024-05-30T16:52:29.667382+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:40:43.349838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667032+0000","last_change":"2024-05-30T16:52:28.220747+0000","last_active":"2024-05-30T16:52:29.667032+0000","last_peered":"2024-05-30T16:52:29.667032+0000","last_clean":"2024-05-30T16:52:29.667032+0000","last_became_active":"2024-05-30T16:52:28.220458+0000","last_became_peered":"2024-05-30T16:52:28.220458+0000","last_unstale":"2024-05-30T16:52:29.667032+0000","last_undegraded":"2024-05-30T16:52:29.667032+0000","last_fullsized":"2024-05-30T16:52:29.667032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:20:20.196221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230186+0000","last_change":"2024-05-30T16:52:28.225802+0000","last_active":"2024-05-30T16:52:28.230186+0000","last_peered":"2024-05-30T16:52:28.230186+0000","last_clean":"2024-05-30T16:52:28.230186+0000","last_became_active":"2024-05-30T16:52:28.225366+0000","last_became_peered":"2024-05-30T16:52:28.225366+0000","last_unstale":"2024-05-30T16:52:28.230186+0000","last_undegraded":"2024-05-30T16:52:28.230186+0000","last_fullsized":"2024-05-30T16:52:28.230186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:11:24.392435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.234506+0000","last_change":"2024-05-30T16:52:28.223194+0000","last_active":"2024-05-30T16:52:29.234506+0000","last_peered":"2024-05-30T16:52:29.234506+0000","last_clean":"2024-05-30T16:52:29.234506+0000","last_became_active":"2024-05-30T16:52:28.221888+0000","last_became_peered":"2024-05-30T16:52:28.221888+0000","last_unstale":"2024-05-30T16:52:29.234506+0000","last_undegraded":"2024-05-30T16:52:29.234506+0000","last_fullsized":"2024-05-30T16:52:29.234506+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:26.623648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235333+0000","last_change":"2024-05-30T16:52:28.232643+0000","last_active":"2024-05-30T16:52:28.235333+0000","last_peered":"2024-05-30T16:52:28.235333+0000","last_clean":"2024-05-30T16:52:28.235333+0000","last_became_active":"2024-05-30T16:52:28.231215+0000","last_became_peered":"2024-05-30T16:52:28.231215+0000","last_unstale":"2024-05-30T16:52:28.235333+0000","last_undegraded":"2024-05-30T16:52:28.235333+0000","last_fullsized":"2024-05-30T16:52:28.235333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:15.325955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231247+0000","last_change":"2024-05-30T16:52:28.228070+0000","last_active":"2024-05-30T16:52:29.231247+0000","last_peered":"2024-05-30T16:52:29.231247+0000","last_clean":"2024-05-30T16:52:29.231247+0000","last_became_active":"2024-05-30T16:52:28.227771+0000","last_became_peered":"2024-05-30T16:52:28.227771+0000","last_unstale":"2024-05-30T16:52:29.231247+0000","last_undegraded":"2024-05-30T16:52:29.231247+0000","last_fullsized":"2024-05-30T16:52:29.231247+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:49.844872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233536+0000","last_change":"2024-05-30T16:52:28.227305+0000","last_active":"2024-05-30T16:52:29.233536+0000","last_peered":"2024-05-30T16:52:29.233536+0000","last_clean":"2024-05-30T16:52:29.233536+0000","last_became_active":"2024-05-30T16:52:28.226993+0000","last_became_peered":"2024-05-30T16:52:28.226993+0000","last_unstale":"2024-05-30T16:52:29.233536+0000","last_undegraded":"2024-05-30T16:52:29.233536+0000","last_fullsized":"2024-05-30T16:52:29.233536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:48:49.281817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235274+0000","last_change":"2024-05-30T16:52:28.232418+0000","last_active":"2024-05-30T16:52:28.235274+0000","last_peered":"2024-05-30T16:52:28.235274+0000","last_clean":"2024-05-30T16:52:28.235274+0000","last_became_active":"2024-05-30T16:52:28.231934+0000","last_became_peered":"2024-05-30T16:52:28.231934+0000","last_unstale":"2024-05-30T16:52:28.235274+0000","last_undegraded":"2024-05-30T16:52:28.235274+0000","last_fullsized":"2024-05-30T16:52:28.235274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:13.568129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.229893+0000","last_change":"2024-05-30T16:52:28.232229+0000","last_active":"2024-05-30T16:52:29.229893+0000","last_peered":"2024-05-30T16:52:29.229893+0000","last_clean":"2024-05-30T16:52:29.229893+0000","last_became_active":"2024-05-30T16:52:28.229698+0000","last_became_peered":"2024-05-30T16:52:28.229698+0000","last_unstale":"2024-05-30T16:52:29.229893+0000","last_undegraded":"2024-05-30T16:52:29.229893+0000","last_fullsized":"2024-05-30T16:52:29.229893+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:56:01.564145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230781+0000","last_change":"2024-05-30T16:52:28.227789+0000","last_active":"2024-05-30T16:52:28.230781+0000","last_peered":"2024-05-30T16:52:28.230781+0000","last_clean":"2024-05-30T16:52:28.230781+0000","last_became_active":"2024-05-30T16:52:28.227492+0000","last_became_peered":"2024-05-30T16:52:28.227492+0000","last_unstale":"2024-05-30T16:52:28.230781+0000","last_undegraded":"2024-05-30T16:52:28.230781+0000","last_fullsized":"2024-05-30T16:52:28.230781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:36.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668578+0000","last_change":"2024-05-30T16:52:28.232008+0000","last_active":"2024-05-30T16:52:29.668578+0000","last_peered":"2024-05-30T16:52:29.668578+0000","last_clean":"2024-05-30T16:52:29.668578+0000","last_became_active":"2024-05-30T16:52:28.230283+0000","last_became_peered":"2024-05-30T16:52:28.230283+0000","last_unstale":"2024-05-30T16:52:29.668578+0000","last_undegraded":"2024-05-30T16:52:29.668578+0000","last_fullsized":"2024-05-30T16:52:29.668578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:47:00.613414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231828+0000","last_change":"2024-05-30T16:52:28.228832+0000","last_active":"2024-05-30T16:52:29.231828+0000","last_peered":"2024-05-30T16:52:29.231828+0000","last_clean":"2024-05-30T16:52:29.231828+0000","last_became_active":"2024-05-30T16:52:28.228498+0000","last_became_peered":"2024-05-30T16:52:28.228498+0000","last_unstale":"2024-05-30T16:52:29.231828+0000","last_undegraded":"2024-05-30T16:52:29.231828+0000","last_fullsized":"2024-05-30T16:52:29.231828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:57:08.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.224326+0000","last_change":"2024-05-30T16:52:28.216836+0000","last_active":"2024-05-30T16:52:28.224326+0000","last_peered":"2024-05-30T16:52:28.224326+0000","last_clean":"2024-05-30T16:52:28.224326+0000","last_became_active":"2024-05-30T16:52:28.216551+0000","last_became_peered":"2024-05-30T16:52:28.216551+0000","last_unstale":"2024-05-30T16:52:28.224326+0000","last_undegraded":"2024-05-30T16:52:28.224326+0000","last_fullsized":"2024-05-30T16:52:28.224326+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:33.302790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235599+0000","last_change":"2024-05-30T16:52:28.230069+0000","last_active":"2024-05-30T16:52:28.235599+0000","last_peered":"2024-05-30T16:52:28.235599+0000","last_clean":"2024-05-30T16:52:28.235599+0000","last_became_active":"2024-05-30T16:52:28.229721+0000","last_became_peered":"2024-05-30T16:52:28.229721+0000","last_unstale":"2024-05-30T16:52:28.235599+0000","last_undegraded":"2024-05-30T16:52:28.235599+0000","last_fullsized":"2024-05-30T16:52:28.235599+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:36.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231342+0000","last_change":"2024-05-30T16:52:28.235586+0000","last_active":"2024-05-30T16:52:29.231342+0000","last_peered":"2024-05-30T16:52:29.231342+0000","last_clean":"2024-05-30T16:52:29.231342+0000","last_became_active":"2024-05-30T16:52:28.235317+0000","last_became_peered":"2024-05-30T16:52:28.235317+0000","last_unstale":"2024-05-30T16:52:29.231342+0000","last_undegraded":"2024-05-30T16:52:29.231342+0000","last_fullsized":"2024-05-30T16:52:29.231342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:51.107093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233228+0000","last_change":"2024-05-30T16:52:28.220267+0000","last_active":"2024-05-30T16:52:29.233228+0000","last_peered":"2024-05-30T16:52:29.233228+0000","last_clean":"2024-05-30T16:52:29.233228+0000","last_became_active":"2024-05-30T16:52:28.219886+0000","last_became_peered":"2024-05-30T16:52:28.219886+0000","last_unstale":"2024-05-30T16:52:29.233228+0000","last_undegraded":"2024-05-30T16:52:29.233228+0000","last_fullsized":"2024-05-30T16:52:29.233228+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:19:03.977651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.226746+0000","last_change":"2024-05-30T16:52:28.222902+0000","last_active":"2024-05-30T16:52:28.226746+0000","last_peered":"2024-05-30T16:52:28.226746+0000","last_clean":"2024-05-30T16:52:28.226746+0000","last_became_active":"2024-05-30T16:52:28.222572+0000","last_became_peered":"2024-05-30T16:52:28.222572+0000","last_unstale":"2024-05-30T16:52:28.226746+0000","last_undegraded":"2024-05-30T16:52:28.226746+0000","last_fullsized":"2024-05-30T16:52:28.226746+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:18.702444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230696+0000","last_change":"2024-05-30T16:52:28.223983+0000","last_active":"2024-05-30T16:52:29.230696+0000","last_peered":"2024-05-30T16:52:29.230696+0000","last_clean":"2024-05-30T16:52:29.230696+0000","last_became_active":"2024-05-30T16:52:28.223668+0000","last_became_peered":"2024-05-30T16:52:28.223668+0000","last_unstale":"2024-05-30T16:52:29.230696+0000","last_undegraded":"2024-05-30T16:52:29.230696+0000","last_fullsized":"2024-05-30T16:52:29.230696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:38:04.772837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.247401+0000","last_change":"2024-05-30T16:52:28.245829+0000","last_active":"2024-05-30T16:52:28.247401+0000","last_peered":"2024-05-30T16:52:28.247401+0000","last_clean":"2024-05-30T16:52:28.247401+0000","last_became_active":"2024-05-30T16:52:28.245500+0000","last_became_peered":"2024-05-30T16:52:28.245500+0000","last_unstale":"2024-05-30T16:52:28.247401+0000","last_undegraded":"2024-05-30T16:52:28.247401+0000","last_fullsized":"2024-05-30T16:52:28.247401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:28.718841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.245115+0000","last_change":"2024-05-30T16:52:28.240515+0000","last_active":"2024-05-30T16:52:28.245115+0000","last_peered":"2024-05-30T16:52:28.245115+0000","last_clean":"2024-05-30T16:52:28.245115+0000","last_became_active":"2024-05-30T16:52:28.240030+0000","last_became_peered":"2024-05-30T16:52:28.240030+0000","last_unstale":"2024-05-30T16:52:28.245115+0000","last_undegraded":"2024-05-30T16:52:28.245115+0000","last_fullsized":"2024-05-30T16:52:28.245115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:02:13.568129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.241653+0000","last_change":"2024-05-30T16:52:27.223522+0000","last_active":"2024-05-30T16:52:28.241653+0000","last_peered":"2024-05-30T16:52:28.241653+0000","last_clean":"2024-05-30T16:52:28.241653+0000","last_became_active":"2024-05-30T16:52:27.223234+0000","last_became_peered":"2024-05-30T16:52:27.223234+0000","last_unstale":"2024-05-30T16:52:28.241653+0000","last_undegraded":"2024-05-30T16:52:28.241653+0000","last_fullsized":"2024-05-30T16:52:28.241653+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:29.073369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670863+0000","last_change":"2024-05-30T16:52:28.238004+0000","last_active":"2024-05-30T16:52:29.670863+0000","last_peered":"2024-05-30T16:52:29.670863+0000","last_clean":"2024-05-30T16:52:29.670863+0000","last_became_active":"2024-05-30T16:52:28.237617+0000","last_became_peered":"2024-05-30T16:52:28.237617+0000","last_unstale":"2024-05-30T16:52:29.670863+0000","last_undegraded":"2024-05-30T16:52:29.670863+0000","last_fullsized":"2024-05-30T16:52:29.670863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:20.237839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233991+0000","last_change":"2024-05-30T16:52:27.232664+0000","last_active":"2024-05-30T16:52:29.233991+0000","last_peered":"2024-05-30T16:52:29.233991+0000","last_clean":"2024-05-30T16:52:29.233991+0000","last_became_active":"2024-05-30T16:52:27.232414+0000","last_became_peered":"2024-05-30T16:52:27.232414+0000","last_unstale":"2024-05-30T16:52:29.233991+0000","last_undegraded":"2024-05-30T16:52:29.233991+0000","last_fullsized":"2024-05-30T16:52:29.233991+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:30.720551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230177+0000","last_change":"2024-05-30T16:52:28.225173+0000","last_active":"2024-05-30T16:52:28.230177+0000","last_peered":"2024-05-30T16:52:28.230177+0000","last_clean":"2024-05-30T16:52:28.230177+0000","last_became_active":"2024-05-30T16:52:28.224753+0000","last_became_peered":"2024-05-30T16:52:28.224753+0000","last_unstale":"2024-05-30T16:52:28.230177+0000","last_undegraded":"2024-05-30T16:52:28.230177+0000","last_fullsized":"2024-05-30T16:52:28.230177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:48.871632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.247799+0000","last_change":"2024-05-30T16:52:27.224465+0000","last_active":"2024-05-30T16:52:28.247799+0000","last_peered":"2024-05-30T16:52:28.247799+0000","last_clean":"2024-05-30T16:52:28.247799+0000","last_became_active":"2024-05-30T16:52:27.224176+0000","last_became_peered":"2024-05-30T16:52:27.224176+0000","last_unstale":"2024-05-30T16:52:28.247799+0000","last_undegraded":"2024-05-30T16:52:28.247799+0000","last_fullsized":"2024-05-30T16:52:28.247799+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:13:38.253232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.239687+0000","last_change":"2024-05-30T16:52:28.236088+0000","last_active":"2024-05-30T16:52:28.239687+0000","last_peered":"2024-05-30T16:52:28.239687+0000","last_clean":"2024-05-30T16:52:28.239687+0000","last_became_active":"2024-05-30T16:52:28.235840+0000","last_became_peered":"2024-05-30T16:52:28.235840+0000","last_unstale":"2024-05-30T16:52:28.239687+0000","last_undegraded":"2024-05-30T16:52:28.239687+0000","last_fullsized":"2024-05-30T16:52:28.239687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:39:06.774775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235824+0000","last_change":"2024-05-30T16:52:27.232144+0000","last_active":"2024-05-30T16:52:28.235824+0000","last_peered":"2024-05-30T16:52:28.235824+0000","last_clean":"2024-05-30T16:52:28.235824+0000","last_became_active":"2024-05-30T16:52:27.231860+0000","last_became_peered":"2024-05-30T16:52:27.231860+0000","last_unstale":"2024-05-30T16:52:28.235824+0000","last_undegraded":"2024-05-30T16:52:28.235824+0000","last_fullsized":"2024-05-30T16:52:28.235824+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:05:10.878114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.231684+0000","last_change":"2024-05-30T16:52:28.229395+0000","last_active":"2024-05-30T16:52:29.231684+0000","last_peered":"2024-05-30T16:52:29.231684+0000","last_clean":"2024-05-30T16:52:29.231684+0000","last_became_active":"2024-05-30T16:52:28.228882+0000","last_became_peered":"2024-05-30T16:52:28.228882+0000","last_unstale":"2024-05-30T16:52:29.231684+0000","last_undegraded":"2024-05-30T16:52:29.231684+0000","last_fullsized":"2024-05-30T16:52:29.231684+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:05:33.766019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667298+0000","last_change":"2024-05-30T16:52:27.221692+0000","last_active":"2024-05-30T16:52:29.667298+0000","last_peered":"2024-05-30T16:52:29.667298+0000","last_clean":"2024-05-30T16:52:29.667298+0000","last_became_active":"2024-05-30T16:52:27.221222+0000","last_became_peered":"2024-05-30T16:52:27.221222+0000","last_unstale":"2024-05-30T16:52:29.667298+0000","last_undegraded":"2024-05-30T16:52:29.667298+0000","last_fullsized":"2024-05-30T16:52:29.667298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:15:03.761475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233069+0000","last_change":"2024-05-30T16:52:28.227194+0000","last_active":"2024-05-30T16:52:29.233069+0000","last_peered":"2024-05-30T16:52:29.233069+0000","last_clean":"2024-05-30T16:52:29.233069+0000","last_became_active":"2024-05-30T16:52:28.226829+0000","last_became_peered":"2024-05-30T16:52:28.226829+0000","last_unstale":"2024-05-30T16:52:29.233069+0000","last_undegraded":"2024-05-30T16:52:29.233069+0000","last_fullsized":"2024-05-30T16:52:29.233069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:20.196845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.234660+0000","last_change":"2024-05-30T16:52:27.222100+0000","last_active":"2024-05-30T16:52:28.234660+0000","last_peered":"2024-05-30T16:52:28.234660+0000","last_clean":"2024-05-30T16:52:28.234660+0000","last_became_active":"2024-05-30T16:52:27.221718+0000","last_became_peered":"2024-05-30T16:52:27.221718+0000","last_unstale":"2024-05-30T16:52:28.234660+0000","last_undegraded":"2024-05-30T16:52:28.234660+0000","last_fullsized":"2024-05-30T16:52:28.234660+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:32.296488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668635+0000","last_change":"2024-05-30T16:52:28.220825+0000","last_active":"2024-05-30T16:52:29.668635+0000","last_peered":"2024-05-30T16:52:29.668635+0000","last_clean":"2024-05-30T16:52:29.668635+0000","last_became_active":"2024-05-30T16:52:28.220547+0000","last_became_peered":"2024-05-30T16:52:28.220547+0000","last_unstale":"2024-05-30T16:52:29.668635+0000","last_undegraded":"2024-05-30T16:52:29.668635+0000","last_fullsized":"2024-05-30T16:52:29.668635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:03:38.973622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.209872+0000","last_change":"2024-05-30T16:52:27.219695+0000","last_active":"2024-05-30T16:52:28.209872+0000","last_peered":"2024-05-30T16:52:28.209872+0000","last_clean":"2024-05-30T16:52:28.209872+0000","last_became_active":"2024-05-30T16:52:27.219366+0000","last_became_peered":"2024-05-30T16:52:27.219366+0000","last_unstale":"2024-05-30T16:52:28.209872+0000","last_undegraded":"2024-05-30T16:52:28.209872+0000","last_fullsized":"2024-05-30T16:52:28.209872+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:47.865330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230102+0000","last_change":"2024-05-30T16:52:28.216745+0000","last_active":"2024-05-30T16:52:29.230102+0000","last_peered":"2024-05-30T16:52:29.230102+0000","last_clean":"2024-05-30T16:52:29.230102+0000","last_became_active":"2024-05-30T16:52:28.216245+0000","last_became_peered":"2024-05-30T16:52:28.216245+0000","last_unstale":"2024-05-30T16:52:29.230102+0000","last_undegraded":"2024-05-30T16:52:29.230102+0000","last_fullsized":"2024-05-30T16:52:29.230102+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:40:29.196239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.666727+0000","last_change":"2024-05-30T16:52:27.227619+0000","last_active":"2024-05-30T16:52:29.666727+0000","last_peered":"2024-05-30T16:52:29.666727+0000","last_clean":"2024-05-30T16:52:29.666727+0000","last_became_active":"2024-05-30T16:52:27.227326+0000","last_became_peered":"2024-05-30T16:52:27.227326+0000","last_unstale":"2024-05-30T16:52:29.666727+0000","last_undegraded":"2024-05-30T16:52:29.666727+0000","last_fullsized":"2024-05-30T16:52:29.666727+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:03:37.967320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668259+0000","last_change":"2024-05-30T16:52:28.224110+0000","last_active":"2024-05-30T16:52:29.668259+0000","last_peered":"2024-05-30T16:52:29.668259+0000","last_clean":"2024-05-30T16:52:29.668259+0000","last_became_active":"2024-05-30T16:52:28.223620+0000","last_became_peered":"2024-05-30T16:52:28.223620+0000","last_unstale":"2024-05-30T16:52:29.668259+0000","last_undegraded":"2024-05-30T16:52:29.668259+0000","last_fullsized":"2024-05-30T16:52:29.668259+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:00:48.375080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.217209+0000","last_change":"2024-05-30T16:52:27.229397+0000","last_active":"2024-05-30T16:52:28.217209+0000","last_peered":"2024-05-30T16:52:28.217209+0000","last_clean":"2024-05-30T16:52:28.217209+0000","last_became_active":"2024-05-30T16:52:27.229136+0000","last_became_peered":"2024-05-30T16:52:27.229136+0000","last_unstale":"2024-05-30T16:52:28.217209+0000","last_undegraded":"2024-05-30T16:52:28.217209+0000","last_fullsized":"2024-05-30T16:52:28.217209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:32.296488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232494+0000","last_change":"2024-05-30T16:52:28.230499+0000","last_active":"2024-05-30T16:52:28.232494+0000","last_peered":"2024-05-30T16:52:28.232494+0000","last_clean":"2024-05-30T16:52:28.232494+0000","last_became_active":"2024-05-30T16:52:28.230165+0000","last_became_peered":"2024-05-30T16:52:28.230165+0000","last_unstale":"2024-05-30T16:52:28.232494+0000","last_undegraded":"2024-05-30T16:52:28.232494+0000","last_fullsized":"2024-05-30T16:52:28.232494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:19:36.748624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.234149+0000","last_change":"2024-05-30T16:52:27.231492+0000","last_active":"2024-05-30T16:52:29.234149+0000","last_peered":"2024-05-30T16:52:29.234149+0000","last_clean":"2024-05-30T16:52:29.234149+0000","last_became_active":"2024-05-30T16:52:27.231145+0000","last_became_peered":"2024-05-30T16:52:27.231145+0000","last_unstale":"2024-05-30T16:52:29.234149+0000","last_undegraded":"2024-05-30T16:52:29.234149+0000","last_fullsized":"2024-05-30T16:52:29.234149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:38:03.513990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668714+0000","last_change":"2024-05-30T16:52:28.228720+0000","last_active":"2024-05-30T16:52:29.668714+0000","last_peered":"2024-05-30T16:52:29.668714+0000","last_clean":"2024-05-30T16:52:29.668714+0000","last_became_active":"2024-05-30T16:52:28.222768+0000","last_became_peered":"2024-05-30T16:52:28.222768+0000","last_unstale":"2024-05-30T16:52:29.668714+0000","last_undegraded":"2024-05-30T16:52:29.668714+0000","last_fullsized":"2024-05-30T16:52:29.668714+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:01.515201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.242729+0000","last_change":"2024-05-30T16:52:27.235155+0000","last_active":"2024-05-30T16:52:28.242729+0000","last_peered":"2024-05-30T16:52:28.242729+0000","last_clean":"2024-05-30T16:52:28.242729+0000","last_became_active":"2024-05-30T16:52:27.234883+0000","last_became_peered":"2024-05-30T16:52:27.234883+0000","last_unstale":"2024-05-30T16:52:28.242729+0000","last_undegraded":"2024-05-30T16:52:28.242729+0000","last_fullsized":"2024-05-30T16:52:28.242729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:11:23.386133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.239634+0000","last_change":"2024-05-30T16:52:28.238075+0000","last_active":"2024-05-30T16:52:28.239634+0000","last_peered":"2024-05-30T16:52:28.239634+0000","last_clean":"2024-05-30T16:52:28.239634+0000","last_became_active":"2024-05-30T16:52:28.237775+0000","last_became_peered":"2024-05-30T16:52:28.237775+0000","last_unstale":"2024-05-30T16:52:28.239634+0000","last_undegraded":"2024-05-30T16:52:28.239634+0000","last_fullsized":"2024-05-30T16:52:28.239634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:28.718841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667941+0000","last_change":"2024-05-30T16:52:27.218529+0000","last_active":"2024-05-30T16:52:29.667941+0000","last_peered":"2024-05-30T16:52:29.667941+0000","last_clean":"2024-05-30T16:52:29.667941+0000","last_became_active":"2024-05-30T16:52:27.218063+0000","last_became_peered":"2024-05-30T16:52:27.218063+0000","last_unstale":"2024-05-30T16:52:29.667941+0000","last_undegraded":"2024-05-30T16:52:29.667941+0000","last_fullsized":"2024-05-30T16:52:29.667941+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:46:59.607113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.229690+0000","last_change":"2024-05-30T16:52:28.223595+0000","last_active":"2024-05-30T16:52:28.229690+0000","last_peered":"2024-05-30T16:52:28.229690+0000","last_clean":"2024-05-30T16:52:28.229690+0000","last_became_active":"2024-05-30T16:52:28.223293+0000","last_became_peered":"2024-05-30T16:52:28.223293+0000","last_unstale":"2024-05-30T16:52:28.229690+0000","last_undegraded":"2024-05-30T16:52:28.229690+0000","last_fullsized":"2024-05-30T16:52:28.229690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:11:24.392435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233161+0000","last_change":"2024-05-30T16:52:27.229883+0000","last_active":"2024-05-30T16:52:29.233161+0000","last_peered":"2024-05-30T16:52:29.233161+0000","last_clean":"2024-05-30T16:52:29.233161+0000","last_became_active":"2024-05-30T16:52:27.228306+0000","last_became_peered":"2024-05-30T16:52:27.228306+0000","last_unstale":"2024-05-30T16:52:29.233161+0000","last_undegraded":"2024-05-30T16:52:29.233161+0000","last_fullsized":"2024-05-30T16:52:29.233161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:49.542333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232555+0000","last_change":"2024-05-30T16:52:28.228631+0000","last_active":"2024-05-30T16:52:28.232555+0000","last_peered":"2024-05-30T16:52:28.232555+0000","last_clean":"2024-05-30T16:52:28.232555+0000","last_became_active":"2024-05-30T16:52:28.225966+0000","last_became_peered":"2024-05-30T16:52:28.225966+0000","last_unstale":"2024-05-30T16:52:28.232555+0000","last_undegraded":"2024-05-30T16:52:28.232555+0000","last_fullsized":"2024-05-30T16:52:28.232555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:49:30.079671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669125+0000","last_change":"2024-05-30T16:52:27.233165+0000","last_active":"2024-05-30T16:52:29.669125+0000","last_peered":"2024-05-30T16:52:29.669125+0000","last_clean":"2024-05-30T16:52:29.669125+0000","last_became_active":"2024-05-30T16:52:27.232819+0000","last_became_peered":"2024-05-30T16:52:27.232819+0000","last_unstale":"2024-05-30T16:52:29.669125+0000","last_undegraded":"2024-05-30T16:52:29.669125+0000","last_fullsized":"2024-05-30T16:52:29.669125+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:19.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.666727+0000","last_change":"2024-05-30T16:52:28.223161+0000","last_active":"2024-05-30T16:52:29.666727+0000","last_peered":"2024-05-30T16:52:29.666727+0000","last_clean":"2024-05-30T16:52:29.666727+0000","last_became_active":"2024-05-30T16:52:28.222599+0000","last_became_peered":"2024-05-30T16:52:28.222599+0000","last_unstale":"2024-05-30T16:52:29.666727+0000","last_undegraded":"2024-05-30T16:52:29.666727+0000","last_fullsized":"2024-05-30T16:52:29.666727+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:55:05.755482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.208909+0000","last_change":"2024-05-30T16:52:27.229295+0000","last_active":"2024-05-30T16:52:28.208909+0000","last_peered":"2024-05-30T16:52:28.208909+0000","last_clean":"2024-05-30T16:52:28.208909+0000","last_became_active":"2024-05-30T16:52:27.228977+0000","last_became_peered":"2024-05-30T16:52:27.228977+0000","last_unstale":"2024-05-30T16:52:28.208909+0000","last_undegraded":"2024-05-30T16:52:28.208909+0000","last_fullsized":"2024-05-30T16:52:28.208909+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:21.449739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232534+0000","last_change":"2024-05-30T16:52:28.230635+0000","last_active":"2024-05-30T16:52:28.232534+0000","last_peered":"2024-05-30T16:52:28.232534+0000","last_clean":"2024-05-30T16:52:28.232534+0000","last_became_active":"2024-05-30T16:52:28.229433+0000","last_became_peered":"2024-05-30T16:52:28.229433+0000","last_unstale":"2024-05-30T16:52:28.232534+0000","last_undegraded":"2024-05-30T16:52:28.232534+0000","last_fullsized":"2024-05-30T16:52:28.232534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:11.006848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.240815+0000","last_change":"2024-05-30T16:52:27.230985+0000","last_active":"2024-05-30T16:52:28.240815+0000","last_peered":"2024-05-30T16:52:28.240815+0000","last_clean":"2024-05-30T16:52:28.240815+0000","last_became_active":"2024-05-30T16:52:27.230551+0000","last_became_peered":"2024-05-30T16:52:27.230551+0000","last_unstale":"2024-05-30T16:52:28.240815+0000","last_undegraded":"2024-05-30T16:52:28.240815+0000","last_fullsized":"2024-05-30T16:52:28.240815+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:03:47.865330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.232771+0000","last_change":"2024-05-30T16:52:28.228841+0000","last_active":"2024-05-30T16:52:28.232771+0000","last_peered":"2024-05-30T16:52:28.232771+0000","last_clean":"2024-05-30T16:52:28.232771+0000","last_became_active":"2024-05-30T16:52:28.228518+0000","last_became_peered":"2024-05-30T16:52:28.228518+0000","last_unstale":"2024-05-30T16:52:28.232771+0000","last_undegraded":"2024-05-30T16:52:28.232771+0000","last_fullsized":"2024-05-30T16:52:28.232771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:34:59.657491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.234529+0000","last_change":"2024-05-30T16:52:27.229987+0000","last_active":"2024-05-30T16:52:29.234529+0000","last_peered":"2024-05-30T16:52:29.234529+0000","last_clean":"2024-05-30T16:52:29.234529+0000","last_became_active":"2024-05-30T16:52:27.227660+0000","last_became_peered":"2024-05-30T16:52:27.227660+0000","last_unstale":"2024-05-30T16:52:29.234529+0000","last_undegraded":"2024-05-30T16:52:29.234529+0000","last_fullsized":"2024-05-30T16:52:29.234529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:48.577658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.229846+0000","last_change":"2024-05-30T16:52:28.225425+0000","last_active":"2024-05-30T16:52:28.229846+0000","last_peered":"2024-05-30T16:52:28.229846+0000","last_clean":"2024-05-30T16:52:28.229846+0000","last_became_active":"2024-05-30T16:52:28.225152+0000","last_became_peered":"2024-05-30T16:52:28.225152+0000","last_unstale":"2024-05-30T16:52:28.229846+0000","last_undegraded":"2024-05-30T16:52:28.229846+0000","last_fullsized":"2024-05-30T16:52:28.229846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:04.705529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.247944+0000","last_change":"2024-05-30T16:52:27.227849+0000","last_active":"2024-05-30T16:52:28.247944+0000","last_peered":"2024-05-30T16:52:28.247944+0000","last_clean":"2024-05-30T16:52:28.247944+0000","last_became_active":"2024-05-30T16:52:27.227465+0000","last_became_peered":"2024-05-30T16:52:27.227465+0000","last_unstale":"2024-05-30T16:52:28.247944+0000","last_undegraded":"2024-05-30T16:52:28.247944+0000","last_fullsized":"2024-05-30T16:52:28.247944+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:34:58.651189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.234488+0000","last_change":"2024-05-30T16:52:28.220537+0000","last_active":"2024-05-30T16:52:29.234488+0000","last_peered":"2024-05-30T16:52:29.234488+0000","last_clean":"2024-05-30T16:52:29.234488+0000","last_became_active":"2024-05-30T16:52:28.220232+0000","last_became_peered":"2024-05-30T16:52:28.220232+0000","last_unstale":"2024-05-30T16:52:29.234488+0000","last_undegraded":"2024-05-30T16:52:29.234488+0000","last_fullsized":"2024-05-30T16:52:29.234488+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:08.526023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.237004+0000","last_change":"2024-05-30T16:52:27.232326+0000","last_active":"2024-05-30T16:52:28.237004+0000","last_peered":"2024-05-30T16:52:28.237004+0000","last_clean":"2024-05-30T16:52:28.237004+0000","last_became_active":"2024-05-30T16:52:27.232034+0000","last_became_peered":"2024-05-30T16:52:27.232034+0000","last_unstale":"2024-05-30T16:52:28.237004+0000","last_undegraded":"2024-05-30T16:52:28.237004+0000","last_fullsized":"2024-05-30T16:52:28.237004+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:48:01.498972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235444+0000","last_change":"2024-05-30T16:52:28.232285+0000","last_active":"2024-05-30T16:52:28.235444+0000","last_peered":"2024-05-30T16:52:28.235444+0000","last_clean":"2024-05-30T16:52:28.235444+0000","last_became_active":"2024-05-30T16:52:28.231386+0000","last_became_peered":"2024-05-30T16:52:28.231386+0000","last_unstale":"2024-05-30T16:52:28.235444+0000","last_undegraded":"2024-05-30T16:52:28.235444+0000","last_fullsized":"2024-05-30T16:52:28.235444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:39:06.774775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.231462+0000","last_change":"2024-05-30T16:52:27.218134+0000","last_active":"2024-05-30T16:52:28.231462+0000","last_peered":"2024-05-30T16:52:28.231462+0000","last_clean":"2024-05-30T16:52:28.231462+0000","last_became_active":"2024-05-30T16:52:27.217838+0000","last_became_peered":"2024-05-30T16:52:27.217838+0000","last_unstale":"2024-05-30T16:52:28.231462+0000","last_undegraded":"2024-05-30T16:52:28.231462+0000","last_fullsized":"2024-05-30T16:52:28.231462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:05:10.000547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.231569+0000","last_change":"2024-05-30T16:52:28.227266+0000","last_active":"2024-05-30T16:52:28.231569+0000","last_peered":"2024-05-30T16:52:28.231569+0000","last_clean":"2024-05-30T16:52:28.231569+0000","last_became_active":"2024-05-30T16:52:28.223692+0000","last_became_peered":"2024-05-30T16:52:28.223692+0000","last_unstale":"2024-05-30T16:52:28.231569+0000","last_undegraded":"2024-05-30T16:52:28.231569+0000","last_fullsized":"2024-05-30T16:52:28.231569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:08:36.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.233330+0000","last_change":"2024-05-30T16:52:27.230550+0000","last_active":"2024-05-30T16:52:29.233330+0000","last_peered":"2024-05-30T16:52:29.233330+0000","last_clean":"2024-05-30T16:52:29.233330+0000","last_became_active":"2024-05-30T16:52:27.230236+0000","last_became_peered":"2024-05-30T16:52:27.230236+0000","last_unstale":"2024-05-30T16:52:29.233330+0000","last_undegraded":"2024-05-30T16:52:29.233330+0000","last_fullsized":"2024-05-30T16:52:29.233330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:11:56.573368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.223370+0000","last_change":"2024-05-30T16:52:28.222454+0000","last_active":"2024-05-30T16:52:28.223370+0000","last_peered":"2024-05-30T16:52:28.223370+0000","last_clean":"2024-05-30T16:52:28.223370+0000","last_became_active":"2024-05-30T16:52:28.221376+0000","last_became_peered":"2024-05-30T16:52:28.221376+0000","last_unstale":"2024-05-30T16:52:28.223370+0000","last_undegraded":"2024-05-30T16:52:28.223370+0000","last_fullsized":"2024-05-30T16:52:28.223370+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:15:26.192897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669368+0000","last_change":"2024-05-30T16:52:27.229665+0000","last_active":"2024-05-30T16:52:29.669368+0000","last_peered":"2024-05-30T16:52:29.669368+0000","last_clean":"2024-05-30T16:52:29.669368+0000","last_became_active":"2024-05-30T16:52:27.229334+0000","last_became_peered":"2024-05-30T16:52:27.229334+0000","last_unstale":"2024-05-30T16:52:29.669368+0000","last_undegraded":"2024-05-30T16:52:29.669368+0000","last_fullsized":"2024-05-30T16:52:29.669368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:32:54.351039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230312+0000","last_change":"2024-05-30T16:52:28.226084+0000","last_active":"2024-05-30T16:52:28.230312+0000","last_peered":"2024-05-30T16:52:28.230312+0000","last_clean":"2024-05-30T16:52:28.230312+0000","last_became_active":"2024-05-30T16:52:28.225511+0000","last_became_peered":"2024-05-30T16:52:28.225511+0000","last_unstale":"2024-05-30T16:52:28.230312+0000","last_undegraded":"2024-05-30T16:52:28.230312+0000","last_fullsized":"2024-05-30T16:52:28.230312+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:18.702444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.207835+0000","last_change":"2024-05-30T16:52:27.221456+0000","last_active":"2024-05-30T16:52:28.207835+0000","last_peered":"2024-05-30T16:52:28.207835+0000","last_clean":"2024-05-30T16:52:28.207835+0000","last_became_active":"2024-05-30T16:52:27.220980+0000","last_became_peered":"2024-05-30T16:52:27.220980+0000","last_unstale":"2024-05-30T16:52:28.207835+0000","last_undegraded":"2024-05-30T16:52:28.207835+0000","last_fullsized":"2024-05-30T16:52:28.207835+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:39:35.473873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230642+0000","last_change":"2024-05-30T16:52:28.228414+0000","last_active":"2024-05-30T16:52:28.230642+0000","last_peered":"2024-05-30T16:52:28.230642+0000","last_clean":"2024-05-30T16:52:28.230642+0000","last_became_active":"2024-05-30T16:52:28.228160+0000","last_became_peered":"2024-05-30T16:52:28.228160+0000","last_unstale":"2024-05-30T16:52:28.230642+0000","last_undegraded":"2024-05-30T16:52:28.230642+0000","last_fullsized":"2024-05-30T16:52:28.230642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:19:36.748624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.670928+0000","last_change":"2024-05-30T16:52:27.227842+0000","last_active":"2024-05-30T16:52:29.670928+0000","last_peered":"2024-05-30T16:52:29.670928+0000","last_clean":"2024-05-30T16:52:29.670928+0000","last_became_active":"2024-05-30T16:52:27.227553+0000","last_became_peered":"2024-05-30T16:52:27.227553+0000","last_unstale":"2024-05-30T16:52:29.670928+0000","last_undegraded":"2024-05-30T16:52:29.670928+0000","last_fullsized":"2024-05-30T16:52:29.670928+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:57:07.352356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230645+0000","last_change":"2024-05-30T16:52:28.227492+0000","last_active":"2024-05-30T16:52:28.230645+0000","last_peered":"2024-05-30T16:52:28.230645+0000","last_clean":"2024-05-30T16:52:28.230645+0000","last_became_active":"2024-05-30T16:52:28.226966+0000","last_became_peered":"2024-05-30T16:52:28.226966+0000","last_unstale":"2024-05-30T16:52:28.230645+0000","last_undegraded":"2024-05-30T16:52:28.230645+0000","last_fullsized":"2024-05-30T16:52:28.230645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:34:59.657491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.230760+0000","last_change":"2024-05-30T16:52:27.222452+0000","last_active":"2024-05-30T16:52:28.230760+0000","last_peered":"2024-05-30T16:52:28.230760+0000","last_clean":"2024-05-30T16:52:28.230760+0000","last_became_active":"2024-05-30T16:52:27.221459+0000","last_became_peered":"2024-05-30T16:52:27.221459+0000","last_unstale":"2024-05-30T16:52:28.230760+0000","last_undegraded":"2024-05-30T16:52:28.230760+0000","last_fullsized":"2024-05-30T16:52:28.230760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:19:35.742322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669161+0000","last_change":"2024-05-30T16:52:28.221634+0000","last_active":"2024-05-30T16:52:29.669161+0000","last_peered":"2024-05-30T16:52:29.669161+0000","last_clean":"2024-05-30T16:52:29.669161+0000","last_became_active":"2024-05-30T16:52:28.221348+0000","last_became_peered":"2024-05-30T16:52:28.221348+0000","last_unstale":"2024-05-30T16:52:29.669161+0000","last_undegraded":"2024-05-30T16:52:29.669161+0000","last_fullsized":"2024-05-30T16:52:29.669161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:09.342892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669827+0000","last_change":"2024-05-30T16:52:27.225833+0000","last_active":"2024-05-30T16:52:29.669827+0000","last_peered":"2024-05-30T16:52:29.669827+0000","last_clean":"2024-05-30T16:52:29.669827+0000","last_became_active":"2024-05-30T16:52:27.225533+0000","last_became_peered":"2024-05-30T16:52:27.225533+0000","last_unstale":"2024-05-30T16:52:29.669827+0000","last_undegraded":"2024-05-30T16:52:29.669827+0000","last_fullsized":"2024-05-30T16:52:29.669827+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:08:27.766617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.235579+0000","last_change":"2024-05-30T16:52:28.227720+0000","last_active":"2024-05-30T16:52:28.235579+0000","last_peered":"2024-05-30T16:52:28.235579+0000","last_clean":"2024-05-30T16:52:28.235579+0000","last_became_active":"2024-05-30T16:52:28.227323+0000","last_became_peered":"2024-05-30T16:52:28.227323+0000","last_unstale":"2024-05-30T16:52:28.235579+0000","last_undegraded":"2024-05-30T16:52:28.235579+0000","last_fullsized":"2024-05-30T16:52:28.235579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:04.705529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.209057+0000","last_change":"2024-05-30T16:52:27.220057+0000","last_active":"2024-05-30T16:52:28.209057+0000","last_peered":"2024-05-30T16:52:28.209057+0000","last_clean":"2024-05-30T16:52:28.209057+0000","last_became_active":"2024-05-30T16:52:27.219483+0000","last_became_peered":"2024-05-30T16:52:27.219483+0000","last_unstale":"2024-05-30T16:52:28.209057+0000","last_undegraded":"2024-05-30T16:52:28.209057+0000","last_fullsized":"2024-05-30T16:52:28.209057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:15:25.186595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.232263+0000","last_change":"2024-05-30T16:52:28.229162+0000","last_active":"2024-05-30T16:52:29.232263+0000","last_peered":"2024-05-30T16:52:29.232263+0000","last_clean":"2024-05-30T16:52:29.232263+0000","last_became_active":"2024-05-30T16:52:28.227949+0000","last_became_peered":"2024-05-30T16:52:28.227949+0000","last_unstale":"2024-05-30T16:52:29.232263+0000","last_undegraded":"2024-05-30T16:52:29.232263+0000","last_fullsized":"2024-05-30T16:52:29.232263+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:48:54.104397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668739+0000","last_change":"2024-05-30T16:52:27.224539+0000","last_active":"2024-05-30T16:52:29.668739+0000","last_peered":"2024-05-30T16:52:29.668739+0000","last_clean":"2024-05-30T16:52:29.668739+0000","last_became_active":"2024-05-30T16:52:27.224228+0000","last_became_peered":"2024-05-30T16:52:27.224228+0000","last_unstale":"2024-05-30T16:52:29.668739+0000","last_undegraded":"2024-05-30T16:52:29.668739+0000","last_fullsized":"2024-05-30T16:52:29.668739+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:50.100791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.233490+0000","last_change":"2024-05-30T16:52:28.231009+0000","last_active":"2024-05-30T16:52:28.233490+0000","last_peered":"2024-05-30T16:52:28.233490+0000","last_clean":"2024-05-30T16:52:28.233490+0000","last_became_active":"2024-05-30T16:52:28.229859+0000","last_became_peered":"2024-05-30T16:52:28.229859+0000","last_unstale":"2024-05-30T16:52:28.233490+0000","last_undegraded":"2024-05-30T16:52:28.233490+0000","last_fullsized":"2024-05-30T16:52:28.233490+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:13:39.259533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667484+0000","last_change":"2024-05-30T16:52:27.214583+0000","last_active":"2024-05-30T16:52:29.667484+0000","last_peered":"2024-05-30T16:52:29.667484+0000","last_clean":"2024-05-30T16:52:29.667484+0000","last_became_active":"2024-05-30T16:52:27.214182+0000","last_became_peered":"2024-05-30T16:52:27.214182+0000","last_unstale":"2024-05-30T16:52:29.667484+0000","last_undegraded":"2024-05-30T16:52:29.667484+0000","last_fullsized":"2024-05-30T16:52:29.667484+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:39:04.956426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.667689+0000","last_change":"2024-05-30T16:52:28.233315+0000","last_active":"2024-05-30T16:52:29.667689+0000","last_peered":"2024-05-30T16:52:29.667689+0000","last_clean":"2024-05-30T16:52:29.667689+0000","last_became_active":"2024-05-30T16:52:28.233041+0000","last_became_peered":"2024-05-30T16:52:28.233041+0000","last_unstale":"2024-05-30T16:52:29.667689+0000","last_undegraded":"2024-05-30T16:52:29.667689+0000","last_fullsized":"2024-05-30T16:52:29.667689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:51:30.479286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.668431+0000","last_change":"2024-05-30T16:52:27.227072+0000","last_active":"2024-05-30T16:52:29.668431+0000","last_peered":"2024-05-30T16:52:29.668431+0000","last_clean":"2024-05-30T16:52:29.668431+0000","last_became_active":"2024-05-30T16:52:27.226804+0000","last_became_peered":"2024-05-30T16:52:27.226804+0000","last_unstale":"2024-05-30T16:52:29.668431+0000","last_undegraded":"2024-05-30T16:52:29.668431+0000","last_fullsized":"2024-05-30T16:52:29.668431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:48.838570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230543+0000","last_change":"2024-05-30T16:52:28.216511+0000","last_active":"2024-05-30T16:52:29.230543+0000","last_peered":"2024-05-30T16:52:29.230543+0000","last_clean":"2024-05-30T16:52:29.230543+0000","last_became_active":"2024-05-30T16:52:28.215895+0000","last_became_peered":"2024-05-30T16:52:28.215895+0000","last_unstale":"2024-05-30T16:52:29.230543+0000","last_undegraded":"2024-05-30T16:52:29.230543+0000","last_fullsized":"2024-05-30T16:52:29.230543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:13:02.686629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.230079+0000","last_change":"2024-05-30T16:52:27.224814+0000","last_active":"2024-05-30T16:52:29.230079+0000","last_peered":"2024-05-30T16:52:29.230079+0000","last_clean":"2024-05-30T16:52:29.230079+0000","last_became_active":"2024-05-30T16:52:27.224354+0000","last_became_peered":"2024-05-30T16:52:27.224354+0000","last_unstale":"2024-05-30T16:52:29.230079+0000","last_undegraded":"2024-05-30T16:52:29.230079+0000","last_fullsized":"2024-05-30T16:52:29.230079+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:43:58.690696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.229631+0000","last_change":"2024-05-30T16:52:28.238951+0000","last_active":"2024-05-30T16:52:29.229631+0000","last_peered":"2024-05-30T16:52:29.229631+0000","last_clean":"2024-05-30T16:52:29.229631+0000","last_became_active":"2024-05-30T16:52:28.234929+0000","last_became_peered":"2024-05-30T16:52:28.234929+0000","last_unstale":"2024-05-30T16:52:29.229631+0000","last_undegraded":"2024-05-30T16:52:29.229631+0000","last_fullsized":"2024-05-30T16:52:29.229631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:27.178461+0000","last_clean_scrub_stamp":"2024-05-30T16:52:27.178461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:50:56.945930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-30T16:52:29.669009+0000","last_change":"2024-05-30T16:52:27.223659+0000","last_active":"2024-05-30T16:52:29.669009+0000","last_peered":"2024-05-30T16:52:29.669009+0000","last_clean":"2024-05-30T16:52:29.669009+0000","last_became_active":"2024-05-30T16:52:27.223321+0000","last_became_peered":"2024-05-30T16:52:27.223321+0000","last_unstale":"2024-05-30T16:52:29.669009+0000","last_undegraded":"2024-05-30T16:52:29.669009+0000","last_fullsized":"2024-05-30T16:52:29.669009+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:52:26.172159+0000","last_clean_scrub_stamp":"2024-05-30T16:52:26.172159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:51:44.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":247,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-05-30T16:52:28.214126+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:52:28.214126+0000","last_peered":"2024-05-30T16:52:28.214126+0000","last_clean":"2024-05-30T16:52:28.214126+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:52:28.214126+0000","last_undegraded":"2024-05-30T16:52:28.214126+0000","last_fullsized":"2024-05-30T16:52:28.214126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509549,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45788,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326052,"statfs":{"total":96636764160,"available":96589877248,"internally_reserved":0,"allocated":737280,"data_stored":310527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45788,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326052,"statfs":{"total":96636764160,"available":96589877248,"internally_reserved":0,"allocated":737280,"data_stored":305811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509550,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45796,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326044,"statfs":{"total":96636764160,"available":96589869056,"internally_reserved":0,"allocated":745472,"data_stored":310583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":310583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509549,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45796,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326044,"statfs":{"total":96636764160,"available":96589869056,"internally_reserved":0,"allocated":741376,"data_stored":310617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542252,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44960,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326880,"statfs":{"total":96636764160,"available":96590725120,"internally_reserved":0,"allocated":1327104,"data_stored":894727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542253,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40216,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331624,"statfs":{"total":96636764160,"available":96595582976,"internally_reserved":0,"allocated":733184,"data_stored":304851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542253,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46380,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325460,"statfs":{"total":96636764160,"available":96589271040,"internally_reserved":0,"allocated":1335296,"data_stored":896761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:52:33.201 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T16:52:33.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-30T16:52:33.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.462+0000 7fd4a5e00640 1 Processor -- start 2024-05-30T16:52:33.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.462+0000 7fd4a5e00640 1 -- start start 2024-05-30T16:52:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.464+0000 7fd4a5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4a00ce1f0 0x7fd4a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.464+0000 7fd4a5e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd4a00ce5d0 0x7fd4a015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.464+0000 7fd4a5e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a00b3580 con 0x7fd4a00b3d60 2024-05-30T16:52:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.464+0000 7fd4a5e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd4a015f490 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.464+0000 7fd4a5e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd4a015f770 con 0x7fd4a00ce5d0 2024-05-30T16:52:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.464+0000 7fd4a4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4a00ce1f0 0x7fd4a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.464+0000 7fd4a4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4a00ce1f0 0x7fd4a00b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41346/0 (socket says 172.21.15.67:41346) 2024-05-30T16:52:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.464+0000 7fd4a4a00640 1 -- 172.21.15.67:0/3308990986 learned_addr learned my addr 172.21.15.67:0/3308990986 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.465+0000 7fd49fe00640 1 --2- 172.21.15.67:0/3308990986 >> v2:172.21.15.169:3300/0 conn(0x7fd4a00ce5d0 0x7fd4a015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.465+0000 7fd4a4a00640 1 -- 172.21.15.67:0/3308990986 >> v2:172.21.15.169:3300/0 conn(0x7fd4a00ce5d0 msgr2=0x7fd4a015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.465+0000 7fd4a4a00640 1 --2- 172.21.15.67:0/3308990986 >> v2:172.21.15.169:3300/0 conn(0x7fd4a00ce5d0 0x7fd4a015ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.465+0000 7fd4a4a00640 1 -- 172.21.15.67:0/3308990986 >> v1:172.21.15.169:6789/0 conn(0x7fd4a00b3d60 legacy=0x7fd4a00ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:33.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.465+0000 7fd4a4a00640 1 -- 172.21.15.67:0/3308990986 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a015fa50 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.466+0000 7fd4a4a00640 1 --2- 172.21.15.67:0/3308990986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 0x7fd4a00b4dd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0070060 tx=0x7fd49002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb8ef5203ec84de server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:33.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.467+0000 7fd49f400640 1 -- 172.21.15.67:0/3308990986 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490036070 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.467+0000 7fd49f400640 1 -- 172.21.15.67:0/3308990986 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd490031600 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.467+0000 7fd49f400640 1 -- 172.21.15.67:0/3308990986 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4900319d0 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.467+0000 7fd4a5e00640 1 -- 172.21.15.67:0/3308990986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 msgr2=0x7fd4a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.467+0000 7fd4a5e00640 1 --2- 172.21.15.67:0/3308990986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 0x7fd4a00b4dd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0070060 tx=0x7fd49002a5d0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.468+0000 7fd4a5e00640 1 -- 172.21.15.67:0/3308990986 shutdown_connections 2024-05-30T16:52:33.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.468+0000 7fd4a5e00640 1 --2- 172.21.15.67:0/3308990986 >> v2:172.21.15.169:3300/0 conn(0x7fd4a00ce5d0 0x7fd4a015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.468+0000 7fd4a5e00640 1 --2- 172.21.15.67:0/3308990986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 0x7fd4a00b4dd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.468+0000 7fd4a5e00640 1 -- 172.21.15.67:0/3308990986 >> 172.21.15.67:0/3308990986 conn(0x7fd4a006b900 msgr2=0x7fd4a006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.468+0000 7fd4a5e00640 1 -- 172.21.15.67:0/3308990986 shutdown_connections 2024-05-30T16:52:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.468+0000 7fd4a5e00640 1 -- 172.21.15.67:0/3308990986 wait complete. 2024-05-30T16:52:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.470+0000 7fd4a5e00640 1 Processor -- start 2024-05-30T16:52:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.470+0000 7fd4a5e00640 1 -- start start 2024-05-30T16:52:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.471+0000 7fd4a5e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a00b3d60 0x7fd4a00afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.471+0000 7fd4a5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 0x7fd4a01411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.471+0000 7fd4a5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4a00ce5d0 0x7fd4a01416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.471+0000 7fd4a5e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a00cd2a0 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.471+0000 7fd4a5e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a015f180 con 0x7fd4a00b3d60 2024-05-30T16:52:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.471+0000 7fd4a5e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a00cd420 con 0x7fd4a00ce5d0 2024-05-30T16:52:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.471+0000 7fd49fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 0x7fd4a01411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.471+0000 7fd49fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 0x7fd4a01411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41354/0 (socket says 172.21.15.67:41354) 2024-05-30T16:52:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.471+0000 7fd49fe00640 1 -- 172.21.15.67:0/2947529340 learned_addr learned my addr 172.21.15.67:0/2947529340 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.472+0000 7fd4a4a00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a00b3d60 0x7fd4a00afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.472+0000 7fd4a5400640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4a00ce5d0 0x7fd4a01416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.472+0000 7fd49fe00640 1 -- 172.21.15.67:0/2947529340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4a00ce5d0 msgr2=0x7fd4a01416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.472+0000 7fd49fe00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4a00ce5d0 0x7fd4a01416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.472+0000 7fd49fe00640 1 -- 172.21.15.67:0/2947529340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a00b3d60 msgr2=0x7fd4a00afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.472+0000 7fd49fe00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a00b3d60 0x7fd4a00afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.472+0000 7fd49fe00640 1 -- 172.21.15.67:0/2947529340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a006e5c0 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.472+0000 7fd49fe00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 0x7fd4a01411a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00d800 tx=0x7fd48c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.473+0000 7fd49cc00640 1 -- 172.21.15.67:0/2947529340 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00b6c0 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.473+0000 7fd49cc00640 1 -- 172.21.15.67:0/2947529340 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c019070 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.473+0000 7fd49cc00640 1 -- 172.21.15.67:0/2947529340 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00ba80 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.473+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a00b00a0 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.473+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a006e840 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.474+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a00689e0 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.475+0000 7fd49cc00640 1 -- 172.21.15.67:0/2947529340 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7fd48c00bc20 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.476+0000 7fd49cc00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48006d0d0 0x7fd48006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.477+0000 7fd49cc00640 1 -- 172.21.15.67:0/2947529340 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fd48c08d2d0 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.477+0000 7fd4a4a00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48006d0d0 0x7fd48006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.478+0000 7fd4a4a00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48006d0d0 0x7fd48006f590 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd490033000 tx=0x7fd490009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:33.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.481+0000 7fd49cc00640 1 -- 172.21.15.67:0/2947529340 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4a006e5c0 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.755+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd4a006bd80 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.761+0000 7fd49cc00640 1 -- 172.21.15.67:0/2947529340 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fd48c05a750 con 0x7fd4a00ce1f0 2024-05-30T16:52:33.761 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-05-30T16:52:33.762 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T16:52:33.762 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-05-30T16:52:33.762 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-05-30T16:52:33.762 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-30T16:52:33.762 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-05-30T16:52:33.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48006d0d0 msgr2=0x7fd48006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48006d0d0 0x7fd48006f590 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd490033000 tx=0x7fd490009000 comp rx=0 tx=0).stop 2024-05-30T16:52:33.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 msgr2=0x7fd4a01411a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 0x7fd4a01411a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00d800 tx=0x7fd48c00dcc0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 shutdown_connections 2024-05-30T16:52:33.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48006d0d0 0x7fd48006f590 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4a00ce5d0 0x7fd4a01416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4a00ce1f0 0x7fd4a01411a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 --2- 172.21.15.67:0/2947529340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a00b3d60 0x7fd4a00afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.763+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 >> 172.21.15.67:0/2947529340 conn(0x7fd4a006b900 msgr2=0x7fd4a00732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:33.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.764+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 shutdown_connections 2024-05-30T16:52:33.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.764+0000 7fd4a5e00640 1 -- 172.21.15.67:0/2947529340 wait complete. 2024-05-30T16:52:33.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:52:33.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.982+0000 7f20f1c00640 1 Processor -- start 2024-05-30T16:52:33.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.982+0000 7f20f1c00640 1 -- start start 2024-05-30T16:52:33.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.984+0000 7f20f1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec073310 0x7f20ec0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.984+0000 7f20f1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f20ec06dc20 0x7f20ec15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.984+0000 7f20f1c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20ec0afd60 con 0x7f20ec15b3a0 2024-05-30T16:52:33.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.984+0000 7f20f1c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f20ec06f980 con 0x7f20ec06dc20 2024-05-30T16:52:33.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.984+0000 7f20f1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec0ae0c0 con 0x7f20ec073310 2024-05-30T16:52:33.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.984+0000 7f20ebe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f20ec15b3a0 0x7f20ec072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40558/0 (socket says 172.21.15.67:40558) 2024-05-30T16:52:33.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.984+0000 7f20ebe00640 1 -- 172.21.15.67:0/2320214899 learned_addr learned my addr 172.21.15.67:0/2320214899 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:33.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.984+0000 7f20f1200640 1 --2- 172.21.15.67:0/2320214899 >> v2:172.21.15.67:3300/0 conn(0x7f20ec06dc20 0x7f20ec15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.985+0000 7f20eb400640 1 -- 172.21.15.67:0/2320214899 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3484601717 0 0) 0x7f20ec0afd60 con 0x7f20ec15b3a0 2024-05-30T16:52:33.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.985+0000 7f20e3e00640 1 --2- 172.21.15.67:0/2320214899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec073310 0x7f20ec0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.985+0000 7f20eb400640 1 -- 172.21.15.67:0/2320214899 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20d0003520 con 0x7f20ec15b3a0 2024-05-30T16:52:33.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.985+0000 7f20f1200640 1 -- 172.21.15.67:0/2320214899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec073310 msgr2=0x7f20ec0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:33.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.985+0000 7f20f1200640 1 --2- 172.21.15.67:0/2320214899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec073310 0x7f20ec0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.985+0000 7f20f1200640 1 -- 172.21.15.67:0/2320214899 >> v1:172.21.15.67:6789/0 conn(0x7f20ec15b3a0 legacy=0x7f20ec072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.985+0000 7f20f1200640 1 -- 172.21.15.67:0/2320214899 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec0af7a0 con 0x7f20ec06dc20 2024-05-30T16:52:33.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.985+0000 7f20e3e00640 1 --2- 172.21.15.67:0/2320214899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec073310 0x7f20ec0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:33.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.986+0000 7f20f1200640 1 --2- 172.21.15.67:0/2320214899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec06dc20 0x7f20ec15f9a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009d90 tx=0x7f20e402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20589e62b2e4876 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:33.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.986+0000 7f20eb400640 1 -- 172.21.15.67:0/2320214899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4007740 con 0x7f20ec06dc20 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.986+0000 7f20eb400640 1 -- 172.21.15.67:0/2320214899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e40078e0 con 0x7f20ec06dc20 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.986+0000 7f20eb400640 1 -- 172.21.15.67:0/2320214899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4007cb0 con 0x7f20ec06dc20 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.987+0000 7f20f1c00640 1 -- 172.21.15.67:0/2320214899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec06dc20 msgr2=0x7f20ec15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.987+0000 7f20f1c00640 1 --2- 172.21.15.67:0/2320214899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec06dc20 0x7f20ec15f9a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009d90 tx=0x7f20e402a660 comp rx=0 tx=0).stop 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.987+0000 7f20f1c00640 1 -- 172.21.15.67:0/2320214899 shutdown_connections 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.987+0000 7f20f1c00640 1 --2- 172.21.15.67:0/2320214899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec06dc20 0x7f20ec15f9a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.987+0000 7f20f1c00640 1 --2- 172.21.15.67:0/2320214899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec073310 0x7f20ec0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.987+0000 7f20f1c00640 1 -- 172.21.15.67:0/2320214899 >> 172.21.15.67:0/2320214899 conn(0x7f20ec0b1fc0 msgr2=0x7f20ec0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.987+0000 7f20f1c00640 1 -- 172.21.15.67:0/2320214899 shutdown_connections 2024-05-30T16:52:33.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.987+0000 7f20f1c00640 1 -- 172.21.15.67:0/2320214899 wait complete. 2024-05-30T16:52:33.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.989+0000 7f20f1c00640 1 Processor -- start 2024-05-30T16:52:33.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.989+0000 7f20f1c00640 1 -- start start 2024-05-30T16:52:33.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20f1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ec06dc20 0x7f20ec0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20f1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec073310 0x7f20ec14e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20f1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec15b3a0 0x7f20ec0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20f1c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec15b780 con 0x7f20ec073310 2024-05-30T16:52:33.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20f1c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec15fee0 con 0x7f20ec06dc20 2024-05-30T16:52:33.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20f1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec0ae550 con 0x7f20ec15b3a0 2024-05-30T16:52:33.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec073310 0x7f20ec14e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec073310 0x7f20ec14e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41368/0 (socket says 172.21.15.67:41368) 2024-05-30T16:52:33.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20e3e00640 1 -- 172.21.15.67:0/2214859290 learned_addr learned my addr 172.21.15.67:0/2214859290 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:33.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20e3e00640 1 -- 172.21.15.67:0/2214859290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec15b3a0 msgr2=0x7f20ec0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20e3e00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec15b3a0 0x7f20ec0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20e3e00640 1 -- 172.21.15.67:0/2214859290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ec06dc20 msgr2=0x7f20ec0b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:33.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20e3e00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ec06dc20 0x7f20ec0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:33.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.990+0000 7f20e3e00640 1 -- 172.21.15.67:0/2214859290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec14ebf0 con 0x7f20ec073310 2024-05-30T16:52:33.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.991+0000 7f20e3e00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec073310 0x7f20ec14e6b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c7f0 tx=0x7f20d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:33.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.991+0000 7f20e8c00640 1 -- 172.21.15.67:0/2214859290 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4015070 con 0x7f20ec073310 2024-05-30T16:52:33.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.991+0000 7f20e8c00640 1 -- 172.21.15.67:0/2214859290 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d4007a30 con 0x7f20ec073310 2024-05-30T16:52:33.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.991+0000 7f20e8c00640 1 -- 172.21.15.67:0/2214859290 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4007e00 con 0x7f20ec073310 2024-05-30T16:52:33.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.991+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec0b01b0 con 0x7f20ec073310 2024-05-30T16:52:33.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.991+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec0b0770 con 0x7f20ec073310 2024-05-30T16:52:33.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.992+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ec0689e0 con 0x7f20ec073310 2024-05-30T16:52:33.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.993+0000 7f20e8c00640 1 -- 172.21.15.67:0/2214859290 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f20d4018070 con 0x7f20ec073310 2024-05-30T16:52:33.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.994+0000 7f20e8c00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20c806d0d0 0x7f20c806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:33.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.994+0000 7f20e8c00640 1 -- 172.21.15.67:0/2214859290 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f20d408cf10 con 0x7f20ec073310 2024-05-30T16:52:33.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.994+0000 7f20ebe00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20c806d0d0 0x7f20c806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:33.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.995+0000 7f20ebe00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20c806d0d0 0x7f20c806f590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000f60 tx=0x7f20dc0078f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:33.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:33.999+0000 7f20e8c00640 1 -- 172.21.15.67:0/2214859290 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20d405a1f0 con 0x7f20ec073310 2024-05-30T16:52:34.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.243+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20ec06e6e0 con 0x7f20ec073310 2024-05-30T16:52:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.245+0000 7f20e8c00640 1 -- 172.21.15.67:0/2214859290 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5181 (secure 0 0 0) 0x7f20d4059af0 con 0x7f20ec073310 2024-05-30T16:52:34.245 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:52:34.246 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2932035652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2932035652},{"type":"v1","addr":"172.21.15.67:6837","nonce":2932035652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/327493818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":327493818},{"type":"v1","addr":"172.21.15.67:6836","nonce":327493818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1437754767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1437754767},{"type":"v1","addr":"172.21.15.169:6834","nonce":1437754767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:52:27.681449+0000","modified":"2024-05-30T16:52:30.749556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/898694760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":898694760},{"type":"v1","addr":"172.21.15.169:6835","nonce":898694760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5937,"qdb_cluster":[5937]},"id":4}]} 2024-05-30T16:52:34.246 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 35 2024-05-30T16:52:34.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.247+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20c806d0d0 msgr2=0x7f20c806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.247+0000 7f20f1c00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20c806d0d0 0x7f20c806f590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000f60 tx=0x7f20dc0078f0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.247+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec073310 msgr2=0x7f20ec14e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.247+0000 7f20f1c00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec073310 0x7f20ec14e6b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c7f0 tx=0x7f20d400cbf0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.248+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 shutdown_connections 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.248+0000 7f20f1c00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20c806d0d0 0x7f20c806f590 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.248+0000 7f20f1c00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec15b3a0 0x7f20ec0b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.248+0000 7f20f1c00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20ec073310 0x7f20ec14e6b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.248+0000 7f20f1c00640 1 --2- 172.21.15.67:0/2214859290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ec06dc20 0x7f20ec0b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.248+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 >> 172.21.15.67:0/2214859290 conn(0x7f20ec0b1fc0 msgr2=0x7f20ec0d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.248+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 shutdown_connections 2024-05-30T16:52:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.248+0000 7f20f1c00640 1 -- 172.21.15.67:0/2214859290 wait complete. 2024-05-30T16:52:34.670 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-30T16:52:27.681449+0000', 'modified': '2024-05-30T16:52:30.749556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5937}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5937': {'gid': 5937, 'name': 'd', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/898694760', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6833', 'nonce': 898694760}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 898694760}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5937, 'qdb_cluster': [5937]} 2024-05-30T16:52:34.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-30T16:52:34.671 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-30T16:52:34.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.900+0000 7f9ab2000640 1 Processor -- start 2024-05-30T16:52:34.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.900+0000 7f9ab2000640 1 -- start start 2024-05-30T16:52:34.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.902+0000 7f9ab2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9aac158b80 0x7f9aac06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.902+0000 7f9ab2000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aac06e6e0 con 0x7f9aac06e160 2024-05-30T16:52:34.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.902+0000 7f9ab2000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aac06e9c0 con 0x7f9aac0722d0 2024-05-30T16:52:34.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.902+0000 7f9ab2000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9aac06eca0 con 0x7f9aac158b80 2024-05-30T16:52:34.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.902+0000 7f9ab0c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9aac158b80 0x7f9aac06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.902+0000 7f9ab0c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9aac158b80 0x7f9aac06dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41374/0 (socket says 172.21.15.67:41374) 2024-05-30T16:52:34.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.902+0000 7f9aa3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9aac06e160 0x7f9aac071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40566/0 (socket says 172.21.15.67:40566) 2024-05-30T16:52:34.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.902+0000 7f9ab0c00640 1 -- 172.21.15.67:0/2020994741 learned_addr learned my addr 172.21.15.67:0/2020994741 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:34.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.903+0000 7f9ab0c00640 1 -- 172.21.15.67:0/2020994741 >> v1:172.21.15.169:6789/0 conn(0x7f9aac0722d0 legacy=0x7f9aac0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.903+0000 7f9ab0c00640 1 -- 172.21.15.67:0/2020994741 >> v1:172.21.15.67:6789/0 conn(0x7f9aac06e160 legacy=0x7f9aac071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.903+0000 7f9ab0c00640 1 -- 172.21.15.67:0/2020994741 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac06ef80 con 0x7f9aac158b80 2024-05-30T16:52:34.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.903+0000 7f9aabe00640 1 -- 172.21.15.67:0/2020994741 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 544034556 0 0) 0x7f9aac06e6e0 con 0x7f9aac06e160 2024-05-30T16:52:34.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.905+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/2020994741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 0x7f9aac06dc20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009870 tx=0x7f9a9c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27675cbbc8556c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:34.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.905+0000 7f9aabe00640 1 -- 172.21.15.67:0/2020994741 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c037070 con 0x7f9aac158b80 2024-05-30T16:52:34.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.905+0000 7f9aabe00640 1 -- 172.21.15.67:0/2020994741 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a9c02ae50 con 0x7f9aac158b80 2024-05-30T16:52:34.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.905+0000 7f9aabe00640 1 -- 172.21.15.67:0/2020994741 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c032860 con 0x7f9aac158b80 2024-05-30T16:52:34.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.905+0000 7f9ab2000640 1 -- 172.21.15.67:0/2020994741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 msgr2=0x7f9aac06dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.905+0000 7f9ab2000640 1 --2- 172.21.15.67:0/2020994741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 0x7f9aac06dc20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009870 tx=0x7f9a9c02a300 comp rx=0 tx=0).stop 2024-05-30T16:52:34.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.906+0000 7f9ab2000640 1 -- 172.21.15.67:0/2020994741 shutdown_connections 2024-05-30T16:52:34.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.906+0000 7f9ab2000640 1 --2- 172.21.15.67:0/2020994741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 0x7f9aac06dc20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.906+0000 7f9ab2000640 1 -- 172.21.15.67:0/2020994741 >> 172.21.15.67:0/2020994741 conn(0x7f9aac0b1fc0 msgr2=0x7f9aac0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:34.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.906+0000 7f9ab2000640 1 -- 172.21.15.67:0/2020994741 shutdown_connections 2024-05-30T16:52:34.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.906+0000 7f9ab2000640 1 -- 172.21.15.67:0/2020994741 wait complete. 2024-05-30T16:52:34.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.907+0000 7f9ab2000640 1 Processor -- start 2024-05-30T16:52:34.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.908+0000 7f9ab2000640 1 -- start start 2024-05-30T16:52:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac06e160 0x7f9aac09d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aac0722d0 0x7f9aac1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 0x7f9aac1485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab2000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac06f780 con 0x7f9aac158b80 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac0b2d30 con 0x7f9aac06e160 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac1527e0 con 0x7f9aac0722d0 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 0x7f9aac1485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 0x7f9aac1485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41376/0 (socket says 172.21.15.67:41376) 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab1600640 1 -- 172.21.15.67:0/4067636394 learned_addr learned my addr 172.21.15.67:0/4067636394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac06e160 0x7f9aac09d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9aa3e00640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aac0722d0 0x7f9aac1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab1600640 1 -- 172.21.15.67:0/4067636394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aac0722d0 msgr2=0x7f9aac1522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab1600640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aac0722d0 0x7f9aac1522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab1600640 1 -- 172.21.15.67:0/4067636394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac06e160 msgr2=0x7f9aac09d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab1600640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac06e160 0x7f9aac09d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.909+0000 7f9ab1600640 1 -- 172.21.15.67:0/4067636394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac09dbd0 con 0x7f9aac158b80 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.910+0000 7f9ab1600640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 0x7f9aac1485e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c760 tx=0x7f9aa400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.910+0000 7f9aa9600640 1 -- 172.21.15.67:0/4067636394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4003b70 con 0x7f9aac158b80 2024-05-30T16:52:34.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.910+0000 7f9aa9600640 1 -- 172.21.15.67:0/4067636394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa4003d30 con 0x7f9aac158b80 2024-05-30T16:52:34.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.910+0000 7f9ab2000640 1 -- 172.21.15.67:0/4067636394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac09deb0 con 0x7f9aac158b80 2024-05-30T16:52:34.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.910+0000 7f9aa9600640 1 -- 172.21.15.67:0/4067636394 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400aa90 con 0x7f9aac158b80 2024-05-30T16:52:34.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.910+0000 7f9ab2000640 1 -- 172.21.15.67:0/4067636394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac0ad3b0 con 0x7f9aac158b80 2024-05-30T16:52:34.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.912+0000 7f9aa9600640 1 -- 172.21.15.67:0/4067636394 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f9aa400ac30 con 0x7f9aac158b80 2024-05-30T16:52:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.914+0000 7f9aa9600640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a8806d0d0 0x7f9a8806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.914+0000 7f9aa9600640 1 -- 172.21.15.67:0/4067636394 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f9aa408cd30 con 0x7f9aac158b80 2024-05-30T16:52:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.914+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a8806d0d0 0x7f9a8806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.914+0000 7f9ab6568740 1 Processor -- start 2024-05-30T16:52:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.914+0000 7f9ab6568740 1 -- start start 2024-05-30T16:52:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a8806d0d0 0x7f9a8806f590 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c034000 tx=0x7f9a9c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec292490 0x55ffec295c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec296150 0x55ffec2b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab6568740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ffec1cfb10 con 0x55ffec2b25d0 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab6568740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ffec1762b0 con 0x55ffec292490 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab6568740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ffec1cbad0 con 0x55ffec296150 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41380/0 (socket says 172.21.15.67:41380) 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.915+0000 7f9ab1600640 1 -- 172.21.15.67:0/4290817226 learned_addr learned my addr 172.21.15.67:0/4290817226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/4290817226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec292490 0x55ffec295c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9ab1600640 1 -- 172.21.15.67:0/4290817226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec296150 msgr2=0x55ffec2b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9ab1600640 1 --2- 172.21.15.67:0/4290817226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec296150 0x55ffec2b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9ab1600640 1 -- 172.21.15.67:0/4290817226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec292490 msgr2=0x55ffec295c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9ab1600640 1 --2- 172.21.15.67:0/4290817226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec292490 0x55ffec295c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9ab1600640 1 -- 172.21.15.67:0/4290817226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffebee22d0 con 0x55ffec2b25d0 2024-05-30T16:52:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9ab1600640 1 --2- 172.21.15.67:0/4290817226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2b2a30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c730 tx=0x7f9aa4059440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927ba85b9ee7debf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:34.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9aa1200640 1 -- 172.21.15.67:0/4290817226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa405ddd0 con 0x55ffec2b25d0 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9aa1200640 1 -- 172.21.15.67:0/4290817226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa4062600 con 0x55ffec2b25d0 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.916+0000 7f9aa1200640 1 -- 172.21.15.67:0/4290817226 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400ce10 con 0x55ffec2b25d0 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.917+0000 7f9ab6568740 1 -- 172.21.15.67:0/4290817226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 msgr2=0x55ffec2b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.917+0000 7f9ab6568740 1 --2- 172.21.15.67:0/4290817226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2b2a30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c730 tx=0x7f9aa4059440 comp rx=0 tx=0).stop 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.917+0000 7f9ab6568740 1 -- 172.21.15.67:0/4290817226 shutdown_connections 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.917+0000 7f9ab6568740 1 --2- 172.21.15.67:0/4290817226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2b2a30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.917+0000 7f9ab6568740 1 --2- 172.21.15.67:0/4290817226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec296150 0x55ffec2b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.917+0000 7f9ab6568740 1 --2- 172.21.15.67:0/4290817226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec292490 0x55ffec295c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.917+0000 7f9ab6568740 1 -- 172.21.15.67:0/4290817226 >> 172.21.15.67:0/4290817226 conn(0x55ffec268470 msgr2=0x55ffec269470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.917+0000 7f9ab6568740 1 -- 172.21.15.67:0/4290817226 shutdown_connections 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.917+0000 7f9ab6568740 1 -- 172.21.15.67:0/4290817226 wait complete. 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.918+0000 7f9ab6568740 1 -- start start 2024-05-30T16:52:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.918+0000 7f9ab6568740 1 Processor -- start 2024-05-30T16:52:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec28e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2b25d0 0x55ffec296db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec2972f0 0x55ffec2a8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab6568740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ffec2662a0 con 0x55ffec296150 2024-05-30T16:52:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab6568740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ffec189570 con 0x55ffec2972f0 2024-05-30T16:52:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab6568740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ffec266580 con 0x55ffec2b25d0 2024-05-30T16:52:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab6568740 10 client.0 _ll_register_callbacks cb 0x55ffec157fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec28e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec28e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41388/0 (socket says 172.21.15.67:41388) 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.919+0000 7f9ab0c00640 1 -- 172.21.15.67:0/3144914304 learned_addr learned my addr 172.21.15.67:0/3144914304 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9ab0c00640 1 -- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2b25d0 msgr2=0x55ffec296db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9aa3e00640 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2b25d0 0x55ffec296db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2b25d0 0x55ffec296db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9ab1600640 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec2972f0 0x55ffec2a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9ab0c00640 1 -- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec2972f0 msgr2=0x55ffec2a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec2972f0 0x55ffec2a8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9ab0c00640 1 -- 172.21.15.67:0/3144914304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffebfac6b0 con 0x55ffec296150 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec28e410 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c000c00 tx=0x7f9a9c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c4b0bc1f4cf5d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9a99600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.920+0000 7f9a99600640 1 -- 172.21.15.67:0/3144914304 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c037040 con 0x55ffec296150 2024-05-30T16:52:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9a99600640 1 -- 172.21.15.67:0/3144914304 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f9a9c03cab0 con 0x55ffec296150 2024-05-30T16:52:34.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9ab6568740 1 -- 172.21.15.67:0/3144914304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa4062600 con 0x55ffec296150 2024-05-30T16:52:34.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9a99600640 1 -- 172.21.15.67:0/3144914304 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9a9c055050 con 0x55ffec296150 2024-05-30T16:52:34.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9ab6568740 10 client.6060 fetch_fsmap learned FSMap version 35 2024-05-30T16:52:34.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9ab6568740 1 -- 172.21.15.67:0/3144914304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ffec299780 con 0x55ffec296150 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9a99600640 1 -- 172.21.15.67:0/3144914304 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa4062600 con 0x55ffec296150 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9a99600640 1 -- 172.21.15.67:0/3144914304 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f9a9c058020 con 0x55ffec296150 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9ab6568740 10 client.6060 fetch_fsmap finished waiting for FSMap version 35 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9ab6568740 10 client.6060 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9ab6568740 20 client.6060 populate_metadata read hostname 'smithi067' 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] conn(0x55ffec2cca90 0x55ffec2cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9ab6568740 4 client.6060 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.921+0000 7f9ab6568740 1 -- 172.21.15.67:0/3144914304 --> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffec1dd530 con 0x55ffec2cca90 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.922+0000 7f9aa3e00640 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] conn(0x55ffec2cca90 0x55ffec2cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.923+0000 7f9aa3e00640 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] conn(0x55ffec2cca90 0x55ffec2cee80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ca2d8b77c0ea65 server_cookie=d391eeb3d558858a in_seq=0 out_seq=0 2024-05-30T16:52:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.923+0000 7f9a99600640 10 client.6060 ms_handle_connect on v2:172.21.15.169:6833/898694760 2024-05-30T16:52:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9a99600640 1 -- 172.21.15.67:0/3144914304 <== mds.0 v2:172.21.15.169:6833/898694760 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffec1dd530 con 0x55ffec2cca90 2024-05-30T16:52:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9a99600640 10 client.6060 handle_command_reply: tid=0 2024-05-30T16:52:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9ab6568740 1 client.6060 shutdown 2024-05-30T16:52:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9ab6568740 1 -- 172.21.15.67:0/3144914304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 msgr2=0x55ffec28e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec28e410 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c000c00 tx=0x7f9a9c02c070 comp rx=0 tx=0).stop 2024-05-30T16:52:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9ab6568740 1 -- 172.21.15.67:0/3144914304 shutdown_connections 2024-05-30T16:52:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] conn(0x55ffec2cca90 0x55ffec2cee80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec2972f0 0x55ffec2a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2b25d0 0x55ffec296db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3144914304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec28e410 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9ab6568740 1 -- 172.21.15.67:0/3144914304 >> 172.21.15.67:0/3144914304 conn(0x55ffec292490 msgr2=0x55ffec2651b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.926+0000 7f9a99600640 0 client.6060 ms_handle_reset on v2:172.21.15.169:6833/898694760 2024-05-30T16:52:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.927+0000 7f9ab6568740 1 -- 172.21.15.67:0/3144914304 shutdown_connections 2024-05-30T16:52:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.927+0000 7f9ab6568740 1 -- 172.21.15.67:0/3144914304 wait complete. 2024-05-30T16:52:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.927+0000 7f9ab6568740 20 client.6060 trim_cache size 0 max 16384 2024-05-30T16:52:34.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.946+0000 7f9ab6568740 1 Processor -- start 2024-05-30T16:52:34.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.946+0000 7f9ab6568740 1 -- start start 2024-05-30T16:52:34.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.947+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec296150 0x55ffec291740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.947+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2972f0 0x55ffec296be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.947+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.947+0000 7f9ab6568740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac1ffbc0 con 0x55ffec2b25d0 2024-05-30T16:52:34.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.947+0000 7f9ab6568740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac1fef50 con 0x55ffec296150 2024-05-30T16:52:34.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.947+0000 7f9ab6568740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac1ff570 con 0x55ffec2972f0 2024-05-30T16:52:34.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41400/0 (socket says 172.21.15.67:41400) 2024-05-30T16:52:34.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab1600640 1 -- 172.21.15.67:0/3664495939 learned_addr learned my addr 172.21.15.67:0/3664495939 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:34.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec296150 0x55ffec291740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9aa3e00640 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2972f0 0x55ffec296be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab1600640 1 -- 172.21.15.67:0/3664495939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2972f0 msgr2=0x55ffec296be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab1600640 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2972f0 0x55ffec296be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab1600640 1 -- 172.21.15.67:0/3664495939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec296150 msgr2=0x55ffec291740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab1600640 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec296150 0x55ffec291740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab1600640 1 -- 172.21.15.67:0/3664495939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffec297120 con 0x55ffec2b25d0 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.948+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec296150 0x55ffec291740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.949+0000 7f9ab1600640 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2651b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c730 tx=0x7f9aa405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3449553723ce7fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.949+0000 7f9a9be00640 1 -- 172.21.15.67:0/3664495939 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa400ce10 con 0x55ffec2b25d0 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.949+0000 7f9a9be00640 1 -- 172.21.15.67:0/3664495939 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa4088070 con 0x55ffec2b25d0 2024-05-30T16:52:34.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.949+0000 7f9a9be00640 1 -- 172.21.15.67:0/3664495939 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa405ddd0 con 0x55ffec2b25d0 2024-05-30T16:52:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.949+0000 7f9ab6568740 1 -- 172.21.15.67:0/3664495939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 msgr2=0x55ffec2651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.949+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2651b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c730 tx=0x7f9aa405b000 comp rx=0 tx=0).stop 2024-05-30T16:52:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.950+0000 7f9ab6568740 1 -- 172.21.15.67:0/3664495939 shutdown_connections 2024-05-30T16:52:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.950+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec2b25d0 0x55ffec2651b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.950+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2972f0 0x55ffec296be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.950+0000 7f9ab6568740 1 --2- 172.21.15.67:0/3664495939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec296150 0x55ffec291740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.950+0000 7f9ab6568740 1 -- 172.21.15.67:0/3664495939 >> 172.21.15.67:0/3664495939 conn(0x55ffec292490 msgr2=0x55ffec2618d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.950+0000 7f9ab6568740 1 -- 172.21.15.67:0/3664495939 shutdown_connections 2024-05-30T16:52:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.950+0000 7f9ab6568740 1 -- 172.21.15.67:0/3664495939 wait complete. 2024-05-30T16:52:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.950+0000 7f9ab6568740 1 -- start start 2024-05-30T16:52:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.951+0000 7f9ab6568740 1 Processor -- start 2024-05-30T16:52:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.951+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec2aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.951+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec2972f0 0x55ffec266fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.951+0000 7f9ab6568740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2b25d0 0x55ffec2674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.951+0000 7f9ab6568740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ffec2a9160 con 0x55ffec296150 2024-05-30T16:52:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.951+0000 7f9ab6568740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ffec2987d0 con 0x55ffec2972f0 2024-05-30T16:52:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.951+0000 7f9ab6568740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ffec26bc20 con 0x55ffec2b25d0 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab6568740 10 client.0 _ll_register_callbacks cb 0x7f9a9c032a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec2aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec2aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41406/0 (socket says 172.21.15.67:41406) 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab0c00640 1 -- 172.21.15.67:0/722426386 learned_addr learned my addr 172.21.15.67:0/722426386 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab0c00640 1 -- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2b25d0 msgr2=0x55ffec2674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2b25d0 0x55ffec2674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab0c00640 1 -- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec2972f0 msgr2=0x55ffec266fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec2972f0 0x55ffec266fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab0c00640 1 -- 172.21.15.67:0/722426386 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffec2ca910 con 0x55ffec296150 2024-05-30T16:52:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9ab0c00640 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec2aa8e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c048850 tx=0x7f9a9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d383173f3b62fc server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.952+0000 7f9a99600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:52:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9a99600640 1 -- 172.21.15.67:0/722426386 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c037040 con 0x55ffec296150 2024-05-30T16:52:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9a99600640 1 -- 172.21.15.67:0/722426386 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f9a9c059600 con 0x55ffec296150 2024-05-30T16:52:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9ab6568740 1 -- 172.21.15.67:0/722426386 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa4088070 con 0x55ffec296150 2024-05-30T16:52:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9a99600640 1 -- 172.21.15.67:0/722426386 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9a9c059800 con 0x55ffec296150 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9ab6568740 10 client.6066 fetch_fsmap learned FSMap version 35 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9ab6568740 1 -- 172.21.15.67:0/722426386 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ffec299080 con 0x55ffec296150 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9a99600640 1 -- 172.21.15.67:0/722426386 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa4088070 con 0x55ffec296150 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9a99600640 1 -- 172.21.15.67:0/722426386 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f9a9c06a020 con 0x55ffec296150 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9ab6568740 10 client.6066 fetch_fsmap finished waiting for FSMap version 35 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9ab6568740 10 client.6066 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9ab6568740 20 client.6066 populate_metadata read hostname 'smithi067' 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9ab6568740 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] conn(0x55ffec2ad1e0 0x55ffec2ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9ab6568740 4 client.6066 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.953+0000 7f9ab6568740 1 -- 172.21.15.67:0/722426386 --> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] -- command(tid 0: {"prefix": "status"}) -- 0x55ffec1d73a0 con 0x55ffec2ad1e0 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.954+0000 7f9aa3e00640 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] conn(0x55ffec2ad1e0 0x55ffec2ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.955+0000 7f9aa3e00640 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] conn(0x55ffec2ad1e0 0x55ffec2ad5c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56337ca5c3ac5901 server_cookie=87e22cef8461e2a8 in_seq=0 out_seq=0 2024-05-30T16:52:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.955+0000 7f9a99600640 10 client.6066 ms_handle_connect on v2:172.21.15.169:6833/898694760 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.955+0000 7f9a99600640 1 -- 172.21.15.67:0/722426386 <== mds.0 v2:172.21.15.169:6833/898694760 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ffec1d73a0 con 0x55ffec2ad1e0 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.955+0000 7f9a99600640 10 client.6066 handle_command_reply: tid=0 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.955+0000 7f9ab6568740 1 client.6066 shutdown 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 -- 172.21.15.67:0/722426386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 msgr2=0x55ffec2aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec2aa8e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c048850 tx=0x7f9a9c004730 comp rx=0 tx=0).stop 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 -- 172.21.15.67:0/722426386 shutdown_connections 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:6833/898694760,v1:172.21.15.169:6835/898694760] conn(0x55ffec2ad1e0 0x55ffec2ad5c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ffec2b25d0 0x55ffec2674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ffec2972f0 0x55ffec266fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 --2- 172.21.15.67:0/722426386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ffec296150 0x55ffec2aa8e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9a99600640 0 client.6066 ms_handle_reset on v2:172.21.15.169:6833/898694760 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 -- 172.21.15.67:0/722426386 >> 172.21.15.67:0/722426386 conn(0x55ffec292490 msgr2=0x55ffec28e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 -- 172.21.15.67:0/722426386 shutdown_connections 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 1 -- 172.21.15.67:0/722426386 wait complete. 2024-05-30T16:52:34.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.956+0000 7f9ab6568740 20 client.6066 trim_cache size 0 max 16384 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5937, 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 6.7554068789999997, 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 35, 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 62, 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 62, 2024-05-30T16:52:34.959 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 11.170036937000001 2024-05-30T16:52:34.960 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:52:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 -- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a8806d0d0 msgr2=0x7f9a8806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a8806d0d0 0x7f9a8806f590 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c034000 tx=0x7f9a9c02e000 comp rx=0 tx=0).stop 2024-05-30T16:52:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 -- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 msgr2=0x7f9aac1485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 0x7f9aac1485e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c760 tx=0x7f9aa400cc20 comp rx=0 tx=0).stop 2024-05-30T16:52:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 -- 172.21.15.67:0/4067636394 shutdown_connections 2024-05-30T16:52:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a8806d0d0 0x7f9a8806f590 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac158b80 0x7f9aac1485e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aac0722d0 0x7f9aac1522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 --2- 172.21.15.67:0/4067636394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac06e160 0x7f9aac09d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 -- 172.21.15.67:0/4067636394 >> 172.21.15.67:0/4067636394 conn(0x7f9aac0b1fc0 msgr2=0x7f9aac1602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 -- 172.21.15.67:0/4067636394 shutdown_connections 2024-05-30T16:52:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:34.957+0000 7f9ab2000640 1 -- 172.21.15.67:0/4067636394 wait complete. 2024-05-30T16:52:34.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T16:52:35.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.173+0000 7f8e3a800640 1 Processor -- start 2024-05-30T16:52:35.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.173+0000 7f8e3a800640 1 -- start start 2024-05-30T16:52:35.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.174+0000 7f8e3a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3415b3a0 0x7f8e340b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:35.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.174+0000 7f8e3a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e34073310 0x7f8e3415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:35.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.174+0000 7f8e3a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e340afd60 con 0x7f8e3406dc20 2024-05-30T16:52:35.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.174+0000 7f8e3a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f8e3406f980 con 0x7f8e34073310 2024-05-30T16:52:35.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.174+0000 7f8e3a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8e340ae0c0 con 0x7f8e3415b3a0 2024-05-30T16:52:35.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.175+0000 7f8e39e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e34073310 0x7f8e3415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:35.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.175+0000 7f8e39e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e34073310 0x7f8e3415f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41422/0 (socket says 172.21.15.67:41422) 2024-05-30T16:52:35.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.175+0000 7f8e39e00640 1 -- 172.21.15.67:0/3517695320 learned_addr learned my addr 172.21.15.67:0/3517695320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:35.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.175+0000 7f8e39400640 1 --2- 172.21.15.67:0/3517695320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3415b3a0 0x7f8e340b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:35.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.175+0000 7f8e39e00640 1 -- 172.21.15.67:0/3517695320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3415b3a0 msgr2=0x7f8e340b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:35.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.175+0000 7f8e39e00640 1 --2- 172.21.15.67:0/3517695320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3415b3a0 0x7f8e340b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.175+0000 7f8e39e00640 1 -- 172.21.15.67:0/3517695320 >> v1:172.21.15.169:6789/0 conn(0x7f8e3406dc20 legacy=0x7f8e34072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:35.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.175+0000 7f8e39e00640 1 -- 172.21.15.67:0/3517695320 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e340af7a0 con 0x7f8e34073310 2024-05-30T16:52:35.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.176+0000 7f8e39e00640 1 --2- 172.21.15.67:0/3517695320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e34073310 0x7f8e3415f9a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d150 tx=0x7f8e2c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fe481ad06a242f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e33e00640 1 -- 172.21.15.67:0/3517695320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00b2e0 con 0x7f8e34073310 2024-05-30T16:52:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e33e00640 1 -- 172.21.15.67:0/3517695320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e2c009600 con 0x7f8e34073310 2024-05-30T16:52:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e33e00640 1 -- 172.21.15.67:0/3517695320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00b480 con 0x7f8e34073310 2024-05-30T16:52:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e3a800640 1 -- 172.21.15.67:0/3517695320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e34073310 msgr2=0x7f8e3415f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e3a800640 1 --2- 172.21.15.67:0/3517695320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e34073310 0x7f8e3415f9a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d150 tx=0x7f8e2c02c8e0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e3a800640 1 -- 172.21.15.67:0/3517695320 shutdown_connections 2024-05-30T16:52:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e3a800640 1 --2- 172.21.15.67:0/3517695320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e34073310 0x7f8e3415f9a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e3a800640 1 --2- 172.21.15.67:0/3517695320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3415b3a0 0x7f8e340b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e3a800640 1 -- 172.21.15.67:0/3517695320 >> 172.21.15.67:0/3517695320 conn(0x7f8e340b1fc0 msgr2=0x7f8e340b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:35.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.177+0000 7f8e3a800640 1 -- 172.21.15.67:0/3517695320 shutdown_connections 2024-05-30T16:52:35.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.178+0000 7f8e3a800640 1 -- 172.21.15.67:0/3517695320 wait complete. 2024-05-30T16:52:35.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.179+0000 7f8e3a800640 1 Processor -- start 2024-05-30T16:52:35.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.179+0000 7f8e3a800640 1 -- start start 2024-05-30T16:52:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.180+0000 7f8e3a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3406dc20 0x7f8e3415a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.180+0000 7f8e3a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e34073310 0x7f8e34157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.180+0000 7f8e3a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3415b3a0 0x7f8e340ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.180+0000 7f8e3a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3415b780 con 0x7f8e3415b3a0 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.180+0000 7f8e3a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3415fee0 con 0x7f8e34073310 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.180+0000 7f8e3a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8e340ae550 con 0x7f8e3406dc20 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3415b3a0 0x7f8e340ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3415b3a0 0x7f8e340ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41438/0 (socket says 172.21.15.67:41438) 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39e00640 1 -- 172.21.15.67:0/303546539 learned_addr learned my addr 172.21.15.67:0/303546539 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39400640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3406dc20 0x7f8e3415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e38a00640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e34073310 0x7f8e34157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39e00640 1 -- 172.21.15.67:0/303546539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3406dc20 msgr2=0x7f8e3415a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39e00640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3406dc20 0x7f8e3415a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39e00640 1 -- 172.21.15.67:0/303546539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e34073310 msgr2=0x7f8e34157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39e00640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e34073310 0x7f8e34157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39e00640 1 -- 172.21.15.67:0/303546539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e341596b0 con 0x7f8e3415b3a0 2024-05-30T16:52:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.181+0000 7f8e39e00640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3415b3a0 0x7f8e340ce770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c02ced0 tx=0x7f8e2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.182+0000 7f8e31600640 1 -- 172.21.15.67:0/303546539 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c040030 con 0x7f8e3415b3a0 2024-05-30T16:52:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.182+0000 7f8e31600640 1 -- 172.21.15.67:0/303546539 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e2c03c640 con 0x7f8e3415b3a0 2024-05-30T16:52:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.182+0000 7f8e31600640 1 -- 172.21.15.67:0/303546539 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c03ca30 con 0x7f8e3415b3a0 2024-05-30T16:52:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.182+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e34159990 con 0x7f8e3415b3a0 2024-05-30T16:52:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.182+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e34159f50 con 0x7f8e3415b3a0 2024-05-30T16:52:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.183+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e340689e0 con 0x7f8e3415b3a0 2024-05-30T16:52:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.184+0000 7f8e31600640 1 -- 172.21.15.67:0/303546539 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f8e2c043030 con 0x7f8e3415b3a0 2024-05-30T16:52:35.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.185+0000 7f8e31600640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e1406d0d0 0x7f8e1406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:35.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.185+0000 7f8e31600640 1 -- 172.21.15.67:0/303546539 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f8e2c030020 con 0x7f8e3415b3a0 2024-05-30T16:52:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.185+0000 7f8e39400640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e1406d0d0 0x7f8e1406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.186+0000 7f8e39400640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e1406d0d0 0x7f8e1406f590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8e3415be10 tx=0x7f8e28008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:35.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.189+0000 7f8e31600640 1 -- 172.21.15.67:0/303546539 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e2c04a020 con 0x7f8e3415b3a0 2024-05-30T16:52:35.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.437+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8e34072c00 con 0x7f8e3415b3a0 2024-05-30T16:52:35.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.438+0000 7f8e31600640 1 -- 172.21.15.67:0/303546539 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f8e2c081030 con 0x7f8e3415b3a0 2024-05-30T16:52:35.439 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T16:52:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e1406d0d0 msgr2=0x7f8e1406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e1406d0d0 0x7f8e1406f590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8e3415be10 tx=0x7f8e28008040 comp rx=0 tx=0).stop 2024-05-30T16:52:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3415b3a0 msgr2=0x7f8e340ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3415b3a0 0x7f8e340ce770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c02ced0 tx=0x7f8e2c004a40 comp rx=0 tx=0).stop 2024-05-30T16:52:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 shutdown_connections 2024-05-30T16:52:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e1406d0d0 0x7f8e1406f590 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3415b3a0 0x7f8e340ce770 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e34073310 0x7f8e34157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 --2- 172.21.15.67:0/303546539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e3406dc20 0x7f8e3415a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:35.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 >> 172.21.15.67:0/303546539 conn(0x7f8e340b1fc0 msgr2=0x7f8e340ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:35.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 shutdown_connections 2024-05-30T16:52:35.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35.441+0000 7f8e3a800640 1 -- 172.21.15.67:0/303546539 wait complete. 2024-05-30T16:52:35.461 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T16:52:35.461 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T16:52:35.461 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T16:52:35.461 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-30T16:52:35.461 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T16:52:35.461 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T16:52:35.461 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T16:52:35.461 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T16:52:35.461 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:52:35.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:35.462 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T16:52:35.511 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T16:52:35.512 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42017sec preferred_lft 42017sec 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::74f5:cfff:fec0:693d/64 scope link 2024-05-30T16:52:35.513 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:52:35.514 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:52:35.514 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:52:35.514 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::a8fe:aeff:fef7:f6a0/64 scope link 2024-05-30T16:52:35.514 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:52:35.514 INFO:teuthology.orchestra.run.smithi067.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:52:35.514 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:52:35.514 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::185e:a4ff:fec6:7d8c/64 scope link 2024-05-30T16:52:35.514 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:52:35.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:35.515 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T16:52:35.598 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T16:52:35.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:35.603 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T16:52:35.658 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T16:52:35.659 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T16:52:35.659 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T16:52:35.659 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T16:52:35.659 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T16:52:35.660 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:52:35.660 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:52:35.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T16:52:35.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T16:52:35.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T16:52:35.660 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:52:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:52:35.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:52:35.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:35.878 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T16:52:35.928 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T16:52:35.928 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-30T16:52:35.929 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:52:35.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:35.929 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-05-30T16:52:36.018 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-30T16:52:36.018 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:52:36.018 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-30T16:52:36.073 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:52:36.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:52:36.074 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:52:36.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:52:36.219 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:52:36.277 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:52:36.278 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:52:36.279 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:52:36.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:36.280 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:52:36.332 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T16:52:36.333 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-30T16:52:36.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:52:36.408 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:52:36.450 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:52:36.451 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:52:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:52:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:52:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:52:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:52:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:52:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:52:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:52:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:52:36.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:36.453 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:52:36.517 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-05-30T16:52:36.516+0000 7ff7aeb4e480 -1 init, newargv = 0x55750ca0c690 newargc=15 2024-05-30T16:52:36.517 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[56210]: starting ceph client 2024-05-30T16:52:36.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[56210]: starting fuse 2024-05-30T16:52:36.538 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T16:52:36.538 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-30T16:52:36.538 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T16:52:36.538 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T16:52:36.538 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T16:52:36.538 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T16:52:36.539 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T16:52:36.540 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T16:52:36.540 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T16:52:36.540 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T16:52:36.540 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T16:52:36.540 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:52:36.540 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-05-30T16:52:36.540 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T16:52:36.732 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.56210.asok 2024-05-30T16:52:36.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:52:36.750 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56210.asok 2024-05-30T16:52:36.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:36.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56210.asok status 2024-05-30T16:52:36.988 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:52:36.988 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T16:52:36.988 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T16:52:36.988 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T16:52:36.988 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-05-30T16:52:36.988 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T16:52:36.988 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-30T16:52:36.988 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "56210", 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6081, 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6081 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 3422599539 2024-05-30T16:52:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 3422599539 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.6081 192.168.0.1:0/3422599539", 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/3422599539", 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 35, 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 62, 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T16:52:36.990 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T16:52:36.991 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:52:37.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:37.004 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T16:52:37.004 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:52:37.058 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T16:52:37.058 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-30T16:52:37.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:37.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:52:37.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-30T16:52:37.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.369+0000 7efe49000640 1 Processor -- start 2024-05-30T16:52:37.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.369+0000 7efe49000640 1 -- start start 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.370+0000 7efe49000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efe440d0430 0x7efe440d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.370+0000 7efe49000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe4415bb30 con 0x7efe440ce560 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.370+0000 7efe49000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe440b3ba0 con 0x7efe4415f0c0 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.370+0000 7efe49000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7efe440b5290 con 0x7efe440d0430 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.371+0000 7efe42a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efe440d0430 0x7efe440d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.371+0000 7efe43400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efe440ce560 0x7efe440b4b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40568/0 (socket says 172.21.15.67:40568) 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.371+0000 7efe43400640 1 -- 172.21.15.67:0/2204271847 learned_addr learned my addr 172.21.15.67:0/2204271847 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.371+0000 7efe42a00640 1 -- 172.21.15.67:0/2204271847 >> v1:172.21.15.169:6789/0 conn(0x7efe4415f0c0 legacy=0x7efe4415f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.371+0000 7efe42a00640 1 -- 172.21.15.67:0/2204271847 >> v1:172.21.15.67:6789/0 conn(0x7efe440ce560 legacy=0x7efe440b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:37.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.371+0000 7efe42a00640 1 -- 172.21.15.67:0/2204271847 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4415bdd0 con 0x7efe440d0430 2024-05-30T16:52:37.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.372+0000 7efe42a00640 1 --2- 172.21.15.67:0/2204271847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe440d0430 0x7efe440d0860 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097a0 tx=0x7efe2c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7785786bd967d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:37.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.373+0000 7efe42000640 1 -- 172.21.15.67:0/2204271847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c037070 con 0x7efe440d0430 2024-05-30T16:52:37.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.373+0000 7efe42000640 1 -- 172.21.15.67:0/2204271847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2c032440 con 0x7efe440d0430 2024-05-30T16:52:37.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.373+0000 7efe42000640 1 -- 172.21.15.67:0/2204271847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c032810 con 0x7efe440d0430 2024-05-30T16:52:37.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.373+0000 7efe49000640 1 -- 172.21.15.67:0/2204271847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe440d0430 msgr2=0x7efe440d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:37.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.373+0000 7efe49000640 1 --2- 172.21.15.67:0/2204271847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe440d0430 0x7efe440d0860 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097a0 tx=0x7efe2c02a1e0 comp rx=0 tx=0).stop 2024-05-30T16:52:37.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.374+0000 7efe49000640 1 -- 172.21.15.67:0/2204271847 shutdown_connections 2024-05-30T16:52:37.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.374+0000 7efe49000640 1 --2- 172.21.15.67:0/2204271847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe440d0430 0x7efe440d0860 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:37.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.374+0000 7efe49000640 1 -- 172.21.15.67:0/2204271847 >> 172.21.15.67:0/2204271847 conn(0x7efe4406bae0 msgr2=0x7efe4406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:37.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.374+0000 7efe49000640 1 -- 172.21.15.67:0/2204271847 shutdown_connections 2024-05-30T16:52:37.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.374+0000 7efe49000640 1 -- 172.21.15.67:0/2204271847 wait complete. 2024-05-30T16:52:37.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.375+0000 7efe49000640 1 Processor -- start 2024-05-30T16:52:37.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.376+0000 7efe49000640 1 -- start start 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.376+0000 7efe49000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe440ce560 0x7efe4414aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.376+0000 7efe49000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe440d0430 0x7efe440ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.376+0000 7efe49000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4415f0c0 0x7efe44072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe49000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe4415c390 con 0x7efe4415f0c0 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe49000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efe440d0da0 con 0x7efe440d0430 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe49000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efe4415c670 con 0x7efe440ce560 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4415f0c0 0x7efe44072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4415f0c0 0x7efe44072a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41464/0 (socket says 172.21.15.67:41464) 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe43e00640 1 -- 172.21.15.67:0/1508046969 learned_addr learned my addr 172.21.15.67:0/1508046969 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe43400640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe440ce560 0x7efe4414aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:37.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe42a00640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe440d0430 0x7efe440ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:37.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe43e00640 1 -- 172.21.15.67:0/1508046969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe440ce560 msgr2=0x7efe4414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:37.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe43e00640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe440ce560 0x7efe4414aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:37.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe43e00640 1 -- 172.21.15.67:0/1508046969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe440d0430 msgr2=0x7efe440ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:37.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe43e00640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe440d0430 0x7efe440ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:37.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.377+0000 7efe43e00640 1 -- 172.21.15.67:0/1508046969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4414b450 con 0x7efe4415f0c0 2024-05-30T16:52:37.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.378+0000 7efe43e00640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4415f0c0 0x7efe44072a90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c780 tx=0x7efe3c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:37.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.378+0000 7efe3b400640 1 -- 172.21.15.67:0/1508046969 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c016030 con 0x7efe4415f0c0 2024-05-30T16:52:37.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.378+0000 7efe3b400640 1 -- 172.21.15.67:0/1508046969 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe3c006b50 con 0x7efe4415f0c0 2024-05-30T16:52:37.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.378+0000 7efe3b400640 1 -- 172.21.15.67:0/1508046969 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00a850 con 0x7efe4415f0c0 2024-05-30T16:52:37.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.378+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4414b730 con 0x7efe4415f0c0 2024-05-30T16:52:37.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.378+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4414bcf0 con 0x7efe4415f0c0 2024-05-30T16:52:37.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.379+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe440689e0 con 0x7efe4415f0c0 2024-05-30T16:52:37.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.380+0000 7efe3b400640 1 -- 172.21.15.67:0/1508046969 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7efe3c019070 con 0x7efe4415f0c0 2024-05-30T16:52:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.381+0000 7efe3b400640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efe2006d100 0x7efe2006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.381+0000 7efe3b400640 1 -- 172.21.15.67:0/1508046969 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7efe3c08cd00 con 0x7efe4415f0c0 2024-05-30T16:52:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.382+0000 7efe43400640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efe2006d100 0x7efe2006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:37.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.382+0000 7efe43400640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efe2006d100 0x7efe2006f5c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe34000f60 tx=0x7efe34009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:37.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.386+0000 7efe3b400640 1 -- 172.21.15.67:0/1508046969 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe3c05a180 con 0x7efe4415f0c0 2024-05-30T16:52:37.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.633+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7efe4415d420 con 0x7efe4415f0c0 2024-05-30T16:52:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.786+0000 7efe3b400640 1 -- 172.21.15.67:0/1508046969 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v36) ==== 136+0+0 (secure 0 0 0) 0x7efe3c059a80 con 0x7efe4415f0c0 2024-05-30T16:52:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efe2006d100 msgr2=0x7efe2006f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efe2006d100 0x7efe2006f5c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe34000f60 tx=0x7efe34009040 comp rx=0 tx=0).stop 2024-05-30T16:52:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4415f0c0 msgr2=0x7efe44072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4415f0c0 0x7efe44072a90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c780 tx=0x7efe3c00cc40 comp rx=0 tx=0).stop 2024-05-30T16:52:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 shutdown_connections 2024-05-30T16:52:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efe2006d100 0x7efe2006f5c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4415f0c0 0x7efe44072a90 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe440d0430 0x7efe440ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 --2- 172.21.15.67:0/1508046969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe440ce560 0x7efe4414aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 >> 172.21.15.67:0/1508046969 conn(0x7efe4406bae0 msgr2=0x7efe4406bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.790+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 shutdown_connections 2024-05-30T16:52:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:37.791+0000 7efe49000640 1 -- 172.21.15.67:0/1508046969 wait complete. 2024-05-30T16:52:37.817 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:52:38.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.036+0000 7f9e93200640 1 Processor -- start 2024-05-30T16:52:38.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.036+0000 7f9e93200640 1 -- start start 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.037+0000 7f9e93200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9e8c0d16a0 0x7f9e8c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.037+0000 7f9e93200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e8c0cf870 0x7f9e8c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.037+0000 7f9e93200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e8c0cfce0 con 0x7f9e8c15e870 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.037+0000 7f9e93200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9e8c0d2150 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.037+0000 7f9e93200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9e8c15ec50 con 0x7f9e8c0cf870 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.038+0000 7f9e91400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9e8c0d16a0 0x7f9e8c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.038+0000 7f9e91400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9e8c0d16a0 0x7f9e8c0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41466/0 (socket says 172.21.15.67:41466) 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.038+0000 7f9e91400640 1 -- 172.21.15.67:0/2663508718 learned_addr learned my addr 172.21.15.67:0/2663508718 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.038+0000 7f9e92800640 1 --2- 172.21.15.67:0/2663508718 >> v2:172.21.15.169:3300/0 conn(0x7f9e8c0cf870 0x7f9e8c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.038+0000 7f9e91400640 1 -- 172.21.15.67:0/2663508718 >> v2:172.21.15.169:3300/0 conn(0x7f9e8c0cf870 msgr2=0x7f9e8c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.038+0000 7f9e91400640 1 --2- 172.21.15.67:0/2663508718 >> v2:172.21.15.169:3300/0 conn(0x7f9e8c0cf870 0x7f9e8c0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.038+0000 7f9e91400640 1 -- 172.21.15.67:0/2663508718 >> v1:172.21.15.169:6789/0 conn(0x7f9e8c15e870 legacy=0x7f9e8c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.038+0000 7f9e91400640 1 -- 172.21.15.67:0/2663508718 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8c0cff80 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.039+0000 7f9e91400640 1 --2- 172.21.15.67:0/2663508718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 0x7f9e8c0cf330 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c163400 tx=0x7f9e7c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853544d0c39ee9f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e90a00640 1 -- 172.21.15.67:0/2663508718 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c004030 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e90a00640 1 -- 172.21.15.67:0/2663508718 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7c0041d0 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e90a00640 1 -- 172.21.15.67:0/2663508718 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c003610 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e93200640 1 -- 172.21.15.67:0/2663508718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 msgr2=0x7f9e8c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e93200640 1 --2- 172.21.15.67:0/2663508718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 0x7f9e8c0cf330 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c163400 tx=0x7f9e7c02a210 comp rx=0 tx=0).stop 2024-05-30T16:52:38.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e93200640 1 -- 172.21.15.67:0/2663508718 shutdown_connections 2024-05-30T16:52:38.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e93200640 1 --2- 172.21.15.67:0/2663508718 >> v2:172.21.15.169:3300/0 conn(0x7f9e8c0cf870 0x7f9e8c0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e93200640 1 --2- 172.21.15.67:0/2663508718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 0x7f9e8c0cf330 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e93200640 1 -- 172.21.15.67:0/2663508718 >> 172.21.15.67:0/2663508718 conn(0x7f9e8c08bfc0 msgr2=0x7f9e8c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:38.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e93200640 1 -- 172.21.15.67:0/2663508718 shutdown_connections 2024-05-30T16:52:38.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.040+0000 7f9e93200640 1 -- 172.21.15.67:0/2663508718 wait complete. 2024-05-30T16:52:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.042+0000 7f9e93200640 1 Processor -- start 2024-05-30T16:52:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.042+0000 7f9e93200640 1 -- start start 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.043+0000 7f9e93200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e8c0cf870 0x7f9e8c0b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.043+0000 7f9e93200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 0x7f9e8c1610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.043+0000 7f9e93200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e8c15e870 0x7f9e8c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.043+0000 7f9e93200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c0d0820 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.043+0000 7f9e93200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c0d0540 con 0x7f9e8c15e870 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.043+0000 7f9e93200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8c0d4fb0 con 0x7f9e8c0cf870 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 0x7f9e8c1610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 0x7f9e8c1610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41482/0 (socket says 172.21.15.67:41482) 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91400640 1 -- 172.21.15.67:0/958967426 learned_addr learned my addr 172.21.15.67:0/958967426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91e00640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e8c0cf870 0x7f9e8c0b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91400640 1 -- 172.21.15.67:0/958967426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e8c0cf870 msgr2=0x7f9e8c0b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91400640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e8c0cf870 0x7f9e8c0b8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e92800640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e8c15e870 0x7f9e8c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91400640 1 -- 172.21.15.67:0/958967426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e8c15e870 msgr2=0x7f9e8c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91400640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e8c15e870 0x7f9e8c159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91400640 1 -- 172.21.15.67:0/958967426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8c0b8dc0 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.044+0000 7f9e91400640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 0x7f9e8c1610e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c02a250 tx=0x7f9e7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.045+0000 7f9e86000640 1 -- 172.21.15.67:0/958967426 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c041020 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.045+0000 7f9e86000640 1 -- 172.21.15.67:0/958967426 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7c003c50 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.045+0000 7f9e86000640 1 -- 172.21.15.67:0/958967426 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c004030 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.045+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8c0b7320 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.045+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8c0b78c0 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.046+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8c0689e0 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.047+0000 7f9e86000640 1 -- 172.21.15.67:0/958967426 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f9e7c044020 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.048+0000 7f9e86000640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e7006d0d0 0x7f9e7006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.048+0000 7f9e86000640 1 -- 172.21.15.67:0/958967426 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f9e7c0ad600 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.049+0000 7f9e91e00640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e7006d0d0 0x7f9e7006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.049+0000 7f9e91e00640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e7006d0d0 0x7f9e7006f590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9e80003d90 tx=0x7f9e80009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:38.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.053+0000 7f9e86000640 1 -- 172.21.15.67:0/958967426 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e7c04b030 con 0x7f9e8c0d16a0 2024-05-30T16:52:38.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.287+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9e8c0ba500 con 0x7f9e7006d0d0 2024-05-30T16:52:38.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.290+0000 7f9e86000640 1 -- 172.21.15.67:0/958967426 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9e8c0ba500 con 0x7f9e7006d0d0 2024-05-30T16:52:38.291 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:52:38.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:52:38.291 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:52:38.291 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:52:38.291 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.292+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e7006d0d0 msgr2=0x7f9e7006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.292+0000 7f9e93200640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e7006d0d0 0x7f9e7006f590 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9e80003d90 tx=0x7f9e80009040 comp rx=0 tx=0).stop 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 msgr2=0x7f9e8c1610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 0x7f9e8c1610e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c02a250 tx=0x7f9e7c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 shutdown_connections 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e7006d0d0 0x7f9e7006f590 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e8c15e870 0x7f9e8c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e8c0d16a0 0x7f9e8c1610e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 --2- 172.21.15.67:0/958967426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e8c0cf870 0x7f9e8c0b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 >> 172.21.15.67:0/958967426 conn(0x7f9e8c08bfc0 msgr2=0x7f9e8c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 shutdown_connections 2024-05-30T16:52:38.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.293+0000 7f9e93200640 1 -- 172.21.15.67:0/958967426 wait complete. 2024-05-30T16:52:38.314 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:52:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.528+0000 7f16b7200640 1 Processor -- start 2024-05-30T16:52:38.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.528+0000 7f16b7200640 1 -- start start 2024-05-30T16:52:38.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.529+0000 7f16b7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b815e870 0x7f16b80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.529+0000 7f16b7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16b80cfa40 0x7f16b80d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.529+0000 7f16b7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16b815ec50 con 0x7f16b80d14d0 2024-05-30T16:52:38.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.529+0000 7f16b7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f16b80d2150 con 0x7f16b80cfa40 2024-05-30T16:52:38.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.529+0000 7f16b7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16b80cff80 con 0x7f16b815e870 2024-05-30T16:52:38.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.530+0000 7f16b5400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f16b80d14d0 0x7f16b80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40572/0 (socket says 172.21.15.67:40572) 2024-05-30T16:52:38.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.530+0000 7f16b5400640 1 -- 172.21.15.67:0/1385676506 learned_addr learned my addr 172.21.15.67:0/1385676506 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.530+0000 7f16b5e00640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b815e870 0x7f16b80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.530+0000 7f16b6800640 1 --2- 172.21.15.67:0/1385676506 >> v2:172.21.15.169:3300/0 conn(0x7f16b80cfa40 0x7f16b80d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.530+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809264398 0 0) 0x7f16b815ec50 con 0x7f16b80d14d0 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.530+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1698003520 con 0x7f16b80d14d0 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1912176442 0 0) 0x7f1698003520 con 0x7f16b80d14d0 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b815e870 msgr2=0x7f16b80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b815e870 0x7f16b80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 >> v2:172.21.15.169:3300/0 conn(0x7f16b80cfa40 msgr2=0x7f16b80d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 --2- 172.21.15.67:0/1385676506 >> v2:172.21.15.169:3300/0 conn(0x7f16b80cfa40 0x7f16b80d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b80d0260 con 0x7f16b80d14d0 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f16a0001b70 con 0x7f16b80d14d0 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 >> v1:172.21.15.67:6789/0 conn(0x7f16b80d14d0 legacy=0x7f16b80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b815e870 0x7f16980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f1698004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698004d40 0x7f16980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f16b80d2150 con 0x7f1698003f30 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16b80cff80 con 0x7f1698004d40 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f169800b9c0 con 0x7f16b815e870 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.531+0000 7f16b5400640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f1698004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.532+0000 7f16b5e00640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b815e870 0x7f16980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.532+0000 7f16b5400640 1 -- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b815e870 msgr2=0x7f16980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.532+0000 7f16b5400640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b815e870 0x7f16980039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.532+0000 7f16b6800640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698004d40 0x7f16980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.532+0000 7f16b5400640 1 -- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698004d40 msgr2=0x7f16980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.532+0000 7f16b5400640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698004d40 0x7f16980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.532+0000 7f16b5400640 1 -- 172.21.15.67:0/1385676506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1698003520 con 0x7f1698003f30 2024-05-30T16:52:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.532+0000 7f16b6800640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698004d40 0x7f16980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.533+0000 7f16b5400640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f1698004800 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f16a000c580 tx=0x7f1698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a470f49159206c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.533+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a000c1a0 con 0x7f1698003f30 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.533+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a002f030 con 0x7f1698003f30 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.533+0000 7f16b4a00640 1 -- 172.21.15.67:0/1385676506 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0007310 con 0x7f1698003f30 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.534+0000 7f16b7200640 1 -- 172.21.15.67:0/1385676506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 msgr2=0x7f1698004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.534+0000 7f16b7200640 1 --2- 172.21.15.67:0/1385676506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f1698004800 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f16a000c580 tx=0x7f1698005830 comp rx=0 tx=0).stop 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.534+0000 7f16b6800640 1 -- 172.21.15.67:0/1385676506 reap_dead start 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.534+0000 7f16b7200640 1 -- 172.21.15.67:0/1385676506 shutdown_connections 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.534+0000 7f16b7200640 1 -- 172.21.15.67:0/1385676506 >> 172.21.15.67:0/1385676506 conn(0x7f16b808bfc0 msgr2=0x7f16b80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.534+0000 7f16b7200640 1 -- 172.21.15.67:0/1385676506 shutdown_connections 2024-05-30T16:52:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.534+0000 7f16b7200640 1 -- 172.21.15.67:0/1385676506 wait complete. 2024-05-30T16:52:38.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.536+0000 7f16b7200640 1 Processor -- start 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.536+0000 7f16b7200640 1 -- start start 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f16b8159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b80b7ae0 0x7f16b81f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b81f85b0 0x7f16b81f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f16b80d4fb0 con 0x7f1698003f30 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16b80d0820 con 0x7f16b81f85b0 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16b80d0f90 con 0x7f16b80b7ae0 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f16b8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f16b8159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41492/0 (socket says 172.21.15.67:41492) 2024-05-30T16:52:38.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b5e00640 1 -- 172.21.15.67:0/2664138063 learned_addr learned my addr 172.21.15.67:0/2664138063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:38.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b6800640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b81f85b0 0x7f16b81f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b5400640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b80b7ae0 0x7f16b81f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b5e00640 1 -- 172.21.15.67:0/2664138063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b80b7ae0 msgr2=0x7f16b81f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b5e00640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b80b7ae0 0x7f16b81f8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b5e00640 1 -- 172.21.15.67:0/2664138063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b81f85b0 msgr2=0x7f16b81f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b5e00640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b81f85b0 0x7f16b81f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.537+0000 7f16b5e00640 1 -- 172.21.15.67:0/2664138063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b80b8ad0 con 0x7f1698003f30 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.538+0000 7f16b5e00640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f16b8159620 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f16a8010860 tx=0x7f16a8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.538+0000 7f16ae000640 1 -- 172.21.15.67:0/2664138063 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800f040 con 0x7f1698003f30 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.538+0000 7f16ae000640 1 -- 172.21.15.67:0/2664138063 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a800eb00 con 0x7f1698003f30 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.538+0000 7f16ae000640 1 -- 172.21.15.67:0/2664138063 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a801c400 con 0x7f1698003f30 2024-05-30T16:52:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.538+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b80b7f60 con 0x7f1698003f30 2024-05-30T16:52:38.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.538+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b80b8510 con 0x7f1698003f30 2024-05-30T16:52:38.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.540+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b80689e0 con 0x7f1698003f30 2024-05-30T16:52:38.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.541+0000 7f16ae000640 1 -- 172.21.15.67:0/2664138063 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f16a801c5a0 con 0x7f1698003f30 2024-05-30T16:52:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.542+0000 7f16ae000640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169406d100 0x7f169406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.542+0000 7f16ae000640 1 -- 172.21.15.67:0/2664138063 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f16a808aa50 con 0x7f1698003f30 2024-05-30T16:52:38.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.542+0000 7f16b5400640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169406d100 0x7f169406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.543+0000 7f16b5400640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169406d100 0x7f169406f5c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f16a0033000 tx=0x7f16a0006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:38.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.546+0000 7f16ae000640 1 -- 172.21.15.67:0/2664138063 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16a8018030 con 0x7f1698003f30 2024-05-30T16:52:38.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.756+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f16b80d3c90 con 0x7f1698003f30 2024-05-30T16:52:38.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.757+0000 7f16ae000640 1 -- 172.21.15.67:0/2664138063 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f16a805a120 con 0x7f1698003f30 2024-05-30T16:52:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169406d100 msgr2=0x7f169406f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169406d100 0x7f169406f5c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f16a0033000 tx=0x7f16a0006000 comp rx=0 tx=0).stop 2024-05-30T16:52:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 msgr2=0x7f16b8159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f16b8159620 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f16a8010860 tx=0x7f16a8010d20 comp rx=0 tx=0).stop 2024-05-30T16:52:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 shutdown_connections 2024-05-30T16:52:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f169406d100 0x7f169406f5c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b81f85b0 0x7f16b81f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16b80b7ae0 0x7f16b81f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 --2- 172.21.15.67:0/2664138063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1698003f30 0x7f16b8159620 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 >> 172.21.15.67:0/2664138063 conn(0x7f16b808bfc0 msgr2=0x7f16b80b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 shutdown_connections 2024-05-30T16:52:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.760+0000 7f16b7200640 1 -- 172.21.15.67:0/2664138063 wait complete. 2024-05-30T16:52:38.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-05-30T16:52:38.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.984+0000 7f68ce600640 1 Processor -- start 2024-05-30T16:52:38.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.984+0000 7f68ce600640 1 -- start start 2024-05-30T16:52:38.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.986+0000 7f68ce600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80c7320 0x7f68c8145a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.986+0000 7f68ce600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f68c80cc810 0x7f68c80ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.986+0000 7f68ce600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68c80c7820 con 0x7f68c80b3e30 2024-05-30T16:52:38.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.986+0000 7f68ce600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f68c80c8430 con 0x7f68c80cc810 2024-05-30T16:52:38.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.986+0000 7f68ce600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68c80c7b00 con 0x7f68c80c7320 2024-05-30T16:52:38.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.986+0000 7f68cd200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f68c80b3e30 0x7f68c80b4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40574/0 (socket says 172.21.15.67:40574) 2024-05-30T16:52:38.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.986+0000 7f68cd200640 1 -- 172.21.15.67:0/1913357286 learned_addr learned my addr 172.21.15.67:0/1913357286 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:38.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.986+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545597441 0 0) 0x7f68c80c7820 con 0x7f68c80b3e30 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68ac003520 con 0x7f68c80b3e30 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7e00640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80c7320 0x7f68c8145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68cdc00640 1 --2- 172.21.15.67:0/1913357286 >> v2:172.21.15.169:3300/0 conn(0x7f68c80cc810 0x7f68c80ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2527711081 0 0) 0x7f68ac003520 con 0x7f68c80b3e30 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80c7320 msgr2=0x7f68c8145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80c7320 0x7f68c8145a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 >> v2:172.21.15.169:3300/0 conn(0x7f68c80cc810 msgr2=0x7f68c80ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 --2- 172.21.15.67:0/1913357286 >> v2:172.21.15.169:3300/0 conn(0x7f68c80cc810 0x7f68c80ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c80c7de0 con 0x7f68c80b3e30 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f68b8001670 con 0x7f68c80b3e30 2024-05-30T16:52:38.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 >> v1:172.21.15.67:6789/0 conn(0x7f68c80b3e30 legacy=0x7f68c80b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68ac0039b0 0x7f68ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80c7320 0x7f68ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68c80c8430 con 0x7f68ac004d40 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f68c80c7b00 con 0x7f68ac0039b0 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68ac00ba70 con 0x7f68c80c7320 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.987+0000 7f68cdc00640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.988+0000 7f68cd200640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68ac0039b0 0x7f68ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.988+0000 7f68c7e00640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80c7320 0x7f68ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.988+0000 7f68cdc00640 1 -- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80c7320 msgr2=0x7f68ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.988+0000 7f68cdc00640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80c7320 0x7f68ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.988+0000 7f68cdc00640 1 -- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68ac0039b0 msgr2=0x7f68ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.988+0000 7f68cdc00640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68ac0039b0 0x7f68ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.988+0000 7f68cdc00640 1 -- 172.21.15.67:0/1913357286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac003520 con 0x7f68ac004d40 2024-05-30T16:52:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.989+0000 7f68cdc00640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68ac0051c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f68c0007390 tx=0x7f68ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30af6c03d7247ee server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:38.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.989+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0009be0 con 0x7f68ac004d40 2024-05-30T16:52:38.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.989+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c002e640 con 0x7f68ac004d40 2024-05-30T16:52:38.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.989+0000 7f68c7400640 1 -- 172.21.15.67:0/1913357286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c002ea10 con 0x7f68ac004d40 2024-05-30T16:52:38.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.990+0000 7f68ce600640 1 -- 172.21.15.67:0/1913357286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 msgr2=0x7f68ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.990+0000 7f68ce600640 1 --2- 172.21.15.67:0/1913357286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68ac0051c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f68c0007390 tx=0x7f68ac005830 comp rx=0 tx=0).stop 2024-05-30T16:52:38.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.990+0000 7f68cdc00640 1 -- 172.21.15.67:0/1913357286 reap_dead start 2024-05-30T16:52:38.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.990+0000 7f68ce600640 1 -- 172.21.15.67:0/1913357286 shutdown_connections 2024-05-30T16:52:38.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.990+0000 7f68ce600640 1 -- 172.21.15.67:0/1913357286 >> 172.21.15.67:0/1913357286 conn(0x7f68c80d2250 msgr2=0x7f68c80d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:38.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.990+0000 7f68ce600640 1 -- 172.21.15.67:0/1913357286 shutdown_connections 2024-05-30T16:52:38.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.990+0000 7f68ce600640 1 -- 172.21.15.67:0/1913357286 wait complete. 2024-05-30T16:52:38.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.992+0000 7f68ce600640 1 Processor -- start 2024-05-30T16:52:38.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.992+0000 7f68ce600640 1 -- start start 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.993+0000 7f68ce600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68c80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.993+0000 7f68ce600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c81f4cd0 0x7f68c80b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.993+0000 7f68ce600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80b7130 0x7f68c80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.993+0000 7f68ce600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81454a0 con 0x7f68ac004d40 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.993+0000 7f68ce600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f68c80c88c0 con 0x7f68c81f4cd0 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.993+0000 7f68ce600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68c8145620 con 0x7f68c80b7130 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.993+0000 7f68cd200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68c80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68cd200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68c80b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41512/0 (socket says 172.21.15.67:41512) 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68cd200640 1 -- 172.21.15.67:0/2112629624 learned_addr learned my addr 172.21.15.67:0/2112629624 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68c7e00640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c81f4cd0 0x7f68c80b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68cd200640 1 -- 172.21.15.67:0/2112629624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80b7130 msgr2=0x7f68c80cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68cd200640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80b7130 0x7f68c80cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68cd200640 1 -- 172.21.15.67:0/2112629624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c81f4cd0 msgr2=0x7f68c80b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68cd200640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c81f4cd0 0x7f68c80b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68cd200640 1 -- 172.21.15.67:0/2112629624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c80b54d0 con 0x7f68ac004d40 2024-05-30T16:52:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68c7e00640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c81f4cd0 0x7f68c80b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:52:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.994+0000 7f68cd200640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68c80b4f90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f68b800e870 tx=0x7f68b800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.995+0000 7f68c4c00640 1 -- 172.21.15.67:0/2112629624 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b800c870 con 0x7f68ac004d40 2024-05-30T16:52:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.995+0000 7f68c4c00640 1 -- 172.21.15.67:0/2112629624 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b8018070 con 0x7f68ac004d40 2024-05-30T16:52:38.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.995+0000 7f68c4c00640 1 -- 172.21.15.67:0/2112629624 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b800cc50 con 0x7f68ac004d40 2024-05-30T16:52:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.995+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c80b75d0 con 0x7f68ac004d40 2024-05-30T16:52:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.995+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c80c97c0 con 0x7f68ac004d40 2024-05-30T16:52:38.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.996+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c80690f0 con 0x7f68ac004d40 2024-05-30T16:52:38.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.997+0000 7f68c4c00640 1 -- 172.21.15.67:0/2112629624 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f68b801d460 con 0x7f68ac004d40 2024-05-30T16:52:38.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.998+0000 7f68c4c00640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68a806d0d0 0x7f68a806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:38.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.998+0000 7f68c4c00640 1 -- 172.21.15.67:0/2112629624 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f68b808cce0 con 0x7f68ac004d40 2024-05-30T16:52:38.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.998+0000 7f68c7e00640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68a806d0d0 0x7f68a806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:38.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:38.999+0000 7f68c7e00640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68a806d0d0 0x7f68a806f590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f68c80b8110 tx=0x7f68b4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:39.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.002+0000 7f68c4c00640 1 -- 172.21.15.67:0/2112629624 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68b805a0d0 con 0x7f68ac004d40 2024-05-30T16:52:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.208+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f68c80b18e0 con 0x7f68ac004d40 2024-05-30T16:52:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.214+0000 7f68c4c00640 1 -- 172.21.15.67:0/2112629624 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v15)=0 v15) ==== 136+0+0 (secure 0 0 0) 0x7f68b80599d0 con 0x7f68ac004d40 2024-05-30T16:52:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68a806d0d0 msgr2=0x7f68a806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68a806d0d0 0x7f68a806f590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f68c80b8110 tx=0x7f68b4008040 comp rx=0 tx=0).stop 2024-05-30T16:52:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 msgr2=0x7f68c80b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68c80b4f90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f68b800e870 tx=0x7f68b800ed30 comp rx=0 tx=0).stop 2024-05-30T16:52:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 shutdown_connections 2024-05-30T16:52:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f68a806d0d0 0x7f68a806f590 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68c80b7130 0x7f68c80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c81f4cd0 0x7f68c80b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 --2- 172.21.15.67:0/2112629624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68ac004d40 0x7f68c80b4f90 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.217+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 >> 172.21.15.67:0/2112629624 conn(0x7f68c80d2250 msgr2=0x7f68c80ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.218+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 shutdown_connections 2024-05-30T16:52:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.218+0000 7f68ce600640 1 -- 172.21.15.67:0/2112629624 wait complete. 2024-05-30T16:52:39.241 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.240+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.241 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.240+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.241 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-30T16:52:39.242 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.240+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.242 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.240+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.242 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.240+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.436+0000 7ff181800640 1 Processor -- start 2024-05-30T16:52:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.436+0000 7ff181800640 1 -- start start 2024-05-30T16:52:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.438+0000 7ff181800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 0x7ff17c1608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.438+0000 7ff181800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff17c0d0a30 0x7ff17c0b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.438+0000 7ff181800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff17c15c7d0 con 0x7ff17c06ef00 2024-05-30T16:52:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.438+0000 7ff181800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff17c0ad9a0 con 0x7ff17c0d0a30 2024-05-30T16:52:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.438+0000 7ff181800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff17c160e10 con 0x7ff17c15c3f0 2024-05-30T16:52:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.438+0000 7ff180e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff17c0d0a30 0x7ff17c0b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.439+0000 7ff180e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff17c0d0a30 0x7ff17c0b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41542/0 (socket says 172.21.15.67:41542) 2024-05-30T16:52:39.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.439+0000 7ff180e00640 1 -- 172.21.15.67:0/3125395255 learned_addr learned my addr 172.21.15.67:0/3125395255 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:39.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.439+0000 7ff17be00640 1 --2- 172.21.15.67:0/3125395255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 0x7ff17c1608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:39.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.439+0000 7ff180e00640 1 -- 172.21.15.67:0/3125395255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 msgr2=0x7ff17c1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:39.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.439+0000 7ff180e00640 1 --2- 172.21.15.67:0/3125395255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 0x7ff17c1608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.439+0000 7ff180e00640 1 -- 172.21.15.67:0/3125395255 >> v1:172.21.15.169:6789/0 conn(0x7ff17c06ef00 legacy=0x7ff17c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:39.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.439+0000 7ff180e00640 1 -- 172.21.15.67:0/3125395255 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c0ad3e0 con 0x7ff17c0d0a30 2024-05-30T16:52:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.440+0000 7ff180e00640 1 --2- 172.21.15.67:0/3125395255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c0d0a30 0x7ff17c0b30b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff174009e50 tx=0x7ff17402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcf38d53bff2447 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.440+0000 7ff17aa00640 1 -- 172.21.15.67:0/3125395255 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174036070 con 0x7ff17c0d0a30 2024-05-30T16:52:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.440+0000 7ff17aa00640 1 -- 172.21.15.67:0/3125395255 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff174031600 con 0x7ff17c0d0a30 2024-05-30T16:52:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.440+0000 7ff17aa00640 1 -- 172.21.15.67:0/3125395255 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1740319d0 con 0x7ff17c0d0a30 2024-05-30T16:52:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.441+0000 7ff181800640 1 -- 172.21.15.67:0/3125395255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c0d0a30 msgr2=0x7ff17c0b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.441+0000 7ff181800640 1 --2- 172.21.15.67:0/3125395255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c0d0a30 0x7ff17c0b30b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff174009e50 tx=0x7ff17402a6c0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.441+0000 7ff181800640 1 -- 172.21.15.67:0/3125395255 shutdown_connections 2024-05-30T16:52:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.441+0000 7ff181800640 1 --2- 172.21.15.67:0/3125395255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c0d0a30 0x7ff17c0b30b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.441+0000 7ff181800640 1 --2- 172.21.15.67:0/3125395255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 0x7ff17c1608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.441+0000 7ff181800640 1 -- 172.21.15.67:0/3125395255 >> 172.21.15.67:0/3125395255 conn(0x7ff17c06bae0 msgr2=0x7ff17c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:39.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.441+0000 7ff181800640 1 -- 172.21.15.67:0/3125395255 shutdown_connections 2024-05-30T16:52:39.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.441+0000 7ff181800640 1 -- 172.21.15.67:0/3125395255 wait complete. 2024-05-30T16:52:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.443+0000 7ff181800640 1 Processor -- start 2024-05-30T16:52:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.443+0000 7ff181800640 1 -- start start 2024-05-30T16:52:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.444+0000 7ff181800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c06ef00 0x7ff17c1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.444+0000 7ff181800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0d0a30 0x7ff17c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.444+0000 7ff181800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 0x7ff17c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.444+0000 7ff181800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1603b0 con 0x7ff17c06ef00 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.444+0000 7ff181800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1600d0 con 0x7ff17c0d0a30 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.444+0000 7ff181800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff17c0ad090 con 0x7ff17c15c3f0 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c06ef00 0x7ff17c1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c06ef00 0x7ff17c1523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41546/0 (socket says 172.21.15.67:41546) 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17be00640 1 -- 172.21.15.67:0/134493737 learned_addr learned my addr 172.21.15.67:0/134493737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff180e00640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 0x7ff17c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17b400640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0d0a30 0x7ff17c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17be00640 1 -- 172.21.15.67:0/134493737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 msgr2=0x7ff17c1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17be00640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 0x7ff17c1579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17be00640 1 -- 172.21.15.67:0/134493737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0d0a30 msgr2=0x7ff17c1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17be00640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0d0a30 0x7ff17c1574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17be00640 1 -- 172.21.15.67:0/134493737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c15cbf0 con 0x7ff17c06ef00 2024-05-30T16:52:39.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.445+0000 7ff17be00640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c06ef00 0x7ff17c1523e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00d5f0 tx=0x7ff16c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:39.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.446+0000 7ff173e00640 1 -- 172.21.15.67:0/134493737 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0125f0 con 0x7ff17c06ef00 2024-05-30T16:52:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.446+0000 7ff173e00640 1 -- 172.21.15.67:0/134493737 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16c019070 con 0x7ff17c06ef00 2024-05-30T16:52:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.446+0000 7ff173e00640 1 -- 172.21.15.67:0/134493737 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0129d0 con 0x7ff17c06ef00 2024-05-30T16:52:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.446+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c15ced0 con 0x7ff17c06ef00 2024-05-30T16:52:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.446+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c15d490 con 0x7ff17c06ef00 2024-05-30T16:52:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.447+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17c0689e0 con 0x7ff17c06ef00 2024-05-30T16:52:39.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.448+0000 7ff173e00640 1 -- 172.21.15.67:0/134493737 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7ff16c012b70 con 0x7ff17c06ef00 2024-05-30T16:52:39.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.449+0000 7ff173e00640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff15c06d0d0 0x7ff15c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.449+0000 7ff173e00640 1 -- 172.21.15.67:0/134493737 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7ff16c08d220 con 0x7ff17c06ef00 2024-05-30T16:52:39.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.450+0000 7ff17b400640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff15c06d0d0 0x7ff15c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:39.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.450+0000 7ff17b400640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff15c06d0d0 0x7ff15c06f590 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff17c152cc0 tx=0x7ff168009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:39.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.454+0000 7ff173e00640 1 -- 172.21.15.67:0/134493737 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff16c05a6a0 con 0x7ff17c06ef00 2024-05-30T16:52:39.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.686+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7ff17c15c9a0 con 0x7ff15c06d0d0 2024-05-30T16:52:39.698 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.697+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.698 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.697+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.698 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.697+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.698 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.697+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.698 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:52:39.697+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:52:39.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.770+0000 7ff173e00640 1 -- 172.21.15.67:0/134493737 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff17c15c9a0 con 0x7ff15c06d0d0 2024-05-30T16:52:39.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff15c06d0d0 msgr2=0x7ff15c06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff15c06d0d0 0x7ff15c06f590 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff17c152cc0 tx=0x7ff168009040 comp rx=0 tx=0).stop 2024-05-30T16:52:39.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c06ef00 msgr2=0x7ff17c1523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c06ef00 0x7ff17c1523e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00d5f0 tx=0x7ff16c00dab0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 shutdown_connections 2024-05-30T16:52:39.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff15c06d0d0 0x7ff15c06f590 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c15c3f0 0x7ff17c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0d0a30 0x7ff17c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 --2- 172.21.15.67:0/134493737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff17c06ef00 0x7ff17c1523e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 >> 172.21.15.67:0/134493737 conn(0x7ff17c06bae0 msgr2=0x7ff17c15ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:39.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 shutdown_connections 2024-05-30T16:52:39.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.773+0000 7ff181800640 1 -- 172.21.15.67:0/134493737 wait complete. 2024-05-30T16:52:39.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:52:39.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.985+0000 7f6ab3200640 1 Processor -- start 2024-05-30T16:52:39.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.985+0000 7f6ab3200640 1 -- start start 2024-05-30T16:52:39.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.986+0000 7f6ab3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6aac15b5f0 0x7f6aac0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.987+0000 7f6ab3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aac15fef0 con 0x7f6aac0b4320 2024-05-30T16:52:39.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.987+0000 7f6ab3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aac15b9d0 con 0x7f6aac0b04a0 2024-05-30T16:52:39.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.987+0000 7f6ab3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f6aac06fed0 con 0x7f6aac15b5f0 2024-05-30T16:52:39.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.987+0000 7f6aab400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6aac0b4320 0x7f6aac15f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40582/0 (socket says 172.21.15.67:40582) 2024-05-30T16:52:39.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.987+0000 7f6aab400640 1 -- 172.21.15.67:0/1872453230 learned_addr learned my addr 172.21.15.67:0/1872453230 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:52:39.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.987+0000 7f6aabe00640 1 --2- 172.21.15.67:0/1872453230 >> v2:172.21.15.67:3300/0 conn(0x7f6aac15b5f0 0x7f6aac0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:39.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.987+0000 7f6aabe00640 1 -- 172.21.15.67:0/1872453230 >> v1:172.21.15.169:6789/0 conn(0x7f6aac0b04a0 legacy=0x7f6aac15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:52:39.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.987+0000 7f6aabe00640 1 -- 172.21.15.67:0/1872453230 >> v1:172.21.15.67:6789/0 conn(0x7f6aac0b4320 legacy=0x7f6aac15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.987+0000 7f6aabe00640 1 -- 172.21.15.67:0/1872453230 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac0b4890 con 0x7f6aac15b5f0 2024-05-30T16:52:39.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.989+0000 7f6aabe00640 1 --2- 172.21.15.67:0/1872453230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac15b5f0 0x7f6aac0b4e80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0097d0 tx=0x7f6a9c02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6c8536eb8a6620 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:39.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.989+0000 7f6aaaa00640 1 -- 172.21.15.67:0/1872453230 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c037070 con 0x7f6aac15b5f0 2024-05-30T16:52:39.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.989+0000 7f6aaaa00640 1 -- 172.21.15.67:0/1872453230 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9c02ae00 con 0x7f6aac15b5f0 2024-05-30T16:52:39.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.989+0000 7f6aaaa00640 1 -- 172.21.15.67:0/1872453230 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c032660 con 0x7f6aac15b5f0 2024-05-30T16:52:39.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.989+0000 7f6ab3200640 1 -- 172.21.15.67:0/1872453230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac15b5f0 msgr2=0x7f6aac0b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.989+0000 7f6ab3200640 1 --2- 172.21.15.67:0/1872453230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac15b5f0 0x7f6aac0b4e80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0097d0 tx=0x7f6a9c02a0f0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.990+0000 7f6ab3200640 1 -- 172.21.15.67:0/1872453230 shutdown_connections 2024-05-30T16:52:39.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.990+0000 7f6ab3200640 1 --2- 172.21.15.67:0/1872453230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac15b5f0 0x7f6aac0b4e80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.990+0000 7f6ab3200640 1 -- 172.21.15.67:0/1872453230 >> 172.21.15.67:0/1872453230 conn(0x7f6aac09aa30 msgr2=0x7f6aac0b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:39.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.990+0000 7f6ab3200640 1 -- 172.21.15.67:0/1872453230 shutdown_connections 2024-05-30T16:52:39.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.990+0000 7f6ab3200640 1 -- 172.21.15.67:0/1872453230 wait complete. 2024-05-30T16:52:39.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.991+0000 7f6ab3200640 1 Processor -- start 2024-05-30T16:52:39.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.991+0000 7f6ab3200640 1 -- start start 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.992+0000 7f6ab3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac0b04a0 0x7f6aac1456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.992+0000 7f6ab3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aac0b4320 0x7f6aac150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.992+0000 7f6ab3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aac15b5f0 0x7f6aac14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.992+0000 7f6ab3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b01c0 con 0x7f6aac0b04a0 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.992+0000 7f6ab3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0af9c0 con 0x7f6aac0b4320 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.992+0000 7f6ab3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6aac0b6fe0 con 0x7f6aac15b5f0 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aabe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac0b04a0 0x7f6aac1456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aabe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac0b04a0 0x7f6aac1456d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41586/0 (socket says 172.21.15.67:41586) 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aabe00640 1 -- 172.21.15.67:0/3517728578 learned_addr learned my addr 172.21.15.67:0/3517728578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aabe00640 1 -- 172.21.15.67:0/3517728578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aac15b5f0 msgr2=0x7f6aac14c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aabe00640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aac15b5f0 0x7f6aac14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aabe00640 1 -- 172.21.15.67:0/3517728578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aac0b4320 msgr2=0x7f6aac150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aabe00640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aac0b4320 0x7f6aac150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aabe00640 1 -- 172.21.15.67:0/3517728578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac1463b0 con 0x7f6aac0b04a0 2024-05-30T16:52:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aabe00640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac0b04a0 0x7f6aac1456d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0098f0 tx=0x7f6a9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.993+0000 7f6aa3e00640 1 -- 172.21.15.67:0/3517728578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c037040 con 0x7f6aac0b04a0 2024-05-30T16:52:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.994+0000 7f6aa3e00640 1 -- 172.21.15.67:0/3517728578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9c004030 con 0x7f6aac0b04a0 2024-05-30T16:52:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.994+0000 7f6aa3e00640 1 -- 172.21.15.67:0/3517728578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c004420 con 0x7f6aac0b04a0 2024-05-30T16:52:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.994+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac14f4d0 con 0x7f6aac0b04a0 2024-05-30T16:52:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.994+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac14fa90 con 0x7f6aac0b04a0 2024-05-30T16:52:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.995+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac0690f0 con 0x7f6aac0b04a0 2024-05-30T16:52:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.996+0000 7f6aa3e00640 1 -- 172.21.15.67:0/3517728578 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 91754+0+0 (secure 0 0 0) 0x7f6a9c03a070 con 0x7f6aac0b04a0 2024-05-30T16:52:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.997+0000 7f6aa3e00640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a8806d0d0 0x7f6a8806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:52:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.997+0000 7f6aa3e00640 1 -- 172.21.15.67:0/3517728578 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f6a9c0ad410 con 0x7f6aac0b04a0 2024-05-30T16:52:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.997+0000 7f6aab400640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a8806d0d0 0x7f6a8806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:52:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:39.998+0000 7f6aab400640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a8806d0d0 0x7f6a8806f590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a94000f60 tx=0x7f6a94007a80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:52:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.002+0000 7f6aa3e00640 1 -- 172.21.15.67:0/3517728578 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a9c04a030 con 0x7f6aac0b04a0 2024-05-30T16:52:40.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.239+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f6aac0d2060 con 0x7f6a8806d0d0 2024-05-30T16:52:40.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.245+0000 7f6aa3e00640 1 -- 172.21.15.67:0/3517728578 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f6aac0d2060 con 0x7f6a8806d0d0 2024-05-30T16:52:40.246 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1 2024-05-30T16:52:40.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a8806d0d0 msgr2=0x7f6a8806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:40.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a8806d0d0 0x7f6a8806f590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6a94000f60 tx=0x7f6a94007a80 comp rx=0 tx=0).stop 2024-05-30T16:52:40.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac0b04a0 msgr2=0x7f6aac1456d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:52:40.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac0b04a0 0x7f6aac1456d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0098f0 tx=0x7f6a9c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:52:40.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 shutdown_connections 2024-05-30T16:52:40.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a8806d0d0 0x7f6a8806f590 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:40.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aac15b5f0 0x7f6aac14c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:40.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aac0b4320 0x7f6aac150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:40.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 --2- 172.21.15.67:0/3517728578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aac0b04a0 0x7f6aac1456d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:52:40.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 >> 172.21.15.67:0/3517728578 conn(0x7f6aac09aa30 msgr2=0x7f6aac074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:52:40.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 shutdown_connections 2024-05-30T16:52:40.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:52:40.248+0000 7f6ab3200640 1 -- 172.21.15.67:0/3517728578 wait complete. 2024-05-30T16:52:40.268 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1 2024-05-30T16:52:40.268 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:52:40.268 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:40.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:40.320 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:52:41.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:52:41.389 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:52:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:52:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:52:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.3152 s, 65.8 MB/s 2024-05-30T16:52:56.722 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:52:56.722 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:52:56.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:52:56.777 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:53:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:53:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:53:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.6076 s, 73.5 MB/s 2024-05-30T16:53:11.453 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:53:11.453 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:53:11.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:53:11.510 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:53:11.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:53:11.561 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:53:25.436 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:53:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:53:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.8694 s, 77.4 MB/s 2024-05-30T16:53:25.438 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-30T16:53:25.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.667+0000 7f11d9a00640 1 Processor -- start 2024-05-30T16:53:25.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.667+0000 7f11d9a00640 1 -- start start 2024-05-30T16:53:25.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.668+0000 7f11d9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f11d415e5a0 0x7f11d40d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:25.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.668+0000 7f11d9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cfb40 0x7f11d40d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.668+0000 7f11d9a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f11d40cffa0 0x7f11d40d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.669+0000 7f11d9a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f11d40d2b30 con 0x7f11d415e5a0 2024-05-30T16:53:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.669+0000 7f11d9a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f11d40d3500 con 0x7f11d40cffa0 2024-05-30T16:53:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.669+0000 7f11d9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11d40d2200 con 0x7f11d40cfb40 2024-05-30T16:53:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.669+0000 7f11d3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f11d415e5a0 0x7f11d40d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.669+0000 7f11d3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f11d415e5a0 0x7f11d40d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53316/0 (socket says 172.21.15.67:53316) 2024-05-30T16:53:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.669+0000 7f11d3e00640 1 -- 172.21.15.67:0/3072569732 learned_addr learned my addr 172.21.15.67:0/3072569732 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.669+0000 7f11d3e00640 1 -- 172.21.15.67:0/3072569732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cfb40 msgr2=0x7f11d40d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.669+0000 7f11d3400640 1 --2- 172.21.15.67:0/3072569732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cfb40 0x7f11d40d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.669+0000 7f11d9000640 1 --2- 172.21.15.67:0/3072569732 >> v2:172.21.15.169:3300/0 conn(0x7f11d40cffa0 0x7f11d40d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.670+0000 7f11d3e00640 1 --2- 172.21.15.67:0/3072569732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cfb40 0x7f11d40d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.670+0000 7f11d3e00640 1 -- 172.21.15.67:0/3072569732 >> v2:172.21.15.169:3300/0 conn(0x7f11d40cffa0 msgr2=0x7f11d40d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.670+0000 7f11d3e00640 1 --2- 172.21.15.67:0/3072569732 >> v2:172.21.15.169:3300/0 conn(0x7f11d40cffa0 0x7f11d40d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.670+0000 7f11d3e00640 1 -- 172.21.15.67:0/3072569732 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d40d24e0 con 0x7f11d415e5a0 2024-05-30T16:53:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.670+0000 7f11d9000640 1 --2- 172.21.15.67:0/3072569732 >> v2:172.21.15.169:3300/0 conn(0x7f11d40cffa0 0x7f11d40d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:25.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.671+0000 7f11d3e00640 1 --2- 172.21.15.67:0/3072569732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d415e5a0 0x7f11d40d2fc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f11d40d4460 tx=0x7f11c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b156733ffff9d398 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:25.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.671+0000 7f11d2a00640 1 -- 172.21.15.67:0/3072569732 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4037070 con 0x7f11d415e5a0 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.671+0000 7f11d2a00640 1 -- 172.21.15.67:0/3072569732 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c4032440 con 0x7f11d415e5a0 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.671+0000 7f11d2a00640 1 -- 172.21.15.67:0/3072569732 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4032810 con 0x7f11d415e5a0 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.672+0000 7f11d9a00640 1 -- 172.21.15.67:0/3072569732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d415e5a0 msgr2=0x7f11d40d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.672+0000 7f11d9a00640 1 --2- 172.21.15.67:0/3072569732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d415e5a0 0x7f11d40d2fc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f11d40d4460 tx=0x7f11c402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.672+0000 7f11d9a00640 1 -- 172.21.15.67:0/3072569732 shutdown_connections 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.672+0000 7f11d9a00640 1 --2- 172.21.15.67:0/3072569732 >> v2:172.21.15.169:3300/0 conn(0x7f11d40cffa0 0x7f11d40d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.672+0000 7f11d9a00640 1 --2- 172.21.15.67:0/3072569732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cfb40 0x7f11d40d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.672+0000 7f11d9a00640 1 --2- 172.21.15.67:0/3072569732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d415e5a0 0x7f11d40d2fc0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.672+0000 7f11d9a00640 1 -- 172.21.15.67:0/3072569732 >> 172.21.15.67:0/3072569732 conn(0x7f11d408bfc0 msgr2=0x7f11d40d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.672+0000 7f11d9a00640 1 -- 172.21.15.67:0/3072569732 shutdown_connections 2024-05-30T16:53:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.672+0000 7f11d9a00640 1 -- 172.21.15.67:0/3072569732 wait complete. 2024-05-30T16:53:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.674+0000 7f11d9a00640 1 Processor -- start 2024-05-30T16:53:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.674+0000 7f11d9a00640 1 -- start start 2024-05-30T16:53:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.675+0000 7f11d9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d40cfb40 0x7f11d40b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.675+0000 7f11d9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cffa0 0x7f11d4160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.675+0000 7f11d9a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d415e5a0 0x7f11d40b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.675+0000 7f11d9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f11d415eb00 con 0x7f11d40cfb40 2024-05-30T16:53:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.675+0000 7f11d9a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11d415e980 con 0x7f11d415e5a0 2024-05-30T16:53:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.675+0000 7f11d9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11d40d5220 con 0x7f11d40cffa0 2024-05-30T16:53:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.675+0000 7f11d3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d40cfb40 0x7f11d40b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.675+0000 7f11d3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d40cfb40 0x7f11d40b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53332/0 (socket says 172.21.15.67:53332) 2024-05-30T16:53:25.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.675+0000 7f11d3e00640 1 -- 172.21.15.67:0/54394988 learned_addr learned my addr 172.21.15.67:0/54394988 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:25.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d3400640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cffa0 0x7f11d4160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:25.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d9000640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d415e5a0 0x7f11d40b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:25.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d3e00640 1 -- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cffa0 msgr2=0x7f11d4160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:25.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d3e00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cffa0 0x7f11d4160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d3e00640 1 -- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d415e5a0 msgr2=0x7f11d40b94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:25.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d3e00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d415e5a0 0x7f11d40b94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d3e00640 1 -- 172.21.15.67:0/54394988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d40b53e0 con 0x7f11d40cfb40 2024-05-30T16:53:25.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d3400640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cffa0 0x7f11d4160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d9000640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d415e5a0 0x7f11d40b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.676+0000 7f11d3e00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d40cfb40 0x7f11d40b4ab0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.677+0000 7f11cbe00640 1 -- 172.21.15.67:0/54394988 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4041070 con 0x7f11d40cfb40 2024-05-30T16:53:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.677+0000 7f11cbe00640 1 -- 172.21.15.67:0/54394988 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c403c870 con 0x7f11d40cfb40 2024-05-30T16:53:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.677+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d40b89e0 con 0x7f11d40cfb40 2024-05-30T16:53:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.677+0000 7f11cbe00640 1 -- 172.21.15.67:0/54394988 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4037040 con 0x7f11d40cfb40 2024-05-30T16:53:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.677+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d40b8fa0 con 0x7f11d40cfb40 2024-05-30T16:53:25.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.678+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d40689e0 con 0x7f11d40cfb40 2024-05-30T16:53:25.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.679+0000 7f11cbe00640 1 -- 172.21.15.67:0/54394988 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 91856+0+0 (secure 0 0 0) 0x7f11c4044070 con 0x7f11d40cfb40 2024-05-30T16:53:25.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.680+0000 7f11cbe00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f11b406d1f0 0x7f11b406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:25.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.680+0000 7f11cbe00640 1 -- 172.21.15.67:0/54394988 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f11c40ab290 con 0x7f11d40cfb40 2024-05-30T16:53:25.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.680+0000 7f11d3400640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f11b406d1f0 0x7f11b406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:25.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.681+0000 7f11d3400640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f11b406d1f0 0x7f11b406f6b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11d415ad40 tx=0x7f11c0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.685+0000 7f11cbe00640 1 -- 172.21.15.67:0/54394988 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11c407b040 con 0x7f11d40cfb40 2024-05-30T16:53:25.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.923+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f11d40d1d30 con 0x7f11b406d1f0 2024-05-30T16:53:25.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.956+0000 7f11cbe00640 1 -- 172.21.15.67:0/54394988 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f11d40d1d30 con 0x7f11b406d1f0 2024-05-30T16:53:25.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f11b406d1f0 msgr2=0x7f11b406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:25.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f11b406d1f0 0x7f11b406f6b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11d415ad40 tx=0x7f11c0008040 comp rx=0 tx=0).stop 2024-05-30T16:53:25.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d40cfb40 msgr2=0x7f11d40b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:25.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d40cfb40 0x7f11d40b4ab0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c40046c0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 shutdown_connections 2024-05-30T16:53:25.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f11b406d1f0 0x7f11b406f6b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d415e5a0 0x7f11d40b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d40cffa0 0x7f11d4160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 --2- 172.21.15.67:0/54394988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11d40cfb40 0x7f11d40b4ab0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:25.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 >> 172.21.15.67:0/54394988 conn(0x7f11d408bfc0 msgr2=0x7f11d4159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:25.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 shutdown_connections 2024-05-30T16:53:25.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:25.959+0000 7f11d9a00640 1 -- 172.21.15.67:0/54394988 wait complete. 2024-05-30T16:53:25.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:53:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.183+0000 7f3ccd400640 1 Processor -- start 2024-05-30T16:53:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.183+0000 7f3ccd400640 1 -- start start 2024-05-30T16:53:26.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.184+0000 7f3ccd400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3cc80b32c0 0x7f3cc80b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:26.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.184+0000 7f3ccd400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc80d0060 con 0x7f3cc80d0640 2024-05-30T16:53:26.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.184+0000 7f3ccd400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc815cb10 con 0x7f3cc815d5d0 2024-05-30T16:53:26.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.184+0000 7f3ccd400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3cc80adbb0 con 0x7f3cc80b32c0 2024-05-30T16:53:26.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.184+0000 7f3cc6800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3cc80d0640 0x7f3cc80d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41014/0 (socket says 172.21.15.67:41014) 2024-05-30T16:53:26.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.184+0000 7f3cc6800640 1 -- 172.21.15.67:0/2292514204 learned_addr learned my addr 172.21.15.67:0/2292514204 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc7200640 1 --2- 172.21.15.67:0/2292514204 >> v2:172.21.15.169:3300/0 conn(0x7f3cc80b32c0 0x7f3cc80b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3288488398 0 0) 0x7f3cc80d0060 con 0x7f3cc80d0640 2024-05-30T16:53:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ca8003520 con 0x7f3cc80d0640 2024-05-30T16:53:26.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3641071172 0 0) 0x7f3ca8003520 con 0x7f3cc80d0640 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 >> v2:172.21.15.169:3300/0 conn(0x7f3cc80b32c0 msgr2=0x7f3cc80b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 --2- 172.21.15.67:0/2292514204 >> v2:172.21.15.169:3300/0 conn(0x7f3cc80b32c0 0x7f3cc80b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 >> v1:172.21.15.169:6789/0 conn(0x7f3cc815d5d0 legacy=0x7f3cc80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc80ad5f0 con 0x7f3cc80d0640 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f3cb8001670 con 0x7f3cc80d0640 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 >> v1:172.21.15.67:6789/0 conn(0x7f3cc80d0640 legacy=0x7f3cc80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.185+0000 7f3cc5400640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5400640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca80051c0 0x7f3ca800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca800c130 con 0x7f3ca8004800 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca800c2e0 con 0x7f3ca80051c0 2024-05-30T16:53:26.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca800c4e0 con 0x7f3ca80039b0 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5e00640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc6800640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc7200640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca80051c0 0x7f3ca800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5e00640 1 -- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ca80039b0 msgr2=0x7f3ca8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5e00640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5e00640 1 -- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca80051c0 msgr2=0x7f3ca800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5e00640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca80051c0 0x7f3ca800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc5e00640 1 -- 172.21.15.67:0/2292514204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca8003520 con 0x7f3ca8004800 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc6800640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc7200640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca80051c0 0x7f3ca800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:26.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.186+0000 7f3cc7200640 1 -- 172.21.15.67:0/2292514204 reap_dead start 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.187+0000 7f3cc5e00640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0001e50 tx=0x7f3ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6369dc36e84b3978 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.188+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0001a70 con 0x7f3ca8004800 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.188+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb002ae40 con 0x7f3ca8004800 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.188+0000 7f3cc5400640 1 -- 172.21.15.67:0/2292514204 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0003bb0 con 0x7f3ca8004800 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.188+0000 7f3ccd400640 1 -- 172.21.15.67:0/2292514204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca8004800 msgr2=0x7f3ca8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.188+0000 7f3ccd400640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0001e50 tx=0x7f3ca8005830 comp rx=0 tx=0).stop 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.188+0000 7f3ccd400640 1 -- 172.21.15.67:0/2292514204 shutdown_connections 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.188+0000 7f3ccd400640 1 --2- 172.21.15.67:0/2292514204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.188+0000 7f3ccd400640 1 -- 172.21.15.67:0/2292514204 >> 172.21.15.67:0/2292514204 conn(0x7f3cc806bd20 msgr2=0x7f3cc806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:26.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.188+0000 7f3ccd400640 1 -- 172.21.15.67:0/2292514204 shutdown_connections 2024-05-30T16:53:26.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.189+0000 7f3ccd400640 1 -- 172.21.15.67:0/2292514204 wait complete. 2024-05-30T16:53:26.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.190+0000 7f3ccd400640 1 Processor -- start 2024-05-30T16:53:26.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.190+0000 7f3ccd400640 1 -- start start 2024-05-30T16:53:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.191+0000 7f3ccd400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca8004800 0x7f3cc80b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.191+0000 7f3ccd400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81f5790 0x7f3cc81f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.191+0000 7f3ccd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cc814a770 0x7f3cc814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.191+0000 7f3ccd400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc815d0b0 con 0x7f3cc814a770 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.191+0000 7f3ccd400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc815cdd0 con 0x7f3ca8004800 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.191+0000 7f3ccd400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc815da50 con 0x7f3cc81f5790 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cc814a770 0x7f3cc814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cc814a770 0x7f3cc814ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53346/0 (socket says 172.21.15.67:53346) 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc7200640 1 -- 172.21.15.67:0/513032752 learned_addr learned my addr 172.21.15.67:0/513032752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc6800640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca8004800 0x7f3cc80b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc5e00640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81f5790 0x7f3cc81f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc7200640 1 -- 172.21.15.67:0/513032752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81f5790 msgr2=0x7f3cc81f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:26.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc7200640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81f5790 0x7f3cc81f5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc7200640 1 -- 172.21.15.67:0/513032752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca8004800 msgr2=0x7f3cc80b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc7200640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca8004800 0x7f3cc80b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.192+0000 7f3cc7200640 1 -- 172.21.15.67:0/513032752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc814b4c0 con 0x7f3cc814a770 2024-05-30T16:53:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.193+0000 7f3cc7200640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cc814a770 0x7f3cc814ac00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80ae290 tx=0x7f3cc0010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.193+0000 7f3cbea00640 1 -- 172.21.15.67:0/513032752 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc000f040 con 0x7f3cc814a770 2024-05-30T16:53:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.193+0000 7f3cbea00640 1 -- 172.21.15.67:0/513032752 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc001b070 con 0x7f3cc814a770 2024-05-30T16:53:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.193+0000 7f3cbea00640 1 -- 172.21.15.67:0/513032752 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0004430 con 0x7f3cc814a770 2024-05-30T16:53:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.193+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc814b7a0 con 0x7f3cc814a770 2024-05-30T16:53:26.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.193+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc80b2d70 con 0x7f3cc814a770 2024-05-30T16:53:26.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.194+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc814b9a0 con 0x7f3cc814a770 2024-05-30T16:53:26.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.196+0000 7f3cbea00640 1 -- 172.21.15.67:0/513032752 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 91856+0+0 (secure 0 0 0) 0x7f3cc000e670 con 0x7f3cc814a770 2024-05-30T16:53:26.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.197+0000 7f3cbea00640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3ca406d220 0x7f3ca406f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:26.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.197+0000 7f3cbea00640 1 -- 172.21.15.67:0/513032752 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f3cc008a530 con 0x7f3cc814a770 2024-05-30T16:53:26.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.197+0000 7f3cc6800640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3ca406d220 0x7f3ca406f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:26.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.198+0000 7f3cc6800640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3ca406d220 0x7f3ca406f6e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8069c40 tx=0x7f3cb8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.201+0000 7f3cbea00640 1 -- 172.21.15.67:0/513032752 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3cc814b9a0 con 0x7f3cc814a770 2024-05-30T16:53:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.443+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f3cc80689e0 con 0x7f3ca406d220 2024-05-30T16:53:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.451+0000 7f3cbea00640 1 -- 172.21.15.67:0/513032752 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f3cc80689e0 con 0x7f3ca406d220 2024-05-30T16:53:26.452 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1 2024-05-30T16:53:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3ca406d220 msgr2=0x7f3ca406f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3ca406d220 0x7f3ca406f6e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8069c40 tx=0x7f3cb8009040 comp rx=0 tx=0).stop 2024-05-30T16:53:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cc814a770 msgr2=0x7f3cc814ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cc814a770 0x7f3cc814ac00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80ae290 tx=0x7f3cc0010c20 comp rx=0 tx=0).stop 2024-05-30T16:53:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 shutdown_connections 2024-05-30T16:53:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3ca406d220 0x7f3ca406f6e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cc814a770 0x7f3cc814ac00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cc81f5790 0x7f3cc81f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 --2- 172.21.15.67:0/513032752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca8004800 0x7f3cc80b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 >> 172.21.15.67:0/513032752 conn(0x7f3cc806bd20 msgr2=0x7f3cc80cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 shutdown_connections 2024-05-30T16:53:26.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:26.455+0000 7f3ccd400640 1 -- 172.21.15.67:0/513032752 wait complete. 2024-05-30T16:53:26.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:53:26.474 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1 2024-05-30T16:53:26.474 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:53:26.531 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1 2024-05-30T16:53:26.532 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3138256896" 2024-05-30T16:53:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:53:26.533 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-30T16:53:28.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:53:28.533 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1 2024-05-30T16:53:28.534 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:53:28.604 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1 2024-05-30T16:53:28.604 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3138256896" 2024-05-30T16:53:28.604 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:53:28.605 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-30T16:53:30.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:53:30.606 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1 2024-05-30T16:53:30.606 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:53:30.675 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/21c40302-b2f7-4b7e-bf32-0e263588d4a1 2024-05-30T16:53:30.675 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3221225472" 2024-05-30T16:53:30.675 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:53:30.676 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-30T16:53:30.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15501 2024-05-30T16:53:30.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.905+0000 7ff22fe00640 1 Processor -- start 2024-05-30T16:53:30.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.906+0000 7ff22fe00640 1 -- start start 2024-05-30T16:53:30.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.907+0000 7ff22fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff230161d00 0x7ff2300b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:30.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.907+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2300d1550 0x7ff2300d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:30.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.907+0000 7ff22fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2300cfb40 con 0x7ff2300d4d90 2024-05-30T16:53:30.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.907+0000 7ff22fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff2300d2040 con 0x7ff230161d00 2024-05-30T16:53:30.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.907+0000 7ff22fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff2300cfdc0 con 0x7ff2300d1550 2024-05-30T16:53:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.908+0000 7ff22e000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff2300d4d90 0x7ff2300d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54900/0 (socket says 172.21.15.67:54900) 2024-05-30T16:53:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.908+0000 7ff22e000640 1 -- 172.21.15.67:0/168345079 learned_addr learned my addr 172.21.15.67:0/168345079 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.908+0000 7ff22ea00640 1 --2- 172.21.15.67:0/168345079 >> v2:172.21.15.67:3300/0 conn(0x7ff230161d00 0x7ff2300b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.908+0000 7ff22f400640 1 --2- 172.21.15.67:0/168345079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2300d1550 0x7ff2300d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.908+0000 7ff22ea00640 1 -- 172.21.15.67:0/168345079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2300d1550 msgr2=0x7ff2300d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.908+0000 7ff22ea00640 1 --2- 172.21.15.67:0/168345079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2300d1550 0x7ff2300d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.908+0000 7ff22ea00640 1 -- 172.21.15.67:0/168345079 >> v1:172.21.15.67:6789/0 conn(0x7ff2300d4d90 legacy=0x7ff2300d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.908+0000 7ff22ea00640 1 -- 172.21.15.67:0/168345079 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2300d00a0 con 0x7ff230161d00 2024-05-30T16:53:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.908+0000 7ff22d600640 1 -- 172.21.15.67:0/168345079 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464332407 0 0) 0x7ff21c001110 con 0x7ff2300d4d90 2024-05-30T16:53:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.910+0000 7ff22ea00640 1 --2- 172.21.15.67:0/168345079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff230161d00 0x7ff2300b3d00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff2200097a0 tx=0x7ff22002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23173b1087f12b90 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.910+0000 7ff22d600640 1 -- 172.21.15.67:0/168345079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220037070 con 0x7ff230161d00 2024-05-30T16:53:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.910+0000 7ff22d600640 1 -- 172.21.15.67:0/168345079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff220032440 con 0x7ff230161d00 2024-05-30T16:53:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.910+0000 7ff22d600640 1 -- 172.21.15.67:0/168345079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220032810 con 0x7ff230161d00 2024-05-30T16:53:30.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.911+0000 7ff22fe00640 1 -- 172.21.15.67:0/168345079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff230161d00 msgr2=0x7ff2300b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:30.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.911+0000 7ff22fe00640 1 --2- 172.21.15.67:0/168345079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff230161d00 0x7ff2300b3d00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff2200097a0 tx=0x7ff22002a180 comp rx=0 tx=0).stop 2024-05-30T16:53:30.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.911+0000 7ff22fe00640 1 -- 172.21.15.67:0/168345079 shutdown_connections 2024-05-30T16:53:30.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.911+0000 7ff22fe00640 1 --2- 172.21.15.67:0/168345079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2300d1550 0x7ff2300d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:30.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.911+0000 7ff22fe00640 1 --2- 172.21.15.67:0/168345079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff230161d00 0x7ff2300b3d00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:30.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.911+0000 7ff22fe00640 1 -- 172.21.15.67:0/168345079 >> 172.21.15.67:0/168345079 conn(0x7ff23008bfc0 msgr2=0x7ff2300d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:30.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.911+0000 7ff22fe00640 1 -- 172.21.15.67:0/168345079 shutdown_connections 2024-05-30T16:53:30.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.911+0000 7ff22fe00640 1 -- 172.21.15.67:0/168345079 wait complete. 2024-05-30T16:53:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.913+0000 7ff22fe00640 1 Processor -- start 2024-05-30T16:53:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.913+0000 7ff22fe00640 1 -- start start 2024-05-30T16:53:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.914+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2300d1550 0x7ff2301513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.914+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2300d4d90 0x7ff230160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.914+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff230161d00 0x7ff2301f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.914+0000 7ff22fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff2300b5560 con 0x7ff2300d1550 2024-05-30T16:53:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.914+0000 7ff22fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff2300d2290 con 0x7ff2300d4d90 2024-05-30T16:53:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.914+0000 7ff22fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff2300b56e0 con 0x7ff230161d00 2024-05-30T16:53:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.914+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2300d1550 0x7ff2301513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.914+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2300d1550 0x7ff2301513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33488/0 (socket says 172.21.15.67:33488) 2024-05-30T16:53:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.914+0000 7ff22ea00640 1 -- 172.21.15.67:0/170228478 learned_addr learned my addr 172.21.15.67:0/170228478 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:30.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff22ea00640 1 -- 172.21.15.67:0/170228478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff230161d00 msgr2=0x7ff2301f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff22ea00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff230161d00 0x7ff2301f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff22ea00640 1 -- 172.21.15.67:0/170228478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2300d4d90 msgr2=0x7ff230160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff22ea00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2300d4d90 0x7ff230160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff22ea00640 1 -- 172.21.15.67:0/170228478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2300b72e0 con 0x7ff2300d1550 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff22ea00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2300d1550 0x7ff2301513f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff220000c00 tx=0x7ff2200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff226a00640 1 -- 172.21.15.67:0/170228478 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220037050 con 0x7ff2300d1550 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff226a00640 1 -- 172.21.15.67:0/170228478 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff220038040 con 0x7ff2300d1550 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2300b75c0 con 0x7ff2300d1550 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.915+0000 7ff226a00640 1 -- 172.21.15.67:0/170228478 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220004490 con 0x7ff2300d1550 2024-05-30T16:53:30.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.916+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2300b99e0 con 0x7ff2300d1550 2024-05-30T16:53:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.917+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2300689e0 con 0x7ff2300d1550 2024-05-30T16:53:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.918+0000 7ff226a00640 1 -- 172.21.15.67:0/170228478 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 91856+0+0 (secure 0 0 0) 0x7ff220030900 con 0x7ff2300d1550 2024-05-30T16:53:30.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.919+0000 7ff226a00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff21006d1f0 0x7ff21006f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:30.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.919+0000 7ff226a00640 1 -- 172.21.15.67:0/170228478 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7ff2200aaf30 con 0x7ff2300d1550 2024-05-30T16:53:30.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.919+0000 7ff22e000640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff21006d1f0 0x7ff21006f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:30.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.920+0000 7ff22e000640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff21006d1f0 0x7ff21006f6b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff2300ca280 tx=0x7ff21c001310 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:30.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:30.925+0000 7ff226a00640 1 -- 172.21.15.67:0/170228478 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff22007adb0 con 0x7ff2300d1550 2024-05-30T16:53:31.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.166+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15501", "target": ["mon-mgr", ""]}) -- 0x7ff2300b7090 con 0x7ff21006d1f0 2024-05-30T16:53:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.318+0000 7ff226a00640 1 -- 172.21.15.67:0/170228478 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2300b7090 con 0x7ff21006d1f0 2024-05-30T16:53:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.321+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff21006d1f0 msgr2=0x7ff21006f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.321+0000 7ff22fe00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff21006d1f0 0x7ff21006f6b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff2300ca280 tx=0x7ff21c001310 comp rx=0 tx=0).stop 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.321+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2300d1550 msgr2=0x7ff2301513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.321+0000 7ff22fe00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2300d1550 0x7ff2301513f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff220000c00 tx=0x7ff2200046c0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.322+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 shutdown_connections 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.322+0000 7ff22fe00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff21006d1f0 0x7ff21006f6b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.322+0000 7ff22fe00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff230161d00 0x7ff2301f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.322+0000 7ff22fe00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2300d4d90 0x7ff230160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.322+0000 7ff22fe00640 1 --2- 172.21.15.67:0/170228478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2300d1550 0x7ff2301513f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.322+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 >> 172.21.15.67:0/170228478 conn(0x7ff23008bfc0 msgr2=0x7ff2300d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.322+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 shutdown_connections 2024-05-30T16:53:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.322+0000 7ff22fe00640 1 -- 172.21.15.67:0/170228478 wait complete. 2024-05-30T16:53:31.324 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.323+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.324 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.323+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.324 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.323+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.324 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.323+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.324 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.323+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15502 2024-05-30T16:53:31.353 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.351+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.353 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.351+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.353 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.351+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.353 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.351+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.353 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:31.352+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.542+0000 7f0778c00640 1 Processor -- start 2024-05-30T16:53:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.542+0000 7f0778c00640 1 -- start start 2024-05-30T16:53:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0778c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774158670 0x7f0774071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0778c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f077406db20 0x7f0774072460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0778c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f077406dfa0 0x7f07740729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0778c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f077415beb0 con 0x7f077406db20 2024-05-30T16:53:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0778c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f077406f670 con 0x7f077406dfa0 2024-05-30T16:53:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0778c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f077415b330 con 0x7f0774158670 2024-05-30T16:53:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0772a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f077406db20 0x7f0774072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0772a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f077406db20 0x7f0774072460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33544/0 (socket says 172.21.15.67:33544) 2024-05-30T16:53:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0772a00640 1 -- 172.21.15.67:0/2027845075 learned_addr learned my addr 172.21.15.67:0/2027845075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0773400640 1 --2- 172.21.15.67:0/2027845075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774158670 0x7f0774071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0772a00640 1 -- 172.21.15.67:0/2027845075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774158670 msgr2=0x7f0774071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0772a00640 1 --2- 172.21.15.67:0/2027845075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774158670 0x7f0774071ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0772a00640 1 -- 172.21.15.67:0/2027845075 >> v2:172.21.15.169:3300/0 conn(0x7f077406dfa0 msgr2=0x7f07740729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.544+0000 7f0772a00640 1 --2- 172.21.15.67:0/2027845075 >> v2:172.21.15.169:3300/0 conn(0x7f077406dfa0 0x7f07740729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.545+0000 7f0772a00640 1 -- 172.21.15.67:0/2027845075 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077415b610 con 0x7f077406db20 2024-05-30T16:53:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.546+0000 7f0772a00640 1 --2- 172.21.15.67:0/2027845075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406db20 0x7f0774072460 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f07740b2c70 tx=0x7f076002c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c43ab590c3ed3d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:31.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.546+0000 7f0772000640 1 -- 172.21.15.67:0/2027845075 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760032700 con 0x7f077406db20 2024-05-30T16:53:31.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.546+0000 7f0772000640 1 -- 172.21.15.67:0/2027845075 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07600328a0 con 0x7f077406db20 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.546+0000 7f0772000640 1 -- 172.21.15.67:0/2027845075 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760032c70 con 0x7f077406db20 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.546+0000 7f0778c00640 1 -- 172.21.15.67:0/2027845075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406db20 msgr2=0x7f0774072460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.546+0000 7f0778c00640 1 --2- 172.21.15.67:0/2027845075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406db20 0x7f0774072460 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f07740b2c70 tx=0x7f076002c800 comp rx=0 tx=0).stop 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.547+0000 7f0778c00640 1 -- 172.21.15.67:0/2027845075 shutdown_connections 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.547+0000 7f0778c00640 1 --2- 172.21.15.67:0/2027845075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406db20 0x7f0774072460 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.547+0000 7f0778c00640 1 --2- 172.21.15.67:0/2027845075 >> v2:172.21.15.169:3300/0 conn(0x7f077406dfa0 0x7f07740729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.547+0000 7f0778c00640 1 --2- 172.21.15.67:0/2027845075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774158670 0x7f0774071ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.547+0000 7f0778c00640 1 -- 172.21.15.67:0/2027845075 >> 172.21.15.67:0/2027845075 conn(0x7f07740b1db0 msgr2=0x7f07740b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.547+0000 7f0778c00640 1 -- 172.21.15.67:0/2027845075 shutdown_connections 2024-05-30T16:53:31.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.547+0000 7f0778c00640 1 -- 172.21.15.67:0/2027845075 wait complete. 2024-05-30T16:53:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.548+0000 7f0778c00640 1 Processor -- start 2024-05-30T16:53:31.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.549+0000 7f0778c00640 1 -- start start 2024-05-30T16:53:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.549+0000 7f0778c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f077406db20 0x7f077414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.549+0000 7f0778c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406dfa0 0x7f077414f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0778c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0774158670 0x7f077414ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0778c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0774072000 con 0x7f077406dfa0 2024-05-30T16:53:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0778c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f077415bbd0 con 0x7f0774158670 2024-05-30T16:53:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0778c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f077415af00 con 0x7f077406db20 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0772a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406dfa0 0x7f077414f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0772a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406dfa0 0x7f077414f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33560/0 (socket says 172.21.15.67:33560) 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0772a00640 1 -- 172.21.15.67:0/1108347532 learned_addr learned my addr 172.21.15.67:0/1108347532 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0773400640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f077406db20 0x7f077414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0773e00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0774158670 0x7f077414ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0772a00640 1 -- 172.21.15.67:0/1108347532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f077406db20 msgr2=0x7f077414b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0772a00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f077406db20 0x7f077414b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0772a00640 1 -- 172.21.15.67:0/1108347532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0774158670 msgr2=0x7f077414ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0772a00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0774158670 0x7f077414ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.550+0000 7f0772a00640 1 -- 172.21.15.67:0/1108347532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0774150310 con 0x7f077406dfa0 2024-05-30T16:53:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.551+0000 7f0772a00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406dfa0 0x7f077414f930 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f076002c840 tx=0x7f0760004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:31.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.551+0000 7f076b400640 1 -- 172.21.15.67:0/1108347532 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760044070 con 0x7f077406dfa0 2024-05-30T16:53:31.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.551+0000 7f076b400640 1 -- 172.21.15.67:0/1108347532 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0760034d00 con 0x7f077406dfa0 2024-05-30T16:53:31.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.551+0000 7f076b400640 1 -- 172.21.15.67:0/1108347532 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076003e680 con 0x7f077406dfa0 2024-05-30T16:53:31.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.551+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077414e0a0 con 0x7f077406dfa0 2024-05-30T16:53:31.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.551+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077414e660 con 0x7f077406dfa0 2024-05-30T16:53:31.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.553+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07740689e0 con 0x7f077406dfa0 2024-05-30T16:53:31.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.554+0000 7f076b400640 1 -- 172.21.15.67:0/1108347532 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) ==== 91856+0+0 (secure 0 0 0) 0x7f0760047070 con 0x7f077406dfa0 2024-05-30T16:53:31.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.555+0000 7f076b400640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f075406d1f0 0x7f075406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:31.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.555+0000 7f076b400640 1 -- 172.21.15.67:0/1108347532 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f07600ad660 con 0x7f077406dfa0 2024-05-30T16:53:31.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.555+0000 7f0773400640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f075406d1f0 0x7f075406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:31.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.556+0000 7f0773400640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f075406d1f0 0x7f075406f6b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f076400a970 tx=0x7f0764008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:31.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.559+0000 7f076b400640 1 -- 172.21.15.67:0/1108347532 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0760031070 con 0x7f077406dfa0 2024-05-30T16:53:31.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:31.802+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15502", "target": ["mon-mgr", ""]}) -- 0x7f07740b2470 con 0x7f075406d1f0 2024-05-30T16:53:32.330 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.329+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.330 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.329+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.330 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.329+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.330 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.329+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.330 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.329+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.449+0000 7f076b400640 1 -- 172.21.15.67:0/1108347532 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 20) ==== 91958+0+0 (secure 0 0 0) 0x7f076003c040 con 0x7f077406dfa0 2024-05-30T16:53:32.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.610+0000 7f076b400640 1 -- 172.21.15.67:0/1108347532 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f07740b2470 con 0x7f075406d1f0 2024-05-30T16:53:32.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.615+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f075406d1f0 msgr2=0x7f075406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:32.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.615+0000 7f0778c00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f075406d1f0 0x7f075406f6b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f076400a970 tx=0x7f0764008040 comp rx=0 tx=0).stop 2024-05-30T16:53:32.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.615+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406dfa0 msgr2=0x7f077414f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:32.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.615+0000 7f0778c00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406dfa0 0x7f077414f930 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f076002c840 tx=0x7f0760004730 comp rx=0 tx=0).stop 2024-05-30T16:53:32.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.616+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 shutdown_connections 2024-05-30T16:53:32.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.616+0000 7f0778c00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f075406d1f0 0x7f075406f6b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:32.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.616+0000 7f0778c00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0774158670 0x7f077414ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:32.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.616+0000 7f0778c00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f077406dfa0 0x7f077414f930 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:32.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.616+0000 7f0778c00640 1 --2- 172.21.15.67:0/1108347532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f077406db20 0x7f077414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:32.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.616+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 >> 172.21.15.67:0/1108347532 conn(0x7f07740b1db0 msgr2=0x7f07740ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:32.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.624+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 shutdown_connections 2024-05-30T16:53:32.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.624+0000 7f0778c00640 1 -- 172.21.15.67:0/1108347532 wait complete. 2024-05-30T16:53:32.665 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15503 2024-05-30T16:53:32.838 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.837+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.838 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.837+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.838 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.837+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.838 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.837+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.838 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:32.837+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:32.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.993+0000 7f725f000640 1 Processor -- start 2024-05-30T16:53:32.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.993+0000 7f725f000640 1 -- start start 2024-05-30T16:53:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.995+0000 7f725f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7258072aa0 0x7f7258072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.995+0000 7f725f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f725806e610 0x7f72580b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.995+0000 7f725f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72580733c0 con 0x7f725815adb0 2024-05-30T16:53:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.995+0000 7f725f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f725806ea60 con 0x7f725806e610 2024-05-30T16:53:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.995+0000 7f725f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f725806ec90 con 0x7f7258072aa0 2024-05-30T16:53:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.996+0000 7f725d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f725806e610 0x7f72580b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:32.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.996+0000 7f725d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f725806e610 0x7f72580b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33606/0 (socket says 172.21.15.67:33606) 2024-05-30T16:53:32.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.996+0000 7f725d000640 1 -- 172.21.15.67:0/616651362 learned_addr learned my addr 172.21.15.67:0/616651362 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:32.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.996+0000 7f7257e00640 1 --2- 172.21.15.67:0/616651362 >> v2:172.21.15.169:3300/0 conn(0x7f7258072aa0 0x7f7258072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:32.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.996+0000 7f725d000640 1 -- 172.21.15.67:0/616651362 >> v2:172.21.15.169:3300/0 conn(0x7f7258072aa0 msgr2=0x7f7258072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:32.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.997+0000 7f725d000640 1 --2- 172.21.15.67:0/616651362 >> v2:172.21.15.169:3300/0 conn(0x7f7258072aa0 0x7f7258072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:32.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.997+0000 7f725d000640 1 -- 172.21.15.67:0/616651362 >> v1:172.21.15.169:6789/0 conn(0x7f725815adb0 legacy=0x7f725806df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:32.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.997+0000 7f725d000640 1 -- 172.21.15.67:0/616651362 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725806ef70 con 0x7f725806e610 2024-05-30T16:53:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:32.999+0000 7f725d000640 1 --2- 172.21.15.67:0/616651362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 0x7f72580b3ad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f725806c840 tx=0x7f724802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258263d010c4f3e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:33.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.002+0000 7f7256a00640 1 -- 172.21.15.67:0/616651362 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72480325a0 con 0x7f725806e610 2024-05-30T16:53:33.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.003+0000 7f7256a00640 1 -- 172.21.15.67:0/616651362 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7248038070 con 0x7f725806e610 2024-05-30T16:53:33.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.003+0000 7f7256a00640 1 -- 172.21.15.67:0/616651362 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248032960 con 0x7f725806e610 2024-05-30T16:53:33.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.004+0000 7f725f000640 1 -- 172.21.15.67:0/616651362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 msgr2=0x7f72580b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:33.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.004+0000 7f725f000640 1 --2- 172.21.15.67:0/616651362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 0x7f72580b3ad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f725806c840 tx=0x7f724802c8e0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.006+0000 7f725f000640 1 -- 172.21.15.67:0/616651362 shutdown_connections 2024-05-30T16:53:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.006+0000 7f725f000640 1 --2- 172.21.15.67:0/616651362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 0x7f72580b3ad0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.006+0000 7f725f000640 1 --2- 172.21.15.67:0/616651362 >> v2:172.21.15.169:3300/0 conn(0x7f7258072aa0 0x7f7258072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.006+0000 7f725f000640 1 -- 172.21.15.67:0/616651362 >> 172.21.15.67:0/616651362 conn(0x7f725806bd20 msgr2=0x7f725806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.007+0000 7f725f000640 1 -- 172.21.15.67:0/616651362 shutdown_connections 2024-05-30T16:53:33.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.007+0000 7f725f000640 1 -- 172.21.15.67:0/616651362 wait complete. 2024-05-30T16:53:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.009+0000 7f725f000640 1 Processor -- start 2024-05-30T16:53:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.010+0000 7f725f000640 1 -- start start 2024-05-30T16:53:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.011+0000 7f725f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 0x7f72581fb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.011+0000 7f725f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7258072aa0 0x7f72580b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:33.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.011+0000 7f725f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725815adb0 0x7f72580b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.011+0000 7f725f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f725806f250 con 0x7f725806e610 2024-05-30T16:53:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.011+0000 7f725f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72580b35b0 con 0x7f7258072aa0 2024-05-30T16:53:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.011+0000 7f725f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72580d06d0 con 0x7f725815adb0 2024-05-30T16:53:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.012+0000 7f7257e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 0x7f72581fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.012+0000 7f7257e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 0x7f72581fb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33620/0 (socket says 172.21.15.67:33620) 2024-05-30T16:53:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.012+0000 7f7257e00640 1 -- 172.21.15.67:0/2969085309 learned_addr learned my addr 172.21.15.67:0/2969085309 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.012+0000 7f7257e00640 1 -- 172.21.15.67:0/2969085309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725815adb0 msgr2=0x7f72580b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.012+0000 7f7257e00640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725815adb0 0x7f72580b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.012+0000 7f7257e00640 1 -- 172.21.15.67:0/2969085309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7258072aa0 msgr2=0x7f72580b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.012+0000 7f7257e00640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7258072aa0 0x7f72580b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.012+0000 7f7257e00640 1 -- 172.21.15.67:0/2969085309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7258156060 con 0x7f725806e610 2024-05-30T16:53:33.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.013+0000 7f7257e00640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 0x7f72581fb910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7250007f20 tx=0x7f725000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:33.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.013+0000 7f724fe00640 1 -- 172.21.15.67:0/2969085309 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250014620 con 0x7f725806e610 2024-05-30T16:53:33.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.013+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72581fbe50 con 0x7f725806e610 2024-05-30T16:53:33.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.013+0000 7f724fe00640 1 -- 172.21.15.67:0/2969085309 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7250019070 con 0x7f725806e610 2024-05-30T16:53:33.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.014+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72581552a0 con 0x7f725806e610 2024-05-30T16:53:33.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.014+0000 7f724fe00640 1 -- 172.21.15.67:0/2969085309 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72500149c0 con 0x7f725806e610 2024-05-30T16:53:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.017+0000 7f724fe00640 1 -- 172.21.15.67:0/2969085309 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 91958+0+0 (secure 0 0 0) 0x7f7250014b60 con 0x7f725806e610 2024-05-30T16:53:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.018+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7230005290 con 0x7f725806e610 2024-05-30T16:53:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.019+0000 7f724fe00640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723806d510 0x7f723806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.019+0000 7f724fe00640 1 -- 172.21.15.67:0/2969085309 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f72500147c0 con 0x7f725806e610 2024-05-30T16:53:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.020+0000 7f7257400640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723806d510 0x7f723806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:33.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.023+0000 7f7257400640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723806d510 0x7f723806f9d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f72580b4770 tx=0x7f7244006be0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:33.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.032+0000 7f724fe00640 1 -- 172.21.15.67:0/2969085309 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7250023020 con 0x7f725806e610 2024-05-30T16:53:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.528+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15503", "target": ["mon-mgr", ""]}) -- 0x7f7230000c80 con 0x7f723806d510 2024-05-30T16:53:33.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.955+0000 7f724fe00640 1 -- 172.21.15.67:0/2969085309 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7230000c80 con 0x7f723806d510 2024-05-30T16:53:33.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.959+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723806d510 msgr2=0x7f723806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:33.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.959+0000 7f725f000640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723806d510 0x7f723806f9d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f72580b4770 tx=0x7f7244006be0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.959+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 msgr2=0x7f72581fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.959+0000 7f725f000640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 0x7f72581fb910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7250007f20 tx=0x7f725000e570 comp rx=0 tx=0).stop 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.960+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 shutdown_connections 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.960+0000 7f725f000640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f723806d510 0x7f723806f9d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.960+0000 7f725f000640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725815adb0 0x7f72580b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.960+0000 7f725f000640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7258072aa0 0x7f72580b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.960+0000 7f725f000640 1 --2- 172.21.15.67:0/2969085309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725806e610 0x7f72581fb910 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.960+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 >> 172.21.15.67:0/2969085309 conn(0x7f725806bd20 msgr2=0x7f725814c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.961+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 shutdown_connections 2024-05-30T16:53:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:33.962+0000 7f725f000640 1 -- 172.21.15.67:0/2969085309 wait complete. 2024-05-30T16:53:34.005 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:34.005+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:34.005 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:34.005+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:34.005 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:34.005+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:34.006 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:34.005+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:34.006 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:34.005+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:34.012 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15504 2024-05-30T16:53:34.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.426+0000 7f54fca00640 1 Processor -- start 2024-05-30T16:53:34.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.426+0000 7f54fca00640 1 -- start start 2024-05-30T16:53:34.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.430+0000 7f54fca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f54f8071ac0 0x7f54f8162ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:34.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.431+0000 7f54fca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f54f806ef10 0x7f54f80ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:34.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.431+0000 7f54efe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f54f80af840 0x7f54f8071ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54912/0 (socket says 172.21.15.67:54912) 2024-05-30T16:53:34.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.431+0000 7f54fca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f806de70 con 0x7f54f80af840 2024-05-30T16:53:34.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.431+0000 7f54fca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f54f806e150 con 0x7f54f806ef10 2024-05-30T16:53:34.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.431+0000 7f54fca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f54f806e430 con 0x7f54f8071ac0 2024-05-30T16:53:34.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.431+0000 7f54efe00640 1 -- 172.21.15.67:0/3877909663 learned_addr learned my addr 172.21.15.67:0/3877909663 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:34.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.431+0000 7f54f6800640 1 --2- 172.21.15.67:0/3877909663 >> v2:172.21.15.169:3300/0 conn(0x7f54f8071ac0 0x7f54f8162ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:34.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.432+0000 7f54f5e00640 1 -- 172.21.15.67:0/3877909663 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741010327 0 0) 0x7f54f806de70 con 0x7f54f80af840 2024-05-30T16:53:34.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.432+0000 7f54f5e00640 1 -- 172.21.15.67:0/3877909663 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54dc003520 con 0x7f54f80af840 2024-05-30T16:53:34.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.432+0000 7f54f6800640 1 -- 172.21.15.67:0/3877909663 >> v2:172.21.15.67:3300/0 conn(0x7f54f806ef10 msgr2=0x7f54f80ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:34.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.432+0000 7f54f6800640 1 --2- 172.21.15.67:0/3877909663 >> v2:172.21.15.67:3300/0 conn(0x7f54f806ef10 0x7f54f80ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:34.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.432+0000 7f54f6800640 1 -- 172.21.15.67:0/3877909663 >> v1:172.21.15.67:6789/0 conn(0x7f54f80af840 legacy=0x7f54f8071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:34.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.432+0000 7f54f6800640 1 -- 172.21.15.67:0/3877909663 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f806e710 con 0x7f54f8071ac0 2024-05-30T16:53:34.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.434+0000 7f54f6800640 1 --2- 172.21.15.67:0/3877909663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f8071ac0 0x7f54f8162ee0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54f8070e90 tx=0x7f54e802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab4a512e8b33d32 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:34.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.435+0000 7f54f5e00640 1 -- 172.21.15.67:0/3877909663 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8037070 con 0x7f54f8071ac0 2024-05-30T16:53:34.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.435+0000 7f54f5e00640 1 -- 172.21.15.67:0/3877909663 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e8032440 con 0x7f54f8071ac0 2024-05-30T16:53:34.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.435+0000 7f54f5e00640 1 -- 172.21.15.67:0/3877909663 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8032810 con 0x7f54f8071ac0 2024-05-30T16:53:34.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.436+0000 7f54fca00640 1 -- 172.21.15.67:0/3877909663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f8071ac0 msgr2=0x7f54f8162ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:34.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.436+0000 7f54fca00640 1 --2- 172.21.15.67:0/3877909663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f8071ac0 0x7f54f8162ee0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54f8070e90 tx=0x7f54e802a1a0 comp rx=0 tx=0).stop 2024-05-30T16:53:34.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.437+0000 7f54fca00640 1 -- 172.21.15.67:0/3877909663 shutdown_connections 2024-05-30T16:53:34.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.437+0000 7f54fca00640 1 --2- 172.21.15.67:0/3877909663 >> v2:172.21.15.67:3300/0 conn(0x7f54f806ef10 0x7f54f80ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:34.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.437+0000 7f54fca00640 1 --2- 172.21.15.67:0/3877909663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f8071ac0 0x7f54f8162ee0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:34.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.437+0000 7f54fca00640 1 -- 172.21.15.67:0/3877909663 >> 172.21.15.67:0/3877909663 conn(0x7f54f80b1db0 msgr2=0x7f54f80b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:34.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.444+0000 7f54fca00640 1 -- 172.21.15.67:0/3877909663 shutdown_connections 2024-05-30T16:53:34.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.444+0000 7f54fca00640 1 -- 172.21.15.67:0/3877909663 wait complete. 2024-05-30T16:53:34.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.448+0000 7f54fca00640 1 Processor -- start 2024-05-30T16:53:34.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.448+0000 7f54fca00640 1 -- start start 2024-05-30T16:53:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.452+0000 7f54fca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f806ef10 0x7f54f81f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.452+0000 7f54fca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f8071ac0 0x7f54f814d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:34.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.452+0000 7f54fca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f80af840 0x7f54f8159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.453+0000 7f54fca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f54f815d1e0 con 0x7f54f806ef10 2024-05-30T16:53:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.453+0000 7f54fca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f54f806db20 con 0x7f54f8071ac0 2024-05-30T16:53:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.453+0000 7f54fca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f54f815d360 con 0x7f54f80af840 2024-05-30T16:53:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.454+0000 7f54f6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f806ef10 0x7f54f81f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.454+0000 7f54f6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f806ef10 0x7f54f81f8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33662/0 (socket says 172.21.15.67:33662) 2024-05-30T16:53:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.454+0000 7f54f6800640 1 -- 172.21.15.67:0/2191008614 learned_addr learned my addr 172.21.15.67:0/2191008614 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.456+0000 7f54f6800640 1 -- 172.21.15.67:0/2191008614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f80af840 msgr2=0x7f54f8159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.456+0000 7f54f6800640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f80af840 0x7f54f8159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.456+0000 7f54f6800640 1 -- 172.21.15.67:0/2191008614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f8071ac0 msgr2=0x7f54f814d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.456+0000 7f54f6800640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f8071ac0 0x7f54f814d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.456+0000 7f54f6800640 1 -- 172.21.15.67:0/2191008614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f814dda0 con 0x7f54f806ef10 2024-05-30T16:53:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.459+0000 7f54f6800640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f806ef10 0x7f54f81f8820 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009890 tx=0x7f54e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.459+0000 7f54eea00640 1 -- 172.21.15.67:0/2191008614 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8037050 con 0x7f54f806ef10 2024-05-30T16:53:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.460+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f815e8c0 con 0x7f54f806ef10 2024-05-30T16:53:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.460+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f815ee80 con 0x7f54f806ef10 2024-05-30T16:53:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.461+0000 7f54eea00640 1 -- 172.21.15.67:0/2191008614 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e8038040 con 0x7f54f806ef10 2024-05-30T16:53:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.462+0000 7f54eea00640 1 -- 172.21.15.67:0/2191008614 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e803cb30 con 0x7f54f806ef10 2024-05-30T16:53:34.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.465+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d0005290 con 0x7f54f806ef10 2024-05-30T16:53:34.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.472+0000 7f54eea00640 1 -- 172.21.15.67:0/2191008614 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) ==== 91958+0+0 (secure 0 0 0) 0x7f54e803ccd0 con 0x7f54f806ef10 2024-05-30T16:53:34.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.475+0000 7f54eea00640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54d406d310 0x7f54d406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:34.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.475+0000 7f54eea00640 1 -- 172.21.15.67:0/2191008614 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f54e80aafc0 con 0x7f54f806ef10 2024-05-30T16:53:34.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.490+0000 7f54efe00640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54d406d310 0x7f54d406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:34.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.493+0000 7f54eea00640 1 -- 172.21.15.67:0/2191008614 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mgrmap(e 21) ==== 92111+0+0 (secure 0 0 0) 0x7f54e8076d20 con 0x7f54f806ef10 2024-05-30T16:53:34.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.499+0000 7f54efe00640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54d406d310 0x7f54d406f7d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54e0000f60 tx=0x7f54e00078f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:34.504+0000 7f54eea00640 1 -- 172.21.15.67:0/2191008614 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54e807adb0 con 0x7f54f806ef10 2024-05-30T16:53:35.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.016+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15504", "target": ["mon-mgr", ""]}) -- 0x7f54d0000c80 con 0x7f54d406d310 2024-05-30T16:53:35.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.787+0000 7f54eea00640 1 -- 172.21.15.67:0/2191008614 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f54d0000c80 con 0x7f54d406d310 2024-05-30T16:53:35.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.802+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54d406d310 msgr2=0x7f54d406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:35.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.802+0000 7f54fca00640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54d406d310 0x7f54d406f7d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54e0000f60 tx=0x7f54e00078f0 comp rx=0 tx=0).stop 2024-05-30T16:53:35.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.802+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f806ef10 msgr2=0x7f54f81f8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:35.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.802+0000 7f54fca00640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f806ef10 0x7f54f81f8820 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009890 tx=0x7f54e80046c0 comp rx=0 tx=0).stop 2024-05-30T16:53:35.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.806+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 shutdown_connections 2024-05-30T16:53:35.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.806+0000 7f54fca00640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54d406d310 0x7f54d406f7d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:35.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.806+0000 7f54fca00640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f80af840 0x7f54f8159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:35.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.806+0000 7f54fca00640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f8071ac0 0x7f54f814d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:35.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.806+0000 7f54fca00640 1 --2- 172.21.15.67:0/2191008614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54f806ef10 0x7f54f81f8820 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:35.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.806+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 >> 172.21.15.67:0/2191008614 conn(0x7f54f80b1db0 msgr2=0x7f54f80b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:35.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.807+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 shutdown_connections 2024-05-30T16:53:35.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:35.807+0000 7f54fca00640 1 -- 172.21.15.67:0/2191008614 wait complete. 2024-05-30T16:53:35.864 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15505 2024-05-30T16:53:36.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.313+0000 7f7967e00640 1 Processor -- start 2024-05-30T16:53:36.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.314+0000 7f7967e00640 1 -- start start 2024-05-30T16:53:36.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.316+0000 7f7967e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79680b3d00 0x7f79680d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:36.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.316+0000 7f7967e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79680d1380 0x7f79680d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:36.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.317+0000 7f7967e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79680cfb40 con 0x7f7968161d00 2024-05-30T16:53:36.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.317+0000 7f7967e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f79680d2040 con 0x7f79680b3d00 2024-05-30T16:53:36.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.317+0000 7f7967e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f79680b4140 con 0x7f79680d1380 2024-05-30T16:53:36.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.317+0000 7f7966a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7968161d00 0x7f79680d4d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54916/0 (socket says 172.21.15.67:54916) 2024-05-30T16:53:36.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.317+0000 7f7966a00640 1 -- 172.21.15.67:0/1080000102 learned_addr learned my addr 172.21.15.67:0/1080000102 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:36.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.319+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 327701248 0 0) 0x7f7960001490 con 0x7f7968161d00 2024-05-30T16:53:36.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.319+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f794c003520 con 0x7f7968161d00 2024-05-30T16:53:36.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.321+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1377930718 0 0) 0x7f794c003520 con 0x7f7968161d00 2024-05-30T16:53:36.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.321+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 >> v2:172.21.15.169:3300/0 conn(0x7f79680d1380 msgr2=0x7f79680d27b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:53:36.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.321+0000 7f7965600640 1 --2- 172.21.15.67:0/1080000102 >> v2:172.21.15.169:3300/0 conn(0x7f79680d1380 0x7f79680d27b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.321+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 >> v2:172.21.15.67:3300/0 conn(0x7f79680b3d00 msgr2=0x7f79680d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.321+0000 7f7966000640 1 --2- 172.21.15.67:0/1080000102 >> v2:172.21.15.67:3300/0 conn(0x7f79680b3d00 0x7f79680d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:36.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.322+0000 7f7965600640 1 --2- 172.21.15.67:0/1080000102 >> v2:172.21.15.67:3300/0 conn(0x7f79680b3d00 0x7f79680d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:36.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.324+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79680cfdc0 con 0x7f7968161d00 2024-05-30T16:53:36.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.324+0000 7f7966000640 1 --2- 172.21.15.67:0/1080000102 >> v2:172.21.15.67:3300/0 conn(0x7f79680b3d00 0x7f79680d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:36.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.325+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7960001ba0 con 0x7f7968161d00 2024-05-30T16:53:36.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.325+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 >> v1:172.21.15.67:6789/0 conn(0x7f7968161d00 legacy=0x7f79680d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.325+0000 7f7965600640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c003a80 0x7f794c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.325+0000 7f7965600640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f794c004800 0x7f794c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.326+0000 7f7965600640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794c0053c0 0x7f794c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.326+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f79680d2040 con 0x7f794c004800 2024-05-30T16:53:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.326+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79680b4140 con 0x7f794c003a80 2024-05-30T16:53:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.326+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f794c00c390 con 0x7f794c0053c0 2024-05-30T16:53:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.326+0000 7f7966000640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f794c004800 0x7f794c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:36.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.328+0000 7f7967400640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794c0053c0 0x7f794c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.328+0000 7f7966000640 1 -- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794c0053c0 msgr2=0x7f794c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.328+0000 7f7966000640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794c0053c0 0x7f794c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:36.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.332+0000 7f7966a00640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c003a80 0x7f794c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:36.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.333+0000 7f7966000640 1 -- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c003a80 msgr2=0x7f794c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.333+0000 7f7966000640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c003a80 0x7f794c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.333+0000 7f7966000640 1 -- 172.21.15.67:0/1080000102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c003520 con 0x7f794c004800 2024-05-30T16:53:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.333+0000 7f7967400640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f794c0053c0 0x7f794c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.333+0000 7f7967400640 1 -- 172.21.15.67:0/1080000102 reap_dead start 2024-05-30T16:53:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.335+0000 7f7966000640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f794c004800 0x7f794c004c80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7958014260 tx=0x7f794c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c9f80bd251967c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.335+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795803e070 con 0x7f794c004800 2024-05-30T16:53:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.335+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7958010ac0 con 0x7f794c004800 2024-05-30T16:53:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.335+0000 7f7965600640 1 -- 172.21.15.67:0/1080000102 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958012e60 con 0x7f794c004800 2024-05-30T16:53:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.337+0000 7f7967e00640 1 -- 172.21.15.67:0/1080000102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f794c004800 msgr2=0x7f794c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.337+0000 7f7967e00640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f794c004800 0x7f794c004c80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7958014260 tx=0x7f794c005830 comp rx=0 tx=0).stop 2024-05-30T16:53:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.339+0000 7f7966a00640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c003a80 0x7f794c003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.339+0000 7f7967e00640 1 -- 172.21.15.67:0/1080000102 shutdown_connections 2024-05-30T16:53:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.339+0000 7f7967e00640 1 --2- 172.21.15.67:0/1080000102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f794c004800 0x7f794c004c80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.339+0000 7f7967e00640 1 -- 172.21.15.67:0/1080000102 >> 172.21.15.67:0/1080000102 conn(0x7f796808bfc0 msgr2=0x7f79680d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.346+0000 7f7967e00640 1 -- 172.21.15.67:0/1080000102 shutdown_connections 2024-05-30T16:53:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.348+0000 7f7967e00640 1 -- 172.21.15.67:0/1080000102 wait complete. 2024-05-30T16:53:36.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.357+0000 7f7967e00640 1 Processor -- start 2024-05-30T16:53:36.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.358+0000 7f7967e00640 1 -- start start 2024-05-30T16:53:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.369+0000 7f7967e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c004800 0x7f796815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:36.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.370+0000 7f7967e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680ca920 0x7f79681582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.370+0000 7f7967e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f796815bc70 0x7f79680c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.371+0000 7f7966a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c004800 0x7f796815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.371+0000 7f7966a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c004800 0x7f796815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36808/0 (socket says 172.21.15.67:36808) 2024-05-30T16:53:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.371+0000 7f7966a00640 1 -- 172.21.15.67:0/3716693349 learned_addr learned my addr 172.21.15.67:0/3716693349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:36.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.372+0000 7f7966000640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680ca920 0x7f79681582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:36.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.373+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f79680d2290 con 0x7f796815bc70 2024-05-30T16:53:36.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.373+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79680d0380 con 0x7f794c004800 2024-05-30T16:53:36.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.373+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79680b5560 con 0x7f79680ca920 2024-05-30T16:53:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.372+0000 7f7967400640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f796815bc70 0x7f79680c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:36.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.381+0000 7f7967400640 1 -- 172.21.15.67:0/3716693349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680ca920 msgr2=0x7f79681582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.381+0000 7f7967400640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680ca920 0x7f79681582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.381+0000 7f7967400640 1 -- 172.21.15.67:0/3716693349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c004800 msgr2=0x7f796815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.381+0000 7f7967400640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c004800 0x7f796815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.381+0000 7f7967400640 1 -- 172.21.15.67:0/3716693349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79680cada0 con 0x7f796815bc70 2024-05-30T16:53:36.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.382+0000 7f7966000640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680ca920 0x7f79681582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.388+0000 7f7967400640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f796815bc70 0x7f79680c8790 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f79680b7000 tx=0x7f795000a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.389+0000 7f795ea00640 1 -- 172.21.15.67:0/3716693349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950011820 con 0x7f796815bc70 2024-05-30T16:53:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.389+0000 7f795ea00640 1 -- 172.21.15.67:0/3716693349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7950017070 con 0x7f796815bc70 2024-05-30T16:53:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.389+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796815de20 con 0x7f796815bc70 2024-05-30T16:53:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.390+0000 7f795ea00640 1 -- 172.21.15.67:0/3716693349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950011c20 con 0x7f796815bc70 2024-05-30T16:53:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.390+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796815b470 con 0x7f796815bc70 2024-05-30T16:53:36.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.393+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79680689e0 con 0x7f796815bc70 2024-05-30T16:53:36.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.394+0000 7f795ea00640 1 -- 172.21.15.67:0/3716693349 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 21) ==== 92111+0+0 (secure 0 0 0) 0x7f795001b410 con 0x7f796815bc70 2024-05-30T16:53:36.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.397+0000 7f795ea00640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f794406d4f0 0x7f794406f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:36.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.397+0000 7f795ea00640 1 -- 172.21.15.67:0/3716693349 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f7950089ab0 con 0x7f796815bc70 2024-05-30T16:53:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.403+0000 7f795ea00640 1 -- 172.21.15.67:0/3716693349 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7950059930 con 0x7f796815bc70 2024-05-30T16:53:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.428+0000 7f7966a00640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f794406d4f0 0x7f794406f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:36.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.515+0000 7f7966a00640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f794406d4f0 0x7f794406f9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7960004d90 tx=0x7f796000a040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:36.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:36.967+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15505", "target": ["mon-mgr", ""]}) -- 0x7f796815e9b0 con 0x7f794406d4f0 2024-05-30T16:53:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.977+0000 7f795ea00640 1 -- 172.21.15.67:0/3716693349 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f796815e9b0 con 0x7f794406d4f0 2024-05-30T16:53:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.981+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f794406d4f0 msgr2=0x7f794406f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.981+0000 7f7967e00640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f794406d4f0 0x7f794406f9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7960004d90 tx=0x7f796000a040 comp rx=0 tx=0).stop 2024-05-30T16:53:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.981+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f796815bc70 msgr2=0x7f79680c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.981+0000 7f7967e00640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f796815bc70 0x7f79680c8790 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f79680b7000 tx=0x7f795000a870 comp rx=0 tx=0).stop 2024-05-30T16:53:37.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.985+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 shutdown_connections 2024-05-30T16:53:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.985+0000 7f7967e00640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f794406d4f0 0x7f794406f9b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.985+0000 7f7967e00640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f796815bc70 0x7f79680c8790 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.985+0000 7f7967e00640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680ca920 0x7f79681582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.985+0000 7f7967e00640 1 --2- 172.21.15.67:0/3716693349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f794c004800 0x7f796815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.985+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 >> 172.21.15.67:0/3716693349 conn(0x7f796808bfc0 msgr2=0x7f796806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.985+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 shutdown_connections 2024-05-30T16:53:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:37.986+0000 7f7967e00640 1 -- 172.21.15.67:0/3716693349 wait complete. 2024-05-30T16:53:38.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15506 2024-05-30T16:53:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.513+0000 7f4673a00640 1 Processor -- start 2024-05-30T16:53:38.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.517+0000 7f4673a00640 1 -- start start 2024-05-30T16:53:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.525+0000 7f4673a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f466c162af0 0x7f466c0d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:38.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.526+0000 7f4673a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f466c0b7960 0x7f466c0d52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.527+0000 7f4671a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f466c0d5820 0x7f466c06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54932/0 (socket says 172.21.15.67:54932) 2024-05-30T16:53:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.527+0000 7f4671a00640 1 -- 172.21.15.67:0/83171246 learned_addr learned my addr 172.21.15.67:0/83171246 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.527+0000 7f4673a00640 1 -- 172.21.15.67:0/83171246 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f466c162ed0 con 0x7f466c0d5820 2024-05-30T16:53:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.527+0000 7f4673a00640 1 -- 172.21.15.67:0/83171246 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f466c0d5cb0 con 0x7f466c162af0 2024-05-30T16:53:38.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.528+0000 7f4673a00640 1 -- 172.21.15.67:0/83171246 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f466c1665d0 con 0x7f466c0b7960 2024-05-30T16:53:38.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.528+0000 7f4671000640 1 --2- 172.21.15.67:0/83171246 >> v2:172.21.15.67:3300/0 conn(0x7f466c162af0 0x7f466c0d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.530+0000 7f4671000640 1 -- 172.21.15.67:0/83171246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f466c0b7960 msgr2=0x7f466c0d52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.530+0000 7f4671000640 1 --2- 172.21.15.67:0/83171246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f466c0b7960 0x7f466c0d52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.530+0000 7f4671000640 1 -- 172.21.15.67:0/83171246 >> v1:172.21.15.67:6789/0 conn(0x7f466c0d5820 legacy=0x7f466c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:38.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.531+0000 7f4671000640 1 -- 172.21.15.67:0/83171246 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c0b7dc0 con 0x7f466c162af0 2024-05-30T16:53:38.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.538+0000 7f4671000640 1 --2- 172.21.15.67:0/83171246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c162af0 0x7f466c0d82a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f466400b050 tx=0x7f466402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce2d8458e3ec5ee server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.539+0000 7f466b400640 1 -- 172.21.15.67:0/83171246 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466402d070 con 0x7f466c162af0 2024-05-30T16:53:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.540+0000 7f466b400640 1 -- 172.21.15.67:0/83171246 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4664036070 con 0x7f466c162af0 2024-05-30T16:53:38.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.540+0000 7f466b400640 1 -- 172.21.15.67:0/83171246 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46640318d0 con 0x7f466c162af0 2024-05-30T16:53:38.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.543+0000 7f4673a00640 1 -- 172.21.15.67:0/83171246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c162af0 msgr2=0x7f466c0d82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:38.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.543+0000 7f4673a00640 1 --2- 172.21.15.67:0/83171246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c162af0 0x7f466c0d82a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f466400b050 tx=0x7f466402a8e0 comp rx=0 tx=0).stop 2024-05-30T16:53:38.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.547+0000 7f4673a00640 1 -- 172.21.15.67:0/83171246 shutdown_connections 2024-05-30T16:53:38.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.547+0000 7f4673a00640 1 --2- 172.21.15.67:0/83171246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f466c0b7960 0x7f466c0d52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.550+0000 7f4673a00640 1 --2- 172.21.15.67:0/83171246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c162af0 0x7f466c0d82a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.551+0000 7f4673a00640 1 -- 172.21.15.67:0/83171246 >> 172.21.15.67:0/83171246 conn(0x7f466c0ba450 msgr2=0x7f466c0ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.556+0000 7f4673a00640 1 -- 172.21.15.67:0/83171246 shutdown_connections 2024-05-30T16:53:38.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.556+0000 7f4673a00640 1 -- 172.21.15.67:0/83171246 wait complete. 2024-05-30T16:53:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.561+0000 7f4673a00640 1 Processor -- start 2024-05-30T16:53:38.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.562+0000 7f4673a00640 1 -- start start 2024-05-30T16:53:38.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.565+0000 7f4673a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c0b7960 0x7f466c1f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:38.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.565+0000 7f4673a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f466c0d5820 0x7f466c0a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:38.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.567+0000 7f4673a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c162af0 0x7f466c0a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:38.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.572+0000 7f4671000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c0b7960 0x7f466c1f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:38.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.573+0000 7f4673a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f466c166820 con 0x7f466c0b7960 2024-05-30T16:53:38.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.573+0000 7f4671000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c0b7960 0x7f466c1f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33704/0 (socket says 172.21.15.67:33704) 2024-05-30T16:53:38.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.573+0000 7f4671000640 1 -- 172.21.15.67:0/2228172871 learned_addr learned my addr 172.21.15.67:0/2228172871 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:38.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.573+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f466c166e70 con 0x7f466c162af0 2024-05-30T16:53:38.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.573+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f466c166b00 con 0x7f466c0d5820 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.575+0000 7f466be00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f466c0d5820 0x7f466c0a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.576+0000 7f4671a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c162af0 0x7f466c0a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.577+0000 7f4671a00640 1 -- 172.21.15.67:0/2228172871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f466c0d5820 msgr2=0x7f466c0a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.577+0000 7f4671a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f466c0d5820 0x7f466c0a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.577+0000 7f4671a00640 1 -- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c0b7960 msgr2=0x7f466c1f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.577+0000 7f4671a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c0b7960 0x7f466c1f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.577+0000 7f4671a00640 1 -- 172.21.15.67:0/2228172871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c0a4050 con 0x7f466c162af0 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.577+0000 7f4671000640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c0b7960 0x7f466c1f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.578+0000 7f4671a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c162af0 0x7f466c0a5c00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f465800c390 tx=0x7f465800c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.578+0000 7f4668c00640 1 -- 172.21.15.67:0/2228172871 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658017070 con 0x7f466c162af0 2024-05-30T16:53:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.578+0000 7f4668c00640 1 -- 172.21.15.67:0/2228172871 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4658005ce0 con 0x7f466c162af0 2024-05-30T16:53:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.578+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c0a4330 con 0x7f466c162af0 2024-05-30T16:53:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.579+0000 7f4668c00640 1 -- 172.21.15.67:0/2228172871 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46580072d0 con 0x7f466c162af0 2024-05-30T16:53:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.579+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c0a48f0 con 0x7f466c162af0 2024-05-30T16:53:38.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.581+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466c0689e0 con 0x7f466c162af0 2024-05-30T16:53:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.586+0000 7f4668c00640 1 -- 172.21.15.67:0/2228172871 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 21) ==== 92111+0+0 (secure 0 0 0) 0x7f465801a070 con 0x7f466c162af0 2024-05-30T16:53:38.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.589+0000 7f4668c00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f464c06d540 0x7f464c06fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.590+0000 7f4668c00640 1 -- 172.21.15.67:0/2228172871 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f46580899e0 con 0x7f466c162af0 2024-05-30T16:53:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.616+0000 7f4668c00640 1 -- 172.21.15.67:0/2228172871 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4658059020 con 0x7f466c162af0 2024-05-30T16:53:38.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.626+0000 7f4671000640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f464c06d540 0x7f464c06fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:38.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:38.642+0000 7f4671000640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f464c06d540 0x7f464c06fa00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f466400b050 tx=0x7f4664030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:39.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:39.213+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15506", "target": ["mon-mgr", ""]}) -- 0x7f466c06f250 con 0x7f464c06d540 2024-05-30T16:53:40.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.309+0000 7f4668c00640 1 -- 172.21.15.67:0/2228172871 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f466c06f250 con 0x7f464c06d540 2024-05-30T16:53:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.318+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f464c06d540 msgr2=0x7f464c06fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.318+0000 7f4673a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f464c06d540 0x7f464c06fa00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f466400b050 tx=0x7f4664030000 comp rx=0 tx=0).stop 2024-05-30T16:53:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.318+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c162af0 msgr2=0x7f466c0a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.318+0000 7f4673a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c162af0 0x7f466c0a5c00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f465800c390 tx=0x7f465800c790 comp rx=0 tx=0).stop 2024-05-30T16:53:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.320+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 shutdown_connections 2024-05-30T16:53:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.320+0000 7f4673a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f464c06d540 0x7f464c06fa00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.320+0000 7f4673a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c162af0 0x7f466c0a5c00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.320+0000 7f4673a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f466c0d5820 0x7f466c0a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.320+0000 7f4673a00640 1 --2- 172.21.15.67:0/2228172871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466c0b7960 0x7f466c1f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.320+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 >> 172.21.15.67:0/2228172871 conn(0x7f466c0ba450 msgr2=0x7f466c0721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.321+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 shutdown_connections 2024-05-30T16:53:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.321+0000 7f4673a00640 1 -- 172.21.15.67:0/2228172871 wait complete. 2024-05-30T16:53:40.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15507 2024-05-30T16:53:40.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.870+0000 7f5dcac00640 1 Processor -- start 2024-05-30T16:53:40.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.870+0000 7f5dcac00640 1 -- start start 2024-05-30T16:53:40.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.873+0000 7f5dcac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40ad130 0x7f5dc40ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:40.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.874+0000 7f5dca200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5dc406e530 0x7f5dc406e9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45786/0 (socket says 172.21.15.67:45786) 2024-05-30T16:53:40.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.874+0000 7f5dcac00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc40cffb0 con 0x7f5dc406e530 2024-05-30T16:53:40.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.874+0000 7f5dcac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc40adad0 con 0x7f5dc415e820 2024-05-30T16:53:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.874+0000 7f5dcac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc40af3f0 con 0x7f5dc40ad130 2024-05-30T16:53:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.874+0000 7f5dca200640 1 -- 172.21.15.67:0/486096964 learned_addr learned my addr 172.21.15.67:0/486096964 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.874+0000 7f5dc8e00640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40ad130 0x7f5dc40ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:40.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.875+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410868414 0 0) 0x7f5dc40cffb0 con 0x7f5dc406e530 2024-05-30T16:53:40.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.875+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dac003520 con 0x7f5dc406e530 2024-05-30T16:53:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.876+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2831635419 0 0) 0x7f5dac003520 con 0x7f5dc406e530 2024-05-30T16:53:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.876+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40ad130 msgr2=0x7f5dc40ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.876+0000 7f5dc3e00640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40ad130 0x7f5dc40ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.876+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 >> v1:172.21.15.169:6789/0 conn(0x7f5dc415e820 legacy=0x7f5dc40cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.876+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc415ec00 con 0x7f5dc406e530 2024-05-30T16:53:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.876+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5dbc001670 con 0x7f5dc406e530 2024-05-30T16:53:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.876+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 >> v1:172.21.15.67:6789/0 conn(0x7f5dc406e530 legacy=0x7f5dc406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.876+0000 7f5dc3e00640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40ad130 0x7f5dac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.876+0000 7f5dc3e00640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dac003f40 0x7f5dac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.877+0000 7f5dc3e00640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dac004d40 0x7f5dac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.877+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40af3f0 con 0x7f5dac004d40 2024-05-30T16:53:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.877+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac00baa0 con 0x7f5dac003f40 2024-05-30T16:53:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.877+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5dac00bca0 con 0x7f5dc40ad130 2024-05-30T16:53:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.878+0000 7f5dca200640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dac004d40 0x7f5dac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.878+0000 7f5dc9800640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40ad130 0x7f5dac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.879+0000 7f5dc9800640 1 -- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dac003f40 msgr2=0x7f5dac004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:53:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.879+0000 7f5dc9800640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dac003f40 0x7f5dac004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.879+0000 7f5dc9800640 1 -- 172.21.15.67:0/486096964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dac004d40 msgr2=0x7f5dac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.879+0000 7f5dc9800640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dac004d40 0x7f5dac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.879+0000 7f5dc9800640 1 -- 172.21.15.67:0/486096964 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc415ec00 con 0x7f5dc40ad130 2024-05-30T16:53:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.884+0000 7f5dc9800640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40ad130 0x7f5dac0039e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5db0003940 tx=0x7f5dac005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a00b544612fff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.886+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db003e070 con 0x7f5dc40ad130 2024-05-30T16:53:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.886+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db0039620 con 0x7f5dc40ad130 2024-05-30T16:53:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.886+0000 7f5dc3e00640 1 -- 172.21.15.67:0/486096964 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0003980 con 0x7f5dc40ad130 2024-05-30T16:53:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.887+0000 7f5dcac00640 1 -- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40ad130 msgr2=0x7f5dac0039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.887+0000 7f5dcac00640 1 --2- 172.21.15.67:0/486096964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40ad130 0x7f5dac0039e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5db0003940 tx=0x7f5dac005830 comp rx=0 tx=0).stop 2024-05-30T16:53:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.888+0000 7f5dca200640 1 -- 172.21.15.67:0/486096964 reap_dead start 2024-05-30T16:53:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.888+0000 7f5dcac00640 1 -- 172.21.15.67:0/486096964 shutdown_connections 2024-05-30T16:53:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.888+0000 7f5dcac00640 1 -- 172.21.15.67:0/486096964 >> 172.21.15.67:0/486096964 conn(0x7f5dc40b1e50 msgr2=0x7f5dc40b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:40.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.889+0000 7f5dcac00640 1 -- 172.21.15.67:0/486096964 shutdown_connections 2024-05-30T16:53:40.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.889+0000 7f5dcac00640 1 -- 172.21.15.67:0/486096964 wait complete. 2024-05-30T16:53:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.892+0000 7f5dcac00640 1 Processor -- start 2024-05-30T16:53:40.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.892+0000 7f5dcac00640 1 -- start start 2024-05-30T16:53:40.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.894+0000 7f5dcac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc4161210 0x7f5dc40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:40.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.895+0000 7f5dcac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc41615f0 0x7f5dc40b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:40.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.895+0000 7f5dcac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc4156ec0 0x7f5dc4157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:40.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.895+0000 7f5dcac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc406dfa0 con 0x7f5dc4161210 2024-05-30T16:53:40.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.896+0000 7f5dcac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40cf410 con 0x7f5dc41615f0 2024-05-30T16:53:40.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.896+0000 7f5dcac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc406e280 con 0x7f5dc4156ec0 2024-05-30T16:53:40.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.896+0000 7f5dc9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc4161210 0x7f5dc40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:40.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.896+0000 7f5dc8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc41615f0 0x7f5dc40b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:40.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.896+0000 7f5dc8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc41615f0 0x7f5dc40b4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36234/0 (socket says 172.21.15.67:36234) 2024-05-30T16:53:40.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.896+0000 7f5dc8e00640 1 -- 172.21.15.67:0/1122646382 learned_addr learned my addr 172.21.15.67:0/1122646382 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:40.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.897+0000 7f5dca200640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc4156ec0 0x7f5dc4157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:40.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.899+0000 7f5dca200640 1 -- 172.21.15.67:0/1122646382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc41615f0 msgr2=0x7f5dc40b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:40.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.899+0000 7f5dca200640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc41615f0 0x7f5dc40b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:40.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.899+0000 7f5dca200640 1 -- 172.21.15.67:0/1122646382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc4161210 msgr2=0x7f5dc40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:40.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.899+0000 7f5dca200640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc4161210 0x7f5dc40b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:40.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.899+0000 7f5dca200640 1 -- 172.21.15.67:0/1122646382 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc41f0230 con 0x7f5dc4156ec0 2024-05-30T16:53:40.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.900+0000 7f5dca200640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc4156ec0 0x7f5dc4157340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40717f0 tx=0x7f5dbc007720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:40.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.900+0000 7f5dc1600640 1 -- 172.21.15.67:0/1122646382 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00f910 con 0x7f5dc4156ec0 2024-05-30T16:53:40.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.900+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc40b48c0 con 0x7f5dc4156ec0 2024-05-30T16:53:40.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.900+0000 7f5dc1600640 1 -- 172.21.15.67:0/1122646382 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dbc018070 con 0x7f5dc4156ec0 2024-05-30T16:53:40.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.900+0000 7f5dc1600640 1 -- 172.21.15.67:0/1122646382 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00fcd0 con 0x7f5dc4156ec0 2024-05-30T16:53:40.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.900+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc41f07d0 con 0x7f5dc4156ec0 2024-05-30T16:53:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.903+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc4071db0 con 0x7f5dc4156ec0 2024-05-30T16:53:40.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.904+0000 7f5dc1600640 1 -- 172.21.15.67:0/1122646382 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 21) ==== 92111+0+0 (secure 0 0 0) 0x7f5dbc01d460 con 0x7f5dc4156ec0 2024-05-30T16:53:40.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.906+0000 7f5dc1600640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406d4f0 0x7f5da406f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:40.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.907+0000 7f5dc1600640 1 -- 172.21.15.67:0/1122646382 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f5dbc08a5e0 con 0x7f5dc4156ec0 2024-05-30T16:53:40.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.907+0000 7f5dc9800640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406d4f0 0x7f5da406f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.913+0000 7f5dc1600640 1 -- 172.21.15.67:0/1122646382 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dc4071db0 con 0x7f5dc4156ec0 2024-05-30T16:53:40.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:40.915+0000 7f5dc9800640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406d4f0 0x7f5da406f9b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5db003b000 tx=0x7f5db0009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:41.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:41.444+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15507", "target": ["mon-mgr", ""]}) -- 0x7f5dc40726e0 con 0x7f5da406d4f0 2024-05-30T16:53:41.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:53:41.697 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:53:42.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.565+0000 7f5dc1600640 1 -- 172.21.15.67:0/1122646382 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5dc40726e0 con 0x7f5da406d4f0 2024-05-30T16:53:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.578+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406d4f0 msgr2=0x7f5da406f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.578+0000 7f5dcac00640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406d4f0 0x7f5da406f9b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5db003b000 tx=0x7f5db0009030 comp rx=0 tx=0).stop 2024-05-30T16:53:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.578+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc4156ec0 msgr2=0x7f5dc4157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.578+0000 7f5dcac00640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc4156ec0 0x7f5dc4157340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40717f0 tx=0x7f5dbc007720 comp rx=0 tx=0).stop 2024-05-30T16:53:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.579+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 shutdown_connections 2024-05-30T16:53:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.579+0000 7f5dcac00640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406d4f0 0x7f5da406f9b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.579+0000 7f5dcac00640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc4156ec0 0x7f5dc4157340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.579+0000 7f5dcac00640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc41615f0 0x7f5dc40b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.579+0000 7f5dcac00640 1 --2- 172.21.15.67:0/1122646382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc4161210 0x7f5dc40b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.579+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 >> 172.21.15.67:0/1122646382 conn(0x7f5dc40b1e50 msgr2=0x7f5dc40cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.580+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 shutdown_connections 2024-05-30T16:53:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:42.580+0000 7f5dcac00640 1 -- 172.21.15.67:0/1122646382 wait complete. 2024-05-30T16:53:42.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:53:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.087+0000 7fdfc4c00640 1 Processor -- start 2024-05-30T16:53:43.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.087+0000 7fdfc4c00640 1 -- start start 2024-05-30T16:53:43.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.089+0000 7fdfc4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:43.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.090+0000 7fdfc4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdfc015ab70 0x7fdfc006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:43.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.090+0000 7fdfc4c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfc00c5810 con 0x7fdfc00c50c0 2024-05-30T16:53:43.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.090+0000 7fdfc4c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fdfc00c5af0 con 0x7fdfc015ab70 2024-05-30T16:53:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.090+0000 7fdfc4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc00c5dd0 con 0x7fdfc00b2970 2024-05-30T16:53:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.094+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.094+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:41118/0 (socket says 172.21.15.67:41118) 2024-05-30T16:53:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.094+0000 7fdfbf400640 1 -- 172.21.15.67:0/3657741697 learned_addr learned my addr 172.21.15.67:0/3657741697 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.095+0000 7fdfbf400640 1 -- 172.21.15.67:0/3657741697 >> v2:172.21.15.67:3300/0 conn(0x7fdfc015ab70 msgr2=0x7fdfc006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.095+0000 7fdfbf400640 1 --2- 172.21.15.67:0/3657741697 >> v2:172.21.15.67:3300/0 conn(0x7fdfc015ab70 0x7fdfc006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.095+0000 7fdfbf400640 1 -- 172.21.15.67:0/3657741697 >> v1:172.21.15.169:6789/0 conn(0x7fdfc00c50c0 legacy=0x7fdfc015a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:43.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.095+0000 7fdfbf400640 1 -- 172.21.15.67:0/3657741697 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc00c60b0 con 0x7fdfc00b2970 2024-05-30T16:53:43.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.098+0000 7fdfbf400640 1 --2- 172.21.15.67:0/3657741697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015ef00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009870 tx=0x7fdfb002a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799a7986b2d8fb01 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.102+0000 7fdfbea00640 1 -- 172.21.15.67:0/3657741697 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0037070 con 0x7fdfc00b2970 2024-05-30T16:53:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.102+0000 7fdfbea00640 1 -- 172.21.15.67:0/3657741697 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb0032440 con 0x7fdfc00b2970 2024-05-30T16:53:43.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.102+0000 7fdfbea00640 1 -- 172.21.15.67:0/3657741697 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0032810 con 0x7fdfc00b2970 2024-05-30T16:53:43.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.107+0000 7fdfc4c00640 1 -- 172.21.15.67:0/3657741697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 msgr2=0x7fdfc015ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:43.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.107+0000 7fdfc4c00640 1 --2- 172.21.15.67:0/3657741697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015ef00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009870 tx=0x7fdfb002a1f0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.107+0000 7fdfc4c00640 1 -- 172.21.15.67:0/3657741697 shutdown_connections 2024-05-30T16:53:43.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.107+0000 7fdfc4c00640 1 --2- 172.21.15.67:0/3657741697 >> v2:172.21.15.67:3300/0 conn(0x7fdfc015ab70 0x7fdfc006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.107+0000 7fdfc4c00640 1 --2- 172.21.15.67:0/3657741697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015ef00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.107+0000 7fdfc4c00640 1 -- 172.21.15.67:0/3657741697 >> 172.21.15.67:0/3657741697 conn(0x7fdfc00b1e50 msgr2=0x7fdfc00b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:43.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.108+0000 7fdfc4c00640 1 -- 172.21.15.67:0/3657741697 shutdown_connections 2024-05-30T16:53:43.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.108+0000 7fdfc4c00640 1 -- 172.21.15.67:0/3657741697 wait complete. 2024-05-30T16:53:43.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.112+0000 7fdfc4c00640 1 Processor -- start 2024-05-30T16:53:43.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.112+0000 7fdfc4c00640 1 -- start start 2024-05-30T16:53:43.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.115+0000 7fdfc4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:43.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.115+0000 7fdfc4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfc00c50c0 0x7fdfc00999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:43.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.116+0000 7fdfc4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfc014ee40 0x7fdfc014f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:43.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.116+0000 7fdfc4c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc006e3b0 con 0x7fdfc00c50c0 2024-05-30T16:53:43.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.117+0000 7fdfc4c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc006e230 con 0x7fdfc014ee40 2024-05-30T16:53:43.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.117+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:43.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.117+0000 7fdfbf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:41126/0 (socket says 172.21.15.67:41126) 2024-05-30T16:53:43.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.118+0000 7fdfbf400640 1 -- 172.21.15.67:0/956642360 learned_addr learned my addr 172.21.15.67:0/956642360 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:43.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.118+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc006e530 con 0x7fdfc00b2970 2024-05-30T16:53:43.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.119+0000 7fdfb6a00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfc00c50c0 0x7fdfc00999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:43.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.122+0000 7fdfbfe00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfc014ee40 0x7fdfc014f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.124+0000 7fdfbf400640 1 -- 172.21.15.67:0/956642360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfc014ee40 msgr2=0x7fdfc014f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.125+0000 7fdfbf400640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfc014ee40 0x7fdfc014f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.125+0000 7fdfbf400640 1 -- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfc00c50c0 msgr2=0x7fdfc00999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:43.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.125+0000 7fdfbf400640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfc00c50c0 0x7fdfc00999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.125+0000 7fdfbf400640 1 -- 172.21.15.67:0/956642360 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc00b5340 con 0x7fdfc00b2970 2024-05-30T16:53:43.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.128+0000 7fdfbf400640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015d860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009870 tx=0x7fdfb00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.131+0000 7fdfb7e00640 1 -- 172.21.15.67:0/956642360 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0037040 con 0x7fdfc00b2970 2024-05-30T16:53:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.131+0000 7fdfb7e00640 1 -- 172.21.15.67:0/956642360 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb0004230 con 0x7fdfc00b2970 2024-05-30T16:53:43.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.132+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc00b5620 con 0x7fdfc00b2970 2024-05-30T16:53:43.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.136+0000 7fdfb7e00640 1 -- 172.21.15.67:0/956642360 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb003c670 con 0x7fdfc00b2970 2024-05-30T16:53:43.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.131+0000 7fdfb6a00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfc00c50c0 0x7fdfc00999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:53:43.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.138+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc00b5be0 con 0x7fdfc00b2970 2024-05-30T16:53:43.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.142+0000 7fdfb7e00640 1 -- 172.21.15.67:0/956642360 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 21) ==== 92111+0+0 (secure 0 0 0) 0x7fdfb003a070 con 0x7fdfc00b2970 2024-05-30T16:53:43.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.144+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc00689e0 con 0x7fdfc00b2970 2024-05-30T16:53:43.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.144+0000 7fdfb7e00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdf9c06d590 0x7fdf9c06fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:43.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.145+0000 7fdfb7e00640 1 -- 172.21.15.67:0/956642360 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fdfb00ab040 con 0x7fdfc00b2970 2024-05-30T16:53:43.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.149+0000 7fdfbfe00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfc014ee40 0x7fdfc014f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:43.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.177+0000 7fdfb7e00640 1 -- 172.21.15.67:0/956642360 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdfb0041020 con 0x7fdfc00b2970 2024-05-30T16:53:43.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.181+0000 7fdfb6a00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdf9c06d590 0x7fdf9c06fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:43.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.183+0000 7fdfb6a00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdf9c06d590 0x7fdf9c06fa50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdfc014fd90 tx=0x7fdfa8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:43.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.911+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdfc00cd910 con 0x7fdfc00b2970 2024-05-30T16:53:43.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.929+0000 7fdfb7e00640 1 -- 172.21.15.67:0/956642360 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2401 (secure 0 0 0) 0x7fdfb007a610 con 0x7fdfc00b2970 2024-05-30T16:53:43.936 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:43.937 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 617, 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 62, 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:53:43.938 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1868, 2024-05-30T16:53:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 7689467157, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 15490154496, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 757603958784, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 383641755, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 371819463, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 93, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 103 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 36, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:53:43.940 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 4, 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5937 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:53:43.941 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 11, 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:53:36.221449+0000", 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:53:43.942 INFO:teuthology.orchestra.run.smithi067.stdout: "a6b1c249-71a9-4e88-ae98-e1a218b10b75": { 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.927% (0s)\n [========....................] ", 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31927001476287842, 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: "fa4e9196-4ba4-4af2-a603-a487b633bc42": { 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 23.047% (8s)\n [======......................] (remaining: 29s)", 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.23047000169754028, 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:53:43.943 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:53:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.939+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdf9c06d590 msgr2=0x7fdf9c06fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.939+0000 7fdfc4c00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdf9c06d590 0x7fdf9c06fa50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdfc014fd90 tx=0x7fdfa8009040 comp rx=0 tx=0).stop 2024-05-30T16:53:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.939+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 msgr2=0x7fdfc015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.939+0000 7fdfc4c00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015d860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009870 tx=0x7fdfb00046c0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.940+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 shutdown_connections 2024-05-30T16:53:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.940+0000 7fdfc4c00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdf9c06d590 0x7fdf9c06fa50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.940+0000 7fdfc4c00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfc014ee40 0x7fdfc014f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.940+0000 7fdfc4c00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfc00c50c0 0x7fdfc00999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.940+0000 7fdfc4c00640 1 --2- 172.21.15.67:0/956642360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfc00b2970 0x7fdfc015d860 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:43.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.940+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 >> 172.21.15.67:0/956642360 conn(0x7fdfc00b1e50 msgr2=0x7fdfc014b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:43.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.941+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 shutdown_connections 2024-05-30T16:53:43.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:43.941+0000 7fdfc4c00640 1 -- 172.21.15.67:0/956642360 wait complete. 2024-05-30T16:53:44.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15501 2024-05-30T16:53:44.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.515+0000 7fc21c000640 1 Processor -- start 2024-05-30T16:53:44.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.515+0000 7fc21c000640 1 -- start start 2024-05-30T16:53:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.521+0000 7fc21c000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc21415ed80 0x7fc2140cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.521+0000 7fc21c000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc21415f160 con 0x7fc2140b7e50 2024-05-30T16:53:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.521+0000 7fc21c000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc214162930 con 0x7fc2140b68f0 2024-05-30T16:53:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.521+0000 7fc21c000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc214163540 con 0x7fc21415ed80 2024-05-30T16:53:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.522+0000 7fc219600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc21415ed80 0x7fc2140cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.522+0000 7fc219600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc21415ed80 0x7fc2140cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36256/0 (socket says 172.21.15.67:36256) 2024-05-30T16:53:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.522+0000 7fc219600640 1 -- 172.21.15.67:0/2522000610 learned_addr learned my addr 172.21.15.67:0/2522000610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.523+0000 7fc219600640 1 -- 172.21.15.67:0/2522000610 >> v1:172.21.15.169:6789/0 conn(0x7fc2140b68f0 legacy=0x7fc2140677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.523+0000 7fc219600640 1 -- 172.21.15.67:0/2522000610 >> v1:172.21.15.67:6789/0 conn(0x7fc2140b7e50 legacy=0x7fc2140b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.523+0000 7fc219600640 1 -- 172.21.15.67:0/2522000610 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc214162c10 con 0x7fc21415ed80 2024-05-30T16:53:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.523+0000 7fc213e00640 1 -- 172.21.15.67:0/2522000610 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173374716 0 0) 0x7fc21415f160 con 0x7fc2140b7e50 2024-05-30T16:53:44.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.525+0000 7fc219600640 1 --2- 172.21.15.67:0/2522000610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21415ed80 0x7fc2140cff80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc208009870 tx=0x7fc20802a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eebf8e578939b2d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.531+0000 7fc213e00640 1 -- 172.21.15.67:0/2522000610 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208037070 con 0x7fc21415ed80 2024-05-30T16:53:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.532+0000 7fc213e00640 1 -- 172.21.15.67:0/2522000610 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc208032640 con 0x7fc21415ed80 2024-05-30T16:53:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.532+0000 7fc213e00640 1 -- 172.21.15.67:0/2522000610 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208032a10 con 0x7fc21415ed80 2024-05-30T16:53:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.535+0000 7fc21c000640 1 -- 172.21.15.67:0/2522000610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21415ed80 msgr2=0x7fc2140cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:44.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.535+0000 7fc21c000640 1 --2- 172.21.15.67:0/2522000610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21415ed80 0x7fc2140cff80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc208009870 tx=0x7fc20802a2f0 comp rx=0 tx=0).stop 2024-05-30T16:53:44.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.541+0000 7fc21c000640 1 -- 172.21.15.67:0/2522000610 shutdown_connections 2024-05-30T16:53:44.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.542+0000 7fc21c000640 1 --2- 172.21.15.67:0/2522000610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21415ed80 0x7fc2140cff80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:44.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.542+0000 7fc21c000640 1 -- 172.21.15.67:0/2522000610 >> 172.21.15.67:0/2522000610 conn(0x7fc2140d13c0 msgr2=0x7fc2140d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.544+0000 7fc21c000640 1 -- 172.21.15.67:0/2522000610 shutdown_connections 2024-05-30T16:53:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.548+0000 7fc21c000640 1 -- 172.21.15.67:0/2522000610 wait complete. 2024-05-30T16:53:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.551+0000 7fc21c000640 1 Processor -- start 2024-05-30T16:53:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.552+0000 7fc21c000640 1 -- start start 2024-05-30T16:53:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.557+0000 7fc21c000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2140b68f0 0x7fc2140cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.557+0000 7fc21c000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2140b7e50 0x7fc2140cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.557+0000 7fc21c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc21415ed80 0x7fc214150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.558+0000 7fc219600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2140b68f0 0x7fc2140cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:44.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.558+0000 7fc21c000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc2140cedb0 con 0x7fc2140b7e50 2024-05-30T16:53:44.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.558+0000 7fc21c000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2141631d0 con 0x7fc2140b68f0 2024-05-30T16:53:44.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.558+0000 7fc21c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc2140cef30 con 0x7fc21415ed80 2024-05-30T16:53:44.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.559+0000 7fc219600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2140b68f0 0x7fc2140cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36270/0 (socket says 172.21.15.67:36270) 2024-05-30T16:53:44.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.559+0000 7fc21a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc21415ed80 0x7fc214150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:44.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.559+0000 7fc219600640 1 -- 172.21.15.67:0/1015113485 learned_addr learned my addr 172.21.15.67:0/1015113485 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.560+0000 7fc218c00640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2140b7e50 0x7fc2140cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.560+0000 7fc21a000640 1 -- 172.21.15.67:0/1015113485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2140b68f0 msgr2=0x7fc2140cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.560+0000 7fc21a000640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2140b68f0 0x7fc2140cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.560+0000 7fc21a000640 1 -- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2140b7e50 msgr2=0x7fc2140cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.560+0000 7fc21a000640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2140b7e50 0x7fc2140cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.561+0000 7fc21a000640 1 -- 172.21.15.67:0/1015113485 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2141518c0 con 0x7fc21415ed80 2024-05-30T16:53:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.562+0000 7fc21a000640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc21415ed80 0x7fc214150fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00c690 tx=0x7fc20c00cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.564+0000 7fc211600640 1 -- 172.21.15.67:0/1015113485 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c003b70 con 0x7fc21415ed80 2024-05-30T16:53:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.564+0000 7fc211600640 1 -- 172.21.15.67:0/1015113485 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc20c003d10 con 0x7fc21415ed80 2024-05-30T16:53:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.564+0000 7fc211600640 1 -- 172.21.15.67:0/1015113485 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20c005310 con 0x7fc21415ed80 2024-05-30T16:53:44.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.568+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc214151ba0 con 0x7fc21415ed80 2024-05-30T16:53:44.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.568+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc214152160 con 0x7fc21415ed80 2024-05-30T16:53:44.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.571+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc214150470 con 0x7fc21415ed80 2024-05-30T16:53:44.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.575+0000 7fc211600640 1 -- 172.21.15.67:0/1015113485 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 21) ==== 92111+0+0 (secure 0 0 0) 0x7fc20c0054b0 con 0x7fc21415ed80 2024-05-30T16:53:44.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.577+0000 7fc211600640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc1f406d510 0x7fc1f406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:44.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.577+0000 7fc211600640 1 -- 172.21.15.67:0/1015113485 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fc20c08a5f0 con 0x7fc21415ed80 2024-05-30T16:53:44.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.577+0000 7fc219600640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc1f406d510 0x7fc1f406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:44.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.579+0000 7fc219600640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc1f406d510 0x7fc1f406f9d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc208009630 tx=0x7fc20802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:44.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:44.587+0000 7fc211600640 1 -- 172.21.15.67:0/1015113485 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc20c05a250 con 0x7fc21415ed80 2024-05-30T16:53:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:45.209+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15501", "target": ["mon-mgr", ""]}) -- 0x7fc2140d23d0 con 0x7fc1f406d510 2024-05-30T16:53:45.254 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:45.251+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:45.254 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:45.251+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:45.254 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:45.251+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:45.254 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:45.251+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:45.254 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:45.251+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:53:46.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.792+0000 7fc211600640 1 -- 172.21.15.67:0/1015113485 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc2140d23d0 con 0x7fc1f406d510 2024-05-30T16:53:46.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.803+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc1f406d510 msgr2=0x7fc1f406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.804+0000 7fc21c000640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc1f406d510 0x7fc1f406f9d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc208009630 tx=0x7fc20802e000 comp rx=0 tx=0).stop 2024-05-30T16:53:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.804+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc21415ed80 msgr2=0x7fc214150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:46.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.804+0000 7fc21c000640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc21415ed80 0x7fc214150fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00c690 tx=0x7fc20c00cb50 comp rx=0 tx=0).stop 2024-05-30T16:53:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.805+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 shutdown_connections 2024-05-30T16:53:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.805+0000 7fc21c000640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc1f406d510 0x7fc1f406f9d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.805+0000 7fc21c000640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc21415ed80 0x7fc214150fe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.805+0000 7fc21c000640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2140b7e50 0x7fc2140cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.805+0000 7fc21c000640 1 --2- 172.21.15.67:0/1015113485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2140b68f0 0x7fc2140cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:46.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.805+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 >> 172.21.15.67:0/1015113485 conn(0x7fc2140d13c0 msgr2=0x7fc2140d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:46.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.808+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 shutdown_connections 2024-05-30T16:53:46.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:46.815+0000 7fc21c000640 1 -- 172.21.15.67:0/1015113485 wait complete. 2024-05-30T16:53:46.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15502 2024-05-30T16:53:47.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.315+0000 7fbd8e000640 1 Processor -- start 2024-05-30T16:53:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.316+0000 7fbd8e000640 1 -- start start 2024-05-30T16:53:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.318+0000 7fbd8e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 0x7fbd88073a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.319+0000 7fbd8e000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd8815bf10 con 0x7fbd8815ab20 2024-05-30T16:53:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.319+0000 7fbd8e000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd880cc920 con 0x7fbd880b3fb0 2024-05-30T16:53:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.319+0000 7fbd8e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd880b17c0 con 0x7fbd880b3330 2024-05-30T16:53:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.320+0000 7fbd87400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbd8815ab20 0x7fbd880b38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45804/0 (socket says 172.21.15.67:45804) 2024-05-30T16:53:47.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.320+0000 7fbd87400640 1 -- 172.21.15.67:0/367915143 learned_addr learned my addr 172.21.15.67:0/367915143 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.321+0000 7fbd87e00640 1 --2- 172.21.15.67:0/367915143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 0x7fbd88073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.322+0000 7fbd86000640 1 -- 172.21.15.67:0/367915143 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767134327 0 0) 0x7fbd8815bf10 con 0x7fbd8815ab20 2024-05-30T16:53:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.322+0000 7fbd86000640 1 -- 172.21.15.67:0/367915143 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd6c003520 con 0x7fbd8815ab20 2024-05-30T16:53:47.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.323+0000 7fbd87e00640 1 -- 172.21.15.67:0/367915143 >> v1:172.21.15.169:6789/0 conn(0x7fbd880b3fb0 legacy=0x7fbd8815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:47.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.323+0000 7fbd87e00640 1 -- 172.21.15.67:0/367915143 >> v1:172.21.15.67:6789/0 conn(0x7fbd8815ab20 legacy=0x7fbd880b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:47.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.323+0000 7fbd86000640 1 -- 172.21.15.67:0/367915143 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 666487127 0 0) 0x7fbd6c003520 con 0x7fbd8815ab20 2024-05-30T16:53:47.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.323+0000 7fbd87e00640 1 -- 172.21.15.67:0/367915143 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd880b1aa0 con 0x7fbd880b3330 2024-05-30T16:53:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.329+0000 7fbd87e00640 1 --2- 172.21.15.67:0/367915143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 0x7fbd88073a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000aed0 tx=0x7fbd8002b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b41fa7e24438ede server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.333+0000 7fbd86000640 1 -- 172.21.15.67:0/367915143 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74004df0 con 0x7fbd880b3330 2024-05-30T16:53:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.333+0000 7fbd86000640 1 -- 172.21.15.67:0/367915143 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd8000a0d0 con 0x7fbd880b3330 2024-05-30T16:53:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.333+0000 7fbd86000640 1 -- 172.21.15.67:0/367915143 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80008660 con 0x7fbd880b3330 2024-05-30T16:53:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.335+0000 7fbd8e000640 1 -- 172.21.15.67:0/367915143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 msgr2=0x7fbd88073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.335+0000 7fbd8e000640 1 --2- 172.21.15.67:0/367915143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 0x7fbd88073a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000aed0 tx=0x7fbd8002b6d0 comp rx=0 tx=0).stop 2024-05-30T16:53:47.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.336+0000 7fbd8e000640 1 -- 172.21.15.67:0/367915143 shutdown_connections 2024-05-30T16:53:47.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.336+0000 7fbd8e000640 1 --2- 172.21.15.67:0/367915143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 0x7fbd88073a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:47.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.336+0000 7fbd8e000640 1 -- 172.21.15.67:0/367915143 >> 172.21.15.67:0/367915143 conn(0x7fbd880b1fc0 msgr2=0x7fbd880b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:47.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.340+0000 7fbd8e000640 1 -- 172.21.15.67:0/367915143 shutdown_connections 2024-05-30T16:53:47.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.341+0000 7fbd8e000640 1 -- 172.21.15.67:0/367915143 wait complete. 2024-05-30T16:53:47.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.351+0000 7fbd8e000640 1 Processor -- start 2024-05-30T16:53:47.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.353+0000 7fbd8e000640 1 -- start start 2024-05-30T16:53:47.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.357+0000 7fbd8e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 0x7fbd880ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:47.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.357+0000 7fbd8e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd880b3fb0 0x7fbd88200cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:47.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.357+0000 7fbd8e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd8815ab20 0x7fbd880aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:47.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.357+0000 7fbd8e000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbd880cc540 con 0x7fbd880b3fb0 2024-05-30T16:53:47.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.357+0000 7fbd8e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8815c1b0 con 0x7fbd8815ab20 2024-05-30T16:53:47.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.357+0000 7fbd8e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd880cc6c0 con 0x7fbd880b3330 2024-05-30T16:53:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.364+0000 7fbd87e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd8815ab20 0x7fbd880aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.364+0000 7fbd87e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd8815ab20 0x7fbd880aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36304/0 (socket says 172.21.15.67:36304) 2024-05-30T16:53:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.364+0000 7fbd87e00640 1 -- 172.21.15.67:0/1408182521 learned_addr learned my addr 172.21.15.67:0/1408182521 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.361+0000 7fbd86a00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd880b3fb0 0x7fbd88200cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.364+0000 7fbd87400640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 0x7fbd880ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.369+0000 7fbd87e00640 1 -- 172.21.15.67:0/1408182521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 msgr2=0x7fbd880ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:47.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.369+0000 7fbd87e00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 0x7fbd880ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.369+0000 7fbd87e00640 1 -- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd880b3fb0 msgr2=0x7fbd88200cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.372+0000 7fbd87e00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd880b3fb0 0x7fbd88200cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.372+0000 7fbd87e00640 1 -- 172.21.15.67:0/1408182521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd880cf070 con 0x7fbd8815ab20 2024-05-30T16:53:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.375+0000 7fbd86a00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd880b3fb0 0x7fbd88200cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:53:47.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.379+0000 7fbd87e00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd8815ab20 0x7fbd880aeaf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbd8002b2b0 tx=0x7fbd80004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.383+0000 7fbd7f400640 1 -- 172.21.15.67:0/1408182521 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80009a50 con 0x7fbd8815ab20 2024-05-30T16:53:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.383+0000 7fbd7f400640 1 -- 172.21.15.67:0/1408182521 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd80009bf0 con 0x7fbd8815ab20 2024-05-30T16:53:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.383+0000 7fbd7f400640 1 -- 172.21.15.67:0/1408182521 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80003990 con 0x7fbd8815ab20 2024-05-30T16:53:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.383+0000 7fbd8e000640 1 -- 172.21.15.67:0/1408182521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd880cd850 con 0x7fbd8815ab20 2024-05-30T16:53:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.384+0000 7fbd8e000640 1 -- 172.21.15.67:0/1408182521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd880cde10 con 0x7fbd8815ab20 2024-05-30T16:53:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.386+0000 7fbd7cc00640 1 -- 172.21.15.67:0/1408182521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c005290 con 0x7fbd8815ab20 2024-05-30T16:53:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.390+0000 7fbd7f400640 1 -- 172.21.15.67:0/1408182521 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 22) ==== 92162+0+0 (secure 0 0 0) 0x7fbd8003b070 con 0x7fbd8815ab20 2024-05-30T16:53:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.392+0000 7fbd7f400640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd6806d620 0x7fbd6806fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.394+0000 7fbd7f400640 1 -- 172.21.15.67:0/1408182521 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fbd800aaf00 con 0x7fbd8815ab20 2024-05-30T16:53:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.395+0000 7fbd7f400640 1 -- 172.21.15.67:0/1408182521 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd8007ad80 con 0x7fbd8815ab20 2024-05-30T16:53:47.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.415+0000 7fbd87400640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd6806d620 0x7fbd6806fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:47.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.417+0000 7fbd87400640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd6806d620 0x7fbd6806fae0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbd74004f60 tx=0x7fbd74004e10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:47.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:47.904+0000 7fbd7cc00640 1 -- 172.21.15.67:0/1408182521 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15502", "target": ["mon-mgr", ""]}) -- 0x7fbd5c000bc0 con 0x7fbd6806d620 2024-05-30T16:53:49.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.054+0000 7fbd7f400640 1 -- 172.21.15.67:0/1408182521 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbd5c000bc0 con 0x7fbd6806d620 2024-05-30T16:53:49.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.069+0000 7fbd7cc00640 1 -- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd6806d620 msgr2=0x7fbd6806fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:49.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.069+0000 7fbd7cc00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd6806d620 0x7fbd6806fae0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbd74004f60 tx=0x7fbd74004e10 comp rx=0 tx=0).stop 2024-05-30T16:53:49.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.069+0000 7fbd7cc00640 1 -- 172.21.15.67:0/1408182521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd8815ab20 msgr2=0x7fbd880aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:49.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.069+0000 7fbd7cc00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd8815ab20 0x7fbd880aeaf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbd8002b2b0 tx=0x7fbd80004a40 comp rx=0 tx=0).stop 2024-05-30T16:53:49.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.077+0000 7fbd7cc00640 1 -- 172.21.15.67:0/1408182521 shutdown_connections 2024-05-30T16:53:49.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.077+0000 7fbd7cc00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbd6806d620 0x7fbd6806fae0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.077+0000 7fbd7cc00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd8815ab20 0x7fbd880aeaf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.077+0000 7fbd7cc00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd880b3fb0 0x7fbd88200cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.077+0000 7fbd7cc00640 1 --2- 172.21.15.67:0/1408182521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd880b3330 0x7fbd880ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.077+0000 7fbd7cc00640 1 -- 172.21.15.67:0/1408182521 >> 172.21.15.67:0/1408182521 conn(0x7fbd880b1fc0 msgr2=0x7fbd88159bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:49.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.078+0000 7fbd7cc00640 1 -- 172.21.15.67:0/1408182521 shutdown_connections 2024-05-30T16:53:49.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.078+0000 7fbd7cc00640 1 -- 172.21.15.67:0/1408182521 wait complete. 2024-05-30T16:53:49.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15503 2024-05-30T16:53:49.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.602+0000 7f94d9600640 1 Processor -- start 2024-05-30T16:53:49.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.604+0000 7f94d9600640 1 -- start start 2024-05-30T16:53:49.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.605+0000 7f94d9600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f94d415eef0 0x7f94d40b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:49.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.605+0000 7f94d9600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d415ba20 con 0x7f94d4075220 2024-05-30T16:53:49.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.605+0000 7f94d9600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d415e410 con 0x7f94d415b3c0 2024-05-30T16:53:49.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.605+0000 7f94d9600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f94d415e1c0 con 0x7f94d415eef0 2024-05-30T16:53:49.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.607+0000 7f94d3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f94d4075220 0x7f94d415acb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45810/0 (socket says 172.21.15.67:45810) 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.607+0000 7f94d3400640 1 -- 172.21.15.67:0/2971249486 learned_addr learned my addr 172.21.15.67:0/2971249486 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.608+0000 7f94d3e00640 1 --2- 172.21.15.67:0/2971249486 >> v2:172.21.15.169:3300/0 conn(0x7f94d415eef0 0x7f94d40b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.608+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4166993235 0 0) 0x7f94d415ba20 con 0x7f94d4075220 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.608+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94b8005730 con 0x7f94d4075220 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 576634278 0 0) 0x7f94b8005730 con 0x7f94d4075220 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 >> v2:172.21.15.169:3300/0 conn(0x7f94d415eef0 msgr2=0x7f94d40b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 --2- 172.21.15.67:0/2971249486 >> v2:172.21.15.169:3300/0 conn(0x7f94d415eef0 0x7f94d40b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 >> v1:172.21.15.169:6789/0 conn(0x7f94d415b3c0 legacy=0x7f94d406dc80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d415e6f0 con 0x7f94d4075220 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f94c0001670 con 0x7f94d4075220 2024-05-30T16:53:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 >> v1:172.21.15.67:6789/0 conn(0x7f94d4075220 legacy=0x7f94d415acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 --2- 172.21.15.67:0/2971249486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94b8005bc0 0x7f94b8005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 --2- 172.21.15.67:0/2971249486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8006a10 0x7f94b8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 --2- 172.21.15.67:0/2971249486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94b80073d0 0x7f94b800ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94d415e1c0 con 0x7f94b8006a10 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94b80105f0 con 0x7f94b8005bc0 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.610+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f94b80107f0 con 0x7f94b80073d0 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.611+0000 7f94d3400640 1 --2- 172.21.15.67:0/2971249486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8006a10 0x7f94b8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.611+0000 7f94d3400640 1 -- 172.21.15.67:0/2971249486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94b80073d0 msgr2=0x7f94b800ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.611+0000 7f94d3400640 1 --2- 172.21.15.67:0/2971249486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94b80073d0 0x7f94b800ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.611+0000 7f94d3400640 1 -- 172.21.15.67:0/2971249486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94b8005bc0 msgr2=0x7f94b8005fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:53:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.611+0000 7f94d3400640 1 --2- 172.21.15.67:0/2971249486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94b8005bc0 0x7f94b8005fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.611+0000 7f94d3400640 1 -- 172.21.15.67:0/2971249486 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d415e6f0 con 0x7f94b8006a10 2024-05-30T16:53:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.612+0000 7f94d8c00640 1 -- 172.21.15.67:0/2971249486 reap_dead start 2024-05-30T16:53:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.615+0000 7f94d3400640 1 --2- 172.21.15.67:0/2971249486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8006a10 0x7f94b8006e90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f94c0007d30 tx=0x7f94b8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35d201c805c615 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.615+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0007d70 con 0x7f94b8006a10 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.615+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c0037070 con 0x7f94b8006a10 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.615+0000 7f94d2a00640 1 -- 172.21.15.67:0/2971249486 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0033e40 con 0x7f94b8006a10 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.616+0000 7f94d9600640 1 -- 172.21.15.67:0/2971249486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8006a10 msgr2=0x7f94b8006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.616+0000 7f94d9600640 1 --2- 172.21.15.67:0/2971249486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8006a10 0x7f94b8006e90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f94c0007d30 tx=0x7f94b8007a40 comp rx=0 tx=0).stop 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.618+0000 7f94d9600640 1 -- 172.21.15.67:0/2971249486 shutdown_connections 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.618+0000 7f94d9600640 1 --2- 172.21.15.67:0/2971249486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94b8006a10 0x7f94b8006e90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.618+0000 7f94d9600640 1 -- 172.21.15.67:0/2971249486 >> 172.21.15.67:0/2971249486 conn(0x7f94d4067ce0 msgr2=0x7f94d406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.618+0000 7f94d9600640 1 -- 172.21.15.67:0/2971249486 shutdown_connections 2024-05-30T16:53:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.618+0000 7f94d9600640 1 -- 172.21.15.67:0/2971249486 wait complete. 2024-05-30T16:53:49.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.623+0000 7f94d9600640 1 Processor -- start 2024-05-30T16:53:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.628+0000 7f94d9600640 1 -- start start 2024-05-30T16:53:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.630+0000 7f94d9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94b8006a10 0x7f94d41410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.630+0000 7f94d9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d41439b0 0x7f94d4143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.630+0000 7f94d9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d40a9490 0x7f94d40a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.630+0000 7f94d9600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94d415ae30 con 0x7f94d41439b0 2024-05-30T16:53:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.630+0000 7f94d9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94d415acb0 con 0x7f94b8006a10 2024-05-30T16:53:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.630+0000 7f94d9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f94d415afb0 con 0x7f94d40a9490 2024-05-30T16:53:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.631+0000 7f94d3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d41439b0 0x7f94d4143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.631+0000 7f94d3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d41439b0 0x7f94d4143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33486/0 (socket says 172.21.15.67:33486) 2024-05-30T16:53:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.631+0000 7f94d3400640 1 -- 172.21.15.67:0/2509196310 learned_addr learned my addr 172.21.15.67:0/2509196310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:49.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.631+0000 7f94d8c00640 1 --2- 172.21.15.67:0/2509196310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d40a9490 0x7f94d40a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:49.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.633+0000 7f94d3400640 1 -- 172.21.15.67:0/2509196310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d40a9490 msgr2=0x7f94d40a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:49.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.633+0000 7f94d3400640 1 --2- 172.21.15.67:0/2509196310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d40a9490 0x7f94d40a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.633+0000 7f94d3400640 1 -- 172.21.15.67:0/2509196310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94b8006a10 msgr2=0x7f94d41410b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T16:53:49.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.633+0000 7f94d3400640 1 --2- 172.21.15.67:0/2509196310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94b8006a10 0x7f94d41410b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:49.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.633+0000 7f94d3400640 1 -- 172.21.15.67:0/2509196310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d41444a0 con 0x7f94d41439b0 2024-05-30T16:53:49.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.634+0000 7f94d8c00640 1 --2- 172.21.15.67:0/2509196310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94d40a9490 0x7f94d40a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:53:49.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.634+0000 7f94d3400640 1 --2- 172.21.15.67:0/2509196310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d41439b0 0x7f94d4143e30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f94c000c320 tx=0x7f94c0001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.635+0000 7f94cbe00640 1 -- 172.21.15.67:0/2509196310 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c004c070 con 0x7f94d41439b0 2024-05-30T16:53:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.635+0000 7f94cbe00640 1 -- 172.21.15.67:0/2509196310 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c0004bf0 con 0x7f94d41439b0 2024-05-30T16:53:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.635+0000 7f94cbe00640 1 -- 172.21.15.67:0/2509196310 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0007070 con 0x7f94d41439b0 2024-05-30T16:53:49.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.635+0000 7f94d9600640 1 -- 172.21.15.67:0/2509196310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d4144780 con 0x7f94d41439b0 2024-05-30T16:53:49.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.635+0000 7f94d9600640 1 -- 172.21.15.67:0/2509196310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41f38c0 con 0x7f94d41439b0 2024-05-30T16:53:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.642+0000 7f94cbe00640 1 -- 172.21.15.67:0/2509196310 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 92162+0+0 (secure 0 0 0) 0x7f94c0007210 con 0x7f94d41439b0 2024-05-30T16:53:49.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.645+0000 7f94cbe00640 1 --2- 172.21.15.67:0/2509196310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f94b406d650 0x7f94b406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:49.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.646+0000 7f94cbe00640 1 -- 172.21.15.67:0/2509196310 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f94c00b3250 con 0x7f94d41439b0 2024-05-30T16:53:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.666+0000 7f94d9600640 1 -- 172.21.15.67:0/2509196310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d4144950 con 0x7f94d41439b0 2024-05-30T16:53:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.681+0000 7f94cbe00640 1 -- 172.21.15.67:0/2509196310 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f94c0082800 con 0x7f94d41439b0 2024-05-30T16:53:49.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.706+0000 7f94d3e00640 1 --2- 172.21.15.67:0/2509196310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f94b406d650 0x7f94b406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:49.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:49.712+0000 7f94d3e00640 1 --2- 172.21.15.67:0/2509196310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f94b406d650 0x7f94b406fb10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f94d4051690 tx=0x7f94cc007130 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:50.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:50.208+0000 7f94d9600640 1 -- 172.21.15.67:0/2509196310 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15503", "target": ["mon-mgr", ""]}) -- 0x7f94d40689e0 con 0x7f94b406d650 2024-05-30T16:53:51.466 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:53:51.466+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:53:51.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.471+0000 7f94cbe00640 1 -- 172.21.15.67:0/2509196310 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f94d40689e0 con 0x7f94b406d650 2024-05-30T16:53:51.472 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:53:51.497 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:53:51.498 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15504 2024-05-30T16:53:51.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.878+0000 7f8d3d800640 1 Processor -- start 2024-05-30T16:53:51.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.879+0000 7f8d3d800640 1 -- start start 2024-05-30T16:53:51.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.887+0000 7f8d3d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3815e850 0x7f8d380b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:51.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.888+0000 7f8d3d800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d380cef30 0x7f8d380d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:51.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.890+0000 7f8d3d800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d380b8000 con 0x7f8d380d1b80 2024-05-30T16:53:51.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.891+0000 7f8d3d800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8d3815ec30 con 0x7f8d380cef30 2024-05-30T16:53:51.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.891+0000 7f8d3d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8d380d1fd0 con 0x7f8d3815e850 2024-05-30T16:53:51.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.889+0000 7f8d36000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d380cef30 0x7f8d380d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:51.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.891+0000 7f8d37400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8d380d1b80 0x7f8d38162340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:60480/0 (socket says 172.21.15.67:60480) 2024-05-30T16:53:51.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.891+0000 7f8d37400640 1 -- 172.21.15.67:0/3065087912 learned_addr learned my addr 172.21.15.67:0/3065087912 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:51.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.889+0000 7f8d36a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3815e850 0x7f8d380b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:51.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.893+0000 7f8d35600640 1 -- 172.21.15.67:0/3065087912 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1722447809 0 0) 0x7f8d380b8000 con 0x7f8d380d1b80 2024-05-30T16:53:51.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.893+0000 7f8d35600640 1 -- 172.21.15.67:0/3065087912 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d1c003520 con 0x7f8d380d1b80 2024-05-30T16:53:51.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.893+0000 7f8d36a00640 1 -- 172.21.15.67:0/3065087912 >> v2:172.21.15.169:3300/0 conn(0x7f8d380cef30 msgr2=0x7f8d380d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:51.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.893+0000 7f8d36a00640 1 --2- 172.21.15.67:0/3065087912 >> v2:172.21.15.169:3300/0 conn(0x7f8d380cef30 0x7f8d380d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.893+0000 7f8d36a00640 1 -- 172.21.15.67:0/3065087912 >> v1:172.21.15.169:6789/0 conn(0x7f8d380d1b80 legacy=0x7f8d38162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.894+0000 7f8d36a00640 1 -- 172.21.15.67:0/3065087912 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d380b82e0 con 0x7f8d3815e850 2024-05-30T16:53:51.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.896+0000 7f8d36a00640 1 --2- 172.21.15.67:0/3065087912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3815e850 0x7f8d380b77b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d280097a0 tx=0x7f8d2802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac9ab5030f6765f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:51.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.897+0000 7f8d35600640 1 -- 172.21.15.67:0/3065087912 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28037070 con 0x7f8d3815e850 2024-05-30T16:53:51.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.897+0000 7f8d35600640 1 -- 172.21.15.67:0/3065087912 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d2802ae40 con 0x7f8d3815e850 2024-05-30T16:53:51.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.897+0000 7f8d35600640 1 -- 172.21.15.67:0/3065087912 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28032660 con 0x7f8d3815e850 2024-05-30T16:53:51.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.898+0000 7f8d3d800640 1 -- 172.21.15.67:0/3065087912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3815e850 msgr2=0x7f8d380b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.898+0000 7f8d3d800640 1 --2- 172.21.15.67:0/3065087912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3815e850 0x7f8d380b77b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d280097a0 tx=0x7f8d2802a0f0 comp rx=0 tx=0).stop 2024-05-30T16:53:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.899+0000 7f8d3d800640 1 -- 172.21.15.67:0/3065087912 shutdown_connections 2024-05-30T16:53:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.899+0000 7f8d3d800640 1 --2- 172.21.15.67:0/3065087912 >> v2:172.21.15.169:3300/0 conn(0x7f8d380cef30 0x7f8d380d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.899+0000 7f8d3d800640 1 --2- 172.21.15.67:0/3065087912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3815e850 0x7f8d380b77b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.899+0000 7f8d3d800640 1 -- 172.21.15.67:0/3065087912 >> 172.21.15.67:0/3065087912 conn(0x7f8d38067aa0 msgr2=0x7f8d38067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.899+0000 7f8d3d800640 1 -- 172.21.15.67:0/3065087912 shutdown_connections 2024-05-30T16:53:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.899+0000 7f8d3d800640 1 -- 172.21.15.67:0/3065087912 wait complete. 2024-05-30T16:53:51.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.902+0000 7f8d3d800640 1 Processor -- start 2024-05-30T16:53:51.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.903+0000 7f8d3d800640 1 -- start start 2024-05-30T16:53:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.906+0000 7f8d3d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d380cef30 0x7f8d380ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:51.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.906+0000 7f8d3d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d380d1b80 0x7f8d38150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.907+0000 7f8d3d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d3815e850 0x7f8d380d4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:51.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.908+0000 7f8d3d800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8d380b7cf0 con 0x7f8d3815e850 2024-05-30T16:53:51.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.908+0000 7f8d3d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8d380b88a0 con 0x7f8d380cef30 2024-05-30T16:53:51.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.908+0000 7f8d3d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8d380ba0b0 con 0x7f8d380d1b80 2024-05-30T16:53:51.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.909+0000 7f8d36a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d380cef30 0x7f8d380ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:51.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.909+0000 7f8d36a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d380cef30 0x7f8d380ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:46440/0 (socket says 172.21.15.67:46440) 2024-05-30T16:53:51.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.909+0000 7f8d36a00640 1 -- 172.21.15.67:0/3751242543 learned_addr learned my addr 172.21.15.67:0/3751242543 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:51.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.909+0000 7f8d36000640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d380d1b80 0x7f8d38150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.910+0000 7f8d37400640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d3815e850 0x7f8d380d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:51.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.910+0000 7f8d36000640 1 -- 172.21.15.67:0/3751242543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d380cef30 msgr2=0x7f8d380ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.910+0000 7f8d36000640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d380cef30 0x7f8d380ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.910+0000 7f8d36000640 1 -- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d3815e850 msgr2=0x7f8d380d4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.910+0000 7f8d36000640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d3815e850 0x7f8d380d4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.910+0000 7f8d36000640 1 -- 172.21.15.67:0/3751242543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d380cd9c0 con 0x7f8d380d1b80 2024-05-30T16:53:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.911+0000 7f8d37400640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d3815e850 0x7f8d380d4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:53:51.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.911+0000 7f8d36000640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d380d1b80 0x7f8d38150070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400e870 tx=0x7f8d2400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.912+0000 7f8d2ea00640 1 -- 172.21.15.67:0/3751242543 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24017070 con 0x7f8d380d1b80 2024-05-30T16:53:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.912+0000 7f8d2ea00640 1 -- 172.21.15.67:0/3751242543 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d2400ee60 con 0x7f8d380d1b80 2024-05-30T16:53:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.912+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d380cdca0 con 0x7f8d380d1b80 2024-05-30T16:53:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.913+0000 7f8d2ea00640 1 -- 172.21.15.67:0/3751242543 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24021060 con 0x7f8d380d1b80 2024-05-30T16:53:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.913+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d380ce260 con 0x7f8d380d1b80 2024-05-30T16:53:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.915+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d380ced20 con 0x7f8d380d1b80 2024-05-30T16:53:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.918+0000 7f8d2ea00640 1 -- 172.21.15.67:0/3751242543 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 22) ==== 92162+0+0 (secure 0 0 0) 0x7f8d2401a070 con 0x7f8d380d1b80 2024-05-30T16:53:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.919+0000 7f8d2ea00640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d1806d550 0x7f8d1806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.920+0000 7f8d2ea00640 1 -- 172.21.15.67:0/3751242543 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f8d2408a900 con 0x7f8d380d1b80 2024-05-30T16:53:51.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.924+0000 7f8d36a00640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d1806d550 0x7f8d1806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.928+0000 7f8d36a00640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d1806d550 0x7f8d1806fa10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009560 tx=0x7f8d2802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:51.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:51.930+0000 7f8d2ea00640 1 -- 172.21.15.67:0/3751242543 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d24059e90 con 0x7f8d380d1b80 2024-05-30T16:53:52.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:52.491+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15504", "target": ["mon-mgr", ""]}) -- 0x7f8d380ce430 con 0x7f8d1806d550 2024-05-30T16:53:53.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.703+0000 7f8d2ea00640 1 -- 172.21.15.67:0/3751242543 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d380ce430 con 0x7f8d1806d550 2024-05-30T16:53:53.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d1806d550 msgr2=0x7f8d1806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:53.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d1806d550 0x7f8d1806fa10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009560 tx=0x7f8d2802e000 comp rx=0 tx=0).stop 2024-05-30T16:53:53.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d380d1b80 msgr2=0x7f8d38150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:53.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d380d1b80 0x7f8d38150070 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400e870 tx=0x7f8d2400ed30 comp rx=0 tx=0).stop 2024-05-30T16:53:53.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 shutdown_connections 2024-05-30T16:53:53.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d1806d550 0x7f8d1806fa10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:53.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d3815e850 0x7f8d380d4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:53.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d380d1b80 0x7f8d38150070 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:53.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 --2- 172.21.15.67:0/3751242543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d380cef30 0x7f8d380ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:53.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 >> 172.21.15.67:0/3751242543 conn(0x7f8d38067aa0 msgr2=0x7f8d3815ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:53.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 shutdown_connections 2024-05-30T16:53:53.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:53.714+0000 7f8d3d800640 1 -- 172.21.15.67:0/3751242543 wait complete. 2024-05-30T16:53:53.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15505 2024-05-30T16:53:54.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.243+0000 7f44db600640 1 Processor -- start 2024-05-30T16:53:54.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.243+0000 7f44db600640 1 -- start start 2024-05-30T16:53:54.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.247+0000 7f44db600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d415ea20 0x7f44d40d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:54.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.247+0000 7f44db600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44d40cfc50 0x7f44d40b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:54.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.247+0000 7f44db600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d40d3a40 con 0x7f44d40d32d0 2024-05-30T16:53:54.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.247+0000 7f44db600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f44d40b74d0 con 0x7f44d40cfc50 2024-05-30T16:53:54.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.247+0000 7f44db600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44d40b77b0 con 0x7f44d415ea20 2024-05-30T16:53:54.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.248+0000 7f44d3e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f44d40d32d0 0x7f44d40cf540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:60496/0 (socket says 172.21.15.67:60496) 2024-05-30T16:53:54.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.248+0000 7f44d3e00640 1 -- 172.21.15.67:0/1619901070 learned_addr learned my addr 172.21.15.67:0/1619901070 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.248+0000 7f44d8c00640 1 --2- 172.21.15.67:0/1619901070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d415ea20 0x7f44d40d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.249+0000 7f44d9600640 1 --2- 172.21.15.67:0/1619901070 >> v2:172.21.15.67:3300/0 conn(0x7f44d40cfc50 0x7f44d40b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.250+0000 7f44d8c00640 1 -- 172.21.15.67:0/1619901070 >> v2:172.21.15.67:3300/0 conn(0x7f44d40cfc50 msgr2=0x7f44d40b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.250+0000 7f44d8c00640 1 --2- 172.21.15.67:0/1619901070 >> v2:172.21.15.67:3300/0 conn(0x7f44d40cfc50 0x7f44d40b7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.250+0000 7f44d8c00640 1 -- 172.21.15.67:0/1619901070 >> v1:172.21.15.169:6789/0 conn(0x7f44d40d32d0 legacy=0x7f44d40cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.250+0000 7f44d8c00640 1 -- 172.21.15.67:0/1619901070 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d40b7a90 con 0x7f44d415ea20 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.251+0000 7f44d9600640 1 --2- 172.21.15.67:0/1619901070 >> v2:172.21.15.67:3300/0 conn(0x7f44d40cfc50 0x7f44d40b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.254+0000 7f44d8c00640 1 --2- 172.21.15.67:0/1619901070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d415ea20 0x7f44d40d4f50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0097a0 tx=0x7f44bc02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a137dc7e3020cfd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.254+0000 7f44d3400640 1 -- 172.21.15.67:0/1619901070 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc037070 con 0x7f44d415ea20 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.254+0000 7f44d3400640 1 -- 172.21.15.67:0/1619901070 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc02ad10 con 0x7f44d415ea20 2024-05-30T16:53:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.254+0000 7f44d3400640 1 -- 172.21.15.67:0/1619901070 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc032660 con 0x7f44d415ea20 2024-05-30T16:53:54.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.255+0000 7f44db600640 1 -- 172.21.15.67:0/1619901070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d415ea20 msgr2=0x7f44d40d4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:54.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.255+0000 7f44db600640 1 --2- 172.21.15.67:0/1619901070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d415ea20 0x7f44d40d4f50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0097a0 tx=0x7f44bc02a080 comp rx=0 tx=0).stop 2024-05-30T16:53:54.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.256+0000 7f44db600640 1 -- 172.21.15.67:0/1619901070 shutdown_connections 2024-05-30T16:53:54.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.256+0000 7f44db600640 1 --2- 172.21.15.67:0/1619901070 >> v2:172.21.15.67:3300/0 conn(0x7f44d40cfc50 0x7f44d40b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:54.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.256+0000 7f44db600640 1 --2- 172.21.15.67:0/1619901070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d415ea20 0x7f44d40d4f50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:54.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.256+0000 7f44db600640 1 -- 172.21.15.67:0/1619901070 >> 172.21.15.67:0/1619901070 conn(0x7f44d408bfc0 msgr2=0x7f44d40d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:54.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.256+0000 7f44db600640 1 -- 172.21.15.67:0/1619901070 shutdown_connections 2024-05-30T16:53:54.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.256+0000 7f44db600640 1 -- 172.21.15.67:0/1619901070 wait complete. 2024-05-30T16:53:54.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.259+0000 7f44db600640 1 Processor -- start 2024-05-30T16:53:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.259+0000 7f44db600640 1 -- start start 2024-05-30T16:53:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.260+0000 7f44db600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40cfc50 0x7f44d41fc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.260+0000 7f44db600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44d40d32d0 0x7f44d40caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.261+0000 7f44db600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d415ea20 0x7f44d415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.261+0000 7f44db600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f44d40b8670 con 0x7f44d40d32d0 2024-05-30T16:53:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.261+0000 7f44db600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44d40b84f0 con 0x7f44d415ea20 2024-05-30T16:53:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.261+0000 7f44db600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44d40b87f0 con 0x7f44d40cfc50 2024-05-30T16:53:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.262+0000 7f44d8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40cfc50 0x7f44d41fc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.262+0000 7f44d8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40cfc50 0x7f44d41fc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:35472/0 (socket says 172.21.15.67:35472) 2024-05-30T16:53:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.262+0000 7f44d8c00640 1 -- 172.21.15.67:0/603616359 learned_addr learned my addr 172.21.15.67:0/603616359 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.262+0000 7f44d9600640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d415ea20 0x7f44d415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.263+0000 7f44d8c00640 1 -- 172.21.15.67:0/603616359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d415ea20 msgr2=0x7f44d415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.263+0000 7f44d8c00640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d415ea20 0x7f44d415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.263+0000 7f44d8c00640 1 -- 172.21.15.67:0/603616359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44d40d32d0 msgr2=0x7f44d40caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.263+0000 7f44d8c00640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44d40d32d0 0x7f44d40caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.263+0000 7f44d8c00640 1 -- 172.21.15.67:0/603616359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d40c7fe0 con 0x7f44d40cfc50 2024-05-30T16:53:54.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.264+0000 7f44d8c00640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40cfc50 0x7f44d41fc940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0098c0 tx=0x7f44bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:54.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.265+0000 7f44d0c00640 1 -- 172.21.15.67:0/603616359 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc037040 con 0x7f44d40cfc50 2024-05-30T16:53:54.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.265+0000 7f44d0c00640 1 -- 172.21.15.67:0/603616359 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc05d650 con 0x7f44d40cfc50 2024-05-30T16:53:54.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.265+0000 7f44d0c00640 1 -- 172.21.15.67:0/603616359 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc05da20 con 0x7f44d40cfc50 2024-05-30T16:53:54.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.265+0000 7f44db600640 1 -- 172.21.15.67:0/603616359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d40c82c0 con 0x7f44d40cfc50 2024-05-30T16:53:54.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.265+0000 7f44db600640 1 -- 172.21.15.67:0/603616359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d40c8880 con 0x7f44d40cfc50 2024-05-30T16:53:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.270+0000 7f44ca000640 1 -- 172.21.15.67:0/603616359 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a4005290 con 0x7f44d40cfc50 2024-05-30T16:53:54.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.271+0000 7f44d0c00640 1 -- 172.21.15.67:0/603616359 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 22) ==== 92162+0+0 (secure 0 0 0) 0x7f44bc066020 con 0x7f44d40cfc50 2024-05-30T16:53:54.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.272+0000 7f44d0c00640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f44b006d5a0 0x7f44b006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:54.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.272+0000 7f44d0c00640 1 -- 172.21.15.67:0/603616359 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f44bc0cc730 con 0x7f44d40cfc50 2024-05-30T16:53:54.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.273+0000 7f44d3e00640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f44b006d5a0 0x7f44b006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:54.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.275+0000 7f44d3e00640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f44b006d5a0 0x7f44b006fa60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f44d41615a0 tx=0x7f44cc007d20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.279+0000 7f44d0c00640 1 -- 172.21.15.67:0/603616359 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44bc06d020 con 0x7f44d40cfc50 2024-05-30T16:53:54.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:54.813+0000 7f44ca000640 1 -- 172.21.15.67:0/603616359 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15505", "target": ["mon-mgr", ""]}) -- 0x7f44a4000bc0 con 0x7f44b006d5a0 2024-05-30T16:53:55.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.438+0000 7f44d0c00640 1 -- 172.21.15.67:0/603616359 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f44a4000bc0 con 0x7f44b006d5a0 2024-05-30T16:53:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.443+0000 7f44ca000640 1 -- 172.21.15.67:0/603616359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f44b006d5a0 msgr2=0x7f44b006fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.443+0000 7f44ca000640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f44b006d5a0 0x7f44b006fa60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f44d41615a0 tx=0x7f44cc007d20 comp rx=0 tx=0).stop 2024-05-30T16:53:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.443+0000 7f44ca000640 1 -- 172.21.15.67:0/603616359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40cfc50 msgr2=0x7f44d41fc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.443+0000 7f44ca000640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40cfc50 0x7f44d41fc940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0098c0 tx=0x7f44bc0046c0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.444+0000 7f44ca000640 1 -- 172.21.15.67:0/603616359 shutdown_connections 2024-05-30T16:53:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.444+0000 7f44ca000640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f44b006d5a0 0x7f44b006fa60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.444+0000 7f44ca000640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d415ea20 0x7f44d415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.444+0000 7f44ca000640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44d40d32d0 0x7f44d40caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.444+0000 7f44ca000640 1 --2- 172.21.15.67:0/603616359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40cfc50 0x7f44d41fc940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.444+0000 7f44ca000640 1 -- 172.21.15.67:0/603616359 >> 172.21.15.67:0/603616359 conn(0x7f44d408bfc0 msgr2=0x7f44d4074b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:55.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.444+0000 7f44ca000640 1 -- 172.21.15.67:0/603616359 shutdown_connections 2024-05-30T16:53:55.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.444+0000 7f44ca000640 1 -- 172.21.15.67:0/603616359 wait complete. 2024-05-30T16:53:55.494 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15506 2024-05-30T16:53:55.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.939+0000 7f22b3a00640 1 Processor -- start 2024-05-30T16:53:55.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.939+0000 7f22b3a00640 1 -- start start 2024-05-30T16:53:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.941+0000 7f22b3a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22ac15b6f0 0x7f22ac06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.941+0000 7f22b3a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.941+0000 7f22b3a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22ac0ade90 con 0x7f22ac0d0640 2024-05-30T16:53:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.941+0000 7f22b3a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f22ac071c40 con 0x7f22ac15b6f0 2024-05-30T16:53:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.941+0000 7f22b3a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22ac071e70 con 0x7f22ac0725e0 2024-05-30T16:53:55.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.942+0000 7f22b1000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f22ac0d0640 0x7f22ac0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:60502/0 (socket says 172.21.15.67:60502) 2024-05-30T16:53:55.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.942+0000 7f22b1000640 1 -- 172.21.15.67:0/1861781754 learned_addr learned my addr 172.21.15.67:0/1861781754 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:53:55.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.943+0000 7f22b1a00640 1 --2- 172.21.15.67:0/1861781754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:55.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.943+0000 7f22ab400640 1 -- 172.21.15.67:0/1861781754 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488012253 0 0) 0x7f22ac0ade90 con 0x7f22ac0d0640 2024-05-30T16:53:55.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.943+0000 7f22ab400640 1 -- 172.21.15.67:0/1861781754 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2290003520 con 0x7f22ac0d0640 2024-05-30T16:53:55.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.943+0000 7f22b1a00640 1 -- 172.21.15.67:0/1861781754 >> v2:172.21.15.67:3300/0 conn(0x7f22ac15b6f0 msgr2=0x7f22ac06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:55.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.943+0000 7f22b1a00640 1 --2- 172.21.15.67:0/1861781754 >> v2:172.21.15.67:3300/0 conn(0x7f22ac15b6f0 0x7f22ac06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.943+0000 7f22ab400640 1 -- 172.21.15.67:0/1861781754 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3715136156 0 0) 0x7f2290003520 con 0x7f22ac0d0640 2024-05-30T16:53:55.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.943+0000 7f22b1a00640 1 -- 172.21.15.67:0/1861781754 >> v1:172.21.15.169:6789/0 conn(0x7f22ac0d0640 legacy=0x7f22ac0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:55.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.943+0000 7f22b1a00640 1 -- 172.21.15.67:0/1861781754 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac072130 con 0x7f22ac0725e0 2024-05-30T16:53:55.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.946+0000 7f22b1a00640 1 --2- 172.21.15.67:0/1861781754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac0b3370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009d90 tx=0x7f22a402a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b23c50616293e4b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:55.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.947+0000 7f22ab400640 1 -- 172.21.15.67:0/1861781754 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c001670 con 0x7f22ac0725e0 2024-05-30T16:53:55.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.947+0000 7f22ab400640 1 -- 172.21.15.67:0/1861781754 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a4007540 con 0x7f22ac0725e0 2024-05-30T16:53:55.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.947+0000 7f22ab400640 1 -- 172.21.15.67:0/1861781754 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4007910 con 0x7f22ac0725e0 2024-05-30T16:53:55.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.951+0000 7f22b3a00640 1 -- 172.21.15.67:0/1861781754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 msgr2=0x7f22ac0b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:55.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.951+0000 7f22b3a00640 1 --2- 172.21.15.67:0/1861781754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac0b3370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009d90 tx=0x7f22a402a610 comp rx=0 tx=0).stop 2024-05-30T16:53:55.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.952+0000 7f22b3a00640 1 -- 172.21.15.67:0/1861781754 shutdown_connections 2024-05-30T16:53:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.952+0000 7f22b3a00640 1 --2- 172.21.15.67:0/1861781754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac0b3370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.952+0000 7f22b3a00640 1 --2- 172.21.15.67:0/1861781754 >> v2:172.21.15.67:3300/0 conn(0x7f22ac15b6f0 0x7f22ac06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.952+0000 7f22b3a00640 1 -- 172.21.15.67:0/1861781754 >> 172.21.15.67:0/1861781754 conn(0x7f22ac06bd20 msgr2=0x7f22ac06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.953+0000 7f22b3a00640 1 -- 172.21.15.67:0/1861781754 shutdown_connections 2024-05-30T16:53:55.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.953+0000 7f22b3a00640 1 -- 172.21.15.67:0/1861781754 wait complete. 2024-05-30T16:53:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.957+0000 7f22b3a00640 1 Processor -- start 2024-05-30T16:53:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.957+0000 7f22b3a00640 1 -- start start 2024-05-30T16:53:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.960+0000 7f22b3a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac16be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.960+0000 7f22b3a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac0d0640 0x7f22ac154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.960+0000 7f22b3a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22ac15b6f0 0x7f22ac1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.960+0000 7f22b3a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac0b38b0 con 0x7f22ac15b6f0 2024-05-30T16:53:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.960+0000 7f22b3a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac06e890 con 0x7f22ac0d0640 2024-05-30T16:53:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.960+0000 7f22b3a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22ac0b3a30 con 0x7f22ac0725e0 2024-05-30T16:53:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.961+0000 7f22b1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac16be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.961+0000 7f22b1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac16be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:35490/0 (socket says 172.21.15.67:35490) 2024-05-30T16:53:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.961+0000 7f22b1000640 1 -- 172.21.15.67:0/1763488081 learned_addr learned my addr 172.21.15.67:0/1763488081 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.961+0000 7f22abe00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac0d0640 0x7f22ac154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.962+0000 7f22b1a00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22ac15b6f0 0x7f22ac1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.962+0000 7f22b1000640 1 -- 172.21.15.67:0/1763488081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac0d0640 msgr2=0x7f22ac154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.962+0000 7f22b1000640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac0d0640 0x7f22ac154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.962+0000 7f22b1000640 1 -- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22ac15b6f0 msgr2=0x7f22ac1578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.962+0000 7f22b1000640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22ac15b6f0 0x7f22ac1578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.962+0000 7f22b1000640 1 -- 172.21.15.67:0/1763488081 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac0af570 con 0x7f22ac0725e0 2024-05-30T16:53:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.963+0000 7f22b1000640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac16be90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f229c001eb0 tx=0x7f229c00cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.964+0000 7f22a8c00640 1 -- 172.21.15.67:0/1763488081 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c017070 con 0x7f22ac0725e0 2024-05-30T16:53:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.964+0000 7f22a8c00640 1 -- 172.21.15.67:0/1763488081 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229c005ad0 con 0x7f22ac0725e0 2024-05-30T16:53:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.964+0000 7f22a8c00640 1 -- 172.21.15.67:0/1763488081 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c007070 con 0x7f22ac0725e0 2024-05-30T16:53:55.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.964+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ac0af810 con 0x7f22ac0725e0 2024-05-30T16:53:55.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.964+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ac1496d0 con 0x7f22ac0725e0 2024-05-30T16:53:55.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.965+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ac0689e0 con 0x7f22ac0725e0 2024-05-30T16:53:55.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.968+0000 7f22a8c00640 1 -- 172.21.15.67:0/1763488081 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 22) ==== 92162+0+0 (secure 0 0 0) 0x7f229c01a070 con 0x7f22ac0725e0 2024-05-30T16:53:55.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.970+0000 7f22a8c00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f228c06d650 0x7f228c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:55.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.970+0000 7f22a8c00640 1 -- 172.21.15.67:0/1763488081 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f229c089c90 con 0x7f22ac0725e0 2024-05-30T16:53:55.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.971+0000 7f22abe00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f228c06d650 0x7f228c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:55.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.981+0000 7f22a8c00640 1 -- 172.21.15.67:0/1763488081 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f229c059240 con 0x7f22ac0725e0 2024-05-30T16:53:55.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:55.983+0000 7f22abe00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f228c06d650 0x7f228c06fb10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f22ac073740 tx=0x7f2298009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.586+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15506", "target": ["mon-mgr", ""]}) -- 0x7f22ac0cfbf0 con 0x7f228c06d650 2024-05-30T16:53:56.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.965+0000 7f22a8c00640 1 -- 172.21.15.67:0/1763488081 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f22ac0cfbf0 con 0x7f228c06d650 2024-05-30T16:53:56.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.969+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f228c06d650 msgr2=0x7f228c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:56.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.969+0000 7f22b3a00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f228c06d650 0x7f228c06fb10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f22ac073740 tx=0x7f2298009060 comp rx=0 tx=0).stop 2024-05-30T16:53:56.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.969+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 msgr2=0x7f22ac16be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:56.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.969+0000 7f22b3a00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac16be90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f229c001eb0 tx=0x7f229c00cbe0 comp rx=0 tx=0).stop 2024-05-30T16:53:56.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.970+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 shutdown_connections 2024-05-30T16:53:56.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.970+0000 7f22b3a00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f228c06d650 0x7f228c06fb10 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f22ac073740 tx=0x7f2298009060 comp rx=0 tx=0).stop 2024-05-30T16:53:56.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.970+0000 7f22b3a00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22ac15b6f0 0x7f22ac1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:56.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.970+0000 7f22b3a00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac0d0640 0x7f22ac154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:56.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.970+0000 7f22b3a00640 1 --2- 172.21.15.67:0/1763488081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0725e0 0x7f22ac16be90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:56.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.970+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 >> 172.21.15.67:0/1763488081 conn(0x7f22ac06bd20 msgr2=0x7f22ac0b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:56.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.971+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 shutdown_connections 2024-05-30T16:53:56.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:56.971+0000 7f22b3a00640 1 -- 172.21.15.67:0/1763488081 wait complete. 2024-05-30T16:53:57.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15507 2024-05-30T16:53:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.344+0000 7f7097c00640 1 Processor -- start 2024-05-30T16:53:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.344+0000 7f7097c00640 1 -- start start 2024-05-30T16:53:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.346+0000 7f7097c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f70900ae220 0x7f709015b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.347+0000 7f7097c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70900cf7a0 0x7f70900cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.347+0000 7f7097c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f709006b900 0x7f709006bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.347+0000 7f7097c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f70900af680 con 0x7f70900ae220 2024-05-30T16:53:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.347+0000 7f7097c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7090162540 con 0x7f709006b900 2024-05-30T16:53:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.347+0000 7f7097c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70900aea70 con 0x7f70900cf7a0 2024-05-30T16:53:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.347+0000 7f7095200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f70900ae220 0x7f709015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.347+0000 7f7095200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f70900ae220 0x7f709015b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43514/0 (socket says 172.21.15.67:43514) 2024-05-30T16:53:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.347+0000 7f7095200640 1 -- 172.21.15.67:0/4161280422 learned_addr learned my addr 172.21.15.67:0/4161280422 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:57.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.348+0000 7f7095c00640 1 --2- 172.21.15.67:0/4161280422 >> v2:172.21.15.169:3300/0 conn(0x7f709006b900 0x7f709006bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:57.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.349+0000 7f708fe00640 1 --2- 172.21.15.67:0/4161280422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70900cf7a0 0x7f70900cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:57.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.349+0000 7f7095200640 1 -- 172.21.15.67:0/4161280422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70900cf7a0 msgr2=0x7f70900cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:57.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.349+0000 7f7095200640 1 --2- 172.21.15.67:0/4161280422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70900cf7a0 0x7f70900cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:57.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.349+0000 7f7095200640 1 -- 172.21.15.67:0/4161280422 >> v2:172.21.15.169:3300/0 conn(0x7f709006b900 msgr2=0x7f709006bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:57.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.349+0000 7f7095200640 1 --2- 172.21.15.67:0/4161280422 >> v2:172.21.15.169:3300/0 conn(0x7f709006b900 0x7f709006bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:57.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.350+0000 7f7095200640 1 -- 172.21.15.67:0/4161280422 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70900aed50 con 0x7f70900ae220 2024-05-30T16:53:57.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.352+0000 7f7095200640 1 --2- 172.21.15.67:0/4161280422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900ae220 0x7f709015b760 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7090072cd0 tx=0x7f707802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccf2fbe19561e06 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:57.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.352+0000 7f708f400640 1 -- 172.21.15.67:0/4161280422 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078037070 con 0x7f70900ae220 2024-05-30T16:53:57.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.352+0000 7f708f400640 1 -- 172.21.15.67:0/4161280422 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7078032440 con 0x7f70900ae220 2024-05-30T16:53:57.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.352+0000 7f708f400640 1 -- 172.21.15.67:0/4161280422 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078032810 con 0x7f70900ae220 2024-05-30T16:53:57.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.354+0000 7f7097c00640 1 -- 172.21.15.67:0/4161280422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900ae220 msgr2=0x7f709015b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:57.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.354+0000 7f7097c00640 1 --2- 172.21.15.67:0/4161280422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900ae220 0x7f709015b760 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7090072cd0 tx=0x7f707802a1e0 comp rx=0 tx=0).stop 2024-05-30T16:53:57.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.354+0000 7f7097c00640 1 -- 172.21.15.67:0/4161280422 shutdown_connections 2024-05-30T16:53:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.354+0000 7f7097c00640 1 --2- 172.21.15.67:0/4161280422 >> v2:172.21.15.169:3300/0 conn(0x7f709006b900 0x7f709006bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.354+0000 7f7097c00640 1 --2- 172.21.15.67:0/4161280422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70900cf7a0 0x7f70900cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.354+0000 7f7097c00640 1 --2- 172.21.15.67:0/4161280422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900ae220 0x7f709015b760 secure :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7090072cd0 tx=0x7f707802a1e0 comp rx=0 tx=0).stop 2024-05-30T16:53:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.354+0000 7f7097c00640 1 -- 172.21.15.67:0/4161280422 >> 172.21.15.67:0/4161280422 conn(0x7f70900b2f40 msgr2=0x7f70900b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.354+0000 7f7097c00640 1 -- 172.21.15.67:0/4161280422 shutdown_connections 2024-05-30T16:53:57.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.354+0000 7f7097c00640 1 -- 172.21.15.67:0/4161280422 wait complete. 2024-05-30T16:53:57.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.357+0000 7f7097c00640 1 Processor -- start 2024-05-30T16:53:57.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.357+0000 7f7097c00640 1 -- start start 2024-05-30T16:53:57.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.358+0000 7f7097c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709006b900 0x7f709015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:57.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.358+0000 7f7097c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70900cf7a0 0x7f709015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:57.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.358+0000 7f7097c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900c75b0 0x7f709015c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.359+0000 7f7097c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f70900cbcf0 con 0x7f70900c75b0 2024-05-30T16:53:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.359+0000 7f7097c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f70900af310 con 0x7f70900cf7a0 2024-05-30T16:53:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.359+0000 7f7097c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70900cbe70 con 0x7f709006b900 2024-05-30T16:53:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.359+0000 7f7095c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900c75b0 0x7f709015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.359+0000 7f7095c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900c75b0 0x7f709015c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43528/0 (socket says 172.21.15.67:43528) 2024-05-30T16:53:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.359+0000 7f7095c00640 1 -- 172.21.15.67:0/4122113509 learned_addr learned my addr 172.21.15.67:0/4122113509 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.360+0000 7f708fe00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70900cf7a0 0x7f709015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.360+0000 7f7095c00640 1 -- 172.21.15.67:0/4122113509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709006b900 msgr2=0x7f709015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.360+0000 7f7095c00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709006b900 0x7f709015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.360+0000 7f7095c00640 1 -- 172.21.15.67:0/4122113509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70900cf7a0 msgr2=0x7f709015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.360+0000 7f7095c00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70900cf7a0 0x7f709015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.360+0000 7f7095c00640 1 -- 172.21.15.67:0/4122113509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70900b61c0 con 0x7f70900c75b0 2024-05-30T16:53:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.361+0000 7f7095c00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900c75b0 0x7f709015c390 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7088007e60 tx=0x7f708800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.361+0000 7f708cc00640 1 -- 172.21.15.67:0/4122113509 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088014620 con 0x7f70900c75b0 2024-05-30T16:53:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.361+0000 7f708cc00640 1 -- 172.21.15.67:0/4122113509 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7088019070 con 0x7f70900c75b0 2024-05-30T16:53:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.361+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70900b64a0 con 0x7f70900c75b0 2024-05-30T16:53:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.361+0000 7f708cc00640 1 -- 172.21.15.67:0/4122113509 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088014a20 con 0x7f70900c75b0 2024-05-30T16:53:57.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.361+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709015e2f0 con 0x7f70900c75b0 2024-05-30T16:53:57.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.363+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70900689e0 con 0x7f70900c75b0 2024-05-30T16:53:57.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.367+0000 7f708cc00640 1 -- 172.21.15.67:0/4122113509 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 92162+0+0 (secure 0 0 0) 0x7f7088014bc0 con 0x7f70900c75b0 2024-05-30T16:53:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.369+0000 7f708cc00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706c06d620 0x7f706c06fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.369+0000 7f708cc00640 1 -- 172.21.15.67:0/4122113509 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f708808bcf0 con 0x7f70900c75b0 2024-05-30T16:53:57.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.370+0000 7f7095200640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706c06d620 0x7f706c06fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.371+0000 7f7095200640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706c06d620 0x7f706c06fae0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7078034000 tx=0x7f707802c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:57.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.373+0000 7f708cc00640 1 -- 172.21.15.67:0/4122113509 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f708805bb70 con 0x7f70900c75b0 2024-05-30T16:53:57.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:57.836+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15507", "target": ["mon-mgr", ""]}) -- 0x7f709015e6d0 con 0x7f706c06d620 2024-05-30T16:53:58.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.079+0000 7f708cc00640 1 -- 172.21.15.67:0/4122113509 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7078032580 con 0x7f706c06d620 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.081+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706c06d620 msgr2=0x7f706c06fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.081+0000 7f7097c00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706c06d620 0x7f706c06fae0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7078034000 tx=0x7f707802c000 comp rx=0 tx=0).stop 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.081+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900c75b0 msgr2=0x7f709015c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.081+0000 7f7097c00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900c75b0 0x7f709015c390 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7088007e60 tx=0x7f708800e4b0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.082+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 shutdown_connections 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.082+0000 7f7097c00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f706c06d620 0x7f706c06fae0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.082+0000 7f7097c00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70900c75b0 0x7f709015c390 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.082+0000 7f7097c00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70900cf7a0 0x7f709015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.082+0000 7f7097c00640 1 --2- 172.21.15.67:0/4122113509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709006b900 0x7f709015ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.082+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 >> 172.21.15.67:0/4122113509 conn(0x7f70900b2f40 msgr2=0x7f70900cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.082+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 shutdown_connections 2024-05-30T16:53:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.082+0000 7f7097c00640 1 -- 172.21.15.67:0/4122113509 wait complete. 2024-05-30T16:53:58.101 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:53:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.309+0000 7f4f92c00640 1 Processor -- start 2024-05-30T16:53:58.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.309+0000 7f4f92c00640 1 -- start start 2024-05-30T16:53:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.310+0000 7f4f92c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f8c073310 0x7f4f8c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.310+0000 7f4f92c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f8c06dc20 0x7f4f8c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.310+0000 7f4f92c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f8c0afd60 con 0x7f4f8c15b3a0 2024-05-30T16:53:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.310+0000 7f4f92c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4f8c06f980 con 0x7f4f8c06dc20 2024-05-30T16:53:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.310+0000 7f4f92c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4f8c0ae0c0 con 0x7f4f8c073310 2024-05-30T16:53:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.311+0000 7f4f92200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f8c06dc20 0x7f4f8c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.311+0000 7f4f92200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f8c06dc20 0x7f4f8c15f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43542/0 (socket says 172.21.15.67:43542) 2024-05-30T16:53:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.311+0000 7f4f92200640 1 -- 172.21.15.67:0/1180552666 learned_addr learned my addr 172.21.15.67:0/1180552666 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.311+0000 7f4f90e00640 1 --2- 172.21.15.67:0/1180552666 >> v2:172.21.15.169:3300/0 conn(0x7f4f8c073310 0x7f4f8c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.311+0000 7f4f92200640 1 -- 172.21.15.67:0/1180552666 >> v2:172.21.15.169:3300/0 conn(0x7f4f8c073310 msgr2=0x7f4f8c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:58.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.311+0000 7f4f92200640 1 --2- 172.21.15.67:0/1180552666 >> v2:172.21.15.169:3300/0 conn(0x7f4f8c073310 0x7f4f8c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.311+0000 7f4f92200640 1 -- 172.21.15.67:0/1180552666 >> v1:172.21.15.169:6789/0 conn(0x7f4f8c15b3a0 legacy=0x7f4f8c072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:58.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.311+0000 7f4f92200640 1 -- 172.21.15.67:0/1180552666 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c0af7a0 con 0x7f4f8c06dc20 2024-05-30T16:53:58.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.312+0000 7f4f92200640 1 --2- 172.21.15.67:0/1180552666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 0x7f4f8c15f9a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0711f0 tx=0x7f4f8402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c80bc4f3dca2bf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:58.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.313+0000 7f4f8be00640 1 -- 172.21.15.67:0/1180552666 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84036070 con 0x7f4f8c06dc20 2024-05-30T16:53:58.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.313+0000 7f4f8be00640 1 -- 172.21.15.67:0/1180552666 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f84031600 con 0x7f4f8c06dc20 2024-05-30T16:53:58.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.313+0000 7f4f8be00640 1 -- 172.21.15.67:0/1180552666 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f840319d0 con 0x7f4f8c06dc20 2024-05-30T16:53:58.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.313+0000 7f4f92c00640 1 -- 172.21.15.67:0/1180552666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 msgr2=0x7f4f8c15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.313+0000 7f4f92c00640 1 --2- 172.21.15.67:0/1180552666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 0x7f4f8c15f9a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0711f0 tx=0x7f4f8402a6f0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.314+0000 7f4f92c00640 1 -- 172.21.15.67:0/1180552666 shutdown_connections 2024-05-30T16:53:58.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.314+0000 7f4f92c00640 1 --2- 172.21.15.67:0/1180552666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 0x7f4f8c15f9a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.314+0000 7f4f92c00640 1 --2- 172.21.15.67:0/1180552666 >> v2:172.21.15.169:3300/0 conn(0x7f4f8c073310 0x7f4f8c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.314+0000 7f4f92c00640 1 -- 172.21.15.67:0/1180552666 >> 172.21.15.67:0/1180552666 conn(0x7f4f8c0b1fc0 msgr2=0x7f4f8c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:58.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.314+0000 7f4f92c00640 1 -- 172.21.15.67:0/1180552666 shutdown_connections 2024-05-30T16:53:58.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.314+0000 7f4f92c00640 1 -- 172.21.15.67:0/1180552666 wait complete. 2024-05-30T16:53:58.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.316+0000 7f4f92c00640 1 Processor -- start 2024-05-30T16:53:58.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.316+0000 7f4f92c00640 1 -- start start 2024-05-30T16:53:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.317+0000 7f4f92c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 0x7f4f8c146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.317+0000 7f4f92c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f8c073310 0x7f4f8c166b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.317+0000 7f4f92c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f8c15b3a0 0x7f4f8c0b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.317+0000 7f4f92c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c15b780 con 0x7f4f8c06dc20 2024-05-30T16:53:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.317+0000 7f4f92c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c15fee0 con 0x7f4f8c073310 2024-05-30T16:53:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.317+0000 7f4f92c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f8c0ae550 con 0x7f4f8c15b3a0 2024-05-30T16:53:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.317+0000 7f4f91800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 0x7f4f8c146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.317+0000 7f4f91800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 0x7f4f8c146320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43558/0 (socket says 172.21.15.67:43558) 2024-05-30T16:53:58.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.317+0000 7f4f91800640 1 -- 172.21.15.67:0/1585414733 learned_addr learned my addr 172.21.15.67:0/1585414733 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f92200640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f8c15b3a0 0x7f4f8c0b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f90e00640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f8c073310 0x7f4f8c166b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f91800640 1 -- 172.21.15.67:0/1585414733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f8c15b3a0 msgr2=0x7f4f8c0b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f91800640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f8c15b3a0 0x7f4f8c0b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f91800640 1 -- 172.21.15.67:0/1585414733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f8c073310 msgr2=0x7f4f8c166b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f91800640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f8c073310 0x7f4f8c166b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f91800640 1 -- 172.21.15.67:0/1585414733 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c1670d0 con 0x7f4f8c06dc20 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f92200640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f8c15b3a0 0x7f4f8c0b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f91800640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 0x7f4f8c146320 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000b3f0 tx=0x7f4f8000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f89600640 1 -- 172.21.15.67:0/1585414733 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80015020 con 0x7f4f8c06dc20 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f89600640 1 -- 172.21.15.67:0/1585414733 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f80011830 con 0x7f4f8c06dc20 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f89600640 1 -- 172.21.15.67:0/1585414733 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80011c20 con 0x7f4f8c06dc20 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.318+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c146900 con 0x7f4f8c06dc20 2024-05-30T16:53:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.319+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c146ec0 con 0x7f4f8c06dc20 2024-05-30T16:53:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.320+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8c0689e0 con 0x7f4f8c06dc20 2024-05-30T16:53:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.321+0000 7f4f89600640 1 -- 172.21.15.67:0/1585414733 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 92162+0+0 (secure 0 0 0) 0x7f4f80018020 con 0x7f4f8c06dc20 2024-05-30T16:53:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.322+0000 7f4f89600640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f6c06d550 0x7f4f6c06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.322+0000 7f4f89600640 1 -- 172.21.15.67:0/1585414733 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f4f8008a440 con 0x7f4f8c06dc20 2024-05-30T16:53:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.322+0000 7f4f90e00640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f6c06d550 0x7f4f6c06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.323+0000 7f4f90e00640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f6c06d550 0x7f4f6c06fa10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0b1a70 tx=0x7f4f78008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:58.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.327+0000 7f4f89600640 1 -- 172.21.15.67:0/1585414733 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f8001f030 con 0x7f4f8c06dc20 2024-05-30T16:53:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.561+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f4f8c0cc680 con 0x7f4f6c06d550 2024-05-30T16:53:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.564+0000 7f4f89600640 1 -- 172.21.15.67:0/1585414733 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f4f8c0cc680 con 0x7f4f6c06d550 2024-05-30T16:53:58.565 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:53:58.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:53:58.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:53:58.565 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:53:58.565 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:53:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.566+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f6c06d550 msgr2=0x7f4f6c06fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.566+0000 7f4f92c00640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f6c06d550 0x7f4f6c06fa10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0b1a70 tx=0x7f4f78008040 comp rx=0 tx=0).stop 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.566+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 msgr2=0x7f4f8c146320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.566+0000 7f4f92c00640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 0x7f4f8c146320 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000b3f0 tx=0x7f4f8000b8b0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.567+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 shutdown_connections 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.567+0000 7f4f92c00640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f6c06d550 0x7f4f6c06fa10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.567+0000 7f4f92c00640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f8c15b3a0 0x7f4f8c0b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.567+0000 7f4f92c00640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f8c073310 0x7f4f8c166b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.567+0000 7f4f92c00640 1 --2- 172.21.15.67:0/1585414733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f8c06dc20 0x7f4f8c146320 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.567+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 >> 172.21.15.67:0/1585414733 conn(0x7f4f8c0b1fc0 msgr2=0x7f4f8c1512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.567+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 shutdown_connections 2024-05-30T16:53:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.567+0000 7f4f92c00640 1 -- 172.21.15.67:0/1585414733 wait complete. 2024-05-30T16:53:58.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:53:58.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.776+0000 7fc5b8e00640 1 Processor -- start 2024-05-30T16:53:58.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.776+0000 7fc5b8e00640 1 -- start start 2024-05-30T16:53:58.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.777+0000 7fc5b8e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5b415c6b0 0x7fc5b40b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.777+0000 7fc5b8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b40d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.777+0000 7fc5b8e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5b40cd1a0 con 0x7fc5b40ccc30 2024-05-30T16:53:58.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.777+0000 7fc5b8e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b406e3a0 con 0x7fc5b415c6b0 2024-05-30T16:53:58.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.777+0000 7fc5b8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b40cd480 con 0x7fc5b40d0460 2024-05-30T16:53:58.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.778+0000 7fc5b3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b40d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.778+0000 7fc5b3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b40d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:35530/0 (socket says 172.21.15.67:35530) 2024-05-30T16:53:58.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.778+0000 7fc5b3e00640 1 -- 172.21.15.67:0/3016172680 learned_addr learned my addr 172.21.15.67:0/3016172680 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:58.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.778+0000 7fc5b3400640 1 --2- 172.21.15.67:0/3016172680 >> v2:172.21.15.169:3300/0 conn(0x7fc5b415c6b0 0x7fc5b40b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.779+0000 7fc5b3e00640 1 -- 172.21.15.67:0/3016172680 >> v2:172.21.15.169:3300/0 conn(0x7fc5b415c6b0 msgr2=0x7fc5b40b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:53:58.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.779+0000 7fc5b3e00640 1 --2- 172.21.15.67:0/3016172680 >> v2:172.21.15.169:3300/0 conn(0x7fc5b415c6b0 0x7fc5b40b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.779+0000 7fc5b3e00640 1 -- 172.21.15.67:0/3016172680 >> v1:172.21.15.169:6789/0 conn(0x7fc5b40ccc30 legacy=0x7fc5b406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.779+0000 7fc5b3e00640 1 -- 172.21.15.67:0/3016172680 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b40cd760 con 0x7fc5b40d0460 2024-05-30T16:53:58.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.780+0000 7fc5b3e00640 1 --2- 172.21.15.67:0/3016172680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b40d08f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009e80 tx=0x7fc5ac02a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3174cb5d1006591 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:58.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.781+0000 7fc5b2a00640 1 -- 172.21.15.67:0/3016172680 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac036070 con 0x7fc5b40d0460 2024-05-30T16:53:58.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.781+0000 7fc5b2a00640 1 -- 172.21.15.67:0/3016172680 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac031600 con 0x7fc5b40d0460 2024-05-30T16:53:58.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.781+0000 7fc5b2a00640 1 -- 172.21.15.67:0/3016172680 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0319d0 con 0x7fc5b40d0460 2024-05-30T16:53:58.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.781+0000 7fc5b8e00640 1 -- 172.21.15.67:0/3016172680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 msgr2=0x7fc5b40d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.781+0000 7fc5b8e00640 1 --2- 172.21.15.67:0/3016172680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b40d08f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009e80 tx=0x7fc5ac02a750 comp rx=0 tx=0).stop 2024-05-30T16:53:58.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.782+0000 7fc5b8e00640 1 -- 172.21.15.67:0/3016172680 shutdown_connections 2024-05-30T16:53:58.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.782+0000 7fc5b8e00640 1 --2- 172.21.15.67:0/3016172680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b40d08f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.782+0000 7fc5b8e00640 1 --2- 172.21.15.67:0/3016172680 >> v2:172.21.15.169:3300/0 conn(0x7fc5b415c6b0 0x7fc5b40b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.782+0000 7fc5b8e00640 1 -- 172.21.15.67:0/3016172680 >> 172.21.15.67:0/3016172680 conn(0x7fc5b406bae0 msgr2=0x7fc5b406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:58.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.782+0000 7fc5b8e00640 1 -- 172.21.15.67:0/3016172680 shutdown_connections 2024-05-30T16:53:58.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.782+0000 7fc5b8e00640 1 -- 172.21.15.67:0/3016172680 wait complete. 2024-05-30T16:53:58.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.783+0000 7fc5b8e00640 1 Processor -- start 2024-05-30T16:53:58.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.783+0000 7fc5b8e00640 1 -- start start 2024-05-30T16:53:58.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.784+0000 7fc5b8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b40ccc30 0x7fc5b414ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.784+0000 7fc5b8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b41f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.784+0000 7fc5b8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b415c6b0 0x7fc5b40c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.784+0000 7fc5b8e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b40b42f0 con 0x7fc5b40ccc30 2024-05-30T16:53:58.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.784+0000 7fc5b8e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b40cba10 con 0x7fc5b415c6b0 2024-05-30T16:53:58.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.784+0000 7fc5b8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b40b4470 con 0x7fc5b40d0460 2024-05-30T16:53:58.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b40ccc30 0x7fc5b414ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b40ccc30 0x7fc5b414ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43574/0 (socket says 172.21.15.67:43574) 2024-05-30T16:53:58.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3400640 1 -- 172.21.15.67:0/2911125208 learned_addr learned my addr 172.21.15.67:0/2911125208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5abe00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b41f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3e00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b415c6b0 0x7fc5b40c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3400640 1 -- 172.21.15.67:0/2911125208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 msgr2=0x7fc5b41f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3400640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b41f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3400640 1 -- 172.21.15.67:0/2911125208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b415c6b0 msgr2=0x7fc5b40c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3400640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b415c6b0 0x7fc5b40c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3400640 1 -- 172.21.15.67:0/2911125208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b41f41f0 con 0x7fc5b40ccc30 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3e00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b415c6b0 0x7fc5b40c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.785+0000 7fc5b3400640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b40ccc30 0x7fc5b414ed20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400e8a0 tx=0x7fc5a400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.786+0000 7fc5ab400640 1 -- 172.21.15.67:0/2911125208 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4017070 con 0x7fc5b40ccc30 2024-05-30T16:53:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.786+0000 7fc5ab400640 1 -- 172.21.15.67:0/2911125208 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a400c940 con 0x7fc5b40ccc30 2024-05-30T16:53:58.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.786+0000 7fc5ab400640 1 -- 172.21.15.67:0/2911125208 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400cd30 con 0x7fc5b40ccc30 2024-05-30T16:53:58.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.786+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b40ad310 con 0x7fc5b40ccc30 2024-05-30T16:53:58.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.786+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b40ad8d0 con 0x7fc5b40ccc30 2024-05-30T16:53:58.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.787+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b40689e0 con 0x7fc5b40ccc30 2024-05-30T16:53:58.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.788+0000 7fc5ab400640 1 -- 172.21.15.67:0/2911125208 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 22) ==== 92162+0+0 (secure 0 0 0) 0x7fc5a401a070 con 0x7fc5b40ccc30 2024-05-30T16:53:58.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.789+0000 7fc5ab400640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc59006d550 0x7fc59006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:53:58.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.789+0000 7fc5ab400640 1 -- 172.21.15.67:0/2911125208 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fc5a408a5e0 con 0x7fc5b40ccc30 2024-05-30T16:53:58.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.789+0000 7fc5abe00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc59006d550 0x7fc59006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:53:58.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.790+0000 7fc5abe00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc59006d550 0x7fc59006fa10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40c6750 tx=0x7fc59c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:53:58.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:58.793+0000 7fc5ab400640 1 -- 172.21.15.67:0/2911125208 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5b41f41f0 con 0x7fc5b40ccc30 2024-05-30T16:53:59.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.063+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc5b40ced80 con 0x7fc5b40ccc30 2024-05-30T16:53:59.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.064+0000 7fc5ab400640 1 -- 172.21.15.67:0/2911125208 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2379 (secure 0 0 0) 0x7fc5a405a240 con 0x7fc5b40ccc30 2024-05-30T16:53:59.064 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:53:59.064 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:53:59.064 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:53:59.065 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 632, 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 62, 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:53:59.066 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4251, 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 17668421957, 2024-05-30T16:53:59.067 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 35565821952, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 737528291328, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 551314892, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 551013901, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 134, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 147 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 36, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:53:59.068 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 4, 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5937 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:53:59.069 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 11, 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:53:36.221449+0000", 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "a6b1c249-71a9-4e88-ae98-e1a218b10b75": { 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 30.0 seconds before exiting (16s)\n [============================] ", 2024-05-30T16:53:59.070 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:53:59.071 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:53:59.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:53:59.071 INFO:teuthology.orchestra.run.smithi067.stdout: "fa4e9196-4ba4-4af2-a603-a487b633bc42": { 2024-05-30T16:53:59.071 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 30.0 seconds before exiting (25s)\n [============================] ", 2024-05-30T16:53:59.071 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:53:59.071 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:53:59.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:53:59.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:53:59.071 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:53:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc59006d550 msgr2=0x7fc59006fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc59006d550 0x7fc59006fa10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40c6750 tx=0x7fc59c009040 comp rx=0 tx=0).stop 2024-05-30T16:53:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b40ccc30 msgr2=0x7fc5b414ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:53:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b40ccc30 0x7fc5b414ed20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400e8a0 tx=0x7fc5a400ed60 comp rx=0 tx=0).stop 2024-05-30T16:53:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 shutdown_connections 2024-05-30T16:53:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc59006d550 0x7fc59006fa10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5b415c6b0 0x7fc5b40c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5b40d0460 0x7fc5b41f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 --2- 172.21.15.67:0/2911125208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b40ccc30 0x7fc5b414ed20 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:53:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.066+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 >> 172.21.15.67:0/2911125208 conn(0x7fc5b406bae0 msgr2=0x7fc5b406dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:53:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.067+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 shutdown_connections 2024-05-30T16:53:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:53:59.067+0000 7fc5b8e00640 1 -- 172.21.15.67:0/2911125208 wait complete. 2024-05-30T16:54:04.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:54:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.311+0000 7fcc94000640 1 Processor -- start 2024-05-30T16:54:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.311+0000 7fcc94000640 1 -- start start 2024-05-30T16:54:04.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.313+0000 7fcc94000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc8c1621e0 0x7fcc8c0b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.313+0000 7fcc94000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc8c0d3530 0x7fcc8c0d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.313+0000 7fcc94000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc8c0cf560 con 0x7fcc8c0b34e0 2024-05-30T16:54:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.313+0000 7fcc94000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fcc8c0d0860 con 0x7fcc8c0d3530 2024-05-30T16:54:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.313+0000 7fcc94000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcc8c0cf840 con 0x7fcc8c1621e0 2024-05-30T16:54:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.314+0000 7fcc90c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc8c0d3530 0x7fcc8c0d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.314+0000 7fcc92000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcc8c0b34e0 0x7fcc8c0d28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38752/0 (socket says 172.21.15.67:38752) 2024-05-30T16:54:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.314+0000 7fcc92000640 1 -- 172.21.15.67:0/3399428622 learned_addr learned my addr 172.21.15.67:0/3399428622 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:04.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.314+0000 7fcc91600640 1 --2- 172.21.15.67:0/3399428622 >> v2:172.21.15.169:3300/0 conn(0x7fcc8c1621e0 0x7fcc8c0b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:04.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.314+0000 7fcc90c00640 1 -- 172.21.15.67:0/3399428622 >> v2:172.21.15.169:3300/0 conn(0x7fcc8c1621e0 msgr2=0x7fcc8c0b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:04.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.314+0000 7fcc90c00640 1 --2- 172.21.15.67:0/3399428622 >> v2:172.21.15.169:3300/0 conn(0x7fcc8c1621e0 0x7fcc8c0b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.314+0000 7fcc90c00640 1 -- 172.21.15.67:0/3399428622 >> v1:172.21.15.67:6789/0 conn(0x7fcc8c0b34e0 legacy=0x7fcc8c0d28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:04.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.314+0000 7fcc90c00640 1 -- 172.21.15.67:0/3399428622 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c0cfb20 con 0x7fcc8c0d3530 2024-05-30T16:54:04.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.314+0000 7fcc8be00640 1 -- 172.21.15.67:0/3399428622 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180883204 0 0) 0x7fcc8c0cf560 con 0x7fcc8c0b34e0 2024-05-30T16:54:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.315+0000 7fcc90c00640 1 --2- 172.21.15.67:0/3399428622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 0x7fcc8c0d0320 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c08d170 tx=0x7fcc7802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d41f21db42d1a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.316+0000 7fcc8be00640 1 -- 172.21.15.67:0/3399428622 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78037070 con 0x7fcc8c0d3530 2024-05-30T16:54:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.316+0000 7fcc8be00640 1 -- 172.21.15.67:0/3399428622 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc78032440 con 0x7fcc8c0d3530 2024-05-30T16:54:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.316+0000 7fcc8be00640 1 -- 172.21.15.67:0/3399428622 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7802ae60 con 0x7fcc8c0d3530 2024-05-30T16:54:04.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.316+0000 7fcc94000640 1 -- 172.21.15.67:0/3399428622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 msgr2=0x7fcc8c0d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:04.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.316+0000 7fcc94000640 1 --2- 172.21.15.67:0/3399428622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 0x7fcc8c0d0320 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c08d170 tx=0x7fcc7802a110 comp rx=0 tx=0).stop 2024-05-30T16:54:04.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.316+0000 7fcc94000640 1 -- 172.21.15.67:0/3399428622 shutdown_connections 2024-05-30T16:54:04.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.316+0000 7fcc94000640 1 --2- 172.21.15.67:0/3399428622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 0x7fcc8c0d0320 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.316+0000 7fcc94000640 1 --2- 172.21.15.67:0/3399428622 >> v2:172.21.15.169:3300/0 conn(0x7fcc8c1621e0 0x7fcc8c0b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.316+0000 7fcc94000640 1 -- 172.21.15.67:0/3399428622 >> 172.21.15.67:0/3399428622 conn(0x7fcc8c08bfc0 msgr2=0x7fcc8c0d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:04.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.317+0000 7fcc94000640 1 -- 172.21.15.67:0/3399428622 shutdown_connections 2024-05-30T16:54:04.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.317+0000 7fcc94000640 1 -- 172.21.15.67:0/3399428622 wait complete. 2024-05-30T16:54:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.318+0000 7fcc94000640 1 Processor -- start 2024-05-30T16:54:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.318+0000 7fcc94000640 1 -- start start 2024-05-30T16:54:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.319+0000 7fcc94000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc8c0b34e0 0x7fcc8c1f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.319+0000 7fcc94000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 0x7fcc8c1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.319+0000 7fcc94000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc8c1621e0 0x7fcc8c0b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.319+0000 7fcc94000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c0d3170 con 0x7fcc8c0d3530 2024-05-30T16:54:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.319+0000 7fcc94000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c0d2ff0 con 0x7fcc8c0b34e0 2024-05-30T16:54:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.319+0000 7fcc94000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8c0b4010 con 0x7fcc8c1621e0 2024-05-30T16:54:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.320+0000 7fcc90c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 0x7fcc8c1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.320+0000 7fcc90c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 0x7fcc8c1574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34216/0 (socket says 172.21.15.67:34216) 2024-05-30T16:54:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.320+0000 7fcc90c00640 1 -- 172.21.15.67:0/344639436 learned_addr learned my addr 172.21.15.67:0/344639436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:04.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.320+0000 7fcc90c00640 1 -- 172.21.15.67:0/344639436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc8c1621e0 msgr2=0x7fcc8c0b92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:04.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.320+0000 7fcc90c00640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc8c1621e0 0x7fcc8c0b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.320+0000 7fcc90c00640 1 -- 172.21.15.67:0/344639436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc8c0b34e0 msgr2=0x7fcc8c1f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:04.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.320+0000 7fcc90c00640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc8c0b34e0 0x7fcc8c1f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.320+0000 7fcc90c00640 1 -- 172.21.15.67:0/344639436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c158710 con 0x7fcc8c0d3530 2024-05-30T16:54:04.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.320+0000 7fcc90c00640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 0x7fcc8c1574d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcc78000c00 tx=0x7fcc780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.321+0000 7fcc89600640 1 -- 172.21.15.67:0/344639436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78037040 con 0x7fcc8c0d3530 2024-05-30T16:54:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.321+0000 7fcc89600640 1 -- 172.21.15.67:0/344639436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc78042070 con 0x7fcc8c0d3530 2024-05-30T16:54:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.321+0000 7fcc89600640 1 -- 172.21.15.67:0/344639436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78030920 con 0x7fcc8c0d3530 2024-05-30T16:54:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.321+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8c0b9810 con 0x7fcc8c0d3530 2024-05-30T16:54:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.321+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8c157e70 con 0x7fcc8c0d3530 2024-05-30T16:54:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.322+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8c0690f0 con 0x7fcc8c0d3530 2024-05-30T16:54:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.323+0000 7fcc89600640 1 -- 172.21.15.67:0/344639436 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7fcc78030ac0 con 0x7fcc8c0d3530 2024-05-30T16:54:04.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.324+0000 7fcc89600640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc6c06d4c0 0x7fcc6c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:04.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.324+0000 7fcc89600640 1 -- 172.21.15.67:0/344639436 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fcc780ab0f0 con 0x7fcc8c0d3530 2024-05-30T16:54:04.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.324+0000 7fcc91600640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc6c06d4c0 0x7fcc6c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:04.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.325+0000 7fcc91600640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc6c06d4c0 0x7fcc6c06f980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009fa0 tx=0x7fcc7c00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:04.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.329+0000 7fcc89600640 1 -- 172.21.15.67:0/344639436 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc7807a720 con 0x7fcc8c0d3530 2024-05-30T16:54:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.614+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcc8c1602c0 con 0x7fcc8c0d3530 2024-05-30T16:54:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.616+0000 7fcc89600640 1 -- 172.21.15.67:0/344639436 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2379 (secure 0 0 0) 0x7fcc7807ea20 con 0x7fcc8c0d3530 2024-05-30T16:54:04.616 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:04.616 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:54:04.617 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 638, 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 62, 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:54:04.618 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:04.619 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4699, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 19556067278, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 39534219264, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 733559894016, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 420423972, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 427452542, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 103, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 117 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 36, 2024-05-30T16:54:04.620 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4, 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 4, 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5937 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:04.621 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 11, 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:53:36.221449+0000", 2024-05-30T16:54:04.622 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: "a6b1c249-71a9-4e88-ae98-e1a218b10b75": { 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 25.0 seconds before exiting (21s)\n [============================] ", 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: "fa4e9196-4ba4-4af2-a603-a487b633bc42": { 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 25.0 seconds before exiting (30s)\n [============================] ", 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:04.623 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:54:04.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.618+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc6c06d4c0 msgr2=0x7fcc6c06f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:04.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.618+0000 7fcc94000640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc6c06d4c0 0x7fcc6c06f980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009fa0 tx=0x7fcc7c00b040 comp rx=0 tx=0).stop 2024-05-30T16:54:04.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.618+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 msgr2=0x7fcc8c1574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:04.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.618+0000 7fcc94000640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 0x7fcc8c1574d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcc78000c00 tx=0x7fcc780046c0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.619+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 shutdown_connections 2024-05-30T16:54:04.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.619+0000 7fcc94000640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc6c06d4c0 0x7fcc6c06f980 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.619+0000 7fcc94000640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc8c1621e0 0x7fcc8c0b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.619+0000 7fcc94000640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc8c0d3530 0x7fcc8c1574d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.619+0000 7fcc94000640 1 --2- 172.21.15.67:0/344639436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc8c0b34e0 0x7fcc8c1f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:04.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.619+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 >> 172.21.15.67:0/344639436 conn(0x7fcc8c08bfc0 msgr2=0x7fcc8c06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:04.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.619+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 shutdown_connections 2024-05-30T16:54:04.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:04.619+0000 7fcc94000640 1 -- 172.21.15.67:0/344639436 wait complete. 2024-05-30T16:54:09.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:54:09.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.865+0000 7efc68a00640 1 Processor -- start 2024-05-30T16:54:09.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.866+0000 7efc68a00640 1 -- start start 2024-05-30T16:54:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.867+0000 7efc68a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc640723d0 0x7efc640b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.867+0000 7efc68a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efc640b41e0 0x7efc640cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.867+0000 7efc68a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc640cf0c0 con 0x7efc64145320 2024-05-30T16:54:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.867+0000 7efc68a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7efc640b4a20 con 0x7efc640723d0 2024-05-30T16:54:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.867+0000 7efc68a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efc640b4d00 con 0x7efc640b41e0 2024-05-30T16:54:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.867+0000 7efc61e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efc64145320 0x7efc64071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38758/0 (socket says 172.21.15.67:38758) 2024-05-30T16:54:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.867+0000 7efc61e00640 1 -- 172.21.15.67:0/2722341012 learned_addr learned my addr 172.21.15.67:0/2722341012 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:09.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.867+0000 7efc61400640 1 --2- 172.21.15.67:0/2722341012 >> v2:172.21.15.67:3300/0 conn(0x7efc640723d0 0x7efc640b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.868+0000 7efc62800640 1 --2- 172.21.15.67:0/2722341012 >> v2:172.21.15.169:3300/0 conn(0x7efc640b41e0 0x7efc640cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.868+0000 7efc60a00640 1 -- 172.21.15.67:0/2722341012 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2988579891 0 0) 0x7efc640cf0c0 con 0x7efc64145320 2024-05-30T16:54:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.868+0000 7efc60a00640 1 -- 172.21.15.67:0/2722341012 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc48003520 con 0x7efc64145320 2024-05-30T16:54:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.868+0000 7efc61400640 1 -- 172.21.15.67:0/2722341012 >> v2:172.21.15.169:3300/0 conn(0x7efc640b41e0 msgr2=0x7efc640cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.868+0000 7efc61400640 1 --2- 172.21.15.67:0/2722341012 >> v2:172.21.15.169:3300/0 conn(0x7efc640b41e0 0x7efc640cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:09.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.868+0000 7efc61400640 1 -- 172.21.15.67:0/2722341012 >> v1:172.21.15.67:6789/0 conn(0x7efc64145320 legacy=0x7efc64071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:09.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.868+0000 7efc61400640 1 -- 172.21.15.67:0/2722341012 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc640b4fe0 con 0x7efc640723d0 2024-05-30T16:54:09.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.868+0000 7efc62800640 1 --2- 172.21.15.67:0/2722341012 >> v2:172.21.15.169:3300/0 conn(0x7efc640b41e0 0x7efc640cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.869+0000 7efc61400640 1 --2- 172.21.15.67:0/2722341012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640723d0 0x7efc640b3ca0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7efc640ce950 tx=0x7efc4c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feed2f410fc20510 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.869+0000 7efc60a00640 1 -- 172.21.15.67:0/2722341012 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c037070 con 0x7efc640723d0 2024-05-30T16:54:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.869+0000 7efc60a00640 1 -- 172.21.15.67:0/2722341012 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4c032440 con 0x7efc640723d0 2024-05-30T16:54:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.869+0000 7efc60a00640 1 -- 172.21.15.67:0/2722341012 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c032810 con 0x7efc640723d0 2024-05-30T16:54:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.870+0000 7efc68a00640 1 -- 172.21.15.67:0/2722341012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640723d0 msgr2=0x7efc640b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.870+0000 7efc68a00640 1 --2- 172.21.15.67:0/2722341012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640723d0 0x7efc640b3ca0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7efc640ce950 tx=0x7efc4c02a1a0 comp rx=0 tx=0).stop 2024-05-30T16:54:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.870+0000 7efc68a00640 1 -- 172.21.15.67:0/2722341012 shutdown_connections 2024-05-30T16:54:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.870+0000 7efc68a00640 1 --2- 172.21.15.67:0/2722341012 >> v2:172.21.15.169:3300/0 conn(0x7efc640b41e0 0x7efc640cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:09.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.870+0000 7efc68a00640 1 --2- 172.21.15.67:0/2722341012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640723d0 0x7efc640b3ca0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.870+0000 7efc68a00640 1 -- 172.21.15.67:0/2722341012 >> 172.21.15.67:0/2722341012 conn(0x7efc64067ce0 msgr2=0x7efc6406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.870+0000 7efc68a00640 1 -- 172.21.15.67:0/2722341012 shutdown_connections 2024-05-30T16:54:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.870+0000 7efc68a00640 1 -- 172.21.15.67:0/2722341012 wait complete. 2024-05-30T16:54:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.872+0000 7efc68a00640 1 Processor -- start 2024-05-30T16:54:09.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.872+0000 7efc68a00640 1 -- start start 2024-05-30T16:54:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc68a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc640723d0 0x7efc64152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc68a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640b41e0 0x7efc64143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc68a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc64145320 0x7efc64144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc68a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efc640b4650 con 0x7efc640b41e0 2024-05-30T16:54:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc68a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efc64145700 con 0x7efc64145320 2024-05-30T16:54:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc68a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efc640b47d0 con 0x7efc640723d0 2024-05-30T16:54:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc61400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640b41e0 0x7efc64143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc61400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640b41e0 0x7efc64143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34242/0 (socket says 172.21.15.67:34242) 2024-05-30T16:54:09.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc61400640 1 -- 172.21.15.67:0/2497379581 learned_addr learned my addr 172.21.15.67:0/2497379581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:09.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc61400640 1 -- 172.21.15.67:0/2497379581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc640723d0 msgr2=0x7efc64152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:09.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.873+0000 7efc61400640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc640723d0 0x7efc64152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:09.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.874+0000 7efc61400640 1 -- 172.21.15.67:0/2497379581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc64145320 msgr2=0x7efc64144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:09.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.874+0000 7efc61400640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc64145320 0x7efc64144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:09.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.874+0000 7efc61400640 1 -- 172.21.15.67:0/2497379581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc641430c0 con 0x7efc640b41e0 2024-05-30T16:54:09.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.874+0000 7efc61400640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640b41e0 0x7efc64143d10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efc4c000c00 tx=0x7efc4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.874+0000 7efc5a000640 1 -- 172.21.15.67:0/2497379581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c037040 con 0x7efc640b41e0 2024-05-30T16:54:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.874+0000 7efc5a000640 1 -- 172.21.15.67:0/2497379581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4c042070 con 0x7efc640b41e0 2024-05-30T16:54:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.874+0000 7efc5a000640 1 -- 172.21.15.67:0/2497379581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c030960 con 0x7efc640b41e0 2024-05-30T16:54:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.874+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc64142020 con 0x7efc640b41e0 2024-05-30T16:54:09.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.874+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc641425e0 con 0x7efc640b41e0 2024-05-30T16:54:09.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.876+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc640689e0 con 0x7efc640b41e0 2024-05-30T16:54:09.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.877+0000 7efc5a000640 1 -- 172.21.15.67:0/2497379581 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7efc4c030b00 con 0x7efc640b41e0 2024-05-30T16:54:09.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.877+0000 7efc5a000640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc4006d4c0 0x7efc4006f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:09.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.878+0000 7efc5a000640 1 -- 172.21.15.67:0/2497379581 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7efc4c0ab0f0 con 0x7efc640b41e0 2024-05-30T16:54:09.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.878+0000 7efc61e00640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc4006d4c0 0x7efc4006f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:09.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.879+0000 7efc61e00640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc4006d4c0 0x7efc4006f980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efc54000f60 tx=0x7efc540078f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:09.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:09.883+0000 7efc5a000640 1 -- 172.21.15.67:0/2497379581 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc4c045360 con 0x7efc640b41e0 2024-05-30T16:54:10.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.160+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efc640b34e0 con 0x7efc640b41e0 2024-05-30T16:54:10.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.161+0000 7efc5a000640 1 -- 172.21.15.67:0/2497379581 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2377 (secure 0 0 0) 0x7efc4c07ad10 con 0x7efc640b41e0 2024-05-30T16:54:10.161 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:54:10.162 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 643, 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:10.163 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 62, 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:54:10.164 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4697, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 19556112263, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 39534288896, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 733559824384, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 152333638, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 157248653, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 38, 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 46 2024-05-30T16:54:10.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 36, 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4, 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 4, 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5937 2024-05-30T16:54:10.166 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:10.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 11, 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:53:36.221449+0000", 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "a6b1c249-71a9-4e88-ae98-e1a218b10b75": { 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 21.0 seconds before exiting (25s)\n [============================] ", 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "fa4e9196-4ba4-4af2-a603-a487b633bc42": { 2024-05-30T16:54:10.168 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (34s)\n [============================] ", 2024-05-30T16:54:10.169 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:10.169 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:10.169 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:10.169 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:10.169 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.163+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc4006d4c0 msgr2=0x7efc4006f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.163+0000 7efc68a00640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc4006d4c0 0x7efc4006f980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efc54000f60 tx=0x7efc540078f0 comp rx=0 tx=0).stop 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.163+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640b41e0 msgr2=0x7efc64143d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.163+0000 7efc68a00640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640b41e0 0x7efc64143d10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efc4c000c00 tx=0x7efc4c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.163+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 shutdown_connections 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.164+0000 7efc68a00640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc4006d4c0 0x7efc4006f980 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.164+0000 7efc68a00640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc64145320 0x7efc64144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.164+0000 7efc68a00640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc640b41e0 0x7efc64143d10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.164+0000 7efc68a00640 1 --2- 172.21.15.67:0/2497379581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc640723d0 0x7efc64152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:10.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.164+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 >> 172.21.15.67:0/2497379581 conn(0x7efc64067ce0 msgr2=0x7efc640b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:10.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.164+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 shutdown_connections 2024-05-30T16:54:10.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:10.164+0000 7efc68a00640 1 -- 172.21.15.67:0/2497379581 wait complete. 2024-05-30T16:54:11.894 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:54:11.902 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:54:15.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:54:15.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.396+0000 7fad62400640 1 Processor -- start 2024-05-30T16:54:15.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.396+0000 7fad62400640 1 -- start start 2024-05-30T16:54:15.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.398+0000 7fad62400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad5c15d170 0x7fad5c0c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:15.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.398+0000 7fad62400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fad5c163dc0 0x7fad5c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:15.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.398+0000 7fad62400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 0x7fad5c0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.398+0000 7fad62400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fad5c0adb00 con 0x7fad5c163dc0 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.398+0000 7fad62400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fad5c0d0e30 con 0x7fad5c15d170 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.398+0000 7fad62400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c15d550 con 0x7fad5c0d09a0 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.398+0000 7fad5be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fad5c163dc0 0x7fad5c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.398+0000 7fad5be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fad5c163dc0 0x7fad5c0d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47980/0 (socket says 172.21.15.67:47980) 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.398+0000 7fad5be00640 1 -- 172.21.15.67:0/4149033304 learned_addr learned my addr 172.21.15.67:0/4149033304 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.399+0000 7fad61000640 1 --2- 172.21.15.67:0/4149033304 >> v2:172.21.15.169:3300/0 conn(0x7fad5c15d170 0x7fad5c0c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.399+0000 7fad61a00640 1 --2- 172.21.15.67:0/4149033304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 0x7fad5c0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.399+0000 7fad5be00640 1 -- 172.21.15.67:0/4149033304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 msgr2=0x7fad5c0b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.399+0000 7fad5be00640 1 --2- 172.21.15.67:0/4149033304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 0x7fad5c0b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.399+0000 7fad5be00640 1 -- 172.21.15.67:0/4149033304 >> v2:172.21.15.169:3300/0 conn(0x7fad5c15d170 msgr2=0x7fad5c0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.399+0000 7fad5be00640 1 --2- 172.21.15.67:0/4149033304 >> v2:172.21.15.169:3300/0 conn(0x7fad5c15d170 0x7fad5c0c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.399+0000 7fad5be00640 1 -- 172.21.15.67:0/4149033304 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c0ad1d0 con 0x7fad5c163dc0 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.400+0000 7fad5be00640 1 --2- 172.21.15.67:0/4149033304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c163dc0 0x7fad5c0d0460 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fad5c06c800 tx=0x7fad4402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b42db97b7b3b68f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.400+0000 7fad5b400640 1 -- 172.21.15.67:0/4149033304 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44037070 con 0x7fad5c163dc0 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.400+0000 7fad5b400640 1 -- 172.21.15.67:0/4149033304 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad44032440 con 0x7fad5c163dc0 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.400+0000 7fad5b400640 1 -- 172.21.15.67:0/4149033304 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44032810 con 0x7fad5c163dc0 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.401+0000 7fad62400640 1 -- 172.21.15.67:0/4149033304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c163dc0 msgr2=0x7fad5c0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.401+0000 7fad62400640 1 --2- 172.21.15.67:0/4149033304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c163dc0 0x7fad5c0d0460 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fad5c06c800 tx=0x7fad4402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.401+0000 7fad62400640 1 -- 172.21.15.67:0/4149033304 shutdown_connections 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.401+0000 7fad62400640 1 --2- 172.21.15.67:0/4149033304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c163dc0 0x7fad5c0d0460 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.401+0000 7fad62400640 1 --2- 172.21.15.67:0/4149033304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 0x7fad5c0b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.401+0000 7fad62400640 1 --2- 172.21.15.67:0/4149033304 >> v2:172.21.15.169:3300/0 conn(0x7fad5c15d170 0x7fad5c0c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.401+0000 7fad62400640 1 -- 172.21.15.67:0/4149033304 >> 172.21.15.67:0/4149033304 conn(0x7fad5c06bae0 msgr2=0x7fad5c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:15.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.401+0000 7fad62400640 1 -- 172.21.15.67:0/4149033304 shutdown_connections 2024-05-30T16:54:15.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.401+0000 7fad62400640 1 -- 172.21.15.67:0/4149033304 wait complete. 2024-05-30T16:54:15.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.403+0000 7fad62400640 1 Processor -- start 2024-05-30T16:54:15.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.403+0000 7fad62400640 1 -- start start 2024-05-30T16:54:15.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.404+0000 7fad62400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 0x7fad5c157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:15.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.404+0000 7fad62400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c15d170 0x7fad5c0ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:15.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.404+0000 7fad62400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad5c163dc0 0x7fad5c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:15.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.404+0000 7fad62400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c0b3150 con 0x7fad5c15d170 2024-05-30T16:54:15.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.404+0000 7fad62400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c0ad790 con 0x7fad5c163dc0 2024-05-30T16:54:15.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.404+0000 7fad62400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c071bf0 con 0x7fad5c0d09a0 2024-05-30T16:54:15.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.404+0000 7fad5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c15d170 0x7fad5c0ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:15.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.404+0000 7fad5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c15d170 0x7fad5c0ae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47988/0 (socket says 172.21.15.67:47988) 2024-05-30T16:54:15.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.404+0000 7fad5be00640 1 -- 172.21.15.67:0/3053723186 learned_addr learned my addr 172.21.15.67:0/3053723186 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:15.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.405+0000 7fad61a00640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad5c163dc0 0x7fad5c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:15.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.405+0000 7fad61000640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 0x7fad5c157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:15.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.405+0000 7fad5be00640 1 -- 172.21.15.67:0/3053723186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 msgr2=0x7fad5c157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:15.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.405+0000 7fad5be00640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 0x7fad5c157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.405+0000 7fad5be00640 1 -- 172.21.15.67:0/3053723186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad5c163dc0 msgr2=0x7fad5c0ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:15.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.405+0000 7fad5be00640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad5c163dc0 0x7fad5c0ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.405+0000 7fad5be00640 1 -- 172.21.15.67:0/3053723186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c071ed0 con 0x7fad5c15d170 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.405+0000 7fad5be00640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c15d170 0x7fad5c0ae1c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fad44000c00 tx=0x7fad440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.406+0000 7fad58c00640 1 -- 172.21.15.67:0/3053723186 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44041020 con 0x7fad5c15d170 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.406+0000 7fad58c00640 1 -- 172.21.15.67:0/3053723186 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad440306c0 con 0x7fad5c15d170 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.406+0000 7fad58c00640 1 -- 172.21.15.67:0/3053723186 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44037040 con 0x7fad5c15d170 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.406+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c0721b0 con 0x7fad5c15d170 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.406+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c0cd6b0 con 0x7fad5c15d170 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.407+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5c0689e0 con 0x7fad5c15d170 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.408+0000 7fad58c00640 1 -- 172.21.15.67:0/3053723186 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7fad44044020 con 0x7fad5c15d170 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.409+0000 7fad58c00640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad3806d4c0 0x7fad3806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.409+0000 7fad58c00640 1 -- 172.21.15.67:0/3053723186 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fad440aafe0 con 0x7fad5c15d170 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.409+0000 7fad61000640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad3806d4c0 0x7fad3806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:15.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.410+0000 7fad61000640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad3806d4c0 0x7fad3806f980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009a60 tx=0x7fad4c003990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:15.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.414+0000 7fad58c00640 1 -- 172.21.15.67:0/3053723186 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad4404b030 con 0x7fad5c15d170 2024-05-30T16:54:15.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.697+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fad5c099810 con 0x7fad5c15d170 2024-05-30T16:54:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.698+0000 7fad58c00640 1 -- 172.21.15.67:0/3053723186 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2364 (secure 0 0 0) 0x7fad4407ac00 con 0x7fad5c15d170 2024-05-30T16:54:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:15.699 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:54:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:54:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:54:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:54:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:54:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:54:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 649, 2024-05-30T16:54:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 62, 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:54:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4697, 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 19556122597, 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 39534305280, 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 733559808000, 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:54:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 255, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 5714, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 2 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 36, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 4, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:54:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5937 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 11, 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:53:36.221449+0000", 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "a6b1c249-71a9-4e88-ae98-e1a218b10b75": { 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 15.0 seconds before exiting (31s)\n [============================] ", 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "fa4e9196-4ba4-4af2-a603-a487b633bc42": { 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (40s)\n [============================] ", 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:15.706 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad3806d4c0 msgr2=0x7fad3806f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad3806d4c0 0x7fad3806f980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009a60 tx=0x7fad4c003990 comp rx=0 tx=0).stop 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c15d170 msgr2=0x7fad5c0ae1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c15d170 0x7fad5c0ae1c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fad44000c00 tx=0x7fad440046c0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 shutdown_connections 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad3806d4c0 0x7fad3806f980 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad5c163dc0 0x7fad5c0ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad5c15d170 0x7fad5c0ae1c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 --2- 172.21.15.67:0/3053723186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad5c0d09a0 0x7fad5c157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.701+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 >> 172.21.15.67:0/3053723186 conn(0x7fad5c06bae0 msgr2=0x7fad5c06bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.702+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 shutdown_connections 2024-05-30T16:54:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:15.702+0000 7fad62400640 1 -- 172.21.15.67:0/3053723186 wait complete. 2024-05-30T16:54:20.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:54:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.944+0000 7f22f7c00640 1 Processor -- start 2024-05-30T16:54:20.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.944+0000 7f22f7c00640 1 -- start start 2024-05-30T16:54:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.945+0000 7f22f7c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22f0145320 0x7f22f0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.945+0000 7f22f7c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f00cf0c0 con 0x7f22f0072200 2024-05-30T16:54:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.945+0000 7f22f7c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f00b4a20 con 0x7f22f00b43b0 2024-05-30T16:54:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.945+0000 7f22f7c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f22f00b4d00 con 0x7f22f0145320 2024-05-30T16:54:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.946+0000 7f22efe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f22f0072200 0x7f22f00b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52924/0 (socket says 172.21.15.67:52924) 2024-05-30T16:54:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.946+0000 7f22efe00640 1 -- 172.21.15.67:0/1198959448 learned_addr learned my addr 172.21.15.67:0/1198959448 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.946+0000 7f22f5200640 1 --2- 172.21.15.67:0/1198959448 >> v2:172.21.15.67:3300/0 conn(0x7f22f0145320 0x7f22f0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.946+0000 7f22ef400640 1 -- 172.21.15.67:0/1198959448 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 883985095 0 0) 0x7f22f00cf0c0 con 0x7f22f0072200 2024-05-30T16:54:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.946+0000 7f22ef400640 1 -- 172.21.15.67:0/1198959448 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22d4003520 con 0x7f22f0072200 2024-05-30T16:54:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.946+0000 7f22f5200640 1 -- 172.21.15.67:0/1198959448 >> v1:172.21.15.169:6789/0 conn(0x7f22f00b43b0 legacy=0x7f22f00cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.946+0000 7f22f5200640 1 -- 172.21.15.67:0/1198959448 >> v1:172.21.15.67:6789/0 conn(0x7f22f0072200 legacy=0x7f22f00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.946+0000 7f22f5200640 1 -- 172.21.15.67:0/1198959448 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f00b4fe0 con 0x7f22f0145320 2024-05-30T16:54:20.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.947+0000 7f22f5200640 1 --2- 172.21.15.67:0/1198959448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f0145320 0x7f22f0071cc0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f22e00097a0 tx=0x7f22e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e63c20d1524ca4b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:20.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.948+0000 7f22ef400640 1 -- 172.21.15.67:0/1198959448 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0037070 con 0x7f22f0145320 2024-05-30T16:54:20.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.948+0000 7f22ef400640 1 -- 172.21.15.67:0/1198959448 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e0032440 con 0x7f22f0145320 2024-05-30T16:54:20.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.948+0000 7f22ef400640 1 -- 172.21.15.67:0/1198959448 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0032810 con 0x7f22f0145320 2024-05-30T16:54:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.948+0000 7f22f7c00640 1 -- 172.21.15.67:0/1198959448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f0145320 msgr2=0x7f22f0071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.948+0000 7f22f7c00640 1 --2- 172.21.15.67:0/1198959448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f0145320 0x7f22f0071cc0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f22e00097a0 tx=0x7f22e002a180 comp rx=0 tx=0).stop 2024-05-30T16:54:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.948+0000 7f22f7c00640 1 -- 172.21.15.67:0/1198959448 shutdown_connections 2024-05-30T16:54:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.948+0000 7f22f7c00640 1 --2- 172.21.15.67:0/1198959448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f0145320 0x7f22f0071cc0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.948+0000 7f22f7c00640 1 -- 172.21.15.67:0/1198959448 >> 172.21.15.67:0/1198959448 conn(0x7f22f0067ce0 msgr2=0x7f22f006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:20.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.948+0000 7f22f7c00640 1 -- 172.21.15.67:0/1198959448 shutdown_connections 2024-05-30T16:54:20.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.949+0000 7f22f7c00640 1 -- 172.21.15.67:0/1198959448 wait complete. 2024-05-30T16:54:20.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.950+0000 7f22f7c00640 1 Processor -- start 2024-05-30T16:54:20.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.950+0000 7f22f7c00640 1 -- start start 2024-05-30T16:54:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.951+0000 7f22f7c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f0072200 0x7f22f0133a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.951+0000 7f22f7c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f00b43b0 0x7f22f015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.951+0000 7f22f7c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0145320 0x7f22f0144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.951+0000 7f22f7c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22f00cff90 con 0x7f22f00b43b0 2024-05-30T16:54:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.951+0000 7f22f7c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0145700 con 0x7f22f0072200 2024-05-30T16:54:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.951+0000 7f22f7c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22f00d0110 con 0x7f22f0145320 2024-05-30T16:54:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.952+0000 7f22efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f00b43b0 0x7f22f015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:20.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.952+0000 7f22efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f00b43b0 0x7f22f015ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34162/0 (socket says 172.21.15.67:34162) 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.952+0000 7f22efe00640 1 -- 172.21.15.67:0/1440401804 learned_addr learned my addr 172.21.15.67:0/1440401804 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.952+0000 7f22efe00640 1 -- 172.21.15.67:0/1440401804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0145320 msgr2=0x7f22f0144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.952+0000 7f22efe00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0145320 0x7f22f0144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.952+0000 7f22efe00640 1 -- 172.21.15.67:0/1440401804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f0072200 msgr2=0x7f22f0133a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.952+0000 7f22efe00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f0072200 0x7f22f0133a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.952+0000 7f22efe00640 1 -- 172.21.15.67:0/1440401804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f0142680 con 0x7f22f00b43b0 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.952+0000 7f22efe00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f00b43b0 0x7f22f015ef40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c8c0 tx=0x7f22dc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.953+0000 7f22ecc00640 1 -- 172.21.15.67:0/1440401804 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc018070 con 0x7f22f00b43b0 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.953+0000 7f22ecc00640 1 -- 172.21.15.67:0/1440401804 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22dc00a640 con 0x7f22f00b43b0 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.953+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f0142960 con 0x7f22f00b43b0 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.953+0000 7f22ecc00640 1 -- 172.21.15.67:0/1440401804 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00aa70 con 0x7f22f00b43b0 2024-05-30T16:54:20.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.953+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f0142f20 con 0x7f22f00b43b0 2024-05-30T16:54:20.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.954+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f00689e0 con 0x7f22f00b43b0 2024-05-30T16:54:20.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.955+0000 7f22ecc00640 1 -- 172.21.15.67:0/1440401804 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7f22dc01b070 con 0x7f22f00b43b0 2024-05-30T16:54:20.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.956+0000 7f22ecc00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f22d006d4f0 0x7f22d006f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:20.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.957+0000 7f22ecc00640 1 -- 172.21.15.67:0/1440401804 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f22dc089f50 con 0x7f22f00b43b0 2024-05-30T16:54:20.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.957+0000 7f22f5200640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f22d006d4f0 0x7f22d006f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:20.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.957+0000 7f22f5200640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f22d006d4f0 0x7f22d006f9b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f22e0034000 tx=0x7f22e002c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:20.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:20.961+0000 7f22ecc00640 1 -- 172.21.15.67:0/1440401804 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22dc059ae0 con 0x7f22f00b43b0 2024-05-30T16:54:21.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.249+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f22f00b34e0 con 0x7f22f00b43b0 2024-05-30T16:54:21.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.250+0000 7f22ecc00640 1 -- 172.21.15.67:0/1440401804 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2331 (secure 0 0 0) 0x7f22dc0593e0 con 0x7f22f00b43b0 2024-05-30T16:54:21.250 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:54:21.251 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 654, 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 62, 2024-05-30T16:54:21.252 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:54:21.253 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:54:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:54:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:54:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4697, 2024-05-30T16:54:21.254 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 19556127805, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 39534321664, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 733559791616, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 1535, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 36, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:54:21.255 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 4, 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5937 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:54:21.256 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 11, 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:53:36.221449+0000", 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:21.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "a6b1c249-71a9-4e88-ae98-e1a218b10b75": { 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 9.0 seconds before exiting (37s)\n [============================] ", 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "fa4e9196-4ba4-4af2-a603-a487b633bc42": { 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 9.0 seconds before exiting (46s)\n [============================] ", 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:21.258 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.252+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f22d006d4f0 msgr2=0x7f22d006f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f22d006d4f0 0x7f22d006f9b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f22e0034000 tx=0x7f22e002c000 comp rx=0 tx=0).stop 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f00b43b0 msgr2=0x7f22f015ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f00b43b0 0x7f22f015ef40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c8c0 tx=0x7f22dc00cd80 comp rx=0 tx=0).stop 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 shutdown_connections 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f22d006d4f0 0x7f22d006f9b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f0145320 0x7f22f0144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22f00b43b0 0x7f22f015ef40 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 --2- 172.21.15.67:0/1440401804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f0072200 0x7f22f0133a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 >> 172.21.15.67:0/1440401804 conn(0x7f22f0067ce0 msgr2=0x7f22f00cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 shutdown_connections 2024-05-30T16:54:21.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:21.253+0000 7f22f7c00640 1 -- 172.21.15.67:0/1440401804 wait complete. 2024-05-30T16:54:26.276 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:54:26.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.496+0000 7f52ede00640 1 Processor -- start 2024-05-30T16:54:26.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.496+0000 7f52ede00640 1 -- start start 2024-05-30T16:54:26.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.497+0000 7f52ede00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52e815c6b0 0x7f52e80b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:26.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.497+0000 7f52ede00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f52e80ccc30 0x7f52e806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:26.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.497+0000 7f52ede00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e80cd1a0 con 0x7f52e80d0460 2024-05-30T16:54:26.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.497+0000 7f52ede00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f52e80cd480 con 0x7f52e80ccc30 2024-05-30T16:54:26.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.497+0000 7f52ede00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f52e80cd760 con 0x7f52e815c6b0 2024-05-30T16:54:26.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.498+0000 7f52e7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f52e80ccc30 0x7f52e806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:26.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.498+0000 7f52ed400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f52e80d0460 0x7f52e80cbf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52934/0 (socket says 172.21.15.67:52934) 2024-05-30T16:54:26.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.498+0000 7f52ed400640 1 -- 172.21.15.67:0/61388635 learned_addr learned my addr 172.21.15.67:0/61388635 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:26.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.498+0000 7f52eca00640 1 --2- 172.21.15.67:0/61388635 >> v2:172.21.15.169:3300/0 conn(0x7f52e815c6b0 0x7f52e80b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:26.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.498+0000 7f52e7e00640 1 -- 172.21.15.67:0/61388635 >> v2:172.21.15.169:3300/0 conn(0x7f52e815c6b0 msgr2=0x7f52e80b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:26.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.498+0000 7f52e7e00640 1 --2- 172.21.15.67:0/61388635 >> v2:172.21.15.169:3300/0 conn(0x7f52e815c6b0 0x7f52e80b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.498+0000 7f52e7e00640 1 -- 172.21.15.67:0/61388635 >> v1:172.21.15.67:6789/0 conn(0x7f52e80d0460 legacy=0x7f52e80cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:26.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.498+0000 7f52e7e00640 1 -- 172.21.15.67:0/61388635 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e80cda40 con 0x7f52e80ccc30 2024-05-30T16:54:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.498+0000 7f52e7400640 1 -- 172.21.15.67:0/61388635 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2693961267 0 0) 0x7f52e80cd1a0 con 0x7f52e80d0460 2024-05-30T16:54:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.499+0000 7f52e7e00640 1 --2- 172.21.15.67:0/61388635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ccc30 0x7f52e806dc90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f52d40097a0 tx=0x7f52d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5861282443a1a94e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52e7400640 1 -- 172.21.15.67:0/61388635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4037070 con 0x7f52e80ccc30 2024-05-30T16:54:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52e7400640 1 -- 172.21.15.67:0/61388635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d4032440 con 0x7f52e80ccc30 2024-05-30T16:54:26.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52e7400640 1 -- 172.21.15.67:0/61388635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4032810 con 0x7f52e80ccc30 2024-05-30T16:54:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52ede00640 1 -- 172.21.15.67:0/61388635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ccc30 msgr2=0x7f52e806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52ede00640 1 --2- 172.21.15.67:0/61388635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ccc30 0x7f52e806dc90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f52d40097a0 tx=0x7f52d402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52ede00640 1 -- 172.21.15.67:0/61388635 shutdown_connections 2024-05-30T16:54:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52ede00640 1 --2- 172.21.15.67:0/61388635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80ccc30 0x7f52e806dc90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52ede00640 1 --2- 172.21.15.67:0/61388635 >> v2:172.21.15.169:3300/0 conn(0x7f52e815c6b0 0x7f52e80b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52ede00640 1 -- 172.21.15.67:0/61388635 >> 172.21.15.67:0/61388635 conn(0x7f52e806bae0 msgr2=0x7f52e806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:26.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52ede00640 1 -- 172.21.15.67:0/61388635 shutdown_connections 2024-05-30T16:54:26.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.500+0000 7f52ede00640 1 -- 172.21.15.67:0/61388635 wait complete. 2024-05-30T16:54:26.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.502+0000 7f52ede00640 1 Processor -- start 2024-05-30T16:54:26.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.502+0000 7f52ede00640 1 -- start start 2024-05-30T16:54:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.503+0000 7f52ede00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e80ccc30 0x7f52e81f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.503+0000 7f52ede00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80d0460 0x7f52e81f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.503+0000 7f52ede00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e815c6b0 0x7f52e815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.503+0000 7f52ede00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80b4470 con 0x7f52e80d0460 2024-05-30T16:54:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.503+0000 7f52ede00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80b42f0 con 0x7f52e815c6b0 2024-05-30T16:54:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.503+0000 7f52ede00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f52e806e1d0 con 0x7f52e80ccc30 2024-05-30T16:54:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80d0460 0x7f52e81f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80d0460 0x7f52e81f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34190/0 (socket says 172.21.15.67:34190) 2024-05-30T16:54:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52e7e00640 1 -- 172.21.15.67:0/3853995202 learned_addr learned my addr 172.21.15.67:0/3853995202 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52eca00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e80ccc30 0x7f52e81f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52e7e00640 1 -- 172.21.15.67:0/3853995202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e80ccc30 msgr2=0x7f52e81f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52e7e00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e80ccc30 0x7f52e81f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52ed400640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e815c6b0 0x7f52e815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52e7e00640 1 -- 172.21.15.67:0/3853995202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e815c6b0 msgr2=0x7f52e815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52e7e00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e815c6b0 0x7f52e815ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52e7e00640 1 -- 172.21.15.67:0/3853995202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e80b0f90 con 0x7f52e80d0460 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.504+0000 7f52e7e00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80d0460 0x7f52e81f4d20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f52d4035040 tx=0x7f52d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.505+0000 7f52e4c00640 1 -- 172.21.15.67:0/3853995202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4037040 con 0x7f52e80d0460 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.505+0000 7f52e4c00640 1 -- 172.21.15.67:0/3853995202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d4030770 con 0x7f52e80d0460 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.505+0000 7f52e4c00640 1 -- 172.21.15.67:0/3853995202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4030b80 con 0x7f52e80d0460 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.505+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e81f7f90 con 0x7f52e80d0460 2024-05-30T16:54:26.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.505+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e81f8550 con 0x7f52e80d0460 2024-05-30T16:54:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.506+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e80689e0 con 0x7f52e80d0460 2024-05-30T16:54:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.507+0000 7f52e4c00640 1 -- 172.21.15.67:0/3853995202 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7f52d4044020 con 0x7f52e80d0460 2024-05-30T16:54:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.508+0000 7f52e4c00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52c806d510 0x7f52c806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.508+0000 7f52e4c00640 1 -- 172.21.15.67:0/3853995202 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f52d40aac80 con 0x7f52e80d0460 2024-05-30T16:54:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.509+0000 7f52eca00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52c806d510 0x7f52c806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:26.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.509+0000 7f52eca00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52c806d510 0x7f52c806f9d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f52dc006fd0 tx=0x7f52dc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.514+0000 7f52e4c00640 1 -- 172.21.15.67:0/3853995202 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52d4032e20 con 0x7f52e80d0460 2024-05-30T16:54:26.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.796+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52e80cfce0 con 0x7f52e80d0460 2024-05-30T16:54:26.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.797+0000 7f52e4c00640 1 -- 172.21.15.67:0/3853995202 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2330 (secure 0 0 0) 0x7f52d407a8a0 con 0x7f52e80d0460 2024-05-30T16:54:26.798 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:26.798 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:54:26.799 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 660, 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 62, 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:54:26.800 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:54:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4697, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 19556127805, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 39534321664, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 733559791616, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 511, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 36, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:54:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 4, 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5937 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:54:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 11, 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:53:36.221449+0000", 2024-05-30T16:54:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "a6b1c249-71a9-4e88-ae98-e1a218b10b75": { 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 3.0 seconds before exiting (43s)\n [============================] ", 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "fa4e9196-4ba4-4af2-a603-a487b633bc42": { 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 3.0 seconds before exiting (53s)\n [============================] ", 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:26.805 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:54:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52c806d510 msgr2=0x7f52c806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52c806d510 0x7f52c806f9d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f52dc006fd0 tx=0x7f52dc008040 comp rx=0 tx=0).stop 2024-05-30T16:54:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80d0460 msgr2=0x7f52e81f4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80d0460 0x7f52e81f4d20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f52d4035040 tx=0x7f52d40046c0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 shutdown_connections 2024-05-30T16:54:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52c806d510 0x7f52c806f9d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e815c6b0 0x7f52e815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e80d0460 0x7f52e81f4d20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 --2- 172.21.15.67:0/3853995202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52e80ccc30 0x7f52e81f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.799+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 >> 172.21.15.67:0/3853995202 conn(0x7f52e806bae0 msgr2=0x7f52e80b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.800+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 shutdown_connections 2024-05-30T16:54:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:26.800+0000 7f52ede00640 1 -- 172.21.15.67:0/3853995202 wait complete. 2024-05-30T16:54:31.823 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:54:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.039+0000 7fb226e00640 1 Processor -- start 2024-05-30T16:54:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.039+0000 7fb226e00640 1 -- start start 2024-05-30T16:54:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.040+0000 7fb226e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb22015c6b0 0x7fb2200b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.041+0000 7fb226e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2200cd1a0 con 0x7fb2200ccc30 2024-05-30T16:54:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.041+0000 7fb226e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2200cd480 con 0x7fb2200d0460 2024-05-30T16:54:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.041+0000 7fb226e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb22006e3a0 con 0x7fb22015c6b0 2024-05-30T16:54:32.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.041+0000 7fb225000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb2200ccc30 0x7fb22006dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52504/0 (socket says 172.21.15.67:52504) 2024-05-30T16:54:32.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.041+0000 7fb225000640 1 -- 172.21.15.67:0/272887376 learned_addr learned my addr 172.21.15.67:0/272887376 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:32.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.041+0000 7fb225a00640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb22015c6b0 0x7fb2200b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.041+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1916318084 0 0) 0x7fb2200cd1a0 con 0x7fb2200ccc30 2024-05-30T16:54:32.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.041+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb208003520 con 0x7fb2200ccc30 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 480898281 0 0) 0x7fb208003520 con 0x7fb2200ccc30 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb22015c6b0 msgr2=0x7fb2200b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb22015c6b0 0x7fb2200b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 >> v1:172.21.15.169:6789/0 conn(0x7fb2200d0460 legacy=0x7fb2200cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200cd760 con 0x7fb2200ccc30 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fb20c001670 con 0x7fb2200ccc30 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 >> v1:172.21.15.67:6789/0 conn(0x7fb2200ccc30 legacy=0x7fb22006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2080039e0 0x7fb208003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb208004800 0x7fb208004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb22015c6b0 0x7fb208005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb22006e3a0 con 0x7fb208004800 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb20800bb50 con 0x7fb2080039e0 2024-05-30T16:54:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb20800bd50 con 0x7fb22015c6b0 2024-05-30T16:54:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb225000640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb208004800 0x7fb208004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.042+0000 7fb225a00640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2080039e0 0x7fb208003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.043+0000 7fb225000640 1 -- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb22015c6b0 msgr2=0x7fb208005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.043+0000 7fb225000640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb22015c6b0 0x7fb208005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.043+0000 7fb225000640 1 -- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2080039e0 msgr2=0x7fb208003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.043+0000 7fb225000640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2080039e0 0x7fb208003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.043+0000 7fb225000640 1 -- 172.21.15.67:0/272887376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb208003520 con 0x7fb208004800 2024-05-30T16:54:32.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.044+0000 7fb225000640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb208004800 0x7fb208004c80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c0e0 tx=0x7fb208005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ab7dc370f0ce9f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:32.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.044+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c007c20 con 0x7fb208004800 2024-05-30T16:54:32.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.044+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20c005af0 con 0x7fb208004800 2024-05-30T16:54:32.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.044+0000 7fb21fe00640 1 -- 172.21.15.67:0/272887376 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c007070 con 0x7fb208004800 2024-05-30T16:54:32.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.045+0000 7fb226e00640 1 -- 172.21.15.67:0/272887376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb208004800 msgr2=0x7fb208004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.045+0000 7fb226e00640 1 --2- 172.21.15.67:0/272887376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb208004800 0x7fb208004c80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c0e0 tx=0x7fb208005830 comp rx=0 tx=0).stop 2024-05-30T16:54:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.045+0000 7fb226400640 1 -- 172.21.15.67:0/272887376 reap_dead start 2024-05-30T16:54:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.045+0000 7fb226e00640 1 -- 172.21.15.67:0/272887376 shutdown_connections 2024-05-30T16:54:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.045+0000 7fb226e00640 1 -- 172.21.15.67:0/272887376 >> 172.21.15.67:0/272887376 conn(0x7fb22006bae0 msgr2=0x7fb22006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.045+0000 7fb226e00640 1 -- 172.21.15.67:0/272887376 shutdown_connections 2024-05-30T16:54:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.045+0000 7fb226e00640 1 -- 172.21.15.67:0/272887376 wait complete. 2024-05-30T16:54:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.047+0000 7fb226e00640 1 Processor -- start 2024-05-30T16:54:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.047+0000 7fb226e00640 1 -- start start 2024-05-30T16:54:32.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.048+0000 7fb226e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb208004800 0x7fb2200b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.048+0000 7fb226e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2200728d0 0x7fb2201f1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.048+0000 7fb226e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2201f1750 0x7fb2200ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.048+0000 7fb226e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb2200cc640 con 0x7fb2201f1750 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.048+0000 7fb226e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb2200cba10 con 0x7fb208004800 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.048+0000 7fb226e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb2200cc7c0 con 0x7fb2200728d0 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.048+0000 7fb226400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2201f1750 0x7fb2200ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.048+0000 7fb226400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2201f1750 0x7fb2200ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56708/0 (socket says 172.21.15.67:56708) 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.048+0000 7fb226400640 1 -- 172.21.15.67:0/2478682608 learned_addr learned my addr 172.21.15.67:0/2478682608 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb225a00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb208004800 0x7fb2200b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb225000640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2200728d0 0x7fb2201f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb226400640 1 -- 172.21.15.67:0/2478682608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2200728d0 msgr2=0x7fb2201f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb226400640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2200728d0 0x7fb2201f1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb226400640 1 -- 172.21.15.67:0/2478682608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb208004800 msgr2=0x7fb2200b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb226400640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb208004800 0x7fb2200b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb226400640 1 -- 172.21.15.67:0/2478682608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb220072d50 con 0x7fb2201f1750 2024-05-30T16:54:32.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb225a00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb208004800 0x7fb2200b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:32.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb226400640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2201f1750 0x7fb2200ae180 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb2200ca770 tx=0x7fb21800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:32.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb21d600640 1 -- 172.21.15.67:0/2478682608 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218017070 con 0x7fb2201f1750 2024-05-30T16:54:32.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb21d600640 1 -- 172.21.15.67:0/2478682608 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb218005d60 con 0x7fb2201f1750 2024-05-30T16:54:32.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb21d600640 1 -- 172.21.15.67:0/2478682608 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21800a880 con 0x7fb2201f1750 2024-05-30T16:54:32.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2200c7390 con 0x7fb2201f1750 2024-05-30T16:54:32.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.049+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2200c7950 con 0x7fb2201f1750 2024-05-30T16:54:32.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.050+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2200689e0 con 0x7fb2201f1750 2024-05-30T16:54:32.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.052+0000 7fb21d600640 1 -- 172.21.15.67:0/2478682608 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7fb21801a070 con 0x7fb2201f1750 2024-05-30T16:54:32.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.053+0000 7fb21d600640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb20006d4f0 0x7fb20006f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.053+0000 7fb21d600640 1 -- 172.21.15.67:0/2478682608 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fb218089a60 con 0x7fb2201f1750 2024-05-30T16:54:32.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.053+0000 7fb225a00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb20006d4f0 0x7fb20006f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.054+0000 7fb225a00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb20006d4f0 0x7fb20006f9b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb214006fd0 tx=0x7fb214008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:32.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.058+0000 7fb21d600640 1 -- 172.21.15.67:0/2478682608 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2180595c0 con 0x7fb2201f1750 2024-05-30T16:54:32.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.337+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb2200b1f60 con 0x7fb2201f1750 2024-05-30T16:54:32.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.338+0000 7fb21d600640 1 -- 172.21.15.67:0/2478682608 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1731 (secure 0 0 0) 0x7fb218058ec0 con 0x7fb2201f1750 2024-05-30T16:54:32.340 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:32.340 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:54:32.340 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:54:32.340 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:54:32.340 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 665, 2024-05-30T16:54:32.341 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 62, 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:54:32.342 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4697, 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 19556127805, 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 39534321664, 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 733559791616, 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:54:32.343 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 36, 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4, 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 4, 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 5937 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:32.344 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 11, 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:53:36.221449+0000", 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:54:32.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.346 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": {} 2024-05-30T16:54:32.346 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:54:32.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb20006d4f0 msgr2=0x7fb20006f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb20006d4f0 0x7fb20006f9b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb214006fd0 tx=0x7fb214008040 comp rx=0 tx=0).stop 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2201f1750 msgr2=0x7fb2200ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2201f1750 0x7fb2200ae180 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb2200ca770 tx=0x7fb21800cb90 comp rx=0 tx=0).stop 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 shutdown_connections 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb20006d4f0 0x7fb20006f9b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2201f1750 0x7fb2200ae180 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2200728d0 0x7fb2201f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 --2- 172.21.15.67:0/2478682608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb208004800 0x7fb2200b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.341+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 >> 172.21.15.67:0/2478682608 conn(0x7fb22006bae0 msgr2=0x7fb2200d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.342+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 shutdown_connections 2024-05-30T16:54:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.342+0000 7fb226e00640 1 -- 172.21.15.67:0/2478682608 wait complete. 2024-05-30T16:54:32.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-30T16:54:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.575+0000 7fabb3c00640 1 Processor -- start 2024-05-30T16:54:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.575+0000 7fabb3c00640 1 -- start start 2024-05-30T16:54:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.577+0000 7fabb3c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fabac0d0640 0x7fabac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.577+0000 7fabb3c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0b32c0 0x7fabac0b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.577+0000 7fabb3c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabac0d0060 con 0x7fabac0d0a20 2024-05-30T16:54:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.577+0000 7fabb3c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fabac0adbb0 con 0x7fabac0d0640 2024-05-30T16:54:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.577+0000 7fabb3c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fabac15db10 con 0x7fabac0b32c0 2024-05-30T16:54:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.577+0000 7fabb1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fabac0d0640 0x7fabac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.577+0000 7fababe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fabac0d0a20 0x7fabac0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52512/0 (socket says 172.21.15.67:52512) 2024-05-30T16:54:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.577+0000 7fababe00640 1 -- 172.21.15.67:0/1774493205 learned_addr learned my addr 172.21.15.67:0/1774493205 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.578+0000 7fabb1c00640 1 --2- 172.21.15.67:0/1774493205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0b32c0 0x7fabac0b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.578+0000 7fabb1200640 1 -- 172.21.15.67:0/1774493205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0b32c0 msgr2=0x7fabac0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.578+0000 7fabb1200640 1 --2- 172.21.15.67:0/1774493205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0b32c0 0x7fabac0b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.578+0000 7fabb1200640 1 -- 172.21.15.67:0/1774493205 >> v1:172.21.15.67:6789/0 conn(0x7fabac0d0a20 legacy=0x7fabac0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.578+0000 7fabb1200640 1 -- 172.21.15.67:0/1774493205 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac15cb10 con 0x7fabac0d0640 2024-05-30T16:54:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.578+0000 7fabb1c00640 1 --2- 172.21.15.67:0/1774493205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0b32c0 0x7fabac0b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.578+0000 7fabab400640 1 -- 172.21.15.67:0/1774493205 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158856867 0 0) 0x7fabac0d0060 con 0x7fabac0d0a20 2024-05-30T16:54:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.579+0000 7fabb1200640 1 --2- 172.21.15.67:0/1774493205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0640 0x7fabac15d5d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7faba00097a0 tx=0x7faba002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7caa64ab3aeea56 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.579+0000 7fabab400640 1 -- 172.21.15.67:0/1774493205 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0037070 con 0x7fabac0d0640 2024-05-30T16:54:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.579+0000 7fabab400640 1 -- 172.21.15.67:0/1774493205 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba0032440 con 0x7fabac0d0640 2024-05-30T16:54:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.579+0000 7fabab400640 1 -- 172.21.15.67:0/1774493205 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0032810 con 0x7fabac0d0640 2024-05-30T16:54:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.580+0000 7fabb3c00640 1 -- 172.21.15.67:0/1774493205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0640 msgr2=0x7fabac15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.580+0000 7fabb3c00640 1 --2- 172.21.15.67:0/1774493205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0640 0x7fabac15d5d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7faba00097a0 tx=0x7faba002a180 comp rx=0 tx=0).stop 2024-05-30T16:54:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.580+0000 7fabb3c00640 1 -- 172.21.15.67:0/1774493205 shutdown_connections 2024-05-30T16:54:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.580+0000 7fabb3c00640 1 --2- 172.21.15.67:0/1774493205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0b32c0 0x7fabac0b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.580+0000 7fabb3c00640 1 --2- 172.21.15.67:0/1774493205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0640 0x7fabac15d5d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.580+0000 7fabb3c00640 1 -- 172.21.15.67:0/1774493205 >> 172.21.15.67:0/1774493205 conn(0x7fabac06bd20 msgr2=0x7fabac06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.580+0000 7fabb3c00640 1 -- 172.21.15.67:0/1774493205 shutdown_connections 2024-05-30T16:54:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.580+0000 7fabb3c00640 1 -- 172.21.15.67:0/1774493205 wait complete. 2024-05-30T16:54:32.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.582+0000 7fabb3c00640 1 Processor -- start 2024-05-30T16:54:32.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.582+0000 7fabb3c00640 1 -- start start 2024-05-30T16:54:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.583+0000 7fabb3c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabac0b32c0 0x7fabac1456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.583+0000 7fabb3c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0d0640 0x7fabac153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.583+0000 7fabb3c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0a20 0x7fabac072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.583+0000 7fabb3c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fabac15cdd0 con 0x7fabac0d0a20 2024-05-30T16:54:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.583+0000 7fabb3c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fabac0ad8d0 con 0x7fabac0b32c0 2024-05-30T16:54:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.583+0000 7fabb3c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fabac15d0b0 con 0x7fabac0d0640 2024-05-30T16:54:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.583+0000 7fabb1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0a20 0x7fabac072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.583+0000 7fabb1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0a20 0x7fabac072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56718/0 (socket says 172.21.15.67:56718) 2024-05-30T16:54:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.583+0000 7fabb1c00640 1 -- 172.21.15.67:0/2879494102 learned_addr learned my addr 172.21.15.67:0/2879494102 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fababe00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0d0640 0x7fabac153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fabb1200640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabac0b32c0 0x7fabac1456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fabb1c00640 1 -- 172.21.15.67:0/2879494102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0d0640 msgr2=0x7fabac153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fabb1c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0d0640 0x7fabac153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fabb1c00640 1 -- 172.21.15.67:0/2879494102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabac0b32c0 msgr2=0x7fabac1456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fabb1c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabac0b32c0 0x7fabac1456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fabb1c00640 1 -- 172.21.15.67:0/2879494102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac153ce0 con 0x7fabac0d0a20 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fabb1200640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabac0b32c0 0x7fabac1456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fabb1c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0a20 0x7fabac072110 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faba400cef0 tx=0x7faba400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7faba8c00640 1 -- 172.21.15.67:0/2879494102 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba401a070 con 0x7fabac0d0a20 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7faba8c00640 1 -- 172.21.15.67:0/2879494102 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba400d860 con 0x7fabac0d0a20 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.584+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabac153fc0 con 0x7fabac0d0a20 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.585+0000 7faba8c00640 1 -- 172.21.15.67:0/2879494102 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba400dc50 con 0x7fabac0d0a20 2024-05-30T16:54:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.585+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabac154580 con 0x7fabac0d0a20 2024-05-30T16:54:32.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.586+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabac0689e0 con 0x7fabac0d0a20 2024-05-30T16:54:32.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.587+0000 7faba8c00640 1 -- 172.21.15.67:0/2879494102 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7faba401d070 con 0x7fabac0d0a20 2024-05-30T16:54:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.588+0000 7faba8c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab8c06d4f0 0x7fab8c06f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.589+0000 7faba8c00640 1 -- 172.21.15.67:0/2879494102 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7faba408adc0 con 0x7fabac0d0a20 2024-05-30T16:54:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.589+0000 7fabb1200640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab8c06d4f0 0x7fab8c06f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.590+0000 7fabb1200640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab8c06d4f0 0x7fab8c06f9b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faba0034000 tx=0x7faba002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:32.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.593+0000 7faba8c00640 1 -- 172.21.15.67:0/2879494102 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faba405ad00 con 0x7fabac0d0a20 2024-05-30T16:54:32.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.828+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabac072de0 con 0x7fab8c06d4f0 2024-05-30T16:54:32.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.849+0000 7faba8c00640 1 -- 172.21.15.67:0/2879494102 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7fabac072de0 con 0x7fab8c06d4f0 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sv1" 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone15506" 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone15507" 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone15502" 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone15505" 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone15503" 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone15501" 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone15504" 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:32.852 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:54:32.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab8c06d4f0 msgr2=0x7fab8c06f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab8c06d4f0 0x7fab8c06f9b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faba0034000 tx=0x7faba002e000 comp rx=0 tx=0).stop 2024-05-30T16:54:32.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0a20 msgr2=0x7fabac072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:32.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0a20 0x7fabac072110 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faba400cef0 tx=0x7faba400a710 comp rx=0 tx=0).stop 2024-05-30T16:54:32.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 shutdown_connections 2024-05-30T16:54:32.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab8c06d4f0 0x7fab8c06f9b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabac0d0a20 0x7fabac072110 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabac0d0640 0x7fabac153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 --2- 172.21.15.67:0/2879494102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabac0b32c0 0x7fabac1456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:32.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 >> 172.21.15.67:0/2879494102 conn(0x7fabac06bd20 msgr2=0x7fabac15f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:32.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.852+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 shutdown_connections 2024-05-30T16:54:32.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:32.853+0000 7fabb3c00640 1 -- 172.21.15.67:0/2879494102 wait complete. 2024-05-30T16:54:32.874 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-30T16:54:33.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.087+0000 7f441e600640 1 Processor -- start 2024-05-30T16:54:33.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.087+0000 7f441e600640 1 -- start start 2024-05-30T16:54:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.088+0000 7f441e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f441815b340 0x7f441806f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.088+0000 7f441e600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f441815c2c0 0x7f44180b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.088+0000 7f441e600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f441806fa90 con 0x7f441815b720 2024-05-30T16:54:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.088+0000 7f441e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f44180af9b0 con 0x7f441815b340 2024-05-30T16:54:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.088+0000 7f441e600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f44180aeda0 con 0x7f441815c2c0 2024-05-30T16:54:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.088+0000 7f4417400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f441815b340 0x7f441806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.088+0000 7f4417400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f441815b340 0x7f441806f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56734/0 (socket says 172.21.15.67:56734) 2024-05-30T16:54:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.088+0000 7f4417400640 1 -- 172.21.15.67:0/1116392452 learned_addr learned my addr 172.21.15.67:0/1116392452 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:33.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.089+0000 7f4417400640 1 -- 172.21.15.67:0/1116392452 >> v2:172.21.15.169:3300/0 conn(0x7f441815c2c0 msgr2=0x7f44180b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:33.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.089+0000 7f4417400640 1 --2- 172.21.15.67:0/1116392452 >> v2:172.21.15.169:3300/0 conn(0x7f441815c2c0 0x7f44180b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.089+0000 7f4417400640 1 -- 172.21.15.67:0/1116392452 >> v1:172.21.15.169:6789/0 conn(0x7f441815b720 legacy=0x7f441815bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:33.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.089+0000 7f4417400640 1 -- 172.21.15.67:0/1116392452 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44180af080 con 0x7f441815b340 2024-05-30T16:54:33.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.090+0000 7f4417400640 1 --2- 172.21.15.67:0/1116392452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815b340 0x7f441806f550 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f44180af920 tx=0x7f440802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4fac4cbe349b92 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:33.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.090+0000 7f4416a00640 1 -- 172.21.15.67:0/1116392452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408037070 con 0x7f441815b340 2024-05-30T16:54:33.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.090+0000 7f4416a00640 1 -- 172.21.15.67:0/1116392452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4408032440 con 0x7f441815b340 2024-05-30T16:54:33.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.090+0000 7f4416a00640 1 -- 172.21.15.67:0/1116392452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408032810 con 0x7f441815b340 2024-05-30T16:54:33.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.091+0000 7f441e600640 1 -- 172.21.15.67:0/1116392452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815b340 msgr2=0x7f441806f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.091+0000 7f441e600640 1 --2- 172.21.15.67:0/1116392452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815b340 0x7f441806f550 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f44180af920 tx=0x7f440802a230 comp rx=0 tx=0).stop 2024-05-30T16:54:33.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.091+0000 7f441e600640 1 -- 172.21.15.67:0/1116392452 shutdown_connections 2024-05-30T16:54:33.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.091+0000 7f441e600640 1 --2- 172.21.15.67:0/1116392452 >> v2:172.21.15.169:3300/0 conn(0x7f441815c2c0 0x7f44180b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.091+0000 7f441e600640 1 --2- 172.21.15.67:0/1116392452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815b340 0x7f441806f550 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.091+0000 7f441e600640 1 -- 172.21.15.67:0/1116392452 >> 172.21.15.67:0/1116392452 conn(0x7f44180b1e50 msgr2=0x7f44180b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:33.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.091+0000 7f441e600640 1 -- 172.21.15.67:0/1116392452 shutdown_connections 2024-05-30T16:54:33.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.091+0000 7f441e600640 1 -- 172.21.15.67:0/1116392452 wait complete. 2024-05-30T16:54:33.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.093+0000 7f441e600640 1 Processor -- start 2024-05-30T16:54:33.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.093+0000 7f441e600640 1 -- start start 2024-05-30T16:54:33.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f441e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441815b340 0x7f4418098dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f441e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f441815b720 0x7f44181467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f441e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815c2c0 0x7f4418146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f441e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f44180aea30 con 0x7f441815c2c0 2024-05-30T16:54:33.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f441e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44180af640 con 0x7f441815b340 2024-05-30T16:54:33.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f441e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f441815c8c0 con 0x7f441815b720 2024-05-30T16:54:33.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f4417e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815c2c0 0x7f4418146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f4417e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815c2c0 0x7f4418146d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56744/0 (socket says 172.21.15.67:56744) 2024-05-30T16:54:33.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f4417e00640 1 -- 172.21.15.67:0/3889283280 learned_addr learned my addr 172.21.15.67:0/3889283280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:33.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f4417400640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441815b340 0x7f4418098dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.094+0000 7f440ea00640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f441815b720 0x7f44181467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f4417e00640 1 -- 172.21.15.67:0/3889283280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f441815b720 msgr2=0x7f44181467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f4417e00640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f441815b720 0x7f44181467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f4417e00640 1 -- 172.21.15.67:0/3889283280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441815b340 msgr2=0x7f4418098dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f4417e00640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441815b340 0x7f4418098dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f4417e00640 1 -- 172.21.15.67:0/3889283280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44181475f0 con 0x7f441815c2c0 2024-05-30T16:54:33.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f440ea00640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f441815b720 0x7f44181467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:33.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f4417e00640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815c2c0 0x7f4418146d00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f441000b9a0 tx=0x7f441000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:33.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f440fe00640 1 -- 172.21.15.67:0/3889283280 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410019070 con 0x7f441815c2c0 2024-05-30T16:54:33.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f440fe00640 1 -- 172.21.15.67:0/3889283280 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f441000cbc0 con 0x7f441815c2c0 2024-05-30T16:54:33.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44181478d0 con 0x7f441815c2c0 2024-05-30T16:54:33.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f440fe00640 1 -- 172.21.15.67:0/3889283280 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410014440 con 0x7f441815c2c0 2024-05-30T16:54:33.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.095+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4418147bb0 con 0x7f441815c2c0 2024-05-30T16:54:33.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.097+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44180689e0 con 0x7f441815c2c0 2024-05-30T16:54:33.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.098+0000 7f440fe00640 1 -- 172.21.15.67:0/3889283280 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7f441001c070 con 0x7f441815c2c0 2024-05-30T16:54:33.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.100+0000 7f440fe00640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43f406d4f0 0x7f43f406f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.100+0000 7f440fe00640 1 -- 172.21.15.67:0/3889283280 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f441008a540 con 0x7f441815c2c0 2024-05-30T16:54:33.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.100+0000 7f4417400640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43f406d4f0 0x7f43f406f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.101+0000 7f4417400640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43f406d4f0 0x7f43f406f9b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4408034000 tx=0x7f440802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:33.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.104+0000 7f440fe00640 1 -- 172.21.15.67:0/3889283280 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f441005a160 con 0x7f441815c2c0 2024-05-30T16:54:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.347+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f441815bbb0 con 0x7f43f406d4f0 2024-05-30T16:54:33.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.376+0000 7f440fe00640 1 -- 172.21.15.67:0/3889283280 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f441815bbb0 con 0x7f43f406d4f0 2024-05-30T16:54:33.377 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:33.377 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:54:33.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:33.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1" 2024-05-30T16:54:33.378 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:33.378 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:54:33.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.378+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43f406d4f0 msgr2=0x7f43f406f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.378+0000 7f441e600640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43f406d4f0 0x7f43f406f9b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4408034000 tx=0x7f440802e000 comp rx=0 tx=0).stop 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.378+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815c2c0 msgr2=0x7f4418146d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.378+0000 7f441e600640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815c2c0 0x7f4418146d00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f441000b9a0 tx=0x7f441000be60 comp rx=0 tx=0).stop 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.379+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 shutdown_connections 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.379+0000 7f441e600640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43f406d4f0 0x7f43f406f9b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.379+0000 7f441e600640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f441815c2c0 0x7f4418146d00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.379+0000 7f441e600640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f441815b720 0x7f44181467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.379+0000 7f441e600640 1 --2- 172.21.15.67:0/3889283280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441815b340 0x7f4418098dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.379+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 >> 172.21.15.67:0/3889283280 conn(0x7f44180b1e50 msgr2=0x7f4418151210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.379+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 shutdown_connections 2024-05-30T16:54:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.379+0000 7f441e600640 1 -- 172.21.15.67:0/3889283280 wait complete. 2024-05-30T16:54:33.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-30T16:54:33.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.620+0000 7f27cfe00640 1 Processor -- start 2024-05-30T16:54:33.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.620+0000 7f27cfe00640 1 -- start start 2024-05-30T16:54:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.622+0000 7f27cfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f27c8161a90 0x7f27c806e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.622+0000 7f27cfe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c80b3d40 con 0x7f27c806e9b0 2024-05-30T16:54:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.622+0000 7f27cfe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c80b4020 con 0x7f27c8161e70 2024-05-30T16:54:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.622+0000 7f27cfe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f27c80b4300 con 0x7f27c8161a90 2024-05-30T16:54:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.622+0000 7f27cde00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f27c806e9b0 0x7f27c80ad2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52518/0 (socket says 172.21.15.67:52518) 2024-05-30T16:54:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.622+0000 7f27cde00640 1 -- 172.21.15.67:0/332113633 learned_addr learned my addr 172.21.15.67:0/332113633 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715602567 0 0) 0x7f27c80b3d40 con 0x7f27c806e9b0 2024-05-30T16:54:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27cd400640 1 --2- 172.21.15.67:0/332113633 >> v2:172.21.15.169:3300/0 conn(0x7f27c8161a90 0x7f27c806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27b0003520 con 0x7f27c806e9b0 2024-05-30T16:54:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 972811284 0 0) 0x7f27b0003520 con 0x7f27c806e9b0 2024-05-30T16:54:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 >> v2:172.21.15.169:3300/0 conn(0x7f27c8161a90 msgr2=0x7f27c806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 --2- 172.21.15.67:0/332113633 >> v2:172.21.15.169:3300/0 conn(0x7f27c8161a90 0x7f27c806e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 >> v1:172.21.15.169:6789/0 conn(0x7f27c8161e70 legacy=0x7f27c80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c806dc70 con 0x7f27c806e9b0 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f27c00019f0 con 0x7f27c806e9b0 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 >> v1:172.21.15.67:6789/0 conn(0x7f27c806e9b0 legacy=0x7f27c80ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.623+0000 7f27c7e00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27b0003a00 0x7f27b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27c7e00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27c7e00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0005210 0x7f27b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f27c80b4300 con 0x7f27b0004800 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27b000c160 con 0x7f27b0005210 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f27b000c360 con 0x7f27b0003a00 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cca00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cd400640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27b0003a00 0x7f27b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cde00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0005210 0x7f27b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cca00640 1 -- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27b0003a00 msgr2=0x7f27b0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cca00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27b0003a00 0x7f27b0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cca00640 1 -- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0005210 msgr2=0x7f27b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cca00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0005210 0x7f27b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cca00640 1 -- 172.21.15.67:0/332113633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b0003520 con 0x7f27b0004800 2024-05-30T16:54:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cde00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0005210 0x7f27b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cde00640 1 -- 172.21.15.67:0/332113633 reap_dead start 2024-05-30T16:54:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.624+0000 7f27cd400640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27b0003a00 0x7f27b0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:33.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.625+0000 7f27cca00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f27b4001720 tx=0x7f27b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f699bf6c7569a9f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:33.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.626+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4001370 con 0x7f27b0004800 2024-05-30T16:54:33.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.626+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b40052d0 con 0x7f27b0004800 2024-05-30T16:54:33.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.626+0000 7f27c7e00640 1 -- 172.21.15.67:0/332113633 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b402ab90 con 0x7f27b0004800 2024-05-30T16:54:33.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.626+0000 7f27cfe00640 1 -- 172.21.15.67:0/332113633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27b0004800 msgr2=0x7f27b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.626+0000 7f27cfe00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f27b4001720 tx=0x7f27b0005830 comp rx=0 tx=0).stop 2024-05-30T16:54:33.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.627+0000 7f27cfe00640 1 -- 172.21.15.67:0/332113633 shutdown_connections 2024-05-30T16:54:33.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.627+0000 7f27cfe00640 1 --2- 172.21.15.67:0/332113633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.627+0000 7f27cfe00640 1 -- 172.21.15.67:0/332113633 >> 172.21.15.67:0/332113633 conn(0x7f27c806bd20 msgr2=0x7f27c806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:33.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.627+0000 7f27cfe00640 1 -- 172.21.15.67:0/332113633 shutdown_connections 2024-05-30T16:54:33.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.627+0000 7f27cfe00640 1 -- 172.21.15.67:0/332113633 wait complete. 2024-05-30T16:54:33.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.629+0000 7f27cfe00640 1 Processor -- start 2024-05-30T16:54:33.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.629+0000 7f27cfe00640 1 -- start start 2024-05-30T16:54:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0004800 0x7f27c81f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27c814eaf0 0x7f27c8160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27c815c2d0 0x7f27c815c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cfe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f27c806e470 con 0x7f27c815c2d0 2024-05-30T16:54:33.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27c80b5020 con 0x7f27b0004800 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f27c806e5f0 con 0x7f27c814eaf0 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27c815c2d0 0x7f27c815c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27c815c2d0 0x7f27c815c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56768/0 (socket says 172.21.15.67:56768) 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cde00640 1 -- 172.21.15.67:0/3009724223 learned_addr learned my addr 172.21.15.67:0/3009724223 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cd400640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0004800 0x7f27c81f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cca00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27c814eaf0 0x7f27c8160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cde00640 1 -- 172.21.15.67:0/3009724223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27c814eaf0 msgr2=0x7f27c8160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cde00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27c814eaf0 0x7f27c8160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cde00640 1 -- 172.21.15.67:0/3009724223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0004800 msgr2=0x7f27c81f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cde00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0004800 0x7f27c81f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cde00640 1 -- 172.21.15.67:0/3009724223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c8160770 con 0x7f27c815c2d0 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.630+0000 7f27cd400640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0004800 0x7f27c81f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:33.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.631+0000 7f27cde00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27c815c2d0 0x7f27c815c750 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f27c8073570 tx=0x7f27c000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:33.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.631+0000 7f27c5600640 1 -- 172.21.15.67:0/3009724223 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0019070 con 0x7f27c815c2d0 2024-05-30T16:54:33.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.631+0000 7f27c5600640 1 -- 172.21.15.67:0/3009724223 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c0004d60 con 0x7f27c815c2d0 2024-05-30T16:54:33.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.631+0000 7f27c5600640 1 -- 172.21.15.67:0/3009724223 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c000bd70 con 0x7f27c815c2d0 2024-05-30T16:54:33.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.631+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c8159190 con 0x7f27c815c2d0 2024-05-30T16:54:33.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.631+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c809c160 con 0x7f27c815c2d0 2024-05-30T16:54:33.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.633+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c80b4790 con 0x7f27c815c2d0 2024-05-30T16:54:33.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.634+0000 7f27c5600640 1 -- 172.21.15.67:0/3009724223 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7f27c00070d0 con 0x7f27c815c2d0 2024-05-30T16:54:33.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.635+0000 7f27c5600640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f27a806d4f0 0x7f27a806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:33.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.635+0000 7f27c5600640 1 -- 172.21.15.67:0/3009724223 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f27c008a100 con 0x7f27c815c2d0 2024-05-30T16:54:33.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.635+0000 7f27cd400640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f27a806d4f0 0x7f27a806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:33.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.636+0000 7f27cd400640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f27a806d4f0 0x7f27a806f9b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f27c8069c40 tx=0x7f27bc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:33.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.640+0000 7f27c5600640 1 -- 172.21.15.67:0/3009724223 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27c80b4790 con 0x7f27c815c2d0 2024-05-30T16:54:33.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.884+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27c806e190 con 0x7f27a806d4f0 2024-05-30T16:54:33.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.967+0000 7f27c5600640 1 -- 172.21.15.67:0/3009724223 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f27c806e190 con 0x7f27a806d4f0 2024-05-30T16:54:33.968 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:33.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f27a806d4f0 msgr2=0x7f27a806f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f27a806d4f0 0x7f27a806f9b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f27c8069c40 tx=0x7f27bc008040 comp rx=0 tx=0).stop 2024-05-30T16:54:33.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27c815c2d0 msgr2=0x7f27c815c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:33.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27c815c2d0 0x7f27c815c750 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f27c8073570 tx=0x7f27c000b4a0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 shutdown_connections 2024-05-30T16:54:33.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f27a806d4f0 0x7f27a806f9b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27c815c2d0 0x7f27c815c750 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27c814eaf0 0x7f27c8160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 --2- 172.21.15.67:0/3009724223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0004800 0x7f27c81f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.971+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 >> 172.21.15.67:0/3009724223 conn(0x7f27c806bd20 msgr2=0x7f27c80b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.972+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 shutdown_connections 2024-05-30T16:54:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:33.972+0000 7f27cfe00640 1 -- 172.21.15.67:0/3009724223 wait complete. 2024-05-30T16:54:33.995 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-30T16:54:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.201+0000 7f4217e00640 1 Processor -- start 2024-05-30T16:54:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.201+0000 7f4217e00640 1 -- start start 2024-05-30T16:54:34.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.203+0000 7f4217e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42180cae20 0x7f4218151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.203+0000 7f4217e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42180b3e80 con 0x7f42180caa40 2024-05-30T16:54:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.203+0000 7f4217e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42180cde80 con 0x7f42181522f0 2024-05-30T16:54:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.203+0000 7f4217e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f42180d0b30 con 0x7f42180cae20 2024-05-30T16:54:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.203+0000 7f4216a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f42180caa40 0x7f42180ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52534/0 (socket says 172.21.15.67:52534) 2024-05-30T16:54:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.203+0000 7f4216a00640 1 -- 172.21.15.67:0/1393257942 learned_addr learned my addr 172.21.15.67:0/1393257942 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.204+0000 7f420e000640 1 --2- 172.21.15.67:0/1393257942 >> v2:172.21.15.67:3300/0 conn(0x7f42180cae20 0x7f4218151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.204+0000 7f4216000640 1 -- 172.21.15.67:0/1393257942 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1102177705 0 0) 0x7f42180b3e80 con 0x7f42180caa40 2024-05-30T16:54:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.204+0000 7f4216000640 1 -- 172.21.15.67:0/1393257942 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41fc003520 con 0x7f42180caa40 2024-05-30T16:54:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.204+0000 7f420e000640 1 -- 172.21.15.67:0/1393257942 >> v1:172.21.15.169:6789/0 conn(0x7f42181522f0 legacy=0x7f42180cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.204+0000 7f420e000640 1 -- 172.21.15.67:0/1393257942 >> v1:172.21.15.67:6789/0 conn(0x7f42180caa40 legacy=0x7f42180ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.204+0000 7f420e000640 1 -- 172.21.15.67:0/1393257942 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42180ce160 con 0x7f42180cae20 2024-05-30T16:54:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.205+0000 7f420e000640 1 --2- 172.21.15.67:0/1393257942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 0x7f4218151db0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f42000097a0 tx=0x7f420002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10b2fe066bf81c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4216000640 1 -- 172.21.15.67:0/1393257942 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200037070 con 0x7f42180cae20 2024-05-30T16:54:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4216000640 1 -- 172.21.15.67:0/1393257942 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420002ae50 con 0x7f42180cae20 2024-05-30T16:54:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4216000640 1 -- 172.21.15.67:0/1393257942 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200032660 con 0x7f42180cae20 2024-05-30T16:54:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4217e00640 1 -- 172.21.15.67:0/1393257942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 msgr2=0x7f4218151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4217e00640 1 --2- 172.21.15.67:0/1393257942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 0x7f4218151db0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f42000097a0 tx=0x7f420002a140 comp rx=0 tx=0).stop 2024-05-30T16:54:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4217e00640 1 -- 172.21.15.67:0/1393257942 shutdown_connections 2024-05-30T16:54:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4217e00640 1 --2- 172.21.15.67:0/1393257942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 0x7f4218151db0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4217e00640 1 -- 172.21.15.67:0/1393257942 >> 172.21.15.67:0/1393257942 conn(0x7f421806b900 msgr2=0x7f421806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4217e00640 1 -- 172.21.15.67:0/1393257942 shutdown_connections 2024-05-30T16:54:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.206+0000 7f4217e00640 1 -- 172.21.15.67:0/1393257942 wait complete. 2024-05-30T16:54:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.208+0000 7f4217e00640 1 Processor -- start 2024-05-30T16:54:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.208+0000 7f4217e00640 1 -- start start 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.209+0000 7f4217e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42180caa40 0x7f421814a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.209+0000 7f4217e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 0x7f4218158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.209+0000 7f4217e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42181522f0 0x7f4218154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.209+0000 7f4217e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f42180cf070 con 0x7f42180cae20 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.209+0000 7f4217e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42180b4160 con 0x7f42181522f0 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.209+0000 7f4217e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4218152710 con 0x7f42180caa40 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f420e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 0x7f4218158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f420e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 0x7f4218158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56784/0 (socket says 172.21.15.67:56784) 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f420e000640 1 -- 172.21.15.67:0/1261054811 learned_addr learned my addr 172.21.15.67:0/1261054811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f4216a00640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42180caa40 0x7f421814a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f4217400640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42181522f0 0x7f4218154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f420e000640 1 -- 172.21.15.67:0/1261054811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42180caa40 msgr2=0x7f421814a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f420e000640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42180caa40 0x7f421814a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f420e000640 1 -- 172.21.15.67:0/1261054811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42181522f0 msgr2=0x7f4218154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f420e000640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42181522f0 0x7f4218154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.210+0000 7f420e000640 1 -- 172.21.15.67:0/1261054811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42180728e0 con 0x7f42180cae20 2024-05-30T16:54:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.211+0000 7f420e000640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 0x7f4218158690 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f42000098c0 tx=0x7f42000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.211+0000 7f420f400640 1 -- 172.21.15.67:0/1261054811 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200041020 con 0x7f42180cae20 2024-05-30T16:54:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.211+0000 7f420f400640 1 -- 172.21.15.67:0/1261054811 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420003c670 con 0x7f42180cae20 2024-05-30T16:54:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.211+0000 7f420f400640 1 -- 172.21.15.67:0/1261054811 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200037040 con 0x7f42180cae20 2024-05-30T16:54:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.211+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4218072bc0 con 0x7f42180cae20 2024-05-30T16:54:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.211+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4218073180 con 0x7f42180cae20 2024-05-30T16:54:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.212+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42180689e0 con 0x7f42180cae20 2024-05-30T16:54:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.213+0000 7f420f400640 1 -- 172.21.15.67:0/1261054811 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7f4200044020 con 0x7f42180cae20 2024-05-30T16:54:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.215+0000 7f420f400640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f41f406d4c0 0x7f41f406f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.215+0000 7f420f400640 1 -- 172.21.15.67:0/1261054811 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f42000aadc0 con 0x7f42180cae20 2024-05-30T16:54:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.215+0000 7f4216a00640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f41f406d4c0 0x7f41f406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.216+0000 7f4216a00640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f41f406d4c0 0x7f41f406f980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4208005880 tx=0x7f4208009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:34.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.219+0000 7f420f400640 1 -- 172.21.15.67:0/1261054811 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f420004b030 con 0x7f42180cae20 2024-05-30T16:54:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.454+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f42180b4ee0 con 0x7f41f406d4c0 2024-05-30T16:54:34.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.507+0000 7f420f400640 1 -- 172.21.15.67:0/1261054811 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42180b4ee0 con 0x7f41f406d4c0 2024-05-30T16:54:34.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f41f406d4c0 msgr2=0x7f41f406f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:34.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f41f406d4c0 0x7f41f406f980 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4208005880 tx=0x7f4208009040 comp rx=0 tx=0).stop 2024-05-30T16:54:34.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 msgr2=0x7f4218158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:34.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 0x7f4218158690 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f42000098c0 tx=0x7f42000046c0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 shutdown_connections 2024-05-30T16:54:34.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f41f406d4c0 0x7f41f406f980 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42181522f0 0x7f4218154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42180cae20 0x7f4218158690 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 --2- 172.21.15.67:0/1261054811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42180caa40 0x7f421814a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.511+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 >> 172.21.15.67:0/1261054811 conn(0x7f421806b900 msgr2=0x7f42180ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:34.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.512+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 shutdown_connections 2024-05-30T16:54:34.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.512+0000 7f4217e00640 1 -- 172.21.15.67:0/1261054811 wait complete. 2024-05-30T16:54:34.515 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.515+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.515 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.515+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.516 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.515+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.516 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.515+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.516 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.515+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15506 2024-05-30T16:54:34.547 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.546+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.547 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.546+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.547 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.546+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.547 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.547+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.547 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:54:34.547+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:54:34.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.725+0000 7f8ac2000640 1 Processor -- start 2024-05-30T16:54:34.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.725+0000 7f8ac2000640 1 -- start start 2024-05-30T16:54:34.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.727+0000 7f8ac2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8abc145fb0 0x7f8abc0b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.727+0000 7f8ac2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc15ccb0 0x7f8abc0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.727+0000 7f8ac2000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8abc0b0db0 con 0x7f8abc160280 2024-05-30T16:54:34.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.727+0000 7f8ac2000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f8abc146390 con 0x7f8abc145fb0 2024-05-30T16:54:34.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.727+0000 7f8ac2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc0ae760 con 0x7f8abc15ccb0 2024-05-30T16:54:34.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.727+0000 7f8abb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8abc145fb0 0x7f8abc0b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.727+0000 7f8abb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8abc145fb0 0x7f8abc0b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56848/0 (socket says 172.21.15.67:56848) 2024-05-30T16:54:34.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.727+0000 7f8abb400640 1 -- 172.21.15.67:0/2531245610 learned_addr learned my addr 172.21.15.67:0/2531245610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:34.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.728+0000 7f8abaa00640 1 --2- 172.21.15.67:0/2531245610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc15ccb0 0x7f8abc0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.728+0000 7f8abb400640 1 -- 172.21.15.67:0/2531245610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc15ccb0 msgr2=0x7f8abc0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:34.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.728+0000 7f8abb400640 1 --2- 172.21.15.67:0/2531245610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc15ccb0 0x7f8abc0ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.728+0000 7f8abb400640 1 -- 172.21.15.67:0/2531245610 >> v1:172.21.15.169:6789/0 conn(0x7f8abc160280 legacy=0x7f8abc06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:34.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.728+0000 7f8abb400640 1 -- 172.21.15.67:0/2531245610 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc0b1050 con 0x7f8abc145fb0 2024-05-30T16:54:34.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.729+0000 7f8abb400640 1 --2- 172.21.15.67:0/2531245610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc145fb0 0x7f8abc0b0650 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009e50 tx=0x7f8aac02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e6e12b57f8c175 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:34.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.729+0000 7f8aba000640 1 -- 172.21.15.67:0/2531245610 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac036070 con 0x7f8abc145fb0 2024-05-30T16:54:34.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.729+0000 7f8aba000640 1 -- 172.21.15.67:0/2531245610 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aac031600 con 0x7f8abc145fb0 2024-05-30T16:54:34.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.729+0000 7f8aba000640 1 -- 172.21.15.67:0/2531245610 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0319d0 con 0x7f8abc145fb0 2024-05-30T16:54:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.730+0000 7f8ac2000640 1 -- 172.21.15.67:0/2531245610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc145fb0 msgr2=0x7f8abc0b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.730+0000 7f8ac2000640 1 --2- 172.21.15.67:0/2531245610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc145fb0 0x7f8abc0b0650 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009e50 tx=0x7f8aac02a6c0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.730+0000 7f8ac2000640 1 -- 172.21.15.67:0/2531245610 shutdown_connections 2024-05-30T16:54:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.730+0000 7f8ac2000640 1 --2- 172.21.15.67:0/2531245610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc15ccb0 0x7f8abc0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.730+0000 7f8ac2000640 1 --2- 172.21.15.67:0/2531245610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc145fb0 0x7f8abc0b0650 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.730+0000 7f8ac2000640 1 -- 172.21.15.67:0/2531245610 >> 172.21.15.67:0/2531245610 conn(0x7f8abc0b6760 msgr2=0x7f8abc0b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.730+0000 7f8ac2000640 1 -- 172.21.15.67:0/2531245610 shutdown_connections 2024-05-30T16:54:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.730+0000 7f8ac2000640 1 -- 172.21.15.67:0/2531245610 wait complete. 2024-05-30T16:54:34.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.732+0000 7f8ac2000640 1 Processor -- start 2024-05-30T16:54:34.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.732+0000 7f8ac2000640 1 -- start start 2024-05-30T16:54:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.733+0000 7f8ac2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc145fb0 0x7f8abc0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.733+0000 7f8ac2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc15ccb0 0x7f8abc12d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.733+0000 7f8ac2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc160280 0x7f8abc12dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.733+0000 7f8ac2000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0b18f0 con 0x7f8abc160280 2024-05-30T16:54:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.733+0000 7f8ac2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0b1610 con 0x7f8abc15ccb0 2024-05-30T16:54:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.733+0000 7f8ac2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc160710 con 0x7f8abc145fb0 2024-05-30T16:54:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.733+0000 7f8abbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc160280 0x7f8abc12dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.733+0000 7f8abbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc160280 0x7f8abc12dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56850/0 (socket says 172.21.15.67:56850) 2024-05-30T16:54:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.733+0000 7f8abbe00640 1 -- 172.21.15.67:0/879558127 learned_addr learned my addr 172.21.15.67:0/879558127 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8abb400640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc145fb0 0x7f8abc0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8abbe00640 1 -- 172.21.15.67:0/879558127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc145fb0 msgr2=0x7f8abc0b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8abbe00640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc145fb0 0x7f8abc0b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8abbe00640 1 -- 172.21.15.67:0/879558127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc15ccb0 msgr2=0x7f8abc12d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8abbe00640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc15ccb0 0x7f8abc12d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:34.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8abbe00640 1 -- 172.21.15.67:0/879558127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc09c9c0 con 0x7f8abc160280 2024-05-30T16:54:34.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8abbe00640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc160280 0x7f8abc12dd40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400b410 tx=0x7f8ab400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:34.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8ab3400640 1 -- 172.21.15.67:0/879558127 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4015030 con 0x7f8abc160280 2024-05-30T16:54:34.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8ab3400640 1 -- 172.21.15.67:0/879558127 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab4004020 con 0x7f8abc160280 2024-05-30T16:54:34.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8ab3400640 1 -- 172.21.15.67:0/879558127 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4004410 con 0x7f8abc160280 2024-05-30T16:54:34.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.734+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc09cca0 con 0x7f8abc160280 2024-05-30T16:54:34.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.735+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc09dc10 con 0x7f8abc160280 2024-05-30T16:54:34.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.736+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc09de10 con 0x7f8abc160280 2024-05-30T16:54:34.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.737+0000 7f8ab3400640 1 -- 172.21.15.67:0/879558127 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7f8ab4018070 con 0x7f8abc160280 2024-05-30T16:54:34.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.738+0000 7f8ab3400640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a9c06d4f0 0x7f8a9c06f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:34.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.738+0000 7f8ab3400640 1 -- 172.21.15.67:0/879558127 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f8ab408a2c0 con 0x7f8abc160280 2024-05-30T16:54:34.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.738+0000 7f8abb400640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a9c06d4f0 0x7f8a9c06f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:34.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.739+0000 7f8abb400640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a9c06d4f0 0x7f8a9c06f9b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8aac033000 tx=0x7f8aac009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.742+0000 7f8ab3400640 1 -- 172.21.15.67:0/879558127 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8abc09de10 con 0x7f8abc160280 2024-05-30T16:54:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:34.980+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15506", "target": ["mon-mgr", ""]}) -- 0x7f8abc09d720 con 0x7f8a9c06d4f0 2024-05-30T16:54:35.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.032+0000 7f8ab3400640 1 -- 172.21.15.67:0/879558127 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8abc09d720 con 0x7f8a9c06d4f0 2024-05-30T16:54:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a9c06d4f0 msgr2=0x7f8a9c06f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a9c06d4f0 0x7f8a9c06f9b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8aac033000 tx=0x7f8aac009000 comp rx=0 tx=0).stop 2024-05-30T16:54:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc160280 msgr2=0x7f8abc12dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc160280 0x7f8abc12dd40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400b410 tx=0x7f8ab400b8d0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 shutdown_connections 2024-05-30T16:54:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8a9c06d4f0 0x7f8a9c06f9b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8abc160280 0x7f8abc12dd40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc15ccb0 0x7f8abc12d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 --2- 172.21.15.67:0/879558127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8abc145fb0 0x7f8abc0b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.035+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 >> 172.21.15.67:0/879558127 conn(0x7f8abc0b6760 msgr2=0x7f8abc15d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:35.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.036+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 shutdown_connections 2024-05-30T16:54:35.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.036+0000 7f8ac2000640 1 -- 172.21.15.67:0/879558127 wait complete. 2024-05-30T16:54:35.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15507 2024-05-30T16:54:35.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.291+0000 7fef35a00640 1 Processor -- start 2024-05-30T16:54:35.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.291+0000 7fef35a00640 1 -- start start 2024-05-30T16:54:35.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.293+0000 7fef35a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef300cfb40 0x7fef300d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.293+0000 7fef35a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fef300cffd0 0x7fef300d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.293+0000 7fef35a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef300d2200 con 0x7fef3015e5a0 2024-05-30T16:54:35.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.293+0000 7fef35a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fef300d2b30 con 0x7fef300cffd0 2024-05-30T16:54:35.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.293+0000 7fef35a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fef300d24e0 con 0x7fef300cfb40 2024-05-30T16:54:35.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.293+0000 7fef2fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fef3015e5a0 0x7fef300d2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52536/0 (socket says 172.21.15.67:52536) 2024-05-30T16:54:35.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.293+0000 7fef2fe00640 1 -- 172.21.15.67:0/342933644 learned_addr learned my addr 172.21.15.67:0/342933644 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:35.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.293+0000 7fef35000640 1 --2- 172.21.15.67:0/342933644 >> v2:172.21.15.67:3300/0 conn(0x7fef300cffd0 0x7fef300d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.294+0000 7fef27e00640 1 --2- 172.21.15.67:0/342933644 >> v2:172.21.15.169:3300/0 conn(0x7fef300cfb40 0x7fef300d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.294+0000 7fef2f400640 1 -- 172.21.15.67:0/342933644 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399898288 0 0) 0x7fef300d2200 con 0x7fef3015e5a0 2024-05-30T16:54:35.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.294+0000 7fef2f400640 1 -- 172.21.15.67:0/342933644 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef14003520 con 0x7fef3015e5a0 2024-05-30T16:54:35.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.294+0000 7fef35000640 1 -- 172.21.15.67:0/342933644 >> v2:172.21.15.169:3300/0 conn(0x7fef300cfb40 msgr2=0x7fef300d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:35.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.294+0000 7fef35000640 1 --2- 172.21.15.67:0/342933644 >> v2:172.21.15.169:3300/0 conn(0x7fef300cfb40 0x7fef300d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.294+0000 7fef35000640 1 -- 172.21.15.67:0/342933644 >> v1:172.21.15.67:6789/0 conn(0x7fef3015e5a0 legacy=0x7fef300d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.294+0000 7fef35000640 1 -- 172.21.15.67:0/342933644 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef300d27c0 con 0x7fef300cffd0 2024-05-30T16:54:35.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.295+0000 7fef35000640 1 --2- 172.21.15.67:0/342933644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 0x7fef300d1250 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fef30073fa0 tx=0x7fef2802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7272621c1d466dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.295+0000 7fef2f400640 1 -- 172.21.15.67:0/342933644 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28007740 con 0x7fef300cffd0 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.295+0000 7fef2f400640 1 -- 172.21.15.67:0/342933644 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef280078e0 con 0x7fef300cffd0 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.295+0000 7fef2f400640 1 -- 172.21.15.67:0/342933644 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28007cb0 con 0x7fef300cffd0 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.296+0000 7fef35a00640 1 -- 172.21.15.67:0/342933644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 msgr2=0x7fef300d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.296+0000 7fef35a00640 1 --2- 172.21.15.67:0/342933644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 0x7fef300d1250 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fef30073fa0 tx=0x7fef2802a680 comp rx=0 tx=0).stop 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.296+0000 7fef35a00640 1 -- 172.21.15.67:0/342933644 shutdown_connections 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.296+0000 7fef35a00640 1 --2- 172.21.15.67:0/342933644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 0x7fef300d1250 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.296+0000 7fef35a00640 1 --2- 172.21.15.67:0/342933644 >> v2:172.21.15.169:3300/0 conn(0x7fef300cfb40 0x7fef300d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.296+0000 7fef35a00640 1 -- 172.21.15.67:0/342933644 >> 172.21.15.67:0/342933644 conn(0x7fef3008bfc0 msgr2=0x7fef300d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.296+0000 7fef35a00640 1 -- 172.21.15.67:0/342933644 shutdown_connections 2024-05-30T16:54:35.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.296+0000 7fef35a00640 1 -- 172.21.15.67:0/342933644 wait complete. 2024-05-30T16:54:35.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.298+0000 7fef35a00640 1 Processor -- start 2024-05-30T16:54:35.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.298+0000 7fef35a00640 1 -- start start 2024-05-30T16:54:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef35a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef300cfb40 0x7fef301f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef35a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 0x7fef30154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef35a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef3015e5a0 0x7fef300cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef35a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fef300d53a0 con 0x7fef300cffd0 2024-05-30T16:54:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef35a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef300d5220 con 0x7fef3015e5a0 2024-05-30T16:54:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef35a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fef3006c740 con 0x7fef300cfb40 2024-05-30T16:54:35.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 0x7fef30154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 0x7fef30154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56864/0 (socket says 172.21.15.67:56864) 2024-05-30T16:54:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef27e00640 1 -- 172.21.15.67:0/2573907439 learned_addr learned my addr 172.21.15.67:0/2573907439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef2fe00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef300cfb40 0x7fef301f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef27e00640 1 -- 172.21.15.67:0/2573907439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef300cfb40 msgr2=0x7fef301f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef27e00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef300cfb40 0x7fef301f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef27e00640 1 -- 172.21.15.67:0/2573907439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef3015e5a0 msgr2=0x7fef300cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef27e00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef3015e5a0 0x7fef300cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.300+0000 7fef27e00640 1 -- 172.21.15.67:0/2573907439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef301f87a0 con 0x7fef300cffd0 2024-05-30T16:54:35.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.301+0000 7fef27e00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 0x7fef30154a30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fef1800d800 tx=0x7fef1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.301+0000 7fef2cc00640 1 -- 172.21.15.67:0/2573907439 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1800b6c0 con 0x7fef300cffd0 2024-05-30T16:54:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.301+0000 7fef2cc00640 1 -- 172.21.15.67:0/2573907439 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef18019070 con 0x7fef300cffd0 2024-05-30T16:54:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.301+0000 7fef2cc00640 1 -- 172.21.15.67:0/2573907439 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1800ba80 con 0x7fef300cffd0 2024-05-30T16:54:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.301+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef301f4400 con 0x7fef300cffd0 2024-05-30T16:54:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.301+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef301f49c0 con 0x7fef300cffd0 2024-05-30T16:54:35.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.302+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef300689e0 con 0x7fef300cffd0 2024-05-30T16:54:35.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.304+0000 7fef2cc00640 1 -- 172.21.15.67:0/2573907439 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7fef1800bc20 con 0x7fef300cffd0 2024-05-30T16:54:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.305+0000 7fef2cc00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0c06d4c0 0x7fef0c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.305+0000 7fef2cc00640 1 -- 172.21.15.67:0/2573907439 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fef1808ac50 con 0x7fef300cffd0 2024-05-30T16:54:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.305+0000 7fef2fe00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0c06d4c0 0x7fef0c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.306+0000 7fef2fe00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0c06d4c0 0x7fef0c06f980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fef20000f60 tx=0x7fef20009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.310+0000 7fef2cc00640 1 -- 172.21.15.67:0/2573907439 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef301f87a0 con 0x7fef300cffd0 2024-05-30T16:54:35.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.543+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15507", "target": ["mon-mgr", ""]}) -- 0x7fef300d1d30 con 0x7fef0c06d4c0 2024-05-30T16:54:35.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.598+0000 7fef2cc00640 1 -- 172.21.15.67:0/2573907439 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fef300d1d30 con 0x7fef0c06d4c0 2024-05-30T16:54:35.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0c06d4c0 msgr2=0x7fef0c06f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0c06d4c0 0x7fef0c06f980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fef20000f60 tx=0x7fef20009040 comp rx=0 tx=0).stop 2024-05-30T16:54:35.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 msgr2=0x7fef30154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 0x7fef30154a30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fef1800d800 tx=0x7fef1800dcc0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 shutdown_connections 2024-05-30T16:54:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0c06d4c0 0x7fef0c06f980 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef3015e5a0 0x7fef300cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef300cffd0 0x7fef30154a30 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 --2- 172.21.15.67:0/2573907439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef300cfb40 0x7fef301f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 >> 172.21.15.67:0/2573907439 conn(0x7fef3008bfc0 msgr2=0x7fef3006ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 shutdown_connections 2024-05-30T16:54:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.602+0000 7fef35a00640 1 -- 172.21.15.67:0/2573907439 wait complete. 2024-05-30T16:54:35.625 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15502 2024-05-30T16:54:35.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.831+0000 7f7e85e00640 1 Processor -- start 2024-05-30T16:54:35.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.831+0000 7f7e85e00640 1 -- start start 2024-05-30T16:54:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.833+0000 7f7e85e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e8015fa00 0x7f7e8006e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.833+0000 7f7e85e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7e800d0b40 0x7f7e80072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.833+0000 7f7e85e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8006fa00 con 0x7f7e801600f0 2024-05-30T16:54:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.833+0000 7f7e85e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7e8006eeb0 con 0x7f7e800d0b40 2024-05-30T16:54:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.833+0000 7f7e85e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7e8006f160 con 0x7f7e8015fa00 2024-05-30T16:54:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.833+0000 7f7e7fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7e801600f0 0x7f7e800d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52546/0 (socket says 172.21.15.67:52546) 2024-05-30T16:54:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.834+0000 7f7e7fe00640 1 -- 172.21.15.67:0/3174001760 learned_addr learned my addr 172.21.15.67:0/3174001760 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.834+0000 7f7e85400640 1 --2- 172.21.15.67:0/3174001760 >> v2:172.21.15.67:3300/0 conn(0x7f7e800d0b40 0x7f7e80072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.834+0000 7f7e84a00640 1 --2- 172.21.15.67:0/3174001760 >> v2:172.21.15.169:3300/0 conn(0x7f7e8015fa00 0x7f7e8006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.834+0000 7f7e7f400640 1 -- 172.21.15.67:0/3174001760 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27372031 0 0) 0x7f7e8006fa00 con 0x7f7e801600f0 2024-05-30T16:54:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.834+0000 7f7e7f400640 1 -- 172.21.15.67:0/3174001760 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e68003520 con 0x7f7e801600f0 2024-05-30T16:54:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.834+0000 7f7e85400640 1 -- 172.21.15.67:0/3174001760 >> v2:172.21.15.169:3300/0 conn(0x7f7e8015fa00 msgr2=0x7f7e8006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.834+0000 7f7e85400640 1 --2- 172.21.15.67:0/3174001760 >> v2:172.21.15.169:3300/0 conn(0x7f7e8015fa00 0x7f7e8006e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.834+0000 7f7e85400640 1 -- 172.21.15.67:0/3174001760 >> v1:172.21.15.67:6789/0 conn(0x7f7e801600f0 legacy=0x7f7e800d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.834+0000 7f7e85400640 1 -- 172.21.15.67:0/3174001760 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8006f440 con 0x7f7e800d0b40 2024-05-30T16:54:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.835+0000 7f7e85400640 1 --2- 172.21.15.67:0/3174001760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e800d0b40 0x7f7e80072190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7e800d0130 tx=0x7f7e7802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5190156bce9dcf7b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.835+0000 7f7e7f400640 1 -- 172.21.15.67:0/3174001760 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78007740 con 0x7f7e800d0b40 2024-05-30T16:54:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e7f400640 1 -- 172.21.15.67:0/3174001760 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e780078e0 con 0x7f7e800d0b40 2024-05-30T16:54:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e7f400640 1 -- 172.21.15.67:0/3174001760 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78007cb0 con 0x7f7e800d0b40 2024-05-30T16:54:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e85e00640 1 -- 172.21.15.67:0/3174001760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e800d0b40 msgr2=0x7f7e80072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e85e00640 1 --2- 172.21.15.67:0/3174001760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e800d0b40 0x7f7e80072190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7e800d0130 tx=0x7f7e7802a560 comp rx=0 tx=0).stop 2024-05-30T16:54:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e85e00640 1 -- 172.21.15.67:0/3174001760 shutdown_connections 2024-05-30T16:54:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e85e00640 1 --2- 172.21.15.67:0/3174001760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e800d0b40 0x7f7e80072190 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e85e00640 1 --2- 172.21.15.67:0/3174001760 >> v2:172.21.15.169:3300/0 conn(0x7f7e8015fa00 0x7f7e8006e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e85e00640 1 -- 172.21.15.67:0/3174001760 >> 172.21.15.67:0/3174001760 conn(0x7f7e8006bae0 msgr2=0x7f7e8006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e85e00640 1 -- 172.21.15.67:0/3174001760 shutdown_connections 2024-05-30T16:54:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.836+0000 7f7e85e00640 1 -- 172.21.15.67:0/3174001760 wait complete. 2024-05-30T16:54:35.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.838+0000 7f7e85e00640 1 Processor -- start 2024-05-30T16:54:35.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.838+0000 7f7e85e00640 1 -- start start 2024-05-30T16:54:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.839+0000 7f7e85e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800d0b40 0x7f7e8009dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.839+0000 7f7e85e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e8015fa00 0x7f7e800c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.839+0000 7f7e85e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e801600f0 0x7f7e801f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.839+0000 7f7e85e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8015fde0 con 0x7f7e8015fa00 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.839+0000 7f7e85e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80071c70 con 0x7f7e800d0b40 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.839+0000 7f7e85e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e800726d0 con 0x7f7e801600f0 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e7fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e8015fa00 0x7f7e800c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e7fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e8015fa00 0x7f7e800c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56872/0 (socket says 172.21.15.67:56872) 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e7fe00640 1 -- 172.21.15.67:0/431094424 learned_addr learned my addr 172.21.15.67:0/431094424 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e85400640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e801600f0 0x7f7e801f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e7fe00640 1 -- 172.21.15.67:0/431094424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e801600f0 msgr2=0x7f7e801f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e7fe00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e801600f0 0x7f7e801f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e84a00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800d0b40 0x7f7e8009dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e7fe00640 1 -- 172.21.15.67:0/431094424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800d0b40 msgr2=0x7f7e8009dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e7fe00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800d0b40 0x7f7e8009dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e7fe00640 1 -- 172.21.15.67:0/431094424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e801f6170 con 0x7f7e8015fa00 2024-05-30T16:54:35.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.840+0000 7f7e7fe00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e8015fa00 0x7f7e800c5130 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0010e0 tx=0x7f7e6c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.841+0000 7f7e7cc00640 1 -- 172.21.15.67:0/431094424 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c017070 con 0x7f7e8015fa00 2024-05-30T16:54:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.841+0000 7f7e7cc00640 1 -- 172.21.15.67:0/431094424 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e6c005d00 con 0x7f7e8015fa00 2024-05-30T16:54:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.841+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8014bff0 con 0x7f7e8015fa00 2024-05-30T16:54:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.841+0000 7f7e7cc00640 1 -- 172.21.15.67:0/431094424 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c007310 con 0x7f7e8015fa00 2024-05-30T16:54:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.841+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e801f5e40 con 0x7f7e8015fa00 2024-05-30T16:54:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.842+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e800689e0 con 0x7f7e8015fa00 2024-05-30T16:54:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.843+0000 7f7e7cc00640 1 -- 172.21.15.67:0/431094424 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7f7e6c01a070 con 0x7f7e8015fa00 2024-05-30T16:54:35.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.844+0000 7f7e7cc00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6006d4c0 0x7f7e6006f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.844+0000 7f7e7cc00640 1 -- 172.21.15.67:0/431094424 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f7e6c089b80 con 0x7f7e8015fa00 2024-05-30T16:54:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.844+0000 7f7e84a00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6006d4c0 0x7f7e6006f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.845+0000 7f7e84a00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6006d4c0 0x7f7e6006f980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e74006fd0 tx=0x7f7e74008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:35.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:35.848+0000 7f7e7cc00640 1 -- 172.21.15.67:0/431094424 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e6c0597a0 con 0x7f7e8015fa00 2024-05-30T16:54:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.080+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15502", "target": ["mon-mgr", ""]}) -- 0x7f7e801f6340 con 0x7f7e6006d4c0 2024-05-30T16:54:36.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.132+0000 7f7e7cc00640 1 -- 172.21.15.67:0/431094424 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e801f6340 con 0x7f7e6006d4c0 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6006d4c0 msgr2=0x7f7e6006f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6006d4c0 0x7f7e6006f980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e74006fd0 tx=0x7f7e74008040 comp rx=0 tx=0).stop 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e8015fa00 msgr2=0x7f7e800c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e8015fa00 0x7f7e800c5130 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0010e0 tx=0x7f7e6c00c890 comp rx=0 tx=0).stop 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 shutdown_connections 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6006d4c0 0x7f7e6006f980 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e801600f0 0x7f7e801f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e8015fa00 0x7f7e800c5130 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 --2- 172.21.15.67:0/431094424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e800d0b40 0x7f7e8009dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.135+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 >> 172.21.15.67:0/431094424 conn(0x7f7e8006bae0 msgr2=0x7f7e800cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:36.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.136+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 shutdown_connections 2024-05-30T16:54:36.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.136+0000 7f7e85e00640 1 -- 172.21.15.67:0/431094424 wait complete. 2024-05-30T16:54:36.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15505 2024-05-30T16:54:36.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.379+0000 7f5d4f200640 1 Processor -- start 2024-05-30T16:54:36.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.379+0000 7f5d4f200640 1 -- start start 2024-05-30T16:54:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.381+0000 7f5d4f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d500d03e0 0x7f5d50162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.381+0000 7f5d4f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d500cf620 con 0x7f5d5015e5e0 2024-05-30T16:54:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.381+0000 7f5d4f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d500cf900 con 0x7f5d500d3270 2024-05-30T16:54:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.381+0000 7f5d4f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5d500cfbe0 con 0x7f5d500d03e0 2024-05-30T16:54:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.381+0000 7f5d4de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5d5015e5e0 0x7f5d500d2b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52554/0 (socket says 172.21.15.67:52554) 2024-05-30T16:54:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.381+0000 7f5d4de00640 1 -- 172.21.15.67:0/3944234012 learned_addr learned my addr 172.21.15.67:0/3944234012 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:36.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4e800640 1 --2- 172.21.15.67:0/3944234012 >> v2:172.21.15.169:3300/0 conn(0x7f5d500d03e0 0x7f5d50162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1479663528 0 0) 0x7f5d500cf620 con 0x7f5d5015e5e0 2024-05-30T16:54:36.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d34003520 con 0x7f5d5015e5e0 2024-05-30T16:54:36.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1864276059 0 0) 0x7f5d34003520 con 0x7f5d5015e5e0 2024-05-30T16:54:36.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 >> v2:172.21.15.169:3300/0 conn(0x7f5d500d03e0 msgr2=0x7f5d50162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4d400640 1 --2- 172.21.15.67:0/3944234012 >> v2:172.21.15.169:3300/0 conn(0x7f5d500d03e0 0x7f5d50162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 >> v1:172.21.15.169:6789/0 conn(0x7f5d500d3270 legacy=0x7f5d50166710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d500cfec0 con 0x7f5d5015e5e0 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5d40001670 con 0x7f5d5015e5e0 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.382+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 >> v1:172.21.15.67:6789/0 conn(0x7f5d5015e5e0 legacy=0x7f5d500d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4d400640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d34003a00 0x7f5d34003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4d400640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d34004800 0x7f5d34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4d400640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d340051c0 0x7f5d3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d500cfbe0 con 0x7f5d34003a00 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3400c0c0 con 0x7f5d34004800 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3400c2c0 con 0x7f5d340051c0 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4de00640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d34003a00 0x7f5d34003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d47e00640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d34004800 0x7f5d34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4e800640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d340051c0 0x7f5d3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4de00640 1 -- 172.21.15.67:0/3944234012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d340051c0 msgr2=0x7f5d3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4de00640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d340051c0 0x7f5d3400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4de00640 1 -- 172.21.15.67:0/3944234012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d34004800 msgr2=0x7f5d34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4de00640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d34004800 0x7f5d34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4de00640 1 -- 172.21.15.67:0/3944234012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d34003520 con 0x7f5d34003a00 2024-05-30T16:54:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.383+0000 7f5d4e800640 1 -- 172.21.15.67:0/3944234012 reap_dead start 2024-05-30T16:54:36.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.384+0000 7f5d4de00640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d34003a00 0x7f5d34003e20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c0e0 tx=0x7f5d34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d17d262f98ccf4b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:36.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.385+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40007c20 con 0x7f5d34003a00 2024-05-30T16:54:36.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.385+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d40005af0 con 0x7f5d34003a00 2024-05-30T16:54:36.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.385+0000 7f5d4d400640 1 -- 172.21.15.67:0/3944234012 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40007070 con 0x7f5d34003a00 2024-05-30T16:54:36.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.385+0000 7f5d4f200640 1 -- 172.21.15.67:0/3944234012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d34003a00 msgr2=0x7f5d34003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.385+0000 7f5d4f200640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d34003a00 0x7f5d34003e20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c0e0 tx=0x7f5d34005830 comp rx=0 tx=0).stop 2024-05-30T16:54:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.385+0000 7f5d4f200640 1 -- 172.21.15.67:0/3944234012 shutdown_connections 2024-05-30T16:54:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.385+0000 7f5d4f200640 1 --2- 172.21.15.67:0/3944234012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d34003a00 0x7f5d34003e20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.385+0000 7f5d4f200640 1 -- 172.21.15.67:0/3944234012 >> 172.21.15.67:0/3944234012 conn(0x7f5d5008bfc0 msgr2=0x7f5d500d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.386+0000 7f5d4f200640 1 -- 172.21.15.67:0/3944234012 shutdown_connections 2024-05-30T16:54:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.386+0000 7f5d4f200640 1 -- 172.21.15.67:0/3944234012 wait complete. 2024-05-30T16:54:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.387+0000 7f5d4f200640 1 Processor -- start 2024-05-30T16:54:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.388+0000 7f5d4f200640 1 -- start start 2024-05-30T16:54:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.388+0000 7f5d4f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d34003a00 0x7f5d501f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.388+0000 7f5d4f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d500ca9a0 0x7f5d501525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.388+0000 7f5d4f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d50152dd0 0x7f5d500ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.388+0000 7f5d4f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5015eb40 con 0x7f5d500ca9a0 2024-05-30T16:54:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.388+0000 7f5d4f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5015e9c0 con 0x7f5d50152dd0 2024-05-30T16:54:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.388+0000 7f5d4f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5015ecc0 con 0x7f5d34003a00 2024-05-30T16:54:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d500ca9a0 0x7f5d501525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d500ca9a0 0x7f5d501525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56886/0 (socket says 172.21.15.67:56886) 2024-05-30T16:54:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d47e00640 1 -- 172.21.15.67:0/1919453820 learned_addr learned my addr 172.21.15.67:0/1919453820 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d4de00640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d34003a00 0x7f5d501f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d4e800640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d50152dd0 0x7f5d500ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d47e00640 1 -- 172.21.15.67:0/1919453820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d34003a00 msgr2=0x7f5d501f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d47e00640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d34003a00 0x7f5d501f85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d47e00640 1 -- 172.21.15.67:0/1919453820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d50152dd0 msgr2=0x7f5d500ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d47e00640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d50152dd0 0x7f5d500ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d47e00640 1 -- 172.21.15.67:0/1919453820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d501f8b30 con 0x7f5d500ca9a0 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.389+0000 7f5d4e800640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d50152dd0 0x7f5d500ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.390+0000 7f5d47e00640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d500ca9a0 0x7f5d501525b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e440 tx=0x7f5d3800e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.390+0000 7f5d46000640 1 -- 172.21.15.67:0/1919453820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38019070 con 0x7f5d500ca9a0 2024-05-30T16:54:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.390+0000 7f5d46000640 1 -- 172.21.15.67:0/1919453820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d380052e0 con 0x7f5d500ca9a0 2024-05-30T16:54:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.390+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5015f0f0 con 0x7f5d500ca9a0 2024-05-30T16:54:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.390+0000 7f5d46000640 1 -- 172.21.15.67:0/1919453820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38003830 con 0x7f5d500ca9a0 2024-05-30T16:54:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.390+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d501f80d0 con 0x7f5d500ca9a0 2024-05-30T16:54:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.391+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d500690f0 con 0x7f5d500ca9a0 2024-05-30T16:54:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.392+0000 7f5d46000640 1 -- 172.21.15.67:0/1919453820 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7f5d3801c070 con 0x7f5d500ca9a0 2024-05-30T16:54:36.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.393+0000 7f5d46000640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d2c06d4c0 0x7f5d2c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.393+0000 7f5d46000640 1 -- 172.21.15.67:0/1919453820 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f5d38089d00 con 0x7f5d500ca9a0 2024-05-30T16:54:36.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.394+0000 7f5d4de00640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d2c06d4c0 0x7f5d2c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.394+0000 7f5d4de00640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d2c06d4c0 0x7f5d2c06f980 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5d40032000 tx=0x7f5d40009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:36.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.398+0000 7f5d46000640 1 -- 172.21.15.67:0/1919453820 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d380592d0 con 0x7f5d500ca9a0 2024-05-30T16:54:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.633+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15505", "target": ["mon-mgr", ""]}) -- 0x7f5d5015eea0 con 0x7f5d2c06d4c0 2024-05-30T16:54:36.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.675+0000 7f5d46000640 1 -- 172.21.15.67:0/1919453820 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5d5015eea0 con 0x7f5d2c06d4c0 2024-05-30T16:54:36.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.677+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d2c06d4c0 msgr2=0x7f5d2c06f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.677+0000 7f5d4f200640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d2c06d4c0 0x7f5d2c06f980 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5d40032000 tx=0x7f5d40009030 comp rx=0 tx=0).stop 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.677+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d500ca9a0 msgr2=0x7f5d501525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.677+0000 7f5d4f200640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d500ca9a0 0x7f5d501525b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e440 tx=0x7f5d3800e840 comp rx=0 tx=0).stop 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.677+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 shutdown_connections 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.677+0000 7f5d4f200640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5d2c06d4c0 0x7f5d2c06f980 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.677+0000 7f5d4f200640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d50152dd0 0x7f5d500ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.677+0000 7f5d4f200640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d500ca9a0 0x7f5d501525b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.677+0000 7f5d4f200640 1 --2- 172.21.15.67:0/1919453820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d34003a00 0x7f5d501f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.678+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 >> 172.21.15.67:0/1919453820 conn(0x7f5d5008bfc0 msgr2=0x7f5d50162130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.678+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 shutdown_connections 2024-05-30T16:54:36.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.678+0000 7f5d4f200640 1 -- 172.21.15.67:0/1919453820 wait complete. 2024-05-30T16:54:36.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15503 2024-05-30T16:54:36.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.902+0000 7faeeca00640 1 Processor -- start 2024-05-30T16:54:36.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.902+0000 7faeeca00640 1 -- start start 2024-05-30T16:54:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.904+0000 7faeeca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faee815fa00 0x7faee806e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.904+0000 7faeeca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faee81600f0 0x7faee80d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.904+0000 7faeeca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee806fa00 con 0x7faee80d0970 2024-05-30T16:54:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.904+0000 7faeeca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7faee80d0dc0 con 0x7faee81600f0 2024-05-30T16:54:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.904+0000 7faeeca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faee806eeb0 con 0x7faee815fa00 2024-05-30T16:54:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.904+0000 7faedfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faee81600f0 0x7faee80d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.904+0000 7faedfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faee81600f0 0x7faee80d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56894/0 (socket says 172.21.15.67:56894) 2024-05-30T16:54:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.904+0000 7faedfe00640 1 -- 172.21.15.67:0/54989508 learned_addr learned my addr 172.21.15.67:0/54989508 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:36.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.905+0000 7faee7400640 1 --2- 172.21.15.67:0/54989508 >> v2:172.21.15.169:3300/0 conn(0x7faee815fa00 0x7faee806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.905+0000 7faedfe00640 1 -- 172.21.15.67:0/54989508 >> v2:172.21.15.169:3300/0 conn(0x7faee815fa00 msgr2=0x7faee806e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:36.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.905+0000 7faedfe00640 1 --2- 172.21.15.67:0/54989508 >> v2:172.21.15.169:3300/0 conn(0x7faee815fa00 0x7faee806e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.905+0000 7faedfe00640 1 -- 172.21.15.67:0/54989508 >> v1:172.21.15.169:6789/0 conn(0x7faee80d0970 legacy=0x7faee80b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:36.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.905+0000 7faedfe00640 1 -- 172.21.15.67:0/54989508 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee806f160 con 0x7faee81600f0 2024-05-30T16:54:36.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.905+0000 7faee7400640 1 --2- 172.21.15.67:0/54989508 >> v2:172.21.15.169:3300/0 conn(0x7faee815fa00 0x7faee806e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:36.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.906+0000 7faedfe00640 1 --2- 172.21.15.67:0/54989508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee81600f0 0x7faee80d0430 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faee806d980 tx=0x7faed002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af848a178efcb285 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.906+0000 7faee6a00640 1 -- 172.21.15.67:0/54989508 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0037070 con 0x7faee81600f0 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.906+0000 7faee6a00640 1 -- 172.21.15.67:0/54989508 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed0032440 con 0x7faee81600f0 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.906+0000 7faee6a00640 1 -- 172.21.15.67:0/54989508 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0032810 con 0x7faee81600f0 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.907+0000 7faeeca00640 1 -- 172.21.15.67:0/54989508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee81600f0 msgr2=0x7faee80d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.907+0000 7faeeca00640 1 --2- 172.21.15.67:0/54989508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee81600f0 0x7faee80d0430 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faee806d980 tx=0x7faed002a160 comp rx=0 tx=0).stop 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.907+0000 7faeeca00640 1 -- 172.21.15.67:0/54989508 shutdown_connections 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.907+0000 7faeeca00640 1 --2- 172.21.15.67:0/54989508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee81600f0 0x7faee80d0430 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.907+0000 7faeeca00640 1 --2- 172.21.15.67:0/54989508 >> v2:172.21.15.169:3300/0 conn(0x7faee815fa00 0x7faee806e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.907+0000 7faeeca00640 1 -- 172.21.15.67:0/54989508 >> 172.21.15.67:0/54989508 conn(0x7faee806bae0 msgr2=0x7faee806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.907+0000 7faeeca00640 1 -- 172.21.15.67:0/54989508 shutdown_connections 2024-05-30T16:54:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.907+0000 7faeeca00640 1 -- 172.21.15.67:0/54989508 wait complete. 2024-05-30T16:54:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.909+0000 7faeeca00640 1 Processor -- start 2024-05-30T16:54:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.909+0000 7faeeca00640 1 -- start start 2024-05-30T16:54:36.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.910+0000 7faeeca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80d0970 0x7faee814abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.910+0000 7faeeca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee815fa00 0x7faee8147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.910+0000 7faeeca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee81600f0 0x7faee809a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.910+0000 7faeeca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faee8071c70 con 0x7faee80d0970 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.910+0000 7faeeca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faee806f720 con 0x7faee81600f0 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.910+0000 7faeeca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faee815fde0 con 0x7faee815fa00 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.910+0000 7faee7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80d0970 0x7faee814abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.910+0000 7faee7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80d0970 0x7faee814abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56896/0 (socket says 172.21.15.67:56896) 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.910+0000 7faee7400640 1 -- 172.21.15.67:0/234587811 learned_addr learned my addr 172.21.15.67:0/234587811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faee7400640 1 -- 172.21.15.67:0/234587811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee815fa00 msgr2=0x7faee8147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faedfe00640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee815fa00 0x7faee8147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faee7400640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee815fa00 0x7faee8147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faee7400640 1 -- 172.21.15.67:0/234587811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee81600f0 msgr2=0x7faee809a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:36.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faee7400640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee81600f0 0x7faee809a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:36.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faee7400640 1 -- 172.21.15.67:0/234587811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee814b120 con 0x7faee80d0970 2024-05-30T16:54:36.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faee7400640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80d0970 0x7faee814abe0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7faed800c7f0 tx=0x7faed800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faedf400640 1 -- 172.21.15.67:0/234587811 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8015070 con 0x7faee80d0970 2024-05-30T16:54:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faedf400640 1 -- 172.21.15.67:0/234587811 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed8007820 con 0x7faee80d0970 2024-05-30T16:54:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee81f26a0 con 0x7faee80d0970 2024-05-30T16:54:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.911+0000 7faedf400640 1 -- 172.21.15.67:0/234587811 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8007bd0 con 0x7faee80d0970 2024-05-30T16:54:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.912+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee81f2ee0 con 0x7faee80d0970 2024-05-30T16:54:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.913+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee81f3220 con 0x7faee80d0970 2024-05-30T16:54:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.914+0000 7faedf400640 1 -- 172.21.15.67:0/234587811 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 23) ==== 92111+0+0 (secure 0 0 0) 0x7faed8018070 con 0x7faee80d0970 2024-05-30T16:54:36.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.914+0000 7faedf400640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faec406d4c0 0x7faec406f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:36.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.915+0000 7faedf400640 1 -- 172.21.15.67:0/234587811 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7faed808a4b0 con 0x7faee80d0970 2024-05-30T16:54:36.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.915+0000 7faedfe00640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faec406d4c0 0x7faec406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:36.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.916+0000 7faedfe00640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faec406d4c0 0x7faec406f980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faed0034000 tx=0x7faed002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:36.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:36.919+0000 7faedf400640 1 -- 172.21.15.67:0/234587811 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faee81f3220 con 0x7faee80d0970 2024-05-30T16:54:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.111+0000 7faedf400640 1 -- 172.21.15.67:0/234587811 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7faed80562a0 con 0x7faee80d0970 2024-05-30T16:54:37.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.147+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15503", "target": ["mon-mgr", ""]}) -- 0x7faee80689e0 con 0x7faec406d4c0 2024-05-30T16:54:37.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.204+0000 7faedf400640 1 -- 172.21.15.67:0/234587811 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faee80689e0 con 0x7faec406d4c0 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faec406d4c0 msgr2=0x7faec406f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faec406d4c0 0x7faec406f980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faed0034000 tx=0x7faed002e000 comp rx=0 tx=0).stop 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80d0970 msgr2=0x7faee814abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80d0970 0x7faee814abe0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7faed800c7f0 tx=0x7faed800ccb0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 shutdown_connections 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faec406d4c0 0x7faec406f980 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faee81600f0 0x7faee809a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee815fa00 0x7faee8147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 --2- 172.21.15.67:0/234587811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80d0970 0x7faee814abe0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 >> 172.21.15.67:0/234587811 conn(0x7faee806bae0 msgr2=0x7faee806bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:37.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 shutdown_connections 2024-05-30T16:54:37.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.207+0000 7faeeca00640 1 -- 172.21.15.67:0/234587811 wait complete. 2024-05-30T16:54:37.227 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15501 2024-05-30T16:54:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.441+0000 7fbbe7e00640 1 Processor -- start 2024-05-30T16:54:37.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.441+0000 7fbbe7e00640 1 -- start start 2024-05-30T16:54:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.443+0000 7fbbe7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe8162d20 0x7fbbe806e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.443+0000 7fbbe7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbbe80cf800 0x7fbbe80cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.443+0000 7fbbe7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbe80b49f0 con 0x7fbbe8162940 2024-05-30T16:54:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.443+0000 7fbbe7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbbe80b4cd0 con 0x7fbbe80cf800 2024-05-30T16:54:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.443+0000 7fbbe7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe80cf020 con 0x7fbbe8162d20 2024-05-30T16:54:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.443+0000 7fbbe6a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbbe8162940 0x7fbbe80b4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52560/0 (socket says 172.21.15.67:52560) 2024-05-30T16:54:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.443+0000 7fbbe6a00640 1 -- 172.21.15.67:0/2632584007 learned_addr learned my addr 172.21.15.67:0/2632584007 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.444+0000 7fbbe7400640 1 --2- 172.21.15.67:0/2632584007 >> v2:172.21.15.169:3300/0 conn(0x7fbbe80cf800 0x7fbbe80cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.444+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1636650116 0 0) 0x7fbbe80b49f0 con 0x7fbbe8162940 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.444+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbcc003520 con 0x7fbbe8162940 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4140690234 0 0) 0x7fbbcc003520 con 0x7fbbe8162940 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe8162d20 msgr2=0x7fbbe806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 --2- 172.21.15.67:0/2632584007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe8162d20 0x7fbbe806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 >> v2:172.21.15.169:3300/0 conn(0x7fbbe80cf800 msgr2=0x7fbbe80cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 --2- 172.21.15.67:0/2632584007 >> v2:172.21.15.169:3300/0 conn(0x7fbbe80cf800 0x7fbbe80cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe80cecd0 con 0x7fbbe8162940 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fbbd0001670 con 0x7fbbe8162940 2024-05-30T16:54:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 >> v1:172.21.15.67:6789/0 conn(0x7fbbe8162940 legacy=0x7fbbe80b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 --2- 172.21.15.67:0/2632584007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe8162d20 0x7fbbcc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 --2- 172.21.15.67:0/2632584007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbcc003fe0 0x7fbbcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 --2- 172.21.15.67:0/2632584007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbcc004d90 0x7fbbcc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80b4cd0 con 0x7fbbcc004d90 2024-05-30T16:54:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80cf020 con 0x7fbbcc003fe0 2024-05-30T16:54:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.445+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc00bcf0 con 0x7fbbe8162d20 2024-05-30T16:54:37.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.446+0000 7fbbe7400640 1 --2- 172.21.15.67:0/2632584007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbcc004d90 0x7fbbcc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:37.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.446+0000 7fbbe7400640 1 -- 172.21.15.67:0/2632584007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe8162d20 msgr2=0x7fbbcc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:37.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.446+0000 7fbbe7400640 1 --2- 172.21.15.67:0/2632584007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe8162d20 0x7fbbcc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.446+0000 7fbbe7400640 1 -- 172.21.15.67:0/2632584007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbcc003fe0 msgr2=0x7fbbcc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:37.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.446+0000 7fbbe7400640 1 --2- 172.21.15.67:0/2632584007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbcc003fe0 0x7fbbcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.446+0000 7fbbe7400640 1 -- 172.21.15.67:0/2632584007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe80cecd0 con 0x7fbbcc004d90 2024-05-30T16:54:37.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.447+0000 7fbbe7400640 1 --2- 172.21.15.67:0/2632584007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbcc004d90 0x7fbbcc005210 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0007360 tx=0x7fbbcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb7763122859b19 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:37.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.447+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0009be0 con 0x7fbbcc004d90 2024-05-30T16:54:37.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.447+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe002e620 con 0x7fbbcc004d90 2024-05-30T16:54:37.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.448+0000 7fbbe5600640 1 -- 172.21.15.67:0/2632584007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe002e9f0 con 0x7fbbcc004d90 2024-05-30T16:54:37.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.448+0000 7fbbe7e00640 1 -- 172.21.15.67:0/2632584007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbcc004d90 msgr2=0x7fbbcc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.448+0000 7fbbe7e00640 1 --2- 172.21.15.67:0/2632584007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbcc004d90 0x7fbbcc005210 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0007360 tx=0x7fbbcc005830 comp rx=0 tx=0).stop 2024-05-30T16:54:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.448+0000 7fbbe7400640 1 -- 172.21.15.67:0/2632584007 reap_dead start 2024-05-30T16:54:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.448+0000 7fbbe7e00640 1 -- 172.21.15.67:0/2632584007 shutdown_connections 2024-05-30T16:54:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.448+0000 7fbbe7e00640 1 -- 172.21.15.67:0/2632584007 >> 172.21.15.67:0/2632584007 conn(0x7fbbe806bae0 msgr2=0x7fbbe806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.449+0000 7fbbe7e00640 1 -- 172.21.15.67:0/2632584007 shutdown_connections 2024-05-30T16:54:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.449+0000 7fbbe7e00640 1 -- 172.21.15.67:0/2632584007 wait complete. 2024-05-30T16:54:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.451+0000 7fbbe7e00640 1 Processor -- start 2024-05-30T16:54:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.452+0000 7fbbe7e00640 1 -- start start 2024-05-30T16:54:37.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.453+0000 7fbbe7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbe81f8d00 0x7fbbe809a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:37.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.453+0000 7fbbe7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe809a7f0 0x7fbbe80cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:37.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.453+0000 7fbbe7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe80cbeb0 0x7fbbe81fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:37.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.453+0000 7fbbe7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe806e8f0 con 0x7fbbe81f8d00 2024-05-30T16:54:37.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.453+0000 7fbbe7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe806e770 con 0x7fbbe80cbeb0 2024-05-30T16:54:37.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.453+0000 7fbbe7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe8071b40 con 0x7fbbe809a7f0 2024-05-30T16:54:37.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.454+0000 7fbbe6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbe81f8d00 0x7fbbe809a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:37.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.454+0000 7fbbe6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbe81f8d00 0x7fbbe809a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56918/0 (socket says 172.21.15.67:56918) 2024-05-30T16:54:37.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.454+0000 7fbbe6a00640 1 -- 172.21.15.67:0/555496882 learned_addr learned my addr 172.21.15.67:0/555496882 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:37.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.454+0000 7fbbe6000640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe809a7f0 0x7fbbe80cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.455+0000 7fbbe6a00640 1 -- 172.21.15.67:0/555496882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe809a7f0 msgr2=0x7fbbe80cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.455+0000 7fbbe6a00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe809a7f0 0x7fbbe80cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.455+0000 7fbbe7400640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe80cbeb0 0x7fbbe81fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.455+0000 7fbbe6a00640 1 -- 172.21.15.67:0/555496882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe80cbeb0 msgr2=0x7fbbe81fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.455+0000 7fbbe6a00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe80cbeb0 0x7fbbe81fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.455+0000 7fbbe6a00640 1 -- 172.21.15.67:0/555496882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe809c260 con 0x7fbbe81f8d00 2024-05-30T16:54:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.455+0000 7fbbe7400640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe80cbeb0 0x7fbbe81fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.455+0000 7fbbe6a00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbe81f8d00 0x7fbbe809a2b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c700 tx=0x7fbbd000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:37.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.455+0000 7fbbdea00640 1 -- 172.21.15.67:0/555496882 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0018070 con 0x7fbbe81f8d00 2024-05-30T16:54:37.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.456+0000 7fbbdea00640 1 -- 172.21.15.67:0/555496882 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd0004cb0 con 0x7fbbe81f8d00 2024-05-30T16:54:37.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.456+0000 7fbbdea00640 1 -- 172.21.15.67:0/555496882 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00072d0 con 0x7fbbe81f8d00 2024-05-30T16:54:37.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.456+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe80cc6f0 con 0x7fbbe81f8d00 2024-05-30T16:54:37.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.456+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe80cccb0 con 0x7fbbe81f8d00 2024-05-30T16:54:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.457+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe80689e0 con 0x7fbbe81f8d00 2024-05-30T16:54:37.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.459+0000 7fbbdea00640 1 -- 172.21.15.67:0/555496882 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fbbd0007470 con 0x7fbbe81f8d00 2024-05-30T16:54:37.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.460+0000 7fbbdea00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbbc406d610 0x7fbbc406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:37.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.460+0000 7fbbdea00640 1 -- 172.21.15.67:0/555496882 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fbbd0089b90 con 0x7fbbe81f8d00 2024-05-30T16:54:37.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.464+0000 7fbbe6000640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbbc406d610 0x7fbbc406fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:37.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.467+0000 7fbbe6000640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbbc406d610 0x7fbbc406fad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80b4490 tx=0x7fbbd8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:37.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.468+0000 7fbbdea00640 1 -- 172.21.15.67:0/555496882 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbbd0059a10 con 0x7fbbe81f8d00 2024-05-30T16:54:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.908+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15501", "target": ["mon-mgr", ""]}) -- 0x7fbbe809abd0 con 0x7fbbc406d610 2024-05-30T16:54:37.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.941+0000 7fbbdea00640 1 -- 172.21.15.67:0/555496882 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbbe809abd0 con 0x7fbbc406d610 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.945+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbbc406d610 msgr2=0x7fbbc406fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.945+0000 7fbbe7e00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbbc406d610 0x7fbbc406fad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80b4490 tx=0x7fbbd8008040 comp rx=0 tx=0).stop 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbe81f8d00 msgr2=0x7fbbe809a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbe81f8d00 0x7fbbe809a2b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c700 tx=0x7fbbd000cbc0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 shutdown_connections 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbbc406d610 0x7fbbc406fad0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbe80cbeb0 0x7fbbe81fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbe809a7f0 0x7fbbe80cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 --2- 172.21.15.67:0/555496882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbe81f8d00 0x7fbbe809a2b0 secure :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c700 tx=0x7fbbd000cbc0 comp rx=0 tx=0).stop 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 >> 172.21.15.67:0/555496882 conn(0x7fbbe806bae0 msgr2=0x7fbbe80b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 shutdown_connections 2024-05-30T16:54:37.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:37.946+0000 7fbbe7e00640 1 -- 172.21.15.67:0/555496882 wait complete. 2024-05-30T16:54:37.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15504 2024-05-30T16:54:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.165+0000 7f519f600640 1 Processor -- start 2024-05-30T16:54:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.165+0000 7f519f600640 1 -- start start 2024-05-30T16:54:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.166+0000 7f519f600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5198140500 0x7f51980ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.166+0000 7f519f600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51980d0020 con 0x7f51980cf2f0 2024-05-30T16:54:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.166+0000 7f519f600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f519806dd50 con 0x7f519806edf0 2024-05-30T16:54:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.166+0000 7f519f600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f519806e030 con 0x7f5198140500 2024-05-30T16:54:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.167+0000 7f519cc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f51980cf2f0 0x7f51980cf910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52570/0 (socket says 172.21.15.67:52570) 2024-05-30T16:54:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.167+0000 7f519cc00640 1 -- 172.21.15.67:0/153994293 learned_addr learned my addr 172.21.15.67:0/153994293 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.167+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3187306833 0 0) 0x7f51980d0020 con 0x7f51980cf2f0 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.167+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5180003520 con 0x7f51980cf2f0 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.167+0000 7f519d600640 1 --2- 172.21.15.67:0/153994293 >> v2:172.21.15.169:3300/0 conn(0x7f5198140500 0x7f51980ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.167+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2523812788 0 0) 0x7f5180003520 con 0x7f51980cf2f0 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.167+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 >> v2:172.21.15.169:3300/0 conn(0x7f5198140500 msgr2=0x7f51980ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.167+0000 7f5197400640 1 --2- 172.21.15.67:0/153994293 >> v2:172.21.15.169:3300/0 conn(0x7f5198140500 0x7f51980ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.167+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 >> v1:172.21.15.169:6789/0 conn(0x7f519806edf0 legacy=0x7f519813fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519806e310 con 0x7f51980cf2f0 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f518c001670 con 0x7f51980cf2f0 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 >> v1:172.21.15.67:6789/0 conn(0x7f51980cf2f0 legacy=0x7f51980cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f5197400640 1 --2- 172.21.15.67:0/153994293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51800039b0 0x7f5180003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f5197400640 1 --2- 172.21.15.67:0/153994293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5180004800 0x7f5180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f5197400640 1 --2- 172.21.15.67:0/153994293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51800052e0 0x7f518000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f519806e030 con 0x7f51800039b0 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f518000c1b0 con 0x7f5180004800 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f518000c3b0 con 0x7f51800052e0 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f519cc00640 1 --2- 172.21.15.67:0/153994293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51800039b0 0x7f5180003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f519cc00640 1 -- 172.21.15.67:0/153994293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51800052e0 msgr2=0x7f518000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f519cc00640 1 --2- 172.21.15.67:0/153994293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51800052e0 0x7f518000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f519cc00640 1 -- 172.21.15.67:0/153994293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5180004800 msgr2=0x7f5180004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f519cc00640 1 --2- 172.21.15.67:0/153994293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5180004800 0x7f5180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f519cc00640 1 -- 172.21.15.67:0/153994293 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5180003520 con 0x7f51800039b0 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.168+0000 7f519d600640 1 -- 172.21.15.67:0/153994293 reap_dead start 2024-05-30T16:54:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.169+0000 7f519cc00640 1 --2- 172.21.15.67:0/153994293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51800039b0 0x7f5180003dd0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c210 tx=0x7f5180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac0a3410f3b39f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:38.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.169+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c007d10 con 0x7f51800039b0 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.169+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518c005ad0 con 0x7f51800039b0 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.170+0000 7f5197400640 1 -- 172.21.15.67:0/153994293 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c007070 con 0x7f51800039b0 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.170+0000 7f519f600640 1 -- 172.21.15.67:0/153994293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51800039b0 msgr2=0x7f5180003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.170+0000 7f519f600640 1 --2- 172.21.15.67:0/153994293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51800039b0 0x7f5180003dd0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c210 tx=0x7f5180005830 comp rx=0 tx=0).stop 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.170+0000 7f519f600640 1 -- 172.21.15.67:0/153994293 shutdown_connections 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.170+0000 7f519f600640 1 --2- 172.21.15.67:0/153994293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51800039b0 0x7f5180003dd0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.170+0000 7f519f600640 1 -- 172.21.15.67:0/153994293 >> 172.21.15.67:0/153994293 conn(0x7f51980b1e50 msgr2=0x7f51980b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.170+0000 7f519f600640 1 -- 172.21.15.67:0/153994293 shutdown_connections 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.170+0000 7f519f600640 1 -- 172.21.15.67:0/153994293 wait complete. 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.172+0000 7f519f600640 1 Processor -- start 2024-05-30T16:54:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.172+0000 7f519f600640 1 -- start start 2024-05-30T16:54:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f519f600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51800039b0 0x7f519815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f519f600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f519815dbd0 0x7f519815ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f519f600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51980b4440 0x7f51980b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f519f600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5198146020 con 0x7f519815dbd0 2024-05-30T16:54:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f519f600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f519806e8d0 con 0x7f51800039b0 2024-05-30T16:54:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f519f600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f51981461a0 con 0x7f51980b4440 2024-05-30T16:54:38.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f5197e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f519815dbd0 0x7f519815ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f5197e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f519815dbd0 0x7f519815ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56936/0 (socket says 172.21.15.67:56936) 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f5197e00640 1 -- 172.21.15.67:0/3913591664 learned_addr learned my addr 172.21.15.67:0/3913591664 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f5197e00640 1 -- 172.21.15.67:0/3913591664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51980b4440 msgr2=0x7f51980b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f5197e00640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51980b4440 0x7f51980b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f5197e00640 1 -- 172.21.15.67:0/3913591664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51800039b0 msgr2=0x7f519815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f5197e00640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51800039b0 0x7f519815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.173+0000 7f5197e00640 1 -- 172.21.15.67:0/3913591664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519815ce10 con 0x7f519815dbd0 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.174+0000 7f5197e00640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f519815dbd0 0x7f519815ead0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f518400c430 tx=0x7f518400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.174+0000 7f5194c00640 1 -- 172.21.15.67:0/3913591664 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184017070 con 0x7f519815dbd0 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.174+0000 7f5194c00640 1 -- 172.21.15.67:0/3913591664 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51840052f0 con 0x7f519815dbd0 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.174+0000 7f5194c00640 1 -- 172.21.15.67:0/3913591664 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184003830 con 0x7f519815dbd0 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.174+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519815d0f0 con 0x7f519815dbd0 2024-05-30T16:54:38.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.174+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51980b4e10 con 0x7f519815dbd0 2024-05-30T16:54:38.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.175+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51980689e0 con 0x7f519815dbd0 2024-05-30T16:54:38.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.176+0000 7f5194c00640 1 -- 172.21.15.67:0/3913591664 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f518401a070 con 0x7f519815dbd0 2024-05-30T16:54:38.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.177+0000 7f5194c00640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f517806d5e0 0x7f517806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:38.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.177+0000 7f5194c00640 1 -- 172.21.15.67:0/3913591664 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f51840899d0 con 0x7f519815dbd0 2024-05-30T16:54:38.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.177+0000 7f519cc00640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f517806d5e0 0x7f517806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:38.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.178+0000 7f519cc00640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f517806d5e0 0x7f517806faa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f518c032000 tx=0x7f518c009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:38.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.181+0000 7f5194c00640 1 -- 172.21.15.67:0/3913591664 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5184059850 con 0x7f519815dbd0 2024-05-30T16:54:38.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.412+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15504", "target": ["mon-mgr", ""]}) -- 0x7f5198073f30 con 0x7f517806d5e0 2024-05-30T16:54:38.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.456+0000 7f5194c00640 1 -- 172.21.15.67:0/3913591664 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5198073f30 con 0x7f517806d5e0 2024-05-30T16:54:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.459+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f517806d5e0 msgr2=0x7f517806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.459+0000 7f519f600640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f517806d5e0 0x7f517806faa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f518c032000 tx=0x7f518c009030 comp rx=0 tx=0).stop 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.459+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f519815dbd0 msgr2=0x7f519815ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.459+0000 7f519f600640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f519815dbd0 0x7f519815ead0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f518400c430 tx=0x7f518400c8f0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.460+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 shutdown_connections 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.460+0000 7f519f600640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f517806d5e0 0x7f517806faa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.460+0000 7f519f600640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51980b4440 0x7f51980b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.460+0000 7f519f600640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f519815dbd0 0x7f519815ead0 secure :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f518400c430 tx=0x7f518400c8f0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.460+0000 7f519f600640 1 --2- 172.21.15.67:0/3913591664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51800039b0 0x7f519815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.460+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 >> 172.21.15.67:0/3913591664 conn(0x7f51980b1e50 msgr2=0x7f5198072090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.460+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 shutdown_connections 2024-05-30T16:54:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38.461+0000 7f519f600640 1 -- 172.21.15.67:0/3913591664 wait complete. 2024-05-30T16:54:38.491 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-30T16:54:38.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-30T16:54:38.644 INFO:teuthology.orchestra.run.smithi067.stdout:2 2024-05-30T16:54:38.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:54:38.659 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T16:54:38.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:54:38.768 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:54:38.876 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T16:54:38.876 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:54:38.876 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:54:38.876 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:54:38.931 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:54:38.931 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:54:38.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:54:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.191+0000 7fa121c00640 1 Processor -- start 2024-05-30T16:54:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.191+0000 7fa121c00640 1 -- start start 2024-05-30T16:54:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa121c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa11c0707d0 0x7fa11c06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa121c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c06c310 0x7fa11c0d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:39.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa121c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa11c15fbc0 con 0x7fa11c0b01e0 2024-05-30T16:54:39.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa121c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa11c070d50 con 0x7fa11c0707d0 2024-05-30T16:54:39.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa121c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c0b6140 con 0x7fa11c06c310 2024-05-30T16:54:39.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa112000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa11c0707d0 0x7fa11c06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:39.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa112000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa11c0707d0 0x7fa11c06ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56950/0 (socket says 172.21.15.67:56950) 2024-05-30T16:54:39.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa112000640 1 -- 172.21.15.67:0/1531326926 learned_addr learned my addr 172.21.15.67:0/1531326926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa112000640 1 -- 172.21.15.67:0/1531326926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c06c310 msgr2=0x7fa11c0d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa112000640 1 --2- 172.21.15.67:0/1531326926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c06c310 0x7fa11c0d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa112000640 1 -- 172.21.15.67:0/1531326926 >> v1:172.21.15.169:6789/0 conn(0x7fa11c0b01e0 legacy=0x7fa11c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.193+0000 7fa112000640 1 -- 172.21.15.67:0/1531326926 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c0b6420 con 0x7fa11c0707d0 2024-05-30T16:54:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.194+0000 7fa112000640 1 --2- 172.21.15.67:0/1531326926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c0707d0 0x7fa11c06ee90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa104009870 tx=0x7fa10402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8fdc40eb4b9cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa11a000640 1 -- 172.21.15.67:0/1531326926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104037070 con 0x7fa11c0707d0 2024-05-30T16:54:39.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa11a000640 1 -- 172.21.15.67:0/1531326926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa104032440 con 0x7fa11c0707d0 2024-05-30T16:54:39.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa11a000640 1 -- 172.21.15.67:0/1531326926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104032810 con 0x7fa11c0707d0 2024-05-30T16:54:39.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa121c00640 1 -- 172.21.15.67:0/1531326926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c0707d0 msgr2=0x7fa11c06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:39.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa121c00640 1 --2- 172.21.15.67:0/1531326926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c0707d0 0x7fa11c06ee90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa104009870 tx=0x7fa10402a210 comp rx=0 tx=0).stop 2024-05-30T16:54:39.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa121c00640 1 -- 172.21.15.67:0/1531326926 shutdown_connections 2024-05-30T16:54:39.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa121c00640 1 --2- 172.21.15.67:0/1531326926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c06c310 0x7fa11c0d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:39.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa121c00640 1 --2- 172.21.15.67:0/1531326926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c0707d0 0x7fa11c06ee90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:39.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa121c00640 1 -- 172.21.15.67:0/1531326926 >> 172.21.15.67:0/1531326926 conn(0x7fa11c09aa30 msgr2=0x7fa11c0b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:39.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa121c00640 1 -- 172.21.15.67:0/1531326926 shutdown_connections 2024-05-30T16:54:39.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.195+0000 7fa121c00640 1 -- 172.21.15.67:0/1531326926 wait complete. 2024-05-30T16:54:39.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.197+0000 7fa121c00640 1 Processor -- start 2024-05-30T16:54:39.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.197+0000 7fa121c00640 1 -- start start 2024-05-30T16:54:39.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.198+0000 7fa121c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c06c310 0x7fa11c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:39.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.198+0000 7fa121c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0707d0 0x7fa11c161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:39.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.198+0000 7fa121c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c0b01e0 0x7fa11c153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:39.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.198+0000 7fa121c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c0d19b0 con 0x7fa11c06c310 2024-05-30T16:54:39.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.198+0000 7fa121c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c0b5df0 con 0x7fa11c0707d0 2024-05-30T16:54:39.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.198+0000 7fa121c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c0d1b30 con 0x7fa11c0b01e0 2024-05-30T16:54:39.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.198+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c06c310 0x7fa11c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:39.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.198+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c06c310 0x7fa11c156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56962/0 (socket says 172.21.15.67:56962) 2024-05-30T16:54:39.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.198+0000 7fa11aa00640 1 -- 172.21.15.67:0/3758325272 learned_addr learned my addr 172.21.15.67:0/3758325272 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa112000640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0707d0 0x7fa11c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa11aa00640 1 -- 172.21.15.67:0/3758325272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c0b01e0 msgr2=0x7fa11c153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa11aa00640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c0b01e0 0x7fa11c153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa11aa00640 1 -- 172.21.15.67:0/3758325272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0707d0 msgr2=0x7fa11c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa11aa00640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0707d0 0x7fa11c161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa11aa00640 1 -- 172.21.15.67:0/3758325272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c156000 con 0x7fa11c06c310 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa11aa00640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c06c310 0x7fa11c156aa0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00ba50 tx=0x7fa10c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa113400640 1 -- 172.21.15.67:0/3758325272 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c019070 con 0x7fa11c06c310 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa113400640 1 -- 172.21.15.67:0/3758325272 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10c00cc20 con 0x7fa11c06c310 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa113400640 1 -- 172.21.15.67:0/3758325272 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c014680 con 0x7fa11c06c310 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c154600 con 0x7fa11c06c310 2024-05-30T16:54:39.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.199+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11c1548e0 con 0x7fa11c06c310 2024-05-30T16:54:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.201+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11c0689e0 con 0x7fa11c06c310 2024-05-30T16:54:39.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.201+0000 7fa113400640 1 -- 172.21.15.67:0/3758325272 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fa10c01c070 con 0x7fa11c06c310 2024-05-30T16:54:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.203+0000 7fa113400640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa0f806d5e0 0x7fa0f806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.203+0000 7fa113400640 1 -- 172.21.15.67:0/3758325272 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5793+0+0 (secure 0 0 0) 0x7fa10c08a240 con 0x7fa11c06c310 2024-05-30T16:54:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.203+0000 7fa112000640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa0f806d5e0 0x7fa0f806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.204+0000 7fa112000640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa0f806d5e0 0x7fa0f806faa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa104034000 tx=0x7fa10402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:39.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.207+0000 7fa113400640 1 -- 172.21.15.67:0/3758325272 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa10c059a70 con 0x7fa11c06c310 2024-05-30T16:54:39.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.419+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa11c06ddb0 con 0x7fa11c06c310 2024-05-30T16:54:39.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.419+0000 7fa113400640 1 -- 172.21.15.67:0/3758325272 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7fa10c05dd70 con 0x7fa11c06c310 2024-05-30T16:54:39.420 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa0f806d5e0 msgr2=0x7fa0f806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa0f806d5e0 0x7fa0f806faa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa104034000 tx=0x7fa10402e000 comp rx=0 tx=0).stop 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c06c310 msgr2=0x7fa11c156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c06c310 0x7fa11c156aa0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00ba50 tx=0x7fa10c00bf10 comp rx=0 tx=0).stop 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 shutdown_connections 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa0f806d5e0 0x7fa0f806faa0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa11c0b01e0 0x7fa11c153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa11c0707d0 0x7fa11c161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 --2- 172.21.15.67:0/3758325272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa11c06c310 0x7fa11c156aa0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 >> 172.21.15.67:0/3758325272 conn(0x7fa11c09aa30 msgr2=0x7fa11c0ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 shutdown_connections 2024-05-30T16:54:39.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:39.422+0000 7fa121c00640 1 -- 172.21.15.67:0/3758325272 wait complete. 2024-05-30T16:54:39.442 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T16:54:39.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:54:39.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-30T16:54:39.545 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[56210]: fuse finished with error 0 and tester_r 0 2024-05-30T16:54:39.546 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:54:39.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:54:39.546 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-30T16:54:39.715 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:54:39.715 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:54:39.715 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:54:39.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:54:39.771 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:54:39.771 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:54:39.771 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:54:39.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:54:39.832 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:54:39.832 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:54:39.832 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:54:39.889 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:54:39.889 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:54:40.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.142+0000 7f7601600640 1 Processor -- start 2024-05-30T16:54:40.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.142+0000 7f7601600640 1 -- start start 2024-05-30T16:54:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.143+0000 7f7601600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f75fc15e9d0 0x7f75fc0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.143+0000 7f7601600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75fc0d52a0 0x7f75fc0d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.143+0000 7f7601600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75fc0d5730 con 0x7f75fc0d4150 2024-05-30T16:54:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.143+0000 7f7601600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f75fc15edb0 con 0x7f75fc15e9d0 2024-05-30T16:54:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.143+0000 7f7601600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f75fc0d2b70 con 0x7f75fc0d52a0 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75faa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f75fc15e9d0 0x7f75fc0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75fa000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75fc0d52a0 0x7f75fc0d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75fa000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75fc0d52a0 0x7f75fc0d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:53714/0 (socket says 172.21.15.67:53714) 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75fa000640 1 -- 172.21.15.67:0/3537646095 learned_addr learned my addr 172.21.15.67:0/3537646095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75fb400640 1 --1- 172.21.15.67:0/3537646095 >> v1:172.21.15.169:6789/0 conn(0x7f75fc0d4150 0x7f75fc0d1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:49854/0 (socket says 172.21.15.67:49854) 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75faa00640 1 -- 172.21.15.67:0/3537646095 >> v2:172.21.15.169:3300/0 conn(0x7f75fc0d52a0 msgr2=0x7f75fc0d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75faa00640 1 --2- 172.21.15.67:0/3537646095 >> v2:172.21.15.169:3300/0 conn(0x7f75fc0d52a0 0x7f75fc0d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75faa00640 1 -- 172.21.15.67:0/3537646095 >> v1:172.21.15.169:6789/0 conn(0x7f75fc0d4150 legacy=0x7f75fc0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75f9600640 1 -- 172.21.15.67:0/3537646095 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4267815286 0 0) 0x7f75f4001490 con 0x7f75fc0d4150 2024-05-30T16:54:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.144+0000 7f75faa00640 1 -- 172.21.15.67:0/3537646095 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc0d2e50 con 0x7f75fc15e9d0 2024-05-30T16:54:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.146+0000 7f75faa00640 1 --2- 172.21.15.67:0/3537646095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc15e9d0 0x7f75fc0d24d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0b3dc0 tx=0x7f75ec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267c9e9ee531a017 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.147+0000 7f75f9600640 1 -- 172.21.15.67:0/3537646095 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec037070 con 0x7f75fc15e9d0 2024-05-30T16:54:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.147+0000 7f75f9600640 1 -- 172.21.15.67:0/3537646095 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec032440 con 0x7f75fc15e9d0 2024-05-30T16:54:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.148+0000 7f75f9600640 1 -- 172.21.15.67:0/3537646095 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec02ae60 con 0x7f75fc15e9d0 2024-05-30T16:54:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.149+0000 7f7601600640 1 -- 172.21.15.67:0/3537646095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc15e9d0 msgr2=0x7f75fc0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.149+0000 7f7601600640 1 --2- 172.21.15.67:0/3537646095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc15e9d0 0x7f75fc0d24d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f75fc0b3dc0 tx=0x7f75ec02a110 comp rx=0 tx=0).stop 2024-05-30T16:54:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.150+0000 7f7601600640 1 -- 172.21.15.67:0/3537646095 shutdown_connections 2024-05-30T16:54:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.150+0000 7f7601600640 1 --2- 172.21.15.67:0/3537646095 >> v2:172.21.15.169:3300/0 conn(0x7f75fc0d52a0 0x7f75fc0d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.150+0000 7f7601600640 1 --2- 172.21.15.67:0/3537646095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc15e9d0 0x7f75fc0d24d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.150+0000 7f7601600640 1 -- 172.21.15.67:0/3537646095 >> 172.21.15.67:0/3537646095 conn(0x7f75fc067890 msgr2=0x7f75fc067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.150+0000 7f7601600640 1 -- 172.21.15.67:0/3537646095 shutdown_connections 2024-05-30T16:54:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.151+0000 7f7601600640 1 -- 172.21.15.67:0/3537646095 wait complete. 2024-05-30T16:54:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.154+0000 7f7601600640 1 Processor -- start 2024-05-30T16:54:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.154+0000 7f7601600640 1 -- start start 2024-05-30T16:54:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f7601600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc0d52a0 0x7f75fc1e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f7601600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75fc15e9d0 0x7f75fc1f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f7601600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75fc153570 0x7f75fc153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f7601600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0d36f0 con 0x7f75fc0d52a0 2024-05-30T16:54:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f7601600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0d3410 con 0x7f75fc153570 2024-05-30T16:54:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f7601600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc0d1d60 con 0x7f75fc15e9d0 2024-05-30T16:54:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f75faa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc0d52a0 0x7f75fc1e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f75faa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc0d52a0 0x7f75fc1e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56972/0 (socket says 172.21.15.67:56972) 2024-05-30T16:54:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f75faa00640 1 -- 172.21.15.67:0/246227777 learned_addr learned my addr 172.21.15.67:0/246227777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f75fb400640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75fc153570 0x7f75fc153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.156+0000 7f75fa000640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75fc15e9d0 0x7f75fc1f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.157+0000 7f75faa00640 1 -- 172.21.15.67:0/246227777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75fc15e9d0 msgr2=0x7f75fc1f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.157+0000 7f75faa00640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75fc15e9d0 0x7f75fc1f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.157+0000 7f75faa00640 1 -- 172.21.15.67:0/246227777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75fc153570 msgr2=0x7f75fc153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.157+0000 7f75faa00640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75fc153570 0x7f75fc153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.157+0000 7f75faa00640 1 -- 172.21.15.67:0/246227777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc158990 con 0x7f75fc0d52a0 2024-05-30T16:54:40.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.157+0000 7f75faa00640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc0d52a0 0x7f75fc1e70a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f75ec035040 tx=0x7f75ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:40.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.158+0000 7f75f2a00640 1 -- 172.21.15.67:0/246227777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec037040 con 0x7f75fc0d52a0 2024-05-30T16:54:40.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.158+0000 7f75f2a00640 1 -- 172.21.15.67:0/246227777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec004230 con 0x7f75fc0d52a0 2024-05-30T16:54:40.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.158+0000 7f75f2a00640 1 -- 172.21.15.67:0/246227777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec003610 con 0x7f75fc0d52a0 2024-05-30T16:54:40.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.158+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc1e75e0 con 0x7f75fc0d52a0 2024-05-30T16:54:40.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.158+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc156270 con 0x7f75fc0d52a0 2024-05-30T16:54:40.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.160+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75fc0689e0 con 0x7f75fc0d52a0 2024-05-30T16:54:40.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.163+0000 7f75f2a00640 1 -- 172.21.15.67:0/246227777 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f75ec0037b0 con 0x7f75fc0d52a0 2024-05-30T16:54:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.164+0000 7f75f2a00640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f75dc06d5e0 0x7f75dc06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.164+0000 7f75f2a00640 1 -- 172.21.15.67:0/246227777 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5793+0+0 (secure 0 0 0) 0x7f75ec0ab2b0 con 0x7f75fc0d52a0 2024-05-30T16:54:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.165+0000 7f75fa000640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f75dc06d5e0 0x7f75dc06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.167+0000 7f75fa000640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f75dc06d5e0 0x7f75dc06faa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f75fc1f7e70 tx=0x7f75e8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:40.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.172+0000 7f75f2a00640 1 -- 172.21.15.67:0/246227777 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75ec07b130 con 0x7f75fc0d52a0 2024-05-30T16:54:40.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.409+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75fc0b5a80 con 0x7f75fc0d52a0 2024-05-30T16:54:40.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.409+0000 7f75f2a00640 1 -- 172.21.15.67:0/246227777 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5181 (secure 0 0 0) 0x7f75ec07aa30 con 0x7f75fc0d52a0 2024-05-30T16:54:40.410 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:40.410 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2932035652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2932035652},{"type":"v1","addr":"172.21.15.67:6837","nonce":2932035652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/327493818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":327493818},{"type":"v1","addr":"172.21.15.67:6836","nonce":327493818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1437754767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1437754767},{"type":"v1","addr":"172.21.15.169:6834","nonce":1437754767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:52:27.681449+0000","modified":"2024-05-30T16:52:30.749556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/898694760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":898694760},{"type":"v1","addr":"172.21.15.169:6835","nonce":898694760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5937,"qdb_cluster":[5937]},"id":4}]} 2024-05-30T16:54:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 36 2024-05-30T16:54:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f75dc06d5e0 msgr2=0x7f75dc06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f75dc06d5e0 0x7f75dc06faa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f75fc1f7e70 tx=0x7f75e8008040 comp rx=0 tx=0).stop 2024-05-30T16:54:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc0d52a0 msgr2=0x7f75fc1e70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc0d52a0 0x7f75fc1e70a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f75ec035040 tx=0x7f75ec0046c0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 shutdown_connections 2024-05-30T16:54:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f75dc06d5e0 0x7f75dc06faa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75fc153570 0x7f75fc153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75fc15e9d0 0x7f75fc1f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 --2- 172.21.15.67:0/246227777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75fc0d52a0 0x7f75fc1e70a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 >> 172.21.15.67:0/246227777 conn(0x7f75fc067890 msgr2=0x7f75fc15ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.412+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 shutdown_connections 2024-05-30T16:54:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.413+0000 7f7601600640 1 -- 172.21.15.67:0/246227777 wait complete. 2024-05-30T16:54:40.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:54:40.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.684+0000 7f5dd9e00640 1 Processor -- start 2024-05-30T16:54:40.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.685+0000 7f5dd9e00640 1 -- start start 2024-05-30T16:54:40.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.686+0000 7f5dd9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5dd40d2db0 0x7f5dd40b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.686+0000 7f5dd9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40d4d60 0x7f5dd40d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.687+0000 7f5dd9e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dd40d36d0 con 0x7f5dd40cf490 2024-05-30T16:54:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.687+0000 7f5dd9e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd40d5540 con 0x7f5dd40d2db0 2024-05-30T16:54:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.687+0000 7f5dd9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd4163500 con 0x7f5dd40d4d60 2024-05-30T16:54:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.687+0000 7f5dd3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5dd40cf490 0x7f5dd40cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52576/0 (socket says 172.21.15.67:52576) 2024-05-30T16:54:40.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.687+0000 7f5dd3400640 1 -- 172.21.15.67:0/1622331343 learned_addr learned my addr 172.21.15.67:0/1622331343 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:40.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.687+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/1622331343 >> v2:172.21.15.67:3300/0 conn(0x7f5dd40d2db0 0x7f5dd40b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.687+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396774023 0 0) 0x7f5dd40d36d0 con 0x7f5dd40cf490 2024-05-30T16:54:40.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.687+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5db8003520 con 0x7f5dd40cf490 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1646027079 0 0) 0x7f5db8003520 con 0x7f5dd40cf490 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40d4d60 msgr2=0x7f5dd40d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40d4d60 0x7f5dd40d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 >> v2:172.21.15.67:3300/0 conn(0x7f5dd40d2db0 msgr2=0x7f5dd40b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 --2- 172.21.15.67:0/1622331343 >> v2:172.21.15.67:3300/0 conn(0x7f5dd40d2db0 0x7f5dd40b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd4162740 con 0x7f5dd40cf490 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5dc0001670 con 0x7f5dd40cf490 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 >> v1:172.21.15.67:6789/0 conn(0x7f5dd40cf490 legacy=0x7f5dd40cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40d4d60 0x7f5db80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db8003f30 0x7f5db8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5db8004d40 0x7f5db80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd40d5540 con 0x7f5db8003f30 2024-05-30T16:54:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd4163500 con 0x7f5db8004d40 2024-05-30T16:54:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5db80143b0 con 0x7f5dd40d4d60 2024-05-30T16:54:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.688+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/1622331343 >> v2:172.21.15.67:3300/0 conn(0x7f5dd40d2db0 0x7f5dd40b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:54:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.689+0000 7f5dd2000640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db8003f30 0x7f5db8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.689+0000 7f5dd3400640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5db8004d40 0x7f5db80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.690+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40d4d60 0x7f5db80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.691+0000 7f5dd3400640 1 -- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40d4d60 msgr2=0x7f5db80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.691+0000 7f5dd3400640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40d4d60 0x7f5db80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.691+0000 7f5dd3400640 1 -- 172.21.15.67:0/1622331343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db8003f30 msgr2=0x7f5db8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.691+0000 7f5dd3400640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db8003f30 0x7f5db8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.691+0000 7f5dd3400640 1 -- 172.21.15.67:0/1622331343 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd4162740 con 0x7f5db8004d40 2024-05-30T16:54:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.691+0000 7f5dd2000640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5db8003f30 0x7f5db8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:54:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.692+0000 7f5dd3400640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5db8004d40 0x7f5db80051c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c290 tx=0x7f5db8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ba635466b4fce13 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.692+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0036070 con 0x7f5db8004d40 2024-05-30T16:54:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.692+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc0005ad0 con 0x7f5db8004d40 2024-05-30T16:54:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.692+0000 7f5dd1600640 1 -- 172.21.15.67:0/1622331343 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0007d70 con 0x7f5db8004d40 2024-05-30T16:54:40.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.694+0000 7f5dd9e00640 1 -- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5db8004d40 msgr2=0x7f5db80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.694+0000 7f5dd9e00640 1 --2- 172.21.15.67:0/1622331343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5db8004d40 0x7f5db80051c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c290 tx=0x7f5db8005830 comp rx=0 tx=0).stop 2024-05-30T16:54:40.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.694+0000 7f5dd3400640 1 -- 172.21.15.67:0/1622331343 reap_dead start 2024-05-30T16:54:40.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.694+0000 7f5dd9e00640 1 -- 172.21.15.67:0/1622331343 shutdown_connections 2024-05-30T16:54:40.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.694+0000 7f5dd9e00640 1 -- 172.21.15.67:0/1622331343 >> 172.21.15.67:0/1622331343 conn(0x7f5dd408bfc0 msgr2=0x7f5dd40d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:40.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.694+0000 7f5dd9e00640 1 -- 172.21.15.67:0/1622331343 shutdown_connections 2024-05-30T16:54:40.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.694+0000 7f5dd9e00640 1 -- 172.21.15.67:0/1622331343 wait complete. 2024-05-30T16:54:40.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.696+0000 7f5dd9e00640 1 Processor -- start 2024-05-30T16:54:40.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.696+0000 7f5dd9e00640 1 -- start start 2024-05-30T16:54:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.698+0000 7f5dd9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5db8004d40 0x7f5dd40b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.698+0000 7f5dd9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd40ca360 0x7f5dd40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.698+0000 7f5dd9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40ca7c0 0x7f5dd40b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.698+0000 7f5dd9e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd4162fe0 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.698+0000 7f5dd9e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd4162d00 con 0x7f5dd40ca360 2024-05-30T16:54:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.698+0000 7f5dd9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd40d51a0 con 0x7f5db8004d40 2024-05-30T16:54:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.698+0000 7f5dd2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd40ca360 0x7f5dd40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.698+0000 7f5dd3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40ca7c0 0x7f5dd40b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.699+0000 7f5dd3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40ca7c0 0x7f5dd40b90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56990/0 (socket says 172.21.15.67:56990) 2024-05-30T16:54:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.699+0000 7f5dd2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd40ca360 0x7f5dd40b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:53738/0 (socket says 172.21.15.67:53738) 2024-05-30T16:54:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.699+0000 7f5dd3400640 1 -- 172.21.15.67:0/3438087031 learned_addr learned my addr 172.21.15.67:0/3438087031 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.699+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5db8004d40 0x7f5dd40b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.700+0000 7f5dd3400640 1 -- 172.21.15.67:0/3438087031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5db8004d40 msgr2=0x7f5dd40b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.700+0000 7f5dd3400640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5db8004d40 0x7f5dd40b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.700+0000 7f5dd3400640 1 -- 172.21.15.67:0/3438087031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd40ca360 msgr2=0x7f5dd40b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.700+0000 7f5dd3400640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd40ca360 0x7f5dd40b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.701+0000 7f5dd3400640 1 -- 172.21.15.67:0/3438087031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd40b7720 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.701+0000 7f5dd2000640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd40ca360 0x7f5dd40b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:54:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.701+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5db8004d40 0x7f5dd40b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:54:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.701+0000 7f5dd3400640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40ca7c0 0x7f5dd40b90d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c3b0 tx=0x7f5dc0001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.702+0000 7f5dcb400640 1 -- 172.21.15.67:0/3438087031 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0045070 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.702+0000 7f5dcb400640 1 -- 172.21.15.67:0/3438087031 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc0007270 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.702+0000 7f5dcb400640 1 -- 172.21.15.67:0/3438087031 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0036040 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.702+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd40b9c30 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.703+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd40cba20 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.704+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd40ba150 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.710+0000 7f5dcb400640 1 -- 172.21.15.67:0/3438087031 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f5dc0048070 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.711+0000 7f5dcb400640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5db406d6b0 0x7f5db406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.711+0000 7f5dcb400640 1 -- 172.21.15.67:0/3438087031 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5793+0+0 (secure 0 0 0) 0x7f5dc00a9e10 con 0x7f5dd40ca7c0 2024-05-30T16:54:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.712+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5db406d6b0 0x7f5db406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.713+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5db406d6b0 0x7f5db406fb70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc007a90 tx=0x7f5dcc00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:40.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:40.717+0000 7f5dcb400640 1 -- 172.21.15.67:0/3438087031 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dd40ba150 con 0x7f5dd40ca7c0 2024-05-30T16:54:41.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.154+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dd40689e0 con 0x7f5dd40ca7c0 2024-05-30T16:54:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.155+0000 7f5dcb400640 1 -- 172.21.15.67:0/3438087031 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5181 (secure 0 0 0) 0x7f5dc0079640 con 0x7f5dd40ca7c0 2024-05-30T16:54:41.156 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:41.156 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2932035652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2932035652},{"type":"v1","addr":"172.21.15.67:6837","nonce":2932035652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/327493818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":327493818},{"type":"v1","addr":"172.21.15.67:6836","nonce":327493818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1437754767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1437754767},{"type":"v1","addr":"172.21.15.169:6834","nonce":1437754767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:52:27.681449+0000","modified":"2024-05-30T16:52:30.749556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/898694760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":898694760},{"type":"v1","addr":"172.21.15.169:6835","nonce":898694760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5937,"qdb_cluster":[5937]},"id":4}]} 2024-05-30T16:54:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 36 2024-05-30T16:54:41.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5db406d6b0 msgr2=0x7f5db406fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5db406d6b0 0x7f5db406fb70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc007a90 tx=0x7f5dcc00b040 comp rx=0 tx=0).stop 2024-05-30T16:54:41.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40ca7c0 msgr2=0x7f5dd40b90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40ca7c0 0x7f5dd40b90d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c3b0 tx=0x7f5dc0001970 comp rx=0 tx=0).stop 2024-05-30T16:54:41.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 shutdown_connections 2024-05-30T16:54:41.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5db406d6b0 0x7f5db406fb70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40ca7c0 0x7f5dd40b90d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd40ca360 0x7f5dd40b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 --2- 172.21.15.67:0/3438087031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5db8004d40 0x7f5dd40b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.159+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 >> 172.21.15.67:0/3438087031 conn(0x7f5dd408bfc0 msgr2=0x7f5dd40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.160+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 shutdown_connections 2024-05-30T16:54:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.160+0000 7f5dd9e00640 1 -- 172.21.15.67:0/3438087031 wait complete. 2024-05-30T16:54:41.187 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:54:41.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.381+0000 7f0d30e00640 1 Processor -- start 2024-05-30T16:54:41.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.382+0000 7f0d30e00640 1 -- start start 2024-05-30T16:54:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.383+0000 7f0d30e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d2c15b3a0 0x7f0d2c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.383+0000 7f0d30e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0d2c073310 0x7f0d2c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.383+0000 7f0d30e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d2c0afd60 con 0x7f0d2c06dc20 2024-05-30T16:54:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.383+0000 7f0d30e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f0d2c06f980 con 0x7f0d2c073310 2024-05-30T16:54:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.383+0000 7f0d30e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0d2c0ae0c0 con 0x7f0d2c15b3a0 2024-05-30T16:54:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.383+0000 7f0d2be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0d2c073310 0x7f0d2c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.383+0000 7f0d2be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0d2c073310 0x7f0d2c15f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43060/0 (socket says 172.21.15.67:43060) 2024-05-30T16:54:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.383+0000 7f0d2be00640 1 -- 172.21.15.67:0/632937210 learned_addr learned my addr 172.21.15.67:0/632937210 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.383+0000 7f0d2aa00640 1 --1- 172.21.15.67:0/632937210 >> v1:172.21.15.67:6789/0 conn(0x7f0d2c06dc20 0x7f0d2c072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37474/0 (socket says 172.21.15.67:37474) 2024-05-30T16:54:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.384+0000 7f0d2b400640 1 --2- 172.21.15.67:0/632937210 >> v2:172.21.15.169:3300/0 conn(0x7f0d2c15b3a0 0x7f0d2c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.384+0000 7f0d2be00640 1 -- 172.21.15.67:0/632937210 >> v2:172.21.15.169:3300/0 conn(0x7f0d2c15b3a0 msgr2=0x7f0d2c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.384+0000 7f0d2be00640 1 --2- 172.21.15.67:0/632937210 >> v2:172.21.15.169:3300/0 conn(0x7f0d2c15b3a0 0x7f0d2c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.384+0000 7f0d2be00640 1 -- 172.21.15.67:0/632937210 >> v1:172.21.15.67:6789/0 conn(0x7f0d2c06dc20 legacy=0x7f0d2c072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.384+0000 7f0d2be00640 1 -- 172.21.15.67:0/632937210 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c0af7a0 con 0x7f0d2c073310 2024-05-30T16:54:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.384+0000 7f0d2a000640 1 -- 172.21.15.67:0/632937210 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336528152 0 0) 0x7f0d2c0afd60 con 0x7f0d2c06dc20 2024-05-30T16:54:41.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.385+0000 7f0d2be00640 1 --2- 172.21.15.67:0/632937210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 0x7f0d2c15f9a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c072bd0 tx=0x7f0d2402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9324175d8065fc00 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.385+0000 7f0d2a000640 1 -- 172.21.15.67:0/632937210 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2400b2e0 con 0x7f0d2c073310 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.385+0000 7f0d2a000640 1 -- 172.21.15.67:0/632937210 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d24037070 con 0x7f0d2c073310 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.385+0000 7f0d2a000640 1 -- 172.21.15.67:0/632937210 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2400b480 con 0x7f0d2c073310 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.386+0000 7f0d30e00640 1 -- 172.21.15.67:0/632937210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 msgr2=0x7f0d2c15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.386+0000 7f0d30e00640 1 --2- 172.21.15.67:0/632937210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 0x7f0d2c15f9a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c072bd0 tx=0x7f0d2402c940 comp rx=0 tx=0).stop 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.386+0000 7f0d30e00640 1 -- 172.21.15.67:0/632937210 shutdown_connections 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.386+0000 7f0d30e00640 1 --2- 172.21.15.67:0/632937210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 0x7f0d2c15f9a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.386+0000 7f0d30e00640 1 --2- 172.21.15.67:0/632937210 >> v2:172.21.15.169:3300/0 conn(0x7f0d2c15b3a0 0x7f0d2c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.386+0000 7f0d30e00640 1 -- 172.21.15.67:0/632937210 >> 172.21.15.67:0/632937210 conn(0x7f0d2c0b1fc0 msgr2=0x7f0d2c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.386+0000 7f0d30e00640 1 -- 172.21.15.67:0/632937210 shutdown_connections 2024-05-30T16:54:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.386+0000 7f0d30e00640 1 -- 172.21.15.67:0/632937210 wait complete. 2024-05-30T16:54:41.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.388+0000 7f0d30e00640 1 Processor -- start 2024-05-30T16:54:41.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.388+0000 7f0d30e00640 1 -- start start 2024-05-30T16:54:41.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.388+0000 7f0d30e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d2c06dc20 0x7f0d2c15e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d30e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 0x7f0d2c157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d30e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d2c15b3a0 0x7f0d2c0cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d30e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c15b780 con 0x7f0d2c073310 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d30e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c15fee0 con 0x7f0d2c06dc20 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d30e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2c0ae550 con 0x7f0d2c15b3a0 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 0x7f0d2c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 0x7f0d2c157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43072/0 (socket says 172.21.15.67:43072) 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2aa00640 1 -- 172.21.15.67:0/4155857998 learned_addr learned my addr 172.21.15.67:0/4155857998 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2be00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d2c15b3a0 0x7f0d2c0cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2aa00640 1 -- 172.21.15.67:0/4155857998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d2c15b3a0 msgr2=0x7f0d2c0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2aa00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d2c15b3a0 0x7f0d2c0cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2aa00640 1 -- 172.21.15.67:0/4155857998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d2c06dc20 msgr2=0x7f0d2c15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2aa00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d2c06dc20 0x7f0d2c15e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2aa00640 1 -- 172.21.15.67:0/4155857998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c1598b0 con 0x7f0d2c073310 2024-05-30T16:54:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.389+0000 7f0d2aa00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 0x7f0d2c157230 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c500 tx=0x7f0d1800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:41.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.390+0000 7f0d23400640 1 -- 172.21.15.67:0/4155857998 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18018070 con 0x7f0d2c073310 2024-05-30T16:54:41.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.390+0000 7f0d23400640 1 -- 172.21.15.67:0/4155857998 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18007580 con 0x7f0d2c073310 2024-05-30T16:54:41.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.390+0000 7f0d23400640 1 -- 172.21.15.67:0/4155857998 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18004a20 con 0x7f0d2c073310 2024-05-30T16:54:41.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.390+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2c159b90 con 0x7f0d2c073310 2024-05-30T16:54:41.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.390+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2c15a150 con 0x7f0d2c073310 2024-05-30T16:54:41.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.391+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2c0689e0 con 0x7f0d2c073310 2024-05-30T16:54:41.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.392+0000 7f0d23400640 1 -- 172.21.15.67:0/4155857998 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f0d1801b070 con 0x7f0d2c073310 2024-05-30T16:54:41.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.393+0000 7f0d23400640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0d0c06d5e0 0x7f0d0c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.393+0000 7f0d23400640 1 -- 172.21.15.67:0/4155857998 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5793+0+0 (secure 0 0 0) 0x7f0d18089960 con 0x7f0d2c073310 2024-05-30T16:54:41.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.393+0000 7f0d2b400640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0d0c06d5e0 0x7f0d0c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.394+0000 7f0d2b400640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0d0c06d5e0 0x7f0d0c06faa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c006fd0 tx=0x7f0d1c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:41.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.397+0000 7f0d23400640 1 -- 172.21.15.67:0/4155857998 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d180597e0 con 0x7f0d2c073310 2024-05-30T16:54:41.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.594+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d2c15a320 con 0x7f0d2c073310 2024-05-30T16:54:41.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.596+0000 7f0d23400640 1 -- 172.21.15.67:0/4155857998 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16805 (secure 0 0 0) 0x7f0d180590e0 con 0x7f0d2c073310 2024-05-30T16:54:41.597 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:41.597 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:54:39.101022+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:52:25.604028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-30T16:52:26.652938+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:54:41.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0d0c06d5e0 msgr2=0x7f0d0c06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0d0c06d5e0 0x7f0d0c06faa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c006fd0 tx=0x7f0d1c008040 comp rx=0 tx=0).stop 2024-05-30T16:54:41.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 msgr2=0x7f0d2c157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 0x7f0d2c157230 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c500 tx=0x7f0d1800c9c0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 shutdown_connections 2024-05-30T16:54:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0d0c06d5e0 0x7f0d0c06faa0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d2c15b3a0 0x7f0d2c0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c073310 0x7f0d2c157230 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 --2- 172.21.15.67:0/4155857998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d2c06dc20 0x7f0d2c15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.598+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 >> 172.21.15.67:0/4155857998 conn(0x7f0d2c0b1fc0 msgr2=0x7f0d2c071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.599+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 shutdown_connections 2024-05-30T16:54:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.599+0000 7f0d30e00640 1 -- 172.21.15.67:0/4155857998 wait complete. 2024-05-30T16:54:41.617 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-30T16:54:41.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:54:41.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.822+0000 7f64b8000640 1 Processor -- start 2024-05-30T16:54:41.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.823+0000 7f64b8000640 1 -- start start 2024-05-30T16:54:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.824+0000 7f64b8000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f64b00b4b60 0x7f64b00cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.824+0000 7f64b8000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 0x7f64b00d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.824+0000 7f64b8000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64b00ced40 con 0x7f64b00b3460 2024-05-30T16:54:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.824+0000 7f64b8000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f64b00d0e40 con 0x7f64b00b4b60 2024-05-30T16:54:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.824+0000 7f64b8000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64b00b3fe0 con 0x7f64b00d04a0 2024-05-30T16:54:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.824+0000 7f64b5600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f64b00b4b60 0x7f64b00cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.824+0000 7f64b6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f64b00b3460 0x7f64b00b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37486/0 (socket says 172.21.15.67:37486) 2024-05-30T16:54:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.825+0000 7f64b6000640 1 -- 172.21.15.67:0/2113065666 learned_addr learned my addr 172.21.15.67:0/2113065666 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.825+0000 7f64b5600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f64b00b4b60 0x7f64b00cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43078/0 (socket says 172.21.15.67:43078) 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.825+0000 7f64b4c00640 1 --2- 172.21.15.67:0/2113065666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 0x7f64b00d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.825+0000 7f64b5600640 1 -- 172.21.15.67:0/2113065666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 msgr2=0x7f64b00d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.825+0000 7f64b5600640 1 --2- 172.21.15.67:0/2113065666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 0x7f64b00d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.825+0000 7f64b5600640 1 -- 172.21.15.67:0/2113065666 >> v1:172.21.15.67:6789/0 conn(0x7f64b00b3460 legacy=0x7f64b00b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.825+0000 7f64b5600640 1 -- 172.21.15.67:0/2113065666 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b015fbb0 con 0x7f64b00b4b60 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.825+0000 7f64afe00640 1 -- 172.21.15.67:0/2113065666 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2453681620 0 0) 0x7f64a8001490 con 0x7f64b00b3460 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.826+0000 7f64b5600640 1 --2- 172.21.15.67:0/2113065666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 0x7f64b00cfaa0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f64a00097a0 tx=0x7f64a002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a08c40a62fe3d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.826+0000 7f64afe00640 1 -- 172.21.15.67:0/2113065666 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0037070 con 0x7f64b00b4b60 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.826+0000 7f64afe00640 1 -- 172.21.15.67:0/2113065666 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a0032440 con 0x7f64b00b4b60 2024-05-30T16:54:41.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.826+0000 7f64afe00640 1 -- 172.21.15.67:0/2113065666 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0032810 con 0x7f64b00b4b60 2024-05-30T16:54:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.827+0000 7f64b8000640 1 -- 172.21.15.67:0/2113065666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 msgr2=0x7f64b00cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.827+0000 7f64b8000640 1 --2- 172.21.15.67:0/2113065666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 0x7f64b00cfaa0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f64a00097a0 tx=0x7f64a002a180 comp rx=0 tx=0).stop 2024-05-30T16:54:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.827+0000 7f64b8000640 1 -- 172.21.15.67:0/2113065666 shutdown_connections 2024-05-30T16:54:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.827+0000 7f64b8000640 1 --2- 172.21.15.67:0/2113065666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 0x7f64b00d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.827+0000 7f64b8000640 1 --2- 172.21.15.67:0/2113065666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 0x7f64b00cfaa0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.827+0000 7f64b8000640 1 -- 172.21.15.67:0/2113065666 >> 172.21.15.67:0/2113065666 conn(0x7f64b006bae0 msgr2=0x7f64b006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.827+0000 7f64b8000640 1 -- 172.21.15.67:0/2113065666 shutdown_connections 2024-05-30T16:54:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.827+0000 7f64b8000640 1 -- 172.21.15.67:0/2113065666 wait complete. 2024-05-30T16:54:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.829+0000 7f64b8000640 1 Processor -- start 2024-05-30T16:54:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.829+0000 7f64b8000640 1 -- start start 2024-05-30T16:54:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b8000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b00b3460 0x7f64b009da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b8000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 0x7f64b015d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b8000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 0x7f64b00c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b8000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64b00cf580 con 0x7f64b00b4b60 2024-05-30T16:54:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b8000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64b00cf2a0 con 0x7f64b00b3460 2024-05-30T16:54:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b8000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64b00cffe0 con 0x7f64b00d04a0 2024-05-30T16:54:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 0x7f64b015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 0x7f64b015d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43090/0 (socket says 172.21.15.67:43090) 2024-05-30T16:54:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b4c00640 1 -- 172.21.15.67:0/1322752889 learned_addr learned my addr 172.21.15.67:0/1322752889 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:41.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b4c00640 1 -- 172.21.15.67:0/1322752889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 msgr2=0x7f64b00c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.830+0000 7f64b6000640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 0x7f64b00c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64b4c00640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 0x7f64b00c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64b4c00640 1 -- 172.21.15.67:0/1322752889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b00b3460 msgr2=0x7f64b009da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64b4c00640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b00b3460 0x7f64b009da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64b4c00640 1 -- 172.21.15.67:0/1322752889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b015deb0 con 0x7f64b00b4b60 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64b6000640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 0x7f64b00c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64b4c00640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 0x7f64b015d360 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f649c00d800 tx=0x7f649c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64ad600640 1 -- 172.21.15.67:0/1322752889 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c00b6c0 con 0x7f64b00b4b60 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64ad600640 1 -- 172.21.15.67:0/1322752889 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649c019070 con 0x7f64b00b4b60 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64ad600640 1 -- 172.21.15.67:0/1322752889 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c00ba80 con 0x7f64b00b4b60 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b009a2b0 con 0x7f64b00b4b60 2024-05-30T16:54:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.831+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b009a550 con 0x7f64b00b4b60 2024-05-30T16:54:41.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.833+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b00689e0 con 0x7f64b00b4b60 2024-05-30T16:54:41.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.834+0000 7f64ad600640 1 -- 172.21.15.67:0/1322752889 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f649c00bc20 con 0x7f64b00b4b60 2024-05-30T16:54:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.835+0000 7f64ad600640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649006d5e0 0x7f649006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:41.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.835+0000 7f64ad600640 1 -- 172.21.15.67:0/1322752889 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5793+0+0 (secure 0 0 0) 0x7f649c08ad30 con 0x7f64b00b4b60 2024-05-30T16:54:41.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.835+0000 7f64b5600640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649006d5e0 0x7f649006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:41.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.836+0000 7f64b5600640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649006d5e0 0x7f649006faa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64a0034000 tx=0x7f64a002c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:41.840+0000 7f64ad600640 1 -- 172.21.15.67:0/1322752889 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64b015deb0 con 0x7f64b00b4b60 2024-05-30T16:54:42.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:54:42.060 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:54:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.073+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64b00b3b70 con 0x7f64b00b4b60 2024-05-30T16:54:42.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.074+0000 7f64ad600640 1 -- 172.21.15.67:0/1322752889 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5181 (secure 0 0 0) 0x7f649c05a920 con 0x7f64b00b4b60 2024-05-30T16:54:42.075 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:42.075 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2932035652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2932035652},{"type":"v1","addr":"172.21.15.67:6837","nonce":2932035652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/327493818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":327493818},{"type":"v1","addr":"172.21.15.67:6836","nonce":327493818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1437754767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1437754767},{"type":"v1","addr":"172.21.15.169:6834","nonce":1437754767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:52:27.681449+0000","modified":"2024-05-30T16:52:30.749556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/898694760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":898694760},{"type":"v1","addr":"172.21.15.169:6835","nonce":898694760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5937,"qdb_cluster":[5937]},"id":4}]} 2024-05-30T16:54:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 36 2024-05-30T16:54:42.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649006d5e0 msgr2=0x7f649006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649006d5e0 0x7f649006faa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64a0034000 tx=0x7f64a002c000 comp rx=0 tx=0).stop 2024-05-30T16:54:42.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 msgr2=0x7f64b015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 0x7f64b015d360 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f649c00d800 tx=0x7f649c00dcc0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 shutdown_connections 2024-05-30T16:54:42.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f649006d5e0 0x7f649006faa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b00d04a0 0x7f64b00c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64b00b4b60 0x7f64b015d360 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 --2- 172.21.15.67:0/1322752889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b00b3460 0x7f64b009da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 >> 172.21.15.67:0/1322752889 conn(0x7f64b006bae0 msgr2=0x7f64b00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:42.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 shutdown_connections 2024-05-30T16:54:42.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.077+0000 7f64b8000640 1 -- 172.21.15.67:0/1322752889 wait complete. 2024-05-30T16:54:42.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:54:42.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.320+0000 7f09cce00640 1 Processor -- start 2024-05-30T16:54:42.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.321+0000 7f09cce00640 1 -- start start 2024-05-30T16:54:42.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09cce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f09c8156b30 0x7f09c80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09cce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09c80ad940 0x7f09c80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09cce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c8156f10 con 0x7f09c806ee40 2024-05-30T16:54:42.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09cce00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f09c806dda0 con 0x7f09c80ad940 2024-05-30T16:54:42.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09cce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f09c806e080 con 0x7f09c8156b30 2024-05-30T16:54:42.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09c7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09c80ad940 0x7f09c80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:42.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09c7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09c80ad940 0x7f09c80b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43098/0 (socket says 172.21.15.67:43098) 2024-05-30T16:54:42.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09c7e00640 1 -- 172.21.15.67:0/3685353007 learned_addr learned my addr 172.21.15.67:0/3685353007 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:42.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09c7e00640 1 -- 172.21.15.67:0/3685353007 >> v2:172.21.15.169:3300/0 conn(0x7f09c8156b30 msgr2=0x7f09c80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:42.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09c7e00640 1 --2- 172.21.15.67:0/3685353007 >> v2:172.21.15.169:3300/0 conn(0x7f09c8156b30 0x7f09c80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09c7e00640 1 -- 172.21.15.67:0/3685353007 >> v1:172.21.15.67:6789/0 conn(0x7f09c806ee40 legacy=0x7f09c80ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.322+0000 7f09c7e00640 1 -- 172.21.15.67:0/3685353007 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c806e360 con 0x7f09c80ad940 2024-05-30T16:54:42.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.323+0000 7f09c7e00640 1 --2- 172.21.15.67:0/3685353007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c80ad940 0x7f09c80b32c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f09c806c8b0 tx=0x7f09c002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fb58837af42bd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:42.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09c6a00640 1 -- 172.21.15.67:0/3685353007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0037070 con 0x7f09c80ad940 2024-05-30T16:54:42.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09c6a00640 1 -- 172.21.15.67:0/3685353007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c0032440 con 0x7f09c80ad940 2024-05-30T16:54:42.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09c6a00640 1 -- 172.21.15.67:0/3685353007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0032810 con 0x7f09c80ad940 2024-05-30T16:54:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09cce00640 1 -- 172.21.15.67:0/3685353007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c80ad940 msgr2=0x7f09c80b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09cce00640 1 --2- 172.21.15.67:0/3685353007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c80ad940 0x7f09c80b32c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f09c806c8b0 tx=0x7f09c002a200 comp rx=0 tx=0).stop 2024-05-30T16:54:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09cce00640 1 -- 172.21.15.67:0/3685353007 shutdown_connections 2024-05-30T16:54:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09cce00640 1 --2- 172.21.15.67:0/3685353007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c80ad940 0x7f09c80b32c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09cce00640 1 --2- 172.21.15.67:0/3685353007 >> v2:172.21.15.169:3300/0 conn(0x7f09c8156b30 0x7f09c80af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09cce00640 1 -- 172.21.15.67:0/3685353007 >> 172.21.15.67:0/3685353007 conn(0x7f09c80b1fc0 msgr2=0x7f09c80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09cce00640 1 -- 172.21.15.67:0/3685353007 shutdown_connections 2024-05-30T16:54:42.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.324+0000 7f09cce00640 1 -- 172.21.15.67:0/3685353007 wait complete. 2024-05-30T16:54:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.326+0000 7f09cce00640 1 Processor -- start 2024-05-30T16:54:42.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.326+0000 7f09cce00640 1 -- start start 2024-05-30T16:54:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.327+0000 7f09cce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09c806ee40 0x7f09c80b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.327+0000 7f09cce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09c80ad940 0x7f09c81f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.327+0000 7f09cce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c8156b30 0x7f09c81f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.327+0000 7f09cce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80afc20 con 0x7f09c8156b30 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.327+0000 7f09cce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80b2c30 con 0x7f09c806ee40 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.327+0000 7f09cce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f09c814b630 con 0x7f09c80ad940 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.327+0000 7f09c7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c8156b30 0x7f09c81f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.327+0000 7f09c7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c8156b30 0x7f09c81f8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43114/0 (socket says 172.21.15.67:43114) 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.327+0000 7f09c7e00640 1 -- 172.21.15.67:0/4128729968 learned_addr learned my addr 172.21.15.67:0/4128729968 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09c7e00640 1 -- 172.21.15.67:0/4128729968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09c80ad940 msgr2=0x7f09c81f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09c7e00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09c80ad940 0x7f09c81f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09c7e00640 1 -- 172.21.15.67:0/4128729968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09c806ee40 msgr2=0x7f09c80b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09c7e00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09c806ee40 0x7f09c80b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09c7e00640 1 -- 172.21.15.67:0/4128729968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c80cea50 con 0x7f09c8156b30 2024-05-30T16:54:42.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09c7e00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c8156b30 0x7f09c81f8670 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f09c00090b0 tx=0x7f09c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09bfe00640 1 -- 172.21.15.67:0/4128729968 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0041020 con 0x7f09c8156b30 2024-05-30T16:54:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09bfe00640 1 -- 172.21.15.67:0/4128729968 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c0030820 con 0x7f09c8156b30 2024-05-30T16:54:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09bfe00640 1 -- 172.21.15.67:0/4128729968 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0037040 con 0x7f09c8156b30 2024-05-30T16:54:42.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c80cf0a0 con 0x7f09c8156b30 2024-05-30T16:54:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.328+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c80b48b0 con 0x7f09c8156b30 2024-05-30T16:54:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.329+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c80689e0 con 0x7f09c8156b30 2024-05-30T16:54:42.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.331+0000 7f09bfe00640 1 -- 172.21.15.67:0/4128729968 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f09c0044020 con 0x7f09c8156b30 2024-05-30T16:54:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.331+0000 7f09bfe00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f09a406d5e0 0x7f09a406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.332+0000 7f09bfe00640 1 -- 172.21.15.67:0/4128729968 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5793+0+0 (secure 0 0 0) 0x7f09c00ab630 con 0x7f09c8156b30 2024-05-30T16:54:42.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.332+0000 7f09c7400640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f09a406d5e0 0x7f09a406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:42.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.332+0000 7f09c7400640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f09a406d5e0 0x7f09a406faa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f09b800ab90 tx=0x7f09b8009250 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:42.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.335+0000 7f09bfe00640 1 -- 172.21.15.67:0/4128729968 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09c00abe20 con 0x7f09c8156b30 2024-05-30T16:54:42.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.567+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09c815fb50 con 0x7f09c8156b30 2024-05-30T16:54:42.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.568+0000 7f09bfe00640 1 -- 172.21.15.67:0/4128729968 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5181 (secure 0 0 0) 0x7f09c004b030 con 0x7f09c8156b30 2024-05-30T16:54:42.569 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:42.569 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2932035652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2932035652},{"type":"v1","addr":"172.21.15.67:6837","nonce":2932035652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/327493818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":327493818},{"type":"v1","addr":"172.21.15.67:6836","nonce":327493818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1437754767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1437754767},{"type":"v1","addr":"172.21.15.169:6834","nonce":1437754767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:52:27.681449+0000","modified":"2024-05-30T16:52:30.749556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/898694760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":898694760},{"type":"v1","addr":"172.21.15.169:6835","nonce":898694760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5937,"qdb_cluster":[5937]},"id":4}]} 2024-05-30T16:54:42.569 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 36 2024-05-30T16:54:42.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.570+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f09a406d5e0 msgr2=0x7f09a406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.570+0000 7f09cce00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f09a406d5e0 0x7f09a406faa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f09b800ab90 tx=0x7f09b8009250 comp rx=0 tx=0).stop 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.570+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c8156b30 msgr2=0x7f09c81f8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.570+0000 7f09cce00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c8156b30 0x7f09c81f8670 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f09c00090b0 tx=0x7f09c00046c0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.571+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 shutdown_connections 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.571+0000 7f09cce00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f09a406d5e0 0x7f09a406faa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.571+0000 7f09cce00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09c8156b30 0x7f09c81f8670 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.571+0000 7f09cce00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09c80ad940 0x7f09c81f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.571+0000 7f09cce00640 1 --2- 172.21.15.67:0/4128729968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09c806ee40 0x7f09c80b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.571+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 >> 172.21.15.67:0/4128729968 conn(0x7f09c80b1fc0 msgr2=0x7f09c80b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.571+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 shutdown_connections 2024-05-30T16:54:42.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.571+0000 7f09cce00640 1 -- 172.21.15.67:0/4128729968 wait complete. 2024-05-30T16:54:42.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:54:42.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.784+0000 7f1ea7e00640 1 Processor -- start 2024-05-30T16:54:42.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.784+0000 7f1ea7e00640 1 -- start start 2024-05-30T16:54:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.786+0000 7f1ea7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1ea8156b30 0x7f1ea80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.786+0000 7f1ea7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ea806ee40 0x7f1ea80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.786+0000 7f1ea7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea8156f10 con 0x7f1ea806f2f0 2024-05-30T16:54:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.786+0000 7f1ea7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea806dda0 con 0x7f1ea806ee40 2024-05-30T16:54:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.786+0000 7f1ea7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea806e080 con 0x7f1ea8156b30 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.786+0000 7f1ea6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ea806ee40 0x7f1ea80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.786+0000 7f1ea6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ea806ee40 0x7f1ea80ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43118/0 (socket says 172.21.15.67:43118) 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.786+0000 7f1ea6000640 1 -- 172.21.15.67:0/1247653727 learned_addr learned my addr 172.21.15.67:0/1247653727 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.786+0000 7f1ea7400640 1 --1- 172.21.15.67:0/1247653727 >> v1:172.21.15.67:6789/0 conn(0x7f1ea806f2f0 0x7f1ea80b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37494/0 (socket says 172.21.15.67:37494) 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.787+0000 7f1ea6a00640 1 --2- 172.21.15.67:0/1247653727 >> v2:172.21.15.169:3300/0 conn(0x7f1ea8156b30 0x7f1ea80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.787+0000 7f1ea6000640 1 -- 172.21.15.67:0/1247653727 >> v2:172.21.15.169:3300/0 conn(0x7f1ea8156b30 msgr2=0x7f1ea80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.787+0000 7f1ea6000640 1 --2- 172.21.15.67:0/1247653727 >> v2:172.21.15.169:3300/0 conn(0x7f1ea8156b30 0x7f1ea80af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.787+0000 7f1ea6000640 1 -- 172.21.15.67:0/1247653727 >> v1:172.21.15.67:6789/0 conn(0x7f1ea806f2f0 legacy=0x7f1ea80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.787+0000 7f1ea6000640 1 -- 172.21.15.67:0/1247653727 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea806e360 con 0x7f1ea806ee40 2024-05-30T16:54:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.787+0000 7f1ea6a00640 1 --2- 172.21.15.67:0/1247653727 >> v2:172.21.15.169:3300/0 conn(0x7f1ea8156b30 0x7f1ea80af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.788+0000 7f1ea6000640 1 --2- 172.21.15.67:0/1247653727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806ee40 0x7f1ea80ad230 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1ea806c8b0 tx=0x7f1e9402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dc23d05ff9de33 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.788+0000 7f1ea5600640 1 -- 172.21.15.67:0/1247653727 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94037070 con 0x7f1ea806ee40 2024-05-30T16:54:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.788+0000 7f1ea5600640 1 -- 172.21.15.67:0/1247653727 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e94032440 con 0x7f1ea806ee40 2024-05-30T16:54:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.788+0000 7f1ea5600640 1 -- 172.21.15.67:0/1247653727 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94032810 con 0x7f1ea806ee40 2024-05-30T16:54:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.789+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1247653727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806ee40 msgr2=0x7f1ea80ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.789+0000 7f1ea7e00640 1 --2- 172.21.15.67:0/1247653727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806ee40 0x7f1ea80ad230 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1ea806c8b0 tx=0x7f1e9402a1f0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.789+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1247653727 shutdown_connections 2024-05-30T16:54:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.789+0000 7f1ea7e00640 1 --2- 172.21.15.67:0/1247653727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806ee40 0x7f1ea80ad230 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.789+0000 7f1ea7e00640 1 --2- 172.21.15.67:0/1247653727 >> v2:172.21.15.169:3300/0 conn(0x7f1ea8156b30 0x7f1ea80af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.789+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1247653727 >> 172.21.15.67:0/1247653727 conn(0x7f1ea80b1fc0 msgr2=0x7f1ea80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.789+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1247653727 shutdown_connections 2024-05-30T16:54:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.789+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1247653727 wait complete. 2024-05-30T16:54:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.790+0000 7f1ea7e00640 1 Processor -- start 2024-05-30T16:54:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.791+0000 7f1ea7e00640 1 -- start start 2024-05-30T16:54:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.791+0000 7f1ea7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ea806ee40 0x7f1ea8145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.791+0000 7f1ea7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806f2f0 0x7f1ea8158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.791+0000 7f1ea7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ea8156b30 0x7f1ea815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.791+0000 7f1ea7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea80afc20 con 0x7f1ea806f2f0 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.791+0000 7f1ea7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea80b2c30 con 0x7f1ea806ee40 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.791+0000 7f1ea7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea8158f90 con 0x7f1ea8156b30 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806f2f0 0x7f1ea8158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806f2f0 0x7f1ea8158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43124/0 (socket says 172.21.15.67:43124) 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6000640 1 -- 172.21.15.67:0/1764406519 learned_addr learned my addr 172.21.15.67:0/1764406519 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6a00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ea806ee40 0x7f1ea8145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea7400640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ea8156b30 0x7f1ea815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6000640 1 -- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ea8156b30 msgr2=0x7f1ea815e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6000640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ea8156b30 0x7f1ea815e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6000640 1 -- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ea806ee40 msgr2=0x7f1ea8145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6000640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ea806ee40 0x7f1ea8145380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6000640 1 -- 172.21.15.67:0/1764406519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea8157c90 con 0x7f1ea806f2f0 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea6a00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ea806ee40 0x7f1ea8145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.792+0000 7f1ea7400640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ea8156b30 0x7f1ea815e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.793+0000 7f1ea6000640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806f2f0 0x7f1ea8158a50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1e94000c00 tx=0x7f1e940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.793+0000 7f1e9ea00640 1 -- 172.21.15.67:0/1764406519 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94037040 con 0x7f1ea806f2f0 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.793+0000 7f1e9ea00640 1 -- 172.21.15.67:0/1764406519 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e94042070 con 0x7f1ea806f2f0 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.793+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea8157f70 con 0x7f1ea806f2f0 2024-05-30T16:54:42.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.793+0000 7f1e9ea00640 1 -- 172.21.15.67:0/1764406519 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94030bb0 con 0x7f1ea806f2f0 2024-05-30T16:54:42.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.793+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea8158530 con 0x7f1ea806f2f0 2024-05-30T16:54:42.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.794+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea80689e0 con 0x7f1ea806f2f0 2024-05-30T16:54:42.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.795+0000 7f1e9ea00640 1 -- 172.21.15.67:0/1764406519 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f1e94047460 con 0x7f1ea806f2f0 2024-05-30T16:54:42.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.796+0000 7f1e9ea00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1e8806d5e0 0x7f1e8806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:42.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.796+0000 7f1e9ea00640 1 -- 172.21.15.67:0/1764406519 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5793+0+0 (secure 0 0 0) 0x7f1e940ab4a0 con 0x7f1ea806f2f0 2024-05-30T16:54:42.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.796+0000 7f1ea6a00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1e8806d5e0 0x7f1e8806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:42.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.797+0000 7f1ea6a00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1e8806d5e0 0x7f1e8806faa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1e98006fd0 tx=0x7f1e98008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:42.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.802+0000 7f1e9ea00640 1 -- 172.21.15.67:0/1764406519 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e9407b090 con 0x7f1ea806f2f0 2024-05-30T16:54:43.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:42.999+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ea80b05c0 con 0x7f1ea806f2f0 2024-05-30T16:54:43.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.003+0000 7f1e9ea00640 1 -- 172.21.15.67:0/1764406519 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16805 (secure 0 0 0) 0x7f1e9407a990 con 0x7f1ea806f2f0 2024-05-30T16:54:43.004 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:43.004 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:54:39.101022+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:52:25.604028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-30T16:52:26.652938+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1e8806d5e0 msgr2=0x7f1e8806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1e8806d5e0 0x7f1e8806faa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1e98006fd0 tx=0x7f1e98008040 comp rx=0 tx=0).stop 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806f2f0 msgr2=0x7f1ea8158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806f2f0 0x7f1ea8158a50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1e94000c00 tx=0x7f1e940046c0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 shutdown_connections 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1e8806d5e0 0x7f1e8806faa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ea8156b30 0x7f1ea815e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ea806f2f0 0x7f1ea8158a50 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 --2- 172.21.15.67:0/1764406519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ea806ee40 0x7f1ea8145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 >> 172.21.15.67:0/1764406519 conn(0x7f1ea80b1fc0 msgr2=0x7f1ea80b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 shutdown_connections 2024-05-30T16:54:43.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.006+0000 7f1ea7e00640 1 -- 172.21.15.67:0/1764406519 wait complete. 2024-05-30T16:54:43.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-30T16:54:43.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.231+0000 7fc5a6a00640 1 Processor -- start 2024-05-30T16:54:43.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.231+0000 7fc5a6a00640 1 -- start start 2024-05-30T16:54:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.233+0000 7fc5a6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc5a015b3a0 0x7fc5a00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.233+0000 7fc5a6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a006dc20 0x7fc5a015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.233+0000 7fc5a6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5a0072c00 0x7fc5a0073050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.233+0000 7fc5a6a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc5a006f980 con 0x7fc5a015b3a0 2024-05-30T16:54:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.233+0000 7fc5a6a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc5a0073590 con 0x7fc5a0072c00 2024-05-30T16:54:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.233+0000 7fc5a6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a00ae0c0 con 0x7fc5a006dc20 2024-05-30T16:54:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.233+0000 7fc5a5600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc5a015b3a0 0x7fc5a00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.233+0000 7fc5a5600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc5a015b3a0 0x7fc5a00b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43128/0 (socket says 172.21.15.67:43128) 2024-05-30T16:54:43.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.233+0000 7fc5a5600640 1 -- 172.21.15.67:0/4229302120 learned_addr learned my addr 172.21.15.67:0/4229302120 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.234+0000 7fc5a4c00640 1 --2- 172.21.15.67:0/4229302120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a006dc20 0x7fc5a015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.234+0000 7fc5a6000640 1 --2- 172.21.15.67:0/4229302120 >> v2:172.21.15.169:3300/0 conn(0x7fc5a0072c00 0x7fc5a0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.234+0000 7fc5a5600640 1 -- 172.21.15.67:0/4229302120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a006dc20 msgr2=0x7fc5a015f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.234+0000 7fc5a5600640 1 --2- 172.21.15.67:0/4229302120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a006dc20 0x7fc5a015f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.234+0000 7fc5a5600640 1 -- 172.21.15.67:0/4229302120 >> v2:172.21.15.169:3300/0 conn(0x7fc5a0072c00 msgr2=0x7fc5a0073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.234+0000 7fc5a5600640 1 --2- 172.21.15.67:0/4229302120 >> v2:172.21.15.169:3300/0 conn(0x7fc5a0072c00 0x7fc5a0073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.234+0000 7fc5a5600640 1 -- 172.21.15.67:0/4229302120 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a00afd60 con 0x7fc5a015b3a0 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.234+0000 7fc5a4c00640 1 --2- 172.21.15.67:0/4229302120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a006dc20 0x7fc5a015f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.234+0000 7fc5a6000640 1 --2- 172.21.15.67:0/4229302120 >> v2:172.21.15.169:3300/0 conn(0x7fc5a0072c00 0x7fc5a0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:43.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.235+0000 7fc5a5600640 1 --2- 172.21.15.67:0/4229302120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a015b3a0 0x7fc5a00b32c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc5a006fd40 tx=0x7fc59402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1041ea366de86cf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.235+0000 7fc59fe00640 1 -- 172.21.15.67:0/4229302120 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594037070 con 0x7fc5a015b3a0 2024-05-30T16:54:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc59fe00640 1 -- 172.21.15.67:0/4229302120 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc594032440 con 0x7fc5a015b3a0 2024-05-30T16:54:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc59fe00640 1 -- 172.21.15.67:0/4229302120 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594032810 con 0x7fc5a015b3a0 2024-05-30T16:54:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4229302120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a015b3a0 msgr2=0x7fc5a00b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4229302120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a015b3a0 0x7fc5a00b32c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc5a006fd40 tx=0x7fc59402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4229302120 shutdown_connections 2024-05-30T16:54:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4229302120 >> v2:172.21.15.169:3300/0 conn(0x7fc5a0072c00 0x7fc5a0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4229302120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a006dc20 0x7fc5a015f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4229302120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a015b3a0 0x7fc5a00b32c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4229302120 >> 172.21.15.67:0/4229302120 conn(0x7fc5a00b1fc0 msgr2=0x7fc5a00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4229302120 shutdown_connections 2024-05-30T16:54:43.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.236+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4229302120 wait complete. 2024-05-30T16:54:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.238+0000 7fc5a6a00640 1 Processor -- start 2024-05-30T16:54:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.238+0000 7fc5a6a00640 1 -- start start 2024-05-30T16:54:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.239+0000 7fc5a6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a006dc20 0x7fc5a0148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.239+0000 7fc5a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a0072c00 0x7fc5a00ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.239+0000 7fc5a6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a015b3a0 0x7fc5a00aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.239+0000 7fc5a6a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a00afa80 con 0x7fc5a0072c00 2024-05-30T16:54:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.239+0000 7fc5a6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a015fee0 con 0x7fc5a006dc20 2024-05-30T16:54:43.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.239+0000 7fc5a6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a015b780 con 0x7fc5a015b3a0 2024-05-30T16:54:43.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a0072c00 0x7fc5a00ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a0072c00 0x7fc5a00ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43138/0 (socket says 172.21.15.67:43138) 2024-05-30T16:54:43.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a4c00640 1 -- 172.21.15.67:0/4290445791 learned_addr learned my addr 172.21.15.67:0/4290445791 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a6000640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a015b3a0 0x7fc5a00aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a5600640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a006dc20 0x7fc5a0148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a4c00640 1 -- 172.21.15.67:0/4290445791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a015b3a0 msgr2=0x7fc5a00aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a4c00640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a015b3a0 0x7fc5a00aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a4c00640 1 -- 172.21.15.67:0/4290445791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a006dc20 msgr2=0x7fc5a0148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a4c00640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a006dc20 0x7fc5a0148770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a4c00640 1 -- 172.21.15.67:0/4290445791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a0149cc0 con 0x7fc5a0072c00 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.240+0000 7fc5a6000640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a015b3a0 0x7fc5a00aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.241+0000 7fc5a4c00640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a0072c00 0x7fc5a00ae550 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c820 tx=0x7fc58c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.241+0000 7fc59d600640 1 -- 172.21.15.67:0/4290445791 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c015070 con 0x7fc5a0072c00 2024-05-30T16:54:43.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.241+0000 7fc59d600640 1 -- 172.21.15.67:0/4290445791 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58c007a30 con 0x7fc5a0072c00 2024-05-30T16:54:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.241+0000 7fc59d600640 1 -- 172.21.15.67:0/4290445791 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c007de0 con 0x7fc5a0072c00 2024-05-30T16:54:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.241+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a0149fa0 con 0x7fc5a0072c00 2024-05-30T16:54:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.241+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a014a560 con 0x7fc5a0072c00 2024-05-30T16:54:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.242+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a00689e0 con 0x7fc5a0072c00 2024-05-30T16:54:43.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.243+0000 7fc59d600640 1 -- 172.21.15.67:0/4290445791 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fc58c018070 con 0x7fc5a0072c00 2024-05-30T16:54:43.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.244+0000 7fc59d600640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc58006d5e0 0x7fc58006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.244+0000 7fc59d600640 1 -- 172.21.15.67:0/4290445791 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5793+0+0 (secure 0 0 0) 0x7fc58c08a430 con 0x7fc5a0072c00 2024-05-30T16:54:43.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.244+0000 7fc5a5600640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc58006d5e0 0x7fc58006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.245+0000 7fc5a5600640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc58006d5e0 0x7fc58006faa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc594034000 tx=0x7fc59402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:43.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.249+0000 7fc59d600640 1 -- 172.21.15.67:0/4290445791 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc58c05a2b0 con 0x7fc5a0072c00 2024-05-30T16:54:43.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.487+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc5a00b2e90 con 0x7fc5a0072c00 2024-05-30T16:54:43.488 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-05-30T16:54:43.488+0000 7f4982000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-30T16:54:43.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.492+0000 7fc59d600640 1 -- 172.21.15.67:0/4290445791 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7fc58c059bb0 con 0x7fc5a0072c00 2024-05-30T16:54:43.493 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc58006d5e0 msgr2=0x7fc58006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc58006d5e0 0x7fc58006faa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc594034000 tx=0x7fc59402e000 comp rx=0 tx=0).stop 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a0072c00 msgr2=0x7fc5a00ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a0072c00 0x7fc5a00ae550 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c820 tx=0x7fc58c00cce0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 shutdown_connections 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc58006d5e0 0x7fc58006faa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a015b3a0 0x7fc5a00aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a0072c00 0x7fc5a00ae550 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 --2- 172.21.15.67:0/4290445791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5a006dc20 0x7fc5a0148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 >> 172.21.15.67:0/4290445791 conn(0x7fc5a00b1fc0 msgr2=0x7fc5a0151bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:43.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 shutdown_connections 2024-05-30T16:54:43.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.495+0000 7fc5a6a00640 1 -- 172.21.15.67:0/4290445791 wait complete. 2024-05-30T16:54:43.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-30T16:54:43.615 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T16:54:43.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.703+0000 7f6994400640 1 Processor -- start 2024-05-30T16:54:43.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.704+0000 7f6994400640 1 -- start start 2024-05-30T16:54:43.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.705+0000 7f6994400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f698c15d5d0 0x7f698c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.705+0000 7f6994400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f698c0b32c0 0x7f698c0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.705+0000 7f6994400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698c15cb10 con 0x7f698c0d0640 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.705+0000 7f6994400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f698c0adbb0 con 0x7f698c0b32c0 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.705+0000 7f6994400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f698c0ad5f0 con 0x7f698c15d5d0 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.705+0000 7f6991a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f698c0d0640 0x7f698c0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37508/0 (socket says 172.21.15.67:37508) 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.705+0000 7f6991a00640 1 -- 172.21.15.67:0/2102255471 learned_addr learned my addr 172.21.15.67:0/2102255471 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.706+0000 7f6992400640 1 --2- 172.21.15.67:0/2102255471 >> v2:172.21.15.67:3300/0 conn(0x7f698c0b32c0 0x7f698c0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.706+0000 7f6991000640 1 --2- 172.21.15.67:0/2102255471 >> v2:172.21.15.169:3300/0 conn(0x7f698c15d5d0 0x7f698c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.706+0000 7f6992400640 1 -- 172.21.15.67:0/2102255471 >> v2:172.21.15.169:3300/0 conn(0x7f698c15d5d0 msgr2=0x7f698c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.706+0000 7f6992400640 1 --2- 172.21.15.67:0/2102255471 >> v2:172.21.15.169:3300/0 conn(0x7f698c15d5d0 0x7f698c0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.706+0000 7f6992400640 1 -- 172.21.15.67:0/2102255471 >> v1:172.21.15.67:6789/0 conn(0x7f698c0d0640 legacy=0x7f698c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.706+0000 7f6992400640 1 -- 172.21.15.67:0/2102255471 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c0ad8d0 con 0x7f698c0b32c0 2024-05-30T16:54:43.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.706+0000 7f698be00640 1 -- 172.21.15.67:0/2102255471 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829434273 0 0) 0x7f698c15cb10 con 0x7f698c0d0640 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.707+0000 7f6992400640 1 --2- 172.21.15.67:0/2102255471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 0x7f698c0b3700 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f698c074120 tx=0x7f698402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4e02302daee80a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.707+0000 7f698be00640 1 -- 172.21.15.67:0/2102255471 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984007740 con 0x7f698c0b32c0 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.707+0000 7f698be00640 1 -- 172.21.15.67:0/2102255471 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69840078e0 con 0x7f698c0b32c0 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.707+0000 7f698be00640 1 -- 172.21.15.67:0/2102255471 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984007cb0 con 0x7f698c0b32c0 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.708+0000 7f6994400640 1 -- 172.21.15.67:0/2102255471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 msgr2=0x7f698c0b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.708+0000 7f6994400640 1 --2- 172.21.15.67:0/2102255471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 0x7f698c0b3700 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f698c074120 tx=0x7f698402a5f0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.708+0000 7f6994400640 1 -- 172.21.15.67:0/2102255471 shutdown_connections 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.708+0000 7f6994400640 1 --2- 172.21.15.67:0/2102255471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 0x7f698c0b3700 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.708+0000 7f6994400640 1 --2- 172.21.15.67:0/2102255471 >> v2:172.21.15.169:3300/0 conn(0x7f698c15d5d0 0x7f698c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.708+0000 7f6994400640 1 -- 172.21.15.67:0/2102255471 >> 172.21.15.67:0/2102255471 conn(0x7f698c06bd20 msgr2=0x7f698c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:43.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.708+0000 7f6994400640 1 -- 172.21.15.67:0/2102255471 shutdown_connections 2024-05-30T16:54:43.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.708+0000 7f6994400640 1 -- 172.21.15.67:0/2102255471 wait complete. 2024-05-30T16:54:43.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.710+0000 7f6994400640 1 Processor -- start 2024-05-30T16:54:43.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.710+0000 7f6994400640 1 -- start start 2024-05-30T16:54:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.711+0000 7f6994400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 0x7f698c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.711+0000 7f6994400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f698c0d0640 0x7f698c1f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.711+0000 7f6994400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f698c15d5d0 0x7f698c1f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.711+0000 7f6994400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f698c15da50 con 0x7f698c0b32c0 2024-05-30T16:54:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.711+0000 7f6994400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f698c15d0b0 con 0x7f698c0d0640 2024-05-30T16:54:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.711+0000 7f6994400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f698c0ad2a0 con 0x7f698c15d5d0 2024-05-30T16:54:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.711+0000 7f6991a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 0x7f698c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6991a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 0x7f698c154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43150/0 (socket says 172.21.15.67:43150) 2024-05-30T16:54:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6991a00640 1 -- 172.21.15.67:0/503392429 learned_addr learned my addr 172.21.15.67:0/503392429 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:43.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6991a00640 1 -- 172.21.15.67:0/503392429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f698c15d5d0 msgr2=0x7f698c1f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6991a00640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f698c15d5d0 0x7f698c1f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6991a00640 1 -- 172.21.15.67:0/503392429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f698c0d0640 msgr2=0x7f698c1f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6991a00640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f698c0d0640 0x7f698c1f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6991a00640 1 -- 172.21.15.67:0/503392429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c14f540 con 0x7f698c0b32c0 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6991a00640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 0x7f698c154f00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f698000c460 tx=0x7f698000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6989600640 1 -- 172.21.15.67:0/503392429 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980018070 con 0x7f698c0b32c0 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6989600640 1 -- 172.21.15.67:0/503392429 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6980007500 con 0x7f698c0b32c0 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.712+0000 7f6989600640 1 -- 172.21.15.67:0/503392429 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980004a20 con 0x7f698c0b32c0 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.713+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c14e780 con 0x7f698c0b32c0 2024-05-30T16:54:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.713+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c14ed40 con 0x7f698c0b32c0 2024-05-30T16:54:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.714+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c0689e0 con 0x7f698c0b32c0 2024-05-30T16:54:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.715+0000 7f6989600640 1 -- 172.21.15.67:0/503392429 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f6980004bc0 con 0x7f698c0b32c0 2024-05-30T16:54:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.715+0000 7f6989600640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f696c06d5e0 0x7f696c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.716+0000 7f6989600640 1 -- 172.21.15.67:0/503392429 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5851+0+0 (secure 0 0 0) 0x7f6980089990 con 0x7f698c0b32c0 2024-05-30T16:54:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.716+0000 7f6991000640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f696c06d5e0 0x7f696c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:43.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.716+0000 7f6991000640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f696c06d5e0 0x7f696c06faa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f698c145140 tx=0x7f6978008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:43.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.720+0000 7f6989600640 1 -- 172.21.15.67:0/503392429 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6980059710 con 0x7f698c0b32c0 2024-05-30T16:54:43.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:43.954+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f698c06f220 con 0x7f698c0b32c0 2024-05-30T16:54:44.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.496+0000 7f6989600640 1 -- 172.21.15.67:0/503392429 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f6980059010 con 0x7f698c0b32c0 2024-05-30T16:54:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.500+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f696c06d5e0 msgr2=0x7f696c06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.500+0000 7f6994400640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f696c06d5e0 0x7f696c06faa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f698c145140 tx=0x7f6978008040 comp rx=0 tx=0).stop 2024-05-30T16:54:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.500+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 msgr2=0x7f698c154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.500+0000 7f6994400640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 0x7f698c154f00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f698000c460 tx=0x7f698000c920 comp rx=0 tx=0).stop 2024-05-30T16:54:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.500+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 shutdown_connections 2024-05-30T16:54:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.500+0000 7f6994400640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f696c06d5e0 0x7f696c06faa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:44.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.500+0000 7f6994400640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f698c15d5d0 0x7f698c1f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:44.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.500+0000 7f6994400640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f698c0d0640 0x7f698c1f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:44.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.500+0000 7f6994400640 1 --2- 172.21.15.67:0/503392429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f698c0b32c0 0x7f698c154f00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:44.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.501+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 >> 172.21.15.67:0/503392429 conn(0x7f698c06bd20 msgr2=0x7f698c15dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:44.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.501+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 shutdown_connections 2024-05-30T16:54:44.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.501+0000 7f6994400640 1 -- 172.21.15.67:0/503392429 wait complete. 2024-05-30T16:54:44.521 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-30T16:54:44.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.731+0000 7f0cb3e00640 1 Processor -- start 2024-05-30T16:54:44.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.731+0000 7f0cb3e00640 1 -- start start 2024-05-30T16:54:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.733+0000 7f0cb3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0cb415e810 0x7f0cb406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.733+0000 7f0cb3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0cb40cb290 0x7f0cb40ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.733+0000 7f0cb3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb415ebf0 con 0x7f0cb40caa90 2024-05-30T16:54:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.733+0000 7f0cb3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb406fba0 con 0x7f0cb415e810 2024-05-30T16:54:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.733+0000 7f0cb3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb40d1010 con 0x7f0cb40cb290 2024-05-30T16:54:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.733+0000 7f0cb2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0cb415e810 0x7f0cb406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.733+0000 7f0cb2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0cb415e810 0x7f0cb406c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43154/0 (socket says 172.21.15.67:43154) 2024-05-30T16:54:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.733+0000 7f0cb2a00640 1 -- 172.21.15.67:0/2756022416 learned_addr learned my addr 172.21.15.67:0/2756022416 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.734+0000 7f0cb2a00640 1 -- 172.21.15.67:0/2756022416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0cb40cb290 msgr2=0x7f0cb40ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.734+0000 7f0cb2a00640 1 --2- 172.21.15.67:0/2756022416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0cb40cb290 0x7f0cb40ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.734+0000 7f0cb2a00640 1 -- 172.21.15.67:0/2756022416 >> v1:172.21.15.169:6789/0 conn(0x7f0cb40caa90 legacy=0x7f0cb40cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.734+0000 7f0cb2a00640 1 -- 172.21.15.67:0/2756022416 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb406ee40 con 0x7f0cb415e810 2024-05-30T16:54:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.735+0000 7f0cb2a00640 1 --2- 172.21.15.67:0/2756022416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb415e810 0x7f0cb406c310 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009e50 tx=0x7f0ca402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbb82b6421a9f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.735+0000 7f0cb1600640 1 -- 172.21.15.67:0/2756022416 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4036070 con 0x7f0cb415e810 2024-05-30T16:54:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.735+0000 7f0cb1600640 1 -- 172.21.15.67:0/2756022416 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca4031600 con 0x7f0cb415e810 2024-05-30T16:54:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.735+0000 7f0cb1600640 1 -- 172.21.15.67:0/2756022416 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca40319d0 con 0x7f0cb415e810 2024-05-30T16:54:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.736+0000 7f0cb3e00640 1 -- 172.21.15.67:0/2756022416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb415e810 msgr2=0x7f0cb406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.736+0000 7f0cb3e00640 1 --2- 172.21.15.67:0/2756022416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb415e810 0x7f0cb406c310 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009e50 tx=0x7f0ca402a630 comp rx=0 tx=0).stop 2024-05-30T16:54:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.736+0000 7f0cb3e00640 1 -- 172.21.15.67:0/2756022416 shutdown_connections 2024-05-30T16:54:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.736+0000 7f0cb3e00640 1 --2- 172.21.15.67:0/2756022416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0cb40cb290 0x7f0cb40ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.736+0000 7f0cb3e00640 1 --2- 172.21.15.67:0/2756022416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb415e810 0x7f0cb406c310 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.736+0000 7f0cb3e00640 1 -- 172.21.15.67:0/2756022416 >> 172.21.15.67:0/2756022416 conn(0x7f0cb40b2ed0 msgr2=0x7f0cb40b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.736+0000 7f0cb3e00640 1 -- 172.21.15.67:0/2756022416 shutdown_connections 2024-05-30T16:54:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.736+0000 7f0cb3e00640 1 -- 172.21.15.67:0/2756022416 wait complete. 2024-05-30T16:54:44.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.739+0000 7f0cb3e00640 1 Processor -- start 2024-05-30T16:54:44.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.739+0000 7f0cb3e00640 1 -- start start 2024-05-30T16:54:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.740+0000 7f0cb3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0cb40caa90 0x7f0cb4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.740+0000 7f0cb3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb40cb290 0x7f0cb41618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.740+0000 7f0cb3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb415e810 0x7f0cb41f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.740+0000 7f0cb3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb40d0250 con 0x7f0cb40cb290 2024-05-30T16:54:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.740+0000 7f0cb3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb406f830 con 0x7f0cb415e810 2024-05-30T16:54:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.740+0000 7f0cb3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb40d03d0 con 0x7f0cb40caa90 2024-05-30T16:54:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.740+0000 7f0cb2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb40cb290 0x7f0cb41618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.741+0000 7f0cb2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb40cb290 0x7f0cb41618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43160/0 (socket says 172.21.15.67:43160) 2024-05-30T16:54:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.741+0000 7f0cb2000640 1 -- 172.21.15.67:0/3764104470 learned_addr learned my addr 172.21.15.67:0/3764104470 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.741+0000 7f0cb2a00640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0cb40caa90 0x7f0cb4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.741+0000 7f0cb2000640 1 -- 172.21.15.67:0/3764104470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0cb40caa90 msgr2=0x7f0cb4160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.741+0000 7f0cb2000640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0cb40caa90 0x7f0cb4160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.741+0000 7f0cb2000640 1 -- 172.21.15.67:0/3764104470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb415e810 msgr2=0x7f0cb41f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.741+0000 7f0cb2000640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb415e810 0x7f0cb41f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.741+0000 7f0cb2000640 1 -- 172.21.15.67:0/3764104470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb4157be0 con 0x7f0cb40cb290 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.741+0000 7f0cb2000640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb40cb290 0x7f0cb41618e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00b450 tx=0x7f0c9c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.742+0000 7f0cab400640 1 -- 172.21.15.67:0/3764104470 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c015020 con 0x7f0cb40cb290 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.742+0000 7f0cab400640 1 -- 172.21.15.67:0/3764104470 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9c011890 con 0x7f0cb40cb290 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.742+0000 7f0cab400640 1 -- 172.21.15.67:0/3764104470 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c011c80 con 0x7f0cb40cb290 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.742+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb4157ec0 con 0x7f0cb40cb290 2024-05-30T16:54:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.742+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb4158480 con 0x7f0cb40cb290 2024-05-30T16:54:44.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.743+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb40690f0 con 0x7f0cb40cb290 2024-05-30T16:54:44.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.744+0000 7f0cab400640 1 -- 172.21.15.67:0/3764104470 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f0c9c018020 con 0x7f0cb40cb290 2024-05-30T16:54:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.745+0000 7f0cab400640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0c9006d5e0 0x7f0c9006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.745+0000 7f0cab400640 1 -- 172.21.15.67:0/3764104470 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5851+0+0 (secure 0 0 0) 0x7f0c9c08a6c0 con 0x7f0cb40cb290 2024-05-30T16:54:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.745+0000 7f0cb2a00640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0c9006d5e0 0x7f0c9006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.746+0000 7f0cb2a00640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0c9006d5e0 0x7f0c9006faa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4033000 tx=0x7f0ca4009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.749+0000 7f0cab400640 1 -- 172.21.15.67:0/3764104470 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c9c01f020 con 0x7f0cb40cb290 2024-05-30T16:54:44.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:44.953+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0cb406dc90 con 0x7f0cb40cb290 2024-05-30T16:54:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.499+0000 7f0cab400640 1 -- 172.21.15.67:0/3764104470 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) ==== 181+0+0 (secure 0 0 0) 0x7f0c9c00d040 con 0x7f0cb40cb290 2024-05-30T16:54:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-05-30T16:54:45.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.503+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0c9006d5e0 msgr2=0x7f0c9006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.503+0000 7f0cb3e00640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0c9006d5e0 0x7f0c9006faa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4033000 tx=0x7f0ca4009000 comp rx=0 tx=0).stop 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.503+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb40cb290 msgr2=0x7f0cb41618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.503+0000 7f0cb3e00640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb40cb290 0x7f0cb41618e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00b450 tx=0x7f0c9c00b910 comp rx=0 tx=0).stop 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.504+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 shutdown_connections 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.504+0000 7f0cb3e00640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0c9006d5e0 0x7f0c9006faa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.504+0000 7f0cb3e00640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cb415e810 0x7f0cb41f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.504+0000 7f0cb3e00640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0cb40cb290 0x7f0cb41618e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.504+0000 7f0cb3e00640 1 --2- 172.21.15.67:0/3764104470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0cb40caa90 0x7f0cb4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.504+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 >> 172.21.15.67:0/3764104470 conn(0x7f0cb40b2ed0 msgr2=0x7f0cb40ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.504+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 shutdown_connections 2024-05-30T16:54:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.504+0000 7f0cb3e00640 1 -- 172.21.15.67:0/3764104470 wait complete. 2024-05-30T16:54:45.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-30T16:54:45.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.741+0000 7f80b3200640 1 Processor -- start 2024-05-30T16:54:45.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.742+0000 7f80b3200640 1 -- start start 2024-05-30T16:54:45.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.744+0000 7f80b3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80b415fa00 0x7f80b406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.744+0000 7f80b3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b406fa00 con 0x7f80b40d0b40 2024-05-30T16:54:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.744+0000 7f80b3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b406eeb0 con 0x7f80b41600f0 2024-05-30T16:54:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.744+0000 7f80b3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f80b406f160 con 0x7f80b415fa00 2024-05-30T16:54:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.746+0000 7f80b1400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f80b41600f0 0x7f80b40d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:59518/0 (socket says 172.21.15.67:59518) 2024-05-30T16:54:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.746+0000 7f80b1400640 1 -- 172.21.15.67:0/3032185997 learned_addr learned my addr 172.21.15.67:0/3032185997 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:45.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.748+0000 7f80b1e00640 1 --2- 172.21.15.67:0/3032185997 >> v2:172.21.15.67:3300/0 conn(0x7f80b415fa00 0x7f80b406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.748+0000 7f80b0a00640 1 -- 172.21.15.67:0/3032185997 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1814501560 0 0) 0x7f80b406eeb0 con 0x7f80b41600f0 2024-05-30T16:54:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.748+0000 7f80b0a00640 1 -- 172.21.15.67:0/3032185997 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8098003520 con 0x7f80b41600f0 2024-05-30T16:54:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.749+0000 7f80b1e00640 1 -- 172.21.15.67:0/3032185997 >> v1:172.21.15.169:6789/0 conn(0x7f80b41600f0 legacy=0x7f80b40d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:45.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.749+0000 7f80b1e00640 1 -- 172.21.15.67:0/3032185997 >> v1:172.21.15.67:6789/0 conn(0x7f80b40d0b40 legacy=0x7f80b4072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:45.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.749+0000 7f80b1e00640 1 -- 172.21.15.67:0/3032185997 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b406f440 con 0x7f80b415fa00 2024-05-30T16:54:45.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.751+0000 7f80b1e00640 1 --2- 172.21.15.67:0/3032185997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b415fa00 0x7f80b406e970 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f80a40097a0 tx=0x7f80a402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bd60d4c94ec6ac server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:45.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.752+0000 7f80b0a00640 1 -- 172.21.15.67:0/3032185997 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4037070 con 0x7f80b415fa00 2024-05-30T16:54:45.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.752+0000 7f80b0a00640 1 -- 172.21.15.67:0/3032185997 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a4032440 con 0x7f80b415fa00 2024-05-30T16:54:45.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.752+0000 7f80b0a00640 1 -- 172.21.15.67:0/3032185997 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a402ae60 con 0x7f80b415fa00 2024-05-30T16:54:45.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.753+0000 7f80b3200640 1 -- 172.21.15.67:0/3032185997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b415fa00 msgr2=0x7f80b406e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:45.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.753+0000 7f80b3200640 1 --2- 172.21.15.67:0/3032185997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b415fa00 0x7f80b406e970 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f80a40097a0 tx=0x7f80a402a110 comp rx=0 tx=0).stop 2024-05-30T16:54:45.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.754+0000 7f80b3200640 1 -- 172.21.15.67:0/3032185997 shutdown_connections 2024-05-30T16:54:45.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.754+0000 7f80b3200640 1 --2- 172.21.15.67:0/3032185997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b415fa00 0x7f80b406e970 secure :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f80a40097a0 tx=0x7f80a402a110 comp rx=0 tx=0).stop 2024-05-30T16:54:45.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.754+0000 7f80b3200640 1 -- 172.21.15.67:0/3032185997 >> 172.21.15.67:0/3032185997 conn(0x7f80b406bae0 msgr2=0x7f80b406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:45.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.754+0000 7f80b3200640 1 -- 172.21.15.67:0/3032185997 shutdown_connections 2024-05-30T16:54:45.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.754+0000 7f80b3200640 1 -- 172.21.15.67:0/3032185997 wait complete. 2024-05-30T16:54:45.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.757+0000 7f80b3200640 1 Processor -- start 2024-05-30T16:54:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.757+0000 7f80b3200640 1 -- start start 2024-05-30T16:54:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.758+0000 7f80b3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b40d0b40 0x7f80b41e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.758+0000 7f80b3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b41600f0 0x7f80b409c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.758+0000 7f80b3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b409c6f0 0x7f80b41f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.758+0000 7f80b3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f80b415fde0 con 0x7f80b409c6f0 2024-05-30T16:54:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.758+0000 7f80b3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4071c70 con 0x7f80b40d0b40 2024-05-30T16:54:45.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.758+0000 7f80b3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f80b40728a0 con 0x7f80b41600f0 2024-05-30T16:54:45.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.759+0000 7f80b2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b409c6f0 0x7f80b41f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:45.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.759+0000 7f80b2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b409c6f0 0x7f80b41f2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43170/0 (socket says 172.21.15.67:43170) 2024-05-30T16:54:45.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.759+0000 7f80b2800640 1 -- 172.21.15.67:0/3521827115 learned_addr learned my addr 172.21.15.67:0/3521827115 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:45.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.759+0000 7f80b1e00640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b40d0b40 0x7f80b41e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:45.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.759+0000 7f80b1400640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b41600f0 0x7f80b409c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:45.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.759+0000 7f80b2800640 1 -- 172.21.15.67:0/3521827115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b41600f0 msgr2=0x7f80b409c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:45.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.760+0000 7f80b2800640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b41600f0 0x7f80b409c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:45.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.760+0000 7f80b2800640 1 -- 172.21.15.67:0/3521827115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b40d0b40 msgr2=0x7f80b41e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:45.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.760+0000 7f80b2800640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b40d0b40 0x7f80b41e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:45.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.760+0000 7f80b2800640 1 -- 172.21.15.67:0/3521827115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b409b110 con 0x7f80b409c6f0 2024-05-30T16:54:45.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.760+0000 7f80b1400640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b41600f0 0x7f80b409c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:45.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.760+0000 7f80b2800640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b409c6f0 0x7f80b41f2920 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c730 tx=0x7f80ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:45.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.761+0000 7f80aa000640 1 -- 172.21.15.67:0/3521827115 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac003b90 con 0x7f80b409c6f0 2024-05-30T16:54:45.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.761+0000 7f80aa000640 1 -- 172.21.15.67:0/3521827115 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ac003d50 con 0x7f80b409c6f0 2024-05-30T16:54:45.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.761+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b409b3f0 con 0x7f80b409c6f0 2024-05-30T16:54:45.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.761+0000 7f80aa000640 1 -- 172.21.15.67:0/3521827115 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00aa90 con 0x7f80b409c6f0 2024-05-30T16:54:45.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.761+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b409b9b0 con 0x7f80b409c6f0 2024-05-30T16:54:45.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.763+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b40b1390 con 0x7f80b409c6f0 2024-05-30T16:54:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.764+0000 7f80aa000640 1 -- 172.21.15.67:0/3521827115 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f80ac00ac30 con 0x7f80b409c6f0 2024-05-30T16:54:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.765+0000 7f80aa000640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f809406d610 0x7f809406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.765+0000 7f80aa000640 1 -- 172.21.15.67:0/3521827115 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5413+0+0 (secure 0 0 0) 0x7f80ac08a450 con 0x7f80b409c6f0 2024-05-30T16:54:45.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.765+0000 7f80b1e00640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f809406d610 0x7f809406fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.766+0000 7f80b1e00640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f809406d610 0x7f809406fad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f80a4034000 tx=0x7f80a402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:45.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.769+0000 7f80aa000640 1 -- 172.21.15.67:0/3521827115 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80b40b1390 con 0x7f80b409c6f0 2024-05-30T16:54:45.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:45.975+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f80b40689e0 con 0x7f80b409c6f0 2024-05-30T16:54:46.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.512+0000 7f80aa000640 1 -- 172.21.15.67:0/3521827115 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) ==== 169+0+0 (secure 0 0 0) 0x7f80ac05a2d0 con 0x7f80b409c6f0 2024-05-30T16:54:46.513 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-05-30T16:54:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.516+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f809406d610 msgr2=0x7f809406fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.516+0000 7f80b3200640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f809406d610 0x7f809406fad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f80a4034000 tx=0x7f80a402e000 comp rx=0 tx=0).stop 2024-05-30T16:54:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.516+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b409c6f0 msgr2=0x7f80b41f2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.516+0000 7f80b3200640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b409c6f0 0x7f80b41f2920 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c730 tx=0x7f80ac00cbf0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.517+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 shutdown_connections 2024-05-30T16:54:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.517+0000 7f80b3200640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f809406d610 0x7f809406fad0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.517+0000 7f80b3200640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b409c6f0 0x7f80b41f2920 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.517+0000 7f80b3200640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b41600f0 0x7f80b409c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.517+0000 7f80b3200640 1 --2- 172.21.15.67:0/3521827115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b40d0b40 0x7f80b41e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.517+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 >> 172.21.15.67:0/3521827115 conn(0x7f80b406bae0 msgr2=0x7f80b40b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:46.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.518+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 shutdown_connections 2024-05-30T16:54:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.519+0000 7f80b3200640 1 -- 172.21.15.67:0/3521827115 wait complete. 2024-05-30T16:54:46.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_no_wait 2024-05-30T16:54:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.760+0000 7fa801a00640 1 Processor -- start 2024-05-30T16:54:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.760+0000 7fa801a00640 1 -- start start 2024-05-30T16:54:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.761+0000 7fa801a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa7fc0d0640 0x7fa7fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa801a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc0b32c0 0x7fa7fc0b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa801a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc0d0060 con 0x7fa7fc0d0a20 2024-05-30T16:54:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa801a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc0adbb0 con 0x7fa7fc0d0640 2024-05-30T16:54:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa801a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc15db10 con 0x7fa7fc0b32c0 2024-05-30T16:54:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa7fa000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa7fc0d0a20 0x7fa7fc0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37536/0 (socket says 172.21.15.67:37536) 2024-05-30T16:54:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa7fa000640 1 -- 172.21.15.67:0/1105003137 learned_addr learned my addr 172.21.15.67:0/1105003137 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa7faa00640 1 --2- 172.21.15.67:0/1105003137 >> v2:172.21.15.169:3300/0 conn(0x7fa7fc0d0640 0x7fa7fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2630385410 0 0) 0x7fa7fc0d0060 con 0x7fa7fc0d0a20 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7e0003520 con 0x7fa7fc0d0a20 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.762+0000 7fa7fb400640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc0b32c0 0x7fa7fc0b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2276605737 0 0) 0x7fa7e0003520 con 0x7fa7fc0d0a20 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc0b32c0 msgr2=0x7fa7fc0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc0b32c0 0x7fa7fc0b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 >> v2:172.21.15.169:3300/0 conn(0x7fa7fc0d0640 msgr2=0x7fa7fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 --2- 172.21.15.67:0/1105003137 >> v2:172.21.15.169:3300/0 conn(0x7fa7fc0d0640 0x7fa7fc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc15cb10 con 0x7fa7fc0d0a20 2024-05-30T16:54:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fa7e4001670 con 0x7fa7fc0d0a20 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 >> v1:172.21.15.67:6789/0 conn(0x7fa7fc0d0a20 legacy=0x7fa7fc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc0b32c0 0x7fa7e00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0003f80 0x7fa7e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e0004d40 0x7fa7e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc15db10 con 0x7fa7e0004d40 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e000b9f0 con 0x7fa7e0003f80 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e000bbf0 con 0x7fa7fc0b32c0 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.763+0000 7fa7fb400640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e0004d40 0x7fa7e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.764+0000 7fa7fa000640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0003f80 0x7fa7e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.764+0000 7fa7faa00640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc0b32c0 0x7fa7e00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.764+0000 7fa7fb400640 1 -- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc0b32c0 msgr2=0x7fa7e00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.764+0000 7fa7fb400640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc0b32c0 0x7fa7e00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.764+0000 7fa7fb400640 1 -- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0003f80 msgr2=0x7fa7e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:46.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.764+0000 7fa7fb400640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0003f80 0x7fa7e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.764+0000 7fa7fb400640 1 -- 172.21.15.67:0/1105003137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e0003520 con 0x7fa7e0004d40 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.765+0000 7fa7fb400640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e0004d40 0x7fa7e00051c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4007360 tx=0x7fa7e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36eb8f126a8219e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.765+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4009bc0 con 0x7fa7e0004d40 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.765+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f402e640 con 0x7fa7e0004d40 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.765+0000 7fa7f9600640 1 -- 172.21.15.67:0/1105003137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f402ea30 con 0x7fa7e0004d40 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.766+0000 7fa801a00640 1 -- 172.21.15.67:0/1105003137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e0004d40 msgr2=0x7fa7e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.766+0000 7fa801a00640 1 --2- 172.21.15.67:0/1105003137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7e0004d40 0x7fa7e00051c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4007360 tx=0x7fa7e0005830 comp rx=0 tx=0).stop 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.766+0000 7fa7fb400640 1 -- 172.21.15.67:0/1105003137 reap_dead start 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.766+0000 7fa801a00640 1 -- 172.21.15.67:0/1105003137 shutdown_connections 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.766+0000 7fa801a00640 1 -- 172.21.15.67:0/1105003137 >> 172.21.15.67:0/1105003137 conn(0x7fa7fc06bd20 msgr2=0x7fa7fc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.766+0000 7fa801a00640 1 -- 172.21.15.67:0/1105003137 shutdown_connections 2024-05-30T16:54:46.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.766+0000 7fa801a00640 1 -- 172.21.15.67:0/1105003137 wait complete. 2024-05-30T16:54:46.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.768+0000 7fa801a00640 1 Processor -- start 2024-05-30T16:54:46.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.768+0000 7fa801a00640 1 -- start start 2024-05-30T16:54:46.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.769+0000 7fa801a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0004d40 0x7fa7fc154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:46.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.769+0000 7fa801a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7fc151e40 0x7fa7fc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:46.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.769+0000 7fa801a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc073070 0x7fa7fc1f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.769+0000 7fa801a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc15cdd0 con 0x7fa7fc151e40 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.769+0000 7fa801a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0ad8d0 con 0x7fa7e0004d40 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.769+0000 7fa801a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc15d0b0 con 0x7fa7fc073070 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.769+0000 7fa7fa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7fc151e40 0x7fa7fc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.769+0000 7fa7fa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7fc151e40 0x7fa7fc155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43182/0 (socket says 172.21.15.67:43182) 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.769+0000 7fa7fa000640 1 -- 172.21.15.67:0/3824524358 learned_addr learned my addr 172.21.15.67:0/3824524358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7faa00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0004d40 0x7fa7fc154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7fa000640 1 -- 172.21.15.67:0/3824524358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc073070 msgr2=0x7fa7fc1f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7fa000640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc073070 0x7fa7fc1f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7fa000640 1 -- 172.21.15.67:0/3824524358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0004d40 msgr2=0x7fa7fc154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7fa000640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0004d40 0x7fa7fc154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7fa000640 1 -- 172.21.15.67:0/3824524358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc154b70 con 0x7fa7fc151e40 2024-05-30T16:54:46.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7faa00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0004d40 0x7fa7fc154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:46.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7fa000640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7fc151e40 0x7fa7fc155c10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4001050 tx=0x7fa7e400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:46.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7f2a00640 1 -- 172.21.15.67:0/3824524358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e400ee50 con 0x7fa7fc151e40 2024-05-30T16:54:46.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7f2a00640 1 -- 172.21.15.67:0/3824524358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e4018070 con 0x7fa7fc151e40 2024-05-30T16:54:46.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa7f2a00640 1 -- 172.21.15.67:0/3824524358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e400cb10 con 0x7fa7fc151e40 2024-05-30T16:54:46.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.770+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc154e50 con 0x7fa7fc151e40 2024-05-30T16:54:46.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.771+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc155410 con 0x7fa7fc151e40 2024-05-30T16:54:46.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.772+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc0689e0 con 0x7fa7fc151e40 2024-05-30T16:54:46.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.773+0000 7fa7f2a00640 1 -- 172.21.15.67:0/3824524358 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fa7e400ccb0 con 0x7fa7fc151e40 2024-05-30T16:54:46.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.774+0000 7fa7f2a00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa7d806d7b0 0x7fa7d806fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:46.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.774+0000 7fa7f2a00640 1 -- 172.21.15.67:0/3824524358 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7fa7e408a730 con 0x7fa7fc151e40 2024-05-30T16:54:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.774+0000 7fa7faa00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa7d806d7b0 0x7fa7d806fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.775+0000 7fa7faa00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa7d806d7b0 0x7fa7d806fc70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009a80 tx=0x7fa7ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:46.779+0000 7fa7f2a00640 1 -- 172.21.15.67:0/3824524358 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7e405a320 con 0x7fa7fc151e40 2024-05-30T16:54:47.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.002+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"} v 0) -- 0x7fa7fc15dcb0 con 0x7fa7fc151e40 2024-05-30T16:54:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.010+0000 7fa7f2a00640 1 -- 172.21.15.67:0/3824524358 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"}]=0 v16) ==== 117+0+0 (secure 0 0 0) 0x7fa7e4059c20 con 0x7fa7fc151e40 2024-05-30T16:54:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa7d806d7b0 msgr2=0x7fa7d806fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa7d806d7b0 0x7fa7d806fc70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009a80 tx=0x7fa7ec008040 comp rx=0 tx=0).stop 2024-05-30T16:54:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7fc151e40 msgr2=0x7fa7fc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7fc151e40 0x7fa7fc155c10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4001050 tx=0x7fa7e400ec00 comp rx=0 tx=0).stop 2024-05-30T16:54:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 shutdown_connections 2024-05-30T16:54:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa7d806d7b0 0x7fa7d806fc70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7fc073070 0x7fa7fc1f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7fc151e40 0x7fa7fc155c10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 --2- 172.21.15.67:0/3824524358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7e0004d40 0x7fa7fc154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.013+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 >> 172.21.15.67:0/3824524358 conn(0x7fa7fc06bd20 msgr2=0x7fa7fc072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.014+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 shutdown_connections 2024-05-30T16:54:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.014+0000 7fa801a00640 1 -- 172.21.15.67:0/3824524358 wait complete. 2024-05-30T16:54:47.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-30T16:54:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.246+0000 7f0101600640 1 Processor -- start 2024-05-30T16:54:47.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.246+0000 7f0101600640 1 -- start start 2024-05-30T16:54:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.247+0000 7f0101600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f00fc0708e0 0x7f00fc0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f0101600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f00fc0af360 0x7f00fc0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f0101600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00fc0b6120 con 0x7f00fc06b900 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f0101600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f00fc0b49b0 con 0x7f00fc0708e0 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f0101600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f00fc0b6400 con 0x7f00fc0af360 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f00fa000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f00fc0708e0 0x7f00fc0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f00fa000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f00fc0708e0 0x7f00fc0b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43192/0 (socket says 172.21.15.67:43192) 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f00fa000640 1 -- 172.21.15.67:0/1560467045 learned_addr learned my addr 172.21.15.67:0/1560467045 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f00fb400640 1 --2- 172.21.15.67:0/1560467045 >> v2:172.21.15.169:3300/0 conn(0x7f00fc0af360 0x7f00fc0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f00fa000640 1 -- 172.21.15.67:0/1560467045 >> v2:172.21.15.169:3300/0 conn(0x7f00fc0af360 msgr2=0x7f00fc0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f00fa000640 1 --2- 172.21.15.67:0/1560467045 >> v2:172.21.15.169:3300/0 conn(0x7f00fc0af360 0x7f00fc0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f00fa000640 1 -- 172.21.15.67:0/1560467045 >> v1:172.21.15.169:6789/0 conn(0x7f00fc06b900 legacy=0x7f00fc06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.248+0000 7f00fa000640 1 -- 172.21.15.67:0/1560467045 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc0aee40 con 0x7f00fc0708e0 2024-05-30T16:54:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.250+0000 7f00fa000640 1 --2- 172.21.15.67:0/1560467045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0708e0 0x7f00fc0b4470 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f00fc070680 tx=0x7f00e402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e498b334b7db674b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.250+0000 7f00f9600640 1 -- 172.21.15.67:0/1560467045 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4036070 con 0x7f00fc0708e0 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.250+0000 7f00f9600640 1 -- 172.21.15.67:0/1560467045 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e4031600 con 0x7f00fc0708e0 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.250+0000 7f00f9600640 1 -- 172.21.15.67:0/1560467045 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40319d0 con 0x7f00fc0708e0 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.251+0000 7f0101600640 1 -- 172.21.15.67:0/1560467045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0708e0 msgr2=0x7f00fc0b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.251+0000 7f0101600640 1 --2- 172.21.15.67:0/1560467045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0708e0 0x7f00fc0b4470 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f00fc070680 tx=0x7f00e402a6f0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.251+0000 7f0101600640 1 -- 172.21.15.67:0/1560467045 shutdown_connections 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.251+0000 7f0101600640 1 --2- 172.21.15.67:0/1560467045 >> v2:172.21.15.169:3300/0 conn(0x7f00fc0af360 0x7f00fc0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.251+0000 7f0101600640 1 --2- 172.21.15.67:0/1560467045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0708e0 0x7f00fc0b4470 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.251+0000 7f0101600640 1 -- 172.21.15.67:0/1560467045 >> 172.21.15.67:0/1560467045 conn(0x7f00fc09aa30 msgr2=0x7f00fc0b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.251+0000 7f0101600640 1 -- 172.21.15.67:0/1560467045 shutdown_connections 2024-05-30T16:54:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.251+0000 7f0101600640 1 -- 172.21.15.67:0/1560467045 wait complete. 2024-05-30T16:54:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.253+0000 7f0101600640 1 Processor -- start 2024-05-30T16:54:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.253+0000 7f0101600640 1 -- start start 2024-05-30T16:54:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.254+0000 7f0101600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06b900 0x7f00fc0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.254+0000 7f0101600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00fc0708e0 0x7f00fc0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.254+0000 7f0101600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0af360 0x7f00fc159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.254+0000 7f0101600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1451f0 con 0x7f00fc0af360 2024-05-30T16:54:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.254+0000 7f0101600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc0b5df0 con 0x7f00fc0708e0 2024-05-30T16:54:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.254+0000 7f0101600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc145370 con 0x7f00fc06b900 2024-05-30T16:54:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.254+0000 7f00fb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0af360 0x7f00fc159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.254+0000 7f00fb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0af360 0x7f00fc159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43194/0 (socket says 172.21.15.67:43194) 2024-05-30T16:54:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.254+0000 7f00fb400640 1 -- 172.21.15.67:0/1949937690 learned_addr learned my addr 172.21.15.67:0/1949937690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00faa00640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06b900 0x7f00fc0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00fa000640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00fc0708e0 0x7f00fc0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00fb400640 1 -- 172.21.15.67:0/1949937690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06b900 msgr2=0x7f00fc0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00fb400640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06b900 0x7f00fc0b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00fb400640 1 -- 172.21.15.67:0/1949937690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00fc0708e0 msgr2=0x7f00fc0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00fb400640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00fc0708e0 0x7f00fc0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00fb400640 1 -- 172.21.15.67:0/1949937690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc0b0e40 con 0x7f00fc0af360 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00fa000640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00fc0708e0 0x7f00fc0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00fb400640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0af360 0x7f00fc159c70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f00f400d5d0 tx=0x7f00f400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00f2a00640 1 -- 172.21.15.67:0/1949937690 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f40125f0 con 0x7f00fc0af360 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00f2a00640 1 -- 172.21.15.67:0/1949937690 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f4019070 con 0x7f00fc0af360 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc0b1120 con 0x7f00fc0af360 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f00f2a00640 1 -- 172.21.15.67:0/1949937690 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f40129f0 con 0x7f00fc0af360 2024-05-30T16:54:47.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.255+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc0b16e0 con 0x7f00fc0af360 2024-05-30T16:54:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.256+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00fc0690f0 con 0x7f00fc0af360 2024-05-30T16:54:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.258+0000 7f00f2a00640 1 -- 172.21.15.67:0/1949937690 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f00f4012b90 con 0x7f00fc0af360 2024-05-30T16:54:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.258+0000 7f00f2a00640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f00d806d5e0 0x7f00d806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.259+0000 7f00f2a00640 1 -- 172.21.15.67:0/1949937690 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f00f408acb0 con 0x7f00fc0af360 2024-05-30T16:54:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.259+0000 7f00faa00640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f00d806d5e0 0x7f00d806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.260+0000 7f00faa00640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f00d806d5e0 0x7f00d806faa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0b03f0 tx=0x7f00ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:47.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.265+0000 7f00f2a00640 1 -- 172.21.15.67:0/1949937690 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00f405a820 con 0x7f00fc0af360 2024-05-30T16:54:47.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.472+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f00fc1608a0 con 0x7f00fc0af360 2024-05-30T16:54:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.477+0000 7f00f2a00640 1 -- 172.21.15.67:0/1949937690 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f00f405a120 con 0x7f00fc0af360 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f00d806d5e0 msgr2=0x7f00d806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f00d806d5e0 0x7f00d806faa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0b03f0 tx=0x7f00ec009040 comp rx=0 tx=0).stop 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0af360 msgr2=0x7f00fc159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0af360 0x7f00fc159c70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f00f400d5d0 tx=0x7f00f400da90 comp rx=0 tx=0).stop 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 shutdown_connections 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f00d806d5e0 0x7f00d806faa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00fc0af360 0x7f00fc159c70 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00fc0708e0 0x7f00fc0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 --2- 172.21.15.67:0/1949937690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00fc06b900 0x7f00fc0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 >> 172.21.15.67:0/1949937690 conn(0x7f00fc09aa30 msgr2=0x7f00fc15fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:47.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 shutdown_connections 2024-05-30T16:54:47.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.480+0000 7f0101600640 1 -- 172.21.15.67:0/1949937690 wait complete. 2024-05-30T16:54:47.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones' 2024-05-30T16:54:47.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.717+0000 7ff093200640 1 Processor -- start 2024-05-30T16:54:47.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.718+0000 7ff093200640 1 -- start start 2024-05-30T16:54:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.719+0000 7ff093200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff0940d3280 0x7ff0940d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.719+0000 7ff093200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0941625c0 0x7ff094162a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.719+0000 7ff093200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff094163330 con 0x7ff09415e8a0 2024-05-30T16:54:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.719+0000 7ff093200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff0940cf2d0 con 0x7ff0940d3280 2024-05-30T16:54:47.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.719+0000 7ff093200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff0940cf5b0 con 0x7ff0941625c0 2024-05-30T16:54:47.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.720+0000 7ff091e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff09415e8a0 0x7ff0940d2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37542/0 (socket says 172.21.15.67:37542) 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.720+0000 7ff091e00640 1 -- 172.21.15.67:0/1196576304 learned_addr learned my addr 172.21.15.67:0/1196576304 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.720+0000 7ff092800640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0941625c0 0x7ff094162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.720+0000 7ff091400640 1 --2- 172.21.15.67:0/1196576304 >> v2:172.21.15.169:3300/0 conn(0x7ff0940d3280 0x7ff0940d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.720+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935105032 0 0) 0x7ff094163330 con 0x7ff09415e8a0 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.720+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff078003520 con 0x7ff09415e8a0 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1713487183 0 0) 0x7ff078003520 con 0x7ff09415e8a0 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0941625c0 msgr2=0x7ff094162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0941625c0 0x7ff094162a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 >> v2:172.21.15.169:3300/0 conn(0x7ff0940d3280 msgr2=0x7ff0940d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 --2- 172.21.15.67:0/1196576304 >> v2:172.21.15.169:3300/0 conn(0x7ff0940d3280 0x7ff0940d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0940cf890 con 0x7ff09415e8a0 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff084001670 con 0x7ff09415e8a0 2024-05-30T16:54:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 >> v1:172.21.15.67:6789/0 conn(0x7ff09415e8a0 legacy=0x7ff0940d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0941625c0 0x7ff0780039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff078003f30 0x7ff078004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff078004d40 0x7ff0780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff0940cf2d0 con 0x7ff078004d40 2024-05-30T16:54:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff0940cf5b0 con 0x7ff078003f30 2024-05-30T16:54:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff07800b9c0 con 0x7ff0941625c0 2024-05-30T16:54:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff092800640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff078004d40 0x7ff0780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff091400640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff078003f30 0x7ff078004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff092800640 1 -- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0941625c0 msgr2=0x7ff0780039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:47.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.721+0000 7ff092800640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0941625c0 0x7ff0780039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.722+0000 7ff092800640 1 -- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff078003f30 msgr2=0x7ff078004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.722+0000 7ff092800640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff078003f30 0x7ff078004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.722+0000 7ff092800640 1 -- 172.21.15.67:0/1196576304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff078003520 con 0x7ff078004d40 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.722+0000 7ff091400640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff078003f30 0x7ff078004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.723+0000 7ff092800640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff078004d40 0x7ff0780051c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009fc0 tx=0x7ff078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8048cbc6557dd86 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.723+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c009c10 con 0x7ff078004d40 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.723+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c036070 con 0x7ff078004d40 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.723+0000 7ff090a00640 1 -- 172.21.15.67:0/1196576304 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c02d070 con 0x7ff078004d40 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.723+0000 7ff093200640 1 -- 172.21.15.67:0/1196576304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff078004d40 msgr2=0x7ff0780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.723+0000 7ff093200640 1 --2- 172.21.15.67:0/1196576304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff078004d40 0x7ff0780051c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009fc0 tx=0x7ff078005830 comp rx=0 tx=0).stop 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.724+0000 7ff092800640 1 -- 172.21.15.67:0/1196576304 reap_dead start 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.724+0000 7ff093200640 1 -- 172.21.15.67:0/1196576304 shutdown_connections 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.724+0000 7ff093200640 1 -- 172.21.15.67:0/1196576304 >> 172.21.15.67:0/1196576304 conn(0x7ff09408bfc0 msgr2=0x7ff0940d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.724+0000 7ff093200640 1 -- 172.21.15.67:0/1196576304 shutdown_connections 2024-05-30T16:54:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.724+0000 7ff093200640 1 -- 172.21.15.67:0/1196576304 wait complete. 2024-05-30T16:54:47.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.725+0000 7ff093200640 1 Processor -- start 2024-05-30T16:54:47.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.726+0000 7ff093200640 1 -- start start 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.726+0000 7ff093200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff078004d40 0x7ff0940c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.726+0000 7ff093200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff094157d80 0x7ff0941570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff093200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0940c7f40 0x7ff0941e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff093200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff09415ee00 con 0x7ff0940c7f40 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff093200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff09415ec80 con 0x7ff094157d80 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff093200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff094162a20 con 0x7ff078004d40 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff092800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0940c7f40 0x7ff0941e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff092800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0940c7f40 0x7ff0941e9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43202/0 (socket says 172.21.15.67:43202) 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff092800640 1 -- 172.21.15.67:0/1915540038 learned_addr learned my addr 172.21.15.67:0/1915540038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff091400640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff094157d80 0x7ff0941570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff092800640 1 -- 172.21.15.67:0/1915540038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff078004d40 msgr2=0x7ff0940c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff091e00640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff078004d40 0x7ff0940c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff092800640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff078004d40 0x7ff0940c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff092800640 1 -- 172.21.15.67:0/1915540038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff094157d80 msgr2=0x7ff0941570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff092800640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff094157d80 0x7ff0941570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff092800640 1 -- 172.21.15.67:0/1915540038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0940c8440 con 0x7ff0940c7f40 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.727+0000 7ff091400640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff094157d80 0x7ff0941570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.728+0000 7ff092800640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0940c7f40 0x7ff0941e9e20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00e230 tx=0x7ff08c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.728+0000 7ff08a000640 1 -- 172.21.15.67:0/1915540038 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c009c10 con 0x7ff0940c7f40 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.728+0000 7ff08a000640 1 -- 172.21.15.67:0/1915540038 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c003990 con 0x7ff0940c7f40 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.728+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0940c8720 con 0x7ff0940c7f40 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.728+0000 7ff08a000640 1 -- 172.21.15.67:0/1915540038 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c003d60 con 0x7ff0940c7f40 2024-05-30T16:54:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.728+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0940c8ce0 con 0x7ff0940c7f40 2024-05-30T16:54:47.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.729+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0940689e0 con 0x7ff0940c7f40 2024-05-30T16:54:47.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.730+0000 7ff08a000640 1 -- 172.21.15.67:0/1915540038 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7ff08c005d10 con 0x7ff0940c7f40 2024-05-30T16:54:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.731+0000 7ff08a000640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff07406d5e0 0x7ff07406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.732+0000 7ff08a000640 1 -- 172.21.15.67:0/1915540038 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7ff08c0abd00 con 0x7ff0940c7f40 2024-05-30T16:54:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.732+0000 7ff091e00640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff07406d5e0 0x7ff07406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:47.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.733+0000 7ff091e00640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff07406d5e0 0x7ff07406faa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff084007a70 tx=0x7ff084009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:47.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:47.736+0000 7ff08a000640 1 -- 172.21.15.67:0/1915540038 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff08c04a060 con 0x7ff0940c7f40 2024-05-30T16:54:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.013+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]} v 0) -- 0x7ff0940d1c80 con 0x7ff0940c7f40 2024-05-30T16:54:48.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.525+0000 7ff08a000640 1 -- 172.21.15.67:0/1915540038 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]}]=0 v446) ==== 162+0+0 (secure 0 0 0) 0x7ff08c07b850 con 0x7ff0940c7f40 2024-05-30T16:54:48.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff07406d5e0 msgr2=0x7ff07406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:48.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff07406d5e0 0x7ff07406faa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff084007a70 tx=0x7ff084009040 comp rx=0 tx=0).stop 2024-05-30T16:54:48.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0940c7f40 msgr2=0x7ff0941e9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:48.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0940c7f40 0x7ff0941e9e20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00e230 tx=0x7ff08c009750 comp rx=0 tx=0).stop 2024-05-30T16:54:48.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 shutdown_connections 2024-05-30T16:54:48.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff07406d5e0 0x7ff07406faa0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0940c7f40 0x7ff0941e9e20 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff094157d80 0x7ff0941570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 --2- 172.21.15.67:0/1915540038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff078004d40 0x7ff0940c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 >> 172.21.15.67:0/1915540038 conn(0x7ff09408bfc0 msgr2=0x7ff0940b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:48.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.530+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 shutdown_connections 2024-05-30T16:54:48.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.531+0000 7ff093200640 1 -- 172.21.15.67:0/1915540038 wait complete. 2024-05-30T16:54:48.553 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed when number of clone jobs is ... ok 2024-05-30T16:54:48.553 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-05-30T16:54:48.554 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T16:54:48.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:54:48.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.769+0000 7f7356400640 1 Processor -- start 2024-05-30T16:54:48.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.770+0000 7f7356400640 1 -- start start 2024-05-30T16:54:48.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.771+0000 7f7356400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73500d16a0 0x7f73500cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:48.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.771+0000 7f7356400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f73500cf870 0x7f73500d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:48.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.771+0000 7f7356400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f735015ec50 con 0x7f735015e870 2024-05-30T16:54:48.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.771+0000 7f7356400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f73500d2150 con 0x7f73500cf870 2024-05-30T16:54:48.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.771+0000 7f7356400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73500cfd20 con 0x7f73500d16a0 2024-05-30T16:54:48.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.771+0000 7f7355000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f735015e870 0x7f73500d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37550/0 (socket says 172.21.15.67:37550) 2024-05-30T16:54:48.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.771+0000 7f7355000640 1 -- 172.21.15.67:0/2379557555 learned_addr learned my addr 172.21.15.67:0/2379557555 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:48.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f7355a00640 1 --2- 172.21.15.67:0/2379557555 >> v2:172.21.15.169:3300/0 conn(0x7f73500cf870 0x7f73500d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:48.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963566685 0 0) 0x7f735015ec50 con 0x7f735015e870 2024-05-30T16:54:48.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7334003520 con 0x7f735015e870 2024-05-30T16:54:48.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f734fe00640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73500d16a0 0x7f73500cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:48.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1729835880 0 0) 0x7f7334003520 con 0x7f735015e870 2024-05-30T16:54:48.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73500d16a0 msgr2=0x7f73500cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:48.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f734f400640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73500d16a0 0x7f73500cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 >> v2:172.21.15.169:3300/0 conn(0x7f73500cf870 msgr2=0x7f73500d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f734f400640 1 --2- 172.21.15.67:0/2379557555 >> v2:172.21.15.169:3300/0 conn(0x7f73500cf870 0x7f73500d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.772+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73500cff80 con 0x7f735015e870 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f73400019f0 con 0x7f735015e870 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 >> v1:172.21.15.67:6789/0 conn(0x7f735015e870 legacy=0x7f73500d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734f400640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734f400640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7334004800 0x7f7334004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734f400640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73500d16a0 0x7f73340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f73500d2150 con 0x7f7334004800 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73500cfd20 con 0x7f73340039b0 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f733400bac0 con 0x7f73500d16a0 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734fe00640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7334004800 0x7f7334004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:48.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f7355000640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:48.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734fe00640 1 -- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73500d16a0 msgr2=0x7f73340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:48.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734fe00640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73500d16a0 0x7f73340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734fe00640 1 -- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73340039b0 msgr2=0x7f7334003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:48.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734fe00640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.773+0000 7f734fe00640 1 -- 172.21.15.67:0/2379557555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7334003520 con 0x7f7334004800 2024-05-30T16:54:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.775+0000 7f734fe00640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7334004800 0x7f7334004c80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f733800b9e0 tx=0x7f7334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d4196b0252692e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.775+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733800b5e0 con 0x7f7334004800 2024-05-30T16:54:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.775+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733802ada0 con 0x7f7334004800 2024-05-30T16:54:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.775+0000 7f734f400640 1 -- 172.21.15.67:0/2379557555 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338005bb0 con 0x7f7334004800 2024-05-30T16:54:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.775+0000 7f7356400640 1 -- 172.21.15.67:0/2379557555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7334004800 msgr2=0x7f7334004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.775+0000 7f7356400640 1 --2- 172.21.15.67:0/2379557555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7334004800 0x7f7334004c80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f733800b9e0 tx=0x7f7334005830 comp rx=0 tx=0).stop 2024-05-30T16:54:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.775+0000 7f7355a00640 1 -- 172.21.15.67:0/2379557555 reap_dead start 2024-05-30T16:54:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.776+0000 7f7356400640 1 -- 172.21.15.67:0/2379557555 shutdown_connections 2024-05-30T16:54:48.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.776+0000 7f7356400640 1 -- 172.21.15.67:0/2379557555 >> 172.21.15.67:0/2379557555 conn(0x7f735008bfc0 msgr2=0x7f73500d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:48.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.776+0000 7f7356400640 1 -- 172.21.15.67:0/2379557555 shutdown_connections 2024-05-30T16:54:48.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.776+0000 7f7356400640 1 -- 172.21.15.67:0/2379557555 wait complete. 2024-05-30T16:54:48.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.777+0000 7f7356400640 1 Processor -- start 2024-05-30T16:54:48.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.778+0000 7f7356400640 1 -- start start 2024-05-30T16:54:48.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7356400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7334004800 0x7f73500ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:48.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7356400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73500b91b0 0x7f73500b9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:48.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7356400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73501f4460 0x7f73501f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:48.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7356400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f73500d0820 con 0x7f73501f4460 2024-05-30T16:54:48.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7356400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73500d0540 con 0x7f73500b91b0 2024-05-30T16:54:48.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7356400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73500d4fb0 con 0x7f7334004800 2024-05-30T16:54:48.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7355a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73501f4460 0x7f73501f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:48.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7355a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73501f4460 0x7f73501f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43226/0 (socket says 172.21.15.67:43226) 2024-05-30T16:54:48.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7355a00640 1 -- 172.21.15.67:0/1585193510 learned_addr learned my addr 172.21.15.67:0/1585193510 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:48.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7355000640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7334004800 0x7f73500ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:48.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f734fe00640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73500b91b0 0x7f73500b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:48.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7355a00640 1 -- 172.21.15.67:0/1585193510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7334004800 msgr2=0x7f73500ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:48.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7355a00640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7334004800 0x7f73500ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7355a00640 1 -- 172.21.15.67:0/1585193510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73500b91b0 msgr2=0x7f73500b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:48.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7355a00640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73500b91b0 0x7f73500b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:48.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.779+0000 7f7355a00640 1 -- 172.21.15.67:0/1585193510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73501f4e20 con 0x7f73501f4460 2024-05-30T16:54:48.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.780+0000 7f7355a00640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73501f4460 0x7f73501f48e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f735008d9e0 tx=0x7f734800d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.780+0000 7f734cc00640 1 -- 172.21.15.67:0/1585193510 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348004020 con 0x7f73501f4460 2024-05-30T16:54:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.780+0000 7f734cc00640 1 -- 172.21.15.67:0/1585193510 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73480041e0 con 0x7f73501f4460 2024-05-30T16:54:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.780+0000 7f734cc00640 1 -- 172.21.15.67:0/1585193510 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73480043d0 con 0x7f73501f4460 2024-05-30T16:54:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.780+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73501f50b0 con 0x7f73501f4460 2024-05-30T16:54:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.780+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73501f59a0 con 0x7f73501f4460 2024-05-30T16:54:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.782+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73500b9b70 con 0x7f73501f4460 2024-05-30T16:54:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.782+0000 7f734cc00640 1 -- 172.21.15.67:0/1585193510 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f734801a020 con 0x7f73501f4460 2024-05-30T16:54:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.783+0000 7f734cc00640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f732c06d5e0 0x7f732c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.784+0000 7f734cc00640 1 -- 172.21.15.67:0/1585193510 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f734808a020 con 0x7f73501f4460 2024-05-30T16:54:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.784+0000 7f7355000640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f732c06d5e0 0x7f732c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:48.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.784+0000 7f7355000640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f732c06d5e0 0x7f732c06faa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7340001330 tx=0x7f7340009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:48.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.788+0000 7f734cc00640 1 -- 172.21.15.67:0/1585193510 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7348059c10 con 0x7f73501f4460 2024-05-30T16:54:48.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:48.998+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73500689e0 con 0x7f73501f4460 2024-05-30T16:54:49.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.000+0000 7f734cc00640 1 -- 172.21.15.67:0/1585193510 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+13095 (secure 0 0 0) 0x7f7348059510 con 0x7f73501f4460 2024-05-30T16:54:49.000 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:49.001 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":66,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:54:46.501045+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/898694760":"2024-05-31T16:54:43.488637+0000","172.21.15.169:6833/898694760":"2024-05-31T16:54:43.488637+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:54:49.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.002+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f732c06d5e0 msgr2=0x7f732c06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.002+0000 7f7356400640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f732c06d5e0 0x7f732c06faa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7340001330 tx=0x7f7340009040 comp rx=0 tx=0).stop 2024-05-30T16:54:49.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.002+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73501f4460 msgr2=0x7f73501f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.002+0000 7f7356400640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73501f4460 0x7f73501f48e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f735008d9e0 tx=0x7f734800d820 comp rx=0 tx=0).stop 2024-05-30T16:54:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.003+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 shutdown_connections 2024-05-30T16:54:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.003+0000 7f7356400640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f732c06d5e0 0x7f732c06faa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.003+0000 7f7356400640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73501f4460 0x7f73501f48e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.003+0000 7f7356400640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73500b91b0 0x7f73500b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.003+0000 7f7356400640 1 --2- 172.21.15.67:0/1585193510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7334004800 0x7f73500ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.003+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 >> 172.21.15.67:0/1585193510 conn(0x7f735008bfc0 msgr2=0x7f73500b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.003+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 shutdown_connections 2024-05-30T16:54:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.003+0000 7f7356400640 1 -- 172.21.15.67:0/1585193510 wait complete. 2024-05-30T16:54:49.023 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-30T16:43:40.009725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-30T16:54:49.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-30T16:54:49.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.241+0000 7f2a27000640 1 Processor -- start 2024-05-30T16:54:49.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.241+0000 7f2a27000640 1 -- start start 2024-05-30T16:54:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.243+0000 7f2a27000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a20153510 0x7f2a200b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.243+0000 7f2a27000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2a200ccb90 0x7f2a200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.243+0000 7f2a27000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2015d900 con 0x7f2a2015cb60 2024-05-30T16:54:49.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.243+0000 7f2a27000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2a201538f0 con 0x7f2a200ccb90 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.243+0000 7f2a27000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a200cd530 con 0x7f2a20153510 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.243+0000 7f2a26600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2a2015cb60 0x7f2a2015cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37558/0 (socket says 172.21.15.67:37558) 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.243+0000 7f2a26600640 1 -- 172.21.15.67:0/2199907720 learned_addr learned my addr 172.21.15.67:0/2199907720 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.243+0000 7f2a25c00640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a20153510 0x7f2a200b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.243+0000 7f2a25200640 1 --2- 172.21.15.67:0/2199907720 >> v2:172.21.15.169:3300/0 conn(0x7f2a200ccb90 0x7f2a200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 33412659 0 0) 0x7f2a2015d900 con 0x7f2a2015cb60 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a08003520 con 0x7f2a2015cb60 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3405165771 0 0) 0x7f2a08003520 con 0x7f2a2015cb60 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a20153510 msgr2=0x7f2a200b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a20153510 0x7f2a200b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 >> v2:172.21.15.169:3300/0 conn(0x7f2a200ccb90 msgr2=0x7f2a200ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 --2- 172.21.15.67:0/2199907720 >> v2:172.21.15.169:3300/0 conn(0x7f2a200ccb90 0x7f2a200ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a200b3f90 con 0x7f2a2015cb60 2024-05-30T16:54:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f2a180019f0 con 0x7f2a2015cb60 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 >> v1:172.21.15.67:6789/0 conn(0x7f2a2015cb60 legacy=0x7f2a2015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a20153510 0x7f2a080039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08003f80 0x7f2a08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a08004d40 0x7f2a080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0800b9c0 con 0x7f2a08004d40 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0800bb70 con 0x7f2a08003f80 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0800bd70 con 0x7f2a20153510 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.244+0000 7f2a26600640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a08004d40 0x7f2a080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.245+0000 7f2a25c00640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a20153510 0x7f2a080039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.245+0000 7f2a25200640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08003f80 0x7f2a08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.245+0000 7f2a26600640 1 -- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a20153510 msgr2=0x7f2a080039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.245+0000 7f2a26600640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a20153510 0x7f2a080039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.245+0000 7f2a26600640 1 -- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08003f80 msgr2=0x7f2a08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.245+0000 7f2a26600640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08003f80 0x7f2a08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.245+0000 7f2a26600640 1 -- 172.21.15.67:0/2199907720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a08003520 con 0x7f2a08004d40 2024-05-30T16:54:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.245+0000 7f2a25200640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08003f80 0x7f2a08004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.246+0000 7f2a26600640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a08004d40 0x7f2a080051c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800a7a0 tx=0x7f2a08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca7f044587f3268 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.246+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800a3f0 con 0x7f2a08004d40 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.246+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1802f070 con 0x7f2a08004d40 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.246+0000 7f2a1fe00640 1 -- 172.21.15.67:0/2199907720 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1802bc60 con 0x7f2a08004d40 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.247+0000 7f2a27000640 1 -- 172.21.15.67:0/2199907720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a08004d40 msgr2=0x7f2a080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.247+0000 7f2a27000640 1 --2- 172.21.15.67:0/2199907720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a08004d40 0x7f2a080051c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800a7a0 tx=0x7f2a08005830 comp rx=0 tx=0).stop 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.247+0000 7f2a26600640 1 -- 172.21.15.67:0/2199907720 reap_dead start 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.247+0000 7f2a27000640 1 -- 172.21.15.67:0/2199907720 shutdown_connections 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.247+0000 7f2a27000640 1 -- 172.21.15.67:0/2199907720 >> 172.21.15.67:0/2199907720 conn(0x7f2a2006b900 msgr2=0x7f2a2006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.247+0000 7f2a27000640 1 -- 172.21.15.67:0/2199907720 shutdown_connections 2024-05-30T16:54:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.247+0000 7f2a27000640 1 -- 172.21.15.67:0/2199907720 wait complete. 2024-05-30T16:54:49.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.249+0000 7f2a27000640 1 Processor -- start 2024-05-30T16:54:49.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.249+0000 7f2a27000640 1 -- start start 2024-05-30T16:54:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.250+0000 7f2a27000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08004d40 0x7f2a200cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.250+0000 7f2a27000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a20148440 0x7f2a201f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.250+0000 7f2a27000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a200c94d0 0x7f2a200c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.250+0000 7f2a27000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2a200b4550 con 0x7f2a20148440 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.250+0000 7f2a27000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2a200b48c0 con 0x7f2a08004d40 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.250+0000 7f2a27000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a200b4d50 con 0x7f2a200c94d0 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.250+0000 7f2a25200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a20148440 0x7f2a201f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a25200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a20148440 0x7f2a201f42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43258/0 (socket says 172.21.15.67:43258) 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a25200640 1 -- 172.21.15.67:0/602806580 learned_addr learned my addr 172.21.15.67:0/602806580 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a25c00640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08004d40 0x7f2a200cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a26600640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a200c94d0 0x7f2a200c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a25200640 1 -- 172.21.15.67:0/602806580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a200c94d0 msgr2=0x7f2a200c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a25200640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a200c94d0 0x7f2a200c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a25200640 1 -- 172.21.15.67:0/602806580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08004d40 msgr2=0x7f2a200cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a25200640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08004d40 0x7f2a200cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a25200640 1 -- 172.21.15.67:0/602806580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a200aeca0 con 0x7f2a20148440 2024-05-30T16:54:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.251+0000 7f2a25200640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a20148440 0x7f2a201f42a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0109d0 tx=0x7f2a0c010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.252+0000 7f2a1d600640 1 -- 172.21.15.67:0/602806580 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c00f040 con 0x7f2a20148440 2024-05-30T16:54:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.252+0000 7f2a1d600640 1 -- 172.21.15.67:0/602806580 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0c00ec20 con 0x7f2a20148440 2024-05-30T16:54:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.252+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a200aef40 con 0x7f2a20148440 2024-05-30T16:54:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.252+0000 7f2a1d600640 1 -- 172.21.15.67:0/602806580 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c01c400 con 0x7f2a20148440 2024-05-30T16:54:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.252+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a20150620 con 0x7f2a20148440 2024-05-30T16:54:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.253+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a200689e0 con 0x7f2a20148440 2024-05-30T16:54:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.254+0000 7f2a1d600640 1 -- 172.21.15.67:0/602806580 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f2a0c01c5a0 con 0x7f2a20148440 2024-05-30T16:54:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.255+0000 7f2a1d600640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a0006d5e0 0x7f2a0006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.255+0000 7f2a1d600640 1 -- 172.21.15.67:0/602806580 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f2a0c08ae00 con 0x7f2a20148440 2024-05-30T16:54:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.256+0000 7f2a25c00640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a0006d5e0 0x7f2a0006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.256+0000 7f2a25c00640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a0006d5e0 0x7f2a0006faa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009a10 tx=0x7f2a14008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.260+0000 7f2a1d600640 1 -- 172.21.15.67:0/602806580 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a0c018030 con 0x7f2a20148440 2024-05-30T16:54:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.461+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2a200b3c40 con 0x7f2a20148440 2024-05-30T16:54:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.462+0000 7f2a1d600640 1 -- 172.21.15.67:0/602806580 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f2a0c05abf0 con 0x7f2a20148440 2024-05-30T16:54:49.463 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-05-30T16:54:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a0006d5e0 msgr2=0x7f2a0006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a0006d5e0 0x7f2a0006faa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009a10 tx=0x7f2a14008040 comp rx=0 tx=0).stop 2024-05-30T16:54:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a20148440 msgr2=0x7f2a201f42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a20148440 0x7f2a201f42a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0109d0 tx=0x7f2a0c010e90 comp rx=0 tx=0).stop 2024-05-30T16:54:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 shutdown_connections 2024-05-30T16:54:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2a0006d5e0 0x7f2a0006faa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a200c94d0 0x7f2a200c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2a20148440 0x7f2a201f42a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 --2- 172.21.15.67:0/602806580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a08004d40 0x7f2a200cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.464+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 >> 172.21.15.67:0/602806580 conn(0x7f2a2006b900 msgr2=0x7f2a2006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:49.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.465+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 shutdown_connections 2024-05-30T16:54:49.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.465+0000 7f2a27000640 1 -- 172.21.15.67:0/602806580 wait complete. 2024-05-30T16:54:49.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-05-30T16:54:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.680+0000 7f25d5400640 1 Processor -- start 2024-05-30T16:54:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.680+0000 7f25d5400640 1 -- start start 2024-05-30T16:54:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.682+0000 7f25d5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d006dc20 0x7f25d00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.682+0000 7f25d5400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d00afd60 con 0x7f25d0073310 2024-05-30T16:54:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.682+0000 7f25d5400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d00af7a0 con 0x7f25d015b3a0 2024-05-30T16:54:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.682+0000 7f25d5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25d006f980 con 0x7f25d006dc20 2024-05-30T16:54:49.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.682+0000 7f25c7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f25d0073310 0x7f25d00ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37566/0 (socket says 172.21.15.67:37566) 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.682+0000 7f25c7e00640 1 -- 172.21.15.67:0/116537868 learned_addr learned my addr 172.21.15.67:0/116537868 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.682+0000 7f25d4a00640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d006dc20 0x7f25d00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125804537 0 0) 0x7f25d00afd60 con 0x7f25d0073310 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25b4005730 con 0x7f25d0073310 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2943664686 0 0) 0x7f25b4005730 con 0x7f25d0073310 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d006dc20 msgr2=0x7f25d00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d006dc20 0x7f25d00b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 >> v1:172.21.15.169:6789/0 conn(0x7f25d015b3a0 legacy=0x7f25d0072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d00afa80 con 0x7f25d0073310 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f25b8001670 con 0x7f25d0073310 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 >> v1:172.21.15.67:6789/0 conn(0x7f25d0073310 legacy=0x7f25d00ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d006dc20 0x7f25b4005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4006170 0x7f25b4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006f50 0x7f25b40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f25d006f980 con 0x7f25b4006170 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25b400dd60 con 0x7f25b4006f50 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25b4003520 con 0x7f25d006dc20 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.683+0000 7f25c7e00640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4006170 0x7f25b4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.684+0000 7f25cfe00640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d006dc20 0x7f25b4005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.684+0000 7f25d4a00640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006f50 0x7f25b40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.684+0000 7f25c7e00640 1 -- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d006dc20 msgr2=0x7f25b4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.684+0000 7f25c7e00640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d006dc20 0x7f25b4005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.684+0000 7f25c7e00640 1 -- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006f50 msgr2=0x7f25b40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.684+0000 7f25c7e00640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006f50 0x7f25b40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.684+0000 7f25c7e00640 1 -- 172.21.15.67:0/116537868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b4005730 con 0x7f25b4006170 2024-05-30T16:54:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.684+0000 7f25d4a00640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006f50 0x7f25b40073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:49.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.685+0000 7f25c7e00640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4006170 0x7f25b4006a10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f25b800c210 tx=0x7f25b4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7db141f80cc23a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:49.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.685+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8007d30 con 0x7f25b4006170 2024-05-30T16:54:49.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.685+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b8005ad0 con 0x7f25b4006170 2024-05-30T16:54:49.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.685+0000 7f25cf400640 1 -- 172.21.15.67:0/116537868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8007070 con 0x7f25b4006170 2024-05-30T16:54:49.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.686+0000 7f25d5400640 1 -- 172.21.15.67:0/116537868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4006170 msgr2=0x7f25b4006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.686+0000 7f25d5400640 1 --2- 172.21.15.67:0/116537868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25b4006170 0x7f25b4006a10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f25b800c210 tx=0x7f25b4007a40 comp rx=0 tx=0).stop 2024-05-30T16:54:49.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.686+0000 7f25d4a00640 1 -- 172.21.15.67:0/116537868 reap_dead start 2024-05-30T16:54:49.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.686+0000 7f25d5400640 1 -- 172.21.15.67:0/116537868 shutdown_connections 2024-05-30T16:54:49.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.686+0000 7f25d5400640 1 -- 172.21.15.67:0/116537868 >> 172.21.15.67:0/116537868 conn(0x7f25d00b1fc0 msgr2=0x7f25d00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:49.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.686+0000 7f25d5400640 1 -- 172.21.15.67:0/116537868 shutdown_connections 2024-05-30T16:54:49.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.686+0000 7f25d5400640 1 -- 172.21.15.67:0/116537868 wait complete. 2024-05-30T16:54:49.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.688+0000 7f25d5400640 1 Processor -- start 2024-05-30T16:54:49.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.688+0000 7f25d5400640 1 -- start start 2024-05-30T16:54:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.689+0000 7f25d5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006170 0x7f25d014c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.689+0000 7f25d5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d014dd70 0x7f25d014e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.689+0000 7f25d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d01f3b40 0x7f25d01f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.689+0000 7f25d5400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f25d00b32c0 con 0x7f25d01f3b40 2024-05-30T16:54:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.689+0000 7f25d5400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25d00ae0c0 con 0x7f25b4006170 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.689+0000 7f25d5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25d00b3440 con 0x7f25d014dd70 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25d4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d01f3b40 0x7f25d01f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25d4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d01f3b40 0x7f25d01f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43276/0 (socket says 172.21.15.67:43276) 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25d4a00640 1 -- 172.21.15.67:0/1593452268 learned_addr learned my addr 172.21.15.67:0/1593452268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25c7e00640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d014dd70 0x7f25d014e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25cfe00640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006170 0x7f25d014c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25d4a00640 1 -- 172.21.15.67:0/1593452268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d014dd70 msgr2=0x7f25d014e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25d4a00640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d014dd70 0x7f25d014e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25d4a00640 1 -- 172.21.15.67:0/1593452268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006170 msgr2=0x7f25d014c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25d4a00640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006170 0x7f25d014c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25d4a00640 1 -- 172.21.15.67:0/1593452268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d014c570 con 0x7f25d01f3b40 2024-05-30T16:54:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.690+0000 7f25d4a00640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d01f3b40 0x7f25d01f3ff0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f25d0076c10 tx=0x7f25c800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.691+0000 7f25ccc00640 1 -- 172.21.15.67:0/1593452268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c801c070 con 0x7f25d01f3b40 2024-05-30T16:54:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.691+0000 7f25ccc00640 1 -- 172.21.15.67:0/1593452268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c800fe50 con 0x7f25d01f3b40 2024-05-30T16:54:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.691+0000 7f25ccc00640 1 -- 172.21.15.67:0/1593452268 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8017640 con 0x7f25d01f3b40 2024-05-30T16:54:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.691+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d014e910 con 0x7f25d01f3b40 2024-05-30T16:54:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.691+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d014eed0 con 0x7f25d01f3b40 2024-05-30T16:54:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.692+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d00ad780 con 0x7f25d01f3b40 2024-05-30T16:54:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.693+0000 7f25ccc00640 1 -- 172.21.15.67:0/1593452268 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f25c8005d10 con 0x7f25d01f3b40 2024-05-30T16:54:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.695+0000 7f25ccc00640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25ac06d610 0x7f25ac06fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.695+0000 7f25ccc00640 1 -- 172.21.15.67:0/1593452268 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f25c808a770 con 0x7f25d01f3b40 2024-05-30T16:54:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.695+0000 7f25cfe00640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25ac06d610 0x7f25ac06fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.696+0000 7f25cfe00640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25ac06d610 0x7f25ac06fad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f25c0007fd0 tx=0x7f25c0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.699+0000 7f25ccc00640 1 -- 172.21.15.67:0/1593452268 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25d00ad780 con 0x7f25d01f3b40 2024-05-30T16:54:49.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:49.979+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f25d00689e0 con 0x7f25d01f3b40 2024-05-30T16:54:50.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.538+0000 7f25ccc00640 1 -- 172.21.15.67:0/1593452268 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v448) ==== 177+0+0 (secure 0 0 0) 0x7f25c8012070 con 0x7f25d01f3b40 2024-05-30T16:54:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.541+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25ac06d610 msgr2=0x7f25ac06fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.541+0000 7f25d5400640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25ac06d610 0x7f25ac06fad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f25c0007fd0 tx=0x7f25c0009040 comp rx=0 tx=0).stop 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.541+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d01f3b40 msgr2=0x7f25d01f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.541+0000 7f25d5400640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d01f3b40 0x7f25d01f3ff0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f25d0076c10 tx=0x7f25c800f4a0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.542+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 shutdown_connections 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.542+0000 7f25d5400640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25ac06d610 0x7f25ac06fad0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.542+0000 7f25d5400640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d01f3b40 0x7f25d01f3ff0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.542+0000 7f25d5400640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25d014dd70 0x7f25d014e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.542+0000 7f25d5400640 1 --2- 172.21.15.67:0/1593452268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b4006170 0x7f25d014c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.542+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 >> 172.21.15.67:0/1593452268 conn(0x7f25d00b1fc0 msgr2=0x7f25d00b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.542+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 shutdown_connections 2024-05-30T16:54:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.542+0000 7f25d5400640 1 -- 172.21.15.67:0/1593452268 wait complete. 2024-05-30T16:54:50.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:54:50.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.774+0000 7f99b8c00640 1 Processor -- start 2024-05-30T16:54:50.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.775+0000 7f99b8c00640 1 -- start start 2024-05-30T16:54:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.776+0000 7f99b8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99b415e870 0x7f99b40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.776+0000 7f99b8c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f99b40cfa40 0x7f99b40d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.776+0000 7f99b8c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b415ec50 con 0x7f99b40d14d0 2024-05-30T16:54:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.776+0000 7f99b8c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f99b40d2150 con 0x7f99b415e870 2024-05-30T16:54:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.776+0000 7f99b8c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f99b40cff80 con 0x7f99b40cfa40 2024-05-30T16:54:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.776+0000 7f99b3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99b415e870 0x7f99b40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.776+0000 7f99b3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99b415e870 0x7f99b40d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43284/0 (socket says 172.21.15.67:43284) 2024-05-30T16:54:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.776+0000 7f99b3400640 1 -- 172.21.15.67:0/3676277611 learned_addr learned my addr 172.21.15.67:0/3676277611 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.777+0000 7f99b3e00640 1 --2- 172.21.15.67:0/3676277611 >> v2:172.21.15.169:3300/0 conn(0x7f99b40cfa40 0x7f99b40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.777+0000 7f99b3400640 1 -- 172.21.15.67:0/3676277611 >> v2:172.21.15.169:3300/0 conn(0x7f99b40cfa40 msgr2=0x7f99b40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.777+0000 7f99b3400640 1 --2- 172.21.15.67:0/3676277611 >> v2:172.21.15.169:3300/0 conn(0x7f99b40cfa40 0x7f99b40d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.777+0000 7f99b3400640 1 -- 172.21.15.67:0/3676277611 >> v1:172.21.15.67:6789/0 conn(0x7f99b40d14d0 legacy=0x7f99b40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.777+0000 7f99b3400640 1 -- 172.21.15.67:0/3676277611 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b40d0260 con 0x7f99b415e870 2024-05-30T16:54:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.778+0000 7f99b3400640 1 --2- 172.21.15.67:0/3676277611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b415e870 0x7f99b40d0f90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f99b408cad0 tx=0x7f99a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab8c698270c905e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b2000640 1 -- 172.21.15.67:0/3676277611 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4037070 con 0x7f99b415e870 2024-05-30T16:54:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b2000640 1 -- 172.21.15.67:0/3676277611 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a4032440 con 0x7f99b415e870 2024-05-30T16:54:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b2000640 1 -- 172.21.15.67:0/3676277611 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4032810 con 0x7f99b415e870 2024-05-30T16:54:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b8c00640 1 -- 172.21.15.67:0/3676277611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b415e870 msgr2=0x7f99b40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b8c00640 1 --2- 172.21.15.67:0/3676277611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b415e870 0x7f99b40d0f90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f99b408cad0 tx=0x7f99a402a270 comp rx=0 tx=0).stop 2024-05-30T16:54:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b8c00640 1 -- 172.21.15.67:0/3676277611 shutdown_connections 2024-05-30T16:54:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b8c00640 1 --2- 172.21.15.67:0/3676277611 >> v2:172.21.15.169:3300/0 conn(0x7f99b40cfa40 0x7f99b40d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b8c00640 1 --2- 172.21.15.67:0/3676277611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b415e870 0x7f99b40d0f90 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b8c00640 1 -- 172.21.15.67:0/3676277611 >> 172.21.15.67:0/3676277611 conn(0x7f99b408bfc0 msgr2=0x7f99b40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:50.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b8c00640 1 -- 172.21.15.67:0/3676277611 shutdown_connections 2024-05-30T16:54:50.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.779+0000 7f99b8c00640 1 -- 172.21.15.67:0/3676277611 wait complete. 2024-05-30T16:54:50.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.781+0000 7f99b8c00640 1 Processor -- start 2024-05-30T16:54:50.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.781+0000 7f99b8c00640 1 -- start start 2024-05-30T16:54:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.782+0000 7f99b8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99b40cfa40 0x7f99b40cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.782+0000 7f99b8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b40d14d0 0x7f99b40cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.782+0000 7f99b8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b415e870 0x7f99b414f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.782+0000 7f99b8c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f99b40d4fb0 con 0x7f99b40d14d0 2024-05-30T16:54:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.782+0000 7f99b8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f99b40d0820 con 0x7f99b415e870 2024-05-30T16:54:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.782+0000 7f99b8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f99b40d2b20 con 0x7f99b40cfa40 2024-05-30T16:54:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.782+0000 7f99b2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b40d14d0 0x7f99b40cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.782+0000 7f99b2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b40d14d0 0x7f99b40cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49140/0 (socket says 172.21.15.67:49140) 2024-05-30T16:54:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.782+0000 7f99b2a00640 1 -- 172.21.15.67:0/3587968785 learned_addr learned my addr 172.21.15.67:0/3587968785 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99b3400640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99b40cfa40 0x7f99b40cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99b2a00640 1 -- 172.21.15.67:0/3587968785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99b40cfa40 msgr2=0x7f99b40cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99b2a00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99b40cfa40 0x7f99b40cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99b2a00640 1 -- 172.21.15.67:0/3587968785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b415e870 msgr2=0x7f99b414f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99b3e00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b415e870 0x7f99b414f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99b2a00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b415e870 0x7f99b414f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99b2a00640 1 -- 172.21.15.67:0/3587968785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b40b7240 con 0x7f99b40d14d0 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99b3400640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99b40cfa40 0x7f99b40cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99b2a00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b40d14d0 0x7f99b40cdf50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c7a0 tx=0x7f999c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.783+0000 7f99ab400640 1 -- 172.21.15.67:0/3587968785 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c017070 con 0x7f99b40d14d0 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.784+0000 7f99ab400640 1 -- 172.21.15.67:0/3587968785 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f999c003da0 con 0x7f99b40d14d0 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.784+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b40b74b0 con 0x7f99b40d14d0 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.784+0000 7f99ab400640 1 -- 172.21.15.67:0/3587968785 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c005290 con 0x7f99b40d14d0 2024-05-30T16:54:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.784+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b40b7a70 con 0x7f99b40d14d0 2024-05-30T16:54:50.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.785+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b40d28e0 con 0x7f99b40d14d0 2024-05-30T16:54:50.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.786+0000 7f99ab400640 1 -- 172.21.15.67:0/3587968785 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f999c01a070 con 0x7f99b40d14d0 2024-05-30T16:54:50.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.787+0000 7f99ab400640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f999006d5e0 0x7f999006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:50.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.787+0000 7f99ab400640 1 -- 172.21.15.67:0/3587968785 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f999c089c70 con 0x7f99b40d14d0 2024-05-30T16:54:50.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.787+0000 7f99b3400640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f999006d5e0 0x7f999006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:50.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.787+0000 7f99b3400640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f999006d5e0 0x7f999006faa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99a4034000 tx=0x7f99a402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:50.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.791+0000 7f99ab400640 1 -- 172.21.15.67:0/3587968785 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99b40d28e0 con 0x7f99b40d14d0 2024-05-30T16:54:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.988+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f99b40cf610 con 0x7f99b40d14d0 2024-05-30T16:54:50.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.995+0000 7f99ab400640 1 -- 172.21.15.67:0/3587968785 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f999c059a60 con 0x7f99b40d14d0 2024-05-30T16:54:50.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f999006d5e0 msgr2=0x7f999006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:50.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f999006d5e0 0x7f999006faa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f99a4034000 tx=0x7f99a402e000 comp rx=0 tx=0).stop 2024-05-30T16:54:50.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b40d14d0 msgr2=0x7f99b40cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:50.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b40d14d0 0x7f99b40cdf50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c7a0 tx=0x7f999c00cc60 comp rx=0 tx=0).stop 2024-05-30T16:54:50.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 shutdown_connections 2024-05-30T16:54:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f999006d5e0 0x7f999006faa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b415e870 0x7f99b414f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99b40d14d0 0x7f99b40cdf50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 --2- 172.21.15.67:0/3587968785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99b40cfa40 0x7f99b40cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 >> 172.21.15.67:0/3587968785 conn(0x7f99b408bfc0 msgr2=0x7f99b40b6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 shutdown_connections 2024-05-30T16:54:51.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:50.998+0000 7f99b8c00640 1 -- 172.21.15.67:0/3587968785 wait complete. 2024-05-30T16:54:51.019 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T16:54:51.019 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:54:51.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:54:51.242 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T16:54:51.242 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:54:51.242 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:54:51.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:54:51.301 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:54:51.301 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:54:51.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:54:51.359 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:54:51.359 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:54:51.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:54:51.419 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:54:51.420 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:54:51.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:54:51.476 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-30T16:54:51.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:54:51.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.698+0000 7f3686a00640 1 Processor -- start 2024-05-30T16:54:51.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.698+0000 7f3686a00640 1 -- start start 2024-05-30T16:54:51.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.700+0000 7f3686a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f368015e870 0x7f36800d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:51.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.700+0000 7f3686a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f36800cfa40 0x7f36800d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:51.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.700+0000 7f3686a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f368015ec50 con 0x7f36800d14d0 2024-05-30T16:54:51.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.700+0000 7f3686a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f36800d2150 con 0x7f36800cfa40 2024-05-30T16:54:51.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.700+0000 7f3686a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f36800cff80 con 0x7f368015e870 2024-05-30T16:54:51.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.700+0000 7f3686000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f36800cfa40 0x7f36800d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:51.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.700+0000 7f3686000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f36800cfa40 0x7f36800d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49146/0 (socket says 172.21.15.67:49146) 2024-05-30T16:54:51.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.700+0000 7f3686000640 1 -- 172.21.15.67:0/670589113 learned_addr learned my addr 172.21.15.67:0/670589113 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:51.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.701+0000 7f3685600640 1 --2- 172.21.15.67:0/670589113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f368015e870 0x7f36800d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:51.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.701+0000 7f3686000640 1 -- 172.21.15.67:0/670589113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f368015e870 msgr2=0x7f36800d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:51.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.701+0000 7f3686000640 1 --2- 172.21.15.67:0/670589113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f368015e870 0x7f36800d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.701+0000 7f3686000640 1 -- 172.21.15.67:0/670589113 >> v1:172.21.15.169:6789/0 conn(0x7f36800d14d0 legacy=0x7f36800cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:51.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.701+0000 7f3686000640 1 -- 172.21.15.67:0/670589113 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36800d0260 con 0x7f36800cfa40 2024-05-30T16:54:51.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.702+0000 7f3686000640 1 --2- 172.21.15.67:0/670589113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 0x7f36800d25e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3678009e50 tx=0x7f367802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe92fffacf95964 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:51.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.702+0000 7f367fe00640 1 -- 172.21.15.67:0/670589113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3678036070 con 0x7f36800cfa40 2024-05-30T16:54:51.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.702+0000 7f367fe00640 1 -- 172.21.15.67:0/670589113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3678031600 con 0x7f36800cfa40 2024-05-30T16:54:51.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.702+0000 7f367fe00640 1 -- 172.21.15.67:0/670589113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36780319d0 con 0x7f36800cfa40 2024-05-30T16:54:51.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.703+0000 7f3686a00640 1 -- 172.21.15.67:0/670589113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 msgr2=0x7f36800d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:51.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.703+0000 7f3686a00640 1 --2- 172.21.15.67:0/670589113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 0x7f36800d25e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3678009e50 tx=0x7f367802a6d0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.703+0000 7f3686a00640 1 -- 172.21.15.67:0/670589113 shutdown_connections 2024-05-30T16:54:51.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.703+0000 7f3686a00640 1 --2- 172.21.15.67:0/670589113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 0x7f36800d25e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.703+0000 7f3686a00640 1 --2- 172.21.15.67:0/670589113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f368015e870 0x7f36800d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.703+0000 7f3686a00640 1 -- 172.21.15.67:0/670589113 >> 172.21.15.67:0/670589113 conn(0x7f368008bfc0 msgr2=0x7f36800d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:51.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.703+0000 7f3686a00640 1 -- 172.21.15.67:0/670589113 shutdown_connections 2024-05-30T16:54:51.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.703+0000 7f3686a00640 1 -- 172.21.15.67:0/670589113 wait complete. 2024-05-30T16:54:51.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.704+0000 7f3686a00640 1 Processor -- start 2024-05-30T16:54:51.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.705+0000 7f3686a00640 1 -- start start 2024-05-30T16:54:51.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.705+0000 7f3686a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 0x7f368014c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:51.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.705+0000 7f3686a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36800d14d0 0x7f3680160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:51.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.705+0000 7f3686a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f368015e870 0x7f36800b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3686a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f36800d4fb0 con 0x7f36800cfa40 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3686a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f36800d0820 con 0x7f368015e870 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3686a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f36800d2b20 con 0x7f36800d14d0 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3685600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 0x7f368014c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3685600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 0x7f368014c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49148/0 (socket says 172.21.15.67:49148) 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3685600640 1 -- 172.21.15.67:0/976604091 learned_addr learned my addr 172.21.15.67:0/976604091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3684c00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36800d14d0 0x7f3680160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3685600640 1 -- 172.21.15.67:0/976604091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36800d14d0 msgr2=0x7f3680160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3685600640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36800d14d0 0x7f3680160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3685600640 1 -- 172.21.15.67:0/976604091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f368015e870 msgr2=0x7f36800b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3685600640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f368015e870 0x7f36800b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3685600640 1 -- 172.21.15.67:0/976604091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368014cee0 con 0x7f36800cfa40 2024-05-30T16:54:51.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.706+0000 7f3684c00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36800d14d0 0x7f3680160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:51.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.707+0000 7f3685600640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 0x7f368014c9a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f367400d5f0 tx=0x7f367400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:51.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.707+0000 7f367d600640 1 -- 172.21.15.67:0/976604091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36740125f0 con 0x7f36800cfa40 2024-05-30T16:54:51.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.707+0000 7f367d600640 1 -- 172.21.15.67:0/976604091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3674019070 con 0x7f36800cfa40 2024-05-30T16:54:51.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.707+0000 7f367d600640 1 -- 172.21.15.67:0/976604091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36740129d0 con 0x7f36800cfa40 2024-05-30T16:54:51.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.707+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3680161470 con 0x7f36800cfa40 2024-05-30T16:54:51.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.707+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3680161a30 con 0x7f36800cfa40 2024-05-30T16:54:51.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.708+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36800689e0 con 0x7f36800cfa40 2024-05-30T16:54:51.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.709+0000 7f367d600640 1 -- 172.21.15.67:0/976604091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f3674012b70 con 0x7f36800cfa40 2024-05-30T16:54:51.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.710+0000 7f367d600640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f366006d5e0 0x7f366006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:51.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.710+0000 7f367d600640 1 -- 172.21.15.67:0/976604091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f367408ac30 con 0x7f36800cfa40 2024-05-30T16:54:51.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.710+0000 7f3684c00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f366006d5e0 0x7f366006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:51.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.711+0000 7f3684c00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f366006d5e0 0x7f366006faa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3680161e90 tx=0x7f366c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:51.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.715+0000 7f367d600640 1 -- 172.21.15.67:0/976604091 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f368014cee0 con 0x7f36800cfa40 2024-05-30T16:54:51.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.948+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36800d28a0 con 0x7f36800cfa40 2024-05-30T16:54:51.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.949+0000 7f367d600640 1 -- 172.21.15.67:0/976604091 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3872 (secure 0 0 0) 0x7f367405a820 con 0x7f36800cfa40 2024-05-30T16:54:51.950 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:51.950 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2932035652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2932035652},{"type":"v1","addr":"172.21.15.67:6837","nonce":2932035652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2940614672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2940614672},{"type":"v1","addr":"172.21.15.169:6835","nonce":2940614672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/327493818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":327493818},{"type":"v1","addr":"172.21.15.67:6836","nonce":327493818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1437754767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1437754767},{"type":"v1","addr":"172.21.15.169:6834","nonce":1437754767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-05-30T16:54:51.950 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 38 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f366006d5e0 msgr2=0x7f366006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f366006d5e0 0x7f366006faa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3680161e90 tx=0x7f366c009040 comp rx=0 tx=0).stop 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 msgr2=0x7f368014c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 0x7f368014c9a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f367400d5f0 tx=0x7f367400dab0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 shutdown_connections 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f366006d5e0 0x7f366006faa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f368015e870 0x7f36800b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36800d14d0 0x7f3680160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 --2- 172.21.15.67:0/976604091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36800cfa40 0x7f368014c9a0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:51.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 >> 172.21.15.67:0/976604091 conn(0x7f368008bfc0 msgr2=0x7f36800b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 shutdown_connections 2024-05-30T16:54:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:51.952+0000 7f3686a00640 1 -- 172.21.15.67:0/976604091 wait complete. 2024-05-30T16:54:51.971 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-30T16:54:51.971 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-30T16:54:51.972 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-30T16:54:51.972 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:54:51.972 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-30T16:54:51.972 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-30T16:54:51.972 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-30T16:54:51.972 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:54:51.972 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-30T16:54:51.973 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-30T16:54:51.973 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-30T16:54:51.973 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:54:51.973 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-30T16:54:51.973 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-30T16:54:51.973 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-30T16:54:51.973 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:54:51.987 INFO:tasks.ceph.mds.c:Stopped 2024-05-30T16:54:51.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-30T16:54:51.995 INFO:tasks.ceph.mds.b:Stopped 2024-05-30T16:54:51.995 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-30T16:54:51.996 INFO:tasks.ceph.mds.d:Stopped 2024-05-30T16:54:51.996 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-30T16:54:51.998 INFO:tasks.ceph.mds.a:Stopped 2024-05-30T16:54:51.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-30T16:54:52.043 INFO:tasks.ceph.mds.b:Started 2024-05-30T16:54:52.043 DEBUG:teuthology.parallel:result is None 2024-05-30T16:54:52.044 INFO:tasks.ceph.mds.c:Started 2024-05-30T16:54:52.044 DEBUG:teuthology.parallel:result is None 2024-05-30T16:54:52.048 INFO:tasks.ceph.mds.d:Started 2024-05-30T16:54:52.048 DEBUG:teuthology.parallel:result is None 2024-05-30T16:54:52.050 INFO:tasks.ceph.mds.a:Started 2024-05-30T16:54:52.050 DEBUG:teuthology.parallel:result is None 2024-05-30T16:54:52.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-30T16:54:52.186 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T16:54:52.193 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T16:54:52.196 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T16:54:52.198 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T16:54:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.226+0000 7f4aca600640 1 Processor -- start 2024-05-30T16:54:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.226+0000 7f4aca600640 1 -- start start 2024-05-30T16:54:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.227+0000 7f4aca600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4ac406f6e0 0x7f4ac40cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.227+0000 7f4aca600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 0x7f4ac40ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.227+0000 7f4aca600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac40b4350 con 0x7f4ac40d04a0 2024-05-30T16:54:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.227+0000 7f4aca600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac40b3ec0 con 0x7f4ac406f6e0 2024-05-30T16:54:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.227+0000 7f4aca600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac40b32d0 con 0x7f4ac4162e70 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.227+0000 7f4ac3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4ac40d04a0 0x7f4ac40d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34544/0 (socket says 172.21.15.67:34544) 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.227+0000 7f4ac3400640 1 -- 172.21.15.67:0/2729387591 learned_addr learned my addr 172.21.15.67:0/2729387591 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.227+0000 7f4ac3e00640 1 --2- 172.21.15.67:0/2729387591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 0x7f4ac40ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.227+0000 7f4ac2a00640 1 --2- 172.21.15.67:0/2729387591 >> v2:172.21.15.169:3300/0 conn(0x7f4ac406f6e0 0x7f4ac40cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.228+0000 7f4ac2000640 1 -- 172.21.15.67:0/2729387591 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3187468074 0 0) 0x7f4ac40b4350 con 0x7f4ac40d04a0 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.228+0000 7f4ac2000640 1 -- 172.21.15.67:0/2729387591 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4aa8003520 con 0x7f4ac40d04a0 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.228+0000 7f4ac3e00640 1 -- 172.21.15.67:0/2729387591 >> v2:172.21.15.169:3300/0 conn(0x7f4ac406f6e0 msgr2=0x7f4ac40cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.228+0000 7f4ac3e00640 1 --2- 172.21.15.67:0/2729387591 >> v2:172.21.15.169:3300/0 conn(0x7f4ac406f6e0 0x7f4ac40cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.228+0000 7f4ac3e00640 1 -- 172.21.15.67:0/2729387591 >> v1:172.21.15.67:6789/0 conn(0x7f4ac40d04a0 legacy=0x7f4ac40d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.228+0000 7f4ac3e00640 1 -- 172.21.15.67:0/2729387591 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac40b3590 con 0x7f4ac4162e70 2024-05-30T16:54:52.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.229+0000 7f4ac3e00640 1 --2- 172.21.15.67:0/2729387591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 0x7f4ac40ce230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=663ffdaecd515111 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:52.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.229+0000 7f4ac2000640 1 -- 172.21.15.67:0/2729387591 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc031600 con 0x7f4ac4162e70 2024-05-30T16:54:52.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.229+0000 7f4ac2000640 1 -- 172.21.15.67:0/2729387591 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4abc0317a0 con 0x7f4ac4162e70 2024-05-30T16:54:52.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.229+0000 7f4ac2000640 1 -- 172.21.15.67:0/2729387591 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc031b70 con 0x7f4ac4162e70 2024-05-30T16:54:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.230+0000 7f4aca600640 1 -- 172.21.15.67:0/2729387591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 msgr2=0x7f4ac40ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.230+0000 7f4aca600640 1 --2- 172.21.15.67:0/2729387591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 0x7f4ac40ce230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc02a5d0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.230+0000 7f4aca600640 1 -- 172.21.15.67:0/2729387591 shutdown_connections 2024-05-30T16:54:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.230+0000 7f4aca600640 1 --2- 172.21.15.67:0/2729387591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 0x7f4ac40ce230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.230+0000 7f4aca600640 1 --2- 172.21.15.67:0/2729387591 >> v2:172.21.15.169:3300/0 conn(0x7f4ac406f6e0 0x7f4ac40cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.230+0000 7f4aca600640 1 -- 172.21.15.67:0/2729387591 >> 172.21.15.67:0/2729387591 conn(0x7f4ac406bae0 msgr2=0x7f4ac406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.230+0000 7f4aca600640 1 -- 172.21.15.67:0/2729387591 shutdown_connections 2024-05-30T16:54:52.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.230+0000 7f4aca600640 1 -- 172.21.15.67:0/2729387591 wait complete. 2024-05-30T16:54:52.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.232+0000 7f4aca600640 1 Processor -- start 2024-05-30T16:54:52.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.232+0000 7f4aca600640 1 -- start start 2024-05-30T16:54:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4aca600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ac406f6e0 0x7f4ac4165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4aca600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ac40d04a0 0x7f4ac4163d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4aca600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 0x7f4ac40ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4aca600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac40cda30 con 0x7f4ac40d04a0 2024-05-30T16:54:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4aca600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac40b3b50 con 0x7f4ac406f6e0 2024-05-30T16:54:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4aca600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac40ce770 con 0x7f4ac4162e70 2024-05-30T16:54:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4ac2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ac40d04a0 0x7f4ac4163d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4ac3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ac406f6e0 0x7f4ac4165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4ac2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ac40d04a0 0x7f4ac4163d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49174/0 (socket says 172.21.15.67:49174) 2024-05-30T16:54:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4ac2a00640 1 -- 172.21.15.67:0/3872865043 learned_addr learned my addr 172.21.15.67:0/3872865043 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.233+0000 7f4ac3e00640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 0x7f4ac40ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4ac2a00640 1 -- 172.21.15.67:0/3872865043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 msgr2=0x7f4ac40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4ac2a00640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 0x7f4ac40ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4ac2a00640 1 -- 172.21.15.67:0/3872865043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ac406f6e0 msgr2=0x7f4ac4165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4ac2a00640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ac406f6e0 0x7f4ac4165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4ac2a00640 1 -- 172.21.15.67:0/3872865043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac40c4740 con 0x7f4ac40d04a0 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4ac2a00640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ac40d04a0 0x7f4ac4163d20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000e8b0 tx=0x7f4ab000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4abb400640 1 -- 172.21.15.67:0/3872865043 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0017070 con 0x7f4ac40d04a0 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4abb400640 1 -- 172.21.15.67:0/3872865043 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab000c940 con 0x7f4ac40d04a0 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac40c4a20 con 0x7f4ac40d04a0 2024-05-30T16:54:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4abb400640 1 -- 172.21.15.67:0/3872865043 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000cd10 con 0x7f4ac40d04a0 2024-05-30T16:54:52.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.234+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac4159f20 con 0x7f4ac40d04a0 2024-05-30T16:54:52.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.235+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac40ae0b0 con 0x7f4ac40d04a0 2024-05-30T16:54:52.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.237+0000 7f4abb400640 1 -- 172.21.15.67:0/3872865043 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f4ab001a070 con 0x7f4ac40d04a0 2024-05-30T16:54:52.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.238+0000 7f4abb400640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4aa406d5e0 0x7f4aa406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.238+0000 7f4abb400640 1 -- 172.21.15.67:0/3872865043 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f4ab008a8e0 con 0x7f4ac40d04a0 2024-05-30T16:54:52.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.238+0000 7f4ac3400640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4aa406d5e0 0x7f4aa406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.239+0000 7f4ac3400640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4aa406d5e0 0x7f4aa406faa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4005880 tx=0x7f4ab4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:52.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.242+0000 7f4abb400640 1 -- 172.21.15.67:0/3872865043 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ab005a080 con 0x7f4ac40d04a0 2024-05-30T16:54:52.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.457+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4ac40689e0 con 0x7f4ac40d04a0 2024-05-30T16:54:52.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.564+0000 7f4abb400640 1 -- 172.21.15.67:0/3872865043 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) ==== 97+0+0 (secure 0 0 0) 0x7f4ab005e380 con 0x7f4ac40d04a0 2024-05-30T16:54:52.565 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-05-30T16:54:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4aa406d5e0 msgr2=0x7f4aa406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4aa406d5e0 0x7f4aa406faa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4005880 tx=0x7f4ab4009040 comp rx=0 tx=0).stop 2024-05-30T16:54:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ac40d04a0 msgr2=0x7f4ac4163d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ac40d04a0 0x7f4ac4163d20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000e8b0 tx=0x7f4ab000ed70 comp rx=0 tx=0).stop 2024-05-30T16:54:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 shutdown_connections 2024-05-30T16:54:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4aa406d5e0 0x7f4aa406faa0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ac4162e70 0x7f4ac40ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ac40d04a0 0x7f4ac4163d20 secure :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000e8b0 tx=0x7f4ab000ed70 comp rx=0 tx=0).stop 2024-05-30T16:54:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 --2- 172.21.15.67:0/3872865043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ac406f6e0 0x7f4ac4165bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.568+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 >> 172.21.15.67:0/3872865043 conn(0x7f4ac406bae0 msgr2=0x7f4ac406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.569+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 shutdown_connections 2024-05-30T16:54:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.569+0000 7f4aca600640 1 -- 172.21.15.67:0/3872865043 wait complete. 2024-05-30T16:54:52.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-30T16:54:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.798+0000 7feecbe00640 1 Processor -- start 2024-05-30T16:54:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.798+0000 7feecbe00640 1 -- start start 2024-05-30T16:54:52.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.799+0000 7feecbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feec40d04a0 0x7feec40d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.799+0000 7feecbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feec406f6e0 0x7feec40cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feecbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec4162e70 0x7feec406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feecbe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7feec40b3ec0 con 0x7feec406f6e0 2024-05-30T16:54:52.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feecbe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7feec40b4350 con 0x7feec40d04a0 2024-05-30T16:54:52.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feecbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feec40b32d0 con 0x7feec4162e70 2024-05-30T16:54:52.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feec8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feec406f6e0 0x7feec40cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feec8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feec406f6e0 0x7feec40cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49186/0 (socket says 172.21.15.67:49186) 2024-05-30T16:54:52.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feec8a00640 1 -- 172.21.15.67:0/378384072 learned_addr learned my addr 172.21.15.67:0/378384072 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:52.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feec9e00640 1 --2- 172.21.15.67:0/378384072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec4162e70 0x7feec406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feec8a00640 1 -- 172.21.15.67:0/378384072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec4162e70 msgr2=0x7feec406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feec9400640 1 --2- 172.21.15.67:0/378384072 >> v2:172.21.15.169:3300/0 conn(0x7feec40d04a0 0x7feec40d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feec8a00640 1 --2- 172.21.15.67:0/378384072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec4162e70 0x7feec406e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.800+0000 7feec8a00640 1 -- 172.21.15.67:0/378384072 >> v2:172.21.15.169:3300/0 conn(0x7feec40d04a0 msgr2=0x7feec40d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.801+0000 7feec8a00640 1 --2- 172.21.15.67:0/378384072 >> v2:172.21.15.169:3300/0 conn(0x7feec40d04a0 0x7feec40d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.801+0000 7feec8a00640 1 -- 172.21.15.67:0/378384072 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec40b3590 con 0x7feec406f6e0 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.801+0000 7feec9400640 1 --2- 172.21.15.67:0/378384072 >> v2:172.21.15.169:3300/0 conn(0x7feec40d04a0 0x7feec40d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.802+0000 7feec8a00640 1 --2- 172.21.15.67:0/378384072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec406f6e0 0x7feec40cd4f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7feec40c3470 tx=0x7feeb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d77d628d775fc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.802+0000 7feec3e00640 1 -- 172.21.15.67:0/378384072 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4037070 con 0x7feec406f6e0 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.802+0000 7feec3e00640 1 -- 172.21.15.67:0/378384072 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb4032440 con 0x7feec406f6e0 2024-05-30T16:54:52.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.802+0000 7feec3e00640 1 -- 172.21.15.67:0/378384072 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4032810 con 0x7feec406f6e0 2024-05-30T16:54:52.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.803+0000 7feecbe00640 1 -- 172.21.15.67:0/378384072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec406f6e0 msgr2=0x7feec40cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:52.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.803+0000 7feecbe00640 1 --2- 172.21.15.67:0/378384072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec406f6e0 0x7feec40cd4f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7feec40c3470 tx=0x7feeb402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.803+0000 7feecbe00640 1 -- 172.21.15.67:0/378384072 shutdown_connections 2024-05-30T16:54:52.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.803+0000 7feecbe00640 1 --2- 172.21.15.67:0/378384072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec406f6e0 0x7feec40cd4f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.803+0000 7feecbe00640 1 --2- 172.21.15.67:0/378384072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec4162e70 0x7feec406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.803+0000 7feecbe00640 1 --2- 172.21.15.67:0/378384072 >> v2:172.21.15.169:3300/0 conn(0x7feec40d04a0 0x7feec40d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.803+0000 7feecbe00640 1 -- 172.21.15.67:0/378384072 >> 172.21.15.67:0/378384072 conn(0x7feec406bae0 msgr2=0x7feec406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:52.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.803+0000 7feecbe00640 1 -- 172.21.15.67:0/378384072 shutdown_connections 2024-05-30T16:54:52.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.803+0000 7feecbe00640 1 -- 172.21.15.67:0/378384072 wait complete. 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.804+0000 7feecbe00640 1 Processor -- start 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.805+0000 7feecbe00640 1 -- start start 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.805+0000 7feecbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec406f6e0 0x7feec40c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.805+0000 7feecbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feec40d04a0 0x7feec4163250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feecbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec4162e70 0x7feec4099550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feecbe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feec40d0dc0 con 0x7feec4162e70 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feecbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feec40b3b50 con 0x7feec40d04a0 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feecbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feec40cda30 con 0x7feec406f6e0 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec4162e70 0x7feec4099550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec4162e70 0x7feec4099550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49192/0 (socket says 172.21.15.67:49192) 2024-05-30T16:54:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec9e00640 1 -- 172.21.15.67:0/222042774 learned_addr learned my addr 172.21.15.67:0/222042774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec9400640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec406f6e0 0x7feec40c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec8a00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feec40d04a0 0x7feec4163250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec9e00640 1 -- 172.21.15.67:0/222042774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec406f6e0 msgr2=0x7feec40c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec9e00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec406f6e0 0x7feec40c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec9e00640 1 -- 172.21.15.67:0/222042774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feec40d04a0 msgr2=0x7feec4163250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec9e00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feec40d04a0 0x7feec4163250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec9e00640 1 -- 172.21.15.67:0/222042774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec4163d10 con 0x7feec4162e70 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.806+0000 7feec8a00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feec40d04a0 0x7feec4163250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.807+0000 7feec9e00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec4162e70 0x7feec4099550 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7feebc00cc80 tx=0x7feebc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:52.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.807+0000 7feec1600640 1 -- 172.21.15.67:0/222042774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc003990 con 0x7feec4162e70 2024-05-30T16:54:52.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.807+0000 7feec1600640 1 -- 172.21.15.67:0/222042774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc003b50 con 0x7feec4162e70 2024-05-30T16:54:52.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.807+0000 7feec1600640 1 -- 172.21.15.67:0/222042774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc004020 con 0x7feec4162e70 2024-05-30T16:54:52.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.807+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec4163ff0 con 0x7feec4162e70 2024-05-30T16:54:52.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.807+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec41645b0 con 0x7feec4162e70 2024-05-30T16:54:52.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.808+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec40689e0 con 0x7feec4162e70 2024-05-30T16:54:52.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.809+0000 7feec1600640 1 -- 172.21.15.67:0/222042774 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7feebc0041c0 con 0x7feec4162e70 2024-05-30T16:54:52.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.811+0000 7feec1600640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feea806d5e0 0x7feea806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:52.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.811+0000 7feec1600640 1 -- 172.21.15.67:0/222042774 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4913+0+0 (secure 0 0 0) 0x7feebc08aa30 con 0x7feec4162e70 2024-05-30T16:54:52.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.811+0000 7feec9400640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feea806d5e0 0x7feea806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:52.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.812+0000 7feec9400640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feea806d5e0 0x7feea806faa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feeb8006b10 tx=0x7feeb8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:52.815+0000 7feec1600640 1 -- 172.21.15.67:0/222042774 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feebc014030 con 0x7feec4162e70 2024-05-30T16:54:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.094+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7feec4072520 con 0x7feec4162e70 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.095+0000 7feec1600640 1 -- 172.21.15.67:0/222042774 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7feebc05a8f0 con 0x7feec4162e70 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA==", 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:54:53.096 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw==", 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.097 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw==", 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg==", 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-05-30T16:54:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng==", 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg==", 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.100 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w==", 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ==", 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.102 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A==", 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ==", 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w==", 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A==", 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:54:53.104 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg==", 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-05-30T16:54:53.105 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ==", 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.2", 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ==", 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:54:53.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.3", 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A==", 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:54:53.107 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCArFhmF5mhNhAA4H3cgcH6YdD53fANDW4seA==", 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ==", 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw==", 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:54:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:54:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:54:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:54:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-05-30T16:54:53.110 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:54:53.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feea806d5e0 msgr2=0x7feea806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feea806d5e0 0x7feea806faa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feeb8006b10 tx=0x7feeb8008040 comp rx=0 tx=0).stop 2024-05-30T16:54:53.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec4162e70 msgr2=0x7feec4099550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec4162e70 0x7feec4099550 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7feebc00cc80 tx=0x7feebc00a5c0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 shutdown_connections 2024-05-30T16:54:53.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feea806d5e0 0x7feea806faa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feec4162e70 0x7feec4099550 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feec40d04a0 0x7feec4163250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 --2- 172.21.15.67:0/222042774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feec406f6e0 0x7feec40c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 >> 172.21.15.67:0/222042774 conn(0x7feec406bae0 msgr2=0x7feec4071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:53.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 shutdown_connections 2024-05-30T16:54:53.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.098+0000 7feecbe00640 1 -- 172.21.15.67:0/222042774 wait complete. 2024-05-30T16:54:53.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-30T16:54:53.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.334+0000 7fa054000640 1 Processor -- start 2024-05-30T16:54:53.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.334+0000 7fa054000640 1 -- start start 2024-05-30T16:54:53.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.336+0000 7fa054000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa04c06f860 0x7fa04c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.336+0000 7fa054000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa04c0d0bb0 0x7fa04c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.336+0000 7fa054000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04c06ed10 con 0x7fa04c0b3690 2024-05-30T16:54:53.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.336+0000 7fa054000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c15cb00 con 0x7fa04c06f860 2024-05-30T16:54:53.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.336+0000 7fa054000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c0b3cc0 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.336+0000 7fa051600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa04c06f860 0x7fa04c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.336+0000 7fa051600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa04c06f860 0x7fa04c0b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49206/0 (socket says 172.21.15.67:49206) 2024-05-30T16:54:53.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.336+0000 7fa051600640 1 -- 172.21.15.67:0/1431654319 learned_addr learned my addr 172.21.15.67:0/1431654319 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:53.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.336+0000 7fa050c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa04c0b3690 0x7fa04c0d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34550/0 (socket says 172.21.15.67:34550) 2024-05-30T16:54:53.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.337+0000 7fa052000640 1 --2- 172.21.15.67:0/1431654319 >> v2:172.21.15.169:3300/0 conn(0x7fa04c0d0bb0 0x7fa04c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.337+0000 7fa051600640 1 -- 172.21.15.67:0/1431654319 >> v2:172.21.15.169:3300/0 conn(0x7fa04c0d0bb0 msgr2=0x7fa04c0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:53.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.337+0000 7fa051600640 1 --2- 172.21.15.67:0/1431654319 >> v2:172.21.15.169:3300/0 conn(0x7fa04c0d0bb0 0x7fa04c0ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.337+0000 7fa051600640 1 -- 172.21.15.67:0/1431654319 >> v1:172.21.15.67:6789/0 conn(0x7fa04c0b3690 legacy=0x7fa04c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.337+0000 7fa051600640 1 -- 172.21.15.67:0/1431654319 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0b3fa0 con 0x7fa04c06f860 2024-05-30T16:54:53.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.338+0000 7fa051600640 1 --2- 172.21.15.67:0/1431654319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c06f860 0x7fa04c0b3150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0af6e0 tx=0x7fa04002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b9d14cddd245ca server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:53.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.339+0000 7fa04be00640 1 -- 172.21.15.67:0/1431654319 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040037070 con 0x7fa04c06f860 2024-05-30T16:54:53.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.339+0000 7fa04be00640 1 -- 172.21.15.67:0/1431654319 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa040032440 con 0x7fa04c06f860 2024-05-30T16:54:53.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.339+0000 7fa04be00640 1 -- 172.21.15.67:0/1431654319 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040032810 con 0x7fa04c06f860 2024-05-30T16:54:53.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.339+0000 7fa054000640 1 -- 172.21.15.67:0/1431654319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c06f860 msgr2=0x7fa04c0b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.339+0000 7fa054000640 1 --2- 172.21.15.67:0/1431654319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c06f860 0x7fa04c0b3150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0af6e0 tx=0x7fa04002a270 comp rx=0 tx=0).stop 2024-05-30T16:54:53.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.339+0000 7fa054000640 1 -- 172.21.15.67:0/1431654319 shutdown_connections 2024-05-30T16:54:53.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.339+0000 7fa054000640 1 --2- 172.21.15.67:0/1431654319 >> v2:172.21.15.169:3300/0 conn(0x7fa04c0d0bb0 0x7fa04c0ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.340+0000 7fa054000640 1 --2- 172.21.15.67:0/1431654319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c06f860 0x7fa04c0b3150 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.340+0000 7fa054000640 1 -- 172.21.15.67:0/1431654319 >> 172.21.15.67:0/1431654319 conn(0x7fa04c06bae0 msgr2=0x7fa04c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:53.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.340+0000 7fa054000640 1 -- 172.21.15.67:0/1431654319 shutdown_connections 2024-05-30T16:54:53.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.340+0000 7fa054000640 1 -- 172.21.15.67:0/1431654319 wait complete. 2024-05-30T16:54:53.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.341+0000 7fa054000640 1 Processor -- start 2024-05-30T16:54:53.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.341+0000 7fa054000640 1 -- start start 2024-05-30T16:54:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.342+0000 7fa054000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa04c06f860 0x7fa04c0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.342+0000 7fa054000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa04c0b3690 0x7fa04c1f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.342+0000 7fa054000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c0d0bb0 0x7fa04c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.342+0000 7fa054000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c0b4840 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.342+0000 7fa054000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c0b4560 con 0x7fa04c06f860 2024-05-30T16:54:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.342+0000 7fa054000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c0ad670 con 0x7fa04c0b3690 2024-05-30T16:54:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.343+0000 7fa052000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c0d0bb0 0x7fa04c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.343+0000 7fa052000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c0d0bb0 0x7fa04c15f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49214/0 (socket says 172.21.15.67:49214) 2024-05-30T16:54:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.343+0000 7fa052000640 1 -- 172.21.15.67:0/3105849657 learned_addr learned my addr 172.21.15.67:0/3105849657 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:53.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.343+0000 7fa051600640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa04c06f860 0x7fa04c0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.343+0000 7fa052000640 1 -- 172.21.15.67:0/3105849657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa04c0b3690 msgr2=0x7fa04c1f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.343+0000 7fa052000640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa04c0b3690 0x7fa04c1f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.343+0000 7fa052000640 1 -- 172.21.15.67:0/3105849657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa04c06f860 msgr2=0x7fa04c0c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.343+0000 7fa052000640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa04c06f860 0x7fa04c0c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.343+0000 7fa052000640 1 -- 172.21.15.67:0/3105849657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0ca1e0 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.344+0000 7fa052000640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c0d0bb0 0x7fa04c15f050 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa044007e60 tx=0x7fa04400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.344+0000 7fa049600640 1 -- 172.21.15.67:0/3105849657 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044014420 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.344+0000 7fa049600640 1 -- 172.21.15.67:0/3105849657 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa044019070 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.344+0000 7fa049600640 1 -- 172.21.15.67:0/3105849657 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044014820 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.344+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c158fd0 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.344+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c1f5d40 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.345+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c06fdc0 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.346+0000 7fa049600640 1 -- 172.21.15.67:0/3105849657 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fa0440149c0 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.347+0000 7fa049600640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa02c06d5e0 0x7fa02c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.347+0000 7fa049600640 1 -- 172.21.15.67:0/3105849657 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4913+0+0 (secure 0 0 0) 0x7fa04408ada0 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.348+0000 7fa051600640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa02c06d5e0 0x7fa02c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.348+0000 7fa051600640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa02c06d5e0 0x7fa02c06faa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa040034000 tx=0x7fa04002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:53.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.352+0000 7fa049600640 1 -- 172.21.15.67:0/3105849657 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa04c06fdc0 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.561+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa04c0689e0 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.562+0000 7fa049600640 1 -- 172.21.15.67:0/3105849657 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7fa04405ac80 con 0x7fa04c0d0bb0 2024-05-30T16:54:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:53.563 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:54:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:54:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-05-30T16:54:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-05-30T16:54:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:54:53.563 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:54:53.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa02c06d5e0 msgr2=0x7fa02c06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa02c06d5e0 0x7fa02c06faa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa040034000 tx=0x7fa04002e000 comp rx=0 tx=0).stop 2024-05-30T16:54:53.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c0d0bb0 msgr2=0x7fa04c15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c0d0bb0 0x7fa04c15f050 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa044007e60 tx=0x7fa04400e4b0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 shutdown_connections 2024-05-30T16:54:53.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa02c06d5e0 0x7fa02c06faa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa04c0d0bb0 0x7fa04c15f050 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa04c0b3690 0x7fa04c1f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 --2- 172.21.15.67:0/3105849657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa04c06f860 0x7fa04c0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 >> 172.21.15.67:0/3105849657 conn(0x7fa04c06bae0 msgr2=0x7fa04c0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:53.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 shutdown_connections 2024-05-30T16:54:53.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.565+0000 7fa054000640 1 -- 172.21.15.67:0/3105849657 wait complete. 2024-05-30T16:54:53.586 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-30T16:54:53.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-30T16:54:53.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.780+0000 7fc1a6000640 1 Processor -- start 2024-05-30T16:54:53.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.780+0000 7fc1a6000640 1 -- start start 2024-05-30T16:54:53.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.782+0000 7fc1a6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a01621d0 0x7fc1a00d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.782+0000 7fc1a6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc1a01625b0 0x7fc1a00d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.782+0000 7fc1a6000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a00d0e50 con 0x7fc1a015ea10 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.782+0000 7fc1a6000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a00d1a60 con 0x7fc1a01625b0 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.782+0000 7fc1a6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a00d1130 con 0x7fc1a01621d0 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.782+0000 7fc19fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc1a01625b0 0x7fc1a00d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.782+0000 7fc19fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc1a01625b0 0x7fc1a00d4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49224/0 (socket says 172.21.15.67:49224) 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.782+0000 7fc19fe00640 1 -- 172.21.15.67:0/2966995984 learned_addr learned my addr 172.21.15.67:0/2966995984 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.783+0000 7fc1a4c00640 1 --2- 172.21.15.67:0/2966995984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a01621d0 0x7fc1a00d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.783+0000 7fc19fe00640 1 -- 172.21.15.67:0/2966995984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a01621d0 msgr2=0x7fc1a00d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.783+0000 7fc19fe00640 1 --2- 172.21.15.67:0/2966995984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a01621d0 0x7fc1a00d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.783+0000 7fc19fe00640 1 -- 172.21.15.67:0/2966995984 >> v1:172.21.15.169:6789/0 conn(0x7fc1a015ea10 legacy=0x7fc1a015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.783+0000 7fc19fe00640 1 -- 172.21.15.67:0/2966995984 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a00d1410 con 0x7fc1a01625b0 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.784+0000 7fc19fe00640 1 --2- 172.21.15.67:0/2966995984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 0x7fc1a00d4f30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009850 tx=0x7fc18c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db713026da5f51db server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.784+0000 7fc19f400640 1 -- 172.21.15.67:0/2966995984 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004030 con 0x7fc1a01625b0 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.784+0000 7fc19f400640 1 -- 172.21.15.67:0/2966995984 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c0041d0 con 0x7fc1a01625b0 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.784+0000 7fc19f400640 1 -- 172.21.15.67:0/2966995984 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c003610 con 0x7fc1a01625b0 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.785+0000 7fc1a6000640 1 -- 172.21.15.67:0/2966995984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 msgr2=0x7fc1a00d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.785+0000 7fc1a6000640 1 --2- 172.21.15.67:0/2966995984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 0x7fc1a00d4f30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009850 tx=0x7fc18c02a250 comp rx=0 tx=0).stop 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.785+0000 7fc1a6000640 1 -- 172.21.15.67:0/2966995984 shutdown_connections 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.785+0000 7fc1a6000640 1 --2- 172.21.15.67:0/2966995984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 0x7fc1a00d4f30 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.785+0000 7fc1a6000640 1 --2- 172.21.15.67:0/2966995984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a01621d0 0x7fc1a00d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.785+0000 7fc1a6000640 1 -- 172.21.15.67:0/2966995984 >> 172.21.15.67:0/2966995984 conn(0x7fc1a008bfc0 msgr2=0x7fc1a00d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:53.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.785+0000 7fc1a6000640 1 -- 172.21.15.67:0/2966995984 shutdown_connections 2024-05-30T16:54:53.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.785+0000 7fc1a6000640 1 -- 172.21.15.67:0/2966995984 wait complete. 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.787+0000 7fc1a6000640 1 Processor -- start 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.787+0000 7fc1a6000640 1 -- start start 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.788+0000 7fc1a6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a015ea10 0x7fc1a0152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.788+0000 7fc1a6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1a01621d0 0x7fc1a01f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.788+0000 7fc1a6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 0x7fc1a01f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.788+0000 7fc1a6000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a00d5470 con 0x7fc1a01625b0 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.788+0000 7fc1a6000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a00d1ef0 con 0x7fc1a01621d0 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.788+0000 7fc1a6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a0162a10 con 0x7fc1a015ea10 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.788+0000 7fc1a5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 0x7fc1a01f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.788+0000 7fc1a5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 0x7fc1a01f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49230/0 (socket says 172.21.15.67:49230) 2024-05-30T16:54:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.788+0000 7fc1a5600640 1 -- 172.21.15.67:0/2426723331 learned_addr learned my addr 172.21.15.67:0/2426723331 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc1a4c00640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a015ea10 0x7fc1a0152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc19fe00640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1a01621d0 0x7fc1a01f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc1a5600640 1 -- 172.21.15.67:0/2426723331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a015ea10 msgr2=0x7fc1a0152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc1a5600640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a015ea10 0x7fc1a0152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc1a5600640 1 -- 172.21.15.67:0/2426723331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1a01621d0 msgr2=0x7fc1a01f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc1a5600640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1a01621d0 0x7fc1a01f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc1a5600640 1 -- 172.21.15.67:0/2426723331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a00a0140 con 0x7fc1a01625b0 2024-05-30T16:54:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc1a5600640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 0x7fc1a01f81a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc19800c6b0 tx=0x7fc19800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc19cc00640 1 -- 172.21.15.67:0/2426723331 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198003b70 con 0x7fc1a01625b0 2024-05-30T16:54:53.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc19cc00640 1 -- 172.21.15.67:0/2426723331 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc198003d30 con 0x7fc1a01625b0 2024-05-30T16:54:53.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc19cc00640 1 -- 172.21.15.67:0/2426723331 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198005310 con 0x7fc1a01625b0 2024-05-30T16:54:53.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a00a0420 con 0x7fc1a01625b0 2024-05-30T16:54:53.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.789+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a00a09e0 con 0x7fc1a01625b0 2024-05-30T16:54:53.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.791+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a00689e0 con 0x7fc1a01625b0 2024-05-30T16:54:53.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.792+0000 7fc19cc00640 1 -- 172.21.15.67:0/2426723331 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fc1980054b0 con 0x7fc1a01625b0 2024-05-30T16:54:53.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.793+0000 7fc19cc00640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc18006d610 0x7fc18006fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.793+0000 7fc19cc00640 1 -- 172.21.15.67:0/2426723331 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4913+0+0 (secure 0 0 0) 0x7fc19808a400 con 0x7fc1a01625b0 2024-05-30T16:54:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.793+0000 7fc1a4c00640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc18006d610 0x7fc18006fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.794+0000 7fc1a4c00640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc18006d610 0x7fc18006fad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc194008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:53.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:53.797+0000 7fc19cc00640 1 -- 172.21.15.67:0/2426723331 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc19805a280 con 0x7fc1a01625b0 2024-05-30T16:54:54.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.001+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc1a00d4d10 con 0x7fc1a01625b0 2024-05-30T16:54:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.580+0000 7fc19cc00640 1 -- 172.21.15.67:0/2426723331 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7fc198059b80 con 0x7fc1a01625b0 2024-05-30T16:54:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-05-30T16:54:54.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.584+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc18006d610 msgr2=0x7fc18006fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:54.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.584+0000 7fc1a6000640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc18006d610 0x7fc18006fad0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc194008040 comp rx=0 tx=0).stop 2024-05-30T16:54:54.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.584+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 msgr2=0x7fc1a01f81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:54.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.584+0000 7fc1a6000640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 0x7fc1a01f81a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc19800c6b0 tx=0x7fc19800cb70 comp rx=0 tx=0).stop 2024-05-30T16:54:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.585+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 shutdown_connections 2024-05-30T16:54:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.585+0000 7fc1a6000640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc18006d610 0x7fc18006fad0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.585+0000 7fc1a6000640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a01625b0 0x7fc1a01f81a0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.585+0000 7fc1a6000640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1a01621d0 0x7fc1a01f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.585+0000 7fc1a6000640 1 --2- 172.21.15.67:0/2426723331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1a015ea10 0x7fc1a0152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.585+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 >> 172.21.15.67:0/2426723331 conn(0x7fc1a008bfc0 msgr2=0x7fc1a015db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.585+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 shutdown_connections 2024-05-30T16:54:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.585+0000 7fc1a6000640 1 -- 172.21.15.67:0/2426723331 wait complete. 2024-05-30T16:54:54.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-30T16:54:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.825+0000 7f42bae00640 1 Processor -- start 2024-05-30T16:54:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.825+0000 7f42bae00640 1 -- start start 2024-05-30T16:54:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.827+0000 7f42bae00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42b4153510 0x7f42b40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.827+0000 7f42bae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b415d270 0x7f42b40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.827+0000 7f42bae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42b415d900 con 0x7f42b40ccb90 2024-05-30T16:54:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.827+0000 7f42bae00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f42b415d6e0 con 0x7f42b4153510 2024-05-30T16:54:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.827+0000 7f42bae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42b41538f0 con 0x7f42b415d270 2024-05-30T16:54:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.827+0000 7f42b9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42b4153510 0x7f42b40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.827+0000 7f42b9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42b4153510 0x7f42b40b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49244/0 (socket says 172.21.15.67:49244) 2024-05-30T16:54:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.827+0000 7f42b9a00640 1 -- 172.21.15.67:0/4205324083 learned_addr learned my addr 172.21.15.67:0/4205324083 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.828+0000 7f42b9a00640 1 -- 172.21.15.67:0/4205324083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b415d270 msgr2=0x7f42b40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.828+0000 7f42b9a00640 1 --2- 172.21.15.67:0/4205324083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b415d270 0x7f42b40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.828+0000 7f42b9a00640 1 -- 172.21.15.67:0/4205324083 >> v1:172.21.15.169:6789/0 conn(0x7f42b40ccb90 legacy=0x7f42b415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.828+0000 7f42b9a00640 1 -- 172.21.15.67:0/4205324083 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b40b3f90 con 0x7f42b4153510 2024-05-30T16:54:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.829+0000 7f42b9a00640 1 --2- 172.21.15.67:0/4205324083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b4153510 0x7f42b40b4d50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009e50 tx=0x7f42a802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200b74681fe1823b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:54.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.829+0000 7f42b3e00640 1 -- 172.21.15.67:0/4205324083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8036070 con 0x7f42b4153510 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.829+0000 7f42b3e00640 1 -- 172.21.15.67:0/4205324083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a8031600 con 0x7f42b4153510 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.829+0000 7f42b3e00640 1 -- 172.21.15.67:0/4205324083 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a80319d0 con 0x7f42b4153510 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.830+0000 7f42bae00640 1 -- 172.21.15.67:0/4205324083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b4153510 msgr2=0x7f42b40b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.830+0000 7f42bae00640 1 --2- 172.21.15.67:0/4205324083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b4153510 0x7f42b40b4d50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009e50 tx=0x7f42a802a640 comp rx=0 tx=0).stop 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.830+0000 7f42bae00640 1 -- 172.21.15.67:0/4205324083 shutdown_connections 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.830+0000 7f42bae00640 1 --2- 172.21.15.67:0/4205324083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b415d270 0x7f42b40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.830+0000 7f42bae00640 1 --2- 172.21.15.67:0/4205324083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b4153510 0x7f42b40b4d50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.830+0000 7f42bae00640 1 -- 172.21.15.67:0/4205324083 >> 172.21.15.67:0/4205324083 conn(0x7f42b406b900 msgr2=0x7f42b406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.830+0000 7f42bae00640 1 -- 172.21.15.67:0/4205324083 shutdown_connections 2024-05-30T16:54:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.830+0000 7f42bae00640 1 -- 172.21.15.67:0/4205324083 wait complete. 2024-05-30T16:54:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.832+0000 7f42bae00640 1 Processor -- start 2024-05-30T16:54:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.832+0000 7f42bae00640 1 -- start start 2024-05-30T16:54:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.833+0000 7f42bae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b40ccb90 0x7f42b4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.833+0000 7f42bae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42b4153510 0x7f42b4073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.833+0000 7f42bae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b415d270 0x7f42b41f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.833+0000 7f42bae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f42b40b48c0 con 0x7f42b415d270 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.833+0000 7f42bae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42b40b4550 con 0x7f42b4153510 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.833+0000 7f42bae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42b406e940 con 0x7f42b40ccb90 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.833+0000 7f42ba400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b415d270 0x7f42b41f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.833+0000 7f42ba400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b415d270 0x7f42b41f4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49256/0 (socket says 172.21.15.67:49256) 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.833+0000 7f42ba400640 1 -- 172.21.15.67:0/720841161 learned_addr learned my addr 172.21.15.67:0/720841161 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42b9a00640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b40ccb90 0x7f42b4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42b9000640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42b4153510 0x7f42b4073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42ba400640 1 -- 172.21.15.67:0/720841161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b40ccb90 msgr2=0x7f42b4071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42ba400640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b40ccb90 0x7f42b4071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42ba400640 1 -- 172.21.15.67:0/720841161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42b4153510 msgr2=0x7f42b4073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42ba400640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42b4153510 0x7f42b4073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42ba400640 1 -- 172.21.15.67:0/720841161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b4072160 con 0x7f42b415d270 2024-05-30T16:54:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42ba400640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b415d270 0x7f42b41f4770 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00b410 tx=0x7f42ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42b1600640 1 -- 172.21.15.67:0/720841161 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac015020 con 0x7f42b415d270 2024-05-30T16:54:54.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42b1600640 1 -- 172.21.15.67:0/720841161 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ac011850 con 0x7f42b415d270 2024-05-30T16:54:54.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.834+0000 7f42b1600640 1 -- 172.21.15.67:0/720841161 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac011c40 con 0x7f42b415d270 2024-05-30T16:54:54.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.835+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b4072440 con 0x7f42b415d270 2024-05-30T16:54:54.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.835+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b4072a00 con 0x7f42b415d270 2024-05-30T16:54:54.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.836+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b40689e0 con 0x7f42b415d270 2024-05-30T16:54:54.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.837+0000 7f42b1600640 1 -- 172.21.15.67:0/720841161 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f42ac018020 con 0x7f42b415d270 2024-05-30T16:54:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.838+0000 7f42b1600640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f429406d5e0 0x7f429406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.838+0000 7f42b1600640 1 -- 172.21.15.67:0/720841161 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5283+0+0 (secure 0 0 0) 0x7f42ac08ab90 con 0x7f42b415d270 2024-05-30T16:54:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.838+0000 7f42b9a00640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f429406d5e0 0x7f429406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.839+0000 7f42b9a00640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f429406d5e0 0x7f429406faa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f42a8033000 tx=0x7f42a8009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:54.842+0000 7f42b1600640 1 -- 172.21.15.67:0/720841161 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42ac01f020 con 0x7f42b415d270 2024-05-30T16:54:55.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.048+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f42b40cab00 con 0x7f42b415d270 2024-05-30T16:54:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.587+0000 7f42b1600640 1 -- 172.21.15.67:0/720841161 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7f42ac00f2e0 con 0x7f42b415d270 2024-05-30T16:54:55.589 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-05-30T16:54:55.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f429406d5e0 msgr2=0x7f429406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:55.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f429406d5e0 0x7f429406faa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f42a8033000 tx=0x7f42a8009000 comp rx=0 tx=0).stop 2024-05-30T16:54:55.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b415d270 msgr2=0x7f42b41f4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:55.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b415d270 0x7f42b41f4770 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00b410 tx=0x7f42ac00b8d0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 shutdown_connections 2024-05-30T16:54:55.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f429406d5e0 0x7f429406faa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42b415d270 0x7f42b41f4770 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42b4153510 0x7f42b4073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 --2- 172.21.15.67:0/720841161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b40ccb90 0x7f42b4071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.592+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 >> 172.21.15.67:0/720841161 conn(0x7f42b406b900 msgr2=0x7f42b406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:55.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.594+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 shutdown_connections 2024-05-30T16:54:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.594+0000 7f42bae00640 1 -- 172.21.15.67:0/720841161 wait complete. 2024-05-30T16:54:55.628 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-30T16:54:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.809+0000 7fed62000640 1 Processor -- start 2024-05-30T16:54:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.809+0000 7fed62000640 1 -- start start 2024-05-30T16:54:55.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.811+0000 7fed62000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fed5c0cd2a0 0x7fed5c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.811+0000 7fed62000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fed5c15cb60 0x7fed5c15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.811+0000 7fed62000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed5c15d900 con 0x7fed5c153510 2024-05-30T16:54:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.811+0000 7fed62000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fed5c1538f0 con 0x7fed5c0cd2a0 2024-05-30T16:54:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.811+0000 7fed62000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fed5c0b48c0 con 0x7fed5c15cb60 2024-05-30T16:54:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.811+0000 7fed53e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fed5c0cd2a0 0x7fed5c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.811+0000 7fed53e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fed5c0cd2a0 0x7fed5c0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49264/0 (socket says 172.21.15.67:49264) 2024-05-30T16:54:55.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.811+0000 7fed53e00640 1 -- 172.21.15.67:0/2487993513 learned_addr learned my addr 172.21.15.67:0/2487993513 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.812+0000 7fed53e00640 1 -- 172.21.15.67:0/2487993513 >> v2:172.21.15.169:3300/0 conn(0x7fed5c15cb60 msgr2=0x7fed5c15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.812+0000 7fed53e00640 1 --2- 172.21.15.67:0/2487993513 >> v2:172.21.15.169:3300/0 conn(0x7fed5c15cb60 0x7fed5c15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.812+0000 7fed53e00640 1 -- 172.21.15.67:0/2487993513 >> v1:172.21.15.67:6789/0 conn(0x7fed5c153510 legacy=0x7fed5c0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.812+0000 7fed53e00640 1 -- 172.21.15.67:0/2487993513 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c0b3f90 con 0x7fed5c0cd2a0 2024-05-30T16:54:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.813+0000 7fed53e00640 1 --2- 172.21.15.67:0/2487993513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c0cd2a0 0x7fed5c0b4d50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0b4a40 tx=0x7fed4402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a304d2b7973b83f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:55.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.813+0000 7fed5be00640 1 -- 172.21.15.67:0/2487993513 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44037070 con 0x7fed5c0cd2a0 2024-05-30T16:54:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.813+0000 7fed5be00640 1 -- 172.21.15.67:0/2487993513 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed44032440 con 0x7fed5c0cd2a0 2024-05-30T16:54:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.813+0000 7fed5be00640 1 -- 172.21.15.67:0/2487993513 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44032810 con 0x7fed5c0cd2a0 2024-05-30T16:54:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.814+0000 7fed62000640 1 -- 172.21.15.67:0/2487993513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c0cd2a0 msgr2=0x7fed5c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:55.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.814+0000 7fed62000640 1 --2- 172.21.15.67:0/2487993513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c0cd2a0 0x7fed5c0b4d50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0b4a40 tx=0x7fed4402a1d0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.814+0000 7fed62000640 1 -- 172.21.15.67:0/2487993513 shutdown_connections 2024-05-30T16:54:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.814+0000 7fed62000640 1 --2- 172.21.15.67:0/2487993513 >> v2:172.21.15.169:3300/0 conn(0x7fed5c15cb60 0x7fed5c15cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.814+0000 7fed62000640 1 --2- 172.21.15.67:0/2487993513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c0cd2a0 0x7fed5c0b4d50 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.814+0000 7fed62000640 1 -- 172.21.15.67:0/2487993513 >> 172.21.15.67:0/2487993513 conn(0x7fed5c06b900 msgr2=0x7fed5c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.814+0000 7fed62000640 1 -- 172.21.15.67:0/2487993513 shutdown_connections 2024-05-30T16:54:55.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.814+0000 7fed62000640 1 -- 172.21.15.67:0/2487993513 wait complete. 2024-05-30T16:54:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.816+0000 7fed62000640 1 Processor -- start 2024-05-30T16:54:55.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.816+0000 7fed62000640 1 -- start start 2024-05-30T16:54:55.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed62000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5c0cd2a0 0x7fed5c1f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed62000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c153510 0x7fed5c14a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed62000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5c15cb60 0x7fed5c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed62000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c15d4b0 con 0x7fed5c153510 2024-05-30T16:54:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed62000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c0b4550 con 0x7fed5c15cb60 2024-05-30T16:54:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed62000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed5c15d630 con 0x7fed5c0cd2a0 2024-05-30T16:54:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed53e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c153510 0x7fed5c14a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed53e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c153510 0x7fed5c14a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49268/0 (socket says 172.21.15.67:49268) 2024-05-30T16:54:55.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed53e00640 1 -- 172.21.15.67:0/925657965 learned_addr learned my addr 172.21.15.67:0/925657965 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.817+0000 7fed60c00640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5c0cd2a0 0x7fed5c1f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:55.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed53e00640 1 -- 172.21.15.67:0/925657965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5c0cd2a0 msgr2=0x7fed5c1f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed61600640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5c15cb60 0x7fed5c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed53e00640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5c0cd2a0 0x7fed5c1f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed53e00640 1 -- 172.21.15.67:0/925657965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5c15cb60 msgr2=0x7fed5c15b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed53e00640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5c15cb60 0x7fed5c15b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed53e00640 1 -- 172.21.15.67:0/925657965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c1f8120 con 0x7fed5c153510 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed60c00640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5c0cd2a0 0x7fed5c1f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed53e00640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c153510 0x7fed5c14a590 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fed44000c00 tx=0x7fed440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed59600640 1 -- 172.21.15.67:0/925657965 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44041020 con 0x7fed5c153510 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed59600640 1 -- 172.21.15.67:0/925657965 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed44030970 con 0x7fed5c153510 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed59600640 1 -- 172.21.15.67:0/925657965 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44037040 con 0x7fed5c153510 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.818+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c1f8400 con 0x7fed5c153510 2024-05-30T16:54:55.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.819+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c154f30 con 0x7fed5c153510 2024-05-30T16:54:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.820+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5c0689e0 con 0x7fed5c153510 2024-05-30T16:54:55.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.821+0000 7fed59600640 1 -- 172.21.15.67:0/925657965 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fed44044020 con 0x7fed5c153510 2024-05-30T16:54:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.823+0000 7fed59600640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fed3806d5e0 0x7fed3806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:55.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.823+0000 7fed59600640 1 -- 172.21.15.67:0/925657965 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5665+0+0 (secure 0 0 0) 0x7fed440ab700 con 0x7fed5c153510 2024-05-30T16:54:55.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.823+0000 7fed60c00640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fed3806d5e0 0x7fed3806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:55.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.824+0000 7fed60c00640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fed3806d5e0 0x7fed3806faa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed4c001680 tx=0x7fed4c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:55.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:55.828+0000 7fed59600640 1 -- 172.21.15.67:0/925657965 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed4404b030 con 0x7fed5c153510 2024-05-30T16:54:56.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.077+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fed5c0b3c40 con 0x7fed5c153510 2024-05-30T16:54:56.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.634+0000 7fed59600640 1 -- 172.21.15.67:0/925657965 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v41) ==== 174+0+0 (secure 0 0 0) 0x7fed4407b330 con 0x7fed5c153510 2024-05-30T16:54:56.640 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 10 and data pool 11 2024-05-30T16:54:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.647+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fed3806d5e0 msgr2=0x7fed3806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.647+0000 7fed62000640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fed3806d5e0 0x7fed3806faa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed4c001680 tx=0x7fed4c009040 comp rx=0 tx=0).stop 2024-05-30T16:54:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.647+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c153510 msgr2=0x7fed5c14a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.647+0000 7fed62000640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c153510 0x7fed5c14a590 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fed44000c00 tx=0x7fed440046c0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.650+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 shutdown_connections 2024-05-30T16:54:56.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.650+0000 7fed62000640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fed3806d5e0 0x7fed3806faa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.650+0000 7fed62000640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5c15cb60 0x7fed5c15b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.650+0000 7fed62000640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed5c153510 0x7fed5c14a590 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.650+0000 7fed62000640 1 --2- 172.21.15.67:0/925657965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5c0cd2a0 0x7fed5c1f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.650+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 >> 172.21.15.67:0/925657965 conn(0x7fed5c06b900 msgr2=0x7fed5c06e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.651+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 shutdown_connections 2024-05-30T16:54:56.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.651+0000 7fed62000640 1 -- 172.21.15.67:0/925657965 wait complete. 2024-05-30T16:54:56.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:54:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.850+0000 7fc877200640 1 Processor -- start 2024-05-30T16:54:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.850+0000 7fc877200640 1 -- start start 2024-05-30T16:54:56.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.852+0000 7fc877200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc8780d2610 0x7fc87815f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:56.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.852+0000 7fc877200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8780d2bd0 con 0x7fc8780b5f90 2024-05-30T16:54:56.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.852+0000 7fc877200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8780d2eb0 con 0x7fc8780cebf0 2024-05-30T16:54:56.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.852+0000 7fc877200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc8780d37e0 con 0x7fc8780d2610 2024-05-30T16:54:56.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.852+0000 7fc875e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc8780b5f90 0x7fc8780b6370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34578/0 (socket says 172.21.15.67:34578) 2024-05-30T16:54:56.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.852+0000 7fc875e00640 1 -- 172.21.15.67:0/2760735271 learned_addr learned my addr 172.21.15.67:0/2760735271 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:56.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.852+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1567481762 0 0) 0x7fc8780d2bd0 con 0x7fc8780b5f90 2024-05-30T16:54:56.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.852+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc85c003520 con 0x7fc8780b5f90 2024-05-30T16:54:56.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc86d400640 1 --2- 172.21.15.67:0/2760735271 >> v2:172.21.15.169:3300/0 conn(0x7fc8780d2610 0x7fc87815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:56.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3101832595 0 0) 0x7fc85c003520 con 0x7fc8780b5f90 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 >> v2:172.21.15.169:3300/0 conn(0x7fc8780d2610 msgr2=0x7fc87815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 --2- 172.21.15.67:0/2760735271 >> v2:172.21.15.169:3300/0 conn(0x7fc8780d2610 0x7fc87815f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 >> v1:172.21.15.169:6789/0 conn(0x7fc8780cebf0 legacy=0x7fc8780cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8780d3190 con 0x7fc8780b5f90 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fc868001670 con 0x7fc8780b5f90 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 >> v1:172.21.15.67:6789/0 conn(0x7fc8780b5f90 legacy=0x7fc8780b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc85c0039b0 0x7fc85c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc85c004800 0x7fc85c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc85c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc8780d37e0 con 0x7fc85c0052e0 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c00c190 con 0x7fc85c004800 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc85c00c390 con 0x7fc85c0039b0 2024-05-30T16:54:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.853+0000 7fc876800640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc85c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.854+0000 7fc875e00640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc85c0039b0 0x7fc85c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.854+0000 7fc86d400640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc85c004800 0x7fc85c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.854+0000 7fc876800640 1 -- 172.21.15.67:0/2760735271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc85c0039b0 msgr2=0x7fc85c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.854+0000 7fc876800640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc85c0039b0 0x7fc85c003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.854+0000 7fc876800640 1 -- 172.21.15.67:0/2760735271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc85c004800 msgr2=0x7fc85c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.854+0000 7fc876800640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc85c004800 0x7fc85c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.854+0000 7fc876800640 1 -- 172.21.15.67:0/2760735271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c003520 con 0x7fc85c0052e0 2024-05-30T16:54:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.854+0000 7fc876800640 1 -- 172.21.15.67:0/2760735271 reap_dead start 2024-05-30T16:54:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.855+0000 7fc876800640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc85c00b9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc870001d20 tx=0x7fc85c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230e2ea4e3f8504e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.855+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870001920 con 0x7fc85c0052e0 2024-05-30T16:54:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.855+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc87002e070 con 0x7fc85c0052e0 2024-05-30T16:54:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.855+0000 7fc875400640 1 -- 172.21.15.67:0/2760735271 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870003970 con 0x7fc85c0052e0 2024-05-30T16:54:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.855+0000 7fc877200640 1 -- 172.21.15.67:0/2760735271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 msgr2=0x7fc85c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.855+0000 7fc877200640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc85c00b9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc870001d20 tx=0x7fc85c005830 comp rx=0 tx=0).stop 2024-05-30T16:54:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.855+0000 7fc877200640 1 -- 172.21.15.67:0/2760735271 shutdown_connections 2024-05-30T16:54:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.855+0000 7fc877200640 1 --2- 172.21.15.67:0/2760735271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc85c00b9c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.855+0000 7fc877200640 1 -- 172.21.15.67:0/2760735271 >> 172.21.15.67:0/2760735271 conn(0x7fc87808bfc0 msgr2=0x7fc8780d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.856+0000 7fc877200640 1 -- 172.21.15.67:0/2760735271 shutdown_connections 2024-05-30T16:54:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.856+0000 7fc877200640 1 -- 172.21.15.67:0/2760735271 wait complete. 2024-05-30T16:54:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.857+0000 7fc877200640 1 Processor -- start 2024-05-30T16:54:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.858+0000 7fc877200640 1 -- start start 2024-05-30T16:54:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.858+0000 7fc877200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc87814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc877200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8781f78d0 0x7fc87814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc877200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8780b7360 0x7fc8780b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc877200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc87815eac0 con 0x7fc85c0052e0 2024-05-30T16:54:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc877200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8780d4ee0 con 0x7fc8781f78d0 2024-05-30T16:54:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc877200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc87815f790 con 0x7fc8780b7360 2024-05-30T16:54:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc875e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc87814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc875e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc87814eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49284/0 (socket says 172.21.15.67:49284) 2024-05-30T16:54:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc875e00640 1 -- 172.21.15.67:0/3066470785 learned_addr learned my addr 172.21.15.67:0/3066470785 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc875e00640 1 -- 172.21.15.67:0/3066470785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8780b7360 msgr2=0x7fc8780b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc86d400640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8781f78d0 0x7fc87814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc875e00640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8780b7360 0x7fc8780b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc875e00640 1 -- 172.21.15.67:0/3066470785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8781f78d0 msgr2=0x7fc87814bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc875e00640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8781f78d0 0x7fc87814bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.859+0000 7fc875e00640 1 -- 172.21.15.67:0/3066470785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc878152280 con 0x7fc85c0052e0 2024-05-30T16:54:56.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.860+0000 7fc875e00640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc87814eea0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc86800d800 tx=0x7fc86800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:56.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.860+0000 7fc86ea00640 1 -- 172.21.15.67:0/3066470785 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868019070 con 0x7fc85c0052e0 2024-05-30T16:54:56.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.860+0000 7fc86ea00640 1 -- 172.21.15.67:0/3066470785 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8680074e0 con 0x7fc85c0052e0 2024-05-30T16:54:56.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.860+0000 7fc86ea00640 1 -- 172.21.15.67:0/3066470785 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868004a20 con 0x7fc85c0052e0 2024-05-30T16:54:56.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.860+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87814f3e0 con 0x7fc85c0052e0 2024-05-30T16:54:56.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.860+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8781f8070 con 0x7fc85c0052e0 2024-05-30T16:54:56.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.861+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8780689e0 con 0x7fc85c0052e0 2024-05-30T16:54:56.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.862+0000 7fc86ea00640 1 -- 172.21.15.67:0/3066470785 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fc86801c070 con 0x7fc85c0052e0 2024-05-30T16:54:56.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.863+0000 7fc86ea00640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc85406d5e0 0x7fc85406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.863+0000 7fc86ea00640 1 -- 172.21.15.67:0/3066470785 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5765+0+0 (secure 0 0 0) 0x7fc86808a050 con 0x7fc85c0052e0 2024-05-30T16:54:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.863+0000 7fc86d400640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc85406d5e0 0x7fc85406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.864+0000 7fc86d400640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc85406d5e0 0x7fc85406faa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc8780cba90 tx=0x7fc860008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:56.867+0000 7fc86ea00640 1 -- 172.21.15.67:0/3066470785 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc87814f3e0 con 0x7fc85c0052e0 2024-05-30T16:54:57.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.064+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8780b3970 con 0x7fc85c0052e0 2024-05-30T16:54:57.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.066+0000 7fc86ea00640 1 -- 172.21.15.67:0/3066470785 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16756 (secure 0 0 0) 0x7fc878152280 con 0x7fc85c0052e0 2024-05-30T16:54:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:57.068 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":73,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:54:56.078905+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:54:54.017222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-30T16:54:55.063707+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:54:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.069+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc85406d5e0 msgr2=0x7fc85406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.069+0000 7fc877200640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc85406d5e0 0x7fc85406faa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc8780cba90 tx=0x7fc860008040 comp rx=0 tx=0).stop 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.069+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 msgr2=0x7fc87814eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.069+0000 7fc877200640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc87814eea0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc86800d800 tx=0x7fc86800dcc0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.070+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 shutdown_connections 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.070+0000 7fc877200640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc85406d5e0 0x7fc85406faa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.070+0000 7fc877200640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8780b7360 0x7fc8780b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.070+0000 7fc877200640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8781f78d0 0x7fc87814bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.070+0000 7fc877200640 1 --2- 172.21.15.67:0/3066470785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc85c0052e0 0x7fc87814eea0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.070+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 >> 172.21.15.67:0/3066470785 conn(0x7fc87808bfc0 msgr2=0x7fc8780b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.070+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 shutdown_connections 2024-05-30T16:54:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.070+0000 7fc877200640 1 -- 172.21.15.67:0/3066470785 wait complete. 2024-05-30T16:54:57.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:54:57.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.294+0000 7ff8ebe00640 1 Processor -- start 2024-05-30T16:54:57.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.294+0000 7ff8ebe00640 1 -- start start 2024-05-30T16:54:57.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.296+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40d0930 0x7ff8e40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.296+0000 7ff8ebe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8e406ed10 con 0x7ff8e406f860 2024-05-30T16:54:57.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.296+0000 7ff8ebe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8e415cb00 con 0x7ff8e40d04a0 2024-05-30T16:54:57.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.296+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e40b3cc0 con 0x7ff8e40d0930 2024-05-30T16:54:57.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.296+0000 7ff8e9400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff8e406f860 0x7ff8e40b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34580/0 (socket says 172.21.15.67:34580) 2024-05-30T16:54:57.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.296+0000 7ff8e9400640 1 -- 172.21.15.67:0/529513592 learned_addr learned my addr 172.21.15.67:0/529513592 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.296+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597029716 0 0) 0x7ff8d8001110 con 0x7ff8e406f860 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8cc003520 con 0x7ff8e406f860 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e9e00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40d0930 0x7ff8e40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2761802587 0 0) 0x7ff8cc003520 con 0x7ff8e406f860 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40d0930 msgr2=0x7ff8e40ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40d0930 0x7ff8e40ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 >> v1:172.21.15.169:6789/0 conn(0x7ff8e40d04a0 legacy=0x7ff8e40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e40b3fa0 con 0x7ff8e406f860 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff8d8001840 con 0x7ff8e406f860 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 >> v1:172.21.15.67:6789/0 conn(0x7ff8e406f860 legacy=0x7ff8e40b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8cc0039e0 0x7ff8cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40d0930 0x7ff8cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e40b3cc0 con 0x7ff8cc004d40 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc00bb50 con 0x7ff8cc0039e0 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc00bd50 con 0x7ff8e40d0930 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.297+0000 7ff8e9e00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.298+0000 7ff8e9400640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8cc0039e0 0x7ff8cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.298+0000 7ff8e9e00640 1 -- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40d0930 msgr2=0x7ff8cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.298+0000 7ff8e8a00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40d0930 0x7ff8cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.298+0000 7ff8e9e00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40d0930 0x7ff8cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.298+0000 7ff8e9e00640 1 -- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8cc0039e0 msgr2=0x7ff8cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.298+0000 7ff8e9e00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8cc0039e0 0x7ff8cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.298+0000 7ff8e9e00640 1 -- 172.21.15.67:0/529513592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc003520 con 0x7ff8cc004d40 2024-05-30T16:54:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.298+0000 7ff8e8a00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40d0930 0x7ff8cc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:57.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.299+0000 7ff8e9e00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8cc0051c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc007360 tx=0x7ff8cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b5d38e9bd2723b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.299+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc009be0 con 0x7ff8cc004d40 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.299+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8dc036600 con 0x7ff8cc004d40 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.299+0000 7ff8e3e00640 1 -- 172.21.15.67:0/529513592 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0369d0 con 0x7ff8cc004d40 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.300+0000 7ff8ebe00640 1 -- 172.21.15.67:0/529513592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 msgr2=0x7ff8cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.300+0000 7ff8ebe00640 1 --2- 172.21.15.67:0/529513592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8cc0051c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc007360 tx=0x7ff8cc005830 comp rx=0 tx=0).stop 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.300+0000 7ff8e9e00640 1 -- 172.21.15.67:0/529513592 reap_dead start 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.300+0000 7ff8ebe00640 1 -- 172.21.15.67:0/529513592 shutdown_connections 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.300+0000 7ff8ebe00640 1 -- 172.21.15.67:0/529513592 >> 172.21.15.67:0/529513592 conn(0x7ff8e406bae0 msgr2=0x7ff8e406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.300+0000 7ff8ebe00640 1 -- 172.21.15.67:0/529513592 shutdown_connections 2024-05-30T16:54:57.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.300+0000 7ff8ebe00640 1 -- 172.21.15.67:0/529513592 wait complete. 2024-05-30T16:54:57.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.302+0000 7ff8ebe00640 1 Processor -- start 2024-05-30T16:54:57.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.302+0000 7ff8ebe00640 1 -- start start 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.303+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8e41f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.303+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40c55d0 0x7ff8e40c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.303+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e415ce30 0x7ff8e415d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.303+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e40b4840 con 0x7ff8cc004d40 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.303+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e40b4560 con 0x7ff8e415ce30 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.303+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e40d0d80 con 0x7ff8e40c55d0 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.303+0000 7ff8e9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8e41f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.303+0000 7ff8e9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8e41f4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49300/0 (socket says 172.21.15.67:49300) 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.303+0000 7ff8e9400640 1 -- 172.21.15.67:0/3819511679 learned_addr learned my addr 172.21.15.67:0/3819511679 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e9e00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e415ce30 0x7ff8e415d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e8a00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40c55d0 0x7ff8e40c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e9400640 1 -- 172.21.15.67:0/3819511679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40c55d0 msgr2=0x7ff8e40c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e9400640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40c55d0 0x7ff8e40c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e9400640 1 -- 172.21.15.67:0/3819511679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e415ce30 msgr2=0x7ff8e415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e9400640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e415ce30 0x7ff8e415d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e9400640 1 -- 172.21.15.67:0/3819511679 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e415dbd0 con 0x7ff8cc004d40 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e9400640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8e41f4030 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800e780 tx=0x7ff8d800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e1600640 1 -- 172.21.15.67:0/3819511679 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800c660 con 0x7ff8cc004d40 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e1600640 1 -- 172.21.15.67:0/3819511679 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d8018070 con 0x7ff8cc004d40 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.304+0000 7ff8e1600640 1 -- 172.21.15.67:0/3819511679 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800ca40 con 0x7ff8cc004d40 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.305+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e415deb0 con 0x7ff8cc004d40 2024-05-30T16:54:57.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.305+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e415e470 con 0x7ff8cc004d40 2024-05-30T16:54:57.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.306+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e40689e0 con 0x7ff8cc004d40 2024-05-30T16:54:57.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.307+0000 7ff8e1600640 1 -- 172.21.15.67:0/3819511679 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7ff8d800cbe0 con 0x7ff8cc004d40 2024-05-30T16:54:57.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.308+0000 7ff8e1600640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff8c406d5e0 0x7ff8c406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.308+0000 7ff8e1600640 1 -- 172.21.15.67:0/3819511679 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5765+0+0 (secure 0 0 0) 0x7ff8d808a6f0 con 0x7ff8cc004d40 2024-05-30T16:54:57.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.308+0000 7ff8e8a00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff8c406d5e0 0x7ff8c406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.309+0000 7ff8e8a00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff8c406d5e0 0x7ff8c406faa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40c33f0 tx=0x7ff8d0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:57.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.312+0000 7ff8e1600640 1 -- 172.21.15.67:0/3819511679 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8e406ed10 con 0x7ff8cc004d40 2024-05-30T16:54:57.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.508+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8e4072e80 con 0x7ff8cc004d40 2024-05-30T16:54:57.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.511+0000 7ff8e1600640 1 -- 172.21.15.67:0/3819511679 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16756 (secure 0 0 0) 0x7ff8d805a2e0 con 0x7ff8cc004d40 2024-05-30T16:54:57.512 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:54:57.512 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":73,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:54:56.078905+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:54:54.017222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-30T16:54:55.063707+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:54:57.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff8c406d5e0 msgr2=0x7ff8c406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff8c406d5e0 0x7ff8c406faa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40c33f0 tx=0x7ff8d0009040 comp rx=0 tx=0).stop 2024-05-30T16:54:57.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 msgr2=0x7ff8e41f4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8e41f4030 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800e780 tx=0x7ff8d800ec40 comp rx=0 tx=0).stop 2024-05-30T16:54:57.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 shutdown_connections 2024-05-30T16:54:57.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff8c406d5e0 0x7ff8c406faa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e415ce30 0x7ff8e415d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8e40c55d0 0x7ff8e40c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 --2- 172.21.15.67:0/3819511679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8cc004d40 0x7ff8e41f4030 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 >> 172.21.15.67:0/3819511679 conn(0x7ff8e406bae0 msgr2=0x7ff8e4072260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:57.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.513+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 shutdown_connections 2024-05-30T16:54:57.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.514+0000 7ff8ebe00640 1 -- 172.21.15.67:0/3819511679 wait complete. 2024-05-30T16:54:57.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-30T16:54:57.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.738+0000 7f43da800640 1 Processor -- start 2024-05-30T16:54:57.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.738+0000 7f43da800640 1 -- start start 2024-05-30T16:54:57.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.740+0000 7f43da800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0640 0x7f43d415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.740+0000 7f43da800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f43d40b32c0 0x7f43d40b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.740+0000 7f43da800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d40d0060 con 0x7f43d40d0a20 2024-05-30T16:54:57.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.740+0000 7f43da800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f43d40adbb0 con 0x7f43d40b32c0 2024-05-30T16:54:57.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.740+0000 7f43da800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f43d415db10 con 0x7f43d40d0640 2024-05-30T16:54:57.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.740+0000 7f43d3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f43d40b32c0 0x7f43d40b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.740+0000 7f43d3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f43d40b32c0 0x7f43d40b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49308/0 (socket says 172.21.15.67:49308) 2024-05-30T16:54:57.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.741+0000 7f43d3e00640 1 -- 172.21.15.67:0/1544021934 learned_addr learned my addr 172.21.15.67:0/1544021934 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:57.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.741+0000 7f43d3400640 1 --2- 172.21.15.67:0/1544021934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0640 0x7f43d415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.742+0000 7f43d3e00640 1 -- 172.21.15.67:0/1544021934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0640 msgr2=0x7f43d415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:57.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.742+0000 7f43d3e00640 1 --2- 172.21.15.67:0/1544021934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0640 0x7f43d415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.742+0000 7f43d3e00640 1 -- 172.21.15.67:0/1544021934 >> v1:172.21.15.67:6789/0 conn(0x7f43d40d0a20 legacy=0x7f43d40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.742+0000 7f43d3e00640 1 -- 172.21.15.67:0/1544021934 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d415cb10 con 0x7f43d40b32c0 2024-05-30T16:54:57.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.744+0000 7f43d3e00640 1 --2- 172.21.15.67:0/1544021934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 0x7f43d40b3750 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009e50 tx=0x7f43cc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba44518e4d7e4af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:57.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.744+0000 7f43d2000640 1 -- 172.21.15.67:0/1544021934 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc036070 con 0x7f43d40b32c0 2024-05-30T16:54:57.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.744+0000 7f43d2000640 1 -- 172.21.15.67:0/1544021934 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc031600 con 0x7f43d40b32c0 2024-05-30T16:54:57.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.744+0000 7f43d2000640 1 -- 172.21.15.67:0/1544021934 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0319d0 con 0x7f43d40b32c0 2024-05-30T16:54:57.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.745+0000 7f43da800640 1 -- 172.21.15.67:0/1544021934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 msgr2=0x7f43d40b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.745+0000 7f43da800640 1 --2- 172.21.15.67:0/1544021934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 0x7f43d40b3750 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009e50 tx=0x7f43cc02a730 comp rx=0 tx=0).stop 2024-05-30T16:54:57.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.745+0000 7f43da800640 1 -- 172.21.15.67:0/1544021934 shutdown_connections 2024-05-30T16:54:57.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.745+0000 7f43da800640 1 --2- 172.21.15.67:0/1544021934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 0x7f43d40b3750 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.745+0000 7f43da800640 1 --2- 172.21.15.67:0/1544021934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0640 0x7f43d415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.745+0000 7f43da800640 1 -- 172.21.15.67:0/1544021934 >> 172.21.15.67:0/1544021934 conn(0x7f43d406bd20 msgr2=0x7f43d406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.745+0000 7f43da800640 1 -- 172.21.15.67:0/1544021934 shutdown_connections 2024-05-30T16:54:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.745+0000 7f43da800640 1 -- 172.21.15.67:0/1544021934 wait complete. 2024-05-30T16:54:57.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.747+0000 7f43da800640 1 Processor -- start 2024-05-30T16:54:57.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.747+0000 7f43da800640 1 -- start start 2024-05-30T16:54:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.748+0000 7f43da800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 0x7f43d409aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.748+0000 7f43da800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d40d0640 0x7f43d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.748+0000 7f43da800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0a20 0x7f43d415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.748+0000 7f43da800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f43d415cdd0 con 0x7f43d40b32c0 2024-05-30T16:54:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.748+0000 7f43da800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43d40ad8d0 con 0x7f43d40d0640 2024-05-30T16:54:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.748+0000 7f43da800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f43d415d0b0 con 0x7f43d40d0a20 2024-05-30T16:54:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 0x7f43d409aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 0x7f43d409aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49312/0 (socket says 172.21.15.67:49312) 2024-05-30T16:54:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d3400640 1 -- 172.21.15.67:0/3397723928 learned_addr learned my addr 172.21.15.67:0/3397723928 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d2a00640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d40d0640 0x7f43d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d3e00640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0a20 0x7f43d415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d3400640 1 -- 172.21.15.67:0/3397723928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0a20 msgr2=0x7f43d415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d3400640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0a20 0x7f43d415b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d3400640 1 -- 172.21.15.67:0/3397723928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d40d0640 msgr2=0x7f43d406d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d3400640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d40d0640 0x7f43d406d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d3400640 1 -- 172.21.15.67:0/3397723928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d409c580 con 0x7f43d40b32c0 2024-05-30T16:54:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.749+0000 7f43d2a00640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d40d0640 0x7f43d406d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.750+0000 7f43d3400640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 0x7f43d409aa80 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9b0 tx=0x7f43c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.750+0000 7f43cb400640 1 -- 172.21.15.67:0/3397723928 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4017070 con 0x7f43d40b32c0 2024-05-30T16:54:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.750+0000 7f43cb400640 1 -- 172.21.15.67:0/3397723928 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c400c9a0 con 0x7f43d40b32c0 2024-05-30T16:54:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.750+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d409c860 con 0x7f43d40b32c0 2024-05-30T16:54:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.750+0000 7f43cb400640 1 -- 172.21.15.67:0/3397723928 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c400cd90 con 0x7f43d40b32c0 2024-05-30T16:54:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.750+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d41f5270 con 0x7f43d40b32c0 2024-05-30T16:54:57.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.751+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d40689e0 con 0x7f43d40b32c0 2024-05-30T16:54:57.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.752+0000 7f43cb400640 1 -- 172.21.15.67:0/3397723928 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f43c401a070 con 0x7f43d40b32c0 2024-05-30T16:54:57.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.753+0000 7f43cb400640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43b406d5e0 0x7f43b406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:57.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.753+0000 7f43cb400640 1 -- 172.21.15.67:0/3397723928 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f43c408a9a0 con 0x7f43d40b32c0 2024-05-30T16:54:57.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.753+0000 7f43d2a00640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43b406d5e0 0x7f43b406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:57.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.754+0000 7f43d2a00640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43b406d5e0 0x7f43b406faa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f43d4155fb0 tx=0x7f43c0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:57.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.758+0000 7f43cb400640 1 -- 172.21.15.67:0/3397723928 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43c405a820 con 0x7f43d40b32c0 2024-05-30T16:54:57.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:57.994+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f43d415c1c0 con 0x7f43d40b32c0 2024-05-30T16:54:58.578 INFO:tasks.ceph.mds.a.smithi067.stderr:2024-05-30T16:54:58.578+0000 7f85f7600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T16:54:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.659+0000 7f43cb400640 1 -- 172.21.15.67:0/3397723928 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f43c405a120 con 0x7f43d40b32c0 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43b406d5e0 msgr2=0x7f43b406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43b406d5e0 0x7f43b406faa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f43d4155fb0 tx=0x7f43c0009040 comp rx=0 tx=0).stop 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 msgr2=0x7f43d409aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 0x7f43d409aa80 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9b0 tx=0x7f43c400ee70 comp rx=0 tx=0).stop 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 shutdown_connections 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f43b406d5e0 0x7f43b406faa0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d40d0a20 0x7f43d415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d40d0640 0x7f43d406d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 --2- 172.21.15.67:0/3397723928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d40b32c0 0x7f43d409aa80 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.663+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 >> 172.21.15.67:0/3397723928 conn(0x7f43d406bd20 msgr2=0x7f43d41505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.664+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 shutdown_connections 2024-05-30T16:54:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.664+0000 7f43da800640 1 -- 172.21.15.67:0/3397723928 wait complete. 2024-05-30T16:54:58.688 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-30T16:54:58.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-30T16:54:58.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.902+0000 7f1ccf200640 1 Processor -- start 2024-05-30T16:54:58.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.902+0000 7f1ccf200640 1 -- start start 2024-05-30T16:54:58.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.904+0000 7f1ccf200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1cd0153510 0x7f1cd00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:58.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.904+0000 7f1ccf200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd015d900 con 0x7f1cd00ccb90 2024-05-30T16:54:58.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.904+0000 7f1ccf200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd00b3f90 con 0x7f1cd015d270 2024-05-30T16:54:58.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.904+0000 7f1ccf200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd01538f0 con 0x7f1cd0153510 2024-05-30T16:54:58.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.904+0000 7f1ccd400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1cd00ccb90 0x7f1cd015cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34592/0 (socket says 172.21.15.67:34592) 2024-05-30T16:54:58.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.904+0000 7f1ccd400640 1 -- 172.21.15.67:0/1670821389 learned_addr learned my addr 172.21.15.67:0/1670821389 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccde00640 1 --2- 172.21.15.67:0/1670821389 >> v2:172.21.15.169:3300/0 conn(0x7f1cd0153510 0x7f1cd00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3684935857 0 0) 0x7f1cd015d900 con 0x7f1cd00ccb90 2024-05-30T16:54:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cb4003520 con 0x7f1cd00ccb90 2024-05-30T16:54:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3596829649 0 0) 0x7f1cb4003520 con 0x7f1cd00ccb90 2024-05-30T16:54:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 >> v2:172.21.15.169:3300/0 conn(0x7f1cd0153510 msgr2=0x7f1cd00b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 --2- 172.21.15.67:0/1670821389 >> v2:172.21.15.169:3300/0 conn(0x7f1cd0153510 0x7f1cd00b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 >> v1:172.21.15.169:6789/0 conn(0x7f1cd015d270 legacy=0x7f1cd006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd00b4270 con 0x7f1cd00ccb90 2024-05-30T16:54:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f1cb8001670 con 0x7f1cd00ccb90 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 >> v1:172.21.15.67:6789/0 conn(0x7f1cd00ccb90 legacy=0x7f1cd015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb40039b0 0x7f1cb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb40051c0 0x7f1cb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb400c130 con 0x7f1cb4004800 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb400c2e0 con 0x7f1cb40051c0 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.905+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb400c4e0 con 0x7f1cb40039b0 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1ccd400640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1ccde00640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb40039b0 0x7f1cb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1cce800640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb40051c0 0x7f1cb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1ccd400640 1 -- 172.21.15.67:0/1670821389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb40039b0 msgr2=0x7f1cb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:58.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1ccd400640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb40039b0 0x7f1cb4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1ccd400640 1 -- 172.21.15.67:0/1670821389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb40051c0 msgr2=0x7f1cb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1ccd400640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb40051c0 0x7f1cb400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1ccd400640 1 -- 172.21.15.67:0/1670821389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb4003520 con 0x7f1cb4004800 2024-05-30T16:54:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1cce800640 1 -- 172.21.15.67:0/1670821389 reap_dead start 2024-05-30T16:54:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.906+0000 7f1ccde00640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb40039b0 0x7f1cb4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:54:58.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.907+0000 7f1ccd400640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cb4004c80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c240 tx=0x7f1cb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec85ff16f04b189 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:58.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.908+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8007cf0 con 0x7f1cb4004800 2024-05-30T16:54:58.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.908+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb80058d0 con 0x7f1cb4004800 2024-05-30T16:54:58.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.908+0000 7f1ccca00640 1 -- 172.21.15.67:0/1670821389 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8005d20 con 0x7f1cb4004800 2024-05-30T16:54:58.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.908+0000 7f1ccf200640 1 -- 172.21.15.67:0/1670821389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 msgr2=0x7f1cb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:58.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.908+0000 7f1ccf200640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cb4004c80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800c240 tx=0x7f1cb4005830 comp rx=0 tx=0).stop 2024-05-30T16:54:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.909+0000 7f1ccf200640 1 -- 172.21.15.67:0/1670821389 shutdown_connections 2024-05-30T16:54:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.909+0000 7f1ccf200640 1 --2- 172.21.15.67:0/1670821389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cb4004c80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.909+0000 7f1ccf200640 1 -- 172.21.15.67:0/1670821389 >> 172.21.15.67:0/1670821389 conn(0x7f1cd006b900 msgr2=0x7f1cd006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.909+0000 7f1ccf200640 1 -- 172.21.15.67:0/1670821389 shutdown_connections 2024-05-30T16:54:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.909+0000 7f1ccf200640 1 -- 172.21.15.67:0/1670821389 wait complete. 2024-05-30T16:54:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.911+0000 7f1ccf200640 1 Processor -- start 2024-05-30T16:54:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.911+0000 7f1ccf200640 1 -- start start 2024-05-30T16:54:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.912+0000 7f1ccf200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cd014fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.912+0000 7f1ccf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cd0150300 0x7f1cd01e6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.912+0000 7f1ccf200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd00736a0 0x7f1cd00b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.912+0000 7f1ccf200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd015d6a0 con 0x7f1cb4004800 2024-05-30T16:54:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.912+0000 7f1ccf200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd00b48c0 con 0x7f1cd00736a0 2024-05-30T16:54:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.912+0000 7f1ccf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd00b4d50 con 0x7f1cd0150300 2024-05-30T16:54:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.912+0000 7f1ccde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cd014fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.912+0000 7f1ccde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cd014fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49330/0 (socket says 172.21.15.67:49330) 2024-05-30T16:54:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.912+0000 7f1ccde00640 1 -- 172.21.15.67:0/1926985797 learned_addr learned my addr 172.21.15.67:0/1926985797 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:58.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1cce800640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd00736a0 0x7f1cd00b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:58.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1ccd400640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cd0150300 0x7f1cd01e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:58.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1ccde00640 1 -- 172.21.15.67:0/1926985797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cd0150300 msgr2=0x7f1cd01e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:58.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1ccde00640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cd0150300 0x7f1cd01e6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1ccde00640 1 -- 172.21.15.67:0/1926985797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd00736a0 msgr2=0x7f1cd00b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:58.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1ccde00640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd00736a0 0x7f1cd00b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:58.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1ccde00640 1 -- 172.21.15.67:0/1926985797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd01e6b10 con 0x7f1cb4004800 2024-05-30T16:54:58.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1ccde00640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cd014fac0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00107f0 tx=0x7f1cc0010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:58.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1cc6000640 1 -- 172.21.15.67:0/1926985797 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000f040 con 0x7f1cb4004800 2024-05-30T16:54:58.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1cc6000640 1 -- 172.21.15.67:0/1926985797 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc000eb00 con 0x7f1cb4004800 2024-05-30T16:54:58.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd0072bc0 con 0x7f1cb4004800 2024-05-30T16:54:58.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.913+0000 7f1cc6000640 1 -- 172.21.15.67:0/1926985797 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0010ea0 con 0x7f1cb4004800 2024-05-30T16:54:58.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.914+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd0150000 con 0x7f1cb4004800 2024-05-30T16:54:58.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.915+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd00689e0 con 0x7f1cb4004800 2024-05-30T16:54:58.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.916+0000 7f1cc6000640 1 -- 172.21.15.67:0/1926985797 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f1cc001c400 con 0x7f1cb4004800 2024-05-30T16:54:58.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.917+0000 7f1cc6000640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1cac06d610 0x7f1cac06fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:58.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.917+0000 7f1cc6000640 1 -- 172.21.15.67:0/1926985797 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f1cc008a9d0 con 0x7f1cb4004800 2024-05-30T16:54:58.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.917+0000 7f1ccd400640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1cac06d610 0x7f1cac06fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:58.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.918+0000 7f1ccd400640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1cac06d610 0x7f1cac06fad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8033000 tx=0x7f1cb8009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:58.921+0000 7f1cc6000640 1 -- 172.21.15.67:0/1926985797 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1cc0018030 con 0x7f1cb4004800 2024-05-30T16:54:59.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.174+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1cd00af380 con 0x7f1cb4004800 2024-05-30T16:54:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.662+0000 7f1cc6000640 1 -- 172.21.15.67:0/1926985797 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f1cc005a850 con 0x7f1cb4004800 2024-05-30T16:54:59.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.666+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1cac06d610 msgr2=0x7f1cac06fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:59.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.666+0000 7f1ccf200640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1cac06d610 0x7f1cac06fad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8033000 tx=0x7f1cb8009030 comp rx=0 tx=0).stop 2024-05-30T16:54:59.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.666+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 msgr2=0x7f1cd014fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.666+0000 7f1ccf200640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cd014fac0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00107f0 tx=0x7f1cc0010cb0 comp rx=0 tx=0).stop 2024-05-30T16:54:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.667+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 shutdown_connections 2024-05-30T16:54:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.667+0000 7f1ccf200640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1cac06d610 0x7f1cac06fad0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.667+0000 7f1ccf200640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cd00736a0 0x7f1cd00b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.667+0000 7f1ccf200640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cd0150300 0x7f1cd01e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.667+0000 7f1ccf200640 1 --2- 172.21.15.67:0/1926985797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cb4004800 0x7f1cd014fac0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.667+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 >> 172.21.15.67:0/1926985797 conn(0x7f1cd006b900 msgr2=0x7f1cd0151140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.667+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 shutdown_connections 2024-05-30T16:54:59.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.667+0000 7f1ccf200640 1 -- 172.21.15.67:0/1926985797 wait complete. 2024-05-30T16:54:59.689 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-30T16:54:59.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:54:59.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.908+0000 7fc497200640 1 Processor -- start 2024-05-30T16:54:59.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.909+0000 7fc497200640 1 -- start start 2024-05-30T16:54:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.910+0000 7fc497200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc49015b3a0 0x7fc4900b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.910+0000 7fc497200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4900afd60 con 0x7fc49006dc20 2024-05-30T16:54:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.910+0000 7fc497200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4900af7a0 con 0x7fc490073310 2024-05-30T16:54:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.910+0000 7fc497200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc49006f980 con 0x7fc49015b3a0 2024-05-30T16:54:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.910+0000 7fc495e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc49015b3a0 0x7fc4900b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.910+0000 7fc495400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc49006dc20 0x7fc490072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34594/0 (socket says 172.21.15.67:34594) 2024-05-30T16:54:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.910+0000 7fc495400640 1 -- 172.21.15.67:0/1244777999 learned_addr learned my addr 172.21.15.67:0/1244777999 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:54:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.910+0000 7fc495e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc49015b3a0 0x7fc4900b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49334/0 (socket says 172.21.15.67:49334) 2024-05-30T16:54:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.911+0000 7fc495e00640 1 -- 172.21.15.67:0/1244777999 >> v1:172.21.15.169:6789/0 conn(0x7fc490073310 legacy=0x7fc4900ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:54:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.911+0000 7fc495e00640 1 -- 172.21.15.67:0/1244777999 >> v1:172.21.15.67:6789/0 conn(0x7fc49006dc20 legacy=0x7fc490072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.911+0000 7fc495e00640 1 -- 172.21.15.67:0/1244777999 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4900afa80 con 0x7fc49015b3a0 2024-05-30T16:54:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.912+0000 7fc495e00640 1 --2- 172.21.15.67:0/1244777999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc49015b3a0 0x7fc4900b32c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc4840097a0 tx=0x7fc48402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8dc1cd772655731 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.912+0000 7fc494a00640 1 -- 172.21.15.67:0/1244777999 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484037070 con 0x7fc49015b3a0 2024-05-30T16:54:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.912+0000 7fc494a00640 1 -- 172.21.15.67:0/1244777999 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc484032440 con 0x7fc49015b3a0 2024-05-30T16:54:59.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.912+0000 7fc494a00640 1 -- 172.21.15.67:0/1244777999 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484032810 con 0x7fc49015b3a0 2024-05-30T16:54:59.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.913+0000 7fc497200640 1 -- 172.21.15.67:0/1244777999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc49015b3a0 msgr2=0x7fc4900b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:59.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.913+0000 7fc497200640 1 --2- 172.21.15.67:0/1244777999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc49015b3a0 0x7fc4900b32c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc4840097a0 tx=0x7fc48402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:54:59.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.913+0000 7fc497200640 1 -- 172.21.15.67:0/1244777999 shutdown_connections 2024-05-30T16:54:59.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.913+0000 7fc497200640 1 --2- 172.21.15.67:0/1244777999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc49015b3a0 0x7fc4900b32c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:59.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.913+0000 7fc497200640 1 -- 172.21.15.67:0/1244777999 >> 172.21.15.67:0/1244777999 conn(0x7fc4900b1fc0 msgr2=0x7fc4900b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:54:59.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.913+0000 7fc497200640 1 -- 172.21.15.67:0/1244777999 shutdown_connections 2024-05-30T16:54:59.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.913+0000 7fc497200640 1 -- 172.21.15.67:0/1244777999 wait complete. 2024-05-30T16:54:59.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.915+0000 7fc497200640 1 Processor -- start 2024-05-30T16:54:59.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.915+0000 7fc497200640 1 -- start start 2024-05-30T16:54:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc497200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc49006dc20 0x7fc49014d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc497200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc490073310 0x7fc4901502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:59.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc497200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc49015b3a0 0x7fc4901507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:59.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc497200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc49015f9a0 con 0x7fc490073310 2024-05-30T16:54:59.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc497200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc4900ae0c0 con 0x7fc49006dc20 2024-05-30T16:54:59.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc497200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc49015fb20 con 0x7fc49015b3a0 2024-05-30T16:54:59.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc495400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc490073310 0x7fc4901502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:59.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc495400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc490073310 0x7fc4901502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49338/0 (socket says 172.21.15.67:49338) 2024-05-30T16:54:59.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc495400640 1 -- 172.21.15.67:0/3091583028 learned_addr learned my addr 172.21.15.67:0/3091583028 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:54:59.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc495400640 1 -- 172.21.15.67:0/3091583028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc49015b3a0 msgr2=0x7fc4901507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:59.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc495400640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc49015b3a0 0x7fc4901507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:59.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc495400640 1 -- 172.21.15.67:0/3091583028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc49006dc20 msgr2=0x7fc49014d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:54:59.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.916+0000 7fc495400640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc49006dc20 0x7fc49014d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:54:59.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.917+0000 7fc495400640 1 -- 172.21.15.67:0/3091583028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc490152ba0 con 0x7fc490073310 2024-05-30T16:54:59.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.917+0000 7fc495400640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc490073310 0x7fc4901502b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c400 tx=0x7fc48000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:59.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.917+0000 7fc48a000640 1 -- 172.21.15.67:0/3091583028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480017070 con 0x7fc490073310 2024-05-30T16:54:59.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.917+0000 7fc48a000640 1 -- 172.21.15.67:0/3091583028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc480005ce0 con 0x7fc490073310 2024-05-30T16:54:59.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.917+0000 7fc48a000640 1 -- 172.21.15.67:0/3091583028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4800072f0 con 0x7fc490073310 2024-05-30T16:54:59.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.917+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49014eed0 con 0x7fc490073310 2024-05-30T16:54:59.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.917+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49014f490 con 0x7fc490073310 2024-05-30T16:54:59.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.918+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4900689e0 con 0x7fc490073310 2024-05-30T16:54:59.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.920+0000 7fc48a000640 1 -- 172.21.15.67:0/3091583028 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fc48001a070 con 0x7fc490073310 2024-05-30T16:54:59.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.920+0000 7fc48a000640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc47406d5e0 0x7fc47406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:54:59.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.921+0000 7fc48a000640 1 -- 172.21.15.67:0/3091583028 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fc4800899c0 con 0x7fc490073310 2024-05-30T16:54:59.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.921+0000 7fc495e00640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc47406d5e0 0x7fc47406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:54:59.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.922+0000 7fc495e00640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc47406d5e0 0x7fc47406faa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc484034000 tx=0x7fc48402c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:54:59.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:54:59.925+0000 7fc48a000640 1 -- 172.21.15.67:0/3091583028 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc480059840 con 0x7fc490073310 2024-05-30T16:55:00.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.172+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4900b2e90 con 0x7fc490073310 2024-05-30T16:55:00.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.173+0000 7fc48a000640 1 -- 172.21.15.67:0/3091583028 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5189 (secure 0 0 0) 0x7fc480059140 con 0x7fc490073310 2024-05-30T16:55:00.174 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:55:00.174 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5171,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1912096869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1912096869},{"type":"v1","addr":"172.21.15.169:6835","nonce":1912096869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5608,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3963396329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3963396329},{"type":"v1","addr":"172.21.15.67:6835","nonce":3963396329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2169821644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2169821644},{"type":"v1","addr":"172.21.15.169:6834","nonce":2169821644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:54:56.078869+0000","modified":"2024-05-30T16:54:59.175686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6558":{"gid":6558,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/2706164583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2706164583},{"type":"v1","addr":"172.21.15.67:6837","nonce":2706164583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6558,"qdb_cluster":[6558]},"id":5}]} 2024-05-30T16:55:00.174 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 45 2024-05-30T16:55:00.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc47406d5e0 msgr2=0x7fc47406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc47406d5e0 0x7fc47406faa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc484034000 tx=0x7fc48402c000 comp rx=0 tx=0).stop 2024-05-30T16:55:00.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc490073310 msgr2=0x7fc4901502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc490073310 0x7fc4901502b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c400 tx=0x7fc48000c8c0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 shutdown_connections 2024-05-30T16:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc47406d5e0 0x7fc47406faa0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc49015b3a0 0x7fc4901507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc490073310 0x7fc4901502b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 --2- 172.21.15.67:0/3091583028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc49006dc20 0x7fc49014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 >> 172.21.15.67:0/3091583028 conn(0x7fc4900b1fc0 msgr2=0x7fc49006e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 shutdown_connections 2024-05-30T16:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.176+0000 7fc497200640 1 -- 172.21.15.67:0/3091583028 wait complete. 2024-05-30T16:55:00.195 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:55:00.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.408+0000 7ff87d400640 1 Processor -- start 2024-05-30T16:55:00.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.408+0000 7ff87d400640 1 -- start start 2024-05-30T16:55:00.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.410+0000 7ff87d400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff878158b80 0x7ff87806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.410+0000 7ff87d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff87806e160 0x7ff878071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.410+0000 7ff87d400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff878072510 con 0x7ff878072100 2024-05-30T16:55:00.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.410+0000 7ff87d400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff8780b42d0 con 0x7ff87806e160 2024-05-30T16:55:00.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.410+0000 7ff87d400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff87806e6e0 con 0x7ff878158b80 2024-05-30T16:55:00.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.410+0000 7ff877400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff87806e160 0x7ff878071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.410+0000 7ff877400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff87806e160 0x7ff878071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49352/0 (socket says 172.21.15.67:49352) 2024-05-30T16:55:00.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.410+0000 7ff877400640 1 -- 172.21.15.67:0/285946622 learned_addr learned my addr 172.21.15.67:0/285946622 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:00.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.411+0000 7ff877e00640 1 --2- 172.21.15.67:0/285946622 >> v2:172.21.15.169:3300/0 conn(0x7ff878158b80 0x7ff87806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.411+0000 7ff877400640 1 -- 172.21.15.67:0/285946622 >> v2:172.21.15.169:3300/0 conn(0x7ff878158b80 msgr2=0x7ff87806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:00.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.411+0000 7ff877400640 1 --2- 172.21.15.67:0/285946622 >> v2:172.21.15.169:3300/0 conn(0x7ff878158b80 0x7ff87806dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.411+0000 7ff877400640 1 -- 172.21.15.67:0/285946622 >> v1:172.21.15.67:6789/0 conn(0x7ff878072100 legacy=0x7ff8781621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.411+0000 7ff877400640 1 -- 172.21.15.67:0/285946622 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87806e9c0 con 0x7ff87806e160 2024-05-30T16:55:00.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.411+0000 7ff877e00640 1 --2- 172.21.15.67:0/285946622 >> v2:172.21.15.169:3300/0 conn(0x7ff878158b80 0x7ff87806dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:00.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.412+0000 7ff877400640 1 --2- 172.21.15.67:0/285946622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff87806e160 0x7ff878071bc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff87806c830 tx=0x7ff86402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb05ae973828359 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.412+0000 7ff876a00640 1 -- 172.21.15.67:0/285946622 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864037070 con 0x7ff87806e160 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.412+0000 7ff876a00640 1 -- 172.21.15.67:0/285946622 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff864032440 con 0x7ff87806e160 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.412+0000 7ff876a00640 1 -- 172.21.15.67:0/285946622 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864032810 con 0x7ff87806e160 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.413+0000 7ff87d400640 1 -- 172.21.15.67:0/285946622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff87806e160 msgr2=0x7ff878071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.413+0000 7ff87d400640 1 --2- 172.21.15.67:0/285946622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff87806e160 0x7ff878071bc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff87806c830 tx=0x7ff86402a270 comp rx=0 tx=0).stop 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.413+0000 7ff87d400640 1 -- 172.21.15.67:0/285946622 shutdown_connections 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.413+0000 7ff87d400640 1 --2- 172.21.15.67:0/285946622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff87806e160 0x7ff878071bc0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.413+0000 7ff87d400640 1 --2- 172.21.15.67:0/285946622 >> v2:172.21.15.169:3300/0 conn(0x7ff878158b80 0x7ff87806dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.413+0000 7ff87d400640 1 -- 172.21.15.67:0/285946622 >> 172.21.15.67:0/285946622 conn(0x7ff8780b1fc0 msgr2=0x7ff8780b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:00.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.413+0000 7ff87d400640 1 -- 172.21.15.67:0/285946622 shutdown_connections 2024-05-30T16:55:00.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.413+0000 7ff87d400640 1 -- 172.21.15.67:0/285946622 wait complete. 2024-05-30T16:55:00.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.415+0000 7ff87d400640 1 Processor -- start 2024-05-30T16:55:00.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.415+0000 7ff87d400640 1 -- start start 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.416+0000 7ff87d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87806e160 0x7ff8781f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.416+0000 7ff87d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff878072100 0x7ff878146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.416+0000 7ff87d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff878158b80 0x7ff87809de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.416+0000 7ff87d400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff87806f260 con 0x7ff878158b80 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.416+0000 7ff87d400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff87806ef80 con 0x7ff87806e160 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.416+0000 7ff87d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8780b2d30 con 0x7ff878072100 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff87ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff878158b80 0x7ff87809de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff87ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff878158b80 0x7ff87809de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49364/0 (socket says 172.21.15.67:49364) 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff87ca00640 1 -- 172.21.15.67:0/3269778868 learned_addr learned my addr 172.21.15.67:0/3269778868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff877400640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff878072100 0x7ff878146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff877e00640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87806e160 0x7ff8781f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff87ca00640 1 -- 172.21.15.67:0/3269778868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff878072100 msgr2=0x7ff878146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff87ca00640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff878072100 0x7ff878146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff87ca00640 1 -- 172.21.15.67:0/3269778868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87806e160 msgr2=0x7ff8781f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff87ca00640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87806e160 0x7ff8781f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff87ca00640 1 -- 172.21.15.67:0/3269778868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87809e380 con 0x7ff878158b80 2024-05-30T16:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.417+0000 7ff87ca00640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff878158b80 0x7ff87809de40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff87000c3c0 tx=0x7ff87000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.418+0000 7ff86fe00640 1 -- 172.21.15.67:0/3269778868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870017070 con 0x7ff878158b80 2024-05-30T16:55:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.418+0000 7ff86fe00640 1 -- 172.21.15.67:0/3269778868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8700052d0 con 0x7ff878158b80 2024-05-30T16:55:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.418+0000 7ff86fe00640 1 -- 172.21.15.67:0/3269778868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870003830 con 0x7ff878158b80 2024-05-30T16:55:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.418+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8781f48b0 con 0x7ff878158b80 2024-05-30T16:55:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.418+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781f4e70 con 0x7ff878158b80 2024-05-30T16:55:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.419+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8780689e0 con 0x7ff878158b80 2024-05-30T16:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.420+0000 7ff86fe00640 1 -- 172.21.15.67:0/3269778868 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7ff87001a070 con 0x7ff878158b80 2024-05-30T16:55:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.422+0000 7ff86fe00640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85806d5e0 0x7ff85806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.422+0000 7ff86fe00640 1 -- 172.21.15.67:0/3269778868 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7ff8700898e0 con 0x7ff878158b80 2024-05-30T16:55:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.422+0000 7ff877e00640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85806d5e0 0x7ff85806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.423+0000 7ff877e00640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85806d5e0 0x7ff85806faa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff868008ee0 tx=0x7ff868008430 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.427+0000 7ff86fe00640 1 -- 172.21.15.67:0/3269778868 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff878072510 con 0x7ff878158b80 2024-05-30T16:55:00.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.634+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8780d02b0 con 0x7ff878158b80 2024-05-30T16:55:00.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.636+0000 7ff86fe00640 1 -- 172.21.15.67:0/3269778868 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16743 (secure 0 0 0) 0x7ff8700596a0 con 0x7ff878158b80 2024-05-30T16:55:00.637 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:55:00.637 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":74,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:54:57.635318+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:54:54.017222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-30T16:54:55.063707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:55:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.638+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85806d5e0 msgr2=0x7ff85806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.638+0000 7ff87d400640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85806d5e0 0x7ff85806faa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff868008ee0 tx=0x7ff868008430 comp rx=0 tx=0).stop 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.638+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff878158b80 msgr2=0x7ff87809de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.638+0000 7ff87d400640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff878158b80 0x7ff87809de40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff87000c3c0 tx=0x7ff87000c880 comp rx=0 tx=0).stop 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.639+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 shutdown_connections 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.639+0000 7ff87d400640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85806d5e0 0x7ff85806faa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.639+0000 7ff87d400640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff878158b80 0x7ff87809de40 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.639+0000 7ff87d400640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff878072100 0x7ff878146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.639+0000 7ff87d400640 1 --2- 172.21.15.67:0/3269778868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff87806e160 0x7ff8781f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.639+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 >> 172.21.15.67:0/3269778868 conn(0x7ff8780b1fc0 msgr2=0x7ff878152250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.639+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 shutdown_connections 2024-05-30T16:55:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.639+0000 7ff87d400640 1 -- 172.21.15.67:0/3269778868 wait complete. 2024-05-30T16:55:00.658 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T16:55:00.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:55:00.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.860+0000 7f2196600640 1 Processor -- start 2024-05-30T16:55:00.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.861+0000 7f2196600640 1 -- start start 2024-05-30T16:55:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.862+0000 7f2196600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21900b4e10 0x7f21900cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.862+0000 7f2196600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21900b51f0 con 0x7f21900ce370 2024-05-30T16:55:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.862+0000 7f2196600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21900aef60 con 0x7f21900d0ca0 2024-05-30T16:55:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.862+0000 7f2196600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f21900ad1d0 con 0x7f21900b4e10 2024-05-30T16:55:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.863+0000 7f218f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21900b4e10 0x7f21900cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.863+0000 7f218f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21900b4e10 0x7f21900cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48126/0 (socket says 172.21.15.67:48126) 2024-05-30T16:55:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.863+0000 7f218f400640 1 -- 172.21.15.67:0/330441498 learned_addr learned my addr 172.21.15.67:0/330441498 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.863+0000 7f218f400640 1 -- 172.21.15.67:0/330441498 >> v1:172.21.15.169:6789/0 conn(0x7f21900d0ca0 legacy=0x7f2190099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.863+0000 7f218f400640 1 -- 172.21.15.67:0/330441498 >> v1:172.21.15.67:6789/0 conn(0x7f21900ce370 legacy=0x7f21900d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.863+0000 7f218f400640 1 -- 172.21.15.67:0/330441498 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21900ad470 con 0x7f21900b4e10 2024-05-30T16:55:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.864+0000 7f218f400640 1 --2- 172.21.15.67:0/330441498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900b4e10 0x7f21900cde30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2180009870 tx=0x7f218002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bf41bd8feebe67 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.865+0000 7f218ea00640 1 -- 172.21.15.67:0/330441498 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180037070 con 0x7f21900b4e10 2024-05-30T16:55:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.865+0000 7f218ea00640 1 -- 172.21.15.67:0/330441498 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2180032440 con 0x7f21900b4e10 2024-05-30T16:55:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.865+0000 7f218ea00640 1 -- 172.21.15.67:0/330441498 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180032810 con 0x7f21900b4e10 2024-05-30T16:55:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.865+0000 7f2196600640 1 -- 172.21.15.67:0/330441498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900b4e10 msgr2=0x7f21900cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.865+0000 7f2196600640 1 --2- 172.21.15.67:0/330441498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900b4e10 0x7f21900cde30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2180009870 tx=0x7f218002a210 comp rx=0 tx=0).stop 2024-05-30T16:55:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.866+0000 7f2196600640 1 -- 172.21.15.67:0/330441498 shutdown_connections 2024-05-30T16:55:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.866+0000 7f2196600640 1 --2- 172.21.15.67:0/330441498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900b4e10 0x7f21900cde30 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.866+0000 7f2196600640 1 -- 172.21.15.67:0/330441498 >> 172.21.15.67:0/330441498 conn(0x7f219006bae0 msgr2=0x7f219006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.866+0000 7f2196600640 1 -- 172.21.15.67:0/330441498 shutdown_connections 2024-05-30T16:55:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.866+0000 7f2196600640 1 -- 172.21.15.67:0/330441498 wait complete. 2024-05-30T16:55:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.868+0000 7f2196600640 1 Processor -- start 2024-05-30T16:55:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.868+0000 7f2196600640 1 -- start start 2024-05-30T16:55:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.869+0000 7f2196600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21900b4e10 0x7f21901533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.869+0000 7f2196600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21900ce370 0x7f21900afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.869+0000 7f2196600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900d0ca0 0x7f2190072930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.869+0000 7f2196600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21900cea00 con 0x7f21900d0ca0 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.869+0000 7f2196600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21900ce880 con 0x7f21900b4e10 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.869+0000 7f2196600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21900b3150 con 0x7f21900ce370 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.869+0000 7f218fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900d0ca0 0x7f2190072930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.869+0000 7f218fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900d0ca0 0x7f2190072930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48132/0 (socket says 172.21.15.67:48132) 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.869+0000 7f218fe00640 1 -- 172.21.15.67:0/3784835851 learned_addr learned my addr 172.21.15.67:0/3784835851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f218f400640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21900b4e10 0x7f21901533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f2186a00640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21900ce370 0x7f21900afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f218fe00640 1 -- 172.21.15.67:0/3784835851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21900ce370 msgr2=0x7f21900afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f218fe00640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21900ce370 0x7f21900afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f218fe00640 1 -- 172.21.15.67:0/3784835851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21900b4e10 msgr2=0x7f21901533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f218fe00640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21900b4e10 0x7f21901533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f218fe00640 1 -- 172.21.15.67:0/3784835851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21901539c0 con 0x7f21900d0ca0 2024-05-30T16:55:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f218f400640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21900b4e10 0x7f21901533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f218fe00640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900d0ca0 0x7f2190072930 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f218800c730 tx=0x7f218800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f2187e00640 1 -- 172.21.15.67:0/3784835851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188016020 con 0x7f21900d0ca0 2024-05-30T16:55:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f2187e00640 1 -- 172.21.15.67:0/3784835851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218800aa50 con 0x7f21900d0ca0 2024-05-30T16:55:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f2187e00640 1 -- 172.21.15.67:0/3784835851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218800ce40 con 0x7f21900d0ca0 2024-05-30T16:55:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.870+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2190153ca0 con 0x7f21900d0ca0 2024-05-30T16:55:00.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.871+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2190154260 con 0x7f21900d0ca0 2024-05-30T16:55:00.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.872+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21900cce30 con 0x7f21900d0ca0 2024-05-30T16:55:00.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.873+0000 7f2187e00640 1 -- 172.21.15.67:0/3784835851 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f2188019020 con 0x7f21900d0ca0 2024-05-30T16:55:00.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.874+0000 7f2187e00640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f216c06d5e0 0x7f216c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:00.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.874+0000 7f2187e00640 1 -- 172.21.15.67:0/3784835851 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f218808a570 con 0x7f21900d0ca0 2024-05-30T16:55:00.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.874+0000 7f218f400640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f216c06d5e0 0x7f216c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:00.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.875+0000 7f218f400640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f216c06d5e0 0x7f216c06faa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2180034000 tx=0x7f218002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:00.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:00.878+0000 7f2187e00640 1 -- 172.21.15.67:0/3784835851 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21900cce30 con 0x7f21900d0ca0 2024-05-30T16:55:01.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.080+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21900689e0 con 0x7f216c06d5e0 2024-05-30T16:55:01.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.101+0000 7f2187e00640 1 -- 172.21.15.67:0/3784835851 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f21900689e0 con 0x7f216c06d5e0 2024-05-30T16:55:01.106 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:55:01.107 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:55:01.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.107+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f216c06d5e0 msgr2=0x7f216c06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.107+0000 7f2196600640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f216c06d5e0 0x7f216c06faa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2180034000 tx=0x7f218002e000 comp rx=0 tx=0).stop 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.107+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900d0ca0 msgr2=0x7f2190072930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.107+0000 7f2196600640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900d0ca0 0x7f2190072930 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f218800c730 tx=0x7f218800cbf0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.108+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 shutdown_connections 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.108+0000 7f2196600640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f216c06d5e0 0x7f216c06faa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.108+0000 7f2196600640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21900d0ca0 0x7f2190072930 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.108+0000 7f2196600640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21900ce370 0x7f21900afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.108+0000 7f2196600640 1 --2- 172.21.15.67:0/3784835851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21900b4e10 0x7f21901533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.108+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 >> 172.21.15.67:0/3784835851 conn(0x7f219006bae0 msgr2=0x7f219006e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.108+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 shutdown_connections 2024-05-30T16:55:01.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.108+0000 7f2196600640 1 -- 172.21.15.67:0/3784835851 wait complete. 2024-05-30T16:55:01.129 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-05-30T16:55:00.274360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":447800,"kb_used_data":8200,"kb_used_omap":12,"kb_used_meta":439475,"kb_avail":754526920,"statfs":{"total":773094113280,"available":772635566080,"internally_reserved":0,"allocated":8396800,"data_stored":4231596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":450022960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.691960"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746720+0000","last_change":"2024-05-30T16:54:56.668387+0000","last_active":"2024-05-30T16:54:57.746720+0000","last_peered":"2024-05-30T16:54:57.746720+0000","last_clean":"2024-05-30T16:54:57.746720+0000","last_became_active":"2024-05-30T16:54:56.668003+0000","last_became_peered":"2024-05-30T16:54:56.668003+0000","last_unstale":"2024-05-30T16:54:57.746720+0000","last_undegraded":"2024-05-30T16:54:57.746720+0000","last_fullsized":"2024-05-30T16:54:57.746720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:40:56.375503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654299+0000","last_change":"2024-05-30T16:54:56.658136+0000","last_active":"2024-05-30T16:54:57.654299+0000","last_peered":"2024-05-30T16:54:57.654299+0000","last_clean":"2024-05-30T16:54:57.654299+0000","last_became_active":"2024-05-30T16:54:56.656742+0000","last_became_peered":"2024-05-30T16:54:56.656742+0000","last_unstale":"2024-05-30T16:54:57.654299+0000","last_undegraded":"2024-05-30T16:54:57.654299+0000","last_fullsized":"2024-05-30T16:54:57.654299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:35:20.382926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745196+0000","last_change":"2024-05-30T16:54:56.653171+0000","last_active":"2024-05-30T16:54:57.745196+0000","last_peered":"2024-05-30T16:54:57.745196+0000","last_clean":"2024-05-30T16:54:57.745196+0000","last_became_active":"2024-05-30T16:54:56.651534+0000","last_became_peered":"2024-05-30T16:54:56.651534+0000","last_unstale":"2024-05-30T16:54:57.745196+0000","last_undegraded":"2024-05-30T16:54:57.745196+0000","last_fullsized":"2024-05-30T16:54:57.745196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:03.881908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649428+0000","last_change":"2024-05-30T16:54:56.661986+0000","last_active":"2024-05-30T16:54:57.649428+0000","last_peered":"2024-05-30T16:54:57.649428+0000","last_clean":"2024-05-30T16:54:57.649428+0000","last_became_active":"2024-05-30T16:54:56.661453+0000","last_became_peered":"2024-05-30T16:54:56.661453+0000","last_unstale":"2024-05-30T16:54:57.649428+0000","last_undegraded":"2024-05-30T16:54:57.649428+0000","last_fullsized":"2024-05-30T16:54:57.649428+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:53:37.735789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743511+0000","last_change":"2024-05-30T16:54:56.646452+0000","last_active":"2024-05-30T16:54:57.743511+0000","last_peered":"2024-05-30T16:54:57.743511+0000","last_clean":"2024-05-30T16:54:57.743511+0000","last_became_active":"2024-05-30T16:54:56.645189+0000","last_became_peered":"2024-05-30T16:54:56.645189+0000","last_unstale":"2024-05-30T16:54:57.743511+0000","last_undegraded":"2024-05-30T16:54:57.743511+0000","last_fullsized":"2024-05-30T16:54:57.743511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:56:25.427282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652631+0000","last_change":"2024-05-30T16:54:56.660046+0000","last_active":"2024-05-30T16:54:57.652631+0000","last_peered":"2024-05-30T16:54:57.652631+0000","last_clean":"2024-05-30T16:54:57.652631+0000","last_became_active":"2024-05-30T16:54:56.659780+0000","last_became_peered":"2024-05-30T16:54:56.659780+0000","last_unstale":"2024-05-30T16:54:57.652631+0000","last_undegraded":"2024-05-30T16:54:57.652631+0000","last_fullsized":"2024-05-30T16:54:57.652631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:38:11.617688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.651802+0000","last_change":"2024-05-30T16:54:56.637859+0000","last_active":"2024-05-30T16:54:57.651802+0000","last_peered":"2024-05-30T16:54:57.651802+0000","last_clean":"2024-05-30T16:54:57.651802+0000","last_became_active":"2024-05-30T16:54:56.636940+0000","last_became_peered":"2024-05-30T16:54:56.636940+0000","last_unstale":"2024-05-30T16:54:57.651802+0000","last_undegraded":"2024-05-30T16:54:57.651802+0000","last_fullsized":"2024-05-30T16:54:57.651802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:59.281724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652873+0000","last_change":"2024-05-30T16:54:56.648376+0000","last_active":"2024-05-30T16:54:57.652873+0000","last_peered":"2024-05-30T16:54:57.652873+0000","last_clean":"2024-05-30T16:54:57.652873+0000","last_became_active":"2024-05-30T16:54:56.647888+0000","last_became_peered":"2024-05-30T16:54:56.647888+0000","last_unstale":"2024-05-30T16:54:57.652873+0000","last_undegraded":"2024-05-30T16:54:57.652873+0000","last_fullsized":"2024-05-30T16:54:57.652873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:04.447059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744146+0000","last_change":"2024-05-30T16:54:56.655198+0000","last_active":"2024-05-30T16:54:57.744146+0000","last_peered":"2024-05-30T16:54:57.744146+0000","last_clean":"2024-05-30T16:54:57.744146+0000","last_became_active":"2024-05-30T16:54:56.654928+0000","last_became_peered":"2024-05-30T16:54:56.654928+0000","last_unstale":"2024-05-30T16:54:57.744146+0000","last_undegraded":"2024-05-30T16:54:57.744146+0000","last_fullsized":"2024-05-30T16:54:57.744146+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:04:16.402646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747523+0000","last_change":"2024-05-30T16:54:56.655011+0000","last_active":"2024-05-30T16:54:57.747523+0000","last_peered":"2024-05-30T16:54:57.747523+0000","last_clean":"2024-05-30T16:54:57.747523+0000","last_became_active":"2024-05-30T16:54:56.650949+0000","last_became_peered":"2024-05-30T16:54:56.650949+0000","last_unstale":"2024-05-30T16:54:57.747523+0000","last_undegraded":"2024-05-30T16:54:57.747523+0000","last_fullsized":"2024-05-30T16:54:57.747523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:27.907456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746765+0000","last_change":"2024-05-30T16:54:56.656036+0000","last_active":"2024-05-30T16:54:57.746765+0000","last_peered":"2024-05-30T16:54:57.746765+0000","last_clean":"2024-05-30T16:54:57.746765+0000","last_became_active":"2024-05-30T16:54:56.655646+0000","last_became_peered":"2024-05-30T16:54:56.655646+0000","last_unstale":"2024-05-30T16:54:57.746765+0000","last_undegraded":"2024-05-30T16:54:57.746765+0000","last_fullsized":"2024-05-30T16:54:57.746765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:41:45.993512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649658+0000","last_change":"2024-05-30T16:54:56.654191+0000","last_active":"2024-05-30T16:54:57.649658+0000","last_peered":"2024-05-30T16:54:57.649658+0000","last_clean":"2024-05-30T16:54:57.649658+0000","last_became_active":"2024-05-30T16:54:56.653132+0000","last_became_peered":"2024-05-30T16:54:56.653132+0000","last_unstale":"2024-05-30T16:54:57.649658+0000","last_undegraded":"2024-05-30T16:54:57.649658+0000","last_fullsized":"2024-05-30T16:54:57.649658+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:02.366944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743852+0000","last_change":"2024-05-30T16:54:56.658124+0000","last_active":"2024-05-30T16:54:57.743852+0000","last_peered":"2024-05-30T16:54:57.743852+0000","last_clean":"2024-05-30T16:54:57.743852+0000","last_became_active":"2024-05-30T16:54:56.655085+0000","last_became_peered":"2024-05-30T16:54:56.655085+0000","last_unstale":"2024-05-30T16:54:57.743852+0000","last_undegraded":"2024-05-30T16:54:57.743852+0000","last_fullsized":"2024-05-30T16:54:57.743852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:14:01.523291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743565+0000","last_change":"2024-05-30T16:54:56.646454+0000","last_active":"2024-05-30T16:54:57.743565+0000","last_peered":"2024-05-30T16:54:57.743565+0000","last_clean":"2024-05-30T16:54:57.743565+0000","last_became_active":"2024-05-30T16:54:56.645488+0000","last_became_peered":"2024-05-30T16:54:56.645488+0000","last_unstale":"2024-05-30T16:54:57.743565+0000","last_undegraded":"2024-05-30T16:54:57.743565+0000","last_fullsized":"2024-05-30T16:54:57.743565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:04.334329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743607+0000","last_change":"2024-05-30T16:54:56.656993+0000","last_active":"2024-05-30T16:54:57.743607+0000","last_peered":"2024-05-30T16:54:57.743607+0000","last_clean":"2024-05-30T16:54:57.743607+0000","last_became_active":"2024-05-30T16:54:56.656664+0000","last_became_peered":"2024-05-30T16:54:56.656664+0000","last_unstale":"2024-05-30T16:54:57.743607+0000","last_undegraded":"2024-05-30T16:54:57.743607+0000","last_fullsized":"2024-05-30T16:54:57.743607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:45:02.436795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745066+0000","last_change":"2024-05-30T16:54:56.654092+0000","last_active":"2024-05-30T16:54:57.745066+0000","last_peered":"2024-05-30T16:54:57.745066+0000","last_clean":"2024-05-30T16:54:57.745066+0000","last_became_active":"2024-05-30T16:54:56.649755+0000","last_became_peered":"2024-05-30T16:54:56.649755+0000","last_unstale":"2024-05-30T16:54:57.745066+0000","last_undegraded":"2024-05-30T16:54:57.745066+0000","last_fullsized":"2024-05-30T16:54:57.745066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:38:11.617688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744174+0000","last_change":"2024-05-30T16:54:56.653022+0000","last_active":"2024-05-30T16:54:57.744174+0000","last_peered":"2024-05-30T16:54:57.744174+0000","last_clean":"2024-05-30T16:54:57.744174+0000","last_became_active":"2024-05-30T16:54:56.652636+0000","last_became_peered":"2024-05-30T16:54:56.652636+0000","last_unstale":"2024-05-30T16:54:57.744174+0000","last_undegraded":"2024-05-30T16:54:57.744174+0000","last_fullsized":"2024-05-30T16:54:57.744174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:53:37.735789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649826+0000","last_change":"2024-05-30T16:54:56.654605+0000","last_active":"2024-05-30T16:54:57.649826+0000","last_peered":"2024-05-30T16:54:57.649826+0000","last_clean":"2024-05-30T16:54:57.649826+0000","last_became_active":"2024-05-30T16:54:56.654328+0000","last_became_peered":"2024-05-30T16:54:56.654328+0000","last_unstale":"2024-05-30T16:54:57.649826+0000","last_undegraded":"2024-05-30T16:54:57.649826+0000","last_fullsized":"2024-05-30T16:54:57.649826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:23.342774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653977+0000","last_change":"2024-05-30T16:54:56.647710+0000","last_active":"2024-05-30T16:54:57.653977+0000","last_peered":"2024-05-30T16:54:57.653977+0000","last_clean":"2024-05-30T16:54:57.653977+0000","last_became_active":"2024-05-30T16:54:56.647270+0000","last_became_peered":"2024-05-30T16:54:56.647270+0000","last_unstale":"2024-05-30T16:54:57.653977+0000","last_undegraded":"2024-05-30T16:54:57.653977+0000","last_fullsized":"2024-05-30T16:54:57.653977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:13.200522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743282+0000","last_change":"2024-05-30T16:54:56.651160+0000","last_active":"2024-05-30T16:54:57.743282+0000","last_peered":"2024-05-30T16:54:57.743282+0000","last_clean":"2024-05-30T16:54:57.743282+0000","last_became_active":"2024-05-30T16:54:56.650855+0000","last_became_peered":"2024-05-30T16:54:56.650855+0000","last_unstale":"2024-05-30T16:54:57.743282+0000","last_undegraded":"2024-05-30T16:54:57.743282+0000","last_fullsized":"2024-05-30T16:54:57.743282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:30.080641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745876+0000","last_change":"2024-05-30T16:54:56.651661+0000","last_active":"2024-05-30T16:54:57.745876+0000","last_peered":"2024-05-30T16:54:57.745876+0000","last_clean":"2024-05-30T16:54:57.745876+0000","last_became_active":"2024-05-30T16:54:56.651355+0000","last_became_peered":"2024-05-30T16:54:56.651355+0000","last_unstale":"2024-05-30T16:54:57.745876+0000","last_undegraded":"2024-05-30T16:54:57.745876+0000","last_fullsized":"2024-05-30T16:54:57.745876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:08:13.510748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745077+0000","last_change":"2024-05-30T16:54:56.656794+0000","last_active":"2024-05-30T16:54:57.745077+0000","last_peered":"2024-05-30T16:54:57.745077+0000","last_clean":"2024-05-30T16:54:57.745077+0000","last_became_active":"2024-05-30T16:54:56.655613+0000","last_became_peered":"2024-05-30T16:54:56.655613+0000","last_unstale":"2024-05-30T16:54:57.745077+0000","last_undegraded":"2024-05-30T16:54:57.745077+0000","last_fullsized":"2024-05-30T16:54:57.745077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:04.447059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747605+0000","last_change":"2024-05-30T16:54:56.656425+0000","last_active":"2024-05-30T16:54:57.747605+0000","last_peered":"2024-05-30T16:54:57.747605+0000","last_clean":"2024-05-30T16:54:57.747605+0000","last_became_active":"2024-05-30T16:54:56.652392+0000","last_became_peered":"2024-05-30T16:54:56.652392+0000","last_unstale":"2024-05-30T16:54:57.747605+0000","last_undegraded":"2024-05-30T16:54:57.747605+0000","last_fullsized":"2024-05-30T16:54:57.747605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:29:18.084001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746811+0000","last_change":"2024-05-30T16:54:56.654512+0000","last_active":"2024-05-30T16:54:57.746811+0000","last_peered":"2024-05-30T16:54:57.746811+0000","last_clean":"2024-05-30T16:54:57.746811+0000","last_became_active":"2024-05-30T16:54:56.654226+0000","last_became_peered":"2024-05-30T16:54:56.654226+0000","last_unstale":"2024-05-30T16:54:57.746811+0000","last_undegraded":"2024-05-30T16:54:57.746811+0000","last_fullsized":"2024-05-30T16:54:57.746811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:33.721422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746512+0000","last_change":"2024-05-30T16:54:56.660027+0000","last_active":"2024-05-30T16:54:57.746512+0000","last_peered":"2024-05-30T16:54:57.746512+0000","last_clean":"2024-05-30T16:54:57.746512+0000","last_became_active":"2024-05-30T16:54:56.659709+0000","last_became_peered":"2024-05-30T16:54:56.659709+0000","last_unstale":"2024-05-30T16:54:57.746512+0000","last_undegraded":"2024-05-30T16:54:57.746512+0000","last_fullsized":"2024-05-30T16:54:57.746512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:35:20.382926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654788+0000","last_change":"2024-05-30T16:54:56.650514+0000","last_active":"2024-05-30T16:54:57.654788+0000","last_peered":"2024-05-30T16:54:57.654788+0000","last_clean":"2024-05-30T16:54:57.654788+0000","last_became_active":"2024-05-30T16:54:56.650185+0000","last_became_peered":"2024-05-30T16:54:56.650185+0000","last_unstale":"2024-05-30T16:54:57.654788+0000","last_undegraded":"2024-05-30T16:54:57.654788+0000","last_fullsized":"2024-05-30T16:54:57.654788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:11.984521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745216+0000","last_change":"2024-05-30T16:54:56.657943+0000","last_active":"2024-05-30T16:54:57.745216+0000","last_peered":"2024-05-30T16:54:57.745216+0000","last_clean":"2024-05-30T16:54:57.745216+0000","last_became_active":"2024-05-30T16:54:56.657471+0000","last_became_peered":"2024-05-30T16:54:56.657471+0000","last_unstale":"2024-05-30T16:54:57.745216+0000","last_undegraded":"2024-05-30T16:54:57.745216+0000","last_fullsized":"2024-05-30T16:54:57.745216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:49.601902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746196+0000","last_change":"2024-05-30T16:54:56.651927+0000","last_active":"2024-05-30T16:54:57.746196+0000","last_peered":"2024-05-30T16:54:57.746196+0000","last_clean":"2024-05-30T16:54:57.746196+0000","last_became_active":"2024-05-30T16:54:56.651606+0000","last_became_peered":"2024-05-30T16:54:56.651606+0000","last_unstale":"2024-05-30T16:54:57.746196+0000","last_undegraded":"2024-05-30T16:54:57.746196+0000","last_fullsized":"2024-05-30T16:54:57.746196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:41.464064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745196+0000","last_change":"2024-05-30T16:54:56.652611+0000","last_active":"2024-05-30T16:54:57.745196+0000","last_peered":"2024-05-30T16:54:57.745196+0000","last_clean":"2024-05-30T16:54:57.745196+0000","last_became_active":"2024-05-30T16:54:56.650324+0000","last_became_peered":"2024-05-30T16:54:56.650324+0000","last_unstale":"2024-05-30T16:54:57.745196+0000","last_undegraded":"2024-05-30T16:54:57.745196+0000","last_fullsized":"2024-05-30T16:54:57.745196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:38.341289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653606+0000","last_change":"2024-05-30T16:54:56.651181+0000","last_active":"2024-05-30T16:54:57.653606+0000","last_peered":"2024-05-30T16:54:57.653606+0000","last_clean":"2024-05-30T16:54:57.653606+0000","last_became_active":"2024-05-30T16:54:56.650465+0000","last_became_peered":"2024-05-30T16:54:56.650465+0000","last_unstale":"2024-05-30T16:54:57.653606+0000","last_undegraded":"2024-05-30T16:54:57.653606+0000","last_fullsized":"2024-05-30T16:54:57.653606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:35:41.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654805+0000","last_change":"2024-05-30T16:54:56.649580+0000","last_active":"2024-05-30T16:54:57.654805+0000","last_peered":"2024-05-30T16:54:57.654805+0000","last_clean":"2024-05-30T16:54:57.654805+0000","last_became_active":"2024-05-30T16:54:56.649298+0000","last_became_peered":"2024-05-30T16:54:56.649298+0000","last_unstale":"2024-05-30T16:54:57.654805+0000","last_undegraded":"2024-05-30T16:54:57.654805+0000","last_fullsized":"2024-05-30T16:54:57.654805+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:55:02.203424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.651516+0000","last_change":"2024-05-30T16:54:56.660910+0000","last_active":"2024-05-30T16:54:57.651516+0000","last_peered":"2024-05-30T16:54:57.651516+0000","last_clean":"2024-05-30T16:54:57.651516+0000","last_became_active":"2024-05-30T16:54:56.660627+0000","last_became_peered":"2024-05-30T16:54:56.660627+0000","last_unstale":"2024-05-30T16:54:57.651516+0000","last_undegraded":"2024-05-30T16:54:57.651516+0000","last_fullsized":"2024-05-30T16:54:57.651516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:56:12.172501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747211+0000","last_change":"2024-05-30T16:54:56.667544+0000","last_active":"2024-05-30T16:54:57.747211+0000","last_peered":"2024-05-30T16:54:57.747211+0000","last_clean":"2024-05-30T16:54:57.747211+0000","last_became_active":"2024-05-30T16:54:56.665510+0000","last_became_peered":"2024-05-30T16:54:56.665510+0000","last_unstale":"2024-05-30T16:54:57.747211+0000","last_undegraded":"2024-05-30T16:54:57.747211+0000","last_fullsized":"2024-05-30T16:54:57.747211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:29.099619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.748032+0000","last_change":"2024-05-30T16:54:55.628463+0000","last_active":"2024-05-30T16:54:57.748032+0000","last_peered":"2024-05-30T16:54:57.748032+0000","last_clean":"2024-05-30T16:54:57.748032+0000","last_became_active":"2024-05-30T16:54:55.628026+0000","last_became_peered":"2024-05-30T16:54:55.628026+0000","last_unstale":"2024-05-30T16:54:57.748032+0000","last_undegraded":"2024-05-30T16:54:57.748032+0000","last_fullsized":"2024-05-30T16:54:57.748032+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:56.321162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744234+0000","last_change":"2024-05-30T16:54:56.652959+0000","last_active":"2024-05-30T16:54:57.744234+0000","last_peered":"2024-05-30T16:54:57.744234+0000","last_clean":"2024-05-30T16:54:57.744234+0000","last_became_active":"2024-05-30T16:54:56.652190+0000","last_became_peered":"2024-05-30T16:54:56.652190+0000","last_unstale":"2024-05-30T16:54:57.744234+0000","last_undegraded":"2024-05-30T16:54:57.744234+0000","last_fullsized":"2024-05-30T16:54:57.744234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:02.366944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743407+0000","last_change":"2024-05-30T16:54:55.623039+0000","last_active":"2024-05-30T16:54:57.743407+0000","last_peered":"2024-05-30T16:54:57.743407+0000","last_clean":"2024-05-30T16:54:57.743407+0000","last_became_active":"2024-05-30T16:54:55.622700+0000","last_became_peered":"2024-05-30T16:54:55.622700+0000","last_unstale":"2024-05-30T16:54:57.743407+0000","last_undegraded":"2024-05-30T16:54:57.743407+0000","last_fullsized":"2024-05-30T16:54:57.743407+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:35:40.493510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653060+0000","last_change":"2024-05-30T16:54:56.662477+0000","last_active":"2024-05-30T16:54:57.653060+0000","last_peered":"2024-05-30T16:54:57.653060+0000","last_clean":"2024-05-30T16:54:57.653060+0000","last_became_active":"2024-05-30T16:54:56.661843+0000","last_became_peered":"2024-05-30T16:54:56.661843+0000","last_unstale":"2024-05-30T16:54:57.653060+0000","last_undegraded":"2024-05-30T16:54:57.653060+0000","last_fullsized":"2024-05-30T16:54:57.653060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:08:13.510748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746053+0000","last_change":"2024-05-30T16:54:55.631759+0000","last_active":"2024-05-30T16:54:57.746053+0000","last_peered":"2024-05-30T16:54:57.746053+0000","last_clean":"2024-05-30T16:54:57.746053+0000","last_became_active":"2024-05-30T16:54:55.630908+0000","last_became_peered":"2024-05-30T16:54:55.630908+0000","last_unstale":"2024-05-30T16:54:57.746053+0000","last_undegraded":"2024-05-30T16:54:57.746053+0000","last_fullsized":"2024-05-30T16:54:57.746053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:20.115543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747203+0000","last_change":"2024-05-30T16:54:56.655912+0000","last_active":"2024-05-30T16:54:57.747203+0000","last_peered":"2024-05-30T16:54:57.747203+0000","last_clean":"2024-05-30T16:54:57.747203+0000","last_became_active":"2024-05-30T16:54:56.655496+0000","last_became_peered":"2024-05-30T16:54:56.655496+0000","last_unstale":"2024-05-30T16:54:57.747203+0000","last_undegraded":"2024-05-30T16:54:57.747203+0000","last_fullsized":"2024-05-30T16:54:57.747203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:39.899066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653179+0000","last_change":"2024-05-30T16:54:55.623100+0000","last_active":"2024-05-30T16:54:57.653179+0000","last_peered":"2024-05-30T16:54:57.653179+0000","last_clean":"2024-05-30T16:54:57.653179+0000","last_became_active":"2024-05-30T16:54:55.622544+0000","last_became_peered":"2024-05-30T16:54:55.622544+0000","last_unstale":"2024-05-30T16:54:57.653179+0000","last_undegraded":"2024-05-30T16:54:57.653179+0000","last_fullsized":"2024-05-30T16:54:57.653179+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:50.348074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653629+0000","last_change":"2024-05-30T16:54:56.651407+0000","last_active":"2024-05-30T16:54:57.653629+0000","last_peered":"2024-05-30T16:54:57.653629+0000","last_clean":"2024-05-30T16:54:57.653629+0000","last_became_active":"2024-05-30T16:54:56.650844+0000","last_became_peered":"2024-05-30T16:54:56.650844+0000","last_unstale":"2024-05-30T16:54:57.653629+0000","last_undegraded":"2024-05-30T16:54:57.653629+0000","last_fullsized":"2024-05-30T16:54:57.653629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:29.380010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745262+0000","last_change":"2024-05-30T16:54:55.618125+0000","last_active":"2024-05-30T16:54:57.745262+0000","last_peered":"2024-05-30T16:54:57.745262+0000","last_clean":"2024-05-30T16:54:57.745262+0000","last_became_active":"2024-05-30T16:54:55.617577+0000","last_became_peered":"2024-05-30T16:54:55.617577+0000","last_unstale":"2024-05-30T16:54:57.745262+0000","last_undegraded":"2024-05-30T16:54:57.745262+0000","last_fullsized":"2024-05-30T16:54:57.745262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:43:18.083619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747136+0000","last_change":"2024-05-30T16:54:56.657326+0000","last_active":"2024-05-30T16:54:57.747136+0000","last_peered":"2024-05-30T16:54:57.747136+0000","last_clean":"2024-05-30T16:54:57.747136+0000","last_became_active":"2024-05-30T16:54:56.656754+0000","last_became_peered":"2024-05-30T16:54:56.656754+0000","last_unstale":"2024-05-30T16:54:57.747136+0000","last_undegraded":"2024-05-30T16:54:57.747136+0000","last_fullsized":"2024-05-30T16:54:57.747136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:13.200522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652801+0000","last_change":"2024-05-30T16:54:55.626901+0000","last_active":"2024-05-30T16:54:57.652801+0000","last_peered":"2024-05-30T16:54:57.652801+0000","last_clean":"2024-05-30T16:54:57.652801+0000","last_became_active":"2024-05-30T16:54:55.626578+0000","last_became_peered":"2024-05-30T16:54:55.626578+0000","last_unstale":"2024-05-30T16:54:57.652801+0000","last_undegraded":"2024-05-30T16:54:57.652801+0000","last_fullsized":"2024-05-30T16:54:57.652801+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:01.357036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.742652+0000","last_change":"2024-05-30T16:54:56.659662+0000","last_active":"2024-05-30T16:54:57.742652+0000","last_peered":"2024-05-30T16:54:57.742652+0000","last_clean":"2024-05-30T16:54:57.742652+0000","last_became_active":"2024-05-30T16:54:56.657971+0000","last_became_peered":"2024-05-30T16:54:56.657971+0000","last_unstale":"2024-05-30T16:54:57.742652+0000","last_undegraded":"2024-05-30T16:54:57.742652+0000","last_fullsized":"2024-05-30T16:54:57.742652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:31.815811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649175+0000","last_change":"2024-05-30T16:54:55.629842+0000","last_active":"2024-05-30T16:54:57.649175+0000","last_peered":"2024-05-30T16:54:57.649175+0000","last_clean":"2024-05-30T16:54:57.649175+0000","last_became_active":"2024-05-30T16:54:55.629495+0000","last_became_peered":"2024-05-30T16:54:55.629495+0000","last_unstale":"2024-05-30T16:54:57.649175+0000","last_undegraded":"2024-05-30T16:54:57.649175+0000","last_fullsized":"2024-05-30T16:54:57.649175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:54.016165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743799+0000","last_change":"2024-05-30T16:54:56.656743+0000","last_active":"2024-05-30T16:54:57.743799+0000","last_peered":"2024-05-30T16:54:57.743799+0000","last_clean":"2024-05-30T16:54:57.743799+0000","last_became_active":"2024-05-30T16:54:56.656476+0000","last_became_peered":"2024-05-30T16:54:56.656476+0000","last_unstale":"2024-05-30T16:54:57.743799+0000","last_undegraded":"2024-05-30T16:54:57.743799+0000","last_fullsized":"2024-05-30T16:54:57.743799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:07.005681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.742573+0000","last_change":"2024-05-30T16:54:55.611884+0000","last_active":"2024-05-30T16:54:57.742573+0000","last_peered":"2024-05-30T16:54:57.742573+0000","last_clean":"2024-05-30T16:54:57.742573+0000","last_became_active":"2024-05-30T16:54:55.611572+0000","last_became_peered":"2024-05-30T16:54:55.611572+0000","last_unstale":"2024-05-30T16:54:57.742573+0000","last_undegraded":"2024-05-30T16:54:57.742573+0000","last_fullsized":"2024-05-30T16:54:57.742573+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:24.338448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654704+0000","last_change":"2024-05-30T16:54:56.653034+0000","last_active":"2024-05-30T16:54:57.654704+0000","last_peered":"2024-05-30T16:54:57.654704+0000","last_clean":"2024-05-30T16:54:57.654704+0000","last_became_active":"2024-05-30T16:54:56.652759+0000","last_became_peered":"2024-05-30T16:54:56.652759+0000","last_unstale":"2024-05-30T16:54:57.654704+0000","last_undegraded":"2024-05-30T16:54:57.654704+0000","last_fullsized":"2024-05-30T16:54:57.654704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:43:19.421505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746377+0000","last_change":"2024-05-30T16:54:55.624791+0000","last_active":"2024-05-30T16:54:57.746377+0000","last_peered":"2024-05-30T16:54:57.746377+0000","last_clean":"2024-05-30T16:54:57.746377+0000","last_became_active":"2024-05-30T16:54:55.621959+0000","last_became_peered":"2024-05-30T16:54:55.621959+0000","last_unstale":"2024-05-30T16:54:57.746377+0000","last_undegraded":"2024-05-30T16:54:57.746377+0000","last_fullsized":"2024-05-30T16:54:57.746377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:44.846266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747766+0000","last_change":"2024-05-30T16:54:56.667857+0000","last_active":"2024-05-30T16:54:57.747766+0000","last_peered":"2024-05-30T16:54:57.747766+0000","last_clean":"2024-05-30T16:54:57.747766+0000","last_became_active":"2024-05-30T16:54:56.667272+0000","last_became_peered":"2024-05-30T16:54:56.667272+0000","last_unstale":"2024-05-30T16:54:57.747766+0000","last_undegraded":"2024-05-30T16:54:57.747766+0000","last_fullsized":"2024-05-30T16:54:57.747766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:30.039187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654433+0000","last_change":"2024-05-30T16:54:55.627531+0000","last_active":"2024-05-30T16:54:57.654433+0000","last_peered":"2024-05-30T16:54:57.654433+0000","last_clean":"2024-05-30T16:54:57.654433+0000","last_became_active":"2024-05-30T16:54:55.627166+0000","last_became_peered":"2024-05-30T16:54:55.627166+0000","last_unstale":"2024-05-30T16:54:57.654433+0000","last_undegraded":"2024-05-30T16:54:57.654433+0000","last_fullsized":"2024-05-30T16:54:57.654433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:25.538140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653312+0000","last_change":"2024-05-30T16:54:56.658827+0000","last_active":"2024-05-30T16:54:57.653312+0000","last_peered":"2024-05-30T16:54:57.653312+0000","last_clean":"2024-05-30T16:54:57.653312+0000","last_became_active":"2024-05-30T16:54:56.658375+0000","last_became_peered":"2024-05-30T16:54:56.658375+0000","last_unstale":"2024-05-30T16:54:57.653312+0000","last_undegraded":"2024-05-30T16:54:57.653312+0000","last_fullsized":"2024-05-30T16:54:57.653312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:41.464064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745259+0000","last_change":"2024-05-30T16:54:55.615825+0000","last_active":"2024-05-30T16:54:57.745259+0000","last_peered":"2024-05-30T16:54:57.745259+0000","last_clean":"2024-05-30T16:54:57.745259+0000","last_became_active":"2024-05-30T16:54:55.615172+0000","last_became_peered":"2024-05-30T16:54:55.615172+0000","last_unstale":"2024-05-30T16:54:57.745259+0000","last_undegraded":"2024-05-30T16:54:57.745259+0000","last_fullsized":"2024-05-30T16:54:57.745259+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:25:13.012127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743833+0000","last_change":"2024-05-30T16:54:56.649822+0000","last_active":"2024-05-30T16:54:57.743833+0000","last_peered":"2024-05-30T16:54:57.743833+0000","last_clean":"2024-05-30T16:54:57.743833+0000","last_became_active":"2024-05-30T16:54:56.647069+0000","last_became_peered":"2024-05-30T16:54:56.647069+0000","last_unstale":"2024-05-30T16:54:57.743833+0000","last_undegraded":"2024-05-30T16:54:57.743833+0000","last_fullsized":"2024-05-30T16:54:57.743833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:33.417787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649953+0000","last_change":"2024-05-30T16:54:55.613399+0000","last_active":"2024-05-30T16:54:57.649953+0000","last_peered":"2024-05-30T16:54:57.649953+0000","last_clean":"2024-05-30T16:54:57.649953+0000","last_became_active":"2024-05-30T16:54:55.612527+0000","last_became_peered":"2024-05-30T16:54:55.612527+0000","last_unstale":"2024-05-30T16:54:57.649953+0000","last_undegraded":"2024-05-30T16:54:57.649953+0000","last_fullsized":"2024-05-30T16:54:57.649953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:50:59.288715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745817+0000","last_change":"2024-05-30T16:54:56.654144+0000","last_active":"2024-05-30T16:54:57.745817+0000","last_peered":"2024-05-30T16:54:57.745817+0000","last_clean":"2024-05-30T16:54:57.745817+0000","last_became_active":"2024-05-30T16:54:56.653861+0000","last_became_peered":"2024-05-30T16:54:56.653861+0000","last_unstale":"2024-05-30T16:54:57.745817+0000","last_undegraded":"2024-05-30T16:54:57.745817+0000","last_fullsized":"2024-05-30T16:54:57.745817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:14:01.523291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649787+0000","last_change":"2024-05-30T16:54:55.624179+0000","last_active":"2024-05-30T16:54:57.649787+0000","last_peered":"2024-05-30T16:54:57.649787+0000","last_clean":"2024-05-30T16:54:57.649787+0000","last_became_active":"2024-05-30T16:54:55.623735+0000","last_became_peered":"2024-05-30T16:54:55.623735+0000","last_unstale":"2024-05-30T16:54:57.649787+0000","last_undegraded":"2024-05-30T16:54:57.649787+0000","last_fullsized":"2024-05-30T16:54:57.649787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:30.805904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654332+0000","last_change":"2024-05-30T16:54:56.656732+0000","last_active":"2024-05-30T16:54:57.654332+0000","last_peered":"2024-05-30T16:54:57.654332+0000","last_clean":"2024-05-30T16:54:57.654332+0000","last_became_active":"2024-05-30T16:54:56.656329+0000","last_became_peered":"2024-05-30T16:54:56.656329+0000","last_unstale":"2024-05-30T16:54:57.654332+0000","last_undegraded":"2024-05-30T16:54:57.654332+0000","last_fullsized":"2024-05-30T16:54:57.654332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:57.331070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653086+0000","last_change":"2024-05-30T16:54:55.627550+0000","last_active":"2024-05-30T16:54:57.653086+0000","last_peered":"2024-05-30T16:54:57.653086+0000","last_clean":"2024-05-30T16:54:57.653086+0000","last_became_active":"2024-05-30T16:54:55.627217+0000","last_became_peered":"2024-05-30T16:54:55.627217+0000","last_unstale":"2024-05-30T16:54:57.653086+0000","last_undegraded":"2024-05-30T16:54:57.653086+0000","last_fullsized":"2024-05-30T16:54:57.653086+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:43:36.263157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743028+0000","last_change":"2024-05-30T16:54:56.651514+0000","last_active":"2024-05-30T16:54:57.743028+0000","last_peered":"2024-05-30T16:54:57.743028+0000","last_clean":"2024-05-30T16:54:57.743028+0000","last_became_active":"2024-05-30T16:54:56.650990+0000","last_became_peered":"2024-05-30T16:54:56.650990+0000","last_unstale":"2024-05-30T16:54:57.743028+0000","last_undegraded":"2024-05-30T16:54:57.743028+0000","last_fullsized":"2024-05-30T16:54:57.743028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:17.578200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652897+0000","last_change":"2024-05-30T16:54:55.619882+0000","last_active":"2024-05-30T16:54:57.652897+0000","last_peered":"2024-05-30T16:54:57.652897+0000","last_clean":"2024-05-30T16:54:57.652897+0000","last_became_active":"2024-05-30T16:54:55.619553+0000","last_became_peered":"2024-05-30T16:54:55.619553+0000","last_unstale":"2024-05-30T16:54:57.652897+0000","last_undegraded":"2024-05-30T16:54:57.652897+0000","last_fullsized":"2024-05-30T16:54:57.652897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:00:49.949175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.742806+0000","last_change":"2024-05-30T16:54:56.648216+0000","last_active":"2024-05-30T16:54:57.742806+0000","last_peered":"2024-05-30T16:54:57.742806+0000","last_clean":"2024-05-30T16:54:57.742806+0000","last_became_active":"2024-05-30T16:54:56.647745+0000","last_became_peered":"2024-05-30T16:54:56.647745+0000","last_unstale":"2024-05-30T16:54:57.742806+0000","last_undegraded":"2024-05-30T16:54:57.742806+0000","last_fullsized":"2024-05-30T16:54:57.742806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:51:00.298622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744979+0000","last_change":"2024-05-30T16:54:55.629445+0000","last_active":"2024-05-30T16:54:57.744979+0000","last_peered":"2024-05-30T16:54:57.744979+0000","last_clean":"2024-05-30T16:54:57.744979+0000","last_became_active":"2024-05-30T16:54:55.627481+0000","last_became_peered":"2024-05-30T16:54:55.627481+0000","last_unstale":"2024-05-30T16:54:57.744979+0000","last_undegraded":"2024-05-30T16:54:57.744979+0000","last_fullsized":"2024-05-30T16:54:57.744979+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:08.663109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747164+0000","last_change":"2024-05-30T16:54:56.652717+0000","last_active":"2024-05-30T16:54:57.747164+0000","last_peered":"2024-05-30T16:54:57.747164+0000","last_clean":"2024-05-30T16:54:57.747164+0000","last_became_active":"2024-05-30T16:54:56.651285+0000","last_became_peered":"2024-05-30T16:54:56.651285+0000","last_unstale":"2024-05-30T16:54:57.747164+0000","last_undegraded":"2024-05-30T16:54:57.747164+0000","last_fullsized":"2024-05-30T16:54:57.747164+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:23:19.300945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654749+0000","last_change":"2024-05-30T16:54:55.626454+0000","last_active":"2024-05-30T16:54:57.654749+0000","last_peered":"2024-05-30T16:54:57.654749+0000","last_clean":"2024-05-30T16:54:57.654749+0000","last_became_active":"2024-05-30T16:54:55.625789+0000","last_became_peered":"2024-05-30T16:54:55.625789+0000","last_unstale":"2024-05-30T16:54:57.654749+0000","last_undegraded":"2024-05-30T16:54:57.654749+0000","last_fullsized":"2024-05-30T16:54:57.654749+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:08.663109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747984+0000","last_change":"2024-05-30T16:54:56.663267+0000","last_active":"2024-05-30T16:54:57.747984+0000","last_peered":"2024-05-30T16:54:57.747984+0000","last_clean":"2024-05-30T16:54:57.747984+0000","last_became_active":"2024-05-30T16:54:56.662116+0000","last_became_peered":"2024-05-30T16:54:56.662116+0000","last_unstale":"2024-05-30T16:54:57.747984+0000","last_undegraded":"2024-05-30T16:54:57.747984+0000","last_fullsized":"2024-05-30T16:54:57.747984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:44:21.798658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747315+0000","last_change":"2024-05-30T16:54:55.631123+0000","last_active":"2024-05-30T16:54:57.747315+0000","last_peered":"2024-05-30T16:54:57.747315+0000","last_clean":"2024-05-30T16:54:57.747315+0000","last_became_active":"2024-05-30T16:54:55.630836+0000","last_became_peered":"2024-05-30T16:54:55.630836+0000","last_unstale":"2024-05-30T16:54:57.747315+0000","last_undegraded":"2024-05-30T16:54:57.747315+0000","last_fullsized":"2024-05-30T16:54:57.747315+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:22.133301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649867+0000","last_change":"2024-05-30T16:54:56.653431+0000","last_active":"2024-05-30T16:54:57.649867+0000","last_peered":"2024-05-30T16:54:57.649867+0000","last_clean":"2024-05-30T16:54:57.649867+0000","last_became_active":"2024-05-30T16:54:56.652792+0000","last_became_peered":"2024-05-30T16:54:56.652792+0000","last_unstale":"2024-05-30T16:54:57.649867+0000","last_undegraded":"2024-05-30T16:54:57.649867+0000","last_fullsized":"2024-05-30T16:54:57.649867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:43:37.273065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654112+0000","last_change":"2024-05-30T16:54:55.630437+0000","last_active":"2024-05-30T16:54:57.654112+0000","last_peered":"2024-05-30T16:54:57.654112+0000","last_clean":"2024-05-30T16:54:57.654112+0000","last_became_active":"2024-05-30T16:54:55.630081+0000","last_became_peered":"2024-05-30T16:54:55.630081+0000","last_unstale":"2024-05-30T16:54:57.654112+0000","last_undegraded":"2024-05-30T16:54:57.654112+0000","last_fullsized":"2024-05-30T16:54:57.654112+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:44.846266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743943+0000","last_change":"2024-05-30T16:54:56.649578+0000","last_active":"2024-05-30T16:54:57.743943+0000","last_peered":"2024-05-30T16:54:57.743943+0000","last_clean":"2024-05-30T16:54:57.743943+0000","last_became_active":"2024-05-30T16:54:56.646637+0000","last_became_peered":"2024-05-30T16:54:56.646637+0000","last_unstale":"2024-05-30T16:54:57.743943+0000","last_undegraded":"2024-05-30T16:54:57.743943+0000","last_fullsized":"2024-05-30T16:54:57.743943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:10.287840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747299+0000","last_change":"2024-05-30T16:54:55.632199+0000","last_active":"2024-05-30T16:54:57.747299+0000","last_peered":"2024-05-30T16:54:57.747299+0000","last_clean":"2024-05-30T16:54:57.747299+0000","last_became_active":"2024-05-30T16:54:55.631811+0000","last_became_peered":"2024-05-30T16:54:55.631811+0000","last_unstale":"2024-05-30T16:54:57.747299+0000","last_undegraded":"2024-05-30T16:54:57.747299+0000","last_fullsized":"2024-05-30T16:54:57.747299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:28.370103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745262+0000","last_change":"2024-05-30T16:54:56.650664+0000","last_active":"2024-05-30T16:54:57.745262+0000","last_peered":"2024-05-30T16:54:57.745262+0000","last_clean":"2024-05-30T16:54:57.745262+0000","last_became_active":"2024-05-30T16:54:56.649814+0000","last_became_peered":"2024-05-30T16:54:56.649814+0000","last_unstale":"2024-05-30T16:54:57.745262+0000","last_undegraded":"2024-05-30T16:54:57.745262+0000","last_fullsized":"2024-05-30T16:54:57.745262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:19:48.958217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653365+0000","last_change":"2024-05-30T16:54:55.625995+0000","last_active":"2024-05-30T16:54:57.653365+0000","last_peered":"2024-05-30T16:54:57.653365+0000","last_clean":"2024-05-30T16:54:57.653365+0000","last_became_active":"2024-05-30T16:54:55.625720+0000","last_became_peered":"2024-05-30T16:54:55.625720+0000","last_unstale":"2024-05-30T16:54:57.653365+0000","last_undegraded":"2024-05-30T16:54:57.653365+0000","last_fullsized":"2024-05-30T16:54:57.653365+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:03:11.650715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744998+0000","last_change":"2024-05-30T16:54:56.654539+0000","last_active":"2024-05-30T16:54:57.744998+0000","last_peered":"2024-05-30T16:54:57.744998+0000","last_clean":"2024-05-30T16:54:57.744998+0000","last_became_active":"2024-05-30T16:54:56.653412+0000","last_became_peered":"2024-05-30T16:54:56.653412+0000","last_unstale":"2024-05-30T16:54:57.744998+0000","last_undegraded":"2024-05-30T16:54:57.744998+0000","last_fullsized":"2024-05-30T16:54:57.744998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:33:39.658399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653206+0000","last_change":"2024-05-30T16:54:55.621183+0000","last_active":"2024-05-30T16:54:57.653206+0000","last_peered":"2024-05-30T16:54:57.653206+0000","last_clean":"2024-05-30T16:54:57.653206+0000","last_became_active":"2024-05-30T16:54:55.620893+0000","last_became_peered":"2024-05-30T16:54:55.620893+0000","last_unstale":"2024-05-30T16:54:57.653206+0000","last_undegraded":"2024-05-30T16:54:57.653206+0000","last_fullsized":"2024-05-30T16:54:57.653206+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:48.476553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744974+0000","last_change":"2024-05-30T16:54:56.664758+0000","last_active":"2024-05-30T16:54:57.744974+0000","last_peered":"2024-05-30T16:54:57.744974+0000","last_clean":"2024-05-30T16:54:57.744974+0000","last_became_active":"2024-05-30T16:54:56.664324+0000","last_became_peered":"2024-05-30T16:54:56.664324+0000","last_unstale":"2024-05-30T16:54:57.744974+0000","last_undegraded":"2024-05-30T16:54:57.744974+0000","last_fullsized":"2024-05-30T16:54:57.744974+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:14:46.367921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.742706+0000","last_change":"2024-05-30T16:54:55.623249+0000","last_active":"2024-05-30T16:54:57.742706+0000","last_peered":"2024-05-30T16:54:57.742706+0000","last_clean":"2024-05-30T16:54:57.742706+0000","last_became_active":"2024-05-30T16:54:55.622929+0000","last_became_peered":"2024-05-30T16:54:55.622929+0000","last_unstale":"2024-05-30T16:54:57.742706+0000","last_undegraded":"2024-05-30T16:54:57.742706+0000","last_fullsized":"2024-05-30T16:54:57.742706+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:58.271816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743340+0000","last_change":"2024-05-30T16:54:56.655795+0000","last_active":"2024-05-30T16:54:57.743340+0000","last_peered":"2024-05-30T16:54:57.743340+0000","last_clean":"2024-05-30T16:54:57.743340+0000","last_became_active":"2024-05-30T16:54:56.655488+0000","last_became_peered":"2024-05-30T16:54:56.655488+0000","last_unstale":"2024-05-30T16:54:57.743340+0000","last_undegraded":"2024-05-30T16:54:57.743340+0000","last_fullsized":"2024-05-30T16:54:57.743340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:33.255023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654775+0000","last_change":"2024-05-30T16:54:55.626336+0000","last_active":"2024-05-30T16:54:57.654775+0000","last_peered":"2024-05-30T16:54:57.654775+0000","last_clean":"2024-05-30T16:54:57.654775+0000","last_became_active":"2024-05-30T16:54:55.625166+0000","last_became_peered":"2024-05-30T16:54:55.625166+0000","last_unstale":"2024-05-30T16:54:57.654775+0000","last_undegraded":"2024-05-30T16:54:57.654775+0000","last_fullsized":"2024-05-30T16:54:57.654775+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:20.115543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654604+0000","last_change":"2024-05-30T16:54:56.643257+0000","last_active":"2024-05-30T16:54:57.654604+0000","last_peered":"2024-05-30T16:54:57.654604+0000","last_clean":"2024-05-30T16:54:57.654604+0000","last_became_active":"2024-05-30T16:54:56.642794+0000","last_became_peered":"2024-05-30T16:54:56.642794+0000","last_unstale":"2024-05-30T16:54:57.654604+0000","last_undegraded":"2024-05-30T16:54:57.654604+0000","last_fullsized":"2024-05-30T16:54:57.654604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:17:13.411216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654988+0000","last_change":"2024-05-30T16:54:55.623374+0000","last_active":"2024-05-30T16:54:57.654988+0000","last_peered":"2024-05-30T16:54:57.654988+0000","last_clean":"2024-05-30T16:54:57.654988+0000","last_became_active":"2024-05-30T16:54:55.623068+0000","last_became_peered":"2024-05-30T16:54:55.623068+0000","last_unstale":"2024-05-30T16:54:57.654988+0000","last_undegraded":"2024-05-30T16:54:57.654988+0000","last_fullsized":"2024-05-30T16:54:57.654988+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:11.720295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747185+0000","last_change":"2024-05-30T16:54:56.666080+0000","last_active":"2024-05-30T16:54:57.747185+0000","last_peered":"2024-05-30T16:54:57.747185+0000","last_clean":"2024-05-30T16:54:57.747185+0000","last_became_active":"2024-05-30T16:54:56.665788+0000","last_became_peered":"2024-05-30T16:54:56.665788+0000","last_unstale":"2024-05-30T16:54:57.747185+0000","last_undegraded":"2024-05-30T16:54:57.747185+0000","last_fullsized":"2024-05-30T16:54:57.747185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:00.934684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743910+0000","last_change":"2024-05-30T16:54:55.622679+0000","last_active":"2024-05-30T16:54:57.743910+0000","last_peered":"2024-05-30T16:54:57.743910+0000","last_clean":"2024-05-30T16:54:57.743910+0000","last_became_active":"2024-05-30T16:54:55.622295+0000","last_became_peered":"2024-05-30T16:54:55.622295+0000","last_unstale":"2024-05-30T16:54:57.743910+0000","last_undegraded":"2024-05-30T16:54:57.743910+0000","last_fullsized":"2024-05-30T16:54:57.743910+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:28.956662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747598+0000","last_change":"2024-05-30T16:54:56.655813+0000","last_active":"2024-05-30T16:54:57.747598+0000","last_peered":"2024-05-30T16:54:57.747598+0000","last_clean":"2024-05-30T16:54:57.747598+0000","last_became_active":"2024-05-30T16:54:56.654959+0000","last_became_peered":"2024-05-30T16:54:56.654959+0000","last_unstale":"2024-05-30T16:54:57.747598+0000","last_undegraded":"2024-05-30T16:54:57.747598+0000","last_fullsized":"2024-05-30T16:54:57.747598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.799865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743406+0000","last_change":"2024-05-30T16:54:55.629963+0000","last_active":"2024-05-30T16:54:57.743406+0000","last_peered":"2024-05-30T16:54:57.743406+0000","last_clean":"2024-05-30T16:54:57.743406+0000","last_became_active":"2024-05-30T16:54:55.629607+0000","last_became_peered":"2024-05-30T16:54:55.629607+0000","last_unstale":"2024-05-30T16:54:57.743406+0000","last_undegraded":"2024-05-30T16:54:57.743406+0000","last_fullsized":"2024-05-30T16:54:57.743406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:28.370103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743716+0000","last_change":"2024-05-30T16:54:56.654299+0000","last_active":"2024-05-30T16:54:57.743716+0000","last_peered":"2024-05-30T16:54:57.743716+0000","last_clean":"2024-05-30T16:54:57.743716+0000","last_became_active":"2024-05-30T16:54:56.652304+0000","last_became_peered":"2024-05-30T16:54:56.652304+0000","last_unstale":"2024-05-30T16:54:57.743716+0000","last_undegraded":"2024-05-30T16:54:57.743716+0000","last_fullsized":"2024-05-30T16:54:57.743716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:02.453084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743490+0000","last_change":"2024-05-30T16:54:55.611765+0000","last_active":"2024-05-30T16:54:57.743490+0000","last_peered":"2024-05-30T16:54:57.743490+0000","last_clean":"2024-05-30T16:54:57.743490+0000","last_became_active":"2024-05-30T16:54:55.611426+0000","last_became_peered":"2024-05-30T16:54:55.611426+0000","last_unstale":"2024-05-30T16:54:57.743490+0000","last_undegraded":"2024-05-30T16:54:57.743490+0000","last_fullsized":"2024-05-30T16:54:57.743490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:47:13.879124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652216+0000","last_change":"2024-05-30T16:54:56.656597+0000","last_active":"2024-05-30T16:54:57.652216+0000","last_peered":"2024-05-30T16:54:57.652216+0000","last_clean":"2024-05-30T16:54:57.652216+0000","last_became_active":"2024-05-30T16:54:56.656321+0000","last_became_peered":"2024-05-30T16:54:56.656321+0000","last_unstale":"2024-05-30T16:54:57.652216+0000","last_undegraded":"2024-05-30T16:54:57.652216+0000","last_fullsized":"2024-05-30T16:54:57.652216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:08.792991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649028+0000","last_change":"2024-05-30T16:54:55.621069+0000","last_active":"2024-05-30T16:54:57.649028+0000","last_peered":"2024-05-30T16:54:57.649028+0000","last_clean":"2024-05-30T16:54:57.649028+0000","last_became_active":"2024-05-30T16:54:55.620559+0000","last_became_peered":"2024-05-30T16:54:55.620559+0000","last_unstale":"2024-05-30T16:54:57.649028+0000","last_undegraded":"2024-05-30T16:54:57.649028+0000","last_fullsized":"2024-05-30T16:54:57.649028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:06:35.918541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649896+0000","last_change":"2024-05-30T16:54:56.648552+0000","last_active":"2024-05-30T16:54:57.649896+0000","last_peered":"2024-05-30T16:54:57.649896+0000","last_clean":"2024-05-30T16:54:57.649896+0000","last_became_active":"2024-05-30T16:54:56.648264+0000","last_became_peered":"2024-05-30T16:54:56.648264+0000","last_unstale":"2024-05-30T16:54:57.649896+0000","last_undegraded":"2024-05-30T16:54:57.649896+0000","last_fullsized":"2024-05-30T16:54:57.649896+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:03:12.660623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744835+0000","last_change":"2024-05-30T16:54:55.629713+0000","last_active":"2024-05-30T16:54:57.744835+0000","last_peered":"2024-05-30T16:54:57.744835+0000","last_clean":"2024-05-30T16:54:57.744835+0000","last_became_active":"2024-05-30T16:54:55.629182+0000","last_became_peered":"2024-05-30T16:54:55.629182+0000","last_unstale":"2024-05-30T16:54:57.744835+0000","last_undegraded":"2024-05-30T16:54:57.744835+0000","last_fullsized":"2024-05-30T16:54:57.744835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:20:43.048793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.651562+0000","last_change":"2024-05-30T16:54:56.662049+0000","last_active":"2024-05-30T16:54:57.651562+0000","last_peered":"2024-05-30T16:54:57.651562+0000","last_clean":"2024-05-30T16:54:57.651562+0000","last_became_active":"2024-05-30T16:54:56.661686+0000","last_became_peered":"2024-05-30T16:54:56.661686+0000","last_unstale":"2024-05-30T16:54:57.651562+0000","last_undegraded":"2024-05-30T16:54:57.651562+0000","last_fullsized":"2024-05-30T16:54:57.651562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:28.212134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.651696+0000","last_change":"2024-05-30T16:54:55.623173+0000","last_active":"2024-05-30T16:54:57.651696+0000","last_peered":"2024-05-30T16:54:57.651696+0000","last_clean":"2024-05-30T16:54:57.651696+0000","last_became_active":"2024-05-30T16:54:55.622874+0000","last_became_peered":"2024-05-30T16:54:55.622874+0000","last_unstale":"2024-05-30T16:54:57.651696+0000","last_undegraded":"2024-05-30T16:54:57.651696+0000","last_fullsized":"2024-05-30T16:54:57.651696+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:53:36.725881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744975+0000","last_change":"2024-05-30T16:54:56.666000+0000","last_active":"2024-05-30T16:54:57.744975+0000","last_peered":"2024-05-30T16:54:57.744975+0000","last_clean":"2024-05-30T16:54:57.744975+0000","last_became_active":"2024-05-30T16:54:56.665688+0000","last_became_peered":"2024-05-30T16:54:56.665688+0000","last_unstale":"2024-05-30T16:54:57.744975+0000","last_undegraded":"2024-05-30T16:54:57.744975+0000","last_fullsized":"2024-05-30T16:54:57.744975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:30.857494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652351+0000","last_change":"2024-05-30T16:54:55.620733+0000","last_active":"2024-05-30T16:54:57.652351+0000","last_peered":"2024-05-30T16:54:57.652351+0000","last_clean":"2024-05-30T16:54:57.652351+0000","last_became_active":"2024-05-30T16:54:55.620460+0000","last_became_peered":"2024-05-30T16:54:55.620460+0000","last_unstale":"2024-05-30T16:54:57.652351+0000","last_undegraded":"2024-05-30T16:54:57.652351+0000","last_fullsized":"2024-05-30T16:54:57.652351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:23:27.426633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":272,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746749+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:54:57.746749+0000","last_peered":"2024-05-30T16:54:57.746749+0000","last_clean":"2024-05-30T16:54:57.746749+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:54:57.746749+0000","last_undegraded":"2024-05-30T16:54:57.746749+0000","last_fullsized":"2024-05-30T16:54:57.746749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509579,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54268,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317572,"statfs":{"total":96636764160,"available":96581193728,"internally_reserved":0,"allocated":901120,"data_stored":380847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509579,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58428,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313412,"statfs":{"total":96636764160,"available":96576933888,"internally_reserved":0,"allocated":897024,"data_stored":380825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58428,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313412,"statfs":{"total":96636764160,"available":96576933888,"internally_reserved":0,"allocated":901120,"data_stored":380867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509580,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50116,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321724,"statfs":{"total":96636764160,"available":96585445376,"internally_reserved":0,"allocated":905216,"data_stored":382367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58440,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313400,"statfs":{"total":96636764160,"available":96576921600,"internally_reserved":0,"allocated":913408,"data_stored":382983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542282,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54848,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316992,"statfs":{"total":96636764160,"available":96580599808,"internally_reserved":0,"allocated":1495040,"data_stored":971739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542283,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54272,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317568,"statfs":{"total":96636764160,"available":96581189632,"internally_reserved":0,"allocated":897024,"data_stored":380845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59000,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312840,"statfs":{"total":96636764160,"available":96576348160,"internally_reserved":0,"allocated":1486848,"data_stored":971123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:55:01.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:55:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.385+0000 7f2606c00640 1 Processor -- start 2024-05-30T16:55:01.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.385+0000 7f2606c00640 1 -- start start 2024-05-30T16:55:01.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.387+0000 7f2606c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f260015e870 0x7f26000d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.387+0000 7f2606c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260015ec50 con 0x7f26000cfa40 2024-05-30T16:55:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.387+0000 7f2606c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26000cff80 con 0x7f26000d14d0 2024-05-30T16:55:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.387+0000 7f2606c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f26000d2150 con 0x7f260015e870 2024-05-30T16:55:01.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.387+0000 7f2606200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f26000cfa40 0x7f26000d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34722/0 (socket says 172.21.15.67:34722) 2024-05-30T16:55:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.388+0000 7f2606200640 1 -- 172.21.15.67:0/3226315124 learned_addr learned my addr 172.21.15.67:0/3226315124 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:55:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.388+0000 7f2605800640 1 --2- 172.21.15.67:0/3226315124 >> v2:172.21.15.169:3300/0 conn(0x7f260015e870 0x7f26000d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.388+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 670755227 0 0) 0x7f260015ec50 con 0x7f26000cfa40 2024-05-30T16:55:01.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.388+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25e8003520 con 0x7f26000cfa40 2024-05-30T16:55:01.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.388+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4212651302 0 0) 0x7f25e8003520 con 0x7f26000cfa40 2024-05-30T16:55:01.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.388+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 >> v2:172.21.15.169:3300/0 conn(0x7f260015e870 msgr2=0x7f26000d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:01.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.388+0000 7f25ffe00640 1 --2- 172.21.15.67:0/3226315124 >> v2:172.21.15.169:3300/0 conn(0x7f260015e870 0x7f26000d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 >> v1:172.21.15.169:6789/0 conn(0x7f26000d14d0 legacy=0x7f26000cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26000d0260 con 0x7f26000cfa40 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f25f80019f0 con 0x7f26000cfa40 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 >> v1:172.21.15.67:6789/0 conn(0x7f26000cfa40 legacy=0x7f26000d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e8003a00 0x7f25e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8004800 0x7f25e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25e80051c0 0x7f25e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26000d2150 con 0x7f25e8003a00 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25e800c0c0 con 0x7f25e8004800 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25e800c2c0 con 0x7f25e80051c0 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f2605800640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e8003a00 0x7f25e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f2604e00640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8004800 0x7f25e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f2605800640 1 -- 172.21.15.67:0/3226315124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25e80051c0 msgr2=0x7f25e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:01.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f2605800640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25e80051c0 0x7f25e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f2605800640 1 -- 172.21.15.67:0/3226315124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8004800 msgr2=0x7f25e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:01.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f2605800640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8004800 0x7f25e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.389+0000 7f2605800640 1 -- 172.21.15.67:0/3226315124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e8003520 con 0x7f25e8003a00 2024-05-30T16:55:01.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.390+0000 7f2604e00640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8004800 0x7f25e8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:01.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.390+0000 7f2606200640 1 -- 172.21.15.67:0/3226315124 reap_dead start 2024-05-30T16:55:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.391+0000 7f2605800640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e8003a00 0x7f25e8003e20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009ac0 tx=0x7f25e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fa62d598f5968d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.391+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4036070 con 0x7f25e8003a00 2024-05-30T16:55:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.391+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f4031400 con 0x7f25e8003a00 2024-05-30T16:55:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.391+0000 7f25ffe00640 1 -- 172.21.15.67:0/3226315124 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4009690 con 0x7f25e8003a00 2024-05-30T16:55:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.391+0000 7f2606c00640 1 -- 172.21.15.67:0/3226315124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e8003a00 msgr2=0x7f25e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.391+0000 7f2606c00640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e8003a00 0x7f25e8003e20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009ac0 tx=0x7f25e8005830 comp rx=0 tx=0).stop 2024-05-30T16:55:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.392+0000 7f2606c00640 1 -- 172.21.15.67:0/3226315124 shutdown_connections 2024-05-30T16:55:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.392+0000 7f2606c00640 1 --2- 172.21.15.67:0/3226315124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e8003a00 0x7f25e8003e20 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.392+0000 7f2606c00640 1 -- 172.21.15.67:0/3226315124 >> 172.21.15.67:0/3226315124 conn(0x7f260008bfc0 msgr2=0x7f26000d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:01.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.392+0000 7f2606c00640 1 -- 172.21.15.67:0/3226315124 shutdown_connections 2024-05-30T16:55:01.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.392+0000 7f2606c00640 1 -- 172.21.15.67:0/3226315124 wait complete. 2024-05-30T16:55:01.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.393+0000 7f2606c00640 1 Processor -- start 2024-05-30T16:55:01.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.394+0000 7f2606c00640 1 -- start start 2024-05-30T16:55:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.394+0000 7f2606c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8003a00 0x7f260015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.394+0000 7f2606c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26000c8ac0 0x7f260015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.394+0000 7f2606c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26000c8f20 0x7f26000c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26000d4fb0 con 0x7f26000c8f20 2024-05-30T16:55:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26000d0820 con 0x7f25e8003a00 2024-05-30T16:55:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26000d0f90 con 0x7f26000c8ac0 2024-05-30T16:55:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26000c8f20 0x7f26000c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26000c8f20 0x7f26000c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48152/0 (socket says 172.21.15.67:48152) 2024-05-30T16:55:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606200640 1 -- 172.21.15.67:0/3185773360 learned_addr learned my addr 172.21.15.67:0/3185773360 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2605800640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8003a00 0x7f260015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2604e00640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26000c8ac0 0x7f260015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606200640 1 -- 172.21.15.67:0/3185773360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26000c8ac0 msgr2=0x7f260015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606200640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26000c8ac0 0x7f260015b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606200640 1 -- 172.21.15.67:0/3185773360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8003a00 msgr2=0x7f260015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606200640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8003a00 0x7f260015d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.395+0000 7f2606200640 1 -- 172.21.15.67:0/3185773360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260015d680 con 0x7f26000c8f20 2024-05-30T16:55:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.396+0000 7f2606200640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26000c8f20 0x7f26000c9d50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f25f8001370 tx=0x7f25f800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:01.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.396+0000 7f25fd600640 1 -- 172.21.15.67:0/3185773360 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8004d60 con 0x7f26000c8f20 2024-05-30T16:55:01.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.396+0000 7f25fd600640 1 -- 172.21.15.67:0/3185773360 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f800a610 con 0x7f26000c8f20 2024-05-30T16:55:01.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.396+0000 7f25fd600640 1 -- 172.21.15.67:0/3185773360 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f800aa00 con 0x7f26000c8f20 2024-05-30T16:55:01.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.396+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26000b7340 con 0x7f26000c8f20 2024-05-30T16:55:01.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.396+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26000b7900 con 0x7f26000c8f20 2024-05-30T16:55:01.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.398+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260015e3f0 con 0x7f26000c8f20 2024-05-30T16:55:01.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.398+0000 7f25fd600640 1 -- 172.21.15.67:0/3185773360 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f25f8011070 con 0x7f26000c8f20 2024-05-30T16:55:01.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.399+0000 7f25fd600640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25e006d5e0 0x7f25e006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.400+0000 7f25fd600640 1 -- 172.21.15.67:0/3185773360 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f25f8089aa0 con 0x7f26000c8f20 2024-05-30T16:55:01.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.400+0000 7f2605800640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25e006d5e0 0x7f25e006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.400+0000 7f2605800640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25e006d5e0 0x7f25e006faa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f25f4033000 tx=0x7f25f4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:01.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.404+0000 7f25fd600640 1 -- 172.21.15.67:0/3185773360 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f260015e3f0 con 0x7f26000c8f20 2024-05-30T16:55:01.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.604+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26000b6840 con 0x7f25e006d5e0 2024-05-30T16:55:01.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.618+0000 7f25fd600640 1 -- 172.21.15.67:0/3185773360 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f26000b6840 con 0x7f25e006d5e0 2024-05-30T16:55:01.623 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:55:01.624 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:55:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25e006d5e0 msgr2=0x7f25e006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25e006d5e0 0x7f25e006faa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f25f4033000 tx=0x7f25f4008030 comp rx=0 tx=0).stop 2024-05-30T16:55:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26000c8f20 msgr2=0x7f26000c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26000c8f20 0x7f26000c9d50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f25f8001370 tx=0x7f25f800cb80 comp rx=0 tx=0).stop 2024-05-30T16:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 shutdown_connections 2024-05-30T16:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25e006d5e0 0x7f25e006faa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26000c8f20 0x7f26000c9d50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26000c8ac0 0x7f260015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 --2- 172.21.15.67:0/3185773360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e8003a00 0x7f260015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 >> 172.21.15.67:0/3185773360 conn(0x7f260008bfc0 msgr2=0x7f26000b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 shutdown_connections 2024-05-30T16:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.625+0000 7f2606c00640 1 -- 172.21.15.67:0/3185773360 wait complete. 2024-05-30T16:55:01.645 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-05-30T16:55:00.274360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":447800,"kb_used_data":8200,"kb_used_omap":12,"kb_used_meta":439475,"kb_avail":754526920,"statfs":{"total":773094113280,"available":772635566080,"internally_reserved":0,"allocated":8396800,"data_stored":4231596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":450022960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.691960"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746720+0000","last_change":"2024-05-30T16:54:56.668387+0000","last_active":"2024-05-30T16:54:57.746720+0000","last_peered":"2024-05-30T16:54:57.746720+0000","last_clean":"2024-05-30T16:54:57.746720+0000","last_became_active":"2024-05-30T16:54:56.668003+0000","last_became_peered":"2024-05-30T16:54:56.668003+0000","last_unstale":"2024-05-30T16:54:57.746720+0000","last_undegraded":"2024-05-30T16:54:57.746720+0000","last_fullsized":"2024-05-30T16:54:57.746720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:40:56.375503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654299+0000","last_change":"2024-05-30T16:54:56.658136+0000","last_active":"2024-05-30T16:54:57.654299+0000","last_peered":"2024-05-30T16:54:57.654299+0000","last_clean":"2024-05-30T16:54:57.654299+0000","last_became_active":"2024-05-30T16:54:56.656742+0000","last_became_peered":"2024-05-30T16:54:56.656742+0000","last_unstale":"2024-05-30T16:54:57.654299+0000","last_undegraded":"2024-05-30T16:54:57.654299+0000","last_fullsized":"2024-05-30T16:54:57.654299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:35:20.382926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745196+0000","last_change":"2024-05-30T16:54:56.653171+0000","last_active":"2024-05-30T16:54:57.745196+0000","last_peered":"2024-05-30T16:54:57.745196+0000","last_clean":"2024-05-30T16:54:57.745196+0000","last_became_active":"2024-05-30T16:54:56.651534+0000","last_became_peered":"2024-05-30T16:54:56.651534+0000","last_unstale":"2024-05-30T16:54:57.745196+0000","last_undegraded":"2024-05-30T16:54:57.745196+0000","last_fullsized":"2024-05-30T16:54:57.745196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:03.881908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649428+0000","last_change":"2024-05-30T16:54:56.661986+0000","last_active":"2024-05-30T16:54:57.649428+0000","last_peered":"2024-05-30T16:54:57.649428+0000","last_clean":"2024-05-30T16:54:57.649428+0000","last_became_active":"2024-05-30T16:54:56.661453+0000","last_became_peered":"2024-05-30T16:54:56.661453+0000","last_unstale":"2024-05-30T16:54:57.649428+0000","last_undegraded":"2024-05-30T16:54:57.649428+0000","last_fullsized":"2024-05-30T16:54:57.649428+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:53:37.735789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743511+0000","last_change":"2024-05-30T16:54:56.646452+0000","last_active":"2024-05-30T16:54:57.743511+0000","last_peered":"2024-05-30T16:54:57.743511+0000","last_clean":"2024-05-30T16:54:57.743511+0000","last_became_active":"2024-05-30T16:54:56.645189+0000","last_became_peered":"2024-05-30T16:54:56.645189+0000","last_unstale":"2024-05-30T16:54:57.743511+0000","last_undegraded":"2024-05-30T16:54:57.743511+0000","last_fullsized":"2024-05-30T16:54:57.743511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:56:25.427282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652631+0000","last_change":"2024-05-30T16:54:56.660046+0000","last_active":"2024-05-30T16:54:57.652631+0000","last_peered":"2024-05-30T16:54:57.652631+0000","last_clean":"2024-05-30T16:54:57.652631+0000","last_became_active":"2024-05-30T16:54:56.659780+0000","last_became_peered":"2024-05-30T16:54:56.659780+0000","last_unstale":"2024-05-30T16:54:57.652631+0000","last_undegraded":"2024-05-30T16:54:57.652631+0000","last_fullsized":"2024-05-30T16:54:57.652631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:38:11.617688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.651802+0000","last_change":"2024-05-30T16:54:56.637859+0000","last_active":"2024-05-30T16:54:57.651802+0000","last_peered":"2024-05-30T16:54:57.651802+0000","last_clean":"2024-05-30T16:54:57.651802+0000","last_became_active":"2024-05-30T16:54:56.636940+0000","last_became_peered":"2024-05-30T16:54:56.636940+0000","last_unstale":"2024-05-30T16:54:57.651802+0000","last_undegraded":"2024-05-30T16:54:57.651802+0000","last_fullsized":"2024-05-30T16:54:57.651802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:59.281724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652873+0000","last_change":"2024-05-30T16:54:56.648376+0000","last_active":"2024-05-30T16:54:57.652873+0000","last_peered":"2024-05-30T16:54:57.652873+0000","last_clean":"2024-05-30T16:54:57.652873+0000","last_became_active":"2024-05-30T16:54:56.647888+0000","last_became_peered":"2024-05-30T16:54:56.647888+0000","last_unstale":"2024-05-30T16:54:57.652873+0000","last_undegraded":"2024-05-30T16:54:57.652873+0000","last_fullsized":"2024-05-30T16:54:57.652873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:04.447059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744146+0000","last_change":"2024-05-30T16:54:56.655198+0000","last_active":"2024-05-30T16:54:57.744146+0000","last_peered":"2024-05-30T16:54:57.744146+0000","last_clean":"2024-05-30T16:54:57.744146+0000","last_became_active":"2024-05-30T16:54:56.654928+0000","last_became_peered":"2024-05-30T16:54:56.654928+0000","last_unstale":"2024-05-30T16:54:57.744146+0000","last_undegraded":"2024-05-30T16:54:57.744146+0000","last_fullsized":"2024-05-30T16:54:57.744146+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:04:16.402646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747523+0000","last_change":"2024-05-30T16:54:56.655011+0000","last_active":"2024-05-30T16:54:57.747523+0000","last_peered":"2024-05-30T16:54:57.747523+0000","last_clean":"2024-05-30T16:54:57.747523+0000","last_became_active":"2024-05-30T16:54:56.650949+0000","last_became_peered":"2024-05-30T16:54:56.650949+0000","last_unstale":"2024-05-30T16:54:57.747523+0000","last_undegraded":"2024-05-30T16:54:57.747523+0000","last_fullsized":"2024-05-30T16:54:57.747523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:27.907456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746765+0000","last_change":"2024-05-30T16:54:56.656036+0000","last_active":"2024-05-30T16:54:57.746765+0000","last_peered":"2024-05-30T16:54:57.746765+0000","last_clean":"2024-05-30T16:54:57.746765+0000","last_became_active":"2024-05-30T16:54:56.655646+0000","last_became_peered":"2024-05-30T16:54:56.655646+0000","last_unstale":"2024-05-30T16:54:57.746765+0000","last_undegraded":"2024-05-30T16:54:57.746765+0000","last_fullsized":"2024-05-30T16:54:57.746765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:41:45.993512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649658+0000","last_change":"2024-05-30T16:54:56.654191+0000","last_active":"2024-05-30T16:54:57.649658+0000","last_peered":"2024-05-30T16:54:57.649658+0000","last_clean":"2024-05-30T16:54:57.649658+0000","last_became_active":"2024-05-30T16:54:56.653132+0000","last_became_peered":"2024-05-30T16:54:56.653132+0000","last_unstale":"2024-05-30T16:54:57.649658+0000","last_undegraded":"2024-05-30T16:54:57.649658+0000","last_fullsized":"2024-05-30T16:54:57.649658+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:02.366944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743852+0000","last_change":"2024-05-30T16:54:56.658124+0000","last_active":"2024-05-30T16:54:57.743852+0000","last_peered":"2024-05-30T16:54:57.743852+0000","last_clean":"2024-05-30T16:54:57.743852+0000","last_became_active":"2024-05-30T16:54:56.655085+0000","last_became_peered":"2024-05-30T16:54:56.655085+0000","last_unstale":"2024-05-30T16:54:57.743852+0000","last_undegraded":"2024-05-30T16:54:57.743852+0000","last_fullsized":"2024-05-30T16:54:57.743852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:14:01.523291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743565+0000","last_change":"2024-05-30T16:54:56.646454+0000","last_active":"2024-05-30T16:54:57.743565+0000","last_peered":"2024-05-30T16:54:57.743565+0000","last_clean":"2024-05-30T16:54:57.743565+0000","last_became_active":"2024-05-30T16:54:56.645488+0000","last_became_peered":"2024-05-30T16:54:56.645488+0000","last_unstale":"2024-05-30T16:54:57.743565+0000","last_undegraded":"2024-05-30T16:54:57.743565+0000","last_fullsized":"2024-05-30T16:54:57.743565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:04.334329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743607+0000","last_change":"2024-05-30T16:54:56.656993+0000","last_active":"2024-05-30T16:54:57.743607+0000","last_peered":"2024-05-30T16:54:57.743607+0000","last_clean":"2024-05-30T16:54:57.743607+0000","last_became_active":"2024-05-30T16:54:56.656664+0000","last_became_peered":"2024-05-30T16:54:56.656664+0000","last_unstale":"2024-05-30T16:54:57.743607+0000","last_undegraded":"2024-05-30T16:54:57.743607+0000","last_fullsized":"2024-05-30T16:54:57.743607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:45:02.436795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745066+0000","last_change":"2024-05-30T16:54:56.654092+0000","last_active":"2024-05-30T16:54:57.745066+0000","last_peered":"2024-05-30T16:54:57.745066+0000","last_clean":"2024-05-30T16:54:57.745066+0000","last_became_active":"2024-05-30T16:54:56.649755+0000","last_became_peered":"2024-05-30T16:54:56.649755+0000","last_unstale":"2024-05-30T16:54:57.745066+0000","last_undegraded":"2024-05-30T16:54:57.745066+0000","last_fullsized":"2024-05-30T16:54:57.745066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:38:11.617688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744174+0000","last_change":"2024-05-30T16:54:56.653022+0000","last_active":"2024-05-30T16:54:57.744174+0000","last_peered":"2024-05-30T16:54:57.744174+0000","last_clean":"2024-05-30T16:54:57.744174+0000","last_became_active":"2024-05-30T16:54:56.652636+0000","last_became_peered":"2024-05-30T16:54:56.652636+0000","last_unstale":"2024-05-30T16:54:57.744174+0000","last_undegraded":"2024-05-30T16:54:57.744174+0000","last_fullsized":"2024-05-30T16:54:57.744174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:53:37.735789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649826+0000","last_change":"2024-05-30T16:54:56.654605+0000","last_active":"2024-05-30T16:54:57.649826+0000","last_peered":"2024-05-30T16:54:57.649826+0000","last_clean":"2024-05-30T16:54:57.649826+0000","last_became_active":"2024-05-30T16:54:56.654328+0000","last_became_peered":"2024-05-30T16:54:56.654328+0000","last_unstale":"2024-05-30T16:54:57.649826+0000","last_undegraded":"2024-05-30T16:54:57.649826+0000","last_fullsized":"2024-05-30T16:54:57.649826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:23.342774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653977+0000","last_change":"2024-05-30T16:54:56.647710+0000","last_active":"2024-05-30T16:54:57.653977+0000","last_peered":"2024-05-30T16:54:57.653977+0000","last_clean":"2024-05-30T16:54:57.653977+0000","last_became_active":"2024-05-30T16:54:56.647270+0000","last_became_peered":"2024-05-30T16:54:56.647270+0000","last_unstale":"2024-05-30T16:54:57.653977+0000","last_undegraded":"2024-05-30T16:54:57.653977+0000","last_fullsized":"2024-05-30T16:54:57.653977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:13.200522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743282+0000","last_change":"2024-05-30T16:54:56.651160+0000","last_active":"2024-05-30T16:54:57.743282+0000","last_peered":"2024-05-30T16:54:57.743282+0000","last_clean":"2024-05-30T16:54:57.743282+0000","last_became_active":"2024-05-30T16:54:56.650855+0000","last_became_peered":"2024-05-30T16:54:56.650855+0000","last_unstale":"2024-05-30T16:54:57.743282+0000","last_undegraded":"2024-05-30T16:54:57.743282+0000","last_fullsized":"2024-05-30T16:54:57.743282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:30.080641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745876+0000","last_change":"2024-05-30T16:54:56.651661+0000","last_active":"2024-05-30T16:54:57.745876+0000","last_peered":"2024-05-30T16:54:57.745876+0000","last_clean":"2024-05-30T16:54:57.745876+0000","last_became_active":"2024-05-30T16:54:56.651355+0000","last_became_peered":"2024-05-30T16:54:56.651355+0000","last_unstale":"2024-05-30T16:54:57.745876+0000","last_undegraded":"2024-05-30T16:54:57.745876+0000","last_fullsized":"2024-05-30T16:54:57.745876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:08:13.510748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745077+0000","last_change":"2024-05-30T16:54:56.656794+0000","last_active":"2024-05-30T16:54:57.745077+0000","last_peered":"2024-05-30T16:54:57.745077+0000","last_clean":"2024-05-30T16:54:57.745077+0000","last_became_active":"2024-05-30T16:54:56.655613+0000","last_became_peered":"2024-05-30T16:54:56.655613+0000","last_unstale":"2024-05-30T16:54:57.745077+0000","last_undegraded":"2024-05-30T16:54:57.745077+0000","last_fullsized":"2024-05-30T16:54:57.745077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:04.447059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747605+0000","last_change":"2024-05-30T16:54:56.656425+0000","last_active":"2024-05-30T16:54:57.747605+0000","last_peered":"2024-05-30T16:54:57.747605+0000","last_clean":"2024-05-30T16:54:57.747605+0000","last_became_active":"2024-05-30T16:54:56.652392+0000","last_became_peered":"2024-05-30T16:54:56.652392+0000","last_unstale":"2024-05-30T16:54:57.747605+0000","last_undegraded":"2024-05-30T16:54:57.747605+0000","last_fullsized":"2024-05-30T16:54:57.747605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:29:18.084001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746811+0000","last_change":"2024-05-30T16:54:56.654512+0000","last_active":"2024-05-30T16:54:57.746811+0000","last_peered":"2024-05-30T16:54:57.746811+0000","last_clean":"2024-05-30T16:54:57.746811+0000","last_became_active":"2024-05-30T16:54:56.654226+0000","last_became_peered":"2024-05-30T16:54:56.654226+0000","last_unstale":"2024-05-30T16:54:57.746811+0000","last_undegraded":"2024-05-30T16:54:57.746811+0000","last_fullsized":"2024-05-30T16:54:57.746811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:30:33.721422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746512+0000","last_change":"2024-05-30T16:54:56.660027+0000","last_active":"2024-05-30T16:54:57.746512+0000","last_peered":"2024-05-30T16:54:57.746512+0000","last_clean":"2024-05-30T16:54:57.746512+0000","last_became_active":"2024-05-30T16:54:56.659709+0000","last_became_peered":"2024-05-30T16:54:56.659709+0000","last_unstale":"2024-05-30T16:54:57.746512+0000","last_undegraded":"2024-05-30T16:54:57.746512+0000","last_fullsized":"2024-05-30T16:54:57.746512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:35:20.382926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654788+0000","last_change":"2024-05-30T16:54:56.650514+0000","last_active":"2024-05-30T16:54:57.654788+0000","last_peered":"2024-05-30T16:54:57.654788+0000","last_clean":"2024-05-30T16:54:57.654788+0000","last_became_active":"2024-05-30T16:54:56.650185+0000","last_became_peered":"2024-05-30T16:54:56.650185+0000","last_unstale":"2024-05-30T16:54:57.654788+0000","last_undegraded":"2024-05-30T16:54:57.654788+0000","last_fullsized":"2024-05-30T16:54:57.654788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:43:11.984521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745216+0000","last_change":"2024-05-30T16:54:56.657943+0000","last_active":"2024-05-30T16:54:57.745216+0000","last_peered":"2024-05-30T16:54:57.745216+0000","last_clean":"2024-05-30T16:54:57.745216+0000","last_became_active":"2024-05-30T16:54:56.657471+0000","last_became_peered":"2024-05-30T16:54:56.657471+0000","last_unstale":"2024-05-30T16:54:57.745216+0000","last_undegraded":"2024-05-30T16:54:57.745216+0000","last_fullsized":"2024-05-30T16:54:57.745216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:49.601902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746196+0000","last_change":"2024-05-30T16:54:56.651927+0000","last_active":"2024-05-30T16:54:57.746196+0000","last_peered":"2024-05-30T16:54:57.746196+0000","last_clean":"2024-05-30T16:54:57.746196+0000","last_became_active":"2024-05-30T16:54:56.651606+0000","last_became_peered":"2024-05-30T16:54:56.651606+0000","last_unstale":"2024-05-30T16:54:57.746196+0000","last_undegraded":"2024-05-30T16:54:57.746196+0000","last_fullsized":"2024-05-30T16:54:57.746196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:41.464064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745196+0000","last_change":"2024-05-30T16:54:56.652611+0000","last_active":"2024-05-30T16:54:57.745196+0000","last_peered":"2024-05-30T16:54:57.745196+0000","last_clean":"2024-05-30T16:54:57.745196+0000","last_became_active":"2024-05-30T16:54:56.650324+0000","last_became_peered":"2024-05-30T16:54:56.650324+0000","last_unstale":"2024-05-30T16:54:57.745196+0000","last_undegraded":"2024-05-30T16:54:57.745196+0000","last_fullsized":"2024-05-30T16:54:57.745196+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:38.341289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653606+0000","last_change":"2024-05-30T16:54:56.651181+0000","last_active":"2024-05-30T16:54:57.653606+0000","last_peered":"2024-05-30T16:54:57.653606+0000","last_clean":"2024-05-30T16:54:57.653606+0000","last_became_active":"2024-05-30T16:54:56.650465+0000","last_became_peered":"2024-05-30T16:54:56.650465+0000","last_unstale":"2024-05-30T16:54:57.653606+0000","last_undegraded":"2024-05-30T16:54:57.653606+0000","last_fullsized":"2024-05-30T16:54:57.653606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:35:41.503418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654805+0000","last_change":"2024-05-30T16:54:56.649580+0000","last_active":"2024-05-30T16:54:57.654805+0000","last_peered":"2024-05-30T16:54:57.654805+0000","last_clean":"2024-05-30T16:54:57.654805+0000","last_became_active":"2024-05-30T16:54:56.649298+0000","last_became_peered":"2024-05-30T16:54:56.649298+0000","last_unstale":"2024-05-30T16:54:57.654805+0000","last_undegraded":"2024-05-30T16:54:57.654805+0000","last_fullsized":"2024-05-30T16:54:57.654805+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:55:02.203424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.651516+0000","last_change":"2024-05-30T16:54:56.660910+0000","last_active":"2024-05-30T16:54:57.651516+0000","last_peered":"2024-05-30T16:54:57.651516+0000","last_clean":"2024-05-30T16:54:57.651516+0000","last_became_active":"2024-05-30T16:54:56.660627+0000","last_became_peered":"2024-05-30T16:54:56.660627+0000","last_unstale":"2024-05-30T16:54:57.651516+0000","last_undegraded":"2024-05-30T16:54:57.651516+0000","last_fullsized":"2024-05-30T16:54:57.651516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:56:12.172501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747211+0000","last_change":"2024-05-30T16:54:56.667544+0000","last_active":"2024-05-30T16:54:57.747211+0000","last_peered":"2024-05-30T16:54:57.747211+0000","last_clean":"2024-05-30T16:54:57.747211+0000","last_became_active":"2024-05-30T16:54:56.665510+0000","last_became_peered":"2024-05-30T16:54:56.665510+0000","last_unstale":"2024-05-30T16:54:57.747211+0000","last_undegraded":"2024-05-30T16:54:57.747211+0000","last_fullsized":"2024-05-30T16:54:57.747211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:29.099619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.748032+0000","last_change":"2024-05-30T16:54:55.628463+0000","last_active":"2024-05-30T16:54:57.748032+0000","last_peered":"2024-05-30T16:54:57.748032+0000","last_clean":"2024-05-30T16:54:57.748032+0000","last_became_active":"2024-05-30T16:54:55.628026+0000","last_became_peered":"2024-05-30T16:54:55.628026+0000","last_unstale":"2024-05-30T16:54:57.748032+0000","last_undegraded":"2024-05-30T16:54:57.748032+0000","last_fullsized":"2024-05-30T16:54:57.748032+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:56.321162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744234+0000","last_change":"2024-05-30T16:54:56.652959+0000","last_active":"2024-05-30T16:54:57.744234+0000","last_peered":"2024-05-30T16:54:57.744234+0000","last_clean":"2024-05-30T16:54:57.744234+0000","last_became_active":"2024-05-30T16:54:56.652190+0000","last_became_peered":"2024-05-30T16:54:56.652190+0000","last_unstale":"2024-05-30T16:54:57.744234+0000","last_undegraded":"2024-05-30T16:54:57.744234+0000","last_fullsized":"2024-05-30T16:54:57.744234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:02.366944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743407+0000","last_change":"2024-05-30T16:54:55.623039+0000","last_active":"2024-05-30T16:54:57.743407+0000","last_peered":"2024-05-30T16:54:57.743407+0000","last_clean":"2024-05-30T16:54:57.743407+0000","last_became_active":"2024-05-30T16:54:55.622700+0000","last_became_peered":"2024-05-30T16:54:55.622700+0000","last_unstale":"2024-05-30T16:54:57.743407+0000","last_undegraded":"2024-05-30T16:54:57.743407+0000","last_fullsized":"2024-05-30T16:54:57.743407+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:35:40.493510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653060+0000","last_change":"2024-05-30T16:54:56.662477+0000","last_active":"2024-05-30T16:54:57.653060+0000","last_peered":"2024-05-30T16:54:57.653060+0000","last_clean":"2024-05-30T16:54:57.653060+0000","last_became_active":"2024-05-30T16:54:56.661843+0000","last_became_peered":"2024-05-30T16:54:56.661843+0000","last_unstale":"2024-05-30T16:54:57.653060+0000","last_undegraded":"2024-05-30T16:54:57.653060+0000","last_fullsized":"2024-05-30T16:54:57.653060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:08:13.510748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746053+0000","last_change":"2024-05-30T16:54:55.631759+0000","last_active":"2024-05-30T16:54:57.746053+0000","last_peered":"2024-05-30T16:54:57.746053+0000","last_clean":"2024-05-30T16:54:57.746053+0000","last_became_active":"2024-05-30T16:54:55.630908+0000","last_became_peered":"2024-05-30T16:54:55.630908+0000","last_unstale":"2024-05-30T16:54:57.746053+0000","last_undegraded":"2024-05-30T16:54:57.746053+0000","last_fullsized":"2024-05-30T16:54:57.746053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:20.115543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747203+0000","last_change":"2024-05-30T16:54:56.655912+0000","last_active":"2024-05-30T16:54:57.747203+0000","last_peered":"2024-05-30T16:54:57.747203+0000","last_clean":"2024-05-30T16:54:57.747203+0000","last_became_active":"2024-05-30T16:54:56.655496+0000","last_became_peered":"2024-05-30T16:54:56.655496+0000","last_unstale":"2024-05-30T16:54:57.747203+0000","last_undegraded":"2024-05-30T16:54:57.747203+0000","last_fullsized":"2024-05-30T16:54:57.747203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:39.899066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653179+0000","last_change":"2024-05-30T16:54:55.623100+0000","last_active":"2024-05-30T16:54:57.653179+0000","last_peered":"2024-05-30T16:54:57.653179+0000","last_clean":"2024-05-30T16:54:57.653179+0000","last_became_active":"2024-05-30T16:54:55.622544+0000","last_became_peered":"2024-05-30T16:54:55.622544+0000","last_unstale":"2024-05-30T16:54:57.653179+0000","last_undegraded":"2024-05-30T16:54:57.653179+0000","last_fullsized":"2024-05-30T16:54:57.653179+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:16:50.348074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653629+0000","last_change":"2024-05-30T16:54:56.651407+0000","last_active":"2024-05-30T16:54:57.653629+0000","last_peered":"2024-05-30T16:54:57.653629+0000","last_clean":"2024-05-30T16:54:57.653629+0000","last_became_active":"2024-05-30T16:54:56.650844+0000","last_became_peered":"2024-05-30T16:54:56.650844+0000","last_unstale":"2024-05-30T16:54:57.653629+0000","last_undegraded":"2024-05-30T16:54:57.653629+0000","last_fullsized":"2024-05-30T16:54:57.653629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:29.380010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745262+0000","last_change":"2024-05-30T16:54:55.618125+0000","last_active":"2024-05-30T16:54:57.745262+0000","last_peered":"2024-05-30T16:54:57.745262+0000","last_clean":"2024-05-30T16:54:57.745262+0000","last_became_active":"2024-05-30T16:54:55.617577+0000","last_became_peered":"2024-05-30T16:54:55.617577+0000","last_unstale":"2024-05-30T16:54:57.745262+0000","last_undegraded":"2024-05-30T16:54:57.745262+0000","last_fullsized":"2024-05-30T16:54:57.745262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:43:18.083619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747136+0000","last_change":"2024-05-30T16:54:56.657326+0000","last_active":"2024-05-30T16:54:57.747136+0000","last_peered":"2024-05-30T16:54:57.747136+0000","last_clean":"2024-05-30T16:54:57.747136+0000","last_became_active":"2024-05-30T16:54:56.656754+0000","last_became_peered":"2024-05-30T16:54:56.656754+0000","last_unstale":"2024-05-30T16:54:57.747136+0000","last_undegraded":"2024-05-30T16:54:57.747136+0000","last_fullsized":"2024-05-30T16:54:57.747136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:16:13.200522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652801+0000","last_change":"2024-05-30T16:54:55.626901+0000","last_active":"2024-05-30T16:54:57.652801+0000","last_peered":"2024-05-30T16:54:57.652801+0000","last_clean":"2024-05-30T16:54:57.652801+0000","last_became_active":"2024-05-30T16:54:55.626578+0000","last_became_peered":"2024-05-30T16:54:55.626578+0000","last_unstale":"2024-05-30T16:54:57.652801+0000","last_undegraded":"2024-05-30T16:54:57.652801+0000","last_fullsized":"2024-05-30T16:54:57.652801+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:01.357036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.742652+0000","last_change":"2024-05-30T16:54:56.659662+0000","last_active":"2024-05-30T16:54:57.742652+0000","last_peered":"2024-05-30T16:54:57.742652+0000","last_clean":"2024-05-30T16:54:57.742652+0000","last_became_active":"2024-05-30T16:54:56.657971+0000","last_became_peered":"2024-05-30T16:54:56.657971+0000","last_unstale":"2024-05-30T16:54:57.742652+0000","last_undegraded":"2024-05-30T16:54:57.742652+0000","last_fullsized":"2024-05-30T16:54:57.742652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:31.815811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649175+0000","last_change":"2024-05-30T16:54:55.629842+0000","last_active":"2024-05-30T16:54:57.649175+0000","last_peered":"2024-05-30T16:54:57.649175+0000","last_clean":"2024-05-30T16:54:57.649175+0000","last_became_active":"2024-05-30T16:54:55.629495+0000","last_became_peered":"2024-05-30T16:54:55.629495+0000","last_unstale":"2024-05-30T16:54:57.649175+0000","last_undegraded":"2024-05-30T16:54:57.649175+0000","last_fullsized":"2024-05-30T16:54:57.649175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:36:54.016165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743799+0000","last_change":"2024-05-30T16:54:56.656743+0000","last_active":"2024-05-30T16:54:57.743799+0000","last_peered":"2024-05-30T16:54:57.743799+0000","last_clean":"2024-05-30T16:54:57.743799+0000","last_became_active":"2024-05-30T16:54:56.656476+0000","last_became_peered":"2024-05-30T16:54:56.656476+0000","last_unstale":"2024-05-30T16:54:57.743799+0000","last_undegraded":"2024-05-30T16:54:57.743799+0000","last_fullsized":"2024-05-30T16:54:57.743799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:07.005681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.742573+0000","last_change":"2024-05-30T16:54:55.611884+0000","last_active":"2024-05-30T16:54:57.742573+0000","last_peered":"2024-05-30T16:54:57.742573+0000","last_clean":"2024-05-30T16:54:57.742573+0000","last_became_active":"2024-05-30T16:54:55.611572+0000","last_became_peered":"2024-05-30T16:54:55.611572+0000","last_unstale":"2024-05-30T16:54:57.742573+0000","last_undegraded":"2024-05-30T16:54:57.742573+0000","last_fullsized":"2024-05-30T16:54:57.742573+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:24.338448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654704+0000","last_change":"2024-05-30T16:54:56.653034+0000","last_active":"2024-05-30T16:54:57.654704+0000","last_peered":"2024-05-30T16:54:57.654704+0000","last_clean":"2024-05-30T16:54:57.654704+0000","last_became_active":"2024-05-30T16:54:56.652759+0000","last_became_peered":"2024-05-30T16:54:56.652759+0000","last_unstale":"2024-05-30T16:54:57.654704+0000","last_undegraded":"2024-05-30T16:54:57.654704+0000","last_fullsized":"2024-05-30T16:54:57.654704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:43:19.421505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746377+0000","last_change":"2024-05-30T16:54:55.624791+0000","last_active":"2024-05-30T16:54:57.746377+0000","last_peered":"2024-05-30T16:54:57.746377+0000","last_clean":"2024-05-30T16:54:57.746377+0000","last_became_active":"2024-05-30T16:54:55.621959+0000","last_became_peered":"2024-05-30T16:54:55.621959+0000","last_unstale":"2024-05-30T16:54:57.746377+0000","last_undegraded":"2024-05-30T16:54:57.746377+0000","last_fullsized":"2024-05-30T16:54:57.746377+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:44.846266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747766+0000","last_change":"2024-05-30T16:54:56.667857+0000","last_active":"2024-05-30T16:54:57.747766+0000","last_peered":"2024-05-30T16:54:57.747766+0000","last_clean":"2024-05-30T16:54:57.747766+0000","last_became_active":"2024-05-30T16:54:56.667272+0000","last_became_peered":"2024-05-30T16:54:56.667272+0000","last_unstale":"2024-05-30T16:54:57.747766+0000","last_undegraded":"2024-05-30T16:54:57.747766+0000","last_fullsized":"2024-05-30T16:54:57.747766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:27:30.039187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654433+0000","last_change":"2024-05-30T16:54:55.627531+0000","last_active":"2024-05-30T16:54:57.654433+0000","last_peered":"2024-05-30T16:54:57.654433+0000","last_clean":"2024-05-30T16:54:57.654433+0000","last_became_active":"2024-05-30T16:54:55.627166+0000","last_became_peered":"2024-05-30T16:54:55.627166+0000","last_unstale":"2024-05-30T16:54:57.654433+0000","last_undegraded":"2024-05-30T16:54:57.654433+0000","last_fullsized":"2024-05-30T16:54:57.654433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:25.538140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653312+0000","last_change":"2024-05-30T16:54:56.658827+0000","last_active":"2024-05-30T16:54:57.653312+0000","last_peered":"2024-05-30T16:54:57.653312+0000","last_clean":"2024-05-30T16:54:57.653312+0000","last_became_active":"2024-05-30T16:54:56.658375+0000","last_became_peered":"2024-05-30T16:54:56.658375+0000","last_unstale":"2024-05-30T16:54:57.653312+0000","last_undegraded":"2024-05-30T16:54:57.653312+0000","last_fullsized":"2024-05-30T16:54:57.653312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:41.464064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745259+0000","last_change":"2024-05-30T16:54:55.615825+0000","last_active":"2024-05-30T16:54:57.745259+0000","last_peered":"2024-05-30T16:54:57.745259+0000","last_clean":"2024-05-30T16:54:57.745259+0000","last_became_active":"2024-05-30T16:54:55.615172+0000","last_became_peered":"2024-05-30T16:54:55.615172+0000","last_unstale":"2024-05-30T16:54:57.745259+0000","last_undegraded":"2024-05-30T16:54:57.745259+0000","last_fullsized":"2024-05-30T16:54:57.745259+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:25:13.012127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743833+0000","last_change":"2024-05-30T16:54:56.649822+0000","last_active":"2024-05-30T16:54:57.743833+0000","last_peered":"2024-05-30T16:54:57.743833+0000","last_clean":"2024-05-30T16:54:57.743833+0000","last_became_active":"2024-05-30T16:54:56.647069+0000","last_became_peered":"2024-05-30T16:54:56.647069+0000","last_unstale":"2024-05-30T16:54:57.743833+0000","last_undegraded":"2024-05-30T16:54:57.743833+0000","last_fullsized":"2024-05-30T16:54:57.743833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:33.417787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649953+0000","last_change":"2024-05-30T16:54:55.613399+0000","last_active":"2024-05-30T16:54:57.649953+0000","last_peered":"2024-05-30T16:54:57.649953+0000","last_clean":"2024-05-30T16:54:57.649953+0000","last_became_active":"2024-05-30T16:54:55.612527+0000","last_became_peered":"2024-05-30T16:54:55.612527+0000","last_unstale":"2024-05-30T16:54:57.649953+0000","last_undegraded":"2024-05-30T16:54:57.649953+0000","last_fullsized":"2024-05-30T16:54:57.649953+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:50:59.288715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745817+0000","last_change":"2024-05-30T16:54:56.654144+0000","last_active":"2024-05-30T16:54:57.745817+0000","last_peered":"2024-05-30T16:54:57.745817+0000","last_clean":"2024-05-30T16:54:57.745817+0000","last_became_active":"2024-05-30T16:54:56.653861+0000","last_became_peered":"2024-05-30T16:54:56.653861+0000","last_unstale":"2024-05-30T16:54:57.745817+0000","last_undegraded":"2024-05-30T16:54:57.745817+0000","last_fullsized":"2024-05-30T16:54:57.745817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:14:01.523291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649787+0000","last_change":"2024-05-30T16:54:55.624179+0000","last_active":"2024-05-30T16:54:57.649787+0000","last_peered":"2024-05-30T16:54:57.649787+0000","last_clean":"2024-05-30T16:54:57.649787+0000","last_became_active":"2024-05-30T16:54:55.623735+0000","last_became_peered":"2024-05-30T16:54:55.623735+0000","last_unstale":"2024-05-30T16:54:57.649787+0000","last_undegraded":"2024-05-30T16:54:57.649787+0000","last_fullsized":"2024-05-30T16:54:57.649787+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:54:30.805904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654332+0000","last_change":"2024-05-30T16:54:56.656732+0000","last_active":"2024-05-30T16:54:57.654332+0000","last_peered":"2024-05-30T16:54:57.654332+0000","last_clean":"2024-05-30T16:54:57.654332+0000","last_became_active":"2024-05-30T16:54:56.656329+0000","last_became_peered":"2024-05-30T16:54:56.656329+0000","last_unstale":"2024-05-30T16:54:57.654332+0000","last_undegraded":"2024-05-30T16:54:57.654332+0000","last_fullsized":"2024-05-30T16:54:57.654332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:57.331070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653086+0000","last_change":"2024-05-30T16:54:55.627550+0000","last_active":"2024-05-30T16:54:57.653086+0000","last_peered":"2024-05-30T16:54:57.653086+0000","last_clean":"2024-05-30T16:54:57.653086+0000","last_became_active":"2024-05-30T16:54:55.627217+0000","last_became_peered":"2024-05-30T16:54:55.627217+0000","last_unstale":"2024-05-30T16:54:57.653086+0000","last_undegraded":"2024-05-30T16:54:57.653086+0000","last_fullsized":"2024-05-30T16:54:57.653086+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:43:36.263157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743028+0000","last_change":"2024-05-30T16:54:56.651514+0000","last_active":"2024-05-30T16:54:57.743028+0000","last_peered":"2024-05-30T16:54:57.743028+0000","last_clean":"2024-05-30T16:54:57.743028+0000","last_became_active":"2024-05-30T16:54:56.650990+0000","last_became_peered":"2024-05-30T16:54:56.650990+0000","last_unstale":"2024-05-30T16:54:57.743028+0000","last_undegraded":"2024-05-30T16:54:57.743028+0000","last_fullsized":"2024-05-30T16:54:57.743028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:45:17.578200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652897+0000","last_change":"2024-05-30T16:54:55.619882+0000","last_active":"2024-05-30T16:54:57.652897+0000","last_peered":"2024-05-30T16:54:57.652897+0000","last_clean":"2024-05-30T16:54:57.652897+0000","last_became_active":"2024-05-30T16:54:55.619553+0000","last_became_peered":"2024-05-30T16:54:55.619553+0000","last_unstale":"2024-05-30T16:54:57.652897+0000","last_undegraded":"2024-05-30T16:54:57.652897+0000","last_fullsized":"2024-05-30T16:54:57.652897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:00:49.949175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.742806+0000","last_change":"2024-05-30T16:54:56.648216+0000","last_active":"2024-05-30T16:54:57.742806+0000","last_peered":"2024-05-30T16:54:57.742806+0000","last_clean":"2024-05-30T16:54:57.742806+0000","last_became_active":"2024-05-30T16:54:56.647745+0000","last_became_peered":"2024-05-30T16:54:56.647745+0000","last_unstale":"2024-05-30T16:54:57.742806+0000","last_undegraded":"2024-05-30T16:54:57.742806+0000","last_fullsized":"2024-05-30T16:54:57.742806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:51:00.298622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744979+0000","last_change":"2024-05-30T16:54:55.629445+0000","last_active":"2024-05-30T16:54:57.744979+0000","last_peered":"2024-05-30T16:54:57.744979+0000","last_clean":"2024-05-30T16:54:57.744979+0000","last_became_active":"2024-05-30T16:54:55.627481+0000","last_became_peered":"2024-05-30T16:54:55.627481+0000","last_unstale":"2024-05-30T16:54:57.744979+0000","last_undegraded":"2024-05-30T16:54:57.744979+0000","last_fullsized":"2024-05-30T16:54:57.744979+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:08.663109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747164+0000","last_change":"2024-05-30T16:54:56.652717+0000","last_active":"2024-05-30T16:54:57.747164+0000","last_peered":"2024-05-30T16:54:57.747164+0000","last_clean":"2024-05-30T16:54:57.747164+0000","last_became_active":"2024-05-30T16:54:56.651285+0000","last_became_peered":"2024-05-30T16:54:56.651285+0000","last_unstale":"2024-05-30T16:54:57.747164+0000","last_undegraded":"2024-05-30T16:54:57.747164+0000","last_fullsized":"2024-05-30T16:54:57.747164+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:23:19.300945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654749+0000","last_change":"2024-05-30T16:54:55.626454+0000","last_active":"2024-05-30T16:54:57.654749+0000","last_peered":"2024-05-30T16:54:57.654749+0000","last_clean":"2024-05-30T16:54:57.654749+0000","last_became_active":"2024-05-30T16:54:55.625789+0000","last_became_peered":"2024-05-30T16:54:55.625789+0000","last_unstale":"2024-05-30T16:54:57.654749+0000","last_undegraded":"2024-05-30T16:54:57.654749+0000","last_fullsized":"2024-05-30T16:54:57.654749+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:17:08.663109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747984+0000","last_change":"2024-05-30T16:54:56.663267+0000","last_active":"2024-05-30T16:54:57.747984+0000","last_peered":"2024-05-30T16:54:57.747984+0000","last_clean":"2024-05-30T16:54:57.747984+0000","last_became_active":"2024-05-30T16:54:56.662116+0000","last_became_peered":"2024-05-30T16:54:56.662116+0000","last_unstale":"2024-05-30T16:54:57.747984+0000","last_undegraded":"2024-05-30T16:54:57.747984+0000","last_fullsized":"2024-05-30T16:54:57.747984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:44:21.798658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747315+0000","last_change":"2024-05-30T16:54:55.631123+0000","last_active":"2024-05-30T16:54:57.747315+0000","last_peered":"2024-05-30T16:54:57.747315+0000","last_clean":"2024-05-30T16:54:57.747315+0000","last_became_active":"2024-05-30T16:54:55.630836+0000","last_became_peered":"2024-05-30T16:54:55.630836+0000","last_unstale":"2024-05-30T16:54:57.747315+0000","last_undegraded":"2024-05-30T16:54:57.747315+0000","last_fullsized":"2024-05-30T16:54:57.747315+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:03:22.133301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649867+0000","last_change":"2024-05-30T16:54:56.653431+0000","last_active":"2024-05-30T16:54:57.649867+0000","last_peered":"2024-05-30T16:54:57.649867+0000","last_clean":"2024-05-30T16:54:57.649867+0000","last_became_active":"2024-05-30T16:54:56.652792+0000","last_became_peered":"2024-05-30T16:54:56.652792+0000","last_unstale":"2024-05-30T16:54:57.649867+0000","last_undegraded":"2024-05-30T16:54:57.649867+0000","last_fullsized":"2024-05-30T16:54:57.649867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:43:37.273065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654112+0000","last_change":"2024-05-30T16:54:55.630437+0000","last_active":"2024-05-30T16:54:57.654112+0000","last_peered":"2024-05-30T16:54:57.654112+0000","last_clean":"2024-05-30T16:54:57.654112+0000","last_became_active":"2024-05-30T16:54:55.630081+0000","last_became_peered":"2024-05-30T16:54:55.630081+0000","last_unstale":"2024-05-30T16:54:57.654112+0000","last_undegraded":"2024-05-30T16:54:57.654112+0000","last_fullsized":"2024-05-30T16:54:57.654112+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:44.846266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743943+0000","last_change":"2024-05-30T16:54:56.649578+0000","last_active":"2024-05-30T16:54:57.743943+0000","last_peered":"2024-05-30T16:54:57.743943+0000","last_clean":"2024-05-30T16:54:57.743943+0000","last_became_active":"2024-05-30T16:54:56.646637+0000","last_became_peered":"2024-05-30T16:54:56.646637+0000","last_unstale":"2024-05-30T16:54:57.743943+0000","last_undegraded":"2024-05-30T16:54:57.743943+0000","last_fullsized":"2024-05-30T16:54:57.743943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:32:10.287840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747299+0000","last_change":"2024-05-30T16:54:55.632199+0000","last_active":"2024-05-30T16:54:57.747299+0000","last_peered":"2024-05-30T16:54:57.747299+0000","last_clean":"2024-05-30T16:54:57.747299+0000","last_became_active":"2024-05-30T16:54:55.631811+0000","last_became_peered":"2024-05-30T16:54:55.631811+0000","last_unstale":"2024-05-30T16:54:57.747299+0000","last_undegraded":"2024-05-30T16:54:57.747299+0000","last_fullsized":"2024-05-30T16:54:57.747299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:28.370103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.745262+0000","last_change":"2024-05-30T16:54:56.650664+0000","last_active":"2024-05-30T16:54:57.745262+0000","last_peered":"2024-05-30T16:54:57.745262+0000","last_clean":"2024-05-30T16:54:57.745262+0000","last_became_active":"2024-05-30T16:54:56.649814+0000","last_became_peered":"2024-05-30T16:54:56.649814+0000","last_unstale":"2024-05-30T16:54:57.745262+0000","last_undegraded":"2024-05-30T16:54:57.745262+0000","last_fullsized":"2024-05-30T16:54:57.745262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:19:48.958217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653365+0000","last_change":"2024-05-30T16:54:55.625995+0000","last_active":"2024-05-30T16:54:57.653365+0000","last_peered":"2024-05-30T16:54:57.653365+0000","last_clean":"2024-05-30T16:54:57.653365+0000","last_became_active":"2024-05-30T16:54:55.625720+0000","last_became_peered":"2024-05-30T16:54:55.625720+0000","last_unstale":"2024-05-30T16:54:57.653365+0000","last_undegraded":"2024-05-30T16:54:57.653365+0000","last_fullsized":"2024-05-30T16:54:57.653365+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:03:11.650715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744998+0000","last_change":"2024-05-30T16:54:56.654539+0000","last_active":"2024-05-30T16:54:57.744998+0000","last_peered":"2024-05-30T16:54:57.744998+0000","last_clean":"2024-05-30T16:54:57.744998+0000","last_became_active":"2024-05-30T16:54:56.653412+0000","last_became_peered":"2024-05-30T16:54:56.653412+0000","last_unstale":"2024-05-30T16:54:57.744998+0000","last_undegraded":"2024-05-30T16:54:57.744998+0000","last_fullsized":"2024-05-30T16:54:57.744998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:33:39.658399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.653206+0000","last_change":"2024-05-30T16:54:55.621183+0000","last_active":"2024-05-30T16:54:57.653206+0000","last_peered":"2024-05-30T16:54:57.653206+0000","last_clean":"2024-05-30T16:54:57.653206+0000","last_became_active":"2024-05-30T16:54:55.620893+0000","last_became_peered":"2024-05-30T16:54:55.620893+0000","last_unstale":"2024-05-30T16:54:57.653206+0000","last_undegraded":"2024-05-30T16:54:57.653206+0000","last_fullsized":"2024-05-30T16:54:57.653206+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:38:48.476553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744974+0000","last_change":"2024-05-30T16:54:56.664758+0000","last_active":"2024-05-30T16:54:57.744974+0000","last_peered":"2024-05-30T16:54:57.744974+0000","last_clean":"2024-05-30T16:54:57.744974+0000","last_became_active":"2024-05-30T16:54:56.664324+0000","last_became_peered":"2024-05-30T16:54:56.664324+0000","last_unstale":"2024-05-30T16:54:57.744974+0000","last_undegraded":"2024-05-30T16:54:57.744974+0000","last_fullsized":"2024-05-30T16:54:57.744974+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:14:46.367921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.742706+0000","last_change":"2024-05-30T16:54:55.623249+0000","last_active":"2024-05-30T16:54:57.742706+0000","last_peered":"2024-05-30T16:54:57.742706+0000","last_clean":"2024-05-30T16:54:57.742706+0000","last_became_active":"2024-05-30T16:54:55.622929+0000","last_became_peered":"2024-05-30T16:54:55.622929+0000","last_unstale":"2024-05-30T16:54:57.742706+0000","last_undegraded":"2024-05-30T16:54:57.742706+0000","last_fullsized":"2024-05-30T16:54:57.742706+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:58.271816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743340+0000","last_change":"2024-05-30T16:54:56.655795+0000","last_active":"2024-05-30T16:54:57.743340+0000","last_peered":"2024-05-30T16:54:57.743340+0000","last_clean":"2024-05-30T16:54:57.743340+0000","last_became_active":"2024-05-30T16:54:56.655488+0000","last_became_peered":"2024-05-30T16:54:56.655488+0000","last_unstale":"2024-05-30T16:54:57.743340+0000","last_undegraded":"2024-05-30T16:54:57.743340+0000","last_fullsized":"2024-05-30T16:54:57.743340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:35:33.255023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654775+0000","last_change":"2024-05-30T16:54:55.626336+0000","last_active":"2024-05-30T16:54:57.654775+0000","last_peered":"2024-05-30T16:54:57.654775+0000","last_clean":"2024-05-30T16:54:57.654775+0000","last_became_active":"2024-05-30T16:54:55.625166+0000","last_became_peered":"2024-05-30T16:54:55.625166+0000","last_unstale":"2024-05-30T16:54:57.654775+0000","last_undegraded":"2024-05-30T16:54:57.654775+0000","last_fullsized":"2024-05-30T16:54:57.654775+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:15:20.115543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654604+0000","last_change":"2024-05-30T16:54:56.643257+0000","last_active":"2024-05-30T16:54:57.654604+0000","last_peered":"2024-05-30T16:54:57.654604+0000","last_clean":"2024-05-30T16:54:57.654604+0000","last_became_active":"2024-05-30T16:54:56.642794+0000","last_became_peered":"2024-05-30T16:54:56.642794+0000","last_unstale":"2024-05-30T16:54:57.654604+0000","last_undegraded":"2024-05-30T16:54:57.654604+0000","last_fullsized":"2024-05-30T16:54:57.654604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:17:13.411216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.654988+0000","last_change":"2024-05-30T16:54:55.623374+0000","last_active":"2024-05-30T16:54:57.654988+0000","last_peered":"2024-05-30T16:54:57.654988+0000","last_clean":"2024-05-30T16:54:57.654988+0000","last_became_active":"2024-05-30T16:54:55.623068+0000","last_became_peered":"2024-05-30T16:54:55.623068+0000","last_unstale":"2024-05-30T16:54:57.654988+0000","last_undegraded":"2024-05-30T16:54:57.654988+0000","last_fullsized":"2024-05-30T16:54:57.654988+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:11.720295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747185+0000","last_change":"2024-05-30T16:54:56.666080+0000","last_active":"2024-05-30T16:54:57.747185+0000","last_peered":"2024-05-30T16:54:57.747185+0000","last_clean":"2024-05-30T16:54:57.747185+0000","last_became_active":"2024-05-30T16:54:56.665788+0000","last_became_peered":"2024-05-30T16:54:56.665788+0000","last_unstale":"2024-05-30T16:54:57.747185+0000","last_undegraded":"2024-05-30T16:54:57.747185+0000","last_fullsized":"2024-05-30T16:54:57.747185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:52:00.934684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743910+0000","last_change":"2024-05-30T16:54:55.622679+0000","last_active":"2024-05-30T16:54:57.743910+0000","last_peered":"2024-05-30T16:54:57.743910+0000","last_clean":"2024-05-30T16:54:57.743910+0000","last_became_active":"2024-05-30T16:54:55.622295+0000","last_became_peered":"2024-05-30T16:54:55.622295+0000","last_unstale":"2024-05-30T16:54:57.743910+0000","last_undegraded":"2024-05-30T16:54:57.743910+0000","last_fullsized":"2024-05-30T16:54:57.743910+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:28.956662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.747598+0000","last_change":"2024-05-30T16:54:56.655813+0000","last_active":"2024-05-30T16:54:57.747598+0000","last_peered":"2024-05-30T16:54:57.747598+0000","last_clean":"2024-05-30T16:54:57.747598+0000","last_became_active":"2024-05-30T16:54:56.654959+0000","last_became_peered":"2024-05-30T16:54:56.654959+0000","last_unstale":"2024-05-30T16:54:57.747598+0000","last_undegraded":"2024-05-30T16:54:57.747598+0000","last_fullsized":"2024-05-30T16:54:57.747598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.799865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743406+0000","last_change":"2024-05-30T16:54:55.629963+0000","last_active":"2024-05-30T16:54:57.743406+0000","last_peered":"2024-05-30T16:54:57.743406+0000","last_clean":"2024-05-30T16:54:57.743406+0000","last_became_active":"2024-05-30T16:54:55.629607+0000","last_became_peered":"2024-05-30T16:54:55.629607+0000","last_unstale":"2024-05-30T16:54:57.743406+0000","last_undegraded":"2024-05-30T16:54:57.743406+0000","last_fullsized":"2024-05-30T16:54:57.743406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:19:28.370103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743716+0000","last_change":"2024-05-30T16:54:56.654299+0000","last_active":"2024-05-30T16:54:57.743716+0000","last_peered":"2024-05-30T16:54:57.743716+0000","last_clean":"2024-05-30T16:54:57.743716+0000","last_became_active":"2024-05-30T16:54:56.652304+0000","last_became_peered":"2024-05-30T16:54:56.652304+0000","last_unstale":"2024-05-30T16:54:57.743716+0000","last_undegraded":"2024-05-30T16:54:57.743716+0000","last_fullsized":"2024-05-30T16:54:57.743716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:22:02.453084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.743490+0000","last_change":"2024-05-30T16:54:55.611765+0000","last_active":"2024-05-30T16:54:57.743490+0000","last_peered":"2024-05-30T16:54:57.743490+0000","last_clean":"2024-05-30T16:54:57.743490+0000","last_became_active":"2024-05-30T16:54:55.611426+0000","last_became_peered":"2024-05-30T16:54:55.611426+0000","last_unstale":"2024-05-30T16:54:57.743490+0000","last_undegraded":"2024-05-30T16:54:57.743490+0000","last_fullsized":"2024-05-30T16:54:57.743490+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:47:13.879124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652216+0000","last_change":"2024-05-30T16:54:56.656597+0000","last_active":"2024-05-30T16:54:57.652216+0000","last_peered":"2024-05-30T16:54:57.652216+0000","last_clean":"2024-05-30T16:54:57.652216+0000","last_became_active":"2024-05-30T16:54:56.656321+0000","last_became_peered":"2024-05-30T16:54:56.656321+0000","last_unstale":"2024-05-30T16:54:57.652216+0000","last_undegraded":"2024-05-30T16:54:57.652216+0000","last_fullsized":"2024-05-30T16:54:57.652216+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:12:08.792991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649028+0000","last_change":"2024-05-30T16:54:55.621069+0000","last_active":"2024-05-30T16:54:57.649028+0000","last_peered":"2024-05-30T16:54:57.649028+0000","last_clean":"2024-05-30T16:54:57.649028+0000","last_became_active":"2024-05-30T16:54:55.620559+0000","last_became_peered":"2024-05-30T16:54:55.620559+0000","last_unstale":"2024-05-30T16:54:57.649028+0000","last_undegraded":"2024-05-30T16:54:57.649028+0000","last_fullsized":"2024-05-30T16:54:57.649028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:06:35.918541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.649896+0000","last_change":"2024-05-30T16:54:56.648552+0000","last_active":"2024-05-30T16:54:57.649896+0000","last_peered":"2024-05-30T16:54:57.649896+0000","last_clean":"2024-05-30T16:54:57.649896+0000","last_became_active":"2024-05-30T16:54:56.648264+0000","last_became_peered":"2024-05-30T16:54:56.648264+0000","last_unstale":"2024-05-30T16:54:57.649896+0000","last_undegraded":"2024-05-30T16:54:57.649896+0000","last_fullsized":"2024-05-30T16:54:57.649896+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:03:12.660623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744835+0000","last_change":"2024-05-30T16:54:55.629713+0000","last_active":"2024-05-30T16:54:57.744835+0000","last_peered":"2024-05-30T16:54:57.744835+0000","last_clean":"2024-05-30T16:54:57.744835+0000","last_became_active":"2024-05-30T16:54:55.629182+0000","last_became_peered":"2024-05-30T16:54:55.629182+0000","last_unstale":"2024-05-30T16:54:57.744835+0000","last_undegraded":"2024-05-30T16:54:57.744835+0000","last_fullsized":"2024-05-30T16:54:57.744835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:20:43.048793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.651562+0000","last_change":"2024-05-30T16:54:56.662049+0000","last_active":"2024-05-30T16:54:57.651562+0000","last_peered":"2024-05-30T16:54:57.651562+0000","last_clean":"2024-05-30T16:54:57.651562+0000","last_became_active":"2024-05-30T16:54:56.661686+0000","last_became_peered":"2024-05-30T16:54:56.661686+0000","last_unstale":"2024-05-30T16:54:57.651562+0000","last_undegraded":"2024-05-30T16:54:57.651562+0000","last_fullsized":"2024-05-30T16:54:57.651562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:28.212134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.651696+0000","last_change":"2024-05-30T16:54:55.623173+0000","last_active":"2024-05-30T16:54:57.651696+0000","last_peered":"2024-05-30T16:54:57.651696+0000","last_clean":"2024-05-30T16:54:57.651696+0000","last_became_active":"2024-05-30T16:54:55.622874+0000","last_became_peered":"2024-05-30T16:54:55.622874+0000","last_unstale":"2024-05-30T16:54:57.651696+0000","last_undegraded":"2024-05-30T16:54:57.651696+0000","last_fullsized":"2024-05-30T16:54:57.651696+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:53:36.725881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.744975+0000","last_change":"2024-05-30T16:54:56.666000+0000","last_active":"2024-05-30T16:54:57.744975+0000","last_peered":"2024-05-30T16:54:57.744975+0000","last_clean":"2024-05-30T16:54:57.744975+0000","last_became_active":"2024-05-30T16:54:56.665688+0000","last_became_peered":"2024-05-30T16:54:56.665688+0000","last_unstale":"2024-05-30T16:54:57.744975+0000","last_undegraded":"2024-05-30T16:54:57.744975+0000","last_fullsized":"2024-05-30T16:54:57.744975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:55.580886+0000","last_clean_scrub_stamp":"2024-05-30T16:54:55.580886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:41:30.857494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.652351+0000","last_change":"2024-05-30T16:54:55.620733+0000","last_active":"2024-05-30T16:54:57.652351+0000","last_peered":"2024-05-30T16:54:57.652351+0000","last_clean":"2024-05-30T16:54:57.652351+0000","last_became_active":"2024-05-30T16:54:55.620460+0000","last_became_peered":"2024-05-30T16:54:55.620460+0000","last_unstale":"2024-05-30T16:54:57.652351+0000","last_undegraded":"2024-05-30T16:54:57.652351+0000","last_fullsized":"2024-05-30T16:54:57.652351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:54:54.570978+0000","last_clean_scrub_stamp":"2024-05-30T16:54:54.570978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:23:27.426633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":272,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-30T16:54:57.746749+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:54:57.746749+0000","last_peered":"2024-05-30T16:54:57.746749+0000","last_clean":"2024-05-30T16:54:57.746749+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:54:57.746749+0000","last_undegraded":"2024-05-30T16:54:57.746749+0000","last_fullsized":"2024-05-30T16:54:57.746749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509579,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54268,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317572,"statfs":{"total":96636764160,"available":96581193728,"internally_reserved":0,"allocated":901120,"data_stored":380847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509579,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58428,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313412,"statfs":{"total":96636764160,"available":96576933888,"internally_reserved":0,"allocated":897024,"data_stored":380825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58428,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313412,"statfs":{"total":96636764160,"available":96576933888,"internally_reserved":0,"allocated":901120,"data_stored":380867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509580,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50116,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321724,"statfs":{"total":96636764160,"available":96585445376,"internally_reserved":0,"allocated":905216,"data_stored":382367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58440,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313400,"statfs":{"total":96636764160,"available":96576921600,"internally_reserved":0,"allocated":913408,"data_stored":382983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542282,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54848,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316992,"statfs":{"total":96636764160,"available":96580599808,"internally_reserved":0,"allocated":1495040,"data_stored":971739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542283,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54272,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317568,"statfs":{"total":96636764160,"available":96581189632,"internally_reserved":0,"allocated":897024,"data_stored":380845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59000,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312840,"statfs":{"total":96636764160,"available":96576348160,"internally_reserved":0,"allocated":1486848,"data_stored":971123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:55:01.648 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T16:55:01.649 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-30T16:55:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.912+0000 7f34c9200640 1 Processor -- start 2024-05-30T16:55:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.912+0000 7f34c9200640 1 -- start start 2024-05-30T16:55:01.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.913+0000 7f34c9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40ce1f0 0x7f34c40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.913+0000 7f34c9200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f34c40ce5d0 0x7f34c415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.913+0000 7f34c9200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34c40b3d60 0x7f34c40b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.913+0000 7f34c9200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f34c40b3580 con 0x7f34c40b3d60 2024-05-30T16:55:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.913+0000 7f34c9200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f34c415f490 con 0x7f34c40ce5d0 2024-05-30T16:55:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.913+0000 7f34c9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34c415f770 con 0x7f34c40ce1f0 2024-05-30T16:55:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.914+0000 7f34c3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34c40b3d60 0x7f34c40b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.914+0000 7f34c3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34c40b3d60 0x7f34c40b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48154/0 (socket says 172.21.15.67:48154) 2024-05-30T16:55:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.914+0000 7f34c3e00640 1 -- 172.21.15.67:0/655028655 learned_addr learned my addr 172.21.15.67:0/655028655 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.914+0000 7f34c3400640 1 --2- 172.21.15.67:0/655028655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40ce1f0 0x7f34c40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.914+0000 7f34bbe00640 1 --2- 172.21.15.67:0/655028655 >> v2:172.21.15.169:3300/0 conn(0x7f34c40ce5d0 0x7f34c415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.914+0000 7f34c3e00640 1 -- 172.21.15.67:0/655028655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40ce1f0 msgr2=0x7f34c40b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.915+0000 7f34c3e00640 1 --2- 172.21.15.67:0/655028655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40ce1f0 0x7f34c40b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.915+0000 7f34c3e00640 1 -- 172.21.15.67:0/655028655 >> v2:172.21.15.169:3300/0 conn(0x7f34c40ce5d0 msgr2=0x7f34c415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.915+0000 7f34c3e00640 1 --2- 172.21.15.67:0/655028655 >> v2:172.21.15.169:3300/0 conn(0x7f34c40ce5d0 0x7f34c415ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.915+0000 7f34c3e00640 1 -- 172.21.15.67:0/655028655 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c415fa50 con 0x7f34c40b3d60 2024-05-30T16:55:01.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.916+0000 7f34c3e00640 1 --2- 172.21.15.67:0/655028655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40b3d60 0x7f34c40b41d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f34c40d11c0 tx=0x7f34bc02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7361d6796b09ea38 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:01.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.916+0000 7f34c2a00640 1 -- 172.21.15.67:0/655028655 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc036070 con 0x7f34c40b3d60 2024-05-30T16:55:01.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.916+0000 7f34c2a00640 1 -- 172.21.15.67:0/655028655 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc031600 con 0x7f34c40b3d60 2024-05-30T16:55:01.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.916+0000 7f34c2a00640 1 -- 172.21.15.67:0/655028655 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0319d0 con 0x7f34c40b3d60 2024-05-30T16:55:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.917+0000 7f34c9200640 1 -- 172.21.15.67:0/655028655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40b3d60 msgr2=0x7f34c40b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.917+0000 7f34c9200640 1 --2- 172.21.15.67:0/655028655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40b3d60 0x7f34c40b41d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f34c40d11c0 tx=0x7f34bc02a570 comp rx=0 tx=0).stop 2024-05-30T16:55:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.917+0000 7f34c9200640 1 -- 172.21.15.67:0/655028655 shutdown_connections 2024-05-30T16:55:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.917+0000 7f34c9200640 1 --2- 172.21.15.67:0/655028655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40b3d60 0x7f34c40b41d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.917+0000 7f34c9200640 1 --2- 172.21.15.67:0/655028655 >> v2:172.21.15.169:3300/0 conn(0x7f34c40ce5d0 0x7f34c415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.917+0000 7f34c9200640 1 --2- 172.21.15.67:0/655028655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40ce1f0 0x7f34c40b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.917+0000 7f34c9200640 1 -- 172.21.15.67:0/655028655 >> 172.21.15.67:0/655028655 conn(0x7f34c406b900 msgr2=0x7f34c406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.917+0000 7f34c9200640 1 -- 172.21.15.67:0/655028655 shutdown_connections 2024-05-30T16:55:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.917+0000 7f34c9200640 1 -- 172.21.15.67:0/655028655 wait complete. 2024-05-30T16:55:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.919+0000 7f34c9200640 1 Processor -- start 2024-05-30T16:55:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.920+0000 7f34c9200640 1 -- start start 2024-05-30T16:55:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.920+0000 7f34c9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40b3d60 0x7f34c415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.920+0000 7f34c9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40ce1f0 0x7f34c41e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34c9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40ce5d0 0x7f34c41471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34c9200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34c40ccb90 con 0x7f34c40ce5d0 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34c9200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34c415f180 con 0x7f34c40ce1f0 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34c9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34c40ccd10 con 0x7f34c40b3d60 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40ce5d0 0x7f34c41471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40ce5d0 0x7f34c41471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48164/0 (socket says 172.21.15.67:48164) 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34c3e00640 1 -- 172.21.15.67:0/327811112 learned_addr learned my addr 172.21.15.67:0/327811112 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34bbe00640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40ce1f0 0x7f34c41e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34c3400640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40b3d60 0x7f34c415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.921+0000 7f34c3e00640 1 -- 172.21.15.67:0/327811112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40b3d60 msgr2=0x7f34c415bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34c3e00640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40b3d60 0x7f34c415bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34c3e00640 1 -- 172.21.15.67:0/327811112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40ce1f0 msgr2=0x7f34c41e8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34c3e00640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40ce1f0 0x7f34c41e8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34c3e00640 1 -- 172.21.15.67:0/327811112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c415aa60 con 0x7f34c40ce5d0 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34c3400640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40b3d60 0x7f34c415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34c3e00640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40ce5d0 0x7f34c41471c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009f40 tx=0x7f34bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34bb400640 1 -- 172.21.15.67:0/327811112 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc036050 con 0x7f34c40ce5d0 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34bb400640 1 -- 172.21.15.67:0/327811112 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc042070 con 0x7f34c40ce5d0 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34bb400640 1 -- 172.21.15.67:0/327811112 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc007bd0 con 0x7f34c40ce5d0 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.922+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c415ad40 con 0x7f34c40ce5d0 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.923+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c415b300 con 0x7f34c40ce5d0 2024-05-30T16:55:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.924+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c40689e0 con 0x7f34c40ce5d0 2024-05-30T16:55:01.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.925+0000 7f34bb400640 1 -- 172.21.15.67:0/327811112 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f34bc047460 con 0x7f34c40ce5d0 2024-05-30T16:55:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.926+0000 7f34bb400640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34a006d5e0 0x7f34a006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.926+0000 7f34bb400640 1 -- 172.21.15.67:0/327811112 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f34bc0ab530 con 0x7f34c40ce5d0 2024-05-30T16:55:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.927+0000 7f34c3400640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34a006d5e0 0x7f34a006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.928+0000 7f34c3400640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34a006d5e0 0x7f34a006faa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f34b4006fd0 tx=0x7f34b4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:01.932+0000 7f34bb400640 1 -- 172.21.15.67:0/327811112 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34bc045360 con 0x7f34c40ce5d0 2024-05-30T16:55:02.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.209+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f34c406fbf0 con 0x7f34c40ce5d0 2024-05-30T16:55:02.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.214+0000 7f34bb400640 1 -- 172.21.15.67:0/327811112 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f34bc07b3b0 con 0x7f34c40ce5d0 2024-05-30T16:55:02.214 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-05-30T16:55:02.214 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T16:55:02.215 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-05-30T16:55:02.215 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-05-30T16:55:02.215 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-30T16:55:02.215 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-05-30T16:55:02.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34a006d5e0 msgr2=0x7f34a006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:02.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34a006d5e0 0x7f34a006faa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f34b4006fd0 tx=0x7f34b4008040 comp rx=0 tx=0).stop 2024-05-30T16:55:02.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40ce5d0 msgr2=0x7f34c41471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40ce5d0 0x7f34c41471c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009f40 tx=0x7f34bc004a40 comp rx=0 tx=0).stop 2024-05-30T16:55:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 shutdown_connections 2024-05-30T16:55:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34a006d5e0 0x7f34a006faa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c40ce5d0 0x7f34c41471c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40ce1f0 0x7f34c41e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 --2- 172.21.15.67:0/327811112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c40b3d60 0x7f34c415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.216+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 >> 172.21.15.67:0/327811112 conn(0x7f34c406b900 msgr2=0x7f34c406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.217+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 shutdown_connections 2024-05-30T16:55:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.217+0000 7f34c9200640 1 -- 172.21.15.67:0/327811112 wait complete. 2024-05-30T16:55:02.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:55:02.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.447+0000 7f752b200640 1 Processor -- start 2024-05-30T16:55:02.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.447+0000 7f752b200640 1 -- start start 2024-05-30T16:55:02.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.449+0000 7f752b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f752c073310 0x7f752c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:02.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.449+0000 7f752b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f752c06dc20 0x7f752c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:02.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.449+0000 7f752b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752c0afd60 con 0x7f752c15b3a0 2024-05-30T16:55:02.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.449+0000 7f752b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f752c06f980 con 0x7f752c06dc20 2024-05-30T16:55:02.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.449+0000 7f752b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f752c0ae0c0 con 0x7f752c073310 2024-05-30T16:55:02.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.449+0000 7f752a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f752c06dc20 0x7f752c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:02.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.449+0000 7f752a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f752c06dc20 0x7f752c15f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48170/0 (socket says 172.21.15.67:48170) 2024-05-30T16:55:02.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.449+0000 7f752a800640 1 -- 172.21.15.67:0/1181507130 learned_addr learned my addr 172.21.15.67:0/1181507130 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:02.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.450+0000 7f7529400640 1 --2- 172.21.15.67:0/1181507130 >> v2:172.21.15.169:3300/0 conn(0x7f752c073310 0x7f752c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:02.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.450+0000 7f752a800640 1 -- 172.21.15.67:0/1181507130 >> v2:172.21.15.169:3300/0 conn(0x7f752c073310 msgr2=0x7f752c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:02.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.450+0000 7f752a800640 1 --2- 172.21.15.67:0/1181507130 >> v2:172.21.15.169:3300/0 conn(0x7f752c073310 0x7f752c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.450+0000 7f752a800640 1 -- 172.21.15.67:0/1181507130 >> v1:172.21.15.169:6789/0 conn(0x7f752c15b3a0 legacy=0x7f752c072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:02.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.450+0000 7f752a800640 1 -- 172.21.15.67:0/1181507130 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c0af7a0 con 0x7f752c06dc20 2024-05-30T16:55:02.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.451+0000 7f752a800640 1 --2- 172.21.15.67:0/1181507130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c06dc20 0x7f752c15f9a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f752c0711f0 tx=0x7f752402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55aa90cb8b675bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:02.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.451+0000 7f7528a00640 1 -- 172.21.15.67:0/1181507130 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524036070 con 0x7f752c06dc20 2024-05-30T16:55:02.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.451+0000 7f7528a00640 1 -- 172.21.15.67:0/1181507130 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7524031600 con 0x7f752c06dc20 2024-05-30T16:55:02.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.451+0000 7f7528a00640 1 -- 172.21.15.67:0/1181507130 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75240319d0 con 0x7f752c06dc20 2024-05-30T16:55:02.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.452+0000 7f752b200640 1 -- 172.21.15.67:0/1181507130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c06dc20 msgr2=0x7f752c15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:02.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.452+0000 7f752b200640 1 --2- 172.21.15.67:0/1181507130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c06dc20 0x7f752c15f9a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f752c0711f0 tx=0x7f752402a6f0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.452+0000 7f752b200640 1 -- 172.21.15.67:0/1181507130 shutdown_connections 2024-05-30T16:55:02.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.452+0000 7f752b200640 1 --2- 172.21.15.67:0/1181507130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c06dc20 0x7f752c15f9a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.452+0000 7f752b200640 1 --2- 172.21.15.67:0/1181507130 >> v2:172.21.15.169:3300/0 conn(0x7f752c073310 0x7f752c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.452+0000 7f752b200640 1 -- 172.21.15.67:0/1181507130 >> 172.21.15.67:0/1181507130 conn(0x7f752c0b1fc0 msgr2=0x7f752c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:02.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.452+0000 7f752b200640 1 -- 172.21.15.67:0/1181507130 shutdown_connections 2024-05-30T16:55:02.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.452+0000 7f752b200640 1 -- 172.21.15.67:0/1181507130 wait complete. 2024-05-30T16:55:02.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.454+0000 7f752b200640 1 Processor -- start 2024-05-30T16:55:02.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.454+0000 7f752b200640 1 -- start start 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.455+0000 7f752b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c06dc20 0x7f752c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.455+0000 7f752b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c073310 0x7f752c15d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.455+0000 7f752b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c15b3a0 0x7f752c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.455+0000 7f752b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f752c15b780 con 0x7f752c15b3a0 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.455+0000 7f752b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f752c15fee0 con 0x7f752c06dc20 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.455+0000 7f752b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f752c0ae550 con 0x7f752c073310 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.455+0000 7f752a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c15b3a0 0x7f752c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.455+0000 7f752a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c15b3a0 0x7f752c0aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48182/0 (socket says 172.21.15.67:48182) 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.455+0000 7f752a800640 1 -- 172.21.15.67:0/655624997 learned_addr learned my addr 172.21.15.67:0/655624997 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f7529e00640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c06dc20 0x7f752c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f7529400640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c073310 0x7f752c15d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f752a800640 1 -- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c073310 msgr2=0x7f752c15d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f752a800640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c073310 0x7f752c15d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f752a800640 1 -- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c06dc20 msgr2=0x7f752c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f752a800640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c06dc20 0x7f752c0cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f752a800640 1 -- 172.21.15.67:0/655624997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c15e280 con 0x7f752c15b3a0 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f7529e00640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c06dc20 0x7f752c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f7529400640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c073310 0x7f752c15d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f752a800640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c15b3a0 0x7f752c0aff30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7524034040 tx=0x7f7524004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f7522000640 1 -- 172.21.15.67:0/655624997 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524036050 con 0x7f752c15b3a0 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f7522000640 1 -- 172.21.15.67:0/655624997 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7524042070 con 0x7f752c15b3a0 2024-05-30T16:55:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f7522000640 1 -- 172.21.15.67:0/655624997 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524009510 con 0x7f752c15b3a0 2024-05-30T16:55:02.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.456+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c15e560 con 0x7f752c15b3a0 2024-05-30T16:55:02.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.457+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c0cf2f0 con 0x7f752c15b3a0 2024-05-30T16:55:02.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.458+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c0689e0 con 0x7f752c15b3a0 2024-05-30T16:55:02.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.459+0000 7f7522000640 1 -- 172.21.15.67:0/655624997 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f7524007bd0 con 0x7f752c15b3a0 2024-05-30T16:55:02.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.460+0000 7f7522000640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f750c06d5e0 0x7f750c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:02.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.460+0000 7f7522000640 1 -- 172.21.15.67:0/655624997 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f75240ab3f0 con 0x7f752c15b3a0 2024-05-30T16:55:02.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.460+0000 7f7529e00640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f750c06d5e0 0x7f750c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:02.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.461+0000 7f7529e00640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f750c06d5e0 0x7f750c06faa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f751c00a830 tx=0x7f751c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:02.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.464+0000 7f7522000640 1 -- 172.21.15.67:0/655624997 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f752407b270 con 0x7f752c15b3a0 2024-05-30T16:55:02.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.711+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f752c0b2e90 con 0x7f752c15b3a0 2024-05-30T16:55:02.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.712+0000 7f7522000640 1 -- 172.21.15.67:0/655624997 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5189 (secure 0 0 0) 0x7f752407ab70 con 0x7f752c15b3a0 2024-05-30T16:55:02.713 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:55:02.713 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5171,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1912096869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1912096869},{"type":"v1","addr":"172.21.15.169:6835","nonce":1912096869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5608,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3963396329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3963396329},{"type":"v1","addr":"172.21.15.67:6835","nonce":3963396329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2169821644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2169821644},{"type":"v1","addr":"172.21.15.169:6834","nonce":2169821644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:54:56.078869+0000","modified":"2024-05-30T16:54:59.175686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6558":{"gid":6558,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/2706164583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2706164583},{"type":"v1","addr":"172.21.15.67:6837","nonce":2706164583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6558,"qdb_cluster":[6558]},"id":5}]} 2024-05-30T16:55:02.713 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 45 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f750c06d5e0 msgr2=0x7f750c06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f750c06d5e0 0x7f750c06faa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f751c00a830 tx=0x7f751c008040 comp rx=0 tx=0).stop 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c15b3a0 msgr2=0x7f752c0aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c15b3a0 0x7f752c0aff30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7524034040 tx=0x7f7524004a40 comp rx=0 tx=0).stop 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 shutdown_connections 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f750c06d5e0 0x7f750c06faa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f752c15b3a0 0x7f752c0aff30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f752c073310 0x7f752c15d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 --2- 172.21.15.67:0/655624997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f752c06dc20 0x7f752c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:02.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 >> 172.21.15.67:0/655624997 conn(0x7f752c0b1fc0 msgr2=0x7f752c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:02.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 shutdown_connections 2024-05-30T16:55:02.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:02.715+0000 7f752b200640 1 -- 172.21.15.67:0/655624997 wait complete. 2024-05-30T16:55:03.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-30T16:54:56.078869+0000', 'modified': '2024-05-30T16:54:59.175686+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6558}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6558': {'gid': 6558, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.67:6837/2706164583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6836', 'nonce': 2706164583}, {'type': 'v1', 'addr': '172.21.15.67:6837', 'nonce': 2706164583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6558, 'qdb_cluster': [6558]} 2024-05-30T16:55:03.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-30T16:55:03.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-30T16:55:03.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.363+0000 7fb707200640 1 Processor -- start 2024-05-30T16:55:03.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.363+0000 7fb707200640 1 -- start start 2024-05-30T16:55:03.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb707200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb708073310 0x7fb7080b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb707200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb70806dc20 0x7fb70815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb707200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7080afd60 con 0x7fb70815b3a0 2024-05-30T16:55:03.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb707200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb70806f980 con 0x7fb70806dc20 2024-05-30T16:55:03.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb707200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb7080ae0c0 con 0x7fb708073310 2024-05-30T16:55:03.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb706800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb70806dc20 0x7fb70815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb706800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb70806dc20 0x7fb70815f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48194/0 (socket says 172.21.15.67:48194) 2024-05-30T16:55:03.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb706800640 1 -- 172.21.15.67:0/3522594265 learned_addr learned my addr 172.21.15.67:0/3522594265 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:03.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb706800640 1 -- 172.21.15.67:0/3522594265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb708073310 msgr2=0x7fb7080b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb706800640 1 --2- 172.21.15.67:0/3522594265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb708073310 0x7fb7080b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb706800640 1 -- 172.21.15.67:0/3522594265 >> v1:172.21.15.169:6789/0 conn(0x7fb70815b3a0 legacy=0x7fb708072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.365+0000 7fb706800640 1 -- 172.21.15.67:0/3522594265 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7080af7a0 con 0x7fb70806dc20 2024-05-30T16:55:03.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.367+0000 7fb706800640 1 --2- 172.21.15.67:0/3522594265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb70806dc20 0x7fb70815f9a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb700009870 tx=0x7fb70002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982286bc24efcf96 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.367+0000 7fb704a00640 1 -- 172.21.15.67:0/3522594265 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700037070 con 0x7fb70806dc20 2024-05-30T16:55:03.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.367+0000 7fb704a00640 1 -- 172.21.15.67:0/3522594265 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb700032440 con 0x7fb70806dc20 2024-05-30T16:55:03.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.367+0000 7fb704a00640 1 -- 172.21.15.67:0/3522594265 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700032810 con 0x7fb70806dc20 2024-05-30T16:55:03.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.368+0000 7fb707200640 1 -- 172.21.15.67:0/3522594265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb70806dc20 msgr2=0x7fb70815f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.368+0000 7fb707200640 1 --2- 172.21.15.67:0/3522594265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb70806dc20 0x7fb70815f9a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb700009870 tx=0x7fb70002a200 comp rx=0 tx=0).stop 2024-05-30T16:55:03.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.368+0000 7fb707200640 1 -- 172.21.15.67:0/3522594265 shutdown_connections 2024-05-30T16:55:03.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.368+0000 7fb707200640 1 --2- 172.21.15.67:0/3522594265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb70806dc20 0x7fb70815f9a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.368+0000 7fb707200640 1 --2- 172.21.15.67:0/3522594265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb708073310 0x7fb7080b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.368+0000 7fb707200640 1 -- 172.21.15.67:0/3522594265 >> 172.21.15.67:0/3522594265 conn(0x7fb7080b1fc0 msgr2=0x7fb7080b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:03.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.368+0000 7fb707200640 1 -- 172.21.15.67:0/3522594265 shutdown_connections 2024-05-30T16:55:03.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.368+0000 7fb707200640 1 -- 172.21.15.67:0/3522594265 wait complete. 2024-05-30T16:55:03.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.370+0000 7fb707200640 1 Processor -- start 2024-05-30T16:55:03.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.370+0000 7fb707200640 1 -- start start 2024-05-30T16:55:03.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.371+0000 7fb707200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb70806dc20 0x7fb70815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.371+0000 7fb707200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb708073310 0x7fb7080ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.371+0000 7fb707200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb70815b3a0 0x7fb70809d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.371+0000 7fb707200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb70815b780 con 0x7fb708073310 2024-05-30T16:55:03.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.371+0000 7fb707200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb70815fee0 con 0x7fb70806dc20 2024-05-30T16:55:03.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.371+0000 7fb707200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb7080ae550 con 0x7fb70815b3a0 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.371+0000 7fb705400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb708073310 0x7fb7080ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.371+0000 7fb705400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb708073310 0x7fb7080ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48196/0 (socket says 172.21.15.67:48196) 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.371+0000 7fb705400640 1 -- 172.21.15.67:0/1051620663 learned_addr learned my addr 172.21.15.67:0/1051620663 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb706800640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb70815b3a0 0x7fb70809d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb705e00640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb70806dc20 0x7fb70815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb705400640 1 -- 172.21.15.67:0/1051620663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb70815b3a0 msgr2=0x7fb70809d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb705400640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb70815b3a0 0x7fb70809d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb705400640 1 -- 172.21.15.67:0/1051620663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb70806dc20 msgr2=0x7fb70815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb705400640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb70806dc20 0x7fb70815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb705400640 1 -- 172.21.15.67:0/1051620663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70815cf00 con 0x7fb708073310 2024-05-30T16:55:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb705e00640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb70806dc20 0x7fb70815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb705400640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb708073310 0x7fb7080ce800 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b4a0 tx=0x7fb6f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb6fe000640 1 -- 172.21.15.67:0/1051620663 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4014070 con 0x7fb708073310 2024-05-30T16:55:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb6fe000640 1 -- 172.21.15.67:0/1051620663 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f4010610 con 0x7fb708073310 2024-05-30T16:55:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb707200640 1 -- 172.21.15.67:0/1051620663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70815d1e0 con 0x7fb708073310 2024-05-30T16:55:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.372+0000 7fb6fe000640 1 -- 172.21.15.67:0/1051620663 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4010a00 con 0x7fb708073310 2024-05-30T16:55:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.373+0000 7fb707200640 1 -- 172.21.15.67:0/1051620663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70815d7a0 con 0x7fb708073310 2024-05-30T16:55:03.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.375+0000 7fb6fe000640 1 -- 172.21.15.67:0/1051620663 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fb6f4017070 con 0x7fb708073310 2024-05-30T16:55:03.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.376+0000 7fb6fe000640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb6e806d610 0x7fb6e806fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.376+0000 7fb6fe000640 1 -- 172.21.15.67:0/1051620663 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fb6f408a020 con 0x7fb708073310 2024-05-30T16:55:03.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.376+0000 7fb70fd8e740 1 Processor -- start 2024-05-30T16:55:03.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.376+0000 7fb705e00640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb6e806d610 0x7fb6e806fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.376+0000 7fb70fd8e740 1 -- start start 2024-05-30T16:55:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.377+0000 7fb705e00640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb6e806d610 0x7fb6e806fad0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8007e90 tx=0x7fb6f80073a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.377+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x5623336000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.377+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562333601090 0x5623335f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.377+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e1d30 0x5623335e21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.377+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56233351eb10 con 0x5623335ffcc0 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.377+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5623334c52b0 con 0x5623335e1d30 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.377+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56233351aad0 con 0x562333601090 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb705e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x5623336000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb705e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x5623336000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48210/0 (socket says 172.21.15.67:48210) 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb705e00640 1 -- 172.21.15.67:0/3791158113 learned_addr learned my addr 172.21.15.67:0/3791158113 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb706800640 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e1d30 0x5623335e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb705400640 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562333601090 0x5623335f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb705e00640 1 -- 172.21.15.67:0/3791158113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562333601090 msgr2=0x5623335f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb705e00640 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562333601090 0x5623335f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb705e00640 1 -- 172.21.15.67:0/3791158113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e1d30 msgr2=0x5623335e21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb705e00640 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e1d30 0x5623335e21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb705e00640 1 -- 172.21.15.67:0/3791158113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623332312d0 con 0x5623335ffcc0 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.378+0000 7fb706800640 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e1d30 0x5623335e21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.379+0000 7fb705e00640 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x5623336000a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800f5b0 tx=0x7fb6f800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b4154c20f706e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.379+0000 7fb6f2a00640 1 -- 172.21.15.67:0/3791158113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8003970 con 0x5623335ffcc0 2024-05-30T16:55:03.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.379+0000 7fb6f2a00640 1 -- 172.21.15.67:0/3791158113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f8015030 con 0x5623335ffcc0 2024-05-30T16:55:03.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.379+0000 7fb6f2a00640 1 -- 172.21.15.67:0/3791158113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8003d50 con 0x5623335ffcc0 2024-05-30T16:55:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.379+0000 7fb70fd8e740 1 -- 172.21.15.67:0/3791158113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 msgr2=0x5623336000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.379+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x5623336000a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800f5b0 tx=0x7fb6f800f000 comp rx=0 tx=0).stop 2024-05-30T16:55:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.380+0000 7fb70fd8e740 1 -- 172.21.15.67:0/3791158113 shutdown_connections 2024-05-30T16:55:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.380+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e1d30 0x5623335e21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.380+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562333601090 0x5623335f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.380+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/3791158113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x5623336000a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.380+0000 7fb70fd8e740 1 -- 172.21.15.67:0/3791158113 >> 172.21.15.67:0/3791158113 conn(0x5623335b7470 msgr2=0x5623335b8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.380+0000 7fb70fd8e740 1 -- 172.21.15.67:0/3791158113 shutdown_connections 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.380+0000 7fb70fd8e740 1 -- 172.21.15.67:0/3791158113 wait complete. 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.380+0000 7fb70fd8e740 1 -- start start 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.381+0000 7fb70fd8e740 1 Processor -- start 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 0x5623335e2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 0x5623335fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335e7110 0x5623335e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5623334d8570 con 0x5623335e7110 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5623335fde40 con 0x562333601090 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5623335fbca0 con 0x5623335ffcc0 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb70fd8e740 10 client.0 _ll_register_callbacks cb 0x5623334a6fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:55:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb706800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335e7110 0x5623335e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb706800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335e7110 0x5623335e3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48214/0 (socket says 172.21.15.67:48214) 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb706800640 1 -- 172.21.15.67:0/2526389823 learned_addr learned my addr 172.21.15.67:0/2526389823 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb705400640 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 0x5623335fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb705e00640 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 0x5623335e2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb706800640 1 -- 172.21.15.67:0/2526389823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 msgr2=0x5623335e2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb706800640 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 0x5623335e2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.382+0000 7fb706800640 1 -- 172.21.15.67:0/2526389823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 msgr2=0x5623335fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.383+0000 7fb706800640 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 0x5623335fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.383+0000 7fb706800640 1 -- 172.21.15.67:0/2526389823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623332fb6b0 con 0x5623335e7110 2024-05-30T16:55:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.383+0000 7fb705400640 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 0x5623335fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:03.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.383+0000 7fb706800640 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335e7110 0x5623335e3da0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb700009990 tx=0x7fb700004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b887920ab6ef260 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.383+0000 7fb6e7400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.383+0000 7fb6e7400640 1 -- 172.21.15.67:0/2526389823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700043070 con 0x5623335e7110 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.383+0000 7fb6e7400640 1 -- 172.21.15.67:0/2526389823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fb700032d50 con 0x5623335e7110 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.383+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2526389823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f8015030 con 0x5623335e7110 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb6e7400640 1 -- 172.21.15.67:0/2526389823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb70004e050 con 0x5623335e7110 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb70fd8e740 10 client.6681 fetch_fsmap learned FSMap version 45 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2526389823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x5623335fb760 con 0x5623335e7110 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb6e7400640 1 -- 172.21.15.67:0/2526389823 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f8015030 con 0x5623335e7110 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb6e7400640 1 -- 172.21.15.67:0/2526389823 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7fb700051020 con 0x5623335e7110 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb70fd8e740 10 client.6681 fetch_fsmap finished waiting for FSMap version 45 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb70fd8e740 10 client.6681 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb70fd8e740 20 client.6681 populate_metadata read hostname 'smithi067' 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] conn(0x5623335e4b10 0x562333625f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb70fd8e740 4 client.6681 mds_command: new command op to 6558 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.384+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2526389823 --> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56233352c530 con 0x5623335e4b10 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.385+0000 7fb705e00640 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] conn(0x5623335e4b10 0x562333625f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.385+0000 7fb705e00640 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] conn(0x5623335e4b10 0x562333625f40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93470acc6d12273a server_cookie=35af7eba5294cbc7 in_seq=0 out_seq=0 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.385+0000 7fb6e7400640 10 client.6681 ms_handle_connect on v2:172.21.15.67:6836/2706164583 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.386+0000 7fb6e7400640 1 -- 172.21.15.67:0/2526389823 <== mds.0 v2:172.21.15.67:6836/2706164583 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56233352c530 con 0x5623335e4b10 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.386+0000 7fb6e7400640 10 client.6681 handle_command_reply: tid=0 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.386+0000 7fb70fd8e740 1 client.6681 shutdown 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.386+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2526389823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335e7110 msgr2=0x5623335e3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.386+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335e7110 0x5623335e3da0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb700009990 tx=0x7fb700004730 comp rx=0 tx=0).stop 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2526389823 shutdown_connections 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] conn(0x5623335e4b10 0x562333625f40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335e7110 0x5623335e3da0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 0x5623335fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2526389823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 0x5623335e2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb6e7400640 0 client.6681 ms_handle_reset on v2:172.21.15.67:6836/2706164583 2024-05-30T16:55:03.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2526389823 >> 172.21.15.67:0/2526389823 conn(0x5623335e1d30 msgr2=0x5623335fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:03.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2526389823 shutdown_connections 2024-05-30T16:55:03.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2526389823 wait complete. 2024-05-30T16:55:03.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.387+0000 7fb70fd8e740 20 client.6681 trim_cache size 0 max 16384 2024-05-30T16:55:03.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.407+0000 7fb70fd8e740 1 Processor -- start 2024-05-30T16:55:03.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.407+0000 7fb70fd8e740 1 -- start start 2024-05-30T16:55:03.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.408+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e7110 0x5623336000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.408+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 0x5623335f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.408+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562333601090 0x5623335f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.408+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb70809be80 con 0x562333601090 2024-05-30T16:55:03.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.408+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb70809b1e0 con 0x5623335e7110 2024-05-30T16:55:03.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.408+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb70809b830 con 0x5623335ffcc0 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.408+0000 7fb706800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562333601090 0x5623335f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.408+0000 7fb706800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562333601090 0x5623335f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48220/0 (socket says 172.21.15.67:48220) 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.408+0000 7fb706800640 1 -- 172.21.15.67:0/2428598072 learned_addr learned my addr 172.21.15.67:0/2428598072 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb705400640 1 --2- 172.21.15.67:0/2428598072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 0x5623335f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb706800640 1 -- 172.21.15.67:0/2428598072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 msgr2=0x5623335f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb706800640 1 --2- 172.21.15.67:0/2428598072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 0x5623335f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb705e00640 1 --2- 172.21.15.67:0/2428598072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e7110 0x5623336000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb706800640 1 -- 172.21.15.67:0/2428598072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e7110 msgr2=0x5623336000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb706800640 1 --2- 172.21.15.67:0/2428598072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e7110 0x5623336000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb706800640 1 -- 172.21.15.67:0/2428598072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623335e4110 con 0x562333601090 2024-05-30T16:55:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb706800640 1 --2- 172.21.15.67:0/2428598072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562333601090 0x5623335f62b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb700001eb0 tx=0x7fb70003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af11345de8c7fdf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb6f2000640 1 -- 172.21.15.67:0/2428598072 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700043030 con 0x562333601090 2024-05-30T16:55:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb6f2000640 1 -- 172.21.15.67:0/2428598072 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb700032b50 con 0x562333601090 2024-05-30T16:55:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.409+0000 7fb6f2000640 1 -- 172.21.15.67:0/2428598072 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700037040 con 0x562333601090 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.410+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2428598072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562333601090 msgr2=0x5623335f62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.410+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2428598072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562333601090 0x5623335f62b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb700001eb0 tx=0x7fb70003c000 comp rx=0 tx=0).stop 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.410+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2428598072 shutdown_connections 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.410+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2428598072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562333601090 0x5623335f62b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.410+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2428598072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335ffcc0 0x5623335f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.410+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2428598072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623335e7110 0x5623336000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.410+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2428598072 >> 172.21.15.67:0/2428598072 conn(0x5623335e1d30 msgr2=0x5623335fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.410+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2428598072 shutdown_connections 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.410+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2428598072 wait complete. 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.411+0000 7fb70fd8e740 1 -- start start 2024-05-30T16:55:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.411+0000 7fb70fd8e740 1 Processor -- start 2024-05-30T16:55:03.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335e7110 0x5623335b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x562333682a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb70fd8e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 0x5623335e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5623335e4990 con 0x5623335ffcc0 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5623335fb9e0 con 0x562333601090 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb70fd8e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5623335e4f80 con 0x5623335e7110 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb70fd8e740 10 client.0 _ll_register_callbacks cb 0x7fb7000096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb705400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x562333682a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb705400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x562333682a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48228/0 (socket says 172.21.15.67:48228) 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.412+0000 7fb705400640 1 -- 172.21.15.67:0/2929463971 learned_addr learned my addr 172.21.15.67:0/2929463971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb705e00640 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335e7110 0x5623335b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb705400640 1 -- 172.21.15.67:0/2929463971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335e7110 msgr2=0x5623335b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb705400640 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335e7110 0x5623335b67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb706800640 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 0x5623335e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb705400640 1 -- 172.21.15.67:0/2929463971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 msgr2=0x5623335e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb705400640 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 0x5623335e0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb705400640 1 -- 172.21.15.67:0/2929463971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623336207f0 con 0x5623335ffcc0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb705400640 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x562333682a70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4061e50 tx=0x7fb6f4059ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d945e373119b0799 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb6e7400640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb6e7400640 1 -- 172.21.15.67:0/2929463971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4088060 con 0x5623335ffcc0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb6e7400640 1 -- 172.21.15.67:0/2929463971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fb6f4010da0 con 0x5623335ffcc0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.413+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2929463971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb700032b50 con 0x5623335ffcc0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb6e7400640 1 -- 172.21.15.67:0/2929463971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb6f401e040 con 0x5623335ffcc0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb70fd8e740 10 client.6687 fetch_fsmap learned FSMap version 45 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2929463971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x562333683070 con 0x5623335ffcc0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb6e7400640 1 -- 172.21.15.67:0/2929463971 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb700032b50 con 0x5623335ffcc0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb6e7400640 1 -- 172.21.15.67:0/2929463971 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7fb6f4071030 con 0x5623335ffcc0 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb70fd8e740 10 client.6687 fetch_fsmap finished waiting for FSMap version 45 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb70fd8e740 10 client.6687 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb70fd8e740 20 client.6687 populate_metadata read hostname 'smithi067' 2024-05-30T16:55:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] conn(0x5623335e11b0 0x562333629f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb70fd8e740 4 client.6687 mds_command: new command op to 6558 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-30T16:55:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2929463971 --> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] -- command(tid 0: {"prefix": "status"}) -- 0x5623335263a0 con 0x5623335e11b0 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.414+0000 7fb705e00640 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] conn(0x5623335e11b0 0x562333629f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb705e00640 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] conn(0x5623335e11b0 0x562333629f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc7488d99bb9eb84 server_cookie=c58efde2c743a61b in_seq=0 out_seq=0 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb6e7400640 10 client.6687 ms_handle_connect on v2:172.21.15.67:6836/2706164583 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb6e7400640 1 -- 172.21.15.67:0/2929463971 <== mds.0 v2:172.21.15.67:6836/2706164583 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5623335263a0 con 0x5623335e11b0 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb6e7400640 10 client.6687 handle_command_reply: tid=0 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb70fd8e740 1 client.6687 shutdown 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2929463971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 msgr2=0x562333682a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x562333682a70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4061e50 tx=0x7fb6f4059ea0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2929463971 shutdown_connections 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.67:6836/2706164583,v1:172.21.15.67:6837/2706164583] conn(0x5623335e11b0 0x562333629f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562333601090 0x5623335e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5623335ffcc0 0x562333682a70 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb70fd8e740 1 --2- 172.21.15.67:0/2929463971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623335e7110 0x5623335b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb6e7400640 0 client.6687 ms_handle_reset on v2:172.21.15.67:6836/2706164583 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.415+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2929463971 >> 172.21.15.67:0/2929463971 conn(0x5623335e1d30 msgr2=0x5623335f5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2929463971 shutdown_connections 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb70fd8e740 1 -- 172.21.15.67:0/2929463971 wait complete. 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb70fd8e740 20 client.6687 trim_cache size 0 max 16384 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6558, 2024-05-30T16:55:03.419 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-05-30T16:55:03.420 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-05-30T16:55:03.420 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-05-30T16:55:03.420 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 6.7790314589999996, 2024-05-30T16:55:03.420 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 45, 2024-05-30T16:55:03.420 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 74, 2024-05-30T16:55:03.420 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 74, 2024-05-30T16:55:03.420 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 11.218326119 2024-05-30T16:55:03.420 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 -- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb6e806d610 msgr2=0x7fb6e806fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb6e806d610 0x7fb6e806fad0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8007e90 tx=0x7fb6f80073a0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 -- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb708073310 msgr2=0x7fb7080ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb708073310 0x7fb7080ce800 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b4a0 tx=0x7fb6f400b960 comp rx=0 tx=0).stop 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 -- 172.21.15.67:0/1051620663 shutdown_connections 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb6e806d610 0x7fb6e806fad0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb70815b3a0 0x7fb70809d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb708073310 0x7fb7080ce800 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 --2- 172.21.15.67:0/1051620663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb70806dc20 0x7fb70815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.416+0000 7fb707200640 1 -- 172.21.15.67:0/1051620663 >> 172.21.15.67:0/1051620663 conn(0x7fb7080b1fc0 msgr2=0x7fb708153940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.417+0000 7fb707200640 1 -- 172.21.15.67:0/1051620663 shutdown_connections 2024-05-30T16:55:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.417+0000 7fb707200640 1 -- 172.21.15.67:0/1051620663 wait complete. 2024-05-30T16:55:03.435 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T16:55:03.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.645+0000 7f65a4e00640 1 Processor -- start 2024-05-30T16:55:03.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.645+0000 7f65a4e00640 1 -- start start 2024-05-30T16:55:03.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.646+0000 7f65a4e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f65a0073310 0x7f65a00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.646+0000 7f65a4e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a00afd60 con 0x7f65a006dc20 2024-05-30T16:55:03.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.647+0000 7f65a4e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a00af7a0 con 0x7f65a015b3a0 2024-05-30T16:55:03.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.647+0000 7f65a4e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f65a006f980 con 0x7f65a0073310 2024-05-30T16:55:03.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.647+0000 7f659ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f65a0073310 0x7f65a00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.647+0000 7f659ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f65a0073310 0x7f65a00b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48244/0 (socket says 172.21.15.67:48244) 2024-05-30T16:55:03.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.647+0000 7f659ea00640 1 -- 172.21.15.67:0/3637330171 learned_addr learned my addr 172.21.15.67:0/3637330171 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:03.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.647+0000 7f659ea00640 1 -- 172.21.15.67:0/3637330171 >> v1:172.21.15.169:6789/0 conn(0x7f65a015b3a0 legacy=0x7f65a0072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:03.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.647+0000 7f659ea00640 1 -- 172.21.15.67:0/3637330171 >> v1:172.21.15.67:6789/0 conn(0x7f65a006dc20 legacy=0x7f65a00ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.647+0000 7f659ea00640 1 -- 172.21.15.67:0/3637330171 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a00afa80 con 0x7f65a0073310 2024-05-30T16:55:03.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.648+0000 7f659ea00640 1 --2- 172.21.15.67:0/3637330171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 0x7f65a00b32c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f659000bb10 tx=0x7f659002c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ae6b29394579a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.649+0000 7f659e000640 1 -- 172.21.15.67:0/3637330171 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590036020 con 0x7f65a0073310 2024-05-30T16:55:03.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.649+0000 7f659e000640 1 -- 172.21.15.67:0/3637330171 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6590032820 con 0x7f65a0073310 2024-05-30T16:55:03.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.649+0000 7f659e000640 1 -- 172.21.15.67:0/3637330171 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590032bf0 con 0x7f65a0073310 2024-05-30T16:55:03.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.649+0000 7f65a4e00640 1 -- 172.21.15.67:0/3637330171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 msgr2=0x7f65a00b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.649+0000 7f65a4e00640 1 --2- 172.21.15.67:0/3637330171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 0x7f65a00b32c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f659000bb10 tx=0x7f659002c960 comp rx=0 tx=0).stop 2024-05-30T16:55:03.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.649+0000 7f65a4e00640 1 -- 172.21.15.67:0/3637330171 shutdown_connections 2024-05-30T16:55:03.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.649+0000 7f65a4e00640 1 --2- 172.21.15.67:0/3637330171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 0x7f65a00b32c0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.649+0000 7f65a4e00640 1 -- 172.21.15.67:0/3637330171 >> 172.21.15.67:0/3637330171 conn(0x7f65a00b1fc0 msgr2=0x7f65a00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:03.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.649+0000 7f65a4e00640 1 -- 172.21.15.67:0/3637330171 shutdown_connections 2024-05-30T16:55:03.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.650+0000 7f65a4e00640 1 -- 172.21.15.67:0/3637330171 wait complete. 2024-05-30T16:55:03.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.651+0000 7f65a4e00640 1 Processor -- start 2024-05-30T16:55:03.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.651+0000 7f65a4e00640 1 -- start start 2024-05-30T16:55:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.652+0000 7f65a4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a006dc20 0x7f65a00cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.652+0000 7f65a4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 0x7f65a0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.652+0000 7f65a4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65a015b3a0 0x7f65a0154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.652+0000 7f65a4e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65a015f9a0 con 0x7f65a0073310 2024-05-30T16:55:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.652+0000 7f65a4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f65a00ae0c0 con 0x7f65a006dc20 2024-05-30T16:55:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.652+0000 7f65a4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f65a015fb20 con 0x7f65a015b3a0 2024-05-30T16:55:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 0x7f65a0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 0x7f65a0154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48250/0 (socket says 172.21.15.67:48250) 2024-05-30T16:55:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659ea00640 1 -- 172.21.15.67:0/3647469092 learned_addr learned my addr 172.21.15.67:0/3647469092 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:03.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659ea00640 1 -- 172.21.15.67:0/3647469092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65a015b3a0 msgr2=0x7f65a0154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659ea00640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65a015b3a0 0x7f65a0154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659ea00640 1 -- 172.21.15.67:0/3647469092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a006dc20 msgr2=0x7f65a00cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659f400640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a006dc20 0x7f65a00cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659ea00640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a006dc20 0x7f65a00cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659ea00640 1 -- 172.21.15.67:0/3647469092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a00cacd0 con 0x7f65a0073310 2024-05-30T16:55:03.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659f400640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a006dc20 0x7f65a00cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:03.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.653+0000 7f659ea00640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 0x7f65a0154ab0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f659000b2c0 tx=0x7f65900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.654+0000 7f6597400640 1 -- 172.21.15.67:0/3647469092 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590036040 con 0x7f65a0073310 2024-05-30T16:55:03.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.654+0000 7f6597400640 1 -- 172.21.15.67:0/3647469092 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6590031c30 con 0x7f65a0073310 2024-05-30T16:55:03.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.654+0000 7f6597400640 1 -- 172.21.15.67:0/3647469092 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659003b420 con 0x7f65a0073310 2024-05-30T16:55:03.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.654+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a00cafb0 con 0x7f65a0073310 2024-05-30T16:55:03.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.654+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a00cb570 con 0x7f65a0073310 2024-05-30T16:55:03.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.655+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a00689e0 con 0x7f65a0073310 2024-05-30T16:55:03.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.656+0000 7f6597400640 1 -- 172.21.15.67:0/3647469092 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f6590043020 con 0x7f65a0073310 2024-05-30T16:55:03.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.657+0000 7f6597400640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f657c06d5e0 0x7f657c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:03.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.657+0000 7f6597400640 1 -- 172.21.15.67:0/3647469092 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f65900ab880 con 0x7f65a0073310 2024-05-30T16:55:03.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.657+0000 7f659f400640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f657c06d5e0 0x7f657c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:03.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.658+0000 7f659f400640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f657c06d5e0 0x7f657c06faa0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f658800a830 tx=0x7f6588008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:03.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.662+0000 7f6597400640 1 -- 172.21.15.67:0/3647469092 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f659002f070 con 0x7f65a0073310 2024-05-30T16:55:03.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.910+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f65a00729c0 con 0x7f65a0073310 2024-05-30T16:55:03.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.911+0000 7f6597400640 1 -- 172.21.15.67:0/3647469092 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7f659007b450 con 0x7f65a0073310 2024-05-30T16:55:03.912 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T16:55:03.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f657c06d5e0 msgr2=0x7f657c06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f657c06d5e0 0x7f657c06faa0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f658800a830 tx=0x7f6588008040 comp rx=0 tx=0).stop 2024-05-30T16:55:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 msgr2=0x7f65a0154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 0x7f65a0154ab0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f659000b2c0 tx=0x7f65900046c0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 shutdown_connections 2024-05-30T16:55:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f657c06d5e0 0x7f657c06faa0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65a015b3a0 0x7f65a0154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a0073310 0x7f65a0154ab0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 --2- 172.21.15.67:0/3647469092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65a006dc20 0x7f65a00cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 >> 172.21.15.67:0/3647469092 conn(0x7f65a00b1fc0 msgr2=0x7f65a00b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 shutdown_connections 2024-05-30T16:55:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:03.914+0000 7f65a4e00640 1 -- 172.21.15.67:0/3647469092 wait complete. 2024-05-30T16:55:03.934 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T16:55:03.934 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T16:55:03.934 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T16:55:03.934 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-30T16:55:03.934 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T16:55:03.934 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T16:55:03.934 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T16:55:03.934 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T16:55:03.934 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:55:03.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:03.935 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:55:03.988 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41868sec preferred_lft 41868sec 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::74f5:cfff:fec0:693d/64 scope link 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:55:03.989 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:55:03.990 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::a8fe:aeff:fef7:f6a0/64 scope link 2024-05-30T16:55:03.990 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:55:03.990 INFO:teuthology.orchestra.run.smithi067.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:55:03.990 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:55:03.990 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::185e:a4ff:fec6:7d8c/64 scope link 2024-05-30T16:55:03.990 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:55:03.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:03.991 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T16:55:04.073 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T16:55:04.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:04.078 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T16:55:04.131 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T16:55:04.131 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T16:55:04.131 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T16:55:04.131 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T16:55:04.131 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T16:55:04.132 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:55:04.132 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:55:04.132 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T16:55:04.132 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T16:55:04.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T16:55:04.133 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:55:04.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:55:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:55:04.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:04.350 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T16:55:04.397 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T16:55:04.397 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-30T16:55:04.397 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:55:04.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:04.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-05-30T16:55:04.486 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-30T16:55:04.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:55:04.487 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-30T16:55:04.545 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:55:04.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:55:04.545 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:55:04.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:55:04.692 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:55:04.748 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:55:04.749 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:55:04.750 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:55:04.751 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:55:04.751 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:55:04.751 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:55:04.751 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:55:04.751 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:55:04.751 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:55:04.751 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:55:04.751 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:55:04.751 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:55:04.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:04.752 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:55:04.802 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T16:55:04.802 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-30T16:55:04.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:55:04.876 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:55:04.917 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:55:04.917 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:55:04.917 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:55:04.918 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:55:04.919 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:55:04.920 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:55:04.920 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:55:04.920 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:55:04.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:04.921 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:55:04.975 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-05-30T16:55:04.975+0000 7fd89aad2480 -1 init, newargv = 0x555f23681a70 newargc=15 2024-05-30T16:55:04.975 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[60682]: starting ceph client 2024-05-30T16:55:04.983 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[60682]: starting fuse 2024-05-30T16:55:05.004 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T16:55:05.004 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T16:55:05.005 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-05-30T16:55:05.006 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T16:55:05.202 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.60682.asok 2024-05-30T16:55:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:55:05.220 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60682.asok 2024-05-30T16:55:05.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:05.220 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60682.asok status 2024-05-30T16:55:05.454 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "60682", 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6702, 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T16:55:05.455 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6702 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 387659309 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 387659309 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:55:05.456 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.6702 192.168.0.1:0/387659309", 2024-05-30T16:55:05.457 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/387659309", 2024-05-30T16:55:05.457 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T16:55:05.457 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 45, 2024-05-30T16:55:05.457 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 74, 2024-05-30T16:55:05.457 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T16:55:05.457 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T16:55:05.457 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T16:55:05.457 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:55:05.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:05.470 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T16:55:05.471 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:55:05.524 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T16:55:05.524 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-30T16:55:05.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:05.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:55:05.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-30T16:55:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.837+0000 7f76dbe00640 1 Processor -- start 2024-05-30T16:55:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.838+0000 7f76dbe00640 1 -- start start 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.839+0000 7f76dbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76dc071f70 0x7f76dc072350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.839+0000 7f76dbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f76dc15cb50 0x7f76dc15cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.839+0000 7f76dbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc141b80 0x7f76dc141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.839+0000 7f76dbe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f76dc15d8c0 con 0x7f76dc071f70 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.839+0000 7f76dbe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f76dc13f820 con 0x7f76dc15cb50 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.839+0000 7f76dbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76dc072890 con 0x7f76dc141b80 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76daa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76dc071f70 0x7f76dc072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76daa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76dc071f70 0x7f76dc072350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48254/0 (socket says 172.21.15.67:48254) 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76daa00640 1 -- 172.21.15.67:0/1202652700 learned_addr learned my addr 172.21.15.67:0/1202652700 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76d3e00640 1 --2- 172.21.15.67:0/1202652700 >> v2:172.21.15.169:3300/0 conn(0x7f76dc15cb50 0x7f76dc15cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76db400640 1 --2- 172.21.15.67:0/1202652700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc141b80 0x7f76dc141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:05.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76daa00640 1 -- 172.21.15.67:0/1202652700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc141b80 msgr2=0x7f76dc141ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:05.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76daa00640 1 --2- 172.21.15.67:0/1202652700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc141b80 0x7f76dc141ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:05.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76daa00640 1 -- 172.21.15.67:0/1202652700 >> v2:172.21.15.169:3300/0 conn(0x7f76dc15cb50 msgr2=0x7f76dc15cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:05.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76daa00640 1 --2- 172.21.15.67:0/1202652700 >> v2:172.21.15.169:3300/0 conn(0x7f76dc15cb50 0x7f76dc15cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:05.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76daa00640 1 -- 172.21.15.67:0/1202652700 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc0b01a0 con 0x7f76dc071f70 2024-05-30T16:55:05.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.840+0000 7f76db400640 1 --2- 172.21.15.67:0/1202652700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc141b80 0x7f76dc141ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:05.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.841+0000 7f76daa00640 1 --2- 172.21.15.67:0/1202652700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc071f70 0x7f76dc072350 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0b23e0 tx=0x7f76cc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1212ea10673e6e76 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:05.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.842+0000 7f76da000640 1 -- 172.21.15.67:0/1202652700 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc037070 con 0x7f76dc071f70 2024-05-30T16:55:05.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.842+0000 7f76da000640 1 -- 172.21.15.67:0/1202652700 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc02ae00 con 0x7f76dc071f70 2024-05-30T16:55:05.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.842+0000 7f76da000640 1 -- 172.21.15.67:0/1202652700 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc032660 con 0x7f76dc071f70 2024-05-30T16:55:05.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.842+0000 7f76dbe00640 1 -- 172.21.15.67:0/1202652700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc071f70 msgr2=0x7f76dc072350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:05.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.842+0000 7f76dbe00640 1 --2- 172.21.15.67:0/1202652700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc071f70 0x7f76dc072350 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0b23e0 tx=0x7f76cc02a0b0 comp rx=0 tx=0).stop 2024-05-30T16:55:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.843+0000 7f76dbe00640 1 -- 172.21.15.67:0/1202652700 shutdown_connections 2024-05-30T16:55:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.843+0000 7f76dbe00640 1 --2- 172.21.15.67:0/1202652700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc141b80 0x7f76dc141ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.843+0000 7f76dbe00640 1 --2- 172.21.15.67:0/1202652700 >> v2:172.21.15.169:3300/0 conn(0x7f76dc15cb50 0x7f76dc15cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.843+0000 7f76dbe00640 1 --2- 172.21.15.67:0/1202652700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc071f70 0x7f76dc072350 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.843+0000 7f76dbe00640 1 -- 172.21.15.67:0/1202652700 >> 172.21.15.67:0/1202652700 conn(0x7f76dc067ce0 msgr2=0x7f76dc06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.843+0000 7f76dbe00640 1 -- 172.21.15.67:0/1202652700 shutdown_connections 2024-05-30T16:55:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.843+0000 7f76dbe00640 1 -- 172.21.15.67:0/1202652700 wait complete. 2024-05-30T16:55:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.844+0000 7f76dbe00640 1 Processor -- start 2024-05-30T16:55:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.845+0000 7f76dbe00640 1 -- start start 2024-05-30T16:55:05.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76dbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76dc071f70 0x7f76dc1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:05.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76dbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc141b80 0x7f76dc135d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:05.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76dbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc15cb50 0x7f76dc137240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:05.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76dbe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc06e3e0 con 0x7f76dc141b80 2024-05-30T16:55:05.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76dbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc06e100 con 0x7f76dc071f70 2024-05-30T16:55:05.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76dbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76dc15d520 con 0x7f76dc15cb50 2024-05-30T16:55:05.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc141b80 0x7f76dc135d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc141b80 0x7f76dc135d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48270/0 (socket says 172.21.15.67:48270) 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76d3e00640 1 -- 172.21.15.67:0/2397926021 learned_addr learned my addr 172.21.15.67:0/2397926021 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76daa00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76dc071f70 0x7f76dc1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76db400640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc15cb50 0x7f76dc137240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76d3e00640 1 -- 172.21.15.67:0/2397926021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc15cb50 msgr2=0x7f76dc137240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76d3e00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc15cb50 0x7f76dc137240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76d3e00640 1 -- 172.21.15.67:0/2397926021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76dc071f70 msgr2=0x7f76dc1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76d3e00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76dc071f70 0x7f76dc1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.846+0000 7f76d3e00640 1 -- 172.21.15.67:0/2397926021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc0cfc60 con 0x7f76dc141b80 2024-05-30T16:55:05.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.847+0000 7f76d3e00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc141b80 0x7f76dc135d60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f76c400d5f0 tx=0x7f76c400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:05.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.847+0000 7f76d2a00640 1 -- 172.21.15.67:0/2397926021 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40125f0 con 0x7f76dc141b80 2024-05-30T16:55:05.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.847+0000 7f76d2a00640 1 -- 172.21.15.67:0/2397926021 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c4019070 con 0x7f76dc141b80 2024-05-30T16:55:05.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.847+0000 7f76d2a00640 1 -- 172.21.15.67:0/2397926021 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40129d0 con 0x7f76dc141b80 2024-05-30T16:55:05.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.847+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76dc136460 con 0x7f76dc141b80 2024-05-30T16:55:05.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.847+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76dc136a20 con 0x7f76dc141b80 2024-05-30T16:55:05.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.848+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76dc0690f0 con 0x7f76dc141b80 2024-05-30T16:55:05.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.850+0000 7f76d2a00640 1 -- 172.21.15.67:0/2397926021 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f76c4012b70 con 0x7f76dc141b80 2024-05-30T16:55:05.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.850+0000 7f76d2a00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76b806d610 0x7f76b806fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:05.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.851+0000 7f76d2a00640 1 -- 172.21.15.67:0/2397926021 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f76c408ac30 con 0x7f76dc141b80 2024-05-30T16:55:05.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.851+0000 7f76daa00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76b806d610 0x7f76b806fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:05.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.851+0000 7f76daa00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76b806d610 0x7f76b806fad0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f76cc034000 tx=0x7f76cc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:05.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:05.855+0000 7f76d2a00640 1 -- 172.21.15.67:0/2397926021 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76c405a820 con 0x7f76dc141b80 2024-05-30T16:55:06.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.103+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f76dc137020 con 0x7f76dc141b80 2024-05-30T16:55:06.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.707+0000 7f76d2a00640 1 -- 172.21.15.67:0/2397926021 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v46) ==== 136+0+0 (secure 0 0 0) 0x7f76c405a120 con 0x7f76dc141b80 2024-05-30T16:55:06.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76b806d610 msgr2=0x7f76b806fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:06.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76b806d610 0x7f76b806fad0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f76cc034000 tx=0x7f76cc02e000 comp rx=0 tx=0).stop 2024-05-30T16:55:06.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc141b80 msgr2=0x7f76dc135d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:06.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc141b80 0x7f76dc135d60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f76c400d5f0 tx=0x7f76c400dab0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 shutdown_connections 2024-05-30T16:55:06.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f76b806d610 0x7f76b806fad0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76dc15cb50 0x7f76dc137240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76dc141b80 0x7f76dc135d60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 --2- 172.21.15.67:0/2397926021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76dc071f70 0x7f76dc1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.710+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 >> 172.21.15.67:0/2397926021 conn(0x7f76dc067ce0 msgr2=0x7f76dc15b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.711+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 shutdown_connections 2024-05-30T16:55:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.711+0000 7f76dbe00640 1 -- 172.21.15.67:0/2397926021 wait complete. 2024-05-30T16:55:06.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:55:06.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.953+0000 7f9a43200640 1 Processor -- start 2024-05-30T16:55:06.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.953+0000 7f9a43200640 1 -- start start 2024-05-30T16:55:06.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.954+0000 7f9a43200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a440ce3a0 0x7f9a4406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:06.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.954+0000 7f9a43200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a4406d960 0x7f9a4409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:06.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.954+0000 7f9a43200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a440d04c0 con 0x7f9a4409af70 2024-05-30T16:55:06.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.954+0000 7f9a43200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9a440d1350 con 0x7f9a4406d960 2024-05-30T16:55:06.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.954+0000 7f9a43200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9a4409b3a0 con 0x7f9a440ce3a0 2024-05-30T16:55:06.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.955+0000 7f9a41400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a4406d960 0x7f9a4409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:06.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.955+0000 7f9a41400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a4406d960 0x7f9a4409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48286/0 (socket says 172.21.15.67:48286) 2024-05-30T16:55:06.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.955+0000 7f9a41400640 1 -- 172.21.15.67:0/3172287907 learned_addr learned my addr 172.21.15.67:0/3172287907 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:06.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.955+0000 7f9a42800640 1 --1- 172.21.15.67:0/3172287907 >> v1:172.21.15.67:6789/0 conn(0x7f9a4409af70 0x7f9a441624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34736/0 (socket says 172.21.15.67:34736) 2024-05-30T16:55:06.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.955+0000 7f9a41e00640 1 --2- 172.21.15.67:0/3172287907 >> v2:172.21.15.169:3300/0 conn(0x7f9a440ce3a0 0x7f9a4406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:06.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.955+0000 7f9a41400640 1 -- 172.21.15.67:0/3172287907 >> v2:172.21.15.169:3300/0 conn(0x7f9a440ce3a0 msgr2=0x7f9a4406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:06.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.955+0000 7f9a41400640 1 --2- 172.21.15.67:0/3172287907 >> v2:172.21.15.169:3300/0 conn(0x7f9a440ce3a0 0x7f9a4406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.955+0000 7f9a41400640 1 -- 172.21.15.67:0/3172287907 >> v1:172.21.15.67:6789/0 conn(0x7f9a4409af70 legacy=0x7f9a441624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:06.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.955+0000 7f9a41400640 1 -- 172.21.15.67:0/3172287907 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a440ce780 con 0x7f9a4406d960 2024-05-30T16:55:06.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.956+0000 7f9a41400640 1 --2- 172.21.15.67:0/3172287907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 0x7f9a4409aa30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9a44074150 tx=0x7f9a3402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6973551cffe625cb server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:06.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.957+0000 7f9a40a00640 1 -- 172.21.15.67:0/3172287907 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34036070 con 0x7f9a4406d960 2024-05-30T16:55:06.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.957+0000 7f9a40a00640 1 -- 172.21.15.67:0/3172287907 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a34031600 con 0x7f9a4406d960 2024-05-30T16:55:06.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.957+0000 7f9a40a00640 1 -- 172.21.15.67:0/3172287907 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a340319f0 con 0x7f9a4406d960 2024-05-30T16:55:06.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.957+0000 7f9a43200640 1 -- 172.21.15.67:0/3172287907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 msgr2=0x7f9a4409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:06.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.957+0000 7f9a43200640 1 --2- 172.21.15.67:0/3172287907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 0x7f9a4409aa30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9a44074150 tx=0x7f9a3402a7c0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.958+0000 7f9a43200640 1 -- 172.21.15.67:0/3172287907 shutdown_connections 2024-05-30T16:55:06.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.958+0000 7f9a43200640 1 --2- 172.21.15.67:0/3172287907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 0x7f9a4409aa30 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.958+0000 7f9a43200640 1 --2- 172.21.15.67:0/3172287907 >> v2:172.21.15.169:3300/0 conn(0x7f9a440ce3a0 0x7f9a4406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.958+0000 7f9a43200640 1 -- 172.21.15.67:0/3172287907 >> 172.21.15.67:0/3172287907 conn(0x7f9a440d2330 msgr2=0x7f9a440d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:06.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.958+0000 7f9a43200640 1 -- 172.21.15.67:0/3172287907 shutdown_connections 2024-05-30T16:55:06.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.958+0000 7f9a43200640 1 -- 172.21.15.67:0/3172287907 wait complete. 2024-05-30T16:55:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.960+0000 7f9a43200640 1 Processor -- start 2024-05-30T16:55:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.960+0000 7f9a43200640 1 -- start start 2024-05-30T16:55:06.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a43200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 0x7f9a440c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:06.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a43200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a4409af70 0x7f9a440b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:06.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a43200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a440ce3a0 0x7f9a4415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a43200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a440d0a20 con 0x7f9a4406d960 2024-05-30T16:55:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a43200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a440d0740 con 0x7f9a4409af70 2024-05-30T16:55:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a43200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a440d0d00 con 0x7f9a440ce3a0 2024-05-30T16:55:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a41e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 0x7f9a440c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a41e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 0x7f9a440c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48298/0 (socket says 172.21.15.67:48298) 2024-05-30T16:55:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a41e00640 1 -- 172.21.15.67:0/1615947219 learned_addr learned my addr 172.21.15.67:0/1615947219 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a42800640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a440ce3a0 0x7f9a4415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.961+0000 7f9a41400640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a4409af70 0x7f9a440b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a41e00640 1 -- 172.21.15.67:0/1615947219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a440ce3a0 msgr2=0x7f9a4415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a41e00640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a440ce3a0 0x7f9a4415b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a41e00640 1 -- 172.21.15.67:0/1615947219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a4409af70 msgr2=0x7f9a440b6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a41e00640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a4409af70 0x7f9a440b6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a41e00640 1 -- 172.21.15.67:0/1615947219 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a440cae00 con 0x7f9a4406d960 2024-05-30T16:55:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a41400640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a4409af70 0x7f9a440b6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a41e00640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 0x7f9a440c9e10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d5f0 tx=0x7f9a2c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a3a000640 1 -- 172.21.15.67:0/1615947219 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0125f0 con 0x7f9a4406d960 2024-05-30T16:55:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a3a000640 1 -- 172.21.15.67:0/1615947219 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2c019070 con 0x7f9a4406d960 2024-05-30T16:55:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a3a000640 1 -- 172.21.15.67:0/1615947219 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0129d0 con 0x7f9a4406d960 2024-05-30T16:55:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a441f8570 con 0x7f9a4406d960 2024-05-30T16:55:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.962+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441f8b30 con 0x7f9a4406d960 2024-05-30T16:55:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.963+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a440690f0 con 0x7f9a4406d960 2024-05-30T16:55:06.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.965+0000 7f9a3a000640 1 -- 172.21.15.67:0/1615947219 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f9a2c012b70 con 0x7f9a4406d960 2024-05-30T16:55:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.965+0000 7f9a3a000640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a2006d5e0 0x7f9a2006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.966+0000 7f9a3a000640 1 -- 172.21.15.67:0/1615947219 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f9a2c08ac80 con 0x7f9a4406d960 2024-05-30T16:55:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.966+0000 7f9a41400640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a2006d5e0 0x7f9a2006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.967+0000 7f9a41400640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a2006d5e0 0x7f9a2006faa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9a34033000 tx=0x7f9a34009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:06.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:06.970+0000 7f9a3a000640 1 -- 172.21.15.67:0/1615947219 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a2c05a870 con 0x7f9a4406d960 2024-05-30T16:55:07.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.204+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9a440d1dc0 con 0x7f9a2006d5e0 2024-05-30T16:55:07.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.207+0000 7f9a3a000640 1 -- 172.21.15.67:0/1615947219 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9a440d1dc0 con 0x7f9a2006d5e0 2024-05-30T16:55:07.207 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:55:07.208 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:55:07.208 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:55:07.208 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:55:07.208 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:55:07.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a2006d5e0 msgr2=0x7f9a2006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a2006d5e0 0x7f9a2006faa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9a34033000 tx=0x7f9a34009000 comp rx=0 tx=0).stop 2024-05-30T16:55:07.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 msgr2=0x7f9a440c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 0x7f9a440c9e10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d5f0 tx=0x7f9a2c00dab0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 shutdown_connections 2024-05-30T16:55:07.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a2006d5e0 0x7f9a2006faa0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a440ce3a0 0x7f9a4415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a4409af70 0x7f9a440b6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 --2- 172.21.15.67:0/1615947219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a4406d960 0x7f9a440c9e10 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 >> 172.21.15.67:0/1615947219 conn(0x7f9a440d2330 msgr2=0x7f9a441615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.209+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 shutdown_connections 2024-05-30T16:55:07.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.210+0000 7f9a43200640 1 -- 172.21.15.67:0/1615947219 wait complete. 2024-05-30T16:55:07.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:55:07.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.441+0000 7fcd3ea00640 1 Processor -- start 2024-05-30T16:55:07.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.442+0000 7fcd3ea00640 1 -- start start 2024-05-30T16:55:07.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.443+0000 7fcd3ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcd3809aca0 0x7fcd3809b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.443+0000 7fcd3ea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd380d11d0 con 0x7fcd380b13d0 2024-05-30T16:55:07.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.443+0000 7fcd3ea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd380cdb60 con 0x7fcd3809b5c0 2024-05-30T16:55:07.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.443+0000 7fcd3ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcd3809ba00 con 0x7fcd3809aca0 2024-05-30T16:55:07.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.443+0000 7fcd37400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcd380b13d0 0x7fcd380d03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34748/0 (socket says 172.21.15.67:34748) 2024-05-30T16:55:07.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.443+0000 7fcd37400640 1 -- 172.21.15.67:0/1356085626 learned_addr learned my addr 172.21.15.67:0/1356085626 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:55:07.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd37e00640 1 --2- 172.21.15.67:0/1356085626 >> v2:172.21.15.169:3300/0 conn(0x7fcd3809aca0 0x7fcd3809b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967913029 0 0) 0x7fcd380d11d0 con 0x7fcd380b13d0 2024-05-30T16:55:07.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd1c003520 con 0x7fcd380b13d0 2024-05-30T16:55:07.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 528920207 0 0) 0x7fcd1c003520 con 0x7fcd380b13d0 2024-05-30T16:55:07.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 >> v2:172.21.15.169:3300/0 conn(0x7fcd3809aca0 msgr2=0x7fcd3809b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:07.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 --2- 172.21.15.67:0/1356085626 >> v2:172.21.15.169:3300/0 conn(0x7fcd3809aca0 0x7fcd3809b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 >> v1:172.21.15.169:6789/0 conn(0x7fcd3809b5c0 legacy=0x7fcd380d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd380b1990 con 0x7fcd380b13d0 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fcd24001670 con 0x7fcd380b13d0 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 >> v1:172.21.15.67:6789/0 conn(0x7fcd380b13d0 legacy=0x7fcd380d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.444+0000 7fcd36a00640 1 --2- 172.21.15.67:0/1356085626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd36a00640 1 --2- 172.21.15.67:0/1356085626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd1c004800 0x7fcd1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd36a00640 1 --2- 172.21.15.67:0/1356085626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd1c0051c0 0x7fcd1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c00c070 con 0x7fcd1c003a00 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c00c220 con 0x7fcd1c004800 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c00c420 con 0x7fcd1c0051c0 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd37e00640 1 --2- 172.21.15.67:0/1356085626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd37e00640 1 -- 172.21.15.67:0/1356085626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd1c0051c0 msgr2=0x7fcd1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd37e00640 1 --2- 172.21.15.67:0/1356085626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd1c0051c0 0x7fcd1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd37e00640 1 -- 172.21.15.67:0/1356085626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd1c004800 msgr2=0x7fcd1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:07.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd37e00640 1 --2- 172.21.15.67:0/1356085626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd1c004800 0x7fcd1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd37e00640 1 -- 172.21.15.67:0/1356085626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c003520 con 0x7fcd1c003a00 2024-05-30T16:55:07.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.445+0000 7fcd3ca00640 1 -- 172.21.15.67:0/1356085626 reap_dead start 2024-05-30T16:55:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.446+0000 7fcd37e00640 1 --2- 172.21.15.67:0/1356085626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd1c003e20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009ac0 tx=0x7fcd1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7133adc42add4304 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28036070 con 0x7fcd1c003a00 2024-05-30T16:55:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd28031400 con 0x7fcd1c003a00 2024-05-30T16:55:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd36a00640 1 -- 172.21.15.67:0/1356085626 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28009690 con 0x7fcd1c003a00 2024-05-30T16:55:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd3ea00640 1 -- 172.21.15.67:0/1356085626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 msgr2=0x7fcd1c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/1356085626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd1c003e20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009ac0 tx=0x7fcd1c005830 comp rx=0 tx=0).stop 2024-05-30T16:55:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd3ea00640 1 -- 172.21.15.67:0/1356085626 shutdown_connections 2024-05-30T16:55:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/1356085626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd1c003e20 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd3ea00640 1 -- 172.21.15.67:0/1356085626 >> 172.21.15.67:0/1356085626 conn(0x7fcd380d24b0 msgr2=0x7fcd380d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:07.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd3ea00640 1 -- 172.21.15.67:0/1356085626 shutdown_connections 2024-05-30T16:55:07.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.447+0000 7fcd3ea00640 1 -- 172.21.15.67:0/1356085626 wait complete. 2024-05-30T16:55:07.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.449+0000 7fcd3ea00640 1 Processor -- start 2024-05-30T16:55:07.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.449+0000 7fcd3ea00640 1 -- start start 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.450+0000 7fcd3ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd380c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.450+0000 7fcd3ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd38155da0 0x7fcd380afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.450+0000 7fcd3ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd380b0000 0x7fcd381f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.450+0000 7fcd3ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd380ce3c0 con 0x7fcd1c003a00 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.450+0000 7fcd3ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd380ce0e0 con 0x7fcd38155da0 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.450+0000 7fcd3ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3809b080 con 0x7fcd380b0000 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd380c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd380c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48302/0 (socket says 172.21.15.67:48302) 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37e00640 1 -- 172.21.15.67:0/2342834583 learned_addr learned my addr 172.21.15.67:0/2342834583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37400640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd38155da0 0x7fcd380afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd3ca00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd380b0000 0x7fcd381f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37e00640 1 -- 172.21.15.67:0/2342834583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd380b0000 msgr2=0x7fcd381f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37e00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd380b0000 0x7fcd381f7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37e00640 1 -- 172.21.15.67:0/2342834583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd38155da0 msgr2=0x7fcd380afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37e00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd38155da0 0x7fcd380afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37e00640 1 -- 172.21.15.67:0/2342834583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd381f8560 con 0x7fcd1c003a00 2024-05-30T16:55:07.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.451+0000 7fcd37e00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd380c9580 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009be0 tx=0x7fcd28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:07.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.452+0000 7fcd2fe00640 1 -- 172.21.15.67:0/2342834583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2803f020 con 0x7fcd1c003a00 2024-05-30T16:55:07.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.452+0000 7fcd2fe00640 1 -- 172.21.15.67:0/2342834583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd28037040 con 0x7fcd1c003a00 2024-05-30T16:55:07.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.452+0000 7fcd2fe00640 1 -- 172.21.15.67:0/2342834583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28036070 con 0x7fcd1c003a00 2024-05-30T16:55:07.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.452+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd380c7680 con 0x7fcd1c003a00 2024-05-30T16:55:07.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.452+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd380c7c40 con 0x7fcd1c003a00 2024-05-30T16:55:07.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.453+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd380c7470 con 0x7fcd1c003a00 2024-05-30T16:55:07.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.454+0000 7fcd2fe00640 1 -- 172.21.15.67:0/2342834583 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fcd28042020 con 0x7fcd1c003a00 2024-05-30T16:55:07.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.455+0000 7fcd2fe00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1806d5e0 0x7fcd1806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.455+0000 7fcd2fe00640 1 -- 172.21.15.67:0/2342834583 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fcd280aa2b0 con 0x7fcd1c003a00 2024-05-30T16:55:07.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.455+0000 7fcd37400640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1806d5e0 0x7fcd1806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.456+0000 7fcd37400640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1806d5e0 0x7fcd1806faa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcd38156730 tx=0x7fcd24009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:07.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.460+0000 7fcd2fe00640 1 -- 172.21.15.67:0/2342834583 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd28049070 con 0x7fcd1c003a00 2024-05-30T16:55:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.657+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcd380690f0 con 0x7fcd1c003a00 2024-05-30T16:55:07.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.658+0000 7fcd2fe00640 1 -- 172.21.15.67:0/2342834583 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fcd2807a130 con 0x7fcd1c003a00 2024-05-30T16:55:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1806d5e0 msgr2=0x7fcd1806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1806d5e0 0x7fcd1806faa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcd38156730 tx=0x7fcd24009040 comp rx=0 tx=0).stop 2024-05-30T16:55:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 msgr2=0x7fcd380c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd380c9580 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009be0 tx=0x7fcd28004730 comp rx=0 tx=0).stop 2024-05-30T16:55:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 shutdown_connections 2024-05-30T16:55:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1806d5e0 0x7fcd1806faa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd380b0000 0x7fcd381f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd38155da0 0x7fcd380afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/2342834583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd1c003a00 0x7fcd380c9580 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 >> 172.21.15.67:0/2342834583 conn(0x7fcd380d24b0 msgr2=0x7fcd3806ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:07.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.661+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 shutdown_connections 2024-05-30T16:55:07.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.662+0000 7fcd3ea00640 1 -- 172.21.15.67:0/2342834583 wait complete. 2024-05-30T16:55:07.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-05-30T16:55:07.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.890+0000 7f069be00640 1 Processor -- start 2024-05-30T16:55:07.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.890+0000 7f069be00640 1 -- start start 2024-05-30T16:55:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.892+0000 7f069be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f069c073ba0 0x7f069c15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.892+0000 7f069be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069c15a2e0 con 0x7f069c0af330 2024-05-30T16:55:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.892+0000 7f069be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069c0ad230 con 0x7f069c159f00 2024-05-30T16:55:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.892+0000 7f069be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f069c073fd0 con 0x7f069c073ba0 2024-05-30T16:55:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.892+0000 7f069a000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f069c0af330 0x7f069c071f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34752/0 (socket says 172.21.15.67:34752) 2024-05-30T16:55:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.892+0000 7f069a000640 1 -- 172.21.15.67:0/2628590953 learned_addr learned my addr 172.21.15.67:0/2628590953 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:55:07.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.892+0000 7f069b400640 1 --2- 172.21.15.67:0/2628590953 >> v2:172.21.15.169:3300/0 conn(0x7f069c073ba0 0x7f069c15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.892+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1056455441 0 0) 0x7f0688001110 con 0x7f069c0af330 2024-05-30T16:55:07.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0680003520 con 0x7f069c0af330 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 102352668 0 0) 0x7f0680003520 con 0x7f069c0af330 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 >> v2:172.21.15.169:3300/0 conn(0x7f069c073ba0 msgr2=0x7f069c15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 --2- 172.21.15.67:0/2628590953 >> v2:172.21.15.169:3300/0 conn(0x7f069c073ba0 0x7f069c15d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 >> v1:172.21.15.169:6789/0 conn(0x7f069c159f00 legacy=0x7f069c0aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c0ad510 con 0x7f069c0af330 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f0688001840 con 0x7f069c0af330 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 >> v1:172.21.15.67:6789/0 conn(0x7f069c0af330 legacy=0x7f069c071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680003a00 0x7f0680003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0680004800 0x7f0680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0680005210 0x7f068000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f069c073fd0 con 0x7f0680004800 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0680010570 con 0x7f0680003a00 2024-05-30T16:55:07.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.893+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0680010770 con 0x7f0680005210 2024-05-30T16:55:07.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.894+0000 7f069a000640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0680004800 0x7f0680004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.894+0000 7f069b400640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0680005210 0x7f068000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.894+0000 7f069aa00640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680003a00 0x7f0680003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.894+0000 7f069a000640 1 -- 172.21.15.67:0/2628590953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0680005210 msgr2=0x7f068000fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:07.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.894+0000 7f069a000640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0680005210 0x7f068000fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.894+0000 7f069a000640 1 -- 172.21.15.67:0/2628590953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680003a00 msgr2=0x7f0680003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:07.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.894+0000 7f069a000640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680003a00 0x7f0680003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.894+0000 7f069a000640 1 -- 172.21.15.67:0/2628590953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0680003520 con 0x7f0680004800 2024-05-30T16:55:07.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.894+0000 7f069b400640 1 -- 172.21.15.67:0/2628590953 reap_dead start 2024-05-30T16:55:07.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.896+0000 7f069a000640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0680004800 0x7f0680004c80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0688001050 tx=0x7f0680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8dcbdb9a51f558 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:07.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.896+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688007e10 con 0x7f0680004800 2024-05-30T16:55:07.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.896+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0688005af0 con 0x7f0680004800 2024-05-30T16:55:07.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.896+0000 7f0699600640 1 -- 172.21.15.67:0/2628590953 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688007070 con 0x7f0680004800 2024-05-30T16:55:07.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.897+0000 7f069be00640 1 -- 172.21.15.67:0/2628590953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0680004800 msgr2=0x7f0680004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.897+0000 7f069be00640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0680004800 0x7f0680004c80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0688001050 tx=0x7f0680005830 comp rx=0 tx=0).stop 2024-05-30T16:55:07.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.897+0000 7f069be00640 1 -- 172.21.15.67:0/2628590953 shutdown_connections 2024-05-30T16:55:07.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.897+0000 7f069be00640 1 --2- 172.21.15.67:0/2628590953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0680004800 0x7f0680004c80 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.897+0000 7f069be00640 1 -- 172.21.15.67:0/2628590953 >> 172.21.15.67:0/2628590953 conn(0x7f069c0b1db0 msgr2=0x7f069c0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:07.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.897+0000 7f069be00640 1 -- 172.21.15.67:0/2628590953 shutdown_connections 2024-05-30T16:55:07.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.897+0000 7f069be00640 1 -- 172.21.15.67:0/2628590953 wait complete. 2024-05-30T16:55:07.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.899+0000 7f069be00640 1 Processor -- start 2024-05-30T16:55:07.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.899+0000 7f069be00640 1 -- start start 2024-05-30T16:55:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.900+0000 7f069be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680004800 0x7f069c157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.900+0000 7f069be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f069c15b120 0x7f069c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.900+0000 7f069be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f069c145a10 0x7f069c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.900+0000 7f069be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f069c15d3b0 con 0x7f069c15b120 2024-05-30T16:55:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.900+0000 7f069be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f069c0cfbc0 con 0x7f0680004800 2024-05-30T16:55:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.900+0000 7f069be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f069c15d530 con 0x7f069c145a10 2024-05-30T16:55:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.900+0000 7f069a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f069c15b120 0x7f069c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.900+0000 7f069a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f069c15b120 0x7f069c155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48318/0 (socket says 172.21.15.67:48318) 2024-05-30T16:55:07.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.900+0000 7f069a000640 1 -- 172.21.15.67:0/1345258083 learned_addr learned my addr 172.21.15.67:0/1345258083 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:07.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.901+0000 7f069b400640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f069c145a10 0x7f069c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.901+0000 7f069aa00640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680004800 0x7f069c157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.901+0000 7f069a000640 1 -- 172.21.15.67:0/1345258083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f069c145a10 msgr2=0x7f069c1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.901+0000 7f069a000640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f069c145a10 0x7f069c1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.901+0000 7f069a000640 1 -- 172.21.15.67:0/1345258083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680004800 msgr2=0x7f069c157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.901+0000 7f069a000640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680004800 0x7f069c157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.901+0000 7f069a000640 1 -- 172.21.15.67:0/1345258083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c158430 con 0x7f069c15b120 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.901+0000 7f069a000640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f069c15b120 0x7f069c155d70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0688034040 tx=0x7f069c06d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.902+0000 7f0693400640 1 -- 172.21.15.67:0/1345258083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688042020 con 0x7f069c15b120 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.902+0000 7f0693400640 1 -- 172.21.15.67:0/1345258083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0688004bf0 con 0x7f069c15b120 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.902+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069c158710 con 0x7f069c15b120 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.902+0000 7f0693400640 1 -- 172.21.15.67:0/1345258083 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688001110 con 0x7f069c15b120 2024-05-30T16:55:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.902+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069c158cd0 con 0x7f069c15b120 2024-05-30T16:55:07.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.903+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069c0689e0 con 0x7f069c15b120 2024-05-30T16:55:07.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.905+0000 7f0693400640 1 -- 172.21.15.67:0/1345258083 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f0688045020 con 0x7f069c15b120 2024-05-30T16:55:07.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.906+0000 7f0693400640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f067c06d5e0 0x7f067c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:07.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.906+0000 7f0693400640 1 -- 172.21.15.67:0/1345258083 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f06880aaa70 con 0x7f069c15b120 2024-05-30T16:55:07.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.906+0000 7f069aa00640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f067c06d5e0 0x7f067c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:07.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.907+0000 7f069aa00640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f067c06d5e0 0x7f067c06faa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f068c005410 tx=0x7f068c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:07.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:07.911+0000 7f0693400640 1 -- 172.21.15.67:0/1345258083 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f068804c070 con 0x7f069c15b120 2024-05-30T16:55:08.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.115+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f069c153450 con 0x7f069c15b120 2024-05-30T16:55:08.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.122+0000 7f0693400640 1 -- 172.21.15.67:0/1345258083 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v19)=0 v19) ==== 136+0+0 (secure 0 0 0) 0x7f069c15a2e0 con 0x7f069c15b120 2024-05-30T16:55:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f067c06d5e0 msgr2=0x7f067c06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f067c06d5e0 0x7f067c06faa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f068c005410 tx=0x7f068c009040 comp rx=0 tx=0).stop 2024-05-30T16:55:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f069c15b120 msgr2=0x7f069c155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f069c15b120 0x7f069c155d70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0688034040 tx=0x7f069c06d870 comp rx=0 tx=0).stop 2024-05-30T16:55:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 shutdown_connections 2024-05-30T16:55:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f067c06d5e0 0x7f067c06faa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f069c145a10 0x7f069c1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f069c15b120 0x7f069c155d70 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 --2- 172.21.15.67:0/1345258083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680004800 0x7f069c157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.125+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 >> 172.21.15.67:0/1345258083 conn(0x7f069c0b1db0 msgr2=0x7f069c15b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.126+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 shutdown_connections 2024-05-30T16:55:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.126+0000 7f069be00640 1 -- 172.21.15.67:0/1345258083 wait complete. 2024-05-30T16:55:08.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-30T16:55:08.155 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.150+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.155 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.150+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.155 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.150+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.155 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.150+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.155 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.150+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.335+0000 7fe5a1c00640 1 Processor -- start 2024-05-30T16:55:08.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.335+0000 7fe5a1c00640 1 -- start start 2024-05-30T16:55:08.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.337+0000 7fe5a1c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe59c0d30d0 0x7fe59c169660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.337+0000 7fe5a1c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe59c0cf660 con 0x7fe59c0ceff0 2024-05-30T16:55:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.337+0000 7fe5a1c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe59c0cf940 con 0x7fe59c0b38e0 2024-05-30T16:55:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.337+0000 7fe5a1c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe59c0b3cc0 con 0x7fe59c0d30d0 2024-05-30T16:55:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.337+0000 7fe593e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe59c0ceff0 0x7fe59c0d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34768/0 (socket says 172.21.15.67:34768) 2024-05-30T16:55:08.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.337+0000 7fe593e00640 1 -- 172.21.15.67:0/1684836496 learned_addr learned my addr 172.21.15.67:0/1684836496 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe5a1200640 1 --2- 172.21.15.67:0/1684836496 >> v2:172.21.15.169:3300/0 conn(0x7fe59c0d30d0 0x7fe59c169660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 312786743 0 0) 0x7fe59c0cf660 con 0x7fe59c0ceff0 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe580003520 con 0x7fe59c0ceff0 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3256731502 0 0) 0x7fe580003520 con 0x7fe59c0ceff0 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 >> v2:172.21.15.169:3300/0 conn(0x7fe59c0d30d0 msgr2=0x7fe59c169660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 --2- 172.21.15.67:0/1684836496 >> v2:172.21.15.169:3300/0 conn(0x7fe59c0d30d0 0x7fe59c169660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 >> v1:172.21.15.169:6789/0 conn(0x7fe59c0b38e0 legacy=0x7fe59c0d4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c0cfc20 con 0x7fe59c0ceff0 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fe584001670 con 0x7fe59c0ceff0 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 >> v1:172.21.15.67:6789/0 conn(0x7fe59c0ceff0 legacy=0x7fe59c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5800039b0 0x7fe580003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe580004800 0x7fe580004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe58000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c0b3cc0 con 0x7fe5800051c0 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe58000c060 con 0x7fe5800039b0 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.338+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe58000c260 con 0x7fe580004800 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.339+0000 7fe5a1200640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe58000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.339+0000 7fe59be00640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5800039b0 0x7fe580003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.339+0000 7fe593e00640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe580004800 0x7fe580004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.339+0000 7fe5a1200640 1 -- 172.21.15.67:0/1684836496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe580004800 msgr2=0x7fe580004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.339+0000 7fe5a1200640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe580004800 0x7fe580004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.339+0000 7fe5a1200640 1 -- 172.21.15.67:0/1684836496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5800039b0 msgr2=0x7fe580003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.339+0000 7fe5a1200640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5800039b0 0x7fe580003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.339+0000 7fe5a1200640 1 -- 172.21.15.67:0/1684836496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe580003520 con 0x7fe5800051c0 2024-05-30T16:55:08.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.340+0000 7fe5a1200640 1 -- 172.21.15.67:0/1684836496 reap_dead start 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.340+0000 7fe5a1200640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe58000b9c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe594009fc0 tx=0x7fe580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee7433b6643a24c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594035020 con 0x7fe5800051c0 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe594003b90 con 0x7fe5800051c0 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe59b400640 1 -- 172.21.15.67:0/1684836496 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594009c10 con 0x7fe5800051c0 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe5a1c00640 1 -- 172.21.15.67:0/1684836496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 msgr2=0x7fe58000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe5a1c00640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe58000b9c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe594009fc0 tx=0x7fe580005830 comp rx=0 tx=0).stop 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe5a1c00640 1 -- 172.21.15.67:0/1684836496 shutdown_connections 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe5a1c00640 1 --2- 172.21.15.67:0/1684836496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe58000b9c0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe5a1c00640 1 -- 172.21.15.67:0/1684836496 >> 172.21.15.67:0/1684836496 conn(0x7fe59c09d770 msgr2=0x7fe59c0d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe5a1c00640 1 -- 172.21.15.67:0/1684836496 shutdown_connections 2024-05-30T16:55:08.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.341+0000 7fe5a1c00640 1 -- 172.21.15.67:0/1684836496 wait complete. 2024-05-30T16:55:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.343+0000 7fe5a1c00640 1 Processor -- start 2024-05-30T16:55:08.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.343+0000 7fe5a1c00640 1 -- start start 2024-05-30T16:55:08.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.344+0000 7fe5a1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe59c157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.344+0000 7fe5a1c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59c0ce0d0 0x7fe59c0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.344+0000 7fe5a1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c0b2740 0x7fe59c1f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.344+0000 7fe5a1c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c0d29c0 con 0x7fe5800051c0 2024-05-30T16:55:08.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.344+0000 7fe5a1c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c0d01e0 con 0x7fe59c0b2740 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.344+0000 7fe5a1c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe59c0d2b40 con 0x7fe59c0ce0d0 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.344+0000 7fe59be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe59c157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.344+0000 7fe59be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe59c157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48348/0 (socket says 172.21.15.67:48348) 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.344+0000 7fe59be00640 1 -- 172.21.15.67:0/2161180255 learned_addr learned my addr 172.21.15.67:0/2161180255 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe59be00640 1 -- 172.21.15.67:0/2161180255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59c0ce0d0 msgr2=0x7fe59c0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe59be00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59c0ce0d0 0x7fe59c0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe59be00640 1 -- 172.21.15.67:0/2161180255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c0b2740 msgr2=0x7fe59c1f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe59be00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c0b2740 0x7fe59c1f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe59be00640 1 -- 172.21.15.67:0/2161180255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c0b7c20 con 0x7fe5800051c0 2024-05-30T16:55:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe59be00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe59c157940 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00e480 tx=0x7fe58c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe598c00640 1 -- 172.21.15.67:0/2161180255 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0052e0 con 0x7fe5800051c0 2024-05-30T16:55:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe598c00640 1 -- 172.21.15.67:0/2161180255 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58c018070 con 0x7fe5800051c0 2024-05-30T16:55:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59c0cd030 con 0x7fe5800051c0 2024-05-30T16:55:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe598c00640 1 -- 172.21.15.67:0/2161180255 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c003830 con 0x7fe5800051c0 2024-05-30T16:55:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.345+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59c0cd5f0 con 0x7fe5800051c0 2024-05-30T16:55:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.346+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59c0b2b60 con 0x7fe5800051c0 2024-05-30T16:55:08.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.348+0000 7fe598c00640 1 -- 172.21.15.67:0/2161180255 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7fe58c0039d0 con 0x7fe5800051c0 2024-05-30T16:55:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.348+0000 7fe598c00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe57806d610 0x7fe57806fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.349+0000 7fe598c00640 1 -- 172.21.15.67:0/2161180255 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fe58c08a5a0 con 0x7fe5800051c0 2024-05-30T16:55:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.349+0000 7fe593e00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe57806d610 0x7fe57806fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.349+0000 7fe593e00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe57806d610 0x7fe57806fad0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe584001200 tx=0x7fe584000ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.353+0000 7fe598c00640 1 -- 172.21.15.67:0/2161180255 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe58c05a190 con 0x7fe5800051c0 2024-05-30T16:55:08.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.574+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fe59c0ba260 con 0x7fe57806d610 2024-05-30T16:55:08.591 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.590+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.592 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.590+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.592 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.590+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.592 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.590+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.592 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:08.590+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:08.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.637+0000 7fe598c00640 1 -- 172.21.15.67:0/2161180255 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe59c0ba260 con 0x7fe57806d610 2024-05-30T16:55:08.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe57806d610 msgr2=0x7fe57806fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe57806d610 0x7fe57806fad0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe584001200 tx=0x7fe584000ed0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 msgr2=0x7fe59c157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe59c157940 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00e480 tx=0x7fe58c00e940 comp rx=0 tx=0).stop 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 shutdown_connections 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe57806d610 0x7fe57806fad0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c0b2740 0x7fe59c1f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59c0ce0d0 0x7fe59c0b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 --2- 172.21.15.67:0/2161180255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5800051c0 0x7fe59c157940 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 >> 172.21.15.67:0/2161180255 conn(0x7fe59c09d770 msgr2=0x7fe59c0b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 shutdown_connections 2024-05-30T16:55:08.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.640+0000 7fe5a1c00640 1 -- 172.21.15.67:0/2161180255 wait complete. 2024-05-30T16:55:08.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:55:08.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.858+0000 7f044d000640 1 Processor -- start 2024-05-30T16:55:08.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.858+0000 7f044d000640 1 -- start start 2024-05-30T16:55:08.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.860+0000 7f044d000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f04480cae20 0x7f0448151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.860+0000 7f044d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f04481522f0 0x7f04480cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.860+0000 7f044d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04480b3e80 con 0x7f04480caa40 2024-05-30T16:55:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.860+0000 7f044d000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f04480d0b30 con 0x7f04481522f0 2024-05-30T16:55:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.860+0000 7f044d000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f04480b4160 con 0x7f04480cae20 2024-05-30T16:55:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.860+0000 7f0447e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f04481522f0 0x7f04480cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.860+0000 7f0447400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f04480caa40 0x7f04480ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34770/0 (socket says 172.21.15.67:34770) 2024-05-30T16:55:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.860+0000 7f0447400640 1 -- 172.21.15.67:0/1212377771 learned_addr learned my addr 172.21.15.67:0/1212377771 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:55:08.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.860+0000 7f0446a00640 1 --2- 172.21.15.67:0/1212377771 >> v2:172.21.15.169:3300/0 conn(0x7f04480cae20 0x7f0448151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.861+0000 7f0446000640 1 -- 172.21.15.67:0/1212377771 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3401191681 0 0) 0x7f04480b3e80 con 0x7f04480caa40 2024-05-30T16:55:08.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.861+0000 7f0447e00640 1 -- 172.21.15.67:0/1212377771 >> v2:172.21.15.169:3300/0 conn(0x7f04480cae20 msgr2=0x7f0448151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:08.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.861+0000 7f0447e00640 1 --2- 172.21.15.67:0/1212377771 >> v2:172.21.15.169:3300/0 conn(0x7f04480cae20 0x7f0448151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.861+0000 7f0447e00640 1 -- 172.21.15.67:0/1212377771 >> v1:172.21.15.67:6789/0 conn(0x7f04480caa40 legacy=0x7f04480ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.861+0000 7f0447e00640 1 -- 172.21.15.67:0/1212377771 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04480cde80 con 0x7f04481522f0 2024-05-30T16:55:08.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.862+0000 7f0447e00640 1 --2- 172.21.15.67:0/1212377771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04481522f0 0x7f04480cf710 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f04480d0fb0 tx=0x7f044002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd58852187e1f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:08.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.862+0000 7f0446000640 1 -- 172.21.15.67:0/1212377771 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440007740 con 0x7f04481522f0 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.862+0000 7f0446000640 1 -- 172.21.15.67:0/1212377771 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04400078e0 con 0x7f04481522f0 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.862+0000 7f0446000640 1 -- 172.21.15.67:0/1212377771 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440007cb0 con 0x7f04481522f0 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.863+0000 7f044d000640 1 -- 172.21.15.67:0/1212377771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04481522f0 msgr2=0x7f04480cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.863+0000 7f044d000640 1 --2- 172.21.15.67:0/1212377771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04481522f0 0x7f04480cf710 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f04480d0fb0 tx=0x7f044002a5f0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.863+0000 7f044d000640 1 -- 172.21.15.67:0/1212377771 shutdown_connections 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.863+0000 7f044d000640 1 --2- 172.21.15.67:0/1212377771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04481522f0 0x7f04480cf710 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.863+0000 7f044d000640 1 --2- 172.21.15.67:0/1212377771 >> v2:172.21.15.169:3300/0 conn(0x7f04480cae20 0x7f0448151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.863+0000 7f044d000640 1 -- 172.21.15.67:0/1212377771 >> 172.21.15.67:0/1212377771 conn(0x7f044806b900 msgr2=0x7f044806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.863+0000 7f044d000640 1 -- 172.21.15.67:0/1212377771 shutdown_connections 2024-05-30T16:55:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.863+0000 7f044d000640 1 -- 172.21.15.67:0/1212377771 wait complete. 2024-05-30T16:55:08.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.865+0000 7f044d000640 1 Processor -- start 2024-05-30T16:55:08.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.865+0000 7f044d000640 1 -- start start 2024-05-30T16:55:08.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.866+0000 7f044d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04480caa40 0x7f0448072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.866+0000 7f044d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04480cae20 0x7f0448147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.866+0000 7f044d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04481522f0 0x7f0448148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.866+0000 7f044d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f04480ce440 con 0x7f04480caa40 2024-05-30T16:55:08.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.866+0000 7f044d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04480cf070 con 0x7f04481522f0 2024-05-30T16:55:08.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.866+0000 7f044d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0448152710 con 0x7f04480cae20 2024-05-30T16:55:08.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0447400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04480caa40 0x7f0448072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0447400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04480caa40 0x7f0448072e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48396/0 (socket says 172.21.15.67:48396) 2024-05-30T16:55:08.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0447400640 1 -- 172.21.15.67:0/1503403269 learned_addr learned my addr 172.21.15.67:0/1503403269 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0447e00640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04481522f0 0x7f0448148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0446a00640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04480cae20 0x7f0448147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0447400640 1 -- 172.21.15.67:0/1503403269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04480cae20 msgr2=0x7f0448147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0447400640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04480cae20 0x7f0448147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0447400640 1 -- 172.21.15.67:0/1503403269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04481522f0 msgr2=0x7f0448148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0447400640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04481522f0 0x7f0448148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.867+0000 7f0447400640 1 -- 172.21.15.67:0/1503403269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0448149070 con 0x7f04480caa40 2024-05-30T16:55:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.868+0000 7f0447400640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04480caa40 0x7f0448072e50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f043800c500 tx=0x7f043800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.868+0000 7f043f400640 1 -- 172.21.15.67:0/1503403269 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438018070 con 0x7f04480caa40 2024-05-30T16:55:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.868+0000 7f043f400640 1 -- 172.21.15.67:0/1503403269 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0438007580 con 0x7f04480caa40 2024-05-30T16:55:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.868+0000 7f043f400640 1 -- 172.21.15.67:0/1503403269 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438004a20 con 0x7f04480caa40 2024-05-30T16:55:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.868+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0448149350 con 0x7f04480caa40 2024-05-30T16:55:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.868+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0448149910 con 0x7f04480caa40 2024-05-30T16:55:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.869+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04480689e0 con 0x7f04480caa40 2024-05-30T16:55:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.870+0000 7f043f400640 1 -- 172.21.15.67:0/1503403269 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 24) ==== 92213+0+0 (secure 0 0 0) 0x7f043801b070 con 0x7f04480caa40 2024-05-30T16:55:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.871+0000 7f043f400640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f042806d5e0 0x7f042806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:08.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.871+0000 7f043f400640 1 -- 172.21.15.67:0/1503403269 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f0438089930 con 0x7f04480caa40 2024-05-30T16:55:08.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.871+0000 7f0446a00640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f042806d5e0 0x7f042806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:08.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.872+0000 7f0446a00640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f042806d5e0 0x7f042806faa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f04480ca2c0 tx=0x7f0434008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:08.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:08.876+0000 7f043f400640 1 -- 172.21.15.67:0/1503403269 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04380597b0 con 0x7f04480caa40 2024-05-30T16:55:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.114+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f044815b100 con 0x7f042806d5e0 2024-05-30T16:55:09.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.123+0000 7f043f400640 1 -- 172.21.15.67:0/1503403269 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f044815b100 con 0x7f042806d5e0 2024-05-30T16:55:09.125 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160 2024-05-30T16:55:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.127+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f042806d5e0 msgr2=0x7f042806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.127+0000 7f044d000640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f042806d5e0 0x7f042806faa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f04480ca2c0 tx=0x7f0434008040 comp rx=0 tx=0).stop 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.127+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04480caa40 msgr2=0x7f0448072e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.127+0000 7f044d000640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04480caa40 0x7f0448072e50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f043800c500 tx=0x7f043800c9c0 comp rx=0 tx=0).stop 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.128+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 shutdown_connections 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.128+0000 7f044d000640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f042806d5e0 0x7f042806faa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.128+0000 7f044d000640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04481522f0 0x7f0448148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.128+0000 7f044d000640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04480cae20 0x7f0448147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.128+0000 7f044d000640 1 --2- 172.21.15.67:0/1503403269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04480caa40 0x7f0448072e50 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.128+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 >> 172.21.15.67:0/1503403269 conn(0x7f044806b900 msgr2=0x7f04480b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.128+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 shutdown_connections 2024-05-30T16:55:09.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:09.128+0000 7f044d000640 1 -- 172.21.15.67:0/1503403269 wait complete. 2024-05-30T16:55:09.155 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160 2024-05-30T16:55:09.155 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:55:09.155 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:55:09.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:09.206 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:55:12.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:55:12.216 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:55:25.848 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:55:25.848 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:55:25.848 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.5219 s, 65.0 MB/s 2024-05-30T16:55:25.850 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:55:25.850 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:55:25.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:25.907 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:55:40.656 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:55:40.656 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:55:40.657 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.6921 s, 73.1 MB/s 2024-05-30T16:55:40.658 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:55:40.658 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:55:40.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:40.718 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:55:42.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:55:42.652 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:55:55.277 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:55:55.277 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:55:55.277 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.496 s, 74.1 MB/s 2024-05-30T16:55:55.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-30T16:55:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.502+0000 7f140f200640 1 Processor -- start 2024-05-30T16:55:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.502+0000 7f140f200640 1 -- start start 2024-05-30T16:55:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.504+0000 7f140f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100d0860 0x7f141015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.504+0000 7f140f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f141015bb30 con 0x7f14100ce560 2024-05-30T16:55:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.504+0000 7f140f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14100b3ba0 con 0x7f14100d0430 2024-05-30T16:55:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.504+0000 7f140f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14100b5290 con 0x7f14100d0860 2024-05-30T16:55:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.504+0000 7f140de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f14100ce560 0x7f14100b4b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55010/0 (socket says 172.21.15.67:55010) 2024-05-30T16:55:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.504+0000 7f140de00640 1 -- 172.21.15.67:0/166113334 learned_addr learned my addr 172.21.15.67:0/166113334 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.505+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 516298141 0 0) 0x7f141015bb30 con 0x7f14100ce560 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.505+0000 7f140e800640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100d0860 0x7f141015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.505+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13f4003520 con 0x7f14100ce560 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.505+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 674641693 0 0) 0x7f13f4003520 con 0x7f14100ce560 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.505+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100d0860 msgr2=0x7f141015f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.505+0000 7f140d400640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100d0860 0x7f141015f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.505+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 >> v1:172.21.15.169:6789/0 conn(0x7f14100d0430 legacy=0x7f141015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.505+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141015bdd0 con 0x7f14100ce560 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.505+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f14000019f0 con 0x7f14100ce560 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 >> v1:172.21.15.67:6789/0 conn(0x7f14100ce560 legacy=0x7f14100b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140d400640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f13f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140d400640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100d0860 0x7f13f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140d400640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13f4004d40 0x7f13f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14100b5290 con 0x7f13f40039e0 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f13f400ba70 con 0x7f13f4004d40 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f13f400de80 con 0x7f14100d0860 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140de00640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f13f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140de00640 1 -- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100d0860 msgr2=0x7f13f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140de00640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100d0860 0x7f13f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140de00640 1 -- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13f4004d40 msgr2=0x7f13f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140e800640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13f4004d40 0x7f13f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140de00640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13f4004d40 0x7f13f40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.506+0000 7f140de00640 1 -- 172.21.15.67:0/166113334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f4003520 con 0x7f13f40039e0 2024-05-30T16:55:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.507+0000 7f140de00640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f13f4003e00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f140000a830 tx=0x7f13f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6939ff461866506a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.508+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140000a480 con 0x7f13f40039e0 2024-05-30T16:55:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.508+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f140002f070 con 0x7f13f40039e0 2024-05-30T16:55:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.508+0000 7f140d400640 1 -- 172.21.15.67:0/166113334 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140002bd30 con 0x7f13f40039e0 2024-05-30T16:55:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.508+0000 7f140f200640 1 -- 172.21.15.67:0/166113334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 msgr2=0x7f13f4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.508+0000 7f140f200640 1 --2- 172.21.15.67:0/166113334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f13f4003e00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f140000a830 tx=0x7f13f4005830 comp rx=0 tx=0).stop 2024-05-30T16:55:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.508+0000 7f140e800640 1 -- 172.21.15.67:0/166113334 reap_dead start 2024-05-30T16:55:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.509+0000 7f140f200640 1 -- 172.21.15.67:0/166113334 shutdown_connections 2024-05-30T16:55:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.509+0000 7f140f200640 1 -- 172.21.15.67:0/166113334 >> 172.21.15.67:0/166113334 conn(0x7f141006bae0 msgr2=0x7f141006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.509+0000 7f140f200640 1 -- 172.21.15.67:0/166113334 shutdown_connections 2024-05-30T16:55:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.509+0000 7f140f200640 1 -- 172.21.15.67:0/166113334 wait complete. 2024-05-30T16:55:55.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.510+0000 7f140f200640 1 Processor -- start 2024-05-30T16:55:55.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.511+0000 7f140f200640 1 -- start start 2024-05-30T16:55:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f1410153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100ad610 0x7f1410072450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100adb40 0x7f14100adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f141015c670 con 0x7f13f40039e0 2024-05-30T16:55:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f141015c390 con 0x7f14100adb40 2024-05-30T16:55:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14100d0cf0 con 0x7f14100ad610 2024-05-30T16:55:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f1410153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f1410153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52692/0 (socket says 172.21.15.67:52692) 2024-05-30T16:55:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140de00640 1 -- 172.21.15.67:0/3702968835 learned_addr learned my addr 172.21.15.67:0/3702968835 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f1405400640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100ad610 0x7f1410072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140e800640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100adb40 0x7f14100adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140de00640 1 -- 172.21.15.67:0/3702968835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100ad610 msgr2=0x7f1410072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140de00640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100ad610 0x7f1410072450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140de00640 1 -- 172.21.15.67:0/3702968835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100adb40 msgr2=0x7f14100adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140de00640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100adb40 0x7f14100adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.512+0000 7f140de00640 1 -- 172.21.15.67:0/3702968835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1410072d50 con 0x7f13f40039e0 2024-05-30T16:55:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.513+0000 7f140de00640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f1410153920 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f140000a920 tx=0x7f141015bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.513+0000 7f1406a00640 1 -- 172.21.15.67:0/3702968835 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140000a480 con 0x7f13f40039e0 2024-05-30T16:55:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.513+0000 7f1406a00640 1 -- 172.21.15.67:0/3702968835 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14000019e0 con 0x7f13f40039e0 2024-05-30T16:55:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.513+0000 7f1406a00640 1 -- 172.21.15.67:0/3702968835 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400004da0 con 0x7f13f40039e0 2024-05-30T16:55:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.513+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1410073030 con 0x7f13f40039e0 2024-05-30T16:55:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.513+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14101545e0 con 0x7f13f40039e0 2024-05-30T16:55:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.514+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14100689e0 con 0x7f13f40039e0 2024-05-30T16:55:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.515+0000 7f1406a00640 1 -- 172.21.15.67:0/3702968835 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) ==== 92315+0+0 (secure 0 0 0) 0x7f1400047020 con 0x7f13f40039e0 2024-05-30T16:55:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.516+0000 7f1406a00640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f13ec06d700 0x7f13ec06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.516+0000 7f1406a00640 1 -- 172.21.15.67:0/3702968835 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f14000aacb0 con 0x7f13f40039e0 2024-05-30T16:55:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.517+0000 7f1405400640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f13ec06d700 0x7f13ec06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.517+0000 7f1405400640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f13ec06d700 0x7f13ec06fbc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f14100b4b10 tx=0x7f13f8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:55.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.521+0000 7f1406a00640 1 -- 172.21.15.67:0/3702968835 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f140007a870 con 0x7f13f40039e0 2024-05-30T16:55:55.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.758+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f141006bf90 con 0x7f13ec06d700 2024-05-30T16:55:55.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.789+0000 7f1406a00640 1 -- 172.21.15.67:0/3702968835 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f141006bf90 con 0x7f13ec06d700 2024-05-30T16:55:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.791+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f13ec06d700 msgr2=0x7f13ec06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.791+0000 7f140f200640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f13ec06d700 0x7f13ec06fbc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f14100b4b10 tx=0x7f13f8009040 comp rx=0 tx=0).stop 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.791+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 msgr2=0x7f1410153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.791+0000 7f140f200640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f1410153920 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f140000a920 tx=0x7f141015bfa0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.792+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 shutdown_connections 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.792+0000 7f140f200640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f13ec06d700 0x7f13ec06fbc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.792+0000 7f140f200640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100adb40 0x7f14100adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.792+0000 7f140f200640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14100ad610 0x7f1410072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.792+0000 7f140f200640 1 --2- 172.21.15.67:0/3702968835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13f40039e0 0x7f1410153920 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.792+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 >> 172.21.15.67:0/3702968835 conn(0x7f141006bae0 msgr2=0x7f14100ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.792+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 shutdown_connections 2024-05-30T16:55:55.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:55.792+0000 7f140f200640 1 -- 172.21.15.67:0/3702968835 wait complete. 2024-05-30T16:55:55.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:55:56.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.019+0000 7f8d79600640 1 Processor -- start 2024-05-30T16:55:56.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.019+0000 7f8d79600640 1 -- start start 2024-05-30T16:55:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.020+0000 7f8d79600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d740caa40 0x7f8d740ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.021+0000 7f8d79600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d741524c0 0x7f8d740cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.021+0000 7f8d79600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d740b3e80 con 0x7f8d740cae20 2024-05-30T16:55:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.021+0000 7f8d79600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8d740d0b30 con 0x7f8d740caa40 2024-05-30T16:55:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.021+0000 7f8d79600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8d740b4160 con 0x7f8d741524c0 2024-05-30T16:55:56.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.021+0000 7f8d73e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d740caa40 0x7f8d740ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:56.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.021+0000 7f8d78c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d741524c0 0x7f8d740cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:56.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.021+0000 7f8d73e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d740caa40 0x7f8d740ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:56846/0 (socket says 172.21.15.67:56846) 2024-05-30T16:55:56.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.021+0000 7f8d73e00640 1 -- 172.21.15.67:0/1343125005 learned_addr learned my addr 172.21.15.67:0/1343125005 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.022+0000 7f8d73e00640 1 -- 172.21.15.67:0/1343125005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d741524c0 msgr2=0x7f8d740cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.022+0000 7f8d73e00640 1 --2- 172.21.15.67:0/1343125005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d741524c0 0x7f8d740cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.022+0000 7f8d73e00640 1 -- 172.21.15.67:0/1343125005 >> v1:172.21.15.169:6789/0 conn(0x7f8d740cae20 legacy=0x7f8d74151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.022+0000 7f8d73e00640 1 -- 172.21.15.67:0/1343125005 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d740cde80 con 0x7f8d740caa40 2024-05-30T16:55:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.024+0000 7f8d73e00640 1 --2- 172.21.15.67:0/1343125005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d740caa40 0x7f8d740ce960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009870 tx=0x7f8d6402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a7f89deec8c690 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.024+0000 7f8d73400640 1 -- 172.21.15.67:0/1343125005 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64037070 con 0x7f8d740caa40 2024-05-30T16:55:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.024+0000 7f8d73400640 1 -- 172.21.15.67:0/1343125005 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d64032440 con 0x7f8d740caa40 2024-05-30T16:55:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.024+0000 7f8d73400640 1 -- 172.21.15.67:0/1343125005 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64032810 con 0x7f8d740caa40 2024-05-30T16:55:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.025+0000 7f8d79600640 1 -- 172.21.15.67:0/1343125005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d740caa40 msgr2=0x7f8d740ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.025+0000 7f8d79600640 1 --2- 172.21.15.67:0/1343125005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d740caa40 0x7f8d740ce960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009870 tx=0x7f8d6402a1c0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.025+0000 7f8d79600640 1 -- 172.21.15.67:0/1343125005 shutdown_connections 2024-05-30T16:55:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.025+0000 7f8d79600640 1 --2- 172.21.15.67:0/1343125005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d741524c0 0x7f8d740cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.025+0000 7f8d79600640 1 --2- 172.21.15.67:0/1343125005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d740caa40 0x7f8d740ce960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.025+0000 7f8d79600640 1 -- 172.21.15.67:0/1343125005 >> 172.21.15.67:0/1343125005 conn(0x7f8d7406b900 msgr2=0x7f8d7406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.025+0000 7f8d79600640 1 -- 172.21.15.67:0/1343125005 shutdown_connections 2024-05-30T16:55:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.025+0000 7f8d79600640 1 -- 172.21.15.67:0/1343125005 wait complete. 2024-05-30T16:55:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.027+0000 7f8d79600640 1 Processor -- start 2024-05-30T16:55:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.027+0000 7f8d79600640 1 -- start start 2024-05-30T16:55:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d79600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d740caa40 0x7f8d7414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d79600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d740cae20 0x7f8d7415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d79600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d741524c0 0x7f8d7415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d79600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740ceea0 con 0x7f8d740caa40 2024-05-30T16:55:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d79600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740ce440 con 0x7f8d741524c0 2024-05-30T16:55:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d79600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8d740cf020 con 0x7f8d740cae20 2024-05-30T16:55:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d73e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d740caa40 0x7f8d7414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d73e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d740caa40 0x7f8d7414b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52694/0 (socket says 172.21.15.67:52694) 2024-05-30T16:55:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d73e00640 1 -- 172.21.15.67:0/2997082507 learned_addr learned my addr 172.21.15.67:0/2997082507 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.028+0000 7f8d78c00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d741524c0 0x7f8d7415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d73e00640 1 -- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d740cae20 msgr2=0x7f8d7415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d6be00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d740cae20 0x7f8d7415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d73e00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d740cae20 0x7f8d7415a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d73e00640 1 -- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d741524c0 msgr2=0x7f8d7415aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d73e00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d741524c0 0x7f8d7415aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d73e00640 1 -- 172.21.15.67:0/2997082507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d740b1640 con 0x7f8d740caa40 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d78c00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d741524c0 0x7f8d7415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d6be00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d740cae20 0x7f8d7415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d73e00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d740caa40 0x7f8d7414b1e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000c00 tx=0x7f8d640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d70c00640 1 -- 172.21.15.67:0/2997082507 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64037040 con 0x7f8d740caa40 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d70c00640 1 -- 172.21.15.67:0/2997082507 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d64042070 con 0x7f8d740caa40 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d70c00640 1 -- 172.21.15.67:0/2997082507 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64030860 con 0x7f8d740caa40 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.029+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d74149180 con 0x7f8d740caa40 2024-05-30T16:55:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.030+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d74149740 con 0x7f8d740caa40 2024-05-30T16:55:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.031+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d740689e0 con 0x7f8d740caa40 2024-05-30T16:55:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.032+0000 7f8d70c00640 1 -- 172.21.15.67:0/2997082507 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) ==== 92315+0+0 (secure 0 0 0) 0x7f8d6402ad10 con 0x7f8d740caa40 2024-05-30T16:55:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.033+0000 7f8d70c00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d5006d700 0x7f8d5006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.033+0000 7f8d70c00640 1 -- 172.21.15.67:0/2997082507 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f8d640aaf00 con 0x7f8d740caa40 2024-05-30T16:55:56.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.033+0000 7f8d6be00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d5006d700 0x7f8d5006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:56.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.034+0000 7f8d6be00640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d5006d700 0x7f8d5006fbc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8d7416a480 tx=0x7f8d5c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:56.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.037+0000 7f8d70c00640 1 -- 172.21.15.67:0/2997082507 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d740b1640 con 0x7f8d740caa40 2024-05-30T16:55:56.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.271+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f8d7406dcf0 con 0x7f8d5006d700 2024-05-30T16:55:56.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.283+0000 7f8d70c00640 1 -- 172.21.15.67:0/2997082507 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f8d7406dcf0 con 0x7f8d5006d700 2024-05-30T16:55:56.285 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160 2024-05-30T16:55:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.285+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d5006d700 msgr2=0x7f8d5006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.285+0000 7f8d79600640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d5006d700 0x7f8d5006fbc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8d7416a480 tx=0x7f8d5c009040 comp rx=0 tx=0).stop 2024-05-30T16:55:56.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.285+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d740caa40 msgr2=0x7f8d7414b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.285+0000 7f8d79600640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d740caa40 0x7f8d7414b1e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000c00 tx=0x7f8d640046c0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.286+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 shutdown_connections 2024-05-30T16:55:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.286+0000 7f8d79600640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8d5006d700 0x7f8d5006fbc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.286+0000 7f8d79600640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d741524c0 0x7f8d7415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.286+0000 7f8d79600640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d740cae20 0x7f8d7415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.286+0000 7f8d79600640 1 --2- 172.21.15.67:0/2997082507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d740caa40 0x7f8d7414b1e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.286+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 >> 172.21.15.67:0/2997082507 conn(0x7f8d7406b900 msgr2=0x7f8d7406ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.286+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 shutdown_connections 2024-05-30T16:55:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:56.286+0000 7f8d79600640 1 -- 172.21.15.67:0/2997082507 wait complete. 2024-05-30T16:55:56.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:56.306 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160 2024-05-30T16:55:56.307 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:55:56.366 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160 2024-05-30T16:55:56.366 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3138256896" 2024-05-30T16:55:56.367 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:55:56.368 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-30T16:55:58.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:55:58.369 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160 2024-05-30T16:55:58.369 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:55:58.434 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/74b76f74-9223-4964-830c-8bc1b77fc160 2024-05-30T16:55:58.435 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3221225472" 2024-05-30T16:55:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:55:58.436 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-30T16:55:58.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92281 2024-05-30T16:55:58.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.653+0000 7f17c8c00640 1 Processor -- start 2024-05-30T16:55:58.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.653+0000 7f17c8c00640 1 -- start start 2024-05-30T16:55:58.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.655+0000 7f17c8c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f17c4159f00 0x7f17c4073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:58.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.655+0000 7f17c8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f17c415d3b0 0x7f17c40aec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:58.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.655+0000 7f17c8c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c415a2e0 con 0x7f17c40af160 2024-05-30T16:55:58.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.655+0000 7f17c8c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f17c415d860 con 0x7f17c415d3b0 2024-05-30T16:55:58.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.655+0000 7f17c8c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f17c40ad230 con 0x7f17c4159f00 2024-05-30T16:55:58.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.655+0000 7f17c2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f17c415d3b0 0x7f17c40aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:58.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.655+0000 7f17c2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f17c415d3b0 0x7f17c40aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52700/0 (socket says 172.21.15.67:52700) 2024-05-30T16:55:58.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.655+0000 7f17c2a00640 1 -- 172.21.15.67:0/585435269 learned_addr learned my addr 172.21.15.67:0/585435269 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:58.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.655+0000 7f17c3400640 1 --2- 172.21.15.67:0/585435269 >> v2:172.21.15.169:3300/0 conn(0x7f17c4159f00 0x7f17c4073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:58.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.656+0000 7f17c2a00640 1 -- 172.21.15.67:0/585435269 >> v2:172.21.15.169:3300/0 conn(0x7f17c4159f00 msgr2=0x7f17c4073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:58.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.656+0000 7f17c2a00640 1 --2- 172.21.15.67:0/585435269 >> v2:172.21.15.169:3300/0 conn(0x7f17c4159f00 0x7f17c4073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:58.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.656+0000 7f17c2a00640 1 -- 172.21.15.67:0/585435269 >> v1:172.21.15.67:6789/0 conn(0x7f17c40af160 legacy=0x7f17c415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:58.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.656+0000 7f17c2a00640 1 -- 172.21.15.67:0/585435269 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c40ad510 con 0x7f17c415d3b0 2024-05-30T16:55:58.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.656+0000 7f17c2000640 1 -- 172.21.15.67:0/585435269 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 832712748 0 0) 0x7f17bc001490 con 0x7f17c40af160 2024-05-30T16:55:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.657+0000 7f17c2a00640 1 --2- 172.21.15.67:0/585435269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 0x7f17c40aec20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f17c4070e60 tx=0x7f17b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e21acde9132866f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.657+0000 7f17c2000640 1 -- 172.21.15.67:0/585435269 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0037070 con 0x7f17c415d3b0 2024-05-30T16:55:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.657+0000 7f17c2000640 1 -- 172.21.15.67:0/585435269 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b0032440 con 0x7f17c415d3b0 2024-05-30T16:55:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.657+0000 7f17c2000640 1 -- 172.21.15.67:0/585435269 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0032810 con 0x7f17c415d3b0 2024-05-30T16:55:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.658+0000 7f17c8c00640 1 -- 172.21.15.67:0/585435269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 msgr2=0x7f17c40aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.658+0000 7f17c8c00640 1 --2- 172.21.15.67:0/585435269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 0x7f17c40aec20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f17c4070e60 tx=0x7f17b002a1e0 comp rx=0 tx=0).stop 2024-05-30T16:55:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.658+0000 7f17c8c00640 1 -- 172.21.15.67:0/585435269 shutdown_connections 2024-05-30T16:55:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.658+0000 7f17c8c00640 1 --2- 172.21.15.67:0/585435269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 0x7f17c40aec20 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.658+0000 7f17c8c00640 1 --2- 172.21.15.67:0/585435269 >> v2:172.21.15.169:3300/0 conn(0x7f17c4159f00 0x7f17c4073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.658+0000 7f17c8c00640 1 -- 172.21.15.67:0/585435269 >> 172.21.15.67:0/585435269 conn(0x7f17c40b1db0 msgr2=0x7f17c40b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.658+0000 7f17c8c00640 1 -- 172.21.15.67:0/585435269 shutdown_connections 2024-05-30T16:55:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.658+0000 7f17c8c00640 1 -- 172.21.15.67:0/585435269 wait complete. 2024-05-30T16:55:58.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.660+0000 7f17c8c00640 1 Processor -- start 2024-05-30T16:55:58.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.660+0000 7f17c8c00640 1 -- start start 2024-05-30T16:55:58.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40af160 0x7f17c41580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4159f00 0x7f17c414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 0x7f17c414daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c8c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f17c40af590 con 0x7f17c415d3b0 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17c40cfbc0 con 0x7f17c40af160 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f17c4072f10 con 0x7f17c4159f00 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 0x7f17c414daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 0x7f17c414daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52708/0 (socket says 172.21.15.67:52708) 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c3e00640 1 -- 172.21.15.67:0/3319731200 learned_addr learned my addr 172.21.15.67:0/3319731200 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c2a00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4159f00 0x7f17c414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c3400640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40af160 0x7f17c41580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c3e00640 1 -- 172.21.15.67:0/3319731200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4159f00 msgr2=0x7f17c414d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c3e00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4159f00 0x7f17c414d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c3e00640 1 -- 172.21.15.67:0/3319731200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40af160 msgr2=0x7f17c41580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c3e00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40af160 0x7f17c41580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.661+0000 7f17c3e00640 1 -- 172.21.15.67:0/3319731200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c415f450 con 0x7f17c415d3b0 2024-05-30T16:55:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.662+0000 7f17c3e00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 0x7f17c414daa0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f17bc001730 tx=0x7f17bc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.662+0000 7f17bb400640 1 -- 172.21.15.67:0/3319731200 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc017070 con 0x7f17c415d3b0 2024-05-30T16:55:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.662+0000 7f17bb400640 1 -- 172.21.15.67:0/3319731200 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc003dc0 con 0x7f17c415d3b0 2024-05-30T16:55:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.662+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c415e840 con 0x7f17c415d3b0 2024-05-30T16:55:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.662+0000 7f17bb400640 1 -- 172.21.15.67:0/3319731200 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0052b0 con 0x7f17c415d3b0 2024-05-30T16:55:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.662+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c415ee00 con 0x7f17c415d3b0 2024-05-30T16:55:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.663+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c40689e0 con 0x7f17c415d3b0 2024-05-30T16:55:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.664+0000 7f17bb400640 1 -- 172.21.15.67:0/3319731200 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) ==== 92315+0+0 (secure 0 0 0) 0x7f17bc01a070 con 0x7f17c415d3b0 2024-05-30T16:55:58.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.665+0000 7f17bb400640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f17a406d700 0x7f17a406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:58.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.665+0000 7f17bb400640 1 -- 172.21.15.67:0/3319731200 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f17bc089f90 con 0x7f17c415d3b0 2024-05-30T16:55:58.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.666+0000 7f17c3400640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f17a406d700 0x7f17a406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.666+0000 7f17c3400640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f17a406d700 0x7f17a406fbc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f17b4006fd0 tx=0x7f17b4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:58.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.670+0000 7f17bb400640 1 -- 172.21.15.67:0/3319731200 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17c415a2e0 con 0x7f17c415d3b0 2024-05-30T16:55:58.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:58.922+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92281", "target": ["mon-mgr", ""]}) -- 0x7f17c40ae320 con 0x7f17a406d700 2024-05-30T16:55:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.068+0000 7f17bb400640 1 -- 172.21.15.67:0/3319731200 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f17c40ae320 con 0x7f17a406d700 2024-05-30T16:55:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f17a406d700 msgr2=0x7f17a406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f17a406d700 0x7f17a406fbc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f17b4006fd0 tx=0x7f17b4008040 comp rx=0 tx=0).stop 2024-05-30T16:55:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 msgr2=0x7f17c414daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 0x7f17c414daa0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f17bc001730 tx=0x7f17bc00cc70 comp rx=0 tx=0).stop 2024-05-30T16:55:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 shutdown_connections 2024-05-30T16:55:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f17a406d700 0x7f17a406fbc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17c415d3b0 0x7f17c414daa0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17c4159f00 0x7f17c414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 --2- 172.21.15.67:0/3319731200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c40af160 0x7f17c41580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.071+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 >> 172.21.15.67:0/3319731200 conn(0x7f17c40b1db0 msgr2=0x7f17c415bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.072+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 shutdown_connections 2024-05-30T16:55:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.072+0000 7f17c8c00640 1 -- 172.21.15.67:0/3319731200 wait complete. 2024-05-30T16:55:59.076 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.074+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.076 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.074+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.076 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.074+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.076 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.074+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.076 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.074+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92282 2024-05-30T16:55:59.096 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.095+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.097 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.095+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.097 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.095+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.097 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.095+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.097 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:55:59.095+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:55:59.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.289+0000 7f3488e00640 1 Processor -- start 2024-05-30T16:55:59.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.289+0000 7f3488e00640 1 -- start start 2024-05-30T16:55:59.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.290+0000 7f3488e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f348415c740 0x7f3484072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:59.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.290+0000 7f3488e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34840ae0a0 0x7f348406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:59.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.291+0000 7f3488e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f348415cb20 con 0x7f348406be40 2024-05-30T16:55:59.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.291+0000 7f3488e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f34840b1540 con 0x7f34840ae0a0 2024-05-30T16:55:59.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.291+0000 7f3488e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f348406c270 con 0x7f348415c740 2024-05-30T16:55:59.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.291+0000 7f3483e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f348406be40 0x7f34840d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55024/0 (socket says 172.21.15.67:55024) 2024-05-30T16:55:59.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.291+0000 7f3483e00640 1 -- 172.21.15.67:0/2963394609 learned_addr learned my addr 172.21.15.67:0/2963394609 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:55:59.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.291+0000 7f3482a00640 1 --2- 172.21.15.67:0/2963394609 >> v2:172.21.15.67:3300/0 conn(0x7f34840ae0a0 0x7f348406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:59.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.291+0000 7f3483400640 1 --2- 172.21.15.67:0/2963394609 >> v2:172.21.15.169:3300/0 conn(0x7f348415c740 0x7f3484072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:59.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.291+0000 7f3482000640 1 -- 172.21.15.67:0/2963394609 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 318080739 0 0) 0x7f348415cb20 con 0x7f348406be40 2024-05-30T16:55:59.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.292+0000 7f3482000640 1 -- 172.21.15.67:0/2963394609 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3468003520 con 0x7f348406be40 2024-05-30T16:55:59.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.292+0000 7f3482a00640 1 -- 172.21.15.67:0/2963394609 >> v2:172.21.15.169:3300/0 conn(0x7f348415c740 msgr2=0x7f3484072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:55:59.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.292+0000 7f3482a00640 1 --2- 172.21.15.67:0/2963394609 >> v2:172.21.15.169:3300/0 conn(0x7f348415c740 0x7f3484072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:59.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.292+0000 7f3482a00640 1 -- 172.21.15.67:0/2963394609 >> v1:172.21.15.67:6789/0 conn(0x7f348406be40 legacy=0x7f34840d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:59.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.292+0000 7f3482a00640 1 -- 172.21.15.67:0/2963394609 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34840b3810 con 0x7f34840ae0a0 2024-05-30T16:55:59.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.293+0000 7f3482a00640 1 --2- 172.21.15.67:0/2963394609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34840ae0a0 0x7f348406b900 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f34840721b0 tx=0x7f346c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10ceab8a9daf538 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:59.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.293+0000 7f3482000640 1 -- 172.21.15.67:0/2963394609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c037070 con 0x7f34840ae0a0 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.293+0000 7f3482000640 1 -- 172.21.15.67:0/2963394609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c032440 con 0x7f34840ae0a0 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.293+0000 7f3482000640 1 -- 172.21.15.67:0/2963394609 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c02ae60 con 0x7f34840ae0a0 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.294+0000 7f3488e00640 1 -- 172.21.15.67:0/2963394609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34840ae0a0 msgr2=0x7f348406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.294+0000 7f3488e00640 1 --2- 172.21.15.67:0/2963394609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34840ae0a0 0x7f348406b900 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f34840721b0 tx=0x7f346c02a110 comp rx=0 tx=0).stop 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.294+0000 7f3488e00640 1 -- 172.21.15.67:0/2963394609 shutdown_connections 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.294+0000 7f3488e00640 1 --2- 172.21.15.67:0/2963394609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34840ae0a0 0x7f348406b900 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.294+0000 7f3488e00640 1 --2- 172.21.15.67:0/2963394609 >> v2:172.21.15.169:3300/0 conn(0x7f348415c740 0x7f3484072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.294+0000 7f3488e00640 1 -- 172.21.15.67:0/2963394609 >> 172.21.15.67:0/2963394609 conn(0x7f34840b6800 msgr2=0x7f34840b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.294+0000 7f3488e00640 1 -- 172.21.15.67:0/2963394609 shutdown_connections 2024-05-30T16:55:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.294+0000 7f3488e00640 1 -- 172.21.15.67:0/2963394609 wait complete. 2024-05-30T16:55:59.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.295+0000 7f3488e00640 1 Processor -- start 2024-05-30T16:55:59.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.296+0000 7f3488e00640 1 -- start start 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.296+0000 7f3488e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348406be40 0x7f34840cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3488e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34840ae0a0 0x7f34840cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3488e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348415c740 0x7f3484148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3488e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34840b1a20 con 0x7f348406be40 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3488e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34840b1740 con 0x7f34840ae0a0 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3488e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f348406c5f0 con 0x7f348415c740 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3483400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348406be40 0x7f34840cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3483400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348406be40 0x7f34840cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52768/0 (socket says 172.21.15.67:52768) 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3483400640 1 -- 172.21.15.67:0/3797645079 learned_addr learned my addr 172.21.15.67:0/3797645079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:55:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3482a00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34840ae0a0 0x7f34840cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3483e00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348415c740 0x7f3484148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3483400640 1 -- 172.21.15.67:0/3797645079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348415c740 msgr2=0x7f3484148b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3483400640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348415c740 0x7f3484148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3483400640 1 -- 172.21.15.67:0/3797645079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34840ae0a0 msgr2=0x7f34840cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:55:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3483400640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34840ae0a0 0x7f34840cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:55:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.297+0000 7f3483400640 1 -- 172.21.15.67:0/3797645079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34840ccbd0 con 0x7f348406be40 2024-05-30T16:55:59.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.298+0000 7f3483400640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348406be40 0x7f34840cbe30 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f347400d5f0 tx=0x7f347400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:59.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.298+0000 7f347b400640 1 -- 172.21.15.67:0/3797645079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740125f0 con 0x7f348406be40 2024-05-30T16:55:59.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.298+0000 7f347b400640 1 -- 172.21.15.67:0/3797645079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3474019070 con 0x7f348406be40 2024-05-30T16:55:59.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.298+0000 7f347b400640 1 -- 172.21.15.67:0/3797645079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740129d0 con 0x7f348406be40 2024-05-30T16:55:59.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.298+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34840cceb0 con 0x7f348406be40 2024-05-30T16:55:59.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.298+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3484154790 con 0x7f348406be40 2024-05-30T16:55:59.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.299+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34840690f0 con 0x7f348406be40 2024-05-30T16:55:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.300+0000 7f347b400640 1 -- 172.21.15.67:0/3797645079 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) ==== 92315+0+0 (secure 0 0 0) 0x7f3474012b70 con 0x7f348406be40 2024-05-30T16:55:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.301+0000 7f347b400640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f346006d700 0x7f346006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:55:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.301+0000 7f347b400640 1 -- 172.21.15.67:0/3797645079 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f347408ad20 con 0x7f348406be40 2024-05-30T16:55:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.302+0000 7f3482a00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f346006d700 0x7f346006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:55:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.302+0000 7f3482a00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f346006d700 0x7f346006fbc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f346c034000 tx=0x7f346c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:55:59.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.306+0000 7f347b400640 1 -- 172.21.15.67:0/3797645079 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f347405a8e0 con 0x7f348406be40 2024-05-30T16:55:59.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:55:59.547+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92282", "target": ["mon-mgr", ""]}) -- 0x7f34840cf760 con 0x7f346006d700 2024-05-30T16:56:00.077 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.076+0000 7f0c4d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.078 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.076+0000 7f0c4d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.078 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.076+0000 7f0c4d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.078 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.076+0000 7f0c4d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.078 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.076+0000 7f0c4d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.193+0000 7f347b400640 1 -- 172.21.15.67:0/3797645079 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f34840cf760 con 0x7f346006d700 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.196+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f346006d700 msgr2=0x7f346006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.196+0000 7f3488e00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f346006d700 0x7f346006fbc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f346c034000 tx=0x7f346c02e000 comp rx=0 tx=0).stop 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.196+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348406be40 msgr2=0x7f34840cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.196+0000 7f3488e00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348406be40 0x7f34840cbe30 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f347400d5f0 tx=0x7f347400dab0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.197+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 shutdown_connections 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.197+0000 7f3488e00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f346006d700 0x7f346006fbc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.197+0000 7f3488e00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348415c740 0x7f3484148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.197+0000 7f3488e00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34840ae0a0 0x7f34840cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.197+0000 7f3488e00640 1 --2- 172.21.15.67:0/3797645079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f348406be40 0x7f34840cbe30 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.197+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 >> 172.21.15.67:0/3797645079 conn(0x7f34840b6800 msgr2=0x7f34840b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.197+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 shutdown_connections 2024-05-30T16:56:00.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.197+0000 7f3488e00640 1 -- 172.21.15.67:0/3797645079 wait complete. 2024-05-30T16:56:00.207 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.205+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.207 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.205+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.207 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.205+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.207 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.205+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.207 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:00.205+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:00.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92283 2024-05-30T16:56:00.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.602+0000 7f1973200640 1 Processor -- start 2024-05-30T16:56:00.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.602+0000 7f1973200640 1 -- start start 2024-05-30T16:56:00.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.606+0000 7f1973200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 0x7f197415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:00.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.606+0000 7f1973200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19740affb0 0x7f197406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:00.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.606+0000 7f1973200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19740b4720 con 0x7f19740ccc50 2024-05-30T16:56:00.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.606+0000 7f1973200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f197415da10 con 0x7f19740affb0 2024-05-30T16:56:00.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.606+0000 7f1973200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19740b4a00 con 0x7f19740afbd0 2024-05-30T16:56:00.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.607+0000 7f1971e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 0x7f197415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.607+0000 7f1972800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f19740ccc50 0x7f1974160040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55040/0 (socket says 172.21.15.67:55040) 2024-05-30T16:56:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.607+0000 7f1972800640 1 -- 172.21.15.67:0/2570311101 learned_addr learned my addr 172.21.15.67:0/2570311101 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.607+0000 7f1971400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19740affb0 0x7f197406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.608+0000 7f1971e00640 1 -- 172.21.15.67:0/2570311101 >> v2:172.21.15.169:3300/0 conn(0x7f19740affb0 msgr2=0x7f197406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:00.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.608+0000 7f1971e00640 1 --2- 172.21.15.67:0/2570311101 >> v2:172.21.15.169:3300/0 conn(0x7f19740affb0 0x7f197406c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.608+0000 7f1971e00640 1 -- 172.21.15.67:0/2570311101 >> v1:172.21.15.67:6789/0 conn(0x7f19740ccc50 legacy=0x7f1974160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:00.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.608+0000 7f1971e00640 1 -- 172.21.15.67:0/2570311101 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740b4ce0 con 0x7f19740afbd0 2024-05-30T16:56:00.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.610+0000 7f1971e00640 1 --2- 172.21.15.67:0/2570311101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 0x7f197415d4d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f19640097a0 tx=0x7f196402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ec9f2b1bd919eb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:00.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.610+0000 7f1970a00640 1 -- 172.21.15.67:0/2570311101 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964037070 con 0x7f19740afbd0 2024-05-30T16:56:00.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.610+0000 7f1970a00640 1 -- 172.21.15.67:0/2570311101 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1964032440 con 0x7f19740afbd0 2024-05-30T16:56:00.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.610+0000 7f1970a00640 1 -- 172.21.15.67:0/2570311101 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964032810 con 0x7f19740afbd0 2024-05-30T16:56:00.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.612+0000 7f1973200640 1 -- 172.21.15.67:0/2570311101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 msgr2=0x7f197415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:00.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.612+0000 7f1973200640 1 --2- 172.21.15.67:0/2570311101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 0x7f197415d4d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f19640097a0 tx=0x7f196402a180 comp rx=0 tx=0).stop 2024-05-30T16:56:00.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.612+0000 7f1973200640 1 -- 172.21.15.67:0/2570311101 shutdown_connections 2024-05-30T16:56:00.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.612+0000 7f1973200640 1 --2- 172.21.15.67:0/2570311101 >> v2:172.21.15.169:3300/0 conn(0x7f19740affb0 0x7f197406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.612+0000 7f1973200640 1 --2- 172.21.15.67:0/2570311101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 0x7f197415d4d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.612+0000 7f1973200640 1 -- 172.21.15.67:0/2570311101 >> 172.21.15.67:0/2570311101 conn(0x7f1974072260 msgr2=0x7f1974072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:00.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.612+0000 7f1973200640 1 -- 172.21.15.67:0/2570311101 shutdown_connections 2024-05-30T16:56:00.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.612+0000 7f1973200640 1 -- 172.21.15.67:0/2570311101 wait complete. 2024-05-30T16:56:00.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.615+0000 7f1973200640 1 Processor -- start 2024-05-30T16:56:00.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.616+0000 7f1973200640 1 -- start start 2024-05-30T16:56:00.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.617+0000 7f1973200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 0x7f197415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:00.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.617+0000 7f1973200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19740affb0 0x7f19740b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:00.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.617+0000 7f1973200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19740ccc50 0x7f197414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:00.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.617+0000 7f1973200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f19740cf8c0 con 0x7f19740ccc50 2024-05-30T16:56:00.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.617+0000 7f1973200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19740b52a0 con 0x7f19740affb0 2024-05-30T16:56:00.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.617+0000 7f1973200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19740cfa40 con 0x7f19740afbd0 2024-05-30T16:56:00.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.617+0000 7f1972800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19740ccc50 0x7f197414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:00.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.618+0000 7f1972800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19740ccc50 0x7f197414f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52804/0 (socket says 172.21.15.67:52804) 2024-05-30T16:56:00.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.618+0000 7f1972800640 1 -- 172.21.15.67:0/3366681016 learned_addr learned my addr 172.21.15.67:0/3366681016 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:00.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.618+0000 7f1971e00640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 0x7f197415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:00.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.618+0000 7f1971400640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19740affb0 0x7f19740b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:00.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.618+0000 7f1972800640 1 -- 172.21.15.67:0/3366681016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 msgr2=0x7f197415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:00.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.618+0000 7f1972800640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 0x7f197415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.619+0000 7f1972800640 1 -- 172.21.15.67:0/3366681016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19740affb0 msgr2=0x7f19740b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:00.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.619+0000 7f1972800640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19740affb0 0x7f19740b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:00.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.619+0000 7f1972800640 1 -- 172.21.15.67:0/3366681016 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19741499d0 con 0x7f19740ccc50 2024-05-30T16:56:00.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.619+0000 7f1972800640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19740ccc50 0x7f197414f120 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f196c00cad0 tx=0x7f196c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:00.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.620+0000 7f196a000640 1 -- 172.21.15.67:0/3366681016 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c017070 con 0x7f19740ccc50 2024-05-30T16:56:00.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.620+0000 7f196a000640 1 -- 172.21.15.67:0/3366681016 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c00ac30 con 0x7f19740ccc50 2024-05-30T16:56:00.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.620+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1974149cb0 con 0x7f19740ccc50 2024-05-30T16:56:00.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.620+0000 7f196a000640 1 -- 172.21.15.67:0/3366681016 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c012400 con 0x7f19740ccc50 2024-05-30T16:56:00.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.620+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1974149670 con 0x7f19740ccc50 2024-05-30T16:56:00.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.623+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19740690f0 con 0x7f19740ccc50 2024-05-30T16:56:00.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.624+0000 7f196a000640 1 -- 172.21.15.67:0/3366681016 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 92519+0+0 (secure 0 0 0) 0x7f196c01a070 con 0x7f19740ccc50 2024-05-30T16:56:00.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.626+0000 7f196a000640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f195406d970 0x7f195406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:00.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.627+0000 7f196a000640 1 -- 172.21.15.67:0/3366681016 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f196c08a620 con 0x7f19740ccc50 2024-05-30T16:56:00.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.627+0000 7f1971e00640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f195406d970 0x7f195406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.630+0000 7f196a000640 1 -- 172.21.15.67:0/3366681016 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f196c059b90 con 0x7f19740ccc50 2024-05-30T16:56:00.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:00.631+0000 7f1971e00640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f195406d970 0x7f195406fe30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1964034000 tx=0x7f196402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:01.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.152+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92283", "target": ["mon-mgr", ""]}) -- 0x7f197414c000 con 0x7f195406d970 2024-05-30T16:56:01.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.634+0000 7f196a000640 1 -- 172.21.15.67:0/3366681016 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f197414c000 con 0x7f195406d970 2024-05-30T16:56:01.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.639+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f195406d970 msgr2=0x7f195406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:01.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.639+0000 7f1973200640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f195406d970 0x7f195406fe30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1964034000 tx=0x7f196402e000 comp rx=0 tx=0).stop 2024-05-30T16:56:01.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.639+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19740ccc50 msgr2=0x7f197414f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:01.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.639+0000 7f1973200640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19740ccc50 0x7f197414f120 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f196c00cad0 tx=0x7f196c00cf90 comp rx=0 tx=0).stop 2024-05-30T16:56:01.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.640+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 shutdown_connections 2024-05-30T16:56:01.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.640+0000 7f1973200640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f195406d970 0x7f195406fe30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:01.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.640+0000 7f1973200640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19740ccc50 0x7f197414f120 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:01.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.640+0000 7f1973200640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19740affb0 0x7f19740b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:01.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.640+0000 7f1973200640 1 --2- 172.21.15.67:0/3366681016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19740afbd0 0x7f197415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:01.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.640+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 >> 172.21.15.67:0/3366681016 conn(0x7f1974072260 msgr2=0x7f197415dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:01.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.640+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 shutdown_connections 2024-05-30T16:56:01.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:01.640+0000 7f1973200640 1 -- 172.21.15.67:0/3366681016 wait complete. 2024-05-30T16:56:01.698 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92284 2024-05-30T16:56:01.712 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:01.712+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:01.712 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:01.712+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:01.712 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:01.712+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:01.713 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:01.712+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:01.713 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:01.712+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:02.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.124+0000 7f4ed8800640 1 Processor -- start 2024-05-30T16:56:02.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.124+0000 7f4ed8800640 1 -- start start 2024-05-30T16:56:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.127+0000 7f4ed8800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4ed00d0de0 0x7f4ed0072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.128+0000 7f4ed5e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4ed0162c20 0x7f4ed00d06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35260/0 (socket says 172.21.15.67:35260) 2024-05-30T16:56:02.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.130+0000 7f4ed8800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed006e450 0x7f4ed00ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:02.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.131+0000 7f4ed8800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed0072db0 con 0x7f4ed0162c20 2024-05-30T16:56:02.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.131+0000 7f4ed8800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed0072b80 con 0x7f4ed00d0de0 2024-05-30T16:56:02.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.131+0000 7f4ed8800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed0163000 con 0x7f4ed006e450 2024-05-30T16:56:02.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.131+0000 7f4ed5400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4ed00d0de0 0x7f4ed0072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:02.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.131+0000 7f4ed5400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4ed00d0de0 0x7f4ed0072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:51400/0 (socket says 172.21.15.67:51400) 2024-05-30T16:56:02.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.131+0000 7f4ed5400640 1 -- 172.21.15.67:0/1966407698 learned_addr learned my addr 172.21.15.67:0/1966407698 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:02.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.133+0000 7f4ed5400640 1 -- 172.21.15.67:0/1966407698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed006e450 msgr2=0x7f4ed00ad1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:56:02.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.133+0000 7f4ed5400640 1 --2- 172.21.15.67:0/1966407698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed006e450 0x7f4ed00ad1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:02.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.133+0000 7f4ed5400640 1 -- 172.21.15.67:0/1966407698 >> v1:172.21.15.67:6789/0 conn(0x7f4ed0162c20 legacy=0x7f4ed00d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:02.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.133+0000 7f4ed5400640 1 -- 172.21.15.67:0/1966407698 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed0073090 con 0x7f4ed00d0de0 2024-05-30T16:56:02.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.134+0000 7f4ed5e00640 1 --1- 172.21.15.67:0/1966407698 >> v1:172.21.15.67:6789/0 conn(0x7f4ed0162c20 0x7f4ed00d06d0 :-1 s=CLOSED pgs=0 cs=0 l=0).handle_server_banner_and_identify state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-30T16:56:02.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.138+0000 7f4ed5400640 1 --2- 172.21.15.67:0/1966407698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed00d0de0 0x7f4ed0072640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00afb30 tx=0x7f4ecc02a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2afd2a83f9f20f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:02.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.140+0000 7f4ed4a00640 1 -- 172.21.15.67:0/1966407698 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc036070 con 0x7f4ed00d0de0 2024-05-30T16:56:02.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.140+0000 7f4ed4a00640 1 -- 172.21.15.67:0/1966407698 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ecc031600 con 0x7f4ed00d0de0 2024-05-30T16:56:02.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.140+0000 7f4ed4a00640 1 -- 172.21.15.67:0/1966407698 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc036070 con 0x7f4ed00d0de0 2024-05-30T16:56:02.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.146+0000 7f4ed8800640 1 -- 172.21.15.67:0/1966407698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed00d0de0 msgr2=0x7f4ed0072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:02.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.147+0000 7f4ed8800640 1 --2- 172.21.15.67:0/1966407698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed00d0de0 0x7f4ed0072640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00afb30 tx=0x7f4ecc02a640 comp rx=0 tx=0).stop 2024-05-30T16:56:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.147+0000 7f4ed8800640 1 -- 172.21.15.67:0/1966407698 shutdown_connections 2024-05-30T16:56:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.148+0000 7f4ed8800640 1 --2- 172.21.15.67:0/1966407698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed006e450 0x7f4ed00ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.148+0000 7f4ed8800640 1 --2- 172.21.15.67:0/1966407698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed00d0de0 0x7f4ed0072640 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.148+0000 7f4ed8800640 1 -- 172.21.15.67:0/1966407698 >> 172.21.15.67:0/1966407698 conn(0x7f4ed006bd20 msgr2=0x7f4ed006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:02.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.152+0000 7f4ed8800640 1 -- 172.21.15.67:0/1966407698 shutdown_connections 2024-05-30T16:56:02.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.154+0000 7f4ed8800640 1 -- 172.21.15.67:0/1966407698 wait complete. 2024-05-30T16:56:02.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.159+0000 7f4ed8800640 1 Processor -- start 2024-05-30T16:56:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.160+0000 7f4ed8800640 1 -- start start 2024-05-30T16:56:02.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.162+0000 7f4ed8800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed006e450 0x7f4ed01f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:02.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.163+0000 7f4ed8800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00d0de0 0x7f4ed0165b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:02.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.163+0000 7f4ed5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed006e450 0x7f4ed01f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:02.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.163+0000 7f4ed8800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed0162c20 0x7f4ed01557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:02.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.163+0000 7f4ed8800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed0073650 con 0x7f4ed006e450 2024-05-30T16:56:02.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.163+0000 7f4ed8800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed00739c0 con 0x7f4ed0162c20 2024-05-30T16:56:02.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.163+0000 7f4ed8800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed0073e50 con 0x7f4ed00d0de0 2024-05-30T16:56:02.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.164+0000 7f4ed6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed0162c20 0x7f4ed01557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:02.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.165+0000 7f4ed6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed0162c20 0x7f4ed01557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:51412/0 (socket says 172.21.15.67:51412) 2024-05-30T16:56:02.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.165+0000 7f4ed6800640 1 -- 172.21.15.67:0/1986092394 learned_addr learned my addr 172.21.15.67:0/1986092394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:02.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.166+0000 7f4ed5400640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00d0de0 0x7f4ed0165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:02.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.167+0000 7f4ed5e00640 1 -- 172.21.15.67:0/1986092394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00d0de0 msgr2=0x7f4ed0165b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:02.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.167+0000 7f4ed5e00640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00d0de0 0x7f4ed0165b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:02.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.167+0000 7f4ed5e00640 1 -- 172.21.15.67:0/1986092394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed0162c20 msgr2=0x7f4ed01557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:02.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.167+0000 7f4ed5e00640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed0162c20 0x7f4ed01557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:02.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.167+0000 7f4ed5e00640 1 -- 172.21.15.67:0/1986092394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed0156a90 con 0x7f4ed006e450 2024-05-30T16:56:02.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.168+0000 7f4ed5e00640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed006e450 0x7f4ed01f9ab0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b410 tx=0x7f4ec000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:02.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.170+0000 7f4eca000640 1 -- 172.21.15.67:0/1986092394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0015020 con 0x7f4ed006e450 2024-05-30T16:56:02.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.170+0000 7f4eca000640 1 -- 172.21.15.67:0/1986092394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0011830 con 0x7f4ed006e450 2024-05-30T16:56:02.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.170+0000 7f4eca000640 1 -- 172.21.15.67:0/1986092394 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0011c20 con 0x7f4ed006e450 2024-05-30T16:56:02.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.171+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed0156d70 con 0x7f4ed006e450 2024-05-30T16:56:02.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.171+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed0157330 con 0x7f4ed006e450 2024-05-30T16:56:02.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.174+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed00689e0 con 0x7f4ed006e450 2024-05-30T16:56:02.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.180+0000 7f4eca000640 1 -- 172.21.15.67:0/1986092394 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) ==== 92519+0+0 (secure 0 0 0) 0x7f4ec0018020 con 0x7f4ed006e450 2024-05-30T16:56:02.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.182+0000 7f4eca000640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406da10 0x7f4eb406fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:02.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.182+0000 7f4eca000640 1 -- 172.21.15.67:0/1986092394 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f4ec008af10 con 0x7f4ed006e450 2024-05-30T16:56:02.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.186+0000 7f4ed5400640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406da10 0x7f4eb406fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:02.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.193+0000 7f4eca000640 1 -- 172.21.15.67:0/1986092394 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ec001f020 con 0x7f4ed006e450 2024-05-30T16:56:02.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.211+0000 7f4ed5400640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406da10 0x7f4eb406fed0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc033000 tx=0x7f4ecc009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:02.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.577+0000 7f4eca000640 1 -- 172.21.15.67:0/1986092394 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f4ec00569e0 con 0x7f4ed006e450 2024-05-30T16:56:02.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:02.689+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92284", "target": ["mon-mgr", ""]}) -- 0x7f4ed0074a90 con 0x7f4eb406da10 2024-05-30T16:56:03.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.411+0000 7f4eca000640 1 -- 172.21.15.67:0/1986092394 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ed0074a90 con 0x7f4eb406da10 2024-05-30T16:56:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406da10 msgr2=0x7f4eb406fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406da10 0x7f4eb406fed0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc033000 tx=0x7f4ecc009000 comp rx=0 tx=0).stop 2024-05-30T16:56:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed006e450 msgr2=0x7f4ed01f9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed006e450 0x7f4ed01f9ab0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b410 tx=0x7f4ec000b8d0 comp rx=0 tx=0).stop 2024-05-30T16:56:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 shutdown_connections 2024-05-30T16:56:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406da10 0x7f4eb406fed0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed0162c20 0x7f4ed01557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00d0de0 0x7f4ed0165b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 --2- 172.21.15.67:0/1986092394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed006e450 0x7f4ed01f9ab0 secure :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b410 tx=0x7f4ec000b8d0 comp rx=0 tx=0).stop 2024-05-30T16:56:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.415+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 >> 172.21.15.67:0/1986092394 conn(0x7f4ed006bd20 msgr2=0x7f4ed00aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.416+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 shutdown_connections 2024-05-30T16:56:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.416+0000 7f4ed8800640 1 -- 172.21.15.67:0/1986092394 wait complete. 2024-05-30T16:56:03.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92285 2024-05-30T16:56:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.914+0000 7f1760800640 1 Processor -- start 2024-05-30T16:56:03.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.914+0000 7f1760800640 1 -- start start 2024-05-30T16:56:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.916+0000 7f1760800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f17580b3670 0x7f1758162650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.916+0000 7f1760800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17580b5310 con 0x7f17580b3a50 2024-05-30T16:56:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.916+0000 7f1760800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17580d2d30 con 0x7f17580b4e80 2024-05-30T16:56:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.916+0000 7f1760800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1758162450 con 0x7f17580b3670 2024-05-30T16:56:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.916+0000 7f175d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f17580b3a50 0x7f1758162b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35274/0 (socket says 172.21.15.67:35274) 2024-05-30T16:56:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.916+0000 7f175d400640 1 -- 172.21.15.67:0/1906547614 learned_addr learned my addr 172.21.15.67:0/1906547614 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.916+0000 7f175de00640 1 --2- 172.21.15.67:0/1906547614 >> v2:172.21.15.67:3300/0 conn(0x7f17580b3670 0x7f1758162650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.917+0000 7f175de00640 1 -- 172.21.15.67:0/1906547614 >> v1:172.21.15.169:6789/0 conn(0x7f17580b4e80 legacy=0x7f1758160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:03.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.917+0000 7f175de00640 1 -- 172.21.15.67:0/1906547614 >> v1:172.21.15.67:6789/0 conn(0x7f17580b3a50 legacy=0x7f1758162b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:03.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.917+0000 7f175de00640 1 -- 172.21.15.67:0/1906547614 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17580d3010 con 0x7f17580b3670 2024-05-30T16:56:03.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.918+0000 7f175de00640 1 --2- 172.21.15.67:0/1906547614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b3670 0x7f1758162650 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1754009de0 tx=0x7f175402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da84d4fd2366ba6a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:03.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.919+0000 7f175ca00640 1 -- 172.21.15.67:0/1906547614 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754007740 con 0x7f17580b3670 2024-05-30T16:56:03.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.919+0000 7f175ca00640 1 -- 172.21.15.67:0/1906547614 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17540078e0 con 0x7f17580b3670 2024-05-30T16:56:03.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.919+0000 7f175ca00640 1 -- 172.21.15.67:0/1906547614 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754007cb0 con 0x7f17580b3670 2024-05-30T16:56:03.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.921+0000 7f1760800640 1 -- 172.21.15.67:0/1906547614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b3670 msgr2=0x7f1758162650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:03.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.921+0000 7f1760800640 1 --2- 172.21.15.67:0/1906547614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b3670 0x7f1758162650 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1754009de0 tx=0x7f175402a6c0 comp rx=0 tx=0).stop 2024-05-30T16:56:03.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.922+0000 7f1760800640 1 -- 172.21.15.67:0/1906547614 shutdown_connections 2024-05-30T16:56:03.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.922+0000 7f1760800640 1 --2- 172.21.15.67:0/1906547614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b3670 0x7f1758162650 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:03.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.922+0000 7f1760800640 1 -- 172.21.15.67:0/1906547614 >> 172.21.15.67:0/1906547614 conn(0x7f175808bfc0 msgr2=0x7f17580d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:03.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.922+0000 7f1760800640 1 -- 172.21.15.67:0/1906547614 shutdown_connections 2024-05-30T16:56:03.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.923+0000 7f1760800640 1 -- 172.21.15.67:0/1906547614 wait complete. 2024-05-30T16:56:03.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.926+0000 7f1760800640 1 Processor -- start 2024-05-30T16:56:03.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.927+0000 7f1760800640 1 -- start start 2024-05-30T16:56:03.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f1760800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580b3670 0x7f1758150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:03.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f1760800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17580b3a50 0x7f17580b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:03.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f1760800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b8ef0 0x7f17580b8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:03.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f1760800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f17581632a0 con 0x7f17580b8ef0 2024-05-30T16:56:03.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f1760800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17580b56e0 con 0x7f17580b3670 2024-05-30T16:56:03.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f1760800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1758163420 con 0x7f17580b3a50 2024-05-30T16:56:03.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f175de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580b3670 0x7f1758150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:03.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f175de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580b3670 0x7f1758150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:51418/0 (socket says 172.21.15.67:51418) 2024-05-30T16:56:03.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f175de00640 1 -- 172.21.15.67:0/1337903601 learned_addr learned my addr 172.21.15.67:0/1337903601 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:03.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.929+0000 7f175e800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b8ef0 0x7f17580b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:03.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.930+0000 7f175d400640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17580b3a50 0x7f17580b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:03.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.930+0000 7f175e800640 1 -- 172.21.15.67:0/1337903601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17580b3a50 msgr2=0x7f17580b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:03.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.930+0000 7f175e800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17580b3a50 0x7f17580b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:03.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.930+0000 7f175e800640 1 -- 172.21.15.67:0/1337903601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580b3670 msgr2=0x7f1758150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:03.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.930+0000 7f175e800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580b3670 0x7f1758150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:03.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.930+0000 7f175e800640 1 -- 172.21.15.67:0/1337903601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17581e89d0 con 0x7f17580b8ef0 2024-05-30T16:56:03.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.931+0000 7f175e800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b8ef0 0x7f17580b8320 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f174800c3f0 tx=0x7f174800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:03.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.931+0000 7f1752000640 1 -- 172.21.15.67:0/1337903601 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748017070 con 0x7f17580b8ef0 2024-05-30T16:56:03.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.932+0000 7f1752000640 1 -- 172.21.15.67:0/1337903601 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17480052d0 con 0x7f17580b8ef0 2024-05-30T16:56:03.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.932+0000 7f1752000640 1 -- 172.21.15.67:0/1337903601 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748003830 con 0x7f17580b8ef0 2024-05-30T16:56:03.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.932+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17581e8430 con 0x7f17580b8ef0 2024-05-30T16:56:03.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.934+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17581e86f0 con 0x7f17580b8ef0 2024-05-30T16:56:03.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.936+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f175806e520 con 0x7f17580b8ef0 2024-05-30T16:56:03.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.939+0000 7f1752000640 1 -- 172.21.15.67:0/1337903601 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f174801a070 con 0x7f17580b8ef0 2024-05-30T16:56:03.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.941+0000 7f1752000640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f173c06da20 0x7f173c06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:03.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.941+0000 7f1752000640 1 -- 172.21.15.67:0/1337903601 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f174808a0f0 con 0x7f17580b8ef0 2024-05-30T16:56:03.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.942+0000 7f175de00640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f173c06da20 0x7f173c06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:03.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.947+0000 7f1752000640 1 -- 172.21.15.67:0/1337903601 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f175806e520 con 0x7f17580b8ef0 2024-05-30T16:56:03.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:03.948+0000 7f175de00640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f173c06da20 0x7f173c06fee0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f175402a700 tx=0x7f175402f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:04.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:04.457+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92285", "target": ["mon-mgr", ""]}) -- 0x7f1758161360 con 0x7f173c06da20 2024-05-30T16:56:05.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.131+0000 7f1752000640 1 -- 172.21.15.67:0/1337903601 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1758161360 con 0x7f173c06da20 2024-05-30T16:56:05.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.138+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f173c06da20 msgr2=0x7f173c06fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:05.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.138+0000 7f1760800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f173c06da20 0x7f173c06fee0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f175402a700 tx=0x7f175402f000 comp rx=0 tx=0).stop 2024-05-30T16:56:05.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.138+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b8ef0 msgr2=0x7f17580b8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:05.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.138+0000 7f1760800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b8ef0 0x7f17580b8320 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f174800c3f0 tx=0x7f174800c8b0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.139+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 shutdown_connections 2024-05-30T16:56:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.139+0000 7f1760800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f173c06da20 0x7f173c06fee0 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f175402a700 tx=0x7f175402f000 comp rx=0 tx=0).stop 2024-05-30T16:56:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.139+0000 7f1760800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17580b8ef0 0x7f17580b8320 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.139+0000 7f1760800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17580b3a50 0x7f17580b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.139+0000 7f1760800640 1 --2- 172.21.15.67:0/1337903601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17580b3670 0x7f1758150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.139+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 >> 172.21.15.67:0/1337903601 conn(0x7f175808bfc0 msgr2=0x7f17580d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.139+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 shutdown_connections 2024-05-30T16:56:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.139+0000 7f1760800640 1 -- 172.21.15.67:0/1337903601 wait complete. 2024-05-30T16:56:05.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:56:05.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.640+0000 7ff1d2a00640 1 Processor -- start 2024-05-30T16:56:05.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.640+0000 7ff1d2a00640 1 -- start start 2024-05-30T16:56:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.643+0000 7ff1d2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1cc06eec0 0x7ff1cc0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.643+0000 7ff1d2a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1cc15b190 con 0x7ff1cc15a9a0 2024-05-30T16:56:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.643+0000 7ff1d2a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1cc0b3960 con 0x7ff1cc15ad80 2024-05-30T16:56:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.643+0000 7ff1d2a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff1cc0b3c40 con 0x7ff1cc06eec0 2024-05-30T16:56:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.643+0000 7ff1cbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff1cc15a9a0 0x7ff1cc06e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35276/0 (socket says 172.21.15.67:35276) 2024-05-30T16:56:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.644+0000 7ff1cbe00640 1 -- 172.21.15.67:0/3268409070 learned_addr learned my addr 172.21.15.67:0/3268409070 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.646+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375987313 0 0) 0x7ff1cc15b190 con 0x7ff1cc15a9a0 2024-05-30T16:56:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.646+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1b0003520 con 0x7ff1cc15a9a0 2024-05-30T16:56:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.649+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2008894689 0 0) 0x7ff1b0003520 con 0x7ff1cc15a9a0 2024-05-30T16:56:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.649+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 >> v2:172.21.15.67:3300/0 conn(0x7ff1cc06eec0 msgr2=0x7ff1cc0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:05.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.649+0000 7ff1cb400640 1 --2- 172.21.15.67:0/3268409070 >> v2:172.21.15.67:3300/0 conn(0x7ff1cc06eec0 0x7ff1cc0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.649+0000 7ff1caa00640 1 --2- 172.21.15.67:0/3268409070 >> v2:172.21.15.67:3300/0 conn(0x7ff1cc06eec0 0x7ff1cc0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.650+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 >> v1:172.21.15.169:6789/0 conn(0x7ff1cc15ad80 legacy=0x7ff1cc15c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.650+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc0b3f20 con 0x7ff1cc15a9a0 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.650+0000 7ff1cb400640 1 --2- 172.21.15.67:0/3268409070 >> v2:172.21.15.67:3300/0 conn(0x7ff1cc06eec0 0x7ff1cc0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.650+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff1bc001670 con 0x7ff1cc15a9a0 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.650+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 >> v1:172.21.15.67:6789/0 conn(0x7ff1cc15a9a0 legacy=0x7ff1cc06e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.650+0000 7ff1caa00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b00037f0 0x7ff1b0003c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.651+0000 7ff1caa00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b000f3b0 0x7ff1b00117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.651+0000 7ff1caa00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b0011ce0 0x7ff1b0014170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.651+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc0b3c40 con 0x7ff1b00037f0 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.651+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0004bf0 con 0x7ff1b000f3b0 2024-05-30T16:56:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.651+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b0004df0 con 0x7ff1b0011ce0 2024-05-30T16:56:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.656+0000 7ff1d0a00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b0011ce0 0x7ff1b0014170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.656+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b00037f0 0x7ff1b0003c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:05.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.657+0000 7ff1cbe00640 1 -- 172.21.15.67:0/3268409070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b0011ce0 msgr2=0x7ff1b0014170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:05.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.657+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b0011ce0 0x7ff1b0014170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.657+0000 7ff1cbe00640 1 -- 172.21.15.67:0/3268409070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b000f3b0 msgr2=0x7ff1b00117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.657+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b000f3b0 0x7ff1b00117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.657+0000 7ff1cbe00640 1 -- 172.21.15.67:0/3268409070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b0003520 con 0x7ff1b00037f0 2024-05-30T16:56:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.661+0000 7ff1d0a00640 1 -- 172.21.15.67:0/3268409070 reap_dead start 2024-05-30T16:56:05.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.663+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b00037f0 0x7ff1b0003c10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00c110 tx=0x7ff1b0004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1975dbcdab8c4f51 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.667+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc007c40 con 0x7ff1b00037f0 2024-05-30T16:56:05.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.667+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc005ad0 con 0x7ff1b00037f0 2024-05-30T16:56:05.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.667+0000 7ff1caa00640 1 -- 172.21.15.67:0/3268409070 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc007070 con 0x7ff1b00037f0 2024-05-30T16:56:05.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.668+0000 7ff1d2a00640 1 -- 172.21.15.67:0/3268409070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b00037f0 msgr2=0x7ff1b0003c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:05.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.668+0000 7ff1d2a00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b00037f0 0x7ff1b0003c10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00c110 tx=0x7ff1b0004820 comp rx=0 tx=0).stop 2024-05-30T16:56:05.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.673+0000 7ff1d2a00640 1 -- 172.21.15.67:0/3268409070 shutdown_connections 2024-05-30T16:56:05.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.673+0000 7ff1d2a00640 1 --2- 172.21.15.67:0/3268409070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b00037f0 0x7ff1b0003c10 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.673+0000 7ff1d2a00640 1 -- 172.21.15.67:0/3268409070 >> 172.21.15.67:0/3268409070 conn(0x7ff1cc067ce0 msgr2=0x7ff1cc06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.676+0000 7ff1d2a00640 1 -- 172.21.15.67:0/3268409070 shutdown_connections 2024-05-30T16:56:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.677+0000 7ff1d2a00640 1 -- 172.21.15.67:0/3268409070 wait complete. 2024-05-30T16:56:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.683+0000 7ff1d2a00640 1 Processor -- start 2024-05-30T16:56:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.683+0000 7ff1d2a00640 1 -- start start 2024-05-30T16:56:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.686+0000 7ff1d2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b00037f0 0x7ff1cc2159c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.686+0000 7ff1d2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1cc1557c0 0x7ff1cc14f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.686+0000 7ff1d2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc1535b0 0x7ff1cc1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.687+0000 7ff1d2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc06f500 con 0x7ff1cc1557c0 2024-05-30T16:56:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.687+0000 7ff1d2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc0b44e0 con 0x7ff1cc1535b0 2024-05-30T16:56:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.687+0000 7ff1d2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc0b4a00 con 0x7ff1b00037f0 2024-05-30T16:56:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.687+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b00037f0 0x7ff1cc2159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.687+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b00037f0 0x7ff1cc2159c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:46968/0 (socket says 172.21.15.67:46968) 2024-05-30T16:56:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.687+0000 7ff1cbe00640 1 -- 172.21.15.67:0/2294455457 learned_addr learned my addr 172.21.15.67:0/2294455457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.688+0000 7ff1d0a00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc1535b0 0x7ff1cc1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.689+0000 7ff1cbe00640 1 -- 172.21.15.67:0/2294455457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc1535b0 msgr2=0x7ff1cc1575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.689+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc1535b0 0x7ff1cc1575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.689+0000 7ff1cbe00640 1 -- 172.21.15.67:0/2294455457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1cc1557c0 msgr2=0x7ff1cc14f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.689+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1cc1557c0 0x7ff1cc14f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.692+0000 7ff1cbe00640 1 -- 172.21.15.67:0/2294455457 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc215fa0 con 0x7ff1b00037f0 2024-05-30T16:56:05.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.694+0000 7ff1cbe00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b00037f0 0x7ff1cc2159c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc02ad80 tx=0x7ff1cc0b3b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:05.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.755+0000 7ff1c3e00640 1 -- 172.21.15.67:0/2294455457 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc037040 con 0x7ff1b00037f0 2024-05-30T16:56:05.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.756+0000 7ff1c3e00640 1 -- 172.21.15.67:0/2294455457 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc041070 con 0x7ff1b00037f0 2024-05-30T16:56:05.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.756+0000 7ff1c3e00640 1 -- 172.21.15.67:0/2294455457 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc004a20 con 0x7ff1b00037f0 2024-05-30T16:56:05.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.759+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc216280 con 0x7ff1b00037f0 2024-05-30T16:56:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.759+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc216840 con 0x7ff1b00037f0 2024-05-30T16:56:05.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.765+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc0689e0 con 0x7ff1b00037f0 2024-05-30T16:56:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.766+0000 7ff1c3e00640 1 -- 172.21.15.67:0/2294455457 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7ff1bc004bc0 con 0x7ff1b00037f0 2024-05-30T16:56:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.769+0000 7ff1c3e00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff1ac06da00 0x7ff1ac06fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.769+0000 7ff1c3e00640 1 -- 172.21.15.67:0/2294455457 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7ff1bc0a9ff0 con 0x7ff1b00037f0 2024-05-30T16:56:05.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.775+0000 7ff1c3e00640 1 -- 172.21.15.67:0/2294455457 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1bc001630 con 0x7ff1b00037f0 2024-05-30T16:56:05.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.871+0000 7ff1cb400640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff1ac06da00 0x7ff1ac06fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:05.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:05.889+0000 7ff1cb400640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff1ac06da00 0x7ff1ac06fec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc1547c0 tx=0x7ff1b8006b10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.517+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff1cc0cc360 con 0x7ff1b00037f0 2024-05-30T16:56:06.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.520+0000 7ff1c3e00640 1 -- 172.21.15.67:0/2294455457 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2136 (secure 0 0 0) 0x7ff1bc079e70 con 0x7ff1b00037f0 2024-05-30T16:56:06.522 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:56:06.522 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:56:06.523 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:56:06.524 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:56:06.524 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:56:06.524 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:06.524 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 759, 2024-05-30T16:56:06.524 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:56:06.524 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:56:06.524 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:56:06.524 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:56:06.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:56:06.525 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 970, 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 3926805137, 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 7916879872, 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 765177233408, 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 66733445, 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 92602315, 2024-05-30T16:56:06.526 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 16, 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 26 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:56:06.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:56:06.528 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 16.406% (4s)\n [====........................] (remaining: 20s)", 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.16405999660491943, 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:06.529 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:56:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.525+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff1ac06da00 msgr2=0x7ff1ac06fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.525+0000 7ff1d2a00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff1ac06da00 0x7ff1ac06fec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc1547c0 tx=0x7ff1b8006b10 comp rx=0 tx=0).stop 2024-05-30T16:56:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.525+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b00037f0 msgr2=0x7ff1cc2159c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.525+0000 7ff1d2a00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b00037f0 0x7ff1cc2159c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc02ad80 tx=0x7ff1cc0b3b30 comp rx=0 tx=0).stop 2024-05-30T16:56:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.526+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 shutdown_connections 2024-05-30T16:56:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.526+0000 7ff1d2a00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff1ac06da00 0x7ff1ac06fec0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.526+0000 7ff1d2a00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc1535b0 0x7ff1cc1575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.526+0000 7ff1d2a00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1cc1557c0 0x7ff1cc14f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.526+0000 7ff1d2a00640 1 --2- 172.21.15.67:0/2294455457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1b00037f0 0x7ff1cc2159c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.526+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 >> 172.21.15.67:0/2294455457 conn(0x7ff1cc067ce0 msgr2=0x7ff1cc159a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.527+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 shutdown_connections 2024-05-30T16:56:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:06.533+0000 7ff1d2a00640 1 -- 172.21.15.67:0/2294455457 wait complete. 2024-05-30T16:56:07.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:56:08.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.022+0000 7f7653e00640 1 Processor -- start 2024-05-30T16:56:08.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.023+0000 7f7653e00640 1 -- start start 2024-05-30T16:56:08.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.025+0000 7f7653e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f764c071cc0 0x7f764c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:08.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.026+0000 7f7653e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c0b3a10 con 0x7f764c15b850 2024-05-30T16:56:08.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.026+0000 7f7653e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764c0b3cf0 con 0x7f764c06e620 2024-05-30T16:56:08.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.026+0000 7f7653e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f764c06e190 con 0x7f764c071cc0 2024-05-30T16:56:08.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.027+0000 7f7651e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f764c06e620 0x7f764c06ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:58596/0 (socket says 172.21.15.67:58596) 2024-05-30T16:56:08.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.027+0000 7f7651400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f764c071cc0 0x7f764c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:08.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.027+0000 7f7651e00640 1 -- 172.21.15.67:0/612510619 learned_addr learned my addr 172.21.15.67:0/612510619 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:08.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.028+0000 7f7651400640 1 -- 172.21.15.67:0/612510619 >> v1:172.21.15.169:6789/0 conn(0x7f764c06e620 legacy=0x7f764c06ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:08.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.028+0000 7f764be00640 1 -- 172.21.15.67:0/612510619 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91816729 0 0) 0x7f764c0b3cf0 con 0x7f764c06e620 2024-05-30T16:56:08.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.028+0000 7f7651400640 1 -- 172.21.15.67:0/612510619 >> v1:172.21.15.67:6789/0 conn(0x7f764c15b850 legacy=0x7f764c0720a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:56:08.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.028+0000 7f7651400640 1 -- 172.21.15.67:0/612510619 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c0b3fd0 con 0x7f764c071cc0 2024-05-30T16:56:08.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.031+0000 7f7651400640 1 --2- 172.21.15.67:0/612510619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c071cc0 0x7f764c0b3380 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7638007430 tx=0x7f763802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfc9a792e129d19 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:08.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.033+0000 7f764be00640 1 -- 172.21.15.67:0/612510619 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763802ae40 con 0x7f764c071cc0 2024-05-30T16:56:08.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.034+0000 7f764be00640 1 -- 172.21.15.67:0/612510619 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76380089e0 con 0x7f764c071cc0 2024-05-30T16:56:08.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.034+0000 7f764be00640 1 -- 172.21.15.67:0/612510619 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638008d90 con 0x7f764c071cc0 2024-05-30T16:56:08.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.036+0000 7f7653e00640 1 -- 172.21.15.67:0/612510619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c071cc0 msgr2=0x7f764c0b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:08.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.037+0000 7f7653e00640 1 --2- 172.21.15.67:0/612510619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c071cc0 0x7f764c0b3380 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7638007430 tx=0x7f763802a340 comp rx=0 tx=0).stop 2024-05-30T16:56:08.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.038+0000 7f7653e00640 1 -- 172.21.15.67:0/612510619 shutdown_connections 2024-05-30T16:56:08.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.038+0000 7f7653e00640 1 --2- 172.21.15.67:0/612510619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c071cc0 0x7f764c0b3380 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:08.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.039+0000 7f7653e00640 1 -- 172.21.15.67:0/612510619 >> 172.21.15.67:0/612510619 conn(0x7f764c06bae0 msgr2=0x7f764c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:08.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.041+0000 7f7653e00640 1 -- 172.21.15.67:0/612510619 shutdown_connections 2024-05-30T16:56:08.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.041+0000 7f7653e00640 1 -- 172.21.15.67:0/612510619 wait complete. 2024-05-30T16:56:08.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.045+0000 7f7653e00640 1 Processor -- start 2024-05-30T16:56:08.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.045+0000 7f7653e00640 1 -- start start 2024-05-30T16:56:08.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.047+0000 7f7653e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764c06e620 0x7f764c1efe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:08.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.048+0000 7f7653e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764c071cc0 0x7f764c0a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:08.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.048+0000 7f7651400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764c06e620 0x7f764c1efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:08.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.048+0000 7f7651400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764c06e620 0x7f764c1efe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:46970/0 (socket says 172.21.15.67:46970) 2024-05-30T16:56:08.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.048+0000 7f7651400640 1 -- 172.21.15.67:0/1754335972 learned_addr learned my addr 172.21.15.67:0/1754335972 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:08.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.049+0000 7f7653e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c15b850 0x7f764c150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:08.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.049+0000 7f7651e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c15b850 0x7f764c150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:08.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.049+0000 7f7650a00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764c071cc0 0x7f764c0a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:08.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.050+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1550d0 con 0x7f764c15b850 2024-05-30T16:56:08.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.050+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f764c15bcd0 con 0x7f764c071cc0 2024-05-30T16:56:08.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.050+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f764c155250 con 0x7f764c06e620 2024-05-30T16:56:08.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.052+0000 7f7651e00640 1 -- 172.21.15.67:0/1754335972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764c06e620 msgr2=0x7f764c1efe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:08.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.052+0000 7f7651e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764c06e620 0x7f764c1efe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:08.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.052+0000 7f7651e00640 1 -- 172.21.15.67:0/1754335972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764c071cc0 msgr2=0x7f764c0a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:08.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.052+0000 7f7651e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764c071cc0 0x7f764c0a9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:08.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.052+0000 7f7651e00640 1 -- 172.21.15.67:0/1754335972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c1585c0 con 0x7f764c15b850 2024-05-30T16:56:08.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.052+0000 7f7651e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c15b850 0x7f764c150b20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f764400c6d0 tx=0x7f764400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:08.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.054+0000 7f7649600640 1 -- 172.21.15.67:0/1754335972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644018070 con 0x7f764c15b850 2024-05-30T16:56:08.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.054+0000 7f7649600640 1 -- 172.21.15.67:0/1754335972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7644004d60 con 0x7f764c15b850 2024-05-30T16:56:08.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.054+0000 7f7649600640 1 -- 172.21.15.67:0/1754335972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644018070 con 0x7f764c15b850 2024-05-30T16:56:08.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.055+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c1588a0 con 0x7f764c15b850 2024-05-30T16:56:08.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.055+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c159140 con 0x7f764c15b850 2024-05-30T16:56:08.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.058+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764c159340 con 0x7f764c15b850 2024-05-30T16:56:08.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.062+0000 7f7649600640 1 -- 172.21.15.67:0/1754335972 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f764400a5e0 con 0x7f764c15b850 2024-05-30T16:56:08.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.063+0000 7f7649600640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762c06dbd0 0x7f762c070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:08.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.064+0000 7f7649600640 1 -- 172.21.15.67:0/1754335972 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f7644089780 con 0x7f764c15b850 2024-05-30T16:56:08.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.065+0000 7f7651400640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762c06dbd0 0x7f762c070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:08.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.069+0000 7f7651400640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762c06dbd0 0x7f762c070090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f763802a380 tx=0x7f763802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:08.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.072+0000 7f7649600640 1 -- 172.21.15.67:0/1754335972 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f764c159340 con 0x7f764c15b850 2024-05-30T16:56:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.763+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f764c0689e0 con 0x7f764c15b850 2024-05-30T16:56:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.766+0000 7f7649600640 1 -- 172.21.15.67:0/1754335972 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2140 (secure 0 0 0) 0x7f7644058fb0 con 0x7f764c15b850 2024-05-30T16:56:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:56:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 762, 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:56:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1504, 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 6158463131, 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 12460113920, 2024-05-30T16:56:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 760633999360, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 257845850, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 269415630, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 63, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 76 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:56:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:56:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 16.406% (4s)\n [====........................] (remaining: 20s)", 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.16405999660491943, 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:08.773 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:56:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.772+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762c06dbd0 msgr2=0x7f762c070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.772+0000 7f7653e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762c06dbd0 0x7f762c070090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f763802a380 tx=0x7f763802e000 comp rx=0 tx=0).stop 2024-05-30T16:56:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.772+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c15b850 msgr2=0x7f764c150b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.772+0000 7f7653e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c15b850 0x7f764c150b20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f764400c6d0 tx=0x7f764400cb90 comp rx=0 tx=0).stop 2024-05-30T16:56:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.775+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 shutdown_connections 2024-05-30T16:56:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.775+0000 7f7653e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762c06dbd0 0x7f762c070090 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.775+0000 7f7653e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764c15b850 0x7f764c150b20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.775+0000 7f7653e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764c071cc0 0x7f764c0a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.775+0000 7f7653e00640 1 --2- 172.21.15.67:0/1754335972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764c06e620 0x7f764c1efe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.776+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 >> 172.21.15.67:0/1754335972 conn(0x7f764c06bae0 msgr2=0x7f764c151fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.780+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 shutdown_connections 2024-05-30T16:56:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:08.780+0000 7f7653e00640 1 -- 172.21.15.67:0/1754335972 wait complete. 2024-05-30T16:56:09.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:56:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.235+0000 7f6201e00640 1 Processor -- start 2024-05-30T16:56:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.235+0000 7f6201e00640 1 -- start start 2024-05-30T16:56:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.241+0000 7f6201e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc071ac0 0x7f61fc071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.241+0000 7f6201e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61fc06dfb0 con 0x7f61fc06db20 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.241+0000 7f6201e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61fc06e290 con 0x7f61fc163480 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.241+0000 7f6201e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f61fc072470 con 0x7f61fc071ac0 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.242+0000 7f6200a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f61fc163480 0x7f61fc163860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:58608/0 (socket says 172.21.15.67:58608) 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.242+0000 7f6200a00640 1 -- 172.21.15.67:0/434045808 learned_addr learned my addr 172.21.15.67:0/434045808 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.243+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3637132891 0 0) 0x7f61fc06e290 con 0x7f61fc163480 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.243+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61e0003520 con 0x7f61fc163480 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.244+0000 7f6201400640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc071ac0 0x7f61fc071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.244+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2045563362 0 0) 0x7f61e0003520 con 0x7f61fc163480 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.244+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc071ac0 msgr2=0x7f61fc071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.244+0000 7f61fb400640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc071ac0 0x7f61fc071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.244+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 >> v1:172.21.15.67:6789/0 conn(0x7f61fc06db20 legacy=0x7f61fc06f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.244+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc06e570 con 0x7f61fc163480 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.244+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f61ec001670 con 0x7f61fc163480 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.244+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 >> v1:172.21.15.169:6789/0 conn(0x7f61fc163480 legacy=0x7f61fc163860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.244+0000 7f61fb400640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61e00039b0 0x7f61e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.245+0000 7f61fb400640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0004800 0x7f61e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.245+0000 7f61fb400640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0005210 0x7f61e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.245+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc072470 con 0x7f61e00039b0 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.245+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61e000c140 con 0x7f61e0005210 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.245+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f61e000c340 con 0x7f61e0004800 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.245+0000 7f6201400640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc071ac0 0x7f61fc071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.246+0000 7f6200a00640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61e00039b0 0x7f61e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.246+0000 7f61fbe00640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0004800 0x7f61e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.247+0000 7f6201400640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0005210 0x7f61e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.247+0000 7f6200a00640 1 -- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0004800 msgr2=0x7f61e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:10.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.247+0000 7f6200a00640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61e0004800 0x7f61e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:10.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.247+0000 7f6200a00640 1 -- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0005210 msgr2=0x7f61e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:10.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.247+0000 7f6200a00640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e0005210 0x7f61e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:10.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.247+0000 7f6200a00640 1 -- 172.21.15.67:0/434045808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc06e570 con 0x7f61e00039b0 2024-05-30T16:56:10.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.251+0000 7f6200a00640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61e00039b0 0x7f61e0003dd0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f61ec001eb0 tx=0x7f61e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bc28e28f50660e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.252+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec032020 con 0x7f61e00039b0 2024-05-30T16:56:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.252+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61ec005ad0 con 0x7f61e00039b0 2024-05-30T16:56:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.252+0000 7f61fb400640 1 -- 172.21.15.67:0/434045808 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec007d10 con 0x7f61e00039b0 2024-05-30T16:56:10.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.254+0000 7f6201e00640 1 -- 172.21.15.67:0/434045808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61e00039b0 msgr2=0x7f61e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:10.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.254+0000 7f6201e00640 1 --2- 172.21.15.67:0/434045808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61e00039b0 0x7f61e0003dd0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f61ec001eb0 tx=0x7f61e0005830 comp rx=0 tx=0).stop 2024-05-30T16:56:10.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.254+0000 7f6201400640 1 -- 172.21.15.67:0/434045808 reap_dead start 2024-05-30T16:56:10.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.255+0000 7f6201e00640 1 -- 172.21.15.67:0/434045808 shutdown_connections 2024-05-30T16:56:10.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.255+0000 7f6201e00640 1 -- 172.21.15.67:0/434045808 >> 172.21.15.67:0/434045808 conn(0x7f61fc0b1db0 msgr2=0x7f61fc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:10.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.259+0000 7f6201e00640 1 -- 172.21.15.67:0/434045808 shutdown_connections 2024-05-30T16:56:10.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.260+0000 7f6201e00640 1 -- 172.21.15.67:0/434045808 wait complete. 2024-05-30T16:56:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.269+0000 7f6201e00640 1 Processor -- start 2024-05-30T16:56:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.270+0000 7f6201e00640 1 -- start start 2024-05-30T16:56:10.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.272+0000 7f6201e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e00039b0 0x7f61fc14e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:10.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.273+0000 7f6201e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61fc151b50 0x7f61fc151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.273+0000 7f6201e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc152750 0x7f61fc152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.273+0000 7f6201e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc071f30 con 0x7f61fc151b50 2024-05-30T16:56:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.273+0000 7f6201e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc073eb0 con 0x7f61e00039b0 2024-05-30T16:56:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.273+0000 7f6201e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f61fc0720b0 con 0x7f61fc152750 2024-05-30T16:56:10.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.276+0000 7f6200a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e00039b0 0x7f61fc14e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:10.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.277+0000 7f6201400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc152750 0x7f61fc152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:10.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.277+0000 7f6201400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc152750 0x7f61fc152bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:46996/0 (socket says 172.21.15.67:46996) 2024-05-30T16:56:10.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.277+0000 7f6201400640 1 -- 172.21.15.67:0/2769362677 learned_addr learned my addr 172.21.15.67:0/2769362677 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:10.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.278+0000 7f61fbe00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61fc151b50 0x7f61fc151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.280+0000 7f61fbe00640 1 -- 172.21.15.67:0/2769362677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc152750 msgr2=0x7f61fc152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:10.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.280+0000 7f61fbe00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc152750 0x7f61fc152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:10.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.280+0000 7f61fbe00640 1 -- 172.21.15.67:0/2769362677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e00039b0 msgr2=0x7f61fc14e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:10.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.282+0000 7f61fbe00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e00039b0 0x7f61fc14e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:10.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.282+0000 7f61fbe00640 1 -- 172.21.15.67:0/2769362677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc1524f0 con 0x7f61fc151b50 2024-05-30T16:56:10.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.282+0000 7f6200a00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e00039b0 0x7f61fc14e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:10.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.282+0000 7f61fbe00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61fc151b50 0x7f61fc151fb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0b2e40 tx=0x7f61f400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:10.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.284+0000 7f61f8c00640 1 -- 172.21.15.67:0/2769362677 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f400ee60 con 0x7f61fc151b50 2024-05-30T16:56:10.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.285+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61fc1534a0 con 0x7f61fc151b50 2024-05-30T16:56:10.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.284+0000 7f61f8c00640 1 -- 172.21.15.67:0/2769362677 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f4018070 con 0x7f61fc151b50 2024-05-30T16:56:10.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.285+0000 7f61f8c00640 1 -- 172.21.15.67:0/2769362677 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4003dc0 con 0x7f61fc151b50 2024-05-30T16:56:10.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.285+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61fc14f3b0 con 0x7f61fc151b50 2024-05-30T16:56:10.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.287+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d4005290 con 0x7f61fc151b50 2024-05-30T16:56:10.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.299+0000 7f61f8c00640 1 -- 172.21.15.67:0/2769362677 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f61f40050d0 con 0x7f61fc151b50 2024-05-30T16:56:10.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.305+0000 7f61f8c00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f61dc06da20 0x7f61dc06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:10.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.306+0000 7f61f8c00640 1 -- 172.21.15.67:0/2769362677 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f61f408b120 con 0x7f61fc151b50 2024-05-30T16:56:10.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.309+0000 7f6200a00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f61dc06da20 0x7f61dc06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:10.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.311+0000 7f61f8c00640 1 -- 172.21.15.67:0/2769362677 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61f405afe0 con 0x7f61fc151b50 2024-05-30T16:56:10.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:10.311+0000 7f6200a00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f61dc06da20 0x7f61dc06fee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f61ec033000 tx=0x7f61ec02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:11.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.004+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f61d4005850 con 0x7f61fc151b50 2024-05-30T16:56:11.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.006+0000 7f61f8c00640 1 -- 172.21.15.67:0/2769362677 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2417 (secure 0 0 0) 0x7f61f405fe20 con 0x7f61fc151b50 2024-05-30T16:56:11.011 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:11.012 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 764, 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:56:11.013 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 2313, 2024-05-30T16:56:11.014 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 9564237519, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 18811998208, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 754282115072, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 530978903, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 531018731, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 128, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 140 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:56:11.015 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:56:11.016 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "7c6f9121-d563-4406-bd90-584b3290d1b2": { 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 35.781% (3s)\n [==========..................] (remaining: 5s)", 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.35780999064445496, 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:56:11.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:11.018 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:56:11.018 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 36.394% (9s)\n [==========..................] (remaining: 16s)", 2024-05-30T16:56:11.018 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.36394000053405762, 2024-05-30T16:56:11.018 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:56:11.018 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:11.018 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:11.018 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:56:11.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.019+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f61dc06da20 msgr2=0x7f61dc06fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:11.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.019+0000 7f6201e00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f61dc06da20 0x7f61dc06fee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f61ec033000 tx=0x7f61ec02e000 comp rx=0 tx=0).stop 2024-05-30T16:56:11.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.019+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61fc151b50 msgr2=0x7f61fc151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:11.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.019+0000 7f6201e00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61fc151b50 0x7f61fc151fb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0b2e40 tx=0x7f61f400ec10 comp rx=0 tx=0).stop 2024-05-30T16:56:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.020+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 shutdown_connections 2024-05-30T16:56:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.020+0000 7f6201e00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f61dc06da20 0x7f61dc06fee0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.020+0000 7f6201e00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61fc152750 0x7f61fc152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.020+0000 7f6201e00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61fc151b50 0x7f61fc151fb0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.020+0000 7f6201e00640 1 --2- 172.21.15.67:0/2769362677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61e00039b0 0x7f61fc14e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:11.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.020+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 >> 172.21.15.67:0/2769362677 conn(0x7f61fc0b1db0 msgr2=0x7f61fc15c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:11.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.022+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 shutdown_connections 2024-05-30T16:56:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.025+0000 7f6201e00640 1 -- 172.21.15.67:0/2769362677 wait complete. 2024-05-30T16:56:11.082 INFO:tasks.cephfs.test_volumes:pevs - {'7c6f9121-d563-4406-bd90-584b3290d1b2': {'message': '5 ongoing+pending clones; Avg progress is 35.781% (3s)\n [==========..................] (remaining: 5s)', 'progress': 0.35780999064445496, 'add_to_ceph_s': True}, 'e0a834ca-3f6a-4236-843d-35f9b733ad76': {'message': '4 ongoing clones; Avg progress is 36.394% (9s)\n [==========..................] (remaining: 16s)', 'progress': 0.3639400005340576, 'add_to_ceph_s': True}} 2024-05-30T16:56:11.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92286 2024-05-30T16:56:11.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.525+0000 7ff250e00640 1 Processor -- start 2024-05-30T16:56:11.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.526+0000 7ff250e00640 1 -- start start 2024-05-30T16:56:11.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.528+0000 7ff250e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff24c0b4e80 0x7ff24c162d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:11.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.528+0000 7ff250e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff24c0b5310 con 0x7ff24c0b3670 2024-05-30T16:56:11.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.528+0000 7ff250e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff24c0d2d30 con 0x7ff24c0b3a50 2024-05-30T16:56:11.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.528+0000 7ff250e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff24c162450 con 0x7ff24c0b4e80 2024-05-30T16:56:11.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.529+0000 7ff249e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff24c0b3670 0x7ff24c162650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36022/0 (socket says 172.21.15.67:36022) 2024-05-30T16:56:11.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.529+0000 7ff249e00640 1 -- 172.21.15.67:0/4048811593 learned_addr learned my addr 172.21.15.67:0/4048811593 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:11.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.529+0000 7ff243e00640 1 --2- 172.21.15.67:0/4048811593 >> v2:172.21.15.67:3300/0 conn(0x7ff24c0b4e80 0x7ff24c162d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:11.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.530+0000 7ff249400640 1 -- 172.21.15.67:0/4048811593 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257168128 0 0) 0x7ff24c0b5310 con 0x7ff24c0b3670 2024-05-30T16:56:11.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.530+0000 7ff249400640 1 -- 172.21.15.67:0/4048811593 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff230003520 con 0x7ff24c0b3670 2024-05-30T16:56:11.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.531+0000 7ff243e00640 1 -- 172.21.15.67:0/4048811593 >> v1:172.21.15.169:6789/0 conn(0x7ff24c0b3a50 legacy=0x7ff24c160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:11.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.531+0000 7ff243e00640 1 -- 172.21.15.67:0/4048811593 >> v1:172.21.15.67:6789/0 conn(0x7ff24c0b3670 legacy=0x7ff24c162650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:11.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.531+0000 7ff243e00640 1 -- 172.21.15.67:0/4048811593 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c0d3010 con 0x7ff24c0b4e80 2024-05-30T16:56:11.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.532+0000 7ff243e00640 1 --2- 172.21.15.67:0/4048811593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b4e80 0x7ff24c162d60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009a00 tx=0x7ff23c02a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a628e3e5f4734a1d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.532+0000 7ff249400640 1 -- 172.21.15.67:0/4048811593 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c037070 con 0x7ff24c0b4e80 2024-05-30T16:56:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.532+0000 7ff249400640 1 -- 172.21.15.67:0/4048811593 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff23c032440 con 0x7ff24c0b4e80 2024-05-30T16:56:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.532+0000 7ff249400640 1 -- 172.21.15.67:0/4048811593 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c032850 con 0x7ff24c0b4e80 2024-05-30T16:56:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.533+0000 7ff250e00640 1 -- 172.21.15.67:0/4048811593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b4e80 msgr2=0x7ff24c162d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.533+0000 7ff250e00640 1 --2- 172.21.15.67:0/4048811593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b4e80 0x7ff24c162d60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009a00 tx=0x7ff23c02a3c0 comp rx=0 tx=0).stop 2024-05-30T16:56:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.534+0000 7ff250e00640 1 -- 172.21.15.67:0/4048811593 shutdown_connections 2024-05-30T16:56:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.534+0000 7ff250e00640 1 --2- 172.21.15.67:0/4048811593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b4e80 0x7ff24c162d60 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.534+0000 7ff250e00640 1 -- 172.21.15.67:0/4048811593 >> 172.21.15.67:0/4048811593 conn(0x7ff24c08bfc0 msgr2=0x7ff24c0d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:11.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.534+0000 7ff250e00640 1 -- 172.21.15.67:0/4048811593 shutdown_connections 2024-05-30T16:56:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.534+0000 7ff250e00640 1 -- 172.21.15.67:0/4048811593 wait complete. 2024-05-30T16:56:11.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.537+0000 7ff250e00640 1 Processor -- start 2024-05-30T16:56:11.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.538+0000 7ff250e00640 1 -- start start 2024-05-30T16:56:11.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.540+0000 7ff250e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b3670 0x7ff24c0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:11.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.540+0000 7ff250e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0b3a50 0x7ff24c1f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:11.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.540+0000 7ff250e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0b7980 0x7ff24c1f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:11.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.540+0000 7ff250e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1632a0 con 0x7ff24c0b3670 2024-05-30T16:56:11.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.540+0000 7ff250e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c0b56e0 con 0x7ff24c0b3a50 2024-05-30T16:56:11.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.540+0000 7ff250e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff24c163420 con 0x7ff24c0b7980 2024-05-30T16:56:11.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.541+0000 7ff249e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b3670 0x7ff24c0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:11.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.541+0000 7ff249e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b3670 0x7ff24c0c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58590/0 (socket says 172.21.15.67:58590) 2024-05-30T16:56:11.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.541+0000 7ff249e00640 1 -- 172.21.15.67:0/1459639306 learned_addr learned my addr 172.21.15.67:0/1459639306 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:11.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.541+0000 7ff24a800640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0b7980 0x7ff24c1f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:11.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.542+0000 7ff249e00640 1 -- 172.21.15.67:0/1459639306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0b7980 msgr2=0x7ff24c1f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:11.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.542+0000 7ff243e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0b3a50 0x7ff24c1f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:11.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.542+0000 7ff249e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0b7980 0x7ff24c1f78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:11.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.542+0000 7ff249e00640 1 -- 172.21.15.67:0/1459639306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0b3a50 msgr2=0x7ff24c1f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:11.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.542+0000 7ff249e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0b3a50 0x7ff24c1f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:11.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.542+0000 7ff249e00640 1 -- 172.21.15.67:0/1459639306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c1f5090 con 0x7ff24c0b3670 2024-05-30T16:56:11.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.542+0000 7ff243e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0b3a50 0x7ff24c1f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:11.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.543+0000 7ff249e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b3670 0x7ff24c0c8bd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff23400c450 tx=0x7ff23400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.544+0000 7ff242000640 1 -- 172.21.15.67:0/1459639306 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234017070 con 0x7ff24c0b3670 2024-05-30T16:56:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.545+0000 7ff242000640 1 -- 172.21.15.67:0/1459639306 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff234005ce0 con 0x7ff24c0b3670 2024-05-30T16:56:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.545+0000 7ff242000640 1 -- 172.21.15.67:0/1459639306 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234017070 con 0x7ff24c0b3670 2024-05-30T16:56:11.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.545+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c0c92d0 con 0x7ff24c0b3670 2024-05-30T16:56:11.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.546+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c0c9890 con 0x7ff24c0b3670 2024-05-30T16:56:11.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.553+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24c162fb0 con 0x7ff24c0b3670 2024-05-30T16:56:11.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.551+0000 7ff242000640 1 -- 172.21.15.67:0/1459639306 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7ff23401a070 con 0x7ff24c0b3670 2024-05-30T16:56:11.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.556+0000 7ff242000640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff22806dbd0 0x7ff228070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.563+0000 7ff242000640 1 -- 172.21.15.67:0/1459639306 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7ff2340898f0 con 0x7ff24c0b3670 2024-05-30T16:56:11.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.578+0000 7ff242000640 1 -- 172.21.15.67:0/1459639306 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff24c162fb0 con 0x7ff24c0b3670 2024-05-30T16:56:11.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.581+0000 7ff243e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff22806dbd0 0x7ff228070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:11.584+0000 7ff243e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff22806dbd0 0x7ff228070090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff23c034000 tx=0x7ff23c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:12.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.167+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92286", "target": ["mon-mgr", ""]}) -- 0x7ff24c0689e0 con 0x7ff22806dbd0 2024-05-30T16:56:12.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:56:12.784 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:56:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.847+0000 7ff242000640 1 -- 172.21.15.67:0/1459639306 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff24c0689e0 con 0x7ff22806dbd0 2024-05-30T16:56:12.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff22806dbd0 msgr2=0x7ff228070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:12.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff22806dbd0 0x7ff228070090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff23c034000 tx=0x7ff23c02e000 comp rx=0 tx=0).stop 2024-05-30T16:56:12.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b3670 msgr2=0x7ff24c0c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:12.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b3670 0x7ff24c0c8bd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff23400c450 tx=0x7ff23400c910 comp rx=0 tx=0).stop 2024-05-30T16:56:12.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 shutdown_connections 2024-05-30T16:56:12.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff22806dbd0 0x7ff228070090 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:12.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0b7980 0x7ff24c1f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:12.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0b3a50 0x7ff24c1f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:12.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 --2- 172.21.15.67:0/1459639306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff24c0b3670 0x7ff24c0c8bd0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:12.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.854+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 >> 172.21.15.67:0/1459639306 conn(0x7ff24c08bfc0 msgr2=0x7ff24c0b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:12.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.855+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 shutdown_connections 2024-05-30T16:56:12.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:12.855+0000 7ff250e00640 1 -- 172.21.15.67:0/1459639306 wait complete. 2024-05-30T16:56:12.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92287 2024-05-30T16:56:13.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.090+0000 7f302c800640 1 Processor -- start 2024-05-30T16:56:13.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.090+0000 7f302c800640 1 -- start start 2024-05-30T16:56:13.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.094+0000 7f302c800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3024161ed0 0x7f30240676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:13.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.094+0000 7f302c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30240b0470 0x7f3024067bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:13.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.094+0000 7f302c800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30240b0ea0 con 0x7f30240b4f30 2024-05-30T16:56:13.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.094+0000 7f302c800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f30240b2650 con 0x7f3024161ed0 2024-05-30T16:56:13.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.094+0000 7f302c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30240b2ed0 con 0x7f30240b0470 2024-05-30T16:56:13.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.094+0000 7f3029e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3024161ed0 0x7f30240676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:13.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.094+0000 7f3029e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3024161ed0 0x7f30240676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58604/0 (socket says 172.21.15.67:58604) 2024-05-30T16:56:13.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.094+0000 7f3029e00640 1 -- 172.21.15.67:0/325539190 learned_addr learned my addr 172.21.15.67:0/325539190 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.095+0000 7f3029e00640 1 -- 172.21.15.67:0/325539190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30240b0470 msgr2=0x7f3024067bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.095+0000 7f3029e00640 1 --2- 172.21.15.67:0/325539190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30240b0470 0x7f3024067bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.095+0000 7f3029e00640 1 -- 172.21.15.67:0/325539190 >> v1:172.21.15.169:6789/0 conn(0x7f30240b4f30 legacy=0x7f302406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:13.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.095+0000 7f3029e00640 1 -- 172.21.15.67:0/325539190 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240b2880 con 0x7f3024161ed0 2024-05-30T16:56:13.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.097+0000 7f3029e00640 1 --2- 172.21.15.67:0/325539190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024161ed0 0x7f30240676b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3018009870 tx=0x7f301802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97504d6254d22cba server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:13.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.097+0000 7f3028a00640 1 -- 172.21.15.67:0/325539190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018037070 con 0x7f3024161ed0 2024-05-30T16:56:13.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.098+0000 7f3028a00640 1 -- 172.21.15.67:0/325539190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3018032440 con 0x7f3024161ed0 2024-05-30T16:56:13.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.098+0000 7f3028a00640 1 -- 172.21.15.67:0/325539190 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018032810 con 0x7f3024161ed0 2024-05-30T16:56:13.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.098+0000 7f302c800640 1 -- 172.21.15.67:0/325539190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024161ed0 msgr2=0x7f30240676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:13.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.098+0000 7f302c800640 1 --2- 172.21.15.67:0/325539190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024161ed0 0x7f30240676b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3018009870 tx=0x7f301802a210 comp rx=0 tx=0).stop 2024-05-30T16:56:13.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.099+0000 7f302c800640 1 -- 172.21.15.67:0/325539190 shutdown_connections 2024-05-30T16:56:13.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.099+0000 7f302c800640 1 --2- 172.21.15.67:0/325539190 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30240b0470 0x7f3024067bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:13.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.099+0000 7f302c800640 1 --2- 172.21.15.67:0/325539190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3024161ed0 0x7f30240676b0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:13.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.099+0000 7f302c800640 1 -- 172.21.15.67:0/325539190 >> 172.21.15.67:0/325539190 conn(0x7f30240732b0 msgr2=0x7f302409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.099+0000 7f302c800640 1 -- 172.21.15.67:0/325539190 shutdown_connections 2024-05-30T16:56:13.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.100+0000 7f302c800640 1 -- 172.21.15.67:0/325539190 wait complete. 2024-05-30T16:56:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.105+0000 7f302c800640 1 Processor -- start 2024-05-30T16:56:13.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.105+0000 7f302c800640 1 -- start start 2024-05-30T16:56:13.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.108+0000 7f302c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30240b0470 0x7f302415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:13.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.108+0000 7f302c800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30240b4f30 0x7f302415d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:13.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.108+0000 7f302c800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3024161ed0 0x7f30241f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:13.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.108+0000 7f302c800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f30240b0b50 con 0x7f30240b4f30 2024-05-30T16:56:13.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.108+0000 7f302c800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30240b1180 con 0x7f3024161ed0 2024-05-30T16:56:13.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.108+0000 7f302c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30240af610 con 0x7f30240b0470 2024-05-30T16:56:13.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.108+0000 7f3029400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30240b4f30 0x7f302415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:13.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.108+0000 7f3029400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30240b4f30 0x7f302415d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58610/0 (socket says 172.21.15.67:58610) 2024-05-30T16:56:13.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.108+0000 7f3029400640 1 -- 172.21.15.67:0/2224469864 learned_addr learned my addr 172.21.15.67:0/2224469864 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.109+0000 7f3029400640 1 -- 172.21.15.67:0/2224469864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30240b0470 msgr2=0x7f302415e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.109+0000 7f3029400640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30240b0470 0x7f302415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.109+0000 7f3029400640 1 -- 172.21.15.67:0/2224469864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3024161ed0 msgr2=0x7f30241f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.109+0000 7f3029400640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3024161ed0 0x7f30241f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.109+0000 7f3029400640 1 -- 172.21.15.67:0/2224469864 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302409e7b0 con 0x7f30240b4f30 2024-05-30T16:56:13.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.110+0000 7f3029400640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30240b4f30 0x7f302415d520 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f301000b450 tx=0x7f301000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.116+0000 7f301e000640 1 -- 172.21.15.67:0/2224469864 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010015020 con 0x7f30240b4f30 2024-05-30T16:56:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.116+0000 7f301e000640 1 -- 172.21.15.67:0/2224469864 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3010011890 con 0x7f30240b4f30 2024-05-30T16:56:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.116+0000 7f301e000640 1 -- 172.21.15.67:0/2224469864 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010011c80 con 0x7f30240b4f30 2024-05-30T16:56:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.116+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302409ea90 con 0x7f30240b4f30 2024-05-30T16:56:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.117+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302409f050 con 0x7f30240b4f30 2024-05-30T16:56:13.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.122+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3000005290 con 0x7f30240b4f30 2024-05-30T16:56:13.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.123+0000 7f301e000640 1 -- 172.21.15.67:0/2224469864 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f3010018020 con 0x7f30240b4f30 2024-05-30T16:56:13.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.125+0000 7f301e000640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f300406daa0 0x7f300406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:13.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.125+0000 7f301e000640 1 -- 172.21.15.67:0/2224469864 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f301008a8a0 con 0x7f30240b4f30 2024-05-30T16:56:13.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.126+0000 7f3029e00640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f300406daa0 0x7f300406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:13.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.127+0000 7f3029e00640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f300406daa0 0x7f300406ff60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3018034000 tx=0x7f301802c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:13.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.137+0000 7f301e000640 1 -- 172.21.15.67:0/2224469864 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f301001f020 con 0x7f30240b4f30 2024-05-30T16:56:13.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:13.719+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92287", "target": ["mon-mgr", ""]}) -- 0x7f3000000c80 con 0x7f300406daa0 2024-05-30T16:56:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.431+0000 7f301e000640 1 -- 172.21.15.67:0/2224469864 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3000000c80 con 0x7f300406daa0 2024-05-30T16:56:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.444+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f300406daa0 msgr2=0x7f300406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.444+0000 7f302c800640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f300406daa0 0x7f300406ff60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3018034000 tx=0x7f301802c000 comp rx=0 tx=0).stop 2024-05-30T16:56:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.444+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30240b4f30 msgr2=0x7f302415d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.444+0000 7f302c800640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30240b4f30 0x7f302415d520 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f301000b450 tx=0x7f301000b910 comp rx=0 tx=0).stop 2024-05-30T16:56:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.445+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 shutdown_connections 2024-05-30T16:56:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.445+0000 7f302c800640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f300406daa0 0x7f300406ff60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.445+0000 7f302c800640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3024161ed0 0x7f30241f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.445+0000 7f302c800640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30240b4f30 0x7f302415d520 secure :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f301000b450 tx=0x7f301000b910 comp rx=0 tx=0).stop 2024-05-30T16:56:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.445+0000 7f302c800640 1 --2- 172.21.15.67:0/2224469864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30240b0470 0x7f302415e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.445+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 >> 172.21.15.67:0/2224469864 conn(0x7f30240732b0 msgr2=0x7f30240702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.446+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 shutdown_connections 2024-05-30T16:56:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.448+0000 7f302c800640 1 -- 172.21.15.67:0/2224469864 wait complete. 2024-05-30T16:56:14.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92288 2024-05-30T16:56:14.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.967+0000 7f41a8a00640 1 Processor -- start 2024-05-30T16:56:14.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.967+0000 7f41a8a00640 1 -- start start 2024-05-30T16:56:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.969+0000 7f41a8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f41a40b5140 0x7f41a4071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.969+0000 7f41a8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f41a4072360 0x7f41a406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.969+0000 7f41a8a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a40b3580 con 0x7f41a406fbb0 2024-05-30T16:56:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.969+0000 7f41a8a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f41a4071c80 con 0x7f41a40b5140 2024-05-30T16:56:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.969+0000 7f41a8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f41a40b4470 con 0x7f41a4072360 2024-05-30T16:56:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.970+0000 7f41a1e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f41a40b5140 0x7f41a4071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.970+0000 7f41a1e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f41a40b5140 0x7f41a4071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58616/0 (socket says 172.21.15.67:58616) 2024-05-30T16:56:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.970+0000 7f41a1e00640 1 -- 172.21.15.67:0/1405493596 learned_addr learned my addr 172.21.15.67:0/1405493596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:14.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.971+0000 7f41a3200640 1 --2- 172.21.15.67:0/1405493596 >> v2:172.21.15.169:3300/0 conn(0x7f41a4072360 0x7f41a406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:14.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.972+0000 7f41a1e00640 1 -- 172.21.15.67:0/1405493596 >> v2:172.21.15.169:3300/0 conn(0x7f41a4072360 msgr2=0x7f41a406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:14.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.972+0000 7f41a1e00640 1 --2- 172.21.15.67:0/1405493596 >> v2:172.21.15.169:3300/0 conn(0x7f41a4072360 0x7f41a406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:14.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.972+0000 7f41a1e00640 1 -- 172.21.15.67:0/1405493596 >> v1:172.21.15.67:6789/0 conn(0x7f41a406fbb0 legacy=0x7f41a415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:14.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.972+0000 7f41a1e00640 1 -- 172.21.15.67:0/1405493596 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a40b3860 con 0x7f41a40b5140 2024-05-30T16:56:14.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.972+0000 7f41a3200640 1 --2- 172.21.15.67:0/1405493596 >> v2:172.21.15.169:3300/0 conn(0x7f41a4072360 0x7f41a406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:14.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.975+0000 7f41a1e00640 1 --2- 172.21.15.67:0/1405493596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41a40b5140 0x7f41a4071e20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f41a40cfb60 tx=0x7f419402a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fefe84adf4090c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:14.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.976+0000 7f41a1400640 1 -- 172.21.15.67:0/1405493596 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194037070 con 0x7f41a40b5140 2024-05-30T16:56:14.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.976+0000 7f41a1400640 1 -- 172.21.15.67:0/1405493596 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4194032640 con 0x7f41a40b5140 2024-05-30T16:56:14.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.976+0000 7f41a1400640 1 -- 172.21.15.67:0/1405493596 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194032a50 con 0x7f41a40b5140 2024-05-30T16:56:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.977+0000 7f41a8a00640 1 -- 172.21.15.67:0/1405493596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41a40b5140 msgr2=0x7f41a4071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.977+0000 7f41a8a00640 1 --2- 172.21.15.67:0/1405493596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41a40b5140 0x7f41a4071e20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f41a40cfb60 tx=0x7f419402a330 comp rx=0 tx=0).stop 2024-05-30T16:56:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.978+0000 7f41a8a00640 1 -- 172.21.15.67:0/1405493596 shutdown_connections 2024-05-30T16:56:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.978+0000 7f41a8a00640 1 --2- 172.21.15.67:0/1405493596 >> v2:172.21.15.169:3300/0 conn(0x7f41a4072360 0x7f41a406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.978+0000 7f41a8a00640 1 --2- 172.21.15.67:0/1405493596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41a40b5140 0x7f41a4071e20 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.978+0000 7f41a8a00640 1 -- 172.21.15.67:0/1405493596 >> 172.21.15.67:0/1405493596 conn(0x7f41a40b1db0 msgr2=0x7f41a40b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:14.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.978+0000 7f41a8a00640 1 -- 172.21.15.67:0/1405493596 shutdown_connections 2024-05-30T16:56:14.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.979+0000 7f41a8a00640 1 -- 172.21.15.67:0/1405493596 wait complete. 2024-05-30T16:56:14.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.982+0000 7f41a8a00640 1 Processor -- start 2024-05-30T16:56:14.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.982+0000 7f41a8a00640 1 -- start start 2024-05-30T16:56:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.984+0000 7f41a8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41a406fbb0 0x7f41a4099b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.984+0000 7f41a8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41a4072360 0x7f41a40ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.984+0000 7f41a8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41a415b2a0 0x7f41a4145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.984+0000 7f41a8a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f41a40b4100 con 0x7f41a4072360 2024-05-30T16:56:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.984+0000 7f41a8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f41a40b3e20 con 0x7f41a406fbb0 2024-05-30T16:56:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.984+0000 7f41a8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f41a415acd0 con 0x7f41a415b2a0 2024-05-30T16:56:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.984+0000 7f41a3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41a415b2a0 0x7f41a4145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.985+0000 7f41a3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41a415b2a0 0x7f41a4145020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:37110/0 (socket says 172.21.15.67:37110) 2024-05-30T16:56:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.985+0000 7f41a2800640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41a406fbb0 0x7f41a4099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.985+0000 7f41a1e00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41a4072360 0x7f41a40ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.986+0000 7f41a3200640 1 -- 172.21.15.67:0/294156556 learned_addr learned my addr 172.21.15.67:0/294156556 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.986+0000 7f41a2800640 1 -- 172.21.15.67:0/294156556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41a415b2a0 msgr2=0x7f41a4145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.986+0000 7f41a2800640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41a415b2a0 0x7f41a4145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.986+0000 7f41a2800640 1 -- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41a4072360 msgr2=0x7f41a40ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.986+0000 7f41a2800640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41a4072360 0x7f41a40ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.986+0000 7f41a2800640 1 -- 172.21.15.67:0/294156556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a40cdea0 con 0x7f41a406fbb0 2024-05-30T16:56:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.986+0000 7f41a3200640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41a415b2a0 0x7f41a4145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:56:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.987+0000 7f41a2800640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41a406fbb0 0x7f41a4099b30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f419000c3c0 tx=0x7f419000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.987+0000 7f419aa00640 1 -- 172.21.15.67:0/294156556 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190017070 con 0x7f41a406fbb0 2024-05-30T16:56:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.988+0000 7f419aa00640 1 -- 172.21.15.67:0/294156556 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41900052d0 con 0x7f41a406fbb0 2024-05-30T16:56:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.988+0000 7f419aa00640 1 -- 172.21.15.67:0/294156556 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190003830 con 0x7f41a406fbb0 2024-05-30T16:56:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.988+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a40aec90 con 0x7f41a406fbb0 2024-05-30T16:56:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.988+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a40ceb80 con 0x7f41a406fbb0 2024-05-30T16:56:14.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.991+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a40689e0 con 0x7f41a406fbb0 2024-05-30T16:56:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:14.996+0000 7f419aa00640 1 -- 172.21.15.67:0/294156556 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f419001a070 con 0x7f41a406fbb0 2024-05-30T16:56:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:15.001+0000 7f419aa00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f418406daa0 0x7f418406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:15.002+0000 7f419aa00640 1 -- 172.21.15.67:0/294156556 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f4190089db0 con 0x7f41a406fbb0 2024-05-30T16:56:15.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:15.014+0000 7f419aa00640 1 -- 172.21.15.67:0/294156556 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4190059210 con 0x7f41a406fbb0 2024-05-30T16:56:15.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:15.027+0000 7f41a1e00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f418406daa0 0x7f418406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:15.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:15.032+0000 7f41a1e00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f418406daa0 0x7f418406ff60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4194034000 tx=0x7f419402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:15.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:15.526+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92288", "target": ["mon-mgr", ""]}) -- 0x7f41a414c340 con 0x7f418406daa0 2024-05-30T16:56:16.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.018+0000 7f419aa00640 1 -- 172.21.15.67:0/294156556 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f41a414c340 con 0x7f418406daa0 2024-05-30T16:56:16.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f418406daa0 msgr2=0x7f418406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:16.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f418406daa0 0x7f418406ff60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4194034000 tx=0x7f419402e000 comp rx=0 tx=0).stop 2024-05-30T16:56:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41a406fbb0 msgr2=0x7f41a4099b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41a406fbb0 0x7f41a4099b30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f419000c3c0 tx=0x7f419000c880 comp rx=0 tx=0).stop 2024-05-30T16:56:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 shutdown_connections 2024-05-30T16:56:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f418406daa0 0x7f418406ff60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41a415b2a0 0x7f41a4145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41a4072360 0x7f41a40ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 --2- 172.21.15.67:0/294156556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41a406fbb0 0x7f41a4099b30 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f419000c3c0 tx=0x7f419000c880 comp rx=0 tx=0).stop 2024-05-30T16:56:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.030+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 >> 172.21.15.67:0/294156556 conn(0x7f41a40b1db0 msgr2=0x7f41a415c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.032+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 shutdown_connections 2024-05-30T16:56:16.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.035+0000 7f41a8a00640 1 -- 172.21.15.67:0/294156556 wait complete. 2024-05-30T16:56:16.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92289 2024-05-30T16:56:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.517+0000 7f7e8d000640 1 Processor -- start 2024-05-30T16:56:16.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.518+0000 7f7e8d000640 1 -- start start 2024-05-30T16:56:16.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.521+0000 7f7e8d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7e880d0de0 0x7f7e88072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:16.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.521+0000 7f7e8d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 0x7f7e880ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:16.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.521+0000 7f7e8d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e88072db0 con 0x7f7e88162c20 2024-05-30T16:56:16.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.521+0000 7f7e8d000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7e88072b80 con 0x7f7e880d0de0 2024-05-30T16:56:16.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.521+0000 7f7e8d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e88163000 con 0x7f7e8806e450 2024-05-30T16:56:16.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.524+0000 7f7e85e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7e88162c20 0x7f7e880d06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36052/0 (socket says 172.21.15.67:36052) 2024-05-30T16:56:16.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.524+0000 7f7e85e00640 1 -- 172.21.15.67:0/715666639 learned_addr learned my addr 172.21.15.67:0/715666639 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:16.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.530+0000 7f7e85400640 1 --2- 172.21.15.67:0/715666639 >> v2:172.21.15.67:3300/0 conn(0x7f7e880d0de0 0x7f7e88072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:16.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.531+0000 7f7e86800640 1 --2- 172.21.15.67:0/715666639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 0x7f7e880ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:16.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.535+0000 7f7e85400640 1 -- 172.21.15.67:0/715666639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 msgr2=0x7f7e880ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:16.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.535+0000 7f7e85400640 1 --2- 172.21.15.67:0/715666639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 0x7f7e880ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:16.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.535+0000 7f7e85400640 1 -- 172.21.15.67:0/715666639 >> v1:172.21.15.67:6789/0 conn(0x7f7e88162c20 legacy=0x7f7e880d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:16.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.535+0000 7f7e84a00640 1 -- 172.21.15.67:0/715666639 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4205650694 0 0) 0x7f7e88072db0 con 0x7f7e88162c20 2024-05-30T16:56:16.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.535+0000 7f7e85400640 1 -- 172.21.15.67:0/715666639 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e88073090 con 0x7f7e880d0de0 2024-05-30T16:56:16.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.541+0000 7f7e85400640 1 --2- 172.21.15.67:0/715666639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e880d0de0 0x7f7e88072640 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7e740097a0 tx=0x7f7e7402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267ee631907a8998 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.543+0000 7f7e84a00640 1 -- 172.21.15.67:0/715666639 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74037070 con 0x7f7e880d0de0 2024-05-30T16:56:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.543+0000 7f7e84a00640 1 -- 172.21.15.67:0/715666639 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7402ae50 con 0x7f7e880d0de0 2024-05-30T16:56:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.543+0000 7f7e84a00640 1 -- 172.21.15.67:0/715666639 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74032660 con 0x7f7e880d0de0 2024-05-30T16:56:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.545+0000 7f7e8d000640 1 -- 172.21.15.67:0/715666639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e880d0de0 msgr2=0x7f7e88072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.545+0000 7f7e8d000640 1 --2- 172.21.15.67:0/715666639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e880d0de0 0x7f7e88072640 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7e740097a0 tx=0x7f7e7402a100 comp rx=0 tx=0).stop 2024-05-30T16:56:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.546+0000 7f7e8d000640 1 -- 172.21.15.67:0/715666639 shutdown_connections 2024-05-30T16:56:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.546+0000 7f7e8d000640 1 --2- 172.21.15.67:0/715666639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 0x7f7e880ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.546+0000 7f7e8d000640 1 --2- 172.21.15.67:0/715666639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e880d0de0 0x7f7e88072640 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.546+0000 7f7e8d000640 1 -- 172.21.15.67:0/715666639 >> 172.21.15.67:0/715666639 conn(0x7f7e8806bd20 msgr2=0x7f7e8806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.546+0000 7f7e8d000640 1 -- 172.21.15.67:0/715666639 shutdown_connections 2024-05-30T16:56:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.546+0000 7f7e8d000640 1 -- 172.21.15.67:0/715666639 wait complete. 2024-05-30T16:56:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.550+0000 7f7e8d000640 1 Processor -- start 2024-05-30T16:56:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.550+0000 7f7e8d000640 1 -- start start 2024-05-30T16:56:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.552+0000 7f7e8d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 0x7f7e8815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.552+0000 7f7e8d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88162c20 0x7f7e88155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.552+0000 7f7e8d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e88159cb0 0x7f7e88156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.552+0000 7f7e8d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88073650 con 0x7f7e88159cb0 2024-05-30T16:56:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.552+0000 7f7e8d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e880739c0 con 0x7f7e88162c20 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.552+0000 7f7e8d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e88073e50 con 0x7f7e8806e450 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.553+0000 7f7e86800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e88159cb0 0x7f7e88156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.553+0000 7f7e86800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e88159cb0 0x7f7e88156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58640/0 (socket says 172.21.15.67:58640) 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.553+0000 7f7e86800640 1 -- 172.21.15.67:0/1792334721 learned_addr learned my addr 172.21.15.67:0/1792334721 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.553+0000 7f7e85e00640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 0x7f7e8815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.554+0000 7f7e85400640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88162c20 0x7f7e88155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.554+0000 7f7e86800640 1 -- 172.21.15.67:0/1792334721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 msgr2=0x7f7e8815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.554+0000 7f7e86800640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 0x7f7e8815bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.554+0000 7f7e86800640 1 -- 172.21.15.67:0/1792334721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88162c20 msgr2=0x7f7e88155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.554+0000 7f7e86800640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88162c20 0x7f7e88155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.554+0000 7f7e86800640 1 -- 172.21.15.67:0/1792334721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8815af40 con 0x7f7e88159cb0 2024-05-30T16:56:16.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.555+0000 7f7e86800640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e88159cb0 0x7f7e88156d60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009dd0 tx=0x7f7e8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:16.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.556+0000 7f7e7e000640 1 -- 172.21.15.67:0/1792334721 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80018020 con 0x7f7e88159cb0 2024-05-30T16:56:16.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.556+0000 7f7e7e000640 1 -- 172.21.15.67:0/1792334721 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80014480 con 0x7f7e88159cb0 2024-05-30T16:56:16.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.556+0000 7f7e7e000640 1 -- 172.21.15.67:0/1792334721 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80018020 con 0x7f7e88159cb0 2024-05-30T16:56:16.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.556+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8815b220 con 0x7f7e88159cb0 2024-05-30T16:56:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.561+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8815b7e0 con 0x7f7e88159cb0 2024-05-30T16:56:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.564+0000 7f7e7e000640 1 -- 172.21.15.67:0/1792334721 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f7e8001b020 con 0x7f7e88159cb0 2024-05-30T16:56:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.565+0000 7f7e7e000640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6806de00 0x7f7e680702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.568+0000 7f7e7e000640 1 -- 172.21.15.67:0/1792334721 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f7e8000cd30 con 0x7f7e88159cb0 2024-05-30T16:56:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.568+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e880689e0 con 0x7f7e88159cb0 2024-05-30T16:56:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.570+0000 7f7e85e00640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6806de00 0x7f7e680702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:16.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.584+0000 7f7e7e000640 1 -- 172.21.15.67:0/1792334721 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e80010040 con 0x7f7e88159cb0 2024-05-30T16:56:16.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:16.615+0000 7f7e85e00640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6806de00 0x7f7e680702c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000f60 tx=0x7f7e78009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:17.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.111+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92289", "target": ["mon-mgr", ""]}) -- 0x7f7e880cfd80 con 0x7f7e6806de00 2024-05-30T16:56:17.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.720+0000 7f7e7e000640 1 -- 172.21.15.67:0/1792334721 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e880cfd80 con 0x7f7e6806de00 2024-05-30T16:56:17.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.730+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6806de00 msgr2=0x7f7e680702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:17.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.730+0000 7f7e8d000640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6806de00 0x7f7e680702c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000f60 tx=0x7f7e78009040 comp rx=0 tx=0).stop 2024-05-30T16:56:17.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.730+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e88159cb0 msgr2=0x7f7e88156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:17.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.730+0000 7f7e8d000640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e88159cb0 0x7f7e88156d60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009dd0 tx=0x7f7e8000c490 comp rx=0 tx=0).stop 2024-05-30T16:56:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.735+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 shutdown_connections 2024-05-30T16:56:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.735+0000 7f7e8d000640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e6806de00 0x7f7e680702c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.735+0000 7f7e8d000640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e88159cb0 0x7f7e88156d60 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.735+0000 7f7e8d000640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e88162c20 0x7f7e88155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.735+0000 7f7e8d000640 1 --2- 172.21.15.67:0/1792334721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e8806e450 0x7f7e8815bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.735+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 >> 172.21.15.67:0/1792334721 conn(0x7f7e8806bd20 msgr2=0x7f7e880afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:17.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.736+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 shutdown_connections 2024-05-30T16:56:17.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:17.736+0000 7f7e8d000640 1 -- 172.21.15.67:0/1792334721 wait complete. 2024-05-30T16:56:17.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922810 2024-05-30T16:56:18.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.192+0000 7f9d0d000640 1 Processor -- start 2024-05-30T16:56:18.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.193+0000 7f9d0d000640 1 -- start start 2024-05-30T16:56:18.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.195+0000 7f9d0d000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d08162ac0 0x7f9d080d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:18.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.195+0000 7f9d0d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9d080b4120 0x7f9d080d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:18.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.195+0000 7f9d0d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081633b0 0x7f9d08163800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:18.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.195+0000 7f9d0d000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9d080b3c90 con 0x7f9d080b4120 2024-05-30T16:56:18.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.195+0000 7f9d0d000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9d081640e0 con 0x7f9d08162ac0 2024-05-30T16:56:18.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.195+0000 7f9d0d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9d080b3360 con 0x7f9d081633b0 2024-05-30T16:56:18.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.196+0000 7f9d05e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d08162ac0 0x7f9d080d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:18.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.197+0000 7f9d05e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d08162ac0 0x7f9d080d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:58394/0 (socket says 172.21.15.67:58394) 2024-05-30T16:56:18.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.197+0000 7f9d05e00640 1 -- 172.21.15.67:0/1446985532 learned_addr learned my addr 172.21.15.67:0/1446985532 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:18.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.197+0000 7f9d05400640 1 --2- 172.21.15.67:0/1446985532 >> v2:172.21.15.67:3300/0 conn(0x7f9d080b4120 0x7f9d080d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:18.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.198+0000 7f9d05400640 1 -- 172.21.15.67:0/1446985532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081633b0 msgr2=0x7f9d08163800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:56:18.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.199+0000 7f9d05400640 1 --2- 172.21.15.67:0/1446985532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081633b0 0x7f9d08163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:18.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.199+0000 7f9d05400640 1 -- 172.21.15.67:0/1446985532 >> v2:172.21.15.169:3300/0 conn(0x7f9d08162ac0 msgr2=0x7f9d080d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:18.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.199+0000 7f9d05400640 1 --2- 172.21.15.67:0/1446985532 >> v2:172.21.15.169:3300/0 conn(0x7f9d08162ac0 0x7f9d080d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:18.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.199+0000 7f9d05400640 1 -- 172.21.15.67:0/1446985532 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d080b3640 con 0x7f9d080b4120 2024-05-30T16:56:18.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.201+0000 7f9d05400640 1 --2- 172.21.15.67:0/1446985532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d080b4120 0x7f9d080d0b80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9d0806fe90 tx=0x7f9cf002a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28963d959cf1faa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:18.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.202+0000 7f9d04a00640 1 -- 172.21.15.67:0/1446985532 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0037070 con 0x7f9d080b4120 2024-05-30T16:56:18.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.202+0000 7f9d04a00640 1 -- 172.21.15.67:0/1446985532 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf0032440 con 0x7f9d080b4120 2024-05-30T16:56:18.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.202+0000 7f9d04a00640 1 -- 172.21.15.67:0/1446985532 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0032810 con 0x7f9d080b4120 2024-05-30T16:56:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.204+0000 7f9d0d000640 1 -- 172.21.15.67:0/1446985532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d080b4120 msgr2=0x7f9d080d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.205+0000 7f9d0d000640 1 --2- 172.21.15.67:0/1446985532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d080b4120 0x7f9d080d0b80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9d0806fe90 tx=0x7f9cf002a370 comp rx=0 tx=0).stop 2024-05-30T16:56:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.205+0000 7f9d0d000640 1 -- 172.21.15.67:0/1446985532 shutdown_connections 2024-05-30T16:56:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.205+0000 7f9d0d000640 1 --2- 172.21.15.67:0/1446985532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d080b4120 0x7f9d080d0b80 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.205+0000 7f9d0d000640 1 --2- 172.21.15.67:0/1446985532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081633b0 0x7f9d08163800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.205+0000 7f9d0d000640 1 --2- 172.21.15.67:0/1446985532 >> v2:172.21.15.169:3300/0 conn(0x7f9d08162ac0 0x7f9d080d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.205+0000 7f9d0d000640 1 -- 172.21.15.67:0/1446985532 >> 172.21.15.67:0/1446985532 conn(0x7f9d0806bd20 msgr2=0x7f9d0806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:18.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.206+0000 7f9d0d000640 1 -- 172.21.15.67:0/1446985532 shutdown_connections 2024-05-30T16:56:18.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.207+0000 7f9d0d000640 1 -- 172.21.15.67:0/1446985532 wait complete. 2024-05-30T16:56:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.214+0000 7f9d0d000640 1 Processor -- start 2024-05-30T16:56:18.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.215+0000 7f9d0d000640 1 -- start start 2024-05-30T16:56:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.218+0000 7f9d0d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d08162ac0 0x7f9d0814f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.218+0000 7f9d0d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d081633b0 0x7f9d080ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.218+0000 7f9d0d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081fccd0 0x7f9d0815d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:18.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.218+0000 7f9d0d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9d08163ec0 con 0x7f9d081633b0 2024-05-30T16:56:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.219+0000 7f9d0d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9d08163d40 con 0x7f9d08162ac0 2024-05-30T16:56:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.219+0000 7f9d0d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9d08162ea0 con 0x7f9d081fccd0 2024-05-30T16:56:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.219+0000 7f9d05e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d08162ac0 0x7f9d0814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.219+0000 7f9d06800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081fccd0 0x7f9d0815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:18.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.219+0000 7f9d06800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081fccd0 0x7f9d0815d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:37138/0 (socket says 172.21.15.67:37138) 2024-05-30T16:56:18.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.219+0000 7f9d06800640 1 -- 172.21.15.67:0/3831823215 learned_addr learned my addr 172.21.15.67:0/3831823215 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:18.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.219+0000 7f9d05400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d081633b0 0x7f9d080ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:18.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.222+0000 7f9d05400640 1 -- 172.21.15.67:0/3831823215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081fccd0 msgr2=0x7f9d0815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:18.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.222+0000 7f9d05400640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081fccd0 0x7f9d0815d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:18.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.223+0000 7f9d05400640 1 -- 172.21.15.67:0/3831823215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d08162ac0 msgr2=0x7f9d0814f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:18.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.223+0000 7f9d05400640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d08162ac0 0x7f9d0814f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:18.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.223+0000 7f9d05400640 1 -- 172.21.15.67:0/3831823215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d080cea60 con 0x7f9d081633b0 2024-05-30T16:56:18.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.223+0000 7f9d05e00640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d08162ac0 0x7f9d0814f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:18.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.227+0000 7f9d05400640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d081633b0 0x7f9d080ce160 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0031000 tx=0x7f9cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:18.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.233+0000 7f9cfe000640 1 -- 172.21.15.67:0/3831823215 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0040020 con 0x7f9d081633b0 2024-05-30T16:56:18.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.233+0000 7f9cfe000640 1 -- 172.21.15.67:0/3831823215 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf0038040 con 0x7f9d081633b0 2024-05-30T16:56:18.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.233+0000 7f9cfe000640 1 -- 172.21.15.67:0/3831823215 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0037070 con 0x7f9d081633b0 2024-05-30T16:56:18.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.235+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d080ced40 con 0x7f9d081633b0 2024-05-30T16:56:18.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.235+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d080cf300 con 0x7f9d081633b0 2024-05-30T16:56:18.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.240+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d080689e0 con 0x7f9d081633b0 2024-05-30T16:56:18.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.249+0000 7f9cfe000640 1 -- 172.21.15.67:0/3831823215 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f9cf0043020 con 0x7f9d081633b0 2024-05-30T16:56:18.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.256+0000 7f9cfe000640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ce406daa0 0x7f9ce406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:18.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.256+0000 7f9cfe000640 1 -- 172.21.15.67:0/3831823215 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f9cf00aaf00 con 0x7f9d081633b0 2024-05-30T16:56:18.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.258+0000 7f9d05e00640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ce406daa0 0x7f9ce406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:18.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.259+0000 7f9cfe000640 1 -- 172.21.15.67:0/3831823215 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9cf004a030 con 0x7f9d081633b0 2024-05-30T16:56:18.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.262+0000 7f9d05e00640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ce406daa0 0x7f9ce406ff60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9d00003b80 tx=0x7f9d000091b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:18.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:18.722+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922810", "target": ["mon-mgr", ""]}) -- 0x7f9d080af4d0 con 0x7f9ce406daa0 2024-05-30T16:56:19.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.169+0000 7f9cfe000640 1 -- 172.21.15.67:0/3831823215 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d080af4d0 con 0x7f9ce406daa0 2024-05-30T16:56:19.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ce406daa0 msgr2=0x7f9ce406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:19.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ce406daa0 0x7f9ce406ff60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9d00003b80 tx=0x7f9d000091b0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d081633b0 msgr2=0x7f9d080ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:19.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d081633b0 0x7f9d080ce160 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0031000 tx=0x7f9cf00046c0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 shutdown_connections 2024-05-30T16:56:19.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9ce406daa0 0x7f9ce406ff60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d081fccd0 0x7f9d0815d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d081633b0 0x7f9d080ce160 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 --2- 172.21.15.67:0/3831823215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d08162ac0 0x7f9d0814f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.173+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 >> 172.21.15.67:0/3831823215 conn(0x7f9d0806bd20 msgr2=0x7f9d08071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:19.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.174+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 shutdown_connections 2024-05-30T16:56:19.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.174+0000 7f9d0d000640 1 -- 172.21.15.67:0/3831823215 wait complete. 2024-05-30T16:56:19.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922811 2024-05-30T16:56:19.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.510+0000 7f76a0e00640 1 Processor -- start 2024-05-30T16:56:19.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.510+0000 7f76a0e00640 1 -- start start 2024-05-30T16:56:19.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.512+0000 7f76a0e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f769c162480 0x7f769c160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:19.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.512+0000 7f76a0e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f769c06ca10 0x7f769c162860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:19.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.512+0000 7f76a0e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f769c162da0 con 0x7f769c09b5d0 2024-05-30T16:56:19.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.512+0000 7f76a0e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f769c06d1f0 con 0x7f769c162480 2024-05-30T16:56:19.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.512+0000 7f76a0e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f769c06d4d0 con 0x7f769c06ca10 2024-05-30T16:56:19.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.513+0000 7f7699e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f769c162480 0x7f769c160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:19.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.513+0000 7f769a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f769c09b5d0 0x7f769c0d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36068/0 (socket says 172.21.15.67:36068) 2024-05-30T16:56:19.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.513+0000 7f769a800640 1 -- 172.21.15.67:0/3162818696 learned_addr learned my addr 172.21.15.67:0/3162818696 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:19.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.513+0000 7f7699e00640 1 -- 172.21.15.67:0/3162818696 >> v2:172.21.15.169:3300/0 conn(0x7f769c06ca10 msgr2=0x7f769c162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:19.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.513+0000 7f7699e00640 1 --2- 172.21.15.67:0/3162818696 >> v2:172.21.15.169:3300/0 conn(0x7f769c06ca10 0x7f769c162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.514+0000 7f7699e00640 1 -- 172.21.15.67:0/3162818696 >> v1:172.21.15.67:6789/0 conn(0x7f769c09b5d0 legacy=0x7f769c0d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:19.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.514+0000 7f7699e00640 1 -- 172.21.15.67:0/3162818696 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c06d7b0 con 0x7f769c162480 2024-05-30T16:56:19.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.515+0000 7f7699e00640 1 --2- 172.21.15.67:0/3162818696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c162480 0x7f769c160280 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f768c0097a0 tx=0x7f768c02a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a27ba1483b4f5f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:19.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.515+0000 7f7699400640 1 -- 172.21.15.67:0/3162818696 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c037070 con 0x7f769c162480 2024-05-30T16:56:19.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.516+0000 7f7699400640 1 -- 172.21.15.67:0/3162818696 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c032440 con 0x7f769c162480 2024-05-30T16:56:19.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.516+0000 7f7699400640 1 -- 172.21.15.67:0/3162818696 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c032850 con 0x7f769c162480 2024-05-30T16:56:19.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.517+0000 7f76a0e00640 1 -- 172.21.15.67:0/3162818696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c162480 msgr2=0x7f769c160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:19.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.517+0000 7f76a0e00640 1 --2- 172.21.15.67:0/3162818696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c162480 0x7f769c160280 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f768c0097a0 tx=0x7f768c02a1f0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.518+0000 7f76a0e00640 1 -- 172.21.15.67:0/3162818696 shutdown_connections 2024-05-30T16:56:19.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.518+0000 7f76a0e00640 1 --2- 172.21.15.67:0/3162818696 >> v2:172.21.15.169:3300/0 conn(0x7f769c06ca10 0x7f769c162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.518+0000 7f76a0e00640 1 --2- 172.21.15.67:0/3162818696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c162480 0x7f769c160280 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.518+0000 7f76a0e00640 1 -- 172.21.15.67:0/3162818696 >> 172.21.15.67:0/3162818696 conn(0x7f769c0d2430 msgr2=0x7f769c0d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:19.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.519+0000 7f76a0e00640 1 -- 172.21.15.67:0/3162818696 shutdown_connections 2024-05-30T16:56:19.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.519+0000 7f76a0e00640 1 -- 172.21.15.67:0/3162818696 wait complete. 2024-05-30T16:56:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.525+0000 7f76a0e00640 1 Processor -- start 2024-05-30T16:56:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.525+0000 7f76a0e00640 1 -- start start 2024-05-30T16:56:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.527+0000 7f76a0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c06ca10 0x7f769c0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.527+0000 7f76a0e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c09b5d0 0x7f769c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.527+0000 7f76a0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c162480 0x7f769c159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.527+0000 7f76a0e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f769c06cea0 con 0x7f769c06ca10 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.527+0000 7f76a0e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f769c0d08a0 con 0x7f769c09b5d0 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.527+0000 7f76a0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f769c09ba60 con 0x7f769c162480 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.527+0000 7f7699e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c06ca10 0x7f769c0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.528+0000 7f7699e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c06ca10 0x7f769c0c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58666/0 (socket says 172.21.15.67:58666) 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.528+0000 7f7699e00640 1 -- 172.21.15.67:0/1221357459 learned_addr learned my addr 172.21.15.67:0/1221357459 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.528+0000 7f7693e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c09b5d0 0x7f769c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.528+0000 7f7699e00640 1 -- 172.21.15.67:0/1221357459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c162480 msgr2=0x7f769c159f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.528+0000 7f7699e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c162480 0x7f769c159f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.528+0000 7f7699e00640 1 -- 172.21.15.67:0/1221357459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c09b5d0 msgr2=0x7f769c0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.528+0000 7f7699e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c09b5d0 0x7f769c0c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.528+0000 7f7699e00640 1 -- 172.21.15.67:0/1221357459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c0c8450 con 0x7f769c06ca10 2024-05-30T16:56:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.528+0000 7f7693e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c09b5d0 0x7f769c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:56:19.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.529+0000 7f7699e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c06ca10 0x7f769c0c87b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f768c02a7e0 tx=0x7f768c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:19.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.529+0000 7f7692000640 1 -- 172.21.15.67:0/1221357459 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c041070 con 0x7f769c06ca10 2024-05-30T16:56:19.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.529+0000 7f7692000640 1 -- 172.21.15.67:0/1221357459 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c03c930 con 0x7f769c06ca10 2024-05-30T16:56:19.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.529+0000 7f7692000640 1 -- 172.21.15.67:0/1221357459 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c037040 con 0x7f769c06ca10 2024-05-30T16:56:19.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.530+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c0c8170 con 0x7f769c06ca10 2024-05-30T16:56:19.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.530+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c0c6930 con 0x7f769c06ca10 2024-05-30T16:56:19.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.534+0000 7f7692000640 1 -- 172.21.15.67:0/1221357459 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f768c044070 con 0x7f769c06ca10 2024-05-30T16:56:19.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.534+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c0689e0 con 0x7f769c06ca10 2024-05-30T16:56:19.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.535+0000 7f7692000640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f767806da00 0x7f767806fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:19.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.535+0000 7f7692000640 1 -- 172.21.15.67:0/1221357459 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f768c0ab6b0 con 0x7f769c06ca10 2024-05-30T16:56:19.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.536+0000 7f7693e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f767806da00 0x7f767806fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:19.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.541+0000 7f7693e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f767806da00 0x7f767806fec0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f769c0a0480 tx=0x7f7684008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:19.543+0000 7f7692000640 1 -- 172.21.15.67:0/1221357459 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f768c07b370 con 0x7f769c06ca10 2024-05-30T16:56:20.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.028+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922811", "target": ["mon-mgr", ""]}) -- 0x7f769c0d3020 con 0x7f767806da00 2024-05-30T16:56:20.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.643+0000 7f7692000640 1 -- 172.21.15.67:0/1221357459 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f769c0d3020 con 0x7f767806da00 2024-05-30T16:56:20.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f767806da00 msgr2=0x7f767806fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:20.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f767806da00 0x7f767806fec0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f769c0a0480 tx=0x7f7684008040 comp rx=0 tx=0).stop 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c06ca10 msgr2=0x7f769c0c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c06ca10 0x7f769c0c87b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f768c02a7e0 tx=0x7f768c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 shutdown_connections 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f767806da00 0x7f767806fec0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f769c162480 0x7f769c159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769c09b5d0 0x7f769c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 --2- 172.21.15.67:0/1221357459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f769c06ca10 0x7f769c0c87b0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.650+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 >> 172.21.15.67:0/1221357459 conn(0x7f769c0d2430 msgr2=0x7f769c15cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.651+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 shutdown_connections 2024-05-30T16:56:20.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:20.651+0000 7f76a0e00640 1 -- 172.21.15.67:0/1221357459 wait complete. 2024-05-30T16:56:20.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922812 2024-05-30T16:56:21.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.108+0000 7f47ccc00640 1 Processor -- start 2024-05-30T16:56:21.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.108+0000 7f47ccc00640 1 -- start start 2024-05-30T16:56:21.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.111+0000 7f47ccc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c815e940 0x7f47c80d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:21.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.111+0000 7f47ccc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47c80ba070 0x7f47c80b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:21.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.111+0000 7f47ccc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c80d3060 con 0x7f47c80b66a0 2024-05-30T16:56:21.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.111+0000 7f47ccc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f47c815ed20 con 0x7f47c80ba070 2024-05-30T16:56:21.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.111+0000 7f47ccc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f47c80d32e0 con 0x7f47c815e940 2024-05-30T16:56:21.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.112+0000 7f47c5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c815e940 0x7f47c80d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:21.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.112+0000 7f47c5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c815e940 0x7f47c80d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:33502/0 (socket says 172.21.15.67:33502) 2024-05-30T16:56:21.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.112+0000 7f47c5e00640 1 -- 172.21.15.67:0/2466955988 learned_addr learned my addr 172.21.15.67:0/2466955988 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:21.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.112+0000 7f47c5400640 1 --2- 172.21.15.67:0/2466955988 >> v2:172.21.15.67:3300/0 conn(0x7f47c80ba070 0x7f47c80b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:21.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.113+0000 7f47c5e00640 1 -- 172.21.15.67:0/2466955988 >> v2:172.21.15.67:3300/0 conn(0x7f47c80ba070 msgr2=0x7f47c80b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:21.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.113+0000 7f47c5e00640 1 --2- 172.21.15.67:0/2466955988 >> v2:172.21.15.67:3300/0 conn(0x7f47c80ba070 0x7f47c80b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:21.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.113+0000 7f47c5e00640 1 -- 172.21.15.67:0/2466955988 >> v1:172.21.15.67:6789/0 conn(0x7f47c80b66a0 legacy=0x7f47c80676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:21.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.113+0000 7f47c5e00640 1 -- 172.21.15.67:0/2466955988 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c80d35c0 con 0x7f47c815e940 2024-05-30T16:56:21.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.114+0000 7f47c5400640 1 --2- 172.21.15.67:0/2466955988 >> v2:172.21.15.67:3300/0 conn(0x7f47c80ba070 0x7f47c80b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:21.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.116+0000 7f47c5e00640 1 --2- 172.21.15.67:0/2466955988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c815e940 0x7f47c80d2b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009870 tx=0x7f47b402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de5c63d38dd2699 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:21.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.117+0000 7f47c4a00640 1 -- 172.21.15.67:0/2466955988 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4037070 con 0x7f47c815e940 2024-05-30T16:56:21.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.117+0000 7f47c4a00640 1 -- 172.21.15.67:0/2466955988 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b4032440 con 0x7f47c815e940 2024-05-30T16:56:21.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.117+0000 7f47c4a00640 1 -- 172.21.15.67:0/2466955988 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4032810 con 0x7f47c815e940 2024-05-30T16:56:21.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.119+0000 7f47ccc00640 1 -- 172.21.15.67:0/2466955988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c815e940 msgr2=0x7f47c80d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:21.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.119+0000 7f47ccc00640 1 --2- 172.21.15.67:0/2466955988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c815e940 0x7f47c80d2b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009870 tx=0x7f47b402a270 comp rx=0 tx=0).stop 2024-05-30T16:56:21.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.122+0000 7f47ccc00640 1 -- 172.21.15.67:0/2466955988 shutdown_connections 2024-05-30T16:56:21.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.122+0000 7f47ccc00640 1 --2- 172.21.15.67:0/2466955988 >> v2:172.21.15.67:3300/0 conn(0x7f47c80ba070 0x7f47c80b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:21.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.122+0000 7f47ccc00640 1 --2- 172.21.15.67:0/2466955988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c815e940 0x7f47c80d2b20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:21.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.122+0000 7f47ccc00640 1 -- 172.21.15.67:0/2466955988 >> 172.21.15.67:0/2466955988 conn(0x7f47c806d150 msgr2=0x7f47c80d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:21.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.122+0000 7f47ccc00640 1 -- 172.21.15.67:0/2466955988 shutdown_connections 2024-05-30T16:56:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.122+0000 7f47ccc00640 1 -- 172.21.15.67:0/2466955988 wait complete. 2024-05-30T16:56:21.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.127+0000 7f47ccc00640 1 Processor -- start 2024-05-30T16:56:21.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.127+0000 7f47ccc00640 1 -- start start 2024-05-30T16:56:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.130+0000 7f47ccc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c80b66a0 0x7f47c80cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.131+0000 7f47ccc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c80ba070 0x7f47c8154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.132+0000 7f47ccc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c815e940 0x7f47c8154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.132+0000 7f47ccc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f47c80d3e60 con 0x7f47c80b66a0 2024-05-30T16:56:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.132+0000 7f47ccc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f47c80d3b80 con 0x7f47c815e940 2024-05-30T16:56:21.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.132+0000 7f47ccc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f47c80d4380 con 0x7f47c80ba070 2024-05-30T16:56:21.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.133+0000 7f47c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c80b66a0 0x7f47c80cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:21.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.133+0000 7f47c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c80b66a0 0x7f47c80cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57188/0 (socket says 172.21.15.67:57188) 2024-05-30T16:56:21.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.133+0000 7f47c5e00640 1 -- 172.21.15.67:0/137484848 learned_addr learned my addr 172.21.15.67:0/137484848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:21.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.134+0000 7f47c6800640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c815e940 0x7f47c8154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.136+0000 7f47c5e00640 1 -- 172.21.15.67:0/137484848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c80ba070 msgr2=0x7f47c8154110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T16:56:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.136+0000 7f47c5e00640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c80ba070 0x7f47c8154110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.136+0000 7f47c5e00640 1 -- 172.21.15.67:0/137484848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c815e940 msgr2=0x7f47c8154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:21.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.137+0000 7f47c5e00640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c815e940 0x7f47c8154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:21.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.137+0000 7f47c5e00640 1 -- 172.21.15.67:0/137484848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c81ec040 con 0x7f47c80b66a0 2024-05-30T16:56:21.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.137+0000 7f47c6800640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c815e940 0x7f47c8154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:56:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.139+0000 7f47c5e00640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c80b66a0 0x7f47c80cdd90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f47b40090b0 tx=0x7f47b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:21.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.140+0000 7f47be000640 1 -- 172.21.15.67:0/137484848 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4041070 con 0x7f47c80b66a0 2024-05-30T16:56:21.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.140+0000 7f47be000640 1 -- 172.21.15.67:0/137484848 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b403c890 con 0x7f47c80b66a0 2024-05-30T16:56:21.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.140+0000 7f47be000640 1 -- 172.21.15.67:0/137484848 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4037040 con 0x7f47c80b66a0 2024-05-30T16:56:21.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.140+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c81ebd60 con 0x7f47c80b66a0 2024-05-30T16:56:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.141+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c80cd590 con 0x7f47c80b66a0 2024-05-30T16:56:21.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.144+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c80689e0 con 0x7f47c80b66a0 2024-05-30T16:56:21.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.150+0000 7f47be000640 1 -- 172.21.15.67:0/137484848 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f47b4044070 con 0x7f47c80b66a0 2024-05-30T16:56:21.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.152+0000 7f47be000640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f47a806da20 0x7f47a806fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:21.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.153+0000 7f47be000640 1 -- 172.21.15.67:0/137484848 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f47b40aba30 con 0x7f47c80b66a0 2024-05-30T16:56:21.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.155+0000 7f47be000640 1 -- 172.21.15.67:0/137484848 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47b407b570 con 0x7f47c80b66a0 2024-05-30T16:56:21.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.162+0000 7f47c5400640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f47a806da20 0x7f47a806fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.170+0000 7f47c5400640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f47a806da20 0x7f47a806fee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f47b8006fd0 tx=0x7f47b8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:21.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:21.691+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922812", "target": ["mon-mgr", ""]}) -- 0x7f47c80b9250 con 0x7f47a806da20 2024-05-30T16:56:22.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.935+0000 7f47be000640 1 -- 172.21.15.67:0/137484848 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f47c80b9250 con 0x7f47a806da20 2024-05-30T16:56:22.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.943+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f47a806da20 msgr2=0x7f47a806fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:22.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.943+0000 7f47ccc00640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f47a806da20 0x7f47a806fee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f47b8006fd0 tx=0x7f47b8008040 comp rx=0 tx=0).stop 2024-05-30T16:56:22.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.943+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c80b66a0 msgr2=0x7f47c80cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:22.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.943+0000 7f47ccc00640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c80b66a0 0x7f47c80cdd90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f47b40090b0 tx=0x7f47b40046c0 comp rx=0 tx=0).stop 2024-05-30T16:56:22.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.944+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 shutdown_connections 2024-05-30T16:56:22.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.944+0000 7f47ccc00640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f47a806da20 0x7f47a806fee0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:22.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.944+0000 7f47ccc00640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47c815e940 0x7f47c8154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:22.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.944+0000 7f47ccc00640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47c80ba070 0x7f47c8154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:22.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.944+0000 7f47ccc00640 1 --2- 172.21.15.67:0/137484848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c80b66a0 0x7f47c80cdd90 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:22.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.944+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 >> 172.21.15.67:0/137484848 conn(0x7f47c806d150 msgr2=0x7f47c80b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:22.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.945+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 shutdown_connections 2024-05-30T16:56:22.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:22.945+0000 7f47ccc00640 1 -- 172.21.15.67:0/137484848 wait complete. 2024-05-30T16:56:23.007 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922813 2024-05-30T16:56:23.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.502+0000 7f8af3e00640 1 Processor -- start 2024-05-30T16:56:23.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.503+0000 7f8af3e00640 1 -- start start 2024-05-30T16:56:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.505+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af4169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.505+0000 7f8af3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af415ec50 con 0x7f8af40d16a0 2024-05-30T16:56:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.505+0000 7f8af3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af40cff80 con 0x7f8af415e870 2024-05-30T16:56:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.505+0000 7f8af3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8af40d2150 con 0x7f8af40cfa40 2024-05-30T16:56:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.508+0000 7f8af3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af4169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.508+0000 7f8af3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af4169690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:33518/0 (socket says 172.21.15.67:33518) 2024-05-30T16:56:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.508+0000 7f8af3400640 1 -- 172.21.15.67:0/1094596279 learned_addr learned my addr 172.21.15.67:0/1094596279 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.509+0000 7f8af1600640 1 -- 172.21.15.67:0/1094596279 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804185172 0 0) 0x7f8af415ec50 con 0x7f8af40d16a0 2024-05-30T16:56:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.510+0000 7f8af1600640 1 -- 172.21.15.67:0/1094596279 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ad8003520 con 0x7f8af40d16a0 2024-05-30T16:56:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.510+0000 7f8af3400640 1 -- 172.21.15.67:0/1094596279 >> v1:172.21.15.169:6789/0 conn(0x7f8af415e870 legacy=0x7f8af40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:23.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.510+0000 7f8af3400640 1 -- 172.21.15.67:0/1094596279 >> v1:172.21.15.67:6789/0 conn(0x7f8af40d16a0 legacy=0x7f8af40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:23.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.510+0000 7f8af3400640 1 -- 172.21.15.67:0/1094596279 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af40d0260 con 0x7f8af40cfa40 2024-05-30T16:56:23.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.513+0000 7f8af3400640 1 --2- 172.21.15.67:0/1094596279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af4169690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40098a0 tx=0x7f8ae402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2730f0aa5f4d3aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.513+0000 7f8af1600640 1 -- 172.21.15.67:0/1094596279 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4037070 con 0x7f8af40cfa40 2024-05-30T16:56:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.513+0000 7f8af1600640 1 -- 172.21.15.67:0/1094596279 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae4032440 con 0x7f8af40cfa40 2024-05-30T16:56:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.513+0000 7f8af1600640 1 -- 172.21.15.67:0/1094596279 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4032810 con 0x7f8af40cfa40 2024-05-30T16:56:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.515+0000 7f8af3e00640 1 -- 172.21.15.67:0/1094596279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 msgr2=0x7f8af4169690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.515+0000 7f8af3e00640 1 --2- 172.21.15.67:0/1094596279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af4169690 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40098a0 tx=0x7f8ae402a290 comp rx=0 tx=0).stop 2024-05-30T16:56:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.516+0000 7f8af3e00640 1 -- 172.21.15.67:0/1094596279 shutdown_connections 2024-05-30T16:56:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.516+0000 7f8af3e00640 1 --2- 172.21.15.67:0/1094596279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af4169690 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.516+0000 7f8af3e00640 1 -- 172.21.15.67:0/1094596279 >> 172.21.15.67:0/1094596279 conn(0x7f8af408bfc0 msgr2=0x7f8af40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.517+0000 7f8af3e00640 1 -- 172.21.15.67:0/1094596279 shutdown_connections 2024-05-30T16:56:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.517+0000 7f8af3e00640 1 -- 172.21.15.67:0/1094596279 wait complete. 2024-05-30T16:56:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.521+0000 7f8af3e00640 1 Processor -- start 2024-05-30T16:56:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.522+0000 7f8af3e00640 1 -- start start 2024-05-30T16:56:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.524+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af40b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.524+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af40d16a0 0x7f8af40b9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.524+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8af415e870 0x7f8af415bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.524+0000 7f8af3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8af40d4fb0 con 0x7f8af415e870 2024-05-30T16:56:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.524+0000 7f8af3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8af40d0820 con 0x7f8af40d16a0 2024-05-30T16:56:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.524+0000 7f8af3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8af40d35f0 con 0x7f8af40cfa40 2024-05-30T16:56:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.525+0000 7f8af2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af40b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.525+0000 7f8af2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af40b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:33528/0 (socket says 172.21.15.67:33528) 2024-05-30T16:56:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.525+0000 7f8af2a00640 1 -- 172.21.15.67:0/1194148743 learned_addr learned my addr 172.21.15.67:0/1194148743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.526+0000 7f8af2a00640 1 -- 172.21.15.67:0/1194148743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af40d16a0 msgr2=0x7f8af40b9d20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T16:56:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.526+0000 7f8af2a00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af40d16a0 0x7f8af40b9d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.526+0000 7f8af2a00640 1 -- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8af415e870 msgr2=0x7f8af415bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.527+0000 7f8af3400640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8af415e870 0x7f8af415bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:23.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.527+0000 7f8af2a00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8af415e870 0x7f8af415bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:23.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.527+0000 7f8af2a00640 1 -- 172.21.15.67:0/1194148743 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af41f91a0 con 0x7f8af40cfa40 2024-05-30T16:56:23.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.528+0000 7f8af2a00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af40b71a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000c3c0 tx=0x7f8ae000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:23.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.529+0000 7f8aeaa00640 1 -- 172.21.15.67:0/1194148743 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00052d0 con 0x7f8af40cfa40 2024-05-30T16:56:23.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.530+0000 7f8aeaa00640 1 -- 172.21.15.67:0/1194148743 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0005490 con 0x7f8af40cfa40 2024-05-30T16:56:23.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.530+0000 7f8aeaa00640 1 -- 172.21.15.67:0/1194148743 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0007260 con 0x7f8af40cfa40 2024-05-30T16:56:23.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.530+0000 7f8af3400640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8af415e870 0x7f8af415bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:23.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.530+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af40b76e0 con 0x7f8af40cfa40 2024-05-30T16:56:23.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.530+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af415b100 con 0x7f8af40cfa40 2024-05-30T16:56:23.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.533+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af40d3ec0 con 0x7f8af40cfa40 2024-05-30T16:56:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.587+0000 7f8aeaa00640 1 -- 172.21.15.67:0/1194148743 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f8ae0007400 con 0x7f8af40cfa40 2024-05-30T16:56:23.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.589+0000 7f8aeaa00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8ad406daa0 0x7f8ad406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:23.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.590+0000 7f8aeaa00640 1 -- 172.21.15.67:0/1194148743 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f8ae008a120 con 0x7f8af40cfa40 2024-05-30T16:56:23.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.590+0000 7f8aeaa00640 1 -- 172.21.15.67:0/1194148743 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8af40d3ec0 con 0x7f8af40cfa40 2024-05-30T16:56:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.608+0000 7f8af2000640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8ad406daa0 0x7f8ad406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:23.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:23.638+0000 7f8af2000640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8ad406daa0 0x7f8ad406ff60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8af4073000 tx=0x7f8aec007c10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:24.207+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922813", "target": ["mon-mgr", ""]}) -- 0x7f8af40689e0 con 0x7f8ad406daa0 2024-05-30T16:56:25.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.484+0000 7f8aeaa00640 1 -- 172.21.15.67:0/1194148743 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8af40689e0 con 0x7f8ad406daa0 2024-05-30T16:56:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8ad406daa0 msgr2=0x7f8ad406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8ad406daa0 0x7f8ad406ff60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8af4073000 tx=0x7f8aec007c10 comp rx=0 tx=0).stop 2024-05-30T16:56:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 msgr2=0x7f8af40b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af40b71a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000c3c0 tx=0x7f8ae000c880 comp rx=0 tx=0).stop 2024-05-30T16:56:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 shutdown_connections 2024-05-30T16:56:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8ad406daa0 0x7f8ad406ff60 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8af4073000 tx=0x7f8aec007c10 comp rx=0 tx=0).stop 2024-05-30T16:56:25.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8af415e870 0x7f8af415bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:25.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af40d16a0 0x7f8af40b9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:25.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 --2- 172.21.15.67:0/1194148743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af40cfa40 0x7f8af40b71a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:25.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.493+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 >> 172.21.15.67:0/1194148743 conn(0x7f8af408bfc0 msgr2=0x7f8af41591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:25.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.496+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 shutdown_connections 2024-05-30T16:56:25.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:25.496+0000 7f8af3e00640 1 -- 172.21.15.67:0/1194148743 wait complete. 2024-05-30T16:56:25.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922814 2024-05-30T16:56:26.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.066+0000 7f12b2e00640 1 Processor -- start 2024-05-30T16:56:26.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.066+0000 7f12b2e00640 1 -- start start 2024-05-30T16:56:26.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.069+0000 7f12b2e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f12ac15ab20 0x7f12ac073a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.069+0000 7f12b2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac0b3fb0 0x7f12ac15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.069+0000 7f12b2e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12ac15bf10 con 0x7f12ac0b3330 2024-05-30T16:56:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.069+0000 7f12b2e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f12ac0cc920 con 0x7f12ac15ab20 2024-05-30T16:56:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.069+0000 7f12b2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12ac0b17c0 con 0x7f12ac0b3fb0 2024-05-30T16:56:26.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.070+0000 7f12abe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f12ac15ab20 0x7f12ac073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:26.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.070+0000 7f12abe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f12ac15ab20 0x7f12ac073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:54148/0 (socket says 172.21.15.67:54148) 2024-05-30T16:56:26.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.070+0000 7f12abe00640 1 -- 172.21.15.67:0/1435825292 learned_addr learned my addr 172.21.15.67:0/1435825292 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:26.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.070+0000 7f12b0e00640 1 --2- 172.21.15.67:0/1435825292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac0b3fb0 0x7f12ac15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:26.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.070+0000 7f12abe00640 1 -- 172.21.15.67:0/1435825292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac0b3fb0 msgr2=0x7f12ac15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:26.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.070+0000 7f12abe00640 1 --2- 172.21.15.67:0/1435825292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac0b3fb0 0x7f12ac15c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:26.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.070+0000 7f12abe00640 1 -- 172.21.15.67:0/1435825292 >> v1:172.21.15.169:6789/0 conn(0x7f12ac0b3330 legacy=0x7f12ac0b38a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:56:26.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.071+0000 7f12abe00640 1 -- 172.21.15.67:0/1435825292 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac0b1aa0 con 0x7f12ac15ab20 2024-05-30T16:56:26.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.073+0000 7f12abe00640 1 --2- 172.21.15.67:0/1435825292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac15ab20 0x7f12ac073a40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009a50 tx=0x7f12a402a9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57d52ec37a8eefb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.075+0000 7f12aaa00640 1 -- 172.21.15.67:0/1435825292 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4030680 con 0x7f12ac15ab20 2024-05-30T16:56:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.075+0000 7f12aaa00640 1 -- 172.21.15.67:0/1435825292 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a4035070 con 0x7f12ac15ab20 2024-05-30T16:56:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.075+0000 7f12aaa00640 1 -- 172.21.15.67:0/1435825292 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4030a80 con 0x7f12ac15ab20 2024-05-30T16:56:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.077+0000 7f12b2e00640 1 -- 172.21.15.67:0/1435825292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac15ab20 msgr2=0x7f12ac073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.077+0000 7f12b2e00640 1 --2- 172.21.15.67:0/1435825292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac15ab20 0x7f12ac073a40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009a50 tx=0x7f12a402a9d0 comp rx=0 tx=0).stop 2024-05-30T16:56:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.081+0000 7f12b2e00640 1 -- 172.21.15.67:0/1435825292 shutdown_connections 2024-05-30T16:56:26.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.081+0000 7f12b2e00640 1 --2- 172.21.15.67:0/1435825292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac0b3fb0 0x7f12ac15c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.081+0000 7f12b2e00640 1 --2- 172.21.15.67:0/1435825292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac15ab20 0x7f12ac073a40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.081+0000 7f12b2e00640 1 -- 172.21.15.67:0/1435825292 >> 172.21.15.67:0/1435825292 conn(0x7f12ac0b1fc0 msgr2=0x7f12ac0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.081+0000 7f12b2e00640 1 -- 172.21.15.67:0/1435825292 shutdown_connections 2024-05-30T16:56:26.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.082+0000 7f12b2e00640 1 -- 172.21.15.67:0/1435825292 wait complete. 2024-05-30T16:56:26.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.085+0000 7f12b2e00640 1 Processor -- start 2024-05-30T16:56:26.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.088+0000 7f12b2e00640 1 -- start start 2024-05-30T16:56:26.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.090+0000 7f12b2e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12ac0b3330 0x7f12ac14d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:26.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.090+0000 7f12b2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac0b3fb0 0x7f12ac0999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:26.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.090+0000 7f12b2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac15ab20 0x7f12ac152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:26.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.090+0000 7f12b2e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac0cc540 con 0x7f12ac0b3330 2024-05-30T16:56:26.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.090+0000 7f12b2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac15c1b0 con 0x7f12ac0b3fb0 2024-05-30T16:56:26.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.090+0000 7f12b2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12ac0cc6c0 con 0x7f12ac15ab20 2024-05-30T16:56:26.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.090+0000 7f12abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12ac0b3330 0x7f12ac14d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:26.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.090+0000 7f12abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12ac0b3330 0x7f12ac14d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57206/0 (socket says 172.21.15.67:57206) 2024-05-30T16:56:26.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.090+0000 7f12abe00640 1 -- 172.21.15.67:0/237558297 learned_addr learned my addr 172.21.15.67:0/237558297 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:26.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.091+0000 7f12ab400640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac0b3fb0 0x7f12ac0999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:26.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.091+0000 7f12b0e00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac15ab20 0x7f12ac152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:26.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.093+0000 7f12abe00640 1 -- 172.21.15.67:0/237558297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac15ab20 msgr2=0x7f12ac152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:26.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.093+0000 7f12abe00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac15ab20 0x7f12ac152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:26.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.094+0000 7f12abe00640 1 -- 172.21.15.67:0/237558297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac0b3fb0 msgr2=0x7f12ac0999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:26.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.094+0000 7f12abe00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac0b3fb0 0x7f12ac0999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:26.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.094+0000 7f12abe00640 1 -- 172.21.15.67:0/237558297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac151cb0 con 0x7f12ac0b3330 2024-05-30T16:56:26.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.094+0000 7f12ab400640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac0b3fb0 0x7f12ac0999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:56:26.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.097+0000 7f12abe00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12ac0b3330 0x7f12ac14d370 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009a50 tx=0x7f12a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:26.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.098+0000 7f12a3e00640 1 -- 172.21.15.67:0/237558297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4039810 con 0x7f12ac0b3330 2024-05-30T16:56:26.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.098+0000 7f12a3e00640 1 -- 172.21.15.67:0/237558297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a40399d0 con 0x7f12ac0b3330 2024-05-30T16:56:26.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.098+0000 7f12a3e00640 1 -- 172.21.15.67:0/237558297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4039dc0 con 0x7f12ac0b3330 2024-05-30T16:56:26.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.098+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac146e10 con 0x7f12ac0b3330 2024-05-30T16:56:26.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.099+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac144a60 con 0x7f12ac0b3330 2024-05-30T16:56:26.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.101+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac0689e0 con 0x7f12ac0b3330 2024-05-30T16:56:26.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.104+0000 7f12a3e00640 1 -- 172.21.15.67:0/237558297 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f12a402e050 con 0x7f12ac0b3330 2024-05-30T16:56:26.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.105+0000 7f12a3e00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f128c06d9d0 0x7f128c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:26.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.105+0000 7f12a3e00640 1 -- 172.21.15.67:0/237558297 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f12a40ab5d0 con 0x7f12ac0b3330 2024-05-30T16:56:26.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.127+0000 7f12a3e00640 1 -- 172.21.15.67:0/237558297 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12a402c040 con 0x7f12ac0b3330 2024-05-30T16:56:26.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.133+0000 7f12ab400640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f128c06d9d0 0x7f128c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.149+0000 7f12ab400640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f128c06d9d0 0x7f128c06fe90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f129c006fd0 tx=0x7f129c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:26.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:26.705+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922814", "target": ["mon-mgr", ""]}) -- 0x7f12ac0b26c0 con 0x7f128c06d9d0 2024-05-30T16:56:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.673+0000 7f12a3e00640 1 -- 172.21.15.67:0/237558297 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12ac0b26c0 con 0x7f128c06d9d0 2024-05-30T16:56:27.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f128c06d9d0 msgr2=0x7f128c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:27.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f128c06d9d0 0x7f128c06fe90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f129c006fd0 tx=0x7f129c008040 comp rx=0 tx=0).stop 2024-05-30T16:56:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12ac0b3330 msgr2=0x7f12ac14d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12ac0b3330 0x7f12ac14d370 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009a50 tx=0x7f12a4004a40 comp rx=0 tx=0).stop 2024-05-30T16:56:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 shutdown_connections 2024-05-30T16:56:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f128c06d9d0 0x7f128c06fe90 secure :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f129c006fd0 tx=0x7f129c008040 comp rx=0 tx=0).stop 2024-05-30T16:56:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12ac15ab20 0x7f12ac152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12ac0b3fb0 0x7f12ac0999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 --2- 172.21.15.67:0/237558297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12ac0b3330 0x7f12ac14d370 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:27.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.689+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 >> 172.21.15.67:0/237558297 conn(0x7f12ac0b1fc0 msgr2=0x7f12ac15af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:27.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.691+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 shutdown_connections 2024-05-30T16:56:27.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:27.691+0000 7f12b2e00640 1 -- 172.21.15.67:0/237558297 wait complete. 2024-05-30T16:56:27.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922815 2024-05-30T16:56:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.198+0000 7f07f3e00640 1 Processor -- start 2024-05-30T16:56:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.198+0000 7f07f3e00640 1 -- start start 2024-05-30T16:56:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.205+0000 7f07f3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.205+0000 7f07f3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f40cffb0 con 0x7f07f40ad130 2024-05-30T16:56:28.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.207+0000 7f07f3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f415ec00 con 0x7f07f415e820 2024-05-30T16:56:28.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.207+0000 7f07f3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07f40af3f0 con 0x7f07f40ad5c0 2024-05-30T16:56:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.207+0000 7f07f3400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.207+0000 7f07f2000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f07f40ad130 0x7f07f406e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55774/0 (socket says 172.21.15.67:55774) 2024-05-30T16:56:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.207+0000 7f07f2000640 1 -- 172.21.15.67:0/3581282449 learned_addr learned my addr 172.21.15.67:0/3581282449 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:28.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.209+0000 7f07f3400640 1 -- 172.21.15.67:0/3581282449 >> v1:172.21.15.169:6789/0 conn(0x7f07f415e820 legacy=0x7f07f40cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:28.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.209+0000 7f07f3400640 1 -- 172.21.15.67:0/3581282449 >> v1:172.21.15.67:6789/0 conn(0x7f07f40ad130 legacy=0x7f07f406e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:28.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.209+0000 7f07f3400640 1 -- 172.21.15.67:0/3581282449 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f406dcc0 con 0x7f07f40ad5c0 2024-05-30T16:56:28.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.211+0000 7f07f3400640 1 --2- 172.21.15.67:0/3581282449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f406ec40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07e00097d0 tx=0x7f07e002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73fb0523fa32a79e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:28.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.212+0000 7f07f1600640 1 -- 172.21.15.67:0/3581282449 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0037070 con 0x7f07f40ad5c0 2024-05-30T16:56:28.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.212+0000 7f07f1600640 1 -- 172.21.15.67:0/3581282449 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e0032440 con 0x7f07f40ad5c0 2024-05-30T16:56:28.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.212+0000 7f07f1600640 1 -- 172.21.15.67:0/3581282449 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0032810 con 0x7f07f40ad5c0 2024-05-30T16:56:28.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.214+0000 7f07f3e00640 1 -- 172.21.15.67:0/3581282449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 msgr2=0x7f07f406ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:28.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.214+0000 7f07f3e00640 1 --2- 172.21.15.67:0/3581282449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f406ec40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07e00097d0 tx=0x7f07e002a210 comp rx=0 tx=0).stop 2024-05-30T16:56:28.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.217+0000 7f07f3e00640 1 -- 172.21.15.67:0/3581282449 shutdown_connections 2024-05-30T16:56:28.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.217+0000 7f07f3e00640 1 --2- 172.21.15.67:0/3581282449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f406ec40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:28.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.217+0000 7f07f3e00640 1 -- 172.21.15.67:0/3581282449 >> 172.21.15.67:0/3581282449 conn(0x7f07f40b1e50 msgr2=0x7f07f40b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:28.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.220+0000 7f07f3e00640 1 -- 172.21.15.67:0/3581282449 shutdown_connections 2024-05-30T16:56:28.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.220+0000 7f07f3e00640 1 -- 172.21.15.67:0/3581282449 wait complete. 2024-05-30T16:56:28.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.224+0000 7f07f3e00640 1 Processor -- start 2024-05-30T16:56:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.224+0000 7f07f3e00640 1 -- start start 2024-05-30T16:56:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.226+0000 7f07f3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07f40ad130 0x7f07f41f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.226+0000 7f07f3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f4158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.226+0000 7f07f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f415e820 0x7f07f4159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.226+0000 7f07f3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f07f406e280 con 0x7f07f415e820 2024-05-30T16:56:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.226+0000 7f07f3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07f40cf410 con 0x7f07f40ad130 2024-05-30T16:56:28.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.226+0000 7f07f3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07f40ada50 con 0x7f07f40ad5c0 2024-05-30T16:56:28.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.228+0000 7f07f3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f415e820 0x7f07f4159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.229+0000 7f07f3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f415e820 0x7f07f4159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57220/0 (socket says 172.21.15.67:57220) 2024-05-30T16:56:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.229+0000 7f07f3400640 1 -- 172.21.15.67:0/3609487921 learned_addr learned my addr 172.21.15.67:0/3609487921 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.229+0000 7f07f2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f4158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:28.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.230+0000 7f07f2a00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07f40ad130 0x7f07f41f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:28.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.232+0000 7f07f3400640 1 -- 172.21.15.67:0/3609487921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 msgr2=0x7f07f4158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:28.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.232+0000 7f07f3400640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f4158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:28.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.232+0000 7f07f3400640 1 -- 172.21.15.67:0/3609487921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07f40ad130 msgr2=0x7f07f41f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:28.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.233+0000 7f07f3400640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07f40ad130 0x7f07f41f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:28.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.233+0000 7f07f3400640 1 -- 172.21.15.67:0/3609487921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f415a7d0 con 0x7f07f415e820 2024-05-30T16:56:28.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.234+0000 7f07f2a00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07f40ad130 0x7f07f41f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:28.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.234+0000 7f07f2000640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f4158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:28.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.235+0000 7f07f3400640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f415e820 0x7f07f4159dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f07e00097d0 tx=0x7f07e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:28.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.237+0000 7f07eaa00640 1 -- 172.21.15.67:0/3609487921 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0040020 con 0x7f07f415e820 2024-05-30T16:56:28.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.238+0000 7f07eaa00640 1 -- 172.21.15.67:0/3609487921 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e0038040 con 0x7f07f415e820 2024-05-30T16:56:28.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.238+0000 7f07eaa00640 1 -- 172.21.15.67:0/3609487921 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0037070 con 0x7f07f415e820 2024-05-30T16:56:28.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.239+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f415aab0 con 0x7f07f415e820 2024-05-30T16:56:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.244+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f41f83e0 con 0x7f07f415e820 2024-05-30T16:56:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.248+0000 7f07eaa00640 1 -- 172.21.15.67:0/3609487921 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f07e0043020 con 0x7f07f415e820 2024-05-30T16:56:28.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.248+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f40689e0 con 0x7f07f415e820 2024-05-30T16:56:28.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.250+0000 7f07eaa00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f07d406d9d0 0x7f07d406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:28.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.270+0000 7f07f2a00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f07d406d9d0 0x7f07d406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.270+0000 7f07eaa00640 1 -- 172.21.15.67:0/3609487921 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f07e0075860 con 0x7f07f415e820 2024-05-30T16:56:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.275+0000 7f07eaa00640 1 -- 172.21.15.67:0/3609487921 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07e0032e20 con 0x7f07f415e820 2024-05-30T16:56:28.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.279+0000 7f07f2a00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f07d406d9d0 0x7f07d406fe90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f07ec007fd0 tx=0x7f07ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:28.835+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922815", "target": ["mon-mgr", ""]}) -- 0x7f07f4069210 con 0x7f07d406d9d0 2024-05-30T16:56:29.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.928+0000 7f07eaa00640 1 -- 172.21.15.67:0/3609487921 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f07f4069210 con 0x7f07d406d9d0 2024-05-30T16:56:29.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.939+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f07d406d9d0 msgr2=0x7f07d406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:29.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.939+0000 7f07f3e00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f07d406d9d0 0x7f07d406fe90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f07ec007fd0 tx=0x7f07ec009040 comp rx=0 tx=0).stop 2024-05-30T16:56:29.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.939+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f415e820 msgr2=0x7f07f4159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:29.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.939+0000 7f07f3e00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f415e820 0x7f07f4159dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f07e00097d0 tx=0x7f07e00046c0 comp rx=0 tx=0).stop 2024-05-30T16:56:29.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.943+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 shutdown_connections 2024-05-30T16:56:29.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.943+0000 7f07f3e00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f07d406d9d0 0x7f07d406fe90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:29.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.943+0000 7f07f3e00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07f415e820 0x7f07f4159dd0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:29.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.943+0000 7f07f3e00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07f40ad5c0 0x7f07f4158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:29.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.943+0000 7f07f3e00640 1 --2- 172.21.15.67:0/3609487921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07f40ad130 0x7f07f41f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:29.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.943+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 >> 172.21.15.67:0/3609487921 conn(0x7f07f40b1e50 msgr2=0x7f07f40afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:29.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.952+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 shutdown_connections 2024-05-30T16:56:29.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:29.952+0000 7f07f3e00640 1 -- 172.21.15.67:0/3609487921 wait complete. 2024-05-30T16:56:30.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922816 2024-05-30T16:56:30.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.544+0000 7f6ed1600640 1 Processor -- start 2024-05-30T16:56:30.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.544+0000 7f6ed1600640 1 -- start start 2024-05-30T16:56:30.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.549+0000 7f6ed1600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6ecc15d260 0x7f6ecc0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:30.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.549+0000 7f6ed1600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ecc0cc780 con 0x7f6ecc0b2a70 2024-05-30T16:56:30.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.549+0000 7f6ed1600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ecc0ca970 con 0x7f6ecc15c770 2024-05-30T16:56:30.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.549+0000 7f6ed1600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6ecc1602d0 con 0x7f6ecc15d260 2024-05-30T16:56:30.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.549+0000 7f6ecb400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6ecc0b2a70 0x7f6ecc06f1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55776/0 (socket says 172.21.15.67:55776) 2024-05-30T16:56:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.549+0000 7f6ecb400640 1 -- 172.21.15.67:0/146800216 learned_addr learned my addr 172.21.15.67:0/146800216 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.551+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488130779 0 0) 0x7f6ecc0cc780 con 0x7f6ecc0b2a70 2024-05-30T16:56:30.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.551+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6eb0003520 con 0x7f6ecc0b2a70 2024-05-30T16:56:30.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.552+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3178905024 0 0) 0x7f6ecc0ca970 con 0x7f6ecc15c770 2024-05-30T16:56:30.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.552+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ecc0cc780 con 0x7f6ecc15c770 2024-05-30T16:56:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4157629246 0 0) 0x7f6eb0003520 con 0x7f6ecc0b2a70 2024-05-30T16:56:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 >> v2:172.21.15.169:3300/0 conn(0x7f6ecc15d260 msgr2=0x7f6ecc0cd5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:56:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 --2- 172.21.15.67:0/146800216 >> v2:172.21.15.169:3300/0 conn(0x7f6ecc15d260 0x7f6ecc0cd5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 >> v1:172.21.15.169:6789/0 conn(0x7f6ecc15c770 legacy=0x7f6ecc15cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc0cac50 con 0x7f6ecc0b2a70 2024-05-30T16:56:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f6ebc001670 con 0x7f6ecc0b2a70 2024-05-30T16:56:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 >> v1:172.21.15.67:6789/0 conn(0x7f6ecc0b2a70 legacy=0x7f6ecc06f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004ba0 0x7f6eb0004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00037a0 0x7f6eb0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb0004360 0x7f6eb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1602d0 con 0x7f6eb0004ba0 2024-05-30T16:56:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb000c280 con 0x7f6eb0004360 2024-05-30T16:56:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.553+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb000c480 con 0x7f6eb00037a0 2024-05-30T16:56:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.554+0000 7f6ecbe00640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004ba0 0x7f6eb0004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.555+0000 7f6ed0c00640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb0004360 0x7f6eb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.555+0000 7f6ecb400640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00037a0 0x7f6eb0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:30.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.556+0000 7f6ed0c00640 1 -- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00037a0 msgr2=0x7f6eb0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.556+0000 7f6ed0c00640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6eb00037a0 0x7f6eb0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.556+0000 7f6ed0c00640 1 -- 172.21.15.67:0/146800216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004ba0 msgr2=0x7f6eb0004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.556+0000 7f6ed0c00640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004ba0 0x7f6eb0004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.556+0000 7f6ed0c00640 1 -- 172.21.15.67:0/146800216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc0cac50 con 0x7f6eb0004360 2024-05-30T16:56:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.558+0000 7f6ed0c00640 1 -- 172.21.15.67:0/146800216 reap_dead start 2024-05-30T16:56:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.559+0000 7f6ed0c00640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb0004360 0x7f6eb000b9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009860 tx=0x7f6eb0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ff9997a77c4b1a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.560+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb803e070 con 0x7f6eb0004360 2024-05-30T16:56:30.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.560+0000 7f6ecaa00640 1 -- 172.21.15.67:0/146800216 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eb8039600 con 0x7f6eb0004360 2024-05-30T16:56:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.561+0000 7f6ed1600640 1 -- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb0004360 msgr2=0x7f6eb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.561+0000 7f6ed1600640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb0004360 0x7f6eb000b9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009860 tx=0x7f6eb0005830 comp rx=0 tx=0).stop 2024-05-30T16:56:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.562+0000 7f6ed1600640 1 -- 172.21.15.67:0/146800216 shutdown_connections 2024-05-30T16:56:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.562+0000 7f6ed1600640 1 --2- 172.21.15.67:0/146800216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb0004360 0x7f6eb000b9c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.562+0000 7f6ed1600640 1 -- 172.21.15.67:0/146800216 >> 172.21.15.67:0/146800216 conn(0x7f6ecc0b1e50 msgr2=0x7f6ecc0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.562+0000 7f6ed1600640 1 -- 172.21.15.67:0/146800216 shutdown_connections 2024-05-30T16:56:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.562+0000 7f6ed1600640 1 -- 172.21.15.67:0/146800216 wait complete. 2024-05-30T16:56:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.570+0000 7f6ed1600640 1 Processor -- start 2024-05-30T16:56:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.570+0000 7f6ed1600640 1 -- start start 2024-05-30T16:56:30.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.572+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004360 0x7f6ecc148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.572+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0999f0 0x7f6ecc0ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.572+0000 7f6ed1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0ad9e0 0x7f6ecc0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.572+0000 7f6ed1600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc0cd5a0 con 0x7f6eb0004360 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.572+0000 7f6ed1600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc0cc540 con 0x7f6ecc0999f0 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.572+0000 7f6ed1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ecc0cd720 con 0x7f6ecc0ad9e0 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.573+0000 7f6ecbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004360 0x7f6ecc148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.573+0000 7f6ecbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004360 0x7f6ecc148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57242/0 (socket says 172.21.15.67:57242) 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.573+0000 7f6ecbe00640 1 -- 172.21.15.67:0/3717195125 learned_addr learned my addr 172.21.15.67:0/3717195125 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.573+0000 7f6ed0c00640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0ad9e0 0x7f6ecc0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.573+0000 7f6ecb400640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0999f0 0x7f6ecc0ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.574+0000 7f6ecbe00640 1 -- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0ad9e0 msgr2=0x7f6ecc0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.574+0000 7f6ecbe00640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0ad9e0 0x7f6ecc0b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.574+0000 7f6ecbe00640 1 -- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0999f0 msgr2=0x7f6ecc0ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.574+0000 7f6ecbe00640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0999f0 0x7f6ecc0ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.574+0000 7f6ecbe00640 1 -- 172.21.15.67:0/3717195125 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc099e70 con 0x7f6eb0004360 2024-05-30T16:56:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.574+0000 7f6ed0c00640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0ad9e0 0x7f6ecc0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.574+0000 7f6ecb400640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0999f0 0x7f6ecc0ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.575+0000 7f6ecbe00640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004360 0x7f6ecc148530 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400e8f0 tx=0x7f6ec400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.576+0000 7f6ec3e00640 1 -- 172.21.15.67:0/3717195125 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4003b80 con 0x7f6eb0004360 2024-05-30T16:56:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.576+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc145040 con 0x7f6eb0004360 2024-05-30T16:56:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.577+0000 7f6ec3e00640 1 -- 172.21.15.67:0/3717195125 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4018070 con 0x7f6eb0004360 2024-05-30T16:56:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.577+0000 7f6ec3e00640 1 -- 172.21.15.67:0/3717195125 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4003b80 con 0x7f6eb0004360 2024-05-30T16:56:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.578+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc145600 con 0x7f6eb0004360 2024-05-30T16:56:30.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.581+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc0addc0 con 0x7f6eb0004360 2024-05-30T16:56:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.588+0000 7f6ec3e00640 1 -- 172.21.15.67:0/3717195125 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f6ec40050d0 con 0x7f6eb0004360 2024-05-30T16:56:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.592+0000 7f6ec3e00640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6eac06dc00 0x7f6eac0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.593+0000 7f6ec3e00640 1 -- 172.21.15.67:0/3717195125 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f6ec4089fc0 con 0x7f6eb0004360 2024-05-30T16:56:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.600+0000 7f6ecb400640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6eac06dc00 0x7f6eac0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.603+0000 7f6ec3e00640 1 -- 172.21.15.67:0/3717195125 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ec4022020 con 0x7f6eb0004360 2024-05-30T16:56:30.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:30.608+0000 7f6ecb400640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6eac06dc00 0x7f6eac0700c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0b60c0 tx=0x7f6ebc0010a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:31.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:31.178+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922816", "target": ["mon-mgr", ""]}) -- 0x7f6ecc15fd50 con 0x7f6eac06dc00 2024-05-30T16:56:32.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.006+0000 7f6ec3e00640 1 -- 172.21.15.67:0/3717195125 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6ecc15fd50 con 0x7f6eac06dc00 2024-05-30T16:56:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.018+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6eac06dc00 msgr2=0x7f6eac0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.018+0000 7f6ed1600640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6eac06dc00 0x7f6eac0700c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0b60c0 tx=0x7f6ebc0010a0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.018+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004360 msgr2=0x7f6ecc148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:32.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.018+0000 7f6ed1600640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004360 0x7f6ecc148530 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400e8f0 tx=0x7f6ec400ecf0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.019+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 shutdown_connections 2024-05-30T16:56:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.019+0000 7f6ed1600640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6eac06dc00 0x7f6eac0700c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.019+0000 7f6ed1600640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0ad9e0 0x7f6ecc0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.019+0000 7f6ed1600640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0999f0 0x7f6ecc0ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.019+0000 7f6ed1600640 1 --2- 172.21.15.67:0/3717195125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6eb0004360 0x7f6ecc148530 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.019+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 >> 172.21.15.67:0/3717195125 conn(0x7f6ecc0b1e50 msgr2=0x7f6ecc0afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:32.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.021+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 shutdown_connections 2024-05-30T16:56:32.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.021+0000 7f6ed1600640 1 -- 172.21.15.67:0/3717195125 wait complete. 2024-05-30T16:56:32.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922817 2024-05-30T16:56:32.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.550+0000 7f3147200640 1 Processor -- start 2024-05-30T16:56:32.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.550+0000 7f3147200640 1 -- start start 2024-05-30T16:56:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.554+0000 7f3147200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f31480d00b0 0x7f31480b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.554+0000 7f3147200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f31480b0840 0x7f31480b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.555+0000 7f3147200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31480b3bd0 con 0x7f31481583d0 2024-05-30T16:56:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.555+0000 7f3147200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f31480ad100 con 0x7f31480b0840 2024-05-30T16:56:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.555+0000 7f3147200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f31480b03b0 con 0x7f31480d00b0 2024-05-30T16:56:32.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.556+0000 7f3145400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f31481583d0 0x7f314815b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:46978/0 (socket says 172.21.15.67:46978) 2024-05-30T16:56:32.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.556+0000 7f3145400640 1 -- 172.21.15.67:0/816718415 learned_addr learned my addr 172.21.15.67:0/816718415 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:32.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.556+0000 7f3145e00640 1 --2- 172.21.15.67:0/816718415 >> v2:172.21.15.169:3300/0 conn(0x7f31480d00b0 0x7f31480b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:32.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.557+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 416217198 0 0) 0x7f31480b3bd0 con 0x7f31481583d0 2024-05-30T16:56:32.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.557+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f312c003520 con 0x7f31481583d0 2024-05-30T16:56:32.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.557+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 68712896 0 0) 0x7f312c003520 con 0x7f31481583d0 2024-05-30T16:56:32.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.557+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 >> v2:172.21.15.169:3300/0 conn(0x7f31480d00b0 msgr2=0x7f31480b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:32.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.557+0000 7f3144a00640 1 --2- 172.21.15.67:0/816718415 >> v2:172.21.15.169:3300/0 conn(0x7f31480d00b0 0x7f31480b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.557+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 >> v2:172.21.15.67:3300/0 conn(0x7f31480b0840 msgr2=0x7f31480b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:32.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 --2- 172.21.15.67:0/816718415 >> v2:172.21.15.67:3300/0 conn(0x7f31480b0840 0x7f31480b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31480b3eb0 con 0x7f31481583d0 2024-05-30T16:56:32.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f31400019d0 con 0x7f31481583d0 2024-05-30T16:56:32.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 >> v1:172.21.15.169:6789/0 conn(0x7f31481583d0 legacy=0x7f314815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:32.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f312c003a00 0x7f312c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:32.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c004800 0x7f312c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:32.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0051c0 0x7f312c0145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:32.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f31480b03b0 con 0x7f312c003a00 2024-05-30T16:56:32.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f312c014b00 con 0x7f312c0051c0 2024-05-30T16:56:32.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.558+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f312c014c80 con 0x7f312c004800 2024-05-30T16:56:32.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.561+0000 7f3145400640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c004800 0x7f312c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:32.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.561+0000 7f3146800640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0051c0 0x7f312c0145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:32.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.562+0000 7f3145e00640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f312c003a00 0x7f312c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:32.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.563+0000 7f3146800640 1 -- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c004800 msgr2=0x7f312c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:32.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.563+0000 7f3146800640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c004800 0x7f312c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.563+0000 7f3146800640 1 -- 172.21.15.67:0/816718415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f312c003a00 msgr2=0x7f312c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:32.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.563+0000 7f3146800640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f312c003a00 0x7f312c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.563+0000 7f3146800640 1 -- 172.21.15.67:0/816718415 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c003520 con 0x7f312c0051c0 2024-05-30T16:56:32.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.567+0000 7f3146800640 1 -- 172.21.15.67:0/816718415 reap_dead start 2024-05-30T16:56:32.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.568+0000 7f3146800640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0051c0 0x7f312c0145c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3138009860 tx=0x7f312c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16466aa8792a61e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:32.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.569+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138009460 con 0x7f312c0051c0 2024-05-30T16:56:32.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.571+0000 7f3144a00640 1 -- 172.21.15.67:0/816718415 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3138004240 con 0x7f312c0051c0 2024-05-30T16:56:32.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.573+0000 7f3147200640 1 -- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0051c0 msgr2=0x7f312c0145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:32.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.573+0000 7f3147200640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0051c0 0x7f312c0145c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3138009860 tx=0x7f312c005830 comp rx=0 tx=0).stop 2024-05-30T16:56:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.574+0000 7f3147200640 1 -- 172.21.15.67:0/816718415 shutdown_connections 2024-05-30T16:56:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.574+0000 7f3147200640 1 --2- 172.21.15.67:0/816718415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c0051c0 0x7f312c0145c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.574+0000 7f3147200640 1 -- 172.21.15.67:0/816718415 >> 172.21.15.67:0/816718415 conn(0x7f31480b55d0 msgr2=0x7f31480b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.574+0000 7f3147200640 1 -- 172.21.15.67:0/816718415 shutdown_connections 2024-05-30T16:56:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.574+0000 7f3147200640 1 -- 172.21.15.67:0/816718415 wait complete. 2024-05-30T16:56:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.578+0000 7f3147200640 1 Processor -- start 2024-05-30T16:56:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.579+0000 7f3147200640 1 -- start start 2024-05-30T16:56:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.581+0000 7f3147200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0051c0 0x7f31480ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.581+0000 7f3147200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31480ae970 0x7f3148147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.581+0000 7f3147200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31481565c0 0x7f3148156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.582+0000 7f3145400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31480ae970 0x7f3148147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:32.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.583+0000 7f3145400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31480ae970 0x7f3148147e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37116/0 (socket says 172.21.15.67:37116) 2024-05-30T16:56:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.583+0000 7f3145e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0051c0 0x7f31480ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.583+0000 7f3145e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0051c0 0x7f31480ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:39928/0 (socket says 172.21.15.67:39928) 2024-05-30T16:56:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.583+0000 7f3145e00640 1 -- 172.21.15.67:0/1186782437 learned_addr learned my addr 172.21.15.67:0/1186782437 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.583+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f31480b3230 con 0x7f31480ae970 2024-05-30T16:56:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.583+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f31480b30b0 con 0x7f31481565c0 2024-05-30T16:56:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.584+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f31480b33b0 con 0x7f312c0051c0 2024-05-30T16:56:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.585+0000 7f3146800640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31481565c0 0x7f3148156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.586+0000 7f3145400640 1 -- 172.21.15.67:0/1186782437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0051c0 msgr2=0x7f31480ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.587+0000 7f3145400640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0051c0 0x7f31480ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.587+0000 7f3145400640 1 -- 172.21.15.67:0/1186782437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31481565c0 msgr2=0x7f3148156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:32.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.587+0000 7f3145400640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31481565c0 0x7f3148156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:32.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.588+0000 7f3145400640 1 -- 172.21.15.67:0/1186782437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3148148350 con 0x7f31480ae970 2024-05-30T16:56:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.588+0000 7f3146800640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31481565c0 0x7f3148156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.589+0000 7f3145400640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31480ae970 0x7f3148147e10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3148158850 tx=0x7f3140007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.590+0000 7f313e000640 1 -- 172.21.15.67:0/1186782437 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140017020 con 0x7f31480ae970 2024-05-30T16:56:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.591+0000 7f313e000640 1 -- 172.21.15.67:0/1186782437 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3140001a40 con 0x7f31480ae970 2024-05-30T16:56:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.592+0000 7f313e000640 1 -- 172.21.15.67:0/1186782437 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314000ac40 con 0x7f31480ae970 2024-05-30T16:56:32.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.592+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31480cd0f0 con 0x7f31480ae970 2024-05-30T16:56:32.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.597+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314815c5f0 con 0x7f31480ae970 2024-05-30T16:56:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.602+0000 7f313e000640 1 -- 172.21.15.67:0/1186782437 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) ==== 92570+0+0 (secure 0 0 0) 0x7f314001a020 con 0x7f31480ae970 2024-05-30T16:56:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.604+0000 7f313e000640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f312406da50 0x7f312406ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:32.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.604+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31480689e0 con 0x7f31480ae970 2024-05-30T16:56:32.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.606+0000 7f313e000640 1 -- 172.21.15.67:0/1186782437 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f314000ade0 con 0x7f31480ae970 2024-05-30T16:56:32.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.622+0000 7f313e000640 1 -- 172.21.15.67:0/1186782437 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31400c1b80 con 0x7f31480ae970 2024-05-30T16:56:32.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.651+0000 7f3145e00640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f312406da50 0x7f312406ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:32.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.668+0000 7f3145e00640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f312406da50 0x7f312406ff10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3130005f60 tx=0x7f313000b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:32.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:32.743+0000 7f313e000640 1 -- 172.21.15.67:0/1186782437 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 29) ==== 92519+0+0 (secure 0 0 0) 0x7f3140055d70 con 0x7f31480ae970 2024-05-30T16:56:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:33.233+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922817", "target": ["mon-mgr", ""]}) -- 0x7f3148158d70 con 0x7f312406da50 2024-05-30T16:56:33.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:33.996+0000 7f313e000640 1 -- 172.21.15.67:0/1186782437 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3148158d70 con 0x7f312406da50 2024-05-30T16:56:34.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.003+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f312406da50 msgr2=0x7f312406ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:34.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.003+0000 7f3147200640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f312406da50 0x7f312406ff10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3130005f60 tx=0x7f313000b040 comp rx=0 tx=0).stop 2024-05-30T16:56:34.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.003+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31480ae970 msgr2=0x7f3148147e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:34.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.003+0000 7f3147200640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31480ae970 0x7f3148147e10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f3148158850 tx=0x7f3140007b10 comp rx=0 tx=0).stop 2024-05-30T16:56:34.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.004+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 shutdown_connections 2024-05-30T16:56:34.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.004+0000 7f3147200640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f312406da50 0x7f312406ff10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:34.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.004+0000 7f3147200640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31481565c0 0x7f3148156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:34.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.004+0000 7f3147200640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31480ae970 0x7f3148147e10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:34.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.004+0000 7f3147200640 1 --2- 172.21.15.67:0/1186782437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f312c0051c0 0x7f31480ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:34.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.004+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 >> 172.21.15.67:0/1186782437 conn(0x7f31480b55d0 msgr2=0x7f31481576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:34.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.004+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 shutdown_connections 2024-05-30T16:56:34.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.004+0000 7f3147200640 1 -- 172.21.15.67:0/1186782437 wait complete. 2024-05-30T16:56:34.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922818 2024-05-30T16:56:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.530+0000 7f149a800640 1 Processor -- start 2024-05-30T16:56:34.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.531+0000 7f149a800640 1 -- start start 2024-05-30T16:56:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.534+0000 7f149a800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f148c0e7800 0x7f148c006f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.535+0000 7f1493400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f148c0e7800 0x7f148c006f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.535+0000 7f1493400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f148c0e7800 0x7f148c006f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:37452/0 (socket says 172.21.15.67:37452) 2024-05-30T16:56:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.535+0000 7f149a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f148c038ea0 0x7f148c051920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.537+0000 7f149a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f148c0e7be0 con 0x7f148c0548e0 2024-05-30T16:56:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.537+0000 7f149a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f148c04a5e0 con 0x7f148c038ea0 2024-05-30T16:56:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.537+0000 7f149a800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f148c0eb3f0 con 0x7f148c0e7800 2024-05-30T16:56:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.536+0000 7f1492a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f148c0548e0 0x7f148c038790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:46994/0 (socket says 172.21.15.67:46994) 2024-05-30T16:56:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.537+0000 7f1492a00640 1 -- 172.21.15.67:0/4025291731 learned_addr learned my addr 172.21.15.67:0/4025291731 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.538+0000 7f1492000640 1 -- 172.21.15.67:0/4025291731 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245766713 0 0) 0x7f1488001110 con 0x7f148c0548e0 2024-05-30T16:56:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.539+0000 7f1492000640 1 -- 172.21.15.67:0/4025291731 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f147c003520 con 0x7f148c0548e0 2024-05-30T16:56:34.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.539+0000 7f1493400640 1 -- 172.21.15.67:0/4025291731 >> v2:172.21.15.67:3300/0 conn(0x7f148c038ea0 msgr2=0x7f148c051920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:34.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.539+0000 7f1493e00640 1 --2- 172.21.15.67:0/4025291731 >> v2:172.21.15.67:3300/0 conn(0x7f148c038ea0 0x7f148c051920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.540+0000 7f1493400640 1 --2- 172.21.15.67:0/4025291731 >> v2:172.21.15.67:3300/0 conn(0x7f148c038ea0 0x7f148c051920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.540+0000 7f1493400640 1 -- 172.21.15.67:0/4025291731 >> v1:172.21.15.169:6789/0 conn(0x7f148c0548e0 legacy=0x7f148c038790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.540+0000 7f1493400640 1 -- 172.21.15.67:0/4025291731 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c0eb6d0 con 0x7f148c0e7800 2024-05-30T16:56:34.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.541+0000 7f1493e00640 1 --2- 172.21.15.67:0/4025291731 >> v2:172.21.15.67:3300/0 conn(0x7f148c038ea0 0x7f148c051920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.544+0000 7f1493400640 1 --2- 172.21.15.67:0/4025291731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 0x7f148c006f80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f148c00ed60 tx=0x7f148002a120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b698ec4d0b4be4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:34.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.545+0000 7f1492000640 1 -- 172.21.15.67:0/4025291731 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480037070 con 0x7f148c0e7800 2024-05-30T16:56:34.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.546+0000 7f1492000640 1 -- 172.21.15.67:0/4025291731 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1480032440 con 0x7f148c0e7800 2024-05-30T16:56:34.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.546+0000 7f1492000640 1 -- 172.21.15.67:0/4025291731 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480032810 con 0x7f148c0e7800 2024-05-30T16:56:34.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.554+0000 7f149a800640 1 -- 172.21.15.67:0/4025291731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 msgr2=0x7f148c006f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:34.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.554+0000 7f149a800640 1 --2- 172.21.15.67:0/4025291731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 0x7f148c006f80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f148c00ed60 tx=0x7f148002a120 comp rx=0 tx=0).stop 2024-05-30T16:56:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.556+0000 7f149a800640 1 -- 172.21.15.67:0/4025291731 shutdown_connections 2024-05-30T16:56:34.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.556+0000 7f149a800640 1 --2- 172.21.15.67:0/4025291731 >> v2:172.21.15.67:3300/0 conn(0x7f148c038ea0 0x7f148c051920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:34.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.556+0000 7f149a800640 1 --2- 172.21.15.67:0/4025291731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 0x7f148c006f80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:34.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.556+0000 7f149a800640 1 -- 172.21.15.67:0/4025291731 >> 172.21.15.67:0/4025291731 conn(0x7f148c053790 msgr2=0x7f148c053ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:34.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.559+0000 7f149a800640 1 -- 172.21.15.67:0/4025291731 shutdown_connections 2024-05-30T16:56:34.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.560+0000 7f149a800640 1 -- 172.21.15.67:0/4025291731 wait complete. 2024-05-30T16:56:34.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.566+0000 7f149a800640 1 Processor -- start 2024-05-30T16:56:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.566+0000 7f149a800640 1 -- start start 2024-05-30T16:56:34.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.569+0000 7f149a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148c038ea0 0x7f148c050d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.570+0000 7f149a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f148c0548e0 0x7f148c03c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:34.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.570+0000 7f149a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 0x7f148c0de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:34.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.574+0000 7f149a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f148c0ebf70 con 0x7f148c0548e0 2024-05-30T16:56:34.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.574+0000 7f149a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f148c0ebc90 con 0x7f148c0e7800 2024-05-30T16:56:34.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.574+0000 7f149a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f148c051e60 con 0x7f148c038ea0 2024-05-30T16:56:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.575+0000 7f1492a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f148c0548e0 0x7f148c03c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.575+0000 7f1493400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148c038ea0 0x7f148c050d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.575+0000 7f1493400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148c038ea0 0x7f148c050d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:39944/0 (socket says 172.21.15.67:39944) 2024-05-30T16:56:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.575+0000 7f1492a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f148c0548e0 0x7f148c03c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37140/0 (socket says 172.21.15.67:37140) 2024-05-30T16:56:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.575+0000 7f1492a00640 1 -- 172.21.15.67:0/518773784 learned_addr learned my addr 172.21.15.67:0/518773784 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:34.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.578+0000 7f1493e00640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 0x7f148c0de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:34.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.581+0000 7f1493400640 1 -- 172.21.15.67:0/518773784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 msgr2=0x7f148c0de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:34.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.581+0000 7f1493400640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 0x7f148c0de5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:34.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.581+0000 7f1493400640 1 -- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f148c0548e0 msgr2=0x7f148c03c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:34.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.581+0000 7f1493400640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f148c0548e0 0x7f148c03c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:34.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.581+0000 7f1493400640 1 -- 172.21.15.67:0/518773784 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c0e35a0 con 0x7f148c038ea0 2024-05-30T16:56:34.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.581+0000 7f1492a00640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f148c0548e0 0x7f148c03c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:34.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.583+0000 7f1493e00640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 0x7f148c0de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:34.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.587+0000 7f1493400640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148c038ea0 0x7f148c050d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1480031000 tx=0x7f14800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:34.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.588+0000 7f1487400640 1 -- 172.21.15.67:0/518773784 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480042020 con 0x7f148c038ea0 2024-05-30T16:56:34.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.588+0000 7f1487400640 1 -- 172.21.15.67:0/518773784 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148003d730 con 0x7f148c038ea0 2024-05-30T16:56:34.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.588+0000 7f1487400640 1 -- 172.21.15.67:0/518773784 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480037040 con 0x7f148c038ea0 2024-05-30T16:56:34.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.588+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148c0e3880 con 0x7f148c038ea0 2024-05-30T16:56:34.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.589+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f148c0e3e40 con 0x7f148c038ea0 2024-05-30T16:56:34.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.592+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148c006500 con 0x7f148c038ea0 2024-05-30T16:56:34.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.596+0000 7f1487400640 1 -- 172.21.15.67:0/518773784 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 29) ==== 92519+0+0 (secure 0 0 0) 0x7f1480045020 con 0x7f148c038ea0 2024-05-30T16:56:34.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.602+0000 7f1487400640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f147406d990 0x7f147406fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:34.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.602+0000 7f1487400640 1 -- 172.21.15.67:0/518773784 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f14800ab440 con 0x7f148c038ea0 2024-05-30T16:56:34.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.619+0000 7f1487400640 1 -- 172.21.15.67:0/518773784 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f148004c030 con 0x7f148c038ea0 2024-05-30T16:56:34.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.638+0000 7f1492a00640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f147406d990 0x7f147406fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:34.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:34.692+0000 7f1492a00640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f147406d990 0x7f147406fe50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f148c03ca60 tx=0x7f1488007bd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:35.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.237+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922818", "target": ["mon-mgr", ""]}) -- 0x7f148c0e4010 con 0x7f147406d990 2024-05-30T16:56:35.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.935+0000 7f1487400640 1 -- 172.21.15.67:0/518773784 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f148c0e4010 con 0x7f147406d990 2024-05-30T16:56:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.940+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f147406d990 msgr2=0x7f147406fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.940+0000 7f149a800640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f147406d990 0x7f147406fe50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f148c03ca60 tx=0x7f1488007bd0 comp rx=0 tx=0).stop 2024-05-30T16:56:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.940+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148c038ea0 msgr2=0x7f148c050d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:35.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.940+0000 7f149a800640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148c038ea0 0x7f148c050d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1480031000 tx=0x7f14800046c0 comp rx=0 tx=0).stop 2024-05-30T16:56:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.945+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 shutdown_connections 2024-05-30T16:56:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.945+0000 7f149a800640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f147406d990 0x7f147406fe50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.945+0000 7f149a800640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f148c0e7800 0x7f148c0de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.945+0000 7f149a800640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f148c0548e0 0x7f148c03c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.945+0000 7f149a800640 1 --2- 172.21.15.67:0/518773784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148c038ea0 0x7f148c050d00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.945+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 >> 172.21.15.67:0/518773784 conn(0x7f148c053790 msgr2=0x7f148c0080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.946+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 shutdown_connections 2024-05-30T16:56:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:35.946+0000 7f149a800640 1 -- 172.21.15.67:0/518773784 wait complete. 2024-05-30T16:56:36.004 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922819 2024-05-30T16:56:36.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.466+0000 7f6f0de00640 1 Processor -- start 2024-05-30T16:56:36.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.466+0000 7f6f0de00640 1 -- start start 2024-05-30T16:56:36.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.469+0000 7f6f0de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f08162cf0 0x7f6f080b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:36.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.469+0000 7f6f0de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08073740 0x7f6f080ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:36.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.469+0000 7f6f0de00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f080b1dc0 con 0x7f6f08099b50 2024-05-30T16:56:36.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.469+0000 7f6f0de00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6f080b3d70 con 0x7f6f08162cf0 2024-05-30T16:56:36.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.469+0000 7f6f0de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f080b0da0 con 0x7f6f08073740 2024-05-30T16:56:36.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.470+0000 7f6f06a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f08162cf0 0x7f6f080b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:36.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.470+0000 7f6f06000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f6f08099b50 0x7f6f08073030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:47006/0 (socket says 172.21.15.67:47006) 2024-05-30T16:56:36.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.470+0000 7f6f06000640 1 -- 172.21.15.67:0/25280408 learned_addr learned my addr 172.21.15.67:0/25280408 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:36.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.470+0000 7f6f07400640 1 --2- 172.21.15.67:0/25280408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08073740 0x7f6f080ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:36.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.471+0000 7f6f07400640 1 -- 172.21.15.67:0/25280408 >> v2:172.21.15.169:3300/0 conn(0x7f6f08162cf0 msgr2=0x7f6f080b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:36.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.471+0000 7f6f07400640 1 --2- 172.21.15.67:0/25280408 >> v2:172.21.15.169:3300/0 conn(0x7f6f08162cf0 0x7f6f080b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:36.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.471+0000 7f6f07400640 1 -- 172.21.15.67:0/25280408 >> v1:172.21.15.169:6789/0 conn(0x7f6f08099b50 legacy=0x7f6f08073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:36.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.472+0000 7f6f07400640 1 -- 172.21.15.67:0/25280408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f080b1080 con 0x7f6f08073740 2024-05-30T16:56:36.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.481+0000 7f6f07400640 1 --2- 172.21.15.67:0/25280408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08073740 0x7f6f080ad2a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6f000097a0 tx=0x7f6f0002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6b1e4d617891492 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:36.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.482+0000 7f6f05600640 1 -- 172.21.15.67:0/25280408 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00037070 con 0x7f6f08073740 2024-05-30T16:56:36.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.482+0000 7f6f05600640 1 -- 172.21.15.67:0/25280408 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0002ad10 con 0x7f6f08073740 2024-05-30T16:56:36.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.482+0000 7f6f05600640 1 -- 172.21.15.67:0/25280408 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00032660 con 0x7f6f08073740 2024-05-30T16:56:36.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.484+0000 7f6f0de00640 1 -- 172.21.15.67:0/25280408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08073740 msgr2=0x7f6f080ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:36.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.484+0000 7f6f0de00640 1 --2- 172.21.15.67:0/25280408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08073740 0x7f6f080ad2a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6f000097a0 tx=0x7f6f0002a080 comp rx=0 tx=0).stop 2024-05-30T16:56:36.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.484+0000 7f6f0de00640 1 -- 172.21.15.67:0/25280408 shutdown_connections 2024-05-30T16:56:36.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.485+0000 7f6f0de00640 1 --2- 172.21.15.67:0/25280408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08073740 0x7f6f080ad2a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:36.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.485+0000 7f6f0de00640 1 --2- 172.21.15.67:0/25280408 >> v2:172.21.15.169:3300/0 conn(0x7f6f08162cf0 0x7f6f080b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:36.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.485+0000 7f6f0de00640 1 -- 172.21.15.67:0/25280408 >> 172.21.15.67:0/25280408 conn(0x7f6f080b1fc0 msgr2=0x7f6f080b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:36.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.486+0000 7f6f0de00640 1 -- 172.21.15.67:0/25280408 shutdown_connections 2024-05-30T16:56:36.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.487+0000 7f6f0de00640 1 -- 172.21.15.67:0/25280408 wait complete. 2024-05-30T16:56:36.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.501+0000 7f6f0de00640 1 Processor -- start 2024-05-30T16:56:36.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.501+0000 7f6f0de00640 1 -- start start 2024-05-30T16:56:36.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.507+0000 7f6f0de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f08073740 0x7f6f0814e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:36.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.507+0000 7f6f0de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08099b50 0x7f6f0814af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:36.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.508+0000 7f6f0de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f08162cf0 0x7f6f081612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:36.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.513+0000 7f6f06a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f08073740 0x7f6f0814e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:36.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.513+0000 7f6f0de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081630d0 con 0x7f6f08162cf0 2024-05-30T16:56:36.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.513+0000 7f6f0de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f080b4050 con 0x7f6f08073740 2024-05-30T16:56:36.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.513+0000 7f6f0de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f08163250 con 0x7f6f08099b50 2024-05-30T16:56:36.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.514+0000 7f6f07400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f08162cf0 0x7f6f081612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:36.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.514+0000 7f6f07400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f08162cf0 0x7f6f081612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37144/0 (socket says 172.21.15.67:37144) 2024-05-30T16:56:36.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.514+0000 7f6f07400640 1 -- 172.21.15.67:0/2167224850 learned_addr learned my addr 172.21.15.67:0/2167224850 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:36.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.514+0000 7f6f06000640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08099b50 0x7f6f0814af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:36.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.515+0000 7f6f06000640 1 -- 172.21.15.67:0/2167224850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f08073740 msgr2=0x7f6f0814e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:36.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.515+0000 7f6f06000640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f08073740 0x7f6f0814e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:36.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.515+0000 7f6f06000640 1 -- 172.21.15.67:0/2167224850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f08162cf0 msgr2=0x7f6f081612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:36.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.515+0000 7f6f06000640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f08162cf0 0x7f6f081612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:36.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.515+0000 7f6f06000640 1 -- 172.21.15.67:0/2167224850 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0814b9e0 con 0x7f6f08099b50 2024-05-30T16:56:36.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.518+0000 7f6f06000640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08099b50 0x7f6f0814af90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800cac0 tx=0x7f6ef800cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:36.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.520+0000 7f6eff400640 1 -- 172.21.15.67:0/2167224850 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8018070 con 0x7f6f08099b50 2024-05-30T16:56:36.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.520+0000 7f6eff400640 1 -- 172.21.15.67:0/2167224850 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef800a640 con 0x7f6f08099b50 2024-05-30T16:56:36.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.520+0000 7f6eff400640 1 -- 172.21.15.67:0/2167224850 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800aa30 con 0x7f6f08099b50 2024-05-30T16:56:36.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.520+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0814bcc0 con 0x7f6f08099b50 2024-05-30T16:56:36.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.520+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0814ee70 con 0x7f6f08099b50 2024-05-30T16:56:36.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.523+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee0005290 con 0x7f6f08099b50 2024-05-30T16:56:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.528+0000 7f6eff400640 1 -- 172.21.15.67:0/2167224850 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 29) ==== 92519+0+0 (secure 0 0 0) 0x7f6ef801b070 con 0x7f6f08099b50 2024-05-30T16:56:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.529+0000 7f6eff400640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee406da10 0x7f6ee406fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:36.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.531+0000 7f6eff400640 1 -- 172.21.15.67:0/2167224850 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f6ef808a160 con 0x7f6f08099b50 2024-05-30T16:56:36.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.537+0000 7f6eff400640 1 -- 172.21.15.67:0/2167224850 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ef8059670 con 0x7f6f08099b50 2024-05-30T16:56:36.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.565+0000 7f6f06a00640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee406da10 0x7f6ee406fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:36.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:36.636+0000 7f6f06a00640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee406da10 0x7f6ee406fed0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6f081626a0 tx=0x7f6ef0009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:37.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.194+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922819", "target": ["mon-mgr", ""]}) -- 0x7f6ee0000c80 con 0x7f6ee406da10 2024-05-30T16:56:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.982+0000 7f6eff400640 1 -- 172.21.15.67:0/2167224850 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6ee0000c80 con 0x7f6ee406da10 2024-05-30T16:56:37.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.991+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee406da10 msgr2=0x7f6ee406fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:37.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.991+0000 7f6f0de00640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee406da10 0x7f6ee406fed0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6f081626a0 tx=0x7f6ef0009190 comp rx=0 tx=0).stop 2024-05-30T16:56:37.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.991+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08099b50 msgr2=0x7f6f0814af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:37.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.991+0000 7f6f0de00640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08099b50 0x7f6f0814af90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800cac0 tx=0x7f6ef800cf80 comp rx=0 tx=0).stop 2024-05-30T16:56:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.993+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 shutdown_connections 2024-05-30T16:56:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.993+0000 7f6f0de00640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ee406da10 0x7f6ee406fed0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:37.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.993+0000 7f6f0de00640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f08162cf0 0x7f6f081612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:37.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.993+0000 7f6f0de00640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f08099b50 0x7f6f0814af90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:37.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.993+0000 7f6f0de00640 1 --2- 172.21.15.67:0/2167224850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f08073740 0x7f6f0814e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:37.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.994+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 >> 172.21.15.67:0/2167224850 conn(0x7f6f080b1fc0 msgr2=0x7f6f080aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:37.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.995+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 shutdown_connections 2024-05-30T16:56:37.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:37.995+0000 7f6f0de00640 1 -- 172.21.15.67:0/2167224850 wait complete. 2024-05-30T16:56:38.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone922820 2024-05-30T16:56:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.419+0000 7f1b0e200640 1 Processor -- start 2024-05-30T16:56:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.419+0000 7f1b0e200640 1 -- start start 2024-05-30T16:56:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.421+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 0x7f1b081633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.421+0000 7f1b0e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1b080b4120 0x7f1b081638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.421+0000 7f1b0e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b08163e30 con 0x7f1b08162ac0 2024-05-30T16:56:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.421+0000 7f1b0e200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1b080b3c90 con 0x7f1b080b4120 2024-05-30T16:56:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.421+0000 7f1b0e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1b081640e0 con 0x7f1b080d0d50 2024-05-30T16:56:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.421+0000 7f1b07400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f1b08162ac0 0x7f1b080d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:47012/0 (socket says 172.21.15.67:47012) 2024-05-30T16:56:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.421+0000 7f1b07400640 1 -- 172.21.15.67:0/2207671203 learned_addr learned my addr 172.21.15.67:0/2207671203 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.421+0000 7f1b07e00640 1 --2- 172.21.15.67:0/2207671203 >> v2:172.21.15.169:3300/0 conn(0x7f1b080b4120 0x7f1b081638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.421+0000 7f1b06a00640 1 --2- 172.21.15.67:0/2207671203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 0x7f1b081633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.422+0000 7f1b06000640 1 -- 172.21.15.67:0/2207671203 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2378290213 0 0) 0x7f1b08163e30 con 0x7f1b08162ac0 2024-05-30T16:56:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.422+0000 7f1b07e00640 1 -- 172.21.15.67:0/2207671203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 msgr2=0x7f1b081633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.422+0000 7f1b07e00640 1 --2- 172.21.15.67:0/2207671203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 0x7f1b081633b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.422+0000 7f1b07e00640 1 -- 172.21.15.67:0/2207671203 >> v1:172.21.15.169:6789/0 conn(0x7f1b08162ac0 legacy=0x7f1b080d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.422+0000 7f1b07e00640 1 -- 172.21.15.67:0/2207671203 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b080b3360 con 0x7f1b080b4120 2024-05-30T16:56:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.422+0000 7f1b06a00640 1 --2- 172.21.15.67:0/2207671203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 0x7f1b081633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:56:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.423+0000 7f1b07e00640 1 --2- 172.21.15.67:0/2207671203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b080b4120 0x7f1b081638f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009d90 tx=0x7f1b0002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b501142de34da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.423+0000 7f1b06000640 1 -- 172.21.15.67:0/2207671203 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00007740 con 0x7f1b080b4120 2024-05-30T16:56:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.423+0000 7f1b06000640 1 -- 172.21.15.67:0/2207671203 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b000078e0 con 0x7f1b080b4120 2024-05-30T16:56:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.423+0000 7f1b06000640 1 -- 172.21.15.67:0/2207671203 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00007cb0 con 0x7f1b080b4120 2024-05-30T16:56:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.424+0000 7f1b0e200640 1 -- 172.21.15.67:0/2207671203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b080b4120 msgr2=0x7f1b081638f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.424+0000 7f1b0e200640 1 --2- 172.21.15.67:0/2207671203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b080b4120 0x7f1b081638f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009d90 tx=0x7f1b0002a660 comp rx=0 tx=0).stop 2024-05-30T16:56:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.424+0000 7f1b0e200640 1 -- 172.21.15.67:0/2207671203 shutdown_connections 2024-05-30T16:56:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.424+0000 7f1b0e200640 1 --2- 172.21.15.67:0/2207671203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b080b4120 0x7f1b081638f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.424+0000 7f1b0e200640 1 --2- 172.21.15.67:0/2207671203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 0x7f1b081633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.424+0000 7f1b0e200640 1 -- 172.21.15.67:0/2207671203 >> 172.21.15.67:0/2207671203 conn(0x7f1b0806bd20 msgr2=0x7f1b0806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.424+0000 7f1b0e200640 1 -- 172.21.15.67:0/2207671203 shutdown_connections 2024-05-30T16:56:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.424+0000 7f1b0e200640 1 -- 172.21.15.67:0/2207671203 wait complete. 2024-05-30T16:56:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.427+0000 7f1b0e200640 1 Processor -- start 2024-05-30T16:56:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.427+0000 7f1b0e200640 1 -- start start 2024-05-30T16:56:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.428+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b080b4120 0x7f1b08161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.428+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 0x7f1b0809be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.428+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b08162ac0 0x7f1b080ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.428+0000 7f1b0e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1b08162ea0 con 0x7f1b080b4120 2024-05-30T16:56:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.428+0000 7f1b0e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080b3920 con 0x7f1b08162ac0 2024-05-30T16:56:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.428+0000 7f1b0e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1b08163020 con 0x7f1b080d0d50 2024-05-30T16:56:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.428+0000 7f1b07400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b080b4120 0x7f1b08161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.428+0000 7f1b07400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b080b4120 0x7f1b08161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37150/0 (socket says 172.21.15.67:37150) 2024-05-30T16:56:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.428+0000 7f1b07400640 1 -- 172.21.15.67:0/187815450 learned_addr learned my addr 172.21.15.67:0/187815450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.429+0000 7f1b06a00640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 0x7f1b0809be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.429+0000 7f1b07e00640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b08162ac0 0x7f1b080ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.429+0000 7f1b07400640 1 -- 172.21.15.67:0/187815450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 msgr2=0x7f1b0809be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.429+0000 7f1b07400640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 0x7f1b0809be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.429+0000 7f1b07400640 1 -- 172.21.15.67:0/187815450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b08162ac0 msgr2=0x7f1b080ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.429+0000 7f1b07400640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b08162ac0 0x7f1b080ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.429+0000 7f1b07400640 1 -- 172.21.15.67:0/187815450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0809ad60 con 0x7f1b080b4120 2024-05-30T16:56:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.429+0000 7f1b07400640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b080b4120 0x7f1b08161620 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c430 tx=0x7f1af800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.430+0000 7f1aff400640 1 -- 172.21.15.67:0/187815450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af80074e0 con 0x7f1b080b4120 2024-05-30T16:56:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.430+0000 7f1aff400640 1 -- 172.21.15.67:0/187815450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af80047e0 con 0x7f1b080b4120 2024-05-30T16:56:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.430+0000 7f1aff400640 1 -- 172.21.15.67:0/187815450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8004bd0 con 0x7f1b080b4120 2024-05-30T16:56:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.430+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0809b040 con 0x7f1b080b4120 2024-05-30T16:56:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.430+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0809b600 con 0x7f1b080b4120 2024-05-30T16:56:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.432+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b080689e0 con 0x7f1b080b4120 2024-05-30T16:56:38.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.434+0000 7f1aff400640 1 -- 172.21.15.67:0/187815450 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 29) ==== 92519+0+0 (secure 0 0 0) 0x7f1af8011070 con 0x7f1b080b4120 2024-05-30T16:56:38.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.435+0000 7f1aff400640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae806d970 0x7f1ae806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:38.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.435+0000 7f1aff400640 1 -- 172.21.15.67:0/187815450 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f1af808a480 con 0x7f1b080b4120 2024-05-30T16:56:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.435+0000 7f1b06a00640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae806d970 0x7f1ae806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.436+0000 7f1b06a00640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae806d970 0x7f1ae806fe30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1b0809a540 tx=0x7f1af4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:38.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.442+0000 7f1aff400640 1 -- 172.21.15.67:0/187815450 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1af80599f0 con 0x7f1b080b4120 2024-05-30T16:56:38.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:38.895+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone922820", "target": ["mon-mgr", ""]}) -- 0x7f1b0806e9f0 con 0x7f1ae806d970 2024-05-30T16:56:39.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.247+0000 7f1aff400640 1 -- 172.21.15.67:0/187815450 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1b0806e9f0 con 0x7f1ae806d970 2024-05-30T16:56:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae806d970 msgr2=0x7f1ae806fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae806d970 0x7f1ae806fe30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1b0809a540 tx=0x7f1af4008040 comp rx=0 tx=0).stop 2024-05-30T16:56:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b080b4120 msgr2=0x7f1b08161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b080b4120 0x7f1b08161620 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c430 tx=0x7f1af800c8f0 comp rx=0 tx=0).stop 2024-05-30T16:56:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 shutdown_connections 2024-05-30T16:56:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ae806d970 0x7f1ae806fe30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b08162ac0 0x7f1b080ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b080d0d50 0x7f1b0809be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:39.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 --2- 172.21.15.67:0/187815450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b080b4120 0x7f1b08161620 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 >> 172.21.15.67:0/187815450 conn(0x7f1b0806bd20 msgr2=0x7f1b080d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 shutdown_connections 2024-05-30T16:56:39.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:39.250+0000 7f1b0e200640 1 -- 172.21.15.67:0/187815450 wait complete. 2024-05-30T16:56:41.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:56:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.760+0000 7f2546800640 1 Processor -- start 2024-05-30T16:56:41.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.760+0000 7f2546800640 1 -- start start 2024-05-30T16:56:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.762+0000 7f2546800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254015e9e0 0x7f25400ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.763+0000 7f2546800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25400ae1f0 con 0x7f25400d0970 2024-05-30T16:56:41.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.763+0000 7f2546800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25400ae4d0 con 0x7f25400afa30 2024-05-30T16:56:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.763+0000 7f2546800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25400aee00 con 0x7f254015e9e0 2024-05-30T16:56:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.763+0000 7f253fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f25400d0970 0x7f254006e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42710/0 (socket says 172.21.15.67:42710) 2024-05-30T16:56:41.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.763+0000 7f253fe00640 1 -- 172.21.15.67:0/2448672522 learned_addr learned my addr 172.21.15.67:0/2448672522 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.764+0000 7f253f400640 1 --2- 172.21.15.67:0/2448672522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254015e9e0 0x7f25400ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.765+0000 7f253f400640 1 -- 172.21.15.67:0/2448672522 >> v1:172.21.15.169:6789/0 conn(0x7f25400afa30 legacy=0x7f25400d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.765+0000 7f253f400640 1 -- 172.21.15.67:0/2448672522 >> v1:172.21.15.67:6789/0 conn(0x7f25400d0970 legacy=0x7f254006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:41.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.765+0000 7f253f400640 1 -- 172.21.15.67:0/2448672522 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25400ae7b0 con 0x7f254015e9e0 2024-05-30T16:56:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.770+0000 7f253f400640 1 --2- 172.21.15.67:0/2448672522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254015e9e0 0x7f25400ad950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f252c0097d0 tx=0x7f252c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc50b3745b627e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.771+0000 7f253e000640 1 -- 172.21.15.67:0/2448672522 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c037070 con 0x7f254015e9e0 2024-05-30T16:56:41.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.771+0000 7f253e000640 1 -- 172.21.15.67:0/2448672522 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f252c032440 con 0x7f254015e9e0 2024-05-30T16:56:41.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.772+0000 7f253e000640 1 -- 172.21.15.67:0/2448672522 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c032850 con 0x7f254015e9e0 2024-05-30T16:56:41.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.776+0000 7f2546800640 1 -- 172.21.15.67:0/2448672522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254015e9e0 msgr2=0x7f25400ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:41.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.776+0000 7f2546800640 1 --2- 172.21.15.67:0/2448672522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254015e9e0 0x7f25400ad950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f252c0097d0 tx=0x7f252c02a270 comp rx=0 tx=0).stop 2024-05-30T16:56:41.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.779+0000 7f2546800640 1 -- 172.21.15.67:0/2448672522 shutdown_connections 2024-05-30T16:56:41.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.779+0000 7f2546800640 1 --2- 172.21.15.67:0/2448672522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254015e9e0 0x7f25400ad950 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:41.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.779+0000 7f2546800640 1 -- 172.21.15.67:0/2448672522 >> 172.21.15.67:0/2448672522 conn(0x7f2540067ce0 msgr2=0x7f254006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.780+0000 7f2546800640 1 -- 172.21.15.67:0/2448672522 shutdown_connections 2024-05-30T16:56:41.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.780+0000 7f2546800640 1 -- 172.21.15.67:0/2448672522 wait complete. 2024-05-30T16:56:41.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.783+0000 7f2546800640 1 Processor -- start 2024-05-30T16:56:41.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.784+0000 7f2546800640 1 -- start start 2024-05-30T16:56:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.786+0000 7f2546800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25400afa30 0x7f2540154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.786+0000 7f2546800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25400d0970 0x7f25401f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.786+0000 7f2546800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254015e9e0 0x7f25401f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.786+0000 7f2546800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f254006ed50 con 0x7f25400d0970 2024-05-30T16:56:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.786+0000 7f2546800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25400ade90 con 0x7f254015e9e0 2024-05-30T16:56:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.786+0000 7f2546800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f254006eed0 con 0x7f25400afa30 2024-05-30T16:56:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.787+0000 7f253f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25400afa30 0x7f2540154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.787+0000 7f253f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25400afa30 0x7f2540154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:59652/0 (socket says 172.21.15.67:59652) 2024-05-30T16:56:41.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.787+0000 7f253f400640 1 -- 172.21.15.67:0/1340622610 learned_addr learned my addr 172.21.15.67:0/1340622610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.787+0000 7f253fe00640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254015e9e0 0x7f25401f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.787+0000 7f253ea00640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25400d0970 0x7f25401f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.787+0000 7f253f400640 1 -- 172.21.15.67:0/1340622610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254015e9e0 msgr2=0x7f25401f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.787+0000 7f253f400640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254015e9e0 0x7f25401f8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.788+0000 7f253f400640 1 -- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25400d0970 msgr2=0x7f25401f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.788+0000 7f253f400640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25400d0970 0x7f25401f7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.788+0000 7f253f400640 1 -- 172.21.15.67:0/1340622610 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254014d120 con 0x7f25400afa30 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.788+0000 7f253fe00640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254015e9e0 0x7f25401f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.788+0000 7f253f400640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25400afa30 0x7f2540154620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f252c0097d0 tx=0x7f252c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:41.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.789+0000 7f2537400640 1 -- 172.21.15.67:0/1340622610 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c042020 con 0x7f25400afa30 2024-05-30T16:56:41.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.789+0000 7f2537400640 1 -- 172.21.15.67:0/1340622610 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f252c030910 con 0x7f25400afa30 2024-05-30T16:56:41.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.789+0000 7f2537400640 1 -- 172.21.15.67:0/1340622610 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c037040 con 0x7f25400afa30 2024-05-30T16:56:41.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.789+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2540154c20 con 0x7f25400afa30 2024-05-30T16:56:41.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.789+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f25401551e0 con 0x7f25400afa30 2024-05-30T16:56:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.793+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2518005290 con 0x7f25400afa30 2024-05-30T16:56:41.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.793+0000 7f2537400640 1 -- 172.21.15.67:0/1340622610 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 29) ==== 92519+0+0 (secure 0 0 0) 0x7f252c045020 con 0x7f25400afa30 2024-05-30T16:56:41.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.800+0000 7f2537400640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f252006d940 0x7f252006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:41.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.801+0000 7f2537400640 1 -- 172.21.15.67:0/1340622610 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f252c0ab230 con 0x7f25400afa30 2024-05-30T16:56:41.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.803+0000 7f253ea00640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f252006d940 0x7f252006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:41.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.805+0000 7f253ea00640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f252006d940 0x7f252006fe00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f25400cd100 tx=0x7f2530009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:41.805+0000 7f2537400640 1 -- 172.21.15.67:0/1340622610 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f252c04c070 con 0x7f25400afa30 2024-05-30T16:56:42.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.436+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2518005850 con 0x7f25400afa30 2024-05-30T16:56:42.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.438+0000 7f2537400640 1 -- 172.21.15.67:0/1340622610 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2386 (secure 0 0 0) 0x7f254014d120 con 0x7f25400afa30 2024-05-30T16:56:42.444 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:56:42.444 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:56:42.444 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:56:42.444 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:56:42.444 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:56:42.444 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:56:42.444 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 795, 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:56:42.445 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:56:42.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 6806, 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 28326864732, 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 56216186880, 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 716877926400, 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 687533744, 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 703281290, 2024-05-30T16:56:42.447 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 166, 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 183 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:56:42.448 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:56:42.449 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "7c6f9121-d563-4406-bd90-584b3290d1b2": { 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "13 ongoing+pending clones; Avg progress is 23.077% (32s)\n [======......................] (remaining: 108s)", 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.23077000677585602, 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 0.0% (38s)\n [............................] ", 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0, 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:56:42.450 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:42.451 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:56:42.451 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:56:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.458+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f252006d940 msgr2=0x7f252006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.458+0000 7f2546800640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f252006d940 0x7f252006fe00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f25400cd100 tx=0x7f2530009040 comp rx=0 tx=0).stop 2024-05-30T16:56:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.458+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25400afa30 msgr2=0x7f2540154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.458+0000 7f2546800640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25400afa30 0x7f2540154620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f252c0097d0 tx=0x7f252c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:56:42.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.461+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 shutdown_connections 2024-05-30T16:56:42.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.461+0000 7f2546800640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f252006d940 0x7f252006fe00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:42.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.461+0000 7f2546800640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f254015e9e0 0x7f25401f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:42.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.461+0000 7f2546800640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25400d0970 0x7f25401f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:42.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.461+0000 7f2546800640 1 --2- 172.21.15.67:0/1340622610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25400afa30 0x7f2540154620 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:42.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.461+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 >> 172.21.15.67:0/1340622610 conn(0x7f2540067ce0 msgr2=0x7f25400b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.465+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 shutdown_connections 2024-05-30T16:56:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:42.465+0000 7f2546800640 1 -- 172.21.15.67:0/1340622610 wait complete. 2024-05-30T16:56:42.516 INFO:tasks.cephfs.test_volumes:pevs - {'7c6f9121-d563-4406-bd90-584b3290d1b2': {'message': '13 ongoing+pending clones; Avg progress is 23.077% (32s)\n [======......................] (remaining: 108s)', 'progress': 0.23077000677585602, 'add_to_ceph_s': True}, 'e0a834ca-3f6a-4236-843d-35f9b733ad76': {'message': '4 ongoing clones; Avg progress is 0.0% (38s)\n [............................] ', 'progress': 0, 'add_to_ceph_s': True}} 2024-05-30T16:56:42.516 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-05-30T16:56:42.516 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.23077000677585602; onpen_p = 0.35780999064445496 2024-05-30T16:56:42.516 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92281 2024-05-30T16:56:43.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.046+0000 7f871f400640 1 Processor -- start 2024-05-30T16:56:43.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.047+0000 7f871f400640 1 -- start start 2024-05-30T16:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.051+0000 7f871f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87180b3330 0x7f8718073a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.051+0000 7f871f400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871815bf10 con 0x7f87180b3fb0 2024-05-30T16:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.051+0000 7f871f400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87180cc920 con 0x7f871815ab20 2024-05-30T16:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.051+0000 7f871f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f87180b17c0 con 0x7f87180b3330 2024-05-30T16:56:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.053+0000 7f871d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87180b3330 0x7f8718073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.053+0000 7f871d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87180b3330 0x7f8718073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:59658/0 (socket says 172.21.15.67:59658) 2024-05-30T16:56:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.053+0000 7f871d400640 1 -- 172.21.15.67:0/4235506465 learned_addr learned my addr 172.21.15.67:0/4235506465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.054+0000 7f871d400640 1 -- 172.21.15.67:0/4235506465 >> v1:172.21.15.169:6789/0 conn(0x7f871815ab20 legacy=0x7f87180b38a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:56:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.054+0000 7f871d400640 1 -- 172.21.15.67:0/4235506465 >> v1:172.21.15.67:6789/0 conn(0x7f87180b3fb0 legacy=0x7f871815c6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:56:43.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.054+0000 7f871d400640 1 -- 172.21.15.67:0/4235506465 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87180b1aa0 con 0x7f87180b3330 2024-05-30T16:56:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.057+0000 7f871d400640 1 --2- 172.21.15.67:0/4235506465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87180b3330 0x7f8718073a40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8718070ce0 tx=0x7f870802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a920d6aab16614f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:43.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.061+0000 7f8717400640 1 -- 172.21.15.67:0/4235506465 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708037070 con 0x7f87180b3330 2024-05-30T16:56:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.061+0000 7f8717400640 1 -- 172.21.15.67:0/4235506465 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8708032440 con 0x7f87180b3330 2024-05-30T16:56:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.063+0000 7f871f400640 1 -- 172.21.15.67:0/4235506465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87180b3330 msgr2=0x7f8718073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.063+0000 7f8717400640 1 -- 172.21.15.67:0/4235506465 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708037070 con 0x7f87180b3330 2024-05-30T16:56:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.063+0000 7f871f400640 1 --2- 172.21.15.67:0/4235506465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87180b3330 0x7f8718073a40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8718070ce0 tx=0x7f870802a230 comp rx=0 tx=0).stop 2024-05-30T16:56:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.063+0000 7f871f400640 1 -- 172.21.15.67:0/4235506465 shutdown_connections 2024-05-30T16:56:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.063+0000 7f871f400640 1 --2- 172.21.15.67:0/4235506465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87180b3330 0x7f8718073a40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:43.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.064+0000 7f871f400640 1 -- 172.21.15.67:0/4235506465 >> 172.21.15.67:0/4235506465 conn(0x7f87180b1fc0 msgr2=0x7f87180b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.065+0000 7f871f400640 1 -- 172.21.15.67:0/4235506465 shutdown_connections 2024-05-30T16:56:43.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.065+0000 7f871f400640 1 -- 172.21.15.67:0/4235506465 wait complete. 2024-05-30T16:56:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.070+0000 7f871f400640 1 Processor -- start 2024-05-30T16:56:43.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.070+0000 7f871f400640 1 -- start start 2024-05-30T16:56:43.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.072+0000 7f871f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87180b3330 0x7f8718146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:43.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.072+0000 7f871f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87180b3fb0 0x7f87180ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:43.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.072+0000 7f871f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f871815ab20 0x7f8718147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:43.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.072+0000 7f871f400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f87180cc540 con 0x7f87180b3fb0 2024-05-30T16:56:43.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.072+0000 7f871f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f871815c1b0 con 0x7f87180b3330 2024-05-30T16:56:43.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.072+0000 7f871f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f87180cc6c0 con 0x7f871815ab20 2024-05-30T16:56:43.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.073+0000 7f8717e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87180b3fb0 0x7f87180ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:43.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.073+0000 7f8717e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87180b3fb0 0x7f87180ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34122/0 (socket says 172.21.15.67:34122) 2024-05-30T16:56:43.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.073+0000 7f8717e00640 1 -- 172.21.15.67:0/2315432673 learned_addr learned my addr 172.21.15.67:0/2315432673 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:43.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.073+0000 7f871d400640 1 --2- 172.21.15.67:0/2315432673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f871815ab20 0x7f8718147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:43.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.074+0000 7f871ca00640 1 --2- 172.21.15.67:0/2315432673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87180b3330 0x7f8718146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:43.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.074+0000 7f871d400640 1 -- 172.21.15.67:0/2315432673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87180b3330 msgr2=0x7f8718146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:43.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.074+0000 7f871d400640 1 --2- 172.21.15.67:0/2315432673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87180b3330 0x7f8718146cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:43.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.074+0000 7f871d400640 1 -- 172.21.15.67:0/2315432673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87180b3fb0 msgr2=0x7f87180ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:43.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.074+0000 7f871d400640 1 --2- 172.21.15.67:0/2315432673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87180b3fb0 0x7f87180ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:43.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.074+0000 7f871d400640 1 -- 172.21.15.67:0/2315432673 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87180cd720 con 0x7f871815ab20 2024-05-30T16:56:43.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.075+0000 7f871d400640 1 --2- 172.21.15.67:0/2315432673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f871815ab20 0x7f8718147580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8708031000 tx=0x7f87080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:43.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.076+0000 7f8714c00640 1 -- 172.21.15.67:0/2315432673 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708041020 con 0x7f871815ab20 2024-05-30T16:56:43.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.076+0000 7f8714c00640 1 -- 172.21.15.67:0/2315432673 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87080325f0 con 0x7f871815ab20 2024-05-30T16:56:43.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.076+0000 7f8714c00640 1 -- 172.21.15.67:0/2315432673 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708037040 con 0x7f871815ab20 2024-05-30T16:56:43.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.078+0000 7f871f400640 1 -- 172.21.15.67:0/2315432673 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87180cda00 con 0x7f871815ab20 2024-05-30T16:56:43.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.081+0000 7f871f400640 1 -- 172.21.15.67:0/2315432673 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87180cdfc0 con 0x7f871815ab20 2024-05-30T16:56:43.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.085+0000 7f871f400640 1 -- 172.21.15.67:0/2315432673 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87180689e0 con 0x7f871815ab20 2024-05-30T16:56:43.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.086+0000 7f8714c00640 1 -- 172.21.15.67:0/2315432673 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 29) ==== 92519+0+0 (secure 0 0 0) 0x7f8708044020 con 0x7f871815ab20 2024-05-30T16:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.088+0000 7f8714c00640 1 --2- 172.21.15.67:0/2315432673 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f86f806d940 0x7f86f806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:43.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.089+0000 7f8714c00640 1 -- 172.21.15.67:0/2315432673 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f87080aaf70 con 0x7f871815ab20 2024-05-30T16:56:43.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.093+0000 7f8714c00640 1 -- 172.21.15.67:0/2315432673 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f870804b030 con 0x7f871815ab20 2024-05-30T16:56:43.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.098+0000 7f871ca00640 1 --2- 172.21.15.67:0/2315432673 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f86f806d940 0x7f86f806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:43.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.104+0000 7f871ca00640 1 --2- 172.21.15.67:0/2315432673 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f86f806d940 0x7f86f806fe00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8710007dc0 tx=0x7f87100072f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:43.126 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:56:43.131 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:56:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:43.714+0000 7f871f400640 1 -- 172.21.15.67:0/2315432673 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92281", "target": ["mon-mgr", ""]}) -- 0x7f87180cfab0 con 0x7f86f806d940 2024-05-30T16:56:43.911 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:43.909+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:43.911 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:43.909+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:43.911 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:43.909+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:43.911 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:43.909+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:43.911 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:43.909+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:56:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.083+0000 7f8714c00640 1 -- 172.21.15.67:0/2315432673 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f8708076dc0 con 0x7f871815ab20 2024-05-30T16:56:45.346 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:45.342+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:45.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.349+0000 7f8714c00640 1 -- 172.21.15.67:0/2315432673 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f87180cfab0 con 0x7f86f806d940 2024-05-30T16:56:45.350 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:45.376 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:45.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92282 2024-05-30T16:56:45.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.792+0000 7f5883000640 1 Processor -- start 2024-05-30T16:56:45.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.794+0000 7f5883000640 1 -- start start 2024-05-30T16:56:45.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.803+0000 7f5883000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f587c1639b0 0x7f587c0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.803+0000 7f5883000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f587c163d90 0x7f587c166670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.803+0000 7f5883000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f587c166bb0 0x7f587c0ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:45.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.803+0000 7f5883000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f587c0ceca0 con 0x7f587c166bb0 2024-05-30T16:56:45.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.803+0000 7f5883000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f587c1670c0 con 0x7f587c163d90 2024-05-30T16:56:45.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.803+0000 7f5883000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f587c1673a0 con 0x7f587c1639b0 2024-05-30T16:56:45.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.805+0000 7f5882600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f587c166bb0 0x7f587c0ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:45.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.805+0000 7f5882600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f587c166bb0 0x7f587c0ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34152/0 (socket says 172.21.15.67:34152) 2024-05-30T16:56:45.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.805+0000 7f5882600640 1 -- 172.21.15.67:0/2084017660 learned_addr learned my addr 172.21.15.67:0/2084017660 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:45.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.807+0000 7f5882600640 1 -- 172.21.15.67:0/2084017660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f587c1639b0 msgr2=0x7f587c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:45.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.807+0000 7f5882600640 1 --2- 172.21.15.67:0/2084017660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f587c1639b0 0x7f587c0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:45.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.807+0000 7f5882600640 1 -- 172.21.15.67:0/2084017660 >> v2:172.21.15.169:3300/0 conn(0x7f587c163d90 msgr2=0x7f587c166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:45.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.807+0000 7f5881200640 1 --2- 172.21.15.67:0/2084017660 >> v2:172.21.15.169:3300/0 conn(0x7f587c163d90 0x7f587c166670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:45.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.807+0000 7f5882600640 1 --2- 172.21.15.67:0/2084017660 >> v2:172.21.15.169:3300/0 conn(0x7f587c163d90 0x7f587c166670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:45.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.807+0000 7f5882600640 1 -- 172.21.15.67:0/2084017660 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c0cdf60 con 0x7f587c166bb0 2024-05-30T16:56:45.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.813+0000 7f5882600640 1 --2- 172.21.15.67:0/2084017660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f587c166bb0 0x7f587c0ceee0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f587c0d11c0 tx=0x7f587402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a513d4c55d9f0676 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.815+0000 7f587be00640 1 -- 172.21.15.67:0/2084017660 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874036070 con 0x7f587c166bb0 2024-05-30T16:56:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.815+0000 7f587be00640 1 -- 172.21.15.67:0/2084017660 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5874031600 con 0x7f587c166bb0 2024-05-30T16:56:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.816+0000 7f587be00640 1 -- 172.21.15.67:0/2084017660 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58740319d0 con 0x7f587c166bb0 2024-05-30T16:56:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.818+0000 7f5883000640 1 -- 172.21.15.67:0/2084017660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f587c166bb0 msgr2=0x7f587c0ceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:45.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.818+0000 7f5883000640 1 --2- 172.21.15.67:0/2084017660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f587c166bb0 0x7f587c0ceee0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f587c0d11c0 tx=0x7f587402a6a0 comp rx=0 tx=0).stop 2024-05-30T16:56:45.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.820+0000 7f5883000640 1 -- 172.21.15.67:0/2084017660 shutdown_connections 2024-05-30T16:56:45.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.820+0000 7f5883000640 1 --2- 172.21.15.67:0/2084017660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f587c166bb0 0x7f587c0ceee0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:45.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.820+0000 7f5883000640 1 --2- 172.21.15.67:0/2084017660 >> v2:172.21.15.169:3300/0 conn(0x7f587c163d90 0x7f587c166670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:45.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.820+0000 7f5883000640 1 --2- 172.21.15.67:0/2084017660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f587c1639b0 0x7f587c0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:45.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.820+0000 7f5883000640 1 -- 172.21.15.67:0/2084017660 >> 172.21.15.67:0/2084017660 conn(0x7f587c06b900 msgr2=0x7f587c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:45.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.821+0000 7f5883000640 1 -- 172.21.15.67:0/2084017660 shutdown_connections 2024-05-30T16:56:45.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.821+0000 7f5883000640 1 -- 172.21.15.67:0/2084017660 wait complete. 2024-05-30T16:56:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.838+0000 7f5883000640 1 Processor -- start 2024-05-30T16:56:45.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.838+0000 7f5883000640 1 -- start start 2024-05-30T16:56:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.846+0000 7f5883000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f587c1639b0 0x7f587c1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.847+0000 7f5883000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f587c163d90 0x7f587c14c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:45.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.850+0000 7f5883000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f587c166bb0 0x7f587c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.850+0000 7f5881c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f587c1639b0 0x7f587c1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:45.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.851+0000 7f5881c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f587c1639b0 0x7f587c1620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:40090/0 (socket says 172.21.15.67:40090) 2024-05-30T16:56:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.851+0000 7f5881c00640 1 -- 172.21.15.67:0/1859645036 learned_addr learned my addr 172.21.15.67:0/1859645036 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.851+0000 7f5883000640 1 -- 172.21.15.67:0/1859645036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f587c0afbf0 con 0x7f587c166bb0 2024-05-30T16:56:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.853+0000 7f5883000640 1 -- 172.21.15.67:0/1859645036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f587c0afa70 con 0x7f587c1639b0 2024-05-30T16:56:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.853+0000 7f5883000640 1 -- 172.21.15.67:0/1859645036 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f587c0cf420 con 0x7f587c163d90 2024-05-30T16:56:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.853+0000 7f5881200640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f587c163d90 0x7f587c14c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:45.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.853+0000 7f5882600640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f587c166bb0 0x7f587c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:45.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.860+0000 7f5882600640 1 -- 172.21.15.67:0/1859645036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f587c163d90 msgr2=0x7f587c14c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.860+0000 7f5882600640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f587c163d90 0x7f587c14c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.860+0000 7f5882600640 1 -- 172.21.15.67:0/1859645036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f587c1639b0 msgr2=0x7f587c1620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.860+0000 7f5882600640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f587c1639b0 0x7f587c1620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.860+0000 7f5882600640 1 -- 172.21.15.67:0/1859645036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c15e140 con 0x7f587c166bb0 2024-05-30T16:56:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.861+0000 7f5882600640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f587c166bb0 0x7f587c0b0330 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5874030000 tx=0x7f5874004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.861+0000 7f5881c00640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f587c1639b0 0x7f587c1620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:45.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.861+0000 7f5881200640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f587c163d90 0x7f587c14c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:45.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.862+0000 7f5879600640 1 -- 172.21.15.67:0/1859645036 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874036050 con 0x7f587c166bb0 2024-05-30T16:56:45.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.862+0000 7f5879600640 1 -- 172.21.15.67:0/1859645036 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5874042070 con 0x7f587c166bb0 2024-05-30T16:56:45.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.862+0000 7f5883000640 1 -- 172.21.15.67:0/1859645036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c15e420 con 0x7f587c166bb0 2024-05-30T16:56:45.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.862+0000 7f5879600640 1 -- 172.21.15.67:0/1859645036 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874009510 con 0x7f587c166bb0 2024-05-30T16:56:45.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.862+0000 7f5883000640 1 -- 172.21.15.67:0/1859645036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c15e9e0 con 0x7f587c166bb0 2024-05-30T16:56:45.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.865+0000 7f5883000640 1 -- 172.21.15.67:0/1859645036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c0689e0 con 0x7f587c166bb0 2024-05-30T16:56:45.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.874+0000 7f5879600640 1 -- 172.21.15.67:0/1859645036 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f5874007bb0 con 0x7f587c166bb0 2024-05-30T16:56:45.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.876+0000 7f5879600640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f585c06daa0 0x7f585c06ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:45.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.876+0000 7f5879600640 1 -- 172.21.15.67:0/1859645036 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f58740ac0f0 con 0x7f587c166bb0 2024-05-30T16:56:45.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.885+0000 7f5879600640 1 -- 172.21.15.67:0/1859645036 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f587407b9b0 con 0x7f587c166bb0 2024-05-30T16:56:45.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.910+0000 7f5881c00640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f585c06daa0 0x7f585c06ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:45.965+0000 7f5881c00640 1 --2- 172.21.15.67:0/1859645036 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f585c06daa0 0x7f585c06ff60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5870009e10 tx=0x7f58700091c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:46.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:46.494+0000 7f5883000640 1 -- 172.21.15.67:0/1859645036 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92282", "target": ["mon-mgr", ""]}) -- 0x7f587c0b3df0 con 0x7f585c06daa0 2024-05-30T16:56:46.749 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:46.748+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:46.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:46.751+0000 7f5879600640 1 -- 172.21.15.67:0/1859645036 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f587c0b3df0 con 0x7f585c06daa0 2024-05-30T16:56:46.752 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:46.780 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:46.781 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92283 2024-05-30T16:56:47.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.167+0000 7facd7400640 1 Processor -- start 2024-05-30T16:56:47.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.168+0000 7facd7400640 1 -- start start 2024-05-30T16:56:47.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.169+0000 7facd7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd006fd30 0x7facd01581a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:47.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.169+0000 7facd7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7facd006ecb0 0x7facd015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:47.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.169+0000 7facd7400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd00b3660 con 0x7facd0072780 2024-05-30T16:56:47.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.169+0000 7facd7400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7facd01586e0 con 0x7facd006ecb0 2024-05-30T16:56:47.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.169+0000 7facd7400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7facd00b3940 con 0x7facd006fd30 2024-05-30T16:56:47.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.170+0000 7facd5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7facd006ecb0 0x7facd015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:47.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.170+0000 7faccfe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7facd0072780 0x7facd015e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:52582/0 (socket says 172.21.15.67:52582) 2024-05-30T16:56:47.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.170+0000 7faccfe00640 1 -- 172.21.15.67:0/2142163637 learned_addr learned my addr 172.21.15.67:0/2142163637 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:47.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.170+0000 7facd4a00640 1 --2- 172.21.15.67:0/2142163637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd006fd30 0x7facd01581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:47.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.171+0000 7facd5400640 1 -- 172.21.15.67:0/2142163637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd006fd30 msgr2=0x7facd01581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:47.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.171+0000 7facd5400640 1 --2- 172.21.15.67:0/2142163637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd006fd30 0x7facd01581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:47.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.171+0000 7facd5400640 1 -- 172.21.15.67:0/2142163637 >> v1:172.21.15.169:6789/0 conn(0x7facd0072780 legacy=0x7facd015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:47.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.171+0000 7facd5400640 1 -- 172.21.15.67:0/2142163637 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd00b3c20 con 0x7facd006ecb0 2024-05-30T16:56:47.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.171+0000 7facd4a00640 1 --2- 172.21.15.67:0/2142163637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd006fd30 0x7facd01581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.173+0000 7facd5400640 1 --2- 172.21.15.67:0/2142163637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facd006ecb0 0x7facd015ea00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7facc800be30 tx=0x7facc80092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e1df360885def4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.174+0000 7faccf400640 1 -- 172.21.15.67:0/2142163637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8036070 con 0x7facd006ecb0 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.175+0000 7faccf400640 1 -- 172.21.15.67:0/2142163637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc8009540 con 0x7facd006ecb0 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.175+0000 7faccf400640 1 -- 172.21.15.67:0/2142163637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8036070 con 0x7facd006ecb0 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.176+0000 7facd7400640 1 -- 172.21.15.67:0/2142163637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facd006ecb0 msgr2=0x7facd015ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.176+0000 7facd7400640 1 --2- 172.21.15.67:0/2142163637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facd006ecb0 0x7facd015ea00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7facc800be30 tx=0x7facc80092f0 comp rx=0 tx=0).stop 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.177+0000 7facd7400640 1 -- 172.21.15.67:0/2142163637 shutdown_connections 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.177+0000 7facd7400640 1 --2- 172.21.15.67:0/2142163637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facd006ecb0 0x7facd015ea00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.177+0000 7facd7400640 1 --2- 172.21.15.67:0/2142163637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd006fd30 0x7facd01581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.177+0000 7facd7400640 1 -- 172.21.15.67:0/2142163637 >> 172.21.15.67:0/2142163637 conn(0x7facd006bd20 msgr2=0x7facd006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.177+0000 7facd7400640 1 -- 172.21.15.67:0/2142163637 shutdown_connections 2024-05-30T16:56:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.177+0000 7facd7400640 1 -- 172.21.15.67:0/2142163637 wait complete. 2024-05-30T16:56:47.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.187+0000 7facd7400640 1 Processor -- start 2024-05-30T16:56:47.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.187+0000 7facd7400640 1 -- start start 2024-05-30T16:56:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.194+0000 7facd7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facd006ecb0 0x7facd0099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.194+0000 7facd7400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd006fd30 0x7facd01520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:47.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.194+0000 7facd7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd0072780 0x7facd0152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:47.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.194+0000 7facd7400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7facd0072c00 con 0x7facd006ecb0 2024-05-30T16:56:47.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.194+0000 7facd7400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7facd00b41e0 con 0x7facd006fd30 2024-05-30T16:56:47.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.194+0000 7facd7400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7facd00b32c0 con 0x7facd0072780 2024-05-30T16:56:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.195+0000 7faccfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd006fd30 0x7facd01520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.196+0000 7facd5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd0072780 0x7facd0152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.196+0000 7facd5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd0072780 0x7facd0152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:59720/0 (socket says 172.21.15.67:59720) 2024-05-30T16:56:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.196+0000 7facd5400640 1 -- 172.21.15.67:0/3379744090 learned_addr learned my addr 172.21.15.67:0/3379744090 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.196+0000 7facd4a00640 1 --2- 172.21.15.67:0/3379744090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facd006ecb0 0x7facd0099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:47.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.199+0000 7facd4a00640 1 -- 172.21.15.67:0/3379744090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd0072780 msgr2=0x7facd0152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:47.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.200+0000 7facd4a00640 1 --2- 172.21.15.67:0/3379744090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd0072780 0x7facd0152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:47.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.200+0000 7facd4a00640 1 -- 172.21.15.67:0/3379744090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd006fd30 msgr2=0x7facd01520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:47.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.200+0000 7facd4a00640 1 --2- 172.21.15.67:0/3379744090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd006fd30 0x7facd01520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:47.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.200+0000 7facd4a00640 1 -- 172.21.15.67:0/3379744090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd00aeba0 con 0x7facd006ecb0 2024-05-30T16:56:47.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.200+0000 7facd5400640 1 --2- 172.21.15.67:0/3379744090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facd0072780 0x7facd0152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:47.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.200+0000 7faccfe00640 1 --2- 172.21.15.67:0/3379744090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facd006fd30 0x7facd01520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:47.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.201+0000 7facd4a00640 1 --2- 172.21.15.67:0/3379744090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facd006ecb0 0x7facd0099850 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7facc400b390 tx=0x7facc400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:47.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.202+0000 7facccc00640 1 -- 172.21.15.67:0/3379744090 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4023070 con 0x7facd006ecb0 2024-05-30T16:56:47.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.202+0000 7facccc00640 1 -- 172.21.15.67:0/3379744090 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc401b6a0 con 0x7facd006ecb0 2024-05-30T16:56:47.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.202+0000 7facccc00640 1 -- 172.21.15.67:0/3379744090 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc401ba90 con 0x7facd006ecb0 2024-05-30T16:56:47.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.202+0000 7facd7400640 1 -- 172.21.15.67:0/3379744090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd00b0020 con 0x7facd006ecb0 2024-05-30T16:56:47.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.203+0000 7facd7400640 1 -- 172.21.15.67:0/3379744090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd00b05e0 con 0x7facd006ecb0 2024-05-30T16:56:47.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.206+0000 7facd7400640 1 -- 172.21.15.67:0/3379744090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd00689e0 con 0x7facd006ecb0 2024-05-30T16:56:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.207+0000 7facccc00640 1 -- 172.21.15.67:0/3379744090 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7facc4026070 con 0x7facd006ecb0 2024-05-30T16:56:47.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.209+0000 7facccc00640 1 --2- 172.21.15.67:0/3379744090 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7facb006daa0 0x7facb006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:47.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.212+0000 7facccc00640 1 -- 172.21.15.67:0/3379744090 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7facc4092bd0 con 0x7facd006ecb0 2024-05-30T16:56:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.216+0000 7faccfe00640 1 --2- 172.21.15.67:0/3379744090 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7facb006daa0 0x7facb006ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:47.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.217+0000 7facccc00640 1 -- 172.21.15.67:0/3379744090 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7facc40620c0 con 0x7facd006ecb0 2024-05-30T16:56:47.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.218+0000 7faccfe00640 1 --2- 172.21.15.67:0/3379744090 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7facb006daa0 0x7facb006ff60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7facd0153f30 tx=0x7facbc007e80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:47.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.811+0000 7facd7400640 1 -- 172.21.15.67:0/3379744090 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92283", "target": ["mon-mgr", ""]}) -- 0x7facd00af8f0 con 0x7facb006daa0 2024-05-30T16:56:47.999 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:47.998+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:47.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:47.999+0000 7facccc00640 1 -- 172.21.15.67:0/3379744090 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7facd00af8f0 con 0x7facb006daa0 2024-05-30T16:56:48.004 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:48.035 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:48.036 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92284 2024-05-30T16:56:48.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.469+0000 7fd222600640 1 Processor -- start 2024-05-30T16:56:48.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.469+0000 7fd222600640 1 -- start start 2024-05-30T16:56:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.474+0000 7fd222600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd21c0b2ae0 0x7fd21c073a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:48.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.474+0000 7fd222600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21c0b3330 0x7fd21c0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.474+0000 7fd222600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd21c0cc920 con 0x7fd21c0b3de0 2024-05-30T16:56:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.474+0000 7fd222600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd21c0b17c0 con 0x7fd21c0b2ae0 2024-05-30T16:56:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.474+0000 7fd222600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c0b1aa0 con 0x7fd21c0b3330 2024-05-30T16:56:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.475+0000 7fd21b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd21c0b2ae0 0x7fd21c073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.475+0000 7fd21b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd21c0b2ae0 0x7fd21c073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34192/0 (socket says 172.21.15.67:34192) 2024-05-30T16:56:48.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.475+0000 7fd21b400640 1 -- 172.21.15.67:0/3590702493 learned_addr learned my addr 172.21.15.67:0/3590702493 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.477+0000 7fd21aa00640 1 --2- 172.21.15.67:0/3590702493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21c0b3330 0x7fd21c0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.478+0000 7fd21b400640 1 -- 172.21.15.67:0/3590702493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21c0b3330 msgr2=0x7fd21c0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:48.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.478+0000 7fd21b400640 1 --2- 172.21.15.67:0/3590702493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21c0b3330 0x7fd21c0b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.478+0000 7fd21b400640 1 -- 172.21.15.67:0/3590702493 >> v1:172.21.15.169:6789/0 conn(0x7fd21c0b3de0 legacy=0x7fd21c15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.478+0000 7fd21b400640 1 -- 172.21.15.67:0/3590702493 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c0b1d80 con 0x7fd21c0b2ae0 2024-05-30T16:56:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.481+0000 7fd21b400640 1 --2- 172.21.15.67:0/3590702493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd21c0b2ae0 0x7fd21c073a40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd214013150 tx=0x7fd214032910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b2a529e82b51ab server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.481+0000 7fd21a000640 1 -- 172.21.15.67:0/3590702493 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2140114e0 con 0x7fd21c0b2ae0 2024-05-30T16:56:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.482+0000 7fd21a000640 1 -- 172.21.15.67:0/3590702493 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd214011680 con 0x7fd21c0b2ae0 2024-05-30T16:56:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.482+0000 7fd21a000640 1 -- 172.21.15.67:0/3590702493 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21400fa70 con 0x7fd21c0b2ae0 2024-05-30T16:56:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.483+0000 7fd222600640 1 -- 172.21.15.67:0/3590702493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd21c0b2ae0 msgr2=0x7fd21c073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.483+0000 7fd222600640 1 --2- 172.21.15.67:0/3590702493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd21c0b2ae0 0x7fd21c073a40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd214013150 tx=0x7fd214032910 comp rx=0 tx=0).stop 2024-05-30T16:56:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.483+0000 7fd222600640 1 -- 172.21.15.67:0/3590702493 shutdown_connections 2024-05-30T16:56:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.483+0000 7fd222600640 1 --2- 172.21.15.67:0/3590702493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21c0b3330 0x7fd21c0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.483+0000 7fd222600640 1 --2- 172.21.15.67:0/3590702493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd21c0b2ae0 0x7fd21c073a40 secure :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd214013150 tx=0x7fd214032910 comp rx=0 tx=0).stop 2024-05-30T16:56:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.483+0000 7fd222600640 1 -- 172.21.15.67:0/3590702493 >> 172.21.15.67:0/3590702493 conn(0x7fd21c0b1fc0 msgr2=0x7fd21c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:48.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.484+0000 7fd222600640 1 -- 172.21.15.67:0/3590702493 shutdown_connections 2024-05-30T16:56:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.485+0000 7fd222600640 1 -- 172.21.15.67:0/3590702493 wait complete. 2024-05-30T16:56:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.489+0000 7fd222600640 1 Processor -- start 2024-05-30T16:56:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.489+0000 7fd222600640 1 -- start start 2024-05-30T16:56:48.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.491+0000 7fd222600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd21c0b3330 0x7fd21c14d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.491+0000 7fd222600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21c0b3de0 0x7fd21c0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.491+0000 7fd222600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c0ceef0 0x7fd21c0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.491+0000 7fd222600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0cc6c0 con 0x7fd21c0b3330 2024-05-30T16:56:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.491+0000 7fd222600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0cc540 con 0x7fd21c0ceef0 2024-05-30T16:56:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.491+0000 7fd222600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c15bb30 con 0x7fd21c0b3de0 2024-05-30T16:56:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.492+0000 7fd21aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21c0b3de0 0x7fd21c0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.492+0000 7fd21aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21c0b3de0 0x7fd21c0ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:59742/0 (socket says 172.21.15.67:59742) 2024-05-30T16:56:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.492+0000 7fd21aa00640 1 -- 172.21.15.67:0/230538289 learned_addr learned my addr 172.21.15.67:0/230538289 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.492+0000 7fd21be00640 1 --2- 172.21.15.67:0/230538289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c0ceef0 0x7fd21c0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:48.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.493+0000 7fd21aa00640 1 -- 172.21.15.67:0/230538289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c0ceef0 msgr2=0x7fd21c0aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:48.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.493+0000 7fd21aa00640 1 --2- 172.21.15.67:0/230538289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c0ceef0 0x7fd21c0aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:48.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.493+0000 7fd21aa00640 1 -- 172.21.15.67:0/230538289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd21c0b3330 msgr2=0x7fd21c14d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:48.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.493+0000 7fd21aa00640 1 --2- 172.21.15.67:0/230538289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd21c0b3330 0x7fd21c14d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:48.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.493+0000 7fd21aa00640 1 -- 172.21.15.67:0/230538289 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c1f48e0 con 0x7fd21c0b3de0 2024-05-30T16:56:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.493+0000 7fd21be00640 1 --2- 172.21.15.67:0/230538289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c0ceef0 0x7fd21c0aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:56:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.496+0000 7fd21aa00640 1 --2- 172.21.15.67:0/230538289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd21c0b3de0 0x7fd21c0ce9b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd20800d800 tx=0x7fd20800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.498+0000 7fd213400640 1 -- 172.21.15.67:0/230538289 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20800b660 con 0x7fd21c0b3de0 2024-05-30T16:56:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.498+0000 7fd222600640 1 -- 172.21.15.67:0/230538289 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c0ad550 con 0x7fd21c0b3de0 2024-05-30T16:56:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.499+0000 7fd222600640 1 -- 172.21.15.67:0/230538289 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c0adb10 con 0x7fd21c0b3de0 2024-05-30T16:56:48.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.500+0000 7fd213400640 1 -- 172.21.15.67:0/230538289 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd208019070 con 0x7fd21c0b3de0 2024-05-30T16:56:48.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.500+0000 7fd213400640 1 -- 172.21.15.67:0/230538289 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20800ba20 con 0x7fd21c0b3de0 2024-05-30T16:56:48.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.503+0000 7fd222600640 1 -- 172.21.15.67:0/230538289 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f4005290 con 0x7fd21c0b3de0 2024-05-30T16:56:48.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.505+0000 7fd213400640 1 -- 172.21.15.67:0/230538289 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7fd20800bbc0 con 0x7fd21c0b3de0 2024-05-30T16:56:48.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.508+0000 7fd213400640 1 --2- 172.21.15.67:0/230538289 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd1fc06da00 0x7fd1fc06fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.509+0000 7fd213400640 1 -- 172.21.15.67:0/230538289 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fd20808a9c0 con 0x7fd21c0b3de0 2024-05-30T16:56:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.518+0000 7fd213400640 1 -- 172.21.15.67:0/230538289 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd20805a8a0 con 0x7fd21c0b3de0 2024-05-30T16:56:48.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.527+0000 7fd21b400640 1 --2- 172.21.15.67:0/230538289 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd1fc06da00 0x7fd1fc06fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:48.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:48.580+0000 7fd21b400640 1 --2- 172.21.15.67:0/230538289 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd1fc06da00 0x7fd1fc06fec0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd21403c000 tx=0x7fd214037000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:49.153+0000 7fd222600640 1 -- 172.21.15.67:0/230538289 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92284", "target": ["mon-mgr", ""]}) -- 0x7fd1f4000bc0 con 0x7fd1fc06da00 2024-05-30T16:56:49.956 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:49.955+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:49.960+0000 7fd213400640 1 -- 172.21.15.67:0/230538289 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fd1f4000bc0 con 0x7fd1fc06da00 2024-05-30T16:56:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:49.989 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:49.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92285 2024-05-30T16:56:50.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.509+0000 7f7129400640 1 Processor -- start 2024-05-30T16:56:50.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.509+0000 7f7129400640 1 -- start start 2024-05-30T16:56:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.511+0000 7f7129400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7124162ac0 0x7f71240d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.511+0000 7f7129400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 0x7f71240d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.511+0000 7f7129400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71241640e0 con 0x7f71241633b0 2024-05-30T16:56:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.511+0000 7f7129400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f71240b3c90 con 0x7f7124162ac0 2024-05-30T16:56:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.511+0000 7f7129400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71240b3360 con 0x7f71240b4120 2024-05-30T16:56:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.512+0000 7f7122800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7124162ac0 0x7f71240d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.512+0000 7f7123200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f71241633b0 0x7f7124163820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42718/0 (socket says 172.21.15.67:42718) 2024-05-30T16:56:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.512+0000 7f7123200640 1 -- 172.21.15.67:0/1399931457 learned_addr learned my addr 172.21.15.67:0/1399931457 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.513+0000 7f7121e00640 1 --2- 172.21.15.67:0/1399931457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 0x7f71240d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:50.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.513+0000 7f7121400640 1 -- 172.21.15.67:0/1399931457 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3424449916 0 0) 0x7f71241640e0 con 0x7f71241633b0 2024-05-30T16:56:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.513+0000 7f7121400640 1 -- 172.21.15.67:0/1399931457 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7108003520 con 0x7f71241633b0 2024-05-30T16:56:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.513+0000 7f7121e00640 1 -- 172.21.15.67:0/1399931457 >> v2:172.21.15.169:3300/0 conn(0x7f7124162ac0 msgr2=0x7f71240d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.513+0000 7f7121e00640 1 --2- 172.21.15.67:0/1399931457 >> v2:172.21.15.169:3300/0 conn(0x7f7124162ac0 0x7f71240d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.513+0000 7f7121e00640 1 -- 172.21.15.67:0/1399931457 >> v1:172.21.15.67:6789/0 conn(0x7f71241633b0 legacy=0x7f7124163820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:50.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.513+0000 7f7121e00640 1 -- 172.21.15.67:0/1399931457 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71240b3640 con 0x7f71240b4120 2024-05-30T16:56:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.515+0000 7f7121e00640 1 --2- 172.21.15.67:0/1399931457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 0x7f71240d0b80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71100097a0 tx=0x7f711002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c9d2d6a33acff4a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.516+0000 7f7121400640 1 -- 172.21.15.67:0/1399931457 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110037070 con 0x7f71240b4120 2024-05-30T16:56:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.516+0000 7f7121400640 1 -- 172.21.15.67:0/1399931457 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7110032440 con 0x7f71240b4120 2024-05-30T16:56:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.516+0000 7f7121400640 1 -- 172.21.15.67:0/1399931457 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110032850 con 0x7f71240b4120 2024-05-30T16:56:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.518+0000 7f7129400640 1 -- 172.21.15.67:0/1399931457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 msgr2=0x7f71240d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.518+0000 7f7129400640 1 --2- 172.21.15.67:0/1399931457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 0x7f71240d0b80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71100097a0 tx=0x7f711002a1a0 comp rx=0 tx=0).stop 2024-05-30T16:56:50.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.519+0000 7f7129400640 1 -- 172.21.15.67:0/1399931457 shutdown_connections 2024-05-30T16:56:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.519+0000 7f7129400640 1 --2- 172.21.15.67:0/1399931457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 0x7f71240d0b80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.519+0000 7f7129400640 1 --2- 172.21.15.67:0/1399931457 >> v2:172.21.15.169:3300/0 conn(0x7f7124162ac0 0x7f71240d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.519+0000 7f7129400640 1 -- 172.21.15.67:0/1399931457 >> 172.21.15.67:0/1399931457 conn(0x7f712406bd20 msgr2=0x7f712406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:50.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.525+0000 7f7129400640 1 -- 172.21.15.67:0/1399931457 shutdown_connections 2024-05-30T16:56:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.527+0000 7f7129400640 1 -- 172.21.15.67:0/1399931457 wait complete. 2024-05-30T16:56:50.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.539+0000 7f7129400640 1 Processor -- start 2024-05-30T16:56:50.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.539+0000 7f7129400640 1 -- start start 2024-05-30T16:56:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.545+0000 7f7129400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 0x7f71240cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.545+0000 7f7129400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7124162ac0 0x7f712415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.545+0000 7f7129400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71241633b0 0x7f712415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.545+0000 7f7129400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7124163020 con 0x7f71241633b0 2024-05-30T16:56:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.545+0000 7f7129400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7124162ea0 con 0x7f7124162ac0 2024-05-30T16:56:50.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.545+0000 7f7129400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f712406dbd0 con 0x7f71240b4120 2024-05-30T16:56:50.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.546+0000 7f7122800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 0x7f71240cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:50.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.546+0000 7f7122800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 0x7f71240cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:59772/0 (socket says 172.21.15.67:59772) 2024-05-30T16:56:50.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.546+0000 7f7122800640 1 -- 172.21.15.67:0/3901976515 learned_addr learned my addr 172.21.15.67:0/3901976515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:50.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.546+0000 7f7122800640 1 -- 172.21.15.67:0/3901976515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7124162ac0 msgr2=0x7f712415a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-30T16:56:50.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.546+0000 7f7122800640 1 --2- 172.21.15.67:0/3901976515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7124162ac0 0x7f712415a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:50.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.546+0000 7f7122800640 1 -- 172.21.15.67:0/3901976515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71241633b0 msgr2=0x7f712415bfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-30T16:56:50.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.546+0000 7f7122800640 1 --2- 172.21.15.67:0/3901976515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71241633b0 0x7f712415bfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:50.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.546+0000 7f7122800640 1 -- 172.21.15.67:0/3901976515 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712415b5b0 con 0x7f71240b4120 2024-05-30T16:56:50.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.547+0000 7f7122800640 1 --2- 172.21.15.67:0/3901976515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71240b4120 0x7f71240cd3e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f711400d800 tx=0x7f711400dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:50.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.547+0000 7f711aa00640 1 -- 172.21.15.67:0/3901976515 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711400b6c0 con 0x7f71240b4120 2024-05-30T16:56:50.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.547+0000 7f711aa00640 1 -- 172.21.15.67:0/3901976515 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7114019070 con 0x7f71240b4120 2024-05-30T16:56:50.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.547+0000 7f711aa00640 1 -- 172.21.15.67:0/3901976515 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711400ba60 con 0x7f71240b4120 2024-05-30T16:56:50.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.548+0000 7f7129400640 1 -- 172.21.15.67:0/3901976515 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712415d670 con 0x7f71240b4120 2024-05-30T16:56:50.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.548+0000 7f7129400640 1 -- 172.21.15.67:0/3901976515 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f712415d910 con 0x7f71240b4120 2024-05-30T16:56:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.552+0000 7f7129400640 1 -- 172.21.15.67:0/3901976515 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71240689e0 con 0x7f71240b4120 2024-05-30T16:56:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.552+0000 7f711aa00640 1 -- 172.21.15.67:0/3901976515 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f711400bc00 con 0x7f71240b4120 2024-05-30T16:56:50.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.555+0000 7f711aa00640 1 --2- 172.21.15.67:0/3901976515 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f710406d9d0 0x7f710406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:50.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.555+0000 7f711aa00640 1 -- 172.21.15.67:0/3901976515 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f711408ab80 con 0x7f71240b4120 2024-05-30T16:56:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.561+0000 7f7121e00640 1 --2- 172.21.15.67:0/3901976515 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f710406d9d0 0x7f710406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:50.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.565+0000 7f7121e00640 1 --2- 172.21.15.67:0/3901976515 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f710406d9d0 0x7f710406fe90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7110034000 tx=0x7f711002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:50.567+0000 7f711aa00640 1 -- 172.21.15.67:0/3901976515 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f711405aa60 con 0x7f71240b4120 2024-05-30T16:56:51.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:51.328+0000 7f7129400640 1 -- 172.21.15.67:0/3901976515 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92285", "target": ["mon-mgr", ""]}) -- 0x7f712406c2a0 con 0x7f710406d9d0 2024-05-30T16:56:52.163 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:52.163+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.163+0000 7f711aa00640 1 -- 172.21.15.67:0/3901976515 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f712406c2a0 con 0x7f710406d9d0 2024-05-30T16:56:52.165 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:52.194 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:52.195 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92286 2024-05-30T16:56:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.610+0000 7faae7000640 1 Processor -- start 2024-05-30T16:56:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.611+0000 7faae7000640 1 -- start start 2024-05-30T16:56:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.613+0000 7faae7000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faae0162ac0 0x7faae00d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.613+0000 7faae7000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae01640e0 con 0x7faae0163ac0 2024-05-30T16:56:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.613+0000 7faae7000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae00b3360 con 0x7faae00b4120 2024-05-30T16:56:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.613+0000 7faae7000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7faae00b3c90 con 0x7faae0162ac0 2024-05-30T16:56:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.613+0000 7faadfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faae0162ac0 0x7faae00d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.614+0000 7faadfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faae0162ac0 0x7faae00d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54884/0 (socket says 172.21.15.67:54884) 2024-05-30T16:56:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.614+0000 7faadfe00640 1 -- 172.21.15.67:0/3743684290 learned_addr learned my addr 172.21.15.67:0/3743684290 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.614+0000 7faadfe00640 1 -- 172.21.15.67:0/3743684290 >> v1:172.21.15.169:6789/0 conn(0x7faae00b4120 legacy=0x7faae01633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.614+0000 7faadfe00640 1 -- 172.21.15.67:0/3743684290 >> v1:172.21.15.67:6789/0 conn(0x7faae0163ac0 legacy=0x7faae006dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.614+0000 7faadfe00640 1 -- 172.21.15.67:0/3743684290 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae00b3640 con 0x7faae0162ac0 2024-05-30T16:56:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.617+0000 7faadfe00640 1 --2- 172.21.15.67:0/3743684290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae0162ac0 0x7faae00d0640 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7faae00b3e10 tx=0x7faacc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc2dd128776cb30 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:52.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.619+0000 7faadea00640 1 -- 172.21.15.67:0/3743684290 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc037070 con 0x7faae0162ac0 2024-05-30T16:56:52.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.619+0000 7faadea00640 1 -- 172.21.15.67:0/3743684290 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faacc032440 con 0x7faae0162ac0 2024-05-30T16:56:52.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.620+0000 7faadea00640 1 -- 172.21.15.67:0/3743684290 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc032810 con 0x7faae0162ac0 2024-05-30T16:56:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.621+0000 7faae7000640 1 -- 172.21.15.67:0/3743684290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae0162ac0 msgr2=0x7faae00d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.621+0000 7faae7000640 1 --2- 172.21.15.67:0/3743684290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae0162ac0 0x7faae00d0640 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7faae00b3e10 tx=0x7faacc02a250 comp rx=0 tx=0).stop 2024-05-30T16:56:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.621+0000 7faae7000640 1 -- 172.21.15.67:0/3743684290 shutdown_connections 2024-05-30T16:56:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.621+0000 7faae7000640 1 --2- 172.21.15.67:0/3743684290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae0162ac0 0x7faae00d0640 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.621+0000 7faae7000640 1 -- 172.21.15.67:0/3743684290 >> 172.21.15.67:0/3743684290 conn(0x7faae006bd20 msgr2=0x7faae006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.621+0000 7faae7000640 1 -- 172.21.15.67:0/3743684290 shutdown_connections 2024-05-30T16:56:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.621+0000 7faae7000640 1 -- 172.21.15.67:0/3743684290 wait complete. 2024-05-30T16:56:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.624+0000 7faae7000640 1 Processor -- start 2024-05-30T16:56:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.625+0000 7faae7000640 1 -- start start 2024-05-30T16:56:52.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.626+0000 7faae7000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae00b4120 0x7faae01525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:52.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.626+0000 7faae7000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae0162ac0 0x7faae0160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:52.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.626+0000 7faae7000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae0163ac0 0x7faae015d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:52.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.627+0000 7faae7000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faae0163020 con 0x7faae0162ac0 2024-05-30T16:56:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.627+0000 7faae7000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faae0162ea0 con 0x7faae00b4120 2024-05-30T16:56:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.627+0000 7faae7000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faae00d0b80 con 0x7faae0163ac0 2024-05-30T16:56:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.628+0000 7faae5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae0163ac0 0x7faae015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:52.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.628+0000 7faadfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae00b4120 0x7faae01525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:52.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.628+0000 7faadfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae00b4120 0x7faae01525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:37946/0 (socket says 172.21.15.67:37946) 2024-05-30T16:56:52.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.628+0000 7faadfe00640 1 -- 172.21.15.67:0/1783003427 learned_addr learned my addr 172.21.15.67:0/1783003427 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:52.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.628+0000 7faadf400640 1 --2- 172.21.15.67:0/1783003427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae0162ac0 0x7faae0160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.629+0000 7faadfe00640 1 -- 172.21.15.67:0/1783003427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae0163ac0 msgr2=0x7faae015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.629+0000 7faadfe00640 1 --2- 172.21.15.67:0/1783003427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae0163ac0 0x7faae015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.629+0000 7faadfe00640 1 -- 172.21.15.67:0/1783003427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae0162ac0 msgr2=0x7faae0160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.629+0000 7faadfe00640 1 --2- 172.21.15.67:0/1783003427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae0162ac0 0x7faae0160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.629+0000 7faadfe00640 1 -- 172.21.15.67:0/1783003427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae0152b20 con 0x7faae00b4120 2024-05-30T16:56:52.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.630+0000 7faadfe00640 1 --2- 172.21.15.67:0/1783003427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae00b4120 0x7faae01525e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faacc000c00 tx=0x7faacc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.631+0000 7faad7e00640 1 -- 172.21.15.67:0/1783003427 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc037040 con 0x7faae00b4120 2024-05-30T16:56:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.631+0000 7faad7e00640 1 -- 172.21.15.67:0/1783003427 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faacc004250 con 0x7faae00b4120 2024-05-30T16:56:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.631+0000 7faad7e00640 1 -- 172.21.15.67:0/1783003427 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc03c730 con 0x7faae00b4120 2024-05-30T16:56:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.632+0000 7faae7000640 1 -- 172.21.15.67:0/1783003427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae0160150 con 0x7faae00b4120 2024-05-30T16:56:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.632+0000 7faae7000640 1 -- 172.21.15.67:0/1783003427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae0160710 con 0x7faae00b4120 2024-05-30T16:56:52.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.635+0000 7faae7000640 1 -- 172.21.15.67:0/1783003427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae00689e0 con 0x7faae00b4120 2024-05-30T16:56:52.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.637+0000 7faad7e00640 1 -- 172.21.15.67:0/1783003427 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7faacc03a070 con 0x7faae00b4120 2024-05-30T16:56:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.639+0000 7faad7e00640 1 --2- 172.21.15.67:0/1783003427 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faac006da00 0x7faac006fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.639+0000 7faad7e00640 1 -- 172.21.15.67:0/1783003427 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7faacc0ab270 con 0x7faae00b4120 2024-05-30T16:56:52.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.651+0000 7faad7e00640 1 -- 172.21.15.67:0/1783003427 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faacc041020 con 0x7faae00b4120 2024-05-30T16:56:52.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.689+0000 7faadf400640 1 --2- 172.21.15.67:0/1783003427 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faac006da00 0x7faac006fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:52.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:52.761+0000 7faadf400640 1 --2- 172.21.15.67:0/1783003427 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faac006da00 0x7faac006fec0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faae01618e0 tx=0x7faad0007e50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:53.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:53.261+0000 7faae7000640 1 -- 172.21.15.67:0/1783003427 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92286", "target": ["mon-mgr", ""]}) -- 0x7faae015ff00 con 0x7faac006da00 2024-05-30T16:56:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:53.508+0000 7faad7e00640 1 -- 172.21.15.67:0/1783003427 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7faae015ff00 con 0x7faac006da00 2024-05-30T16:56:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:53.514 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:53.506+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:53.557 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:53.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92287 2024-05-30T16:56:54.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.041+0000 7f7c67200640 1 Processor -- start 2024-05-30T16:56:54.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.043+0000 7f7c67200640 1 -- start start 2024-05-30T16:56:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.046+0000 7f7c67200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7c680d0740 0x7f7c680b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:54.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.046+0000 7f7c67200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c680afc70 con 0x7f7c680b36d0 2024-05-30T16:56:54.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.046+0000 7f7c67200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c680aff50 con 0x7f7c6815c200 2024-05-30T16:56:54.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.046+0000 7f7c67200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7c680b0880 con 0x7f7c680d0740 2024-05-30T16:56:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.048+0000 7f7c65e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7c6815c200 0x7f7c680d0030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:55464/0 (socket says 172.21.15.67:55464) 2024-05-30T16:56:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.048+0000 7f7c65e00640 1 -- 172.21.15.67:0/3562552805 learned_addr learned my addr 172.21.15.67:0/3562552805 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.048+0000 7f7c66800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7c680b36d0 0x7f7c680b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36662/0 (socket says 172.21.15.67:36662) 2024-05-30T16:56:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.049+0000 7f7c65400640 1 --2- 172.21.15.67:0/3562552805 >> v2:172.21.15.67:3300/0 conn(0x7f7c680d0740 0x7f7c680b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.050+0000 7f7c65400640 1 -- 172.21.15.67:0/3562552805 >> v1:172.21.15.169:6789/0 conn(0x7f7c6815c200 legacy=0x7f7c680d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.051+0000 7f7c65400640 1 -- 172.21.15.67:0/3562552805 >> v1:172.21.15.67:6789/0 conn(0x7f7c680b36d0 legacy=0x7f7c680b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.051+0000 7f7c65400640 1 -- 172.21.15.67:0/3562552805 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c680b0230 con 0x7f7c680d0740 2024-05-30T16:56:54.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.054+0000 7f7c65400640 1 --2- 172.21.15.67:0/3562552805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c680d0740 0x7f7c680b3190 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009a00 tx=0x7f7c5402a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68d9d3bfc6893e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:54.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.059+0000 7f7c64a00640 1 -- 172.21.15.67:0/3562552805 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54037070 con 0x7f7c680d0740 2024-05-30T16:56:54.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.059+0000 7f7c64a00640 1 -- 172.21.15.67:0/3562552805 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c5402ae10 con 0x7f7c680d0740 2024-05-30T16:56:54.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.059+0000 7f7c64a00640 1 -- 172.21.15.67:0/3562552805 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54032660 con 0x7f7c680d0740 2024-05-30T16:56:54.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.061+0000 7f7c67200640 1 -- 172.21.15.67:0/3562552805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c680d0740 msgr2=0x7f7c680b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:54.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.061+0000 7f7c67200640 1 --2- 172.21.15.67:0/3562552805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c680d0740 0x7f7c680b3190 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009a00 tx=0x7f7c5402a390 comp rx=0 tx=0).stop 2024-05-30T16:56:54.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.062+0000 7f7c67200640 1 -- 172.21.15.67:0/3562552805 shutdown_connections 2024-05-30T16:56:54.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.062+0000 7f7c67200640 1 --2- 172.21.15.67:0/3562552805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c680d0740 0x7f7c680b3190 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:54.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.062+0000 7f7c67200640 1 -- 172.21.15.67:0/3562552805 >> 172.21.15.67:0/3562552805 conn(0x7f7c680b55d0 msgr2=0x7f7c680b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:54.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.066+0000 7f7c67200640 1 -- 172.21.15.67:0/3562552805 shutdown_connections 2024-05-30T16:56:54.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.068+0000 7f7c67200640 1 -- 172.21.15.67:0/3562552805 wait complete. 2024-05-30T16:56:54.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.073+0000 7f7c67200640 1 Processor -- start 2024-05-30T16:56:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.074+0000 7f7c67200640 1 -- start start 2024-05-30T16:56:54.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.077+0000 7f7c67200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c680b36d0 0x7f7c681faff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:54.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.078+0000 7f7c67200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c680d0740 0x7f7c680c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:54.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.079+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c680b36d0 0x7f7c681faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:54.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.079+0000 7f7c67200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c6815c200 0x7f7c682021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:54.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.079+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c680b36d0 0x7f7c681faff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:37954/0 (socket says 172.21.15.67:37954) 2024-05-30T16:56:54.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.080+0000 7f7c65400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c680d0740 0x7f7c680c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:54.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.080+0000 7f7c65400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c680d0740 0x7f7c680c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:34218/0 (socket says 172.21.15.67:34218) 2024-05-30T16:56:54.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.080+0000 7f7c67200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c680b3b20 con 0x7f7c6815c200 2024-05-30T16:56:54.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.080+0000 7f7c67200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6815c5e0 con 0x7f7c680b36d0 2024-05-30T16:56:54.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.080+0000 7f7c67200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c680b3ca0 con 0x7f7c680d0740 2024-05-30T16:56:54.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.081+0000 7f7c65e00640 1 -- 172.21.15.67:0/3275368800 learned_addr learned my addr 172.21.15.67:0/3275368800 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:54.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.084+0000 7f7c66800640 1 --2- 172.21.15.67:0/3275368800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c6815c200 0x7f7c682021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:54.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.085+0000 7f7c65400640 1 -- 172.21.15.67:0/3275368800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c680b36d0 msgr2=0x7f7c681faff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:54.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.085+0000 7f7c65400640 1 --2- 172.21.15.67:0/3275368800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c680b36d0 0x7f7c681faff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:54.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.085+0000 7f7c65400640 1 -- 172.21.15.67:0/3275368800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c6815c200 msgr2=0x7f7c682021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:54.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.085+0000 7f7c65400640 1 --2- 172.21.15.67:0/3275368800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c6815c200 0x7f7c682021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:54.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.086+0000 7f7c65400640 1 -- 172.21.15.67:0/3275368800 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c680c5b90 con 0x7f7c680d0740 2024-05-30T16:56:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.087+0000 7f7c65e00640 1 --2- 172.21.15.67:0/3275368800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c680b36d0 0x7f7c681faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:56:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.087+0000 7f7c66800640 1 --2- 172.21.15.67:0/3275368800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c6815c200 0x7f7c682021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:56:54.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.089+0000 7f7c65400640 1 --2- 172.21.15.67:0/3275368800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c680d0740 0x7f7c680c8840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7c54031000 tx=0x7f7c540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.090+0000 7f7c5e000640 1 -- 172.21.15.67:0/3275368800 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54037040 con 0x7f7c680d0740 2024-05-30T16:56:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.090+0000 7f7c5e000640 1 -- 172.21.15.67:0/3275368800 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54032e50 con 0x7f7c680d0740 2024-05-30T16:56:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.090+0000 7f7c5e000640 1 -- 172.21.15.67:0/3275368800 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54004460 con 0x7f7c680d0740 2024-05-30T16:56:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.090+0000 7f7c67200640 1 -- 172.21.15.67:0/3275368800 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c680c5e70 con 0x7f7c680d0740 2024-05-30T16:56:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.090+0000 7f7c67200640 1 -- 172.21.15.67:0/3275368800 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c680c6430 con 0x7f7c680d0740 2024-05-30T16:56:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.100+0000 7f7c67200640 1 -- 172.21.15.67:0/3275368800 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c680754f0 con 0x7f7c680d0740 2024-05-30T16:56:54.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.103+0000 7f7c5e000640 1 -- 172.21.15.67:0/3275368800 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f7c5403a070 con 0x7f7c680d0740 2024-05-30T16:56:54.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.106+0000 7f7c5e000640 1 --2- 172.21.15.67:0/3275368800 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c4806da50 0x7f7c4806ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:54.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.106+0000 7f7c5e000640 1 -- 172.21.15.67:0/3275368800 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f7c540ab810 con 0x7f7c680d0740 2024-05-30T16:56:54.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.112+0000 7f7c5e000640 1 -- 172.21.15.67:0/3275368800 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c54041030 con 0x7f7c680d0740 2024-05-30T16:56:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.132+0000 7f7c65e00640 1 --2- 172.21.15.67:0/3275368800 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c4806da50 0x7f7c4806ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:54.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.181+0000 7f7c65e00640 1 --2- 172.21.15.67:0/3275368800 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c4806da50 0x7f7c4806ff10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005880 tx=0x7f7c58009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:54.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:54.719+0000 7f7c67200640 1 -- 172.21.15.67:0/3275368800 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92287", "target": ["mon-mgr", ""]}) -- 0x7f7c68074b30 con 0x7f7c4806da50 2024-05-30T16:56:55.524 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:55.523+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.532+0000 7f7c5e000640 1 -- 172.21.15.67:0/3275368800 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f7c68074b30 con 0x7f7c4806da50 2024-05-30T16:56:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:55.562 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:55.563 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92288 2024-05-30T16:56:55.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.981+0000 7f7921600640 1 Processor -- start 2024-05-30T16:56:55.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.981+0000 7f7921600640 1 -- start start 2024-05-30T16:56:55.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.987+0000 7f7921600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f791c1581a0 0x7f791c072780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:55.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.987+0000 7f7921600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c06ecb0 0x7f791c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:55.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.987+0000 7f7921600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f791c072cc0 con 0x7f791c0d0770 2024-05-30T16:56:55.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.987+0000 7f7921600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f791c0b3660 con 0x7f791c1581a0 2024-05-30T16:56:55.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.987+0000 7f7921600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f791c0b3940 con 0x7f791c06ecb0 2024-05-30T16:56:55.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.989+0000 7f791a000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f791c0d0770 0x7f791c15e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:55468/0 (socket says 172.21.15.67:55468) 2024-05-30T16:56:55.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.989+0000 7f791a000640 1 -- 172.21.15.67:0/1600411492 learned_addr learned my addr 172.21.15.67:0/1600411492 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:55.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.990+0000 7f791b400640 1 --2- 172.21.15.67:0/1600411492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c06ecb0 0x7f791c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:55.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.991+0000 7f791aa00640 1 --2- 172.21.15.67:0/1600411492 >> v2:172.21.15.67:3300/0 conn(0x7f791c1581a0 0x7f791c072780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:55.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.992+0000 7f7919600640 1 -- 172.21.15.67:0/1600411492 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1630181778 0 0) 0x7f791c072cc0 con 0x7f791c0d0770 2024-05-30T16:56:55.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.992+0000 7f7919600640 1 -- 172.21.15.67:0/1600411492 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7900003520 con 0x7f791c0d0770 2024-05-30T16:56:55.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.993+0000 7f791b400640 1 -- 172.21.15.67:0/1600411492 >> v2:172.21.15.67:3300/0 conn(0x7f791c1581a0 msgr2=0x7f791c072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:55.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.993+0000 7f791b400640 1 --2- 172.21.15.67:0/1600411492 >> v2:172.21.15.67:3300/0 conn(0x7f791c1581a0 0x7f791c072780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:55.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.993+0000 7f791b400640 1 -- 172.21.15.67:0/1600411492 >> v1:172.21.15.169:6789/0 conn(0x7f791c0d0770 legacy=0x7f791c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:55.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.993+0000 7f791b400640 1 -- 172.21.15.67:0/1600411492 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c0b3c20 con 0x7f791c06ecb0 2024-05-30T16:56:55.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.993+0000 7f791aa00640 1 --2- 172.21.15.67:0/1600411492 >> v2:172.21.15.67:3300/0 conn(0x7f791c1581a0 0x7f791c072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:56:55.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.995+0000 7f791b400640 1 --2- 172.21.15.67:0/1600411492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c06ecb0 0x7f791c0d0c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7914009d90 tx=0x7f791402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58464932fbc7a2e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:55.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.996+0000 7f7919600640 1 -- 172.21.15.67:0/1600411492 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914035020 con 0x7f791c06ecb0 2024-05-30T16:56:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.997+0000 7f7919600640 1 -- 172.21.15.67:0/1600411492 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7914007540 con 0x7f791c06ecb0 2024-05-30T16:56:55.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:55.997+0000 7f7919600640 1 -- 172.21.15.67:0/1600411492 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908001650 con 0x7f791c06ecb0 2024-05-30T16:56:56.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.000+0000 7f7921600640 1 -- 172.21.15.67:0/1600411492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c06ecb0 msgr2=0x7f791c0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:56.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.000+0000 7f7921600640 1 --2- 172.21.15.67:0/1600411492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c06ecb0 0x7f791c0d0c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7914009d90 tx=0x7f791402a5d0 comp rx=0 tx=0).stop 2024-05-30T16:56:56.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.001+0000 7f7921600640 1 -- 172.21.15.67:0/1600411492 shutdown_connections 2024-05-30T16:56:56.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.001+0000 7f7921600640 1 --2- 172.21.15.67:0/1600411492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c06ecb0 0x7f791c0d0c20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:56.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.001+0000 7f7921600640 1 --2- 172.21.15.67:0/1600411492 >> v2:172.21.15.67:3300/0 conn(0x7f791c1581a0 0x7f791c072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:56.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.001+0000 7f7921600640 1 -- 172.21.15.67:0/1600411492 >> 172.21.15.67:0/1600411492 conn(0x7f791c06bd20 msgr2=0x7f791c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.001+0000 7f7921600640 1 -- 172.21.15.67:0/1600411492 shutdown_connections 2024-05-30T16:56:56.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.001+0000 7f7921600640 1 -- 172.21.15.67:0/1600411492 wait complete. 2024-05-30T16:56:56.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.006+0000 7f7921600640 1 Processor -- start 2024-05-30T16:56:56.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.007+0000 7f7921600640 1 -- start start 2024-05-30T16:56:56.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.010+0000 7f7921600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f791c06ecb0 0x7f791c14d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:56.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.011+0000 7f7921600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c0d0770 0x7f791c099570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:56.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.011+0000 7f7921600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f791c1581a0 0x7f791c099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:56.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.011+0000 7f7921600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f791c0b32c0 con 0x7f791c1581a0 2024-05-30T16:56:56.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.011+0000 7f7921600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f791c0b41e0 con 0x7f791c06ecb0 2024-05-30T16:56:56.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.011+0000 7f7921600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f791c0b3440 con 0x7f791c0d0770 2024-05-30T16:56:56.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.012+0000 7f791a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c0d0770 0x7f791c099570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:56.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.012+0000 7f791a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c0d0770 0x7f791c099570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:34242/0 (socket says 172.21.15.67:34242) 2024-05-30T16:56:56.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.012+0000 7f791a000640 1 -- 172.21.15.67:0/1749797862 learned_addr learned my addr 172.21.15.67:0/1749797862 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:56.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.013+0000 7f791b400640 1 --2- 172.21.15.67:0/1749797862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f791c1581a0 0x7f791c099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.017+0000 7f791a000640 1 -- 172.21.15.67:0/1749797862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f791c06ecb0 msgr2=0x7f791c14d590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T16:56:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.017+0000 7f791a000640 1 --2- 172.21.15.67:0/1749797862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f791c06ecb0 0x7f791c14d590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.017+0000 7f791a000640 1 -- 172.21.15.67:0/1749797862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f791c1581a0 msgr2=0x7f791c099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.017+0000 7f791a000640 1 --2- 172.21.15.67:0/1749797862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f791c1581a0 0x7f791c099ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.017+0000 7f791a000640 1 -- 172.21.15.67:0/1749797862 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c0b2920 con 0x7f791c0d0770 2024-05-30T16:56:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.026+0000 7f791a000640 1 --2- 172.21.15.67:0/1749797862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791c0d0770 0x7f791c099570 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f790800caf0 tx=0x7f7908001d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.030+0000 7f7912a00640 1 -- 172.21.15.67:0/1749797862 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908017070 con 0x7f791c0d0770 2024-05-30T16:56:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.030+0000 7f7912a00640 1 -- 172.21.15.67:0/1749797862 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7908005ce0 con 0x7f791c0d0770 2024-05-30T16:56:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.030+0000 7f7912a00640 1 -- 172.21.15.67:0/1749797862 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79080072b0 con 0x7f791c0d0770 2024-05-30T16:56:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.030+0000 7f7921600640 1 -- 172.21.15.67:0/1749797862 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c0b2c00 con 0x7f791c0d0770 2024-05-30T16:56:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.030+0000 7f7921600640 1 -- 172.21.15.67:0/1749797862 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c1501e0 con 0x7f791c0d0770 2024-05-30T16:56:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.035+0000 7f7921600640 1 -- 172.21.15.67:0/1749797862 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f4005290 con 0x7f791c0d0770 2024-05-30T16:56:56.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.038+0000 7f7912a00640 1 -- 172.21.15.67:0/1749797862 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f790801a070 con 0x7f791c0d0770 2024-05-30T16:56:56.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.039+0000 7f7912a00640 1 --2- 172.21.15.67:0/1749797862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f78fc06daa0 0x7f78fc06ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:56.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.040+0000 7f7912a00640 1 -- 172.21.15.67:0/1749797862 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f790808a160 con 0x7f791c0d0770 2024-05-30T16:56:56.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.049+0000 7f7912a00640 1 -- 172.21.15.67:0/1749797862 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7908059ca0 con 0x7f791c0d0770 2024-05-30T16:56:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.062+0000 7f791aa00640 1 --2- 172.21.15.67:0/1749797862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f78fc06daa0 0x7f78fc06ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:56.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.129+0000 7f791aa00640 1 --2- 172.21.15.67:0/1749797862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f78fc06daa0 0x7f78fc06ff60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f790c00fc20 tx=0x7f790c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:56.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:56.595+0000 7f7921600640 1 -- 172.21.15.67:0/1749797862 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92288", "target": ["mon-mgr", ""]}) -- 0x7f78f4000bc0 con 0x7f78fc06daa0 2024-05-30T16:56:57.026 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:57.025+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:57.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.058+0000 7f7912a00640 1 -- 172.21.15.67:0/1749797862 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f78f4000bc0 con 0x7f78fc06daa0 2024-05-30T16:56:57.058 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:57.113 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:57.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92289 2024-05-30T16:56:57.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.666+0000 7fc698600640 1 Processor -- start 2024-05-30T16:56:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.668+0000 7fc698600640 1 -- start start 2024-05-30T16:56:57.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.671+0000 7fc698600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc69015ed80 0x7fc6900cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:57.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.671+0000 7fc698600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6900b68f0 0x7fc6900cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:57.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.671+0000 7fc698600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69015f160 con 0x7fc6900b7e50 2024-05-30T16:56:57.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.671+0000 7fc698600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc690163540 con 0x7fc6900b68f0 2024-05-30T16:56:57.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.671+0000 7fc698600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc690162930 con 0x7fc69015ed80 2024-05-30T16:56:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.673+0000 7fc696600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6900b68f0 0x7fc6900cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.673+0000 7fc696600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6900b68f0 0x7fc6900cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54954/0 (socket says 172.21.15.67:54954) 2024-05-30T16:56:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.673+0000 7fc696600640 1 -- 172.21.15.67:0/3636604634 learned_addr learned my addr 172.21.15.67:0/3636604634 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.675+0000 7fc695c00640 1 --2- 172.21.15.67:0/3636604634 >> v2:172.21.15.169:3300/0 conn(0x7fc69015ed80 0x7fc6900cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.675+0000 7fc696600640 1 -- 172.21.15.67:0/3636604634 >> v2:172.21.15.169:3300/0 conn(0x7fc69015ed80 msgr2=0x7fc6900cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.675+0000 7fc696600640 1 --2- 172.21.15.67:0/3636604634 >> v2:172.21.15.169:3300/0 conn(0x7fc69015ed80 0x7fc6900cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.675+0000 7fc696600640 1 -- 172.21.15.67:0/3636604634 >> v1:172.21.15.169:6789/0 conn(0x7fc6900b7e50 legacy=0x7fc6900b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:56:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.675+0000 7fc696600640 1 -- 172.21.15.67:0/3636604634 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc690162c10 con 0x7fc6900b68f0 2024-05-30T16:56:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.677+0000 7fc696600640 1 --2- 172.21.15.67:0/3636604634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6900b68f0 0x7fc6900cedb0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc69008cc30 tx=0x7fc68802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d2cd31fbf32c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.679+0000 7fc68fe00640 1 -- 172.21.15.67:0/3636604634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688036070 con 0x7fc6900b68f0 2024-05-30T16:56:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.679+0000 7fc68fe00640 1 -- 172.21.15.67:0/3636604634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc688031600 con 0x7fc6900b68f0 2024-05-30T16:56:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.679+0000 7fc68fe00640 1 -- 172.21.15.67:0/3636604634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6880319d0 con 0x7fc6900b68f0 2024-05-30T16:56:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.680+0000 7fc698600640 1 -- 172.21.15.67:0/3636604634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6900b68f0 msgr2=0x7fc6900cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.680+0000 7fc698600640 1 --2- 172.21.15.67:0/3636604634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6900b68f0 0x7fc6900cedb0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc69008cc30 tx=0x7fc68802a6f0 comp rx=0 tx=0).stop 2024-05-30T16:56:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.680+0000 7fc698600640 1 -- 172.21.15.67:0/3636604634 shutdown_connections 2024-05-30T16:56:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.681+0000 7fc698600640 1 --2- 172.21.15.67:0/3636604634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6900b68f0 0x7fc6900cedb0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.681+0000 7fc698600640 1 --2- 172.21.15.67:0/3636604634 >> v2:172.21.15.169:3300/0 conn(0x7fc69015ed80 0x7fc6900cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.681+0000 7fc698600640 1 -- 172.21.15.67:0/3636604634 >> 172.21.15.67:0/3636604634 conn(0x7fc6900d13c0 msgr2=0x7fc6900d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.681+0000 7fc698600640 1 -- 172.21.15.67:0/3636604634 shutdown_connections 2024-05-30T16:56:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.681+0000 7fc698600640 1 -- 172.21.15.67:0/3636604634 wait complete. 2024-05-30T16:56:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.685+0000 7fc698600640 1 Processor -- start 2024-05-30T16:56:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.686+0000 7fc698600640 1 -- start start 2024-05-30T16:56:57.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.689+0000 7fc698600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6900b68f0 0x7fc69015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:57.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.690+0000 7fc698600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6900b7e50 0x7fc69014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:57.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.690+0000 7fc698600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc69015ed80 0x7fc6900b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:57.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.690+0000 7fc698600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc6900677b0 con 0x7fc6900b68f0 2024-05-30T16:56:57.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.690+0000 7fc698600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901631d0 con 0x7fc69015ed80 2024-05-30T16:56:57.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.690+0000 7fc698600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc690067930 con 0x7fc6900b7e50 2024-05-30T16:56:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.692+0000 7fc695c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6900b68f0 0x7fc69015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.692+0000 7fc695200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6900b7e50 0x7fc69014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.692+0000 7fc695200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6900b7e50 0x7fc69014fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:34258/0 (socket says 172.21.15.67:34258) 2024-05-30T16:56:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.692+0000 7fc695200640 1 -- 172.21.15.67:0/219595279 learned_addr learned my addr 172.21.15.67:0/219595279 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.693+0000 7fc696600640 1 --2- 172.21.15.67:0/219595279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc69015ed80 0x7fc6900b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.694+0000 7fc695c00640 1 -- 172.21.15.67:0/219595279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6900b7e50 msgr2=0x7fc69014fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.695+0000 7fc695c00640 1 --2- 172.21.15.67:0/219595279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6900b7e50 0x7fc69014fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.695+0000 7fc695c00640 1 -- 172.21.15.67:0/219595279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc69015ed80 msgr2=0x7fc6900b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.695+0000 7fc695c00640 1 --2- 172.21.15.67:0/219595279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc69015ed80 0x7fc6900b8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.695+0000 7fc695c00640 1 -- 172.21.15.67:0/219595279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6900cdaf0 con 0x7fc6900b68f0 2024-05-30T16:56:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.696+0000 7fc695c00640 1 --2- 172.21.15.67:0/219595279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6900b68f0 0x7fc69015dea0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc68400c5f0 tx=0x7fc68400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.697+0000 7fc68d600640 1 -- 172.21.15.67:0/219595279 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684015070 con 0x7fc6900b68f0 2024-05-30T16:56:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.697+0000 7fc68d600640 1 -- 172.21.15.67:0/219595279 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc684011800 con 0x7fc6900b68f0 2024-05-30T16:56:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.697+0000 7fc68d600640 1 -- 172.21.15.67:0/219595279 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684011bd0 con 0x7fc6900b68f0 2024-05-30T16:56:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.697+0000 7fc698600640 1 -- 172.21.15.67:0/219595279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6900cddd0 con 0x7fc6900b68f0 2024-05-30T16:56:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.698+0000 7fc698600640 1 -- 172.21.15.67:0/219595279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6900ce390 con 0x7fc6900b68f0 2024-05-30T16:56:57.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.707+0000 7fc698600640 1 -- 172.21.15.67:0/219595279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66c005290 con 0x7fc6900b68f0 2024-05-30T16:56:57.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.708+0000 7fc68d600640 1 -- 172.21.15.67:0/219595279 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7fc684018070 con 0x7fc6900b68f0 2024-05-30T16:56:57.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.709+0000 7fc68d600640 1 --2- 172.21.15.67:0/219595279 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc67006da20 0x7fc67006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:57.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.710+0000 7fc68d600640 1 -- 172.21.15.67:0/219595279 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fc68408a740 con 0x7fc6900b68f0 2024-05-30T16:56:57.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.720+0000 7fc68d600640 1 -- 172.21.15.67:0/219595279 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc684059c30 con 0x7fc6900b68f0 2024-05-30T16:56:57.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.725+0000 7fc695200640 1 --2- 172.21.15.67:0/219595279 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc67006da20 0x7fc67006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:57.767+0000 7fc695200640 1 --2- 172.21.15.67:0/219595279 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc67006da20 0x7fc67006fee0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc69015a830 tx=0x7fc67c007ef0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:58.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.332+0000 7fc698600640 1 -- 172.21.15.67:0/219595279 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92289", "target": ["mon-mgr", ""]}) -- 0x7fc66c000bc0 con 0x7fc67006da20 2024-05-30T16:56:58.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.435+0000 7fc68d600640 1 -- 172.21.15.67:0/219595279 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fc66c000bc0 con 0x7fc67006da20 2024-05-30T16:56:58.435 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:58.437 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:58.434+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:58.462 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:58.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922810 2024-05-30T16:56:58.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.805+0000 7f1635800640 1 Processor -- start 2024-05-30T16:56:58.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.805+0000 7f1635800640 1 -- start start 2024-05-30T16:56:58.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.809+0000 7f1635800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0c00 0x7f163006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:58.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.809+0000 7f1635800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16300b3660 con 0x7f16300d0770 2024-05-30T16:56:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.809+0000 7f1635800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16300b3940 con 0x7f16301581a0 2024-05-30T16:56:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.809+0000 7f1635800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16300b3c20 con 0x7f16300d0c00 2024-05-30T16:56:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.809+0000 7f162ea00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f16301581a0 0x7f1630072780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:55488/0 (socket says 172.21.15.67:55488) 2024-05-30T16:56:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.809+0000 7f162e000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f16300d0770 0x7f163006dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36670/0 (socket says 172.21.15.67:36670) 2024-05-30T16:56:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.809+0000 7f162ea00640 1 -- 172.21.15.67:0/2577674886 learned_addr learned my addr 172.21.15.67:0/2577674886 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.809+0000 7f162f400640 1 --2- 172.21.15.67:0/2577674886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0c00 0x7f163006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.811+0000 7f162f400640 1 -- 172.21.15.67:0/2577674886 >> v1:172.21.15.169:6789/0 conn(0x7f16301581a0 legacy=0x7f1630072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.811+0000 7f162f400640 1 -- 172.21.15.67:0/2577674886 >> v1:172.21.15.67:6789/0 conn(0x7f16300d0770 legacy=0x7f163006dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:58.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.811+0000 7f162f400640 1 -- 172.21.15.67:0/2577674886 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16300b3f00 con 0x7f16300d0c00 2024-05-30T16:56:58.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.812+0000 7f162f400640 1 --2- 172.21.15.67:0/2577674886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0c00 0x7f163006e3b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1628009dc0 tx=0x7f162802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73892bde97574a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:58.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.813+0000 7f162d600640 1 -- 172.21.15.67:0/2577674886 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628007740 con 0x7f16300d0c00 2024-05-30T16:56:58.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.813+0000 7f162d600640 1 -- 172.21.15.67:0/2577674886 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16280078e0 con 0x7f16300d0c00 2024-05-30T16:56:58.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.813+0000 7f162d600640 1 -- 172.21.15.67:0/2577674886 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628007cb0 con 0x7f16300d0c00 2024-05-30T16:56:58.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.814+0000 7f1635800640 1 -- 172.21.15.67:0/2577674886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0c00 msgr2=0x7f163006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:58.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.814+0000 7f1635800640 1 --2- 172.21.15.67:0/2577674886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0c00 0x7f163006e3b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1628009dc0 tx=0x7f162802a6d0 comp rx=0 tx=0).stop 2024-05-30T16:56:58.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.815+0000 7f1635800640 1 -- 172.21.15.67:0/2577674886 shutdown_connections 2024-05-30T16:56:58.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.815+0000 7f1635800640 1 --2- 172.21.15.67:0/2577674886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0c00 0x7f163006e3b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:58.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.815+0000 7f1635800640 1 -- 172.21.15.67:0/2577674886 >> 172.21.15.67:0/2577674886 conn(0x7f163006bd20 msgr2=0x7f163006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:58.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.815+0000 7f1635800640 1 -- 172.21.15.67:0/2577674886 shutdown_connections 2024-05-30T16:56:58.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.815+0000 7f1635800640 1 -- 172.21.15.67:0/2577674886 wait complete. 2024-05-30T16:56:58.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.818+0000 7f1635800640 1 Processor -- start 2024-05-30T16:56:58.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.818+0000 7f1635800640 1 -- start start 2024-05-30T16:56:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.820+0000 7f1635800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0770 0x7f163015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.820+0000 7f1635800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16300d0c00 0x7f163013c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.820+0000 7f1635800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16301581a0 0x7f16301f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.820+0000 7f162e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16300d0c00 0x7f163013c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.820+0000 7f162e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16300d0c00 0x7f163013c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:38004/0 (socket says 172.21.15.67:38004) 2024-05-30T16:56:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.820+0000 7f162ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0770 0x7f163015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.820+0000 7f162ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0770 0x7f163015c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:34276/0 (socket says 172.21.15.67:34276) 2024-05-30T16:56:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.821+0000 7f1635800640 1 -- 172.21.15.67:0/464809123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f16300b32c0 con 0x7f16301581a0 2024-05-30T16:56:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.821+0000 7f162e000640 1 -- 172.21.15.67:0/464809123 learned_addr learned my addr 172.21.15.67:0/464809123 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.821+0000 7f1635800640 1 -- 172.21.15.67:0/464809123 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16300d1040 con 0x7f16300d0c00 2024-05-30T16:56:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.821+0000 7f1635800640 1 -- 172.21.15.67:0/464809123 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16300b3440 con 0x7f16300d0770 2024-05-30T16:56:58.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.822+0000 7f162f400640 1 --2- 172.21.15.67:0/464809123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16301581a0 0x7f16301f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:58.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.822+0000 7f162ea00640 1 -- 172.21.15.67:0/464809123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16300d0c00 msgr2=0x7f163013c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:58.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.822+0000 7f162ea00640 1 --2- 172.21.15.67:0/464809123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16300d0c00 0x7f163013c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:58.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.822+0000 7f162ea00640 1 -- 172.21.15.67:0/464809123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16301581a0 msgr2=0x7f16301f7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:58.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.822+0000 7f162ea00640 1 --2- 172.21.15.67:0/464809123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16301581a0 0x7f16301f7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:58.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.822+0000 7f162ea00640 1 -- 172.21.15.67:0/464809123 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163015c800 con 0x7f16300d0770 2024-05-30T16:56:58.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.823+0000 7f162ea00640 1 --2- 172.21.15.67:0/464809123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16300d0770 0x7f163015c220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f161c00c440 tx=0x7f161c00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:58.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.823+0000 7f1626a00640 1 -- 172.21.15.67:0/464809123 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c017070 con 0x7f16300d0770 2024-05-30T16:56:58.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.823+0000 7f1626a00640 1 -- 172.21.15.67:0/464809123 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c005ce0 con 0x7f16300d0770 2024-05-30T16:56:58.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.823+0000 7f1626a00640 1 -- 172.21.15.67:0/464809123 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0072f0 con 0x7f16300d0770 2024-05-30T16:56:58.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.824+0000 7f1635800640 1 -- 172.21.15.67:0/464809123 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163009afb0 con 0x7f16300d0770 2024-05-30T16:56:58.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.824+0000 7f1635800640 1 -- 172.21.15.67:0/464809123 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1630099b60 con 0x7f16300d0770 2024-05-30T16:56:58.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.825+0000 7f1635800640 1 -- 172.21.15.67:0/464809123 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16300689e0 con 0x7f16300d0770 2024-05-30T16:56:58.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.827+0000 7f1626a00640 1 -- 172.21.15.67:0/464809123 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f161c01a070 con 0x7f16300d0770 2024-05-30T16:56:58.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.828+0000 7f1626a00640 1 --2- 172.21.15.67:0/464809123 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f161006d9d0 0x7f161006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:58.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.828+0000 7f1626a00640 1 -- 172.21.15.67:0/464809123 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f161c089cf0 con 0x7f16300d0770 2024-05-30T16:56:58.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.829+0000 7f162e000640 1 --2- 172.21.15.67:0/464809123 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f161006d9d0 0x7f161006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:58.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.831+0000 7f162e000640 1 --2- 172.21.15.67:0/464809123 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f161006d9d0 0x7f161006fe90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f163013d680 tx=0x7f1620007f10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:58.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:58.840+0000 7f1626a00640 1 -- 172.21.15.67:0/464809123 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f161c0591e0 con 0x7f16300d0770 2024-05-30T16:56:59.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.309+0000 7f1635800640 1 -- 172.21.15.67:0/464809123 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922810", "target": ["mon-mgr", ""]}) -- 0x7f163013cc60 con 0x7f161006d9d0 2024-05-30T16:56:59.448 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:56:59.447+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:59.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.453+0000 7f1626a00640 1 -- 172.21.15.67:0/464809123 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f163013cc60 con 0x7f161006d9d0 2024-05-30T16:56:59.453 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:56:59.472 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:56:59.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922811 2024-05-30T16:56:59.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.784+0000 7f4203e00640 1 Processor -- start 2024-05-30T16:56:59.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.784+0000 7f4203e00640 1 -- start start 2024-05-30T16:56:59.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.786+0000 7f4203e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42040d16a0 0x7f42040cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:59.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.786+0000 7f4203e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f420415ec50 con 0x7f420415e870 2024-05-30T16:56:59.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.786+0000 7f4203e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42040cff80 con 0x7f42040cf870 2024-05-30T16:56:59.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.786+0000 7f4203e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42040d2150 con 0x7f42040d16a0 2024-05-30T16:56:59.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.787+0000 7f4202a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f420415e870 0x7f42040d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36682/0 (socket says 172.21.15.67:36682) 2024-05-30T16:56:59.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.787+0000 7f4202a00640 1 -- 172.21.15.67:0/1858960329 learned_addr learned my addr 172.21.15.67:0/1858960329 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:56:59.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.790+0000 7f4201600640 1 -- 172.21.15.67:0/1858960329 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3928702029 0 0) 0x7f420415ec50 con 0x7f420415e870 2024-05-30T16:56:59.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.791+0000 7f4202000640 1 --2- 172.21.15.67:0/1858960329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42040d16a0 0x7f42040cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:59.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.791+0000 7f4201600640 1 -- 172.21.15.67:0/1858960329 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41e8003520 con 0x7f420415e870 2024-05-30T16:56:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.792+0000 7f4201600640 1 -- 172.21.15.67:0/1858960329 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1175520625 0 0) 0x7f42040cff80 con 0x7f42040cf870 2024-05-30T16:56:59.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.792+0000 7f4201600640 1 -- 172.21.15.67:0/1858960329 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f420415ec50 con 0x7f42040cf870 2024-05-30T16:56:59.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.793+0000 7f4202000640 1 -- 172.21.15.67:0/1858960329 >> v1:172.21.15.169:6789/0 conn(0x7f42040cf870 legacy=0x7f42040d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:59.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.793+0000 7f4202000640 1 -- 172.21.15.67:0/1858960329 >> v1:172.21.15.67:6789/0 conn(0x7f420415e870 legacy=0x7f42040d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:59.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.793+0000 7f4202000640 1 -- 172.21.15.67:0/1858960329 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42040d0260 con 0x7f42040d16a0 2024-05-30T16:56:59.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.796+0000 7f4202000640 1 --2- 172.21.15.67:0/1858960329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42040d16a0 0x7f42040cf330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f41f40097c0 tx=0x7f41f402a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a423d849ca0cd74d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:59.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.797+0000 7f4201600640 1 -- 172.21.15.67:0/1858960329 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4037070 con 0x7f42040d16a0 2024-05-30T16:56:59.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.797+0000 7f4201600640 1 -- 172.21.15.67:0/1858960329 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f4032440 con 0x7f42040d16a0 2024-05-30T16:56:59.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.797+0000 7f4201600640 1 -- 172.21.15.67:0/1858960329 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4032810 con 0x7f42040d16a0 2024-05-30T16:56:59.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.798+0000 7f4203e00640 1 -- 172.21.15.67:0/1858960329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42040d16a0 msgr2=0x7f42040cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:59.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.798+0000 7f4203e00640 1 --2- 172.21.15.67:0/1858960329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42040d16a0 0x7f42040cf330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f41f40097c0 tx=0x7f41f402a1f0 comp rx=0 tx=0).stop 2024-05-30T16:56:59.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.803+0000 7f4203e00640 1 -- 172.21.15.67:0/1858960329 shutdown_connections 2024-05-30T16:56:59.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.803+0000 7f4203e00640 1 --2- 172.21.15.67:0/1858960329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42040d16a0 0x7f42040cf330 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:59.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.803+0000 7f4203e00640 1 -- 172.21.15.67:0/1858960329 >> 172.21.15.67:0/1858960329 conn(0x7f420408bfc0 msgr2=0x7f42040d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:56:59.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.803+0000 7f4203e00640 1 -- 172.21.15.67:0/1858960329 shutdown_connections 2024-05-30T16:56:59.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.803+0000 7f4203e00640 1 -- 172.21.15.67:0/1858960329 wait complete. 2024-05-30T16:56:59.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.806+0000 7f4203e00640 1 Processor -- start 2024-05-30T16:56:59.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.807+0000 7f4203e00640 1 -- start start 2024-05-30T16:56:59.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.809+0000 7f4203e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42040cf870 0x7f4204159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:59.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.809+0000 7f4203e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42040d16a0 0x7f42040b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:59.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.809+0000 7f4203e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f420415e870 0x7f42040b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:59.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.810+0000 7f4203e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f42040d0820 con 0x7f42040cf870 2024-05-30T16:56:59.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.810+0000 7f4203e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42040cfd20 con 0x7f42040d16a0 2024-05-30T16:56:59.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.810+0000 7f4203e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42040d4fb0 con 0x7f420415e870 2024-05-30T16:56:59.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.811+0000 7f4203400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f420415e870 0x7f42040b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:59.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.811+0000 7f4203400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f420415e870 0x7f42040b8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:34304/0 (socket says 172.21.15.67:34304) 2024-05-30T16:56:59.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.811+0000 7f4203400640 1 -- 172.21.15.67:0/3883075723 learned_addr learned my addr 172.21.15.67:0/3883075723 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:56:59.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.811+0000 7f4202000640 1 --2- 172.21.15.67:0/3883075723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42040d16a0 0x7f42040b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:59.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.811+0000 7f4202a00640 1 --2- 172.21.15.67:0/3883075723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42040cf870 0x7f4204159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:59.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.813+0000 7f4202a00640 1 -- 172.21.15.67:0/3883075723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f420415e870 msgr2=0x7f42040b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:59.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.813+0000 7f4202a00640 1 --2- 172.21.15.67:0/3883075723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f420415e870 0x7f42040b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:59.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.813+0000 7f4202a00640 1 -- 172.21.15.67:0/3883075723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42040d16a0 msgr2=0x7f42040b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:56:59.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.813+0000 7f4202a00640 1 --2- 172.21.15.67:0/3883075723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42040d16a0 0x7f42040b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:56:59.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.813+0000 7f4202a00640 1 -- 172.21.15.67:0/3883075723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42040b73f0 con 0x7f42040cf870 2024-05-30T16:56:59.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.815+0000 7f4202a00640 1 --2- 172.21.15.67:0/3883075723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42040cf870 0x7f4204159c00 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f41f000c3f0 tx=0x7f41f0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:56:59.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.816+0000 7f41faa00640 1 -- 172.21.15.67:0/3883075723 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0017070 con 0x7f42040cf870 2024-05-30T16:56:59.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.816+0000 7f41faa00640 1 -- 172.21.15.67:0/3883075723 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f0005ce0 con 0x7f42040cf870 2024-05-30T16:56:59.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.816+0000 7f41faa00640 1 -- 172.21.15.67:0/3883075723 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00072d0 con 0x7f42040cf870 2024-05-30T16:56:59.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.818+0000 7f4203e00640 1 -- 172.21.15.67:0/3883075723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42040b76d0 con 0x7f42040cf870 2024-05-30T16:56:59.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.818+0000 7f4203e00640 1 -- 172.21.15.67:0/3883075723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42040b7c90 con 0x7f42040cf870 2024-05-30T16:56:59.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.823+0000 7f4203e00640 1 -- 172.21.15.67:0/3883075723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42040689e0 con 0x7f42040cf870 2024-05-30T16:56:59.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.827+0000 7f41faa00640 1 -- 172.21.15.67:0/3883075723 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f41f001a070 con 0x7f42040cf870 2024-05-30T16:56:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.828+0000 7f41faa00640 1 --2- 172.21.15.67:0/3883075723 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f41e406daa0 0x7f41e406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:56:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.829+0000 7f41faa00640 1 -- 172.21.15.67:0/3883075723 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f41f0089d50 con 0x7f42040cf870 2024-05-30T16:56:59.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.834+0000 7f4202000640 1 --2- 172.21.15.67:0/3883075723 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f41e406daa0 0x7f41e406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:56:59.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.836+0000 7f41faa00640 1 -- 172.21.15.67:0/3883075723 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41f0059240 con 0x7f42040cf870 2024-05-30T16:56:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:56:59.838+0000 7f4202000640 1 --2- 172.21.15.67:0/3883075723 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f41e406daa0 0x7f41e406ff60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f41f4034000 tx=0x7f41f402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:00.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:00.344+0000 7f4203e00640 1 -- 172.21.15.67:0/3883075723 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922811", "target": ["mon-mgr", ""]}) -- 0x7f42040b71a0 con 0x7f41e406daa0 2024-05-30T16:57:00.633 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:00.628+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:57:00.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:00.644+0000 7f41faa00640 1 -- 172.21.15.67:0/3883075723 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f42040b71a0 con 0x7f41e406daa0 2024-05-30T16:57:00.649 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:57:00.679 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:57:00.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922812 2024-05-30T16:57:01.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.118+0000 7fe721600640 1 Processor -- start 2024-05-30T16:57:01.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.118+0000 7fe721600640 1 -- start start 2024-05-30T16:57:01.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.120+0000 7fe721600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe71c0ccf30 0x7fe71c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:01.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.121+0000 7fe721600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71c0b1af0 con 0x7fe71c09af70 2024-05-30T16:57:01.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.121+0000 7fe721600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71c09b380 con 0x7fe71c0b1710 2024-05-30T16:57:01.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.121+0000 7fe721600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe71c09bb90 con 0x7fe71c0ccf30 2024-05-30T16:57:01.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.123+0000 7fe720c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe71c09af70 0x7fe71c15dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40240/0 (socket says 172.21.15.67:40240) 2024-05-30T16:57:01.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.123+0000 7fe720c00640 1 -- 172.21.15.67:0/1183297907 learned_addr learned my addr 172.21.15.67:0/1183297907 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:01.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.123+0000 7fe71be00640 1 --1- 172.21.15.67:0/1183297907 >> v1:172.21.15.169:6789/0 conn(0x7fe71c0b1710 0x7fe71c0d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:33132/0 (socket says 172.21.15.67:33132) 2024-05-30T16:57:01.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.123+0000 7fe71b400640 1 --2- 172.21.15.67:0/1183297907 >> v2:172.21.15.67:3300/0 conn(0x7fe71c0ccf30 0x7fe71c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.125+0000 7fe71b400640 1 -- 172.21.15.67:0/1183297907 >> v1:172.21.15.169:6789/0 conn(0x7fe71c0b1710 legacy=0x7fe71c0d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:01.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.125+0000 7fe71b400640 1 -- 172.21.15.67:0/1183297907 >> v1:172.21.15.67:6789/0 conn(0x7fe71c09af70 legacy=0x7fe71c15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:01.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.125+0000 7fe71b400640 1 -- 172.21.15.67:0/1183297907 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c0cd3f0 con 0x7fe71c0ccf30 2024-05-30T16:57:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.130+0000 7fe71b400640 1 --2- 172.21.15.67:0/1183297907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71c0ccf30 0x7fe71c09aa30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe7080097a0 tx=0x7fe70802a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a87fd2a61ef96c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:01.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.134+0000 7fe71aa00640 1 -- 172.21.15.67:0/1183297907 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708037070 con 0x7fe71c0ccf30 2024-05-30T16:57:01.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.136+0000 7fe71aa00640 1 -- 172.21.15.67:0/1183297907 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70802acd0 con 0x7fe71c0ccf30 2024-05-30T16:57:01.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.136+0000 7fe71aa00640 1 -- 172.21.15.67:0/1183297907 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708032660 con 0x7fe71c0ccf30 2024-05-30T16:57:01.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.139+0000 7fe721600640 1 -- 172.21.15.67:0/1183297907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71c0ccf30 msgr2=0x7fe71c09aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:01.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.139+0000 7fe721600640 1 --2- 172.21.15.67:0/1183297907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71c0ccf30 0x7fe71c09aa30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe7080097a0 tx=0x7fe70802a040 comp rx=0 tx=0).stop 2024-05-30T16:57:01.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.142+0000 7fe721600640 1 -- 172.21.15.67:0/1183297907 shutdown_connections 2024-05-30T16:57:01.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.142+0000 7fe721600640 1 --2- 172.21.15.67:0/1183297907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71c0ccf30 0x7fe71c09aa30 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:01.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.142+0000 7fe721600640 1 -- 172.21.15.67:0/1183297907 >> 172.21.15.67:0/1183297907 conn(0x7fe71c0b3d60 msgr2=0x7fe71c0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:01.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.143+0000 7fe721600640 1 -- 172.21.15.67:0/1183297907 shutdown_connections 2024-05-30T16:57:01.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.143+0000 7fe721600640 1 -- 172.21.15.67:0/1183297907 wait complete. 2024-05-30T16:57:01.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.152+0000 7fe721600640 1 Processor -- start 2024-05-30T16:57:01.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.152+0000 7fe721600640 1 -- start start 2024-05-30T16:57:01.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.154+0000 7fe721600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe71c09af70 0x7fe71c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:01.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.154+0000 7fe721600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe71c0b1710 0x7fe71c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:01.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.154+0000 7fe721600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71c0ccf30 0x7fe71c0cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:01.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.154+0000 7fe721600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c0cdc90 con 0x7fe71c0ccf30 2024-05-30T16:57:01.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.154+0000 7fe721600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c0cd9b0 con 0x7fe71c0b1710 2024-05-30T16:57:01.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.154+0000 7fe721600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe71c0cdf70 con 0x7fe71c09af70 2024-05-30T16:57:01.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.157+0000 7fe71b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe71c0b1710 0x7fe71c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:01.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.157+0000 7fe71be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe71c09af70 0x7fe71c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:01.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.158+0000 7fe71b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe71c0b1710 0x7fe71c15e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:59312/0 (socket says 172.21.15.67:59312) 2024-05-30T16:57:01.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.158+0000 7fe71b400640 1 -- 172.21.15.67:0/3780071202 learned_addr learned my addr 172.21.15.67:0/3780071202 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:01.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.158+0000 7fe720c00640 1 --2- 172.21.15.67:0/3780071202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71c0ccf30 0x7fe71c0cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:01.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.160+0000 7fe720c00640 1 -- 172.21.15.67:0/3780071202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe71c09af70 msgr2=0x7fe71c1570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:01.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.160+0000 7fe720c00640 1 --2- 172.21.15.67:0/3780071202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe71c09af70 0x7fe71c1570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:01.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.160+0000 7fe720c00640 1 -- 172.21.15.67:0/3780071202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe71c0b1710 msgr2=0x7fe71c15e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:01.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.160+0000 7fe720c00640 1 --2- 172.21.15.67:0/3780071202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe71c0b1710 0x7fe71c15e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:01.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.160+0000 7fe720c00640 1 -- 172.21.15.67:0/3780071202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c15ee10 con 0x7fe71c0ccf30 2024-05-30T16:57:01.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.161+0000 7fe71b400640 1 --2- 172.21.15.67:0/3780071202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe71c0b1710 0x7fe71c15e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:57:01.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.161+0000 7fe71be00640 1 --2- 172.21.15.67:0/3780071202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe71c09af70 0x7fe71c1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:57:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.163+0000 7fe720c00640 1 --2- 172.21.15.67:0/3780071202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe71c0ccf30 0x7fe71c0cb720 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe71400cb20 tx=0x7fe714002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:01.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.164+0000 7fe713e00640 1 -- 172.21.15.67:0/3780071202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714018070 con 0x7fe71c0ccf30 2024-05-30T16:57:01.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.164+0000 7fe721600640 1 -- 172.21.15.67:0/3780071202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c15f0f0 con 0x7fe71c0ccf30 2024-05-30T16:57:01.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.165+0000 7fe721600640 1 -- 172.21.15.67:0/3780071202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c15f6b0 con 0x7fe71c0ccf30 2024-05-30T16:57:01.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.166+0000 7fe713e00640 1 -- 172.21.15.67:0/3780071202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe71400a7c0 con 0x7fe71c0ccf30 2024-05-30T16:57:01.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.166+0000 7fe713e00640 1 -- 172.21.15.67:0/3780071202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71400abb0 con 0x7fe71c0ccf30 2024-05-30T16:57:01.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.168+0000 7fe721600640 1 -- 172.21.15.67:0/3780071202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c0689e0 con 0x7fe71c0ccf30 2024-05-30T16:57:01.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.171+0000 7fe713e00640 1 -- 172.21.15.67:0/3780071202 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7fe71401b070 con 0x7fe71c0ccf30 2024-05-30T16:57:01.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.173+0000 7fe713e00640 1 --2- 172.21.15.67:0/3780071202 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe6fc06d9d0 0x7fe6fc06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:01.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.173+0000 7fe713e00640 1 -- 172.21.15.67:0/3780071202 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fe71408a2b0 con 0x7fe71c0ccf30 2024-05-30T16:57:01.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.179+0000 7fe71be00640 1 --2- 172.21.15.67:0/3780071202 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe6fc06d9d0 0x7fe6fc06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.185+0000 7fe713e00640 1 -- 172.21.15.67:0/3780071202 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe71c0b1af0 con 0x7fe71c0ccf30 2024-05-30T16:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.189+0000 7fe71be00640 1 --2- 172.21.15.67:0/3780071202 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe6fc06d9d0 0x7fe6fc06fe90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007bd0 tx=0x7fe70c007b20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:01.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:01.693+0000 7fe721600640 1 -- 172.21.15.67:0/3780071202 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922812", "target": ["mon-mgr", ""]}) -- 0x7fe71c0c8d40 con 0x7fe6fc06d9d0 2024-05-30T16:57:02.141 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:02.140+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T16:57:02.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.140+0000 7fe713e00640 1 -- 172.21.15.67:0/3780071202 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fe71c0c8d40 con 0x7fe6fc06d9d0 2024-05-30T16:57:02.144 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T16:57:02.170 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T16:57:02.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922813 2024-05-30T16:57:02.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.639+0000 7febaee00640 1 Processor -- start 2024-05-30T16:57:02.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.642+0000 7febaee00640 1 -- start start 2024-05-30T16:57:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.644+0000 7febaee00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feba80d0640 0x7feba80d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.644+0000 7febaee00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feba80725e0 0x7feba80ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.644+0000 7febaee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba80ade90 con 0x7feba815b6f0 2024-05-30T16:57:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.644+0000 7febaee00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7feba8071c40 con 0x7feba80d0640 2024-05-30T16:57:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.644+0000 7febaee00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7feba80d0f60 con 0x7feba80725e0 2024-05-30T16:57:02.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.645+0000 7feba7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7feba815b6f0 0x7feba806e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40254/0 (socket says 172.21.15.67:40254) 2024-05-30T16:57:02.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.645+0000 7feba7400640 1 -- 172.21.15.67:0/3684793884 learned_addr learned my addr 172.21.15.67:0/3684793884 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:02.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.646+0000 7febace00640 1 --2- 172.21.15.67:0/3684793884 >> v2:172.21.15.169:3300/0 conn(0x7feba80725e0 0x7feba80ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:02.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.647+0000 7feba7e00640 1 --2- 172.21.15.67:0/3684793884 >> v2:172.21.15.67:3300/0 conn(0x7feba80d0640 0x7feba80d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:02.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.647+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4123399785 0 0) 0x7feba80ade90 con 0x7feba815b6f0 2024-05-30T16:57:02.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.647+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb8c003520 con 0x7feba815b6f0 2024-05-30T16:57:02.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.648+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 15456039 0 0) 0x7feb8c003520 con 0x7feba815b6f0 2024-05-30T16:57:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.648+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 >> v2:172.21.15.169:3300/0 conn(0x7feba80725e0 msgr2=0x7feba80ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.648+0000 7feba6a00640 1 --2- 172.21.15.67:0/3684793884 >> v2:172.21.15.169:3300/0 conn(0x7feba80725e0 0x7feba80ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.648+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 >> v2:172.21.15.67:3300/0 conn(0x7feba80d0640 msgr2=0x7feba80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:02.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.649+0000 7feba6a00640 1 --2- 172.21.15.67:0/3684793884 >> v2:172.21.15.67:3300/0 conn(0x7feba80d0640 0x7feba80d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:02.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.649+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba8071e70 con 0x7feba815b6f0 2024-05-30T16:57:02.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.649+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7feb94001670 con 0x7feba815b6f0 2024-05-30T16:57:02.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.649+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 >> v1:172.21.15.67:6789/0 conn(0x7feba815b6f0 legacy=0x7feba806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:02.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.649+0000 7feba6a00640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb8c0039b0 0x7feb8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:02.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.650+0000 7feba6a00640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb8c004800 0x7feb8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:02.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.650+0000 7feba6a00640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb8c0051c0 0x7feb8c00ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:02.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.650+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feba80d0f60 con 0x7feb8c0051c0 2024-05-30T16:57:02.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.650+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c010510 con 0x7feb8c0039b0 2024-05-30T16:57:02.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.650+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feb8c010690 con 0x7feb8c004800 2024-05-30T16:57:02.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.650+0000 7feba7e00640 1 --2- 172.21.15.67:0/3684793884 >> v2:172.21.15.67:3300/0 conn(0x7feba80d0640 0x7feba80d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:57:02.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.651+0000 7febace00640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb8c0051c0 0x7feb8c00ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:02.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.651+0000 7feba7400640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb8c004800 0x7feb8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:02.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.653+0000 7febace00640 1 -- 172.21.15.67:0/3684793884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb8c004800 msgr2=0x7feb8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:02.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.653+0000 7febace00640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb8c004800 0x7feb8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:02.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.653+0000 7febace00640 1 -- 172.21.15.67:0/3684793884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb8c0039b0 msgr2=0x7feb8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:02.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.653+0000 7febace00640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb8c0039b0 0x7feb8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:02.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.653+0000 7febace00640 1 -- 172.21.15.67:0/3684793884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba8071e70 con 0x7feb8c0051c0 2024-05-30T16:57:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.654+0000 7febace00640 1 -- 172.21.15.67:0/3684793884 reap_dead start 2024-05-30T16:57:02.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.655+0000 7febace00640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb8c0051c0 0x7feb8c00ffd0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7feba0009fc0 tx=0x7feb8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89ff6829965886 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:02.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.656+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0035020 con 0x7feb8c0051c0 2024-05-30T16:57:02.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.657+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba0003bd0 con 0x7feb8c0051c0 2024-05-30T16:57:02.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.657+0000 7feba6a00640 1 -- 172.21.15.67:0/3684793884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0009c10 con 0x7feb8c0051c0 2024-05-30T16:57:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.657+0000 7febaee00640 1 -- 172.21.15.67:0/3684793884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb8c0051c0 msgr2=0x7feb8c00ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.657+0000 7febaee00640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb8c0051c0 0x7feb8c00ffd0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7feba0009fc0 tx=0x7feb8c005830 comp rx=0 tx=0).stop 2024-05-30T16:57:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.658+0000 7febaee00640 1 -- 172.21.15.67:0/3684793884 shutdown_connections 2024-05-30T16:57:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.658+0000 7febaee00640 1 --2- 172.21.15.67:0/3684793884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb8c0051c0 0x7feb8c00ffd0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.658+0000 7febaee00640 1 -- 172.21.15.67:0/3684793884 >> 172.21.15.67:0/3684793884 conn(0x7feba806bd20 msgr2=0x7feba806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.658+0000 7febaee00640 1 -- 172.21.15.67:0/3684793884 shutdown_connections 2024-05-30T16:57:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.658+0000 7febaee00640 1 -- 172.21.15.67:0/3684793884 wait complete. 2024-05-30T16:57:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.660+0000 7febaee00640 1 Processor -- start 2024-05-30T16:57:02.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.660+0000 7febaee00640 1 -- start start 2024-05-30T16:57:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.662+0000 7febaee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb8c0051c0 0x7feba815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.662+0000 7febaee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba81593c0 0x7feba80ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.662+0000 7febaee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba81fae20 0x7feba81fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.662+0000 7febaee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feba80d0a20 con 0x7feba81fae20 2024-05-30T16:57:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.662+0000 7febaee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feba80ae130 con 0x7feb8c0051c0 2024-05-30T16:57:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.662+0000 7febaee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feba80d0ba0 con 0x7feba81593c0 2024-05-30T16:57:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.663+0000 7feba7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba81593c0 0x7feba80ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.663+0000 7feba7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba81593c0 0x7feba80ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:57426/0 (socket says 172.21.15.67:57426) 2024-05-30T16:57:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.663+0000 7feba7400640 1 -- 172.21.15.67:0/2898140535 learned_addr learned my addr 172.21.15.67:0/2898140535 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.664+0000 7febace00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba81fae20 0x7feba81fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.665+0000 7febace00640 1 -- 172.21.15.67:0/2898140535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba81593c0 msgr2=0x7feba80ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.665+0000 7febace00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba81593c0 0x7feba80ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.665+0000 7febace00640 1 -- 172.21.15.67:0/2898140535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb8c0051c0 msgr2=0x7feba815a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.665+0000 7febace00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb8c0051c0 0x7feba815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.665+0000 7febace00640 1 -- 172.21.15.67:0/2898140535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba8159940 con 0x7feba81fae20 2024-05-30T16:57:02.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.666+0000 7febace00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba81fae20 0x7feba81fb2a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7feba80ad920 tx=0x7feba0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:02.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.667+0000 7feb9fe00640 1 -- 172.21.15.67:0/2898140535 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0035030 con 0x7feba81fae20 2024-05-30T16:57:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.667+0000 7feb9fe00640 1 -- 172.21.15.67:0/2898140535 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba0043070 con 0x7feba81fae20 2024-05-30T16:57:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.667+0000 7feb9fe00640 1 -- 172.21.15.67:0/2898140535 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba002d070 con 0x7feba81fae20 2024-05-30T16:57:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.668+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba8159c20 con 0x7feba81fae20 2024-05-30T16:57:02.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.668+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba815a1e0 con 0x7feba81fae20 2024-05-30T16:57:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.670+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba80689e0 con 0x7feba81fae20 2024-05-30T16:57:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.673+0000 7feb9fe00640 1 -- 172.21.15.67:0/2898140535 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7feba0005d10 con 0x7feba81fae20 2024-05-30T16:57:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.674+0000 7feb9fe00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8806da20 0x7feb8806fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.674+0000 7feb9fe00640 1 -- 172.21.15.67:0/2898140535 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7feba00abfc0 con 0x7feba81fae20 2024-05-30T16:57:02.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.683+0000 7feba7e00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8806da20 0x7feb8806fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:02.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.687+0000 7feb9fe00640 1 -- 172.21.15.67:0/2898140535 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feba004d020 con 0x7feba81fae20 2024-05-30T16:57:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:02.752+0000 7feba7e00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8806da20 0x7feb8806fee0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feb98009b40 tx=0x7feb98004730 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:03.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:03.353+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922813", "target": ["mon-mgr", ""]}) -- 0x7feba8156610 con 0x7feb8806da20 2024-05-30T16:57:04.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.540+0000 7feb9fe00640 1 -- 172.21.15.67:0/2898140535 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feba8156610 con 0x7feb8806da20 2024-05-30T16:57:04.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.545+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8806da20 msgr2=0x7feb8806fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:04.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8806da20 0x7feb8806fee0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feb98009b40 tx=0x7feb98004730 comp rx=0 tx=0).stop 2024-05-30T16:57:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba81fae20 msgr2=0x7feba81fb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba81fae20 0x7feba81fb2a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7feba80ad920 tx=0x7feba0009750 comp rx=0 tx=0).stop 2024-05-30T16:57:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 shutdown_connections 2024-05-30T16:57:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feb8806da20 0x7feb8806fee0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feba81fae20 0x7feba81fb2a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feba81593c0 0x7feba80ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 --2- 172.21.15.67:0/2898140535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb8c0051c0 0x7feba815a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 >> 172.21.15.67:0/2898140535 conn(0x7feba806bd20 msgr2=0x7feba80b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 shutdown_connections 2024-05-30T16:57:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:04.546+0000 7febaee00640 1 -- 172.21.15.67:0/2898140535 wait complete. 2024-05-30T16:57:04.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922814 2024-05-30T16:57:05.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.035+0000 7f90a4a00640 1 Processor -- start 2024-05-30T16:57:05.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.036+0000 7f90a4a00640 1 -- start start 2024-05-30T16:57:05.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.040+0000 7f90a4a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f90a0162ac0 0x7f90a00d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:05.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.040+0000 7f90a4a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a01640e0 con 0x7f90a0163ac0 2024-05-30T16:57:05.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.040+0000 7f90a4a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a00b3360 con 0x7f90a00b4120 2024-05-30T16:57:05.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.040+0000 7f90a4a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f90a00b3c90 con 0x7f90a0162ac0 2024-05-30T16:57:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.041+0000 7f909e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f90a0163ac0 0x7f90a006dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40270/0 (socket says 172.21.15.67:40270) 2024-05-30T16:57:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.041+0000 7f909e800640 1 -- 172.21.15.67:0/4256288511 learned_addr learned my addr 172.21.15.67:0/4256288511 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:05.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.042+0000 7f909de00640 1 --2- 172.21.15.67:0/4256288511 >> v2:172.21.15.169:3300/0 conn(0x7f90a0162ac0 0x7f90a00d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:05.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.042+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2945866192 0 0) 0x7f90a00b3360 con 0x7f90a00b4120 2024-05-30T16:57:05.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.042+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9084003520 con 0x7f90a00b4120 2024-05-30T16:57:05.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.042+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2452418902 0 0) 0x7f9084003520 con 0x7f90a00b4120 2024-05-30T16:57:05.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.042+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 >> v2:172.21.15.169:3300/0 conn(0x7f90a0162ac0 msgr2=0x7f90a00d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:05.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.042+0000 7f909ca00640 1 --2- 172.21.15.67:0/4256288511 >> v2:172.21.15.169:3300/0 conn(0x7f90a0162ac0 0x7f90a00d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:05.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.042+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 >> v1:172.21.15.67:6789/0 conn(0x7f90a0163ac0 legacy=0x7f90a006dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:05.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.042+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a00b3640 con 0x7f90a00b4120 2024-05-30T16:57:05.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.042+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f908c001670 con 0x7f90a00b4120 2024-05-30T16:57:05.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.043+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 >> v1:172.21.15.169:6789/0 conn(0x7f90a00b4120 legacy=0x7f90a01633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:05.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.043+0000 7f909ca00640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90840039b0 0x7f9084003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:05.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.043+0000 7f909ca00640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f9084004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.043+0000 7f909ca00640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90840051c0 0x7f908400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.043+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90a00b3c90 con 0x7f9084004800 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.043+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f908400c090 con 0x7f90840039b0 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.043+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f908400c290 con 0x7f90840051c0 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.044+0000 7f909d400640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f9084004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.044+0000 7f909e800640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90840051c0 0x7f908400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.044+0000 7f909de00640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90840039b0 0x7f9084003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.044+0000 7f909d400640 1 -- 172.21.15.67:0/4256288511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90840051c0 msgr2=0x7f908400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.044+0000 7f909d400640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90840051c0 0x7f908400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.044+0000 7f909d400640 1 -- 172.21.15.67:0/4256288511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90840039b0 msgr2=0x7f9084003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.045+0000 7f909d400640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90840039b0 0x7f9084003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.045+0000 7f909d400640 1 -- 172.21.15.67:0/4256288511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a00b3640 con 0x7f9084004800 2024-05-30T16:57:05.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.045+0000 7f909e800640 1 -- 172.21.15.67:0/4256288511 reap_dead start 2024-05-30T16:57:05.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.047+0000 7f909de00640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90840039b0 0x7f9084003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:05.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.047+0000 7f909d400640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f9084004c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c1e0 tx=0x7f9084005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1998dff4a0f3ba9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:05.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.047+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c007c40 con 0x7f9084004800 2024-05-30T16:57:05.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.048+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f908c031600 con 0x7f9084004800 2024-05-30T16:57:05.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.048+0000 7f909ca00640 1 -- 172.21.15.67:0/4256288511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c031a30 con 0x7f9084004800 2024-05-30T16:57:05.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.049+0000 7f90a4a00640 1 -- 172.21.15.67:0/4256288511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 msgr2=0x7f9084004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:05.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.049+0000 7f90a4a00640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f9084004c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c1e0 tx=0x7f9084005830 comp rx=0 tx=0).stop 2024-05-30T16:57:05.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.049+0000 7f90a4a00640 1 -- 172.21.15.67:0/4256288511 shutdown_connections 2024-05-30T16:57:05.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.049+0000 7f90a4a00640 1 --2- 172.21.15.67:0/4256288511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f9084004c80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:05.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.049+0000 7f90a4a00640 1 -- 172.21.15.67:0/4256288511 >> 172.21.15.67:0/4256288511 conn(0x7f90a006bd20 msgr2=0x7f90a006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:05.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.050+0000 7f90a4a00640 1 -- 172.21.15.67:0/4256288511 shutdown_connections 2024-05-30T16:57:05.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.050+0000 7f90a4a00640 1 -- 172.21.15.67:0/4256288511 wait complete. 2024-05-30T16:57:05.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.053+0000 7f90a4a00640 1 Processor -- start 2024-05-30T16:57:05.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.054+0000 7f90a4a00640 1 -- start start 2024-05-30T16:57:05.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.055+0000 7f90a4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f90a009c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:05.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.056+0000 7f90a4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a009a950 0x7f90a015f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:05.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.056+0000 7f90a4a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a00b0260 0x7f90a00b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:05.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.056+0000 7f90a4a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0163020 con 0x7f9084004800 2024-05-30T16:57:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.056+0000 7f90a4a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0162ea0 con 0x7f90a00b0260 2024-05-30T16:57:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.056+0000 7f90a4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f90a01633b0 con 0x7f90a009a950 2024-05-30T16:57:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.057+0000 7f909e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a00b0260 0x7f90a00b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.057+0000 7f909e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a00b0260 0x7f90a00b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:59366/0 (socket says 172.21.15.67:59366) 2024-05-30T16:57:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.057+0000 7f909e800640 1 -- 172.21.15.67:0/901805640 learned_addr learned my addr 172.21.15.67:0/901805640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.057+0000 7f909d400640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a009a950 0x7f90a015f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.058+0000 7f909e800640 1 -- 172.21.15.67:0/901805640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a009a950 msgr2=0x7f90a015f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.058+0000 7f909e800640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a009a950 0x7f90a015f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.058+0000 7f909e800640 1 -- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 msgr2=0x7f90a009c3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-30T16:57:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.058+0000 7f909de00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f90a009c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.059+0000 7f909e800640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f90a009c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.059+0000 7f909e800640 1 -- 172.21.15.67:0/901805640 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a009c920 con 0x7f90a00b0260 2024-05-30T16:57:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.059+0000 7f909de00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f90a009c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.060+0000 7f909e800640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a00b0260 0x7f90a00b06f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f90a0072a20 tx=0x7f9098007bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.062+0000 7f9096000640 1 -- 172.21.15.67:0/901805640 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909800fa10 con 0x7f90a00b0260 2024-05-30T16:57:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.062+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a014c4f0 con 0x7f90a00b0260 2024-05-30T16:57:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.062+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a014cab0 con 0x7f90a00b0260 2024-05-30T16:57:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.063+0000 7f9096000640 1 -- 172.21.15.67:0/901805640 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9098018070 con 0x7f90a00b0260 2024-05-30T16:57:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.063+0000 7f9096000640 1 -- 172.21.15.67:0/901805640 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909800fdd0 con 0x7f90a00b0260 2024-05-30T16:57:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.064+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9078005290 con 0x7f90a00b0260 2024-05-30T16:57:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.067+0000 7f9096000640 1 -- 172.21.15.67:0/901805640 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f909801d460 con 0x7f90a00b0260 2024-05-30T16:57:05.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.068+0000 7f9096000640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f908006daa0 0x7f908006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:05.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.068+0000 7f9096000640 1 -- 172.21.15.67:0/901805640 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f909808ae50 con 0x7f90a00b0260 2024-05-30T16:57:05.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.077+0000 7f9096000640 1 -- 172.21.15.67:0/901805640 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f909805acd0 con 0x7f90a00b0260 2024-05-30T16:57:05.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.091+0000 7f909de00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f908006daa0 0x7f908006ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:05.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.107+0000 7f909de00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f908006daa0 0x7f908006ff60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f90a00b0f30 tx=0x7f9090008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:05.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:05.694+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922814", "target": ["mon-mgr", ""]}) -- 0x7f9078000bc0 con 0x7f908006daa0 2024-05-30T16:57:07.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.087+0000 7f9096000640 1 -- 172.21.15.67:0/901805640 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9078000bc0 con 0x7f908006daa0 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f908006daa0 msgr2=0x7f908006ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f908006daa0 0x7f908006ff60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f90a00b0f30 tx=0x7f9090008040 comp rx=0 tx=0).stop 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a00b0260 msgr2=0x7f90a00b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a00b0260 0x7f90a00b06f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f90a0072a20 tx=0x7f9098007bd0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 shutdown_connections 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f908006daa0 0x7f908006ff60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a00b0260 0x7f90a00b06f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a009a950 0x7f90a015f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 --2- 172.21.15.67:0/901805640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9084004800 0x7f90a009c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 >> 172.21.15.67:0/901805640 conn(0x7f90a006bd20 msgr2=0x7f90a006eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 shutdown_connections 2024-05-30T16:57:07.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.091+0000 7f90a4a00640 1 -- 172.21.15.67:0/901805640 wait complete. 2024-05-30T16:57:07.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922815 2024-05-30T16:57:07.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.583+0000 7f7c1f800640 1 Processor -- start 2024-05-30T16:57:07.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.583+0000 7f7c1f800640 1 -- start start 2024-05-30T16:57:07.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.587+0000 7f7c1f800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c18162b20 0x7f7c18167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:07.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.587+0000 7f7c1f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180d1570 0x7f7c180d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:07.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.587+0000 7f7c1f800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c18162f00 con 0x7f7c180b7500 2024-05-30T16:57:07.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.588+0000 7f7c1f800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7c180d1f20 con 0x7f7c18162b20 2024-05-30T16:57:07.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.588+0000 7f7c1f800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c180cf4f0 con 0x7f7c180d1570 2024-05-30T16:57:07.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.588+0000 7f7c1ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c18162b20 0x7f7c18167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:07.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.588+0000 7f7c1ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c18162b20 0x7f7c18167080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:59382/0 (socket says 172.21.15.67:59382) 2024-05-30T16:57:07.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.588+0000 7f7c1ce00640 1 -- 172.21.15.67:0/2344867910 learned_addr learned my addr 172.21.15.67:0/2344867910 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:07.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.589+0000 7f7c1ce00640 1 -- 172.21.15.67:0/2344867910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180d1570 msgr2=0x7f7c180d19e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:57:07.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.589+0000 7f7c1ce00640 1 --2- 172.21.15.67:0/2344867910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180d1570 0x7f7c180d19e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.589+0000 7f7c1ce00640 1 -- 172.21.15.67:0/2344867910 >> v1:172.21.15.67:6789/0 conn(0x7f7c180b7500 legacy=0x7f7c180d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:07.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.590+0000 7f7c1ce00640 1 -- 172.21.15.67:0/2344867910 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c180d4190 con 0x7f7c18162b20 2024-05-30T16:57:07.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.589+0000 7f7c17400640 1 -- 172.21.15.67:0/2344867910 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 202042439 0 0) 0x7f7c18162f00 con 0x7f7c180b7500 2024-05-30T16:57:07.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.591+0000 7f7c1ce00640 1 --2- 172.21.15.67:0/2344867910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c18162b20 0x7f7c18167080 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009870 tx=0x7f7c0c02a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c13244c891bdd21e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:07.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.592+0000 7f7c17400640 1 -- 172.21.15.67:0/2344867910 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c037070 con 0x7f7c18162b20 2024-05-30T16:57:07.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.592+0000 7f7c17400640 1 -- 172.21.15.67:0/2344867910 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c032440 con 0x7f7c18162b20 2024-05-30T16:57:07.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.592+0000 7f7c17400640 1 -- 172.21.15.67:0/2344867910 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c032810 con 0x7f7c18162b20 2024-05-30T16:57:07.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.595+0000 7f7c1f800640 1 -- 172.21.15.67:0/2344867910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c18162b20 msgr2=0x7f7c18167080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:07.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.595+0000 7f7c1f800640 1 --2- 172.21.15.67:0/2344867910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c18162b20 0x7f7c18167080 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009870 tx=0x7f7c0c02a1f0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.596+0000 7f7c1f800640 1 -- 172.21.15.67:0/2344867910 shutdown_connections 2024-05-30T16:57:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.596+0000 7f7c1f800640 1 --2- 172.21.15.67:0/2344867910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180d1570 0x7f7c180d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.596+0000 7f7c1f800640 1 --2- 172.21.15.67:0/2344867910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c18162b20 0x7f7c18167080 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.596+0000 7f7c1f800640 1 -- 172.21.15.67:0/2344867910 >> 172.21.15.67:0/2344867910 conn(0x7f7c18067aa0 msgr2=0x7f7c18067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.597+0000 7f7c1f800640 1 -- 172.21.15.67:0/2344867910 shutdown_connections 2024-05-30T16:57:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.597+0000 7f7c1f800640 1 -- 172.21.15.67:0/2344867910 wait complete. 2024-05-30T16:57:07.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.601+0000 7f7c1f800640 1 Processor -- start 2024-05-30T16:57:07.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.601+0000 7f7c1f800640 1 -- start start 2024-05-30T16:57:07.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.603+0000 7f7c1f800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180b7500 0x7f7c18164d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:07.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.603+0000 7f7c1f800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c180d1570 0x7f7c181ec400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:07.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.604+0000 7f7c1f800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c18162b20 0x7f7c181568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:07.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.604+0000 7f7c1f800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180d4670 con 0x7f7c18162b20 2024-05-30T16:57:07.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.604+0000 7f7c1ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180b7500 0x7f7c18164d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:07.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.604+0000 7f7c1ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180b7500 0x7f7c18164d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:57466/0 (socket says 172.21.15.67:57466) 2024-05-30T16:57:07.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.604+0000 7f7c1ce00640 1 -- 172.21.15.67:0/753996372 learned_addr learned my addr 172.21.15.67:0/753996372 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:07.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.604+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180cf210 con 0x7f7c180d1570 2024-05-30T16:57:07.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.604+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c180d47f0 con 0x7f7c180b7500 2024-05-30T16:57:07.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.604+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c180d1570 0x7f7c181ec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:07.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.605+0000 7f7c1d800640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c18162b20 0x7f7c181568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:07.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.606+0000 7f7c1ce00640 1 -- 172.21.15.67:0/753996372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c180d1570 msgr2=0x7f7c181ec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:07.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.606+0000 7f7c1ce00640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c180d1570 0x7f7c181ec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.606+0000 7f7c1ce00640 1 -- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c18162b20 msgr2=0x7f7c181568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:07.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.606+0000 7f7c1ce00640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c18162b20 0x7f7c181568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:07.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.606+0000 7f7c1ce00640 1 -- 172.21.15.67:0/753996372 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c181ecb00 con 0x7f7c180b7500 2024-05-30T16:57:07.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.606+0000 7f7c1ce00640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180b7500 0x7f7c18164d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c000c00 tx=0x7f7c0c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:07.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.607+0000 7f7c14c00640 1 -- 172.21.15.67:0/753996372 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c041070 con 0x7f7c180b7500 2024-05-30T16:57:07.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.607+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c181ecde0 con 0x7f7c180b7500 2024-05-30T16:57:07.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.607+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c181ed3a0 con 0x7f7c180b7500 2024-05-30T16:57:07.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.608+0000 7f7c14c00640 1 -- 172.21.15.67:0/753996372 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c032440 con 0x7f7c180b7500 2024-05-30T16:57:07.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.609+0000 7f7c14c00640 1 -- 172.21.15.67:0/753996372 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c041070 con 0x7f7c180b7500 2024-05-30T16:57:07.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.610+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf4005290 con 0x7f7c180b7500 2024-05-30T16:57:07.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.614+0000 7f7c14c00640 1 -- 172.21.15.67:0/753996372 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f7c0c044070 con 0x7f7c180b7500 2024-05-30T16:57:07.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.618+0000 7f7c14c00640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bf806ddd0 0x7f7bf8070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:07.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.620+0000 7f7c14c00640 1 -- 172.21.15.67:0/753996372 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f7c0c03c670 con 0x7f7c180b7500 2024-05-30T16:57:07.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.627+0000 7f7c14c00640 1 -- 172.21.15.67:0/753996372 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c0c0ab2d0 con 0x7f7c180b7500 2024-05-30T16:57:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.630+0000 7f7c17e00640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bf806ddd0 0x7f7bf8070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:07.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:07.695+0000 7f7c17e00640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bf806ddd0 0x7f7bf8070290 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7c180ce690 tx=0x7f7c04009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:08.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:08.260+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922815", "target": ["mon-mgr", ""]}) -- 0x7f7bf4000bc0 con 0x7f7bf806ddd0 2024-05-30T16:57:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.766+0000 7f7c14c00640 1 -- 172.21.15.67:0/753996372 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7bf4000bc0 con 0x7f7bf806ddd0 2024-05-30T16:57:09.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.772+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bf806ddd0 msgr2=0x7f7bf8070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.773+0000 7f7c1f800640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bf806ddd0 0x7f7bf8070290 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7c180ce690 tx=0x7f7c04009040 comp rx=0 tx=0).stop 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.773+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180b7500 msgr2=0x7f7c18164d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.773+0000 7f7c1f800640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180b7500 0x7f7c18164d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c000c00 tx=0x7f7c0c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.773+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 shutdown_connections 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.773+0000 7f7c1f800640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7bf806ddd0 0x7f7bf8070290 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.773+0000 7f7c1f800640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c18162b20 0x7f7c181568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.773+0000 7f7c1f800640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c180d1570 0x7f7c181ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.773+0000 7f7c1f800640 1 --2- 172.21.15.67:0/753996372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c180b7500 0x7f7c18164d10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.773+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 >> 172.21.15.67:0/753996372 conn(0x7f7c18067aa0 msgr2=0x7f7c180b9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.774+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 shutdown_connections 2024-05-30T16:57:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:09.774+0000 7f7c1f800640 1 -- 172.21.15.67:0/753996372 wait complete. 2024-05-30T16:57:09.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922816 2024-05-30T16:57:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.268+0000 7f21f9a00640 1 Processor -- start 2024-05-30T16:57:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.268+0000 7f21f9a00640 1 -- start start 2024-05-30T16:57:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.271+0000 7f21f9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21f406fd30 0x7f21f4072780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:10.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.271+0000 7f21f9a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21f4157f30 0x7f21f40d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:10.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.273+0000 7f21f2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21f406fd30 0x7f21f4072780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:10.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.273+0000 7f21f2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21f406fd30 0x7f21f4072780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49636/0 (socket says 172.21.15.67:49636) 2024-05-30T16:57:10.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.274+0000 7f21f9a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f4072cc0 con 0x7f21f41583e0 2024-05-30T16:57:10.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.274+0000 7f21f9a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f21f4161430 con 0x7f21f406fd30 2024-05-30T16:57:10.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.274+0000 7f21f9a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f21f40b3940 con 0x7f21f4157f30 2024-05-30T16:57:10.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.274+0000 7f21f2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21f4157f30 0x7f21f40d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:10.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.274+0000 7f21f2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21f4157f30 0x7f21f40d0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:59404/0 (socket says 172.21.15.67:59404) 2024-05-30T16:57:10.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.274+0000 7f21f2000640 1 -- 172.21.15.67:0/1553994530 learned_addr learned my addr 172.21.15.67:0/1553994530 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:10.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.275+0000 7f21f2000640 1 -- 172.21.15.67:0/1553994530 >> v2:172.21.15.67:3300/0 conn(0x7f21f406fd30 msgr2=0x7f21f4072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:10.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.279+0000 7f21f2000640 1 --2- 172.21.15.67:0/1553994530 >> v2:172.21.15.67:3300/0 conn(0x7f21f406fd30 0x7f21f4072780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.279+0000 7f21f2000640 1 -- 172.21.15.67:0/1553994530 >> v1:172.21.15.67:6789/0 conn(0x7f21f41583e0 legacy=0x7f21f415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.279+0000 7f21f2000640 1 -- 172.21.15.67:0/1553994530 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f40b3c20 con 0x7f21f4157f30 2024-05-30T16:57:10.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.281+0000 7f21f2a00640 1 --2- 172.21.15.67:0/1553994530 >> v2:172.21.15.67:3300/0 conn(0x7f21f406fd30 0x7f21f4072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:57:10.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.284+0000 7f21f2000640 1 --2- 172.21.15.67:0/1553994530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f4157f30 0x7f21f40d0770 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f21f406f980 tx=0x7f21e402a760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb74dbc79c41ff5a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:10.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.286+0000 7f21f1600640 1 -- 172.21.15.67:0/1553994530 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4036070 con 0x7f21f4157f30 2024-05-30T16:57:10.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.287+0000 7f21f1600640 1 -- 172.21.15.67:0/1553994530 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4031600 con 0x7f21f4157f30 2024-05-30T16:57:10.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.287+0000 7f21f1600640 1 -- 172.21.15.67:0/1553994530 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4031a10 con 0x7f21f4157f30 2024-05-30T16:57:10.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.288+0000 7f21f9a00640 1 -- 172.21.15.67:0/1553994530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f4157f30 msgr2=0x7f21f40d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:10.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.288+0000 7f21f9a00640 1 --2- 172.21.15.67:0/1553994530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f4157f30 0x7f21f40d0770 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f21f406f980 tx=0x7f21e402a760 comp rx=0 tx=0).stop 2024-05-30T16:57:10.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.290+0000 7f21f9a00640 1 -- 172.21.15.67:0/1553994530 shutdown_connections 2024-05-30T16:57:10.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.290+0000 7f21f9a00640 1 --2- 172.21.15.67:0/1553994530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f4157f30 0x7f21f40d0770 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:10.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.290+0000 7f21f9a00640 1 --2- 172.21.15.67:0/1553994530 >> v2:172.21.15.67:3300/0 conn(0x7f21f406fd30 0x7f21f4072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:10.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.290+0000 7f21f9a00640 1 -- 172.21.15.67:0/1553994530 >> 172.21.15.67:0/1553994530 conn(0x7f21f406bd20 msgr2=0x7f21f406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:10.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.292+0000 7f21f9a00640 1 -- 172.21.15.67:0/1553994530 shutdown_connections 2024-05-30T16:57:10.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.292+0000 7f21f9a00640 1 -- 172.21.15.67:0/1553994530 wait complete. 2024-05-30T16:57:10.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.303+0000 7f21f9a00640 1 Processor -- start 2024-05-30T16:57:10.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.305+0000 7f21f9a00640 1 -- start start 2024-05-30T16:57:10.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.315+0000 7f21f9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f406fd30 0x7f21f41525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:10.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.315+0000 7f21f9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21f4157f30 0x7f21f408bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:10.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.315+0000 7f21f9a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f41583e0 0x7f21f408d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:10.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.316+0000 7f21f2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f406fd30 0x7f21f41525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:10.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.316+0000 7f21f2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f406fd30 0x7f21f41525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:57482/0 (socket says 172.21.15.67:57482) 2024-05-30T16:57:10.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.317+0000 7f21f2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21f4157f30 0x7f21f408bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:10.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.317+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21f40b32c0 con 0x7f21f4157f30 2024-05-30T16:57:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.317+0000 7f21f2a00640 1 -- 172.21.15.67:0/2712600985 learned_addr learned my addr 172.21.15.67:0/2712600985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.318+0000 7f21f3400640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f41583e0 0x7f21f408d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.318+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21f40b41e0 con 0x7f21f41583e0 2024-05-30T16:57:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.318+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21f40b3440 con 0x7f21f406fd30 2024-05-30T16:57:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.321+0000 7f21f2000640 1 -- 172.21.15.67:0/2712600985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f406fd30 msgr2=0x7f21f41525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.322+0000 7f21f2000640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f406fd30 0x7f21f41525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.322+0000 7f21f2000640 1 -- 172.21.15.67:0/2712600985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f41583e0 msgr2=0x7f21f408d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.322+0000 7f21f2000640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f41583e0 0x7f21f408d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.322+0000 7f21f2000640 1 -- 172.21.15.67:0/2712600985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f40b28c0 con 0x7f21f4157f30 2024-05-30T16:57:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.323+0000 7f21f2000640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21f4157f30 0x7f21f408bfc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f21f40b3ac0 tx=0x7f21e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:10.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.323+0000 7f21eaa00640 1 -- 172.21.15.67:0/2712600985 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4041020 con 0x7f21f4157f30 2024-05-30T16:57:10.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.328+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f408c810 con 0x7f21f4157f30 2024-05-30T16:57:10.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.328+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f408cdd0 con 0x7f21f4157f30 2024-05-30T16:57:10.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.333+0000 7f21eaa00640 1 -- 172.21.15.67:0/2712600985 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4031600 con 0x7f21f4157f30 2024-05-30T16:57:10.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.333+0000 7f21eaa00640 1 -- 172.21.15.67:0/2712600985 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4041020 con 0x7f21f4157f30 2024-05-30T16:57:10.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.335+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc005290 con 0x7f21f4157f30 2024-05-30T16:57:10.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.340+0000 7f21eaa00640 1 -- 172.21.15.67:0/2712600985 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f21e4044020 con 0x7f21f4157f30 2024-05-30T16:57:10.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.343+0000 7f21eaa00640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21d406dbd0 0x7f21d4070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:10.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.344+0000 7f21eaa00640 1 -- 172.21.15.67:0/2712600985 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f21e40abb00 con 0x7f21f4157f30 2024-05-30T16:57:10.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.346+0000 7f21eaa00640 1 -- 172.21.15.67:0/2712600985 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21e407b980 con 0x7f21f4157f30 2024-05-30T16:57:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.386+0000 7f21f2a00640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21d406dbd0 0x7f21d4070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:10.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.436+0000 7f21f2a00640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21d406dbd0 0x7f21d4070090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f21f406f9f0 tx=0x7f21ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:10.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:10.888+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922816", "target": ["mon-mgr", ""]}) -- 0x7f21cc000bc0 con 0x7f21d406dbd0 2024-05-30T16:57:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.081+0000 7f21eaa00640 1 -- 172.21.15.67:0/2712600985 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f21cc000bc0 con 0x7f21d406dbd0 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.093+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21d406dbd0 msgr2=0x7f21d4070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.093+0000 7f21f9a00640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21d406dbd0 0x7f21d4070090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f21f406f9f0 tx=0x7f21ec008040 comp rx=0 tx=0).stop 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.093+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21f4157f30 msgr2=0x7f21f408bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.093+0000 7f21f9a00640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21f4157f30 0x7f21f408bfc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f21f40b3ac0 tx=0x7f21e4004a40 comp rx=0 tx=0).stop 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.094+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 shutdown_connections 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.094+0000 7f21f9a00640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f21d406dbd0 0x7f21d4070090 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.094+0000 7f21f9a00640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f41583e0 0x7f21f408d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.094+0000 7f21f9a00640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21f4157f30 0x7f21f408bfc0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.094+0000 7f21f9a00640 1 --2- 172.21.15.67:0/2712600985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f406fd30 0x7f21f41525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.094+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 >> 172.21.15.67:0/2712600985 conn(0x7f21f406bd20 msgr2=0x7f21f40ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.094+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 shutdown_connections 2024-05-30T16:57:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.094+0000 7f21f9a00640 1 -- 172.21.15.67:0/2712600985 wait complete. 2024-05-30T16:57:12.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922817 2024-05-30T16:57:12.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.587+0000 7f3ff5a00640 1 Processor -- start 2024-05-30T16:57:12.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.587+0000 7f3ff5a00640 1 -- start start 2024-05-30T16:57:12.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.590+0000 7f3ff5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff01581a0 0x7f3ff0072780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:12.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.590+0000 7f3ff5a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3ff00d0770 0x7f3ff00d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:12.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.590+0000 7f3ff5a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ff006fd30 0x7f3ff015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:12.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.590+0000 7f3ff5a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff0072cc0 con 0x7f3ff00d0770 2024-05-30T16:57:12.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.590+0000 7f3ff5a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff00b3660 con 0x7f3ff006fd30 2024-05-30T16:57:12.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.590+0000 7f3ff5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff00b3940 con 0x7f3ff01581a0 2024-05-30T16:57:12.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.591+0000 7f3feea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff01581a0 0x7f3ff0072780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:12.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.591+0000 7f3feea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff01581a0 0x7f3ff0072780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:53190/0 (socket says 172.21.15.67:53190) 2024-05-30T16:57:12.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.591+0000 7f3feea00640 1 -- 172.21.15.67:0/3074811274 learned_addr learned my addr 172.21.15.67:0/3074811274 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:12.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.592+0000 7f3fef400640 1 --2- 172.21.15.67:0/3074811274 >> v2:172.21.15.169:3300/0 conn(0x7f3ff006fd30 0x7f3ff015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:12.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.592+0000 7f3feea00640 1 -- 172.21.15.67:0/3074811274 >> v2:172.21.15.169:3300/0 conn(0x7f3ff006fd30 msgr2=0x7f3ff015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:12.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.592+0000 7f3feea00640 1 --2- 172.21.15.67:0/3074811274 >> v2:172.21.15.169:3300/0 conn(0x7f3ff006fd30 0x7f3ff015eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.592+0000 7f3feea00640 1 -- 172.21.15.67:0/3074811274 >> v2:172.21.15.67:3300/0 conn(0x7f3ff00d0770 msgr2=0x7f3ff00d0bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:57:12.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.594+0000 7f3fee000640 1 --2- 172.21.15.67:0/3074811274 >> v2:172.21.15.67:3300/0 conn(0x7f3ff00d0770 0x7f3ff00d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.594+0000 7f3feea00640 1 --2- 172.21.15.67:0/3074811274 >> v2:172.21.15.67:3300/0 conn(0x7f3ff00d0770 0x7f3ff00d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.594+0000 7f3feea00640 1 -- 172.21.15.67:0/3074811274 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff00b3c20 con 0x7f3ff01581a0 2024-05-30T16:57:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.594+0000 7f3fee000640 1 --2- 172.21.15.67:0/3074811274 >> v2:172.21.15.67:3300/0 conn(0x7f3ff00d0770 0x7f3ff00d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:12.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.604+0000 7f3feea00640 1 --2- 172.21.15.67:0/3074811274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff01581a0 0x7f3ff0072780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00afda0 tx=0x7f3fe002a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32a5270ca6b9665c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:12.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.608+0000 7f3fed600640 1 -- 172.21.15.67:0/3074811274 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0037070 con 0x7f3ff01581a0 2024-05-30T16:57:12.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.608+0000 7f3fed600640 1 -- 172.21.15.67:0/3074811274 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe0032440 con 0x7f3ff01581a0 2024-05-30T16:57:12.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.608+0000 7f3fed600640 1 -- 172.21.15.67:0/3074811274 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0032850 con 0x7f3ff01581a0 2024-05-30T16:57:12.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.610+0000 7f3ff5a00640 1 -- 172.21.15.67:0/3074811274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff01581a0 msgr2=0x7f3ff0072780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:12.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.610+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/3074811274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff01581a0 0x7f3ff0072780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00afda0 tx=0x7f3fe002a160 comp rx=0 tx=0).stop 2024-05-30T16:57:12.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.619+0000 7f3ff5a00640 1 -- 172.21.15.67:0/3074811274 shutdown_connections 2024-05-30T16:57:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.619+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/3074811274 >> v2:172.21.15.67:3300/0 conn(0x7f3ff00d0770 0x7f3ff00d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.619+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/3074811274 >> v2:172.21.15.169:3300/0 conn(0x7f3ff006fd30 0x7f3ff015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.619+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/3074811274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff01581a0 0x7f3ff0072780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.619+0000 7f3ff5a00640 1 -- 172.21.15.67:0/3074811274 >> 172.21.15.67:0/3074811274 conn(0x7f3ff006bd20 msgr2=0x7f3ff006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.621+0000 7f3ff5a00640 1 -- 172.21.15.67:0/3074811274 shutdown_connections 2024-05-30T16:57:12.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.621+0000 7f3ff5a00640 1 -- 172.21.15.67:0/3074811274 wait complete. 2024-05-30T16:57:12.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.623+0000 7f3ff5a00640 1 Processor -- start 2024-05-30T16:57:12.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.624+0000 7f3ff5a00640 1 -- start start 2024-05-30T16:57:12.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.626+0000 7f3ff5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff006fd30 0x7f3ff00afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:12.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.626+0000 7f3ff5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00d0770 0x7f3ff0099550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:12.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.626+0000 7f3ff5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff01581a0 0x7f3ff01f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:12.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.626+0000 7f3ff5a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff015f0c0 con 0x7f3ff01581a0 2024-05-30T16:57:12.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.626+0000 7f3ff5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff00b41e0 con 0x7f3ff00d0770 2024-05-30T16:57:12.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.626+0000 7f3ff5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff00b32c0 con 0x7f3ff006fd30 2024-05-30T16:57:12.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.627+0000 7f3fef400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff01581a0 0x7f3ff01f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:12.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.627+0000 7f3fef400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff01581a0 0x7f3ff01f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55928/0 (socket says 172.21.15.67:55928) 2024-05-30T16:57:12.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.627+0000 7f3fef400640 1 -- 172.21.15.67:0/2293581364 learned_addr learned my addr 172.21.15.67:0/2293581364 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:12.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.631+0000 7f3feea00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff006fd30 0x7f3ff00afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:12.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.632+0000 7f3fee000640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00d0770 0x7f3ff0099550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:12.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.632+0000 7f3fef400640 1 -- 172.21.15.67:0/2293581364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff006fd30 msgr2=0x7f3ff00afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:12.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.632+0000 7f3fef400640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff006fd30 0x7f3ff00afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.632+0000 7f3fef400640 1 -- 172.21.15.67:0/2293581364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00d0770 msgr2=0x7f3ff0099550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:12.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.632+0000 7f3fef400640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00d0770 0x7f3ff0099550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:12.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.632+0000 7f3fef400640 1 -- 172.21.15.67:0/2293581364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff0098830 con 0x7f3ff01581a0 2024-05-30T16:57:12.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.632+0000 7f3fee000640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00d0770 0x7f3ff0099550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:12.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.637+0000 7f3fef400640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff01581a0 0x7f3ff01f78c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800efc0 tx=0x7f3fe800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:12.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.644+0000 7f3fe6a00640 1 -- 172.21.15.67:0/2293581364 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8018020 con 0x7f3ff01581a0 2024-05-30T16:57:12.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.644+0000 7f3fe6a00640 1 -- 172.21.15.67:0/2293581364 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe8014480 con 0x7f3ff01581a0 2024-05-30T16:57:12.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.644+0000 7f3fe6a00640 1 -- 172.21.15.67:0/2293581364 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8014870 con 0x7f3ff01581a0 2024-05-30T16:57:12.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.644+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff0098270 con 0x7f3ff01581a0 2024-05-30T16:57:12.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.645+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff00b18d0 con 0x7f3ff01581a0 2024-05-30T16:57:12.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.652+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc8005290 con 0x7f3ff01581a0 2024-05-30T16:57:12.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.674+0000 7f3fe6a00640 1 -- 172.21.15.67:0/2293581364 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f3fe8005d10 con 0x7f3ff01581a0 2024-05-30T16:57:12.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.681+0000 7f3fe6a00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3fd006daa0 0x7f3fd006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:12.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.682+0000 7f3fe6a00640 1 -- 172.21.15.67:0/2293581364 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f3fe808ba90 con 0x7f3ff01581a0 2024-05-30T16:57:12.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.684+0000 7f3fe6a00640 1 -- 172.21.15.67:0/2293581364 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fe8022030 con 0x7f3ff01581a0 2024-05-30T16:57:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.693+0000 7f3feea00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3fd006daa0 0x7f3fd006ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:12.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:12.706+0000 7f3feea00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3fd006daa0 0x7f3fd006ff60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0034000 tx=0x7f3fe002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:13.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:13.272+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922817", "target": ["mon-mgr", ""]}) -- 0x7f3fc8000bc0 con 0x7f3fd006daa0 2024-05-30T16:57:13.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:57:13.320 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:57:15.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.094+0000 7f3fe6a00640 1 -- 172.21.15.67:0/2293581364 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3fc8000bc0 con 0x7f3fd006daa0 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3fd006daa0 msgr2=0x7f3fd006ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3fd006daa0 0x7f3fd006ff60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0034000 tx=0x7f3fe002e000 comp rx=0 tx=0).stop 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff01581a0 msgr2=0x7f3ff01f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff01581a0 0x7f3ff01f78c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800efc0 tx=0x7f3fe800c490 comp rx=0 tx=0).stop 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 shutdown_connections 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3fd006daa0 0x7f3fd006ff60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ff01581a0 0x7f3ff01f78c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00d0770 0x7f3ff0099550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 --2- 172.21.15.67:0/2293581364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ff006fd30 0x7f3ff00afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 >> 172.21.15.67:0/2293581364 conn(0x7f3ff006bd20 msgr2=0x7f3ff006dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 shutdown_connections 2024-05-30T16:57:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.097+0000 7f3ff5a00640 1 -- 172.21.15.67:0/2293581364 wait complete. 2024-05-30T16:57:15.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922818 2024-05-30T16:57:15.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.536+0000 7f3b14c00640 1 Processor -- start 2024-05-30T16:57:15.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.536+0000 7f3b14c00640 1 -- start start 2024-05-30T16:57:15.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.540+0000 7f3b14c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161e70 0x7f3b100b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:15.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.540+0000 7f3b14c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3b100b3800 0x7f3b100ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:15.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.540+0000 7f3b14c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b100b3d40 con 0x7f3b10161a90 2024-05-30T16:57:15.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.540+0000 7f3b14c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3b100b4020 con 0x7f3b100b3800 2024-05-30T16:57:15.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.540+0000 7f3b14c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3b100b4300 con 0x7f3b10161e70 2024-05-30T16:57:15.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.541+0000 7f3b0de00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f3b10161a90 0x7f3b1006e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:42184/0 (socket says 172.21.15.67:42184) 2024-05-30T16:57:15.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.541+0000 7f3b0de00640 1 -- 172.21.15.67:0/3771093649 learned_addr learned my addr 172.21.15.67:0/3771093649 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.543+0000 7f3b0e800640 1 --2- 172.21.15.67:0/3771093649 >> v2:172.21.15.169:3300/0 conn(0x7f3b100b3800 0x7f3b100ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.544+0000 7f3b05400640 1 --2- 172.21.15.67:0/3771093649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161e70 0x7f3b100b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.544+0000 7f3b0d400640 1 -- 172.21.15.67:0/3771093649 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318419273 0 0) 0x7f3b100b3d40 con 0x7f3b10161a90 2024-05-30T16:57:15.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.544+0000 7f3b0d400640 1 -- 172.21.15.67:0/3771093649 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3af4003520 con 0x7f3b10161a90 2024-05-30T16:57:15.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.545+0000 7f3b05400640 1 -- 172.21.15.67:0/3771093649 >> v2:172.21.15.169:3300/0 conn(0x7f3b100b3800 msgr2=0x7f3b100ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:15.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.545+0000 7f3b05400640 1 --2- 172.21.15.67:0/3771093649 >> v2:172.21.15.169:3300/0 conn(0x7f3b100b3800 0x7f3b100ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:15.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.545+0000 7f3b05400640 1 -- 172.21.15.67:0/3771093649 >> v1:172.21.15.169:6789/0 conn(0x7f3b10161a90 legacy=0x7f3b1006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:15.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.545+0000 7f3b05400640 1 -- 172.21.15.67:0/3771093649 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1006dc70 con 0x7f3b10161e70 2024-05-30T16:57:15.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.549+0000 7f3b05400640 1 --2- 172.21.15.67:0/3771093649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161e70 0x7f3b100b32c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3af80097a0 tx=0x7f3af802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f09e2a847f268b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:15.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.550+0000 7f3b0d400640 1 -- 172.21.15.67:0/3771093649 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8037070 con 0x7f3b10161e70 2024-05-30T16:57:15.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.550+0000 7f3b0d400640 1 -- 172.21.15.67:0/3771093649 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af802ae40 con 0x7f3b10161e70 2024-05-30T16:57:15.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.550+0000 7f3b0d400640 1 -- 172.21.15.67:0/3771093649 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8032660 con 0x7f3b10161e70 2024-05-30T16:57:15.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.551+0000 7f3b14c00640 1 -- 172.21.15.67:0/3771093649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161e70 msgr2=0x7f3b100b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:15.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.551+0000 7f3b14c00640 1 --2- 172.21.15.67:0/3771093649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161e70 0x7f3b100b32c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3af80097a0 tx=0x7f3af802a130 comp rx=0 tx=0).stop 2024-05-30T16:57:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.553+0000 7f3b14c00640 1 -- 172.21.15.67:0/3771093649 shutdown_connections 2024-05-30T16:57:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.553+0000 7f3b14c00640 1 --2- 172.21.15.67:0/3771093649 >> v2:172.21.15.169:3300/0 conn(0x7f3b100b3800 0x7f3b100ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.553+0000 7f3b14c00640 1 --2- 172.21.15.67:0/3771093649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161e70 0x7f3b100b32c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.553+0000 7f3b14c00640 1 -- 172.21.15.67:0/3771093649 >> 172.21.15.67:0/3771093649 conn(0x7f3b1006bd20 msgr2=0x7f3b1006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.554+0000 7f3b14c00640 1 -- 172.21.15.67:0/3771093649 shutdown_connections 2024-05-30T16:57:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.554+0000 7f3b14c00640 1 -- 172.21.15.67:0/3771093649 wait complete. 2024-05-30T16:57:15.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.558+0000 7f3b14c00640 1 Processor -- start 2024-05-30T16:57:15.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.571+0000 7f3b14c00640 1 -- start start 2024-05-30T16:57:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.578+0000 7f3b14c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b100b3800 0x7f3b1014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.580+0000 7f3b14c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161a90 0x7f3b10140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:15.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.582+0000 7f3b14c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10161e70 0x7f3b10141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:15.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.582+0000 7f3b0de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b100b3800 0x7f3b1014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:15.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.582+0000 7f3b0de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b100b3800 0x7f3b1014a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:50274/0 (socket says 172.21.15.67:50274) 2024-05-30T16:57:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.585+0000 7f3b05400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161a90 0x7f3b10140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.585+0000 7f3b05400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161a90 0x7f3b10140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:53218/0 (socket says 172.21.15.67:53218) 2024-05-30T16:57:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.585+0000 7f3b0e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10161e70 0x7f3b10141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.585+0000 7f3b0e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10161e70 0x7f3b10141310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55940/0 (socket says 172.21.15.67:55940) 2024-05-30T16:57:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.585+0000 7f3b14c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1006eb80 con 0x7f3b10161e70 2024-05-30T16:57:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.585+0000 7f3b14c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3b100b5020 con 0x7f3b100b3800 2024-05-30T16:57:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.585+0000 7f3b14c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3b100ad7e0 con 0x7f3b10161a90 2024-05-30T16:57:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.586+0000 7f3b0de00640 1 -- 172.21.15.67:0/839776259 learned_addr learned my addr 172.21.15.67:0/839776259 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:15.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.587+0000 7f3b0e800640 1 -- 172.21.15.67:0/839776259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161a90 msgr2=0x7f3b10140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:15.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.587+0000 7f3b0e800640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161a90 0x7f3b10140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:15.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.587+0000 7f3b0e800640 1 -- 172.21.15.67:0/839776259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b100b3800 msgr2=0x7f3b1014a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:15.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.587+0000 7f3b0e800640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b100b3800 0x7f3b1014a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:15.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.587+0000 7f3b0e800640 1 -- 172.21.15.67:0/839776259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1014a9d0 con 0x7f3b10161e70 2024-05-30T16:57:15.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.590+0000 7f3b0e800640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10161e70 0x7f3b10141310 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800d560 tx=0x7f3b0800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:15.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.591+0000 7f3b06a00640 1 -- 172.21.15.67:0/839776259 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b080125f0 con 0x7f3b10161e70 2024-05-30T16:57:15.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.592+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1014acb0 con 0x7f3b10161e70 2024-05-30T16:57:15.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.593+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1014b270 con 0x7f3b10161e70 2024-05-30T16:57:15.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.593+0000 7f3b06a00640 1 -- 172.21.15.67:0/839776259 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b08019070 con 0x7f3b10161e70 2024-05-30T16:57:15.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.593+0000 7f3b06a00640 1 -- 172.21.15.67:0/839776259 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b080125f0 con 0x7f3b10161e70 2024-05-30T16:57:15.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.604+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b100cf690 con 0x7f3b10161e70 2024-05-30T16:57:15.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.607+0000 7f3b06a00640 1 -- 172.21.15.67:0/839776259 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f3b08012790 con 0x7f3b10161e70 2024-05-30T16:57:15.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.610+0000 7f3b06a00640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3aec06dc00 0x7f3aec0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:15.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.611+0000 7f3b06a00640 1 -- 172.21.15.67:0/839776259 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f3b0808a680 con 0x7f3b10161e70 2024-05-30T16:57:15.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.622+0000 7f3b0de00640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3aec06dc00 0x7f3aec0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:15.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.626+0000 7f3b0de00640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3aec06dc00 0x7f3aec0700c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3b00001560 tx=0x7f3b00005b70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:15.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:15.627+0000 7f3b06a00640 1 -- 172.21.15.67:0/839776259 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b100cf690 con 0x7f3b10161e70 2024-05-30T16:57:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:16.158+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922818", "target": ["mon-mgr", ""]}) -- 0x7f3b100689e0 con 0x7f3aec06dc00 2024-05-30T16:57:17.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.224+0000 7f3b06a00640 1 -- 172.21.15.67:0/839776259 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b100689e0 con 0x7f3aec06dc00 2024-05-30T16:57:17.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.230+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3aec06dc00 msgr2=0x7f3aec0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:17.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.230+0000 7f3b14c00640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3aec06dc00 0x7f3aec0700c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3b00001560 tx=0x7f3b00005b70 comp rx=0 tx=0).stop 2024-05-30T16:57:17.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.230+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10161e70 msgr2=0x7f3b10141310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:17.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.230+0000 7f3b14c00640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10161e70 0x7f3b10141310 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800d560 tx=0x7f3b0800da20 comp rx=0 tx=0).stop 2024-05-30T16:57:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.231+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 shutdown_connections 2024-05-30T16:57:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.231+0000 7f3b14c00640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3aec06dc00 0x7f3aec0700c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.231+0000 7f3b14c00640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b10161e70 0x7f3b10141310 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.231+0000 7f3b14c00640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b10161a90 0x7f3b10140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.231+0000 7f3b14c00640 1 --2- 172.21.15.67:0/839776259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b100b3800 0x7f3b1014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.231+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 >> 172.21.15.67:0/839776259 conn(0x7f3b1006bd20 msgr2=0x7f3b100b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:17.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.232+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 shutdown_connections 2024-05-30T16:57:17.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.233+0000 7f3b14c00640 1 -- 172.21.15.67:0/839776259 wait complete. 2024-05-30T16:57:17.293 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922819 2024-05-30T16:57:17.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.689+0000 7f7ebb600640 1 Processor -- start 2024-05-30T16:57:17.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.691+0000 7f7ebb600640 1 -- start start 2024-05-30T16:57:17.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.695+0000 7f7ebb600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7eb40cef30 0x7f7eb40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:17.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.695+0000 7f7ebb600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7eb40d1b80 0x7f7eb40b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:17.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.695+0000 7f7ebb600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb40d1f60 con 0x7f7eb415e850 2024-05-30T16:57:17.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.695+0000 7f7ebb600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb415ec30 con 0x7f7eb40d1b80 2024-05-30T16:57:17.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.695+0000 7f7ebb600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb40b8000 con 0x7f7eb40cef30 2024-05-30T16:57:17.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.696+0000 7f7eb3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7eb40cef30 0x7f7eb40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:17.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.696+0000 7f7eb3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7eb40cef30 0x7f7eb40d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:50290/0 (socket says 172.21.15.67:50290) 2024-05-30T16:57:17.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.696+0000 7f7eb3e00640 1 -- 172.21.15.67:0/1438069691 learned_addr learned my addr 172.21.15.67:0/1438069691 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:17.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.697+0000 7f7eb3e00640 1 -- 172.21.15.67:0/1438069691 >> v2:172.21.15.67:3300/0 conn(0x7f7eb40d1b80 msgr2=0x7f7eb40b90a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-30T16:57:17.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.697+0000 7f7eb3e00640 1 --2- 172.21.15.67:0/1438069691 >> v2:172.21.15.67:3300/0 conn(0x7f7eb40d1b80 0x7f7eb40b90a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.697+0000 7f7eb3e00640 1 -- 172.21.15.67:0/1438069691 >> v1:172.21.15.169:6789/0 conn(0x7f7eb415e850 legacy=0x7f7eb40b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:17.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.697+0000 7f7eb3e00640 1 -- 172.21.15.67:0/1438069691 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb40b82e0 con 0x7f7eb40cef30 2024-05-30T16:57:17.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.699+0000 7f7eb3e00640 1 --2- 172.21.15.67:0/1438069691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 0x7f7eb40d1640 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40cb730 tx=0x7f7eac02a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aad78a89ae3524ad server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:17.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.700+0000 7f7eb3400640 1 -- 172.21.15.67:0/1438069691 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac036070 con 0x7f7eb40cef30 2024-05-30T16:57:17.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.700+0000 7f7eb3400640 1 -- 172.21.15.67:0/1438069691 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eac031600 con 0x7f7eb40cef30 2024-05-30T16:57:17.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.700+0000 7f7eb3400640 1 -- 172.21.15.67:0/1438069691 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0319d0 con 0x7f7eb40cef30 2024-05-30T16:57:17.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.702+0000 7f7ebb600640 1 -- 172.21.15.67:0/1438069691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 msgr2=0x7f7eb40d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:17.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.702+0000 7f7ebb600640 1 --2- 172.21.15.67:0/1438069691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 0x7f7eb40d1640 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40cb730 tx=0x7f7eac02a6c0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.706+0000 7f7ebb600640 1 -- 172.21.15.67:0/1438069691 shutdown_connections 2024-05-30T16:57:17.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.706+0000 7f7ebb600640 1 --2- 172.21.15.67:0/1438069691 >> v2:172.21.15.67:3300/0 conn(0x7f7eb40d1b80 0x7f7eb40b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.706+0000 7f7ebb600640 1 --2- 172.21.15.67:0/1438069691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 0x7f7eb40d1640 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.706+0000 7f7ebb600640 1 -- 172.21.15.67:0/1438069691 >> 172.21.15.67:0/1438069691 conn(0x7f7eb4067aa0 msgr2=0x7f7eb4067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:17.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.706+0000 7f7ebb600640 1 -- 172.21.15.67:0/1438069691 shutdown_connections 2024-05-30T16:57:17.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.706+0000 7f7ebb600640 1 -- 172.21.15.67:0/1438069691 wait complete. 2024-05-30T16:57:17.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.715+0000 7f7ebb600640 1 Processor -- start 2024-05-30T16:57:17.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.715+0000 7f7ebb600640 1 -- start start 2024-05-30T16:57:17.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.725+0000 7f7ebb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 0x7f7eb414db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:17.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.725+0000 7f7ebb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eb40d1b80 0x7f7eb415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:17.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.726+0000 7f7ebb600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7eb415e850 0x7f7eb415d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:17.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.729+0000 7f7ebb600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40b95e0 con 0x7f7eb415e850 2024-05-30T16:57:17.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.729+0000 7f7ebb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40b88a0 con 0x7f7eb40cef30 2024-05-30T16:57:17.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.729+0000 7f7ebb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb40b9760 con 0x7f7eb40d1b80 2024-05-30T16:57:17.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.729+0000 7f7eb3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eb40d1b80 0x7f7eb415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:17.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.730+0000 7f7eb3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eb40d1b80 0x7f7eb415cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:53224/0 (socket says 172.21.15.67:53224) 2024-05-30T16:57:17.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.730+0000 7f7eb3e00640 1 -- 172.21.15.67:0/2972327161 learned_addr learned my addr 172.21.15.67:0/2972327161 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:17.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.729+0000 7f7eb8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 0x7f7eb414db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:17.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.731+0000 7f7eb9600640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7eb415e850 0x7f7eb415d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:17.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.734+0000 7f7eb8c00640 1 -- 172.21.15.67:0/2972327161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eb40d1b80 msgr2=0x7f7eb415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:17.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.734+0000 7f7eb8c00640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eb40d1b80 0x7f7eb415cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.734+0000 7f7eb8c00640 1 -- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7eb415e850 msgr2=0x7f7eb415d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.735+0000 7f7eb8c00640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7eb415e850 0x7f7eb415d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.735+0000 7f7eb8c00640 1 -- 172.21.15.67:0/2972327161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb414e0b0 con 0x7f7eb40cef30 2024-05-30T16:57:17.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.736+0000 7f7eb9600640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7eb415e850 0x7f7eb415d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:57:17.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.736+0000 7f7eb3e00640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eb40d1b80 0x7f7eb415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:57:17.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.738+0000 7f7eb8c00640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 0x7f7eb414db70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c3c0 tx=0x7f7ea000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:17.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.740+0000 7f7eb0c00640 1 -- 172.21.15.67:0/2972327161 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0005220 con 0x7f7eb40cef30 2024-05-30T16:57:17.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.741+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb41f5480 con 0x7f7eb40cef30 2024-05-30T16:57:17.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.741+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb41f8250 con 0x7f7eb40cef30 2024-05-30T16:57:17.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.741+0000 7f7eb0c00640 1 -- 172.21.15.67:0/2972327161 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea00053c0 con 0x7f7eb40cef30 2024-05-30T16:57:17.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.741+0000 7f7eb0c00640 1 -- 172.21.15.67:0/2972327161 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0007230 con 0x7f7eb40cef30 2024-05-30T16:57:17.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.750+0000 7f7eb0c00640 1 -- 172.21.15.67:0/2972327161 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f7ea0019020 con 0x7f7eb40cef30 2024-05-30T16:57:17.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.751+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb4157e70 con 0x7f7eb40cef30 2024-05-30T16:57:17.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.756+0000 7f7eb0c00640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e9406da20 0x7f7e9406fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:17.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.757+0000 7f7eb0c00640 1 -- 172.21.15.67:0/2972327161 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f7ea00898d0 con 0x7f7eb40cef30 2024-05-30T16:57:17.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.766+0000 7f7eb3e00640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e9406da20 0x7f7e9406fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:17.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.772+0000 7f7eb0c00640 1 -- 172.21.15.67:0/2972327161 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7eb414e0b0 con 0x7f7eb40cef30 2024-05-30T16:57:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:17.793+0000 7f7eb3e00640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e9406da20 0x7f7e9406fee0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40c8190 tx=0x7f7eac009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:18.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.223+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922819", "target": ["mon-mgr", ""]}) -- 0x7f7eb40689e0 con 0x7f7e9406da20 2024-05-30T16:57:18.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.605+0000 7f7eb0c00640 1 -- 172.21.15.67:0/2972327161 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7eb40689e0 con 0x7f7e9406da20 2024-05-30T16:57:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e9406da20 msgr2=0x7f7e9406fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e9406da20 0x7f7e9406fee0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40c8190 tx=0x7f7eac009000 comp rx=0 tx=0).stop 2024-05-30T16:57:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 msgr2=0x7f7eb414db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 0x7f7eb414db70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c3c0 tx=0x7f7ea000c880 comp rx=0 tx=0).stop 2024-05-30T16:57:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 shutdown_connections 2024-05-30T16:57:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7e9406da20 0x7f7e9406fee0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7eb415e850 0x7f7eb415d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eb40d1b80 0x7f7eb415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 --2- 172.21.15.67:0/2972327161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eb40cef30 0x7f7eb414db70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 >> 172.21.15.67:0/2972327161 conn(0x7f7eb4067aa0 msgr2=0x7f7eb40c8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.613+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 shutdown_connections 2024-05-30T16:57:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.614+0000 7f7ebb600640 1 -- 172.21.15.67:0/2972327161 wait complete. 2024-05-30T16:57:18.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone922820 2024-05-30T16:57:18.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.969+0000 7f9eb7c00640 1 Processor -- start 2024-05-30T16:57:18.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.970+0000 7f9eb7c00640 1 -- start start 2024-05-30T16:57:18.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.973+0000 7f9eb7c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9eb00b68f0 0x7f9eb00cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:18.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.973+0000 7f9eb7c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b7e50 0x7f9eb00cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:18.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.973+0000 7f9eb7c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb015f160 con 0x7f9eb015ed80 2024-05-30T16:57:18.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.973+0000 7f9eb7c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb0163540 con 0x7f9eb00b68f0 2024-05-30T16:57:18.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.973+0000 7f9eb7c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb0162930 con 0x7f9eb00b7e50 2024-05-30T16:57:18.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.973+0000 7f9eb5200640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f9eb015ed80 0x7f9eb00b61e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:42206/0 (socket says 172.21.15.67:42206) 2024-05-30T16:57:18.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.973+0000 7f9eb5200640 1 -- 172.21.15.67:0/811955884 learned_addr learned my addr 172.21.15.67:0/811955884 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:18.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.974+0000 7f9eafe00640 1 --2- 172.21.15.67:0/811955884 >> v2:172.21.15.169:3300/0 conn(0x7f9eb00b68f0 0x7f9eb00cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:18.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.974+0000 7f9eaf400640 1 -- 172.21.15.67:0/811955884 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397332879 0 0) 0x7f9eb015f160 con 0x7f9eb015ed80 2024-05-30T16:57:18.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.974+0000 7f9eaf400640 1 -- 172.21.15.67:0/811955884 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e94003520 con 0x7f9eb015ed80 2024-05-30T16:57:18.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.974+0000 7f9eafe00640 1 -- 172.21.15.67:0/811955884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b7e50 msgr2=0x7f9eb00cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:18.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.974+0000 7f9eafe00640 1 --2- 172.21.15.67:0/811955884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b7e50 0x7f9eb00cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:18.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.974+0000 7f9eafe00640 1 -- 172.21.15.67:0/811955884 >> v1:172.21.15.169:6789/0 conn(0x7f9eb015ed80 legacy=0x7f9eb00b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:18.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.974+0000 7f9eafe00640 1 -- 172.21.15.67:0/811955884 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb0162c10 con 0x7f9eb00b68f0 2024-05-30T16:57:18.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.975+0000 7f9eafe00640 1 --2- 172.21.15.67:0/811955884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb00b68f0 0x7f9eb00cff80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009d90 tx=0x7f9ea802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95d0dde62fef7d59 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:18.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.976+0000 7f9eaf400640 1 -- 172.21.15.67:0/811955884 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8007740 con 0x7f9eb00b68f0 2024-05-30T16:57:18.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.976+0000 7f9eaf400640 1 -- 172.21.15.67:0/811955884 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea80078e0 con 0x7f9eb00b68f0 2024-05-30T16:57:18.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.976+0000 7f9eaf400640 1 -- 172.21.15.67:0/811955884 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8007c90 con 0x7f9eb00b68f0 2024-05-30T16:57:18.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.977+0000 7f9eb7c00640 1 -- 172.21.15.67:0/811955884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb00b68f0 msgr2=0x7f9eb00cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:18.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.977+0000 7f9eb7c00640 1 --2- 172.21.15.67:0/811955884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb00b68f0 0x7f9eb00cff80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009d90 tx=0x7f9ea802a660 comp rx=0 tx=0).stop 2024-05-30T16:57:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.977+0000 7f9eb7c00640 1 -- 172.21.15.67:0/811955884 shutdown_connections 2024-05-30T16:57:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.977+0000 7f9eb7c00640 1 --2- 172.21.15.67:0/811955884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b7e50 0x7f9eb00cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.977+0000 7f9eb7c00640 1 --2- 172.21.15.67:0/811955884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb00b68f0 0x7f9eb00cff80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.977+0000 7f9eb7c00640 1 -- 172.21.15.67:0/811955884 >> 172.21.15.67:0/811955884 conn(0x7f9eb00d13c0 msgr2=0x7f9eb00d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.977+0000 7f9eb7c00640 1 -- 172.21.15.67:0/811955884 shutdown_connections 2024-05-30T16:57:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.977+0000 7f9eb7c00640 1 -- 172.21.15.67:0/811955884 wait complete. 2024-05-30T16:57:18.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.979+0000 7f9eb7c00640 1 Processor -- start 2024-05-30T16:57:18.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.979+0000 7f9eb7c00640 1 -- start start 2024-05-30T16:57:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.980+0000 7f9eb7c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b68f0 0x7f9eb00c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.980+0000 7f9eb7c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb00b7e50 0x7f9eb01504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.980+0000 7f9eb7c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb015ed80 0x7f9eb0150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.980+0000 7f9eb7c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb00677b0 con 0x7f9eb00b7e50 2024-05-30T16:57:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.980+0000 7f9eb7c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01631d0 con 0x7f9eb015ed80 2024-05-30T16:57:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.980+0000 7f9eb7c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb0067930 con 0x7f9eb00b68f0 2024-05-30T16:57:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.980+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb00b7e50 0x7f9eb01504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.981+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb00b7e50 0x7f9eb01504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55956/0 (socket says 172.21.15.67:55956) 2024-05-30T16:57:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.981+0000 7f9eafe00640 1 -- 172.21.15.67:0/1728165921 learned_addr learned my addr 172.21.15.67:0/1728165921 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.981+0000 7f9eb5200640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b68f0 0x7f9eb00c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.981+0000 7f9eb5c00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb015ed80 0x7f9eb0150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.981+0000 7f9eafe00640 1 -- 172.21.15.67:0/1728165921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b68f0 msgr2=0x7f9eb00c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.981+0000 7f9eafe00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b68f0 0x7f9eb00c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.981+0000 7f9eafe00640 1 -- 172.21.15.67:0/1728165921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb015ed80 msgr2=0x7f9eb0150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.981+0000 7f9eafe00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb015ed80 0x7f9eb0150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.981+0000 7f9eafe00640 1 -- 172.21.15.67:0/1728165921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb00c84e0 con 0x7f9eb00b7e50 2024-05-30T16:57:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.982+0000 7f9eafe00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb00b7e50 0x7f9eb01504f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009eb0 tx=0x7f9ea8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.982+0000 7f9eacc00640 1 -- 172.21.15.67:0/1728165921 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8040070 con 0x7f9eb00b7e50 2024-05-30T16:57:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.982+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb00c87c0 con 0x7f9eb00b7e50 2024-05-30T16:57:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.983+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb0160620 con 0x7f9eb00b7e50 2024-05-30T16:57:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.983+0000 7f9eacc00640 1 -- 172.21.15.67:0/1728165921 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea8007840 con 0x7f9eb00b7e50 2024-05-30T16:57:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.983+0000 7f9eacc00640 1 -- 172.21.15.67:0/1728165921 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8009450 con 0x7f9eb00b7e50 2024-05-30T16:57:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.984+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e88005290 con 0x7f9eb00b7e50 2024-05-30T16:57:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.987+0000 7f9eacc00640 1 -- 172.21.15.67:0/1728165921 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f9ea8039040 con 0x7f9eb00b7e50 2024-05-30T16:57:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.988+0000 7f9eacc00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e9006d9d0 0x7f9e9006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.988+0000 7f9eacc00640 1 -- 172.21.15.67:0/1728165921 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f9ea80ac380 con 0x7f9eb00b7e50 2024-05-30T16:57:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.989+0000 7f9eb5200640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e9006d9d0 0x7f9e9006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:18.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:18.993+0000 7f9eacc00640 1 -- 172.21.15.67:0/1728165921 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ea807bcd0 con 0x7f9eb00b7e50 2024-05-30T16:57:19.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.038+0000 7f9eb5200640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e9006d9d0 0x7f9e9006fe90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005880 tx=0x7f9e9c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.430+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone922820", "target": ["mon-mgr", ""]}) -- 0x7f9e88000bc0 con 0x7f9e9006d9d0 2024-05-30T16:57:19.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.637+0000 7f9eacc00640 1 -- 172.21.15.67:0/1728165921 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9e88000bc0 con 0x7f9e9006d9d0 2024-05-30T16:57:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e9006d9d0 msgr2=0x7f9e9006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e9006d9d0 0x7f9e9006fe90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005880 tx=0x7f9e9c009040 comp rx=0 tx=0).stop 2024-05-30T16:57:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb00b7e50 msgr2=0x7f9eb01504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb00b7e50 0x7f9eb01504f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009eb0 tx=0x7f9ea8004a40 comp rx=0 tx=0).stop 2024-05-30T16:57:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 shutdown_connections 2024-05-30T16:57:19.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e9006d9d0 0x7f9e9006fe90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb015ed80 0x7f9eb0150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9eb00b7e50 0x7f9eb01504f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 --2- 172.21.15.67:0/1728165921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b68f0 0x7f9eb00c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.639+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 >> 172.21.15.67:0/1728165921 conn(0x7f9eb00d13c0 msgr2=0x7f9eb0159440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:19.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.640+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 shutdown_connections 2024-05-30T16:57:19.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.640+0000 7f9eb7c00640 1 -- 172.21.15.67:0/1728165921 wait complete. 2024-05-30T16:57:19.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:57:19.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.830+0000 7f6776800640 1 Processor -- start 2024-05-30T16:57:19.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.830+0000 7f6776800640 1 -- start start 2024-05-30T16:57:19.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.832+0000 7f6776800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6770156b30 0x7f67700af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:19.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.832+0000 7f6776800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f67700ad940 0x7f67700b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:19.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.832+0000 7f6776800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6770156f10 con 0x7f677006ee40 2024-05-30T16:57:19.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.832+0000 7f6776800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f677006dda0 con 0x7f6770156b30 2024-05-30T16:57:19.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.832+0000 7f6776800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f677006e080 con 0x7f67700ad940 2024-05-30T16:57:19.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.832+0000 7f6775400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6770156b30 0x7f67700af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:19.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.832+0000 7f6775400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6770156b30 0x7f67700af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55964/0 (socket says 172.21.15.67:55964) 2024-05-30T16:57:19.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.832+0000 7f6775400640 1 -- 172.21.15.67:0/2708691033 learned_addr learned my addr 172.21.15.67:0/2708691033 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:19.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.832+0000 7f6774a00640 1 --1- 172.21.15.67:0/2708691033 >> v1:172.21.15.67:6789/0 conn(0x7f677006ee40 0x7f67700ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45416/0 (socket says 172.21.15.67:45416) 2024-05-30T16:57:19.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.833+0000 7f6775e00640 1 --2- 172.21.15.67:0/2708691033 >> v2:172.21.15.169:3300/0 conn(0x7f67700ad940 0x7f67700b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:19.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.833+0000 7f6775400640 1 -- 172.21.15.67:0/2708691033 >> v2:172.21.15.169:3300/0 conn(0x7f67700ad940 msgr2=0x7f67700b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:19.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.833+0000 7f6775400640 1 --2- 172.21.15.67:0/2708691033 >> v2:172.21.15.169:3300/0 conn(0x7f67700ad940 0x7f67700b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.833+0000 7f6775400640 1 -- 172.21.15.67:0/2708691033 >> v1:172.21.15.67:6789/0 conn(0x7f677006ee40 legacy=0x7f67700ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:19.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.833+0000 7f6775400640 1 -- 172.21.15.67:0/2708691033 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677006e360 con 0x7f6770156b30 2024-05-30T16:57:19.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.833+0000 7f6775e00640 1 --2- 172.21.15.67:0/2708691033 >> v2:172.21.15.169:3300/0 conn(0x7f67700ad940 0x7f67700b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.834+0000 7f6775400640 1 --2- 172.21.15.67:0/2708691033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 0x7f67700af6e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f67700d0180 tx=0x7f676402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c32eb8f7567f2d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f676fe00640 1 -- 172.21.15.67:0/2708691033 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764037070 con 0x7f6770156b30 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f676fe00640 1 -- 172.21.15.67:0/2708691033 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6764032440 con 0x7f6770156b30 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f676fe00640 1 -- 172.21.15.67:0/2708691033 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764032810 con 0x7f6770156b30 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f6776800640 1 -- 172.21.15.67:0/2708691033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 msgr2=0x7f67700af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f6776800640 1 --2- 172.21.15.67:0/2708691033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 0x7f67700af6e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f67700d0180 tx=0x7f676402a1e0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f6776800640 1 -- 172.21.15.67:0/2708691033 shutdown_connections 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f6776800640 1 --2- 172.21.15.67:0/2708691033 >> v2:172.21.15.169:3300/0 conn(0x7f67700ad940 0x7f67700b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f6776800640 1 --2- 172.21.15.67:0/2708691033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 0x7f67700af6e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f6776800640 1 -- 172.21.15.67:0/2708691033 >> 172.21.15.67:0/2708691033 conn(0x7f67700b1fc0 msgr2=0x7f67700b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f6776800640 1 -- 172.21.15.67:0/2708691033 shutdown_connections 2024-05-30T16:57:19.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.835+0000 7f6776800640 1 -- 172.21.15.67:0/2708691033 wait complete. 2024-05-30T16:57:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.837+0000 7f6776800640 1 Processor -- start 2024-05-30T16:57:19.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.837+0000 7f6776800640 1 -- start start 2024-05-30T16:57:19.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.838+0000 7f6776800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677006ee40 0x7f67700b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:19.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.838+0000 7f6776800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67700ad940 0x7f67700af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:19.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.838+0000 7f6776800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 0x7f67700af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:19.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.838+0000 7f6776800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f67700afc20 con 0x7f6770156b30 2024-05-30T16:57:19.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.838+0000 7f6776800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f67700b2c30 con 0x7f677006ee40 2024-05-30T16:57:19.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.838+0000 7f6776800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f67700cf4f0 con 0x7f67700ad940 2024-05-30T16:57:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.838+0000 7f6775e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 0x7f67700af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.838+0000 7f6775e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 0x7f67700af5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55968/0 (socket says 172.21.15.67:55968) 2024-05-30T16:57:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.838+0000 7f6775e00640 1 -- 172.21.15.67:0/1264847305 learned_addr learned my addr 172.21.15.67:0/1264847305 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f6775e00640 1 -- 172.21.15.67:0/1264847305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67700ad940 msgr2=0x7f67700af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f6775400640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677006ee40 0x7f67700b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f6775e00640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67700ad940 0x7f67700af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f6775e00640 1 -- 172.21.15.67:0/1264847305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677006ee40 msgr2=0x7f67700b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f6775e00640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677006ee40 0x7f67700b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:19.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f6775e00640 1 -- 172.21.15.67:0/1264847305 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67700b3230 con 0x7f6770156b30 2024-05-30T16:57:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f6775e00640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 0x7f67700af5a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f676800ce40 tx=0x7f676800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f676d600640 1 -- 172.21.15.67:0/1264847305 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676801a070 con 0x7f6770156b30 2024-05-30T16:57:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f676d600640 1 -- 172.21.15.67:0/1264847305 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676800d840 con 0x7f6770156b30 2024-05-30T16:57:19.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67700b34f0 con 0x7f6770156b30 2024-05-30T16:57:19.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f676d600640 1 -- 172.21.15.67:0/1264847305 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676800dc30 con 0x7f6770156b30 2024-05-30T16:57:19.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.839+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67700b3fd0 con 0x7f6770156b30 2024-05-30T16:57:19.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.841+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67700689e0 con 0x7f6770156b30 2024-05-30T16:57:19.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.841+0000 7f676d600640 1 -- 172.21.15.67:0/1264847305 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f676801d070 con 0x7f6770156b30 2024-05-30T16:57:19.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.842+0000 7f676d600640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f675006da00 0x7f675006fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:19.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.842+0000 7f676d600640 1 -- 172.21.15.67:0/1264847305 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f676808b220 con 0x7f6770156b30 2024-05-30T16:57:19.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.842+0000 7f6775400640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f675006da00 0x7f675006fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:19.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.843+0000 7f6775400640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f675006da00 0x7f675006fec0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6764034000 tx=0x7f676402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:19.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:19.846+0000 7f676d600640 1 -- 172.21.15.67:0/1264847305 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f676805ad60 con 0x7f6770156b30 2024-05-30T16:57:20.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.069+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f677006f7f0 con 0x7f675006da00 2024-05-30T16:57:20.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.072+0000 7f676d600640 1 -- 172.21.15.67:0/1264847305 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f677006f7f0 con 0x7f675006da00 2024-05-30T16:57:20.074 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:57:20.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:20.074 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:57:20.074 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:20.074 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:57:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.074+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f675006da00 msgr2=0x7f675006fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.074+0000 7f6776800640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f675006da00 0x7f675006fec0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6764034000 tx=0x7f676402e000 comp rx=0 tx=0).stop 2024-05-30T16:57:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.074+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 msgr2=0x7f67700af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.074+0000 7f6776800640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 0x7f67700af5a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f676800ce40 tx=0x7f676800a710 comp rx=0 tx=0).stop 2024-05-30T16:57:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.075+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 shutdown_connections 2024-05-30T16:57:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.075+0000 7f6776800640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f675006da00 0x7f675006fec0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.075+0000 7f6776800640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6770156b30 0x7f67700af5a0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.075+0000 7f6776800640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f67700ad940 0x7f67700af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.075+0000 7f6776800640 1 --2- 172.21.15.67:0/1264847305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f677006ee40 0x7f67700b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.075+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 >> 172.21.15.67:0/1264847305 conn(0x7f67700b1fc0 msgr2=0x7f6770071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:20.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.075+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 shutdown_connections 2024-05-30T16:57:20.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.075+0000 7f6776800640 1 -- 172.21.15.67:0/1264847305 wait complete. 2024-05-30T16:57:20.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:57:20.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.282+0000 7f185dc00640 1 Processor -- start 2024-05-30T16:57:20.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.282+0000 7f185dc00640 1 -- start start 2024-05-30T16:57:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.284+0000 7f185dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f18580d0b70 0x7f1858169460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.284+0000 7f185dc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18581621d0 con 0x7f1858161df0 2024-05-30T16:57:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.284+0000 7f185dc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18580ccf10 con 0x7f18580cc2b0 2024-05-30T16:57:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.284+0000 7f185dc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f18580cd840 con 0x7f18580d0b70 2024-05-30T16:57:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.284+0000 7f1857e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1858161df0 0x7f1858099800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45432/0 (socket says 172.21.15.67:45432) 2024-05-30T16:57:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.284+0000 7f1857e00640 1 -- 172.21.15.67:0/3886744375 learned_addr learned my addr 172.21.15.67:0/3886744375 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4109893761 0 0) 0x7f1848001110 con 0x7f1858161df0 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f183c003520 con 0x7f1858161df0 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f185d200640 1 --2- 172.21.15.67:0/3886744375 >> v2:172.21.15.169:3300/0 conn(0x7f18580d0b70 0x7f1858169460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2443706417 0 0) 0x7f183c003520 con 0x7f1858161df0 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 >> v2:172.21.15.169:3300/0 conn(0x7f18580d0b70 msgr2=0x7f1858169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 --2- 172.21.15.67:0/3886744375 >> v2:172.21.15.169:3300/0 conn(0x7f18580d0b70 0x7f1858169460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 >> v1:172.21.15.169:6789/0 conn(0x7f18580cc2b0 legacy=0x7f18580d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580cd1f0 con 0x7f1858161df0 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f1848001840 con 0x7f1858161df0 2024-05-30T16:57:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 >> v1:172.21.15.67:6789/0 conn(0x7f1858161df0 legacy=0x7f1858099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0039b0 0x7f183c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c004800 0x7f183c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f183c0051c0 0x7f183c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00c0c0 con 0x7f183c004800 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f183c00c270 con 0x7f183c0039b0 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.285+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f183c00c470 con 0x7f183c0051c0 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.286+0000 7f184fe00640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c004800 0x7f183c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.286+0000 7f1857e00640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0039b0 0x7f183c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.286+0000 7f185d200640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f183c0051c0 0x7f183c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.286+0000 7f184fe00640 1 -- 172.21.15.67:0/3886744375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f183c0051c0 msgr2=0x7f183c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.286+0000 7f184fe00640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f183c0051c0 0x7f183c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.286+0000 7f184fe00640 1 -- 172.21.15.67:0/3886744375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0039b0 msgr2=0x7f183c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.286+0000 7f184fe00640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c0039b0 0x7f183c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.286+0000 7f185d200640 1 -- 172.21.15.67:0/3886744375 reap_dead start 2024-05-30T16:57:20.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.286+0000 7f184fe00640 1 -- 172.21.15.67:0/3886744375 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c003520 con 0x7f183c004800 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.287+0000 7f184fe00640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c004800 0x7f183c004c80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1840003d80 tx=0x7f183c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf2262fe78d5f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.288+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840037070 con 0x7f183c004800 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.288+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f184002a990 con 0x7f183c004800 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.288+0000 7f1857400640 1 -- 172.21.15.67:0/3886744375 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840003980 con 0x7f183c004800 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.288+0000 7f185dc00640 1 -- 172.21.15.67:0/3886744375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c004800 msgr2=0x7f183c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.288+0000 7f185dc00640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c004800 0x7f183c004c80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1840003d80 tx=0x7f183c005830 comp rx=0 tx=0).stop 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.288+0000 7f185dc00640 1 -- 172.21.15.67:0/3886744375 shutdown_connections 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.288+0000 7f185dc00640 1 --2- 172.21.15.67:0/3886744375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f183c004800 0x7f183c004c80 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.288+0000 7f185dc00640 1 -- 172.21.15.67:0/3886744375 >> 172.21.15.67:0/3886744375 conn(0x7f185806bae0 msgr2=0x7f185806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.288+0000 7f185dc00640 1 -- 172.21.15.67:0/3886744375 shutdown_connections 2024-05-30T16:57:20.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.289+0000 7f185dc00640 1 -- 172.21.15.67:0/3886744375 wait complete. 2024-05-30T16:57:20.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.290+0000 7f185dc00640 1 Processor -- start 2024-05-30T16:57:20.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.290+0000 7f185dc00640 1 -- start start 2024-05-30T16:57:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.291+0000 7f185dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c004800 0x7f185806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.291+0000 7f185dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858153f30 0x7f18580add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.291+0000 7f185dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18580ae2c0 0x7f185806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.291+0000 7f185dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f18580ca870 con 0x7f1858153f30 2024-05-30T16:57:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.291+0000 7f185dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f18580ccbc0 con 0x7f183c004800 2024-05-30T16:57:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.291+0000 7f185dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f18580ca9f0 con 0x7f18580ae2c0 2024-05-30T16:57:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.291+0000 7f184fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858153f30 0x7f18580add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.291+0000 7f184fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858153f30 0x7f18580add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55986/0 (socket says 172.21.15.67:55986) 2024-05-30T16:57:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.291+0000 7f184fe00640 1 -- 172.21.15.67:0/1587391490 learned_addr learned my addr 172.21.15.67:0/1587391490 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:20.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f184fe00640 1 -- 172.21.15.67:0/1587391490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18580ae2c0 msgr2=0x7f185806dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:20.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f184fe00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18580ae2c0 0x7f185806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f184fe00640 1 -- 172.21.15.67:0/1587391490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c004800 msgr2=0x7f185806f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:20.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f184fe00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c004800 0x7f185806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f184fe00640 1 -- 172.21.15.67:0/1587391490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580c72f0 con 0x7f1858153f30 2024-05-30T16:57:20.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f184fe00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858153f30 0x7f18580add80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1840003d80 tx=0x7f18400036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:20.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f1854c00640 1 -- 172.21.15.67:0/1587391490 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840037050 con 0x7f1858153f30 2024-05-30T16:57:20.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f1854c00640 1 -- 172.21.15.67:0/1587391490 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1840038040 con 0x7f1858153f30 2024-05-30T16:57:20.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f1854c00640 1 -- 172.21.15.67:0/1587391490 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840006e50 con 0x7f1858153f30 2024-05-30T16:57:20.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.292+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185806fb00 con 0x7f1858153f30 2024-05-30T16:57:20.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.293+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1858161200 con 0x7f1858153f30 2024-05-30T16:57:20.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.294+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18580ae6e0 con 0x7f1858153f30 2024-05-30T16:57:20.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.294+0000 7f1854c00640 1 -- 172.21.15.67:0/1587391490 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f1840045020 con 0x7f1858153f30 2024-05-30T16:57:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.295+0000 7f1854c00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f183406dba0 0x7f1834070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.295+0000 7f1854c00640 1 -- 172.21.15.67:0/1587391490 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f18400aa450 con 0x7f1858153f30 2024-05-30T16:57:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.296+0000 7f1857e00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f183406dba0 0x7f1834070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.296+0000 7f1857e00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f183406dba0 0x7f1834070060 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1848001d70 tx=0x7f1848001e00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:20.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.299+0000 7f1854c00640 1 -- 172.21.15.67:0/1587391490 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f184007a2d0 con 0x7f1858153f30 2024-05-30T16:57:20.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.568+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f18580af410 con 0x7f1858153f30 2024-05-30T16:57:20.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.569+0000 7f1854c00640 1 -- 172.21.15.67:0/1587391490 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2419 (secure 0 0 0) 0x7f1840079bd0 con 0x7f1858153f30 2024-05-30T16:57:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:20.571 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:57:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:57:20.571 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:57:20.572 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 833, 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:57:20.573 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:20.574 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 12600, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 52634895915, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 105491951616, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 667602161664, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 536220504, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 534525970, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 131, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 144 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:57:20.575 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:20.576 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:57:20.577 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "7c6f9121-d563-4406-bd90-584b3290d1b2": { 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "2 ongoing+pending clones; Avg progress is 31.38% (71s)\n [========....................] (remaining: 2m)", 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31380000710487366, 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "2 ongoing clones; Avg progress is 31.38% (77s)\n [========....................] (remaining: 2m)", 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31380000710487366, 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:20.579 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:20.579 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:57:20.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f183406dba0 msgr2=0x7f1834070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:20.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f183406dba0 0x7f1834070060 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1848001d70 tx=0x7f1848001e00 comp rx=0 tx=0).stop 2024-05-30T16:57:20.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858153f30 msgr2=0x7f18580add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:20.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858153f30 0x7f18580add80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1840003d80 tx=0x7f18400036f0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 shutdown_connections 2024-05-30T16:57:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f183406dba0 0x7f1834070060 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18580ae2c0 0x7f185806dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858153f30 0x7f18580add80 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 --2- 172.21.15.67:0/1587391490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f183c004800 0x7f185806f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 >> 172.21.15.67:0/1587391490 conn(0x7f185806bae0 msgr2=0x7f18580cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 shutdown_connections 2024-05-30T16:57:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:20.572+0000 7f185dc00640 1 -- 172.21.15.67:0/1587391490 wait complete. 2024-05-30T16:57:25.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:57:25.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.810+0000 7f7569000640 1 Processor -- start 2024-05-30T16:57:25.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.811+0000 7f7569000640 1 -- start start 2024-05-30T16:57:25.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.812+0000 7f7569000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7564155a00 0x7f756415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:25.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.812+0000 7f7569000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f756406f390 0x7f756406f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.812+0000 7f7569000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f756406e3d0 con 0x7f75640d0970 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.812+0000 7f7569000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f756406fd40 con 0x7f7564155a00 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.812+0000 7f7569000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f75640ccf20 con 0x7f756406f390 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.813+0000 7f7561e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7564155a00 0x7f756415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.813+0000 7f7561e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7564155a00 0x7f756415baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55146/0 (socket says 172.21.15.67:55146) 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.813+0000 7f7561e00640 1 -- 172.21.15.67:0/3497665956 learned_addr learned my addr 172.21.15.67:0/3497665956 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.813+0000 7f7561e00640 1 -- 172.21.15.67:0/3497665956 >> v2:172.21.15.169:3300/0 conn(0x7f756406f390 msgr2=0x7f756406f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.813+0000 7f7561e00640 1 --2- 172.21.15.67:0/3497665956 >> v2:172.21.15.169:3300/0 conn(0x7f756406f390 0x7f756406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.813+0000 7f7561e00640 1 -- 172.21.15.67:0/3497665956 >> v1:172.21.15.169:6789/0 conn(0x7f75640d0970 legacy=0x7f756406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:25.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.813+0000 7f7561e00640 1 -- 172.21.15.67:0/3497665956 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756415c0b0 con 0x7f7564155a00 2024-05-30T16:57:25.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.814+0000 7f7561e00640 1 --2- 172.21.15.67:0/3497665956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7564155a00 0x7f756415baa0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f75640cfee0 tx=0x7f755402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfc35407028cc1f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:25.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.815+0000 7f7560a00640 1 -- 172.21.15.67:0/3497665956 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554036070 con 0x7f7564155a00 2024-05-30T16:57:25.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.815+0000 7f7560a00640 1 -- 172.21.15.67:0/3497665956 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7554031600 con 0x7f7564155a00 2024-05-30T16:57:25.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.815+0000 7f7560a00640 1 -- 172.21.15.67:0/3497665956 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540319d0 con 0x7f7564155a00 2024-05-30T16:57:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.815+0000 7f7569000640 1 -- 172.21.15.67:0/3497665956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7564155a00 msgr2=0x7f756415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.815+0000 7f7569000640 1 --2- 172.21.15.67:0/3497665956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7564155a00 0x7f756415baa0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f75640cfee0 tx=0x7f755402a6f0 comp rx=0 tx=0).stop 2024-05-30T16:57:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.815+0000 7f7569000640 1 -- 172.21.15.67:0/3497665956 shutdown_connections 2024-05-30T16:57:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.815+0000 7f7569000640 1 --2- 172.21.15.67:0/3497665956 >> v2:172.21.15.169:3300/0 conn(0x7f756406f390 0x7f756406f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.815+0000 7f7569000640 1 --2- 172.21.15.67:0/3497665956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7564155a00 0x7f756415baa0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.815+0000 7f7569000640 1 -- 172.21.15.67:0/3497665956 >> 172.21.15.67:0/3497665956 conn(0x7f7564067ce0 msgr2=0x7f756406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.816+0000 7f7569000640 1 -- 172.21.15.67:0/3497665956 shutdown_connections 2024-05-30T16:57:25.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.816+0000 7f7569000640 1 -- 172.21.15.67:0/3497665956 wait complete. 2024-05-30T16:57:25.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.817+0000 7f7569000640 1 Processor -- start 2024-05-30T16:57:25.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.817+0000 7f7569000640 1 -- start start 2024-05-30T16:57:25.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.818+0000 7f7569000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f756406f390 0x7f7564168580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:25.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.818+0000 7f7569000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75640d0970 0x7f7564155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:25.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.818+0000 7f7569000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7564155a00 0x7f756416c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:25.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.818+0000 7f7569000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f756415c670 con 0x7f756406f390 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.818+0000 7f7569000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f756415c950 con 0x7f75640d0970 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.818+0000 7f7569000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f756415f090 con 0x7f7564155a00 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f756406f390 0x7f7564168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f756406f390 0x7f7564168580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55158/0 (socket says 172.21.15.67:55158) 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561e00640 1 -- 172.21.15.67:0/745526752 learned_addr learned my addr 172.21.15.67:0/745526752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561400640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75640d0970 0x7f7564155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561e00640 1 -- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7564155a00 msgr2=0x7f756416c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7562800640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7564155a00 0x7f756416c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561e00640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7564155a00 0x7f756416c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561e00640 1 -- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75640d0970 msgr2=0x7f7564155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561e00640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75640d0970 0x7f7564155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:25.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561e00640 1 -- 172.21.15.67:0/745526752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75641555a0 con 0x7f756406f390 2024-05-30T16:57:25.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7561400640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75640d0970 0x7f7564155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:25.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.819+0000 7f7562800640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7564155a00 0x7f756416c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:25.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.820+0000 7f7561e00640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f756406f390 0x7f7564168580 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f75540093e0 tx=0x7f7554004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:25.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.820+0000 7f755a000640 1 -- 172.21.15.67:0/745526752 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554041020 con 0x7f756406f390 2024-05-30T16:57:25.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.820+0000 7f755a000640 1 -- 172.21.15.67:0/745526752 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7554007800 con 0x7f756406f390 2024-05-30T16:57:25.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.820+0000 7f755a000640 1 -- 172.21.15.67:0/745526752 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554036040 con 0x7f756406f390 2024-05-30T16:57:25.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.820+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756416c5b0 con 0x7f756406f390 2024-05-30T16:57:25.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.820+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756416cb60 con 0x7f756406f390 2024-05-30T16:57:25.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.821+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75640689e0 con 0x7f756406f390 2024-05-30T16:57:25.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.822+0000 7f755a000640 1 -- 172.21.15.67:0/745526752 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f7554044020 con 0x7f756406f390 2024-05-30T16:57:25.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.823+0000 7f755a000640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f754406d9d0 0x7f754406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:25.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.823+0000 7f755a000640 1 -- 172.21.15.67:0/745526752 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f75540ab720 con 0x7f756406f390 2024-05-30T16:57:25.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.824+0000 7f7561400640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f754406d9d0 0x7f754406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:25.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.824+0000 7f7561400640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f754406d9d0 0x7f754406fe90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7550007fd0 tx=0x7f7550009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:25.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:25.828+0000 7f755a000640 1 -- 172.21.15.67:0/745526752 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f755404b030 con 0x7f756406f390 2024-05-30T16:57:26.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.107+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f756406f170 con 0x7f756406f390 2024-05-30T16:57:26.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.108+0000 7f755a000640 1 -- 172.21.15.67:0/745526752 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2381 (secure 0 0 0) 0x7f755407b5d0 con 0x7f756406f390 2024-05-30T16:57:26.109 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:57:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 839, 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:57:26.111 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:57:26.112 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 13242, 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 55327874317, 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 111369601024, 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 661724512256, 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 486131010, 2024-05-30T16:57:26.113 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 488273281, 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 119, 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 133 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:57:26.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:57:26.115 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "7c6f9121-d563-4406-bd90-584b3290d1b2": { 2024-05-30T16:57:26.116 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 26.0 seconds before exiting (77s)\n [============================] ", 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 26.0 seconds before exiting (83s)\n [============================] ", 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:26.117 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:57:26.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f754406d9d0 msgr2=0x7f754406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:26.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f754406d9d0 0x7f754406fe90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7550007fd0 tx=0x7f7550009040 comp rx=0 tx=0).stop 2024-05-30T16:57:26.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f756406f390 msgr2=0x7f7564168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:26.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f756406f390 0x7f7564168580 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f75540093e0 tx=0x7f7554004a40 comp rx=0 tx=0).stop 2024-05-30T16:57:26.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 shutdown_connections 2024-05-30T16:57:26.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f754406d9d0 0x7f754406fe90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:26.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7564155a00 0x7f756416c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:26.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75640d0970 0x7f7564155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:26.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 --2- 172.21.15.67:0/745526752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f756406f390 0x7f7564168580 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:26.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 >> 172.21.15.67:0/745526752 conn(0x7f7564067ce0 msgr2=0x7f75640cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:26.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.110+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 shutdown_connections 2024-05-30T16:57:26.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:26.111+0000 7f7569000640 1 -- 172.21.15.67:0/745526752 wait complete. 2024-05-30T16:57:31.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:57:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.356+0000 7f8e52400640 1 Processor -- start 2024-05-30T16:57:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.356+0000 7f8e52400640 1 -- start start 2024-05-30T16:57:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e52400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e4c0723d0 0x7f8e4c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e52400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8e4c0b41e0 0x7f8e4c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e52400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4c0cf0c0 con 0x7f8e4c145320 2024-05-30T16:57:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e52400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4c0b4a20 con 0x7f8e4c0723d0 2024-05-30T16:57:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e52400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4c0b4d00 con 0x7f8e4c0b41e0 2024-05-30T16:57:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e42a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e4c0723d0 0x7f8e4c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e4b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8e4c145320 0x7f8e4c071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57172/0 (socket says 172.21.15.67:57172) 2024-05-30T16:57:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e4b400640 1 -- 172.21.15.67:0/1328968254 learned_addr learned my addr 172.21.15.67:0/1328968254 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e4be00640 1 --2- 172.21.15.67:0/1328968254 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c0b41e0 0x7f8e4c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.358+0000 7f8e42a00640 1 -- 172.21.15.67:0/1328968254 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c0b41e0 msgr2=0x7f8e4c0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.359+0000 7f8e42a00640 1 --2- 172.21.15.67:0/1328968254 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c0b41e0 0x7f8e4c0cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.359+0000 7f8e42a00640 1 -- 172.21.15.67:0/1328968254 >> v1:172.21.15.67:6789/0 conn(0x7f8e4c145320 legacy=0x7f8e4c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.359+0000 7f8e4aa00640 1 -- 172.21.15.67:0/1328968254 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1417139687 0 0) 0x7f8e4c0cf0c0 con 0x7f8e4c145320 2024-05-30T16:57:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.359+0000 7f8e42a00640 1 -- 172.21.15.67:0/1328968254 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c0b4fe0 con 0x7f8e4c0723d0 2024-05-30T16:57:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.359+0000 7f8e4be00640 1 --2- 172.21.15.67:0/1328968254 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c0b41e0 0x7f8e4c0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.360+0000 7f8e42a00640 1 --2- 172.21.15.67:0/1328968254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c0723d0 0x7f8e4c0b3ca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0ce950 tx=0x7f8e3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277258776a5364ce server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.360+0000 7f8e4aa00640 1 -- 172.21.15.67:0/1328968254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34037070 con 0x7f8e4c0723d0 2024-05-30T16:57:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.360+0000 7f8e4aa00640 1 -- 172.21.15.67:0/1328968254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e34032440 con 0x7f8e4c0723d0 2024-05-30T16:57:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.360+0000 7f8e4aa00640 1 -- 172.21.15.67:0/1328968254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34032810 con 0x7f8e4c0723d0 2024-05-30T16:57:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.361+0000 7f8e52400640 1 -- 172.21.15.67:0/1328968254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c0723d0 msgr2=0x7f8e4c0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.361+0000 7f8e52400640 1 --2- 172.21.15.67:0/1328968254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c0723d0 0x7f8e4c0b3ca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0ce950 tx=0x7f8e3402a1a0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.361+0000 7f8e52400640 1 -- 172.21.15.67:0/1328968254 shutdown_connections 2024-05-30T16:57:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.361+0000 7f8e52400640 1 --2- 172.21.15.67:0/1328968254 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c0b41e0 0x7f8e4c0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.361+0000 7f8e52400640 1 --2- 172.21.15.67:0/1328968254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c0723d0 0x7f8e4c0b3ca0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.361+0000 7f8e52400640 1 -- 172.21.15.67:0/1328968254 >> 172.21.15.67:0/1328968254 conn(0x7f8e4c067ce0 msgr2=0x7f8e4c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.361+0000 7f8e52400640 1 -- 172.21.15.67:0/1328968254 shutdown_connections 2024-05-30T16:57:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.361+0000 7f8e52400640 1 -- 172.21.15.67:0/1328968254 wait complete. 2024-05-30T16:57:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.363+0000 7f8e52400640 1 Processor -- start 2024-05-30T16:57:31.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.363+0000 7f8e52400640 1 -- start start 2024-05-30T16:57:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.364+0000 7f8e52400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0723d0 0x7f8e4c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.364+0000 7f8e52400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c0b41e0 0x7f8e4c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:31.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.364+0000 7f8e52400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c145320 0x7f8e4c1f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.364+0000 7f8e52400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c0b4650 con 0x7f8e4c145320 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.364+0000 7f8e52400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c145700 con 0x7f8e4c0b41e0 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.364+0000 7f8e52400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4c0b47d0 con 0x7f8e4c0723d0 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.364+0000 7f8e4be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c145320 0x7f8e4c1f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.364+0000 7f8e4be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c145320 0x7f8e4c1f4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57410/0 (socket says 172.21.15.67:57410) 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.364+0000 7f8e4be00640 1 -- 172.21.15.67:0/369724197 learned_addr learned my addr 172.21.15.67:0/369724197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e4b400640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0723d0 0x7f8e4c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e42a00640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c0b41e0 0x7f8e4c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e4be00640 1 -- 172.21.15.67:0/369724197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0723d0 msgr2=0x7f8e4c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e4be00640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0723d0 0x7f8e4c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e4be00640 1 -- 172.21.15.67:0/369724197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c0b41e0 msgr2=0x7f8e4c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:31.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e4be00640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c0b41e0 0x7f8e4c15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e4be00640 1 -- 172.21.15.67:0/369724197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c15f790 con 0x7f8e4c145320 2024-05-30T16:57:31.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e42a00640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c0b41e0 0x7f8e4c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:31.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e4be00640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c145320 0x7f8e4c1f4260 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400c7f0 tx=0x7f8e4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e43e00640 1 -- 172.21.15.67:0/369724197 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44015070 con 0x7f8e4c145320 2024-05-30T16:57:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e43e00640 1 -- 172.21.15.67:0/369724197 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e440077c0 con 0x7f8e4c145320 2024-05-30T16:57:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e43e00640 1 -- 172.21.15.67:0/369724197 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44007bf0 con 0x7f8e4c145320 2024-05-30T16:57:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.365+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c15fa70 con 0x7f8e4c145320 2024-05-30T16:57:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.366+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c15da10 con 0x7f8e4c145320 2024-05-30T16:57:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.367+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c0d0d60 con 0x7f8e4c145320 2024-05-30T16:57:31.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.368+0000 7f8e43e00640 1 -- 172.21.15.67:0/369724197 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f8e44018070 con 0x7f8e4c145320 2024-05-30T16:57:31.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.369+0000 7f8e43e00640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e2806d9d0 0x7f8e2806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:31.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.369+0000 7f8e43e00640 1 -- 172.21.15.67:0/369724197 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f8e4408aba0 con 0x7f8e4c145320 2024-05-30T16:57:31.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.369+0000 7f8e4b400640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e2806d9d0 0x7f8e2806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:31.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.370+0000 7f8e4b400640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e2806d9d0 0x7f8e2806fe90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0012e0 tx=0x7f8e3c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:31.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.374+0000 7f8e43e00640 1 -- 172.21.15.67:0/369724197 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e4c0d0d60 con 0x7f8e4c145320 2024-05-30T16:57:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.656+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8e4c0b34e0 con 0x7f8e4c145320 2024-05-30T16:57:31.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.657+0000 7f8e43e00640 1 -- 172.21.15.67:0/369724197 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2379 (secure 0 0 0) 0x7f8e4405a6e0 con 0x7f8e4c145320 2024-05-30T16:57:31.657 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:57:31.658 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 845, 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:57:31.659 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:31.660 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 13239, 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 55327917464, 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 111415861248, 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 661678252032, 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 222211766, 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 224334272, 2024-05-30T16:57:31.661 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 54, 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 63 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:57:31.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:57:31.663 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "7c6f9121-d563-4406-bd90-584b3290d1b2": { 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 20.0 seconds before exiting (83s)\n [============================] ", 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:31.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:31.665 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:57:31.665 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 20.0 seconds before exiting (89s)\n [============================] ", 2024-05-30T16:57:31.665 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:31.665 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:31.665 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:31.665 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:31.665 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:57:31.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e2806d9d0 msgr2=0x7f8e2806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:31.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e2806d9d0 0x7f8e2806fe90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0012e0 tx=0x7f8e3c009040 comp rx=0 tx=0).stop 2024-05-30T16:57:31.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c145320 msgr2=0x7f8e4c1f4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:31.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c145320 0x7f8e4c1f4260 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400c7f0 tx=0x7f8e4400ccb0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 shutdown_connections 2024-05-30T16:57:31.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8e2806d9d0 0x7f8e2806fe90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e4c145320 0x7f8e4c1f4260 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c0b41e0 0x7f8e4c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 --2- 172.21.15.67:0/369724197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0723d0 0x7f8e4c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.659+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 >> 172.21.15.67:0/369724197 conn(0x7f8e4c067ce0 msgr2=0x7f8e4c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.660+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 shutdown_connections 2024-05-30T16:57:31.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:31.660+0000 7f8e52400640 1 -- 172.21.15.67:0/369724197 wait complete. 2024-05-30T16:57:36.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:57:36.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.899+0000 7f14e9600640 1 Processor -- start 2024-05-30T16:57:36.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.899+0000 7f14e9600640 1 -- start start 2024-05-30T16:57:36.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.901+0000 7f14e9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4145320 0x7f14e4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:36.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.901+0000 7f14e9600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14e40b43b0 0x7f14e40cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:36.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.901+0000 7f14e9600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14e40cf0c0 con 0x7f14e4072200 2024-05-30T16:57:36.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.901+0000 7f14e9600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f14e4072660 con 0x7f14e40b43b0 2024-05-30T16:57:36.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.901+0000 7f14e9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14e40b4a20 con 0x7f14e4145320 2024-05-30T16:57:36.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.902+0000 7f14da000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f14e4072200 0x7f14e40b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57178/0 (socket says 172.21.15.67:57178) 2024-05-30T16:57:36.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.902+0000 7f14da000640 1 -- 172.21.15.67:0/3231545773 learned_addr learned my addr 172.21.15.67:0/3231545773 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:36.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.902+0000 7f14e3400640 1 --2- 172.21.15.67:0/3231545773 >> v2:172.21.15.169:3300/0 conn(0x7f14e40b43b0 0x7f14e40cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:36.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.902+0000 7f14e2a00640 1 --2- 172.21.15.67:0/3231545773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4145320 0x7f14e4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:36.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.902+0000 7f14e2000640 1 -- 172.21.15.67:0/3231545773 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312349499 0 0) 0x7f14e40cf0c0 con 0x7f14e4072200 2024-05-30T16:57:36.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.902+0000 7f14e2000640 1 -- 172.21.15.67:0/3231545773 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14c8003520 con 0x7f14e4072200 2024-05-30T16:57:36.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.903+0000 7f14e3400640 1 -- 172.21.15.67:0/3231545773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4145320 msgr2=0x7f14e4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:36.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.903+0000 7f14e3400640 1 --2- 172.21.15.67:0/3231545773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4145320 0x7f14e4071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:36.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.903+0000 7f14e3400640 1 -- 172.21.15.67:0/3231545773 >> v1:172.21.15.67:6789/0 conn(0x7f14e4072200 legacy=0x7f14e40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:36.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.903+0000 7f14e2000640 1 -- 172.21.15.67:0/3231545773 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3933706636 0 0) 0x7f14c8003520 con 0x7f14e4072200 2024-05-30T16:57:36.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.903+0000 7f14e3400640 1 -- 172.21.15.67:0/3231545773 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e40b4d00 con 0x7f14e40b43b0 2024-05-30T16:57:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.904+0000 7f14e3400640 1 --2- 172.21.15.67:0/3231545773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14e40b43b0 0x7f14e40cf880 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0097a0 tx=0x7f14dc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30517d8aaaccde65 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.905+0000 7f14e2000640 1 -- 172.21.15.67:0/3231545773 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc037070 con 0x7f14e40b43b0 2024-05-30T16:57:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.905+0000 7f14e2000640 1 -- 172.21.15.67:0/3231545773 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14dc02aba0 con 0x7f14e40b43b0 2024-05-30T16:57:36.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.905+0000 7f14e2000640 1 -- 172.21.15.67:0/3231545773 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc001670 con 0x7f14e40b43b0 2024-05-30T16:57:36.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.906+0000 7f14e9600640 1 -- 172.21.15.67:0/3231545773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14e40b43b0 msgr2=0x7f14e40cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:36.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.906+0000 7f14e9600640 1 --2- 172.21.15.67:0/3231545773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14e40b43b0 0x7f14e40cf880 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0097a0 tx=0x7f14dc02a110 comp rx=0 tx=0).stop 2024-05-30T16:57:36.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.906+0000 7f14e9600640 1 -- 172.21.15.67:0/3231545773 shutdown_connections 2024-05-30T16:57:36.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.906+0000 7f14e9600640 1 --2- 172.21.15.67:0/3231545773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14e40b43b0 0x7f14e40cf880 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:36.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.906+0000 7f14e9600640 1 --2- 172.21.15.67:0/3231545773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4145320 0x7f14e4071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:36.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.906+0000 7f14e9600640 1 -- 172.21.15.67:0/3231545773 >> 172.21.15.67:0/3231545773 conn(0x7f14e4067ce0 msgr2=0x7f14e406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:36.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.906+0000 7f14e9600640 1 -- 172.21.15.67:0/3231545773 shutdown_connections 2024-05-30T16:57:36.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.906+0000 7f14e9600640 1 -- 172.21.15.67:0/3231545773 wait complete. 2024-05-30T16:57:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.908+0000 7f14e9600640 1 Processor -- start 2024-05-30T16:57:36.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.908+0000 7f14e9600640 1 -- start start 2024-05-30T16:57:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14e9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4072200 0x7f14e415f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14e9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14e40b43b0 0x7f14e41e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14e9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14e4145320 0x7f14e40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14e9600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4145700 con 0x7f14e40b43b0 2024-05-30T16:57:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14e9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14e40cf360 con 0x7f14e4145320 2024-05-30T16:57:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14e9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14e40cfdc0 con 0x7f14e4072200 2024-05-30T16:57:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14da000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14e40b43b0 0x7f14e41e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14da000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14e40b43b0 0x7f14e41e4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57426/0 (socket says 172.21.15.67:57426) 2024-05-30T16:57:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14da000640 1 -- 172.21.15.67:0/3712670106 learned_addr learned my addr 172.21.15.67:0/3712670106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:36.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14e2a00640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4072200 0x7f14e415f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:36.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14e3400640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14e4145320 0x7f14e40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:36.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14da000640 1 -- 172.21.15.67:0/3712670106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4072200 msgr2=0x7f14e415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:36.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14da000640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4072200 0x7f14e415f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:36.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14da000640 1 -- 172.21.15.67:0/3712670106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14e4145320 msgr2=0x7f14e40aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:36.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14da000640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14e4145320 0x7f14e40aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:36.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.909+0000 7f14da000640 1 -- 172.21.15.67:0/3712670106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e41e5b70 con 0x7f14e40b43b0 2024-05-30T16:57:36.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.910+0000 7f14da000640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14e40b43b0 0x7f14e41e4c70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0011e0 tx=0x7f14cc00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.910+0000 7f14db400640 1 -- 172.21.15.67:0/3712670106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc017070 con 0x7f14e40b43b0 2024-05-30T16:57:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.910+0000 7f14db400640 1 -- 172.21.15.67:0/3712670106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14cc005ad0 con 0x7f14e40b43b0 2024-05-30T16:57:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.910+0000 7f14db400640 1 -- 172.21.15.67:0/3712670106 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc007070 con 0x7f14e40b43b0 2024-05-30T16:57:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.910+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e415ebb0 con 0x7f14e40b43b0 2024-05-30T16:57:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.910+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e41e5e50 con 0x7f14e40b43b0 2024-05-30T16:57:36.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.912+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e40689e0 con 0x7f14e40b43b0 2024-05-30T16:57:36.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.913+0000 7f14db400640 1 -- 172.21.15.67:0/3712670106 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f14cc01a070 con 0x7f14e40b43b0 2024-05-30T16:57:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.913+0000 7f14db400640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f14c006d9d0 0x7f14c006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.914+0000 7f14db400640 1 -- 172.21.15.67:0/3712670106 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f14cc089fe0 con 0x7f14e40b43b0 2024-05-30T16:57:36.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.914+0000 7f14e2a00640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f14c006d9d0 0x7f14c006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:36.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.914+0000 7f14e2a00640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f14c006d9d0 0x7f14c006fe90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007380 tx=0x7f14d400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:36.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:36.918+0000 7f14db400640 1 -- 172.21.15.67:0/3712670106 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14cc059b20 con 0x7f14e40b43b0 2024-05-30T16:57:37.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.200+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f14e41e51b0 con 0x7f14e40b43b0 2024-05-30T16:57:37.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.201+0000 7f14db400640 1 -- 172.21.15.67:0/3712670106 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2366 (secure 0 0 0) 0x7f14cc059420 con 0x7f14e40b43b0 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:57:37.202 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 850, 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:57:37.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:37.204 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 13239, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 55327959968, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 111415918592, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 661678194688, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 426, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 8529, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:57:37.205 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 2 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:57:37.206 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:37.207 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "7c6f9121-d563-4406-bd90-584b3290d1b2": { 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 14.0 seconds before exiting (89s)\n [============================] ", 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:57:37.208 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 14.0 seconds before exiting (95s)\n [============================] ", 2024-05-30T16:57:37.209 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:37.209 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:37.209 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:37.209 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:37.209 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:57:37.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.203+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f14c006d9d0 msgr2=0x7f14c006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.203+0000 7f14e9600640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f14c006d9d0 0x7f14c006fe90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007380 tx=0x7f14d400b040 comp rx=0 tx=0).stop 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.203+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14e40b43b0 msgr2=0x7f14e41e4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.203+0000 7f14e9600640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14e40b43b0 0x7f14e41e4c70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0011e0 tx=0x7f14cc00cbe0 comp rx=0 tx=0).stop 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.204+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 shutdown_connections 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.204+0000 7f14e9600640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f14c006d9d0 0x7f14c006fe90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.204+0000 7f14e9600640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14e4145320 0x7f14e40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.204+0000 7f14e9600640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14e40b43b0 0x7f14e41e4c70 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.204+0000 7f14e9600640 1 --2- 172.21.15.67:0/3712670106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14e4072200 0x7f14e415f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.204+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 >> 172.21.15.67:0/3712670106 conn(0x7f14e4067ce0 msgr2=0x7f14e40d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.204+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 shutdown_connections 2024-05-30T16:57:37.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:37.204+0000 7f14e9600640 1 -- 172.21.15.67:0/3712670106 wait complete. 2024-05-30T16:57:42.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:57:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.448+0000 7f3315400640 1 Processor -- start 2024-05-30T16:57:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.448+0000 7f3315400640 1 -- start start 2024-05-30T16:57:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.450+0000 7f3315400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f331015ac80 0x7f331015f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.450+0000 7f3315400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33100ca870 0x7f33100ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.450+0000 7f3315400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33100b4be0 con 0x7f33100af720 2024-05-30T16:57:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.450+0000 7f3315400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f331015f630 con 0x7f331015ac80 2024-05-30T16:57:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.450+0000 7f3315400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33100cf8a0 con 0x7f33100ca870 2024-05-30T16:57:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.450+0000 7f3314a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f33100af720 0x7f33100b3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49790/0 (socket says 172.21.15.67:49790) 2024-05-30T16:57:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.450+0000 7f3314a00640 1 -- 172.21.15.67:0/2553584599 learned_addr learned my addr 172.21.15.67:0/2553584599 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330f400640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33100ca870 0x7f33100ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330fe00640 1 --2- 172.21.15.67:0/2553584599 >> v2:172.21.15.169:3300/0 conn(0x7f331015ac80 0x7f331015f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3779852004 0 0) 0x7f33100b4be0 con 0x7f33100af720 2024-05-30T16:57:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f4003520 con 0x7f33100af720 2024-05-30T16:57:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3240127573 0 0) 0x7f32f4003520 con 0x7f33100af720 2024-05-30T16:57:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33100ca870 msgr2=0x7f33100ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33100ca870 0x7f33100ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 >> v2:172.21.15.169:3300/0 conn(0x7f331015ac80 msgr2=0x7f331015f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 --2- 172.21.15.67:0/2553584599 >> v2:172.21.15.169:3300/0 conn(0x7f331015ac80 0x7f331015f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33100cde90 con 0x7f33100af720 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f3308001670 con 0x7f33100af720 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 >> v1:172.21.15.67:6789/0 conn(0x7f33100af720 legacy=0x7f33100b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33100ca870 0x7f32f40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f4003f80 0x7f32f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f32f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f331015f630 con 0x7f32f4004d40 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33100cf8a0 con 0x7f32f4003f80 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.451+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32f400ba70 con 0x7f33100ca870 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.452+0000 7f3314a00640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f32f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.452+0000 7f330fe00640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33100ca870 0x7f32f40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.452+0000 7f330f400640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f4003f80 0x7f32f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.452+0000 7f3314a00640 1 -- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33100ca870 msgr2=0x7f32f40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.452+0000 7f3314a00640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33100ca870 0x7f32f40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.452+0000 7f3314a00640 1 -- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f4003f80 msgr2=0x7f32f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.452+0000 7f3314a00640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f4003f80 0x7f32f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.452+0000 7f3314a00640 1 -- 172.21.15.67:0/2553584599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f4003520 con 0x7f32f4004d40 2024-05-30T16:57:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.452+0000 7f330f400640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f4003f80 0x7f32f4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.453+0000 7f3314a00640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f32f40051c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f330800c160 tx=0x7f32f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83c6cac0d172e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.453+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308007c80 con 0x7f32f4004d40 2024-05-30T16:57:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.453+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3308005ad0 con 0x7f32f4004d40 2024-05-30T16:57:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.453+0000 7f330ea00640 1 -- 172.21.15.67:0/2553584599 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308007070 con 0x7f32f4004d40 2024-05-30T16:57:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.454+0000 7f3315400640 1 -- 172.21.15.67:0/2553584599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 msgr2=0x7f32f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.454+0000 7f3315400640 1 --2- 172.21.15.67:0/2553584599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f32f40051c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f330800c160 tx=0x7f32f4005830 comp rx=0 tx=0).stop 2024-05-30T16:57:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.454+0000 7f3314a00640 1 -- 172.21.15.67:0/2553584599 reap_dead start 2024-05-30T16:57:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.454+0000 7f3315400640 1 -- 172.21.15.67:0/2553584599 shutdown_connections 2024-05-30T16:57:42.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.454+0000 7f3315400640 1 -- 172.21.15.67:0/2553584599 >> 172.21.15.67:0/2553584599 conn(0x7f331006bae0 msgr2=0x7f331006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:42.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.454+0000 7f3315400640 1 -- 172.21.15.67:0/2553584599 shutdown_connections 2024-05-30T16:57:42.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.454+0000 7f3315400640 1 -- 172.21.15.67:0/2553584599 wait complete. 2024-05-30T16:57:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.456+0000 7f3315400640 1 Processor -- start 2024-05-30T16:57:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.456+0000 7f3315400640 1 -- start start 2024-05-30T16:57:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.457+0000 7f3315400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f33101477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.457+0000 7f3315400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3310099750 0x7f33100aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.457+0000 7f3315400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3310098a90 0x7f3310098f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.457+0000 7f3315400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f33100cfe20 con 0x7f32f4004d40 2024-05-30T16:57:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.457+0000 7f3315400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33100cfb40 con 0x7f3310098a90 2024-05-30T16:57:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.457+0000 7f3315400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f33100d0460 con 0x7f3310099750 2024-05-30T16:57:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.457+0000 7f330fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f33101477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.457+0000 7f330fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f33101477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57286/0 (socket says 172.21.15.67:57286) 2024-05-30T16:57:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.457+0000 7f330fe00640 1 -- 172.21.15.67:0/2826052794 learned_addr learned my addr 172.21.15.67:0/2826052794 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.458+0000 7f330f400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3310099750 0x7f33100aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.458+0000 7f3314a00640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3310098a90 0x7f3310098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.458+0000 7f330fe00640 1 -- 172.21.15.67:0/2826052794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3310099750 msgr2=0x7f33100aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.458+0000 7f330fe00640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3310099750 0x7f33100aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.458+0000 7f330fe00640 1 -- 172.21.15.67:0/2826052794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3310098a90 msgr2=0x7f3310098f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.458+0000 7f330fe00640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3310098a90 0x7f3310098f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.458+0000 7f330fe00640 1 -- 172.21.15.67:0/2826052794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3310099480 con 0x7f32f4004d40 2024-05-30T16:57:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.458+0000 7f330f400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3310099750 0x7f33100aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.458+0000 7f330fe00640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f33101477f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f32f80116d0 tx=0x7f32f8011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.459+0000 7f3307e00640 1 -- 172.21.15.67:0/2826052794 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f800f9c0 con 0x7f32f4004d40 2024-05-30T16:57:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.459+0000 7f3307e00640 1 -- 172.21.15.67:0/2826052794 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f800fb60 con 0x7f32f4004d40 2024-05-30T16:57:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.459+0000 7f3307e00640 1 -- 172.21.15.67:0/2826052794 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8015460 con 0x7f32f4004d40 2024-05-30T16:57:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.459+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33100b2c00 con 0x7f32f4004d40 2024-05-30T16:57:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.459+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33100b2640 con 0x7f32f4004d40 2024-05-30T16:57:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.460+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33100689e0 con 0x7f32f4004d40 2024-05-30T16:57:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.461+0000 7f3307e00640 1 -- 172.21.15.67:0/2826052794 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f32f8015600 con 0x7f32f4004d40 2024-05-30T16:57:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.462+0000 7f3307e00640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32ec06da00 0x7f32ec06fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.462+0000 7f3307e00640 1 -- 172.21.15.67:0/2826052794 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f32f808b000 con 0x7f32f4004d40 2024-05-30T16:57:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.462+0000 7f330f400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32ec06da00 0x7f32ec06fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.463+0000 7f330f400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32ec06da00 0x7f32ec06fec0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3310147df0 tx=0x7f330000d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:42.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.467+0000 7f3307e00640 1 -- 172.21.15.67:0/2826052794 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32f8019030 con 0x7f32f4004d40 2024-05-30T16:57:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.745+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f33100b4790 con 0x7f32f4004d40 2024-05-30T16:57:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.746+0000 7f3307e00640 1 -- 172.21.15.67:0/2826052794 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2335 (secure 0 0 0) 0x7f32f805a4f0 con 0x7f32f4004d40 2024-05-30T16:57:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:42.746 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:57:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:57:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 856, 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:57:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 13239, 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 55327959968, 2024-05-30T16:57:42.749 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 111415918592, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 661678194688, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 3838, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:57:42.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:57:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "7c6f9121-d563-4406-bd90-584b3290d1b2": { 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 10.0 seconds before exiting (93s)\n [============================] ", 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:42.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:42.753 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:57:42.753 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 10.0 seconds before exiting (99s)\n [============================] ", 2024-05-30T16:57:42.753 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:42.753 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:42.753 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:42.753 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:42.753 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32ec06da00 msgr2=0x7f32ec06fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32ec06da00 0x7f32ec06fec0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3310147df0 tx=0x7f330000d040 comp rx=0 tx=0).stop 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 msgr2=0x7f33101477f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f33101477f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f32f80116d0 tx=0x7f32f8011b90 comp rx=0 tx=0).stop 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 shutdown_connections 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32ec06da00 0x7f32ec06fec0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3310098a90 0x7f3310098f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3310099750 0x7f33100aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 --2- 172.21.15.67:0/2826052794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32f4004d40 0x7f33101477f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.748+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 >> 172.21.15.67:0/2826052794 conn(0x7f331006bae0 msgr2=0x7f33100cbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:42.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.749+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 shutdown_connections 2024-05-30T16:57:42.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:42.749+0000 7f3315400640 1 -- 172.21.15.67:0/2826052794 wait complete. 2024-05-30T16:57:43.509 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:57:43.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:57:47.767 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:57:47.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.987+0000 7fe093e00640 1 Processor -- start 2024-05-30T16:57:47.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.987+0000 7fe093e00640 1 -- start start 2024-05-30T16:57:47.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.989+0000 7fe093e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe09415e690 0x7fe0940d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:47.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.989+0000 7fe093e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0940d1ca0 0x7fe0940d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:47.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.989+0000 7fe093e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe09415ea70 0x7fe0940d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:47.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.989+0000 7fe093e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe0940d3060 con 0x7fe09415e690 2024-05-30T16:57:47.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.989+0000 7fe093e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe0940d3340 con 0x7fe09415ea70 2024-05-30T16:57:47.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.989+0000 7fe093e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe0940d3620 con 0x7fe0940d1ca0 2024-05-30T16:57:47.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.989+0000 7fe092a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe09415e690 0x7fe0940d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:47.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.989+0000 7fe092a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe09415e690 0x7fe0940d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57302/0 (socket says 172.21.15.67:57302) 2024-05-30T16:57:47.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.989+0000 7fe092a00640 1 -- 172.21.15.67:0/1431372206 learned_addr learned my addr 172.21.15.67:0/1431372206 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:47.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.990+0000 7fe092a00640 1 -- 172.21.15.67:0/1431372206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0940d1ca0 msgr2=0x7fe0940d4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:47.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.990+0000 7fe092a00640 1 --2- 172.21.15.67:0/1431372206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0940d1ca0 0x7fe0940d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:47.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.990+0000 7fe092a00640 1 -- 172.21.15.67:0/1431372206 >> v2:172.21.15.169:3300/0 conn(0x7fe09415ea70 msgr2=0x7fe0940d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:47.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.990+0000 7fe092a00640 1 --2- 172.21.15.67:0/1431372206 >> v2:172.21.15.169:3300/0 conn(0x7fe09415ea70 0x7fe0940d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:47.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.990+0000 7fe092a00640 1 -- 172.21.15.67:0/1431372206 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0940cf770 con 0x7fe09415e690 2024-05-30T16:57:47.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.991+0000 7fe092a00640 1 --2- 172.21.15.67:0/1431372206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe09415e690 0x7fe0940d4530 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe0940d55a0 tx=0x7fe08402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81106028aff4e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:47.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.991+0000 7fe091600640 1 -- 172.21.15.67:0/1431372206 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084037070 con 0x7fe09415e690 2024-05-30T16:57:47.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.991+0000 7fe091600640 1 -- 172.21.15.67:0/1431372206 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe084032440 con 0x7fe09415e690 2024-05-30T16:57:47.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.991+0000 7fe091600640 1 -- 172.21.15.67:0/1431372206 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084032810 con 0x7fe09415e690 2024-05-30T16:57:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.992+0000 7fe093e00640 1 -- 172.21.15.67:0/1431372206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe09415e690 msgr2=0x7fe0940d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.992+0000 7fe093e00640 1 --2- 172.21.15.67:0/1431372206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe09415e690 0x7fe0940d4530 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe0940d55a0 tx=0x7fe08402a1d0 comp rx=0 tx=0).stop 2024-05-30T16:57:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.992+0000 7fe093e00640 1 -- 172.21.15.67:0/1431372206 shutdown_connections 2024-05-30T16:57:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.992+0000 7fe093e00640 1 --2- 172.21.15.67:0/1431372206 >> v2:172.21.15.169:3300/0 conn(0x7fe09415ea70 0x7fe0940d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.992+0000 7fe093e00640 1 --2- 172.21.15.67:0/1431372206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0940d1ca0 0x7fe0940d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.992+0000 7fe093e00640 1 --2- 172.21.15.67:0/1431372206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe09415e690 0x7fe0940d4530 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.992+0000 7fe093e00640 1 -- 172.21.15.67:0/1431372206 >> 172.21.15.67:0/1431372206 conn(0x7fe09409d910 msgr2=0x7fe09408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.992+0000 7fe093e00640 1 -- 172.21.15.67:0/1431372206 shutdown_connections 2024-05-30T16:57:47.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.992+0000 7fe093e00640 1 -- 172.21.15.67:0/1431372206 wait complete. 2024-05-30T16:57:47.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.994+0000 7fe093e00640 1 Processor -- start 2024-05-30T16:57:47.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.994+0000 7fe093e00640 1 -- start start 2024-05-30T16:57:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.995+0000 7fe093e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0940d1ca0 0x7fe09415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.995+0000 7fe093e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe09415e690 0x7fe09406d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.995+0000 7fe093e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe09415ea70 0x7fe09406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.995+0000 7fe093e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe0940d29f0 con 0x7fe0940d1ca0 2024-05-30T16:57:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.995+0000 7fe093e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0940d2870 con 0x7fe09415ea70 2024-05-30T16:57:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.995+0000 7fe093e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe0940d2b70 con 0x7fe09415e690 2024-05-30T16:57:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.995+0000 7fe092a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0940d1ca0 0x7fe09415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.995+0000 7fe092a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0940d1ca0 0x7fe09415a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57312/0 (socket says 172.21.15.67:57312) 2024-05-30T16:57:47.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.995+0000 7fe092a00640 1 -- 172.21.15.67:0/681119460 learned_addr learned my addr 172.21.15.67:0/681119460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:47.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe092000640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe09415e690 0x7fe09406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:47.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe092a00640 1 -- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe09415e690 msgr2=0x7fe09406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe093400640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe09415ea70 0x7fe09406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe092a00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe09415e690 0x7fe09406d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe092a00640 1 -- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe09415ea70 msgr2=0x7fe09406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe092a00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe09415ea70 0x7fe09406d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe092a00640 1 -- 172.21.15.67:0/681119460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0940b9c20 con 0x7fe0940d1ca0 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe092000640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe09415e690 0x7fe09406d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe093400640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe09415ea70 0x7fe09406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe092a00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0940d1ca0 0x7fe09415a0e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fe084000c00 tx=0x7fe0840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.996+0000 7fe08aa00640 1 -- 172.21.15.67:0/681119460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084041070 con 0x7fe0940d1ca0 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.997+0000 7fe08aa00640 1 -- 172.21.15.67:0/681119460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe08403c870 con 0x7fe0940d1ca0 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.997+0000 7fe08aa00640 1 -- 172.21.15.67:0/681119460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084037040 con 0x7fe0940d1ca0 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.997+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0940cd320 con 0x7fe0940d1ca0 2024-05-30T16:57:47.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.997+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0940cd8e0 con 0x7fe0940d1ca0 2024-05-30T16:57:47.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.998+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0940689e0 con 0x7fe0940d1ca0 2024-05-30T16:57:47.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:47.999+0000 7fe08aa00640 1 -- 172.21.15.67:0/681119460 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7fe084044070 con 0x7fe0940d1ca0 2024-05-30T16:57:48.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.000+0000 7fe08aa00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe07406d9d0 0x7fe07406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:48.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.000+0000 7fe08aa00640 1 -- 172.21.15.67:0/681119460 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fe0840ab600 con 0x7fe0940d1ca0 2024-05-30T16:57:48.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.000+0000 7fe092000640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe07406d9d0 0x7fe07406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:48.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.001+0000 7fe092000640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe07406d9d0 0x7fe07406fe90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe0940b9910 tx=0x7fe080008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:48.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.005+0000 7fe08aa00640 1 -- 172.21.15.67:0/681119460 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe08407b140 con 0x7fe0940d1ca0 2024-05-30T16:57:48.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.289+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe09406cbc0 con 0x7fe0940d1ca0 2024-05-30T16:57:48.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.290+0000 7fe08aa00640 1 -- 172.21.15.67:0/681119460 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2334 (secure 0 0 0) 0x7fe08407aa40 con 0x7fe0940d1ca0 2024-05-30T16:57:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:48.291 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:57:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:57:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:57:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:57:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:57:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 861, 2024-05-30T16:57:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 13239, 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 55327962574, 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 111415918592, 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 661678194688, 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:57:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 1364, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:57:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:57:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "7c6f9121-d563-4406-bd90-584b3290d1b2": { 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "No pending clones left, waiting 4.0 seconds before exiting (99s)\n [============================] ", 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "e0a834ca-3f6a-4236-843d-35f9b733ad76": { 2024-05-30T16:57:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 4.0 seconds before exiting (105s)\n [============================] ", 2024-05-30T16:57:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:57:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:57:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:48.298 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:57:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe07406d9d0 msgr2=0x7fe07406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe07406d9d0 0x7fe07406fe90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe0940b9910 tx=0x7fe080008040 comp rx=0 tx=0).stop 2024-05-30T16:57:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0940d1ca0 msgr2=0x7fe09415a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0940d1ca0 0x7fe09415a0e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fe084000c00 tx=0x7fe0840046c0 comp rx=0 tx=0).stop 2024-05-30T16:57:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 shutdown_connections 2024-05-30T16:57:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe07406d9d0 0x7fe07406fe90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe09415ea70 0x7fe09406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe09415e690 0x7fe09406d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:48.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 --2- 172.21.15.67:0/681119460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0940d1ca0 0x7fe09415a0e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:48.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 >> 172.21.15.67:0/681119460 conn(0x7fe09409d910 msgr2=0x7fe0940b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:48.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 shutdown_connections 2024-05-30T16:57:48.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:48.293+0000 7fe093e00640 1 -- 172.21.15.67:0/681119460 wait complete. 2024-05-30T16:57:53.314 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:57:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.532+0000 7fb7b2600640 1 Processor -- start 2024-05-30T16:57:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.533+0000 7fb7b2600640 1 -- start start 2024-05-30T16:57:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.534+0000 7fb7b2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac0d5150 0x7fb7ac0d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.534+0000 7fb7b2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb7ac0d3490 0x7fb7ac0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.534+0000 7fb7b2600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7ac0d2200 con 0x7fb7ac15fc70 2024-05-30T16:57:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.534+0000 7fb7b2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb7ac160050 con 0x7fb7ac0d3490 2024-05-30T16:57:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.534+0000 7fb7b2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ac0d2ac0 con 0x7fb7ac0d5150 2024-05-30T16:57:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.534+0000 7fb7b1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb7ac0d3490 0x7fb7ac0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.534+0000 7fb7b1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb7ac0d3490 0x7fb7ac0d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46970/0 (socket says 172.21.15.67:46970) 2024-05-30T16:57:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.534+0000 7fb7b1c00640 1 -- 172.21.15.67:0/2321512764 learned_addr learned my addr 172.21.15.67:0/2321512764 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.535+0000 7fb7b1c00640 1 -- 172.21.15.67:0/2321512764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac0d5150 msgr2=0x7fb7ac0d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.535+0000 7fb7abe00640 1 --2- 172.21.15.67:0/2321512764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac0d5150 0x7fb7ac0d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.535+0000 7fb7b1c00640 1 --2- 172.21.15.67:0/2321512764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac0d5150 0x7fb7ac0d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.535+0000 7fb7b1c00640 1 -- 172.21.15.67:0/2321512764 >> v1:172.21.15.169:6789/0 conn(0x7fb7ac15fc70 legacy=0x7fb7ac0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.535+0000 7fb7b1c00640 1 -- 172.21.15.67:0/2321512764 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac0d2470 con 0x7fb7ac0d3490 2024-05-30T16:57:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.535+0000 7fb7abe00640 1 --2- 172.21.15.67:0/2321512764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac0d5150 0x7fb7ac0d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.536+0000 7fb7b1c00640 1 --2- 172.21.15.67:0/2321512764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 0x7fb7ac0d0f10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009870 tx=0x7fb7a402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db0eaaf1d3c0557 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.536+0000 7fb7ab400640 1 -- 172.21.15.67:0/2321512764 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4037070 con 0x7fb7ac0d3490 2024-05-30T16:57:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.536+0000 7fb7ab400640 1 -- 172.21.15.67:0/2321512764 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a4032440 con 0x7fb7ac0d3490 2024-05-30T16:57:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.536+0000 7fb7ab400640 1 -- 172.21.15.67:0/2321512764 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4032810 con 0x7fb7ac0d3490 2024-05-30T16:57:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.537+0000 7fb7b2600640 1 -- 172.21.15.67:0/2321512764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 msgr2=0x7fb7ac0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.537+0000 7fb7b2600640 1 --2- 172.21.15.67:0/2321512764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 0x7fb7ac0d0f10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009870 tx=0x7fb7a402a210 comp rx=0 tx=0).stop 2024-05-30T16:57:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.537+0000 7fb7b2600640 1 -- 172.21.15.67:0/2321512764 shutdown_connections 2024-05-30T16:57:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.537+0000 7fb7b2600640 1 --2- 172.21.15.67:0/2321512764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 0x7fb7ac0d0f10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.537+0000 7fb7b2600640 1 --2- 172.21.15.67:0/2321512764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac0d5150 0x7fb7ac0d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.537+0000 7fb7b2600640 1 -- 172.21.15.67:0/2321512764 >> 172.21.15.67:0/2321512764 conn(0x7fb7ac08bfc0 msgr2=0x7fb7ac0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.537+0000 7fb7b2600640 1 -- 172.21.15.67:0/2321512764 shutdown_connections 2024-05-30T16:57:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.537+0000 7fb7b2600640 1 -- 172.21.15.67:0/2321512764 wait complete. 2024-05-30T16:57:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.539+0000 7fb7b2600640 1 Processor -- start 2024-05-30T16:57:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.539+0000 7fb7b2600640 1 -- start start 2024-05-30T16:57:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.540+0000 7fb7b2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 0x7fb7ac15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.540+0000 7fb7b2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7ac0d5150 0x7fb7ac14da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.540+0000 7fb7b2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac15fc70 0x7fb7ac09e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.540+0000 7fb7b2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac0d1450 con 0x7fb7ac0d3490 2024-05-30T16:57:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.540+0000 7fb7b2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac0d0570 con 0x7fb7ac0d5150 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.540+0000 7fb7b2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ac0d15d0 con 0x7fb7ac15fc70 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.540+0000 7fb7b1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 0x7fb7ac15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.540+0000 7fb7b1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 0x7fb7ac15b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46982/0 (socket says 172.21.15.67:46982) 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.540+0000 7fb7b1200640 1 -- 172.21.15.67:0/2781136199 learned_addr learned my addr 172.21.15.67:0/2781136199 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7b1200640 1 -- 172.21.15.67:0/2781136199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac15fc70 msgr2=0x7fb7ac09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7b1200640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac15fc70 0x7fb7ac09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7b1200640 1 -- 172.21.15.67:0/2781136199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7ac0d5150 msgr2=0x7fb7ac14da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7b1200640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7ac0d5150 0x7fb7ac14da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7b1200640 1 -- 172.21.15.67:0/2781136199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac161b70 con 0x7fb7ac0d3490 2024-05-30T16:57:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7b1200640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 0x7fb7ac15b0a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b3e0 tx=0x7fb79400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7a8c00640 1 -- 172.21.15.67:0/2781136199 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794015020 con 0x7fb7ac0d3490 2024-05-30T16:57:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7a8c00640 1 -- 172.21.15.67:0/2781136199 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb794011850 con 0x7fb7ac0d3490 2024-05-30T16:57:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ac0a0370 con 0x7fb7ac0d3490 2024-05-30T16:57:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.541+0000 7fb7a8c00640 1 -- 172.21.15.67:0/2781136199 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794011c80 con 0x7fb7ac0d3490 2024-05-30T16:57:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.542+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ac0a0930 con 0x7fb7ac0d3490 2024-05-30T16:57:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.543+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ac0694c0 con 0x7fb7ac0d3490 2024-05-30T16:57:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.544+0000 7fb7a8c00640 1 -- 172.21.15.67:0/2781136199 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7fb794018020 con 0x7fb7ac0d3490 2024-05-30T16:57:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.545+0000 7fb7a8c00640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb78806da00 0x7fb78806fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.545+0000 7fb7a8c00640 1 -- 172.21.15.67:0/2781136199 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fb79408a630 con 0x7fb7ac0d3490 2024-05-30T16:57:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.545+0000 7fb7abe00640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb78806da00 0x7fb78806fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:53.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.546+0000 7fb7abe00640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb78806da00 0x7fb78806fec0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac15ab70 tx=0x7fb79c008420 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.550+0000 7fb7a8c00640 1 -- 172.21.15.67:0/2781136199 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7ac0694c0 con 0x7fb7ac0d3490 2024-05-30T16:57:53.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.830+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb7ac0689e0 con 0x7fb7ac0d3490 2024-05-30T16:57:53.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.831+0000 7fb7a8c00640 1 -- 172.21.15.67:0/2781136199 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1826 (secure 0 0 0) 0x7fb79401f020 con 0x7fb7ac0d3490 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:57:53.832 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 867, 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:57:53.833 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 76, 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:53.834 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 13239, 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 55327962574, 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 111415918592, 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 661678194688, 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 255, 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T16:57:53.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 46, 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5, 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 5, 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6558 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:53.836 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 13, 2024-05-30T16:57:53.837 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:56:04.302846+0000", 2024-05-30T16:57:53.838 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:57:53.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:53.838 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": {} 2024-05-30T16:57:53.838 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:57:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb78806da00 msgr2=0x7fb78806fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb78806da00 0x7fb78806fec0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac15ab70 tx=0x7fb79c008420 comp rx=0 tx=0).stop 2024-05-30T16:57:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 msgr2=0x7fb7ac15b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 0x7fb7ac15b0a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b3e0 tx=0x7fb79400b8a0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 shutdown_connections 2024-05-30T16:57:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb78806da00 0x7fb78806fec0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7ac15fc70 0x7fb7ac09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7ac0d5150 0x7fb7ac14da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 --2- 172.21.15.67:0/2781136199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7ac0d3490 0x7fb7ac15b0a0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:53.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 >> 172.21.15.67:0/2781136199 conn(0x7fb7ac08bfc0 msgr2=0x7fb7ac0b8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:53.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 shutdown_connections 2024-05-30T16:57:53.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:53.834+0000 7fb7b2600640 1 -- 172.21.15.67:0/2781136199 wait complete. 2024-05-30T16:57:53.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-30T16:57:54.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.064+0000 7f0e4bc00640 1 Processor -- start 2024-05-30T16:57:54.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.064+0000 7f0e4bc00640 1 -- start start 2024-05-30T16:57:54.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.066+0000 7f0e4bc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0e440d0640 0x7f0e4415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.066+0000 7f0e4bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e440b32c0 0x7f0e440b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.066+0000 7f0e4bc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e440d0060 con 0x7f0e440d0a20 2024-05-30T16:57:54.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.066+0000 7f0e4bc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0e440adbb0 con 0x7f0e440d0640 2024-05-30T16:57:54.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.066+0000 7f0e4bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4415db10 con 0x7f0e440b32c0 2024-05-30T16:57:54.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.066+0000 7f0e43e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0e440d0a20 0x7f0e440cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33664/0 (socket says 172.21.15.67:33664) 2024-05-30T16:57:54.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.066+0000 7f0e43e00640 1 -- 172.21.15.67:0/1599899776 learned_addr learned my addr 172.21.15.67:0/1599899776 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:54.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.066+0000 7f0e49c00640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e440b32c0 0x7f0e440b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.066+0000 7f0e49200640 1 --2- 172.21.15.67:0/1599899776 >> v2:172.21.15.169:3300/0 conn(0x7f0e440d0640 0x7f0e4415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1868329681 0 0) 0x7f0e440d0060 con 0x7f0e440d0a20 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e28003520 con 0x7f0e440d0a20 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 743548786 0 0) 0x7f0e28003520 con 0x7f0e440d0a20 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e440b32c0 msgr2=0x7f0e440b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e440b32c0 0x7f0e440b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 >> v2:172.21.15.169:3300/0 conn(0x7f0e440d0640 msgr2=0x7f0e4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 --2- 172.21.15.67:0/1599899776 >> v2:172.21.15.169:3300/0 conn(0x7f0e440d0640 0x7f0e4415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4415cb10 con 0x7f0e440d0a20 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f0e340019f0 con 0x7f0e440d0a20 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 >> v1:172.21.15.67:6789/0 conn(0x7f0e440d0a20 legacy=0x7f0e440cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e440b32c0 0x7f0e280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e28003f30 0x7f0e28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e28004d40 0x7f0e280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4415db10 con 0x7f0e28003f30 2024-05-30T16:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2800b9c0 con 0x7f0e28004d40 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.067+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2800bb40 con 0x7f0e440b32c0 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.068+0000 7f0e43e00640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e28003f30 0x7f0e28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.068+0000 7f0e49200640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e440b32c0 0x7f0e280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.068+0000 7f0e49c00640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e28004d40 0x7f0e280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.068+0000 7f0e43e00640 1 -- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e440b32c0 msgr2=0x7f0e280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.068+0000 7f0e43e00640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e440b32c0 0x7f0e280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.068+0000 7f0e43e00640 1 -- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e28004d40 msgr2=0x7f0e280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.068+0000 7f0e43e00640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e28004d40 0x7f0e280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.068+0000 7f0e43e00640 1 -- 172.21.15.67:0/1599899776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e28003520 con 0x7f0e28003f30 2024-05-30T16:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.068+0000 7f0e49c00640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e28004d40 0x7f0e280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:54.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.069+0000 7f0e43e00640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e28003f30 0x7f0e28004800 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400a920 tx=0x7f0e28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a7e4b7a09560d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:54.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.069+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400a4c0 con 0x7f0e28003f30 2024-05-30T16:57:54.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.069+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3402f070 con 0x7f0e28003f30 2024-05-30T16:57:54.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.069+0000 7f0e43400640 1 -- 172.21.15.67:0/1599899776 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3402bd40 con 0x7f0e28003f30 2024-05-30T16:57:54.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.070+0000 7f0e4bc00640 1 -- 172.21.15.67:0/1599899776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e28003f30 msgr2=0x7f0e28004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.070+0000 7f0e4bc00640 1 --2- 172.21.15.67:0/1599899776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e28003f30 0x7f0e28004800 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400a920 tx=0x7f0e28005830 comp rx=0 tx=0).stop 2024-05-30T16:57:54.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.070+0000 7f0e49c00640 1 -- 172.21.15.67:0/1599899776 reap_dead start 2024-05-30T16:57:54.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.070+0000 7f0e4bc00640 1 -- 172.21.15.67:0/1599899776 shutdown_connections 2024-05-30T16:57:54.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.070+0000 7f0e4bc00640 1 -- 172.21.15.67:0/1599899776 >> 172.21.15.67:0/1599899776 conn(0x7f0e4406bd20 msgr2=0x7f0e4406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:54.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.070+0000 7f0e4bc00640 1 -- 172.21.15.67:0/1599899776 shutdown_connections 2024-05-30T16:57:54.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.070+0000 7f0e4bc00640 1 -- 172.21.15.67:0/1599899776 wait complete. 2024-05-30T16:57:54.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.072+0000 7f0e4bc00640 1 Processor -- start 2024-05-30T16:57:54.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.072+0000 7f0e4bc00640 1 -- start start 2024-05-30T16:57:54.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.073+0000 7f0e4bc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e28003f30 0x7f0e440cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.073+0000 7f0e4bc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e4409dc50 0x7f0e441f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.073+0000 7f0e4bc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4409afa0 0x7f0e4409bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.073+0000 7f0e4bc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4415cdd0 con 0x7f0e4409dc50 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.073+0000 7f0e4bc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e440ad8d0 con 0x7f0e4409afa0 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.073+0000 7f0e4bc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4415d0b0 con 0x7f0e28003f30 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e4409dc50 0x7f0e441f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e4409dc50 0x7f0e441f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46992/0 (socket says 172.21.15.67:46992) 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e43e00640 1 -- 172.21.15.67:0/2508665214 learned_addr learned my addr 172.21.15.67:0/2508665214 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e49200640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e28003f30 0x7f0e440cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e49c00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4409afa0 0x7f0e4409bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e43e00640 1 -- 172.21.15.67:0/2508665214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e28003f30 msgr2=0x7f0e440cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e43e00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e28003f30 0x7f0e440cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e43e00640 1 -- 172.21.15.67:0/2508665214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4409afa0 msgr2=0x7f0e4409bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e43e00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4409afa0 0x7f0e4409bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e43e00640 1 -- 172.21.15.67:0/2508665214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4409e3f0 con 0x7f0e4409dc50 2024-05-30T16:57:54.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e49c00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4409afa0 0x7f0e4409bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.074+0000 7f0e43e00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e4409dc50 0x7f0e441f86e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400aa40 tx=0x7f0e440ae570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.075+0000 7f0e40c00640 1 -- 172.21.15.67:0/2508665214 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34044070 con 0x7f0e4409dc50 2024-05-30T16:57:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.075+0000 7f0e40c00640 1 -- 172.21.15.67:0/2508665214 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3402bcb0 con 0x7f0e4409dc50 2024-05-30T16:57:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.075+0000 7f0e40c00640 1 -- 172.21.15.67:0/2508665214 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3402be60 con 0x7f0e4409dc50 2024-05-30T16:57:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.075+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e441f8c20 con 0x7f0e4409dc50 2024-05-30T16:57:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.075+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4409a4c0 con 0x7f0e4409dc50 2024-05-30T16:57:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.076+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e440689e0 con 0x7f0e4409dc50 2024-05-30T16:57:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.077+0000 7f0e40c00640 1 -- 172.21.15.67:0/2508665214 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f0e34047070 con 0x7f0e4409dc50 2024-05-30T16:57:54.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.079+0000 7f0e40c00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e2006d9d0 0x7f0e2006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.079+0000 7f0e40c00640 1 -- 172.21.15.67:0/2508665214 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f0e340aaa50 con 0x7f0e4409dc50 2024-05-30T16:57:54.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.079+0000 7f0e49200640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e2006d9d0 0x7f0e2006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.080+0000 7f0e49200640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e2006d9d0 0x7f0e2006fe90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009a90 tx=0x7f0e2c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:54.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.084+0000 7f0e40c00640 1 -- 172.21.15.67:0/2508665214 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e34079f40 con 0x7f0e4409dc50 2024-05-30T16:57:54.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.322+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e44073560 con 0x7f0e2006d9d0 2024-05-30T16:57:54.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.359+0000 7f0e40c00640 1 -- 172.21.15.67:0/2508665214 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1029 (secure 0 0 0) 0x7f0e44073560 con 0x7f0e2006d9d0 2024-05-30T16:57:54.360 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:54.360 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:57:54.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone92282" 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone92283" 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922816" 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sv1" 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone92286" 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922813" 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922819" 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone92284" 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922820" 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922814" 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922817" 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone92285" 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922812" 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922815" 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922811" 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone92288" 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone92289" 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922818" 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.365 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone92281" 2024-05-30T16:57:54.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.365 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone92287" 2024-05-30T16:57:54.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:57:54.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:54.365 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone922810" 2024-05-30T16:57:54.365 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:54.365 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:57:54.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e2006d9d0 msgr2=0x7f0e2006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e2006d9d0 0x7f0e2006fe90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009a90 tx=0x7f0e2c008040 comp rx=0 tx=0).stop 2024-05-30T16:57:54.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e4409dc50 msgr2=0x7f0e441f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e4409dc50 0x7f0e441f86e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400aa40 tx=0x7f0e440ae570 comp rx=0 tx=0).stop 2024-05-30T16:57:54.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 shutdown_connections 2024-05-30T16:57:54.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e2006d9d0 0x7f0e2006fe90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4409afa0 0x7f0e4409bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e4409dc50 0x7f0e441f86e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 --2- 172.21.15.67:0/2508665214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e28003f30 0x7f0e440cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 >> 172.21.15.67:0/2508665214 conn(0x7f0e4406bd20 msgr2=0x7f0e4415e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:54.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 shutdown_connections 2024-05-30T16:57:54.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.362+0000 7f0e4bc00640 1 -- 172.21.15.67:0/2508665214 wait complete. 2024-05-30T16:57:54.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92282 2024-05-30T16:57:54.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.600+0000 7fc535800640 1 Processor -- start 2024-05-30T16:57:54.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.600+0000 7fc535800640 1 -- start start 2024-05-30T16:57:54.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.602+0000 7fc535800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc530161e40 0x7fc5300b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.602+0000 7fc535800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc530073030 0x7fc53006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.602+0000 7fc535800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5300703e0 con 0x7fc53006ce50 2024-05-30T16:57:54.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.602+0000 7fc535800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc53006d2c0 con 0x7fc530161e40 2024-05-30T16:57:54.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.602+0000 7fc535800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc530162220 con 0x7fc530073030 2024-05-30T16:57:54.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.602+0000 7fc52ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc530161e40 0x7fc5300b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.602+0000 7fc52ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc530161e40 0x7fc5300b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47008/0 (socket says 172.21.15.67:47008) 2024-05-30T16:57:54.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.602+0000 7fc52ea00640 1 -- 172.21.15.67:0/1930514418 learned_addr learned my addr 172.21.15.67:0/1930514418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.603+0000 7fc52ea00640 1 -- 172.21.15.67:0/1930514418 >> v2:172.21.15.169:3300/0 conn(0x7fc530073030 msgr2=0x7fc53006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.603+0000 7fc52ea00640 1 --2- 172.21.15.67:0/1930514418 >> v2:172.21.15.169:3300/0 conn(0x7fc530073030 0x7fc53006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.603+0000 7fc52ea00640 1 -- 172.21.15.67:0/1930514418 >> v1:172.21.15.67:6789/0 conn(0x7fc53006ce50 legacy=0x7fc530162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.603+0000 7fc52ea00640 1 -- 172.21.15.67:0/1930514418 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53006e610 con 0x7fc530161e40 2024-05-30T16:57:54.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.604+0000 7fc52ea00640 1 --2- 172.21.15.67:0/1930514418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530161e40 0x7fc5300b3a10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc5300724c0 tx=0x7fc52002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb6d37cfb12ca4c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:54.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.604+0000 7fc52e000640 1 -- 172.21.15.67:0/1930514418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520037070 con 0x7fc530161e40 2024-05-30T16:57:54.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.604+0000 7fc52e000640 1 -- 172.21.15.67:0/1930514418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc520032440 con 0x7fc530161e40 2024-05-30T16:57:54.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.605+0000 7fc52e000640 1 -- 172.21.15.67:0/1930514418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520032810 con 0x7fc530161e40 2024-05-30T16:57:54.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.605+0000 7fc535800640 1 -- 172.21.15.67:0/1930514418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530161e40 msgr2=0x7fc5300b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.605+0000 7fc535800640 1 --2- 172.21.15.67:0/1930514418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530161e40 0x7fc5300b3a10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc5300724c0 tx=0x7fc52002a270 comp rx=0 tx=0).stop 2024-05-30T16:57:54.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.605+0000 7fc535800640 1 -- 172.21.15.67:0/1930514418 shutdown_connections 2024-05-30T16:57:54.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.605+0000 7fc535800640 1 --2- 172.21.15.67:0/1930514418 >> v2:172.21.15.169:3300/0 conn(0x7fc530073030 0x7fc53006c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.605+0000 7fc535800640 1 --2- 172.21.15.67:0/1930514418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530161e40 0x7fc5300b3a10 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.605+0000 7fc535800640 1 -- 172.21.15.67:0/1930514418 >> 172.21.15.67:0/1930514418 conn(0x7fc530072510 msgr2=0x7fc530072920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.605+0000 7fc535800640 1 -- 172.21.15.67:0/1930514418 shutdown_connections 2024-05-30T16:57:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.605+0000 7fc535800640 1 -- 172.21.15.67:0/1930514418 wait complete. 2024-05-30T16:57:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.607+0000 7fc535800640 1 Processor -- start 2024-05-30T16:57:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.607+0000 7fc535800640 1 -- start start 2024-05-30T16:57:54.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.608+0000 7fc535800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc53006ce50 0x7fc53014ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.608+0000 7fc535800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530073030 0x7fc5301f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.608+0000 7fc535800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc530161e40 0x7fc5301f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.608+0000 7fc535800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc53006eeb0 con 0x7fc530073030 2024-05-30T16:57:54.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.608+0000 7fc535800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc53006ebd0 con 0x7fc530161e40 2024-05-30T16:57:54.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.608+0000 7fc535800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc530067af0 con 0x7fc53006ce50 2024-05-30T16:57:54.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc527e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530073030 0x7fc5301f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc527e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530073030 0x7fc5301f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47016/0 (socket says 172.21.15.67:47016) 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc527e00640 1 -- 172.21.15.67:0/1177317307 learned_addr learned my addr 172.21.15.67:0/1177317307 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc52ea00640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc53006ce50 0x7fc53014ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc527e00640 1 -- 172.21.15.67:0/1177317307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc53006ce50 msgr2=0x7fc53014ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc527e00640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc53006ce50 0x7fc53014ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc527e00640 1 -- 172.21.15.67:0/1177317307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc530161e40 msgr2=0x7fc5301f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc52f400640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc530161e40 0x7fc5301f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc527e00640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc530161e40 0x7fc5301f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc527e00640 1 -- 172.21.15.67:0/1177317307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53014ea00 con 0x7fc530073030 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc52f400640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc530161e40 0x7fc5301f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.609+0000 7fc527e00640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530073030 0x7fc5301f7ef0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc51800b4d0 tx=0x7fc51800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.610+0000 7fc526a00640 1 -- 172.21.15.67:0/1177317307 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518014070 con 0x7fc530073030 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.610+0000 7fc526a00640 1 -- 172.21.15.67:0/1177317307 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc518010870 con 0x7fc530073030 2024-05-30T16:57:54.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.610+0000 7fc526a00640 1 -- 172.21.15.67:0/1177317307 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518010c60 con 0x7fc530073030 2024-05-30T16:57:54.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.610+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53014ece0 con 0x7fc530073030 2024-05-30T16:57:54.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.610+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53014f2a0 con 0x7fc530073030 2024-05-30T16:57:54.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.611+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5300cd900 con 0x7fc530073030 2024-05-30T16:57:54.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.612+0000 7fc526a00640 1 -- 172.21.15.67:0/1177317307 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7fc518017070 con 0x7fc530073030 2024-05-30T16:57:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.613+0000 7fc526a00640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc50c06d9d0 0x7fc50c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.613+0000 7fc526a00640 1 -- 172.21.15.67:0/1177317307 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fc51808a7e0 con 0x7fc530073030 2024-05-30T16:57:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.614+0000 7fc52ea00640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc50c06d9d0 0x7fc50c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.614+0000 7fc52ea00640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc50c06d9d0 0x7fc50c06fe90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc520034000 tx=0x7fc52002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:54.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.618+0000 7fc526a00640 1 -- 172.21.15.67:0/1177317307 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5300cd900 con 0x7fc530073030 2024-05-30T16:57:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.852+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92282", "target": ["mon-mgr", ""]}) -- 0x7fc53014f680 con 0x7fc50c06d9d0 2024-05-30T16:57:54.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.894+0000 7fc526a00640 1 -- 172.21.15.67:0/1177317307 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc53014f680 con 0x7fc50c06d9d0 2024-05-30T16:57:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.897+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc50c06d9d0 msgr2=0x7fc50c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.897+0000 7fc535800640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc50c06d9d0 0x7fc50c06fe90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc520034000 tx=0x7fc52002e000 comp rx=0 tx=0).stop 2024-05-30T16:57:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.897+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530073030 msgr2=0x7fc5301f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.897+0000 7fc535800640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530073030 0x7fc5301f7ef0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc51800b4d0 tx=0x7fc51800b990 comp rx=0 tx=0).stop 2024-05-30T16:57:54.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.898+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 shutdown_connections 2024-05-30T16:57:54.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.898+0000 7fc535800640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc50c06d9d0 0x7fc50c06fe90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.898+0000 7fc535800640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc530161e40 0x7fc5301f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.898+0000 7fc535800640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc530073030 0x7fc5301f7ef0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.898+0000 7fc535800640 1 --2- 172.21.15.67:0/1177317307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc53006ce50 0x7fc53014ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:54.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.898+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 >> 172.21.15.67:0/1177317307 conn(0x7fc530072510 msgr2=0x7fc5300b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:54.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.898+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 shutdown_connections 2024-05-30T16:57:54.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:54.898+0000 7fc535800640 1 -- 172.21.15.67:0/1177317307 wait complete. 2024-05-30T16:57:54.902 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.902+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:54.902 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.902+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:54.902 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.902+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:54.902 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.902+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:54.902 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.902+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:54.923 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92283 2024-05-30T16:57:54.923 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.922+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:54.924 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.922+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:54.924 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.922+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:54.924 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.922+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:54.924 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:57:54.922+0000 7f0c8bc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:57:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.109+0000 7f44af200640 1 Processor -- start 2024-05-30T16:57:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.110+0000 7f44af200640 1 -- start start 2024-05-30T16:57:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.111+0000 7f44af200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f44b01480a0 0x7f44b00d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.111+0000 7f44af200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b01428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.111+0000 7f44af200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44b0162cb0 con 0x7f44b00af6f0 2024-05-30T16:57:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.111+0000 7f44af200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f44b00d0dc0 con 0x7f44b01480a0 2024-05-30T16:57:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.111+0000 7f44af200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44b0162f90 con 0x7f44b00d0970 2024-05-30T16:57:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.112+0000 7f44ae800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b01428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.112+0000 7f44ae800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b01428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:37374/0 (socket says 172.21.15.67:37374) 2024-05-30T16:57:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.112+0000 7f44ae800640 1 -- 172.21.15.67:0/4096292336 learned_addr learned my addr 172.21.15.67:0/4096292336 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.112+0000 7f44a7e00640 1 --2- 172.21.15.67:0/4096292336 >> v2:172.21.15.169:3300/0 conn(0x7f44b01480a0 0x7f44b00d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.113+0000 7f44ae800640 1 -- 172.21.15.67:0/4096292336 >> v2:172.21.15.169:3300/0 conn(0x7f44b01480a0 msgr2=0x7f44b00d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.113+0000 7f44ae800640 1 --2- 172.21.15.67:0/4096292336 >> v2:172.21.15.169:3300/0 conn(0x7f44b01480a0 0x7f44b00d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.113+0000 7f44ae800640 1 -- 172.21.15.67:0/4096292336 >> v1:172.21.15.169:6789/0 conn(0x7f44b00af6f0 legacy=0x7f44b01623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.113+0000 7f44ae800640 1 -- 172.21.15.67:0/4096292336 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b0163270 con 0x7f44b00d0970 2024-05-30T16:57:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.114+0000 7f44ae800640 1 --2- 172.21.15.67:0/4096292336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b01428d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009e50 tx=0x7f44a802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9b1d2ac4c4e56e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.115+0000 7f44ad400640 1 -- 172.21.15.67:0/4096292336 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8036070 con 0x7f44b00d0970 2024-05-30T16:57:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.115+0000 7f44ad400640 1 -- 172.21.15.67:0/4096292336 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a8031600 con 0x7f44b00d0970 2024-05-30T16:57:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.115+0000 7f44ad400640 1 -- 172.21.15.67:0/4096292336 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80319d0 con 0x7f44b00d0970 2024-05-30T16:57:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.115+0000 7f44af200640 1 -- 172.21.15.67:0/4096292336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 msgr2=0x7f44b01428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.115+0000 7f44af200640 1 --2- 172.21.15.67:0/4096292336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b01428d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009e50 tx=0x7f44a802a710 comp rx=0 tx=0).stop 2024-05-30T16:57:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.115+0000 7f44af200640 1 -- 172.21.15.67:0/4096292336 shutdown_connections 2024-05-30T16:57:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.115+0000 7f44af200640 1 --2- 172.21.15.67:0/4096292336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b01428d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.115+0000 7f44af200640 1 --2- 172.21.15.67:0/4096292336 >> v2:172.21.15.169:3300/0 conn(0x7f44b01480a0 0x7f44b00d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.115+0000 7f44af200640 1 -- 172.21.15.67:0/4096292336 >> 172.21.15.67:0/4096292336 conn(0x7f44b006bae0 msgr2=0x7f44b006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.116+0000 7f44af200640 1 -- 172.21.15.67:0/4096292336 shutdown_connections 2024-05-30T16:57:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.116+0000 7f44af200640 1 -- 172.21.15.67:0/4096292336 wait complete. 2024-05-30T16:57:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.117+0000 7f44af200640 1 Processor -- start 2024-05-30T16:57:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.117+0000 7f44af200640 1 -- start start 2024-05-30T16:57:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.118+0000 7f44af200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b00af6f0 0x7f44b015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.118+0000 7f44af200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b00ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.118+0000 7f44af200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b01480a0 0x7f44b00ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.118+0000 7f44af200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f44b00afad0 con 0x7f44b01480a0 2024-05-30T16:57:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.118+0000 7f44af200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01423b0 con 0x7f44b00af6f0 2024-05-30T16:57:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.118+0000 7f44af200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44b00afc50 con 0x7f44b00d0970 2024-05-30T16:57:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44ae800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b01480a0 0x7f44b00ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44ae800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b01480a0 0x7f44b00ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47054/0 (socket says 172.21.15.67:47054) 2024-05-30T16:57:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44ae800640 1 -- 172.21.15.67:0/126890632 learned_addr learned my addr 172.21.15.67:0/126890632 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44ade00640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b00af6f0 0x7f44b015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44a7e00640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b00ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44ae800640 1 -- 172.21.15.67:0/126890632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 msgr2=0x7f44b00ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44ae800640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b00ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44ae800640 1 -- 172.21.15.67:0/126890632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b00af6f0 msgr2=0x7f44b015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44ae800640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b00af6f0 0x7f44b015cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.119+0000 7f44ae800640 1 -- 172.21.15.67:0/126890632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b015f860 con 0x7f44b01480a0 2024-05-30T16:57:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.120+0000 7f44ae800640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b01480a0 0x7f44b00ae860 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f44a8034040 tx=0x7f44a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.120+0000 7f44a6000640 1 -- 172.21.15.67:0/126890632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a803f020 con 0x7f44b01480a0 2024-05-30T16:57:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.120+0000 7f44a6000640 1 -- 172.21.15.67:0/126890632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a8037040 con 0x7f44b01480a0 2024-05-30T16:57:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.120+0000 7f44a6000640 1 -- 172.21.15.67:0/126890632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8036070 con 0x7f44b01480a0 2024-05-30T16:57:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.120+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b0161040 con 0x7f44b01480a0 2024-05-30T16:57:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.120+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b0161600 con 0x7f44b01480a0 2024-05-30T16:57:55.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.121+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b00689e0 con 0x7f44b01480a0 2024-05-30T16:57:55.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.123+0000 7f44a6000640 1 -- 172.21.15.67:0/126890632 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f44a8042020 con 0x7f44b01480a0 2024-05-30T16:57:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.123+0000 7f44a6000640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f448c06d9d0 0x7f448c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.124+0000 7f44a6000640 1 -- 172.21.15.67:0/126890632 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f44a80aaa70 con 0x7f44b01480a0 2024-05-30T16:57:55.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.124+0000 7f44ade00640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f448c06d9d0 0x7f448c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.124+0000 7f44ade00640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f448c06d9d0 0x7f448c06fe90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f44a00016e0 tx=0x7f44a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:55.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.131+0000 7f44a6000640 1 -- 172.21.15.67:0/126890632 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44a8049030 con 0x7f44b01480a0 2024-05-30T16:57:55.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.365+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92283", "target": ["mon-mgr", ""]}) -- 0x7f44b0147360 con 0x7f448c06d9d0 2024-05-30T16:57:55.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.408+0000 7f44a6000640 1 -- 172.21.15.67:0/126890632 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f44b0147360 con 0x7f448c06d9d0 2024-05-30T16:57:55.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f448c06d9d0 msgr2=0x7f448c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f448c06d9d0 0x7f448c06fe90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f44a00016e0 tx=0x7f44a0009040 comp rx=0 tx=0).stop 2024-05-30T16:57:55.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b01480a0 msgr2=0x7f44b00ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b01480a0 0x7f44b00ae860 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f44a8034040 tx=0x7f44a8004a40 comp rx=0 tx=0).stop 2024-05-30T16:57:55.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 shutdown_connections 2024-05-30T16:57:55.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f448c06d9d0 0x7f448c06fe90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b01480a0 0x7f44b00ae860 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44b00d0970 0x7f44b00ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 --2- 172.21.15.67:0/126890632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44b00af6f0 0x7f44b015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.412+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 >> 172.21.15.67:0/126890632 conn(0x7f44b006bae0 msgr2=0x7f44b00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.413+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 shutdown_connections 2024-05-30T16:57:55.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.413+0000 7f44af200640 1 -- 172.21.15.67:0/126890632 wait complete. 2024-05-30T16:57:55.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922816 2024-05-30T16:57:55.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.630+0000 7f0b16a00640 1 Processor -- start 2024-05-30T16:57:55.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.631+0000 7f0b16a00640 1 -- start start 2024-05-30T16:57:55.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.632+0000 7f0b16a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b100d0430 0x7f0b100d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.632+0000 7f0b16a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b100d0d70 con 0x7f0b1015f0c0 2024-05-30T16:57:55.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.632+0000 7f0b16a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b1015bb30 con 0x7f0b100ce560 2024-05-30T16:57:55.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.632+0000 7f0b16a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100b5290 con 0x7f0b100d0430 2024-05-30T16:57:55.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.633+0000 7f0b16000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0b1015f0c0 0x7f0b1015f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33672/0 (socket says 172.21.15.67:33672) 2024-05-30T16:57:55.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.633+0000 7f0b16000640 1 -- 172.21.15.67:0/1918706324 learned_addr learned my addr 172.21.15.67:0/1918706324 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:55.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.633+0000 7f0b14c00640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b100d0430 0x7f0b100d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.633+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310961127 0 0) 0x7f0b100d0d70 con 0x7f0b1015f0c0 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.633+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af8003520 con 0x7f0b1015f0c0 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3389229661 0 0) 0x7f0af8003520 con 0x7f0b1015f0c0 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b100d0430 msgr2=0x7f0b100d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b100d0430 0x7f0b100d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 >> v1:172.21.15.169:6789/0 conn(0x7f0b100ce560 legacy=0x7f0b100b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b100b3ba0 con 0x7f0b1015f0c0 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f0b080019f0 con 0x7f0b1015f0c0 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 >> v1:172.21.15.67:6789/0 conn(0x7f0b1015f0c0 legacy=0x7f0b1015f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af80039e0 0x7f0af8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b100d0430 0x7f0af8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af8004d40 0x7f0af80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100b5290 con 0x7f0af8004d40 2024-05-30T16:57:55.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0af800ba70 con 0x7f0af80039e0 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0af800bc70 con 0x7f0b100d0430 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.634+0000 7f0b16000640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af8004d40 0x7f0af80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.635+0000 7f0b15600640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af80039e0 0x7f0af8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.635+0000 7f0b14c00640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b100d0430 0x7f0af8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.635+0000 7f0b16000640 1 -- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b100d0430 msgr2=0x7f0af8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.635+0000 7f0b16000640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b100d0430 0x7f0af8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.635+0000 7f0b16000640 1 -- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af80039e0 msgr2=0x7f0af8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.635+0000 7f0b16000640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0af80039e0 0x7f0af8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.635+0000 7f0b16000640 1 -- 172.21.15.67:0/1918706324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af8003520 con 0x7f0af8004d40 2024-05-30T16:57:55.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.635+0000 7f0b14c00640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b100d0430 0x7f0af8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:55.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.636+0000 7f0b16000640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af8004d40 0x7f0af80051c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0b08002230 tx=0x7f0af8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5ec939e062abb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:55.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.636+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800a4a0 con 0x7f0af8004d40 2024-05-30T16:57:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.636+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0802f070 con 0x7f0af8004d40 2024-05-30T16:57:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.636+0000 7f0b0fe00640 1 -- 172.21.15.67:0/1918706324 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0802bcb0 con 0x7f0af8004d40 2024-05-30T16:57:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.637+0000 7f0b16a00640 1 -- 172.21.15.67:0/1918706324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af8004d40 msgr2=0x7f0af80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.637+0000 7f0b16a00640 1 --2- 172.21.15.67:0/1918706324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af8004d40 0x7f0af80051c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0b08002230 tx=0x7f0af8005830 comp rx=0 tx=0).stop 2024-05-30T16:57:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.637+0000 7f0b16000640 1 -- 172.21.15.67:0/1918706324 reap_dead start 2024-05-30T16:57:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.637+0000 7f0b16a00640 1 -- 172.21.15.67:0/1918706324 shutdown_connections 2024-05-30T16:57:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.637+0000 7f0b16a00640 1 -- 172.21.15.67:0/1918706324 >> 172.21.15.67:0/1918706324 conn(0x7f0b1006bae0 msgr2=0x7f0b1006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.637+0000 7f0b16a00640 1 -- 172.21.15.67:0/1918706324 shutdown_connections 2024-05-30T16:57:55.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.637+0000 7f0b16a00640 1 -- 172.21.15.67:0/1918706324 wait complete. 2024-05-30T16:57:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.639+0000 7f0b16a00640 1 Processor -- start 2024-05-30T16:57:55.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.639+0000 7f0b16a00640 1 -- start start 2024-05-30T16:57:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.640+0000 7f0b16a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af8004d40 0x7f0b1009abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.640+0000 7f0b16a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b1014c8c0 0x7f0b1014c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.640+0000 7f0b16a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b100b15b0 0x7f0b100b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.640+0000 7f0b16a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015c390 con 0x7f0b100b15b0 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.640+0000 7f0b16a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015c0b0 con 0x7f0b1014c8c0 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.640+0000 7f0b16a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1015c670 con 0x7f0af8004d40 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.640+0000 7f0b16000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b100b15b0 0x7f0b100b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.640+0000 7f0b16000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b100b15b0 0x7f0b100b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47068/0 (socket says 172.21.15.67:47068) 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.640+0000 7f0b16000640 1 -- 172.21.15.67:0/3545989246 learned_addr learned my addr 172.21.15.67:0/3545989246 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b15600640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af8004d40 0x7f0b1009abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b16000640 1 -- 172.21.15.67:0/3545989246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af8004d40 msgr2=0x7f0b1009abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b14c00640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b1014c8c0 0x7f0b1014c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b16000640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af8004d40 0x7f0b1009abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b16000640 1 -- 172.21.15.67:0/3545989246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b1014c8c0 msgr2=0x7f0b1014c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b16000640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b1014c8c0 0x7f0b1014c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b16000640 1 -- 172.21.15.67:0/3545989246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1009d2c0 con 0x7f0b100b15b0 2024-05-30T16:57:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b14c00640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b1014c8c0 0x7f0b1014c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b16000640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b100b15b0 0x7f0b100b1a30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0b0802b500 tx=0x7f0b08001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.641+0000 7f0b0d600640 1 -- 172.21.15.67:0/3545989246 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b080019e0 con 0x7f0b100b15b0 2024-05-30T16:57:55.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.642+0000 7f0b0d600640 1 -- 172.21.15.67:0/3545989246 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0800a480 con 0x7f0b100b15b0 2024-05-30T16:57:55.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.642+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1009d5a0 con 0x7f0b100b15b0 2024-05-30T16:57:55.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.642+0000 7f0b0d600640 1 -- 172.21.15.67:0/3545989246 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004da0 con 0x7f0b100b15b0 2024-05-30T16:57:55.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.642+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1009db60 con 0x7f0b100b15b0 2024-05-30T16:57:55.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.643+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b100689e0 con 0x7f0b100b15b0 2024-05-30T16:57:55.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.644+0000 7f0b0d600640 1 -- 172.21.15.67:0/3545989246 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f0b08034470 con 0x7f0b100b15b0 2024-05-30T16:57:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.645+0000 7f0b0d600640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0af006db70 0x7f0af0070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.646+0000 7f0b0d600640 1 -- 172.21.15.67:0/3545989246 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f0b080aace0 con 0x7f0b100b15b0 2024-05-30T16:57:55.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.646+0000 7f0b15600640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0af006db70 0x7f0af0070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:55.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.647+0000 7f0b15600640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0af006db70 0x7f0af0070030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0b04007fd0 tx=0x7f0b04009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:55.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.650+0000 7f0b0d600640 1 -- 172.21.15.67:0/3545989246 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b0807abc0 con 0x7f0b100b15b0 2024-05-30T16:57:55.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.886+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922816", "target": ["mon-mgr", ""]}) -- 0x7f0b100b1f70 con 0x7f0af006db70 2024-05-30T16:57:55.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.937+0000 7f0b0d600640 1 -- 172.21.15.67:0/3545989246 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b0400f510 con 0x7f0af006db70 2024-05-30T16:57:55.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0af006db70 msgr2=0x7f0af0070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0af006db70 0x7f0af0070030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0b04007fd0 tx=0x7f0b04009040 comp rx=0 tx=0).stop 2024-05-30T16:57:55.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b100b15b0 msgr2=0x7f0b100b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:55.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b100b15b0 0x7f0b100b1a30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0b0802b500 tx=0x7f0b08001dc0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 shutdown_connections 2024-05-30T16:57:55.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0af006db70 0x7f0af0070030 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b100b15b0 0x7f0b100b1a30 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b1014c8c0 0x7f0b1014c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 --2- 172.21.15.67:0/3545989246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0af8004d40 0x7f0b1009abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:55.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 >> 172.21.15.67:0/3545989246 conn(0x7f0b1006bae0 msgr2=0x7f0b100ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 shutdown_connections 2024-05-30T16:57:55.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:55.940+0000 7f0b16a00640 1 -- 172.21.15.67:0/3545989246 wait complete. 2024-05-30T16:57:55.965 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-30T16:57:56.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.165+0000 7f5151800640 1 Processor -- start 2024-05-30T16:57:56.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.165+0000 7f5151800640 1 -- start start 2024-05-30T16:57:56.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.166+0000 7f5151800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c15e5a0 0x7f514c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.166+0000 7f5151800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f514c0d1420 0x7f514c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.166+0000 7f5151800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f514c0d3500 con 0x7f514c0cfb40 2024-05-30T16:57:56.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.166+0000 7f5151800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f514c0d2b30 con 0x7f514c0d1420 2024-05-30T16:57:56.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.166+0000 7f5151800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f514c0d2200 con 0x7f514c15e5a0 2024-05-30T16:57:56.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.167+0000 7f514be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c15e5a0 0x7f514c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.167+0000 7f514b400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f514c0cfb40 0x7f514c0d0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:38240/0 (socket says 172.21.15.67:38240) 2024-05-30T16:57:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.167+0000 7f514b400640 1 -- 172.21.15.67:0/1719021851 learned_addr learned my addr 172.21.15.67:0/1719021851 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.167+0000 7f5150e00640 1 --2- 172.21.15.67:0/1719021851 >> v2:172.21.15.169:3300/0 conn(0x7f514c0d1420 0x7f514c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.168+0000 7f514be00640 1 -- 172.21.15.67:0/1719021851 >> v2:172.21.15.169:3300/0 conn(0x7f514c0d1420 msgr2=0x7f514c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.168+0000 7f514be00640 1 --2- 172.21.15.67:0/1719021851 >> v2:172.21.15.169:3300/0 conn(0x7f514c0d1420 0x7f514c06c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.168+0000 7f514aa00640 1 -- 172.21.15.67:0/1719021851 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2713685323 0 0) 0x7f514c0d3500 con 0x7f514c0cfb40 2024-05-30T16:57:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.168+0000 7f514be00640 1 -- 172.21.15.67:0/1719021851 >> v1:172.21.15.169:6789/0 conn(0x7f514c0cfb40 legacy=0x7f514c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.168+0000 7f514be00640 1 -- 172.21.15.67:0/1719021851 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c0d24e0 con 0x7f514c15e5a0 2024-05-30T16:57:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.168+0000 7f5150e00640 1 --2- 172.21.15.67:0/1719021851 >> v2:172.21.15.169:3300/0 conn(0x7f514c0d1420 0x7f514c06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:57:56.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.170+0000 7f514be00640 1 --2- 172.21.15.67:0/1719021851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c15e5a0 0x7f514c0d2fc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f513c0097a0 tx=0x7f513c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330be009d514988d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:56.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.170+0000 7f514aa00640 1 -- 172.21.15.67:0/1719021851 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c037070 con 0x7f514c15e5a0 2024-05-30T16:57:56.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.170+0000 7f514aa00640 1 -- 172.21.15.67:0/1719021851 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513c032440 con 0x7f514c15e5a0 2024-05-30T16:57:56.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.170+0000 7f514aa00640 1 -- 172.21.15.67:0/1719021851 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c032810 con 0x7f514c15e5a0 2024-05-30T16:57:56.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.171+0000 7f5151800640 1 -- 172.21.15.67:0/1719021851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c15e5a0 msgr2=0x7f514c0d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.171+0000 7f5151800640 1 --2- 172.21.15.67:0/1719021851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c15e5a0 0x7f514c0d2fc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f513c0097a0 tx=0x7f513c02a180 comp rx=0 tx=0).stop 2024-05-30T16:57:56.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.171+0000 7f5151800640 1 -- 172.21.15.67:0/1719021851 shutdown_connections 2024-05-30T16:57:56.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.171+0000 7f5151800640 1 --2- 172.21.15.67:0/1719021851 >> v2:172.21.15.169:3300/0 conn(0x7f514c0d1420 0x7f514c06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.171+0000 7f5151800640 1 --2- 172.21.15.67:0/1719021851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c15e5a0 0x7f514c0d2fc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.171+0000 7f5151800640 1 -- 172.21.15.67:0/1719021851 >> 172.21.15.67:0/1719021851 conn(0x7f514c08bfc0 msgr2=0x7f514c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:56.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.171+0000 7f5151800640 1 -- 172.21.15.67:0/1719021851 shutdown_connections 2024-05-30T16:57:56.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.171+0000 7f5151800640 1 -- 172.21.15.67:0/1719021851 wait complete. 2024-05-30T16:57:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.173+0000 7f5151800640 1 Processor -- start 2024-05-30T16:57:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.173+0000 7f5151800640 1 -- start start 2024-05-30T16:57:56.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.174+0000 7f5151800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f514c0cfb40 0x7f514c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.174+0000 7f5151800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c0d1420 0x7f514c14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.174+0000 7f5151800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f514c15e5a0 0x7f514c147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.174+0000 7f5151800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f514c15eb00 con 0x7f514c15e5a0 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.174+0000 7f5151800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f514c15e980 con 0x7f514c0cfb40 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.174+0000 7f5151800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f514c0d5220 con 0x7f514c0d1420 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.174+0000 7f5150e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f514c15e5a0 0x7f514c147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.174+0000 7f5150e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f514c15e5a0 0x7f514c147fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47076/0 (socket says 172.21.15.67:47076) 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.174+0000 7f5150e00640 1 -- 172.21.15.67:0/1887751777 learned_addr learned my addr 172.21.15.67:0/1887751777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.175+0000 7f514b400640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c0d1420 0x7f514c14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.175+0000 7f5150e00640 1 -- 172.21.15.67:0/1887751777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c0d1420 msgr2=0x7f514c14aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.175+0000 7f514be00640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f514c0cfb40 0x7f514c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.175+0000 7f5150e00640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c0d1420 0x7f514c14aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.175+0000 7f5150e00640 1 -- 172.21.15.67:0/1887751777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f514c0cfb40 msgr2=0x7f514c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.175+0000 7f5150e00640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f514c0cfb40 0x7f514c0cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.175+0000 7f5150e00640 1 -- 172.21.15.67:0/1887751777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c1f4310 con 0x7f514c15e5a0 2024-05-30T16:57:56.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.175+0000 7f5150e00640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f514c15e5a0 0x7f514c147fa0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f514400efc0 tx=0x7f514400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:56.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.176+0000 7f5143e00640 1 -- 172.21.15.67:0/1887751777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144018030 con 0x7f514c15e5a0 2024-05-30T16:57:56.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.176+0000 7f5143e00640 1 -- 172.21.15.67:0/1887751777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514400cdf0 con 0x7f514c15e5a0 2024-05-30T16:57:56.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.176+0000 7f5143e00640 1 -- 172.21.15.67:0/1887751777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144003bb0 con 0x7f514c15e5a0 2024-05-30T16:57:56.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.176+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c0c9c90 con 0x7f514c15e5a0 2024-05-30T16:57:56.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.176+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c0ca250 con 0x7f514c15e5a0 2024-05-30T16:57:56.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.177+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514c0689e0 con 0x7f514c15e5a0 2024-05-30T16:57:56.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.178+0000 7f5143e00640 1 -- 172.21.15.67:0/1887751777 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7f514401b070 con 0x7f514c15e5a0 2024-05-30T16:57:56.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.179+0000 7f5143e00640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f512806d9d0 0x7f512806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.180+0000 7f5143e00640 1 -- 172.21.15.67:0/1887751777 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f514408b480 con 0x7f514c15e5a0 2024-05-30T16:57:56.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.180+0000 7f514be00640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f512806d9d0 0x7f512806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.180+0000 7f514be00640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f512806d9d0 0x7f512806fe90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f513c034000 tx=0x7f513c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:56.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.184+0000 7f5143e00640 1 -- 172.21.15.67:0/1887751777 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5144010040 con 0x7f514c15e5a0 2024-05-30T16:57:56.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.420+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f514c0d1d30 con 0x7f512806d9d0 2024-05-30T16:57:56.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.448+0000 7f5143e00640 1 -- 172.21.15.67:0/1887751777 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f514c0d1d30 con 0x7f512806d9d0 2024-05-30T16:57:56.449 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:56.449 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:57:56.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:57:56.449 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1" 2024-05-30T16:57:56.450 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:57:56.450 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:57:56.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.451+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f512806d9d0 msgr2=0x7f512806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.451+0000 7f5151800640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f512806d9d0 0x7f512806fe90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f513c034000 tx=0x7f513c02e000 comp rx=0 tx=0).stop 2024-05-30T16:57:56.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.451+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f514c15e5a0 msgr2=0x7f514c147fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.451+0000 7f5151800640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f514c15e5a0 0x7f514c147fa0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f514400efc0 tx=0x7f514400c490 comp rx=0 tx=0).stop 2024-05-30T16:57:56.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.452+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 shutdown_connections 2024-05-30T16:57:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.452+0000 7f5151800640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f512806d9d0 0x7f512806fe90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.452+0000 7f5151800640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f514c15e5a0 0x7f514c147fa0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.452+0000 7f5151800640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f514c0d1420 0x7f514c14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.452+0000 7f5151800640 1 --2- 172.21.15.67:0/1887751777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f514c0cfb40 0x7f514c0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.452+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 >> 172.21.15.67:0/1887751777 conn(0x7f514c08bfc0 msgr2=0x7f514c0b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.452+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 shutdown_connections 2024-05-30T16:57:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.452+0000 7f5151800640 1 -- 172.21.15.67:0/1887751777 wait complete. 2024-05-30T16:57:56.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-30T16:57:56.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.682+0000 7fe7dea00640 1 Processor -- start 2024-05-30T16:57:56.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.682+0000 7fe7dea00640 1 -- start start 2024-05-30T16:57:56.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.684+0000 7fe7dea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe7d815e870 0x7fe7d80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.684+0000 7fe7dea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80cfa40 0x7fe7d80d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.684+0000 7fe7dea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d815ec50 con 0x7fe7d80d14d0 2024-05-30T16:57:56.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.684+0000 7fe7dea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d80d2150 con 0x7fe7d815e870 2024-05-30T16:57:56.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.684+0000 7fe7dea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d80cff80 con 0x7fe7d80cfa40 2024-05-30T16:57:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.685+0000 7fe7dd600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe7d815e870 0x7fe7d80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.685+0000 7fe7dd600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe7d815e870 0x7fe7d80d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47080/0 (socket says 172.21.15.67:47080) 2024-05-30T16:57:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.685+0000 7fe7dd600640 1 -- 172.21.15.67:0/80987882 learned_addr learned my addr 172.21.15.67:0/80987882 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.685+0000 7fe7de000640 1 --2- 172.21.15.67:0/80987882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80cfa40 0x7fe7d80d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.685+0000 7fe7dd600640 1 -- 172.21.15.67:0/80987882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80cfa40 msgr2=0x7fe7d80d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.685+0000 7fe7dd600640 1 --2- 172.21.15.67:0/80987882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80cfa40 0x7fe7d80d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.685+0000 7fe7dd600640 1 -- 172.21.15.67:0/80987882 >> v1:172.21.15.67:6789/0 conn(0x7fe7d80d14d0 legacy=0x7fe7d80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.685+0000 7fe7dd600640 1 -- 172.21.15.67:0/80987882 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d80d0260 con 0x7fe7d815e870 2024-05-30T16:57:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.686+0000 7fe7dd600640 1 --2- 172.21.15.67:0/80987882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 0x7fe7d80d0f90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009e50 tx=0x7fe7cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88bf987b92a27ed server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.687+0000 7fe7d7e00640 1 -- 172.21.15.67:0/80987882 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc036070 con 0x7fe7d815e870 2024-05-30T16:57:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.687+0000 7fe7d7e00640 1 -- 172.21.15.67:0/80987882 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7cc031600 con 0x7fe7d815e870 2024-05-30T16:57:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.687+0000 7fe7d7e00640 1 -- 172.21.15.67:0/80987882 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc0319d0 con 0x7fe7d815e870 2024-05-30T16:57:56.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.687+0000 7fe7dea00640 1 -- 172.21.15.67:0/80987882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 msgr2=0x7fe7d80d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.687+0000 7fe7dea00640 1 --2- 172.21.15.67:0/80987882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 0x7fe7d80d0f90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009e50 tx=0x7fe7cc02a710 comp rx=0 tx=0).stop 2024-05-30T16:57:56.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.687+0000 7fe7dea00640 1 -- 172.21.15.67:0/80987882 shutdown_connections 2024-05-30T16:57:56.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.687+0000 7fe7dea00640 1 --2- 172.21.15.67:0/80987882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80cfa40 0x7fe7d80d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.687+0000 7fe7dea00640 1 --2- 172.21.15.67:0/80987882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 0x7fe7d80d0f90 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.687+0000 7fe7dea00640 1 -- 172.21.15.67:0/80987882 >> 172.21.15.67:0/80987882 conn(0x7fe7d808bfc0 msgr2=0x7fe7d80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:56.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.688+0000 7fe7dea00640 1 -- 172.21.15.67:0/80987882 shutdown_connections 2024-05-30T16:57:56.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.688+0000 7fe7dea00640 1 -- 172.21.15.67:0/80987882 wait complete. 2024-05-30T16:57:56.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.689+0000 7fe7dea00640 1 Processor -- start 2024-05-30T16:57:56.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.689+0000 7fe7dea00640 1 -- start start 2024-05-30T16:57:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.690+0000 7fe7dea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7d80cfa40 0x7fe7d815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.690+0000 7fe7dea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80d14d0 0x7fe7d80b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.690+0000 7fe7dea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 0x7fe7d80a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.690+0000 7fe7dea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d80d4fb0 con 0x7fe7d815e870 2024-05-30T16:57:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.690+0000 7fe7dea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d80d0820 con 0x7fe7d80cfa40 2024-05-30T16:57:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.690+0000 7fe7dea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d80d2b20 con 0x7fe7d80d14d0 2024-05-30T16:57:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7de000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 0x7fe7d80a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7de000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 0x7fe7d80a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47092/0 (socket says 172.21.15.67:47092) 2024-05-30T16:57:56.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7de000640 1 -- 172.21.15.67:0/1920874138 learned_addr learned my addr 172.21.15.67:0/1920874138 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7dd600640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7d80cfa40 0x7fe7d815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7de000640 1 -- 172.21.15.67:0/1920874138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80d14d0 msgr2=0x7fe7d80b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7dcc00640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80d14d0 0x7fe7d80b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7de000640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80d14d0 0x7fe7d80b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7de000640 1 -- 172.21.15.67:0/1920874138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7d80cfa40 msgr2=0x7fe7d815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7de000640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7d80cfa40 0x7fe7d815da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7de000640 1 -- 172.21.15.67:0/1920874138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d80b72d0 con 0x7fe7d815e870 2024-05-30T16:57:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.691+0000 7fe7dd600640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7d80cfa40 0x7fe7d815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:56.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.692+0000 7fe7de000640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 0x7fe7d80a1850 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d5d0 tx=0x7fe7d000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.692+0000 7fe7d5600640 1 -- 172.21.15.67:0/1920874138 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00125f0 con 0x7fe7d815e870 2024-05-30T16:57:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.692+0000 7fe7d5600640 1 -- 172.21.15.67:0/1920874138 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d0019070 con 0x7fe7d815e870 2024-05-30T16:57:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.692+0000 7fe7d5600640 1 -- 172.21.15.67:0/1920874138 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00129d0 con 0x7fe7d815e870 2024-05-30T16:57:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.692+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d80b75b0 con 0x7fe7d815e870 2024-05-30T16:57:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.692+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d80b7b70 con 0x7fe7d815e870 2024-05-30T16:57:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.693+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d80b8060 con 0x7fe7d815e870 2024-05-30T16:57:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.694+0000 7fe7d5600640 1 -- 172.21.15.67:0/1920874138 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 30) ==== 92570+0+0 (secure 0 0 0) 0x7fe7d0012b70 con 0x7fe7d815e870 2024-05-30T16:57:56.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.696+0000 7fe7d5600640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe7b806d9d0 0x7fe7b806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:56.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.696+0000 7fe7d5600640 1 -- 172.21.15.67:0/1920874138 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fe7d008aba0 con 0x7fe7d815e870 2024-05-30T16:57:56.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.696+0000 7fe7dd600640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe7b806d9d0 0x7fe7b806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:56.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.697+0000 7fe7dd600640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe7b806d9d0 0x7fe7b806fe90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc033000 tx=0x7fe7cc009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:56.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.701+0000 7fe7d5600640 1 -- 172.21.15.67:0/1920874138 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7d005aa20 con 0x7fe7d815e870 2024-05-30T16:57:56.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:56.941+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7d80ba500 con 0x7fe7b806d9d0 2024-05-30T16:57:57.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.002+0000 7fe7d5600640 1 -- 172.21.15.67:0/1920874138 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7d80ba500 con 0x7fe7b806d9d0 2024-05-30T16:57:57.004 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:57:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.006+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe7b806d9d0 msgr2=0x7fe7b806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.006+0000 7fe7dea00640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe7b806d9d0 0x7fe7b806fe90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc033000 tx=0x7fe7cc009000 comp rx=0 tx=0).stop 2024-05-30T16:57:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.006+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 msgr2=0x7fe7d80a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.006+0000 7fe7dea00640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 0x7fe7d80a1850 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d5d0 tx=0x7fe7d000da90 comp rx=0 tx=0).stop 2024-05-30T16:57:57.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.007+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 shutdown_connections 2024-05-30T16:57:57.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.007+0000 7fe7dea00640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe7b806d9d0 0x7fe7b806fe90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.007+0000 7fe7dea00640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7d815e870 0x7fe7d80a1850 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.007+0000 7fe7dea00640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7d80d14d0 0x7fe7d80b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.007+0000 7fe7dea00640 1 --2- 172.21.15.67:0/1920874138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7d80cfa40 0x7fe7d815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.007+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 >> 172.21.15.67:0/1920874138 conn(0x7fe7d808bfc0 msgr2=0x7fe7d80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:57.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.007+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 shutdown_connections 2024-05-30T16:57:57.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.007+0000 7fe7dea00640 1 -- 172.21.15.67:0/1920874138 wait complete. 2024-05-30T16:57:57.036 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-30T16:57:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.299+0000 7f9de6000640 1 Processor -- start 2024-05-30T16:57:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.299+0000 7f9de6000640 1 -- start start 2024-05-30T16:57:57.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.302+0000 7f9de6000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9de015d5d0 0x7f9de00cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.303+0000 7f9de6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9de00b32c0 0x7f9de00b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.303+0000 7f9de6000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de015cb10 con 0x7f9de00d0640 2024-05-30T16:57:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.303+0000 7f9de6000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9de00adbb0 con 0x7f9de00b32c0 2024-05-30T16:57:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.303+0000 7f9de6000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9de00ad5f0 con 0x7f9de015d5d0 2024-05-30T16:57:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.303+0000 7f9ddea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9de015d5d0 0x7f9de00cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.303+0000 7f9ddf400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9de00d0640 0x7f9de00d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33688/0 (socket says 172.21.15.67:33688) 2024-05-30T16:57:57.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.303+0000 7f9ddf400640 1 -- 172.21.15.67:0/3476800700 learned_addr learned my addr 172.21.15.67:0/3476800700 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.304+0000 7f9dde000640 1 -- 172.21.15.67:0/3476800700 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277838184 0 0) 0x7f9de015cb10 con 0x7f9de00d0640 2024-05-30T16:57:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.304+0000 7f9dde000640 1 -- 172.21.15.67:0/3476800700 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dc4003520 con 0x7f9de00d0640 2024-05-30T16:57:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.304+0000 7f9ddfe00640 1 --2- 172.21.15.67:0/3476800700 >> v2:172.21.15.67:3300/0 conn(0x7f9de00b32c0 0x7f9de00b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.304+0000 7f9ddea00640 1 -- 172.21.15.67:0/3476800700 >> v2:172.21.15.67:3300/0 conn(0x7f9de00b32c0 msgr2=0x7f9de00b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.304+0000 7f9ddea00640 1 --2- 172.21.15.67:0/3476800700 >> v2:172.21.15.67:3300/0 conn(0x7f9de00b32c0 0x7f9de00b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.304+0000 7f9ddea00640 1 -- 172.21.15.67:0/3476800700 >> v1:172.21.15.67:6789/0 conn(0x7f9de00d0640 legacy=0x7f9de00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.304+0000 7f9ddea00640 1 -- 172.21.15.67:0/3476800700 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de00ad8d0 con 0x7f9de015d5d0 2024-05-30T16:57:57.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.304+0000 7f9ddfe00640 1 --2- 172.21.15.67:0/3476800700 >> v2:172.21.15.67:3300/0 conn(0x7f9de00b32c0 0x7f9de00b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:57.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.306+0000 7f9ddea00640 1 --2- 172.21.15.67:0/3476800700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015d5d0 0x7f9de00cf950 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9de0074120 tx=0x7f9dcc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d8c36efaaa5c5df server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:57.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.307+0000 7f9dde000640 1 -- 172.21.15.67:0/3476800700 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc037070 con 0x7f9de015d5d0 2024-05-30T16:57:57.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.307+0000 7f9dde000640 1 -- 172.21.15.67:0/3476800700 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc032440 con 0x7f9de015d5d0 2024-05-30T16:57:57.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.307+0000 7f9dde000640 1 -- 172.21.15.67:0/3476800700 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc02ae60 con 0x7f9de015d5d0 2024-05-30T16:57:57.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.309+0000 7f9de6000640 1 -- 172.21.15.67:0/3476800700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015d5d0 msgr2=0x7f9de00cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.309+0000 7f9de6000640 1 --2- 172.21.15.67:0/3476800700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015d5d0 0x7f9de00cf950 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9de0074120 tx=0x7f9dcc02a110 comp rx=0 tx=0).stop 2024-05-30T16:57:57.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.309+0000 7f9de6000640 1 -- 172.21.15.67:0/3476800700 shutdown_connections 2024-05-30T16:57:57.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.309+0000 7f9de6000640 1 --2- 172.21.15.67:0/3476800700 >> v2:172.21.15.67:3300/0 conn(0x7f9de00b32c0 0x7f9de00b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.309+0000 7f9de6000640 1 --2- 172.21.15.67:0/3476800700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015d5d0 0x7f9de00cf950 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9de0074120 tx=0x7f9dcc02a110 comp rx=0 tx=0).stop 2024-05-30T16:57:57.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.309+0000 7f9de6000640 1 -- 172.21.15.67:0/3476800700 >> 172.21.15.67:0/3476800700 conn(0x7f9de006bd20 msgr2=0x7f9de006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:57.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.310+0000 7f9de6000640 1 -- 172.21.15.67:0/3476800700 shutdown_connections 2024-05-30T16:57:57.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.311+0000 7f9de6000640 1 -- 172.21.15.67:0/3476800700 wait complete. 2024-05-30T16:57:57.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.314+0000 7f9de6000640 1 Processor -- start 2024-05-30T16:57:57.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.314+0000 7f9de6000640 1 -- start start 2024-05-30T16:57:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.315+0000 7f9de6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de00b32c0 0x7f9de01f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.315+0000 7f9de6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9de00d0640 0x7f9de0152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.315+0000 7f9de6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de0152a70 0x7f9de014ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.315+0000 7f9de6000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9de015da50 con 0x7f9de00b32c0 2024-05-30T16:57:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.315+0000 7f9de6000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9de015d0b0 con 0x7f9de0152a70 2024-05-30T16:57:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.315+0000 7f9de6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9de00ad2a0 con 0x7f9de00d0640 2024-05-30T16:57:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.316+0000 7f9ddea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9de00d0640 0x7f9de0152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.317+0000 7f9ddf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de00b32c0 0x7f9de01f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.317+0000 7f9ddea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9de00d0640 0x7f9de0152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:37452/0 (socket says 172.21.15.67:37452) 2024-05-30T16:57:57.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.317+0000 7f9ddea00640 1 -- 172.21.15.67:0/3859752312 learned_addr learned my addr 172.21.15.67:0/3859752312 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:57.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.318+0000 7f9ddf400640 1 -- 172.21.15.67:0/3859752312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9de00d0640 msgr2=0x7f9de0152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.318+0000 7f9ddf400640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9de00d0640 0x7f9de0152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.318+0000 7f9ddf400640 1 -- 172.21.15.67:0/3859752312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de0152a70 msgr2=0x7f9de014ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.319+0000 7f9ddf400640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de0152a70 0x7f9de014ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.319+0000 7f9ddf400640 1 -- 172.21.15.67:0/3859752312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de014a240 con 0x7f9de00b32c0 2024-05-30T16:57:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.319+0000 7f9ddea00640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9de00d0640 0x7f9de0152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:57:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.319+0000 7f9ddf400640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de00b32c0 0x7f9de01f6e50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000c430 tx=0x7f9dd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.320+0000 7f9dd7400640 1 -- 172.21.15.67:0/3859752312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0017070 con 0x7f9de00b32c0 2024-05-30T16:57:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.320+0000 7f9dd7400640 1 -- 172.21.15.67:0/3859752312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd0005ce0 con 0x7f9de00b32c0 2024-05-30T16:57:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.320+0000 7f9dd7400640 1 -- 172.21.15.67:0/3859752312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00072d0 con 0x7f9de00b32c0 2024-05-30T16:57:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.320+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de014a520 con 0x7f9de00b32c0 2024-05-30T16:57:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.320+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de014aae0 con 0x7f9de00b32c0 2024-05-30T16:57:57.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.322+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de00689e0 con 0x7f9de00b32c0 2024-05-30T16:57:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.325+0000 7f9dd7400640 1 -- 172.21.15.67:0/3859752312 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 31) ==== 92672+0+0 (secure 0 0 0) 0x7f9dd001a070 con 0x7f9de00b32c0 2024-05-30T16:57:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.326+0000 7f9dd7400640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9dc006db40 0x7f9dc0070000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.326+0000 7f9dd7400640 1 -- 172.21.15.67:0/3859752312 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f9dd0089de0 con 0x7f9de00b32c0 2024-05-30T16:57:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.326+0000 7f9ddea00640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9dc006db40 0x7f9dc0070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.327+0000 7f9ddea00640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9dc006db40 0x7f9dc0070000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc02a150 tx=0x7f9dcc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.334+0000 7f9dd7400640 1 -- 172.21.15.67:0/3859752312 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9dd00592e0 con 0x7f9de00b32c0 2024-05-30T16:57:57.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.656+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f9de00b4e50 con 0x7f9dc006db40 2024-05-30T16:57:57.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.691+0000 7f9dd7400640 1 -- 172.21.15.67:0/3859752312 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9de00b4e50 con 0x7f9dc006db40 2024-05-30T16:57:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9dc006db40 msgr2=0x7f9dc0070000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9dc006db40 0x7f9dc0070000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc02a150 tx=0x7f9dcc02e000 comp rx=0 tx=0).stop 2024-05-30T16:57:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de00b32c0 msgr2=0x7f9de01f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de00b32c0 0x7f9de01f6e50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000c430 tx=0x7f9dd000c8f0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 shutdown_connections 2024-05-30T16:57:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9dc006db40 0x7f9dc0070000 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de0152a70 0x7f9de014ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9de00d0640 0x7f9de0152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 --2- 172.21.15.67:0/3859752312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9de00b32c0 0x7f9de01f6e50 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.694+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 >> 172.21.15.67:0/3859752312 conn(0x7f9de006bd20 msgr2=0x7f9de015dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.695+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 shutdown_connections 2024-05-30T16:57:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.695+0000 7f9de6000640 1 -- 172.21.15.67:0/3859752312 wait complete. 2024-05-30T16:57:57.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92286 2024-05-30T16:57:57.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.901+0000 7fd195200640 1 Processor -- start 2024-05-30T16:57:57.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.901+0000 7fd195200640 1 -- start start 2024-05-30T16:57:57.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.903+0000 7fd195200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd19015e5a0 0x7fd1900d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.903+0000 7fd195200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cfb40 0x7fd1900d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.903+0000 7fd195200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1900d3500 con 0x7fd1900cffa0 2024-05-30T16:57:57.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.903+0000 7fd195200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd1900d2b30 con 0x7fd19015e5a0 2024-05-30T16:57:57.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.903+0000 7fd195200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd1900d2200 con 0x7fd1900cfb40 2024-05-30T16:57:57.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.903+0000 7fd18f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd19015e5a0 0x7fd1900d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.903+0000 7fd18f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd19015e5a0 0x7fd1900d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47114/0 (socket says 172.21.15.67:47114) 2024-05-30T16:57:57.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.903+0000 7fd18f400640 1 -- 172.21.15.67:0/2241946468 learned_addr learned my addr 172.21.15.67:0/2241946468 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:57.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.904+0000 7fd18f400640 1 -- 172.21.15.67:0/2241946468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cfb40 msgr2=0x7fd1900d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:57.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.904+0000 7fd18ea00640 1 --2- 172.21.15.67:0/2241946468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cfb40 0x7fd1900d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.904+0000 7fd18f400640 1 --2- 172.21.15.67:0/2241946468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cfb40 0x7fd1900d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.904+0000 7fd18e000640 1 -- 172.21.15.67:0/2241946468 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819726760 0 0) 0x7fd1900d3500 con 0x7fd1900cffa0 2024-05-30T16:57:57.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.904+0000 7fd18f400640 1 -- 172.21.15.67:0/2241946468 >> v1:172.21.15.67:6789/0 conn(0x7fd1900cffa0 legacy=0x7fd19006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.904+0000 7fd18f400640 1 -- 172.21.15.67:0/2241946468 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1900d24e0 con 0x7fd19015e5a0 2024-05-30T16:57:57.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.904+0000 7fd18ea00640 1 --2- 172.21.15.67:0/2241946468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cfb40 0x7fd1900d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:57.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.906+0000 7fd18f400640 1 --2- 172.21.15.67:0/2241946468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 0x7fd1900d2fc0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd180009870 tx=0x7fd18002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63efec26a96fa72 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:57.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.906+0000 7fd18e000640 1 -- 172.21.15.67:0/2241946468 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180037070 con 0x7fd19015e5a0 2024-05-30T16:57:57.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.906+0000 7fd18e000640 1 -- 172.21.15.67:0/2241946468 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd180032440 con 0x7fd19015e5a0 2024-05-30T16:57:57.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.906+0000 7fd18e000640 1 -- 172.21.15.67:0/2241946468 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180032810 con 0x7fd19015e5a0 2024-05-30T16:57:57.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.907+0000 7fd195200640 1 -- 172.21.15.67:0/2241946468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 msgr2=0x7fd1900d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.907+0000 7fd195200640 1 --2- 172.21.15.67:0/2241946468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 0x7fd1900d2fc0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd180009870 tx=0x7fd18002a250 comp rx=0 tx=0).stop 2024-05-30T16:57:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.907+0000 7fd195200640 1 -- 172.21.15.67:0/2241946468 shutdown_connections 2024-05-30T16:57:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.907+0000 7fd195200640 1 --2- 172.21.15.67:0/2241946468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cfb40 0x7fd1900d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.907+0000 7fd195200640 1 --2- 172.21.15.67:0/2241946468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 0x7fd1900d2fc0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.907+0000 7fd195200640 1 -- 172.21.15.67:0/2241946468 >> 172.21.15.67:0/2241946468 conn(0x7fd19008bfc0 msgr2=0x7fd1900d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.907+0000 7fd195200640 1 -- 172.21.15.67:0/2241946468 shutdown_connections 2024-05-30T16:57:57.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.907+0000 7fd195200640 1 -- 172.21.15.67:0/2241946468 wait complete. 2024-05-30T16:57:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.908+0000 7fd195200640 1 Processor -- start 2024-05-30T16:57:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.909+0000 7fd195200640 1 -- start start 2024-05-30T16:57:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.909+0000 7fd195200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1900cfb40 0x7fd1900b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd195200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cffa0 0x7fd1900b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd195200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 0x7fd19015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd195200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd19015eb00 con 0x7fd19015e5a0 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd195200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd19015e980 con 0x7fd1900cfb40 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd195200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd1900d5220 con 0x7fd1900cffa0 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 0x7fd19015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 0x7fd19015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47130/0 (socket says 172.21.15.67:47130) 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18fe00640 1 -- 172.21.15.67:0/3932408464 learned_addr learned my addr 172.21.15.67:0/3932408464 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18f400640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1900cfb40 0x7fd1900b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18fe00640 1 -- 172.21.15.67:0/3932408464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cffa0 msgr2=0x7fd1900b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18fe00640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cffa0 0x7fd1900b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18fe00640 1 -- 172.21.15.67:0/3932408464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1900cfb40 msgr2=0x7fd1900b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18fe00640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1900cfb40 0x7fd1900b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18fe00640 1 -- 172.21.15.67:0/3932408464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1900b7080 con 0x7fd19015e5a0 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.910+0000 7fd18f400640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1900cfb40 0x7fd1900b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:57.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.911+0000 7fd18fe00640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 0x7fd19015a4b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c730 tx=0x7fd18800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.911+0000 7fd187400640 1 -- 172.21.15.67:0/3932408464 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188003da0 con 0x7fd19015e5a0 2024-05-30T16:57:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.911+0000 7fd187400640 1 -- 172.21.15.67:0/3932408464 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd18800ce40 con 0x7fd19015e5a0 2024-05-30T16:57:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.911+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1900b5460 con 0x7fd19015e5a0 2024-05-30T16:57:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.911+0000 7fd187400640 1 -- 172.21.15.67:0/3932408464 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1880052b0 con 0x7fd19015e5a0 2024-05-30T16:57:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.911+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1900b7620 con 0x7fd19015e5a0 2024-05-30T16:57:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.912+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1900689e0 con 0x7fd19015e5a0 2024-05-30T16:57:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.914+0000 7fd187400640 1 -- 172.21.15.67:0/3932408464 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 31) ==== 92672+0+0 (secure 0 0 0) 0x7fd188005450 con 0x7fd19015e5a0 2024-05-30T16:57:57.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.915+0000 7fd187400640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd17006daf0 0x7fd17006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:57.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.915+0000 7fd187400640 1 -- 172.21.15.67:0/3932408464 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fd18808aad0 con 0x7fd19015e5a0 2024-05-30T16:57:57.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.916+0000 7fd18f400640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd17006daf0 0x7fd17006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.916+0000 7fd18f400640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd17006daf0 0x7fd17006ffb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd180034000 tx=0x7fd18002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:57.920+0000 7fd187400640 1 -- 172.21.15.67:0/3932408464 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd18805a560 con 0x7fd19015e5a0 2024-05-30T16:57:58.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.273+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92286", "target": ["mon-mgr", ""]}) -- 0x7fd1900d1d30 con 0x7fd17006daf0 2024-05-30T16:57:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.311+0000 7fd187400640 1 -- 172.21.15.67:0/3932408464 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd1900d1d30 con 0x7fd17006daf0 2024-05-30T16:57:58.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.315+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd17006daf0 msgr2=0x7fd17006ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:58.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.315+0000 7fd195200640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd17006daf0 0x7fd17006ffb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd180034000 tx=0x7fd18002e000 comp rx=0 tx=0).stop 2024-05-30T16:57:58.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.315+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 msgr2=0x7fd19015a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:58.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.315+0000 7fd195200640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 0x7fd19015a4b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c730 tx=0x7fd18800cbf0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.316+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 shutdown_connections 2024-05-30T16:57:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.316+0000 7fd195200640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd17006daf0 0x7fd17006ffb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.316+0000 7fd195200640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd19015e5a0 0x7fd19015a4b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.316+0000 7fd195200640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1900cffa0 0x7fd1900b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.316+0000 7fd195200640 1 --2- 172.21.15.67:0/3932408464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1900cfb40 0x7fd1900b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.316+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 >> 172.21.15.67:0/3932408464 conn(0x7fd19008bfc0 msgr2=0x7fd1900b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.316+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 shutdown_connections 2024-05-30T16:57:58.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.317+0000 7fd195200640 1 -- 172.21.15.67:0/3932408464 wait complete. 2024-05-30T16:57:58.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922813 2024-05-30T16:57:58.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.618+0000 7f8145400640 1 Processor -- start 2024-05-30T16:57:58.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.618+0000 7f8145400640 1 -- start start 2024-05-30T16:57:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.620+0000 7f8145400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814009b140 0x7f81400d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.620+0000 7f8145400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f814009b770 0x7f81400b7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.620+0000 7f8145400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81400cd240 con 0x7f814015c7a0 2024-05-30T16:57:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.620+0000 7f8145400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f81400cde50 con 0x7f814009b770 2024-05-30T16:57:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.620+0000 7f8145400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f81400b6360 con 0x7f814009b140 2024-05-30T16:57:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.620+0000 7f813f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f814009b770 0x7f81400b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.621+0000 7f813f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f814009b770 0x7f81400b7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:39388/0 (socket says 172.21.15.67:39388) 2024-05-30T16:57:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.621+0000 7f813f200640 1 -- 172.21.15.67:0/976533836 learned_addr learned my addr 172.21.15.67:0/976533836 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.621+0000 7f813de00640 1 --2- 172.21.15.67:0/976533836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814009b140 0x7f81400d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.621+0000 7f813de00640 1 -- 172.21.15.67:0/976533836 >> v2:172.21.15.169:3300/0 conn(0x7f814009b770 msgr2=0x7f81400b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.621+0000 7f813de00640 1 --2- 172.21.15.67:0/976533836 >> v2:172.21.15.169:3300/0 conn(0x7f814009b770 0x7f81400b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.621+0000 7f813de00640 1 -- 172.21.15.67:0/976533836 >> v1:172.21.15.169:6789/0 conn(0x7f814015c7a0 legacy=0x7f814009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.621+0000 7f813de00640 1 -- 172.21.15.67:0/976533836 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81400cd520 con 0x7f814009b140 2024-05-30T16:57:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.621+0000 7f813f200640 1 --2- 172.21.15.67:0/976533836 >> v2:172.21.15.169:3300/0 conn(0x7f814009b770 0x7f81400b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:57:58.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.623+0000 7f813de00640 1 --2- 172.21.15.67:0/976533836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814009b140 0x7f81400d0f90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f812c0097a0 tx=0x7f812c02a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=532f7b8757f16798 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.623+0000 7f813d400640 1 -- 172.21.15.67:0/976533836 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c037070 con 0x7f814009b140 2024-05-30T16:57:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.623+0000 7f813d400640 1 -- 172.21.15.67:0/976533836 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812c02ae40 con 0x7f814009b140 2024-05-30T16:57:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.623+0000 7f813d400640 1 -- 172.21.15.67:0/976533836 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c032660 con 0x7f814009b140 2024-05-30T16:57:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.624+0000 7f8145400640 1 -- 172.21.15.67:0/976533836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814009b140 msgr2=0x7f81400d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.624+0000 7f8145400640 1 --2- 172.21.15.67:0/976533836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814009b140 0x7f81400d0f90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f812c0097a0 tx=0x7f812c02a0f0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.624+0000 7f8145400640 1 -- 172.21.15.67:0/976533836 shutdown_connections 2024-05-30T16:57:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.624+0000 7f8145400640 1 --2- 172.21.15.67:0/976533836 >> v2:172.21.15.169:3300/0 conn(0x7f814009b770 0x7f81400b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.624+0000 7f8145400640 1 --2- 172.21.15.67:0/976533836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814009b140 0x7f81400d0f90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.624+0000 7f8145400640 1 -- 172.21.15.67:0/976533836 >> 172.21.15.67:0/976533836 conn(0x7f81400b3ff0 msgr2=0x7f81400b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.625+0000 7f8145400640 1 -- 172.21.15.67:0/976533836 shutdown_connections 2024-05-30T16:57:58.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.625+0000 7f8145400640 1 -- 172.21.15.67:0/976533836 wait complete. 2024-05-30T16:57:58.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.627+0000 7f8145400640 1 Processor -- start 2024-05-30T16:57:58.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.627+0000 7f8145400640 1 -- start start 2024-05-30T16:57:58.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.629+0000 7f8145400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814009b140 0x7f81400cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:58.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.629+0000 7f8145400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814009b770 0x7f81400c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:58.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.629+0000 7f8145400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814015c7a0 0x7f81400c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.629+0000 7f8145400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f814015cb80 con 0x7f814009b140 2024-05-30T16:57:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.629+0000 7f8145400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f81400cdae0 con 0x7f814009b770 2024-05-30T16:57:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.629+0000 7f8145400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f81400ce2e0 con 0x7f814015c7a0 2024-05-30T16:57:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.629+0000 7f813e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814009b140 0x7f81400cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.629+0000 7f813de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814009b770 0x7f81400c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.630+0000 7f813de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814009b770 0x7f81400c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:39404/0 (socket says 172.21.15.67:39404) 2024-05-30T16:57:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.630+0000 7f813de00640 1 -- 172.21.15.67:0/2735724726 learned_addr learned my addr 172.21.15.67:0/2735724726 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.630+0000 7f813e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814009b140 0x7f81400cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47144/0 (socket says 172.21.15.67:47144) 2024-05-30T16:57:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.630+0000 7f813f200640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814015c7a0 0x7f81400c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:58.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.630+0000 7f813f200640 1 -- 172.21.15.67:0/2735724726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814009b770 msgr2=0x7f81400c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:58.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.630+0000 7f813f200640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814009b770 0x7f81400c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.630+0000 7f813f200640 1 -- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814009b140 msgr2=0x7f81400cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:58.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.630+0000 7f813f200640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814009b140 0x7f81400cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:58.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.631+0000 7f813f200640 1 -- 172.21.15.67:0/2735724726 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8140150c40 con 0x7f814015c7a0 2024-05-30T16:57:58.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.631+0000 7f813e800640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814009b140 0x7f81400cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:57:58.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.631+0000 7f813f200640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814015c7a0 0x7f81400c88b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8138007a60 tx=0x7f8138007f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:58.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.631+0000 7f8136a00640 1 -- 172.21.15.67:0/2735724726 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813800dc40 con 0x7f814015c7a0 2024-05-30T16:57:58.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.631+0000 7f8136a00640 1 -- 172.21.15.67:0/2735724726 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813800de00 con 0x7f814015c7a0 2024-05-30T16:57:58.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.631+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8140150f20 con 0x7f814015c7a0 2024-05-30T16:57:58.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.632+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401514e0 con 0x7f814015c7a0 2024-05-30T16:57:58.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.632+0000 7f8136a00640 1 -- 172.21.15.67:0/2735724726 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138015640 con 0x7f814015c7a0 2024-05-30T16:57:58.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.633+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8118005290 con 0x7f814015c7a0 2024-05-30T16:57:58.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.636+0000 7f8136a00640 1 -- 172.21.15.67:0/2735724726 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 31) ==== 92672+0+0 (secure 0 0 0) 0x7f81380157e0 con 0x7f814015c7a0 2024-05-30T16:57:58.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.637+0000 7f8136a00640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f812006daf0 0x7f812006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:58.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.638+0000 7f8136a00640 1 -- 172.21.15.67:0/2735724726 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f813808a9a0 con 0x7f814015c7a0 2024-05-30T16:57:58.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.638+0000 7f813e800640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f812006daf0 0x7f812006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:58.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.639+0000 7f813e800640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f812006daf0 0x7f812006ffb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f81400c9030 tx=0x7f8130009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:58.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.642+0000 7f8136a00640 1 -- 172.21.15.67:0/2735724726 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8138010070 con 0x7f814015c7a0 2024-05-30T16:57:58.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:58.978+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922813", "target": ["mon-mgr", ""]}) -- 0x7f8118000bc0 con 0x7f812006daf0 2024-05-30T16:57:59.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.005+0000 7f8136a00640 1 -- 172.21.15.67:0/2735724726 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8118000bc0 con 0x7f812006daf0 2024-05-30T16:57:59.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f812006daf0 msgr2=0x7f812006ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f812006daf0 0x7f812006ffb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f81400c9030 tx=0x7f8130009040 comp rx=0 tx=0).stop 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814015c7a0 msgr2=0x7f81400c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814015c7a0 0x7f81400c88b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8138007a60 tx=0x7f8138007f20 comp rx=0 tx=0).stop 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 shutdown_connections 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f812006daf0 0x7f812006ffb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f814015c7a0 0x7f81400c88b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814009b770 0x7f81400c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 --2- 172.21.15.67:0/2735724726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814009b140 0x7f81400cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 >> 172.21.15.67:0/2735724726 conn(0x7f81400b3ff0 msgr2=0x7f81400d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 shutdown_connections 2024-05-30T16:57:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.008+0000 7f8145400640 1 -- 172.21.15.67:0/2735724726 wait complete. 2024-05-30T16:57:59.031 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922819 2024-05-30T16:57:59.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.210+0000 7f2ec4200640 1 Processor -- start 2024-05-30T16:57:59.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.210+0000 7f2ec4200640 1 -- start start 2024-05-30T16:57:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ec4200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2ebc0d34e0 0x7f2ebc0d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ec4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0b5570 0x7f2ebc0b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ec4200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ebc0d5500 con 0x7f2ebc1622b0 2024-05-30T16:57:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ec4200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2ebc0b4f50 con 0x7f2ebc0d34e0 2024-05-30T16:57:59.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ec4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ebc0d2720 con 0x7f2ebc0b5570 2024-05-30T16:57:59.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ec1800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2ebc1622b0 0x7f2ebc0d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33704/0 (socket says 172.21.15.67:33704) 2024-05-30T16:57:59.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ec1800640 1 -- 172.21.15.67:0/2646539858 learned_addr learned my addr 172.21.15.67:0/2646539858 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:59.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 410251822 0 0) 0x7f2ebc0d5500 con 0x7f2ebc1622b0 2024-05-30T16:57:59.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ea4003520 con 0x7f2ebc1622b0 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.212+0000 7f2ec2200640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0b5570 0x7f2ebc0b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ec0e00640 1 --2- 172.21.15.67:0/2646539858 >> v2:172.21.15.169:3300/0 conn(0x7f2ebc0d34e0 0x7f2ebc0d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3548689139 0 0) 0x7f2ea4003520 con 0x7f2ebc1622b0 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0b5570 msgr2=0x7f2ebc0b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0b5570 0x7f2ebc0b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 >> v2:172.21.15.169:3300/0 conn(0x7f2ebc0d34e0 msgr2=0x7f2ebc0d4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 --2- 172.21.15.67:0/2646539858 >> v2:172.21.15.169:3300/0 conn(0x7f2ebc0d34e0 0x7f2ebc0d4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc0d2a00 con 0x7f2ebc1622b0 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f2eb0001670 con 0x7f2ebc1622b0 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 >> v1:172.21.15.67:6789/0 conn(0x7f2ebc1622b0 legacy=0x7f2ebc0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ea40039b0 0x7f2ea4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea4004800 0x7f2ea4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0b5570 0x7f2ea400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc0b4f50 con 0x7f2ea40039b0 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc0d2720 con 0x7f2ea4004800 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea40055a0 con 0x7f2ebc0b5570 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.213+0000 7f2ec1800640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ea40039b0 0x7f2ea4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.214+0000 7f2ec2200640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0b5570 0x7f2ea400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.214+0000 7f2ec1800640 1 -- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0b5570 msgr2=0x7f2ea400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.214+0000 7f2ec1800640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc0b5570 0x7f2ea400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.214+0000 7f2ec1800640 1 -- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea4004800 msgr2=0x7f2ea4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.214+0000 7f2ec1800640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea4004800 0x7f2ea4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.214+0000 7f2ec1800640 1 -- 172.21.15.67:0/2646539858 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea4003520 con 0x7f2ea40039b0 2024-05-30T16:57:59.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.215+0000 7f2ec1800640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ea40039b0 0x7f2ea4003dd0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c190 tx=0x7f2ea4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c683f6dee97ee2d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:59.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.215+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0007c80 con 0x7f2ea40039b0 2024-05-30T16:57:59.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.215+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb0005ad0 con 0x7f2ea40039b0 2024-05-30T16:57:59.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.215+0000 7f2ebbe00640 1 -- 172.21.15.67:0/2646539858 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0007070 con 0x7f2ea40039b0 2024-05-30T16:57:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.215+0000 7f2ec4200640 1 -- 172.21.15.67:0/2646539858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ea40039b0 msgr2=0x7f2ea4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.215+0000 7f2ec4200640 1 --2- 172.21.15.67:0/2646539858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ea40039b0 0x7f2ea4003dd0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c190 tx=0x7f2ea4005830 comp rx=0 tx=0).stop 2024-05-30T16:57:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.216+0000 7f2ec2200640 1 -- 172.21.15.67:0/2646539858 reap_dead start 2024-05-30T16:57:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.216+0000 7f2ec4200640 1 -- 172.21.15.67:0/2646539858 shutdown_connections 2024-05-30T16:57:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.216+0000 7f2ec4200640 1 -- 172.21.15.67:0/2646539858 >> 172.21.15.67:0/2646539858 conn(0x7f2ebc08bfc0 msgr2=0x7f2ebc0d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.216+0000 7f2ec4200640 1 -- 172.21.15.67:0/2646539858 shutdown_connections 2024-05-30T16:57:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.216+0000 7f2ec4200640 1 -- 172.21.15.67:0/2646539858 wait complete. 2024-05-30T16:57:59.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.217+0000 7f2ec4200640 1 Processor -- start 2024-05-30T16:57:59.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.218+0000 7f2ec4200640 1 -- start start 2024-05-30T16:57:59.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.218+0000 7f2ec4200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea40039b0 0x7f2ebc0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec4200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ebc15abf0 0x7f2ebc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc15b070 0x7f2ebc0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec4200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc0b5b80 con 0x7f2ebc15abf0 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec4200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc0b5a00 con 0x7f2ea40039b0 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ebc0b5d00 con 0x7f2ebc15b070 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ebc15abf0 0x7f2ebc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ebc15abf0 0x7f2ebc0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47160/0 (socket says 172.21.15.67:47160) 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec0e00640 1 -- 172.21.15.67:0/1754172521 learned_addr learned my addr 172.21.15.67:0/1754172521 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec1800640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea40039b0 0x7f2ebc0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec2200640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc15b070 0x7f2ebc0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec0e00640 1 -- 172.21.15.67:0/1754172521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc15b070 msgr2=0x7f2ebc0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec0e00640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc15b070 0x7f2ebc0b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec0e00640 1 -- 172.21.15.67:0/1754172521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea40039b0 msgr2=0x7f2ebc0c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec0e00640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea40039b0 0x7f2ebc0c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.219+0000 7f2ec0e00640 1 -- 172.21.15.67:0/1754172521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc0c8c00 con 0x7f2ebc15abf0 2024-05-30T16:57:59.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.220+0000 7f2ec0e00640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ebc15abf0 0x7f2ebc0cd300 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c7f0 tx=0x7f2ea800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.220+0000 7f2eb9600640 1 -- 172.21.15.67:0/1754172521 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8004230 con 0x7f2ebc15abf0 2024-05-30T16:57:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.220+0000 7f2eb9600640 1 -- 172.21.15.67:0/1754172521 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea80043d0 con 0x7f2ebc15abf0 2024-05-30T16:57:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.220+0000 7f2eb9600640 1 -- 172.21.15.67:0/1754172521 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8007860 con 0x7f2ebc15abf0 2024-05-30T16:57:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.220+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ebc0b7310 con 0x7f2ebc15abf0 2024-05-30T16:57:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.220+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ebc0b78d0 con 0x7f2ebc15abf0 2024-05-30T16:57:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.221+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ebc0689e0 con 0x7f2ebc15abf0 2024-05-30T16:57:59.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.222+0000 7f2eb9600640 1 -- 172.21.15.67:0/1754172521 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 31) ==== 92672+0+0 (secure 0 0 0) 0x7f2ea8007c10 con 0x7f2ebc15abf0 2024-05-30T16:57:59.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.223+0000 7f2eb9600640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e9c06daf0 0x7f2e9c06ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.224+0000 7f2eb9600640 1 -- 172.21.15.67:0/1754172521 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f2ea808acc0 con 0x7f2ebc15abf0 2024-05-30T16:57:59.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.224+0000 7f2ec1800640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e9c06daf0 0x7f2e9c06ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.224+0000 7f2ec1800640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e9c06daf0 0x7f2e9c06ffb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0032000 tx=0x7f2eb0009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:59.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.228+0000 7f2eb9600640 1 -- 172.21.15.67:0/1754172521 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ea8015020 con 0x7f2ebc15abf0 2024-05-30T16:57:59.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.459+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922819", "target": ["mon-mgr", ""]}) -- 0x7f2ebc0d4790 con 0x7f2e9c06daf0 2024-05-30T16:57:59.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.492+0000 7f2eb9600640 1 -- 172.21.15.67:0/1754172521 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ebc0d4790 con 0x7f2e9c06daf0 2024-05-30T16:57:59.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.496+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e9c06daf0 msgr2=0x7f2e9c06ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.496+0000 7f2ec4200640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e9c06daf0 0x7f2e9c06ffb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0032000 tx=0x7f2eb0009030 comp rx=0 tx=0).stop 2024-05-30T16:57:59.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.496+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ebc15abf0 msgr2=0x7f2ebc0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.496+0000 7f2ec4200640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ebc15abf0 0x7f2ebc0cd300 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c7f0 tx=0x7f2ea800ccb0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.496+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 shutdown_connections 2024-05-30T16:57:59.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.497+0000 7f2ec4200640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2e9c06daf0 0x7f2e9c06ffb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.497+0000 7f2ec4200640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc15b070 0x7f2ebc0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.497+0000 7f2ec4200640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ebc15abf0 0x7f2ebc0cd300 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.497+0000 7f2ec4200640 1 --2- 172.21.15.67:0/1754172521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea40039b0 0x7f2ebc0c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.497+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 >> 172.21.15.67:0/1754172521 conn(0x7f2ebc08bfc0 msgr2=0x7f2ebc06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:59.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.497+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 shutdown_connections 2024-05-30T16:57:59.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.497+0000 7f2ec4200640 1 -- 172.21.15.67:0/1754172521 wait complete. 2024-05-30T16:57:59.517 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92284 2024-05-30T16:57:59.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.710+0000 7f3bb2400640 1 Processor -- start 2024-05-30T16:57:59.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.711+0000 7f3bb2400640 1 -- start start 2024-05-30T16:57:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.712+0000 7f3bb2400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3bac15e5e0 0x7f3bac162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.712+0000 7f3bb2400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0d3270 0x7f3bac0d4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.712+0000 7f3bb2400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac0cf620 con 0x7f3bac0d03e0 2024-05-30T16:57:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.712+0000 7f3bb2400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3bac0d53e0 con 0x7f3bac15e5e0 2024-05-30T16:57:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.712+0000 7f3bb2400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0cf900 con 0x7f3bac0d3270 2024-05-30T16:57:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3babe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3bac0d03e0 0x7f3bac0d2b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33710/0 (socket says 172.21.15.67:33710) 2024-05-30T16:57:59.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3babe00640 1 -- 172.21.15.67:0/3043771282 learned_addr learned my addr 172.21.15.67:0/3043771282 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:57:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0d3270 0x7f3bac0d4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3bb1000640 1 --2- 172.21.15.67:0/3043771282 >> v2:172.21.15.169:3300/0 conn(0x7f3bac15e5e0 0x7f3bac162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 604245550 0 0) 0x7f3bac0cf620 con 0x7f3bac0d03e0 2024-05-30T16:57:59.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b94003520 con 0x7f3bac0d03e0 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1627748994 0 0) 0x7f3b94003520 con 0x7f3bac0d03e0 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0d3270 msgr2=0x7f3bac0d4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3bab400640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0d3270 0x7f3bac0d4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 >> v2:172.21.15.169:3300/0 conn(0x7f3bac15e5e0 msgr2=0x7f3bac162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.713+0000 7f3bab400640 1 --2- 172.21.15.67:0/3043771282 >> v2:172.21.15.169:3300/0 conn(0x7f3bac15e5e0 0x7f3bac162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac0cfbe0 con 0x7f3bac0d03e0 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f3b98001670 con 0x7f3bac0d03e0 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bb1000640 1 --2- 172.21.15.67:0/3043771282 >> v2:172.21.15.169:3300/0 conn(0x7f3bac15e5e0 0x7f3bac162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 >> v1:172.21.15.67:6789/0 conn(0x7f3bac0d03e0 legacy=0x7f3bac0d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bab400640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0d3270 0x7f3b940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bab400640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b94004030 0x7f3b94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bab400640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0cf900 con 0x7f3b94004030 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9400baa0 con 0x7f3b94004d40 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9400bca0 con 0x7f3bac0d3270 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3babe00640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b94004030 0x7f3b94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3bb1000640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0d3270 0x7f3b940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3babe00640 1 -- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0d3270 msgr2=0x7f3b940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3babe00640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac0d3270 0x7f3b940039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3babe00640 1 -- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004d40 msgr2=0x7f3b940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3babe00640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.714+0000 7f3babe00640 1 -- 172.21.15.67:0/3043771282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b94003520 con 0x7f3b94004030 2024-05-30T16:57:59.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.715+0000 7f3babe00640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b94004030 0x7f3b94004800 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800c160 tx=0x7f3b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d76c3402a49c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:59.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98007c60 con 0x7f3b94004030 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b98005af0 con 0x7f3b94004030 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bab400640 1 -- 172.21.15.67:0/3043771282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98007070 con 0x7f3b94004030 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bb2400640 1 -- 172.21.15.67:0/3043771282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b94004030 msgr2=0x7f3b94004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bb2400640 1 --2- 172.21.15.67:0/3043771282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b94004030 0x7f3b94004800 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800c160 tx=0x7f3b94005830 comp rx=0 tx=0).stop 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bb1a00640 1 -- 172.21.15.67:0/3043771282 reap_dead start 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bb2400640 1 -- 172.21.15.67:0/3043771282 shutdown_connections 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bb2400640 1 -- 172.21.15.67:0/3043771282 >> 172.21.15.67:0/3043771282 conn(0x7f3bac08bfc0 msgr2=0x7f3bac0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bb2400640 1 -- 172.21.15.67:0/3043771282 shutdown_connections 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.716+0000 7f3bb2400640 1 -- 172.21.15.67:0/3043771282 wait complete. 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.718+0000 7f3bb2400640 1 Processor -- start 2024-05-30T16:57:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.718+0000 7f3bb2400640 1 -- start start 2024-05-30T16:57:59.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.719+0000 7f3bb2400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004030 0x7f3bac0b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.719+0000 7f3bb2400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac152740 0x7f3bac1e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.719+0000 7f3bb2400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac152bc0 0x7f3bac1e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.719+0000 7f3bb2400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac15e9c0 con 0x7f3bac152740 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.719+0000 7f3bb2400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0cf2d0 con 0x7f3b94004030 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.719+0000 7f3bb2400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac15eb40 con 0x7f3bac152bc0 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.719+0000 7f3babe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac152740 0x7f3bac1e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.719+0000 7f3babe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac152740 0x7f3bac1e8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47176/0 (socket says 172.21.15.67:47176) 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.719+0000 7f3babe00640 1 -- 172.21.15.67:0/1958377949 learned_addr learned my addr 172.21.15.67:0/1958377949 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3bb1000640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004030 0x7f3bac0b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac152bc0 0x7f3bac1e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3babe00640 1 -- 172.21.15.67:0/1958377949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac152bc0 msgr2=0x7f3bac1e8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3babe00640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac152bc0 0x7f3bac1e8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3babe00640 1 -- 172.21.15.67:0/1958377949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004030 msgr2=0x7f3bac0b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3babe00640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004030 0x7f3bac0b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3babe00640 1 -- 172.21.15.67:0/1958377949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac160f40 con 0x7f3bac152740 2024-05-30T16:57:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3bb1a00640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac152bc0 0x7f3bac1e8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:57:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3babe00640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac152740 0x7f3bac1e8630 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f3b9802ad30 tx=0x7f3bac0cf7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.720+0000 7f3ba8c00640 1 -- 172.21.15.67:0/1958377949 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98041020 con 0x7f3bac152740 2024-05-30T16:57:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.721+0000 7f3ba8c00640 1 -- 172.21.15.67:0/1958377949 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b98001430 con 0x7f3bac152740 2024-05-30T16:57:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.721+0000 7f3ba8c00640 1 -- 172.21.15.67:0/1958377949 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98007070 con 0x7f3bac152740 2024-05-30T16:57:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.721+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac161220 con 0x7f3bac152740 2024-05-30T16:57:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.721+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac1617e0 con 0x7f3bac152740 2024-05-30T16:57:59.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.722+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac0690f0 con 0x7f3bac152740 2024-05-30T16:57:59.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.723+0000 7f3ba8c00640 1 -- 172.21.15.67:0/1958377949 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 31) ==== 92672+0+0 (secure 0 0 0) 0x7f3b98044020 con 0x7f3bac152740 2024-05-30T16:57:59.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.723+0000 7f3ba8c00640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3b8c06daf0 0x7f3b8c06ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:57:59.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.724+0000 7f3ba8c00640 1 -- 172.21.15.67:0/1958377949 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f3b980aaad0 con 0x7f3bac152740 2024-05-30T16:57:59.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.724+0000 7f3bb1000640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3b8c06daf0 0x7f3b8c06ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:57:59.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.724+0000 7f3bb1000640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3b8c06daf0 0x7f3b8c06ffb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006fd0 tx=0x7f3ba0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:57:59.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.728+0000 7f3ba8c00640 1 -- 172.21.15.67:0/1958377949 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b9807a680 con 0x7f3bac152740 2024-05-30T16:57:59.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.951+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92284", "target": ["mon-mgr", ""]}) -- 0x7f3bac160cf0 con 0x7f3b8c06daf0 2024-05-30T16:57:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.986+0000 7f3ba8c00640 1 -- 172.21.15.67:0/1958377949 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3bac160cf0 con 0x7f3b8c06daf0 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3b8c06daf0 msgr2=0x7f3b8c06ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3b8c06daf0 0x7f3b8c06ffb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006fd0 tx=0x7f3ba0008040 comp rx=0 tx=0).stop 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac152740 msgr2=0x7f3bac1e8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac152740 0x7f3bac1e8630 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f3b9802ad30 tx=0x7f3bac0cf7f0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 shutdown_connections 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3b8c06daf0 0x7f3b8c06ffb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bac152bc0 0x7f3bac1e8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bac152740 0x7f3bac1e8630 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 --2- 172.21.15.67:0/1958377949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b94004030 0x7f3bac0b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:57:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 >> 172.21.15.67:0/1958377949 conn(0x7f3bac08bfc0 msgr2=0x7f3bac0b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:57:59.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 shutdown_connections 2024-05-30T16:57:59.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:57:59.988+0000 7f3bb2400640 1 -- 172.21.15.67:0/1958377949 wait complete. 2024-05-30T16:58:00.010 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922820 2024-05-30T16:58:00.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.246+0000 7ff4eb600640 1 Processor -- start 2024-05-30T16:58:00.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.246+0000 7ff4eb600640 1 -- start start 2024-05-30T16:58:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.247+0000 7ff4eb600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4e415b720 0x7ff4e415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.247+0000 7ff4eb600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e40aeda0 con 0x7ff4e415c0f0 2024-05-30T16:58:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.247+0000 7ff4eb600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e40af080 con 0x7ff4e415b340 2024-05-30T16:58:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.247+0000 7ff4eb600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e415c510 con 0x7ff4e415b720 2024-05-30T16:58:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.247+0000 7ff4e3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4e415b720 0x7ff4e415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.248+0000 7ff4e3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4e415b720 0x7ff4e415bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47190/0 (socket says 172.21.15.67:47190) 2024-05-30T16:58:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.248+0000 7ff4e3e00640 1 -- 172.21.15.67:0/2500106315 learned_addr learned my addr 172.21.15.67:0/2500106315 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:00.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.248+0000 7ff4e9600640 1 --1- 172.21.15.67:0/2500106315 >> v1:172.21.15.67:6789/0 conn(0x7ff4e415c0f0 0x7ff4e40b3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33718/0 (socket says 172.21.15.67:33718) 2024-05-30T16:58:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.248+0000 7ff4e3e00640 1 -- 172.21.15.67:0/2500106315 >> v1:172.21.15.169:6789/0 conn(0x7ff4e415b340 legacy=0x7ff4e406f550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:58:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.248+0000 7ff4e3e00640 1 -- 172.21.15.67:0/2500106315 >> v1:172.21.15.67:6789/0 conn(0x7ff4e415c0f0 legacy=0x7ff4e40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.248+0000 7ff4e3e00640 1 -- 172.21.15.67:0/2500106315 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e40af360 con 0x7ff4e415b720 2024-05-30T16:58:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.248+0000 7ff4e3400640 1 -- 172.21.15.67:0/2500106315 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 318050167 0 0) 0x7ff4e40aeda0 con 0x7ff4e415c0f0 2024-05-30T16:58:00.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.249+0000 7ff4e3e00640 1 --2- 172.21.15.67:0/2500106315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b720 0x7ff4e415bbb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009870 tx=0x7ff4cc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd94beed6ab588ce server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:00.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.249+0000 7ff4e3400640 1 -- 172.21.15.67:0/2500106315 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc037070 con 0x7ff4e415b720 2024-05-30T16:58:00.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.249+0000 7ff4e3400640 1 -- 172.21.15.67:0/2500106315 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc032440 con 0x7ff4e415b720 2024-05-30T16:58:00.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.249+0000 7ff4e3400640 1 -- 172.21.15.67:0/2500106315 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc032810 con 0x7ff4e415b720 2024-05-30T16:58:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.250+0000 7ff4eb600640 1 -- 172.21.15.67:0/2500106315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b720 msgr2=0x7ff4e415bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.250+0000 7ff4eb600640 1 --2- 172.21.15.67:0/2500106315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b720 0x7ff4e415bbb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009870 tx=0x7ff4cc02a2a0 comp rx=0 tx=0).stop 2024-05-30T16:58:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.250+0000 7ff4eb600640 1 -- 172.21.15.67:0/2500106315 shutdown_connections 2024-05-30T16:58:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.250+0000 7ff4eb600640 1 --2- 172.21.15.67:0/2500106315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b720 0x7ff4e415bbb0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.250+0000 7ff4eb600640 1 -- 172.21.15.67:0/2500106315 >> 172.21.15.67:0/2500106315 conn(0x7ff4e40b1e50 msgr2=0x7ff4e40b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.250+0000 7ff4eb600640 1 -- 172.21.15.67:0/2500106315 shutdown_connections 2024-05-30T16:58:00.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.250+0000 7ff4eb600640 1 -- 172.21.15.67:0/2500106315 wait complete. 2024-05-30T16:58:00.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.252+0000 7ff4eb600640 1 Processor -- start 2024-05-30T16:58:00.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.252+0000 7ff4eb600640 1 -- start start 2024-05-30T16:58:00.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4eb600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b340 0x7ff4e41590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:00.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4eb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e415b720 0x7ff4e40cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:00.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4eb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e415c0f0 0x7ff4e40b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4eb600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40aea30 con 0x7ff4e415b340 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4eb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40af9b0 con 0x7ff4e415c0f0 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4eb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e415c8c0 con 0x7ff4e415b720 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4e8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b340 0x7ff4e41590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4e8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b340 0x7ff4e41590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47192/0 (socket says 172.21.15.67:47192) 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4e8c00640 1 -- 172.21.15.67:0/332844354 learned_addr learned my addr 172.21.15.67:0/332844354 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.253+0000 7ff4e3e00640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e415b720 0x7ff4e40cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.254+0000 7ff4e9600640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e415c0f0 0x7ff4e40b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.254+0000 7ff4e8c00640 1 -- 172.21.15.67:0/332844354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e415b720 msgr2=0x7ff4e40cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.254+0000 7ff4e8c00640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e415b720 0x7ff4e40cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.254+0000 7ff4e8c00640 1 -- 172.21.15.67:0/332844354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e415c0f0 msgr2=0x7ff4e40b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.254+0000 7ff4e8c00640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e415c0f0 0x7ff4e40b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.254+0000 7ff4e8c00640 1 -- 172.21.15.67:0/332844354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e40ce730 con 0x7ff4e415b340 2024-05-30T16:58:00.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.254+0000 7ff4e8c00640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b340 0x7ff4e41590a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400ba80 tx=0x7ff4d400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:00.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.254+0000 7ff4e0c00640 1 -- 172.21.15.67:0/332844354 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4019070 con 0x7ff4e415b340 2024-05-30T16:58:00.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.255+0000 7ff4e0c00640 1 -- 172.21.15.67:0/332844354 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d400cc90 con 0x7ff4e415b340 2024-05-30T16:58:00.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.255+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e40cea10 con 0x7ff4e415b340 2024-05-30T16:58:00.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.255+0000 7ff4e0c00640 1 -- 172.21.15.67:0/332844354 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4014680 con 0x7ff4e415b340 2024-05-30T16:58:00.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.255+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e40cefd0 con 0x7ff4e415b340 2024-05-30T16:58:00.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.256+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e40689e0 con 0x7ff4e415b340 2024-05-30T16:58:00.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.257+0000 7ff4e0c00640 1 -- 172.21.15.67:0/332844354 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 31) ==== 92672+0+0 (secure 0 0 0) 0x7ff4d401c070 con 0x7ff4e415b340 2024-05-30T16:58:00.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.258+0000 7ff4e0c00640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c006daf0 0x7ff4c006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:00.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.258+0000 7ff4e0c00640 1 -- 172.21.15.67:0/332844354 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7ff4d408a4d0 con 0x7ff4e415b340 2024-05-30T16:58:00.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.258+0000 7ff4e3e00640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c006daf0 0x7ff4c006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:00.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.259+0000 7ff4e3e00640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c006daf0 0x7ff4c006ffb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc034000 tx=0x7ff4cc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:00.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.262+0000 7ff4e0c00640 1 -- 172.21.15.67:0/332844354 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4d4059d00 con 0x7ff4e415b340 2024-05-30T16:58:00.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:00.493+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922820", "target": ["mon-mgr", ""]}) -- 0x7ff4e40b4e60 con 0x7ff4c006daf0 2024-05-30T16:58:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.225+0000 7ff4e0c00640 1 -- 172.21.15.67:0/332844354 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff4e40b4e60 con 0x7ff4c006daf0 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c006daf0 msgr2=0x7ff4c006ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c006daf0 0x7ff4c006ffb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc034000 tx=0x7ff4cc02e000 comp rx=0 tx=0).stop 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b340 msgr2=0x7ff4e41590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b340 0x7ff4e41590a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400ba80 tx=0x7ff4d400bf40 comp rx=0 tx=0).stop 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 shutdown_connections 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c006daf0 0x7ff4c006ffb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e415c0f0 0x7ff4e40b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e415b720 0x7ff4e40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 --2- 172.21.15.67:0/332844354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e415b340 0x7ff4e41590a0 secure :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400ba80 tx=0x7ff4d400bf40 comp rx=0 tx=0).stop 2024-05-30T16:58:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.228+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 >> 172.21.15.67:0/332844354 conn(0x7ff4e40b1e50 msgr2=0x7ff4e40b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.229+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 shutdown_connections 2024-05-30T16:58:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.229+0000 7ff4eb600640 1 -- 172.21.15.67:0/332844354 wait complete. 2024-05-30T16:58:02.254 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922814 2024-05-30T16:58:02.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.509+0000 7f1fd3600640 1 Processor -- start 2024-05-30T16:58:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.509+0000 7f1fd3600640 1 -- start start 2024-05-30T16:58:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.511+0000 7f1fd3600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0af5f0 0x7f1fcc0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.511+0000 7f1fd3600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1fcc0af9d0 0x7f1fcc071c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.511+0000 7f1fd3600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fcc0b4100 con 0x7f1fcc0b3c40 2024-05-30T16:58:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.511+0000 7f1fd3600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1fcc0b51a0 con 0x7f1fcc0af9d0 2024-05-30T16:58:02.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.511+0000 7f1fd3600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc0b43e0 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.511+0000 7f1fd0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0af5f0 0x7f1fcc0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:02.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.512+0000 7f1fd0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0af5f0 0x7f1fcc0b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:54960/0 (socket says 172.21.15.67:54960) 2024-05-30T16:58:02.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.512+0000 7f1fd0c00640 1 -- 172.21.15.67:0/1731838568 learned_addr learned my addr 172.21.15.67:0/1731838568 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:02.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.512+0000 7f1fcbe00640 1 --2- 172.21.15.67:0/1731838568 >> v2:172.21.15.67:3300/0 conn(0x7f1fcc0af9d0 0x7f1fcc071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:02.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.512+0000 7f1fd0c00640 1 -- 172.21.15.67:0/1731838568 >> v2:172.21.15.67:3300/0 conn(0x7f1fcc0af9d0 msgr2=0x7f1fcc071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:02.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.512+0000 7f1fd0c00640 1 --2- 172.21.15.67:0/1731838568 >> v2:172.21.15.67:3300/0 conn(0x7f1fcc0af9d0 0x7f1fcc071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.512+0000 7f1fd0c00640 1 -- 172.21.15.67:0/1731838568 >> v1:172.21.15.169:6789/0 conn(0x7f1fcc0b3c40 legacy=0x7f1fcc15fef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:58:02.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.512+0000 7f1fd0c00640 1 -- 172.21.15.67:0/1731838568 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc0b46c0 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.512+0000 7f1fcbe00640 1 --2- 172.21.15.67:0/1731838568 >> v2:172.21.15.67:3300/0 conn(0x7f1fcc0af9d0 0x7f1fcc071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:58:02.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.513+0000 7f1fd0c00640 1 --2- 172.21.15.67:0/1731838568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0af5f0 0x7f1fcc0b3700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009e50 tx=0x7f1fc402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e50636d261e895f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:02.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.514+0000 7f1fcb400640 1 -- 172.21.15.67:0/1731838568 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4036070 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.514+0000 7f1fcb400640 1 -- 172.21.15.67:0/1731838568 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc4031600 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.514+0000 7f1fcb400640 1 -- 172.21.15.67:0/1731838568 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc40319d0 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.515+0000 7f1fd3600640 1 -- 172.21.15.67:0/1731838568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0af5f0 msgr2=0x7f1fcc0b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:02.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.515+0000 7f1fd3600640 1 --2- 172.21.15.67:0/1731838568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0af5f0 0x7f1fcc0b3700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009e50 tx=0x7f1fc402a6c0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.515+0000 7f1fd3600640 1 -- 172.21.15.67:0/1731838568 shutdown_connections 2024-05-30T16:58:02.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.515+0000 7f1fd3600640 1 --2- 172.21.15.67:0/1731838568 >> v2:172.21.15.67:3300/0 conn(0x7f1fcc0af9d0 0x7f1fcc071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.515+0000 7f1fd3600640 1 --2- 172.21.15.67:0/1731838568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0af5f0 0x7f1fcc0b3700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.515+0000 7f1fd3600640 1 -- 172.21.15.67:0/1731838568 >> 172.21.15.67:0/1731838568 conn(0x7f1fcc0b1e50 msgr2=0x7f1fcc0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:02.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.516+0000 7f1fd3600640 1 -- 172.21.15.67:0/1731838568 shutdown_connections 2024-05-30T16:58:02.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.516+0000 7f1fd3600640 1 -- 172.21.15.67:0/1731838568 wait complete. 2024-05-30T16:58:02.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.518+0000 7f1fd3600640 1 Processor -- start 2024-05-30T16:58:02.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.518+0000 7f1fd3600640 1 -- start start 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd3600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fcc0af5f0 0x7f1fcc0d0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd3600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0b3c40 0x7f1fcc1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd3600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fcc155920 0x7f1fcc165580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd3600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc163440 con 0x7f1fcc155920 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd3600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0b4c80 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd3600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc1635c0 con 0x7f1fcc0b3c40 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fcc0af5f0 0x7f1fcc0d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fcc0af5f0 0x7f1fcc0d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:49928/0 (socket says 172.21.15.67:49928) 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd0c00640 1 -- 172.21.15.67:0/3928488759 learned_addr learned my addr 172.21.15.67:0/3928488759 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fcbe00640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0b3c40 0x7f1fcc1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.519+0000 7f1fd1600640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fcc155920 0x7f1fcc165580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:02.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.520+0000 7f1fd0c00640 1 -- 172.21.15.67:0/3928488759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0b3c40 msgr2=0x7f1fcc1553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:02.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.520+0000 7f1fd0c00640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0b3c40 0x7f1fcc1553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.520+0000 7f1fd0c00640 1 -- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fcc155920 msgr2=0x7f1fcc165580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:02.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.520+0000 7f1fd0c00640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fcc155920 0x7f1fcc165580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.520+0000 7f1fd0c00640 1 -- 172.21.15.67:0/3928488759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc16cd50 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.520+0000 7f1fcbe00640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0b3c40 0x7f1fcc1553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:58:02.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.521+0000 7f1fd0c00640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fcc0af5f0 0x7f1fcc0d0830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009e50 tx=0x7f1fc4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:02.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.522+0000 7f1fc8c00640 1 -- 172.21.15.67:0/3928488759 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4036050 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.522+0000 7f1fc8c00640 1 -- 172.21.15.67:0/3928488759 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc4042070 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.522+0000 7f1fc8c00640 1 -- 172.21.15.67:0/3928488759 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4009510 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.522+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc16cff0 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.522+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc16d5b0 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.524+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc155d30 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.526+0000 7f1fc8c00640 1 -- 172.21.15.67:0/3928488759 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 31) ==== 92672+0+0 (secure 0 0 0) 0x7f1fc4007c70 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.528+0000 7f1fc8c00640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1fac06dbc0 0x7f1fac070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:02.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.528+0000 7f1fc8c00640 1 -- 172.21.15.67:0/3928488759 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f1fc40abe30 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.530+0000 7f1fcbe00640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1fac06dbc0 0x7f1fac070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.532+0000 7f1fcbe00640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1fac06dbc0 0x7f1fac070080 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009560 tx=0x7f1fbc009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.532+0000 7f1fc8c00640 1 -- 172.21.15.67:0/3928488759 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fcc155d30 con 0x7f1fcc0af5f0 2024-05-30T16:58:02.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.886+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922814", "target": ["mon-mgr", ""]}) -- 0x7f1fcc0689e0 con 0x7f1fac06dbc0 2024-05-30T16:58:02.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.921+0000 7f1fc8c00640 1 -- 172.21.15.67:0/3928488759 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1fcc0689e0 con 0x7f1fac06dbc0 2024-05-30T16:58:02.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.923+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1fac06dbc0 msgr2=0x7f1fac070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:02.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1fac06dbc0 0x7f1fac070080 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009560 tx=0x7f1fbc009190 comp rx=0 tx=0).stop 2024-05-30T16:58:02.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fcc0af5f0 msgr2=0x7f1fcc0d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:02.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fcc0af5f0 0x7f1fcc0d0830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009e50 tx=0x7f1fc4004a40 comp rx=0 tx=0).stop 2024-05-30T16:58:02.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 shutdown_connections 2024-05-30T16:58:02.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1fac06dbc0 0x7f1fac070080 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fcc155920 0x7f1fcc165580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fcc0b3c40 0x7f1fcc1553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 --2- 172.21.15.67:0/3928488759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fcc0af5f0 0x7f1fcc0d0830 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:02.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 >> 172.21.15.67:0/3928488759 conn(0x7f1fcc0b1e50 msgr2=0x7f1fcc0b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:02.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 shutdown_connections 2024-05-30T16:58:02.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:02.924+0000 7f1fd3600640 1 -- 172.21.15.67:0/3928488759 wait complete. 2024-05-30T16:58:02.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922817 2024-05-30T16:58:03.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.125+0000 7f77c4c00640 1 Processor -- start 2024-05-30T16:58:03.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.126+0000 7f77c4c00640 1 -- start start 2024-05-30T16:58:03.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.127+0000 7f77c4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c015fa00 0x7f77c006e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.127+0000 7f77c4c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f77c00d0b40 0x7f77c0072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.127+0000 7f77c4c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c006fa00 con 0x7f77c01600f0 2024-05-30T16:58:03.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.127+0000 7f77c4c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f77c006eeb0 con 0x7f77c00d0b40 2024-05-30T16:58:03.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.127+0000 7f77c4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f77c006f160 con 0x7f77c015fa00 2024-05-30T16:58:03.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.128+0000 7f77bf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c015fa00 0x7f77c006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.128+0000 7f77bf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c015fa00 0x7f77c006e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:54982/0 (socket says 172.21.15.67:54982) 2024-05-30T16:58:03.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.128+0000 7f77bf400640 1 -- 172.21.15.67:0/1380475645 learned_addr learned my addr 172.21.15.67:0/1380475645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:03.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.128+0000 7f77bea00640 1 --1- 172.21.15.67:0/1380475645 >> v1:172.21.15.169:6789/0 conn(0x7f77c01600f0 0x7f77c00d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:41616/0 (socket says 172.21.15.67:41616) 2024-05-30T16:58:03.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.128+0000 7f77bfe00640 1 --2- 172.21.15.67:0/1380475645 >> v2:172.21.15.169:3300/0 conn(0x7f77c00d0b40 0x7f77c0072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.129+0000 7f77bfe00640 1 -- 172.21.15.67:0/1380475645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c015fa00 msgr2=0x7f77c006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:03.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.129+0000 7f77bfe00640 1 --2- 172.21.15.67:0/1380475645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c015fa00 0x7f77c006e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.129+0000 7f77bfe00640 1 -- 172.21.15.67:0/1380475645 >> v1:172.21.15.169:6789/0 conn(0x7f77c01600f0 legacy=0x7f77c00d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.129+0000 7f77bfe00640 1 -- 172.21.15.67:0/1380475645 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c006f440 con 0x7f77c00d0b40 2024-05-30T16:58:03.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.130+0000 7f77bfe00640 1 --2- 172.21.15.67:0/1380475645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c00d0b40 0x7f77c0072190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009d90 tx=0x7f77b802a540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703e42bd4e27fea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:03.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.130+0000 7f77be000640 1 -- 172.21.15.67:0/1380475645 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8007740 con 0x7f77c00d0b40 2024-05-30T16:58:03.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77be000640 1 -- 172.21.15.67:0/1380475645 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b80078e0 con 0x7f77c00d0b40 2024-05-30T16:58:03.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77be000640 1 -- 172.21.15.67:0/1380475645 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8007cb0 con 0x7f77c00d0b40 2024-05-30T16:58:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77c4c00640 1 -- 172.21.15.67:0/1380475645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c00d0b40 msgr2=0x7f77c0072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77c4c00640 1 --2- 172.21.15.67:0/1380475645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c00d0b40 0x7f77c0072190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009d90 tx=0x7f77b802a540 comp rx=0 tx=0).stop 2024-05-30T16:58:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77c4c00640 1 -- 172.21.15.67:0/1380475645 shutdown_connections 2024-05-30T16:58:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77c4c00640 1 --2- 172.21.15.67:0/1380475645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c00d0b40 0x7f77c0072190 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77c4c00640 1 --2- 172.21.15.67:0/1380475645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c015fa00 0x7f77c006e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77c4c00640 1 -- 172.21.15.67:0/1380475645 >> 172.21.15.67:0/1380475645 conn(0x7f77c006bae0 msgr2=0x7f77c006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77c4c00640 1 -- 172.21.15.67:0/1380475645 shutdown_connections 2024-05-30T16:58:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.131+0000 7f77c4c00640 1 -- 172.21.15.67:0/1380475645 wait complete. 2024-05-30T16:58:03.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.133+0000 7f77c4c00640 1 Processor -- start 2024-05-30T16:58:03.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.133+0000 7f77c4c00640 1 -- start start 2024-05-30T16:58:03.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.134+0000 7f77c4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c00d0b40 0x7f77c00cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.134+0000 7f77c4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c015fa00 0x7f77c00ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.134+0000 7f77c4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c01600f0 0x7f77c00cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.134+0000 7f77c4c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f77c015fde0 con 0x7f77c00d0b40 2024-05-30T16:58:03.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.134+0000 7f77c4c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f77c0071c70 con 0x7f77c015fa00 2024-05-30T16:58:03.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.134+0000 7f77c4c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00726d0 con 0x7f77c01600f0 2024-05-30T16:58:03.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.134+0000 7f77bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c00d0b40 0x7f77c00cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.134+0000 7f77bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c00d0b40 0x7f77c00cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34832/0 (socket says 172.21.15.67:34832) 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.134+0000 7f77bf400640 1 -- 172.21.15.67:0/3698339248 learned_addr learned my addr 172.21.15.67:0/3698339248 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77bfe00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c01600f0 0x7f77c00cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77bea00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c015fa00 0x7f77c00ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77bf400640 1 -- 172.21.15.67:0/3698339248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c01600f0 msgr2=0x7f77c00cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77bf400640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c01600f0 0x7f77c00cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77bf400640 1 -- 172.21.15.67:0/3698339248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c015fa00 msgr2=0x7f77c00ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77bf400640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c015fa00 0x7f77c00ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77bf400640 1 -- 172.21.15.67:0/3698339248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c00cba60 con 0x7f77c00d0b40 2024-05-30T16:58:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77bea00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c015fa00 0x7f77c00ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:03.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77bf400640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c00d0b40 0x7f77c00cf420 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f77b000e8b0 tx=0x7f77b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:03.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77b7400640 1 -- 172.21.15.67:0/3698339248 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0017070 con 0x7f77c00d0b40 2024-05-30T16:58:03.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77b7400640 1 -- 172.21.15.67:0/3698339248 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b000c8e0 con 0x7f77c00d0b40 2024-05-30T16:58:03.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77b7400640 1 -- 172.21.15.67:0/3698339248 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b000ccb0 con 0x7f77c00d0b40 2024-05-30T16:58:03.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.135+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c00cbce0 con 0x7f77c00d0b40 2024-05-30T16:58:03.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.136+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c00cc2a0 con 0x7f77c00d0b40 2024-05-30T16:58:03.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.137+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c00689e0 con 0x7f77c00d0b40 2024-05-30T16:58:03.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.138+0000 7f77b7400640 1 -- 172.21.15.67:0/3698339248 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 31) ==== 92672+0+0 (secure 0 0 0) 0x7f77b001a070 con 0x7f77c00d0b40 2024-05-30T16:58:03.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.138+0000 7f77b7400640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77a006daf0 0x7f77a006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.139+0000 7f77b7400640 1 -- 172.21.15.67:0/3698339248 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f77b008a9f0 con 0x7f77c00d0b40 2024-05-30T16:58:03.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.139+0000 7f77bea00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77a006daf0 0x7f77a006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.139+0000 7f77bea00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77a006daf0 0x7f77a006ffb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f77c01542d0 tx=0x7f77ac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:03.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.143+0000 7f77b7400640 1 -- 172.21.15.67:0/3698339248 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77c00cba60 con 0x7f77c00d0b40 2024-05-30T16:58:03.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.368+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922817", "target": ["mon-mgr", ""]}) -- 0x7f77c006e200 con 0x7f77a006daf0 2024-05-30T16:58:03.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.389+0000 7f77b7400640 1 -- 172.21.15.67:0/3698339248 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77c006e200 con 0x7f77a006daf0 2024-05-30T16:58:03.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77a006daf0 msgr2=0x7f77a006ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77a006daf0 0x7f77a006ffb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f77c01542d0 tx=0x7f77ac009040 comp rx=0 tx=0).stop 2024-05-30T16:58:03.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c00d0b40 msgr2=0x7f77c00cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c00d0b40 0x7f77c00cf420 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f77b000e8b0 tx=0x7f77b000ed70 comp rx=0 tx=0).stop 2024-05-30T16:58:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 shutdown_connections 2024-05-30T16:58:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77a006daf0 0x7f77a006ffb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77c01600f0 0x7f77c00cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77c015fa00 0x7f77c00ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 --2- 172.21.15.67:0/3698339248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c00d0b40 0x7f77c00cf420 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 >> 172.21.15.67:0/3698339248 conn(0x7f77c006bae0 msgr2=0x7f77c006bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 shutdown_connections 2024-05-30T16:58:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.392+0000 7f77c4c00640 1 -- 172.21.15.67:0/3698339248 wait complete. 2024-05-30T16:58:03.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92285 2024-05-30T16:58:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.604+0000 7ffa2f200640 1 Processor -- start 2024-05-30T16:58:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.604+0000 7ffa2f200640 1 -- start start 2024-05-30T16:58:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.606+0000 7ffa2f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffa301600f0 0x7ffa300d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.606+0000 7ffa2f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffa300d0970 0x7ffa30072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.606+0000 7ffa2f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa300d0d90 con 0x7ffa3015fa00 2024-05-30T16:58:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.606+0000 7ffa2f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ffa3006fa00 con 0x7ffa301600f0 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.606+0000 7ffa2f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ffa3006f160 con 0x7ffa300d0970 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.606+0000 7ffa25400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffa301600f0 0x7ffa300d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.606+0000 7ffa2de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ffa3015fa00 0x7ffa3006e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56638/0 (socket says 172.21.15.67:56638) 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.606+0000 7ffa25400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffa301600f0 0x7ffa300d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34838/0 (socket says 172.21.15.67:34838) 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.606+0000 7ffa2de00640 1 -- 172.21.15.67:0/2822192113 learned_addr learned my addr 172.21.15.67:0/2822192113 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.607+0000 7ffa2e800640 1 --2- 172.21.15.67:0/2822192113 >> v2:172.21.15.169:3300/0 conn(0x7ffa300d0970 0x7ffa30072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.607+0000 7ffa25400640 1 -- 172.21.15.67:0/2822192113 >> v2:172.21.15.169:3300/0 conn(0x7ffa300d0970 msgr2=0x7ffa30072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.607+0000 7ffa25400640 1 --2- 172.21.15.67:0/2822192113 >> v2:172.21.15.169:3300/0 conn(0x7ffa300d0970 0x7ffa30072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.607+0000 7ffa25400640 1 -- 172.21.15.67:0/2822192113 >> v1:172.21.15.67:6789/0 conn(0x7ffa3015fa00 legacy=0x7ffa3006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.607+0000 7ffa25400640 1 -- 172.21.15.67:0/2822192113 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3006f440 con 0x7ffa301600f0 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.607+0000 7ffa2d400640 1 -- 172.21.15.67:0/2822192113 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522256650 0 0) 0x7ffa300d0d90 con 0x7ffa3015fa00 2024-05-30T16:58:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.607+0000 7ffa2e800640 1 --2- 172.21.15.67:0/2822192113 >> v2:172.21.15.169:3300/0 conn(0x7ffa300d0970 0x7ffa30072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.608+0000 7ffa25400640 1 --2- 172.21.15.67:0/2822192113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa301600f0 0x7ffa300d0430 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ffa3006e890 tx=0x7ffa1802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b82baaeb95295fe server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.608+0000 7ffa2d400640 1 -- 172.21.15.67:0/2822192113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18037070 con 0x7ffa301600f0 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.608+0000 7ffa2d400640 1 -- 172.21.15.67:0/2822192113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1802add0 con 0x7ffa301600f0 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.608+0000 7ffa2d400640 1 -- 172.21.15.67:0/2822192113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18032660 con 0x7ffa301600f0 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.609+0000 7ffa2f200640 1 -- 172.21.15.67:0/2822192113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa301600f0 msgr2=0x7ffa300d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.609+0000 7ffa2f200640 1 --2- 172.21.15.67:0/2822192113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa301600f0 0x7ffa300d0430 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ffa3006e890 tx=0x7ffa1802a080 comp rx=0 tx=0).stop 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.609+0000 7ffa2f200640 1 -- 172.21.15.67:0/2822192113 shutdown_connections 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.609+0000 7ffa2f200640 1 --2- 172.21.15.67:0/2822192113 >> v2:172.21.15.169:3300/0 conn(0x7ffa300d0970 0x7ffa30072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.609+0000 7ffa2f200640 1 --2- 172.21.15.67:0/2822192113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa301600f0 0x7ffa300d0430 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.609+0000 7ffa2f200640 1 -- 172.21.15.67:0/2822192113 >> 172.21.15.67:0/2822192113 conn(0x7ffa3006bae0 msgr2=0x7ffa3006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.609+0000 7ffa2f200640 1 -- 172.21.15.67:0/2822192113 shutdown_connections 2024-05-30T16:58:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.609+0000 7ffa2f200640 1 -- 172.21.15.67:0/2822192113 wait complete. 2024-05-30T16:58:03.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.610+0000 7ffa2f200640 1 Processor -- start 2024-05-30T16:58:03.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.611+0000 7ffa2f200640 1 -- start start 2024-05-30T16:58:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.611+0000 7ffa2f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa300d0970 0x7ffa3015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.611+0000 7ffa2f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa3015fa00 0x7ffa300c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.611+0000 7ffa2f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa301600f0 0x7ffa301f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.611+0000 7ffa2f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3015fde0 con 0x7ffa3015fa00 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.611+0000 7ffa2f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30071c70 con 0x7ffa300d0970 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.611+0000 7ffa2f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa300726d0 con 0x7ffa301600f0 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.612+0000 7ffa25400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa3015fa00 0x7ffa300c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.612+0000 7ffa25400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa3015fa00 0x7ffa300c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34854/0 (socket says 172.21.15.67:34854) 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.612+0000 7ffa25400640 1 -- 172.21.15.67:0/3647502731 learned_addr learned my addr 172.21.15.67:0/3647502731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.612+0000 7ffa25400640 1 -- 172.21.15.67:0/3647502731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa301600f0 msgr2=0x7ffa301f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.612+0000 7ffa25400640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa301600f0 0x7ffa301f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.612+0000 7ffa25400640 1 -- 172.21.15.67:0/3647502731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa300d0970 msgr2=0x7ffa3015e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.612+0000 7ffa25400640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa300d0970 0x7ffa3015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.612+0000 7ffa25400640 1 -- 172.21.15.67:0/3647502731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa301f4110 con 0x7ffa3015fa00 2024-05-30T16:58:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.612+0000 7ffa25400640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa3015fa00 0x7ffa300c9b50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009890 tx=0x7ffa180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.613+0000 7ffa26a00640 1 -- 172.21.15.67:0/3647502731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18041020 con 0x7ffa3015fa00 2024-05-30T16:58:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.613+0000 7ffa26a00640 1 -- 172.21.15.67:0/3647502731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1803d640 con 0x7ffa3015fa00 2024-05-30T16:58:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.613+0000 7ffa26a00640 1 -- 172.21.15.67:0/3647502731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18037040 con 0x7ffa3015fa00 2024-05-30T16:58:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.613+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3015eb20 con 0x7ffa3015fa00 2024-05-30T16:58:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.613+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa301f43b0 con 0x7ffa3015fa00 2024-05-30T16:58:03.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.614+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa300689e0 con 0x7ffa3015fa00 2024-05-30T16:58:03.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.615+0000 7ffa26a00640 1 -- 172.21.15.67:0/3647502731 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7ffa18044020 con 0x7ffa3015fa00 2024-05-30T16:58:03.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.616+0000 7ffa26a00640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffa0c06da60 0x7ffa0c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:03.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.616+0000 7ffa26a00640 1 -- 172.21.15.67:0/3647502731 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7ffa180ab000 con 0x7ffa3015fa00 2024-05-30T16:58:03.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.616+0000 7ffa2de00640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffa0c06da60 0x7ffa0c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:03.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.617+0000 7ffa2de00640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffa0c06da60 0x7ffa0c06ff20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffa20007fd0 tx=0x7ffa20007920 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:03.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.620+0000 7ffa26a00640 1 -- 172.21.15.67:0/3647502731 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa1804b030 con 0x7ffa3015fa00 2024-05-30T16:58:03.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.850+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92285", "target": ["mon-mgr", ""]}) -- 0x7ffa300cda60 con 0x7ffa0c06da60 2024-05-30T16:58:03.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.893+0000 7ffa26a00640 1 -- 172.21.15.67:0/3647502731 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa300cda60 con 0x7ffa0c06da60 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffa0c06da60 msgr2=0x7ffa0c06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffa0c06da60 0x7ffa0c06ff20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffa20007fd0 tx=0x7ffa20007920 comp rx=0 tx=0).stop 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa3015fa00 msgr2=0x7ffa300c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa3015fa00 0x7ffa300c9b50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009890 tx=0x7ffa180046c0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 shutdown_connections 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffa0c06da60 0x7ffa0c06ff20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa301600f0 0x7ffa301f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa3015fa00 0x7ffa300c9b50 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 --2- 172.21.15.67:0/3647502731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa300d0970 0x7ffa3015e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 >> 172.21.15.67:0/3647502731 conn(0x7ffa3006bae0 msgr2=0x7ffa3006bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 shutdown_connections 2024-05-30T16:58:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:03.896+0000 7ffa2f200640 1 -- 172.21.15.67:0/3647502731 wait complete. 2024-05-30T16:58:03.923 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922812 2024-05-30T16:58:04.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.110+0000 7fa2cda00640 1 Processor -- start 2024-05-30T16:58:04.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.110+0000 7fa2cda00640 1 -- start start 2024-05-30T16:58:04.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.112+0000 7fa2cda00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa2c80af6f0 0x7fa2c81623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.112+0000 7fa2cda00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa2c80d0b40 0x7fa2c81428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.112+0000 7fa2cda00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c8162cb0 con 0x7fa2c81480a0 2024-05-30T16:58:04.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.112+0000 7fa2cda00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c8162f90 con 0x7fa2c80d0b40 2024-05-30T16:58:04.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.112+0000 7fa2cda00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c8163270 con 0x7fa2c80af6f0 2024-05-30T16:58:04.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.112+0000 7fa2cd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa2c80d0b40 0x7fa2c81428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.112+0000 7fa2cd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa2c80d0b40 0x7fa2c81428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34858/0 (socket says 172.21.15.67:34858) 2024-05-30T16:58:04.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.112+0000 7fa2cd000640 1 -- 172.21.15.67:0/703223513 learned_addr learned my addr 172.21.15.67:0/703223513 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:04.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.113+0000 7fa2c7e00640 1 --2- 172.21.15.67:0/703223513 >> v2:172.21.15.169:3300/0 conn(0x7fa2c80af6f0 0x7fa2c81623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.113+0000 7fa2cd000640 1 -- 172.21.15.67:0/703223513 >> v2:172.21.15.169:3300/0 conn(0x7fa2c80af6f0 msgr2=0x7fa2c81623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:04.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.113+0000 7fa2cd000640 1 --2- 172.21.15.67:0/703223513 >> v2:172.21.15.169:3300/0 conn(0x7fa2c80af6f0 0x7fa2c81623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.113+0000 7fa2cd000640 1 -- 172.21.15.67:0/703223513 >> v1:172.21.15.67:6789/0 conn(0x7fa2c81480a0 legacy=0x7fa2c80d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.113+0000 7fa2cd000640 1 -- 172.21.15.67:0/703223513 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c81420d0 con 0x7fa2c80d0b40 2024-05-30T16:58:04.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.114+0000 7fa2cd000640 1 --2- 172.21.15.67:0/703223513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c80d0b40 0x7fa2c81428d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80d10c0 tx=0x7fa2c002a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3feaea47afe78c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:04.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.114+0000 7fa2c7400640 1 -- 172.21.15.67:0/703223513 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0036070 con 0x7fa2c80d0b40 2024-05-30T16:58:04.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.115+0000 7fa2c7400640 1 -- 172.21.15.67:0/703223513 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0031600 con 0x7fa2c80d0b40 2024-05-30T16:58:04.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.115+0000 7fa2c7400640 1 -- 172.21.15.67:0/703223513 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00319d0 con 0x7fa2c80d0b40 2024-05-30T16:58:04.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.115+0000 7fa2cda00640 1 -- 172.21.15.67:0/703223513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c80d0b40 msgr2=0x7fa2c81428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.115+0000 7fa2cda00640 1 --2- 172.21.15.67:0/703223513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c80d0b40 0x7fa2c81428d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80d10c0 tx=0x7fa2c002a740 comp rx=0 tx=0).stop 2024-05-30T16:58:04.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.115+0000 7fa2cda00640 1 -- 172.21.15.67:0/703223513 shutdown_connections 2024-05-30T16:58:04.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.115+0000 7fa2cda00640 1 --2- 172.21.15.67:0/703223513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c80d0b40 0x7fa2c81428d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.115+0000 7fa2cda00640 1 --2- 172.21.15.67:0/703223513 >> v2:172.21.15.169:3300/0 conn(0x7fa2c80af6f0 0x7fa2c81623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.115+0000 7fa2cda00640 1 -- 172.21.15.67:0/703223513 >> 172.21.15.67:0/703223513 conn(0x7fa2c806bae0 msgr2=0x7fa2c806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:04.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.115+0000 7fa2cda00640 1 -- 172.21.15.67:0/703223513 shutdown_connections 2024-05-30T16:58:04.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.116+0000 7fa2cda00640 1 -- 172.21.15.67:0/703223513 wait complete. 2024-05-30T16:58:04.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.118+0000 7fa2cda00640 1 Processor -- start 2024-05-30T16:58:04.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.118+0000 7fa2cda00640 1 -- start start 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.119+0000 7fa2cda00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c80af6f0 0x7fa2c8153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.119+0000 7fa2cda00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c80d0b40 0x7fa2c80cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.119+0000 7fa2cda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c81480a0 0x7fa2c81f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.119+0000 7fa2cda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c8162a70 con 0x7fa2c81480a0 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.119+0000 7fa2cda00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c81628f0 con 0x7fa2c80d0b40 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.119+0000 7fa2cda00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c80afad0 con 0x7fa2c80af6f0 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2cd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c81480a0 0x7fa2c81f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2cd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c81480a0 0x7fa2c81f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34868/0 (socket says 172.21.15.67:34868) 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2cd000640 1 -- 172.21.15.67:0/3397010342 learned_addr learned my addr 172.21.15.67:0/3397010342 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2cd000640 1 -- 172.21.15.67:0/3397010342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c80af6f0 msgr2=0x7fa2c8153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2c7e00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c80af6f0 0x7fa2c8153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2bfe00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c80d0b40 0x7fa2c80cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2cd000640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c80af6f0 0x7fa2c8153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2cd000640 1 -- 172.21.15.67:0/3397010342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c80d0b40 msgr2=0x7fa2c80cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2cd000640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c80d0b40 0x7fa2c80cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2cd000640 1 -- 172.21.15.67:0/3397010342 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c8154330 con 0x7fa2c81480a0 2024-05-30T16:58:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2bfe00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c80d0b40 0x7fa2c80cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:04.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.120+0000 7fa2cd000640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c81480a0 0x7fa2c81f4640 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009f40 tx=0x7fa2c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:04.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.121+0000 7fa2c4c00640 1 -- 172.21.15.67:0/3397010342 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0036050 con 0x7fa2c81480a0 2024-05-30T16:58:04.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.121+0000 7fa2c4c00640 1 -- 172.21.15.67:0/3397010342 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0042070 con 0x7fa2c81480a0 2024-05-30T16:58:04.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.121+0000 7fa2c4c00640 1 -- 172.21.15.67:0/3397010342 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0009510 con 0x7fa2c81480a0 2024-05-30T16:58:04.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.121+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c8143340 con 0x7fa2c81480a0 2024-05-30T16:58:04.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.121+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c8143900 con 0x7fa2c81480a0 2024-05-30T16:58:04.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.122+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c80689e0 con 0x7fa2c81480a0 2024-05-30T16:58:04.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.123+0000 7fa2c4c00640 1 -- 172.21.15.67:0/3397010342 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fa2c0007bd0 con 0x7fa2c81480a0 2024-05-30T16:58:04.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.124+0000 7fa2c4c00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2a406dab0 0x7fa2a406ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.124+0000 7fa2c4c00640 1 -- 172.21.15.67:0/3397010342 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7fa2c00ab810 con 0x7fa2c81480a0 2024-05-30T16:58:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.124+0000 7fa2c7e00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2a406dab0 0x7fa2a406ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.125+0000 7fa2c7e00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2a406dab0 0x7fa2a406ff70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8006fd0 tx=0x7fa2b8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:04.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.129+0000 7fa2c4c00640 1 -- 172.21.15.67:0/3397010342 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2c003a360 con 0x7fa2c81480a0 2024-05-30T16:58:04.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.350+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922812", "target": ["mon-mgr", ""]}) -- 0x7fa2c80c9e50 con 0x7fa2a406dab0 2024-05-30T16:58:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.390+0000 7fa2c4c00640 1 -- 172.21.15.67:0/3397010342 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa2c80c9e50 con 0x7fa2a406dab0 2024-05-30T16:58:04.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.392+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2a406dab0 msgr2=0x7fa2a406ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.392+0000 7fa2cda00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2a406dab0 0x7fa2a406ff70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8006fd0 tx=0x7fa2b8008040 comp rx=0 tx=0).stop 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.392+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c81480a0 msgr2=0x7fa2c81f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.392+0000 7fa2cda00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c81480a0 0x7fa2c81f4640 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009f40 tx=0x7fa2c0004a40 comp rx=0 tx=0).stop 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.393+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 shutdown_connections 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.393+0000 7fa2cda00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2a406dab0 0x7fa2a406ff70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.393+0000 7fa2cda00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c81480a0 0x7fa2c81f4640 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.393+0000 7fa2cda00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c80d0b40 0x7fa2c80cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.393+0000 7fa2cda00640 1 --2- 172.21.15.67:0/3397010342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c80af6f0 0x7fa2c8153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.393+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 >> 172.21.15.67:0/3397010342 conn(0x7fa2c806bae0 msgr2=0x7fa2c80b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.393+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 shutdown_connections 2024-05-30T16:58:04.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.393+0000 7fa2cda00640 1 -- 172.21.15.67:0/3397010342 wait complete. 2024-05-30T16:58:04.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922815 2024-05-30T16:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.610+0000 7f522be00640 1 Processor -- start 2024-05-30T16:58:04.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.610+0000 7f522be00640 1 -- start start 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.611+0000 7f522be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f522c15e5a0 0x7f522c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.611+0000 7f522be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c0cfb40 0x7f522c0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.611+0000 7f522be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f522c0d3500 con 0x7f522c0cffa0 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.611+0000 7f522be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f522c0d2b30 con 0x7f522c15e5a0 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.611+0000 7f522be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f522c0d2200 con 0x7f522c0cfb40 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.612+0000 7f522aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f522c15e5a0 0x7f522c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.612+0000 7f522b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f522c0cffa0 0x7f522c06c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56654/0 (socket says 172.21.15.67:56654) 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.612+0000 7f522b400640 1 -- 172.21.15.67:0/3353477588 learned_addr learned my addr 172.21.15.67:0/3353477588 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.612+0000 7f522a000640 1 --2- 172.21.15.67:0/3353477588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c0cfb40 0x7f522c0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.612+0000 7f522aa00640 1 -- 172.21.15.67:0/3353477588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c0cfb40 msgr2=0x7f522c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.612+0000 7f522aa00640 1 --2- 172.21.15.67:0/3353477588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c0cfb40 0x7f522c0d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.612+0000 7f522aa00640 1 -- 172.21.15.67:0/3353477588 >> v1:172.21.15.67:6789/0 conn(0x7f522c0cffa0 legacy=0x7f522c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.612+0000 7f522aa00640 1 -- 172.21.15.67:0/3353477588 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c0d24e0 con 0x7f522c15e5a0 2024-05-30T16:58:04.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.612+0000 7f5229600640 1 -- 172.21.15.67:0/3353477588 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1398234581 0 0) 0x7f522c0d3500 con 0x7f522c0cffa0 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.613+0000 7f522aa00640 1 --2- 172.21.15.67:0/3353477588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c15e5a0 0x7f522c0d2fc0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f521c0097a0 tx=0x7f521c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d646ab001aa9fa server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f5229600640 1 -- 172.21.15.67:0/3353477588 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c037070 con 0x7f522c15e5a0 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f5229600640 1 -- 172.21.15.67:0/3353477588 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c032440 con 0x7f522c15e5a0 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f5229600640 1 -- 172.21.15.67:0/3353477588 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c032810 con 0x7f522c15e5a0 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f522be00640 1 -- 172.21.15.67:0/3353477588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c15e5a0 msgr2=0x7f522c0d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f522be00640 1 --2- 172.21.15.67:0/3353477588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c15e5a0 0x7f522c0d2fc0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f521c0097a0 tx=0x7f521c02a180 comp rx=0 tx=0).stop 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f522be00640 1 -- 172.21.15.67:0/3353477588 shutdown_connections 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f522be00640 1 --2- 172.21.15.67:0/3353477588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c0cfb40 0x7f522c0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f522be00640 1 --2- 172.21.15.67:0/3353477588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c15e5a0 0x7f522c0d2fc0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f522be00640 1 -- 172.21.15.67:0/3353477588 >> 172.21.15.67:0/3353477588 conn(0x7f522c08bfc0 msgr2=0x7f522c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:04.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f522be00640 1 -- 172.21.15.67:0/3353477588 shutdown_connections 2024-05-30T16:58:04.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.614+0000 7f522be00640 1 -- 172.21.15.67:0/3353477588 wait complete. 2024-05-30T16:58:04.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.616+0000 7f522be00640 1 Processor -- start 2024-05-30T16:58:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.616+0000 7f522be00640 1 -- start start 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.617+0000 7f522be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c0cfb40 0x7f522c0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.617+0000 7f522be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522c0cffa0 0x7f522c0cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.617+0000 7f522be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c15e5a0 0x7f522c160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.617+0000 7f522be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f522c15eb00 con 0x7f522c0cfb40 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.617+0000 7f522be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f522c15e980 con 0x7f522c0cffa0 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.617+0000 7f522be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f522c0d5220 con 0x7f522c15e5a0 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c0cfb40 0x7f522c0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c0cfb40 0x7f522c0b59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34898/0 (socket says 172.21.15.67:34898) 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522aa00640 1 -- 172.21.15.67:0/43987048 learned_addr learned my addr 172.21.15.67:0/43987048 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522a000640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522c0cffa0 0x7f522c0cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522aa00640 1 -- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c15e5a0 msgr2=0x7f522c160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522b400640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c15e5a0 0x7f522c160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522aa00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c15e5a0 0x7f522c160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522aa00640 1 -- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522c0cffa0 msgr2=0x7f522c0cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522aa00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522c0cffa0 0x7f522c0cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522aa00640 1 -- 172.21.15.67:0/43987048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c0b6950 con 0x7f522c0cfb40 2024-05-30T16:58:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522a000640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522c0cffa0 0x7f522c0cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.618+0000 7f522b400640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c15e5a0 0x7f522c160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.619+0000 7f522aa00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c0cfb40 0x7f522c0b59c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f521c035040 tx=0x7f521c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.619+0000 7f5222a00640 1 -- 172.21.15.67:0/43987048 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c037050 con 0x7f522c0cfb40 2024-05-30T16:58:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.619+0000 7f5222a00640 1 -- 172.21.15.67:0/43987048 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c038040 con 0x7f522c0cfb40 2024-05-30T16:58:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.619+0000 7f5222a00640 1 -- 172.21.15.67:0/43987048 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c004490 con 0x7f522c0cfb40 2024-05-30T16:58:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.619+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522c0b6c30 con 0x7f522c0cfb40 2024-05-30T16:58:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.619+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522c0ccf20 con 0x7f522c0cfb40 2024-05-30T16:58:04.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.620+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522c0b5f00 con 0x7f522c0cfb40 2024-05-30T16:58:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.621+0000 7f5222a00640 1 -- 172.21.15.67:0/43987048 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f521c030980 con 0x7f522c0cfb40 2024-05-30T16:58:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.622+0000 7f5222a00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520806da60 0x7f520806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.622+0000 7f5222a00640 1 -- 172.21.15.67:0/43987048 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f521c0ab700 con 0x7f522c0cfb40 2024-05-30T16:58:04.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.622+0000 7f522a000640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520806da60 0x7f520806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:04.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.623+0000 7f522a000640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520806da60 0x7f520806ff20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f522c1f7b10 tx=0x7f5214008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:04.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.627+0000 7f5222a00640 1 -- 172.21.15.67:0/43987048 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f521c07b320 con 0x7f522c0cfb40 2024-05-30T16:58:04.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.862+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922815", "target": ["mon-mgr", ""]}) -- 0x7f522c0d1d30 con 0x7f520806da60 2024-05-30T16:58:04.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.896+0000 7f5222a00640 1 -- 172.21.15.67:0/43987048 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f522c0d1d30 con 0x7f520806da60 2024-05-30T16:58:04.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.899+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520806da60 msgr2=0x7f520806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520806da60 0x7f520806ff20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f522c1f7b10 tx=0x7f5214008040 comp rx=0 tx=0).stop 2024-05-30T16:58:04.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c0cfb40 msgr2=0x7f522c0b59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:04.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c0cfb40 0x7f522c0b59c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f521c035040 tx=0x7f521c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 shutdown_connections 2024-05-30T16:58:04.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520806da60 0x7f520806ff20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522c15e5a0 0x7f522c160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522c0cffa0 0x7f522c0cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 --2- 172.21.15.67:0/43987048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f522c0cfb40 0x7f522c0b59c0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:04.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 >> 172.21.15.67:0/43987048 conn(0x7f522c08bfc0 msgr2=0x7f522c0b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:04.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 shutdown_connections 2024-05-30T16:58:04.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:04.900+0000 7f522be00640 1 -- 172.21.15.67:0/43987048 wait complete. 2024-05-30T16:58:04.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922811 2024-05-30T16:58:05.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.176+0000 7f7209200640 1 Processor -- start 2024-05-30T16:58:05.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.176+0000 7f7209200640 1 -- start start 2024-05-30T16:58:05.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.178+0000 7f7209200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f72040af5f0 0x7f72040b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:05.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.178+0000 7f7209200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f72040af9d0 0x7f7204071c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:05.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.179+0000 7f7209200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72040b4100 con 0x7f72040b3c40 2024-05-30T16:58:05.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.179+0000 7f7209200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f72040b51a0 con 0x7f72040af9d0 2024-05-30T16:58:05.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.179+0000 7f7209200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f72040b43e0 con 0x7f72040af5f0 2024-05-30T16:58:05.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.179+0000 7f7201e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f72040af5f0 0x7f72040b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:05.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.179+0000 7f7201e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f72040af5f0 0x7f72040b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:50014/0 (socket says 172.21.15.67:50014) 2024-05-30T16:58:05.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.179+0000 7f7201e00640 1 -- 172.21.15.67:0/3780040102 learned_addr learned my addr 172.21.15.67:0/3780040102 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:05.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.179+0000 7f7201400640 1 --2- 172.21.15.67:0/3780040102 >> v2:172.21.15.67:3300/0 conn(0x7f72040af9d0 0x7f7204071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:05.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.180+0000 7f7201400640 1 -- 172.21.15.67:0/3780040102 >> v2:172.21.15.169:3300/0 conn(0x7f72040af5f0 msgr2=0x7f72040b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:05.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.180+0000 7f7201400640 1 --2- 172.21.15.67:0/3780040102 >> v2:172.21.15.169:3300/0 conn(0x7f72040af5f0 0x7f72040b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:05.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.180+0000 7f7201400640 1 -- 172.21.15.67:0/3780040102 >> v1:172.21.15.67:6789/0 conn(0x7f72040b3c40 legacy=0x7f720415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:05.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.180+0000 7f7201400640 1 -- 172.21.15.67:0/3780040102 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72040b46c0 con 0x7f72040af9d0 2024-05-30T16:58:05.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.182+0000 7f7201400640 1 --2- 172.21.15.67:0/3780040102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72040af9d0 0x7f7204071c00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f71f00097a0 tx=0x7f71f002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98df904517756b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:05.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.183+0000 7f7200a00640 1 -- 172.21.15.67:0/3780040102 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0037070 con 0x7f72040af9d0 2024-05-30T16:58:05.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.183+0000 7f7200a00640 1 -- 172.21.15.67:0/3780040102 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f0032440 con 0x7f72040af9d0 2024-05-30T16:58:05.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.183+0000 7f7200a00640 1 -- 172.21.15.67:0/3780040102 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0037070 con 0x7f72040af9d0 2024-05-30T16:58:05.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.184+0000 7f7209200640 1 -- 172.21.15.67:0/3780040102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72040af9d0 msgr2=0x7f7204071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:05.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.184+0000 7f7209200640 1 --2- 172.21.15.67:0/3780040102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72040af9d0 0x7f7204071c00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f71f00097a0 tx=0x7f71f002a1d0 comp rx=0 tx=0).stop 2024-05-30T16:58:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.185+0000 7f7209200640 1 -- 172.21.15.67:0/3780040102 shutdown_connections 2024-05-30T16:58:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.185+0000 7f7209200640 1 --2- 172.21.15.67:0/3780040102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72040af9d0 0x7f7204071c00 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.185+0000 7f7209200640 1 --2- 172.21.15.67:0/3780040102 >> v2:172.21.15.169:3300/0 conn(0x7f72040af5f0 0x7f72040b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.185+0000 7f7209200640 1 -- 172.21.15.67:0/3780040102 >> 172.21.15.67:0/3780040102 conn(0x7f72040b1e50 msgr2=0x7f72040b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.185+0000 7f7209200640 1 -- 172.21.15.67:0/3780040102 shutdown_connections 2024-05-30T16:58:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.185+0000 7f7209200640 1 -- 172.21.15.67:0/3780040102 wait complete. 2024-05-30T16:58:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.188+0000 7f7209200640 1 Processor -- start 2024-05-30T16:58:05.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.188+0000 7f7209200640 1 -- start start 2024-05-30T16:58:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.190+0000 7f7209200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040af5f0 0x7f720415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.190+0000 7f7209200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72040b3c40 0x7f720415db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.190+0000 7f7209200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720414a000 0x7f720415e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.190+0000 7f7209200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7204163440 con 0x7f720414a000 2024-05-30T16:58:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.190+0000 7f7209200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72040b4c80 con 0x7f72040af5f0 2024-05-30T16:58:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.190+0000 7f7209200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72041635c0 con 0x7f72040b3c40 2024-05-30T16:58:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.191+0000 7f7202800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720414a000 0x7f720415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.191+0000 7f7201400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72040b3c40 0x7f720415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.191+0000 7f7201400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72040b3c40 0x7f720415db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:55006/0 (socket says 172.21.15.67:55006) 2024-05-30T16:58:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.191+0000 7f7201400640 1 -- 172.21.15.67:0/2441718312 learned_addr learned my addr 172.21.15.67:0/2441718312 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:05.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.191+0000 7f7201e00640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040af5f0 0x7f720415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:05.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.192+0000 7f7201e00640 1 -- 172.21.15.67:0/2441718312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72040b3c40 msgr2=0x7f720415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:05.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.192+0000 7f7201e00640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72040b3c40 0x7f720415db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:05.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.192+0000 7f7201e00640 1 -- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720414a000 msgr2=0x7f720415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:05.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.192+0000 7f7201e00640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720414a000 0x7f720415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:05.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.192+0000 7f7201e00640 1 -- 172.21.15.67:0/2441718312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720414a6d0 con 0x7f72040af5f0 2024-05-30T16:58:05.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.192+0000 7f7202800640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720414a000 0x7f720415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:58:05.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.193+0000 7f7201e00640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040af5f0 0x7f720415b1c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71f400e850 tx=0x7f71f400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.193+0000 7f71fa000640 1 -- 172.21.15.67:0/2441718312 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4017070 con 0x7f72040af5f0 2024-05-30T16:58:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.193+0000 7f71fa000640 1 -- 172.21.15.67:0/2441718312 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f400c940 con 0x7f72040af5f0 2024-05-30T16:58:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.193+0000 7f71fa000640 1 -- 172.21.15.67:0/2441718312 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f400ccf0 con 0x7f72040af5f0 2024-05-30T16:58:05.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.194+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720414a9b0 con 0x7f72040af5f0 2024-05-30T16:58:05.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.194+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720414af70 con 0x7f72040af5f0 2024-05-30T16:58:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.197+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72040689e0 con 0x7f72040af5f0 2024-05-30T16:58:05.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.198+0000 7f71fa000640 1 -- 172.21.15.67:0/2441718312 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f71f401a070 con 0x7f72040af5f0 2024-05-30T16:58:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.199+0000 7f71fa000640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406dae0 0x7f71e406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.199+0000 7f71fa000640 1 -- 172.21.15.67:0/2441718312 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f71f408a940 con 0x7f72040af5f0 2024-05-30T16:58:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.200+0000 7f7201400640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406dae0 0x7f71e406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.201+0000 7f7201400640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406dae0 0x7f71e406ffa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f71f00097a0 tx=0x7f71f002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:05.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.208+0000 7f71fa000640 1 -- 172.21.15.67:0/2441718312 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71f4059e20 con 0x7f72040af5f0 2024-05-30T16:58:05.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:05.563+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922811", "target": ["mon-mgr", ""]}) -- 0x7f720406f3c0 con 0x7f71e406dae0 2024-05-30T16:58:07.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.256+0000 7f71fa000640 1 -- 172.21.15.67:0/2441718312 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f720406f3c0 con 0x7f71e406dae0 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406dae0 msgr2=0x7f71e406ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406dae0 0x7f71e406ffa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f71f00097a0 tx=0x7f71f002e000 comp rx=0 tx=0).stop 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040af5f0 msgr2=0x7f720415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040af5f0 0x7f720415b1c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71f400e850 tx=0x7f71f400ed10 comp rx=0 tx=0).stop 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 shutdown_connections 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71e406dae0 0x7f71e406ffa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720414a000 0x7f720415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72040b3c40 0x7f720415db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 --2- 172.21.15.67:0/2441718312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040af5f0 0x7f720415b1c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.259+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 >> 172.21.15.67:0/2441718312 conn(0x7f72040b1e50 msgr2=0x7f7204161310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.262+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 shutdown_connections 2024-05-30T16:58:07.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.262+0000 7f7209200640 1 -- 172.21.15.67:0/2441718312 wait complete. 2024-05-30T16:58:07.301 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92288 2024-05-30T16:58:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.568+0000 7f0266e00640 1 Processor -- start 2024-05-30T16:58:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.569+0000 7f0266e00640 1 -- start start 2024-05-30T16:58:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.570+0000 7f0266e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0260072310 0x7f026015fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:07.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.570+0000 7f0266e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02600b4100 con 0x7f02600af9d0 2024-05-30T16:58:07.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.570+0000 7f0266e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02600b43e0 con 0x7f02600af5f0 2024-05-30T16:58:07.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.570+0000 7f0266e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f02600b51a0 con 0x7f0260072310 2024-05-30T16:58:07.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.570+0000 7f025fe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f02600af5f0 0x7f02600b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:41628/0 (socket says 172.21.15.67:41628) 2024-05-30T16:58:07.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.570+0000 7f025fe00640 1 -- 172.21.15.67:0/1002130478 learned_addr learned my addr 172.21.15.67:0/1002130478 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:07.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.571+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2230873991 0 0) 0x7f02600b43e0 con 0x7f02600af5f0 2024-05-30T16:58:07.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.571+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0244003520 con 0x7f02600af5f0 2024-05-30T16:58:07.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.571+0000 7f0264e00640 1 --2- 172.21.15.67:0/1002130478 >> v2:172.21.15.67:3300/0 conn(0x7f0260072310 0x7f026015fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:07.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4077448555 0 0) 0x7f02600b4100 con 0x7f02600af9d0 2024-05-30T16:58:07.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02600b43e0 con 0x7f02600af9d0 2024-05-30T16:58:07.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 256081125 0 0) 0x7f0244003520 con 0x7f02600af5f0 2024-05-30T16:58:07.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 >> v2:172.21.15.67:3300/0 conn(0x7f0260072310 msgr2=0x7f026015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:07.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 --2- 172.21.15.67:0/1002130478 >> v2:172.21.15.67:3300/0 conn(0x7f0260072310 0x7f026015fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 >> v1:172.21.15.67:6789/0 conn(0x7f02600af9d0 legacy=0x7f0260071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:07.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02600b46c0 con 0x7f02600af5f0 2024-05-30T16:58:07.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f02580019f0 con 0x7f02600af5f0 2024-05-30T16:58:07.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 >> v1:172.21.15.169:6789/0 conn(0x7f02600af5f0 legacy=0x7f02600b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:07.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244003b90 0x7f0244003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:07.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02440079a0 0x7f0244009d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:07.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02440049b0 0x7f0244004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:07.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f02600b51a0 con 0x7f02440049b0 2024-05-30T16:58:07.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f024400c2e0 con 0x7f02440079a0 2024-05-30T16:58:07.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f024400c460 con 0x7f0244003b90 2024-05-30T16:58:07.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.572+0000 7f025fe00640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244003b90 0x7f0244003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:07.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.573+0000 7f025f400640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02440079a0 0x7f0244009d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:07.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.573+0000 7f025fe00640 1 -- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02440079a0 msgr2=0x7f0244009d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:07.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.573+0000 7f025fe00640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02440079a0 0x7f0244009d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.573+0000 7f025fe00640 1 -- 172.21.15.67:0/1002130478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02440049b0 msgr2=0x7f0244004e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:07.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.573+0000 7f025fe00640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02440049b0 0x7f0244004e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.573+0000 7f025fe00640 1 -- 172.21.15.67:0/1002130478 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0244003520 con 0x7f0244003b90 2024-05-30T16:58:07.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.573+0000 7f0264e00640 1 -- 172.21.15.67:0/1002130478 reap_dead start 2024-05-30T16:58:07.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.575+0000 7f025fe00640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244003b90 0x7f0244003f90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f025800a710 tx=0x7f0244005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c7dcd49fc8f2fa server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:07.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.575+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025800a330 con 0x7f0244003b90 2024-05-30T16:58:07.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.575+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025802f070 con 0x7f0244003b90 2024-05-30T16:58:07.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.575+0000 7f025ea00640 1 -- 172.21.15.67:0/1002130478 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025802bb80 con 0x7f0244003b90 2024-05-30T16:58:07.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.577+0000 7f0266e00640 1 -- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244003b90 msgr2=0x7f0244003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:07.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.577+0000 7f0266e00640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244003b90 0x7f0244003f90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f025800a710 tx=0x7f0244005830 comp rx=0 tx=0).stop 2024-05-30T16:58:07.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.577+0000 7f0266e00640 1 -- 172.21.15.67:0/1002130478 shutdown_connections 2024-05-30T16:58:07.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.577+0000 7f0266e00640 1 --2- 172.21.15.67:0/1002130478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244003b90 0x7f0244003f90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.577+0000 7f0266e00640 1 -- 172.21.15.67:0/1002130478 >> 172.21.15.67:0/1002130478 conn(0x7f02600b1e50 msgr2=0x7f02600b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:07.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.577+0000 7f0266e00640 1 -- 172.21.15.67:0/1002130478 shutdown_connections 2024-05-30T16:58:07.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.577+0000 7f0266e00640 1 -- 172.21.15.67:0/1002130478 wait complete. 2024-05-30T16:58:07.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.580+0000 7f0266e00640 1 Processor -- start 2024-05-30T16:58:07.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.581+0000 7f0266e00640 1 -- start start 2024-05-30T16:58:07.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.583+0000 7f0266e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0244003b90 0x7f02601531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:07.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.583+0000 7f0266e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02601603b0 0x7f0260160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:07.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.584+0000 7f0266e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260158c80 0x7f0260159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:07.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.584+0000 7f0266e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0260163440 con 0x7f0244003b90 2024-05-30T16:58:07.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.584+0000 7f0266e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02600b4c80 con 0x7f0260158c80 2024-05-30T16:58:07.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.584+0000 7f0266e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02601635c0 con 0x7f02601603b0 2024-05-30T16:58:07.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.584+0000 7f0264e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260158c80 0x7f0260159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:07.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.584+0000 7f025f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02601603b0 0x7f0260160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:07.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.584+0000 7f0264e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260158c80 0x7f0260159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:50044/0 (socket says 172.21.15.67:50044) 2024-05-30T16:58:07.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.584+0000 7f0264e00640 1 -- 172.21.15.67:0/2479483420 learned_addr learned my addr 172.21.15.67:0/2479483420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:07.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.585+0000 7f025fe00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0244003b90 0x7f02601531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:07.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.585+0000 7f025f400640 1 -- 172.21.15.67:0/2479483420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260158c80 msgr2=0x7f0260159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:07.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.585+0000 7f025f400640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260158c80 0x7f0260159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.585+0000 7f025f400640 1 -- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0244003b90 msgr2=0x7f02601531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:07.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.585+0000 7f025f400640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0244003b90 0x7f02601531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.585+0000 7f025f400640 1 -- 172.21.15.67:0/2479483420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0260159a00 con 0x7f02601603b0 2024-05-30T16:58:07.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.585+0000 7f025fe00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0244003b90 0x7f02601531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:58:07.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.586+0000 7f025f400640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02601603b0 0x7f0260160830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f024c00f7a0 tx=0x7f024c00fc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:07.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.587+0000 7f0257e00640 1 -- 172.21.15.67:0/2479483420 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c0052e0 con 0x7f02601603b0 2024-05-30T16:58:07.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.587+0000 7f0257e00640 1 -- 172.21.15.67:0/2479483420 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c018070 con 0x7f02601603b0 2024-05-30T16:58:07.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.587+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0260159ce0 con 0x7f02601603b0 2024-05-30T16:58:07.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.587+0000 7f0257e00640 1 -- 172.21.15.67:0/2479483420 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c00d970 con 0x7f02601603b0 2024-05-30T16:58:07.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.587+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f026015a2a0 con 0x7f02601603b0 2024-05-30T16:58:07.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.590+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0260160d70 con 0x7f02601603b0 2024-05-30T16:58:07.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.591+0000 7f0257e00640 1 -- 172.21.15.67:0/2479483420 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f024c00db10 con 0x7f02601603b0 2024-05-30T16:58:07.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.593+0000 7f0257e00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f024006da90 0x7f024006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:07.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.593+0000 7f0257e00640 1 -- 172.21.15.67:0/2479483420 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f024c08a860 con 0x7f02601603b0 2024-05-30T16:58:07.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.594+0000 7f025fe00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f024006da90 0x7f024006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:07.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.595+0000 7f025fe00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f024006da90 0x7f024006ff50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0258033000 tx=0x7f0258007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:07.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.601+0000 7f0257e00640 1 -- 172.21.15.67:0/2479483420 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f024c05a3d0 con 0x7f02601603b0 2024-05-30T16:58:07.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.870+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92288", "target": ["mon-mgr", ""]}) -- 0x7f02600689e0 con 0x7f024006da90 2024-05-30T16:58:07.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.916+0000 7f0257e00640 1 -- 172.21.15.67:0/2479483420 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02600689e0 con 0x7f024006da90 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f024006da90 msgr2=0x7f024006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f024006da90 0x7f024006ff50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0258033000 tx=0x7f0258007000 comp rx=0 tx=0).stop 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02601603b0 msgr2=0x7f0260160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02601603b0 0x7f0260160830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f024c00f7a0 tx=0x7f024c00fc60 comp rx=0 tx=0).stop 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 shutdown_connections 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f024006da90 0x7f024006ff50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0260158c80 0x7f0260159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02601603b0 0x7f0260160830 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 --2- 172.21.15.67:0/2479483420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0244003b90 0x7f02601531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 >> 172.21.15.67:0/2479483420 conn(0x7f02600b1e50 msgr2=0x7f0260161210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 shutdown_connections 2024-05-30T16:58:07.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:07.919+0000 7f0266e00640 1 -- 172.21.15.67:0/2479483420 wait complete. 2024-05-30T16:58:07.941 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92289 2024-05-30T16:58:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.130+0000 7feefbc00640 1 Processor -- start 2024-05-30T16:58:08.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.130+0000 7feefbc00640 1 -- start start 2024-05-30T16:58:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.131+0000 7feefbc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feef415e830 0x7feef406dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.131+0000 7feefbc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 0x7feef406f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.131+0000 7feefbc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef406e190 con 0x7feef4071c00 2024-05-30T16:58:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.131+0000 7feefbc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7feef406f770 con 0x7feef415e830 2024-05-30T16:58:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.131+0000 7feefbc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feef406e470 con 0x7feef40b3510 2024-05-30T16:58:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef9c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7feef4071c00 0x7feef40ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56690/0 (socket says 172.21.15.67:56690) 2024-05-30T16:58:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef9c00640 1 -- 172.21.15.67:0/3234584910 learned_addr learned my addr 172.21.15.67:0/3234584910 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef9200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feef415e830 0x7feef406dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef3e00640 1 --2- 172.21.15.67:0/3234584910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 0x7feef406f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef3400640 1 -- 172.21.15.67:0/3234584910 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751649035 0 0) 0x7feef406e190 con 0x7feef4071c00 2024-05-30T16:58:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef3400640 1 -- 172.21.15.67:0/3234584910 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feedc003520 con 0x7feef4071c00 2024-05-30T16:58:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef9200640 1 -- 172.21.15.67:0/3234584910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 msgr2=0x7feef406f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef9200640 1 --2- 172.21.15.67:0/3234584910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 0x7feef406f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef9200640 1 -- 172.21.15.67:0/3234584910 >> v1:172.21.15.67:6789/0 conn(0x7feef4071c00 legacy=0x7feef40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.132+0000 7feef9200640 1 -- 172.21.15.67:0/3234584910 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef406e750 con 0x7feef415e830 2024-05-30T16:58:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.133+0000 7feef9200640 1 --2- 172.21.15.67:0/3234584910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 0x7feef406dbb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7feee80097a0 tx=0x7feee802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afe776fa4ad4421 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.134+0000 7feef3400640 1 -- 172.21.15.67:0/3234584910 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8037070 con 0x7feef415e830 2024-05-30T16:58:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.134+0000 7feef3400640 1 -- 172.21.15.67:0/3234584910 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee8032440 con 0x7feef415e830 2024-05-30T16:58:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.134+0000 7feef3400640 1 -- 172.21.15.67:0/3234584910 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee80327f0 con 0x7feef415e830 2024-05-30T16:58:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.134+0000 7feefbc00640 1 -- 172.21.15.67:0/3234584910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 msgr2=0x7feef406dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.134+0000 7feefbc00640 1 --2- 172.21.15.67:0/3234584910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 0x7feef406dbb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7feee80097a0 tx=0x7feee802a180 comp rx=0 tx=0).stop 2024-05-30T16:58:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.134+0000 7feefbc00640 1 -- 172.21.15.67:0/3234584910 shutdown_connections 2024-05-30T16:58:08.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.134+0000 7feefbc00640 1 --2- 172.21.15.67:0/3234584910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 0x7feef406f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.134+0000 7feefbc00640 1 --2- 172.21.15.67:0/3234584910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 0x7feef406dbb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.134+0000 7feefbc00640 1 -- 172.21.15.67:0/3234584910 >> 172.21.15.67:0/3234584910 conn(0x7feef40b1e50 msgr2=0x7feef40b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.135+0000 7feefbc00640 1 -- 172.21.15.67:0/3234584910 shutdown_connections 2024-05-30T16:58:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.135+0000 7feefbc00640 1 -- 172.21.15.67:0/3234584910 wait complete. 2024-05-30T16:58:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.136+0000 7feefbc00640 1 Processor -- start 2024-05-30T16:58:08.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.136+0000 7feefbc00640 1 -- start start 2024-05-30T16:58:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.137+0000 7feefbc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef4071c00 0x7feef4149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.137+0000 7feefbc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 0x7feef4160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.137+0000 7feefbc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 0x7feef4161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.137+0000 7feefbc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feef415ec10 con 0x7feef415e830 2024-05-30T16:58:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.137+0000 7feefbc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feef406ed10 con 0x7feef4071c00 2024-05-30T16:58:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.137+0000 7feefbc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feef40728c0 con 0x7feef40b3510 2024-05-30T16:58:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.137+0000 7feef9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 0x7feef4161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.137+0000 7feef9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 0x7feef4161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34960/0 (socket says 172.21.15.67:34960) 2024-05-30T16:58:08.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.137+0000 7feef9c00640 1 -- 172.21.15.67:0/3898435304 learned_addr learned my addr 172.21.15.67:0/3898435304 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:08.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef3e00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 0x7feef4160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef9200640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef4071c00 0x7feef4149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef9c00640 1 -- 172.21.15.67:0/3898435304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 msgr2=0x7feef4160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef9c00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 0x7feef4160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef9c00640 1 -- 172.21.15.67:0/3898435304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef4071c00 msgr2=0x7feef4149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef9c00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef4071c00 0x7feef4149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef9c00640 1 -- 172.21.15.67:0/3898435304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef414a2d0 con 0x7feef415e830 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef9c00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 0x7feef4161080 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7feeec00cb40 tx=0x7feeec00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef0c00640 1 -- 172.21.15.67:0/3898435304 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec017070 con 0x7feef415e830 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef0c00640 1 -- 172.21.15.67:0/3898435304 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec00aca0 con 0x7feef415e830 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feef0c00640 1 -- 172.21.15.67:0/3898435304 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec012620 con 0x7feef415e830 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef414a7f0 con 0x7feef415e830 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.138+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef414adb0 con 0x7feef415e830 2024-05-30T16:58:08.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.139+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef40689e0 con 0x7feef415e830 2024-05-30T16:58:08.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.141+0000 7feef0c00640 1 -- 172.21.15.67:0/3898435304 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7feeec01a070 con 0x7feef415e830 2024-05-30T16:58:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.141+0000 7feef0c00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feed406da60 0x7feed406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.142+0000 7feef0c00640 1 -- 172.21.15.67:0/3898435304 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7feeec08a8b0 con 0x7feef415e830 2024-05-30T16:58:08.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.142+0000 7feef9200640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feed406da60 0x7feed406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.142+0000 7feef9200640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feed406da60 0x7feed406ff20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feee8034000 tx=0x7feee802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:08.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.146+0000 7feef0c00640 1 -- 172.21.15.67:0/3898435304 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feeec05a3e0 con 0x7feef415e830 2024-05-30T16:58:08.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.372+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92289", "target": ["mon-mgr", ""]}) -- 0x7feef41621f0 con 0x7feed406da60 2024-05-30T16:58:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.405+0000 7feef0c00640 1 -- 172.21.15.67:0/3898435304 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feef41621f0 con 0x7feed406da60 2024-05-30T16:58:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feed406da60 msgr2=0x7feed406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feed406da60 0x7feed406ff20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feee8034000 tx=0x7feee802e000 comp rx=0 tx=0).stop 2024-05-30T16:58:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 msgr2=0x7feef4161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 0x7feef4161080 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7feeec00cb40 tx=0x7feeec00a060 comp rx=0 tx=0).stop 2024-05-30T16:58:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 shutdown_connections 2024-05-30T16:58:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7feed406da60 0x7feed406ff20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feef415e830 0x7feef4161080 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feef40b3510 0x7feef4160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 --2- 172.21.15.67:0/3898435304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feef4071c00 0x7feef4149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.407+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 >> 172.21.15.67:0/3898435304 conn(0x7feef40b1e50 msgr2=0x7feef415fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.408+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 shutdown_connections 2024-05-30T16:58:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.408+0000 7feefbc00640 1 -- 172.21.15.67:0/3898435304 wait complete. 2024-05-30T16:58:08.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922818 2024-05-30T16:58:08.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.617+0000 7f3069000640 1 Processor -- start 2024-05-30T16:58:08.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.617+0000 7f3069000640 1 -- start start 2024-05-30T16:58:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.618+0000 7f3069000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f306415fa00 0x7f306406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.618+0000 7f3069000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30640d0b40 0x7f3064072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.618+0000 7f3069000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f306406fa00 con 0x7f30641600f0 2024-05-30T16:58:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.618+0000 7f3069000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f306406eeb0 con 0x7f30640d0b40 2024-05-30T16:58:08.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.618+0000 7f3069000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f306406f160 con 0x7f306415fa00 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.619+0000 7f305be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f30641600f0 0x7f30640d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56692/0 (socket says 172.21.15.67:56692) 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.619+0000 7f305be00640 1 -- 172.21.15.67:0/3290088760 learned_addr learned my addr 172.21.15.67:0/3290088760 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.619+0000 7f3063e00640 1 --2- 172.21.15.67:0/3290088760 >> v2:172.21.15.169:3300/0 conn(0x7f30640d0b40 0x7f3064072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.619+0000 7f3063400640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f306415fa00 0x7f306406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.619+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160060369 0 0) 0x7f306406fa00 con 0x7f30641600f0 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.619+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3048003520 con 0x7f30641600f0 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 492036844 0 0) 0x7f3048003520 con 0x7f30641600f0 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f306415fa00 msgr2=0x7f306406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f306415fa00 0x7f306406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 >> v2:172.21.15.169:3300/0 conn(0x7f30640d0b40 msgr2=0x7f3064072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 --2- 172.21.15.67:0/3290088760 >> v2:172.21.15.169:3300/0 conn(0x7f30640d0b40 0x7f3064072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306406f440 con 0x7f30641600f0 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f304c001690 con 0x7f30641600f0 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 >> v1:172.21.15.67:6789/0 conn(0x7f30641600f0 legacy=0x7f30640d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f3048003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f306415fa00 0x7f3048004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3048004d40 0x7f30480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f306406eeb0 con 0x7f30480039b0 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f306406f160 con 0x7f3048004d40 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f304800bb40 con 0x7f306415fa00 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3063e00640 1 --2- 172.21.15.67:0/3290088760 >> v2:172.21.15.169:3300/0 conn(0x7f30640d0b40 0x7f3064072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.620+0000 7f3063400640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f3048003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.621+0000 7f3063400640 1 -- 172.21.15.67:0/3290088760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f306415fa00 msgr2=0x7f3048004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.621+0000 7f3063400640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f306415fa00 0x7f3048004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.621+0000 7f3063400640 1 -- 172.21.15.67:0/3290088760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3048004d40 msgr2=0x7f30480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:08.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.621+0000 7f3063400640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3048004d40 0x7f30480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.621+0000 7f3063400640 1 -- 172.21.15.67:0/3290088760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3048003520 con 0x7f30480039b0 2024-05-30T16:58:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.622+0000 7f3063400640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f3048003dd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3054009ac0 tx=0x7f3048005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dd64e894a5de86 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.622+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054036070 con 0x7f30480039b0 2024-05-30T16:58:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.622+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3054031400 con 0x7f30480039b0 2024-05-30T16:58:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.622+0000 7f3062a00640 1 -- 172.21.15.67:0/3290088760 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054009690 con 0x7f30480039b0 2024-05-30T16:58:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.623+0000 7f3069000640 1 -- 172.21.15.67:0/3290088760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 msgr2=0x7f3048003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.623+0000 7f3069000640 1 --2- 172.21.15.67:0/3290088760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f3048003dd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3054009ac0 tx=0x7f3048005830 comp rx=0 tx=0).stop 2024-05-30T16:58:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.623+0000 7f3063e00640 1 -- 172.21.15.67:0/3290088760 reap_dead start 2024-05-30T16:58:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.623+0000 7f3069000640 1 -- 172.21.15.67:0/3290088760 shutdown_connections 2024-05-30T16:58:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.623+0000 7f3069000640 1 -- 172.21.15.67:0/3290088760 >> 172.21.15.67:0/3290088760 conn(0x7f306406bae0 msgr2=0x7f306406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.623+0000 7f3069000640 1 -- 172.21.15.67:0/3290088760 shutdown_connections 2024-05-30T16:58:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.623+0000 7f3069000640 1 -- 172.21.15.67:0/3290088760 wait complete. 2024-05-30T16:58:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.625+0000 7f3069000640 1 Processor -- start 2024-05-30T16:58:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.625+0000 7f3069000640 1 -- start start 2024-05-30T16:58:08.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3069000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f30640cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3069000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30640cddc0 0x7f306415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3069000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3064153940 0x7f30641f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3069000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f306415fde0 con 0x7f30480039b0 2024-05-30T16:58:08.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3069000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3064071c70 con 0x7f30640cddc0 2024-05-30T16:58:08.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3069000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f306406e970 con 0x7f3064153940 2024-05-30T16:58:08.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3063400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f30640cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3063400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f30640cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34976/0 (socket says 172.21.15.67:34976) 2024-05-30T16:58:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3063400640 1 -- 172.21.15.67:0/608256250 learned_addr learned my addr 172.21.15.67:0/608256250 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3063400640 1 -- 172.21.15.67:0/608256250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3064153940 msgr2=0x7f30641f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3063400640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3064153940 0x7f30641f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3063400640 1 -- 172.21.15.67:0/608256250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30640cddc0 msgr2=0x7f306415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.626+0000 7f3063400640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30640cddc0 0x7f306415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.627+0000 7f3063400640 1 -- 172.21.15.67:0/608256250 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30640ce430 con 0x7f30480039b0 2024-05-30T16:58:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.627+0000 7f3063400640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f30640cf4d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3054034040 tx=0x7f3054004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.627+0000 7f305b400640 1 -- 172.21.15.67:0/608256250 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054040020 con 0x7f30480039b0 2024-05-30T16:58:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.627+0000 7f305b400640 1 -- 172.21.15.67:0/608256250 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3054003940 con 0x7f30480039b0 2024-05-30T16:58:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.627+0000 7f305b400640 1 -- 172.21.15.67:0/608256250 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054036040 con 0x7f30480039b0 2024-05-30T16:58:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.627+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30640ce710 con 0x7f30480039b0 2024-05-30T16:58:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.627+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30640cecd0 con 0x7f30480039b0 2024-05-30T16:58:08.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.629+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306406db40 con 0x7f30480039b0 2024-05-30T16:58:08.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.630+0000 7f305b400640 1 -- 172.21.15.67:0/608256250 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f3054043020 con 0x7f30480039b0 2024-05-30T16:58:08.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.630+0000 7f305b400640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f304006da60 0x7f304006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:08.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.631+0000 7f305b400640 1 -- 172.21.15.67:0/608256250 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f30540aace0 con 0x7f30480039b0 2024-05-30T16:58:08.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.631+0000 7f305be00640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f304006da60 0x7f304006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:08.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.631+0000 7f305be00640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f304006da60 0x7f304006ff20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f304c0011e0 tx=0x7f304c001450 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:08.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.635+0000 7f305b400640 1 -- 172.21.15.67:0/608256250 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f306406db40 con 0x7f30480039b0 2024-05-30T16:58:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.874+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922818", "target": ["mon-mgr", ""]}) -- 0x7f30640689e0 con 0x7f304006da60 2024-05-30T16:58:08.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.909+0000 7f305b400640 1 -- 172.21.15.67:0/608256250 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f30640689e0 con 0x7f304006da60 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f304006da60 msgr2=0x7f304006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f304006da60 0x7f304006ff20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f304c0011e0 tx=0x7f304c001450 comp rx=0 tx=0).stop 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 msgr2=0x7f30640cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f30640cf4d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3054034040 tx=0x7f3054004730 comp rx=0 tx=0).stop 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 shutdown_connections 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f304006da60 0x7f304006ff20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3064153940 0x7f30641f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30640cddc0 0x7f306415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 --2- 172.21.15.67:0/608256250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30480039b0 0x7f30640cf4d0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:08.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 >> 172.21.15.67:0/608256250 conn(0x7f306406bae0 msgr2=0x7f30640b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:08.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 shutdown_connections 2024-05-30T16:58:08.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:08.912+0000 7f3069000640 1 -- 172.21.15.67:0/608256250 wait complete. 2024-05-30T16:58:08.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92281 2024-05-30T16:58:09.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.201+0000 7f2d82400640 1 Processor -- start 2024-05-30T16:58:09.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.202+0000 7f2d82400640 1 -- start start 2024-05-30T16:58:09.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.204+0000 7f2d82400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0d34e0 0x7f2d7c0d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.204+0000 7f2d82400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2d7c0b5570 0x7f2d7c0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.204+0000 7f2d82400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7c0d5500 con 0x7f2d7c1622b0 2024-05-30T16:58:09.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.204+0000 7f2d82400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2d7c0b4f50 con 0x7f2d7c0b5570 2024-05-30T16:58:09.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.204+0000 7f2d82400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7c0d2720 con 0x7f2d7c0d34e0 2024-05-30T16:58:09.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.204+0000 7f2d7b400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f2d7c1622b0 0x7f2d7c0d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:41644/0 (socket says 172.21.15.67:41644) 2024-05-30T16:58:09.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.204+0000 7f2d7b400640 1 -- 172.21.15.67:0/3933943996 learned_addr learned my addr 172.21.15.67:0/3933943996 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:09.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.204+0000 7f2d7aa00640 1 --2- 172.21.15.67:0/3933943996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0d34e0 0x7f2d7c0d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.206+0000 7f2d7be00640 1 --2- 172.21.15.67:0/3933943996 >> v2:172.21.15.169:3300/0 conn(0x7f2d7c0b5570 0x7f2d7c0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.206+0000 7f2d7a000640 1 -- 172.21.15.67:0/3933943996 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 939324605 0 0) 0x7f2d7c0d5500 con 0x7f2d7c1622b0 2024-05-30T16:58:09.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.206+0000 7f2d7a000640 1 -- 172.21.15.67:0/3933943996 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d60003520 con 0x7f2d7c1622b0 2024-05-30T16:58:09.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.206+0000 7f2d7be00640 1 -- 172.21.15.67:0/3933943996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0d34e0 msgr2=0x7f2d7c0d4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:09.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.207+0000 7f2d7a000640 1 -- 172.21.15.67:0/3933943996 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3918755966 0 0) 0x7f2d60003520 con 0x7f2d7c1622b0 2024-05-30T16:58:09.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.207+0000 7f2d7be00640 1 --2- 172.21.15.67:0/3933943996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0d34e0 0x7f2d7c0d4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.207+0000 7f2d7be00640 1 -- 172.21.15.67:0/3933943996 >> v1:172.21.15.169:6789/0 conn(0x7f2d7c1622b0 legacy=0x7f2d7c0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.207+0000 7f2d7be00640 1 -- 172.21.15.67:0/3933943996 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c0d2a00 con 0x7f2d7c0b5570 2024-05-30T16:58:09.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.207+0000 7f2d7aa00640 1 --2- 172.21.15.67:0/3933943996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0d34e0 0x7f2d7c0d4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:58:09.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.208+0000 7f2d7be00640 1 --2- 172.21.15.67:0/3933943996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7c0b5570 0x7f2d7c0b5a20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009d90 tx=0x7f2d7402a550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a02d33f2eea8cd16 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:09.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.209+0000 7f2d7a000640 1 -- 172.21.15.67:0/3933943996 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c001670 con 0x7f2d7c0b5570 2024-05-30T16:58:09.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.209+0000 7f2d7a000640 1 -- 172.21.15.67:0/3933943996 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d74007540 con 0x7f2d7c0b5570 2024-05-30T16:58:09.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.209+0000 7f2d7a000640 1 -- 172.21.15.67:0/3933943996 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74007910 con 0x7f2d7c0b5570 2024-05-30T16:58:09.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.210+0000 7f2d82400640 1 -- 172.21.15.67:0/3933943996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7c0b5570 msgr2=0x7f2d7c0b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.210+0000 7f2d82400640 1 --2- 172.21.15.67:0/3933943996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7c0b5570 0x7f2d7c0b5a20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009d90 tx=0x7f2d7402a550 comp rx=0 tx=0).stop 2024-05-30T16:58:09.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.211+0000 7f2d82400640 1 -- 172.21.15.67:0/3933943996 shutdown_connections 2024-05-30T16:58:09.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.211+0000 7f2d82400640 1 --2- 172.21.15.67:0/3933943996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7c0b5570 0x7f2d7c0b5a20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.211+0000 7f2d82400640 1 --2- 172.21.15.67:0/3933943996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0d34e0 0x7f2d7c0d4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.211+0000 7f2d82400640 1 -- 172.21.15.67:0/3933943996 >> 172.21.15.67:0/3933943996 conn(0x7f2d7c08bfc0 msgr2=0x7f2d7c0d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.211+0000 7f2d82400640 1 -- 172.21.15.67:0/3933943996 shutdown_connections 2024-05-30T16:58:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.211+0000 7f2d82400640 1 -- 172.21.15.67:0/3933943996 wait complete. 2024-05-30T16:58:09.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.214+0000 7f2d82400640 1 Processor -- start 2024-05-30T16:58:09.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.214+0000 7f2d82400640 1 -- start start 2024-05-30T16:58:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.215+0000 7f2d82400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0b5570 0x7f2d7c1f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.215+0000 7f2d82400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7c0d34e0 0x7f2d7c0b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.215+0000 7f2d82400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c1622b0 0x7f2d7c0d1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.215+0000 7f2d82400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0b40c0 con 0x7f2d7c1622b0 2024-05-30T16:58:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.215+0000 7f2d82400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7c0b3f40 con 0x7f2d7c0d34e0 2024-05-30T16:58:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.215+0000 7f2d82400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7c0b4240 con 0x7f2d7c0b5570 2024-05-30T16:58:09.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.216+0000 7f2d7aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7c0d34e0 0x7f2d7c0b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.216+0000 7f2d7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c1622b0 0x7f2d7c0d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.217+0000 7f2d7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c1622b0 0x7f2d7c0d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34988/0 (socket says 172.21.15.67:34988) 2024-05-30T16:58:09.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.217+0000 7f2d7be00640 1 -- 172.21.15.67:0/2851816504 learned_addr learned my addr 172.21.15.67:0/2851816504 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:09.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.217+0000 7f2d7b400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0b5570 0x7f2d7c1f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.218+0000 7f2d7be00640 1 -- 172.21.15.67:0/2851816504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0b5570 msgr2=0x7f2d7c1f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.218+0000 7f2d7be00640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0b5570 0x7f2d7c1f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.218+0000 7f2d7be00640 1 -- 172.21.15.67:0/2851816504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7c0d34e0 msgr2=0x7f2d7c0b9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.218+0000 7f2d7be00640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7c0d34e0 0x7f2d7c0b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.218+0000 7f2d7be00640 1 -- 172.21.15.67:0/2851816504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7c1512f0 con 0x7f2d7c1622b0 2024-05-30T16:58:09.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.220+0000 7f2d7be00640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c1622b0 0x7f2d7c0d1020 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009d90 tx=0x7f2d74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.221+0000 7f2d73400640 1 -- 172.21.15.67:0/2851816504 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74040070 con 0x7f2d7c1622b0 2024-05-30T16:58:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.221+0000 7f2d73400640 1 -- 172.21.15.67:0/2851816504 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d74031b40 con 0x7f2d7c1622b0 2024-05-30T16:58:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.221+0000 7f2d73400640 1 -- 172.21.15.67:0/2851816504 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74009450 con 0x7f2d7c1622b0 2024-05-30T16:58:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.221+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7c151590 con 0x7f2d7c1622b0 2024-05-30T16:58:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.221+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7c151b50 con 0x7f2d7c1622b0 2024-05-30T16:58:09.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.223+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7c0689e0 con 0x7f2d7c1622b0 2024-05-30T16:58:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.227+0000 7f2d73400640 1 -- 172.21.15.67:0/2851816504 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f2d74039040 con 0x7f2d7c1622b0 2024-05-30T16:58:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.230+0000 7f2d73400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d5c06dab0 0x7f2d5c06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.230+0000 7f2d73400640 1 -- 172.21.15.67:0/2851816504 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f2d740ac590 con 0x7f2d7c1622b0 2024-05-30T16:58:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.231+0000 7f2d7b400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d5c06dab0 0x7f2d5c06ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.232+0000 7f2d7b400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d5c06dab0 0x7f2d5c06ff70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0011e0 tx=0x7f2d6c0016d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.234+0000 7f2d73400640 1 -- 172.21.15.67:0/2851816504 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d74053020 con 0x7f2d7c1622b0 2024-05-30T16:58:09.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.591+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92281", "target": ["mon-mgr", ""]}) -- 0x7f2d7c08c6c0 con 0x7f2d5c06dab0 2024-05-30T16:58:09.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.613+0000 7f2d73400640 1 -- 172.21.15.67:0/2851816504 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2d7c08c6c0 con 0x7f2d5c06dab0 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d5c06dab0 msgr2=0x7f2d5c06ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d5c06dab0 0x7f2d5c06ff70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0011e0 tx=0x7f2d6c0016d0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c1622b0 msgr2=0x7f2d7c0d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c1622b0 0x7f2d7c0d1020 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009d90 tx=0x7f2d74004a40 comp rx=0 tx=0).stop 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 shutdown_connections 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d5c06dab0 0x7f2d5c06ff70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c1622b0 0x7f2d7c0d1020 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d7c0d34e0 0x7f2d7c0b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 --2- 172.21.15.67:0/2851816504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d7c0b5570 0x7f2d7c1f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 >> 172.21.15.67:0/2851816504 conn(0x7f2d7c08bfc0 msgr2=0x7f2d7c0b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 shutdown_connections 2024-05-30T16:58:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.616+0000 7f2d82400640 1 -- 172.21.15.67:0/2851816504 wait complete. 2024-05-30T16:58:09.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92287 2024-05-30T16:58:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.837+0000 7f9f63200640 1 Processor -- start 2024-05-30T16:58:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.837+0000 7f9f63200640 1 -- start start 2024-05-30T16:58:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.839+0000 7f9f63200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.839+0000 7f9f63200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9f5c0d1420 0x7f9f5c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.839+0000 7f9f63200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f5c0d3500 con 0x7f9f5c0cfb40 2024-05-30T16:58:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.839+0000 7f9f63200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9f5c0d2b30 con 0x7f9f5c0d1420 2024-05-30T16:58:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.839+0000 7f9f63200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5c0d2200 con 0x7f9f5c15e5a0 2024-05-30T16:58:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.839+0000 7f9f61e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.839+0000 7f9f61e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:55080/0 (socket says 172.21.15.67:55080) 2024-05-30T16:58:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.839+0000 7f9f61e00640 1 -- 172.21.15.67:0/3691303906 learned_addr learned my addr 172.21.15.67:0/3691303906 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:09.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.840+0000 7f9f62800640 1 --2- 172.21.15.67:0/3691303906 >> v2:172.21.15.169:3300/0 conn(0x7f9f5c0d1420 0x7f9f5c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.840+0000 7f9f61e00640 1 -- 172.21.15.67:0/3691303906 >> v2:172.21.15.169:3300/0 conn(0x7f9f5c0d1420 msgr2=0x7f9f5c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.840+0000 7f9f61e00640 1 --2- 172.21.15.67:0/3691303906 >> v2:172.21.15.169:3300/0 conn(0x7f9f5c0d1420 0x7f9f5c06c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.840+0000 7f9f61e00640 1 -- 172.21.15.67:0/3691303906 >> v1:172.21.15.169:6789/0 conn(0x7f9f5c0cfb40 legacy=0x7f9f5c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.840+0000 7f9f61e00640 1 -- 172.21.15.67:0/3691303906 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c0d24e0 con 0x7f9f5c15e5a0 2024-05-30T16:58:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.840+0000 7f9f62800640 1 --2- 172.21.15.67:0/3691303906 >> v2:172.21.15.169:3300/0 conn(0x7f9f5c0d1420 0x7f9f5c06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:58:09.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.842+0000 7f9f61e00640 1 --2- 172.21.15.67:0/3691303906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0d2fc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009870 tx=0x7f9f5002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ee6a9fab7e82b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:09.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.842+0000 7f9f60a00640 1 -- 172.21.15.67:0/3691303906 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50037070 con 0x7f9f5c15e5a0 2024-05-30T16:58:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.842+0000 7f9f60a00640 1 -- 172.21.15.67:0/3691303906 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f50032440 con 0x7f9f5c15e5a0 2024-05-30T16:58:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.842+0000 7f9f60a00640 1 -- 172.21.15.67:0/3691303906 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50032810 con 0x7f9f5c15e5a0 2024-05-30T16:58:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.843+0000 7f9f63200640 1 -- 172.21.15.67:0/3691303906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 msgr2=0x7f9f5c0d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.843+0000 7f9f63200640 1 --2- 172.21.15.67:0/3691303906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0d2fc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009870 tx=0x7f9f5002a250 comp rx=0 tx=0).stop 2024-05-30T16:58:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.843+0000 7f9f63200640 1 -- 172.21.15.67:0/3691303906 shutdown_connections 2024-05-30T16:58:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.843+0000 7f9f63200640 1 --2- 172.21.15.67:0/3691303906 >> v2:172.21.15.169:3300/0 conn(0x7f9f5c0d1420 0x7f9f5c06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.843+0000 7f9f63200640 1 --2- 172.21.15.67:0/3691303906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0d2fc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.843+0000 7f9f63200640 1 -- 172.21.15.67:0/3691303906 >> 172.21.15.67:0/3691303906 conn(0x7f9f5c08bfc0 msgr2=0x7f9f5c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.843+0000 7f9f63200640 1 -- 172.21.15.67:0/3691303906 shutdown_connections 2024-05-30T16:58:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.843+0000 7f9f63200640 1 -- 172.21.15.67:0/3691303906 wait complete. 2024-05-30T16:58:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.845+0000 7f9f63200640 1 Processor -- start 2024-05-30T16:58:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.845+0000 7f9f63200640 1 -- start start 2024-05-30T16:58:09.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f63200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f5c0cfb40 0x7f9f5c157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f63200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c0d1420 0x7f9f5c0b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f63200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0b7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f63200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c15eb00 con 0x7f9f5c0d1420 2024-05-30T16:58:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f63200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c15e980 con 0x7f9f5c0cfb40 2024-05-30T16:58:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f63200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5c0d5220 con 0x7f9f5c15e5a0 2024-05-30T16:58:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f61400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c0d1420 0x7f9f5c0b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f61400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c0d1420 0x7f9f5c0b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35004/0 (socket says 172.21.15.67:35004) 2024-05-30T16:58:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f61400640 1 -- 172.21.15.67:0/264633329 learned_addr learned my addr 172.21.15.67:0/264633329 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.846+0000 7f9f62800640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f61e00640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f5c0cfb40 0x7f9f5c157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f61400640 1 -- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 msgr2=0x7f9f5c0b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f61400640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f61400640 1 -- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f5c0cfb40 msgr2=0x7f9f5c157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f61400640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f5c0cfb40 0x7f9f5c157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f61400640 1 -- 172.21.15.67:0/264633329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c0b9f00 con 0x7f9f5c0d1420 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f61e00640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f5c0cfb40 0x7f9f5c157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f62800640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f61400640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c0d1420 0x7f9f5c0b7240 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c430 tx=0x7f9f4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f56000640 1 -- 172.21.15.67:0/264633329 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0052d0 con 0x7f9f5c0d1420 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f56000640 1 -- 172.21.15.67:0/264633329 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4c005490 con 0x7f9f5c0d1420 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c1f44e0 con 0x7f9f5c0d1420 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f56000640 1 -- 172.21.15.67:0/264633329 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c007260 con 0x7f9f5c0d1420 2024-05-30T16:58:09.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.847+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c1f4a80 con 0x7f9f5c0d1420 2024-05-30T16:58:09.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.849+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c1f5420 con 0x7f9f5c0d1420 2024-05-30T16:58:09.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.850+0000 7f9f56000640 1 -- 172.21.15.67:0/264633329 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f9f4c007400 con 0x7f9f5c0d1420 2024-05-30T16:58:09.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.851+0000 7f9f56000640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9f4006da60 0x7f9f4006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:09.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.851+0000 7f9f56000640 1 -- 172.21.15.67:0/264633329 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f9f4c089fd0 con 0x7f9f5c0d1420 2024-05-30T16:58:09.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.851+0000 7f9f61e00640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9f4006da60 0x7f9f4006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:09.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.852+0000 7f9f61e00640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9f4006da60 0x7f9f4006ff20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f50034000 tx=0x7f9f5002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:09.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:09.855+0000 7f9f56000640 1 -- 172.21.15.67:0/264633329 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f5c1f5420 con 0x7f9f5c0d1420 2024-05-30T16:58:10.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.142+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92287", "target": ["mon-mgr", ""]}) -- 0x7f9f5c0cb2b0 con 0x7f9f4006da60 2024-05-30T16:58:10.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.180+0000 7f9f56000640 1 -- 172.21.15.67:0/264633329 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f5c0cb2b0 con 0x7f9f4006da60 2024-05-30T16:58:10.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9f4006da60 msgr2=0x7f9f4006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:10.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9f4006da60 0x7f9f4006ff20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f50034000 tx=0x7f9f5002e000 comp rx=0 tx=0).stop 2024-05-30T16:58:10.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c0d1420 msgr2=0x7f9f5c0b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:10.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c0d1420 0x7f9f5c0b7240 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c430 tx=0x7f9f4c00c8f0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 shutdown_connections 2024-05-30T16:58:10.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9f4006da60 0x7f9f4006ff20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f5c15e5a0 0x7f9f5c0b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f5c0d1420 0x7f9f5c0b7240 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 --2- 172.21.15.67:0/264633329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f5c0cfb40 0x7f9f5c157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.183+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 >> 172.21.15.67:0/264633329 conn(0x7f9f5c08bfc0 msgr2=0x7f9f5c15ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.184+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 shutdown_connections 2024-05-30T16:58:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.184+0000 7f9f63200640 1 -- 172.21.15.67:0/264633329 wait complete. 2024-05-30T16:58:10.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone922810 2024-05-30T16:58:10.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.494+0000 7fa947800640 1 Processor -- start 2024-05-30T16:58:10.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.495+0000 7fa947800640 1 -- start start 2024-05-30T16:58:10.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.496+0000 7fa947800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa940161ce0 0x7fa9400d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:10.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.496+0000 7fa947800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9401620c0 0x7fa9400d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:10.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.496+0000 7fa947800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9400d5370 con 0x7fa9400b5a60 2024-05-30T16:58:10.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.496+0000 7fa947800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa9400cfea0 con 0x7fa940161ce0 2024-05-30T16:58:10.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.496+0000 7fa947800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9400d0180 con 0x7fa9401620c0 2024-05-30T16:58:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.496+0000 7fa944e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa940161ce0 0x7fa9400d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.496+0000 7fa944e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa940161ce0 0x7fa9400d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35010/0 (socket says 172.21.15.67:35010) 2024-05-30T16:58:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.496+0000 7fa944e00640 1 -- 172.21.15.67:0/1852326039 learned_addr learned my addr 172.21.15.67:0/1852326039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.497+0000 7fa945800640 1 --2- 172.21.15.67:0/1852326039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9401620c0 0x7fa9400d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.497+0000 7fa944e00640 1 -- 172.21.15.67:0/1852326039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9401620c0 msgr2=0x7fa9400d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.497+0000 7fa944e00640 1 --2- 172.21.15.67:0/1852326039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9401620c0 0x7fa9400d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.497+0000 7fa944e00640 1 -- 172.21.15.67:0/1852326039 >> v1:172.21.15.169:6789/0 conn(0x7fa9400b5a60 legacy=0x7fa9401625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.497+0000 7fa944e00640 1 -- 172.21.15.67:0/1852326039 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9400d5020 con 0x7fa940161ce0 2024-05-30T16:58:10.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.498+0000 7fa944e00640 1 --2- 172.21.15.67:0/1852326039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa940161ce0 0x7fa9400d0f40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa934009870 tx=0x7fa93402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c0024998aea5c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:10.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.498+0000 7fa93f400640 1 -- 172.21.15.67:0/1852326039 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934037070 con 0x7fa940161ce0 2024-05-30T16:58:10.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.498+0000 7fa93f400640 1 -- 172.21.15.67:0/1852326039 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa934032440 con 0x7fa940161ce0 2024-05-30T16:58:10.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.498+0000 7fa93f400640 1 -- 172.21.15.67:0/1852326039 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934032810 con 0x7fa940161ce0 2024-05-30T16:58:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.499+0000 7fa947800640 1 -- 172.21.15.67:0/1852326039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa940161ce0 msgr2=0x7fa9400d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.499+0000 7fa947800640 1 --2- 172.21.15.67:0/1852326039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa940161ce0 0x7fa9400d0f40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa934009870 tx=0x7fa93402a1d0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.499+0000 7fa947800640 1 -- 172.21.15.67:0/1852326039 shutdown_connections 2024-05-30T16:58:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.499+0000 7fa947800640 1 --2- 172.21.15.67:0/1852326039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9401620c0 0x7fa9400d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.499+0000 7fa947800640 1 --2- 172.21.15.67:0/1852326039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa940161ce0 0x7fa9400d0f40 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.499+0000 7fa947800640 1 -- 172.21.15.67:0/1852326039 >> 172.21.15.67:0/1852326039 conn(0x7fa94008bfc0 msgr2=0x7fa9400d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.499+0000 7fa947800640 1 -- 172.21.15.67:0/1852326039 shutdown_connections 2024-05-30T16:58:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.499+0000 7fa947800640 1 -- 172.21.15.67:0/1852326039 wait complete. 2024-05-30T16:58:10.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.501+0000 7fa947800640 1 Processor -- start 2024-05-30T16:58:10.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.501+0000 7fa947800640 1 -- start start 2024-05-30T16:58:10.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa947800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9400b5a60 0x7fa94015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa947800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa940161ce0 0x7fa940159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa947800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9401620c0 0x7fa9400ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa947800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400b54c0 con 0x7fa9401620c0 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa947800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400d3690 con 0x7fa940161ce0 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa947800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9400b5640 con 0x7fa9400b5a60 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa945800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9401620c0 0x7fa9400ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa944e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9400b5a60 0x7fa94015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa945800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9401620c0 0x7fa9400ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35026/0 (socket says 172.21.15.67:35026) 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa945800640 1 -- 172.21.15.67:0/4145438381 learned_addr learned my addr 172.21.15.67:0/4145438381 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.502+0000 7fa93fe00640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa940161ce0 0x7fa940159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa945800640 1 -- 172.21.15.67:0/4145438381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9400b5a60 msgr2=0x7fa94015da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa945800640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9400b5a60 0x7fa94015da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa945800640 1 -- 172.21.15.67:0/4145438381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa940161ce0 msgr2=0x7fa940159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa945800640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa940161ce0 0x7fa940159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa945800640 1 -- 172.21.15.67:0/4145438381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94015df50 con 0x7fa9401620c0 2024-05-30T16:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa944e00640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9400b5a60 0x7fa94015da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa945800640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9401620c0 0x7fa9400ca0b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa938007d80 tx=0x7fa93800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa93cc00640 1 -- 172.21.15.67:0/4145438381 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938014420 con 0x7fa9401620c0 2024-05-30T16:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa93cc00640 1 -- 172.21.15.67:0/4145438381 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938019070 con 0x7fa9401620c0 2024-05-30T16:58:10.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.503+0000 7fa93cc00640 1 -- 172.21.15.67:0/4145438381 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938014800 con 0x7fa9401620c0 2024-05-30T16:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.504+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa940159fa0 con 0x7fa9401620c0 2024-05-30T16:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.504+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9400cc430 con 0x7fa9401620c0 2024-05-30T16:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.505+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9400690f0 con 0x7fa9401620c0 2024-05-30T16:58:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.506+0000 7fa93cc00640 1 -- 172.21.15.67:0/4145438381 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fa9380149a0 con 0x7fa9401620c0 2024-05-30T16:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.507+0000 7fa93cc00640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa92006da90 0x7fa92006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.507+0000 7fa93cc00640 1 -- 172.21.15.67:0/4145438381 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7fa93808b130 con 0x7fa9401620c0 2024-05-30T16:58:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.507+0000 7fa944e00640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa92006da90 0x7fa92006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:10.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.508+0000 7fa944e00640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa92006da90 0x7fa92006ff50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa934034000 tx=0x7fa93402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.512+0000 7fa93cc00640 1 -- 172.21.15.67:0/4145438381 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa93805a5e0 con 0x7fa9401620c0 2024-05-30T16:58:10.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.741+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone922810", "target": ["mon-mgr", ""]}) -- 0x7fa9400d1d60 con 0x7fa92006da90 2024-05-30T16:58:10.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.771+0000 7fa93cc00640 1 -- 172.21.15.67:0/4145438381 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa9400d1d60 con 0x7fa92006da90 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa92006da90 msgr2=0x7fa92006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa92006da90 0x7fa92006ff50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa934034000 tx=0x7fa93402e000 comp rx=0 tx=0).stop 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9401620c0 msgr2=0x7fa9400ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9401620c0 0x7fa9400ca0b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa938007d80 tx=0x7fa93800e4b0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 shutdown_connections 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa92006da90 0x7fa92006ff50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9401620c0 0x7fa9400ca0b0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa940161ce0 0x7fa940159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 --2- 172.21.15.67:0/4145438381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9400b5a60 0x7fa94015da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.774+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 >> 172.21.15.67:0/4145438381 conn(0x7fa94008bfc0 msgr2=0x7fa94015e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:10.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.775+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 shutdown_connections 2024-05-30T16:58:10.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10.775+0000 7fa947800640 1 -- 172.21.15.67:0/4145438381 wait complete. 2024-05-30T16:58:10.797 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-30T16:58:10.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-30T16:58:10.902 INFO:teuthology.orchestra.run.smithi067.stdout:2 2024-05-30T16:58:10.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:58:10.911 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T16:58:10.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:58:11.026 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:58:11.150 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T16:58:11.150 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:58:11.150 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:58:11.150 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:11.209 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:58:11.209 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:11.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T16:58:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.531+0000 7fbddd000640 1 Processor -- start 2024-05-30T16:58:11.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.532+0000 7fbddd000640 1 -- start start 2024-05-30T16:58:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.533+0000 7fbddd000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd8161fe0 0x7fbdd8073020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.533+0000 7fbddd000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbdd80cfda0 0x7fbdd8073560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.534+0000 7fbddd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbdd80b10f0 0x7fbdd80af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.534+0000 7fbddd000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd80b0ed0 con 0x7fbdd80b10f0 2024-05-30T16:58:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.534+0000 7fbddd000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd80caac0 con 0x7fbdd80cfda0 2024-05-30T16:58:11.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.534+0000 7fbddd000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd80b1770 con 0x7fbdd8161fe0 2024-05-30T16:58:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.534+0000 7fbdd5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd8161fe0 0x7fbdd8073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.534+0000 7fbdd5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd8161fe0 0x7fbdd8073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:48616/0 (socket says 172.21.15.67:48616) 2024-05-30T16:58:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.534+0000 7fbdd5e00640 1 -- 172.21.15.67:0/3713712898 learned_addr learned my addr 172.21.15.67:0/3713712898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.534+0000 7fbdd5400640 1 --2- 172.21.15.67:0/3713712898 >> v2:172.21.15.169:3300/0 conn(0x7fbdd80cfda0 0x7fbdd8073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.535+0000 7fbdd5e00640 1 -- 172.21.15.67:0/3713712898 >> v2:172.21.15.169:3300/0 conn(0x7fbdd80cfda0 msgr2=0x7fbdd8073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:11.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.535+0000 7fbdd5e00640 1 --2- 172.21.15.67:0/3713712898 >> v2:172.21.15.169:3300/0 conn(0x7fbdd80cfda0 0x7fbdd8073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.535+0000 7fbdd5e00640 1 -- 172.21.15.67:0/3713712898 >> v2:172.21.15.67:3300/0 conn(0x7fbdd80b10f0 msgr2=0x7fbdd80af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:11.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.535+0000 7fbdd5e00640 1 --2- 172.21.15.67:0/3713712898 >> v2:172.21.15.67:3300/0 conn(0x7fbdd80b10f0 0x7fbdd80af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.535+0000 7fbdd5e00640 1 -- 172.21.15.67:0/3713712898 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd80b1a50 con 0x7fbdd8161fe0 2024-05-30T16:58:11.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.536+0000 7fbdd5e00640 1 --2- 172.21.15.67:0/3713712898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd8161fe0 0x7fbdd8073020 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdd806d470 tx=0x7fbdd002a600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1faa8500695a15c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:11.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.537+0000 7fbdd4a00640 1 -- 172.21.15.67:0/3713712898 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0036070 con 0x7fbdd8161fe0 2024-05-30T16:58:11.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.537+0000 7fbdd4a00640 1 -- 172.21.15.67:0/3713712898 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd0031600 con 0x7fbdd8161fe0 2024-05-30T16:58:11.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.537+0000 7fbdd4a00640 1 -- 172.21.15.67:0/3713712898 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00319d0 con 0x7fbdd8161fe0 2024-05-30T16:58:11.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.538+0000 7fbddd000640 1 -- 172.21.15.67:0/3713712898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd8161fe0 msgr2=0x7fbdd8073020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:11.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.538+0000 7fbddd000640 1 --2- 172.21.15.67:0/3713712898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd8161fe0 0x7fbdd8073020 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdd806d470 tx=0x7fbdd002a600 comp rx=0 tx=0).stop 2024-05-30T16:58:11.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.538+0000 7fbddd000640 1 -- 172.21.15.67:0/3713712898 shutdown_connections 2024-05-30T16:58:11.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.538+0000 7fbddd000640 1 --2- 172.21.15.67:0/3713712898 >> v2:172.21.15.67:3300/0 conn(0x7fbdd80b10f0 0x7fbdd80af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.538+0000 7fbddd000640 1 --2- 172.21.15.67:0/3713712898 >> v2:172.21.15.169:3300/0 conn(0x7fbdd80cfda0 0x7fbdd8073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.538+0000 7fbddd000640 1 --2- 172.21.15.67:0/3713712898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd8161fe0 0x7fbdd8073020 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.538+0000 7fbddd000640 1 -- 172.21.15.67:0/3713712898 >> 172.21.15.67:0/3713712898 conn(0x7fbdd80d0260 msgr2=0x7fbdd80d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:11.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.538+0000 7fbddd000640 1 -- 172.21.15.67:0/3713712898 shutdown_connections 2024-05-30T16:58:11.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.538+0000 7fbddd000640 1 -- 172.21.15.67:0/3713712898 wait complete. 2024-05-30T16:58:11.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.541+0000 7fbddd000640 1 Processor -- start 2024-05-30T16:58:11.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.541+0000 7fbddd000640 1 -- start start 2024-05-30T16:58:11.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.542+0000 7fbddd000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b10f0 0x7fbdd81f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:11.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.542+0000 7fbddd000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80cfda0 0x7fbdd80c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:11.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.542+0000 7fbddd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd8161fe0 0x7fbdd80b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:11.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.542+0000 7fbddd000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd80b00b0 con 0x7fbdd8161fe0 2024-05-30T16:58:11.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.542+0000 7fbddd000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd80aff30 con 0x7fbdd80cfda0 2024-05-30T16:58:11.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.542+0000 7fbddd000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd80b0230 con 0x7fbdd80b10f0 2024-05-30T16:58:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.543+0000 7fbdd5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80cfda0 0x7fbdd80c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.543+0000 7fbdd5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80cfda0 0x7fbdd80c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36818/0 (socket says 172.21.15.67:36818) 2024-05-30T16:58:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.543+0000 7fbdd5400640 1 -- 172.21.15.67:0/3744041995 learned_addr learned my addr 172.21.15.67:0/3744041995 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.543+0000 7fbdd5e00640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b10f0 0x7fbdd81f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.543+0000 7fbdd6800640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd8161fe0 0x7fbdd80b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:11.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.544+0000 7fbdd5400640 1 -- 172.21.15.67:0/3744041995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b10f0 msgr2=0x7fbdd81f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:11.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.544+0000 7fbdd5400640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b10f0 0x7fbdd81f8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.544+0000 7fbdd5400640 1 -- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd8161fe0 msgr2=0x7fbdd80b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:11.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.544+0000 7fbdd5400640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd8161fe0 0x7fbdd80b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.544+0000 7fbdd5400640 1 -- 172.21.15.67:0/3744041995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd80c98f0 con 0x7fbdd80cfda0 2024-05-30T16:58:11.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.544+0000 7fbdd6800640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd8161fe0 0x7fbdd80b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:58:11.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.545+0000 7fbdd5400640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80cfda0 0x7fbdd80c8f30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400e9b0 tx=0x7fbdc400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:11.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.545+0000 7fbdce000640 1 -- 172.21.15.67:0/3744041995 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4017070 con 0x7fbdd80cfda0 2024-05-30T16:58:11.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.546+0000 7fbdce000640 1 -- 172.21.15.67:0/3744041995 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc400c9a0 con 0x7fbdd80cfda0 2024-05-30T16:58:11.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.546+0000 7fbdce000640 1 -- 172.21.15.67:0/3744041995 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4017070 con 0x7fbdd80cfda0 2024-05-30T16:58:11.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.547+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd80c9bd0 con 0x7fbdd80cfda0 2024-05-30T16:58:11.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.547+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd80ca190 con 0x7fbdd80cfda0 2024-05-30T16:58:11.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.548+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd80689e0 con 0x7fbdd80cfda0 2024-05-30T16:58:11.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.550+0000 7fbdce000640 1 -- 172.21.15.67:0/3744041995 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fbdc401a070 con 0x7fbdd80cfda0 2024-05-30T16:58:11.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.552+0000 7fbdce000640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbdb806dc90 0x7fbdb8070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:11.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.552+0000 7fbdce000640 1 -- 172.21.15.67:0/3744041995 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7fbdc40581b0 con 0x7fbdd80cfda0 2024-05-30T16:58:11.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.554+0000 7fbdd5e00640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbdb806dc90 0x7fbdb8070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:11.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.556+0000 7fbdd5e00640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbdb806dc90 0x7fbdb8070150 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0033000 tx=0x7fbdd0009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:11.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.560+0000 7fbdce000640 1 -- 172.21.15.67:0/3744041995 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbdc405a110 con 0x7fbdd80cfda0 2024-05-30T16:58:11.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.825+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbdd80af780 con 0x7fbdd80cfda0 2024-05-30T16:58:11.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.827+0000 7fbdce000640 1 -- 172.21.15.67:0/3744041995 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7fbdc405e410 con 0x7fbdd80cfda0 2024-05-30T16:58:11.828 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T16:58:11.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbdb806dc90 msgr2=0x7fbdb8070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:11.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbdb806dc90 0x7fbdb8070150 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0033000 tx=0x7fbdd0009000 comp rx=0 tx=0).stop 2024-05-30T16:58:11.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80cfda0 msgr2=0x7fbdd80c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:11.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80cfda0 0x7fbdd80c8f30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400e9b0 tx=0x7fbdc400ee70 comp rx=0 tx=0).stop 2024-05-30T16:58:11.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 shutdown_connections 2024-05-30T16:58:11.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbdb806dc90 0x7fbdb8070150 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd8161fe0 0x7fbdd80b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80cfda0 0x7fbdd80c8f30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 --2- 172.21.15.67:0/3744041995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b10f0 0x7fbdd81f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 >> 172.21.15.67:0/3744041995 conn(0x7fbdd80d0260 msgr2=0x7fbdd80749f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.829+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 shutdown_connections 2024-05-30T16:58:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:11.830+0000 7fbddd000640 1 -- 172.21.15.67:0/3744041995 wait complete. 2024-05-30T16:58:11.848 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T16:58:11.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:11.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-30T16:58:11.927 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T16:58:11.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:11.927 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-30T16:58:11.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[60682]: fuse finished with error 0 and tester_r 0 2024-05-30T16:58:12.134 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:58:12.135 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:58:12.135 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:12.194 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:58:12.195 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:58:12.195 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:58:12.195 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:12.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:58:12.257 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T16:58:12.257 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:58:12.257 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:12.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:58:12.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:58:12.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.578+0000 7f9cd6e00640 1 Processor -- start 2024-05-30T16:58:12.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.578+0000 7f9cd6e00640 1 -- start start 2024-05-30T16:58:12.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.580+0000 7f9cd6e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9cd00d04a0 0x7f9cd00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:12.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.580+0000 7f9cd6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd0162e70 0x7f9cd00ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:12.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.580+0000 7f9cd6e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd00b4350 con 0x7f9cd006f6e0 2024-05-30T16:58:12.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.580+0000 7f9cd6e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd00b3ec0 con 0x7f9cd00d04a0 2024-05-30T16:58:12.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.580+0000 7f9cd6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd00b32d0 con 0x7f9cd0162e70 2024-05-30T16:58:12.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.581+0000 7f9ccf400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f9cd006f6e0 0x7f9cd00cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:50424/0 (socket says 172.21.15.67:50424) 2024-05-30T16:58:12.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.581+0000 7f9ccf400640 1 -- 172.21.15.67:0/3401798999 learned_addr learned my addr 172.21.15.67:0/3401798999 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:12.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.581+0000 7f9ccfe00640 1 --2- 172.21.15.67:0/3401798999 >> v2:172.21.15.67:3300/0 conn(0x7f9cd00d04a0 0x7f9cd00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:12.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.581+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 827345774 0 0) 0x7f9cd00b4350 con 0x7f9cd006f6e0 2024-05-30T16:58:12.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.582+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cb4003520 con 0x7f9cd006f6e0 2024-05-30T16:58:12.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.583+0000 7f9cd4e00640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd0162e70 0x7f9cd00ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:12.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.583+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 779559933 0 0) 0x7f9cb4003520 con 0x7f9cd006f6e0 2024-05-30T16:58:12.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.583+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd0162e70 msgr2=0x7f9cd00ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:12.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.583+0000 7f9ccea00640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd0162e70 0x7f9cd00ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.583+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 >> v2:172.21.15.67:3300/0 conn(0x7f9cd00d04a0 msgr2=0x7f9cd00d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:12.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.583+0000 7f9ccea00640 1 --2- 172.21.15.67:0/3401798999 >> v2:172.21.15.67:3300/0 conn(0x7f9cd00d04a0 0x7f9cd00d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.583+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd00b3590 con 0x7f9cd006f6e0 2024-05-30T16:58:12.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.583+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9cbc001670 con 0x7f9cd006f6e0 2024-05-30T16:58:12.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 >> v1:172.21.15.169:6789/0 conn(0x7f9cd006f6e0 legacy=0x7f9cd00cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9ccea00640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb40039b0 0x7f9cb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9ccea00640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd0162e70 0x7f9cb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9ccea00640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb4004d40 0x7f9cb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd00b3ec0 con 0x7f9cb4004d40 2024-05-30T16:58:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd00b32d0 con 0x7f9cb40039b0 2024-05-30T16:58:12.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb400bac0 con 0x7f9cd0162e70 2024-05-30T16:58:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9ccf400640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd0162e70 0x7f9cb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9ccfe00640 1 --2- 172.21.15.67:0/3401798999 >> v2:172.21.15.67:3300/0 conn(0x7f9cd00d04a0 0x7f9cd00d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:58:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.584+0000 7f9cd4e00640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb4004d40 0x7f9cb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.585+0000 7f9ccfe00640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb40039b0 0x7f9cb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.585+0000 7f9ccf400640 1 -- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb40039b0 msgr2=0x7f9cb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.585+0000 7f9ccf400640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cb40039b0 0x7f9cb4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.585+0000 7f9ccf400640 1 -- 172.21.15.67:0/3401798999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb4004d40 msgr2=0x7f9cb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:12.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.586+0000 7f9ccf400640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb4004d40 0x7f9cb40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.586+0000 7f9ccf400640 1 -- 172.21.15.67:0/3401798999 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd00b3590 con 0x7f9cd0162e70 2024-05-30T16:58:12.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.587+0000 7f9cd4e00640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb4004d40 0x7f9cb40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:58:12.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.590+0000 7f9ccf400640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd0162e70 0x7f9cb4004800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9cd006c920 tx=0x7f9cb4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989df98bf45f2e8f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:12.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.590+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc007d90 con 0x7f9cd0162e70 2024-05-30T16:58:12.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.590+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cbc005ad0 con 0x7f9cd0162e70 2024-05-30T16:58:12.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.590+0000 7f9ccea00640 1 -- 172.21.15.67:0/3401798999 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc007070 con 0x7f9cd0162e70 2024-05-30T16:58:12.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.591+0000 7f9cd6e00640 1 -- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd0162e70 msgr2=0x7f9cb4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:12.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.591+0000 7f9cd6e00640 1 --2- 172.21.15.67:0/3401798999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd0162e70 0x7f9cb4004800 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9cd006c920 tx=0x7f9cb4005830 comp rx=0 tx=0).stop 2024-05-30T16:58:12.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.591+0000 7f9cd4e00640 1 -- 172.21.15.67:0/3401798999 reap_dead start 2024-05-30T16:58:12.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.591+0000 7f9cd6e00640 1 -- 172.21.15.67:0/3401798999 shutdown_connections 2024-05-30T16:58:12.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.591+0000 7f9cd6e00640 1 -- 172.21.15.67:0/3401798999 >> 172.21.15.67:0/3401798999 conn(0x7f9cd006bae0 msgr2=0x7f9cd006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:12.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.592+0000 7f9cd6e00640 1 -- 172.21.15.67:0/3401798999 shutdown_connections 2024-05-30T16:58:12.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.592+0000 7f9cd6e00640 1 -- 172.21.15.67:0/3401798999 wait complete. 2024-05-30T16:58:12.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.594+0000 7f9cd6e00640 1 Processor -- start 2024-05-30T16:58:12.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.594+0000 7f9cd6e00640 1 -- start start 2024-05-30T16:58:12.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.595+0000 7f9cd6e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cd0162e70 0x7f9cd00c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:12.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.595+0000 7f9cd6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd01561f0 0x7f9cd0164f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:12.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.595+0000 7f9cd6e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0165460 0x7f9cd00c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.595+0000 7f9cd6e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd00d0dc0 con 0x7f9cd0162e70 2024-05-30T16:58:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.595+0000 7f9cd6e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd00b3b50 con 0x7f9cd0165460 2024-05-30T16:58:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.595+0000 7f9cd6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd00ce230 con 0x7f9cd01561f0 2024-05-30T16:58:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.595+0000 7f9ccfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cd0162e70 0x7f9cd00c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.595+0000 7f9ccfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cd0162e70 0x7f9cd00c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43872/0 (socket says 172.21.15.67:43872) 2024-05-30T16:58:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.595+0000 7f9ccfe00640 1 -- 172.21.15.67:0/2894929695 learned_addr learned my addr 172.21.15.67:0/2894929695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:12.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.596+0000 7f9ccf400640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd01561f0 0x7f9cd0164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:12.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.596+0000 7f9cd4e00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0165460 0x7f9cd00c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:12.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.596+0000 7f9ccf400640 1 -- 172.21.15.67:0/2894929695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0165460 msgr2=0x7f9cd00c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:12.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.596+0000 7f9ccf400640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0165460 0x7f9cd00c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.596+0000 7f9ccf400640 1 -- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cd0162e70 msgr2=0x7f9cd00c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:12.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.597+0000 7f9ccf400640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cd0162e70 0x7f9cd00c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.597+0000 7f9ccf400640 1 -- 172.21.15.67:0/2894929695 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd00c4ac0 con 0x7f9cd01561f0 2024-05-30T16:58:12.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.597+0000 7f9ccfe00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cd0162e70 0x7f9cd00c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:58:12.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.598+0000 7f9ccf400640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd01561f0 0x7f9cd0164f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc030000 tx=0x7f9cbc001970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:12.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.598+0000 7f9cc7e00640 1 -- 172.21.15.67:0/2894929695 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc042070 con 0x7f9cd01561f0 2024-05-30T16:58:12.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.598+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd00c4da0 con 0x7f9cd01561f0 2024-05-30T16:58:12.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.599+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd00c5360 con 0x7f9cd01561f0 2024-05-30T16:58:12.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.599+0000 7f9cc7e00640 1 -- 172.21.15.67:0/2894929695 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cbc007480 con 0x7f9cd01561f0 2024-05-30T16:58:12.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.599+0000 7f9cc7e00640 1 -- 172.21.15.67:0/2894929695 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc03e450 con 0x7f9cd01561f0 2024-05-30T16:58:12.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.600+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd0165860 con 0x7f9cd01561f0 2024-05-30T16:58:12.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.601+0000 7f9cc7e00640 1 -- 172.21.15.67:0/2894929695 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f9cbc045070 con 0x7f9cd01561f0 2024-05-30T16:58:12.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.604+0000 7f9cc7e00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cb006db30 0x7f9cb006fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:12.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.604+0000 7f9cc7e00640 1 -- 172.21.15.67:0/2894929695 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f9cbc0abb00 con 0x7f9cd01561f0 2024-05-30T16:58:12.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.604+0000 7f9ccfe00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cb006db30 0x7f9cb006fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:12.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.606+0000 7f9ccfe00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cb006db30 0x7f9cb006fff0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9cd015fc10 tx=0x7f9cc0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:12.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.614+0000 7f9cc7e00640 1 -- 172.21.15.67:0/2894929695 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9cbc07b100 con 0x7f9cd01561f0 2024-05-30T16:58:12.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.974+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cd00689e0 con 0x7f9cd01561f0 2024-05-30T16:58:12.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.975+0000 7f9cc7e00640 1 -- 172.21.15.67:0/2894929695 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5189 (secure 0 0 0) 0x7f9cbc07f400 con 0x7f9cd01561f0 2024-05-30T16:58:12.976 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:12.976 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5171,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1912096869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1912096869},{"type":"v1","addr":"172.21.15.169:6835","nonce":1912096869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5608,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3963396329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3963396329},{"type":"v1","addr":"172.21.15.67:6835","nonce":3963396329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2169821644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2169821644},{"type":"v1","addr":"172.21.15.169:6834","nonce":2169821644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:54:56.078869+0000","modified":"2024-05-30T16:54:59.175686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6558":{"gid":6558,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/2706164583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2706164583},{"type":"v1","addr":"172.21.15.67:6837","nonce":2706164583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6558,"qdb_cluster":[6558]},"id":5}]} 2024-05-30T16:58:12.976 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 46 2024-05-30T16:58:12.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.977+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cb006db30 msgr2=0x7f9cb006fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:12.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.977+0000 7f9cd6e00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cb006db30 0x7f9cb006fff0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9cd015fc10 tx=0x7f9cc0008040 comp rx=0 tx=0).stop 2024-05-30T16:58:12.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.977+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd01561f0 msgr2=0x7f9cd0164f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:12.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.977+0000 7f9cd6e00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd01561f0 0x7f9cd0164f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc030000 tx=0x7f9cbc001970 comp rx=0 tx=0).stop 2024-05-30T16:58:12.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.978+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 shutdown_connections 2024-05-30T16:58:12.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.978+0000 7f9cd6e00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cb006db30 0x7f9cb006fff0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.978+0000 7f9cd6e00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cd0165460 0x7f9cd00c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.978+0000 7f9cd6e00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9cd01561f0 0x7f9cd0164f20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.978+0000 7f9cd6e00640 1 --2- 172.21.15.67:0/2894929695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cd0162e70 0x7f9cd00c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:12.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.978+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 >> 172.21.15.67:0/2894929695 conn(0x7f9cd006bae0 msgr2=0x7f9cd00cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:12.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.978+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 shutdown_connections 2024-05-30T16:58:12.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:12.978+0000 7f9cd6e00640 1 -- 172.21.15.67:0/2894929695 wait complete. 2024-05-30T16:58:12.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:58:13.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.238+0000 7f5982e00640 1 Processor -- start 2024-05-30T16:58:13.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.238+0000 7f5982e00640 1 -- start start 2024-05-30T16:58:13.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.240+0000 7f5982e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f597c162e70 0x7f597c16d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.240+0000 7f5982e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f597c0b4350 con 0x7f597c06f6e0 2024-05-30T16:58:13.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.240+0000 7f5982e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f597c0b32d0 con 0x7f597c0d04a0 2024-05-30T16:58:13.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.240+0000 7f5982e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f597c0b3ec0 con 0x7f597c162e70 2024-05-30T16:58:13.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.241+0000 7f597be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f597c0d04a0 0x7f597c0d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:50428/0 (socket says 172.21.15.67:50428) 2024-05-30T16:58:13.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.241+0000 7f597be00640 1 -- 172.21.15.67:0/3554720185 learned_addr learned my addr 172.21.15.67:0/3554720185 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:13.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f5980e00640 1 --2- 172.21.15.67:0/3554720185 >> v2:172.21.15.169:3300/0 conn(0x7f597c162e70 0x7f597c16d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2896362004 0 0) 0x7f597c0b32d0 con 0x7f597c0d04a0 2024-05-30T16:58:13.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5960003520 con 0x7f597c0d04a0 2024-05-30T16:58:13.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1420401279 0 0) 0x7f5960003520 con 0x7f597c0d04a0 2024-05-30T16:58:13.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 >> v2:172.21.15.169:3300/0 conn(0x7f597c162e70 msgr2=0x7f597c16d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:13.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 --2- 172.21.15.67:0/3554720185 >> v2:172.21.15.169:3300/0 conn(0x7f597c162e70 0x7f597c16d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 >> v1:172.21.15.67:6789/0 conn(0x7f597c06f6e0 legacy=0x7f597c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c0b3590 con 0x7f597c0d04a0 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f596c001670 con 0x7f597c0d04a0 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 >> v1:172.21.15.169:6789/0 conn(0x7f597c0d04a0 legacy=0x7f597c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59600039b0 0x7f5960003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59600043b0 0x7f5960004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5960005120 0x7f596000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f597c0b3ec0 con 0x7f5960005120 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f596000c060 con 0x7f59600039b0 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.242+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f596000c260 con 0x7f59600043b0 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.243+0000 7f5980e00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5960005120 0x7f596000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.243+0000 7f597be00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59600039b0 0x7f5960003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.243+0000 7f597b400640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59600043b0 0x7f5960004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.244+0000 7f597be00640 1 -- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59600043b0 msgr2=0x7f5960004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:13.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.244+0000 7f597be00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59600043b0 0x7f5960004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.244+0000 7f597be00640 1 -- 172.21.15.67:0/3554720185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5960005120 msgr2=0x7f596000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:13.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.245+0000 7f597be00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5960005120 0x7f596000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.245+0000 7f597be00640 1 -- 172.21.15.67:0/3554720185 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c0b3590 con 0x7f59600039b0 2024-05-30T16:58:13.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.245+0000 7f5980e00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5960005120 0x7f596000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:58:13.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.245+0000 7f5980e00640 1 -- 172.21.15.67:0/3554720185 reap_dead start 2024-05-30T16:58:13.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.246+0000 7f597be00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59600039b0 0x7f5960003d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c210 tx=0x7f5960005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=513830bdbb30aad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:13.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.247+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c02e070 con 0x7f59600039b0 2024-05-30T16:58:13.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.247+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596c005af0 con 0x7f59600039b0 2024-05-30T16:58:13.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.247+0000 7f597aa00640 1 -- 172.21.15.67:0/3554720185 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c007d30 con 0x7f59600039b0 2024-05-30T16:58:13.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.248+0000 7f5982e00640 1 -- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59600039b0 msgr2=0x7f5960003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.248+0000 7f5982e00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59600039b0 0x7f5960003d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c210 tx=0x7f5960005830 comp rx=0 tx=0).stop 2024-05-30T16:58:13.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.249+0000 7f5982e00640 1 -- 172.21.15.67:0/3554720185 shutdown_connections 2024-05-30T16:58:13.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.249+0000 7f5982e00640 1 --2- 172.21.15.67:0/3554720185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59600039b0 0x7f5960003d90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.249+0000 7f5982e00640 1 -- 172.21.15.67:0/3554720185 >> 172.21.15.67:0/3554720185 conn(0x7f597c06bae0 msgr2=0x7f597c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:13.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.250+0000 7f5982e00640 1 -- 172.21.15.67:0/3554720185 shutdown_connections 2024-05-30T16:58:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.250+0000 7f5982e00640 1 -- 172.21.15.67:0/3554720185 wait complete. 2024-05-30T16:58:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.253+0000 7f5982e00640 1 Processor -- start 2024-05-30T16:58:13.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.253+0000 7f5982e00640 1 -- start start 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f5982e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59600039b0 0x7f597c156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f5982e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c163c60 0x7f597c164b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f5982e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f597c0c5bf0 0x7f597c0c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f5982e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f597c0d0880 con 0x7f597c0c5bf0 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f5982e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f597c0b3b50 con 0x7f597c163c60 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f5982e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f597c0d0a00 con 0x7f59600039b0 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f5980e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f597c0c5bf0 0x7f597c0c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f597b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c163c60 0x7f597c164b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f5980e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f597c0c5bf0 0x7f597c0c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43888/0 (socket says 172.21.15.67:43888) 2024-05-30T16:58:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.255+0000 7f5980e00640 1 -- 172.21.15.67:0/834199862 learned_addr learned my addr 172.21.15.67:0/834199862 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.256+0000 7f597be00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59600039b0 0x7f597c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.256+0000 7f5980e00640 1 -- 172.21.15.67:0/834199862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59600039b0 msgr2=0x7f597c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.256+0000 7f5980e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59600039b0 0x7f597c156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.256+0000 7f5980e00640 1 -- 172.21.15.67:0/834199862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c163c60 msgr2=0x7f597c164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.256+0000 7f5980e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c163c60 0x7f597c164b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.256+0000 7f5980e00640 1 -- 172.21.15.67:0/834199862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c0c5170 con 0x7f597c0c5bf0 2024-05-30T16:58:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.257+0000 7f5980e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f597c0c5bf0 0x7f597c0c60a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f597c06dff0 tx=0x7f5974010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:13.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.257+0000 7f5973e00640 1 -- 172.21.15.67:0/834199862 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974003990 con 0x7f597c0c5bf0 2024-05-30T16:58:13.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.257+0000 7f5973e00640 1 -- 172.21.15.67:0/834199862 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597401b070 con 0x7f597c0c5bf0 2024-05-30T16:58:13.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.257+0000 7f5973e00640 1 -- 172.21.15.67:0/834199862 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974003d90 con 0x7f597c0c5bf0 2024-05-30T16:58:13.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.258+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c1641d0 con 0x7f597c0c5bf0 2024-05-30T16:58:13.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.258+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c0c53f0 con 0x7f597c0c5bf0 2024-05-30T16:58:13.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.259+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c1646f0 con 0x7f597c0c5bf0 2024-05-30T16:58:13.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.261+0000 7f5973e00640 1 -- 172.21.15.67:0/834199862 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f5974004030 con 0x7f597c0c5bf0 2024-05-30T16:58:13.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.263+0000 7f5973e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f595c06dae0 0x7f595c06ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.263+0000 7f5973e00640 1 -- 172.21.15.67:0/834199862 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f597408b770 con 0x7f597c0c5bf0 2024-05-30T16:58:13.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.264+0000 7f597be00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f595c06dae0 0x7f595c06ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.265+0000 7f597be00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f595c06dae0 0x7f595c06ffa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f596c033000 tx=0x7f596c02f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:13.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.272+0000 7f5973e00640 1 -- 172.21.15.67:0/834199862 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f597c0c5170 con 0x7f597c0c5bf0 2024-05-30T16:58:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.590+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f597c0689e0 con 0x7f597c0c5bf0 2024-05-30T16:58:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.591+0000 7f5973e00640 1 -- 172.21.15.67:0/834199862 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5189 (secure 0 0 0) 0x7f597405b2a0 con 0x7f597c0c5bf0 2024-05-30T16:58:13.592 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:13.592 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5171,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1912096869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1912096869},{"type":"v1","addr":"172.21.15.169:6835","nonce":1912096869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5608,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3963396329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3963396329},{"type":"v1","addr":"172.21.15.67:6835","nonce":3963396329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2169821644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2169821644},{"type":"v1","addr":"172.21.15.169:6834","nonce":2169821644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:54:56.078869+0000","modified":"2024-05-30T16:54:59.175686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6558":{"gid":6558,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/2706164583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2706164583},{"type":"v1","addr":"172.21.15.67:6837","nonce":2706164583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6558,"qdb_cluster":[6558]},"id":5}]} 2024-05-30T16:58:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 46 2024-05-30T16:58:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.593+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f595c06dae0 msgr2=0x7f595c06ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.593+0000 7f5982e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f595c06dae0 0x7f595c06ffa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f596c033000 tx=0x7f596c02f000 comp rx=0 tx=0).stop 2024-05-30T16:58:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f597c0c5bf0 msgr2=0x7f597c0c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f597c0c5bf0 0x7f597c0c60a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f597c06dff0 tx=0x7f5974010590 comp rx=0 tx=0).stop 2024-05-30T16:58:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 shutdown_connections 2024-05-30T16:58:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f595c06dae0 0x7f595c06ffa0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f597c0c5bf0 0x7f597c0c60a0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c163c60 0x7f597c164b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 --2- 172.21.15.67:0/834199862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59600039b0 0x7f597c156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 >> 172.21.15.67:0/834199862 conn(0x7f597c06bae0 msgr2=0x7f597c072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 shutdown_connections 2024-05-30T16:58:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.594+0000 7f5982e00640 1 -- 172.21.15.67:0/834199862 wait complete. 2024-05-30T16:58:13.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:58:13.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:58:13.660 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:58:13.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.789+0000 7f508a800640 1 Processor -- start 2024-05-30T16:58:13.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.790+0000 7f508a800640 1 -- start start 2024-05-30T16:58:13.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.791+0000 7f508a800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f50840d0780 0x7f508409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.791+0000 7f508a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f508409af70 0x7f508415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f508a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f508409b380 con 0x7f50840b37a0 2024-05-30T16:58:13.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f508a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f50840d0ef0 con 0x7f508409af70 2024-05-30T16:58:13.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f508a800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f50840d11d0 con 0x7f50840d0780 2024-05-30T16:58:13.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f5083400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f50840b37a0 0x7f508406d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54324/0 (socket says 172.21.15.67:54324) 2024-05-30T16:58:13.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f5083400640 1 -- 172.21.15.67:0/2318284939 learned_addr learned my addr 172.21.15.67:0/2318284939 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:13.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f5083e00640 1 --2- 172.21.15.67:0/2318284939 >> v2:172.21.15.67:3300/0 conn(0x7f508409af70 0x7f508415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f5082a00640 1 --2- 172.21.15.67:0/2318284939 >> v2:172.21.15.169:3300/0 conn(0x7f50840d0780 0x7f508409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f5083e00640 1 -- 172.21.15.67:0/2318284939 >> v2:172.21.15.169:3300/0 conn(0x7f50840d0780 msgr2=0x7f508409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f5083e00640 1 --2- 172.21.15.67:0/2318284939 >> v2:172.21.15.169:3300/0 conn(0x7f50840d0780 0x7f508409aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f5083e00640 1 -- 172.21.15.67:0/2318284939 >> v1:172.21.15.67:6789/0 conn(0x7f50840b37a0 legacy=0x7f508406d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.792+0000 7f5083e00640 1 -- 172.21.15.67:0/2318284939 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50840d14b0 con 0x7f508409af70 2024-05-30T16:58:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.793+0000 7f5083e00640 1 --2- 172.21.15.67:0/2318284939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f508409af70 0x7f508415e0c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f50840734e0 tx=0x7f507c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605bdc51dc4d0f24 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:13.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f5082000640 1 -- 172.21.15.67:0/2318284939 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c007740 con 0x7f508409af70 2024-05-30T16:58:13.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f5082000640 1 -- 172.21.15.67:0/2318284939 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c0078e0 con 0x7f508409af70 2024-05-30T16:58:13.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f5082000640 1 -- 172.21.15.67:0/2318284939 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c007cb0 con 0x7f508409af70 2024-05-30T16:58:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f508a800640 1 -- 172.21.15.67:0/2318284939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f508409af70 msgr2=0x7f508415e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f508a800640 1 --2- 172.21.15.67:0/2318284939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f508409af70 0x7f508415e0c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f50840734e0 tx=0x7f507c02a710 comp rx=0 tx=0).stop 2024-05-30T16:58:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f508a800640 1 -- 172.21.15.67:0/2318284939 shutdown_connections 2024-05-30T16:58:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f508a800640 1 --2- 172.21.15.67:0/2318284939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f508409af70 0x7f508415e0c0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f508a800640 1 --2- 172.21.15.67:0/2318284939 >> v2:172.21.15.169:3300/0 conn(0x7f50840d0780 0x7f508409aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f508a800640 1 -- 172.21.15.67:0/2318284939 >> 172.21.15.67:0/2318284939 conn(0x7f50840b3f40 msgr2=0x7f50840b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.794+0000 7f508a800640 1 -- 172.21.15.67:0/2318284939 shutdown_connections 2024-05-30T16:58:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.795+0000 7f508a800640 1 -- 172.21.15.67:0/2318284939 wait complete. 2024-05-30T16:58:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.796+0000 7f508a800640 1 Processor -- start 2024-05-30T16:58:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.796+0000 7f508a800640 1 -- start start 2024-05-30T16:58:13.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.797+0000 7f508a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508409af70 0x7f50840b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.797+0000 7f508a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50840b37a0 0x7f50840b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.797+0000 7f508a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50840d0780 0x7f50840b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.797+0000 7f508a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f50840d0bf0 con 0x7f50840b37a0 2024-05-30T16:58:13.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.797+0000 7f508a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50840d1a70 con 0x7f508409af70 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.797+0000 7f508a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50840b3b80 con 0x7f50840d0780 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.797+0000 7f5082a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50840b37a0 0x7f50840b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.797+0000 7f5082a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50840b37a0 0x7f50840b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43904/0 (socket says 172.21.15.67:43904) 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5082a00640 1 -- 172.21.15.67:0/1587754968 learned_addr learned my addr 172.21.15.67:0/1587754968 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5083400640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508409af70 0x7f50840b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5083e00640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50840d0780 0x7f50840b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5082a00640 1 -- 172.21.15.67:0/1587754968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50840d0780 msgr2=0x7f50840b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5082a00640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50840d0780 0x7f50840b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5082a00640 1 -- 172.21.15.67:0/1587754968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508409af70 msgr2=0x7f50840b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5082a00640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508409af70 0x7f50840b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5082a00640 1 -- 172.21.15.67:0/1587754968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50841f4470 con 0x7f50840b37a0 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5083400640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508409af70 0x7f50840b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:13.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f5082a00640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50840b37a0 0x7f50840b64c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f506c00d5f0 tx=0x7f506c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:13.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f507b400640 1 -- 172.21.15.67:0/1587754968 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c0125f0 con 0x7f50840b37a0 2024-05-30T16:58:13.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.798+0000 7f507b400640 1 -- 172.21.15.67:0/1587754968 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c019070 con 0x7f50840b37a0 2024-05-30T16:58:13.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.799+0000 7f507b400640 1 -- 172.21.15.67:0/1587754968 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c0129d0 con 0x7f50840b37a0 2024-05-30T16:58:13.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.799+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841f8790 con 0x7f50840b37a0 2024-05-30T16:58:13.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.799+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841f4a10 con 0x7f50840b37a0 2024-05-30T16:58:13.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.800+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508406fa40 con 0x7f50840b37a0 2024-05-30T16:58:13.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.801+0000 7f507b400640 1 -- 172.21.15.67:0/1587754968 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f506c012b70 con 0x7f50840b37a0 2024-05-30T16:58:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.802+0000 7f507b400640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f506006da60 0x7f506006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.802+0000 7f507b400640 1 -- 172.21.15.67:0/1587754968 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f506c08aed0 con 0x7f50840b37a0 2024-05-30T16:58:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.802+0000 7f5083400640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f506006da60 0x7f506006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.803+0000 7f5083400640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f506006da60 0x7f506006ff20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5074000f60 tx=0x7f5074009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:13.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:13.806+0000 7f507b400640 1 -- 172.21.15.67:0/1587754968 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f508406fa40 con 0x7f50840b37a0 2024-05-30T16:58:14.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.004+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50840689e0 con 0x7f50840b37a0 2024-05-30T16:58:14.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.006+0000 7f507b400640 1 -- 172.21.15.67:0/1587754968 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16907 (secure 0 0 0) 0x7f506c05aa00 con 0x7f50840b37a0 2024-05-30T16:58:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:14.007 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":78,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:58:06.617050+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:54:54.017222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-30T16:54:55.063707+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1c","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:58:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f506006da60 msgr2=0x7f506006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f506006da60 0x7f506006ff20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5074000f60 tx=0x7f5074009040 comp rx=0 tx=0).stop 2024-05-30T16:58:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50840b37a0 msgr2=0x7f50840b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50840b37a0 0x7f50840b64c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f506c00d5f0 tx=0x7f506c00dab0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 shutdown_connections 2024-05-30T16:58:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f506006da60 0x7f506006ff20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50840d0780 0x7f50840b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50840b37a0 0x7f50840b64c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 --2- 172.21.15.67:0/1587754968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508409af70 0x7f50840b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 >> 172.21.15.67:0/1587754968 conn(0x7f50840b3f40 msgr2=0x7f50840b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:14.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 shutdown_connections 2024-05-30T16:58:14.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.008+0000 7f508a800640 1 -- 172.21.15.67:0/1587754968 wait complete. 2024-05-30T16:58:14.027 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-30T16:58:14.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:58:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.271+0000 7fd73be00640 1 Processor -- start 2024-05-30T16:58:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.272+0000 7fd73be00640 1 -- start start 2024-05-30T16:58:14.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.273+0000 7fd73be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.273+0000 7fd73be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd73c0cf870 0x7fd73c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.273+0000 7fd73be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c15ec50 con 0x7fd73c15e870 2024-05-30T16:58:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.273+0000 7fd73be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c0d2150 con 0x7fd73c0cf870 2024-05-30T16:58:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.273+0000 7fd73be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c0cfd20 con 0x7fd73c0d16a0 2024-05-30T16:58:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.274+0000 7fd73a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.274+0000 7fd73a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:48722/0 (socket says 172.21.15.67:48722) 2024-05-30T16:58:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.274+0000 7fd73a000640 1 -- 172.21.15.67:0/596260506 learned_addr learned my addr 172.21.15.67:0/596260506 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.274+0000 7fd73b400640 1 --2- 172.21.15.67:0/596260506 >> v2:172.21.15.169:3300/0 conn(0x7fd73c0cf870 0x7fd73c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.275+0000 7fd73a000640 1 -- 172.21.15.67:0/596260506 >> v2:172.21.15.169:3300/0 conn(0x7fd73c0cf870 msgr2=0x7fd73c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.275+0000 7fd73a000640 1 --2- 172.21.15.67:0/596260506 >> v2:172.21.15.169:3300/0 conn(0x7fd73c0cf870 0x7fd73c0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.275+0000 7fd73a000640 1 -- 172.21.15.67:0/596260506 >> v1:172.21.15.169:6789/0 conn(0x7fd73c15e870 legacy=0x7fd73c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.275+0000 7fd73a000640 1 -- 172.21.15.67:0/596260506 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c0cff80 con 0x7fd73c0d16a0 2024-05-30T16:58:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.276+0000 7fd73a000640 1 --2- 172.21.15.67:0/596260506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0cf330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd724009ea0 tx=0x7fd72402a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b19c1c1e3c3aa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:14.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd739600640 1 -- 172.21.15.67:0/596260506 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724036070 con 0x7fd73c0d16a0 2024-05-30T16:58:14.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd739600640 1 -- 172.21.15.67:0/596260506 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd724031600 con 0x7fd73c0d16a0 2024-05-30T16:58:14.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd739600640 1 -- 172.21.15.67:0/596260506 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240319d0 con 0x7fd73c0d16a0 2024-05-30T16:58:14.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd73be00640 1 -- 172.21.15.67:0/596260506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 msgr2=0x7fd73c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd73be00640 1 --2- 172.21.15.67:0/596260506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0cf330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd724009ea0 tx=0x7fd72402a7b0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd73be00640 1 -- 172.21.15.67:0/596260506 shutdown_connections 2024-05-30T16:58:14.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd73be00640 1 --2- 172.21.15.67:0/596260506 >> v2:172.21.15.169:3300/0 conn(0x7fd73c0cf870 0x7fd73c0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd73be00640 1 --2- 172.21.15.67:0/596260506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0cf330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd73be00640 1 -- 172.21.15.67:0/596260506 >> 172.21.15.67:0/596260506 conn(0x7fd73c08bfc0 msgr2=0x7fd73c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:14.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd73be00640 1 -- 172.21.15.67:0/596260506 shutdown_connections 2024-05-30T16:58:14.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.277+0000 7fd73be00640 1 -- 172.21.15.67:0/596260506 wait complete. 2024-05-30T16:58:14.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.279+0000 7fd73be00640 1 Processor -- start 2024-05-30T16:58:14.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.279+0000 7fd73be00640 1 -- start start 2024-05-30T16:58:14.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.280+0000 7fd73be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd73c0cf870 0x7fd73c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.280+0000 7fd73be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.280+0000 7fd73be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73c15e870 0x7fd73c156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.280+0000 7fd73be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0d0820 con 0x7fd73c0cf870 2024-05-30T16:58:14.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.280+0000 7fd73be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0d0540 con 0x7fd73c15e870 2024-05-30T16:58:14.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.280+0000 7fd73be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c0d4fb0 con 0x7fd73c0d16a0 2024-05-30T16:58:14.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd73c0cf870 0x7fd73c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd73c0cf870 0x7fd73c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43918/0 (socket says 172.21.15.67:43918) 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73aa00640 1 -- 172.21.15.67:0/3482318008 learned_addr learned my addr 172.21.15.67:0/3482318008 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73a000640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73b400640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73c15e870 0x7fd73c156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73aa00640 1 -- 172.21.15.67:0/3482318008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 msgr2=0x7fd73c0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73aa00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73aa00640 1 -- 172.21.15.67:0/3482318008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73c15e870 msgr2=0x7fd73c156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73aa00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73c15e870 0x7fd73c156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73aa00640 1 -- 172.21.15.67:0/3482318008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c0cace0 con 0x7fd73c0cf870 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73b400640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73c15e870 0x7fd73c156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.281+0000 7fd73aa00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd73c0cf870 0x7fd73c0ca700 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c400 tx=0x7fd72c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.282+0000 7fd732a00640 1 -- 172.21.15.67:0/3482318008 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c017070 con 0x7fd73c0cf870 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.282+0000 7fd732a00640 1 -- 172.21.15.67:0/3482318008 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c0052d0 con 0x7fd73c0cf870 2024-05-30T16:58:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.282+0000 7fd732a00640 1 -- 172.21.15.67:0/3482318008 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c003830 con 0x7fd73c0cf870 2024-05-30T16:58:14.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.282+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c0cb810 con 0x7fd73c0cf870 2024-05-30T16:58:14.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.282+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c0c9070 con 0x7fd73c0cf870 2024-05-30T16:58:14.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.283+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c0689e0 con 0x7fd73c0cf870 2024-05-30T16:58:14.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.284+0000 7fd732a00640 1 -- 172.21.15.67:0/3482318008 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fd72c01a070 con 0x7fd73c0cf870 2024-05-30T16:58:14.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.285+0000 7fd732a00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd71806da90 0x7fd71806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.286+0000 7fd732a00640 1 -- 172.21.15.67:0/3482318008 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7fd72c089dc0 con 0x7fd73c0cf870 2024-05-30T16:58:14.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.286+0000 7fd73a000640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd71806da90 0x7fd71806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.286+0000 7fd73a000640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd71806da90 0x7fd71806ff50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd724033000 tx=0x7fd72402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:14.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.290+0000 7fd732a00640 1 -- 172.21.15.67:0/3482318008 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd72c0598f0 con 0x7fd73c0cf870 2024-05-30T16:58:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.524+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd73c0ba500 con 0x7fd73c0cf870 2024-05-30T16:58:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.525+0000 7fd732a00640 1 -- 172.21.15.67:0/3482318008 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5189 (secure 0 0 0) 0x7fd72c0591f0 con 0x7fd73c0cf870 2024-05-30T16:58:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:14.526 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5171,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1912096869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1912096869},{"type":"v1","addr":"172.21.15.169:6835","nonce":1912096869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5608,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3963396329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3963396329},{"type":"v1","addr":"172.21.15.67:6835","nonce":3963396329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2169821644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2169821644},{"type":"v1","addr":"172.21.15.169:6834","nonce":2169821644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:54:56.078869+0000","modified":"2024-05-30T16:54:59.175686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6558":{"gid":6558,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/2706164583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2706164583},{"type":"v1","addr":"172.21.15.67:6837","nonce":2706164583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6558,"qdb_cluster":[6558]},"id":5}]} 2024-05-30T16:58:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 46 2024-05-30T16:58:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd71806da90 msgr2=0x7fd71806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd71806da90 0x7fd71806ff50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd724033000 tx=0x7fd72402e000 comp rx=0 tx=0).stop 2024-05-30T16:58:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd73c0cf870 msgr2=0x7fd73c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd73c0cf870 0x7fd73c0ca700 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c400 tx=0x7fd72c00c8c0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 shutdown_connections 2024-05-30T16:58:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd71806da90 0x7fd71806ff50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd73c15e870 0x7fd73c156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd73c0d16a0 0x7fd73c0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 --2- 172.21.15.67:0/3482318008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd73c0cf870 0x7fd73c0ca700 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 >> 172.21.15.67:0/3482318008 conn(0x7fd73c08bfc0 msgr2=0x7fd73c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 shutdown_connections 2024-05-30T16:58:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.528+0000 7fd73be00640 1 -- 172.21.15.67:0/3482318008 wait complete. 2024-05-30T16:58:14.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:58:14.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.758+0000 7fd45e200640 1 Processor -- start 2024-05-30T16:58:14.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.758+0000 7fd45e200640 1 -- start start 2024-05-30T16:58:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.759+0000 7fd45e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd458156b30 0x7fd4580af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.759+0000 7fd45e200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd458156f10 con 0x7fd45806ee40 2024-05-30T16:58:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.759+0000 7fd45e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd45806dda0 con 0x7fd4580ad940 2024-05-30T16:58:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.759+0000 7fd45e200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd45806e080 con 0x7fd458156b30 2024-05-30T16:58:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.760+0000 7fd457e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd45806ee40 0x7fd4580ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54340/0 (socket says 172.21.15.67:54340) 2024-05-30T16:58:14.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.760+0000 7fd457e00640 1 -- 172.21.15.67:0/2366135336 learned_addr learned my addr 172.21.15.67:0/2366135336 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:14.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.760+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 575014467 0 0) 0x7fd458156f10 con 0x7fd45806ee40 2024-05-30T16:58:14.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.760+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd43c003520 con 0x7fd45806ee40 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.760+0000 7fd45ce00640 1 --2- 172.21.15.67:0/2366135336 >> v2:172.21.15.169:3300/0 conn(0x7fd458156b30 0x7fd4580af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.760+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3846986187 0 0) 0x7fd43c003520 con 0x7fd45806ee40 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 >> v2:172.21.15.169:3300/0 conn(0x7fd458156b30 msgr2=0x7fd4580af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 --2- 172.21.15.67:0/2366135336 >> v2:172.21.15.169:3300/0 conn(0x7fd458156b30 0x7fd4580af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 >> v1:172.21.15.169:6789/0 conn(0x7fd4580ad940 legacy=0x7fd4580b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45806e360 con 0x7fd45806ee40 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fd440001670 con 0x7fd45806ee40 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 >> v1:172.21.15.67:6789/0 conn(0x7fd45806ee40 legacy=0x7fd4580ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd43c003a00 0x7fd43c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43c004800 0x7fd43c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c005370 0x7fd43c00fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd45806e080 con 0x7fd43c004800 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c0106d0 con 0x7fd43c005370 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c0108d0 con 0x7fd43c003a00 2024-05-30T16:58:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457e00640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43c004800 0x7fd43c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd45ce00640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd43c003a00 0x7fd43c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457e00640 1 -- 172.21.15.67:0/2366135336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd43c003a00 msgr2=0x7fd43c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457e00640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd43c003a00 0x7fd43c003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd45d800640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c005370 0x7fd43c00fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457e00640 1 -- 172.21.15.67:0/2366135336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c005370 msgr2=0x7fd43c00fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457e00640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c005370 0x7fd43c00fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd457e00640 1 -- 172.21.15.67:0/2366135336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c003520 con 0x7fd43c004800 2024-05-30T16:58:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.761+0000 7fd45d800640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c005370 0x7fd43c00fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.762+0000 7fd45d800640 1 -- 172.21.15.67:0/2366135336 reap_dead start 2024-05-30T16:58:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.762+0000 7fd457e00640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43c004800 0x7fd43c004c80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c110 tx=0x7fd43c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9854c4bbe92f7a24 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.763+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440007c40 con 0x7fd43c004800 2024-05-30T16:58:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.763+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440005ad0 con 0x7fd43c004800 2024-05-30T16:58:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.763+0000 7fd457400640 1 -- 172.21.15.67:0/2366135336 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440007070 con 0x7fd43c004800 2024-05-30T16:58:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.763+0000 7fd45e200640 1 -- 172.21.15.67:0/2366135336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43c004800 msgr2=0x7fd43c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.763+0000 7fd45e200640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43c004800 0x7fd43c004c80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c110 tx=0x7fd43c005830 comp rx=0 tx=0).stop 2024-05-30T16:58:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.763+0000 7fd45e200640 1 -- 172.21.15.67:0/2366135336 shutdown_connections 2024-05-30T16:58:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.763+0000 7fd45e200640 1 --2- 172.21.15.67:0/2366135336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43c004800 0x7fd43c004c80 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.763+0000 7fd45e200640 1 -- 172.21.15.67:0/2366135336 >> 172.21.15.67:0/2366135336 conn(0x7fd4580b1fc0 msgr2=0x7fd4580b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.764+0000 7fd45e200640 1 -- 172.21.15.67:0/2366135336 shutdown_connections 2024-05-30T16:58:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.764+0000 7fd45e200640 1 -- 172.21.15.67:0/2366135336 wait complete. 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.765+0000 7fd45e200640 1 Processor -- start 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.765+0000 7fd45e200640 1 -- start start 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.766+0000 7fd45e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c004800 0x7fd458145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.766+0000 7fd45e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4581519d0 0x7fd458154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.766+0000 7fd45e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd458151e50 0x7fd458154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.766+0000 7fd45e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd4580b2db0 con 0x7fd4581519d0 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.766+0000 7fd45e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4580b2c30 con 0x7fd43c004800 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.766+0000 7fd45e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd45806f2f0 con 0x7fd458151e50 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd457e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4581519d0 0x7fd458154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd457e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4581519d0 0x7fd458154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43932/0 (socket says 172.21.15.67:43932) 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd457e00640 1 -- 172.21.15.67:0/3443865106 learned_addr learned my addr 172.21.15.67:0/3443865106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd45ce00640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c004800 0x7fd458145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd45d800640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd458151e50 0x7fd458154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd457e00640 1 -- 172.21.15.67:0/3443865106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd458151e50 msgr2=0x7fd458154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd457e00640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd458151e50 0x7fd458154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd457e00640 1 -- 172.21.15.67:0/3443865106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c004800 msgr2=0x7fd458145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd457e00640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c004800 0x7fd458145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd457e00640 1 -- 172.21.15.67:0/3443865106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4580b3300 con 0x7fd4581519d0 2024-05-30T16:58:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd45d800640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd458151e50 0x7fd458154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.767+0000 7fd457e00640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4581519d0 0x7fd458154810 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c230 tx=0x7fd45815ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.768+0000 7fd454c00640 1 -- 172.21.15.67:0/3443865106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44003f070 con 0x7fd4581519d0 2024-05-30T16:58:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.768+0000 7fd454c00640 1 -- 172.21.15.67:0/3443865106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4400013b0 con 0x7fd4581519d0 2024-05-30T16:58:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.768+0000 7fd454c00640 1 -- 172.21.15.67:0/3443865106 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440007070 con 0x7fd4581519d0 2024-05-30T16:58:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.768+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4580b35e0 con 0x7fd4581519d0 2024-05-30T16:58:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.768+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd458145a20 con 0x7fd4581519d0 2024-05-30T16:58:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.769+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4580689e0 con 0x7fd4581519d0 2024-05-30T16:58:14.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.770+0000 7fd454c00640 1 -- 172.21.15.67:0/3443865106 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fd440042070 con 0x7fd4581519d0 2024-05-30T16:58:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.771+0000 7fd454c00640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd43406da90 0x7fd43406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.771+0000 7fd454c00640 1 -- 172.21.15.67:0/3443865106 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7fd4400aaa90 con 0x7fd4581519d0 2024-05-30T16:58:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.771+0000 7fd45ce00640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd43406da90 0x7fd43406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.772+0000 7fd45ce00640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd43406da90 0x7fd43406ff50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd448006fd0 tx=0x7fd448008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:14.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:14.775+0000 7fd454c00640 1 -- 172.21.15.67:0/3443865106 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd440049020 con 0x7fd4581519d0 2024-05-30T16:58:15.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.018+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4580cfce0 con 0x7fd4581519d0 2024-05-30T16:58:15.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.020+0000 7fd454c00640 1 -- 172.21.15.67:0/3443865106 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5189 (secure 0 0 0) 0x7fd44007a5c0 con 0x7fd4581519d0 2024-05-30T16:58:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:15.021 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5171,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1912096869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1912096869},{"type":"v1","addr":"172.21.15.169:6835","nonce":1912096869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5608,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3963396329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3963396329},{"type":"v1","addr":"172.21.15.67:6835","nonce":3963396329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2169821644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2169821644},{"type":"v1","addr":"172.21.15.169:6834","nonce":2169821644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:54:56.078869+0000","modified":"2024-05-30T16:54:59.175686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6558":{"gid":6558,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6837/2706164583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2706164583},{"type":"v1","addr":"172.21.15.67:6837","nonce":2706164583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6558,"qdb_cluster":[6558]},"id":5}]} 2024-05-30T16:58:15.021 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 46 2024-05-30T16:58:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.022+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd43406da90 msgr2=0x7fd43406ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.022+0000 7fd45e200640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd43406da90 0x7fd43406ff50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd448006fd0 tx=0x7fd448008040 comp rx=0 tx=0).stop 2024-05-30T16:58:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.022+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4581519d0 msgr2=0x7fd458154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.022+0000 7fd45e200640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4581519d0 0x7fd458154810 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c230 tx=0x7fd45815ca90 comp rx=0 tx=0).stop 2024-05-30T16:58:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.022+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 shutdown_connections 2024-05-30T16:58:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.022+0000 7fd45e200640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd43406da90 0x7fd43406ff50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.022+0000 7fd45e200640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd458151e50 0x7fd458154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.022+0000 7fd45e200640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4581519d0 0x7fd458154810 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.022+0000 7fd45e200640 1 --2- 172.21.15.67:0/3443865106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd43c004800 0x7fd458145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.023+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 >> 172.21.15.67:0/3443865106 conn(0x7fd4580b1fc0 msgr2=0x7fd45815f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.023+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 shutdown_connections 2024-05-30T16:58:15.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.023+0000 7fd45e200640 1 -- 172.21.15.67:0/3443865106 wait complete. 2024-05-30T16:58:15.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:58:15.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.236+0000 7f79a1600640 1 Processor -- start 2024-05-30T16:58:15.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.236+0000 7f79a1600640 1 -- start start 2024-05-30T16:58:15.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.237+0000 7f79a1600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f799c162d20 0x7f799c071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.237+0000 7f79a1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c072100 0x7f799c0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.237+0000 7f79a1600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f799c06f590 con 0x7f799c162940 2024-05-30T16:58:15.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.237+0000 7f79a1600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f799c072530 con 0x7f799c162d20 2024-05-30T16:58:15.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.237+0000 7f79a1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f799c06f870 con 0x7f799c072100 2024-05-30T16:58:15.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.238+0000 7f799be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f799c162940 0x7f799c06eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54350/0 (socket says 172.21.15.67:54350) 2024-05-30T16:58:15.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.238+0000 7f799be00640 1 -- 172.21.15.67:0/749780953 learned_addr learned my addr 172.21.15.67:0/749780953 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:15.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.238+0000 7f799b400640 1 --2- 172.21.15.67:0/749780953 >> v2:172.21.15.169:3300/0 conn(0x7f799c162d20 0x7f799c071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.238+0000 7f79a0c00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c072100 0x7f799c0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.238+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550141458 0 0) 0x7f799c06f590 con 0x7f799c162940 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.238+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7980003520 con 0x7f799c162940 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1068812890 0 0) 0x7f7980003520 con 0x7f799c162940 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c072100 msgr2=0x7f799c0adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c072100 0x7f799c0adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 >> v2:172.21.15.169:3300/0 conn(0x7f799c162d20 msgr2=0x7f799c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 --2- 172.21.15.67:0/749780953 >> v2:172.21.15.169:3300/0 conn(0x7f799c162d20 0x7f799c071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c0ad3d0 con 0x7f799c162940 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f798c001670 con 0x7f799c162940 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 >> v1:172.21.15.67:6789/0 conn(0x7f799c162940 legacy=0x7f799c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79800039b0 0x7f7980003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c072100 0x7f7980004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7980004d40 0x7f79800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f799c072530 con 0x7f7980004d40 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f799c06f870 con 0x7f79800039b0 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f798000ba40 con 0x7f799c072100 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.239+0000 7f79a0c00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7980004d40 0x7f79800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.240+0000 7f799be00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79800039b0 0x7f7980003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.240+0000 7f799b400640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c072100 0x7f7980004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.240+0000 7f79a0c00640 1 -- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c072100 msgr2=0x7f7980004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.240+0000 7f79a0c00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c072100 0x7f7980004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.240+0000 7f79a0c00640 1 -- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79800039b0 msgr2=0x7f7980003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.240+0000 7f79a0c00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79800039b0 0x7f7980003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.240+0000 7f79a0c00640 1 -- 172.21.15.67:0/749780953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7980003520 con 0x7f7980004d40 2024-05-30T16:58:15.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.240+0000 7f799b400640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c072100 0x7f7980004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:15.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.241+0000 7f79a0c00640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7980004d40 0x7f79800051c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7994009fc0 tx=0x7f7980005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36269e6f6a3d9593 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:15.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.241+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994009c10 con 0x7f7980004d40 2024-05-30T16:58:15.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.241+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7994036070 con 0x7f7980004d40 2024-05-30T16:58:15.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.241+0000 7f799aa00640 1 -- 172.21.15.67:0/749780953 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994031880 con 0x7f7980004d40 2024-05-30T16:58:15.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.242+0000 7f79a1600640 1 -- 172.21.15.67:0/749780953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7980004d40 msgr2=0x7f79800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.242+0000 7f79a1600640 1 --2- 172.21.15.67:0/749780953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7980004d40 0x7f79800051c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7994009fc0 tx=0x7f7980005830 comp rx=0 tx=0).stop 2024-05-30T16:58:15.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.242+0000 7f79a0c00640 1 -- 172.21.15.67:0/749780953 reap_dead start 2024-05-30T16:58:15.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.242+0000 7f79a1600640 1 -- 172.21.15.67:0/749780953 shutdown_connections 2024-05-30T16:58:15.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.242+0000 7f79a1600640 1 -- 172.21.15.67:0/749780953 >> 172.21.15.67:0/749780953 conn(0x7f799c0b1fc0 msgr2=0x7f799c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:15.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.242+0000 7f79a1600640 1 -- 172.21.15.67:0/749780953 shutdown_connections 2024-05-30T16:58:15.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.242+0000 7f79a1600640 1 -- 172.21.15.67:0/749780953 wait complete. 2024-05-30T16:58:15.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.243+0000 7f79a1600640 1 Processor -- start 2024-05-30T16:58:15.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.244+0000 7f79a1600640 1 -- start start 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.244+0000 7f79a1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7980004d40 0x7f799c164690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f79a1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f799c165da0 0x7f799c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f79a1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c156680 0x7f799c0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f79a1600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f799c06df60 con 0x7f799c165da0 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f79a1600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f799c06dde0 con 0x7f7980004d40 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f79a1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f799c0adbb0 con 0x7f799c156680 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f799b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f799c165da0 0x7f799c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f799b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f799c165da0 0x7f799c0ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43944/0 (socket says 172.21.15.67:43944) 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f799b400640 1 -- 172.21.15.67:0/3890507526 learned_addr learned my addr 172.21.15.67:0/3890507526 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f799be00640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7980004d40 0x7f799c164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f799b400640 1 -- 172.21.15.67:0/3890507526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c156680 msgr2=0x7f799c0b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f79a0c00640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c156680 0x7f799c0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f799b400640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c156680 0x7f799c0b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f799b400640 1 -- 172.21.15.67:0/3890507526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7980004d40 msgr2=0x7f799c164690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f799b400640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7980004d40 0x7f799c164690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.245+0000 7f799b400640 1 -- 172.21.15.67:0/3890507526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c164d00 con 0x7f799c165da0 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.246+0000 7f799b400640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f799c165da0 0x7f799c0ce150 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f7988010900 tx=0x7f7988010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.246+0000 7f7993e00640 1 -- 172.21.15.67:0/3890507526 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798800f040 con 0x7f799c165da0 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.246+0000 7f7993e00640 1 -- 172.21.15.67:0/3890507526 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f798800ec20 con 0x7f799c165da0 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.246+0000 7f7993e00640 1 -- 172.21.15.67:0/3890507526 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798801c400 con 0x7f799c165da0 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.246+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c164fe0 con 0x7f799c165da0 2024-05-30T16:58:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.246+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1655a0 con 0x7f799c165da0 2024-05-30T16:58:15.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.247+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c0689e0 con 0x7f799c165da0 2024-05-30T16:58:15.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.248+0000 7f7993e00640 1 -- 172.21.15.67:0/3890507526 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f798801c5a0 con 0x7f799c165da0 2024-05-30T16:58:15.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.249+0000 7f7993e00640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f797c06da60 0x7f797c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.249+0000 7f7993e00640 1 -- 172.21.15.67:0/3890507526 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f798808b270 con 0x7f799c165da0 2024-05-30T16:58:15.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.249+0000 7f799be00640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f797c06da60 0x7f797c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.250+0000 7f799be00640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f797c06da60 0x7f797c06ff20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f799c0b4c10 tx=0x7f798c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.254+0000 7f7993e00640 1 -- 172.21.15.67:0/3890507526 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7988018030 con 0x7f799c165da0 2024-05-30T16:58:15.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.463+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f799c099dc0 con 0x7f799c165da0 2024-05-30T16:58:15.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.465+0000 7f7993e00640 1 -- 172.21.15.67:0/3890507526 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16907 (secure 0 0 0) 0x7f798805a750 con 0x7f799c165da0 2024-05-30T16:58:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:15.466 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":78,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:58:06.617050+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:54:54.017222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-30T16:54:55.063707+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1c","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f797c06da60 msgr2=0x7f797c06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f797c06da60 0x7f797c06ff20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f799c0b4c10 tx=0x7f798c009040 comp rx=0 tx=0).stop 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f799c165da0 msgr2=0x7f799c0ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f799c165da0 0x7f799c0ce150 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f7988010900 tx=0x7f7988010dc0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 shutdown_connections 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f797c06da60 0x7f797c06ff20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f799c156680 0x7f799c0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f799c165da0 0x7f799c0ce150 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 --2- 172.21.15.67:0/3890507526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7980004d40 0x7f799c164690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 >> 172.21.15.67:0/3890507526 conn(0x7f799c0b1fc0 msgr2=0x7f799c0afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 shutdown_connections 2024-05-30T16:58:15.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.468+0000 7f79a1600640 1 -- 172.21.15.67:0/3890507526 wait complete. 2024-05-30T16:58:15.489 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-30T16:58:15.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.704+0000 7fd7fb200640 1 Processor -- start 2024-05-30T16:58:15.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.704+0000 7fd7fb200640 1 -- start start 2024-05-30T16:58:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.705+0000 7fd7fb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7f406f860 0x7fd7f40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.705+0000 7fd7fb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd7f40d0bb0 0x7fd7f40ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.705+0000 7fd7fb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f406ed10 con 0x7fd7f40b3690 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.705+0000 7fd7fb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f415cb00 con 0x7fd7f40d0bb0 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.705+0000 7fd7fb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f40b3cc0 con 0x7fd7f406f860 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.705+0000 7fd7f9200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd7f40d0bb0 0x7fd7f40ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.705+0000 7fd7f3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd7f40b3690 0x7fd7f40d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54362/0 (socket says 172.21.15.67:54362) 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.705+0000 7fd7f3400640 1 -- 172.21.15.67:0/342791963 learned_addr learned my addr 172.21.15.67:0/342791963 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.706+0000 7fd7f3e00640 1 --2- 172.21.15.67:0/342791963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7f406f860 0x7fd7f40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.706+0000 7fd7f9200640 1 -- 172.21.15.67:0/342791963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7f406f860 msgr2=0x7fd7f40b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.706+0000 7fd7f9200640 1 --2- 172.21.15.67:0/342791963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7f406f860 0x7fd7f40b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.706+0000 7fd7f9200640 1 -- 172.21.15.67:0/342791963 >> v1:172.21.15.67:6789/0 conn(0x7fd7f40b3690 legacy=0x7fd7f40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.706+0000 7fd7f9200640 1 -- 172.21.15.67:0/342791963 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f40b3fa0 con 0x7fd7f40d0bb0 2024-05-30T16:58:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.706+0000 7fd7f2a00640 1 -- 172.21.15.67:0/342791963 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 573490312 0 0) 0x7fd7e0001110 con 0x7fd7f40b3690 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.707+0000 7fd7f9200640 1 --2- 172.21.15.67:0/342791963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f40d0bb0 0x7fd7f40ad130 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009d90 tx=0x7fd7ec02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e017c138eec17bd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.707+0000 7fd7f2a00640 1 -- 172.21.15.67:0/342791963 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec007740 con 0x7fd7f40d0bb0 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.707+0000 7fd7f2a00640 1 -- 172.21.15.67:0/342791963 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ec0078e0 con 0x7fd7f40d0bb0 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.707+0000 7fd7f2a00640 1 -- 172.21.15.67:0/342791963 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec007cb0 con 0x7fd7f40d0bb0 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.708+0000 7fd7fb200640 1 -- 172.21.15.67:0/342791963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f40d0bb0 msgr2=0x7fd7f40ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.708+0000 7fd7fb200640 1 --2- 172.21.15.67:0/342791963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f40d0bb0 0x7fd7f40ad130 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009d90 tx=0x7fd7ec02a660 comp rx=0 tx=0).stop 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.708+0000 7fd7fb200640 1 -- 172.21.15.67:0/342791963 shutdown_connections 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.708+0000 7fd7fb200640 1 --2- 172.21.15.67:0/342791963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f40d0bb0 0x7fd7f40ad130 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.708+0000 7fd7fb200640 1 --2- 172.21.15.67:0/342791963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7f406f860 0x7fd7f40b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.708+0000 7fd7fb200640 1 -- 172.21.15.67:0/342791963 >> 172.21.15.67:0/342791963 conn(0x7fd7f406bae0 msgr2=0x7fd7f406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.708+0000 7fd7fb200640 1 -- 172.21.15.67:0/342791963 shutdown_connections 2024-05-30T16:58:15.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.708+0000 7fd7fb200640 1 -- 172.21.15.67:0/342791963 wait complete. 2024-05-30T16:58:15.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.710+0000 7fd7fb200640 1 Processor -- start 2024-05-30T16:58:15.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.710+0000 7fd7fb200640 1 -- start start 2024-05-30T16:58:15.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.711+0000 7fd7fb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f406f860 0x7fd7f40c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.711+0000 7fd7fb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7f40b3690 0x7fd7f40af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.711+0000 7fd7fb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7f40d0bb0 0x7fd7f40ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.711+0000 7fd7fb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f40b4840 con 0x7fd7f406f860 2024-05-30T16:58:15.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.711+0000 7fd7fb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f40b4560 con 0x7fd7f40d0bb0 2024-05-30T16:58:15.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.711+0000 7fd7fb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f40ad670 con 0x7fd7f40b3690 2024-05-30T16:58:15.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.711+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f406f860 0x7fd7f40c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.711+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f406f860 0x7fd7f40c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43952/0 (socket says 172.21.15.67:43952) 2024-05-30T16:58:15.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.711+0000 7fd7f3e00640 1 -- 172.21.15.67:0/1732873618 learned_addr learned my addr 172.21.15.67:0/1732873618 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:15.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7f3e00640 1 -- 172.21.15.67:0/1732873618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7f40b3690 msgr2=0x7fd7f40af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7f3e00640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7f40b3690 0x7fd7f40af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7f3e00640 1 -- 172.21.15.67:0/1732873618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7f40d0bb0 msgr2=0x7fd7f40ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7f3e00640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7f40d0bb0 0x7fd7f40ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7f3e00640 1 -- 172.21.15.67:0/1732873618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f40aff80 con 0x7fd7f406f860 2024-05-30T16:58:15.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7f3e00640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f406f860 0x7fd7f40c9f10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400d800 tx=0x7fd7e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:15.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7ebe00640 1 -- 172.21.15.67:0/1732873618 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400b6c0 con 0x7fd7f406f860 2024-05-30T16:58:15.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7ebe00640 1 -- 172.21.15.67:0/1732873618 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e4019070 con 0x7fd7f406f860 2024-05-30T16:58:15.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f40b0260 con 0x7fd7f406f860 2024-05-30T16:58:15.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7ebe00640 1 -- 172.21.15.67:0/1732873618 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400ba80 con 0x7fd7f406f860 2024-05-30T16:58:15.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.712+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f40b0b00 con 0x7fd7f406f860 2024-05-30T16:58:15.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.714+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f40689e0 con 0x7fd7f406f860 2024-05-30T16:58:15.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.715+0000 7fd7ebe00640 1 -- 172.21.15.67:0/1732873618 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fd7e400bc20 con 0x7fd7f406f860 2024-05-30T16:58:15.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.716+0000 7fd7ebe00640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7d406da60 0x7fd7d406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:15.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.716+0000 7fd7ebe00640 1 -- 172.21.15.67:0/1732873618 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7fd7e408afa0 con 0x7fd7f406f860 2024-05-30T16:58:15.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.716+0000 7fd7f3400640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7d406da60 0x7fd7d406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:15.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.717+0000 7fd7f3400640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7d406da60 0x7fd7d406ff20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40cb0f0 tx=0x7fd7e0001310 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:15.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.720+0000 7fd7ebe00640 1 -- 172.21.15.67:0/1732873618 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd7e405aaa0 con 0x7fd7f406f860 2024-05-30T16:58:15.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.962+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd7f406e2f0 con 0x7fd7f406f860 2024-05-30T16:58:15.963 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-05-30T16:58:15.963+0000 7f4982000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-30T16:58:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.967+0000 7fd7ebe00640 1 -- 172.21.15.67:0/1732873618 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7fd7e405a3a0 con 0x7fd7f406f860 2024-05-30T16:58:15.967 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-30T16:58:15.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.970+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7d406da60 msgr2=0x7fd7d406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.970+0000 7fd7fb200640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7d406da60 0x7fd7d406ff20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40cb0f0 tx=0x7fd7e0001310 comp rx=0 tx=0).stop 2024-05-30T16:58:15.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.970+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f406f860 msgr2=0x7fd7f40c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:15.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.970+0000 7fd7fb200640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f406f860 0x7fd7f40c9f10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400d800 tx=0x7fd7e400dcc0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.971+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 shutdown_connections 2024-05-30T16:58:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.971+0000 7fd7fb200640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7d406da60 0x7fd7d406ff20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.971+0000 7fd7fb200640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7f40d0bb0 0x7fd7f40ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.971+0000 7fd7fb200640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7f40b3690 0x7fd7f40af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.971+0000 7fd7fb200640 1 --2- 172.21.15.67:0/1732873618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7f406f860 0x7fd7f40c9f10 secure :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400d800 tx=0x7fd7e400dcc0 comp rx=0 tx=0).stop 2024-05-30T16:58:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.971+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 >> 172.21.15.67:0/1732873618 conn(0x7fd7f406bae0 msgr2=0x7fd7f40cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.971+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 shutdown_connections 2024-05-30T16:58:15.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:15.971+0000 7fd7fb200640 1 -- 172.21.15.67:0/1732873618 wait complete. 2024-05-30T16:58:15.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-30T16:58:16.070 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T16:58:16.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.177+0000 7f3284800640 1 Processor -- start 2024-05-30T16:58:16.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.177+0000 7f3284800640 1 -- start start 2024-05-30T16:58:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.178+0000 7f3284800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f327c15b6f0 0x7f327c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.178+0000 7f3284800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c0725e0 0x7f327c0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.178+0000 7f3284800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f327c0ade90 con 0x7f327c0d0640 2024-05-30T16:58:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.178+0000 7f3284800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f327c071c40 con 0x7f327c15b6f0 2024-05-30T16:58:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.178+0000 7f3284800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f327c071e70 con 0x7f327c0725e0 2024-05-30T16:58:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.179+0000 7f3282800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c0725e0 0x7f327c0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.179+0000 7f3282800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c0725e0 0x7f327c0ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:48836/0 (socket says 172.21.15.67:48836) 2024-05-30T16:58:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.179+0000 7f3282800640 1 -- 172.21.15.67:0/3887226407 learned_addr learned my addr 172.21.15.67:0/3887226407 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.179+0000 7f3281400640 1 --2- 172.21.15.67:0/3887226407 >> v2:172.21.15.169:3300/0 conn(0x7f327c15b6f0 0x7f327c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.180+0000 7f3282800640 1 -- 172.21.15.67:0/3887226407 >> v2:172.21.15.169:3300/0 conn(0x7f327c15b6f0 msgr2=0x7f327c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.180+0000 7f3282800640 1 --2- 172.21.15.67:0/3887226407 >> v2:172.21.15.169:3300/0 conn(0x7f327c15b6f0 0x7f327c06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.180+0000 7f3282800640 1 -- 172.21.15.67:0/3887226407 >> v1:172.21.15.169:6789/0 conn(0x7f327c0d0640 legacy=0x7f327c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.180+0000 7f3280a00640 1 -- 172.21.15.67:0/3887226407 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141837906 0 0) 0x7f327c0ade90 con 0x7f327c0d0640 2024-05-30T16:58:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.180+0000 7f3282800640 1 -- 172.21.15.67:0/3887226407 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c072130 con 0x7f327c0725e0 2024-05-30T16:58:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.181+0000 7f3282800640 1 --2- 172.21.15.67:0/3887226407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c0725e0 0x7f327c0ae650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3278009e50 tx=0x7f327802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64e2ed5bc9101ead server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.182+0000 7f3280a00640 1 -- 172.21.15.67:0/3887226407 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278036070 con 0x7f327c0725e0 2024-05-30T16:58:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.182+0000 7f3280a00640 1 -- 172.21.15.67:0/3887226407 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3278031600 con 0x7f327c0725e0 2024-05-30T16:58:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.182+0000 7f3280a00640 1 -- 172.21.15.67:0/3887226407 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780319d0 con 0x7f327c0725e0 2024-05-30T16:58:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.182+0000 7f3284800640 1 -- 172.21.15.67:0/3887226407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c0725e0 msgr2=0x7f327c0ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.182+0000 7f3284800640 1 --2- 172.21.15.67:0/3887226407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c0725e0 0x7f327c0ae650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3278009e50 tx=0x7f327802a710 comp rx=0 tx=0).stop 2024-05-30T16:58:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.182+0000 7f3284800640 1 -- 172.21.15.67:0/3887226407 shutdown_connections 2024-05-30T16:58:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.182+0000 7f3284800640 1 --2- 172.21.15.67:0/3887226407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c0725e0 0x7f327c0ae650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.182+0000 7f3284800640 1 --2- 172.21.15.67:0/3887226407 >> v2:172.21.15.169:3300/0 conn(0x7f327c15b6f0 0x7f327c06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.182+0000 7f3284800640 1 -- 172.21.15.67:0/3887226407 >> 172.21.15.67:0/3887226407 conn(0x7f327c06bd20 msgr2=0x7f327c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.183+0000 7f3284800640 1 -- 172.21.15.67:0/3887226407 shutdown_connections 2024-05-30T16:58:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.183+0000 7f3284800640 1 -- 172.21.15.67:0/3887226407 wait complete. 2024-05-30T16:58:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.184+0000 7f3284800640 1 Processor -- start 2024-05-30T16:58:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.184+0000 7f3284800640 1 -- start start 2024-05-30T16:58:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.185+0000 7f3284800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f327c0725e0 0x7f327c149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.185+0000 7f3284800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c0d0640 0x7f327c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.185+0000 7f3284800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c15b6f0 0x7f327c0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.185+0000 7f3284800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f327c0aeb90 con 0x7f327c0d0640 2024-05-30T16:58:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.185+0000 7f3284800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f327c06e890 con 0x7f327c0725e0 2024-05-30T16:58:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.185+0000 7f3284800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f327c0aed10 con 0x7f327c15b6f0 2024-05-30T16:58:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.185+0000 7f3281400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c0d0640 0x7f327c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c0d0640 0x7f327c151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43974/0 (socket says 172.21.15.67:43974) 2024-05-30T16:58:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281400640 1 -- 172.21.15.67:0/1372268905 learned_addr learned my addr 172.21.15.67:0/1372268905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3282800640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c15b6f0 0x7f327c0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281400640 1 -- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c15b6f0 msgr2=0x7f327c0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281400640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c15b6f0 0x7f327c0ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281e00640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f327c0725e0 0x7f327c149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281400640 1 -- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f327c0725e0 msgr2=0x7f327c149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281400640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f327c0725e0 0x7f327c149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281400640 1 -- 172.21.15.67:0/1372268905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c154240 con 0x7f327c0d0640 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3282800640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c15b6f0 0x7f327c0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281e00640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f327c0725e0 0x7f327c149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.186+0000 7f3281400640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c0d0640 0x7f327c151b20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f326c00e870 tx=0x7f326c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.187+0000 7f3276000640 1 -- 172.21.15.67:0/1372268905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c017070 con 0x7f327c0d0640 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.187+0000 7f3276000640 1 -- 172.21.15.67:0/1372268905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c00c940 con 0x7f327c0d0640 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.187+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c15db40 con 0x7f327c0d0640 2024-05-30T16:58:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.187+0000 7f3276000640 1 -- 172.21.15.67:0/1372268905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c00cd30 con 0x7f327c0d0640 2024-05-30T16:58:16.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.187+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c15e100 con 0x7f327c0d0640 2024-05-30T16:58:16.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.188+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327c0689e0 con 0x7f327c0d0640 2024-05-30T16:58:16.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.189+0000 7f3276000640 1 -- 172.21.15.67:0/1372268905 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f326c01a070 con 0x7f327c0d0640 2024-05-30T16:58:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.190+0000 7f3276000640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f326006da60 0x7f326006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.190+0000 7f3276000640 1 -- 172.21.15.67:0/1372268905 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5909+0+0 (secure 0 0 0) 0x7f326c08acb0 con 0x7f327c0d0640 2024-05-30T16:58:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.190+0000 7f3281e00640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f326006da60 0x7f326006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.191+0000 7f3281e00640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f326006da60 0x7f326006ff20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3270000f60 tx=0x7f3270009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:16.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.194+0000 7f3276000640 1 -- 172.21.15.67:0/1372268905 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f326c05a7e0 con 0x7f327c0d0640 2024-05-30T16:58:16.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.439+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f327c0cc9b0 con 0x7f327c0d0640 2024-05-30T16:58:16.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.971+0000 7f3276000640 1 -- 172.21.15.67:0/1372268905 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) ==== 104+0+0 (secure 0 0 0) 0x7f326c05a0e0 con 0x7f327c0d0640 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f326006da60 msgr2=0x7f326006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f326006da60 0x7f326006ff20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3270000f60 tx=0x7f3270009040 comp rx=0 tx=0).stop 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c0d0640 msgr2=0x7f327c151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c0d0640 0x7f327c151b20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f326c00e870 tx=0x7f326c00ed30 comp rx=0 tx=0).stop 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 shutdown_connections 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f326006da60 0x7f326006ff20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f327c15b6f0 0x7f327c0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f327c0d0640 0x7f327c151b20 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 --2- 172.21.15.67:0/1372268905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f327c0725e0 0x7f327c149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.974+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 >> 172.21.15.67:0/1372268905 conn(0x7f327c06bd20 msgr2=0x7f327c15a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:16.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.975+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 shutdown_connections 2024-05-30T16:58:16.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:16.975+0000 7f3284800640 1 -- 172.21.15.67:0/1372268905 wait complete. 2024-05-30T16:58:17.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-30T16:58:17.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.223+0000 7f7c9d000640 1 Processor -- start 2024-05-30T16:58:17.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.223+0000 7f7c9d000640 1 -- start start 2024-05-30T16:58:17.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.224+0000 7f7c9d000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c980d1100 0x7f7c980d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:17.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.224+0000 7f7c9d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c98163020 con 0x7f7c980d31d0 2024-05-30T16:58:17.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.225+0000 7f7c9d000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c98163300 con 0x7f7c9815e990 2024-05-30T16:58:17.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.225+0000 7f7c9d000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7c980d1aa0 con 0x7f7c980d1100 2024-05-30T16:58:17.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.225+0000 7f7c96800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7c980d31d0 0x7f7c980d3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54390/0 (socket says 172.21.15.67:54390) 2024-05-30T16:58:17.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.225+0000 7f7c96800640 1 -- 172.21.15.67:0/1588477671 learned_addr learned my addr 172.21.15.67:0/1588477671 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:17.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.225+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494139075 0 0) 0x7f7c98163020 con 0x7f7c980d31d0 2024-05-30T16:58:17.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.225+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c7c003520 con 0x7f7c980d31d0 2024-05-30T16:58:17.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.225+0000 7f7c8fe00640 1 --2- 172.21.15.67:0/1588477671 >> v2:172.21.15.169:3300/0 conn(0x7f7c980d1100 0x7f7c980d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:17.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4124884484 0 0) 0x7f7c7c003520 con 0x7f7c980d31d0 2024-05-30T16:58:17.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 >> v2:172.21.15.169:3300/0 conn(0x7f7c980d1100 msgr2=0x7f7c980d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 --2- 172.21.15.67:0/1588477671 >> v2:172.21.15.169:3300/0 conn(0x7f7c980d1100 0x7f7c980d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 >> v1:172.21.15.169:6789/0 conn(0x7f7c9815e990 legacy=0x7f7c98162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c980d29d0 con 0x7f7c980d31d0 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7c900019f0 con 0x7f7c980d31d0 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 >> v1:172.21.15.67:6789/0 conn(0x7f7c980d31d0 legacy=0x7f7c980d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 --2- 172.21.15.67:0/1588477671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c7c0039b0 0x7f7c7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 --2- 172.21.15.67:0/1588477671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7c004800 0x7f7c7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 --2- 172.21.15.67:0/1588477671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7c0051c0 0x7f7c7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c00c0e0 con 0x7f7c7c0051c0 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c00c290 con 0x7f7c7c004800 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c7c00c490 con 0x7f7c7c0039b0 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.226+0000 7f7c96800640 1 --2- 172.21.15.67:0/1588477671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7c0051c0 0x7f7c7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.227+0000 7f7c96800640 1 -- 172.21.15.67:0/1588477671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c7c0039b0 msgr2=0x7f7c7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:17.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.227+0000 7f7c96800640 1 --2- 172.21.15.67:0/1588477671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c7c0039b0 0x7f7c7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.227+0000 7f7c96800640 1 -- 172.21.15.67:0/1588477671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7c004800 msgr2=0x7f7c7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:17.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.227+0000 7f7c96800640 1 --2- 172.21.15.67:0/1588477671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c7c004800 0x7f7c7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.227+0000 7f7c96800640 1 -- 172.21.15.67:0/1588477671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7c003520 con 0x7f7c7c0051c0 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.227+0000 7f7c96800640 1 -- 172.21.15.67:0/1588477671 reap_dead start 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.228+0000 7f7c96800640 1 --2- 172.21.15.67:0/1588477671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7c0051c0 0x7f7c7c00b9c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000a850 tx=0x7f7c7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f926fc947888d221 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.228+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000a4a0 con 0x7f7c7c0051c0 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.228+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c9002f070 con 0x7f7c7c0051c0 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.228+0000 7f7c95400640 1 -- 172.21.15.67:0/1588477671 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9002bce0 con 0x7f7c7c0051c0 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.228+0000 7f7c9d000640 1 -- 172.21.15.67:0/1588477671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7c0051c0 msgr2=0x7f7c7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.228+0000 7f7c9d000640 1 --2- 172.21.15.67:0/1588477671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7c0051c0 0x7f7c7c00b9c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000a850 tx=0x7f7c7c005830 comp rx=0 tx=0).stop 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.229+0000 7f7c9d000640 1 -- 172.21.15.67:0/1588477671 shutdown_connections 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.229+0000 7f7c9d000640 1 --2- 172.21.15.67:0/1588477671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c7c0051c0 0x7f7c7c00b9c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.229+0000 7f7c9d000640 1 -- 172.21.15.67:0/1588477671 >> 172.21.15.67:0/1588477671 conn(0x7f7c9806cb00 msgr2=0x7f7c9806cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:17.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.229+0000 7f7c9d000640 1 -- 172.21.15.67:0/1588477671 shutdown_connections 2024-05-30T16:58:17.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.229+0000 7f7c9d000640 1 -- 172.21.15.67:0/1588477671 wait complete. 2024-05-30T16:58:17.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.230+0000 7f7c9d000640 1 Processor -- start 2024-05-30T16:58:17.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.231+0000 7f7c9d000640 1 -- start start 2024-05-30T16:58:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.231+0000 7f7c9d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c7c0051c0 0x7f7c98159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.231+0000 7f7c9d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c980c9a90 0x7f7c980b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.231+0000 7f7c9d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c980c9f10 0x7f7c980b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.231+0000 7f7c9d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7c980d2d40 con 0x7f7c980c9f10 2024-05-30T16:58:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.231+0000 7f7c9d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c980d2780 con 0x7f7c980c9a90 2024-05-30T16:58:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.231+0000 7f7c9d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9815ed70 con 0x7f7c7c0051c0 2024-05-30T16:58:17.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.232+0000 7f7c96800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c980c9f10 0x7f7c980b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:17.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.232+0000 7f7c96800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c980c9f10 0x7f7c980b9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44004/0 (socket says 172.21.15.67:44004) 2024-05-30T16:58:17.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.232+0000 7f7c96800640 1 -- 172.21.15.67:0/2279534905 learned_addr learned my addr 172.21.15.67:0/2279534905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:17.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.232+0000 7f7c95e00640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c7c0051c0 0x7f7c98159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:17.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.232+0000 7f7c96800640 1 -- 172.21.15.67:0/2279534905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c7c0051c0 msgr2=0x7f7c98159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:17.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.232+0000 7f7c96800640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c7c0051c0 0x7f7c98159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.232+0000 7f7c96800640 1 -- 172.21.15.67:0/2279534905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c980c9a90 msgr2=0x7f7c980b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:17.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.232+0000 7f7c96800640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c980c9a90 0x7f7c980b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.232+0000 7f7c96800640 1 -- 172.21.15.67:0/2279534905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c980c83c0 con 0x7f7c980c9f10 2024-05-30T16:58:17.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.233+0000 7f7c96800640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c980c9f10 0x7f7c980b9e80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000a970 tx=0x7f7c90001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:17.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.233+0000 7f7c8e000640 1 -- 172.21.15.67:0/2279534905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000a480 con 0x7f7c980c9f10 2024-05-30T16:58:17.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.233+0000 7f7c8e000640 1 -- 172.21.15.67:0/2279534905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c90042070 con 0x7f7c980c9f10 2024-05-30T16:58:17.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.233+0000 7f7c8e000640 1 -- 172.21.15.67:0/2279534905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90034670 con 0x7f7c980c9f10 2024-05-30T16:58:17.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.233+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c980c8660 con 0x7f7c980c9f10 2024-05-30T16:58:17.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.233+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c980c8c20 con 0x7f7c980c9f10 2024-05-30T16:58:17.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.234+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c980689e0 con 0x7f7c980c9f10 2024-05-30T16:58:17.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.235+0000 7f7c8e000640 1 -- 172.21.15.67:0/2279534905 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f7c9003c070 con 0x7f7c980c9f10 2024-05-30T16:58:17.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.236+0000 7f7c8e000640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c7406da60 0x7f7c7406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:17.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.236+0000 7f7c8e000640 1 -- 172.21.15.67:0/2279534905 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5909+0+0 (secure 0 0 0) 0x7f7c900aaa30 con 0x7f7c980c9f10 2024-05-30T16:58:17.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.237+0000 7f7c95e00640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c7406da60 0x7f7c7406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:17.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.237+0000 7f7c95e00640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c7406da60 0x7f7c7406ff20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7c88001590 tx=0x7f7c88009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:17.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.242+0000 7f7c8e000640 1 -- 172.21.15.67:0/2279534905 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c9007a9d0 con 0x7f7c980c9f10 2024-05-30T16:58:17.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.456+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7c980d3b30 con 0x7f7c980c9f10 2024-05-30T16:58:17.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.984+0000 7f7c8e000640 1 -- 172.21.15.67:0/2279534905 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) ==== 181+0+0 (secure 0 0 0) 0x7f7c9007a2d0 con 0x7f7c980c9f10 2024-05-30T16:58:17.985 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-05-30T16:58:17.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c7406da60 msgr2=0x7f7c7406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:17.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c7406da60 0x7f7c7406ff20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7c88001590 tx=0x7f7c88009040 comp rx=0 tx=0).stop 2024-05-30T16:58:17.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c980c9f10 msgr2=0x7f7c980b9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:17.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c980c9f10 0x7f7c980b9e80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000a970 tx=0x7f7c90001dc0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 shutdown_connections 2024-05-30T16:58:17.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7c7406da60 0x7f7c7406ff20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c980c9f10 0x7f7c980b9e80 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c980c9a90 0x7f7c980b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 --2- 172.21.15.67:0/2279534905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c7c0051c0 0x7f7c98159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:17.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.988+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 >> 172.21.15.67:0/2279534905 conn(0x7f7c9806cb00 msgr2=0x7f7c9815dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:17.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.990+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 shutdown_connections 2024-05-30T16:58:17.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:17.991+0000 7f7c9d000640 1 -- 172.21.15.67:0/2279534905 wait complete. 2024-05-30T16:58:18.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-30T16:58:18.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.208+0000 7fa12be00640 1 Processor -- start 2024-05-30T16:58:18.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.208+0000 7fa12be00640 1 -- start start 2024-05-30T16:58:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.210+0000 7fa12be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa12c0d0b40 0x7fa12c169740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.210+0000 7fa12be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa12c162cb0 con 0x7fa12c1480a0 2024-05-30T16:58:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.210+0000 7fa12be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa12c162f90 con 0x7fa12c0af6f0 2024-05-30T16:58:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.210+0000 7fa12be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa12c163270 con 0x7fa12c0d0b40 2024-05-30T16:58:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.210+0000 7fa123e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa12c1480a0 0x7fa12c0d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54396/0 (socket says 172.21.15.67:54396) 2024-05-30T16:58:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.210+0000 7fa123e00640 1 -- 172.21.15.67:0/3362244094 learned_addr learned my addr 172.21.15.67:0/3362244094 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.210+0000 7fa12b400640 1 --2- 172.21.15.67:0/3362244094 >> v2:172.21.15.67:3300/0 conn(0x7fa12c0d0b40 0x7fa12c169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.211+0000 7fa12b400640 1 -- 172.21.15.67:0/3362244094 >> v1:172.21.15.169:6789/0 conn(0x7fa12c0af6f0 legacy=0x7fa12c1623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.211+0000 7fa12b400640 1 -- 172.21.15.67:0/3362244094 >> v1:172.21.15.67:6789/0 conn(0x7fa12c1480a0 legacy=0x7fa12c0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.211+0000 7fa12b400640 1 -- 172.21.15.67:0/3362244094 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c1420d0 con 0x7fa12c0d0b40 2024-05-30T16:58:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.211+0000 7fa12a000640 1 -- 172.21.15.67:0/3362244094 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2717427854 0 0) 0x7fa12c162cb0 con 0x7fa12c1480a0 2024-05-30T16:58:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.212+0000 7fa12b400640 1 --2- 172.21.15.67:0/3362244094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c0d0b40 0x7fa12c169740 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa124009d90 tx=0x7fa12402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d68197845216cbe server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:18.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.212+0000 7fa12a000640 1 -- 172.21.15.67:0/3362244094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124007740 con 0x7fa12c0d0b40 2024-05-30T16:58:18.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.212+0000 7fa12a000640 1 -- 172.21.15.67:0/3362244094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1240078e0 con 0x7fa12c0d0b40 2024-05-30T16:58:18.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.212+0000 7fa12a000640 1 -- 172.21.15.67:0/3362244094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124007cb0 con 0x7fa12c0d0b40 2024-05-30T16:58:18.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.213+0000 7fa12be00640 1 -- 172.21.15.67:0/3362244094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c0d0b40 msgr2=0x7fa12c169740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:18.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.213+0000 7fa12be00640 1 --2- 172.21.15.67:0/3362244094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c0d0b40 0x7fa12c169740 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa124009d90 tx=0x7fa12402a6f0 comp rx=0 tx=0).stop 2024-05-30T16:58:18.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.213+0000 7fa12be00640 1 -- 172.21.15.67:0/3362244094 shutdown_connections 2024-05-30T16:58:18.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.213+0000 7fa12be00640 1 --2- 172.21.15.67:0/3362244094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c0d0b40 0x7fa12c169740 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:18.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.213+0000 7fa12be00640 1 -- 172.21.15.67:0/3362244094 >> 172.21.15.67:0/3362244094 conn(0x7fa12c06bae0 msgr2=0x7fa12c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:18.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.213+0000 7fa12be00640 1 -- 172.21.15.67:0/3362244094 shutdown_connections 2024-05-30T16:58:18.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.213+0000 7fa12be00640 1 -- 172.21.15.67:0/3362244094 wait complete. 2024-05-30T16:58:18.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.215+0000 7fa12be00640 1 Processor -- start 2024-05-30T16:58:18.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.216+0000 7fa12be00640 1 -- start start 2024-05-30T16:58:18.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.217+0000 7fa12be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0af6f0 0x7fa12c099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.217+0000 7fa12be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0d0b40 0x7fa12c06ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.217+0000 7fa12be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c1480a0 0x7fa12c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.217+0000 7fa12be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c0afc50 con 0x7fa12c1480a0 2024-05-30T16:58:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.217+0000 7fa12be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c0afad0 con 0x7fa12c0af6f0 2024-05-30T16:58:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.217+0000 7fa12be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa12c1438e0 con 0x7fa12c0d0b40 2024-05-30T16:58:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.217+0000 7fa12b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c1480a0 0x7fa12c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.217+0000 7fa12b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c1480a0 0x7fa12c06f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44020/0 (socket says 172.21.15.67:44020) 2024-05-30T16:58:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.217+0000 7fa12b400640 1 -- 172.21.15.67:0/3984725551 learned_addr learned my addr 172.21.15.67:0/3984725551 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:18.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa123e00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0d0b40 0x7fa12c06ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa12aa00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0af6f0 0x7fa12c099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa12b400640 1 -- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0d0b40 msgr2=0x7fa12c06ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa12b400640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0d0b40 0x7fa12c06ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa12b400640 1 -- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0af6f0 msgr2=0x7fa12c099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa12b400640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0af6f0 0x7fa12c099aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa12b400640 1 -- 172.21.15.67:0/3984725551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c0adff0 con 0x7fa12c1480a0 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa12aa00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0af6f0 0x7fa12c099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa123e00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0d0b40 0x7fa12c06ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa12b400640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c1480a0 0x7fa12c06f1b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa124009d90 tx=0x7fa124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa122a00640 1 -- 172.21.15.67:0/3984725551 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124040070 con 0x7fa12c1480a0 2024-05-30T16:58:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa122a00640 1 -- 172.21.15.67:0/3984725551 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124031c90 con 0x7fa12c1480a0 2024-05-30T16:58:18.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa122a00640 1 -- 172.21.15.67:0/3984725551 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12403c640 con 0x7fa12c1480a0 2024-05-30T16:58:18.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.218+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c160d20 con 0x7fa12c1480a0 2024-05-30T16:58:18.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.219+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c160940 con 0x7fa12c1480a0 2024-05-30T16:58:18.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.220+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12c0689e0 con 0x7fa12c1480a0 2024-05-30T16:58:18.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.221+0000 7fa122a00640 1 -- 172.21.15.67:0/3984725551 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fa124039040 con 0x7fa12c1480a0 2024-05-30T16:58:18.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.222+0000 7fa122a00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa10806da90 0x7fa10806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:18.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.222+0000 7fa122a00640 1 -- 172.21.15.67:0/3984725551 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5471+0+0 (secure 0 0 0) 0x7fa1240ac0c0 con 0x7fa12c1480a0 2024-05-30T16:58:18.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.223+0000 7fa12aa00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa10806da90 0x7fa10806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:18.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.224+0000 7fa12aa00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa10806da90 0x7fa10806ff50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000f60 tx=0x7fa11c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:18.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.228+0000 7fa122a00640 1 -- 172.21.15.67:0/3984725551 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa12404b340 con 0x7fa12c1480a0 2024-05-30T16:58:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.432+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa12c0c9e50 con 0x7fa12c1480a0 2024-05-30T16:58:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.988+0000 7fa122a00640 1 -- 172.21.15.67:0/3984725551 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) ==== 169+0+0 (secure 0 0 0) 0x7fa12407bd10 con 0x7fa12c1480a0 2024-05-30T16:58:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-05-30T16:58:18.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.993+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa10806da90 msgr2=0x7fa10806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:18.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.993+0000 7fa12be00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa10806da90 0x7fa10806ff50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000f60 tx=0x7fa11c009040 comp rx=0 tx=0).stop 2024-05-30T16:58:18.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.993+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c1480a0 msgr2=0x7fa12c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.993+0000 7fa12be00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c1480a0 0x7fa12c06f1b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa124009d90 tx=0x7fa124004a40 comp rx=0 tx=0).stop 2024-05-30T16:58:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.995+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 shutdown_connections 2024-05-30T16:58:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.995+0000 7fa12be00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa10806da90 0x7fa10806ff50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.995+0000 7fa12be00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa12c1480a0 0x7fa12c06f1b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.995+0000 7fa12be00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0d0b40 0x7fa12c06ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.995+0000 7fa12be00640 1 --2- 172.21.15.67:0/3984725551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0af6f0 0x7fa12c099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.995+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 >> 172.21.15.67:0/3984725551 conn(0x7fa12c06bae0 msgr2=0x7fa12c162500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.995+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 shutdown_connections 2024-05-30T16:58:18.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:18.995+0000 7fa12be00640 1 -- 172.21.15.67:0/3984725551 wait complete. 2024-05-30T16:58:19.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_no_wait 2024-05-30T16:58:19.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.252+0000 7f586fa00640 1 Processor -- start 2024-05-30T16:58:19.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.252+0000 7f586fa00640 1 -- start start 2024-05-30T16:58:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.254+0000 7f586fa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5868162ac0 0x7f58680d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.254+0000 7f586fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58680b4120 0x7f58680d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.254+0000 7f586fa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58681640e0 con 0x7f58681633b0 2024-05-30T16:58:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.254+0000 7f586fa00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f58680b3c90 con 0x7f5868162ac0 2024-05-30T16:58:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.254+0000 7f586fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f58680b3360 con 0x7f58680b4120 2024-05-30T16:58:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.254+0000 7f586d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5868162ac0 0x7f58680d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.254+0000 7f586d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5868162ac0 0x7f58680d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44024/0 (socket says 172.21.15.67:44024) 2024-05-30T16:58:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.254+0000 7f586d000640 1 -- 172.21.15.67:0/1892283776 learned_addr learned my addr 172.21.15.67:0/1892283776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.255+0000 7f586d000640 1 -- 172.21.15.67:0/1892283776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58680b4120 msgr2=0x7f58680d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:19.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.255+0000 7f586d000640 1 --2- 172.21.15.67:0/1892283776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58680b4120 0x7f58680d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.255+0000 7f586d000640 1 -- 172.21.15.67:0/1892283776 >> v1:172.21.15.169:6789/0 conn(0x7f58681633b0 legacy=0x7f5868163820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:19.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.255+0000 7f586d000640 1 -- 172.21.15.67:0/1892283776 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58680b3640 con 0x7f5868162ac0 2024-05-30T16:58:19.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.256+0000 7f586d000640 1 --2- 172.21.15.67:0/1892283776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5868162ac0 0x7f58680d0640 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f585c009870 tx=0x7f585c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5934e4aada4efc88 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:19.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.256+0000 7f5867400640 1 -- 172.21.15.67:0/1892283776 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c037070 con 0x7f5868162ac0 2024-05-30T16:58:19.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.256+0000 7f5867400640 1 -- 172.21.15.67:0/1892283776 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f585c032440 con 0x7f5868162ac0 2024-05-30T16:58:19.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.256+0000 7f5867400640 1 -- 172.21.15.67:0/1892283776 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c032810 con 0x7f5868162ac0 2024-05-30T16:58:19.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.257+0000 7f586fa00640 1 -- 172.21.15.67:0/1892283776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5868162ac0 msgr2=0x7f58680d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.257+0000 7f586fa00640 1 --2- 172.21.15.67:0/1892283776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5868162ac0 0x7f58680d0640 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f585c009870 tx=0x7f585c02a210 comp rx=0 tx=0).stop 2024-05-30T16:58:19.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.257+0000 7f586fa00640 1 -- 172.21.15.67:0/1892283776 shutdown_connections 2024-05-30T16:58:19.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.257+0000 7f586fa00640 1 --2- 172.21.15.67:0/1892283776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58680b4120 0x7f58680d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.257+0000 7f586fa00640 1 --2- 172.21.15.67:0/1892283776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5868162ac0 0x7f58680d0640 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.257+0000 7f586fa00640 1 -- 172.21.15.67:0/1892283776 >> 172.21.15.67:0/1892283776 conn(0x7f586806bd20 msgr2=0x7f586806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:19.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.257+0000 7f586fa00640 1 -- 172.21.15.67:0/1892283776 shutdown_connections 2024-05-30T16:58:19.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.257+0000 7f586fa00640 1 -- 172.21.15.67:0/1892283776 wait complete. 2024-05-30T16:58:19.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.259+0000 7f586fa00640 1 Processor -- start 2024-05-30T16:58:19.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.259+0000 7f586fa00640 1 -- start start 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586fa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58680b4120 0x7f5868151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5868162ac0 0x7f58680cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586fa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58681633b0 0x7f58680cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586fa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5868163020 con 0x7f58681633b0 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586fa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5868162ea0 con 0x7f58680b4120 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f586806dbd0 con 0x7f5868162ac0 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58681633b0 0x7f58680cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58681633b0 0x7f58680cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44032/0 (socket says 172.21.15.67:44032) 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586da00640 1 -- 172.21.15.67:0/2012740950 learned_addr learned my addr 172.21.15.67:0/2012740950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:19.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.260+0000 7f586d000640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58680b4120 0x7f5868151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f586da00640 1 -- 172.21.15.67:0/2012740950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5868162ac0 msgr2=0x7f58680cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f586da00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5868162ac0 0x7f58680cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f586da00640 1 -- 172.21.15.67:0/2012740950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58680b4120 msgr2=0x7f5868151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f586da00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58680b4120 0x7f5868151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f586da00640 1 -- 172.21.15.67:0/2012740950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586809cda0 con 0x7f58681633b0 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f586da00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58681633b0 0x7f58680cf1c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f586000b990 tx=0x7f586000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f5864c00640 1 -- 172.21.15.67:0/2012740950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860019070 con 0x7f58681633b0 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f5864c00640 1 -- 172.21.15.67:0/2012740950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586000cba0 con 0x7f58681633b0 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f5864c00640 1 -- 172.21.15.67:0/2012740950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860014440 con 0x7f58681633b0 2024-05-30T16:58:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.261+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58681fc890 con 0x7f58681633b0 2024-05-30T16:58:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.262+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681fce50 con 0x7f58681633b0 2024-05-30T16:58:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.263+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58680689e0 con 0x7f58681633b0 2024-05-30T16:58:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.264+0000 7f5864c00640 1 -- 172.21.15.67:0/2012740950 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f586001c070 con 0x7f58681633b0 2024-05-30T16:58:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.265+0000 7f5864c00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f584806da60 0x7f584806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.265+0000 7f5864c00640 1 -- 172.21.15.67:0/2012740950 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f586008a800 con 0x7f58681633b0 2024-05-30T16:58:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.265+0000 7f586d000640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f584806da60 0x7f584806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.266+0000 7f586d000640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f584806da60 0x7f584806ff20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f585c034000 tx=0x7f585c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:19.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.270+0000 7f5864c00640 1 -- 172.21.15.67:0/2012740950 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f586005a330 con 0x7f58681633b0 2024-05-30T16:58:19.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.479+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"} v 0) -- 0x7f58680afc60 con 0x7f58681633b0 2024-05-30T16:58:19.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.486+0000 7f5864c00640 1 -- 172.21.15.67:0/2012740950 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"}]=0 v20) ==== 117+0+0 (secure 0 0 0) 0x7f5860059c30 con 0x7f58681633b0 2024-05-30T16:58:19.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f584806da60 msgr2=0x7f584806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f584806da60 0x7f584806ff20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f585c034000 tx=0x7f585c02e000 comp rx=0 tx=0).stop 2024-05-30T16:58:19.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58681633b0 msgr2=0x7f58680cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58681633b0 0x7f58680cf1c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f586000b990 tx=0x7f586000be50 comp rx=0 tx=0).stop 2024-05-30T16:58:19.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 shutdown_connections 2024-05-30T16:58:19.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f584806da60 0x7f584806ff20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58681633b0 0x7f58680cf1c0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5868162ac0 0x7f58680cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 --2- 172.21.15.67:0/2012740950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58680b4120 0x7f5868151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 >> 172.21.15.67:0/2012740950 conn(0x7f586806bd20 msgr2=0x7f58681647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:19.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 shutdown_connections 2024-05-30T16:58:19.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.489+0000 7f586fa00640 1 -- 172.21.15.67:0/2012740950 wait complete. 2024-05-30T16:58:19.511 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-30T16:58:19.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.716+0000 7f1bda200640 1 Processor -- start 2024-05-30T16:58:19.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.716+0000 7f1bda200640 1 -- start start 2024-05-30T16:58:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.717+0000 7f1bda200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd4153510 0x7f1bd40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bda200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1bd415d270 0x7f1bd40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bda200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd415d900 con 0x7f1bd40ccb90 2024-05-30T16:58:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bda200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd41538f0 con 0x7f1bd415d270 2024-05-30T16:58:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bda200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd40cd530 con 0x7f1bd4153510 2024-05-30T16:58:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bd3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1bd40ccb90 0x7f1bd415cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54412/0 (socket says 172.21.15.67:54412) 2024-05-30T16:58:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bd3e00640 1 -- 172.21.15.67:0/1327909006 learned_addr learned my addr 172.21.15.67:0/1327909006 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bd9800640 1 --2- 172.21.15.67:0/1327909006 >> v2:172.21.15.67:3300/0 conn(0x7f1bd415d270 0x7f1bd40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bd8e00640 1 --2- 172.21.15.67:0/1327909006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd4153510 0x7f1bd40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bd3400640 1 -- 172.21.15.67:0/1327909006 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1278717953 0 0) 0x7f1bd415d900 con 0x7f1bd40ccb90 2024-05-30T16:58:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.718+0000 7f1bd3400640 1 -- 172.21.15.67:0/1327909006 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bb8003520 con 0x7f1bd40ccb90 2024-05-30T16:58:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.719+0000 7f1bd9800640 1 -- 172.21.15.67:0/1327909006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd4153510 msgr2=0x7f1bd40b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.719+0000 7f1bd9800640 1 --2- 172.21.15.67:0/1327909006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd4153510 0x7f1bd40b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.719+0000 7f1bd9800640 1 -- 172.21.15.67:0/1327909006 >> v1:172.21.15.67:6789/0 conn(0x7f1bd40ccb90 legacy=0x7f1bd415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.719+0000 7f1bd9800640 1 -- 172.21.15.67:0/1327909006 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd40b3f90 con 0x7f1bd415d270 2024-05-30T16:58:19.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.720+0000 7f1bd9800640 1 --2- 172.21.15.67:0/1327909006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd415d270 0x7f1bd40ccff0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009d90 tx=0x7f1bcc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f44c7d85c274876 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:19.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.720+0000 7f1bd3400640 1 -- 172.21.15.67:0/1327909006 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc031600 con 0x7f1bd415d270 2024-05-30T16:58:19.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.720+0000 7f1bd3400640 1 -- 172.21.15.67:0/1327909006 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0317a0 con 0x7f1bd415d270 2024-05-30T16:58:19.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.720+0000 7f1bd3400640 1 -- 172.21.15.67:0/1327909006 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc031b70 con 0x7f1bd415d270 2024-05-30T16:58:19.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.721+0000 7f1bda200640 1 -- 172.21.15.67:0/1327909006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd415d270 msgr2=0x7f1bd40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.721+0000 7f1bda200640 1 --2- 172.21.15.67:0/1327909006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd415d270 0x7f1bd40ccff0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009d90 tx=0x7f1bcc02a5d0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.721+0000 7f1bda200640 1 -- 172.21.15.67:0/1327909006 shutdown_connections 2024-05-30T16:58:19.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.721+0000 7f1bda200640 1 --2- 172.21.15.67:0/1327909006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd415d270 0x7f1bd40ccff0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.721+0000 7f1bda200640 1 --2- 172.21.15.67:0/1327909006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd4153510 0x7f1bd40b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.721+0000 7f1bda200640 1 -- 172.21.15.67:0/1327909006 >> 172.21.15.67:0/1327909006 conn(0x7f1bd406b900 msgr2=0x7f1bd406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:19.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.721+0000 7f1bda200640 1 -- 172.21.15.67:0/1327909006 shutdown_connections 2024-05-30T16:58:19.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.721+0000 7f1bda200640 1 -- 172.21.15.67:0/1327909006 wait complete. 2024-05-30T16:58:19.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.723+0000 7f1bda200640 1 Processor -- start 2024-05-30T16:58:19.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.723+0000 7f1bda200640 1 -- start start 2024-05-30T16:58:19.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bda200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd40ccb90 0x7f1bd414a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bda200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd4153510 0x7f1bd4072d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bda200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd415d270 0x7f1bd40ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bda200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd40b4550 con 0x7f1bd40ccb90 2024-05-30T16:58:19.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bda200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd40b48c0 con 0x7f1bd4153510 2024-05-30T16:58:19.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bda200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd406e940 con 0x7f1bd415d270 2024-05-30T16:58:19.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bd8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd40ccb90 0x7f1bd414a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:19.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bd8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd40ccb90 0x7f1bd414a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44052/0 (socket says 172.21.15.67:44052) 2024-05-30T16:58:19.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bd8e00640 1 -- 172.21.15.67:0/3749166731 learned_addr learned my addr 172.21.15.67:0/3749166731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:19.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bd3e00640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd4153510 0x7f1bd4072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:19.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bd8e00640 1 -- 172.21.15.67:0/3749166731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd415d270 msgr2=0x7f1bd40ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bd8e00640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd415d270 0x7f1bd40ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bd8e00640 1 -- 172.21.15.67:0/3749166731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd4153510 msgr2=0x7f1bd4072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bd8e00640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd4153510 0x7f1bd4072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.724+0000 7f1bd8e00640 1 -- 172.21.15.67:0/3749166731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd40ad260 con 0x7f1bd40ccb90 2024-05-30T16:58:19.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.725+0000 7f1bd8e00640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd40ccb90 0x7f1bd414a8b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400d830 tx=0x7f1bc400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:19.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.725+0000 7f1bd0c00640 1 -- 172.21.15.67:0/3749166731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc400b6c0 con 0x7f1bd40ccb90 2024-05-30T16:58:19.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.725+0000 7f1bd0c00640 1 -- 172.21.15.67:0/3749166731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc4019070 con 0x7f1bd40ccb90 2024-05-30T16:58:19.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.725+0000 7f1bd0c00640 1 -- 172.21.15.67:0/3749166731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc400ba80 con 0x7f1bd40ccb90 2024-05-30T16:58:19.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.725+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd40ad540 con 0x7f1bd40ccb90 2024-05-30T16:58:19.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.725+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd40adb00 con 0x7f1bd40ccb90 2024-05-30T16:58:19.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.726+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd40689e0 con 0x7f1bd40ccb90 2024-05-30T16:58:19.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.727+0000 7f1bd0c00640 1 -- 172.21.15.67:0/3749166731 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f1bc401e460 con 0x7f1bd40ccb90 2024-05-30T16:58:19.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.728+0000 7f1bd0c00640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bb406da60 0x7f1bb406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:19.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.728+0000 7f1bd0c00640 1 -- 172.21.15.67:0/3749166731 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f1bc408b120 con 0x7f1bd40ccb90 2024-05-30T16:58:19.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.729+0000 7f1bd3e00640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bb406da60 0x7f1bb406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:19.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.730+0000 7f1bd3e00640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bb406da60 0x7f1bb406ff20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4150a00 tx=0x7f1bc0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:19.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.733+0000 7f1bd0c00640 1 -- 172.21.15.67:0/3749166731 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bc405acb0 con 0x7f1bd40ccb90 2024-05-30T16:58:19.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.941+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1bd40b3c40 con 0x7f1bd40ccb90 2024-05-30T16:58:19.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.946+0000 7f1bd0c00640 1 -- 172.21.15.67:0/3749166731 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f1bc405a5b0 con 0x7f1bd40ccb90 2024-05-30T16:58:19.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bb406da60 msgr2=0x7f1bb406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bb406da60 0x7f1bb406ff20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4150a00 tx=0x7f1bc0009040 comp rx=0 tx=0).stop 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd40ccb90 msgr2=0x7f1bd414a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd40ccb90 0x7f1bd414a8b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400d830 tx=0x7f1bc400dcf0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 shutdown_connections 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1bb406da60 0x7f1bb406ff20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bd415d270 0x7f1bd40ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bd4153510 0x7f1bd4072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 --2- 172.21.15.67:0/3749166731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bd40ccb90 0x7f1bd414a8b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 >> 172.21.15.67:0/3749166731 conn(0x7f1bd406b900 msgr2=0x7f1bd40ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 shutdown_connections 2024-05-30T16:58:19.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:19.949+0000 7f1bda200640 1 -- 172.21.15.67:0/3749166731 wait complete. 2024-05-30T16:58:19.970 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-05-30T16:58:20.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.161+0000 7fce5fe00640 1 Processor -- start 2024-05-30T16:58:20.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.162+0000 7fce5fe00640 1 -- start start 2024-05-30T16:58:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.163+0000 7fce5fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fce580d04a0 0x7fce580d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.163+0000 7fce5fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fce580b3460 0x7fce580b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.163+0000 7fce5fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce580ced40 con 0x7fce580b4b60 2024-05-30T16:58:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.163+0000 7fce5fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fce5815fbb0 con 0x7fce580d04a0 2024-05-30T16:58:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.163+0000 7fce5fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fce580cefc0 con 0x7fce580b3460 2024-05-30T16:58:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.163+0000 7fce5ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fce580d04a0 0x7fce580d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.163+0000 7fce5d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fce580b4b60 0x7fce580cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54420/0 (socket says 172.21.15.67:54420) 2024-05-30T16:58:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.163+0000 7fce5d400640 1 -- 172.21.15.67:0/524031846 learned_addr learned my addr 172.21.15.67:0/524031846 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.164+0000 7fce5de00640 1 --2- 172.21.15.67:0/524031846 >> v2:172.21.15.169:3300/0 conn(0x7fce580b3460 0x7fce580b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.164+0000 7fce5ca00640 1 -- 172.21.15.67:0/524031846 >> v2:172.21.15.169:3300/0 conn(0x7fce580b3460 msgr2=0x7fce580b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.164+0000 7fce5ca00640 1 --2- 172.21.15.67:0/524031846 >> v2:172.21.15.169:3300/0 conn(0x7fce580b3460 0x7fce580b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.164+0000 7fce5ca00640 1 -- 172.21.15.67:0/524031846 >> v1:172.21.15.67:6789/0 conn(0x7fce580b4b60 legacy=0x7fce580cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.164+0000 7fce57e00640 1 -- 172.21.15.67:0/524031846 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 802702898 0 0) 0x7fce4c001110 con 0x7fce580b4b60 2024-05-30T16:58:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.164+0000 7fce5ca00640 1 -- 172.21.15.67:0/524031846 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce580cf2a0 con 0x7fce580d04a0 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.165+0000 7fce5ca00640 1 --2- 172.21.15.67:0/524031846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580d04a0 0x7fce580d0930 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fce580af320 tx=0x7fce4402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5216ac7c54d2ec4a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.165+0000 7fce57e00640 1 -- 172.21.15.67:0/524031846 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44037070 con 0x7fce580d04a0 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.165+0000 7fce57e00640 1 -- 172.21.15.67:0/524031846 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce44032440 con 0x7fce580d04a0 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.165+0000 7fce57e00640 1 -- 172.21.15.67:0/524031846 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce44032810 con 0x7fce580d04a0 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.166+0000 7fce5fe00640 1 -- 172.21.15.67:0/524031846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580d04a0 msgr2=0x7fce580d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.166+0000 7fce5fe00640 1 --2- 172.21.15.67:0/524031846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580d04a0 0x7fce580d0930 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fce580af320 tx=0x7fce4402a1a0 comp rx=0 tx=0).stop 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.166+0000 7fce5fe00640 1 -- 172.21.15.67:0/524031846 shutdown_connections 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.166+0000 7fce5fe00640 1 --2- 172.21.15.67:0/524031846 >> v2:172.21.15.169:3300/0 conn(0x7fce580b3460 0x7fce580b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.166+0000 7fce5fe00640 1 --2- 172.21.15.67:0/524031846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580d04a0 0x7fce580d0930 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.166+0000 7fce5fe00640 1 -- 172.21.15.67:0/524031846 >> 172.21.15.67:0/524031846 conn(0x7fce5806bae0 msgr2=0x7fce5806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.166+0000 7fce5fe00640 1 -- 172.21.15.67:0/524031846 shutdown_connections 2024-05-30T16:58:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.166+0000 7fce5fe00640 1 -- 172.21.15.67:0/524031846 wait complete. 2024-05-30T16:58:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.168+0000 7fce5fe00640 1 Processor -- start 2024-05-30T16:58:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.168+0000 7fce5fe00640 1 -- start start 2024-05-30T16:58:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.169+0000 7fce5fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580b3460 0x7fce581f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.169+0000 7fce5fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce580b4b60 0x7fce5809c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.169+0000 7fce5fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce580d04a0 0x7fce581f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.169+0000 7fce5fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fce580b3f40 con 0x7fce580b3460 2024-05-30T16:58:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.169+0000 7fce5fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce580b3dc0 con 0x7fce580b4b60 2024-05-30T16:58:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.169+0000 7fce5fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fce5815ede0 con 0x7fce580d04a0 2024-05-30T16:58:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.169+0000 7fce5d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580b3460 0x7fce581f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.170+0000 7fce5d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580b3460 0x7fce581f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44082/0 (socket says 172.21.15.67:44082) 2024-05-30T16:58:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.170+0000 7fce5d400640 1 -- 172.21.15.67:0/808854145 learned_addr learned my addr 172.21.15.67:0/808854145 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.170+0000 7fce5ca00640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce580b4b60 0x7fce5809c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.170+0000 7fce5d400640 1 -- 172.21.15.67:0/808854145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce580d04a0 msgr2=0x7fce581f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.170+0000 7fce5d400640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce580d04a0 0x7fce581f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.170+0000 7fce5d400640 1 -- 172.21.15.67:0/808854145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce580b4b60 msgr2=0x7fce5809c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.170+0000 7fce5d400640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce580b4b60 0x7fce5809c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.170+0000 7fce5d400640 1 -- 172.21.15.67:0/808854145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce581f81e0 con 0x7fce580b3460 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.170+0000 7fce5d400640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580b3460 0x7fce581f4c20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c780 tx=0x7fce4c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.171+0000 7fce55600640 1 -- 172.21.15.67:0/808854145 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c018070 con 0x7fce580b3460 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.171+0000 7fce55600640 1 -- 172.21.15.67:0/808854145 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4c0072d0 con 0x7fce580b3460 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.171+0000 7fce55600640 1 -- 172.21.15.67:0/808854145 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0047e0 con 0x7fce580b3460 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.171+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce581f84c0 con 0x7fce580b3460 2024-05-30T16:58:20.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.171+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce581f4700 con 0x7fce580b3460 2024-05-30T16:58:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.172+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce580689e0 con 0x7fce580b3460 2024-05-30T16:58:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.173+0000 7fce55600640 1 -- 172.21.15.67:0/808854145 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fce4c01b070 con 0x7fce580b3460 2024-05-30T16:58:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.174+0000 7fce55600640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fce3806da60 0x7fce3806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.174+0000 7fce55600640 1 -- 172.21.15.67:0/808854145 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7fce4c089de0 con 0x7fce580b3460 2024-05-30T16:58:20.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.174+0000 7fce5ca00640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fce3806da60 0x7fce3806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:20.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.175+0000 7fce5ca00640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fce3806da60 0x7fce3806ff20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce44034000 tx=0x7fce4402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.179+0000 7fce55600640 1 -- 172.21.15.67:0/808854145 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce580ced40 con 0x7fce580b3460 2024-05-30T16:58:20.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:20.469+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7fce58161880 con 0x7fce580b3460 2024-05-30T16:58:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.018+0000 7fce55600640 1 -- 172.21.15.67:0/808854145 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v605) ==== 174+0+0 (secure 0 0 0) 0x7fce4c059910 con 0x7fce580b3460 2024-05-30T16:58:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.021+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fce3806da60 msgr2=0x7fce3806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.021+0000 7fce5fe00640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fce3806da60 0x7fce3806ff20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce44034000 tx=0x7fce4402e000 comp rx=0 tx=0).stop 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.021+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580b3460 msgr2=0x7fce581f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.021+0000 7fce5fe00640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580b3460 0x7fce581f4c20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c780 tx=0x7fce4c00cc40 comp rx=0 tx=0).stop 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.022+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 shutdown_connections 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.022+0000 7fce5fe00640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fce3806da60 0x7fce3806ff20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.022+0000 7fce5fe00640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce580d04a0 0x7fce581f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.022+0000 7fce5fe00640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce580b4b60 0x7fce5809c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.022+0000 7fce5fe00640 1 --2- 172.21.15.67:0/808854145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce580b3460 0x7fce581f4c20 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.022+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 >> 172.21.15.67:0/808854145 conn(0x7fce5806bae0 msgr2=0x7fce580b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.022+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 shutdown_connections 2024-05-30T16:58:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.022+0000 7fce5fe00640 1 -- 172.21.15.67:0/808854145 wait complete. 2024-05-30T16:58:21.046 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-05-30T16:58:21.046 INFO:tasks.cephfs_test_runner:Starting test: test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-05-30T16:58:21.046 INFO:tasks.cephfs_test_runner:test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T16:58:21.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:58:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.260+0000 7f4733c00640 1 Processor -- start 2024-05-30T16:58:21.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.260+0000 7f4733c00640 1 -- start start 2024-05-30T16:58:21.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.261+0000 7f4733c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f472c162e70 0x7f472c16d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.261+0000 7f4733c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f472c0b4350 con 0x7f472c0d04a0 2024-05-30T16:58:21.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.261+0000 7f4733c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f472c0b32d0 con 0x7f472c06f6e0 2024-05-30T16:58:21.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.261+0000 7f4733c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f472c0b3ec0 con 0x7f472c162e70 2024-05-30T16:58:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.262+0000 7f4731200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f472c0d04a0 0x7f472c0d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56284/0 (socket says 172.21.15.67:56284) 2024-05-30T16:58:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.262+0000 7f4731c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f472c162e70 0x7f472c16d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.262+0000 7f4731200640 1 -- 172.21.15.67:0/3933403591 learned_addr learned my addr 172.21.15.67:0/3933403591 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.262+0000 7f4731c00640 1 -- 172.21.15.67:0/3933403591 >> v1:172.21.15.169:6789/0 conn(0x7f472c06f6e0 legacy=0x7f472c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.262+0000 7f4731c00640 1 -- 172.21.15.67:0/3933403591 >> v1:172.21.15.67:6789/0 conn(0x7f472c0d04a0 legacy=0x7f472c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.262+0000 7f4731c00640 1 -- 172.21.15.67:0/3933403591 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c0b3590 con 0x7f472c162e70 2024-05-30T16:58:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.262+0000 7f472b400640 1 -- 172.21.15.67:0/3933403591 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3638949488 0 0) 0x7f472c0b4350 con 0x7f472c0d04a0 2024-05-30T16:58:21.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.263+0000 7f4731c00640 1 --2- 172.21.15.67:0/3933403591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c162e70 0x7f472c16d7f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f47240097a0 tx=0x7f472402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b1de36ea23180d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:21.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.264+0000 7f472b400640 1 -- 172.21.15.67:0/3933403591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724037070 con 0x7f472c162e70 2024-05-30T16:58:21.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.264+0000 7f472b400640 1 -- 172.21.15.67:0/3933403591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4724032440 con 0x7f472c162e70 2024-05-30T16:58:21.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.264+0000 7f472b400640 1 -- 172.21.15.67:0/3933403591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724032810 con 0x7f472c162e70 2024-05-30T16:58:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.264+0000 7f4733c00640 1 -- 172.21.15.67:0/3933403591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c162e70 msgr2=0x7f472c16d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.264+0000 7f4733c00640 1 --2- 172.21.15.67:0/3933403591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c162e70 0x7f472c16d7f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f47240097a0 tx=0x7f472402a1a0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.264+0000 7f4733c00640 1 -- 172.21.15.67:0/3933403591 shutdown_connections 2024-05-30T16:58:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.264+0000 7f4733c00640 1 --2- 172.21.15.67:0/3933403591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c162e70 0x7f472c16d7f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.264+0000 7f4733c00640 1 -- 172.21.15.67:0/3933403591 >> 172.21.15.67:0/3933403591 conn(0x7f472c06bae0 msgr2=0x7f472c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.265+0000 7f4733c00640 1 -- 172.21.15.67:0/3933403591 shutdown_connections 2024-05-30T16:58:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.265+0000 7f4733c00640 1 -- 172.21.15.67:0/3933403591 wait complete. 2024-05-30T16:58:21.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.266+0000 7f4733c00640 1 Processor -- start 2024-05-30T16:58:21.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.266+0000 7f4733c00640 1 -- start start 2024-05-30T16:58:21.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.267+0000 7f4733c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f472c06f6e0 0x7f472c159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.267+0000 7f4733c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c0d04a0 0x7f472c15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.267+0000 7f4733c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f472c162e70 0x7f472c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.267+0000 7f4733c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f472c0cf240 con 0x7f472c0d04a0 2024-05-30T16:58:21.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.267+0000 7f4733c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f472c0b3b50 con 0x7f472c162e70 2024-05-30T16:58:21.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.267+0000 7f4733c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f472c0cf3c0 con 0x7f472c06f6e0 2024-05-30T16:58:21.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f472be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c0d04a0 0x7f472c15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f472be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c0d04a0 0x7f472c15dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49202/0 (socket says 172.21.15.67:49202) 2024-05-30T16:58:21.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f472be00640 1 -- 172.21.15.67:0/1360198607 learned_addr learned my addr 172.21.15.67:0/1360198607 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f4731200640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f472c06f6e0 0x7f472c159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f472be00640 1 -- 172.21.15.67:0/1360198607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f472c06f6e0 msgr2=0x7f472c159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f472be00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f472c06f6e0 0x7f472c159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f472be00640 1 -- 172.21.15.67:0/1360198607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f472c162e70 msgr2=0x7f472c0c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f4731c00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f472c162e70 0x7f472c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f472be00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f472c162e70 0x7f472c0c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f472be00640 1 -- 172.21.15.67:0/1360198607 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c159860 con 0x7f472c0d04a0 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.268+0000 7f472be00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c0d04a0 0x7f472c15dfd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f471c00c7a0 tx=0x7f471c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.269+0000 7f4728c00640 1 -- 172.21.15.67:0/1360198607 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c003b70 con 0x7f472c0d04a0 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.269+0000 7f4728c00640 1 -- 172.21.15.67:0/1360198607 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f471c003d10 con 0x7f472c0d04a0 2024-05-30T16:58:21.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.269+0000 7f4728c00640 1 -- 172.21.15.67:0/1360198607 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c00aa30 con 0x7f472c0d04a0 2024-05-30T16:58:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.269+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c163250 con 0x7f472c0d04a0 2024-05-30T16:58:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.269+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c163810 con 0x7f472c0d04a0 2024-05-30T16:58:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.270+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f472c0689e0 con 0x7f472c0d04a0 2024-05-30T16:58:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.271+0000 7f4728c00640 1 -- 172.21.15.67:0/1360198607 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f471c00abd0 con 0x7f472c0d04a0 2024-05-30T16:58:21.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.272+0000 7f4728c00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f470806da60 0x7f470806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.272+0000 7f4728c00640 1 -- 172.21.15.67:0/1360198607 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f471c08a9b0 con 0x7f472c0d04a0 2024-05-30T16:58:21.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.272+0000 7f4731200640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f470806da60 0x7f470806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.273+0000 7f4731200640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f470806da60 0x7f470806ff20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4714007fd0 tx=0x7f4714009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:21.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.277+0000 7f4728c00640 1 -- 172.21.15.67:0/1360198607 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f472c159860 con 0x7f472c0d04a0 2024-05-30T16:58:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.481+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f472c06cc30 con 0x7f472c0d04a0 2024-05-30T16:58:21.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.484+0000 7f4728c00640 1 -- 172.21.15.67:0/1360198607 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+13096 (secure 0 0 0) 0x7f471c05a4e0 con 0x7f472c0d04a0 2024-05-30T16:58:21.485 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:21.485 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":81,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:58:18.984609+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6837/2706164583":"2024-05-31T16:58:15.963769+0000","172.21.15.67:6836/2706164583":"2024-05-31T16:58:15.963769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:58:21.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.486+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f470806da60 msgr2=0x7f470806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.486+0000 7f4733c00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f470806da60 0x7f470806ff20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4714007fd0 tx=0x7f4714009040 comp rx=0 tx=0).stop 2024-05-30T16:58:21.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.486+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c0d04a0 msgr2=0x7f472c15dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.486+0000 7f4733c00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c0d04a0 0x7f472c15dfd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f471c00c7a0 tx=0x7f471c00cc60 comp rx=0 tx=0).stop 2024-05-30T16:58:21.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.487+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 shutdown_connections 2024-05-30T16:58:21.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.487+0000 7f4733c00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f470806da60 0x7f470806ff20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.487+0000 7f4733c00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f472c162e70 0x7f472c0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.487+0000 7f4733c00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f472c0d04a0 0x7f472c15dfd0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.487+0000 7f4733c00640 1 --2- 172.21.15.67:0/1360198607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f472c06f6e0 0x7f472c159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.487+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 >> 172.21.15.67:0/1360198607 conn(0x7f472c06bae0 msgr2=0x7f472c0ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:21.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.487+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 shutdown_connections 2024-05-30T16:58:21.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.487+0000 7f4733c00640 1 -- 172.21.15.67:0/1360198607 wait complete. 2024-05-30T16:58:21.506 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-30T16:43:40.009725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-30T16:58:21.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-30T16:58:21.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.725+0000 7f9646a00640 1 Processor -- start 2024-05-30T16:58:21.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.725+0000 7f9646a00640 1 -- start start 2024-05-30T16:58:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.727+0000 7f9646a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f964015d270 0x7f96400b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.727+0000 7f9646a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f964015d900 con 0x7f96400cd2a0 2024-05-30T16:58:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.727+0000 7f9646a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96400b3f90 con 0x7f9640153510 2024-05-30T16:58:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.727+0000 7f9646a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f96401538f0 con 0x7f964015d270 2024-05-30T16:58:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.727+0000 7f9644c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f96400cd2a0 0x7f964015cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56296/0 (socket says 172.21.15.67:56296) 2024-05-30T16:58:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.727+0000 7f9644c00640 1 -- 172.21.15.67:0/2998260653 learned_addr learned my addr 172.21.15.67:0/2998260653 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f9646000640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f964015d270 0x7f96400b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3540634687 0 0) 0x7f964015d900 con 0x7f96400cd2a0 2024-05-30T16:58:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9628003520 con 0x7f96400cd2a0 2024-05-30T16:58:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1826242563 0 0) 0x7f9628003520 con 0x7f96400cd2a0 2024-05-30T16:58:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f964015d270 msgr2=0x7f96400b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f964015d270 0x7f96400b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 >> v1:172.21.15.169:6789/0 conn(0x7f9640153510 legacy=0x7f96400ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96400b4270 con 0x7f96400cd2a0 2024-05-30T16:58:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f962c001670 con 0x7f96400cd2a0 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 >> v1:172.21.15.67:6789/0 conn(0x7f96400cd2a0 legacy=0x7f964015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f964015d270 0x7f96280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.728+0000 7f963fe00640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9628003fb0 0x7f9628004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f963fe00640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9628004d40 0x7f96280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f962800bb80 con 0x7f9628004d40 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f962800bd30 con 0x7f9628003fb0 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f962800bf30 con 0x7f964015d270 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f9646000640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9628004d40 0x7f96280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f9645600640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f964015d270 0x7f96280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f9644c00640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9628003fb0 0x7f9628004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f9646000640 1 -- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f964015d270 msgr2=0x7f96280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f9646000640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f964015d270 0x7f96280039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f9646000640 1 -- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9628003fb0 msgr2=0x7f9628004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:21.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f9646000640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9628003fb0 0x7f9628004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.729+0000 7f9646000640 1 -- 172.21.15.67:0/2998260653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9628003520 con 0x7f9628004d40 2024-05-30T16:58:21.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.730+0000 7f9646000640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9628004d40 0x7f96280051c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fc0 tx=0x7f9628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62912dd8486e75c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.731+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638009c10 con 0x7f9628004d40 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.731+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9638036070 con 0x7f9628004d40 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.731+0000 7f963fe00640 1 -- 172.21.15.67:0/2998260653 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638031880 con 0x7f9628004d40 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.731+0000 7f9646a00640 1 -- 172.21.15.67:0/2998260653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9628004d40 msgr2=0x7f96280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.731+0000 7f9646a00640 1 --2- 172.21.15.67:0/2998260653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9628004d40 0x7f96280051c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fc0 tx=0x7f9628005830 comp rx=0 tx=0).stop 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.731+0000 7f9646000640 1 -- 172.21.15.67:0/2998260653 reap_dead start 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.731+0000 7f9646a00640 1 -- 172.21.15.67:0/2998260653 shutdown_connections 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.731+0000 7f9646a00640 1 -- 172.21.15.67:0/2998260653 >> 172.21.15.67:0/2998260653 conn(0x7f964006b900 msgr2=0x7f964006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.732+0000 7f9646a00640 1 -- 172.21.15.67:0/2998260653 shutdown_connections 2024-05-30T16:58:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.732+0000 7f9646a00640 1 -- 172.21.15.67:0/2998260653 wait complete. 2024-05-30T16:58:21.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.733+0000 7f9646a00640 1 Processor -- start 2024-05-30T16:58:21.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.733+0000 7f9646a00640 1 -- start start 2024-05-30T16:58:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.734+0000 7f9646a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f964014ddc0 0x7f9640072270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.734+0000 7f9646a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964014e1a0 0x7f96400727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.734+0000 7f9646a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9640072ef0 0x7f964015dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.734+0000 7f9646a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f96400b48c0 con 0x7f964014ddc0 2024-05-30T16:58:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.734+0000 7f9646a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f964015d6c0 con 0x7f964014e1a0 2024-05-30T16:58:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.734+0000 7f9646a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f96400b4d50 con 0x7f9640072ef0 2024-05-30T16:58:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9645600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f964014ddc0 0x7f9640072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9645600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f964014ddc0 0x7f9640072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49232/0 (socket says 172.21.15.67:49232) 2024-05-30T16:58:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9645600640 1 -- 172.21.15.67:0/480059395 learned_addr learned my addr 172.21.15.67:0/480059395 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9644c00640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964014e1a0 0x7f96400727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9646000640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9640072ef0 0x7f964015dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9645600640 1 -- 172.21.15.67:0/480059395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9640072ef0 msgr2=0x7f964015dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9645600640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9640072ef0 0x7f964015dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9645600640 1 -- 172.21.15.67:0/480059395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964014e1a0 msgr2=0x7f96400727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9645600640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964014e1a0 0x7f96400727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.735+0000 7f9645600640 1 -- 172.21.15.67:0/480059395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96400ce740 con 0x7f964014ddc0 2024-05-30T16:58:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.736+0000 7f9645600640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f964014ddc0 0x7f9640072270 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f963400e550 tx=0x7f963400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.736+0000 7f963d600640 1 -- 172.21.15.67:0/480059395 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96340052e0 con 0x7f964014ddc0 2024-05-30T16:58:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.736+0000 7f963d600640 1 -- 172.21.15.67:0/480059395 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9634018070 con 0x7f964014ddc0 2024-05-30T16:58:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.736+0000 7f963d600640 1 -- 172.21.15.67:0/480059395 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634003810 con 0x7f964014ddc0 2024-05-30T16:58:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.736+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96400cea20 con 0x7f964014ddc0 2024-05-30T16:58:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.736+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96400cefe0 con 0x7f964014ddc0 2024-05-30T16:58:21.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.737+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96400689e0 con 0x7f964014ddc0 2024-05-30T16:58:21.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.738+0000 7f963d600640 1 -- 172.21.15.67:0/480059395 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f96340039b0 con 0x7f964014ddc0 2024-05-30T16:58:21.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.740+0000 7f963d600640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f962006da90 0x7f962006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:21.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.740+0000 7f963d600640 1 -- 172.21.15.67:0/480059395 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f963408a750 con 0x7f964014ddc0 2024-05-30T16:58:21.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.740+0000 7f9644c00640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f962006da90 0x7f962006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:21.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.741+0000 7f9644c00640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f962006da90 0x7f962006ff50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f96400cb2f0 tx=0x7f962c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:21.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.745+0000 7f963d600640 1 -- 172.21.15.67:0/480059395 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9634092070 con 0x7f964014ddc0 2024-05-30T16:58:21.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.952+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96400b3c40 con 0x7f964014ddc0 2024-05-30T16:58:21.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.953+0000 7f963d600640 1 -- 172.21.15.67:0/480059395 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f963405a2e0 con 0x7f964014ddc0 2024-05-30T16:58:21.953 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-05-30T16:58:21.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.955+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f962006da90 msgr2=0x7f962006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.955+0000 7f9646a00640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f962006da90 0x7f962006ff50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f96400cb2f0 tx=0x7f962c009040 comp rx=0 tx=0).stop 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.955+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f964014ddc0 msgr2=0x7f9640072270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.955+0000 7f9646a00640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f964014ddc0 0x7f9640072270 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f963400e550 tx=0x7f963400ea10 comp rx=0 tx=0).stop 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.956+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 shutdown_connections 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.956+0000 7f9646a00640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f962006da90 0x7f962006ff50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.956+0000 7f9646a00640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9640072ef0 0x7f964015dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.956+0000 7f9646a00640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f964014e1a0 0x7f96400727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.956+0000 7f9646a00640 1 --2- 172.21.15.67:0/480059395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f964014ddc0 0x7f9640072270 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.956+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 >> 172.21.15.67:0/480059395 conn(0x7f964006b900 msgr2=0x7f964015c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.956+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 shutdown_connections 2024-05-30T16:58:21.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:21.956+0000 7f9646a00640 1 -- 172.21.15.67:0/480059395 wait complete. 2024-05-30T16:58:21.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed' 2024-05-30T16:58:22.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.179+0000 7fd982000640 1 Processor -- start 2024-05-30T16:58:22.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.180+0000 7fd982000640 1 -- start start 2024-05-30T16:58:22.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.181+0000 7fd982000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c15e870 0x7fd97c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:22.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.181+0000 7fd982000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd97c0cfa40 0x7fd97c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:22.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.181+0000 7fd982000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd97c15ec50 con 0x7fd97c0d14d0 2024-05-30T16:58:22.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.181+0000 7fd982000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd97c0d2150 con 0x7fd97c0cfa40 2024-05-30T16:58:22.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.181+0000 7fd982000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c0cff80 con 0x7fd97c15e870 2024-05-30T16:58:22.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.182+0000 7fd980c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c15e870 0x7fd97c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:22.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.182+0000 7fd981600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd97c0cfa40 0x7fd97c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:22.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.182+0000 7fd97be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd97c0d14d0 0x7fd97c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:50182/0 (socket says 172.21.15.67:50182) 2024-05-30T16:58:22.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.182+0000 7fd97be00640 1 -- 172.21.15.67:0/9389220 learned_addr learned my addr 172.21.15.67:0/9389220 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:22.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.183+0000 7fd980c00640 1 -- 172.21.15.67:0/9389220 >> v2:172.21.15.169:3300/0 conn(0x7fd97c0cfa40 msgr2=0x7fd97c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:22.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.183+0000 7fd980c00640 1 --2- 172.21.15.67:0/9389220 >> v2:172.21.15.169:3300/0 conn(0x7fd97c0cfa40 0x7fd97c0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:22.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.183+0000 7fd980c00640 1 -- 172.21.15.67:0/9389220 >> v1:172.21.15.169:6789/0 conn(0x7fd97c0d14d0 legacy=0x7fd97c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:22.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.183+0000 7fd980c00640 1 -- 172.21.15.67:0/9389220 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c0d0260 con 0x7fd97c15e870 2024-05-30T16:58:22.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.184+0000 7fd980c00640 1 --2- 172.21.15.67:0/9389220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c15e870 0x7fd97c0d0f90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009d90 tx=0x7fd96c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=664968062174d4e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:22.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.185+0000 7fd97b400640 1 -- 172.21.15.67:0/9389220 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c007740 con 0x7fd97c15e870 2024-05-30T16:58:22.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.185+0000 7fd97b400640 1 -- 172.21.15.67:0/9389220 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd96c0078e0 con 0x7fd97c15e870 2024-05-30T16:58:22.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.185+0000 7fd97b400640 1 -- 172.21.15.67:0/9389220 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c007cb0 con 0x7fd97c15e870 2024-05-30T16:58:22.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.185+0000 7fd982000640 1 -- 172.21.15.67:0/9389220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c15e870 msgr2=0x7fd97c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:22.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.185+0000 7fd982000640 1 --2- 172.21.15.67:0/9389220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c15e870 0x7fd97c0d0f90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009d90 tx=0x7fd96c02a660 comp rx=0 tx=0).stop 2024-05-30T16:58:22.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.186+0000 7fd982000640 1 -- 172.21.15.67:0/9389220 shutdown_connections 2024-05-30T16:58:22.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.186+0000 7fd982000640 1 --2- 172.21.15.67:0/9389220 >> v2:172.21.15.169:3300/0 conn(0x7fd97c0cfa40 0x7fd97c0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:22.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.186+0000 7fd982000640 1 --2- 172.21.15.67:0/9389220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c15e870 0x7fd97c0d0f90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:22.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.186+0000 7fd982000640 1 -- 172.21.15.67:0/9389220 >> 172.21.15.67:0/9389220 conn(0x7fd97c08bfc0 msgr2=0x7fd97c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:22.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.186+0000 7fd982000640 1 -- 172.21.15.67:0/9389220 shutdown_connections 2024-05-30T16:58:22.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.186+0000 7fd982000640 1 -- 172.21.15.67:0/9389220 wait complete. 2024-05-30T16:58:22.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.188+0000 7fd982000640 1 Processor -- start 2024-05-30T16:58:22.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.188+0000 7fd982000640 1 -- start start 2024-05-30T16:58:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.189+0000 7fd982000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c0cfa40 0x7fd97c159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.189+0000 7fd982000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd97c0d14d0 0x7fd97c0b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.189+0000 7fd982000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97c15e870 0x7fd97c0b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.189+0000 7fd982000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c0d4fb0 con 0x7fd97c0d14d0 2024-05-30T16:58:22.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.189+0000 7fd982000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c0d0820 con 0x7fd97c15e870 2024-05-30T16:58:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.189+0000 7fd982000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd97c0d2b20 con 0x7fd97c0cfa40 2024-05-30T16:58:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd97be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd97c0d14d0 0x7fd97c0b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd97be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd97c0d14d0 0x7fd97c0b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49244/0 (socket says 172.21.15.67:49244) 2024-05-30T16:58:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd97be00640 1 -- 172.21.15.67:0/53226554 learned_addr learned my addr 172.21.15.67:0/53226554 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd980c00640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c0cfa40 0x7fd97c159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:22.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd981600640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97c15e870 0x7fd97c0b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:22.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd97be00640 1 -- 172.21.15.67:0/53226554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c0cfa40 msgr2=0x7fd97c159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:22.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd97be00640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c0cfa40 0x7fd97c159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:22.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd97be00640 1 -- 172.21.15.67:0/53226554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97c15e870 msgr2=0x7fd97c0b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:22.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd97be00640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97c15e870 0x7fd97c0b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:22.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd97be00640 1 -- 172.21.15.67:0/53226554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c0b72d0 con 0x7fd97c0d14d0 2024-05-30T16:58:22.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.190+0000 7fd981600640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97c15e870 0x7fd97c0b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:58:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.191+0000 7fd97be00640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd97c0d14d0 0x7fd97c0b8370 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c430 tx=0x7fd96400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.191+0000 7fd978c00640 1 -- 172.21.15.67:0/53226554 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964017070 con 0x7fd97c0d14d0 2024-05-30T16:58:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.191+0000 7fd978c00640 1 -- 172.21.15.67:0/53226554 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd964005d30 con 0x7fd97c0d14d0 2024-05-30T16:58:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.191+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c0b75b0 con 0x7fd97c0d14d0 2024-05-30T16:58:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.191+0000 7fd978c00640 1 -- 172.21.15.67:0/53226554 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640072f0 con 0x7fd97c0d14d0 2024-05-30T16:58:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.191+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c0b7b70 con 0x7fd97c0d14d0 2024-05-30T16:58:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.193+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c0689e0 con 0x7fd97c0d14d0 2024-05-30T16:58:22.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.194+0000 7fd978c00640 1 -- 172.21.15.67:0/53226554 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fd96401a070 con 0x7fd97c0d14d0 2024-05-30T16:58:22.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.195+0000 7fd978c00640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806da90 0x7fd95806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:22.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.195+0000 7fd978c00640 1 -- 172.21.15.67:0/53226554 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7fd964089e10 con 0x7fd97c0d14d0 2024-05-30T16:58:22.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.195+0000 7fd980c00640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806da90 0x7fd95806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:22.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.196+0000 7fd980c00640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806da90 0x7fd95806ff50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd96c034000 tx=0x7fd96c02f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:22.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.199+0000 7fd978c00640 1 -- 172.21.15.67:0/53226554 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9640599a0 con 0x7fd97c0d14d0 2024-05-30T16:58:22.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:22.475+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]} v 0) -- 0x7fd97c0ba500 con 0x7fd97c0d14d0 2024-05-30T16:58:23.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.031+0000 7fd978c00640 1 -- 172.21.15.67:0/53226554 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]}]=0 v607) ==== 156+0+0 (secure 0 0 0) 0x7fd9640592a0 con 0x7fd97c0d14d0 2024-05-30T16:58:23.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.034+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806da90 msgr2=0x7fd95806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:23.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.034+0000 7fd982000640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806da90 0x7fd95806ff50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd96c034000 tx=0x7fd96c02f000 comp rx=0 tx=0).stop 2024-05-30T16:58:23.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.034+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd97c0d14d0 msgr2=0x7fd97c0b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:23.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.034+0000 7fd982000640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd97c0d14d0 0x7fd97c0b8370 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c430 tx=0x7fd96400c8f0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.034+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 shutdown_connections 2024-05-30T16:58:23.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.035+0000 7fd982000640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806da90 0x7fd95806ff50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.035+0000 7fd982000640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97c15e870 0x7fd97c0b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.035+0000 7fd982000640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd97c0d14d0 0x7fd97c0b8370 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.035+0000 7fd982000640 1 --2- 172.21.15.67:0/53226554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97c0cfa40 0x7fd97c159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.035+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 >> 172.21.15.67:0/53226554 conn(0x7fd97c08bfc0 msgr2=0x7fd97c0d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:23.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.035+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 shutdown_connections 2024-05-30T16:58:23.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.035+0000 7fd982000640 1 -- 172.21.15.67:0/53226554 wait complete. 2024-05-30T16:58:23.057 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:58:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.264+0000 7fcfdb200640 1 Processor -- start 2024-05-30T16:58:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.265+0000 7fcfdb200640 1 -- start start 2024-05-30T16:58:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.266+0000 7fcfdb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcfdc15e870 0x7fcfdc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.266+0000 7fcfdb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcfdc0d14d0 0x7fcfdc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.266+0000 7fcfdb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfdc0cfce0 con 0x7fcfdc0cf870 2024-05-30T16:58:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.266+0000 7fcfdb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fcfdc0d2150 con 0x7fcfdc0d14d0 2024-05-30T16:58:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.266+0000 7fcfdb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcfdc15ec50 con 0x7fcfdc15e870 2024-05-30T16:58:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.266+0000 7fcfd3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcfdc0d14d0 0x7fcfdc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.267+0000 7fcfd3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcfdc0d14d0 0x7fcfdc0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49252/0 (socket says 172.21.15.67:49252) 2024-05-30T16:58:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.267+0000 7fcfd3e00640 1 -- 172.21.15.67:0/281653812 learned_addr learned my addr 172.21.15.67:0/281653812 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.267+0000 7fcfd3e00640 1 -- 172.21.15.67:0/281653812 >> v2:172.21.15.169:3300/0 conn(0x7fcfdc15e870 msgr2=0x7fcfdc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.267+0000 7fcfd3e00640 1 --2- 172.21.15.67:0/281653812 >> v2:172.21.15.169:3300/0 conn(0x7fcfdc15e870 0x7fcfdc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.267+0000 7fcfd3e00640 1 -- 172.21.15.67:0/281653812 >> v1:172.21.15.169:6789/0 conn(0x7fcfdc0cf870 legacy=0x7fcfdc162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.267+0000 7fcfd3e00640 1 -- 172.21.15.67:0/281653812 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc0cff80 con 0x7fcfdc0d14d0 2024-05-30T16:58:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.268+0000 7fcfd3e00640 1 --2- 172.21.15.67:0/281653812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc0d14d0 0x7fcfdc0cf330 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc08e720 tx=0x7fcfc402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234b3877b1a4208b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.269+0000 7fcfd9400640 1 -- 172.21.15.67:0/281653812 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004030 con 0x7fcfdc0d14d0 2024-05-30T16:58:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.269+0000 7fcfd9400640 1 -- 172.21.15.67:0/281653812 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc40041d0 con 0x7fcfdc0d14d0 2024-05-30T16:58:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.269+0000 7fcfd9400640 1 -- 172.21.15.67:0/281653812 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4003610 con 0x7fcfdc0d14d0 2024-05-30T16:58:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.269+0000 7fcfdb200640 1 -- 172.21.15.67:0/281653812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc0d14d0 msgr2=0x7fcfdc0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.269+0000 7fcfdb200640 1 --2- 172.21.15.67:0/281653812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc0d14d0 0x7fcfdc0cf330 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc08e720 tx=0x7fcfc402a210 comp rx=0 tx=0).stop 2024-05-30T16:58:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.270+0000 7fcfdb200640 1 -- 172.21.15.67:0/281653812 shutdown_connections 2024-05-30T16:58:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.270+0000 7fcfdb200640 1 --2- 172.21.15.67:0/281653812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc0d14d0 0x7fcfdc0cf330 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.270+0000 7fcfdb200640 1 --2- 172.21.15.67:0/281653812 >> v2:172.21.15.169:3300/0 conn(0x7fcfdc15e870 0x7fcfdc0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.270+0000 7fcfdb200640 1 -- 172.21.15.67:0/281653812 >> 172.21.15.67:0/281653812 conn(0x7fcfdc08bfc0 msgr2=0x7fcfdc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.270+0000 7fcfdb200640 1 -- 172.21.15.67:0/281653812 shutdown_connections 2024-05-30T16:58:23.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.270+0000 7fcfdb200640 1 -- 172.21.15.67:0/281653812 wait complete. 2024-05-30T16:58:23.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.272+0000 7fcfdb200640 1 Processor -- start 2024-05-30T16:58:23.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.272+0000 7fcfdb200640 1 -- start start 2024-05-30T16:58:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfdb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfdc0cf870 0x7fcfdc159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfdb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfdc0d14d0 0x7fcfdc0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfdb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc15e870 0x7fcfdc0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfdb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc0d0820 con 0x7fcfdc15e870 2024-05-30T16:58:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfdb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc0d0540 con 0x7fcfdc0d14d0 2024-05-30T16:58:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfdb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcfdc0d4fb0 con 0x7fcfdc0cf870 2024-05-30T16:58:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfda800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc15e870 0x7fcfdc0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfda800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc15e870 0x7fcfdc0c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49268/0 (socket says 172.21.15.67:49268) 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfda800640 1 -- 172.21.15.67:0/3308950455 learned_addr learned my addr 172.21.15.67:0/3308950455 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.273+0000 7fcfd9e00640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfdc0cf870 0x7fcfdc159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfda800640 1 -- 172.21.15.67:0/3308950455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfdc0cf870 msgr2=0x7fcfdc159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfd3e00640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfdc0d14d0 0x7fcfdc0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfda800640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfdc0cf870 0x7fcfdc159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfda800640 1 -- 172.21.15.67:0/3308950455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfdc0d14d0 msgr2=0x7fcfdc0b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfda800640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfdc0d14d0 0x7fcfdc0b71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfda800640 1 -- 172.21.15.67:0/3308950455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc0b7c60 con 0x7fcfdc15e870 2024-05-30T16:58:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfd3e00640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfdc0d14d0 0x7fcfdc0b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:23.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfda800640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc15e870 0x7fcfdc0c98b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b970 tx=0x7fcfd400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:23.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfd2000640 1 -- 172.21.15.67:0/3308950455 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4019070 con 0x7fcfdc15e870 2024-05-30T16:58:23.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfd2000640 1 -- 172.21.15.67:0/3308950455 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd400cb40 con 0x7fcfdc15e870 2024-05-30T16:58:23.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfd2000640 1 -- 172.21.15.67:0/3308950455 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4014440 con 0x7fcfdc15e870 2024-05-30T16:58:23.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc0b7f40 con 0x7fcfdc15e870 2024-05-30T16:58:23.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.274+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc0b8500 con 0x7fcfdc15e870 2024-05-30T16:58:23.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.275+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc0689e0 con 0x7fcfdc15e870 2024-05-30T16:58:23.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.276+0000 7fcfd2000640 1 -- 172.21.15.67:0/3308950455 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fcfd401c070 con 0x7fcfdc15e870 2024-05-30T16:58:23.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.277+0000 7fcfd2000640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfb806da60 0x7fcfb806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:23.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.277+0000 7fcfd2000640 1 -- 172.21.15.67:0/3308950455 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7fcfd408a740 con 0x7fcfdc15e870 2024-05-30T16:58:23.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.278+0000 7fcfd9e00640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfb806da60 0x7fcfb806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:23.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.278+0000 7fcfd9e00640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfb806da60 0x7fcfb806ff20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009890 tx=0x7fcfcc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:23.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.282+0000 7fcfd2000640 1 -- 172.21.15.67:0/3308950455 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfd405a270 con 0x7fcfdc15e870 2024-05-30T16:58:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.485+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcfdc0b8c00 con 0x7fcfdc15e870 2024-05-30T16:58:23.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.491+0000 7fcfd2000640 1 -- 172.21.15.67:0/3308950455 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fcfd4059b70 con 0x7fcfdc15e870 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfb806da60 msgr2=0x7fcfb806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfb806da60 0x7fcfb806ff20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009890 tx=0x7fcfcc008040 comp rx=0 tx=0).stop 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc15e870 msgr2=0x7fcfdc0c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc15e870 0x7fcfdc0c98b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b970 tx=0x7fcfd400be30 comp rx=0 tx=0).stop 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 shutdown_connections 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcfb806da60 0x7fcfb806ff20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfdc15e870 0x7fcfdc0c98b0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfdc0d14d0 0x7fcfdc0b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 --2- 172.21.15.67:0/3308950455 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcfdc0cf870 0x7fcfdc159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 >> 172.21.15.67:0/3308950455 conn(0x7fcfdc08bfc0 msgr2=0x7fcfdc0b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 shutdown_connections 2024-05-30T16:58:23.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:23.494+0000 7fcfdb200640 1 -- 172.21.15.67:0/3308950455 wait complete. 2024-05-30T16:58:23.513 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T16:58:23.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:58:23.626 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T16:58:23.739 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T16:58:23.740 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:58:23.740 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:23.801 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:58:23.801 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:58:23.801 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:23.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:58:23.857 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:58:23.857 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:23.917 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T16:58:23.917 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T16:58:23.917 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:23.975 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T16:58:23.975 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-30T16:58:23.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:58:24.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.194+0000 7f63e9400640 1 Processor -- start 2024-05-30T16:58:24.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.195+0000 7f63e9400640 1 -- start start 2024-05-30T16:58:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.196+0000 7f63e9400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f63e40cfa40 0x7f63e4169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.196+0000 7f63e9400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e415ec50 con 0x7f63e40d16a0 2024-05-30T16:58:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.196+0000 7f63e9400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e40cff80 con 0x7f63e415e870 2024-05-30T16:58:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.196+0000 7f63e9400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f63e40d2150 con 0x7f63e40cfa40 2024-05-30T16:58:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.197+0000 7f63e3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f63e40d16a0 0x7f63e40cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56310/0 (socket says 172.21.15.67:56310) 2024-05-30T16:58:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.197+0000 7f63e3400640 1 -- 172.21.15.67:0/416492514 learned_addr learned my addr 172.21.15.67:0/416492514 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.197+0000 7f63e8a00640 1 --2- 172.21.15.67:0/416492514 >> v2:172.21.15.67:3300/0 conn(0x7f63e40cfa40 0x7f63e4169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.197+0000 7f63e2a00640 1 -- 172.21.15.67:0/416492514 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1835767167 0 0) 0x7f63e415ec50 con 0x7f63e40d16a0 2024-05-30T16:58:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.197+0000 7f63e2a00640 1 -- 172.21.15.67:0/416492514 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63c8003520 con 0x7f63e40d16a0 2024-05-30T16:58:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.197+0000 7f63e8a00640 1 -- 172.21.15.67:0/416492514 >> v1:172.21.15.169:6789/0 conn(0x7f63e415e870 legacy=0x7f63e40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.197+0000 7f63e8a00640 1 -- 172.21.15.67:0/416492514 >> v1:172.21.15.67:6789/0 conn(0x7f63e40d16a0 legacy=0x7f63e40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.197+0000 7f63e8a00640 1 -- 172.21.15.67:0/416492514 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e40d0260 con 0x7f63e40cfa40 2024-05-30T16:58:24.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.199+0000 7f63e8a00640 1 --2- 172.21.15.67:0/416492514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e40cfa40 0x7f63e4169690 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0097a0 tx=0x7f63dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379f0a2f762ffc95 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:24.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.199+0000 7f63e2a00640 1 -- 172.21.15.67:0/416492514 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc037070 con 0x7f63e40cfa40 2024-05-30T16:58:24.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.199+0000 7f63e2a00640 1 -- 172.21.15.67:0/416492514 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63dc032440 con 0x7f63e40cfa40 2024-05-30T16:58:24.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.199+0000 7f63e2a00640 1 -- 172.21.15.67:0/416492514 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0327f0 con 0x7f63e40cfa40 2024-05-30T16:58:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.200+0000 7f63e9400640 1 -- 172.21.15.67:0/416492514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e40cfa40 msgr2=0x7f63e4169690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.200+0000 7f63e9400640 1 --2- 172.21.15.67:0/416492514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e40cfa40 0x7f63e4169690 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0097a0 tx=0x7f63dc02a230 comp rx=0 tx=0).stop 2024-05-30T16:58:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.200+0000 7f63e9400640 1 -- 172.21.15.67:0/416492514 shutdown_connections 2024-05-30T16:58:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.200+0000 7f63e9400640 1 --2- 172.21.15.67:0/416492514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e40cfa40 0x7f63e4169690 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.200+0000 7f63e9400640 1 -- 172.21.15.67:0/416492514 >> 172.21.15.67:0/416492514 conn(0x7f63e408bfc0 msgr2=0x7f63e40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.200+0000 7f63e9400640 1 -- 172.21.15.67:0/416492514 shutdown_connections 2024-05-30T16:58:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.200+0000 7f63e9400640 1 -- 172.21.15.67:0/416492514 wait complete. 2024-05-30T16:58:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.203+0000 7f63e9400640 1 Processor -- start 2024-05-30T16:58:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.203+0000 7f63e9400640 1 -- start start 2024-05-30T16:58:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.204+0000 7f63e9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e40cfa40 0x7f63e415c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.204+0000 7f63e9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e40d16a0 0x7f63e41f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.204+0000 7f63e9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e415e870 0x7f63e41fc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.204+0000 7f63e9400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40d4fb0 con 0x7f63e415e870 2024-05-30T16:58:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.204+0000 7f63e9400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40d0820 con 0x7f63e40cfa40 2024-05-30T16:58:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.204+0000 7f63e9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f63e40d35f0 con 0x7f63e40d16a0 2024-05-30T16:58:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e415e870 0x7f63e41fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e415e870 0x7f63e41fc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49274/0 (socket says 172.21.15.67:49274) 2024-05-30T16:58:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e8a00640 1 -- 172.21.15.67:0/3600763499 learned_addr learned my addr 172.21.15.67:0/3600763499 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e3400640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e40d16a0 0x7f63e41f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e3e00640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e40cfa40 0x7f63e415c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e8a00640 1 -- 172.21.15.67:0/3600763499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e40d16a0 msgr2=0x7f63e41f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e8a00640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e40d16a0 0x7f63e41f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e8a00640 1 -- 172.21.15.67:0/3600763499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e40cfa40 msgr2=0x7f63e415c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e8a00640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e40cfa40 0x7f63e415c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e8a00640 1 -- 172.21.15.67:0/3600763499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e41f8f60 con 0x7f63e415e870 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.205+0000 7f63e3e00640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e40cfa40 0x7f63e415c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.206+0000 7f63e8a00640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e415e870 0x7f63e41fc410 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0097a0 tx=0x7f63dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.206+0000 7f63dbe00640 1 -- 172.21.15.67:0/3600763499 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc037050 con 0x7f63e415e870 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.206+0000 7f63dbe00640 1 -- 172.21.15.67:0/3600763499 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63dc043070 con 0x7f63e415e870 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.206+0000 7f63dbe00640 1 -- 172.21.15.67:0/3600763499 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc032810 con 0x7f63e415e870 2024-05-30T16:58:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.206+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e41f9240 con 0x7f63e415e870 2024-05-30T16:58:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.206+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e41f8c70 con 0x7f63e415e870 2024-05-30T16:58:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.207+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e40d4bb0 con 0x7f63e415e870 2024-05-30T16:58:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.208+0000 7f63dbe00640 1 -- 172.21.15.67:0/3600763499 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f63dc0329b0 con 0x7f63e415e870 2024-05-30T16:58:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.210+0000 7f63dbe00640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f63c006da60 0x7f63c006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.210+0000 7f63dbe00640 1 -- 172.21.15.67:0/3600763499 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f63dc0ab530 con 0x7f63e415e870 2024-05-30T16:58:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.210+0000 7f63e3e00640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f63c006da60 0x7f63c006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.211+0000 7f63e3e00640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f63c006da60 0x7f63c006ff20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f63cc000f60 tx=0x7f63cc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.214+0000 7f63dbe00640 1 -- 172.21.15.67:0/3600763499 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63e40d4bb0 con 0x7f63e415e870 2024-05-30T16:58:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.463+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e40689e0 con 0x7f63e415e870 2024-05-30T16:58:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.464+0000 7f63dbe00640 1 -- 172.21.15.67:0/3600763499 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+3875 (secure 0 0 0) 0x7f63dc07b0c0 con 0x7f63e415e870 2024-05-30T16:58:24.465 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:24.465 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":48,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5171,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1912096869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1912096869},{"type":"v1","addr":"172.21.15.169:6835","nonce":1912096869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5608,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3963396329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3963396329},{"type":"v1","addr":"172.21.15.67:6835","nonce":3963396329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2169821644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2169821644},{"type":"v1","addr":"172.21.15.169:6834","nonce":2169821644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2593339581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2593339581},{"type":"v1","addr":"172.21.15.67:6837","nonce":2593339581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[]} 2024-05-30T16:58:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 48 2024-05-30T16:58:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f63c006da60 msgr2=0x7f63c006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f63c006da60 0x7f63c006ff20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f63cc000f60 tx=0x7f63cc009040 comp rx=0 tx=0).stop 2024-05-30T16:58:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e415e870 msgr2=0x7f63e41fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e415e870 0x7f63e41fc410 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0097a0 tx=0x7f63dc0046c0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 shutdown_connections 2024-05-30T16:58:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f63c006da60 0x7f63c006ff20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e415e870 0x7f63e41fc410 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63e40d16a0 0x7f63e41f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 --2- 172.21.15.67:0/3600763499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63e40cfa40 0x7f63e415c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 >> 172.21.15.67:0/3600763499 conn(0x7f63e408bfc0 msgr2=0x7f63e40b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 shutdown_connections 2024-05-30T16:58:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.467+0000 7f63e9400640 1 -- 172.21.15.67:0/3600763499 wait complete. 2024-05-30T16:58:24.488 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-30T16:58:24.488 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-30T16:58:24.489 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-30T16:58:24.489 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:58:24.489 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-30T16:58:24.489 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-30T16:58:24.489 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-30T16:58:24.489 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:58:24.489 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-30T16:58:24.490 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-30T16:58:24.490 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-30T16:58:24.490 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:58:24.490 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-30T16:58:24.490 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-30T16:58:24.490 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-30T16:58:24.490 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T16:58:24.506 INFO:tasks.ceph.mds.c:Stopped 2024-05-30T16:58:24.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-30T16:58:24.507 INFO:tasks.ceph.mds.a:Stopped 2024-05-30T16:58:24.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-30T16:58:24.515 INFO:tasks.ceph.mds.b:Stopped 2024-05-30T16:58:24.515 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-30T16:58:24.516 INFO:tasks.ceph.mds.d:Stopped 2024-05-30T16:58:24.516 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-30T16:58:24.553 INFO:tasks.ceph.mds.c:Started 2024-05-30T16:58:24.553 DEBUG:teuthology.parallel:result is None 2024-05-30T16:58:24.559 INFO:tasks.ceph.mds.a:Started 2024-05-30T16:58:24.559 DEBUG:teuthology.parallel:result is None 2024-05-30T16:58:24.563 INFO:tasks.ceph.mds.b:Started 2024-05-30T16:58:24.563 DEBUG:teuthology.parallel:result is None 2024-05-30T16:58:24.568 INFO:tasks.ceph.mds.d:Started 2024-05-30T16:58:24.569 DEBUG:teuthology.parallel:result is None 2024-05-30T16:58:24.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-30T16:58:24.709 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T16:58:24.709 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T16:58:24.716 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T16:58:24.717 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T16:58:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.753+0000 7fc840800640 1 Processor -- start 2024-05-30T16:58:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.753+0000 7fc840800640 1 -- start start 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.754+0000 7fc840800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8380d04a0 0x7fc8380d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.754+0000 7fc840800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8380b4350 con 0x7fc83806f6e0 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.754+0000 7fc840800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8380b32d0 con 0x7fc838162e70 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.754+0000 7fc840800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc8380b3ec0 con 0x7fc8380d04a0 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.754+0000 7fc83d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc83806f6e0 0x7fc8380cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56326/0 (socket says 172.21.15.67:56326) 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.754+0000 7fc83d400640 1 -- 172.21.15.67:0/3754994673 learned_addr learned my addr 172.21.15.67:0/3754994673 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.755+0000 7fc83de00640 1 --2- 172.21.15.67:0/3754994673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8380d04a0 0x7fc8380d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.755+0000 7fc83ca00640 1 -- 172.21.15.67:0/3754994673 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740517367 0 0) 0x7fc8380b4350 con 0x7fc83806f6e0 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.755+0000 7fc83ca00640 1 -- 172.21.15.67:0/3754994673 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc820003520 con 0x7fc83806f6e0 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.755+0000 7fc83ca00640 1 -- 172.21.15.67:0/3754994673 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1079072763 0 0) 0x7fc8380b32d0 con 0x7fc838162e70 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.755+0000 7fc83ca00640 1 -- 172.21.15.67:0/3754994673 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8380b4350 con 0x7fc838162e70 2024-05-30T16:58:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.755+0000 7fc83de00640 1 -- 172.21.15.67:0/3754994673 >> v1:172.21.15.169:6789/0 conn(0x7fc838162e70 legacy=0x7fc8380ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.755+0000 7fc83de00640 1 -- 172.21.15.67:0/3754994673 >> v1:172.21.15.67:6789/0 conn(0x7fc83806f6e0 legacy=0x7fc8380cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.755+0000 7fc83de00640 1 -- 172.21.15.67:0/3754994673 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8380b3590 con 0x7fc8380d04a0 2024-05-30T16:58:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.757+0000 7fc83de00640 1 --2- 172.21.15.67:0/3754994673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8380d04a0 0x7fc8380d0880 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0097d0 tx=0x7fc82c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3504b01cbc7cd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.757+0000 7fc83ca00640 1 -- 172.21.15.67:0/3754994673 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c037070 con 0x7fc8380d04a0 2024-05-30T16:58:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.757+0000 7fc83ca00640 1 -- 172.21.15.67:0/3754994673 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c032440 con 0x7fc8380d04a0 2024-05-30T16:58:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.757+0000 7fc83ca00640 1 -- 172.21.15.67:0/3754994673 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c032810 con 0x7fc8380d04a0 2024-05-30T16:58:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.758+0000 7fc840800640 1 -- 172.21.15.67:0/3754994673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8380d04a0 msgr2=0x7fc8380d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.758+0000 7fc840800640 1 --2- 172.21.15.67:0/3754994673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8380d04a0 0x7fc8380d0880 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0097d0 tx=0x7fc82c02a210 comp rx=0 tx=0).stop 2024-05-30T16:58:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.758+0000 7fc840800640 1 -- 172.21.15.67:0/3754994673 shutdown_connections 2024-05-30T16:58:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.758+0000 7fc840800640 1 --2- 172.21.15.67:0/3754994673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8380d04a0 0x7fc8380d0880 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.758+0000 7fc840800640 1 -- 172.21.15.67:0/3754994673 >> 172.21.15.67:0/3754994673 conn(0x7fc83806bae0 msgr2=0x7fc83806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:24.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.758+0000 7fc840800640 1 -- 172.21.15.67:0/3754994673 shutdown_connections 2024-05-30T16:58:24.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.758+0000 7fc840800640 1 -- 172.21.15.67:0/3754994673 wait complete. 2024-05-30T16:58:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.759+0000 7fc840800640 1 Processor -- start 2024-05-30T16:58:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.760+0000 7fc840800640 1 -- start start 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc840800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83806f6e0 0x7fc838163250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc840800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8380d04a0 0x7fc8380998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc840800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc838162e70 0x7fc838164b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc840800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc8380d0dc0 con 0x7fc8380d04a0 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc840800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8380b3b50 con 0x7fc83806f6e0 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc840800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc8380ce940 con 0x7fc838162e70 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc83d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8380d04a0 0x7fc8380998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc83d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8380d04a0 0x7fc8380998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49294/0 (socket says 172.21.15.67:49294) 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc83d400640 1 -- 172.21.15.67:0/1789541123 learned_addr learned my addr 172.21.15.67:0/1789541123 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:24.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc83de00640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83806f6e0 0x7fc838163250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc83e800640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc838162e70 0x7fc838164b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc83d400640 1 -- 172.21.15.67:0/1789541123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc838162e70 msgr2=0x7fc838164b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc83d400640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc838162e70 0x7fc838164b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc83d400640 1 -- 172.21.15.67:0/1789541123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83806f6e0 msgr2=0x7fc838163250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:24.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.761+0000 7fc83d400640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83806f6e0 0x7fc838163250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:24.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.762+0000 7fc83d400640 1 -- 172.21.15.67:0/1789541123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc838163a80 con 0x7fc8380d04a0 2024-05-30T16:58:24.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.762+0000 7fc83d400640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8380d04a0 0x7fc8380998f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc82800c4d0 tx=0x7fc82800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:24.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.762+0000 7fc832000640 1 -- 172.21.15.67:0/1789541123 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828017070 con 0x7fc8380d04a0 2024-05-30T16:58:24.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.762+0000 7fc832000640 1 -- 172.21.15.67:0/1789541123 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc828005ce0 con 0x7fc8380d04a0 2024-05-30T16:58:24.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.762+0000 7fc832000640 1 -- 172.21.15.67:0/1789541123 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828007270 con 0x7fc8380d04a0 2024-05-30T16:58:24.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.762+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc838163d60 con 0x7fc8380d04a0 2024-05-30T16:58:24.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.762+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc838164320 con 0x7fc8380d04a0 2024-05-30T16:58:24.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.763+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8380689e0 con 0x7fc8380d04a0 2024-05-30T16:58:24.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.765+0000 7fc832000640 1 -- 172.21.15.67:0/1789541123 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fc82801a070 con 0x7fc8380d04a0 2024-05-30T16:58:24.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.766+0000 7fc832000640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc81c06da60 0x7fc81c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:24.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.766+0000 7fc832000640 1 -- 172.21.15.67:0/1789541123 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7fc828089d70 con 0x7fc8380d04a0 2024-05-30T16:58:24.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.766+0000 7fc83de00640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc81c06da60 0x7fc81c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:24.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.767+0000 7fc83de00640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc81c06da60 0x7fc81c06ff20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc82c034000 tx=0x7fc82c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:24.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.770+0000 7fc832000640 1 -- 172.21.15.67:0/1789541123 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc828059250 con 0x7fc8380d04a0 2024-05-30T16:58:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:24.985+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc8380b0450 con 0x7fc8380d04a0 2024-05-30T16:58:25.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.062+0000 7fc832000640 1 -- 172.21.15.67:0/1789541123 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) ==== 97+0+0 (secure 0 0 0) 0x7fc82805d550 con 0x7fc8380d04a0 2024-05-30T16:58:25.063 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-05-30T16:58:25.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc81c06da60 msgr2=0x7fc81c06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc81c06da60 0x7fc81c06ff20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc82c034000 tx=0x7fc82c02e000 comp rx=0 tx=0).stop 2024-05-30T16:58:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8380d04a0 msgr2=0x7fc8380998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8380d04a0 0x7fc8380998f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc82800c4d0 tx=0x7fc82800c990 comp rx=0 tx=0).stop 2024-05-30T16:58:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 shutdown_connections 2024-05-30T16:58:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc81c06da60 0x7fc81c06ff20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc838162e70 0x7fc838164b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8380d04a0 0x7fc8380998f0 secure :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc82800c4d0 tx=0x7fc82800c990 comp rx=0 tx=0).stop 2024-05-30T16:58:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 --2- 172.21.15.67:0/1789541123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83806f6e0 0x7fc838163250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 >> 172.21.15.67:0/1789541123 conn(0x7fc83806bae0 msgr2=0x7fc83806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:25.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.066+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 shutdown_connections 2024-05-30T16:58:25.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.067+0000 7fc840800640 1 -- 172.21.15.67:0/1789541123 wait complete. 2024-05-30T16:58:25.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-30T16:58:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.277+0000 7fda35c00640 1 Processor -- start 2024-05-30T16:58:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.278+0000 7fda35c00640 1 -- start start 2024-05-30T16:58:25.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.280+0000 7fda35c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fda30156b30 0x7fda300af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.280+0000 7fda35c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda3006ee40 0x7fda300ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.280+0000 7fda35c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda30156f10 con 0x7fda3006f2f0 2024-05-30T16:58:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.280+0000 7fda35c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fda3006dda0 con 0x7fda30156b30 2024-05-30T16:58:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.280+0000 7fda35c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fda3006e080 con 0x7fda3006ee40 2024-05-30T16:58:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.280+0000 7fda2fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fda30156b30 0x7fda300af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.280+0000 7fda2fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fda30156b30 0x7fda300af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49304/0 (socket says 172.21.15.67:49304) 2024-05-30T16:58:25.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.280+0000 7fda2fe00640 1 -- 172.21.15.67:0/1891156171 learned_addr learned my addr 172.21.15.67:0/1891156171 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.281+0000 7fda2fe00640 1 -- 172.21.15.67:0/1891156171 >> v2:172.21.15.169:3300/0 conn(0x7fda3006ee40 msgr2=0x7fda300ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.281+0000 7fda2fe00640 1 --2- 172.21.15.67:0/1891156171 >> v2:172.21.15.169:3300/0 conn(0x7fda3006ee40 0x7fda300ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.281+0000 7fda2fe00640 1 -- 172.21.15.67:0/1891156171 >> v1:172.21.15.169:6789/0 conn(0x7fda3006f2f0 legacy=0x7fda300b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.281+0000 7fda2fe00640 1 -- 172.21.15.67:0/1891156171 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3006e360 con 0x7fda30156b30 2024-05-30T16:58:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.282+0000 7fda2fe00640 1 --2- 172.21.15.67:0/1891156171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 0x7fda300af6e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fda3006fdf0 tx=0x7fda2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8eb37843169529d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.282+0000 7fda2f400640 1 -- 172.21.15.67:0/1891156171 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20037070 con 0x7fda30156b30 2024-05-30T16:58:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.282+0000 7fda2f400640 1 -- 172.21.15.67:0/1891156171 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda20032440 con 0x7fda30156b30 2024-05-30T16:58:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.282+0000 7fda2f400640 1 -- 172.21.15.67:0/1891156171 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20032810 con 0x7fda30156b30 2024-05-30T16:58:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.283+0000 7fda35c00640 1 -- 172.21.15.67:0/1891156171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 msgr2=0x7fda300af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.283+0000 7fda35c00640 1 --2- 172.21.15.67:0/1891156171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 0x7fda300af6e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fda3006fdf0 tx=0x7fda2002a1a0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.283+0000 7fda35c00640 1 -- 172.21.15.67:0/1891156171 shutdown_connections 2024-05-30T16:58:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.283+0000 7fda35c00640 1 --2- 172.21.15.67:0/1891156171 >> v2:172.21.15.169:3300/0 conn(0x7fda3006ee40 0x7fda300ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.283+0000 7fda35c00640 1 --2- 172.21.15.67:0/1891156171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 0x7fda300af6e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.283+0000 7fda35c00640 1 -- 172.21.15.67:0/1891156171 >> 172.21.15.67:0/1891156171 conn(0x7fda300b1fc0 msgr2=0x7fda300b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.283+0000 7fda35c00640 1 -- 172.21.15.67:0/1891156171 shutdown_connections 2024-05-30T16:58:25.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.283+0000 7fda35c00640 1 -- 172.21.15.67:0/1891156171 wait complete. 2024-05-30T16:58:25.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.285+0000 7fda35c00640 1 Processor -- start 2024-05-30T16:58:25.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.285+0000 7fda35c00640 1 -- start start 2024-05-30T16:58:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.286+0000 7fda35c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3006ee40 0x7fda301f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.286+0000 7fda35c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3006f2f0 0x7fda3014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.286+0000 7fda35c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 0x7fda3014b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.286+0000 7fda35c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fda300afc20 con 0x7fda30156b30 2024-05-30T16:58:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.286+0000 7fda35c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda300b2c30 con 0x7fda3006f2f0 2024-05-30T16:58:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.286+0000 7fda35c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda30158be0 con 0x7fda3006ee40 2024-05-30T16:58:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.286+0000 7fda35200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 0x7fda3014b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.286+0000 7fda35200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 0x7fda3014b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49312/0 (socket says 172.21.15.67:49312) 2024-05-30T16:58:25.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.286+0000 7fda35200640 1 -- 172.21.15.67:0/3431510958 learned_addr learned my addr 172.21.15.67:0/3431510958 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda2fe00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3006ee40 0x7fda301f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda27e00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3006f2f0 0x7fda3014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda35200640 1 -- 172.21.15.67:0/3431510958 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3006ee40 msgr2=0x7fda301f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda35200640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3006ee40 0x7fda301f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda35200640 1 -- 172.21.15.67:0/3431510958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3006f2f0 msgr2=0x7fda3014ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda35200640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3006f2f0 0x7fda3014ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda35200640 1 -- 172.21.15.67:0/3431510958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda30073010 con 0x7fda30156b30 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda35200640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 0x7fda3014b140 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fda2800ba10 tx=0x7fda2800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda2cc00640 1 -- 172.21.15.67:0/3431510958 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28019070 con 0x7fda30156b30 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda2cc00640 1 -- 172.21.15.67:0/3431510958 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2800cb20 con 0x7fda30156b30 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda2cc00640 1 -- 172.21.15.67:0/3431510958 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28014440 con 0x7fda30156b30 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.287+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda300b1040 con 0x7fda30156b30 2024-05-30T16:58:25.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.288+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda30073590 con 0x7fda30156b30 2024-05-30T16:58:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.289+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda300689e0 con 0x7fda30156b30 2024-05-30T16:58:25.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.290+0000 7fda2cc00640 1 -- 172.21.15.67:0/3431510958 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fda2801c070 con 0x7fda30156b30 2024-05-30T16:58:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.291+0000 7fda2cc00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda0c06da60 0x7fda0c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.291+0000 7fda2cc00640 1 -- 172.21.15.67:0/3431510958 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 4913+0+0 (secure 0 0 0) 0x7fda2808a730 con 0x7fda30156b30 2024-05-30T16:58:25.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.291+0000 7fda2fe00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda0c06da60 0x7fda0c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.292+0000 7fda2fe00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda0c06da60 0x7fda0c06ff20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda20034000 tx=0x7fda2002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.295+0000 7fda2cc00640 1 -- 172.21.15.67:0/3431510958 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda2805a260 con 0x7fda30156b30 2024-05-30T16:58:25.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.576+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fda3006f7f0 con 0x7fda30156b30 2024-05-30T16:58:25.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.578+0000 7fda2cc00640 1 -- 172.21.15.67:0/3431510958 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fda28059b60 con 0x7fda30156b30 2024-05-30T16:58:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:25.579 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:58:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-05-30T16:58:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-05-30T16:58:25.579 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA==", 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw==", 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:58:25.580 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw==", 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.581 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg==", 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng==", 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg==", 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:58:25.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w==", 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ==", 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A==", 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ==", 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w==", 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A==", 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T16:58:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg==", 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ==", 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.2", 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ==", 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:58:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:58:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:58:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:58:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.3", 2024-05-30T16:58:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A==", 2024-05-30T16:58:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCArFhmF5mhNhAA4H3cgcH6YdD53fANDW4seA==", 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T16:58:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ==", 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw==", 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T16:58:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T16:58:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T16:58:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T16:58:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-05-30T16:58:25.593 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda0c06da60 msgr2=0x7fda0c06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda0c06da60 0x7fda0c06ff20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda20034000 tx=0x7fda2002e000 comp rx=0 tx=0).stop 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 msgr2=0x7fda3014b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 0x7fda3014b140 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fda2800ba10 tx=0x7fda2800bed0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 shutdown_connections 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fda0c06da60 0x7fda0c06ff20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fda30156b30 0x7fda3014b140 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3006f2f0 0x7fda3014ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 --2- 172.21.15.67:0/3431510958 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3006ee40 0x7fda301f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 >> 172.21.15.67:0/3431510958 conn(0x7fda300b1fc0 msgr2=0x7fda3015b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 shutdown_connections 2024-05-30T16:58:25.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.581+0000 7fda35c00640 1 -- 172.21.15.67:0/3431510958 wait complete. 2024-05-30T16:58:25.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-30T16:58:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.788+0000 7fcb77e00640 1 Processor -- start 2024-05-30T16:58:25.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.789+0000 7fcb77e00640 1 -- start start 2024-05-30T16:58:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.790+0000 7fcb77e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb7815e870 0x7fcb780d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.790+0000 7fcb77e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcb780d14d0 0x7fcb780cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.790+0000 7fcb77e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb780cf870 0x7fcb78162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.790+0000 7fcb77e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fcb780d2150 con 0x7fcb780d14d0 2024-05-30T16:58:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.790+0000 7fcb77e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcb780cfd00 con 0x7fcb7815e870 2024-05-30T16:58:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.790+0000 7fcb77e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb7815ec50 con 0x7fcb780cf870 2024-05-30T16:58:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcb780d14d0 0x7fcb780cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcb780d14d0 0x7fcb780cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49322/0 (socket says 172.21.15.67:49322) 2024-05-30T16:58:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76000640 1 -- 172.21.15.67:0/4099922569 learned_addr learned my addr 172.21.15.67:0/4099922569 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb77400640 1 --2- 172.21.15.67:0/4099922569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb780cf870 0x7fcb78162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76a00640 1 --2- 172.21.15.67:0/4099922569 >> v2:172.21.15.169:3300/0 conn(0x7fcb7815e870 0x7fcb780d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76000640 1 -- 172.21.15.67:0/4099922569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb780cf870 msgr2=0x7fcb78162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:25.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76000640 1 --2- 172.21.15.67:0/4099922569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb780cf870 0x7fcb78162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76000640 1 -- 172.21.15.67:0/4099922569 >> v2:172.21.15.169:3300/0 conn(0x7fcb7815e870 msgr2=0x7fcb780d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:25.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76000640 1 --2- 172.21.15.67:0/4099922569 >> v2:172.21.15.169:3300/0 conn(0x7fcb7815e870 0x7fcb780d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76000640 1 -- 172.21.15.67:0/4099922569 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb780cff80 con 0x7fcb780d14d0 2024-05-30T16:58:25.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb77400640 1 --2- 172.21.15.67:0/4099922569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb780cf870 0x7fcb78162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:25.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.791+0000 7fcb76a00640 1 --2- 172.21.15.67:0/4099922569 >> v2:172.21.15.169:3300/0 conn(0x7fcb7815e870 0x7fcb780d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:25.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.793+0000 7fcb76000640 1 --2- 172.21.15.67:0/4099922569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 0x7fcb780cf330 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fcb7808e720 tx=0x7fcb6002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bf9f7a6202918e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:25.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.793+0000 7fcb75600640 1 -- 172.21.15.67:0/4099922569 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60036070 con 0x7fcb780d14d0 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.793+0000 7fcb75600640 1 -- 172.21.15.67:0/4099922569 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb60031600 con 0x7fcb780d14d0 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.793+0000 7fcb75600640 1 -- 172.21.15.67:0/4099922569 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb600319d0 con 0x7fcb780d14d0 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.794+0000 7fcb77e00640 1 -- 172.21.15.67:0/4099922569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 msgr2=0x7fcb780cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.794+0000 7fcb77e00640 1 --2- 172.21.15.67:0/4099922569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 0x7fcb780cf330 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fcb7808e720 tx=0x7fcb6002a6f0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.794+0000 7fcb77e00640 1 -- 172.21.15.67:0/4099922569 shutdown_connections 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.794+0000 7fcb77e00640 1 --2- 172.21.15.67:0/4099922569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 0x7fcb780cf330 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.794+0000 7fcb77e00640 1 --2- 172.21.15.67:0/4099922569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb780cf870 0x7fcb78162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.794+0000 7fcb77e00640 1 --2- 172.21.15.67:0/4099922569 >> v2:172.21.15.169:3300/0 conn(0x7fcb7815e870 0x7fcb780d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.794+0000 7fcb77e00640 1 -- 172.21.15.67:0/4099922569 >> 172.21.15.67:0/4099922569 conn(0x7fcb7808bfc0 msgr2=0x7fcb780d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.794+0000 7fcb77e00640 1 -- 172.21.15.67:0/4099922569 shutdown_connections 2024-05-30T16:58:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.794+0000 7fcb77e00640 1 -- 172.21.15.67:0/4099922569 wait complete. 2024-05-30T16:58:25.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.796+0000 7fcb77e00640 1 Processor -- start 2024-05-30T16:58:25.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.796+0000 7fcb77e00640 1 -- start start 2024-05-30T16:58:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.797+0000 7fcb77e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb780cf870 0x7fcb780b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.797+0000 7fcb77e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 0x7fcb780b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.797+0000 7fcb77e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb7815e870 0x7fcb781509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb77e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcb780d0820 con 0x7fcb780d14d0 2024-05-30T16:58:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb77e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb780d0540 con 0x7fcb780cf870 2024-05-30T16:58:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb77e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb780d4fb0 con 0x7fcb7815e870 2024-05-30T16:58:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb76000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 0x7fcb780b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb76000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 0x7fcb780b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49330/0 (socket says 172.21.15.67:49330) 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb76000640 1 -- 172.21.15.67:0/3080986400 learned_addr learned my addr 172.21.15.67:0/3080986400 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb76a00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb780cf870 0x7fcb780b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb77400640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb7815e870 0x7fcb781509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb76000640 1 -- 172.21.15.67:0/3080986400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb7815e870 msgr2=0x7fcb781509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb76000640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb7815e870 0x7fcb781509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb76000640 1 -- 172.21.15.67:0/3080986400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb780cf870 msgr2=0x7fcb780b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb76000640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb780cf870 0x7fcb780b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb76000640 1 -- 172.21.15.67:0/3080986400 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb78150f20 con 0x7fcb780d14d0 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.798+0000 7fcb77400640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb7815e870 0x7fcb781509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.799+0000 7fcb76000640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 0x7fcb780b71a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009e70 tx=0x7fcb60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:25.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.799+0000 7fcb6ea00640 1 -- 172.21.15.67:0/3080986400 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6003f020 con 0x7fcb780d14d0 2024-05-30T16:58:25.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.799+0000 7fcb6ea00640 1 -- 172.21.15.67:0/3080986400 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb60037040 con 0x7fcb780d14d0 2024-05-30T16:58:25.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.799+0000 7fcb6ea00640 1 -- 172.21.15.67:0/3080986400 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60036070 con 0x7fcb780d14d0 2024-05-30T16:58:25.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.799+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb780b78d0 con 0x7fcb780d14d0 2024-05-30T16:58:25.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.799+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb780b7e90 con 0x7fcb780d14d0 2024-05-30T16:58:25.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.800+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb780689e0 con 0x7fcb780d14d0 2024-05-30T16:58:25.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.802+0000 7fcb6ea00640 1 -- 172.21.15.67:0/3080986400 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fcb60042020 con 0x7fcb780d14d0 2024-05-30T16:58:25.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.803+0000 7fcb6ea00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb5406da60 0x7fcb5406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:25.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.803+0000 7fcb6ea00640 1 -- 172.21.15.67:0/3080986400 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 4913+0+0 (secure 0 0 0) 0x7fcb600aaf90 con 0x7fcb780d14d0 2024-05-30T16:58:25.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.803+0000 7fcb76a00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb5406da60 0x7fcb5406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:25.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.804+0000 7fcb76a00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb5406da60 0x7fcb5406ff20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800a910 tx=0x7fcb68008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:25.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:25.807+0000 7fcb6ea00640 1 -- 172.21.15.67:0/3080986400 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb60049030 con 0x7fcb780d14d0 2024-05-30T16:58:26.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.023+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcb780b6630 con 0x7fcb780d14d0 2024-05-30T16:58:26.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.024+0000 7fcb6ea00640 1 -- 172.21.15.67:0/3080986400 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) ==== 84+0+65 (secure 0 0 0) 0x7fcb6007a4d0 con 0x7fcb780d14d0 2024-05-30T16:58:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:26.025 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:58:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-05-30T16:58:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-05-30T16:58:26.025 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:26.025 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:58:26.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.026+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb5406da60 msgr2=0x7fcb5406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:26.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb5406da60 0x7fcb5406ff20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800a910 tx=0x7fcb68008040 comp rx=0 tx=0).stop 2024-05-30T16:58:26.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 msgr2=0x7fcb780b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:26.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 0x7fcb780b71a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009e70 tx=0x7fcb60004a40 comp rx=0 tx=0).stop 2024-05-30T16:58:26.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 shutdown_connections 2024-05-30T16:58:26.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb5406da60 0x7fcb5406ff20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:26.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb7815e870 0x7fcb781509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:26.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb780d14d0 0x7fcb780b71a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:26.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 --2- 172.21.15.67:0/3080986400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb780cf870 0x7fcb780b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:26.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 >> 172.21.15.67:0/3080986400 conn(0x7fcb7808bfc0 msgr2=0x7fcb780b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:26.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 shutdown_connections 2024-05-30T16:58:26.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.027+0000 7fcb77e00640 1 -- 172.21.15.67:0/3080986400 wait complete. 2024-05-30T16:58:26.046 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-30T16:58:26.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-30T16:58:26.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.243+0000 7fe3f2200640 1 Processor -- start 2024-05-30T16:58:26.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.243+0000 7fe3f2200640 1 -- start start 2024-05-30T16:58:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.244+0000 7fe3f2200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe3ec1621d0 0x7fe3ec0d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.244+0000 7fe3f2200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3ec0cf180 0x7fe3ec0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.244+0000 7fe3f2200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3ec0d0e50 con 0x7fe3ec1625b0 2024-05-30T16:58:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.244+0000 7fe3f2200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe3ec0d1a60 con 0x7fe3ec1621d0 2024-05-30T16:58:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.244+0000 7fe3f2200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ec0d1130 con 0x7fe3ec0cf180 2024-05-30T16:58:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.245+0000 7fe3f0e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe3ec1621d0 0x7fe3ec0d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.245+0000 7fe3f0e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe3ec1621d0 0x7fe3ec0d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49346/0 (socket says 172.21.15.67:49346) 2024-05-30T16:58:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.245+0000 7fe3f0e00640 1 -- 172.21.15.67:0/2276575756 learned_addr learned my addr 172.21.15.67:0/2276575756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.245+0000 7fe3f0e00640 1 -- 172.21.15.67:0/2276575756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3ec0cf180 msgr2=0x7fe3ec0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.245+0000 7fe3f0e00640 1 --2- 172.21.15.67:0/2276575756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3ec0cf180 0x7fe3ec0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.245+0000 7fe3f0e00640 1 -- 172.21.15.67:0/2276575756 >> v1:172.21.15.67:6789/0 conn(0x7fe3ec1625b0 legacy=0x7fe3ec0d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:26.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.245+0000 7fe3f0e00640 1 -- 172.21.15.67:0/2276575756 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec0d1410 con 0x7fe3ec1621d0 2024-05-30T16:58:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.246+0000 7fe3f0e00640 1 --2- 172.21.15.67:0/2276575756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec1621d0 0x7fe3ec0d3220 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009870 tx=0x7fe3e002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb89b3778ca7cac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.247+0000 7fe3eb400640 1 -- 172.21.15.67:0/2276575756 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0037070 con 0x7fe3ec1621d0 2024-05-30T16:58:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.247+0000 7fe3eb400640 1 -- 172.21.15.67:0/2276575756 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e0032440 con 0x7fe3ec1621d0 2024-05-30T16:58:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.247+0000 7fe3eb400640 1 -- 172.21.15.67:0/2276575756 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0032810 con 0x7fe3ec1621d0 2024-05-30T16:58:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.247+0000 7fe3f2200640 1 -- 172.21.15.67:0/2276575756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec1621d0 msgr2=0x7fe3ec0d3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.247+0000 7fe3f2200640 1 --2- 172.21.15.67:0/2276575756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec1621d0 0x7fe3ec0d3220 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009870 tx=0x7fe3e002a250 comp rx=0 tx=0).stop 2024-05-30T16:58:26.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.248+0000 7fe3f2200640 1 -- 172.21.15.67:0/2276575756 shutdown_connections 2024-05-30T16:58:26.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.248+0000 7fe3f2200640 1 --2- 172.21.15.67:0/2276575756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3ec0cf180 0x7fe3ec0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:26.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.248+0000 7fe3f2200640 1 --2- 172.21.15.67:0/2276575756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec1621d0 0x7fe3ec0d3220 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:26.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.248+0000 7fe3f2200640 1 -- 172.21.15.67:0/2276575756 >> 172.21.15.67:0/2276575756 conn(0x7fe3ec08bfc0 msgr2=0x7fe3ec0d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:26.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.248+0000 7fe3f2200640 1 -- 172.21.15.67:0/2276575756 shutdown_connections 2024-05-30T16:58:26.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.248+0000 7fe3f2200640 1 -- 172.21.15.67:0/2276575756 wait complete. 2024-05-30T16:58:26.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.249+0000 7fe3f2200640 1 Processor -- start 2024-05-30T16:58:26.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.250+0000 7fe3f2200640 1 -- start start 2024-05-30T16:58:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.250+0000 7fe3f2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec0cf180 0x7fe3ec0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.250+0000 7fe3f2200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3ec1621d0 0x7fe3ec1f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f2200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3ec1625b0 0x7fe3ec1f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec162a40 con 0x7fe3ec0cf180 2024-05-30T16:58:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f2200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec0d1ef0 con 0x7fe3ec1621d0 2024-05-30T16:58:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f2200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ec15fa20 con 0x7fe3ec1625b0 2024-05-30T16:58:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec0cf180 0x7fe3ec0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec0cf180 0x7fe3ec0c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49352/0 (socket says 172.21.15.67:49352) 2024-05-30T16:58:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f0e00640 1 -- 172.21.15.67:0/696953143 learned_addr learned my addr 172.21.15.67:0/696953143 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f1800640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3ec1625b0 0x7fe3ec1f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3ebe00640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3ec1621d0 0x7fe3ec1f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f0e00640 1 -- 172.21.15.67:0/696953143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3ec1625b0 msgr2=0x7fe3ec1f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f0e00640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3ec1625b0 0x7fe3ec1f8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f0e00640 1 -- 172.21.15.67:0/696953143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3ec1621d0 msgr2=0x7fe3ec1f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f0e00640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3ec1621d0 0x7fe3ec1f56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3f0e00640 1 -- 172.21.15.67:0/696953143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec1f5c10 con 0x7fe3ec0cf180 2024-05-30T16:58:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.251+0000 7fe3ebe00640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3ec1621d0 0x7fe3ec1f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.252+0000 7fe3f0e00640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec0cf180 0x7fe3ec0c9ec0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00097c0 tx=0x7fe3e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.252+0000 7fe3e8c00640 1 -- 172.21.15.67:0/696953143 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0041020 con 0x7fe3ec0cf180 2024-05-30T16:58:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.252+0000 7fe3e8c00640 1 -- 172.21.15.67:0/696953143 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e0030970 con 0x7fe3ec0cf180 2024-05-30T16:58:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.252+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ec0ca540 con 0x7fe3ec0cf180 2024-05-30T16:58:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.252+0000 7fe3e8c00640 1 -- 172.21.15.67:0/696953143 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0037040 con 0x7fe3ec0cf180 2024-05-30T16:58:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.252+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ec0cab00 con 0x7fe3ec0cf180 2024-05-30T16:58:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.254+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ec0689e0 con 0x7fe3ec0cf180 2024-05-30T16:58:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.254+0000 7fe3e8c00640 1 -- 172.21.15.67:0/696953143 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fe3e0044020 con 0x7fe3ec0cf180 2024-05-30T16:58:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.255+0000 7fe3e8c00640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe3cc06da60 0x7fe3cc06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.256+0000 7fe3e8c00640 1 -- 172.21.15.67:0/696953143 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 4913+0+0 (secure 0 0 0) 0x7fe3e00ab400 con 0x7fe3ec0cf180 2024-05-30T16:58:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.256+0000 7fe3ebe00640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe3cc06da60 0x7fe3cc06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:26.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.256+0000 7fe3ebe00640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe3cc06da60 0x7fe3cc06ff20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec1472e0 tx=0x7fe3d8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:26.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.260+0000 7fe3e8c00640 1 -- 172.21.15.67:0/696953143 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3e004b030 con 0x7fe3ec0cf180 2024-05-30T16:58:26.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:26.466+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe3ec0d4d10 con 0x7fe3ec0cf180 2024-05-30T16:58:27.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.069+0000 7fe3e8c00640 1 -- 172.21.15.67:0/696953143 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) ==== 138+0+0 (secure 0 0 0) 0x7fe3ec1f5c10 con 0x7fe3ec0cf180 2024-05-30T16:58:27.071 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-05-30T16:58:27.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.073+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe3cc06da60 msgr2=0x7fe3cc06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.073+0000 7fe3f2200640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe3cc06da60 0x7fe3cc06ff20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec1472e0 tx=0x7fe3d8009040 comp rx=0 tx=0).stop 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec0cf180 msgr2=0x7fe3ec0c9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec0cf180 0x7fe3ec0c9ec0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00097c0 tx=0x7fe3e00046c0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 shutdown_connections 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe3cc06da60 0x7fe3cc06ff20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3ec1625b0 0x7fe3ec1f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3ec1621d0 0x7fe3ec1f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 --2- 172.21.15.67:0/696953143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3ec0cf180 0x7fe3ec0c9ec0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 >> 172.21.15.67:0/696953143 conn(0x7fe3ec08bfc0 msgr2=0x7fe3ec15fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 shutdown_connections 2024-05-30T16:58:27.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.074+0000 7fe3f2200640 1 -- 172.21.15.67:0/696953143 wait complete. 2024-05-30T16:58:27.101 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-30T16:58:27.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.308+0000 7fcb6e600640 1 Processor -- start 2024-05-30T16:58:27.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.309+0000 7fcb6e600640 1 -- start start 2024-05-30T16:58:27.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.310+0000 7fcb6e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb680ce5d0 0x7fcb680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:27.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.310+0000 7fcb6e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb680b3580 con 0x7fcb680b3d60 2024-05-30T16:58:27.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.311+0000 7fcb6e600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb6815f490 con 0x7fcb680ce1f0 2024-05-30T16:58:27.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.311+0000 7fcb6e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6815f770 con 0x7fcb680ce5d0 2024-05-30T16:58:27.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.311+0000 7fcb6dc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcb680b3d60 0x7fcb680ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56340/0 (socket says 172.21.15.67:56340) 2024-05-30T16:58:27.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.311+0000 7fcb6dc00640 1 -- 172.21.15.67:0/4269490920 learned_addr learned my addr 172.21.15.67:0/4269490920 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:27.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.311+0000 7fcb67e00640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb680ce5d0 0x7fcb680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:27.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.311+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768027399 0 0) 0x7fcb680b3580 con 0x7fcb680b3d60 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.311+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb50003520 con 0x7fcb680b3d60 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3183028876 0 0) 0x7fcb50003520 con 0x7fcb680b3d60 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb680ce5d0 msgr2=0x7fcb680b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb680ce5d0 0x7fcb680b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 >> v1:172.21.15.169:6789/0 conn(0x7fcb680ce1f0 legacy=0x7fcb6815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6815fa50 con 0x7fcb680b3d60 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fcb600019f0 con 0x7fcb680b3d60 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 >> v1:172.21.15.67:6789/0 conn(0x7fcb680b3d60 legacy=0x7fcb680ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb500039e0 0x7fcb50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb680ce5d0 0x7fcb5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6815f770 con 0x7fcb500039e0 2024-05-30T16:58:27.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50005580 con 0x7fcb50004800 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5000bf00 con 0x7fcb680ce5d0 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.312+0000 7fcb6d200640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb500039e0 0x7fcb50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.313+0000 7fcb67e00640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.313+0000 7fcb6dc00640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb680ce5d0 0x7fcb5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.313+0000 7fcb6d200640 1 -- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb680ce5d0 msgr2=0x7fcb5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.313+0000 7fcb6d200640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb680ce5d0 0x7fcb5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.313+0000 7fcb6d200640 1 -- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb50004800 msgr2=0x7fcb50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.313+0000 7fcb6d200640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.313+0000 7fcb6d200640 1 -- 172.21.15.67:0/4269490920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb50003520 con 0x7fcb500039e0 2024-05-30T16:58:27.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.313+0000 7fcb6dc00640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb680ce5d0 0x7fcb5000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:27.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.314+0000 7fcb6d200640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb500039e0 0x7fcb50003e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c001780 tx=0x7fcb50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84a4ad5c9c7ee16 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:27.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.314+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c001370 con 0x7fcb500039e0 2024-05-30T16:58:27.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.314+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5c0052d0 con 0x7fcb500039e0 2024-05-30T16:58:27.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.314+0000 7fcb67400640 1 -- 172.21.15.67:0/4269490920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c02ac00 con 0x7fcb500039e0 2024-05-30T16:58:27.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.315+0000 7fcb6e600640 1 -- 172.21.15.67:0/4269490920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb500039e0 msgr2=0x7fcb50003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:27.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.315+0000 7fcb6e600640 1 --2- 172.21.15.67:0/4269490920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb500039e0 0x7fcb50003e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c001780 tx=0x7fcb50005830 comp rx=0 tx=0).stop 2024-05-30T16:58:27.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.315+0000 7fcb6dc00640 1 -- 172.21.15.67:0/4269490920 reap_dead start 2024-05-30T16:58:27.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.315+0000 7fcb6e600640 1 -- 172.21.15.67:0/4269490920 shutdown_connections 2024-05-30T16:58:27.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.315+0000 7fcb6e600640 1 -- 172.21.15.67:0/4269490920 >> 172.21.15.67:0/4269490920 conn(0x7fcb6806b900 msgr2=0x7fcb6806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:27.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.315+0000 7fcb6e600640 1 -- 172.21.15.67:0/4269490920 shutdown_connections 2024-05-30T16:58:27.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.315+0000 7fcb6e600640 1 -- 172.21.15.67:0/4269490920 wait complete. 2024-05-30T16:58:27.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.317+0000 7fcb6e600640 1 Processor -- start 2024-05-30T16:58:27.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.317+0000 7fcb6e600640 1 -- start start 2024-05-30T16:58:27.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.318+0000 7fcb6e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb500039e0 0x7fcb681f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:27.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.318+0000 7fcb6e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb6809a2b0 0x7fcb681f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:27.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.318+0000 7fcb6e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb681f5370 0x7fcb68161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:27.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.318+0000 7fcb6e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcb680cd420 con 0x7fcb681f5370 2024-05-30T16:58:27.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.318+0000 7fcb6e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb680cd2a0 con 0x7fcb500039e0 2024-05-30T16:58:27.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.318+0000 7fcb6e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb680b4dd0 con 0x7fcb6809a2b0 2024-05-30T16:58:27.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.318+0000 7fcb6dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb681f5370 0x7fcb68161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:27.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.318+0000 7fcb6dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb681f5370 0x7fcb68161fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49384/0 (socket says 172.21.15.67:49384) 2024-05-30T16:58:27.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.318+0000 7fcb6dc00640 1 -- 172.21.15.67:0/366202345 learned_addr learned my addr 172.21.15.67:0/366202345 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:27.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.319+0000 7fcb6d200640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb500039e0 0x7fcb681f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:27.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.319+0000 7fcb67e00640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb6809a2b0 0x7fcb681f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.319+0000 7fcb6dc00640 1 -- 172.21.15.67:0/366202345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb6809a2b0 msgr2=0x7fcb681f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.319+0000 7fcb6dc00640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb6809a2b0 0x7fcb681f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.319+0000 7fcb6dc00640 1 -- 172.21.15.67:0/366202345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb500039e0 msgr2=0x7fcb681f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.319+0000 7fcb6dc00640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb500039e0 0x7fcb681f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.319+0000 7fcb6dc00640 1 -- 172.21.15.67:0/366202345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb68160f80 con 0x7fcb681f5370 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.319+0000 7fcb6dc00640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb681f5370 0x7fcb68161fe0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcb680b3340 tx=0x7fcb6000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.319+0000 7fcb64c00640 1 -- 172.21.15.67:0/366202345 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60004d60 con 0x7fcb681f5370 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.320+0000 7fcb64c00640 1 -- 172.21.15.67:0/366202345 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb6000bb20 con 0x7fcb681f5370 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.320+0000 7fcb64c00640 1 -- 172.21.15.67:0/366202345 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60007110 con 0x7fcb681f5370 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.320+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb68161220 con 0x7fcb681f5370 2024-05-30T16:58:27.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.320+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb681617e0 con 0x7fcb681f5370 2024-05-30T16:58:27.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.321+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb680689e0 con 0x7fcb681f5370 2024-05-30T16:58:27.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.322+0000 7fcb64c00640 1 -- 172.21.15.67:0/366202345 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fcb6001b020 con 0x7fcb681f5370 2024-05-30T16:58:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.323+0000 7fcb64c00640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb4806da90 0x7fcb4806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.323+0000 7fcb64c00640 1 -- 172.21.15.67:0/366202345 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5283+0+0 (secure 0 0 0) 0x7fcb6008ac60 con 0x7fcb681f5370 2024-05-30T16:58:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.323+0000 7fcb6d200640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb4806da90 0x7fcb4806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.324+0000 7fcb6d200640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb4806da90 0x7fcb4806ff50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c032000 tx=0x7fcb5c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:27.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.328+0000 7fcb64c00640 1 -- 172.21.15.67:0/366202345 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb6005a790 con 0x7fcb681f5370 2024-05-30T16:58:27.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:27.534+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcb680adde0 con 0x7fcb681f5370 2024-05-30T16:58:28.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.077+0000 7fcb64c00640 1 -- 172.21.15.67:0/366202345 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) ==== 170+0+0 (secure 0 0 0) 0x7fcb6005a090 con 0x7fcb681f5370 2024-05-30T16:58:28.078 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-05-30T16:58:28.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb4806da90 msgr2=0x7fcb4806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:28.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb4806da90 0x7fcb4806ff50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c032000 tx=0x7fcb5c02e000 comp rx=0 tx=0).stop 2024-05-30T16:58:28.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb681f5370 msgr2=0x7fcb68161fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb681f5370 0x7fcb68161fe0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcb680b3340 tx=0x7fcb6000b4a0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 shutdown_connections 2024-05-30T16:58:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb4806da90 0x7fcb4806ff50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb681f5370 0x7fcb68161fe0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb6809a2b0 0x7fcb681f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 --2- 172.21.15.67:0/366202345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb500039e0 0x7fcb681f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.082+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 >> 172.21.15.67:0/366202345 conn(0x7fcb6806b900 msgr2=0x7fcb680ae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:28.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.083+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 shutdown_connections 2024-05-30T16:58:28.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.083+0000 7fcb6e600640 1 -- 172.21.15.67:0/366202345 wait complete. 2024-05-30T16:58:28.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-30T16:58:28.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.305+0000 7f717e200640 1 Processor -- start 2024-05-30T16:58:28.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.305+0000 7f717e200640 1 -- start start 2024-05-30T16:58:28.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.306+0000 7f717e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71780ce5d0 0x7f71780b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:28.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.306+0000 7f717e200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71780b3580 con 0x7f71780b3d60 2024-05-30T16:58:28.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.306+0000 7f717e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717815f490 con 0x7f71780ce1f0 2024-05-30T16:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.307+0000 7f717e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f717815f770 con 0x7f71780ce5d0 2024-05-30T16:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.307+0000 7f717d800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f71780b3d60 0x7f71780ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56346/0 (socket says 172.21.15.67:56346) 2024-05-30T16:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.307+0000 7f717d800640 1 -- 172.21.15.67:0/1810568768 learned_addr learned my addr 172.21.15.67:0/1810568768 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:28.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.307+0000 7f7177e00640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71780ce5d0 0x7f71780b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:28.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.307+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297906489 0 0) 0x7f71780b3580 con 0x7f71780b3d60 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.307+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7160003520 con 0x7f71780b3d60 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1493073428 0 0) 0x7f7160003520 con 0x7f71780b3d60 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71780ce5d0 msgr2=0x7f71780b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71780ce5d0 0x7f71780b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 >> v1:172.21.15.169:6789/0 conn(0x7f71780ce1f0 legacy=0x7f717815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717815fa50 con 0x7f71780b3d60 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f71700019f0 con 0x7f71780b3d60 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 >> v1:172.21.15.67:6789/0 conn(0x7f71780b3d60 legacy=0x7f71780ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71780ce5d0 0x7f71600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7160003f60 0x7f7160004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7160004d40 0x7f71600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f717815f770 con 0x7f7160003f60 2024-05-30T16:58:28.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f716000bbb0 con 0x7f7160004d40 2024-05-30T16:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f716000bdb0 con 0x7f71780ce5d0 2024-05-30T16:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.308+0000 7f7177e00640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7160003f60 0x7f7160004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.309+0000 7f717ce00640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71780ce5d0 0x7f71600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.309+0000 7f7177e00640 1 -- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71780ce5d0 msgr2=0x7f71600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.309+0000 7f7177e00640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71780ce5d0 0x7f71600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.309+0000 7f7177e00640 1 -- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7160004d40 msgr2=0x7f71600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.309+0000 7f7177e00640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7160004d40 0x7f71600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.309+0000 7f7177e00640 1 -- 172.21.15.67:0/1810568768 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717815fa50 con 0x7f7160003f60 2024-05-30T16:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.310+0000 7f7177e00640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7160003f60 0x7f7160004800 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7164009ac0 tx=0x7f7160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e2aee57f09cb24 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.310+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164036070 con 0x7f7160003f60 2024-05-30T16:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.310+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7164031400 con 0x7f7160003f60 2024-05-30T16:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.310+0000 7f7177400640 1 -- 172.21.15.67:0/1810568768 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164009690 con 0x7f7160003f60 2024-05-30T16:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.311+0000 7f717e200640 1 -- 172.21.15.67:0/1810568768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7160003f60 msgr2=0x7f7160004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.311+0000 7f717e200640 1 --2- 172.21.15.67:0/1810568768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7160003f60 0x7f7160004800 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7164009ac0 tx=0x7f7160005830 comp rx=0 tx=0).stop 2024-05-30T16:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.311+0000 7f717d800640 1 -- 172.21.15.67:0/1810568768 reap_dead start 2024-05-30T16:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.311+0000 7f717e200640 1 -- 172.21.15.67:0/1810568768 shutdown_connections 2024-05-30T16:58:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.311+0000 7f717e200640 1 -- 172.21.15.67:0/1810568768 >> 172.21.15.67:0/1810568768 conn(0x7f717806b900 msgr2=0x7f717806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.311+0000 7f717e200640 1 -- 172.21.15.67:0/1810568768 shutdown_connections 2024-05-30T16:58:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.311+0000 7f717e200640 1 -- 172.21.15.67:0/1810568768 wait complete. 2024-05-30T16:58:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.313+0000 7f717e200640 1 Processor -- start 2024-05-30T16:58:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.313+0000 7f717e200640 1 -- start start 2024-05-30T16:58:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.314+0000 7f717e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7160003f60 0x7f717815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.314+0000 7f717e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71781527d0 0x7f71781615c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.314+0000 7f717e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71781484d0 0x7f7178154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.314+0000 7f717e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f71780cd420 con 0x7f71781527d0 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.314+0000 7f717e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71780cd2a0 con 0x7f71781484d0 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.314+0000 7f717e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71780b4dd0 con 0x7f7160003f60 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.314+0000 7f7177e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71781527d0 0x7f71781615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.314+0000 7f7177e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71781527d0 0x7f71781615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49392/0 (socket says 172.21.15.67:49392) 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.314+0000 7f7177e00640 1 -- 172.21.15.67:0/1551406307 learned_addr learned my addr 172.21.15.67:0/1551406307 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.315+0000 7f717ce00640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7160003f60 0x7f717815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.315+0000 7f7177e00640 1 -- 172.21.15.67:0/1551406307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7160003f60 msgr2=0x7f717815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.315+0000 7f7177e00640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7160003f60 0x7f717815fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.315+0000 7f7177e00640 1 -- 172.21.15.67:0/1551406307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71781484d0 msgr2=0x7f7178154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.315+0000 7f717d800640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71781484d0 0x7f7178154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.315+0000 7f7177e00640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71781484d0 0x7f7178154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:28.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.315+0000 7f7177e00640 1 -- 172.21.15.67:0/1551406307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7178160520 con 0x7f71781527d0 2024-05-30T16:58:28.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.315+0000 7f717d800640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71781484d0 0x7f7178154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:28.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.315+0000 7f7177e00640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71781527d0 0x7f71781615c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7164000c00 tx=0x7f7164004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:28.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.316+0000 7f7174c00640 1 -- 172.21.15.67:0/1551406307 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164040020 con 0x7f71781527d0 2024-05-30T16:58:28.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.316+0000 7f7174c00640 1 -- 172.21.15.67:0/1551406307 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7164003c60 con 0x7f71781527d0 2024-05-30T16:58:28.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.316+0000 7f7174c00640 1 -- 172.21.15.67:0/1551406307 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164036040 con 0x7f71781527d0 2024-05-30T16:58:28.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.316+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7178160800 con 0x7f71781527d0 2024-05-30T16:58:28.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.316+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7178160dc0 con 0x7f71781527d0 2024-05-30T16:58:28.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.317+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71780689e0 con 0x7f71781527d0 2024-05-30T16:58:28.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.319+0000 7f7174c00640 1 -- 172.21.15.67:0/1551406307 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f7164043020 con 0x7f71781527d0 2024-05-30T16:58:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.319+0000 7f7174c00640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f715806da90 0x7f715806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.320+0000 7f7174c00640 1 -- 172.21.15.67:0/1551406307 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5665+0+0 (secure 0 0 0) 0x7f71640ab490 con 0x7f71781527d0 2024-05-30T16:58:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.320+0000 7f717ce00640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f715806da90 0x7f715806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.321+0000 7f717ce00640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f715806da90 0x7f715806ff50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f716c007fd0 tx=0x7f716c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.324+0000 7f7174c00640 1 -- 172.21.15.67:0/1551406307 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f716404a070 con 0x7f71781527d0 2024-05-30T16:58:28.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:28.571+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f71780ca070 con 0x7f71781527d0 2024-05-30T16:58:29.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.117+0000 7f7174c00640 1 -- 172.21.15.67:0/1551406307 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v51) ==== 174+0+0 (secure 0 0 0) 0x7f71640044e0 con 0x7f71781527d0 2024-05-30T16:58:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 12 and data pool 13 2024-05-30T16:58:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.127+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f715806da90 msgr2=0x7f715806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.127+0000 7f717e200640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f715806da90 0x7f715806ff50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f716c007fd0 tx=0x7f716c009040 comp rx=0 tx=0).stop 2024-05-30T16:58:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.127+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71781527d0 msgr2=0x7f71781615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.127+0000 7f717e200640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71781527d0 0x7f71781615c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7164000c00 tx=0x7f7164004730 comp rx=0 tx=0).stop 2024-05-30T16:58:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.128+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 shutdown_connections 2024-05-30T16:58:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.128+0000 7f717e200640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f715806da90 0x7f715806ff50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.128+0000 7f717e200640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71781484d0 0x7f7178154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.128+0000 7f717e200640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71781527d0 0x7f71781615c0 secure :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7164000c00 tx=0x7f7164004730 comp rx=0 tx=0).stop 2024-05-30T16:58:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.128+0000 7f717e200640 1 --2- 172.21.15.67:0/1551406307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7160003f60 0x7f717815fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.129+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 >> 172.21.15.67:0/1551406307 conn(0x7f717806b900 msgr2=0x7f71780ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:29.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.132+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 shutdown_connections 2024-05-30T16:58:29.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.132+0000 7f717e200640 1 -- 172.21.15.67:0/1551406307 wait complete. 2024-05-30T16:58:29.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:58:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.344+0000 7f9a33200640 1 Processor -- start 2024-05-30T16:58:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.345+0000 7f9a33200640 1 -- start start 2024-05-30T16:58:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.346+0000 7f9a33200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a3415e870 0x7f9a340d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.346+0000 7f9a33200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a340cfa40 0x7f9a340d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.346+0000 7f9a33200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a3415ec50 con 0x7f9a340d14d0 2024-05-30T16:58:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.346+0000 7f9a33200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9a340d2150 con 0x7f9a3415e870 2024-05-30T16:58:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.346+0000 7f9a33200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9a340cff80 con 0x7f9a340cfa40 2024-05-30T16:58:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.346+0000 7f9a31e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a3415e870 0x7f9a340d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.346+0000 7f9a31e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a3415e870 0x7f9a340d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49394/0 (socket says 172.21.15.67:49394) 2024-05-30T16:58:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.346+0000 7f9a31e00640 1 -- 172.21.15.67:0/4232837341 learned_addr learned my addr 172.21.15.67:0/4232837341 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:29.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.346+0000 7f9a29400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9a340d14d0 0x7f9a340cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56354/0 (socket says 172.21.15.67:56354) 2024-05-30T16:58:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.347+0000 7f9a32800640 1 --2- 172.21.15.67:0/4232837341 >> v2:172.21.15.169:3300/0 conn(0x7f9a340cfa40 0x7f9a340d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.347+0000 7f9a31e00640 1 -- 172.21.15.67:0/4232837341 >> v2:172.21.15.169:3300/0 conn(0x7f9a340cfa40 msgr2=0x7f9a340d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.347+0000 7f9a31e00640 1 --2- 172.21.15.67:0/4232837341 >> v2:172.21.15.169:3300/0 conn(0x7f9a340cfa40 0x7f9a340d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.347+0000 7f9a31e00640 1 -- 172.21.15.67:0/4232837341 >> v1:172.21.15.67:6789/0 conn(0x7f9a340d14d0 legacy=0x7f9a340cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.347+0000 7f9a31400640 1 -- 172.21.15.67:0/4232837341 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2129031922 0 0) 0x7f9a3415ec50 con 0x7f9a340d14d0 2024-05-30T16:58:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.347+0000 7f9a31e00640 1 -- 172.21.15.67:0/4232837341 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a340d0260 con 0x7f9a3415e870 2024-05-30T16:58:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.348+0000 7f9a31e00640 1 --2- 172.21.15.67:0/4232837341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 0x7f9a340d0f90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f9a3408cad0 tx=0x7f9a2402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567ea7bc5073514f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.348+0000 7f9a31400640 1 -- 172.21.15.67:0/4232837341 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24036070 con 0x7f9a3415e870 2024-05-30T16:58:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.348+0000 7f9a31400640 1 -- 172.21.15.67:0/4232837341 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a24031600 con 0x7f9a3415e870 2024-05-30T16:58:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.348+0000 7f9a31400640 1 -- 172.21.15.67:0/4232837341 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a240319d0 con 0x7f9a3415e870 2024-05-30T16:58:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.349+0000 7f9a33200640 1 -- 172.21.15.67:0/4232837341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 msgr2=0x7f9a340d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.349+0000 7f9a33200640 1 --2- 172.21.15.67:0/4232837341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 0x7f9a340d0f90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f9a3408cad0 tx=0x7f9a2402a730 comp rx=0 tx=0).stop 2024-05-30T16:58:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.349+0000 7f9a33200640 1 -- 172.21.15.67:0/4232837341 shutdown_connections 2024-05-30T16:58:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.349+0000 7f9a33200640 1 --2- 172.21.15.67:0/4232837341 >> v2:172.21.15.169:3300/0 conn(0x7f9a340cfa40 0x7f9a340d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.349+0000 7f9a33200640 1 --2- 172.21.15.67:0/4232837341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 0x7f9a340d0f90 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.349+0000 7f9a33200640 1 -- 172.21.15.67:0/4232837341 >> 172.21.15.67:0/4232837341 conn(0x7f9a3408bfc0 msgr2=0x7f9a340d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.349+0000 7f9a33200640 1 -- 172.21.15.67:0/4232837341 shutdown_connections 2024-05-30T16:58:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.349+0000 7f9a33200640 1 -- 172.21.15.67:0/4232837341 wait complete. 2024-05-30T16:58:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.351+0000 7f9a33200640 1 Processor -- start 2024-05-30T16:58:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.351+0000 7f9a33200640 1 -- start start 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.352+0000 7f9a33200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a340cfa40 0x7f9a3414f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.352+0000 7f9a33200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a340d14d0 0x7f9a340b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.352+0000 7f9a33200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 0x7f9a340b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.352+0000 7f9a33200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9a340d4fb0 con 0x7f9a3415e870 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.352+0000 7f9a33200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a340d0820 con 0x7f9a340cfa40 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.352+0000 7f9a33200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a340d2b20 con 0x7f9a340d14d0 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a32800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 0x7f9a340b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a32800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 0x7f9a340b8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49402/0 (socket says 172.21.15.67:49402) 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a32800640 1 -- 172.21.15.67:0/1719340164 learned_addr learned my addr 172.21.15.67:0/1719340164 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a31e00640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a340cfa40 0x7f9a3414f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a29400640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a340d14d0 0x7f9a340b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a32800640 1 -- 172.21.15.67:0/1719340164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a340d14d0 msgr2=0x7f9a340b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a32800640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a340d14d0 0x7f9a340b71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a32800640 1 -- 172.21.15.67:0/1719340164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a340cfa40 msgr2=0x7f9a3414f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a32800640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a340cfa40 0x7f9a3414f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a32800640 1 -- 172.21.15.67:0/1719340164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a340b7b60 con 0x7f9a3415e870 2024-05-30T16:58:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a31e00640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a340cfa40 0x7f9a3414f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T16:58:29.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.353+0000 7f9a32800640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 0x7f9a340b8c70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d5d0 tx=0x7f9a2c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.354+0000 7f9a2aa00640 1 -- 172.21.15.67:0/1719340164 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0125f0 con 0x7f9a3415e870 2024-05-30T16:58:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.354+0000 7f9a2aa00640 1 -- 172.21.15.67:0/1719340164 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2c019070 con 0x7f9a3415e870 2024-05-30T16:58:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.354+0000 7f9a2aa00640 1 -- 172.21.15.67:0/1719340164 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0129d0 con 0x7f9a3415e870 2024-05-30T16:58:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.354+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a340b7e40 con 0x7f9a3415e870 2024-05-30T16:58:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.354+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a340c8430 con 0x7f9a3415e870 2024-05-30T16:58:29.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.355+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a340c8630 con 0x7f9a3415e870 2024-05-30T16:58:29.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.356+0000 7f9a2aa00640 1 -- 172.21.15.67:0/1719340164 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f9a2c012b70 con 0x7f9a3415e870 2024-05-30T16:58:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.357+0000 7f9a2aa00640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1006da90 0x7f9a1006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.357+0000 7f9a2aa00640 1 -- 172.21.15.67:0/1719340164 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5765+0+0 (secure 0 0 0) 0x7f9a2c08ae70 con 0x7f9a3415e870 2024-05-30T16:58:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.357+0000 7f9a31e00640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1006da90 0x7f9a1006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.358+0000 7f9a31e00640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1006da90 0x7f9a1006ff50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9a24033000 tx=0x7f9a24009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:29.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.362+0000 7f9a2aa00640 1 -- 172.21.15.67:0/1719340164 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a340c8630 con 0x7f9a3415e870 2024-05-30T16:58:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.568+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a340ba500 con 0x7f9a3415e870 2024-05-30T16:58:29.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.570+0000 7f9a2aa00640 1 -- 172.21.15.67:0/1719340164 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16748 (secure 0 0 0) 0x7f9a2c05aa00 con 0x7f9a3415e870 2024-05-30T16:58:29.571 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:29.571 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":88,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:58:28.573129+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:58:26.480292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-30T16:58:27.548326+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1006da90 msgr2=0x7f9a1006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1006da90 0x7f9a1006ff50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9a24033000 tx=0x7f9a24009000 comp rx=0 tx=0).stop 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 msgr2=0x7f9a340b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 0x7f9a340b8c70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00d5d0 tx=0x7f9a2c00da90 comp rx=0 tx=0).stop 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 shutdown_connections 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9a1006da90 0x7f9a1006ff50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a3415e870 0x7f9a340b8c70 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a340d14d0 0x7f9a340b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 --2- 172.21.15.67:0/1719340164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a340cfa40 0x7f9a3414f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 >> 172.21.15.67:0/1719340164 conn(0x7f9a3408bfc0 msgr2=0x7f9a340b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:29.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 shutdown_connections 2024-05-30T16:58:29.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.573+0000 7f9a33200640 1 -- 172.21.15.67:0/1719340164 wait complete. 2024-05-30T16:58:29.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:58:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.801+0000 7f806e400640 1 Processor -- start 2024-05-30T16:58:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.801+0000 7f806e400640 1 -- start start 2024-05-30T16:58:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.802+0000 7f806e400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f806815e870 0x7f80680d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.802+0000 7f806e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80680d14d0 0x7f80680cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.802+0000 7f806e400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80680cfd00 con 0x7f80680cf870 2024-05-30T16:58:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.802+0000 7f806e400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f80680d2150 con 0x7f806815e870 2024-05-30T16:58:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.802+0000 7f806e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f806815ec50 con 0x7f80680d14d0 2024-05-30T16:58:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.803+0000 7f806da00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f80680cf870 0x7f8068162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56362/0 (socket says 172.21.15.67:56362) 2024-05-30T16:58:29.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.803+0000 7f806da00640 1 -- 172.21.15.67:0/1688745987 learned_addr learned my addr 172.21.15.67:0/1688745987 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:29.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.803+0000 7f806d000640 1 --2- 172.21.15.67:0/1688745987 >> v2:172.21.15.169:3300/0 conn(0x7f806815e870 0x7f80680d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.803+0000 7f8067e00640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80680d14d0 0x7f80680cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.803+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2891848322 0 0) 0x7f8060001110 con 0x7f80680cf870 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.803+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f804c003520 con 0x7f80680cf870 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1168164183 0 0) 0x7f804c003520 con 0x7f80680cf870 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80680d14d0 msgr2=0x7f80680cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80680d14d0 0x7f80680cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 >> v2:172.21.15.169:3300/0 conn(0x7f806815e870 msgr2=0x7f80680d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 --2- 172.21.15.67:0/1688745987 >> v2:172.21.15.169:3300/0 conn(0x7f806815e870 0x7f80680d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80680cff80 con 0x7f80680cf870 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f8060001840 con 0x7f80680cf870 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 >> v1:172.21.15.67:6789/0 conn(0x7f80680cf870 legacy=0x7f8068162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0039b0 0x7f804c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f804c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80680d14d0 0x7f804c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f80680d2150 con 0x7f804c004800 2024-05-30T16:58:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f806815ec50 con 0x7f804c0039b0 2024-05-30T16:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f804c00b9c0 con 0x7f80680d14d0 2024-05-30T16:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.804+0000 7f8067e00640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f804c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.805+0000 7f806d000640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0039b0 0x7f804c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.805+0000 7f8067e00640 1 -- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80680d14d0 msgr2=0x7f804c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.805+0000 7f8067e00640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80680d14d0 0x7f804c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.805+0000 7f8067e00640 1 -- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0039b0 msgr2=0x7f804c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.805+0000 7f8067e00640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0039b0 0x7f804c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.805+0000 7f8067e00640 1 -- 172.21.15.67:0/1688745987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c003520 con 0x7f804c004800 2024-05-30T16:58:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.805+0000 7f806d000640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f804c0039b0 0x7f804c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.806+0000 7f8067e00640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f804c004c80 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8054009ab0 tx=0x7f804c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52addb0bdd59d0e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.806+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054036070 con 0x7f804c004800 2024-05-30T16:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.806+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8054005990 con 0x7f804c004800 2024-05-30T16:58:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.806+0000 7f8067400640 1 -- 172.21.15.67:0/1688745987 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054009680 con 0x7f804c004800 2024-05-30T16:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.807+0000 7f806e400640 1 -- 172.21.15.67:0/1688745987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 msgr2=0x7f804c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.807+0000 7f806e400640 1 --2- 172.21.15.67:0/1688745987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f804c004c80 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8054009ab0 tx=0x7f804c005830 comp rx=0 tx=0).stop 2024-05-30T16:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.807+0000 7f806da00640 1 -- 172.21.15.67:0/1688745987 reap_dead start 2024-05-30T16:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.807+0000 7f806e400640 1 -- 172.21.15.67:0/1688745987 shutdown_connections 2024-05-30T16:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.807+0000 7f806e400640 1 -- 172.21.15.67:0/1688745987 >> 172.21.15.67:0/1688745987 conn(0x7f806808bfc0 msgr2=0x7f80680d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.807+0000 7f806e400640 1 -- 172.21.15.67:0/1688745987 shutdown_connections 2024-05-30T16:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.807+0000 7f806e400640 1 -- 172.21.15.67:0/1688745987 wait complete. 2024-05-30T16:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.809+0000 7f806e400640 1 Processor -- start 2024-05-30T16:58:29.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.809+0000 7f806e400640 1 -- start start 2024-05-30T16:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.810+0000 7f806e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f8068159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.810+0000 7f806e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80681571f0 0x7f80680b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.810+0000 7f806e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80680b9310 0x7f80680cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.810+0000 7f806e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f80680d0820 con 0x7f804c004800 2024-05-30T16:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.810+0000 7f806e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f80680d0540 con 0x7f80680b9310 2024-05-30T16:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.810+0000 7f806e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f80680d4fb0 con 0x7f80681571f0 2024-05-30T16:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.810+0000 7f806d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f8068159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.810+0000 7f806d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f8068159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49416/0 (socket says 172.21.15.67:49416) 2024-05-30T16:58:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.810+0000 7f806d000640 1 -- 172.21.15.67:0/3125676141 learned_addr learned my addr 172.21.15.67:0/3125676141 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f8067e00640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80681571f0 0x7f80680b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f806da00640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80680b9310 0x7f80680cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f806d000640 1 -- 172.21.15.67:0/3125676141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80681571f0 msgr2=0x7f80680b8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f806d000640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80681571f0 0x7f80680b8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f806d000640 1 -- 172.21.15.67:0/3125676141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80680b9310 msgr2=0x7f80680cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f806d000640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80680b9310 0x7f80680cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f806d000640 1 -- 172.21.15.67:0/3125676141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80680cb6f0 con 0x7f804c004800 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f8067e00640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80681571f0 0x7f80680b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f806d000640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f8068159040 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f805800f400 tx=0x7f805800f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f8064c00640 1 -- 172.21.15.67:0/3125676141 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058003990 con 0x7f804c004800 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f8064c00640 1 -- 172.21.15.67:0/3125676141 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8058003b30 con 0x7f804c004800 2024-05-30T16:58:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f8064c00640 1 -- 172.21.15.67:0/3125676141 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058004020 con 0x7f804c004800 2024-05-30T16:58:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.811+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80680cb9d0 con 0x7f804c004800 2024-05-30T16:58:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.812+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8068159a20 con 0x7f804c004800 2024-05-30T16:58:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.813+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80680689e0 con 0x7f804c004800 2024-05-30T16:58:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.814+0000 7f8064c00640 1 -- 172.21.15.67:0/3125676141 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f8058005d10 con 0x7f804c004800 2024-05-30T16:58:29.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.815+0000 7f8064c00640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f804806da60 0x7f804806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:29.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.815+0000 7f8064c00640 1 -- 172.21.15.67:0/3125676141 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5765+0+0 (secure 0 0 0) 0x7f805808b5a0 con 0x7f804c004800 2024-05-30T16:58:29.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.816+0000 7f8067e00640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f804806da60 0x7f804806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:29.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.816+0000 7f8067e00640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f804806da60 0x7f804806ff20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8054033000 tx=0x7f8054008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:29.820+0000 7f8064c00640 1 -- 172.21.15.67:0/3125676141 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8058024020 con 0x7f804c004800 2024-05-30T16:58:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.025+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80680ba500 con 0x7f804c004800 2024-05-30T16:58:30.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.028+0000 7f8064c00640 1 -- 172.21.15.67:0/3125676141 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16748 (secure 0 0 0) 0x7f805805b0d0 con 0x7f804c004800 2024-05-30T16:58:30.029 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:30.029 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":88,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:58:28.573129+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:58:26.480292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-30T16:58:27.548326+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:58:30.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.030+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f804806da60 msgr2=0x7f804806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.030+0000 7f806e400640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f804806da60 0x7f804806ff20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8054033000 tx=0x7f8054008030 comp rx=0 tx=0).stop 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.030+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 msgr2=0x7f8068159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.030+0000 7f806e400640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f8068159040 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f805800f400 tx=0x7f805800f8c0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.031+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 shutdown_connections 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.031+0000 7f806e400640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f804806da60 0x7f804806ff20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.031+0000 7f806e400640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80680b9310 0x7f80680cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.031+0000 7f806e400640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80681571f0 0x7f80680b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.031+0000 7f806e400640 1 --2- 172.21.15.67:0/3125676141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f804c004800 0x7f8068159040 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.031+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 >> 172.21.15.67:0/3125676141 conn(0x7f806808bfc0 msgr2=0x7f80680b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.031+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 shutdown_connections 2024-05-30T16:58:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.031+0000 7f806e400640 1 -- 172.21.15.67:0/3125676141 wait complete. 2024-05-30T16:58:30.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-30T16:58:30.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.265+0000 7f850b200640 1 Processor -- start 2024-05-30T16:58:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.266+0000 7f850b200640 1 -- start start 2024-05-30T16:58:30.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.267+0000 7f850b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f850c1639b0 0x7f850c0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:30.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.267+0000 7f850b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f850c163d90 0x7f850c166670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:30.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.267+0000 7f850b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850c1670c0 con 0x7f850c166bb0 2024-05-30T16:58:30.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.267+0000 7f850b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f850c0ceca0 con 0x7f850c163d90 2024-05-30T16:58:30.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.267+0000 7f850b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f850c1673a0 con 0x7f850c1639b0 2024-05-30T16:58:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.268+0000 7f8503e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f850c163d90 0x7f850c166670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.268+0000 7f850a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f850c166bb0 0x7f850c0ceee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56370/0 (socket says 172.21.15.67:56370) 2024-05-30T16:58:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.268+0000 7f850a800640 1 -- 172.21.15.67:0/1257540515 learned_addr learned my addr 172.21.15.67:0/1257540515 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.268+0000 7f8509e00640 1 --2- 172.21.15.67:0/1257540515 >> v2:172.21.15.169:3300/0 conn(0x7f850c1639b0 0x7f850c0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.268+0000 7f8503e00640 1 -- 172.21.15.67:0/1257540515 >> v2:172.21.15.169:3300/0 conn(0x7f850c1639b0 msgr2=0x7f850c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:30.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.268+0000 7f8503e00640 1 --2- 172.21.15.67:0/1257540515 >> v2:172.21.15.169:3300/0 conn(0x7f850c1639b0 0x7f850c0ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.268+0000 7f8503e00640 1 -- 172.21.15.67:0/1257540515 >> v1:172.21.15.67:6789/0 conn(0x7f850c166bb0 legacy=0x7f850c0ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:30.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.268+0000 7f8503e00640 1 -- 172.21.15.67:0/1257540515 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c0cdf60 con 0x7f850c163d90 2024-05-30T16:58:30.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.268+0000 7f8509400640 1 -- 172.21.15.67:0/1257540515 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969140181 0 0) 0x7f850c1670c0 con 0x7f850c166bb0 2024-05-30T16:58:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.269+0000 7f8503e00640 1 --2- 172.21.15.67:0/1257540515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c163d90 0x7f850c166670 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f850c0d1190 tx=0x7f84f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c3d6daa191a87a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.270+0000 7f8509400640 1 -- 172.21.15.67:0/1257540515 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4037070 con 0x7f850c163d90 2024-05-30T16:58:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.270+0000 7f8509400640 1 -- 172.21.15.67:0/1257540515 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f4032440 con 0x7f850c163d90 2024-05-30T16:58:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.270+0000 7f8509400640 1 -- 172.21.15.67:0/1257540515 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4032810 con 0x7f850c163d90 2024-05-30T16:58:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.270+0000 7f850b200640 1 -- 172.21.15.67:0/1257540515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c163d90 msgr2=0x7f850c166670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.270+0000 7f850b200640 1 --2- 172.21.15.67:0/1257540515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c163d90 0x7f850c166670 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f850c0d1190 tx=0x7f84f402a1a0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.270+0000 7f850b200640 1 -- 172.21.15.67:0/1257540515 shutdown_connections 2024-05-30T16:58:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.270+0000 7f850b200640 1 --2- 172.21.15.67:0/1257540515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c163d90 0x7f850c166670 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.270+0000 7f850b200640 1 --2- 172.21.15.67:0/1257540515 >> v2:172.21.15.169:3300/0 conn(0x7f850c1639b0 0x7f850c0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.271+0000 7f850b200640 1 -- 172.21.15.67:0/1257540515 >> 172.21.15.67:0/1257540515 conn(0x7f850c06b900 msgr2=0x7f850c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.271+0000 7f850b200640 1 -- 172.21.15.67:0/1257540515 shutdown_connections 2024-05-30T16:58:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.271+0000 7f850b200640 1 -- 172.21.15.67:0/1257540515 wait complete. 2024-05-30T16:58:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.272+0000 7f850b200640 1 Processor -- start 2024-05-30T16:58:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.273+0000 7f850b200640 1 -- start start 2024-05-30T16:58:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f850b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c1639b0 0x7f850c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f850b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f850c163d90 0x7f850c072ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f850b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f850c166bb0 0x7f850c0733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f850b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0afbf0 con 0x7f850c1639b0 2024-05-30T16:58:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f850b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0afa70 con 0x7f850c163d90 2024-05-30T16:58:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f850b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f850c0cf5f0 con 0x7f850c166bb0 2024-05-30T16:58:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f8509e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c1639b0 0x7f850c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f8509e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c1639b0 0x7f850c15da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49428/0 (socket says 172.21.15.67:49428) 2024-05-30T16:58:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f8509e00640 1 -- 172.21.15.67:0/974592069 learned_addr learned my addr 172.21.15.67:0/974592069 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f850a800640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f850c166bb0 0x7f850c0733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f8509e00640 1 -- 172.21.15.67:0/974592069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f850c166bb0 msgr2=0x7f850c0733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f8509e00640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f850c166bb0 0x7f850c0733e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f8503e00640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f850c163d90 0x7f850c072ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f8509e00640 1 -- 172.21.15.67:0/974592069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f850c163d90 msgr2=0x7f850c072ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.274+0000 7f8509e00640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f850c163d90 0x7f850c072ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.275+0000 7f8509e00640 1 -- 172.21.15.67:0/974592069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c15e320 con 0x7f850c1639b0 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.275+0000 7f8509e00640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c1639b0 0x7f850c15da20 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00d800 tx=0x7f84fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.275+0000 7f8502000640 1 -- 172.21.15.67:0/974592069 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00b6c0 con 0x7f850c1639b0 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.275+0000 7f8502000640 1 -- 172.21.15.67:0/974592069 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc019070 con 0x7f850c1639b0 2024-05-30T16:58:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.275+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c15e600 con 0x7f850c1639b0 2024-05-30T16:58:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.275+0000 7f8502000640 1 -- 172.21.15.67:0/974592069 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00ba80 con 0x7f850c1639b0 2024-05-30T16:58:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.275+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c15ebc0 con 0x7f850c1639b0 2024-05-30T16:58:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.277+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c15edc0 con 0x7f850c1639b0 2024-05-30T16:58:30.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.278+0000 7f8502000640 1 -- 172.21.15.67:0/974592069 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f84fc00bc20 con 0x7f850c1639b0 2024-05-30T16:58:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.279+0000 7f8502000640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f84e806da60 0x7f84e806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.280+0000 7f8502000640 1 -- 172.21.15.67:0/974592069 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5765+0+0 (secure 0 0 0) 0x7f84fc08aef0 con 0x7f850c1639b0 2024-05-30T16:58:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.280+0000 7f8503e00640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f84e806da60 0x7f84e806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:30.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.280+0000 7f8503e00640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f84e806da60 0x7f84e806ff20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:30.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.284+0000 7f8502000640 1 -- 172.21.15.67:0/974592069 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84fc01c360 con 0x7f850c1639b0 2024-05-30T16:58:30.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:30.529+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f850c0689e0 con 0x7f850c1639b0 2024-05-30T16:58:31.066 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-05-30T16:58:31.065+0000 7f8a02400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T16:58:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.140+0000 7f8502000640 1 -- 172.21.15.67:0/974592069 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) ==== 118+0+0 (secure 0 0 0) 0x7f84fc05aa80 con 0x7f850c1639b0 2024-05-30T16:58:31.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f84e806da60 msgr2=0x7f84e806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:31.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f84e806da60 0x7f84e806ff20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f402e000 comp rx=0 tx=0).stop 2024-05-30T16:58:31.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c1639b0 msgr2=0x7f850c15da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:31.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c1639b0 0x7f850c15da20 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00d800 tx=0x7f84fc00dcc0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 shutdown_connections 2024-05-30T16:58:31.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f84e806da60 0x7f84e806ff20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f850c166bb0 0x7f850c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f850c163d90 0x7f850c072ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 --2- 172.21.15.67:0/974592069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f850c1639b0 0x7f850c15da20 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.144+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 >> 172.21.15.67:0/974592069 conn(0x7f850c06b900 msgr2=0x7f850c1605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:31.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.145+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 shutdown_connections 2024-05-30T16:58:31.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.145+0000 7f850b200640 1 -- 172.21.15.67:0/974592069 wait complete. 2024-05-30T16:58:31.171 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-30T16:58:31.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-30T16:58:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.389+0000 7fc809400640 1 Processor -- start 2024-05-30T16:58:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.390+0000 7fc809400640 1 -- start start 2024-05-30T16:58:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.391+0000 7fc809400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc804153510 0x7fc8040b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.391+0000 7fc809400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc80415d270 0x7fc8040ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.391+0000 7fc809400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc80415d900 con 0x7fc8040ccb90 2024-05-30T16:58:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.391+0000 7fc809400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc8041538f0 con 0x7fc80415d270 2024-05-30T16:58:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.391+0000 7fc809400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc8040cd530 con 0x7fc804153510 2024-05-30T16:58:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.392+0000 7fc808a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc80415d270 0x7fc8040ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.392+0000 7fc803e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc804153510 0x7fc8040b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.392+0000 7fc803400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc8040ccb90 0x7fc80415cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:38590/0 (socket says 172.21.15.67:38590) 2024-05-30T16:58:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.392+0000 7fc803400640 1 -- 172.21.15.67:0/1457690283 learned_addr learned my addr 172.21.15.67:0/1457690283 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:31.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.393+0000 7fc802a00640 1 -- 172.21.15.67:0/1457690283 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1467730913 0 0) 0x7fc80415d900 con 0x7fc8040ccb90 2024-05-30T16:58:31.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.393+0000 7fc802a00640 1 -- 172.21.15.67:0/1457690283 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7e8003520 con 0x7fc8040ccb90 2024-05-30T16:58:31.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.393+0000 7fc803e00640 1 -- 172.21.15.67:0/1457690283 >> v2:172.21.15.169:3300/0 conn(0x7fc80415d270 msgr2=0x7fc8040ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:31.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.393+0000 7fc803e00640 1 --2- 172.21.15.67:0/1457690283 >> v2:172.21.15.169:3300/0 conn(0x7fc80415d270 0x7fc8040ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.393+0000 7fc803e00640 1 -- 172.21.15.67:0/1457690283 >> v1:172.21.15.169:6789/0 conn(0x7fc8040ccb90 legacy=0x7fc80415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:31.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.393+0000 7fc803e00640 1 -- 172.21.15.67:0/1457690283 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8040b3f90 con 0x7fc804153510 2024-05-30T16:58:31.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.395+0000 7fc803e00640 1 --2- 172.21.15.67:0/1457690283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc804153510 0x7fc8040b4d50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40097a0 tx=0x7fc7f402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f16adafd76f6cf8c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:31.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.396+0000 7fc802a00640 1 -- 172.21.15.67:0/1457690283 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4037070 con 0x7fc804153510 2024-05-30T16:58:31.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.396+0000 7fc802a00640 1 -- 172.21.15.67:0/1457690283 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f402ae50 con 0x7fc804153510 2024-05-30T16:58:31.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.396+0000 7fc802a00640 1 -- 172.21.15.67:0/1457690283 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4032660 con 0x7fc804153510 2024-05-30T16:58:31.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.396+0000 7fc809400640 1 -- 172.21.15.67:0/1457690283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc804153510 msgr2=0x7fc8040b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:31.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.396+0000 7fc809400640 1 --2- 172.21.15.67:0/1457690283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc804153510 0x7fc8040b4d50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40097a0 tx=0x7fc7f402a100 comp rx=0 tx=0).stop 2024-05-30T16:58:31.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.396+0000 7fc809400640 1 -- 172.21.15.67:0/1457690283 shutdown_connections 2024-05-30T16:58:31.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.396+0000 7fc809400640 1 --2- 172.21.15.67:0/1457690283 >> v2:172.21.15.169:3300/0 conn(0x7fc80415d270 0x7fc8040ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.396+0000 7fc809400640 1 --2- 172.21.15.67:0/1457690283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc804153510 0x7fc8040b4d50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.396+0000 7fc809400640 1 -- 172.21.15.67:0/1457690283 >> 172.21.15.67:0/1457690283 conn(0x7fc80406b900 msgr2=0x7fc80406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:31.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.397+0000 7fc809400640 1 -- 172.21.15.67:0/1457690283 shutdown_connections 2024-05-30T16:58:31.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.397+0000 7fc809400640 1 -- 172.21.15.67:0/1457690283 wait complete. 2024-05-30T16:58:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.399+0000 7fc809400640 1 Processor -- start 2024-05-30T16:58:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.399+0000 7fc809400640 1 -- start start 2024-05-30T16:58:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc809400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8040ccb90 0x7fc8040729a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc809400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc804153510 0x7fc80414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc809400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc80415d270 0x7fc8040cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc809400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc8040b4550 con 0x7fc804153510 2024-05-30T16:58:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc809400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8040b48c0 con 0x7fc80415d270 2024-05-30T16:58:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc809400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc80406e940 con 0x7fc8040ccb90 2024-05-30T16:58:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc803400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc804153510 0x7fc80414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc803400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc804153510 0x7fc80414a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50154/0 (socket says 172.21.15.67:50154) 2024-05-30T16:58:31.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc803400640 1 -- 172.21.15.67:0/314254744 learned_addr learned my addr 172.21.15.67:0/314254744 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc803e00640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8040ccb90 0x7fc8040729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc808a00640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc80415d270 0x7fc8040cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc803400640 1 -- 172.21.15.67:0/314254744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8040ccb90 msgr2=0x7fc8040729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc803400640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8040ccb90 0x7fc8040729a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc803400640 1 -- 172.21.15.67:0/314254744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc80415d270 msgr2=0x7fc8040cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc803400640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc80415d270 0x7fc8040cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:31.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.401+0000 7fc803400640 1 -- 172.21.15.67:0/314254744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc804071c20 con 0x7fc804153510 2024-05-30T16:58:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.402+0000 7fc803400640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc804153510 0x7fc80414a720 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000c500 tx=0x7fc7f000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.402+0000 7fc7fbe00640 1 -- 172.21.15.67:0/314254744 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0017070 con 0x7fc804153510 2024-05-30T16:58:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.402+0000 7fc7fbe00640 1 -- 172.21.15.67:0/314254744 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f0005ce0 con 0x7fc804153510 2024-05-30T16:58:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.402+0000 7fc7fbe00640 1 -- 172.21.15.67:0/314254744 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0007330 con 0x7fc804153510 2024-05-30T16:58:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.402+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc804071ec0 con 0x7fc804153510 2024-05-30T16:58:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.402+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc804072480 con 0x7fc804153510 2024-05-30T16:58:31.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.404+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8040689e0 con 0x7fc804153510 2024-05-30T16:58:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.405+0000 7fc7fbe00640 1 -- 172.21.15.67:0/314254744 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fc7f001a070 con 0x7fc804153510 2024-05-30T16:58:31.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.406+0000 7fc7fbe00640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7e406da60 0x7fc7e406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:31.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.406+0000 7fc7fbe00640 1 -- 172.21.15.67:0/314254744 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5765+0+0 (secure 0 0 0) 0x7fc7f0089f00 con 0x7fc804153510 2024-05-30T16:58:31.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.406+0000 7fc803e00640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7e406da60 0x7fc7e406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:31.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.407+0000 7fc803e00640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7e406da60 0x7fc7e406ff20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4034000 tx=0x7fc7f402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:31.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.410+0000 7fc7fbe00640 1 -- 172.21.15.67:0/314254744 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7f0059af0 con 0x7fc804153510 2024-05-30T16:58:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:31.655+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc8040b3c40 con 0x7fc804153510 2024-05-30T16:58:32.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.153+0000 7fc7fbe00640 1 -- 172.21.15.67:0/314254744 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7fc7f00593f0 con 0x7fc804153510 2024-05-30T16:58:32.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.156+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7e406da60 msgr2=0x7fc7e406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.156+0000 7fc809400640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7e406da60 0x7fc7e406ff20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4034000 tx=0x7fc7f402e000 comp rx=0 tx=0).stop 2024-05-30T16:58:32.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.156+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc804153510 msgr2=0x7fc80414a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.156+0000 7fc809400640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc804153510 0x7fc80414a720 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000c500 tx=0x7fc7f000c9c0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.157+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 shutdown_connections 2024-05-30T16:58:32.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.157+0000 7fc809400640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7e406da60 0x7fc7e406ff20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.157+0000 7fc809400640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc80415d270 0x7fc8040cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.157+0000 7fc809400640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc804153510 0x7fc80414a720 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.157+0000 7fc809400640 1 --2- 172.21.15.67:0/314254744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8040ccb90 0x7fc8040729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.157+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 >> 172.21.15.67:0/314254744 conn(0x7fc80406b900 msgr2=0x7fc80406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:32.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.157+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 shutdown_connections 2024-05-30T16:58:32.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.157+0000 7fc809400640 1 -- 172.21.15.67:0/314254744 wait complete. 2024-05-30T16:58:32.178 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-30T16:58:32.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:58:32.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.390+0000 7f7935a00640 1 Processor -- start 2024-05-30T16:58:32.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.390+0000 7f7935a00640 1 -- start start 2024-05-30T16:58:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.391+0000 7f7935a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f793006f860 0x7f79300b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f7935a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300b3690 0x7f79300d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f7935a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79300d09e0 0x7f7930160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f7935a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f79300d0e50 con 0x7f79300d09e0 2024-05-30T16:58:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f7935a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f793006ed10 con 0x7f793006f860 2024-05-30T16:58:32.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f7935a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f793015cb00 con 0x7f79300b3690 2024-05-30T16:58:32.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f792f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79300d09e0 0x7f7930160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f792f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79300d09e0 0x7f7930160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50156/0 (socket says 172.21.15.67:50156) 2024-05-30T16:58:32.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f792f400640 1 -- 172.21.15.67:0/1551240403 learned_addr learned my addr 172.21.15.67:0/1551240403 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:32.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f7926000640 1 --2- 172.21.15.67:0/1551240403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300b3690 0x7f79300d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f792f400640 1 -- 172.21.15.67:0/1551240403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300b3690 msgr2=0x7f79300d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:32.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f792f400640 1 --2- 172.21.15.67:0/1551240403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300b3690 0x7f79300d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f792f400640 1 -- 172.21.15.67:0/1551240403 >> v2:172.21.15.169:3300/0 conn(0x7f793006f860 msgr2=0x7f79300b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:32.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f792ea00640 1 --2- 172.21.15.67:0/1551240403 >> v2:172.21.15.169:3300/0 conn(0x7f793006f860 0x7f79300b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f792f400640 1 --2- 172.21.15.67:0/1551240403 >> v2:172.21.15.169:3300/0 conn(0x7f793006f860 0x7f79300b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.392+0000 7f792f400640 1 -- 172.21.15.67:0/1551240403 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79300b3cc0 con 0x7f79300d09e0 2024-05-30T16:58:32.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.393+0000 7f792f400640 1 --2- 172.21.15.67:0/1551240403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300d09e0 0x7f7930160210 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f793006c8d0 tx=0x7f792802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1862adb60b6ab346 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.394+0000 7f792e000640 1 -- 172.21.15.67:0/1551240403 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928036070 con 0x7f79300d09e0 2024-05-30T16:58:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.394+0000 7f792e000640 1 -- 172.21.15.67:0/1551240403 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7928031600 con 0x7f79300d09e0 2024-05-30T16:58:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.394+0000 7f792e000640 1 -- 172.21.15.67:0/1551240403 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79280319d0 con 0x7f79300d09e0 2024-05-30T16:58:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.394+0000 7f7935a00640 1 -- 172.21.15.67:0/1551240403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300d09e0 msgr2=0x7f7930160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.394+0000 7f7935a00640 1 --2- 172.21.15.67:0/1551240403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300d09e0 0x7f7930160210 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f793006c8d0 tx=0x7f792802a6a0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.394+0000 7f7935a00640 1 -- 172.21.15.67:0/1551240403 shutdown_connections 2024-05-30T16:58:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.395+0000 7f7935a00640 1 --2- 172.21.15.67:0/1551240403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300d09e0 0x7f7930160210 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.395+0000 7f7935a00640 1 --2- 172.21.15.67:0/1551240403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300b3690 0x7f79300d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.395+0000 7f7935a00640 1 --2- 172.21.15.67:0/1551240403 >> v2:172.21.15.169:3300/0 conn(0x7f793006f860 0x7f79300b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.395+0000 7f7935a00640 1 -- 172.21.15.67:0/1551240403 >> 172.21.15.67:0/1551240403 conn(0x7f793006bae0 msgr2=0x7f793006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.395+0000 7f7935a00640 1 -- 172.21.15.67:0/1551240403 shutdown_connections 2024-05-30T16:58:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.395+0000 7f7935a00640 1 -- 172.21.15.67:0/1551240403 wait complete. 2024-05-30T16:58:32.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.396+0000 7f7935a00640 1 Processor -- start 2024-05-30T16:58:32.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.396+0000 7f7935a00640 1 -- start start 2024-05-30T16:58:32.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.397+0000 7f7935a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f793006f860 0x7f7930099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.397+0000 7f7935a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300b3690 0x7f793015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.397+0000 7f7935a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300d09e0 0x7f793015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.397+0000 7f7935a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f79300b4280 con 0x7f79300b3690 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.397+0000 7f7935a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79300b3a70 con 0x7f793006f860 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.397+0000 7f7935a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79300b4560 con 0x7f79300d09e0 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f7926000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300b3690 0x7f793015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f7926000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300b3690 0x7f793015e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50172/0 (socket says 172.21.15.67:50172) 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f7926000640 1 -- 172.21.15.67:0/2671367067 learned_addr learned my addr 172.21.15.67:0/2671367067 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f792f400640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300d09e0 0x7f793015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f792ea00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f793006f860 0x7f7930099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f7926000640 1 -- 172.21.15.67:0/2671367067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300d09e0 msgr2=0x7f793015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f7926000640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300d09e0 0x7f793015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f7926000640 1 -- 172.21.15.67:0/2671367067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f793006f860 msgr2=0x7f7930099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f7926000640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f793006f860 0x7f7930099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f7926000640 1 -- 172.21.15.67:0/2671367067 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79300c8fe0 con 0x7f79300b3690 2024-05-30T16:58:32.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.398+0000 7f792ea00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f793006f860 0x7f7930099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:32.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.399+0000 7f7926000640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300b3690 0x7f793015e050 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f791800c7f0 tx=0x7f791800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.399+0000 7f7927400640 1 -- 172.21.15.67:0/2671367067 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918015070 con 0x7f79300b3690 2024-05-30T16:58:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.399+0000 7f7927400640 1 -- 172.21.15.67:0/2671367067 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7918004240 con 0x7f79300b3690 2024-05-30T16:58:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.399+0000 7f7927400640 1 -- 172.21.15.67:0/2671367067 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918007620 con 0x7f79300b3690 2024-05-30T16:58:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.399+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79300c92c0 con 0x7f79300b3690 2024-05-30T16:58:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.399+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79300c9880 con 0x7f79300b3690 2024-05-30T16:58:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.400+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79300689e0 con 0x7f79300b3690 2024-05-30T16:58:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.401+0000 7f7927400640 1 -- 172.21.15.67:0/2671367067 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f79180079d0 con 0x7f79300b3690 2024-05-30T16:58:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.402+0000 7f7927400640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f790c06da60 0x7f790c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.402+0000 7f7927400640 1 -- 172.21.15.67:0/2671367067 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5765+0+0 (secure 0 0 0) 0x7f791808ae90 con 0x7f79300b3690 2024-05-30T16:58:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.403+0000 7f792ea00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f790c06da60 0x7f790c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.403+0000 7f792ea00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f790c06da60 0x7f790c06ff20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7920008890 tx=0x7f7920008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.407+0000 7f7927400640 1 -- 172.21.15.67:0/2671367067 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f791805a9c0 con 0x7f79300b3690 2024-05-30T16:58:32.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.656+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79300b2400 con 0x7f79300b3690 2024-05-30T16:58:32.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.658+0000 7f7927400640 1 -- 172.21.15.67:0/2671367067 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5189 (secure 0 0 0) 0x7f791805a2c0 con 0x7f79300b3690 2024-05-30T16:58:32.658 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:32.659 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1054215955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1054215955},{"type":"v1","addr":"172.21.15.67:6837","nonce":1054215955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":5591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1236503792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1236503792},{"type":"v1","addr":"172.21.15.169:6835","nonce":1236503792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/4047293074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4047293074},{"type":"v1","addr":"172.21.15.67:6836","nonce":4047293074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:58:28.573056+0000","modified":"2024-05-30T16:58:31.656482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7398":{"gid":7398,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/2733959172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2733959172},{"type":"v1","addr":"172.21.15.169:6834","nonce":2733959172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7398,"qdb_cluster":[7398]},"id":6}]} 2024-05-30T16:58:32.659 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 55 2024-05-30T16:58:32.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f790c06da60 msgr2=0x7f790c06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f790c06da60 0x7f790c06ff20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7920008890 tx=0x7f7920008040 comp rx=0 tx=0).stop 2024-05-30T16:58:32.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300b3690 msgr2=0x7f793015e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300b3690 0x7f793015e050 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f791800c7f0 tx=0x7f791800ccb0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 shutdown_connections 2024-05-30T16:58:32.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f790c06da60 0x7f790c06ff20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79300d09e0 0x7f793015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79300b3690 0x7f793015e050 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 --2- 172.21.15.67:0/2671367067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f793006f860 0x7f7930099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 >> 172.21.15.67:0/2671367067 conn(0x7f793006bae0 msgr2=0x7f79300ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:32.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 shutdown_connections 2024-05-30T16:58:32.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.661+0000 7f7935a00640 1 -- 172.21.15.67:0/2671367067 wait complete. 2024-05-30T16:58:32.681 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T16:58:32.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.885+0000 7ff4eea00640 1 Processor -- start 2024-05-30T16:58:32.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.885+0000 7ff4eea00640 1 -- start start 2024-05-30T16:58:32.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.886+0000 7ff4eea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.886+0000 7ff4eea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff4e80d0ca0 0x7ff4e8099090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.886+0000 7ff4eea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e80b51f0 con 0x7ff4e80ce370 2024-05-30T16:58:32.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.886+0000 7ff4eea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e80aef60 con 0x7ff4e80d0ca0 2024-05-30T16:58:32.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.886+0000 7ff4eea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e80ad1d0 con 0x7ff4e80b4e10 2024-05-30T16:58:32.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.887+0000 7ff4e7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.887+0000 7ff4e7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:37158/0 (socket says 172.21.15.67:37158) 2024-05-30T16:58:32.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.887+0000 7ff4e7e00640 1 -- 172.21.15.67:0/2770585098 learned_addr learned my addr 172.21.15.67:0/2770585098 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:32.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.887+0000 7ff4eca00640 1 --2- 172.21.15.67:0/2770585098 >> v2:172.21.15.169:3300/0 conn(0x7ff4e80d0ca0 0x7ff4e8099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.888+0000 7ff4e7e00640 1 -- 172.21.15.67:0/2770585098 >> v2:172.21.15.169:3300/0 conn(0x7ff4e80d0ca0 msgr2=0x7ff4e8099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:32.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.888+0000 7ff4e7e00640 1 --2- 172.21.15.67:0/2770585098 >> v2:172.21.15.169:3300/0 conn(0x7ff4e80d0ca0 0x7ff4e8099090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.888+0000 7ff4e7e00640 1 -- 172.21.15.67:0/2770585098 >> v1:172.21.15.169:6789/0 conn(0x7ff4e80ce370 legacy=0x7ff4e80d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.888+0000 7ff4e7e00640 1 -- 172.21.15.67:0/2770585098 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e80ad470 con 0x7ff4e80b4e10 2024-05-30T16:58:32.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.889+0000 7ff4e7e00640 1 --2- 172.21.15.67:0/2770585098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80cde30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009870 tx=0x7ff4d802a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3853827b041caf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:32.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.889+0000 7ff4e6a00640 1 -- 172.21.15.67:0/2770585098 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8037070 con 0x7ff4e80b4e10 2024-05-30T16:58:32.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4e6a00640 1 -- 172.21.15.67:0/2770585098 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d8032440 con 0x7ff4e80b4e10 2024-05-30T16:58:32.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4e6a00640 1 -- 172.21.15.67:0/2770585098 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8032810 con 0x7ff4e80b4e10 2024-05-30T16:58:32.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4eea00640 1 -- 172.21.15.67:0/2770585098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 msgr2=0x7ff4e80cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4eea00640 1 --2- 172.21.15.67:0/2770585098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80cde30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009870 tx=0x7ff4d802a1d0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4eea00640 1 -- 172.21.15.67:0/2770585098 shutdown_connections 2024-05-30T16:58:32.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4eea00640 1 --2- 172.21.15.67:0/2770585098 >> v2:172.21.15.169:3300/0 conn(0x7ff4e80d0ca0 0x7ff4e8099090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4eea00640 1 --2- 172.21.15.67:0/2770585098 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80cde30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4eea00640 1 -- 172.21.15.67:0/2770585098 >> 172.21.15.67:0/2770585098 conn(0x7ff4e806bae0 msgr2=0x7ff4e806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:32.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4eea00640 1 -- 172.21.15.67:0/2770585098 shutdown_connections 2024-05-30T16:58:32.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.890+0000 7ff4eea00640 1 -- 172.21.15.67:0/2770585098 wait complete. 2024-05-30T16:58:32.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.892+0000 7ff4eea00640 1 Processor -- start 2024-05-30T16:58:32.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.892+0000 7ff4eea00640 1 -- start start 2024-05-30T16:58:32.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.893+0000 7ff4eea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.893+0000 7ff4eea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e80ce370 0x7ff4e80c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.893+0000 7ff4eea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e80d0ca0 0x7ff4e80c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.893+0000 7ff4eea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e80cea00 con 0x7ff4e80ce370 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.893+0000 7ff4eea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e80ce880 con 0x7ff4e80d0ca0 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.893+0000 7ff4eea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e80b3150 con 0x7ff4e80b4e10 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.893+0000 7ff4e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e80ce370 0x7ff4e80c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e80ce370 0x7ff4e80c3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50180/0 (socket says 172.21.15.67:50180) 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4e7400640 1 -- 172.21.15.67:0/4182038585 learned_addr learned my addr 172.21.15.67:0/4182038585 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4eca00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e80d0ca0 0x7ff4e80c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4e7e00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4e7400640 1 -- 172.21.15.67:0/4182038585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 msgr2=0x7ff4e80c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4e7400640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4e7400640 1 -- 172.21.15.67:0/4182038585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e80d0ca0 msgr2=0x7ff4e80c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4e7400640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e80d0ca0 0x7ff4e80c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4e7400640 1 -- 172.21.15.67:0/4182038585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e80c9700 con 0x7ff4e80ce370 2024-05-30T16:58:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.894+0000 7ff4e7400640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e80ce370 0x7ff4e80c3590 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400c440 tx=0x7ff4d400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.895+0000 7ff4dfe00640 1 -- 172.21.15.67:0/4182038585 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40052d0 con 0x7ff4e80ce370 2024-05-30T16:58:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.895+0000 7ff4dfe00640 1 -- 172.21.15.67:0/4182038585 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d4005470 con 0x7ff4e80ce370 2024-05-30T16:58:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.895+0000 7ff4dfe00640 1 -- 172.21.15.67:0/4182038585 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4007260 con 0x7ff4e80ce370 2024-05-30T16:58:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.895+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e80c99e0 con 0x7ff4e80ce370 2024-05-30T16:58:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.895+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e80c9fa0 con 0x7ff4e80ce370 2024-05-30T16:58:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.896+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e80689e0 con 0x7ff4e80ce370 2024-05-30T16:58:32.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.898+0000 7ff4dfe00640 1 -- 172.21.15.67:0/4182038585 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7ff4d40074c0 con 0x7ff4e80ce370 2024-05-30T16:58:32.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.899+0000 7ff4dfe00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c806da60 0x7ff4c806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:32.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.899+0000 7ff4dfe00640 1 -- 172.21.15.67:0/4182038585 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5765+0+0 (secure 0 0 0) 0x7ff4d4089f20 con 0x7ff4e80ce370 2024-05-30T16:58:32.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.899+0000 7ff4e7e00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c806da60 0x7ff4c806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:32.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.900+0000 7ff4e7e00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c806da60 0x7ff4c806ff20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8034000 tx=0x7ff4d802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:32.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:32.904+0000 7ff4dfe00640 1 -- 172.21.15.67:0/4182038585 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4d4059a50 con 0x7ff4e80ce370 2024-05-30T16:58:33.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.113+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4e806f540 con 0x7ff4e80ce370 2024-05-30T16:58:33.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.115+0000 7ff4dfe00640 1 -- 172.21.15.67:0/4182038585 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16735 (secure 0 0 0) 0x7ff4d4059350 con 0x7ff4e80ce370 2024-05-30T16:58:33.116 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:33.116 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:58:30.117530+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:58:26.480292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-30T16:58:27.548326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T16:58:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c806da60 msgr2=0x7ff4c806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c806da60 0x7ff4c806ff20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8034000 tx=0x7ff4d802e000 comp rx=0 tx=0).stop 2024-05-30T16:58:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e80ce370 msgr2=0x7ff4e80c3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e80ce370 0x7ff4e80c3590 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400c440 tx=0x7ff4d400c900 comp rx=0 tx=0).stop 2024-05-30T16:58:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 shutdown_connections 2024-05-30T16:58:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4c806da60 0x7ff4c806ff20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4e80d0ca0 0x7ff4e80c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4e80ce370 0x7ff4e80c3590 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 --2- 172.21.15.67:0/4182038585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4e80b4e10 0x7ff4e80c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 >> 172.21.15.67:0/4182038585 conn(0x7ff4e806bae0 msgr2=0x7ff4e81522e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:33.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 shutdown_connections 2024-05-30T16:58:33.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.118+0000 7ff4eea00640 1 -- 172.21.15.67:0/4182038585 wait complete. 2024-05-30T16:58:33.138 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T16:58:33.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:58:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.348+0000 7feea5a00640 1 Processor -- start 2024-05-30T16:58:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.348+0000 7feea5a00640 1 -- start start 2024-05-30T16:58:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.350+0000 7feea5a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feea0156b30 0x7feea00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.350+0000 7feea5a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feea006ee40 0x7feea00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.350+0000 7feea5a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea0156f10 con 0x7feea006f2f0 2024-05-30T16:58:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.350+0000 7feea5a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7feea006dda0 con 0x7feea006ee40 2024-05-30T16:58:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.350+0000 7feea5a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7feea006e080 con 0x7feea0156b30 2024-05-30T16:58:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.350+0000 7fee9f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feea006ee40 0x7feea00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.350+0000 7fee9f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feea006ee40 0x7feea00ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50184/0 (socket says 172.21.15.67:50184) 2024-05-30T16:58:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.350+0000 7fee9f400640 1 -- 172.21.15.67:0/2937334865 learned_addr learned my addr 172.21.15.67:0/2937334865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.351+0000 7fee9fe00640 1 --2- 172.21.15.67:0/2937334865 >> v2:172.21.15.169:3300/0 conn(0x7feea0156b30 0x7feea00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.351+0000 7fee9f400640 1 -- 172.21.15.67:0/2937334865 >> v2:172.21.15.169:3300/0 conn(0x7feea0156b30 msgr2=0x7feea00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.351+0000 7fee9f400640 1 --2- 172.21.15.67:0/2937334865 >> v2:172.21.15.169:3300/0 conn(0x7feea0156b30 0x7feea00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.351+0000 7fee9f400640 1 -- 172.21.15.67:0/2937334865 >> v1:172.21.15.169:6789/0 conn(0x7feea006f2f0 legacy=0x7feea00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.351+0000 7fee9f400640 1 -- 172.21.15.67:0/2937334865 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea006e360 con 0x7feea006ee40 2024-05-30T16:58:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.352+0000 7fee9f400640 1 --2- 172.21.15.67:0/2937334865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006ee40 0x7feea00ad230 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7feea006c8b0 tx=0x7fee8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71420e4c8a6b0408 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.352+0000 7fee9ea00640 1 -- 172.21.15.67:0/2937334865 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c037070 con 0x7feea006ee40 2024-05-30T16:58:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.352+0000 7fee9ea00640 1 -- 172.21.15.67:0/2937334865 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c032440 con 0x7feea006ee40 2024-05-30T16:58:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.352+0000 7fee9ea00640 1 -- 172.21.15.67:0/2937334865 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c032810 con 0x7feea006ee40 2024-05-30T16:58:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.353+0000 7feea5a00640 1 -- 172.21.15.67:0/2937334865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006ee40 msgr2=0x7feea00ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.353+0000 7feea5a00640 1 --2- 172.21.15.67:0/2937334865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006ee40 0x7feea00ad230 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7feea006c8b0 tx=0x7fee8c02a1a0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.353+0000 7feea5a00640 1 -- 172.21.15.67:0/2937334865 shutdown_connections 2024-05-30T16:58:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.353+0000 7feea5a00640 1 --2- 172.21.15.67:0/2937334865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006ee40 0x7feea00ad230 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.353+0000 7feea5a00640 1 --2- 172.21.15.67:0/2937334865 >> v2:172.21.15.169:3300/0 conn(0x7feea0156b30 0x7feea00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.353+0000 7feea5a00640 1 -- 172.21.15.67:0/2937334865 >> 172.21.15.67:0/2937334865 conn(0x7feea00b1fc0 msgr2=0x7feea00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.353+0000 7feea5a00640 1 -- 172.21.15.67:0/2937334865 shutdown_connections 2024-05-30T16:58:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.353+0000 7feea5a00640 1 -- 172.21.15.67:0/2937334865 wait complete. 2024-05-30T16:58:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.355+0000 7feea5a00640 1 Processor -- start 2024-05-30T16:58:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.355+0000 7feea5a00640 1 -- start start 2024-05-30T16:58:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.356+0000 7feea5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea006ee40 0x7feea0146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.356+0000 7feea5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006f2f0 0x7feea0155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.356+0000 7feea5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea0156b30 0x7feea0072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.356+0000 7feea5a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7feea00afc20 con 0x7feea006f2f0 2024-05-30T16:58:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.356+0000 7feea5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feea00b2c30 con 0x7feea006ee40 2024-05-30T16:58:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.356+0000 7feea5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feea0159710 con 0x7feea0156b30 2024-05-30T16:58:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.356+0000 7fee9f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006f2f0 0x7feea0155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.356+0000 7fee9f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006f2f0 0x7feea0155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50196/0 (socket says 172.21.15.67:50196) 2024-05-30T16:58:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.356+0000 7fee9f400640 1 -- 172.21.15.67:0/4285873914 learned_addr learned my addr 172.21.15.67:0/4285873914 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.357+0000 7feea5000640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea0156b30 0x7feea0072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.357+0000 7fee9fe00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea006ee40 0x7feea0146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.357+0000 7fee9f400640 1 -- 172.21.15.67:0/4285873914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea0156b30 msgr2=0x7feea0072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.357+0000 7fee9f400640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea0156b30 0x7feea0072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.357+0000 7fee9f400640 1 -- 172.21.15.67:0/4285873914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea006ee40 msgr2=0x7feea0146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.357+0000 7fee9f400640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea006ee40 0x7feea0146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.357+0000 7fee9f400640 1 -- 172.21.15.67:0/4285873914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea0158670 con 0x7feea006f2f0 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.357+0000 7fee9fe00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea006ee40 0x7feea0146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.357+0000 7fee9f400640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006f2f0 0x7feea0155dc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fee8c000c00 tx=0x7fee8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.358+0000 7fee97e00640 1 -- 172.21.15.67:0/4285873914 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c041020 con 0x7feea006f2f0 2024-05-30T16:58:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.358+0000 7fee97e00640 1 -- 172.21.15.67:0/4285873914 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c0306c0 con 0x7feea006f2f0 2024-05-30T16:58:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.358+0000 7fee97e00640 1 -- 172.21.15.67:0/4285873914 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c037040 con 0x7feea006f2f0 2024-05-30T16:58:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.358+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea0158950 con 0x7feea006f2f0 2024-05-30T16:58:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.358+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea0158f10 con 0x7feea006f2f0 2024-05-30T16:58:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.359+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea00689e0 con 0x7feea006f2f0 2024-05-30T16:58:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.360+0000 7fee97e00640 1 -- 172.21.15.67:0/4285873914 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fee8c044020 con 0x7feea006f2f0 2024-05-30T16:58:33.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.361+0000 7fee97e00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee8006da60 0x7fee8006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.361+0000 7fee97e00640 1 -- 172.21.15.67:0/4285873914 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5823+0+0 (secure 0 0 0) 0x7fee8c0aafb0 con 0x7feea006f2f0 2024-05-30T16:58:33.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.361+0000 7fee9fe00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee8006da60 0x7fee8006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.362+0000 7fee9fe00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee8006da60 0x7fee8006ff20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee90006fd0 tx=0x7fee90008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:33.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.366+0000 7fee97e00640 1 -- 172.21.15.67:0/4285873914 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee8c04b030 con 0x7feea006f2f0 2024-05-30T16:58:33.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.566+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feea006f7f0 con 0x7fee8006da60 2024-05-30T16:58:33.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.588+0000 7fee97e00640 1 -- 172.21.15.67:0/4285873914 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252079 (secure 0 0 0) 0x7feea006f7f0 con 0x7fee8006da60 2024-05-30T16:58:33.592 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:33.594 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:58:33.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.594+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee8006da60 msgr2=0x7fee8006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.594+0000 7feea5a00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee8006da60 0x7fee8006ff20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee90006fd0 tx=0x7fee90008040 comp rx=0 tx=0).stop 2024-05-30T16:58:33.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.594+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006f2f0 msgr2=0x7feea0155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.594+0000 7feea5a00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006f2f0 0x7feea0155dc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fee8c000c00 tx=0x7fee8c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.595+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 shutdown_connections 2024-05-30T16:58:33.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.595+0000 7feea5a00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee8006da60 0x7fee8006ff20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.595+0000 7feea5a00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea0156b30 0x7feea0072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.595+0000 7feea5a00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feea006f2f0 0x7feea0155dc0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.595+0000 7feea5a00640 1 --2- 172.21.15.67:0/4285873914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea006ee40 0x7feea0146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.595+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 >> 172.21.15.67:0/4285873914 conn(0x7feea00b1fc0 msgr2=0x7feea00b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:33.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.595+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 shutdown_connections 2024-05-30T16:58:33.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.595+0000 7feea5a00640 1 -- 172.21.15.67:0/4285873914 wait complete. 2024-05-30T16:58:33.617 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2024-05-30T16:58:33.163344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":861056,"kb_used_data":9600,"kb_used_omap":12,"kb_used_meta":851315,"kb_avail":754113664,"statfs":{"total":773094113280,"available":772212391936,"internally_reserved":0,"allocated":9830400,"data_stored":4893790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":871747120},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.083687"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173826+0000","last_change":"2024-05-30T16:58:29.134700+0000","last_active":"2024-05-30T16:58:30.173826+0000","last_peered":"2024-05-30T16:58:30.173826+0000","last_clean":"2024-05-30T16:58:30.173826+0000","last_became_active":"2024-05-30T16:58:29.134424+0000","last_became_peered":"2024-05-30T16:58:29.134424+0000","last_unstale":"2024-05-30T16:58:30.173826+0000","last_undegraded":"2024-05-30T16:58:30.173826+0000","last_fullsized":"2024-05-30T16:58:30.173826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:44.238425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142500+0000","last_change":"2024-05-30T16:58:29.134283+0000","last_active":"2024-05-30T16:58:30.142500+0000","last_peered":"2024-05-30T16:58:30.142500+0000","last_clean":"2024-05-30T16:58:30.142500+0000","last_became_active":"2024-05-30T16:58:29.133622+0000","last_became_peered":"2024-05-30T16:58:29.133622+0000","last_unstale":"2024-05-30T16:58:30.142500+0000","last_undegraded":"2024-05-30T16:58:30.142500+0000","last_fullsized":"2024-05-30T16:58:30.142500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:49.663228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.137575+0000","last_change":"2024-05-30T16:58:29.135737+0000","last_active":"2024-05-30T16:58:29.137575+0000","last_peered":"2024-05-30T16:58:29.137575+0000","last_clean":"2024-05-30T16:58:29.137575+0000","last_became_active":"2024-05-30T16:58:29.130881+0000","last_became_peered":"2024-05-30T16:58:29.130881+0000","last_unstale":"2024-05-30T16:58:29.137575+0000","last_undegraded":"2024-05-30T16:58:29.137575+0000","last_fullsized":"2024-05-30T16:58:29.137575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:15:59.315185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128147+0000","last_change":"2024-05-30T16:58:29.143594+0000","last_active":"2024-05-30T16:58:30.128147+0000","last_peered":"2024-05-30T16:58:30.128147+0000","last_clean":"2024-05-30T16:58:30.128147+0000","last_became_active":"2024-05-30T16:58:29.142724+0000","last_became_peered":"2024-05-30T16:58:29.142724+0000","last_unstale":"2024-05-30T16:58:30.128147+0000","last_undegraded":"2024-05-30T16:58:30.128147+0000","last_fullsized":"2024-05-30T16:58:30.128147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:25.159552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216284+0000","last_change":"2024-05-30T16:58:29.142988+0000","last_active":"2024-05-30T16:58:30.216284+0000","last_peered":"2024-05-30T16:58:30.216284+0000","last_clean":"2024-05-30T16:58:30.216284+0000","last_became_active":"2024-05-30T16:58:29.141432+0000","last_became_peered":"2024-05-30T16:58:29.141432+0000","last_unstale":"2024-05-30T16:58:30.216284+0000","last_undegraded":"2024-05-30T16:58:30.216284+0000","last_fullsized":"2024-05-30T16:58:30.216284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:09.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.148985+0000","last_change":"2024-05-30T16:58:29.147576+0000","last_active":"2024-05-30T16:58:29.148985+0000","last_peered":"2024-05-30T16:58:29.148985+0000","last_clean":"2024-05-30T16:58:29.148985+0000","last_became_active":"2024-05-30T16:58:29.146398+0000","last_became_peered":"2024-05-30T16:58:29.146398+0000","last_unstale":"2024-05-30T16:58:29.148985+0000","last_undegraded":"2024-05-30T16:58:29.148985+0000","last_fullsized":"2024-05-30T16:58:29.148985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:17.147362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127708+0000","last_change":"2024-05-30T16:58:29.139475+0000","last_active":"2024-05-30T16:58:30.127708+0000","last_peered":"2024-05-30T16:58:30.127708+0000","last_clean":"2024-05-30T16:58:30.127708+0000","last_became_active":"2024-05-30T16:58:29.139069+0000","last_became_peered":"2024-05-30T16:58:29.139069+0000","last_unstale":"2024-05-30T16:58:30.127708+0000","last_undegraded":"2024-05-30T16:58:30.127708+0000","last_fullsized":"2024-05-30T16:58:30.127708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:10:56.904673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.135616+0000","last_change":"2024-05-30T16:58:29.132440+0000","last_active":"2024-05-30T16:58:29.135616+0000","last_peered":"2024-05-30T16:58:29.135616+0000","last_clean":"2024-05-30T16:58:29.135616+0000","last_became_active":"2024-05-30T16:58:29.132102+0000","last_became_peered":"2024-05-30T16:58:29.132102+0000","last_unstale":"2024-05-30T16:58:29.135616+0000","last_undegraded":"2024-05-30T16:58:29.135616+0000","last_fullsized":"2024-05-30T16:58:29.135616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:08:50.542702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.131863+0000","last_change":"2024-05-30T16:58:29.129581+0000","last_active":"2024-05-30T16:58:29.131863+0000","last_peered":"2024-05-30T16:58:29.131863+0000","last_clean":"2024-05-30T16:58:29.131863+0000","last_became_active":"2024-05-30T16:58:29.126155+0000","last_became_peered":"2024-05-30T16:58:29.126155+0000","last_unstale":"2024-05-30T16:58:29.131863+0000","last_undegraded":"2024-05-30T16:58:29.131863+0000","last_fullsized":"2024-05-30T16:58:29.131863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:54:32.272063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173936+0000","last_change":"2024-05-30T16:58:29.127811+0000","last_active":"2024-05-30T16:58:30.173936+0000","last_peered":"2024-05-30T16:58:30.173936+0000","last_clean":"2024-05-30T16:58:30.173936+0000","last_became_active":"2024-05-30T16:58:29.126934+0000","last_became_peered":"2024-05-30T16:58:29.126934+0000","last_unstale":"2024-05-30T16:58:30.173936+0000","last_undegraded":"2024-05-30T16:58:30.173936+0000","last_fullsized":"2024-05-30T16:58:30.173936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:02.038829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142452+0000","last_change":"2024-05-30T16:58:29.138717+0000","last_active":"2024-05-30T16:58:30.142452+0000","last_peered":"2024-05-30T16:58:30.142452+0000","last_clean":"2024-05-30T16:58:30.142452+0000","last_became_active":"2024-05-30T16:58:29.132811+0000","last_became_peered":"2024-05-30T16:58:29.132811+0000","last_unstale":"2024-05-30T16:58:30.142452+0000","last_undegraded":"2024-05-30T16:58:30.142452+0000","last_fullsized":"2024-05-30T16:58:30.142452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:18.832238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173109+0000","last_change":"2024-05-30T16:58:29.132568+0000","last_active":"2024-05-30T16:58:30.173109+0000","last_peered":"2024-05-30T16:58:30.173109+0000","last_clean":"2024-05-30T16:58:30.173109+0000","last_became_active":"2024-05-30T16:58:29.130928+0000","last_became_peered":"2024-05-30T16:58:29.130928+0000","last_unstale":"2024-05-30T16:58:30.173109+0000","last_undegraded":"2024-05-30T16:58:30.173109+0000","last_fullsized":"2024-05-30T16:58:30.173109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:09.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.136139+0000","last_change":"2024-05-30T16:58:29.127628+0000","last_active":"2024-05-30T16:58:29.136139+0000","last_peered":"2024-05-30T16:58:29.136139+0000","last_clean":"2024-05-30T16:58:29.136139+0000","last_became_active":"2024-05-30T16:58:29.127101+0000","last_became_peered":"2024-05-30T16:58:29.127101+0000","last_unstale":"2024-05-30T16:58:29.136139+0000","last_undegraded":"2024-05-30T16:58:29.136139+0000","last_fullsized":"2024-05-30T16:58:29.136139+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:44.398093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128254+0000","last_change":"2024-05-30T16:58:29.146655+0000","last_active":"2024-05-30T16:58:30.128254+0000","last_peered":"2024-05-30T16:58:30.128254+0000","last_clean":"2024-05-30T16:58:30.128254+0000","last_became_active":"2024-05-30T16:58:29.145851+0000","last_became_peered":"2024-05-30T16:58:29.145851+0000","last_unstale":"2024-05-30T16:58:30.128254+0000","last_undegraded":"2024-05-30T16:58:30.128254+0000","last_fullsized":"2024-05-30T16:58:30.128254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:50.575170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.124364+0000","last_change":"2024-05-30T16:58:29.125194+0000","last_active":"2024-05-30T16:58:30.124364+0000","last_peered":"2024-05-30T16:58:30.124364+0000","last_clean":"2024-05-30T16:58:30.124364+0000","last_became_active":"2024-05-30T16:58:29.119557+0000","last_became_peered":"2024-05-30T16:58:29.119557+0000","last_unstale":"2024-05-30T16:58:30.124364+0000","last_undegraded":"2024-05-30T16:58:30.124364+0000","last_fullsized":"2024-05-30T16:58:30.124364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:05.947318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.215865+0000","last_change":"2024-05-30T16:58:29.132349+0000","last_active":"2024-05-30T16:58:30.215865+0000","last_peered":"2024-05-30T16:58:30.215865+0000","last_clean":"2024-05-30T16:58:30.215865+0000","last_became_active":"2024-05-30T16:58:29.132085+0000","last_became_peered":"2024-05-30T16:58:29.132085+0000","last_unstale":"2024-05-30T16:58:30.215865+0000","last_undegraded":"2024-05-30T16:58:30.215865+0000","last_fullsized":"2024-05-30T16:58:30.215865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:35.736510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.217573+0000","last_change":"2024-05-30T16:58:29.146119+0000","last_active":"2024-05-30T16:58:30.217573+0000","last_peered":"2024-05-30T16:58:30.217573+0000","last_clean":"2024-05-30T16:58:30.217573+0000","last_became_active":"2024-05-30T16:58:29.145798+0000","last_became_peered":"2024-05-30T16:58:29.145798+0000","last_unstale":"2024-05-30T16:58:30.217573+0000","last_undegraded":"2024-05-30T16:58:30.217573+0000","last_fullsized":"2024-05-30T16:58:30.217573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:22.544018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128218+0000","last_change":"2024-05-30T16:58:29.135098+0000","last_active":"2024-05-30T16:58:30.128218+0000","last_peered":"2024-05-30T16:58:30.128218+0000","last_clean":"2024-05-30T16:58:30.128218+0000","last_became_active":"2024-05-30T16:58:29.134804+0000","last_became_peered":"2024-05-30T16:58:29.134804+0000","last_unstale":"2024-05-30T16:58:30.128218+0000","last_undegraded":"2024-05-30T16:58:30.128218+0000","last_fullsized":"2024-05-30T16:58:30.128218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:12.142920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.139319+0000","last_change":"2024-05-30T16:58:29.132473+0000","last_active":"2024-05-30T16:58:29.139319+0000","last_peered":"2024-05-30T16:58:29.139319+0000","last_clean":"2024-05-30T16:58:29.139319+0000","last_became_active":"2024-05-30T16:58:29.132174+0000","last_became_peered":"2024-05-30T16:58:29.132174+0000","last_unstale":"2024-05-30T16:58:29.139319+0000","last_undegraded":"2024-05-30T16:58:29.139319+0000","last_fullsized":"2024-05-30T16:58:29.139319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:36:37.836774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128475+0000","last_change":"2024-05-30T16:58:29.143185+0000","last_active":"2024-05-30T16:58:30.128475+0000","last_peered":"2024-05-30T16:58:30.128475+0000","last_clean":"2024-05-30T16:58:30.128475+0000","last_became_active":"2024-05-30T16:58:29.142896+0000","last_became_peered":"2024-05-30T16:58:29.142896+0000","last_unstale":"2024-05-30T16:58:30.128475+0000","last_undegraded":"2024-05-30T16:58:30.128475+0000","last_fullsized":"2024-05-30T16:58:30.128475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:34.754725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127756+0000","last_change":"2024-05-30T16:58:29.143222+0000","last_active":"2024-05-30T16:58:30.127756+0000","last_peered":"2024-05-30T16:58:30.127756+0000","last_clean":"2024-05-30T16:58:30.127756+0000","last_became_active":"2024-05-30T16:58:29.142927+0000","last_became_peered":"2024-05-30T16:58:29.142927+0000","last_unstale":"2024-05-30T16:58:30.127756+0000","last_undegraded":"2024-05-30T16:58:30.127756+0000","last_fullsized":"2024-05-30T16:58:30.127756+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:45:51.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141815+0000","last_change":"2024-05-30T16:58:29.145994+0000","last_active":"2024-05-30T16:58:30.141815+0000","last_peered":"2024-05-30T16:58:30.141815+0000","last_clean":"2024-05-30T16:58:30.141815+0000","last_became_active":"2024-05-30T16:58:29.145634+0000","last_became_peered":"2024-05-30T16:58:29.145634+0000","last_unstale":"2024-05-30T16:58:30.141815+0000","last_undegraded":"2024-05-30T16:58:30.141815+0000","last_fullsized":"2024-05-30T16:58:30.141815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:50.575170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173257+0000","last_change":"2024-05-30T16:58:29.130507+0000","last_active":"2024-05-30T16:58:30.173257+0000","last_peered":"2024-05-30T16:58:30.173257+0000","last_clean":"2024-05-30T16:58:30.173257+0000","last_became_active":"2024-05-30T16:58:29.130178+0000","last_became_peered":"2024-05-30T16:58:29.130178+0000","last_unstale":"2024-05-30T16:58:30.173257+0000","last_undegraded":"2024-05-30T16:58:30.173257+0000","last_fullsized":"2024-05-30T16:58:30.173257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:56:26.493747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127616+0000","last_change":"2024-05-30T16:58:29.136667+0000","last_active":"2024-05-30T16:58:30.127616+0000","last_peered":"2024-05-30T16:58:30.127616+0000","last_clean":"2024-05-30T16:58:30.127616+0000","last_became_active":"2024-05-30T16:58:29.135160+0000","last_became_peered":"2024-05-30T16:58:29.135160+0000","last_unstale":"2024-05-30T16:58:30.127616+0000","last_undegraded":"2024-05-30T16:58:30.127616+0000","last_fullsized":"2024-05-30T16:58:30.127616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:53.332755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142768+0000","last_change":"2024-05-30T16:58:29.137037+0000","last_active":"2024-05-30T16:58:30.142768+0000","last_peered":"2024-05-30T16:58:30.142768+0000","last_clean":"2024-05-30T16:58:30.142768+0000","last_became_active":"2024-05-30T16:58:29.136760+0000","last_became_peered":"2024-05-30T16:58:29.136760+0000","last_unstale":"2024-05-30T16:58:30.142768+0000","last_undegraded":"2024-05-30T16:58:30.142768+0000","last_fullsized":"2024-05-30T16:58:30.142768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:11:01.264686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.142596+0000","last_change":"2024-05-30T16:58:29.140538+0000","last_active":"2024-05-30T16:58:29.142596+0000","last_peered":"2024-05-30T16:58:29.142596+0000","last_clean":"2024-05-30T16:58:29.142596+0000","last_became_active":"2024-05-30T16:58:29.140052+0000","last_became_peered":"2024-05-30T16:58:29.140052+0000","last_unstale":"2024-05-30T16:58:29.142596+0000","last_undegraded":"2024-05-30T16:58:29.142596+0000","last_fullsized":"2024-05-30T16:58:29.142596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:10.807493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.215931+0000","last_change":"2024-05-30T16:58:29.137903+0000","last_active":"2024-05-30T16:58:30.215931+0000","last_peered":"2024-05-30T16:58:30.215931+0000","last_clean":"2024-05-30T16:58:30.215931+0000","last_became_active":"2024-05-30T16:58:29.135438+0000","last_became_peered":"2024-05-30T16:58:29.135438+0000","last_unstale":"2024-05-30T16:58:30.215931+0000","last_undegraded":"2024-05-30T16:58:30.215931+0000","last_fullsized":"2024-05-30T16:58:30.215931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:54:46.848424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173592+0000","last_change":"2024-05-30T16:58:29.132320+0000","last_active":"2024-05-30T16:58:30.173592+0000","last_peered":"2024-05-30T16:58:30.173592+0000","last_clean":"2024-05-30T16:58:30.173592+0000","last_became_active":"2024-05-30T16:58:29.132037+0000","last_became_peered":"2024-05-30T16:58:29.132037+0000","last_unstale":"2024-05-30T16:58:30.173592+0000","last_undegraded":"2024-05-30T16:58:30.173592+0000","last_fullsized":"2024-05-30T16:58:30.173592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:22.544018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141751+0000","last_change":"2024-05-30T16:58:29.135169+0000","last_active":"2024-05-30T16:58:30.141751+0000","last_peered":"2024-05-30T16:58:30.141751+0000","last_clean":"2024-05-30T16:58:30.141751+0000","last_became_active":"2024-05-30T16:58:29.134724+0000","last_became_peered":"2024-05-30T16:58:29.134724+0000","last_unstale":"2024-05-30T16:58:30.141751+0000","last_undegraded":"2024-05-30T16:58:30.141751+0000","last_fullsized":"2024-05-30T16:58:30.141751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:12.142920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141995+0000","last_change":"2024-05-30T16:58:29.147599+0000","last_active":"2024-05-30T16:58:30.141995+0000","last_peered":"2024-05-30T16:58:30.141995+0000","last_clean":"2024-05-30T16:58:30.141995+0000","last_became_active":"2024-05-30T16:58:29.147146+0000","last_became_peered":"2024-05-30T16:58:29.147146+0000","last_unstale":"2024-05-30T16:58:30.141995+0000","last_undegraded":"2024-05-30T16:58:30.141995+0000","last_fullsized":"2024-05-30T16:58:30.141995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:34.754725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216206+0000","last_change":"2024-05-30T16:58:29.131709+0000","last_active":"2024-05-30T16:58:30.216206+0000","last_peered":"2024-05-30T16:58:30.216206+0000","last_clean":"2024-05-30T16:58:30.216206+0000","last_became_active":"2024-05-30T16:58:29.131345+0000","last_became_peered":"2024-05-30T16:58:29.131345+0000","last_unstale":"2024-05-30T16:58:30.216206+0000","last_undegraded":"2024-05-30T16:58:30.216206+0000","last_fullsized":"2024-05-30T16:58:30.216206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:02:19.809216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173691+0000","last_change":"2024-05-30T16:58:29.137413+0000","last_active":"2024-05-30T16:58:30.173691+0000","last_peered":"2024-05-30T16:58:30.173691+0000","last_clean":"2024-05-30T16:58:30.173691+0000","last_became_active":"2024-05-30T16:58:29.137101+0000","last_became_peered":"2024-05-30T16:58:29.137101+0000","last_unstale":"2024-05-30T16:58:30.173691+0000","last_undegraded":"2024-05-30T16:58:30.173691+0000","last_fullsized":"2024-05-30T16:58:30.173691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:17.255770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.217703+0000","last_change":"2024-05-30T16:58:29.131881+0000","last_active":"2024-05-30T16:58:30.217703+0000","last_peered":"2024-05-30T16:58:30.217703+0000","last_clean":"2024-05-30T16:58:30.217703+0000","last_became_active":"2024-05-30T16:58:29.131511+0000","last_became_peered":"2024-05-30T16:58:29.131511+0000","last_unstale":"2024-05-30T16:58:30.217703+0000","last_undegraded":"2024-05-30T16:58:30.217703+0000","last_fullsized":"2024-05-30T16:58:30.217703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:44.238425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174143+0000","last_change":"2024-05-30T16:58:28.108634+0000","last_active":"2024-05-30T16:58:30.174143+0000","last_peered":"2024-05-30T16:58:30.174143+0000","last_clean":"2024-05-30T16:58:30.174143+0000","last_became_active":"2024-05-30T16:58:28.108356+0000","last_became_peered":"2024-05-30T16:58:28.108356+0000","last_unstale":"2024-05-30T16:58:30.174143+0000","last_undegraded":"2024-05-30T16:58:30.174143+0000","last_fullsized":"2024-05-30T16:58:30.174143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:44.065918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173977+0000","last_change":"2024-05-30T16:58:29.133335+0000","last_active":"2024-05-30T16:58:30.173977+0000","last_peered":"2024-05-30T16:58:30.173977+0000","last_clean":"2024-05-30T16:58:30.173977+0000","last_became_active":"2024-05-30T16:58:29.130543+0000","last_became_peered":"2024-05-30T16:58:29.130543+0000","last_unstale":"2024-05-30T16:58:30.173977+0000","last_undegraded":"2024-05-30T16:58:30.173977+0000","last_fullsized":"2024-05-30T16:58:30.173977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:59.208770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.217777+0000","last_change":"2024-05-30T16:58:28.108639+0000","last_active":"2024-05-30T16:58:30.217777+0000","last_peered":"2024-05-30T16:58:30.217777+0000","last_clean":"2024-05-30T16:58:30.217777+0000","last_became_active":"2024-05-30T16:58:28.106892+0000","last_became_peered":"2024-05-30T16:58:28.106892+0000","last_unstale":"2024-05-30T16:58:30.217777+0000","last_undegraded":"2024-05-30T16:58:30.217777+0000","last_fullsized":"2024-05-30T16:58:30.217777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:15:24.374426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173489+0000","last_change":"2024-05-30T16:58:29.140924+0000","last_active":"2024-05-30T16:58:30.173489+0000","last_peered":"2024-05-30T16:58:30.173489+0000","last_clean":"2024-05-30T16:58:30.173489+0000","last_became_active":"2024-05-30T16:58:29.140535+0000","last_became_peered":"2024-05-30T16:58:29.140535+0000","last_unstale":"2024-05-30T16:58:30.173489+0000","last_undegraded":"2024-05-30T16:58:30.173489+0000","last_fullsized":"2024-05-30T16:58:30.173489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:22.645412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.155860+0000","last_change":"2024-05-30T16:58:28.105704+0000","last_active":"2024-05-30T16:58:29.155860+0000","last_peered":"2024-05-30T16:58:29.155860+0000","last_clean":"2024-05-30T16:58:29.155860+0000","last_became_active":"2024-05-30T16:58:28.104703+0000","last_became_peered":"2024-05-30T16:58:28.104703+0000","last_unstale":"2024-05-30T16:58:29.155860+0000","last_undegraded":"2024-05-30T16:58:29.155860+0000","last_fullsized":"2024-05-30T16:58:29.155860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:17:11.646160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.142641+0000","last_change":"2024-05-30T16:58:29.140789+0000","last_active":"2024-05-30T16:58:29.142641+0000","last_peered":"2024-05-30T16:58:29.142641+0000","last_clean":"2024-05-30T16:58:29.142641+0000","last_became_active":"2024-05-30T16:58:29.139977+0000","last_became_peered":"2024-05-30T16:58:29.139977+0000","last_unstale":"2024-05-30T16:58:29.142641+0000","last_undegraded":"2024-05-30T16:58:29.142641+0000","last_fullsized":"2024-05-30T16:58:29.142641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:31:43.364677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.156258+0000","last_change":"2024-05-30T16:58:28.108974+0000","last_active":"2024-05-30T16:58:29.156258+0000","last_peered":"2024-05-30T16:58:29.156258+0000","last_clean":"2024-05-30T16:58:29.156258+0000","last_became_active":"2024-05-30T16:58:28.108604+0000","last_became_peered":"2024-05-30T16:58:28.108604+0000","last_unstale":"2024-05-30T16:58:29.156258+0000","last_undegraded":"2024-05-30T16:58:29.156258+0000","last_fullsized":"2024-05-30T16:58:29.156258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:31.549762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142086+0000","last_change":"2024-05-30T16:58:29.146801+0000","last_active":"2024-05-30T16:58:30.142086+0000","last_peered":"2024-05-30T16:58:30.142086+0000","last_clean":"2024-05-30T16:58:30.142086+0000","last_became_active":"2024-05-30T16:58:29.146517+0000","last_became_peered":"2024-05-30T16:58:29.146517+0000","last_unstale":"2024-05-30T16:58:30.142086+0000","last_undegraded":"2024-05-30T16:58:30.142086+0000","last_fullsized":"2024-05-30T16:58:30.142086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:17:41.362206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216600+0000","last_change":"2024-05-30T16:58:28.104793+0000","last_active":"2024-05-30T16:58:30.216600+0000","last_peered":"2024-05-30T16:58:30.216600+0000","last_clean":"2024-05-30T16:58:30.216600+0000","last_became_active":"2024-05-30T16:58:28.104138+0000","last_became_peered":"2024-05-30T16:58:28.104138+0000","last_unstale":"2024-05-30T16:58:30.216600+0000","last_undegraded":"2024-05-30T16:58:30.216600+0000","last_fullsized":"2024-05-30T16:58:30.216600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:00:46.771796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216889+0000","last_change":"2024-05-30T16:58:29.141417+0000","last_active":"2024-05-30T16:58:30.216889+0000","last_peered":"2024-05-30T16:58:30.216889+0000","last_clean":"2024-05-30T16:58:30.216889+0000","last_became_active":"2024-05-30T16:58:29.140692+0000","last_became_peered":"2024-05-30T16:58:29.140692+0000","last_unstale":"2024-05-30T16:58:30.216889+0000","last_undegraded":"2024-05-30T16:58:30.216889+0000","last_fullsized":"2024-05-30T16:58:30.216889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:22.645412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.215780+0000","last_change":"2024-05-30T16:58:28.109377+0000","last_active":"2024-05-30T16:58:30.215780+0000","last_peered":"2024-05-30T16:58:30.215780+0000","last_clean":"2024-05-30T16:58:30.215780+0000","last_became_active":"2024-05-30T16:58:28.108720+0000","last_became_peered":"2024-05-30T16:58:28.108720+0000","last_unstale":"2024-05-30T16:58:30.215780+0000","last_undegraded":"2024-05-30T16:58:30.215780+0000","last_fullsized":"2024-05-30T16:58:30.215780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:43:10.868544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.137783+0000","last_change":"2024-05-30T16:58:29.132392+0000","last_active":"2024-05-30T16:58:29.137783+0000","last_peered":"2024-05-30T16:58:29.137783+0000","last_clean":"2024-05-30T16:58:29.137783+0000","last_became_active":"2024-05-30T16:58:29.132083+0000","last_became_peered":"2024-05-30T16:58:29.132083+0000","last_unstale":"2024-05-30T16:58:29.137783+0000","last_undegraded":"2024-05-30T16:58:29.137783+0000","last_fullsized":"2024-05-30T16:58:29.137783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:18:36.476929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.127119+0000","last_change":"2024-05-30T16:58:28.108001+0000","last_active":"2024-05-30T16:58:29.127119+0000","last_peered":"2024-05-30T16:58:29.127119+0000","last_clean":"2024-05-30T16:58:29.127119+0000","last_became_active":"2024-05-30T16:58:28.107703+0000","last_became_peered":"2024-05-30T16:58:28.107703+0000","last_unstale":"2024-05-30T16:58:29.127119+0000","last_undegraded":"2024-05-30T16:58:29.127119+0000","last_fullsized":"2024-05-30T16:58:29.127119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:36.468903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.124294+0000","last_change":"2024-05-30T16:58:29.120318+0000","last_active":"2024-05-30T16:58:30.124294+0000","last_peered":"2024-05-30T16:58:30.124294+0000","last_clean":"2024-05-30T16:58:30.124294+0000","last_became_active":"2024-05-30T16:58:29.119920+0000","last_became_peered":"2024-05-30T16:58:29.119920+0000","last_unstale":"2024-05-30T16:58:30.124294+0000","last_undegraded":"2024-05-30T16:58:30.124294+0000","last_fullsized":"2024-05-30T16:58:30.124294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:16.700840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142186+0000","last_change":"2024-05-30T16:58:28.107649+0000","last_active":"2024-05-30T16:58:30.142186+0000","last_peered":"2024-05-30T16:58:30.142186+0000","last_clean":"2024-05-30T16:58:30.142186+0000","last_became_active":"2024-05-30T16:58:28.105597+0000","last_became_peered":"2024-05-30T16:58:28.105597+0000","last_unstale":"2024-05-30T16:58:30.142186+0000","last_undegraded":"2024-05-30T16:58:30.142186+0000","last_fullsized":"2024-05-30T16:58:30.142186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:56.153715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.125643+0000","last_change":"2024-05-30T16:58:29.136646+0000","last_active":"2024-05-30T16:58:30.125643+0000","last_peered":"2024-05-30T16:58:30.125643+0000","last_clean":"2024-05-30T16:58:30.125643+0000","last_became_active":"2024-05-30T16:58:29.136370+0000","last_became_peered":"2024-05-30T16:58:29.136370+0000","last_unstale":"2024-05-30T16:58:30.125643+0000","last_undegraded":"2024-05-30T16:58:30.125643+0000","last_fullsized":"2024-05-30T16:58:30.125643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:28.798742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174310+0000","last_change":"2024-05-30T16:58:28.099840+0000","last_active":"2024-05-30T16:58:30.174310+0000","last_peered":"2024-05-30T16:58:30.174310+0000","last_clean":"2024-05-30T16:58:30.174310+0000","last_became_active":"2024-05-30T16:58:28.099453+0000","last_became_peered":"2024-05-30T16:58:28.099453+0000","last_unstale":"2024-05-30T16:58:30.174310+0000","last_undegraded":"2024-05-30T16:58:30.174310+0000","last_fullsized":"2024-05-30T16:58:30.174310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:29:57.171394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.148518+0000","last_change":"2024-05-30T16:58:29.145237+0000","last_active":"2024-05-30T16:58:29.148518+0000","last_peered":"2024-05-30T16:58:29.148518+0000","last_clean":"2024-05-30T16:58:29.148518+0000","last_became_active":"2024-05-30T16:58:29.144954+0000","last_became_peered":"2024-05-30T16:58:29.144954+0000","last_unstale":"2024-05-30T16:58:29.148518+0000","last_undegraded":"2024-05-30T16:58:29.148518+0000","last_fullsized":"2024-05-30T16:58:29.148518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:31:09.043735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174004+0000","last_change":"2024-05-30T16:58:28.102851+0000","last_active":"2024-05-30T16:58:30.174004+0000","last_peered":"2024-05-30T16:58:30.174004+0000","last_clean":"2024-05-30T16:58:30.174004+0000","last_became_active":"2024-05-30T16:58:28.102547+0000","last_became_peered":"2024-05-30T16:58:28.102547+0000","last_unstale":"2024-05-30T16:58:30.174004+0000","last_undegraded":"2024-05-30T16:58:30.174004+0000","last_fullsized":"2024-05-30T16:58:30.174004+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:00:46.771796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.147515+0000","last_change":"2024-05-30T16:58:29.137538+0000","last_active":"2024-05-30T16:58:29.147515+0000","last_peered":"2024-05-30T16:58:29.147515+0000","last_clean":"2024-05-30T16:58:29.147515+0000","last_became_active":"2024-05-30T16:58:29.137227+0000","last_became_peered":"2024-05-30T16:58:29.137227+0000","last_unstale":"2024-05-30T16:58:29.147515+0000","last_undegraded":"2024-05-30T16:58:29.147515+0000","last_fullsized":"2024-05-30T16:58:29.147515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:18:36.476929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173172+0000","last_change":"2024-05-30T16:58:28.099059+0000","last_active":"2024-05-30T16:58:30.173172+0000","last_peered":"2024-05-30T16:58:30.173172+0000","last_clean":"2024-05-30T16:58:30.173172+0000","last_became_active":"2024-05-30T16:58:28.098711+0000","last_became_peered":"2024-05-30T16:58:28.098711+0000","last_unstale":"2024-05-30T16:58:30.173172+0000","last_undegraded":"2024-05-30T16:58:30.173172+0000","last_fullsized":"2024-05-30T16:58:30.173172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:14.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216162+0000","last_change":"2024-05-30T16:58:29.135114+0000","last_active":"2024-05-30T16:58:30.216162+0000","last_peered":"2024-05-30T16:58:30.216162+0000","last_clean":"2024-05-30T16:58:30.216162+0000","last_became_active":"2024-05-30T16:58:29.134827+0000","last_became_peered":"2024-05-30T16:58:29.134827+0000","last_unstale":"2024-05-30T16:58:30.216162+0000","last_undegraded":"2024-05-30T16:58:30.216162+0000","last_fullsized":"2024-05-30T16:58:30.216162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:58:16.952470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.157024+0000","last_change":"2024-05-30T16:58:28.110149+0000","last_active":"2024-05-30T16:58:29.157024+0000","last_peered":"2024-05-30T16:58:29.157024+0000","last_clean":"2024-05-30T16:58:29.157024+0000","last_became_active":"2024-05-30T16:58:28.109562+0000","last_became_peered":"2024-05-30T16:58:28.109562+0000","last_unstale":"2024-05-30T16:58:29.157024+0000","last_undegraded":"2024-05-30T16:58:29.157024+0000","last_fullsized":"2024-05-30T16:58:29.157024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:47.986563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.144938+0000","last_change":"2024-05-30T16:58:29.134694+0000","last_active":"2024-05-30T16:58:29.144938+0000","last_peered":"2024-05-30T16:58:29.144938+0000","last_clean":"2024-05-30T16:58:29.144938+0000","last_became_active":"2024-05-30T16:58:29.129590+0000","last_became_peered":"2024-05-30T16:58:29.129590+0000","last_unstale":"2024-05-30T16:58:29.144938+0000","last_undegraded":"2024-05-30T16:58:29.144938+0000","last_fullsized":"2024-05-30T16:58:29.144938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:57.354731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141960+0000","last_change":"2024-05-30T16:58:28.111410+0000","last_active":"2024-05-30T16:58:30.141960+0000","last_peered":"2024-05-30T16:58:30.141960+0000","last_clean":"2024-05-30T16:58:30.141960+0000","last_became_active":"2024-05-30T16:58:28.111098+0000","last_became_peered":"2024-05-30T16:58:28.111098+0000","last_unstale":"2024-05-30T16:58:30.141960+0000","last_undegraded":"2024-05-30T16:58:30.141960+0000","last_fullsized":"2024-05-30T16:58:30.141960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:02.958884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.136115+0000","last_change":"2024-05-30T16:58:29.127379+0000","last_active":"2024-05-30T16:58:29.136115+0000","last_peered":"2024-05-30T16:58:29.136115+0000","last_clean":"2024-05-30T16:58:29.136115+0000","last_became_active":"2024-05-30T16:58:29.126691+0000","last_became_peered":"2024-05-30T16:58:29.126691+0000","last_unstale":"2024-05-30T16:58:29.136115+0000","last_undegraded":"2024-05-30T16:58:29.136115+0000","last_fullsized":"2024-05-30T16:58:29.136115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:36:37.836774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.124881+0000","last_change":"2024-05-30T16:58:28.102571+0000","last_active":"2024-05-30T16:58:30.124881+0000","last_peered":"2024-05-30T16:58:30.124881+0000","last_clean":"2024-05-30T16:58:30.124881+0000","last_became_active":"2024-05-30T16:58:28.097342+0000","last_became_peered":"2024-05-30T16:58:28.097342+0000","last_unstale":"2024-05-30T16:58:30.124881+0000","last_undegraded":"2024-05-30T16:58:30.124881+0000","last_fullsized":"2024-05-30T16:58:30.124881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:32:17.040752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141900+0000","last_change":"2024-05-30T16:58:29.135084+0000","last_active":"2024-05-30T16:58:30.141900+0000","last_peered":"2024-05-30T16:58:30.141900+0000","last_clean":"2024-05-30T16:58:30.141900+0000","last_became_active":"2024-05-30T16:58:29.134733+0000","last_became_peered":"2024-05-30T16:58:29.134733+0000","last_unstale":"2024-05-30T16:58:30.141900+0000","last_undegraded":"2024-05-30T16:58:30.141900+0000","last_fullsized":"2024-05-30T16:58:30.141900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:43.227674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.107862+0000","last_change":"2024-05-30T16:58:28.107855+0000","last_active":"2024-05-30T16:58:29.107862+0000","last_peered":"2024-05-30T16:58:29.107862+0000","last_clean":"2024-05-30T16:58:29.107862+0000","last_became_active":"2024-05-30T16:58:28.106737+0000","last_became_peered":"2024-05-30T16:58:28.106737+0000","last_unstale":"2024-05-30T16:58:29.107862+0000","last_undegraded":"2024-05-30T16:58:29.107862+0000","last_fullsized":"2024-05-30T16:58:29.107862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:59.119587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141681+0000","last_change":"2024-05-30T16:58:29.132469+0000","last_active":"2024-05-30T16:58:30.141681+0000","last_peered":"2024-05-30T16:58:30.141681+0000","last_clean":"2024-05-30T16:58:30.141681+0000","last_became_active":"2024-05-30T16:58:29.132137+0000","last_became_peered":"2024-05-30T16:58:29.132137+0000","last_unstale":"2024-05-30T16:58:30.141681+0000","last_undegraded":"2024-05-30T16:58:30.141681+0000","last_fullsized":"2024-05-30T16:58:30.141681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:25.159552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.159131+0000","last_change":"2024-05-30T16:58:28.107196+0000","last_active":"2024-05-30T16:58:29.159131+0000","last_peered":"2024-05-30T16:58:29.159131+0000","last_clean":"2024-05-30T16:58:29.159131+0000","last_became_active":"2024-05-30T16:58:28.102125+0000","last_became_peered":"2024-05-30T16:58:28.102125+0000","last_unstale":"2024-05-30T16:58:29.159131+0000","last_undegraded":"2024-05-30T16:58:29.159131+0000","last_fullsized":"2024-05-30T16:58:29.159131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:31.419109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.122143+0000","last_change":"2024-05-30T16:58:29.118801+0000","last_active":"2024-05-30T16:58:29.122143+0000","last_peered":"2024-05-30T16:58:29.122143+0000","last_clean":"2024-05-30T16:58:29.122143+0000","last_became_active":"2024-05-30T16:58:29.116528+0000","last_became_peered":"2024-05-30T16:58:29.116528+0000","last_unstale":"2024-05-30T16:58:29.122143+0000","last_undegraded":"2024-05-30T16:58:29.122143+0000","last_fullsized":"2024-05-30T16:58:29.122143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:17:12.660722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174405+0000","last_change":"2024-05-30T16:58:28.106524+0000","last_active":"2024-05-30T16:58:30.174405+0000","last_peered":"2024-05-30T16:58:30.174405+0000","last_clean":"2024-05-30T16:58:30.174405+0000","last_became_active":"2024-05-30T16:58:28.105165+0000","last_became_peered":"2024-05-30T16:58:28.105165+0000","last_unstale":"2024-05-30T16:58:30.174405+0000","last_undegraded":"2024-05-30T16:58:30.174405+0000","last_fullsized":"2024-05-30T16:58:30.174405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:05:12.760513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174149+0000","last_change":"2024-05-30T16:58:29.128564+0000","last_active":"2024-05-30T16:58:30.174149+0000","last_peered":"2024-05-30T16:58:30.174149+0000","last_clean":"2024-05-30T16:58:30.174149+0000","last_became_active":"2024-05-30T16:58:29.128258+0000","last_became_peered":"2024-05-30T16:58:29.128258+0000","last_unstale":"2024-05-30T16:58:30.174149+0000","last_undegraded":"2024-05-30T16:58:30.174149+0000","last_fullsized":"2024-05-30T16:58:30.174149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:40:44.960616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.156522+0000","last_change":"2024-05-30T16:58:28.111737+0000","last_active":"2024-05-30T16:58:29.156522+0000","last_peered":"2024-05-30T16:58:29.156522+0000","last_clean":"2024-05-30T16:58:29.156522+0000","last_became_active":"2024-05-30T16:58:28.109345+0000","last_became_peered":"2024-05-30T16:58:28.109345+0000","last_unstale":"2024-05-30T16:58:29.156522+0000","last_undegraded":"2024-05-30T16:58:29.156522+0000","last_fullsized":"2024-05-30T16:58:29.156522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:50.840179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216407+0000","last_change":"2024-05-30T16:58:29.142398+0000","last_active":"2024-05-30T16:58:30.216407+0000","last_peered":"2024-05-30T16:58:30.216407+0000","last_clean":"2024-05-30T16:58:30.216407+0000","last_became_active":"2024-05-30T16:58:29.142103+0000","last_became_peered":"2024-05-30T16:58:29.142103+0000","last_unstale":"2024-05-30T16:58:30.216407+0000","last_undegraded":"2024-05-30T16:58:30.216407+0000","last_fullsized":"2024-05-30T16:58:30.216407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:32.109950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.130579+0000","last_change":"2024-05-30T16:58:28.106755+0000","last_active":"2024-05-30T16:58:29.130579+0000","last_peered":"2024-05-30T16:58:29.130579+0000","last_clean":"2024-05-30T16:58:29.130579+0000","last_became_active":"2024-05-30T16:58:28.106425+0000","last_became_peered":"2024-05-30T16:58:28.106425+0000","last_unstale":"2024-05-30T16:58:29.130579+0000","last_undegraded":"2024-05-30T16:58:29.130579+0000","last_fullsized":"2024-05-30T16:58:29.130579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:50:27.592384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128345+0000","last_change":"2024-05-30T16:58:29.145344+0000","last_active":"2024-05-30T16:58:30.128345+0000","last_peered":"2024-05-30T16:58:30.128345+0000","last_clean":"2024-05-30T16:58:30.128345+0000","last_became_active":"2024-05-30T16:58:29.142645+0000","last_became_peered":"2024-05-30T16:58:29.142645+0000","last_unstale":"2024-05-30T16:58:30.128345+0000","last_undegraded":"2024-05-30T16:58:30.128345+0000","last_fullsized":"2024-05-30T16:58:30.128345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:43.227674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.124889+0000","last_change":"2024-05-30T16:58:28.105411+0000","last_active":"2024-05-30T16:58:30.124889+0000","last_peered":"2024-05-30T16:58:30.124889+0000","last_clean":"2024-05-30T16:58:30.124889+0000","last_became_active":"2024-05-30T16:58:28.105098+0000","last_became_peered":"2024-05-30T16:58:28.105098+0000","last_unstale":"2024-05-30T16:58:30.124889+0000","last_undegraded":"2024-05-30T16:58:30.124889+0000","last_fullsized":"2024-05-30T16:58:30.124889+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:44:17.323649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.128742+0000","last_change":"2024-05-30T16:58:29.120696+0000","last_active":"2024-05-30T16:58:29.128742+0000","last_peered":"2024-05-30T16:58:29.128742+0000","last_clean":"2024-05-30T16:58:29.128742+0000","last_became_active":"2024-05-30T16:58:29.120145+0000","last_became_peered":"2024-05-30T16:58:29.120145+0000","last_unstale":"2024-05-30T16:58:29.128742+0000","last_undegraded":"2024-05-30T16:58:29.128742+0000","last_fullsized":"2024-05-30T16:58:29.128742+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:10.642499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128536+0000","last_change":"2024-05-30T16:58:28.112852+0000","last_active":"2024-05-30T16:58:30.128536+0000","last_peered":"2024-05-30T16:58:30.128536+0000","last_clean":"2024-05-30T16:58:30.128536+0000","last_became_active":"2024-05-30T16:58:28.112553+0000","last_became_peered":"2024-05-30T16:58:28.112553+0000","last_unstale":"2024-05-30T16:58:30.128536+0000","last_undegraded":"2024-05-30T16:58:30.128536+0000","last_fullsized":"2024-05-30T16:58:30.128536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:14:08.346680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127338+0000","last_change":"2024-05-30T16:58:29.135993+0000","last_active":"2024-05-30T16:58:30.127338+0000","last_peered":"2024-05-30T16:58:30.127338+0000","last_clean":"2024-05-30T16:58:30.127338+0000","last_became_active":"2024-05-30T16:58:29.131946+0000","last_became_peered":"2024-05-30T16:58:29.131946+0000","last_unstale":"2024-05-30T16:58:30.127338+0000","last_undegraded":"2024-05-30T16:58:30.127338+0000","last_fullsized":"2024-05-30T16:58:30.127338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:49.132327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127500+0000","last_change":"2024-05-30T16:58:28.109200+0000","last_active":"2024-05-30T16:58:30.127500+0000","last_peered":"2024-05-30T16:58:30.127500+0000","last_clean":"2024-05-30T16:58:30.127500+0000","last_became_active":"2024-05-30T16:58:28.108518+0000","last_became_peered":"2024-05-30T16:58:28.108518+0000","last_unstale":"2024-05-30T16:58:30.127500+0000","last_undegraded":"2024-05-30T16:58:30.127500+0000","last_fullsized":"2024-05-30T16:58:30.127500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:02.958884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.132950+0000","last_change":"2024-05-30T16:58:29.129344+0000","last_active":"2024-05-30T16:58:29.132950+0000","last_peered":"2024-05-30T16:58:29.132950+0000","last_clean":"2024-05-30T16:58:29.132950+0000","last_became_active":"2024-05-30T16:58:29.126344+0000","last_became_peered":"2024-05-30T16:58:29.126344+0000","last_unstale":"2024-05-30T16:58:29.132950+0000","last_undegraded":"2024-05-30T16:58:29.132950+0000","last_fullsized":"2024-05-30T16:58:29.132950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:02:40.890296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.125054+0000","last_change":"2024-05-30T16:58:28.104915+0000","last_active":"2024-05-30T16:58:30.125054+0000","last_peered":"2024-05-30T16:58:30.125054+0000","last_clean":"2024-05-30T16:58:30.125054+0000","last_became_active":"2024-05-30T16:58:28.104530+0000","last_became_peered":"2024-05-30T16:58:28.104530+0000","last_unstale":"2024-05-30T16:58:30.125054+0000","last_undegraded":"2024-05-30T16:58:30.125054+0000","last_fullsized":"2024-05-30T16:58:30.125054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:17.172719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.217588+0000","last_change":"2024-05-30T16:58:29.134490+0000","last_active":"2024-05-30T16:58:30.217588+0000","last_peered":"2024-05-30T16:58:30.217588+0000","last_clean":"2024-05-30T16:58:30.217588+0000","last_became_active":"2024-05-30T16:58:29.133664+0000","last_became_peered":"2024-05-30T16:58:29.133664+0000","last_unstale":"2024-05-30T16:58:30.217588+0000","last_undegraded":"2024-05-30T16:58:30.217588+0000","last_fullsized":"2024-05-30T16:58:30.217588+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:17.255770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142543+0000","last_change":"2024-05-30T16:58:28.105396+0000","last_active":"2024-05-30T16:58:30.142543+0000","last_peered":"2024-05-30T16:58:30.142543+0000","last_clean":"2024-05-30T16:58:30.142543+0000","last_became_active":"2024-05-30T16:58:28.105122+0000","last_became_peered":"2024-05-30T16:58:28.105122+0000","last_unstale":"2024-05-30T16:58:30.142543+0000","last_undegraded":"2024-05-30T16:58:30.142543+0000","last_fullsized":"2024-05-30T16:58:30.142543+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:25:06.422803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.136774+0000","last_change":"2024-05-30T16:58:29.129226+0000","last_active":"2024-05-30T16:58:29.136774+0000","last_peered":"2024-05-30T16:58:29.136774+0000","last_clean":"2024-05-30T16:58:29.136774+0000","last_became_active":"2024-05-30T16:58:29.128958+0000","last_became_peered":"2024-05-30T16:58:29.128958+0000","last_unstale":"2024-05-30T16:58:29.136774+0000","last_undegraded":"2024-05-30T16:58:29.136774+0000","last_fullsized":"2024-05-30T16:58:29.136774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:57.354731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.144701+0000","last_change":"2024-05-30T16:58:28.112685+0000","last_active":"2024-05-30T16:58:30.144701+0000","last_peered":"2024-05-30T16:58:30.144701+0000","last_clean":"2024-05-30T16:58:30.144701+0000","last_became_active":"2024-05-30T16:58:28.112391+0000","last_became_peered":"2024-05-30T16:58:28.112391+0000","last_unstale":"2024-05-30T16:58:30.144701+0000","last_undegraded":"2024-05-30T16:58:30.144701+0000","last_fullsized":"2024-05-30T16:58:30.144701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:31:24.605145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173927+0000","last_change":"2024-05-30T16:58:29.144167+0000","last_active":"2024-05-30T16:58:30.173927+0000","last_peered":"2024-05-30T16:58:30.173927+0000","last_clean":"2024-05-30T16:58:30.173927+0000","last_became_active":"2024-05-30T16:58:29.143882+0000","last_became_peered":"2024-05-30T16:58:29.143882+0000","last_unstale":"2024-05-30T16:58:30.173927+0000","last_undegraded":"2024-05-30T16:58:30.173927+0000","last_fullsized":"2024-05-30T16:58:30.173927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:39:55.738510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173394+0000","last_change":"2024-05-30T16:58:28.107379+0000","last_active":"2024-05-30T16:58:30.173394+0000","last_peered":"2024-05-30T16:58:30.173394+0000","last_clean":"2024-05-30T16:58:30.173394+0000","last_became_active":"2024-05-30T16:58:28.107003+0000","last_became_peered":"2024-05-30T16:58:28.107003+0000","last_unstale":"2024-05-30T16:58:30.173394+0000","last_undegraded":"2024-05-30T16:58:30.173394+0000","last_fullsized":"2024-05-30T16:58:30.173394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:15:24.374426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216771+0000","last_change":"2024-05-30T16:58:29.131232+0000","last_active":"2024-05-30T16:58:30.216771+0000","last_peered":"2024-05-30T16:58:30.216771+0000","last_clean":"2024-05-30T16:58:30.216771+0000","last_became_active":"2024-05-30T16:58:29.130843+0000","last_became_peered":"2024-05-30T16:58:29.130843+0000","last_unstale":"2024-05-30T16:58:30.216771+0000","last_undegraded":"2024-05-30T16:58:30.216771+0000","last_fullsized":"2024-05-30T16:58:30.216771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:56:26.493747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173907+0000","last_change":"2024-05-30T16:58:28.097741+0000","last_active":"2024-05-30T16:58:30.173907+0000","last_peered":"2024-05-30T16:58:30.173907+0000","last_clean":"2024-05-30T16:58:30.173907+0000","last_became_active":"2024-05-30T16:58:28.097436+0000","last_became_peered":"2024-05-30T16:58:28.097436+0000","last_unstale":"2024-05-30T16:58:30.173907+0000","last_undegraded":"2024-05-30T16:58:30.173907+0000","last_fullsized":"2024-05-30T16:58:30.173907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:42:25.356966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.119180+0000","last_change":"2024-05-30T16:58:29.115480+0000","last_active":"2024-05-30T16:58:29.119180+0000","last_peered":"2024-05-30T16:58:29.119180+0000","last_clean":"2024-05-30T16:58:29.119180+0000","last_became_active":"2024-05-30T16:58:29.114866+0000","last_became_peered":"2024-05-30T16:58:29.114866+0000","last_unstale":"2024-05-30T16:58:29.119180+0000","last_undegraded":"2024-05-30T16:58:29.119180+0000","last_fullsized":"2024-05-30T16:58:29.119180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:54.389890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.126207+0000","last_change":"2024-05-30T16:58:28.106856+0000","last_active":"2024-05-30T16:58:29.126207+0000","last_peered":"2024-05-30T16:58:29.126207+0000","last_clean":"2024-05-30T16:58:29.126207+0000","last_became_active":"2024-05-30T16:58:28.106562+0000","last_became_peered":"2024-05-30T16:58:28.106562+0000","last_unstale":"2024-05-30T16:58:29.126207+0000","last_undegraded":"2024-05-30T16:58:29.126207+0000","last_fullsized":"2024-05-30T16:58:29.126207+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:19.426018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.140279+0000","last_change":"2024-05-30T16:58:29.133484+0000","last_active":"2024-05-30T16:58:29.140279+0000","last_peered":"2024-05-30T16:58:29.140279+0000","last_clean":"2024-05-30T16:58:29.140279+0000","last_became_active":"2024-05-30T16:58:29.133096+0000","last_became_peered":"2024-05-30T16:58:29.133096+0000","last_unstale":"2024-05-30T16:58:29.140279+0000","last_undegraded":"2024-05-30T16:58:29.140279+0000","last_fullsized":"2024-05-30T16:58:29.140279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:44.398093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216062+0000","last_change":"2024-05-30T16:58:28.110991+0000","last_active":"2024-05-30T16:58:30.216062+0000","last_peered":"2024-05-30T16:58:30.216062+0000","last_clean":"2024-05-30T16:58:30.216062+0000","last_became_active":"2024-05-30T16:58:28.110572+0000","last_became_peered":"2024-05-30T16:58:28.110572+0000","last_unstale":"2024-05-30T16:58:30.216062+0000","last_undegraded":"2024-05-30T16:58:30.216062+0000","last_fullsized":"2024-05-30T16:58:30.216062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:14.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.141826+0000","last_change":"2024-05-30T16:58:29.139570+0000","last_active":"2024-05-30T16:58:29.141826+0000","last_peered":"2024-05-30T16:58:29.141826+0000","last_clean":"2024-05-30T16:58:29.141826+0000","last_became_active":"2024-05-30T16:58:29.139242+0000","last_became_peered":"2024-05-30T16:58:29.139242+0000","last_unstale":"2024-05-30T16:58:29.141826+0000","last_undegraded":"2024-05-30T16:58:29.141826+0000","last_fullsized":"2024-05-30T16:58:29.141826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:32:32.154520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173818+0000","last_change":"2024-05-30T16:58:28.101054+0000","last_active":"2024-05-30T16:58:30.173818+0000","last_peered":"2024-05-30T16:58:30.173818+0000","last_clean":"2024-05-30T16:58:30.173818+0000","last_became_active":"2024-05-30T16:58:28.100722+0000","last_became_peered":"2024-05-30T16:58:28.100722+0000","last_unstale":"2024-05-30T16:58:30.173818+0000","last_undegraded":"2024-05-30T16:58:30.173818+0000","last_fullsized":"2024-05-30T16:58:30.173818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:43:10.868544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173250+0000","last_change":"2024-05-30T16:58:29.136069+0000","last_active":"2024-05-30T16:58:30.173250+0000","last_peered":"2024-05-30T16:58:30.173250+0000","last_clean":"2024-05-30T16:58:30.173250+0000","last_became_active":"2024-05-30T16:58:29.135758+0000","last_became_peered":"2024-05-30T16:58:29.135758+0000","last_unstale":"2024-05-30T16:58:30.173250+0000","last_undegraded":"2024-05-30T16:58:30.173250+0000","last_fullsized":"2024-05-30T16:58:30.173250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:32.109950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.215352+0000","last_change":"2024-05-30T16:58:28.106102+0000","last_active":"2024-05-30T16:58:30.215352+0000","last_peered":"2024-05-30T16:58:30.215352+0000","last_clean":"2024-05-30T16:58:30.215352+0000","last_became_active":"2024-05-30T16:58:28.105833+0000","last_became_peered":"2024-05-30T16:58:28.105833+0000","last_unstale":"2024-05-30T16:58:30.215352+0000","last_undegraded":"2024-05-30T16:58:30.215352+0000","last_fullsized":"2024-05-30T16:58:30.215352+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:59:16.277759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142195+0000","last_change":"2024-05-30T16:58:29.130395+0000","last_active":"2024-05-30T16:58:30.142195+0000","last_peered":"2024-05-30T16:58:30.142195+0000","last_clean":"2024-05-30T16:58:30.142195+0000","last_became_active":"2024-05-30T16:58:29.130094+0000","last_became_peered":"2024-05-30T16:58:29.130094+0000","last_unstale":"2024-05-30T16:58:30.142195+0000","last_undegraded":"2024-05-30T16:58:30.142195+0000","last_fullsized":"2024-05-30T16:58:30.142195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:36:18.504283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.151689+0000","last_change":"2024-05-30T16:58:28.104823+0000","last_active":"2024-05-30T16:58:29.151689+0000","last_peered":"2024-05-30T16:58:29.151689+0000","last_clean":"2024-05-30T16:58:29.151689+0000","last_became_active":"2024-05-30T16:58:28.104492+0000","last_became_peered":"2024-05-30T16:58:28.104492+0000","last_unstale":"2024-05-30T16:58:29.151689+0000","last_undegraded":"2024-05-30T16:58:29.151689+0000","last_fullsized":"2024-05-30T16:58:29.151689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:50:27.592384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":299,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.108772+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:58:29.108772+0000","last_peered":"2024-05-30T16:58:29.108772+0000","last_clean":"2024-05-30T16:58:29.108772+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:58:29.108772+0000","last_undegraded":"2024-05-30T16:58:29.108772+0000","last_fullsized":"2024-05-30T16:58:29.108772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509621,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126584,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245256,"statfs":{"total":96636764160,"available":96507142144,"internally_reserved":0,"allocated":1085440,"data_stored":463756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509621,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86312,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285528,"statfs":{"total":96636764160,"available":96548380672,"internally_reserved":0,"allocated":1073152,"data_stored":463062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509622,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126596,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245244,"statfs":{"total":96636764160,"available":96507129856,"internally_reserved":0,"allocated":1097728,"data_stored":471030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509622,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140468,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231372,"statfs":{"total":96636764160,"available":96492924928,"internally_reserved":0,"allocated":1089536,"data_stored":470000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509621,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104384,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267456,"statfs":{"total":96636764160,"available":96529874944,"internally_reserved":0,"allocated":1097728,"data_stored":469612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542324,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122996,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248844,"statfs":{"total":96636764160,"available":96510816256,"internally_reserved":0,"allocated":1671168,"data_stored":1053500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542325,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76568,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295272,"statfs":{"total":96636764160,"available":96558358528,"internally_reserved":0,"allocated":1060864,"data_stored":456231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542325,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77148,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294692,"statfs":{"total":96636764160,"available":96557764608,"internally_reserved":0,"allocated":1654784,"data_stored":1046599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:58:33.621 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T16:58:33.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.871+0000 7f9bf6800640 1 Processor -- start 2024-05-30T16:58:33.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.871+0000 7f9bf6800640 1 -- start start 2024-05-30T16:58:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.873+0000 7f9bf6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9bf006ee40 0x7f9bf00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.873+0000 7f9bf6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006f2f0 0x7f9bf00ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.873+0000 7f9bf6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf0156f10 con 0x7f9bf0156b30 2024-05-30T16:58:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.873+0000 7f9bf6800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf006dda0 con 0x7f9bf006ee40 2024-05-30T16:58:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.873+0000 7f9bf6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf006e080 con 0x7f9bf006f2f0 2024-05-30T16:58:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.873+0000 7f9bf4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9bf006ee40 0x7f9bf00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.873+0000 7f9bf4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9bf006ee40 0x7f9bf00ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50200/0 (socket says 172.21.15.67:50200) 2024-05-30T16:58:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.873+0000 7f9bf4a00640 1 -- 172.21.15.67:0/4009710331 learned_addr learned my addr 172.21.15.67:0/4009710331 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.874+0000 7f9bf5e00640 1 --2- 172.21.15.67:0/4009710331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006f2f0 0x7f9bf00ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.874+0000 7f9bf4a00640 1 -- 172.21.15.67:0/4009710331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006f2f0 msgr2=0x7f9bf00ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.874+0000 7f9bf4a00640 1 --2- 172.21.15.67:0/4009710331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006f2f0 0x7f9bf00ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.874+0000 7f9bf4a00640 1 -- 172.21.15.67:0/4009710331 >> v1:172.21.15.169:6789/0 conn(0x7f9bf0156b30 legacy=0x7f9bf00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.874+0000 7f9bf4a00640 1 -- 172.21.15.67:0/4009710331 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf006e360 con 0x7f9bf006ee40 2024-05-30T16:58:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.875+0000 7f9bf4a00640 1 --2- 172.21.15.67:0/4009710331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006ee40 0x7f9bf00ad230 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009870 tx=0x7f9bdc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3278f356fb7c9338 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.875+0000 7f9befe00640 1 -- 172.21.15.67:0/4009710331 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc037070 con 0x7f9bf006ee40 2024-05-30T16:58:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9befe00640 1 -- 172.21.15.67:0/4009710331 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bdc032440 con 0x7f9bf006ee40 2024-05-30T16:58:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9befe00640 1 -- 172.21.15.67:0/4009710331 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc032810 con 0x7f9bf006ee40 2024-05-30T16:58:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9bf6800640 1 -- 172.21.15.67:0/4009710331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006ee40 msgr2=0x7f9bf00ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9bf6800640 1 --2- 172.21.15.67:0/4009710331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006ee40 0x7f9bf00ad230 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009870 tx=0x7f9bdc02a1c0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9bf6800640 1 -- 172.21.15.67:0/4009710331 shutdown_connections 2024-05-30T16:58:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9bf6800640 1 --2- 172.21.15.67:0/4009710331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006f2f0 0x7f9bf00ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9bf6800640 1 --2- 172.21.15.67:0/4009710331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006ee40 0x7f9bf00ad230 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9bf6800640 1 -- 172.21.15.67:0/4009710331 >> 172.21.15.67:0/4009710331 conn(0x7f9bf00b1fc0 msgr2=0x7f9bf00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9bf6800640 1 -- 172.21.15.67:0/4009710331 shutdown_connections 2024-05-30T16:58:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.876+0000 7f9bf6800640 1 -- 172.21.15.67:0/4009710331 wait complete. 2024-05-30T16:58:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.878+0000 7f9bf6800640 1 Processor -- start 2024-05-30T16:58:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.878+0000 7f9bf6800640 1 -- start start 2024-05-30T16:58:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.879+0000 7f9bf6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006ee40 0x7f9bf0099300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.879+0000 7f9bf6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006f2f0 0x7f9bf0150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.879+0000 7f9bf6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0156b30 0x7f9bf01510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.879+0000 7f9bf6800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf00b2db0 con 0x7f9bf006f2f0 2024-05-30T16:58:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.879+0000 7f9bf6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf00b2c30 con 0x7f9bf0156b30 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.879+0000 7f9bf6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf00b32c0 con 0x7f9bf006ee40 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.879+0000 7f9bf4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006f2f0 0x7f9bf0150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.879+0000 7f9bf4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006f2f0 0x7f9bf0150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50202/0 (socket says 172.21.15.67:50202) 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.879+0000 7f9bf4a00640 1 -- 172.21.15.67:0/20325971 learned_addr learned my addr 172.21.15.67:0/20325971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.880+0000 7f9bf5e00640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0156b30 0x7f9bf01510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.880+0000 7f9bf5400640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006ee40 0x7f9bf0099300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.880+0000 7f9bf4a00640 1 -- 172.21.15.67:0/20325971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006ee40 msgr2=0x7f9bf0099300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.880+0000 7f9bf4a00640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006ee40 0x7f9bf0099300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.880+0000 7f9bf4a00640 1 -- 172.21.15.67:0/20325971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0156b30 msgr2=0x7f9bf01510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.880+0000 7f9bf4a00640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0156b30 0x7f9bf01510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.880+0000 7f9bf4a00640 1 -- 172.21.15.67:0/20325971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf01e87c0 con 0x7f9bf006f2f0 2024-05-30T16:58:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.880+0000 7f9bf4a00640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006f2f0 0x7f9bf0150ba0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc000c00 tx=0x7f9bdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.881+0000 7f9bed600640 1 -- 172.21.15.67:0/20325971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc041020 con 0x7f9bf006f2f0 2024-05-30T16:58:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.881+0000 7f9bed600640 1 -- 172.21.15.67:0/20325971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bdc0306c0 con 0x7f9bf006f2f0 2024-05-30T16:58:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.881+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf0157690 con 0x7f9bf006f2f0 2024-05-30T16:58:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.881+0000 7f9bed600640 1 -- 172.21.15.67:0/20325971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc037040 con 0x7f9bf006f2f0 2024-05-30T16:58:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.881+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf0157c50 con 0x7f9bf006f2f0 2024-05-30T16:58:33.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.882+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf00689e0 con 0x7f9bf006f2f0 2024-05-30T16:58:33.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.883+0000 7f9bed600640 1 -- 172.21.15.67:0/20325971 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f9bdc044020 con 0x7f9bf006f2f0 2024-05-30T16:58:33.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.884+0000 7f9bed600640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006da90 0x7f9bd006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:33.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.884+0000 7f9bed600640 1 -- 172.21.15.67:0/20325971 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5823+0+0 (secure 0 0 0) 0x7f9bdc0ab280 con 0x7f9bf006f2f0 2024-05-30T16:58:33.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.885+0000 7f9bf5400640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006da90 0x7f9bd006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:33.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.885+0000 7f9bf5400640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006da90 0x7f9bd006ff50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9be4000f60 tx=0x7f9be4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:33.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:33.889+0000 7f9bed600640 1 -- 172.21.15.67:0/20325971 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bdc04b030 con 0x7f9bf006f2f0 2024-05-30T16:58:34.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.085+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bf00b4950 con 0x7f9bd006da90 2024-05-30T16:58:34.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.104+0000 7f9bed600640 1 -- 172.21.15.67:0/20325971 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252079 (secure 0 0 0) 0x7f9bf00b4950 con 0x7f9bd006da90 2024-05-30T16:58:34.107 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:34.110 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T16:58:34.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006da90 msgr2=0x7f9bd006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006da90 0x7f9bd006ff50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9be4000f60 tx=0x7f9be4009040 comp rx=0 tx=0).stop 2024-05-30T16:58:34.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006f2f0 msgr2=0x7f9bf0150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006f2f0 0x7f9bf0150ba0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc000c00 tx=0x7f9bdc0046c0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 shutdown_connections 2024-05-30T16:58:34.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006da90 0x7f9bd006ff50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0156b30 0x7f9bf01510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf006f2f0 0x7f9bf0150ba0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 --2- 172.21.15.67:0/20325971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf006ee40 0x7f9bf0099300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.111+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 >> 172.21.15.67:0/20325971 conn(0x7f9bf00b1fc0 msgr2=0x7f9bf00b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:34.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.112+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 shutdown_connections 2024-05-30T16:58:34.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.112+0000 7f9bf6800640 1 -- 172.21.15.67:0/20325971 wait complete. 2024-05-30T16:58:34.132 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2024-05-30T16:58:33.163344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":861056,"kb_used_data":9600,"kb_used_omap":12,"kb_used_meta":851315,"kb_avail":754113664,"statfs":{"total":773094113280,"available":772212391936,"internally_reserved":0,"allocated":9830400,"data_stored":4893790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":871747120},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.083687"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173826+0000","last_change":"2024-05-30T16:58:29.134700+0000","last_active":"2024-05-30T16:58:30.173826+0000","last_peered":"2024-05-30T16:58:30.173826+0000","last_clean":"2024-05-30T16:58:30.173826+0000","last_became_active":"2024-05-30T16:58:29.134424+0000","last_became_peered":"2024-05-30T16:58:29.134424+0000","last_unstale":"2024-05-30T16:58:30.173826+0000","last_undegraded":"2024-05-30T16:58:30.173826+0000","last_fullsized":"2024-05-30T16:58:30.173826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:44.238425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142500+0000","last_change":"2024-05-30T16:58:29.134283+0000","last_active":"2024-05-30T16:58:30.142500+0000","last_peered":"2024-05-30T16:58:30.142500+0000","last_clean":"2024-05-30T16:58:30.142500+0000","last_became_active":"2024-05-30T16:58:29.133622+0000","last_became_peered":"2024-05-30T16:58:29.133622+0000","last_unstale":"2024-05-30T16:58:30.142500+0000","last_undegraded":"2024-05-30T16:58:30.142500+0000","last_fullsized":"2024-05-30T16:58:30.142500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:05:49.663228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.137575+0000","last_change":"2024-05-30T16:58:29.135737+0000","last_active":"2024-05-30T16:58:29.137575+0000","last_peered":"2024-05-30T16:58:29.137575+0000","last_clean":"2024-05-30T16:58:29.137575+0000","last_became_active":"2024-05-30T16:58:29.130881+0000","last_became_peered":"2024-05-30T16:58:29.130881+0000","last_unstale":"2024-05-30T16:58:29.137575+0000","last_undegraded":"2024-05-30T16:58:29.137575+0000","last_fullsized":"2024-05-30T16:58:29.137575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:15:59.315185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128147+0000","last_change":"2024-05-30T16:58:29.143594+0000","last_active":"2024-05-30T16:58:30.128147+0000","last_peered":"2024-05-30T16:58:30.128147+0000","last_clean":"2024-05-30T16:58:30.128147+0000","last_became_active":"2024-05-30T16:58:29.142724+0000","last_became_peered":"2024-05-30T16:58:29.142724+0000","last_unstale":"2024-05-30T16:58:30.128147+0000","last_undegraded":"2024-05-30T16:58:30.128147+0000","last_fullsized":"2024-05-30T16:58:30.128147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:25.159552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216284+0000","last_change":"2024-05-30T16:58:29.142988+0000","last_active":"2024-05-30T16:58:30.216284+0000","last_peered":"2024-05-30T16:58:30.216284+0000","last_clean":"2024-05-30T16:58:30.216284+0000","last_became_active":"2024-05-30T16:58:29.141432+0000","last_became_peered":"2024-05-30T16:58:29.141432+0000","last_unstale":"2024-05-30T16:58:30.216284+0000","last_undegraded":"2024-05-30T16:58:30.216284+0000","last_fullsized":"2024-05-30T16:58:30.216284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:09.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.148985+0000","last_change":"2024-05-30T16:58:29.147576+0000","last_active":"2024-05-30T16:58:29.148985+0000","last_peered":"2024-05-30T16:58:29.148985+0000","last_clean":"2024-05-30T16:58:29.148985+0000","last_became_active":"2024-05-30T16:58:29.146398+0000","last_became_peered":"2024-05-30T16:58:29.146398+0000","last_unstale":"2024-05-30T16:58:29.148985+0000","last_undegraded":"2024-05-30T16:58:29.148985+0000","last_fullsized":"2024-05-30T16:58:29.148985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:53:17.147362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127708+0000","last_change":"2024-05-30T16:58:29.139475+0000","last_active":"2024-05-30T16:58:30.127708+0000","last_peered":"2024-05-30T16:58:30.127708+0000","last_clean":"2024-05-30T16:58:30.127708+0000","last_became_active":"2024-05-30T16:58:29.139069+0000","last_became_peered":"2024-05-30T16:58:29.139069+0000","last_unstale":"2024-05-30T16:58:30.127708+0000","last_undegraded":"2024-05-30T16:58:30.127708+0000","last_fullsized":"2024-05-30T16:58:30.127708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:10:56.904673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.135616+0000","last_change":"2024-05-30T16:58:29.132440+0000","last_active":"2024-05-30T16:58:29.135616+0000","last_peered":"2024-05-30T16:58:29.135616+0000","last_clean":"2024-05-30T16:58:29.135616+0000","last_became_active":"2024-05-30T16:58:29.132102+0000","last_became_peered":"2024-05-30T16:58:29.132102+0000","last_unstale":"2024-05-30T16:58:29.135616+0000","last_undegraded":"2024-05-30T16:58:29.135616+0000","last_fullsized":"2024-05-30T16:58:29.135616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:08:50.542702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.131863+0000","last_change":"2024-05-30T16:58:29.129581+0000","last_active":"2024-05-30T16:58:29.131863+0000","last_peered":"2024-05-30T16:58:29.131863+0000","last_clean":"2024-05-30T16:58:29.131863+0000","last_became_active":"2024-05-30T16:58:29.126155+0000","last_became_peered":"2024-05-30T16:58:29.126155+0000","last_unstale":"2024-05-30T16:58:29.131863+0000","last_undegraded":"2024-05-30T16:58:29.131863+0000","last_fullsized":"2024-05-30T16:58:29.131863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:54:32.272063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173936+0000","last_change":"2024-05-30T16:58:29.127811+0000","last_active":"2024-05-30T16:58:30.173936+0000","last_peered":"2024-05-30T16:58:30.173936+0000","last_clean":"2024-05-30T16:58:30.173936+0000","last_became_active":"2024-05-30T16:58:29.126934+0000","last_became_peered":"2024-05-30T16:58:29.126934+0000","last_unstale":"2024-05-30T16:58:30.173936+0000","last_undegraded":"2024-05-30T16:58:30.173936+0000","last_fullsized":"2024-05-30T16:58:30.173936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:02.038829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142452+0000","last_change":"2024-05-30T16:58:29.138717+0000","last_active":"2024-05-30T16:58:30.142452+0000","last_peered":"2024-05-30T16:58:30.142452+0000","last_clean":"2024-05-30T16:58:30.142452+0000","last_became_active":"2024-05-30T16:58:29.132811+0000","last_became_peered":"2024-05-30T16:58:29.132811+0000","last_unstale":"2024-05-30T16:58:30.142452+0000","last_undegraded":"2024-05-30T16:58:30.142452+0000","last_fullsized":"2024-05-30T16:58:30.142452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:31:18.832238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173109+0000","last_change":"2024-05-30T16:58:29.132568+0000","last_active":"2024-05-30T16:58:30.173109+0000","last_peered":"2024-05-30T16:58:30.173109+0000","last_clean":"2024-05-30T16:58:30.173109+0000","last_became_active":"2024-05-30T16:58:29.130928+0000","last_became_peered":"2024-05-30T16:58:29.130928+0000","last_unstale":"2024-05-30T16:58:30.173109+0000","last_undegraded":"2024-05-30T16:58:30.173109+0000","last_fullsized":"2024-05-30T16:58:30.173109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:09.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.136139+0000","last_change":"2024-05-30T16:58:29.127628+0000","last_active":"2024-05-30T16:58:29.136139+0000","last_peered":"2024-05-30T16:58:29.136139+0000","last_clean":"2024-05-30T16:58:29.136139+0000","last_became_active":"2024-05-30T16:58:29.127101+0000","last_became_peered":"2024-05-30T16:58:29.127101+0000","last_unstale":"2024-05-30T16:58:29.136139+0000","last_undegraded":"2024-05-30T16:58:29.136139+0000","last_fullsized":"2024-05-30T16:58:29.136139+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:44.398093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128254+0000","last_change":"2024-05-30T16:58:29.146655+0000","last_active":"2024-05-30T16:58:30.128254+0000","last_peered":"2024-05-30T16:58:30.128254+0000","last_clean":"2024-05-30T16:58:30.128254+0000","last_became_active":"2024-05-30T16:58:29.145851+0000","last_became_peered":"2024-05-30T16:58:29.145851+0000","last_unstale":"2024-05-30T16:58:30.128254+0000","last_undegraded":"2024-05-30T16:58:30.128254+0000","last_fullsized":"2024-05-30T16:58:30.128254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:50.575170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.124364+0000","last_change":"2024-05-30T16:58:29.125194+0000","last_active":"2024-05-30T16:58:30.124364+0000","last_peered":"2024-05-30T16:58:30.124364+0000","last_clean":"2024-05-30T16:58:30.124364+0000","last_became_active":"2024-05-30T16:58:29.119557+0000","last_became_peered":"2024-05-30T16:58:29.119557+0000","last_unstale":"2024-05-30T16:58:30.124364+0000","last_undegraded":"2024-05-30T16:58:30.124364+0000","last_fullsized":"2024-05-30T16:58:30.124364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:05.947318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.215865+0000","last_change":"2024-05-30T16:58:29.132349+0000","last_active":"2024-05-30T16:58:30.215865+0000","last_peered":"2024-05-30T16:58:30.215865+0000","last_clean":"2024-05-30T16:58:30.215865+0000","last_became_active":"2024-05-30T16:58:29.132085+0000","last_became_peered":"2024-05-30T16:58:29.132085+0000","last_unstale":"2024-05-30T16:58:30.215865+0000","last_undegraded":"2024-05-30T16:58:30.215865+0000","last_fullsized":"2024-05-30T16:58:30.215865+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:27:35.736510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.217573+0000","last_change":"2024-05-30T16:58:29.146119+0000","last_active":"2024-05-30T16:58:30.217573+0000","last_peered":"2024-05-30T16:58:30.217573+0000","last_clean":"2024-05-30T16:58:30.217573+0000","last_became_active":"2024-05-30T16:58:29.145798+0000","last_became_peered":"2024-05-30T16:58:29.145798+0000","last_unstale":"2024-05-30T16:58:30.217573+0000","last_undegraded":"2024-05-30T16:58:30.217573+0000","last_fullsized":"2024-05-30T16:58:30.217573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:22.544018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128218+0000","last_change":"2024-05-30T16:58:29.135098+0000","last_active":"2024-05-30T16:58:30.128218+0000","last_peered":"2024-05-30T16:58:30.128218+0000","last_clean":"2024-05-30T16:58:30.128218+0000","last_became_active":"2024-05-30T16:58:29.134804+0000","last_became_peered":"2024-05-30T16:58:29.134804+0000","last_unstale":"2024-05-30T16:58:30.128218+0000","last_undegraded":"2024-05-30T16:58:30.128218+0000","last_fullsized":"2024-05-30T16:58:30.128218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:12.142920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.139319+0000","last_change":"2024-05-30T16:58:29.132473+0000","last_active":"2024-05-30T16:58:29.139319+0000","last_peered":"2024-05-30T16:58:29.139319+0000","last_clean":"2024-05-30T16:58:29.139319+0000","last_became_active":"2024-05-30T16:58:29.132174+0000","last_became_peered":"2024-05-30T16:58:29.132174+0000","last_unstale":"2024-05-30T16:58:29.139319+0000","last_undegraded":"2024-05-30T16:58:29.139319+0000","last_fullsized":"2024-05-30T16:58:29.139319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:36:37.836774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128475+0000","last_change":"2024-05-30T16:58:29.143185+0000","last_active":"2024-05-30T16:58:30.128475+0000","last_peered":"2024-05-30T16:58:30.128475+0000","last_clean":"2024-05-30T16:58:30.128475+0000","last_became_active":"2024-05-30T16:58:29.142896+0000","last_became_peered":"2024-05-30T16:58:29.142896+0000","last_unstale":"2024-05-30T16:58:30.128475+0000","last_undegraded":"2024-05-30T16:58:30.128475+0000","last_fullsized":"2024-05-30T16:58:30.128475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:34.754725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127756+0000","last_change":"2024-05-30T16:58:29.143222+0000","last_active":"2024-05-30T16:58:30.127756+0000","last_peered":"2024-05-30T16:58:30.127756+0000","last_clean":"2024-05-30T16:58:30.127756+0000","last_became_active":"2024-05-30T16:58:29.142927+0000","last_became_peered":"2024-05-30T16:58:29.142927+0000","last_unstale":"2024-05-30T16:58:30.127756+0000","last_undegraded":"2024-05-30T16:58:30.127756+0000","last_fullsized":"2024-05-30T16:58:30.127756+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:45:51.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141815+0000","last_change":"2024-05-30T16:58:29.145994+0000","last_active":"2024-05-30T16:58:30.141815+0000","last_peered":"2024-05-30T16:58:30.141815+0000","last_clean":"2024-05-30T16:58:30.141815+0000","last_became_active":"2024-05-30T16:58:29.145634+0000","last_became_peered":"2024-05-30T16:58:29.145634+0000","last_unstale":"2024-05-30T16:58:30.141815+0000","last_undegraded":"2024-05-30T16:58:30.141815+0000","last_fullsized":"2024-05-30T16:58:30.141815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:50.575170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173257+0000","last_change":"2024-05-30T16:58:29.130507+0000","last_active":"2024-05-30T16:58:30.173257+0000","last_peered":"2024-05-30T16:58:30.173257+0000","last_clean":"2024-05-30T16:58:30.173257+0000","last_became_active":"2024-05-30T16:58:29.130178+0000","last_became_peered":"2024-05-30T16:58:29.130178+0000","last_unstale":"2024-05-30T16:58:30.173257+0000","last_undegraded":"2024-05-30T16:58:30.173257+0000","last_fullsized":"2024-05-30T16:58:30.173257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:56:26.493747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127616+0000","last_change":"2024-05-30T16:58:29.136667+0000","last_active":"2024-05-30T16:58:30.127616+0000","last_peered":"2024-05-30T16:58:30.127616+0000","last_clean":"2024-05-30T16:58:30.127616+0000","last_became_active":"2024-05-30T16:58:29.135160+0000","last_became_peered":"2024-05-30T16:58:29.135160+0000","last_unstale":"2024-05-30T16:58:30.127616+0000","last_undegraded":"2024-05-30T16:58:30.127616+0000","last_fullsized":"2024-05-30T16:58:30.127616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:13:53.332755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142768+0000","last_change":"2024-05-30T16:58:29.137037+0000","last_active":"2024-05-30T16:58:30.142768+0000","last_peered":"2024-05-30T16:58:30.142768+0000","last_clean":"2024-05-30T16:58:30.142768+0000","last_became_active":"2024-05-30T16:58:29.136760+0000","last_became_peered":"2024-05-30T16:58:29.136760+0000","last_unstale":"2024-05-30T16:58:30.142768+0000","last_undegraded":"2024-05-30T16:58:30.142768+0000","last_fullsized":"2024-05-30T16:58:30.142768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:11:01.264686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.142596+0000","last_change":"2024-05-30T16:58:29.140538+0000","last_active":"2024-05-30T16:58:29.142596+0000","last_peered":"2024-05-30T16:58:29.142596+0000","last_clean":"2024-05-30T16:58:29.142596+0000","last_became_active":"2024-05-30T16:58:29.140052+0000","last_became_peered":"2024-05-30T16:58:29.140052+0000","last_unstale":"2024-05-30T16:58:29.142596+0000","last_undegraded":"2024-05-30T16:58:29.142596+0000","last_fullsized":"2024-05-30T16:58:29.142596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:52:10.807493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.215931+0000","last_change":"2024-05-30T16:58:29.137903+0000","last_active":"2024-05-30T16:58:30.215931+0000","last_peered":"2024-05-30T16:58:30.215931+0000","last_clean":"2024-05-30T16:58:30.215931+0000","last_became_active":"2024-05-30T16:58:29.135438+0000","last_became_peered":"2024-05-30T16:58:29.135438+0000","last_unstale":"2024-05-30T16:58:30.215931+0000","last_undegraded":"2024-05-30T16:58:30.215931+0000","last_fullsized":"2024-05-30T16:58:30.215931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:54:46.848424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173592+0000","last_change":"2024-05-30T16:58:29.132320+0000","last_active":"2024-05-30T16:58:30.173592+0000","last_peered":"2024-05-30T16:58:30.173592+0000","last_clean":"2024-05-30T16:58:30.173592+0000","last_became_active":"2024-05-30T16:58:29.132037+0000","last_became_peered":"2024-05-30T16:58:29.132037+0000","last_unstale":"2024-05-30T16:58:30.173592+0000","last_undegraded":"2024-05-30T16:58:30.173592+0000","last_fullsized":"2024-05-30T16:58:30.173592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:27:22.544018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141751+0000","last_change":"2024-05-30T16:58:29.135169+0000","last_active":"2024-05-30T16:58:30.141751+0000","last_peered":"2024-05-30T16:58:30.141751+0000","last_clean":"2024-05-30T16:58:30.141751+0000","last_became_active":"2024-05-30T16:58:29.134724+0000","last_became_peered":"2024-05-30T16:58:29.134724+0000","last_unstale":"2024-05-30T16:58:30.141751+0000","last_undegraded":"2024-05-30T16:58:30.141751+0000","last_fullsized":"2024-05-30T16:58:30.141751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:33:12.142920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141995+0000","last_change":"2024-05-30T16:58:29.147599+0000","last_active":"2024-05-30T16:58:30.141995+0000","last_peered":"2024-05-30T16:58:30.141995+0000","last_clean":"2024-05-30T16:58:30.141995+0000","last_became_active":"2024-05-30T16:58:29.147146+0000","last_became_peered":"2024-05-30T16:58:29.147146+0000","last_unstale":"2024-05-30T16:58:30.141995+0000","last_undegraded":"2024-05-30T16:58:30.141995+0000","last_fullsized":"2024-05-30T16:58:30.141995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:34.754725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216206+0000","last_change":"2024-05-30T16:58:29.131709+0000","last_active":"2024-05-30T16:58:30.216206+0000","last_peered":"2024-05-30T16:58:30.216206+0000","last_clean":"2024-05-30T16:58:30.216206+0000","last_became_active":"2024-05-30T16:58:29.131345+0000","last_became_peered":"2024-05-30T16:58:29.131345+0000","last_unstale":"2024-05-30T16:58:30.216206+0000","last_undegraded":"2024-05-30T16:58:30.216206+0000","last_fullsized":"2024-05-30T16:58:30.216206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:02:19.809216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173691+0000","last_change":"2024-05-30T16:58:29.137413+0000","last_active":"2024-05-30T16:58:30.173691+0000","last_peered":"2024-05-30T16:58:30.173691+0000","last_clean":"2024-05-30T16:58:30.173691+0000","last_became_active":"2024-05-30T16:58:29.137101+0000","last_became_peered":"2024-05-30T16:58:29.137101+0000","last_unstale":"2024-05-30T16:58:30.173691+0000","last_undegraded":"2024-05-30T16:58:30.173691+0000","last_fullsized":"2024-05-30T16:58:30.173691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:17.255770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.217703+0000","last_change":"2024-05-30T16:58:29.131881+0000","last_active":"2024-05-30T16:58:30.217703+0000","last_peered":"2024-05-30T16:58:30.217703+0000","last_clean":"2024-05-30T16:58:30.217703+0000","last_became_active":"2024-05-30T16:58:29.131511+0000","last_became_peered":"2024-05-30T16:58:29.131511+0000","last_unstale":"2024-05-30T16:58:30.217703+0000","last_undegraded":"2024-05-30T16:58:30.217703+0000","last_fullsized":"2024-05-30T16:58:30.217703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:44.238425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174143+0000","last_change":"2024-05-30T16:58:28.108634+0000","last_active":"2024-05-30T16:58:30.174143+0000","last_peered":"2024-05-30T16:58:30.174143+0000","last_clean":"2024-05-30T16:58:30.174143+0000","last_became_active":"2024-05-30T16:58:28.108356+0000","last_became_peered":"2024-05-30T16:58:28.108356+0000","last_unstale":"2024-05-30T16:58:30.174143+0000","last_undegraded":"2024-05-30T16:58:30.174143+0000","last_fullsized":"2024-05-30T16:58:30.174143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:26:44.065918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173977+0000","last_change":"2024-05-30T16:58:29.133335+0000","last_active":"2024-05-30T16:58:30.173977+0000","last_peered":"2024-05-30T16:58:30.173977+0000","last_clean":"2024-05-30T16:58:30.173977+0000","last_became_active":"2024-05-30T16:58:29.130543+0000","last_became_peered":"2024-05-30T16:58:29.130543+0000","last_unstale":"2024-05-30T16:58:30.173977+0000","last_undegraded":"2024-05-30T16:58:30.173977+0000","last_fullsized":"2024-05-30T16:58:30.173977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:59.208770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.217777+0000","last_change":"2024-05-30T16:58:28.108639+0000","last_active":"2024-05-30T16:58:30.217777+0000","last_peered":"2024-05-30T16:58:30.217777+0000","last_clean":"2024-05-30T16:58:30.217777+0000","last_became_active":"2024-05-30T16:58:28.106892+0000","last_became_peered":"2024-05-30T16:58:28.106892+0000","last_unstale":"2024-05-30T16:58:30.217777+0000","last_undegraded":"2024-05-30T16:58:30.217777+0000","last_fullsized":"2024-05-30T16:58:30.217777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:15:24.374426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173489+0000","last_change":"2024-05-30T16:58:29.140924+0000","last_active":"2024-05-30T16:58:30.173489+0000","last_peered":"2024-05-30T16:58:30.173489+0000","last_clean":"2024-05-30T16:58:30.173489+0000","last_became_active":"2024-05-30T16:58:29.140535+0000","last_became_peered":"2024-05-30T16:58:29.140535+0000","last_unstale":"2024-05-30T16:58:30.173489+0000","last_undegraded":"2024-05-30T16:58:30.173489+0000","last_fullsized":"2024-05-30T16:58:30.173489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:22.645412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.155860+0000","last_change":"2024-05-30T16:58:28.105704+0000","last_active":"2024-05-30T16:58:29.155860+0000","last_peered":"2024-05-30T16:58:29.155860+0000","last_clean":"2024-05-30T16:58:29.155860+0000","last_became_active":"2024-05-30T16:58:28.104703+0000","last_became_peered":"2024-05-30T16:58:28.104703+0000","last_unstale":"2024-05-30T16:58:29.155860+0000","last_undegraded":"2024-05-30T16:58:29.155860+0000","last_fullsized":"2024-05-30T16:58:29.155860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:17:11.646160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.142641+0000","last_change":"2024-05-30T16:58:29.140789+0000","last_active":"2024-05-30T16:58:29.142641+0000","last_peered":"2024-05-30T16:58:29.142641+0000","last_clean":"2024-05-30T16:58:29.142641+0000","last_became_active":"2024-05-30T16:58:29.139977+0000","last_became_peered":"2024-05-30T16:58:29.139977+0000","last_unstale":"2024-05-30T16:58:29.142641+0000","last_undegraded":"2024-05-30T16:58:29.142641+0000","last_fullsized":"2024-05-30T16:58:29.142641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:31:43.364677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.156258+0000","last_change":"2024-05-30T16:58:28.108974+0000","last_active":"2024-05-30T16:58:29.156258+0000","last_peered":"2024-05-30T16:58:29.156258+0000","last_clean":"2024-05-30T16:58:29.156258+0000","last_became_active":"2024-05-30T16:58:28.108604+0000","last_became_peered":"2024-05-30T16:58:28.108604+0000","last_unstale":"2024-05-30T16:58:29.156258+0000","last_undegraded":"2024-05-30T16:58:29.156258+0000","last_fullsized":"2024-05-30T16:58:29.156258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:35:31.549762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142086+0000","last_change":"2024-05-30T16:58:29.146801+0000","last_active":"2024-05-30T16:58:30.142086+0000","last_peered":"2024-05-30T16:58:30.142086+0000","last_clean":"2024-05-30T16:58:30.142086+0000","last_became_active":"2024-05-30T16:58:29.146517+0000","last_became_peered":"2024-05-30T16:58:29.146517+0000","last_unstale":"2024-05-30T16:58:30.142086+0000","last_undegraded":"2024-05-30T16:58:30.142086+0000","last_fullsized":"2024-05-30T16:58:30.142086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:17:41.362206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216600+0000","last_change":"2024-05-30T16:58:28.104793+0000","last_active":"2024-05-30T16:58:30.216600+0000","last_peered":"2024-05-30T16:58:30.216600+0000","last_clean":"2024-05-30T16:58:30.216600+0000","last_became_active":"2024-05-30T16:58:28.104138+0000","last_became_peered":"2024-05-30T16:58:28.104138+0000","last_unstale":"2024-05-30T16:58:30.216600+0000","last_undegraded":"2024-05-30T16:58:30.216600+0000","last_fullsized":"2024-05-30T16:58:30.216600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:00:46.771796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216889+0000","last_change":"2024-05-30T16:58:29.141417+0000","last_active":"2024-05-30T16:58:30.216889+0000","last_peered":"2024-05-30T16:58:30.216889+0000","last_clean":"2024-05-30T16:58:30.216889+0000","last_became_active":"2024-05-30T16:58:29.140692+0000","last_became_peered":"2024-05-30T16:58:29.140692+0000","last_unstale":"2024-05-30T16:58:30.216889+0000","last_undegraded":"2024-05-30T16:58:30.216889+0000","last_fullsized":"2024-05-30T16:58:30.216889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:22.645412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.215780+0000","last_change":"2024-05-30T16:58:28.109377+0000","last_active":"2024-05-30T16:58:30.215780+0000","last_peered":"2024-05-30T16:58:30.215780+0000","last_clean":"2024-05-30T16:58:30.215780+0000","last_became_active":"2024-05-30T16:58:28.108720+0000","last_became_peered":"2024-05-30T16:58:28.108720+0000","last_unstale":"2024-05-30T16:58:30.215780+0000","last_undegraded":"2024-05-30T16:58:30.215780+0000","last_fullsized":"2024-05-30T16:58:30.215780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:43:10.868544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.137783+0000","last_change":"2024-05-30T16:58:29.132392+0000","last_active":"2024-05-30T16:58:29.137783+0000","last_peered":"2024-05-30T16:58:29.137783+0000","last_clean":"2024-05-30T16:58:29.137783+0000","last_became_active":"2024-05-30T16:58:29.132083+0000","last_became_peered":"2024-05-30T16:58:29.132083+0000","last_unstale":"2024-05-30T16:58:29.137783+0000","last_undegraded":"2024-05-30T16:58:29.137783+0000","last_fullsized":"2024-05-30T16:58:29.137783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:18:36.476929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.127119+0000","last_change":"2024-05-30T16:58:28.108001+0000","last_active":"2024-05-30T16:58:29.127119+0000","last_peered":"2024-05-30T16:58:29.127119+0000","last_clean":"2024-05-30T16:58:29.127119+0000","last_became_active":"2024-05-30T16:58:28.107703+0000","last_became_peered":"2024-05-30T16:58:28.107703+0000","last_unstale":"2024-05-30T16:58:29.127119+0000","last_undegraded":"2024-05-30T16:58:29.127119+0000","last_fullsized":"2024-05-30T16:58:29.127119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:36.468903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.124294+0000","last_change":"2024-05-30T16:58:29.120318+0000","last_active":"2024-05-30T16:58:30.124294+0000","last_peered":"2024-05-30T16:58:30.124294+0000","last_clean":"2024-05-30T16:58:30.124294+0000","last_became_active":"2024-05-30T16:58:29.119920+0000","last_became_peered":"2024-05-30T16:58:29.119920+0000","last_unstale":"2024-05-30T16:58:30.124294+0000","last_undegraded":"2024-05-30T16:58:30.124294+0000","last_fullsized":"2024-05-30T16:58:30.124294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:16.700840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142186+0000","last_change":"2024-05-30T16:58:28.107649+0000","last_active":"2024-05-30T16:58:30.142186+0000","last_peered":"2024-05-30T16:58:30.142186+0000","last_clean":"2024-05-30T16:58:30.142186+0000","last_became_active":"2024-05-30T16:58:28.105597+0000","last_became_peered":"2024-05-30T16:58:28.105597+0000","last_unstale":"2024-05-30T16:58:30.142186+0000","last_undegraded":"2024-05-30T16:58:30.142186+0000","last_fullsized":"2024-05-30T16:58:30.142186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:56.153715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.125643+0000","last_change":"2024-05-30T16:58:29.136646+0000","last_active":"2024-05-30T16:58:30.125643+0000","last_peered":"2024-05-30T16:58:30.125643+0000","last_clean":"2024-05-30T16:58:30.125643+0000","last_became_active":"2024-05-30T16:58:29.136370+0000","last_became_peered":"2024-05-30T16:58:29.136370+0000","last_unstale":"2024-05-30T16:58:30.125643+0000","last_undegraded":"2024-05-30T16:58:30.125643+0000","last_fullsized":"2024-05-30T16:58:30.125643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:09:28.798742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174310+0000","last_change":"2024-05-30T16:58:28.099840+0000","last_active":"2024-05-30T16:58:30.174310+0000","last_peered":"2024-05-30T16:58:30.174310+0000","last_clean":"2024-05-30T16:58:30.174310+0000","last_became_active":"2024-05-30T16:58:28.099453+0000","last_became_peered":"2024-05-30T16:58:28.099453+0000","last_unstale":"2024-05-30T16:58:30.174310+0000","last_undegraded":"2024-05-30T16:58:30.174310+0000","last_fullsized":"2024-05-30T16:58:30.174310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:29:57.171394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.148518+0000","last_change":"2024-05-30T16:58:29.145237+0000","last_active":"2024-05-30T16:58:29.148518+0000","last_peered":"2024-05-30T16:58:29.148518+0000","last_clean":"2024-05-30T16:58:29.148518+0000","last_became_active":"2024-05-30T16:58:29.144954+0000","last_became_peered":"2024-05-30T16:58:29.144954+0000","last_unstale":"2024-05-30T16:58:29.148518+0000","last_undegraded":"2024-05-30T16:58:29.148518+0000","last_fullsized":"2024-05-30T16:58:29.148518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:31:09.043735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174004+0000","last_change":"2024-05-30T16:58:28.102851+0000","last_active":"2024-05-30T16:58:30.174004+0000","last_peered":"2024-05-30T16:58:30.174004+0000","last_clean":"2024-05-30T16:58:30.174004+0000","last_became_active":"2024-05-30T16:58:28.102547+0000","last_became_peered":"2024-05-30T16:58:28.102547+0000","last_unstale":"2024-05-30T16:58:30.174004+0000","last_undegraded":"2024-05-30T16:58:30.174004+0000","last_fullsized":"2024-05-30T16:58:30.174004+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:00:46.771796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.147515+0000","last_change":"2024-05-30T16:58:29.137538+0000","last_active":"2024-05-30T16:58:29.147515+0000","last_peered":"2024-05-30T16:58:29.147515+0000","last_clean":"2024-05-30T16:58:29.147515+0000","last_became_active":"2024-05-30T16:58:29.137227+0000","last_became_peered":"2024-05-30T16:58:29.137227+0000","last_unstale":"2024-05-30T16:58:29.147515+0000","last_undegraded":"2024-05-30T16:58:29.147515+0000","last_fullsized":"2024-05-30T16:58:29.147515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:18:36.476929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173172+0000","last_change":"2024-05-30T16:58:28.099059+0000","last_active":"2024-05-30T16:58:30.173172+0000","last_peered":"2024-05-30T16:58:30.173172+0000","last_clean":"2024-05-30T16:58:30.173172+0000","last_became_active":"2024-05-30T16:58:28.098711+0000","last_became_peered":"2024-05-30T16:58:28.098711+0000","last_unstale":"2024-05-30T16:58:30.173172+0000","last_undegraded":"2024-05-30T16:58:30.173172+0000","last_fullsized":"2024-05-30T16:58:30.173172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:14.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216162+0000","last_change":"2024-05-30T16:58:29.135114+0000","last_active":"2024-05-30T16:58:30.216162+0000","last_peered":"2024-05-30T16:58:30.216162+0000","last_clean":"2024-05-30T16:58:30.216162+0000","last_became_active":"2024-05-30T16:58:29.134827+0000","last_became_peered":"2024-05-30T16:58:29.134827+0000","last_unstale":"2024-05-30T16:58:30.216162+0000","last_undegraded":"2024-05-30T16:58:30.216162+0000","last_fullsized":"2024-05-30T16:58:30.216162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:58:16.952470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.157024+0000","last_change":"2024-05-30T16:58:28.110149+0000","last_active":"2024-05-30T16:58:29.157024+0000","last_peered":"2024-05-30T16:58:29.157024+0000","last_clean":"2024-05-30T16:58:29.157024+0000","last_became_active":"2024-05-30T16:58:28.109562+0000","last_became_peered":"2024-05-30T16:58:28.109562+0000","last_unstale":"2024-05-30T16:58:29.157024+0000","last_undegraded":"2024-05-30T16:58:29.157024+0000","last_fullsized":"2024-05-30T16:58:29.157024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:47.986563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.144938+0000","last_change":"2024-05-30T16:58:29.134694+0000","last_active":"2024-05-30T16:58:29.144938+0000","last_peered":"2024-05-30T16:58:29.144938+0000","last_clean":"2024-05-30T16:58:29.144938+0000","last_became_active":"2024-05-30T16:58:29.129590+0000","last_became_peered":"2024-05-30T16:58:29.129590+0000","last_unstale":"2024-05-30T16:58:29.144938+0000","last_undegraded":"2024-05-30T16:58:29.144938+0000","last_fullsized":"2024-05-30T16:58:29.144938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:57.354731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141960+0000","last_change":"2024-05-30T16:58:28.111410+0000","last_active":"2024-05-30T16:58:30.141960+0000","last_peered":"2024-05-30T16:58:30.141960+0000","last_clean":"2024-05-30T16:58:30.141960+0000","last_became_active":"2024-05-30T16:58:28.111098+0000","last_became_peered":"2024-05-30T16:58:28.111098+0000","last_unstale":"2024-05-30T16:58:30.141960+0000","last_undegraded":"2024-05-30T16:58:30.141960+0000","last_fullsized":"2024-05-30T16:58:30.141960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:02.958884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.136115+0000","last_change":"2024-05-30T16:58:29.127379+0000","last_active":"2024-05-30T16:58:29.136115+0000","last_peered":"2024-05-30T16:58:29.136115+0000","last_clean":"2024-05-30T16:58:29.136115+0000","last_became_active":"2024-05-30T16:58:29.126691+0000","last_became_peered":"2024-05-30T16:58:29.126691+0000","last_unstale":"2024-05-30T16:58:29.136115+0000","last_undegraded":"2024-05-30T16:58:29.136115+0000","last_fullsized":"2024-05-30T16:58:29.136115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:36:37.836774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.124881+0000","last_change":"2024-05-30T16:58:28.102571+0000","last_active":"2024-05-30T16:58:30.124881+0000","last_peered":"2024-05-30T16:58:30.124881+0000","last_clean":"2024-05-30T16:58:30.124881+0000","last_became_active":"2024-05-30T16:58:28.097342+0000","last_became_peered":"2024-05-30T16:58:28.097342+0000","last_unstale":"2024-05-30T16:58:30.124881+0000","last_undegraded":"2024-05-30T16:58:30.124881+0000","last_fullsized":"2024-05-30T16:58:30.124881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:32:17.040752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141900+0000","last_change":"2024-05-30T16:58:29.135084+0000","last_active":"2024-05-30T16:58:30.141900+0000","last_peered":"2024-05-30T16:58:30.141900+0000","last_clean":"2024-05-30T16:58:30.141900+0000","last_became_active":"2024-05-30T16:58:29.134733+0000","last_became_peered":"2024-05-30T16:58:29.134733+0000","last_unstale":"2024-05-30T16:58:30.141900+0000","last_undegraded":"2024-05-30T16:58:30.141900+0000","last_fullsized":"2024-05-30T16:58:30.141900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:43.227674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.107862+0000","last_change":"2024-05-30T16:58:28.107855+0000","last_active":"2024-05-30T16:58:29.107862+0000","last_peered":"2024-05-30T16:58:29.107862+0000","last_clean":"2024-05-30T16:58:29.107862+0000","last_became_active":"2024-05-30T16:58:28.106737+0000","last_became_peered":"2024-05-30T16:58:28.106737+0000","last_unstale":"2024-05-30T16:58:29.107862+0000","last_undegraded":"2024-05-30T16:58:29.107862+0000","last_fullsized":"2024-05-30T16:58:29.107862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:59.119587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.141681+0000","last_change":"2024-05-30T16:58:29.132469+0000","last_active":"2024-05-30T16:58:30.141681+0000","last_peered":"2024-05-30T16:58:30.141681+0000","last_clean":"2024-05-30T16:58:30.141681+0000","last_became_active":"2024-05-30T16:58:29.132137+0000","last_became_peered":"2024-05-30T16:58:29.132137+0000","last_unstale":"2024-05-30T16:58:30.141681+0000","last_undegraded":"2024-05-30T16:58:30.141681+0000","last_fullsized":"2024-05-30T16:58:30.141681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:25.159552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.159131+0000","last_change":"2024-05-30T16:58:28.107196+0000","last_active":"2024-05-30T16:58:29.159131+0000","last_peered":"2024-05-30T16:58:29.159131+0000","last_clean":"2024-05-30T16:58:29.159131+0000","last_became_active":"2024-05-30T16:58:28.102125+0000","last_became_peered":"2024-05-30T16:58:28.102125+0000","last_unstale":"2024-05-30T16:58:29.159131+0000","last_undegraded":"2024-05-30T16:58:29.159131+0000","last_fullsized":"2024-05-30T16:58:29.159131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:31.419109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.122143+0000","last_change":"2024-05-30T16:58:29.118801+0000","last_active":"2024-05-30T16:58:29.122143+0000","last_peered":"2024-05-30T16:58:29.122143+0000","last_clean":"2024-05-30T16:58:29.122143+0000","last_became_active":"2024-05-30T16:58:29.116528+0000","last_became_peered":"2024-05-30T16:58:29.116528+0000","last_unstale":"2024-05-30T16:58:29.122143+0000","last_undegraded":"2024-05-30T16:58:29.122143+0000","last_fullsized":"2024-05-30T16:58:29.122143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:17:12.660722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174405+0000","last_change":"2024-05-30T16:58:28.106524+0000","last_active":"2024-05-30T16:58:30.174405+0000","last_peered":"2024-05-30T16:58:30.174405+0000","last_clean":"2024-05-30T16:58:30.174405+0000","last_became_active":"2024-05-30T16:58:28.105165+0000","last_became_peered":"2024-05-30T16:58:28.105165+0000","last_unstale":"2024-05-30T16:58:30.174405+0000","last_undegraded":"2024-05-30T16:58:30.174405+0000","last_fullsized":"2024-05-30T16:58:30.174405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:05:12.760513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.174149+0000","last_change":"2024-05-30T16:58:29.128564+0000","last_active":"2024-05-30T16:58:30.174149+0000","last_peered":"2024-05-30T16:58:30.174149+0000","last_clean":"2024-05-30T16:58:30.174149+0000","last_became_active":"2024-05-30T16:58:29.128258+0000","last_became_peered":"2024-05-30T16:58:29.128258+0000","last_unstale":"2024-05-30T16:58:30.174149+0000","last_undegraded":"2024-05-30T16:58:30.174149+0000","last_fullsized":"2024-05-30T16:58:30.174149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:40:44.960616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.156522+0000","last_change":"2024-05-30T16:58:28.111737+0000","last_active":"2024-05-30T16:58:29.156522+0000","last_peered":"2024-05-30T16:58:29.156522+0000","last_clean":"2024-05-30T16:58:29.156522+0000","last_became_active":"2024-05-30T16:58:28.109345+0000","last_became_peered":"2024-05-30T16:58:28.109345+0000","last_unstale":"2024-05-30T16:58:29.156522+0000","last_undegraded":"2024-05-30T16:58:29.156522+0000","last_fullsized":"2024-05-30T16:58:29.156522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:46:50.840179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216407+0000","last_change":"2024-05-30T16:58:29.142398+0000","last_active":"2024-05-30T16:58:30.216407+0000","last_peered":"2024-05-30T16:58:30.216407+0000","last_clean":"2024-05-30T16:58:30.216407+0000","last_became_active":"2024-05-30T16:58:29.142103+0000","last_became_peered":"2024-05-30T16:58:29.142103+0000","last_unstale":"2024-05-30T16:58:30.216407+0000","last_undegraded":"2024-05-30T16:58:30.216407+0000","last_fullsized":"2024-05-30T16:58:30.216407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:32.109950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.130579+0000","last_change":"2024-05-30T16:58:28.106755+0000","last_active":"2024-05-30T16:58:29.130579+0000","last_peered":"2024-05-30T16:58:29.130579+0000","last_clean":"2024-05-30T16:58:29.130579+0000","last_became_active":"2024-05-30T16:58:28.106425+0000","last_became_peered":"2024-05-30T16:58:28.106425+0000","last_unstale":"2024-05-30T16:58:29.130579+0000","last_undegraded":"2024-05-30T16:58:29.130579+0000","last_fullsized":"2024-05-30T16:58:29.130579+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:50:27.592384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128345+0000","last_change":"2024-05-30T16:58:29.145344+0000","last_active":"2024-05-30T16:58:30.128345+0000","last_peered":"2024-05-30T16:58:30.128345+0000","last_clean":"2024-05-30T16:58:30.128345+0000","last_became_active":"2024-05-30T16:58:29.142645+0000","last_became_peered":"2024-05-30T16:58:29.142645+0000","last_unstale":"2024-05-30T16:58:30.128345+0000","last_undegraded":"2024-05-30T16:58:30.128345+0000","last_fullsized":"2024-05-30T16:58:30.128345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:43.227674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.124889+0000","last_change":"2024-05-30T16:58:28.105411+0000","last_active":"2024-05-30T16:58:30.124889+0000","last_peered":"2024-05-30T16:58:30.124889+0000","last_clean":"2024-05-30T16:58:30.124889+0000","last_became_active":"2024-05-30T16:58:28.105098+0000","last_became_peered":"2024-05-30T16:58:28.105098+0000","last_unstale":"2024-05-30T16:58:30.124889+0000","last_undegraded":"2024-05-30T16:58:30.124889+0000","last_fullsized":"2024-05-30T16:58:30.124889+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:44:17.323649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.128742+0000","last_change":"2024-05-30T16:58:29.120696+0000","last_active":"2024-05-30T16:58:29.128742+0000","last_peered":"2024-05-30T16:58:29.128742+0000","last_clean":"2024-05-30T16:58:29.128742+0000","last_became_active":"2024-05-30T16:58:29.120145+0000","last_became_peered":"2024-05-30T16:58:29.120145+0000","last_unstale":"2024-05-30T16:58:29.128742+0000","last_undegraded":"2024-05-30T16:58:29.128742+0000","last_fullsized":"2024-05-30T16:58:29.128742+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:10.642499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.128536+0000","last_change":"2024-05-30T16:58:28.112852+0000","last_active":"2024-05-30T16:58:30.128536+0000","last_peered":"2024-05-30T16:58:30.128536+0000","last_clean":"2024-05-30T16:58:30.128536+0000","last_became_active":"2024-05-30T16:58:28.112553+0000","last_became_peered":"2024-05-30T16:58:28.112553+0000","last_unstale":"2024-05-30T16:58:30.128536+0000","last_undegraded":"2024-05-30T16:58:30.128536+0000","last_fullsized":"2024-05-30T16:58:30.128536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:14:08.346680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127338+0000","last_change":"2024-05-30T16:58:29.135993+0000","last_active":"2024-05-30T16:58:30.127338+0000","last_peered":"2024-05-30T16:58:30.127338+0000","last_clean":"2024-05-30T16:58:30.127338+0000","last_became_active":"2024-05-30T16:58:29.131946+0000","last_became_peered":"2024-05-30T16:58:29.131946+0000","last_unstale":"2024-05-30T16:58:30.127338+0000","last_undegraded":"2024-05-30T16:58:30.127338+0000","last_fullsized":"2024-05-30T16:58:30.127338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:49.132327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.127500+0000","last_change":"2024-05-30T16:58:28.109200+0000","last_active":"2024-05-30T16:58:30.127500+0000","last_peered":"2024-05-30T16:58:30.127500+0000","last_clean":"2024-05-30T16:58:30.127500+0000","last_became_active":"2024-05-30T16:58:28.108518+0000","last_became_peered":"2024-05-30T16:58:28.108518+0000","last_unstale":"2024-05-30T16:58:30.127500+0000","last_undegraded":"2024-05-30T16:58:30.127500+0000","last_fullsized":"2024-05-30T16:58:30.127500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:02.958884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.132950+0000","last_change":"2024-05-30T16:58:29.129344+0000","last_active":"2024-05-30T16:58:29.132950+0000","last_peered":"2024-05-30T16:58:29.132950+0000","last_clean":"2024-05-30T16:58:29.132950+0000","last_became_active":"2024-05-30T16:58:29.126344+0000","last_became_peered":"2024-05-30T16:58:29.126344+0000","last_unstale":"2024-05-30T16:58:29.132950+0000","last_undegraded":"2024-05-30T16:58:29.132950+0000","last_fullsized":"2024-05-30T16:58:29.132950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:02:40.890296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.125054+0000","last_change":"2024-05-30T16:58:28.104915+0000","last_active":"2024-05-30T16:58:30.125054+0000","last_peered":"2024-05-30T16:58:30.125054+0000","last_clean":"2024-05-30T16:58:30.125054+0000","last_became_active":"2024-05-30T16:58:28.104530+0000","last_became_peered":"2024-05-30T16:58:28.104530+0000","last_unstale":"2024-05-30T16:58:30.125054+0000","last_undegraded":"2024-05-30T16:58:30.125054+0000","last_fullsized":"2024-05-30T16:58:30.125054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:17.172719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.217588+0000","last_change":"2024-05-30T16:58:29.134490+0000","last_active":"2024-05-30T16:58:30.217588+0000","last_peered":"2024-05-30T16:58:30.217588+0000","last_clean":"2024-05-30T16:58:30.217588+0000","last_became_active":"2024-05-30T16:58:29.133664+0000","last_became_peered":"2024-05-30T16:58:29.133664+0000","last_unstale":"2024-05-30T16:58:30.217588+0000","last_undegraded":"2024-05-30T16:58:30.217588+0000","last_fullsized":"2024-05-30T16:58:30.217588+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:29:17.255770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142543+0000","last_change":"2024-05-30T16:58:28.105396+0000","last_active":"2024-05-30T16:58:30.142543+0000","last_peered":"2024-05-30T16:58:30.142543+0000","last_clean":"2024-05-30T16:58:30.142543+0000","last_became_active":"2024-05-30T16:58:28.105122+0000","last_became_peered":"2024-05-30T16:58:28.105122+0000","last_unstale":"2024-05-30T16:58:30.142543+0000","last_undegraded":"2024-05-30T16:58:30.142543+0000","last_fullsized":"2024-05-30T16:58:30.142543+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:25:06.422803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.136774+0000","last_change":"2024-05-30T16:58:29.129226+0000","last_active":"2024-05-30T16:58:29.136774+0000","last_peered":"2024-05-30T16:58:29.136774+0000","last_clean":"2024-05-30T16:58:29.136774+0000","last_became_active":"2024-05-30T16:58:29.128958+0000","last_became_peered":"2024-05-30T16:58:29.128958+0000","last_unstale":"2024-05-30T16:58:29.136774+0000","last_undegraded":"2024-05-30T16:58:29.136774+0000","last_fullsized":"2024-05-30T16:58:29.136774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:57.354731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.144701+0000","last_change":"2024-05-30T16:58:28.112685+0000","last_active":"2024-05-30T16:58:30.144701+0000","last_peered":"2024-05-30T16:58:30.144701+0000","last_clean":"2024-05-30T16:58:30.144701+0000","last_became_active":"2024-05-30T16:58:28.112391+0000","last_became_peered":"2024-05-30T16:58:28.112391+0000","last_unstale":"2024-05-30T16:58:30.144701+0000","last_undegraded":"2024-05-30T16:58:30.144701+0000","last_fullsized":"2024-05-30T16:58:30.144701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:31:24.605145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173927+0000","last_change":"2024-05-30T16:58:29.144167+0000","last_active":"2024-05-30T16:58:30.173927+0000","last_peered":"2024-05-30T16:58:30.173927+0000","last_clean":"2024-05-30T16:58:30.173927+0000","last_became_active":"2024-05-30T16:58:29.143882+0000","last_became_peered":"2024-05-30T16:58:29.143882+0000","last_unstale":"2024-05-30T16:58:30.173927+0000","last_undegraded":"2024-05-30T16:58:30.173927+0000","last_fullsized":"2024-05-30T16:58:30.173927+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:39:55.738510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173394+0000","last_change":"2024-05-30T16:58:28.107379+0000","last_active":"2024-05-30T16:58:30.173394+0000","last_peered":"2024-05-30T16:58:30.173394+0000","last_clean":"2024-05-30T16:58:30.173394+0000","last_became_active":"2024-05-30T16:58:28.107003+0000","last_became_peered":"2024-05-30T16:58:28.107003+0000","last_unstale":"2024-05-30T16:58:30.173394+0000","last_undegraded":"2024-05-30T16:58:30.173394+0000","last_fullsized":"2024-05-30T16:58:30.173394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:15:24.374426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216771+0000","last_change":"2024-05-30T16:58:29.131232+0000","last_active":"2024-05-30T16:58:30.216771+0000","last_peered":"2024-05-30T16:58:30.216771+0000","last_clean":"2024-05-30T16:58:30.216771+0000","last_became_active":"2024-05-30T16:58:29.130843+0000","last_became_peered":"2024-05-30T16:58:29.130843+0000","last_unstale":"2024-05-30T16:58:30.216771+0000","last_undegraded":"2024-05-30T16:58:30.216771+0000","last_fullsized":"2024-05-30T16:58:30.216771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:56:26.493747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173907+0000","last_change":"2024-05-30T16:58:28.097741+0000","last_active":"2024-05-30T16:58:30.173907+0000","last_peered":"2024-05-30T16:58:30.173907+0000","last_clean":"2024-05-30T16:58:30.173907+0000","last_became_active":"2024-05-30T16:58:28.097436+0000","last_became_peered":"2024-05-30T16:58:28.097436+0000","last_unstale":"2024-05-30T16:58:30.173907+0000","last_undegraded":"2024-05-30T16:58:30.173907+0000","last_fullsized":"2024-05-30T16:58:30.173907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:42:25.356966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.119180+0000","last_change":"2024-05-30T16:58:29.115480+0000","last_active":"2024-05-30T16:58:29.119180+0000","last_peered":"2024-05-30T16:58:29.119180+0000","last_clean":"2024-05-30T16:58:29.119180+0000","last_became_active":"2024-05-30T16:58:29.114866+0000","last_became_peered":"2024-05-30T16:58:29.114866+0000","last_unstale":"2024-05-30T16:58:29.119180+0000","last_undegraded":"2024-05-30T16:58:29.119180+0000","last_fullsized":"2024-05-30T16:58:29.119180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:34:54.389890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.126207+0000","last_change":"2024-05-30T16:58:28.106856+0000","last_active":"2024-05-30T16:58:29.126207+0000","last_peered":"2024-05-30T16:58:29.126207+0000","last_clean":"2024-05-30T16:58:29.126207+0000","last_became_active":"2024-05-30T16:58:28.106562+0000","last_became_peered":"2024-05-30T16:58:28.106562+0000","last_unstale":"2024-05-30T16:58:29.126207+0000","last_undegraded":"2024-05-30T16:58:29.126207+0000","last_fullsized":"2024-05-30T16:58:29.126207+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:51:19.426018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.140279+0000","last_change":"2024-05-30T16:58:29.133484+0000","last_active":"2024-05-30T16:58:29.140279+0000","last_peered":"2024-05-30T16:58:29.140279+0000","last_clean":"2024-05-30T16:58:29.140279+0000","last_became_active":"2024-05-30T16:58:29.133096+0000","last_became_peered":"2024-05-30T16:58:29.133096+0000","last_unstale":"2024-05-30T16:58:29.140279+0000","last_undegraded":"2024-05-30T16:58:29.140279+0000","last_fullsized":"2024-05-30T16:58:29.140279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:35:44.398093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.216062+0000","last_change":"2024-05-30T16:58:28.110991+0000","last_active":"2024-05-30T16:58:30.216062+0000","last_peered":"2024-05-30T16:58:30.216062+0000","last_clean":"2024-05-30T16:58:30.216062+0000","last_became_active":"2024-05-30T16:58:28.110572+0000","last_became_peered":"2024-05-30T16:58:28.110572+0000","last_unstale":"2024-05-30T16:58:30.216062+0000","last_undegraded":"2024-05-30T16:58:30.216062+0000","last_fullsized":"2024-05-30T16:58:30.216062+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:14.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.141826+0000","last_change":"2024-05-30T16:58:29.139570+0000","last_active":"2024-05-30T16:58:29.141826+0000","last_peered":"2024-05-30T16:58:29.141826+0000","last_clean":"2024-05-30T16:58:29.141826+0000","last_became_active":"2024-05-30T16:58:29.139242+0000","last_became_peered":"2024-05-30T16:58:29.139242+0000","last_unstale":"2024-05-30T16:58:29.141826+0000","last_undegraded":"2024-05-30T16:58:29.141826+0000","last_fullsized":"2024-05-30T16:58:29.141826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:32:32.154520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173818+0000","last_change":"2024-05-30T16:58:28.101054+0000","last_active":"2024-05-30T16:58:30.173818+0000","last_peered":"2024-05-30T16:58:30.173818+0000","last_clean":"2024-05-30T16:58:30.173818+0000","last_became_active":"2024-05-30T16:58:28.100722+0000","last_became_peered":"2024-05-30T16:58:28.100722+0000","last_unstale":"2024-05-30T16:58:30.173818+0000","last_undegraded":"2024-05-30T16:58:30.173818+0000","last_fullsized":"2024-05-30T16:58:30.173818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:43:10.868544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.173250+0000","last_change":"2024-05-30T16:58:29.136069+0000","last_active":"2024-05-30T16:58:30.173250+0000","last_peered":"2024-05-30T16:58:30.173250+0000","last_clean":"2024-05-30T16:58:30.173250+0000","last_became_active":"2024-05-30T16:58:29.135758+0000","last_became_peered":"2024-05-30T16:58:29.135758+0000","last_unstale":"2024-05-30T16:58:30.173250+0000","last_undegraded":"2024-05-30T16:58:30.173250+0000","last_fullsized":"2024-05-30T16:58:30.173250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:49:32.109950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.215352+0000","last_change":"2024-05-30T16:58:28.106102+0000","last_active":"2024-05-30T16:58:30.215352+0000","last_peered":"2024-05-30T16:58:30.215352+0000","last_clean":"2024-05-30T16:58:30.215352+0000","last_became_active":"2024-05-30T16:58:28.105833+0000","last_became_peered":"2024-05-30T16:58:28.105833+0000","last_unstale":"2024-05-30T16:58:30.215352+0000","last_undegraded":"2024-05-30T16:58:30.215352+0000","last_fullsized":"2024-05-30T16:58:30.215352+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:59:16.277759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-30T16:58:30.142195+0000","last_change":"2024-05-30T16:58:29.130395+0000","last_active":"2024-05-30T16:58:30.142195+0000","last_peered":"2024-05-30T16:58:30.142195+0000","last_clean":"2024-05-30T16:58:30.142195+0000","last_became_active":"2024-05-30T16:58:29.130094+0000","last_became_peered":"2024-05-30T16:58:29.130094+0000","last_unstale":"2024-05-30T16:58:30.142195+0000","last_undegraded":"2024-05-30T16:58:30.142195+0000","last_fullsized":"2024-05-30T16:58:30.142195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:28.070194+0000","last_clean_scrub_stamp":"2024-05-30T16:58:28.070194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:36:18.504283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.151689+0000","last_change":"2024-05-30T16:58:28.104823+0000","last_active":"2024-05-30T16:58:29.151689+0000","last_peered":"2024-05-30T16:58:29.151689+0000","last_clean":"2024-05-30T16:58:29.151689+0000","last_became_active":"2024-05-30T16:58:28.104492+0000","last_became_peered":"2024-05-30T16:58:28.104492+0000","last_unstale":"2024-05-30T16:58:29.151689+0000","last_undegraded":"2024-05-30T16:58:29.151689+0000","last_fullsized":"2024-05-30T16:58:29.151689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:58:27.055632+0000","last_clean_scrub_stamp":"2024-05-30T16:58:27.055632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:50:27.592384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":299,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-30T16:58:29.108772+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T16:58:29.108772+0000","last_peered":"2024-05-30T16:58:29.108772+0000","last_clean":"2024-05-30T16:58:29.108772+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T16:58:29.108772+0000","last_undegraded":"2024-05-30T16:58:29.108772+0000","last_fullsized":"2024-05-30T16:58:29.108772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509621,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126584,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245256,"statfs":{"total":96636764160,"available":96507142144,"internally_reserved":0,"allocated":1085440,"data_stored":463756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":128514502},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509621,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86312,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285528,"statfs":{"total":96636764160,"available":96548380672,"internally_reserved":0,"allocated":1073152,"data_stored":463062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509622,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126596,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245244,"statfs":{"total":96636764160,"available":96507129856,"internally_reserved":0,"allocated":1097728,"data_stored":471030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509622,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140468,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231372,"statfs":{"total":96636764160,"available":96492924928,"internally_reserved":0,"allocated":1089536,"data_stored":470000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509621,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104384,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267456,"statfs":{"total":96636764160,"available":96529874944,"internally_reserved":0,"allocated":1097728,"data_stored":469612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542324,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122996,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248844,"statfs":{"total":96636764160,"available":96510816256,"internally_reserved":0,"allocated":1671168,"data_stored":1053500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542325,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76568,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295272,"statfs":{"total":96636764160,"available":96558358528,"internally_reserved":0,"allocated":1060864,"data_stored":456231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542325,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77148,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294692,"statfs":{"total":96636764160,"available":96557764608,"internally_reserved":0,"allocated":1654784,"data_stored":1046599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T16:58:34.136 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T16:58:34.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-30T16:58:34.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.394+0000 7fd5ab200640 1 Processor -- start 2024-05-30T16:58:34.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.394+0000 7fd5ab200640 1 -- start start 2024-05-30T16:58:34.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.396+0000 7fd5ab200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5a40d0640 0x7fd5a415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.396+0000 7fd5ab200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5a40d0060 con 0x7fd5a40b32c0 2024-05-30T16:58:34.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.396+0000 7fd5ab200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5a415cb10 con 0x7fd5a40d0a20 2024-05-30T16:58:34.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.396+0000 7fd5ab200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd5a40adbb0 con 0x7fd5a40d0640 2024-05-30T16:58:34.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.396+0000 7fd5a9200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd5a40b32c0 0x7fd5a40b3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59534/0 (socket says 172.21.15.67:59534) 2024-05-30T16:58:34.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.396+0000 7fd5a9200640 1 -- 172.21.15.67:0/1147426817 learned_addr learned my addr 172.21.15.67:0/1147426817 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:34.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070745339 0 0) 0x7fd5a40d0060 con 0x7fd5a40b32c0 2024-05-30T16:58:34.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3e00640 1 --2- 172.21.15.67:0/1147426817 >> v2:172.21.15.169:3300/0 conn(0x7fd5a40d0640 0x7fd5a415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd588003520 con 0x7fd5a40b32c0 2024-05-30T16:58:34.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 123547234 0 0) 0x7fd588003520 con 0x7fd5a40b32c0 2024-05-30T16:58:34.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 >> v2:172.21.15.169:3300/0 conn(0x7fd5a40d0640 msgr2=0x7fd5a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:34.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 --2- 172.21.15.67:0/1147426817 >> v2:172.21.15.169:3300/0 conn(0x7fd5a40d0640 0x7fd5a415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 >> v1:172.21.15.169:6789/0 conn(0x7fd5a40d0a20 legacy=0x7fd5a40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a40ad5f0 con 0x7fd5a40b32c0 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fd59c0019f0 con 0x7fd5a40b32c0 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 >> v1:172.21.15.67:6789/0 conn(0x7fd5a40b32c0 legacy=0x7fd5a40b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd588003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd588004800 0x7fd588004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd588005210 0x7fd58800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd58800c130 con 0x7fd588003a00 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd58800c2e0 con 0x7fd588005210 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.397+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd58800c4e0 con 0x7fd588004800 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd5a3e00640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd588003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd5a9200640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd588005210 0x7fd58800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd5a3e00640 1 -- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd588004800 msgr2=0x7fd588004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:34.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd59be00640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd588004800 0x7fd588004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd5a3e00640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd588004800 0x7fd588004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd5a3e00640 1 -- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd588005210 msgr2=0x7fd58800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:34.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd5a3e00640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd588005210 0x7fd58800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd5a3e00640 1 -- 172.21.15.67:0/1147426817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd588003520 con 0x7fd588003a00 2024-05-30T16:58:34.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd5a9200640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd588005210 0x7fd58800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:34.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd5a9200640 1 -- 172.21.15.67:0/1147426817 reap_dead start 2024-05-30T16:58:34.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.398+0000 7fd59be00640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd588004800 0x7fd588004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.399+0000 7fd5a3e00640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd588003e20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd594009a90 tx=0x7fd588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb5b3c867874936 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594036070 con 0x7fd588003a00 2024-05-30T16:58:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd594031400 con 0x7fd588003a00 2024-05-30T16:58:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5a3400640 1 -- 172.21.15.67:0/1147426817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594009690 con 0x7fd588003a00 2024-05-30T16:58:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5ab200640 1 -- 172.21.15.67:0/1147426817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 msgr2=0x7fd588003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5ab200640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd588003e20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd594009a90 tx=0x7fd588005830 comp rx=0 tx=0).stop 2024-05-30T16:58:34.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5ab200640 1 -- 172.21.15.67:0/1147426817 shutdown_connections 2024-05-30T16:58:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5ab200640 1 --2- 172.21.15.67:0/1147426817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd588003e20 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5ab200640 1 -- 172.21.15.67:0/1147426817 >> 172.21.15.67:0/1147426817 conn(0x7fd5a406bd20 msgr2=0x7fd5a406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5ab200640 1 -- 172.21.15.67:0/1147426817 shutdown_connections 2024-05-30T16:58:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.400+0000 7fd5ab200640 1 -- 172.21.15.67:0/1147426817 wait complete. 2024-05-30T16:58:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.402+0000 7fd5ab200640 1 Processor -- start 2024-05-30T16:58:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.402+0000 7fd5ab200640 1 -- start start 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.403+0000 7fd5ab200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd5a40b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.403+0000 7fd5ab200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a41f50e0 0x7fd5a41476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.403+0000 7fd5ab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a415b240 0x7fd5a415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.403+0000 7fd5ab200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a415d0b0 con 0x7fd588003a00 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.403+0000 7fd5ab200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a415cdd0 con 0x7fd5a41f50e0 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.403+0000 7fd5ab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a40ad2a0 con 0x7fd5a415b240 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd5a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd5a40b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd5a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd5a40b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50216/0 (socket says 172.21.15.67:50216) 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd5a3e00640 1 -- 172.21.15.67:0/2145123802 learned_addr learned my addr 172.21.15.67:0/2145123802 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd5a9200640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a415b240 0x7fd5a415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd59be00640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a41f50e0 0x7fd5a41476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd5a3e00640 1 -- 172.21.15.67:0/2145123802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a415b240 msgr2=0x7fd5a415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd5a3e00640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a415b240 0x7fd5a415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd5a3e00640 1 -- 172.21.15.67:0/2145123802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a41f50e0 msgr2=0x7fd5a41476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd5a3e00640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a41f50e0 0x7fd5a41476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd5a3e00640 1 -- 172.21.15.67:0/2145123802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a4147c10 con 0x7fd588003a00 2024-05-30T16:58:34.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.404+0000 7fd59be00640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a41f50e0 0x7fd5a41476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.405+0000 7fd5a3e00640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd5a40b18b0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd594009bb0 tx=0x7fd594004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.405+0000 7fd5a0c00640 1 -- 172.21.15.67:0/2145123802 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59403f020 con 0x7fd588003a00 2024-05-30T16:58:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.405+0000 7fd5a0c00640 1 -- 172.21.15.67:0/2145123802 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd594037040 con 0x7fd588003a00 2024-05-30T16:58:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.405+0000 7fd5a0c00640 1 -- 172.21.15.67:0/2145123802 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594036070 con 0x7fd588003a00 2024-05-30T16:58:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.405+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a40b1e90 con 0x7fd588003a00 2024-05-30T16:58:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.405+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a40b2450 con 0x7fd588003a00 2024-05-30T16:58:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.406+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a415bc00 con 0x7fd588003a00 2024-05-30T16:58:34.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.407+0000 7fd5a0c00640 1 -- 172.21.15.67:0/2145123802 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fd594042020 con 0x7fd588003a00 2024-05-30T16:58:34.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.408+0000 7fd5a0c00640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006da90 0x7fd58006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.408+0000 7fd5a0c00640 1 -- 172.21.15.67:0/2145123802 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fd5940aaa50 con 0x7fd588003a00 2024-05-30T16:58:34.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.408+0000 7fd59be00640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006da90 0x7fd58006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.409+0000 7fd59be00640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006da90 0x7fd58006ff50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd5a415dda0 tx=0x7fd58c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:34.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.413+0000 7fd5a0c00640 1 -- 172.21.15.67:0/2145123802 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd594049030 con 0x7fd588003a00 2024-05-30T16:58:34.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.693+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd5a40689e0 con 0x7fd588003a00 2024-05-30T16:58:34.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.698+0000 7fd5a0c00640 1 -- 172.21.15.67:0/2145123802 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fd5a4147c10 con 0x7fd588003a00 2024-05-30T16:58:34.699 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-05-30T16:58:34.699 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T16:58:34.699 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-05-30T16:58:34.700 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-05-30T16:58:34.700 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-30T16:58:34.700 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-05-30T16:58:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006da90 msgr2=0x7fd58006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006da90 0x7fd58006ff50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd5a415dda0 tx=0x7fd58c009040 comp rx=0 tx=0).stop 2024-05-30T16:58:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 msgr2=0x7fd5a40b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd5a40b18b0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fd594009bb0 tx=0x7fd594004730 comp rx=0 tx=0).stop 2024-05-30T16:58:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 shutdown_connections 2024-05-30T16:58:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006da90 0x7fd58006ff50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a415b240 0x7fd5a415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a41f50e0 0x7fd5a41476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 --2- 172.21.15.67:0/2145123802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd588003a00 0x7fd5a40b18b0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 >> 172.21.15.67:0/2145123802 conn(0x7fd5a406bd20 msgr2=0x7fd5a40aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.701+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 shutdown_connections 2024-05-30T16:58:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.702+0000 7fd5ab200640 1 -- 172.21.15.67:0/2145123802 wait complete. 2024-05-30T16:58:34.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T16:58:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.916+0000 7f6657200640 1 Processor -- start 2024-05-30T16:58:34.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.917+0000 7f6657200640 1 -- start start 2024-05-30T16:58:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.918+0000 7f6657200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6658162940 0x7f665806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.918+0000 7f6657200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66580722d0 0x7f66580adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.918+0000 7f6657200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665806f590 con 0x7f6658162d20 2024-05-30T16:58:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.918+0000 7f6657200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f665806f870 con 0x7f6658162940 2024-05-30T16:58:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.918+0000 7f6657200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66580ad3d0 con 0x7f66580722d0 2024-05-30T16:58:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.919+0000 7f6655e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6658162940 0x7f665806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.919+0000 7f6655e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6658162940 0x7f665806eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50228/0 (socket says 172.21.15.67:50228) 2024-05-30T16:58:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.919+0000 7f6655e00640 1 -- 172.21.15.67:0/294807439 learned_addr learned my addr 172.21.15.67:0/294807439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.919+0000 7f6655e00640 1 -- 172.21.15.67:0/294807439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66580722d0 msgr2=0x7f66580adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.919+0000 7f6655e00640 1 --2- 172.21.15.67:0/294807439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66580722d0 0x7f66580adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.919+0000 7f6655e00640 1 -- 172.21.15.67:0/294807439 >> v1:172.21.15.169:6789/0 conn(0x7f6658162d20 legacy=0x7f6658071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.919+0000 7f6655e00640 1 -- 172.21.15.67:0/294807439 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665806dde0 con 0x7f6658162940 2024-05-30T16:58:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.920+0000 7f6655e00640 1 --2- 172.21.15.67:0/294807439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162940 0x7f665806eb00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6648009870 tx=0x7f664802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616be4e3df52406f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.920+0000 7f6655400640 1 -- 172.21.15.67:0/294807439 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648037070 con 0x7f6658162940 2024-05-30T16:58:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.920+0000 7f6655400640 1 -- 172.21.15.67:0/294807439 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664802ae10 con 0x7f6658162940 2024-05-30T16:58:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.921+0000 7f6655400640 1 -- 172.21.15.67:0/294807439 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648032660 con 0x7f6658162940 2024-05-30T16:58:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.921+0000 7f6657200640 1 -- 172.21.15.67:0/294807439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162940 msgr2=0x7f665806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.921+0000 7f6657200640 1 --2- 172.21.15.67:0/294807439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162940 0x7f665806eb00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6648009870 tx=0x7f664802a100 comp rx=0 tx=0).stop 2024-05-30T16:58:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.921+0000 7f6657200640 1 -- 172.21.15.67:0/294807439 shutdown_connections 2024-05-30T16:58:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.921+0000 7f6657200640 1 --2- 172.21.15.67:0/294807439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66580722d0 0x7f66580adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.921+0000 7f6657200640 1 --2- 172.21.15.67:0/294807439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162940 0x7f665806eb00 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.921+0000 7f6657200640 1 -- 172.21.15.67:0/294807439 >> 172.21.15.67:0/294807439 conn(0x7f66580b1fc0 msgr2=0x7f66580b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.921+0000 7f6657200640 1 -- 172.21.15.67:0/294807439 shutdown_connections 2024-05-30T16:58:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.921+0000 7f6657200640 1 -- 172.21.15.67:0/294807439 wait complete. 2024-05-30T16:58:34.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.923+0000 7f6657200640 1 Processor -- start 2024-05-30T16:58:34.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.923+0000 7f6657200640 1 -- start start 2024-05-30T16:58:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.924+0000 7f6657200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66580722d0 0x7f6658165af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.924+0000 7f6657200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6658162940 0x7f66580b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.924+0000 7f6657200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162d20 0x7f66581522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.924+0000 7f6657200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66580ae0f0 con 0x7f6658162d20 2024-05-30T16:58:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.924+0000 7f6657200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f665806f250 con 0x7f66580722d0 2024-05-30T16:58:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.924+0000 7f6657200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66580ae270 con 0x7f6658162940 2024-05-30T16:58:34.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.924+0000 7f6656800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162d20 0x7f66581522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.924+0000 7f6656800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162d20 0x7f66581522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50236/0 (socket says 172.21.15.67:50236) 2024-05-30T16:58:34.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.924+0000 7f6656800640 1 -- 172.21.15.67:0/3525846781 learned_addr learned my addr 172.21.15.67:0/3525846781 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:34.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f664fe00640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6658162940 0x7f66580b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f6656800640 1 -- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6658162940 msgr2=0x7f66580b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f6656800640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6658162940 0x7f66580b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f6655e00640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66580722d0 0x7f6658165af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f6656800640 1 -- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66580722d0 msgr2=0x7f6658165af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f6656800640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66580722d0 0x7f6658165af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f6656800640 1 -- 172.21.15.67:0/3525846781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66581fccd0 con 0x7f6658162d20 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f664fe00640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6658162940 0x7f66580b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f6655e00640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66580722d0 0x7f6658165af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f6656800640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162d20 0x7f66581522e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f665000b9e0 tx=0x7f665000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f664e000640 1 -- 172.21.15.67:0/3525846781 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650019070 con 0x7f6658162d20 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f664e000640 1 -- 172.21.15.67:0/3525846781 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665000cac0 con 0x7f6658162d20 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f664e000640 1 -- 172.21.15.67:0/3525846781 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650014440 con 0x7f6658162d20 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.925+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6658165010 con 0x7f6658162d20 2024-05-30T16:58:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.926+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6658166030 con 0x7f6658162d20 2024-05-30T16:58:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.927+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66580689e0 con 0x7f6658162d20 2024-05-30T16:58:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.928+0000 7f664e000640 1 -- 172.21.15.67:0/3525846781 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f665001c070 con 0x7f6658162d20 2024-05-30T16:58:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.929+0000 7f664e000640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663406da60 0x7f663406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.929+0000 7f664e000640 1 -- 172.21.15.67:0/3525846781 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f665008a700 con 0x7f6658162d20 2024-05-30T16:58:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.929+0000 7f6655e00640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663406da60 0x7f663406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.930+0000 7f6655e00640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663406da60 0x7f663406ff20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6648034000 tx=0x7f664802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:34.933+0000 7f664e000640 1 -- 172.21.15.67:0/3525846781 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f665005a290 con 0x7f6658162d20 2024-05-30T16:58:35.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.179+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66580b4e50 con 0x7f6658162d20 2024-05-30T16:58:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.182+0000 7f664e000640 1 -- 172.21.15.67:0/3525846781 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5189 (secure 0 0 0) 0x7f6650059b90 con 0x7f6658162d20 2024-05-30T16:58:35.182 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:58:35.183 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1054215955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1054215955},{"type":"v1","addr":"172.21.15.67:6837","nonce":1054215955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":5591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1236503792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1236503792},{"type":"v1","addr":"172.21.15.169:6835","nonce":1236503792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/4047293074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4047293074},{"type":"v1","addr":"172.21.15.67:6836","nonce":4047293074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:58:28.573056+0000","modified":"2024-05-30T16:58:31.656482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7398":{"gid":7398,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/2733959172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2733959172},{"type":"v1","addr":"172.21.15.169:6834","nonce":2733959172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7398,"qdb_cluster":[7398]},"id":6}]} 2024-05-30T16:58:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 55 2024-05-30T16:58:35.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.184+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663406da60 msgr2=0x7f663406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.184+0000 7f6657200640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663406da60 0x7f663406ff20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6648034000 tx=0x7f664802e000 comp rx=0 tx=0).stop 2024-05-30T16:58:35.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162d20 msgr2=0x7f66581522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162d20 0x7f66581522e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f665000b9e0 tx=0x7f665000bea0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 shutdown_connections 2024-05-30T16:58:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663406da60 0x7f663406ff20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6658162d20 0x7f66581522e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6658162940 0x7f66580b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 --2- 172.21.15.67:0/3525846781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66580722d0 0x7f6658165af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 >> 172.21.15.67:0/3525846781 conn(0x7f66580b1fc0 msgr2=0x7f66580b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 shutdown_connections 2024-05-30T16:58:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.185+0000 7f6657200640 1 -- 172.21.15.67:0/3525846781 wait complete. 2024-05-30T16:58:35.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-30T16:58:28.573056+0000', 'modified': '2024-05-30T16:58:31.656482+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7398': {'gid': 7398, 'name': 'd', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6834/2733959172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6833', 'nonce': 2733959172}, {'type': 'v1', 'addr': '172.21.15.169:6834', 'nonce': 2733959172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7398, 'qdb_cluster': [7398]} 2024-05-30T16:58:35.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-30T16:58:35.609 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-30T16:58:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.829+0000 7fbe26e00640 1 Processor -- start 2024-05-30T16:58:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.829+0000 7fbe26e00640 1 -- start start 2024-05-30T16:58:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.831+0000 7fbe26e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbe200722d0 0x7fbe2016d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.831+0000 7fbe26e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2006f590 con 0x7fbe20162d20 2024-05-30T16:58:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.831+0000 7fbe26e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2006f870 con 0x7fbe20162940 2024-05-30T16:58:35.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.831+0000 7fbe26e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbe200ad3d0 con 0x7fbe200722d0 2024-05-30T16:58:35.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.831+0000 7fbe25000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbe20162d20 0x7fbe20071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59536/0 (socket says 172.21.15.67:59536) 2024-05-30T16:58:35.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.831+0000 7fbe25000640 1 -- 172.21.15.67:0/2079948934 learned_addr learned my addr 172.21.15.67:0/2079948934 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.831+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1619694662 0 0) 0x7fbe2006f590 con 0x7fbe20162d20 2024-05-30T16:58:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.831+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe08003520 con 0x7fbe20162d20 2024-05-30T16:58:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.831+0000 7fbe26400640 1 --2- 172.21.15.67:0/2079948934 >> v2:172.21.15.169:3300/0 conn(0x7fbe200722d0 0x7fbe2016d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3033935693 0 0) 0x7fbe08003520 con 0x7fbe20162d20 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 >> v2:172.21.15.169:3300/0 conn(0x7fbe200722d0 msgr2=0x7fbe2016d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 --2- 172.21.15.67:0/2079948934 >> v2:172.21.15.169:3300/0 conn(0x7fbe200722d0 0x7fbe2016d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 >> v1:172.21.15.169:6789/0 conn(0x7fbe20162940 legacy=0x7fbe2006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2006f210 con 0x7fbe20162d20 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fbe0c001670 con 0x7fbe20162d20 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 >> v1:172.21.15.67:6789/0 conn(0x7fbe20162d20 legacy=0x7fbe20071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08003a00 0x7fbe08003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08004800 0x7fbe08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe080053c0 0x7fbe0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200ad3d0 con 0x7fbe08003a00 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0800c2a0 con 0x7fbe080053c0 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0800c4a0 con 0x7fbe08004800 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.832+0000 7fbe25a00640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08003a00 0x7fbe08003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.833+0000 7fbe25000640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08004800 0x7fbe08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.833+0000 7fbe25a00640 1 -- 172.21.15.67:0/2079948934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08004800 msgr2=0x7fbe08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.833+0000 7fbe25a00640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08004800 0x7fbe08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.833+0000 7fbe25a00640 1 -- 172.21.15.67:0/2079948934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe080053c0 msgr2=0x7fbe0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.833+0000 7fbe25a00640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe080053c0 0x7fbe0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.833+0000 7fbe25a00640 1 -- 172.21.15.67:0/2079948934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe08003520 con 0x7fbe08003a00 2024-05-30T16:58:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.833+0000 7fbe26400640 1 -- 172.21.15.67:0/2079948934 reap_dead start 2024-05-30T16:58:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.833+0000 7fbe25000640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08004800 0x7fbe08004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.834+0000 7fbe25a00640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08003a00 0x7fbe08003e20 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fbe14001e50 tx=0x7fbe08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11788b4dbc21821f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.834+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14001a70 con 0x7fbe08003a00 2024-05-30T16:58:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.834+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe14003970 con 0x7fbe08003a00 2024-05-30T16:58:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.834+0000 7fbe1fe00640 1 -- 172.21.15.67:0/2079948934 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14003d60 con 0x7fbe08003a00 2024-05-30T16:58:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.835+0000 7fbe26e00640 1 -- 172.21.15.67:0/2079948934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08003a00 msgr2=0x7fbe08003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.835+0000 7fbe26e00640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08003a00 0x7fbe08003e20 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fbe14001e50 tx=0x7fbe08005830 comp rx=0 tx=0).stop 2024-05-30T16:58:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.835+0000 7fbe26e00640 1 -- 172.21.15.67:0/2079948934 shutdown_connections 2024-05-30T16:58:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.835+0000 7fbe26e00640 1 --2- 172.21.15.67:0/2079948934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe08003a00 0x7fbe08003e20 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.835+0000 7fbe26e00640 1 -- 172.21.15.67:0/2079948934 >> 172.21.15.67:0/2079948934 conn(0x7fbe200b1fc0 msgr2=0x7fbe200b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.835+0000 7fbe26e00640 1 -- 172.21.15.67:0/2079948934 shutdown_connections 2024-05-30T16:58:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.835+0000 7fbe26e00640 1 -- 172.21.15.67:0/2079948934 wait complete. 2024-05-30T16:58:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.837+0000 7fbe26e00640 1 Processor -- start 2024-05-30T16:58:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.837+0000 7fbe26e00640 1 -- start start 2024-05-30T16:58:35.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe26e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe08003a00 0x7fbe200ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe26e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe2015eea0 0x7fbe200b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe26e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe200ce520 0x7fbe20154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe26e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2006ec80 con 0x7fbe200ce520 2024-05-30T16:58:35.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe26e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2006eb00 con 0x7fbe08003a00 2024-05-30T16:58:35.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe26e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2006ee00 con 0x7fbe2015eea0 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe26400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe200ce520 0x7fbe20154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe26400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe200ce520 0x7fbe20154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50260/0 (socket says 172.21.15.67:50260) 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe26400640 1 -- 172.21.15.67:0/1960814188 learned_addr learned my addr 172.21.15.67:0/1960814188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe25000640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe2015eea0 0x7fbe200b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.838+0000 7fbe25a00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe08003a00 0x7fbe200ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe26400640 1 -- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe2015eea0 msgr2=0x7fbe200b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe26400640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe2015eea0 0x7fbe200b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe26400640 1 -- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe08003a00 msgr2=0x7fbe200ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe26400640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe08003a00 0x7fbe200ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe26400640 1 -- 172.21.15.67:0/1960814188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe200cd1e0 con 0x7fbe200ce520 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe25000640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe2015eea0 0x7fbe200b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe25a00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe08003a00 0x7fbe200ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe26400640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe200ce520 0x7fbe20154250 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fbe20071430 tx=0x7fbe18007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe1d600640 1 -- 172.21.15.67:0/1960814188 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18018070 con 0x7fbe200ce520 2024-05-30T16:58:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe1d600640 1 -- 172.21.15.67:0/1960814188 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe18004240 con 0x7fbe200ce520 2024-05-30T16:58:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe1d600640 1 -- 172.21.15.67:0/1960814188 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18003610 con 0x7fbe200ce520 2024-05-30T16:58:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe26e00640 1 -- 172.21.15.67:0/1960814188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe200cd480 con 0x7fbe200ce520 2024-05-30T16:58:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.839+0000 7fbe26e00640 1 -- 172.21.15.67:0/1960814188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe200cda40 con 0x7fbe200ce520 2024-05-30T16:58:35.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.842+0000 7fbe1d600640 1 -- 172.21.15.67:0/1960814188 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7fbe1801b070 con 0x7fbe200ce520 2024-05-30T16:58:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.842+0000 7fbe1d600640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe0006da60 0x7fbe0006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.843+0000 7fbe1d600640 1 -- 172.21.15.67:0/1960814188 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fbe18089e80 con 0x7fbe200ce520 2024-05-30T16:58:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.843+0000 7fbe25a00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe0006da60 0x7fbe0006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.843+0000 7fbe2b211740 1 Processor -- start 2024-05-30T16:58:35.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.843+0000 7fbe2b211740 1 -- start start 2024-05-30T16:58:35.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.844+0000 7fbe25a00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe0006da60 0x7fbe0006ff20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbe14032000 tx=0x7fbe1402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.844+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a0f490 0x564105a12c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.844+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a13150 0x564105a2f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.844+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a2fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.844+0000 7fbe2b211740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56410594cb10 con 0x564105a13150 2024-05-30T16:58:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.844+0000 7fbe2b211740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5641058f32b0 con 0x564105a0f490 2024-05-30T16:58:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.844+0000 7fbe2b211740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564105948ad0 con 0x564105a2f5d0 2024-05-30T16:58:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe25000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a13150 0x564105a2f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe25000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a13150 0x564105a2f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50276/0 (socket says 172.21.15.67:50276) 2024-05-30T16:58:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe25000640 1 -- 172.21.15.67:0/3068178820 learned_addr learned my addr 172.21.15.67:0/3068178820 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe25a00640 1 --2- 172.21.15.67:0/3068178820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a0f490 0x564105a12c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe26400640 1 --2- 172.21.15.67:0/3068178820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a2fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe25000640 1 -- 172.21.15.67:0/3068178820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 msgr2=0x564105a2fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe25000640 1 --2- 172.21.15.67:0/3068178820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a2fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe25000640 1 -- 172.21.15.67:0/3068178820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a0f490 msgr2=0x564105a12c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe25000640 1 --2- 172.21.15.67:0/3068178820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a0f490 0x564105a12c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.845+0000 7fbe25000640 1 -- 172.21.15.67:0/3068178820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56410565f2d0 con 0x564105a13150 2024-05-30T16:58:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.846+0000 7fbe25000640 1 --2- 172.21.15.67:0/3068178820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a13150 0x564105a2f090 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c007970 tx=0x7fbe0c007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f208e6a087dc88 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.846+0000 7fbe12000640 1 -- 172.21.15.67:0/3068178820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c019070 con 0x564105a13150 2024-05-30T16:58:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.846+0000 7fbe12000640 1 -- 172.21.15.67:0/3068178820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0c0074e0 con 0x564105a13150 2024-05-30T16:58:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.846+0000 7fbe12000640 1 -- 172.21.15.67:0/3068178820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c00ea80 con 0x564105a13150 2024-05-30T16:58:35.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.847+0000 7fbe2b211740 1 -- 172.21.15.67:0/3068178820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a13150 msgr2=0x564105a2f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.847+0000 7fbe2b211740 1 --2- 172.21.15.67:0/3068178820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a13150 0x564105a2f090 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c007970 tx=0x7fbe0c007e30 comp rx=0 tx=0).stop 2024-05-30T16:58:35.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.847+0000 7fbe2b211740 1 -- 172.21.15.67:0/3068178820 shutdown_connections 2024-05-30T16:58:35.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.847+0000 7fbe2b211740 1 --2- 172.21.15.67:0/3068178820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.847+0000 7fbe2b211740 1 --2- 172.21.15.67:0/3068178820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a13150 0x564105a2f090 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.847+0000 7fbe2b211740 1 --2- 172.21.15.67:0/3068178820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a0f490 0x564105a12c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.847+0000 7fbe2b211740 1 -- 172.21.15.67:0/3068178820 >> 172.21.15.67:0/3068178820 conn(0x5641059e5470 msgr2=0x5641059e6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:35.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.847+0000 7fbe2b211740 1 -- 172.21.15.67:0/3068178820 shutdown_connections 2024-05-30T16:58:35.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.847+0000 7fbe2b211740 1 -- 172.21.15.67:0/3068178820 wait complete. 2024-05-30T16:58:35.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.848+0000 7fbe2b211740 1 -- start start 2024-05-30T16:58:35.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.848+0000 7fbe2b211740 1 Processor -- start 2024-05-30T16:58:35.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 0x564105a40a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a16850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a3df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe2b211740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5641059e32a0 con 0x564105a3dae0 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe2b211740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564105906570 con 0x564105a13150 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe2b211740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5641059e3580 con 0x564105a2f5d0 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe2b211740 10 client.0 _ll_register_callbacks cb 0x5641058d4fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe26400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a3df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe26400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a3df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50278/0 (socket says 172.21.15.67:50278) 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.849+0000 7fbe26400640 1 -- 172.21.15.67:0/1607158947 learned_addr learned my addr 172.21.15.67:0/1607158947 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe25000640 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a16850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe26400640 1 -- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 msgr2=0x564105a16850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe25a00640 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 0x564105a40a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe26400640 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a16850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe26400640 1 -- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 msgr2=0x564105a40a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe26400640 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 0x564105a40a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe26400640 1 -- 172.21.15.67:0/1607158947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641057296b0 con 0x564105a3dae0 2024-05-30T16:58:35.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe25000640 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a16850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe26400640 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a3df60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbe18059060 tx=0x7fbe1805d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdc5ddf1bbde6f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe06a00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe06a00640 1 -- 172.21.15.67:0/1607158947 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe180589b0 con 0x564105a3dae0 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe06a00640 1 -- 172.21.15.67:0/1607158947 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fbe180550b0 con 0x564105a3dae0 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe2b211740 1 -- 172.21.15.67:0/1607158947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe0c0074e0 con 0x564105a3dae0 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.850+0000 7fbe06a00640 1 -- 172.21.15.67:0/1607158947 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fbe18078050 con 0x564105a3dae0 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe2b211740 10 client.7512 fetch_fsmap learned FSMap version 55 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe2b211740 1 -- 172.21.15.67:0/1607158947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x564105a29750 con 0x564105a3dae0 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe06a00640 1 -- 172.21.15.67:0/1607158947 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0c0074e0 con 0x564105a3dae0 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe06a00640 1 -- 172.21.15.67:0/1607158947 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 55) ==== 3494+0+0 (secure 0 0 0) 0x7fbe1805d0c0 con 0x564105a3dae0 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe2b211740 10 client.7512 fetch_fsmap finished waiting for FSMap version 55 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe2b211740 10 client.7512 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe2b211740 20 client.7512 populate_metadata read hostname 'smithi067' 2024-05-30T16:58:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe2b211740 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] conn(0x564105a0e270 0x564105a2a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe2b211740 4 client.7512 mds_command: new command op to 7398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-30T16:58:35.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.851+0000 7fbe2b211740 1 -- 172.21.15.67:0/1607158947 --> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56410595a530 con 0x564105a0e270 2024-05-30T16:58:35.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.852+0000 7fbe25a00640 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] conn(0x564105a0e270 0x564105a2a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.853+0000 7fbe25a00640 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] conn(0x564105a0e270 0x564105a2a7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f17cdef2b458477 server_cookie=35e3b8431b1e46f1 in_seq=0 out_seq=0 2024-05-30T16:58:35.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.853+0000 7fbe06a00640 10 client.7512 ms_handle_connect on v2:172.21.15.169:6833/2733959172 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe06a00640 1 -- 172.21.15.67:0/1607158947 <== mds.0 v2:172.21.15.169:6833/2733959172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56410595a530 con 0x564105a0e270 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe06a00640 10 client.7512 handle_command_reply: tid=0 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 client.7512 shutdown 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 -- 172.21.15.67:0/1607158947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 msgr2=0x564105a3df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a3df60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbe18059060 tx=0x7fbe1805d720 comp rx=0 tx=0).stop 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 -- 172.21.15.67:0/1607158947 shutdown_connections 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] conn(0x564105a0e270 0x564105a2a7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a3df60 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a16850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 --2- 172.21.15.67:0/1607158947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 0x564105a40a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 -- 172.21.15.67:0/1607158947 >> 172.21.15.67:0/1607158947 conn(0x564105a0f490 msgr2=0x5641059e21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:35.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe06a00640 0 client.7512 ms_handle_reset on v2:172.21.15.169:6833/2733959172 2024-05-30T16:58:35.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 -- 172.21.15.67:0/1607158947 shutdown_connections 2024-05-30T16:58:35.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.855+0000 7fbe2b211740 1 -- 172.21.15.67:0/1607158947 wait complete. 2024-05-30T16:58:35.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.856+0000 7fbe2b211740 20 client.7512 trim_cache size 0 max 16384 2024-05-30T16:58:35.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.876+0000 7fbe2b211740 1 Processor -- start 2024-05-30T16:58:35.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.876+0000 7fbe2b211740 1 -- start start 2024-05-30T16:58:35.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.877+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 0x564105a27a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.877+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a0e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.877+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a0ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.877+0000 7fbe2b211740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20204900 con 0x564105a3dae0 2024-05-30T16:58:35.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.877+0000 7fbe2b211740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20164cf0 con 0x564105a13150 2024-05-30T16:58:35.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.877+0000 7fbe2b211740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe202042b0 con 0x564105a2f5d0 2024-05-30T16:58:35.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.877+0000 7fbe26400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a0ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.877+0000 7fbe26400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a0ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50284/0 (socket says 172.21.15.67:50284) 2024-05-30T16:58:35.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.877+0000 7fbe26400640 1 -- 172.21.15.67:0/3077201053 learned_addr learned my addr 172.21.15.67:0/3077201053 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:35.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe25000640 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a0e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe25a00640 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 0x564105a27a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe26400640 1 -- 172.21.15.67:0/3077201053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 msgr2=0x564105a0e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe26400640 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a0e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe26400640 1 -- 172.21.15.67:0/3077201053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 msgr2=0x564105a27a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe26400640 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 0x564105a27a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe26400640 1 -- 172.21.15.67:0/3077201053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564105a0b710 con 0x564105a3dae0 2024-05-30T16:58:35.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe25a00640 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 0x564105a27a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:35.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe26400640 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a0ee50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fbe18008010 tx=0x7fbe1805c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad819f8d7c204691 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:35.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe11600640 1 -- 172.21.15.67:0/3077201053 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18057720 con 0x564105a3dae0 2024-05-30T16:58:35.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe11600640 1 -- 172.21.15.67:0/3077201053 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe18088070 con 0x564105a3dae0 2024-05-30T16:58:35.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.878+0000 7fbe11600640 1 -- 172.21.15.67:0/3077201053 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe180589b0 con 0x564105a3dae0 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.879+0000 7fbe2b211740 1 -- 172.21.15.67:0/3077201053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 msgr2=0x564105a0ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.879+0000 7fbe2b211740 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a0ee50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fbe18008010 tx=0x7fbe1805c050 comp rx=0 tx=0).stop 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.879+0000 7fbe2b211740 1 -- 172.21.15.67:0/3077201053 shutdown_connections 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.879+0000 7fbe2b211740 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a3dae0 0x564105a0ee50 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.879+0000 7fbe2b211740 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a2f5d0 0x564105a0e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.879+0000 7fbe2b211740 1 --2- 172.21.15.67:0/3077201053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a13150 0x564105a27a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.879+0000 7fbe2b211740 1 -- 172.21.15.67:0/3077201053 >> 172.21.15.67:0/3077201053 conn(0x564105a0f490 msgr2=0x564105a13550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.879+0000 7fbe2b211740 1 -- 172.21.15.67:0/3077201053 shutdown_connections 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.879+0000 7fbe2b211740 1 -- 172.21.15.67:0/3077201053 wait complete. 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.880+0000 7fbe2b211740 1 -- start start 2024-05-30T16:58:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.880+0000 7fbe2b211740 1 Processor -- start 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a13150 0x5641059e4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a2f5d0 0x564105a3fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe2b211740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a3dae0 0x564105aa74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe2b211740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x564105a0e0f0 con 0x564105a2f5d0 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe2b211740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564105a3e550 con 0x564105a3dae0 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe2b211740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564105a0e740 con 0x564105a13150 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe2b211740 10 client.0 _ll_register_callbacks cb 0x7fbe18059360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe25000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a2f5d0 0x564105a3fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe25000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a2f5d0 0x564105a3fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50294/0 (socket says 172.21.15.67:50294) 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.881+0000 7fbe25000640 1 -- 172.21.15.67:0/2193172155 learned_addr learned my addr 172.21.15.67:0/2193172155 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe25a00640 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a13150 0x5641059e4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe25000640 1 -- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a13150 msgr2=0x5641059e4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe25000640 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a13150 0x5641059e4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe25000640 1 -- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a3dae0 msgr2=0x564105aa74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe26400640 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a3dae0 0x564105aa74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe25000640 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a3dae0 0x564105aa74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe25000640 1 -- 172.21.15.67:0/2193172155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564105a27fb0 con 0x564105a2f5d0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe25a00640 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a13150 0x5641059e4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe25000640 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a2f5d0 0x564105a3fe30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c007940 tx=0x7fbe0c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa73ca0a25df9993 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.882+0000 7fbe06a00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe06a00640 1 -- 172.21.15.67:0/2193172155 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c024070 con 0x564105a2f5d0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe06a00640 1 -- 172.21.15.67:0/2193172155 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fbe0c00ec70 con 0x564105a2f5d0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe2b211740 1 -- 172.21.15.67:0/2193172155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe18088070 con 0x564105a2f5d0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe06a00640 1 -- 172.21.15.67:0/2193172155 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fbe0c019040 con 0x564105a2f5d0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe2b211740 10 client.7518 fetch_fsmap learned FSMap version 55 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe2b211740 1 -- 172.21.15.67:0/2193172155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x564105a2f400 con 0x564105a2f5d0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe06a00640 1 -- 172.21.15.67:0/2193172155 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe18088070 con 0x564105a2f5d0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe06a00640 1 -- 172.21.15.67:0/2193172155 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 55) ==== 3494+0+0 (secure 0 0 0) 0x7fbe0c01f040 con 0x564105a2f5d0 2024-05-30T16:58:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe2b211740 10 client.7518 fetch_fsmap finished waiting for FSMap version 55 2024-05-30T16:58:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe2b211740 10 client.7518 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-30T16:58:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe2b211740 20 client.7518 populate_metadata read hostname 'smithi067' 2024-05-30T16:58:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe2b211740 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] conn(0x564105a0eb40 0x564105a0a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe2b211740 4 client.7518 mds_command: new command op to 7398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-30T16:58:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.883+0000 7fbe2b211740 1 -- 172.21.15.67:0/2193172155 --> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] -- command(tid 0: {"prefix": "status"}) -- 0x5641059543a0 con 0x564105a0eb40 2024-05-30T16:58:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.884+0000 7fbe25a00640 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] conn(0x564105a0eb40 0x564105a0a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.885+0000 7fbe25a00640 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] conn(0x564105a0eb40 0x564105a0a4f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a77fc4d2ccc0ab6 server_cookie=8f2900295cf1a6e3 in_seq=0 out_seq=0 2024-05-30T16:58:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.885+0000 7fbe06a00640 10 client.7518 ms_handle_connect on v2:172.21.15.169:6833/2733959172 2024-05-30T16:58:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe06a00640 1 -- 172.21.15.67:0/2193172155 <== mds.0 v2:172.21.15.169:6833/2733959172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5641059543a0 con 0x564105a0eb40 2024-05-30T16:58:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe06a00640 10 client.7518 handle_command_reply: tid=0 2024-05-30T16:58:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 client.7518 shutdown 2024-05-30T16:58:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 -- 172.21.15.67:0/2193172155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a2f5d0 msgr2=0x564105a3fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a2f5d0 0x564105a3fe30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c007940 tx=0x7fbe0c001970 comp rx=0 tx=0).stop 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 -- 172.21.15.67:0/2193172155 shutdown_connections 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:6833/2733959172,v1:172.21.15.169:6834/2733959172] conn(0x564105a0eb40 0x564105a0a4f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564105a3dae0 0x564105aa74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564105a2f5d0 0x564105a3fe30 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 --2- 172.21.15.67:0/2193172155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564105a13150 0x5641059e4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe06a00640 0 client.7518 ms_handle_reset on v2:172.21.15.169:6833/2733959172 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 -- 172.21.15.67:0/2193172155 >> 172.21.15.67:0/2193172155 conn(0x564105a0f490 msgr2=0x5641059e3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 -- 172.21.15.67:0/2193172155 shutdown_connections 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 1 -- 172.21.15.67:0/2193172155 wait complete. 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.886+0000 7fbe2b211740 20 client.7518 trim_cache size 0 max 16384 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:58:35.889 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7398, 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 6.7656453499999998, 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 55, 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 11.165397434999999 2024-05-30T16:58:35.890 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:58:35.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 -- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe0006da60 msgr2=0x7fbe0006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe0006da60 0x7fbe0006ff20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbe14032000 tx=0x7fbe1402e000 comp rx=0 tx=0).stop 2024-05-30T16:58:35.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 -- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe200ce520 msgr2=0x7fbe20154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:35.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe200ce520 0x7fbe20154250 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fbe20071430 tx=0x7fbe18007860 comp rx=0 tx=0).stop 2024-05-30T16:58:35.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 -- 172.21.15.67:0/1960814188 shutdown_connections 2024-05-30T16:58:35.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe0006da60 0x7fbe0006ff20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe200ce520 0x7fbe20154250 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe2015eea0 0x7fbe200b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 --2- 172.21.15.67:0/1960814188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe08003a00 0x7fbe200ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:35.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 -- 172.21.15.67:0/1960814188 >> 172.21.15.67:0/1960814188 conn(0x7fbe200b1fc0 msgr2=0x7fbe200b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:35.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 -- 172.21.15.67:0/1960814188 shutdown_connections 2024-05-30T16:58:35.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:35.887+0000 7fbe26e00640 1 -- 172.21.15.67:0/1960814188 wait complete. 2024-05-30T16:58:35.906 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T16:58:36.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.110+0000 7f2c03200640 1 Processor -- start 2024-05-30T16:58:36.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.110+0000 7f2c03200640 1 -- start start 2024-05-30T16:58:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.112+0000 7f2c03200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c04158b80 0x7f2c0406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.112+0000 7f2c03200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c040722d0 0x7f2c040b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.112+0000 7f2c03200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c0406e6e0 con 0x7f2c0406e160 2024-05-30T16:58:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.112+0000 7f2c03200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2c0406e9c0 con 0x7f2c040722d0 2024-05-30T16:58:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.112+0000 7f2c03200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0406eca0 con 0x7f2c04158b80 2024-05-30T16:58:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.112+0000 7f2c02800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c040722d0 0x7f2c040b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.112+0000 7f2c01400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2c0406e160 0x7f2c04071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59538/0 (socket says 172.21.15.67:59538) 2024-05-30T16:58:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.112+0000 7f2c01400640 1 -- 172.21.15.67:0/2115670959 learned_addr learned my addr 172.21.15.67:0/2115670959 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.113+0000 7f2c01e00640 1 --2- 172.21.15.67:0/2115670959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c04158b80 0x7f2c0406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.113+0000 7f2c02800640 1 -- 172.21.15.67:0/2115670959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c04158b80 msgr2=0x7f2c0406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.113+0000 7f2c02800640 1 --2- 172.21.15.67:0/2115670959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c04158b80 0x7f2c0406dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.113+0000 7f2c02800640 1 -- 172.21.15.67:0/2115670959 >> v1:172.21.15.67:6789/0 conn(0x7f2c0406e160 legacy=0x7f2c04071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.113+0000 7f2c02800640 1 -- 172.21.15.67:0/2115670959 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0406ef80 con 0x7f2c040722d0 2024-05-30T16:58:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.113+0000 7f2c00a00640 1 -- 172.21.15.67:0/2115670959 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 364455226 0 0) 0x7f2c0406e6e0 con 0x7f2c0406e160 2024-05-30T16:58:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.113+0000 7f2c01e00640 1 --2- 172.21.15.67:0/2115670959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c04158b80 0x7f2c0406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:36.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.114+0000 7f2c02800640 1 --2- 172.21.15.67:0/2115670959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c040722d0 0x7f2c040b32c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fbcf9f1e0d7bd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.114+0000 7f2c00a00640 1 -- 172.21.15.67:0/2115670959 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc007740 con 0x7f2c040722d0 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.114+0000 7f2c00a00640 1 -- 172.21.15.67:0/2115670959 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0078e0 con 0x7f2c040722d0 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.114+0000 7f2c00a00640 1 -- 172.21.15.67:0/2115670959 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc007cb0 con 0x7f2c040722d0 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.115+0000 7f2c03200640 1 -- 172.21.15.67:0/2115670959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c040722d0 msgr2=0x7f2c040b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.115+0000 7f2c03200640 1 --2- 172.21.15.67:0/2115670959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c040722d0 0x7f2c040b32c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc02a660 comp rx=0 tx=0).stop 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.115+0000 7f2c03200640 1 -- 172.21.15.67:0/2115670959 shutdown_connections 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.115+0000 7f2c03200640 1 --2- 172.21.15.67:0/2115670959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c040722d0 0x7f2c040b32c0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.115+0000 7f2c03200640 1 --2- 172.21.15.67:0/2115670959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c04158b80 0x7f2c0406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.115+0000 7f2c03200640 1 -- 172.21.15.67:0/2115670959 >> 172.21.15.67:0/2115670959 conn(0x7f2c040b1fc0 msgr2=0x7f2c040b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.115+0000 7f2c03200640 1 -- 172.21.15.67:0/2115670959 shutdown_connections 2024-05-30T16:58:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.115+0000 7f2c03200640 1 -- 172.21.15.67:0/2115670959 wait complete. 2024-05-30T16:58:36.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.117+0000 7f2c03200640 1 Processor -- start 2024-05-30T16:58:36.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.117+0000 7f2c03200640 1 -- start start 2024-05-30T16:58:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c03200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c0406e160 0x7f2c040ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c03200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c040722d0 0x7f2c04151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c03200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c04158b80 0x7f2c0414e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c03200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0406f780 con 0x7f2c04158b80 2024-05-30T16:58:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c03200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040b2d30 con 0x7f2c040722d0 2024-05-30T16:58:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c03200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0409ae60 con 0x7f2c0406e160 2024-05-30T16:58:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c02800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c04158b80 0x7f2c0414e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c02800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c04158b80 0x7f2c0414e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50308/0 (socket says 172.21.15.67:50308) 2024-05-30T16:58:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c02800640 1 -- 172.21.15.67:0/73122387 learned_addr learned my addr 172.21.15.67:0/73122387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c01400640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c040722d0 0x7f2c04151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c02800640 1 -- 172.21.15.67:0/73122387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c0406e160 msgr2=0x7f2c040ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c02800640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c0406e160 0x7f2c040ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c02800640 1 -- 172.21.15.67:0/73122387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c040722d0 msgr2=0x7f2c04151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c02800640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c040722d0 0x7f2c04151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.118+0000 7f2c02800640 1 -- 172.21.15.67:0/73122387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0409b0d0 con 0x7f2c04158b80 2024-05-30T16:58:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.119+0000 7f2c02800640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c04158b80 0x7f2c0414e700 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.119+0000 7f2bfa000640 1 -- 172.21.15.67:0/73122387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc040070 con 0x7f2c04158b80 2024-05-30T16:58:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.119+0000 7f2bfa000640 1 -- 172.21.15.67:0/73122387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc031c40 con 0x7f2c04158b80 2024-05-30T16:58:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.119+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0409b3b0 con 0x7f2c04158b80 2024-05-30T16:58:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.119+0000 7f2bfa000640 1 -- 172.21.15.67:0/73122387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc003990 con 0x7f2c04158b80 2024-05-30T16:58:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.119+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0414ec40 con 0x7f2c04158b80 2024-05-30T16:58:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.121+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c04157750 con 0x7f2c04158b80 2024-05-30T16:58:36.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.122+0000 7f2bfa000640 1 -- 172.21.15.67:0/73122387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f2bfc039040 con 0x7f2c04158b80 2024-05-30T16:58:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.122+0000 7f2bfa000640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2be406da60 0x7f2be406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.123+0000 7f2bfa000640 1 -- 172.21.15.67:0/73122387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f2bfc0aba60 con 0x7f2c04158b80 2024-05-30T16:58:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.123+0000 7f2c01e00640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2be406da60 0x7f2be406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.123+0000 7f2c01e00640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2be406da60 0x7f2be406ff20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4006fd0 tx=0x7f2bf4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:36.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.127+0000 7f2bfa000640 1 -- 172.21.15.67:0/73122387 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c04157750 con 0x7f2c04158b80 2024-05-30T16:58:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.370+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2c040689e0 con 0x7f2c04158b80 2024-05-30T16:58:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.371+0000 7f2bfa000640 1 -- 172.21.15.67:0/73122387 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7f2bfc07b590 con 0x7f2c04158b80 2024-05-30T16:58:36.372 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2be406da60 msgr2=0x7f2be406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2be406da60 0x7f2be406ff20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4006fd0 tx=0x7f2bf4008040 comp rx=0 tx=0).stop 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c04158b80 msgr2=0x7f2c0414e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c04158b80 0x7f2c0414e700 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc004a40 comp rx=0 tx=0).stop 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 shutdown_connections 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2be406da60 0x7f2be406ff20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c04158b80 0x7f2c0414e700 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c040722d0 0x7f2c04151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 --2- 172.21.15.67:0/73122387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c0406e160 0x7f2c040ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 >> 172.21.15.67:0/73122387 conn(0x7f2c040b1fc0 msgr2=0x7f2c04153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 shutdown_connections 2024-05-30T16:58:36.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36.374+0000 7f2c03200640 1 -- 172.21.15.67:0/73122387 wait complete. 2024-05-30T16:58:36.395 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T16:58:36.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T16:58:36.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T16:58:36.395 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-30T16:58:36.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T16:58:36.395 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T16:58:36.395 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T16:58:36.395 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T16:58:36.396 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:58:36.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:36.396 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T16:58:36.452 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41656sec preferred_lft 41656sec 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::74f5:cfff:fec0:693d/64 scope link 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:58:36.453 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:58:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:58:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::a8fe:aeff:fef7:f6a0/64 scope link 2024-05-30T16:58:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:58:36.454 INFO:teuthology.orchestra.run.smithi067.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T16:58:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:58:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::185e:a4ff:fec6:7d8c/64 scope link 2024-05-30T16:58:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T16:58:36.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:36.455 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T16:58:36.540 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T16:58:36.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:36.544 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T16:58:36.598 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T16:58:36.599 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T16:58:36.599 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T16:58:36.599 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T16:58:36.599 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T16:58:36.600 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T16:58:36.600 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T16:58:36.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T16:58:36.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T16:58:36.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T16:58:36.600 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T16:58:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T16:58:36.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:58:36.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:36.824 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T16:58:36.876 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T16:58:36.876 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-30T16:58:36.877 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:58:36.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:36.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-05-30T16:58:36.964 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-30T16:58:36.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:58:36.965 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-30T16:58:37.023 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-30T16:58:37.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T16:58:37.023 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:58:37.077 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:58:37.175 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:58:37.231 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:58:37.231 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:58:37.231 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:58:37.231 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:58:37.231 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:58:37.231 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:58:37.231 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:58:37.231 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:58:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:58:37.233 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:58:37.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:37.234 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:58:37.286 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T16:58:37.286 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-30T16:58:37.291 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T16:58:37.364 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T16:58:37.408 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T16:58:37.408 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T16:58:37.409 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T16:58:37.410 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T16:58:37.411 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T16:58:37.411 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T16:58:37.411 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T16:58:37.411 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T16:58:37.411 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:58:37.411 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T16:58:37.411 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:58:37.411 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T16:58:37.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:37.412 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T16:58:37.471 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-05-30T16:58:37.471+0000 7fa95320c480 -1 init, newargv = 0x5614bc365c80 newargc=15 2024-05-30T16:58:37.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[66929]: starting ceph client 2024-05-30T16:58:37.483 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[66929]: starting fuse 2024-05-30T16:58:37.499 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T16:58:37.500 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-30T16:58:37.500 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T16:58:37.501 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T16:58:37.502 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-05-30T16:58:37.502 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T16:58:37.698 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.66929.asok 2024-05-30T16:58:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T16:58:37.715 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.66929.asok 2024-05-30T16:58:37.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:37.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.66929.asok status 2024-05-30T16:58:37.950 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:58:37.950 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "66929", 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7533, 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T16:58:37.951 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7533 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2950363266 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2950363266 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.7533 192.168.0.1:0/2950363266", 2024-05-30T16:58:37.952 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/2950363266", 2024-05-30T16:58:37.953 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T16:58:37.953 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 55, 2024-05-30T16:58:37.953 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 91, 2024-05-30T16:58:37.953 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T16:58:37.953 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T16:58:37.953 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T16:58:37.953 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:58:37.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:37.966 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T16:58:37.966 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-30T16:58:38.019 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T16:58:38.019 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-30T16:58:38.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:38.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-30T16:58:38.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-30T16:58:38.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.326+0000 7f2663000640 1 Processor -- start 2024-05-30T16:58:38.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.326+0000 7f2663000640 1 -- start start 2024-05-30T16:58:38.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.328+0000 7f2663000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f265c0cf2f0 0x7f265c0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.328+0000 7f2663000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c140500 0x7f265c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.328+0000 7f2663000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265c06dd50 con 0x7f265c06edf0 2024-05-30T16:58:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.328+0000 7f2663000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f265c06e030 con 0x7f265c0cf2f0 2024-05-30T16:58:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.328+0000 7f2663000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f265c06e310 con 0x7f265c140500 2024-05-30T16:58:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.329+0000 7f2661000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c140500 0x7f265c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:38.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.329+0000 7f2653e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f265c06edf0 0x7f265c13fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:38654/0 (socket says 172.21.15.67:38654) 2024-05-30T16:58:38.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.329+0000 7f2653e00640 1 -- 172.21.15.67:0/3816007451 learned_addr learned my addr 172.21.15.67:0/3816007451 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:38.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.329+0000 7f265be00640 1 --2- 172.21.15.67:0/3816007451 >> v2:172.21.15.169:3300/0 conn(0x7f265c0cf2f0 0x7f265c0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:38.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.330+0000 7f2661000640 1 -- 172.21.15.67:0/3816007451 >> v2:172.21.15.169:3300/0 conn(0x7f265c0cf2f0 msgr2=0x7f265c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:38.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.330+0000 7f2661000640 1 --2- 172.21.15.67:0/3816007451 >> v2:172.21.15.169:3300/0 conn(0x7f265c0cf2f0 0x7f265c0cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:38.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.330+0000 7f2661000640 1 -- 172.21.15.67:0/3816007451 >> v1:172.21.15.169:6789/0 conn(0x7f265c06edf0 legacy=0x7f265c13fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:38.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.330+0000 7f2661000640 1 -- 172.21.15.67:0/3816007451 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c06e5f0 con 0x7f265c140500 2024-05-30T16:58:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.331+0000 7f2661000640 1 --2- 172.21.15.67:0/3816007451 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c140500 0x7f265c0ad130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2654009d90 tx=0x7f265402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8d45e3419d0ff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.332+0000 7f265b400640 1 -- 172.21.15.67:0/3816007451 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654031600 con 0x7f265c140500 2024-05-30T16:58:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.332+0000 7f265b400640 1 -- 172.21.15.67:0/3816007451 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26540317a0 con 0x7f265c140500 2024-05-30T16:58:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.332+0000 7f265b400640 1 -- 172.21.15.67:0/3816007451 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654031b70 con 0x7f265c140500 2024-05-30T16:58:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.333+0000 7f2663000640 1 -- 172.21.15.67:0/3816007451 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c140500 msgr2=0x7f265c0ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.333+0000 7f2663000640 1 --2- 172.21.15.67:0/3816007451 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c140500 0x7f265c0ad130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2654009d90 tx=0x7f265402a5d0 comp rx=0 tx=0).stop 2024-05-30T16:58:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.333+0000 7f2663000640 1 -- 172.21.15.67:0/3816007451 shutdown_connections 2024-05-30T16:58:38.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.333+0000 7f2663000640 1 --2- 172.21.15.67:0/3816007451 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c140500 0x7f265c0ad130 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:38.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.333+0000 7f2663000640 1 --2- 172.21.15.67:0/3816007451 >> v2:172.21.15.169:3300/0 conn(0x7f265c0cf2f0 0x7f265c0cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:38.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.333+0000 7f2663000640 1 -- 172.21.15.67:0/3816007451 >> 172.21.15.67:0/3816007451 conn(0x7f265c0b1e50 msgr2=0x7f265c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:38.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.333+0000 7f2663000640 1 -- 172.21.15.67:0/3816007451 shutdown_connections 2024-05-30T16:58:38.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.333+0000 7f2663000640 1 -- 172.21.15.67:0/3816007451 wait complete. 2024-05-30T16:58:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.335+0000 7f2663000640 1 Processor -- start 2024-05-30T16:58:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.335+0000 7f2663000640 1 -- start start 2024-05-30T16:58:38.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.336+0000 7f2663000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c06edf0 0x7f265c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:38.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.336+0000 7f2663000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cf2f0 0x7f265c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:38.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.336+0000 7f2663000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c073890 0x7f265c1f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:38.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.336+0000 7f2663000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1461a0 con 0x7f265c073890 2024-05-30T16:58:38.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.336+0000 7f2663000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f265c146020 con 0x7f265c0cf2f0 2024-05-30T16:58:38.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.336+0000 7f2663000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f265c0cfe50 con 0x7f265c06edf0 2024-05-30T16:58:38.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.336+0000 7f2661000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c073890 0x7f265c1f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:38.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.336+0000 7f2661000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c073890 0x7f265c1f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50324/0 (socket says 172.21.15.67:50324) 2024-05-30T16:58:38.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.336+0000 7f2661000640 1 -- 172.21.15.67:0/2672943915 learned_addr learned my addr 172.21.15.67:0/2672943915 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:38.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.337+0000 7f2653e00640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cf2f0 0x7f265c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:38.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.337+0000 7f265be00640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c06edf0 0x7f265c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.337+0000 7f2661000640 1 -- 172.21.15.67:0/2672943915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c06edf0 msgr2=0x7f265c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.337+0000 7f2661000640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c06edf0 0x7f265c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.337+0000 7f2661000640 1 -- 172.21.15.67:0/2672943915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cf2f0 msgr2=0x7f265c071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.337+0000 7f2661000640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cf2f0 0x7f265c071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.337+0000 7f2661000640 1 -- 172.21.15.67:0/2672943915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c1f5790 con 0x7f265c073890 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.337+0000 7f2661000640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c073890 0x7f265c1f4830 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2654009eb0 tx=0x7f2654004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.337+0000 7f2658c00640 1 -- 172.21.15.67:0/2672943915 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654040070 con 0x7f265c073890 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.338+0000 7f2658c00640 1 -- 172.21.15.67:0/2672943915 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2654030ae0 con 0x7f265c073890 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.338+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c1f51d0 con 0x7f265c073890 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.338+0000 7f2658c00640 1 -- 172.21.15.67:0/2672943915 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654045460 con 0x7f265c073890 2024-05-30T16:58:38.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.338+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c157ee0 con 0x7f265c073890 2024-05-30T16:58:38.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.339+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c0689e0 con 0x7f265c073890 2024-05-30T16:58:38.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.340+0000 7f2658c00640 1 -- 172.21.15.67:0/2672943915 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f2654039040 con 0x7f265c073890 2024-05-30T16:58:38.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.341+0000 7f2658c00640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263806da60 0x7f263806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:38.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.341+0000 7f2658c00640 1 -- 172.21.15.67:0/2672943915 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f26540ac0c0 con 0x7f265c073890 2024-05-30T16:58:38.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.341+0000 7f265be00640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263806da60 0x7f263806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:38.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.342+0000 7f265be00640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263806da60 0x7f263806ff20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f265c0b0970 tx=0x7f264c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:38.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.346+0000 7f2658c00640 1 -- 172.21.15.67:0/2672943915 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26540443c0 con 0x7f265c073890 2024-05-30T16:58:38.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:38.594+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f265c073f30 con 0x7f265c073890 2024-05-30T16:58:39.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.491+0000 7f2658c00640 1 -- 172.21.15.67:0/2672943915 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7f265407bbf0 con 0x7f265c073890 2024-05-30T16:58:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.495+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263806da60 msgr2=0x7f263806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.495+0000 7f2663000640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263806da60 0x7f263806ff20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f265c0b0970 tx=0x7f264c008040 comp rx=0 tx=0).stop 2024-05-30T16:58:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.495+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c073890 msgr2=0x7f265c1f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.495+0000 7f2663000640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c073890 0x7f265c1f4830 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2654009eb0 tx=0x7f2654004a40 comp rx=0 tx=0).stop 2024-05-30T16:58:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.495+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 shutdown_connections 2024-05-30T16:58:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.495+0000 7f2663000640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f263806da60 0x7f263806ff20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.495+0000 7f2663000640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f265c073890 0x7f265c1f4830 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.496+0000 7f2663000640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cf2f0 0x7f265c071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.496+0000 7f2663000640 1 --2- 172.21.15.67:0/2672943915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c06edf0 0x7f265c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.496+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 >> 172.21.15.67:0/2672943915 conn(0x7f265c0b1e50 msgr2=0x7f265c0cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:39.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.496+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 shutdown_connections 2024-05-30T16:58:39.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.496+0000 7f2663000640 1 -- 172.21.15.67:0/2672943915 wait complete. 2024-05-30T16:58:39.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:58:39.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.736+0000 7f2615800640 1 Processor -- start 2024-05-30T16:58:39.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.737+0000 7f2615800640 1 -- start start 2024-05-30T16:58:39.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.738+0000 7f2615800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26100d16a0 0x7f26100cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:39.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.738+0000 7f2615800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26100cfce0 con 0x7f261015e870 2024-05-30T16:58:39.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.738+0000 7f2615800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f261015ec50 con 0x7f26100cf870 2024-05-30T16:58:39.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.738+0000 7f2615800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f26100d2150 con 0x7f26100d16a0 2024-05-30T16:58:39.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.738+0000 7f260fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f261015e870 0x7f26100d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59548/0 (socket says 172.21.15.67:59548) 2024-05-30T16:58:39.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.738+0000 7f260fe00640 1 -- 172.21.15.67:0/2614822161 learned_addr learned my addr 172.21.15.67:0/2614822161 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:39.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260f400640 1 --2- 172.21.15.67:0/2614822161 >> v2:172.21.15.169:3300/0 conn(0x7f26100d16a0 0x7f26100cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:39.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1660877307 0 0) 0x7f26100cfce0 con 0x7f261015e870 2024-05-30T16:58:39.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25f4003520 con 0x7f261015e870 2024-05-30T16:58:39.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2218634798 0 0) 0x7f25f4003520 con 0x7f261015e870 2024-05-30T16:58:39.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 >> v2:172.21.15.169:3300/0 conn(0x7f26100d16a0 msgr2=0x7f26100cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:39.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 --2- 172.21.15.67:0/2614822161 >> v2:172.21.15.169:3300/0 conn(0x7f26100d16a0 0x7f26100cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 >> v1:172.21.15.169:6789/0 conn(0x7f26100cf870 legacy=0x7f26100d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:39.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26100cff80 con 0x7f261015e870 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f26000019f0 con 0x7f261015e870 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 >> v1:172.21.15.67:6789/0 conn(0x7f261015e870 legacy=0x7f26100d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.739+0000 7f260ea00640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40039b0 0x7f25f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260ea00640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f25f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260ea00640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f40051c0 0x7f25f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26100d2150 con 0x7f25f4004800 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25f400c060 con 0x7f25f40039b0 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25f400c260 con 0x7f25f40051c0 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260f400640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f25f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260fe00640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40039b0 0x7f25f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f2614e00640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f40051c0 0x7f25f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260f400640 1 -- 172.21.15.67:0/2614822161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f40051c0 msgr2=0x7f25f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:39.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260f400640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25f40051c0 0x7f25f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260f400640 1 -- 172.21.15.67:0/2614822161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40039b0 msgr2=0x7f25f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:39.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260f400640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40039b0 0x7f25f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260f400640 1 -- 172.21.15.67:0/2614822161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f4003520 con 0x7f25f4004800 2024-05-30T16:58:39.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f260fe00640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25f40039b0 0x7f25f4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:39.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.740+0000 7f2614e00640 1 -- 172.21.15.67:0/2614822161 reap_dead start 2024-05-30T16:58:39.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.742+0000 7f260f400640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f25f4004c80 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ab0 tx=0x7f25f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4893743fef1dab8b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:39.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.742+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc036070 con 0x7f25f4004800 2024-05-30T16:58:39.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.742+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25fc005990 con 0x7f25f4004800 2024-05-30T16:58:39.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.742+0000 7f260ea00640 1 -- 172.21.15.67:0/2614822161 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc009680 con 0x7f25f4004800 2024-05-30T16:58:39.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.742+0000 7f2615800640 1 -- 172.21.15.67:0/2614822161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 msgr2=0x7f25f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:39.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.742+0000 7f2615800640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f25f4004c80 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ab0 tx=0x7f25f4005830 comp rx=0 tx=0).stop 2024-05-30T16:58:39.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.743+0000 7f2615800640 1 -- 172.21.15.67:0/2614822161 shutdown_connections 2024-05-30T16:58:39.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.743+0000 7f2615800640 1 --2- 172.21.15.67:0/2614822161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f25f4004c80 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.743+0000 7f2615800640 1 -- 172.21.15.67:0/2614822161 >> 172.21.15.67:0/2614822161 conn(0x7f261008bfc0 msgr2=0x7f26100d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:39.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.743+0000 7f2615800640 1 -- 172.21.15.67:0/2614822161 shutdown_connections 2024-05-30T16:58:39.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.743+0000 7f2615800640 1 -- 172.21.15.67:0/2614822161 wait complete. 2024-05-30T16:58:39.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.745+0000 7f2615800640 1 Processor -- start 2024-05-30T16:58:39.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.745+0000 7f2615800640 1 -- start start 2024-05-30T16:58:39.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f2615800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f26101f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:39.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f2615800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26100c8940 0x7f26100b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:39.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f2615800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26100b27f0 0x7f261015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f2615800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f26100d0820 con 0x7f25f4004800 2024-05-30T16:58:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f2615800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26100d0540 con 0x7f26100c8940 2024-05-30T16:58:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f2615800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26100d4fb0 con 0x7f26100b27f0 2024-05-30T16:58:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f26101f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f26101f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50342/0 (socket says 172.21.15.67:50342) 2024-05-30T16:58:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260fe00640 1 -- 172.21.15.67:0/4056735822 learned_addr learned my addr 172.21.15.67:0/4056735822 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f2614e00640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26100b27f0 0x7f261015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260fe00640 1 -- 172.21.15.67:0/4056735822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26100b27f0 msgr2=0x7f261015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260fe00640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26100b27f0 0x7f261015ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260f400640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26100c8940 0x7f26100b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260fe00640 1 -- 172.21.15.67:0/4056735822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26100c8940 msgr2=0x7f26100b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260fe00640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26100c8940 0x7f26100b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260fe00640 1 -- 172.21.15.67:0/4056735822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26100c8eb0 con 0x7f25f4004800 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.746+0000 7f260f400640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26100c8940 0x7f26100b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.747+0000 7f260fe00640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f26101f3ea0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f2600001a20 tx=0x7f260000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.747+0000 7f2607e00640 1 -- 172.21.15.67:0/4056735822 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600019070 con 0x7f25f4004800 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.747+0000 7f2607e00640 1 -- 172.21.15.67:0/4056735822 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2600004d60 con 0x7f25f4004800 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.747+0000 7f2607e00640 1 -- 172.21.15.67:0/4056735822 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000be60 con 0x7f25f4004800 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.747+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26100c9190 con 0x7f25f4004800 2024-05-30T16:58:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.747+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26100c9750 con 0x7f25f4004800 2024-05-30T16:58:39.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.748+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26100689e0 con 0x7f25f4004800 2024-05-30T16:58:39.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.749+0000 7f2607e00640 1 -- 172.21.15.67:0/4056735822 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f26000072f0 con 0x7f25f4004800 2024-05-30T16:58:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.750+0000 7f2607e00640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25f006da60 0x7f25f006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.750+0000 7f2607e00640 1 -- 172.21.15.67:0/4056735822 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f260008ab90 con 0x7f25f4004800 2024-05-30T16:58:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.751+0000 7f260f400640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25f006da60 0x7f25f006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:39.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.751+0000 7f260f400640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25f006da60 0x7f25f006ff20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f25fc033000 tx=0x7f25fc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.757+0000 7f2607e00640 1 -- 172.21.15.67:0/4056735822 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f260005a6c0 con 0x7f25f4004800 2024-05-30T16:58:39.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:39.998+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f26100ba260 con 0x7f25f006da60 2024-05-30T16:58:40.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.000+0000 7f2607e00640 1 -- 172.21.15.67:0/4056735822 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f26100ba260 con 0x7f25f006da60 2024-05-30T16:58:40.001 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:58:40.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:58:40.001 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:58:40.001 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:58:40.001 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:58:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25f006da60 msgr2=0x7f25f006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25f006da60 0x7f25f006ff20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f25fc033000 tx=0x7f25fc008030 comp rx=0 tx=0).stop 2024-05-30T16:58:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 msgr2=0x7f26101f3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f26101f3ea0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f2600001a20 tx=0x7f260000b4a0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 shutdown_connections 2024-05-30T16:58:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f25f006da60 0x7f25f006ff20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26100b27f0 0x7f261015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26100c8940 0x7f26100b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 --2- 172.21.15.67:0/4056735822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25f4004800 0x7f26101f3ea0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 >> 172.21.15.67:0/4056735822 conn(0x7f261008bfc0 msgr2=0x7f26100b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.003+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 shutdown_connections 2024-05-30T16:58:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.004+0000 7f2615800640 1 -- 172.21.15.67:0/4056735822 wait complete. 2024-05-30T16:58:40.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T16:58:40.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.229+0000 7f0359400640 1 Processor -- start 2024-05-30T16:58:40.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.229+0000 7f0359400640 1 -- start start 2024-05-30T16:58:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.231+0000 7f0359400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0354153510 0x7f03540b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.231+0000 7f0359400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f03540ccb90 0x7f03540ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.231+0000 7f0359400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f035415d900 con 0x7f035415cb60 2024-05-30T16:58:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.231+0000 7f0359400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f03541538f0 con 0x7f03540ccb90 2024-05-30T16:58:40.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.231+0000 7f0359400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f03540cd530 con 0x7f0354153510 2024-05-30T16:58:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.232+0000 7f034be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f03540ccb90 0x7f03540ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.232+0000 7f0358a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f035415cb60 0x7f035415cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59560/0 (socket says 172.21.15.67:59560) 2024-05-30T16:58:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.232+0000 7f0358a00640 1 -- 172.21.15.67:0/3969237406 learned_addr learned my addr 172.21.15.67:0/3969237406 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.232+0000 7f0353e00640 1 --2- 172.21.15.67:0/3969237406 >> v2:172.21.15.169:3300/0 conn(0x7f0354153510 0x7f03540b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.232+0000 7f034be00640 1 -- 172.21.15.67:0/3969237406 >> v2:172.21.15.169:3300/0 conn(0x7f0354153510 msgr2=0x7f03540b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.232+0000 7f034be00640 1 --2- 172.21.15.67:0/3969237406 >> v2:172.21.15.169:3300/0 conn(0x7f0354153510 0x7f03540b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.232+0000 7f034be00640 1 -- 172.21.15.67:0/3969237406 >> v1:172.21.15.67:6789/0 conn(0x7f035415cb60 legacy=0x7f035415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.232+0000 7f034be00640 1 -- 172.21.15.67:0/3969237406 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03540b3f90 con 0x7f03540ccb90 2024-05-30T16:58:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.232+0000 7f0353400640 1 -- 172.21.15.67:0/3969237406 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220570401 0 0) 0x7f035415d900 con 0x7f035415cb60 2024-05-30T16:58:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.233+0000 7f034be00640 1 --2- 172.21.15.67:0/3969237406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 0x7f03540ccff0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f03540b3970 tx=0x7f033c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcc85f551ae16e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0353400640 1 -- 172.21.15.67:0/3969237406 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c037070 con 0x7f03540ccb90 2024-05-30T16:58:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0353400640 1 -- 172.21.15.67:0/3969237406 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033c032440 con 0x7f03540ccb90 2024-05-30T16:58:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0353400640 1 -- 172.21.15.67:0/3969237406 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c02ae60 con 0x7f03540ccb90 2024-05-30T16:58:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0359400640 1 -- 172.21.15.67:0/3969237406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 msgr2=0x7f03540ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0359400640 1 --2- 172.21.15.67:0/3969237406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 0x7f03540ccff0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f03540b3970 tx=0x7f033c02a110 comp rx=0 tx=0).stop 2024-05-30T16:58:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0359400640 1 -- 172.21.15.67:0/3969237406 shutdown_connections 2024-05-30T16:58:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0359400640 1 --2- 172.21.15.67:0/3969237406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 0x7f03540ccff0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0359400640 1 --2- 172.21.15.67:0/3969237406 >> v2:172.21.15.169:3300/0 conn(0x7f0354153510 0x7f03540b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0359400640 1 -- 172.21.15.67:0/3969237406 >> 172.21.15.67:0/3969237406 conn(0x7f035406b900 msgr2=0x7f035406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0359400640 1 -- 172.21.15.67:0/3969237406 shutdown_connections 2024-05-30T16:58:40.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.234+0000 7f0359400640 1 -- 172.21.15.67:0/3969237406 wait complete. 2024-05-30T16:58:40.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.236+0000 7f0359400640 1 Processor -- start 2024-05-30T16:58:40.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.236+0000 7f0359400640 1 -- start start 2024-05-30T16:58:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.237+0000 7f0359400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 0x7f035415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.237+0000 7f0359400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0354153510 0x7f03540ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.237+0000 7f0359400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f035415cb60 0x7f03540ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.237+0000 7f0359400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f035415d6a0 con 0x7f03540ccb90 2024-05-30T16:58:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.237+0000 7f0359400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f03540b48c0 con 0x7f035415cb60 2024-05-30T16:58:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.237+0000 7f0359400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f03540b4550 con 0x7f0354153510 2024-05-30T16:58:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.237+0000 7f0353e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 0x7f035415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.238+0000 7f0353e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 0x7f035415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50352/0 (socket says 172.21.15.67:50352) 2024-05-30T16:58:40.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.238+0000 7f0353e00640 1 -- 172.21.15.67:0/1540925550 learned_addr learned my addr 172.21.15.67:0/1540925550 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.238+0000 7f0353e00640 1 -- 172.21.15.67:0/1540925550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0354153510 msgr2=0x7f03540ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.238+0000 7f0353e00640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0354153510 0x7f03540ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.238+0000 7f0353e00640 1 -- 172.21.15.67:0/1540925550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f035415cb60 msgr2=0x7f03540ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.238+0000 7f0353e00640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f035415cb60 0x7f03540ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.238+0000 7f0353e00640 1 -- 172.21.15.67:0/1540925550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0354150a30 con 0x7f03540ccb90 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.238+0000 7f0353e00640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 0x7f035415b6a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f034400d800 tx=0x7f034400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.239+0000 7f0350c00640 1 -- 172.21.15.67:0/1540925550 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034400b6c0 con 0x7f03540ccb90 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.239+0000 7f0350c00640 1 -- 172.21.15.67:0/1540925550 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0344019070 con 0x7f03540ccb90 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.239+0000 7f0350c00640 1 -- 172.21.15.67:0/1540925550 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034400ba80 con 0x7f03540ccb90 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.239+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03540adcf0 con 0x7f03540ccb90 2024-05-30T16:58:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.239+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03540ae2b0 con 0x7f03540ccb90 2024-05-30T16:58:40.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.240+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03540689e0 con 0x7f03540ccb90 2024-05-30T16:58:40.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.241+0000 7f0350c00640 1 -- 172.21.15.67:0/1540925550 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f034400bc20 con 0x7f03540ccb90 2024-05-30T16:58:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.242+0000 7f0350c00640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f033006da60 0x7f033006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.242+0000 7f0350c00640 1 -- 172.21.15.67:0/1540925550 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f034408af40 con 0x7f03540ccb90 2024-05-30T16:58:40.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.242+0000 7f034be00640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f033006da60 0x7f033006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.243+0000 7f034be00640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f033006da60 0x7f033006ff20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f033c034000 tx=0x7f033c02c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:40.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.247+0000 7f0350c00640 1 -- 172.21.15.67:0/1540925550 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f034405aab0 con 0x7f03540ccb90 2024-05-30T16:58:40.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.455+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f03540b3c40 con 0x7f03540ccb90 2024-05-30T16:58:40.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.456+0000 7f0350c00640 1 -- 172.21.15.67:0/1540925550 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f034405a3b0 con 0x7f03540ccb90 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f033006da60 msgr2=0x7f033006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f033006da60 0x7f033006ff20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f033c034000 tx=0x7f033c02c000 comp rx=0 tx=0).stop 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 msgr2=0x7f035415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 0x7f035415b6a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f034400d800 tx=0x7f034400dcc0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 shutdown_connections 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f033006da60 0x7f033006ff20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f035415cb60 0x7f03540ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0354153510 0x7f03540ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 --2- 172.21.15.67:0/1540925550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03540ccb90 0x7f035415b6a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.459+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 >> 172.21.15.67:0/1540925550 conn(0x7f035406b900 msgr2=0x7f035415cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.460+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 shutdown_connections 2024-05-30T16:58:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.460+0000 7f0359400640 1 -- 172.21.15.67:0/1540925550 wait complete. 2024-05-30T16:58:40.479 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-30T16:58:40.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.694+0000 7f52ff200640 1 Processor -- start 2024-05-30T16:58:40.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.694+0000 7f52ff200640 1 -- start start 2024-05-30T16:58:40.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.696+0000 7f52ff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53001621d0 0x7f53000d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.696+0000 7f52ff200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f53001625b0 0x7f53000d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.696+0000 7f52ff200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53000d0e50 con 0x7f530015ea10 2024-05-30T16:58:40.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.696+0000 7f52ff200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f53000d1a60 con 0x7f53001625b0 2024-05-30T16:58:40.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.696+0000 7f52ff200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f53000d1130 con 0x7f53001621d0 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.696+0000 7f52fe800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f530015ea10 0x7f530015eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59576/0 (socket says 172.21.15.67:59576) 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.696+0000 7f52fe800640 1 -- 172.21.15.67:0/220615394 learned_addr learned my addr 172.21.15.67:0/220615394 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fd400640 1 --2- 172.21.15.67:0/220615394 >> v2:172.21.15.169:3300/0 conn(0x7f53001625b0 0x7f53000d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fde00640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53001621d0 0x7f53000d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 255084294 0 0) 0x7f53000d0e50 con 0x7f530015ea10 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52e4003520 con 0x7f530015ea10 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 359465311 0 0) 0x7f52e4003520 con 0x7f530015ea10 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53001621d0 msgr2=0x7f53000d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53001621d0 0x7f53000d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 >> v2:172.21.15.169:3300/0 conn(0x7f53001625b0 msgr2=0x7f53000d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 --2- 172.21.15.67:0/220615394 >> v2:172.21.15.169:3300/0 conn(0x7f53001625b0 0x7f53000d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53000d1410 con 0x7f530015ea10 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f52f8001670 con 0x7f530015ea10 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 >> v1:172.21.15.67:6789/0 conn(0x7f530015ea10 legacy=0x7f530015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e40039b0 0x7f52e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e4004800 0x7f52e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53001621d0 0x7f52e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f53000d1a60 con 0x7f52e4004800 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53000d1130 con 0x7f52e40039b0 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.697+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f52e400b9c0 con 0x7f53001621d0 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.698+0000 7f52fd400640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e4004800 0x7f52e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.698+0000 7f52fe800640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53001621d0 0x7f52e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.698+0000 7f52fde00640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e40039b0 0x7f52e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.698+0000 7f52fd400640 1 -- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53001621d0 msgr2=0x7f52e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.698+0000 7f52fd400640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53001621d0 0x7f52e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.698+0000 7f52fd400640 1 -- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e40039b0 msgr2=0x7f52e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.698+0000 7f52fd400640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e40039b0 0x7f52e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.698+0000 7f52fd400640 1 -- 172.21.15.67:0/220615394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e4003520 con 0x7f52e4004800 2024-05-30T16:58:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.698+0000 7f52fde00640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e40039b0 0x7f52e4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.700+0000 7f52fd400640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e4004800 0x7f52e4004c80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009ab0 tx=0x7f52e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb5691edf3d9eea server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.700+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8036070 con 0x7f52e4004800 2024-05-30T16:58:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.700+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e8005990 con 0x7f52e4004800 2024-05-30T16:58:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.700+0000 7f52fca00640 1 -- 172.21.15.67:0/220615394 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8009680 con 0x7f52e4004800 2024-05-30T16:58:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.700+0000 7f52ff200640 1 -- 172.21.15.67:0/220615394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e4004800 msgr2=0x7f52e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.700+0000 7f52ff200640 1 --2- 172.21.15.67:0/220615394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e4004800 0x7f52e4004c80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009ab0 tx=0x7f52e4005830 comp rx=0 tx=0).stop 2024-05-30T16:58:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.700+0000 7f52fe800640 1 -- 172.21.15.67:0/220615394 reap_dead start 2024-05-30T16:58:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.701+0000 7f52ff200640 1 -- 172.21.15.67:0/220615394 shutdown_connections 2024-05-30T16:58:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.701+0000 7f52ff200640 1 -- 172.21.15.67:0/220615394 >> 172.21.15.67:0/220615394 conn(0x7f530008bfc0 msgr2=0x7f53000d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.701+0000 7f52ff200640 1 -- 172.21.15.67:0/220615394 shutdown_connections 2024-05-30T16:58:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.701+0000 7f52ff200640 1 -- 172.21.15.67:0/220615394 wait complete. 2024-05-30T16:58:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.702+0000 7f52ff200640 1 Processor -- start 2024-05-30T16:58:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.703+0000 7f52ff200640 1 -- start start 2024-05-30T16:58:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.703+0000 7f52ff200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e4004800 0x7f53000a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.703+0000 7f52ff200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f530015d900 0x7f53001f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52ff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53000ca590 0x7f53001e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52ff200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f53000d5470 con 0x7f530015d900 2024-05-30T16:58:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52ff200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53000d1ef0 con 0x7f52e4004800 2024-05-30T16:58:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52ff200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5300162a10 con 0x7f53000ca590 2024-05-30T16:58:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f530015d900 0x7f53001f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f530015d900 0x7f53001f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50362/0 (socket says 172.21.15.67:50362) 2024-05-30T16:58:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fd400640 1 -- 172.21.15.67:0/3108801297 learned_addr learned my addr 172.21.15.67:0/3108801297 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:40.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fe800640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53000ca590 0x7f53001e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fde00640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e4004800 0x7f53000a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fd400640 1 -- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53000ca590 msgr2=0x7f53001e9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fd400640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53000ca590 0x7f53001e9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fd400640 1 -- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e4004800 msgr2=0x7f53000a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fd400640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e4004800 0x7f53000a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fd400640 1 -- 172.21.15.67:0/3108801297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53000cb410 con 0x7f530015d900 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fe800640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53000ca590 0x7f53001e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.704+0000 7f52fde00640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e4004800 0x7f53000a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.705+0000 7f52fd400640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f530015d900 0x7f53001f51b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f52e80099d0 tx=0x7f52e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.705+0000 7f52f6000640 1 -- 172.21.15.67:0/3108801297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8040030 con 0x7f530015d900 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.705+0000 7f52f6000640 1 -- 172.21.15.67:0/3108801297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e8037040 con 0x7f530015d900 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.705+0000 7f52f6000640 1 -- 172.21.15.67:0/3108801297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8036030 con 0x7f530015d900 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.705+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53000cae50 con 0x7f530015d900 2024-05-30T16:58:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.705+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53001f49b0 con 0x7f530015d900 2024-05-30T16:58:40.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.706+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53000689e0 con 0x7f530015d900 2024-05-30T16:58:40.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.707+0000 7f52f6000640 1 -- 172.21.15.67:0/3108801297 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f52e8043070 con 0x7f530015d900 2024-05-30T16:58:40.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.708+0000 7f52f6000640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52dc06da90 0x7f52dc06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:40.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.709+0000 7f52f6000640 1 -- 172.21.15.67:0/3108801297 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f52e80aa380 con 0x7f530015d900 2024-05-30T16:58:40.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.709+0000 7f52fde00640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52dc06da90 0x7f52dc06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.710+0000 7f52fde00640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52dc06da90 0x7f52dc06ff50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009a20 tx=0x7f52f000d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.714+0000 7f52f6000640 1 -- 172.21.15.67:0/3108801297 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52e8079eb0 con 0x7f530015d900 2024-05-30T16:58:40.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:40.947+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f53000d4d10 con 0x7f52dc06da90 2024-05-30T16:58:40.960 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:58:40.960+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:58:40.961 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:58:40.960+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:58:40.961 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:58:40.960+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:58:40.961 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:58:40.960+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:58:40.961 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:58:40.960+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:58:41.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.038+0000 7f52f6000640 1 -- 172.21.15.67:0/3108801297 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f53000d4d10 con 0x7f52dc06da90 2024-05-30T16:58:41.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52dc06da90 msgr2=0x7f52dc06ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:41.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52dc06da90 0x7f52dc06ff50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009a20 tx=0x7f52f000d040 comp rx=0 tx=0).stop 2024-05-30T16:58:41.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f530015d900 msgr2=0x7f53001f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:41.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f530015d900 0x7f53001f51b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f52e80099d0 tx=0x7f52e8004730 comp rx=0 tx=0).stop 2024-05-30T16:58:41.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 shutdown_connections 2024-05-30T16:58:41.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f52dc06da90 0x7f52dc06ff50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53000ca590 0x7f53001e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f530015d900 0x7f53001f51b0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 --2- 172.21.15.67:0/3108801297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52e4004800 0x7f53000a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 >> 172.21.15.67:0/3108801297 conn(0x7f530008bfc0 msgr2=0x7f530015fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:41.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 shutdown_connections 2024-05-30T16:58:41.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.042+0000 7f52ff200640 1 -- 172.21.15.67:0/3108801297 wait complete. 2024-05-30T16:58:41.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:58:41.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.276+0000 7f20ef200640 1 Processor -- start 2024-05-30T16:58:41.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.276+0000 7f20ef200640 1 -- start start 2024-05-30T16:58:41.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.278+0000 7f20ef200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f20f0153510 0x7f20f00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:41.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.278+0000 7f20ef200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f015d900 con 0x7f20f015d270 2024-05-30T16:58:41.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.278+0000 7f20ef200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f00b3f90 con 0x7f20f00ccb90 2024-05-30T16:58:41.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.278+0000 7f20ef200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f20f01538f0 con 0x7f20f0153510 2024-05-30T16:58:41.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.278+0000 7f20ee800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f20f015d270 0x7f20f006e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42186/0 (socket says 172.21.15.67:42186) 2024-05-30T16:58:41.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.278+0000 7f20ee800640 1 -- 172.21.15.67:0/860848755 learned_addr learned my addr 172.21.15.67:0/860848755 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:58:41.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20ede00640 1 --2- 172.21.15.67:0/860848755 >> v2:172.21.15.169:3300/0 conn(0x7f20f0153510 0x7f20f00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:41.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1301202926 0 0) 0x7f20f015d900 con 0x7f20f015d270 2024-05-30T16:58:41.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20d0003520 con 0x7f20f015d270 2024-05-30T16:58:41.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3849957583 0 0) 0x7f20d0003520 con 0x7f20f015d270 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 >> v2:172.21.15.169:3300/0 conn(0x7f20f0153510 msgr2=0x7f20f00b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 --2- 172.21.15.67:0/860848755 >> v2:172.21.15.169:3300/0 conn(0x7f20f0153510 0x7f20f00b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 >> v1:172.21.15.169:6789/0 conn(0x7f20f00ccb90 legacy=0x7f20f015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f00b4270 con 0x7f20f015d270 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f20e8001670 con 0x7f20f015d270 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 >> v1:172.21.15.67:6789/0 conn(0x7f20f015d270 legacy=0x7f20f006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d0003a00 0x7f20d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d0004800 0x7f20d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d0005210 0x7f20d000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0010540 con 0x7f20d0005210 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20d00106f0 con 0x7f20d0004800 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.279+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20d00108f0 con 0x7f20d0003a00 2024-05-30T16:58:41.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.280+0000 7f20ee800640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d0005210 0x7f20d000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.280+0000 7f20ed400640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d0004800 0x7f20d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.280+0000 7f20ede00640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d0003a00 0x7f20d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.280+0000 7f20ee800640 1 -- 172.21.15.67:0/860848755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d0003a00 msgr2=0x7f20d0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.280+0000 7f20ee800640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d0003a00 0x7f20d0003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.280+0000 7f20ee800640 1 -- 172.21.15.67:0/860848755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d0004800 msgr2=0x7f20d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:58:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.280+0000 7f20ee800640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d0004800 0x7f20d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.280+0000 7f20ee800640 1 -- 172.21.15.67:0/860848755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d0003520 con 0x7f20d0005210 2024-05-30T16:58:41.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.281+0000 7f20ee800640 1 -- 172.21.15.67:0/860848755 reap_dead start 2024-05-30T16:58:41.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.281+0000 7f20ee800640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d0005210 0x7f20d000fdd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f20e8007fd0 tx=0x7f20d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9158331a25a8bc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:41.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8007c20 con 0x7f20d0005210 2024-05-30T16:58:41.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e8005af0 con 0x7f20d0005210 2024-05-30T16:58:41.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20eca00640 1 -- 172.21.15.67:0/860848755 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8007070 con 0x7f20d0005210 2024-05-30T16:58:41.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20ef200640 1 -- 172.21.15.67:0/860848755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d0005210 msgr2=0x7f20d000fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:41.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20ef200640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d0005210 0x7f20d000fdd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f20e8007fd0 tx=0x7f20d0005830 comp rx=0 tx=0).stop 2024-05-30T16:58:41.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20ef200640 1 -- 172.21.15.67:0/860848755 shutdown_connections 2024-05-30T16:58:41.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20ef200640 1 --2- 172.21.15.67:0/860848755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d0005210 0x7f20d000fdd0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20ef200640 1 -- 172.21.15.67:0/860848755 >> 172.21.15.67:0/860848755 conn(0x7f20f006b900 msgr2=0x7f20f006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:41.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20ef200640 1 -- 172.21.15.67:0/860848755 shutdown_connections 2024-05-30T16:58:41.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.282+0000 7f20ef200640 1 -- 172.21.15.67:0/860848755 wait complete. 2024-05-30T16:58:41.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.284+0000 7f20ef200640 1 Processor -- start 2024-05-30T16:58:41.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.284+0000 7f20ef200640 1 -- start start 2024-05-30T16:58:41.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.285+0000 7f20ef200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d0005210 0x7f20f014fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:41.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.285+0000 7f20ef200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f014e090 0x7f20f014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:41.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.285+0000 7f20ef200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f015fec0 0x7f20f0160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.285+0000 7f20ef200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20f015d6a0 con 0x7f20f015fec0 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.285+0000 7f20ef200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20f00b48c0 con 0x7f20d0005210 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.285+0000 7f20ef200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20f00b4d50 con 0x7f20f014e090 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.285+0000 7f20ee800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f015fec0 0x7f20f0160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ee800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f015fec0 0x7f20f0160340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56244/0 (socket says 172.21.15.67:56244) 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ee800640 1 -- 172.21.15.67:0/1226878194 learned_addr learned my addr 172.21.15.67:0/1226878194 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ed400640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f014e090 0x7f20f014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ede00640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d0005210 0x7f20f014fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ee800640 1 -- 172.21.15.67:0/1226878194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f014e090 msgr2=0x7f20f014d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ee800640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f014e090 0x7f20f014d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ee800640 1 -- 172.21.15.67:0/1226878194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d0005210 msgr2=0x7f20f014fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ee800640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d0005210 0x7f20f014fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ee800640 1 -- 172.21.15.67:0/1226878194 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f014e4f0 con 0x7f20f015fec0 2024-05-30T16:58:41.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.286+0000 7f20ee800640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f015fec0 0x7f20f0160340 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c1d0 tx=0x7f20f00b4830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:41.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.287+0000 7f20e6000640 1 -- 172.21.15.67:0/1226878194 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8001640 con 0x7f20f015fec0 2024-05-30T16:58:41.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.287+0000 7f20e6000640 1 -- 172.21.15.67:0/1226878194 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e80047e0 con 0x7f20f015fec0 2024-05-30T16:58:41.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.287+0000 7f20e6000640 1 -- 172.21.15.67:0/1226878194 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8004bb0 con 0x7f20f015fec0 2024-05-30T16:58:41.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.287+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f00cde60 con 0x7f20f015fec0 2024-05-30T16:58:41.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.287+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f00ce420 con 0x7f20f015fec0 2024-05-30T16:58:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.288+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f0160880 con 0x7f20f015fec0 2024-05-30T16:58:41.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.289+0000 7f20e6000640 1 -- 172.21.15.67:0/1226878194 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 32) ==== 92621+0+0 (secure 0 0 0) 0x7f20e803a070 con 0x7f20f015fec0 2024-05-30T16:58:41.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.290+0000 7f20e6000640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20cc06da60 0x7f20cc06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:58:41.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.290+0000 7f20e6000640 1 -- 172.21.15.67:0/1226878194 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f20e80a9f60 con 0x7f20f015fec0 2024-05-30T16:58:41.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.290+0000 7f20ede00640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20cc06da60 0x7f20cc06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:58:41.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.291+0000 7f20ede00640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20cc06da60 0x7f20cc06ff20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009a10 tx=0x7f20e0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:58:41.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.295+0000 7f20e6000640 1 -- 172.21.15.67:0/1226878194 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20e8079a90 con 0x7f20f015fec0 2024-05-30T16:58:41.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.528+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f20f00b3c40 con 0x7f20cc06da60 2024-05-30T16:58:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.536+0000 7f20e6000640 1 -- 172.21.15.67:0/1226878194 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f20f00b3c40 con 0x7f20cc06da60 2024-05-30T16:58:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4 2024-05-30T16:58:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20cc06da60 msgr2=0x7f20cc06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20cc06da60 0x7f20cc06ff20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009a10 tx=0x7f20e0008040 comp rx=0 tx=0).stop 2024-05-30T16:58:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f015fec0 msgr2=0x7f20f0160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:58:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f015fec0 0x7f20f0160340 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c1d0 tx=0x7f20f00b4830 comp rx=0 tx=0).stop 2024-05-30T16:58:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 shutdown_connections 2024-05-30T16:58:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20cc06da60 0x7f20cc06ff20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f015fec0 0x7f20f0160340 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f014e090 0x7f20f014d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 --2- 172.21.15.67:0/1226878194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d0005210 0x7f20f014fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:58:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.539+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 >> 172.21.15.67:0/1226878194 conn(0x7f20f006b900 msgr2=0x7f20f00afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:58:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.540+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 shutdown_connections 2024-05-30T16:58:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:58:41.540+0000 7f20ef200640 1 -- 172.21.15.67:0/1226878194 wait complete. 2024-05-30T16:58:41.560 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4 2024-05-30T16:58:41.560 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:58:41.560 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:41.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:41.615 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:58:44.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:58:44.058 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:58:57.748 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:58:57.748 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:58:57.748 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.0476 s, 66.9 MB/s 2024-05-30T16:58:57.750 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:58:57.750 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:58:57.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:58:57.805 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:59:12.974 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:59:12.974 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:59:12.974 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.1086 s, 71.1 MB/s 2024-05-30T16:59:12.976 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T16:59:12.976 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T16:59:13.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:59:13.032 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T16:59:14.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:59:14.180 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:59:27.149 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T16:59:27.149 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T16:59:27.149 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.0539 s, 76.4 MB/s 2024-05-30T16:59:27.151 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-30T16:59:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.375+0000 7f0e9ca00640 1 Processor -- start 2024-05-30T16:59:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.375+0000 7f0e9ca00640 1 -- start start 2024-05-30T16:59:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.376+0000 7f0e9ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0e981600f0 0x7f0e980d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.377+0000 7f0e9ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e98072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.377+0000 7f0e9ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e980d0d90 con 0x7f0e9815fa00 2024-05-30T16:59:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.377+0000 7f0e9ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9806fa00 con 0x7f0e981600f0 2024-05-30T16:59:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.377+0000 7f0e9ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9806f160 con 0x7f0e980d0970 2024-05-30T16:59:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.377+0000 7f0e97200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e98072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.378+0000 7f0e97200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e98072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:59110/0 (socket says 172.21.15.67:59110) 2024-05-30T16:59:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.378+0000 7f0e97200640 1 -- 172.21.15.67:0/501907422 learned_addr learned my addr 172.21.15.67:0/501907422 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.378+0000 7f0e8fe00640 1 --2- 172.21.15.67:0/501907422 >> v2:172.21.15.169:3300/0 conn(0x7f0e981600f0 0x7f0e980d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.378+0000 7f0e97200640 1 -- 172.21.15.67:0/501907422 >> v2:172.21.15.169:3300/0 conn(0x7f0e981600f0 msgr2=0x7f0e980d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.378+0000 7f0e97200640 1 --2- 172.21.15.67:0/501907422 >> v2:172.21.15.169:3300/0 conn(0x7f0e981600f0 0x7f0e980d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.378+0000 7f0e97200640 1 -- 172.21.15.67:0/501907422 >> v1:172.21.15.169:6789/0 conn(0x7f0e9815fa00 legacy=0x7f0e9806e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.378+0000 7f0e97200640 1 -- 172.21.15.67:0/501907422 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9806f440 con 0x7f0e980d0970 2024-05-30T16:59:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.380+0000 7f0e97200640 1 --2- 172.21.15.67:0/501907422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e98072190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009e50 tx=0x7f0e9002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9992fb9e1b414064 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.380+0000 7f0e95e00640 1 -- 172.21.15.67:0/501907422 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90036070 con 0x7f0e980d0970 2024-05-30T16:59:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.380+0000 7f0e95e00640 1 -- 172.21.15.67:0/501907422 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e90031600 con 0x7f0e980d0970 2024-05-30T16:59:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.380+0000 7f0e95e00640 1 -- 172.21.15.67:0/501907422 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e900319d0 con 0x7f0e980d0970 2024-05-30T16:59:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.381+0000 7f0e9ca00640 1 -- 172.21.15.67:0/501907422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 msgr2=0x7f0e98072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.381+0000 7f0e9ca00640 1 --2- 172.21.15.67:0/501907422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e98072190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009e50 tx=0x7f0e9002a5f0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.381+0000 7f0e9ca00640 1 -- 172.21.15.67:0/501907422 shutdown_connections 2024-05-30T16:59:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.381+0000 7f0e9ca00640 1 --2- 172.21.15.67:0/501907422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e98072190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.381+0000 7f0e9ca00640 1 --2- 172.21.15.67:0/501907422 >> v2:172.21.15.169:3300/0 conn(0x7f0e981600f0 0x7f0e980d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.381+0000 7f0e9ca00640 1 -- 172.21.15.67:0/501907422 >> 172.21.15.67:0/501907422 conn(0x7f0e9806bae0 msgr2=0x7f0e9806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.381+0000 7f0e9ca00640 1 -- 172.21.15.67:0/501907422 shutdown_connections 2024-05-30T16:59:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.381+0000 7f0e9ca00640 1 -- 172.21.15.67:0/501907422 wait complete. 2024-05-30T16:59:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.383+0000 7f0e9ca00640 1 Processor -- start 2024-05-30T16:59:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.383+0000 7f0e9ca00640 1 -- start start 2024-05-30T16:59:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.384+0000 7f0e9ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e981541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.384+0000 7f0e9ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9815fa00 0x7f0e9809b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.384+0000 7f0e9ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e981600f0 0x7f0e980b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.384+0000 7f0e9ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9815fde0 con 0x7f0e981600f0 2024-05-30T16:59:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.384+0000 7f0e9ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98071c70 con 0x7f0e9815fa00 2024-05-30T16:59:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.384+0000 7f0e9ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e980726d0 con 0x7f0e980d0970 2024-05-30T16:59:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e97200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e981600f0 0x7f0e980b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e97200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e981600f0 0x7f0e980b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59308/0 (socket says 172.21.15.67:59308) 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e97200640 1 -- 172.21.15.67:0/3771701310 learned_addr learned my addr 172.21.15.67:0/3771701310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e8fe00640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9815fa00 0x7f0e9809b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e96800640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e981541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e97200640 1 -- 172.21.15.67:0/3771701310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 msgr2=0x7f0e981541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e97200640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e981541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e97200640 1 -- 172.21.15.67:0/3771701310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9815fa00 msgr2=0x7f0e9809b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e97200640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9815fa00 0x7f0e9809b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e97200640 1 -- 172.21.15.67:0/3771701310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e980af150 con 0x7f0e981600f0 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e96800640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e981541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:27.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.385+0000 7f0e97200640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e981600f0 0x7f0e980b3be0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f0e90034040 tx=0x7f0e90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:27.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.386+0000 7f0e8ea00640 1 -- 172.21.15.67:0/3771701310 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90036050 con 0x7f0e981600f0 2024-05-30T16:59:27.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.386+0000 7f0e8ea00640 1 -- 172.21.15.67:0/3771701310 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e90042070 con 0x7f0e981600f0 2024-05-30T16:59:27.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.386+0000 7f0e8ea00640 1 -- 172.21.15.67:0/3771701310 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90007bd0 con 0x7f0e981600f0 2024-05-30T16:59:27.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.386+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e98154700 con 0x7f0e981600f0 2024-05-30T16:59:27.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.386+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e981586d0 con 0x7f0e981600f0 2024-05-30T16:59:27.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.387+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e980689e0 con 0x7f0e981600f0 2024-05-30T16:59:27.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.388+0000 7f0e8ea00640 1 -- 172.21.15.67:0/3771701310 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 33) ==== 92672+0+0 (secure 0 0 0) 0x7f0e90047460 con 0x7f0e981600f0 2024-05-30T16:59:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.389+0000 7f0e8ea00640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e7406daf0 0x7f0e7406ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.389+0000 7f0e8ea00640 1 -- 172.21.15.67:0/3771701310 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f0e900ab990 con 0x7f0e981600f0 2024-05-30T16:59:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.390+0000 7f0e96800640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e7406daf0 0x7f0e7406ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.390+0000 7f0e96800640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e7406daf0 0x7f0e7406ffb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0e88000f60 tx=0x7f0e88009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:27.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.394+0000 7f0e8ea00640 1 -- 172.21.15.67:0/3771701310 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e9007b500 con 0x7f0e981600f0 2024-05-30T16:59:27.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.632+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f0e980cda60 con 0x7f0e7406daf0 2024-05-30T16:59:27.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.660+0000 7f0e8ea00640 1 -- 172.21.15.67:0/3771701310 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0e980cda60 con 0x7f0e7406daf0 2024-05-30T16:59:27.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e7406daf0 msgr2=0x7f0e7406ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:27.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e7406daf0 0x7f0e7406ffb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0e88000f60 tx=0x7f0e88009040 comp rx=0 tx=0).stop 2024-05-30T16:59:27.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e981600f0 msgr2=0x7f0e980b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:27.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e981600f0 0x7f0e980b3be0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f0e90034040 tx=0x7f0e90004a40 comp rx=0 tx=0).stop 2024-05-30T16:59:27.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 shutdown_connections 2024-05-30T16:59:27.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e7406daf0 0x7f0e7406ffb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e981600f0 0x7f0e980b3be0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9815fa00 0x7f0e9809b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 --2- 172.21.15.67:0/3771701310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e980d0970 0x7f0e981541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.663+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 >> 172.21.15.67:0/3771701310 conn(0x7f0e9806bae0 msgr2=0x7f0e980cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:27.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.664+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 shutdown_connections 2024-05-30T16:59:27.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.664+0000 7f0e9ca00640 1 -- 172.21.15.67:0/3771701310 wait complete. 2024-05-30T16:59:27.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T16:59:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.898+0000 7f1c3b200640 1 Processor -- start 2024-05-30T16:59:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.899+0000 7f1c3b200640 1 -- start start 2024-05-30T16:59:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.900+0000 7f1c3b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 0x7f1c3c0d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.900+0000 7f1c3b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1c3c0d4a60 0x7f1c3c0d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.900+0000 7f1c3b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c3c0d0150 con 0x7f1c3c0b46a0 2024-05-30T16:59:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.900+0000 7f1c3b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1c3c0d28b0 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.900+0000 7f1c3b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3c0d0430 con 0x7f1c3c0d4680 2024-05-30T16:59:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.900+0000 7f1c31400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1c3c0d4a60 0x7f1c3c0d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.900+0000 7f1c31400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1c3c0d4a60 0x7f1c3c0d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59310/0 (socket says 172.21.15.67:59310) 2024-05-30T16:59:27.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.900+0000 7f1c31400640 1 -- 172.21.15.67:0/3155915598 learned_addr learned my addr 172.21.15.67:0/3155915598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:27.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.901+0000 7f1c39e00640 1 --2- 172.21.15.67:0/3155915598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 0x7f1c3c0d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.901+0000 7f1c31400640 1 -- 172.21.15.67:0/3155915598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 msgr2=0x7f1c3c0d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:27.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.901+0000 7f1c31400640 1 --2- 172.21.15.67:0/3155915598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 0x7f1c3c0d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.901+0000 7f1c31400640 1 -- 172.21.15.67:0/3155915598 >> v1:172.21.15.169:6789/0 conn(0x7f1c3c0b46a0 legacy=0x7f1c3c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:27.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.901+0000 7f1c31400640 1 -- 172.21.15.67:0/3155915598 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3c0d2260 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.902+0000 7f1c31400640 1 --2- 172.21.15.67:0/3155915598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 0x7f1c3c0d3280 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009880 tx=0x7f1c2402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f5c420746abac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:27.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.902+0000 7f1c39400640 1 -- 172.21.15.67:0/3155915598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24004030 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.902+0000 7f1c39400640 1 -- 172.21.15.67:0/3155915598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c240041d0 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.902+0000 7f1c39400640 1 -- 172.21.15.67:0/3155915598 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24003610 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.903+0000 7f1c3b200640 1 -- 172.21.15.67:0/3155915598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 msgr2=0x7f1c3c0d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:27.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.903+0000 7f1c3b200640 1 --2- 172.21.15.67:0/3155915598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 0x7f1c3c0d3280 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009880 tx=0x7f1c2402a210 comp rx=0 tx=0).stop 2024-05-30T16:59:27.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.903+0000 7f1c3b200640 1 -- 172.21.15.67:0/3155915598 shutdown_connections 2024-05-30T16:59:27.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.903+0000 7f1c3b200640 1 --2- 172.21.15.67:0/3155915598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 0x7f1c3c0d3280 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.903+0000 7f1c3b200640 1 --2- 172.21.15.67:0/3155915598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 0x7f1c3c0d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.903+0000 7f1c3b200640 1 -- 172.21.15.67:0/3155915598 >> 172.21.15.67:0/3155915598 conn(0x7f1c3c09d810 msgr2=0x7f1c3c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:27.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.903+0000 7f1c3b200640 1 -- 172.21.15.67:0/3155915598 shutdown_connections 2024-05-30T16:59:27.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.903+0000 7f1c3b200640 1 -- 172.21.15.67:0/3155915598 wait complete. 2024-05-30T16:59:27.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.905+0000 7f1c3b200640 1 Processor -- start 2024-05-30T16:59:27.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.905+0000 7f1c3b200640 1 -- start start 2024-05-30T16:59:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3c0b46a0 0x7f1c3c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 0x7f1c3c15f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 0x7f1c3c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c15eb70 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c15e9f0 con 0x7f1c3c0b46a0 2024-05-30T16:59:27.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3c15ecf0 con 0x7f1c3c0d4680 2024-05-30T16:59:27.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 0x7f1c3c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 0x7f1c3c15f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59312/0 (socket says 172.21.15.67:59312) 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3a800640 1 -- 172.21.15.67:0/2844903931 learned_addr learned my addr 172.21.15.67:0/2844903931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c31400640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 0x7f1c3c15f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c39e00640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3c0b46a0 0x7f1c3c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3a800640 1 -- 172.21.15.67:0/2844903931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 msgr2=0x7f1c3c15f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3a800640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 0x7f1c3c15f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3a800640 1 -- 172.21.15.67:0/2844903931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3c0b46a0 msgr2=0x7f1c3c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3a800640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3c0b46a0 0x7f1c3c0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c3a800640 1 -- 172.21.15.67:0/2844903931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3c0c8050 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.906+0000 7f1c39e00640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3c0b46a0 0x7f1c3c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:27.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.907+0000 7f1c3a800640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 0x7f1c3c15f8b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400c780 tx=0x7f1c3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:27.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.907+0000 7f1c32a00640 1 -- 172.21.15.67:0/2844903931 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34004b60 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.907+0000 7f1c32a00640 1 -- 172.21.15.67:0/2844903931 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c34004d00 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.907+0000 7f1c32a00640 1 -- 172.21.15.67:0/2844903931 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400aa90 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.907+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3c0c8330 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.907+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3c0c88f0 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.909+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c3c0689e0 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.910+0000 7f1c32a00640 1 -- 172.21.15.67:0/2844903931 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 33) ==== 92672+0+0 (secure 0 0 0) 0x7f1c3400ac30 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.910+0000 7f1c32a00640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c1806daf0 0x7f1c1806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.911+0000 7f1c32a00640 1 -- 172.21.15.67:0/2844903931 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f1c3408ada0 con 0x7f1c3c0d4a60 2024-05-30T16:59:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.911+0000 7f1c39e00640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c1806daf0 0x7f1c1806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:27.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.911+0000 7f1c39e00640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c1806daf0 0x7f1c1806ffb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c008890 tx=0x7f1c2c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:27.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:27.915+0000 7f1c32a00640 1 -- 172.21.15.67:0/2844903931 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c34020070 con 0x7f1c3c0d4a60 2024-05-30T16:59:28.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.150+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f1c3c0b4090 con 0x7f1c1806daf0 2024-05-30T16:59:28.159 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4 2024-05-30T16:59:28.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.158+0000 7f1c32a00640 1 -- 172.21.15.67:0/2844903931 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f1c3c0b4090 con 0x7f1c1806daf0 2024-05-30T16:59:28.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c1806daf0 msgr2=0x7f1c1806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c1806daf0 0x7f1c1806ffb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c008890 tx=0x7f1c2c008040 comp rx=0 tx=0).stop 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 msgr2=0x7f1c3c15f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 0x7f1c3c15f8b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400c780 tx=0x7f1c3400cc40 comp rx=0 tx=0).stop 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 shutdown_connections 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1c1806daf0 0x7f1c1806ffb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c3c0d4a60 0x7f1c3c15f8b0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c3c0d4680 0x7f1c3c15f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 --2- 172.21.15.67:0/2844903931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3c0b46a0 0x7f1c3c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 >> 172.21.15.67:0/2844903931 conn(0x7f1c3c09d810 msgr2=0x7f1c3c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 shutdown_connections 2024-05-30T16:59:28.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:28.161+0000 7f1c3b200640 1 -- 172.21.15.67:0/2844903931 wait complete. 2024-05-30T16:59:28.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:59:28.182 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4 2024-05-30T16:59:28.182 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:59:28.245 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4 2024-05-30T16:59:28.245 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3138256896" 2024-05-30T16:59:28.245 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:59:28.246 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-30T16:59:30.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T16:59:30.247 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4 2024-05-30T16:59:30.248 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T16:59:30.320 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/946d4bad-85c5-4e0c-a585-a5f5840584e4 2024-05-30T16:59:30.320 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3221225472" 2024-05-30T16:59:30.320 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:59:30.321 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-30T16:59:30.321 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-30T16:59:30.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.545+0000 7fb199000640 1 Processor -- start 2024-05-30T16:59:30.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.545+0000 7fb199000640 1 -- start start 2024-05-30T16:59:30.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.547+0000 7fb199000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb194153510 0x7fb1940b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:30.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.547+0000 7fb199000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1940ccb90 0x7fb1940ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:30.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.547+0000 7fb199000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb19415d900 con 0x7fb19415cb60 2024-05-30T16:59:30.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.547+0000 7fb199000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb1941538f0 con 0x7fb194153510 2024-05-30T16:59:30.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.547+0000 7fb199000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb1940cd530 con 0x7fb1940ccb90 2024-05-30T16:59:30.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.547+0000 7fb193400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb194153510 0x7fb1940b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:30.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.547+0000 7fb193400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb194153510 0x7fb1940b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59316/0 (socket says 172.21.15.67:59316) 2024-05-30T16:59:30.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.547+0000 7fb193400640 1 -- 172.21.15.67:0/3883688097 learned_addr learned my addr 172.21.15.67:0/3883688097 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:30.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.548+0000 7fb193400640 1 -- 172.21.15.67:0/3883688097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1940ccb90 msgr2=0x7fb1940ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:30.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.548+0000 7fb192a00640 1 --2- 172.21.15.67:0/3883688097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1940ccb90 0x7fb1940ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:30.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.548+0000 7fb193400640 1 --2- 172.21.15.67:0/3883688097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1940ccb90 0x7fb1940ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.548+0000 7fb193400640 1 -- 172.21.15.67:0/3883688097 >> v1:172.21.15.169:6789/0 conn(0x7fb19415cb60 legacy=0x7fb19415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:30.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.548+0000 7fb193400640 1 -- 172.21.15.67:0/3883688097 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1940b3f90 con 0x7fb194153510 2024-05-30T16:59:30.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.549+0000 7fb193400640 1 --2- 172.21.15.67:0/3883688097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb194153510 0x7fb1940b4d50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb184009870 tx=0x7fb18402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf9fc2aa25067f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:30.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.549+0000 7fb192000640 1 -- 172.21.15.67:0/3883688097 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184037070 con 0x7fb194153510 2024-05-30T16:59:30.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.549+0000 7fb192000640 1 -- 172.21.15.67:0/3883688097 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb184032440 con 0x7fb194153510 2024-05-30T16:59:30.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.549+0000 7fb192000640 1 -- 172.21.15.67:0/3883688097 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184032810 con 0x7fb194153510 2024-05-30T16:59:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.550+0000 7fb199000640 1 -- 172.21.15.67:0/3883688097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb194153510 msgr2=0x7fb1940b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.550+0000 7fb199000640 1 --2- 172.21.15.67:0/3883688097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb194153510 0x7fb1940b4d50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb184009870 tx=0x7fb18402a180 comp rx=0 tx=0).stop 2024-05-30T16:59:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.550+0000 7fb199000640 1 -- 172.21.15.67:0/3883688097 shutdown_connections 2024-05-30T16:59:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.550+0000 7fb199000640 1 --2- 172.21.15.67:0/3883688097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1940ccb90 0x7fb1940ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.550+0000 7fb199000640 1 --2- 172.21.15.67:0/3883688097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb194153510 0x7fb1940b4d50 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.550+0000 7fb199000640 1 -- 172.21.15.67:0/3883688097 >> 172.21.15.67:0/3883688097 conn(0x7fb19406b900 msgr2=0x7fb19406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.550+0000 7fb199000640 1 -- 172.21.15.67:0/3883688097 shutdown_connections 2024-05-30T16:59:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.550+0000 7fb199000640 1 -- 172.21.15.67:0/3883688097 wait complete. 2024-05-30T16:59:30.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.552+0000 7fb199000640 1 Processor -- start 2024-05-30T16:59:30.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.552+0000 7fb199000640 1 -- start start 2024-05-30T16:59:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.553+0000 7fb199000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1940ccb90 0x7fb19415db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.553+0000 7fb199000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb194153510 0x7fb1941474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.553+0000 7fb199000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19415cb60 0x7fb1940ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.553+0000 7fb199000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb1940b4550 con 0x7fb1940ccb90 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.553+0000 7fb199000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1940b48c0 con 0x7fb19415cb60 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.553+0000 7fb199000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb19406e940 con 0x7fb194153510 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1940ccb90 0x7fb19415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1940ccb90 0x7fb19415db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59320/0 (socket says 172.21.15.67:59320) 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193400640 1 -- 172.21.15.67:0/3327446890 learned_addr learned my addr 172.21.15.67:0/3327446890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193400640 1 -- 172.21.15.67:0/3327446890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb194153510 msgr2=0x7fb1941474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193400640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb194153510 0x7fb1941474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193400640 1 -- 172.21.15.67:0/3327446890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19415cb60 msgr2=0x7fb1940ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193e00640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19415cb60 0x7fb1940ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:30.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193400640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19415cb60 0x7fb1940ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193400640 1 -- 172.21.15.67:0/3327446890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19415e230 con 0x7fb1940ccb90 2024-05-30T16:59:30.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.554+0000 7fb193e00640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19415cb60 0x7fb1940ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:30.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.555+0000 7fb193400640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1940ccb90 0x7fb19415db00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb184009870 tx=0x7fb1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:30.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.555+0000 7fb18b400640 1 -- 172.21.15.67:0/3327446890 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184041020 con 0x7fb1940ccb90 2024-05-30T16:59:30.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.555+0000 7fb18b400640 1 -- 172.21.15.67:0/3327446890 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb184030970 con 0x7fb1940ccb90 2024-05-30T16:59:30.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.555+0000 7fb18b400640 1 -- 172.21.15.67:0/3327446890 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184037040 con 0x7fb1940ccb90 2024-05-30T16:59:30.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.555+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19415e510 con 0x7fb1940ccb90 2024-05-30T16:59:30.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.555+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19415ead0 con 0x7fb1940ccb90 2024-05-30T16:59:30.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.556+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1940689e0 con 0x7fb1940ccb90 2024-05-30T16:59:30.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.557+0000 7fb18b400640 1 -- 172.21.15.67:0/3327446890 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 33) ==== 92672+0+0 (secure 0 0 0) 0x7fb184044020 con 0x7fb1940ccb90 2024-05-30T16:59:30.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.558+0000 7fb18b400640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb17406daf0 0x7fb17406ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:30.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.558+0000 7fb18b400640 1 -- 172.21.15.67:0/3327446890 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fb1840ab0f0 con 0x7fb1940ccb90 2024-05-30T16:59:30.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.559+0000 7fb192a00640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb17406daf0 0x7fb17406ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.559+0000 7fb192a00640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb17406daf0 0x7fb17406ffb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb180006fd0 tx=0x7fb180008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.563+0000 7fb18b400640 1 -- 172.21.15.67:0/3327446890 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb18404b030 con 0x7fb1940ccb90 2024-05-30T16:59:30.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.804+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fb1940af380 con 0x7fb17406daf0 2024-05-30T16:59:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.959+0000 7fb18b400640 1 -- 172.21.15.67:0/3327446890 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb1940af380 con 0x7fb17406daf0 2024-05-30T16:59:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb17406daf0 msgr2=0x7fb17406ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb17406daf0 0x7fb17406ffb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb180006fd0 tx=0x7fb180008040 comp rx=0 tx=0).stop 2024-05-30T16:59:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1940ccb90 msgr2=0x7fb19415db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1940ccb90 0x7fb19415db00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb184009870 tx=0x7fb1840046c0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 shutdown_connections 2024-05-30T16:59:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb17406daf0 0x7fb17406ffb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19415cb60 0x7fb1940ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb194153510 0x7fb1941474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 --2- 172.21.15.67:0/3327446890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1940ccb90 0x7fb19415db00 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 >> 172.21.15.67:0/3327446890 conn(0x7fb19406b900 msgr2=0x7fb1940cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 shutdown_connections 2024-05-30T16:59:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:30.963+0000 7fb199000640 1 -- 172.21.15.67:0/3327446890 wait complete. 2024-05-30T16:59:30.966 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.965+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:30.966 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.965+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:30.966 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.965+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:30.966 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.965+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:30.966 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.965+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:30.986 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-05-30T16:59:30.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:30.997 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.996+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:30.997 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.996+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:30.997 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.996+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:30.997 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.996+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:30.997 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:30.996+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:31.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.177+0000 7fd97fe00640 1 Processor -- start 2024-05-30T16:59:31.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.177+0000 7fd97fe00640 1 -- start start 2024-05-30T16:59:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.178+0000 7fd97fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97815e850 0x7fd9780b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.178+0000 7fd97fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9780cef30 0x7fd9780d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.178+0000 7fd97fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9780b8000 con 0x7fd9780d1b80 2024-05-30T16:59:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.178+0000 7fd97fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd97815ec30 con 0x7fd9780cef30 2024-05-30T16:59:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.178+0000 7fd97fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd9780d1fd0 con 0x7fd97815e850 2024-05-30T16:59:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.179+0000 7fd97d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97815e850 0x7fd9780b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.179+0000 7fd97de00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd9780d1b80 0x7fd978162340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:35568/0 (socket says 172.21.15.67:35568) 2024-05-30T16:59:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.179+0000 7fd97d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97815e850 0x7fd9780b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:34610/0 (socket says 172.21.15.67:34610) 2024-05-30T16:59:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.179+0000 7fd97de00640 1 -- 172.21.15.67:0/1274186898 learned_addr learned my addr 172.21.15.67:0/1274186898 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.180+0000 7fd97ca00640 1 --2- 172.21.15.67:0/1274186898 >> v2:172.21.15.169:3300/0 conn(0x7fd9780cef30 0x7fd9780d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.180+0000 7fd977e00640 1 -- 172.21.15.67:0/1274186898 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1781104118 0 0) 0x7fd9780b8000 con 0x7fd9780d1b80 2024-05-30T16:59:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.180+0000 7fd977e00640 1 -- 172.21.15.67:0/1274186898 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd960003520 con 0x7fd9780d1b80 2024-05-30T16:59:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.180+0000 7fd97d400640 1 -- 172.21.15.67:0/1274186898 >> v2:172.21.15.169:3300/0 conn(0x7fd9780cef30 msgr2=0x7fd9780d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.180+0000 7fd97d400640 1 --2- 172.21.15.67:0/1274186898 >> v2:172.21.15.169:3300/0 conn(0x7fd9780cef30 0x7fd9780d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.180+0000 7fd97d400640 1 -- 172.21.15.67:0/1274186898 >> v1:172.21.15.169:6789/0 conn(0x7fd9780d1b80 legacy=0x7fd978162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.180+0000 7fd97d400640 1 -- 172.21.15.67:0/1274186898 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9780b82e0 con 0x7fd97815e850 2024-05-30T16:59:31.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.182+0000 7fd97d400640 1 --2- 172.21.15.67:0/1274186898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97815e850 0x7fd9780b77b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0097a0 tx=0x7fd96c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6ca9d792546850 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.182+0000 7fd977e00640 1 -- 172.21.15.67:0/1274186898 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c037070 con 0x7fd97815e850 2024-05-30T16:59:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.182+0000 7fd977e00640 1 -- 172.21.15.67:0/1274186898 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd96c02ae50 con 0x7fd97815e850 2024-05-30T16:59:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.182+0000 7fd977e00640 1 -- 172.21.15.67:0/1274186898 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c032660 con 0x7fd97815e850 2024-05-30T16:59:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.183+0000 7fd97fe00640 1 -- 172.21.15.67:0/1274186898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97815e850 msgr2=0x7fd9780b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.183+0000 7fd97fe00640 1 --2- 172.21.15.67:0/1274186898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97815e850 0x7fd9780b77b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0097a0 tx=0x7fd96c02a100 comp rx=0 tx=0).stop 2024-05-30T16:59:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.183+0000 7fd97fe00640 1 -- 172.21.15.67:0/1274186898 shutdown_connections 2024-05-30T16:59:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.183+0000 7fd97fe00640 1 --2- 172.21.15.67:0/1274186898 >> v2:172.21.15.169:3300/0 conn(0x7fd9780cef30 0x7fd9780d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.183+0000 7fd97fe00640 1 --2- 172.21.15.67:0/1274186898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd97815e850 0x7fd9780b77b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.183+0000 7fd97fe00640 1 -- 172.21.15.67:0/1274186898 >> 172.21.15.67:0/1274186898 conn(0x7fd978067aa0 msgr2=0x7fd978067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.183+0000 7fd97fe00640 1 -- 172.21.15.67:0/1274186898 shutdown_connections 2024-05-30T16:59:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.183+0000 7fd97fe00640 1 -- 172.21.15.67:0/1274186898 wait complete. 2024-05-30T16:59:31.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.185+0000 7fd97fe00640 1 Processor -- start 2024-05-30T16:59:31.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.185+0000 7fd97fe00640 1 -- start start 2024-05-30T16:59:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.186+0000 7fd97fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9780cef30 0x7fd97815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.186+0000 7fd97fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780d1b80 0x7fd9780cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.186+0000 7fd97fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97815e850 0x7fd9781f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.186+0000 7fd97fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780b7cf0 con 0x7fd9780cef30 2024-05-30T16:59:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.186+0000 7fd97fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780b88a0 con 0x7fd97815e850 2024-05-30T16:59:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.186+0000 7fd97fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd9780ba0b0 con 0x7fd9780d1b80 2024-05-30T16:59:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.186+0000 7fd97d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9780cef30 0x7fd97815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.186+0000 7fd97d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9780cef30 0x7fd97815cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40120/0 (socket says 172.21.15.67:40120) 2024-05-30T16:59:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.186+0000 7fd97d400640 1 -- 172.21.15.67:0/2338114257 learned_addr learned my addr 172.21.15.67:0/2338114257 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:31.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97ca00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780d1b80 0x7fd9780cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:31.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97de00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97815e850 0x7fd9781f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:31.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97d400640 1 -- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780d1b80 msgr2=0x7fd9780cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97d400640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780d1b80 0x7fd9780cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97d400640 1 -- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97815e850 msgr2=0x7fd9781f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97d400640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97815e850 0x7fd9781f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97d400640 1 -- 172.21.15.67:0/2338114257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9780cc640 con 0x7fd9780cef30 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97ca00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780d1b80 0x7fd9780cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97de00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97815e850 0x7fd9781f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.187+0000 7fd97d400640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9780cef30 0x7fd97815cd10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fd96c000c00 tx=0x7fd96c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.188+0000 7fd975600640 1 -- 172.21.15.67:0/2338114257 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c041070 con 0x7fd9780cef30 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.188+0000 7fd975600640 1 -- 172.21.15.67:0/2338114257 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd96c03c460 con 0x7fd9780cef30 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.188+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9780cc920 con 0x7fd9780cef30 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.188+0000 7fd975600640 1 -- 172.21.15.67:0/2338114257 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c037040 con 0x7fd9780cef30 2024-05-30T16:59:31.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.188+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9780ccee0 con 0x7fd9780cef30 2024-05-30T16:59:31.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.189+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9780689e0 con 0x7fd9780cef30 2024-05-30T16:59:31.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.190+0000 7fd975600640 1 -- 172.21.15.67:0/2338114257 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 33) ==== 92672+0+0 (secure 0 0 0) 0x7fd96c044070 con 0x7fd9780cef30 2024-05-30T16:59:31.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.191+0000 7fd975600640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806daf0 0x7fd95806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:31.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.191+0000 7fd975600640 1 -- 172.21.15.67:0/2338114257 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fd96c0aab00 con 0x7fd9780cef30 2024-05-30T16:59:31.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.191+0000 7fd97ca00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806daf0 0x7fd95806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:31.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.192+0000 7fd97ca00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806daf0 0x7fd95806ffb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd9781f5450 tx=0x7fd964008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.196+0000 7fd975600640 1 -- 172.21.15.67:0/2338114257 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd96c07a980 con 0x7fd9780cef30 2024-05-30T16:59:31.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:31.439+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd9780b3000 con 0x7fd95806daf0 2024-05-30T16:59:31.969 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:31.967+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:31.969 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:31.967+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:31.969 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:31.967+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:31.969 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:31.967+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:31.969 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T16:59:31.967+0000 7f0c7c200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T16:59:32.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.026+0000 7fd975600640 1 -- 172.21.15.67:0/2338114257 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+275 (secure 0 0 0) 0x7fd9780b3000 con 0x7fd95806daf0 2024-05-30T16:59:32.027 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "in-progress", 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "source": { 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "volume": "cephfs", 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "subvolume": "sv1", 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot": "ss1" 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_report": { 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "percentage cloned": 0.0, 2024-05-30T16:59:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "amount cloned": "184/3.0G", 2024-05-30T16:59:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "files cloned": "3/6" 2024-05-30T16:59:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:32.029 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:32.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.029+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806daf0 msgr2=0x7fd95806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:32.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.029+0000 7fd97fe00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806daf0 0x7fd95806ffb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd9781f5450 tx=0x7fd964008040 comp rx=0 tx=0).stop 2024-05-30T16:59:32.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.029+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9780cef30 msgr2=0x7fd97815cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:32.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.029+0000 7fd97fe00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9780cef30 0x7fd97815cd10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fd96c000c00 tx=0x7fd96c0046c0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.030+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 shutdown_connections 2024-05-30T16:59:32.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.030+0000 7fd97fe00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd95806daf0 0x7fd95806ffb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.030+0000 7fd97fe00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd97815e850 0x7fd9781f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.030+0000 7fd97fe00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9780d1b80 0x7fd9780cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.030+0000 7fd97fe00640 1 --2- 172.21.15.67:0/2338114257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9780cef30 0x7fd97815cd10 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.030+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 >> 172.21.15.67:0/2338114257 conn(0x7fd978067aa0 msgr2=0x7fd97815ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:32.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.030+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 shutdown_connections 2024-05-30T16:59:32.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.030+0000 7fd97fe00640 1 -- 172.21.15.67:0/2338114257 wait complete. 2024-05-30T16:59:32.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.354+0000 7fa340400640 1 Processor -- start 2024-05-30T16:59:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.354+0000 7fa340400640 1 -- start start 2024-05-30T16:59:32.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.357+0000 7fa340400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa338070880 0x7fa338070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:32.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.357+0000 7fa340400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3381602e0 0x7fa338070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.357+0000 7fa340400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3380711a0 con 0x7fa3381624e0 2024-05-30T16:59:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.357+0000 7fa340400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa33809ba20 con 0x7fa338070880 2024-05-30T16:59:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.357+0000 7fa340400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa33815f470 con 0x7fa3381602e0 2024-05-30T16:59:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.358+0000 7fa33e400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa3381624e0 0x7fa338162970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:35580/0 (socket says 172.21.15.67:35580) 2024-05-30T16:59:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.358+0000 7fa33e400640 1 -- 172.21.15.67:0/2271390232 learned_addr learned my addr 172.21.15.67:0/2271390232 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.358+0000 7fa33d000640 1 --2- 172.21.15.67:0/2271390232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3381602e0 0x7fa338070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.358+0000 7fa33da00640 1 --2- 172.21.15.67:0/2271390232 >> v2:172.21.15.67:3300/0 conn(0x7fa338070880 0x7fa338070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.359+0000 7fa33da00640 1 -- 172.21.15.67:0/2271390232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3381602e0 msgr2=0x7fa338070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.359+0000 7fa33da00640 1 --2- 172.21.15.67:0/2271390232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3381602e0 0x7fa338070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.359+0000 7fa33da00640 1 -- 172.21.15.67:0/2271390232 >> v1:172.21.15.169:6789/0 conn(0x7fa3381624e0 legacy=0x7fa338162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.359+0000 7fa33da00640 1 -- 172.21.15.67:0/2271390232 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33815f750 con 0x7fa338070880 2024-05-30T16:59:32.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.360+0000 7fa33da00640 1 --2- 172.21.15.67:0/2271390232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa338070880 0x7fa338070c60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa328009dc0 tx=0x7fa32802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575bebf5ca527751 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:32.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.361+0000 7fa337e00640 1 -- 172.21.15.67:0/2271390232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32802ae60 con 0x7fa338070880 2024-05-30T16:59:32.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.361+0000 7fa337e00640 1 -- 172.21.15.67:0/2271390232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa328007740 con 0x7fa338070880 2024-05-30T16:59:32.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.361+0000 7fa337e00640 1 -- 172.21.15.67:0/2271390232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328007b10 con 0x7fa338070880 2024-05-30T16:59:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.362+0000 7fa340400640 1 -- 172.21.15.67:0/2271390232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa338070880 msgr2=0x7fa338070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.362+0000 7fa340400640 1 --2- 172.21.15.67:0/2271390232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa338070880 0x7fa338070c60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa328009dc0 tx=0x7fa32802a680 comp rx=0 tx=0).stop 2024-05-30T16:59:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.362+0000 7fa340400640 1 -- 172.21.15.67:0/2271390232 shutdown_connections 2024-05-30T16:59:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.362+0000 7fa340400640 1 --2- 172.21.15.67:0/2271390232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3381602e0 0x7fa338070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.362+0000 7fa340400640 1 --2- 172.21.15.67:0/2271390232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa338070880 0x7fa338070c60 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.362+0000 7fa340400640 1 -- 172.21.15.67:0/2271390232 >> 172.21.15.67:0/2271390232 conn(0x7fa3380b3ff0 msgr2=0x7fa3380b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.363+0000 7fa340400640 1 -- 172.21.15.67:0/2271390232 shutdown_connections 2024-05-30T16:59:32.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.363+0000 7fa340400640 1 -- 172.21.15.67:0/2271390232 wait complete. 2024-05-30T16:59:32.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.365+0000 7fa340400640 1 Processor -- start 2024-05-30T16:59:32.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.365+0000 7fa340400640 1 -- start start 2024-05-30T16:59:32.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa340400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa338070880 0x7fa338158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa340400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3381602e0 0x7fa3380b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa340400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3381624e0 0x7fa3381f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa340400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa3380705c0 con 0x7fa3381602e0 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa340400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa33815fd10 con 0x7fa3381624e0 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa340400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa3380d07d0 con 0x7fa338070880 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa33d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3381602e0 0x7fa3380b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa33d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3381602e0 0x7fa3380b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40168/0 (socket says 172.21.15.67:40168) 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa33d000640 1 -- 172.21.15.67:0/4127660392 learned_addr learned my addr 172.21.15.67:0/4127660392 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa33da00640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa338070880 0x7fa338158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.366+0000 7fa33e400640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3381624e0 0x7fa3381f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:32.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.367+0000 7fa33d000640 1 -- 172.21.15.67:0/4127660392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa338070880 msgr2=0x7fa338158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:32.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.367+0000 7fa33d000640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa338070880 0x7fa338158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.367+0000 7fa33d000640 1 -- 172.21.15.67:0/4127660392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3381624e0 msgr2=0x7fa3381f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:32.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.367+0000 7fa33d000640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3381624e0 0x7fa3381f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.367+0000 7fa33d000640 1 -- 172.21.15.67:0/4127660392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3380b7a70 con 0x7fa3381602e0 2024-05-30T16:59:32.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.367+0000 7fa33d000640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3381602e0 0x7fa3380b5960 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e870 tx=0x7fa32400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:32.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.367+0000 7fa335600640 1 -- 172.21.15.67:0/4127660392 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324017070 con 0x7fa3381602e0 2024-05-30T16:59:32.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.368+0000 7fa335600640 1 -- 172.21.15.67:0/4127660392 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa32400c940 con 0x7fa3381602e0 2024-05-30T16:59:32.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.368+0000 7fa335600640 1 -- 172.21.15.67:0/4127660392 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32400cd30 con 0x7fa3381602e0 2024-05-30T16:59:32.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.368+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3380b7d50 con 0x7fa3381602e0 2024-05-30T16:59:32.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.368+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3380b75b0 con 0x7fa3381602e0 2024-05-30T16:59:32.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.369+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3380690f0 con 0x7fa3381602e0 2024-05-30T16:59:32.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.370+0000 7fa335600640 1 -- 172.21.15.67:0/4127660392 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 33) ==== 92672+0+0 (secure 0 0 0) 0x7fa32401a070 con 0x7fa3381602e0 2024-05-30T16:59:32.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.372+0000 7fa335600640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa31806daf0 0x7fa31806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:32.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.372+0000 7fa335600640 1 -- 172.21.15.67:0/4127660392 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fa32408abc0 con 0x7fa3381602e0 2024-05-30T16:59:32.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.373+0000 7fa33da00640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa31806daf0 0x7fa31806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:32.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.379+0000 7fa335600640 1 -- 172.21.15.67:0/4127660392 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa32405a0e0 con 0x7fa3381602e0 2024-05-30T16:59:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.394+0000 7fa33da00640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa31806daf0 0x7fa31806ffb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa328034000 tx=0x7fa32802f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:32.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.781+0000 7fa335600640 1 -- 172.21.15.67:0/4127660392 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7fa324056750 con 0x7fa3381602e0 2024-05-30T16:59:32.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.816+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa3380b4d50 con 0x7fa31806daf0 2024-05-30T16:59:32.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.856+0000 7fa335600640 1 -- 172.21.15.67:0/4127660392 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+275 (secure 0 0 0) 0x7fa3380b4d50 con 0x7fa31806daf0 2024-05-30T16:59:32.857 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:32.857 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:32.857 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "in-progress", 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: "source": { 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: "volume": "cephfs", 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: "subvolume": "sv1", 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot": "ss1" 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_report": { 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: "percentage cloned": 0.0, 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: "amount cloned": "184/3.0G", 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: "files cloned": "4/6" 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:32.858 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.861+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa31806daf0 msgr2=0x7fa31806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.861+0000 7fa340400640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa31806daf0 0x7fa31806ffb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa328034000 tx=0x7fa32802f000 comp rx=0 tx=0).stop 2024-05-30T16:59:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.861+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3381602e0 msgr2=0x7fa3380b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.861+0000 7fa340400640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3381602e0 0x7fa3380b5960 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e870 tx=0x7fa32400ed30 comp rx=0 tx=0).stop 2024-05-30T16:59:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.862+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 shutdown_connections 2024-05-30T16:59:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.862+0000 7fa340400640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa31806daf0 0x7fa31806ffb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.862+0000 7fa340400640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3381624e0 0x7fa3381f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.862+0000 7fa340400640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3381602e0 0x7fa3380b5960 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.862+0000 7fa340400640 1 --2- 172.21.15.67:0/4127660392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa338070880 0x7fa338158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.862+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 >> 172.21.15.67:0/4127660392 conn(0x7fa3380b3ff0 msgr2=0x7fa338074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:32.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.862+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 shutdown_connections 2024-05-30T16:59:32.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:32.863+0000 7fa340400640 1 -- 172.21.15.67:0/4127660392 wait complete. 2024-05-30T16:59:32.897 INFO:tasks.cephfs.test_volumes:percentage cloned = 0.0 2024-05-30T16:59:32.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:33.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.229+0000 7f93db800640 1 Processor -- start 2024-05-30T16:59:33.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.230+0000 7f93db800640 1 -- start start 2024-05-30T16:59:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.231+0000 7f93db800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f93d40cef30 0x7f93d40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.231+0000 7f93db800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f93d40d1b80 0x7f93d40b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.231+0000 7f93db800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d40d1f60 con 0x7f93d415e850 2024-05-30T16:59:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.231+0000 7f93db800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f93d415ec30 con 0x7f93d40d1b80 2024-05-30T16:59:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.231+0000 7f93db800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f93d40b8000 con 0x7f93d40cef30 2024-05-30T16:59:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.231+0000 7f93d9800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f93d40d1b80 0x7f93d40b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.231+0000 7f93d9800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f93d40d1b80 0x7f93d40b90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40170/0 (socket says 172.21.15.67:40170) 2024-05-30T16:59:33.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.231+0000 7f93d9800640 1 -- 172.21.15.67:0/2017575434 learned_addr learned my addr 172.21.15.67:0/2017575434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.232+0000 7f93d9800640 1 -- 172.21.15.67:0/2017575434 >> v2:172.21.15.169:3300/0 conn(0x7f93d40cef30 msgr2=0x7f93d40d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.232+0000 7f93cbe00640 1 --2- 172.21.15.67:0/2017575434 >> v2:172.21.15.169:3300/0 conn(0x7f93d40cef30 0x7f93d40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.232+0000 7f93d9800640 1 --2- 172.21.15.67:0/2017575434 >> v2:172.21.15.169:3300/0 conn(0x7f93d40cef30 0x7f93d40d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.232+0000 7f93d9800640 1 -- 172.21.15.67:0/2017575434 >> v1:172.21.15.169:6789/0 conn(0x7f93d415e850 legacy=0x7f93d40b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.232+0000 7f93d9800640 1 -- 172.21.15.67:0/2017575434 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d40b82e0 con 0x7f93d40d1b80 2024-05-30T16:59:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.232+0000 7f93cbe00640 1 --2- 172.21.15.67:0/2017575434 >> v2:172.21.15.169:3300/0 conn(0x7f93d40cef30 0x7f93d40d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.233+0000 7f93d9800640 1 --2- 172.21.15.67:0/2017575434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40d1b80 0x7f93d40b90a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f93d40cb730 tx=0x7f93cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b07f4cce2a3b97 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.234+0000 7f93d3e00640 1 -- 172.21.15.67:0/2017575434 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc036070 con 0x7f93d40d1b80 2024-05-30T16:59:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.234+0000 7f93d3e00640 1 -- 172.21.15.67:0/2017575434 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93cc031600 con 0x7f93d40d1b80 2024-05-30T16:59:33.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.234+0000 7f93d3e00640 1 -- 172.21.15.67:0/2017575434 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc0319d0 con 0x7f93d40d1b80 2024-05-30T16:59:33.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.235+0000 7f93db800640 1 -- 172.21.15.67:0/2017575434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40d1b80 msgr2=0x7f93d40b90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:33.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.235+0000 7f93db800640 1 --2- 172.21.15.67:0/2017575434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40d1b80 0x7f93d40b90a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f93d40cb730 tx=0x7f93cc02a660 comp rx=0 tx=0).stop 2024-05-30T16:59:33.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.235+0000 7f93db800640 1 -- 172.21.15.67:0/2017575434 shutdown_connections 2024-05-30T16:59:33.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.235+0000 7f93db800640 1 --2- 172.21.15.67:0/2017575434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40d1b80 0x7f93d40b90a0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.235+0000 7f93db800640 1 --2- 172.21.15.67:0/2017575434 >> v2:172.21.15.169:3300/0 conn(0x7f93d40cef30 0x7f93d40d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.235+0000 7f93db800640 1 -- 172.21.15.67:0/2017575434 >> 172.21.15.67:0/2017575434 conn(0x7f93d4067aa0 msgr2=0x7f93d4067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:33.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.235+0000 7f93db800640 1 -- 172.21.15.67:0/2017575434 shutdown_connections 2024-05-30T16:59:33.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.235+0000 7f93db800640 1 -- 172.21.15.67:0/2017575434 wait complete. 2024-05-30T16:59:33.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.238+0000 7f93db800640 1 Processor -- start 2024-05-30T16:59:33.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.239+0000 7f93db800640 1 -- start start 2024-05-30T16:59:33.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.241+0000 7f93db800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40cef30 0x7f93d4151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:33.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.241+0000 7f93db800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40d1b80 0x7f93d4151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:33.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.241+0000 7f93db800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e850 0x7f93d4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:33.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.241+0000 7f93db800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40b95e0 con 0x7f93d40cef30 2024-05-30T16:59:33.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.241+0000 7f93db800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93d40b88a0 con 0x7f93d415e850 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.241+0000 7f93db800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f93d40b9760 con 0x7f93d40d1b80 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.241+0000 7f93d8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40cef30 0x7f93d4151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.241+0000 7f93d8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40cef30 0x7f93d4151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40182/0 (socket says 172.21.15.67:40182) 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.241+0000 7f93d8e00640 1 -- 172.21.15.67:0/749779641 learned_addr learned my addr 172.21.15.67:0/749779641 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.242+0000 7f93cbe00640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40d1b80 0x7f93d4151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.242+0000 7f93d9800640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e850 0x7f93d4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.242+0000 7f93d8e00640 1 -- 172.21.15.67:0/749779641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40d1b80 msgr2=0x7f93d4151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.242+0000 7f93d8e00640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40d1b80 0x7f93d4151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.242+0000 7f93d8e00640 1 -- 172.21.15.67:0/749779641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e850 msgr2=0x7f93d4151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.242+0000 7f93d8e00640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e850 0x7f93d4151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.242+0000 7f93d8e00640 1 -- 172.21.15.67:0/749779641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d41504f0 con 0x7f93d40cef30 2024-05-30T16:59:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.242+0000 7f93d8e00640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40cef30 0x7f93d4151570 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f93c400b3e0 tx=0x7f93c400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.243+0000 7f93d1600640 1 -- 172.21.15.67:0/749779641 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4015020 con 0x7f93d40cef30 2024-05-30T16:59:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.243+0000 7f93d1600640 1 -- 172.21.15.67:0/749779641 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c4011850 con 0x7f93d40cef30 2024-05-30T16:59:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.243+0000 7f93d1600640 1 -- 172.21.15.67:0/749779641 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4011c40 con 0x7f93d40cef30 2024-05-30T16:59:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.243+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d41507b0 con 0x7f93d40cef30 2024-05-30T16:59:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.243+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d4150d70 con 0x7f93d40cef30 2024-05-30T16:59:33.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.245+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ac005290 con 0x7f93d40cef30 2024-05-30T16:59:33.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.248+0000 7f93d1600640 1 -- 172.21.15.67:0/749779641 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f93c4018020 con 0x7f93d40cef30 2024-05-30T16:59:33.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.250+0000 7f93d1600640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f93b006dcf0 0x7f93b00701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.250+0000 7f93d1600640 1 -- 172.21.15.67:0/749779641 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f93c408a720 con 0x7f93d40cef30 2024-05-30T16:59:33.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.251+0000 7f93cbe00640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f93b006dcf0 0x7f93b00701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.254+0000 7f93cbe00640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f93b006dcf0 0x7f93b00701b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f93d40ac690 tx=0x7f93bc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.257+0000 7f93d1600640 1 -- 172.21.15.67:0/749779641 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f93c401f020 con 0x7f93d40cef30 2024-05-30T16:59:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.692+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f93ac000bc0 con 0x7f93b006dcf0 2024-05-30T16:59:33.734 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:33.734 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:33.734 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "in-progress", 2024-05-30T16:59:33.734 INFO:teuthology.orchestra.run.smithi067.stdout: "source": { 2024-05-30T16:59:33.734 INFO:teuthology.orchestra.run.smithi067.stdout: "volume": "cephfs", 2024-05-30T16:59:33.734 INFO:teuthology.orchestra.run.smithi067.stdout: "subvolume": "sv1", 2024-05-30T16:59:33.734 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot": "ss1" 2024-05-30T16:59:33.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:33.735 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_report": { 2024-05-30T16:59:33.735 INFO:teuthology.orchestra.run.smithi067.stdout: "percentage cloned": 15.885, 2024-05-30T16:59:33.735 INFO:teuthology.orchestra.run.smithi067.stdout: "amount cloned": "488M/3.0G", 2024-05-30T16:59:33.735 INFO:teuthology.orchestra.run.smithi067.stdout: "files cloned": "4/6" 2024-05-30T16:59:33.735 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:33.735 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:33.735 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.731+0000 7f93d1600640 1 -- 172.21.15.67:0/749779641 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f93ac000bc0 con 0x7f93b006dcf0 2024-05-30T16:59:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f93b006dcf0 msgr2=0x7f93b00701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f93b006dcf0 0x7f93b00701b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f93d40ac690 tx=0x7f93bc008040 comp rx=0 tx=0).stop 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40cef30 msgr2=0x7f93d4151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40cef30 0x7f93d4151570 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f93c400b3e0 tx=0x7f93c400b8a0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 shutdown_connections 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f93b006dcf0 0x7f93b00701b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93d415e850 0x7f93d4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93d40d1b80 0x7f93d4151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 --2- 172.21.15.67:0/749779641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93d40cef30 0x7f93d4151570 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 >> 172.21.15.67:0/749779641 conn(0x7f93d4067aa0 msgr2=0x7f93d4074a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 shutdown_connections 2024-05-30T16:59:33.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:33.735+0000 7f93db800640 1 -- 172.21.15.67:0/749779641 wait complete. 2024-05-30T16:59:33.769 INFO:tasks.cephfs.test_volumes:percentage cloned = 15.885 2024-05-30T16:59:33.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:34.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.067+0000 7f551f200640 1 Processor -- start 2024-05-30T16:59:34.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.067+0000 7f551f200640 1 -- start start 2024-05-30T16:59:34.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.070+0000 7f551f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cef30 0x7f55180d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.070+0000 7f551f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55180b8000 con 0x7f551815e850 2024-05-30T16:59:34.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.070+0000 7f551f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55180b82e0 con 0x7f55180d1b80 2024-05-30T16:59:34.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.070+0000 7f551f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f55180d1ff0 con 0x7f55180cef30 2024-05-30T16:59:34.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.070+0000 7f5517e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f551815e850 0x7f55180b77b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43470/0 (socket says 172.21.15.67:43470) 2024-05-30T16:59:34.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.070+0000 7f5517e00640 1 -- 172.21.15.67:0/2391754406 learned_addr learned my addr 172.21.15.67:0/2391754406 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:34.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.071+0000 7f5516a00640 1 -- 172.21.15.67:0/2391754406 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1400148964 0 0) 0x7f55180b8000 con 0x7f551815e850 2024-05-30T16:59:34.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.071+0000 7f5516a00640 1 -- 172.21.15.67:0/2391754406 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54fc003520 con 0x7f551815e850 2024-05-30T16:59:34.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.072+0000 7f5517400640 1 --2- 172.21.15.67:0/2391754406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cef30 0x7f55180d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.072+0000 7f5517400640 1 -- 172.21.15.67:0/2391754406 >> v1:172.21.15.169:6789/0 conn(0x7f55180d1b80 legacy=0x7f55180b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.073+0000 7f5517400640 1 -- 172.21.15.67:0/2391754406 >> v1:172.21.15.67:6789/0 conn(0x7f551815e850 legacy=0x7f55180b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.073+0000 7f5517400640 1 -- 172.21.15.67:0/2391754406 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55180b85c0 con 0x7f55180cef30 2024-05-30T16:59:34.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.075+0000 7f5517400640 1 --2- 172.21.15.67:0/2391754406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cef30 0x7f55180d1640 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5504009750 tx=0x7f550402a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bf641fcf8f5620 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:34.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.076+0000 7f5516a00640 1 -- 172.21.15.67:0/2391754406 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504037070 con 0x7f55180cef30 2024-05-30T16:59:34.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.076+0000 7f5516a00640 1 -- 172.21.15.67:0/2391754406 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f550402ad20 con 0x7f55180cef30 2024-05-30T16:59:34.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.076+0000 7f5516a00640 1 -- 172.21.15.67:0/2391754406 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504032660 con 0x7f55180cef30 2024-05-30T16:59:34.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.077+0000 7f551f200640 1 -- 172.21.15.67:0/2391754406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cef30 msgr2=0x7f55180d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.077+0000 7f551f200640 1 --2- 172.21.15.67:0/2391754406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cef30 0x7f55180d1640 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5504009750 tx=0x7f550402a010 comp rx=0 tx=0).stop 2024-05-30T16:59:34.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.077+0000 7f551f200640 1 -- 172.21.15.67:0/2391754406 shutdown_connections 2024-05-30T16:59:34.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.077+0000 7f551f200640 1 --2- 172.21.15.67:0/2391754406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cef30 0x7f55180d1640 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.077+0000 7f551f200640 1 -- 172.21.15.67:0/2391754406 >> 172.21.15.67:0/2391754406 conn(0x7f5518067aa0 msgr2=0x7f5518067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:34.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.078+0000 7f551f200640 1 -- 172.21.15.67:0/2391754406 shutdown_connections 2024-05-30T16:59:34.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.078+0000 7f551f200640 1 -- 172.21.15.67:0/2391754406 wait complete. 2024-05-30T16:59:34.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.081+0000 7f551f200640 1 Processor -- start 2024-05-30T16:59:34.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.081+0000 7f551f200640 1 -- start start 2024-05-30T16:59:34.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.082+0000 7f551f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cef30 0x7f5518156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.082+0000 7f551f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55180d1b80 0x7f55181f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.082+0000 7f551f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815e850 0x7f551814b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.082+0000 7f551f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55180b97b0 con 0x7f55180cef30 2024-05-30T16:59:34.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.082+0000 7f551f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55180b9930 con 0x7f55180d1b80 2024-05-30T16:59:34.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.082+0000 7f551f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f55180b9ab0 con 0x7f551815e850 2024-05-30T16:59:34.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.083+0000 7f5517e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cef30 0x7f5518156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.083+0000 7f551d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815e850 0x7f551814b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.083+0000 7f5517e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cef30 0x7f5518156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40190/0 (socket says 172.21.15.67:40190) 2024-05-30T16:59:34.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.083+0000 7f551d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815e850 0x7f551814b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:34686/0 (socket says 172.21.15.67:34686) 2024-05-30T16:59:34.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.083+0000 7f5517e00640 1 -- 172.21.15.67:0/2149560468 learned_addr learned my addr 172.21.15.67:0/2149560468 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:34.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.083+0000 7f5517400640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55180d1b80 0x7f55181f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.084+0000 7f5517e00640 1 -- 172.21.15.67:0/2149560468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815e850 msgr2=0x7f551814b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.084+0000 7f5517e00640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815e850 0x7f551814b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.084+0000 7f5517e00640 1 -- 172.21.15.67:0/2149560468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55180d1b80 msgr2=0x7f55181f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.084+0000 7f5517e00640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55180d1b80 0x7f55181f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.084+0000 7f5517e00640 1 -- 172.21.15.67:0/2149560468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55181f4ab0 con 0x7f55180cef30 2024-05-30T16:59:34.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.085+0000 7f5517e00640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cef30 0x7f5518156170 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f550800c3c0 tx=0x7f550800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:34.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.085+0000 7f550fe00640 1 -- 172.21.15.67:0/2149560468 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508017070 con 0x7f55180cef30 2024-05-30T16:59:34.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.085+0000 7f550fe00640 1 -- 172.21.15.67:0/2149560468 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5508005ce0 con 0x7f55180cef30 2024-05-30T16:59:34.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.085+0000 7f550fe00640 1 -- 172.21.15.67:0/2149560468 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55080072d0 con 0x7f55180cef30 2024-05-30T16:59:34.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.086+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55180d5320 con 0x7f55180cef30 2024-05-30T16:59:34.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.086+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55180cd190 con 0x7f55180cef30 2024-05-30T16:59:34.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.087+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55180689e0 con 0x7f55180cef30 2024-05-30T16:59:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.090+0000 7f550fe00640 1 -- 172.21.15.67:0/2149560468 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f550801a070 con 0x7f55180cef30 2024-05-30T16:59:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.092+0000 7f550fe00640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f806dd70 0x7f54f8070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.092+0000 7f550fe00640 1 -- 172.21.15.67:0/2149560468 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f550808a390 con 0x7f55180cef30 2024-05-30T16:59:34.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.094+0000 7f5517400640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f806dd70 0x7f54f8070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.095+0000 7f5517400640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f806dd70 0x7f54f8070230 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5504034000 tx=0x7f550402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:34.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.098+0000 7f550fe00640 1 -- 172.21.15.67:0/2149560468 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55180d5320 con 0x7f55180cef30 2024-05-30T16:59:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.463+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f55180b94d0 con 0x7f54f806dd70 2024-05-30T16:59:34.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.497+0000 7f550fe00640 1 -- 172.21.15.67:0/2149560468 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f55180b94d0 con 0x7f54f806dd70 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "in-progress", 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "source": { 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "volume": "cephfs", 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "subvolume": "sv1", 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot": "ss1" 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_report": { 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "percentage cloned": 33.333, 2024-05-30T16:59:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "amount cloned": "1.0G/3.0G", 2024-05-30T16:59:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: "files cloned": "4/6" 2024-05-30T16:59:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:34.500 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f806dd70 msgr2=0x7f54f8070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f806dd70 0x7f54f8070230 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5504034000 tx=0x7f550402e000 comp rx=0 tx=0).stop 2024-05-30T16:59:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cef30 msgr2=0x7f5518156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cef30 0x7f5518156170 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f550800c3c0 tx=0x7f550800c880 comp rx=0 tx=0).stop 2024-05-30T16:59:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 shutdown_connections 2024-05-30T16:59:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f806dd70 0x7f54f8070230 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815e850 0x7f551814b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55180d1b80 0x7f55181f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 --2- 172.21.15.67:0/2149560468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cef30 0x7f5518156170 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 >> 172.21.15.67:0/2149560468 conn(0x7f5518067aa0 msgr2=0x7f5518162a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 shutdown_connections 2024-05-30T16:59:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.501+0000 7f551f200640 1 -- 172.21.15.67:0/2149560468 wait complete. 2024-05-30T16:59:34.531 INFO:tasks.cephfs.test_volumes:percentage cloned = 33.333 2024-05-30T16:59:34.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.799+0000 7fc96e600640 1 Processor -- start 2024-05-30T16:59:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.799+0000 7fc96e600640 1 -- start start 2024-05-30T16:59:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.800+0000 7fc96e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9680b5f90 0x7fc96815f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.800+0000 7fc96e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc9680d2610 0x7fc9680b6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.801+0000 7fc96e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9680b68b0 con 0x7fc968162300 2024-05-30T16:59:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.801+0000 7fc96e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc9680d37e0 con 0x7fc9680d2610 2024-05-30T16:59:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.801+0000 7fc96e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc9680d2bd0 con 0x7fc9680b5f90 2024-05-30T16:59:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.801+0000 7fc967e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc9680d2610 0x7fc9680b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.801+0000 7fc967e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc9680d2610 0x7fc9680b6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40198/0 (socket says 172.21.15.67:40198) 2024-05-30T16:59:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.801+0000 7fc967e00640 1 -- 172.21.15.67:0/2546301775 learned_addr learned my addr 172.21.15.67:0/2546301775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.801+0000 7fc96d200640 1 --2- 172.21.15.67:0/2546301775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9680b5f90 0x7fc96815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.802+0000 7fc967e00640 1 -- 172.21.15.67:0/2546301775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9680b5f90 msgr2=0x7fc96815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.802+0000 7fc967e00640 1 --2- 172.21.15.67:0/2546301775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9680b5f90 0x7fc96815f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.802+0000 7fc967e00640 1 -- 172.21.15.67:0/2546301775 >> v1:172.21.15.67:6789/0 conn(0x7fc968162300 legacy=0x7fc968162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.802+0000 7fc967e00640 1 -- 172.21.15.67:0/2546301775 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9680d2eb0 con 0x7fc9680d2610 2024-05-30T16:59:34.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.804+0000 7fc967e00640 1 --2- 172.21.15.67:0/2546301775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9680d2610 0x7fc9680b6370 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc954009850 tx=0x7fc95402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f987f2a8b5b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:34.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.807+0000 7fc967400640 1 -- 172.21.15.67:0/2546301775 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954004030 con 0x7fc9680d2610 2024-05-30T16:59:34.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.807+0000 7fc967400640 1 -- 172.21.15.67:0/2546301775 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9540041d0 con 0x7fc9680d2610 2024-05-30T16:59:34.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.807+0000 7fc967400640 1 -- 172.21.15.67:0/2546301775 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954003610 con 0x7fc9680d2610 2024-05-30T16:59:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.808+0000 7fc96e600640 1 -- 172.21.15.67:0/2546301775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9680d2610 msgr2=0x7fc9680b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.808+0000 7fc96e600640 1 --2- 172.21.15.67:0/2546301775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9680d2610 0x7fc9680b6370 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc954009850 tx=0x7fc95402a230 comp rx=0 tx=0).stop 2024-05-30T16:59:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.809+0000 7fc96e600640 1 -- 172.21.15.67:0/2546301775 shutdown_connections 2024-05-30T16:59:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.809+0000 7fc96e600640 1 --2- 172.21.15.67:0/2546301775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9680d2610 0x7fc9680b6370 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.809+0000 7fc96e600640 1 --2- 172.21.15.67:0/2546301775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9680b5f90 0x7fc96815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.809+0000 7fc96e600640 1 -- 172.21.15.67:0/2546301775 >> 172.21.15.67:0/2546301775 conn(0x7fc96808bfc0 msgr2=0x7fc9680d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.809+0000 7fc96e600640 1 -- 172.21.15.67:0/2546301775 shutdown_connections 2024-05-30T16:59:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.809+0000 7fc96e600640 1 -- 172.21.15.67:0/2546301775 wait complete. 2024-05-30T16:59:34.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.812+0000 7fc96e600640 1 Processor -- start 2024-05-30T16:59:34.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.812+0000 7fc96e600640 1 -- start start 2024-05-30T16:59:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.813+0000 7fc96e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9680b5f90 0x7fc968148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.814+0000 7fc96e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9680d2610 0x7fc9680b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.814+0000 7fc96e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc968162300 0x7fc9680b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.814+0000 7fc96e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc9680d4ee0 con 0x7fc9680d2610 2024-05-30T16:59:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.814+0000 7fc96e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc9680d3470 con 0x7fc9680b5f90 2024-05-30T16:59:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.814+0000 7fc96e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc96815eac0 con 0x7fc968162300 2024-05-30T16:59:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.815+0000 7fc96d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9680b5f90 0x7fc968148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.815+0000 7fc96dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc968162300 0x7fc9680b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.815+0000 7fc96dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc968162300 0x7fc9680b88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:34700/0 (socket says 172.21.15.67:34700) 2024-05-30T16:59:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.815+0000 7fc96dc00640 1 -- 172.21.15.67:0/3277458400 learned_addr learned my addr 172.21.15.67:0/3277458400 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:34.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.815+0000 7fc96d200640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9680b5f90 0x7fc968148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:45924/0 (socket says 172.21.15.67:45924) 2024-05-30T16:59:34.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.815+0000 7fc967e00640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9680d2610 0x7fc9680b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.816+0000 7fc96dc00640 1 -- 172.21.15.67:0/3277458400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9680b5f90 msgr2=0x7fc968148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.816+0000 7fc96dc00640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9680b5f90 0x7fc968148a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.816+0000 7fc96dc00640 1 -- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9680d2610 msgr2=0x7fc9680b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.816+0000 7fc96dc00640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9680d2610 0x7fc9680b83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.816+0000 7fc96dc00640 1 -- 172.21.15.67:0/3277458400 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc968146290 con 0x7fc968162300 2024-05-30T16:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.816+0000 7fc96d200640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9680b5f90 0x7fc968148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.817+0000 7fc96dc00640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc968162300 0x7fc9680b88f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c6b0 tx=0x7fc96000cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.818+0000 7fc964c00640 1 -- 172.21.15.67:0/3277458400 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960003b70 con 0x7fc968162300 2024-05-30T16:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.818+0000 7fc964c00640 1 -- 172.21.15.67:0/3277458400 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc960003d30 con 0x7fc968162300 2024-05-30T16:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.818+0000 7fc964c00640 1 -- 172.21.15.67:0/3277458400 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc960005310 con 0x7fc968162300 2024-05-30T16:59:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.818+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc968146530 con 0x7fc968162300 2024-05-30T16:59:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.818+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc968146af0 con 0x7fc968162300 2024-05-30T16:59:34.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.820+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc968156980 con 0x7fc968162300 2024-05-30T16:59:34.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.823+0000 7fc964c00640 1 -- 172.21.15.67:0/3277458400 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7fc9600054b0 con 0x7fc968162300 2024-05-30T16:59:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.825+0000 7fc964c00640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc94806dd20 0x7fc9480701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.825+0000 7fc964c00640 1 -- 172.21.15.67:0/3277458400 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fc96008a4a0 con 0x7fc968162300 2024-05-30T16:59:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.828+0000 7fc96d200640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc94806dd20 0x7fc9480701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.829+0000 7fc96d200640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc94806dd20 0x7fc9480701e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006fd0 tx=0x7fc95c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:34.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:34.836+0000 7fc964c00640 1 -- 172.21.15.67:0/3277458400 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc960059d10 con 0x7fc968162300 2024-05-30T16:59:35.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.204+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fc9680689e0 con 0x7fc94806dd20 2024-05-30T16:59:35.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.237+0000 7fc964c00640 1 -- 172.21.15.67:0/3277458400 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7fc9680689e0 con 0x7fc94806dd20 2024-05-30T16:59:35.239 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:35.239 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:35.239 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "in-progress", 2024-05-30T16:59:35.239 INFO:teuthology.orchestra.run.smithi067.stdout: "source": { 2024-05-30T16:59:35.239 INFO:teuthology.orchestra.run.smithi067.stdout: "volume": "cephfs", 2024-05-30T16:59:35.239 INFO:teuthology.orchestra.run.smithi067.stdout: "subvolume": "sv1", 2024-05-30T16:59:35.240 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot": "ss1" 2024-05-30T16:59:35.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:35.240 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_report": { 2024-05-30T16:59:35.240 INFO:teuthology.orchestra.run.smithi067.stdout: "percentage cloned": 33.333, 2024-05-30T16:59:35.240 INFO:teuthology.orchestra.run.smithi067.stdout: "amount cloned": "1.0G/3.0G", 2024-05-30T16:59:35.240 INFO:teuthology.orchestra.run.smithi067.stdout: "files cloned": "4/6" 2024-05-30T16:59:35.240 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:35.240 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:35.240 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.240+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc94806dd20 msgr2=0x7fc9480701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.240+0000 7fc96e600640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc94806dd20 0x7fc9480701e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006fd0 tx=0x7fc95c008040 comp rx=0 tx=0).stop 2024-05-30T16:59:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.240+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc968162300 msgr2=0x7fc9680b88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.240+0000 7fc96e600640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc968162300 0x7fc9680b88f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc96000c6b0 tx=0x7fc96000cb70 comp rx=0 tx=0).stop 2024-05-30T16:59:35.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.240+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 shutdown_connections 2024-05-30T16:59:35.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.240+0000 7fc96e600640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc94806dd20 0x7fc9480701e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.240+0000 7fc96e600640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc968162300 0x7fc9680b88f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.241+0000 7fc96e600640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9680d2610 0x7fc9680b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.241+0000 7fc96e600640 1 --2- 172.21.15.67:0/3277458400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9680b5f90 0x7fc968148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.241+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 >> 172.21.15.67:0/3277458400 conn(0x7fc96808bfc0 msgr2=0x7fc9680b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:35.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.241+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 shutdown_connections 2024-05-30T16:59:35.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.241+0000 7fc96e600640 1 -- 172.21.15.67:0/3277458400 wait complete. 2024-05-30T16:59:35.269 INFO:tasks.cephfs.test_volumes:percentage cloned = 33.333 2024-05-30T16:59:35.269 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:35.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.567+0000 7f9acdc00640 1 Processor -- start 2024-05-30T16:59:35.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.567+0000 7f9acdc00640 1 -- start start 2024-05-30T16:59:35.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.569+0000 7f9acdc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9ac80d0640 0x7f9ac80d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:35.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.569+0000 7f9acdc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac80ade90 con 0x7f9ac815b6f0 2024-05-30T16:59:35.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.569+0000 7f9acdc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac80d0f60 con 0x7f9ac80725e0 2024-05-30T16:59:35.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.569+0000 7f9acdc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9ac8071c40 con 0x7f9ac80d0640 2024-05-30T16:59:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.569+0000 7f9ac6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9ac815b6f0 0x7f9ac806e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43500/0 (socket says 172.21.15.67:43500) 2024-05-30T16:59:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.569+0000 7f9ac6000640 1 -- 172.21.15.67:0/3008780800 learned_addr learned my addr 172.21.15.67:0/3008780800 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.569+0000 7f9ac6a00640 1 --2- 172.21.15.67:0/3008780800 >> v2:172.21.15.169:3300/0 conn(0x7f9ac80d0640 0x7f9ac80d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 551091172 0 0) 0x7f9ac80ade90 con 0x7f9ac815b6f0 2024-05-30T16:59:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9aac005730 con 0x7f9ac815b6f0 2024-05-30T16:59:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4130641301 0 0) 0x7f9aac005730 con 0x7f9ac815b6f0 2024-05-30T16:59:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 >> v2:172.21.15.169:3300/0 conn(0x7f9ac80d0640 msgr2=0x7f9ac80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 --2- 172.21.15.67:0/3008780800 >> v2:172.21.15.169:3300/0 conn(0x7f9ac80d0640 0x7f9ac80d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 >> v1:172.21.15.169:6789/0 conn(0x7f9ac80725e0 legacy=0x7f9ac80ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac8071e70 con 0x7f9ac815b6f0 2024-05-30T16:59:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9ab0001670 con 0x7f9ac815b6f0 2024-05-30T16:59:35.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 >> v1:172.21.15.67:6789/0 conn(0x7f9ac815b6f0 legacy=0x7f9ac806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:35.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.570+0000 7f9ac5600640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac005c10 0x7f9aac006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.571+0000 7f9ac5600640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aac006570 0x7f9aac006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.571+0000 7f9ac5600640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac007180 0x7f9aac00ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.571+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac010510 con 0x7f9aac005c10 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.571+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac010690 con 0x7f9aac007180 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.571+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac010840 con 0x7f9aac006570 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.571+0000 7f9ac6a00640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac005c10 0x7f9aac006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.571+0000 7f9ac6000640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aac006570 0x7f9aac006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.572+0000 7f9ac7400640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac007180 0x7f9aac00ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.572+0000 7f9ac6a00640 1 -- 172.21.15.67:0/3008780800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aac006570 msgr2=0x7f9aac006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.572+0000 7f9ac6a00640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9aac006570 0x7f9aac006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.572+0000 7f9ac6a00640 1 -- 172.21.15.67:0/3008780800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac007180 msgr2=0x7f9aac00ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.572+0000 7f9ac6a00640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac007180 0x7f9aac00ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.572+0000 7f9ac7400640 1 -- 172.21.15.67:0/3008780800 reap_dead start 2024-05-30T16:59:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.572+0000 7f9ac6a00640 1 -- 172.21.15.67:0/3008780800 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac005730 con 0x7f9aac005c10 2024-05-30T16:59:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.574+0000 7f9ac6a00640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac005c10 0x7f9aac006030 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009fc0 tx=0x7f9aac007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c3db14cdbb7943 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.575+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0009c10 con 0x7f9aac005c10 2024-05-30T16:59:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.575+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac0036070 con 0x7f9aac005c10 2024-05-30T16:59:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.575+0000 7f9ac5600640 1 -- 172.21.15.67:0/3008780800 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0031880 con 0x7f9aac005c10 2024-05-30T16:59:35.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.576+0000 7f9acdc00640 1 -- 172.21.15.67:0/3008780800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac005c10 msgr2=0x7f9aac006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:35.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.576+0000 7f9acdc00640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac005c10 0x7f9aac006030 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009fc0 tx=0x7f9aac007a40 comp rx=0 tx=0).stop 2024-05-30T16:59:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.577+0000 7f9acdc00640 1 -- 172.21.15.67:0/3008780800 shutdown_connections 2024-05-30T16:59:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.577+0000 7f9acdc00640 1 --2- 172.21.15.67:0/3008780800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9aac005c10 0x7f9aac006030 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.577+0000 7f9acdc00640 1 -- 172.21.15.67:0/3008780800 >> 172.21.15.67:0/3008780800 conn(0x7f9ac806bd20 msgr2=0x7f9ac806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.577+0000 7f9acdc00640 1 -- 172.21.15.67:0/3008780800 shutdown_connections 2024-05-30T16:59:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.577+0000 7f9acdc00640 1 -- 172.21.15.67:0/3008780800 wait complete. 2024-05-30T16:59:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.580+0000 7f9acdc00640 1 Processor -- start 2024-05-30T16:59:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.580+0000 7f9acdc00640 1 -- start start 2024-05-30T16:59:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.581+0000 7f9acdc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac005c10 0x7f9ac814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.581+0000 7f9acdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac80cd780 0x7f9ac80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.581+0000 7f9acdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ac815bad0 0x7f9ac815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.581+0000 7f9acdc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac80d0a20 con 0x7f9ac80cd780 2024-05-30T16:59:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.581+0000 7f9acdc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac80ae130 con 0x7f9aac005c10 2024-05-30T16:59:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.581+0000 7f9acdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac80d0ba0 con 0x7f9ac815bad0 2024-05-30T16:59:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.581+0000 7f9ac6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac80cd780 0x7f9ac80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.581+0000 7f9ac6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac80cd780 0x7f9ac80cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40226/0 (socket says 172.21.15.67:40226) 2024-05-30T16:59:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.581+0000 7f9ac6000640 1 -- 172.21.15.67:0/1666036155 learned_addr learned my addr 172.21.15.67:0/1666036155 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.582+0000 7f9ac6000640 1 -- 172.21.15.67:0/1666036155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ac815bad0 msgr2=0x7f9ac815bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.582+0000 7f9ac6000640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ac815bad0 0x7f9ac815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.582+0000 7f9ac6000640 1 -- 172.21.15.67:0/1666036155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac005c10 msgr2=0x7f9ac814d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.582+0000 7f9ac6000640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac005c10 0x7f9ac814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.582+0000 7f9ac6000640 1 -- 172.21.15.67:0/1666036155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac80ce4f0 con 0x7f9ac80cd780 2024-05-30T16:59:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.582+0000 7f9ac6000640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac80cd780 0x7f9ac80cdc00 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0000c00 tx=0x7f9ab0007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.583+0000 7f9abea00640 1 -- 172.21.15.67:0/1666036155 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000f8e0 con 0x7f9ac80cd780 2024-05-30T16:59:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.583+0000 7f9abea00640 1 -- 172.21.15.67:0/1666036155 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0018070 con 0x7f9ac80cd780 2024-05-30T16:59:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.583+0000 7f9abea00640 1 -- 172.21.15.67:0/1666036155 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000fd00 con 0x7f9ac80cd780 2024-05-30T16:59:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.583+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac80ce7d0 con 0x7f9ac80cd780 2024-05-30T16:59:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.583+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac80ced90 con 0x7f9ac80cd780 2024-05-30T16:59:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.585+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac80689e0 con 0x7f9ac80cd780 2024-05-30T16:59:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.586+0000 7f9abea00640 1 -- 172.21.15.67:0/1666036155 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f9ab001d460 con 0x7f9ac80cd780 2024-05-30T16:59:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.587+0000 7f9abea00640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9aa406dca0 0x7f9aa4070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.587+0000 7f9abea00640 1 -- 172.21.15.67:0/1666036155 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f9ab008ab30 con 0x7f9ac80cd780 2024-05-30T16:59:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.587+0000 7f9ac6a00640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9aa406dca0 0x7f9aa4070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.588+0000 7f9ac6a00640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9aa406dca0 0x7f9aa4070160 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0035000 tx=0x7f9ac002d000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:35.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:35.595+0000 7f9abea00640 1 -- 172.21.15.67:0/1666036155 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ab005a010 con 0x7f9ac80cd780 2024-05-30T16:59:36.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.003+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f9ac80af350 con 0x7f9aa406dca0 2024-05-30T16:59:36.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.055+0000 7f9abea00640 1 -- 172.21.15.67:0/1666036155 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f9ac80af350 con 0x7f9aa406dca0 2024-05-30T16:59:36.058 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:36.058 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:36.058 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "in-progress", 2024-05-30T16:59:36.058 INFO:teuthology.orchestra.run.smithi067.stdout: "source": { 2024-05-30T16:59:36.058 INFO:teuthology.orchestra.run.smithi067.stdout: "volume": "cephfs", 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout: "subvolume": "sv1", 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot": "ss1" 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_report": { 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout: "percentage cloned": 49.219, 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout: "amount cloned": "1.4G/3.0G", 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout: "files cloned": "5/6" 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:36.059 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:36.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.060+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9aa406dca0 msgr2=0x7f9aa4070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:36.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.060+0000 7f9acdc00640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9aa406dca0 0x7f9aa4070160 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0035000 tx=0x7f9ac002d000 comp rx=0 tx=0).stop 2024-05-30T16:59:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.060+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac80cd780 msgr2=0x7f9ac80cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.060+0000 7f9acdc00640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac80cd780 0x7f9ac80cdc00 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0000c00 tx=0x7f9ab0007720 comp rx=0 tx=0).stop 2024-05-30T16:59:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.061+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 shutdown_connections 2024-05-30T16:59:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.061+0000 7f9acdc00640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9aa406dca0 0x7f9aa4070160 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.061+0000 7f9acdc00640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ac815bad0 0x7f9ac815bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.061+0000 7f9acdc00640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ac80cd780 0x7f9ac80cdc00 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.061+0000 7f9acdc00640 1 --2- 172.21.15.67:0/1666036155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9aac005c10 0x7f9ac814d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.061+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 >> 172.21.15.67:0/1666036155 conn(0x7f9ac806bd20 msgr2=0x7f9ac80b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:36.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.061+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 shutdown_connections 2024-05-30T16:59:36.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.061+0000 7f9acdc00640 1 -- 172.21.15.67:0/1666036155 wait complete. 2024-05-30T16:59:36.111 INFO:tasks.cephfs.test_volumes:percentage cloned = 49.219 2024-05-30T16:59:36.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:36.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.397+0000 7f6ffb000640 1 Processor -- start 2024-05-30T16:59:36.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.397+0000 7f6ffb000640 1 -- start start 2024-05-30T16:59:36.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.398+0000 7f6ffb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff415e9e0 0x7f6ff4162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:36.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.399+0000 7f6ffb000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6ff40d4e00 0x7f6ff40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:36.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.399+0000 7f6ffb000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ff40b3990 con 0x7f6ff40cf7a0 2024-05-30T16:59:36.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.399+0000 7f6ffb000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6ff4163310 con 0x7f6ff40d4e00 2024-05-30T16:59:36.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.399+0000 7f6ffb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff40b3c70 con 0x7f6ff415e9e0 2024-05-30T16:59:36.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.399+0000 7f6ff3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6ff40cf7a0 0x7f6ff40d46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43516/0 (socket says 172.21.15.67:43516) 2024-05-30T16:59:36.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.399+0000 7f6ff3400640 1 -- 172.21.15.67:0/3237044507 learned_addr learned my addr 172.21.15.67:0/3237044507 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:36.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.399+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 477646311 0 0) 0x7f6ff40b3990 con 0x7f6ff40cf7a0 2024-05-30T16:59:36.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.399+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fd8003520 con 0x7f6ff40cf7a0 2024-05-30T16:59:36.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4071552072 0 0) 0x7f6fd8003520 con 0x7f6ff40cf7a0 2024-05-30T16:59:36.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff415e9e0 msgr2=0x7f6ff4162dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:36.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 --2- 172.21.15.67:0/3237044507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff415e9e0 0x7f6ff4162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 >> v2:172.21.15.169:3300/0 conn(0x7f6ff40d4e00 msgr2=0x7f6ff40d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 --2- 172.21.15.67:0/3237044507 >> v2:172.21.15.169:3300/0 conn(0x7f6ff40d4e00 0x7f6ff40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff40b3f50 con 0x7f6ff40cf7a0 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f6fe0001670 con 0x7f6ff40cf7a0 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 >> v1:172.21.15.67:6789/0 conn(0x7f6ff40cf7a0 legacy=0x7f6ff40d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 --2- 172.21.15.67:0/3237044507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff415e9e0 0x7f6fd80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 --2- 172.21.15.67:0/3237044507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd8004110 0x7f6fd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 --2- 172.21.15.67:0/3237044507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fd8004e60 0x7f6fd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff4163310 con 0x7f6fd8004110 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff40b3c70 con 0x7f6fd8004e60 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd800bf00 con 0x7f6ff415e9e0 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.400+0000 7f6ff3400640 1 --2- 172.21.15.67:0/3237044507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd8004110 0x7f6fd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.401+0000 7f6ff3400640 1 -- 172.21.15.67:0/3237044507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff415e9e0 msgr2=0x7f6fd80039b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:59:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.401+0000 7f6ff3400640 1 --2- 172.21.15.67:0/3237044507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff415e9e0 0x7f6fd80039b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.401+0000 7f6ff3400640 1 -- 172.21.15.67:0/3237044507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fd8004e60 msgr2=0x7f6fd800b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-30T16:59:36.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.401+0000 7f6ff3400640 1 --2- 172.21.15.67:0/3237044507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fd8004e60 0x7f6fd800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.401+0000 7f6ff3400640 1 -- 172.21.15.67:0/3237044507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd8003520 con 0x7f6fd8004110 2024-05-30T16:59:36.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.402+0000 7f6ff3400640 1 --2- 172.21.15.67:0/3237044507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd8004110 0x7f6fd8004800 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0001050 tx=0x7f6fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587150038cb44cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:36.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.402+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0007ca0 con 0x7f6fd8004110 2024-05-30T16:59:36.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.402+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe0005ad0 con 0x7f6fd8004110 2024-05-30T16:59:36.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.402+0000 7f6ff2a00640 1 -- 172.21.15.67:0/3237044507 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0007070 con 0x7f6fd8004110 2024-05-30T16:59:36.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.403+0000 7f6ffb000640 1 -- 172.21.15.67:0/3237044507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd8004110 msgr2=0x7f6fd8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:36.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.403+0000 7f6ffb000640 1 --2- 172.21.15.67:0/3237044507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd8004110 0x7f6fd8004800 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0001050 tx=0x7f6fd8005830 comp rx=0 tx=0).stop 2024-05-30T16:59:36.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.403+0000 7f6ff9000640 1 -- 172.21.15.67:0/3237044507 reap_dead start 2024-05-30T16:59:36.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.403+0000 7f6ffb000640 1 -- 172.21.15.67:0/3237044507 shutdown_connections 2024-05-30T16:59:36.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.403+0000 7f6ffb000640 1 -- 172.21.15.67:0/3237044507 >> 172.21.15.67:0/3237044507 conn(0x7f6ff4067aa0 msgr2=0x7f6ff4067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.403+0000 7f6ffb000640 1 -- 172.21.15.67:0/3237044507 shutdown_connections 2024-05-30T16:59:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.403+0000 7f6ffb000640 1 -- 172.21.15.67:0/3237044507 wait complete. 2024-05-30T16:59:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.405+0000 7f6ffb000640 1 Processor -- start 2024-05-30T16:59:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.405+0000 7f6ffb000640 1 -- start start 2024-05-30T16:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ffb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fd8004110 0x7f6ff4155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ffb000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff41571d0 0x7f6ff40cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ffb000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff40cde80 0x7f6ff40b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ffb000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff40b7890 con 0x7f6ff40cde80 2024-05-30T16:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ffb000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff40d2eb0 con 0x7f6ff41571d0 2024-05-30T16:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ffb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff40b7a10 con 0x7f6fd8004110 2024-05-30T16:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ff9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff40cde80 0x7f6ff40b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ff9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff40cde80 0x7f6ff40b6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40234/0 (socket says 172.21.15.67:40234) 2024-05-30T16:59:36.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ff9000640 1 -- 172.21.15.67:0/868832216 learned_addr learned my addr 172.21.15.67:0/868832216 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.406+0000 7f6ff3400640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff41571d0 0x7f6ff40cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6ff9000640 1 -- 172.21.15.67:0/868832216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fd8004110 msgr2=0x7f6ff4155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6ff9000640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fd8004110 0x7f6ff4155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6ff9000640 1 -- 172.21.15.67:0/868832216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff41571d0 msgr2=0x7f6ff40cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6ff9000640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff41571d0 0x7f6ff40cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6ff9000640 1 -- 172.21.15.67:0/868832216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff40b4250 con 0x7f6ff40cde80 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6ff3400640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff41571d0 0x7f6ff40cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6ff9000640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff40cde80 0x7f6ff40b6480 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40d1130 tx=0x7f6fec00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6febe00640 1 -- 172.21.15.67:0/868832216 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec017070 con 0x7f6ff40cde80 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6febe00640 1 -- 172.21.15.67:0/868832216 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fec00aa40 con 0x7f6ff40cde80 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6febe00640 1 -- 172.21.15.67:0/868832216 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00ae30 con 0x7f6ff40cde80 2024-05-30T16:59:36.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff40b4530 con 0x7f6ff40cde80 2024-05-30T16:59:36.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.407+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff40b4af0 con 0x7f6ff40cde80 2024-05-30T16:59:36.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.409+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff40689e0 con 0x7f6ff40cde80 2024-05-30T16:59:36.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.410+0000 7f6febe00640 1 -- 172.21.15.67:0/868832216 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f6fec01a070 con 0x7f6ff40cde80 2024-05-30T16:59:36.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.411+0000 7f6febe00640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fd406dd20 0x7f6fd40701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:36.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.411+0000 7f6febe00640 1 -- 172.21.15.67:0/868832216 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f6fec08a450 con 0x7f6ff40cde80 2024-05-30T16:59:36.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.411+0000 7f6ff3e00640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fd406dd20 0x7f6fd40701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:36.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.412+0000 7f6ff3e00640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fd406dd20 0x7f6fd40701e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009690 tx=0x7f6fe4006c90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:36.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.416+0000 7f6febe00640 1 -- 172.21.15.67:0/868832216 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fec059f20 con 0x7f6ff40cde80 2024-05-30T16:59:36.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.794+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f6ff40d2400 con 0x7f6fd406dd20 2024-05-30T16:59:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.833+0000 7f6febe00640 1 -- 172.21.15.67:0/868832216 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f6ff40d2400 con 0x7f6fd406dd20 2024-05-30T16:59:36.834 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "in-progress", 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "source": { 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "volume": "cephfs", 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "subvolume": "sv1", 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot": "ss1" 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_report": { 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "percentage cloned": 66.667, 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "amount cloned": "2.0G/3.0G", 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: "files cloned": "5/6" 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:36.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:36.836 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fd406dd20 msgr2=0x7f6fd40701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fd406dd20 0x7f6fd40701e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009690 tx=0x7f6fe4006c90 comp rx=0 tx=0).stop 2024-05-30T16:59:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff40cde80 msgr2=0x7f6ff40b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff40cde80 0x7f6ff40b6480 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40d1130 tx=0x7f6fec00cf00 comp rx=0 tx=0).stop 2024-05-30T16:59:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 shutdown_connections 2024-05-30T16:59:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6fd406dd20 0x7f6fd40701e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff40cde80 0x7f6ff40b6480 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff41571d0 0x7f6ff40cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 --2- 172.21.15.67:0/868832216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fd8004110 0x7f6ff4155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:36.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 >> 172.21.15.67:0/868832216 conn(0x7f6ff4067aa0 msgr2=0x7f6ff40b8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:36.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 shutdown_connections 2024-05-30T16:59:36.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:36.836+0000 7f6ffb000640 1 -- 172.21.15.67:0/868832216 wait complete. 2024-05-30T16:59:36.879 INFO:tasks.cephfs.test_volumes:percentage cloned = 66.667 2024-05-30T16:59:36.880 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:37.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.174+0000 7f6662c00640 1 Processor -- start 2024-05-30T16:59:37.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.175+0000 7f6662c00640 1 -- start start 2024-05-30T16:59:37.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.177+0000 7f6662c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f665c072cd0 0x7f665c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:37.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.177+0000 7f6662c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665c06e2e0 con 0x7f665c15abe0 2024-05-30T16:59:37.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.177+0000 7f6662c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665c06e5c0 con 0x7f665c0b45d0 2024-05-30T16:59:37.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.177+0000 7f6662c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f665c06e8a0 con 0x7f665c072cd0 2024-05-30T16:59:37.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.177+0000 7f665be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f665c15abe0 0x7f665c06f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43518/0 (socket says 172.21.15.67:43518) 2024-05-30T16:59:37.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.177+0000 7f6660c00640 1 --2- 172.21.15.67:0/3144405505 >> v2:172.21.15.169:3300/0 conn(0x7f665c072cd0 0x7f665c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:37.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.177+0000 7f665be00640 1 -- 172.21.15.67:0/3144405505 learned_addr learned my addr 172.21.15.67:0/3144405505 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:37.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.178+0000 7f6660c00640 1 -- 172.21.15.67:0/3144405505 >> v1:172.21.15.169:6789/0 conn(0x7f665c0b45d0 legacy=0x7f665c0725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.178+0000 7f6660c00640 1 -- 172.21.15.67:0/3144405505 >> v1:172.21.15.67:6789/0 conn(0x7f665c15abe0 legacy=0x7f665c06f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.178+0000 7f665b400640 1 -- 172.21.15.67:0/3144405505 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 586141328 0 0) 0x7f665c06e5c0 con 0x7f665c0b45d0 2024-05-30T16:59:37.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.178+0000 7f6660c00640 1 -- 172.21.15.67:0/3144405505 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c06eb80 con 0x7f665c072cd0 2024-05-30T16:59:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.180+0000 7f6660c00640 1 --2- 172.21.15.67:0/3144405505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c072cd0 0x7f665c0b4a30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6654009c80 tx=0x7f665402b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e9e379b103e7d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:37.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.180+0000 7f665b400640 1 -- 172.21.15.67:0/3144405505 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654007740 con 0x7f665c072cd0 2024-05-30T16:59:37.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.180+0000 7f665b400640 1 -- 172.21.15.67:0/3144405505 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66540078e0 con 0x7f665c072cd0 2024-05-30T16:59:37.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.180+0000 7f665b400640 1 -- 172.21.15.67:0/3144405505 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654007cb0 con 0x7f665c072cd0 2024-05-30T16:59:37.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.182+0000 7f6662c00640 1 -- 172.21.15.67:0/3144405505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c072cd0 msgr2=0x7f665c0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.182+0000 7f6662c00640 1 --2- 172.21.15.67:0/3144405505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c072cd0 0x7f665c0b4a30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6654009c80 tx=0x7f665402b800 comp rx=0 tx=0).stop 2024-05-30T16:59:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.182+0000 7f6662c00640 1 -- 172.21.15.67:0/3144405505 shutdown_connections 2024-05-30T16:59:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.182+0000 7f6662c00640 1 --2- 172.21.15.67:0/3144405505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c072cd0 0x7f665c0b4a30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.182+0000 7f6662c00640 1 -- 172.21.15.67:0/3144405505 >> 172.21.15.67:0/3144405505 conn(0x7f665c06bd20 msgr2=0x7f665c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.182+0000 7f6662c00640 1 -- 172.21.15.67:0/3144405505 shutdown_connections 2024-05-30T16:59:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.183+0000 7f6662c00640 1 -- 172.21.15.67:0/3144405505 wait complete. 2024-05-30T16:59:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.186+0000 7f6662c00640 1 Processor -- start 2024-05-30T16:59:37.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.186+0000 7f6662c00640 1 -- start start 2024-05-30T16:59:37.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.188+0000 7f6662c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c072cd0 0x7f665c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:37.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.188+0000 7f6662c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f665c0b45d0 0x7f665c15d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:37.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.189+0000 7f665be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c072cd0 0x7f665c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:37.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.189+0000 7f6662c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c15abe0 0x7f665c08bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:37.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.189+0000 7f665be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c072cd0 0x7f665c150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40246/0 (socket says 172.21.15.67:40246) 2024-05-30T16:59:37.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.189+0000 7f665be00640 1 -- 172.21.15.67:0/1347923965 learned_addr learned my addr 172.21.15.67:0/1347923965 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:37.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.189+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f665c06fa90 con 0x7f665c072cd0 2024-05-30T16:59:37.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.189+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f665c15afc0 con 0x7f665c15abe0 2024-05-30T16:59:37.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.189+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f665c06df60 con 0x7f665c0b45d0 2024-05-30T16:59:37.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.189+0000 7f6653e00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f665c0b45d0 0x7f665c15d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:37.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.190+0000 7f6660c00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c15abe0 0x7f665c08bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:37.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.190+0000 7f665be00640 1 -- 172.21.15.67:0/1347923965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f665c0b45d0 msgr2=0x7f665c15d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.190+0000 7f665be00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f665c0b45d0 0x7f665c15d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.190+0000 7f665be00640 1 -- 172.21.15.67:0/1347923965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c15abe0 msgr2=0x7f665c08bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.190+0000 7f665be00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c15abe0 0x7f665c08bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.190+0000 7f665be00640 1 -- 172.21.15.67:0/1347923965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c1577c0 con 0x7f665c072cd0 2024-05-30T16:59:37.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.191+0000 7f665be00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c072cd0 0x7f665c150860 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c4b0 tx=0x7f664c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:37.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.191+0000 7f6658c00640 1 -- 172.21.15.67:0/1347923965 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c017070 con 0x7f665c072cd0 2024-05-30T16:59:37.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.192+0000 7f6658c00640 1 -- 172.21.15.67:0/1347923965 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664c005ce0 con 0x7f665c072cd0 2024-05-30T16:59:37.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.192+0000 7f6658c00640 1 -- 172.21.15.67:0/1347923965 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c007270 con 0x7f665c072cd0 2024-05-30T16:59:37.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.192+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c157aa0 con 0x7f665c072cd0 2024-05-30T16:59:37.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.192+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c150340 con 0x7f665c072cd0 2024-05-30T16:59:37.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.197+0000 7f6658c00640 1 -- 172.21.15.67:0/1347923965 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f664c01a070 con 0x7f665c072cd0 2024-05-30T16:59:37.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.198+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c0689e0 con 0x7f665c072cd0 2024-05-30T16:59:37.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.199+0000 7f6658c00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663806dcf0 0x7f66380701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:37.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.199+0000 7f6658c00640 1 -- 172.21.15.67:0/1347923965 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f664c089e40 con 0x7f665c072cd0 2024-05-30T16:59:37.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.200+0000 7f6653e00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663806dcf0 0x7f66380701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:37.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.202+0000 7f6653e00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663806dcf0 0x7f66380701b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f665c0684b0 tx=0x7f6644009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:37.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.209+0000 7f6658c00640 1 -- 172.21.15.67:0/1347923965 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f664c059300 con 0x7f665c072cd0 2024-05-30T16:59:37.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.598+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f665c06dd40 con 0x7f663806dcf0 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.631+0000 7f6658c00640 1 -- 172.21.15.67:0/1347923965 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f665c06dd40 con 0x7f663806dcf0 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "in-progress", 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stdout: "source": { 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stdout: "volume": "cephfs", 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stdout: "subvolume": "sv1", 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot": "ss1" 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:37.633 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_report": { 2024-05-30T16:59:37.634 INFO:teuthology.orchestra.run.smithi067.stdout: "percentage cloned": 82.552, 2024-05-30T16:59:37.634 INFO:teuthology.orchestra.run.smithi067.stdout: "amount cloned": "2.4G/3.0G", 2024-05-30T16:59:37.634 INFO:teuthology.orchestra.run.smithi067.stdout: "files cloned": "6/6" 2024-05-30T16:59:37.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:37.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:37.634 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663806dcf0 msgr2=0x7f66380701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663806dcf0 0x7f66380701b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f665c0684b0 tx=0x7f6644009040 comp rx=0 tx=0).stop 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c072cd0 msgr2=0x7f665c150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c072cd0 0x7f665c150860 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c4b0 tx=0x7f664c00c970 comp rx=0 tx=0).stop 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 shutdown_connections 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f663806dcf0 0x7f66380701b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c15abe0 0x7f665c08bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f665c0b45d0 0x7f665c15d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 --2- 172.21.15.67:0/1347923965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f665c072cd0 0x7f665c150860 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 >> 172.21.15.67:0/1347923965 conn(0x7f665c06bd20 msgr2=0x7f665c158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.635+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 shutdown_connections 2024-05-30T16:59:37.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.636+0000 7f6662c00640 1 -- 172.21.15.67:0/1347923965 wait complete. 2024-05-30T16:59:37.673 INFO:tasks.cephfs.test_volumes:percentage cloned = 82.552 2024-05-30T16:59:37.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:37.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.970+0000 7f3ce7200640 1 Processor -- start 2024-05-30T16:59:37.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.970+0000 7f3ce7200640 1 -- start start 2024-05-30T16:59:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.972+0000 7f3ce7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3ce015d5d0 0x7f3ce00cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.972+0000 7f3ce7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce015cb10 con 0x7f3ce00b32c0 2024-05-30T16:59:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.972+0000 7f3ce7200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce00ad5f0 con 0x7f3ce00d0640 2024-05-30T16:59:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.972+0000 7f3ce7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f3ce00adbb0 con 0x7f3ce015d5d0 2024-05-30T16:59:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.973+0000 7f3cdfe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f3ce00d0640 0x7f3ce00d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:35624/0 (socket says 172.21.15.67:35624) 2024-05-30T16:59:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.973+0000 7f3cdfe00640 1 -- 172.21.15.67:0/782608651 learned_addr learned my addr 172.21.15.67:0/782608651 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.974+0000 7f3cdf400640 1 --2- 172.21.15.67:0/782608651 >> v2:172.21.15.67:3300/0 conn(0x7f3ce015d5d0 0x7f3ce00cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.974+0000 7f3cdf400640 1 -- 172.21.15.67:0/782608651 >> v1:172.21.15.169:6789/0 conn(0x7f3ce00d0640 legacy=0x7f3ce00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.974+0000 7f3cdf400640 1 -- 172.21.15.67:0/782608651 >> v1:172.21.15.67:6789/0 conn(0x7f3ce00b32c0 legacy=0x7f3ce00b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.975+0000 7f3cdf400640 1 -- 172.21.15.67:0/782608651 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce006f9e0 con 0x7f3ce015d5d0 2024-05-30T16:59:37.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.977+0000 7f3cdf400640 1 --2- 172.21.15.67:0/782608651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 0x7f3ce00cf950 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0097a0 tx=0x7f3ccc029fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4faecbb97b01db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.978+0000 7f3cdea00640 1 -- 172.21.15.67:0/782608651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc037070 con 0x7f3ce015d5d0 2024-05-30T16:59:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.978+0000 7f3cdea00640 1 -- 172.21.15.67:0/782608651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ccc02ac40 con 0x7f3ce015d5d0 2024-05-30T16:59:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.978+0000 7f3cdea00640 1 -- 172.21.15.67:0/782608651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc032440 con 0x7f3ce015d5d0 2024-05-30T16:59:37.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.980+0000 7f3ce7200640 1 -- 172.21.15.67:0/782608651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 msgr2=0x7f3ce00cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.980+0000 7f3ce7200640 1 --2- 172.21.15.67:0/782608651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 0x7f3ce00cf950 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0097a0 tx=0x7f3ccc029fb0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.981+0000 7f3ce7200640 1 -- 172.21.15.67:0/782608651 shutdown_connections 2024-05-30T16:59:37.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.981+0000 7f3ce7200640 1 --2- 172.21.15.67:0/782608651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 0x7f3ce00cf950 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.981+0000 7f3ce7200640 1 -- 172.21.15.67:0/782608651 >> 172.21.15.67:0/782608651 conn(0x7f3ce006bd20 msgr2=0x7f3ce006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:37.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.981+0000 7f3ce7200640 1 -- 172.21.15.67:0/782608651 shutdown_connections 2024-05-30T16:59:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.982+0000 7f3ce7200640 1 -- 172.21.15.67:0/782608651 wait complete. 2024-05-30T16:59:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.986+0000 7f3ce7200640 1 Processor -- start 2024-05-30T16:59:37.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.986+0000 7f3ce7200640 1 -- start start 2024-05-30T16:59:37.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.988+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce00b32c0 0x7f3ce0140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:37.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.988+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce00d0640 0x7f3ce0152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:37.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.988+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 0x7f3ce0099090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:37.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.988+0000 7f3ce7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce015d0b0 con 0x7f3ce015d5d0 2024-05-30T16:59:37.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.988+0000 7f3ce7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce015cdd0 con 0x7f3ce00d0640 2024-05-30T16:59:37.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.988+0000 7f3ce7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce015da50 con 0x7f3ce00b32c0 2024-05-30T16:59:37.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.989+0000 7f3ce5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 0x7f3ce0099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:37.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.989+0000 7f3ce5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 0x7f3ce0099090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40252/0 (socket says 172.21.15.67:40252) 2024-05-30T16:59:37.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.989+0000 7f3ce5200640 1 -- 172.21.15.67:0/147330531 learned_addr learned my addr 172.21.15.67:0/147330531 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:37.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.990+0000 7f3cdfe00640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce00b32c0 0x7f3ce0140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:37.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.990+0000 7f3cdf400640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce00d0640 0x7f3ce0152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:37.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.991+0000 7f3ce5200640 1 -- 172.21.15.67:0/147330531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce00b32c0 msgr2=0x7f3ce0140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.991+0000 7f3ce5200640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce00b32c0 0x7f3ce0140c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.991+0000 7f3ce5200640 1 -- 172.21.15.67:0/147330531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce00d0640 msgr2=0x7f3ce0152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:37.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.991+0000 7f3ce5200640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce00d0640 0x7f3ce0152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:37.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.991+0000 7f3ce5200640 1 -- 172.21.15.67:0/147330531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce01532f0 con 0x7f3ce015d5d0 2024-05-30T16:59:37.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.991+0000 7f3ce5200640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 0x7f3ce0099090 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800c8c0 tx=0x7f3cd800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:37.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.991+0000 7f3cd7e00640 1 -- 172.21.15.67:0/147330531 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8017070 con 0x7f3ce015d5d0 2024-05-30T16:59:37.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.992+0000 7f3cd7e00640 1 -- 172.21.15.67:0/147330531 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd8003da0 con 0x7f3ce015d5d0 2024-05-30T16:59:37.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.992+0000 7f3cd7e00640 1 -- 172.21.15.67:0/147330531 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd80052b0 con 0x7f3ce015d5d0 2024-05-30T16:59:37.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.992+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce0147910 con 0x7f3ce015d5d0 2024-05-30T16:59:37.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.992+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce0147ed0 con 0x7f3ce015d5d0 2024-05-30T16:59:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.994+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb8005290 con 0x7f3ce015d5d0 2024-05-30T16:59:37.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.997+0000 7f3cd7e00640 1 -- 172.21.15.67:0/147330531 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f3cd801a070 con 0x7f3ce015d5d0 2024-05-30T16:59:38.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:37.999+0000 7f3cd7e00640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3cc006dca0 0x7f3cc0070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:38.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.000+0000 7f3cd7e00640 1 -- 172.21.15.67:0/147330531 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f3cd808a310 con 0x7f3ce015d5d0 2024-05-30T16:59:38.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.004+0000 7f3cdfe00640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3cc006dca0 0x7f3cc0070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:38.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.005+0000 7f3cdfe00640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3cc006dca0 0x7f3cc0070160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0000f60 tx=0x7f3cd0007ca0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:38.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.006+0000 7f3cd7e00640 1 -- 172.21.15.67:0/147330531 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3cd8059de0 con 0x7f3ce015d5d0 2024-05-30T16:59:38.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.459+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f3cb8000bc0 con 0x7f3cc006dca0 2024-05-30T16:59:38.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.498+0000 7f3cd7e00640 1 -- 172.21.15.67:0/147330531 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f3cb8000bc0 con 0x7f3cc006dca0 2024-05-30T16:59:38.499 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "in-progress", 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "source": { 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "volume": "cephfs", 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "subvolume": "sv1", 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot": "ss1" 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_report": { 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "percentage cloned": 82.552, 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "amount cloned": "2.4G/3.0G", 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: "files cloned": "6/6" 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:38.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:38.501 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:38.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3cc006dca0 msgr2=0x7f3cc0070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:38.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3cc006dca0 0x7f3cc0070160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0000f60 tx=0x7f3cd0007ca0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 msgr2=0x7f3ce0099090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 0x7f3ce0099090 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800c8c0 tx=0x7f3cd800cd80 comp rx=0 tx=0).stop 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 shutdown_connections 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3cc006dca0 0x7f3cc0070160 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ce015d5d0 0x7f3ce0099090 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ce00d0640 0x7f3ce0152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 --2- 172.21.15.67:0/147330531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ce00b32c0 0x7f3ce0140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 >> 172.21.15.67:0/147330531 conn(0x7f3ce006bd20 msgr2=0x7f3ce015e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 shutdown_connections 2024-05-30T16:59:38.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.501+0000 7f3ce7200640 1 -- 172.21.15.67:0/147330531 wait complete. 2024-05-30T16:59:38.534 INFO:tasks.cephfs.test_volumes:percentage cloned = 82.552 2024-05-30T16:59:38.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:38.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.734+0000 7eff23200640 1 Processor -- start 2024-05-30T16:59:38.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.734+0000 7eff23200640 1 -- start start 2024-05-30T16:59:38.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.736+0000 7eff23200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7eff24153510 0x7eff240b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:38.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.736+0000 7eff23200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff2415d270 0x7eff240ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:38.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.736+0000 7eff23200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2415d900 con 0x7eff240ccb90 2024-05-30T16:59:38.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.736+0000 7eff23200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7eff241538f0 con 0x7eff24153510 2024-05-30T16:59:38.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.736+0000 7eff23200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7eff240cd530 con 0x7eff2415d270 2024-05-30T16:59:38.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.736+0000 7eff21e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7eff24153510 0x7eff240b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:38.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.736+0000 7eff21e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7eff24153510 0x7eff240b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40256/0 (socket says 172.21.15.67:40256) 2024-05-30T16:59:38.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.736+0000 7eff21e00640 1 -- 172.21.15.67:0/2743348224 learned_addr learned my addr 172.21.15.67:0/2743348224 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:38.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.736+0000 7eff21400640 1 --1- 172.21.15.67:0/2743348224 >> v1:172.21.15.67:6789/0 conn(0x7eff240ccb90 0x7eff2415cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43528/0 (socket says 172.21.15.67:43528) 2024-05-30T16:59:38.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.737+0000 7eff21e00640 1 -- 172.21.15.67:0/2743348224 >> v2:172.21.15.169:3300/0 conn(0x7eff2415d270 msgr2=0x7eff240ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:38.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.737+0000 7eff21e00640 1 --2- 172.21.15.67:0/2743348224 >> v2:172.21.15.169:3300/0 conn(0x7eff2415d270 0x7eff240ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.737+0000 7eff21e00640 1 -- 172.21.15.67:0/2743348224 >> v1:172.21.15.67:6789/0 conn(0x7eff240ccb90 legacy=0x7eff2415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:38.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.737+0000 7eff21e00640 1 -- 172.21.15.67:0/2743348224 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff240b3f90 con 0x7eff24153510 2024-05-30T16:59:38.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.738+0000 7eff21e00640 1 --2- 172.21.15.67:0/2743348224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 0x7eff240b4d50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7eff240cdce0 tx=0x7eff1402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354ac4b32a452eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:38.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.738+0000 7eff20a00640 1 -- 172.21.15.67:0/2743348224 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14037070 con 0x7eff24153510 2024-05-30T16:59:38.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.738+0000 7eff20a00640 1 -- 172.21.15.67:0/2743348224 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff14032440 con 0x7eff24153510 2024-05-30T16:59:38.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.738+0000 7eff20a00640 1 -- 172.21.15.67:0/2743348224 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14032810 con 0x7eff24153510 2024-05-30T16:59:38.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.739+0000 7eff23200640 1 -- 172.21.15.67:0/2743348224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 msgr2=0x7eff240b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:38.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.739+0000 7eff23200640 1 --2- 172.21.15.67:0/2743348224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 0x7eff240b4d50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7eff240cdce0 tx=0x7eff1402a1f0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.739+0000 7eff23200640 1 -- 172.21.15.67:0/2743348224 shutdown_connections 2024-05-30T16:59:38.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.739+0000 7eff23200640 1 --2- 172.21.15.67:0/2743348224 >> v2:172.21.15.169:3300/0 conn(0x7eff2415d270 0x7eff240ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.739+0000 7eff23200640 1 --2- 172.21.15.67:0/2743348224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 0x7eff240b4d50 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.739+0000 7eff23200640 1 -- 172.21.15.67:0/2743348224 >> 172.21.15.67:0/2743348224 conn(0x7eff2406b900 msgr2=0x7eff2406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:38.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.739+0000 7eff23200640 1 -- 172.21.15.67:0/2743348224 shutdown_connections 2024-05-30T16:59:38.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.739+0000 7eff23200640 1 -- 172.21.15.67:0/2743348224 wait complete. 2024-05-30T16:59:38.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.741+0000 7eff23200640 1 Processor -- start 2024-05-30T16:59:38.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.741+0000 7eff23200640 1 -- start start 2024-05-30T16:59:38.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.742+0000 7eff23200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff240ccb90 0x7eff24072d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:38.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.742+0000 7eff23200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 0x7eff24099570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:38.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.742+0000 7eff23200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2415d270 0x7eff24099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:38.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.742+0000 7eff23200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff2415d6a0 con 0x7eff24153510 2024-05-30T16:59:38.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.742+0000 7eff23200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff240b48c0 con 0x7eff2415d270 2024-05-30T16:59:38.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.742+0000 7eff23200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff240b4550 con 0x7eff240ccb90 2024-05-30T16:59:38.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.742+0000 7eff21400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 0x7eff24099570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:38.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.742+0000 7eff21400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 0x7eff24099570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40272/0 (socket says 172.21.15.67:40272) 2024-05-30T16:59:38.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.742+0000 7eff21400640 1 -- 172.21.15.67:0/809673593 learned_addr learned my addr 172.21.15.67:0/809673593 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:38.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff21e00640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff240ccb90 0x7eff24072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:38.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff21400640 1 -- 172.21.15.67:0/809673593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff240ccb90 msgr2=0x7eff24072d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:38.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff21400640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff240ccb90 0x7eff24072d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff21400640 1 -- 172.21.15.67:0/809673593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2415d270 msgr2=0x7eff24099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:38.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff21400640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2415d270 0x7eff24099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:38.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff21400640 1 -- 172.21.15.67:0/809673593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff24156570 con 0x7eff24153510 2024-05-30T16:59:38.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff21400640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 0x7eff24099570 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c3e0 tx=0x7eff0c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:38.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff1a000640 1 -- 172.21.15.67:0/809673593 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0074e0 con 0x7eff24153510 2024-05-30T16:59:38.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff1a000640 1 -- 172.21.15.67:0/809673593 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c007680 con 0x7eff24153510 2024-05-30T16:59:38.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff1a000640 1 -- 172.21.15.67:0/809673593 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0049e0 con 0x7eff24153510 2024-05-30T16:59:38.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.743+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff24156850 con 0x7eff24153510 2024-05-30T16:59:38.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.744+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff24150650 con 0x7eff24153510 2024-05-30T16:59:38.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.745+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff24150850 con 0x7eff24153510 2024-05-30T16:59:38.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.746+0000 7eff1a000640 1 -- 172.21.15.67:0/809673593 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7eff0c01a020 con 0x7eff24153510 2024-05-30T16:59:38.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.747+0000 7eff1a000640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff0006dcd0 0x7eff00070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:38.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.747+0000 7eff1a000640 1 -- 172.21.15.67:0/809673593 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7eff0c08a340 con 0x7eff24153510 2024-05-30T16:59:38.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.747+0000 7eff21e00640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff0006dcd0 0x7eff00070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:38.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.748+0000 7eff21e00640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff0006dcd0 0x7eff00070190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7eff14034000 tx=0x7eff1402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:38.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.751+0000 7eff1a000640 1 -- 172.21.15.67:0/809673593 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff24150850 con 0x7eff24153510 2024-05-30T16:59:38.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:38.990+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7eff240689e0 con 0x7eff0006dcd0 2024-05-30T16:59:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.750+0000 7eff1a000640 1 -- 172.21.15.67:0/809673593 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7eff240689e0 con 0x7eff0006dcd0 2024-05-30T16:59:39.752 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:39.752 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:39.752 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "complete" 2024-05-30T16:59:39.752 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:39.752 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff0006dcd0 msgr2=0x7eff00070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff0006dcd0 0x7eff00070190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7eff14034000 tx=0x7eff1402e000 comp rx=0 tx=0).stop 2024-05-30T16:59:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 msgr2=0x7eff24099570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 0x7eff24099570 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c3e0 tx=0x7eff0c00c8a0 comp rx=0 tx=0).stop 2024-05-30T16:59:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 shutdown_connections 2024-05-30T16:59:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff0006dcd0 0x7eff00070190 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2415d270 0x7eff24099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff24153510 0x7eff24099570 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 --2- 172.21.15.67:0/809673593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff240ccb90 0x7eff24072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 >> 172.21.15.67:0/809673593 conn(0x7eff2406b900 msgr2=0x7eff2406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 shutdown_connections 2024-05-30T16:59:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.753+0000 7eff23200640 1 -- 172.21.15.67:0/809673593 wait complete. 2024-05-30T16:59:39.776 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-05-30T16:59:39.776 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-30T16:59:39.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.989+0000 7f5297200640 1 Processor -- start 2024-05-30T16:59:39.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.989+0000 7f5297200640 1 -- start start 2024-05-30T16:59:39.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.991+0000 7f5297200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529815e870 0x7f52980d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:39.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.991+0000 7f5297200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52980d14d0 0x7f52980cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:39.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.991+0000 7f5297200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52980cfd00 con 0x7f52980cf870 2024-05-30T16:59:39.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.991+0000 7f5297200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f52980d2150 con 0x7f52980d14d0 2024-05-30T16:59:39.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.991+0000 7f5297200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f529815ec50 con 0x7f529815e870 2024-05-30T16:59:39.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.991+0000 7f5296800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f52980cf870 0x7f5298162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43536/0 (socket says 172.21.15.67:43536) 2024-05-30T16:59:39.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.991+0000 7f5296800640 1 -- 172.21.15.67:0/275246566 learned_addr learned my addr 172.21.15.67:0/275246566 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.991+0000 7f5295e00640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529815e870 0x7f52980d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5295400640 1 --2- 172.21.15.67:0/275246566 >> v2:172.21.15.169:3300/0 conn(0x7f52980d14d0 0x7f52980cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4077555075 0 0) 0x7f5290001490 con 0x7f52980cf870 2024-05-30T16:59:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f527c003520 con 0x7f52980cf870 2024-05-30T16:59:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2312966867 0 0) 0x7f527c003520 con 0x7f52980cf870 2024-05-30T16:59:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529815e870 msgr2=0x7f52980d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529815e870 0x7f52980d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 >> v2:172.21.15.169:3300/0 conn(0x7f52980d14d0 msgr2=0x7f52980cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 --2- 172.21.15.67:0/275246566 >> v2:172.21.15.169:3300/0 conn(0x7f52980d14d0 0x7f52980cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52980cff80 con 0x7f52980cf870 2024-05-30T16:59:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5290001bc0 con 0x7f52980cf870 2024-05-30T16:59:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 >> v1:172.21.15.67:6789/0 conn(0x7f52980cf870 legacy=0x7f5298162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c0039b0 0x7f527c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529815e870 0x7f527c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f527c004d40 0x7f527c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f52980d2150 con 0x7f527c0039b0 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f529815ec50 con 0x7f527c004d40 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f527c00b9c0 con 0x7f529815e870 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.992+0000 7f5295e00640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c0039b0 0x7f527c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.993+0000 7f5295400640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529815e870 0x7f527c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.993+0000 7f5295e00640 1 -- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529815e870 msgr2=0x7f527c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.993+0000 7f5295e00640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529815e870 0x7f527c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.993+0000 7f5295e00640 1 -- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f527c004d40 msgr2=0x7f527c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.993+0000 7f5295e00640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f527c004d40 0x7f527c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.993+0000 7f5295e00640 1 -- 172.21.15.67:0/275246566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c003520 con 0x7f527c0039b0 2024-05-30T16:59:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.993+0000 7f5295400640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529815e870 0x7f527c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:39.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.994+0000 7f5295e00640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c0039b0 0x7f527c003dd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5288009ac0 tx=0x7f527c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984a31487cacc036 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:39.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.994+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288036070 con 0x7f527c0039b0 2024-05-30T16:59:39.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.994+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5288031400 con 0x7f527c0039b0 2024-05-30T16:59:39.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.994+0000 7f5294a00640 1 -- 172.21.15.67:0/275246566 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288009690 con 0x7f527c0039b0 2024-05-30T16:59:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.995+0000 7f5297200640 1 -- 172.21.15.67:0/275246566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c0039b0 msgr2=0x7f527c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.995+0000 7f5297200640 1 --2- 172.21.15.67:0/275246566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c0039b0 0x7f527c003dd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5288009ac0 tx=0x7f527c005830 comp rx=0 tx=0).stop 2024-05-30T16:59:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.995+0000 7f5296800640 1 -- 172.21.15.67:0/275246566 reap_dead start 2024-05-30T16:59:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.995+0000 7f5297200640 1 -- 172.21.15.67:0/275246566 shutdown_connections 2024-05-30T16:59:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.995+0000 7f5297200640 1 -- 172.21.15.67:0/275246566 >> 172.21.15.67:0/275246566 conn(0x7f529808bfc0 msgr2=0x7f52980d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.995+0000 7f5297200640 1 -- 172.21.15.67:0/275246566 shutdown_connections 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.995+0000 7f5297200640 1 -- 172.21.15.67:0/275246566 wait complete. 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.997+0000 7f5297200640 1 Processor -- start 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.997+0000 7f5297200640 1 -- start start 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.998+0000 7f5297200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f527c0039b0 0x7f5298159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.998+0000 7f5297200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52981ea0f0 0x7f52981ea570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.998+0000 7f5297200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52980ccce0 0x7f52980cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.998+0000 7f5297200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f52980d0820 con 0x7f52980ccce0 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.998+0000 7f5297200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f52980d0540 con 0x7f527c0039b0 2024-05-30T16:59:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.998+0000 7f5297200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f52980d4fb0 con 0x7f52981ea0f0 2024-05-30T16:59:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.998+0000 7f5296800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52980ccce0 0x7f52980cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.998+0000 7f5296800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52980ccce0 0x7f52980cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40282/0 (socket says 172.21.15.67:40282) 2024-05-30T16:59:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.998+0000 7f5296800640 1 -- 172.21.15.67:0/1351740581 learned_addr learned my addr 172.21.15.67:0/1351740581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5295400640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52981ea0f0 0x7f52981ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5296800640 1 -- 172.21.15.67:0/1351740581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52981ea0f0 msgr2=0x7f52981ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5296800640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52981ea0f0 0x7f52981ea570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5296800640 1 -- 172.21.15.67:0/1351740581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f527c0039b0 msgr2=0x7f5298159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5295e00640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f527c0039b0 0x7f5298159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5296800640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f527c0039b0 0x7f5298159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5296800640 1 -- 172.21.15.67:0/1351740581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529815a340 con 0x7f52980ccce0 2024-05-30T16:59:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5296800640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52980ccce0 0x7f52980cd160 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f52980d5680 tx=0x7f529000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f528e000640 1 -- 172.21.15.67:0/1351740581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529000ce70 con 0x7f52980ccce0 2024-05-30T16:59:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f528e000640 1 -- 172.21.15.67:0/1351740581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f529000a610 con 0x7f52980ccce0 2024-05-30T16:59:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f528e000640 1 -- 172.21.15.67:0/1351740581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529000a9c0 con 0x7f52980ccce0 2024-05-30T16:59:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52980cb650 con 0x7f52980ccce0 2024-05-30T16:59:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:39.999+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52980cbbd0 con 0x7f52980ccce0 2024-05-30T16:59:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.001+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52980689e0 con 0x7f52980ccce0 2024-05-30T16:59:40.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.002+0000 7f528e000640 1 -- 172.21.15.67:0/1351740581 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f529000ab60 con 0x7f52980ccce0 2024-05-30T16:59:40.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.003+0000 7f528e000640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f527806dca0 0x7f5278070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.003+0000 7f528e000640 1 -- 172.21.15.67:0/1351740581 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f529008a150 con 0x7f52980ccce0 2024-05-30T16:59:40.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.003+0000 7f5295e00640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f527806dca0 0x7f5278070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.003+0000 7f5295e00640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f527806dca0 0x7f5278070160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5288033000 tx=0x7f5288008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:40.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.007+0000 7f528e000640 1 -- 172.21.15.67:0/1351740581 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f529001a030 con 0x7f52980ccce0 2024-05-30T16:59:40.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.247+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f52980ba500 con 0x7f527806dca0 2024-05-30T16:59:40.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.261+0000 7f528e000640 1 -- 172.21.15.67:0/1351740581 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f52980ba500 con 0x7f527806dca0 2024-05-30T16:59:40.263 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "status": { 2024-05-30T16:59:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "complete" 2024-05-30T16:59:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:40.264 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:40.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f527806dca0 msgr2=0x7f5278070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f527806dca0 0x7f5278070160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5288033000 tx=0x7f5288008030 comp rx=0 tx=0).stop 2024-05-30T16:59:40.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52980ccce0 msgr2=0x7f52980cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52980ccce0 0x7f52980cd160 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f52980d5680 tx=0x7f529000cc20 comp rx=0 tx=0).stop 2024-05-30T16:59:40.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 shutdown_connections 2024-05-30T16:59:40.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f527806dca0 0x7f5278070160 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52980ccce0 0x7f52980cd160 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52981ea0f0 0x7f52981ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 --2- 172.21.15.67:0/1351740581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f527c0039b0 0x7f5298159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.264+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 >> 172.21.15.67:0/1351740581 conn(0x7f529808bfc0 msgr2=0x7f52980d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:40.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.265+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 shutdown_connections 2024-05-30T16:59:40.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.265+0000 7f5297200640 1 -- 172.21.15.67:0/1351740581 wait complete. 2024-05-30T16:59:40.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T16:59:40.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.490+0000 7f9134c00640 1 Processor -- start 2024-05-30T16:59:40.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.490+0000 7f9134c00640 1 -- start start 2024-05-30T16:59:40.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.491+0000 7f9134c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9130073310 0x7f91300b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.491+0000 7f9134c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913006dc20 0x7f913015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.491+0000 7f9134c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91300afd60 con 0x7f913015b3a0 2024-05-30T16:59:40.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.491+0000 7f9134c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f913006f980 con 0x7f9130073310 2024-05-30T16:59:40.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.491+0000 7f9134c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f91300ae0c0 con 0x7f913006dc20 2024-05-30T16:59:40.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.492+0000 7f912fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913006dc20 0x7f913015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.492+0000 7f912ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9130073310 0x7f91300b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.492+0000 7f912fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913006dc20 0x7f913015f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:34800/0 (socket says 172.21.15.67:34800) 2024-05-30T16:59:40.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.492+0000 7f912fe00640 1 -- 172.21.15.67:0/1921168160 learned_addr learned my addr 172.21.15.67:0/1921168160 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:40.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.492+0000 7f912f400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f913015b3a0 0x7f9130072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:35626/0 (socket says 172.21.15.67:35626) 2024-05-30T16:59:40.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.493+0000 7f912fe00640 1 -- 172.21.15.67:0/1921168160 >> v2:172.21.15.169:3300/0 conn(0x7f9130073310 msgr2=0x7f91300b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:40.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.493+0000 7f912fe00640 1 --2- 172.21.15.67:0/1921168160 >> v2:172.21.15.169:3300/0 conn(0x7f9130073310 0x7f91300b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.493+0000 7f912fe00640 1 -- 172.21.15.67:0/1921168160 >> v1:172.21.15.169:6789/0 conn(0x7f913015b3a0 legacy=0x7f9130072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.493+0000 7f912fe00640 1 -- 172.21.15.67:0/1921168160 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91300af7a0 con 0x7f913006dc20 2024-05-30T16:59:40.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.493+0000 7f912ea00640 1 --2- 172.21.15.67:0/1921168160 >> v2:172.21.15.169:3300/0 conn(0x7f9130073310 0x7f91300b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T16:59:40.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.495+0000 7f912fe00640 1 --2- 172.21.15.67:0/1921168160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913006dc20 0x7f913015f9a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9128009e50 tx=0x7f912802c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1800095af4f5e90 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:40.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.496+0000 7f912e000640 1 -- 172.21.15.67:0/1921168160 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128039070 con 0x7f913006dc20 2024-05-30T16:59:40.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.496+0000 7f912e000640 1 -- 172.21.15.67:0/1921168160 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9128034680 con 0x7f913006dc20 2024-05-30T16:59:40.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.496+0000 7f912e000640 1 -- 172.21.15.67:0/1921168160 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128034a50 con 0x7f913006dc20 2024-05-30T16:59:40.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.496+0000 7f9134c00640 1 -- 172.21.15.67:0/1921168160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913006dc20 msgr2=0x7f913015f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.496+0000 7f9134c00640 1 --2- 172.21.15.67:0/1921168160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913006dc20 0x7f913015f9a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9128009e50 tx=0x7f912802c890 comp rx=0 tx=0).stop 2024-05-30T16:59:40.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.496+0000 7f9134c00640 1 -- 172.21.15.67:0/1921168160 shutdown_connections 2024-05-30T16:59:40.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.496+0000 7f9134c00640 1 --2- 172.21.15.67:0/1921168160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913006dc20 0x7f913015f9a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.496+0000 7f9134c00640 1 --2- 172.21.15.67:0/1921168160 >> v2:172.21.15.169:3300/0 conn(0x7f9130073310 0x7f91300b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.496+0000 7f9134c00640 1 -- 172.21.15.67:0/1921168160 >> 172.21.15.67:0/1921168160 conn(0x7f91300b1fc0 msgr2=0x7f91300b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:40.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.497+0000 7f9134c00640 1 -- 172.21.15.67:0/1921168160 shutdown_connections 2024-05-30T16:59:40.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.497+0000 7f9134c00640 1 -- 172.21.15.67:0/1921168160 wait complete. 2024-05-30T16:59:40.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.499+0000 7f9134c00640 1 Processor -- start 2024-05-30T16:59:40.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.499+0000 7f9134c00640 1 -- start start 2024-05-30T16:59:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f9134c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f913006dc20 0x7f91300ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f9134c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9130073310 0x7f91300cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f9134c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913015b3a0 0x7f9130149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f9134c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f913015b780 con 0x7f913006dc20 2024-05-30T16:59:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f9134c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f913015fee0 con 0x7f9130073310 2024-05-30T16:59:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f9134c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f91300ae550 con 0x7f913015b3a0 2024-05-30T16:59:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f913006dc20 0x7f91300ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f913006dc20 0x7f91300ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40296/0 (socket says 172.21.15.67:40296) 2024-05-30T16:59:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912f400640 1 -- 172.21.15.67:0/2707597170 learned_addr learned my addr 172.21.15.67:0/2707597170 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:40.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912fe00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913015b3a0 0x7f9130149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912ea00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9130073310 0x7f91300cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912f400640 1 -- 172.21.15.67:0/2707597170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913015b3a0 msgr2=0x7f9130149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912f400640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913015b3a0 0x7f9130149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912f400640 1 -- 172.21.15.67:0/2707597170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9130073310 msgr2=0x7f91300cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912f400640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9130073310 0x7f91300cf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912f400640 1 -- 172.21.15.67:0/2707597170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91300cd1e0 con 0x7f913006dc20 2024-05-30T16:59:40.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912fe00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913015b3a0 0x7f9130149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:40.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f912f400640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f913006dc20 0x7f91300ccca0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f912000c3c0 tx=0x7f912000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:40.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.502+0000 7f9127400640 1 -- 172.21.15.67:0/2707597170 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91200074e0 con 0x7f913006dc20 2024-05-30T16:59:40.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.503+0000 7f9127400640 1 -- 172.21.15.67:0/2707597170 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9120007680 con 0x7f913006dc20 2024-05-30T16:59:40.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.503+0000 7f9127400640 1 -- 172.21.15.67:0/2707597170 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120004a00 con 0x7f913006dc20 2024-05-30T16:59:40.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.503+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91300cd4b0 con 0x7f913006dc20 2024-05-30T16:59:40.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.503+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91300cdd50 con 0x7f913006dc20 2024-05-30T16:59:40.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.504+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91300cd8a0 con 0x7f913006dc20 2024-05-30T16:59:40.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.505+0000 7f9127400640 1 -- 172.21.15.67:0/2707597170 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f912001a020 con 0x7f913006dc20 2024-05-30T16:59:40.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.506+0000 7f9127400640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f911006dca0 0x7f9110070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.506+0000 7f9127400640 1 -- 172.21.15.67:0/2707597170 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f912008a370 con 0x7f913006dc20 2024-05-30T16:59:40.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.506+0000 7f912ea00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f911006dca0 0x7f9110070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.507+0000 7f912ea00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f911006dca0 0x7f9110070160 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f911c009560 tx=0x7f911c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:40.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.511+0000 7f9127400640 1 -- 172.21.15.67:0/2707597170 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9120059e40 con 0x7f913006dc20 2024-05-30T16:59:40.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.741+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f913006fe90 con 0x7f911006dca0 2024-05-30T16:59:40.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.744+0000 7f9127400640 1 -- 172.21.15.67:0/2707597170 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f913006fe90 con 0x7f911006dca0 2024-05-30T16:59:40.746 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T16:59:40.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:59:40.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T16:59:40.746 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:40.747 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f911006dca0 msgr2=0x7f9110070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f911006dca0 0x7f9110070160 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f911c009560 tx=0x7f911c008040 comp rx=0 tx=0).stop 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f913006dc20 msgr2=0x7f91300ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f913006dc20 0x7f91300ccca0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f912000c3c0 tx=0x7f912000c880 comp rx=0 tx=0).stop 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 shutdown_connections 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f911006dca0 0x7f9110070160 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f913015b3a0 0x7f9130149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9130073310 0x7f91300cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 --2- 172.21.15.67:0/2707597170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f913006dc20 0x7f91300ccca0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 >> 172.21.15.67:0/2707597170 conn(0x7f91300b1fc0 msgr2=0x7f91300b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:40.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 shutdown_connections 2024-05-30T16:59:40.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.748+0000 7f9134c00640 1 -- 172.21.15.67:0/2707597170 wait complete. 2024-05-30T16:59:40.768 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:59:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.972+0000 7f687f000640 1 Processor -- start 2024-05-30T16:59:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.972+0000 7f687f000640 1 -- start start 2024-05-30T16:59:40.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f687815c6b0 0x7f68780b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f68780d0460 0x7f68780d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68780cd1a0 con 0x7f68780ccc30 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f687806e3a0 con 0x7f68780d0460 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f68780d0df0 con 0x7f687815c6b0 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f68780d0460 0x7f68780d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f68780d0460 0x7f68780d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55094/0 (socket says 172.21.15.67:55094) 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687e600640 1 -- 172.21.15.67:0/1811006820 learned_addr learned my addr 172.21.15.67:0/1811006820 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687e600640 1 -- 172.21.15.67:0/1811006820 >> v2:172.21.15.169:3300/0 conn(0x7f687815c6b0 msgr2=0x7f68780b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687e600640 1 --2- 172.21.15.67:0/1811006820 >> v2:172.21.15.169:3300/0 conn(0x7f687815c6b0 0x7f68780b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687e600640 1 -- 172.21.15.67:0/1811006820 >> v1:172.21.15.169:6789/0 conn(0x7f68780ccc30 legacy=0x7f687806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:40.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.974+0000 7f687e600640 1 -- 172.21.15.67:0/1811006820 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68780cd480 con 0x7f68780d0460 2024-05-30T16:59:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.975+0000 7f687e600640 1 --2- 172.21.15.67:0/1811006820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780d0460 0x7f68780d08b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6870009870 tx=0x7f687002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f352c33a2fd6e87d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.976+0000 7f6877e00640 1 -- 172.21.15.67:0/1811006820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870037070 con 0x7f68780d0460 2024-05-30T16:59:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.976+0000 7f6877e00640 1 -- 172.21.15.67:0/1811006820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6870032440 con 0x7f68780d0460 2024-05-30T16:59:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.976+0000 7f6877e00640 1 -- 172.21.15.67:0/1811006820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870032810 con 0x7f68780d0460 2024-05-30T16:59:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.976+0000 7f687f000640 1 -- 172.21.15.67:0/1811006820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780d0460 msgr2=0x7f68780d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.976+0000 7f687f000640 1 --2- 172.21.15.67:0/1811006820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780d0460 0x7f68780d08b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6870009870 tx=0x7f687002a250 comp rx=0 tx=0).stop 2024-05-30T16:59:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.977+0000 7f687f000640 1 -- 172.21.15.67:0/1811006820 shutdown_connections 2024-05-30T16:59:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.977+0000 7f687f000640 1 --2- 172.21.15.67:0/1811006820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780d0460 0x7f68780d08b0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.977+0000 7f687f000640 1 --2- 172.21.15.67:0/1811006820 >> v2:172.21.15.169:3300/0 conn(0x7f687815c6b0 0x7f68780b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.977+0000 7f687f000640 1 -- 172.21.15.67:0/1811006820 >> 172.21.15.67:0/1811006820 conn(0x7f687806bae0 msgr2=0x7f687806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.977+0000 7f687f000640 1 -- 172.21.15.67:0/1811006820 shutdown_connections 2024-05-30T16:59:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.977+0000 7f687f000640 1 -- 172.21.15.67:0/1811006820 wait complete. 2024-05-30T16:59:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.978+0000 7f687f000640 1 Processor -- start 2024-05-30T16:59:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.978+0000 7f687f000640 1 -- start start 2024-05-30T16:59:40.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.979+0000 7f687f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780ccc30 0x7f68780c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.979+0000 7f687f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68780d0460 0x7f68780b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.979+0000 7f687f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687815c6b0 0x7f68781599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.979+0000 7f687f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68780cba10 con 0x7f68780ccc30 2024-05-30T16:59:40.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.979+0000 7f687f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f68780cda40 con 0x7f68780d0460 2024-05-30T16:59:40.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.979+0000 7f687f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68780b42f0 con 0x7f687815c6b0 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780ccc30 0x7f68780c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780ccc30 0x7f68780c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55108/0 (socket says 172.21.15.67:55108) 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687dc00640 1 -- 172.21.15.67:0/4226027115 learned_addr learned my addr 172.21.15.67:0/4226027115 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687e600640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687815c6b0 0x7f68781599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687dc00640 1 -- 172.21.15.67:0/4226027115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687815c6b0 msgr2=0x7f68781599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687d200640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68780d0460 0x7f68780b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687dc00640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687815c6b0 0x7f68781599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687dc00640 1 -- 172.21.15.67:0/4226027115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68780d0460 msgr2=0x7f68780b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687dc00640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68780d0460 0x7f68780b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687dc00640 1 -- 172.21.15.67:0/4226027115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68780c87c0 con 0x7f68780ccc30 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.980+0000 7f687dc00640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780ccc30 0x7f68780c80e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f686800ba50 tx=0x7f686800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.981+0000 7f6875600640 1 -- 172.21.15.67:0/4226027115 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868019070 con 0x7f68780ccc30 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.981+0000 7f6875600640 1 -- 172.21.15.67:0/4226027115 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686800cba0 con 0x7f68780ccc30 2024-05-30T16:59:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.981+0000 7f6875600640 1 -- 172.21.15.67:0/4226027115 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868014440 con 0x7f68780ccc30 2024-05-30T16:59:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.981+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68780c8aa0 con 0x7f68780ccc30 2024-05-30T16:59:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.981+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68780c9060 con 0x7f68780ccc30 2024-05-30T16:59:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.982+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68780689e0 con 0x7f68780ccc30 2024-05-30T16:59:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.983+0000 7f6875600640 1 -- 172.21.15.67:0/4226027115 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f686801c070 con 0x7f68780ccc30 2024-05-30T16:59:40.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.984+0000 7f6875600640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f685806dca0 0x7f6858070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:40.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.985+0000 7f6875600640 1 -- 172.21.15.67:0/4226027115 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f686808a960 con 0x7f68780ccc30 2024-05-30T16:59:40.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.985+0000 7f687d200640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f685806dca0 0x7f6858070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.985+0000 7f687d200640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f685806dca0 0x7f6858070160 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f68780cf530 tx=0x7f6864009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:40.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:40.989+0000 7f6875600640 1 -- 172.21.15.67:0/4226027115 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f686805a430 con 0x7f68780ccc30 2024-05-30T16:59:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.272+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f687806fd40 con 0x7f68780ccc30 2024-05-30T16:59:41.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.273+0000 7f6875600640 1 -- 172.21.15.67:0/4226027115 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2122 (secure 0 0 0) 0x7f6868059d30 con 0x7f68780ccc30 2024-05-30T16:59:41.274 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:59:41.274 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:41.274 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:59:41.275 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 974, 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 91, 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:59:41.276 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1394, 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 5726255067, 2024-05-30T16:59:41.277 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 12060483584, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 761033629696, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 205450415, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 220496224, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 49, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 58 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 56, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:59:41.278 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 6, 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7398 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:59:41.279 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:59:40.449642+0000", 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:59:41.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:41.281 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:59:41.281 INFO:teuthology.orchestra.run.smithi067.stdout: "c94bce46-24ca-440d-8c9f-e3b857e30cd5": { 2024-05-30T16:59:41.281 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 29.0 seconds before exiting (8s)\n [============================] ", 2024-05-30T16:59:41.281 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:59:41.281 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:59:41.281 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:41.281 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:41.281 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f685806dca0 msgr2=0x7f6858070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f685806dca0 0x7f6858070160 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f68780cf530 tx=0x7f6864009040 comp rx=0 tx=0).stop 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780ccc30 msgr2=0x7f68780c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780ccc30 0x7f68780c80e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f686800ba50 tx=0x7f686800bf10 comp rx=0 tx=0).stop 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 shutdown_connections 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f685806dca0 0x7f6858070160 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687815c6b0 0x7f68781599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68780d0460 0x7f68780b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 --2- 172.21.15.67:0/4226027115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780ccc30 0x7f68780c80e0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:41.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 >> 172.21.15.67:0/4226027115 conn(0x7f687806bae0 msgr2=0x7f687815b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:41.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 shutdown_connections 2024-05-30T16:59:41.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:41.276+0000 7f687f000640 1 -- 172.21.15.67:0/4226027115 wait complete. 2024-05-30T16:59:44.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:59:44.620 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T16:59:46.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:59:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.517+0000 7fe36e600640 1 Processor -- start 2024-05-30T16:59:46.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.517+0000 7fe36e600640 1 -- start start 2024-05-30T16:59:46.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.518+0000 7fe36e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe3680ccc30 0x7fe36806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:46.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.518+0000 7fe36e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680d0460 0x7fe3680d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:46.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.518+0000 7fe36e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3680cd1a0 con 0x7fe36815c6b0 2024-05-30T16:59:46.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.518+0000 7fe36e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe3680cd480 con 0x7fe3680ccc30 2024-05-30T16:59:46.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.518+0000 7fe36e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3680cd760 con 0x7fe3680d0460 2024-05-30T16:59:46.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.518+0000 7fe36d200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe36815c6b0 0x7fe3680b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42324/0 (socket says 172.21.15.67:42324) 2024-05-30T16:59:46.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.519+0000 7fe36d200640 1 -- 172.21.15.67:0/3511204721 learned_addr learned my addr 172.21.15.67:0/3511204721 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:46.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.519+0000 7fe367e00640 1 --2- 172.21.15.67:0/3511204721 >> v2:172.21.15.67:3300/0 conn(0x7fe3680ccc30 0x7fe36806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.519+0000 7fe36dc00640 1 --2- 172.21.15.67:0/3511204721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680d0460 0x7fe3680d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.519+0000 7fe367400640 1 -- 172.21.15.67:0/3511204721 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1140554076 0 0) 0x7fe3680cd1a0 con 0x7fe36815c6b0 2024-05-30T16:59:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.519+0000 7fe367400640 1 -- 172.21.15.67:0/3511204721 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe34c003520 con 0x7fe36815c6b0 2024-05-30T16:59:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.519+0000 7fe367e00640 1 -- 172.21.15.67:0/3511204721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680d0460 msgr2=0x7fe3680d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.519+0000 7fe367e00640 1 --2- 172.21.15.67:0/3511204721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680d0460 0x7fe3680d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.519+0000 7fe367e00640 1 -- 172.21.15.67:0/3511204721 >> v1:172.21.15.67:6789/0 conn(0x7fe36815c6b0 legacy=0x7fe3680b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:46.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.519+0000 7fe367e00640 1 -- 172.21.15.67:0/3511204721 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3680cda40 con 0x7fe3680ccc30 2024-05-30T16:59:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.520+0000 7fe367e00640 1 --2- 172.21.15.67:0/3511204721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680ccc30 0x7fe36806dc90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe358009dc0 tx=0x7fe35802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44a627e0deaaeba server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.521+0000 7fe367400640 1 -- 172.21.15.67:0/3511204721 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35802ae60 con 0x7fe3680ccc30 2024-05-30T16:59:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.521+0000 7fe367400640 1 -- 172.21.15.67:0/3511204721 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe358007740 con 0x7fe3680ccc30 2024-05-30T16:59:46.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.521+0000 7fe367400640 1 -- 172.21.15.67:0/3511204721 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358007af0 con 0x7fe3680ccc30 2024-05-30T16:59:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.521+0000 7fe36e600640 1 -- 172.21.15.67:0/3511204721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680ccc30 msgr2=0x7fe36806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.521+0000 7fe36e600640 1 --2- 172.21.15.67:0/3511204721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680ccc30 0x7fe36806dc90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe358009dc0 tx=0x7fe35802a680 comp rx=0 tx=0).stop 2024-05-30T16:59:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.521+0000 7fe36e600640 1 -- 172.21.15.67:0/3511204721 shutdown_connections 2024-05-30T16:59:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.521+0000 7fe36e600640 1 --2- 172.21.15.67:0/3511204721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680d0460 0x7fe3680d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.521+0000 7fe36e600640 1 --2- 172.21.15.67:0/3511204721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680ccc30 0x7fe36806dc90 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.521+0000 7fe36e600640 1 -- 172.21.15.67:0/3511204721 >> 172.21.15.67:0/3511204721 conn(0x7fe36806bae0 msgr2=0x7fe36806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:46.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.522+0000 7fe36e600640 1 -- 172.21.15.67:0/3511204721 shutdown_connections 2024-05-30T16:59:46.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.522+0000 7fe36e600640 1 -- 172.21.15.67:0/3511204721 wait complete. 2024-05-30T16:59:46.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.523+0000 7fe36e600640 1 Processor -- start 2024-05-30T16:59:46.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.524+0000 7fe36e600640 1 -- start start 2024-05-30T16:59:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.524+0000 7fe36e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680ccc30 0x7fe3680b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3680d0460 0x7fe3681f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36815c6b0 0x7fe36814b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe36806e350 con 0x7fe36815c6b0 2024-05-30T16:59:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe36806e1d0 con 0x7fe3680d0460 2024-05-30T16:59:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3680cbf30 con 0x7fe3680ccc30 2024-05-30T16:59:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36815c6b0 0x7fe36814b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36815c6b0 0x7fe36814b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55134/0 (socket says 172.21.15.67:55134) 2024-05-30T16:59:46.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36dc00640 1 -- 172.21.15.67:0/509191536 learned_addr learned my addr 172.21.15.67:0/509191536 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36d200640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680ccc30 0x7fe3680b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe367e00640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3680d0460 0x7fe3681f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36dc00640 1 -- 172.21.15.67:0/509191536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680ccc30 msgr2=0x7fe3680b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36dc00640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680ccc30 0x7fe3680b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36dc00640 1 -- 172.21.15.67:0/509191536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3680d0460 msgr2=0x7fe3681f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36dc00640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3680d0460 0x7fe3681f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.525+0000 7fe36dc00640 1 -- 172.21.15.67:0/509191536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3681f8610 con 0x7fe36815c6b0 2024-05-30T16:59:46.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.526+0000 7fe36dc00640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36815c6b0 0x7fe36814b660 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe36000d800 tx=0x7fe36000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:46.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.526+0000 7fe364c00640 1 -- 172.21.15.67:0/509191536 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36000b6c0 con 0x7fe36815c6b0 2024-05-30T16:59:46.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.526+0000 7fe364c00640 1 -- 172.21.15.67:0/509191536 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe360019070 con 0x7fe36815c6b0 2024-05-30T16:59:46.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.526+0000 7fe364c00640 1 -- 172.21.15.67:0/509191536 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36000bac0 con 0x7fe36815c6b0 2024-05-30T16:59:46.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.526+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3680b2b10 con 0x7fe36815c6b0 2024-05-30T16:59:46.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.526+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3680c9090 con 0x7fe36815c6b0 2024-05-30T16:59:46.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.527+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3680689e0 con 0x7fe36815c6b0 2024-05-30T16:59:46.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.528+0000 7fe364c00640 1 -- 172.21.15.67:0/509191536 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7fe36000bc60 con 0x7fe36815c6b0 2024-05-30T16:59:46.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.529+0000 7fe364c00640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe34406dca0 0x7fe344070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:46.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.529+0000 7fe364c00640 1 -- 172.21.15.67:0/509191536 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fe36008b050 con 0x7fe36815c6b0 2024-05-30T16:59:46.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.529+0000 7fe36d200640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe34406dca0 0x7fe344070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:46.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.530+0000 7fe36d200640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe34406dca0 0x7fe344070160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe350005880 tx=0x7fe350009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:46.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.534+0000 7fe364c00640 1 -- 172.21.15.67:0/509191536 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe36005ab20 con 0x7fe36815c6b0 2024-05-30T16:59:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.814+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe3680d0990 con 0x7fe36815c6b0 2024-05-30T16:59:46.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.816+0000 7fe364c00640 1 -- 172.21.15.67:0/509191536 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2123 (secure 0 0 0) 0x7fe36005a420 con 0x7fe36815c6b0 2024-05-30T16:59:46.816 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:59:46.816 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:46.816 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:59:46.817 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 980, 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 91, 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:59:46.818 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:59:46.819 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1566, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 6443504100, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 13782044672, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 759312068608, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 268335264, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 268358210, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 64, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 71 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 56, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:59:46.820 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 6, 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7398 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:59:46.821 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:59:40.449642+0000", 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:59:46.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:46.823 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:59:46.823 INFO:teuthology.orchestra.run.smithi067.stdout: "c94bce46-24ca-440d-8c9f-e3b857e30cd5": { 2024-05-30T16:59:46.823 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 25.0 seconds before exiting (12s)\n [============================] ", 2024-05-30T16:59:46.823 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:59:46.823 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:59:46.823 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:46.823 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:46.823 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe34406dca0 msgr2=0x7fe344070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe34406dca0 0x7fe344070160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe350005880 tx=0x7fe350009040 comp rx=0 tx=0).stop 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36815c6b0 msgr2=0x7fe36814b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36815c6b0 0x7fe36814b660 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe36000d800 tx=0x7fe36000dcc0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 shutdown_connections 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe34406dca0 0x7fe344070160 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36815c6b0 0x7fe36814b660 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3680d0460 0x7fe3681f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 --2- 172.21.15.67:0/509191536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3680ccc30 0x7fe3680b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 >> 172.21.15.67:0/509191536 conn(0x7fe36806bae0 msgr2=0x7fe368099840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 shutdown_connections 2024-05-30T16:59:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:46.818+0000 7fe36e600640 1 -- 172.21.15.67:0/509191536 wait complete. 2024-05-30T16:59:51.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:59:52.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.070+0000 7fcd92400640 1 Processor -- start 2024-05-30T16:59:52.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.070+0000 7fcd92400640 1 -- start start 2024-05-30T16:59:52.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.072+0000 7fcd92400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcd8c145320 0x7fcd8c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:52.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.072+0000 7fcd92400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c0b43b0 0x7fcd8c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:52.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.072+0000 7fcd92400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd8c0cf0c0 con 0x7fcd8c072200 2024-05-30T16:59:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.072+0000 7fcd92400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcd8c0b4a20 con 0x7fcd8c145320 2024-05-30T16:59:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.072+0000 7fcd92400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c0b4d00 con 0x7fcd8c0b43b0 2024-05-30T16:59:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.072+0000 7fcd8aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcd8c072200 0x7fcd8c0b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54076/0 (socket says 172.21.15.67:54076) 2024-05-30T16:59:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.072+0000 7fcd8aa00640 1 -- 172.21.15.67:0/2186392015 learned_addr learned my addr 172.21.15.67:0/2186392015 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T16:59:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8b400640 1 --2- 172.21.15.67:0/2186392015 >> v2:172.21.15.169:3300/0 conn(0x7fcd8c145320 0x7fcd8c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4193699059 0 0) 0x7fcd8c0cf0c0 con 0x7fcd8c072200 2024-05-30T16:59:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8be00640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c0b43b0 0x7fcd8c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd70003520 con 0x7fcd8c072200 2024-05-30T16:59:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3473279577 0 0) 0x7fcd70003520 con 0x7fcd8c072200 2024-05-30T16:59:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c0b43b0 msgr2=0x7fcd8c0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c0b43b0 0x7fcd8c0cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 >> v2:172.21.15.169:3300/0 conn(0x7fcd8c145320 msgr2=0x7fcd8c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 --2- 172.21.15.67:0/2186392015 >> v2:172.21.15.169:3300/0 conn(0x7fcd8c145320 0x7fcd8c071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c0b4fe0 con 0x7fcd8c072200 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fcd78001670 con 0x7fcd8c072200 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 >> v1:172.21.15.67:6789/0 conn(0x7fcd8c072200 legacy=0x7fcd8c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd700039b0 0x7fcd70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd70004800 0x7fcd70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c0b43b0 0x7fcd700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c0b4a20 con 0x7fcd700039b0 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c0b4d00 con 0x7fcd70004800 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.073+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7000fe20 con 0x7fcd8c0b43b0 2024-05-30T16:59:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.074+0000 7fcd8b400640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd700039b0 0x7fcd70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.074+0000 7fcd8aa00640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd70004800 0x7fcd70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.074+0000 7fcd8be00640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c0b43b0 0x7fcd700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.074+0000 7fcd8b400640 1 -- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c0b43b0 msgr2=0x7fcd700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.074+0000 7fcd8b400640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c0b43b0 0x7fcd700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.074+0000 7fcd8b400640 1 -- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd70004800 msgr2=0x7fcd70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.074+0000 7fcd8b400640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd70004800 0x7fcd70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.074+0000 7fcd8b400640 1 -- 172.21.15.67:0/2186392015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd70003520 con 0x7fcd700039b0 2024-05-30T16:59:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.076+0000 7fcd8b400640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd700039b0 0x7fcd70003dd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00ba30 tx=0x7fcd70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8661eaa673635d04 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.076+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c030070 con 0x7fcd700039b0 2024-05-30T16:59:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.076+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c02ae10 con 0x7fcd700039b0 2024-05-30T16:59:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.076+0000 7fcd8a000640 1 -- 172.21.15.67:0/2186392015 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00b600 con 0x7fcd700039b0 2024-05-30T16:59:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.077+0000 7fcd92400640 1 -- 172.21.15.67:0/2186392015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd700039b0 msgr2=0x7fcd70003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.077+0000 7fcd92400640 1 --2- 172.21.15.67:0/2186392015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd700039b0 0x7fcd70003dd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00ba30 tx=0x7fcd70005830 comp rx=0 tx=0).stop 2024-05-30T16:59:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.077+0000 7fcd8be00640 1 -- 172.21.15.67:0/2186392015 reap_dead start 2024-05-30T16:59:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.077+0000 7fcd92400640 1 -- 172.21.15.67:0/2186392015 shutdown_connections 2024-05-30T16:59:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.077+0000 7fcd92400640 1 -- 172.21.15.67:0/2186392015 >> 172.21.15.67:0/2186392015 conn(0x7fcd8c067ce0 msgr2=0x7fcd8c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.077+0000 7fcd92400640 1 -- 172.21.15.67:0/2186392015 shutdown_connections 2024-05-30T16:59:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.077+0000 7fcd92400640 1 -- 172.21.15.67:0/2186392015 wait complete. 2024-05-30T16:59:52.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.079+0000 7fcd92400640 1 Processor -- start 2024-05-30T16:59:52.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.080+0000 7fcd92400640 1 -- start start 2024-05-30T16:59:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.081+0000 7fcd92400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd8c1f35f0 0x7fcd8c1f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.081+0000 7fcd92400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd8c142a30 0x7fcd8c142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.081+0000 7fcd92400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c143d10 0x7fcd8c144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.081+0000 7fcd92400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c0cf880 con 0x7fcd8c142a30 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.081+0000 7fcd92400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c145700 con 0x7fcd8c1f35f0 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.081+0000 7fcd92400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c0cfa00 con 0x7fcd8c143d10 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.081+0000 7fcd8aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd8c142a30 0x7fcd8c142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.081+0000 7fcd8aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd8c142a30 0x7fcd8c142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34746/0 (socket says 172.21.15.67:34746) 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.081+0000 7fcd8aa00640 1 -- 172.21.15.67:0/3494094156 learned_addr learned my addr 172.21.15.67:0/3494094156 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd8be00640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c143d10 0x7fcd8c144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd8b400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd8c1f35f0 0x7fcd8c1f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd8aa00640 1 -- 172.21.15.67:0/3494094156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c143d10 msgr2=0x7fcd8c144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd8aa00640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c143d10 0x7fcd8c144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd8aa00640 1 -- 172.21.15.67:0/3494094156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd8c1f35f0 msgr2=0x7fcd8c1f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd8aa00640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd8c1f35f0 0x7fcd8c1f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd8aa00640 1 -- 172.21.15.67:0/3494094156 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c1f39d0 con 0x7fcd8c142a30 2024-05-30T16:59:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd8b400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd8c1f35f0 0x7fcd8c1f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T16:59:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd8aa00640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd8c142a30 0x7fcd8c142eb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800e6d0 tx=0x7fcd7800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.082+0000 7fcd83400640 1 -- 172.21.15.67:0/3494094156 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78019070 con 0x7fcd8c142a30 2024-05-30T16:59:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.083+0000 7fcd83400640 1 -- 172.21.15.67:0/3494094156 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7800c6c0 con 0x7fcd8c142a30 2024-05-30T16:59:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.083+0000 7fcd83400640 1 -- 172.21.15.67:0/3494094156 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7800cab0 con 0x7fcd8c142a30 2024-05-30T16:59:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.083+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8c15ddc0 con 0x7fcd8c142a30 2024-05-30T16:59:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.083+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8c15e380 con 0x7fcd8c142a30 2024-05-30T16:59:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.084+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd8c0689e0 con 0x7fcd8c142a30 2024-05-30T16:59:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.085+0000 7fcd83400640 1 -- 172.21.15.67:0/3494094156 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7fcd7801c070 con 0x7fcd8c142a30 2024-05-30T16:59:52.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.086+0000 7fcd83400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd6c06dca0 0x7fcd6c070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:52.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.086+0000 7fcd83400640 1 -- 172.21.15.67:0/3494094156 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fcd7808a000 con 0x7fcd8c142a30 2024-05-30T16:59:52.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.086+0000 7fcd8b400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd6c06dca0 0x7fcd6c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:52.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.087+0000 7fcd8b400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd6c06dca0 0x7fcd6c070160 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00ba00 tx=0x7fcd7c030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.091+0000 7fcd83400640 1 -- 172.21.15.67:0/3494094156 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd78059e80 con 0x7fcd8c142a30 2024-05-30T16:59:52.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.384+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcd8c0b34e0 con 0x7fcd8c142a30 2024-05-30T16:59:52.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.385+0000 7fcd83400640 1 -- 172.21.15.67:0/3494094156 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2123 (secure 0 0 0) 0x7fcd78059780 con 0x7fcd8c142a30 2024-05-30T16:59:52.385 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:59:52.386 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 985, 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 91, 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:59:52.387 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:59:52.388 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1565, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 6443524819, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 13782073344, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 759312039936, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 140458464, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 142220808, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 33, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 39 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 56, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6, 2024-05-30T16:59:52.389 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 6, 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7398 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:59:52.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:59:40.449642+0000", 2024-05-30T16:59:52.391 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:59:52.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:52.392 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:59:52.392 INFO:teuthology.orchestra.run.smithi067.stdout: "c94bce46-24ca-440d-8c9f-e3b857e30cd5": { 2024-05-30T16:59:52.392 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 19.0 seconds before exiting (18s)\n [============================] ", 2024-05-30T16:59:52.392 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:59:52.392 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:59:52.392 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:52.392 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:52.392 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd6c06dca0 msgr2=0x7fcd6c070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd6c06dca0 0x7fcd6c070160 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00ba00 tx=0x7fcd7c030000 comp rx=0 tx=0).stop 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd8c142a30 msgr2=0x7fcd8c142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd8c142a30 0x7fcd8c142eb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800e6d0 tx=0x7fcd7800eb90 comp rx=0 tx=0).stop 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 shutdown_connections 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd6c06dca0 0x7fcd6c070160 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd8c143d10 0x7fcd8c144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd8c142a30 0x7fcd8c142eb0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 --2- 172.21.15.67:0/3494094156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd8c1f35f0 0x7fcd8c1f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 >> 172.21.15.67:0/3494094156 conn(0x7fcd8c067ce0 msgr2=0x7fcd8c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 shutdown_connections 2024-05-30T16:59:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:52.389+0000 7fcd92400640 1 -- 172.21.15.67:0/3494094156 wait complete. 2024-05-30T16:59:57.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T16:59:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.629+0000 7f0b9a600640 1 Processor -- start 2024-05-30T16:59:57.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.629+0000 7f0b9a600640 1 -- start start 2024-05-30T16:59:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.631+0000 7f0b9a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0b940cc2b0 0x7f0b940d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.631+0000 7f0b9a600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0b940d09a0 0x7f0b940cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.631+0000 7f0b9a600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b941621d0 con 0x7f0b94161df0 2024-05-30T16:59:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.631+0000 7f0b9a600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f0b940cd840 con 0x7f0b940cc2b0 2024-05-30T16:59:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.631+0000 7f0b9a600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0b940d0dd0 con 0x7f0b940d09a0 2024-05-30T16:59:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.631+0000 7f0b93e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0b940cc2b0 0x7f0b940d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.631+0000 7f0b93e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0b940cc2b0 0x7f0b940d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34750/0 (socket says 172.21.15.67:34750) 2024-05-30T16:59:57.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.631+0000 7f0b93e00640 1 -- 172.21.15.67:0/1760903901 learned_addr learned my addr 172.21.15.67:0/1760903901 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.632+0000 7f0b93e00640 1 -- 172.21.15.67:0/1760903901 >> v2:172.21.15.169:3300/0 conn(0x7f0b940d09a0 msgr2=0x7f0b940cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.632+0000 7f0b93e00640 1 --2- 172.21.15.67:0/1760903901 >> v2:172.21.15.169:3300/0 conn(0x7f0b940d09a0 0x7f0b940cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.632+0000 7f0b93e00640 1 -- 172.21.15.67:0/1760903901 >> v1:172.21.15.169:6789/0 conn(0x7f0b94161df0 legacy=0x7f0b94099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T16:59:57.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.632+0000 7f0b93e00640 1 -- 172.21.15.67:0/1760903901 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b940ccf10 con 0x7f0b940cc2b0 2024-05-30T16:59:57.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.633+0000 7f0b93e00640 1 --2- 172.21.15.67:0/1760903901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940cc2b0 0x7f0b940d0460 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0b940b1de0 tx=0x7f0b8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca4a9e0ea8b657 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:57.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.633+0000 7f0b93400640 1 -- 172.21.15.67:0/1760903901 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80037070 con 0x7f0b940cc2b0 2024-05-30T16:59:57.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.633+0000 7f0b93400640 1 -- 172.21.15.67:0/1760903901 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b80032440 con 0x7f0b940cc2b0 2024-05-30T16:59:57.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.633+0000 7f0b93400640 1 -- 172.21.15.67:0/1760903901 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80032810 con 0x7f0b940cc2b0 2024-05-30T16:59:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.634+0000 7f0b9a600640 1 -- 172.21.15.67:0/1760903901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940cc2b0 msgr2=0x7f0b940d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.634+0000 7f0b9a600640 1 --2- 172.21.15.67:0/1760903901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940cc2b0 0x7f0b940d0460 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0b940b1de0 tx=0x7f0b8002a230 comp rx=0 tx=0).stop 2024-05-30T16:59:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.634+0000 7f0b9a600640 1 -- 172.21.15.67:0/1760903901 shutdown_connections 2024-05-30T16:59:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.634+0000 7f0b9a600640 1 --2- 172.21.15.67:0/1760903901 >> v2:172.21.15.169:3300/0 conn(0x7f0b940d09a0 0x7f0b940cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.634+0000 7f0b9a600640 1 --2- 172.21.15.67:0/1760903901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940cc2b0 0x7f0b940d0460 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.634+0000 7f0b9a600640 1 -- 172.21.15.67:0/1760903901 >> 172.21.15.67:0/1760903901 conn(0x7f0b9406bae0 msgr2=0x7f0b9406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.634+0000 7f0b9a600640 1 -- 172.21.15.67:0/1760903901 shutdown_connections 2024-05-30T16:59:57.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.634+0000 7f0b9a600640 1 -- 172.21.15.67:0/1760903901 wait complete. 2024-05-30T16:59:57.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.636+0000 7f0b9a600640 1 Processor -- start 2024-05-30T16:59:57.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.636+0000 7f0b9a600640 1 -- start start 2024-05-30T16:59:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.637+0000 7f0b9a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b940cc2b0 0x7f0b94154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.637+0000 7f0b9a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940d09a0 0x7f0b941605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.637+0000 7f0b9a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b94161df0 0x7f0b94160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.637+0000 7f0b9a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b940ccbc0 con 0x7f0b940d09a0 2024-05-30T16:59:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.637+0000 7f0b9a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b940cd4d0 con 0x7f0b94161df0 2024-05-30T16:59:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.637+0000 7f0b9a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0b940ca870 con 0x7f0b940cc2b0 2024-05-30T16:59:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.637+0000 7f0b93e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940d09a0 0x7f0b941605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.637+0000 7f0b93e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940d09a0 0x7f0b941605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34760/0 (socket says 172.21.15.67:34760) 2024-05-30T16:59:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.637+0000 7f0b93e00640 1 -- 172.21.15.67:0/2486651534 learned_addr learned my addr 172.21.15.67:0/2486651534 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T16:59:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b99200640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b940cc2b0 0x7f0b94154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b93e00640 1 -- 172.21.15.67:0/2486651534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b940cc2b0 msgr2=0x7f0b94154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b93e00640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b940cc2b0 0x7f0b94154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b93e00640 1 -- 172.21.15.67:0/2486651534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b94161df0 msgr2=0x7f0b94160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b93e00640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b94161df0 0x7f0b94160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b93e00640 1 -- 172.21.15.67:0/2486651534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b940c6f20 con 0x7f0b940d09a0 2024-05-30T16:59:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b93e00640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940d09a0 0x7f0b941605f0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0b80000c00 tx=0x7f0b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b90c00640 1 -- 172.21.15.67:0/2486651534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80041070 con 0x7f0b940d09a0 2024-05-30T16:59:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b90c00640 1 -- 172.21.15.67:0/2486651534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8003c870 con 0x7f0b940d09a0 2024-05-30T16:59:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b90c00640 1 -- 172.21.15.67:0/2486651534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80037040 con 0x7f0b940d09a0 2024-05-30T16:59:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.638+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b94071c80 con 0x7f0b940d09a0 2024-05-30T16:59:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.639+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b94154970 con 0x7f0b940d09a0 2024-05-30T16:59:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.640+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b940689e0 con 0x7f0b940d09a0 2024-05-30T16:59:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.641+0000 7f0b90c00640 1 -- 172.21.15.67:0/2486651534 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f0b80044070 con 0x7f0b940d09a0 2024-05-30T16:59:57.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.641+0000 7f0b90c00640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b7406dca0 0x7f0b74070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T16:59:57.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.642+0000 7f0b90c00640 1 -- 172.21.15.67:0/2486651534 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f0b800ab830 con 0x7f0b940d09a0 2024-05-30T16:59:57.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.642+0000 7f0b99200640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b7406dca0 0x7f0b74070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T16:59:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.643+0000 7f0b99200640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b7406dca0 0x7f0b74070160 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800a730 tx=0x7f0b88008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T16:59:57.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.647+0000 7f0b90c00640 1 -- 172.21.15.67:0/2486651534 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b941621d0 con 0x7f0b940d09a0 2024-05-30T16:59:57.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.928+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0b940ca650 con 0x7f0b940d09a0 2024-05-30T16:59:57.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.929+0000 7f0b90c00640 1 -- 172.21.15.67:0/2486651534 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2109 (secure 0 0 0) 0x7f0b8007b420 con 0x7f0b940d09a0 2024-05-30T16:59:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T16:59:57.929 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T16:59:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T16:59:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T16:59:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T16:59:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 991, 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 91, 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T16:59:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1565, 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 6443530019, 2024-05-30T16:59:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 13782085632, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 759312027648, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 85, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 2644, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 1 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 56, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T16:59:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 6, 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7398 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T16:59:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:59:40.449642+0000", 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T16:59:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T16:59:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "c94bce46-24ca-440d-8c9f-e3b857e30cd5": { 2024-05-30T16:59:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 14.0 seconds before exiting (23s)\n [============================] ", 2024-05-30T16:59:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T16:59:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T16:59:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T16:59:57.936 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T16:59:57.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b7406dca0 msgr2=0x7f0b74070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:57.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b7406dca0 0x7f0b74070160 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800a730 tx=0x7f0b88008040 comp rx=0 tx=0).stop 2024-05-30T16:59:57.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940d09a0 msgr2=0x7f0b941605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T16:59:57.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940d09a0 0x7f0b941605f0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0b80000c00 tx=0x7f0b800046c0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 shutdown_connections 2024-05-30T16:59:57.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b7406dca0 0x7f0b74070160 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b94161df0 0x7f0b94160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b940d09a0 0x7f0b941605f0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 --2- 172.21.15.67:0/2486651534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b940cc2b0 0x7f0b94154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T16:59:57.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.931+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 >> 172.21.15.67:0/2486651534 conn(0x7f0b9406bae0 msgr2=0x7f0b9406bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T16:59:57.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.932+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 shutdown_connections 2024-05-30T16:59:57.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T16:59:57.932+0000 7f0b9a600640 1 -- 172.21.15.67:0/2486651534 wait complete. 2024-05-30T17:00:02.952 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:00:03.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.172+0000 7f4c53200640 1 Processor -- start 2024-05-30T17:00:03.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.172+0000 7f4c53200640 1 -- start start 2024-05-30T17:00:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.173+0000 7f4c53200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c5415c6b0 0x7f4c540b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.173+0000 7f4c53200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c540ccc30 0x7f4c5406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.173+0000 7f4c53200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c540d0460 0x7f4c540d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.173+0000 7f4c53200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4c540cd1a0 con 0x7f4c540d0460 2024-05-30T17:00:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.173+0000 7f4c53200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4c540cd480 con 0x7f4c5415c6b0 2024-05-30T17:00:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.173+0000 7f4c53200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c540cd760 con 0x7f4c540ccc30 2024-05-30T17:00:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c52800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c540d0460 0x7f4c540d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c52800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c540d0460 0x7f4c540d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53260/0 (socket says 172.21.15.67:53260) 2024-05-30T17:00:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c52800640 1 -- 172.21.15.67:0/3270176552 learned_addr learned my addr 172.21.15.67:0/3270176552 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c4be00640 1 --2- 172.21.15.67:0/3270176552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c540ccc30 0x7f4c5406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c51e00640 1 --2- 172.21.15.67:0/3270176552 >> v2:172.21.15.169:3300/0 conn(0x7f4c5415c6b0 0x7f4c540b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c52800640 1 -- 172.21.15.67:0/3270176552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c540ccc30 msgr2=0x7f4c5406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c52800640 1 --2- 172.21.15.67:0/3270176552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c540ccc30 0x7f4c5406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c52800640 1 -- 172.21.15.67:0/3270176552 >> v2:172.21.15.169:3300/0 conn(0x7f4c5415c6b0 msgr2=0x7f4c540b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c52800640 1 --2- 172.21.15.67:0/3270176552 >> v2:172.21.15.169:3300/0 conn(0x7f4c5415c6b0 0x7f4c540b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c52800640 1 -- 172.21.15.67:0/3270176552 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c540cda40 con 0x7f4c540d0460 2024-05-30T17:00:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.174+0000 7f4c51e00640 1 --2- 172.21.15.67:0/3270176552 >> v2:172.21.15.169:3300/0 conn(0x7f4c5415c6b0 0x7f4c540b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:03.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.175+0000 7f4c52800640 1 --2- 172.21.15.67:0/3270176552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540d0460 0x7f4c540d08f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4c540b3770 tx=0x7f4c4c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbcb9008b3c1f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:03.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.176+0000 7f4c51400640 1 -- 172.21.15.67:0/3270176552 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c036070 con 0x7f4c540d0460 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.176+0000 7f4c51400640 1 -- 172.21.15.67:0/3270176552 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4c031600 con 0x7f4c540d0460 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.176+0000 7f4c51400640 1 -- 172.21.15.67:0/3270176552 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c0319d0 con 0x7f4c540d0460 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.176+0000 7f4c53200640 1 -- 172.21.15.67:0/3270176552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540d0460 msgr2=0x7f4c540d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.176+0000 7f4c53200640 1 --2- 172.21.15.67:0/3270176552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540d0460 0x7f4c540d08f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4c540b3770 tx=0x7f4c4c02a6a0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.176+0000 7f4c53200640 1 -- 172.21.15.67:0/3270176552 shutdown_connections 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.176+0000 7f4c53200640 1 --2- 172.21.15.67:0/3270176552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540d0460 0x7f4c540d08f0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.176+0000 7f4c53200640 1 --2- 172.21.15.67:0/3270176552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c540ccc30 0x7f4c5406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.177+0000 7f4c53200640 1 --2- 172.21.15.67:0/3270176552 >> v2:172.21.15.169:3300/0 conn(0x7f4c5415c6b0 0x7f4c540b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.177+0000 7f4c53200640 1 -- 172.21.15.67:0/3270176552 >> 172.21.15.67:0/3270176552 conn(0x7f4c5406bae0 msgr2=0x7f4c5406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.177+0000 7f4c53200640 1 -- 172.21.15.67:0/3270176552 shutdown_connections 2024-05-30T17:00:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.177+0000 7f4c53200640 1 -- 172.21.15.67:0/3270176552 wait complete. 2024-05-30T17:00:03.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.178+0000 7f4c53200640 1 Processor -- start 2024-05-30T17:00:03.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.179+0000 7f4c53200640 1 -- start start 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.179+0000 7f4c53200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540ccc30 0x7f4c540c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.179+0000 7f4c53200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c540d0460 0x7f4c540c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c53200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5415c6b0 0x7f4c541f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c53200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4c540b4470 con 0x7f4c540ccc30 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c53200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c540b42f0 con 0x7f4c540d0460 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c53200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5406e1d0 con 0x7f4c5415c6b0 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c51e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540ccc30 0x7f4c540c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c51e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540ccc30 0x7f4c540c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53268/0 (socket says 172.21.15.67:53268) 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c51e00640 1 -- 172.21.15.67:0/1974322553 learned_addr learned my addr 172.21.15.67:0/1974322553 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c52800640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5415c6b0 0x7f4c541f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c4be00640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c540d0460 0x7f4c540c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c51e00640 1 -- 172.21.15.67:0/1974322553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5415c6b0 msgr2=0x7f4c541f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c51e00640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5415c6b0 0x7f4c541f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c51e00640 1 -- 172.21.15.67:0/1974322553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c540d0460 msgr2=0x7f4c540c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c51e00640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c540d0460 0x7f4c540c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.180+0000 7f4c51e00640 1 -- 172.21.15.67:0/1974322553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c540c6480 con 0x7f4c540ccc30 2024-05-30T17:00:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.181+0000 7f4c51e00640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540ccc30 0x7f4c540c58e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c820 tx=0x7f4c4400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.181+0000 7f4c4a000640 1 -- 172.21.15.67:0/1974322553 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44015070 con 0x7f4c540ccc30 2024-05-30T17:00:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.181+0000 7f4c4a000640 1 -- 172.21.15.67:0/1974322553 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c44007a30 con 0x7f4c540ccc30 2024-05-30T17:00:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.181+0000 7f4c4a000640 1 -- 172.21.15.67:0/1974322553 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44007e00 con 0x7f4c540ccc30 2024-05-30T17:00:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.181+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c540c6760 con 0x7f4c540ccc30 2024-05-30T17:00:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.181+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c540c6d20 con 0x7f4c540ccc30 2024-05-30T17:00:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.183+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c540c7060 con 0x7f4c540ccc30 2024-05-30T17:00:03.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.183+0000 7f4c4a000640 1 -- 172.21.15.67:0/1974322553 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f4c44018070 con 0x7f4c540ccc30 2024-05-30T17:00:03.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.185+0000 7f4c4a000640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c3006dca0 0x7f4c30070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:03.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.185+0000 7f4c4a000640 1 -- 172.21.15.67:0/1974322553 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f4c4408aba0 con 0x7f4c540ccc30 2024-05-30T17:00:03.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.185+0000 7f4c4be00640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c3006dca0 0x7f4c30070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:03.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.186+0000 7f4c4be00640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c3006dca0 0x7f4c30070160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006fd0 tx=0x7f4c3c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.190+0000 7f4c4a000640 1 -- 172.21.15.67:0/1974322553 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c540c7060 con 0x7f4c540ccc30 2024-05-30T17:00:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.468+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4c540689e0 con 0x7f4c540ccc30 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.469+0000 7f4c4a000640 1 -- 172.21.15.67:0/1974322553 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2077 (secure 0 0 0) 0x7f4c4405a670 con 0x7f4c540ccc30 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:03.470 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 996, 2024-05-30T17:00:03.471 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 91, 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:03.472 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1565, 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 6443531634, 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 13782089728, 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 759312023552, 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:00:03.473 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 682, 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 56, 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6, 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 6, 2024-05-30T17:00:03.474 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7398 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:00:03.475 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:59:40.449642+0000", 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "c94bce46-24ca-440d-8c9f-e3b857e30cd5": { 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 8.0 seconds before exiting (29s)\n [============================] ", 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:00:03.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:03.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:03.477 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:00:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c3006dca0 msgr2=0x7f4c30070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c3006dca0 0x7f4c30070160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006fd0 tx=0x7f4c3c008040 comp rx=0 tx=0).stop 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540ccc30 msgr2=0x7f4c540c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540ccc30 0x7f4c540c58e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c820 tx=0x7f4c4400cc20 comp rx=0 tx=0).stop 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 shutdown_connections 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4c3006dca0 0x7f4c30070160 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5415c6b0 0x7f4c541f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c540d0460 0x7f4c540c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 --2- 172.21.15.67:0/1974322553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c540ccc30 0x7f4c540c58e0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 >> 172.21.15.67:0/1974322553 conn(0x7f4c5406bae0 msgr2=0x7f4c5406bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 shutdown_connections 2024-05-30T17:00:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:03.472+0000 7f4c53200640 1 -- 172.21.15.67:0/1974322553 wait complete. 2024-05-30T17:00:08.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:00:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.715+0000 7f35b1800640 1 Processor -- start 2024-05-30T17:00:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.715+0000 7f35b1800640 1 -- start start 2024-05-30T17:00:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.717+0000 7f35b1800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f35ac0d0b70 0x7f35ac169460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.717+0000 7f35b1800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ac1621d0 con 0x7f35ac0cc2b0 2024-05-30T17:00:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.717+0000 7f35b1800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ac0ccf10 con 0x7f35ac161df0 2024-05-30T17:00:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.717+0000 7f35b1800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f35ac0cd840 con 0x7f35ac0d0b70 2024-05-30T17:00:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.717+0000 7f35ab400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f35ac0cc2b0 0x7f35ac0d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34644/0 (socket says 172.21.15.67:34644) 2024-05-30T17:00:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.717+0000 7f35ab400640 1 -- 172.21.15.67:0/645276254 learned_addr learned my addr 172.21.15.67:0/645276254 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362862374 0 0) 0x7f3598001110 con 0x7f35ac0cc2b0 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3590003520 con 0x7f35ac0cc2b0 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35b0e00640 1 --2- 172.21.15.67:0/645276254 >> v2:172.21.15.169:3300/0 conn(0x7f35ac0d0b70 0x7f35ac169460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1299665641 0 0) 0x7f3590003520 con 0x7f35ac0cc2b0 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 >> v2:172.21.15.169:3300/0 conn(0x7f35ac0d0b70 msgr2=0x7f35ac169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 --2- 172.21.15.67:0/645276254 >> v2:172.21.15.169:3300/0 conn(0x7f35ac0d0b70 0x7f35ac169460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 >> v1:172.21.15.169:6789/0 conn(0x7f35ac161df0 legacy=0x7f35ac099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac0cd1f0 con 0x7f35ac0cc2b0 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f3598001840 con 0x7f35ac0cc2b0 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 >> v1:172.21.15.67:6789/0 conn(0x7f35ac0cc2b0 legacy=0x7f35ac0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35900039b0 0x7f3590003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f3590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35900051c0 0x7f359000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f359000c0e0 con 0x7f3590004800 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f359000c290 con 0x7f35900051c0 2024-05-30T17:00:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.718+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f359000c490 con 0x7f35900039b0 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35ab400640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f3590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35b0e00640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35900051c0 0x7f359000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35abe00640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35900039b0 0x7f3590003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35ab400640 1 -- 172.21.15.67:0/645276254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35900039b0 msgr2=0x7f3590003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35ab400640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35900039b0 0x7f3590003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35ab400640 1 -- 172.21.15.67:0/645276254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35900051c0 msgr2=0x7f359000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35ab400640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35900051c0 0x7f359000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35ab400640 1 -- 172.21.15.67:0/645276254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3590003520 con 0x7f3590004800 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35b0e00640 1 -- 172.21.15.67:0/645276254 reap_dead start 2024-05-30T17:00:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.719+0000 7f35abe00640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35900039b0 0x7f3590003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.720+0000 7f35ab400640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f3590004c80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f359800c4c0 tx=0x7f3590005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40783b3829445f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359800c0e0 con 0x7f3590004800 2024-05-30T17:00:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3598034070 con 0x7f3590004800 2024-05-30T17:00:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35aaa00640 1 -- 172.21.15.67:0/645276254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598005d70 con 0x7f3590004800 2024-05-30T17:00:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35b1800640 1 -- 172.21.15.67:0/645276254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 msgr2=0x7f3590004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35b1800640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f3590004c80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f359800c4c0 tx=0x7f3590005830 comp rx=0 tx=0).stop 2024-05-30T17:00:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35b1800640 1 -- 172.21.15.67:0/645276254 shutdown_connections 2024-05-30T17:00:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35b1800640 1 --2- 172.21.15.67:0/645276254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f3590004c80 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35b1800640 1 -- 172.21.15.67:0/645276254 >> 172.21.15.67:0/645276254 conn(0x7f35ac06bae0 msgr2=0x7f35ac06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35b1800640 1 -- 172.21.15.67:0/645276254 shutdown_connections 2024-05-30T17:00:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.721+0000 7f35b1800640 1 -- 172.21.15.67:0/645276254 wait complete. 2024-05-30T17:00:08.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.723+0000 7f35b1800640 1 Processor -- start 2024-05-30T17:00:08.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.723+0000 7f35b1800640 1 -- start start 2024-05-30T17:00:08.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.724+0000 7f35b1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f35ac1f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:08.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.724+0000 7f35b1800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35ac1f77d0 0x7f35ac154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:08.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.724+0000 7f35b1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35ac0c5bc0 0x7f35ac0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:08.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.724+0000 7f35b1800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac0ca870 con 0x7f3590004800 2024-05-30T17:00:08.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.724+0000 7f35b1800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac0ccbc0 con 0x7f35ac1f77d0 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.724+0000 7f35b1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac0ca9f0 con 0x7f35ac0c5bc0 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.724+0000 7f35abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f35ac1f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.725+0000 7f35abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f35ac1f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53276/0 (socket says 172.21.15.67:53276) 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.725+0000 7f35abe00640 1 -- 172.21.15.67:0/3262536279 learned_addr learned my addr 172.21.15.67:0/3262536279 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.725+0000 7f35abe00640 1 -- 172.21.15.67:0/3262536279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35ac0c5bc0 msgr2=0x7f35ac0c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.725+0000 7f35abe00640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35ac0c5bc0 0x7f35ac0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.725+0000 7f35abe00640 1 -- 172.21.15.67:0/3262536279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35ac1f77d0 msgr2=0x7f35ac154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.725+0000 7f35abe00640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35ac1f77d0 0x7f35ac154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.725+0000 7f35abe00640 1 -- 172.21.15.67:0/3262536279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac1f7cd0 con 0x7f3590004800 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.725+0000 7f35abe00640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f35ac1f4c80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d4a0 tx=0x7f359c00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.726+0000 7f35a3e00640 1 -- 172.21.15.67:0/3262536279 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c019070 con 0x7f3590004800 2024-05-30T17:00:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.726+0000 7f35a3e00640 1 -- 172.21.15.67:0/3262536279 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c005480 con 0x7f3590004800 2024-05-30T17:00:08.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.726+0000 7f35a3e00640 1 -- 172.21.15.67:0/3262536279 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c007230 con 0x7f3590004800 2024-05-30T17:00:08.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.726+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac1f7fb0 con 0x7f3590004800 2024-05-30T17:00:08.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.726+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac1f8570 con 0x7f3590004800 2024-05-30T17:00:08.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.727+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac0689e0 con 0x7f3590004800 2024-05-30T17:00:08.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.728+0000 7f35a3e00640 1 -- 172.21.15.67:0/3262536279 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f359c0073f0 con 0x7f3590004800 2024-05-30T17:00:08.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.729+0000 7f35a3e00640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f358c06dcd0 0x7f358c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:08.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.729+0000 7f35a3e00640 1 -- 172.21.15.67:0/3262536279 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f359c08a6a0 con 0x7f3590004800 2024-05-30T17:00:08.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.729+0000 7f35ab400640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f358c06dcd0 0x7f358c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:08.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.730+0000 7f35ab400640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f358c06dcd0 0x7f358c070190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3598033000 tx=0x7f3598009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:08.734+0000 7f35a3e00640 1 -- 172.21.15.67:0/3262536279 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f359c05a170 con 0x7f3590004800 2024-05-30T17:00:09.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.016+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f35ac0af410 con 0x7f3590004800 2024-05-30T17:00:09.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.017+0000 7f35a3e00640 1 -- 172.21.15.67:0/3262536279 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2078 (secure 0 0 0) 0x7f359c059a70 con 0x7f3590004800 2024-05-30T17:00:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:09.017 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:00:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1002, 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 91, 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:00:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:00:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1565, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 6443531634, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 13782089728, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 759312023552, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 170, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 56, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6, 2024-05-30T17:00:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 6, 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7398 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:59:40.449642+0000", 2024-05-30T17:00:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:00:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:00:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "c94bce46-24ca-440d-8c9f-e3b857e30cd5": { 2024-05-30T17:00:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 2.0 seconds before exiting (35s)\n [============================] ", 2024-05-30T17:00:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:00:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:00:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:09.024 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:00:09.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f358c06dcd0 msgr2=0x7f358c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:09.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f358c06dcd0 0x7f358c070190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3598033000 tx=0x7f3598009030 comp rx=0 tx=0).stop 2024-05-30T17:00:09.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 msgr2=0x7f35ac1f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:09.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f35ac1f4c80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d4a0 tx=0x7f359c00d960 comp rx=0 tx=0).stop 2024-05-30T17:00:09.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 shutdown_connections 2024-05-30T17:00:09.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f358c06dcd0 0x7f358c070190 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:09.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35ac0c5bc0 0x7f35ac0c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:09.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35ac1f77d0 0x7f35ac154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:09.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 --2- 172.21.15.67:0/3262536279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3590004800 0x7f35ac1f4c80 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:09.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.019+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 >> 172.21.15.67:0/3262536279 conn(0x7f35ac06bae0 msgr2=0x7f35ac06bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:09.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.020+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 shutdown_connections 2024-05-30T17:00:09.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:09.020+0000 7f35b1800640 1 -- 172.21.15.67:0/3262536279 wait complete. 2024-05-30T17:00:14.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:00:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.260+0000 7f6068c00640 1 Processor -- start 2024-05-30T17:00:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.260+0000 7f6068c00640 1 -- start start 2024-05-30T17:00:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.261+0000 7f6068c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6064155a00 0x7f606415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.261+0000 7f6068c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f606406f390 0x7f606406f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.261+0000 7f6068c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606406e3d0 con 0x7f60640d0970 2024-05-30T17:00:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.261+0000 7f6068c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f606406fd40 con 0x7f606406f390 2024-05-30T17:00:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.261+0000 7f6068c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f60640ccf20 con 0x7f6064155a00 2024-05-30T17:00:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6062800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f606406f390 0x7f606406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6061400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f60640d0970 0x7f606406dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48982/0 (socket says 172.21.15.67:48982) 2024-05-30T17:00:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6061400640 1 -- 172.21.15.67:0/1267186632 learned_addr learned my addr 172.21.15.67:0/1267186632 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6061e00640 1 --2- 172.21.15.67:0/1267186632 >> v2:172.21.15.169:3300/0 conn(0x7f6064155a00 0x7f606415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6062800640 1 -- 172.21.15.67:0/1267186632 >> v2:172.21.15.169:3300/0 conn(0x7f6064155a00 msgr2=0x7f606415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6062800640 1 --2- 172.21.15.67:0/1267186632 >> v2:172.21.15.169:3300/0 conn(0x7f6064155a00 0x7f606415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6062800640 1 -- 172.21.15.67:0/1267186632 >> v1:172.21.15.67:6789/0 conn(0x7f60640d0970 legacy=0x7f606406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6062800640 1 -- 172.21.15.67:0/1267186632 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606415c0b0 con 0x7f606406f390 2024-05-30T17:00:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6060a00640 1 -- 172.21.15.67:0/1267186632 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 522451483 0 0) 0x7f606406e3d0 con 0x7f60640d0970 2024-05-30T17:00:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.262+0000 7f6061e00640 1 --2- 172.21.15.67:0/1267186632 >> v2:172.21.15.169:3300/0 conn(0x7f6064155a00 0x7f606415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.263+0000 7f6062800640 1 --2- 172.21.15.67:0/1267186632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f606406f390 0x7f606406f800 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f606406db90 tx=0x7f605c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04093589e157f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.264+0000 7f6060a00640 1 -- 172.21.15.67:0/1267186632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c007740 con 0x7f606406f390 2024-05-30T17:00:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.264+0000 7f6060a00640 1 -- 172.21.15.67:0/1267186632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605c0078e0 con 0x7f606406f390 2024-05-30T17:00:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.264+0000 7f6060a00640 1 -- 172.21.15.67:0/1267186632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c007cb0 con 0x7f606406f390 2024-05-30T17:00:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.264+0000 7f6068c00640 1 -- 172.21.15.67:0/1267186632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f606406f390 msgr2=0x7f606406f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.264+0000 7f6068c00640 1 --2- 172.21.15.67:0/1267186632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f606406f390 0x7f606406f800 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f606406db90 tx=0x7f605c02a680 comp rx=0 tx=0).stop 2024-05-30T17:00:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.265+0000 7f6068c00640 1 -- 172.21.15.67:0/1267186632 shutdown_connections 2024-05-30T17:00:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.265+0000 7f6068c00640 1 --2- 172.21.15.67:0/1267186632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f606406f390 0x7f606406f800 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.265+0000 7f6068c00640 1 --2- 172.21.15.67:0/1267186632 >> v2:172.21.15.169:3300/0 conn(0x7f6064155a00 0x7f606415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.265+0000 7f6068c00640 1 -- 172.21.15.67:0/1267186632 >> 172.21.15.67:0/1267186632 conn(0x7f6064067ce0 msgr2=0x7f606406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.265+0000 7f6068c00640 1 -- 172.21.15.67:0/1267186632 shutdown_connections 2024-05-30T17:00:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.265+0000 7f6068c00640 1 -- 172.21.15.67:0/1267186632 wait complete. 2024-05-30T17:00:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.266+0000 7f6068c00640 1 Processor -- start 2024-05-30T17:00:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.267+0000 7f6068c00640 1 -- start start 2024-05-30T17:00:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.267+0000 7f6068c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606406f390 0x7f6064149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.267+0000 7f6068c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640d0970 0x7f606414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.267+0000 7f6068c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6064155a00 0x7f606414a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.267+0000 7f6068c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f606415c670 con 0x7f60640d0970 2024-05-30T17:00:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.267+0000 7f6068c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f606415c950 con 0x7f6064155a00 2024-05-30T17:00:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.267+0000 7f6068c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f606415f090 con 0x7f606406f390 2024-05-30T17:00:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640d0970 0x7f606414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640d0970 0x7f606414dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43762/0 (socket says 172.21.15.67:43762) 2024-05-30T17:00:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061400640 1 -- 172.21.15.67:0/1664567181 learned_addr learned my addr 172.21.15.67:0/1664567181 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061e00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606406f390 0x7f6064149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6062800640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6064155a00 0x7f606414a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061400640 1 -- 172.21.15.67:0/1664567181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606406f390 msgr2=0x7f6064149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061400640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606406f390 0x7f6064149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061400640 1 -- 172.21.15.67:0/1664567181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6064155a00 msgr2=0x7f606414a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061400640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6064155a00 0x7f606414a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061400640 1 -- 172.21.15.67:0/1664567181 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606414e2e0 con 0x7f60640d0970 2024-05-30T17:00:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.268+0000 7f6061e00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606406f390 0x7f6064149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.269+0000 7f6061400640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640d0970 0x7f606414dda0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f605000c4d0 tx=0x7f605000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.269+0000 7f605a000640 1 -- 172.21.15.67:0/1664567181 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050018070 con 0x7f60640d0970 2024-05-30T17:00:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.269+0000 7f605a000640 1 -- 172.21.15.67:0/1664567181 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6050007500 con 0x7f60640d0970 2024-05-30T17:00:14.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.269+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60641482f0 con 0x7f60640d0970 2024-05-30T17:00:14.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.269+0000 7f605a000640 1 -- 172.21.15.67:0/1664567181 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050004a20 con 0x7f60640d0970 2024-05-30T17:00:14.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.269+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60641488b0 con 0x7f60640d0970 2024-05-30T17:00:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.270+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60640689e0 con 0x7f60640d0970 2024-05-30T17:00:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.271+0000 7f605a000640 1 -- 172.21.15.67:0/1664567181 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f605001b070 con 0x7f60640d0970 2024-05-30T17:00:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.273+0000 7f605a000640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604406dcd0 0x7f6044070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.273+0000 7f605a000640 1 -- 172.21.15.67:0/1664567181 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f6050089ed0 con 0x7f60640d0970 2024-05-30T17:00:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.273+0000 7f6061e00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604406dcd0 0x7f6044070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.274+0000 7f6061e00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604406dcd0 0x7f6044070190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6054006fd0 tx=0x7f6054008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:14.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.277+0000 7f605a000640 1 -- 172.21.15.67:0/1664567181 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f606414e2e0 con 0x7f60640d0970 2024-05-30T17:00:14.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.556+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f606406f170 con 0x7f60640d0970 2024-05-30T17:00:14.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.557+0000 7f605a000640 1 -- 172.21.15.67:0/1664567181 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1731 (secure 0 0 0) 0x7f60500599a0 con 0x7f60640d0970 2024-05-30T17:00:14.557 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:00:14.558 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1007, 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:00:14.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:00:14.560 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:00:14.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:14.560 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:00:14.560 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 91, 2024-05-30T17:00:14.560 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:00:14.560 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:00:14.560 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:00:14.560 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:00:14.560 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:00:14.561 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1565, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 6443531634, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 13782089728, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 759312023552, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 56, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:00:14.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:14.563 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 6, 2024-05-30T17:00:14.563 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:00:14.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:00:14.563 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:00:14.563 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7398 2024-05-30T17:00:14.564 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:14.564 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:14.564 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:00:14.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:14.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:00:14.564 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:00:14.565 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:00:14.565 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:00:14.565 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:00:14.565 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:00:14.565 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:00:14.566 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:00:14.566 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:00:14.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:14.566 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:00:14.566 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 17, 2024-05-30T17:00:14.566 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T16:59:40.449642+0000", 2024-05-30T17:00:14.566 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:00:14.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:14.567 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": {} 2024-05-30T17:00:14.567 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604406dcd0 msgr2=0x7f6044070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604406dcd0 0x7f6044070190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6054006fd0 tx=0x7f6054008040 comp rx=0 tx=0).stop 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640d0970 msgr2=0x7f606414dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640d0970 0x7f606414dda0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f605000c4d0 tx=0x7f605000c990 comp rx=0 tx=0).stop 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 shutdown_connections 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f604406dcd0 0x7f6044070190 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6064155a00 0x7f606414a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640d0970 0x7f606414dda0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 --2- 172.21.15.67:0/1664567181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f606406f390 0x7f6064149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.559+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 >> 172.21.15.67:0/1664567181 conn(0x7f6064067ce0 msgr2=0x7f60640cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.560+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 shutdown_connections 2024-05-30T17:00:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.560+0000 7f6068c00640 1 -- 172.21.15.67:0/1664567181 wait complete. 2024-05-30T17:00:14.578 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-30T17:00:14.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:00:14.762 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:00:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.776+0000 7fae40000640 1 Processor -- start 2024-05-30T17:00:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.776+0000 7fae40000640 1 -- start start 2024-05-30T17:00:14.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.778+0000 7fae40000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fae3815e850 0x7fae380b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.778+0000 7fae40000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fae380cef30 0x7fae380d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.778+0000 7fae40000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae380d1f60 con 0x7fae380d1b80 2024-05-30T17:00:14.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.778+0000 7fae40000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fae3815ec30 con 0x7fae3815e850 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.778+0000 7fae40000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fae380b8000 con 0x7fae380cef30 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.778+0000 7fae3d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fae3815e850 0x7fae380b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.778+0000 7fae3d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fae3815e850 0x7fae380b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43764/0 (socket says 172.21.15.67:43764) 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.778+0000 7fae3d600640 1 -- 172.21.15.67:0/1134765634 learned_addr learned my addr 172.21.15.67:0/1134765634 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.778+0000 7fae3cc00640 1 --2- 172.21.15.67:0/1134765634 >> v2:172.21.15.169:3300/0 conn(0x7fae380cef30 0x7fae380d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.779+0000 7fae3d600640 1 -- 172.21.15.67:0/1134765634 >> v2:172.21.15.169:3300/0 conn(0x7fae380cef30 msgr2=0x7fae380d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.779+0000 7fae3d600640 1 --2- 172.21.15.67:0/1134765634 >> v2:172.21.15.169:3300/0 conn(0x7fae380cef30 0x7fae380d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.779+0000 7fae3d600640 1 -- 172.21.15.67:0/1134765634 >> v1:172.21.15.169:6789/0 conn(0x7fae380d1b80 legacy=0x7fae38162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.779+0000 7fae3d600640 1 -- 172.21.15.67:0/1134765634 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae380b82e0 con 0x7fae3815e850 2024-05-30T17:00:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.779+0000 7fae3cc00640 1 --2- 172.21.15.67:0/1134765634 >> v2:172.21.15.169:3300/0 conn(0x7fae380cef30 0x7fae380d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:14.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.780+0000 7fae3d600640 1 --2- 172.21.15.67:0/1134765634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 0x7fae380b77b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fae380cb730 tx=0x7fae2c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b67d6576eef8e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:14.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.780+0000 7fae37e00640 1 -- 172.21.15.67:0/1134765634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c037070 con 0x7fae3815e850 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.780+0000 7fae37e00640 1 -- 172.21.15.67:0/1134765634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2c032440 con 0x7fae3815e850 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.780+0000 7fae37e00640 1 -- 172.21.15.67:0/1134765634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c032810 con 0x7fae3815e850 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.781+0000 7fae40000640 1 -- 172.21.15.67:0/1134765634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 msgr2=0x7fae380b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.781+0000 7fae40000640 1 --2- 172.21.15.67:0/1134765634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 0x7fae380b77b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fae380cb730 tx=0x7fae2c02a1a0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.781+0000 7fae40000640 1 -- 172.21.15.67:0/1134765634 shutdown_connections 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.781+0000 7fae40000640 1 --2- 172.21.15.67:0/1134765634 >> v2:172.21.15.169:3300/0 conn(0x7fae380cef30 0x7fae380d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.781+0000 7fae40000640 1 --2- 172.21.15.67:0/1134765634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 0x7fae380b77b0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.781+0000 7fae40000640 1 -- 172.21.15.67:0/1134765634 >> 172.21.15.67:0/1134765634 conn(0x7fae38067aa0 msgr2=0x7fae38067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.781+0000 7fae40000640 1 -- 172.21.15.67:0/1134765634 shutdown_connections 2024-05-30T17:00:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.781+0000 7fae40000640 1 -- 172.21.15.67:0/1134765634 wait complete. 2024-05-30T17:00:14.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.783+0000 7fae40000640 1 Processor -- start 2024-05-30T17:00:14.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.783+0000 7fae40000640 1 -- start start 2024-05-30T17:00:14.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae40000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae380cef30 0x7fae3814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae40000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae380d1b80 0x7fae380d2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae40000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 0x7fae380d2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae40000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fae380b7cf0 con 0x7fae3815e850 2024-05-30T17:00:14.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae40000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fae380b88a0 con 0x7fae380cef30 2024-05-30T17:00:14.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae40000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fae380ba0b0 con 0x7fae380d1b80 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae3e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 0x7fae380d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae3e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 0x7fae380d2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43778/0 (socket says 172.21.15.67:43778) 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae3e000640 1 -- 172.21.15.67:0/2063279149 learned_addr learned my addr 172.21.15.67:0/2063279149 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.784+0000 7fae3cc00640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae380d1b80 0x7fae380d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae3d600640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae380cef30 0x7fae3814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae3e000640 1 -- 172.21.15.67:0/2063279149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae380d1b80 msgr2=0x7fae380d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae3e000640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae380d1b80 0x7fae380d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae3e000640 1 -- 172.21.15.67:0/2063279149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae380cef30 msgr2=0x7fae3814ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae3e000640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae380cef30 0x7fae3814ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae3e000640 1 -- 172.21.15.67:0/2063279149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae380d5190 con 0x7fae3815e850 2024-05-30T17:00:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae3cc00640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae380d1b80 0x7fae380d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:00:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae3e000640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 0x7fae380d2670 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7b0 tx=0x7fae3000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae35600640 1 -- 172.21.15.67:0/2063279149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30004b60 con 0x7fae3815e850 2024-05-30T17:00:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae35600640 1 -- 172.21.15.67:0/2063279149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae30004d00 con 0x7fae3815e850 2024-05-30T17:00:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.785+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae380d5470 con 0x7fae3815e850 2024-05-30T17:00:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.786+0000 7fae35600640 1 -- 172.21.15.67:0/2063279149 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3000aa90 con 0x7fae3815e850 2024-05-30T17:00:14.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.786+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae38160190 con 0x7fae3815e850 2024-05-30T17:00:14.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.787+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae380689e0 con 0x7fae3815e850 2024-05-30T17:00:14.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.789+0000 7fae35600640 1 -- 172.21.15.67:0/2063279149 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7fae3000ac30 con 0x7fae3815e850 2024-05-30T17:00:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.790+0000 7fae35600640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fae1806dca0 0x7fae18070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.790+0000 7fae35600640 1 -- 172.21.15.67:0/2063279149 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fae3008aaf0 con 0x7fae3815e850 2024-05-30T17:00:14.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.790+0000 7fae3d600640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fae1806dca0 0x7fae18070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:14.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.791+0000 7fae3d600640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fae1806dca0 0x7fae18070160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fae2c034000 tx=0x7fae2c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:14.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:14.794+0000 7fae35600640 1 -- 172.21.15.67:0/2063279149 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae30020070 con 0x7fae3815e850 2024-05-30T17:00:15.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.026+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae3815c1e0 con 0x7fae1806dca0 2024-05-30T17:00:15.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.038+0000 7fae35600640 1 -- 172.21.15.67:0/2063279149 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7fae3815c1e0 con 0x7fae1806dca0 2024-05-30T17:00:15.039 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:15.039 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:00:15.039 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:15.039 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sv1" 2024-05-30T17:00:15.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:15.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:15.040 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1clone1" 2024-05-30T17:00:15.040 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:15.040 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:00:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.041+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fae1806dca0 msgr2=0x7fae18070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.041+0000 7fae40000640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fae1806dca0 0x7fae18070160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fae2c034000 tx=0x7fae2c02e000 comp rx=0 tx=0).stop 2024-05-30T17:00:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.041+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 msgr2=0x7fae380d2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.041+0000 7fae40000640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 0x7fae380d2670 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7b0 tx=0x7fae3000cc70 comp rx=0 tx=0).stop 2024-05-30T17:00:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.041+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 shutdown_connections 2024-05-30T17:00:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.042+0000 7fae40000640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fae1806dca0 0x7fae18070160 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.042+0000 7fae40000640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae3815e850 0x7fae380d2670 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.042+0000 7fae40000640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae380d1b80 0x7fae380d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.042+0000 7fae40000640 1 --2- 172.21.15.67:0/2063279149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae380cef30 0x7fae3814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.042+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 >> 172.21.15.67:0/2063279149 conn(0x7fae38067aa0 msgr2=0x7fae3815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.042+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 shutdown_connections 2024-05-30T17:00:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.042+0000 7fae40000640 1 -- 172.21.15.67:0/2063279149 wait complete. 2024-05-30T17:00:15.060 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-30T17:00:15.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.275+0000 7f9e71200640 1 Processor -- start 2024-05-30T17:00:15.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.275+0000 7f9e71200640 1 -- start start 2024-05-30T17:00:15.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.277+0000 7f9e71200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e6c0af6f0 0x7f9e6c1623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.277+0000 7f9e71200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0d0b40 0x7f9e6c1428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.277+0000 7f9e71200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e6c162cb0 con 0x7f9e6c1480a0 2024-05-30T17:00:15.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.277+0000 7f9e71200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9e6c162f90 con 0x7f9e6c0af6f0 2024-05-30T17:00:15.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.277+0000 7f9e71200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c163270 con 0x7f9e6c0d0b40 2024-05-30T17:00:15.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.278+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0d0b40 0x7f9e6c1428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.278+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0d0b40 0x7f9e6c1428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:55360/0 (socket says 172.21.15.67:55360) 2024-05-30T17:00:15.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.278+0000 7f9e6be00640 1 -- 172.21.15.67:0/996510438 learned_addr learned my addr 172.21.15.67:0/996510438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:15.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.278+0000 7f9e6b400640 1 --2- 172.21.15.67:0/996510438 >> v2:172.21.15.169:3300/0 conn(0x7f9e6c0af6f0 0x7f9e6c1623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.278+0000 7f9e6be00640 1 -- 172.21.15.67:0/996510438 >> v2:172.21.15.169:3300/0 conn(0x7f9e6c0af6f0 msgr2=0x7f9e6c1623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:15.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.278+0000 7f9e6be00640 1 --2- 172.21.15.67:0/996510438 >> v2:172.21.15.169:3300/0 conn(0x7f9e6c0af6f0 0x7f9e6c1623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.278+0000 7f9e6be00640 1 -- 172.21.15.67:0/996510438 >> v1:172.21.15.169:6789/0 conn(0x7f9e6c1480a0 legacy=0x7f9e6c0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.278+0000 7f9e6be00640 1 -- 172.21.15.67:0/996510438 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c1420d0 con 0x7f9e6c0d0b40 2024-05-30T17:00:15.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.279+0000 7f9e6be00640 1 --2- 172.21.15.67:0/996510438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0d0b40 0x7f9e6c1428d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009e50 tx=0x7f9e6402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7c2cb0d2c6ff2d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:15.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.279+0000 7f9e6aa00640 1 -- 172.21.15.67:0/996510438 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64036070 con 0x7f9e6c0d0b40 2024-05-30T17:00:15.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.279+0000 7f9e6aa00640 1 -- 172.21.15.67:0/996510438 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e64031600 con 0x7f9e6c0d0b40 2024-05-30T17:00:15.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.279+0000 7f9e6aa00640 1 -- 172.21.15.67:0/996510438 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640319d0 con 0x7f9e6c0d0b40 2024-05-30T17:00:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.280+0000 7f9e71200640 1 -- 172.21.15.67:0/996510438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0d0b40 msgr2=0x7f9e6c1428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.280+0000 7f9e71200640 1 --2- 172.21.15.67:0/996510438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0d0b40 0x7f9e6c1428d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009e50 tx=0x7f9e6402a710 comp rx=0 tx=0).stop 2024-05-30T17:00:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.280+0000 7f9e71200640 1 -- 172.21.15.67:0/996510438 shutdown_connections 2024-05-30T17:00:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.280+0000 7f9e71200640 1 --2- 172.21.15.67:0/996510438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0d0b40 0x7f9e6c1428d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.280+0000 7f9e71200640 1 --2- 172.21.15.67:0/996510438 >> v2:172.21.15.169:3300/0 conn(0x7f9e6c0af6f0 0x7f9e6c1623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.280+0000 7f9e71200640 1 -- 172.21.15.67:0/996510438 >> 172.21.15.67:0/996510438 conn(0x7f9e6c06bae0 msgr2=0x7f9e6c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.280+0000 7f9e71200640 1 -- 172.21.15.67:0/996510438 shutdown_connections 2024-05-30T17:00:15.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.280+0000 7f9e71200640 1 -- 172.21.15.67:0/996510438 wait complete. 2024-05-30T17:00:15.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.282+0000 7f9e71200640 1 Processor -- start 2024-05-30T17:00:15.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.282+0000 7f9e71200640 1 -- start start 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.283+0000 7f9e71200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0af6f0 0x7f9e6c161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.283+0000 7f9e71200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e6c0d0b40 0x7f9e6c0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.283+0000 7f9e71200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e6c1480a0 0x7f9e6c0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.283+0000 7f9e71200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c162a70 con 0x7f9e6c1480a0 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.283+0000 7f9e71200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1628f0 con 0x7f9e6c0d0b40 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.283+0000 7f9e71200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c0afad0 con 0x7f9e6c0af6f0 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.283+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e6c1480a0 0x7f9e6c0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.283+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e6c1480a0 0x7f9e6c0b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43790/0 (socket says 172.21.15.67:43790) 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e6be00640 1 -- 172.21.15.67:0/2189682693 learned_addr learned my addr 172.21.15.67:0/2189682693 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e6b400640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0af6f0 0x7f9e6c161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e63e00640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e6c0d0b40 0x7f9e6c0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e6be00640 1 -- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0af6f0 msgr2=0x7f9e6c161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e6be00640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0af6f0 0x7f9e6c161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e6be00640 1 -- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e6c0d0b40 msgr2=0x7f9e6c0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e6be00640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e6c0d0b40 0x7f9e6c0ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e6be00640 1 -- 172.21.15.67:0/2189682693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c161860 con 0x7f9e6c1480a0 2024-05-30T17:00:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e6b400640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0af6f0 0x7f9e6c161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e63e00640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e6c0d0b40 0x7f9e6c0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.284+0000 7f9e6be00640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e6c1480a0 0x7f9e6c0b0e20 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9e64034040 tx=0x7f9e64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.285+0000 7f9e63400640 1 -- 172.21.15.67:0/2189682693 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6403f020 con 0x7f9e6c1480a0 2024-05-30T17:00:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.285+0000 7f9e63400640 1 -- 172.21.15.67:0/2189682693 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e64037040 con 0x7f9e6c1480a0 2024-05-30T17:00:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.285+0000 7f9e63400640 1 -- 172.21.15.67:0/2189682693 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64036070 con 0x7f9e6c1480a0 2024-05-30T17:00:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.285+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c0cb050 con 0x7f9e6c1480a0 2024-05-30T17:00:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.285+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c0cb610 con 0x7f9e6c1480a0 2024-05-30T17:00:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.286+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6c0689e0 con 0x7f9e6c1480a0 2024-05-30T17:00:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.287+0000 7f9e63400640 1 -- 172.21.15.67:0/2189682693 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f9e64042020 con 0x7f9e6c1480a0 2024-05-30T17:00:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.288+0000 7f9e63400640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e4806dcd0 0x7f9e48070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.288+0000 7f9e63400640 1 -- 172.21.15.67:0/2189682693 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f9e640aaef0 con 0x7f9e6c1480a0 2024-05-30T17:00:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.288+0000 7f9e6b400640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e4806dcd0 0x7f9e48070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.289+0000 7f9e6b400640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e4806dcd0 0x7f9e48070190 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009a20 tx=0x7f9e5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:15.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.292+0000 7f9e63400640 1 -- 172.21.15.67:0/2189682693 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e64049030 con 0x7f9e6c1480a0 2024-05-30T17:00:15.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.520+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e6c0c9e50 con 0x7f9e4806dcd0 2024-05-30T17:00:15.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.550+0000 7f9e63400640 1 -- 172.21.15.67:0/2189682693 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9e6c0c9e50 con 0x7f9e4806dcd0 2024-05-30T17:00:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:15.552 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:00:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1" 2024-05-30T17:00:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:15.552 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:00:15.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e4806dcd0 msgr2=0x7f9e48070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e4806dcd0 0x7f9e48070190 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009a20 tx=0x7f9e5c008040 comp rx=0 tx=0).stop 2024-05-30T17:00:15.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e6c1480a0 msgr2=0x7f9e6c0b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e6c1480a0 0x7f9e6c0b0e20 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9e64034040 tx=0x7f9e64004a40 comp rx=0 tx=0).stop 2024-05-30T17:00:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 shutdown_connections 2024-05-30T17:00:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e4806dcd0 0x7f9e48070190 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e6c1480a0 0x7f9e6c0b0e20 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e6c0d0b40 0x7f9e6c0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 --2- 172.21.15.67:0/2189682693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e6c0af6f0 0x7f9e6c161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 >> 172.21.15.67:0/2189682693 conn(0x7f9e6c06bae0 msgr2=0x7f9e6c06dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:15.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 shutdown_connections 2024-05-30T17:00:15.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.553+0000 7f9e71200640 1 -- 172.21.15.67:0/2189682693 wait complete. 2024-05-30T17:00:15.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-30T17:00:15.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.780+0000 7ff22ec00640 1 Processor -- start 2024-05-30T17:00:15.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.780+0000 7ff22ec00640 1 -- start start 2024-05-30T17:00:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.782+0000 7ff22ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280cae20 0x7ff228151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.782+0000 7ff22ec00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff228152780 con 0x7ff2280caa40 2024-05-30T17:00:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.782+0000 7ff22ec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2280b3e80 con 0x7ff2281522f0 2024-05-30T17:00:15.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.782+0000 7ff22ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff2280d0b30 con 0x7ff2280cae20 2024-05-30T17:00:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.782+0000 7ff22d800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff2280caa40 0x7ff2280ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48994/0 (socket says 172.21.15.67:48994) 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.782+0000 7ff22d800640 1 -- 172.21.15.67:0/2141667640 learned_addr learned my addr 172.21.15.67:0/2141667640 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff22ce00640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280cae20 0x7ff228151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1084232750 0 0) 0x7ff228152780 con 0x7ff2280caa40 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff210003520 con 0x7ff2280caa40 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3118997288 0 0) 0x7ff210003520 con 0x7ff2280caa40 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280cae20 msgr2=0x7ff228151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280cae20 0x7ff228151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 >> v1:172.21.15.169:6789/0 conn(0x7ff2281522f0 legacy=0x7ff2280cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2280cde80 con 0x7ff2280caa40 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff21c001670 con 0x7ff2280caa40 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 >> v1:172.21.15.67:6789/0 conn(0x7ff2280caa40 legacy=0x7ff2280ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2100039e0 0x7ff210003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280cae20 0x7ff210004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff210004d40 0x7ff2100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff2280d0b30 con 0x7ff210004d40 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff21000bb50 con 0x7ff2100039e0 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.783+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff21000bd50 con 0x7ff2280cae20 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.784+0000 7ff22e200640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff210004d40 0x7ff2100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.784+0000 7ff22ce00640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280cae20 0x7ff210004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.784+0000 7ff22d800640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2100039e0 0x7ff210003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.784+0000 7ff22e200640 1 -- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280cae20 msgr2=0x7ff210004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.784+0000 7ff22e200640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280cae20 0x7ff210004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.784+0000 7ff22e200640 1 -- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2100039e0 msgr2=0x7ff210003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.784+0000 7ff22e200640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2100039e0 0x7ff210003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.784+0000 7ff22e200640 1 -- 172.21.15.67:0/2141667640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff210003520 con 0x7ff210004d40 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.785+0000 7ff22e200640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff210004d40 0x7ff2100051c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff220001e50 tx=0x7ff210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fc76211f6e6963 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220001a70 con 0x7ff210004d40 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff220003970 con 0x7ff210004d40 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff227e00640 1 -- 172.21.15.67:0/2141667640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220003d60 con 0x7ff210004d40 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff22ec00640 1 -- 172.21.15.67:0/2141667640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff210004d40 msgr2=0x7ff2100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff22ec00640 1 --2- 172.21.15.67:0/2141667640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff210004d40 0x7ff2100051c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff220001e50 tx=0x7ff210005830 comp rx=0 tx=0).stop 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff22e200640 1 -- 172.21.15.67:0/2141667640 reap_dead start 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff22ec00640 1 -- 172.21.15.67:0/2141667640 shutdown_connections 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff22ec00640 1 -- 172.21.15.67:0/2141667640 >> 172.21.15.67:0/2141667640 conn(0x7ff22806b900 msgr2=0x7ff22806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff22ec00640 1 -- 172.21.15.67:0/2141667640 shutdown_connections 2024-05-30T17:00:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.786+0000 7ff22ec00640 1 -- 172.21.15.67:0/2141667640 wait complete. 2024-05-30T17:00:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.788+0000 7ff22ec00640 1 Processor -- start 2024-05-30T17:00:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.788+0000 7ff22ec00640 1 -- start start 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.789+0000 7ff22ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d40 0x7ff228155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.789+0000 7ff22ec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2280ad130 0x7ff2280ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.789+0000 7ff22ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280b0e50 0x7ff2280b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.789+0000 7ff22ec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff2280cf070 con 0x7ff2280ad130 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.789+0000 7ff22ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff2280b4160 con 0x7ff210004d40 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.789+0000 7ff22ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff2280ce440 con 0x7ff2280b0e50 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2280ad130 0x7ff2280ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2280ad130 0x7ff2280ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43802/0 (socket says 172.21.15.67:43802) 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22ce00640 1 -- 172.21.15.67:0/190471075 learned_addr learned my addr 172.21.15.67:0/190471075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22e200640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280b0e50 0x7ff2280b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22d800640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d40 0x7ff228155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22ce00640 1 -- 172.21.15.67:0/190471075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280b0e50 msgr2=0x7ff2280b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22ce00640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280b0e50 0x7ff2280b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22ce00640 1 -- 172.21.15.67:0/190471075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d40 msgr2=0x7ff228155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22ce00640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d40 0x7ff228155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22ce00640 1 -- 172.21.15.67:0/190471075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2280adbd0 con 0x7ff2280ad130 2024-05-30T17:00:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.790+0000 7ff22e200640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280b0e50 0x7ff2280b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.791+0000 7ff22ce00640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2280ad130 0x7ff2280ad5b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff214011600 tx=0x7ff214011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.791+0000 7ff225600640 1 -- 172.21.15.67:0/190471075 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21400f920 con 0x7ff2280ad130 2024-05-30T17:00:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.791+0000 7ff225600640 1 -- 172.21.15.67:0/190471075 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21400fac0 con 0x7ff2280ad130 2024-05-30T17:00:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.791+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2280adeb0 con 0x7ff2280ad130 2024-05-30T17:00:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.791+0000 7ff225600640 1 -- 172.21.15.67:0/190471075 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21400fe70 con 0x7ff2280ad130 2024-05-30T17:00:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.791+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22814f860 con 0x7ff2280ad130 2024-05-30T17:00:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.792+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2280b4310 con 0x7ff2280ad130 2024-05-30T17:00:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.793+0000 7ff225600640 1 -- 172.21.15.67:0/190471075 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7ff214015460 con 0x7ff2280ad130 2024-05-30T17:00:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.794+0000 7ff225600640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff20806dca0 0x7ff208070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.795+0000 7ff225600640 1 -- 172.21.15.67:0/190471075 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7ff21408b590 con 0x7ff2280ad130 2024-05-30T17:00:15.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.795+0000 7ff22d800640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff20806dca0 0x7ff208070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.796+0000 7ff22d800640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff20806dca0 0x7ff208070160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0057f0 tx=0x7ff21c000ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:15.799+0000 7ff225600640 1 -- 172.21.15.67:0/190471075 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff2280b4310 con 0x7ff2280ad130 2024-05-30T17:00:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.036+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2280689e0 con 0x7ff20806dca0 2024-05-30T17:00:16.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.097+0000 7ff225600640 1 -- 172.21.15.67:0/190471075 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2280689e0 con 0x7ff20806dca0 2024-05-30T17:00:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:16.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff20806dca0 msgr2=0x7ff208070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff20806dca0 0x7ff208070160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0057f0 tx=0x7ff21c000ed0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2280ad130 msgr2=0x7ff2280ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2280ad130 0x7ff2280ad5b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff214011600 tx=0x7ff214011ac0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 shutdown_connections 2024-05-30T17:00:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff20806dca0 0x7ff208070160 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2280b0e50 0x7ff2280b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2280ad130 0x7ff2280ad5b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 --2- 172.21.15.67:0/190471075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff210004d40 0x7ff228155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 >> 172.21.15.67:0/190471075 conn(0x7ff22806b900 msgr2=0x7ff2280ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.100+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 shutdown_connections 2024-05-30T17:00:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.101+0000 7ff22ec00640 1 -- 172.21.15.67:0/190471075 wait complete. 2024-05-30T17:00:16.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-30T17:00:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.335+0000 7faa27000640 1 Processor -- start 2024-05-30T17:00:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.335+0000 7faa27000640 1 -- start start 2024-05-30T17:00:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.337+0000 7faa27000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa200d0640 0x7faa2015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.337+0000 7faa27000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faa200d0a20 0x7faa200cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.337+0000 7faa27000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa2015cb10 con 0x7faa200b32c0 2024-05-30T17:00:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.337+0000 7faa27000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7faa200adbb0 con 0x7faa200d0640 2024-05-30T17:00:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.337+0000 7faa27000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faa2015db10 con 0x7faa200d0a20 2024-05-30T17:00:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.337+0000 7faa1fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa200d0640 0x7faa2015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.337+0000 7faa1fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa200d0640 0x7faa2015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43804/0 (socket says 172.21.15.67:43804) 2024-05-30T17:00:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.337+0000 7faa1fe00640 1 -- 172.21.15.67:0/2570775556 learned_addr learned my addr 172.21.15.67:0/2570775556 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.338+0000 7faa1fe00640 1 -- 172.21.15.67:0/2570775556 >> v2:172.21.15.169:3300/0 conn(0x7faa200d0a20 msgr2=0x7faa200cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.338+0000 7faa1f400640 1 --2- 172.21.15.67:0/2570775556 >> v2:172.21.15.169:3300/0 conn(0x7faa200d0a20 0x7faa200cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.338+0000 7faa1fe00640 1 --2- 172.21.15.67:0/2570775556 >> v2:172.21.15.169:3300/0 conn(0x7faa200d0a20 0x7faa200cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.338+0000 7faa1fe00640 1 -- 172.21.15.67:0/2570775556 >> v1:172.21.15.169:6789/0 conn(0x7faa200b32c0 legacy=0x7faa200b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.338+0000 7faa1fe00640 1 -- 172.21.15.67:0/2570775556 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa200ad5f0 con 0x7faa200d0640 2024-05-30T17:00:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.338+0000 7faa1f400640 1 --2- 172.21.15.67:0/2570775556 >> v2:172.21.15.169:3300/0 conn(0x7faa200d0a20 0x7faa200cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.339+0000 7faa1fe00640 1 --2- 172.21.15.67:0/2570775556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 0x7faa2015d5d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7faa20074120 tx=0x7faa1002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb28381c4e6ffac server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.340+0000 7faa1ea00640 1 -- 172.21.15.67:0/2570775556 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10036070 con 0x7faa200d0640 2024-05-30T17:00:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.340+0000 7faa1ea00640 1 -- 172.21.15.67:0/2570775556 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa10031600 con 0x7faa200d0640 2024-05-30T17:00:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.340+0000 7faa1ea00640 1 -- 172.21.15.67:0/2570775556 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1002ae60 con 0x7faa200d0640 2024-05-30T17:00:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.340+0000 7faa27000640 1 -- 172.21.15.67:0/2570775556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 msgr2=0x7faa2015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.340+0000 7faa27000640 1 --2- 172.21.15.67:0/2570775556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 0x7faa2015d5d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7faa20074120 tx=0x7faa1002a680 comp rx=0 tx=0).stop 2024-05-30T17:00:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.341+0000 7faa27000640 1 -- 172.21.15.67:0/2570775556 shutdown_connections 2024-05-30T17:00:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.341+0000 7faa27000640 1 --2- 172.21.15.67:0/2570775556 >> v2:172.21.15.169:3300/0 conn(0x7faa200d0a20 0x7faa200cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.341+0000 7faa27000640 1 --2- 172.21.15.67:0/2570775556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 0x7faa2015d5d0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.341+0000 7faa27000640 1 -- 172.21.15.67:0/2570775556 >> 172.21.15.67:0/2570775556 conn(0x7faa2006bd20 msgr2=0x7faa2006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:16.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.341+0000 7faa27000640 1 -- 172.21.15.67:0/2570775556 shutdown_connections 2024-05-30T17:00:16.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.341+0000 7faa27000640 1 -- 172.21.15.67:0/2570775556 wait complete. 2024-05-30T17:00:16.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.342+0000 7faa27000640 1 Processor -- start 2024-05-30T17:00:16.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.343+0000 7faa27000640 1 -- start start 2024-05-30T17:00:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.343+0000 7faa27000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa200b32c0 0x7faa201468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.343+0000 7faa27000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 0x7faa201f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa27000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa200d0a20 0x7faa200b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa27000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faa2015d0b0 con 0x7faa200d0640 2024-05-30T17:00:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa27000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa2015cdd0 con 0x7faa200d0a20 2024-05-30T17:00:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa27000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faa200b3e40 con 0x7faa200b32c0 2024-05-30T17:00:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa1f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 0x7faa201f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa1f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 0x7faa201f3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43818/0 (socket says 172.21.15.67:43818) 2024-05-30T17:00:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa1f400640 1 -- 172.21.15.67:0/1511873970 learned_addr learned my addr 172.21.15.67:0/1511873970 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa1fe00640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa200b32c0 0x7faa201468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa25000640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa200d0a20 0x7faa200b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa1f400640 1 -- 172.21.15.67:0/1511873970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa200b32c0 msgr2=0x7faa201468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa1f400640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa200b32c0 0x7faa201468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa1f400640 1 -- 172.21.15.67:0/1511873970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa200d0a20 msgr2=0x7faa200b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa1f400640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa200d0a20 0x7faa200b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa1f400640 1 -- 172.21.15.67:0/1511873970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa20146de0 con 0x7faa200d0640 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.344+0000 7faa25000640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa200d0a20 0x7faa200b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.345+0000 7faa1f400640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 0x7faa201f3b50 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c7f0 tx=0x7faa0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.345+0000 7faa17e00640 1 -- 172.21.15.67:0/1511873970 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c015070 con 0x7faa200d0640 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.345+0000 7faa17e00640 1 -- 172.21.15.67:0/1511873970 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa0c007820 con 0x7faa200d0640 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.345+0000 7faa17e00640 1 -- 172.21.15.67:0/1511873970 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c007bd0 con 0x7faa200d0640 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.345+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa20156440 con 0x7faa200d0640 2024-05-30T17:00:16.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.345+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa201f7d60 con 0x7faa200d0640 2024-05-30T17:00:16.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.346+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa200689e0 con 0x7faa200d0640 2024-05-30T17:00:16.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.347+0000 7faa17e00640 1 -- 172.21.15.67:0/1511873970 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7faa0c018070 con 0x7faa200d0640 2024-05-30T17:00:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.349+0000 7faa17e00640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa0006dca0 0x7faa00070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.349+0000 7faa17e00640 1 -- 172.21.15.67:0/1511873970 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7faa0c08acc0 con 0x7faa200d0640 2024-05-30T17:00:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.349+0000 7faa1fe00640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa0006dca0 0x7faa00070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.350+0000 7faa1fe00640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa0006dca0 0x7faa00070160 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faa10033000 tx=0x7faa10009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:16.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.353+0000 7faa17e00640 1 -- 172.21.15.67:0/1511873970 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa0c05a790 con 0x7faa200d0640 2024-05-30T17:00:16.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.590+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7faa2015f9f0 con 0x7faa0006dca0 2024-05-30T17:00:16.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.640+0000 7faa17e00640 1 -- 172.21.15.67:0/1511873970 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa2015f9f0 con 0x7faa0006dca0 2024-05-30T17:00:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa0006dca0 msgr2=0x7faa00070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa0006dca0 0x7faa00070160 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faa10033000 tx=0x7faa10009000 comp rx=0 tx=0).stop 2024-05-30T17:00:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 msgr2=0x7faa201f3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 0x7faa201f3b50 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c7f0 tx=0x7faa0c00ccb0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 shutdown_connections 2024-05-30T17:00:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faa0006dca0 0x7faa00070160 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa200d0a20 0x7faa200b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa200d0640 0x7faa201f3b50 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 --2- 172.21.15.67:0/1511873970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa200b32c0 0x7faa201468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 >> 172.21.15.67:0/1511873970 conn(0x7faa2006bd20 msgr2=0x7faa200b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 shutdown_connections 2024-05-30T17:00:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.644+0000 7faa27000640 1 -- 172.21.15.67:0/1511873970 wait complete. 2024-05-30T17:00:16.651 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.649+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.651 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.649+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.651 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.649+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.651 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.649+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.651 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.649+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.666 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-30T17:00:16.682 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.681+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.682 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.681+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.683 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.681+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.683 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.681+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.683 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:16.681+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.849+0000 7f7212e00640 1 Processor -- start 2024-05-30T17:00:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.849+0000 7f7212e00640 1 -- start start 2024-05-30T17:00:16.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.851+0000 7f7212e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f720c0ae290 0x7f720c0af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.851+0000 7f7212e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f720c06b900 0x7f720c06bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.851+0000 7f7212e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f720c0ced00 con 0x7f720c146120 2024-05-30T17:00:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.851+0000 7f7212e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f720c06c2b0 con 0x7f720c0ae290 2024-05-30T17:00:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.851+0000 7f7212e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f720c0afed0 con 0x7f720c06b900 2024-05-30T17:00:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.851+0000 7f7211000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f720c0ae290 0x7f720c0af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.851+0000 7f7211a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f720c146120 0x7f720c0ce5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49000/0 (socket says 172.21.15.67:49000) 2024-05-30T17:00:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.851+0000 7f7211000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f720c0ae290 0x7f720c0af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43834/0 (socket says 172.21.15.67:43834) 2024-05-30T17:00:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.851+0000 7f7211a00640 1 -- 172.21.15.67:0/1230092227 learned_addr learned my addr 172.21.15.67:0/1230092227 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.852+0000 7f7212400640 1 --2- 172.21.15.67:0/1230092227 >> v2:172.21.15.169:3300/0 conn(0x7f720c06b900 0x7f720c06bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.852+0000 7f7211000640 1 -- 172.21.15.67:0/1230092227 >> v2:172.21.15.169:3300/0 conn(0x7f720c06b900 msgr2=0x7f720c06bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.852+0000 7f7211000640 1 --2- 172.21.15.67:0/1230092227 >> v2:172.21.15.169:3300/0 conn(0x7f720c06b900 0x7f720c06bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.852+0000 7f7211000640 1 -- 172.21.15.67:0/1230092227 >> v1:172.21.15.67:6789/0 conn(0x7f720c146120 legacy=0x7f720c0ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.852+0000 7f7211000640 1 -- 172.21.15.67:0/1230092227 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c0cefd0 con 0x7f720c0ae290 2024-05-30T17:00:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.852+0000 7f7212400640 1 --2- 172.21.15.67:0/1230092227 >> v2:172.21.15.169:3300/0 conn(0x7f720c06b900 0x7f720c06bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.852+0000 7f720be00640 1 -- 172.21.15.67:0/1230092227 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509823601 0 0) 0x7f720c0ced00 con 0x7f720c146120 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.853+0000 7f7211000640 1 --2- 172.21.15.67:0/1230092227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 0x7f720c0af990 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f71f80097a0 tx=0x7f71f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884cedd1c3f14a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.853+0000 7f720be00640 1 -- 172.21.15.67:0/1230092227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8037070 con 0x7f720c0ae290 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.853+0000 7f720be00640 1 -- 172.21.15.67:0/1230092227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f8032440 con 0x7f720c0ae290 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.853+0000 7f720be00640 1 -- 172.21.15.67:0/1230092227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8032810 con 0x7f720c0ae290 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.854+0000 7f7212e00640 1 -- 172.21.15.67:0/1230092227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 msgr2=0x7f720c0af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.854+0000 7f7212e00640 1 --2- 172.21.15.67:0/1230092227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 0x7f720c0af990 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f71f80097a0 tx=0x7f71f802a1e0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.854+0000 7f7212e00640 1 -- 172.21.15.67:0/1230092227 shutdown_connections 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.854+0000 7f7212e00640 1 --2- 172.21.15.67:0/1230092227 >> v2:172.21.15.169:3300/0 conn(0x7f720c06b900 0x7f720c06bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.854+0000 7f7212e00640 1 --2- 172.21.15.67:0/1230092227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 0x7f720c0af990 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.854+0000 7f7212e00640 1 -- 172.21.15.67:0/1230092227 >> 172.21.15.67:0/1230092227 conn(0x7f720c0b66f0 msgr2=0x7f720c0b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.854+0000 7f7212e00640 1 -- 172.21.15.67:0/1230092227 shutdown_connections 2024-05-30T17:00:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.854+0000 7f7212e00640 1 -- 172.21.15.67:0/1230092227 wait complete. 2024-05-30T17:00:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.856+0000 7f7212e00640 1 Processor -- start 2024-05-30T17:00:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.856+0000 7f7212e00640 1 -- start start 2024-05-30T17:00:16.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7212e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720c06b900 0x7f720c13ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7212e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 0x7f720c135df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7212e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f720c146120 0x7f720c0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7212e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f720c0cf870 con 0x7f720c0ae290 2024-05-30T17:00:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7212e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f720c0cf590 con 0x7f720c146120 2024-05-30T17:00:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7212e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f720c0cfb50 con 0x7f720c06b900 2024-05-30T17:00:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7211000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 0x7f720c135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7211000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 0x7f720c135df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43848/0 (socket says 172.21.15.67:43848) 2024-05-30T17:00:16.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7211000640 1 -- 172.21.15.67:0/2004603917 learned_addr learned my addr 172.21.15.67:0/2004603917 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7211a00640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720c06b900 0x7f720c13ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7211000640 1 -- 172.21.15.67:0/2004603917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720c06b900 msgr2=0x7f720c13ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7211000640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720c06b900 0x7f720c13ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7211000640 1 -- 172.21.15.67:0/2004603917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f720c146120 msgr2=0x7f720c0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7211000640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f720c146120 0x7f720c0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:16.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.857+0000 7f7211000640 1 -- 172.21.15.67:0/2004603917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c140470 con 0x7f720c0ae290 2024-05-30T17:00:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.858+0000 7f7211000640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 0x7f720c135df0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f71f80097a0 tx=0x7f71f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.858+0000 7f7209600640 1 -- 172.21.15.67:0/2004603917 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8037050 con 0x7f720c0ae290 2024-05-30T17:00:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.858+0000 7f7209600640 1 -- 172.21.15.67:0/2004603917 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f8038040 con 0x7f720c0ae290 2024-05-30T17:00:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.858+0000 7f7209600640 1 -- 172.21.15.67:0/2004603917 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8004490 con 0x7f720c0ae290 2024-05-30T17:00:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.858+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720c136840 con 0x7f720c0ae290 2024-05-30T17:00:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.858+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720c136e00 con 0x7f720c0ae290 2024-05-30T17:00:16.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.860+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720c0690f0 con 0x7f720c0ae290 2024-05-30T17:00:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.861+0000 7f7209600640 1 -- 172.21.15.67:0/2004603917 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f71f8030980 con 0x7f720c0ae290 2024-05-30T17:00:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.861+0000 7f7209600640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71ec06dca0 0x7f71ec070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.862+0000 7f7209600640 1 -- 172.21.15.67:0/2004603917 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f71f80ab7a0 con 0x7f720c0ae290 2024-05-30T17:00:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.862+0000 7f7211a00640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71ec06dca0 0x7f71ec070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.862+0000 7f7211a00640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71ec06dca0 0x7f71ec070160 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7200000f60 tx=0x7f7200009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:16.871+0000 7f7209600640 1 -- 172.21.15.67:0/2004603917 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71f807b3f0 con 0x7f720c0ae290 2024-05-30T17:00:17.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.101+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f720c143e10 con 0x7f71ec06dca0 2024-05-30T17:00:17.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.138+0000 7f7209600640 1 -- 172.21.15.67:0/2004603917 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f720c143e10 con 0x7f71ec06dca0 2024-05-30T17:00:17.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.140+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71ec06dca0 msgr2=0x7f71ec070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:17.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.140+0000 7f7212e00640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71ec06dca0 0x7f71ec070160 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7200000f60 tx=0x7f7200009040 comp rx=0 tx=0).stop 2024-05-30T17:00:17.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.140+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 msgr2=0x7f720c135df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:17.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.140+0000 7f7212e00640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 0x7f720c135df0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f71f80097a0 tx=0x7f71f80046c0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.141+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 shutdown_connections 2024-05-30T17:00:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.141+0000 7f7212e00640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f71ec06dca0 0x7f71ec070160 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.141+0000 7f7212e00640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f720c146120 0x7f720c0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.141+0000 7f7212e00640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f720c0ae290 0x7f720c135df0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.141+0000 7f7212e00640 1 --2- 172.21.15.67:0/2004603917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720c06b900 0x7f720c13ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.141+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 >> 172.21.15.67:0/2004603917 conn(0x7f720c0b66f0 msgr2=0x7f720c141d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.141+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 shutdown_connections 2024-05-30T17:00:17.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.141+0000 7f7212e00640 1 -- 172.21.15.67:0/2004603917 wait complete. 2024-05-30T17:00:17.161 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-30T17:00:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-30T17:00:17.322 INFO:teuthology.orchestra.run.smithi067.stdout:2 2024-05-30T17:00:17.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T17:00:17.332 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T17:00:17.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:00:17.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:00:17.569 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T17:00:17.569 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:00:17.570 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:00:17.570 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:17.630 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:00:17.631 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:17.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T17:00:17.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.895+0000 7f6aa5200640 1 Processor -- start 2024-05-30T17:00:17.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.895+0000 7f6aa5200640 1 -- start start 2024-05-30T17:00:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.896+0000 7f6aa5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa0156b30 0x7f6aa00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.896+0000 7f6aa5200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6aa006ee40 0x7f6aa00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6aa5200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa0156f10 con 0x7f6aa006f2c0 2024-05-30T17:00:17.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6aa5200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa006dda0 con 0x7f6aa006ee40 2024-05-30T17:00:17.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6aa5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa006e080 con 0x7f6aa0156b30 2024-05-30T17:00:17.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6a96a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6aa006ee40 0x7f6aa00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:17.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6a96a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6aa006ee40 0x7f6aa00ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43860/0 (socket says 172.21.15.67:43860) 2024-05-30T17:00:17.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6a96a00640 1 -- 172.21.15.67:0/715520959 learned_addr learned my addr 172.21.15.67:0/715520959 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:17.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6a9fe00640 1 --1- 172.21.15.67:0/715520959 >> v1:172.21.15.67:6789/0 conn(0x7f6aa006f2c0 0x7f6aa00b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49012/0 (socket says 172.21.15.67:49012) 2024-05-30T17:00:17.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6a96a00640 1 -- 172.21.15.67:0/715520959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa0156b30 msgr2=0x7f6aa00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:17.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6a9f400640 1 --2- 172.21.15.67:0/715520959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa0156b30 0x7f6aa00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:17.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6a96a00640 1 --2- 172.21.15.67:0/715520959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa0156b30 0x7f6aa00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.897+0000 7f6a96a00640 1 -- 172.21.15.67:0/715520959 >> v1:172.21.15.67:6789/0 conn(0x7f6aa006f2c0 legacy=0x7f6aa00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:17.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.898+0000 7f6a96a00640 1 -- 172.21.15.67:0/715520959 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa006e360 con 0x7f6aa006ee40 2024-05-30T17:00:17.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.898+0000 7f6a9f400640 1 --2- 172.21.15.67:0/715520959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa0156b30 0x7f6aa00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:17.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.899+0000 7f6a96a00640 1 --2- 172.21.15.67:0/715520959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006ee40 0x7f6aa00ad230 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009870 tx=0x7f6a8802a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519bb39bc3ec1967 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:17.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.899+0000 7f6a9ea00640 1 -- 172.21.15.67:0/715520959 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88037070 con 0x7f6aa006ee40 2024-05-30T17:00:17.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.899+0000 7f6a9ea00640 1 -- 172.21.15.67:0/715520959 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8802ad70 con 0x7f6aa006ee40 2024-05-30T17:00:17.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.899+0000 7f6a9ea00640 1 -- 172.21.15.67:0/715520959 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88032660 con 0x7f6aa006ee40 2024-05-30T17:00:17.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.899+0000 7f6aa5200640 1 -- 172.21.15.67:0/715520959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006ee40 msgr2=0x7f6aa00ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:17.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.899+0000 7f6aa5200640 1 --2- 172.21.15.67:0/715520959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006ee40 0x7f6aa00ad230 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009870 tx=0x7f6a8802a0e0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.900+0000 7f6aa5200640 1 -- 172.21.15.67:0/715520959 shutdown_connections 2024-05-30T17:00:17.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.900+0000 7f6aa5200640 1 --2- 172.21.15.67:0/715520959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006ee40 0x7f6aa00ad230 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.900+0000 7f6aa5200640 1 --2- 172.21.15.67:0/715520959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa0156b30 0x7f6aa00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.900+0000 7f6aa5200640 1 -- 172.21.15.67:0/715520959 >> 172.21.15.67:0/715520959 conn(0x7f6aa00b1fc0 msgr2=0x7f6aa00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:17.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.900+0000 7f6aa5200640 1 -- 172.21.15.67:0/715520959 shutdown_connections 2024-05-30T17:00:17.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.900+0000 7f6aa5200640 1 -- 172.21.15.67:0/715520959 wait complete. 2024-05-30T17:00:17.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.901+0000 7f6aa5200640 1 Processor -- start 2024-05-30T17:00:17.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.902+0000 7f6aa5200640 1 -- start start 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.902+0000 7f6aa5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa006ee40 0x7f6aa0146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.902+0000 7f6aa5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006f2c0 0x7f6aa00b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.902+0000 7f6aa5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0156b30 0x7f6aa00b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.902+0000 7f6aa5200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa00afc20 con 0x7f6aa006f2c0 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.902+0000 7f6aa5200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa00b2c30 con 0x7f6aa0156b30 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.902+0000 7f6aa5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa00b38a0 con 0x7f6aa006ee40 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a96a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006f2c0 0x7f6aa00b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a96a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006f2c0 0x7f6aa00b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43866/0 (socket says 172.21.15.67:43866) 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a96a00640 1 -- 172.21.15.67:0/1526779083 learned_addr learned my addr 172.21.15.67:0/1526779083 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a9f400640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa006ee40 0x7f6aa0146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:17.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a9fe00640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0156b30 0x7f6aa00b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:17.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a96a00640 1 -- 172.21.15.67:0/1526779083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa006ee40 msgr2=0x7f6aa0146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:17.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a96a00640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa006ee40 0x7f6aa0146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a96a00640 1 -- 172.21.15.67:0/1526779083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0156b30 msgr2=0x7f6aa00b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:17.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a96a00640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0156b30 0x7f6aa00b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:17.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.903+0000 7f6a96a00640 1 -- 172.21.15.67:0/1526779083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa0147490 con 0x7f6aa006f2c0 2024-05-30T17:00:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.904+0000 7f6a96a00640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006f2c0 0x7f6aa00b4b40 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.904+0000 7f6a97e00640 1 -- 172.21.15.67:0/1526779083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88037040 con 0x7f6aa006f2c0 2024-05-30T17:00:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.904+0000 7f6a97e00640 1 -- 172.21.15.67:0/1526779083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a88004030 con 0x7f6aa006f2c0 2024-05-30T17:00:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.904+0000 7f6a97e00640 1 -- 172.21.15.67:0/1526779083 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88004420 con 0x7f6aa006f2c0 2024-05-30T17:00:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.904+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa0147710 con 0x7f6aa006f2c0 2024-05-30T17:00:17.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.904+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa00b3530 con 0x7f6aa006f2c0 2024-05-30T17:00:17.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.905+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa00689e0 con 0x7f6aa006f2c0 2024-05-30T17:00:17.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.906+0000 7f6a97e00640 1 -- 172.21.15.67:0/1526779083 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f6a88003610 con 0x7f6aa006f2c0 2024-05-30T17:00:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.907+0000 7f6a97e00640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a7c06dca0 0x7f6a7c070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.907+0000 7f6a97e00640 1 -- 172.21.15.67:0/1526779083 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f6a880ab230 con 0x7f6aa006f2c0 2024-05-30T17:00:17.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.908+0000 7f6a9f400640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a7c06dca0 0x7f6a7c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:17.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.908+0000 7f6a9f400640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a7c06dca0 0x7f6a7c070160 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000a950 tx=0x7f6a90008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:17.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:17.912+0000 7f6a97e00640 1 -- 172.21.15.67:0/1526779083 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6aa0147490 con 0x7f6aa006f2c0 2024-05-30T17:00:18.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.134+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6aa006f7f0 con 0x7f6aa006f2c0 2024-05-30T17:00:18.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.135+0000 7f6a97e00640 1 -- 172.21.15.67:0/1526779083 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) ==== 80+0+0 (secure 0 0 0) 0x7f6a8807b0b0 con 0x7f6aa006f2c0 2024-05-30T17:00:18.136 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T17:00:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a7c06dca0 msgr2=0x7f6a7c070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a7c06dca0 0x7f6a7c070160 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000a950 tx=0x7f6a90008040 comp rx=0 tx=0).stop 2024-05-30T17:00:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006f2c0 msgr2=0x7f6aa00b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006f2c0 0x7f6aa00b4b40 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a880046c0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 shutdown_connections 2024-05-30T17:00:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6a7c06dca0 0x7f6a7c070160 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6aa0156b30 0x7f6aa00b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6aa006f2c0 0x7f6aa00b4b40 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 --2- 172.21.15.67:0/1526779083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6aa006ee40 0x7f6aa0146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 >> 172.21.15.67:0/1526779083 conn(0x7f6aa00b1fc0 msgr2=0x7f6aa0071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 shutdown_connections 2024-05-30T17:00:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.137+0000 7f6aa5200640 1 -- 172.21.15.67:0/1526779083 wait complete. 2024-05-30T17:00:18.158 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T17:00:18.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:18.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-30T17:00:18.248 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T17:00:18.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:18.249 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-30T17:00:18.264 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[66929]: fuse finished with error 0 and tester_r 0 2024-05-30T17:00:18.403 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:00:18.403 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:00:18.403 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:18.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:00:18.445 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:00:18.445 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:00:18.446 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:18.488 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:00:18.488 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:00:18.489 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:00:18.489 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:18.537 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:00:18.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:00:18.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.753+0000 7ff1c6200640 1 Processor -- start 2024-05-30T17:00:18.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.753+0000 7ff1c6200640 1 -- start start 2024-05-30T17:00:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.754+0000 7ff1c6200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1c006f860 0x7ff1c00b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.754+0000 7ff1c6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c00d0bb0 0x7ff1c00ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.754+0000 7ff1c6200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c006ed10 con 0x7ff1c00b3690 2024-05-30T17:00:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.754+0000 7ff1c6200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c015cb00 con 0x7ff1c006f860 2024-05-30T17:00:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.754+0000 7ff1c6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c00b3cc0 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.754+0000 7ff1bf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1c006f860 0x7ff1c00b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.755+0000 7ff1bf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1c006f860 0x7ff1c00b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43874/0 (socket says 172.21.15.67:43874) 2024-05-30T17:00:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.755+0000 7ff1bf400640 1 -- 172.21.15.67:0/4094615358 learned_addr learned my addr 172.21.15.67:0/4094615358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:18.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.755+0000 7ff1bfe00640 1 --2- 172.21.15.67:0/4094615358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c00d0bb0 0x7ff1c00ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:18.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.755+0000 7ff1bf400640 1 -- 172.21.15.67:0/4094615358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c00d0bb0 msgr2=0x7ff1c00ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:18.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.755+0000 7ff1bf400640 1 --2- 172.21.15.67:0/4094615358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c00d0bb0 0x7ff1c00ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.755+0000 7ff1bf400640 1 -- 172.21.15.67:0/4094615358 >> v1:172.21.15.169:6789/0 conn(0x7ff1c00b3690 legacy=0x7ff1c00d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:18.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.755+0000 7ff1bf400640 1 -- 172.21.15.67:0/4094615358 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c00b3fa0 con 0x7ff1c006f860 2024-05-30T17:00:18.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.756+0000 7ff1bf400640 1 --2- 172.21.15.67:0/4094615358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c006f860 0x7ff1c00b3150 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c67333a450ece9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:18.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.756+0000 7ff1bea00640 1 -- 172.21.15.67:0/4094615358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0037070 con 0x7ff1c006f860 2024-05-30T17:00:18.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1bea00640 1 -- 172.21.15.67:0/4094615358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b0032440 con 0x7ff1c006f860 2024-05-30T17:00:18.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1bea00640 1 -- 172.21.15.67:0/4094615358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0032810 con 0x7ff1c006f860 2024-05-30T17:00:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1c6200640 1 -- 172.21.15.67:0/4094615358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c006f860 msgr2=0x7ff1c00b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1c6200640 1 --2- 172.21.15.67:0/4094615358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c006f860 0x7ff1c00b3150 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b002a210 comp rx=0 tx=0).stop 2024-05-30T17:00:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1c6200640 1 -- 172.21.15.67:0/4094615358 shutdown_connections 2024-05-30T17:00:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1c6200640 1 --2- 172.21.15.67:0/4094615358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c00d0bb0 0x7ff1c00ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1c6200640 1 --2- 172.21.15.67:0/4094615358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c006f860 0x7ff1c00b3150 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1c6200640 1 -- 172.21.15.67:0/4094615358 >> 172.21.15.67:0/4094615358 conn(0x7ff1c006bae0 msgr2=0x7ff1c006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1c6200640 1 -- 172.21.15.67:0/4094615358 shutdown_connections 2024-05-30T17:00:18.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.757+0000 7ff1c6200640 1 -- 172.21.15.67:0/4094615358 wait complete. 2024-05-30T17:00:18.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.759+0000 7ff1c6200640 1 Processor -- start 2024-05-30T17:00:18.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.759+0000 7ff1c6200640 1 -- start start 2024-05-30T17:00:18.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.760+0000 7ff1c6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c006f860 0x7ff1c015f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.760+0000 7ff1c6200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c00b3690 0x7ff1c00c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.760+0000 7ff1c6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c00d0bb0 0x7ff1c015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.760+0000 7ff1c6200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00b4840 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.760+0000 7ff1c6200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00b4560 con 0x7ff1c00b3690 2024-05-30T17:00:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.760+0000 7ff1c6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c00ad670 con 0x7ff1c006f860 2024-05-30T17:00:18.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c00d0bb0 0x7ff1c015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c00d0bb0 0x7ff1c015c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43876/0 (socket says 172.21.15.67:43876) 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1bfe00640 1 -- 172.21.15.67:0/1769311852 learned_addr learned my addr 172.21.15.67:0/1769311852 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1bf400640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c006f860 0x7ff1c015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1b6a00640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c00b3690 0x7ff1c00c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1bfe00640 1 -- 172.21.15.67:0/1769311852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c006f860 msgr2=0x7ff1c015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1bfe00640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c006f860 0x7ff1c015f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1bfe00640 1 -- 172.21.15.67:0/1769311852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c00b3690 msgr2=0x7ff1c00c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1bfe00640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c00b3690 0x7ff1c00c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.761+0000 7ff1bfe00640 1 -- 172.21.15.67:0/1769311852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c015c550 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.762+0000 7ff1bfe00640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c00d0bb0 0x7ff1c015c010 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800cc00 tx=0x7ff1b800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.762+0000 7ff1b7e00640 1 -- 172.21.15.67:0/1769311852 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800ce10 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.762+0000 7ff1b7e00640 1 -- 172.21.15.67:0/1769311852 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b8003990 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.762+0000 7ff1b7e00640 1 -- 172.21.15.67:0/1769311852 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8003d60 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.762+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c00c8bd0 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.762+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c00c9190 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.763+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c00689e0 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.764+0000 7ff1b7e00640 1 -- 172.21.15.67:0/1769311852 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7ff1b8003f00 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.765+0000 7ff1b7e00640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff19c06dcd0 0x7ff19c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:18.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.766+0000 7ff1b7e00640 1 -- 172.21.15.67:0/1769311852 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7ff1b808afe0 con 0x7ff1c00d0bb0 2024-05-30T17:00:18.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.766+0000 7ff1bf400640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff19c06dcd0 0x7ff19c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:18.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.767+0000 7ff1bf400640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff19c06dcd0 0x7ff19c070190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0034000 tx=0x7ff1b002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:18.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:18.770+0000 7ff1b7e00640 1 -- 172.21.15.67:0/1769311852 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1b8014030 con 0x7ff1c00d0bb0 2024-05-30T17:00:19.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.018+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1c006e2f0 con 0x7ff1c00d0bb0 2024-05-30T17:00:19.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.019+0000 7ff1b7e00640 1 -- 172.21.15.67:0/1769311852 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5189 (secure 0 0 0) 0x7ff1b805aab0 con 0x7ff1c00d0bb0 2024-05-30T17:00:19.021 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:19.021 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1054215955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1054215955},{"type":"v1","addr":"172.21.15.67:6837","nonce":1054215955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":5591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1236503792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1236503792},{"type":"v1","addr":"172.21.15.169:6835","nonce":1236503792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/4047293074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4047293074},{"type":"v1","addr":"172.21.15.67:6836","nonce":4047293074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:58:28.573056+0000","modified":"2024-05-30T16:58:31.656482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7398":{"gid":7398,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/2733959172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2733959172},{"type":"v1","addr":"172.21.15.169:6834","nonce":2733959172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7398,"qdb_cluster":[7398]},"id":6}]} 2024-05-30T17:00:19.021 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 56 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff19c06dcd0 msgr2=0x7ff19c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff19c06dcd0 0x7ff19c070190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0034000 tx=0x7ff1b002e000 comp rx=0 tx=0).stop 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c00d0bb0 msgr2=0x7ff1c015c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c00d0bb0 0x7ff1c015c010 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800cc00 tx=0x7ff1b800a5c0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 shutdown_connections 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff19c06dcd0 0x7ff19c070190 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1c00d0bb0 0x7ff1c015c010 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c00b3690 0x7ff1c00c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 --2- 172.21.15.67:0/1769311852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c006f860 0x7ff1c015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 >> 172.21.15.67:0/1769311852 conn(0x7ff1c006bae0 msgr2=0x7ff1c00d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:19.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 shutdown_connections 2024-05-30T17:00:19.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.022+0000 7ff1c6200640 1 -- 172.21.15.67:0/1769311852 wait complete. 2024-05-30T17:00:19.045 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:00:19.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.259+0000 7f764cc00640 1 Processor -- start 2024-05-30T17:00:19.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.259+0000 7f764cc00640 1 -- start start 2024-05-30T17:00:19.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.260+0000 7f764cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480d04a0 0x7f76480d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:19.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.260+0000 7f764cc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f76480b3460 0x7f76480b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:19.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.260+0000 7f764cc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76480ced40 con 0x7f76480b4b60 2024-05-30T17:00:19.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.260+0000 7f764cc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f76480d0e40 con 0x7f76480b3460 2024-05-30T17:00:19.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.260+0000 7f764cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f764815fbb0 con 0x7f76480d04a0 2024-05-30T17:00:19.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.261+0000 7f7645e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f76480b4b60 0x7f76480cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49020/0 (socket says 172.21.15.67:49020) 2024-05-30T17:00:19.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.261+0000 7f7645e00640 1 -- 172.21.15.67:0/303846512 learned_addr learned my addr 172.21.15.67:0/303846512 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.261+0000 7f7645400640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480d04a0 0x7f76480d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.261+0000 7f7646800640 1 --2- 172.21.15.67:0/303846512 >> v2:172.21.15.169:3300/0 conn(0x7f76480b3460 0x7f76480b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.261+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 444393370 0 0) 0x7f7638001490 con 0x7f76480b4b60 2024-05-30T17:00:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.261+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f762c003520 con 0x7f76480b4b60 2024-05-30T17:00:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2896821657 0 0) 0x7f762c003520 con 0x7f76480b4b60 2024-05-30T17:00:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480d04a0 msgr2=0x7f76480d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:19.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480d04a0 0x7f76480d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 >> v2:172.21.15.169:3300/0 conn(0x7f76480b3460 msgr2=0x7f76480b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 --2- 172.21.15.67:0/303846512 >> v2:172.21.15.169:3300/0 conn(0x7f76480b3460 0x7f76480b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76480cefc0 con 0x7f76480b4b60 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7638001be0 con 0x7f76480b4b60 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 >> v1:172.21.15.67:6789/0 conn(0x7f76480b4b60 legacy=0x7f76480cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480d04a0 0x7f762c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f762c003f80 0x7f762c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c004d40 0x7f762c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f764815fbb0 con 0x7f762c003f80 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f762c00b9c0 con 0x7f762c004d40 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f762c00bb40 con 0x7f76480d04a0 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7645400640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f762c003f80 0x7f762c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7645e00640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480d04a0 0x7f762c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:19.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7645400640 1 -- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480d04a0 msgr2=0x7f762c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7645400640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480d04a0 0x7f762c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7645400640 1 -- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c004d40 msgr2=0x7f762c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7645400640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c004d40 0x7f762c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.262+0000 7f7645400640 1 -- 172.21.15.67:0/303846512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c003520 con 0x7f762c003f80 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.264+0000 7f7645400640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f762c003f80 0x7f762c004800 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7634009af0 tx=0x7f762c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48db967eed7dcf0b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.264+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634036070 con 0x7f762c003f80 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.264+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7634031400 con 0x7f762c003f80 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.264+0000 7f7644a00640 1 -- 172.21.15.67:0/303846512 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634009690 con 0x7f762c003f80 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.265+0000 7f764cc00640 1 -- 172.21.15.67:0/303846512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f762c003f80 msgr2=0x7f762c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.265+0000 7f764cc00640 1 --2- 172.21.15.67:0/303846512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f762c003f80 0x7f762c004800 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7634009af0 tx=0x7f762c005830 comp rx=0 tx=0).stop 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.265+0000 7f7646800640 1 -- 172.21.15.67:0/303846512 reap_dead start 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.265+0000 7f764cc00640 1 -- 172.21.15.67:0/303846512 shutdown_connections 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.265+0000 7f764cc00640 1 -- 172.21.15.67:0/303846512 >> 172.21.15.67:0/303846512 conn(0x7f764806bae0 msgr2=0x7f764806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.265+0000 7f764cc00640 1 -- 172.21.15.67:0/303846512 shutdown_connections 2024-05-30T17:00:19.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.265+0000 7f764cc00640 1 -- 172.21.15.67:0/303846512 wait complete. 2024-05-30T17:00:19.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.267+0000 7f764cc00640 1 Processor -- start 2024-05-30T17:00:19.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.267+0000 7f764cc00640 1 -- start start 2024-05-30T17:00:19.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f764cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c003f80 0x7f76480999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:19.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f764cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764809b1d0 0x7f764809b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:19.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f764cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480ae1f0 0x7f76480ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:19.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f764cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76480b3e10 con 0x7f764809b1d0 2024-05-30T17:00:19.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f764cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76480cf580 con 0x7f762c003f80 2024-05-30T17:00:19.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f764cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76480b3f90 con 0x7f76480ae1f0 2024-05-30T17:00:19.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7645400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764809b1d0 0x7f764809b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:19.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7645400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764809b1d0 0x7f764809b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43890/0 (socket says 172.21.15.67:43890) 2024-05-30T17:00:19.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7645400640 1 -- 172.21.15.67:0/2236698306 learned_addr learned my addr 172.21.15.67:0/2236698306 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:19.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7646800640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480ae1f0 0x7f76480ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:19.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7645400640 1 -- 172.21.15.67:0/2236698306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480ae1f0 msgr2=0x7f76480ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:19.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7645e00640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c003f80 0x7f76480999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:19.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7645400640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480ae1f0 0x7f76480ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7645400640 1 -- 172.21.15.67:0/2236698306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c003f80 msgr2=0x7f76480999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7645400640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c003f80 0x7f76480999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.268+0000 7f7645400640 1 -- 172.21.15.67:0/2236698306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7648150ac0 con 0x7f764809b1d0 2024-05-30T17:00:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.269+0000 7f7645400640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764809b1d0 0x7f764809b650 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7634009af0 tx=0x7f7634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.269+0000 7f763e000640 1 -- 172.21.15.67:0/2236698306 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634040020 con 0x7f764809b1d0 2024-05-30T17:00:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.269+0000 7f763e000640 1 -- 172.21.15.67:0/2236698306 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76340038d0 con 0x7f764809b1d0 2024-05-30T17:00:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.269+0000 7f763e000640 1 -- 172.21.15.67:0/2236698306 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634036040 con 0x7f764809b1d0 2024-05-30T17:00:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.269+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7648150da0 con 0x7f764809b1d0 2024-05-30T17:00:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.270+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7648151360 con 0x7f764809b1d0 2024-05-30T17:00:19.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.271+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7648068240 con 0x7f764809b1d0 2024-05-30T17:00:19.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.272+0000 7f763e000640 1 -- 172.21.15.67:0/2236698306 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 34) ==== 92825+0+0 (secure 0 0 0) 0x7f7634043020 con 0x7f764809b1d0 2024-05-30T17:00:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.273+0000 7f763e000640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762806dcd0 0x7f7628070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.273+0000 7f763e000640 1 -- 172.21.15.67:0/2236698306 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f76340aabb0 con 0x7f764809b1d0 2024-05-30T17:00:19.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.273+0000 7f7645e00640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762806dcd0 0x7f7628070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.274+0000 7f7645e00640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762806dcd0 0x7f7628070190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7638007c40 tx=0x7f7638009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:19.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.277+0000 7f763e000640 1 -- 172.21.15.67:0/2236698306 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7648068240 con 0x7f764809b1d0 2024-05-30T17:00:19.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.574+0000 7f763e000640 1 -- 172.21.15.67:0/2236698306 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f76340765f0 con 0x7f764809b1d0 2024-05-30T17:00:19.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.705+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76480689e0 con 0x7f764809b1d0 2024-05-30T17:00:19.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.706+0000 7f763e000640 1 -- 172.21.15.67:0/2236698306 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5189 (secure 0 0 0) 0x7f763404a070 con 0x7f764809b1d0 2024-05-30T17:00:19.708 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:19.708 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1054215955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1054215955},{"type":"v1","addr":"172.21.15.67:6837","nonce":1054215955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":5591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1236503792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1236503792},{"type":"v1","addr":"172.21.15.169:6835","nonce":1236503792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/4047293074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4047293074},{"type":"v1","addr":"172.21.15.67:6836","nonce":4047293074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:58:28.573056+0000","modified":"2024-05-30T16:58:31.656482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7398":{"gid":7398,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/2733959172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2733959172},{"type":"v1","addr":"172.21.15.169:6834","nonce":2733959172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7398,"qdb_cluster":[7398]},"id":6}]} 2024-05-30T17:00:19.708 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 56 2024-05-30T17:00:19.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.710+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762806dcd0 msgr2=0x7f7628070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:19.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.710+0000 7f764cc00640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762806dcd0 0x7f7628070190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7638007c40 tx=0x7f7638009040 comp rx=0 tx=0).stop 2024-05-30T17:00:19.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.710+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764809b1d0 msgr2=0x7f764809b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:19.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.710+0000 7f764cc00640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764809b1d0 0x7f764809b650 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7634009af0 tx=0x7f7634004730 comp rx=0 tx=0).stop 2024-05-30T17:00:19.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.711+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 shutdown_connections 2024-05-30T17:00:19.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.711+0000 7f764cc00640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f762806dcd0 0x7f7628070190 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.711+0000 7f764cc00640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76480ae1f0 0x7f76480ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.711+0000 7f764cc00640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f764809b1d0 0x7f764809b650 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.711+0000 7f764cc00640 1 --2- 172.21.15.67:0/2236698306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f762c003f80 0x7f76480999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:19.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.711+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 >> 172.21.15.67:0/2236698306 conn(0x7f764806bae0 msgr2=0x7f764806eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:19.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.712+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 shutdown_connections 2024-05-30T17:00:19.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:19.712+0000 7f764cc00640 1 -- 172.21.15.67:0/2236698306 wait complete. 2024-05-30T17:00:19.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:00:20.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.002+0000 7fd1cde00640 1 Processor -- start 2024-05-30T17:00:20.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.003+0000 7fd1cde00640 1 -- start start 2024-05-30T17:00:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.004+0000 7fd1cde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d2db0 0x7fd1c80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.004+0000 7fd1cde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd1c80d4d60 0x7fd1c80d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.005+0000 7fd1cde00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1c80d36d0 con 0x7fd1c80cf490 2024-05-30T17:00:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.005+0000 7fd1cde00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c80d5540 con 0x7fd1c80d4d60 2024-05-30T17:00:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.005+0000 7fd1cde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c8163500 con 0x7fd1c80d2db0 2024-05-30T17:00:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.005+0000 7fd1be000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd1c80d4d60 0x7fd1c80d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.005+0000 7fd1be000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd1c80d4d60 0x7fd1c80d3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:49964/0 (socket says 172.21.15.67:49964) 2024-05-30T17:00:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.005+0000 7fd1be000640 1 -- 172.21.15.67:0/479720708 learned_addr learned my addr 172.21.15.67:0/479720708 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.006+0000 7fd1c6a00640 1 --2- 172.21.15.67:0/479720708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d2db0 0x7fd1c80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.006+0000 7fd1c6a00640 1 -- 172.21.15.67:0/479720708 >> v2:172.21.15.169:3300/0 conn(0x7fd1c80d4d60 msgr2=0x7fd1c80d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:20.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.006+0000 7fd1c6a00640 1 --2- 172.21.15.67:0/479720708 >> v2:172.21.15.169:3300/0 conn(0x7fd1c80d4d60 0x7fd1c80d3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.006+0000 7fd1c6a00640 1 -- 172.21.15.67:0/479720708 >> v1:172.21.15.169:6789/0 conn(0x7fd1c80cf490 legacy=0x7fd1c80cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.006+0000 7fd1c6a00640 1 -- 172.21.15.67:0/479720708 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c8162740 con 0x7fd1c80d2db0 2024-05-30T17:00:20.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.008+0000 7fd1c6a00640 1 --2- 172.21.15.67:0/479720708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d2db0 0x7fd1c80b3b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00097a0 tx=0x7fd1b002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af44ceb95faf6f18 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:20.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.008+0000 7fd1c6000640 1 -- 172.21.15.67:0/479720708 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0037070 con 0x7fd1c80d2db0 2024-05-30T17:00:20.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.009+0000 7fd1c6000640 1 -- 172.21.15.67:0/479720708 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b002ae40 con 0x7fd1c80d2db0 2024-05-30T17:00:20.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.009+0000 7fd1c6000640 1 -- 172.21.15.67:0/479720708 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0032660 con 0x7fd1c80d2db0 2024-05-30T17:00:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.009+0000 7fd1cde00640 1 -- 172.21.15.67:0/479720708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d2db0 msgr2=0x7fd1c80b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.009+0000 7fd1cde00640 1 --2- 172.21.15.67:0/479720708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d2db0 0x7fd1c80b3b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00097a0 tx=0x7fd1b002a0f0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.010+0000 7fd1cde00640 1 -- 172.21.15.67:0/479720708 shutdown_connections 2024-05-30T17:00:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.010+0000 7fd1cde00640 1 --2- 172.21.15.67:0/479720708 >> v2:172.21.15.169:3300/0 conn(0x7fd1c80d4d60 0x7fd1c80d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.010+0000 7fd1cde00640 1 --2- 172.21.15.67:0/479720708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d2db0 0x7fd1c80b3b50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.010+0000 7fd1cde00640 1 -- 172.21.15.67:0/479720708 >> 172.21.15.67:0/479720708 conn(0x7fd1c808bfc0 msgr2=0x7fd1c80d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:20.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.010+0000 7fd1cde00640 1 -- 172.21.15.67:0/479720708 shutdown_connections 2024-05-30T17:00:20.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.010+0000 7fd1cde00640 1 -- 172.21.15.67:0/479720708 wait complete. 2024-05-30T17:00:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.013+0000 7fd1cde00640 1 Processor -- start 2024-05-30T17:00:20.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.013+0000 7fd1cde00640 1 -- start start 2024-05-30T17:00:20.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.014+0000 7fd1cde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1c80cf490 0x7fd1c814ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.014+0000 7fd1cde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1c80d2db0 0x7fd1c80b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.014+0000 7fd1cde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d4d60 0x7fd1c80b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.014+0000 7fd1cde00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c8162fe0 con 0x7fd1c80cf490 2024-05-30T17:00:20.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.014+0000 7fd1cde00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c8162d00 con 0x7fd1c80d2db0 2024-05-30T17:00:20.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.014+0000 7fd1cde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c80d51a0 con 0x7fd1c80d4d60 2024-05-30T17:00:20.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.015+0000 7fd1c6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1c80cf490 0x7fd1c814ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.015+0000 7fd1c6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1c80cf490 0x7fd1c814ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43902/0 (socket says 172.21.15.67:43902) 2024-05-30T17:00:20.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.015+0000 7fd1c6a00640 1 -- 172.21.15.67:0/663787774 learned_addr learned my addr 172.21.15.67:0/663787774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:20.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.015+0000 7fd1be000640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1c80d2db0 0x7fd1c80b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.016+0000 7fd1c6a00640 1 -- 172.21.15.67:0/663787774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d4d60 msgr2=0x7fd1c80b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.016+0000 7fd1c6a00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d4d60 0x7fd1c80b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.016+0000 7fd1c6a00640 1 -- 172.21.15.67:0/663787774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1c80d2db0 msgr2=0x7fd1c80b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.016+0000 7fd1c6a00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1c80d2db0 0x7fd1c80b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.016+0000 7fd1c6a00640 1 -- 172.21.15.67:0/663787774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c80b31f0 con 0x7fd1c80cf490 2024-05-30T17:00:20.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.016+0000 7fd1be000640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1c80d2db0 0x7fd1c80b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:00:20.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.016+0000 7fd1c6a00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1c80cf490 0x7fd1c814ead0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009890 tx=0x7fd1b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:20.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.017+0000 7fd1bfe00640 1 -- 172.21.15.67:0/663787774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0040020 con 0x7fd1c80cf490 2024-05-30T17:00:20.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.017+0000 7fd1bfe00640 1 -- 172.21.15.67:0/663787774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b003c630 con 0x7fd1c80cf490 2024-05-30T17:00:20.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.017+0000 7fd1bfe00640 1 -- 172.21.15.67:0/663787774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0037040 con 0x7fd1c80cf490 2024-05-30T17:00:20.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.017+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c80b12a0 con 0x7fd1c80cf490 2024-05-30T17:00:20.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.017+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c80b1860 con 0x7fd1c80cf490 2024-05-30T17:00:20.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.019+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c80689e0 con 0x7fd1c80cf490 2024-05-30T17:00:20.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.021+0000 7fd1bfe00640 1 -- 172.21.15.67:0/663787774 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fd1b0043020 con 0x7fd1c80cf490 2024-05-30T17:00:20.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.023+0000 7fd1bfe00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd1a406de90 0x7fd1a4070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.023+0000 7fd1bfe00640 1 -- 172.21.15.67:0/663787774 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7fd1b00aaaa0 con 0x7fd1c80cf490 2024-05-30T17:00:20.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.024+0000 7fd1be000640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd1a406de90 0x7fd1a4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.025+0000 7fd1be000640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd1a406de90 0x7fd1a4070350 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd1c81532d0 tx=0x7fd1b800b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:20.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.034+0000 7fd1bfe00640 1 -- 172.21.15.67:0/663787774 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1b007a920 con 0x7fd1c80cf490 2024-05-30T17:00:20.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.291+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1c80684a0 con 0x7fd1c80cf490 2024-05-30T17:00:20.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.292+0000 7fd1bfe00640 1 -- 172.21.15.67:0/663787774 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16832 (secure 0 0 0) 0x7fd1b007a220 con 0x7fd1c80cf490 2024-05-30T17:00:20.294 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:20.294 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T16:58:34.163493+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:58:26.480292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-30T16:58:27.548326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.a","mappings":[{"from":1,"to":5}]},{"pgid":"13.30","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:00:20.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd1a406de90 msgr2=0x7fd1a4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd1a406de90 0x7fd1a4070350 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd1c81532d0 tx=0x7fd1b800b040 comp rx=0 tx=0).stop 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1c80cf490 msgr2=0x7fd1c814ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1c80cf490 0x7fd1c814ead0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009890 tx=0x7fd1b00046c0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 shutdown_connections 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd1a406de90 0x7fd1a4070350 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1c80d4d60 0x7fd1c80b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1c80d2db0 0x7fd1c80b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 --2- 172.21.15.67:0/663787774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1c80cf490 0x7fd1c814ead0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 >> 172.21.15.67:0/663787774 conn(0x7fd1c808bfc0 msgr2=0x7fd1c80d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 shutdown_connections 2024-05-30T17:00:20.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.295+0000 7fd1cde00640 1 -- 172.21.15.67:0/663787774 wait complete. 2024-05-30T17:00:20.313 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-30T17:00:20.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:00:20.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.503+0000 7f7b00c00640 1 Processor -- start 2024-05-30T17:00:20.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.504+0000 7f7b00c00640 1 -- start start 2024-05-30T17:00:20.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.505+0000 7f7b00c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7afc0d5190 0x7f7afc0d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.505+0000 7f7b00c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7afc0cecf0 con 0x7f7afc15ea40 2024-05-30T17:00:20.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.505+0000 7f7b00c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7afc0cefd0 con 0x7f7afc15ee20 2024-05-30T17:00:20.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.505+0000 7f7b00c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7afc0cf2b0 con 0x7f7afc0d5190 2024-05-30T17:00:20.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.505+0000 7f7afaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7afc0d5190 0x7f7afc0d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.505+0000 7f7afaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7afc0d5190 0x7f7afc0d30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43908/0 (socket says 172.21.15.67:43908) 2024-05-30T17:00:20.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.505+0000 7f7afb400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7afc15ea40 0x7f7afc0cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49034/0 (socket says 172.21.15.67:49034) 2024-05-30T17:00:20.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.505+0000 7f7afaa00640 1 -- 172.21.15.67:0/391762981 learned_addr learned my addr 172.21.15.67:0/391762981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:20.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.506+0000 7f7afaa00640 1 -- 172.21.15.67:0/391762981 >> v1:172.21.15.169:6789/0 conn(0x7f7afc15ee20 legacy=0x7f7afc15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:20.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.506+0000 7f7afaa00640 1 -- 172.21.15.67:0/391762981 >> v1:172.21.15.67:6789/0 conn(0x7f7afc15ea40 legacy=0x7f7afc0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.506+0000 7f7afaa00640 1 -- 172.21.15.67:0/391762981 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7afc0cf590 con 0x7f7afc0d5190 2024-05-30T17:00:20.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.507+0000 7f7afaa00640 1 --2- 172.21.15.67:0/391762981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 0x7f7afc0d30a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009850 tx=0x7f7ae402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bb2c0e77a6e635 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:20.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.507+0000 7f7afa000640 1 -- 172.21.15.67:0/391762981 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4004030 con 0x7f7afc0d5190 2024-05-30T17:00:20.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.507+0000 7f7afa000640 1 -- 172.21.15.67:0/391762981 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae40041d0 con 0x7f7afc0d5190 2024-05-30T17:00:20.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.507+0000 7f7afa000640 1 -- 172.21.15.67:0/391762981 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4003610 con 0x7f7afc0d5190 2024-05-30T17:00:20.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.508+0000 7f7b00c00640 1 -- 172.21.15.67:0/391762981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 msgr2=0x7f7afc0d30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.508+0000 7f7b00c00640 1 --2- 172.21.15.67:0/391762981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 0x7f7afc0d30a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009850 tx=0x7f7ae402a2a0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.508+0000 7f7b00c00640 1 -- 172.21.15.67:0/391762981 shutdown_connections 2024-05-30T17:00:20.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.508+0000 7f7b00c00640 1 --2- 172.21.15.67:0/391762981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 0x7f7afc0d30a0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.508+0000 7f7b00c00640 1 -- 172.21.15.67:0/391762981 >> 172.21.15.67:0/391762981 conn(0x7f7afc08bfc0 msgr2=0x7f7afc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:20.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.508+0000 7f7b00c00640 1 -- 172.21.15.67:0/391762981 shutdown_connections 2024-05-30T17:00:20.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.508+0000 7f7b00c00640 1 -- 172.21.15.67:0/391762981 wait complete. 2024-05-30T17:00:20.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.510+0000 7f7b00c00640 1 Processor -- start 2024-05-30T17:00:20.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.510+0000 7f7b00c00640 1 -- start start 2024-05-30T17:00:20.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7b00c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 0x7f7afc157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7b00c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7afc15ea40 0x7f7afc1e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7b00c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7afc15ee20 0x7f7afc0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7b00c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc15f2d0 con 0x7f7afc0d5190 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7b00c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7afc0d37f0 con 0x7f7afc15ee20 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7b00c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7afc15f450 con 0x7f7afc15ea40 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7afb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 0x7f7afc157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7afb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 0x7f7afc157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43924/0 (socket says 172.21.15.67:43924) 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7afb400640 1 -- 172.21.15.67:0/3575101735 learned_addr learned my addr 172.21.15.67:0/3575101735 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7afb400640 1 -- 172.21.15.67:0/3575101735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7afc15ea40 msgr2=0x7f7afc1e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7afb400640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7afc15ea40 0x7f7afc1e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7afb400640 1 -- 172.21.15.67:0/3575101735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7afc15ee20 msgr2=0x7f7afc0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7afb400640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7afc15ee20 0x7f7afc0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.511+0000 7f7afb400640 1 -- 172.21.15.67:0/3575101735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7afc1e9a30 con 0x7f7afc0d5190 2024-05-30T17:00:20.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.512+0000 7f7afb400640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 0x7f7afc157d80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c790 tx=0x7f7aec00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:20.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.512+0000 7f7af3400640 1 -- 172.21.15.67:0/3575101735 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec017070 con 0x7f7afc0d5190 2024-05-30T17:00:20.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.512+0000 7f7af3400640 1 -- 172.21.15.67:0/3575101735 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aec00a640 con 0x7f7afc0d5190 2024-05-30T17:00:20.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.512+0000 7f7af3400640 1 -- 172.21.15.67:0/3575101735 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00a9f0 con 0x7f7afc0d5190 2024-05-30T17:00:20.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.512+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7afc1582c0 con 0x7f7afc0d5190 2024-05-30T17:00:20.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.512+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7afc1e9750 con 0x7f7afc0d5190 2024-05-30T17:00:20.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.513+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7afc0690f0 con 0x7f7afc0d5190 2024-05-30T17:00:20.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.514+0000 7f7af3400640 1 -- 172.21.15.67:0/3575101735 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f7aec01a070 con 0x7f7afc0d5190 2024-05-30T17:00:20.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.515+0000 7f7af3400640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ad806ddc0 0x7f7ad8070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.515+0000 7f7af3400640 1 -- 172.21.15.67:0/3575101735 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5823+0+0 (secure 0 0 0) 0x7f7aec089e20 con 0x7f7afc0d5190 2024-05-30T17:00:20.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.516+0000 7f7afaa00640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ad806ddc0 0x7f7ad8070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.516+0000 7f7afaa00640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ad806ddc0 0x7f7ad8070280 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4033000 tx=0x7f7ae402c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:20.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.520+0000 7f7af3400640 1 -- 172.21.15.67:0/3575101735 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7aec059650 con 0x7f7afc0d5190 2024-05-30T17:00:20.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.763+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7afc08e620 con 0x7f7afc0d5190 2024-05-30T17:00:20.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.764+0000 7f7af3400640 1 -- 172.21.15.67:0/3575101735 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5189 (secure 0 0 0) 0x7f7aec05d950 con 0x7f7afc0d5190 2024-05-30T17:00:20.766 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:20.766 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1054215955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1054215955},{"type":"v1","addr":"172.21.15.67:6837","nonce":1054215955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":5591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1236503792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1236503792},{"type":"v1","addr":"172.21.15.169:6835","nonce":1236503792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/4047293074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4047293074},{"type":"v1","addr":"172.21.15.67:6836","nonce":4047293074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:58:28.573056+0000","modified":"2024-05-30T16:58:31.656482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7398":{"gid":7398,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/2733959172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2733959172},{"type":"v1","addr":"172.21.15.169:6834","nonce":2733959172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7398,"qdb_cluster":[7398]},"id":6}]} 2024-05-30T17:00:20.766 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 56 2024-05-30T17:00:20.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ad806ddc0 msgr2=0x7f7ad8070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ad806ddc0 0x7f7ad8070280 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4033000 tx=0x7f7ae402c000 comp rx=0 tx=0).stop 2024-05-30T17:00:20.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 msgr2=0x7f7afc157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 0x7f7afc157d80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c790 tx=0x7f7aec00cc50 comp rx=0 tx=0).stop 2024-05-30T17:00:20.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 shutdown_connections 2024-05-30T17:00:20.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7ad806ddc0 0x7f7ad8070280 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7afc15ee20 0x7f7afc0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7afc15ea40 0x7f7afc1e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 --2- 172.21.15.67:0/3575101735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7afc0d5190 0x7f7afc157d80 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 >> 172.21.15.67:0/3575101735 conn(0x7f7afc08bfc0 msgr2=0x7f7afc06d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:20.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 shutdown_connections 2024-05-30T17:00:20.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.767+0000 7f7b00c00640 1 -- 172.21.15.67:0/3575101735 wait complete. 2024-05-30T17:00:20.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:00:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.977+0000 7fc0f1800640 1 Processor -- start 2024-05-30T17:00:20.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.978+0000 7fc0f1800640 1 -- start start 2024-05-30T17:00:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.979+0000 7fc0f1800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc0ec15e870 0x7fc0ec0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.979+0000 7fc0f1800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0ec0cfa40 0x7fc0ec0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.979+0000 7fc0f1800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ec15ec50 con 0x7fc0ec0d14d0 2024-05-30T17:00:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.979+0000 7fc0f1800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ec0d2150 con 0x7fc0ec15e870 2024-05-30T17:00:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.979+0000 7fc0f1800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ec0cff80 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.979+0000 7fc0ebe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc0ec15e870 0x7fc0ec0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.979+0000 7fc0ebe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc0ec15e870 0x7fc0ec0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51156/0 (socket says 172.21.15.67:51156) 2024-05-30T17:00:20.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.979+0000 7fc0ebe00640 1 -- 172.21.15.67:0/4115993069 learned_addr learned my addr 172.21.15.67:0/4115993069 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:20.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.980+0000 7fc0ebe00640 1 -- 172.21.15.67:0/4115993069 >> v2:172.21.15.169:3300/0 conn(0x7fc0ec0cfa40 msgr2=0x7fc0ec0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:20.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.980+0000 7fc0ebe00640 1 --2- 172.21.15.67:0/4115993069 >> v2:172.21.15.169:3300/0 conn(0x7fc0ec0cfa40 0x7fc0ec0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.980+0000 7fc0ebe00640 1 -- 172.21.15.67:0/4115993069 >> v1:172.21.15.169:6789/0 conn(0x7fc0ec0d14d0 legacy=0x7fc0ec0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:20.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.980+0000 7fc0ebe00640 1 -- 172.21.15.67:0/4115993069 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0d0260 con 0x7fc0ec15e870 2024-05-30T17:00:20.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.981+0000 7fc0ebe00640 1 --2- 172.21.15.67:0/4115993069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec15e870 0x7fc0ec0d0f90 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec08cad0 tx=0x7fc0dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65d52cc785b5cdd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:20.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.981+0000 7fc0eaa00640 1 -- 172.21.15.67:0/4115993069 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc036070 con 0x7fc0ec15e870 2024-05-30T17:00:20.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.981+0000 7fc0eaa00640 1 -- 172.21.15.67:0/4115993069 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc031600 con 0x7fc0ec15e870 2024-05-30T17:00:20.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.981+0000 7fc0eaa00640 1 -- 172.21.15.67:0/4115993069 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0319d0 con 0x7fc0ec15e870 2024-05-30T17:00:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.982+0000 7fc0f1800640 1 -- 172.21.15.67:0/4115993069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec15e870 msgr2=0x7fc0ec0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.982+0000 7fc0f1800640 1 --2- 172.21.15.67:0/4115993069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec15e870 0x7fc0ec0d0f90 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec08cad0 tx=0x7fc0dc02a6f0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.982+0000 7fc0f1800640 1 -- 172.21.15.67:0/4115993069 shutdown_connections 2024-05-30T17:00:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.982+0000 7fc0f1800640 1 --2- 172.21.15.67:0/4115993069 >> v2:172.21.15.169:3300/0 conn(0x7fc0ec0cfa40 0x7fc0ec0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.982+0000 7fc0f1800640 1 --2- 172.21.15.67:0/4115993069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec15e870 0x7fc0ec0d0f90 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.982+0000 7fc0f1800640 1 -- 172.21.15.67:0/4115993069 >> 172.21.15.67:0/4115993069 conn(0x7fc0ec08bfc0 msgr2=0x7fc0ec0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.982+0000 7fc0f1800640 1 -- 172.21.15.67:0/4115993069 shutdown_connections 2024-05-30T17:00:20.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.982+0000 7fc0f1800640 1 -- 172.21.15.67:0/4115993069 wait complete. 2024-05-30T17:00:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.984+0000 7fc0f1800640 1 Processor -- start 2024-05-30T17:00:20.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.984+0000 7fc0f1800640 1 -- start start 2024-05-30T17:00:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0f1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec0cfa40 0x7fc0ec14f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0f1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0ec0d14d0 0x7fc0ec15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0f1800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0ec15e870 0x7fc0ec15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0f1800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec0d4fb0 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0f1800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec0d0820 con 0x7fc0ec15e870 2024-05-30T17:00:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0f1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec0d2b20 con 0x7fc0ec0d14d0 2024-05-30T17:00:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0ebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec0cfa40 0x7fc0ec14f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0ebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec0cfa40 0x7fc0ec14f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51168/0 (socket says 172.21.15.67:51168) 2024-05-30T17:00:20.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0ebe00640 1 -- 172.21.15.67:0/20242301 learned_addr learned my addr 172.21.15.67:0/20242301 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0ebe00640 1 -- 172.21.15.67:0/20242301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0ec0d14d0 msgr2=0x7fc0ec15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0ebe00640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0ec0d14d0 0x7fc0ec15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0ebe00640 1 -- 172.21.15.67:0/20242301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0ec15e870 msgr2=0x7fc0ec15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0ebe00640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0ec15e870 0x7fc0ec15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.985+0000 7fc0ebe00640 1 -- 172.21.15.67:0/20242301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0c98f0 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.986+0000 7fc0ebe00640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec0cfa40 0x7fc0ec14f810 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0093e0 tx=0x7fc0dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:20.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.986+0000 7fc0e3e00640 1 -- 172.21.15.67:0/20242301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc041020 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.986+0000 7fc0e3e00640 1 -- 172.21.15.67:0/20242301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0dc007820 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.986+0000 7fc0e3e00640 1 -- 172.21.15.67:0/20242301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc036040 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.986+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec0c9bd0 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.986+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec14fd50 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.988+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ec0689e0 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.988+0000 7fc0e3e00640 1 -- 172.21.15.67:0/20242301 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fc0dc044020 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.989+0000 7fc0e3e00640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc0cc06ddc0 0x7fc0cc070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.989+0000 7fc0e3e00640 1 -- 172.21.15.67:0/20242301 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5879+0+0 (secure 0 0 0) 0x7fc0dc0ab720 con 0x7fc0ec0cfa40 2024-05-30T17:00:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.990+0000 7fc0eb400640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc0cc06ddc0 0x7fc0cc070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:20.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.990+0000 7fc0eb400640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc0cc06ddc0 0x7fc0cc070280 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec163400 tx=0x7fc0d80079e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:20.994+0000 7fc0e3e00640 1 -- 172.21.15.67:0/20242301 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0dc04b030 con 0x7fc0ec0cfa40 2024-05-30T17:00:21.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.246+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0ec0d3d90 con 0x7fc0ec0cfa40 2024-05-30T17:00:21.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.247+0000 7fc0e3e00640 1 -- 172.21.15.67:0/20242301 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5189 (secure 0 0 0) 0x7fc0dc07b5a0 con 0x7fc0ec0cfa40 2024-05-30T17:00:21.249 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:21.249 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1054215955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1054215955},{"type":"v1","addr":"172.21.15.67:6837","nonce":1054215955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":5591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1236503792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1236503792},{"type":"v1","addr":"172.21.15.169:6835","nonce":1236503792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/4047293074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4047293074},{"type":"v1","addr":"172.21.15.67:6836","nonce":4047293074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T16:58:28.573056+0000","modified":"2024-05-30T16:58:31.656482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7398":{"gid":7398,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/2733959172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2733959172},{"type":"v1","addr":"172.21.15.169:6834","nonce":2733959172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7398,"qdb_cluster":[7398]},"id":6}]} 2024-05-30T17:00:21.249 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 56 2024-05-30T17:00:21.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc0cc06ddc0 msgr2=0x7fc0cc070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc0cc06ddc0 0x7fc0cc070280 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec163400 tx=0x7fc0d80079e0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec0cfa40 msgr2=0x7fc0ec14f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec0cfa40 0x7fc0ec14f810 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0093e0 tx=0x7fc0dc004a40 comp rx=0 tx=0).stop 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 shutdown_connections 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc0cc06ddc0 0x7fc0cc070280 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0ec15e870 0x7fc0ec15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0ec0d14d0 0x7fc0ec15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 --2- 172.21.15.67:0/20242301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0ec0cfa40 0x7fc0ec14f810 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 >> 172.21.15.67:0/20242301 conn(0x7fc0ec08bfc0 msgr2=0x7fc0ec0b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 shutdown_connections 2024-05-30T17:00:21.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.250+0000 7fc0f1800640 1 -- 172.21.15.67:0/20242301 wait complete. 2024-05-30T17:00:21.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:00:21.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.470+0000 7f217fe00640 1 Processor -- start 2024-05-30T17:00:21.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.471+0000 7f217fe00640 1 -- start start 2024-05-30T17:00:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.472+0000 7f217fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21800b6570 0x7f21801697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.472+0000 7f217fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21800d3300 con 0x7f218015fae0 2024-05-30T17:00:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.472+0000 7f217fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21800d35e0 con 0x7f2180160380 2024-05-30T17:00:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.472+0000 7f217fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f21800ced60 con 0x7f21800b6570 2024-05-30T17:00:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.473+0000 7f217ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f218015fae0 0x7f21800d4e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45646/0 (socket says 172.21.15.67:45646) 2024-05-30T17:00:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.473+0000 7f217ea00640 1 -- 172.21.15.67:0/3784830623 learned_addr learned my addr 172.21.15.67:0/3784830623 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:21.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.473+0000 7f217f400640 1 --2- 172.21.15.67:0/3784830623 >> v2:172.21.15.169:3300/0 conn(0x7f21800b6570 0x7f21801697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.473+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2079555014 0 0) 0x7f21800d3300 con 0x7f218015fae0 2024-05-30T17:00:21.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.473+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2164003520 con 0x7f218015fae0 2024-05-30T17:00:21.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3362630955 0 0) 0x7f2164003520 con 0x7f218015fae0 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 >> v2:172.21.15.169:3300/0 conn(0x7f21800b6570 msgr2=0x7f21801697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 --2- 172.21.15.67:0/3784830623 >> v2:172.21.15.169:3300/0 conn(0x7f21800b6570 0x7f21801697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 >> v1:172.21.15.169:6789/0 conn(0x7f2180160380 legacy=0x7f21800b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21800d38c0 con 0x7f218015fae0 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f2170001670 con 0x7f218015fae0 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 >> v1:172.21.15.67:6789/0 conn(0x7f218015fae0 legacy=0x7f21800d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 --2- 172.21.15.67:0/3784830623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164003a00 0x7f2164003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 --2- 172.21.15.67:0/3784830623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2164004800 0x7f2164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 --2- 172.21.15.67:0/3784830623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2164005230 0x7f216400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21800ced60 con 0x7f2164004800 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f216400c190 con 0x7f2164003a00 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f216400c390 con 0x7f2164005230 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.474+0000 7f217e000640 1 --2- 172.21.15.67:0/3784830623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2164004800 0x7f2164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.475+0000 7f217e000640 1 -- 172.21.15.67:0/3784830623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2164005230 msgr2=0x7f216400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:21.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.475+0000 7f217e000640 1 --2- 172.21.15.67:0/3784830623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2164005230 0x7f216400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.475+0000 7f217e000640 1 -- 172.21.15.67:0/3784830623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164003a00 msgr2=0x7f2164003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:21.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.475+0000 7f217e000640 1 --2- 172.21.15.67:0/3784830623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164003a00 0x7f2164003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.475+0000 7f217e000640 1 -- 172.21.15.67:0/3784830623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2164003520 con 0x7f2164004800 2024-05-30T17:00:21.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.475+0000 7f217f400640 1 -- 172.21.15.67:0/3784830623 reap_dead start 2024-05-30T17:00:21.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.476+0000 7f217e000640 1 --2- 172.21.15.67:0/3784830623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2164004800 0x7f2164004c80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f21801607a0 tx=0x7f2164005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e11d5b131966dd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:21.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.476+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21680013c0 con 0x7f2164004800 2024-05-30T17:00:21.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.476+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21680052d0 con 0x7f2164004800 2024-05-30T17:00:21.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.476+0000 7f217d600640 1 -- 172.21.15.67:0/3784830623 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216802a870 con 0x7f2164004800 2024-05-30T17:00:21.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.477+0000 7f217fe00640 1 -- 172.21.15.67:0/3784830623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2164004800 msgr2=0x7f2164004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.477+0000 7f217fe00640 1 --2- 172.21.15.67:0/3784830623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2164004800 0x7f2164004c80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f21801607a0 tx=0x7f2164005830 comp rx=0 tx=0).stop 2024-05-30T17:00:21.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.477+0000 7f217fe00640 1 -- 172.21.15.67:0/3784830623 shutdown_connections 2024-05-30T17:00:21.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.477+0000 7f217fe00640 1 --2- 172.21.15.67:0/3784830623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2164004800 0x7f2164004c80 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.477+0000 7f217fe00640 1 -- 172.21.15.67:0/3784830623 >> 172.21.15.67:0/3784830623 conn(0x7f218008bfc0 msgr2=0x7f21800d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:21.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.477+0000 7f217fe00640 1 -- 172.21.15.67:0/3784830623 shutdown_connections 2024-05-30T17:00:21.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.477+0000 7f217fe00640 1 -- 172.21.15.67:0/3784830623 wait complete. 2024-05-30T17:00:21.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.479+0000 7f217fe00640 1 Processor -- start 2024-05-30T17:00:21.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.479+0000 7f217fe00640 1 -- start start 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164004800 0x7f21800b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21800cd3a0 0x7f2180153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2180155a30 0x7f218006db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21800b55b0 con 0x7f21800cd3a0 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21800b5430 con 0x7f2164004800 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21800d4e20 con 0x7f2180155a30 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21800cd3a0 0x7f2180153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21800cd3a0 0x7f2180153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51188/0 (socket says 172.21.15.67:51188) 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217e000640 1 -- 172.21.15.67:0/120916402 learned_addr learned my addr 172.21.15.67:0/120916402 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217e000640 1 -- 172.21.15.67:0/120916402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2180155a30 msgr2=0x7f218006db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217e000640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2180155a30 0x7f218006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217e000640 1 -- 172.21.15.67:0/120916402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164004800 msgr2=0x7f21800b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217e000640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164004800 0x7f21800b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.480+0000 7f217e000640 1 -- 172.21.15.67:0/120916402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2180156280 con 0x7f21800cd3a0 2024-05-30T17:00:21.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.481+0000 7f217e000640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21800cd3a0 0x7f2180153a20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2168001860 tx=0x7f2168001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:21.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.481+0000 7f2176a00640 1 -- 172.21.15.67:0/120916402 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216803f020 con 0x7f21800cd3a0 2024-05-30T17:00:21.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.481+0000 7f2176a00640 1 -- 172.21.15.67:0/120916402 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2168006820 con 0x7f21800cd3a0 2024-05-30T17:00:21.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.481+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2180156560 con 0x7f21800cd3a0 2024-05-30T17:00:21.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.481+0000 7f2176a00640 1 -- 172.21.15.67:0/120916402 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168037040 con 0x7f21800cd3a0 2024-05-30T17:00:21.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.481+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2180156b20 con 0x7f21800cd3a0 2024-05-30T17:00:21.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.482+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21800689e0 con 0x7f21800cd3a0 2024-05-30T17:00:21.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.483+0000 7f2176a00640 1 -- 172.21.15.67:0/120916402 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f2168042020 con 0x7f21800cd3a0 2024-05-30T17:00:21.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.484+0000 7f2176a00640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f215c06ddf0 0x7f215c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.484+0000 7f2176a00640 1 -- 172.21.15.67:0/120916402 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5879+0+0 (secure 0 0 0) 0x7f21680aa360 con 0x7f21800cd3a0 2024-05-30T17:00:21.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.485+0000 7f217ea00640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f215c06ddf0 0x7f215c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.485+0000 7f217ea00640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f215c06ddf0 0x7f215c0702b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2180069090 tx=0x7f21700014b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:21.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.489+0000 7f2176a00640 1 -- 172.21.15.67:0/120916402 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2168049030 con 0x7f21800cd3a0 2024-05-30T17:00:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.704+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21800cd8a0 con 0x7f21800cd3a0 2024-05-30T17:00:21.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.706+0000 7f2176a00640 1 -- 172.21.15.67:0/120916402 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16943 (secure 0 0 0) 0x7f216807a1e0 con 0x7f21800cd3a0 2024-05-30T17:00:21.709 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:21.709 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":92,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:00:20.564122+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-30T16:58:26.480292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-30T16:58:27.548326+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":92,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.a","mappings":[{"from":1,"to":5}]},{"pgid":"13.30","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:00:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f215c06ddf0 msgr2=0x7f215c0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f215c06ddf0 0x7f215c0702b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2180069090 tx=0x7f21700014b0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21800cd3a0 msgr2=0x7f2180153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21800cd3a0 0x7f2180153a20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2168001860 tx=0x7f2168001170 comp rx=0 tx=0).stop 2024-05-30T17:00:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 shutdown_connections 2024-05-30T17:00:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f215c06ddf0 0x7f215c0702b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2180155a30 0x7f218006db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21800cd3a0 0x7f2180153a20 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 --2- 172.21.15.67:0/120916402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164004800 0x7f21800b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.709+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 >> 172.21.15.67:0/120916402 conn(0x7f218008bfc0 msgr2=0x7f2180162d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:21.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.710+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 shutdown_connections 2024-05-30T17:00:21.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.710+0000 7f217fe00640 1 -- 172.21.15.67:0/120916402 wait complete. 2024-05-30T17:00:21.731 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-30T17:00:21.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.938+0000 7fab5f200640 1 Processor -- start 2024-05-30T17:00:21.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.938+0000 7fab5f200640 1 -- start start 2024-05-30T17:00:21.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.940+0000 7fab5f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab60156b30 0x7fab600af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.940+0000 7fab5f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab60156f10 con 0x7fab6006ee40 2024-05-30T17:00:21.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.940+0000 7fab5f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab6006dda0 con 0x7fab600ad940 2024-05-30T17:00:21.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.940+0000 7fab5f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab6006e080 con 0x7fab60156b30 2024-05-30T17:00:21.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.940+0000 7fab57e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fab6006ee40 0x7fab600ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45662/0 (socket says 172.21.15.67:45662) 2024-05-30T17:00:21.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.940+0000 7fab57e00640 1 -- 172.21.15.67:0/2794038621 learned_addr learned my addr 172.21.15.67:0/2794038621 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602894469 0 0) 0x7fab60156f10 con 0x7fab6006ee40 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab44003520 con 0x7fab6006ee40 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5de00640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab60156b30 0x7fab600af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 339094323 0 0) 0x7fab44003520 con 0x7fab6006ee40 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab60156b30 msgr2=0x7fab600af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab60156b30 0x7fab600af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 >> v1:172.21.15.169:6789/0 conn(0x7fab600ad940 legacy=0x7fab600b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6006e360 con 0x7fab6006ee40 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fab48001670 con 0x7fab6006ee40 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 >> v1:172.21.15.67:6789/0 conn(0x7fab6006ee40 legacy=0x7fab600ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab60156b30 0x7fab440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004010 0x7fab44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab44004d40 0x7fab440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fab6006e080 con 0x7fab44004d40 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fab4400bc00 con 0x7fab44004010 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.941+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab4400be00 con 0x7fab60156b30 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.942+0000 7fab5e800640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab44004d40 0x7fab440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.942+0000 7fab5de00640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab60156b30 0x7fab440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.942+0000 7fab57e00640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004010 0x7fab44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.942+0000 7fab5e800640 1 -- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab60156b30 msgr2=0x7fab440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.942+0000 7fab5e800640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab60156b30 0x7fab440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.942+0000 7fab5e800640 1 -- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004010 msgr2=0x7fab44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.942+0000 7fab5e800640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004010 0x7fab44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.942+0000 7fab5e800640 1 -- 172.21.15.67:0/2794038621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab44003520 con 0x7fab44004d40 2024-05-30T17:00:21.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.942+0000 7fab57e00640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004010 0x7fab44004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:21.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.943+0000 7fab5e800640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab44004d40 0x7fab440051c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fab58001e20 tx=0x7fab44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78fcf26e69643e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.943+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58001a70 con 0x7fab44004d40 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.943+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5802ae40 con 0x7fab44004d40 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.943+0000 7fab5d400640 1 -- 172.21.15.67:0/2794038621 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58003bb0 con 0x7fab44004d40 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.944+0000 7fab5f200640 1 -- 172.21.15.67:0/2794038621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab44004d40 msgr2=0x7fab440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.944+0000 7fab5f200640 1 --2- 172.21.15.67:0/2794038621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab44004d40 0x7fab440051c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fab58001e20 tx=0x7fab44005830 comp rx=0 tx=0).stop 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.944+0000 7fab5e800640 1 -- 172.21.15.67:0/2794038621 reap_dead start 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.944+0000 7fab5f200640 1 -- 172.21.15.67:0/2794038621 shutdown_connections 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.944+0000 7fab5f200640 1 -- 172.21.15.67:0/2794038621 >> 172.21.15.67:0/2794038621 conn(0x7fab600b1fc0 msgr2=0x7fab600b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.944+0000 7fab5f200640 1 -- 172.21.15.67:0/2794038621 shutdown_connections 2024-05-30T17:00:21.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.944+0000 7fab5f200640 1 -- 172.21.15.67:0/2794038621 wait complete. 2024-05-30T17:00:21.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.946+0000 7fab5f200640 1 Processor -- start 2024-05-30T17:00:21.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.946+0000 7fab5f200640 1 -- start start 2024-05-30T17:00:21.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.947+0000 7fab5f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004d40 0x7fab60156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.947+0000 7fab5f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab600b12b0 0x7fab60072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.947+0000 7fab5f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab600af0a0 0x7fab600af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.947+0000 7fab5f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fab600b2db0 con 0x7fab600af0a0 2024-05-30T17:00:21.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.947+0000 7fab5f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fab600b2c30 con 0x7fab44004d40 2024-05-30T17:00:21.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.947+0000 7fab5f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab600b39d0 con 0x7fab600b12b0 2024-05-30T17:00:21.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.947+0000 7fab5e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab600af0a0 0x7fab600af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab5e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab600af0a0 0x7fab600af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51206/0 (socket says 172.21.15.67:51206) 2024-05-30T17:00:21.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab5e800640 1 -- 172.21.15.67:0/336798627 learned_addr learned my addr 172.21.15.67:0/336798627 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab5de00640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004d40 0x7fab60156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab57e00640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab600b12b0 0x7fab60072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab5e800640 1 -- 172.21.15.67:0/336798627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab600b12b0 msgr2=0x7fab60072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab5e800640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab600b12b0 0x7fab60072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab5e800640 1 -- 172.21.15.67:0/336798627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004d40 msgr2=0x7fab60156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab5e800640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004d40 0x7fab60156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab5e800640 1 -- 172.21.15.67:0/336798627 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab600b0250 con 0x7fab600af0a0 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab57e00640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab600b12b0 0x7fab60072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.948+0000 7fab5e800640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab600af0a0 0x7fab600af520 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fab5802a8f0 tx=0x7fab580014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.949+0000 7fab56000640 1 -- 172.21.15.67:0/336798627 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58001a70 con 0x7fab600af0a0 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.949+0000 7fab56000640 1 -- 172.21.15.67:0/336798627 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab58003e30 con 0x7fab600af0a0 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.949+0000 7fab56000640 1 -- 172.21.15.67:0/336798627 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58007db0 con 0x7fab600af0a0 2024-05-30T17:00:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.949+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab600b04f0 con 0x7fab600af0a0 2024-05-30T17:00:21.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.949+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab600b0ab0 con 0x7fab600af0a0 2024-05-30T17:00:21.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.950+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab600689e0 con 0x7fab600af0a0 2024-05-30T17:00:21.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.951+0000 7fab56000640 1 -- 172.21.15.67:0/336798627 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fab58043020 con 0x7fab600af0a0 2024-05-30T17:00:21.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.952+0000 7fab56000640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:21.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.952+0000 7fab56000640 1 -- 172.21.15.67:0/336798627 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5879+0+0 (secure 0 0 0) 0x7fab580a9cb0 con 0x7fab600af0a0 2024-05-30T17:00:21.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.953+0000 7fab5de00640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:21.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.953+0000 7fab5de00640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fab50006fd0 tx=0x7fab50008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:21.957+0000 7fab56000640 1 -- 172.21.15.67:0/336798627 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab58079b70 con 0x7fab600af0a0 2024-05-30T17:00:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.202+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fab60160ce0 con 0x7fab600af0a0 2024-05-30T17:00:22.204 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-05-30T17:00:22.203+0000 7f4982000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-30T17:00:22.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.207+0000 7fab56000640 1 -- 172.21.15.67:0/336798627 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) ==== 161+0+0 (secure 0 0 0) 0x7fab58079470 con 0x7fab600af0a0 2024-05-30T17:00:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-30T17:00:22.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.210+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 msgr2=0x7fab3c070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:22.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.210+0000 7fab5f200640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fab50006fd0 tx=0x7fab50008040 comp rx=0 tx=0).stop 2024-05-30T17:00:22.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.210+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab600af0a0 msgr2=0x7fab600af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:22.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.210+0000 7fab5f200640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab600af0a0 0x7fab600af520 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fab5802a8f0 tx=0x7fab580014a0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.211+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 shutdown_connections 2024-05-30T17:00:22.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.211+0000 7fab5f200640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.211+0000 7fab5f200640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab600af0a0 0x7fab600af520 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.211+0000 7fab5f200640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab600b12b0 0x7fab60072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.211+0000 7fab5f200640 1 --2- 172.21.15.67:0/336798627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab44004d40 0x7fab60156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.211+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 >> 172.21.15.67:0/336798627 conn(0x7fab600b1fc0 msgr2=0x7fab60160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:22.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.211+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 shutdown_connections 2024-05-30T17:00:22.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.211+0000 7fab5f200640 1 -- 172.21.15.67:0/336798627 wait complete. 2024-05-30T17:00:22.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-30T17:00:22.337 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T17:00:22.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.433+0000 7f58b0e00640 1 Processor -- start 2024-05-30T17:00:22.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.433+0000 7f58b0e00640 1 -- start start 2024-05-30T17:00:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.434+0000 7f58b0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac153510 0x7f58ac0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.434+0000 7f58b0e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58ac0ccb90 0x7f58ac0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.434+0000 7f58b0e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58ac15d900 con 0x7f58ac15cb60 2024-05-30T17:00:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.434+0000 7f58b0e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f58ac1538f0 con 0x7f58ac0ccb90 2024-05-30T17:00:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.434+0000 7f58b0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac0cd530 con 0x7f58ac153510 2024-05-30T17:00:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58aaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58ac0ccb90 0x7f58ac0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58abe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f58ac15cb60 0x7f58ac15cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45666/0 (socket says 172.21.15.67:45666) 2024-05-30T17:00:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58aaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58ac0ccb90 0x7f58ac0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51218/0 (socket says 172.21.15.67:51218) 2024-05-30T17:00:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58abe00640 1 -- 172.21.15.67:0/430300003 learned_addr learned my addr 172.21.15.67:0/430300003 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58ab400640 1 --2- 172.21.15.67:0/430300003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac153510 0x7f58ac0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58aaa00640 1 -- 172.21.15.67:0/430300003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac153510 msgr2=0x7f58ac0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58aaa00640 1 --2- 172.21.15.67:0/430300003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac153510 0x7f58ac0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58aaa00640 1 -- 172.21.15.67:0/430300003 >> v1:172.21.15.67:6789/0 conn(0x7f58ac15cb60 legacy=0x7f58ac15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58aa000640 1 -- 172.21.15.67:0/430300003 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655001933 0 0) 0x7f58ac15d900 con 0x7f58ac15cb60 2024-05-30T17:00:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.435+0000 7f58aaa00640 1 -- 172.21.15.67:0/430300003 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac0b3f90 con 0x7f58ac0ccb90 2024-05-30T17:00:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.436+0000 7f58aaa00640 1 --2- 172.21.15.67:0/430300003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac0ccb90 0x7f58ac0ccff0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f58980097a0 tx=0x7f589802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7599409a6a5d7d08 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58aa000640 1 -- 172.21.15.67:0/430300003 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898037070 con 0x7f58ac0ccb90 2024-05-30T17:00:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58aa000640 1 -- 172.21.15.67:0/430300003 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589802ae50 con 0x7f58ac0ccb90 2024-05-30T17:00:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58aa000640 1 -- 172.21.15.67:0/430300003 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898032660 con 0x7f58ac0ccb90 2024-05-30T17:00:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58b0e00640 1 -- 172.21.15.67:0/430300003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac0ccb90 msgr2=0x7f58ac0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58b0e00640 1 --2- 172.21.15.67:0/430300003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac0ccb90 0x7f58ac0ccff0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f58980097a0 tx=0x7f589802a100 comp rx=0 tx=0).stop 2024-05-30T17:00:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58b0e00640 1 -- 172.21.15.67:0/430300003 shutdown_connections 2024-05-30T17:00:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58b0e00640 1 --2- 172.21.15.67:0/430300003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac0ccb90 0x7f58ac0ccff0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58b0e00640 1 --2- 172.21.15.67:0/430300003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac153510 0x7f58ac0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58b0e00640 1 -- 172.21.15.67:0/430300003 >> 172.21.15.67:0/430300003 conn(0x7f58ac06b900 msgr2=0x7f58ac06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58b0e00640 1 -- 172.21.15.67:0/430300003 shutdown_connections 2024-05-30T17:00:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.437+0000 7f58b0e00640 1 -- 172.21.15.67:0/430300003 wait complete. 2024-05-30T17:00:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.439+0000 7f58b0e00640 1 Processor -- start 2024-05-30T17:00:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.439+0000 7f58b0e00640 1 -- start start 2024-05-30T17:00:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.440+0000 7f58b0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac0ccb90 0x7f58ac14ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.440+0000 7f58b0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac153510 0x7f58ac14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.440+0000 7f58b0e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac15cb60 0x7f58ac15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.440+0000 7f58b0e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac0b4550 con 0x7f58ac153510 2024-05-30T17:00:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.440+0000 7f58b0e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac0b48c0 con 0x7f58ac15cb60 2024-05-30T17:00:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.440+0000 7f58b0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac06e940 con 0x7f58ac0ccb90 2024-05-30T17:00:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.440+0000 7f58aaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac153510 0x7f58ac14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:22.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.440+0000 7f58aaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac153510 0x7f58ac14a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51222/0 (socket says 172.21.15.67:51222) 2024-05-30T17:00:22.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58aaa00640 1 -- 172.21.15.67:0/1779972432 learned_addr learned my addr 172.21.15.67:0/1779972432 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:22.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58ab400640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac0ccb90 0x7f58ac14ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58abe00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac15cb60 0x7f58ac15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58aaa00640 1 -- 172.21.15.67:0/1779972432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac0ccb90 msgr2=0x7f58ac14ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58aaa00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac0ccb90 0x7f58ac14ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58aaa00640 1 -- 172.21.15.67:0/1779972432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac15cb60 msgr2=0x7f58ac15b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58aaa00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac15cb60 0x7f58ac15b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58aaa00640 1 -- 172.21.15.67:0/1779972432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac15ee20 con 0x7f58ac153510 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58abe00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac15cb60 0x7f58ac15b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.441+0000 7f58aaa00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac153510 0x7f58ac14a1d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f58980098c0 tx=0x7f58980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.442+0000 7f58a3400640 1 -- 172.21.15.67:0/1779972432 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898041030 con 0x7f58ac153510 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.442+0000 7f58a3400640 1 -- 172.21.15.67:0/1779972432 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589802ac50 con 0x7f58ac153510 2024-05-30T17:00:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.442+0000 7f58a3400640 1 -- 172.21.15.67:0/1779972432 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898037040 con 0x7f58ac153510 2024-05-30T17:00:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.442+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac15f100 con 0x7f58ac153510 2024-05-30T17:00:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.442+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac1504b0 con 0x7f58ac153510 2024-05-30T17:00:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.443+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ac0689e0 con 0x7f58ac153510 2024-05-30T17:00:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.444+0000 7f58a3400640 1 -- 172.21.15.67:0/1779972432 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f5898044070 con 0x7f58ac153510 2024-05-30T17:00:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.445+0000 7f58a3400640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f588c06ddc0 0x7f588c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:22.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.445+0000 7f58a3400640 1 -- 172.21.15.67:0/1779972432 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5937+0+0 (secure 0 0 0) 0x7f589802ae00 con 0x7f58ac153510 2024-05-30T17:00:22.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.445+0000 7f58ab400640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f588c06ddc0 0x7f588c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:22.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.446+0000 7f58ab400640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f588c06ddc0 0x7f588c070280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f589c009ad0 tx=0x7f589c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:22.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.450+0000 7f58a3400640 1 -- 172.21.15.67:0/1779972432 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f589804b070 con 0x7f58ac153510 2024-05-30T17:00:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:22.696+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f58ac0d0c20 con 0x7f58ac153510 2024-05-30T17:00:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.218+0000 7f58a3400640 1 -- 172.21.15.67:0/1779972432 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) ==== 104+0+0 (secure 0 0 0) 0x7f5898044e40 con 0x7f58ac153510 2024-05-30T17:00:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.221+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f588c06ddc0 msgr2=0x7f588c070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.221+0000 7f58b0e00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f588c06ddc0 0x7f588c070280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f589c009ad0 tx=0x7f589c008040 comp rx=0 tx=0).stop 2024-05-30T17:00:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.221+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac153510 msgr2=0x7f58ac14a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.221+0000 7f58b0e00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac153510 0x7f58ac14a1d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f58980098c0 tx=0x7f58980046c0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.221+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 shutdown_connections 2024-05-30T17:00:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.222+0000 7f58b0e00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f588c06ddc0 0x7f588c070280 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.222+0000 7f58b0e00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58ac15cb60 0x7f58ac15b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.222+0000 7f58b0e00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ac153510 0x7f58ac14a1d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.222+0000 7f58b0e00640 1 --2- 172.21.15.67:0/1779972432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58ac0ccb90 0x7f58ac14ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.222+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 >> 172.21.15.67:0/1779972432 conn(0x7f58ac06b900 msgr2=0x7f58ac0cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.222+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 shutdown_connections 2024-05-30T17:00:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.222+0000 7f58b0e00640 1 -- 172.21.15.67:0/1779972432 wait complete. 2024-05-30T17:00:23.247 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-30T17:00:23.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.460+0000 7f188fe00640 1 Processor -- start 2024-05-30T17:00:23.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.460+0000 7f188fe00640 1 -- start start 2024-05-30T17:00:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.462+0000 7f188fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f18900af6f0 0x7f18901623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.462+0000 7f188fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18900d0b40 0x7f18901428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.462+0000 7f188fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1890162cb0 con 0x7f18901480a0 2024-05-30T17:00:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.462+0000 7f188fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1890162f90 con 0x7f18900af6f0 2024-05-30T17:00:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.462+0000 7f188fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1890163270 con 0x7f18900d0b40 2024-05-30T17:00:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.462+0000 7f188ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f18900af6f0 0x7f18901623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.462+0000 7f1887e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f18901480a0 0x7f18900d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45672/0 (socket says 172.21.15.67:45672) 2024-05-30T17:00:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.462+0000 7f1887e00640 1 -- 172.21.15.67:0/2401558264 learned_addr learned my addr 172.21.15.67:0/2401558264 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.462+0000 7f188f400640 1 --2- 172.21.15.67:0/2401558264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18900d0b40 0x7f18901428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.463+0000 7f188ea00640 1 -- 172.21.15.67:0/2401558264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18900d0b40 msgr2=0x7f18901428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.463+0000 7f188ea00640 1 --2- 172.21.15.67:0/2401558264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18900d0b40 0x7f18901428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.463+0000 7f188ea00640 1 -- 172.21.15.67:0/2401558264 >> v1:172.21.15.67:6789/0 conn(0x7f18901480a0 legacy=0x7f18900d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.463+0000 7f188ea00640 1 -- 172.21.15.67:0/2401558264 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18901420d0 con 0x7f18900af6f0 2024-05-30T17:00:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.463+0000 7f188e000640 1 -- 172.21.15.67:0/2401558264 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1141414980 0 0) 0x7f1890162cb0 con 0x7f18901480a0 2024-05-30T17:00:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.464+0000 7f188ea00640 1 --2- 172.21.15.67:0/2401558264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900af6f0 0x7f18901623b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f18800097a0 tx=0x7f188002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28593ebd551ff10 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.465+0000 7f188e000640 1 -- 172.21.15.67:0/2401558264 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1880037070 con 0x7f18900af6f0 2024-05-30T17:00:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.465+0000 7f188e000640 1 -- 172.21.15.67:0/2401558264 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1880032440 con 0x7f18900af6f0 2024-05-30T17:00:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.465+0000 7f188e000640 1 -- 172.21.15.67:0/2401558264 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1880032810 con 0x7f18900af6f0 2024-05-30T17:00:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.465+0000 7f188fe00640 1 -- 172.21.15.67:0/2401558264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900af6f0 msgr2=0x7f18901623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.465+0000 7f188fe00640 1 --2- 172.21.15.67:0/2401558264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900af6f0 0x7f18901623b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f18800097a0 tx=0x7f188002a180 comp rx=0 tx=0).stop 2024-05-30T17:00:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.465+0000 7f188fe00640 1 -- 172.21.15.67:0/2401558264 shutdown_connections 2024-05-30T17:00:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.465+0000 7f188fe00640 1 --2- 172.21.15.67:0/2401558264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18900d0b40 0x7f18901428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.465+0000 7f188fe00640 1 --2- 172.21.15.67:0/2401558264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900af6f0 0x7f18901623b0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.465+0000 7f188fe00640 1 -- 172.21.15.67:0/2401558264 >> 172.21.15.67:0/2401558264 conn(0x7f189006bae0 msgr2=0x7f189006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.466+0000 7f188fe00640 1 -- 172.21.15.67:0/2401558264 shutdown_connections 2024-05-30T17:00:23.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.466+0000 7f188fe00640 1 -- 172.21.15.67:0/2401558264 wait complete. 2024-05-30T17:00:23.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.467+0000 7f188fe00640 1 Processor -- start 2024-05-30T17:00:23.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.468+0000 7f188fe00640 1 -- start start 2024-05-30T17:00:23.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.468+0000 7f188fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18900af6f0 0x7f18901428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.468+0000 7f188fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900d0b40 0x7f1890143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.468+0000 7f188fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18901480a0 0x7f1890161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.468+0000 7f188fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1890162a70 con 0x7f18900d0b40 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.468+0000 7f188fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f18901628f0 con 0x7f18900af6f0 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.468+0000 7f188fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f18900afad0 con 0x7f18901480a0 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f1887e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900d0b40 0x7f1890143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f1887e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900d0b40 0x7f1890143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51244/0 (socket says 172.21.15.67:51244) 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f1887e00640 1 -- 172.21.15.67:0/1567900346 learned_addr learned my addr 172.21.15.67:0/1567900346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f188ea00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18900af6f0 0x7f18901428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f188f400640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18901480a0 0x7f1890161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f1887e00640 1 -- 172.21.15.67:0/1567900346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18901480a0 msgr2=0x7f1890161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f1887e00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18901480a0 0x7f1890161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f1887e00640 1 -- 172.21.15.67:0/1567900346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18900af6f0 msgr2=0x7f18901428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:23.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f1887e00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18900af6f0 0x7f18901428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:23.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f1887e00640 1 -- 172.21.15.67:0/1567900346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1890142ed0 con 0x7f18900d0b40 2024-05-30T17:00:23.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.469+0000 7f188ea00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18900af6f0 0x7f18901428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.470+0000 7f1887e00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900d0b40 0x7f1890143f50 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f187800c4a0 tx=0x7f187800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.470+0000 7f1886a00640 1 -- 172.21.15.67:0/1567900346 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878017070 con 0x7f18900d0b40 2024-05-30T17:00:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.470+0000 7f1886a00640 1 -- 172.21.15.67:0/1567900346 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1878005ce0 con 0x7f18900d0b40 2024-05-30T17:00:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.470+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18901431b0 con 0x7f18900d0b40 2024-05-30T17:00:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.470+0000 7f1886a00640 1 -- 172.21.15.67:0/1567900346 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18780072b0 con 0x7f18900d0b40 2024-05-30T17:00:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.470+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1890143770 con 0x7f18900d0b40 2024-05-30T17:00:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.471+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18900681e0 con 0x7f18900d0b40 2024-05-30T17:00:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.472+0000 7f1886a00640 1 -- 172.21.15.67:0/1567900346 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f187801a070 con 0x7f18900d0b40 2024-05-30T17:00:23.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.473+0000 7f1886a00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f186c06ddc0 0x7f186c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:23.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.473+0000 7f1886a00640 1 -- 172.21.15.67:0/1567900346 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5937+0+0 (secure 0 0 0) 0x7f1878089cd0 con 0x7f18900d0b40 2024-05-30T17:00:23.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.474+0000 7f188ea00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f186c06ddc0 0x7f186c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:23.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.474+0000 7f188ea00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f186c06ddc0 0x7f186c070280 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1880034000 tx=0x7f188002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:23.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.478+0000 7f1886a00640 1 -- 172.21.15.67:0/1567900346 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18900681e0 con 0x7f18900d0b40 2024-05-30T17:00:23.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:23.685+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f18900689e0 con 0x7f18900d0b40 2024-05-30T17:00:24.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.595+0000 7f1886a00640 1 -- 172.21.15.67:0/1567900346 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7f1878059b50 con 0x7f18900d0b40 2024-05-30T17:00:24.597 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-05-30T17:00:24.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.601+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f186c06ddc0 msgr2=0x7f186c070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:24.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.601+0000 7f188fe00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f186c06ddc0 0x7f186c070280 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1880034000 tx=0x7f188002e000 comp rx=0 tx=0).stop 2024-05-30T17:00:24.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.601+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900d0b40 msgr2=0x7f1890143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:24.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.601+0000 7f188fe00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900d0b40 0x7f1890143f50 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f187800c4a0 tx=0x7f187800c960 comp rx=0 tx=0).stop 2024-05-30T17:00:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.603+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 shutdown_connections 2024-05-30T17:00:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.603+0000 7f188fe00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f186c06ddc0 0x7f186c070280 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.603+0000 7f188fe00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18901480a0 0x7f1890161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.603+0000 7f188fe00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18900d0b40 0x7f1890143f50 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.603+0000 7f188fe00640 1 --2- 172.21.15.67:0/1567900346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18900af6f0 0x7f18901428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.603+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 >> 172.21.15.67:0/1567900346 conn(0x7f189006bae0 msgr2=0x7f189006f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.603+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 shutdown_connections 2024-05-30T17:00:24.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.605+0000 7f188fe00640 1 -- 172.21.15.67:0/1567900346 wait complete. 2024-05-30T17:00:24.632 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-30T17:00:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.833+0000 7f4738e00640 1 Processor -- start 2024-05-30T17:00:24.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.833+0000 7f4738e00640 1 -- start start 2024-05-30T17:00:24.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4738e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47340ce560 0x7f47340b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4738e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 0x7f473415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4738e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f473415bb30 con 0x7f47340d0430 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4738e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f47340b3ba0 con 0x7f47340ce560 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4738e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f473415bdd0 con 0x7f47340d0860 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4733400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47340ce560 0x7f47340b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4733400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47340ce560 0x7f47340b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51250/0 (socket says 172.21.15.67:51250) 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4733400640 1 -- 172.21.15.67:0/15560222 learned_addr learned my addr 172.21.15.67:0/15560222 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4733e00640 1 --2- 172.21.15.67:0/15560222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 0x7f473415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4733400640 1 -- 172.21.15.67:0/15560222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 msgr2=0x7f473415f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4733400640 1 --2- 172.21.15.67:0/15560222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 0x7f473415f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4733400640 1 -- 172.21.15.67:0/15560222 >> v1:172.21.15.169:6789/0 conn(0x7f47340d0430 legacy=0x7f473415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:24.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.835+0000 7f4733400640 1 -- 172.21.15.67:0/15560222 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473415c0b0 con 0x7f47340ce560 2024-05-30T17:00:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.836+0000 7f4733400640 1 --2- 172.21.15.67:0/15560222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340ce560 0x7f47340b4b80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4724009e50 tx=0x7f472402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bac135a6da63be1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.836+0000 7f4732a00640 1 -- 172.21.15.67:0/15560222 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724036070 con 0x7f47340ce560 2024-05-30T17:00:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.837+0000 7f4732a00640 1 -- 172.21.15.67:0/15560222 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4724031600 con 0x7f47340ce560 2024-05-30T17:00:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.837+0000 7f4732a00640 1 -- 172.21.15.67:0/15560222 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47240319d0 con 0x7f47340ce560 2024-05-30T17:00:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.837+0000 7f4738e00640 1 -- 172.21.15.67:0/15560222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340ce560 msgr2=0x7f47340b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.837+0000 7f4738e00640 1 --2- 172.21.15.67:0/15560222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340ce560 0x7f47340b4b80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4724009e50 tx=0x7f472402a6c0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.837+0000 7f4738e00640 1 -- 172.21.15.67:0/15560222 shutdown_connections 2024-05-30T17:00:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.837+0000 7f4738e00640 1 --2- 172.21.15.67:0/15560222 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 0x7f473415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.837+0000 7f4738e00640 1 --2- 172.21.15.67:0/15560222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340ce560 0x7f47340b4b80 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.837+0000 7f4738e00640 1 -- 172.21.15.67:0/15560222 >> 172.21.15.67:0/15560222 conn(0x7f473406bae0 msgr2=0x7f473406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:24.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.838+0000 7f4738e00640 1 -- 172.21.15.67:0/15560222 shutdown_connections 2024-05-30T17:00:24.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.838+0000 7f4738e00640 1 -- 172.21.15.67:0/15560222 wait complete. 2024-05-30T17:00:24.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.839+0000 7f4738e00640 1 Processor -- start 2024-05-30T17:00:24.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.839+0000 7f4738e00640 1 -- start start 2024-05-30T17:00:24.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.840+0000 7f4738e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47340ce560 0x7f47340af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:24.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.840+0000 7f4738e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340d0430 0x7f47341540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:24.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f4738e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 0x7f4734154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:24.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f4738e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f47340d0cf0 con 0x7f47340d0430 2024-05-30T17:00:24.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f4738e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f473415c670 con 0x7f47340ce560 2024-05-30T17:00:24.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f4738e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f473415fd10 con 0x7f47340d0860 2024-05-30T17:00:24.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f472aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340d0430 0x7f47341540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f472aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340d0430 0x7f47341540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51262/0 (socket says 172.21.15.67:51262) 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f472aa00640 1 -- 172.21.15.67:0/2199247276 learned_addr learned my addr 172.21.15.67:0/2199247276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f4733400640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47340ce560 0x7f47340af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f4733e00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 0x7f4734154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f472aa00640 1 -- 172.21.15.67:0/2199247276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 msgr2=0x7f4734154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f472aa00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 0x7f4734154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f472aa00640 1 -- 172.21.15.67:0/2199247276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47340ce560 msgr2=0x7f47340af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f472aa00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47340ce560 0x7f47340af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f472aa00640 1 -- 172.21.15.67:0/2199247276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4734157010 con 0x7f47340d0430 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.841+0000 7f4733400640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47340ce560 0x7f47340af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:00:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.842+0000 7f472aa00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340d0430 0x7f47341540d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f471c00c430 tx=0x7f471c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:24.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.842+0000 7f472be00640 1 -- 172.21.15.67:0/2199247276 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c011790 con 0x7f47340d0430 2024-05-30T17:00:24.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.842+0000 7f472be00640 1 -- 172.21.15.67:0/2199247276 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f471c017070 con 0x7f47340d0430 2024-05-30T17:00:24.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.842+0000 7f472be00640 1 -- 172.21.15.67:0/2199247276 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c011bb0 con 0x7f47340d0430 2024-05-30T17:00:24.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.842+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47341572f0 con 0x7f47340d0430 2024-05-30T17:00:24.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.842+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4734144c70 con 0x7f47340d0430 2024-05-30T17:00:24.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.844+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47340689e0 con 0x7f47340d0430 2024-05-30T17:00:24.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.844+0000 7f472be00640 1 -- 172.21.15.67:0/2199247276 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f471c01c460 con 0x7f47340d0430 2024-05-30T17:00:24.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.846+0000 7f472be00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f471006ddf0 0x7f47100702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:24.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.846+0000 7f472be00640 1 -- 172.21.15.67:0/2199247276 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5499+0+0 (secure 0 0 0) 0x7f471c08aac0 con 0x7f47340d0430 2024-05-30T17:00:24.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.846+0000 7f4733400640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f471006ddf0 0x7f47100702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:24.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.847+0000 7f4733400640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f471006ddf0 0x7f47100702b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4724033000 tx=0x7f4724009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:24.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:24.850+0000 7f472be00640 1 -- 172.21.15.67:0/2199247276 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f471c05a560 con 0x7f47340d0430 2024-05-30T17:00:25.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.066+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f473415b870 con 0x7f47340d0430 2024-05-30T17:00:25.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.600+0000 7f472be00640 1 -- 172.21.15.67:0/2199247276 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f471c059e60 con 0x7f47340d0430 2024-05-30T17:00:25.601 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-05-30T17:00:25.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.604+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f471006ddf0 msgr2=0x7f47100702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:25.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.604+0000 7f4738e00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f471006ddf0 0x7f47100702b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4724033000 tx=0x7f4724009000 comp rx=0 tx=0).stop 2024-05-30T17:00:25.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.604+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340d0430 msgr2=0x7f47341540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:25.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.604+0000 7f4738e00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340d0430 0x7f47341540d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f471c00c430 tx=0x7f471c00c8f0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.605+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 shutdown_connections 2024-05-30T17:00:25.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.605+0000 7f4738e00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f471006ddf0 0x7f47100702b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.605+0000 7f4738e00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47340d0860 0x7f4734154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.605+0000 7f4738e00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47340d0430 0x7f47341540d0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.605+0000 7f4738e00640 1 --2- 172.21.15.67:0/2199247276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47340ce560 0x7f47340af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.605+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 >> 172.21.15.67:0/2199247276 conn(0x7f473406bae0 msgr2=0x7f473415d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:25.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.605+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 shutdown_connections 2024-05-30T17:00:25.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.605+0000 7f4738e00640 1 -- 172.21.15.67:0/2199247276 wait complete. 2024-05-30T17:00:25.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-30T17:00:25.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.832+0000 7fef2fc00640 1 Processor -- start 2024-05-30T17:00:25.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.832+0000 7fef2fc00640 1 -- start start 2024-05-30T17:00:25.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.833+0000 7fef2fc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef280d0640 0x7fef280d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:25.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.833+0000 7fef2fc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fef280725e0 0x7fef280ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:25.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.833+0000 7fef2fc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef280ade90 con 0x7fef2815b6f0 2024-05-30T17:00:25.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.834+0000 7fef2fc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fef28071c40 con 0x7fef280725e0 2024-05-30T17:00:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.834+0000 7fef2fc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fef280d0f60 con 0x7fef280d0640 2024-05-30T17:00:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.834+0000 7fef27e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fef2815b6f0 0x7fef2806e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45678/0 (socket says 172.21.15.67:45678) 2024-05-30T17:00:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.834+0000 7fef27e00640 1 -- 172.21.15.67:0/3673603768 learned_addr learned my addr 172.21.15.67:0/3673603768 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.834+0000 7fef2dc00640 1 --2- 172.21.15.67:0/3673603768 >> v2:172.21.15.67:3300/0 conn(0x7fef280725e0 0x7fef280ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.834+0000 7fef27400640 1 -- 172.21.15.67:0/3673603768 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 605165412 0 0) 0x7fef280ade90 con 0x7fef2815b6f0 2024-05-30T17:00:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.835+0000 7fef27400640 1 -- 172.21.15.67:0/3673603768 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef0c003520 con 0x7fef2815b6f0 2024-05-30T17:00:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.835+0000 7fef2dc00640 1 -- 172.21.15.67:0/3673603768 >> v2:172.21.15.169:3300/0 conn(0x7fef280d0640 msgr2=0x7fef280d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.835+0000 7fef2d200640 1 --2- 172.21.15.67:0/3673603768 >> v2:172.21.15.169:3300/0 conn(0x7fef280d0640 0x7fef280d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.835+0000 7fef2dc00640 1 --2- 172.21.15.67:0/3673603768 >> v2:172.21.15.169:3300/0 conn(0x7fef280d0640 0x7fef280d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.835+0000 7fef2dc00640 1 -- 172.21.15.67:0/3673603768 >> v1:172.21.15.67:6789/0 conn(0x7fef2815b6f0 legacy=0x7fef2806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.835+0000 7fef2dc00640 1 -- 172.21.15.67:0/3673603768 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef28071e70 con 0x7fef280725e0 2024-05-30T17:00:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.835+0000 7fef2d200640 1 --2- 172.21.15.67:0/3673603768 >> v2:172.21.15.169:3300/0 conn(0x7fef280d0640 0x7fef280d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:25.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.836+0000 7fef2dc00640 1 --2- 172.21.15.67:0/3673603768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 0x7fef280ae650 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fef2806fe40 tx=0x7fef1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5effd2caded237b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:25.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.836+0000 7fef27400640 1 -- 172.21.15.67:0/3673603768 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18037070 con 0x7fef280725e0 2024-05-30T17:00:25.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.836+0000 7fef27400640 1 -- 172.21.15.67:0/3673603768 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef18032440 con 0x7fef280725e0 2024-05-30T17:00:25.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.836+0000 7fef27400640 1 -- 172.21.15.67:0/3673603768 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18032810 con 0x7fef280725e0 2024-05-30T17:00:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.837+0000 7fef2fc00640 1 -- 172.21.15.67:0/3673603768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 msgr2=0x7fef280ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.837+0000 7fef2fc00640 1 --2- 172.21.15.67:0/3673603768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 0x7fef280ae650 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fef2806fe40 tx=0x7fef1802a1a0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.837+0000 7fef2fc00640 1 -- 172.21.15.67:0/3673603768 shutdown_connections 2024-05-30T17:00:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.837+0000 7fef2fc00640 1 --2- 172.21.15.67:0/3673603768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 0x7fef280ae650 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.837+0000 7fef2fc00640 1 --2- 172.21.15.67:0/3673603768 >> v2:172.21.15.169:3300/0 conn(0x7fef280d0640 0x7fef280d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.837+0000 7fef2fc00640 1 -- 172.21.15.67:0/3673603768 >> 172.21.15.67:0/3673603768 conn(0x7fef2806bd20 msgr2=0x7fef2806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.837+0000 7fef2fc00640 1 -- 172.21.15.67:0/3673603768 shutdown_connections 2024-05-30T17:00:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.837+0000 7fef2fc00640 1 -- 172.21.15.67:0/3673603768 wait complete. 2024-05-30T17:00:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.840+0000 7fef2fc00640 1 Processor -- start 2024-05-30T17:00:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.840+0000 7fef2fc00640 1 -- start start 2024-05-30T17:00:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.841+0000 7fef2fc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 0x7fef280cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.841+0000 7fef2fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef280d0640 0x7fef2814ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.841+0000 7fef2fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2815b6f0 0x7fef280c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.841+0000 7fef2fc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fef280aeb90 con 0x7fef280725e0 2024-05-30T17:00:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.841+0000 7fef2fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef280ae130 con 0x7fef280d0640 2024-05-30T17:00:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.841+0000 7fef2fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fef280aed10 con 0x7fef2815b6f0 2024-05-30T17:00:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.841+0000 7fef2d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 0x7fef280cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.841+0000 7fef2d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 0x7fef280cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51276/0 (socket says 172.21.15.67:51276) 2024-05-30T17:00:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.841+0000 7fef2d200640 1 -- 172.21.15.67:0/3213973523 learned_addr learned my addr 172.21.15.67:0/3213973523 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.842+0000 7fef27e00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef280d0640 0x7fef2814ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.842+0000 7fef2d200640 1 -- 172.21.15.67:0/3213973523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2815b6f0 msgr2=0x7fef280c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.842+0000 7fef2d200640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2815b6f0 0x7fef280c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.842+0000 7fef2d200640 1 -- 172.21.15.67:0/3213973523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef280d0640 msgr2=0x7fef2814ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.842+0000 7fef2d200640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef280d0640 0x7fef2814ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.842+0000 7fef2d200640 1 -- 172.21.15.67:0/3213973523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef280cf720 con 0x7fef280725e0 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.842+0000 7fef2d200640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 0x7fef280cca60 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fef2000cf90 tx=0x7fef2000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.842+0000 7fef24c00640 1 -- 172.21.15.67:0/3213973523 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2001a070 con 0x7fef280725e0 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.842+0000 7fef24c00640 1 -- 172.21.15.67:0/3213973523 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2000d840 con 0x7fef280725e0 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.843+0000 7fef24c00640 1 -- 172.21.15.67:0/3213973523 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2000dc10 con 0x7fef280725e0 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.843+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2814d480 con 0x7fef280725e0 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.843+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2814da40 con 0x7fef280725e0 2024-05-30T17:00:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.844+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef280689e0 con 0x7fef280725e0 2024-05-30T17:00:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.845+0000 7fef24c00640 1 -- 172.21.15.67:0/3213973523 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fef2001d070 con 0x7fef280725e0 2024-05-30T17:00:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.846+0000 7fef24c00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0406ddc0 0x7fef04070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.846+0000 7fef24c00640 1 -- 172.21.15.67:0/3213973523 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5027+0+0 (secure 0 0 0) 0x7fef2008b000 con 0x7fef280725e0 2024-05-30T17:00:25.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.846+0000 7fef27e00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0406ddc0 0x7fef04070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:25.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.847+0000 7fef27e00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0406ddc0 0x7fef04070280 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fef2815eb40 tx=0x7fef10009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:25.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:25.853+0000 7fef24c00640 1 -- 172.21.15.67:0/3213973523 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef2005a830 con 0x7fef280725e0 2024-05-30T17:00:26.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.057+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fef280ad620 con 0x7fef280725e0 2024-05-30T17:00:26.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.065+0000 7fef24c00640 1 -- 172.21.15.67:0/3213973523 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fef2005eb30 con 0x7fef280725e0 2024-05-30T17:00:26.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0406ddc0 msgr2=0x7fef04070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:26.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0406ddc0 0x7fef04070280 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fef2815eb40 tx=0x7fef10009040 comp rx=0 tx=0).stop 2024-05-30T17:00:26.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 msgr2=0x7fef280cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:26.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 0x7fef280cca60 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fef2000cf90 tx=0x7fef2000a7d0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 shutdown_connections 2024-05-30T17:00:26.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef0406ddc0 0x7fef04070280 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef2815b6f0 0x7fef280c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef280d0640 0x7fef2814ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 --2- 172.21.15.67:0/3213973523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef280725e0 0x7fef280cca60 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 >> 172.21.15.67:0/3213973523 conn(0x7fef2806bd20 msgr2=0x7fef2815a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.068+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 shutdown_connections 2024-05-30T17:00:26.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.069+0000 7fef2fc00640 1 -- 172.21.15.67:0/3213973523 wait complete. 2024-05-30T17:00:26.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed' 2024-05-30T17:00:26.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.292+0000 7f7596400640 1 Processor -- start 2024-05-30T17:00:26.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.292+0000 7f7596400640 1 -- start start 2024-05-30T17:00:26.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.294+0000 7f7596400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759006f860 0x7f75900b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:26.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.294+0000 7f7596400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f759006ed10 con 0x7f75900b3690 2024-05-30T17:00:26.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.294+0000 7f7596400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f759015cb00 con 0x7f75900d0bb0 2024-05-30T17:00:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.294+0000 7f7596400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f75900b3cc0 con 0x7f759006f860 2024-05-30T17:00:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.294+0000 7f7586a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f75900b3690 0x7f75900d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45690/0 (socket says 172.21.15.67:45690) 2024-05-30T17:00:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.294+0000 7f7586a00640 1 -- 172.21.15.67:0/2002802376 learned_addr learned my addr 172.21.15.67:0/2002802376 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758f400640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759006f860 0x7f75900b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001205651 0 0) 0x7f7578001110 con 0x7f75900b3690 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7574003520 con 0x7f75900b3690 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3769210873 0 0) 0x7f7574003520 con 0x7f75900b3690 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759006f860 msgr2=0x7f75900b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759006f860 0x7f75900b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 >> v1:172.21.15.169:6789/0 conn(0x7f75900d0bb0 legacy=0x7f75900ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75900b3fa0 con 0x7f75900b3690 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7578001840 con 0x7f75900b3690 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 >> v1:172.21.15.67:6789/0 conn(0x7f75900b3690 legacy=0x7f75900d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75740039e0 0x7f7574003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:26.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7574004800 0x7f7574004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740051c0 0x7f757400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f75900b3cc0 con 0x7f75740039e0 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f757400c060 con 0x7f7574004800 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f757400c260 con 0x7f75740051c0 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.295+0000 7f758f400640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759006f860 0x7f75900b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.296+0000 7f758f400640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75740039e0 0x7f7574003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.296+0000 7f7586a00640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7574004800 0x7f7574004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.296+0000 7f758fe00640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740051c0 0x7f757400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.296+0000 7f758f400640 1 -- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740051c0 msgr2=0x7f757400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.296+0000 7f758f400640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740051c0 0x7f757400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.296+0000 7f758f400640 1 -- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7574004800 msgr2=0x7f7574004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.296+0000 7f758f400640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7574004800 0x7f7574004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.296+0000 7f758f400640 1 -- 172.21.15.67:0/2002802376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7574003520 con 0x7f75740039e0 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.297+0000 7f758f400640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75740039e0 0x7f7574003e00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7580006fd0 tx=0x7f7574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c3fa6c89cfb611 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580037070 con 0x7f75740039e0 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f758002aae0 con 0x7f75740039e0 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f758ea00640 1 -- 172.21.15.67:0/2002802376 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758000f040 con 0x7f75740039e0 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f7596400640 1 -- 172.21.15.67:0/2002802376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75740039e0 msgr2=0x7f7574003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f7596400640 1 --2- 172.21.15.67:0/2002802376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75740039e0 0x7f7574003e00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7580006fd0 tx=0x7f7574005830 comp rx=0 tx=0).stop 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f758fe00640 1 -- 172.21.15.67:0/2002802376 reap_dead start 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f7596400640 1 -- 172.21.15.67:0/2002802376 shutdown_connections 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f7596400640 1 -- 172.21.15.67:0/2002802376 >> 172.21.15.67:0/2002802376 conn(0x7f759006bae0 msgr2=0x7f759006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f7596400640 1 -- 172.21.15.67:0/2002802376 shutdown_connections 2024-05-30T17:00:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.298+0000 7f7596400640 1 -- 172.21.15.67:0/2002802376 wait complete. 2024-05-30T17:00:26.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.300+0000 7f7596400640 1 Processor -- start 2024-05-30T17:00:26.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.300+0000 7f7596400640 1 -- start start 2024-05-30T17:00:26.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.301+0000 7f7596400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740039e0 0x7f759015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.301+0000 7f7596400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759015f8d0 0x7f759015e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.301+0000 7f7596400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7590099c30 0x7f75901531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.301+0000 7f7596400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f75900b4840 con 0x7f7590099c30 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.301+0000 7f7596400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75900b4560 con 0x7f75740039e0 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.301+0000 7f7596400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f75900b3150 con 0x7f759015f8d0 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7590099c30 0x7f75901531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7590099c30 0x7f75901531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51296/0 (socket says 172.21.15.67:51296) 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758fe00640 1 -- 172.21.15.67:0/2837300132 learned_addr learned my addr 172.21.15.67:0/2837300132 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758f400640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740039e0 0x7f759015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f7586a00640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759015f8d0 0x7f759015e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758fe00640 1 -- 172.21.15.67:0/2837300132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759015f8d0 msgr2=0x7f759015e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758fe00640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759015f8d0 0x7f759015e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758fe00640 1 -- 172.21.15.67:0/2837300132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740039e0 msgr2=0x7f759015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758fe00640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740039e0 0x7f759015e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758fe00640 1 -- 172.21.15.67:0/2837300132 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75901545d0 con 0x7f7590099c30 2024-05-30T17:00:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.302+0000 7f758fe00640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7590099c30 0x7f75901531e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f75900d1360 tx=0x7f758800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.303+0000 7f7587e00640 1 -- 172.21.15.67:0/2837300132 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588003970 con 0x7f7590099c30 2024-05-30T17:00:26.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.303+0000 7f7587e00640 1 -- 172.21.15.67:0/2837300132 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7588018070 con 0x7f7590099c30 2024-05-30T17:00:26.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.303+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75901548b0 con 0x7f7590099c30 2024-05-30T17:00:26.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.303+0000 7f7587e00640 1 -- 172.21.15.67:0/2837300132 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588003d50 con 0x7f7590099c30 2024-05-30T17:00:26.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.303+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7590154e70 con 0x7f7590099c30 2024-05-30T17:00:26.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.304+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75900689e0 con 0x7f7590099c30 2024-05-30T17:00:26.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.305+0000 7f7587e00640 1 -- 172.21.15.67:0/2837300132 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f75880050d0 con 0x7f7590099c30 2024-05-30T17:00:26.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.306+0000 7f7587e00640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f756c06ddf0 0x7f756c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:26.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.306+0000 7f7587e00640 1 -- 172.21.15.67:0/2837300132 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5027+0+0 (secure 0 0 0) 0x7f758808a4d0 con 0x7f7590099c30 2024-05-30T17:00:26.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.306+0000 7f758f400640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f756c06ddf0 0x7f756c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:26.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.307+0000 7f758f400640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f756c06ddf0 0x7f756c0702b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7580034000 tx=0x7f758002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:26.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.311+0000 7f7587e00640 1 -- 172.21.15.67:0/2837300132 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f758805a3b0 con 0x7f7590099c30 2024-05-30T17:00:26.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:26.613+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]} v 0) -- 0x7f759006e2f0 con 0x7f7590099c30 2024-05-30T17:00:27.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.619+0000 7f7587e00640 1 -- 172.21.15.67:0/2837300132 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]}]=0 v694) ==== 153+0+0 (secure 0 0 0) 0x7f7588059cb0 con 0x7f7590099c30 2024-05-30T17:00:27.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f756c06ddf0 msgr2=0x7f756c0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f756c06ddf0 0x7f756c0702b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7580034000 tx=0x7f758002e000 comp rx=0 tx=0).stop 2024-05-30T17:00:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7590099c30 msgr2=0x7f75901531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7590099c30 0x7f75901531e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f75900d1360 tx=0x7f758800ec10 comp rx=0 tx=0).stop 2024-05-30T17:00:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 shutdown_connections 2024-05-30T17:00:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f756c06ddf0 0x7f756c0702b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7590099c30 0x7f75901531e0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f759015f8d0 0x7f759015e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 --2- 172.21.15.67:0/2837300132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740039e0 0x7f759015e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:27.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 >> 172.21.15.67:0/2837300132 conn(0x7f759006bae0 msgr2=0x7f759015b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:27.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.623+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 shutdown_connections 2024-05-30T17:00:27.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.624+0000 7f7596400640 1 -- 172.21.15.67:0/2837300132 wait complete. 2024-05-30T17:00:27.646 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-05-30T17:00:27.647 INFO:tasks.cephfs_test_runner:Starting test: test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar, that is printed for 1 ongoing clone job, is 2024-05-30T17:00:27.647 INFO:tasks.cephfs_test_runner:test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T17:00:27.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:00:27.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.862+0000 7f512da00640 1 Processor -- start 2024-05-30T17:00:27.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.863+0000 7f512da00640 1 -- start start 2024-05-30T17:00:27.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.864+0000 7f512da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5128158b80 0x7f512806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:27.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.864+0000 7f512da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51280722d0 0x7f51280b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:27.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.864+0000 7f512da00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512806e6e0 con 0x7f512806e160 2024-05-30T17:00:27.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.864+0000 7f512da00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f512806e9c0 con 0x7f5128158b80 2024-05-30T17:00:27.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.864+0000 7f512da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f512806eca0 con 0x7f51280722d0 2024-05-30T17:00:27.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.865+0000 7f5127e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5128158b80 0x7f512806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:27.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.865+0000 7f5127e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5128158b80 0x7f512806dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51302/0 (socket says 172.21.15.67:51302) 2024-05-30T17:00:27.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.865+0000 7f5127e00640 1 -- 172.21.15.67:0/3241672170 learned_addr learned my addr 172.21.15.67:0/3241672170 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:27.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.865+0000 7f512d000640 1 --2- 172.21.15.67:0/3241672170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51280722d0 0x7f51280b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:27.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.865+0000 7f5127e00640 1 -- 172.21.15.67:0/3241672170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51280722d0 msgr2=0x7f51280b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:27.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.865+0000 7f5127e00640 1 --2- 172.21.15.67:0/3241672170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51280722d0 0x7f51280b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:27.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.865+0000 7f5127e00640 1 -- 172.21.15.67:0/3241672170 >> v1:172.21.15.169:6789/0 conn(0x7f512806e160 legacy=0x7f5128071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:27.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.865+0000 7f5127e00640 1 -- 172.21.15.67:0/3241672170 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512806ef80 con 0x7f5128158b80 2024-05-30T17:00:27.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.866+0000 7f5127e00640 1 --2- 172.21.15.67:0/3241672170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5128158b80 0x7f512806dc20 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5118009870 tx=0x7f511802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356e4379de67a95f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:27.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.867+0000 7f5127400640 1 -- 172.21.15.67:0/3241672170 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118037070 con 0x7f5128158b80 2024-05-30T17:00:27.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.867+0000 7f5127400640 1 -- 172.21.15.67:0/3241672170 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5118032440 con 0x7f5128158b80 2024-05-30T17:00:27.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.867+0000 7f5127400640 1 -- 172.21.15.67:0/3241672170 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118032810 con 0x7f5128158b80 2024-05-30T17:00:27.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.867+0000 7f512da00640 1 -- 172.21.15.67:0/3241672170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5128158b80 msgr2=0x7f512806dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:27.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.867+0000 7f512da00640 1 --2- 172.21.15.67:0/3241672170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5128158b80 0x7f512806dc20 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5118009870 tx=0x7f511802a210 comp rx=0 tx=0).stop 2024-05-30T17:00:27.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.868+0000 7f512da00640 1 -- 172.21.15.67:0/3241672170 shutdown_connections 2024-05-30T17:00:27.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.868+0000 7f512da00640 1 --2- 172.21.15.67:0/3241672170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51280722d0 0x7f51280b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:27.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.868+0000 7f512da00640 1 --2- 172.21.15.67:0/3241672170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5128158b80 0x7f512806dc20 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:27.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.868+0000 7f512da00640 1 -- 172.21.15.67:0/3241672170 >> 172.21.15.67:0/3241672170 conn(0x7f51280b1fc0 msgr2=0x7f51280b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:27.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.868+0000 7f512da00640 1 -- 172.21.15.67:0/3241672170 shutdown_connections 2024-05-30T17:00:27.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.868+0000 7f512da00640 1 -- 172.21.15.67:0/3241672170 wait complete. 2024-05-30T17:00:27.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.870+0000 7f512da00640 1 Processor -- start 2024-05-30T17:00:27.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.870+0000 7f512da00640 1 -- start start 2024-05-30T17:00:27.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.871+0000 7f512da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f512806e160 0x7f512809ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:27.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.871+0000 7f512da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51280722d0 0x7f512809cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:27.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.871+0000 7f512da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5128158b80 0x7f5128147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:27.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.871+0000 7f512da00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f512806f780 con 0x7f51280722d0 2024-05-30T17:00:27.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.871+0000 7f512da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f51280b2d30 con 0x7f512806e160 2024-05-30T17:00:27.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.871+0000 7f512da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5128145610 con 0x7f5128158b80 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.871+0000 7f511f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51280722d0 0x7f512809cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f511f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51280722d0 0x7f512809cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51312/0 (socket says 172.21.15.67:51312) 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f511f400640 1 -- 172.21.15.67:0/374274767 learned_addr learned my addr 172.21.15.67:0/374274767 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f5127e00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f512806e160 0x7f512809ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f512d000640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5128158b80 0x7f5128147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f511f400640 1 -- 172.21.15.67:0/374274767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5128158b80 msgr2=0x7f5128147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f511f400640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5128158b80 0x7f5128147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f511f400640 1 -- 172.21.15.67:0/374274767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f512806e160 msgr2=0x7f512809ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f511f400640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f512806e160 0x7f512809ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f511f400640 1 -- 172.21.15.67:0/374274767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51281429d0 con 0x7f51280722d0 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f5127e00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f512806e160 0x7f512809ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:27.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.872+0000 7f511f400640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51280722d0 0x7f512809cf40 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f511000c430 tx=0x7f511000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:27.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.873+0000 7f5124c00640 1 -- 172.21.15.67:0/374274767 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110011790 con 0x7f51280722d0 2024-05-30T17:00:27.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.873+0000 7f5124c00640 1 -- 172.21.15.67:0/374274767 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5110017070 con 0x7f51280722d0 2024-05-30T17:00:27.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.873+0000 7f5124c00640 1 -- 172.21.15.67:0/374274767 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110011b70 con 0x7f51280722d0 2024-05-30T17:00:27.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.873+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5128142cb0 con 0x7f51280722d0 2024-05-30T17:00:27.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.873+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51281482a0 con 0x7f51280722d0 2024-05-30T17:00:27.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.874+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51280689e0 con 0x7f51280722d0 2024-05-30T17:00:27.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.875+0000 7f5124c00640 1 -- 172.21.15.67:0/374274767 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f5110011d10 con 0x7f51280722d0 2024-05-30T17:00:27.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.876+0000 7f5124c00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f510406ddc0 0x7f5104070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:27.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.876+0000 7f5124c00640 1 -- 172.21.15.67:0/374274767 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5027+0+0 (secure 0 0 0) 0x7f511008ab80 con 0x7f51280722d0 2024-05-30T17:00:27.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.876+0000 7f5127e00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f510406ddc0 0x7f5104070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:27.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.877+0000 7f5127e00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f510406ddc0 0x7f5104070280 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5118034000 tx=0x7f511802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:27.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:27.881+0000 7f5124c00640 1 -- 172.21.15.67:0/374274767 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f511005a590 con 0x7f51280722d0 2024-05-30T17:00:28.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.091+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51280cfaf0 con 0x7f51280722d0 2024-05-30T17:00:28.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.093+0000 7f5124c00640 1 -- 172.21.15.67:0/374274767 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+13143 (secure 0 0 0) 0x7f5110059e90 con 0x7f51280722d0 2024-05-30T17:00:28.093 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:28.094 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":95,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:00:25.597104+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6834/2733959172":"2024-05-31T17:00:22.204740+0000","172.21.15.169:6833/2733959172":"2024-05-31T17:00:22.204740+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:00:28.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f510406ddc0 msgr2=0x7f5104070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f510406ddc0 0x7f5104070280 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5118034000 tx=0x7f511802e000 comp rx=0 tx=0).stop 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51280722d0 msgr2=0x7f512809cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51280722d0 0x7f512809cf40 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f511000c430 tx=0x7f511000c8f0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 shutdown_connections 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f510406ddc0 0x7f5104070280 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5128158b80 0x7f5128147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51280722d0 0x7f512809cf40 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 --2- 172.21.15.67:0/374274767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f512806e160 0x7f512809ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 >> 172.21.15.67:0/374274767 conn(0x7f51280b1fc0 msgr2=0x7f51280d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 shutdown_connections 2024-05-30T17:00:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.096+0000 7f512da00640 1 -- 172.21.15.67:0/374274767 wait complete. 2024-05-30T17:00:28.117 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-30T16:43:40.009725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-30T17:00:28.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-30T17:00:28.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.330+0000 7fd59f200640 1 Processor -- start 2024-05-30T17:00:28.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.330+0000 7fd59f200640 1 -- start start 2024-05-30T17:00:28.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.332+0000 7fd59f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 0x7fd5a00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.332+0000 7fd59f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5a00ccb90 0x7fd5a00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.332+0000 7fd59f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5a015d900 con 0x7fd5a015cb60 2024-05-30T17:00:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.332+0000 7fd59f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd5a01538f0 con 0x7fd5a00ccb90 2024-05-30T17:00:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.332+0000 7fd59f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a00cd530 con 0x7fd5a0153510 2024-05-30T17:00:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.332+0000 7fd59d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5a00ccb90 0x7fd5a00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.332+0000 7fd59d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5a00ccb90 0x7fd5a00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51326/0 (socket says 172.21.15.67:51326) 2024-05-30T17:00:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.332+0000 7fd59d400640 1 -- 172.21.15.67:0/1999450425 learned_addr learned my addr 172.21.15.67:0/1999450425 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.333+0000 7fd59de00640 1 --2- 172.21.15.67:0/1999450425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 0x7fd5a00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.333+0000 7fd59d400640 1 -- 172.21.15.67:0/1999450425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 msgr2=0x7fd5a00b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.333+0000 7fd59d400640 1 --2- 172.21.15.67:0/1999450425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 0x7fd5a00b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.333+0000 7fd59d400640 1 -- 172.21.15.67:0/1999450425 >> v1:172.21.15.169:6789/0 conn(0x7fd5a015cb60 legacy=0x7fd5a015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:28.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.333+0000 7fd59d400640 1 -- 172.21.15.67:0/1999450425 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a00b3f90 con 0x7fd5a00ccb90 2024-05-30T17:00:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.334+0000 7fd59d400640 1 --2- 172.21.15.67:0/1999450425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a00ccb90 0x7fd5a00ccff0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009870 tx=0x7fd58c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397f1a50a935ccb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.334+0000 7fd59ca00640 1 -- 172.21.15.67:0/1999450425 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c037070 con 0x7fd5a00ccb90 2024-05-30T17:00:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.334+0000 7fd59ca00640 1 -- 172.21.15.67:0/1999450425 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58c032440 con 0x7fd5a00ccb90 2024-05-30T17:00:28.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.334+0000 7fd59ca00640 1 -- 172.21.15.67:0/1999450425 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c032810 con 0x7fd5a00ccb90 2024-05-30T17:00:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.335+0000 7fd59f200640 1 -- 172.21.15.67:0/1999450425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a00ccb90 msgr2=0x7fd5a00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.335+0000 7fd59f200640 1 --2- 172.21.15.67:0/1999450425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a00ccb90 0x7fd5a00ccff0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009870 tx=0x7fd58c02a180 comp rx=0 tx=0).stop 2024-05-30T17:00:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.335+0000 7fd59f200640 1 -- 172.21.15.67:0/1999450425 shutdown_connections 2024-05-30T17:00:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.335+0000 7fd59f200640 1 --2- 172.21.15.67:0/1999450425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a00ccb90 0x7fd5a00ccff0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.335+0000 7fd59f200640 1 --2- 172.21.15.67:0/1999450425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 0x7fd5a00b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.335+0000 7fd59f200640 1 -- 172.21.15.67:0/1999450425 >> 172.21.15.67:0/1999450425 conn(0x7fd5a006b900 msgr2=0x7fd5a006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.335+0000 7fd59f200640 1 -- 172.21.15.67:0/1999450425 shutdown_connections 2024-05-30T17:00:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.336+0000 7fd59f200640 1 -- 172.21.15.67:0/1999450425 wait complete. 2024-05-30T17:00:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.338+0000 7fd59f200640 1 Processor -- start 2024-05-30T17:00:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.338+0000 7fd59f200640 1 -- start start 2024-05-30T17:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.339+0000 7fd59f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a00ccb90 0x7fd5a0072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.339+0000 7fd59f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 0x7fd5a01e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a015cb60 0x7fd5a01e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a00b4550 con 0x7fd5a015cb60 2024-05-30T17:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a00b48c0 con 0x7fd5a00ccb90 2024-05-30T17:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a006e940 con 0x7fd5a0153510 2024-05-30T17:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a015cb60 0x7fd5a01e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a015cb60 0x7fd5a01e62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51334/0 (socket says 172.21.15.67:51334) 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59e800640 1 -- 172.21.15.67:0/132047023 learned_addr learned my addr 172.21.15.67:0/132047023 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59de00640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a00ccb90 0x7fd5a0072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59d400640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 0x7fd5a01e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59e800640 1 -- 172.21.15.67:0/132047023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 msgr2=0x7fd5a01e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59e800640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 0x7fd5a01e5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59e800640 1 -- 172.21.15.67:0/132047023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a00ccb90 msgr2=0x7fd5a0072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59e800640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a00ccb90 0x7fd5a0072e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59e800640 1 -- 172.21.15.67:0/132047023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a01e6950 con 0x7fd5a015cb60 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.340+0000 7fd59d400640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 0x7fd5a01e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.341+0000 7fd59e800640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a015cb60 0x7fd5a01e62b0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd59800b9c0 tx=0x7fd59800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.341+0000 7fd596000640 1 -- 172.21.15.67:0/132047023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598019070 con 0x7fd5a015cb60 2024-05-30T17:00:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.341+0000 7fd596000640 1 -- 172.21.15.67:0/132047023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd59800cba0 con 0x7fd5a015cb60 2024-05-30T17:00:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.341+0000 7fd596000640 1 -- 172.21.15.67:0/132047023 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598014440 con 0x7fd5a015cb60 2024-05-30T17:00:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.341+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a00ae300 con 0x7fd5a015cb60 2024-05-30T17:00:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.341+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a00ae8c0 con 0x7fd5a015cb60 2024-05-30T17:00:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.343+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a00689e0 con 0x7fd5a015cb60 2024-05-30T17:00:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.344+0000 7fd596000640 1 -- 172.21.15.67:0/132047023 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fd59801c070 con 0x7fd5a015cb60 2024-05-30T17:00:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.344+0000 7fd596000640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006ddc0 0x7fd580070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.345+0000 7fd596000640 1 -- 172.21.15.67:0/132047023 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5027+0+0 (secure 0 0 0) 0x7fd59808a600 con 0x7fd5a015cb60 2024-05-30T17:00:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.345+0000 7fd59de00640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006ddc0 0x7fd580070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.345+0000 7fd59de00640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006ddc0 0x7fd580070280 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd590006fd0 tx=0x7fd590008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.349+0000 7fd596000640 1 -- 172.21.15.67:0/132047023 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd59805a480 con 0x7fd5a015cb60 2024-05-30T17:00:28.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.557+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5a00b3c40 con 0x7fd5a015cb60 2024-05-30T17:00:28.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.557+0000 7fd596000640 1 -- 172.21.15.67:0/132047023 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fd598059d80 con 0x7fd5a015cb60 2024-05-30T17:00:28.558 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-05-30T17:00:28.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006ddc0 msgr2=0x7fd580070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006ddc0 0x7fd580070280 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd590006fd0 tx=0x7fd590008040 comp rx=0 tx=0).stop 2024-05-30T17:00:28.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a015cb60 msgr2=0x7fd5a01e62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a015cb60 0x7fd5a01e62b0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd59800b9c0 tx=0x7fd59800be80 comp rx=0 tx=0).stop 2024-05-30T17:00:28.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 shutdown_connections 2024-05-30T17:00:28.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd58006ddc0 0x7fd580070280 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5a015cb60 0x7fd5a01e62b0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5a0153510 0x7fd5a01e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 --2- 172.21.15.67:0/132047023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5a00ccb90 0x7fd5a0072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 >> 172.21.15.67:0/132047023 conn(0x7fd5a006b900 msgr2=0x7fd5a00afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:28.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 shutdown_connections 2024-05-30T17:00:28.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.560+0000 7fd59f200640 1 -- 172.21.15.67:0/132047023 wait complete. 2024-05-30T17:00:28.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled' 2024-05-30T17:00:28.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.785+0000 7fef5be00640 1 Processor -- start 2024-05-30T17:00:28.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.785+0000 7fef5be00640 1 -- start start 2024-05-30T17:00:28.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.787+0000 7fef5be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fef5c0d16a0 0x7fef5c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.787+0000 7fef5be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef5c0cf870 0x7fef5c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.787+0000 7fef5be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef5c0cfce0 con 0x7fef5c15e870 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.787+0000 7fef5be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fef5c0d2150 con 0x7fef5c0d16a0 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.787+0000 7fef5be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fef5c15ec50 con 0x7fef5c0cf870 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.787+0000 7fef52000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fef5c0d16a0 0x7fef5c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.788+0000 7fef52000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fef5c0d16a0 0x7fef5c0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51346/0 (socket says 172.21.15.67:51346) 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.788+0000 7fef52000640 1 -- 172.21.15.67:0/2068266359 learned_addr learned my addr 172.21.15.67:0/2068266359 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.788+0000 7fef5b400640 1 --2- 172.21.15.67:0/2068266359 >> v2:172.21.15.169:3300/0 conn(0x7fef5c0cf870 0x7fef5c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.788+0000 7fef52000640 1 -- 172.21.15.67:0/2068266359 >> v2:172.21.15.169:3300/0 conn(0x7fef5c0cf870 msgr2=0x7fef5c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.788+0000 7fef52000640 1 --2- 172.21.15.67:0/2068266359 >> v2:172.21.15.169:3300/0 conn(0x7fef5c0cf870 0x7fef5c0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.788+0000 7fef52000640 1 -- 172.21.15.67:0/2068266359 >> v1:172.21.15.169:6789/0 conn(0x7fef5c15e870 legacy=0x7fef5c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.788+0000 7fef52000640 1 -- 172.21.15.67:0/2068266359 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c0cff80 con 0x7fef5c0d16a0 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.789+0000 7fef52000640 1 --2- 172.21.15.67:0/2068266359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0d16a0 0x7fef5c0cf330 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fef5c163400 tx=0x7fef4402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8d76e73ac17f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.789+0000 7fef5a000640 1 -- 172.21.15.67:0/2068266359 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44036070 con 0x7fef5c0d16a0 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.789+0000 7fef5a000640 1 -- 172.21.15.67:0/2068266359 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef44031600 con 0x7fef5c0d16a0 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.790+0000 7fef5a000640 1 -- 172.21.15.67:0/2068266359 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef440319d0 con 0x7fef5c0d16a0 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.790+0000 7fef5be00640 1 -- 172.21.15.67:0/2068266359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0d16a0 msgr2=0x7fef5c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.790+0000 7fef5be00640 1 --2- 172.21.15.67:0/2068266359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0d16a0 0x7fef5c0cf330 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fef5c163400 tx=0x7fef4402a730 comp rx=0 tx=0).stop 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.790+0000 7fef5be00640 1 -- 172.21.15.67:0/2068266359 shutdown_connections 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.790+0000 7fef5be00640 1 --2- 172.21.15.67:0/2068266359 >> v2:172.21.15.169:3300/0 conn(0x7fef5c0cf870 0x7fef5c0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.790+0000 7fef5be00640 1 --2- 172.21.15.67:0/2068266359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0d16a0 0x7fef5c0cf330 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.790+0000 7fef5be00640 1 -- 172.21.15.67:0/2068266359 >> 172.21.15.67:0/2068266359 conn(0x7fef5c08bfc0 msgr2=0x7fef5c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.790+0000 7fef5be00640 1 -- 172.21.15.67:0/2068266359 shutdown_connections 2024-05-30T17:00:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.790+0000 7fef5be00640 1 -- 172.21.15.67:0/2068266359 wait complete. 2024-05-30T17:00:28.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.792+0000 7fef5be00640 1 Processor -- start 2024-05-30T17:00:28.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.792+0000 7fef5be00640 1 -- start start 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.793+0000 7fef5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0cf870 0x7fef5c1f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.793+0000 7fef5be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c0d16a0 0x7fef5c153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.793+0000 7fef5be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef5c15e870 0x7fef5c0b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.793+0000 7fef5be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c0d0820 con 0x7fef5c0cf870 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.793+0000 7fef5be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c0d0540 con 0x7fef5c0d16a0 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.793+0000 7fef5be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fef5c0d4fb0 con 0x7fef5c15e870 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0cf870 0x7fef5c1f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0cf870 0x7fef5c1f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51360/0 (socket says 172.21.15.67:51360) 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef5aa00640 1 -- 172.21.15.67:0/539923226 learned_addr learned my addr 172.21.15.67:0/539923226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef52000640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c0d16a0 0x7fef5c153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef5b400640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef5c15e870 0x7fef5c0b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef5aa00640 1 -- 172.21.15.67:0/539923226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef5c15e870 msgr2=0x7fef5c0b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef5aa00640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef5c15e870 0x7fef5c0b8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef5aa00640 1 -- 172.21.15.67:0/539923226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c0d16a0 msgr2=0x7fef5c153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef5aa00640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c0d16a0 0x7fef5c153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef5aa00640 1 -- 172.21.15.67:0/539923226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c0b8260 con 0x7fef5c0cf870 2024-05-30T17:00:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.794+0000 7fef52000640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c0d16a0 0x7fef5c153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.795+0000 7fef5aa00640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0cf870 0x7fef5c1f5700 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c430 tx=0x7fef4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.795+0000 7fef53400640 1 -- 172.21.15.67:0/539923226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c011790 con 0x7fef5c0cf870 2024-05-30T17:00:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.795+0000 7fef53400640 1 -- 172.21.15.67:0/539923226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4c017070 con 0x7fef5c0cf870 2024-05-30T17:00:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.795+0000 7fef53400640 1 -- 172.21.15.67:0/539923226 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c011bb0 con 0x7fef5c0cf870 2024-05-30T17:00:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.795+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c1e8290 con 0x7fef5c0cf870 2024-05-30T17:00:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.795+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c1e8530 con 0x7fef5c0cf870 2024-05-30T17:00:28.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.796+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c0689e0 con 0x7fef5c0cf870 2024-05-30T17:00:28.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.797+0000 7fef53400640 1 -- 172.21.15.67:0/539923226 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fef4c01c460 con 0x7fef5c0cf870 2024-05-30T17:00:28.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.798+0000 7fef53400640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef3806ddc0 0x7fef38070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.798+0000 7fef53400640 1 -- 172.21.15.67:0/539923226 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5027+0+0 (secure 0 0 0) 0x7fef4c08ab50 con 0x7fef5c0cf870 2024-05-30T17:00:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.798+0000 7fef52000640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef3806ddc0 0x7fef38070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.799+0000 7fef52000640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef3806ddc0 0x7fef38070280 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fef44033000 tx=0x7fef44009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:28.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:28.802+0000 7fef53400640 1 -- 172.21.15.67:0/539923226 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef4c05a5f0 con 0x7fef5c0cf870 2024-05-30T17:00:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.089+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]} v 0) -- 0x7fef5c0ba500 con 0x7fef5c0cf870 2024-05-30T17:00:29.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.632+0000 7fef53400640 1 -- 172.21.15.67:0/539923226 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]}]=0 v696) ==== 164+0+0 (secure 0 0 0) 0x7fef4c059ef0 con 0x7fef5c0cf870 2024-05-30T17:00:29.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef3806ddc0 msgr2=0x7fef38070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef3806ddc0 0x7fef38070280 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fef44033000 tx=0x7fef44009000 comp rx=0 tx=0).stop 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0cf870 msgr2=0x7fef5c1f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0cf870 0x7fef5c1f5700 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c430 tx=0x7fef4c00c8f0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 shutdown_connections 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fef3806ddc0 0x7fef38070280 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef5c15e870 0x7fef5c0b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef5c0d16a0 0x7fef5c153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 --2- 172.21.15.67:0/539923226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef5c0cf870 0x7fef5c1f5700 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 >> 172.21.15.67:0/539923226 conn(0x7fef5c08bfc0 msgr2=0x7fef5c0d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 shutdown_connections 2024-05-30T17:00:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.636+0000 7fef5be00640 1 -- 172.21.15.67:0/539923226 wait complete. 2024-05-30T17:00:29.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T17:00:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.873+0000 7f0716200640 1 Processor -- start 2024-05-30T17:00:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.874+0000 7f0716200640 1 -- start start 2024-05-30T17:00:29.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.875+0000 7f0716200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0710153510 0x7f07100b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.875+0000 7f0716200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f071015d900 con 0x7f07100ccb90 2024-05-30T17:00:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.875+0000 7f0716200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07100b3f90 con 0x7f071015d270 2024-05-30T17:00:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.875+0000 7f0716200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07101538f0 con 0x7f0710153510 2024-05-30T17:00:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.876+0000 7f070fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f07100ccb90 0x7f071015cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45698/0 (socket says 172.21.15.67:45698) 2024-05-30T17:00:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.876+0000 7f070fe00640 1 -- 172.21.15.67:0/3502648508 learned_addr learned my addr 172.21.15.67:0/3502648508 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.876+0000 7f0714e00640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0710153510 0x7f07100b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3785873772 0 0) 0x7f071015d900 con 0x7f07100ccb90 2024-05-30T17:00:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06f8003520 con 0x7f07100ccb90 2024-05-30T17:00:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3424970269 0 0) 0x7f06f8003520 con 0x7f07100ccb90 2024-05-30T17:00:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0710153510 msgr2=0x7f07100b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0710153510 0x7f07100b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 >> v1:172.21.15.169:6789/0 conn(0x7f071015d270 legacy=0x7f071006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07100b4270 con 0x7f07100ccb90 2024-05-30T17:00:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f06fc001670 con 0x7f07100ccb90 2024-05-30T17:00:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 >> v1:172.21.15.67:6789/0 conn(0x7f07100ccb90 legacy=0x7f071015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f80039e0 0x7f06f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0710153510 0x7f06f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06f8004d40 0x7f06f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f06f800bac0 con 0x7f06f80039e0 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f06f800bc70 con 0x7f06f8004d40 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f06f800be70 con 0x7f0710153510 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.877+0000 7f0714e00640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f80039e0 0x7f06f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.878+0000 7f0714e00640 1 -- 172.21.15.67:0/3502648508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0710153510 msgr2=0x7f06f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.878+0000 7f0714e00640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0710153510 0x7f06f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.878+0000 7f0714e00640 1 -- 172.21.15.67:0/3502648508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06f8004d40 msgr2=0x7f06f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.878+0000 7f0714e00640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06f8004d40 0x7f06f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.878+0000 7f0714e00640 1 -- 172.21.15.67:0/3502648508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f8003520 con 0x7f06f80039e0 2024-05-30T17:00:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.879+0000 7f0714e00640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f80039e0 0x7f06f8003e00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0704009ac0 tx=0x7f06f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5355624947fb61d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.879+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704036070 con 0x7f06f80039e0 2024-05-30T17:00:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.879+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0704031400 con 0x7f06f80039e0 2024-05-30T17:00:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.879+0000 7f070f400640 1 -- 172.21.15.67:0/3502648508 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704009690 con 0x7f06f80039e0 2024-05-30T17:00:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.880+0000 7f0716200640 1 -- 172.21.15.67:0/3502648508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f80039e0 msgr2=0x7f06f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.880+0000 7f0716200640 1 --2- 172.21.15.67:0/3502648508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06f80039e0 0x7f06f8003e00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0704009ac0 tx=0x7f06f8005830 comp rx=0 tx=0).stop 2024-05-30T17:00:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.880+0000 7f0715800640 1 -- 172.21.15.67:0/3502648508 reap_dead start 2024-05-30T17:00:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.880+0000 7f0716200640 1 -- 172.21.15.67:0/3502648508 shutdown_connections 2024-05-30T17:00:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.880+0000 7f0716200640 1 -- 172.21.15.67:0/3502648508 >> 172.21.15.67:0/3502648508 conn(0x7f071006b900 msgr2=0x7f071006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.880+0000 7f0716200640 1 -- 172.21.15.67:0/3502648508 shutdown_connections 2024-05-30T17:00:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.880+0000 7f0716200640 1 -- 172.21.15.67:0/3502648508 wait complete. 2024-05-30T17:00:29.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.882+0000 7f0716200640 1 Processor -- start 2024-05-30T17:00:29.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.883+0000 7f0716200640 1 -- start start 2024-05-30T17:00:29.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.884+0000 7f0716200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06f80039e0 0x7f0710151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:29.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.884+0000 7f0716200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0710071a80 0x7f07101f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:29.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.884+0000 7f0716200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0710071f00 0x7f07101f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.884+0000 7f0716200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f07100b4d50 con 0x7f0710071f00 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.884+0000 7f0716200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07100b48c0 con 0x7f0710071a80 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.884+0000 7f0716200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07100b4ed0 con 0x7f06f80039e0 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.884+0000 7f0715800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0710071f00 0x7f07101f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.884+0000 7f0715800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0710071f00 0x7f07101f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51388/0 (socket says 172.21.15.67:51388) 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.884+0000 7f0715800640 1 -- 172.21.15.67:0/812714850 learned_addr learned my addr 172.21.15.67:0/812714850 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f0714e00640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06f80039e0 0x7f0710151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f0715800640 1 -- 172.21.15.67:0/812714850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06f80039e0 msgr2=0x7f0710151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f0715800640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06f80039e0 0x7f0710151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f0715800640 1 -- 172.21.15.67:0/812714850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0710071a80 msgr2=0x7f07101f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:29.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f0715800640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0710071a80 0x7f07101f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:29.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f0715800640 1 -- 172.21.15.67:0/812714850 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07100ad2d0 con 0x7f0710071f00 2024-05-30T17:00:29.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f0715800640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0710071f00 0x7f07101f4930 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f071015d650 tx=0x7f070800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:29.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f070cc00640 1 -- 172.21.15.67:0/812714850 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708017070 con 0x7f0710071f00 2024-05-30T17:00:29.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f070cc00640 1 -- 172.21.15.67:0/812714850 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0708005d60 con 0x7f0710071f00 2024-05-30T17:00:29.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f070cc00640 1 -- 172.21.15.67:0/812714850 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070800a850 con 0x7f0710071f00 2024-05-30T17:00:29.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.885+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07100ad5b0 con 0x7f0710071f00 2024-05-30T17:00:29.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.886+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07100adb70 con 0x7f0710071f00 2024-05-30T17:00:29.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.887+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07100681e0 con 0x7f0710071f00 2024-05-30T17:00:29.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.888+0000 7f070cc00640 1 -- 172.21.15.67:0/812714850 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f070801a070 con 0x7f0710071f00 2024-05-30T17:00:29.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.889+0000 7f070cc00640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f06f006ddc0 0x7f06f0070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:29.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.889+0000 7f070cc00640 1 -- 172.21.15.67:0/812714850 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5027+0+0 (secure 0 0 0) 0x7f0708089b80 con 0x7f0710071f00 2024-05-30T17:00:29.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.889+0000 7f0714e00640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f06f006ddc0 0x7f06f0070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:29.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.890+0000 7f0714e00640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f06f006ddc0 0x7f06f0070280 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0704033000 tx=0x7f0704008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:29.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:29.893+0000 7f070cc00640 1 -- 172.21.15.67:0/812714850 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07100681e0 con 0x7f0710071f00 2024-05-30T17:00:30.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.101+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f07100add70 con 0x7f0710071f00 2024-05-30T17:00:30.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.106+0000 7f070cc00640 1 -- 172.21.15.67:0/812714850 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f07100add70 con 0x7f0710071f00 2024-05-30T17:00:30.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.109+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f06f006ddc0 msgr2=0x7f06f0070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:30.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.109+0000 7f0716200640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f06f006ddc0 0x7f06f0070280 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0704033000 tx=0x7f0704008030 comp rx=0 tx=0).stop 2024-05-30T17:00:30.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.109+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0710071f00 msgr2=0x7f07101f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:30.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.109+0000 7f0716200640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0710071f00 0x7f07101f4930 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f071015d650 tx=0x7f070800cb90 comp rx=0 tx=0).stop 2024-05-30T17:00:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.109+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 shutdown_connections 2024-05-30T17:00:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.110+0000 7f0716200640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f06f006ddc0 0x7f06f0070280 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.110+0000 7f0716200640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0710071f00 0x7f07101f4930 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.110+0000 7f0716200640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0710071a80 0x7f07101f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.110+0000 7f0716200640 1 --2- 172.21.15.67:0/812714850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06f80039e0 0x7f0710151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.110+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 >> 172.21.15.67:0/812714850 conn(0x7f071006b900 msgr2=0x7f071006f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.110+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 shutdown_connections 2024-05-30T17:00:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.110+0000 7f0716200640 1 -- 172.21.15.67:0/812714850 wait complete. 2024-05-30T17:00:30.132 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T17:00:30.132 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:00:30.242 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:00:30.350 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T17:00:30.350 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:00:30.350 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:30.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:00:30.411 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:00:30.412 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:30.469 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:00:30.469 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:00:30.469 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:30.530 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:00:30.531 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:00:30.531 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:30.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:00:30.589 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-30T17:00:30.589 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:00:30.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.812+0000 7fab62400640 1 Processor -- start 2024-05-30T17:00:30.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.813+0000 7fab62400640 1 -- start start 2024-05-30T17:00:30.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.814+0000 7fab62400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab5c156b30 0x7fab5c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:30.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.814+0000 7fab62400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fab5c06ee40 0x7fab5c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:30.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.814+0000 7fab62400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c06f2f0 0x7fab5c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:30.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.814+0000 7fab62400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fab5c156f10 con 0x7fab5c156b30 2024-05-30T17:00:30.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.814+0000 7fab62400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fab5c06dda0 con 0x7fab5c06ee40 2024-05-30T17:00:30.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.814+0000 7fab62400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c06e080 con 0x7fab5c06f2f0 2024-05-30T17:00:30.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.815+0000 7fab61000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab5c156b30 0x7fab5c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:30.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.815+0000 7fab61000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab5c156b30 0x7fab5c0af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43278/0 (socket says 172.21.15.67:43278) 2024-05-30T17:00:30.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.815+0000 7fab61000640 1 -- 172.21.15.67:0/130330863 learned_addr learned my addr 172.21.15.67:0/130330863 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.815+0000 7fab61a00640 1 --2- 172.21.15.67:0/130330863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c06f2f0 0x7fab5c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.815+0000 7fab61000640 1 -- 172.21.15.67:0/130330863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c06f2f0 msgr2=0x7fab5c0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.815+0000 7fab61000640 1 --2- 172.21.15.67:0/130330863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c06f2f0 0x7fab5c0ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.815+0000 7fab61000640 1 -- 172.21.15.67:0/130330863 >> v2:172.21.15.169:3300/0 conn(0x7fab5c06ee40 msgr2=0x7fab5c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.815+0000 7fab61000640 1 --2- 172.21.15.67:0/130330863 >> v2:172.21.15.169:3300/0 conn(0x7fab5c06ee40 0x7fab5c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.815+0000 7fab61000640 1 -- 172.21.15.67:0/130330863 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c06e360 con 0x7fab5c156b30 2024-05-30T17:00:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.816+0000 7fab61000640 1 --2- 172.21.15.67:0/130330863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c156b30 0x7fab5c0af6e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fab5c06fdf0 tx=0x7fab5002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43816f703f71d237 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:30.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.817+0000 7fab5b400640 1 -- 172.21.15.67:0/130330863 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50037070 con 0x7fab5c156b30 2024-05-30T17:00:30.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.817+0000 7fab5b400640 1 -- 172.21.15.67:0/130330863 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab50032440 con 0x7fab5c156b30 2024-05-30T17:00:30.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.817+0000 7fab5b400640 1 -- 172.21.15.67:0/130330863 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50032810 con 0x7fab5c156b30 2024-05-30T17:00:30.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.817+0000 7fab62400640 1 -- 172.21.15.67:0/130330863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c156b30 msgr2=0x7fab5c0af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:30.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.817+0000 7fab62400640 1 --2- 172.21.15.67:0/130330863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c156b30 0x7fab5c0af6e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fab5c06fdf0 tx=0x7fab5002a140 comp rx=0 tx=0).stop 2024-05-30T17:00:30.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.818+0000 7fab62400640 1 -- 172.21.15.67:0/130330863 shutdown_connections 2024-05-30T17:00:30.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.818+0000 7fab62400640 1 --2- 172.21.15.67:0/130330863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c06f2f0 0x7fab5c0ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.818+0000 7fab62400640 1 --2- 172.21.15.67:0/130330863 >> v2:172.21.15.169:3300/0 conn(0x7fab5c06ee40 0x7fab5c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.818+0000 7fab62400640 1 --2- 172.21.15.67:0/130330863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c156b30 0x7fab5c0af6e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.818+0000 7fab62400640 1 -- 172.21.15.67:0/130330863 >> 172.21.15.67:0/130330863 conn(0x7fab5c0b1fc0 msgr2=0x7fab5c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:30.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.818+0000 7fab62400640 1 -- 172.21.15.67:0/130330863 shutdown_connections 2024-05-30T17:00:30.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.818+0000 7fab62400640 1 -- 172.21.15.67:0/130330863 wait complete. 2024-05-30T17:00:30.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.819+0000 7fab62400640 1 Processor -- start 2024-05-30T17:00:30.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.819+0000 7fab62400640 1 -- start start 2024-05-30T17:00:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.820+0000 7fab62400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c06ee40 0x7fab5c157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.820+0000 7fab62400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab5c06f2f0 0x7fab5c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.820+0000 7fab62400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c156b30 0x7fab5c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.820+0000 7fab62400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c0afc20 con 0x7fab5c06ee40 2024-05-30T17:00:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.820+0000 7fab62400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c0b2c30 con 0x7fab5c06f2f0 2024-05-30T17:00:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.820+0000 7fab62400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c146100 con 0x7fab5c156b30 2024-05-30T17:00:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab61000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c06ee40 0x7fab5c157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab61000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c06ee40 0x7fab5c157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43290/0 (socket says 172.21.15.67:43290) 2024-05-30T17:00:30.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab61000640 1 -- 172.21.15.67:0/4189837505 learned_addr learned my addr 172.21.15.67:0/4189837505 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab61a00640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c156b30 0x7fab5c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab61000640 1 -- 172.21.15.67:0/4189837505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c156b30 msgr2=0x7fab5c0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab5be00640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab5c06f2f0 0x7fab5c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab61000640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c156b30 0x7fab5c0ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab61000640 1 -- 172.21.15.67:0/4189837505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab5c06f2f0 msgr2=0x7fab5c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab61000640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab5c06f2f0 0x7fab5c0ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.821+0000 7fab61000640 1 -- 172.21.15.67:0/4189837505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c158e90 con 0x7fab5c06ee40 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.822+0000 7fab61000640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c06ee40 0x7fab5c157aa0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fab50009960 tx=0x7fab500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.822+0000 7fab58c00640 1 -- 172.21.15.67:0/4189837505 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50041020 con 0x7fab5c06ee40 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.822+0000 7fab58c00640 1 -- 172.21.15.67:0/4189837505 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab50030820 con 0x7fab5c06ee40 2024-05-30T17:00:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.822+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c15d960 con 0x7fab5c06ee40 2024-05-30T17:00:30.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.822+0000 7fab58c00640 1 -- 172.21.15.67:0/4189837505 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50037040 con 0x7fab5c06ee40 2024-05-30T17:00:30.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.822+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c15df20 con 0x7fab5c06ee40 2024-05-30T17:00:30.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.823+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5c0689e0 con 0x7fab5c06ee40 2024-05-30T17:00:30.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.824+0000 7fab58c00640 1 -- 172.21.15.67:0/4189837505 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fab50044020 con 0x7fab5c06ee40 2024-05-30T17:00:30.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.825+0000 7fab58c00640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:30.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.825+0000 7fab58c00640 1 -- 172.21.15.67:0/4189837505 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5027+0+0 (secure 0 0 0) 0x7fab500ab110 con 0x7fab5c06ee40 2024-05-30T17:00:30.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.826+0000 7fab5be00640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:30.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.826+0000 7fab5be00640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fab5c1e6ef0 tx=0x7fab48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:30.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:30.830+0000 7fab58c00640 1 -- 172.21.15.67:0/4189837505 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab5004b030 con 0x7fab5c06ee40 2024-05-30T17:00:31.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.080+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab5c158940 con 0x7fab5c06ee40 2024-05-30T17:00:31.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.081+0000 7fab58c00640 1 -- 172.21.15.67:0/4189837505 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+3875 (secure 0 0 0) 0x7fab5c158e90 con 0x7fab5c06ee40 2024-05-30T17:00:31.082 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:31.082 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1054215955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1054215955},{"type":"v1","addr":"172.21.15.67:6837","nonce":1054215955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":5591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1236503792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1236503792},{"type":"v1","addr":"172.21.15.169:6835","nonce":1236503792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2825499128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2825499128},{"type":"v1","addr":"172.21.15.169:6834","nonce":2825499128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/4047293074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":4047293074},{"type":"v1","addr":"172.21.15.67:6836","nonce":4047293074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-05-30T17:00:31.082 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 58 2024-05-30T17:00:31.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 msgr2=0x7fab3c070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:31.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fab5c1e6ef0 tx=0x7fab48008040 comp rx=0 tx=0).stop 2024-05-30T17:00:31.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c06ee40 msgr2=0x7fab5c157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:31.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c06ee40 0x7fab5c157aa0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fab50009960 tx=0x7fab500046c0 comp rx=0 tx=0).stop 2024-05-30T17:00:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 shutdown_connections 2024-05-30T17:00:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fab3c06ddc0 0x7fab3c070280 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab5c156b30 0x7fab5c0ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:31.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab5c06f2f0 0x7fab5c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:31.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 --2- 172.21.15.67:0/4189837505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab5c06ee40 0x7fab5c157aa0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:31.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 >> 172.21.15.67:0/4189837505 conn(0x7fab5c0b1fc0 msgr2=0x7fab5c0b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:31.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 shutdown_connections 2024-05-30T17:00:31.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.084+0000 7fab62400640 1 -- 172.21.15.67:0/4189837505 wait complete. 2024-05-30T17:00:31.106 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-30T17:00:31.106 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-30T17:00:31.107 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-30T17:00:31.107 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:00:31.107 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-30T17:00:31.107 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-30T17:00:31.107 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-30T17:00:31.107 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:00:31.108 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-30T17:00:31.108 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-30T17:00:31.108 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-30T17:00:31.108 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:00:31.108 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-30T17:00:31.108 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-30T17:00:31.108 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-30T17:00:31.108 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:00:31.124 INFO:tasks.ceph.mds.c:Stopped 2024-05-30T17:00:31.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-30T17:00:31.131 INFO:tasks.ceph.mds.a:Stopped 2024-05-30T17:00:31.131 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-30T17:00:31.146 INFO:tasks.ceph.mds.d:Stopped 2024-05-30T17:00:31.146 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-30T17:00:31.147 INFO:tasks.ceph.mds.b:Stopped 2024-05-30T17:00:31.147 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-30T17:00:31.178 INFO:tasks.ceph.mds.c:Started 2024-05-30T17:00:31.179 DEBUG:teuthology.parallel:result is None 2024-05-30T17:00:31.184 INFO:tasks.ceph.mds.a:Started 2024-05-30T17:00:31.184 DEBUG:teuthology.parallel:result is None 2024-05-30T17:00:31.193 INFO:tasks.ceph.mds.d:Started 2024-05-30T17:00:31.194 DEBUG:teuthology.parallel:result is None 2024-05-30T17:00:31.199 INFO:tasks.ceph.mds.b:Started 2024-05-30T17:00:31.200 DEBUG:teuthology.parallel:result is None 2024-05-30T17:00:31.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-30T17:00:31.336 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T17:00:31.338 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T17:00:31.348 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T17:00:31.348 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T17:00:31.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.382+0000 7f431a600640 1 Processor -- start 2024-05-30T17:00:31.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.382+0000 7f431a600640 1 -- start start 2024-05-30T17:00:31.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.383+0000 7f431a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4314160380 0x7f43140b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:31.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.383+0000 7f431a600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43140d3300 con 0x7f43140b5430 2024-05-30T17:00:31.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.383+0000 7f431a600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43140d35e0 con 0x7f431415fae0 2024-05-30T17:00:31.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.383+0000 7f431a600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f43140ced60 con 0x7f4314160380 2024-05-30T17:00:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.384+0000 7f430be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4314160380 0x7f43140b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.384+0000 7f4319c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f43140b5430 0x7f43140cf1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60948/0 (socket says 172.21.15.67:60948) 2024-05-30T17:00:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.384+0000 7f4319c00640 1 -- 172.21.15.67:0/641817698 learned_addr learned my addr 172.21.15.67:0/641817698 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.384+0000 7f430be00640 1 -- 172.21.15.67:0/641817698 >> v1:172.21.15.169:6789/0 conn(0x7f431415fae0 legacy=0x7f43140d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.384+0000 7f430be00640 1 -- 172.21.15.67:0/641817698 >> v1:172.21.15.67:6789/0 conn(0x7f43140b5430 legacy=0x7f43140cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.384+0000 7f430be00640 1 -- 172.21.15.67:0/641817698 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43140d38c0 con 0x7f4314160380 2024-05-30T17:00:31.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.385+0000 7f430be00640 1 --2- 172.21.15.67:0/641817698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4314160380 0x7f43140b5e60 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f43140ceee0 tx=0x7f42fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be512c128d289d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:31.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f4313e00640 1 -- 172.21.15.67:0/641817698 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc037070 con 0x7f4314160380 2024-05-30T17:00:31.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f4313e00640 1 -- 172.21.15.67:0/641817698 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42fc032440 con 0x7f4314160380 2024-05-30T17:00:31.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f4313e00640 1 -- 172.21.15.67:0/641817698 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc032810 con 0x7f4314160380 2024-05-30T17:00:31.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f431a600640 1 -- 172.21.15.67:0/641817698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4314160380 msgr2=0x7f43140b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:31.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f431a600640 1 --2- 172.21.15.67:0/641817698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4314160380 0x7f43140b5e60 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f43140ceee0 tx=0x7f42fc02a210 comp rx=0 tx=0).stop 2024-05-30T17:00:31.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f431a600640 1 -- 172.21.15.67:0/641817698 shutdown_connections 2024-05-30T17:00:31.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f431a600640 1 --2- 172.21.15.67:0/641817698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4314160380 0x7f43140b5e60 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:31.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f431a600640 1 -- 172.21.15.67:0/641817698 >> 172.21.15.67:0/641817698 conn(0x7f431408bfc0 msgr2=0x7f43140d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:31.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f431a600640 1 -- 172.21.15.67:0/641817698 shutdown_connections 2024-05-30T17:00:31.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.386+0000 7f431a600640 1 -- 172.21.15.67:0/641817698 wait complete. 2024-05-30T17:00:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.388+0000 7f431a600640 1 Processor -- start 2024-05-30T17:00:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.388+0000 7f431a600640 1 -- start start 2024-05-30T17:00:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.389+0000 7f431a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43140b5430 0x7f431415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.389+0000 7f431a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431415fae0 0x7f43140b9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.389+0000 7f431a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314160380 0x7f43140b9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.389+0000 7f431a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f43140b6520 con 0x7f43140b5430 2024-05-30T17:00:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.389+0000 7f431a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43140b63a0 con 0x7f431415fae0 2024-05-30T17:00:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.389+0000 7f431a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f43140b66a0 con 0x7f4314160380 2024-05-30T17:00:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.389+0000 7f4319200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43140b5430 0x7f431415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f4319200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43140b5430 0x7f431415aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43342/0 (socket says 172.21.15.67:43342) 2024-05-30T17:00:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f4319200640 1 -- 172.21.15.67:0/1427851882 learned_addr learned my addr 172.21.15.67:0/1427851882 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f430be00640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431415fae0 0x7f43140b9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f4319c00640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314160380 0x7f43140b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f4319200640 1 -- 172.21.15.67:0/1427851882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314160380 msgr2=0x7f43140b9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f4319200640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314160380 0x7f43140b9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f4319200640 1 -- 172.21.15.67:0/1427851882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431415fae0 msgr2=0x7f43140b9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f4319200640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431415fae0 0x7f43140b9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f4319200640 1 -- 172.21.15.67:0/1427851882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43140b9d30 con 0x7f43140b5430 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f430be00640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431415fae0 0x7f43140b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.390+0000 7f4319200640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43140b5430 0x7f431415aa00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f430400c7c0 tx=0x7f430400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.391+0000 7f4311600640 1 -- 172.21.15.67:0/1427851882 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304016020 con 0x7f43140b5430 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.391+0000 7f4311600640 1 -- 172.21.15.67:0/1427851882 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4304006d50 con 0x7f43140b5430 2024-05-30T17:00:31.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.391+0000 7f4311600640 1 -- 172.21.15.67:0/1427851882 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304003bb0 con 0x7f43140b5430 2024-05-30T17:00:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.391+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43140ba010 con 0x7f43140b5430 2024-05-30T17:00:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.391+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141e9750 con 0x7f43140b5430 2024-05-30T17:00:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.392+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43140689e0 con 0x7f43140b5430 2024-05-30T17:00:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.393+0000 7f4311600640 1 -- 172.21.15.67:0/1427851882 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f430400a850 con 0x7f43140b5430 2024-05-30T17:00:31.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.395+0000 7f4311600640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f006ddc0 0x7f42f0070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:31.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.395+0000 7f4311600640 1 -- 172.21.15.67:0/1427851882 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5027+0+0 (secure 0 0 0) 0x7f430408a5b0 con 0x7f43140b5430 2024-05-30T17:00:31.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.395+0000 7f430be00640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f006ddc0 0x7f42f0070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:31.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.396+0000 7f430be00640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f006ddc0 0x7f42f0070280 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f42fc034000 tx=0x7f42fc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.399+0000 7f4311600640 1 -- 172.21.15.67:0/1427851882 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4304020020 con 0x7f43140b5430 2024-05-30T17:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:31.615+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f43140b6c90 con 0x7f43140b5430 2024-05-30T17:00:32.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.670+0000 7f4311600640 1 -- 172.21.15.67:0/1427851882 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) ==== 97+0+0 (secure 0 0 0) 0x7f4304019310 con 0x7f43140b5430 2024-05-30T17:00:32.671 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-05-30T17:00:32.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f006ddc0 msgr2=0x7f42f0070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f006ddc0 0x7f42f0070280 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f42fc034000 tx=0x7f42fc02e000 comp rx=0 tx=0).stop 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43140b5430 msgr2=0x7f431415aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43140b5430 0x7f431415aa00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f430400c7c0 tx=0x7f430400cc80 comp rx=0 tx=0).stop 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 shutdown_connections 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f006ddc0 0x7f42f0070280 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314160380 0x7f43140b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431415fae0 0x7f43140b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 --2- 172.21.15.67:0/1427851882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43140b5430 0x7f431415aa00 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 >> 172.21.15.67:0/1427851882 conn(0x7f431408bfc0 msgr2=0x7f4314161460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 shutdown_connections 2024-05-30T17:00:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.675+0000 7f431a600640 1 -- 172.21.15.67:0/1427851882 wait complete. 2024-05-30T17:00:32.696 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-30T17:00:32.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.905+0000 7f7297000640 1 Processor -- start 2024-05-30T17:00:32.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.906+0000 7f7297000640 1 -- start start 2024-05-30T17:00:32.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.908+0000 7f7297000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7290158d20 0x7f729006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:32.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.908+0000 7f7297000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f72900b5e20 0x7f729006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:32.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.908+0000 7f7297000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72900b4b90 con 0x7f72900b3e20 2024-05-30T17:00:32.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.908+0000 7f7297000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f72900b5060 con 0x7f72900b5e20 2024-05-30T17:00:32.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.908+0000 7f7297000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72900b5340 con 0x7f7290158d20 2024-05-30T17:00:32.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.908+0000 7f7295000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f72900b3e20 0x7f72900b42b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60952/0 (socket says 172.21.15.67:60952) 2024-05-30T17:00:32.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.908+0000 7f7295000640 1 -- 172.21.15.67:0/1667331693 learned_addr learned my addr 172.21.15.67:0/1667331693 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728fe00640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7290158d20 0x7f729006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f7287e00640 1 --2- 172.21.15.67:0/1667331693 >> v2:172.21.15.169:3300/0 conn(0x7f72900b5e20 0x7f729006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 318737601 0 0) 0x7f72900b4b90 con 0x7f72900b3e20 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7274003520 con 0x7f72900b3e20 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4927412 0 0) 0x7f7274003520 con 0x7f72900b3e20 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7290158d20 msgr2=0x7f729006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7290158d20 0x7f729006c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 >> v2:172.21.15.169:3300/0 conn(0x7f72900b5e20 msgr2=0x7f729006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 --2- 172.21.15.67:0/1667331693 >> v2:172.21.15.169:3300/0 conn(0x7f72900b5e20 0x7f729006ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72900b5620 con 0x7f72900b3e20 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f72880019f0 con 0x7f72900b3e20 2024-05-30T17:00:32.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 >> v1:172.21.15.67:6789/0 conn(0x7f72900b3e20 legacy=0x7f72900b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72740039b0 0x7f7274003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f7274004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7290158d20 0x7f72740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f72900b5060 con 0x7f7274004800 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72900b5340 con 0x7f72740039b0 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.909+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f727400b9c0 con 0x7f7290158d20 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.910+0000 7f7287e00640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f7274004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.910+0000 7f7287e00640 1 -- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7290158d20 msgr2=0x7f72740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.910+0000 7f7295000640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7290158d20 0x7f72740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.910+0000 7f728fe00640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72740039b0 0x7f7274003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.910+0000 7f7287e00640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7290158d20 0x7f72740051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.910+0000 7f7287e00640 1 -- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72740039b0 msgr2=0x7f7274003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.910+0000 7f7287e00640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72740039b0 0x7f7274003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.910+0000 7f7287e00640 1 -- 172.21.15.67:0/1667331693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7274003520 con 0x7f7274004800 2024-05-30T17:00:32.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.910+0000 7f728fe00640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72740039b0 0x7f7274003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.911+0000 7f7287e00640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f7274004c80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f7278009a90 tx=0x7f7274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454ea774486e1138 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.912+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278036070 con 0x7f7274004800 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.912+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7278031400 con 0x7f7274004800 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.912+0000 7f728f400640 1 -- 172.21.15.67:0/1667331693 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278009690 con 0x7f7274004800 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.912+0000 7f7297000640 1 -- 172.21.15.67:0/1667331693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 msgr2=0x7f7274004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.912+0000 7f7297000640 1 --2- 172.21.15.67:0/1667331693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f7274004c80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f7278009a90 tx=0x7f7274005830 comp rx=0 tx=0).stop 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.912+0000 7f7295000640 1 -- 172.21.15.67:0/1667331693 reap_dead start 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.912+0000 7f7297000640 1 -- 172.21.15.67:0/1667331693 shutdown_connections 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.912+0000 7f7297000640 1 -- 172.21.15.67:0/1667331693 >> 172.21.15.67:0/1667331693 conn(0x7f72900724f0 msgr2=0x7f72900728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.913+0000 7f7297000640 1 -- 172.21.15.67:0/1667331693 shutdown_connections 2024-05-30T17:00:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.913+0000 7f7297000640 1 -- 172.21.15.67:0/1667331693 wait complete. 2024-05-30T17:00:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.914+0000 7f7297000640 1 Processor -- start 2024-05-30T17:00:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.915+0000 7f7297000640 1 -- start start 2024-05-30T17:00:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.915+0000 7f7297000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f729013f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.915+0000 7f7297000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7290150980 0x7f7290155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.915+0000 7f7297000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72900aec00 0x7f72900af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f7297000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f729006c910 con 0x7f7274004800 2024-05-30T17:00:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f7297000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7290159100 con 0x7f7290150980 2024-05-30T17:00:32.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f7297000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f729006ca90 con 0x7f72900aec00 2024-05-30T17:00:32.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f728fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f729013f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:32.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f728fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f729013f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43364/0 (socket says 172.21.15.67:43364) 2024-05-30T17:00:32.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f728fe00640 1 -- 172.21.15.67:0/3305411149 learned_addr learned my addr 172.21.15.67:0/3305411149 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f7287e00640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7290150980 0x7f7290155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f728fe00640 1 -- 172.21.15.67:0/3305411149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72900aec00 msgr2=0x7f72900af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f728fe00640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72900aec00 0x7f72900af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f728fe00640 1 -- 172.21.15.67:0/3305411149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7290150980 msgr2=0x7f7290155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f728fe00640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7290150980 0x7f7290155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.916+0000 7f728fe00640 1 -- 172.21.15.67:0/3305411149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7290156e70 con 0x7f7274004800 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.917+0000 7f728fe00640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f729013f0f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f7280010890 tx=0x7f7280010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.917+0000 7f728cc00640 1 -- 172.21.15.67:0/3305411149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728000f040 con 0x7f7274004800 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.917+0000 7f728cc00640 1 -- 172.21.15.67:0/3305411149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728000eb00 con 0x7f7274004800 2024-05-30T17:00:32.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.917+0000 7f728cc00640 1 -- 172.21.15.67:0/3305411149 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728001c400 con 0x7f7274004800 2024-05-30T17:00:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.917+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7290157150 con 0x7f7274004800 2024-05-30T17:00:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.917+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7290157710 con 0x7f7274004800 2024-05-30T17:00:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.918+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72900689e0 con 0x7f7274004800 2024-05-30T17:00:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.919+0000 7f728cc00640 1 -- 172.21.15.67:0/3305411149 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f728001c5a0 con 0x7f7274004800 2024-05-30T17:00:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.920+0000 7f728cc00640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f726c06ddf0 0x7f726c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.920+0000 7f728cc00640 1 -- 172.21.15.67:0/3305411149 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4941+0+0 (secure 0 0 0) 0x7f728008b120 con 0x7f7274004800 2024-05-30T17:00:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.921+0000 7f7287e00640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f726c06ddf0 0x7f726c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.921+0000 7f7287e00640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f726c06ddf0 0x7f726c0702b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7278033000 tx=0x7f7278008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:32.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:32.925+0000 7f728cc00640 1 -- 172.21.15.67:0/3305411149 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7280018030 con 0x7f7274004800 2024-05-30T17:00:33.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.208+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f72900736a0 con 0x7f7274004800 2024-05-30T17:00:33.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.209+0000 7f728cc00640 1 -- 172.21.15.67:0/3305411149 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f728005a570 con 0x7f7274004800 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA==", 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:00:33.210 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw==", 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.211 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw==", 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg==", 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.212 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng==", 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg==", 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w==", 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:00:33.214 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ==", 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A==", 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.215 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ==", 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-05-30T17:00:33.216 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w==", 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A==", 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg==", 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ==", 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:00:33.218 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.2", 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ==", 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.219 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.3", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A==", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCArFhmF5mhNhAA4H3cgcH6YdD53fANDW4seA==", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:00:33.220 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ==", 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw==", 2024-05-30T17:00:33.221 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:00:33.222 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T17:00:33.222 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T17:00:33.222 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:00:33.222 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.222 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.222 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-05-30T17:00:33.222 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f726c06ddf0 msgr2=0x7f726c0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f726c06ddf0 0x7f726c0702b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7278033000 tx=0x7f7278008030 comp rx=0 tx=0).stop 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 msgr2=0x7f729013f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f729013f0f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f7280010890 tx=0x7f7280010d50 comp rx=0 tx=0).stop 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 shutdown_connections 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f726c06ddf0 0x7f726c0702b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72900aec00 0x7f72900af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7290150980 0x7f7290155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 --2- 172.21.15.67:0/3305411149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7274004800 0x7f729013f0f0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 >> 172.21.15.67:0/3305411149 conn(0x7f72900724f0 msgr2=0x7f7290160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:33.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 shutdown_connections 2024-05-30T17:00:33.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.212+0000 7f7297000640 1 -- 172.21.15.67:0/3305411149 wait complete. 2024-05-30T17:00:33.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-30T17:00:33.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.443+0000 7f46dc200640 1 Processor -- start 2024-05-30T17:00:33.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.444+0000 7f46dc200640 1 -- start start 2024-05-30T17:00:33.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.445+0000 7f46dc200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46d40d04a0 0x7f46d40d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.445+0000 7f46dc200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f46d40b3460 0x7f46d40b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.445+0000 7f46dc200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d40ced40 con 0x7f46d40b4b60 2024-05-30T17:00:33.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.445+0000 7f46dc200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f46d415fbb0 con 0x7f46d40d04a0 2024-05-30T17:00:33.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.445+0000 7f46dc200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f46d40cefc0 con 0x7f46d40b3460 2024-05-30T17:00:33.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.446+0000 7f46d8e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46d40d04a0 0x7f46d40d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.446+0000 7f46d8e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46d40d04a0 0x7f46d40d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43372/0 (socket says 172.21.15.67:43372) 2024-05-30T17:00:33.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.446+0000 7f46d8e00640 1 -- 172.21.15.67:0/1608475261 learned_addr learned my addr 172.21.15.67:0/1608475261 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:33.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.446+0000 7f46da200640 1 --2- 172.21.15.67:0/1608475261 >> v2:172.21.15.169:3300/0 conn(0x7f46d40b3460 0x7f46d40b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.446+0000 7f46d8e00640 1 -- 172.21.15.67:0/1608475261 >> v2:172.21.15.169:3300/0 conn(0x7f46d40b3460 msgr2=0x7f46d40b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:33.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.446+0000 7f46d8e00640 1 --2- 172.21.15.67:0/1608475261 >> v2:172.21.15.169:3300/0 conn(0x7f46d40b3460 0x7f46d40b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.446+0000 7f46d8e00640 1 -- 172.21.15.67:0/1608475261 >> v1:172.21.15.169:6789/0 conn(0x7f46d40b4b60 legacy=0x7f46d40cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:33.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.446+0000 7f46d8e00640 1 -- 172.21.15.67:0/1608475261 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d40cf2a0 con 0x7f46d40d04a0 2024-05-30T17:00:33.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.447+0000 7f46d8e00640 1 --2- 172.21.15.67:0/1608475261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40d04a0 0x7f46d40d0930 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f46d40af320 tx=0x7f46c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863a53d9a5c6de99 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:33.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.448+0000 7f46d3e00640 1 -- 172.21.15.67:0/1608475261 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0037070 con 0x7f46d40d04a0 2024-05-30T17:00:33.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.448+0000 7f46d3e00640 1 -- 172.21.15.67:0/1608475261 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c0032440 con 0x7f46d40d04a0 2024-05-30T17:00:33.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.448+0000 7f46d3e00640 1 -- 172.21.15.67:0/1608475261 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0032810 con 0x7f46d40d04a0 2024-05-30T17:00:33.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.448+0000 7f46dc200640 1 -- 172.21.15.67:0/1608475261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40d04a0 msgr2=0x7f46d40d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.448+0000 7f46dc200640 1 --2- 172.21.15.67:0/1608475261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40d04a0 0x7f46d40d0930 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f46d40af320 tx=0x7f46c002a230 comp rx=0 tx=0).stop 2024-05-30T17:00:33.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.449+0000 7f46dc200640 1 -- 172.21.15.67:0/1608475261 shutdown_connections 2024-05-30T17:00:33.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.449+0000 7f46dc200640 1 --2- 172.21.15.67:0/1608475261 >> v2:172.21.15.169:3300/0 conn(0x7f46d40b3460 0x7f46d40b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.449+0000 7f46dc200640 1 --2- 172.21.15.67:0/1608475261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40d04a0 0x7f46d40d0930 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.449+0000 7f46dc200640 1 -- 172.21.15.67:0/1608475261 >> 172.21.15.67:0/1608475261 conn(0x7f46d406bae0 msgr2=0x7f46d406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:33.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.449+0000 7f46dc200640 1 -- 172.21.15.67:0/1608475261 shutdown_connections 2024-05-30T17:00:33.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.449+0000 7f46dc200640 1 -- 172.21.15.67:0/1608475261 wait complete. 2024-05-30T17:00:33.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.450+0000 7f46dc200640 1 Processor -- start 2024-05-30T17:00:33.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.451+0000 7f46dc200640 1 -- start start 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46dc200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46d40b3460 0x7f46d41f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46dc200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40b4b60 0x7f46d41f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46dc200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d40d04a0 0x7f46d4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46dc200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46d40b3f40 con 0x7f46d40b4b60 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46dc200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46d40b3dc0 con 0x7f46d40d04a0 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46dc200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f46d415ede0 con 0x7f46d40b3460 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40b4b60 0x7f46d41f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40b4b60 0x7f46d41f3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43382/0 (socket says 172.21.15.67:43382) 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d8e00640 1 -- 172.21.15.67:0/16698787 learned_addr learned my addr 172.21.15.67:0/16698787 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:33.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d8e00640 1 -- 172.21.15.67:0/16698787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46d40b3460 msgr2=0x7f46d41f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d9800640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46d40b3460 0x7f46d41f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d8e00640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46d40b3460 0x7f46d41f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d8e00640 1 -- 172.21.15.67:0/16698787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d40d04a0 msgr2=0x7f46d4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d8e00640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d40d04a0 0x7f46d4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d8e00640 1 -- 172.21.15.67:0/16698787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d41f53c0 con 0x7f46d40b4b60 2024-05-30T17:00:33.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.452+0000 7f46d9800640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46d40b3460 0x7f46d41f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.453+0000 7f46d8e00640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40b4b60 0x7f46d41f3cf0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f46c0000c00 tx=0x7f46c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.453+0000 7f46d1600640 1 -- 172.21.15.67:0/16698787 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0041020 con 0x7f46d40b4b60 2024-05-30T17:00:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.453+0000 7f46d1600640 1 -- 172.21.15.67:0/16698787 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c00306c0 con 0x7f46d40b4b60 2024-05-30T17:00:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.453+0000 7f46d1600640 1 -- 172.21.15.67:0/16698787 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0037040 con 0x7f46d40b4b60 2024-05-30T17:00:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.453+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d41f55c0 con 0x7f46d40b4b60 2024-05-30T17:00:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.453+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41f5b40 con 0x7f46d40b4b60 2024-05-30T17:00:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.455+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d40689e0 con 0x7f46d40b4b60 2024-05-30T17:00:33.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.456+0000 7f46d1600640 1 -- 172.21.15.67:0/16698787 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f46c0044020 con 0x7f46d40b4b60 2024-05-30T17:00:33.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.457+0000 7f46d1600640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46b406ddc0 0x7f46b4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.457+0000 7f46d1600640 1 -- 172.21.15.67:0/16698787 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4941+0+0 (secure 0 0 0) 0x7f46c00ab0c0 con 0x7f46d40b4b60 2024-05-30T17:00:33.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.457+0000 7f46d9800640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46b406ddc0 0x7f46b4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.457+0000 7f46d9800640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46b406ddc0 0x7f46b4070280 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f46d40cdb50 tx=0x7f46c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:33.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.461+0000 7f46d1600640 1 -- 172.21.15.67:0/16698787 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46c004b030 con 0x7f46d40b4b60 2024-05-30T17:00:33.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.666+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f46d41f4230 con 0x7f46d40b4b60 2024-05-30T17:00:33.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.666+0000 7f46d1600640 1 -- 172.21.15.67:0/16698787 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) ==== 84+0+65 (secure 0 0 0) 0x7f46d41f53c0 con 0x7f46d40b4b60 2024-05-30T17:00:33.667 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:33.667 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:00:33.667 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:33.667 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-05-30T17:00:33.667 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-05-30T17:00:33.667 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:33.667 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:00:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46b406ddc0 msgr2=0x7f46b4070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46b406ddc0 0x7f46b4070280 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f46d40cdb50 tx=0x7f46c8009040 comp rx=0 tx=0).stop 2024-05-30T17:00:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40b4b60 msgr2=0x7f46d41f3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40b4b60 0x7f46d41f3cf0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f46c0000c00 tx=0x7f46c00046c0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 shutdown_connections 2024-05-30T17:00:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f46b406ddc0 0x7f46b4070280 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46d40d04a0 0x7f46d4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d40b4b60 0x7f46d41f3cf0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 --2- 172.21.15.67:0/16698787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46d40b3460 0x7f46d41f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.669+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 >> 172.21.15.67:0/16698787 conn(0x7f46d406bae0 msgr2=0x7f46d40afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.670+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 shutdown_connections 2024-05-30T17:00:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.670+0000 7f46dc200640 1 -- 172.21.15.67:0/16698787 wait complete. 2024-05-30T17:00:33.690 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-30T17:00:33.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-30T17:00:33.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.910+0000 7f0404000640 1 Processor -- start 2024-05-30T17:00:33.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.910+0000 7f0404000640 1 -- start start 2024-05-30T17:00:33.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.911+0000 7f0404000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f03fc15c830 0x7f03fc0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0404000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0ce770 0x7f03fc0b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0404000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03fc15cc10 con 0x7f03fc0b45f0 2024-05-30T17:00:33.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0404000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f03fc0b4b90 con 0x7f03fc15c830 2024-05-30T17:00:33.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0404000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc0b4e70 con 0x7f03fc0ce770 2024-05-30T17:00:33.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0401600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f03fc15c830 0x7f03fc0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0401600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f03fc15c830 0x7f03fc0ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43392/0 (socket says 172.21.15.67:43392) 2024-05-30T17:00:33.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0401600640 1 -- 172.21.15.67:0/2296628076 learned_addr learned my addr 172.21.15.67:0/2296628076 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:33.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0400c00640 1 --2- 172.21.15.67:0/2296628076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0ce770 0x7f03fc0b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0401600640 1 -- 172.21.15.67:0/2296628076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0ce770 msgr2=0x7f03fc0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:33.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0401600640 1 --2- 172.21.15.67:0/2296628076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0ce770 0x7f03fc0b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0401600640 1 -- 172.21.15.67:0/2296628076 >> v1:172.21.15.67:6789/0 conn(0x7f03fc0b45f0 legacy=0x7f03fc0d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.912+0000 7f0401600640 1 -- 172.21.15.67:0/2296628076 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc0b5150 con 0x7f03fc15c830 2024-05-30T17:00:33.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.914+0000 7f0401600640 1 --2- 172.21.15.67:0/2296628076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 0x7f03fc0ce230 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009870 tx=0x7f03f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce55405744948248 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:33.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.914+0000 7f03fbe00640 1 -- 172.21.15.67:0/2296628076 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0037070 con 0x7f03fc15c830 2024-05-30T17:00:33.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.914+0000 7f03fbe00640 1 -- 172.21.15.67:0/2296628076 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f0032440 con 0x7f03fc15c830 2024-05-30T17:00:33.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.914+0000 7f03fbe00640 1 -- 172.21.15.67:0/2296628076 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0032810 con 0x7f03fc15c830 2024-05-30T17:00:33.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.914+0000 7f0404000640 1 -- 172.21.15.67:0/2296628076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 msgr2=0x7f03fc0ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.914+0000 7f0404000640 1 --2- 172.21.15.67:0/2296628076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 0x7f03fc0ce230 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009870 tx=0x7f03f002a270 comp rx=0 tx=0).stop 2024-05-30T17:00:33.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.915+0000 7f0404000640 1 -- 172.21.15.67:0/2296628076 shutdown_connections 2024-05-30T17:00:33.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.915+0000 7f0404000640 1 --2- 172.21.15.67:0/2296628076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0ce770 0x7f03fc0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.915+0000 7f0404000640 1 --2- 172.21.15.67:0/2296628076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 0x7f03fc0ce230 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.915+0000 7f0404000640 1 -- 172.21.15.67:0/2296628076 >> 172.21.15.67:0/2296628076 conn(0x7f03fc0b20a0 msgr2=0x7f03fc0b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:33.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.915+0000 7f0404000640 1 -- 172.21.15.67:0/2296628076 shutdown_connections 2024-05-30T17:00:33.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.915+0000 7f0404000640 1 -- 172.21.15.67:0/2296628076 wait complete. 2024-05-30T17:00:33.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.917+0000 7f0404000640 1 Processor -- start 2024-05-30T17:00:33.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.917+0000 7f0404000640 1 -- start start 2024-05-30T17:00:33.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0404000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0b45f0 0x7f03fc1f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0404000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03fc0ce770 0x7f03fc14e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0404000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 0x7f03fc15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0404000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0d0160 con 0x7f03fc15c830 2024-05-30T17:00:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0404000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0cffe0 con 0x7f03fc0ce770 2024-05-30T17:00:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0404000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc160480 con 0x7f03fc0b45f0 2024-05-30T17:00:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0402000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 0x7f03fc15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0402000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 0x7f03fc15ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43394/0 (socket says 172.21.15.67:43394) 2024-05-30T17:00:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0402000640 1 -- 172.21.15.67:0/3901824487 learned_addr learned my addr 172.21.15.67:0/3901824487 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.918+0000 7f0401600640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0b45f0 0x7f03fc1f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f0402000640 1 -- 172.21.15.67:0/3901824487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0b45f0 msgr2=0x7f03fc1f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f0402000640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0b45f0 0x7f03fc1f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f0402000640 1 -- 172.21.15.67:0/3901824487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03fc0ce770 msgr2=0x7f03fc14e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f0400c00640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03fc0ce770 0x7f03fc14e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f0402000640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03fc0ce770 0x7f03fc14e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f0402000640 1 -- 172.21.15.67:0/3901824487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc14e5a0 con 0x7f03fc15c830 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f0402000640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 0x7f03fc15ce30 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c7d0 tx=0x7f03f400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f03f9600640 1 -- 172.21.15.67:0/3901824487 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4003b70 con 0x7f03fc15c830 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f03f9600640 1 -- 172.21.15.67:0/3901824487 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4003d30 con 0x7f03fc15c830 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f03f9600640 1 -- 172.21.15.67:0/3901824487 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f400aa90 con 0x7f03fc15c830 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc1e63d0 con 0x7f03fc15c830 2024-05-30T17:00:33.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.919+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc1e6990 con 0x7f03fc15c830 2024-05-30T17:00:33.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.921+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc0757d0 con 0x7f03fc15c830 2024-05-30T17:00:33.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.922+0000 7f03f9600640 1 -- 172.21.15.67:0/3901824487 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f03f400ac30 con 0x7f03fc15c830 2024-05-30T17:00:33.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.923+0000 7f03f9600640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f03dc06ddc0 0x7f03dc070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:33.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.923+0000 7f03f9600640 1 -- 172.21.15.67:0/3901824487 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4941+0+0 (secure 0 0 0) 0x7f03f408ab00 con 0x7f03fc15c830 2024-05-30T17:00:33.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.923+0000 7f0401600640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f03dc06ddc0 0x7f03dc070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:33.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.924+0000 7f0401600640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f03dc06ddc0 0x7f03dc070280 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f03f0034000 tx=0x7f03f002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:33.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:33.928+0000 7f03f9600640 1 -- 172.21.15.67:0/3901824487 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03f405a600 con 0x7f03fc15c830 2024-05-30T17:00:34.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.135+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f03fc06e260 con 0x7f03fc15c830 2024-05-30T17:00:34.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.687+0000 7f03f9600640 1 -- 172.21.15.67:0/3901824487 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) ==== 138+0+0 (secure 0 0 0) 0x7f03f4059f00 con 0x7f03fc15c830 2024-05-30T17:00:34.688 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-05-30T17:00:34.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f03dc06ddc0 msgr2=0x7f03dc070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:34.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f03dc06ddc0 0x7f03dc070280 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f03f0034000 tx=0x7f03f002e000 comp rx=0 tx=0).stop 2024-05-30T17:00:34.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 msgr2=0x7f03fc15ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:34.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 0x7f03fc15ce30 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c7d0 tx=0x7f03f400cc90 comp rx=0 tx=0).stop 2024-05-30T17:00:34.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 shutdown_connections 2024-05-30T17:00:34.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f03dc06ddc0 0x7f03dc070280 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03fc15c830 0x7f03fc15ce30 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03fc0ce770 0x7f03fc14e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 --2- 172.21.15.67:0/3901824487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03fc0b45f0 0x7f03fc1f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.691+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 >> 172.21.15.67:0/3901824487 conn(0x7f03fc0b20a0 msgr2=0x7f03fc0d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:34.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.692+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 shutdown_connections 2024-05-30T17:00:34.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.692+0000 7f0404000640 1 -- 172.21.15.67:0/3901824487 wait complete. 2024-05-30T17:00:34.718 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-30T17:00:34.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.922+0000 7f9c14e00640 1 Processor -- start 2024-05-30T17:00:34.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.923+0000 7f9c14e00640 1 -- start start 2024-05-30T17:00:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.924+0000 7f9c14e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c10153510 0x7f9c100b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.924+0000 7f9c14e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c100ccb90 0x7f9c100ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.924+0000 7f9c14e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c1015d900 con 0x7f9c1015cb60 2024-05-30T17:00:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.924+0000 7f9c14e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9c101538f0 con 0x7f9c100ccb90 2024-05-30T17:00:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.924+0000 7f9c14e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c100cd530 con 0x7f9c10153510 2024-05-30T17:00:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.924+0000 7f9c0fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9c1015cb60 0x7f9c1015cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60956/0 (socket says 172.21.15.67:60956) 2024-05-30T17:00:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.924+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2730273385 learned_addr learned my addr 172.21.15.67:0/2730273385 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0ea00640 1 --2- 172.21.15.67:0/2730273385 >> v2:172.21.15.169:3300/0 conn(0x7f9c100ccb90 0x7f9c100ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0f400640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c10153510 0x7f9c100b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104932182 0 0) 0x7f9c1015d900 con 0x7f9c1015cb60 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bf4003520 con 0x7f9c1015cb60 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3675506646 0 0) 0x7f9bf4003520 con 0x7f9c1015cb60 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c10153510 msgr2=0x7f9c100b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c10153510 0x7f9c100b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 >> v2:172.21.15.169:3300/0 conn(0x7f9c100ccb90 msgr2=0x7f9c100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 --2- 172.21.15.67:0/2730273385 >> v2:172.21.15.169:3300/0 conn(0x7f9c100ccb90 0x7f9c100ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c100b3f90 con 0x7f9c1015cb60 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9c080019f0 con 0x7f9c1015cb60 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 >> v1:172.21.15.67:6789/0 conn(0x7f9c1015cb60 legacy=0x7f9c1015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c10153510 0x7f9bf40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf4003f80 0x7f9bf4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf4004d40 0x7f9bf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf400b9c0 con 0x7f9bf4004d40 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf400bb70 con 0x7f9bf4003f80 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.925+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf400bd70 con 0x7f9c10153510 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0fe00640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf4004d40 0x7f9bf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0f400640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c10153510 0x7f9bf40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0ea00640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf4003f80 0x7f9bf4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c10153510 msgr2=0x7f9bf40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0fe00640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c10153510 0x7f9bf40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf4003f80 msgr2=0x7f9bf4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0fe00640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf4003f80 0x7f9bf4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2730273385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf4003520 con 0x7f9bf4004d40 2024-05-30T17:00:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0ea00640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf4003f80 0x7f9bf4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:00:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.926+0000 7f9c0f400640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c10153510 0x7f9bf40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.927+0000 7f9c0fe00640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf4004d40 0x7f9bf40051c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800a7a0 tx=0x7f9bf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c652d432b68a8a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.928+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800a3f0 con 0x7f9bf4004d40 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.928+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0802f070 con 0x7f9bf4004d40 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.928+0000 7f9c0e000640 1 -- 172.21.15.67:0/2730273385 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0802bc60 con 0x7f9bf4004d40 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.928+0000 7f9c14e00640 1 -- 172.21.15.67:0/2730273385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf4004d40 msgr2=0x7f9bf40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.928+0000 7f9c14e00640 1 --2- 172.21.15.67:0/2730273385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf4004d40 0x7f9bf40051c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800a7a0 tx=0x7f9bf4005830 comp rx=0 tx=0).stop 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.928+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2730273385 reap_dead start 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.928+0000 7f9c14e00640 1 -- 172.21.15.67:0/2730273385 shutdown_connections 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.928+0000 7f9c14e00640 1 -- 172.21.15.67:0/2730273385 >> 172.21.15.67:0/2730273385 conn(0x7f9c1006b900 msgr2=0x7f9c1006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.928+0000 7f9c14e00640 1 -- 172.21.15.67:0/2730273385 shutdown_connections 2024-05-30T17:00:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.929+0000 7f9c14e00640 1 -- 172.21.15.67:0/2730273385 wait complete. 2024-05-30T17:00:34.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.930+0000 7f9c14e00640 1 Processor -- start 2024-05-30T17:00:34.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.931+0000 7f9c14e00640 1 -- start start 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.931+0000 7f9c14e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c10149610 0x7f9c100ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c14e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100ae500 0x7f9c100c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c14e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c100c9a10 0x7f9c10072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c14e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9c100b4550 con 0x7f9c10149610 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c14e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c100b48c0 con 0x7f9c100ae500 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c14e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c100b4d50 con 0x7f9c100c9a10 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c10149610 0x7f9c100ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c10149610 0x7f9c100ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43412/0 (socket says 172.21.15.67:43412) 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0f400640 1 -- 172.21.15.67:0/3096221436 learned_addr learned my addr 172.21.15.67:0/3096221436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0ea00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100ae500 0x7f9c100c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0fe00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c100c9a10 0x7f9c10072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0f400640 1 -- 172.21.15.67:0/3096221436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c100c9a10 msgr2=0x7f9c10072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0f400640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c100c9a10 0x7f9c10072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0f400640 1 -- 172.21.15.67:0/3096221436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100ae500 msgr2=0x7f9c100c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0f400640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100ae500 0x7f9c100c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.932+0000 7f9c0f400640 1 -- 172.21.15.67:0/3096221436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c100cebc0 con 0x7f9c10149610 2024-05-30T17:00:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.933+0000 7f9c0f400640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c10149610 0x7f9c100ce370 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9c00011700 tx=0x7f9c00011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.933+0000 7f9c07400640 1 -- 172.21.15.67:0/3096221436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000f960 con 0x7f9c10149610 2024-05-30T17:00:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.933+0000 7f9c07400640 1 -- 172.21.15.67:0/3096221436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0000fb00 con 0x7f9c10149610 2024-05-30T17:00:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.933+0000 7f9c07400640 1 -- 172.21.15.67:0/3096221436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00015460 con 0x7f9c10149610 2024-05-30T17:00:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.933+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c100ceea0 con 0x7f9c10149610 2024-05-30T17:00:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.933+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c100cf460 con 0x7f9c10149610 2024-05-30T17:00:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.934+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c100689e0 con 0x7f9c10149610 2024-05-30T17:00:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.936+0000 7f9c07400640 1 -- 172.21.15.67:0/3096221436 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f9c00015600 con 0x7f9c10149610 2024-05-30T17:00:34.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.937+0000 7f9c07400640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bf006ddf0 0x7f9bf00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:34.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.937+0000 7f9c07400640 1 -- 172.21.15.67:0/3096221436 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5311+0+0 (secure 0 0 0) 0x7f9c0008b4b0 con 0x7f9c10149610 2024-05-30T17:00:34.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.937+0000 7f9c0ea00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bf006ddf0 0x7f9bf00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.938+0000 7f9c0ea00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bf006ddf0 0x7f9bf00702b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9c10073c00 tx=0x7f9bfc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:34.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:34.942+0000 7f9c07400640 1 -- 172.21.15.67:0/3096221436 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c00019030 con 0x7f9c10149610 2024-05-30T17:00:35.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.151+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9c100b3c40 con 0x7f9c10149610 2024-05-30T17:00:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.693+0000 7f9c07400640 1 -- 172.21.15.67:0/3096221436 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) ==== 170+0+0 (secure 0 0 0) 0x7f9c0005afb0 con 0x7f9c10149610 2024-05-30T17:00:35.694 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-05-30T17:00:35.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.697+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bf006ddf0 msgr2=0x7f9bf00702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bf006ddf0 0x7f9bf00702b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9c10073c00 tx=0x7f9bfc008040 comp rx=0 tx=0).stop 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c10149610 msgr2=0x7f9c100ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c10149610 0x7f9c100ce370 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9c00011700 tx=0x7f9c00011bc0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 shutdown_connections 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bf006ddf0 0x7f9bf00702b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c100c9a10 0x7f9c10072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100ae500 0x7f9c100c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 --2- 172.21.15.67:0/3096221436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c10149610 0x7f9c100ce370 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 >> 172.21.15.67:0/3096221436 conn(0x7f9c1006b900 msgr2=0x7f9c10152710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 shutdown_connections 2024-05-30T17:00:35.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.698+0000 7f9c14e00640 1 -- 172.21.15.67:0/3096221436 wait complete. 2024-05-30T17:00:35.729 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-30T17:00:35.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.913+0000 7efda7e00640 1 Processor -- start 2024-05-30T17:00:35.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.914+0000 7efda7e00640 1 -- start start 2024-05-30T17:00:35.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.915+0000 7efda7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efda80d4680 0x7efda80d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:35.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.915+0000 7efda7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efda80d4a60 0x7efda80d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:35.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.915+0000 7efda7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda80d0150 con 0x7efda80b46a0 2024-05-30T17:00:35.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.915+0000 7efda7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7efda80d28b0 con 0x7efda80d4a60 2024-05-30T17:00:35.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.915+0000 7efda7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efda80d0430 con 0x7efda80d4680 2024-05-30T17:00:35.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.915+0000 7efd9fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efda80d4a60 0x7efda80d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:35.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.915+0000 7efd9fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efda80d4a60 0x7efda80d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43422/0 (socket says 172.21.15.67:43422) 2024-05-30T17:00:35.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.915+0000 7efd9fe00640 1 -- 172.21.15.67:0/3094543293 learned_addr learned my addr 172.21.15.67:0/3094543293 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:35.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.916+0000 7efda6a00640 1 --2- 172.21.15.67:0/3094543293 >> v2:172.21.15.169:3300/0 conn(0x7efda80d4680 0x7efda80d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:35.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.916+0000 7efd9fe00640 1 -- 172.21.15.67:0/3094543293 >> v2:172.21.15.169:3300/0 conn(0x7efda80d4680 msgr2=0x7efda80d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:35.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.916+0000 7efd9fe00640 1 --2- 172.21.15.67:0/3094543293 >> v2:172.21.15.169:3300/0 conn(0x7efda80d4680 0x7efda80d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.916+0000 7efd9fe00640 1 -- 172.21.15.67:0/3094543293 >> v1:172.21.15.67:6789/0 conn(0x7efda80b46a0 legacy=0x7efda815e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:35.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.916+0000 7efd9fe00640 1 -- 172.21.15.67:0/3094543293 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda80d2260 con 0x7efda80d4a60 2024-05-30T17:00:35.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.916+0000 7efda6a00640 1 --2- 172.21.15.67:0/3094543293 >> v2:172.21.15.169:3300/0 conn(0x7efda80d4680 0x7efda80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:35.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.917+0000 7efd9fe00640 1 --2- 172.21.15.67:0/3094543293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80d4a60 0x7efda80d3280 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7efda80d5580 tx=0x7efd9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd46fdc94a93ccb server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:35.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.917+0000 7efda6000640 1 -- 172.21.15.67:0/3094543293 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90037070 con 0x7efda80d4a60 2024-05-30T17:00:35.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.917+0000 7efda6000640 1 -- 172.21.15.67:0/3094543293 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90032440 con 0x7efda80d4a60 2024-05-30T17:00:35.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.917+0000 7efda6000640 1 -- 172.21.15.67:0/3094543293 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90032810 con 0x7efda80d4a60 2024-05-30T17:00:35.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.918+0000 7efda7e00640 1 -- 172.21.15.67:0/3094543293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80d4a60 msgr2=0x7efda80d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:35.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.918+0000 7efda7e00640 1 --2- 172.21.15.67:0/3094543293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80d4a60 0x7efda80d3280 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7efda80d5580 tx=0x7efd9002a270 comp rx=0 tx=0).stop 2024-05-30T17:00:35.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.918+0000 7efda7e00640 1 -- 172.21.15.67:0/3094543293 shutdown_connections 2024-05-30T17:00:35.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.918+0000 7efda7e00640 1 --2- 172.21.15.67:0/3094543293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80d4a60 0x7efda80d3280 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.918+0000 7efda7e00640 1 --2- 172.21.15.67:0/3094543293 >> v2:172.21.15.169:3300/0 conn(0x7efda80d4680 0x7efda80d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.918+0000 7efda7e00640 1 -- 172.21.15.67:0/3094543293 >> 172.21.15.67:0/3094543293 conn(0x7efda809d810 msgr2=0x7efda808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:35.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.918+0000 7efda7e00640 1 -- 172.21.15.67:0/3094543293 shutdown_connections 2024-05-30T17:00:35.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.918+0000 7efda7e00640 1 -- 172.21.15.67:0/3094543293 wait complete. 2024-05-30T17:00:35.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.920+0000 7efda7e00640 1 Processor -- start 2024-05-30T17:00:35.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.920+0000 7efda7e00640 1 -- start start 2024-05-30T17:00:35.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.921+0000 7efda7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80b46a0 0x7efda8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:35.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.921+0000 7efda7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80d4680 0x7efda81567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:35.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.921+0000 7efda7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda80d4a60 0x7efda8156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:35.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.921+0000 7efda7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efda815eb70 con 0x7efda80b46a0 2024-05-30T17:00:35.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.921+0000 7efda7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efda815e9f0 con 0x7efda80d4a60 2024-05-30T17:00:35.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.921+0000 7efda7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efda815ecf0 con 0x7efda80d4680 2024-05-30T17:00:35.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.921+0000 7efda6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80b46a0 0x7efda8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:35.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.921+0000 7efda6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80b46a0 0x7efda8154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43438/0 (socket says 172.21.15.67:43438) 2024-05-30T17:00:35.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.921+0000 7efda6a00640 1 -- 172.21.15.67:0/3478202566 learned_addr learned my addr 172.21.15.67:0/3478202566 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.922+0000 7efd9fe00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80d4680 0x7efda81567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.922+0000 7efda7400640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda80d4a60 0x7efda8156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.922+0000 7efda6a00640 1 -- 172.21.15.67:0/3478202566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80d4680 msgr2=0x7efda81567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.922+0000 7efda6a00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80d4680 0x7efda81567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.922+0000 7efda6a00640 1 -- 172.21.15.67:0/3478202566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda80d4a60 msgr2=0x7efda8156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.922+0000 7efda6a00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda80d4a60 0x7efda8156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.922+0000 7efda6a00640 1 -- 172.21.15.67:0/3478202566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda8157600 con 0x7efda80b46a0 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.922+0000 7efda6a00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80b46a0 0x7efda8154bb0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7efd9800c7f0 tx=0x7efd9800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.923+0000 7efd9f400640 1 -- 172.21.15.67:0/3478202566 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98015070 con 0x7efda80b46a0 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.923+0000 7efd9f400640 1 -- 172.21.15.67:0/3478202566 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd98007a30 con 0x7efda80b46a0 2024-05-30T17:00:35.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.923+0000 7efd9f400640 1 -- 172.21.15.67:0/3478202566 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98007e00 con 0x7efda80b46a0 2024-05-30T17:00:35.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.923+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda815bc90 con 0x7efda80b46a0 2024-05-30T17:00:35.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.923+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda815c250 con 0x7efda80b46a0 2024-05-30T17:00:35.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.924+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda80689e0 con 0x7efda80b46a0 2024-05-30T17:00:35.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.925+0000 7efd9f400640 1 -- 172.21.15.67:0/3478202566 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7efd98018070 con 0x7efda80b46a0 2024-05-30T17:00:35.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.926+0000 7efd9f400640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406de10 0x7efd840702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:35.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.926+0000 7efd9f400640 1 -- 172.21.15.67:0/3478202566 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5693+0+0 (secure 0 0 0) 0x7efd9808ac00 con 0x7efda80b46a0 2024-05-30T17:00:35.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.927+0000 7efd9fe00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406de10 0x7efd840702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:35.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.927+0000 7efd9fe00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406de10 0x7efd840702d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efd90034000 tx=0x7efd9002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:35.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:35.931+0000 7efd9f400640 1 -- 172.21.15.67:0/3478202566 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd9805aac0 con 0x7efda80b46a0 2024-05-30T17:00:36.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.171+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efda815c490 con 0x7efda80b46a0 2024-05-30T17:00:36.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.739+0000 7efd9f400640 1 -- 172.21.15.67:0/3478202566 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v60) ==== 174+0+0 (secure 0 0 0) 0x7efda815c490 con 0x7efda80b46a0 2024-05-30T17:00:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 14 and data pool 15 2024-05-30T17:00:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.748+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406de10 msgr2=0x7efd840702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.748+0000 7efda7e00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406de10 0x7efd840702d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efd90034000 tx=0x7efd9002e000 comp rx=0 tx=0).stop 2024-05-30T17:00:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.748+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80b46a0 msgr2=0x7efda8154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.748+0000 7efda7e00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80b46a0 0x7efda8154bb0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7efd9800c7f0 tx=0x7efd9800cbf0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.751+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 shutdown_connections 2024-05-30T17:00:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.751+0000 7efda7e00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd8406de10 0x7efd840702d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.751+0000 7efda7e00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efda80d4a60 0x7efda8156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.751+0000 7efda7e00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efda80d4680 0x7efda81567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.751+0000 7efda7e00640 1 --2- 172.21.15.67:0/3478202566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efda80b46a0 0x7efda8154bb0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.751+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 >> 172.21.15.67:0/3478202566 conn(0x7efda809d810 msgr2=0x7efda806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.752+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 shutdown_connections 2024-05-30T17:00:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.752+0000 7efda7e00640 1 -- 172.21.15.67:0/3478202566 wait complete. 2024-05-30T17:00:36.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:00:36.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.960+0000 7fa9e9a00640 1 Processor -- start 2024-05-30T17:00:36.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.960+0000 7fa9e9a00640 1 -- start start 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.961+0000 7fa9e9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406f860 0x7fa9e40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.961+0000 7fa9e9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9e40d0bb0 0x7fa9e40ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.961+0000 7fa9e9a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e406ed10 con 0x7fa9e40b3690 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e9a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e415cb00 con 0x7fa9e40d0bb0 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e40b3cc0 con 0x7fa9e406f860 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9e40d0bb0 0x7fa9e40ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9e40d0bb0 0x7fa9e40ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43450/0 (socket says 172.21.15.67:43450) 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e3400640 1 -- 172.21.15.67:0/2177317105 learned_addr learned my addr 172.21.15.67:0/2177317105 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/2177317105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406f860 0x7fa9e40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e3400640 1 -- 172.21.15.67:0/2177317105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406f860 msgr2=0x7fa9e40b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e3400640 1 --2- 172.21.15.67:0/2177317105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406f860 0x7fa9e40b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e3400640 1 -- 172.21.15.67:0/2177317105 >> v1:172.21.15.169:6789/0 conn(0x7fa9e40b3690 legacy=0x7fa9e40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:36.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e3400640 1 -- 172.21.15.67:0/2177317105 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e40b3fa0 con 0x7fa9e40d0bb0 2024-05-30T17:00:36.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.962+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/2177317105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406f860 0x7fa9e40b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.963+0000 7fa9e3400640 1 --2- 172.21.15.67:0/2177317105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e40d0bb0 0x7fa9e40ad130 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009e50 tx=0x7fa9dc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dc2def152a5e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.963+0000 7fa9e1600640 1 -- 172.21.15.67:0/2177317105 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc036070 con 0x7fa9e40d0bb0 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e1600640 1 -- 172.21.15.67:0/2177317105 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc031600 con 0x7fa9e40d0bb0 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e1600640 1 -- 172.21.15.67:0/2177317105 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0319d0 con 0x7fa9e40d0bb0 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e9a00640 1 -- 172.21.15.67:0/2177317105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e40d0bb0 msgr2=0x7fa9e40ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e9a00640 1 --2- 172.21.15.67:0/2177317105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e40d0bb0 0x7fa9e40ad130 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009e50 tx=0x7fa9dc02a6d0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e9a00640 1 -- 172.21.15.67:0/2177317105 shutdown_connections 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e9a00640 1 --2- 172.21.15.67:0/2177317105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e40d0bb0 0x7fa9e40ad130 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e9a00640 1 --2- 172.21.15.67:0/2177317105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406f860 0x7fa9e40b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e9a00640 1 -- 172.21.15.67:0/2177317105 >> 172.21.15.67:0/2177317105 conn(0x7fa9e406bae0 msgr2=0x7fa9e406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:36.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e9a00640 1 -- 172.21.15.67:0/2177317105 shutdown_connections 2024-05-30T17:00:36.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.964+0000 7fa9e9a00640 1 -- 172.21.15.67:0/2177317105 wait complete. 2024-05-30T17:00:36.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.966+0000 7fa9e9a00640 1 Processor -- start 2024-05-30T17:00:36.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.966+0000 7fa9e9a00640 1 -- start start 2024-05-30T17:00:36.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.967+0000 7fa9e9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406f860 0x7fa9e40998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:36.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.967+0000 7fa9e9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e40b3690 0x7fa9e415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:36.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.967+0000 7fa9e9a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9e40d0bb0 0x7fa9e415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:36.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.967+0000 7fa9e9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e40b4840 con 0x7fa9e406f860 2024-05-30T17:00:36.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.967+0000 7fa9e9a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e40b4560 con 0x7fa9e40d0bb0 2024-05-30T17:00:36.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.967+0000 7fa9e9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e40ad670 con 0x7fa9e40b3690 2024-05-30T17:00:36.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.967+0000 7fa9e2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406f860 0x7fa9e40998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:36.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.967+0000 7fa9e2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406f860 0x7fa9e40998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43466/0 (socket says 172.21.15.67:43466) 2024-05-30T17:00:36.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.967+0000 7fa9e2a00640 1 -- 172.21.15.67:0/762452776 learned_addr learned my addr 172.21.15.67:0/762452776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:36.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9e2a00640 1 -- 172.21.15.67:0/762452776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e40b3690 msgr2=0x7fa9e415b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:36.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e40b3690 0x7fa9e415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9e2a00640 1 -- 172.21.15.67:0/762452776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9e40d0bb0 msgr2=0x7fa9e415b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:36.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9e40d0bb0 0x7fa9e415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:36.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9e2a00640 1 -- 172.21.15.67:0/762452776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e415f5e0 con 0x7fa9e406f860 2024-05-30T17:00:36.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406f860 0x7fa9e40998f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400c7f0 tx=0x7fa9d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:36.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9daa00640 1 -- 172.21.15.67:0/762452776 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4015070 con 0x7fa9e406f860 2024-05-30T17:00:36.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9daa00640 1 -- 172.21.15.67:0/762452776 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d4007a30 con 0x7fa9e406f860 2024-05-30T17:00:36.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9daa00640 1 -- 172.21.15.67:0/762452776 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4007e00 con 0x7fa9e406f860 2024-05-30T17:00:36.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.968+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e40c7330 con 0x7fa9e406f860 2024-05-30T17:00:36.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.969+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e40c78f0 con 0x7fa9e406f860 2024-05-30T17:00:36.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.970+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e40689e0 con 0x7fa9e406f860 2024-05-30T17:00:36.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.971+0000 7fa9daa00640 1 -- 172.21.15.67:0/762452776 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fa9d4018070 con 0x7fa9e406f860 2024-05-30T17:00:36.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.972+0000 7fa9daa00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c406ddc0 0x7fa9c4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:36.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.972+0000 7fa9daa00640 1 -- 172.21.15.67:0/762452776 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5793+0+0 (secure 0 0 0) 0x7fa9d408a870 con 0x7fa9e406f860 2024-05-30T17:00:36.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.972+0000 7fa9e2000640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c406ddc0 0x7fa9c4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:36.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.973+0000 7fa9e2000640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c406ddc0 0x7fa9c4070280 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4158a70 tx=0x7fa9d00079b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:36.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:36.976+0000 7fa9daa00640 1 -- 172.21.15.67:0/762452776 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9d405a6f0 con 0x7fa9e406f860 2024-05-30T17:00:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.180+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9e406f1a0 con 0x7fa9e406f860 2024-05-30T17:00:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.183+0000 7fa9daa00640 1 -- 172.21.15.67:0/762452776 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16752 (secure 0 0 0) 0x7fa9d4059ff0 con 0x7fa9e406f860 2024-05-30T17:00:37.185 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:37.185 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":103,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:00:36.172770+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:00:34.151659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-30T17:00:35.170102+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c406ddc0 msgr2=0x7fa9c4070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c406ddc0 0x7fa9c4070280 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4158a70 tx=0x7fa9d00079b0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406f860 msgr2=0x7fa9e40998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406f860 0x7fa9e40998f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400c7f0 tx=0x7fa9d400cbf0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 shutdown_connections 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c406ddc0 0x7fa9c4070280 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9e40d0bb0 0x7fa9e415b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e40b3690 0x7fa9e415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 --2- 172.21.15.67:0/762452776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406f860 0x7fa9e40998f0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 >> 172.21.15.67:0/762452776 conn(0x7fa9e406bae0 msgr2=0x7fa9e406bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:37.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.186+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 shutdown_connections 2024-05-30T17:00:37.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.187+0000 7fa9e9a00640 1 -- 172.21.15.67:0/762452776 wait complete. 2024-05-30T17:00:37.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:00:37.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.413+0000 7f3795a00640 1 Processor -- start 2024-05-30T17:00:37.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.413+0000 7f3795a00640 1 -- start start 2024-05-30T17:00:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.414+0000 7f3795a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f37900722d0 0x7f379016d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.414+0000 7f3795a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379006f590 con 0x7f3790162d20 2024-05-30T17:00:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.414+0000 7f3795a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379006f870 con 0x7f3790162940 2024-05-30T17:00:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.414+0000 7f3795a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f37900ad3d0 con 0x7f37900722d0 2024-05-30T17:00:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.415+0000 7f378f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3790162d20 0x7f3790071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60972/0 (socket says 172.21.15.67:60972) 2024-05-30T17:00:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.415+0000 7f378f400640 1 -- 172.21.15.67:0/377617784 learned_addr learned my addr 172.21.15.67:0/377617784 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.415+0000 7f3795000640 1 --2- 172.21.15.67:0/377617784 >> v2:172.21.15.67:3300/0 conn(0x7f37900722d0 0x7f379016d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.415+0000 7f3795000640 1 -- 172.21.15.67:0/377617784 >> v1:172.21.15.169:6789/0 conn(0x7f3790162940 legacy=0x7f379006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.415+0000 7f3795000640 1 -- 172.21.15.67:0/377617784 >> v1:172.21.15.67:6789/0 conn(0x7f3790162d20 legacy=0x7f3790071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.415+0000 7f3795000640 1 -- 172.21.15.67:0/377617784 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379006f210 con 0x7f37900722d0 2024-05-30T17:00:37.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.415+0000 7f378ea00640 1 -- 172.21.15.67:0/377617784 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1300015987 0 0) 0x7f379006f590 con 0x7f3790162d20 2024-05-30T17:00:37.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.416+0000 7f3795000640 1 --2- 172.21.15.67:0/377617784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37900722d0 0x7f379016d7e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3788009d90 tx=0x7f378802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d484dac06a5b27a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:37.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.417+0000 7f378ea00640 1 -- 172.21.15.67:0/377617784 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788031600 con 0x7f37900722d0 2024-05-30T17:00:37.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.417+0000 7f378ea00640 1 -- 172.21.15.67:0/377617784 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37880317a0 con 0x7f37900722d0 2024-05-30T17:00:37.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.417+0000 7f378ea00640 1 -- 172.21.15.67:0/377617784 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788031b70 con 0x7f37900722d0 2024-05-30T17:00:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.417+0000 7f3795a00640 1 -- 172.21.15.67:0/377617784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37900722d0 msgr2=0x7f379016d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.417+0000 7f3795a00640 1 --2- 172.21.15.67:0/377617784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37900722d0 0x7f379016d7e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3788009d90 tx=0x7f378802a5d0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.417+0000 7f3795a00640 1 -- 172.21.15.67:0/377617784 shutdown_connections 2024-05-30T17:00:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.417+0000 7f3795a00640 1 --2- 172.21.15.67:0/377617784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37900722d0 0x7f379016d7e0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.417+0000 7f3795a00640 1 -- 172.21.15.67:0/377617784 >> 172.21.15.67:0/377617784 conn(0x7f37900b1fc0 msgr2=0x7f37900b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.418+0000 7f3795a00640 1 -- 172.21.15.67:0/377617784 shutdown_connections 2024-05-30T17:00:37.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.418+0000 7f3795a00640 1 -- 172.21.15.67:0/377617784 wait complete. 2024-05-30T17:00:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.419+0000 7f3795a00640 1 Processor -- start 2024-05-30T17:00:37.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.419+0000 7f3795a00640 1 -- start start 2024-05-30T17:00:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.420+0000 7f3795a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37900722d0 0x7f37900b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.420+0000 7f3795a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3790162940 0x7f379015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3790162d20 0x7f37901593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f37900aed40 con 0x7f3790162d20 2024-05-30T17:00:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f37900aebc0 con 0x7f37900722d0 2024-05-30T17:00:37.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f37900aeec0 con 0x7f3790162940 2024-05-30T17:00:37.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3790162d20 0x7f37901593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3790162d20 0x7f37901593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43484/0 (socket says 172.21.15.67:43484) 2024-05-30T17:00:37.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795000640 1 -- 172.21.15.67:0/3318821224 learned_addr learned my addr 172.21.15.67:0/3318821224 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:37.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f378fe00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37900722d0 0x7f37900b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f378f400640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3790162940 0x7f379015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795000640 1 -- 172.21.15.67:0/3318821224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3790162940 msgr2=0x7f379015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795000640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3790162940 0x7f379015ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795000640 1 -- 172.21.15.67:0/3318821224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37900722d0 msgr2=0x7f37900b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795000640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37900722d0 0x7f37900b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.421+0000 7f3795000640 1 -- 172.21.15.67:0/3318821224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37900cead0 con 0x7f3790162d20 2024-05-30T17:00:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.422+0000 7f3795000640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3790162d20 0x7f37901593c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3788009d90 tx=0x7f3788004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.422+0000 7f3787e00640 1 -- 172.21.15.67:0/3318821224 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788040070 con 0x7f3790162d20 2024-05-30T17:00:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.422+0000 7f3787e00640 1 -- 172.21.15.67:0/3318821224 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3788030b60 con 0x7f3790162d20 2024-05-30T17:00:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.422+0000 7f3787e00640 1 -- 172.21.15.67:0/3318821224 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378803c420 con 0x7f3790162d20 2024-05-30T17:00:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.422+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37901632a0 con 0x7f3790162d20 2024-05-30T17:00:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.422+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3790163860 con 0x7f3790162d20 2024-05-30T17:00:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.423+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37900689e0 con 0x7f3790162d20 2024-05-30T17:00:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.424+0000 7f3787e00640 1 -- 172.21.15.67:0/3318821224 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f3788039040 con 0x7f3790162d20 2024-05-30T17:00:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.425+0000 7f3787e00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f377006ddf0 0x7f37700702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.425+0000 7f3787e00640 1 -- 172.21.15.67:0/3318821224 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5793+0+0 (secure 0 0 0) 0x7f37880abba0 con 0x7f3790162d20 2024-05-30T17:00:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.426+0000 7f378fe00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f377006ddf0 0x7f37700702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.426+0000 7f378fe00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f377006ddf0 0x7f37700702b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f37900b18e0 tx=0x7f3780009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:37.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.430+0000 7f3787e00640 1 -- 172.21.15.67:0/3318821224 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f378807b4f0 con 0x7f3790162d20 2024-05-30T17:00:37.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.634+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f379006eaa0 con 0x7f3790162d20 2024-05-30T17:00:37.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.637+0000 7f3787e00640 1 -- 172.21.15.67:0/3318821224 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16752 (secure 0 0 0) 0x7f378807f7f0 con 0x7f3790162d20 2024-05-30T17:00:37.638 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:37.638 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":103,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:00:36.172770+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:00:34.151659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-30T17:00:35.170102+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:00:37.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.639+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f377006ddf0 msgr2=0x7f37700702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.639+0000 7f3795a00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f377006ddf0 0x7f37700702b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f37900b18e0 tx=0x7f3780009040 comp rx=0 tx=0).stop 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.639+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3790162d20 msgr2=0x7f37901593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.639+0000 7f3795a00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3790162d20 0x7f37901593c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3788009d90 tx=0x7f3788004a40 comp rx=0 tx=0).stop 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.640+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 shutdown_connections 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.640+0000 7f3795a00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f377006ddf0 0x7f37700702b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.640+0000 7f3795a00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3790162d20 0x7f37901593c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.640+0000 7f3795a00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3790162940 0x7f379015ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.640+0000 7f3795a00640 1 --2- 172.21.15.67:0/3318821224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37900722d0 0x7f37900b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.640+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 >> 172.21.15.67:0/3318821224 conn(0x7f37900b1fc0 msgr2=0x7f3790071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.640+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 shutdown_connections 2024-05-30T17:00:37.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.640+0000 7f3795a00640 1 -- 172.21.15.67:0/3318821224 wait complete. 2024-05-30T17:00:37.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-30T17:00:37.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.872+0000 7f3a72a00640 1 Processor -- start 2024-05-30T17:00:37.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.873+0000 7f3a72a00640 1 -- start start 2024-05-30T17:00:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.874+0000 7f3a72a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a6c161a90 0x7f3a6c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.874+0000 7f3a72a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3a6c161e70 0x7f3a6c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.874+0000 7f3a72a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a6c0b3d40 con 0x7f3a6c06e9b0 2024-05-30T17:00:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.874+0000 7f3a72a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6c0b4020 con 0x7f3a6c161e70 2024-05-30T17:00:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.874+0000 7f3a72a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6c0b4300 con 0x7f3a6c161a90 2024-05-30T17:00:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.875+0000 7f3a6b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3a6c161e70 0x7f3a6c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.875+0000 7f3a6b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3a6c161e70 0x7f3a6c0b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43498/0 (socket says 172.21.15.67:43498) 2024-05-30T17:00:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.875+0000 7f3a6b400640 1 -- 172.21.15.67:0/236015488 learned_addr learned my addr 172.21.15.67:0/236015488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.875+0000 7f3a6be00640 1 --2- 172.21.15.67:0/236015488 >> v2:172.21.15.169:3300/0 conn(0x7f3a6c161a90 0x7f3a6c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.875+0000 7f3a6b400640 1 -- 172.21.15.67:0/236015488 >> v2:172.21.15.169:3300/0 conn(0x7f3a6c161a90 msgr2=0x7f3a6c06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:37.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.875+0000 7f3a6b400640 1 --2- 172.21.15.67:0/236015488 >> v2:172.21.15.169:3300/0 conn(0x7f3a6c161a90 0x7f3a6c06e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.875+0000 7f3a6b400640 1 -- 172.21.15.67:0/236015488 >> v1:172.21.15.67:6789/0 conn(0x7f3a6c06e9b0 legacy=0x7f3a6c0ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.875+0000 7f3a6b400640 1 -- 172.21.15.67:0/236015488 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c06dc70 con 0x7f3a6c161e70 2024-05-30T17:00:37.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.875+0000 7f3a6be00640 1 --2- 172.21.15.67:0/236015488 >> v2:172.21.15.169:3300/0 conn(0x7f3a6c161a90 0x7f3a6c06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.876+0000 7f3a6b400640 1 --2- 172.21.15.67:0/236015488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 0x7f3a6c0b32c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c06fe90 tx=0x7f3a5402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab560d268f9d3d91 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a6aa00640 1 -- 172.21.15.67:0/236015488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54036070 con 0x7f3a6c161e70 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a6aa00640 1 -- 172.21.15.67:0/236015488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a54031600 con 0x7f3a6c161e70 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a6aa00640 1 -- 172.21.15.67:0/236015488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a540319d0 con 0x7f3a6c161e70 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a72a00640 1 -- 172.21.15.67:0/236015488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 msgr2=0x7f3a6c0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a72a00640 1 --2- 172.21.15.67:0/236015488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 0x7f3a6c0b32c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c06fe90 tx=0x7f3a5402a730 comp rx=0 tx=0).stop 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a72a00640 1 -- 172.21.15.67:0/236015488 shutdown_connections 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a72a00640 1 --2- 172.21.15.67:0/236015488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 0x7f3a6c0b32c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a72a00640 1 --2- 172.21.15.67:0/236015488 >> v2:172.21.15.169:3300/0 conn(0x7f3a6c161a90 0x7f3a6c06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a72a00640 1 -- 172.21.15.67:0/236015488 >> 172.21.15.67:0/236015488 conn(0x7f3a6c06bd20 msgr2=0x7f3a6c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:37.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a72a00640 1 -- 172.21.15.67:0/236015488 shutdown_connections 2024-05-30T17:00:37.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.877+0000 7f3a72a00640 1 -- 172.21.15.67:0/236015488 wait complete. 2024-05-30T17:00:37.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.879+0000 7f3a72a00640 1 Processor -- start 2024-05-30T17:00:37.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.879+0000 7f3a72a00640 1 -- start start 2024-05-30T17:00:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.880+0000 7f3a72a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a6c06e9b0 0x7f3a6c09c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.880+0000 7f3a72a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6c161a90 0x7f3a6c1466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.880+0000 7f3a72a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 0x7f3a6c09d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.880+0000 7f3a72a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c0ad9b0 con 0x7f3a6c161e70 2024-05-30T17:00:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.880+0000 7f3a72a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c0b5020 con 0x7f3a6c161a90 2024-05-30T17:00:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.880+0000 7f3a72a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c0adb30 con 0x7f3a6c06e9b0 2024-05-30T17:00:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a70a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 0x7f3a6c09d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a70a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 0x7f3a6c09d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43500/0 (socket says 172.21.15.67:43500) 2024-05-30T17:00:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a70a00640 1 -- 172.21.15.67:0/2882024756 learned_addr learned my addr 172.21.15.67:0/2882024756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a6b400640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6c161a90 0x7f3a6c1466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a6be00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a6c06e9b0 0x7f3a6c09c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a70a00640 1 -- 172.21.15.67:0/2882024756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a6c06e9b0 msgr2=0x7f3a6c09c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a70a00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a6c06e9b0 0x7f3a6c09c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a70a00640 1 -- 172.21.15.67:0/2882024756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6c161a90 msgr2=0x7f3a6c1466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a70a00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6c161a90 0x7f3a6c1466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a70a00640 1 -- 172.21.15.67:0/2882024756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c09b710 con 0x7f3a6c161e70 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a6be00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a6c06e9b0 0x7f3a6c09c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.881+0000 7f3a70a00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 0x7f3a6c09d390 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400c400 tx=0x7f3a6400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:37.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.882+0000 7f3a63e00640 1 -- 172.21.15.67:0/2882024756 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64017070 con 0x7f3a6c161e70 2024-05-30T17:00:37.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.882+0000 7f3a63e00640 1 -- 172.21.15.67:0/2882024756 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a640052d0 con 0x7f3a6c161e70 2024-05-30T17:00:37.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.882+0000 7f3a63e00640 1 -- 172.21.15.67:0/2882024756 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64003810 con 0x7f3a6c161e70 2024-05-30T17:00:37.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.882+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6c09b9f0 con 0x7f3a6c161e70 2024-05-30T17:00:37.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.882+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6c09bfb0 con 0x7f3a6c161e70 2024-05-30T17:00:37.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.883+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a6c0689e0 con 0x7f3a6c161e70 2024-05-30T17:00:37.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.884+0000 7f3a63e00640 1 -- 172.21.15.67:0/2882024756 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f3a6401a070 con 0x7f3a6c161e70 2024-05-30T17:00:37.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.885+0000 7f3a63e00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a4806ddc0 0x7f3a48070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:37.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.885+0000 7f3a63e00640 1 -- 172.21.15.67:0/2882024756 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f3a64089be0 con 0x7f3a6c161e70 2024-05-30T17:00:37.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.885+0000 7f3a6be00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a4806ddc0 0x7f3a48070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:37.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.886+0000 7f3a6be00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a4806ddc0 0x7f3a48070280 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c006fd0 tx=0x7f3a5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:37.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:37.889+0000 7f3a63e00640 1 -- 172.21.15.67:0/2882024756 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a64059ac0 con 0x7f3a6c161e70 2024-05-30T17:00:38.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.132+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3a6c072770 con 0x7f3a6c161e70 2024-05-30T17:00:38.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.761+0000 7f3a63e00640 1 -- 172.21.15.67:0/2882024756 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) ==== 118+0+0 (secure 0 0 0) 0x7f3a640593c0 con 0x7f3a6c161e70 2024-05-30T17:00:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.765+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a4806ddc0 msgr2=0x7f3a48070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.765+0000 7f3a72a00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a4806ddc0 0x7f3a48070280 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c006fd0 tx=0x7f3a5c008040 comp rx=0 tx=0).stop 2024-05-30T17:00:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.765+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 msgr2=0x7f3a6c09d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.765+0000 7f3a72a00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 0x7f3a6c09d390 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400c400 tx=0x7f3a6400c8c0 comp rx=0 tx=0).stop 2024-05-30T17:00:38.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.766+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 shutdown_connections 2024-05-30T17:00:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.766+0000 7f3a72a00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a4806ddc0 0x7f3a48070280 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.766+0000 7f3a72a00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6c161e70 0x7f3a6c09d390 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.766+0000 7f3a72a00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a6c161a90 0x7f3a6c1466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.766+0000 7f3a72a00640 1 --2- 172.21.15.67:0/2882024756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a6c06e9b0 0x7f3a6c09c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.766+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 >> 172.21.15.67:0/2882024756 conn(0x7f3a6c06bd20 msgr2=0x7f3a6c0ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.766+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 shutdown_connections 2024-05-30T17:00:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:38.766+0000 7f3a72a00640 1 -- 172.21.15.67:0/2882024756 wait complete. 2024-05-30T17:00:38.791 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-30T17:00:38.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-30T17:00:39.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.007+0000 7fd535c00640 1 Processor -- start 2024-05-30T17:00:39.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.008+0000 7fd535c00640 1 -- start start 2024-05-30T17:00:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.009+0000 7fd535c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300cd2a0 0x7fd5300b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.009+0000 7fd535c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd53015d900 con 0x7fd530153510 2024-05-30T17:00:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.009+0000 7fd535c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5300b3f90 con 0x7fd53015cb60 2024-05-30T17:00:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.010+0000 7fd535c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd53015d6e0 con 0x7fd5300cd2a0 2024-05-30T17:00:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.010+0000 7fd52fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd530153510 0x7fd5300ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60992/0 (socket says 172.21.15.67:60992) 2024-05-30T17:00:39.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.010+0000 7fd52fe00640 1 -- 172.21.15.67:0/3455361638 learned_addr learned my addr 172.21.15.67:0/3455361638 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.010+0000 7fd527e00640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300cd2a0 0x7fd5300b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.010+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3334895823 0 0) 0x7fd53015d900 con 0x7fd530153510 2024-05-30T17:00:39.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.010+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd514003520 con 0x7fd530153510 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2125166289 0 0) 0x7fd514003520 con 0x7fd530153510 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300cd2a0 msgr2=0x7fd5300b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300cd2a0 0x7fd5300b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 >> v1:172.21.15.169:6789/0 conn(0x7fd53015cb60 legacy=0x7fd53015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5300b4270 con 0x7fd530153510 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fd520001670 con 0x7fd530153510 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 >> v1:172.21.15.67:6789/0 conn(0x7fd530153510 legacy=0x7fd5300ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5140039e0 0x7fd514003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300cd2a0 0x7fd514004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd514004d40 0x7fd5140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd53015d6e0 con 0x7fd5140039e0 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd51400baf0 con 0x7fd514004d40 2024-05-30T17:00:39.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd51400bcf0 con 0x7fd5300cd2a0 2024-05-30T17:00:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.011+0000 7fd52fe00640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5140039e0 0x7fd514003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.012+0000 7fd535200640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd514004d40 0x7fd5140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.012+0000 7fd527e00640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300cd2a0 0x7fd514004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.012+0000 7fd52fe00640 1 -- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300cd2a0 msgr2=0x7fd514004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.012+0000 7fd52fe00640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300cd2a0 0x7fd514004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.012+0000 7fd52fe00640 1 -- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd514004d40 msgr2=0x7fd5140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.012+0000 7fd52fe00640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd514004d40 0x7fd5140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.012+0000 7fd52fe00640 1 -- 172.21.15.67:0/3455361638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd514003520 con 0x7fd5140039e0 2024-05-30T17:00:39.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.012+0000 7fd527e00640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300cd2a0 0x7fd514004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.013+0000 7fd52fe00640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5140039e0 0x7fd514003e00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c210 tx=0x7fd514005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb0a09ead433dd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.013+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520007d10 con 0x7fd5140039e0 2024-05-30T17:00:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.013+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd520005af0 con 0x7fd5140039e0 2024-05-30T17:00:39.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.013+0000 7fd52f400640 1 -- 172.21.15.67:0/3455361638 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520007070 con 0x7fd5140039e0 2024-05-30T17:00:39.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.014+0000 7fd535c00640 1 -- 172.21.15.67:0/3455361638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5140039e0 msgr2=0x7fd514003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:39.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.014+0000 7fd535c00640 1 --2- 172.21.15.67:0/3455361638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5140039e0 0x7fd514003e00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c210 tx=0x7fd514005830 comp rx=0 tx=0).stop 2024-05-30T17:00:39.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.014+0000 7fd535200640 1 -- 172.21.15.67:0/3455361638 reap_dead start 2024-05-30T17:00:39.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.014+0000 7fd535c00640 1 -- 172.21.15.67:0/3455361638 shutdown_connections 2024-05-30T17:00:39.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.014+0000 7fd535c00640 1 -- 172.21.15.67:0/3455361638 >> 172.21.15.67:0/3455361638 conn(0x7fd53006b900 msgr2=0x7fd53006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:39.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.014+0000 7fd535c00640 1 -- 172.21.15.67:0/3455361638 shutdown_connections 2024-05-30T17:00:39.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.014+0000 7fd535c00640 1 -- 172.21.15.67:0/3455361638 wait complete. 2024-05-30T17:00:39.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.016+0000 7fd535c00640 1 Processor -- start 2024-05-30T17:00:39.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.016+0000 7fd535c00640 1 -- start start 2024-05-30T17:00:39.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd535c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5140039e0 0x7fd5300ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:39.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd535c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300727e0 0x7fd5301f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd535c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5300cb9a0 0x7fd5300cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd535c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd5300b48c0 con 0x7fd5300cb9a0 2024-05-30T17:00:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd535c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5300b4d50 con 0x7fd5140039e0 2024-05-30T17:00:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd535c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5300b4ed0 con 0x7fd5300727e0 2024-05-30T17:00:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd535200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5300cb9a0 0x7fd5300cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd535200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5300cb9a0 0x7fd5300cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43518/0 (socket says 172.21.15.67:43518) 2024-05-30T17:00:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd535200640 1 -- 172.21.15.67:0/1008649941 learned_addr learned my addr 172.21.15.67:0/1008649941 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:39.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.017+0000 7fd52fe00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5140039e0 0x7fd5300ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:39.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd535200640 1 -- 172.21.15.67:0/1008649941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300727e0 msgr2=0x7fd5301f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:39.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd535200640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300727e0 0x7fd5301f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:39.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd535200640 1 -- 172.21.15.67:0/1008649941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5140039e0 msgr2=0x7fd5300ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:39.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd535200640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5140039e0 0x7fd5300ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:39.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd535200640 1 -- 172.21.15.67:0/1008649941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5300cfc10 con 0x7fd5300cb9a0 2024-05-30T17:00:39.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd535200640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5300cb9a0 0x7fd5300cbe20 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd528007fd0 tx=0x7fd52800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:39.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd52cc00640 1 -- 172.21.15.67:0/1008649941 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528003970 con 0x7fd5300cb9a0 2024-05-30T17:00:39.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd52cc00640 1 -- 172.21.15.67:0/1008649941 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd528018070 con 0x7fd5300cb9a0 2024-05-30T17:00:39.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd52cc00640 1 -- 172.21.15.67:0/1008649941 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528003d70 con 0x7fd5300cb9a0 2024-05-30T17:00:39.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5300adc70 con 0x7fd5300cb9a0 2024-05-30T17:00:39.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.018+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5300ce160 con 0x7fd5300cb9a0 2024-05-30T17:00:39.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.019+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5300689e0 con 0x7fd5300cb9a0 2024-05-30T17:00:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.021+0000 7fd52cc00640 1 -- 172.21.15.67:0/1008649941 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fd5280050d0 con 0x7fd5300cb9a0 2024-05-30T17:00:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.022+0000 7fd52cc00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd50c06ddc0 0x7fd50c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.022+0000 7fd52cc00640 1 -- 172.21.15.67:0/1008649941 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fd52808a500 con 0x7fd5300cb9a0 2024-05-30T17:00:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.022+0000 7fd52fe00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd50c06ddc0 0x7fd50c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:39.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.023+0000 7fd52fe00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd50c06ddc0 0x7fd50c070280 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd520032000 tx=0x7fd520009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:39.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.027+0000 7fd52cc00640 1 -- 172.21.15.67:0/1008649941 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd528059d30 con 0x7fd5300cb9a0 2024-05-30T17:00:39.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.269+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd5300cce70 con 0x7fd5300cb9a0 2024-05-30T17:00:39.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.764+0000 7fd52cc00640 1 -- 172.21.15.67:0/1008649941 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) ==== 122+0+0 (secure 0 0 0) 0x7fd52805e030 con 0x7fd5300cb9a0 2024-05-30T17:00:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.767+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd50c06ddc0 msgr2=0x7fd50c070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.767+0000 7fd535c00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd50c06ddc0 0x7fd50c070280 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd520032000 tx=0x7fd520009030 comp rx=0 tx=0).stop 2024-05-30T17:00:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.767+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5300cb9a0 msgr2=0x7fd5300cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.767+0000 7fd535c00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5300cb9a0 0x7fd5300cbe20 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd528007fd0 tx=0x7fd52800ec10 comp rx=0 tx=0).stop 2024-05-30T17:00:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.768+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 shutdown_connections 2024-05-30T17:00:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.768+0000 7fd535c00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd50c06ddc0 0x7fd50c070280 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.768+0000 7fd535c00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5300cb9a0 0x7fd5300cbe20 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.768+0000 7fd535c00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5300727e0 0x7fd5301f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.768+0000 7fd535c00640 1 --2- 172.21.15.67:0/1008649941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5140039e0 0x7fd5300ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.768+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 >> 172.21.15.67:0/1008649941 conn(0x7fd53006b900 msgr2=0x7fd53006f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.768+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 shutdown_connections 2024-05-30T17:00:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:39.768+0000 7fd535c00640 1 -- 172.21.15.67:0/1008649941 wait complete. 2024-05-30T17:00:39.793 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-30T17:00:39.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:00:40.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.008+0000 7f6641e00640 1 Processor -- start 2024-05-30T17:00:40.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.008+0000 7f6641e00640 1 -- start start 2024-05-30T17:00:40.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.009+0000 7f6641e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c156af0 0x7f663c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.010+0000 7f6641e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f663c070050 0x7f663c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.010+0000 7f6641e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f663c156ed0 con 0x7f663c09af70 2024-05-30T17:00:40.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.010+0000 7f6641e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f663c15cb40 con 0x7f663c070050 2024-05-30T17:00:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.010+0000 7f6641e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f663c15ce20 con 0x7f663c156af0 2024-05-30T17:00:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.010+0000 7f6641400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f663c09af70 0x7f663c160020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:32770/0 (socket says 172.21.15.67:32770) 2024-05-30T17:00:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.010+0000 7f6641400640 1 -- 172.21.15.67:0/4135039094 learned_addr learned my addr 172.21.15.67:0/4135039094 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.010+0000 7f6640a00640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c156af0 0x7f663c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.010+0000 7f663be00640 1 --2- 172.21.15.67:0/4135039094 >> v2:172.21.15.169:3300/0 conn(0x7f663c070050 0x7f663c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3801919660 0 0) 0x7f663c156ed0 con 0x7f663c09af70 2024-05-30T17:00:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6620003520 con 0x7f663c09af70 2024-05-30T17:00:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1860127417 0 0) 0x7f6620003520 con 0x7f663c09af70 2024-05-30T17:00:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c156af0 msgr2=0x7f663c06fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c156af0 0x7f663c06fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 >> v2:172.21.15.169:3300/0 conn(0x7f663c070050 msgr2=0x7f663c09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 --2- 172.21.15.67:0/4135039094 >> v2:172.21.15.169:3300/0 conn(0x7f663c070050 0x7f663c09aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c15d100 con 0x7f663c09af70 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f66340019f0 con 0x7f663c09af70 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 >> v1:172.21.15.67:6789/0 conn(0x7f663c09af70 legacy=0x7f663c160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66200039b0 0x7f6620003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c156af0 0x7f6620004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f66200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f663c15cb40 con 0x7f6620004d40 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f663c15ce20 con 0x7f66200039b0 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.011+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f662000b9c0 con 0x7f663c156af0 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.012+0000 7f6641400640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f66200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.012+0000 7f6640a00640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66200039b0 0x7f6620003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.012+0000 7f663be00640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c156af0 0x7f6620004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.012+0000 7f6641400640 1 -- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c156af0 msgr2=0x7f6620004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.012+0000 7f6641400640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c156af0 0x7f6620004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.012+0000 7f6641400640 1 -- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66200039b0 msgr2=0x7f6620003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.012+0000 7f6641400640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66200039b0 0x7f6620003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.012+0000 7f6641400640 1 -- 172.21.15.67:0/4135039094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6620003520 con 0x7f6620004d40 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.013+0000 7f6641400640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f66200051c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f663400a7d0 tx=0x7f6620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c91d8b80435309 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.013+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663400a3f0 con 0x7f6620004d40 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.013+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663402f070 con 0x7f6620004d40 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.013+0000 7f663b400640 1 -- 172.21.15.67:0/4135039094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663402bc50 con 0x7f6620004d40 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.014+0000 7f6641e00640 1 -- 172.21.15.67:0/4135039094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 msgr2=0x7f66200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.014+0000 7f6641e00640 1 --2- 172.21.15.67:0/4135039094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f66200051c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f663400a7d0 tx=0x7f6620005830 comp rx=0 tx=0).stop 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.014+0000 7f6641400640 1 -- 172.21.15.67:0/4135039094 reap_dead start 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.014+0000 7f6641e00640 1 -- 172.21.15.67:0/4135039094 shutdown_connections 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.014+0000 7f6641e00640 1 -- 172.21.15.67:0/4135039094 >> 172.21.15.67:0/4135039094 conn(0x7f663c0b3ec0 msgr2=0x7f663c0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.014+0000 7f6641e00640 1 -- 172.21.15.67:0/4135039094 shutdown_connections 2024-05-30T17:00:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.014+0000 7f6641e00640 1 -- 172.21.15.67:0/4135039094 wait complete. 2024-05-30T17:00:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.016+0000 7f6641e00640 1 Processor -- start 2024-05-30T17:00:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.016+0000 7f6641e00640 1 -- start start 2024-05-30T17:00:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.017+0000 7f6641e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f663c09d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.017+0000 7f6641e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c15b770 0x7f663c0b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.017+0000 7f6641e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f663c0b1580 0x7f663c14aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.017+0000 7f6641e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f663c09bc70 con 0x7f6620004d40 2024-05-30T17:00:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.017+0000 7f6641e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f663c09baf0 con 0x7f663c0b1580 2024-05-30T17:00:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.017+0000 7f6641e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f663c0b61d0 con 0x7f663c15b770 2024-05-30T17:00:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.017+0000 7f6640a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f663c09d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.017+0000 7f6640a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f663c09d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43544/0 (socket says 172.21.15.67:43544) 2024-05-30T17:00:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.017+0000 7f6640a00640 1 -- 172.21.15.67:0/1335071326 learned_addr learned my addr 172.21.15.67:0/1335071326 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.018+0000 7f6641400640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f663c0b1580 0x7f663c14aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.018+0000 7f663be00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c15b770 0x7f663c0b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.018+0000 7f6640a00640 1 -- 172.21.15.67:0/1335071326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c15b770 msgr2=0x7f663c0b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.018+0000 7f6640a00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c15b770 0x7f663c0b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.018+0000 7f6640a00640 1 -- 172.21.15.67:0/1335071326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f663c0b1580 msgr2=0x7f663c14aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.018+0000 7f6640a00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f663c0b1580 0x7f663c14aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.018+0000 7f6640a00640 1 -- 172.21.15.67:0/1335071326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c09c2c0 con 0x7f6620004d40 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.018+0000 7f6640a00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f663c09d360 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f662c011540 tx=0x7f662c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.019+0000 7f6638c00640 1 -- 172.21.15.67:0/1335071326 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c015930 con 0x7f6620004d40 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.019+0000 7f6638c00640 1 -- 172.21.15.67:0/1335071326 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f662c015ad0 con 0x7f6620004d40 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.019+0000 7f6638c00640 1 -- 172.21.15.67:0/1335071326 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c00f460 con 0x7f6620004d40 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.019+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663c09c5a0 con 0x7f6620004d40 2024-05-30T17:00:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.019+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663c09cb60 con 0x7f6620004d40 2024-05-30T17:00:40.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.020+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f663c0690f0 con 0x7f6620004d40 2024-05-30T17:00:40.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.021+0000 7f6638c00640 1 -- 172.21.15.67:0/1335071326 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f662c00f600 con 0x7f6620004d40 2024-05-30T17:00:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.022+0000 7f6638c00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f661c06ddc0 0x7f661c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.022+0000 7f6638c00640 1 -- 172.21.15.67:0/1335071326 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f662c08b460 con 0x7f6620004d40 2024-05-30T17:00:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.022+0000 7f663be00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f661c06ddc0 0x7f661c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.023+0000 7f663be00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f661c06ddc0 0x7f661c070280 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f663c15c240 tx=0x7f6628008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.027+0000 7f6638c00640 1 -- 172.21.15.67:0/1335071326 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f662c019030 con 0x7f6620004d40 2024-05-30T17:00:40.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.274+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f663c0b1bb0 con 0x7f6620004d40 2024-05-30T17:00:40.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.276+0000 7f6638c00640 1 -- 172.21.15.67:0/1335071326 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5189 (secure 0 0 0) 0x7f662c05af60 con 0x7f6620004d40 2024-05-30T17:00:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:40.277 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":64,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2641805294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2641805294},{"type":"v1","addr":"172.21.15.67:6837","nonce":2641805294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2609681849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2609681849},{"type":"v1","addr":"172.21.15.169:6834","nonce":2609681849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2372354106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2372354106},{"type":"v1","addr":"172.21.15.169:6835","nonce":2372354106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:00:36.172699+0000","modified":"2024-05-30T17:00:39.271296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7902":{"gid":7902,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6836/2120049133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2120049133},{"type":"v1","addr":"172.21.15.67:6836","nonce":2120049133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7902,"qdb_cluster":[7902]},"id":7}]} 2024-05-30T17:00:40.277 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 64 2024-05-30T17:00:40.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f661c06ddc0 msgr2=0x7f661c070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f661c06ddc0 0x7f661c070280 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f663c15c240 tx=0x7f6628008040 comp rx=0 tx=0).stop 2024-05-30T17:00:40.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 msgr2=0x7f663c09d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f663c09d360 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f662c011540 tx=0x7f662c011a00 comp rx=0 tx=0).stop 2024-05-30T17:00:40.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 shutdown_connections 2024-05-30T17:00:40.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f661c06ddc0 0x7f661c070280 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f663c0b1580 0x7f663c14aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f663c15b770 0x7f663c0b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 --2- 172.21.15.67:0/1335071326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6620004d40 0x7f663c09d360 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.279+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 >> 172.21.15.67:0/1335071326 conn(0x7f663c0b3ec0 msgr2=0x7f663c0b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:40.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.280+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 shutdown_connections 2024-05-30T17:00:40.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.280+0000 7f6641e00640 1 -- 172.21.15.67:0/1335071326 wait complete. 2024-05-30T17:00:40.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:00:40.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.514+0000 7f5fbd800640 1 Processor -- start 2024-05-30T17:00:40.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.514+0000 7f5fbd800640 1 -- start start 2024-05-30T17:00:40.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.516+0000 7f5fbd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8156b30 0x7f5fb80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.516+0000 7f5fbd800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fb8156f10 con 0x7f5fb80ad940 2024-05-30T17:00:40.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.516+0000 7f5fbd800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fb806dda0 con 0x7f5fb806ee40 2024-05-30T17:00:40.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.516+0000 7f5fbd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb806e080 con 0x7f5fb8156b30 2024-05-30T17:00:40.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.517+0000 7f5fbce00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5fb80ad940 0x7f5fb80b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:32786/0 (socket says 172.21.15.67:32786) 2024-05-30T17:00:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.517+0000 7f5fbce00640 1 -- 172.21.15.67:0/206209048 learned_addr learned my addr 172.21.15.67:0/206209048 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.517+0000 7f5fb7e00640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8156b30 0x7f5fb80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.517+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477819195 0 0) 0x7f5fb8156f10 con 0x7f5fb80ad940 2024-05-30T17:00:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.517+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f9c003520 con 0x7f5fb80ad940 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 584773879 0 0) 0x7f5f9c003520 con 0x7f5fb80ad940 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8156b30 msgr2=0x7f5fb80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8156b30 0x7f5fb80af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 >> v1:172.21.15.169:6789/0 conn(0x7f5fb806ee40 legacy=0x7f5fb80ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb806e360 con 0x7f5fb80ad940 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5fb0001670 con 0x7f5fb80ad940 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 >> v1:172.21.15.67:6789/0 conn(0x7f5fb80ad940 legacy=0x7f5fb80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8156b30 0x7f5f9c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f9c003fb0 0x7f5f9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c004d40 0x7f5f9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb806e080 con 0x7f5f9c003fb0 2024-05-30T17:00:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c00bbb0 con 0x7f5f9c004d40 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9c00bdb0 con 0x7f5fb8156b30 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb7400640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f9c003fb0 0x7f5f9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.518+0000 7f5fb7e00640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8156b30 0x7f5f9c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.519+0000 7f5fbce00640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c004d40 0x7f5f9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.519+0000 7f5fb7400640 1 -- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8156b30 msgr2=0x7f5f9c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.519+0000 7f5fb7400640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8156b30 0x7f5f9c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.519+0000 7f5fb7400640 1 -- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c004d40 msgr2=0x7f5f9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.519+0000 7f5fb7400640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c004d40 0x7f5f9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.519+0000 7f5fb7400640 1 -- 172.21.15.67:0/206209048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c003520 con 0x7f5f9c003fb0 2024-05-30T17:00:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.519+0000 7f5fbce00640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c004d40 0x7f5f9c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.520+0000 7f5fb7400640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f9c003fb0 0x7f5f9c004800 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4001e50 tx=0x7f5f9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df54e046fd59da7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.520+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4001a70 con 0x7f5f9c003fb0 2024-05-30T17:00:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.520+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa4003970 con 0x7f5f9c003fb0 2024-05-30T17:00:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.520+0000 7f5fb6a00640 1 -- 172.21.15.67:0/206209048 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4003d60 con 0x7f5f9c003fb0 2024-05-30T17:00:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.521+0000 7f5fbd800640 1 -- 172.21.15.67:0/206209048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f9c003fb0 msgr2=0x7f5f9c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.521+0000 7f5fbd800640 1 --2- 172.21.15.67:0/206209048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f9c003fb0 0x7f5f9c004800 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4001e50 tx=0x7f5f9c005830 comp rx=0 tx=0).stop 2024-05-30T17:00:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.521+0000 7f5fbce00640 1 -- 172.21.15.67:0/206209048 reap_dead start 2024-05-30T17:00:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.521+0000 7f5fbd800640 1 -- 172.21.15.67:0/206209048 shutdown_connections 2024-05-30T17:00:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.521+0000 7f5fbd800640 1 -- 172.21.15.67:0/206209048 >> 172.21.15.67:0/206209048 conn(0x7f5fb80b1fc0 msgr2=0x7f5fb80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.521+0000 7f5fbd800640 1 -- 172.21.15.67:0/206209048 shutdown_connections 2024-05-30T17:00:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.521+0000 7f5fbd800640 1 -- 172.21.15.67:0/206209048 wait complete. 2024-05-30T17:00:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.523+0000 7f5fbd800640 1 Processor -- start 2024-05-30T17:00:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.523+0000 7f5fbd800640 1 -- start start 2024-05-30T17:00:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbd800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c003fb0 0x7f5fb81486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8072d40 0x7f5fb80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb8158bc0 0x7f5fb8159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbd800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb80b2db0 con 0x7f5fb8158bc0 2024-05-30T17:00:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbd800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb80b2c30 con 0x7f5f9c003fb0 2024-05-30T17:00:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb80b39d0 con 0x7f5fb8072d40 2024-05-30T17:00:40.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb8158bc0 0x7f5fb8159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb8158bc0 0x7f5fb8159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43568/0 (socket says 172.21.15.67:43568) 2024-05-30T17:00:40.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbce00640 1 -- 172.21.15.67:0/1331309170 learned_addr learned my addr 172.21.15.67:0/1331309170 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:40.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fb7e00640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c003fb0 0x7f5fb81486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fb7400640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8072d40 0x7f5fb80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbce00640 1 -- 172.21.15.67:0/1331309170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8072d40 msgr2=0x7f5fb80731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbce00640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8072d40 0x7f5fb80731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbce00640 1 -- 172.21.15.67:0/1331309170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c003fb0 msgr2=0x7f5fb81486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbce00640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c003fb0 0x7f5fb81486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.524+0000 7f5fbce00640 1 -- 172.21.15.67:0/1331309170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb8159580 con 0x7f5fb8158bc0 2024-05-30T17:00:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.525+0000 7f5fbce00640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb8158bc0 0x7f5fb8159040 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5fb806fe10 tx=0x7f5fb000d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.525+0000 7f5fafe00640 1 -- 172.21.15.67:0/1331309170 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00074e0 con 0x7f5fb8158bc0 2024-05-30T17:00:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.525+0000 7f5fafe00640 1 -- 172.21.15.67:0/1331309170 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb00047e0 con 0x7f5fb8158bc0 2024-05-30T17:00:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.525+0000 7f5fafe00640 1 -- 172.21.15.67:0/1331309170 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0004bb0 con 0x7f5fb8158bc0 2024-05-30T17:00:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.525+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb8148c00 con 0x7f5fb8158bc0 2024-05-30T17:00:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.525+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb80b05e0 con 0x7f5fb8158bc0 2024-05-30T17:00:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.527+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb8073700 con 0x7f5fb8158bc0 2024-05-30T17:00:40.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.528+0000 7f5fafe00640 1 -- 172.21.15.67:0/1331309170 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f5fb00133d0 con 0x7f5fb8158bc0 2024-05-30T17:00:40.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.529+0000 7f5fafe00640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f9806ddc0 0x7f5f98070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.529+0000 7f5fafe00640 1 -- 172.21.15.67:0/1331309170 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f5fb0089f00 con 0x7f5fb8158bc0 2024-05-30T17:00:40.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.529+0000 7f5fb7e00640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f9806ddc0 0x7f5f98070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.530+0000 7f5fb7e00640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f9806ddc0 0x7f5f98070280 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009a10 tx=0x7f5fa8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:40.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.534+0000 7f5fafe00640 1 -- 172.21.15.67:0/1331309170 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fb0091030 con 0x7f5fb8158bc0 2024-05-30T17:00:40.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.737+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fb80689e0 con 0x7f5fb8158bc0 2024-05-30T17:00:40.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.740+0000 7f5fafe00640 1 -- 172.21.15.67:0/1331309170 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16739 (secure 0 0 0) 0x7f5fb0059de0 con 0x7f5fb8158bc0 2024-05-30T17:00:40.741 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:40.741 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":104,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:00:37.736006+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:00:34.151659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-30T17:00:35.170102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:00:40.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f9806ddc0 msgr2=0x7f5f98070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f9806ddc0 0x7f5f98070280 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009a10 tx=0x7f5fa8008040 comp rx=0 tx=0).stop 2024-05-30T17:00:40.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb8158bc0 msgr2=0x7f5fb8159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb8158bc0 0x7f5fb8159040 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5fb806fe10 tx=0x7f5fb000d900 comp rx=0 tx=0).stop 2024-05-30T17:00:40.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 shutdown_connections 2024-05-30T17:00:40.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5f9806ddc0 0x7f5f98070280 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fb8158bc0 0x7f5fb8159040 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fb8072d40 0x7f5fb80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 --2- 172.21.15.67:0/1331309170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f9c003fb0 0x7f5fb81486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 >> 172.21.15.67:0/1331309170 conn(0x7f5fb80b1fc0 msgr2=0x7f5fb80b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:40.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.743+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 shutdown_connections 2024-05-30T17:00:40.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.744+0000 7f5fbd800640 1 -- 172.21.15.67:0/1331309170 wait complete. 2024-05-30T17:00:40.762 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T17:00:40.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:00:40.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.963+0000 7f7d6f000640 1 Processor -- start 2024-05-30T17:00:40.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.963+0000 7f7d6f000640 1 -- start start 2024-05-30T17:00:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.965+0000 7f7d6f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d680704e0 0x7f7d680cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.965+0000 7f7d6f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7d680d10c0 0x7f7d680708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.965+0000 7f7d6f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6815fdf0 con 0x7f7d680ce2e0 2024-05-30T17:00:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.965+0000 7f7d6f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6815b780 con 0x7f7d680704e0 2024-05-30T17:00:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.965+0000 7f7d6f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7d68070e00 con 0x7f7d680d10c0 2024-05-30T17:00:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.965+0000 7f7d6da00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7d680ce2e0 0x7f7d680ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56688/0 (socket says 172.21.15.67:56688) 2024-05-30T17:00:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.965+0000 7f7d67e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d680704e0 0x7f7d680cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.965+0000 7f7d6da00640 1 -- 172.21.15.67:0/567306962 learned_addr learned my addr 172.21.15.67:0/567306962 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.966+0000 7f7d67400640 1 --2- 172.21.15.67:0/567306962 >> v2:172.21.15.169:3300/0 conn(0x7f7d680d10c0 0x7f7d680708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.966+0000 7f7d67e00640 1 -- 172.21.15.67:0/567306962 >> v2:172.21.15.169:3300/0 conn(0x7f7d680d10c0 msgr2=0x7f7d680708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.966+0000 7f7d67e00640 1 --2- 172.21.15.67:0/567306962 >> v2:172.21.15.169:3300/0 conn(0x7f7d680d10c0 0x7f7d680708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.966+0000 7f7d67e00640 1 -- 172.21.15.67:0/567306962 >> v1:172.21.15.67:6789/0 conn(0x7f7d680ce2e0 legacy=0x7f7d680ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.966+0000 7f7d67e00640 1 -- 172.21.15.67:0/567306962 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d680d0dc0 con 0x7f7d680704e0 2024-05-30T17:00:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.966+0000 7f7d66a00640 1 -- 172.21.15.67:0/567306962 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 865989294 0 0) 0x7f7d6815fdf0 con 0x7f7d680ce2e0 2024-05-30T17:00:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.966+0000 7f7d67400640 1 --2- 172.21.15.67:0/567306962 >> v2:172.21.15.169:3300/0 conn(0x7f7d680d10c0 0x7f7d680708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.967+0000 7f7d67e00640 1 --2- 172.21.15.67:0/567306962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680704e0 0x7f7d680cca00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f7d6806f0b0 tx=0x7f7d5802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8511f100a7df5f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.967+0000 7f7d66a00640 1 -- 172.21.15.67:0/567306962 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58037070 con 0x7f7d680704e0 2024-05-30T17:00:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.967+0000 7f7d66a00640 1 -- 172.21.15.67:0/567306962 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d58032440 con 0x7f7d680704e0 2024-05-30T17:00:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.967+0000 7f7d66a00640 1 -- 172.21.15.67:0/567306962 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5802ae60 con 0x7f7d680704e0 2024-05-30T17:00:40.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.968+0000 7f7d6f000640 1 -- 172.21.15.67:0/567306962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680704e0 msgr2=0x7f7d680cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.968+0000 7f7d6f000640 1 --2- 172.21.15.67:0/567306962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680704e0 0x7f7d680cca00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f7d6806f0b0 tx=0x7f7d5802a110 comp rx=0 tx=0).stop 2024-05-30T17:00:40.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.968+0000 7f7d6f000640 1 -- 172.21.15.67:0/567306962 shutdown_connections 2024-05-30T17:00:40.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.968+0000 7f7d6f000640 1 --2- 172.21.15.67:0/567306962 >> v2:172.21.15.169:3300/0 conn(0x7f7d680d10c0 0x7f7d680708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.968+0000 7f7d6f000640 1 --2- 172.21.15.67:0/567306962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680704e0 0x7f7d680cca00 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.968+0000 7f7d6f000640 1 -- 172.21.15.67:0/567306962 >> 172.21.15.67:0/567306962 conn(0x7f7d680b3550 msgr2=0x7f7d680b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:40.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.968+0000 7f7d6f000640 1 -- 172.21.15.67:0/567306962 shutdown_connections 2024-05-30T17:00:40.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.968+0000 7f7d6f000640 1 -- 172.21.15.67:0/567306962 wait complete. 2024-05-30T17:00:40.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.970+0000 7f7d6f000640 1 Processor -- start 2024-05-30T17:00:40.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.970+0000 7f7d6f000640 1 -- start start 2024-05-30T17:00:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d6f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d680704e0 0x7f7d680c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d6f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680ce2e0 0x7f7d68156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d6f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d680d10c0 0x7f7d681f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d6f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6815f830 con 0x7f7d680ce2e0 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d6f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6815f550 con 0x7f7d680704e0 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d6f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6815fb10 con 0x7f7d680d10c0 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680ce2e0 0x7f7d68156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680ce2e0 0x7f7d68156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53112/0 (socket says 172.21.15.67:53112) 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67400640 1 -- 172.21.15.67:0/2206472579 learned_addr learned my addr 172.21.15.67:0/2206472579 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67e00640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d680704e0 0x7f7d680c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67400640 1 -- 172.21.15.67:0/2206472579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d680d10c0 msgr2=0x7f7d681f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d6da00640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d680d10c0 0x7f7d681f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67400640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d680d10c0 0x7f7d681f41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67400640 1 -- 172.21.15.67:0/2206472579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d680704e0 msgr2=0x7f7d680c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67400640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d680704e0 0x7f7d680c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67400640 1 -- 172.21.15.67:0/2206472579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d680c61a0 con 0x7f7d680ce2e0 2024-05-30T17:00:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.971+0000 7f7d67e00640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d680704e0 0x7f7d680c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.972+0000 7f7d67400640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680ce2e0 0x7f7d68156de0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c7f0 tx=0x7f7d5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.972+0000 7f7d5fe00640 1 -- 172.21.15.67:0/2206472579 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54015070 con 0x7f7d680ce2e0 2024-05-30T17:00:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.972+0000 7f7d5fe00640 1 -- 172.21.15.67:0/2206472579 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d54007820 con 0x7f7d680ce2e0 2024-05-30T17:00:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.972+0000 7f7d5fe00640 1 -- 172.21.15.67:0/2206472579 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54007bd0 con 0x7f7d680ce2e0 2024-05-30T17:00:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.972+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d680c9c00 con 0x7f7d680ce2e0 2024-05-30T17:00:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.972+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d680ca1c0 con 0x7f7d680ce2e0 2024-05-30T17:00:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.973+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d680689e0 con 0x7f7d680ce2e0 2024-05-30T17:00:40.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.974+0000 7f7d5fe00640 1 -- 172.21.15.67:0/2206472579 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f7d54018070 con 0x7f7d680ce2e0 2024-05-30T17:00:40.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.975+0000 7f7d5fe00640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d4806ddc0 0x7f7d48070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:40.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.975+0000 7f7d5fe00640 1 -- 172.21.15.67:0/2206472579 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f7d5408a970 con 0x7f7d680ce2e0 2024-05-30T17:00:40.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.976+0000 7f7d67e00640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d4806ddc0 0x7f7d48070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.976+0000 7f7d67e00640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d4806ddc0 0x7f7d48070280 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7d58034000 tx=0x7f7d5802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:40.980+0000 7f7d5fe00640 1 -- 172.21.15.67:0/2206472579 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d5405a850 con 0x7f7d680ce2e0 2024-05-30T17:00:41.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.182+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d680b4400 con 0x7f7d4806ddc0 2024-05-30T17:00:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.201+0000 7f7d5fe00640 1 -- 172.21.15.67:0/2206472579 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249442 (secure 0 0 0) 0x7f7d680b4400 con 0x7f7d4806ddc0 2024-05-30T17:00:41.206 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:00:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d4806ddc0 msgr2=0x7f7d48070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d4806ddc0 0x7f7d48070280 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7d58034000 tx=0x7f7d5802e000 comp rx=0 tx=0).stop 2024-05-30T17:00:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680ce2e0 msgr2=0x7f7d68156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680ce2e0 0x7f7d68156de0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c7f0 tx=0x7f7d5400ccb0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 shutdown_connections 2024-05-30T17:00:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7d4806ddc0 0x7f7d48070280 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d680d10c0 0x7f7d681f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d680ce2e0 0x7f7d68156de0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 --2- 172.21.15.67:0/2206472579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d680704e0 0x7f7d680c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 >> 172.21.15.67:0/2206472579 conn(0x7f7d680b3550 msgr2=0x7f7d680ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:41.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 shutdown_connections 2024-05-30T17:00:41.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.207+0000 7f7d6f000640 1 -- 172.21.15.67:0/2206472579 wait complete. 2024-05-30T17:00:41.226 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":608,"stamp":"2024-05-30T17:00:40.472092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":888904,"kb_used_data":11092,"kb_used_omap":12,"kb_used_meta":877683,"kb_avail":754085816,"statfs":{"total":773094113280,"available":772183875584,"internally_reserved":0,"allocated":11358208,"data_stored":5593113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":898747952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.782789"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752224+0000","last_change":"2024-05-30T17:00:36.748181+0000","last_active":"2024-05-30T17:00:37.752224+0000","last_peered":"2024-05-30T17:00:37.752224+0000","last_clean":"2024-05-30T17:00:37.752224+0000","last_became_active":"2024-05-30T17:00:36.747392+0000","last_became_peered":"2024-05-30T17:00:36.747392+0000","last_unstale":"2024-05-30T17:00:37.752224+0000","last_undegraded":"2024-05-30T17:00:37.752224+0000","last_fullsized":"2024-05-30T17:00:37.752224+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.712512+0000","last_change":"2024-05-30T17:00:35.712512+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.712512+0000","last_undegraded":"2024-05-30T17:00:35.712512+0000","last_fullsized":"2024-05-30T17:00:35.712512+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151835+0000","last_change":"2024-05-30T17:00:36.763111+0000","last_active":"2024-05-30T17:00:38.151835+0000","last_peered":"2024-05-30T17:00:38.151835+0000","last_clean":"2024-05-30T17:00:38.151835+0000","last_became_active":"2024-05-30T17:00:36.761404+0000","last_became_peered":"2024-05-30T17:00:36.761404+0000","last_unstale":"2024-05-30T17:00:38.151835+0000","last_undegraded":"2024-05-30T17:00:38.151835+0000","last_fullsized":"2024-05-30T17:00:38.151835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:01.455423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149546+0000","last_change":"2024-05-30T17:00:36.751908+0000","last_active":"2024-05-30T17:00:38.149546+0000","last_peered":"2024-05-30T17:00:38.149546+0000","last_clean":"2024-05-30T17:00:38.149546+0000","last_became_active":"2024-05-30T17:00:36.751604+0000","last_became_peered":"2024-05-30T17:00:36.751604+0000","last_unstale":"2024-05-30T17:00:38.149546+0000","last_undegraded":"2024-05-30T17:00:38.149546+0000","last_fullsized":"2024-05-30T17:00:38.149546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149447+0000","last_change":"2024-05-30T17:00:36.748294+0000","last_active":"2024-05-30T17:00:38.149447+0000","last_peered":"2024-05-30T17:00:38.149447+0000","last_clean":"2024-05-30T17:00:38.149447+0000","last_became_active":"2024-05-30T17:00:36.747984+0000","last_became_peered":"2024-05-30T17:00:36.747984+0000","last_unstale":"2024-05-30T17:00:38.149447+0000","last_undegraded":"2024-05-30T17:00:38.149447+0000","last_fullsized":"2024-05-30T17:00:38.149447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751747+0000","last_change":"2024-05-30T17:00:36.751171+0000","last_active":"2024-05-30T17:00:37.751747+0000","last_peered":"2024-05-30T17:00:37.751747+0000","last_clean":"2024-05-30T17:00:37.751747+0000","last_became_active":"2024-05-30T17:00:36.747956+0000","last_became_peered":"2024-05-30T17:00:36.747956+0000","last_unstale":"2024-05-30T17:00:37.751747+0000","last_undegraded":"2024-05-30T17:00:37.751747+0000","last_fullsized":"2024-05-30T17:00:37.751747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:03:57.450595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760121+0000","last_change":"2024-05-30T17:00:36.762682+0000","last_active":"2024-05-30T17:00:37.760121+0000","last_peered":"2024-05-30T17:00:37.760121+0000","last_clean":"2024-05-30T17:00:37.760121+0000","last_became_active":"2024-05-30T17:00:36.762190+0000","last_became_peered":"2024-05-30T17:00:36.762190+0000","last_unstale":"2024-05-30T17:00:37.760121+0000","last_undegraded":"2024-05-30T17:00:37.760121+0000","last_fullsized":"2024-05-30T17:00:37.760121+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:33.645864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151899+0000","last_change":"2024-05-30T17:00:36.758850+0000","last_active":"2024-05-30T17:00:38.151899+0000","last_peered":"2024-05-30T17:00:38.151899+0000","last_clean":"2024-05-30T17:00:38.151899+0000","last_became_active":"2024-05-30T17:00:36.758522+0000","last_became_peered":"2024-05-30T17:00:36.758522+0000","last_unstale":"2024-05-30T17:00:38.151899+0000","last_undegraded":"2024-05-30T17:00:38.151899+0000","last_fullsized":"2024-05-30T17:00:38.151899+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:20.247961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149815+0000","last_change":"2024-05-30T17:00:36.750772+0000","last_active":"2024-05-30T17:00:38.149815+0000","last_peered":"2024-05-30T17:00:38.149815+0000","last_clean":"2024-05-30T17:00:38.149815+0000","last_became_active":"2024-05-30T17:00:36.750122+0000","last_became_peered":"2024-05-30T17:00:36.750122+0000","last_unstale":"2024-05-30T17:00:38.149815+0000","last_undegraded":"2024-05-30T17:00:38.149815+0000","last_fullsized":"2024-05-30T17:00:38.149815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761029+0000","last_change":"2024-05-30T17:00:36.751475+0000","last_active":"2024-05-30T17:00:37.761029+0000","last_peered":"2024-05-30T17:00:37.761029+0000","last_clean":"2024-05-30T17:00:37.761029+0000","last_became_active":"2024-05-30T17:00:36.751150+0000","last_became_peered":"2024-05-30T17:00:36.751150+0000","last_unstale":"2024-05-30T17:00:37.761029+0000","last_undegraded":"2024-05-30T17:00:37.761029+0000","last_fullsized":"2024-05-30T17:00:37.761029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:55:05.659729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.712050+0000","last_change":"2024-05-30T17:00:35.712050+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.712050+0000","last_undegraded":"2024-05-30T17:00:35.712050+0000","last_fullsized":"2024-05-30T17:00:35.712050+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.721668+0000","last_change":"2024-05-30T17:00:35.721668+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.721668+0000","last_undegraded":"2024-05-30T17:00:35.721668+0000","last_fullsized":"2024-05-30T17:00:35.721668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.711828+0000","last_change":"2024-05-30T17:00:35.711828+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.711828+0000","last_undegraded":"2024-05-30T17:00:35.711828+0000","last_fullsized":"2024-05-30T17:00:35.711828+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149705+0000","last_change":"2024-05-30T17:00:36.758779+0000","last_active":"2024-05-30T17:00:38.149705+0000","last_peered":"2024-05-30T17:00:38.149705+0000","last_clean":"2024-05-30T17:00:38.149705+0000","last_became_active":"2024-05-30T17:00:36.758402+0000","last_became_peered":"2024-05-30T17:00:36.758402+0000","last_unstale":"2024-05-30T17:00:38.149705+0000","last_undegraded":"2024-05-30T17:00:38.149705+0000","last_fullsized":"2024-05-30T17:00:38.149705+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752387+0000","last_change":"2024-05-30T17:00:36.759407+0000","last_active":"2024-05-30T17:00:37.752387+0000","last_peered":"2024-05-30T17:00:37.752387+0000","last_clean":"2024-05-30T17:00:37.752387+0000","last_became_active":"2024-05-30T17:00:36.759115+0000","last_became_peered":"2024-05-30T17:00:36.759115+0000","last_unstale":"2024-05-30T17:00:37.752387+0000","last_undegraded":"2024-05-30T17:00:37.752387+0000","last_fullsized":"2024-05-30T17:00:37.752387+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.710227+0000","last_change":"2024-05-30T17:00:35.710227+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.710227+0000","last_undegraded":"2024-05-30T17:00:35.710227+0000","last_fullsized":"2024-05-30T17:00:35.710227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.719035+0000","last_change":"2024-05-30T17:00:35.719035+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.719035+0000","last_undegraded":"2024-05-30T17:00:35.719035+0000","last_fullsized":"2024-05-30T17:00:35.719035+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760586+0000","last_change":"2024-05-30T17:00:36.762646+0000","last_active":"2024-05-30T17:00:37.760586+0000","last_peered":"2024-05-30T17:00:37.760586+0000","last_clean":"2024-05-30T17:00:37.760586+0000","last_became_active":"2024-05-30T17:00:36.761684+0000","last_became_peered":"2024-05-30T17:00:36.761684+0000","last_unstale":"2024-05-30T17:00:37.760586+0000","last_undegraded":"2024-05-30T17:00:37.760586+0000","last_fullsized":"2024-05-30T17:00:37.760586+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:50.133298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720502+0000","last_change":"2024-05-30T17:00:35.720502+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720502+0000","last_undegraded":"2024-05-30T17:00:35.720502+0000","last_fullsized":"2024-05-30T17:00:35.720502+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751731+0000","last_change":"2024-05-30T17:00:36.745182+0000","last_active":"2024-05-30T17:00:37.751731+0000","last_peered":"2024-05-30T17:00:37.751731+0000","last_clean":"2024-05-30T17:00:37.751731+0000","last_became_active":"2024-05-30T17:00:36.744869+0000","last_became_peered":"2024-05-30T17:00:36.744869+0000","last_unstale":"2024-05-30T17:00:37.751731+0000","last_undegraded":"2024-05-30T17:00:37.751731+0000","last_fullsized":"2024-05-30T17:00:37.751731+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:52:34.241585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751614+0000","last_change":"2024-05-30T17:00:36.746180+0000","last_active":"2024-05-30T17:00:37.751614+0000","last_peered":"2024-05-30T17:00:37.751614+0000","last_clean":"2024-05-30T17:00:37.751614+0000","last_became_active":"2024-05-30T17:00:36.745907+0000","last_became_peered":"2024-05-30T17:00:36.745907+0000","last_unstale":"2024-05-30T17:00:37.751614+0000","last_undegraded":"2024-05-30T17:00:37.751614+0000","last_fullsized":"2024-05-30T17:00:37.751614+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:43.488502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150152+0000","last_change":"2024-05-30T17:00:36.748364+0000","last_active":"2024-05-30T17:00:38.150152+0000","last_peered":"2024-05-30T17:00:38.150152+0000","last_clean":"2024-05-30T17:00:38.150152+0000","last_became_active":"2024-05-30T17:00:36.748051+0000","last_became_peered":"2024-05-30T17:00:36.748051+0000","last_unstale":"2024-05-30T17:00:38.150152+0000","last_undegraded":"2024-05-30T17:00:38.150152+0000","last_fullsized":"2024-05-30T17:00:38.150152+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:28.418390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151939+0000","last_change":"2024-05-30T17:00:36.754727+0000","last_active":"2024-05-30T17:00:38.151939+0000","last_peered":"2024-05-30T17:00:38.151939+0000","last_clean":"2024-05-30T17:00:38.151939+0000","last_became_active":"2024-05-30T17:00:36.753257+0000","last_became_peered":"2024-05-30T17:00:36.753257+0000","last_unstale":"2024-05-30T17:00:38.151939+0000","last_undegraded":"2024-05-30T17:00:38.151939+0000","last_fullsized":"2024-05-30T17:00:38.151939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:36.271526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752343+0000","last_change":"2024-05-30T17:00:36.754136+0000","last_active":"2024-05-30T17:00:37.752343+0000","last_peered":"2024-05-30T17:00:37.752343+0000","last_clean":"2024-05-30T17:00:37.752343+0000","last_became_active":"2024-05-30T17:00:36.753838+0000","last_became_peered":"2024-05-30T17:00:36.753838+0000","last_unstale":"2024-05-30T17:00:37.752343+0000","last_undegraded":"2024-05-30T17:00:37.752343+0000","last_fullsized":"2024-05-30T17:00:37.752343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149843+0000","last_change":"2024-05-30T17:00:36.753152+0000","last_active":"2024-05-30T17:00:38.149843+0000","last_peered":"2024-05-30T17:00:38.149843+0000","last_clean":"2024-05-30T17:00:38.149843+0000","last_became_active":"2024-05-30T17:00:36.752867+0000","last_became_peered":"2024-05-30T17:00:36.752867+0000","last_unstale":"2024-05-30T17:00:38.149843+0000","last_undegraded":"2024-05-30T17:00:38.149843+0000","last_fullsized":"2024-05-30T17:00:38.149843+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:19:07.666127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152257+0000","last_change":"2024-05-30T17:00:36.761708+0000","last_active":"2024-05-30T17:00:38.152257+0000","last_peered":"2024-05-30T17:00:38.152257+0000","last_clean":"2024-05-30T17:00:38.152257+0000","last_became_active":"2024-05-30T17:00:36.760932+0000","last_became_peered":"2024-05-30T17:00:36.760932+0000","last_unstale":"2024-05-30T17:00:38.152257+0000","last_undegraded":"2024-05-30T17:00:38.152257+0000","last_fullsized":"2024-05-30T17:00:38.152257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:40.779484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752069+0000","last_change":"2024-05-30T17:00:36.755732+0000","last_active":"2024-05-30T17:00:37.752069+0000","last_peered":"2024-05-30T17:00:37.752069+0000","last_clean":"2024-05-30T17:00:37.752069+0000","last_became_active":"2024-05-30T17:00:36.755428+0000","last_became_peered":"2024-05-30T17:00:36.755428+0000","last_unstale":"2024-05-30T17:00:37.752069+0000","last_undegraded":"2024-05-30T17:00:37.752069+0000","last_fullsized":"2024-05-30T17:00:37.752069+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:14.194525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.721256+0000","last_change":"2024-05-30T17:00:35.721256+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.721256+0000","last_undegraded":"2024-05-30T17:00:35.721256+0000","last_fullsized":"2024-05-30T17:00:35.721256+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.710277+0000","last_change":"2024-05-30T17:00:35.710277+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.710277+0000","last_undegraded":"2024-05-30T17:00:35.710277+0000","last_fullsized":"2024-05-30T17:00:35.710277+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.726080+0000","last_change":"2024-05-30T17:00:35.726080+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.726080+0000","last_undegraded":"2024-05-30T17:00:35.726080+0000","last_fullsized":"2024-05-30T17:00:35.726080+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752501+0000","last_change":"2024-05-30T17:00:35.735865+0000","last_active":"2024-05-30T17:00:37.752501+0000","last_peered":"2024-05-30T17:00:37.752501+0000","last_clean":"2024-05-30T17:00:37.752501+0000","last_became_active":"2024-05-30T17:00:35.735526+0000","last_became_peered":"2024-05-30T17:00:35.735526+0000","last_unstale":"2024-05-30T17:00:37.752501+0000","last_undegraded":"2024-05-30T17:00:37.752501+0000","last_fullsized":"2024-05-30T17:00:37.752501+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:42:29.413638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.711876+0000","last_change":"2024-05-30T17:00:35.711876+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.711876+0000","last_undegraded":"2024-05-30T17:00:35.711876+0000","last_fullsized":"2024-05-30T17:00:35.711876+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150045+0000","last_change":"2024-05-30T17:00:35.735197+0000","last_active":"2024-05-30T17:00:38.150045+0000","last_peered":"2024-05-30T17:00:38.150045+0000","last_clean":"2024-05-30T17:00:38.150045+0000","last_became_active":"2024-05-30T17:00:35.734913+0000","last_became_peered":"2024-05-30T17:00:35.734913+0000","last_unstale":"2024-05-30T17:00:38.150045+0000","last_undegraded":"2024-05-30T17:00:38.150045+0000","last_fullsized":"2024-05-30T17:00:38.150045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:58:37.442958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752193+0000","last_change":"2024-05-30T17:00:35.729069+0000","last_active":"2024-05-30T17:00:37.752193+0000","last_peered":"2024-05-30T17:00:37.752193+0000","last_clean":"2024-05-30T17:00:37.752193+0000","last_became_active":"2024-05-30T17:00:35.728765+0000","last_became_peered":"2024-05-30T17:00:35.728765+0000","last_unstale":"2024-05-30T17:00:37.752193+0000","last_undegraded":"2024-05-30T17:00:37.752193+0000","last_fullsized":"2024-05-30T17:00:37.752193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:49:59.611402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761260+0000","last_change":"2024-05-30T17:00:36.746352+0000","last_active":"2024-05-30T17:00:37.761260+0000","last_peered":"2024-05-30T17:00:37.761260+0000","last_clean":"2024-05-30T17:00:37.761260+0000","last_became_active":"2024-05-30T17:00:36.745894+0000","last_became_peered":"2024-05-30T17:00:36.745894+0000","last_unstale":"2024-05-30T17:00:37.761260+0000","last_undegraded":"2024-05-30T17:00:37.761260+0000","last_fullsized":"2024-05-30T17:00:37.761260+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:39:58.290648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752014+0000","last_change":"2024-05-30T17:00:35.734108+0000","last_active":"2024-05-30T17:00:37.752014+0000","last_peered":"2024-05-30T17:00:37.752014+0000","last_clean":"2024-05-30T17:00:37.752014+0000","last_became_active":"2024-05-30T17:00:35.733598+0000","last_became_peered":"2024-05-30T17:00:35.733598+0000","last_unstale":"2024-05-30T17:00:37.752014+0000","last_undegraded":"2024-05-30T17:00:37.752014+0000","last_fullsized":"2024-05-30T17:00:37.752014+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:22.323508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758033+0000","last_change":"2024-05-30T17:00:36.743969+0000","last_active":"2024-05-30T17:00:37.758033+0000","last_peered":"2024-05-30T17:00:37.758033+0000","last_clean":"2024-05-30T17:00:37.758033+0000","last_became_active":"2024-05-30T17:00:36.743574+0000","last_became_peered":"2024-05-30T17:00:36.743574+0000","last_unstale":"2024-05-30T17:00:37.758033+0000","last_undegraded":"2024-05-30T17:00:37.758033+0000","last_fullsized":"2024-05-30T17:00:37.758033+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:57.026949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.699719+0000","last_change":"2024-05-30T17:00:34.699719+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.699719+0000","last_undegraded":"2024-05-30T17:00:34.699719+0000","last_fullsized":"2024-05-30T17:00:34.699719+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149522+0000","last_change":"2024-05-30T17:00:36.755868+0000","last_active":"2024-05-30T17:00:38.149522+0000","last_peered":"2024-05-30T17:00:38.149522+0000","last_clean":"2024-05-30T17:00:38.149522+0000","last_became_active":"2024-05-30T17:00:36.755420+0000","last_became_peered":"2024-05-30T17:00:36.755420+0000","last_unstale":"2024-05-30T17:00:38.149522+0000","last_undegraded":"2024-05-30T17:00:38.149522+0000","last_fullsized":"2024-05-30T17:00:38.149522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751802+0000","last_change":"2024-05-30T17:00:35.733623+0000","last_active":"2024-05-30T17:00:37.751802+0000","last_peered":"2024-05-30T17:00:37.751802+0000","last_clean":"2024-05-30T17:00:37.751802+0000","last_became_active":"2024-05-30T17:00:35.733325+0000","last_became_peered":"2024-05-30T17:00:35.733325+0000","last_unstale":"2024-05-30T17:00:37.751802+0000","last_undegraded":"2024-05-30T17:00:37.751802+0000","last_fullsized":"2024-05-30T17:00:37.751802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:14.320132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.710078+0000","last_change":"2024-05-30T17:00:35.710078+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.710078+0000","last_undegraded":"2024-05-30T17:00:35.710078+0000","last_fullsized":"2024-05-30T17:00:35.710078+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151425+0000","last_change":"2024-05-30T17:00:35.734103+0000","last_active":"2024-05-30T17:00:38.151425+0000","last_peered":"2024-05-30T17:00:38.151425+0000","last_clean":"2024-05-30T17:00:38.151425+0000","last_became_active":"2024-05-30T17:00:35.733719+0000","last_became_peered":"2024-05-30T17:00:35.733719+0000","last_unstale":"2024-05-30T17:00:38.151425+0000","last_undegraded":"2024-05-30T17:00:38.151425+0000","last_fullsized":"2024-05-30T17:00:38.151425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:42.872961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759392+0000","last_change":"2024-05-30T17:00:36.747707+0000","last_active":"2024-05-30T17:00:37.759392+0000","last_peered":"2024-05-30T17:00:37.759392+0000","last_clean":"2024-05-30T17:00:37.759392+0000","last_became_active":"2024-05-30T17:00:36.744282+0000","last_became_peered":"2024-05-30T17:00:36.744282+0000","last_unstale":"2024-05-30T17:00:37.759392+0000","last_undegraded":"2024-05-30T17:00:37.759392+0000","last_fullsized":"2024-05-30T17:00:37.759392+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:48.052699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.703036+0000","last_change":"2024-05-30T17:00:34.703036+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.703036+0000","last_undegraded":"2024-05-30T17:00:34.703036+0000","last_fullsized":"2024-05-30T17:00:34.703036+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752394+0000","last_change":"2024-05-30T17:00:36.742303+0000","last_active":"2024-05-30T17:00:37.752394+0000","last_peered":"2024-05-30T17:00:37.752394+0000","last_clean":"2024-05-30T17:00:37.752394+0000","last_became_active":"2024-05-30T17:00:36.741809+0000","last_became_peered":"2024-05-30T17:00:36.741809+0000","last_unstale":"2024-05-30T17:00:37.752394+0000","last_undegraded":"2024-05-30T17:00:37.752394+0000","last_fullsized":"2024-05-30T17:00:37.752394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.699191+0000","last_change":"2024-05-30T17:00:34.699191+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.699191+0000","last_undegraded":"2024-05-30T17:00:34.699191+0000","last_fullsized":"2024-05-30T17:00:34.699191+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760526+0000","last_change":"2024-05-30T17:00:36.747039+0000","last_active":"2024-05-30T17:00:37.760526+0000","last_peered":"2024-05-30T17:00:37.760526+0000","last_clean":"2024-05-30T17:00:37.760526+0000","last_became_active":"2024-05-30T17:00:36.745915+0000","last_became_peered":"2024-05-30T17:00:36.745915+0000","last_unstale":"2024-05-30T17:00:37.760526+0000","last_undegraded":"2024-05-30T17:00:37.760526+0000","last_fullsized":"2024-05-30T17:00:37.760526+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:21.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.723219+0000","last_change":"2024-05-30T17:00:35.718080+0000","last_active":"2024-05-30T17:00:35.723219+0000","last_peered":"2024-05-30T17:00:35.723219+0000","last_clean":"2024-05-30T17:00:35.723219+0000","last_became_active":"2024-05-30T17:00:35.717705+0000","last_became_peered":"2024-05-30T17:00:35.717705+0000","last_unstale":"2024-05-30T17:00:35.723219+0000","last_undegraded":"2024-05-30T17:00:35.723219+0000","last_fullsized":"2024-05-30T17:00:35.723219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:39:57.277812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152539+0000","last_change":"2024-05-30T17:00:36.754484+0000","last_active":"2024-05-30T17:00:38.152539+0000","last_peered":"2024-05-30T17:00:38.152539+0000","last_clean":"2024-05-30T17:00:38.152539+0000","last_became_active":"2024-05-30T17:00:36.754166+0000","last_became_peered":"2024-05-30T17:00:36.754166+0000","last_unstale":"2024-05-30T17:00:38.152539+0000","last_undegraded":"2024-05-30T17:00:38.152539+0000","last_fullsized":"2024-05-30T17:00:38.152539+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:33.091804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150419+0000","last_change":"2024-05-30T17:00:35.732982+0000","last_active":"2024-05-30T17:00:38.150419+0000","last_peered":"2024-05-30T17:00:38.150419+0000","last_clean":"2024-05-30T17:00:38.150419+0000","last_became_active":"2024-05-30T17:00:35.732114+0000","last_became_peered":"2024-05-30T17:00:35.732114+0000","last_unstale":"2024-05-30T17:00:38.150419+0000","last_undegraded":"2024-05-30T17:00:38.150419+0000","last_fullsized":"2024-05-30T17:00:38.150419+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:33.694752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150348+0000","last_change":"2024-05-30T17:00:36.758949+0000","last_active":"2024-05-30T17:00:38.150348+0000","last_peered":"2024-05-30T17:00:38.150348+0000","last_clean":"2024-05-30T17:00:38.150348+0000","last_became_active":"2024-05-30T17:00:36.758069+0000","last_became_peered":"2024-05-30T17:00:36.758069+0000","last_unstale":"2024-05-30T17:00:38.150348+0000","last_undegraded":"2024-05-30T17:00:38.150348+0000","last_fullsized":"2024-05-30T17:00:38.150348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:38:49.283720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760740+0000","last_change":"2024-05-30T17:00:35.731848+0000","last_active":"2024-05-30T17:00:37.760740+0000","last_peered":"2024-05-30T17:00:37.760740+0000","last_clean":"2024-05-30T17:00:37.760740+0000","last_became_active":"2024-05-30T17:00:35.731536+0000","last_became_peered":"2024-05-30T17:00:35.731536+0000","last_unstale":"2024-05-30T17:00:37.760740+0000","last_undegraded":"2024-05-30T17:00:37.760740+0000","last_fullsized":"2024-05-30T17:00:37.760740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:12:27.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.708596+0000","last_change":"2024-05-30T17:00:35.708596+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.708596+0000","last_undegraded":"2024-05-30T17:00:35.708596+0000","last_fullsized":"2024-05-30T17:00:35.708596+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.737819+0000","last_change":"2024-05-30T17:00:35.734783+0000","last_active":"2024-05-30T17:00:35.737819+0000","last_peered":"2024-05-30T17:00:35.737819+0000","last_clean":"2024-05-30T17:00:35.737819+0000","last_became_active":"2024-05-30T17:00:35.733971+0000","last_became_peered":"2024-05-30T17:00:35.733971+0000","last_unstale":"2024-05-30T17:00:35.737819+0000","last_undegraded":"2024-05-30T17:00:35.737819+0000","last_fullsized":"2024-05-30T17:00:35.737819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:44.116430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.739677+0000","last_change":"2024-05-30T17:00:35.738764+0000","last_active":"2024-05-30T17:00:35.739677+0000","last_peered":"2024-05-30T17:00:35.739677+0000","last_clean":"2024-05-30T17:00:35.739677+0000","last_became_active":"2024-05-30T17:00:35.737679+0000","last_became_peered":"2024-05-30T17:00:35.737679+0000","last_unstale":"2024-05-30T17:00:35.739677+0000","last_undegraded":"2024-05-30T17:00:35.739677+0000","last_fullsized":"2024-05-30T17:00:35.739677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:13:25.537216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760261+0000","last_change":"2024-05-30T17:00:36.744702+0000","last_active":"2024-05-30T17:00:37.760261+0000","last_peered":"2024-05-30T17:00:37.760261+0000","last_clean":"2024-05-30T17:00:37.760261+0000","last_became_active":"2024-05-30T17:00:36.744329+0000","last_became_peered":"2024-05-30T17:00:36.744329+0000","last_unstale":"2024-05-30T17:00:37.760261+0000","last_undegraded":"2024-05-30T17:00:37.760261+0000","last_fullsized":"2024-05-30T17:00:37.760261+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:21:27.452364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759287+0000","last_change":"2024-05-30T17:00:35.729767+0000","last_active":"2024-05-30T17:00:37.759287+0000","last_peered":"2024-05-30T17:00:37.759287+0000","last_clean":"2024-05-30T17:00:37.759287+0000","last_became_active":"2024-05-30T17:00:35.729412+0000","last_became_peered":"2024-05-30T17:00:35.729412+0000","last_unstale":"2024-05-30T17:00:37.759287+0000","last_undegraded":"2024-05-30T17:00:37.759287+0000","last_fullsized":"2024-05-30T17:00:37.759287+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:47.793776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.716683+0000","last_change":"2024-05-30T17:00:35.716683+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.716683+0000","last_undegraded":"2024-05-30T17:00:35.716683+0000","last_fullsized":"2024-05-30T17:00:35.716683+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.699431+0000","last_change":"2024-05-30T17:00:34.699431+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.699431+0000","last_undegraded":"2024-05-30T17:00:34.699431+0000","last_fullsized":"2024-05-30T17:00:34.699431+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152237+0000","last_change":"2024-05-30T17:00:36.751676+0000","last_active":"2024-05-30T17:00:38.152237+0000","last_peered":"2024-05-30T17:00:38.152237+0000","last_clean":"2024-05-30T17:00:38.152237+0000","last_became_active":"2024-05-30T17:00:36.751355+0000","last_became_peered":"2024-05-30T17:00:36.751355+0000","last_unstale":"2024-05-30T17:00:38.152237+0000","last_undegraded":"2024-05-30T17:00:38.152237+0000","last_fullsized":"2024-05-30T17:00:38.152237+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:01:43.178806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760541+0000","last_change":"2024-05-30T17:00:35.734613+0000","last_active":"2024-05-30T17:00:37.760541+0000","last_peered":"2024-05-30T17:00:37.760541+0000","last_clean":"2024-05-30T17:00:37.760541+0000","last_became_active":"2024-05-30T17:00:35.734231+0000","last_became_peered":"2024-05-30T17:00:35.734231+0000","last_unstale":"2024-05-30T17:00:37.760541+0000","last_undegraded":"2024-05-30T17:00:37.760541+0000","last_fullsized":"2024-05-30T17:00:37.760541+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:06.120742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758055+0000","last_change":"2024-05-30T17:00:36.757383+0000","last_active":"2024-05-30T17:00:37.758055+0000","last_peered":"2024-05-30T17:00:37.758055+0000","last_clean":"2024-05-30T17:00:37.758055+0000","last_became_active":"2024-05-30T17:00:36.756264+0000","last_became_peered":"2024-05-30T17:00:36.756264+0000","last_unstale":"2024-05-30T17:00:37.758055+0000","last_undegraded":"2024-05-30T17:00:37.758055+0000","last_fullsized":"2024-05-30T17:00:37.758055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:32.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151230+0000","last_change":"2024-05-30T17:00:35.730141+0000","last_active":"2024-05-30T17:00:38.151230+0000","last_peered":"2024-05-30T17:00:38.151230+0000","last_clean":"2024-05-30T17:00:38.151230+0000","last_became_active":"2024-05-30T17:00:35.728948+0000","last_became_peered":"2024-05-30T17:00:35.728948+0000","last_unstale":"2024-05-30T17:00:38.151230+0000","last_undegraded":"2024-05-30T17:00:38.151230+0000","last_fullsized":"2024-05-30T17:00:38.151230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:13.200467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150264+0000","last_change":"2024-05-30T17:00:36.750999+0000","last_active":"2024-05-30T17:00:38.150264+0000","last_peered":"2024-05-30T17:00:38.150264+0000","last_clean":"2024-05-30T17:00:38.150264+0000","last_became_active":"2024-05-30T17:00:36.750499+0000","last_became_peered":"2024-05-30T17:00:36.750499+0000","last_unstale":"2024-05-30T17:00:38.150264+0000","last_undegraded":"2024-05-30T17:00:38.150264+0000","last_fullsized":"2024-05-30T17:00:38.150264+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:10.934086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751996+0000","last_change":"2024-05-30T17:00:35.733883+0000","last_active":"2024-05-30T17:00:37.751996+0000","last_peered":"2024-05-30T17:00:37.751996+0000","last_clean":"2024-05-30T17:00:37.751996+0000","last_became_active":"2024-05-30T17:00:35.733243+0000","last_became_peered":"2024-05-30T17:00:35.733243+0000","last_unstale":"2024-05-30T17:00:37.751996+0000","last_undegraded":"2024-05-30T17:00:37.751996+0000","last_fullsized":"2024-05-30T17:00:37.751996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:46:39.811373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720831+0000","last_change":"2024-05-30T17:00:35.720831+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720831+0000","last_undegraded":"2024-05-30T17:00:35.720831+0000","last_fullsized":"2024-05-30T17:00:35.720831+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752192+0000","last_change":"2024-05-30T17:00:35.729083+0000","last_active":"2024-05-30T17:00:37.752192+0000","last_peered":"2024-05-30T17:00:37.752192+0000","last_clean":"2024-05-30T17:00:37.752192+0000","last_became_active":"2024-05-30T17:00:35.728830+0000","last_became_peered":"2024-05-30T17:00:35.728830+0000","last_unstale":"2024-05-30T17:00:37.752192+0000","last_undegraded":"2024-05-30T17:00:37.752192+0000","last_fullsized":"2024-05-30T17:00:37.752192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:52.170762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.719960+0000","last_change":"2024-05-30T17:00:35.719960+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.719960+0000","last_undegraded":"2024-05-30T17:00:35.719960+0000","last_fullsized":"2024-05-30T17:00:35.719960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757948+0000","last_change":"2024-05-30T17:00:35.730608+0000","last_active":"2024-05-30T17:00:37.757948+0000","last_peered":"2024-05-30T17:00:37.757948+0000","last_clean":"2024-05-30T17:00:37.757948+0000","last_became_active":"2024-05-30T17:00:35.730310+0000","last_became_peered":"2024-05-30T17:00:35.730310+0000","last_unstale":"2024-05-30T17:00:37.757948+0000","last_undegraded":"2024-05-30T17:00:37.757948+0000","last_fullsized":"2024-05-30T17:00:37.757948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:13.646048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.717200+0000","last_change":"2024-05-30T17:00:35.717200+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.717200+0000","last_undegraded":"2024-05-30T17:00:35.717200+0000","last_fullsized":"2024-05-30T17:00:35.717200+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757483+0000","last_change":"2024-05-30T17:00:35.709483+0000","last_active":"2024-05-30T17:00:37.757483+0000","last_peered":"2024-05-30T17:00:37.757483+0000","last_clean":"2024-05-30T17:00:37.757483+0000","last_became_active":"2024-05-30T17:00:35.708526+0000","last_became_peered":"2024-05-30T17:00:35.708526+0000","last_unstale":"2024-05-30T17:00:37.757483+0000","last_undegraded":"2024-05-30T17:00:37.757483+0000","last_fullsized":"2024-05-30T17:00:37.757483+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:40.251625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151273+0000","last_change":"2024-05-30T17:00:36.749874+0000","last_active":"2024-05-30T17:00:38.151273+0000","last_peered":"2024-05-30T17:00:38.151273+0000","last_clean":"2024-05-30T17:00:38.151273+0000","last_became_active":"2024-05-30T17:00:36.749228+0000","last_became_peered":"2024-05-30T17:00:36.749228+0000","last_unstale":"2024-05-30T17:00:38.151273+0000","last_undegraded":"2024-05-30T17:00:38.151273+0000","last_fullsized":"2024-05-30T17:00:38.151273+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:44.276755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751571+0000","last_change":"2024-05-30T17:00:35.736336+0000","last_active":"2024-05-30T17:00:37.751571+0000","last_peered":"2024-05-30T17:00:37.751571+0000","last_clean":"2024-05-30T17:00:37.751571+0000","last_became_active":"2024-05-30T17:00:35.734451+0000","last_became_peered":"2024-05-30T17:00:35.734451+0000","last_unstale":"2024-05-30T17:00:37.751571+0000","last_undegraded":"2024-05-30T17:00:37.751571+0000","last_fullsized":"2024-05-30T17:00:37.751571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:16.292089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752599+0000","last_change":"2024-05-30T17:00:36.748422+0000","last_active":"2024-05-30T17:00:37.752599+0000","last_peered":"2024-05-30T17:00:37.752599+0000","last_clean":"2024-05-30T17:00:37.752599+0000","last_became_active":"2024-05-30T17:00:36.747900+0000","last_became_peered":"2024-05-30T17:00:36.747900+0000","last_unstale":"2024-05-30T17:00:37.752599+0000","last_undegraded":"2024-05-30T17:00:37.752599+0000","last_fullsized":"2024-05-30T17:00:37.752599+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758807+0000","last_change":"2024-05-30T17:00:35.729597+0000","last_active":"2024-05-30T17:00:37.758807+0000","last_peered":"2024-05-30T17:00:37.758807+0000","last_clean":"2024-05-30T17:00:37.758807+0000","last_became_active":"2024-05-30T17:00:35.728519+0000","last_became_peered":"2024-05-30T17:00:35.728519+0000","last_unstale":"2024-05-30T17:00:37.758807+0000","last_undegraded":"2024-05-30T17:00:37.758807+0000","last_fullsized":"2024-05-30T17:00:37.758807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:12.232879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.754450+0000","last_change":"2024-05-30T17:00:35.733609+0000","last_active":"2024-05-30T17:00:37.754450+0000","last_peered":"2024-05-30T17:00:37.754450+0000","last_clean":"2024-05-30T17:00:37.754450+0000","last_became_active":"2024-05-30T17:00:35.733292+0000","last_became_peered":"2024-05-30T17:00:35.733292+0000","last_unstale":"2024-05-30T17:00:37.754450+0000","last_undegraded":"2024-05-30T17:00:37.754450+0000","last_fullsized":"2024-05-30T17:00:37.754450+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:00.979924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752339+0000","last_change":"2024-05-30T17:00:35.734441+0000","last_active":"2024-05-30T17:00:37.752339+0000","last_peered":"2024-05-30T17:00:37.752339+0000","last_clean":"2024-05-30T17:00:37.752339+0000","last_became_active":"2024-05-30T17:00:35.732710+0000","last_became_peered":"2024-05-30T17:00:35.732710+0000","last_unstale":"2024-05-30T17:00:37.752339+0000","last_undegraded":"2024-05-30T17:00:37.752339+0000","last_fullsized":"2024-05-30T17:00:37.752339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:15.063268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151374+0000","last_change":"2024-05-30T17:00:36.764319+0000","last_active":"2024-05-30T17:00:38.151374+0000","last_peered":"2024-05-30T17:00:38.151374+0000","last_clean":"2024-05-30T17:00:38.151374+0000","last_became_active":"2024-05-30T17:00:36.764034+0000","last_became_peered":"2024-05-30T17:00:36.764034+0000","last_unstale":"2024-05-30T17:00:38.151374+0000","last_undegraded":"2024-05-30T17:00:38.151374+0000","last_fullsized":"2024-05-30T17:00:38.151374+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:28.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152390+0000","last_change":"2024-05-30T17:00:35.727312+0000","last_active":"2024-05-30T17:00:38.152390+0000","last_peered":"2024-05-30T17:00:38.152390+0000","last_clean":"2024-05-30T17:00:38.152390+0000","last_became_active":"2024-05-30T17:00:35.726820+0000","last_became_peered":"2024-05-30T17:00:35.726820+0000","last_unstale":"2024-05-30T17:00:38.152390+0000","last_undegraded":"2024-05-30T17:00:38.152390+0000","last_fullsized":"2024-05-30T17:00:38.152390+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:40.105333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720407+0000","last_change":"2024-05-30T17:00:35.720407+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720407+0000","last_undegraded":"2024-05-30T17:00:35.720407+0000","last_fullsized":"2024-05-30T17:00:35.720407+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752300+0000","last_change":"2024-05-30T17:00:35.727143+0000","last_active":"2024-05-30T17:00:37.752300+0000","last_peered":"2024-05-30T17:00:37.752300+0000","last_clean":"2024-05-30T17:00:37.752300+0000","last_became_active":"2024-05-30T17:00:35.725458+0000","last_became_peered":"2024-05-30T17:00:35.725458+0000","last_unstale":"2024-05-30T17:00:37.752300+0000","last_undegraded":"2024-05-30T17:00:37.752300+0000","last_fullsized":"2024-05-30T17:00:37.752300+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:49.887792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760276+0000","last_change":"2024-05-30T17:00:36.756498+0000","last_active":"2024-05-30T17:00:37.760276+0000","last_peered":"2024-05-30T17:00:37.760276+0000","last_clean":"2024-05-30T17:00:37.760276+0000","last_became_active":"2024-05-30T17:00:36.755460+0000","last_became_peered":"2024-05-30T17:00:36.755460+0000","last_unstale":"2024-05-30T17:00:37.760276+0000","last_undegraded":"2024-05-30T17:00:37.760276+0000","last_fullsized":"2024-05-30T17:00:37.760276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:01.424865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150911+0000","last_change":"2024-05-30T17:00:35.731535+0000","last_active":"2024-05-30T17:00:38.150911+0000","last_peered":"2024-05-30T17:00:38.150911+0000","last_clean":"2024-05-30T17:00:38.150911+0000","last_became_active":"2024-05-30T17:00:35.730893+0000","last_became_peered":"2024-05-30T17:00:35.730893+0000","last_unstale":"2024-05-30T17:00:38.150911+0000","last_undegraded":"2024-05-30T17:00:38.150911+0000","last_fullsized":"2024-05-30T17:00:38.150911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:45.698539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.716687+0000","last_change":"2024-05-30T17:00:35.716687+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.716687+0000","last_undegraded":"2024-05-30T17:00:35.716687+0000","last_fullsized":"2024-05-30T17:00:35.716687+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.727740+0000","last_change":"2024-05-30T17:00:35.724537+0000","last_active":"2024-05-30T17:00:35.727740+0000","last_peered":"2024-05-30T17:00:35.727740+0000","last_clean":"2024-05-30T17:00:35.727740+0000","last_became_active":"2024-05-30T17:00:35.724245+0000","last_became_peered":"2024-05-30T17:00:35.724245+0000","last_unstale":"2024-05-30T17:00:35.727740+0000","last_undegraded":"2024-05-30T17:00:35.727740+0000","last_fullsized":"2024-05-30T17:00:35.727740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:07:13.036207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151226+0000","last_change":"2024-05-30T17:00:36.761666+0000","last_active":"2024-05-30T17:00:38.151226+0000","last_peered":"2024-05-30T17:00:38.151226+0000","last_clean":"2024-05-30T17:00:38.151226+0000","last_became_active":"2024-05-30T17:00:36.760872+0000","last_became_peered":"2024-05-30T17:00:36.760872+0000","last_unstale":"2024-05-30T17:00:38.151226+0000","last_undegraded":"2024-05-30T17:00:38.151226+0000","last_fullsized":"2024-05-30T17:00:38.151226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:12:08.423568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.736077+0000","last_change":"2024-05-30T17:00:35.732840+0000","last_active":"2024-05-30T17:00:35.736077+0000","last_peered":"2024-05-30T17:00:35.736077+0000","last_clean":"2024-05-30T17:00:35.736077+0000","last_became_active":"2024-05-30T17:00:35.732525+0000","last_became_peered":"2024-05-30T17:00:35.732525+0000","last_unstale":"2024-05-30T17:00:35.736077+0000","last_undegraded":"2024-05-30T17:00:35.736077+0000","last_fullsized":"2024-05-30T17:00:35.736077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:29.507352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720954+0000","last_change":"2024-05-30T17:00:35.720954+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720954+0000","last_undegraded":"2024-05-30T17:00:35.720954+0000","last_fullsized":"2024-05-30T17:00:35.720954+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149819+0000","last_change":"2024-05-30T17:00:35.729629+0000","last_active":"2024-05-30T17:00:38.149819+0000","last_peered":"2024-05-30T17:00:38.149819+0000","last_clean":"2024-05-30T17:00:38.149819+0000","last_became_active":"2024-05-30T17:00:35.729282+0000","last_became_peered":"2024-05-30T17:00:35.729282+0000","last_unstale":"2024-05-30T17:00:38.149819+0000","last_undegraded":"2024-05-30T17:00:38.149819+0000","last_fullsized":"2024-05-30T17:00:38.149819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:43.450197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":330,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148934+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:00:38.148934+0000","last_peered":"2024-05-30T17:00:38.148934+0000","last_clean":"2024-05-30T17:00:38.148934+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:00:38.148934+0000","last_undegraded":"2024-05-30T17:00:38.148934+0000","last_fullsized":"2024-05-30T17:00:38.148934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509647,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130936,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240904,"statfs":{"total":96636764160,"available":96502685696,"internally_reserved":0,"allocated":1286144,"data_stored":558361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509647,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86528,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285312,"statfs":{"total":96636764160,"available":96548159488,"internally_reserved":0,"allocated":1298432,"data_stored":559445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509647,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126728,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245112,"statfs":{"total":96636764160,"available":96506994688,"internally_reserved":0,"allocated":1241088,"data_stored":538107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509647,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144792,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227048,"statfs":{"total":96636764160,"available":96488497152,"internally_reserved":0,"allocated":1253376,"data_stored":544466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509646,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126732,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245108,"statfs":{"total":96636764160,"available":96506990592,"internally_reserved":0,"allocated":1241088,"data_stored":538107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542350,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127356,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244484,"statfs":{"total":96636764160,"available":96506351616,"internally_reserved":0,"allocated":1880064,"data_stored":1148189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76788,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295052,"statfs":{"total":96636764160,"available":96558133248,"internally_reserved":0,"allocated":1282048,"data_stored":557779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542351,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69044,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302796,"statfs":{"total":96636764160,"available":96566063104,"internally_reserved":0,"allocated":1875968,"data_stored":1148659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:00:41.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:00:41.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.467+0000 7f933e600640 1 Processor -- start 2024-05-30T17:00:41.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.468+0000 7f933e600640 1 -- start start 2024-05-30T17:00:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.469+0000 7f933e600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f93380d04a0 0x7f93380d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.469+0000 7f933e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f93380ad130 0x7f93380ad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.469+0000 7f933e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f933806ed10 con 0x7f933806f860 2024-05-30T17:00:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.469+0000 7f933e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f933815cb00 con 0x7f93380ad130 2024-05-30T17:00:41.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.469+0000 7f933e600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f93380b3cc0 con 0x7f93380d04a0 2024-05-30T17:00:41.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.470+0000 7f9337400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f933806f860 0x7f93380b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56694/0 (socket says 172.21.15.67:56694) 2024-05-30T17:00:41.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.470+0000 7f9337400640 1 -- 172.21.15.67:0/1407167422 learned_addr learned my addr 172.21.15.67:0/1407167422 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:41.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.470+0000 7f9337e00640 1 --2- 172.21.15.67:0/1407167422 >> v2:172.21.15.67:3300/0 conn(0x7f93380ad130 0x7f93380ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:41.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.470+0000 7f9336a00640 1 --2- 172.21.15.67:0/1407167422 >> v2:172.21.15.169:3300/0 conn(0x7f93380d04a0 0x7f93380d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:41.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.470+0000 7f9337e00640 1 -- 172.21.15.67:0/1407167422 >> v2:172.21.15.169:3300/0 conn(0x7f93380d04a0 msgr2=0x7f93380d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:41.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.470+0000 7f9337e00640 1 --2- 172.21.15.67:0/1407167422 >> v2:172.21.15.169:3300/0 conn(0x7f93380d04a0 0x7f93380d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.470+0000 7f9337e00640 1 -- 172.21.15.67:0/1407167422 >> v1:172.21.15.67:6789/0 conn(0x7f933806f860 legacy=0x7f93380b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.470+0000 7f9337e00640 1 -- 172.21.15.67:0/1407167422 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93380b3fa0 con 0x7f93380ad130 2024-05-30T17:00:41.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.471+0000 7f9337e00640 1 --2- 172.21.15.67:0/1407167422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380ad130 0x7f93380ad540 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9338070120 tx=0x7f933002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf9af87daffb3a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:41.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.471+0000 7f9336000640 1 -- 172.21.15.67:0/1407167422 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330037070 con 0x7f93380ad130 2024-05-30T17:00:41.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.471+0000 7f9336000640 1 -- 172.21.15.67:0/1407167422 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9330032440 con 0x7f93380ad130 2024-05-30T17:00:41.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.471+0000 7f9336000640 1 -- 172.21.15.67:0/1407167422 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330032810 con 0x7f93380ad130 2024-05-30T17:00:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.472+0000 7f933e600640 1 -- 172.21.15.67:0/1407167422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380ad130 msgr2=0x7f93380ad540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.472+0000 7f933e600640 1 --2- 172.21.15.67:0/1407167422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380ad130 0x7f93380ad540 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9338070120 tx=0x7f933002a1a0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.472+0000 7f933e600640 1 -- 172.21.15.67:0/1407167422 shutdown_connections 2024-05-30T17:00:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.472+0000 7f933e600640 1 --2- 172.21.15.67:0/1407167422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380ad130 0x7f93380ad540 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.472+0000 7f933e600640 1 --2- 172.21.15.67:0/1407167422 >> v2:172.21.15.169:3300/0 conn(0x7f93380d04a0 0x7f93380d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.472+0000 7f933e600640 1 -- 172.21.15.67:0/1407167422 >> 172.21.15.67:0/1407167422 conn(0x7f933806bae0 msgr2=0x7f933806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.472+0000 7f933e600640 1 -- 172.21.15.67:0/1407167422 shutdown_connections 2024-05-30T17:00:41.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.472+0000 7f933e600640 1 -- 172.21.15.67:0/1407167422 wait complete. 2024-05-30T17:00:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.474+0000 7f933e600640 1 Processor -- start 2024-05-30T17:00:41.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.474+0000 7f933e600640 1 -- start start 2024-05-30T17:00:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.475+0000 7f933e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933806f860 0x7f93381f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.475+0000 7f933e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93380ad130 0x7f93380c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.475+0000 7f933e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380d04a0 0x7f93380c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.475+0000 7f933e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f93380b4840 con 0x7f93380d04a0 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.475+0000 7f933e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93380b4560 con 0x7f933806f860 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.475+0000 7f933e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f93380b3860 con 0x7f93380ad130 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.475+0000 7f9337e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380d04a0 0x7f93380c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.475+0000 7f9337e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380d04a0 0x7f93380c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53134/0 (socket says 172.21.15.67:53134) 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.475+0000 7f9337e00640 1 -- 172.21.15.67:0/1512236032 learned_addr learned my addr 172.21.15.67:0/1512236032 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.476+0000 7f9337e00640 1 -- 172.21.15.67:0/1512236032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93380ad130 msgr2=0x7f93380c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.476+0000 7f9337e00640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93380ad130 0x7f93380c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.476+0000 7f9337e00640 1 -- 172.21.15.67:0/1512236032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933806f860 msgr2=0x7f93381f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.476+0000 7f9337e00640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933806f860 0x7f93381f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.476+0000 7f9337e00640 1 -- 172.21.15.67:0/1512236032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93380c97f0 con 0x7f93380d04a0 2024-05-30T17:00:41.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.476+0000 7f9337e00640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380d04a0 0x7f93380c8f40 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9330035040 tx=0x7f93300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:41.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.476+0000 7f932f400640 1 -- 172.21.15.67:0/1512236032 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330037040 con 0x7f93380d04a0 2024-05-30T17:00:41.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.477+0000 7f932f400640 1 -- 172.21.15.67:0/1512236032 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9330042070 con 0x7f93380d04a0 2024-05-30T17:00:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.477+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93380c9ad0 con 0x7f93380d04a0 2024-05-30T17:00:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.477+0000 7f932f400640 1 -- 172.21.15.67:0/1512236032 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330030b30 con 0x7f93380d04a0 2024-05-30T17:00:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.477+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93380ca090 con 0x7f93380d04a0 2024-05-30T17:00:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.478+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f933815ceb0 con 0x7f93380d04a0 2024-05-30T17:00:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.479+0000 7f932f400640 1 -- 172.21.15.67:0/1512236032 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f9330030cd0 con 0x7f93380d04a0 2024-05-30T17:00:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.480+0000 7f932f400640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f931806ddc0 0x7f9318070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.481+0000 7f932f400640 1 -- 172.21.15.67:0/1512236032 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f93300ab7c0 con 0x7f93380d04a0 2024-05-30T17:00:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.481+0000 7f9337400640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f931806ddc0 0x7f9318070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.481+0000 7f9337400640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f931806ddc0 0x7f9318070280 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f93280012e0 tx=0x7f9328001630 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.485+0000 7f932f400640 1 -- 172.21.15.67:0/1512236032 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9330032e20 con 0x7f93380d04a0 2024-05-30T17:00:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.676+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93380689e0 con 0x7f931806ddc0 2024-05-30T17:00:41.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.688+0000 7f932f400640 1 -- 172.21.15.67:0/1512236032 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249442 (secure 0 0 0) 0x7f93380689e0 con 0x7f931806ddc0 2024-05-30T17:00:41.691 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:41.693 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:00:41.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f931806ddc0 msgr2=0x7f9318070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f931806ddc0 0x7f9318070280 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f93280012e0 tx=0x7f9328001630 comp rx=0 tx=0).stop 2024-05-30T17:00:41.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380d04a0 msgr2=0x7f93380c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380d04a0 0x7f93380c8f40 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9330035040 tx=0x7f93300046c0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 shutdown_connections 2024-05-30T17:00:41.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f931806ddc0 0x7f9318070280 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93380d04a0 0x7f93380c8f40 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93380ad130 0x7f93380c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 --2- 172.21.15.67:0/1512236032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933806f860 0x7f93381f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 >> 172.21.15.67:0/1512236032 conn(0x7f933806bae0 msgr2=0x7f93380cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:41.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 shutdown_connections 2024-05-30T17:00:41.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.693+0000 7f933e600640 1 -- 172.21.15.67:0/1512236032 wait complete. 2024-05-30T17:00:41.713 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":608,"stamp":"2024-05-30T17:00:40.472092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":888904,"kb_used_data":11092,"kb_used_omap":12,"kb_used_meta":877683,"kb_avail":754085816,"statfs":{"total":773094113280,"available":772183875584,"internally_reserved":0,"allocated":11358208,"data_stored":5593113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":898747952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.782789"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752224+0000","last_change":"2024-05-30T17:00:36.748181+0000","last_active":"2024-05-30T17:00:37.752224+0000","last_peered":"2024-05-30T17:00:37.752224+0000","last_clean":"2024-05-30T17:00:37.752224+0000","last_became_active":"2024-05-30T17:00:36.747392+0000","last_became_peered":"2024-05-30T17:00:36.747392+0000","last_unstale":"2024-05-30T17:00:37.752224+0000","last_undegraded":"2024-05-30T17:00:37.752224+0000","last_fullsized":"2024-05-30T17:00:37.752224+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.712512+0000","last_change":"2024-05-30T17:00:35.712512+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.712512+0000","last_undegraded":"2024-05-30T17:00:35.712512+0000","last_fullsized":"2024-05-30T17:00:35.712512+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151835+0000","last_change":"2024-05-30T17:00:36.763111+0000","last_active":"2024-05-30T17:00:38.151835+0000","last_peered":"2024-05-30T17:00:38.151835+0000","last_clean":"2024-05-30T17:00:38.151835+0000","last_became_active":"2024-05-30T17:00:36.761404+0000","last_became_peered":"2024-05-30T17:00:36.761404+0000","last_unstale":"2024-05-30T17:00:38.151835+0000","last_undegraded":"2024-05-30T17:00:38.151835+0000","last_fullsized":"2024-05-30T17:00:38.151835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:01.455423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149546+0000","last_change":"2024-05-30T17:00:36.751908+0000","last_active":"2024-05-30T17:00:38.149546+0000","last_peered":"2024-05-30T17:00:38.149546+0000","last_clean":"2024-05-30T17:00:38.149546+0000","last_became_active":"2024-05-30T17:00:36.751604+0000","last_became_peered":"2024-05-30T17:00:36.751604+0000","last_unstale":"2024-05-30T17:00:38.149546+0000","last_undegraded":"2024-05-30T17:00:38.149546+0000","last_fullsized":"2024-05-30T17:00:38.149546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149447+0000","last_change":"2024-05-30T17:00:36.748294+0000","last_active":"2024-05-30T17:00:38.149447+0000","last_peered":"2024-05-30T17:00:38.149447+0000","last_clean":"2024-05-30T17:00:38.149447+0000","last_became_active":"2024-05-30T17:00:36.747984+0000","last_became_peered":"2024-05-30T17:00:36.747984+0000","last_unstale":"2024-05-30T17:00:38.149447+0000","last_undegraded":"2024-05-30T17:00:38.149447+0000","last_fullsized":"2024-05-30T17:00:38.149447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751747+0000","last_change":"2024-05-30T17:00:36.751171+0000","last_active":"2024-05-30T17:00:37.751747+0000","last_peered":"2024-05-30T17:00:37.751747+0000","last_clean":"2024-05-30T17:00:37.751747+0000","last_became_active":"2024-05-30T17:00:36.747956+0000","last_became_peered":"2024-05-30T17:00:36.747956+0000","last_unstale":"2024-05-30T17:00:37.751747+0000","last_undegraded":"2024-05-30T17:00:37.751747+0000","last_fullsized":"2024-05-30T17:00:37.751747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:03:57.450595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760121+0000","last_change":"2024-05-30T17:00:36.762682+0000","last_active":"2024-05-30T17:00:37.760121+0000","last_peered":"2024-05-30T17:00:37.760121+0000","last_clean":"2024-05-30T17:00:37.760121+0000","last_became_active":"2024-05-30T17:00:36.762190+0000","last_became_peered":"2024-05-30T17:00:36.762190+0000","last_unstale":"2024-05-30T17:00:37.760121+0000","last_undegraded":"2024-05-30T17:00:37.760121+0000","last_fullsized":"2024-05-30T17:00:37.760121+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:33.645864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151899+0000","last_change":"2024-05-30T17:00:36.758850+0000","last_active":"2024-05-30T17:00:38.151899+0000","last_peered":"2024-05-30T17:00:38.151899+0000","last_clean":"2024-05-30T17:00:38.151899+0000","last_became_active":"2024-05-30T17:00:36.758522+0000","last_became_peered":"2024-05-30T17:00:36.758522+0000","last_unstale":"2024-05-30T17:00:38.151899+0000","last_undegraded":"2024-05-30T17:00:38.151899+0000","last_fullsized":"2024-05-30T17:00:38.151899+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:20.247961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149815+0000","last_change":"2024-05-30T17:00:36.750772+0000","last_active":"2024-05-30T17:00:38.149815+0000","last_peered":"2024-05-30T17:00:38.149815+0000","last_clean":"2024-05-30T17:00:38.149815+0000","last_became_active":"2024-05-30T17:00:36.750122+0000","last_became_peered":"2024-05-30T17:00:36.750122+0000","last_unstale":"2024-05-30T17:00:38.149815+0000","last_undegraded":"2024-05-30T17:00:38.149815+0000","last_fullsized":"2024-05-30T17:00:38.149815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761029+0000","last_change":"2024-05-30T17:00:36.751475+0000","last_active":"2024-05-30T17:00:37.761029+0000","last_peered":"2024-05-30T17:00:37.761029+0000","last_clean":"2024-05-30T17:00:37.761029+0000","last_became_active":"2024-05-30T17:00:36.751150+0000","last_became_peered":"2024-05-30T17:00:36.751150+0000","last_unstale":"2024-05-30T17:00:37.761029+0000","last_undegraded":"2024-05-30T17:00:37.761029+0000","last_fullsized":"2024-05-30T17:00:37.761029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:55:05.659729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.712050+0000","last_change":"2024-05-30T17:00:35.712050+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.712050+0000","last_undegraded":"2024-05-30T17:00:35.712050+0000","last_fullsized":"2024-05-30T17:00:35.712050+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.721668+0000","last_change":"2024-05-30T17:00:35.721668+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.721668+0000","last_undegraded":"2024-05-30T17:00:35.721668+0000","last_fullsized":"2024-05-30T17:00:35.721668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.711828+0000","last_change":"2024-05-30T17:00:35.711828+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.711828+0000","last_undegraded":"2024-05-30T17:00:35.711828+0000","last_fullsized":"2024-05-30T17:00:35.711828+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149705+0000","last_change":"2024-05-30T17:00:36.758779+0000","last_active":"2024-05-30T17:00:38.149705+0000","last_peered":"2024-05-30T17:00:38.149705+0000","last_clean":"2024-05-30T17:00:38.149705+0000","last_became_active":"2024-05-30T17:00:36.758402+0000","last_became_peered":"2024-05-30T17:00:36.758402+0000","last_unstale":"2024-05-30T17:00:38.149705+0000","last_undegraded":"2024-05-30T17:00:38.149705+0000","last_fullsized":"2024-05-30T17:00:38.149705+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752387+0000","last_change":"2024-05-30T17:00:36.759407+0000","last_active":"2024-05-30T17:00:37.752387+0000","last_peered":"2024-05-30T17:00:37.752387+0000","last_clean":"2024-05-30T17:00:37.752387+0000","last_became_active":"2024-05-30T17:00:36.759115+0000","last_became_peered":"2024-05-30T17:00:36.759115+0000","last_unstale":"2024-05-30T17:00:37.752387+0000","last_undegraded":"2024-05-30T17:00:37.752387+0000","last_fullsized":"2024-05-30T17:00:37.752387+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.710227+0000","last_change":"2024-05-30T17:00:35.710227+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.710227+0000","last_undegraded":"2024-05-30T17:00:35.710227+0000","last_fullsized":"2024-05-30T17:00:35.710227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.719035+0000","last_change":"2024-05-30T17:00:35.719035+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.719035+0000","last_undegraded":"2024-05-30T17:00:35.719035+0000","last_fullsized":"2024-05-30T17:00:35.719035+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760586+0000","last_change":"2024-05-30T17:00:36.762646+0000","last_active":"2024-05-30T17:00:37.760586+0000","last_peered":"2024-05-30T17:00:37.760586+0000","last_clean":"2024-05-30T17:00:37.760586+0000","last_became_active":"2024-05-30T17:00:36.761684+0000","last_became_peered":"2024-05-30T17:00:36.761684+0000","last_unstale":"2024-05-30T17:00:37.760586+0000","last_undegraded":"2024-05-30T17:00:37.760586+0000","last_fullsized":"2024-05-30T17:00:37.760586+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:50.133298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720502+0000","last_change":"2024-05-30T17:00:35.720502+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720502+0000","last_undegraded":"2024-05-30T17:00:35.720502+0000","last_fullsized":"2024-05-30T17:00:35.720502+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751731+0000","last_change":"2024-05-30T17:00:36.745182+0000","last_active":"2024-05-30T17:00:37.751731+0000","last_peered":"2024-05-30T17:00:37.751731+0000","last_clean":"2024-05-30T17:00:37.751731+0000","last_became_active":"2024-05-30T17:00:36.744869+0000","last_became_peered":"2024-05-30T17:00:36.744869+0000","last_unstale":"2024-05-30T17:00:37.751731+0000","last_undegraded":"2024-05-30T17:00:37.751731+0000","last_fullsized":"2024-05-30T17:00:37.751731+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:52:34.241585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751614+0000","last_change":"2024-05-30T17:00:36.746180+0000","last_active":"2024-05-30T17:00:37.751614+0000","last_peered":"2024-05-30T17:00:37.751614+0000","last_clean":"2024-05-30T17:00:37.751614+0000","last_became_active":"2024-05-30T17:00:36.745907+0000","last_became_peered":"2024-05-30T17:00:36.745907+0000","last_unstale":"2024-05-30T17:00:37.751614+0000","last_undegraded":"2024-05-30T17:00:37.751614+0000","last_fullsized":"2024-05-30T17:00:37.751614+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:43.488502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150152+0000","last_change":"2024-05-30T17:00:36.748364+0000","last_active":"2024-05-30T17:00:38.150152+0000","last_peered":"2024-05-30T17:00:38.150152+0000","last_clean":"2024-05-30T17:00:38.150152+0000","last_became_active":"2024-05-30T17:00:36.748051+0000","last_became_peered":"2024-05-30T17:00:36.748051+0000","last_unstale":"2024-05-30T17:00:38.150152+0000","last_undegraded":"2024-05-30T17:00:38.150152+0000","last_fullsized":"2024-05-30T17:00:38.150152+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:28.418390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151939+0000","last_change":"2024-05-30T17:00:36.754727+0000","last_active":"2024-05-30T17:00:38.151939+0000","last_peered":"2024-05-30T17:00:38.151939+0000","last_clean":"2024-05-30T17:00:38.151939+0000","last_became_active":"2024-05-30T17:00:36.753257+0000","last_became_peered":"2024-05-30T17:00:36.753257+0000","last_unstale":"2024-05-30T17:00:38.151939+0000","last_undegraded":"2024-05-30T17:00:38.151939+0000","last_fullsized":"2024-05-30T17:00:38.151939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:36.271526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752343+0000","last_change":"2024-05-30T17:00:36.754136+0000","last_active":"2024-05-30T17:00:37.752343+0000","last_peered":"2024-05-30T17:00:37.752343+0000","last_clean":"2024-05-30T17:00:37.752343+0000","last_became_active":"2024-05-30T17:00:36.753838+0000","last_became_peered":"2024-05-30T17:00:36.753838+0000","last_unstale":"2024-05-30T17:00:37.752343+0000","last_undegraded":"2024-05-30T17:00:37.752343+0000","last_fullsized":"2024-05-30T17:00:37.752343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149843+0000","last_change":"2024-05-30T17:00:36.753152+0000","last_active":"2024-05-30T17:00:38.149843+0000","last_peered":"2024-05-30T17:00:38.149843+0000","last_clean":"2024-05-30T17:00:38.149843+0000","last_became_active":"2024-05-30T17:00:36.752867+0000","last_became_peered":"2024-05-30T17:00:36.752867+0000","last_unstale":"2024-05-30T17:00:38.149843+0000","last_undegraded":"2024-05-30T17:00:38.149843+0000","last_fullsized":"2024-05-30T17:00:38.149843+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:19:07.666127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152257+0000","last_change":"2024-05-30T17:00:36.761708+0000","last_active":"2024-05-30T17:00:38.152257+0000","last_peered":"2024-05-30T17:00:38.152257+0000","last_clean":"2024-05-30T17:00:38.152257+0000","last_became_active":"2024-05-30T17:00:36.760932+0000","last_became_peered":"2024-05-30T17:00:36.760932+0000","last_unstale":"2024-05-30T17:00:38.152257+0000","last_undegraded":"2024-05-30T17:00:38.152257+0000","last_fullsized":"2024-05-30T17:00:38.152257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:40.779484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752069+0000","last_change":"2024-05-30T17:00:36.755732+0000","last_active":"2024-05-30T17:00:37.752069+0000","last_peered":"2024-05-30T17:00:37.752069+0000","last_clean":"2024-05-30T17:00:37.752069+0000","last_became_active":"2024-05-30T17:00:36.755428+0000","last_became_peered":"2024-05-30T17:00:36.755428+0000","last_unstale":"2024-05-30T17:00:37.752069+0000","last_undegraded":"2024-05-30T17:00:37.752069+0000","last_fullsized":"2024-05-30T17:00:37.752069+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:14.194525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.721256+0000","last_change":"2024-05-30T17:00:35.721256+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.721256+0000","last_undegraded":"2024-05-30T17:00:35.721256+0000","last_fullsized":"2024-05-30T17:00:35.721256+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.710277+0000","last_change":"2024-05-30T17:00:35.710277+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.710277+0000","last_undegraded":"2024-05-30T17:00:35.710277+0000","last_fullsized":"2024-05-30T17:00:35.710277+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.726080+0000","last_change":"2024-05-30T17:00:35.726080+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.726080+0000","last_undegraded":"2024-05-30T17:00:35.726080+0000","last_fullsized":"2024-05-30T17:00:35.726080+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752501+0000","last_change":"2024-05-30T17:00:35.735865+0000","last_active":"2024-05-30T17:00:37.752501+0000","last_peered":"2024-05-30T17:00:37.752501+0000","last_clean":"2024-05-30T17:00:37.752501+0000","last_became_active":"2024-05-30T17:00:35.735526+0000","last_became_peered":"2024-05-30T17:00:35.735526+0000","last_unstale":"2024-05-30T17:00:37.752501+0000","last_undegraded":"2024-05-30T17:00:37.752501+0000","last_fullsized":"2024-05-30T17:00:37.752501+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:42:29.413638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.711876+0000","last_change":"2024-05-30T17:00:35.711876+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.711876+0000","last_undegraded":"2024-05-30T17:00:35.711876+0000","last_fullsized":"2024-05-30T17:00:35.711876+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150045+0000","last_change":"2024-05-30T17:00:35.735197+0000","last_active":"2024-05-30T17:00:38.150045+0000","last_peered":"2024-05-30T17:00:38.150045+0000","last_clean":"2024-05-30T17:00:38.150045+0000","last_became_active":"2024-05-30T17:00:35.734913+0000","last_became_peered":"2024-05-30T17:00:35.734913+0000","last_unstale":"2024-05-30T17:00:38.150045+0000","last_undegraded":"2024-05-30T17:00:38.150045+0000","last_fullsized":"2024-05-30T17:00:38.150045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:58:37.442958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752193+0000","last_change":"2024-05-30T17:00:35.729069+0000","last_active":"2024-05-30T17:00:37.752193+0000","last_peered":"2024-05-30T17:00:37.752193+0000","last_clean":"2024-05-30T17:00:37.752193+0000","last_became_active":"2024-05-30T17:00:35.728765+0000","last_became_peered":"2024-05-30T17:00:35.728765+0000","last_unstale":"2024-05-30T17:00:37.752193+0000","last_undegraded":"2024-05-30T17:00:37.752193+0000","last_fullsized":"2024-05-30T17:00:37.752193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:49:59.611402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761260+0000","last_change":"2024-05-30T17:00:36.746352+0000","last_active":"2024-05-30T17:00:37.761260+0000","last_peered":"2024-05-30T17:00:37.761260+0000","last_clean":"2024-05-30T17:00:37.761260+0000","last_became_active":"2024-05-30T17:00:36.745894+0000","last_became_peered":"2024-05-30T17:00:36.745894+0000","last_unstale":"2024-05-30T17:00:37.761260+0000","last_undegraded":"2024-05-30T17:00:37.761260+0000","last_fullsized":"2024-05-30T17:00:37.761260+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:39:58.290648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752014+0000","last_change":"2024-05-30T17:00:35.734108+0000","last_active":"2024-05-30T17:00:37.752014+0000","last_peered":"2024-05-30T17:00:37.752014+0000","last_clean":"2024-05-30T17:00:37.752014+0000","last_became_active":"2024-05-30T17:00:35.733598+0000","last_became_peered":"2024-05-30T17:00:35.733598+0000","last_unstale":"2024-05-30T17:00:37.752014+0000","last_undegraded":"2024-05-30T17:00:37.752014+0000","last_fullsized":"2024-05-30T17:00:37.752014+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:22.323508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758033+0000","last_change":"2024-05-30T17:00:36.743969+0000","last_active":"2024-05-30T17:00:37.758033+0000","last_peered":"2024-05-30T17:00:37.758033+0000","last_clean":"2024-05-30T17:00:37.758033+0000","last_became_active":"2024-05-30T17:00:36.743574+0000","last_became_peered":"2024-05-30T17:00:36.743574+0000","last_unstale":"2024-05-30T17:00:37.758033+0000","last_undegraded":"2024-05-30T17:00:37.758033+0000","last_fullsized":"2024-05-30T17:00:37.758033+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:57.026949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.699719+0000","last_change":"2024-05-30T17:00:34.699719+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.699719+0000","last_undegraded":"2024-05-30T17:00:34.699719+0000","last_fullsized":"2024-05-30T17:00:34.699719+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149522+0000","last_change":"2024-05-30T17:00:36.755868+0000","last_active":"2024-05-30T17:00:38.149522+0000","last_peered":"2024-05-30T17:00:38.149522+0000","last_clean":"2024-05-30T17:00:38.149522+0000","last_became_active":"2024-05-30T17:00:36.755420+0000","last_became_peered":"2024-05-30T17:00:36.755420+0000","last_unstale":"2024-05-30T17:00:38.149522+0000","last_undegraded":"2024-05-30T17:00:38.149522+0000","last_fullsized":"2024-05-30T17:00:38.149522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751802+0000","last_change":"2024-05-30T17:00:35.733623+0000","last_active":"2024-05-30T17:00:37.751802+0000","last_peered":"2024-05-30T17:00:37.751802+0000","last_clean":"2024-05-30T17:00:37.751802+0000","last_became_active":"2024-05-30T17:00:35.733325+0000","last_became_peered":"2024-05-30T17:00:35.733325+0000","last_unstale":"2024-05-30T17:00:37.751802+0000","last_undegraded":"2024-05-30T17:00:37.751802+0000","last_fullsized":"2024-05-30T17:00:37.751802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:14.320132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.710078+0000","last_change":"2024-05-30T17:00:35.710078+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.710078+0000","last_undegraded":"2024-05-30T17:00:35.710078+0000","last_fullsized":"2024-05-30T17:00:35.710078+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151425+0000","last_change":"2024-05-30T17:00:35.734103+0000","last_active":"2024-05-30T17:00:38.151425+0000","last_peered":"2024-05-30T17:00:38.151425+0000","last_clean":"2024-05-30T17:00:38.151425+0000","last_became_active":"2024-05-30T17:00:35.733719+0000","last_became_peered":"2024-05-30T17:00:35.733719+0000","last_unstale":"2024-05-30T17:00:38.151425+0000","last_undegraded":"2024-05-30T17:00:38.151425+0000","last_fullsized":"2024-05-30T17:00:38.151425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:42.872961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759392+0000","last_change":"2024-05-30T17:00:36.747707+0000","last_active":"2024-05-30T17:00:37.759392+0000","last_peered":"2024-05-30T17:00:37.759392+0000","last_clean":"2024-05-30T17:00:37.759392+0000","last_became_active":"2024-05-30T17:00:36.744282+0000","last_became_peered":"2024-05-30T17:00:36.744282+0000","last_unstale":"2024-05-30T17:00:37.759392+0000","last_undegraded":"2024-05-30T17:00:37.759392+0000","last_fullsized":"2024-05-30T17:00:37.759392+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:48.052699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.703036+0000","last_change":"2024-05-30T17:00:34.703036+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.703036+0000","last_undegraded":"2024-05-30T17:00:34.703036+0000","last_fullsized":"2024-05-30T17:00:34.703036+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752394+0000","last_change":"2024-05-30T17:00:36.742303+0000","last_active":"2024-05-30T17:00:37.752394+0000","last_peered":"2024-05-30T17:00:37.752394+0000","last_clean":"2024-05-30T17:00:37.752394+0000","last_became_active":"2024-05-30T17:00:36.741809+0000","last_became_peered":"2024-05-30T17:00:36.741809+0000","last_unstale":"2024-05-30T17:00:37.752394+0000","last_undegraded":"2024-05-30T17:00:37.752394+0000","last_fullsized":"2024-05-30T17:00:37.752394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.699191+0000","last_change":"2024-05-30T17:00:34.699191+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.699191+0000","last_undegraded":"2024-05-30T17:00:34.699191+0000","last_fullsized":"2024-05-30T17:00:34.699191+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760526+0000","last_change":"2024-05-30T17:00:36.747039+0000","last_active":"2024-05-30T17:00:37.760526+0000","last_peered":"2024-05-30T17:00:37.760526+0000","last_clean":"2024-05-30T17:00:37.760526+0000","last_became_active":"2024-05-30T17:00:36.745915+0000","last_became_peered":"2024-05-30T17:00:36.745915+0000","last_unstale":"2024-05-30T17:00:37.760526+0000","last_undegraded":"2024-05-30T17:00:37.760526+0000","last_fullsized":"2024-05-30T17:00:37.760526+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:21.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.723219+0000","last_change":"2024-05-30T17:00:35.718080+0000","last_active":"2024-05-30T17:00:35.723219+0000","last_peered":"2024-05-30T17:00:35.723219+0000","last_clean":"2024-05-30T17:00:35.723219+0000","last_became_active":"2024-05-30T17:00:35.717705+0000","last_became_peered":"2024-05-30T17:00:35.717705+0000","last_unstale":"2024-05-30T17:00:35.723219+0000","last_undegraded":"2024-05-30T17:00:35.723219+0000","last_fullsized":"2024-05-30T17:00:35.723219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:39:57.277812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152539+0000","last_change":"2024-05-30T17:00:36.754484+0000","last_active":"2024-05-30T17:00:38.152539+0000","last_peered":"2024-05-30T17:00:38.152539+0000","last_clean":"2024-05-30T17:00:38.152539+0000","last_became_active":"2024-05-30T17:00:36.754166+0000","last_became_peered":"2024-05-30T17:00:36.754166+0000","last_unstale":"2024-05-30T17:00:38.152539+0000","last_undegraded":"2024-05-30T17:00:38.152539+0000","last_fullsized":"2024-05-30T17:00:38.152539+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:33.091804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150419+0000","last_change":"2024-05-30T17:00:35.732982+0000","last_active":"2024-05-30T17:00:38.150419+0000","last_peered":"2024-05-30T17:00:38.150419+0000","last_clean":"2024-05-30T17:00:38.150419+0000","last_became_active":"2024-05-30T17:00:35.732114+0000","last_became_peered":"2024-05-30T17:00:35.732114+0000","last_unstale":"2024-05-30T17:00:38.150419+0000","last_undegraded":"2024-05-30T17:00:38.150419+0000","last_fullsized":"2024-05-30T17:00:38.150419+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:33.694752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150348+0000","last_change":"2024-05-30T17:00:36.758949+0000","last_active":"2024-05-30T17:00:38.150348+0000","last_peered":"2024-05-30T17:00:38.150348+0000","last_clean":"2024-05-30T17:00:38.150348+0000","last_became_active":"2024-05-30T17:00:36.758069+0000","last_became_peered":"2024-05-30T17:00:36.758069+0000","last_unstale":"2024-05-30T17:00:38.150348+0000","last_undegraded":"2024-05-30T17:00:38.150348+0000","last_fullsized":"2024-05-30T17:00:38.150348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:38:49.283720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760740+0000","last_change":"2024-05-30T17:00:35.731848+0000","last_active":"2024-05-30T17:00:37.760740+0000","last_peered":"2024-05-30T17:00:37.760740+0000","last_clean":"2024-05-30T17:00:37.760740+0000","last_became_active":"2024-05-30T17:00:35.731536+0000","last_became_peered":"2024-05-30T17:00:35.731536+0000","last_unstale":"2024-05-30T17:00:37.760740+0000","last_undegraded":"2024-05-30T17:00:37.760740+0000","last_fullsized":"2024-05-30T17:00:37.760740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:12:27.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.708596+0000","last_change":"2024-05-30T17:00:35.708596+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.708596+0000","last_undegraded":"2024-05-30T17:00:35.708596+0000","last_fullsized":"2024-05-30T17:00:35.708596+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.737819+0000","last_change":"2024-05-30T17:00:35.734783+0000","last_active":"2024-05-30T17:00:35.737819+0000","last_peered":"2024-05-30T17:00:35.737819+0000","last_clean":"2024-05-30T17:00:35.737819+0000","last_became_active":"2024-05-30T17:00:35.733971+0000","last_became_peered":"2024-05-30T17:00:35.733971+0000","last_unstale":"2024-05-30T17:00:35.737819+0000","last_undegraded":"2024-05-30T17:00:35.737819+0000","last_fullsized":"2024-05-30T17:00:35.737819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:44.116430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.739677+0000","last_change":"2024-05-30T17:00:35.738764+0000","last_active":"2024-05-30T17:00:35.739677+0000","last_peered":"2024-05-30T17:00:35.739677+0000","last_clean":"2024-05-30T17:00:35.739677+0000","last_became_active":"2024-05-30T17:00:35.737679+0000","last_became_peered":"2024-05-30T17:00:35.737679+0000","last_unstale":"2024-05-30T17:00:35.739677+0000","last_undegraded":"2024-05-30T17:00:35.739677+0000","last_fullsized":"2024-05-30T17:00:35.739677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:13:25.537216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760261+0000","last_change":"2024-05-30T17:00:36.744702+0000","last_active":"2024-05-30T17:00:37.760261+0000","last_peered":"2024-05-30T17:00:37.760261+0000","last_clean":"2024-05-30T17:00:37.760261+0000","last_became_active":"2024-05-30T17:00:36.744329+0000","last_became_peered":"2024-05-30T17:00:36.744329+0000","last_unstale":"2024-05-30T17:00:37.760261+0000","last_undegraded":"2024-05-30T17:00:37.760261+0000","last_fullsized":"2024-05-30T17:00:37.760261+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:21:27.452364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759287+0000","last_change":"2024-05-30T17:00:35.729767+0000","last_active":"2024-05-30T17:00:37.759287+0000","last_peered":"2024-05-30T17:00:37.759287+0000","last_clean":"2024-05-30T17:00:37.759287+0000","last_became_active":"2024-05-30T17:00:35.729412+0000","last_became_peered":"2024-05-30T17:00:35.729412+0000","last_unstale":"2024-05-30T17:00:37.759287+0000","last_undegraded":"2024-05-30T17:00:37.759287+0000","last_fullsized":"2024-05-30T17:00:37.759287+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:47.793776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.716683+0000","last_change":"2024-05-30T17:00:35.716683+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.716683+0000","last_undegraded":"2024-05-30T17:00:35.716683+0000","last_fullsized":"2024-05-30T17:00:35.716683+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.699431+0000","last_change":"2024-05-30T17:00:34.699431+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.699431+0000","last_undegraded":"2024-05-30T17:00:34.699431+0000","last_fullsized":"2024-05-30T17:00:34.699431+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152237+0000","last_change":"2024-05-30T17:00:36.751676+0000","last_active":"2024-05-30T17:00:38.152237+0000","last_peered":"2024-05-30T17:00:38.152237+0000","last_clean":"2024-05-30T17:00:38.152237+0000","last_became_active":"2024-05-30T17:00:36.751355+0000","last_became_peered":"2024-05-30T17:00:36.751355+0000","last_unstale":"2024-05-30T17:00:38.152237+0000","last_undegraded":"2024-05-30T17:00:38.152237+0000","last_fullsized":"2024-05-30T17:00:38.152237+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:01:43.178806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760541+0000","last_change":"2024-05-30T17:00:35.734613+0000","last_active":"2024-05-30T17:00:37.760541+0000","last_peered":"2024-05-30T17:00:37.760541+0000","last_clean":"2024-05-30T17:00:37.760541+0000","last_became_active":"2024-05-30T17:00:35.734231+0000","last_became_peered":"2024-05-30T17:00:35.734231+0000","last_unstale":"2024-05-30T17:00:37.760541+0000","last_undegraded":"2024-05-30T17:00:37.760541+0000","last_fullsized":"2024-05-30T17:00:37.760541+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:06.120742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758055+0000","last_change":"2024-05-30T17:00:36.757383+0000","last_active":"2024-05-30T17:00:37.758055+0000","last_peered":"2024-05-30T17:00:37.758055+0000","last_clean":"2024-05-30T17:00:37.758055+0000","last_became_active":"2024-05-30T17:00:36.756264+0000","last_became_peered":"2024-05-30T17:00:36.756264+0000","last_unstale":"2024-05-30T17:00:37.758055+0000","last_undegraded":"2024-05-30T17:00:37.758055+0000","last_fullsized":"2024-05-30T17:00:37.758055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:32.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151230+0000","last_change":"2024-05-30T17:00:35.730141+0000","last_active":"2024-05-30T17:00:38.151230+0000","last_peered":"2024-05-30T17:00:38.151230+0000","last_clean":"2024-05-30T17:00:38.151230+0000","last_became_active":"2024-05-30T17:00:35.728948+0000","last_became_peered":"2024-05-30T17:00:35.728948+0000","last_unstale":"2024-05-30T17:00:38.151230+0000","last_undegraded":"2024-05-30T17:00:38.151230+0000","last_fullsized":"2024-05-30T17:00:38.151230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:13.200467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150264+0000","last_change":"2024-05-30T17:00:36.750999+0000","last_active":"2024-05-30T17:00:38.150264+0000","last_peered":"2024-05-30T17:00:38.150264+0000","last_clean":"2024-05-30T17:00:38.150264+0000","last_became_active":"2024-05-30T17:00:36.750499+0000","last_became_peered":"2024-05-30T17:00:36.750499+0000","last_unstale":"2024-05-30T17:00:38.150264+0000","last_undegraded":"2024-05-30T17:00:38.150264+0000","last_fullsized":"2024-05-30T17:00:38.150264+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:10.934086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751996+0000","last_change":"2024-05-30T17:00:35.733883+0000","last_active":"2024-05-30T17:00:37.751996+0000","last_peered":"2024-05-30T17:00:37.751996+0000","last_clean":"2024-05-30T17:00:37.751996+0000","last_became_active":"2024-05-30T17:00:35.733243+0000","last_became_peered":"2024-05-30T17:00:35.733243+0000","last_unstale":"2024-05-30T17:00:37.751996+0000","last_undegraded":"2024-05-30T17:00:37.751996+0000","last_fullsized":"2024-05-30T17:00:37.751996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:46:39.811373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720831+0000","last_change":"2024-05-30T17:00:35.720831+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720831+0000","last_undegraded":"2024-05-30T17:00:35.720831+0000","last_fullsized":"2024-05-30T17:00:35.720831+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752192+0000","last_change":"2024-05-30T17:00:35.729083+0000","last_active":"2024-05-30T17:00:37.752192+0000","last_peered":"2024-05-30T17:00:37.752192+0000","last_clean":"2024-05-30T17:00:37.752192+0000","last_became_active":"2024-05-30T17:00:35.728830+0000","last_became_peered":"2024-05-30T17:00:35.728830+0000","last_unstale":"2024-05-30T17:00:37.752192+0000","last_undegraded":"2024-05-30T17:00:37.752192+0000","last_fullsized":"2024-05-30T17:00:37.752192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:52.170762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.719960+0000","last_change":"2024-05-30T17:00:35.719960+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.719960+0000","last_undegraded":"2024-05-30T17:00:35.719960+0000","last_fullsized":"2024-05-30T17:00:35.719960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757948+0000","last_change":"2024-05-30T17:00:35.730608+0000","last_active":"2024-05-30T17:00:37.757948+0000","last_peered":"2024-05-30T17:00:37.757948+0000","last_clean":"2024-05-30T17:00:37.757948+0000","last_became_active":"2024-05-30T17:00:35.730310+0000","last_became_peered":"2024-05-30T17:00:35.730310+0000","last_unstale":"2024-05-30T17:00:37.757948+0000","last_undegraded":"2024-05-30T17:00:37.757948+0000","last_fullsized":"2024-05-30T17:00:37.757948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:13.646048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.717200+0000","last_change":"2024-05-30T17:00:35.717200+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.717200+0000","last_undegraded":"2024-05-30T17:00:35.717200+0000","last_fullsized":"2024-05-30T17:00:35.717200+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757483+0000","last_change":"2024-05-30T17:00:35.709483+0000","last_active":"2024-05-30T17:00:37.757483+0000","last_peered":"2024-05-30T17:00:37.757483+0000","last_clean":"2024-05-30T17:00:37.757483+0000","last_became_active":"2024-05-30T17:00:35.708526+0000","last_became_peered":"2024-05-30T17:00:35.708526+0000","last_unstale":"2024-05-30T17:00:37.757483+0000","last_undegraded":"2024-05-30T17:00:37.757483+0000","last_fullsized":"2024-05-30T17:00:37.757483+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:40.251625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151273+0000","last_change":"2024-05-30T17:00:36.749874+0000","last_active":"2024-05-30T17:00:38.151273+0000","last_peered":"2024-05-30T17:00:38.151273+0000","last_clean":"2024-05-30T17:00:38.151273+0000","last_became_active":"2024-05-30T17:00:36.749228+0000","last_became_peered":"2024-05-30T17:00:36.749228+0000","last_unstale":"2024-05-30T17:00:38.151273+0000","last_undegraded":"2024-05-30T17:00:38.151273+0000","last_fullsized":"2024-05-30T17:00:38.151273+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:44.276755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751571+0000","last_change":"2024-05-30T17:00:35.736336+0000","last_active":"2024-05-30T17:00:37.751571+0000","last_peered":"2024-05-30T17:00:37.751571+0000","last_clean":"2024-05-30T17:00:37.751571+0000","last_became_active":"2024-05-30T17:00:35.734451+0000","last_became_peered":"2024-05-30T17:00:35.734451+0000","last_unstale":"2024-05-30T17:00:37.751571+0000","last_undegraded":"2024-05-30T17:00:37.751571+0000","last_fullsized":"2024-05-30T17:00:37.751571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:16.292089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752599+0000","last_change":"2024-05-30T17:00:36.748422+0000","last_active":"2024-05-30T17:00:37.752599+0000","last_peered":"2024-05-30T17:00:37.752599+0000","last_clean":"2024-05-30T17:00:37.752599+0000","last_became_active":"2024-05-30T17:00:36.747900+0000","last_became_peered":"2024-05-30T17:00:36.747900+0000","last_unstale":"2024-05-30T17:00:37.752599+0000","last_undegraded":"2024-05-30T17:00:37.752599+0000","last_fullsized":"2024-05-30T17:00:37.752599+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758807+0000","last_change":"2024-05-30T17:00:35.729597+0000","last_active":"2024-05-30T17:00:37.758807+0000","last_peered":"2024-05-30T17:00:37.758807+0000","last_clean":"2024-05-30T17:00:37.758807+0000","last_became_active":"2024-05-30T17:00:35.728519+0000","last_became_peered":"2024-05-30T17:00:35.728519+0000","last_unstale":"2024-05-30T17:00:37.758807+0000","last_undegraded":"2024-05-30T17:00:37.758807+0000","last_fullsized":"2024-05-30T17:00:37.758807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:12.232879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.754450+0000","last_change":"2024-05-30T17:00:35.733609+0000","last_active":"2024-05-30T17:00:37.754450+0000","last_peered":"2024-05-30T17:00:37.754450+0000","last_clean":"2024-05-30T17:00:37.754450+0000","last_became_active":"2024-05-30T17:00:35.733292+0000","last_became_peered":"2024-05-30T17:00:35.733292+0000","last_unstale":"2024-05-30T17:00:37.754450+0000","last_undegraded":"2024-05-30T17:00:37.754450+0000","last_fullsized":"2024-05-30T17:00:37.754450+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:00.979924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752339+0000","last_change":"2024-05-30T17:00:35.734441+0000","last_active":"2024-05-30T17:00:37.752339+0000","last_peered":"2024-05-30T17:00:37.752339+0000","last_clean":"2024-05-30T17:00:37.752339+0000","last_became_active":"2024-05-30T17:00:35.732710+0000","last_became_peered":"2024-05-30T17:00:35.732710+0000","last_unstale":"2024-05-30T17:00:37.752339+0000","last_undegraded":"2024-05-30T17:00:37.752339+0000","last_fullsized":"2024-05-30T17:00:37.752339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:15.063268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151374+0000","last_change":"2024-05-30T17:00:36.764319+0000","last_active":"2024-05-30T17:00:38.151374+0000","last_peered":"2024-05-30T17:00:38.151374+0000","last_clean":"2024-05-30T17:00:38.151374+0000","last_became_active":"2024-05-30T17:00:36.764034+0000","last_became_peered":"2024-05-30T17:00:36.764034+0000","last_unstale":"2024-05-30T17:00:38.151374+0000","last_undegraded":"2024-05-30T17:00:38.151374+0000","last_fullsized":"2024-05-30T17:00:38.151374+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:28.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152390+0000","last_change":"2024-05-30T17:00:35.727312+0000","last_active":"2024-05-30T17:00:38.152390+0000","last_peered":"2024-05-30T17:00:38.152390+0000","last_clean":"2024-05-30T17:00:38.152390+0000","last_became_active":"2024-05-30T17:00:35.726820+0000","last_became_peered":"2024-05-30T17:00:35.726820+0000","last_unstale":"2024-05-30T17:00:38.152390+0000","last_undegraded":"2024-05-30T17:00:38.152390+0000","last_fullsized":"2024-05-30T17:00:38.152390+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:40.105333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720407+0000","last_change":"2024-05-30T17:00:35.720407+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720407+0000","last_undegraded":"2024-05-30T17:00:35.720407+0000","last_fullsized":"2024-05-30T17:00:35.720407+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752300+0000","last_change":"2024-05-30T17:00:35.727143+0000","last_active":"2024-05-30T17:00:37.752300+0000","last_peered":"2024-05-30T17:00:37.752300+0000","last_clean":"2024-05-30T17:00:37.752300+0000","last_became_active":"2024-05-30T17:00:35.725458+0000","last_became_peered":"2024-05-30T17:00:35.725458+0000","last_unstale":"2024-05-30T17:00:37.752300+0000","last_undegraded":"2024-05-30T17:00:37.752300+0000","last_fullsized":"2024-05-30T17:00:37.752300+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:49.887792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760276+0000","last_change":"2024-05-30T17:00:36.756498+0000","last_active":"2024-05-30T17:00:37.760276+0000","last_peered":"2024-05-30T17:00:37.760276+0000","last_clean":"2024-05-30T17:00:37.760276+0000","last_became_active":"2024-05-30T17:00:36.755460+0000","last_became_peered":"2024-05-30T17:00:36.755460+0000","last_unstale":"2024-05-30T17:00:37.760276+0000","last_undegraded":"2024-05-30T17:00:37.760276+0000","last_fullsized":"2024-05-30T17:00:37.760276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:01.424865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150911+0000","last_change":"2024-05-30T17:00:35.731535+0000","last_active":"2024-05-30T17:00:38.150911+0000","last_peered":"2024-05-30T17:00:38.150911+0000","last_clean":"2024-05-30T17:00:38.150911+0000","last_became_active":"2024-05-30T17:00:35.730893+0000","last_became_peered":"2024-05-30T17:00:35.730893+0000","last_unstale":"2024-05-30T17:00:38.150911+0000","last_undegraded":"2024-05-30T17:00:38.150911+0000","last_fullsized":"2024-05-30T17:00:38.150911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:45.698539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.716687+0000","last_change":"2024-05-30T17:00:35.716687+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.716687+0000","last_undegraded":"2024-05-30T17:00:35.716687+0000","last_fullsized":"2024-05-30T17:00:35.716687+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.727740+0000","last_change":"2024-05-30T17:00:35.724537+0000","last_active":"2024-05-30T17:00:35.727740+0000","last_peered":"2024-05-30T17:00:35.727740+0000","last_clean":"2024-05-30T17:00:35.727740+0000","last_became_active":"2024-05-30T17:00:35.724245+0000","last_became_peered":"2024-05-30T17:00:35.724245+0000","last_unstale":"2024-05-30T17:00:35.727740+0000","last_undegraded":"2024-05-30T17:00:35.727740+0000","last_fullsized":"2024-05-30T17:00:35.727740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:07:13.036207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151226+0000","last_change":"2024-05-30T17:00:36.761666+0000","last_active":"2024-05-30T17:00:38.151226+0000","last_peered":"2024-05-30T17:00:38.151226+0000","last_clean":"2024-05-30T17:00:38.151226+0000","last_became_active":"2024-05-30T17:00:36.760872+0000","last_became_peered":"2024-05-30T17:00:36.760872+0000","last_unstale":"2024-05-30T17:00:38.151226+0000","last_undegraded":"2024-05-30T17:00:38.151226+0000","last_fullsized":"2024-05-30T17:00:38.151226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:12:08.423568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.736077+0000","last_change":"2024-05-30T17:00:35.732840+0000","last_active":"2024-05-30T17:00:35.736077+0000","last_peered":"2024-05-30T17:00:35.736077+0000","last_clean":"2024-05-30T17:00:35.736077+0000","last_became_active":"2024-05-30T17:00:35.732525+0000","last_became_peered":"2024-05-30T17:00:35.732525+0000","last_unstale":"2024-05-30T17:00:35.736077+0000","last_undegraded":"2024-05-30T17:00:35.736077+0000","last_fullsized":"2024-05-30T17:00:35.736077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:29.507352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720954+0000","last_change":"2024-05-30T17:00:35.720954+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720954+0000","last_undegraded":"2024-05-30T17:00:35.720954+0000","last_fullsized":"2024-05-30T17:00:35.720954+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149819+0000","last_change":"2024-05-30T17:00:35.729629+0000","last_active":"2024-05-30T17:00:38.149819+0000","last_peered":"2024-05-30T17:00:38.149819+0000","last_clean":"2024-05-30T17:00:38.149819+0000","last_became_active":"2024-05-30T17:00:35.729282+0000","last_became_peered":"2024-05-30T17:00:35.729282+0000","last_unstale":"2024-05-30T17:00:38.149819+0000","last_undegraded":"2024-05-30T17:00:38.149819+0000","last_fullsized":"2024-05-30T17:00:38.149819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:43.450197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":330,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148934+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:00:38.148934+0000","last_peered":"2024-05-30T17:00:38.148934+0000","last_clean":"2024-05-30T17:00:38.148934+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:00:38.148934+0000","last_undegraded":"2024-05-30T17:00:38.148934+0000","last_fullsized":"2024-05-30T17:00:38.148934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509647,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130936,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240904,"statfs":{"total":96636764160,"available":96502685696,"internally_reserved":0,"allocated":1286144,"data_stored":558361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509647,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86528,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285312,"statfs":{"total":96636764160,"available":96548159488,"internally_reserved":0,"allocated":1298432,"data_stored":559445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509647,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126728,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245112,"statfs":{"total":96636764160,"available":96506994688,"internally_reserved":0,"allocated":1241088,"data_stored":538107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509647,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144792,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227048,"statfs":{"total":96636764160,"available":96488497152,"internally_reserved":0,"allocated":1253376,"data_stored":544466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509646,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126732,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245108,"statfs":{"total":96636764160,"available":96506990592,"internally_reserved":0,"allocated":1241088,"data_stored":538107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542350,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127356,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244484,"statfs":{"total":96636764160,"available":96506351616,"internally_reserved":0,"allocated":1880064,"data_stored":1148189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76788,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295052,"statfs":{"total":96636764160,"available":96558133248,"internally_reserved":0,"allocated":1282048,"data_stored":557779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542351,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69044,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302796,"statfs":{"total":96636764160,"available":96566063104,"internally_reserved":0,"allocated":1875968,"data_stored":1148659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:00:41.774 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:00:41.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.991+0000 7fe48ce00640 1 Processor -- start 2024-05-30T17:00:41.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.992+0000 7fe48ce00640 1 -- start start 2024-05-30T17:00:41.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.993+0000 7fe48ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 0x7fe4880ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:41.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.993+0000 7fe48ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe48806f2c0 0x7fe4880ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:41.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.993+0000 7fe48ce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe488156f10 con 0x7fe488156b30 2024-05-30T17:00:41.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.993+0000 7fe48ce00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe48806dda0 con 0x7fe48806f2c0 2024-05-30T17:00:41.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.993+0000 7fe48ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe48806e080 con 0x7fe48806ee40 2024-05-30T17:00:41.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.993+0000 7fe487400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe488156b30 0x7fe4880af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56706/0 (socket says 172.21.15.67:56706) 2024-05-30T17:00:41.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.993+0000 7fe487400640 1 -- 172.21.15.67:0/3202731793 learned_addr learned my addr 172.21.15.67:0/3202731793 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:41.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.993+0000 7fe487e00640 1 --2- 172.21.15.67:0/3202731793 >> v2:172.21.15.67:3300/0 conn(0x7fe48806f2c0 0x7fe4880ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:41.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.994+0000 7fe486a00640 1 --2- 172.21.15.67:0/3202731793 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 0x7fe4880ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:41.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.994+0000 7fe486000640 1 -- 172.21.15.67:0/3202731793 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034629432 0 0) 0x7fe488156f10 con 0x7fe488156b30 2024-05-30T17:00:41.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.994+0000 7fe486000640 1 -- 172.21.15.67:0/3202731793 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe46c003520 con 0x7fe488156b30 2024-05-30T17:00:41.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.994+0000 7fe487e00640 1 -- 172.21.15.67:0/3202731793 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 msgr2=0x7fe4880ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:41.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.994+0000 7fe487e00640 1 --2- 172.21.15.67:0/3202731793 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 0x7fe4880ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.994+0000 7fe487e00640 1 -- 172.21.15.67:0/3202731793 >> v1:172.21.15.67:6789/0 conn(0x7fe488156b30 legacy=0x7fe4880af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.994+0000 7fe487e00640 1 -- 172.21.15.67:0/3202731793 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48806e360 con 0x7fe48806f2c0 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.995+0000 7fe487e00640 1 --2- 172.21.15.67:0/3202731793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 0x7fe4880ad770 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe4800097a0 tx=0x7fe48002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9efe19b5fcc84f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.995+0000 7fe486000640 1 -- 172.21.15.67:0/3202731793 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480037070 con 0x7fe48806f2c0 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.995+0000 7fe486000640 1 -- 172.21.15.67:0/3202731793 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48002ae50 con 0x7fe48806f2c0 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.995+0000 7fe486000640 1 -- 172.21.15.67:0/3202731793 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480032660 con 0x7fe48806f2c0 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.996+0000 7fe48ce00640 1 -- 172.21.15.67:0/3202731793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 msgr2=0x7fe4880ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.996+0000 7fe48ce00640 1 --2- 172.21.15.67:0/3202731793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 0x7fe4880ad770 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe4800097a0 tx=0x7fe48002a100 comp rx=0 tx=0).stop 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.996+0000 7fe48ce00640 1 -- 172.21.15.67:0/3202731793 shutdown_connections 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.996+0000 7fe48ce00640 1 --2- 172.21.15.67:0/3202731793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 0x7fe4880ad770 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.996+0000 7fe48ce00640 1 --2- 172.21.15.67:0/3202731793 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 0x7fe4880ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:41.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.996+0000 7fe48ce00640 1 -- 172.21.15.67:0/3202731793 >> 172.21.15.67:0/3202731793 conn(0x7fe4880b1fc0 msgr2=0x7fe4880b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:41.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.996+0000 7fe48ce00640 1 -- 172.21.15.67:0/3202731793 shutdown_connections 2024-05-30T17:00:41.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.996+0000 7fe48ce00640 1 -- 172.21.15.67:0/3202731793 wait complete. 2024-05-30T17:00:41.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.998+0000 7fe48ce00640 1 Processor -- start 2024-05-30T17:00:41.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.998+0000 7fe48ce00640 1 -- start start 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.999+0000 7fe48ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 0x7fe4881f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.999+0000 7fe48ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 0x7fe488159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.999+0000 7fe48ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488156b30 0x7fe48814c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.999+0000 7fe48ce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe4880b2db0 con 0x7fe48806f2c0 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.999+0000 7fe48ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4880b2c30 con 0x7fe488156b30 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.999+0000 7fe48ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe4880b32c0 con 0x7fe48806ee40 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.999+0000 7fe486a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 0x7fe488159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.999+0000 7fe486a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 0x7fe488159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53148/0 (socket says 172.21.15.67:53148) 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:41.999+0000 7fe486a00640 1 -- 172.21.15.67:0/1153681575 learned_addr learned my addr 172.21.15.67:0/1153681575 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.000+0000 7fe487400640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 0x7fe4881f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.000+0000 7fe486a00640 1 -- 172.21.15.67:0/1153681575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 msgr2=0x7fe4881f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.000+0000 7fe486a00640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 0x7fe4881f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.000+0000 7fe486a00640 1 -- 172.21.15.67:0/1153681575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488156b30 msgr2=0x7fe48814c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.000+0000 7fe486a00640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488156b30 0x7fe48814c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.000+0000 7fe486a00640 1 -- 172.21.15.67:0/1153681575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe488157510 con 0x7fe48806f2c0 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.000+0000 7fe486a00640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 0x7fe488159870 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe47000d800 tx=0x7fe47000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.000+0000 7fe47f400640 1 -- 172.21.15.67:0/1153681575 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47000b6c0 con 0x7fe48806f2c0 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.001+0000 7fe47f400640 1 -- 172.21.15.67:0/1153681575 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe470019070 con 0x7fe48806f2c0 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.001+0000 7fe47f400640 1 -- 172.21.15.67:0/1153681575 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47000ba80 con 0x7fe48806f2c0 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.001+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881577f0 con 0x7fe48806f2c0 2024-05-30T17:00:42.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.001+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48814cbd0 con 0x7fe48806f2c0 2024-05-30T17:00:42.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.002+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4880689e0 con 0x7fe48806f2c0 2024-05-30T17:00:42.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.003+0000 7fe47f400640 1 -- 172.21.15.67:0/1153681575 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fe47000bc20 con 0x7fe48806f2c0 2024-05-30T17:00:42.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.004+0000 7fe47f400640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe46406ddc0 0x7fe464070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.004+0000 7fe47f400640 1 -- 172.21.15.67:0/1153681575 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fe47008b100 con 0x7fe48806f2c0 2024-05-30T17:00:42.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.004+0000 7fe487400640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe46406ddc0 0x7fe464070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:42.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.005+0000 7fe487400640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe46406ddc0 0x7fe464070280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe478005c00 tx=0x7fe478009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.009+0000 7fe47f400640 1 -- 172.21.15.67:0/1153681575 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe47005ac00 con 0x7fe48806f2c0 2024-05-30T17:00:42.128 INFO:tasks.ceph.mds.c.smithi067.stderr:2024-05-30T17:00:42.127+0000 7ff120000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T17:00:42.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.211+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe48806f7f0 con 0x7fe46406ddc0 2024-05-30T17:00:42.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.230+0000 7fe47f400640 1 -- 172.21.15.67:0/1153681575 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249442 (secure 0 0 0) 0x7fe48806f7f0 con 0x7fe46406ddc0 2024-05-30T17:00:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:42.236 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:00:42.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.237+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe46406ddc0 msgr2=0x7fe464070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:42.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.237+0000 7fe48ce00640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe46406ddc0 0x7fe464070280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe478005c00 tx=0x7fe478009040 comp rx=0 tx=0).stop 2024-05-30T17:00:42.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.237+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 msgr2=0x7fe488159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:42.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.237+0000 7fe48ce00640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 0x7fe488159870 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe47000d800 tx=0x7fe47000dcc0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.238+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 shutdown_connections 2024-05-30T17:00:42.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.238+0000 7fe48ce00640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe46406ddc0 0x7fe464070280 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.238+0000 7fe48ce00640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488156b30 0x7fe48814c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.238+0000 7fe48ce00640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe48806f2c0 0x7fe488159870 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.238+0000 7fe48ce00640 1 --2- 172.21.15.67:0/1153681575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe48806ee40 0x7fe4881f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.238+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 >> 172.21.15.67:0/1153681575 conn(0x7fe4880b1fc0 msgr2=0x7fe488155410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:42.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.238+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 shutdown_connections 2024-05-30T17:00:42.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.238+0000 7fe48ce00640 1 -- 172.21.15.67:0/1153681575 wait complete. 2024-05-30T17:00:42.259 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":608,"stamp":"2024-05-30T17:00:40.472092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":888904,"kb_used_data":11092,"kb_used_omap":12,"kb_used_meta":877683,"kb_avail":754085816,"statfs":{"total":773094113280,"available":772183875584,"internally_reserved":0,"allocated":11358208,"data_stored":5593113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":898747952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.782789"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752224+0000","last_change":"2024-05-30T17:00:36.748181+0000","last_active":"2024-05-30T17:00:37.752224+0000","last_peered":"2024-05-30T17:00:37.752224+0000","last_clean":"2024-05-30T17:00:37.752224+0000","last_became_active":"2024-05-30T17:00:36.747392+0000","last_became_peered":"2024-05-30T17:00:36.747392+0000","last_unstale":"2024-05-30T17:00:37.752224+0000","last_undegraded":"2024-05-30T17:00:37.752224+0000","last_fullsized":"2024-05-30T17:00:37.752224+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.712512+0000","last_change":"2024-05-30T17:00:35.712512+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.712512+0000","last_undegraded":"2024-05-30T17:00:35.712512+0000","last_fullsized":"2024-05-30T17:00:35.712512+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151835+0000","last_change":"2024-05-30T17:00:36.763111+0000","last_active":"2024-05-30T17:00:38.151835+0000","last_peered":"2024-05-30T17:00:38.151835+0000","last_clean":"2024-05-30T17:00:38.151835+0000","last_became_active":"2024-05-30T17:00:36.761404+0000","last_became_peered":"2024-05-30T17:00:36.761404+0000","last_unstale":"2024-05-30T17:00:38.151835+0000","last_undegraded":"2024-05-30T17:00:38.151835+0000","last_fullsized":"2024-05-30T17:00:38.151835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:01.455423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149546+0000","last_change":"2024-05-30T17:00:36.751908+0000","last_active":"2024-05-30T17:00:38.149546+0000","last_peered":"2024-05-30T17:00:38.149546+0000","last_clean":"2024-05-30T17:00:38.149546+0000","last_became_active":"2024-05-30T17:00:36.751604+0000","last_became_peered":"2024-05-30T17:00:36.751604+0000","last_unstale":"2024-05-30T17:00:38.149546+0000","last_undegraded":"2024-05-30T17:00:38.149546+0000","last_fullsized":"2024-05-30T17:00:38.149546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149447+0000","last_change":"2024-05-30T17:00:36.748294+0000","last_active":"2024-05-30T17:00:38.149447+0000","last_peered":"2024-05-30T17:00:38.149447+0000","last_clean":"2024-05-30T17:00:38.149447+0000","last_became_active":"2024-05-30T17:00:36.747984+0000","last_became_peered":"2024-05-30T17:00:36.747984+0000","last_unstale":"2024-05-30T17:00:38.149447+0000","last_undegraded":"2024-05-30T17:00:38.149447+0000","last_fullsized":"2024-05-30T17:00:38.149447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751747+0000","last_change":"2024-05-30T17:00:36.751171+0000","last_active":"2024-05-30T17:00:37.751747+0000","last_peered":"2024-05-30T17:00:37.751747+0000","last_clean":"2024-05-30T17:00:37.751747+0000","last_became_active":"2024-05-30T17:00:36.747956+0000","last_became_peered":"2024-05-30T17:00:36.747956+0000","last_unstale":"2024-05-30T17:00:37.751747+0000","last_undegraded":"2024-05-30T17:00:37.751747+0000","last_fullsized":"2024-05-30T17:00:37.751747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:03:57.450595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760121+0000","last_change":"2024-05-30T17:00:36.762682+0000","last_active":"2024-05-30T17:00:37.760121+0000","last_peered":"2024-05-30T17:00:37.760121+0000","last_clean":"2024-05-30T17:00:37.760121+0000","last_became_active":"2024-05-30T17:00:36.762190+0000","last_became_peered":"2024-05-30T17:00:36.762190+0000","last_unstale":"2024-05-30T17:00:37.760121+0000","last_undegraded":"2024-05-30T17:00:37.760121+0000","last_fullsized":"2024-05-30T17:00:37.760121+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:33.645864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151899+0000","last_change":"2024-05-30T17:00:36.758850+0000","last_active":"2024-05-30T17:00:38.151899+0000","last_peered":"2024-05-30T17:00:38.151899+0000","last_clean":"2024-05-30T17:00:38.151899+0000","last_became_active":"2024-05-30T17:00:36.758522+0000","last_became_peered":"2024-05-30T17:00:36.758522+0000","last_unstale":"2024-05-30T17:00:38.151899+0000","last_undegraded":"2024-05-30T17:00:38.151899+0000","last_fullsized":"2024-05-30T17:00:38.151899+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:20.247961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149815+0000","last_change":"2024-05-30T17:00:36.750772+0000","last_active":"2024-05-30T17:00:38.149815+0000","last_peered":"2024-05-30T17:00:38.149815+0000","last_clean":"2024-05-30T17:00:38.149815+0000","last_became_active":"2024-05-30T17:00:36.750122+0000","last_became_peered":"2024-05-30T17:00:36.750122+0000","last_unstale":"2024-05-30T17:00:38.149815+0000","last_undegraded":"2024-05-30T17:00:38.149815+0000","last_fullsized":"2024-05-30T17:00:38.149815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761029+0000","last_change":"2024-05-30T17:00:36.751475+0000","last_active":"2024-05-30T17:00:37.761029+0000","last_peered":"2024-05-30T17:00:37.761029+0000","last_clean":"2024-05-30T17:00:37.761029+0000","last_became_active":"2024-05-30T17:00:36.751150+0000","last_became_peered":"2024-05-30T17:00:36.751150+0000","last_unstale":"2024-05-30T17:00:37.761029+0000","last_undegraded":"2024-05-30T17:00:37.761029+0000","last_fullsized":"2024-05-30T17:00:37.761029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:55:05.659729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.712050+0000","last_change":"2024-05-30T17:00:35.712050+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.712050+0000","last_undegraded":"2024-05-30T17:00:35.712050+0000","last_fullsized":"2024-05-30T17:00:35.712050+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.721668+0000","last_change":"2024-05-30T17:00:35.721668+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.721668+0000","last_undegraded":"2024-05-30T17:00:35.721668+0000","last_fullsized":"2024-05-30T17:00:35.721668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.711828+0000","last_change":"2024-05-30T17:00:35.711828+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.711828+0000","last_undegraded":"2024-05-30T17:00:35.711828+0000","last_fullsized":"2024-05-30T17:00:35.711828+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149705+0000","last_change":"2024-05-30T17:00:36.758779+0000","last_active":"2024-05-30T17:00:38.149705+0000","last_peered":"2024-05-30T17:00:38.149705+0000","last_clean":"2024-05-30T17:00:38.149705+0000","last_became_active":"2024-05-30T17:00:36.758402+0000","last_became_peered":"2024-05-30T17:00:36.758402+0000","last_unstale":"2024-05-30T17:00:38.149705+0000","last_undegraded":"2024-05-30T17:00:38.149705+0000","last_fullsized":"2024-05-30T17:00:38.149705+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752387+0000","last_change":"2024-05-30T17:00:36.759407+0000","last_active":"2024-05-30T17:00:37.752387+0000","last_peered":"2024-05-30T17:00:37.752387+0000","last_clean":"2024-05-30T17:00:37.752387+0000","last_became_active":"2024-05-30T17:00:36.759115+0000","last_became_peered":"2024-05-30T17:00:36.759115+0000","last_unstale":"2024-05-30T17:00:37.752387+0000","last_undegraded":"2024-05-30T17:00:37.752387+0000","last_fullsized":"2024-05-30T17:00:37.752387+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.710227+0000","last_change":"2024-05-30T17:00:35.710227+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.710227+0000","last_undegraded":"2024-05-30T17:00:35.710227+0000","last_fullsized":"2024-05-30T17:00:35.710227+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.719035+0000","last_change":"2024-05-30T17:00:35.719035+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.719035+0000","last_undegraded":"2024-05-30T17:00:35.719035+0000","last_fullsized":"2024-05-30T17:00:35.719035+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760586+0000","last_change":"2024-05-30T17:00:36.762646+0000","last_active":"2024-05-30T17:00:37.760586+0000","last_peered":"2024-05-30T17:00:37.760586+0000","last_clean":"2024-05-30T17:00:37.760586+0000","last_became_active":"2024-05-30T17:00:36.761684+0000","last_became_peered":"2024-05-30T17:00:36.761684+0000","last_unstale":"2024-05-30T17:00:37.760586+0000","last_undegraded":"2024-05-30T17:00:37.760586+0000","last_fullsized":"2024-05-30T17:00:37.760586+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:50.133298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720502+0000","last_change":"2024-05-30T17:00:35.720502+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720502+0000","last_undegraded":"2024-05-30T17:00:35.720502+0000","last_fullsized":"2024-05-30T17:00:35.720502+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751731+0000","last_change":"2024-05-30T17:00:36.745182+0000","last_active":"2024-05-30T17:00:37.751731+0000","last_peered":"2024-05-30T17:00:37.751731+0000","last_clean":"2024-05-30T17:00:37.751731+0000","last_became_active":"2024-05-30T17:00:36.744869+0000","last_became_peered":"2024-05-30T17:00:36.744869+0000","last_unstale":"2024-05-30T17:00:37.751731+0000","last_undegraded":"2024-05-30T17:00:37.751731+0000","last_fullsized":"2024-05-30T17:00:37.751731+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:52:34.241585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751614+0000","last_change":"2024-05-30T17:00:36.746180+0000","last_active":"2024-05-30T17:00:37.751614+0000","last_peered":"2024-05-30T17:00:37.751614+0000","last_clean":"2024-05-30T17:00:37.751614+0000","last_became_active":"2024-05-30T17:00:36.745907+0000","last_became_peered":"2024-05-30T17:00:36.745907+0000","last_unstale":"2024-05-30T17:00:37.751614+0000","last_undegraded":"2024-05-30T17:00:37.751614+0000","last_fullsized":"2024-05-30T17:00:37.751614+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:43.488502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150152+0000","last_change":"2024-05-30T17:00:36.748364+0000","last_active":"2024-05-30T17:00:38.150152+0000","last_peered":"2024-05-30T17:00:38.150152+0000","last_clean":"2024-05-30T17:00:38.150152+0000","last_became_active":"2024-05-30T17:00:36.748051+0000","last_became_peered":"2024-05-30T17:00:36.748051+0000","last_unstale":"2024-05-30T17:00:38.150152+0000","last_undegraded":"2024-05-30T17:00:38.150152+0000","last_fullsized":"2024-05-30T17:00:38.150152+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:28.418390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151939+0000","last_change":"2024-05-30T17:00:36.754727+0000","last_active":"2024-05-30T17:00:38.151939+0000","last_peered":"2024-05-30T17:00:38.151939+0000","last_clean":"2024-05-30T17:00:38.151939+0000","last_became_active":"2024-05-30T17:00:36.753257+0000","last_became_peered":"2024-05-30T17:00:36.753257+0000","last_unstale":"2024-05-30T17:00:38.151939+0000","last_undegraded":"2024-05-30T17:00:38.151939+0000","last_fullsized":"2024-05-30T17:00:38.151939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:36.271526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752343+0000","last_change":"2024-05-30T17:00:36.754136+0000","last_active":"2024-05-30T17:00:37.752343+0000","last_peered":"2024-05-30T17:00:37.752343+0000","last_clean":"2024-05-30T17:00:37.752343+0000","last_became_active":"2024-05-30T17:00:36.753838+0000","last_became_peered":"2024-05-30T17:00:36.753838+0000","last_unstale":"2024-05-30T17:00:37.752343+0000","last_undegraded":"2024-05-30T17:00:37.752343+0000","last_fullsized":"2024-05-30T17:00:37.752343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149843+0000","last_change":"2024-05-30T17:00:36.753152+0000","last_active":"2024-05-30T17:00:38.149843+0000","last_peered":"2024-05-30T17:00:38.149843+0000","last_clean":"2024-05-30T17:00:38.149843+0000","last_became_active":"2024-05-30T17:00:36.752867+0000","last_became_peered":"2024-05-30T17:00:36.752867+0000","last_unstale":"2024-05-30T17:00:38.149843+0000","last_undegraded":"2024-05-30T17:00:38.149843+0000","last_fullsized":"2024-05-30T17:00:38.149843+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:19:07.666127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152257+0000","last_change":"2024-05-30T17:00:36.761708+0000","last_active":"2024-05-30T17:00:38.152257+0000","last_peered":"2024-05-30T17:00:38.152257+0000","last_clean":"2024-05-30T17:00:38.152257+0000","last_became_active":"2024-05-30T17:00:36.760932+0000","last_became_peered":"2024-05-30T17:00:36.760932+0000","last_unstale":"2024-05-30T17:00:38.152257+0000","last_undegraded":"2024-05-30T17:00:38.152257+0000","last_fullsized":"2024-05-30T17:00:38.152257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:40.779484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752069+0000","last_change":"2024-05-30T17:00:36.755732+0000","last_active":"2024-05-30T17:00:37.752069+0000","last_peered":"2024-05-30T17:00:37.752069+0000","last_clean":"2024-05-30T17:00:37.752069+0000","last_became_active":"2024-05-30T17:00:36.755428+0000","last_became_peered":"2024-05-30T17:00:36.755428+0000","last_unstale":"2024-05-30T17:00:37.752069+0000","last_undegraded":"2024-05-30T17:00:37.752069+0000","last_fullsized":"2024-05-30T17:00:37.752069+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:14.194525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.721256+0000","last_change":"2024-05-30T17:00:35.721256+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.721256+0000","last_undegraded":"2024-05-30T17:00:35.721256+0000","last_fullsized":"2024-05-30T17:00:35.721256+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.710277+0000","last_change":"2024-05-30T17:00:35.710277+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.710277+0000","last_undegraded":"2024-05-30T17:00:35.710277+0000","last_fullsized":"2024-05-30T17:00:35.710277+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.726080+0000","last_change":"2024-05-30T17:00:35.726080+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.726080+0000","last_undegraded":"2024-05-30T17:00:35.726080+0000","last_fullsized":"2024-05-30T17:00:35.726080+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752501+0000","last_change":"2024-05-30T17:00:35.735865+0000","last_active":"2024-05-30T17:00:37.752501+0000","last_peered":"2024-05-30T17:00:37.752501+0000","last_clean":"2024-05-30T17:00:37.752501+0000","last_became_active":"2024-05-30T17:00:35.735526+0000","last_became_peered":"2024-05-30T17:00:35.735526+0000","last_unstale":"2024-05-30T17:00:37.752501+0000","last_undegraded":"2024-05-30T17:00:37.752501+0000","last_fullsized":"2024-05-30T17:00:37.752501+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:42:29.413638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.711876+0000","last_change":"2024-05-30T17:00:35.711876+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.711876+0000","last_undegraded":"2024-05-30T17:00:35.711876+0000","last_fullsized":"2024-05-30T17:00:35.711876+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150045+0000","last_change":"2024-05-30T17:00:35.735197+0000","last_active":"2024-05-30T17:00:38.150045+0000","last_peered":"2024-05-30T17:00:38.150045+0000","last_clean":"2024-05-30T17:00:38.150045+0000","last_became_active":"2024-05-30T17:00:35.734913+0000","last_became_peered":"2024-05-30T17:00:35.734913+0000","last_unstale":"2024-05-30T17:00:38.150045+0000","last_undegraded":"2024-05-30T17:00:38.150045+0000","last_fullsized":"2024-05-30T17:00:38.150045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:58:37.442958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752193+0000","last_change":"2024-05-30T17:00:35.729069+0000","last_active":"2024-05-30T17:00:37.752193+0000","last_peered":"2024-05-30T17:00:37.752193+0000","last_clean":"2024-05-30T17:00:37.752193+0000","last_became_active":"2024-05-30T17:00:35.728765+0000","last_became_peered":"2024-05-30T17:00:35.728765+0000","last_unstale":"2024-05-30T17:00:37.752193+0000","last_undegraded":"2024-05-30T17:00:37.752193+0000","last_fullsized":"2024-05-30T17:00:37.752193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:49:59.611402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761260+0000","last_change":"2024-05-30T17:00:36.746352+0000","last_active":"2024-05-30T17:00:37.761260+0000","last_peered":"2024-05-30T17:00:37.761260+0000","last_clean":"2024-05-30T17:00:37.761260+0000","last_became_active":"2024-05-30T17:00:36.745894+0000","last_became_peered":"2024-05-30T17:00:36.745894+0000","last_unstale":"2024-05-30T17:00:37.761260+0000","last_undegraded":"2024-05-30T17:00:37.761260+0000","last_fullsized":"2024-05-30T17:00:37.761260+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:39:58.290648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752014+0000","last_change":"2024-05-30T17:00:35.734108+0000","last_active":"2024-05-30T17:00:37.752014+0000","last_peered":"2024-05-30T17:00:37.752014+0000","last_clean":"2024-05-30T17:00:37.752014+0000","last_became_active":"2024-05-30T17:00:35.733598+0000","last_became_peered":"2024-05-30T17:00:35.733598+0000","last_unstale":"2024-05-30T17:00:37.752014+0000","last_undegraded":"2024-05-30T17:00:37.752014+0000","last_fullsized":"2024-05-30T17:00:37.752014+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:22.323508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758033+0000","last_change":"2024-05-30T17:00:36.743969+0000","last_active":"2024-05-30T17:00:37.758033+0000","last_peered":"2024-05-30T17:00:37.758033+0000","last_clean":"2024-05-30T17:00:37.758033+0000","last_became_active":"2024-05-30T17:00:36.743574+0000","last_became_peered":"2024-05-30T17:00:36.743574+0000","last_unstale":"2024-05-30T17:00:37.758033+0000","last_undegraded":"2024-05-30T17:00:37.758033+0000","last_fullsized":"2024-05-30T17:00:37.758033+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:57.026949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.699719+0000","last_change":"2024-05-30T17:00:34.699719+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.699719+0000","last_undegraded":"2024-05-30T17:00:34.699719+0000","last_fullsized":"2024-05-30T17:00:34.699719+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149522+0000","last_change":"2024-05-30T17:00:36.755868+0000","last_active":"2024-05-30T17:00:38.149522+0000","last_peered":"2024-05-30T17:00:38.149522+0000","last_clean":"2024-05-30T17:00:38.149522+0000","last_became_active":"2024-05-30T17:00:36.755420+0000","last_became_peered":"2024-05-30T17:00:36.755420+0000","last_unstale":"2024-05-30T17:00:38.149522+0000","last_undegraded":"2024-05-30T17:00:38.149522+0000","last_fullsized":"2024-05-30T17:00:38.149522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751802+0000","last_change":"2024-05-30T17:00:35.733623+0000","last_active":"2024-05-30T17:00:37.751802+0000","last_peered":"2024-05-30T17:00:37.751802+0000","last_clean":"2024-05-30T17:00:37.751802+0000","last_became_active":"2024-05-30T17:00:35.733325+0000","last_became_peered":"2024-05-30T17:00:35.733325+0000","last_unstale":"2024-05-30T17:00:37.751802+0000","last_undegraded":"2024-05-30T17:00:37.751802+0000","last_fullsized":"2024-05-30T17:00:37.751802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:14.320132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.710078+0000","last_change":"2024-05-30T17:00:35.710078+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.710078+0000","last_undegraded":"2024-05-30T17:00:35.710078+0000","last_fullsized":"2024-05-30T17:00:35.710078+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151425+0000","last_change":"2024-05-30T17:00:35.734103+0000","last_active":"2024-05-30T17:00:38.151425+0000","last_peered":"2024-05-30T17:00:38.151425+0000","last_clean":"2024-05-30T17:00:38.151425+0000","last_became_active":"2024-05-30T17:00:35.733719+0000","last_became_peered":"2024-05-30T17:00:35.733719+0000","last_unstale":"2024-05-30T17:00:38.151425+0000","last_undegraded":"2024-05-30T17:00:38.151425+0000","last_fullsized":"2024-05-30T17:00:38.151425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:42.872961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759392+0000","last_change":"2024-05-30T17:00:36.747707+0000","last_active":"2024-05-30T17:00:37.759392+0000","last_peered":"2024-05-30T17:00:37.759392+0000","last_clean":"2024-05-30T17:00:37.759392+0000","last_became_active":"2024-05-30T17:00:36.744282+0000","last_became_peered":"2024-05-30T17:00:36.744282+0000","last_unstale":"2024-05-30T17:00:37.759392+0000","last_undegraded":"2024-05-30T17:00:37.759392+0000","last_fullsized":"2024-05-30T17:00:37.759392+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:48.052699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.703036+0000","last_change":"2024-05-30T17:00:34.703036+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.703036+0000","last_undegraded":"2024-05-30T17:00:34.703036+0000","last_fullsized":"2024-05-30T17:00:34.703036+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752394+0000","last_change":"2024-05-30T17:00:36.742303+0000","last_active":"2024-05-30T17:00:37.752394+0000","last_peered":"2024-05-30T17:00:37.752394+0000","last_clean":"2024-05-30T17:00:37.752394+0000","last_became_active":"2024-05-30T17:00:36.741809+0000","last_became_peered":"2024-05-30T17:00:36.741809+0000","last_unstale":"2024-05-30T17:00:37.752394+0000","last_undegraded":"2024-05-30T17:00:37.752394+0000","last_fullsized":"2024-05-30T17:00:37.752394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.699191+0000","last_change":"2024-05-30T17:00:34.699191+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.699191+0000","last_undegraded":"2024-05-30T17:00:34.699191+0000","last_fullsized":"2024-05-30T17:00:34.699191+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760526+0000","last_change":"2024-05-30T17:00:36.747039+0000","last_active":"2024-05-30T17:00:37.760526+0000","last_peered":"2024-05-30T17:00:37.760526+0000","last_clean":"2024-05-30T17:00:37.760526+0000","last_became_active":"2024-05-30T17:00:36.745915+0000","last_became_peered":"2024-05-30T17:00:36.745915+0000","last_unstale":"2024-05-30T17:00:37.760526+0000","last_undegraded":"2024-05-30T17:00:37.760526+0000","last_fullsized":"2024-05-30T17:00:37.760526+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:21.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.723219+0000","last_change":"2024-05-30T17:00:35.718080+0000","last_active":"2024-05-30T17:00:35.723219+0000","last_peered":"2024-05-30T17:00:35.723219+0000","last_clean":"2024-05-30T17:00:35.723219+0000","last_became_active":"2024-05-30T17:00:35.717705+0000","last_became_peered":"2024-05-30T17:00:35.717705+0000","last_unstale":"2024-05-30T17:00:35.723219+0000","last_undegraded":"2024-05-30T17:00:35.723219+0000","last_fullsized":"2024-05-30T17:00:35.723219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:39:57.277812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152539+0000","last_change":"2024-05-30T17:00:36.754484+0000","last_active":"2024-05-30T17:00:38.152539+0000","last_peered":"2024-05-30T17:00:38.152539+0000","last_clean":"2024-05-30T17:00:38.152539+0000","last_became_active":"2024-05-30T17:00:36.754166+0000","last_became_peered":"2024-05-30T17:00:36.754166+0000","last_unstale":"2024-05-30T17:00:38.152539+0000","last_undegraded":"2024-05-30T17:00:38.152539+0000","last_fullsized":"2024-05-30T17:00:38.152539+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:33.091804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150419+0000","last_change":"2024-05-30T17:00:35.732982+0000","last_active":"2024-05-30T17:00:38.150419+0000","last_peered":"2024-05-30T17:00:38.150419+0000","last_clean":"2024-05-30T17:00:38.150419+0000","last_became_active":"2024-05-30T17:00:35.732114+0000","last_became_peered":"2024-05-30T17:00:35.732114+0000","last_unstale":"2024-05-30T17:00:38.150419+0000","last_undegraded":"2024-05-30T17:00:38.150419+0000","last_fullsized":"2024-05-30T17:00:38.150419+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:33.694752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150348+0000","last_change":"2024-05-30T17:00:36.758949+0000","last_active":"2024-05-30T17:00:38.150348+0000","last_peered":"2024-05-30T17:00:38.150348+0000","last_clean":"2024-05-30T17:00:38.150348+0000","last_became_active":"2024-05-30T17:00:36.758069+0000","last_became_peered":"2024-05-30T17:00:36.758069+0000","last_unstale":"2024-05-30T17:00:38.150348+0000","last_undegraded":"2024-05-30T17:00:38.150348+0000","last_fullsized":"2024-05-30T17:00:38.150348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:38:49.283720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760740+0000","last_change":"2024-05-30T17:00:35.731848+0000","last_active":"2024-05-30T17:00:37.760740+0000","last_peered":"2024-05-30T17:00:37.760740+0000","last_clean":"2024-05-30T17:00:37.760740+0000","last_became_active":"2024-05-30T17:00:35.731536+0000","last_became_peered":"2024-05-30T17:00:35.731536+0000","last_unstale":"2024-05-30T17:00:37.760740+0000","last_undegraded":"2024-05-30T17:00:37.760740+0000","last_fullsized":"2024-05-30T17:00:37.760740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:12:27.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.708596+0000","last_change":"2024-05-30T17:00:35.708596+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.708596+0000","last_undegraded":"2024-05-30T17:00:35.708596+0000","last_fullsized":"2024-05-30T17:00:35.708596+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.737819+0000","last_change":"2024-05-30T17:00:35.734783+0000","last_active":"2024-05-30T17:00:35.737819+0000","last_peered":"2024-05-30T17:00:35.737819+0000","last_clean":"2024-05-30T17:00:35.737819+0000","last_became_active":"2024-05-30T17:00:35.733971+0000","last_became_peered":"2024-05-30T17:00:35.733971+0000","last_unstale":"2024-05-30T17:00:35.737819+0000","last_undegraded":"2024-05-30T17:00:35.737819+0000","last_fullsized":"2024-05-30T17:00:35.737819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:44.116430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.739677+0000","last_change":"2024-05-30T17:00:35.738764+0000","last_active":"2024-05-30T17:00:35.739677+0000","last_peered":"2024-05-30T17:00:35.739677+0000","last_clean":"2024-05-30T17:00:35.739677+0000","last_became_active":"2024-05-30T17:00:35.737679+0000","last_became_peered":"2024-05-30T17:00:35.737679+0000","last_unstale":"2024-05-30T17:00:35.739677+0000","last_undegraded":"2024-05-30T17:00:35.739677+0000","last_fullsized":"2024-05-30T17:00:35.739677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:13:25.537216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760261+0000","last_change":"2024-05-30T17:00:36.744702+0000","last_active":"2024-05-30T17:00:37.760261+0000","last_peered":"2024-05-30T17:00:37.760261+0000","last_clean":"2024-05-30T17:00:37.760261+0000","last_became_active":"2024-05-30T17:00:36.744329+0000","last_became_peered":"2024-05-30T17:00:36.744329+0000","last_unstale":"2024-05-30T17:00:37.760261+0000","last_undegraded":"2024-05-30T17:00:37.760261+0000","last_fullsized":"2024-05-30T17:00:37.760261+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:21:27.452364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759287+0000","last_change":"2024-05-30T17:00:35.729767+0000","last_active":"2024-05-30T17:00:37.759287+0000","last_peered":"2024-05-30T17:00:37.759287+0000","last_clean":"2024-05-30T17:00:37.759287+0000","last_became_active":"2024-05-30T17:00:35.729412+0000","last_became_peered":"2024-05-30T17:00:35.729412+0000","last_unstale":"2024-05-30T17:00:37.759287+0000","last_undegraded":"2024-05-30T17:00:37.759287+0000","last_fullsized":"2024-05-30T17:00:37.759287+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:47.793776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.716683+0000","last_change":"2024-05-30T17:00:35.716683+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.716683+0000","last_undegraded":"2024-05-30T17:00:35.716683+0000","last_fullsized":"2024-05-30T17:00:35.716683+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-30T17:00:34.699431+0000","last_change":"2024-05-30T17:00:34.699431+0000","last_active":"2024-05-30T17:00:34.674517+0000","last_peered":"2024-05-30T17:00:34.674517+0000","last_clean":"2024-05-30T17:00:34.674517+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:34.699431+0000","last_undegraded":"2024-05-30T17:00:34.699431+0000","last_fullsized":"2024-05-30T17:00:34.699431+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152237+0000","last_change":"2024-05-30T17:00:36.751676+0000","last_active":"2024-05-30T17:00:38.152237+0000","last_peered":"2024-05-30T17:00:38.152237+0000","last_clean":"2024-05-30T17:00:38.152237+0000","last_became_active":"2024-05-30T17:00:36.751355+0000","last_became_peered":"2024-05-30T17:00:36.751355+0000","last_unstale":"2024-05-30T17:00:38.152237+0000","last_undegraded":"2024-05-30T17:00:38.152237+0000","last_fullsized":"2024-05-30T17:00:38.152237+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:01:43.178806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760541+0000","last_change":"2024-05-30T17:00:35.734613+0000","last_active":"2024-05-30T17:00:37.760541+0000","last_peered":"2024-05-30T17:00:37.760541+0000","last_clean":"2024-05-30T17:00:37.760541+0000","last_became_active":"2024-05-30T17:00:35.734231+0000","last_became_peered":"2024-05-30T17:00:35.734231+0000","last_unstale":"2024-05-30T17:00:37.760541+0000","last_undegraded":"2024-05-30T17:00:37.760541+0000","last_fullsized":"2024-05-30T17:00:37.760541+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:06.120742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758055+0000","last_change":"2024-05-30T17:00:36.757383+0000","last_active":"2024-05-30T17:00:37.758055+0000","last_peered":"2024-05-30T17:00:37.758055+0000","last_clean":"2024-05-30T17:00:37.758055+0000","last_became_active":"2024-05-30T17:00:36.756264+0000","last_became_peered":"2024-05-30T17:00:36.756264+0000","last_unstale":"2024-05-30T17:00:37.758055+0000","last_undegraded":"2024-05-30T17:00:37.758055+0000","last_fullsized":"2024-05-30T17:00:37.758055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:32.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151230+0000","last_change":"2024-05-30T17:00:35.730141+0000","last_active":"2024-05-30T17:00:38.151230+0000","last_peered":"2024-05-30T17:00:38.151230+0000","last_clean":"2024-05-30T17:00:38.151230+0000","last_became_active":"2024-05-30T17:00:35.728948+0000","last_became_peered":"2024-05-30T17:00:35.728948+0000","last_unstale":"2024-05-30T17:00:38.151230+0000","last_undegraded":"2024-05-30T17:00:38.151230+0000","last_fullsized":"2024-05-30T17:00:38.151230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:13.200467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150264+0000","last_change":"2024-05-30T17:00:36.750999+0000","last_active":"2024-05-30T17:00:38.150264+0000","last_peered":"2024-05-30T17:00:38.150264+0000","last_clean":"2024-05-30T17:00:38.150264+0000","last_became_active":"2024-05-30T17:00:36.750499+0000","last_became_peered":"2024-05-30T17:00:36.750499+0000","last_unstale":"2024-05-30T17:00:38.150264+0000","last_undegraded":"2024-05-30T17:00:38.150264+0000","last_fullsized":"2024-05-30T17:00:38.150264+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:10.934086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751996+0000","last_change":"2024-05-30T17:00:35.733883+0000","last_active":"2024-05-30T17:00:37.751996+0000","last_peered":"2024-05-30T17:00:37.751996+0000","last_clean":"2024-05-30T17:00:37.751996+0000","last_became_active":"2024-05-30T17:00:35.733243+0000","last_became_peered":"2024-05-30T17:00:35.733243+0000","last_unstale":"2024-05-30T17:00:37.751996+0000","last_undegraded":"2024-05-30T17:00:37.751996+0000","last_fullsized":"2024-05-30T17:00:37.751996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:46:39.811373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720831+0000","last_change":"2024-05-30T17:00:35.720831+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720831+0000","last_undegraded":"2024-05-30T17:00:35.720831+0000","last_fullsized":"2024-05-30T17:00:35.720831+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752192+0000","last_change":"2024-05-30T17:00:35.729083+0000","last_active":"2024-05-30T17:00:37.752192+0000","last_peered":"2024-05-30T17:00:37.752192+0000","last_clean":"2024-05-30T17:00:37.752192+0000","last_became_active":"2024-05-30T17:00:35.728830+0000","last_became_peered":"2024-05-30T17:00:35.728830+0000","last_unstale":"2024-05-30T17:00:37.752192+0000","last_undegraded":"2024-05-30T17:00:37.752192+0000","last_fullsized":"2024-05-30T17:00:37.752192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:52.170762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.719960+0000","last_change":"2024-05-30T17:00:35.719960+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.719960+0000","last_undegraded":"2024-05-30T17:00:35.719960+0000","last_fullsized":"2024-05-30T17:00:35.719960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757948+0000","last_change":"2024-05-30T17:00:35.730608+0000","last_active":"2024-05-30T17:00:37.757948+0000","last_peered":"2024-05-30T17:00:37.757948+0000","last_clean":"2024-05-30T17:00:37.757948+0000","last_became_active":"2024-05-30T17:00:35.730310+0000","last_became_peered":"2024-05-30T17:00:35.730310+0000","last_unstale":"2024-05-30T17:00:37.757948+0000","last_undegraded":"2024-05-30T17:00:37.757948+0000","last_fullsized":"2024-05-30T17:00:37.757948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:13.646048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.717200+0000","last_change":"2024-05-30T17:00:35.717200+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.717200+0000","last_undegraded":"2024-05-30T17:00:35.717200+0000","last_fullsized":"2024-05-30T17:00:35.717200+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757483+0000","last_change":"2024-05-30T17:00:35.709483+0000","last_active":"2024-05-30T17:00:37.757483+0000","last_peered":"2024-05-30T17:00:37.757483+0000","last_clean":"2024-05-30T17:00:37.757483+0000","last_became_active":"2024-05-30T17:00:35.708526+0000","last_became_peered":"2024-05-30T17:00:35.708526+0000","last_unstale":"2024-05-30T17:00:37.757483+0000","last_undegraded":"2024-05-30T17:00:37.757483+0000","last_fullsized":"2024-05-30T17:00:37.757483+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:40.251625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151273+0000","last_change":"2024-05-30T17:00:36.749874+0000","last_active":"2024-05-30T17:00:38.151273+0000","last_peered":"2024-05-30T17:00:38.151273+0000","last_clean":"2024-05-30T17:00:38.151273+0000","last_became_active":"2024-05-30T17:00:36.749228+0000","last_became_peered":"2024-05-30T17:00:36.749228+0000","last_unstale":"2024-05-30T17:00:38.151273+0000","last_undegraded":"2024-05-30T17:00:38.151273+0000","last_fullsized":"2024-05-30T17:00:38.151273+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:44.276755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751571+0000","last_change":"2024-05-30T17:00:35.736336+0000","last_active":"2024-05-30T17:00:37.751571+0000","last_peered":"2024-05-30T17:00:37.751571+0000","last_clean":"2024-05-30T17:00:37.751571+0000","last_became_active":"2024-05-30T17:00:35.734451+0000","last_became_peered":"2024-05-30T17:00:35.734451+0000","last_unstale":"2024-05-30T17:00:37.751571+0000","last_undegraded":"2024-05-30T17:00:37.751571+0000","last_fullsized":"2024-05-30T17:00:37.751571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:16.292089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752599+0000","last_change":"2024-05-30T17:00:36.748422+0000","last_active":"2024-05-30T17:00:37.752599+0000","last_peered":"2024-05-30T17:00:37.752599+0000","last_clean":"2024-05-30T17:00:37.752599+0000","last_became_active":"2024-05-30T17:00:36.747900+0000","last_became_peered":"2024-05-30T17:00:36.747900+0000","last_unstale":"2024-05-30T17:00:37.752599+0000","last_undegraded":"2024-05-30T17:00:37.752599+0000","last_fullsized":"2024-05-30T17:00:37.752599+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758807+0000","last_change":"2024-05-30T17:00:35.729597+0000","last_active":"2024-05-30T17:00:37.758807+0000","last_peered":"2024-05-30T17:00:37.758807+0000","last_clean":"2024-05-30T17:00:37.758807+0000","last_became_active":"2024-05-30T17:00:35.728519+0000","last_became_peered":"2024-05-30T17:00:35.728519+0000","last_unstale":"2024-05-30T17:00:37.758807+0000","last_undegraded":"2024-05-30T17:00:37.758807+0000","last_fullsized":"2024-05-30T17:00:37.758807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:12.232879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.754450+0000","last_change":"2024-05-30T17:00:35.733609+0000","last_active":"2024-05-30T17:00:37.754450+0000","last_peered":"2024-05-30T17:00:37.754450+0000","last_clean":"2024-05-30T17:00:37.754450+0000","last_became_active":"2024-05-30T17:00:35.733292+0000","last_became_peered":"2024-05-30T17:00:35.733292+0000","last_unstale":"2024-05-30T17:00:37.754450+0000","last_undegraded":"2024-05-30T17:00:37.754450+0000","last_fullsized":"2024-05-30T17:00:37.754450+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:00.979924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:00:35.687353+0000","last_change":"2024-05-30T17:00:35.687353+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.687353+0000","last_undegraded":"2024-05-30T17:00:35.687353+0000","last_fullsized":"2024-05-30T17:00:35.687353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752339+0000","last_change":"2024-05-30T17:00:35.734441+0000","last_active":"2024-05-30T17:00:37.752339+0000","last_peered":"2024-05-30T17:00:37.752339+0000","last_clean":"2024-05-30T17:00:37.752339+0000","last_became_active":"2024-05-30T17:00:35.732710+0000","last_became_peered":"2024-05-30T17:00:35.732710+0000","last_unstale":"2024-05-30T17:00:37.752339+0000","last_undegraded":"2024-05-30T17:00:37.752339+0000","last_fullsized":"2024-05-30T17:00:37.752339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:15.063268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151374+0000","last_change":"2024-05-30T17:00:36.764319+0000","last_active":"2024-05-30T17:00:38.151374+0000","last_peered":"2024-05-30T17:00:38.151374+0000","last_clean":"2024-05-30T17:00:38.151374+0000","last_became_active":"2024-05-30T17:00:36.764034+0000","last_became_peered":"2024-05-30T17:00:36.764034+0000","last_unstale":"2024-05-30T17:00:38.151374+0000","last_undegraded":"2024-05-30T17:00:38.151374+0000","last_fullsized":"2024-05-30T17:00:38.151374+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:28.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152390+0000","last_change":"2024-05-30T17:00:35.727312+0000","last_active":"2024-05-30T17:00:38.152390+0000","last_peered":"2024-05-30T17:00:38.152390+0000","last_clean":"2024-05-30T17:00:38.152390+0000","last_became_active":"2024-05-30T17:00:35.726820+0000","last_became_peered":"2024-05-30T17:00:35.726820+0000","last_unstale":"2024-05-30T17:00:38.152390+0000","last_undegraded":"2024-05-30T17:00:38.152390+0000","last_fullsized":"2024-05-30T17:00:38.152390+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:40.105333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720407+0000","last_change":"2024-05-30T17:00:35.720407+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720407+0000","last_undegraded":"2024-05-30T17:00:35.720407+0000","last_fullsized":"2024-05-30T17:00:35.720407+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752300+0000","last_change":"2024-05-30T17:00:35.727143+0000","last_active":"2024-05-30T17:00:37.752300+0000","last_peered":"2024-05-30T17:00:37.752300+0000","last_clean":"2024-05-30T17:00:37.752300+0000","last_became_active":"2024-05-30T17:00:35.725458+0000","last_became_peered":"2024-05-30T17:00:35.725458+0000","last_unstale":"2024-05-30T17:00:37.752300+0000","last_undegraded":"2024-05-30T17:00:37.752300+0000","last_fullsized":"2024-05-30T17:00:37.752300+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:49.887792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760276+0000","last_change":"2024-05-30T17:00:36.756498+0000","last_active":"2024-05-30T17:00:37.760276+0000","last_peered":"2024-05-30T17:00:37.760276+0000","last_clean":"2024-05-30T17:00:37.760276+0000","last_became_active":"2024-05-30T17:00:36.755460+0000","last_became_peered":"2024-05-30T17:00:36.755460+0000","last_unstale":"2024-05-30T17:00:37.760276+0000","last_undegraded":"2024-05-30T17:00:37.760276+0000","last_fullsized":"2024-05-30T17:00:37.760276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:01.424865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150911+0000","last_change":"2024-05-30T17:00:35.731535+0000","last_active":"2024-05-30T17:00:38.150911+0000","last_peered":"2024-05-30T17:00:38.150911+0000","last_clean":"2024-05-30T17:00:38.150911+0000","last_became_active":"2024-05-30T17:00:35.730893+0000","last_became_peered":"2024-05-30T17:00:35.730893+0000","last_unstale":"2024-05-30T17:00:38.150911+0000","last_undegraded":"2024-05-30T17:00:38.150911+0000","last_fullsized":"2024-05-30T17:00:38.150911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:45.698539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.716687+0000","last_change":"2024-05-30T17:00:35.716687+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.716687+0000","last_undegraded":"2024-05-30T17:00:35.716687+0000","last_fullsized":"2024-05-30T17:00:35.716687+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.727740+0000","last_change":"2024-05-30T17:00:35.724537+0000","last_active":"2024-05-30T17:00:35.727740+0000","last_peered":"2024-05-30T17:00:35.727740+0000","last_clean":"2024-05-30T17:00:35.727740+0000","last_became_active":"2024-05-30T17:00:35.724245+0000","last_became_peered":"2024-05-30T17:00:35.724245+0000","last_unstale":"2024-05-30T17:00:35.727740+0000","last_undegraded":"2024-05-30T17:00:35.727740+0000","last_fullsized":"2024-05-30T17:00:35.727740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:07:13.036207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151226+0000","last_change":"2024-05-30T17:00:36.761666+0000","last_active":"2024-05-30T17:00:38.151226+0000","last_peered":"2024-05-30T17:00:38.151226+0000","last_clean":"2024-05-30T17:00:38.151226+0000","last_became_active":"2024-05-30T17:00:36.760872+0000","last_became_peered":"2024-05-30T17:00:36.760872+0000","last_unstale":"2024-05-30T17:00:38.151226+0000","last_undegraded":"2024-05-30T17:00:38.151226+0000","last_fullsized":"2024-05-30T17:00:38.151226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:12:08.423568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-30T17:00:35.736077+0000","last_change":"2024-05-30T17:00:35.732840+0000","last_active":"2024-05-30T17:00:35.736077+0000","last_peered":"2024-05-30T17:00:35.736077+0000","last_clean":"2024-05-30T17:00:35.736077+0000","last_became_active":"2024-05-30T17:00:35.732525+0000","last_became_peered":"2024-05-30T17:00:35.732525+0000","last_unstale":"2024-05-30T17:00:35.736077+0000","last_undegraded":"2024-05-30T17:00:35.736077+0000","last_fullsized":"2024-05-30T17:00:35.736077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:33:29.507352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-30T17:00:35.720954+0000","last_change":"2024-05-30T17:00:35.720954+0000","last_active":"2024-05-30T17:00:35.687353+0000","last_peered":"2024-05-30T17:00:35.687353+0000","last_clean":"2024-05-30T17:00:35.687353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:00:35.720954+0000","last_undegraded":"2024-05-30T17:00:35.720954+0000","last_fullsized":"2024-05-30T17:00:35.720954+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149819+0000","last_change":"2024-05-30T17:00:35.729629+0000","last_active":"2024-05-30T17:00:38.149819+0000","last_peered":"2024-05-30T17:00:38.149819+0000","last_clean":"2024-05-30T17:00:38.149819+0000","last_became_active":"2024-05-30T17:00:35.729282+0000","last_became_peered":"2024-05-30T17:00:35.729282+0000","last_unstale":"2024-05-30T17:00:38.149819+0000","last_undegraded":"2024-05-30T17:00:38.149819+0000","last_fullsized":"2024-05-30T17:00:38.149819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:43.450197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":330,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148934+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:00:38.148934+0000","last_peered":"2024-05-30T17:00:38.148934+0000","last_clean":"2024-05-30T17:00:38.148934+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:00:38.148934+0000","last_undegraded":"2024-05-30T17:00:38.148934+0000","last_fullsized":"2024-05-30T17:00:38.148934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509647,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130936,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240904,"statfs":{"total":96636764160,"available":96502685696,"internally_reserved":0,"allocated":1286144,"data_stored":558361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509647,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86528,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285312,"statfs":{"total":96636764160,"available":96548159488,"internally_reserved":0,"allocated":1298432,"data_stored":559445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509647,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126728,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245112,"statfs":{"total":96636764160,"available":96506994688,"internally_reserved":0,"allocated":1241088,"data_stored":538107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509647,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144792,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227048,"statfs":{"total":96636764160,"available":96488497152,"internally_reserved":0,"allocated":1253376,"data_stored":544466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509646,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126732,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245108,"statfs":{"total":96636764160,"available":96506990592,"internally_reserved":0,"allocated":1241088,"data_stored":538107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542350,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127356,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244484,"statfs":{"total":96636764160,"available":96506351616,"internally_reserved":0,"allocated":1880064,"data_stored":1148189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76788,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295052,"statfs":{"total":96636764160,"available":96558133248,"internally_reserved":0,"allocated":1282048,"data_stored":557779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542351,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69044,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302796,"statfs":{"total":96636764160,"available":96566063104,"internally_reserved":0,"allocated":1875968,"data_stored":1148659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:00:42.263 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3f is not active+clean 2024-05-30T17:00:42.263 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:00:35.687353+0000', 'last_change': '2024-05-30T17:00:35.687353+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.687353+0000', 'last_undegraded': '2024-05-30T17:00:35.687353+0000', 'last_fullsized': '2024-05-30T17:00:35.687353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3d is not active+clean 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:00:35.687353+0000', 'last_change': '2024-05-30T17:00:35.687353+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.687353+0000', 'last_undegraded': '2024-05-30T17:00:35.687353+0000', 'last_fullsized': '2024-05-30T17:00:35.687353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3c is not active+clean 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.712512+0000', 'last_change': '2024-05-30T17:00:35.712512+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.712512+0000', 'last_undegraded': '2024-05-30T17:00:35.712512+0000', 'last_fullsized': '2024-05-30T17:00:35.712512+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:PG 15.33 is not active+clean 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.712050+0000', 'last_change': '2024-05-30T17:00:35.712050+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.712050+0000', 'last_undegraded': '2024-05-30T17:00:35.712050+0000', 'last_fullsized': '2024-05-30T17:00:35.712050+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:PG 15.32 is not active+clean 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.721668+0000', 'last_change': '2024-05-30T17:00:35.721668+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.721668+0000', 'last_undegraded': '2024-05-30T17:00:35.721668+0000', 'last_fullsized': '2024-05-30T17:00:35.721668+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:PG 15.31 is not active+clean 2024-05-30T17:00:42.264 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.711828+0000', 'last_change': '2024-05-30T17:00:35.711828+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.711828+0000', 'last_undegraded': '2024-05-30T17:00:35.711828+0000', 'last_fullsized': '2024-05-30T17:00:35.711828+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2e is not active+clean 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.710227+0000', 'last_change': '2024-05-30T17:00:35.710227+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.710227+0000', 'last_undegraded': '2024-05-30T17:00:35.710227+0000', 'last_fullsized': '2024-05-30T17:00:35.710227+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2d is not active+clean 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.719035+0000', 'last_change': '2024-05-30T17:00:35.719035+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.719035+0000', 'last_undegraded': '2024-05-30T17:00:35.719035+0000', 'last_fullsized': '2024-05-30T17:00:35.719035+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2b is not active+clean 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.720502+0000', 'last_change': '2024-05-30T17:00:35.720502+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.720502+0000', 'last_undegraded': '2024-05-30T17:00:35.720502+0000', 'last_fullsized': '2024-05-30T17:00:35.720502+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:PG 15.22 is not active+clean 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.721256+0000', 'last_change': '2024-05-30T17:00:35.721256+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.721256+0000', 'last_undegraded': '2024-05-30T17:00:35.721256+0000', 'last_fullsized': '2024-05-30T17:00:35.721256+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:PG 15.21 is not active+clean 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.710277+0000', 'last_change': '2024-05-30T17:00:35.710277+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.710277+0000', 'last_undegraded': '2024-05-30T17:00:35.710277+0000', 'last_fullsized': '2024-05-30T17:00:35.710277+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:PG 15.20 is not active+clean 2024-05-30T17:00:42.265 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:00:35.687353+0000', 'last_change': '2024-05-30T17:00:35.687353+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.687353+0000', 'last_undegraded': '2024-05-30T17:00:35.687353+0000', 'last_fullsized': '2024-05-30T17:00:35.687353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1e is not active+clean 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.726080+0000', 'last_change': '2024-05-30T17:00:35.726080+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.726080+0000', 'last_undegraded': '2024-05-30T17:00:35.726080+0000', 'last_fullsized': '2024-05-30T17:00:35.726080+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1f is not active+clean 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.711876+0000', 'last_change': '2024-05-30T17:00:35.711876+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.711876+0000', 'last_undegraded': '2024-05-30T17:00:35.711876+0000', 'last_fullsized': '2024-05-30T17:00:35.711876+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1c is not active+clean 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:00:35.687353+0000', 'last_change': '2024-05-30T17:00:35.687353+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.687353+0000', 'last_undegraded': '2024-05-30T17:00:35.687353+0000', 'last_fullsized': '2024-05-30T17:00:35.687353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1b is not active+clean 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:34.699719+0000', 'last_change': '2024-05-30T17:00:34.699719+0000', 'last_active': '2024-05-30T17:00:34.674517+0000', 'last_peered': '2024-05-30T17:00:34.674517+0000', 'last_clean': '2024-05-30T17:00:34.674517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:34.699719+0000', 'last_undegraded': '2024-05-30T17:00:34.699719+0000', 'last_fullsized': '2024-05-30T17:00:34.699719+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:PG 15.18 is not active+clean 2024-05-30T17:00:42.266 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.710078+0000', 'last_change': '2024-05-30T17:00:35.710078+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.710078+0000', 'last_undegraded': '2024-05-30T17:00:35.710078+0000', 'last_fullsized': '2024-05-30T17:00:35.710078+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:PG 14.18 is not active+clean 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:34.703036+0000', 'last_change': '2024-05-30T17:00:34.703036+0000', 'last_active': '2024-05-30T17:00:34.674517+0000', 'last_peered': '2024-05-30T17:00:34.674517+0000', 'last_clean': '2024-05-30T17:00:34.674517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:34.703036+0000', 'last_undegraded': '2024-05-30T17:00:34.703036+0000', 'last_fullsized': '2024-05-30T17:00:34.703036+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:PG 14.17 is not active+clean 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:34.699191+0000', 'last_change': '2024-05-30T17:00:34.699191+0000', 'last_active': '2024-05-30T17:00:34.674517+0000', 'last_peered': '2024-05-30T17:00:34.674517+0000', 'last_clean': '2024-05-30T17:00:34.674517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:34.699191+0000', 'last_undegraded': '2024-05-30T17:00:34.699191+0000', 'last_fullsized': '2024-05-30T17:00:34.699191+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:PG 15.12 is not active+clean 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.708596+0000', 'last_change': '2024-05-30T17:00:35.708596+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.708596+0000', 'last_undegraded': '2024-05-30T17:00:35.708596+0000', 'last_fullsized': '2024-05-30T17:00:35.708596+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:PG 15.13 is not active+clean 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:00:35.687353+0000', 'last_change': '2024-05-30T17:00:35.687353+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.687353+0000', 'last_undegraded': '2024-05-30T17:00:35.687353+0000', 'last_fullsized': '2024-05-30T17:00:35.687353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:PG 15.11 is not active+clean 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.716683+0000', 'last_change': '2024-05-30T17:00:35.716683+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.716683+0000', 'last_undegraded': '2024-05-30T17:00:35.716683+0000', 'last_fullsized': '2024-05-30T17:00:35.716683+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:PG 14.10 is not active+clean 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:34.699431+0000', 'last_change': '2024-05-30T17:00:34.699431+0000', 'last_active': '2024-05-30T17:00:34.674517+0000', 'last_peered': '2024-05-30T17:00:34.674517+0000', 'last_clean': '2024-05-30T17:00:34.674517+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:34.699431+0000', 'last_undegraded': '2024-05-30T17:00:34.699431+0000', 'last_fullsized': '2024-05-30T17:00:34.699431+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:34.674517+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:PG 15.a is not active+clean 2024-05-30T17:00:42.267 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.720831+0000', 'last_change': '2024-05-30T17:00:35.720831+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.720831+0000', 'last_undegraded': '2024-05-30T17:00:35.720831+0000', 'last_fullsized': '2024-05-30T17:00:35.720831+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:PG 15.b is not active+clean 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.719960+0000', 'last_change': '2024-05-30T17:00:35.719960+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.719960+0000', 'last_undegraded': '2024-05-30T17:00:35.719960+0000', 'last_fullsized': '2024-05-30T17:00:35.719960+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:PG 15.8 is not active+clean 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.717200+0000', 'last_change': '2024-05-30T17:00:35.717200+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.717200+0000', 'last_undegraded': '2024-05-30T17:00:35.717200+0000', 'last_fullsized': '2024-05-30T17:00:35.717200+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:PG 15.7 is not active+clean 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:00:35.687353+0000', 'last_change': '2024-05-30T17:00:35.687353+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.687353+0000', 'last_undegraded': '2024-05-30T17:00:35.687353+0000', 'last_fullsized': '2024-05-30T17:00:35.687353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:PG 15.4 is not active+clean 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:00:35.687353+0000', 'last_change': '2024-05-30T17:00:35.687353+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.687353+0000', 'last_undegraded': '2024-05-30T17:00:35.687353+0000', 'last_fullsized': '2024-05-30T17:00:35.687353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2 is not active+clean 2024-05-30T17:00:42.268 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.720407+0000', 'last_change': '2024-05-30T17:00:35.720407+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.720407+0000', 'last_undegraded': '2024-05-30T17:00:35.720407+0000', 'last_fullsized': '2024-05-30T17:00:35.720407+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.269 INFO:tasks.ceph.ceph_manager.ceph:PG 15.0 is not active+clean 2024-05-30T17:00:42.269 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.716687+0000', 'last_change': '2024-05-30T17:00:35.716687+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.716687+0000', 'last_undegraded': '2024-05-30T17:00:35.716687+0000', 'last_fullsized': '2024-05-30T17:00:35.716687+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.269 INFO:tasks.ceph.ceph_manager.ceph:PG 15.e is not active+clean 2024-05-30T17:00:42.269 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:00:35.720954+0000', 'last_change': '2024-05-30T17:00:35.720954+0000', 'last_active': '2024-05-30T17:00:35.687353+0000', 'last_peered': '2024-05-30T17:00:35.687353+0000', 'last_clean': '2024-05-30T17:00:35.687353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:00:35.720954+0000', 'last_undegraded': '2024-05-30T17:00:35.720954+0000', 'last_fullsized': '2024-05-30T17:00:35.720954+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'last_clean_scrub_stamp': '2024-05-30T17:00:35.687353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-30T17:00:42.269 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-30T17:00:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.514+0000 7f62c8600640 1 Processor -- start 2024-05-30T17:00:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.514+0000 7f62c8600640 1 -- start start 2024-05-30T17:00:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.516+0000 7f62c8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c015baa0 0x7f62c00d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.516+0000 7f62c8600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62c006f390 0x7f62c006f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.516+0000 7f62c8600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62c006e3d0 con 0x7f62c0155a00 2024-05-30T17:00:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.516+0000 7f62c8600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f62c00ccf20 con 0x7f62c006f390 2024-05-30T17:00:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.516+0000 7f62c8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f62c00d0490 con 0x7f62c015baa0 2024-05-30T17:00:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.516+0000 7f62c6600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62c006f390 0x7f62c006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.516+0000 7f62c6600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62c006f390 0x7f62c006f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53164/0 (socket says 172.21.15.67:53164) 2024-05-30T17:00:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.516+0000 7f62c6600640 1 -- 172.21.15.67:0/1169392619 learned_addr learned my addr 172.21.15.67:0/1169392619 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.517+0000 7f62c5200640 1 --2- 172.21.15.67:0/1169392619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c015baa0 0x7f62c00d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.517+0000 7f62c6600640 1 -- 172.21.15.67:0/1169392619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c015baa0 msgr2=0x7f62c00d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.517+0000 7f62c6600640 1 --2- 172.21.15.67:0/1169392619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c015baa0 0x7f62c00d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.517+0000 7f62c6600640 1 -- 172.21.15.67:0/1169392619 >> v1:172.21.15.169:6789/0 conn(0x7f62c0155a00 legacy=0x7f62c006dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.517+0000 7f62c6600640 1 -- 172.21.15.67:0/1169392619 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c015c0b0 con 0x7f62c006f390 2024-05-30T17:00:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.517+0000 7f62c5200640 1 --2- 172.21.15.67:0/1169392619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c015baa0 0x7f62c00d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:42.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.518+0000 7f62c6600640 1 --2- 172.21.15.67:0/1169392619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c006f390 0x7f62c006f820 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009e50 tx=0x7f62b802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50af2a48f5497e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:42.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.518+0000 7f62bfe00640 1 -- 172.21.15.67:0/1169392619 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8036070 con 0x7f62c006f390 2024-05-30T17:00:42.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.518+0000 7f62bfe00640 1 -- 172.21.15.67:0/1169392619 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b8031600 con 0x7f62c006f390 2024-05-30T17:00:42.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.518+0000 7f62bfe00640 1 -- 172.21.15.67:0/1169392619 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b80319d0 con 0x7f62c006f390 2024-05-30T17:00:42.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.519+0000 7f62c8600640 1 -- 172.21.15.67:0/1169392619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c006f390 msgr2=0x7f62c006f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:42.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.519+0000 7f62c8600640 1 --2- 172.21.15.67:0/1169392619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c006f390 0x7f62c006f820 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009e50 tx=0x7f62b802a6c0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.519+0000 7f62c8600640 1 -- 172.21.15.67:0/1169392619 shutdown_connections 2024-05-30T17:00:42.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.519+0000 7f62c8600640 1 --2- 172.21.15.67:0/1169392619 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c006f390 0x7f62c006f820 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.519+0000 7f62c8600640 1 --2- 172.21.15.67:0/1169392619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c015baa0 0x7f62c00d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.519+0000 7f62c8600640 1 -- 172.21.15.67:0/1169392619 >> 172.21.15.67:0/1169392619 conn(0x7f62c0067ce0 msgr2=0x7f62c006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:42.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.519+0000 7f62c8600640 1 -- 172.21.15.67:0/1169392619 shutdown_connections 2024-05-30T17:00:42.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.519+0000 7f62c8600640 1 -- 172.21.15.67:0/1169392619 wait complete. 2024-05-30T17:00:42.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.521+0000 7f62c8600640 1 Processor -- start 2024-05-30T17:00:42.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.521+0000 7f62c8600640 1 -- start start 2024-05-30T17:00:42.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.522+0000 7f62c8600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c006f390 0x7f62c0151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.522+0000 7f62c8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c0155a00 0x7f62c0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.522+0000 7f62c8600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c015baa0 0x7f62c0072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.522+0000 7f62c8600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f62c015c950 con 0x7f62c015baa0 2024-05-30T17:00:42.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.522+0000 7f62c8600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f62c015c670 con 0x7f62c006f390 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.522+0000 7f62c8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f62c015f090 con 0x7f62c0155a00 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.522+0000 7f62c6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c015baa0 0x7f62c0072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.522+0000 7f62c6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c015baa0 0x7f62c0072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53168/0 (socket says 172.21.15.67:53168) 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.522+0000 7f62c6600640 1 -- 172.21.15.67:0/302318660 learned_addr learned my addr 172.21.15.67:0/302318660 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62c6600640 1 -- 172.21.15.67:0/302318660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c0155a00 msgr2=0x7f62c0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62c6600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c0155a00 0x7f62c0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62c6600640 1 -- 172.21.15.67:0/302318660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c006f390 msgr2=0x7f62c0151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62c5c00640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c006f390 0x7f62c0151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62c6600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c006f390 0x7f62c0151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62c6600640 1 -- 172.21.15.67:0/302318660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c0157e90 con 0x7f62c015baa0 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62c6600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c015baa0 0x7f62c0072200 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009f70 tx=0x7f62b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62bd600640 1 -- 172.21.15.67:0/302318660 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8036050 con 0x7f62c015baa0 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62bd600640 1 -- 172.21.15.67:0/302318660 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b8042070 con 0x7f62c015baa0 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.523+0000 7f62bd600640 1 -- 172.21.15.67:0/302318660 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8009510 con 0x7f62c015baa0 2024-05-30T17:00:42.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.524+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c0158130 con 0x7f62c015baa0 2024-05-30T17:00:42.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.524+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c01586f0 con 0x7f62c015baa0 2024-05-30T17:00:42.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.525+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c00689e0 con 0x7f62c015baa0 2024-05-30T17:00:42.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.526+0000 7f62bd600640 1 -- 172.21.15.67:0/302318660 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f62b8007bd0 con 0x7f62c015baa0 2024-05-30T17:00:42.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.527+0000 7f62bd600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f62a006ddf0 0x7f62a00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:42.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.527+0000 7f62bd600640 1 -- 172.21.15.67:0/302318660 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f62b80ab770 con 0x7f62c015baa0 2024-05-30T17:00:42.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.527+0000 7f62c5c00640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f62a006ddf0 0x7f62a00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:42.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.528+0000 7f62c5c00640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f62a006ddf0 0x7f62a00702b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f62c0072920 tx=0x7f62b4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:42.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.531+0000 7f62bd600640 1 -- 172.21.15.67:0/302318660 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62b8045360 con 0x7f62c015baa0 2024-05-30T17:00:42.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.815+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f62c006f170 con 0x7f62c015baa0 2024-05-30T17:00:42.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.815+0000 7f62bd600640 1 -- 172.21.15.67:0/302318660 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1016 (secure 0 0 0) 0x7f62b807b650 con 0x7f62c015baa0 2024-05-30T17:00:42.816 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:42.817 INFO:teuthology.orchestra.run.smithi067.stdout:{"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1036,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1717087419,"num_in_osds":8,"osd_in_since":1717087410,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":910356480,"bytes_avail":772183756800,"bytes_total":773094113280,"write_bytes_sec":2115,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":64,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7902}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-05-30T16:59:40.449642+0000","services":{}},"progress_events":{}} 2024-05-30T17:00:42.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f62a006ddf0 msgr2=0x7f62a00702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:42.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f62a006ddf0 0x7f62a00702b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f62c0072920 tx=0x7f62b4009040 comp rx=0 tx=0).stop 2024-05-30T17:00:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c015baa0 msgr2=0x7f62c0072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c015baa0 0x7f62c0072200 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009f70 tx=0x7f62b8004a40 comp rx=0 tx=0).stop 2024-05-30T17:00:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 shutdown_connections 2024-05-30T17:00:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f62a006ddf0 0x7f62a00702b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62c015baa0 0x7f62c0072200 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62c0155a00 0x7f62c0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 --2- 172.21.15.67:0/302318660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62c006f390 0x7f62c0151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 >> 172.21.15.67:0/302318660 conn(0x7f62c0067ce0 msgr2=0x7f62c0155080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.818+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 shutdown_connections 2024-05-30T17:00:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:42.819+0000 7f62c8600640 1 -- 172.21.15.67:0/302318660 wait complete. 2024-05-30T17:00:42.837 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-30T17:00:42.838 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:00:43.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.048+0000 7f3a43e00640 1 Processor -- start 2024-05-30T17:00:43.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.048+0000 7f3a43e00640 1 -- start start 2024-05-30T17:00:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.050+0000 7f3a43e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a44158b80 0x7f3a4406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.050+0000 7f3a43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a44071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.050+0000 7f3a43e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a4406e6e0 con 0x7f3a44072100 2024-05-30T17:00:43.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.050+0000 7f3a43e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3a440b42d0 con 0x7f3a44158b80 2024-05-30T17:00:43.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.050+0000 7f3a43e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3a4406e9c0 con 0x7f3a4406e160 2024-05-30T17:00:43.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.051+0000 7f3a42a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a44158b80 0x7f3a4406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:43.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.051+0000 7f3a3a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a44071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.051+0000 7f3a42a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a44158b80 0x7f3a4406dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:43506/0 (socket says 172.21.15.67:43506) 2024-05-30T17:00:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.051+0000 7f3a42a00640 1 -- 172.21.15.67:0/988436665 learned_addr learned my addr 172.21.15.67:0/988436665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.052+0000 7f3a3a000640 1 -- 172.21.15.67:0/988436665 >> v2:172.21.15.169:3300/0 conn(0x7f3a44158b80 msgr2=0x7f3a4406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.052+0000 7f3a3a000640 1 --2- 172.21.15.67:0/988436665 >> v2:172.21.15.169:3300/0 conn(0x7f3a44158b80 0x7f3a4406dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.052+0000 7f3a3a000640 1 -- 172.21.15.67:0/988436665 >> v1:172.21.15.169:6789/0 conn(0x7f3a44072100 legacy=0x7f3a441621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.052+0000 7f3a3a000640 1 -- 172.21.15.67:0/988436665 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4406eca0 con 0x7f3a4406e160 2024-05-30T17:00:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.052+0000 7f3a42a00640 1 --2- 172.21.15.67:0/988436665 >> v2:172.21.15.169:3300/0 conn(0x7f3a44158b80 0x7f3a4406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:00:43.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.053+0000 7f3a3a000640 1 --2- 172.21.15.67:0/988436665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a44071bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0097a0 tx=0x7f3a2c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab248963aae05e00 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:43.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a42000640 1 -- 172.21.15.67:0/988436665 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c037070 con 0x7f3a4406e160 2024-05-30T17:00:43.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a42000640 1 -- 172.21.15.67:0/988436665 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2c032440 con 0x7f3a4406e160 2024-05-30T17:00:43.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a42000640 1 -- 172.21.15.67:0/988436665 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c032810 con 0x7f3a4406e160 2024-05-30T17:00:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a43e00640 1 -- 172.21.15.67:0/988436665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 msgr2=0x7f3a44071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a43e00640 1 --2- 172.21.15.67:0/988436665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a44071bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0097a0 tx=0x7f3a2c02a180 comp rx=0 tx=0).stop 2024-05-30T17:00:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a43e00640 1 -- 172.21.15.67:0/988436665 shutdown_connections 2024-05-30T17:00:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a43e00640 1 --2- 172.21.15.67:0/988436665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a44071bc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a43e00640 1 --2- 172.21.15.67:0/988436665 >> v2:172.21.15.169:3300/0 conn(0x7f3a44158b80 0x7f3a4406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a43e00640 1 -- 172.21.15.67:0/988436665 >> 172.21.15.67:0/988436665 conn(0x7f3a440b1fc0 msgr2=0x7f3a440b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.054+0000 7f3a43e00640 1 -- 172.21.15.67:0/988436665 shutdown_connections 2024-05-30T17:00:43.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.055+0000 7f3a43e00640 1 -- 172.21.15.67:0/988436665 wait complete. 2024-05-30T17:00:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.056+0000 7f3a43e00640 1 Processor -- start 2024-05-30T17:00:43.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.056+0000 7f3a43e00640 1 -- start start 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.057+0000 7f3a43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a4409a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.057+0000 7f3a43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a44072100 0x7f3a441f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.057+0000 7f3a43e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a44158b80 0x7f3a441504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.057+0000 7f3a43e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3a440b2d30 con 0x7f3a44072100 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.057+0000 7f3a43e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4406f260 con 0x7f3a44158b80 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.057+0000 7f3a43e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3a4406f780 con 0x7f3a4406e160 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a3a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a44072100 0x7f3a441f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a3a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a44072100 0x7f3a441f4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53182/0 (socket says 172.21.15.67:53182) 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a3a000640 1 -- 172.21.15.67:0/891314344 learned_addr learned my addr 172.21.15.67:0/891314344 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a42a00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a4409a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:43.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a43400640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a44158b80 0x7f3a441504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a3a000640 1 -- 172.21.15.67:0/891314344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 msgr2=0x7f3a4409a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a3a000640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a4409a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a3a000640 1 -- 172.21.15.67:0/891314344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a44158b80 msgr2=0x7f3a441504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a3a000640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a44158b80 0x7f3a441504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a3a000640 1 -- 172.21.15.67:0/891314344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a441509e0 con 0x7f3a44072100 2024-05-30T17:00:43.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.058+0000 7f3a42a00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a4409a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.059+0000 7f3a3a000640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a44072100 0x7f3a441f4160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c02a1c0 tx=0x7f3a2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.059+0000 7f3a3b400640 1 -- 172.21.15.67:0/891314344 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c041020 con 0x7f3a44072100 2024-05-30T17:00:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.059+0000 7f3a3b400640 1 -- 172.21.15.67:0/891314344 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2c030700 con 0x7f3a44072100 2024-05-30T17:00:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.059+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a441f48d0 con 0x7f3a44072100 2024-05-30T17:00:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.059+0000 7f3a3b400640 1 -- 172.21.15.67:0/891314344 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c037040 con 0x7f3a44072100 2024-05-30T17:00:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.059+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a441f4e90 con 0x7f3a44072100 2024-05-30T17:00:43.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.060+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a441f50d0 con 0x7f3a44072100 2024-05-30T17:00:43.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.061+0000 7f3a3b400640 1 -- 172.21.15.67:0/891314344 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f3a2c044020 con 0x7f3a44072100 2024-05-30T17:00:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.063+0000 7f3a3b400640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a2006ddc0 0x7f3a20070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.063+0000 7f3a3b400640 1 -- 172.21.15.67:0/891314344 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f3a2c0ab070 con 0x7f3a44072100 2024-05-30T17:00:43.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.063+0000 7f3a42a00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a2006ddc0 0x7f3a20070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:43.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.064+0000 7f3a42a00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a2006ddc0 0x7f3a20070280 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3a44150dc0 tx=0x7f3a34008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:43.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.068+0000 7f3a3b400640 1 -- 172.21.15.67:0/891314344 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a441f50d0 con 0x7f3a44072100 2024-05-30T17:00:43.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.266+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a4409e3d0 con 0x7f3a2006ddc0 2024-05-30T17:00:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.286+0000 7f3a3b400640 1 -- 172.21.15.67:0/891314344 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252502 (secure 0 0 0) 0x7f3a4409e3d0 con 0x7f3a2006ddc0 2024-05-30T17:00:43.290 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:00:43.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.292+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a2006ddc0 msgr2=0x7f3a20070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:43.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.292+0000 7f3a43e00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a2006ddc0 0x7f3a20070280 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3a44150dc0 tx=0x7f3a34008040 comp rx=0 tx=0).stop 2024-05-30T17:00:43.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.292+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a44072100 msgr2=0x7f3a441f4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:43.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.292+0000 7f3a43e00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a44072100 0x7f3a441f4160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c02a1c0 tx=0x7f3a2c0046c0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.293+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 shutdown_connections 2024-05-30T17:00:43.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.293+0000 7f3a43e00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3a2006ddc0 0x7f3a20070280 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.293+0000 7f3a43e00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a44158b80 0x7f3a441504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.293+0000 7f3a43e00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a44072100 0x7f3a441f4160 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.293+0000 7f3a43e00640 1 --2- 172.21.15.67:0/891314344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a4406e160 0x7f3a4409a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:43.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.293+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 >> 172.21.15.67:0/891314344 conn(0x7f3a440b1fc0 msgr2=0x7f3a441513b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:43.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.293+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 shutdown_connections 2024-05-30T17:00:43.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:43.293+0000 7f3a43e00640 1 -- 172.21.15.67:0/891314344 wait complete. 2024-05-30T17:00:43.315 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":609,"stamp":"2024-05-30T17:00:42.472896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":889020,"kb_used_data":11208,"kb_used_omap":12,"kb_used_meta":877683,"kb_avail":754085700,"statfs":{"total":773094113280,"available":772183756800,"internally_reserved":0,"allocated":11476992,"data_stored":5647228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":898747952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.777002"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149145+0000","last_change":"2024-05-30T17:00:36.759629+0000","last_active":"2024-05-30T17:00:38.149145+0000","last_peered":"2024-05-30T17:00:38.149145+0000","last_clean":"2024-05-30T17:00:38.149145+0000","last_became_active":"2024-05-30T17:00:36.759308+0000","last_became_peered":"2024-05-30T17:00:36.759308+0000","last_unstale":"2024-05-30T17:00:38.149145+0000","last_undegraded":"2024-05-30T17:00:38.149145+0000","last_fullsized":"2024-05-30T17:00:38.149145+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:09:42.812288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752224+0000","last_change":"2024-05-30T17:00:36.748181+0000","last_active":"2024-05-30T17:00:37.752224+0000","last_peered":"2024-05-30T17:00:37.752224+0000","last_clean":"2024-05-30T17:00:37.752224+0000","last_became_active":"2024-05-30T17:00:36.747392+0000","last_became_peered":"2024-05-30T17:00:36.747392+0000","last_unstale":"2024-05-30T17:00:37.752224+0000","last_undegraded":"2024-05-30T17:00:37.752224+0000","last_fullsized":"2024-05-30T17:00:37.752224+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149147+0000","last_change":"2024-05-30T17:00:36.764900+0000","last_active":"2024-05-30T17:00:38.149147+0000","last_peered":"2024-05-30T17:00:38.149147+0000","last_clean":"2024-05-30T17:00:38.149147+0000","last_became_active":"2024-05-30T17:00:36.764614+0000","last_became_peered":"2024-05-30T17:00:36.764614+0000","last_unstale":"2024-05-30T17:00:38.149147+0000","last_undegraded":"2024-05-30T17:00:38.149147+0000","last_fullsized":"2024-05-30T17:00:38.149147+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:41:02.666320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151034+0000","last_change":"2024-05-30T17:00:36.757850+0000","last_active":"2024-05-30T17:00:38.151034+0000","last_peered":"2024-05-30T17:00:38.151034+0000","last_clean":"2024-05-30T17:00:38.151034+0000","last_became_active":"2024-05-30T17:00:36.757545+0000","last_became_peered":"2024-05-30T17:00:36.757545+0000","last_unstale":"2024-05-30T17:00:38.151034+0000","last_undegraded":"2024-05-30T17:00:38.151034+0000","last_fullsized":"2024-05-30T17:00:38.151034+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:35:06.226926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151835+0000","last_change":"2024-05-30T17:00:36.763111+0000","last_active":"2024-05-30T17:00:38.151835+0000","last_peered":"2024-05-30T17:00:38.151835+0000","last_clean":"2024-05-30T17:00:38.151835+0000","last_became_active":"2024-05-30T17:00:36.761404+0000","last_became_peered":"2024-05-30T17:00:36.761404+0000","last_unstale":"2024-05-30T17:00:38.151835+0000","last_undegraded":"2024-05-30T17:00:38.151835+0000","last_fullsized":"2024-05-30T17:00:38.151835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:01.455423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149546+0000","last_change":"2024-05-30T17:00:36.751908+0000","last_active":"2024-05-30T17:00:38.149546+0000","last_peered":"2024-05-30T17:00:38.149546+0000","last_clean":"2024-05-30T17:00:38.149546+0000","last_became_active":"2024-05-30T17:00:36.751604+0000","last_became_peered":"2024-05-30T17:00:36.751604+0000","last_unstale":"2024-05-30T17:00:38.149546+0000","last_undegraded":"2024-05-30T17:00:38.149546+0000","last_fullsized":"2024-05-30T17:00:38.149546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149447+0000","last_change":"2024-05-30T17:00:36.748294+0000","last_active":"2024-05-30T17:00:38.149447+0000","last_peered":"2024-05-30T17:00:38.149447+0000","last_clean":"2024-05-30T17:00:38.149447+0000","last_became_active":"2024-05-30T17:00:36.747984+0000","last_became_peered":"2024-05-30T17:00:36.747984+0000","last_unstale":"2024-05-30T17:00:38.149447+0000","last_undegraded":"2024-05-30T17:00:38.149447+0000","last_fullsized":"2024-05-30T17:00:38.149447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751747+0000","last_change":"2024-05-30T17:00:36.751171+0000","last_active":"2024-05-30T17:00:37.751747+0000","last_peered":"2024-05-30T17:00:37.751747+0000","last_clean":"2024-05-30T17:00:37.751747+0000","last_became_active":"2024-05-30T17:00:36.747956+0000","last_became_peered":"2024-05-30T17:00:36.747956+0000","last_unstale":"2024-05-30T17:00:37.751747+0000","last_undegraded":"2024-05-30T17:00:37.751747+0000","last_fullsized":"2024-05-30T17:00:37.751747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:03:57.450595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760121+0000","last_change":"2024-05-30T17:00:36.762682+0000","last_active":"2024-05-30T17:00:37.760121+0000","last_peered":"2024-05-30T17:00:37.760121+0000","last_clean":"2024-05-30T17:00:37.760121+0000","last_became_active":"2024-05-30T17:00:36.762190+0000","last_became_peered":"2024-05-30T17:00:36.762190+0000","last_unstale":"2024-05-30T17:00:37.760121+0000","last_undegraded":"2024-05-30T17:00:37.760121+0000","last_fullsized":"2024-05-30T17:00:37.760121+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:33.645864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151899+0000","last_change":"2024-05-30T17:00:36.758850+0000","last_active":"2024-05-30T17:00:38.151899+0000","last_peered":"2024-05-30T17:00:38.151899+0000","last_clean":"2024-05-30T17:00:38.151899+0000","last_became_active":"2024-05-30T17:00:36.758522+0000","last_became_peered":"2024-05-30T17:00:36.758522+0000","last_unstale":"2024-05-30T17:00:38.151899+0000","last_undegraded":"2024-05-30T17:00:38.151899+0000","last_fullsized":"2024-05-30T17:00:38.151899+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:20.247961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149815+0000","last_change":"2024-05-30T17:00:36.750772+0000","last_active":"2024-05-30T17:00:38.149815+0000","last_peered":"2024-05-30T17:00:38.149815+0000","last_clean":"2024-05-30T17:00:38.149815+0000","last_became_active":"2024-05-30T17:00:36.750122+0000","last_became_peered":"2024-05-30T17:00:36.750122+0000","last_unstale":"2024-05-30T17:00:38.149815+0000","last_undegraded":"2024-05-30T17:00:38.149815+0000","last_fullsized":"2024-05-30T17:00:38.149815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761029+0000","last_change":"2024-05-30T17:00:36.751475+0000","last_active":"2024-05-30T17:00:37.761029+0000","last_peered":"2024-05-30T17:00:37.761029+0000","last_clean":"2024-05-30T17:00:37.761029+0000","last_became_active":"2024-05-30T17:00:36.751150+0000","last_became_peered":"2024-05-30T17:00:36.751150+0000","last_unstale":"2024-05-30T17:00:37.761029+0000","last_undegraded":"2024-05-30T17:00:37.761029+0000","last_fullsized":"2024-05-30T17:00:37.761029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:55:05.659729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150357+0000","last_change":"2024-05-30T17:00:36.755993+0000","last_active":"2024-05-30T17:00:38.150357+0000","last_peered":"2024-05-30T17:00:38.150357+0000","last_clean":"2024-05-30T17:00:38.150357+0000","last_became_active":"2024-05-30T17:00:36.755651+0000","last_became_peered":"2024-05-30T17:00:36.755651+0000","last_unstale":"2024-05-30T17:00:38.150357+0000","last_undegraded":"2024-05-30T17:00:38.150357+0000","last_fullsized":"2024-05-30T17:00:38.150357+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:37:32.519686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755319+0000","last_change":"2024-05-30T17:00:36.759921+0000","last_active":"2024-05-30T17:00:37.755319+0000","last_peered":"2024-05-30T17:00:37.755319+0000","last_clean":"2024-05-30T17:00:37.755319+0000","last_became_active":"2024-05-30T17:00:36.759644+0000","last_became_peered":"2024-05-30T17:00:36.759644+0000","last_unstale":"2024-05-30T17:00:37.755319+0000","last_undegraded":"2024-05-30T17:00:37.755319+0000","last_fullsized":"2024-05-30T17:00:37.755319+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:24:23.085972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150002+0000","last_change":"2024-05-30T17:00:36.765066+0000","last_active":"2024-05-30T17:00:38.150002+0000","last_peered":"2024-05-30T17:00:38.150002+0000","last_clean":"2024-05-30T17:00:38.150002+0000","last_became_active":"2024-05-30T17:00:36.764709+0000","last_became_peered":"2024-05-30T17:00:36.764709+0000","last_unstale":"2024-05-30T17:00:38.150002+0000","last_undegraded":"2024-05-30T17:00:38.150002+0000","last_fullsized":"2024-05-30T17:00:38.150002+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:03:46.211043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149705+0000","last_change":"2024-05-30T17:00:36.758779+0000","last_active":"2024-05-30T17:00:38.149705+0000","last_peered":"2024-05-30T17:00:38.149705+0000","last_clean":"2024-05-30T17:00:38.149705+0000","last_became_active":"2024-05-30T17:00:36.758402+0000","last_became_peered":"2024-05-30T17:00:36.758402+0000","last_unstale":"2024-05-30T17:00:38.149705+0000","last_undegraded":"2024-05-30T17:00:38.149705+0000","last_fullsized":"2024-05-30T17:00:38.149705+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752387+0000","last_change":"2024-05-30T17:00:36.759407+0000","last_active":"2024-05-30T17:00:37.752387+0000","last_peered":"2024-05-30T17:00:37.752387+0000","last_clean":"2024-05-30T17:00:37.752387+0000","last_became_active":"2024-05-30T17:00:36.759115+0000","last_became_peered":"2024-05-30T17:00:36.759115+0000","last_unstale":"2024-05-30T17:00:37.752387+0000","last_undegraded":"2024-05-30T17:00:37.752387+0000","last_fullsized":"2024-05-30T17:00:37.752387+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150755+0000","last_change":"2024-05-30T17:00:36.761710+0000","last_active":"2024-05-30T17:00:38.150755+0000","last_peered":"2024-05-30T17:00:38.150755+0000","last_clean":"2024-05-30T17:00:38.150755+0000","last_became_active":"2024-05-30T17:00:36.761414+0000","last_became_peered":"2024-05-30T17:00:36.761414+0000","last_unstale":"2024-05-30T17:00:38.150755+0000","last_undegraded":"2024-05-30T17:00:38.150755+0000","last_fullsized":"2024-05-30T17:00:38.150755+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:23:39.211805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755722+0000","last_change":"2024-05-30T17:00:36.767174+0000","last_active":"2024-05-30T17:00:37.755722+0000","last_peered":"2024-05-30T17:00:37.755722+0000","last_clean":"2024-05-30T17:00:37.755722+0000","last_became_active":"2024-05-30T17:00:36.766897+0000","last_became_peered":"2024-05-30T17:00:36.766897+0000","last_unstale":"2024-05-30T17:00:37.755722+0000","last_undegraded":"2024-05-30T17:00:37.755722+0000","last_fullsized":"2024-05-30T17:00:37.755722+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:26:37.028697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760586+0000","last_change":"2024-05-30T17:00:36.762646+0000","last_active":"2024-05-30T17:00:37.760586+0000","last_peered":"2024-05-30T17:00:37.760586+0000","last_clean":"2024-05-30T17:00:37.760586+0000","last_became_active":"2024-05-30T17:00:36.761684+0000","last_became_peered":"2024-05-30T17:00:36.761684+0000","last_unstale":"2024-05-30T17:00:37.760586+0000","last_undegraded":"2024-05-30T17:00:37.760586+0000","last_fullsized":"2024-05-30T17:00:37.760586+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:50.133298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755501+0000","last_change":"2024-05-30T17:00:36.765740+0000","last_active":"2024-05-30T17:00:37.755501+0000","last_peered":"2024-05-30T17:00:37.755501+0000","last_clean":"2024-05-30T17:00:37.755501+0000","last_became_active":"2024-05-30T17:00:36.764041+0000","last_became_peered":"2024-05-30T17:00:36.764041+0000","last_unstale":"2024-05-30T17:00:37.755501+0000","last_undegraded":"2024-05-30T17:00:37.755501+0000","last_fullsized":"2024-05-30T17:00:37.755501+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:41:02.666320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751731+0000","last_change":"2024-05-30T17:00:36.745182+0000","last_active":"2024-05-30T17:00:37.751731+0000","last_peered":"2024-05-30T17:00:37.751731+0000","last_clean":"2024-05-30T17:00:37.751731+0000","last_became_active":"2024-05-30T17:00:36.744869+0000","last_became_peered":"2024-05-30T17:00:36.744869+0000","last_unstale":"2024-05-30T17:00:37.751731+0000","last_undegraded":"2024-05-30T17:00:37.751731+0000","last_fullsized":"2024-05-30T17:00:37.751731+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:52:34.241585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751614+0000","last_change":"2024-05-30T17:00:36.746180+0000","last_active":"2024-05-30T17:00:37.751614+0000","last_peered":"2024-05-30T17:00:37.751614+0000","last_clean":"2024-05-30T17:00:37.751614+0000","last_became_active":"2024-05-30T17:00:36.745907+0000","last_became_peered":"2024-05-30T17:00:36.745907+0000","last_unstale":"2024-05-30T17:00:37.751614+0000","last_undegraded":"2024-05-30T17:00:37.751614+0000","last_fullsized":"2024-05-30T17:00:37.751614+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:43.488502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150152+0000","last_change":"2024-05-30T17:00:36.748364+0000","last_active":"2024-05-30T17:00:38.150152+0000","last_peered":"2024-05-30T17:00:38.150152+0000","last_clean":"2024-05-30T17:00:38.150152+0000","last_became_active":"2024-05-30T17:00:36.748051+0000","last_became_peered":"2024-05-30T17:00:36.748051+0000","last_unstale":"2024-05-30T17:00:38.150152+0000","last_undegraded":"2024-05-30T17:00:38.150152+0000","last_fullsized":"2024-05-30T17:00:38.150152+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:28.418390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151939+0000","last_change":"2024-05-30T17:00:36.754727+0000","last_active":"2024-05-30T17:00:38.151939+0000","last_peered":"2024-05-30T17:00:38.151939+0000","last_clean":"2024-05-30T17:00:38.151939+0000","last_became_active":"2024-05-30T17:00:36.753257+0000","last_became_peered":"2024-05-30T17:00:36.753257+0000","last_unstale":"2024-05-30T17:00:38.151939+0000","last_undegraded":"2024-05-30T17:00:38.151939+0000","last_fullsized":"2024-05-30T17:00:38.151939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:36.271526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752343+0000","last_change":"2024-05-30T17:00:36.754136+0000","last_active":"2024-05-30T17:00:37.752343+0000","last_peered":"2024-05-30T17:00:37.752343+0000","last_clean":"2024-05-30T17:00:37.752343+0000","last_became_active":"2024-05-30T17:00:36.753838+0000","last_became_peered":"2024-05-30T17:00:36.753838+0000","last_unstale":"2024-05-30T17:00:37.752343+0000","last_undegraded":"2024-05-30T17:00:37.752343+0000","last_fullsized":"2024-05-30T17:00:37.752343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149843+0000","last_change":"2024-05-30T17:00:36.753152+0000","last_active":"2024-05-30T17:00:38.149843+0000","last_peered":"2024-05-30T17:00:38.149843+0000","last_clean":"2024-05-30T17:00:38.149843+0000","last_became_active":"2024-05-30T17:00:36.752867+0000","last_became_peered":"2024-05-30T17:00:36.752867+0000","last_unstale":"2024-05-30T17:00:38.149843+0000","last_undegraded":"2024-05-30T17:00:38.149843+0000","last_fullsized":"2024-05-30T17:00:38.149843+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:19:07.666127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152257+0000","last_change":"2024-05-30T17:00:36.761708+0000","last_active":"2024-05-30T17:00:38.152257+0000","last_peered":"2024-05-30T17:00:38.152257+0000","last_clean":"2024-05-30T17:00:38.152257+0000","last_became_active":"2024-05-30T17:00:36.760932+0000","last_became_peered":"2024-05-30T17:00:36.760932+0000","last_unstale":"2024-05-30T17:00:38.152257+0000","last_undegraded":"2024-05-30T17:00:38.152257+0000","last_fullsized":"2024-05-30T17:00:38.152257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:40.779484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752069+0000","last_change":"2024-05-30T17:00:36.755732+0000","last_active":"2024-05-30T17:00:37.752069+0000","last_peered":"2024-05-30T17:00:37.752069+0000","last_clean":"2024-05-30T17:00:37.752069+0000","last_became_active":"2024-05-30T17:00:36.755428+0000","last_became_peered":"2024-05-30T17:00:36.755428+0000","last_unstale":"2024-05-30T17:00:37.752069+0000","last_undegraded":"2024-05-30T17:00:37.752069+0000","last_fullsized":"2024-05-30T17:00:37.752069+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:14.194525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755307+0000","last_change":"2024-05-30T17:00:36.766442+0000","last_active":"2024-05-30T17:00:37.755307+0000","last_peered":"2024-05-30T17:00:37.755307+0000","last_clean":"2024-05-30T17:00:37.755307+0000","last_became_active":"2024-05-30T17:00:36.766171+0000","last_became_peered":"2024-05-30T17:00:36.766171+0000","last_unstale":"2024-05-30T17:00:37.755307+0000","last_undegraded":"2024-05-30T17:00:37.755307+0000","last_fullsized":"2024-05-30T17:00:37.755307+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:39:26.605255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150343+0000","last_change":"2024-05-30T17:00:36.759338+0000","last_active":"2024-05-30T17:00:38.150343+0000","last_peered":"2024-05-30T17:00:38.150343+0000","last_clean":"2024-05-30T17:00:38.150343+0000","last_became_active":"2024-05-30T17:00:36.759058+0000","last_became_peered":"2024-05-30T17:00:36.759058+0000","last_unstale":"2024-05-30T17:00:38.150343+0000","last_undegraded":"2024-05-30T17:00:38.150343+0000","last_fullsized":"2024-05-30T17:00:38.150343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:24:35.298988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148191+0000","last_change":"2024-05-30T17:00:36.770891+0000","last_active":"2024-05-30T17:00:38.148191+0000","last_peered":"2024-05-30T17:00:38.148191+0000","last_clean":"2024-05-30T17:00:38.148191+0000","last_became_active":"2024-05-30T17:00:36.768598+0000","last_became_peered":"2024-05-30T17:00:36.768598+0000","last_unstale":"2024-05-30T17:00:38.148191+0000","last_undegraded":"2024-05-30T17:00:38.148191+0000","last_fullsized":"2024-05-30T17:00:38.148191+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:07.554476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.756150+0000","last_change":"2024-05-30T17:00:36.758929+0000","last_active":"2024-05-30T17:00:37.756150+0000","last_peered":"2024-05-30T17:00:37.756150+0000","last_clean":"2024-05-30T17:00:37.756150+0000","last_became_active":"2024-05-30T17:00:36.758651+0000","last_became_peered":"2024-05-30T17:00:36.758651+0000","last_unstale":"2024-05-30T17:00:37.756150+0000","last_undegraded":"2024-05-30T17:00:37.756150+0000","last_fullsized":"2024-05-30T17:00:37.756150+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:48.926394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752501+0000","last_change":"2024-05-30T17:00:35.735865+0000","last_active":"2024-05-30T17:00:37.752501+0000","last_peered":"2024-05-30T17:00:37.752501+0000","last_clean":"2024-05-30T17:00:37.752501+0000","last_became_active":"2024-05-30T17:00:35.735526+0000","last_became_peered":"2024-05-30T17:00:35.735526+0000","last_unstale":"2024-05-30T17:00:37.752501+0000","last_undegraded":"2024-05-30T17:00:37.752501+0000","last_fullsized":"2024-05-30T17:00:37.752501+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:42:29.413638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151409+0000","last_change":"2024-05-30T17:00:36.775696+0000","last_active":"2024-05-30T17:00:38.151409+0000","last_peered":"2024-05-30T17:00:38.151409+0000","last_clean":"2024-05-30T17:00:38.151409+0000","last_became_active":"2024-05-30T17:00:36.774878+0000","last_became_peered":"2024-05-30T17:00:36.774878+0000","last_unstale":"2024-05-30T17:00:38.151409+0000","last_undegraded":"2024-05-30T17:00:38.151409+0000","last_fullsized":"2024-05-30T17:00:38.151409+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:43.497412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150045+0000","last_change":"2024-05-30T17:00:35.735197+0000","last_active":"2024-05-30T17:00:38.150045+0000","last_peered":"2024-05-30T17:00:38.150045+0000","last_clean":"2024-05-30T17:00:38.150045+0000","last_became_active":"2024-05-30T17:00:35.734913+0000","last_became_peered":"2024-05-30T17:00:35.734913+0000","last_unstale":"2024-05-30T17:00:38.150045+0000","last_undegraded":"2024-05-30T17:00:38.150045+0000","last_fullsized":"2024-05-30T17:00:38.150045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:58:37.442958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148864+0000","last_change":"2024-05-30T17:00:36.773283+0000","last_active":"2024-05-30T17:00:38.148864+0000","last_peered":"2024-05-30T17:00:38.148864+0000","last_clean":"2024-05-30T17:00:38.148864+0000","last_became_active":"2024-05-30T17:00:36.772999+0000","last_became_peered":"2024-05-30T17:00:36.772999+0000","last_unstale":"2024-05-30T17:00:38.148864+0000","last_undegraded":"2024-05-30T17:00:38.148864+0000","last_fullsized":"2024-05-30T17:00:38.148864+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:55:05.314618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752193+0000","last_change":"2024-05-30T17:00:35.729069+0000","last_active":"2024-05-30T17:00:37.752193+0000","last_peered":"2024-05-30T17:00:37.752193+0000","last_clean":"2024-05-30T17:00:37.752193+0000","last_became_active":"2024-05-30T17:00:35.728765+0000","last_became_peered":"2024-05-30T17:00:35.728765+0000","last_unstale":"2024-05-30T17:00:37.752193+0000","last_undegraded":"2024-05-30T17:00:37.752193+0000","last_fullsized":"2024-05-30T17:00:37.752193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:49:59.611402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761260+0000","last_change":"2024-05-30T17:00:36.746352+0000","last_active":"2024-05-30T17:00:37.761260+0000","last_peered":"2024-05-30T17:00:37.761260+0000","last_clean":"2024-05-30T17:00:37.761260+0000","last_became_active":"2024-05-30T17:00:36.745894+0000","last_became_peered":"2024-05-30T17:00:36.745894+0000","last_unstale":"2024-05-30T17:00:37.761260+0000","last_undegraded":"2024-05-30T17:00:37.761260+0000","last_fullsized":"2024-05-30T17:00:37.761260+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:39:58.290648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752014+0000","last_change":"2024-05-30T17:00:35.734108+0000","last_active":"2024-05-30T17:00:37.752014+0000","last_peered":"2024-05-30T17:00:37.752014+0000","last_clean":"2024-05-30T17:00:37.752014+0000","last_became_active":"2024-05-30T17:00:35.733598+0000","last_became_peered":"2024-05-30T17:00:35.733598+0000","last_unstale":"2024-05-30T17:00:37.752014+0000","last_undegraded":"2024-05-30T17:00:37.752014+0000","last_fullsized":"2024-05-30T17:00:37.752014+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:22.323508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758033+0000","last_change":"2024-05-30T17:00:36.743969+0000","last_active":"2024-05-30T17:00:37.758033+0000","last_peered":"2024-05-30T17:00:37.758033+0000","last_clean":"2024-05-30T17:00:37.758033+0000","last_became_active":"2024-05-30T17:00:36.743574+0000","last_became_peered":"2024-05-30T17:00:36.743574+0000","last_unstale":"2024-05-30T17:00:37.758033+0000","last_undegraded":"2024-05-30T17:00:37.758033+0000","last_fullsized":"2024-05-30T17:00:37.758033+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:57.026949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148543+0000","last_change":"2024-05-30T17:00:35.731086+0000","last_active":"2024-05-30T17:00:38.148543+0000","last_peered":"2024-05-30T17:00:38.148543+0000","last_clean":"2024-05-30T17:00:38.148543+0000","last_became_active":"2024-05-30T17:00:35.730454+0000","last_became_peered":"2024-05-30T17:00:35.730454+0000","last_unstale":"2024-05-30T17:00:38.148543+0000","last_undegraded":"2024-05-30T17:00:38.148543+0000","last_fullsized":"2024-05-30T17:00:38.148543+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:24.659637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149522+0000","last_change":"2024-05-30T17:00:36.755868+0000","last_active":"2024-05-30T17:00:38.149522+0000","last_peered":"2024-05-30T17:00:38.149522+0000","last_clean":"2024-05-30T17:00:38.149522+0000","last_became_active":"2024-05-30T17:00:36.755420+0000","last_became_peered":"2024-05-30T17:00:36.755420+0000","last_unstale":"2024-05-30T17:00:38.149522+0000","last_undegraded":"2024-05-30T17:00:38.149522+0000","last_fullsized":"2024-05-30T17:00:38.149522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751802+0000","last_change":"2024-05-30T17:00:35.733623+0000","last_active":"2024-05-30T17:00:37.751802+0000","last_peered":"2024-05-30T17:00:37.751802+0000","last_clean":"2024-05-30T17:00:37.751802+0000","last_became_active":"2024-05-30T17:00:35.733325+0000","last_became_peered":"2024-05-30T17:00:35.733325+0000","last_unstale":"2024-05-30T17:00:37.751802+0000","last_undegraded":"2024-05-30T17:00:37.751802+0000","last_fullsized":"2024-05-30T17:00:37.751802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:14.320132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150246+0000","last_change":"2024-05-30T17:00:36.765167+0000","last_active":"2024-05-30T17:00:38.150246+0000","last_peered":"2024-05-30T17:00:38.150246+0000","last_clean":"2024-05-30T17:00:38.150246+0000","last_became_active":"2024-05-30T17:00:36.764873+0000","last_became_peered":"2024-05-30T17:00:36.764873+0000","last_unstale":"2024-05-30T17:00:38.150246+0000","last_undegraded":"2024-05-30T17:00:38.150246+0000","last_fullsized":"2024-05-30T17:00:38.150246+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:55.166729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151425+0000","last_change":"2024-05-30T17:00:35.734103+0000","last_active":"2024-05-30T17:00:38.151425+0000","last_peered":"2024-05-30T17:00:38.151425+0000","last_clean":"2024-05-30T17:00:38.151425+0000","last_became_active":"2024-05-30T17:00:35.733719+0000","last_became_peered":"2024-05-30T17:00:35.733719+0000","last_unstale":"2024-05-30T17:00:38.151425+0000","last_undegraded":"2024-05-30T17:00:38.151425+0000","last_fullsized":"2024-05-30T17:00:38.151425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:42.872961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759392+0000","last_change":"2024-05-30T17:00:36.747707+0000","last_active":"2024-05-30T17:00:37.759392+0000","last_peered":"2024-05-30T17:00:37.759392+0000","last_clean":"2024-05-30T17:00:37.759392+0000","last_became_active":"2024-05-30T17:00:36.744282+0000","last_became_peered":"2024-05-30T17:00:36.744282+0000","last_unstale":"2024-05-30T17:00:37.759392+0000","last_undegraded":"2024-05-30T17:00:37.759392+0000","last_fullsized":"2024-05-30T17:00:37.759392+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:48.052699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148317+0000","last_change":"2024-05-30T17:00:35.732470+0000","last_active":"2024-05-30T17:00:38.148317+0000","last_peered":"2024-05-30T17:00:38.148317+0000","last_clean":"2024-05-30T17:00:38.148317+0000","last_became_active":"2024-05-30T17:00:35.731250+0000","last_became_peered":"2024-05-30T17:00:35.731250+0000","last_unstale":"2024-05-30T17:00:38.148317+0000","last_undegraded":"2024-05-30T17:00:38.148317+0000","last_fullsized":"2024-05-30T17:00:38.148317+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:24.791603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752394+0000","last_change":"2024-05-30T17:00:36.742303+0000","last_active":"2024-05-30T17:00:37.752394+0000","last_peered":"2024-05-30T17:00:37.752394+0000","last_clean":"2024-05-30T17:00:37.752394+0000","last_became_active":"2024-05-30T17:00:36.741809+0000","last_became_peered":"2024-05-30T17:00:36.741809+0000","last_unstale":"2024-05-30T17:00:37.752394+0000","last_undegraded":"2024-05-30T17:00:37.752394+0000","last_fullsized":"2024-05-30T17:00:37.752394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149201+0000","last_change":"2024-05-30T17:00:35.728011+0000","last_active":"2024-05-30T17:00:38.149201+0000","last_peered":"2024-05-30T17:00:38.149201+0000","last_clean":"2024-05-30T17:00:38.149201+0000","last_became_active":"2024-05-30T17:00:35.727728+0000","last_became_peered":"2024-05-30T17:00:35.727728+0000","last_unstale":"2024-05-30T17:00:38.149201+0000","last_undegraded":"2024-05-30T17:00:38.149201+0000","last_fullsized":"2024-05-30T17:00:38.149201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:44:23.366065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760526+0000","last_change":"2024-05-30T17:00:36.747039+0000","last_active":"2024-05-30T17:00:37.760526+0000","last_peered":"2024-05-30T17:00:37.760526+0000","last_clean":"2024-05-30T17:00:37.760526+0000","last_became_active":"2024-05-30T17:00:36.745915+0000","last_became_peered":"2024-05-30T17:00:36.745915+0000","last_unstale":"2024-05-30T17:00:37.760526+0000","last_undegraded":"2024-05-30T17:00:37.760526+0000","last_fullsized":"2024-05-30T17:00:37.760526+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:21.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150639+0000","last_change":"2024-05-30T17:00:35.718080+0000","last_active":"2024-05-30T17:00:38.150639+0000","last_peered":"2024-05-30T17:00:38.150639+0000","last_clean":"2024-05-30T17:00:38.150639+0000","last_became_active":"2024-05-30T17:00:35.717705+0000","last_became_peered":"2024-05-30T17:00:35.717705+0000","last_unstale":"2024-05-30T17:00:38.150639+0000","last_undegraded":"2024-05-30T17:00:38.150639+0000","last_fullsized":"2024-05-30T17:00:38.150639+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:12:07.410732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152539+0000","last_change":"2024-05-30T17:00:36.754484+0000","last_active":"2024-05-30T17:00:38.152539+0000","last_peered":"2024-05-30T17:00:38.152539+0000","last_clean":"2024-05-30T17:00:38.152539+0000","last_became_active":"2024-05-30T17:00:36.754166+0000","last_became_peered":"2024-05-30T17:00:36.754166+0000","last_unstale":"2024-05-30T17:00:38.152539+0000","last_undegraded":"2024-05-30T17:00:38.152539+0000","last_fullsized":"2024-05-30T17:00:38.152539+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:33.091804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150419+0000","last_change":"2024-05-30T17:00:35.732982+0000","last_active":"2024-05-30T17:00:38.150419+0000","last_peered":"2024-05-30T17:00:38.150419+0000","last_clean":"2024-05-30T17:00:38.150419+0000","last_became_active":"2024-05-30T17:00:35.732114+0000","last_became_peered":"2024-05-30T17:00:35.732114+0000","last_unstale":"2024-05-30T17:00:38.150419+0000","last_undegraded":"2024-05-30T17:00:38.150419+0000","last_fullsized":"2024-05-30T17:00:38.150419+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:33.694752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150348+0000","last_change":"2024-05-30T17:00:36.758949+0000","last_active":"2024-05-30T17:00:38.150348+0000","last_peered":"2024-05-30T17:00:38.150348+0000","last_clean":"2024-05-30T17:00:38.150348+0000","last_became_active":"2024-05-30T17:00:36.758069+0000","last_became_peered":"2024-05-30T17:00:36.758069+0000","last_unstale":"2024-05-30T17:00:38.150348+0000","last_undegraded":"2024-05-30T17:00:38.150348+0000","last_fullsized":"2024-05-30T17:00:38.150348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:38:49.283720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760740+0000","last_change":"2024-05-30T17:00:35.731848+0000","last_active":"2024-05-30T17:00:37.760740+0000","last_peered":"2024-05-30T17:00:37.760740+0000","last_clean":"2024-05-30T17:00:37.760740+0000","last_became_active":"2024-05-30T17:00:35.731536+0000","last_became_peered":"2024-05-30T17:00:35.731536+0000","last_unstale":"2024-05-30T17:00:37.760740+0000","last_undegraded":"2024-05-30T17:00:37.760740+0000","last_fullsized":"2024-05-30T17:00:37.760740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:12:27.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150203+0000","last_change":"2024-05-30T17:00:36.762049+0000","last_active":"2024-05-30T17:00:38.150203+0000","last_peered":"2024-05-30T17:00:38.150203+0000","last_clean":"2024-05-30T17:00:38.150203+0000","last_became_active":"2024-05-30T17:00:36.761703+0000","last_became_peered":"2024-05-30T17:00:36.761703+0000","last_unstale":"2024-05-30T17:00:38.150203+0000","last_undegraded":"2024-05-30T17:00:38.150203+0000","last_fullsized":"2024-05-30T17:00:38.150203+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:46:34.225518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755711+0000","last_change":"2024-05-30T17:00:35.734783+0000","last_active":"2024-05-30T17:00:37.755711+0000","last_peered":"2024-05-30T17:00:37.755711+0000","last_clean":"2024-05-30T17:00:37.755711+0000","last_became_active":"2024-05-30T17:00:35.733971+0000","last_became_peered":"2024-05-30T17:00:35.733971+0000","last_unstale":"2024-05-30T17:00:37.755711+0000","last_undegraded":"2024-05-30T17:00:37.755711+0000","last_fullsized":"2024-05-30T17:00:37.755711+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:43.376431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148751+0000","last_change":"2024-05-30T17:00:36.761813+0000","last_active":"2024-05-30T17:00:38.148751+0000","last_peered":"2024-05-30T17:00:38.148751+0000","last_clean":"2024-05-30T17:00:38.148751+0000","last_became_active":"2024-05-30T17:00:36.755862+0000","last_became_peered":"2024-05-30T17:00:36.755862+0000","last_unstale":"2024-05-30T17:00:38.148751+0000","last_undegraded":"2024-05-30T17:00:38.148751+0000","last_fullsized":"2024-05-30T17:00:38.148751+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:08.876736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755595+0000","last_change":"2024-05-30T17:00:35.738764+0000","last_active":"2024-05-30T17:00:37.755595+0000","last_peered":"2024-05-30T17:00:37.755595+0000","last_clean":"2024-05-30T17:00:37.755595+0000","last_became_active":"2024-05-30T17:00:35.737679+0000","last_became_peered":"2024-05-30T17:00:35.737679+0000","last_unstale":"2024-05-30T17:00:37.755595+0000","last_undegraded":"2024-05-30T17:00:37.755595+0000","last_fullsized":"2024-05-30T17:00:37.755595+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:42:33.203512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760261+0000","last_change":"2024-05-30T17:00:36.744702+0000","last_active":"2024-05-30T17:00:37.760261+0000","last_peered":"2024-05-30T17:00:37.760261+0000","last_clean":"2024-05-30T17:00:37.760261+0000","last_became_active":"2024-05-30T17:00:36.744329+0000","last_became_peered":"2024-05-30T17:00:36.744329+0000","last_unstale":"2024-05-30T17:00:37.760261+0000","last_undegraded":"2024-05-30T17:00:37.760261+0000","last_fullsized":"2024-05-30T17:00:37.760261+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:21:27.452364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759287+0000","last_change":"2024-05-30T17:00:35.729767+0000","last_active":"2024-05-30T17:00:37.759287+0000","last_peered":"2024-05-30T17:00:37.759287+0000","last_clean":"2024-05-30T17:00:37.759287+0000","last_became_active":"2024-05-30T17:00:35.729412+0000","last_became_peered":"2024-05-30T17:00:35.729412+0000","last_unstale":"2024-05-30T17:00:37.759287+0000","last_undegraded":"2024-05-30T17:00:37.759287+0000","last_fullsized":"2024-05-30T17:00:37.759287+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:47.793776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755890+0000","last_change":"2024-05-30T17:00:36.754800+0000","last_active":"2024-05-30T17:00:37.755890+0000","last_peered":"2024-05-30T17:00:37.755890+0000","last_clean":"2024-05-30T17:00:37.755890+0000","last_became_active":"2024-05-30T17:00:36.754501+0000","last_became_peered":"2024-05-30T17:00:36.754501+0000","last_unstale":"2024-05-30T17:00:37.755890+0000","last_undegraded":"2024-05-30T17:00:37.755890+0000","last_fullsized":"2024-05-30T17:00:37.755890+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:40:28.215068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":22,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148274+0000","last_change":"2024-05-30T17:00:35.729728+0000","last_active":"2024-05-30T17:00:38.148274+0000","last_peered":"2024-05-30T17:00:38.148274+0000","last_clean":"2024-05-30T17:00:38.148274+0000","last_became_active":"2024-05-30T17:00:35.723478+0000","last_became_peered":"2024-05-30T17:00:35.723478+0000","last_unstale":"2024-05-30T17:00:38.148274+0000","last_undegraded":"2024-05-30T17:00:38.148274+0000","last_fullsized":"2024-05-30T17:00:38.148274+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:46:24.942533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152237+0000","last_change":"2024-05-30T17:00:36.751676+0000","last_active":"2024-05-30T17:00:38.152237+0000","last_peered":"2024-05-30T17:00:38.152237+0000","last_clean":"2024-05-30T17:00:38.152237+0000","last_became_active":"2024-05-30T17:00:36.751355+0000","last_became_peered":"2024-05-30T17:00:36.751355+0000","last_unstale":"2024-05-30T17:00:38.152237+0000","last_undegraded":"2024-05-30T17:00:38.152237+0000","last_fullsized":"2024-05-30T17:00:38.152237+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:01:43.178806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760541+0000","last_change":"2024-05-30T17:00:35.734613+0000","last_active":"2024-05-30T17:00:37.760541+0000","last_peered":"2024-05-30T17:00:37.760541+0000","last_clean":"2024-05-30T17:00:37.760541+0000","last_became_active":"2024-05-30T17:00:35.734231+0000","last_became_peered":"2024-05-30T17:00:35.734231+0000","last_unstale":"2024-05-30T17:00:37.760541+0000","last_undegraded":"2024-05-30T17:00:37.760541+0000","last_fullsized":"2024-05-30T17:00:37.760541+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:06.120742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758055+0000","last_change":"2024-05-30T17:00:36.757383+0000","last_active":"2024-05-30T17:00:37.758055+0000","last_peered":"2024-05-30T17:00:37.758055+0000","last_clean":"2024-05-30T17:00:37.758055+0000","last_became_active":"2024-05-30T17:00:36.756264+0000","last_became_peered":"2024-05-30T17:00:36.756264+0000","last_unstale":"2024-05-30T17:00:37.758055+0000","last_undegraded":"2024-05-30T17:00:37.758055+0000","last_fullsized":"2024-05-30T17:00:37.758055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:32.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151230+0000","last_change":"2024-05-30T17:00:35.730141+0000","last_active":"2024-05-30T17:00:38.151230+0000","last_peered":"2024-05-30T17:00:38.151230+0000","last_clean":"2024-05-30T17:00:38.151230+0000","last_became_active":"2024-05-30T17:00:35.728948+0000","last_became_peered":"2024-05-30T17:00:35.728948+0000","last_unstale":"2024-05-30T17:00:38.151230+0000","last_undegraded":"2024-05-30T17:00:38.151230+0000","last_fullsized":"2024-05-30T17:00:38.151230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:13.200467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150264+0000","last_change":"2024-05-30T17:00:36.750999+0000","last_active":"2024-05-30T17:00:38.150264+0000","last_peered":"2024-05-30T17:00:38.150264+0000","last_clean":"2024-05-30T17:00:38.150264+0000","last_became_active":"2024-05-30T17:00:36.750499+0000","last_became_peered":"2024-05-30T17:00:36.750499+0000","last_unstale":"2024-05-30T17:00:38.150264+0000","last_undegraded":"2024-05-30T17:00:38.150264+0000","last_fullsized":"2024-05-30T17:00:38.150264+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:10.934086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751996+0000","last_change":"2024-05-30T17:00:35.733883+0000","last_active":"2024-05-30T17:00:37.751996+0000","last_peered":"2024-05-30T17:00:37.751996+0000","last_clean":"2024-05-30T17:00:37.751996+0000","last_became_active":"2024-05-30T17:00:35.733243+0000","last_became_peered":"2024-05-30T17:00:35.733243+0000","last_unstale":"2024-05-30T17:00:37.751996+0000","last_undegraded":"2024-05-30T17:00:37.751996+0000","last_fullsized":"2024-05-30T17:00:37.751996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:46:39.811373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755479+0000","last_change":"2024-05-30T17:00:36.763473+0000","last_active":"2024-05-30T17:00:37.755479+0000","last_peered":"2024-05-30T17:00:37.755479+0000","last_clean":"2024-05-30T17:00:37.755479+0000","last_became_active":"2024-05-30T17:00:36.763143+0000","last_became_peered":"2024-05-30T17:00:36.763143+0000","last_unstale":"2024-05-30T17:00:37.755479+0000","last_undegraded":"2024-05-30T17:00:37.755479+0000","last_fullsized":"2024-05-30T17:00:37.755479+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:09:42.812288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752192+0000","last_change":"2024-05-30T17:00:35.729083+0000","last_active":"2024-05-30T17:00:37.752192+0000","last_peered":"2024-05-30T17:00:37.752192+0000","last_clean":"2024-05-30T17:00:37.752192+0000","last_became_active":"2024-05-30T17:00:35.728830+0000","last_became_peered":"2024-05-30T17:00:35.728830+0000","last_unstale":"2024-05-30T17:00:37.752192+0000","last_undegraded":"2024-05-30T17:00:37.752192+0000","last_fullsized":"2024-05-30T17:00:37.752192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:52.170762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755588+0000","last_change":"2024-05-30T17:00:36.764134+0000","last_active":"2024-05-30T17:00:37.755588+0000","last_peered":"2024-05-30T17:00:37.755588+0000","last_clean":"2024-05-30T17:00:37.755588+0000","last_became_active":"2024-05-30T17:00:36.763828+0000","last_became_peered":"2024-05-30T17:00:36.763828+0000","last_unstale":"2024-05-30T17:00:37.755588+0000","last_undegraded":"2024-05-30T17:00:37.755588+0000","last_fullsized":"2024-05-30T17:00:37.755588+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:06.014878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757948+0000","last_change":"2024-05-30T17:00:35.730608+0000","last_active":"2024-05-30T17:00:37.757948+0000","last_peered":"2024-05-30T17:00:37.757948+0000","last_clean":"2024-05-30T17:00:37.757948+0000","last_became_active":"2024-05-30T17:00:35.730310+0000","last_became_peered":"2024-05-30T17:00:35.730310+0000","last_unstale":"2024-05-30T17:00:37.757948+0000","last_undegraded":"2024-05-30T17:00:37.757948+0000","last_fullsized":"2024-05-30T17:00:37.757948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:13.646048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.754917+0000","last_change":"2024-05-30T17:00:36.763307+0000","last_active":"2024-05-30T17:00:37.754917+0000","last_peered":"2024-05-30T17:00:37.754917+0000","last_clean":"2024-05-30T17:00:37.754917+0000","last_became_active":"2024-05-30T17:00:36.763027+0000","last_became_peered":"2024-05-30T17:00:36.763027+0000","last_unstale":"2024-05-30T17:00:37.754917+0000","last_undegraded":"2024-05-30T17:00:37.754917+0000","last_fullsized":"2024-05-30T17:00:37.754917+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:08.876736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757483+0000","last_change":"2024-05-30T17:00:35.709483+0000","last_active":"2024-05-30T17:00:37.757483+0000","last_peered":"2024-05-30T17:00:37.757483+0000","last_clean":"2024-05-30T17:00:37.757483+0000","last_became_active":"2024-05-30T17:00:35.708526+0000","last_became_peered":"2024-05-30T17:00:35.708526+0000","last_unstale":"2024-05-30T17:00:37.757483+0000","last_undegraded":"2024-05-30T17:00:37.757483+0000","last_fullsized":"2024-05-30T17:00:37.757483+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:40.251625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151273+0000","last_change":"2024-05-30T17:00:36.749874+0000","last_active":"2024-05-30T17:00:38.151273+0000","last_peered":"2024-05-30T17:00:38.151273+0000","last_clean":"2024-05-30T17:00:38.151273+0000","last_became_active":"2024-05-30T17:00:36.749228+0000","last_became_peered":"2024-05-30T17:00:36.749228+0000","last_unstale":"2024-05-30T17:00:38.151273+0000","last_undegraded":"2024-05-30T17:00:38.151273+0000","last_fullsized":"2024-05-30T17:00:38.151273+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:44.276755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751571+0000","last_change":"2024-05-30T17:00:35.736336+0000","last_active":"2024-05-30T17:00:37.751571+0000","last_peered":"2024-05-30T17:00:37.751571+0000","last_clean":"2024-05-30T17:00:37.751571+0000","last_became_active":"2024-05-30T17:00:35.734451+0000","last_became_peered":"2024-05-30T17:00:35.734451+0000","last_unstale":"2024-05-30T17:00:37.751571+0000","last_undegraded":"2024-05-30T17:00:37.751571+0000","last_fullsized":"2024-05-30T17:00:37.751571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:16.292089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752599+0000","last_change":"2024-05-30T17:00:36.748422+0000","last_active":"2024-05-30T17:00:37.752599+0000","last_peered":"2024-05-30T17:00:37.752599+0000","last_clean":"2024-05-30T17:00:37.752599+0000","last_became_active":"2024-05-30T17:00:36.747900+0000","last_became_peered":"2024-05-30T17:00:36.747900+0000","last_unstale":"2024-05-30T17:00:37.752599+0000","last_undegraded":"2024-05-30T17:00:37.752599+0000","last_fullsized":"2024-05-30T17:00:37.752599+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758807+0000","last_change":"2024-05-30T17:00:35.729597+0000","last_active":"2024-05-30T17:00:37.758807+0000","last_peered":"2024-05-30T17:00:37.758807+0000","last_clean":"2024-05-30T17:00:37.758807+0000","last_became_active":"2024-05-30T17:00:35.728519+0000","last_became_peered":"2024-05-30T17:00:35.728519+0000","last_unstale":"2024-05-30T17:00:37.758807+0000","last_undegraded":"2024-05-30T17:00:37.758807+0000","last_fullsized":"2024-05-30T17:00:37.758807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:12.232879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149106+0000","last_change":"2024-05-30T17:00:36.753305+0000","last_active":"2024-05-30T17:00:38.149106+0000","last_peered":"2024-05-30T17:00:38.149106+0000","last_clean":"2024-05-30T17:00:38.149106+0000","last_became_active":"2024-05-30T17:00:36.753023+0000","last_became_peered":"2024-05-30T17:00:36.753023+0000","last_unstale":"2024-05-30T17:00:38.149106+0000","last_undegraded":"2024-05-30T17:00:38.149106+0000","last_fullsized":"2024-05-30T17:00:38.149106+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:24:23.085972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.754450+0000","last_change":"2024-05-30T17:00:35.733609+0000","last_active":"2024-05-30T17:00:37.754450+0000","last_peered":"2024-05-30T17:00:37.754450+0000","last_clean":"2024-05-30T17:00:37.754450+0000","last_became_active":"2024-05-30T17:00:35.733292+0000","last_became_peered":"2024-05-30T17:00:35.733292+0000","last_unstale":"2024-05-30T17:00:37.754450+0000","last_undegraded":"2024-05-30T17:00:37.754450+0000","last_fullsized":"2024-05-30T17:00:37.754450+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:00.979924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149021+0000","last_change":"2024-05-30T17:00:36.759105+0000","last_active":"2024-05-30T17:00:38.149021+0000","last_peered":"2024-05-30T17:00:38.149021+0000","last_clean":"2024-05-30T17:00:38.149021+0000","last_became_active":"2024-05-30T17:00:36.758755+0000","last_became_peered":"2024-05-30T17:00:36.758755+0000","last_unstale":"2024-05-30T17:00:38.149021+0000","last_undegraded":"2024-05-30T17:00:38.149021+0000","last_fullsized":"2024-05-30T17:00:38.149021+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:39:26.605255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752339+0000","last_change":"2024-05-30T17:00:35.734441+0000","last_active":"2024-05-30T17:00:37.752339+0000","last_peered":"2024-05-30T17:00:37.752339+0000","last_clean":"2024-05-30T17:00:37.752339+0000","last_became_active":"2024-05-30T17:00:35.732710+0000","last_became_peered":"2024-05-30T17:00:35.732710+0000","last_unstale":"2024-05-30T17:00:37.752339+0000","last_undegraded":"2024-05-30T17:00:37.752339+0000","last_fullsized":"2024-05-30T17:00:37.752339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:15.063268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151374+0000","last_change":"2024-05-30T17:00:36.764319+0000","last_active":"2024-05-30T17:00:38.151374+0000","last_peered":"2024-05-30T17:00:38.151374+0000","last_clean":"2024-05-30T17:00:38.151374+0000","last_became_active":"2024-05-30T17:00:36.764034+0000","last_became_peered":"2024-05-30T17:00:36.764034+0000","last_unstale":"2024-05-30T17:00:38.151374+0000","last_undegraded":"2024-05-30T17:00:38.151374+0000","last_fullsized":"2024-05-30T17:00:38.151374+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:28.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152390+0000","last_change":"2024-05-30T17:00:35.727312+0000","last_active":"2024-05-30T17:00:38.152390+0000","last_peered":"2024-05-30T17:00:38.152390+0000","last_clean":"2024-05-30T17:00:38.152390+0000","last_became_active":"2024-05-30T17:00:35.726820+0000","last_became_peered":"2024-05-30T17:00:35.726820+0000","last_unstale":"2024-05-30T17:00:38.152390+0000","last_undegraded":"2024-05-30T17:00:38.152390+0000","last_fullsized":"2024-05-30T17:00:38.152390+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:40.105333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755538+0000","last_change":"2024-05-30T17:00:36.755404+0000","last_active":"2024-05-30T17:00:37.755538+0000","last_peered":"2024-05-30T17:00:37.755538+0000","last_clean":"2024-05-30T17:00:37.755538+0000","last_became_active":"2024-05-30T17:00:36.755137+0000","last_became_peered":"2024-05-30T17:00:36.755137+0000","last_unstale":"2024-05-30T17:00:37.755538+0000","last_undegraded":"2024-05-30T17:00:37.755538+0000","last_fullsized":"2024-05-30T17:00:37.755538+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:55:07.410908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752300+0000","last_change":"2024-05-30T17:00:35.727143+0000","last_active":"2024-05-30T17:00:37.752300+0000","last_peered":"2024-05-30T17:00:37.752300+0000","last_clean":"2024-05-30T17:00:37.752300+0000","last_became_active":"2024-05-30T17:00:35.725458+0000","last_became_peered":"2024-05-30T17:00:35.725458+0000","last_unstale":"2024-05-30T17:00:37.752300+0000","last_undegraded":"2024-05-30T17:00:37.752300+0000","last_fullsized":"2024-05-30T17:00:37.752300+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:49.887792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760276+0000","last_change":"2024-05-30T17:00:36.756498+0000","last_active":"2024-05-30T17:00:37.760276+0000","last_peered":"2024-05-30T17:00:37.760276+0000","last_clean":"2024-05-30T17:00:37.760276+0000","last_became_active":"2024-05-30T17:00:36.755460+0000","last_became_peered":"2024-05-30T17:00:36.755460+0000","last_unstale":"2024-05-30T17:00:37.760276+0000","last_undegraded":"2024-05-30T17:00:37.760276+0000","last_fullsized":"2024-05-30T17:00:37.760276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:01.424865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150911+0000","last_change":"2024-05-30T17:00:35.731535+0000","last_active":"2024-05-30T17:00:38.150911+0000","last_peered":"2024-05-30T17:00:38.150911+0000","last_clean":"2024-05-30T17:00:38.150911+0000","last_became_active":"2024-05-30T17:00:35.730893+0000","last_became_peered":"2024-05-30T17:00:35.730893+0000","last_unstale":"2024-05-30T17:00:38.150911+0000","last_undegraded":"2024-05-30T17:00:38.150911+0000","last_fullsized":"2024-05-30T17:00:38.150911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:45.698539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.754917+0000","last_change":"2024-05-30T17:00:36.754985+0000","last_active":"2024-05-30T17:00:37.754917+0000","last_peered":"2024-05-30T17:00:37.754917+0000","last_clean":"2024-05-30T17:00:37.754917+0000","last_became_active":"2024-05-30T17:00:36.754666+0000","last_became_peered":"2024-05-30T17:00:36.754666+0000","last_unstale":"2024-05-30T17:00:37.754917+0000","last_undegraded":"2024-05-30T17:00:37.754917+0000","last_fullsized":"2024-05-30T17:00:37.754917+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:55:05.314618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150387+0000","last_change":"2024-05-30T17:00:35.724537+0000","last_active":"2024-05-30T17:00:38.150387+0000","last_peered":"2024-05-30T17:00:38.150387+0000","last_clean":"2024-05-30T17:00:38.150387+0000","last_became_active":"2024-05-30T17:00:35.724245+0000","last_became_peered":"2024-05-30T17:00:35.724245+0000","last_unstale":"2024-05-30T17:00:38.150387+0000","last_undegraded":"2024-05-30T17:00:38.150387+0000","last_fullsized":"2024-05-30T17:00:38.150387+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:43.263919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151226+0000","last_change":"2024-05-30T17:00:36.761666+0000","last_active":"2024-05-30T17:00:38.151226+0000","last_peered":"2024-05-30T17:00:38.151226+0000","last_clean":"2024-05-30T17:00:38.151226+0000","last_became_active":"2024-05-30T17:00:36.760872+0000","last_became_peered":"2024-05-30T17:00:36.760872+0000","last_unstale":"2024-05-30T17:00:38.151226+0000","last_undegraded":"2024-05-30T17:00:38.151226+0000","last_fullsized":"2024-05-30T17:00:38.151226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:12:08.423568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755078+0000","last_change":"2024-05-30T17:00:35.732840+0000","last_active":"2024-05-30T17:00:37.755078+0000","last_peered":"2024-05-30T17:00:37.755078+0000","last_clean":"2024-05-30T17:00:37.755078+0000","last_became_active":"2024-05-30T17:00:35.732525+0000","last_became_peered":"2024-05-30T17:00:35.732525+0000","last_unstale":"2024-05-30T17:00:37.755078+0000","last_undegraded":"2024-05-30T17:00:37.755078+0000","last_fullsized":"2024-05-30T17:00:37.755078+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:35.909328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.756151+0000","last_change":"2024-05-30T17:00:36.765245+0000","last_active":"2024-05-30T17:00:37.756151+0000","last_peered":"2024-05-30T17:00:37.756151+0000","last_clean":"2024-05-30T17:00:37.756151+0000","last_became_active":"2024-05-30T17:00:36.764662+0000","last_became_peered":"2024-05-30T17:00:36.764662+0000","last_unstale":"2024-05-30T17:00:37.756151+0000","last_undegraded":"2024-05-30T17:00:37.756151+0000","last_fullsized":"2024-05-30T17:00:37.756151+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:49.059539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149819+0000","last_change":"2024-05-30T17:00:35.729629+0000","last_active":"2024-05-30T17:00:38.149819+0000","last_peered":"2024-05-30T17:00:38.149819+0000","last_clean":"2024-05-30T17:00:38.149819+0000","last_became_active":"2024-05-30T17:00:35.729282+0000","last_became_peered":"2024-05-30T17:00:35.729282+0000","last_unstale":"2024-05-30T17:00:38.149819+0000","last_undegraded":"2024-05-30T17:00:38.149819+0000","last_fullsized":"2024-05-30T17:00:38.149819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:43.450197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":330,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148934+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:00:38.148934+0000","last_peered":"2024-05-30T17:00:38.148934+0000","last_clean":"2024-05-30T17:00:38.148934+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:00:38.148934+0000","last_undegraded":"2024-05-30T17:00:38.148934+0000","last_fullsized":"2024-05-30T17:00:38.148934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509647,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130936,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240904,"statfs":{"total":96636764160,"available":96502685696,"internally_reserved":0,"allocated":1286144,"data_stored":558361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509647,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86528,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285312,"statfs":{"total":96636764160,"available":96548159488,"internally_reserved":0,"allocated":1298432,"data_stored":559445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509648,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126780,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245060,"statfs":{"total":96636764160,"available":96506941440,"internally_reserved":0,"allocated":1294336,"data_stored":559343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509648,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144820,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227020,"statfs":{"total":96636764160,"available":96488468480,"internally_reserved":0,"allocated":1282048,"data_stored":557743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509647,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126768,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245072,"statfs":{"total":96636764160,"available":96506953728,"internally_reserved":0,"allocated":1277952,"data_stored":557709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542350,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127356,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244484,"statfs":{"total":96636764160,"available":96506351616,"internally_reserved":0,"allocated":1880064,"data_stored":1148189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76788,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295052,"statfs":{"total":96636764160,"available":96558133248,"internally_reserved":0,"allocated":1282048,"data_stored":557779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542351,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69044,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302796,"statfs":{"total":96636764160,"available":96566063104,"internally_reserved":0,"allocated":1875968,"data_stored":1148659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:00:44.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:00:44.903 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:00:46.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:00:46.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.541+0000 7f72dae00640 1 Processor -- start 2024-05-30T17:00:46.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.541+0000 7f72dae00640 1 -- start start 2024-05-30T17:00:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.543+0000 7f72dae00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f72d415e790 0x7f72d40b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.543+0000 7f72dae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d40711c0 0x7f72d4071620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.543+0000 7f72dae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f72d409aa30 0x7f72d409aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.543+0000 7f72dae00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f72d415eb70 con 0x7f72d415e790 2024-05-30T17:00:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.543+0000 7f72dae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f72d40b5840 con 0x7f72d409aa30 2024-05-30T17:00:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.543+0000 7f72dae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72d40b5b20 con 0x7f72d40711c0 2024-05-30T17:00:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.543+0000 7f72d9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f72d415e790 0x7f72d40b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.543+0000 7f72d9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f72d415e790 0x7f72d40b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53192/0 (socket says 172.21.15.67:53192) 2024-05-30T17:00:46.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.543+0000 7f72d9a00640 1 -- 172.21.15.67:0/2123481982 learned_addr learned my addr 172.21.15.67:0/2123481982 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.544+0000 7f72d9000640 1 --2- 172.21.15.67:0/2123481982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d40711c0 0x7f72d4071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.544+0000 7f72da400640 1 --2- 172.21.15.67:0/2123481982 >> v2:172.21.15.169:3300/0 conn(0x7f72d409aa30 0x7f72d409aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.544+0000 7f72d9a00640 1 -- 172.21.15.67:0/2123481982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d40711c0 msgr2=0x7f72d4071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.544+0000 7f72d9a00640 1 --2- 172.21.15.67:0/2123481982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d40711c0 0x7f72d4071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.544+0000 7f72d9a00640 1 -- 172.21.15.67:0/2123481982 >> v2:172.21.15.169:3300/0 conn(0x7f72d409aa30 msgr2=0x7f72d409aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.544+0000 7f72d9a00640 1 --2- 172.21.15.67:0/2123481982 >> v2:172.21.15.169:3300/0 conn(0x7f72d409aa30 0x7f72d409aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.544+0000 7f72d9a00640 1 -- 172.21.15.67:0/2123481982 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d40b5e00 con 0x7f72d415e790 2024-05-30T17:00:46.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.545+0000 7f72d9a00640 1 --2- 172.21.15.67:0/2123481982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d415e790 0x7f72d40b68e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f72d4070390 tx=0x7f72c802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ddaea4e073a853 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:46.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.545+0000 7f72d3e00640 1 -- 172.21.15.67:0/2123481982 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8037070 con 0x7f72d415e790 2024-05-30T17:00:46.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.545+0000 7f72d3e00640 1 -- 172.21.15.67:0/2123481982 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c8032440 con 0x7f72d415e790 2024-05-30T17:00:46.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.545+0000 7f72d3e00640 1 -- 172.21.15.67:0/2123481982 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8032810 con 0x7f72d415e790 2024-05-30T17:00:46.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.546+0000 7f72dae00640 1 -- 172.21.15.67:0/2123481982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d415e790 msgr2=0x7f72d40b68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:46.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.546+0000 7f72dae00640 1 --2- 172.21.15.67:0/2123481982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d415e790 0x7f72d40b68e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f72d4070390 tx=0x7f72c802a140 comp rx=0 tx=0).stop 2024-05-30T17:00:46.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.546+0000 7f72dae00640 1 -- 172.21.15.67:0/2123481982 shutdown_connections 2024-05-30T17:00:46.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.546+0000 7f72dae00640 1 --2- 172.21.15.67:0/2123481982 >> v2:172.21.15.169:3300/0 conn(0x7f72d409aa30 0x7f72d409aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.546+0000 7f72dae00640 1 --2- 172.21.15.67:0/2123481982 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d40711c0 0x7f72d4071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.546+0000 7f72dae00640 1 --2- 172.21.15.67:0/2123481982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d415e790 0x7f72d40b68e0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.546+0000 7f72dae00640 1 -- 172.21.15.67:0/2123481982 >> 172.21.15.67:0/2123481982 conn(0x7f72d40b3ec0 msgr2=0x7f72d40b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:46.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.546+0000 7f72dae00640 1 -- 172.21.15.67:0/2123481982 shutdown_connections 2024-05-30T17:00:46.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.546+0000 7f72dae00640 1 -- 172.21.15.67:0/2123481982 wait complete. 2024-05-30T17:00:46.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.548+0000 7f72dae00640 1 Processor -- start 2024-05-30T17:00:46.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.549+0000 7f72dae00640 1 -- start start 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.549+0000 7f72dae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d40711c0 0x7f72d4155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.549+0000 7f72dae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d409aa30 0x7f72d415f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72dae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d415e790 0x7f72d41f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72dae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f72d40b54d0 con 0x7f72d409aa30 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72dae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72d40b63c0 con 0x7f72d40711c0 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72dae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72d406ca10 con 0x7f72d415e790 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72d9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d409aa30 0x7f72d415f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72d9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d409aa30 0x7f72d415f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53206/0 (socket says 172.21.15.67:53206) 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72d9000640 1 -- 172.21.15.67:0/3973656324 learned_addr learned my addr 172.21.15.67:0/3973656324 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:46.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72d9a00640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d40711c0 0x7f72d4155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72da400640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d415e790 0x7f72d41f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72d9000640 1 -- 172.21.15.67:0/3973656324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d415e790 msgr2=0x7f72d41f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72d9000640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d415e790 0x7f72d41f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72d9000640 1 -- 172.21.15.67:0/3973656324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d40711c0 msgr2=0x7f72d4155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72d9000640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d40711c0 0x7f72d4155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72d9000640 1 -- 172.21.15.67:0/3973656324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d4160000 con 0x7f72d409aa30 2024-05-30T17:00:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.550+0000 7f72da400640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d415e790 0x7f72d41f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.551+0000 7f72d9000640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d409aa30 0x7f72d415f700 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f72c000d800 tx=0x7f72c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.551+0000 7f72d1600640 1 -- 172.21.15.67:0/3973656324 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c000b6c0 con 0x7f72d409aa30 2024-05-30T17:00:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.551+0000 7f72d1600640 1 -- 172.21.15.67:0/3973656324 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c0019070 con 0x7f72d409aa30 2024-05-30T17:00:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.551+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d41602e0 con 0x7f72d409aa30 2024-05-30T17:00:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.551+0000 7f72d1600640 1 -- 172.21.15.67:0/3973656324 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c000ba80 con 0x7f72d409aa30 2024-05-30T17:00:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.551+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d41608a0 con 0x7f72d409aa30 2024-05-30T17:00:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.552+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d40690f0 con 0x7f72d409aa30 2024-05-30T17:00:46.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.553+0000 7f72d1600640 1 -- 172.21.15.67:0/3973656324 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f72c000bc20 con 0x7f72d409aa30 2024-05-30T17:00:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.554+0000 7f72d1600640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f72b406ddc0 0x7f72b4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.554+0000 7f72d1600640 1 -- 172.21.15.67:0/3973656324 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f72c008b140 con 0x7f72d409aa30 2024-05-30T17:00:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.555+0000 7f72d9a00640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f72b406ddc0 0x7f72b4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:46.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.556+0000 7f72d9a00640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f72b406ddc0 0x7f72b4070280 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f72c8034000 tx=0x7f72c802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.560+0000 7f72d1600640 1 -- 172.21.15.67:0/3973656324 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72c005abe0 con 0x7f72d409aa30 2024-05-30T17:00:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.761+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72d4156470 con 0x7f72b406ddc0 2024-05-30T17:00:46.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.772+0000 7f72d1600640 1 -- 172.21.15.67:0/3973656324 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252390 (secure 0 0 0) 0x7f72c803c460 con 0x7f72b406ddc0 2024-05-30T17:00:46.776 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:46.777 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:00:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.777+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f72b406ddc0 msgr2=0x7f72b4070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.777+0000 7f72dae00640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f72b406ddc0 0x7f72b4070280 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f72c8034000 tx=0x7f72c802e000 comp rx=0 tx=0).stop 2024-05-30T17:00:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.777+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d409aa30 msgr2=0x7f72d415f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.777+0000 7f72dae00640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d409aa30 0x7f72d415f700 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f72c000d800 tx=0x7f72c000dcc0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.778+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 shutdown_connections 2024-05-30T17:00:46.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.778+0000 7f72dae00640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f72b406ddc0 0x7f72b4070280 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.778+0000 7f72dae00640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d415e790 0x7f72d41f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.778+0000 7f72dae00640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72d409aa30 0x7f72d415f700 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.778+0000 7f72dae00640 1 --2- 172.21.15.67:0/3973656324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d40711c0 0x7f72d4155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:46.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.778+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 >> 172.21.15.67:0/3973656324 conn(0x7f72d40b3ec0 msgr2=0x7f72d4159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:46.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.778+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 shutdown_connections 2024-05-30T17:00:46.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:46.778+0000 7f72dae00640 1 -- 172.21.15.67:0/3973656324 wait complete. 2024-05-30T17:00:46.797 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":611,"stamp":"2024-05-30T17:00:46.474663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":889020,"kb_used_data":11208,"kb_used_omap":12,"kb_used_meta":877683,"kb_avail":754085700,"statfs":{"total":773094113280,"available":772183756800,"internally_reserved":0,"allocated":11476992,"data_stored":5647228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":898747952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.755354"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149145+0000","last_change":"2024-05-30T17:00:36.759629+0000","last_active":"2024-05-30T17:00:38.149145+0000","last_peered":"2024-05-30T17:00:38.149145+0000","last_clean":"2024-05-30T17:00:38.149145+0000","last_became_active":"2024-05-30T17:00:36.759308+0000","last_became_peered":"2024-05-30T17:00:36.759308+0000","last_unstale":"2024-05-30T17:00:38.149145+0000","last_undegraded":"2024-05-30T17:00:38.149145+0000","last_fullsized":"2024-05-30T17:00:38.149145+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:09:42.812288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752224+0000","last_change":"2024-05-30T17:00:36.748181+0000","last_active":"2024-05-30T17:00:37.752224+0000","last_peered":"2024-05-30T17:00:37.752224+0000","last_clean":"2024-05-30T17:00:37.752224+0000","last_became_active":"2024-05-30T17:00:36.747392+0000","last_became_peered":"2024-05-30T17:00:36.747392+0000","last_unstale":"2024-05-30T17:00:37.752224+0000","last_undegraded":"2024-05-30T17:00:37.752224+0000","last_fullsized":"2024-05-30T17:00:37.752224+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149147+0000","last_change":"2024-05-30T17:00:36.764900+0000","last_active":"2024-05-30T17:00:38.149147+0000","last_peered":"2024-05-30T17:00:38.149147+0000","last_clean":"2024-05-30T17:00:38.149147+0000","last_became_active":"2024-05-30T17:00:36.764614+0000","last_became_peered":"2024-05-30T17:00:36.764614+0000","last_unstale":"2024-05-30T17:00:38.149147+0000","last_undegraded":"2024-05-30T17:00:38.149147+0000","last_fullsized":"2024-05-30T17:00:38.149147+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:41:02.666320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151034+0000","last_change":"2024-05-30T17:00:36.757850+0000","last_active":"2024-05-30T17:00:38.151034+0000","last_peered":"2024-05-30T17:00:38.151034+0000","last_clean":"2024-05-30T17:00:38.151034+0000","last_became_active":"2024-05-30T17:00:36.757545+0000","last_became_peered":"2024-05-30T17:00:36.757545+0000","last_unstale":"2024-05-30T17:00:38.151034+0000","last_undegraded":"2024-05-30T17:00:38.151034+0000","last_fullsized":"2024-05-30T17:00:38.151034+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:35:06.226926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151835+0000","last_change":"2024-05-30T17:00:36.763111+0000","last_active":"2024-05-30T17:00:38.151835+0000","last_peered":"2024-05-30T17:00:38.151835+0000","last_clean":"2024-05-30T17:00:38.151835+0000","last_became_active":"2024-05-30T17:00:36.761404+0000","last_became_peered":"2024-05-30T17:00:36.761404+0000","last_unstale":"2024-05-30T17:00:38.151835+0000","last_undegraded":"2024-05-30T17:00:38.151835+0000","last_fullsized":"2024-05-30T17:00:38.151835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:01.455423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149546+0000","last_change":"2024-05-30T17:00:36.751908+0000","last_active":"2024-05-30T17:00:38.149546+0000","last_peered":"2024-05-30T17:00:38.149546+0000","last_clean":"2024-05-30T17:00:38.149546+0000","last_became_active":"2024-05-30T17:00:36.751604+0000","last_became_peered":"2024-05-30T17:00:36.751604+0000","last_unstale":"2024-05-30T17:00:38.149546+0000","last_undegraded":"2024-05-30T17:00:38.149546+0000","last_fullsized":"2024-05-30T17:00:38.149546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149447+0000","last_change":"2024-05-30T17:00:36.748294+0000","last_active":"2024-05-30T17:00:38.149447+0000","last_peered":"2024-05-30T17:00:38.149447+0000","last_clean":"2024-05-30T17:00:38.149447+0000","last_became_active":"2024-05-30T17:00:36.747984+0000","last_became_peered":"2024-05-30T17:00:36.747984+0000","last_unstale":"2024-05-30T17:00:38.149447+0000","last_undegraded":"2024-05-30T17:00:38.149447+0000","last_fullsized":"2024-05-30T17:00:38.149447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:18:55.581202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751747+0000","last_change":"2024-05-30T17:00:36.751171+0000","last_active":"2024-05-30T17:00:37.751747+0000","last_peered":"2024-05-30T17:00:37.751747+0000","last_clean":"2024-05-30T17:00:37.751747+0000","last_became_active":"2024-05-30T17:00:36.747956+0000","last_became_peered":"2024-05-30T17:00:36.747956+0000","last_unstale":"2024-05-30T17:00:37.751747+0000","last_undegraded":"2024-05-30T17:00:37.751747+0000","last_fullsized":"2024-05-30T17:00:37.751747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:03:57.450595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760121+0000","last_change":"2024-05-30T17:00:36.762682+0000","last_active":"2024-05-30T17:00:37.760121+0000","last_peered":"2024-05-30T17:00:37.760121+0000","last_clean":"2024-05-30T17:00:37.760121+0000","last_became_active":"2024-05-30T17:00:36.762190+0000","last_became_peered":"2024-05-30T17:00:36.762190+0000","last_unstale":"2024-05-30T17:00:37.760121+0000","last_undegraded":"2024-05-30T17:00:37.760121+0000","last_fullsized":"2024-05-30T17:00:37.760121+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:33.645864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151899+0000","last_change":"2024-05-30T17:00:36.758850+0000","last_active":"2024-05-30T17:00:38.151899+0000","last_peered":"2024-05-30T17:00:38.151899+0000","last_clean":"2024-05-30T17:00:38.151899+0000","last_became_active":"2024-05-30T17:00:36.758522+0000","last_became_peered":"2024-05-30T17:00:36.758522+0000","last_unstale":"2024-05-30T17:00:38.151899+0000","last_undegraded":"2024-05-30T17:00:38.151899+0000","last_fullsized":"2024-05-30T17:00:38.151899+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:19:20.247961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149815+0000","last_change":"2024-05-30T17:00:36.750772+0000","last_active":"2024-05-30T17:00:38.149815+0000","last_peered":"2024-05-30T17:00:38.149815+0000","last_clean":"2024-05-30T17:00:38.149815+0000","last_became_active":"2024-05-30T17:00:36.750122+0000","last_became_peered":"2024-05-30T17:00:36.750122+0000","last_unstale":"2024-05-30T17:00:38.149815+0000","last_undegraded":"2024-05-30T17:00:38.149815+0000","last_fullsized":"2024-05-30T17:00:38.149815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761029+0000","last_change":"2024-05-30T17:00:36.751475+0000","last_active":"2024-05-30T17:00:37.761029+0000","last_peered":"2024-05-30T17:00:37.761029+0000","last_clean":"2024-05-30T17:00:37.761029+0000","last_became_active":"2024-05-30T17:00:36.751150+0000","last_became_peered":"2024-05-30T17:00:36.751150+0000","last_unstale":"2024-05-30T17:00:37.761029+0000","last_undegraded":"2024-05-30T17:00:37.761029+0000","last_fullsized":"2024-05-30T17:00:37.761029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:55:05.659729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150357+0000","last_change":"2024-05-30T17:00:36.755993+0000","last_active":"2024-05-30T17:00:38.150357+0000","last_peered":"2024-05-30T17:00:38.150357+0000","last_clean":"2024-05-30T17:00:38.150357+0000","last_became_active":"2024-05-30T17:00:36.755651+0000","last_became_peered":"2024-05-30T17:00:36.755651+0000","last_unstale":"2024-05-30T17:00:38.150357+0000","last_undegraded":"2024-05-30T17:00:38.150357+0000","last_fullsized":"2024-05-30T17:00:38.150357+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:37:32.519686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755319+0000","last_change":"2024-05-30T17:00:36.759921+0000","last_active":"2024-05-30T17:00:37.755319+0000","last_peered":"2024-05-30T17:00:37.755319+0000","last_clean":"2024-05-30T17:00:37.755319+0000","last_became_active":"2024-05-30T17:00:36.759644+0000","last_became_peered":"2024-05-30T17:00:36.759644+0000","last_unstale":"2024-05-30T17:00:37.755319+0000","last_undegraded":"2024-05-30T17:00:37.755319+0000","last_fullsized":"2024-05-30T17:00:37.755319+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:24:23.085972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150002+0000","last_change":"2024-05-30T17:00:36.765066+0000","last_active":"2024-05-30T17:00:38.150002+0000","last_peered":"2024-05-30T17:00:38.150002+0000","last_clean":"2024-05-30T17:00:38.150002+0000","last_became_active":"2024-05-30T17:00:36.764709+0000","last_became_peered":"2024-05-30T17:00:36.764709+0000","last_unstale":"2024-05-30T17:00:38.150002+0000","last_undegraded":"2024-05-30T17:00:38.150002+0000","last_fullsized":"2024-05-30T17:00:38.150002+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:03:46.211043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149705+0000","last_change":"2024-05-30T17:00:36.758779+0000","last_active":"2024-05-30T17:00:38.149705+0000","last_peered":"2024-05-30T17:00:38.149705+0000","last_clean":"2024-05-30T17:00:38.149705+0000","last_became_active":"2024-05-30T17:00:36.758402+0000","last_became_peered":"2024-05-30T17:00:36.758402+0000","last_unstale":"2024-05-30T17:00:38.149705+0000","last_undegraded":"2024-05-30T17:00:38.149705+0000","last_fullsized":"2024-05-30T17:00:38.149705+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752387+0000","last_change":"2024-05-30T17:00:36.759407+0000","last_active":"2024-05-30T17:00:37.752387+0000","last_peered":"2024-05-30T17:00:37.752387+0000","last_clean":"2024-05-30T17:00:37.752387+0000","last_became_active":"2024-05-30T17:00:36.759115+0000","last_became_peered":"2024-05-30T17:00:36.759115+0000","last_unstale":"2024-05-30T17:00:37.752387+0000","last_undegraded":"2024-05-30T17:00:37.752387+0000","last_fullsized":"2024-05-30T17:00:37.752387+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:59.107506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150755+0000","last_change":"2024-05-30T17:00:36.761710+0000","last_active":"2024-05-30T17:00:38.150755+0000","last_peered":"2024-05-30T17:00:38.150755+0000","last_clean":"2024-05-30T17:00:38.150755+0000","last_became_active":"2024-05-30T17:00:36.761414+0000","last_became_peered":"2024-05-30T17:00:36.761414+0000","last_unstale":"2024-05-30T17:00:38.150755+0000","last_undegraded":"2024-05-30T17:00:38.150755+0000","last_fullsized":"2024-05-30T17:00:38.150755+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:23:39.211805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755722+0000","last_change":"2024-05-30T17:00:36.767174+0000","last_active":"2024-05-30T17:00:37.755722+0000","last_peered":"2024-05-30T17:00:37.755722+0000","last_clean":"2024-05-30T17:00:37.755722+0000","last_became_active":"2024-05-30T17:00:36.766897+0000","last_became_peered":"2024-05-30T17:00:36.766897+0000","last_unstale":"2024-05-30T17:00:37.755722+0000","last_undegraded":"2024-05-30T17:00:37.755722+0000","last_fullsized":"2024-05-30T17:00:37.755722+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:26:37.028697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760586+0000","last_change":"2024-05-30T17:00:36.762646+0000","last_active":"2024-05-30T17:00:37.760586+0000","last_peered":"2024-05-30T17:00:37.760586+0000","last_clean":"2024-05-30T17:00:37.760586+0000","last_became_active":"2024-05-30T17:00:36.761684+0000","last_became_peered":"2024-05-30T17:00:36.761684+0000","last_unstale":"2024-05-30T17:00:37.760586+0000","last_undegraded":"2024-05-30T17:00:37.760586+0000","last_fullsized":"2024-05-30T17:00:37.760586+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:50.133298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755501+0000","last_change":"2024-05-30T17:00:36.765740+0000","last_active":"2024-05-30T17:00:37.755501+0000","last_peered":"2024-05-30T17:00:37.755501+0000","last_clean":"2024-05-30T17:00:37.755501+0000","last_became_active":"2024-05-30T17:00:36.764041+0000","last_became_peered":"2024-05-30T17:00:36.764041+0000","last_unstale":"2024-05-30T17:00:37.755501+0000","last_undegraded":"2024-05-30T17:00:37.755501+0000","last_fullsized":"2024-05-30T17:00:37.755501+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:41:02.666320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751731+0000","last_change":"2024-05-30T17:00:36.745182+0000","last_active":"2024-05-30T17:00:37.751731+0000","last_peered":"2024-05-30T17:00:37.751731+0000","last_clean":"2024-05-30T17:00:37.751731+0000","last_became_active":"2024-05-30T17:00:36.744869+0000","last_became_peered":"2024-05-30T17:00:36.744869+0000","last_unstale":"2024-05-30T17:00:37.751731+0000","last_undegraded":"2024-05-30T17:00:37.751731+0000","last_fullsized":"2024-05-30T17:00:37.751731+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:52:34.241585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751614+0000","last_change":"2024-05-30T17:00:36.746180+0000","last_active":"2024-05-30T17:00:37.751614+0000","last_peered":"2024-05-30T17:00:37.751614+0000","last_clean":"2024-05-30T17:00:37.751614+0000","last_became_active":"2024-05-30T17:00:36.745907+0000","last_became_peered":"2024-05-30T17:00:36.745907+0000","last_unstale":"2024-05-30T17:00:37.751614+0000","last_undegraded":"2024-05-30T17:00:37.751614+0000","last_fullsized":"2024-05-30T17:00:37.751614+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:43.488502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150152+0000","last_change":"2024-05-30T17:00:36.748364+0000","last_active":"2024-05-30T17:00:38.150152+0000","last_peered":"2024-05-30T17:00:38.150152+0000","last_clean":"2024-05-30T17:00:38.150152+0000","last_became_active":"2024-05-30T17:00:36.748051+0000","last_became_peered":"2024-05-30T17:00:36.748051+0000","last_unstale":"2024-05-30T17:00:38.150152+0000","last_undegraded":"2024-05-30T17:00:38.150152+0000","last_fullsized":"2024-05-30T17:00:38.150152+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:28.418390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151939+0000","last_change":"2024-05-30T17:00:36.754727+0000","last_active":"2024-05-30T17:00:38.151939+0000","last_peered":"2024-05-30T17:00:38.151939+0000","last_clean":"2024-05-30T17:00:38.151939+0000","last_became_active":"2024-05-30T17:00:36.753257+0000","last_became_peered":"2024-05-30T17:00:36.753257+0000","last_unstale":"2024-05-30T17:00:38.151939+0000","last_undegraded":"2024-05-30T17:00:38.151939+0000","last_fullsized":"2024-05-30T17:00:38.151939+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:36.271526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752343+0000","last_change":"2024-05-30T17:00:36.754136+0000","last_active":"2024-05-30T17:00:37.752343+0000","last_peered":"2024-05-30T17:00:37.752343+0000","last_clean":"2024-05-30T17:00:37.752343+0000","last_became_active":"2024-05-30T17:00:36.753838+0000","last_became_peered":"2024-05-30T17:00:36.753838+0000","last_unstale":"2024-05-30T17:00:37.752343+0000","last_undegraded":"2024-05-30T17:00:37.752343+0000","last_fullsized":"2024-05-30T17:00:37.752343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149843+0000","last_change":"2024-05-30T17:00:36.753152+0000","last_active":"2024-05-30T17:00:38.149843+0000","last_peered":"2024-05-30T17:00:38.149843+0000","last_clean":"2024-05-30T17:00:38.149843+0000","last_became_active":"2024-05-30T17:00:36.752867+0000","last_became_peered":"2024-05-30T17:00:36.752867+0000","last_unstale":"2024-05-30T17:00:38.149843+0000","last_undegraded":"2024-05-30T17:00:38.149843+0000","last_fullsized":"2024-05-30T17:00:38.149843+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:19:07.666127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152257+0000","last_change":"2024-05-30T17:00:36.761708+0000","last_active":"2024-05-30T17:00:38.152257+0000","last_peered":"2024-05-30T17:00:38.152257+0000","last_clean":"2024-05-30T17:00:38.152257+0000","last_became_active":"2024-05-30T17:00:36.760932+0000","last_became_peered":"2024-05-30T17:00:36.760932+0000","last_unstale":"2024-05-30T17:00:38.152257+0000","last_undegraded":"2024-05-30T17:00:38.152257+0000","last_fullsized":"2024-05-30T17:00:38.152257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:14:40.779484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752069+0000","last_change":"2024-05-30T17:00:36.755732+0000","last_active":"2024-05-30T17:00:37.752069+0000","last_peered":"2024-05-30T17:00:37.752069+0000","last_clean":"2024-05-30T17:00:37.752069+0000","last_became_active":"2024-05-30T17:00:36.755428+0000","last_became_peered":"2024-05-30T17:00:36.755428+0000","last_unstale":"2024-05-30T17:00:37.752069+0000","last_undegraded":"2024-05-30T17:00:37.752069+0000","last_fullsized":"2024-05-30T17:00:37.752069+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:14.194525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755307+0000","last_change":"2024-05-30T17:00:36.766442+0000","last_active":"2024-05-30T17:00:37.755307+0000","last_peered":"2024-05-30T17:00:37.755307+0000","last_clean":"2024-05-30T17:00:37.755307+0000","last_became_active":"2024-05-30T17:00:36.766171+0000","last_became_peered":"2024-05-30T17:00:36.766171+0000","last_unstale":"2024-05-30T17:00:37.755307+0000","last_undegraded":"2024-05-30T17:00:37.755307+0000","last_fullsized":"2024-05-30T17:00:37.755307+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:39:26.605255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150343+0000","last_change":"2024-05-30T17:00:36.759338+0000","last_active":"2024-05-30T17:00:38.150343+0000","last_peered":"2024-05-30T17:00:38.150343+0000","last_clean":"2024-05-30T17:00:38.150343+0000","last_became_active":"2024-05-30T17:00:36.759058+0000","last_became_peered":"2024-05-30T17:00:36.759058+0000","last_unstale":"2024-05-30T17:00:38.150343+0000","last_undegraded":"2024-05-30T17:00:38.150343+0000","last_fullsized":"2024-05-30T17:00:38.150343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:24:35.298988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148191+0000","last_change":"2024-05-30T17:00:36.770891+0000","last_active":"2024-05-30T17:00:38.148191+0000","last_peered":"2024-05-30T17:00:38.148191+0000","last_clean":"2024-05-30T17:00:38.148191+0000","last_became_active":"2024-05-30T17:00:36.768598+0000","last_became_peered":"2024-05-30T17:00:36.768598+0000","last_unstale":"2024-05-30T17:00:38.148191+0000","last_undegraded":"2024-05-30T17:00:38.148191+0000","last_fullsized":"2024-05-30T17:00:38.148191+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:06:07.554476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.756150+0000","last_change":"2024-05-30T17:00:36.758929+0000","last_active":"2024-05-30T17:00:37.756150+0000","last_peered":"2024-05-30T17:00:37.756150+0000","last_clean":"2024-05-30T17:00:37.756150+0000","last_became_active":"2024-05-30T17:00:36.758651+0000","last_became_peered":"2024-05-30T17:00:36.758651+0000","last_unstale":"2024-05-30T17:00:37.756150+0000","last_undegraded":"2024-05-30T17:00:37.756150+0000","last_fullsized":"2024-05-30T17:00:37.756150+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:03:48.926394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752501+0000","last_change":"2024-05-30T17:00:35.735865+0000","last_active":"2024-05-30T17:00:37.752501+0000","last_peered":"2024-05-30T17:00:37.752501+0000","last_clean":"2024-05-30T17:00:37.752501+0000","last_became_active":"2024-05-30T17:00:35.735526+0000","last_became_peered":"2024-05-30T17:00:35.735526+0000","last_unstale":"2024-05-30T17:00:37.752501+0000","last_undegraded":"2024-05-30T17:00:37.752501+0000","last_fullsized":"2024-05-30T17:00:37.752501+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:42:29.413638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151409+0000","last_change":"2024-05-30T17:00:36.775696+0000","last_active":"2024-05-30T17:00:38.151409+0000","last_peered":"2024-05-30T17:00:38.151409+0000","last_clean":"2024-05-30T17:00:38.151409+0000","last_became_active":"2024-05-30T17:00:36.774878+0000","last_became_peered":"2024-05-30T17:00:36.774878+0000","last_unstale":"2024-05-30T17:00:38.151409+0000","last_undegraded":"2024-05-30T17:00:38.151409+0000","last_fullsized":"2024-05-30T17:00:38.151409+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:43.497412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150045+0000","last_change":"2024-05-30T17:00:35.735197+0000","last_active":"2024-05-30T17:00:38.150045+0000","last_peered":"2024-05-30T17:00:38.150045+0000","last_clean":"2024-05-30T17:00:38.150045+0000","last_became_active":"2024-05-30T17:00:35.734913+0000","last_became_peered":"2024-05-30T17:00:35.734913+0000","last_unstale":"2024-05-30T17:00:38.150045+0000","last_undegraded":"2024-05-30T17:00:38.150045+0000","last_fullsized":"2024-05-30T17:00:38.150045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:58:37.442958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148864+0000","last_change":"2024-05-30T17:00:36.773283+0000","last_active":"2024-05-30T17:00:38.148864+0000","last_peered":"2024-05-30T17:00:38.148864+0000","last_clean":"2024-05-30T17:00:38.148864+0000","last_became_active":"2024-05-30T17:00:36.772999+0000","last_became_peered":"2024-05-30T17:00:36.772999+0000","last_unstale":"2024-05-30T17:00:38.148864+0000","last_undegraded":"2024-05-30T17:00:38.148864+0000","last_fullsized":"2024-05-30T17:00:38.148864+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:55:05.314618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752193+0000","last_change":"2024-05-30T17:00:35.729069+0000","last_active":"2024-05-30T17:00:37.752193+0000","last_peered":"2024-05-30T17:00:37.752193+0000","last_clean":"2024-05-30T17:00:37.752193+0000","last_became_active":"2024-05-30T17:00:35.728765+0000","last_became_peered":"2024-05-30T17:00:35.728765+0000","last_unstale":"2024-05-30T17:00:37.752193+0000","last_undegraded":"2024-05-30T17:00:37.752193+0000","last_fullsized":"2024-05-30T17:00:37.752193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:49:59.611402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.761260+0000","last_change":"2024-05-30T17:00:36.746352+0000","last_active":"2024-05-30T17:00:37.761260+0000","last_peered":"2024-05-30T17:00:37.761260+0000","last_clean":"2024-05-30T17:00:37.761260+0000","last_became_active":"2024-05-30T17:00:36.745894+0000","last_became_peered":"2024-05-30T17:00:36.745894+0000","last_unstale":"2024-05-30T17:00:37.761260+0000","last_undegraded":"2024-05-30T17:00:37.761260+0000","last_fullsized":"2024-05-30T17:00:37.761260+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:39:58.290648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752014+0000","last_change":"2024-05-30T17:00:35.734108+0000","last_active":"2024-05-30T17:00:37.752014+0000","last_peered":"2024-05-30T17:00:37.752014+0000","last_clean":"2024-05-30T17:00:37.752014+0000","last_became_active":"2024-05-30T17:00:35.733598+0000","last_became_peered":"2024-05-30T17:00:35.733598+0000","last_unstale":"2024-05-30T17:00:37.752014+0000","last_undegraded":"2024-05-30T17:00:37.752014+0000","last_fullsized":"2024-05-30T17:00:37.752014+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:14:22.323508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758033+0000","last_change":"2024-05-30T17:00:36.743969+0000","last_active":"2024-05-30T17:00:37.758033+0000","last_peered":"2024-05-30T17:00:37.758033+0000","last_clean":"2024-05-30T17:00:37.758033+0000","last_became_active":"2024-05-30T17:00:36.743574+0000","last_became_peered":"2024-05-30T17:00:36.743574+0000","last_unstale":"2024-05-30T17:00:37.758033+0000","last_undegraded":"2024-05-30T17:00:37.758033+0000","last_fullsized":"2024-05-30T17:00:37.758033+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:26:57.026949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148543+0000","last_change":"2024-05-30T17:00:35.731086+0000","last_active":"2024-05-30T17:00:38.148543+0000","last_peered":"2024-05-30T17:00:38.148543+0000","last_clean":"2024-05-30T17:00:38.148543+0000","last_became_active":"2024-05-30T17:00:35.730454+0000","last_became_peered":"2024-05-30T17:00:35.730454+0000","last_unstale":"2024-05-30T17:00:38.148543+0000","last_undegraded":"2024-05-30T17:00:38.148543+0000","last_fullsized":"2024-05-30T17:00:38.148543+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:34:24.659637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149522+0000","last_change":"2024-05-30T17:00:36.755868+0000","last_active":"2024-05-30T17:00:38.149522+0000","last_peered":"2024-05-30T17:00:38.149522+0000","last_clean":"2024-05-30T17:00:38.149522+0000","last_became_active":"2024-05-30T17:00:36.755420+0000","last_became_peered":"2024-05-30T17:00:36.755420+0000","last_unstale":"2024-05-30T17:00:38.149522+0000","last_undegraded":"2024-05-30T17:00:38.149522+0000","last_fullsized":"2024-05-30T17:00:38.149522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:56:53.876456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751802+0000","last_change":"2024-05-30T17:00:35.733623+0000","last_active":"2024-05-30T17:00:37.751802+0000","last_peered":"2024-05-30T17:00:37.751802+0000","last_clean":"2024-05-30T17:00:37.751802+0000","last_became_active":"2024-05-30T17:00:35.733325+0000","last_became_peered":"2024-05-30T17:00:35.733325+0000","last_unstale":"2024-05-30T17:00:37.751802+0000","last_undegraded":"2024-05-30T17:00:37.751802+0000","last_fullsized":"2024-05-30T17:00:37.751802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:14.320132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150246+0000","last_change":"2024-05-30T17:00:36.765167+0000","last_active":"2024-05-30T17:00:38.150246+0000","last_peered":"2024-05-30T17:00:38.150246+0000","last_clean":"2024-05-30T17:00:38.150246+0000","last_became_active":"2024-05-30T17:00:36.764873+0000","last_became_peered":"2024-05-30T17:00:36.764873+0000","last_unstale":"2024-05-30T17:00:38.150246+0000","last_undegraded":"2024-05-30T17:00:38.150246+0000","last_fullsized":"2024-05-30T17:00:38.150246+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:32:55.166729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151425+0000","last_change":"2024-05-30T17:00:35.734103+0000","last_active":"2024-05-30T17:00:38.151425+0000","last_peered":"2024-05-30T17:00:38.151425+0000","last_clean":"2024-05-30T17:00:38.151425+0000","last_became_active":"2024-05-30T17:00:35.733719+0000","last_became_peered":"2024-05-30T17:00:35.733719+0000","last_unstale":"2024-05-30T17:00:38.151425+0000","last_undegraded":"2024-05-30T17:00:38.151425+0000","last_fullsized":"2024-05-30T17:00:38.151425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:47:42.872961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759392+0000","last_change":"2024-05-30T17:00:36.747707+0000","last_active":"2024-05-30T17:00:37.759392+0000","last_peered":"2024-05-30T17:00:37.759392+0000","last_clean":"2024-05-30T17:00:37.759392+0000","last_became_active":"2024-05-30T17:00:36.744282+0000","last_became_peered":"2024-05-30T17:00:36.744282+0000","last_unstale":"2024-05-30T17:00:37.759392+0000","last_undegraded":"2024-05-30T17:00:37.759392+0000","last_fullsized":"2024-05-30T17:00:37.759392+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:57:48.052699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148317+0000","last_change":"2024-05-30T17:00:35.732470+0000","last_active":"2024-05-30T17:00:38.148317+0000","last_peered":"2024-05-30T17:00:38.148317+0000","last_clean":"2024-05-30T17:00:38.148317+0000","last_became_active":"2024-05-30T17:00:35.731250+0000","last_became_peered":"2024-05-30T17:00:35.731250+0000","last_unstale":"2024-05-30T17:00:38.148317+0000","last_undegraded":"2024-05-30T17:00:38.148317+0000","last_fullsized":"2024-05-30T17:00:38.148317+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:41:24.791603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752394+0000","last_change":"2024-05-30T17:00:36.742303+0000","last_active":"2024-05-30T17:00:37.752394+0000","last_peered":"2024-05-30T17:00:37.752394+0000","last_clean":"2024-05-30T17:00:37.752394+0000","last_became_active":"2024-05-30T17:00:36.741809+0000","last_became_peered":"2024-05-30T17:00:36.741809+0000","last_unstale":"2024-05-30T17:00:37.752394+0000","last_undegraded":"2024-05-30T17:00:37.752394+0000","last_fullsized":"2024-05-30T17:00:37.752394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:04:12.185624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149201+0000","last_change":"2024-05-30T17:00:35.728011+0000","last_active":"2024-05-30T17:00:38.149201+0000","last_peered":"2024-05-30T17:00:38.149201+0000","last_clean":"2024-05-30T17:00:38.149201+0000","last_became_active":"2024-05-30T17:00:35.727728+0000","last_became_peered":"2024-05-30T17:00:35.727728+0000","last_unstale":"2024-05-30T17:00:38.149201+0000","last_undegraded":"2024-05-30T17:00:38.149201+0000","last_fullsized":"2024-05-30T17:00:38.149201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:44:23.366065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760526+0000","last_change":"2024-05-30T17:00:36.747039+0000","last_active":"2024-05-30T17:00:37.760526+0000","last_peered":"2024-05-30T17:00:37.760526+0000","last_clean":"2024-05-30T17:00:37.760526+0000","last_became_active":"2024-05-30T17:00:36.745915+0000","last_became_peered":"2024-05-30T17:00:36.745915+0000","last_unstale":"2024-05-30T17:00:37.760526+0000","last_undegraded":"2024-05-30T17:00:37.760526+0000","last_fullsized":"2024-05-30T17:00:37.760526+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:21.006414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150639+0000","last_change":"2024-05-30T17:00:35.718080+0000","last_active":"2024-05-30T17:00:38.150639+0000","last_peered":"2024-05-30T17:00:38.150639+0000","last_clean":"2024-05-30T17:00:38.150639+0000","last_became_active":"2024-05-30T17:00:35.717705+0000","last_became_peered":"2024-05-30T17:00:35.717705+0000","last_unstale":"2024-05-30T17:00:38.150639+0000","last_undegraded":"2024-05-30T17:00:38.150639+0000","last_fullsized":"2024-05-30T17:00:38.150639+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:12:07.410732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152539+0000","last_change":"2024-05-30T17:00:36.754484+0000","last_active":"2024-05-30T17:00:38.152539+0000","last_peered":"2024-05-30T17:00:38.152539+0000","last_clean":"2024-05-30T17:00:38.152539+0000","last_became_active":"2024-05-30T17:00:36.754166+0000","last_became_peered":"2024-05-30T17:00:36.754166+0000","last_unstale":"2024-05-30T17:00:38.152539+0000","last_undegraded":"2024-05-30T17:00:38.152539+0000","last_fullsized":"2024-05-30T17:00:38.152539+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:33.091804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150419+0000","last_change":"2024-05-30T17:00:35.732982+0000","last_active":"2024-05-30T17:00:38.150419+0000","last_peered":"2024-05-30T17:00:38.150419+0000","last_clean":"2024-05-30T17:00:38.150419+0000","last_became_active":"2024-05-30T17:00:35.732114+0000","last_became_peered":"2024-05-30T17:00:35.732114+0000","last_unstale":"2024-05-30T17:00:38.150419+0000","last_undegraded":"2024-05-30T17:00:38.150419+0000","last_fullsized":"2024-05-30T17:00:38.150419+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:33.694752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150348+0000","last_change":"2024-05-30T17:00:36.758949+0000","last_active":"2024-05-30T17:00:38.150348+0000","last_peered":"2024-05-30T17:00:38.150348+0000","last_clean":"2024-05-30T17:00:38.150348+0000","last_became_active":"2024-05-30T17:00:36.758069+0000","last_became_peered":"2024-05-30T17:00:36.758069+0000","last_unstale":"2024-05-30T17:00:38.150348+0000","last_undegraded":"2024-05-30T17:00:38.150348+0000","last_fullsized":"2024-05-30T17:00:38.150348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:38:49.283720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760740+0000","last_change":"2024-05-30T17:00:35.731848+0000","last_active":"2024-05-30T17:00:37.760740+0000","last_peered":"2024-05-30T17:00:37.760740+0000","last_clean":"2024-05-30T17:00:37.760740+0000","last_became_active":"2024-05-30T17:00:35.731536+0000","last_became_peered":"2024-05-30T17:00:35.731536+0000","last_unstale":"2024-05-30T17:00:37.760740+0000","last_undegraded":"2024-05-30T17:00:37.760740+0000","last_fullsized":"2024-05-30T17:00:37.760740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:12:27.801691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150203+0000","last_change":"2024-05-30T17:00:36.762049+0000","last_active":"2024-05-30T17:00:38.150203+0000","last_peered":"2024-05-30T17:00:38.150203+0000","last_clean":"2024-05-30T17:00:38.150203+0000","last_became_active":"2024-05-30T17:00:36.761703+0000","last_became_peered":"2024-05-30T17:00:36.761703+0000","last_unstale":"2024-05-30T17:00:38.150203+0000","last_undegraded":"2024-05-30T17:00:38.150203+0000","last_fullsized":"2024-05-30T17:00:38.150203+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:46:34.225518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755711+0000","last_change":"2024-05-30T17:00:35.734783+0000","last_active":"2024-05-30T17:00:37.755711+0000","last_peered":"2024-05-30T17:00:37.755711+0000","last_clean":"2024-05-30T17:00:37.755711+0000","last_became_active":"2024-05-30T17:00:35.733971+0000","last_became_peered":"2024-05-30T17:00:35.733971+0000","last_unstale":"2024-05-30T17:00:37.755711+0000","last_undegraded":"2024-05-30T17:00:37.755711+0000","last_fullsized":"2024-05-30T17:00:37.755711+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:38:43.376431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148751+0000","last_change":"2024-05-30T17:00:36.761813+0000","last_active":"2024-05-30T17:00:38.148751+0000","last_peered":"2024-05-30T17:00:38.148751+0000","last_clean":"2024-05-30T17:00:38.148751+0000","last_became_active":"2024-05-30T17:00:36.755862+0000","last_became_peered":"2024-05-30T17:00:36.755862+0000","last_unstale":"2024-05-30T17:00:38.148751+0000","last_undegraded":"2024-05-30T17:00:38.148751+0000","last_fullsized":"2024-05-30T17:00:38.148751+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:08.876736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755595+0000","last_change":"2024-05-30T17:00:35.738764+0000","last_active":"2024-05-30T17:00:37.755595+0000","last_peered":"2024-05-30T17:00:37.755595+0000","last_clean":"2024-05-30T17:00:37.755595+0000","last_became_active":"2024-05-30T17:00:35.737679+0000","last_became_peered":"2024-05-30T17:00:35.737679+0000","last_unstale":"2024-05-30T17:00:37.755595+0000","last_undegraded":"2024-05-30T17:00:37.755595+0000","last_fullsized":"2024-05-30T17:00:37.755595+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:42:33.203512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760261+0000","last_change":"2024-05-30T17:00:36.744702+0000","last_active":"2024-05-30T17:00:37.760261+0000","last_peered":"2024-05-30T17:00:37.760261+0000","last_clean":"2024-05-30T17:00:37.760261+0000","last_became_active":"2024-05-30T17:00:36.744329+0000","last_became_peered":"2024-05-30T17:00:36.744329+0000","last_unstale":"2024-05-30T17:00:37.760261+0000","last_undegraded":"2024-05-30T17:00:37.760261+0000","last_fullsized":"2024-05-30T17:00:37.760261+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:21:27.452364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.759287+0000","last_change":"2024-05-30T17:00:35.729767+0000","last_active":"2024-05-30T17:00:37.759287+0000","last_peered":"2024-05-30T17:00:37.759287+0000","last_clean":"2024-05-30T17:00:37.759287+0000","last_became_active":"2024-05-30T17:00:35.729412+0000","last_became_peered":"2024-05-30T17:00:35.729412+0000","last_unstale":"2024-05-30T17:00:37.759287+0000","last_undegraded":"2024-05-30T17:00:37.759287+0000","last_fullsized":"2024-05-30T17:00:37.759287+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:09:47.793776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755890+0000","last_change":"2024-05-30T17:00:36.754800+0000","last_active":"2024-05-30T17:00:37.755890+0000","last_peered":"2024-05-30T17:00:37.755890+0000","last_clean":"2024-05-30T17:00:37.755890+0000","last_became_active":"2024-05-30T17:00:36.754501+0000","last_became_peered":"2024-05-30T17:00:36.754501+0000","last_unstale":"2024-05-30T17:00:37.755890+0000","last_undegraded":"2024-05-30T17:00:37.755890+0000","last_fullsized":"2024-05-30T17:00:37.755890+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:40:28.215068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":22,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148274+0000","last_change":"2024-05-30T17:00:35.729728+0000","last_active":"2024-05-30T17:00:38.148274+0000","last_peered":"2024-05-30T17:00:38.148274+0000","last_clean":"2024-05-30T17:00:38.148274+0000","last_became_active":"2024-05-30T17:00:35.723478+0000","last_became_peered":"2024-05-30T17:00:35.723478+0000","last_unstale":"2024-05-30T17:00:38.148274+0000","last_undegraded":"2024-05-30T17:00:38.148274+0000","last_fullsized":"2024-05-30T17:00:38.148274+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:46:24.942533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152237+0000","last_change":"2024-05-30T17:00:36.751676+0000","last_active":"2024-05-30T17:00:38.152237+0000","last_peered":"2024-05-30T17:00:38.152237+0000","last_clean":"2024-05-30T17:00:38.152237+0000","last_became_active":"2024-05-30T17:00:36.751355+0000","last_became_peered":"2024-05-30T17:00:36.751355+0000","last_unstale":"2024-05-30T17:00:38.152237+0000","last_undegraded":"2024-05-30T17:00:38.152237+0000","last_fullsized":"2024-05-30T17:00:38.152237+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:01:43.178806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760541+0000","last_change":"2024-05-30T17:00:35.734613+0000","last_active":"2024-05-30T17:00:37.760541+0000","last_peered":"2024-05-30T17:00:37.760541+0000","last_clean":"2024-05-30T17:00:37.760541+0000","last_became_active":"2024-05-30T17:00:35.734231+0000","last_became_peered":"2024-05-30T17:00:35.734231+0000","last_unstale":"2024-05-30T17:00:37.760541+0000","last_undegraded":"2024-05-30T17:00:37.760541+0000","last_fullsized":"2024-05-30T17:00:37.760541+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:06.120742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758055+0000","last_change":"2024-05-30T17:00:36.757383+0000","last_active":"2024-05-30T17:00:37.758055+0000","last_peered":"2024-05-30T17:00:37.758055+0000","last_clean":"2024-05-30T17:00:37.758055+0000","last_became_active":"2024-05-30T17:00:36.756264+0000","last_became_peered":"2024-05-30T17:00:36.756264+0000","last_unstale":"2024-05-30T17:00:37.758055+0000","last_undegraded":"2024-05-30T17:00:37.758055+0000","last_fullsized":"2024-05-30T17:00:37.758055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:32.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151230+0000","last_change":"2024-05-30T17:00:35.730141+0000","last_active":"2024-05-30T17:00:38.151230+0000","last_peered":"2024-05-30T17:00:38.151230+0000","last_clean":"2024-05-30T17:00:38.151230+0000","last_became_active":"2024-05-30T17:00:35.728948+0000","last_became_peered":"2024-05-30T17:00:35.728948+0000","last_unstale":"2024-05-30T17:00:38.151230+0000","last_undegraded":"2024-05-30T17:00:38.151230+0000","last_fullsized":"2024-05-30T17:00:38.151230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:18:13.200467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150264+0000","last_change":"2024-05-30T17:00:36.750999+0000","last_active":"2024-05-30T17:00:38.150264+0000","last_peered":"2024-05-30T17:00:38.150264+0000","last_clean":"2024-05-30T17:00:38.150264+0000","last_became_active":"2024-05-30T17:00:36.750499+0000","last_became_peered":"2024-05-30T17:00:36.750499+0000","last_unstale":"2024-05-30T17:00:38.150264+0000","last_undegraded":"2024-05-30T17:00:38.150264+0000","last_fullsized":"2024-05-30T17:00:38.150264+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:49:10.934086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751996+0000","last_change":"2024-05-30T17:00:35.733883+0000","last_active":"2024-05-30T17:00:37.751996+0000","last_peered":"2024-05-30T17:00:37.751996+0000","last_clean":"2024-05-30T17:00:37.751996+0000","last_became_active":"2024-05-30T17:00:35.733243+0000","last_became_peered":"2024-05-30T17:00:35.733243+0000","last_unstale":"2024-05-30T17:00:37.751996+0000","last_undegraded":"2024-05-30T17:00:37.751996+0000","last_fullsized":"2024-05-30T17:00:37.751996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:46:39.811373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755479+0000","last_change":"2024-05-30T17:00:36.763473+0000","last_active":"2024-05-30T17:00:37.755479+0000","last_peered":"2024-05-30T17:00:37.755479+0000","last_clean":"2024-05-30T17:00:37.755479+0000","last_became_active":"2024-05-30T17:00:36.763143+0000","last_became_peered":"2024-05-30T17:00:36.763143+0000","last_unstale":"2024-05-30T17:00:37.755479+0000","last_undegraded":"2024-05-30T17:00:37.755479+0000","last_fullsized":"2024-05-30T17:00:37.755479+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:09:42.812288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752192+0000","last_change":"2024-05-30T17:00:35.729083+0000","last_active":"2024-05-30T17:00:37.752192+0000","last_peered":"2024-05-30T17:00:37.752192+0000","last_clean":"2024-05-30T17:00:37.752192+0000","last_became_active":"2024-05-30T17:00:35.728830+0000","last_became_peered":"2024-05-30T17:00:35.728830+0000","last_unstale":"2024-05-30T17:00:37.752192+0000","last_undegraded":"2024-05-30T17:00:37.752192+0000","last_fullsized":"2024-05-30T17:00:37.752192+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:52.170762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755588+0000","last_change":"2024-05-30T17:00:36.764134+0000","last_active":"2024-05-30T17:00:37.755588+0000","last_peered":"2024-05-30T17:00:37.755588+0000","last_clean":"2024-05-30T17:00:37.755588+0000","last_became_active":"2024-05-30T17:00:36.763828+0000","last_became_peered":"2024-05-30T17:00:36.763828+0000","last_unstale":"2024-05-30T17:00:37.755588+0000","last_undegraded":"2024-05-30T17:00:37.755588+0000","last_fullsized":"2024-05-30T17:00:37.755588+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:06.014878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757948+0000","last_change":"2024-05-30T17:00:35.730608+0000","last_active":"2024-05-30T17:00:37.757948+0000","last_peered":"2024-05-30T17:00:37.757948+0000","last_clean":"2024-05-30T17:00:37.757948+0000","last_became_active":"2024-05-30T17:00:35.730310+0000","last_became_peered":"2024-05-30T17:00:35.730310+0000","last_unstale":"2024-05-30T17:00:37.757948+0000","last_undegraded":"2024-05-30T17:00:37.757948+0000","last_fullsized":"2024-05-30T17:00:37.757948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:13.646048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.754917+0000","last_change":"2024-05-30T17:00:36.763307+0000","last_active":"2024-05-30T17:00:37.754917+0000","last_peered":"2024-05-30T17:00:37.754917+0000","last_clean":"2024-05-30T17:00:37.754917+0000","last_became_active":"2024-05-30T17:00:36.763027+0000","last_became_peered":"2024-05-30T17:00:36.763027+0000","last_unstale":"2024-05-30T17:00:37.754917+0000","last_undegraded":"2024-05-30T17:00:37.754917+0000","last_fullsized":"2024-05-30T17:00:37.754917+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:25:08.876736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.757483+0000","last_change":"2024-05-30T17:00:35.709483+0000","last_active":"2024-05-30T17:00:37.757483+0000","last_peered":"2024-05-30T17:00:37.757483+0000","last_clean":"2024-05-30T17:00:37.757483+0000","last_became_active":"2024-05-30T17:00:35.708526+0000","last_became_peered":"2024-05-30T17:00:35.708526+0000","last_unstale":"2024-05-30T17:00:37.757483+0000","last_undegraded":"2024-05-30T17:00:37.757483+0000","last_fullsized":"2024-05-30T17:00:37.757483+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:40.251625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151273+0000","last_change":"2024-05-30T17:00:36.749874+0000","last_active":"2024-05-30T17:00:38.151273+0000","last_peered":"2024-05-30T17:00:38.151273+0000","last_clean":"2024-05-30T17:00:38.151273+0000","last_became_active":"2024-05-30T17:00:36.749228+0000","last_became_peered":"2024-05-30T17:00:36.749228+0000","last_unstale":"2024-05-30T17:00:38.151273+0000","last_undegraded":"2024-05-30T17:00:38.151273+0000","last_fullsized":"2024-05-30T17:00:38.151273+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:44.276755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.751571+0000","last_change":"2024-05-30T17:00:35.736336+0000","last_active":"2024-05-30T17:00:37.751571+0000","last_peered":"2024-05-30T17:00:37.751571+0000","last_clean":"2024-05-30T17:00:37.751571+0000","last_became_active":"2024-05-30T17:00:35.734451+0000","last_became_peered":"2024-05-30T17:00:35.734451+0000","last_unstale":"2024-05-30T17:00:37.751571+0000","last_undegraded":"2024-05-30T17:00:37.751571+0000","last_fullsized":"2024-05-30T17:00:37.751571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:16.292089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752599+0000","last_change":"2024-05-30T17:00:36.748422+0000","last_active":"2024-05-30T17:00:37.752599+0000","last_peered":"2024-05-30T17:00:37.752599+0000","last_clean":"2024-05-30T17:00:37.752599+0000","last_became_active":"2024-05-30T17:00:36.747900+0000","last_became_peered":"2024-05-30T17:00:36.747900+0000","last_unstale":"2024-05-30T17:00:37.752599+0000","last_undegraded":"2024-05-30T17:00:37.752599+0000","last_fullsized":"2024-05-30T17:00:37.752599+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:54:20.914158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.758807+0000","last_change":"2024-05-30T17:00:35.729597+0000","last_active":"2024-05-30T17:00:37.758807+0000","last_peered":"2024-05-30T17:00:37.758807+0000","last_clean":"2024-05-30T17:00:37.758807+0000","last_became_active":"2024-05-30T17:00:35.728519+0000","last_became_peered":"2024-05-30T17:00:35.728519+0000","last_unstale":"2024-05-30T17:00:37.758807+0000","last_undegraded":"2024-05-30T17:00:37.758807+0000","last_fullsized":"2024-05-30T17:00:37.758807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:30:12.232879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149106+0000","last_change":"2024-05-30T17:00:36.753305+0000","last_active":"2024-05-30T17:00:38.149106+0000","last_peered":"2024-05-30T17:00:38.149106+0000","last_clean":"2024-05-30T17:00:38.149106+0000","last_became_active":"2024-05-30T17:00:36.753023+0000","last_became_peered":"2024-05-30T17:00:36.753023+0000","last_unstale":"2024-05-30T17:00:38.149106+0000","last_undegraded":"2024-05-30T17:00:38.149106+0000","last_fullsized":"2024-05-30T17:00:38.149106+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:24:23.085972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.754450+0000","last_change":"2024-05-30T17:00:35.733609+0000","last_active":"2024-05-30T17:00:37.754450+0000","last_peered":"2024-05-30T17:00:37.754450+0000","last_clean":"2024-05-30T17:00:37.754450+0000","last_became_active":"2024-05-30T17:00:35.733292+0000","last_became_peered":"2024-05-30T17:00:35.733292+0000","last_unstale":"2024-05-30T17:00:37.754450+0000","last_undegraded":"2024-05-30T17:00:37.754450+0000","last_fullsized":"2024-05-30T17:00:37.754450+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:06:00.979924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149021+0000","last_change":"2024-05-30T17:00:36.759105+0000","last_active":"2024-05-30T17:00:38.149021+0000","last_peered":"2024-05-30T17:00:38.149021+0000","last_clean":"2024-05-30T17:00:38.149021+0000","last_became_active":"2024-05-30T17:00:36.758755+0000","last_became_peered":"2024-05-30T17:00:36.758755+0000","last_unstale":"2024-05-30T17:00:38.149021+0000","last_undegraded":"2024-05-30T17:00:38.149021+0000","last_fullsized":"2024-05-30T17:00:38.149021+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:39:26.605255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752339+0000","last_change":"2024-05-30T17:00:35.734441+0000","last_active":"2024-05-30T17:00:37.752339+0000","last_peered":"2024-05-30T17:00:37.752339+0000","last_clean":"2024-05-30T17:00:37.752339+0000","last_became_active":"2024-05-30T17:00:35.732710+0000","last_became_peered":"2024-05-30T17:00:35.732710+0000","last_unstale":"2024-05-30T17:00:37.752339+0000","last_undegraded":"2024-05-30T17:00:37.752339+0000","last_fullsized":"2024-05-30T17:00:37.752339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:15:15.063268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151374+0000","last_change":"2024-05-30T17:00:36.764319+0000","last_active":"2024-05-30T17:00:38.151374+0000","last_peered":"2024-05-30T17:00:38.151374+0000","last_clean":"2024-05-30T17:00:38.151374+0000","last_became_active":"2024-05-30T17:00:36.764034+0000","last_became_peered":"2024-05-30T17:00:36.764034+0000","last_unstale":"2024-05-30T17:00:38.151374+0000","last_undegraded":"2024-05-30T17:00:38.151374+0000","last_fullsized":"2024-05-30T17:00:38.151374+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:01:28.081874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.152390+0000","last_change":"2024-05-30T17:00:35.727312+0000","last_active":"2024-05-30T17:00:38.152390+0000","last_peered":"2024-05-30T17:00:38.152390+0000","last_clean":"2024-05-30T17:00:38.152390+0000","last_became_active":"2024-05-30T17:00:35.726820+0000","last_became_peered":"2024-05-30T17:00:35.726820+0000","last_unstale":"2024-05-30T17:00:38.152390+0000","last_undegraded":"2024-05-30T17:00:38.152390+0000","last_fullsized":"2024-05-30T17:00:38.152390+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:52:40.105333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755538+0000","last_change":"2024-05-30T17:00:36.755404+0000","last_active":"2024-05-30T17:00:37.755538+0000","last_peered":"2024-05-30T17:00:37.755538+0000","last_clean":"2024-05-30T17:00:37.755538+0000","last_became_active":"2024-05-30T17:00:36.755137+0000","last_became_peered":"2024-05-30T17:00:36.755137+0000","last_unstale":"2024-05-30T17:00:37.755538+0000","last_undegraded":"2024-05-30T17:00:37.755538+0000","last_fullsized":"2024-05-30T17:00:37.755538+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:55:07.410908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.752300+0000","last_change":"2024-05-30T17:00:35.727143+0000","last_active":"2024-05-30T17:00:37.752300+0000","last_peered":"2024-05-30T17:00:37.752300+0000","last_clean":"2024-05-30T17:00:37.752300+0000","last_became_active":"2024-05-30T17:00:35.725458+0000","last_became_peered":"2024-05-30T17:00:35.725458+0000","last_unstale":"2024-05-30T17:00:37.752300+0000","last_undegraded":"2024-05-30T17:00:37.752300+0000","last_fullsized":"2024-05-30T17:00:37.752300+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:44:49.887792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.760276+0000","last_change":"2024-05-30T17:00:36.756498+0000","last_active":"2024-05-30T17:00:37.760276+0000","last_peered":"2024-05-30T17:00:37.760276+0000","last_clean":"2024-05-30T17:00:37.760276+0000","last_became_active":"2024-05-30T17:00:36.755460+0000","last_became_peered":"2024-05-30T17:00:36.755460+0000","last_unstale":"2024-05-30T17:00:37.760276+0000","last_undegraded":"2024-05-30T17:00:37.760276+0000","last_fullsized":"2024-05-30T17:00:37.760276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:01.424865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150911+0000","last_change":"2024-05-30T17:00:35.731535+0000","last_active":"2024-05-30T17:00:38.150911+0000","last_peered":"2024-05-30T17:00:38.150911+0000","last_clean":"2024-05-30T17:00:38.150911+0000","last_became_active":"2024-05-30T17:00:35.730893+0000","last_became_peered":"2024-05-30T17:00:35.730893+0000","last_unstale":"2024-05-30T17:00:38.150911+0000","last_undegraded":"2024-05-30T17:00:38.150911+0000","last_fullsized":"2024-05-30T17:00:38.150911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:31:45.698539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.754917+0000","last_change":"2024-05-30T17:00:36.754985+0000","last_active":"2024-05-30T17:00:37.754917+0000","last_peered":"2024-05-30T17:00:37.754917+0000","last_clean":"2024-05-30T17:00:37.754917+0000","last_became_active":"2024-05-30T17:00:36.754666+0000","last_became_peered":"2024-05-30T17:00:36.754666+0000","last_unstale":"2024-05-30T17:00:37.754917+0000","last_undegraded":"2024-05-30T17:00:37.754917+0000","last_fullsized":"2024-05-30T17:00:37.754917+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:55:05.314618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.150387+0000","last_change":"2024-05-30T17:00:35.724537+0000","last_active":"2024-05-30T17:00:38.150387+0000","last_peered":"2024-05-30T17:00:38.150387+0000","last_clean":"2024-05-30T17:00:38.150387+0000","last_became_active":"2024-05-30T17:00:35.724245+0000","last_became_peered":"2024-05-30T17:00:35.724245+0000","last_unstale":"2024-05-30T17:00:38.150387+0000","last_undegraded":"2024-05-30T17:00:38.150387+0000","last_fullsized":"2024-05-30T17:00:38.150387+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:12:43.263919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.151226+0000","last_change":"2024-05-30T17:00:36.761666+0000","last_active":"2024-05-30T17:00:38.151226+0000","last_peered":"2024-05-30T17:00:38.151226+0000","last_clean":"2024-05-30T17:00:38.151226+0000","last_became_active":"2024-05-30T17:00:36.760872+0000","last_became_peered":"2024-05-30T17:00:36.760872+0000","last_unstale":"2024-05-30T17:00:38.151226+0000","last_undegraded":"2024-05-30T17:00:38.151226+0000","last_fullsized":"2024-05-30T17:00:38.151226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:12:08.423568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.755078+0000","last_change":"2024-05-30T17:00:35.732840+0000","last_active":"2024-05-30T17:00:37.755078+0000","last_peered":"2024-05-30T17:00:37.755078+0000","last_clean":"2024-05-30T17:00:37.755078+0000","last_became_active":"2024-05-30T17:00:35.732525+0000","last_became_peered":"2024-05-30T17:00:35.732525+0000","last_unstale":"2024-05-30T17:00:37.755078+0000","last_undegraded":"2024-05-30T17:00:37.755078+0000","last_fullsized":"2024-05-30T17:00:37.755078+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:35.909328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:37.756151+0000","last_change":"2024-05-30T17:00:36.765245+0000","last_active":"2024-05-30T17:00:37.756151+0000","last_peered":"2024-05-30T17:00:37.756151+0000","last_clean":"2024-05-30T17:00:37.756151+0000","last_became_active":"2024-05-30T17:00:36.764662+0000","last_became_peered":"2024-05-30T17:00:36.764662+0000","last_unstale":"2024-05-30T17:00:37.756151+0000","last_undegraded":"2024-05-30T17:00:37.756151+0000","last_fullsized":"2024-05-30T17:00:37.756151+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:35.687353+0000","last_clean_scrub_stamp":"2024-05-30T17:00:35.687353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:22:49.059539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.149819+0000","last_change":"2024-05-30T17:00:35.729629+0000","last_active":"2024-05-30T17:00:38.149819+0000","last_peered":"2024-05-30T17:00:38.149819+0000","last_clean":"2024-05-30T17:00:38.149819+0000","last_became_active":"2024-05-30T17:00:35.729282+0000","last_became_peered":"2024-05-30T17:00:35.729282+0000","last_unstale":"2024-05-30T17:00:38.149819+0000","last_undegraded":"2024-05-30T17:00:38.149819+0000","last_fullsized":"2024-05-30T17:00:38.149819+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:00:34.674517+0000","last_clean_scrub_stamp":"2024-05-30T17:00:34.674517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:41:43.450197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":330,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-30T17:00:38.148934+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:00:38.148934+0000","last_peered":"2024-05-30T17:00:38.148934+0000","last_clean":"2024-05-30T17:00:38.148934+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:00:38.148934+0000","last_undegraded":"2024-05-30T17:00:38.148934+0000","last_fullsized":"2024-05-30T17:00:38.148934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509648,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130936,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240904,"statfs":{"total":96636764160,"available":96502685696,"internally_reserved":0,"allocated":1286144,"data_stored":558361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509648,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86528,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285312,"statfs":{"total":96636764160,"available":96548159488,"internally_reserved":0,"allocated":1298432,"data_stored":559445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509649,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126780,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245060,"statfs":{"total":96636764160,"available":96506941440,"internally_reserved":0,"allocated":1294336,"data_stored":559343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509649,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144820,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227020,"statfs":{"total":96636764160,"available":96488468480,"internally_reserved":0,"allocated":1282048,"data_stored":557743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509648,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126768,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245072,"statfs":{"total":96636764160,"available":96506953728,"internally_reserved":0,"allocated":1277952,"data_stored":557709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542351,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127356,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244484,"statfs":{"total":96636764160,"available":96506351616,"internally_reserved":0,"allocated":1880064,"data_stored":1148189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76788,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295052,"statfs":{"total":96636764160,"available":96558133248,"internally_reserved":0,"allocated":1282048,"data_stored":557779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542352,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69044,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302796,"statfs":{"total":96636764160,"available":96566063104,"internally_reserved":0,"allocated":1875968,"data_stored":1148659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:00:46.801 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T17:00:46.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-30T17:00:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.060+0000 7f6fab200640 1 Processor -- start 2024-05-30T17:00:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.060+0000 7f6fab200640 1 -- start start 2024-05-30T17:00:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.061+0000 7f6fab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fa40ce5d0 0x7f6fa40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.061+0000 7f6fab200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6fa40b3d60 0x7f6fa40b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.061+0000 7f6fab200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa40b3580 con 0x7f6fa40ce1f0 2024-05-30T17:00:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.061+0000 7f6fab200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa415f490 con 0x7f6fa40b3d60 2024-05-30T17:00:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.061+0000 7f6fab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa415f770 con 0x7f6fa40ce5d0 2024-05-30T17:00:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.062+0000 7f6fa9e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6fa40ce1f0 0x7f6fa415ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56716/0 (socket says 172.21.15.67:56716) 2024-05-30T17:00:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.062+0000 7f6fa9e00640 1 -- 172.21.15.67:0/3913980680 learned_addr learned my addr 172.21.15.67:0/3913980680 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.062+0000 7f6fa9400640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fa40ce5d0 0x7f6fa40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.062+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286950167 0 0) 0x7f6fa40b3580 con 0x7f6fa40ce1f0 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.062+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f8c003520 con 0x7f6fa40ce1f0 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.062+0000 7f6faa800640 1 --2- 172.21.15.67:0/3913980680 >> v2:172.21.15.169:3300/0 conn(0x7f6fa40b3d60 0x7f6fa40b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3897107595 0 0) 0x7f6f8c003520 con 0x7f6fa40ce1f0 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fa40ce5d0 msgr2=0x7f6fa40b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fa40ce5d0 0x7f6fa40b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 >> v2:172.21.15.169:3300/0 conn(0x7f6fa40b3d60 msgr2=0x7f6fa40b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 --2- 172.21.15.67:0/3913980680 >> v2:172.21.15.169:3300/0 conn(0x7f6fa40b3d60 0x7f6fa40b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa415fa50 con 0x7f6fa40ce1f0 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f6f98001670 con 0x7f6fa40ce1f0 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa9400640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fa40ce5d0 0x7f6fa40b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 >> v1:172.21.15.67:6789/0 conn(0x7f6fa40ce1f0 legacy=0x7f6fa415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f8c0039b0 0x7f6f8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f8c004800 0x7f6f8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fa40ce5d0 0x7f6f8c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa415f490 con 0x7f6f8c004800 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa415f770 con 0x7f6f8c0039b0 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c00bc00 con 0x7f6fa40ce5d0 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.063+0000 7f6fa9400640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f8c004800 0x7f6f8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.064+0000 7f6fa9e00640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f8c0039b0 0x7f6f8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.064+0000 7f6fa9400640 1 -- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fa40ce5d0 msgr2=0x7f6f8c005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.064+0000 7f6fa9400640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fa40ce5d0 0x7f6f8c005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.064+0000 7f6fa9400640 1 -- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f8c0039b0 msgr2=0x7f6f8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.064+0000 7f6fa9400640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f8c0039b0 0x7f6f8c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.064+0000 7f6fa9400640 1 -- 172.21.15.67:0/3913980680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c003520 con 0x7f6f8c004800 2024-05-30T17:00:47.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.064+0000 7f6fa9e00640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f8c0039b0 0x7f6f8c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.065+0000 7f6fa9400640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f8c004800 0x7f6f8c004c80 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6f900098a0 tx=0x7f6f8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adddf1bfd4dfe109 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.065+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000f040 con 0x7f6f8c004800 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.065+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9002ad00 con 0x7f6f8c004800 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.065+0000 7f6fa8a00640 1 -- 172.21.15.67:0/3913980680 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90004250 con 0x7f6f8c004800 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.066+0000 7f6fab200640 1 -- 172.21.15.67:0/3913980680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f8c004800 msgr2=0x7f6f8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.066+0000 7f6fab200640 1 --2- 172.21.15.67:0/3913980680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f8c004800 0x7f6f8c004c80 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6f900098a0 tx=0x7f6f8c005830 comp rx=0 tx=0).stop 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.066+0000 7f6faa800640 1 -- 172.21.15.67:0/3913980680 reap_dead start 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.066+0000 7f6fab200640 1 -- 172.21.15.67:0/3913980680 shutdown_connections 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.066+0000 7f6fab200640 1 -- 172.21.15.67:0/3913980680 >> 172.21.15.67:0/3913980680 conn(0x7f6fa406b900 msgr2=0x7f6fa406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.066+0000 7f6fab200640 1 -- 172.21.15.67:0/3913980680 shutdown_connections 2024-05-30T17:00:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.066+0000 7f6fab200640 1 -- 172.21.15.67:0/3913980680 wait complete. 2024-05-30T17:00:47.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.068+0000 7f6fab200640 1 Processor -- start 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.068+0000 7f6fab200640 1 -- start start 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.069+0000 7f6fab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f8c004800 0x7f6fa4147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.069+0000 7f6fab200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fa415a400 0x7f6fa415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.069+0000 7f6fab200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fa4159350 0x7f6fa41597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.069+0000 7f6fab200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa40b4f50 con 0x7f6fa415a400 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.069+0000 7f6fab200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa40b4dd0 con 0x7f6fa4159350 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.069+0000 7f6fab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa40b50d0 con 0x7f6f8c004800 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.069+0000 7f6fa9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fa415a400 0x7f6fa415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.069+0000 7f6fa9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fa415a400 0x7f6fa415a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53210/0 (socket says 172.21.15.67:53210) 2024-05-30T17:00:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.069+0000 7f6fa9400640 1 -- 172.21.15.67:0/1523148686 learned_addr learned my addr 172.21.15.67:0/1523148686 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:47.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6faa800640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fa4159350 0x7f6fa41597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6fa9400640 1 -- 172.21.15.67:0/1523148686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f8c004800 msgr2=0x7f6fa4147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6fa9400640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f8c004800 0x7f6fa4147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6fa9400640 1 -- 172.21.15.67:0/1523148686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fa4159350 msgr2=0x7f6fa41597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6fa9400640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fa4159350 0x7f6fa41597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6fa9400640 1 -- 172.21.15.67:0/1523148686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa41f5360 con 0x7f6fa415a400 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6fa9400640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fa415a400 0x7f6fa415a880 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6f90034040 tx=0x7f6f90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6f9e000640 1 -- 172.21.15.67:0/1523148686 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90041070 con 0x7f6fa415a400 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6f9e000640 1 -- 172.21.15.67:0/1523148686 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9000f000 con 0x7f6fa415a400 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6f9e000640 1 -- 172.21.15.67:0/1523148686 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90003c50 con 0x7f6fa415a400 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.070+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa4147750 con 0x7f6fa415a400 2024-05-30T17:00:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.071+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa4147cd0 con 0x7f6fa415a400 2024-05-30T17:00:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.072+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa40689e0 con 0x7f6fa415a400 2024-05-30T17:00:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.073+0000 7f6f9e000640 1 -- 172.21.15.67:0/1523148686 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f6f90044070 con 0x7f6fa415a400 2024-05-30T17:00:47.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.074+0000 7f6f9e000640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f8406ddf0 0x7f6f840702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.074+0000 7f6f9e000640 1 -- 172.21.15.67:0/1523148686 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f6f900aae30 con 0x7f6fa415a400 2024-05-30T17:00:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.074+0000 7f6fa9e00640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f8406ddf0 0x7f6f840702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.075+0000 7f6fa9e00640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f8406ddf0 0x7f6f840702b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6f98001860 tx=0x7f6f98009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.079+0000 7f6f9e000640 1 -- 172.21.15.67:0/1523148686 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fa41f5360 con 0x7f6fa415a400 2024-05-30T17:00:47.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.357+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6fa40ca070 con 0x7f6fa415a400 2024-05-30T17:00:47.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.362+0000 7f6f9e000640 1 -- 172.21.15.67:0/1523148686 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f6f9007acb0 con 0x7f6fa415a400 2024-05-30T17:00:47.363 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-05-30T17:00:47.364 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T17:00:47.364 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-05-30T17:00:47.364 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-05-30T17:00:47.364 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-30T17:00:47.364 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-05-30T17:00:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.365+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f8406ddf0 msgr2=0x7f6f840702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.365+0000 7f6fab200640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f8406ddf0 0x7f6f840702b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6f98001860 tx=0x7f6f98009040 comp rx=0 tx=0).stop 2024-05-30T17:00:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.365+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fa415a400 msgr2=0x7f6fa415a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.365+0000 7f6fab200640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fa415a400 0x7f6fa415a880 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6f90034040 tx=0x7f6f90004730 comp rx=0 tx=0).stop 2024-05-30T17:00:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.366+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 shutdown_connections 2024-05-30T17:00:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.366+0000 7f6fab200640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6f8406ddf0 0x7f6f840702b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.366+0000 7f6fab200640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fa4159350 0x7f6fa41597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.366+0000 7f6fab200640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fa415a400 0x7f6fa415a880 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.366+0000 7f6fab200640 1 --2- 172.21.15.67:0/1523148686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f8c004800 0x7f6fa4147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.366+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 >> 172.21.15.67:0/1523148686 conn(0x7f6fa406b900 msgr2=0x7f6fa415e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.366+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 shutdown_connections 2024-05-30T17:00:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.366+0000 7f6fab200640 1 -- 172.21.15.67:0/1523148686 wait complete. 2024-05-30T17:00:47.387 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:00:47.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.588+0000 7f290d800640 1 Processor -- start 2024-05-30T17:00:47.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.588+0000 7f290d800640 1 -- start start 2024-05-30T17:00:47.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.590+0000 7f290d800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f290815b3a0 0x7f29080b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.590+0000 7f290d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908073310 0x7f290815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.590+0000 7f290d800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29080afd60 con 0x7f290806dc20 2024-05-30T17:00:47.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.590+0000 7f290d800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f290806f980 con 0x7f290815b3a0 2024-05-30T17:00:47.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.590+0000 7f290d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f29080ae0c0 con 0x7f2908073310 2024-05-30T17:00:47.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.590+0000 7f28ff400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f290806dc20 0x7f2908072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56718/0 (socket says 172.21.15.67:56718) 2024-05-30T17:00:47.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.590+0000 7f28ff400640 1 -- 172.21.15.67:0/3729172780 learned_addr learned my addr 172.21.15.67:0/3729172780 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003728698 0 0) 0x7f29080afd60 con 0x7f290806dc20 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907e00640 1 --2- 172.21.15.67:0/3729172780 >> v2:172.21.15.169:3300/0 conn(0x7f290815b3a0 0x7f29080b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28ec003520 con 0x7f290806dc20 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f290ce00640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908073310 0x7f290815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1976586363 0 0) 0x7f28ec003520 con 0x7f290806dc20 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908073310 msgr2=0x7f290815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908073310 0x7f290815f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 >> v2:172.21.15.169:3300/0 conn(0x7f290815b3a0 msgr2=0x7f29080b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 --2- 172.21.15.67:0/3729172780 >> v2:172.21.15.169:3300/0 conn(0x7f290815b3a0 0x7f29080b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29080af7a0 con 0x7f290806dc20 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f28f00019f0 con 0x7f290806dc20 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 >> v1:172.21.15.67:6789/0 conn(0x7f290806dc20 legacy=0x7f2908072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28ec0039b0 0x7f28ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908073310 0x7f28ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28ec004d40 0x7f28ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f290806f980 con 0x7f28ec004d40 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f29080ae0c0 con 0x7f28ec0039b0 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.591+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f28ec00b9c0 con 0x7f2908073310 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.592+0000 7f290ce00640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28ec004d40 0x7f28ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.592+0000 7f2907e00640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28ec0039b0 0x7f28ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.592+0000 7f28ff400640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908073310 0x7f28ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.592+0000 7f290ce00640 1 -- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908073310 msgr2=0x7f28ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.592+0000 7f290ce00640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908073310 0x7f28ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.592+0000 7f290ce00640 1 -- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28ec0039b0 msgr2=0x7f28ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.592+0000 7f290ce00640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28ec0039b0 0x7f28ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.592+0000 7f290ce00640 1 -- 172.21.15.67:0/3729172780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec003520 con 0x7f28ec004d40 2024-05-30T17:00:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.592+0000 7f28ff400640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2908073310 0x7f28ec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.593+0000 7f290ce00640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28ec004d40 0x7f28ec0051c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2900009ac0 tx=0x7f28ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1ad2a5258dabe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900036070 con 0x7f28ec004d40 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2900031400 con 0x7f28ec004d40 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f2907400640 1 -- 172.21.15.67:0/3729172780 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900009690 con 0x7f28ec004d40 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f290d800640 1 -- 172.21.15.67:0/3729172780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28ec004d40 msgr2=0x7f28ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f290d800640 1 --2- 172.21.15.67:0/3729172780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28ec004d40 0x7f28ec0051c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2900009ac0 tx=0x7f28ec005830 comp rx=0 tx=0).stop 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f290ce00640 1 -- 172.21.15.67:0/3729172780 reap_dead start 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f290d800640 1 -- 172.21.15.67:0/3729172780 shutdown_connections 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f290d800640 1 -- 172.21.15.67:0/3729172780 >> 172.21.15.67:0/3729172780 conn(0x7f29080b1fc0 msgr2=0x7f29080b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f290d800640 1 -- 172.21.15.67:0/3729172780 shutdown_connections 2024-05-30T17:00:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.594+0000 7f290d800640 1 -- 172.21.15.67:0/3729172780 wait complete. 2024-05-30T17:00:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.596+0000 7f290d800640 1 Processor -- start 2024-05-30T17:00:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.596+0000 7f290d800640 1 -- start start 2024-05-30T17:00:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.597+0000 7f290d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29081f77b0 0x7f29081f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.597+0000 7f290d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29081f7b90 0x7f29080caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.597+0000 7f290d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29080cb000 0x7f2908099090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.597+0000 7f290d800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f290815f9a0 con 0x7f29080cb000 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.597+0000 7f290d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f290815b780 con 0x7f29081f7b90 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.597+0000 7f290d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f290815fb20 con 0x7f29081f77b0 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.597+0000 7f290ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29080cb000 0x7f2908099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f290ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29080cb000 0x7f2908099090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53230/0 (socket says 172.21.15.67:53230) 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f290ce00640 1 -- 172.21.15.67:0/1473242068 learned_addr learned my addr 172.21.15.67:0/1473242068 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f28ff400640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29081f7b90 0x7f29080caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f2907e00640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29081f77b0 0x7f29081f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f290ce00640 1 -- 172.21.15.67:0/1473242068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29081f77b0 msgr2=0x7f29081f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f290ce00640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29081f77b0 0x7f29081f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f290ce00640 1 -- 172.21.15.67:0/1473242068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29081f7b90 msgr2=0x7f29080caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f290ce00640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29081f7b90 0x7f29080caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f290ce00640 1 -- 172.21.15.67:0/1473242068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29080c9a20 con 0x7f29080cb000 2024-05-30T17:00:47.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.598+0000 7f290ce00640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29080cb000 0x7f2908099090 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2900009be0 tx=0x7f2900004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:47.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.599+0000 7f2904c00640 1 -- 172.21.15.67:0/1473242068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900036040 con 0x7f29080cb000 2024-05-30T17:00:47.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.599+0000 7f2904c00640 1 -- 172.21.15.67:0/1473242068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2900041070 con 0x7f29080cb000 2024-05-30T17:00:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.599+0000 7f2904c00640 1 -- 172.21.15.67:0/1473242068 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900009690 con 0x7f29080cb000 2024-05-30T17:00:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.599+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29080c9d00 con 0x7f29080cb000 2024-05-30T17:00:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.599+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29080ca2c0 con 0x7f29080cb000 2024-05-30T17:00:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.600+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29080689e0 con 0x7f29080cb000 2024-05-30T17:00:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.601+0000 7f2904c00640 1 -- 172.21.15.67:0/1473242068 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f2900003a80 con 0x7f29080cb000 2024-05-30T17:00:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.602+0000 7f2904c00640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28e406ddc0 0x7f28e4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.602+0000 7f2904c00640 1 -- 172.21.15.67:0/1473242068 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f29000a9ec0 con 0x7f29080cb000 2024-05-30T17:00:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.602+0000 7f2907e00640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28e406ddc0 0x7f28e4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.603+0000 7f2907e00640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28e406ddc0 0x7f28e4070280 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009a10 tx=0x7f28f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.607+0000 7f2904c00640 1 -- 172.21.15.67:0/1473242068 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f290004a020 con 0x7f29080cb000 2024-05-30T17:00:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.848+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29080b2e90 con 0x7f29080cb000 2024-05-30T17:00:47.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.850+0000 7f2904c00640 1 -- 172.21.15.67:0/1473242068 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5189 (secure 0 0 0) 0x7f2900079660 con 0x7f29080cb000 2024-05-30T17:00:47.850 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:00:47.851 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":64,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2641805294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2641805294},{"type":"v1","addr":"172.21.15.67:6837","nonce":2641805294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2609681849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2609681849},{"type":"v1","addr":"172.21.15.169:6834","nonce":2609681849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2372354106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2372354106},{"type":"v1","addr":"172.21.15.169:6835","nonce":2372354106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:00:36.172699+0000","modified":"2024-05-30T17:00:39.271296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7902":{"gid":7902,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6836/2120049133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2120049133},{"type":"v1","addr":"172.21.15.67:6836","nonce":2120049133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7902,"qdb_cluster":[7902]},"id":7}]} 2024-05-30T17:00:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 64 2024-05-30T17:00:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28e406ddc0 msgr2=0x7f28e4070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28e406ddc0 0x7f28e4070280 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009a10 tx=0x7f28f8008040 comp rx=0 tx=0).stop 2024-05-30T17:00:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29080cb000 msgr2=0x7f2908099090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29080cb000 0x7f2908099090 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2900009be0 tx=0x7f2900004730 comp rx=0 tx=0).stop 2024-05-30T17:00:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 shutdown_connections 2024-05-30T17:00:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28e406ddc0 0x7f28e4070280 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29080cb000 0x7f2908099090 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29081f7b90 0x7f29080caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 --2- 172.21.15.67:0/1473242068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29081f77b0 0x7f29081f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:47.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.852+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 >> 172.21.15.67:0/1473242068 conn(0x7f29080b1fc0 msgr2=0x7f290806eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:47.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.853+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 shutdown_connections 2024-05-30T17:00:47.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:47.853+0000 7f290d800640 1 -- 172.21.15.67:0/1473242068 wait complete. 2024-05-30T17:00:48.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-30T17:00:36.172699+0000', 'modified': '2024-05-30T17:00:39.271296+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7902}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7902': {'gid': 7902, 'name': 'c', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.67:6836/2120049133', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6834', 'nonce': 2120049133}, {'type': 'v1', 'addr': '172.21.15.67:6836', 'nonce': 2120049133}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7902, 'qdb_cluster': [7902]} 2024-05-30T17:00:48.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-30T17:00:48.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-30T17:00:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.494+0000 7f846a000640 1 Processor -- start 2024-05-30T17:00:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.494+0000 7f846a000640 1 -- start start 2024-05-30T17:00:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.495+0000 7f846a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f846415e870 0x7f84640d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.495+0000 7f846a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f84640d14d0 0x7f84640cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.495+0000 7f846a000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84640cfd00 con 0x7f84640cf870 2024-05-30T17:00:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.495+0000 7f846a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f84640d2150 con 0x7f84640d14d0 2024-05-30T17:00:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.495+0000 7f846a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f846415ec50 con 0x7f846415e870 2024-05-30T17:00:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.496+0000 7f8469600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f84640cf870 0x7f8464162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56722/0 (socket says 172.21.15.67:56722) 2024-05-30T17:00:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.496+0000 7f8469600640 1 -- 172.21.15.67:0/1841336781 learned_addr learned my addr 172.21.15.67:0/1841336781 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.496+0000 7f8463e00640 1 --2- 172.21.15.67:0/1841336781 >> v2:172.21.15.169:3300/0 conn(0x7f84640d14d0 0x7f84640cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.496+0000 7f8468c00640 1 --2- 172.21.15.67:0/1841336781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f846415e870 0x7f84640d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.496+0000 7f8463400640 1 -- 172.21.15.67:0/1841336781 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2693127982 0 0) 0x7f845c001110 con 0x7f84640cf870 2024-05-30T17:00:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.496+0000 7f8463400640 1 -- 172.21.15.67:0/1841336781 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8448003520 con 0x7f84640cf870 2024-05-30T17:00:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.497+0000 7f8463e00640 1 -- 172.21.15.67:0/1841336781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f846415e870 msgr2=0x7f84640d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.497+0000 7f8463e00640 1 --2- 172.21.15.67:0/1841336781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f846415e870 0x7f84640d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.497+0000 7f8463400640 1 -- 172.21.15.67:0/1841336781 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2403235925 0 0) 0x7f8448003520 con 0x7f84640cf870 2024-05-30T17:00:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.497+0000 7f8463e00640 1 -- 172.21.15.67:0/1841336781 >> v1:172.21.15.67:6789/0 conn(0x7f84640cf870 legacy=0x7f8464162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.497+0000 7f8463e00640 1 -- 172.21.15.67:0/1841336781 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84640cff80 con 0x7f84640d14d0 2024-05-30T17:00:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.498+0000 7f8463e00640 1 --2- 172.21.15.67:0/1841336781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84640d14d0 0x7f84640cf330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8450009790 tx=0x7f845002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1adcfecada6d2e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f8463400640 1 -- 172.21.15.67:0/1841336781 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450037070 con 0x7f84640d14d0 2024-05-30T17:00:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f8463400640 1 -- 172.21.15.67:0/1841336781 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845002aba0 con 0x7f84640d14d0 2024-05-30T17:00:48.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f8463400640 1 -- 172.21.15.67:0/1841336781 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c001840 con 0x7f84640d14d0 2024-05-30T17:00:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f846a000640 1 -- 172.21.15.67:0/1841336781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84640d14d0 msgr2=0x7f84640cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f846a000640 1 --2- 172.21.15.67:0/1841336781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84640d14d0 0x7f84640cf330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8450009790 tx=0x7f845002a110 comp rx=0 tx=0).stop 2024-05-30T17:00:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f846a000640 1 -- 172.21.15.67:0/1841336781 shutdown_connections 2024-05-30T17:00:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f846a000640 1 --2- 172.21.15.67:0/1841336781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84640d14d0 0x7f84640cf330 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f846a000640 1 --2- 172.21.15.67:0/1841336781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f846415e870 0x7f84640d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f846a000640 1 -- 172.21.15.67:0/1841336781 >> 172.21.15.67:0/1841336781 conn(0x7f846408bfc0 msgr2=0x7f84640d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:48.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f846a000640 1 -- 172.21.15.67:0/1841336781 shutdown_connections 2024-05-30T17:00:48.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.499+0000 7f846a000640 1 -- 172.21.15.67:0/1841336781 wait complete. 2024-05-30T17:00:48.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.501+0000 7f846a000640 1 Processor -- start 2024-05-30T17:00:48.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.501+0000 7f846a000640 1 -- start start 2024-05-30T17:00:48.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.502+0000 7f846a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84640cf870 0x7f8464159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.502+0000 7f846a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84640d14d0 0x7f84640cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.502+0000 7f846a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846415e870 0x7f84640cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.502+0000 7f846a000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f84640d0820 con 0x7f846415e870 2024-05-30T17:00:48.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.502+0000 7f846a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84640d0540 con 0x7f84640d14d0 2024-05-30T17:00:48.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.502+0000 7f846a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84640d4fb0 con 0x7f84640cf870 2024-05-30T17:00:48.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.502+0000 7f8469600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846415e870 0x7f84640cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8469600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846415e870 0x7f84640cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53236/0 (socket says 172.21.15.67:53236) 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8469600640 1 -- 172.21.15.67:0/832865205 learned_addr learned my addr 172.21.15.67:0/832865205 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8468c00640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84640cf870 0x7f8464159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8463e00640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84640d14d0 0x7f84640cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8469600640 1 -- 172.21.15.67:0/832865205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84640cf870 msgr2=0x7f8464159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8469600640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84640cf870 0x7f8464159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8469600640 1 -- 172.21.15.67:0/832865205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84640d14d0 msgr2=0x7f84640cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8469600640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84640d14d0 0x7f84640cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8469600640 1 -- 172.21.15.67:0/832865205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84640cbfc0 con 0x7f846415e870 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.503+0000 7f8469600640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846415e870 0x7f84640cce60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f845c00c500 tx=0x7f845c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.504+0000 7f8460c00640 1 -- 172.21.15.67:0/832865205 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c005ce0 con 0x7f846415e870 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.504+0000 7f8460c00640 1 -- 172.21.15.67:0/832865205 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c007070 con 0x7f846415e870 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.504+0000 7f8460c00640 1 -- 172.21.15.67:0/832865205 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c007460 con 0x7f846415e870 2024-05-30T17:00:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.504+0000 7f846a000640 1 -- 172.21.15.67:0/832865205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84640cc2a0 con 0x7f846415e870 2024-05-30T17:00:48.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.504+0000 7f846a000640 1 -- 172.21.15.67:0/832865205 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84640cc860 con 0x7f846415e870 2024-05-30T17:00:48.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.506+0000 7f8460c00640 1 -- 172.21.15.67:0/832865205 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f845c007600 con 0x7f846415e870 2024-05-30T17:00:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.508+0000 7f8460c00640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f844406ddf0 0x7f84440702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.508+0000 7f8460c00640 1 -- 172.21.15.67:0/832865205 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f845c08aa30 con 0x7f846415e870 2024-05-30T17:00:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.508+0000 7f8468c00640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f844406ddf0 0x7f84440702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.508+0000 7f846e417740 1 Processor -- start 2024-05-30T17:00:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.508+0000 7f846e417740 1 -- start start 2024-05-30T17:00:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.509+0000 7f8468c00640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f844406ddf0 0x7f84440702b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f84640b21b0 tx=0x7f845400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.509+0000 7f846e417740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff57490 0x55eb1ff5ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.509+0000 7f846e417740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff77090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.509+0000 7f846e417740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff775d0 0x55eb1ff77a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.509+0000 7f846e417740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb1fe94b10 con 0x55eb1ff5b150 2024-05-30T17:00:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.509+0000 7f846e417740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eb1fe3b2b0 con 0x55eb1ff775d0 2024-05-30T17:00:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.509+0000 7f846e417740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eb1fe90ad0 con 0x55eb1ff57490 2024-05-30T17:00:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.509+0000 7f8463e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff77090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8463e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff77090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53252/0 (socket says 172.21.15.67:53252) 2024-05-30T17:00:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8463e00640 1 -- 172.21.15.67:0/1014289592 learned_addr learned my addr 172.21.15.67:0/1014289592 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8468c00640 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff57490 0x55eb1ff5ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8463e00640 1 -- 172.21.15.67:0/1014289592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff57490 msgr2=0x55eb1ff5ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8469600640 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff775d0 0x55eb1ff77a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8463e00640 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff57490 0x55eb1ff5ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8463e00640 1 -- 172.21.15.67:0/1014289592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff775d0 msgr2=0x55eb1ff77a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8463e00640 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff775d0 0x55eb1ff77a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8463e00640 1 -- 172.21.15.67:0/1014289592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1fba72d0 con 0x55eb1ff5b150 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8468c00640 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff57490 0x55eb1ff5ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.510+0000 7f8463e00640 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff77090 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8450009790 tx=0x7f845002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e6eeda0cfd4e54 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f8459200640 1 -- 172.21.15.67:0/1014289592 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450037040 con 0x55eb1ff5b150 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f8459200640 1 -- 172.21.15.67:0/1014289592 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845002aba0 con 0x55eb1ff5b150 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f8459200640 1 -- 172.21.15.67:0/1014289592 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450005d90 con 0x55eb1ff5b150 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f846e417740 1 -- 172.21.15.67:0/1014289592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 msgr2=0x55eb1ff77090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f846e417740 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff77090 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8450009790 tx=0x7f845002c000 comp rx=0 tx=0).stop 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f846e417740 1 -- 172.21.15.67:0/1014289592 shutdown_connections 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f846e417740 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff775d0 0x55eb1ff77a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f846e417740 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff77090 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f846e417740 1 --2- 172.21.15.67:0/1014289592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff57490 0x55eb1ff5ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f846e417740 1 -- 172.21.15.67:0/1014289592 >> 172.21.15.67:0/1014289592 conn(0x55eb1ff2d470 msgr2=0x55eb1ff2e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f846e417740 1 -- 172.21.15.67:0/1014289592 shutdown_connections 2024-05-30T17:00:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.511+0000 7f846e417740 1 -- 172.21.15.67:0/1014289592 wait complete. 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.512+0000 7f846e417740 1 -- start start 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.512+0000 7f846e417740 1 Processor -- start 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.513+0000 7f846e417740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff56600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.513+0000 7f846e417740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff775d0 0x55eb1ff72d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.513+0000 7f846e417740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 0x55eb1ff30280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.513+0000 7f846e417740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ff2b2a0 con 0x55eb1ff5b150 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.513+0000 7f846e417740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eb1fe4e570 con 0x55eb1ff775d0 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.513+0000 7f846e417740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eb1ff2b580 con 0x55eb1ff55780 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.513+0000 7f846e417740 10 client.0 _ll_register_callbacks cb 0x55eb1fe1cfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T17:00:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.513+0000 7f8468c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff56600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.514+0000 7f8468c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff56600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53266/0 (socket says 172.21.15.67:53266) 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.514+0000 7f8468c00640 1 -- 172.21.15.67:0/968554116 learned_addr learned my addr 172.21.15.67:0/968554116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.514+0000 7f8468c00640 1 -- 172.21.15.67:0/968554116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 msgr2=0x55eb1ff30280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.514+0000 7f8468c00640 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 0x55eb1ff30280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.514+0000 7f8468c00640 1 -- 172.21.15.67:0/968554116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff775d0 msgr2=0x55eb1ff72d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.514+0000 7f8468c00640 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff775d0 0x55eb1ff72d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.514+0000 7f8468c00640 1 -- 172.21.15.67:0/968554116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1fc716b0 con 0x55eb1ff5b150 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.514+0000 7f8468c00640 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff56600 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8454016ec0 tx=0x7f8454004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825ae96bb3bbccd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.514+0000 7f844d600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f844d600640 1 -- 172.21.15.67:0/968554116 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84540225d0 con 0x55eb1ff5b150 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f844d600640 1 -- 172.21.15.67:0/968554116 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f8454022770 con 0x55eb1ff5b150 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f846e417740 1 -- 172.21.15.67:0/968554116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f845002aba0 con 0x55eb1ff5b150 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f844d600640 1 -- 172.21.15.67:0/968554116 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8454030050 con 0x55eb1ff5b150 2024-05-30T17:00:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f846e417740 10 client.8064 fetch_fsmap learned FSMap version 64 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f846e417740 1 -- 172.21.15.67:0/968554116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x55eb1ff70790 con 0x55eb1ff5b150 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f844d600640 1 -- 172.21.15.67:0/968554116 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845002aba0 con 0x55eb1ff5b150 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f844d600640 1 -- 172.21.15.67:0/968554116 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f8454033020 con 0x55eb1ff5b150 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f846e417740 10 client.8064 fetch_fsmap finished waiting for FSMap version 64 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f846e417740 10 client.8064 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f846e417740 20 client.8064 populate_metadata read hostname 'smithi067' 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f846e417740 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] conn(0x55eb1ff5d330 0x55eb1ff71b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f846e417740 4 client.8064 mds_command: new command op to 7902 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.515+0000 7f846e417740 1 -- 172.21.15.67:0/968554116 --> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb1fea2530 con 0x55eb1ff5d330 2024-05-30T17:00:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.516+0000 7f8463e00640 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] conn(0x55eb1ff5d330 0x55eb1ff71b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.516+0000 7f8463e00640 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] conn(0x55eb1ff5d330 0x55eb1ff71b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13c806d29350064a server_cookie=4044699d15d326e7 in_seq=0 out_seq=0 2024-05-30T17:00:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.516+0000 7f844d600640 10 client.8064 ms_handle_connect on v2:172.21.15.67:6834/2120049133 2024-05-30T17:00:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f844d600640 1 -- 172.21.15.67:0/968554116 <== mds.0 v2:172.21.15.67:6834/2120049133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb1fea2530 con 0x55eb1ff5d330 2024-05-30T17:00:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f844d600640 10 client.8064 handle_command_reply: tid=0 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f846e417740 1 client.8064 shutdown 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f846e417740 1 -- 172.21.15.67:0/968554116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 msgr2=0x55eb1ff56600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f846e417740 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff56600 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8454016ec0 tx=0x7f8454004ab0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f846e417740 1 -- 172.21.15.67:0/968554116 shutdown_connections 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f846e417740 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] conn(0x55eb1ff5d330 0x55eb1ff71b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f846e417740 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 0x55eb1ff30280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f846e417740 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff775d0 0x55eb1ff72d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f846e417740 1 --2- 172.21.15.67:0/968554116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff5b150 0x55eb1ff56600 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f844d600640 0 client.8064 ms_handle_reset on v2:172.21.15.67:6834/2120049133 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.517+0000 7f846e417740 1 -- 172.21.15.67:0/968554116 >> 172.21.15.67:0/968554116 conn(0x55eb1ff57490 msgr2=0x55eb1ff2a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.518+0000 7f846e417740 1 -- 172.21.15.67:0/968554116 shutdown_connections 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.518+0000 7f846e417740 1 -- 172.21.15.67:0/968554116 wait complete. 2024-05-30T17:00:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.518+0000 7f846e417740 20 client.8064 trim_cache size 0 max 16384 2024-05-30T17:00:48.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.537+0000 7f846e417740 1 Processor -- start 2024-05-30T17:00:48.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.538+0000 7f846e417740 1 -- start start 2024-05-30T17:00:48.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.538+0000 7f846e417740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff55780 0x55eb1ff2c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.538+0000 7f846e417740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff5b150 0x55eb1ff6fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.538+0000 7f846e417740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.538+0000 7f846e417740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f84641ffca0 con 0x55eb1ff775d0 2024-05-30T17:00:48.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.538+0000 7f846e417740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84641ff240 con 0x55eb1ff55780 2024-05-30T17:00:48.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.538+0000 7f846e417740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84641ff6e0 con 0x55eb1ff5b150 2024-05-30T17:00:48.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8469600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8469600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53280/0 (socket says 172.21.15.67:53280) 2024-05-30T17:00:48.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8469600640 1 -- 172.21.15.67:0/2850682260 learned_addr learned my addr 172.21.15.67:0/2850682260 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8463e00640 1 --2- 172.21.15.67:0/2850682260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff5b150 0x55eb1ff6fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8468c00640 1 --2- 172.21.15.67:0/2850682260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff55780 0x55eb1ff2c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8469600640 1 -- 172.21.15.67:0/2850682260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff5b150 msgr2=0x55eb1ff6fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8469600640 1 --2- 172.21.15.67:0/2850682260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff5b150 0x55eb1ff6fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8469600640 1 -- 172.21.15.67:0/2850682260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff55780 msgr2=0x55eb1ff2c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8469600640 1 --2- 172.21.15.67:0/2850682260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff55780 0x55eb1ff2c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.539+0000 7f8469600640 1 -- 172.21.15.67:0/2850682260 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1ff56270 con 0x55eb1ff775d0 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.540+0000 7f8469600640 1 --2- 172.21.15.67:0/2850682260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f845c00c4d0 tx=0x7f845c05a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1550f09295257c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.540+0000 7f844fe00640 1 -- 172.21.15.67:0/2850682260 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c05d040 con 0x55eb1ff775d0 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.540+0000 7f844fe00640 1 -- 172.21.15.67:0/2850682260 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c0629b0 con 0x55eb1ff775d0 2024-05-30T17:00:48.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.540+0000 7f844fe00640 1 -- 172.21.15.67:0/2850682260 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c08ad00 con 0x55eb1ff775d0 2024-05-30T17:00:48.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.540+0000 7f846e417740 1 -- 172.21.15.67:0/2850682260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 msgr2=0x55eb1ff5dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.540+0000 7f846e417740 1 --2- 172.21.15.67:0/2850682260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f845c00c4d0 tx=0x7f845c05a4d0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.541+0000 7f846e417740 1 -- 172.21.15.67:0/2850682260 shutdown_connections 2024-05-30T17:00:48.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.541+0000 7f846e417740 1 --2- 172.21.15.67:0/2850682260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.541+0000 7f846e417740 1 --2- 172.21.15.67:0/2850682260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff5b150 0x55eb1ff6fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.541+0000 7f846e417740 1 --2- 172.21.15.67:0/2850682260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff55780 0x55eb1ff2c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.541+0000 7f846e417740 1 -- 172.21.15.67:0/2850682260 >> 172.21.15.67:0/2850682260 conn(0x55eb1ff57490 msgr2=0x55eb1ff268d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:48.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.541+0000 7f846e417740 1 -- 172.21.15.67:0/2850682260 shutdown_connections 2024-05-30T17:00:48.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.541+0000 7f846e417740 1 -- 172.21.15.67:0/2850682260 wait complete. 2024-05-30T17:00:48.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.541+0000 7f846e417740 1 -- start start 2024-05-30T17:00:48.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.542+0000 7f846e417740 1 Processor -- start 2024-05-30T17:00:48.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f846e417740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 0x55eb1ff55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f846e417740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff5b150 0x55eb1ffeecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f846e417740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f846e417740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ff5d1b0 con 0x55eb1ff775d0 2024-05-30T17:00:48.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f846e417740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ff70a70 con 0x55eb1ff5b150 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f846e417740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eb1ff5d830 con 0x55eb1ff55780 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f846e417740 10 client.0 _ll_register_callbacks cb 0x7f8454016b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8469600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8469600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53294/0 (socket says 172.21.15.67:53294) 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8469600640 1 -- 172.21.15.67:0/2071228592 learned_addr learned my addr 172.21.15.67:0/2071228592 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8463e00640 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff5b150 0x55eb1ffeecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8468c00640 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 0x55eb1ff55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8469600640 1 -- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 msgr2=0x55eb1ff55bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8469600640 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 0x55eb1ff55bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8469600640 1 -- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff5b150 msgr2=0x55eb1ffeecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8469600640 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff5b150 0x55eb1ffeecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8469600640 1 -- 172.21.15.67:0/2071228592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1ff5b6d0 con 0x55eb1ff775d0 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8468c00640 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 0x55eb1ff55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:48.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.543+0000 7f8463e00640 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff5b150 0x55eb1ffeecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f8469600640 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f845c05a680 tx=0x7f845c06f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6697ece7e626e48b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f844d600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f844d600640 1 -- 172.21.15.67:0/2071228592 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c076070 con 0x55eb1ff775d0 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f844d600640 1 -- 172.21.15.67:0/2071228592 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f845c062720 con 0x55eb1ff775d0 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f846e417740 1 -- 172.21.15.67:0/2071228592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f845c0629b0 con 0x55eb1ff775d0 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f844d600640 1 -- 172.21.15.67:0/2071228592 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f845c05d070 con 0x55eb1ff775d0 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f846e417740 10 client.8070 fetch_fsmap learned FSMap version 64 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f846e417740 1 -- 172.21.15.67:0/2071228592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x55eb1fff0740 con 0x55eb1ff775d0 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f844d600640 1 -- 172.21.15.67:0/2071228592 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845c0629b0 con 0x55eb1ff775d0 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f844d600640 1 -- 172.21.15.67:0/2071228592 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f845c079020 con 0x55eb1ff775d0 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.544+0000 7f846e417740 10 client.8070 fetch_fsmap finished waiting for FSMap version 64 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.545+0000 7f846e417740 10 client.8070 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.545+0000 7f846e417740 20 client.8070 populate_metadata read hostname 'smithi067' 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.545+0000 7f846e417740 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] conn(0x55eb1ffefca0 0x55eb1ff98240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.545+0000 7f846e417740 4 client.8070 mds_command: new command op to 7902 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-30T17:00:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.545+0000 7f846e417740 1 -- 172.21.15.67:0/2071228592 --> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] -- command(tid 0: {"prefix": "status"}) -- 0x55eb1ff2b8f0 con 0x55eb1ffefca0 2024-05-30T17:00:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.545+0000 7f8468c00640 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] conn(0x55eb1ffefca0 0x55eb1ff98240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.545+0000 7f8468c00640 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] conn(0x55eb1ffefca0 0x55eb1ff98240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ebad10e7afc4175 server_cookie=22ce396196fcf3e in_seq=0 out_seq=0 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.545+0000 7f844d600640 10 client.8070 ms_handle_connect on v2:172.21.15.67:6834/2120049133 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f844d600640 1 -- 172.21.15.67:0/2071228592 <== mds.0 v2:172.21.15.67:6834/2120049133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55eb1ff2b8f0 con 0x55eb1ffefca0 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f844d600640 10 client.8070 handle_command_reply: tid=0 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 client.8070 shutdown 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 -- 172.21.15.67:0/2071228592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 msgr2=0x55eb1ff5dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f845c05a680 tx=0x7f845c06f000 comp rx=0 tx=0).stop 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 -- 172.21.15.67:0/2071228592 shutdown_connections 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.67:6834/2120049133,v1:172.21.15.67:6836/2120049133] conn(0x55eb1ffefca0 0x55eb1ff98240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb1ff775d0 0x55eb1ff5dc30 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb1ff5b150 0x55eb1ffeecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 --2- 172.21.15.67:0/2071228592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb1ff55780 0x55eb1ff55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f844d600640 0 client.8070 ms_handle_reset on v2:172.21.15.67:6834/2120049133 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 -- 172.21.15.67:0/2071228592 >> 172.21.15.67:0/2071228592 conn(0x55eb1ff57490 msgr2=0x55eb1ff31b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 -- 172.21.15.67:0/2071228592 shutdown_connections 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 1 -- 172.21.15.67:0/2071228592 wait complete. 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.546+0000 7f846e417740 20 client.8070 trim_cache size 0 max 16384 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7902, 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-05-30T17:00:48.550 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-05-30T17:00:48.551 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-05-30T17:00:48.551 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 11.805893967999999, 2024-05-30T17:00:48.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 64, 2024-05-30T17:00:48.551 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 104, 2024-05-30T17:00:48.551 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 104, 2024-05-30T17:00:48.551 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 17.205593821000001 2024-05-30T17:00:48.551 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 -- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f844406ddf0 msgr2=0x7f84440702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f844406ddf0 0x7f84440702b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f84640b21b0 tx=0x7f845400b040 comp rx=0 tx=0).stop 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 -- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846415e870 msgr2=0x7f84640cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846415e870 0x7f84640cce60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f845c00c500 tx=0x7f845c00c9c0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 -- 172.21.15.67:0/832865205 shutdown_connections 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f844406ddf0 0x7f84440702b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846415e870 0x7f84640cce60 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84640d14d0 0x7f84640cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 --2- 172.21.15.67:0/832865205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84640cf870 0x7f8464159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 -- 172.21.15.67:0/832865205 >> 172.21.15.67:0/832865205 conn(0x7f846408bfc0 msgr2=0x7f84640b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 -- 172.21.15.67:0/832865205 shutdown_connections 2024-05-30T17:00:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.547+0000 7f846a000640 1 -- 172.21.15.67:0/832865205 wait complete. 2024-05-30T17:00:48.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T17:00:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.750+0000 7fed1da00640 1 Processor -- start 2024-05-30T17:00:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.751+0000 7fed1da00640 1 -- start start 2024-05-30T17:00:48.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.753+0000 7fed1da00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fed180b56f0 0x7fed180af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.753+0000 7fed1da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fed180b0370 0x7fed180af820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.753+0000 7fed1da00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed180b4730 con 0x7fed1815e670 2024-05-30T17:00:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.753+0000 7fed1da00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fed180aee50 con 0x7fed180b0370 2024-05-30T17:00:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.753+0000 7fed1da00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fed180b49d0 con 0x7fed180b56f0 2024-05-30T17:00:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.753+0000 7fed17e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fed1815e670 0x7fed180b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56732/0 (socket says 172.21.15.67:56732) 2024-05-30T17:00:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.753+0000 7fed17e00640 1 -- 172.21.15.67:0/3544421730 learned_addr learned my addr 172.21.15.67:0/3544421730 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.753+0000 7fed1d000640 1 --2- 172.21.15.67:0/3544421730 >> v2:172.21.15.67:3300/0 conn(0x7fed180b0370 0x7fed180af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.753+0000 7fed0fe00640 1 --2- 172.21.15.67:0/3544421730 >> v2:172.21.15.169:3300/0 conn(0x7fed180b56f0 0x7fed180af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.754+0000 7fed1d000640 1 -- 172.21.15.67:0/3544421730 >> v2:172.21.15.169:3300/0 conn(0x7fed180b56f0 msgr2=0x7fed180af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.754+0000 7fed1d000640 1 --2- 172.21.15.67:0/3544421730 >> v2:172.21.15.169:3300/0 conn(0x7fed180b56f0 0x7fed180af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.754+0000 7fed1d000640 1 -- 172.21.15.67:0/3544421730 >> v1:172.21.15.67:6789/0 conn(0x7fed1815e670 legacy=0x7fed180b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.754+0000 7fed1d000640 1 -- 172.21.15.67:0/3544421730 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed180b4cb0 con 0x7fed180b0370 2024-05-30T17:00:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.755+0000 7fed1d000640 1 --2- 172.21.15.67:0/3544421730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed180b0370 0x7fed180af820 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fed180aefd0 tx=0x7fed1002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d4ce43b512d0e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.755+0000 7fed17400640 1 -- 172.21.15.67:0/3544421730 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10031600 con 0x7fed180b0370 2024-05-30T17:00:48.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.755+0000 7fed17400640 1 -- 172.21.15.67:0/3544421730 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed100317a0 con 0x7fed180b0370 2024-05-30T17:00:48.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.755+0000 7fed17400640 1 -- 172.21.15.67:0/3544421730 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10031b70 con 0x7fed180b0370 2024-05-30T17:00:48.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.756+0000 7fed1da00640 1 -- 172.21.15.67:0/3544421730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed180b0370 msgr2=0x7fed180af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.756+0000 7fed1da00640 1 --2- 172.21.15.67:0/3544421730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed180b0370 0x7fed180af820 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fed180aefd0 tx=0x7fed1002a5d0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.756+0000 7fed1da00640 1 -- 172.21.15.67:0/3544421730 shutdown_connections 2024-05-30T17:00:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.756+0000 7fed1da00640 1 --2- 172.21.15.67:0/3544421730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed180b0370 0x7fed180af820 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.756+0000 7fed1da00640 1 --2- 172.21.15.67:0/3544421730 >> v2:172.21.15.169:3300/0 conn(0x7fed180b56f0 0x7fed180af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.756+0000 7fed1da00640 1 -- 172.21.15.67:0/3544421730 >> 172.21.15.67:0/3544421730 conn(0x7fed180732f0 msgr2=0x7fed1806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.756+0000 7fed1da00640 1 -- 172.21.15.67:0/3544421730 shutdown_connections 2024-05-30T17:00:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.756+0000 7fed1da00640 1 -- 172.21.15.67:0/3544421730 wait complete. 2024-05-30T17:00:48.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.758+0000 7fed1da00640 1 Processor -- start 2024-05-30T17:00:48.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.759+0000 7fed1da00640 1 -- start start 2024-05-30T17:00:48.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.760+0000 7fed1da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed180b0370 0x7fed1815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.760+0000 7fed1da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed180b56f0 0x7fed18153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.760+0000 7fed1da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1815e670 0x7fed181e94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.760+0000 7fed1da00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fed180b5d90 con 0x7fed1815e670 2024-05-30T17:00:48.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.760+0000 7fed1da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed180b5c10 con 0x7fed180b56f0 2024-05-30T17:00:48.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.760+0000 7fed1da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed180ae9e0 con 0x7fed180b0370 2024-05-30T17:00:48.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.760+0000 7fed1d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1815e670 0x7fed181e94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.760+0000 7fed1d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1815e670 0x7fed181e94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53300/0 (socket says 172.21.15.67:53300) 2024-05-30T17:00:48.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.760+0000 7fed1d000640 1 -- 172.21.15.67:0/2725813420 learned_addr learned my addr 172.21.15.67:0/2725813420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.761+0000 7fed17e00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed180b0370 0x7fed1815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.761+0000 7fed0fe00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed180b56f0 0x7fed18153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.761+0000 7fed1d000640 1 -- 172.21.15.67:0/2725813420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed180b0370 msgr2=0x7fed1815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.761+0000 7fed1d000640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed180b0370 0x7fed1815d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.761+0000 7fed1d000640 1 -- 172.21.15.67:0/2725813420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed180b56f0 msgr2=0x7fed18153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.761+0000 7fed1d000640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed180b56f0 0x7fed18153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.761+0000 7fed1d000640 1 -- 172.21.15.67:0/2725813420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed181e8400 con 0x7fed1815e670 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.761+0000 7fed1d000640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1815e670 0x7fed181e94a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fed1002abc0 tx=0x7fed10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.761+0000 7fed14c00640 1 -- 172.21.15.67:0/2725813420 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1003f070 con 0x7fed1815e670 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.762+0000 7fed14c00640 1 -- 172.21.15.67:0/2725813420 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed10030b60 con 0x7fed1815e670 2024-05-30T17:00:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.762+0000 7fed14c00640 1 -- 172.21.15.67:0/2725813420 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10009450 con 0x7fed1815e670 2024-05-30T17:00:48.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.762+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed181e86e0 con 0x7fed1815e670 2024-05-30T17:00:48.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.762+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed181e8ca0 con 0x7fed1815e670 2024-05-30T17:00:48.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.763+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed180690f0 con 0x7fed1815e670 2024-05-30T17:00:48.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.764+0000 7fed14c00640 1 -- 172.21.15.67:0/2725813420 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7fed1003b410 con 0x7fed1815e670 2024-05-30T17:00:48.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.765+0000 7fed14c00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fecf406ddc0 0x7fecf4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:48.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.765+0000 7fed14c00640 1 -- 172.21.15.67:0/2725813420 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fed100ac260 con 0x7fed1815e670 2024-05-30T17:00:48.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.765+0000 7fed17e00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fecf406ddc0 0x7fecf4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:48.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.766+0000 7fed17e00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fecf406ddc0 0x7fecf4070280 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fed08000f60 tx=0x7fed08009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:48.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:48.769+0000 7fed14c00640 1 -- 172.21.15.67:0/2725813420 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed1007bd00 con 0x7fed1815e670 2024-05-30T17:00:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.015+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fed18070300 con 0x7fed1815e670 2024-05-30T17:00:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.015+0000 7fed14c00640 1 -- 172.21.15.67:0/2725813420 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7fed1007b600 con 0x7fed1815e670 2024-05-30T17:00:49.016 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T17:00:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fecf406ddc0 msgr2=0x7fecf4070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fecf406ddc0 0x7fecf4070280 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fed08000f60 tx=0x7fed08009040 comp rx=0 tx=0).stop 2024-05-30T17:00:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1815e670 msgr2=0x7fed181e94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1815e670 0x7fed181e94a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fed1002abc0 tx=0x7fed10004a40 comp rx=0 tx=0).stop 2024-05-30T17:00:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 shutdown_connections 2024-05-30T17:00:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fecf406ddc0 0x7fecf4070280 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1815e670 0x7fed181e94a0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed180b56f0 0x7fed18153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:49.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 --2- 172.21.15.67:0/2725813420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed180b0370 0x7fed1815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:49.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 >> 172.21.15.67:0/2725813420 conn(0x7fed180732f0 msgr2=0x7fed1806f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:49.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 shutdown_connections 2024-05-30T17:00:49.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49.018+0000 7fed1da00640 1 -- 172.21.15.67:0/2725813420 wait complete. 2024-05-30T17:00:49.039 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T17:00:49.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T17:00:49.039 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T17:00:49.039 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-30T17:00:49.039 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T17:00:49.039 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T17:00:49.039 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T17:00:49.039 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T17:00:49.039 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T17:00:49.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:49.040 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T17:00:49.092 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41523sec preferred_lft 41523sec 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff 2024-05-30T17:00:49.093 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::74f5:cfff:fec0:693d/64 scope link 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::a8fe:aeff:fef7:f6a0/64 scope link 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::185e:a4ff:fec6:7d8c/64 scope link 2024-05-30T17:00:49.094 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:00:49.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:49.095 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T17:00:49.177 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T17:00:49.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:49.182 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T17:00:49.239 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T17:00:49.239 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T17:00:49.239 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T17:00:49.239 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T17:00:49.239 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T17:00:49.240 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T17:00:49.240 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T17:00:49.240 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T17:00:49.240 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T17:00:49.240 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T17:00:49.240 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T17:00:49.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T17:00:49.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T17:00:49.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:49.460 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T17:00:49.509 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T17:00:49.509 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-30T17:00:49.510 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T17:00:49.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:49.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-05-30T17:00:49.596 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-30T17:00:49.596 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T17:00:49.597 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-30T17:00:49.651 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-30T17:00:49.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T17:00:49.652 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-30T17:00:49.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T17:00:49.793 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T17:00:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T17:00:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T17:00:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T17:00:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T17:00:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T17:00:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T17:00:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T17:00:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T17:00:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:00:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:00:49.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:49.856 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T17:00:49.909 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T17:00:49.909 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-30T17:00:49.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T17:00:49.987 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T17:00:50.029 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T17:00:50.029 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T17:00:50.029 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T17:00:50.029 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T17:00:50.029 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T17:00:50.029 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T17:00:50.029 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T17:00:50.029 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T17:00:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:00:50.031 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:00:50.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:50.033 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T17:00:50.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-05-30T17:00:50.087+0000 7fa141a3c480 -1 init, newargv = 0x56241c5b8b90 newargc=15 2024-05-30T17:00:50.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[70921]: starting ceph client 2024-05-30T17:00:50.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[70921]: starting fuse 2024-05-30T17:00:50.116 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T17:00:50.116 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-30T17:00:50.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T17:00:50.116 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T17:00:50.116 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T17:00:50.116 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T17:00:50.117 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T17:00:50.118 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T17:00:50.118 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T17:00:50.118 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T17:00:50.118 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T17:00:50.118 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T17:00:50.118 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:00:50.118 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-05-30T17:00:50.118 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T17:00:50.316 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.70921.asok 2024-05-30T17:00:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T17:00:50.333 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.70921.asok 2024-05-30T17:00:50.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:50.334 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.70921.asok status 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "70921", 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T17:00:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8088, 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8088 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1857218402 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:50.573 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1857218402 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.8088 192.168.0.1:0/1857218402", 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/1857218402", 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 64, 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 104, 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T17:00:50.574 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T17:00:50.575 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:00:50.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:50.587 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T17:00:50.587 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-30T17:00:50.641 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T17:00:50.641 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-30T17:00:50.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:50.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-30T17:00:50.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-30T17:00:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.952+0000 7f9435c00640 1 Processor -- start 2024-05-30T17:00:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.953+0000 7f9435c00640 1 -- start start 2024-05-30T17:00:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.954+0000 7f9435c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f94300d1420 0x7f943006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.954+0000 7f9435c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94300d2200 con 0x7f94300cfb40 2024-05-30T17:00:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.954+0000 7f9435c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94300d24e0 con 0x7f943015e5a0 2024-05-30T17:00:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.954+0000 7f9435c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f94300d2b30 con 0x7f94300d1420 2024-05-30T17:00:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.955+0000 7f942f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f94300cfb40 0x7f94300d0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34800/0 (socket says 172.21.15.67:34800) 2024-05-30T17:00:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.955+0000 7f942f400640 1 -- 172.21.15.67:0/2656283165 learned_addr learned my addr 172.21.15.67:0/2656283165 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.955+0000 7f9435200640 1 --2- 172.21.15.67:0/2656283165 >> v2:172.21.15.169:3300/0 conn(0x7f94300d1420 0x7f943006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.955+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2361835817 0 0) 0x7f94300d2200 con 0x7f94300cfb40 2024-05-30T17:00:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.955+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9414003520 con 0x7f94300cfb40 2024-05-30T17:00:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2642620372 0 0) 0x7f9414003520 con 0x7f94300cfb40 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 >> v2:172.21.15.169:3300/0 conn(0x7f94300d1420 msgr2=0x7f943006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 --2- 172.21.15.67:0/2656283165 >> v2:172.21.15.169:3300/0 conn(0x7f94300d1420 0x7f943006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 >> v1:172.21.15.169:6789/0 conn(0x7f943015e5a0 legacy=0x7f94300d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94300d27c0 con 0x7f94300cfb40 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9418001670 con 0x7f94300cfb40 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 >> v1:172.21.15.67:6789/0 conn(0x7f94300cfb40 legacy=0x7f94300d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9414003a00 0x7f9414003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f9414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94140051c0 0x7f941400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f941400c090 con 0x7f9414004800 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f941400c240 con 0x7f94140051c0 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f941400c440 con 0x7f9414003a00 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.956+0000 7f942f400640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f9414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.957+0000 7f9435200640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94140051c0 0x7f941400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.957+0000 7f942fe00640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9414003a00 0x7f9414003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.957+0000 7f942f400640 1 -- 172.21.15.67:0/2656283165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9414003a00 msgr2=0x7f9414003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.957+0000 7f942f400640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9414003a00 0x7f9414003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.957+0000 7f942f400640 1 -- 172.21.15.67:0/2656283165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94140051c0 msgr2=0x7f941400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.957+0000 7f942f400640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94140051c0 0x7f941400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.957+0000 7f942f400640 1 -- 172.21.15.67:0/2656283165 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9414003520 con 0x7f9414004800 2024-05-30T17:00:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.957+0000 7f9435200640 1 -- 172.21.15.67:0/2656283165 reap_dead start 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.957+0000 7f942fe00640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9414003a00 0x7f9414003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.958+0000 7f942f400640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f9414004c80 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f941800c110 tx=0x7f9414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cc68365c6cc9e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.958+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418007c40 con 0x7f9414004800 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.958+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9418005af0 con 0x7f9414004800 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.958+0000 7f942ea00640 1 -- 172.21.15.67:0/2656283165 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418007070 con 0x7f9414004800 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.959+0000 7f9435c00640 1 -- 172.21.15.67:0/2656283165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 msgr2=0x7f9414004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.959+0000 7f9435c00640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f9414004c80 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f941800c110 tx=0x7f9414005830 comp rx=0 tx=0).stop 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.959+0000 7f9435c00640 1 -- 172.21.15.67:0/2656283165 shutdown_connections 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.959+0000 7f9435c00640 1 --2- 172.21.15.67:0/2656283165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f9414004c80 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.959+0000 7f9435c00640 1 -- 172.21.15.67:0/2656283165 >> 172.21.15.67:0/2656283165 conn(0x7f943008bfc0 msgr2=0x7f94300d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.959+0000 7f9435c00640 1 -- 172.21.15.67:0/2656283165 shutdown_connections 2024-05-30T17:00:50.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.959+0000 7f9435c00640 1 -- 172.21.15.67:0/2656283165 wait complete. 2024-05-30T17:00:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.961+0000 7f9435c00640 1 Processor -- start 2024-05-30T17:00:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.961+0000 7f9435c00640 1 -- start start 2024-05-30T17:00:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f9435c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f94300c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f9435c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94300b2cd0 0x7f9430151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f9435c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94300b3150 0x7f9430151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f9435c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94300d53a0 con 0x7f9414004800 2024-05-30T17:00:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f9435c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94300d5220 con 0x7f94300b2cd0 2024-05-30T17:00:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f9435c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f94300cffd0 con 0x7f94300b3150 2024-05-30T17:00:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f942fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f94300c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f942fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f94300c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36436/0 (socket says 172.21.15.67:36436) 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f942fe00640 1 -- 172.21.15.67:0/306578951 learned_addr learned my addr 172.21.15.67:0/306578951 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f942f400640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94300b2cd0 0x7f9430151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f942fe00640 1 -- 172.21.15.67:0/306578951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94300b3150 msgr2=0x7f9430151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f942fe00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94300b3150 0x7f9430151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f942fe00640 1 -- 172.21.15.67:0/306578951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94300b2cd0 msgr2=0x7f9430151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f942fe00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94300b2cd0 0x7f9430151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.962+0000 7f942fe00640 1 -- 172.21.15.67:0/306578951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94300ca8c0 con 0x7f9414004800 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.963+0000 7f942fe00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f94300c7fe0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f942000d7a0 tx=0x7f942000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.963+0000 7f9427e00640 1 -- 172.21.15.67:0/306578951 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420003da0 con 0x7f9414004800 2024-05-30T17:00:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.963+0000 7f9427e00640 1 -- 172.21.15.67:0/306578951 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420017070 con 0x7f9414004800 2024-05-30T17:00:50.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.963+0000 7f9427e00640 1 -- 172.21.15.67:0/306578951 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420005290 con 0x7f9414004800 2024-05-30T17:00:50.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.963+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94300caba0 con 0x7f9414004800 2024-05-30T17:00:50.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.963+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94300c8520 con 0x7f9414004800 2024-05-30T17:00:50.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.965+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94300689e0 con 0x7f9414004800 2024-05-30T17:00:50.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.966+0000 7f9427e00640 1 -- 172.21.15.67:0/306578951 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f9420005430 con 0x7f9414004800 2024-05-30T17:00:50.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.967+0000 7f9427e00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f940c06ddc0 0x7f940c070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:50.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.967+0000 7f9427e00640 1 -- 172.21.15.67:0/306578951 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f942008a8b0 con 0x7f9414004800 2024-05-30T17:00:50.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.967+0000 7f942f400640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f940c06ddc0 0x7f940c070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:50.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.968+0000 7f942f400640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f940c06ddc0 0x7f940c070280 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9418032000 tx=0x7f941802d000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:50.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:50.972+0000 7f9427e00640 1 -- 172.21.15.67:0/306578951 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f942005a730 con 0x7f9414004800 2024-05-30T17:00:51.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.226+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f94300d1d30 con 0x7f9414004800 2024-05-30T17:00:51.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.830+0000 7f9427e00640 1 -- 172.21.15.67:0/306578951 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v65) ==== 136+0+0 (secure 0 0 0) 0x7f942005a030 con 0x7f9414004800 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f940c06ddc0 msgr2=0x7f940c070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f940c06ddc0 0x7f940c070280 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9418032000 tx=0x7f941802d000 comp rx=0 tx=0).stop 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 msgr2=0x7f94300c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f94300c7fe0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f942000d7a0 tx=0x7f942000dc60 comp rx=0 tx=0).stop 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 shutdown_connections 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f940c06ddc0 0x7f940c070280 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94300b3150 0x7f9430151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94300b2cd0 0x7f9430151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 --2- 172.21.15.67:0/306578951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9414004800 0x7f94300c7fe0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 >> 172.21.15.67:0/306578951 conn(0x7f943008bfc0 msgr2=0x7f94300b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 shutdown_connections 2024-05-30T17:00:51.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:51.834+0000 7f9435c00640 1 -- 172.21.15.67:0/306578951 wait complete. 2024-05-30T17:00:51.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T17:00:52.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.070+0000 7f148f000640 1 Processor -- start 2024-05-30T17:00:52.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.071+0000 7f148f000640 1 -- start start 2024-05-30T17:00:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.072+0000 7f148f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f148806ee40 0x7f14880ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.072+0000 7f148f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148806f2f0 0x7f14880ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.072+0000 7f148f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1488156f10 con 0x7f1488156b30 2024-05-30T17:00:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.072+0000 7f148f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f148806dda0 con 0x7f148806ee40 2024-05-30T17:00:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.072+0000 7f148f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f148806e080 con 0x7f148806f2f0 2024-05-30T17:00:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.073+0000 7f148dc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1488156b30 0x7f14880af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34816/0 (socket says 172.21.15.67:34816) 2024-05-30T17:00:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.073+0000 7f148dc00640 1 -- 172.21.15.67:0/1612925791 learned_addr learned my addr 172.21.15.67:0/1612925791 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.073+0000 7f148d200640 1 --2- 172.21.15.67:0/1612925791 >> v2:172.21.15.169:3300/0 conn(0x7f148806ee40 0x7f14880ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.073+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124857661 0 0) 0x7f1488156f10 con 0x7f1488156b30 2024-05-30T17:00:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.073+0000 7f148e600640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148806f2f0 0x7f14880ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.073+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1470003520 con 0x7f1488156b30 2024-05-30T17:00:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.073+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1629876068 0 0) 0x7f1470003520 con 0x7f1488156b30 2024-05-30T17:00:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.073+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148806f2f0 msgr2=0x7f14880ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:52.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.073+0000 7f1487e00640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148806f2f0 0x7f14880ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 >> v2:172.21.15.169:3300/0 conn(0x7f148806ee40 msgr2=0x7f14880ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 --2- 172.21.15.67:0/1612925791 >> v2:172.21.15.169:3300/0 conn(0x7f148806ee40 0x7f14880ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148806e360 con 0x7f1488156b30 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f147c001670 con 0x7f1488156b30 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 >> v1:172.21.15.67:6789/0 conn(0x7f1488156b30 legacy=0x7f14880af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14700039b0 0x7f1470003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1470004800 0x7f1470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148806f2f0 0x7f14700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f148806dda0 con 0x7f1470004800 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f148806e080 con 0x7f14700039b0 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f147000ba70 con 0x7f148806f2f0 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f148d200640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1470004800 0x7f1470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f148d200640 1 -- 172.21.15.67:0/1612925791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148806f2f0 msgr2=0x7f14700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f148d200640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148806f2f0 0x7f14700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f148d200640 1 -- 172.21.15.67:0/1612925791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14700039b0 msgr2=0x7f1470003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f148d200640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14700039b0 0x7f1470003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.074+0000 7f148d200640 1 -- 172.21.15.67:0/1612925791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1470003520 con 0x7f1470004800 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.075+0000 7f148d200640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1470004800 0x7f1470004c80 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f147400ba00 tx=0x7f1470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18f6a94b7018fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474030070 con 0x7f1470004800 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147402acf0 con 0x7f1470004800 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f1487e00640 1 -- 172.21.15.67:0/1612925791 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147400b600 con 0x7f1470004800 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f148f000640 1 -- 172.21.15.67:0/1612925791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1470004800 msgr2=0x7f1470004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f148f000640 1 --2- 172.21.15.67:0/1612925791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1470004800 0x7f1470004c80 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f147400ba00 tx=0x7f1470005830 comp rx=0 tx=0).stop 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f148e600640 1 -- 172.21.15.67:0/1612925791 reap_dead start 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f148f000640 1 -- 172.21.15.67:0/1612925791 shutdown_connections 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f148f000640 1 -- 172.21.15.67:0/1612925791 >> 172.21.15.67:0/1612925791 conn(0x7f14880b1fc0 msgr2=0x7f14880b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f148f000640 1 -- 172.21.15.67:0/1612925791 shutdown_connections 2024-05-30T17:00:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.076+0000 7f148f000640 1 -- 172.21.15.67:0/1612925791 wait complete. 2024-05-30T17:00:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.078+0000 7f148f000640 1 Processor -- start 2024-05-30T17:00:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.078+0000 7f148f000640 1 -- start start 2024-05-30T17:00:52.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.079+0000 7f148f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1470004800 0x7f14881455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.079+0000 7f148f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14881593b0 0x7f148815d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.079+0000 7f148f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14880b09a0 0x7f14880b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.079+0000 7f148f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14880b2db0 con 0x7f14881593b0 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.079+0000 7f148f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14880b2c30 con 0x7f1470004800 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.079+0000 7f148f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14880ad770 con 0x7f14880b09a0 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14881593b0 0x7f148815d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14881593b0 0x7f148815d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36462/0 (socket says 172.21.15.67:36462) 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148d200640 1 -- 172.21.15.67:0/3690274252 learned_addr learned my addr 172.21.15.67:0/3690274252 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148dc00640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1470004800 0x7f14881455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148e600640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14880b09a0 0x7f14880b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148d200640 1 -- 172.21.15.67:0/3690274252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14880b09a0 msgr2=0x7f14880b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148d200640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14880b09a0 0x7f14880b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148d200640 1 -- 172.21.15.67:0/3690274252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1470004800 msgr2=0x7f14881455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148d200640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1470004800 0x7f14881455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.080+0000 7f148d200640 1 -- 172.21.15.67:0/3690274252 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14881f43f0 con 0x7f14881593b0 2024-05-30T17:00:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.081+0000 7f148d200640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14881593b0 0x7f148815d2c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f147400ba30 tx=0x7f1474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.081+0000 7f1485600640 1 -- 172.21.15.67:0/3690274252 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474043020 con 0x7f14881593b0 2024-05-30T17:00:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.081+0000 7f1485600640 1 -- 172.21.15.67:0/3690274252 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147403f680 con 0x7f14881593b0 2024-05-30T17:00:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.081+0000 7f1485600640 1 -- 172.21.15.67:0/3690274252 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147400b600 con 0x7f14881593b0 2024-05-30T17:00:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.081+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148815dbb0 con 0x7f14881593b0 2024-05-30T17:00:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.081+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f148815e170 con 0x7f14881593b0 2024-05-30T17:00:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.083+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14880689e0 con 0x7f14881593b0 2024-05-30T17:00:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.084+0000 7f1485600640 1 -- 172.21.15.67:0/3690274252 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f1474046020 con 0x7f14881593b0 2024-05-30T17:00:52.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.086+0000 7f1485600640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f146806ddf0 0x7f14680702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.086+0000 7f1485600640 1 -- 172.21.15.67:0/3690274252 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f14740ab810 con 0x7f14881593b0 2024-05-30T17:00:52.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.086+0000 7f148dc00640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f146806ddf0 0x7f14680702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.087+0000 7f148dc00640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f146806ddf0 0x7f14680702b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f147c001eb0 tx=0x7f147c001620 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.091+0000 7f1485600640 1 -- 172.21.15.67:0/3690274252 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f147404d020 con 0x7f14881593b0 2024-05-30T17:00:52.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.321+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1488073030 con 0x7f146806ddf0 2024-05-30T17:00:52.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.324+0000 7f1485600640 1 -- 172.21.15.67:0/3690274252 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1488073030 con 0x7f146806ddf0 2024-05-30T17:00:52.325 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:00:52.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:00:52.325 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T17:00:52.325 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:00:52.325 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:00:52.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f146806ddf0 msgr2=0x7f14680702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f146806ddf0 0x7f14680702b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f147c001eb0 tx=0x7f147c001620 comp rx=0 tx=0).stop 2024-05-30T17:00:52.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14881593b0 msgr2=0x7f148815d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14881593b0 0x7f148815d2c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f147400ba30 tx=0x7f1474004730 comp rx=0 tx=0).stop 2024-05-30T17:00:52.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 shutdown_connections 2024-05-30T17:00:52.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f146806ddf0 0x7f14680702b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14880b09a0 0x7f14880b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14881593b0 0x7f148815d2c0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 --2- 172.21.15.67:0/3690274252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1470004800 0x7f14881455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 >> 172.21.15.67:0/3690274252 conn(0x7f14880b1fc0 msgr2=0x7f1488160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:52.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 shutdown_connections 2024-05-30T17:00:52.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.327+0000 7f148f000640 1 -- 172.21.15.67:0/3690274252 wait complete. 2024-05-30T17:00:52.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T17:00:52.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.561+0000 7f0aeec00640 1 Processor -- start 2024-05-30T17:00:52.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.561+0000 7f0aeec00640 1 -- start start 2024-05-30T17:00:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.562+0000 7f0aeec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0ae8161e70 0x7f0ae80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.563+0000 7f0aeec00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae80b3d40 con 0x7f0ae8161a90 2024-05-30T17:00:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.563+0000 7f0aeec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae80b4020 con 0x7f0ae80b3800 2024-05-30T17:00:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.563+0000 7f0aeec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0ae80b4300 con 0x7f0ae8161e70 2024-05-30T17:00:52.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.563+0000 7f0ae7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0ae8161a90 0x7f0ae806e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34818/0 (socket says 172.21.15.67:34818) 2024-05-30T17:00:52.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.563+0000 7f0ae7e00640 1 -- 172.21.15.67:0/140434032 learned_addr learned my addr 172.21.15.67:0/140434032 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:00:52.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.563+0000 7f0ae7400640 1 --2- 172.21.15.67:0/140434032 >> v2:172.21.15.169:3300/0 conn(0x7f0ae8161e70 0x7f0ae80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.563+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4041440939 0 0) 0x7f0ae80b3d40 con 0x7f0ae8161a90 2024-05-30T17:00:52.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0acc003520 con 0x7f0ae8161a90 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3956047678 0 0) 0x7f0acc003520 con 0x7f0ae8161a90 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 >> v2:172.21.15.169:3300/0 conn(0x7f0ae8161e70 msgr2=0x7f0ae80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 --2- 172.21.15.67:0/140434032 >> v2:172.21.15.169:3300/0 conn(0x7f0ae8161e70 0x7f0ae80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 >> v1:172.21.15.169:6789/0 conn(0x7f0ae80b3800 legacy=0x7f0ae80ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae806dc70 con 0x7f0ae8161a90 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f0ad8001670 con 0x7f0ae8161a90 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 >> v1:172.21.15.67:6789/0 conn(0x7f0ae8161a90 legacy=0x7f0ae806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0acc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0acc004800 0x7f0acc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0acc0051c0 0x7f0acc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae80b4300 con 0x7f0acc0039b0 2024-05-30T17:00:52.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc00c060 con 0x7f0acc004800 2024-05-30T17:00:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0acc00c260 con 0x7f0acc0051c0 2024-05-30T17:00:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.564+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0acc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.565+0000 7f0ae7400640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0acc004800 0x7f0acc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.565+0000 7f0ae7e00640 1 -- 172.21.15.67:0/140434032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0acc0051c0 msgr2=0x7f0acc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.565+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0acc0051c0 0x7f0acc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.565+0000 7f0ae7e00640 1 -- 172.21.15.67:0/140434032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0acc004800 msgr2=0x7f0acc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.565+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0acc004800 0x7f0acc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.565+0000 7f0ae7e00640 1 -- 172.21.15.67:0/140434032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc003520 con 0x7f0acc0039b0 2024-05-30T17:00:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.565+0000 7f0aecc00640 1 -- 172.21.15.67:0/140434032 reap_dead start 2024-05-30T17:00:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.566+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0acc003dd0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c240 tx=0x7f0acc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447f9b4719598462 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.566+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8007cf0 con 0x7f0acc0039b0 2024-05-30T17:00:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.566+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad80058d0 con 0x7f0acc0039b0 2024-05-30T17:00:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.566+0000 7f0ae6a00640 1 -- 172.21.15.67:0/140434032 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8005d20 con 0x7f0acc0039b0 2024-05-30T17:00:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.567+0000 7f0aeec00640 1 -- 172.21.15.67:0/140434032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 msgr2=0x7f0acc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.567+0000 7f0aeec00640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0acc003dd0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c240 tx=0x7f0acc005830 comp rx=0 tx=0).stop 2024-05-30T17:00:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.567+0000 7f0aeec00640 1 -- 172.21.15.67:0/140434032 shutdown_connections 2024-05-30T17:00:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.567+0000 7f0aeec00640 1 --2- 172.21.15.67:0/140434032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0acc003dd0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.567+0000 7f0aeec00640 1 -- 172.21.15.67:0/140434032 >> 172.21.15.67:0/140434032 conn(0x7f0ae806bd20 msgr2=0x7f0ae806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.567+0000 7f0aeec00640 1 -- 172.21.15.67:0/140434032 shutdown_connections 2024-05-30T17:00:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.567+0000 7f0aeec00640 1 -- 172.21.15.67:0/140434032 wait complete. 2024-05-30T17:00:52.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.569+0000 7f0aeec00640 1 Processor -- start 2024-05-30T17:00:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.569+0000 7f0aeec00640 1 -- start start 2024-05-30T17:00:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.570+0000 7f0aeec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0ae8156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.570+0000 7f0aeec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ae81442f0 0x7f0ae81f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.570+0000 7f0aeec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae81401c0 0x7f0ae8140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.570+0000 7f0aeec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae80b32c0 con 0x7f0acc0039b0 2024-05-30T17:00:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.570+0000 7f0aeec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae80b5020 con 0x7f0ae81401c0 2024-05-30T17:00:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.570+0000 7f0aeec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae80b3440 con 0x7f0ae81442f0 2024-05-30T17:00:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.570+0000 7f0ae7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0ae8156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.570+0000 7f0ae7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0ae8156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36476/0 (socket says 172.21.15.67:36476) 2024-05-30T17:00:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.570+0000 7f0ae7e00640 1 -- 172.21.15.67:0/2616668027 learned_addr learned my addr 172.21.15.67:0/2616668027 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.571+0000 7f0ae7e00640 1 -- 172.21.15.67:0/2616668027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ae81442f0 msgr2=0x7f0ae81f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.571+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ae81442f0 0x7f0ae81f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.571+0000 7f0ae7e00640 1 -- 172.21.15.67:0/2616668027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae81401c0 msgr2=0x7f0ae8140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.571+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae81401c0 0x7f0ae8140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.571+0000 7f0ae7e00640 1 -- 172.21.15.67:0/2616668027 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae8155660 con 0x7f0acc0039b0 2024-05-30T17:00:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.571+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0ae8156700 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c240 tx=0x7f0ae80b41f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.571+0000 7f0adfe00640 1 -- 172.21.15.67:0/2616668027 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8007cf0 con 0x7f0acc0039b0 2024-05-30T17:00:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.571+0000 7f0adfe00640 1 -- 172.21.15.67:0/2616668027 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad8040070 con 0x7f0acc0039b0 2024-05-30T17:00:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.571+0000 7f0adfe00640 1 -- 172.21.15.67:0/2616668027 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8001c30 con 0x7f0acc0039b0 2024-05-30T17:00:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.572+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae8155940 con 0x7f0acc0039b0 2024-05-30T17:00:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.572+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae8155f00 con 0x7f0acc0039b0 2024-05-30T17:00:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.573+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae80682d0 con 0x7f0acc0039b0 2024-05-30T17:00:52.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.574+0000 7f0adfe00640 1 -- 172.21.15.67:0/2616668027 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f0ad80047e0 con 0x7f0acc0039b0 2024-05-30T17:00:52.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.575+0000 7f0adfe00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ac406ddc0 0x7f0ac4070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:52.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.575+0000 7f0adfe00640 1 -- 172.21.15.67:0/2616668027 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f0ad80aa920 con 0x7f0acc0039b0 2024-05-30T17:00:52.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.575+0000 7f0ae7400640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ac406ddc0 0x7f0ac4070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.576+0000 7f0ae7400640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ac406ddc0 0x7f0ac4070280 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ae813e000 tx=0x7f0ad0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.579+0000 7f0adfe00640 1 -- 172.21.15.67:0/2616668027 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ae80682d0 con 0x7f0acc0039b0 2024-05-30T17:00:52.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.791+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0ae815da90 con 0x7f0acc0039b0 2024-05-30T17:00:52.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.792+0000 7f0adfe00640 1 -- 172.21.15.67:0/2616668027 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f0ad807a420 con 0x7f0acc0039b0 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ac406ddc0 msgr2=0x7f0ac4070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ac406ddc0 0x7f0ac4070280 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ae813e000 tx=0x7f0ad0008040 comp rx=0 tx=0).stop 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 msgr2=0x7f0ae8156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0ae8156700 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800c240 tx=0x7f0ae80b41f0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 shutdown_connections 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ac406ddc0 0x7f0ac4070280 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ae81401c0 0x7f0ae8140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ae81442f0 0x7f0ae81f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 --2- 172.21.15.67:0/2616668027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc0039b0 0x7f0ae8156700 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:52.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 >> 172.21.15.67:0/2616668027 conn(0x7f0ae806bd20 msgr2=0x7f0ae806e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:52.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 shutdown_connections 2024-05-30T17:00:52.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:52.795+0000 7f0aeec00640 1 -- 172.21.15.67:0/2616668027 wait complete. 2024-05-30T17:00:52.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-30T17:00:53.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.004+0000 7f5dff000640 1 Processor -- start 2024-05-30T17:00:53.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.005+0000 7f5dff000640 1 -- start start 2024-05-30T17:00:53.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.006+0000 7f5dff000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5df815e590 0x7f5df81621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dff000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5df80d1ce0 0x7f5df806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dff000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df8162830 con 0x7f5df80cc8b0 2024-05-30T17:00:53.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dff000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5df81630d0 con 0x7f5df80d1ce0 2024-05-30T17:00:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dff000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5df80d0390 con 0x7f5df815e590 2024-05-30T17:00:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dfd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5df80d1ce0 0x7f5df806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dfd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5df80d1ce0 0x7f5df806c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36486/0 (socket says 172.21.15.67:36486) 2024-05-30T17:00:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dfd200640 1 -- 172.21.15.67:0/3357676489 learned_addr learned my addr 172.21.15.67:0/3357676489 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dfd200640 1 -- 172.21.15.67:0/3357676489 >> v2:172.21.15.169:3300/0 conn(0x7f5df815e590 msgr2=0x7f5df81621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dfdc00640 1 --2- 172.21.15.67:0/3357676489 >> v2:172.21.15.169:3300/0 conn(0x7f5df815e590 0x7f5df81621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dfd200640 1 --2- 172.21.15.67:0/3357676489 >> v2:172.21.15.169:3300/0 conn(0x7f5df815e590 0x7f5df81621c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dfd200640 1 -- 172.21.15.67:0/3357676489 >> v1:172.21.15.169:6789/0 conn(0x7f5df80cc8b0 legacy=0x7f5df80d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.007+0000 7f5dfd200640 1 -- 172.21.15.67:0/3357676489 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df8162b10 con 0x7f5df80d1ce0 2024-05-30T17:00:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.008+0000 7f5dfd200640 1 --2- 172.21.15.67:0/3357676489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80d1ce0 0x7f5df806c7b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f5df806c680 tx=0x7f5de402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04744c16ccba671 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.009+0000 7f5df7e00640 1 -- 172.21.15.67:0/3357676489 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4037070 con 0x7f5df80d1ce0 2024-05-30T17:00:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.009+0000 7f5df7e00640 1 -- 172.21.15.67:0/3357676489 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de4032440 con 0x7f5df80d1ce0 2024-05-30T17:00:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.009+0000 7f5df7e00640 1 -- 172.21.15.67:0/3357676489 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4032810 con 0x7f5df80d1ce0 2024-05-30T17:00:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.010+0000 7f5dff000640 1 -- 172.21.15.67:0/3357676489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80d1ce0 msgr2=0x7f5df806c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.010+0000 7f5dff000640 1 --2- 172.21.15.67:0/3357676489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80d1ce0 0x7f5df806c7b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f5df806c680 tx=0x7f5de402a1a0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.010+0000 7f5dff000640 1 -- 172.21.15.67:0/3357676489 shutdown_connections 2024-05-30T17:00:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.010+0000 7f5dff000640 1 --2- 172.21.15.67:0/3357676489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80d1ce0 0x7f5df806c7b0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.010+0000 7f5dff000640 1 --2- 172.21.15.67:0/3357676489 >> v2:172.21.15.169:3300/0 conn(0x7f5df815e590 0x7f5df81621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.010+0000 7f5dff000640 1 -- 172.21.15.67:0/3357676489 >> 172.21.15.67:0/3357676489 conn(0x7f5df806fbf0 msgr2=0x7f5df8070000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.010+0000 7f5dff000640 1 -- 172.21.15.67:0/3357676489 shutdown_connections 2024-05-30T17:00:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.010+0000 7f5dff000640 1 -- 172.21.15.67:0/3357676489 wait complete. 2024-05-30T17:00:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.011+0000 7f5dff000640 1 Processor -- start 2024-05-30T17:00:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.012+0000 7f5dff000640 1 -- start start 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.012+0000 7f5dff000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80cc8b0 0x7f5df81590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.012+0000 7f5dff000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5df80d1ce0 0x7f5df80cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dff000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df815e590 0x7f5df80c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dff000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5df815e970 con 0x7f5df80cc8b0 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dff000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5df80cf6d0 con 0x7f5df815e590 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dff000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5df80cbd90 con 0x7f5df80d1ce0 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dfdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80cc8b0 0x7f5df81590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dfdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80cc8b0 0x7f5df81590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36488/0 (socket says 172.21.15.67:36488) 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dfdc00640 1 -- 172.21.15.67:0/379703537 learned_addr learned my addr 172.21.15.67:0/379703537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dfdc00640 1 -- 172.21.15.67:0/379703537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5df80d1ce0 msgr2=0x7f5df80cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dfdc00640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5df80d1ce0 0x7f5df80cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dfdc00640 1 -- 172.21.15.67:0/379703537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df815e590 msgr2=0x7f5df80c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dfdc00640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df815e590 0x7f5df80c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dfdc00640 1 -- 172.21.15.67:0/379703537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df81f50b0 con 0x7f5df80cc8b0 2024-05-30T17:00:53.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.013+0000 7f5dfdc00640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80cc8b0 0x7f5df81590f0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00ce40 tx=0x7f5dec00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:53.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.014+0000 7f5df5600640 1 -- 172.21.15.67:0/379703537 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec003b90 con 0x7f5df80cc8b0 2024-05-30T17:00:53.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.014+0000 7f5df5600640 1 -- 172.21.15.67:0/379703537 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dec003d30 con 0x7f5df80cc8b0 2024-05-30T17:00:53.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.014+0000 7f5df5600640 1 -- 172.21.15.67:0/379703537 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec004240 con 0x7f5df80cc8b0 2024-05-30T17:00:53.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.014+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df81f4dd0 con 0x7f5df80cc8b0 2024-05-30T17:00:53.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.014+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df80ca480 con 0x7f5df80cc8b0 2024-05-30T17:00:53.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.015+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df80690f0 con 0x7f5df80cc8b0 2024-05-30T17:00:53.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.016+0000 7f5df5600640 1 -- 172.21.15.67:0/379703537 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f5dec0043e0 con 0x7f5df80cc8b0 2024-05-30T17:00:53.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.017+0000 7f5df5600640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5dd806ddc0 0x7f5dd8070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.017+0000 7f5df5600640 1 -- 172.21.15.67:0/379703537 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f5dec08b770 con 0x7f5df80cc8b0 2024-05-30T17:00:53.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.017+0000 7f5dfd200640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5dd806ddc0 0x7f5dd8070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.018+0000 7f5dfd200640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5dd806ddc0 0x7f5dd8070280 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5de4034000 tx=0x7f5de402c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:53.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.021+0000 7f5df5600640 1 -- 172.21.15.67:0/379703537 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dec01d340 con 0x7f5df80cc8b0 2024-05-30T17:00:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.244+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f5df80d2e00 con 0x7f5dd806ddc0 2024-05-30T17:00:53.256 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:53.255+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:53.256 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:53.255+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:53.256 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:53.255+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:53.256 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:53.255+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:53.256 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:00:53.255+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:00:53.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.303+0000 7f5df5600640 1 -- 172.21.15.67:0/379703537 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5df80d2e00 con 0x7f5dd806ddc0 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5dd806ddc0 msgr2=0x7f5dd8070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5dd806ddc0 0x7f5dd8070280 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5de4034000 tx=0x7f5de402c000 comp rx=0 tx=0).stop 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80cc8b0 msgr2=0x7f5df81590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80cc8b0 0x7f5df81590f0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00ce40 tx=0x7f5dec00a660 comp rx=0 tx=0).stop 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 shutdown_connections 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5dd806ddc0 0x7f5dd8070280 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df815e590 0x7f5df80c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5df80d1ce0 0x7f5df80cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 --2- 172.21.15.67:0/379703537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5df80cc8b0 0x7f5df81590f0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 >> 172.21.15.67:0/379703537 conn(0x7f5df806fbf0 msgr2=0x7f5df806de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:53.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 shutdown_connections 2024-05-30T17:00:53.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.306+0000 7f5dff000640 1 -- 172.21.15.67:0/379703537 wait complete. 2024-05-30T17:00:53.325 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T17:00:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.525+0000 7f4bf2800640 1 Processor -- start 2024-05-30T17:00:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.525+0000 7f4bf2800640 1 -- start start 2024-05-30T17:00:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.527+0000 7f4bf2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4bec15e550 0x7f4bec0ae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.527+0000 7f4bf2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4bec0704d0 0x7f4bec0b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.527+0000 7f4bf2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bec0b3d70 con 0x7f4bec06edb0 2024-05-30T17:00:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.527+0000 7f4bf2800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4bec0b4050 con 0x7f4bec15e550 2024-05-30T17:00:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.527+0000 7f4bf2800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4bec0b4330 con 0x7f4bec0704d0 2024-05-30T17:00:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.527+0000 7f4bf1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4bec15e550 0x7f4bec0ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.527+0000 7f4bf1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4bec15e550 0x7f4bec0ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36518/0 (socket says 172.21.15.67:36518) 2024-05-30T17:00:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.527+0000 7f4bf1400640 1 -- 172.21.15.67:0/145704156 learned_addr learned my addr 172.21.15.67:0/145704156 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:53.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.528+0000 7f4bf1e00640 1 --2- 172.21.15.67:0/145704156 >> v2:172.21.15.169:3300/0 conn(0x7f4bec0704d0 0x7f4bec0b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.528+0000 7f4bf1400640 1 -- 172.21.15.67:0/145704156 >> v2:172.21.15.169:3300/0 conn(0x7f4bec0704d0 msgr2=0x7f4bec0b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:53.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.528+0000 7f4bf1400640 1 --2- 172.21.15.67:0/145704156 >> v2:172.21.15.169:3300/0 conn(0x7f4bec0704d0 0x7f4bec0b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.528+0000 7f4bf1400640 1 -- 172.21.15.67:0/145704156 >> v1:172.21.15.169:6789/0 conn(0x7f4bec06edb0 legacy=0x7f4bec06fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.528+0000 7f4bf1400640 1 -- 172.21.15.67:0/145704156 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec0b4610 con 0x7f4bec15e550 2024-05-30T17:00:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.528+0000 7f4bf1e00640 1 --2- 172.21.15.67:0/145704156 >> v2:172.21.15.169:3300/0 conn(0x7f4bec0704d0 0x7f4bec0b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.529+0000 7f4bf1400640 1 --2- 172.21.15.67:0/145704156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 0x7f4bec0ae870 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4bec073c10 tx=0x7f4be002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5c14fe4761eca server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.529+0000 7f4bebe00640 1 -- 172.21.15.67:0/145704156 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0037070 con 0x7f4bec15e550 2024-05-30T17:00:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.529+0000 7f4bebe00640 1 -- 172.21.15.67:0/145704156 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be0032440 con 0x7f4bec15e550 2024-05-30T17:00:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.529+0000 7f4bebe00640 1 -- 172.21.15.67:0/145704156 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0032810 con 0x7f4bec15e550 2024-05-30T17:00:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.530+0000 7f4bf2800640 1 -- 172.21.15.67:0/145704156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 msgr2=0x7f4bec0ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.530+0000 7f4bf2800640 1 --2- 172.21.15.67:0/145704156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 0x7f4bec0ae870 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4bec073c10 tx=0x7f4be002a230 comp rx=0 tx=0).stop 2024-05-30T17:00:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.530+0000 7f4bf2800640 1 -- 172.21.15.67:0/145704156 shutdown_connections 2024-05-30T17:00:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.530+0000 7f4bf2800640 1 --2- 172.21.15.67:0/145704156 >> v2:172.21.15.169:3300/0 conn(0x7f4bec0704d0 0x7f4bec0b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.530+0000 7f4bf2800640 1 --2- 172.21.15.67:0/145704156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 0x7f4bec0ae870 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.530+0000 7f4bf2800640 1 -- 172.21.15.67:0/145704156 >> 172.21.15.67:0/145704156 conn(0x7f4bec073280 msgr2=0x7f4bec06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.530+0000 7f4bf2800640 1 -- 172.21.15.67:0/145704156 shutdown_connections 2024-05-30T17:00:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.530+0000 7f4bf2800640 1 -- 172.21.15.67:0/145704156 wait complete. 2024-05-30T17:00:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.532+0000 7f4bf2800640 1 Processor -- start 2024-05-30T17:00:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.532+0000 7f4bf2800640 1 -- start start 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.533+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec06edb0 0x7f4bec1e8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.533+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0704d0 0x7f4bec0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.533+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 0x7f4bec14fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.533+0000 7f4bf2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec0b5fb0 con 0x7f4bec15e550 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.533+0000 7f4bf2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec0b5e30 con 0x7f4bec06edb0 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.533+0000 7f4bf2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4bec0aedb0 con 0x7f4bec0704d0 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 0x7f4bec14fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 0x7f4bec14fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36526/0 (socket says 172.21.15.67:36526) 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf1e00640 1 -- 172.21.15.67:0/3448981720 learned_addr learned my addr 172.21.15.67:0/3448981720 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf1400640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec06edb0 0x7f4bec1e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf0a00640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0704d0 0x7f4bec0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf1e00640 1 -- 172.21.15.67:0/3448981720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0704d0 msgr2=0x7f4bec0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf1e00640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0704d0 0x7f4bec0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf1e00640 1 -- 172.21.15.67:0/3448981720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec06edb0 msgr2=0x7f4bec1e8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf1e00640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec06edb0 0x7f4bec1e8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf1e00640 1 -- 172.21.15.67:0/3448981720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec1e8ea0 con 0x7f4bec15e550 2024-05-30T17:00:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.534+0000 7f4bf0a00640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0704d0 0x7f4bec0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.535+0000 7f4bf1e00640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 0x7f4bec14fec0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4be400cf20 tx=0x7f4be400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.535+0000 7f4be9600640 1 -- 172.21.15.67:0/3448981720 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be401a070 con 0x7f4bec15e550 2024-05-30T17:00:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.535+0000 7f4be9600640 1 -- 172.21.15.67:0/3448981720 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be400d840 con 0x7f4bec15e550 2024-05-30T17:00:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.535+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec1e9cb0 con 0x7f4bec15e550 2024-05-30T17:00:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.535+0000 7f4be9600640 1 -- 172.21.15.67:0/3448981720 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be400dc30 con 0x7f4bec15e550 2024-05-30T17:00:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.535+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec14f6c0 con 0x7f4bec15e550 2024-05-30T17:00:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.536+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec138ca0 con 0x7f4bec15e550 2024-05-30T17:00:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.537+0000 7f4be9600640 1 -- 172.21.15.67:0/3448981720 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f4be401d070 con 0x7f4bec15e550 2024-05-30T17:00:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.538+0000 7f4be9600640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4bcc06ddc0 0x7f4bcc070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.538+0000 7f4be9600640 1 -- 172.21.15.67:0/3448981720 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f4be408b120 con 0x7f4bec15e550 2024-05-30T17:00:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.539+0000 7f4bf1400640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4bcc06ddc0 0x7f4bcc070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.539+0000 7f4bf1400640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4bcc06ddc0 0x7f4bcc070280 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4be0034000 tx=0x7f4be002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.543+0000 7f4be9600640 1 -- 172.21.15.67:0/3448981720 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bec138ca0 con 0x7f4bec15e550 2024-05-30T17:00:53.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.776+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4bec0689e0 con 0x7f4bcc06ddc0 2024-05-30T17:00:53.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.782+0000 7f4be9600640 1 -- 172.21.15.67:0/3448981720 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f4bec0689e0 con 0x7f4bcc06ddc0 2024-05-30T17:00:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f 2024-05-30T17:00:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.784+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4bcc06ddc0 msgr2=0x7f4bcc070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.784+0000 7f4bf2800640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4bcc06ddc0 0x7f4bcc070280 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4be0034000 tx=0x7f4be002e000 comp rx=0 tx=0).stop 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.784+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 msgr2=0x7f4bec14fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.784+0000 7f4bf2800640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 0x7f4bec14fec0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4be400cf20 tx=0x7f4be400a7d0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.785+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 shutdown_connections 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.785+0000 7f4bf2800640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4bcc06ddc0 0x7f4bcc070280 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.785+0000 7f4bf2800640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bec15e550 0x7f4bec14fec0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.785+0000 7f4bf2800640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bec0704d0 0x7f4bec0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.785+0000 7f4bf2800640 1 --2- 172.21.15.67:0/3448981720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bec06edb0 0x7f4bec1e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.785+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 >> 172.21.15.67:0/3448981720 conn(0x7f4bec073280 msgr2=0x7f4bec06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.785+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 shutdown_connections 2024-05-30T17:00:53.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:53.785+0000 7f4bf2800640 1 -- 172.21.15.67:0/3448981720 wait complete. 2024-05-30T17:00:53.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T17:00:54.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.004+0000 7f4b12600640 1 Processor -- start 2024-05-30T17:00:54.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.005+0000 7f4b12600640 1 -- start start 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.006+0000 7f4b12600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4b0c15e870 0x7f4b0c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.006+0000 7f4b12600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b0c0d14d0 0x7f4b0c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.006+0000 7f4b12600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0c0cfce0 con 0x7f4b0c0cf870 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.006+0000 7f4b12600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0c0d2150 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.006+0000 7f4b12600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0c15ec50 con 0x7f4b0c15e870 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.007+0000 7f4b0be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b0c0d14d0 0x7f4b0c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.007+0000 7f4b0be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b0c0d14d0 0x7f4b0c0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36536/0 (socket says 172.21.15.67:36536) 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.007+0000 7f4b0be00640 1 -- 172.21.15.67:0/602257797 learned_addr learned my addr 172.21.15.67:0/602257797 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.007+0000 7f4b11200640 1 --2- 172.21.15.67:0/602257797 >> v2:172.21.15.169:3300/0 conn(0x7f4b0c15e870 0x7f4b0c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.007+0000 7f4b0be00640 1 -- 172.21.15.67:0/602257797 >> v2:172.21.15.169:3300/0 conn(0x7f4b0c15e870 msgr2=0x7f4b0c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.007+0000 7f4b0be00640 1 --2- 172.21.15.67:0/602257797 >> v2:172.21.15.169:3300/0 conn(0x7f4b0c15e870 0x7f4b0c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.007+0000 7f4b0be00640 1 -- 172.21.15.67:0/602257797 >> v1:172.21.15.169:6789/0 conn(0x7f4b0c0cf870 legacy=0x7f4b0c162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:00:54.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.007+0000 7f4b0be00640 1 -- 172.21.15.67:0/602257797 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c0cff80 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.008+0000 7f4b0be00640 1 --2- 172.21.15.67:0/602257797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 0x7f4b0c0cf330 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c08e720 tx=0x7f4af402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de354feb835210fe server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:54.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.009+0000 7f4b0b400640 1 -- 172.21.15.67:0/602257797 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4004030 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.009+0000 7f4b0b400640 1 -- 172.21.15.67:0/602257797 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af40041d0 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.009+0000 7f4b0b400640 1 -- 172.21.15.67:0/602257797 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4003610 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.009+0000 7f4b12600640 1 -- 172.21.15.67:0/602257797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 msgr2=0x7f4b0c0cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.009+0000 7f4b12600640 1 --2- 172.21.15.67:0/602257797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 0x7f4b0c0cf330 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c08e720 tx=0x7f4af402a210 comp rx=0 tx=0).stop 2024-05-30T17:00:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.010+0000 7f4b12600640 1 -- 172.21.15.67:0/602257797 shutdown_connections 2024-05-30T17:00:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.010+0000 7f4b12600640 1 --2- 172.21.15.67:0/602257797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 0x7f4b0c0cf330 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.010+0000 7f4b12600640 1 --2- 172.21.15.67:0/602257797 >> v2:172.21.15.169:3300/0 conn(0x7f4b0c15e870 0x7f4b0c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.010+0000 7f4b12600640 1 -- 172.21.15.67:0/602257797 >> 172.21.15.67:0/602257797 conn(0x7f4b0c08bfc0 msgr2=0x7f4b0c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.010+0000 7f4b12600640 1 -- 172.21.15.67:0/602257797 shutdown_connections 2024-05-30T17:00:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.010+0000 7f4b12600640 1 -- 172.21.15.67:0/602257797 wait complete. 2024-05-30T17:00:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.011+0000 7f4b12600640 1 Processor -- start 2024-05-30T17:00:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.012+0000 7f4b12600640 1 -- start start 2024-05-30T17:00:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.012+0000 7f4b12600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b0c0cf870 0x7f4b0c159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.012+0000 7f4b12600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 0x7f4b0c0b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.012+0000 7f4b12600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b0c15e870 0x7f4b0c0b9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.012+0000 7f4b12600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c0d0820 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.012+0000 7f4b12600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c0d0540 con 0x7f4b0c0cf870 2024-05-30T17:00:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.012+0000 7f4b12600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0c0d4fb0 con 0x7f4b0c15e870 2024-05-30T17:00:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 0x7f4b0c0b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 0x7f4b0c0b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36546/0 (socket says 172.21.15.67:36546) 2024-05-30T17:00:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b0be00640 1 -- 172.21.15.67:0/2031966248 learned_addr learned my addr 172.21.15.67:0/2031966248 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b11200640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b0c0cf870 0x7f4b0c159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b11c00640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b0c15e870 0x7f4b0c0b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b0be00640 1 -- 172.21.15.67:0/2031966248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b0c15e870 msgr2=0x7f4b0c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b0be00640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b0c15e870 0x7f4b0c0b9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b0be00640 1 -- 172.21.15.67:0/2031966248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b0c0cf870 msgr2=0x7f4b0c159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b0be00640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b0c0cf870 0x7f4b0c159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b0be00640 1 -- 172.21.15.67:0/2031966248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c0b4f30 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.013+0000 7f4b11c00640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b0c15e870 0x7f4b0c0b9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.014+0000 7f4b0be00640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 0x7f4b0c0b3fb0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f4af4034040 tx=0x7f4af40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.014+0000 7f4b08c00640 1 -- 172.21.15.67:0/2031966248 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4041020 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.014+0000 7f4b08c00640 1 -- 172.21.15.67:0/2031966248 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af4003c50 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.014+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0c0b4970 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.014+0000 7f4b08c00640 1 -- 172.21.15.67:0/2031966248 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4004030 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.014+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0c15ef00 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.015+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0c0689e0 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.016+0000 7f4b08c00640 1 -- 172.21.15.67:0/2031966248 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 35) ==== 92927+0+0 (secure 0 0 0) 0x7f4af4044020 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.018+0000 7f4b08c00640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4ae806ddc0 0x7f4ae8070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:00:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.018+0000 7f4b08c00640 1 -- 172.21.15.67:0/2031966248 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f4af40ab390 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.018+0000 7f4b11200640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4ae806ddc0 0x7f4ae8070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:00:54.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.019+0000 7f4b11200640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4ae806ddc0 0x7f4ae8070280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009f30 tx=0x7f4afc00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:00:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.022+0000 7f4b08c00640 1 -- 172.21.15.67:0/2031966248 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4af404b030 con 0x7f4b0c0d14d0 2024-05-30T17:00:54.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.256+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4b0c0cb2f0 con 0x7f4ae806ddc0 2024-05-30T17:00:54.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.266+0000 7f4b08c00640 1 -- 172.21.15.67:0/2031966248 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f4b0c0cb2f0 con 0x7f4ae806ddc0 2024-05-30T17:00:54.266 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4ae806ddc0 msgr2=0x7f4ae8070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4ae806ddc0 0x7f4ae8070280 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009f30 tx=0x7f4afc00b040 comp rx=0 tx=0).stop 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 msgr2=0x7f4b0c0b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 0x7f4b0c0b3fb0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f4af4034040 tx=0x7f4af40046c0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 shutdown_connections 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4ae806ddc0 0x7f4ae8070280 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b0c15e870 0x7f4b0c0b9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b0c0d14d0 0x7f4b0c0b3fb0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 --2- 172.21.15.67:0/2031966248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b0c0cf870 0x7f4b0c159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 >> 172.21.15.67:0/2031966248 conn(0x7f4b0c08bfc0 msgr2=0x7f4b0c0ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:00:54.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 shutdown_connections 2024-05-30T17:00:54.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:00:54.269+0000 7f4b12600640 1 -- 172.21.15.67:0/2031966248 wait complete. 2024-05-30T17:00:54.288 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f 2024-05-30T17:00:54.289 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:00:54.289 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:00:54.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:00:54.340 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T17:01:10.324 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T17:01:10.325 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T17:01:10.325 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.9011 s, 67.5 MB/s 2024-05-30T17:01:10.326 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:01:10.326 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:01:10.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:01:10.384 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T17:01:15.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:01:15.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:01:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T17:01:24.223 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T17:01:24.223 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.7817 s, 77.9 MB/s 2024-05-30T17:01:24.225 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:01:24.225 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:01:24.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:01:24.278 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T17:01:38.361 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T17:01:38.361 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T17:01:38.361 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.9732 s, 76.8 MB/s 2024-05-30T17:01:38.363 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-30T17:01:38.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.575+0000 7f2f95600640 1 Processor -- start 2024-05-30T17:01:38.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.575+0000 7f2f95600640 1 -- start start 2024-05-30T17:01:38.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.577+0000 7f2f95600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2f900ce560 0x7f2f900b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:38.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.577+0000 7f2f95600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f900d0430 0x7f2f900d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:38.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.577+0000 7f2f95600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f900d0d70 con 0x7f2f9015f0c0 2024-05-30T17:01:38.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.577+0000 7f2f95600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2f9015bb30 con 0x7f2f900ce560 2024-05-30T17:01:38.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.577+0000 7f2f95600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2f900b3ba0 con 0x7f2f900d0430 2024-05-30T17:01:38.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.578+0000 7f2f94c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2f9015f0c0 0x7f2f9015f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38508/0 (socket says 172.21.15.67:38508) 2024-05-30T17:01:38.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.578+0000 7f2f94c00640 1 -- 172.21.15.67:0/408288569 learned_addr learned my addr 172.21.15.67:0/408288569 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:01:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.578+0000 7f2f8fe00640 1 --2- 172.21.15.67:0/408288569 >> v2:172.21.15.169:3300/0 conn(0x7f2f900ce560 0x7f2f900b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.578+0000 7f2f8f400640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f900d0430 0x7f2f900d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.578+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1403613585 0 0) 0x7f2f900d0d70 con 0x7f2f9015f0c0 2024-05-30T17:01:38.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.578+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f74003520 con 0x7f2f9015f0c0 2024-05-30T17:01:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.578+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 298856209 0 0) 0x7f2f74003520 con 0x7f2f9015f0c0 2024-05-30T17:01:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f900d0430 msgr2=0x7f2f900d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f900d0430 0x7f2f900d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 >> v2:172.21.15.169:3300/0 conn(0x7f2f900ce560 msgr2=0x7f2f900b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 --2- 172.21.15.67:0/408288569 >> v2:172.21.15.169:3300/0 conn(0x7f2f900ce560 0x7f2f900b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9015bdd0 con 0x7f2f9015f0c0 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f2f880019f0 con 0x7f2f9015f0c0 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 >> v1:172.21.15.67:6789/0 conn(0x7f2f9015f0c0 legacy=0x7f2f9015f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f740039b0 0x7f2f74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f900d0430 0x7f2f74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9015bb30 con 0x7f2f74004d40 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f900b3ba0 con 0x7f2f740039b0 2024-05-30T17:01:38.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7400fdd0 con 0x7f2f900d0430 2024-05-30T17:01:38.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f94c00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:38.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8fe00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f740039b0 0x7f2f74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:38.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.579+0000 7f2f8f400640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f900d0430 0x7f2f74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:38.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.580+0000 7f2f94c00640 1 -- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f900d0430 msgr2=0x7f2f74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:38.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.580+0000 7f2f94c00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f900d0430 0x7f2f74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.580+0000 7f2f94c00640 1 -- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f740039b0 msgr2=0x7f2f74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:38.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.580+0000 7f2f94c00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f740039b0 0x7f2f74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.580+0000 7f2f94c00640 1 -- 172.21.15.67:0/408288569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f74003520 con 0x7f2f74004d40 2024-05-30T17:01:38.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.580+0000 7f2f8fe00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f740039b0 0x7f2f74003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:01:38.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.581+0000 7f2f94c00640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f740051c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800a7b0 tx=0x7f2f74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854dcb485784ef17 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:38.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.581+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8800a3d0 con 0x7f2f74004d40 2024-05-30T17:01:38.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.581+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8802f070 con 0x7f2f74004d40 2024-05-30T17:01:38.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.581+0000 7f2f8ea00640 1 -- 172.21.15.67:0/408288569 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8802bc20 con 0x7f2f74004d40 2024-05-30T17:01:38.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.582+0000 7f2f95600640 1 -- 172.21.15.67:0/408288569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 msgr2=0x7f2f740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:38.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.582+0000 7f2f95600640 1 --2- 172.21.15.67:0/408288569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f740051c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800a7b0 tx=0x7f2f74005830 comp rx=0 tx=0).stop 2024-05-30T17:01:38.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.582+0000 7f2f94c00640 1 -- 172.21.15.67:0/408288569 reap_dead start 2024-05-30T17:01:38.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.582+0000 7f2f95600640 1 -- 172.21.15.67:0/408288569 shutdown_connections 2024-05-30T17:01:38.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.582+0000 7f2f95600640 1 -- 172.21.15.67:0/408288569 >> 172.21.15.67:0/408288569 conn(0x7f2f9006bae0 msgr2=0x7f2f9006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:38.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.582+0000 7f2f95600640 1 -- 172.21.15.67:0/408288569 shutdown_connections 2024-05-30T17:01:38.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.582+0000 7f2f95600640 1 -- 172.21.15.67:0/408288569 wait complete. 2024-05-30T17:01:38.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.584+0000 7f2f95600640 1 Processor -- start 2024-05-30T17:01:38.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.584+0000 7f2f95600640 1 -- start start 2024-05-30T17:01:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.585+0000 7f2f95600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f90071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.585+0000 7f2f95600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f90073890 0x7f2f90150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.585+0000 7f2f95600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f9015c890 0x7f2f9015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.585+0000 7f2f95600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9015c670 con 0x7f2f74004d40 2024-05-30T17:01:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.585+0000 7f2f95600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9015c390 con 0x7f2f90073890 2024-05-30T17:01:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.585+0000 7f2f95600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9006f630 con 0x7f2f9015c890 2024-05-30T17:01:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f90071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:38.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f90071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55440/0 (socket says 172.21.15.67:55440) 2024-05-30T17:01:38.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8fe00640 1 -- 172.21.15.67:0/2516294292 learned_addr learned my addr 172.21.15.67:0/2516294292 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8fe00640 1 -- 172.21.15.67:0/2516294292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f9015c890 msgr2=0x7f2f9015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8fe00640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f9015c890 0x7f2f9015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8f400640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f90073890 0x7f2f90150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8fe00640 1 -- 172.21.15.67:0/2516294292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f90073890 msgr2=0x7f2f90150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8fe00640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f90073890 0x7f2f90150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8fe00640 1 -- 172.21.15.67:0/2516294292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f90072020 con 0x7f2f74004d40 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.586+0000 7f2f8fe00640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f90071a40 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2f80011650 tx=0x7f2f80011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.587+0000 7f2f87e00640 1 -- 172.21.15.67:0/2516294292 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000f9c0 con 0x7f2f74004d40 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.587+0000 7f2f87e00640 1 -- 172.21.15.67:0/2516294292 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8000fb60 con 0x7f2f74004d40 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.587+0000 7f2f87e00640 1 -- 172.21.15.67:0/2516294292 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80015460 con 0x7f2f74004d40 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.587+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f90072300 con 0x7f2f74004d40 2024-05-30T17:01:38.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.587+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f900728c0 con 0x7f2f74004d40 2024-05-30T17:01:38.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.588+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f900681e0 con 0x7f2f74004d40 2024-05-30T17:01:38.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.589+0000 7f2f87e00640 1 -- 172.21.15.67:0/2516294292 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 36) ==== 92978+0+0 (secure 0 0 0) 0x7f2f80015600 con 0x7f2f74004d40 2024-05-30T17:01:38.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.590+0000 7f2f87e00640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f7006ded0 0x7f2f70070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:38.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.590+0000 7f2f87e00640 1 -- 172.21.15.67:0/2516294292 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f2f8008b600 con 0x7f2f74004d40 2024-05-30T17:01:38.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.590+0000 7f2f8f400640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f7006ded0 0x7f2f70070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:38.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.591+0000 7f2f8f400640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f7006ded0 0x7f2f70070390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2f9009a360 tx=0x7f2f7c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:38.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.595+0000 7f2f87e00640 1 -- 172.21.15.67:0/2516294292 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f900681e0 con 0x7f2f74004d40 2024-05-30T17:01:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.835+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f2f900689e0 con 0x7f2f7006ded0 2024-05-30T17:01:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.868+0000 7f2f87e00640 1 -- 172.21.15.67:0/2516294292 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2f900689e0 con 0x7f2f7006ded0 2024-05-30T17:01:38.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.871+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f7006ded0 msgr2=0x7f2f70070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:38.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.871+0000 7f2f95600640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f7006ded0 0x7f2f70070390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2f9009a360 tx=0x7f2f7c008040 comp rx=0 tx=0).stop 2024-05-30T17:01:38.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.871+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 msgr2=0x7f2f90071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:38.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.871+0000 7f2f95600640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f90071a40 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2f80011650 tx=0x7f2f80011b10 comp rx=0 tx=0).stop 2024-05-30T17:01:38.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.872+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 shutdown_connections 2024-05-30T17:01:38.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.872+0000 7f2f95600640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f7006ded0 0x7f2f70070390 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.872+0000 7f2f95600640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f9015c890 0x7f2f9015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.872+0000 7f2f95600640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f90073890 0x7f2f90150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.872+0000 7f2f95600640 1 --2- 172.21.15.67:0/2516294292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f74004d40 0x7f2f90071a40 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:38.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.872+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 >> 172.21.15.67:0/2516294292 conn(0x7f2f9006bae0 msgr2=0x7f2f900b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:38.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.872+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 shutdown_connections 2024-05-30T17:01:38.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:38.872+0000 7f2f95600640 1 -- 172.21.15.67:0/2516294292 wait complete. 2024-05-30T17:01:38.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T17:01:39.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.094+0000 7ff973e00640 1 Processor -- start 2024-05-30T17:01:39.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.094+0000 7ff973e00640 1 -- start start 2024-05-30T17:01:39.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.095+0000 7ff973e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff974153510 0x7ff9740b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:39.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.095+0000 7ff973e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 0x7ff9740ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:39.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.095+0000 7ff973e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97415d900 con 0x7ff9740ccb90 2024-05-30T17:01:39.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.095+0000 7ff973e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff97415d6e0 con 0x7ff974153510 2024-05-30T17:01:39.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.095+0000 7ff973e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9741538f0 con 0x7ff97415d270 2024-05-30T17:01:39.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.096+0000 7ff972a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff974153510 0x7ff9740b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:39.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.096+0000 7ff972a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff974153510 0x7ff9740b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55448/0 (socket says 172.21.15.67:55448) 2024-05-30T17:01:39.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.096+0000 7ff972a00640 1 -- 172.21.15.67:0/1659784592 learned_addr learned my addr 172.21.15.67:0/1659784592 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:39.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.096+0000 7ff973400640 1 --2- 172.21.15.67:0/1659784592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 0x7ff9740ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:39.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.096+0000 7ff972a00640 1 -- 172.21.15.67:0/1659784592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 msgr2=0x7ff9740ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:39.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.096+0000 7ff972a00640 1 --2- 172.21.15.67:0/1659784592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 0x7ff9740ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.096+0000 7ff972a00640 1 -- 172.21.15.67:0/1659784592 >> v1:172.21.15.169:6789/0 conn(0x7ff9740ccb90 legacy=0x7ff97415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:39.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.096+0000 7ff972a00640 1 -- 172.21.15.67:0/1659784592 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9740b3f90 con 0x7ff974153510 2024-05-30T17:01:39.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.097+0000 7ff972a00640 1 --2- 172.21.15.67:0/1659784592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 0x7ff9740b4d50 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff964009870 tx=0x7ff96402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490564525de688d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:39.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.098+0000 7ff972000640 1 -- 172.21.15.67:0/1659784592 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964037070 con 0x7ff974153510 2024-05-30T17:01:39.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.098+0000 7ff972000640 1 -- 172.21.15.67:0/1659784592 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff964032440 con 0x7ff974153510 2024-05-30T17:01:39.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.098+0000 7ff972000640 1 -- 172.21.15.67:0/1659784592 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964032810 con 0x7ff974153510 2024-05-30T17:01:39.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.099+0000 7ff973e00640 1 -- 172.21.15.67:0/1659784592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 msgr2=0x7ff9740b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:39.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.099+0000 7ff973e00640 1 --2- 172.21.15.67:0/1659784592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 0x7ff9740b4d50 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff964009870 tx=0x7ff96402a180 comp rx=0 tx=0).stop 2024-05-30T17:01:39.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.099+0000 7ff973e00640 1 -- 172.21.15.67:0/1659784592 shutdown_connections 2024-05-30T17:01:39.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.099+0000 7ff973e00640 1 --2- 172.21.15.67:0/1659784592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 0x7ff9740ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.099+0000 7ff973e00640 1 --2- 172.21.15.67:0/1659784592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 0x7ff9740b4d50 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.099+0000 7ff973e00640 1 -- 172.21.15.67:0/1659784592 >> 172.21.15.67:0/1659784592 conn(0x7ff97406b900 msgr2=0x7ff97406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:39.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.099+0000 7ff973e00640 1 -- 172.21.15.67:0/1659784592 shutdown_connections 2024-05-30T17:01:39.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.099+0000 7ff973e00640 1 -- 172.21.15.67:0/1659784592 wait complete. 2024-05-30T17:01:39.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.101+0000 7ff973e00640 1 Processor -- start 2024-05-30T17:01:39.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.101+0000 7ff973e00640 1 -- start start 2024-05-30T17:01:39.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.102+0000 7ff973e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9740ccb90 0x7ff974099a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:39.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.102+0000 7ff973e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 0x7ff97415db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:39.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.102+0000 7ff973e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 0x7ff974150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:39.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.102+0000 7ff973e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740b48c0 con 0x7ff974153510 2024-05-30T17:01:39.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.102+0000 7ff973e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740b4550 con 0x7ff9740ccb90 2024-05-30T17:01:39.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.102+0000 7ff973e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff97406e940 con 0x7ff97415d270 2024-05-30T17:01:39.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff96a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 0x7ff97415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:39.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff96a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 0x7ff97415db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55458/0 (socket says 172.21.15.67:55458) 2024-05-30T17:01:39.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff96a000640 1 -- 172.21.15.67:0/216580662 learned_addr learned my addr 172.21.15.67:0/216580662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:39.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff972a00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9740ccb90 0x7ff974099a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:39.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff973400640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 0x7ff974150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:39.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff96a000640 1 -- 172.21.15.67:0/216580662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 msgr2=0x7ff974150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:39.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff96a000640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 0x7ff974150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff96a000640 1 -- 172.21.15.67:0/216580662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9740ccb90 msgr2=0x7ff974099a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:39.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff96a000640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9740ccb90 0x7ff974099a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff96a000640 1 -- 172.21.15.67:0/216580662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97415e350 con 0x7ff974153510 2024-05-30T17:01:39.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.103+0000 7ff972a00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9740ccb90 0x7ff974099a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:01:39.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.104+0000 7ff96a000640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 0x7ff97415db00 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00c430 tx=0x7ff95c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:39.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.104+0000 7ff96b400640 1 -- 172.21.15.67:0/216580662 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c011790 con 0x7ff974153510 2024-05-30T17:01:39.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.104+0000 7ff96b400640 1 -- 172.21.15.67:0/216580662 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c017070 con 0x7ff974153510 2024-05-30T17:01:39.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.104+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97415e630 con 0x7ff974153510 2024-05-30T17:01:39.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.104+0000 7ff96b400640 1 -- 172.21.15.67:0/216580662 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c011b70 con 0x7ff974153510 2024-05-30T17:01:39.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.104+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97415e040 con 0x7ff974153510 2024-05-30T17:01:39.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.105+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9740689e0 con 0x7ff974153510 2024-05-30T17:01:39.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.106+0000 7ff96b400640 1 -- 172.21.15.67:0/216580662 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 36) ==== 92978+0+0 (secure 0 0 0) 0x7ff95c011d10 con 0x7ff974153510 2024-05-30T17:01:39.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.107+0000 7ff96b400640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff95006de50 0x7ff950070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:39.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.107+0000 7ff96b400640 1 -- 172.21.15.67:0/216580662 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7ff95c08ab00 con 0x7ff974153510 2024-05-30T17:01:39.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.108+0000 7ff972a00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff95006de50 0x7ff950070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:39.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.108+0000 7ff972a00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff95006de50 0x7ff950070310 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff964034000 tx=0x7ff96402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:39.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.112+0000 7ff96b400640 1 -- 172.21.15.67:0/216580662 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff95c05a580 con 0x7ff974153510 2024-05-30T17:01:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.351+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff97406df90 con 0x7ff95006de50 2024-05-30T17:01:39.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.363+0000 7ff96b400640 1 -- 172.21.15.67:0/216580662 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7ff97406df90 con 0x7ff95006de50 2024-05-30T17:01:39.363 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f 2024-05-30T17:01:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.365+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff95006de50 msgr2=0x7ff950070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.365+0000 7ff973e00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff95006de50 0x7ff950070310 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff964034000 tx=0x7ff96402e000 comp rx=0 tx=0).stop 2024-05-30T17:01:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.365+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 msgr2=0x7ff97415db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.365+0000 7ff973e00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 0x7ff97415db00 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00c430 tx=0x7ff95c00c8f0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.366+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 shutdown_connections 2024-05-30T17:01:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.366+0000 7ff973e00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff95006de50 0x7ff950070310 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.366+0000 7ff973e00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97415d270 0x7ff974150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.366+0000 7ff973e00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff974153510 0x7ff97415db00 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.366+0000 7ff973e00640 1 --2- 172.21.15.67:0/216580662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9740ccb90 0x7ff974099a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.366+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 >> 172.21.15.67:0/216580662 conn(0x7ff97406b900 msgr2=0x7ff97406f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.366+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 shutdown_connections 2024-05-30T17:01:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:39.366+0000 7ff973e00640 1 -- 172.21.15.67:0/216580662 wait complete. 2024-05-30T17:01:39.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:01:39.386 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f 2024-05-30T17:01:39.386 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T17:01:39.447 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f 2024-05-30T17:01:39.447 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3138256896" 2024-05-30T17:01:39.447 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:01:39.448 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-30T17:01:41.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:01:41.449 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f 2024-05-30T17:01:41.449 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T17:01:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/0a046c17-930e-4968-b569-5934a309f88f 2024-05-30T17:01:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3221225472" 2024-05-30T17:01:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:01:41.517 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-30T17:01:41.517 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-30T17:01:41.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.728+0000 7f6ef7200640 1 Processor -- start 2024-05-30T17:01:41.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.728+0000 7f6ef7200640 1 -- start start 2024-05-30T17:01:41.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.730+0000 7f6ef7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6ef8153510 0x7f6ef80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:41.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.730+0000 7f6ef7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef815d270 0x7f6ef80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.730+0000 7f6ef7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef815d900 con 0x7f6ef80ccb90 2024-05-30T17:01:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.730+0000 7f6ef7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef815d6e0 con 0x7f6ef8153510 2024-05-30T17:01:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.730+0000 7f6ef7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef81538f0 con 0x7f6ef815d270 2024-05-30T17:01:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.730+0000 7f6eefe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6ef80ccb90 0x7f6ef815cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60566/0 (socket says 172.21.15.67:60566) 2024-05-30T17:01:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.730+0000 7f6eefe00640 1 -- 172.21.15.67:0/2215560565 learned_addr learned my addr 172.21.15.67:0/2215560565 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:01:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5e00640 1 --2- 172.21.15.67:0/2215560565 >> v2:172.21.15.169:3300/0 conn(0x7f6ef8153510 0x7f6ef80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef6800640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef815d270 0x7f6ef80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504844842 0 0) 0x7f6ef815d900 con 0x7f6ef80ccb90 2024-05-30T17:01:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6edc003520 con 0x7f6ef80ccb90 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 852612315 0 0) 0x7f6edc003520 con 0x7f6ef80ccb90 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef815d270 msgr2=0x7f6ef80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef815d270 0x7f6ef80ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 >> v2:172.21.15.169:3300/0 conn(0x7f6ef8153510 msgr2=0x7f6ef80b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 --2- 172.21.15.67:0/2215560565 >> v2:172.21.15.169:3300/0 conn(0x7f6ef8153510 0x7f6ef80b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef80b3f90 con 0x7f6ef80ccb90 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f6ee0001670 con 0x7f6ef80ccb90 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 >> v1:172.21.15.67:6789/0 conn(0x7f6ef80ccb90 legacy=0x7f6ef815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6edc0039b0 0x7f6edc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc004800 0x7f6edc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef815d270 0x7f6edc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef815d6e0 con 0x7f6edc0039b0 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc00b9c0 con 0x7f6edc004800 2024-05-30T17:01:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.731+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6edc00bb90 con 0x7f6ef815d270 2024-05-30T17:01:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.732+0000 7f6ef5e00640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6edc0039b0 0x7f6edc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.732+0000 7f6ef5e00640 1 -- 172.21.15.67:0/2215560565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef815d270 msgr2=0x7f6edc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.732+0000 7f6ef5e00640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef815d270 0x7f6edc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.732+0000 7f6ef5e00640 1 -- 172.21.15.67:0/2215560565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc004800 msgr2=0x7f6edc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.732+0000 7f6ef5e00640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc004800 0x7f6edc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.732+0000 7f6ef5e00640 1 -- 172.21.15.67:0/2215560565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc003520 con 0x7f6edc0039b0 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.733+0000 7f6ef5e00640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6edc0039b0 0x7f6edc003dd0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009af0 tx=0x7f6edc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efdf5d04a341765 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.733+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8036070 con 0x7f6edc0039b0 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.734+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee8031400 con 0x7f6edc0039b0 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.734+0000 7f6ef5400640 1 -- 172.21.15.67:0/2215560565 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8009690 con 0x7f6edc0039b0 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.734+0000 7f6ef7200640 1 -- 172.21.15.67:0/2215560565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6edc0039b0 msgr2=0x7f6edc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.734+0000 7f6ef7200640 1 --2- 172.21.15.67:0/2215560565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6edc0039b0 0x7f6edc003dd0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009af0 tx=0x7f6edc005830 comp rx=0 tx=0).stop 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.734+0000 7f6ef6800640 1 -- 172.21.15.67:0/2215560565 reap_dead start 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.734+0000 7f6ef7200640 1 -- 172.21.15.67:0/2215560565 shutdown_connections 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.734+0000 7f6ef7200640 1 -- 172.21.15.67:0/2215560565 >> 172.21.15.67:0/2215560565 conn(0x7f6ef806b900 msgr2=0x7f6ef806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.734+0000 7f6ef7200640 1 -- 172.21.15.67:0/2215560565 shutdown_connections 2024-05-30T17:01:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.734+0000 7f6ef7200640 1 -- 172.21.15.67:0/2215560565 wait complete. 2024-05-30T17:01:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.736+0000 7f6ef7200640 1 Processor -- start 2024-05-30T17:01:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.736+0000 7f6ef7200640 1 -- start start 2024-05-30T17:01:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.737+0000 7f6ef7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc0039b0 0x7f6ef814cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.737+0000 7f6ef7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef814c790 0x7f6ef81444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.737+0000 7f6ef7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef80af180 0x7f6ef806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6ef7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef80b4550 con 0x7f6ef814c790 2024-05-30T17:01:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6ef7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef80b48c0 con 0x7f6edc0039b0 2024-05-30T17:01:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6ef7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef80ccff0 con 0x7f6ef80af180 2024-05-30T17:01:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6eefe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef814c790 0x7f6ef81444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6eefe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef814c790 0x7f6ef81444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52880/0 (socket says 172.21.15.67:52880) 2024-05-30T17:01:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6eefe00640 1 -- 172.21.15.67:0/2679579241 learned_addr learned my addr 172.21.15.67:0/2679579241 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6ef5e00640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc0039b0 0x7f6ef814cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6ef6800640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef80af180 0x7f6ef806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6eefe00640 1 -- 172.21.15.67:0/2679579241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef80af180 msgr2=0x7f6ef806ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6eefe00640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef80af180 0x7f6ef806ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6eefe00640 1 -- 172.21.15.67:0/2679579241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc0039b0 msgr2=0x7f6ef814cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6eefe00640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc0039b0 0x7f6ef814cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.738+0000 7f6eefe00640 1 -- 172.21.15.67:0/2679579241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef81f4070 con 0x7f6ef814c790 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.739+0000 7f6eefe00640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef814c790 0x7f6ef81444a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c7a0 tx=0x7f6ee000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.739+0000 7f6eee000640 1 -- 172.21.15.67:0/2679579241 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0018070 con 0x7f6ef814c790 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.739+0000 7f6eee000640 1 -- 172.21.15.67:0/2679579241 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee00074e0 con 0x7f6ef814c790 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.739+0000 7f6eee000640 1 -- 172.21.15.67:0/2679579241 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0004a20 con 0x7f6ef814c790 2024-05-30T17:01:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.739+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef80ae0e0 con 0x7f6ef814c790 2024-05-30T17:01:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.739+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef80ae6a0 con 0x7f6ef814c790 2024-05-30T17:01:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.740+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef80689e0 con 0x7f6ef814c790 2024-05-30T17:01:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.741+0000 7f6eee000640 1 -- 172.21.15.67:0/2679579241 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 36) ==== 92978+0+0 (secure 0 0 0) 0x7f6ee001b070 con 0x7f6ef814c790 2024-05-30T17:01:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.742+0000 7f6eee000640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ed406de50 0x7f6ed4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.742+0000 7f6eee000640 1 -- 172.21.15.67:0/2679579241 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f6ee008a310 con 0x7f6ef814c790 2024-05-30T17:01:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.743+0000 7f6ef5e00640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ed406de50 0x7f6ed4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.743+0000 7f6ef5e00640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ed406de50 0x7f6ed4070310 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8033000 tx=0x7f6ee8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.747+0000 7f6eee000640 1 -- 172.21.15.67:0/2679579241 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ee0059d00 con 0x7f6ef814c790 2024-05-30T17:01:41.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:41.988+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f6ef80b3c40 con 0x7f6ed406de50 2024-05-30T17:01:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.134+0000 7f6eee000640 1 -- 172.21.15.67:0/2679579241 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6ef80b3c40 con 0x7f6ed406de50 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ed406de50 msgr2=0x7f6ed4070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ed406de50 0x7f6ed4070310 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8033000 tx=0x7f6ee8008030 comp rx=0 tx=0).stop 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef814c790 msgr2=0x7f6ef81444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef814c790 0x7f6ef81444a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c7a0 tx=0x7f6ee000cc60 comp rx=0 tx=0).stop 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 shutdown_connections 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6ed406de50 0x7f6ed4070310 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ef80af180 0x7f6ef806ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef814c790 0x7f6ef81444a0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 --2- 172.21.15.67:0/2679579241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc0039b0 0x7f6ef814cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 >> 172.21.15.67:0/2679579241 conn(0x7f6ef806b900 msgr2=0x7f6ef8150ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 shutdown_connections 2024-05-30T17:01:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:42.138+0000 7f6ef7200640 1 -- 172.21.15.67:0/2679579241 wait complete. 2024-05-30T17:01:42.140 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.140+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:42.140 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.140+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:42.140 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.140+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:42.141 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.140+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:42.141 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.140+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:42.163 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.163+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:42.163 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.163+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:42.164 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.163+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:42.164 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.163+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:42.164 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:01:42.163+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:01:43.160 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-05-30T17:01:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.380+0000 7f0a5a000640 1 Processor -- start 2024-05-30T17:01:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.380+0000 7f0a5a000640 1 -- start start 2024-05-30T17:01:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.382+0000 7f0a5a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0a540cd2a0 0x7f0a540b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.382+0000 7f0a5a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a5415cb60 0x7f0a5415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.382+0000 7f0a5a000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a5415d900 con 0x7f0a54153510 2024-05-30T17:01:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.382+0000 7f0a5a000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f0a541538f0 con 0x7f0a540cd2a0 2024-05-30T17:01:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.382+0000 7f0a5a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0a540b48c0 con 0x7f0a5415cb60 2024-05-30T17:01:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.382+0000 7f0a58c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0a54153510 0x7f0a540ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60572/0 (socket says 172.21.15.67:60572) 2024-05-30T17:01:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.382+0000 7f0a58c00640 1 -- 172.21.15.67:0/1678832731 learned_addr learned my addr 172.21.15.67:0/1678832731 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:01:43.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.383+0000 7f0a53e00640 1 --2- 172.21.15.67:0/1678832731 >> v2:172.21.15.67:3300/0 conn(0x7f0a540cd2a0 0x7f0a540b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.383+0000 7f0a59600640 1 --2- 172.21.15.67:0/1678832731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a5415cb60 0x7f0a5415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.383+0000 7f0a53400640 1 -- 172.21.15.67:0/1678832731 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 351746208 0 0) 0x7f0a5415d900 con 0x7f0a54153510 2024-05-30T17:01:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.383+0000 7f0a53400640 1 -- 172.21.15.67:0/1678832731 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a3c003520 con 0x7f0a54153510 2024-05-30T17:01:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.383+0000 7f0a53e00640 1 -- 172.21.15.67:0/1678832731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a5415cb60 msgr2=0x7f0a5415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.383+0000 7f0a53e00640 1 --2- 172.21.15.67:0/1678832731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a5415cb60 0x7f0a5415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.383+0000 7f0a53e00640 1 -- 172.21.15.67:0/1678832731 >> v1:172.21.15.67:6789/0 conn(0x7f0a54153510 legacy=0x7f0a540ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.383+0000 7f0a53e00640 1 -- 172.21.15.67:0/1678832731 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a540b3f90 con 0x7f0a540cd2a0 2024-05-30T17:01:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.384+0000 7f0a53e00640 1 --2- 172.21.15.67:0/1678832731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a540cd2a0 0x7f0a540b4d50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0a400097a0 tx=0x7f0a4002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cfada0b1e45e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.384+0000 7f0a53400640 1 -- 172.21.15.67:0/1678832731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40037070 con 0x7f0a540cd2a0 2024-05-30T17:01:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.384+0000 7f0a53400640 1 -- 172.21.15.67:0/1678832731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4002ae50 con 0x7f0a540cd2a0 2024-05-30T17:01:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.385+0000 7f0a53400640 1 -- 172.21.15.67:0/1678832731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40032660 con 0x7f0a540cd2a0 2024-05-30T17:01:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.385+0000 7f0a5a000640 1 -- 172.21.15.67:0/1678832731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a540cd2a0 msgr2=0x7f0a540b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.385+0000 7f0a5a000640 1 --2- 172.21.15.67:0/1678832731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a540cd2a0 0x7f0a540b4d50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0a400097a0 tx=0x7f0a4002a100 comp rx=0 tx=0).stop 2024-05-30T17:01:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.385+0000 7f0a5a000640 1 -- 172.21.15.67:0/1678832731 shutdown_connections 2024-05-30T17:01:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.385+0000 7f0a5a000640 1 --2- 172.21.15.67:0/1678832731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a5415cb60 0x7f0a5415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.385+0000 7f0a5a000640 1 --2- 172.21.15.67:0/1678832731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a540cd2a0 0x7f0a540b4d50 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.385+0000 7f0a5a000640 1 -- 172.21.15.67:0/1678832731 >> 172.21.15.67:0/1678832731 conn(0x7f0a5406b900 msgr2=0x7f0a5406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.385+0000 7f0a5a000640 1 -- 172.21.15.67:0/1678832731 shutdown_connections 2024-05-30T17:01:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.385+0000 7f0a5a000640 1 -- 172.21.15.67:0/1678832731 wait complete. 2024-05-30T17:01:43.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.387+0000 7f0a5a000640 1 Processor -- start 2024-05-30T17:01:43.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.387+0000 7f0a5a000640 1 -- start start 2024-05-30T17:01:43.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.388+0000 7f0a5a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a540cd2a0 0x7f0a541f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:43.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.388+0000 7f0a5a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a54153510 0x7f0a54153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:43.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.388+0000 7f0a5a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a5415cb60 0x7f0a54072d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.388+0000 7f0a5a000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5415d4f0 con 0x7f0a54153510 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.388+0000 7f0a5a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0a540b4550 con 0x7f0a5415cb60 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.388+0000 7f0a5a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5415d670 con 0x7f0a540cd2a0 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.389+0000 7f0a53e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a54153510 0x7f0a54153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.389+0000 7f0a53e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a54153510 0x7f0a54153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52952/0 (socket says 172.21.15.67:52952) 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.389+0000 7f0a53e00640 1 -- 172.21.15.67:0/134796113 learned_addr learned my addr 172.21.15.67:0/134796113 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.389+0000 7f0a53e00640 1 -- 172.21.15.67:0/134796113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a540cd2a0 msgr2=0x7f0a541f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.389+0000 7f0a53e00640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a540cd2a0 0x7f0a541f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.389+0000 7f0a53e00640 1 -- 172.21.15.67:0/134796113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a5415cb60 msgr2=0x7f0a54072d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.389+0000 7f0a53e00640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a5415cb60 0x7f0a54072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.389+0000 7f0a53e00640 1 -- 172.21.15.67:0/134796113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a541f24c0 con 0x7f0a54153510 2024-05-30T17:01:43.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.389+0000 7f0a53e00640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a54153510 0x7f0a54153b60 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f0a40035040 tx=0x7f0a400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:43.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.390+0000 7f0a50c00640 1 -- 172.21.15.67:0/134796113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40041070 con 0x7f0a54153510 2024-05-30T17:01:43.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.390+0000 7f0a50c00640 1 -- 172.21.15.67:0/134796113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4003c460 con 0x7f0a54153510 2024-05-30T17:01:43.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.390+0000 7f0a50c00640 1 -- 172.21.15.67:0/134796113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40037040 con 0x7f0a54153510 2024-05-30T17:01:43.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.390+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a541f27a0 con 0x7f0a54153510 2024-05-30T17:01:43.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.390+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a541f2d60 con 0x7f0a54153510 2024-05-30T17:01:43.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.391+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a540689e0 con 0x7f0a54153510 2024-05-30T17:01:43.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.392+0000 7f0a50c00640 1 -- 172.21.15.67:0/134796113 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7f0a4003a070 con 0x7f0a54153510 2024-05-30T17:01:43.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.393+0000 7f0a50c00640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a3406df70 0x7f0a34070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:43.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.393+0000 7f0a50c00640 1 -- 172.21.15.67:0/134796113 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f0a400aabc0 con 0x7f0a54153510 2024-05-30T17:01:43.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.393+0000 7f0a58c00640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a3406df70 0x7f0a34070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:43.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.394+0000 7f0a58c00640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a3406df70 0x7f0a34070430 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000f60 tx=0x7f0a480078f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:43.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.398+0000 7f0a50c00640 1 -- 172.21.15.67:0/134796113 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a4007aa50 con 0x7f0a54153510 2024-05-30T17:01:43.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:43.645+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f0a540ccd40 con 0x7f0a3406df70 2024-05-30T17:01:45.234 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:01:45.243 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:01:46.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.606+0000 7f0a50c00640 1 -- 172.21.15.67:0/134796113 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0a540ccd40 con 0x7f0a3406df70 2024-05-30T17:01:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a3406df70 msgr2=0x7f0a34070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a3406df70 0x7f0a34070430 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000f60 tx=0x7f0a480078f0 comp rx=0 tx=0).stop 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a54153510 msgr2=0x7f0a54153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a54153510 0x7f0a54153b60 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f0a40035040 tx=0x7f0a400046c0 comp rx=0 tx=0).stop 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 shutdown_connections 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a3406df70 0x7f0a34070430 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a5415cb60 0x7f0a54072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a54153510 0x7f0a54153b60 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 --2- 172.21.15.67:0/134796113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a540cd2a0 0x7f0a541f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 >> 172.21.15.67:0/134796113 conn(0x7f0a5406b900 msgr2=0x7f0a5406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 shutdown_connections 2024-05-30T17:01:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:46.610+0000 7f0a5a000640 1 -- 172.21.15.67:0/134796113 wait complete. 2024-05-30T17:01:51.632 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:01:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.848+0000 7ff4b0a00640 1 Processor -- start 2024-05-30T17:01:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.848+0000 7ff4b0a00640 1 -- start start 2024-05-30T17:01:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.850+0000 7ff4b0a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff4ac0d4c40 0x7ff4ac0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.850+0000 7ff4b0a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ac0d3060 con 0x7ff4ac0d1ca0 2024-05-30T17:01:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.850+0000 7ff4b0a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ac0d3340 con 0x7ff4ac15e690 2024-05-30T17:01:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.850+0000 7ff4b0a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ac0d3620 con 0x7ff4ac0d4c40 2024-05-30T17:01:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.850+0000 7ff4ab200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff4ac0d1ca0 0x7ff4ac0d2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42698/0 (socket says 172.21.15.67:42698) 2024-05-30T17:01:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.850+0000 7ff4ab200640 1 -- 172.21.15.67:0/2558997609 learned_addr learned my addr 172.21.15.67:0/2558997609 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:01:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9e00640 1 --2- 172.21.15.67:0/2558997609 >> v2:172.21.15.169:3300/0 conn(0x7ff4ac0d4c40 0x7ff4ac0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2975005805 0 0) 0x7ff4ac0d3060 con 0x7ff4ac0d1ca0 2024-05-30T17:01:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff490003520 con 0x7ff4ac0d1ca0 2024-05-30T17:01:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2790300109 0 0) 0x7ff490003520 con 0x7ff4ac0d1ca0 2024-05-30T17:01:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 >> v2:172.21.15.169:3300/0 conn(0x7ff4ac0d4c40 msgr2=0x7ff4ac0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 --2- 172.21.15.67:0/2558997609 >> v2:172.21.15.169:3300/0 conn(0x7ff4ac0d4c40 0x7ff4ac0d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 >> v1:172.21.15.169:6789/0 conn(0x7ff4ac15e690 legacy=0x7ff4ac0d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac0cf770 con 0x7ff4ac0d1ca0 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff4a4001670 con 0x7ff4ac0d1ca0 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 >> v1:172.21.15.67:6789/0 conn(0x7ff4ac0d1ca0 legacy=0x7ff4ac0d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490003a00 0x7ff490003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff490004800 0x7ff490004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.851+0000 7ff4a9400640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900051c0 0x7ff49000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac0d3620 con 0x7ff490004800 2024-05-30T17:01:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff49000c090 con 0x7ff490003a00 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff49000c290 con 0x7ff4900051c0 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4a9e00640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff490004800 0x7ff490004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4aa800640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490003a00 0x7ff490003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4a9e00640 1 -- 172.21.15.67:0/2558997609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900051c0 msgr2=0x7ff49000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4a9e00640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900051c0 0x7ff49000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4a9e00640 1 -- 172.21.15.67:0/2558997609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490003a00 msgr2=0x7ff490003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4a9e00640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490003a00 0x7ff490003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4a9e00640 1 -- 172.21.15.67:0/2558997609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff490003520 con 0x7ff490004800 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4aa800640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490003a00 0x7ff490003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:01:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.852+0000 7ff4ab200640 1 -- 172.21.15.67:0/2558997609 reap_dead start 2024-05-30T17:01:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.853+0000 7ff4a9e00640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff490004800 0x7ff490004c80 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff49800e170 tx=0x7ff490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2ccb10badf8737 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498009c60 con 0x7ff490004800 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff498036070 con 0x7ff490004800 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4a9400640 1 -- 172.21.15.67:0/2558997609 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4980318a0 con 0x7ff490004800 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4b0a00640 1 -- 172.21.15.67:0/2558997609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff490004800 msgr2=0x7ff490004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4b0a00640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff490004800 0x7ff490004c80 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff49800e170 tx=0x7ff490005830 comp rx=0 tx=0).stop 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4b0a00640 1 -- 172.21.15.67:0/2558997609 shutdown_connections 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4b0a00640 1 --2- 172.21.15.67:0/2558997609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff490004800 0x7ff490004c80 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4b0a00640 1 -- 172.21.15.67:0/2558997609 >> 172.21.15.67:0/2558997609 conn(0x7ff4ac09d910 msgr2=0x7ff4ac08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4b0a00640 1 -- 172.21.15.67:0/2558997609 shutdown_connections 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.854+0000 7ff4b0a00640 1 -- 172.21.15.67:0/2558997609 wait complete. 2024-05-30T17:01:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.856+0000 7ff4b0a00640 1 Processor -- start 2024-05-30T17:01:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.856+0000 7ff4b0a00640 1 -- start start 2024-05-30T17:01:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.857+0000 7ff4b0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490004800 0x7ff4ac1f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.857+0000 7ff4b0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4ac0d2150 0x7ff4ac0d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.857+0000 7ff4b0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4ac1f73c0 0x7ff4ac1f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.857+0000 7ff4b0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac15ebf0 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.857+0000 7ff4b0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac15ea70 con 0x7ff490004800 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.857+0000 7ff4b0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac15ed70 con 0x7ff4ac0d2150 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4ab200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4ac1f73c0 0x7ff4ac1f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4ab200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4ac1f73c0 0x7ff4ac1f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54256/0 (socket says 172.21.15.67:54256) 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4ab200640 1 -- 172.21.15.67:0/3961371254 learned_addr learned my addr 172.21.15.67:0/3961371254 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4a9e00640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4ac0d2150 0x7ff4ac0d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4ab200640 1 -- 172.21.15.67:0/3961371254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4ac0d2150 msgr2=0x7ff4ac0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4ab200640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4ac0d2150 0x7ff4ac0d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4ab200640 1 -- 172.21.15.67:0/3961371254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490004800 msgr2=0x7ff4ac1f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4ab200640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490004800 0x7ff4ac1f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4ab200640 1 -- 172.21.15.67:0/3961371254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac1f7d80 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.858+0000 7ff4a9e00640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4ac0d2150 0x7ff4ac0d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:01:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.859+0000 7ff4ab200640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4ac1f73c0 0x7ff4ac1f7840 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0b7170 tx=0x7ff4a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.859+0000 7ff4a2a00640 1 -- 172.21.15.67:0/3961371254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40074e0 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.859+0000 7ff4a2a00640 1 -- 172.21.15.67:0/3961371254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a40047e0 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.859+0000 7ff4a2a00640 1 -- 172.21.15.67:0/3961371254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4004bd0 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.859+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac1f8010 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.859+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac0b8890 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.860+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac0689e0 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.862+0000 7ff4a2a00640 1 -- 172.21.15.67:0/3961371254 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7ff4a4013450 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.863+0000 7ff4a2a00640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff48c06df70 0x7ff48c070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.863+0000 7ff4a2a00640 1 -- 172.21.15.67:0/3961371254 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7ff4a4089fc0 con 0x7ff4ac1f73c0 2024-05-30T17:01:51.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.863+0000 7ff4aa800640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff48c06df70 0x7ff48c070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:51.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.864+0000 7ff4aa800640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff48c06df70 0x7ff48c070430 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac051690 tx=0x7ff49c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:51.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:51.868+0000 7ff4a2a00640 1 -- 172.21.15.67:0/3961371254 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4a4059e50 con 0x7ff4ac1f73c0 2024-05-30T17:01:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.153+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff4ac06cbc0 con 0x7ff4ac1f73c0 2024-05-30T17:01:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.154+0000 7ff4a2a00640 1 -- 172.21.15.67:0/3961371254 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2113 (secure 0 0 0) 0x7ff4a4059750 con 0x7ff4ac1f73c0 2024-05-30T17:01:52.156 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:01:52.156 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:01:52.156 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:01:52.156 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:01:52.156 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:01:52.156 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:01:52.156 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:01:52.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:52.156 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1105, 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:01:52.157 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 104, 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:52.158 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 798, 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 3222153432, 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 7440113664, 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 765653999616, 2024-05-30T17:01:52.159 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 511, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 18879554, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 8 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:01:52.160 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 7, 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7902 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:01:52.161 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 19, 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:01:42.499404+0000", 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "1c47c4a3-eeba-4cb3-b4b4-e5c080306dec": { 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (4s)\n [============================] ", 2024-05-30T17:01:52.162 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:01:52.163 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:01:52.163 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:52.163 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:01:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff48c06df70 msgr2=0x7ff48c070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff48c06df70 0x7ff48c070430 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac051690 tx=0x7ff49c009040 comp rx=0 tx=0).stop 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4ac1f73c0 msgr2=0x7ff4ac1f7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4ac1f73c0 0x7ff4ac1f7840 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0b7170 tx=0x7ff4a400c8b0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 shutdown_connections 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff48c06df70 0x7ff48c070430 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4ac1f73c0 0x7ff4ac1f7840 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4ac0d2150 0x7ff4ac0d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 --2- 172.21.15.67:0/3961371254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff490004800 0x7ff4ac1f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 >> 172.21.15.67:0/3961371254 conn(0x7ff4ac09d910 msgr2=0x7ff4ac0b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 shutdown_connections 2024-05-30T17:01:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.157+0000 7ff4b0a00640 1 -- 172.21.15.67:0/3961371254 wait complete. 2024-05-30T17:01:52.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:01:52.179 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/0a046c17-930e-4968-b569-5934a309f88f 2024-05-30T17:01:52.179 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T17:01:52.237 INFO:teuthology.orchestra.run.smithi067.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/0a046c17-930e-4968-b569-5934a309f88f': No such file or directory 2024-05-30T17:01:52.239 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-30T17:01:52.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T17:01:52.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.450+0000 7fa221200640 1 Processor -- start 2024-05-30T17:01:52.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.450+0000 7fa221200640 1 -- start start 2024-05-30T17:01:52.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.452+0000 7fa221200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa21c06ee40 0x7fa21c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.452+0000 7fa221200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06f2f0 0x7fa21c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.452+0000 7fa221200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa21c156f10 con 0x7fa21c156b30 2024-05-30T17:01:52.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.452+0000 7fa221200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa21c06dda0 con 0x7fa21c06ee40 2024-05-30T17:01:52.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.452+0000 7fa221200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa21c06e080 con 0x7fa21c06f2f0 2024-05-30T17:01:52.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.453+0000 7fa21b400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa21c156b30 0x7fa21c0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:33592/0 (socket says 172.21.15.67:33592) 2024-05-30T17:01:52.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.453+0000 7fa21b400640 1 -- 172.21.15.67:0/2642730925 learned_addr learned my addr 172.21.15.67:0/2642730925 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:01:52.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.453+0000 7fa21be00640 1 --2- 172.21.15.67:0/2642730925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06f2f0 0x7fa21c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:52.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.453+0000 7fa21a000640 1 -- 172.21.15.67:0/2642730925 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263792998 0 0) 0x7fa21c156f10 con 0x7fa21c156b30 2024-05-30T17:01:52.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.453+0000 7fa21a000640 1 -- 172.21.15.67:0/2642730925 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa200003520 con 0x7fa21c156b30 2024-05-30T17:01:52.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.454+0000 7fa21be00640 1 -- 172.21.15.67:0/2642730925 >> v2:172.21.15.169:3300/0 conn(0x7fa21c06ee40 msgr2=0x7fa21c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:52.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.454+0000 7fa21be00640 1 --2- 172.21.15.67:0/2642730925 >> v2:172.21.15.169:3300/0 conn(0x7fa21c06ee40 0x7fa21c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.454+0000 7fa21be00640 1 -- 172.21.15.67:0/2642730925 >> v1:172.21.15.169:6789/0 conn(0x7fa21c156b30 legacy=0x7fa21c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.454+0000 7fa21be00640 1 -- 172.21.15.67:0/2642730925 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c06e360 con 0x7fa21c06f2f0 2024-05-30T17:01:52.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.455+0000 7fa21be00640 1 --2- 172.21.15.67:0/2642730925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06f2f0 0x7fa21c0ad770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa214009d90 tx=0x7fa21402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25bf8930ec92652f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.456+0000 7fa21a000640 1 -- 172.21.15.67:0/2642730925 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214031600 con 0x7fa21c06f2f0 2024-05-30T17:01:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.456+0000 7fa21a000640 1 -- 172.21.15.67:0/2642730925 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2140317a0 con 0x7fa21c06f2f0 2024-05-30T17:01:52.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.456+0000 7fa21a000640 1 -- 172.21.15.67:0/2642730925 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214031b50 con 0x7fa21c06f2f0 2024-05-30T17:01:52.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.456+0000 7fa221200640 1 -- 172.21.15.67:0/2642730925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06f2f0 msgr2=0x7fa21c0ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.456+0000 7fa221200640 1 --2- 172.21.15.67:0/2642730925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06f2f0 0x7fa21c0ad770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa214009d90 tx=0x7fa21402a5d0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.457+0000 7fa221200640 1 -- 172.21.15.67:0/2642730925 shutdown_connections 2024-05-30T17:01:52.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.457+0000 7fa221200640 1 --2- 172.21.15.67:0/2642730925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06f2f0 0x7fa21c0ad770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.457+0000 7fa221200640 1 --2- 172.21.15.67:0/2642730925 >> v2:172.21.15.169:3300/0 conn(0x7fa21c06ee40 0x7fa21c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.457+0000 7fa221200640 1 -- 172.21.15.67:0/2642730925 >> 172.21.15.67:0/2642730925 conn(0x7fa21c0b1fc0 msgr2=0x7fa21c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:52.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.457+0000 7fa221200640 1 -- 172.21.15.67:0/2642730925 shutdown_connections 2024-05-30T17:01:52.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.457+0000 7fa221200640 1 -- 172.21.15.67:0/2642730925 wait complete. 2024-05-30T17:01:52.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.458+0000 7fa221200640 1 Processor -- start 2024-05-30T17:01:52.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.459+0000 7fa221200640 1 -- start start 2024-05-30T17:01:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa221200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06ee40 0x7fa21c15e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa221200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa21c06f2f0 0x7fa21c14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa221200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa21c156b30 0x7fa21c1550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa221200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c0b2db0 con 0x7fa21c06f2f0 2024-05-30T17:01:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa221200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c0b2c30 con 0x7fa21c156b30 2024-05-30T17:01:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa221200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa21c0b32c0 con 0x7fa21c06ee40 2024-05-30T17:01:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa21aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa21c06f2f0 0x7fa21c14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa21aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa21c06f2f0 0x7fa21c14e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54264/0 (socket says 172.21.15.67:54264) 2024-05-30T17:01:52.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa21aa00640 1 -- 172.21.15.67:0/3801402415 learned_addr learned my addr 172.21.15.67:0/3801402415 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:52.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa21b400640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06ee40 0x7fa21c15e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa21aa00640 1 -- 172.21.15.67:0/3801402415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06ee40 msgr2=0x7fa21c15e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa21aa00640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06ee40 0x7fa21c15e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa21aa00640 1 -- 172.21.15.67:0/3801402415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa21c156b30 msgr2=0x7fa21c1550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.460+0000 7fa21aa00640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa21c156b30 0x7fa21c1550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.461+0000 7fa21aa00640 1 -- 172.21.15.67:0/3801402415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c14e570 con 0x7fa21c06f2f0 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.461+0000 7fa21aa00640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa21c06f2f0 0x7fa21c14e030 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa20800b4a0 tx=0x7fa20800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.461+0000 7fa213400640 1 -- 172.21.15.67:0/3801402415 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208014070 con 0x7fa21c06f2f0 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.461+0000 7fa213400640 1 -- 172.21.15.67:0/3801402415 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa208010610 con 0x7fa21c06f2f0 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.461+0000 7fa213400640 1 -- 172.21.15.67:0/3801402415 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208010a00 con 0x7fa21c06f2f0 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.461+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21c14f630 con 0x7fa21c06f2f0 2024-05-30T17:01:52.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.461+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21c14fbf0 con 0x7fa21c06f2f0 2024-05-30T17:01:52.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.463+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21c0689e0 con 0x7fa21c06f2f0 2024-05-30T17:01:52.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.464+0000 7fa213400640 1 -- 172.21.15.67:0/3801402415 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7fa208017070 con 0x7fa21c06f2f0 2024-05-30T17:01:52.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.465+0000 7fa213400640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa1fc06dfc0 0x7fa1fc070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.465+0000 7fa213400640 1 -- 172.21.15.67:0/3801402415 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fa20808a2a0 con 0x7fa21c06f2f0 2024-05-30T17:01:52.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.465+0000 7fa21b400640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa1fc06dfc0 0x7fa1fc070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:52.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.466+0000 7fa21b400640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa1fc06dfc0 0x7fa1fc070480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005880 tx=0x7fa20c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:52.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.469+0000 7fa213400640 1 -- 172.21.15.67:0/3801402415 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa20805a1b0 con 0x7fa21c06f2f0 2024-05-30T17:01:52.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.699+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa21c14cc90 con 0x7fa1fc06dfc0 2024-05-30T17:01:52.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.701+0000 7fa213400640 1 -- 172.21.15.67:0/3801402415 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa21c14cc90 con 0x7fa1fc06dfc0 2024-05-30T17:01:52.703 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:01:52.703 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:01:52.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T17:01:52.703 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:52.703 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:01:52.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa1fc06dfc0 msgr2=0x7fa1fc070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa1fc06dfc0 0x7fa1fc070480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005880 tx=0x7fa20c009040 comp rx=0 tx=0).stop 2024-05-30T17:01:52.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa21c06f2f0 msgr2=0x7fa21c14e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa21c06f2f0 0x7fa21c14e030 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa20800b4a0 tx=0x7fa20800b960 comp rx=0 tx=0).stop 2024-05-30T17:01:52.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 shutdown_connections 2024-05-30T17:01:52.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa1fc06dfc0 0x7fa1fc070480 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa21c156b30 0x7fa21c1550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa21c06f2f0 0x7fa21c14e030 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 --2- 172.21.15.67:0/3801402415 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06ee40 0x7fa21c15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 >> 172.21.15.67:0/3801402415 conn(0x7fa21c0b1fc0 msgr2=0x7fa21c0733c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.705+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 shutdown_connections 2024-05-30T17:01:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.706+0000 7fa221200640 1 -- 172.21.15.67:0/3801402415 wait complete. 2024-05-30T17:01:52.726 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:01:52.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.936+0000 7f31e9400640 1 Processor -- start 2024-05-30T17:01:52.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.936+0000 7f31e9400640 1 -- start start 2024-05-30T17:01:52.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e9400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f31e415c6b0 0x7f31e40b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e9400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f31e40d0460 0x7f31e40d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e9400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e40cd1a0 con 0x7f31e40ccc30 2024-05-30T17:01:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e9400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f31e406e3a0 con 0x7f31e40d0460 2024-05-30T17:01:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e9400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f31e40d0df0 con 0x7f31e415c6b0 2024-05-30T17:01:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f31e40ccc30 0x7f31e406dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42706/0 (socket says 172.21.15.67:42706) 2024-05-30T17:01:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e3400640 1 -- 172.21.15.67:0/405650335 learned_addr learned my addr 172.21.15.67:0/405650335 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:01:52.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e8a00640 1 --2- 172.21.15.67:0/405650335 >> v2:172.21.15.67:3300/0 conn(0x7f31e40d0460 0x7f31e40d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:52.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e2a00640 1 -- 172.21.15.67:0/405650335 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1312746889 0 0) 0x7f31e40cd1a0 con 0x7f31e40ccc30 2024-05-30T17:01:52.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e2a00640 1 -- 172.21.15.67:0/405650335 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31c8003520 con 0x7f31e40ccc30 2024-05-30T17:01:52.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e3e00640 1 --2- 172.21.15.67:0/405650335 >> v2:172.21.15.169:3300/0 conn(0x7f31e415c6b0 0x7f31e40b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:52.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e8a00640 1 -- 172.21.15.67:0/405650335 >> v2:172.21.15.169:3300/0 conn(0x7f31e415c6b0 msgr2=0x7f31e40b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:52.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e8a00640 1 --2- 172.21.15.67:0/405650335 >> v2:172.21.15.169:3300/0 conn(0x7f31e415c6b0 0x7f31e40b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e8a00640 1 -- 172.21.15.67:0/405650335 >> v1:172.21.15.67:6789/0 conn(0x7f31e40ccc30 legacy=0x7f31e406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.938+0000 7f31e8a00640 1 -- 172.21.15.67:0/405650335 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40cd480 con 0x7f31e40d0460 2024-05-30T17:01:52.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.939+0000 7f31e8a00640 1 --2- 172.21.15.67:0/405650335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40d0460 0x7f31e40d08b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0097a0 tx=0x7f31dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1702856470fd55ce server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:52.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.939+0000 7f31e2a00640 1 -- 172.21.15.67:0/405650335 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc037070 con 0x7f31e40d0460 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.939+0000 7f31e2a00640 1 -- 172.21.15.67:0/405650335 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31dc032440 con 0x7f31e40d0460 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.939+0000 7f31e2a00640 1 -- 172.21.15.67:0/405650335 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc032810 con 0x7f31e40d0460 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.940+0000 7f31e9400640 1 -- 172.21.15.67:0/405650335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40d0460 msgr2=0x7f31e40d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.940+0000 7f31e9400640 1 --2- 172.21.15.67:0/405650335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40d0460 0x7f31e40d08b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0097a0 tx=0x7f31dc02a1e0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.940+0000 7f31e9400640 1 -- 172.21.15.67:0/405650335 shutdown_connections 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.940+0000 7f31e9400640 1 --2- 172.21.15.67:0/405650335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40d0460 0x7f31e40d08b0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.940+0000 7f31e9400640 1 --2- 172.21.15.67:0/405650335 >> v2:172.21.15.169:3300/0 conn(0x7f31e415c6b0 0x7f31e40b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.940+0000 7f31e9400640 1 -- 172.21.15.67:0/405650335 >> 172.21.15.67:0/405650335 conn(0x7f31e406bae0 msgr2=0x7f31e406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.940+0000 7f31e9400640 1 -- 172.21.15.67:0/405650335 shutdown_connections 2024-05-30T17:01:52.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.940+0000 7f31e9400640 1 -- 172.21.15.67:0/405650335 wait complete. 2024-05-30T17:01:52.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.942+0000 7f31e9400640 1 Processor -- start 2024-05-30T17:01:52.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.942+0000 7f31e9400640 1 -- start start 2024-05-30T17:01:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.943+0000 7f31e9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40ccc30 0x7f31e40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.943+0000 7f31e9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31e40d0460 0x7f31e41f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.943+0000 7f31e9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e415c6b0 0x7f31e40c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.943+0000 7f31e9400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40cba10 con 0x7f31e40ccc30 2024-05-30T17:01:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.943+0000 7f31e9400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40cda40 con 0x7f31e415c6b0 2024-05-30T17:01:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.943+0000 7f31e9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f31e40b42f0 con 0x7f31e40d0460 2024-05-30T17:01:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.943+0000 7f31e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40ccc30 0x7f31e40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.943+0000 7f31e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40ccc30 0x7f31e40af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54280/0 (socket says 172.21.15.67:54280) 2024-05-30T17:01:52.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.943+0000 7f31e3e00640 1 -- 172.21.15.67:0/1108400610 learned_addr learned my addr 172.21.15.67:0/1108400610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31e3e00640 1 -- 172.21.15.67:0/1108400610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31e40d0460 msgr2=0x7f31e41f7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31e3e00640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31e40d0460 0x7f31e41f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31e3e00640 1 -- 172.21.15.67:0/1108400610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e415c6b0 msgr2=0x7f31e40c9540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T17:01:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31e3e00640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e415c6b0 0x7f31e40c9540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31e3e00640 1 -- 172.21.15.67:0/1108400610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40b1a70 con 0x7f31e40ccc30 2024-05-30T17:01:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31e3e00640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40ccc30 0x7f31e40af6e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f31d4007e40 tx=0x7f31d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31dbe00640 1 -- 172.21.15.67:0/1108400610 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4014620 con 0x7f31e40ccc30 2024-05-30T17:01:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31dbe00640 1 -- 172.21.15.67:0/1108400610 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d4019070 con 0x7f31e40ccc30 2024-05-30T17:01:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31dbe00640 1 -- 172.21.15.67:0/1108400610 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4014a00 con 0x7f31e40ccc30 2024-05-30T17:01:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e40b1d50 con 0x7f31e40ccc30 2024-05-30T17:01:52.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.944+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e40b2310 con 0x7f31e40ccc30 2024-05-30T17:01:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.946+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e40689e0 con 0x7f31e40ccc30 2024-05-30T17:01:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.947+0000 7f31dbe00640 1 -- 172.21.15.67:0/1108400610 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7f31d4014ba0 con 0x7f31e40ccc30 2024-05-30T17:01:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.947+0000 7f31dbe00640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f31c406df70 0x7f31c4070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.948+0000 7f31dbe00640 1 -- 172.21.15.67:0/1108400610 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f31d408b1e0 con 0x7f31e40ccc30 2024-05-30T17:01:52.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.948+0000 7f31e3400640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f31c406df70 0x7f31c4070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:52.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.948+0000 7f31e3400640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f31c406df70 0x7f31c4070430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f31d0000f60 tx=0x7f31d00078f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:52.952+0000 7f31dbe00640 1 -- 172.21.15.67:0/1108400610 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31d405b070 con 0x7f31e40ccc30 2024-05-30T17:01:53.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.233+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f31e406fc00 con 0x7f31e40ccc30 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.234+0000 7f31dbe00640 1 -- 172.21.15.67:0/1108400610 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2112 (secure 0 0 0) 0x7f31d405a970 con 0x7f31e40ccc30 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:53.236 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1106, 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:01:53.237 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 104, 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:01:53.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 798, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 3222153735, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 7440162816, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 765653950464, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 682, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 6650322, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:01:53.239 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 5 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7, 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 7, 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-05-30T17:01:53.240 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7902 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:01:53.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 19, 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:01:42.499404+0000", 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: "1c47c4a3-eeba-4cb3-b4b4-e5c080306dec": { 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 25.0 seconds before exiting (6s)\n [============================] ", 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:53.242 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:01:53.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f31c406df70 msgr2=0x7f31c4070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f31c406df70 0x7f31c4070430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f31d0000f60 tx=0x7f31d00078f0 comp rx=0 tx=0).stop 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40ccc30 msgr2=0x7f31e40af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40ccc30 0x7f31e40af6e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f31d4007e40 tx=0x7f31d400e4b0 comp rx=0 tx=0).stop 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 shutdown_connections 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f31c406df70 0x7f31c4070430 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e415c6b0 0x7f31e40c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31e40d0460 0x7f31e41f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 --2- 172.21.15.67:0/1108400610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31e40ccc30 0x7f31e40af6e0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 >> 172.21.15.67:0/1108400610 conn(0x7f31e406bae0 msgr2=0x7f31e40c6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 shutdown_connections 2024-05-30T17:01:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:53.237+0000 7f31e9400640 1 -- 172.21.15.67:0/1108400610 wait complete. 2024-05-30T17:01:58.258 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:01:58.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.479+0000 7f4d7be00640 1 Processor -- start 2024-05-30T17:01:58.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.479+0000 7f4d7be00640 1 -- start start 2024-05-30T17:01:58.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.480+0000 7f4d7be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4d7c161df0 0x7f4d7c099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:58.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7c0cc2b0 0x7f4d7c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:58.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d7c1621d0 con 0x7f4d7c0d09a0 2024-05-30T17:01:58.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7c0cd840 con 0x7f4d7c161df0 2024-05-30T17:01:58.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7c0ccf10 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4d7c161df0 0x7f4d7c099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:58.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4d7c161df0 0x7f4d7c099800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54286/0 (socket says 172.21.15.67:54286) 2024-05-30T17:01:58.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7aa00640 1 -- 172.21.15.67:0/1908960390 learned_addr learned my addr 172.21.15.67:0/1908960390 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:58.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7aa00640 1 -- 172.21.15.67:0/1908960390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7c0cc2b0 msgr2=0x7f4d7c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:58.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7aa00640 1 --2- 172.21.15.67:0/1908960390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7c0cc2b0 0x7f4d7c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7aa00640 1 -- 172.21.15.67:0/1908960390 >> v1:172.21.15.169:6789/0 conn(0x7f4d7c0d09a0 legacy=0x7f4d7c0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:01:58.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.481+0000 7f4d7aa00640 1 -- 172.21.15.67:0/1908960390 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c0cd1f0 con 0x7f4d7c161df0 2024-05-30T17:01:58.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.483+0000 7f4d7aa00640 1 --2- 172.21.15.67:0/1908960390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c161df0 0x7f4d7c099800 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009e50 tx=0x7f4d6c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71dbb14a5569c18 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:58.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.483+0000 7f4d7a000640 1 -- 172.21.15.67:0/1908960390 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c036070 con 0x7f4d7c161df0 2024-05-30T17:01:58.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.483+0000 7f4d7a000640 1 -- 172.21.15.67:0/1908960390 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6c031600 con 0x7f4d7c161df0 2024-05-30T17:01:58.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.483+0000 7f4d7a000640 1 -- 172.21.15.67:0/1908960390 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0319d0 con 0x7f4d7c161df0 2024-05-30T17:01:58.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.484+0000 7f4d7be00640 1 -- 172.21.15.67:0/1908960390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c161df0 msgr2=0x7f4d7c099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:58.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.484+0000 7f4d7be00640 1 --2- 172.21.15.67:0/1908960390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c161df0 0x7f4d7c099800 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009e50 tx=0x7f4d6c02a6c0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.484+0000 7f4d7be00640 1 -- 172.21.15.67:0/1908960390 shutdown_connections 2024-05-30T17:01:58.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.484+0000 7f4d7be00640 1 --2- 172.21.15.67:0/1908960390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7c0cc2b0 0x7f4d7c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.484+0000 7f4d7be00640 1 --2- 172.21.15.67:0/1908960390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c161df0 0x7f4d7c099800 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.484+0000 7f4d7be00640 1 -- 172.21.15.67:0/1908960390 >> 172.21.15.67:0/1908960390 conn(0x7f4d7c06bae0 msgr2=0x7f4d7c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:58.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.484+0000 7f4d7be00640 1 -- 172.21.15.67:0/1908960390 shutdown_connections 2024-05-30T17:01:58.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.484+0000 7f4d7be00640 1 -- 172.21.15.67:0/1908960390 wait complete. 2024-05-30T17:01:58.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.486+0000 7f4d7be00640 1 Processor -- start 2024-05-30T17:01:58.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.486+0000 7f4d7be00640 1 -- start start 2024-05-30T17:01:58.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.487+0000 7f4d7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c0cc2b0 0x7f4d7c15f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:58.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.487+0000 7f4d7be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d7c0d09a0 0x7f4d7c151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.487+0000 7f4d7be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7c161df0 0x7f4d7c072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.487+0000 7f4d7be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c0cffb0 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.487+0000 7f4d7be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c099d40 con 0x7f4d7c0d09a0 2024-05-30T17:01:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.487+0000 7f4d7be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7c0ccbc0 con 0x7f4d7c161df0 2024-05-30T17:01:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.488+0000 7f4d7aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c0cc2b0 0x7f4d7c15f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.488+0000 7f4d7aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c0cc2b0 0x7f4d7c15f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54298/0 (socket says 172.21.15.67:54298) 2024-05-30T17:01:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.488+0000 7f4d7aa00640 1 -- 172.21.15.67:0/2072389961 learned_addr learned my addr 172.21.15.67:0/2072389961 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:01:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.488+0000 7f4d7aa00640 1 -- 172.21.15.67:0/2072389961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7c161df0 msgr2=0x7f4d7c072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:58.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.488+0000 7f4d7b400640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7c161df0 0x7f4d7c072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:58.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.488+0000 7f4d7aa00640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7c161df0 0x7f4d7c072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.488+0000 7f4d7aa00640 1 -- 172.21.15.67:0/2072389961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d7c0d09a0 msgr2=0x7f4d7c151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:58.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.488+0000 7f4d7aa00640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d7c0d09a0 0x7f4d7c151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.488+0000 7f4d7aa00640 1 -- 172.21.15.67:0/2072389961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c0c70d0 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.489+0000 7f4d7aa00640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c0cc2b0 0x7f4d7c15f310 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009f70 tx=0x7f4d6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.489+0000 7f4d73400640 1 -- 172.21.15.67:0/2072389961 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c041020 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.489+0000 7f4d73400640 1 -- 172.21.15.67:0/2072389961 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6c007800 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.489+0000 7f4d73400640 1 -- 172.21.15.67:0/2072389961 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c036040 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.489+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7c15e550 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.489+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7c15eb10 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.490+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d7c0689e0 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.491+0000 7f4d73400640 1 -- 172.21.15.67:0/2072389961 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7f4d6c044020 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.492+0000 7f4d73400640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4d5806df70 0x7f4d58070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:01:58.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.492+0000 7f4d73400640 1 -- 172.21.15.67:0/2072389961 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f4d6c0ab7e0 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.492+0000 7f4d72000640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4d5806df70 0x7f4d58070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:01:58.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.493+0000 7f4d72000640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4d5806df70 0x7f4d58070430 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c073880 tx=0x7f4d64006cb0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:01:58.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.498+0000 7f4d73400640 1 -- 172.21.15.67:0/2072389961 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d6c04b030 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.783+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4d7c06f8b0 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.784+0000 7f4d73400640 1 -- 172.21.15.67:0/2072389961 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2110 (secure 0 0 0) 0x7f4d7c1621d0 con 0x7f4d7c0cc2b0 2024-05-30T17:01:58.784 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:01:58.784 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:58.785 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1112, 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:01:58.786 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 104, 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:01:58.787 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 797, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 3222178536, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 7440207872, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 765653905408, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 511, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 8359, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 3 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:01:58.788 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7, 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 7, 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7902 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:01:58.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 19, 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:01:42.499404+0000", 2024-05-30T17:01:58.790 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:01:58.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:01:58.791 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:01:58.791 INFO:teuthology.orchestra.run.smithi067.stdout: "1c47c4a3-eeba-4cb3-b4b4-e5c080306dec": { 2024-05-30T17:01:58.791 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 20.0 seconds before exiting (11s)\n [============================] ", 2024-05-30T17:01:58.791 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:01:58.791 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:01:58.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:58.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:01:58.791 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:01:58.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4d5806df70 msgr2=0x7f4d58070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:58.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4d5806df70 0x7f4d58070430 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c073880 tx=0x7f4d64006cb0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c0cc2b0 msgr2=0x7f4d7c15f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:01:58.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c0cc2b0 0x7f4d7c15f310 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009f70 tx=0x7f4d6c004a40 comp rx=0 tx=0).stop 2024-05-30T17:01:58.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 shutdown_connections 2024-05-30T17:01:58.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4d5806df70 0x7f4d58070430 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d7c161df0 0x7f4d7c072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d7c0d09a0 0x7f4d7c151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 --2- 172.21.15.67:0/2072389961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d7c0cc2b0 0x7f4d7c15f310 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:01:58.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.786+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 >> 172.21.15.67:0/2072389961 conn(0x7f4d7c06bae0 msgr2=0x7f4d7c06ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:01:58.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.787+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 shutdown_connections 2024-05-30T17:01:58.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:01:58.787+0000 7f4d7be00640 1 -- 172.21.15.67:0/2072389961 wait complete. 2024-05-30T17:02:03.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:02:04.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.017+0000 7f0bb7c00640 1 Processor -- start 2024-05-30T17:02:04.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.017+0000 7f0bb7c00640 1 -- start start 2024-05-30T17:02:04.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.019+0000 7f0bb7c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0bb00b4410 0x7f0bb006f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:04.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.019+0000 7f0bb7c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb01638c0 con 0x7f0bb0162d80 2024-05-30T17:02:04.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.019+0000 7f0bb7c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb0163ba0 con 0x7f0bb0163160 2024-05-30T17:02:04.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.019+0000 7f0bb7c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb0163e80 con 0x7f0bb00b4410 2024-05-30T17:02:04.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0bb5200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0bb0162d80 0x7f0bb00b3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44456/0 (socket says 172.21.15.67:44456) 2024-05-30T17:02:04.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0bb5200640 1 -- 172.21.15.67:0/367816065 learned_addr learned my addr 172.21.15.67:0/367816065 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0bafe00640 1 --2- 172.21.15.67:0/367816065 >> v2:172.21.15.169:3300/0 conn(0x7f0bb00b4410 0x7f0bb006f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4262734427 0 0) 0x7f0bb01638c0 con 0x7f0bb0162d80 2024-05-30T17:02:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b98003520 con 0x7f0bb0162d80 2024-05-30T17:02:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4076339250 0 0) 0x7f0b98003520 con 0x7f0bb0162d80 2024-05-30T17:02:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 >> v2:172.21.15.169:3300/0 conn(0x7f0bb00b4410 msgr2=0x7f0bb006f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0baf400640 1 --2- 172.21.15.67:0/367816065 >> v2:172.21.15.169:3300/0 conn(0x7f0bb00b4410 0x7f0bb006f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 >> v1:172.21.15.169:6789/0 conn(0x7f0bb0163160 legacy=0x7f0bb0164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.020+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb0164160 con 0x7f0bb0162d80 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f0ba4001670 con 0x7f0bb0162d80 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 >> v1:172.21.15.67:6789/0 conn(0x7f0bb0162d80 legacy=0x7f0bb00b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0baf400640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b98003a00 0x7f0b98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0baf400640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0b98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0baf400640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b980051c0 0x7f0b9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb0163e80 con 0x7f0b98004800 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9800c070 con 0x7f0b98003a00 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0b9800c270 con 0x7f0b980051c0 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0bafe00640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0b98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0bb5200640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b98003a00 0x7f0b98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.021+0000 7f0bb5c00640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b980051c0 0x7f0b9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:04.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.022+0000 7f0bafe00640 1 -- 172.21.15.67:0/367816065 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b980051c0 msgr2=0x7f0b9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:04.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.022+0000 7f0bafe00640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b980051c0 0x7f0b9800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.022+0000 7f0bafe00640 1 -- 172.21.15.67:0/367816065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b98003a00 msgr2=0x7f0b98003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:04.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.022+0000 7f0bafe00640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b98003a00 0x7f0b98003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.022+0000 7f0bafe00640 1 -- 172.21.15.67:0/367816065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b98003520 con 0x7f0b98004800 2024-05-30T17:02:04.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.022+0000 7f0bb5c00640 1 -- 172.21.15.67:0/367816065 reap_dead start 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.023+0000 7f0bafe00640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0b98004c80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009ac0 tx=0x7f0b98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c993d20fe93c47e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.023+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c036070 con 0x7f0b98004800 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.023+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b9c031400 con 0x7f0b98004800 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.023+0000 7f0baf400640 1 -- 172.21.15.67:0/367816065 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c009690 con 0x7f0b98004800 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.024+0000 7f0bb7c00640 1 -- 172.21.15.67:0/367816065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 msgr2=0x7f0b98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.024+0000 7f0bb7c00640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0b98004c80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009ac0 tx=0x7f0b98005830 comp rx=0 tx=0).stop 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.024+0000 7f0bb7c00640 1 -- 172.21.15.67:0/367816065 shutdown_connections 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.024+0000 7f0bb7c00640 1 --2- 172.21.15.67:0/367816065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0b98004c80 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.024+0000 7f0bb7c00640 1 -- 172.21.15.67:0/367816065 >> 172.21.15.67:0/367816065 conn(0x7f0bb0067ce0 msgr2=0x7f0bb006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.024+0000 7f0bb7c00640 1 -- 172.21.15.67:0/367816065 shutdown_connections 2024-05-30T17:02:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.024+0000 7f0bb7c00640 1 -- 172.21.15.67:0/367816065 wait complete. 2024-05-30T17:02:04.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.026+0000 7f0bb7c00640 1 Processor -- start 2024-05-30T17:02:04.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.026+0000 7f0bb7c00640 1 -- start start 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb7c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0bb0159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb7c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cf7d0 0x7f0bb00aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb7c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cfc30 0x7f0bb00aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb7c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb006f180 con 0x7f0b98004800 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb7c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb0163590 con 0x7f0bb00cf7d0 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb7c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb006f300 con 0x7f0bb00cfc30 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0bb0159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0bb0159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48738/0 (socket says 172.21.15.67:48738) 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb5200640 1 -- 172.21.15.67:0/3178200853 learned_addr learned my addr 172.21.15.67:0/3178200853 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bafe00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cf7d0 0x7f0bb00aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb5200640 1 -- 172.21.15.67:0/3178200853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cfc30 msgr2=0x7f0bb00aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb5200640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cfc30 0x7f0bb00aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb5200640 1 -- 172.21.15.67:0/3178200853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cf7d0 msgr2=0x7f0bb00aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.027+0000 7f0bb5200640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cf7d0 0x7f0bb00aa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.028+0000 7f0bb5200640 1 -- 172.21.15.67:0/3178200853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb00d0200 con 0x7f0b98004800 2024-05-30T17:02:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.028+0000 7f0bb5200640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0bb0159930 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4001050 tx=0x7f0ba400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:04.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.028+0000 7f0bacc00640 1 -- 172.21.15.67:0/3178200853 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400c6c0 con 0x7f0b98004800 2024-05-30T17:02:04.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.028+0000 7f0bacc00640 1 -- 172.21.15.67:0/3178200853 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba4018070 con 0x7f0b98004800 2024-05-30T17:02:04.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.028+0000 7f0bacc00640 1 -- 172.21.15.67:0/3178200853 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400cac0 con 0x7f0b98004800 2024-05-30T17:02:04.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.028+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb00d04e0 con 0x7f0b98004800 2024-05-30T17:02:04.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.028+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb0159e70 con 0x7f0b98004800 2024-05-30T17:02:04.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.030+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb00d10c0 con 0x7f0b98004800 2024-05-30T17:02:04.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.031+0000 7f0bacc00640 1 -- 172.21.15.67:0/3178200853 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7f0ba401d460 con 0x7f0b98004800 2024-05-30T17:02:04.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.031+0000 7f0bacc00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9006df70 0x7f0b90070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:04.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.032+0000 7f0bacc00640 1 -- 172.21.15.67:0/3178200853 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f0ba408a9f0 con 0x7f0b98004800 2024-05-30T17:02:04.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.032+0000 7f0bafe00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9006df70 0x7f0b90070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:04.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.033+0000 7f0bafe00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9006df70 0x7f0b90070430 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c033000 tx=0x7f0b9c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:04.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.036+0000 7f0bacc00640 1 -- 172.21.15.67:0/3178200853 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0bb00d10c0 con 0x7f0b98004800 2024-05-30T17:02:04.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.317+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0bb00689e0 con 0x7f0b98004800 2024-05-30T17:02:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.318+0000 7f0bacc00640 1 -- 172.21.15.67:0/3178200853 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2110 (secure 0 0 0) 0x7f0ba405a880 con 0x7f0b98004800 2024-05-30T17:02:04.319 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:04.319 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:02:04.319 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:02:04.319 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:02:04.319 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:02:04.319 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:02:04.319 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:02:04.320 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1117, 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 104, 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:02:04.321 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 797, 2024-05-30T17:02:04.322 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 3222188042, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 7440228352, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 765653884928, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 255, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 3923, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 1 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:02:04.323 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 7, 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7902 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:02:04.324 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 19, 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:01:42.499404+0000", 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "1c47c4a3-eeba-4cb3-b4b4-e5c080306dec": { 2024-05-30T17:02:04.325 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 16.0 seconds before exiting (15s)\n [============================] ", 2024-05-30T17:02:04.326 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:02:04.326 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:02:04.326 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:04.326 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:04.326 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:02:04.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.321+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9006df70 msgr2=0x7f0b90070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.321+0000 7f0bb7c00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9006df70 0x7f0b90070430 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c033000 tx=0x7f0b9c008030 comp rx=0 tx=0).stop 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.321+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 msgr2=0x7f0bb0159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.321+0000 7f0bb7c00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0bb0159930 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4001050 tx=0x7f0ba400eb90 comp rx=0 tx=0).stop 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.322+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 shutdown_connections 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.322+0000 7f0bb7c00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0b9006df70 0x7f0b90070430 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.322+0000 7f0bb7c00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bb00cfc30 0x7f0bb00aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.322+0000 7f0bb7c00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bb00cf7d0 0x7f0bb00aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.322+0000 7f0bb7c00640 1 --2- 172.21.15.67:0/3178200853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b98004800 0x7f0bb0159930 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.322+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 >> 172.21.15.67:0/3178200853 conn(0x7f0bb0067ce0 msgr2=0x7f0bb0164960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.322+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 shutdown_connections 2024-05-30T17:02:04.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:04.322+0000 7f0bb7c00640 1 -- 172.21.15.67:0/3178200853 wait complete. 2024-05-30T17:02:09.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:02:09.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.561+0000 7eff61400640 1 Processor -- start 2024-05-30T17:02:09.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.562+0000 7eff61400640 1 -- start start 2024-05-30T17:02:09.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.563+0000 7eff61400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff5c06c7b0 0x7eff5c06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:09.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.563+0000 7eff61400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c0d50d0 con 0x7eff5c0cfd60 2024-05-30T17:02:09.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.563+0000 7eff61400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c0d53b0 con 0x7eff5c161b90 2024-05-30T17:02:09.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.563+0000 7eff61400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7eff5c0d2040 con 0x7eff5c06c7b0 2024-05-30T17:02:09.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.564+0000 7eff60a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7eff5c0cfd60 0x7eff5c0d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44470/0 (socket says 172.21.15.67:44470) 2024-05-30T17:02:09.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.564+0000 7eff60a00640 1 -- 172.21.15.67:0/3202795472 learned_addr learned my addr 172.21.15.67:0/3202795472 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:09.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.564+0000 7eff5b400640 1 --2- 172.21.15.67:0/3202795472 >> v2:172.21.15.169:3300/0 conn(0x7eff5c06c7b0 0x7eff5c06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:09.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.564+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006974772 0 0) 0x7eff5c0d50d0 con 0x7eff5c0cfd60 2024-05-30T17:02:09.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.564+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff40003520 con 0x7eff5c0cfd60 2024-05-30T17:02:09.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1907930297 0 0) 0x7eff40003520 con 0x7eff5c0cfd60 2024-05-30T17:02:09.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 >> v2:172.21.15.169:3300/0 conn(0x7eff5c06c7b0 msgr2=0x7eff5c06cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:09.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 --2- 172.21.15.67:0/3202795472 >> v2:172.21.15.169:3300/0 conn(0x7eff5c06c7b0 0x7eff5c06cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 >> v1:172.21.15.169:6789/0 conn(0x7eff5c161b90 legacy=0x7eff5c0d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:09.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c0cf560 con 0x7eff5c0cfd60 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7eff54001670 con 0x7eff5c0cfd60 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 >> v1:172.21.15.67:6789/0 conn(0x7eff5c0cfd60 legacy=0x7eff5c0d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 --2- 172.21.15.67:0/3202795472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 --2- 172.21.15.67:0/3202795472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff40004800 0x7eff40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 --2- 172.21.15.67:0/3202795472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff400052e0 0x7eff4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff4000c1b0 con 0x7eff40003a00 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff4000c360 con 0x7eff400052e0 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff4000c560 con 0x7eff40004800 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.565+0000 7eff5be00640 1 --2- 172.21.15.67:0/3202795472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.566+0000 7eff5be00640 1 -- 172.21.15.67:0/3202795472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff40004800 msgr2=0x7eff40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.566+0000 7eff5be00640 1 --2- 172.21.15.67:0/3202795472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff40004800 0x7eff40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.566+0000 7eff5be00640 1 -- 172.21.15.67:0/3202795472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff400052e0 msgr2=0x7eff4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.566+0000 7eff5be00640 1 --2- 172.21.15.67:0/3202795472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff400052e0 0x7eff4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.566+0000 7eff5be00640 1 -- 172.21.15.67:0/3202795472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff40003520 con 0x7eff40003a00 2024-05-30T17:02:09.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.566+0000 7eff60a00640 1 -- 172.21.15.67:0/3202795472 reap_dead start 2024-05-30T17:02:09.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.567+0000 7eff5be00640 1 --2- 172.21.15.67:0/3202795472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff40003e20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7eff4c001e50 tx=0x7eff40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c633b391bfdfa4e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.567+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c001a70 con 0x7eff40003a00 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.567+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4c003970 con 0x7eff40003a00 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.567+0000 7eff5aa00640 1 -- 172.21.15.67:0/3202795472 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c003d60 con 0x7eff40003a00 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.568+0000 7eff61400640 1 -- 172.21.15.67:0/3202795472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 msgr2=0x7eff40003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.568+0000 7eff61400640 1 --2- 172.21.15.67:0/3202795472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff40003e20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7eff4c001e50 tx=0x7eff40005830 comp rx=0 tx=0).stop 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.568+0000 7eff61400640 1 -- 172.21.15.67:0/3202795472 shutdown_connections 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.568+0000 7eff61400640 1 --2- 172.21.15.67:0/3202795472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff40003e20 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.568+0000 7eff61400640 1 -- 172.21.15.67:0/3202795472 >> 172.21.15.67:0/3202795472 conn(0x7eff5c08bfc0 msgr2=0x7eff5c0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.568+0000 7eff61400640 1 -- 172.21.15.67:0/3202795472 shutdown_connections 2024-05-30T17:02:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.568+0000 7eff61400640 1 -- 172.21.15.67:0/3202795472 wait complete. 2024-05-30T17:02:09.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.570+0000 7eff61400640 1 Processor -- start 2024-05-30T17:02:09.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.570+0000 7eff61400640 1 -- start start 2024-05-30T17:02:09.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff61400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff5c0b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:09.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff61400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0b9a40 0x7eff5c14d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:09.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff61400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cdee0 0x7eff5c1f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff61400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c0cf210 con 0x7eff40003a00 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff61400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c0cf8d0 con 0x7eff5c0cdee0 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff61400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff5c0d4d30 con 0x7eff5c0b9a40 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff5c0b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff5c0b83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48752/0 (socket says 172.21.15.67:48752) 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff5be00640 1 -- 172.21.15.67:0/2220804918 learned_addr learned my addr 172.21.15.67:0/2220804918 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff5b400640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0b9a40 0x7eff5c14d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff5be00640 1 -- 172.21.15.67:0/2220804918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0b9a40 msgr2=0x7eff5c14d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff60a00640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cdee0 0x7eff5c1f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff5be00640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0b9a40 0x7eff5c14d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff5be00640 1 -- 172.21.15.67:0/2220804918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cdee0 msgr2=0x7eff5c1f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff5be00640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cdee0 0x7eff5c1f7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.571+0000 7eff5be00640 1 -- 172.21.15.67:0/2220804918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c0b89a0 con 0x7eff40003a00 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.572+0000 7eff5be00640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff5c0b83a0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7eff4c001f70 tx=0x7eff4c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:09.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.572+0000 7eff53e00640 1 -- 172.21.15.67:0/2220804918 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c03f020 con 0x7eff40003a00 2024-05-30T17:02:09.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.572+0000 7eff53e00640 1 -- 172.21.15.67:0/2220804918 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4c006d60 con 0x7eff40003a00 2024-05-30T17:02:09.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.572+0000 7eff53e00640 1 -- 172.21.15.67:0/2220804918 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c0052f0 con 0x7eff40003a00 2024-05-30T17:02:09.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.572+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c0b8c80 con 0x7eff40003a00 2024-05-30T17:02:09.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.572+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c0b9240 con 0x7eff40003a00 2024-05-30T17:02:09.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.573+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff5c0d2410 con 0x7eff40003a00 2024-05-30T17:02:09.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.574+0000 7eff53e00640 1 -- 172.21.15.67:0/2220804918 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7eff4c042020 con 0x7eff40003a00 2024-05-30T17:02:09.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.575+0000 7eff53e00640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff3806dfa0 0x7eff38070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:09.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.575+0000 7eff53e00640 1 -- 172.21.15.67:0/2220804918 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7eff4c0aaac0 con 0x7eff40003a00 2024-05-30T17:02:09.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.576+0000 7eff5b400640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff3806dfa0 0x7eff38070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:09.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.576+0000 7eff5b400640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff3806dfa0 0x7eff38070460 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0d1980 tx=0x7eff44008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:09.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.580+0000 7eff53e00640 1 -- 172.21.15.67:0/2220804918 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff5c0d2410 con 0x7eff40003a00 2024-05-30T17:02:09.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.865+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7eff5c0690f0 con 0x7eff40003a00 2024-05-30T17:02:09.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.866+0000 7eff53e00640 1 -- 172.21.15.67:0/2220804918 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2079 (secure 0 0 0) 0x7eff4c049020 con 0x7eff40003a00 2024-05-30T17:02:09.866 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:09.866 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:02:09.866 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:02:09.867 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1123, 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 104, 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:02:09.868 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 797, 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 3222189657, 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 7458709504, 2024-05-30T17:02:09.869 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 765635403776, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 1194, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:09.870 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 7, 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7902 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:02:09.871 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 19, 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:01:42.499404+0000", 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "1c47c4a3-eeba-4cb3-b4b4-e5c080306dec": { 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 10.0 seconds before exiting (21s)\n [============================] ", 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:09.872 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:02:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff3806dfa0 msgr2=0x7eff38070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff3806dfa0 0x7eff38070460 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0d1980 tx=0x7eff44008040 comp rx=0 tx=0).stop 2024-05-30T17:02:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 msgr2=0x7eff5c0b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff5c0b83a0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7eff4c001f70 tx=0x7eff4c0014a0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 shutdown_connections 2024-05-30T17:02:09.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff3806dfa0 0x7eff38070460 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff5c0cdee0 0x7eff5c1f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff5c0b9a40 0x7eff5c14d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 --2- 172.21.15.67:0/2220804918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff40003a00 0x7eff5c0b83a0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 >> 172.21.15.67:0/2220804918 conn(0x7eff5c08bfc0 msgr2=0x7eff5c0d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 shutdown_connections 2024-05-30T17:02:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:09.869+0000 7eff61400640 1 -- 172.21.15.67:0/2220804918 wait complete. 2024-05-30T17:02:14.891 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:02:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.111+0000 7fb419400640 1 Processor -- start 2024-05-30T17:02:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.112+0000 7fb419400640 1 -- start start 2024-05-30T17:02:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.113+0000 7fb419400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4140d1040 0x7fb4140d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.113+0000 7fb419400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb41406c950 0x7fb41406cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.113+0000 7fb419400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4140cf7e0 con 0x7fb41415ea40 2024-05-30T17:02:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.113+0000 7fb419400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb41415ee20 con 0x7fb41406c950 2024-05-30T17:02:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.113+0000 7fb419400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb4140d19d0 con 0x7fb4140d1040 2024-05-30T17:02:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.113+0000 7fb413200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb41406c950 0x7fb41406cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.113+0000 7fb413200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb41406c950 0x7fb41406cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48110/0 (socket says 172.21.15.67:48110) 2024-05-30T17:02:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.113+0000 7fb413200640 1 -- 172.21.15.67:0/379314103 learned_addr learned my addr 172.21.15.67:0/379314103 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.114+0000 7fb40be00640 1 --2- 172.21.15.67:0/379314103 >> v2:172.21.15.169:3300/0 conn(0x7fb4140d1040 0x7fb4140d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.114+0000 7fb413200640 1 -- 172.21.15.67:0/379314103 >> v2:172.21.15.169:3300/0 conn(0x7fb4140d1040 msgr2=0x7fb4140d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.114+0000 7fb413200640 1 --2- 172.21.15.67:0/379314103 >> v2:172.21.15.169:3300/0 conn(0x7fb4140d1040 0x7fb4140d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.114+0000 7fb413200640 1 -- 172.21.15.67:0/379314103 >> v1:172.21.15.169:6789/0 conn(0x7fb41415ea40 legacy=0x7fb4140d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.114+0000 7fb413200640 1 -- 172.21.15.67:0/379314103 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41415fe50 con 0x7fb41406c950 2024-05-30T17:02:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.115+0000 7fb413200640 1 --2- 172.21.15.67:0/379314103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb41406c950 0x7fb41406cda0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb4140ceec0 tx=0x7fb40c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26f7af100259079 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.115+0000 7fb411e00640 1 -- 172.21.15.67:0/379314103 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c036070 con 0x7fb41406c950 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.115+0000 7fb411e00640 1 -- 172.21.15.67:0/379314103 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c031600 con 0x7fb41406c950 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.115+0000 7fb411e00640 1 -- 172.21.15.67:0/379314103 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0319d0 con 0x7fb41406c950 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.116+0000 7fb419400640 1 -- 172.21.15.67:0/379314103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb41406c950 msgr2=0x7fb41406cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.116+0000 7fb419400640 1 --2- 172.21.15.67:0/379314103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb41406c950 0x7fb41406cda0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb4140ceec0 tx=0x7fb40c02a660 comp rx=0 tx=0).stop 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.116+0000 7fb419400640 1 -- 172.21.15.67:0/379314103 shutdown_connections 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.116+0000 7fb419400640 1 --2- 172.21.15.67:0/379314103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb41406c950 0x7fb41406cda0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.116+0000 7fb419400640 1 --2- 172.21.15.67:0/379314103 >> v2:172.21.15.169:3300/0 conn(0x7fb4140d1040 0x7fb4140d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.116+0000 7fb419400640 1 -- 172.21.15.67:0/379314103 >> 172.21.15.67:0/379314103 conn(0x7fb41408bfc0 msgr2=0x7fb4140d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.116+0000 7fb419400640 1 -- 172.21.15.67:0/379314103 shutdown_connections 2024-05-30T17:02:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.116+0000 7fb419400640 1 -- 172.21.15.67:0/379314103 wait complete. 2024-05-30T17:02:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.118+0000 7fb419400640 1 Processor -- start 2024-05-30T17:02:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.118+0000 7fb419400640 1 -- start start 2024-05-30T17:02:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb419400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb41406c950 0x7fb4141627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb419400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4140d1040 0x7fb414160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb419400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb41415ea40 0x7fb414160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb419400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb4140d0040 con 0x7fb4140d1040 2024-05-30T17:02:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb419400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4140cfd60 con 0x7fb41406c950 2024-05-30T17:02:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb419400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4140d0320 con 0x7fb41415ea40 2024-05-30T17:02:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb40be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4140d1040 0x7fb414160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb40be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4140d1040 0x7fb414160020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48116/0 (socket says 172.21.15.67:48116) 2024-05-30T17:02:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb40be00640 1 -- 172.21.15.67:0/975777493 learned_addr learned my addr 172.21.15.67:0/975777493 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb412800640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb41406c950 0x7fb4141627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.119+0000 7fb413200640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb41415ea40 0x7fb414160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb40be00640 1 -- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb41415ea40 msgr2=0x7fb414160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb40be00640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb41415ea40 0x7fb414160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb40be00640 1 -- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb41406c950 msgr2=0x7fb4141627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb40be00640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb41406c950 0x7fb4141627c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb40be00640 1 -- 172.21.15.67:0/975777493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb414160ca0 con 0x7fb4140d1040 2024-05-30T17:02:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb412800640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb41406c950 0x7fb4141627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb413200640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb41415ea40 0x7fb414160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb40be00640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4140d1040 0x7fb414160020 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00c7f0 tx=0x7fb3fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb40aa00640 1 -- 172.21.15.67:0/975777493 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc015070 con 0x7fb4140d1040 2024-05-30T17:02:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb40aa00640 1 -- 172.21.15.67:0/975777493 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3fc004240 con 0x7fb4140d1040 2024-05-30T17:02:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb414160f80 con 0x7fb4140d1040 2024-05-30T17:02:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb40aa00640 1 -- 172.21.15.67:0/975777493 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc007620 con 0x7fb4140d1040 2024-05-30T17:02:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.120+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb414161540 con 0x7fb4140d1040 2024-05-30T17:02:15.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.121+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4140689e0 con 0x7fb4140d1040 2024-05-30T17:02:15.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.123+0000 7fb40aa00640 1 -- 172.21.15.67:0/975777493 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7fb3fc0079d0 con 0x7fb4140d1040 2024-05-30T17:02:15.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.124+0000 7fb40aa00640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3f006df70 0x7fb3f0070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:15.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.124+0000 7fb40aa00640 1 -- 172.21.15.67:0/975777493 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fb3fc08ada0 con 0x7fb4140d1040 2024-05-30T17:02:15.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.124+0000 7fb412800640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3f006df70 0x7fb3f0070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:15.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.125+0000 7fb412800640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3f006df70 0x7fb3f0070430 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb40400a810 tx=0x7fb404008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:15.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.128+0000 7fb40aa00640 1 -- 172.21.15.67:0/975777493 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3fc05ac30 con 0x7fb4140d1040 2024-05-30T17:02:15.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:02:15.369 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:02:15.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.410+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb4140d1c70 con 0x7fb4140d1040 2024-05-30T17:02:15.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.411+0000 7fb40aa00640 1 -- 172.21.15.67:0/975777493 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2077 (secure 0 0 0) 0x7fb3fc05a530 con 0x7fb4140d1040 2024-05-30T17:02:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:15.411 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:02:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:02:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:02:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:02:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:02:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:02:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:02:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:02:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:02:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1128, 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:02:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 104, 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:02:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:02:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 797, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 3222189657, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 7458709504, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 765635403776, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 170, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:02:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 7, 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7902 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:02:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:02:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:02:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:02:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:02:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:02:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 19, 2024-05-30T17:02:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:01:42.499404+0000", 2024-05-30T17:02:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:02:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:02:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "1c47c4a3-eeba-4cb3-b4b4-e5c080306dec": { 2024-05-30T17:02:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 4.0 seconds before exiting (27s)\n [============================] ", 2024-05-30T17:02:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:02:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:02:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:15.419 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:02:15.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3f006df70 msgr2=0x7fb3f0070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:15.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3f006df70 0x7fb3f0070430 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb40400a810 tx=0x7fb404008040 comp rx=0 tx=0).stop 2024-05-30T17:02:15.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4140d1040 msgr2=0x7fb414160020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:15.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4140d1040 0x7fb414160020 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00c7f0 tx=0x7fb3fc00ccb0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 shutdown_connections 2024-05-30T17:02:15.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3f006df70 0x7fb3f0070430 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb41415ea40 0x7fb414160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4140d1040 0x7fb414160020 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 --2- 172.21.15.67:0/975777493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb41406c950 0x7fb4141627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:15.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.413+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 >> 172.21.15.67:0/975777493 conn(0x7fb41408bfc0 msgr2=0x7fb4140d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:15.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.414+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 shutdown_connections 2024-05-30T17:02:15.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:15.414+0000 7fb419400640 1 -- 172.21.15.67:0/975777493 wait complete. 2024-05-30T17:02:20.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:02:20.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.654+0000 7f7023200640 1 Processor -- start 2024-05-30T17:02:20.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.655+0000 7f7023200640 1 -- start start 2024-05-30T17:02:20.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.656+0000 7f7023200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f701c15e690 0x7f701c0d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:20.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.656+0000 7f7023200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c0d1ca0 0x7f701c0d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:20.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.657+0000 7f7023200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f701c0d3060 con 0x7f701c15ea70 2024-05-30T17:02:20.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.657+0000 7f7023200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f701c0d3340 con 0x7f701c15e690 2024-05-30T17:02:20.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.657+0000 7f7023200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f701c0d3620 con 0x7f701c0d1ca0 2024-05-30T17:02:20.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.657+0000 7f7021400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c0d1ca0 0x7f701c0d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:20.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.657+0000 7f7021e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f701c15e690 0x7f701c0d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:20.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.657+0000 7f7021400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c0d1ca0 0x7f701c0d4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:39498/0 (socket says 172.21.15.67:39498) 2024-05-30T17:02:20.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.657+0000 7f7021400640 1 -- 172.21.15.67:0/39702598 learned_addr learned my addr 172.21.15.67:0/39702598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:20.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.658+0000 7f7021400640 1 -- 172.21.15.67:0/39702598 >> v2:172.21.15.169:3300/0 conn(0x7f701c15e690 msgr2=0x7f701c0d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:20.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.658+0000 7f7021400640 1 --2- 172.21.15.67:0/39702598 >> v2:172.21.15.169:3300/0 conn(0x7f701c15e690 0x7f701c0d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.658+0000 7f7021400640 1 -- 172.21.15.67:0/39702598 >> v1:172.21.15.169:6789/0 conn(0x7f701c15ea70 legacy=0x7f701c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:20.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.658+0000 7f7021400640 1 -- 172.21.15.67:0/39702598 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c0cf770 con 0x7f701c0d1ca0 2024-05-30T17:02:20.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.658+0000 7f7021e00640 1 --2- 172.21.15.67:0/39702598 >> v2:172.21.15.169:3300/0 conn(0x7f701c15e690 0x7f701c0d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:02:20.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.658+0000 7f7020a00640 1 -- 172.21.15.67:0/39702598 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2524121180 0 0) 0x7f701c0d3060 con 0x7f701c15ea70 2024-05-30T17:02:20.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.660+0000 7f7021400640 1 --2- 172.21.15.67:0/39702598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c0d1ca0 0x7f701c0d4a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f700c009850 tx=0x7f700c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=357f9ab2430cc946 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:20.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.660+0000 7f7020a00640 1 -- 172.21.15.67:0/39702598 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c004030 con 0x7f701c0d1ca0 2024-05-30T17:02:20.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.660+0000 7f7020a00640 1 -- 172.21.15.67:0/39702598 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c0041d0 con 0x7f701c0d1ca0 2024-05-30T17:02:20.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.660+0000 7f7020a00640 1 -- 172.21.15.67:0/39702598 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c003610 con 0x7f701c0d1ca0 2024-05-30T17:02:20.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.661+0000 7f7023200640 1 -- 172.21.15.67:0/39702598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c0d1ca0 msgr2=0x7f701c0d4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:20.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.661+0000 7f7023200640 1 --2- 172.21.15.67:0/39702598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c0d1ca0 0x7f701c0d4a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f700c009850 tx=0x7f700c02a250 comp rx=0 tx=0).stop 2024-05-30T17:02:20.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.661+0000 7f7023200640 1 -- 172.21.15.67:0/39702598 shutdown_connections 2024-05-30T17:02:20.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.661+0000 7f7023200640 1 --2- 172.21.15.67:0/39702598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c0d1ca0 0x7f701c0d4a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.661+0000 7f7023200640 1 --2- 172.21.15.67:0/39702598 >> v2:172.21.15.169:3300/0 conn(0x7f701c15e690 0x7f701c0d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.661+0000 7f7023200640 1 -- 172.21.15.67:0/39702598 >> 172.21.15.67:0/39702598 conn(0x7f701c09d910 msgr2=0x7f701c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:20.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.661+0000 7f7023200640 1 -- 172.21.15.67:0/39702598 shutdown_connections 2024-05-30T17:02:20.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.661+0000 7f7023200640 1 -- 172.21.15.67:0/39702598 wait complete. 2024-05-30T17:02:20.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.663+0000 7f7023200640 1 Processor -- start 2024-05-30T17:02:20.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.663+0000 7f7023200640 1 -- start start 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.664+0000 7f7023200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0d1ca0 0x7f701c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.664+0000 7f7023200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c15e690 0x7f701c150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.664+0000 7f7023200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f701c15ea70 0x7f701c150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.664+0000 7f7023200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f701c0d2bc0 con 0x7f701c15ea70 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.664+0000 7f7023200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f701c0d2a40 con 0x7f701c0d1ca0 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.664+0000 7f7023200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f701c0d2d40 con 0x7f701c15e690 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7022800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f701c15ea70 0x7f701c150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7022800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f701c15ea70 0x7f701c150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48120/0 (socket says 172.21.15.67:48120) 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7022800640 1 -- 172.21.15.67:0/1188249911 learned_addr learned my addr 172.21.15.67:0/1188249911 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7021e00640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0d1ca0 0x7f701c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7021400640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c15e690 0x7f701c150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7022800640 1 -- 172.21.15.67:0/1188249911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c15e690 msgr2=0x7f701c150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7022800640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c15e690 0x7f701c150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7022800640 1 -- 172.21.15.67:0/1188249911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0d1ca0 msgr2=0x7f701c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7022800640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0d1ca0 0x7f701c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.665+0000 7f7022800640 1 -- 172.21.15.67:0/1188249911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c0b7ea0 con 0x7f701c15ea70 2024-05-30T17:02:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.666+0000 7f7022800640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f701c15ea70 0x7f701c150a80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f701800c730 tx=0x7f701800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.666+0000 7f7016000640 1 -- 172.21.15.67:0/1188249911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018003da0 con 0x7f701c15ea70 2024-05-30T17:02:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.666+0000 7f7016000640 1 -- 172.21.15.67:0/1188249911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701800ce40 con 0x7f701c15ea70 2024-05-30T17:02:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.666+0000 7f7016000640 1 -- 172.21.15.67:0/1188249911 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70180052b0 con 0x7f701c15ea70 2024-05-30T17:02:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.666+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c0b8180 con 0x7f701c15ea70 2024-05-30T17:02:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.666+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c0b8740 con 0x7f701c15ea70 2024-05-30T17:02:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.667+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c0689e0 con 0x7f701c15ea70 2024-05-30T17:02:20.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.668+0000 7f7016000640 1 -- 172.21.15.67:0/1188249911 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7f7018005450 con 0x7f701c15ea70 2024-05-30T17:02:20.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.670+0000 7f7016000640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f700006dfa0 0x7f7000070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:20.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.670+0000 7f7016000640 1 -- 172.21.15.67:0/1188249911 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f701808a870 con 0x7f701c15ea70 2024-05-30T17:02:20.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.670+0000 7f7021e00640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f700006dfa0 0x7f7000070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:20.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.671+0000 7f7021e00640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f700006dfa0 0x7f7000070460 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7010009a30 tx=0x7f7010008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.675+0000 7f7016000640 1 -- 172.21.15.67:0/1188249911 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f701805a700 con 0x7f701c15ea70 2024-05-30T17:02:20.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.964+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f701c06cbc0 con 0x7f701c15ea70 2024-05-30T17:02:20.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.965+0000 7f7016000640 1 -- 172.21.15.67:0/1188249911 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1730 (secure 0 0 0) 0x7f701805a000 con 0x7f701c15ea70 2024-05-30T17:02:20.966 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:20.967 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:02:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:02:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:02:20.967 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:02:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:02:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:02:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:02:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:02:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:02:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:02:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1134, 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:02:20.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:02:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 104, 2024-05-30T17:02:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:02:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:02:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:02:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:02:20.970 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 797, 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 3222189657, 2024-05-30T17:02:20.971 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 7458709504, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 765635403776, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 65, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 7, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 7, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-05-30T17:02:20.972 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 7902 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:02:20.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:02:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 19, 2024-05-30T17:02:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:01:42.499404+0000", 2024-05-30T17:02:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:02:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:20.974 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": {} 2024-05-30T17:02:20.974 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:02:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f700006dfa0 msgr2=0x7f7000070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f700006dfa0 0x7f7000070460 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7010009a30 tx=0x7f7010008040 comp rx=0 tx=0).stop 2024-05-30T17:02:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f701c15ea70 msgr2=0x7f701c150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f701c15ea70 0x7f701c150a80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f701800c730 tx=0x7f701800cbf0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 shutdown_connections 2024-05-30T17:02:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f700006dfa0 0x7f7000070460 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f701c15ea70 0x7f701c150a80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f701c15e690 0x7f701c150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 --2- 172.21.15.67:0/1188249911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0d1ca0 0x7f701c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:20.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 >> 172.21.15.67:0/1188249911 conn(0x7f701c09d910 msgr2=0x7f701c0d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:20.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 shutdown_connections 2024-05-30T17:02:20.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:20.968+0000 7f7023200640 1 -- 172.21.15.67:0/1188249911 wait complete. 2024-05-30T17:02:20.989 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-30T17:02:21.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.214+0000 7fb64a600640 1 Processor -- start 2024-05-30T17:02:21.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.214+0000 7fb64a600640 1 -- start start 2024-05-30T17:02:21.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.216+0000 7fb64a600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6440ce1f0 0x7fb6440b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.216+0000 7fb64a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb6440ce5d0 0x7fb64415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.216+0000 7fb64a600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6440b3580 con 0x7fb6440b3d60 2024-05-30T17:02:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.216+0000 7fb64a600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb64415f490 con 0x7fb6440ce5d0 2024-05-30T17:02:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.216+0000 7fb64a600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb64415f770 con 0x7fb6440ce1f0 2024-05-30T17:02:21.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.216+0000 7fb643e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb6440ce5d0 0x7fb64415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.216+0000 7fb643e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb6440ce5d0 0x7fb64415ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59750/0 (socket says 172.21.15.67:59750) 2024-05-30T17:02:21.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.216+0000 7fb643e00640 1 -- 172.21.15.67:0/338285606 learned_addr learned my addr 172.21.15.67:0/338285606 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:21.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.217+0000 7fb649200640 1 --2- 172.21.15.67:0/338285606 >> v2:172.21.15.169:3300/0 conn(0x7fb6440ce1f0 0x7fb6440b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.217+0000 7fb643e00640 1 -- 172.21.15.67:0/338285606 >> v2:172.21.15.169:3300/0 conn(0x7fb6440ce1f0 msgr2=0x7fb6440b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:21.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.217+0000 7fb643e00640 1 --2- 172.21.15.67:0/338285606 >> v2:172.21.15.169:3300/0 conn(0x7fb6440ce1f0 0x7fb6440b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.217+0000 7fb643e00640 1 -- 172.21.15.67:0/338285606 >> v1:172.21.15.169:6789/0 conn(0x7fb6440b3d60 legacy=0x7fb6440ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:21.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.217+0000 7fb643e00640 1 -- 172.21.15.67:0/338285606 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64415fa50 con 0x7fb6440ce5d0 2024-05-30T17:02:21.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.217+0000 7fb649200640 1 --2- 172.21.15.67:0/338285606 >> v2:172.21.15.169:3300/0 conn(0x7fb6440ce1f0 0x7fb6440b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:21.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.218+0000 7fb643e00640 1 --2- 172.21.15.67:0/338285606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 0x7fb64415ec40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fb6440cdd00 tx=0x7fb63002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94146d8b422f511 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:21.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.218+0000 7fb643400640 1 -- 172.21.15.67:0/338285606 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630037070 con 0x7fb6440ce5d0 2024-05-30T17:02:21.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.219+0000 7fb643400640 1 -- 172.21.15.67:0/338285606 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63002ae20 con 0x7fb6440ce5d0 2024-05-30T17:02:21.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.219+0000 7fb643400640 1 -- 172.21.15.67:0/338285606 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630032660 con 0x7fb6440ce5d0 2024-05-30T17:02:21.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.219+0000 7fb64a600640 1 -- 172.21.15.67:0/338285606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 msgr2=0x7fb64415ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.219+0000 7fb64a600640 1 --2- 172.21.15.67:0/338285606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 0x7fb64415ec40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fb6440cdd00 tx=0x7fb63002a110 comp rx=0 tx=0).stop 2024-05-30T17:02:21.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.219+0000 7fb64a600640 1 -- 172.21.15.67:0/338285606 shutdown_connections 2024-05-30T17:02:21.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.219+0000 7fb64a600640 1 --2- 172.21.15.67:0/338285606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 0x7fb64415ec40 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.219+0000 7fb64a600640 1 --2- 172.21.15.67:0/338285606 >> v2:172.21.15.169:3300/0 conn(0x7fb6440ce1f0 0x7fb6440b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.219+0000 7fb64a600640 1 -- 172.21.15.67:0/338285606 >> 172.21.15.67:0/338285606 conn(0x7fb64406b900 msgr2=0x7fb64406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:21.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.219+0000 7fb64a600640 1 -- 172.21.15.67:0/338285606 shutdown_connections 2024-05-30T17:02:21.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.220+0000 7fb64a600640 1 -- 172.21.15.67:0/338285606 wait complete. 2024-05-30T17:02:21.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.221+0000 7fb64a600640 1 Processor -- start 2024-05-30T17:02:21.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.221+0000 7fb64a600640 1 -- start start 2024-05-30T17:02:21.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.222+0000 7fb64a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6440b3d60 0x7fb6440b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.222+0000 7fb64a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6440ce1f0 0x7fb6440afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.222+0000 7fb64a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 0x7fb64406e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.222+0000 7fb64a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440cd2a0 con 0x7fb6440ce5d0 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.222+0000 7fb64a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb64415f180 con 0x7fb6440b3d60 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.222+0000 7fb64a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6440cd420 con 0x7fb6440ce1f0 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb649c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 0x7fb64406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb649c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 0x7fb64406e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59752/0 (socket says 172.21.15.67:59752) 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb649c00640 1 -- 172.21.15.67:0/3826687306 learned_addr learned my addr 172.21.15.67:0/3826687306 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb649200640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6440b3d60 0x7fb6440b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb643e00640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6440ce1f0 0x7fb6440afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb649c00640 1 -- 172.21.15.67:0/3826687306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6440ce1f0 msgr2=0x7fb6440afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb649c00640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6440ce1f0 0x7fb6440afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb649c00640 1 -- 172.21.15.67:0/3826687306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6440b3d60 msgr2=0x7fb6440b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb649c00640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6440b3d60 0x7fb6440b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.223+0000 7fb649c00640 1 -- 172.21.15.67:0/3826687306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6440b24b0 con 0x7fb6440ce5d0 2024-05-30T17:02:21.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.224+0000 7fb649c00640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 0x7fb64406e510 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00b990 tx=0x7fb63c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:21.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.224+0000 7fb640c00640 1 -- 172.21.15.67:0/3826687306 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c019070 con 0x7fb6440ce5d0 2024-05-30T17:02:21.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.224+0000 7fb640c00640 1 -- 172.21.15.67:0/3826687306 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63c00cba0 con 0x7fb6440ce5d0 2024-05-30T17:02:21.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.224+0000 7fb640c00640 1 -- 172.21.15.67:0/3826687306 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c014440 con 0x7fb6440ce5d0 2024-05-30T17:02:21.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.224+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6440b05e0 con 0x7fb6440ce5d0 2024-05-30T17:02:21.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.224+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64406dd10 con 0x7fb6440ce5d0 2024-05-30T17:02:21.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.225+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6440689e0 con 0x7fb6440ce5d0 2024-05-30T17:02:21.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.226+0000 7fb640c00640 1 -- 172.21.15.67:0/3826687306 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7fb63c01c070 con 0x7fb6440ce5d0 2024-05-30T17:02:21.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.227+0000 7fb640c00640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb62406df70 0x7fb624070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.228+0000 7fb640c00640 1 -- 172.21.15.67:0/3826687306 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fb63c08a6f0 con 0x7fb6440ce5d0 2024-05-30T17:02:21.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.228+0000 7fb649200640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb62406df70 0x7fb624070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.228+0000 7fb649200640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb62406df70 0x7fb624070430 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb63800a990 tx=0x7fb638008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:21.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.232+0000 7fb640c00640 1 -- 172.21.15.67:0/3826687306 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb63c090e20 con 0x7fb6440ce5d0 2024-05-30T17:02:21.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.467+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6440ca070 con 0x7fb62406df70 2024-05-30T17:02:21.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.477+0000 7fb640c00640 1 -- 172.21.15.67:0/3826687306 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7fb6440ca070 con 0x7fb62406df70 2024-05-30T17:02:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:21.479 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:02:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sv1" 2024-05-30T17:02:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1clone1" 2024-05-30T17:02:21.479 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:21.479 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:02:21.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb62406df70 msgr2=0x7fb624070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb62406df70 0x7fb624070430 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb63800a990 tx=0x7fb638008040 comp rx=0 tx=0).stop 2024-05-30T17:02:21.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 msgr2=0x7fb64406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 0x7fb64406e510 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00b990 tx=0x7fb63c00be50 comp rx=0 tx=0).stop 2024-05-30T17:02:21.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 shutdown_connections 2024-05-30T17:02:21.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb62406df70 0x7fb624070430 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ce5d0 0x7fb64406e510 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6440ce1f0 0x7fb6440afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 --2- 172.21.15.67:0/3826687306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6440b3d60 0x7fb6440b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.480+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 >> 172.21.15.67:0/3826687306 conn(0x7fb64406b900 msgr2=0x7fb64406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.481+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 shutdown_connections 2024-05-30T17:02:21.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.481+0000 7fb64a600640 1 -- 172.21.15.67:0/3826687306 wait complete. 2024-05-30T17:02:21.501 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-30T17:02:21.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.714+0000 7efefb200640 1 Processor -- start 2024-05-30T17:02:21.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.715+0000 7efefb200640 1 -- start start 2024-05-30T17:02:21.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.716+0000 7efefb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efefc15e5a0 0x7efefc0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.716+0000 7efefb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efefc0cfb40 0x7efefc0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.716+0000 7efefb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efefc0cffa0 0x7efefc0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.716+0000 7efefb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7efefc0d2b30 con 0x7efefc0cfb40 2024-05-30T17:02:21.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.716+0000 7efefb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efefc0d3500 con 0x7efefc0cffa0 2024-05-30T17:02:21.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.716+0000 7efefb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efefc0d2200 con 0x7efefc15e5a0 2024-05-30T17:02:21.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.717+0000 7efef9400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efefc0cfb40 0x7efefc0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.717+0000 7efef9400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efefc0cfb40 0x7efefc0d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59756/0 (socket says 172.21.15.67:59756) 2024-05-30T17:02:21.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.717+0000 7efef9400640 1 -- 172.21.15.67:0/2637371734 learned_addr learned my addr 172.21.15.67:0/2637371734 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:21.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.717+0000 7efef9400640 1 -- 172.21.15.67:0/2637371734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efefc15e5a0 msgr2=0x7efefc0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:21.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.717+0000 7efef9400640 1 --2- 172.21.15.67:0/2637371734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efefc15e5a0 0x7efefc0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.717+0000 7efef9400640 1 -- 172.21.15.67:0/2637371734 >> v2:172.21.15.169:3300/0 conn(0x7efefc0cffa0 msgr2=0x7efefc0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:21.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.717+0000 7efefa800640 1 --2- 172.21.15.67:0/2637371734 >> v2:172.21.15.169:3300/0 conn(0x7efefc0cffa0 0x7efefc0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.717+0000 7efef9400640 1 --2- 172.21.15.67:0/2637371734 >> v2:172.21.15.169:3300/0 conn(0x7efefc0cffa0 0x7efefc0d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.717+0000 7efef9400640 1 -- 172.21.15.67:0/2637371734 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc0d24e0 con 0x7efefc0cfb40 2024-05-30T17:02:21.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.719+0000 7efef9400640 1 --2- 172.21.15.67:0/2637371734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 0x7efefc0d0d10 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efefc0d4460 tx=0x7efee802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923fa8e659d727b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:21.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.719+0000 7efef8a00640 1 -- 172.21.15.67:0/2637371734 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8037070 con 0x7efefc0cfb40 2024-05-30T17:02:21.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.719+0000 7efef8a00640 1 -- 172.21.15.67:0/2637371734 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee8032440 con 0x7efefc0cfb40 2024-05-30T17:02:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.719+0000 7efef8a00640 1 -- 172.21.15.67:0/2637371734 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8032810 con 0x7efefc0cfb40 2024-05-30T17:02:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.719+0000 7efefb200640 1 -- 172.21.15.67:0/2637371734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 msgr2=0x7efefc0d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.719+0000 7efefb200640 1 --2- 172.21.15.67:0/2637371734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 0x7efefc0d0d10 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efefc0d4460 tx=0x7efee802a1e0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.720+0000 7efefb200640 1 -- 172.21.15.67:0/2637371734 shutdown_connections 2024-05-30T17:02:21.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.720+0000 7efefb200640 1 --2- 172.21.15.67:0/2637371734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 0x7efefc0d0d10 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.720+0000 7efefb200640 1 --2- 172.21.15.67:0/2637371734 >> v2:172.21.15.169:3300/0 conn(0x7efefc0cffa0 0x7efefc0d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.720+0000 7efefb200640 1 --2- 172.21.15.67:0/2637371734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efefc15e5a0 0x7efefc0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.720+0000 7efefb200640 1 -- 172.21.15.67:0/2637371734 >> 172.21.15.67:0/2637371734 conn(0x7efefc08bfc0 msgr2=0x7efefc0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:21.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.720+0000 7efefb200640 1 -- 172.21.15.67:0/2637371734 shutdown_connections 2024-05-30T17:02:21.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.720+0000 7efefb200640 1 -- 172.21.15.67:0/2637371734 wait complete. 2024-05-30T17:02:21.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.721+0000 7efefb200640 1 Processor -- start 2024-05-30T17:02:21.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.722+0000 7efefb200640 1 -- start start 2024-05-30T17:02:21.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.722+0000 7efefb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 0x7efefc15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.722+0000 7efefb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efefc0cffa0 0x7efefc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efefb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efefc15e5a0 0x7efefc1f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efefb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efefc15eb00 con 0x7efefc0cfb40 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efefb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efefc15e980 con 0x7efefc0cffa0 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efefb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efefc0d5220 con 0x7efefc15e5a0 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 0x7efefc15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 0x7efefc15cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59764/0 (socket says 172.21.15.67:59764) 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9e00640 1 -- 172.21.15.67:0/1633405816 learned_addr learned my addr 172.21.15.67:0/1633405816 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9400640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efefc0cffa0 0x7efefc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9e00640 1 -- 172.21.15.67:0/1633405816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efefc15e5a0 msgr2=0x7efefc1f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9e00640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efefc15e5a0 0x7efefc1f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9e00640 1 -- 172.21.15.67:0/1633405816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efefc0cffa0 msgr2=0x7efefc0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9e00640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efefc0cffa0 0x7efefc0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9e00640 1 -- 172.21.15.67:0/1633405816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc0cb7d0 con 0x7efefc0cfb40 2024-05-30T17:02:21.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.723+0000 7efef9400640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efefc0cffa0 0x7efefc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:21.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.724+0000 7efef9e00640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 0x7efefc15cd80 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b4a0 tx=0x7efeec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:21.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.724+0000 7efef2000640 1 -- 172.21.15.67:0/1633405816 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec014070 con 0x7efefc0cfb40 2024-05-30T17:02:21.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.724+0000 7efef2000640 1 -- 172.21.15.67:0/1633405816 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec010610 con 0x7efefc0cfb40 2024-05-30T17:02:21.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.724+0000 7efef2000640 1 -- 172.21.15.67:0/1633405816 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec010a00 con 0x7efefc0cfb40 2024-05-30T17:02:21.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.724+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc0cbab0 con 0x7efefc0cfb40 2024-05-30T17:02:21.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.724+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc0cc070 con 0x7efefc0cfb40 2024-05-30T17:02:21.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.725+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efefc0cb560 con 0x7efefc0cfb40 2024-05-30T17:02:21.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.726+0000 7efef2000640 1 -- 172.21.15.67:0/1633405816 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7efeec017070 con 0x7efefc0cfb40 2024-05-30T17:02:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.727+0000 7efef2000640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efedc06df70 0x7efedc070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.727+0000 7efef2000640 1 -- 172.21.15.67:0/1633405816 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7efeec08a390 con 0x7efefc0cfb40 2024-05-30T17:02:21.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.728+0000 7efef9400640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efedc06df70 0x7efedc070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:21.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.728+0000 7efef9400640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efedc06df70 0x7efedc070430 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efee8034000 tx=0x7efee802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:21.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.733+0000 7efef2000640 1 -- 172.21.15.67:0/1633405816 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efeec05a220 con 0x7efefc0cfb40 2024-05-30T17:02:21.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.968+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efefc0689e0 con 0x7efedc06df70 2024-05-30T17:02:21.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.991+0000 7efef2000640 1 -- 172.21.15.67:0/1633405816 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7efefc0689e0 con 0x7efedc06df70 2024-05-30T17:02:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:21.992 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:02:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1" 2024-05-30T17:02:21.992 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:21.993 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:02:21.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efedc06df70 msgr2=0x7efedc070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efedc06df70 0x7efedc070430 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efee8034000 tx=0x7efee802e000 comp rx=0 tx=0).stop 2024-05-30T17:02:21.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 msgr2=0x7efefc15cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:21.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 0x7efefc15cd80 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b4a0 tx=0x7efeec00b960 comp rx=0 tx=0).stop 2024-05-30T17:02:21.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 shutdown_connections 2024-05-30T17:02:21.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efedc06df70 0x7efedc070430 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efefc15e5a0 0x7efefc1f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efefc0cffa0 0x7efefc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 --2- 172.21.15.67:0/1633405816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efefc0cfb40 0x7efefc15cd80 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:21.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 >> 172.21.15.67:0/1633405816 conn(0x7efefc08bfc0 msgr2=0x7efefc0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:21.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 shutdown_connections 2024-05-30T17:02:21.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:21.994+0000 7efefb200640 1 -- 172.21.15.67:0/1633405816 wait complete. 2024-05-30T17:02:22.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-30T17:02:22.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.224+0000 7fe2f7200640 1 Processor -- start 2024-05-30T17:02:22.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.224+0000 7fe2f7200640 1 -- start start 2024-05-30T17:02:22.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.226+0000 7fe2f7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe2f806f5a0 0x7fe2f815a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.226+0000 7fe2f7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f80cf200 con 0x7fe2f815e1e0 2024-05-30T17:02:22.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.226+0000 7fe2f7200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f80cf4e0 con 0x7fe2f806fc70 2024-05-30T17:02:22.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.226+0000 7fe2f7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f80cefd0 con 0x7fe2f806f5a0 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.226+0000 7fe2f6800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe2f815e1e0 0x7fe2f8071b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52152/0 (socket says 172.21.15.67:52152) 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.226+0000 7fe2f6800640 1 -- 172.21.15.67:0/3936093214 learned_addr learned my addr 172.21.15.67:0/3936093214 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2efe00640 1 --2- 172.21.15.67:0/3936093214 >> v2:172.21.15.169:3300/0 conn(0x7fe2f806f5a0 0x7fe2f815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993806363 0 0) 0x7fe2f80cf200 con 0x7fe2f815e1e0 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2dc003520 con 0x7fe2f815e1e0 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 402592007 0 0) 0x7fe2dc003520 con 0x7fe2f815e1e0 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 >> v2:172.21.15.169:3300/0 conn(0x7fe2f806f5a0 msgr2=0x7fe2f815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 --2- 172.21.15.67:0/3936093214 >> v2:172.21.15.169:3300/0 conn(0x7fe2f806f5a0 0x7fe2f815a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 >> v1:172.21.15.169:6789/0 conn(0x7fe2f806fc70 legacy=0x7fe2f80d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f80cf7c0 con 0x7fe2f815e1e0 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fe2f00019f0 con 0x7fe2f815e1e0 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 >> v1:172.21.15.67:6789/0 conn(0x7fe2f815e1e0 legacy=0x7fe2f8071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.227+0000 7fe2f5400640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2dc003a00 0x7fe2dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2f5400640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2f5400640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2dc0051c0 0x7fe2dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc00c090 con 0x7fe2dc004800 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc00c240 con 0x7fe2dc003a00 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2dc00c440 con 0x7fe2dc0051c0 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2efe00640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2f5e00640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2dc003a00 0x7fe2dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2efe00640 1 -- 172.21.15.67:0/3936093214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2dc0051c0 msgr2=0x7fe2dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2efe00640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2dc0051c0 0x7fe2dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2efe00640 1 -- 172.21.15.67:0/3936093214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2dc003a00 msgr2=0x7fe2dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2efe00640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2dc003a00 0x7fe2dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2efe00640 1 -- 172.21.15.67:0/3936093214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc003520 con 0x7fe2dc004800 2024-05-30T17:02:22.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2f5e00640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2dc003a00 0x7fe2dc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:22.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.228+0000 7fe2f6800640 1 -- 172.21.15.67:0/3936093214 reap_dead start 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.229+0000 7fe2efe00640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2dc004c80 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009a90 tx=0x7fe2dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb7314eab322a8b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.230+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0036070 con 0x7fe2dc004800 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.230+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e0031400 con 0x7fe2dc004800 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.230+0000 7fe2f5400640 1 -- 172.21.15.67:0/3936093214 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0009690 con 0x7fe2dc004800 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.230+0000 7fe2f7200640 1 -- 172.21.15.67:0/3936093214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 msgr2=0x7fe2dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.230+0000 7fe2f7200640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2dc004c80 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009a90 tx=0x7fe2dc005830 comp rx=0 tx=0).stop 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.230+0000 7fe2f7200640 1 -- 172.21.15.67:0/3936093214 shutdown_connections 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.230+0000 7fe2f7200640 1 --2- 172.21.15.67:0/3936093214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2dc004c80 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.230+0000 7fe2f7200640 1 -- 172.21.15.67:0/3936093214 >> 172.21.15.67:0/3936093214 conn(0x7fe2f806b900 msgr2=0x7fe2f806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.230+0000 7fe2f7200640 1 -- 172.21.15.67:0/3936093214 shutdown_connections 2024-05-30T17:02:22.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.231+0000 7fe2f7200640 1 -- 172.21.15.67:0/3936093214 wait complete. 2024-05-30T17:02:22.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.232+0000 7fe2f7200640 1 Processor -- start 2024-05-30T17:02:22.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.232+0000 7fe2f7200640 1 -- start start 2024-05-30T17:02:22.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.233+0000 7fe2f7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2f80b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.233+0000 7fe2f7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f81e84e0 0x7fe2f81e8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.233+0000 7fe2f7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f81f4d30 0x7fe2f81f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.233+0000 7fe2f7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f806ea60 con 0x7fe2dc004800 2024-05-30T17:02:22.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.233+0000 7fe2f7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f80cfd80 con 0x7fe2f81f4d30 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.233+0000 7fe2f7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f806ebe0 con 0x7fe2f81e84e0 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2f80b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2f80b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59784/0 (socket says 172.21.15.67:59784) 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f5e00640 1 -- 172.21.15.67:0/4004877939 learned_addr learned my addr 172.21.15.67:0/4004877939 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f6800640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f81f4d30 0x7fe2f81f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2efe00640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f81e84e0 0x7fe2f81e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f5e00640 1 -- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f81e84e0 msgr2=0x7fe2f81e8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f5e00640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f81e84e0 0x7fe2f81e8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f5e00640 1 -- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f81f4d30 msgr2=0x7fe2f81f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f5e00640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f81f4d30 0x7fe2f81f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f5e00640 1 -- 172.21.15.67:0/4004877939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f81e8ff0 con 0x7fe2dc004800 2024-05-30T17:02:22.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2efe00640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f81e84e0 0x7fe2f81e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:22.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f6800640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f81f4d30 0x7fe2f81f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:22.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.234+0000 7fe2f5e00640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2f80b0670 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800d480 tx=0x7fe2e800d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:22.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.235+0000 7fe2ee000640 1 -- 172.21.15.67:0/4004877939 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8005480 con 0x7fe2dc004800 2024-05-30T17:02:22.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.235+0000 7fe2ee000640 1 -- 172.21.15.67:0/4004877939 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e8017070 con 0x7fe2dc004800 2024-05-30T17:02:22.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.235+0000 7fe2ee000640 1 -- 172.21.15.67:0/4004877939 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8003830 con 0x7fe2dc004800 2024-05-30T17:02:22.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.235+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f80cb930 con 0x7fe2dc004800 2024-05-30T17:02:22.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.235+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f80cbef0 con 0x7fe2dc004800 2024-05-30T17:02:22.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.236+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f80689e0 con 0x7fe2dc004800 2024-05-30T17:02:22.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.237+0000 7fe2ee000640 1 -- 172.21.15.67:0/4004877939 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7fe2e80039d0 con 0x7fe2dc004800 2024-05-30T17:02:22.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.238+0000 7fe2ee000640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406dfa0 0x7fe2d4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.238+0000 7fe2ee000640 1 -- 172.21.15.67:0/4004877939 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fe2e808a720 con 0x7fe2dc004800 2024-05-30T17:02:22.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.239+0000 7fe2efe00640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406dfa0 0x7fe2d4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.239+0000 7fe2efe00640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406dfa0 0x7fe2d4070460 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0033000 tx=0x7fe2e0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:22.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.243+0000 7fe2ee000640 1 -- 172.21.15.67:0/4004877939 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2e805a020 con 0x7fe2dc004800 2024-05-30T17:02:22.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.486+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2f81e92d0 con 0x7fe2d406dfa0 2024-05-30T17:02:22.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.545+0000 7fe2ee000640 1 -- 172.21.15.67:0/4004877939 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe2f81e92d0 con 0x7fe2d406dfa0 2024-05-30T17:02:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406dfa0 msgr2=0x7fe2d4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406dfa0 0x7fe2d4070460 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0033000 tx=0x7fe2e0008030 comp rx=0 tx=0).stop 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 msgr2=0x7fe2f80b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2f80b0670 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800d480 tx=0x7fe2e800d940 comp rx=0 tx=0).stop 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 shutdown_connections 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406dfa0 0x7fe2d4070460 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f81f4d30 0x7fe2f81f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f81e84e0 0x7fe2f81e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 --2- 172.21.15.67:0/4004877939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2dc004800 0x7fe2f80b0670 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 >> 172.21.15.67:0/4004877939 conn(0x7fe2f806b900 msgr2=0x7fe2f80b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 shutdown_connections 2024-05-30T17:02:22.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.548+0000 7fe2f7200640 1 -- 172.21.15.67:0/4004877939 wait complete. 2024-05-30T17:02:22.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-30T17:02:22.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.769+0000 7fd18b200640 1 Processor -- start 2024-05-30T17:02:22.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.769+0000 7fd18b200640 1 -- start start 2024-05-30T17:02:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.770+0000 7fd18b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd18c153510 0x7fd18c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.770+0000 7fd18b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c0ccb90 0x7fd18c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.770+0000 7fd18b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd18c15d900 con 0x7fd18c15cb60 2024-05-30T17:02:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.771+0000 7fd18b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd18c1538f0 con 0x7fd18c153510 2024-05-30T17:02:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.771+0000 7fd18b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd18c0cd530 con 0x7fd18c0ccb90 2024-05-30T17:02:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.771+0000 7fd18a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd18c15cb60 0x7fd18c15cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52160/0 (socket says 172.21.15.67:52160) 2024-05-30T17:02:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.771+0000 7fd18a800640 1 -- 172.21.15.67:0/884666302 learned_addr learned my addr 172.21.15.67:0/884666302 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.771+0000 7fd181400640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c0ccb90 0x7fd18c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.771+0000 7fd189e00640 1 --2- 172.21.15.67:0/884666302 >> v2:172.21.15.169:3300/0 conn(0x7fd18c153510 0x7fd18c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.771+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3722061600 0 0) 0x7fd18c15d900 con 0x7fd18c15cb60 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.771+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd170003520 con 0x7fd18c15cb60 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4012364195 0 0) 0x7fd170003520 con 0x7fd18c15cb60 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c0ccb90 msgr2=0x7fd18c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c0ccb90 0x7fd18c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 >> v2:172.21.15.169:3300/0 conn(0x7fd18c153510 msgr2=0x7fd18c0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 --2- 172.21.15.67:0/884666302 >> v2:172.21.15.169:3300/0 conn(0x7fd18c153510 0x7fd18c0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c0b3f90 con 0x7fd18c15cb60 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fd1840019f0 con 0x7fd18c15cb60 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189e00640 1 --2- 172.21.15.67:0/884666302 >> v2:172.21.15.169:3300/0 conn(0x7fd18c153510 0x7fd18c0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:02:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 >> v1:172.21.15.67:6789/0 conn(0x7fd18c15cb60 legacy=0x7fd18c15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c0ccb90 0x7fd1700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd170003f30 0x7fd170004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170004d40 0x7fd1700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd17000fe80 con 0x7fd170003f30 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd170010000 con 0x7fd170004d40 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd1700101d0 con 0x7fd18c0ccb90 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.772+0000 7fd181400640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd170003f30 0x7fd170004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.773+0000 7fd18a800640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170004d40 0x7fd1700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.773+0000 7fd189e00640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c0ccb90 0x7fd1700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.773+0000 7fd181400640 1 -- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c0ccb90 msgr2=0x7fd1700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.773+0000 7fd181400640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c0ccb90 0x7fd1700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.773+0000 7fd181400640 1 -- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170004d40 msgr2=0x7fd1700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.773+0000 7fd181400640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170004d40 0x7fd1700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.773+0000 7fd181400640 1 -- 172.21.15.67:0/884666302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd170003520 con 0x7fd170003f30 2024-05-30T17:02:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.773+0000 7fd189e00640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c0ccb90 0x7fd1700039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.774+0000 7fd181400640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd170003f30 0x7fd170004800 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd174009af0 tx=0x7fd170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f184ec6ed5802aa server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.774+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174036070 con 0x7fd170003f30 2024-05-30T17:02:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.774+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd174031400 con 0x7fd170003f30 2024-05-30T17:02:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.774+0000 7fd189400640 1 -- 172.21.15.67:0/884666302 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174009690 con 0x7fd170003f30 2024-05-30T17:02:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.775+0000 7fd18b200640 1 -- 172.21.15.67:0/884666302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd170003f30 msgr2=0x7fd170004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.775+0000 7fd18b200640 1 --2- 172.21.15.67:0/884666302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd170003f30 0x7fd170004800 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd174009af0 tx=0x7fd170005830 comp rx=0 tx=0).stop 2024-05-30T17:02:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.775+0000 7fd18a800640 1 -- 172.21.15.67:0/884666302 reap_dead start 2024-05-30T17:02:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.775+0000 7fd18b200640 1 -- 172.21.15.67:0/884666302 shutdown_connections 2024-05-30T17:02:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.775+0000 7fd18b200640 1 -- 172.21.15.67:0/884666302 >> 172.21.15.67:0/884666302 conn(0x7fd18c06b900 msgr2=0x7fd18c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.775+0000 7fd18b200640 1 -- 172.21.15.67:0/884666302 shutdown_connections 2024-05-30T17:02:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.775+0000 7fd18b200640 1 -- 172.21.15.67:0/884666302 wait complete. 2024-05-30T17:02:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.777+0000 7fd18b200640 1 Processor -- start 2024-05-30T17:02:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.777+0000 7fd18b200640 1 -- start start 2024-05-30T17:02:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.778+0000 7fd18b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170003f30 0x7fd18c0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.778+0000 7fd18b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c072ee0 0x7fd18c150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.778+0000 7fd18b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c073360 0x7fd18c14ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.778+0000 7fd18b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c0b4550 con 0x7fd18c072ee0 2024-05-30T17:02:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.778+0000 7fd18b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c0b48c0 con 0x7fd170003f30 2024-05-30T17:02:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.778+0000 7fd18b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd18c0ccff0 con 0x7fd18c073360 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.778+0000 7fd181400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c072ee0 0x7fd18c150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.778+0000 7fd181400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c072ee0 0x7fd18c150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59802/0 (socket says 172.21.15.67:59802) 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.778+0000 7fd181400640 1 -- 172.21.15.67:0/1765673327 learned_addr learned my addr 172.21.15.67:0/1765673327 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd189e00640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170003f30 0x7fd18c0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd18a800640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c073360 0x7fd18c14ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd181400640 1 -- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c073360 msgr2=0x7fd18c14ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd181400640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c073360 0x7fd18c14ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd181400640 1 -- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170003f30 msgr2=0x7fd18c0ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd181400640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170003f30 0x7fd18c0ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd181400640 1 -- 172.21.15.67:0/1765673327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c0ad340 con 0x7fd18c072ee0 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd189e00640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170003f30 0x7fd18c0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd18a800640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c073360 0x7fd18c14ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.779+0000 7fd181400640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c072ee0 0x7fd18c150680 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fd174034040 tx=0x7fd174004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.780+0000 7fd182a00640 1 -- 172.21.15.67:0/1765673327 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174040020 con 0x7fd18c072ee0 2024-05-30T17:02:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.780+0000 7fd182a00640 1 -- 172.21.15.67:0/1765673327 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd174003c60 con 0x7fd18c072ee0 2024-05-30T17:02:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.780+0000 7fd182a00640 1 -- 172.21.15.67:0/1765673327 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174036040 con 0x7fd18c072ee0 2024-05-30T17:02:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.780+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c0ad620 con 0x7fd18c072ee0 2024-05-30T17:02:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.780+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c0adbe0 con 0x7fd18c072ee0 2024-05-30T17:02:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.781+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18c0689e0 con 0x7fd18c072ee0 2024-05-30T17:02:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.782+0000 7fd182a00640 1 -- 172.21.15.67:0/1765673327 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7fd174043020 con 0x7fd18c072ee0 2024-05-30T17:02:22.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.783+0000 7fd182a00640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd16806df70 0x7fd168070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:22.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.783+0000 7fd182a00640 1 -- 172.21.15.67:0/1765673327 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fd1740aa9c0 con 0x7fd18c072ee0 2024-05-30T17:02:22.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.783+0000 7fd189e00640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd16806df70 0x7fd168070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:22.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.784+0000 7fd189e00640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd16806df70 0x7fd168070430 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd17c006e80 tx=0x7fd17c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:22.788+0000 7fd182a00640 1 -- 172.21.15.67:0/1765673327 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd17404a070 con 0x7fd18c072ee0 2024-05-30T17:02:23.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.023+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd18c0b3c40 con 0x7fd16806df70 2024-05-30T17:02:23.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.063+0000 7fd182a00640 1 -- 172.21.15.67:0/1765673327 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd18c0b3c40 con 0x7fd16806df70 2024-05-30T17:02:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd16806df70 msgr2=0x7fd168070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd16806df70 0x7fd168070430 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd17c006e80 tx=0x7fd17c008040 comp rx=0 tx=0).stop 2024-05-30T17:02:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c072ee0 msgr2=0x7fd18c150680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c072ee0 0x7fd18c150680 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fd174034040 tx=0x7fd174004730 comp rx=0 tx=0).stop 2024-05-30T17:02:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 shutdown_connections 2024-05-30T17:02:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd16806df70 0x7fd168070430 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd18c073360 0x7fd18c14ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c072ee0 0x7fd18c150680 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 --2- 172.21.15.67:0/1765673327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd170003f30 0x7fd18c0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 >> 172.21.15.67:0/1765673327 conn(0x7fd18c06b900 msgr2=0x7fd18c1526e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 shutdown_connections 2024-05-30T17:02:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.066+0000 7fd18b200640 1 -- 172.21.15.67:0/1765673327 wait complete. 2024-05-30T17:02:23.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.070+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.070+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.070+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.070+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.070 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.070+0000 7f0c8d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.090 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-30T17:02:23.096 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.092+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.096 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.092+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.096 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.092+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.096 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.092+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.096 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:23.092+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:23.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.283+0000 7fc7fdc00640 1 Processor -- start 2024-05-30T17:02:23.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.283+0000 7fc7fdc00640 1 -- start start 2024-05-30T17:02:23.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.285+0000 7fc7fdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80cfb40 0x7fc7f80d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:23.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.285+0000 7fc7fdc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f80d2200 con 0x7fc7f80cffa0 2024-05-30T17:02:23.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.285+0000 7fc7fdc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f80d24e0 con 0x7fc7f815e5a0 2024-05-30T17:02:23.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.285+0000 7fc7fdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f80d2b30 con 0x7fc7f80cfb40 2024-05-30T17:02:23.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.285+0000 7fc7fd200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc7f80cffa0 0x7fc7f806c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52166/0 (socket says 172.21.15.67:52166) 2024-05-30T17:02:23.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.285+0000 7fc7fd200640 1 -- 172.21.15.67:0/2609187504 learned_addr learned my addr 172.21.15.67:0/2609187504 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7efe00640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80cfb40 0x7fc7f80d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3647711433 0 0) 0x7fc7f80d2200 con 0x7fc7f80cffa0 2024-05-30T17:02:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7dc003520 con 0x7fc7f80cffa0 2024-05-30T17:02:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4116544951 0 0) 0x7fc7dc003520 con 0x7fc7f80cffa0 2024-05-30T17:02:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80cfb40 msgr2=0x7fc7f80d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80cfb40 0x7fc7f80d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 >> v1:172.21.15.169:6789/0 conn(0x7fc7f815e5a0 legacy=0x7fc7f80d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f80d27c0 con 0x7fc7f80cffa0 2024-05-30T17:02:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fc7f00019f0 con 0x7fc7f80cffa0 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 >> v1:172.21.15.67:6789/0 conn(0x7fc7f80cffa0 legacy=0x7fc7f806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7dc004800 0x7fc7dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80cfb40 0x7fc7dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc00ba70 con 0x7fc7dc0039e0 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc00bbf0 con 0x7fc7dc004800 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.286+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc7dc00bdf0 con 0x7fc7f80cfb40 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.287+0000 7fc7f7e00640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.287+0000 7fc7efe00640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7dc004800 0x7fc7dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.287+0000 7fc7fd200640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80cfb40 0x7fc7dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.287+0000 7fc7f7e00640 1 -- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80cfb40 msgr2=0x7fc7dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.287+0000 7fc7f7e00640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80cfb40 0x7fc7dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.287+0000 7fc7f7e00640 1 -- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7dc004800 msgr2=0x7fc7dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.287+0000 7fc7f7e00640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7dc004800 0x7fc7dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.287+0000 7fc7f7e00640 1 -- 172.21.15.67:0/2609187504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc003520 con 0x7fc7dc0039e0 2024-05-30T17:02:23.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.287+0000 7fc7fd200640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80cfb40 0x7fc7dc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.288+0000 7fc7f7e00640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7dc003e00 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8001720 tx=0x7fc7dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea53ada10e0d5609 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.289+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8001370 con 0x7fc7dc0039e0 2024-05-30T17:02:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.289+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e80052d0 con 0x7fc7dc0039e0 2024-05-30T17:02:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.289+0000 7fc7f7400640 1 -- 172.21.15.67:0/2609187504 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e802aba0 con 0x7fc7dc0039e0 2024-05-30T17:02:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.289+0000 7fc7fdc00640 1 -- 172.21.15.67:0/2609187504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 msgr2=0x7fc7dc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.289+0000 7fc7fdc00640 1 --2- 172.21.15.67:0/2609187504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7dc003e00 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8001720 tx=0x7fc7dc005830 comp rx=0 tx=0).stop 2024-05-30T17:02:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.289+0000 7fc7fd200640 1 -- 172.21.15.67:0/2609187504 reap_dead start 2024-05-30T17:02:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.289+0000 7fc7fdc00640 1 -- 172.21.15.67:0/2609187504 shutdown_connections 2024-05-30T17:02:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.289+0000 7fc7fdc00640 1 -- 172.21.15.67:0/2609187504 >> 172.21.15.67:0/2609187504 conn(0x7fc7f808bfc0 msgr2=0x7fc7f80d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:23.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.290+0000 7fc7fdc00640 1 -- 172.21.15.67:0/2609187504 shutdown_connections 2024-05-30T17:02:23.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.290+0000 7fc7fdc00640 1 -- 172.21.15.67:0/2609187504 wait complete. 2024-05-30T17:02:23.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.291+0000 7fc7fdc00640 1 Processor -- start 2024-05-30T17:02:23.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.292+0000 7fc7fdc00640 1 -- start start 2024-05-30T17:02:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.293+0000 7fc7fdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7f80c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.293+0000 7fc7fdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80b62f0 0x7fc7f8155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.293+0000 7fc7fdc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f80b3f10 0x7fc7f81605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.293+0000 7fc7fdc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80d53a0 con 0x7fc7dc0039e0 2024-05-30T17:02:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.293+0000 7fc7fdc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80d5220 con 0x7fc7f80b3f10 2024-05-30T17:02:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.293+0000 7fc7fdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f806ce50 con 0x7fc7f80b62f0 2024-05-30T17:02:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.293+0000 7fc7f7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7f80c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.293+0000 7fc7f7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7f80c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59846/0 (socket says 172.21.15.67:59846) 2024-05-30T17:02:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.293+0000 7fc7f7e00640 1 -- 172.21.15.67:0/3301733337 learned_addr learned my addr 172.21.15.67:0/3301733337 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:23.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7f7e00640 1 -- 172.21.15.67:0/3301733337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80b62f0 msgr2=0x7fc7f8155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:23.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7f7e00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80b62f0 0x7fc7f8155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7f7e00640 1 -- 172.21.15.67:0/3301733337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f80b3f10 msgr2=0x7fc7f81605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:23.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7f7e00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f80b3f10 0x7fc7f81605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7f7e00640 1 -- 172.21.15.67:0/3301733337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f8156250 con 0x7fc7dc0039e0 2024-05-30T17:02:23.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7f7e00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7f80c92a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8008fd0 tx=0x7fc7e8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:23.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7f4c00640 1 -- 172.21.15.67:0/3301733337 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8042070 con 0x7fc7dc0039e0 2024-05-30T17:02:23.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7f4c00640 1 -- 172.21.15.67:0/3301733337 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e8005480 con 0x7fc7dc0039e0 2024-05-30T17:02:23.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7f4c00640 1 -- 172.21.15.67:0/3301733337 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8037040 con 0x7fc7dc0039e0 2024-05-30T17:02:23.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f80b6770 con 0x7fc7dc0039e0 2024-05-30T17:02:23.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.294+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f80ccc90 con 0x7fc7dc0039e0 2024-05-30T17:02:23.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.296+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f80689e0 con 0x7fc7dc0039e0 2024-05-30T17:02:23.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.297+0000 7fc7f4c00640 1 -- 172.21.15.67:0/3301733337 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7fc7e8045070 con 0x7fc7dc0039e0 2024-05-30T17:02:23.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.298+0000 7fc7f4c00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7d406dfa0 0x7fc7d4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:23.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.298+0000 7fc7f4c00640 1 -- 172.21.15.67:0/3301733337 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fc7e80aa0d0 con 0x7fc7dc0039e0 2024-05-30T17:02:23.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.298+0000 7fc7efe00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7d406dfa0 0x7fc7d4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:23.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.299+0000 7fc7efe00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7d406dfa0 0x7fc7d4070460 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80cc7a0 tx=0x7fc7e0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:23.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.306+0000 7fc7f4c00640 1 -- 172.21.15.67:0/3301733337 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7e8079f60 con 0x7fc7dc0039e0 2024-05-30T17:02:23.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.540+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fc7f806c8e0 con 0x7fc7d406dfa0 2024-05-30T17:02:23.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.575+0000 7fc7f4c00640 1 -- 172.21.15.67:0/3301733337 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc7f806c8e0 con 0x7fc7d406dfa0 2024-05-30T17:02:23.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7d406dfa0 msgr2=0x7fc7d4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:23.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7d406dfa0 0x7fc7d4070460 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80cc7a0 tx=0x7fc7e0008040 comp rx=0 tx=0).stop 2024-05-30T17:02:23.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 msgr2=0x7fc7f80c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:23.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7f80c92a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8008fd0 tx=0x7fc7e8001120 comp rx=0 tx=0).stop 2024-05-30T17:02:23.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 shutdown_connections 2024-05-30T17:02:23.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc7d406dfa0 0x7fc7d4070460 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f80b3f10 0x7fc7f81605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f80b62f0 0x7fc7f8155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 --2- 172.21.15.67:0/3301733337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc0039e0 0x7fc7f80c92a0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:23.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.579+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 >> 172.21.15.67:0/3301733337 conn(0x7fc7f808bfc0 msgr2=0x7fc7f80d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:23.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.580+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 shutdown_connections 2024-05-30T17:02:23.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23.580+0000 7fc7fdc00640 1 -- 172.21.15.67:0/3301733337 wait complete. 2024-05-30T17:02:23.601 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-30T17:02:23.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-30T17:02:23.705 INFO:teuthology.orchestra.run.smithi067.stdout:2 2024-05-30T17:02:23.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T17:02:23.714 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T17:02:23.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:02:23.807 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:02:23.917 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T17:02:23.917 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:02:23.918 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:02:23.918 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:23.977 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:02:23.977 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:24.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T17:02:24.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.250+0000 7fa9e8a00640 1 Processor -- start 2024-05-30T17:02:24.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.250+0000 7fa9e8a00640 1 -- start start 2024-05-30T17:02:24.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.251+0000 7fa9e8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9e415b3a0 0x7fa9e40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:24.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406dc20 0x7fa9e415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:24.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e8a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e40afd60 con 0x7fa9e4072c00 2024-05-30T17:02:24.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e8a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e406f980 con 0x7fa9e415b3a0 2024-05-30T17:02:24.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e40ae0c0 con 0x7fa9e406dc20 2024-05-30T17:02:24.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9e415b3a0 0x7fa9e40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:24.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e3200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa9e4072c00 0x7fa9e40ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52172/0 (socket says 172.21.15.67:52172) 2024-05-30T17:02:24.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9e415b3a0 0x7fa9e40b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59862/0 (socket says 172.21.15.67:59862) 2024-05-30T17:02:24.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e3200640 1 -- 172.21.15.67:0/6167298 learned_addr learned my addr 172.21.15.67:0/6167298 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9dbe00640 1 --2- 172.21.15.67:0/6167298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406dc20 0x7fa9e415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e2800640 1 -- 172.21.15.67:0/6167298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406dc20 msgr2=0x7fa9e415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e2800640 1 --2- 172.21.15.67:0/6167298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406dc20 0x7fa9e415f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e2800640 1 -- 172.21.15.67:0/6167298 >> v1:172.21.15.67:6789/0 conn(0x7fa9e4072c00 legacy=0x7fa9e40ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9e2800640 1 -- 172.21.15.67:0/6167298 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e40af7a0 con 0x7fa9e415b3a0 2024-05-30T17:02:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.252+0000 7fa9dbe00640 1 --2- 172.21.15.67:0/6167298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406dc20 0x7fa9e415f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.254+0000 7fa9e2800640 1 --2- 172.21.15.67:0/6167298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e415b3a0 0x7fa9e40b32c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097a0 tx=0x7fa9d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df317c7f5706b0c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:24.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.254+0000 7fa9e1e00640 1 -- 172.21.15.67:0/6167298 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4037070 con 0x7fa9e415b3a0 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.254+0000 7fa9e1e00640 1 -- 172.21.15.67:0/6167298 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d4032440 con 0x7fa9e415b3a0 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.254+0000 7fa9e1e00640 1 -- 172.21.15.67:0/6167298 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4032810 con 0x7fa9e415b3a0 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.254+0000 7fa9e8a00640 1 -- 172.21.15.67:0/6167298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e415b3a0 msgr2=0x7fa9e40b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.254+0000 7fa9e8a00640 1 --2- 172.21.15.67:0/6167298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e415b3a0 0x7fa9e40b32c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097a0 tx=0x7fa9d402a180 comp rx=0 tx=0).stop 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.255+0000 7fa9e8a00640 1 -- 172.21.15.67:0/6167298 shutdown_connections 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.255+0000 7fa9e8a00640 1 --2- 172.21.15.67:0/6167298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e406dc20 0x7fa9e415f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.255+0000 7fa9e8a00640 1 --2- 172.21.15.67:0/6167298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e415b3a0 0x7fa9e40b32c0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.255+0000 7fa9e8a00640 1 -- 172.21.15.67:0/6167298 >> 172.21.15.67:0/6167298 conn(0x7fa9e40b1fc0 msgr2=0x7fa9e40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.255+0000 7fa9e8a00640 1 -- 172.21.15.67:0/6167298 shutdown_connections 2024-05-30T17:02:24.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.255+0000 7fa9e8a00640 1 -- 172.21.15.67:0/6167298 wait complete. 2024-05-30T17:02:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.256+0000 7fa9e8a00640 1 Processor -- start 2024-05-30T17:02:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.257+0000 7fa9e8a00640 1 -- start start 2024-05-30T17:02:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.257+0000 7fa9e8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9e406dc20 0x7fa9e4097840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.257+0000 7fa9e8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e4072c00 0x7fa9e40cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e415b3a0 0x7fa9e40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9e8a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e415b780 con 0x7fa9e4072c00 2024-05-30T17:02:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9e8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e415fee0 con 0x7fa9e406dc20 2024-05-30T17:02:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e4073050 con 0x7fa9e415b3a0 2024-05-30T17:02:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9dbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e4072c00 0x7fa9e40cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9dbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e4072c00 0x7fa9e40cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59874/0 (socket says 172.21.15.67:59874) 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9dbe00640 1 -- 172.21.15.67:0/889780170 learned_addr learned my addr 172.21.15.67:0/889780170 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9e2800640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9e406dc20 0x7fa9e4097840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9e3200640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e415b3a0 0x7fa9e40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9dbe00640 1 -- 172.21.15.67:0/889780170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e415b3a0 msgr2=0x7fa9e40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9dbe00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e415b3a0 0x7fa9e40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9dbe00640 1 -- 172.21.15.67:0/889780170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9e406dc20 msgr2=0x7fa9e4097840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9dbe00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9e406dc20 0x7fa9e4097840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.258+0000 7fa9dbe00640 1 -- 172.21.15.67:0/889780170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e40ccfd0 con 0x7fa9e4072c00 2024-05-30T17:02:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.259+0000 7fa9dbe00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e4072c00 0x7fa9e40cbd50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c820 tx=0x7fa9cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:24.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.259+0000 7fa9daa00640 1 -- 172.21.15.67:0/889780170 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc015070 con 0x7fa9e4072c00 2024-05-30T17:02:24.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.259+0000 7fa9daa00640 1 -- 172.21.15.67:0/889780170 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0079d0 con 0x7fa9e4072c00 2024-05-30T17:02:24.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.259+0000 7fa9daa00640 1 -- 172.21.15.67:0/889780170 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc007da0 con 0x7fa9e4072c00 2024-05-30T17:02:24.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.259+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e40cf610 con 0x7fa9e4072c00 2024-05-30T17:02:24.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.259+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e414fd80 con 0x7fa9e4072c00 2024-05-30T17:02:24.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.260+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e40689e0 con 0x7fa9e4072c00 2024-05-30T17:02:24.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.261+0000 7fa9daa00640 1 -- 172.21.15.67:0/889780170 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 37) ==== 93080+0+0 (secure 0 0 0) 0x7fa9cc018070 con 0x7fa9e4072c00 2024-05-30T17:02:24.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.262+0000 7fa9daa00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c006df70 0x7fa9c0070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:24.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.262+0000 7fa9daa00640 1 -- 172.21.15.67:0/889780170 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fa9cc08a870 con 0x7fa9e4072c00 2024-05-30T17:02:24.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.263+0000 7fa9e2800640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c006df70 0x7fa9c0070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:24.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.263+0000 7fa9e2800640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c006df70 0x7fa9c0070430 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4034000 tx=0x7fa9d402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:24.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.267+0000 7fa9daa00640 1 -- 172.21.15.67:0/889780170 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9cc094020 con 0x7fa9e4072c00 2024-05-30T17:02:24.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.485+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa9e40b2e90 con 0x7fa9e4072c00 2024-05-30T17:02:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.486+0000 7fa9daa00640 1 -- 172.21.15.67:0/889780170 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v104) ==== 80+0+0 (secure 0 0 0) 0x7fa9e40ccfd0 con 0x7fa9e4072c00 2024-05-30T17:02:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T17:02:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.489+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c006df70 msgr2=0x7fa9c0070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.489+0000 7fa9e8a00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c006df70 0x7fa9c0070430 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4034000 tx=0x7fa9d402e000 comp rx=0 tx=0).stop 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.489+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e4072c00 msgr2=0x7fa9e40cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.489+0000 7fa9e8a00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e4072c00 0x7fa9e40cbd50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c820 tx=0x7fa9cc00cc20 comp rx=0 tx=0).stop 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.489+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 shutdown_connections 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.490+0000 7fa9e8a00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa9c006df70 0x7fa9c0070430 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.490+0000 7fa9e8a00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9e415b3a0 0x7fa9e40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.490+0000 7fa9e8a00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e4072c00 0x7fa9e40cbd50 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.490+0000 7fa9e8a00640 1 --2- 172.21.15.67:0/889780170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9e406dc20 0x7fa9e4097840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.490+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 >> 172.21.15.67:0/889780170 conn(0x7fa9e40b1fc0 msgr2=0x7fa9e406e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.490+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 shutdown_connections 2024-05-30T17:02:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:24.490+0000 7fa9e8a00640 1 -- 172.21.15.67:0/889780170 wait complete. 2024-05-30T17:02:24.511 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T17:02:24.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:24.511 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-30T17:02:24.597 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T17:02:24.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:24.597 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-30T17:02:24.604 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[70921]: fuse finished with error 0 and tester_r 0 2024-05-30T17:02:24.795 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:02:24.795 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:02:24.795 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:24.852 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:02:24.852 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:02:24.853 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:02:24.853 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:24.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:02:24.907 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:02:24.907 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:02:24.907 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:24.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:02:24.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:02:25.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.185+0000 7f4ed3e00640 1 Processor -- start 2024-05-30T17:02:25.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.185+0000 7f4ed3e00640 1 -- start start 2024-05-30T17:02:25.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.187+0000 7f4ed3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ed4071bc0 0x7f4ed4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.187+0000 7f4ed3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed406e6e0 con 0x7f4ed4158b80 2024-05-30T17:02:25.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.187+0000 7f4ed3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed406e9c0 con 0x7f4ed40b32c0 2024-05-30T17:02:25.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.187+0000 7f4ed3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed406eca0 con 0x7f4ed4071bc0 2024-05-30T17:02:25.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.187+0000 7f4ed2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ed4071bc0 0x7f4ed4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.187+0000 7f4ed2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ed4071bc0 0x7f4ed4072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59882/0 (socket says 172.21.15.67:59882) 2024-05-30T17:02:25.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.187+0000 7f4ed2000640 1 -- 172.21.15.67:0/3294301833 learned_addr learned my addr 172.21.15.67:0/3294301833 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:25.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.188+0000 7f4ed2000640 1 -- 172.21.15.67:0/3294301833 >> v1:172.21.15.169:6789/0 conn(0x7f4ed40b32c0 legacy=0x7f4ed40b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:25.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.188+0000 7f4ed2000640 1 -- 172.21.15.67:0/3294301833 >> v1:172.21.15.67:6789/0 conn(0x7f4ed4158b80 legacy=0x7f4ed406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.188+0000 7f4ed2000640 1 -- 172.21.15.67:0/3294301833 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed406ef80 con 0x7f4ed4071bc0 2024-05-30T17:02:25.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.189+0000 7f4ed2000640 1 --2- 172.21.15.67:0/3294301833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed4071bc0 0x7f4ed4072070 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009870 tx=0x7f4ec002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec6ea406f5f7bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:25.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.189+0000 7f4ed1600640 1 -- 172.21.15.67:0/3294301833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0037070 con 0x7f4ed4071bc0 2024-05-30T17:02:25.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.189+0000 7f4ed1600640 1 -- 172.21.15.67:0/3294301833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0032440 con 0x7f4ed4071bc0 2024-05-30T17:02:25.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.189+0000 7f4ed1600640 1 -- 172.21.15.67:0/3294301833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0032810 con 0x7f4ed4071bc0 2024-05-30T17:02:25.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.190+0000 7f4ed3e00640 1 -- 172.21.15.67:0/3294301833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed4071bc0 msgr2=0x7f4ed4072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.190+0000 7f4ed3e00640 1 --2- 172.21.15.67:0/3294301833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed4071bc0 0x7f4ed4072070 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009870 tx=0x7f4ec002a2a0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.190+0000 7f4ed3e00640 1 -- 172.21.15.67:0/3294301833 shutdown_connections 2024-05-30T17:02:25.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.190+0000 7f4ed3e00640 1 --2- 172.21.15.67:0/3294301833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed4071bc0 0x7f4ed4072070 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.190+0000 7f4ed3e00640 1 -- 172.21.15.67:0/3294301833 >> 172.21.15.67:0/3294301833 conn(0x7f4ed40b1fc0 msgr2=0x7f4ed40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:25.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.190+0000 7f4ed3e00640 1 -- 172.21.15.67:0/3294301833 shutdown_connections 2024-05-30T17:02:25.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.190+0000 7f4ed3e00640 1 -- 172.21.15.67:0/3294301833 wait complete. 2024-05-30T17:02:25.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.192+0000 7f4ed3e00640 1 Processor -- start 2024-05-30T17:02:25.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.192+0000 7f4ed3e00640 1 -- start start 2024-05-30T17:02:25.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.193+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed4071bc0 0x7f4ed4150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.193+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed40b32c0 0x7f4ed414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.193+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed4158b80 0x7f4ed414b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.193+0000 7f4ed3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed406f780 con 0x7f4ed40b32c0 2024-05-30T17:02:25.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.193+0000 7f4ed3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed40b2d30 con 0x7f4ed4158b80 2024-05-30T17:02:25.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.193+0000 7f4ed3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed414bd60 con 0x7f4ed4071bc0 2024-05-30T17:02:25.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.193+0000 7f4ed2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed40b32c0 0x7f4ed414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.193+0000 7f4ed2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed40b32c0 0x7f4ed414b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59892/0 (socket says 172.21.15.67:59892) 2024-05-30T17:02:25.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.193+0000 7f4ed2000640 1 -- 172.21.15.67:0/854744901 learned_addr learned my addr 172.21.15.67:0/854744901 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:25.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ed2000640 1 -- 172.21.15.67:0/854744901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed4071bc0 msgr2=0x7f4ed4150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ed2000640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed4071bc0 0x7f4ed4150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ed2000640 1 -- 172.21.15.67:0/854744901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed4158b80 msgr2=0x7f4ed414b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ed2000640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed4158b80 0x7f4ed414b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ed2000640 1 -- 172.21.15.67:0/854744901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed409bbe0 con 0x7f4ed40b32c0 2024-05-30T17:02:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ed2000640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed40b32c0 0x7f4ed414b2e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009870 tx=0x7f4ec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ecaa00640 1 -- 172.21.15.67:0/854744901 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0041070 con 0x7f4ed40b32c0 2024-05-30T17:02:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ecaa00640 1 -- 172.21.15.67:0/854744901 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec003c870 con 0x7f4ed40b32c0 2024-05-30T17:02:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ecaa00640 1 -- 172.21.15.67:0/854744901 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0037040 con 0x7f4ed40b32c0 2024-05-30T17:02:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.194+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed414c3b0 con 0x7f4ed40b32c0 2024-05-30T17:02:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.195+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed414c970 con 0x7f4ed40b32c0 2024-05-30T17:02:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.196+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed40689e0 con 0x7f4ed40b32c0 2024-05-30T17:02:25.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.197+0000 7f4ecaa00640 1 -- 172.21.15.67:0/854744901 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f4ec0044070 con 0x7f4ed40b32c0 2024-05-30T17:02:25.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.198+0000 7f4ecaa00640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406e090 0x7f4eb4070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.198+0000 7f4ecaa00640 1 -- 172.21.15.67:0/854744901 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f4ec00ab4d0 con 0x7f4ed40b32c0 2024-05-30T17:02:25.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.198+0000 7f4ed2a00640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406e090 0x7f4eb4070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.199+0000 7f4ed2a00640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406e090 0x7f4eb4070550 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000f60 tx=0x7f4ec40079d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:25.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.202+0000 7f4ecaa00640 1 -- 172.21.15.67:0/854744901 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ec007b480 con 0x7f4ed40b32c0 2024-05-30T17:02:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.447+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ed40724d0 con 0x7f4ed40b32c0 2024-05-30T17:02:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.448+0000 7f4ecaa00640 1 -- 172.21.15.67:0/854744901 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7f4ec007ad80 con 0x7f4ed40b32c0 2024-05-30T17:02:25.449 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:25.449 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2641805294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2641805294},{"type":"v1","addr":"172.21.15.67:6837","nonce":2641805294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2609681849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2609681849},{"type":"v1","addr":"172.21.15.169:6834","nonce":2609681849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2372354106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2372354106},{"type":"v1","addr":"172.21.15.169:6835","nonce":2372354106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:00:36.172699+0000","modified":"2024-05-30T17:00:39.271296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7902":{"gid":7902,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6836/2120049133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2120049133},{"type":"v1","addr":"172.21.15.67:6836","nonce":2120049133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7902,"qdb_cluster":[7902]},"id":7}]} 2024-05-30T17:02:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-05-30T17:02:25.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406e090 msgr2=0x7f4eb4070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406e090 0x7f4eb4070550 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000f60 tx=0x7f4ec40079d0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed40b32c0 msgr2=0x7f4ed414b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed40b32c0 0x7f4ed414b2e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009870 tx=0x7f4ec00046c0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 shutdown_connections 2024-05-30T17:02:25.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4eb406e090 0x7f4eb4070550 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed4158b80 0x7f4ed414b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ed40b32c0 0x7f4ed414b2e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 --2- 172.21.15.67:0/854744901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed4071bc0 0x7f4ed4150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 >> 172.21.15.67:0/854744901 conn(0x7f4ed40b1fc0 msgr2=0x7f4ed4156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.451+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 shutdown_connections 2024-05-30T17:02:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.452+0000 7f4ed3e00640 1 -- 172.21.15.67:0/854744901 wait complete. 2024-05-30T17:02:25.473 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:02:25.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.689+0000 7f1613600640 1 Processor -- start 2024-05-30T17:02:25.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.690+0000 7f1613600640 1 -- start start 2024-05-30T17:02:25.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.691+0000 7f1613600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 0x7f160c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.691+0000 7f1613600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f160c0b3690 0x7f160c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.691+0000 7f1613600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f160c0d09e0 0x7f160c160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.691+0000 7f1613600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f160c0d0e50 con 0x7f160c0b3690 2024-05-30T17:02:25.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.691+0000 7f1613600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f160c06ed10 con 0x7f160c0d09e0 2024-05-30T17:02:25.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.691+0000 7f1613600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f160c15cb00 con 0x7f160c06f860 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.691+0000 7f1603e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f160c0b3690 0x7f160c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.691+0000 7f1603e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f160c0b3690 0x7f160c0d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59896/0 (socket says 172.21.15.67:59896) 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.691+0000 7f1603e00640 1 -- 172.21.15.67:0/3015613342 learned_addr learned my addr 172.21.15.67:0/3015613342 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.692+0000 7f1610c00640 1 --2- 172.21.15.67:0/3015613342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 0x7f160c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.692+0000 7f1611600640 1 --2- 172.21.15.67:0/3015613342 >> v2:172.21.15.169:3300/0 conn(0x7f160c0d09e0 0x7f160c160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.692+0000 7f1603e00640 1 -- 172.21.15.67:0/3015613342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 msgr2=0x7f160c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.692+0000 7f1603e00640 1 --2- 172.21.15.67:0/3015613342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 0x7f160c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.692+0000 7f1603e00640 1 -- 172.21.15.67:0/3015613342 >> v2:172.21.15.169:3300/0 conn(0x7f160c0d09e0 msgr2=0x7f160c160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.692+0000 7f1603e00640 1 --2- 172.21.15.67:0/3015613342 >> v2:172.21.15.169:3300/0 conn(0x7f160c0d09e0 0x7f160c160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.692+0000 7f1603e00640 1 -- 172.21.15.67:0/3015613342 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c0b3cc0 con 0x7f160c0b3690 2024-05-30T17:02:25.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.693+0000 7f1603e00640 1 --2- 172.21.15.67:0/3015613342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0b3690 0x7f160c0d04a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f160c06c900 tx=0x7f15f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb1ff991742e0b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:25.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.693+0000 7f160be00640 1 -- 172.21.15.67:0/3015613342 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4037070 con 0x7f160c0b3690 2024-05-30T17:02:25.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.693+0000 7f160be00640 1 -- 172.21.15.67:0/3015613342 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f4032440 con 0x7f160c0b3690 2024-05-30T17:02:25.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.693+0000 7f160be00640 1 -- 172.21.15.67:0/3015613342 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4032810 con 0x7f160c0b3690 2024-05-30T17:02:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.694+0000 7f1613600640 1 -- 172.21.15.67:0/3015613342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0b3690 msgr2=0x7f160c0d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.694+0000 7f1613600640 1 --2- 172.21.15.67:0/3015613342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0b3690 0x7f160c0d04a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f160c06c900 tx=0x7f15f402a1e0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.694+0000 7f1613600640 1 -- 172.21.15.67:0/3015613342 shutdown_connections 2024-05-30T17:02:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.694+0000 7f1613600640 1 --2- 172.21.15.67:0/3015613342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0b3690 0x7f160c0d04a0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.694+0000 7f1613600640 1 --2- 172.21.15.67:0/3015613342 >> v2:172.21.15.169:3300/0 conn(0x7f160c0d09e0 0x7f160c160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.694+0000 7f1613600640 1 --2- 172.21.15.67:0/3015613342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 0x7f160c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.694+0000 7f1613600640 1 -- 172.21.15.67:0/3015613342 >> 172.21.15.67:0/3015613342 conn(0x7f160c06bae0 msgr2=0x7f160c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.694+0000 7f1613600640 1 -- 172.21.15.67:0/3015613342 shutdown_connections 2024-05-30T17:02:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.694+0000 7f1613600640 1 -- 172.21.15.67:0/3015613342 wait complete. 2024-05-30T17:02:25.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.696+0000 7f1613600640 1 Processor -- start 2024-05-30T17:02:25.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.697+0000 7f1613600640 1 -- start start 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1613600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 0x7f160c15f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1613600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f160c0b3690 0x7f160c1f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1613600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0d09e0 0x7f160c0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1613600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f160c0b4560 con 0x7f160c0d09e0 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1613600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f160c0b4280 con 0x7f160c0b3690 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1613600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f160c0b4840 con 0x7f160c06f860 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1611600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0d09e0 0x7f160c0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1611600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0d09e0 0x7f160c0b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59900/0 (socket says 172.21.15.67:59900) 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1611600640 1 -- 172.21.15.67:0/1354493014 learned_addr learned my addr 172.21.15.67:0/1354493014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1610c00640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 0x7f160c15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.698+0000 7f1603e00640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f160c0b3690 0x7f160c1f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.699+0000 7f1611600640 1 -- 172.21.15.67:0/1354493014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 msgr2=0x7f160c15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.699+0000 7f1611600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 0x7f160c15f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.699+0000 7f1611600640 1 -- 172.21.15.67:0/1354493014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f160c0b3690 msgr2=0x7f160c1f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.699+0000 7f1611600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f160c0b3690 0x7f160c1f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.699+0000 7f1611600640 1 -- 172.21.15.67:0/1354493014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c157310 con 0x7f160c0d09e0 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.699+0000 7f1611600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0d09e0 0x7f160c0b0920 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f160400d600 tx=0x7f160400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.699+0000 7f1609600640 1 -- 172.21.15.67:0/1354493014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16040125f0 con 0x7f160c0d09e0 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.700+0000 7f1609600640 1 -- 172.21.15.67:0/1354493014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1604019070 con 0x7f160c0d09e0 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.700+0000 7f1609600640 1 -- 172.21.15.67:0/1354493014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16040129f0 con 0x7f160c0d09e0 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.700+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c157df0 con 0x7f160c0d09e0 2024-05-30T17:02:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.700+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c1583b0 con 0x7f160c0d09e0 2024-05-30T17:02:25.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.701+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160c0b34b0 con 0x7f160c0d09e0 2024-05-30T17:02:25.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.702+0000 7f1609600640 1 -- 172.21.15.67:0/1354493014 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f1604012b90 con 0x7f160c0d09e0 2024-05-30T17:02:25.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.703+0000 7f1609600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15e806e090 0x7f15e8070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:25.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.704+0000 7f1609600640 1 -- 172.21.15.67:0/1354493014 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f160408ad00 con 0x7f160c0d09e0 2024-05-30T17:02:25.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.704+0000 7f1610c00640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15e806e090 0x7f15e8070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:25.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.704+0000 7f1610c00640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15e806e090 0x7f15e8070550 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009a60 tx=0x7f15fc003990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:25.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.708+0000 7f1609600640 1 -- 172.21.15.67:0/1354493014 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f160c0b34b0 con 0x7f160c0d09e0 2024-05-30T17:02:25.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.974+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f160c0689e0 con 0x7f160c0d09e0 2024-05-30T17:02:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.977+0000 7f1609600640 1 -- 172.21.15.67:0/1354493014 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7f160c157310 con 0x7f160c0d09e0 2024-05-30T17:02:25.978 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:25.978 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2641805294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2641805294},{"type":"v1","addr":"172.21.15.67:6837","nonce":2641805294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2609681849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2609681849},{"type":"v1","addr":"172.21.15.169:6834","nonce":2609681849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2372354106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2372354106},{"type":"v1","addr":"172.21.15.169:6835","nonce":2372354106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:00:36.172699+0000","modified":"2024-05-30T17:00:39.271296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7902":{"gid":7902,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6836/2120049133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2120049133},{"type":"v1","addr":"172.21.15.67:6836","nonce":2120049133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7902,"qdb_cluster":[7902]},"id":7}]} 2024-05-30T17:02:25.978 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-05-30T17:02:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.982+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15e806e090 msgr2=0x7f15e8070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.982+0000 7f1613600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15e806e090 0x7f15e8070550 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009a60 tx=0x7f15fc003990 comp rx=0 tx=0).stop 2024-05-30T17:02:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.982+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0d09e0 msgr2=0x7f160c0b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.982+0000 7f1613600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0d09e0 0x7f160c0b0920 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f160400d600 tx=0x7f160400dac0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.982+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 shutdown_connections 2024-05-30T17:02:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.983+0000 7f1613600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f15e806e090 0x7f15e8070550 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.983+0000 7f1613600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f160c0d09e0 0x7f160c0b0920 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.983+0000 7f1613600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f160c0b3690 0x7f160c1f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.983+0000 7f1613600640 1 --2- 172.21.15.67:0/1354493014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f160c06f860 0x7f160c15f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.983+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 >> 172.21.15.67:0/1354493014 conn(0x7f160c06bae0 msgr2=0x7f160c159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.983+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 shutdown_connections 2024-05-30T17:02:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:25.983+0000 7f1613600640 1 -- 172.21.15.67:0/1354493014 wait complete. 2024-05-30T17:02:26.009 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:02:26.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.282+0000 7fe875c00640 1 Processor -- start 2024-05-30T17:02:26.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.282+0000 7fe875c00640 1 -- start start 2024-05-30T17:02:26.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe875c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe87015e9d0 0x7fe8700d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe875c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe8700d1650 0x7fe8700d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe875c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8700d5730 con 0x7fe8700d52a0 2024-05-30T17:02:26.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe875c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe87015edb0 con 0x7fe87015e9d0 2024-05-30T17:02:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe875c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe8700d2000 con 0x7fe8700d1650 2024-05-30T17:02:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe86f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe8700d1650 0x7fe8700d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe86ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe87015e9d0 0x7fe8700d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe86f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe8700d1650 0x7fe8700d1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:37706/0 (socket says 172.21.15.67:37706) 2024-05-30T17:02:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe86f400640 1 -- 172.21.15.67:0/4286921529 learned_addr learned my addr 172.21.15.67:0/4286921529 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.284+0000 7fe86ea00640 1 --2- 172.21.15.67:0/4286921529 >> v2:172.21.15.67:3300/0 conn(0x7fe87015e9d0 0x7fe8700d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59908/0 (socket says 172.21.15.67:59908) 2024-05-30T17:02:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.285+0000 7fe86f400640 1 -- 172.21.15.67:0/4286921529 >> v2:172.21.15.67:3300/0 conn(0x7fe87015e9d0 msgr2=0x7fe8700d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.285+0000 7fe86f400640 1 --2- 172.21.15.67:0/4286921529 >> v2:172.21.15.67:3300/0 conn(0x7fe87015e9d0 0x7fe8700d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.285+0000 7fe86f400640 1 -- 172.21.15.67:0/4286921529 >> v1:172.21.15.67:6789/0 conn(0x7fe8700d52a0 legacy=0x7fe8700d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.285+0000 7fe86f400640 1 -- 172.21.15.67:0/4286921529 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8700d4320 con 0x7fe8700d1650 2024-05-30T17:02:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.288+0000 7fe86f400640 1 --2- 172.21.15.67:0/4286921529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8700d1650 0x7fe8700d1ac0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe8700d0830 tx=0x7fe86802a6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6598ab88fdea610f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.288+0000 7fe86d600640 1 -- 172.21.15.67:0/4286921529 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868036070 con 0x7fe8700d1650 2024-05-30T17:02:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.288+0000 7fe86d600640 1 -- 172.21.15.67:0/4286921529 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe868031600 con 0x7fe8700d1650 2024-05-30T17:02:26.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.288+0000 7fe86d600640 1 -- 172.21.15.67:0/4286921529 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8680319d0 con 0x7fe8700d1650 2024-05-30T17:02:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.289+0000 7fe875c00640 1 -- 172.21.15.67:0/4286921529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8700d1650 msgr2=0x7fe8700d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.289+0000 7fe875c00640 1 --2- 172.21.15.67:0/4286921529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8700d1650 0x7fe8700d1ac0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe8700d0830 tx=0x7fe86802a6b0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.290+0000 7fe875c00640 1 -- 172.21.15.67:0/4286921529 shutdown_connections 2024-05-30T17:02:26.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.290+0000 7fe875c00640 1 --2- 172.21.15.67:0/4286921529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8700d1650 0x7fe8700d1ac0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.290+0000 7fe875c00640 1 --2- 172.21.15.67:0/4286921529 >> v2:172.21.15.67:3300/0 conn(0x7fe87015e9d0 0x7fe8700d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.290+0000 7fe875c00640 1 -- 172.21.15.67:0/4286921529 >> 172.21.15.67:0/4286921529 conn(0x7fe870067890 msgr2=0x7fe870067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:26.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.294+0000 7fe875c00640 1 -- 172.21.15.67:0/4286921529 shutdown_connections 2024-05-30T17:02:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.294+0000 7fe875c00640 1 -- 172.21.15.67:0/4286921529 wait complete. 2024-05-30T17:02:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.298+0000 7fe875c00640 1 Processor -- start 2024-05-30T17:02:26.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.298+0000 7fe875c00640 1 -- start start 2024-05-30T17:02:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.299+0000 7fe875c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8700d1650 0x7fe870154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.299+0000 7fe875c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8700d52a0 0x7fe870152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.300+0000 7fe875c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe87015e9d0 0x7fe8701fc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.300+0000 7fe875c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8700d3130 con 0x7fe8700d52a0 2024-05-30T17:02:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.300+0000 7fe875c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe8700d2c00 con 0x7fe87015e9d0 2024-05-30T17:02:26.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.300+0000 7fe875c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe8700d3410 con 0x7fe8700d1650 2024-05-30T17:02:26.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.300+0000 7fe86f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe87015e9d0 0x7fe8701fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.300+0000 7fe86e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8700d52a0 0x7fe870152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.300+0000 7fe86f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe87015e9d0 0x7fe8701fc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:37716/0 (socket says 172.21.15.67:37716) 2024-05-30T17:02:26.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.300+0000 7fe86f400640 1 -- 172.21.15.67:0/1789306861 learned_addr learned my addr 172.21.15.67:0/1789306861 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:26.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.301+0000 7fe86ea00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8700d1650 0x7fe870154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.301+0000 7fe86f400640 1 -- 172.21.15.67:0/1789306861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8700d1650 msgr2=0x7fe870154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.301+0000 7fe86f400640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8700d1650 0x7fe870154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.301+0000 7fe86f400640 1 -- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8700d52a0 msgr2=0x7fe870152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.302+0000 7fe86f400640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8700d52a0 0x7fe870152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.302+0000 7fe86f400640 1 -- 172.21.15.67:0/1789306861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8701ec8f0 con 0x7fe87015e9d0 2024-05-30T17:02:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.302+0000 7fe86e000640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8700d52a0 0x7fe870152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:02:26.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.303+0000 7fe86f400640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe87015e9d0 0x7fe8701fc570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe868034040 tx=0x7fe868004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.303+0000 7fe866a00640 1 -- 172.21.15.67:0/1789306861 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868036050 con 0x7fe87015e9d0 2024-05-30T17:02:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.303+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8701ecbd0 con 0x7fe87015e9d0 2024-05-30T17:02:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.304+0000 7fe866a00640 1 -- 172.21.15.67:0/1789306861 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe868042070 con 0x7fe87015e9d0 2024-05-30T17:02:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.304+0000 7fe866a00640 1 -- 172.21.15.67:0/1789306861 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868009510 con 0x7fe87015e9d0 2024-05-30T17:02:26.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.304+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8701ed190 con 0x7fe87015e9d0 2024-05-30T17:02:26.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.307+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8700689e0 con 0x7fe87015e9d0 2024-05-30T17:02:26.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.310+0000 7fe866a00640 1 -- 172.21.15.67:0/1789306861 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fe868007bf0 con 0x7fe87015e9d0 2024-05-30T17:02:26.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.313+0000 7fe866a00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe85006e090 0x7fe850070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.314+0000 7fe866a00640 1 -- 172.21.15.67:0/1789306861 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7fe8680ab7f0 con 0x7fe87015e9d0 2024-05-30T17:02:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.315+0000 7fe86ea00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe85006e090 0x7fe850070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.316+0000 7fe86ea00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe85006e090 0x7fe850070550 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe860009a30 tx=0x7fe860008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:26.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.321+0000 7fe866a00640 1 -- 172.21.15.67:0/1789306861 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe86807b600 con 0x7fe87015e9d0 2024-05-30T17:02:26.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.546+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8700d2480 con 0x7fe87015e9d0 2024-05-30T17:02:26.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.550+0000 7fe866a00640 1 -- 172.21.15.67:0/1789306861 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16739 (secure 0 0 0) 0x7fe86807af00 con 0x7fe87015e9d0 2024-05-30T17:02:26.551 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:26.551 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":104,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:00:37.736006+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:00:34.151659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-30T17:00:35.170102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:02:26.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.552+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe85006e090 msgr2=0x7fe850070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.552+0000 7fe875c00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe85006e090 0x7fe850070550 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe860009a30 tx=0x7fe860008040 comp rx=0 tx=0).stop 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.552+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe87015e9d0 msgr2=0x7fe8701fc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.552+0000 7fe875c00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe87015e9d0 0x7fe8701fc570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe868034040 tx=0x7fe868004a40 comp rx=0 tx=0).stop 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.553+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 shutdown_connections 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.553+0000 7fe875c00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe85006e090 0x7fe850070550 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.553+0000 7fe875c00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe87015e9d0 0x7fe8701fc570 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.553+0000 7fe875c00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8700d52a0 0x7fe870152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.553+0000 7fe875c00640 1 --2- 172.21.15.67:0/1789306861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8700d1650 0x7fe870154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.553+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 >> 172.21.15.67:0/1789306861 conn(0x7fe870067890 msgr2=0x7fe87015a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.553+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 shutdown_connections 2024-05-30T17:02:26.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.553+0000 7fe875c00640 1 -- 172.21.15.67:0/1789306861 wait complete. 2024-05-30T17:02:26.572 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-30T17:02:26.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:02:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.773+0000 7f882ea00640 1 Processor -- start 2024-05-30T17:02:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.773+0000 7f882ea00640 1 -- start start 2024-05-30T17:02:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.774+0000 7f882ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88280d16a0 0x7f88280cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.774+0000 7f882ea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88280cfce0 con 0x7f88280cf870 2024-05-30T17:02:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.774+0000 7f882ea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f882815ec50 con 0x7f882815e870 2024-05-30T17:02:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.774+0000 7f882ea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f88280d2150 con 0x7f88280d16a0 2024-05-30T17:02:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.775+0000 7f882cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88280d16a0 0x7f88280cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.775+0000 7f882cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88280d16a0 0x7f88280cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59936/0 (socket says 172.21.15.67:59936) 2024-05-30T17:02:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.775+0000 7f882cc00640 1 -- 172.21.15.67:0/3663830549 learned_addr learned my addr 172.21.15.67:0/3663830549 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.775+0000 7f882cc00640 1 -- 172.21.15.67:0/3663830549 >> v1:172.21.15.169:6789/0 conn(0x7f882815e870 legacy=0x7f88280d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.775+0000 7f882cc00640 1 -- 172.21.15.67:0/3663830549 >> v1:172.21.15.67:6789/0 conn(0x7f88280cf870 legacy=0x7f88280d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.775+0000 7f882cc00640 1 -- 172.21.15.67:0/3663830549 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88280cff80 con 0x7f88280d16a0 2024-05-30T17:02:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.776+0000 7f882cc00640 1 --2- 172.21.15.67:0/3663830549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 0x7f88280cf330 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8818009850 tx=0x7f881802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6831f3bbafc461 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.777+0000 7f8827e00640 1 -- 172.21.15.67:0/3663830549 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818004030 con 0x7f88280d16a0 2024-05-30T17:02:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.777+0000 7f8827e00640 1 -- 172.21.15.67:0/3663830549 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88180041d0 con 0x7f88280d16a0 2024-05-30T17:02:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.777+0000 7f8827e00640 1 -- 172.21.15.67:0/3663830549 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818003610 con 0x7f88280d16a0 2024-05-30T17:02:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.777+0000 7f882ea00640 1 -- 172.21.15.67:0/3663830549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 msgr2=0x7f88280cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.777+0000 7f882ea00640 1 --2- 172.21.15.67:0/3663830549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 0x7f88280cf330 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8818009850 tx=0x7f881802a290 comp rx=0 tx=0).stop 2024-05-30T17:02:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.777+0000 7f882ea00640 1 -- 172.21.15.67:0/3663830549 shutdown_connections 2024-05-30T17:02:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.777+0000 7f882ea00640 1 --2- 172.21.15.67:0/3663830549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 0x7f88280cf330 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.778+0000 7f882ea00640 1 -- 172.21.15.67:0/3663830549 >> 172.21.15.67:0/3663830549 conn(0x7f882808bfc0 msgr2=0x7f88280d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.778+0000 7f882ea00640 1 -- 172.21.15.67:0/3663830549 shutdown_connections 2024-05-30T17:02:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.778+0000 7f882ea00640 1 -- 172.21.15.67:0/3663830549 wait complete. 2024-05-30T17:02:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.779+0000 7f882ea00640 1 Processor -- start 2024-05-30T17:02:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.780+0000 7f882ea00640 1 -- start start 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.780+0000 7f882ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88280cf870 0x7f8828159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 0x7f88280cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f882815e870 0x7f88280d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f88280d0820 con 0x7f88280d16a0 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88280d0540 con 0x7f88280cf870 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88280d4fb0 con 0x7f882815e870 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 0x7f88280cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 0x7f88280cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59950/0 (socket says 172.21.15.67:59950) 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882cc00640 1 -- 172.21.15.67:0/2791516467 learned_addr learned my addr 172.21.15.67:0/2791516467 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882d600640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88280cf870 0x7f8828159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882e000640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f882815e870 0x7f88280d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882cc00640 1 -- 172.21.15.67:0/2791516467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f882815e870 msgr2=0x7f88280d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882cc00640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f882815e870 0x7f88280d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882cc00640 1 -- 172.21.15.67:0/2791516467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88280cf870 msgr2=0x7f8828159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882cc00640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88280cf870 0x7f8828159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882cc00640 1 -- 172.21.15.67:0/2791516467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88280cd0a0 con 0x7f88280d16a0 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.781+0000 7f882e000640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f882815e870 0x7f88280d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.782+0000 7f882cc00640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 0x7f88280cc6d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8818000c00 tx=0x7f88180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.782+0000 7f8825600640 1 -- 172.21.15.67:0/2791516467 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818041020 con 0x7f88280d16a0 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.782+0000 7f8825600640 1 -- 172.21.15.67:0/2791516467 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8818003bd0 con 0x7f88280d16a0 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.782+0000 7f8825600640 1 -- 172.21.15.67:0/2791516467 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818006040 con 0x7f88280d16a0 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.782+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88281e9270 con 0x7f88280d16a0 2024-05-30T17:02:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.782+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88281e9830 con 0x7f88280d16a0 2024-05-30T17:02:26.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.783+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88280689e0 con 0x7f88280d16a0 2024-05-30T17:02:26.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.785+0000 7f8825600640 1 -- 172.21.15.67:0/2791516467 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f8818044020 con 0x7f88280d16a0 2024-05-30T17:02:26.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.785+0000 7f8825600640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f880c06e090 0x7f880c070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:26.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.786+0000 7f8825600640 1 -- 172.21.15.67:0/2791516467 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5793+0+0 (secure 0 0 0) 0x7f88180ab530 con 0x7f88280d16a0 2024-05-30T17:02:26.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.786+0000 7f882d600640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f880c06e090 0x7f880c070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:26.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.786+0000 7f882d600640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f880c06e090 0x7f880c070550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f881c006fd0 tx=0x7f881c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:26.790+0000 7f8825600640 1 -- 172.21.15.67:0/2791516467 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f881804b030 con 0x7f88280d16a0 2024-05-30T17:02:27.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.047+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88280b20a0 con 0x7f88280d16a0 2024-05-30T17:02:27.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.048+0000 7f8825600640 1 -- 172.21.15.67:0/2791516467 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7f881807b3c0 con 0x7f88280d16a0 2024-05-30T17:02:27.049 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:27.049 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2641805294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2641805294},{"type":"v1","addr":"172.21.15.67:6837","nonce":2641805294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2609681849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2609681849},{"type":"v1","addr":"172.21.15.169:6834","nonce":2609681849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2372354106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2372354106},{"type":"v1","addr":"172.21.15.169:6835","nonce":2372354106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:00:36.172699+0000","modified":"2024-05-30T17:00:39.271296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7902":{"gid":7902,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6836/2120049133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2120049133},{"type":"v1","addr":"172.21.15.67:6836","nonce":2120049133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7902,"qdb_cluster":[7902]},"id":7}]} 2024-05-30T17:02:27.049 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-05-30T17:02:27.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.050+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f880c06e090 msgr2=0x7f880c070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.050+0000 7f882ea00640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f880c06e090 0x7f880c070550 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f881c006fd0 tx=0x7f881c008040 comp rx=0 tx=0).stop 2024-05-30T17:02:27.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.050+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 msgr2=0x7f88280cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.050+0000 7f882ea00640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 0x7f88280cc6d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8818000c00 tx=0x7f88180046c0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.051+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 shutdown_connections 2024-05-30T17:02:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.051+0000 7f882ea00640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f880c06e090 0x7f880c070550 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.051+0000 7f882ea00640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f882815e870 0x7f88280d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.051+0000 7f882ea00640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88280d16a0 0x7f88280cc6d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.051+0000 7f882ea00640 1 --2- 172.21.15.67:0/2791516467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88280cf870 0x7f8828159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.051+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 >> 172.21.15.67:0/2791516467 conn(0x7f882808bfc0 msgr2=0x7f88280b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.051+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 shutdown_connections 2024-05-30T17:02:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.051+0000 7f882ea00640 1 -- 172.21.15.67:0/2791516467 wait complete. 2024-05-30T17:02:27.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:02:27.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.288+0000 7fd4b2e00640 1 Processor -- start 2024-05-30T17:02:27.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.288+0000 7fd4b2e00640 1 -- start start 2024-05-30T17:02:27.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.289+0000 7fd4b2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac15b3a0 0x7fd4ac0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.290+0000 7fd4b2e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ac0afd60 con 0x7fd4ac073310 2024-05-30T17:02:27.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.290+0000 7fd4b2e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ac0af7a0 con 0x7fd4ac06dc20 2024-05-30T17:02:27.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.290+0000 7fd4b2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ac06f980 con 0x7fd4ac15b3a0 2024-05-30T17:02:27.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.290+0000 7fd4b2400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd4ac073310 0x7fd4ac0ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52224/0 (socket says 172.21.15.67:52224) 2024-05-30T17:02:27.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.290+0000 7fd4b2400640 1 -- 172.21.15.67:0/3870460197 learned_addr learned my addr 172.21.15.67:0/3870460197 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.290+0000 7fd4b1a00640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac15b3a0 0x7fd4ac0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449068115 0 0) 0x7fd4ac0afd60 con 0x7fd4ac073310 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd494003520 con 0x7fd4ac073310 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2391101385 0 0) 0x7fd494003520 con 0x7fd4ac073310 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac15b3a0 msgr2=0x7fd4ac0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac15b3a0 0x7fd4ac0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 >> v1:172.21.15.169:6789/0 conn(0x7fd4ac06dc20 legacy=0x7fd4ac072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac0afa80 con 0x7fd4ac073310 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fd4a40019f0 con 0x7fd4ac073310 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 >> v1:172.21.15.67:6789/0 conn(0x7fd4ac073310 legacy=0x7fd4ac0ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4940039e0 0x7fd494003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac15b3a0 0x7fd494004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac06f980 con 0x7fd494004d40 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd49400ba70 con 0x7fd4940039e0 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.291+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd49400bc20 con 0x7fd4ac15b3a0 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.292+0000 7fd4b2400640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.292+0000 7fd4b1a00640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4940039e0 0x7fd494003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.292+0000 7fd4b1000640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac15b3a0 0x7fd494004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.292+0000 7fd4b2400640 1 -- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac15b3a0 msgr2=0x7fd494004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.292+0000 7fd4b2400640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac15b3a0 0x7fd494004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.292+0000 7fd4b2400640 1 -- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4940039e0 msgr2=0x7fd494003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.292+0000 7fd4b2400640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4940039e0 0x7fd494003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.292+0000 7fd4b2400640 1 -- 172.21.15.67:0/3870460197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd494003520 con 0x7fd494004d40 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.294+0000 7fd4b2400640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4940051c0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac070090 tx=0x7fd494005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef39b769afb43fd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.294+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400a3f0 con 0x7fd494004d40 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.294+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a402f070 con 0x7fd494004d40 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.294+0000 7fd4abe00640 1 -- 172.21.15.67:0/3870460197 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a402bbe0 con 0x7fd494004d40 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.295+0000 7fd4b2e00640 1 -- 172.21.15.67:0/3870460197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 msgr2=0x7fd4940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.295+0000 7fd4b2e00640 1 --2- 172.21.15.67:0/3870460197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4940051c0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac070090 tx=0x7fd494005830 comp rx=0 tx=0).stop 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.295+0000 7fd4b2400640 1 -- 172.21.15.67:0/3870460197 reap_dead start 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.295+0000 7fd4b2e00640 1 -- 172.21.15.67:0/3870460197 shutdown_connections 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.295+0000 7fd4b2e00640 1 -- 172.21.15.67:0/3870460197 >> 172.21.15.67:0/3870460197 conn(0x7fd4ac0b1fc0 msgr2=0x7fd4ac0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.295+0000 7fd4b2e00640 1 -- 172.21.15.67:0/3870460197 shutdown_connections 2024-05-30T17:02:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.295+0000 7fd4b2e00640 1 -- 172.21.15.67:0/3870460197 wait complete. 2024-05-30T17:02:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.297+0000 7fd4b2e00640 1 Processor -- start 2024-05-30T17:02:27.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.297+0000 7fd4b2e00640 1 -- start start 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b2e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4ac09aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac09cb50 0x7fd4ac09cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ac14c260 0x7fd4ac14c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b2e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac0b32c0 con 0x7fd494004d40 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac0ae0c0 con 0x7fd4ac14c260 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ac0b3440 con 0x7fd4ac09cb50 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4ac09aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4ac09aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59966/0 (socket says 172.21.15.67:59966) 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b1a00640 1 -- 172.21.15.67:0/327264065 learned_addr learned my addr 172.21.15.67:0/327264065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b2400640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ac14c260 0x7fd4ac14c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.298+0000 7fd4b1000640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac09cb50 0x7fd4ac09cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4b1a00640 1 -- 172.21.15.67:0/327264065 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac09cb50 msgr2=0x7fd4ac09cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4b1a00640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac09cb50 0x7fd4ac09cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4b1a00640 1 -- 172.21.15.67:0/327264065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ac14c260 msgr2=0x7fd4ac14c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4b1a00640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ac14c260 0x7fd4ac14c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4b1a00640 1 -- 172.21.15.67:0/327264065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac09d5c0 con 0x7fd494004d40 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4b2400640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ac14c260 0x7fd4ac14c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4b1a00640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4ac09aea0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0011600 tx=0x7fd4a0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4a9600640 1 -- 172.21.15.67:0/327264065 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000f920 con 0x7fd494004d40 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4a9600640 1 -- 172.21.15.67:0/327264065 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a000fac0 con 0x7fd494004d40 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4a9600640 1 -- 172.21.15.67:0/327264065 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000fe70 con 0x7fd494004d40 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ac09d8a0 con 0x7fd494004d40 2024-05-30T17:02:27.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.299+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ac09de60 con 0x7fd494004d40 2024-05-30T17:02:27.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.301+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ac0689e0 con 0x7fd494004d40 2024-05-30T17:02:27.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.302+0000 7fd4a9600640 1 -- 172.21.15.67:0/327264065 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fd4a0015460 con 0x7fd494004d40 2024-05-30T17:02:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.303+0000 7fd4a9600640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48c06e0e0 0x7fd48c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.303+0000 7fd4a9600640 1 -- 172.21.15.67:0/327264065 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5849+0+0 (secure 0 0 0) 0x7fd4a008b2a0 con 0x7fd494004d40 2024-05-30T17:02:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.303+0000 7fd4b1000640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48c06e0e0 0x7fd48c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.304+0000 7fd4b1000640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48c06e0e0 0x7fd48c0705a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0ca5f0 tx=0x7fd498007b70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:27.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.307+0000 7fd4a9600640 1 -- 172.21.15.67:0/327264065 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4a0019030 con 0x7fd494004d40 2024-05-30T17:02:27.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.557+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4ac0b3a80 con 0x7fd494004d40 2024-05-30T17:02:27.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.558+0000 7fd4a9600640 1 -- 172.21.15.67:0/327264065 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7fd4a005b1b0 con 0x7fd494004d40 2024-05-30T17:02:27.559 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:27.559 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2641805294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2641805294},{"type":"v1","addr":"172.21.15.67:6837","nonce":2641805294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2609681849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2609681849},{"type":"v1","addr":"172.21.15.169:6834","nonce":2609681849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2372354106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2372354106},{"type":"v1","addr":"172.21.15.169:6835","nonce":2372354106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:00:36.172699+0000","modified":"2024-05-30T17:00:39.271296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7902},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7902":{"gid":7902,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6836/2120049133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2120049133},{"type":"v1","addr":"172.21.15.67:6836","nonce":2120049133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7902,"qdb_cluster":[7902]},"id":7}]} 2024-05-30T17:02:27.559 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 65 2024-05-30T17:02:27.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48c06e0e0 msgr2=0x7fd48c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48c06e0e0 0x7fd48c0705a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0ca5f0 tx=0x7fd498007b70 comp rx=0 tx=0).stop 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 msgr2=0x7fd4ac09aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4ac09aea0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0011600 tx=0x7fd4a0011ac0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 shutdown_connections 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd48c06e0e0 0x7fd48c0705a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4ac14c260 0x7fd4ac14c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4ac09cb50 0x7fd4ac09cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 --2- 172.21.15.67:0/327264065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd494004d40 0x7fd4ac09aea0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 >> 172.21.15.67:0/327264065 conn(0x7fd4ac0b1fc0 msgr2=0x7fd4ac0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 shutdown_connections 2024-05-30T17:02:27.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.561+0000 7fd4b2e00640 1 -- 172.21.15.67:0/327264065 wait complete. 2024-05-30T17:02:27.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:02:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.778+0000 7fe5e6600640 1 Processor -- start 2024-05-30T17:02:27.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.778+0000 7fe5e6600640 1 -- start start 2024-05-30T17:02:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.780+0000 7fe5e6600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5e00ad940 0x7fe5e00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.780+0000 7fe5e6600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5e0156f10 con 0x7fe5e006ee40 2024-05-30T17:02:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.780+0000 7fe5e6600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5e006dda0 con 0x7fe5e0156b30 2024-05-30T17:02:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.780+0000 7fe5e6600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe5e006e080 con 0x7fe5e00ad940 2024-05-30T17:02:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.780+0000 7fe5d7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe5e006ee40 0x7fe5e00ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52240/0 (socket says 172.21.15.67:52240) 2024-05-30T17:02:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.780+0000 7fe5d7e00640 1 -- 172.21.15.67:0/3174964393 learned_addr learned my addr 172.21.15.67:0/3174964393 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:27.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.780+0000 7fe5e5c00640 1 --2- 172.21.15.67:0/3174964393 >> v2:172.21.15.169:3300/0 conn(0x7fe5e00ad940 0x7fe5e00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.780+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678732487 0 0) 0x7fe5e0156f10 con 0x7fe5e006ee40 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.780+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5c0003520 con 0x7fe5e006ee40 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3495816745 0 0) 0x7fe5c0003520 con 0x7fe5e006ee40 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 >> v2:172.21.15.169:3300/0 conn(0x7fe5e00ad940 msgr2=0x7fe5e00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 --2- 172.21.15.67:0/3174964393 >> v2:172.21.15.169:3300/0 conn(0x7fe5e00ad940 0x7fe5e00b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 >> v1:172.21.15.169:6789/0 conn(0x7fe5e0156b30 legacy=0x7fe5e00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e006e360 con 0x7fe5e006ee40 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fe5d0001670 con 0x7fe5e006ee40 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 >> v1:172.21.15.67:6789/0 conn(0x7fe5e006ee40 legacy=0x7fe5e00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c0003a00 0x7fe5c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c0004800 0x7fe5c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c0005210 0x7fe5c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e006e080 con 0x7fe5c0003a00 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c000c160 con 0x7fe5c0004800 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c000c360 con 0x7fe5c0005210 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.781+0000 7fe5e5200640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c0003a00 0x7fe5c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.782+0000 7fe5d7e00640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c0004800 0x7fe5c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.782+0000 7fe5e5200640 1 -- 172.21.15.67:0/3174964393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c0005210 msgr2=0x7fe5c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.782+0000 7fe5e5200640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c0005210 0x7fe5c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.782+0000 7fe5e5200640 1 -- 172.21.15.67:0/3174964393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c0004800 msgr2=0x7fe5c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.782+0000 7fe5e5200640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c0004800 0x7fe5c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.782+0000 7fe5e5200640 1 -- 172.21.15.67:0/3174964393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c0003520 con 0x7fe5c0003a00 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.782+0000 7fe5e5c00640 1 -- 172.21.15.67:0/3174964393 reap_dead start 2024-05-30T17:02:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.782+0000 7fe5d7e00640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c0004800 0x7fe5c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.783+0000 7fe5e5200640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c0003a00 0x7fe5c0003e20 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8001720 tx=0x7fe5c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a955f70bf04d6ff server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.783+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8001370 con 0x7fe5c0003a00 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.783+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c80052d0 con 0x7fe5c0003a00 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.783+0000 7fe5dfe00640 1 -- 172.21.15.67:0/3174964393 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c802abc0 con 0x7fe5c0003a00 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.784+0000 7fe5e6600640 1 -- 172.21.15.67:0/3174964393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c0003a00 msgr2=0x7fe5c0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.784+0000 7fe5e6600640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c0003a00 0x7fe5c0003e20 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8001720 tx=0x7fe5c0005830 comp rx=0 tx=0).stop 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.784+0000 7fe5e6600640 1 -- 172.21.15.67:0/3174964393 shutdown_connections 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.784+0000 7fe5e6600640 1 --2- 172.21.15.67:0/3174964393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c0003a00 0x7fe5c0003e20 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.784+0000 7fe5e6600640 1 -- 172.21.15.67:0/3174964393 >> 172.21.15.67:0/3174964393 conn(0x7fe5e00b1fc0 msgr2=0x7fe5e00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.784+0000 7fe5e6600640 1 -- 172.21.15.67:0/3174964393 shutdown_connections 2024-05-30T17:02:27.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.784+0000 7fe5e6600640 1 -- 172.21.15.67:0/3174964393 wait complete. 2024-05-30T17:02:27.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.786+0000 7fe5e6600640 1 Processor -- start 2024-05-30T17:02:27.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.786+0000 7fe5e6600640 1 -- start start 2024-05-30T17:02:27.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.787+0000 7fe5e6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c0003a00 0x7fe5e0073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.787+0000 7fe5e6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5e0099b50 0x7fe5e01f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.787+0000 7fe5e6600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5e00b4610 0x7fe5e00b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.787+0000 7fe5e6600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e00b2db0 con 0x7fe5e0099b50 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.787+0000 7fe5e6600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e00b2c30 con 0x7fe5e00b4610 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.787+0000 7fe5e6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e006f2f0 con 0x7fe5c0003a00 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.787+0000 7fe5d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5e0099b50 0x7fe5e01f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.787+0000 7fe5d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5e0099b50 0x7fe5e01f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59984/0 (socket says 172.21.15.67:59984) 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.787+0000 7fe5d7e00640 1 -- 172.21.15.67:0/1710348900 learned_addr learned my addr 172.21.15.67:0/1710348900 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5e5200640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c0003a00 0x7fe5e0073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5e5c00640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5e00b4610 0x7fe5e00b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5d7e00640 1 -- 172.21.15.67:0/1710348900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c0003a00 msgr2=0x7fe5e0073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5d7e00640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c0003a00 0x7fe5e0073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5d7e00640 1 -- 172.21.15.67:0/1710348900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5e00b4610 msgr2=0x7fe5e00b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5d7e00640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5e00b4610 0x7fe5e00b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5d7e00640 1 -- 172.21.15.67:0/1710348900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e01f45c0 con 0x7fe5e0099b50 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5d7e00640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5e0099b50 0x7fe5e01f4080 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000d7a0 tx=0x7fe5d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5dd600640 1 -- 172.21.15.67:0/1710348900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0019070 con 0x7fe5e0099b50 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5dd600640 1 -- 172.21.15.67:0/1710348900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d00072d0 con 0x7fe5e0099b50 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5dd600640 1 -- 172.21.15.67:0/1710348900 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00047e0 con 0x7fe5e0099b50 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.788+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e0071f80 con 0x7fe5e0099b50 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.789+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e0072540 con 0x7fe5e0099b50 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.790+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e00689e0 con 0x7fe5e0099b50 2024-05-30T17:02:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.791+0000 7fe5dd600640 1 -- 172.21.15.67:0/1710348900 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fe5d0004980 con 0x7fe5e0099b50 2024-05-30T17:02:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.792+0000 7fe5dd600640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5bc06e0c0 0x7fe5bc070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.792+0000 7fe5dd600640 1 -- 172.21.15.67:0/1710348900 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5849+0+0 (secure 0 0 0) 0x7fe5d008a280 con 0x7fe5e0099b50 2024-05-30T17:02:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.792+0000 7fe5e5200640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5bc06e0c0 0x7fe5bc070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.793+0000 7fe5e5200640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5bc06e0c0 0x7fe5bc070580 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8032000 tx=0x7fe5c802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:27.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:27.796+0000 7fe5dd600640 1 -- 172.21.15.67:0/1710348900 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5d005a110 con 0x7fe5e0099b50 2024-05-30T17:02:28.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.002+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5e006f7f0 con 0x7fe5e0099b50 2024-05-30T17:02:28.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.005+0000 7fe5dd600640 1 -- 172.21.15.67:0/1710348900 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16851 (secure 0 0 0) 0x7fe5d0059a10 con 0x7fe5e0099b50 2024-05-30T17:02:28.008 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:28.008 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":105,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:02:26.843672+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:00:34.151659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-30T17:00:35.170102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:02:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5bc06e0c0 msgr2=0x7fe5bc070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5bc06e0c0 0x7fe5bc070580 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8032000 tx=0x7fe5c802e000 comp rx=0 tx=0).stop 2024-05-30T17:02:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5e0099b50 msgr2=0x7fe5e01f4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5e0099b50 0x7fe5e01f4080 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000d7a0 tx=0x7fe5d000dc60 comp rx=0 tx=0).stop 2024-05-30T17:02:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 shutdown_connections 2024-05-30T17:02:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe5bc06e0c0 0x7fe5bc070580 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5e00b4610 0x7fe5e00b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5e0099b50 0x7fe5e01f4080 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 --2- 172.21.15.67:0/1710348900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c0003a00 0x7fe5e0073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 >> 172.21.15.67:0/1710348900 conn(0x7fe5e00b1fc0 msgr2=0x7fe5e015b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.008+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 shutdown_connections 2024-05-30T17:02:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.009+0000 7fe5e6600640 1 -- 172.21.15.67:0/1710348900 wait complete. 2024-05-30T17:02:28.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-30T17:02:28.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.239+0000 7f28f8200640 1 Processor -- start 2024-05-30T17:02:28.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.239+0000 7f28f8200640 1 -- start start 2024-05-30T17:02:28.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.241+0000 7f28f8200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f28f00d04a0 0x7f28f00d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.241+0000 7f28f8200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28f00b3460 0x7f28f00b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.241+0000 7f28f8200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f00ced40 con 0x7f28f00b4b60 2024-05-30T17:02:28.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.241+0000 7f28f8200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f28f015fbb0 con 0x7f28f00d04a0 2024-05-30T17:02:28.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.241+0000 7f28f8200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f28f00cefc0 con 0x7f28f00b3460 2024-05-30T17:02:28.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.241+0000 7f28f5800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f28f00b4b60 0x7f28f00cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52252/0 (socket says 172.21.15.67:52252) 2024-05-30T17:02:28.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.241+0000 7f28f5800640 1 -- 172.21.15.67:0/2934714992 learned_addr learned my addr 172.21.15.67:0/2934714992 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28f4e00640 1 --2- 172.21.15.67:0/2934714992 >> v2:172.21.15.169:3300/0 conn(0x7f28f00d04a0 0x7f28f00d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28f6200640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28f00b3460 0x7f28f00b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 105362785 0 0) 0x7f28e4001110 con 0x7f28f00b4b60 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28d8003520 con 0x7f28f00b4b60 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2276401882 0 0) 0x7f28d8003520 con 0x7f28f00b4b60 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28f00b3460 msgr2=0x7f28f00b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28f00b3460 0x7f28f00b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 >> v2:172.21.15.169:3300/0 conn(0x7f28f00d04a0 msgr2=0x7f28f00d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 --2- 172.21.15.67:0/2934714992 >> v2:172.21.15.169:3300/0 conn(0x7f28f00d04a0 0x7f28f00d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f00cf2a0 con 0x7f28f00b4b60 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f28e4001860 con 0x7f28f00b4b60 2024-05-30T17:02:28.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 >> v1:172.21.15.67:6789/0 conn(0x7f28f00b4b60 legacy=0x7f28f00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28f00b3460 0x7f28d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.242+0000 7f28efe00640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28d8003f30 0x7f28d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28efe00640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28d8004d40 0x7f28d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f28f015fbb0 con 0x7f28d8003f30 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28f00cefc0 con 0x7f28d8004d40 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f28d800b9c0 con 0x7f28f00b3460 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28f4e00640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28d8003f30 0x7f28d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28f5800640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28f00b3460 0x7f28d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28f4e00640 1 -- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28f00b3460 msgr2=0x7f28d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28f6200640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28d8004d40 0x7f28d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28f4e00640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28f00b3460 0x7f28d80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28f4e00640 1 -- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28d8004d40 msgr2=0x7f28d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28f4e00640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28d8004d40 0x7f28d80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.243+0000 7f28f4e00640 1 -- 172.21.15.67:0/2934714992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d8003520 con 0x7f28d8003f30 2024-05-30T17:02:28.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.244+0000 7f28f4e00640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28d8003f30 0x7f28d8004800 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009ac0 tx=0x7f28d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ff682bbc230438 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc036070 con 0x7f28d8003f30 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28dc031400 con 0x7f28d8003f30 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28efe00640 1 -- 172.21.15.67:0/2934714992 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc009690 con 0x7f28d8003f30 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28f8200640 1 -- 172.21.15.67:0/2934714992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28d8003f30 msgr2=0x7f28d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28f8200640 1 --2- 172.21.15.67:0/2934714992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28d8003f30 0x7f28d8004800 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009ac0 tx=0x7f28d8005830 comp rx=0 tx=0).stop 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28f6200640 1 -- 172.21.15.67:0/2934714992 reap_dead start 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28f8200640 1 -- 172.21.15.67:0/2934714992 shutdown_connections 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28f8200640 1 -- 172.21.15.67:0/2934714992 >> 172.21.15.67:0/2934714992 conn(0x7f28f006bae0 msgr2=0x7f28f006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28f8200640 1 -- 172.21.15.67:0/2934714992 shutdown_connections 2024-05-30T17:02:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.245+0000 7f28f8200640 1 -- 172.21.15.67:0/2934714992 wait complete. 2024-05-30T17:02:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.247+0000 7f28f8200640 1 Processor -- start 2024-05-30T17:02:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.247+0000 7f28f8200640 1 -- start start 2024-05-30T17:02:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.248+0000 7f28f8200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d8003f30 0x7f28f015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.248+0000 7f28f8200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f009a2b0 0x7f28f009a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.248+0000 7f28f8200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00af6c0 0x7f28f00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.248+0000 7f28f8200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f28f00b3a50 con 0x7f28f009a2b0 2024-05-30T17:02:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.248+0000 7f28f8200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28f00b38d0 con 0x7f28f00af6c0 2024-05-30T17:02:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.248+0000 7f28f8200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f28f00b3bd0 con 0x7f28d8003f30 2024-05-30T17:02:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f009a2b0 0x7f28f009a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f009a2b0 0x7f28f009a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59996/0 (socket says 172.21.15.67:59996) 2024-05-30T17:02:28.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f4e00640 1 -- 172.21.15.67:0/3895403042 learned_addr learned my addr 172.21.15.67:0/3895403042 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f5800640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d8003f30 0x7f28f015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f6200640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00af6c0 0x7f28f00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f4e00640 1 -- 172.21.15.67:0/3895403042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d8003f30 msgr2=0x7f28f015b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f4e00640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d8003f30 0x7f28f015b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f4e00640 1 -- 172.21.15.67:0/3895403042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00af6c0 msgr2=0x7f28f00afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f4e00640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00af6c0 0x7f28f00afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.249+0000 7f28f4e00640 1 -- 172.21.15.67:0/3895403042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f00b0440 con 0x7f28f009a2b0 2024-05-30T17:02:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.250+0000 7f28f4e00640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f009a2b0 0x7f28f009a710 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f28dc034040 tx=0x7f28dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.250+0000 7f28ed600640 1 -- 172.21.15.67:0/3895403042 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc040020 con 0x7f28f009a2b0 2024-05-30T17:02:28.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.250+0000 7f28ed600640 1 -- 172.21.15.67:0/3895403042 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28dc003c60 con 0x7f28f009a2b0 2024-05-30T17:02:28.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.250+0000 7f28ed600640 1 -- 172.21.15.67:0/3895403042 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc036040 con 0x7f28f009a2b0 2024-05-30T17:02:28.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.250+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f00b0720 con 0x7f28f009a2b0 2024-05-30T17:02:28.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.250+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f00b0ce0 con 0x7f28f009a2b0 2024-05-30T17:02:28.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.251+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f00c6c50 con 0x7f28f009a2b0 2024-05-30T17:02:28.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.252+0000 7f28ed600640 1 -- 172.21.15.67:0/3895403042 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f28dc043020 con 0x7f28f009a2b0 2024-05-30T17:02:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.254+0000 7f28ed600640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28d006e090 0x7f28d0070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.254+0000 7f28ed600640 1 -- 172.21.15.67:0/3895403042 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5849+0+0 (secure 0 0 0) 0x7f28dc0aaaa0 con 0x7f28f009a2b0 2024-05-30T17:02:28.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.254+0000 7f28f5800640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28d006e090 0x7f28d0070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.255+0000 7f28f5800640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28d006e090 0x7f28d0070550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f28e4001b10 tx=0x7f28e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:28.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.259+0000 7f28ed600640 1 -- 172.21.15.67:0/3895403042 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28f00c6c50 con 0x7f28f009a2b0 2024-05-30T17:02:28.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.502+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f28f00689e0 con 0x7f28f009a2b0 2024-05-30T17:02:28.505 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-05-30T17:02:28.504+0000 7f4982000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-30T17:02:28.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.508+0000 7f28ed600640 1 -- 172.21.15.67:0/3895403042 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) ==== 161+0+0 (secure 0 0 0) 0x7f28dc04a070 con 0x7f28f009a2b0 2024-05-30T17:02:28.509 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-30T17:02:28.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.511+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28d006e090 msgr2=0x7f28d0070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.511+0000 7f28f8200640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28d006e090 0x7f28d0070550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f28e4001b10 tx=0x7f28e4009040 comp rx=0 tx=0).stop 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.511+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f009a2b0 msgr2=0x7f28f009a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.512+0000 7f28f8200640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f009a2b0 0x7f28f009a710 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f28dc034040 tx=0x7f28dc004730 comp rx=0 tx=0).stop 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.512+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 shutdown_connections 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.512+0000 7f28f8200640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f28d006e090 0x7f28d0070550 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.512+0000 7f28f8200640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28f00af6c0 0x7f28f00afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.512+0000 7f28f8200640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f009a2b0 0x7f28f009a710 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.512+0000 7f28f8200640 1 --2- 172.21.15.67:0/3895403042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d8003f30 0x7f28f015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.512+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 >> 172.21.15.67:0/3895403042 conn(0x7f28f006bae0 msgr2=0x7f28f00ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.512+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 shutdown_connections 2024-05-30T17:02:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.512+0000 7f28f8200640 1 -- 172.21.15.67:0/3895403042 wait complete. 2024-05-30T17:02:28.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-30T17:02:28.606 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T17:02:28.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.721+0000 7f2cffe00640 1 Processor -- start 2024-05-30T17:02:28.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.721+0000 7f2cffe00640 1 -- start start 2024-05-30T17:02:28.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.723+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d0015d270 0x7f2d000b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.723+0000 7f2cffe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d0015d900 con 0x7f2d000cd2a0 2024-05-30T17:02:28.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.723+0000 7f2cffe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d000b3f90 con 0x7f2d00153510 2024-05-30T17:02:28.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.723+0000 7f2cffe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d001538f0 con 0x7f2d0015d270 2024-05-30T17:02:28.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.723+0000 7f2cfe000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2d000cd2a0 0x7f2d0015cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52264/0 (socket says 172.21.15.67:52264) 2024-05-30T17:02:28.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.723+0000 7f2cfe000640 1 -- 172.21.15.67:0/3375570853 learned_addr learned my addr 172.21.15.67:0/3375570853 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:28.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.723+0000 7f2cff400640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d0015d270 0x7f2d000b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.723+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2484688304 0 0) 0x7f2d0015d900 con 0x7f2d000cd2a0 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ce4003520 con 0x7f2d000cd2a0 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3925909511 0 0) 0x7f2ce4003520 con 0x7f2d000cd2a0 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d0015d270 msgr2=0x7f2d000b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d0015d270 0x7f2d000b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 >> v1:172.21.15.169:6789/0 conn(0x7f2d00153510 legacy=0x7f2d000ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d000b4270 con 0x7f2d000cd2a0 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f2cec001670 con 0x7f2d000cd2a0 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 >> v1:172.21.15.67:6789/0 conn(0x7f2d000cd2a0 legacy=0x7f2d0015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d0015d270 0x7f2ce40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ce4003f60 0x7f2ce4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ce4004d40 0x7f2ce40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce400bb50 con 0x7f2ce4003f60 2024-05-30T17:02:28.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce400bcd0 con 0x7f2ce4004d40 2024-05-30T17:02:28.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce400bed0 con 0x7f2d0015d270 2024-05-30T17:02:28.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.724+0000 7f2cfe000640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ce4003f60 0x7f2ce4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.725+0000 7f2cfe000640 1 -- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d0015d270 msgr2=0x7f2ce40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:28.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.725+0000 7f2cff400640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ce4004d40 0x7f2ce40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.725+0000 7f2cfe000640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d0015d270 0x7f2ce40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.725+0000 7f2cfe000640 1 -- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ce4004d40 msgr2=0x7f2ce40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:28.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.725+0000 7f2cfe000640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ce4004d40 0x7f2ce40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.725+0000 7f2cfe000640 1 -- 172.21.15.67:0/3375570853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce4003520 con 0x7f2ce4003f60 2024-05-30T17:02:28.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.726+0000 7f2cfe000640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ce4003f60 0x7f2ce4004800 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c210 tx=0x7f2ce4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d602d8c8d1c4f11d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:28.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.726+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec007d30 con 0x7f2ce4003f60 2024-05-30T17:02:28.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.726+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cec005ad0 con 0x7f2ce4003f60 2024-05-30T17:02:28.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.726+0000 7f2cfd600640 1 -- 172.21.15.67:0/3375570853 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec007070 con 0x7f2ce4003f60 2024-05-30T17:02:28.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.727+0000 7f2cffe00640 1 -- 172.21.15.67:0/3375570853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ce4003f60 msgr2=0x7f2ce4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.727+0000 7f2cffe00640 1 --2- 172.21.15.67:0/3375570853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ce4003f60 0x7f2ce4004800 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c210 tx=0x7f2ce4005830 comp rx=0 tx=0).stop 2024-05-30T17:02:28.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.727+0000 7f2cff400640 1 -- 172.21.15.67:0/3375570853 reap_dead start 2024-05-30T17:02:28.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.727+0000 7f2cffe00640 1 -- 172.21.15.67:0/3375570853 shutdown_connections 2024-05-30T17:02:28.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.727+0000 7f2cffe00640 1 -- 172.21.15.67:0/3375570853 >> 172.21.15.67:0/3375570853 conn(0x7f2d0006b900 msgr2=0x7f2d0006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:28.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.727+0000 7f2cffe00640 1 -- 172.21.15.67:0/3375570853 shutdown_connections 2024-05-30T17:02:28.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.727+0000 7f2cffe00640 1 -- 172.21.15.67:0/3375570853 wait complete. 2024-05-30T17:02:28.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.729+0000 7f2cffe00640 1 Processor -- start 2024-05-30T17:02:28.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.729+0000 7f2cffe00640 1 -- start start 2024-05-30T17:02:28.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.730+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ce4003f60 0x7f2d0014b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.730+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d000cf940 0x7f2d00154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.730+0000 7f2cffe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d000cb2d0 0x7f2d000cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.730+0000 7f2cffe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000b48c0 con 0x7f2d000cf940 2024-05-30T17:02:28.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.730+0000 7f2cffe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0015d6c0 con 0x7f2d000cb2d0 2024-05-30T17:02:28.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.730+0000 7f2cffe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d000b4d50 con 0x7f2ce4003f60 2024-05-30T17:02:28.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.730+0000 7f2cfe000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d000cf940 0x7f2d00154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.730+0000 7f2cfe000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d000cf940 0x7f2d00154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60044/0 (socket says 172.21.15.67:60044) 2024-05-30T17:02:28.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.730+0000 7f2cfe000640 1 -- 172.21.15.67:0/108106496 learned_addr learned my addr 172.21.15.67:0/108106496 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:28.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cff400640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d000cb2d0 0x7f2d000cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cfea00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ce4003f60 0x7f2d0014b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cfe000640 1 -- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ce4003f60 msgr2=0x7f2d0014b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cfe000640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ce4003f60 0x7f2d0014b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cfe000640 1 -- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d000cb2d0 msgr2=0x7f2d000cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cfe000640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d000cb2d0 0x7f2d000cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cfe000640 1 -- 172.21.15.67:0/108106496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d00155210 con 0x7f2d000cf940 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cff400640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d000cb2d0 0x7f2d000cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cfea00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ce4003f60 0x7f2d0014b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cfe000640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d000cf940 0x7f2d00154930 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c330 tx=0x7f2cec001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cf6a00640 1 -- 172.21.15.67:0/108106496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec004bf0 con 0x7f2d000cf940 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cf6a00640 1 -- 172.21.15.67:0/108106496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cec043070 con 0x7f2d000cf940 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d001554f0 con 0x7f2d000cf940 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.731+0000 7f2cf6a00640 1 -- 172.21.15.67:0/108106496 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec007d10 con 0x7f2d000cf940 2024-05-30T17:02:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.732+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d00155ab0 con 0x7f2d000cf940 2024-05-30T17:02:28.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.733+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d000689e0 con 0x7f2d000cf940 2024-05-30T17:02:28.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.734+0000 7f2cf6a00640 1 -- 172.21.15.67:0/108106496 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f2cec007070 con 0x7f2d000cf940 2024-05-30T17:02:28.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.735+0000 7f2cf6a00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ce006e0c0 0x7f2ce0070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:28.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.735+0000 7f2cf6a00640 1 -- 172.21.15.67:0/108106496 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5907+0+0 (secure 0 0 0) 0x7f2cec0aad40 con 0x7f2d000cf940 2024-05-30T17:02:28.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.735+0000 7f2cfea00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ce006e0c0 0x7f2ce0070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:28.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.736+0000 7f2cfea00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ce006e0c0 0x7f2ce0070580 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0007fd0 tx=0x7f2cf0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:28.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.739+0000 7f2cf6a00640 1 -- 172.21.15.67:0/108106496 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2cec07abd0 con 0x7f2d000cf940 2024-05-30T17:02:28.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:28.989+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2d000cb0b0 con 0x7f2d000cf940 2024-05-30T17:02:29.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.511+0000 7f2cf6a00640 1 -- 172.21.15.67:0/108106496 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) ==== 104+0+0 (secure 0 0 0) 0x7f2cec07a4d0 con 0x7f2d000cf940 2024-05-30T17:02:29.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.514+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ce006e0c0 msgr2=0x7f2ce0070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.514+0000 7f2cffe00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ce006e0c0 0x7f2ce0070580 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0007fd0 tx=0x7f2cf0009040 comp rx=0 tx=0).stop 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d000cf940 msgr2=0x7f2d00154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d000cf940 0x7f2d00154930 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c330 tx=0x7f2cec001a40 comp rx=0 tx=0).stop 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 shutdown_connections 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ce006e0c0 0x7f2ce0070580 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d000cb2d0 0x7f2d000cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d000cf940 0x7f2d00154930 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 --2- 172.21.15.67:0/108106496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ce4003f60 0x7f2d0014b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 >> 172.21.15.67:0/108106496 conn(0x7f2d0006b900 msgr2=0x7f2d0006f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 shutdown_connections 2024-05-30T17:02:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.515+0000 7f2cffe00640 1 -- 172.21.15.67:0/108106496 wait complete. 2024-05-30T17:02:29.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-30T17:02:29.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.756+0000 7f4999000640 1 Processor -- start 2024-05-30T17:02:29.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.756+0000 7f4999000640 1 -- start start 2024-05-30T17:02:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.757+0000 7f4999000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f499406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.758+0000 7f4999000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499406fa00 con 0x7f49941600f0 2024-05-30T17:02:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.758+0000 7f4999000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499406eeb0 con 0x7f49940d0b40 2024-05-30T17:02:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.758+0000 7f4999000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f499406f160 con 0x7f499415fa00 2024-05-30T17:02:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.758+0000 7f4992a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f49941600f0 0x7f49940d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52274/0 (socket says 172.21.15.67:52274) 2024-05-30T17:02:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.758+0000 7f4992a00640 1 -- 172.21.15.67:0/2053270655 learned_addr learned my addr 172.21.15.67:0/2053270655 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.758+0000 7f4993400640 1 --2- 172.21.15.67:0/2053270655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f499406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.759+0000 7f4992000640 1 -- 172.21.15.67:0/2053270655 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010131676 0 0) 0x7f499406fa00 con 0x7f49941600f0 2024-05-30T17:02:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.759+0000 7f4992000640 1 -- 172.21.15.67:0/2053270655 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4978003520 con 0x7f49941600f0 2024-05-30T17:02:29.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.759+0000 7f4993400640 1 -- 172.21.15.67:0/2053270655 >> v1:172.21.15.169:6789/0 conn(0x7f49940d0b40 legacy=0x7f4994072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:29.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.759+0000 7f4993400640 1 -- 172.21.15.67:0/2053270655 >> v1:172.21.15.67:6789/0 conn(0x7f49941600f0 legacy=0x7f49940d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:29.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.759+0000 7f4992000640 1 -- 172.21.15.67:0/2053270655 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2526634755 0 0) 0x7f4978003520 con 0x7f49941600f0 2024-05-30T17:02:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.759+0000 7f4993400640 1 -- 172.21.15.67:0/2053270655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499406f440 con 0x7f499415fa00 2024-05-30T17:02:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.761+0000 7f4993400640 1 --2- 172.21.15.67:0/2053270655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f499406e970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f497c0097d0 tx=0x7f497c02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b5ee45258458b42 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.761+0000 7f4992000640 1 -- 172.21.15.67:0/2053270655 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c037070 con 0x7f499415fa00 2024-05-30T17:02:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.761+0000 7f4992000640 1 -- 172.21.15.67:0/2053270655 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c02ab10 con 0x7f499415fa00 2024-05-30T17:02:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.761+0000 7f4992000640 1 -- 172.21.15.67:0/2053270655 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984001690 con 0x7f499415fa00 2024-05-30T17:02:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.762+0000 7f4999000640 1 -- 172.21.15.67:0/2053270655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 msgr2=0x7f499406e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.762+0000 7f4999000640 1 --2- 172.21.15.67:0/2053270655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f499406e970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f497c0097d0 tx=0x7f497c02a080 comp rx=0 tx=0).stop 2024-05-30T17:02:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.762+0000 7f4999000640 1 -- 172.21.15.67:0/2053270655 shutdown_connections 2024-05-30T17:02:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.762+0000 7f4999000640 1 --2- 172.21.15.67:0/2053270655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f499406e970 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.762+0000 7f4999000640 1 -- 172.21.15.67:0/2053270655 >> 172.21.15.67:0/2053270655 conn(0x7f499406bae0 msgr2=0x7f499406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.762+0000 7f4999000640 1 -- 172.21.15.67:0/2053270655 shutdown_connections 2024-05-30T17:02:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.762+0000 7f4999000640 1 -- 172.21.15.67:0/2053270655 wait complete. 2024-05-30T17:02:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.765+0000 7f4999000640 1 Processor -- start 2024-05-30T17:02:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.765+0000 7f4999000640 1 -- start start 2024-05-30T17:02:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.766+0000 7f4999000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49940d0b40 0x7f499415a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.766+0000 7f4999000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f49940af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.766+0000 7f4999000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49941600f0 0x7f4994159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.766+0000 7f4999000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f499415fde0 con 0x7f49940d0b40 2024-05-30T17:02:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.766+0000 7f4999000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4994071c70 con 0x7f49941600f0 2024-05-30T17:02:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.766+0000 7f4999000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f49940728a0 con 0x7f499415fa00 2024-05-30T17:02:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.766+0000 7f4993400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49940d0b40 0x7f499415a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.766+0000 7f4993400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49940d0b40 0x7f499415a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60052/0 (socket says 172.21.15.67:60052) 2024-05-30T17:02:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.766+0000 7f4993400640 1 -- 172.21.15.67:0/1987546563 learned_addr learned my addr 172.21.15.67:0/1987546563 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4992a00640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f49940af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4993e00640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49941600f0 0x7f4994159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4993400640 1 -- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 msgr2=0x7f49940af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4993400640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f49940af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4993400640 1 -- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49941600f0 msgr2=0x7f4994159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4993400640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49941600f0 0x7f4994159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4993400640 1 -- 172.21.15.67:0/1987546563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49941f59f0 con 0x7f49940d0b40 2024-05-30T17:02:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4992a00640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f49940af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4993e00640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49941600f0 0x7f4994159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.767+0000 7f4993400640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49940d0b40 0x7f499415a9c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f497c0097d0 tx=0x7f497c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.768+0000 7f498b400640 1 -- 172.21.15.67:0/1987546563 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c041070 con 0x7f49940d0b40 2024-05-30T17:02:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.768+0000 7f498b400640 1 -- 172.21.15.67:0/1987546563 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c03c680 con 0x7f49940d0b40 2024-05-30T17:02:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.768+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49941f5c90 con 0x7f49940d0b40 2024-05-30T17:02:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.768+0000 7f498b400640 1 -- 172.21.15.67:0/1987546563 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c037040 con 0x7f49940d0b40 2024-05-30T17:02:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.768+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49941f6250 con 0x7f49940d0b40 2024-05-30T17:02:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.769+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49940689e0 con 0x7f49940d0b40 2024-05-30T17:02:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.770+0000 7f498b400640 1 -- 172.21.15.67:0/1987546563 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f497c044070 con 0x7f49940d0b40 2024-05-30T17:02:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.771+0000 7f498b400640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f497006e090 0x7f4970070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.771+0000 7f498b400640 1 -- 172.21.15.67:0/1987546563 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5907+0+0 (secure 0 0 0) 0x7f497c0ab420 con 0x7f49940d0b40 2024-05-30T17:02:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.771+0000 7f4992a00640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f497006e090 0x7f4970070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.772+0000 7f4992a00640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f497006e090 0x7f4970070550 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4984001200 tx=0x7f4984009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:29.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.775+0000 7f498b400640 1 -- 172.21.15.67:0/1987546563 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f497c07b2d0 con 0x7f49940d0b40 2024-05-30T17:02:29.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:29.981+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f49940af6e0 con 0x7f49940d0b40 2024-05-30T17:02:30.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.878+0000 7f498b400640 1 -- 172.21.15.67:0/1987546563 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) ==== 181+0+0 (secure 0 0 0) 0x7f497c07abd0 con 0x7f49940d0b40 2024-05-30T17:02:30.880 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-05-30T17:02:30.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.881+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f497006e090 msgr2=0x7f4970070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:30.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.881+0000 7f4999000640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f497006e090 0x7f4970070550 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4984001200 tx=0x7f4984009040 comp rx=0 tx=0).stop 2024-05-30T17:02:30.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.881+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49940d0b40 msgr2=0x7f499415a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:30.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.881+0000 7f4999000640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49940d0b40 0x7f499415a9c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f497c0097d0 tx=0x7f497c0046c0 comp rx=0 tx=0).stop 2024-05-30T17:02:30.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.882+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 shutdown_connections 2024-05-30T17:02:30.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.882+0000 7f4999000640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f497006e090 0x7f4970070550 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:30.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.882+0000 7f4999000640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49941600f0 0x7f4994159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:30.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.882+0000 7f4999000640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f499415fa00 0x7f49940af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:30.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.882+0000 7f4999000640 1 --2- 172.21.15.67:0/1987546563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49940d0b40 0x7f499415a9c0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:30.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.882+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 >> 172.21.15.67:0/1987546563 conn(0x7f499406bae0 msgr2=0x7f49940d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:30.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.883+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 shutdown_connections 2024-05-30T17:02:30.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:30.885+0000 7f4999000640 1 -- 172.21.15.67:0/1987546563 wait complete. 2024-05-30T17:02:30.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-30T17:02:31.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.128+0000 7faf91400640 1 Processor -- start 2024-05-30T17:02:31.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.128+0000 7faf91400640 1 -- start start 2024-05-30T17:02:31.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.130+0000 7faf91400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faf8c162940 0x7faf8c06e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:31.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.130+0000 7faf91400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0cf800 0x7faf8c0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:31.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.130+0000 7faf91400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf8c0b49f0 con 0x7faf8c162d20 2024-05-30T17:02:31.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.130+0000 7faf91400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faf8c0b4cd0 con 0x7faf8c162940 2024-05-30T17:02:31.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.130+0000 7faf91400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faf8c0cf020 con 0x7faf8c0cf800 2024-05-30T17:02:31.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.131+0000 7faf8b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faf8c162d20 0x7faf8c0b4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36872/0 (socket says 172.21.15.67:36872) 2024-05-30T17:02:31.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.131+0000 7faf8b400640 1 -- 172.21.15.67:0/2236836304 learned_addr learned my addr 172.21.15.67:0/2236836304 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:31.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.131+0000 7faf90a00640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0cf800 0x7faf8c0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:31.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.131+0000 7faf8be00640 1 --2- 172.21.15.67:0/2236836304 >> v2:172.21.15.169:3300/0 conn(0x7faf8c162940 0x7faf8c06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:31.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.131+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2060888237 0 0) 0x7faf8c0b49f0 con 0x7faf8c162d20 2024-05-30T17:02:31.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.131+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf70003520 con 0x7faf8c162d20 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1749696445 0 0) 0x7faf70003520 con 0x7faf8c162d20 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0cf800 msgr2=0x7faf8c0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0cf800 0x7faf8c0cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 >> v2:172.21.15.169:3300/0 conn(0x7faf8c162940 msgr2=0x7faf8c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 --2- 172.21.15.67:0/2236836304 >> v2:172.21.15.169:3300/0 conn(0x7faf8c162940 0x7faf8c06e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8c0cecd0 con 0x7faf8c162d20 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7faf78001670 con 0x7faf8c162d20 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 >> v1:172.21.15.67:6789/0 conn(0x7faf8c162d20 legacy=0x7faf8c0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0cf800 0x7faf700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf70004d40 0x7faf700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c0b4cd0 con 0x7faf70003f80 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c0cf020 con 0x7faf70004d40 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faf7000baf0 con 0x7faf8c0cf800 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.132+0000 7faf8b400640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.133+0000 7faf90a00640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf70004d40 0x7faf700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.133+0000 7faf8be00640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0cf800 0x7faf700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.133+0000 7faf8b400640 1 -- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0cf800 msgr2=0x7faf700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.133+0000 7faf8b400640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0cf800 0x7faf700039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.133+0000 7faf8b400640 1 -- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf70004d40 msgr2=0x7faf700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.133+0000 7faf8b400640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf70004d40 0x7faf700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.133+0000 7faf8b400640 1 -- 172.21.15.67:0/2236836304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf70003520 con 0x7faf70003f80 2024-05-30T17:02:31.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.133+0000 7faf90a00640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf70004d40 0x7faf700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:31.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.134+0000 7faf8b400640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf70004800 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0b4920 tx=0x7faf70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bb5fe0003c0cea server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:31.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.134+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78007c80 con 0x7faf70003f80 2024-05-30T17:02:31.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.134+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf78005ad0 con 0x7faf70003f80 2024-05-30T17:02:31.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.134+0000 7faf8aa00640 1 -- 172.21.15.67:0/2236836304 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78007070 con 0x7faf70003f80 2024-05-30T17:02:31.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.135+0000 7faf91400640 1 -- 172.21.15.67:0/2236836304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 msgr2=0x7faf70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:31.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.135+0000 7faf91400640 1 --2- 172.21.15.67:0/2236836304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf70004800 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0b4920 tx=0x7faf70005830 comp rx=0 tx=0).stop 2024-05-30T17:02:31.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.135+0000 7faf90a00640 1 -- 172.21.15.67:0/2236836304 reap_dead start 2024-05-30T17:02:31.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.135+0000 7faf91400640 1 -- 172.21.15.67:0/2236836304 shutdown_connections 2024-05-30T17:02:31.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.135+0000 7faf91400640 1 -- 172.21.15.67:0/2236836304 >> 172.21.15.67:0/2236836304 conn(0x7faf8c06bae0 msgr2=0x7faf8c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:31.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.135+0000 7faf91400640 1 -- 172.21.15.67:0/2236836304 shutdown_connections 2024-05-30T17:02:31.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.135+0000 7faf91400640 1 -- 172.21.15.67:0/2236836304 wait complete. 2024-05-30T17:02:31.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.137+0000 7faf91400640 1 Processor -- start 2024-05-30T17:02:31.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.137+0000 7faf91400640 1 -- start start 2024-05-30T17:02:31.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf91400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf8c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf91400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0aed40 0x7faf8c0af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf91400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf8c1f8c70 0x7faf8c1f9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf91400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c0d0020 con 0x7faf70003f80 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf91400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c0cfea0 con 0x7faf8c1f8c70 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf91400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faf8c071b40 con 0x7faf8c0aed40 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf8be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf8c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf8be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf8c0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59614/0 (socket says 172.21.15.67:59614) 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf8be00640 1 -- 172.21.15.67:0/2748330562 learned_addr learned my addr 172.21.15.67:0/2748330562 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf8be00640 1 -- 172.21.15.67:0/2748330562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0aed40 msgr2=0x7faf8c0af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf8be00640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0aed40 0x7faf8c0af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf8be00640 1 -- 172.21.15.67:0/2748330562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf8c1f8c70 msgr2=0x7faf8c1f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:31.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf8be00640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf8c1f8c70 0x7faf8c1f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.138+0000 7faf8be00640 1 -- 172.21.15.67:0/2748330562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8c0ccb30 con 0x7faf70003f80 2024-05-30T17:02:31.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.139+0000 7faf8be00640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf8c0b0950 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0108e0 tx=0x7faf7c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:31.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.139+0000 7faf83e00640 1 -- 172.21.15.67:0/2748330562 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00f040 con 0x7faf70003f80 2024-05-30T17:02:31.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.139+0000 7faf83e00640 1 -- 172.21.15.67:0/2748330562 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7c00ea80 con 0x7faf70003f80 2024-05-30T17:02:31.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.139+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf8c0af8b0 con 0x7faf70003f80 2024-05-30T17:02:31.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.139+0000 7faf83e00640 1 -- 172.21.15.67:0/2748330562 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00ee50 con 0x7faf70003f80 2024-05-30T17:02:31.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.139+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf8c0afe70 con 0x7faf70003f80 2024-05-30T17:02:31.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.140+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf8c0689e0 con 0x7faf70003f80 2024-05-30T17:02:31.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.141+0000 7faf83e00640 1 -- 172.21.15.67:0/2748330562 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7faf7c01c400 con 0x7faf70003f80 2024-05-30T17:02:31.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.142+0000 7faf83e00640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faf6c06e0c0 0x7faf6c070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:31.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.142+0000 7faf83e00640 1 -- 172.21.15.67:0/2748330562 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5469+0+0 (secure 0 0 0) 0x7faf7c08b000 con 0x7faf70003f80 2024-05-30T17:02:31.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.142+0000 7faf8b400640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faf6c06e0c0 0x7faf6c070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:31.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.143+0000 7faf8b400640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faf6c06e0c0 0x7faf6c070580 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7faf78032000 tx=0x7faf78007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:31.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.147+0000 7faf83e00640 1 -- 172.21.15.67:0/2748330562 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf7c018030 con 0x7faf70003f80 2024-05-30T17:02:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.353+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faf8c0b3d10 con 0x7faf70003f80 2024-05-30T17:02:31.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.881+0000 7faf83e00640 1 -- 172.21.15.67:0/2748330562 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7faf7c05ae90 con 0x7faf70003f80 2024-05-30T17:02:31.883 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-05-30T17:02:31.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.885+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faf6c06e0c0 msgr2=0x7faf6c070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:31.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.885+0000 7faf91400640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faf6c06e0c0 0x7faf6c070580 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7faf78032000 tx=0x7faf78007000 comp rx=0 tx=0).stop 2024-05-30T17:02:31.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.886+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 msgr2=0x7faf8c0b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:31.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.886+0000 7faf91400640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf8c0b0950 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0108e0 tx=0x7faf7c010da0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.886+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 shutdown_connections 2024-05-30T17:02:31.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.886+0000 7faf91400640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7faf6c06e0c0 0x7faf6c070580 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.886+0000 7faf91400640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf8c1f8c70 0x7faf8c1f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.886+0000 7faf91400640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf8c0aed40 0x7faf8c0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.886+0000 7faf91400640 1 --2- 172.21.15.67:0/2748330562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf70003f80 0x7faf8c0b0950 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:31.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.886+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 >> 172.21.15.67:0/2748330562 conn(0x7faf8c06bae0 msgr2=0x7faf8c0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:31.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.888+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 shutdown_connections 2024-05-30T17:02:31.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:31.889+0000 7faf91400640 1 -- 172.21.15.67:0/2748330562 wait complete. 2024-05-30T17:02:31.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-30T17:02:32.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.113+0000 7f6cc0a00640 1 Processor -- start 2024-05-30T17:02:32.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.113+0000 7f6cc0a00640 1 -- start start 2024-05-30T17:02:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.114+0000 7f6cc0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc153510 0x7f6cbc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.114+0000 7f6cc0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6cbc15d270 0x7f6cbc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.114+0000 7f6cc0a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cbc15d900 con 0x7f6cbc0ccb90 2024-05-30T17:02:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.114+0000 7f6cc0a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc1538f0 con 0x7f6cbc15d270 2024-05-30T17:02:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.114+0000 7f6cc0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc0cd530 con 0x7f6cbc153510 2024-05-30T17:02:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.114+0000 7f6cbaa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6cbc0ccb90 0x7f6cbc15cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36882/0 (socket says 172.21.15.67:36882) 2024-05-30T17:02:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.114+0000 7f6cbaa00640 1 -- 172.21.15.67:0/4272008376 learned_addr learned my addr 172.21.15.67:0/4272008376 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.114+0000 7f6cbbe00640 1 --2- 172.21.15.67:0/4272008376 >> v2:172.21.15.67:3300/0 conn(0x7f6cbc15d270 0x7f6cbc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.115+0000 7f6cbb400640 1 --2- 172.21.15.67:0/4272008376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc153510 0x7f6cbc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.115+0000 7f6cbbe00640 1 -- 172.21.15.67:0/4272008376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc153510 msgr2=0x7f6cbc0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:32.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.115+0000 7f6cbbe00640 1 --2- 172.21.15.67:0/4272008376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc153510 0x7f6cbc0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.115+0000 7f6cbbe00640 1 -- 172.21.15.67:0/4272008376 >> v1:172.21.15.67:6789/0 conn(0x7f6cbc0ccb90 legacy=0x7f6cbc15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.115+0000 7f6cbbe00640 1 -- 172.21.15.67:0/4272008376 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc0b3f90 con 0x7f6cbc15d270 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.116+0000 7f6cbbe00640 1 --2- 172.21.15.67:0/4272008376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc15d270 0x7f6cbc0ccff0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40097a0 tx=0x7f6cb402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeae1162913a34ef server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.116+0000 7f6cba000640 1 -- 172.21.15.67:0/4272008376 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4037070 con 0x7f6cbc15d270 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.116+0000 7f6cba000640 1 -- 172.21.15.67:0/4272008376 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb402ae40 con 0x7f6cbc15d270 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.116+0000 7f6cba000640 1 -- 172.21.15.67:0/4272008376 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4032660 con 0x7f6cbc15d270 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.117+0000 7f6cc0a00640 1 -- 172.21.15.67:0/4272008376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc15d270 msgr2=0x7f6cbc0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.117+0000 7f6cc0a00640 1 --2- 172.21.15.67:0/4272008376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc15d270 0x7f6cbc0ccff0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40097a0 tx=0x7f6cb402a0f0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.117+0000 7f6cc0a00640 1 -- 172.21.15.67:0/4272008376 shutdown_connections 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.117+0000 7f6cc0a00640 1 --2- 172.21.15.67:0/4272008376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc15d270 0x7f6cbc0ccff0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.117+0000 7f6cc0a00640 1 --2- 172.21.15.67:0/4272008376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc153510 0x7f6cbc0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.117+0000 7f6cc0a00640 1 -- 172.21.15.67:0/4272008376 >> 172.21.15.67:0/4272008376 conn(0x7f6cbc06b900 msgr2=0x7f6cbc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.117+0000 7f6cc0a00640 1 -- 172.21.15.67:0/4272008376 shutdown_connections 2024-05-30T17:02:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.117+0000 7f6cc0a00640 1 -- 172.21.15.67:0/4272008376 wait complete. 2024-05-30T17:02:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.119+0000 7f6cc0a00640 1 Processor -- start 2024-05-30T17:02:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.119+0000 7f6cc0a00640 1 -- start start 2024-05-30T17:02:32.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.120+0000 7f6cc0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc0ccb90 0x7f6cbc1ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.120+0000 7f6cc0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc153510 0x7f6cbc073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.120+0000 7f6cc0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cbc15d270 0x7f6cbc155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.120+0000 7f6cc0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0b4550 con 0x7f6cbc153510 2024-05-30T17:02:32.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.120+0000 7f6cc0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0b48c0 con 0x7f6cbc15d270 2024-05-30T17:02:32.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.120+0000 7f6cc0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc06e940 con 0x7f6cbc0ccb90 2024-05-30T17:02:32.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.120+0000 7f6cbaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc153510 0x7f6cbc073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc153510 0x7f6cbc073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59636/0 (socket says 172.21.15.67:59636) 2024-05-30T17:02:32.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbaa00640 1 -- 172.21.15.67:0/2791936597 learned_addr learned my addr 172.21.15.67:0/2791936597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbb400640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc0ccb90 0x7f6cbc1ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbbe00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cbc15d270 0x7f6cbc155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbaa00640 1 -- 172.21.15.67:0/2791936597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc0ccb90 msgr2=0x7f6cbc1ee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbaa00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc0ccb90 0x7f6cbc1ee3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbaa00640 1 -- 172.21.15.67:0/2791936597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cbc15d270 msgr2=0x7f6cbc155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbaa00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cbc15d270 0x7f6cbc155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbaa00640 1 -- 172.21.15.67:0/2791936597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc071ce0 con 0x7f6cbc153510 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.121+0000 7f6cbaa00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc153510 0x7f6cbc073020 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800c400 tx=0x7f6ca800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.122+0000 7f6cb3400640 1 -- 172.21.15.67:0/2791936597 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8017070 con 0x7f6cbc153510 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.122+0000 7f6cb3400640 1 -- 172.21.15.67:0/2791936597 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca8005ce0 con 0x7f6cbc153510 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.122+0000 7f6cb3400640 1 -- 172.21.15.67:0/2791936597 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80072f0 con 0x7f6cbc153510 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.122+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc071f80 con 0x7f6cbc153510 2024-05-30T17:02:32.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.122+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc072540 con 0x7f6cbc153510 2024-05-30T17:02:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.123+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc0689e0 con 0x7f6cbc153510 2024-05-30T17:02:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.124+0000 7f6cb3400640 1 -- 172.21.15.67:0/2791936597 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f6ca801a070 con 0x7f6cbc153510 2024-05-30T17:02:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.125+0000 7f6cb3400640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6c9c06e090 0x7f6c9c070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.125+0000 7f6cb3400640 1 -- 172.21.15.67:0/2791936597 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f6ca8089cd0 con 0x7f6cbc153510 2024-05-30T17:02:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.125+0000 7f6cbb400640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6c9c06e090 0x7f6c9c070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.126+0000 7f6cbb400640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6c9c06e090 0x7f6c9c070550 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009a60 tx=0x7f6cac003990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.131+0000 7f6cb3400640 1 -- 172.21.15.67:0/2791936597 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ca8059c00 con 0x7f6cbc153510 2024-05-30T17:02:32.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.358+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6cbc06be10 con 0x7f6cbc153510 2024-05-30T17:02:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.362+0000 7f6cb3400640 1 -- 172.21.15.67:0/2791936597 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f6ca8059500 con 0x7f6cbc153510 2024-05-30T17:02:32.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6c9c06e090 msgr2=0x7f6c9c070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6c9c06e090 0x7f6c9c070550 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009a60 tx=0x7f6cac003990 comp rx=0 tx=0).stop 2024-05-30T17:02:32.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc153510 msgr2=0x7f6cbc073020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc153510 0x7f6cbc073020 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800c400 tx=0x7f6ca800c8c0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 shutdown_connections 2024-05-30T17:02:32.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6c9c06e090 0x7f6c9c070550 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cbc15d270 0x7f6cbc155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc153510 0x7f6cbc073020 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 --2- 172.21.15.67:0/2791936597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6cbc0ccb90 0x7f6cbc1ee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 >> 172.21.15.67:0/2791936597 conn(0x7f6cbc06b900 msgr2=0x7f6cbc06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 shutdown_connections 2024-05-30T17:02:32.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.365+0000 7f6cc0a00640 1 -- 172.21.15.67:0/2791936597 wait complete. 2024-05-30T17:02:32.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled' 2024-05-30T17:02:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.575+0000 7f0164a00640 1 Processor -- start 2024-05-30T17:02:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.575+0000 7f0164a00640 1 -- start start 2024-05-30T17:02:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.576+0000 7f0164a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0160156b30 0x7f01600af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.576+0000 7f0164a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f016006ee40 0x7f01600ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.576+0000 7f0164a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0160156f10 con 0x7f016006f2f0 2024-05-30T17:02:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.576+0000 7f0164a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f016006dda0 con 0x7f0160156b30 2024-05-30T17:02:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.576+0000 7f0164a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f016006e080 con 0x7f016006ee40 2024-05-30T17:02:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.577+0000 7f015e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0160156b30 0x7f01600af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.577+0000 7f015e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0160156b30 0x7f01600af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59646/0 (socket says 172.21.15.67:59646) 2024-05-30T17:02:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.577+0000 7f015e800640 1 -- 172.21.15.67:0/3881727673 learned_addr learned my addr 172.21.15.67:0/3881727673 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.577+0000 7f0155e00640 1 --2- 172.21.15.67:0/3881727673 >> v2:172.21.15.169:3300/0 conn(0x7f016006ee40 0x7f01600ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.577+0000 7f015e800640 1 -- 172.21.15.67:0/3881727673 >> v2:172.21.15.169:3300/0 conn(0x7f016006ee40 msgr2=0x7f01600ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.577+0000 7f015e800640 1 --2- 172.21.15.67:0/3881727673 >> v2:172.21.15.169:3300/0 conn(0x7f016006ee40 0x7f01600ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.577+0000 7f015e800640 1 -- 172.21.15.67:0/3881727673 >> v1:172.21.15.67:6789/0 conn(0x7f016006f2f0 legacy=0x7f01600b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.577+0000 7f015e800640 1 -- 172.21.15.67:0/3881727673 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016006e360 con 0x7f0160156b30 2024-05-30T17:02:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.579+0000 7f015e800640 1 --2- 172.21.15.67:0/3881727673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0160156b30 0x7f01600af6e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f016006fdf0 tx=0x7f015002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5c8320e7d82c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.579+0000 7f015de00640 1 -- 172.21.15.67:0/3881727673 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150037070 con 0x7f0160156b30 2024-05-30T17:02:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.579+0000 7f015de00640 1 -- 172.21.15.67:0/3881727673 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0150032440 con 0x7f0160156b30 2024-05-30T17:02:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.579+0000 7f015de00640 1 -- 172.21.15.67:0/3881727673 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150032810 con 0x7f0160156b30 2024-05-30T17:02:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.579+0000 7f0164a00640 1 -- 172.21.15.67:0/3881727673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0160156b30 msgr2=0x7f01600af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.579+0000 7f0164a00640 1 --2- 172.21.15.67:0/3881727673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0160156b30 0x7f01600af6e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f016006fdf0 tx=0x7f015002a1e0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.580+0000 7f0164a00640 1 -- 172.21.15.67:0/3881727673 shutdown_connections 2024-05-30T17:02:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.580+0000 7f0164a00640 1 --2- 172.21.15.67:0/3881727673 >> v2:172.21.15.169:3300/0 conn(0x7f016006ee40 0x7f01600ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.580+0000 7f0164a00640 1 --2- 172.21.15.67:0/3881727673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0160156b30 0x7f01600af6e0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.580+0000 7f0164a00640 1 -- 172.21.15.67:0/3881727673 >> 172.21.15.67:0/3881727673 conn(0x7f01600b1fc0 msgr2=0x7f01600b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.580+0000 7f0164a00640 1 -- 172.21.15.67:0/3881727673 shutdown_connections 2024-05-30T17:02:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.580+0000 7f0164a00640 1 -- 172.21.15.67:0/3881727673 wait complete. 2024-05-30T17:02:32.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.581+0000 7f0164a00640 1 Processor -- start 2024-05-30T17:02:32.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.582+0000 7f0164a00640 1 -- start start 2024-05-30T17:02:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.582+0000 7f0164a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016006ee40 0x7f01601f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.582+0000 7f0164a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016006f2f0 0x7f016015f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.582+0000 7f0164a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0160156b30 0x7f01601570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.582+0000 7f0164a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f01600afc20 con 0x7f016006ee40 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.582+0000 7f0164a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01600b2c30 con 0x7f016006f2f0 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.582+0000 7f0164a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f016015fb40 con 0x7f0160156b30 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f015e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016006ee40 0x7f01601f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f015e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016006ee40 0x7f01601f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59656/0 (socket says 172.21.15.67:59656) 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f015e800640 1 -- 172.21.15.67:0/1495640230 learned_addr learned my addr 172.21.15.67:0/1495640230 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f0155e00640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016006f2f0 0x7f016015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f015e800640 1 -- 172.21.15.67:0/1495640230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0160156b30 msgr2=0x7f01601570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f015e800640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0160156b30 0x7f01601570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f015e800640 1 -- 172.21.15.67:0/1495640230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016006f2f0 msgr2=0x7f016015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f015e800640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016006f2f0 0x7f016015f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f015e800640 1 -- 172.21.15.67:0/1495640230 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01601f5d30 con 0x7f016006ee40 2024-05-30T17:02:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.583+0000 7f015e800640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016006ee40 0x7f01601f56e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f015002a220 tx=0x7f01500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.584+0000 7f0157400640 1 -- 172.21.15.67:0/1495640230 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150037040 con 0x7f016006ee40 2024-05-30T17:02:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.584+0000 7f0157400640 1 -- 172.21.15.67:0/1495640230 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0150042070 con 0x7f016006ee40 2024-05-30T17:02:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.584+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01601f6010 con 0x7f016006ee40 2024-05-30T17:02:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.584+0000 7f0157400640 1 -- 172.21.15.67:0/1495640230 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150030bb0 con 0x7f016006ee40 2024-05-30T17:02:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.584+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01601f68b0 con 0x7f016006ee40 2024-05-30T17:02:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.585+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01601f6400 con 0x7f016006ee40 2024-05-30T17:02:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.586+0000 7f0157400640 1 -- 172.21.15.67:0/1495640230 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f0150047460 con 0x7f016006ee40 2024-05-30T17:02:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.588+0000 7f0157400640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f013c06e090 0x7f013c070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.588+0000 7f0157400640 1 -- 172.21.15.67:0/1495640230 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f01500ab570 con 0x7f016006ee40 2024-05-30T17:02:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.588+0000 7f0155e00640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f013c06e090 0x7f013c070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.589+0000 7f0155e00640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f013c06e090 0x7f013c070550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0148006fd0 tx=0x7f0148008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:32.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.592+0000 7f0157400640 1 -- 172.21.15.67:0/1495640230 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0150045360 con 0x7f016006ee40 2024-05-30T17:02:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.874+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]} v 0) -- 0x7f01600689e0 con 0x7f016006ee40 2024-05-30T17:02:32.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.900+0000 7f0157400640 1 -- 172.21.15.67:0/1495640230 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]}]=0 v779) ==== 161+0+0 (secure 0 0 0) 0x7f015007b4a0 con 0x7f016006ee40 2024-05-30T17:02:32.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f013c06e090 msgr2=0x7f013c070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f013c06e090 0x7f013c070550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0148006fd0 tx=0x7f0148008040 comp rx=0 tx=0).stop 2024-05-30T17:02:32.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016006ee40 msgr2=0x7f01601f56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:32.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016006ee40 0x7f01601f56e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f015002a220 tx=0x7f01500046c0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 shutdown_connections 2024-05-30T17:02:32.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f013c06e090 0x7f013c070550 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0160156b30 0x7f01601570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f016006f2f0 0x7f016015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 --2- 172.21.15.67:0/1495640230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016006ee40 0x7f01601f56e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:32.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.902+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 >> 172.21.15.67:0/1495640230 conn(0x7f01600b1fc0 msgr2=0x7f01600b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:32.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.903+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 shutdown_connections 2024-05-30T17:02:32.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:32.903+0000 7f0164a00640 1 -- 172.21.15.67:0/1495640230 wait complete. 2024-05-30T17:02:32.924 INFO:tasks.cephfs_test_runner:Test that the progress bar, that is printed for 1 ongoing clone job, is ... ok 2024-05-30T17:02:32.924 INFO:tasks.cephfs_test_runner:Starting test: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-05-30T17:02:32.925 INFO:tasks.cephfs_test_runner:test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T17:02:32.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:02:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.139+0000 7fac85200640 1 Processor -- start 2024-05-30T17:02:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.140+0000 7fac85200640 1 -- start start 2024-05-30T17:02:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.141+0000 7fac85200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8015e870 0x7fac800d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.141+0000 7fac85200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac8015ec50 con 0x7fac800cfa40 2024-05-30T17:02:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.141+0000 7fac85200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac800cff80 con 0x7fac800d14d0 2024-05-30T17:02:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.141+0000 7fac85200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac800d2150 con 0x7fac8015e870 2024-05-30T17:02:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.142+0000 7fac7fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fac800cfa40 0x7fac800d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36892/0 (socket says 172.21.15.67:36892) 2024-05-30T17:02:33.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.142+0000 7fac7fe00640 1 -- 172.21.15.67:0/1975673331 learned_addr learned my addr 172.21.15.67:0/1975673331 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:33.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.142+0000 7fac7f400640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8015e870 0x7fac800d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.142+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247607145 0 0) 0x7fac8015ec50 con 0x7fac800cfa40 2024-05-30T17:02:33.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.142+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac64003520 con 0x7fac800cfa40 2024-05-30T17:02:33.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1374124808 0 0) 0x7fac64003520 con 0x7fac800cfa40 2024-05-30T17:02:33.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8015e870 msgr2=0x7fac800d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8015e870 0x7fac800d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 >> v1:172.21.15.169:6789/0 conn(0x7fac800d14d0 legacy=0x7fac800cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac800d0260 con 0x7fac800cfa40 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fac78001670 con 0x7fac800cfa40 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 >> v1:172.21.15.67:6789/0 conn(0x7fac800cfa40 legacy=0x7fac800d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8015e870 0x7fac640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64003f20 0x7fac64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac64004d40 0x7fac640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fac800d2150 con 0x7fac64004d40 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac6400ba70 con 0x7fac64003f20 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac6400bc70 con 0x7fac8015e870 2024-05-30T17:02:33.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.143+0000 7fac7fe00640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac64004d40 0x7fac640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.144+0000 7fac7ea00640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64003f20 0x7fac64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.144+0000 7fac7f400640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8015e870 0x7fac640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.144+0000 7fac7fe00640 1 -- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8015e870 msgr2=0x7fac640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:33.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.144+0000 7fac7fe00640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac8015e870 0x7fac640039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.144+0000 7fac7fe00640 1 -- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64003f20 msgr2=0x7fac64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:33.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.144+0000 7fac7fe00640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac64003f20 0x7fac64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.144+0000 7fac7fe00640 1 -- 172.21.15.67:0/1975673331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac64003520 con 0x7fac64004d40 2024-05-30T17:02:33.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.145+0000 7fac7fe00640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac64004d40 0x7fac640051c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fac800cfeb0 tx=0x7fac64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e422f86fd7409ae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:33.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.145+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78007c80 con 0x7fac64004d40 2024-05-30T17:02:33.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.145+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac78005af0 con 0x7fac64004d40 2024-05-30T17:02:33.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.145+0000 7fac7e000640 1 -- 172.21.15.67:0/1975673331 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78007070 con 0x7fac64004d40 2024-05-30T17:02:33.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.146+0000 7fac85200640 1 -- 172.21.15.67:0/1975673331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac64004d40 msgr2=0x7fac640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.146+0000 7fac85200640 1 --2- 172.21.15.67:0/1975673331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac64004d40 0x7fac640051c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fac800cfeb0 tx=0x7fac64005830 comp rx=0 tx=0).stop 2024-05-30T17:02:33.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.146+0000 7fac7fe00640 1 -- 172.21.15.67:0/1975673331 reap_dead start 2024-05-30T17:02:33.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.146+0000 7fac85200640 1 -- 172.21.15.67:0/1975673331 shutdown_connections 2024-05-30T17:02:33.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.146+0000 7fac85200640 1 -- 172.21.15.67:0/1975673331 >> 172.21.15.67:0/1975673331 conn(0x7fac8008bfc0 msgr2=0x7fac800d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:33.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.146+0000 7fac85200640 1 -- 172.21.15.67:0/1975673331 shutdown_connections 2024-05-30T17:02:33.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.146+0000 7fac85200640 1 -- 172.21.15.67:0/1975673331 wait complete. 2024-05-30T17:02:33.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.148+0000 7fac85200640 1 Processor -- start 2024-05-30T17:02:33.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.148+0000 7fac85200640 1 -- start start 2024-05-30T17:02:33.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.149+0000 7fac85200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac64004d40 0x7fac800cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.149+0000 7fac85200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac800cdd40 0x7fac800ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.149+0000 7fac85200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac80160410 0x7fac80160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.149+0000 7fac85200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fac800d4fb0 con 0x7fac800cdd40 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.149+0000 7fac85200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac800d0820 con 0x7fac80160410 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.149+0000 7fac85200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac800d0f90 con 0x7fac64004d40 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.149+0000 7fac7ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac800cdd40 0x7fac800ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.149+0000 7fac7ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac800cdd40 0x7fac800ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59668/0 (socket says 172.21.15.67:59668) 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.149+0000 7fac7ea00640 1 -- 172.21.15.67:0/3705114902 learned_addr learned my addr 172.21.15.67:0/3705114902 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac7fe00640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac80160410 0x7fac80160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac7ea00640 1 -- 172.21.15.67:0/3705114902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac64004d40 msgr2=0x7fac800cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac7f400640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac64004d40 0x7fac800cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac7ea00640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac64004d40 0x7fac800cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac7ea00640 1 -- 172.21.15.67:0/3705114902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac80160410 msgr2=0x7fac80160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac7ea00640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac80160410 0x7fac80160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac7ea00640 1 -- 172.21.15.67:0/3705114902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac80161d30 con 0x7fac800cdd40 2024-05-30T17:02:33.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac7fe00640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac80160410 0x7fac80160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:33.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac7ea00640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac800cdd40 0x7fac800ce1a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e950 tx=0x7fac7000ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:33.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac77400640 1 -- 172.21.15.67:0/3705114902 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70003bd0 con 0x7fac800cdd40 2024-05-30T17:02:33.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac77400640 1 -- 172.21.15.67:0/3705114902 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac70018070 con 0x7fac800cdd40 2024-05-30T17:02:33.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac77400640 1 -- 172.21.15.67:0/3705114902 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7000c780 con 0x7fac800cdd40 2024-05-30T17:02:33.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac800cb780 con 0x7fac800cdd40 2024-05-30T17:02:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.150+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac80161770 con 0x7fac800cdd40 2024-05-30T17:02:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.151+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac800689e0 con 0x7fac800cdd40 2024-05-30T17:02:33.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.153+0000 7fac77400640 1 -- 172.21.15.67:0/3705114902 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fac7000c920 con 0x7fac800cdd40 2024-05-30T17:02:33.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.154+0000 7fac77400640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fac5c06e0c0 0x7fac5c070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.154+0000 7fac77400640 1 -- 172.21.15.67:0/3705114902 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7fac7008ab10 con 0x7fac800cdd40 2024-05-30T17:02:33.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.154+0000 7fac7f400640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fac5c06e0c0 0x7fac5c070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.155+0000 7fac7f400640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fac5c06e0c0 0x7fac5c070580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fac68009910 tx=0x7fac68008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:33.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.158+0000 7fac77400640 1 -- 172.21.15.67:0/3705114902 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac7005a350 con 0x7fac800cdd40 2024-05-30T17:02:33.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.368+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac800cb950 con 0x7fac800cdd40 2024-05-30T17:02:33.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.370+0000 7fac77400640 1 -- 172.21.15.67:0/3705114902 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+13195 (secure 0 0 0) 0x7fac7005e650 con 0x7fac800cdd40 2024-05-30T17:02:33.373 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:33.373 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":108,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:02:31.879594+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6836/2120049133":"2024-05-31T17:02:28.505312+0000","172.21.15.67:6834/2120049133":"2024-05-31T17:02:28.505312+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fac5c06e0c0 msgr2=0x7fac5c070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fac5c06e0c0 0x7fac5c070580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fac68009910 tx=0x7fac68008040 comp rx=0 tx=0).stop 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac800cdd40 msgr2=0x7fac800ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac800cdd40 0x7fac800ce1a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e950 tx=0x7fac7000ee10 comp rx=0 tx=0).stop 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 shutdown_connections 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fac5c06e0c0 0x7fac5c070580 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac80160410 0x7fac80160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac800cdd40 0x7fac800ce1a0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 --2- 172.21.15.67:0/3705114902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac64004d40 0x7fac800cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.373+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 >> 172.21.15.67:0/3705114902 conn(0x7fac8008bfc0 msgr2=0x7fac800b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:33.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.374+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 shutdown_connections 2024-05-30T17:02:33.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.374+0000 7fac85200640 1 -- 172.21.15.67:0/3705114902 wait complete. 2024-05-30T17:02:33.394 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-30T16:43:40.009725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-30T17:02:33.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-30T17:02:33.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.607+0000 7f3417e00640 1 Processor -- start 2024-05-30T17:02:33.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.607+0000 7f3417e00640 1 -- start start 2024-05-30T17:02:33.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.609+0000 7f3417e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34180d16a0 0x7f34180cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.609+0000 7f3417e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180cf870 0x7f34180d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.609+0000 7f3417e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f341815ec50 con 0x7f341815e870 2024-05-30T17:02:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.609+0000 7f3417e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f34180d2150 con 0x7f34180d16a0 2024-05-30T17:02:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.609+0000 7f3417e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34180cfd20 con 0x7f34180cf870 2024-05-30T17:02:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.609+0000 7f3416a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f341815e870 0x7f34180d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36900/0 (socket says 172.21.15.67:36900) 2024-05-30T17:02:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.609+0000 7f3416a00640 1 -- 172.21.15.67:0/224690530 learned_addr learned my addr 172.21.15.67:0/224690530 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:33.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.609+0000 7f340e000640 1 --2- 172.21.15.67:0/224690530 >> v2:172.21.15.67:3300/0 conn(0x7f34180d16a0 0x7f34180cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.610+0000 7f3417400640 1 --2- 172.21.15.67:0/224690530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180cf870 0x7f34180d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.610+0000 7f340e000640 1 -- 172.21.15.67:0/224690530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180cf870 msgr2=0x7f34180d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.610+0000 7f340e000640 1 --2- 172.21.15.67:0/224690530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180cf870 0x7f34180d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.610+0000 7f340e000640 1 -- 172.21.15.67:0/224690530 >> v1:172.21.15.67:6789/0 conn(0x7f341815e870 legacy=0x7f34180d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.610+0000 7f340e000640 1 -- 172.21.15.67:0/224690530 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34180cff80 con 0x7f34180d16a0 2024-05-30T17:02:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.610+0000 7f3417400640 1 --2- 172.21.15.67:0/224690530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180cf870 0x7f34180d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.611+0000 7f340e000640 1 --2- 172.21.15.67:0/224690530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180d16a0 0x7f34180cf330 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f3400009790 tx=0x7f340002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb4265565b39495 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.611+0000 7f3416000640 1 -- 172.21.15.67:0/224690530 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400037070 con 0x7f34180d16a0 2024-05-30T17:02:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.611+0000 7f3416000640 1 -- 172.21.15.67:0/224690530 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3400032440 con 0x7f34180d16a0 2024-05-30T17:02:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.611+0000 7f3416000640 1 -- 172.21.15.67:0/224690530 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400032810 con 0x7f34180d16a0 2024-05-30T17:02:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.612+0000 7f3417e00640 1 -- 172.21.15.67:0/224690530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180d16a0 msgr2=0x7f34180cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.612+0000 7f3417e00640 1 --2- 172.21.15.67:0/224690530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180d16a0 0x7f34180cf330 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f3400009790 tx=0x7f340002a180 comp rx=0 tx=0).stop 2024-05-30T17:02:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.612+0000 7f3417e00640 1 -- 172.21.15.67:0/224690530 shutdown_connections 2024-05-30T17:02:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.612+0000 7f3417e00640 1 --2- 172.21.15.67:0/224690530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180cf870 0x7f34180d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.612+0000 7f3417e00640 1 --2- 172.21.15.67:0/224690530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180d16a0 0x7f34180cf330 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.612+0000 7f3417e00640 1 -- 172.21.15.67:0/224690530 >> 172.21.15.67:0/224690530 conn(0x7f341808bfc0 msgr2=0x7f34180d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.612+0000 7f3417e00640 1 -- 172.21.15.67:0/224690530 shutdown_connections 2024-05-30T17:02:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.612+0000 7f3417e00640 1 -- 172.21.15.67:0/224690530 wait complete. 2024-05-30T17:02:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.614+0000 7f3417e00640 1 Processor -- start 2024-05-30T17:02:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.614+0000 7f3417e00640 1 -- start start 2024-05-30T17:02:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3417e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180cf870 0x7f3418159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3417e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180d16a0 0x7f341815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3417e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f341815e870 0x7f3418160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3417e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f34180d0820 con 0x7f34180cf870 2024-05-30T17:02:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3417e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34180d0540 con 0x7f341815e870 2024-05-30T17:02:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3417e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34180d4fb0 con 0x7f34180d16a0 2024-05-30T17:02:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3416a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180cf870 0x7f3418159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3416a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180cf870 0x7f3418159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59680/0 (socket says 172.21.15.67:59680) 2024-05-30T17:02:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3416a00640 1 -- 172.21.15.67:0/1152760999 learned_addr learned my addr 172.21.15.67:0/1152760999 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.615+0000 7f3416a00640 1 -- 172.21.15.67:0/1152760999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180d16a0 msgr2=0x7f341815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f3416a00640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180d16a0 0x7f341815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f3416a00640 1 -- 172.21.15.67:0/1152760999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f341815e870 msgr2=0x7f3418160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f3416a00640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f341815e870 0x7f3418160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f3416a00640 1 -- 172.21.15.67:0/1152760999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34181596c0 con 0x7f34180cf870 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f3416a00640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180cf870 0x7f3418159180 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f340800c3c0 tx=0x7f340800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f340f400640 1 -- 172.21.15.67:0/1152760999 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408017070 con 0x7f34180cf870 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f340f400640 1 -- 172.21.15.67:0/1152760999 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3408005ce0 con 0x7f34180cf870 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341815f090 con 0x7f34180cf870 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f340f400640 1 -- 172.21.15.67:0/1152760999 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34080072f0 con 0x7f34180cf870 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.616+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341815f650 con 0x7f34180cf870 2024-05-30T17:02:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.617+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34180689e0 con 0x7f34180cf870 2024-05-30T17:02:33.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.618+0000 7f340f400640 1 -- 172.21.15.67:0/1152760999 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f340801a070 con 0x7f34180cf870 2024-05-30T17:02:33.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.619+0000 7f340f400640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f33f406e090 0x7f33f4070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:33.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.620+0000 7f340f400640 1 -- 172.21.15.67:0/1152760999 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f3408089c70 con 0x7f34180cf870 2024-05-30T17:02:33.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.620+0000 7f340e000640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f33f406e090 0x7f33f4070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:33.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.620+0000 7f340e000640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f33f406e090 0x7f33f4070550 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3400034000 tx=0x7f340002c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.624+0000 7f340f400640 1 -- 172.21.15.67:0/1152760999 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3408059ba0 con 0x7f34180cf870 2024-05-30T17:02:33.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.829+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f341806e430 con 0x7f34180cf870 2024-05-30T17:02:33.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.830+0000 7f340f400640 1 -- 172.21.15.67:0/1152760999 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7f34080594a0 con 0x7f34180cf870 2024-05-30T17:02:33.831 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-05-30T17:02:33.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.832+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f33f406e090 msgr2=0x7f33f4070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.832+0000 7f3417e00640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f33f406e090 0x7f33f4070550 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3400034000 tx=0x7f340002c000 comp rx=0 tx=0).stop 2024-05-30T17:02:33.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.832+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180cf870 msgr2=0x7f3418159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:33.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.832+0000 7f3417e00640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180cf870 0x7f3418159180 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f340800c3c0 tx=0x7f340800c880 comp rx=0 tx=0).stop 2024-05-30T17:02:33.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.833+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 shutdown_connections 2024-05-30T17:02:33.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.833+0000 7f3417e00640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f33f406e090 0x7f33f4070550 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.833+0000 7f3417e00640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f341815e870 0x7f3418160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.833+0000 7f3417e00640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34180d16a0 0x7f341815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.833+0000 7f3417e00640 1 --2- 172.21.15.67:0/1152760999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34180cf870 0x7f3418159180 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:33.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.833+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 >> 172.21.15.67:0/1152760999 conn(0x7f341808bfc0 msgr2=0x7f34180d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:33.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.833+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 shutdown_connections 2024-05-30T17:02:33.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:33.833+0000 7f3417e00640 1 -- 172.21.15.67:0/1152760999 wait complete. 2024-05-30T17:02:33.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled' 2024-05-30T17:02:34.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.056+0000 7ff873e00640 1 Processor -- start 2024-05-30T17:02:34.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.056+0000 7ff873e00640 1 -- start start 2024-05-30T17:02:34.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.057+0000 7ff873e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff874162940 0x7ff87406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:34.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.057+0000 7ff873e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8740722d0 0x7ff8740adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:34.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.057+0000 7ff873e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff87406f590 con 0x7ff874162d20 2024-05-30T17:02:34.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.058+0000 7ff873e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff87406f870 con 0x7ff874162940 2024-05-30T17:02:34.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.058+0000 7ff873e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8740ad3d0 con 0x7ff8740722d0 2024-05-30T17:02:34.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.058+0000 7ff86be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff874162d20 0x7ff874071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36904/0 (socket says 172.21.15.67:36904) 2024-05-30T17:02:34.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.058+0000 7ff86be00640 1 -- 172.21.15.67:0/3744726892 learned_addr learned my addr 172.21.15.67:0/3744726892 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.058+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2171067038 0 0) 0x7ff87406f590 con 0x7ff874162d20 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.058+0000 7ff872a00640 1 --2- 172.21.15.67:0/3744726892 >> v2:172.21.15.169:3300/0 conn(0x7ff874162940 0x7ff87406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.058+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff858003520 con 0x7ff874162d20 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.058+0000 7ff873400640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8740722d0 0x7ff8740adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4052929293 0 0) 0x7ff858003520 con 0x7ff874162d20 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8740722d0 msgr2=0x7ff8740adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8740722d0 0x7ff8740adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 >> v2:172.21.15.169:3300/0 conn(0x7ff874162940 msgr2=0x7ff87406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 --2- 172.21.15.67:0/3744726892 >> v2:172.21.15.169:3300/0 conn(0x7ff874162940 0x7ff87406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87406dde0 con 0x7ff874162d20 2024-05-30T17:02:34.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff85c001670 con 0x7ff874162d20 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 >> v1:172.21.15.67:6789/0 conn(0x7ff874162d20 legacy=0x7ff874071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8740722d0 0x7ff8580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff858003f30 0x7ff858004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff858004d40 0x7ff8580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff87406f870 con 0x7ff858004d40 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff8740ad3d0 con 0x7ff858003f30 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff85800bb50 con 0x7ff8740722d0 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.059+0000 7ff873400640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff858004d40 0x7ff8580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.060+0000 7ff86be00640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff858003f30 0x7ff858004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:34.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.060+0000 7ff872a00640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8740722d0 0x7ff8580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.060+0000 7ff873400640 1 -- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8740722d0 msgr2=0x7ff8580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.060+0000 7ff873400640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8740722d0 0x7ff8580039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.060+0000 7ff873400640 1 -- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff858003f30 msgr2=0x7ff858004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.060+0000 7ff873400640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff858003f30 0x7ff858004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.060+0000 7ff873400640 1 -- 172.21.15.67:0/3744726892 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff858003520 con 0x7ff858004d40 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.061+0000 7ff873400640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff858004d40 0x7ff8580051c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff86c007360 tx=0x7ff858005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66555643bfd12b39 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.061+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c009be0 con 0x7ff858004d40 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.061+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86c036600 con 0x7ff858004d40 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.061+0000 7ff872000640 1 -- 172.21.15.67:0/3744726892 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0369d0 con 0x7ff858004d40 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.062+0000 7ff873e00640 1 -- 172.21.15.67:0/3744726892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff858004d40 msgr2=0x7ff8580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:34.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.062+0000 7ff873e00640 1 --2- 172.21.15.67:0/3744726892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff858004d40 0x7ff8580051c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff86c007360 tx=0x7ff858005830 comp rx=0 tx=0).stop 2024-05-30T17:02:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.062+0000 7ff873400640 1 -- 172.21.15.67:0/3744726892 reap_dead start 2024-05-30T17:02:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.062+0000 7ff873e00640 1 -- 172.21.15.67:0/3744726892 shutdown_connections 2024-05-30T17:02:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.062+0000 7ff873e00640 1 -- 172.21.15.67:0/3744726892 >> 172.21.15.67:0/3744726892 conn(0x7ff8740b1fc0 msgr2=0x7ff8740b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.062+0000 7ff873e00640 1 -- 172.21.15.67:0/3744726892 shutdown_connections 2024-05-30T17:02:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.062+0000 7ff873e00640 1 -- 172.21.15.67:0/3744726892 wait complete. 2024-05-30T17:02:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.064+0000 7ff873e00640 1 Processor -- start 2024-05-30T17:02:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.064+0000 7ff873e00640 1 -- start start 2024-05-30T17:02:34.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.065+0000 7ff873e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff858004d40 0x7ff8741f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:34.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.065+0000 7ff873e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87414c770 0x7ff8741583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:34.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.065+0000 7ff873e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8740b46c0 0x7ff8740b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:34.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.065+0000 7ff873e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff8740adbb0 con 0x7ff8740b46c0 2024-05-30T17:02:34.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.065+0000 7ff873e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff87406f250 con 0x7ff858004d40 2024-05-30T17:02:34.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.065+0000 7ff873e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8740add30 con 0x7ff87414c770 2024-05-30T17:02:34.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.065+0000 7ff873400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8740b46c0 0x7ff8740b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:34.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.065+0000 7ff873400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8740b46c0 0x7ff8740b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59700/0 (socket says 172.21.15.67:59700) 2024-05-30T17:02:34.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.065+0000 7ff873400640 1 -- 172.21.15.67:0/967241355 learned_addr learned my addr 172.21.15.67:0/967241355 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:34.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff873400640 1 -- 172.21.15.67:0/967241355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87414c770 msgr2=0x7ff8741583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:34.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff873400640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87414c770 0x7ff8741583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff873400640 1 -- 172.21.15.67:0/967241355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff858004d40 msgr2=0x7ff8741f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:34.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff872a00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff858004d40 0x7ff8741f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:34.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff873400640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff858004d40 0x7ff8741f98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff873400640 1 -- 172.21.15.67:0/967241355 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff874158d40 con 0x7ff8740b46c0 2024-05-30T17:02:34.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff873400640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8740b46c0 0x7ff8740b4b40 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff86c007450 tx=0x7ff86c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:34.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff86aa00640 1 -- 172.21.15.67:0/967241355 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c045070 con 0x7ff8740b46c0 2024-05-30T17:02:34.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff86aa00640 1 -- 172.21.15.67:0/967241355 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86c003990 con 0x7ff8740b46c0 2024-05-30T17:02:34.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff86aa00640 1 -- 172.21.15.67:0/967241355 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c009be0 con 0x7ff8740b46c0 2024-05-30T17:02:34.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff874159020 con 0x7ff8740b46c0 2024-05-30T17:02:34.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.066+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8740ccd40 con 0x7ff8740b46c0 2024-05-30T17:02:34.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.067+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8740b5080 con 0x7ff8740b46c0 2024-05-30T17:02:34.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.068+0000 7ff86aa00640 1 -- 172.21.15.67:0/967241355 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7ff86c048070 con 0x7ff8740b46c0 2024-05-30T17:02:34.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.069+0000 7ff86aa00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85006e090 0x7ff850070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:34.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.069+0000 7ff86aa00640 1 -- 172.21.15.67:0/967241355 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7ff86c0aadf0 con 0x7ff8740b46c0 2024-05-30T17:02:34.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.070+0000 7ff872a00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85006e090 0x7ff850070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:34.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.070+0000 7ff872a00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85006e090 0x7ff850070550 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff864009990 tx=0x7ff864008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:34.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.074+0000 7ff86aa00640 1 -- 172.21.15.67:0/967241355 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff86c03d070 con 0x7ff8740b46c0 2024-05-30T17:02:34.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.344+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]} v 0) -- 0x7ff87414cbb0 con 0x7ff8740b46c0 2024-05-30T17:02:34.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.905+0000 7ff86aa00640 1 -- 172.21.15.67:0/967241355 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]}]=0 v781) ==== 166+0+0 (secure 0 0 0) 0x7ff86c032030 con 0x7ff8740b46c0 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85006e090 msgr2=0x7ff850070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85006e090 0x7ff850070550 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff864009990 tx=0x7ff864008040 comp rx=0 tx=0).stop 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8740b46c0 msgr2=0x7ff8740b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8740b46c0 0x7ff8740b4b40 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff86c007450 tx=0x7ff86c004ab0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 shutdown_connections 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff85006e090 0x7ff850070550 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8740b46c0 0x7ff8740b4b40 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff87414c770 0x7ff8741583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 --2- 172.21.15.67:0/967241355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff858004d40 0x7ff8741f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 >> 172.21.15.67:0/967241355 conn(0x7ff8740b1fc0 msgr2=0x7ff8740b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 shutdown_connections 2024-05-30T17:02:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:34.909+0000 7ff873e00640 1 -- 172.21.15.67:0/967241355 wait complete. 2024-05-30T17:02:34.931 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T17:02:35.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.143+0000 7f4b8c400640 1 Processor -- start 2024-05-30T17:02:35.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.144+0000 7f4b8c400640 1 -- start start 2024-05-30T17:02:35.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.146+0000 7f4b8c400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b840d1390 0x7f4b840733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:35.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.146+0000 7f4b8c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b840ce590 0x7f4b840cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:35.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.146+0000 7f4b8c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b840cedc0 con 0x7f4b840afae0 2024-05-30T17:02:35.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.146+0000 7f4b8c400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4b840cf0a0 con 0x7f4b840d1390 2024-05-30T17:02:35.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.146+0000 7f4b8c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4b840cf380 con 0x7f4b840ce590 2024-05-30T17:02:35.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.146+0000 7f4b89a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b840d1390 0x7f4b840733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:35.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.146+0000 7f4b89a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b840d1390 0x7f4b840733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59714/0 (socket says 172.21.15.67:59714) 2024-05-30T17:02:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.146+0000 7f4b89a00640 1 -- 172.21.15.67:0/29130341 learned_addr learned my addr 172.21.15.67:0/29130341 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.147+0000 7f4b89a00640 1 -- 172.21.15.67:0/29130341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b840ce590 msgr2=0x7f4b840cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.147+0000 7f4b89a00640 1 --2- 172.21.15.67:0/29130341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b840ce590 0x7f4b840cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.147+0000 7f4b89a00640 1 -- 172.21.15.67:0/29130341 >> v1:172.21.15.169:6789/0 conn(0x7f4b840afae0 legacy=0x7f4b840d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.147+0000 7f4b89a00640 1 -- 172.21.15.67:0/29130341 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b840cf660 con 0x7f4b840d1390 2024-05-30T17:02:35.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.148+0000 7f4b89a00640 1 --2- 172.21.15.67:0/29130341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840d1390 0x7f4b840733a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4b840715a0 tx=0x7f4b7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb476bba2edff86 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:35.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.148+0000 7f4b83e00640 1 -- 172.21.15.67:0/29130341 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78036070 con 0x7f4b840d1390 2024-05-30T17:02:35.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.148+0000 7f4b83e00640 1 -- 172.21.15.67:0/29130341 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b78031600 con 0x7f4b840d1390 2024-05-30T17:02:35.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.148+0000 7f4b83e00640 1 -- 172.21.15.67:0/29130341 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780319d0 con 0x7f4b840d1390 2024-05-30T17:02:35.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.149+0000 7f4b8c400640 1 -- 172.21.15.67:0/29130341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840d1390 msgr2=0x7f4b840733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:35.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.149+0000 7f4b8c400640 1 --2- 172.21.15.67:0/29130341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840d1390 0x7f4b840733a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4b840715a0 tx=0x7f4b7802a6a0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.149+0000 7f4b8c400640 1 -- 172.21.15.67:0/29130341 shutdown_connections 2024-05-30T17:02:35.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.149+0000 7f4b8c400640 1 --2- 172.21.15.67:0/29130341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b840ce590 0x7f4b840cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.149+0000 7f4b8c400640 1 --2- 172.21.15.67:0/29130341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840d1390 0x7f4b840733a0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.149+0000 7f4b8c400640 1 -- 172.21.15.67:0/29130341 >> 172.21.15.67:0/29130341 conn(0x7f4b840b6af0 msgr2=0x7f4b8409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:35.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.149+0000 7f4b8c400640 1 -- 172.21.15.67:0/29130341 shutdown_connections 2024-05-30T17:02:35.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.149+0000 7f4b8c400640 1 -- 172.21.15.67:0/29130341 wait complete. 2024-05-30T17:02:35.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.151+0000 7f4b8c400640 1 Processor -- start 2024-05-30T17:02:35.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.151+0000 7f4b8c400640 1 -- start start 2024-05-30T17:02:35.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b8c400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840ce590 0x7f4b8409ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:35.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b8c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b840d1390 0x7f4b840b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:35.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b8c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b8415a8a0 0x7f4b8415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:35.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b8c400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841463e0 con 0x7f4b840ce590 2024-05-30T17:02:35.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b8c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84146260 con 0x7f4b840d1390 2024-05-30T17:02:35.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b8c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4b840ce9e0 con 0x7f4b8415a8a0 2024-05-30T17:02:35.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b89a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840ce590 0x7f4b8409ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:35.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b89a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840ce590 0x7f4b8409ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59726/0 (socket says 172.21.15.67:59726) 2024-05-30T17:02:35.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b89a00640 1 -- 172.21.15.67:0/4234348290 learned_addr learned my addr 172.21.15.67:0/4234348290 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:35.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b89a00640 1 -- 172.21.15.67:0/4234348290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b8415a8a0 msgr2=0x7f4b8415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:35.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b89a00640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b8415a8a0 0x7f4b8415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b89a00640 1 -- 172.21.15.67:0/4234348290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b840d1390 msgr2=0x7f4b840b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:35.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.152+0000 7f4b89a00640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b840d1390 0x7f4b840b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.153+0000 7f4b89a00640 1 -- 172.21.15.67:0/4234348290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b840b0e30 con 0x7f4b840ce590 2024-05-30T17:02:35.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.153+0000 7f4b89a00640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840ce590 0x7f4b8409ecc0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4b78000f80 tx=0x7f4b78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.153+0000 7f4b81600640 1 -- 172.21.15.67:0/4234348290 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78041020 con 0x7f4b840ce590 2024-05-30T17:02:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.153+0000 7f4b81600640 1 -- 172.21.15.67:0/4234348290 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b78037040 con 0x7f4b840ce590 2024-05-30T17:02:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.153+0000 7f4b81600640 1 -- 172.21.15.67:0/4234348290 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78036070 con 0x7f4b840ce590 2024-05-30T17:02:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.153+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b840b02b0 con 0x7f4b840ce590 2024-05-30T17:02:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.153+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b840b0870 con 0x7f4b840ce590 2024-05-30T17:02:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.154+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b840690f0 con 0x7f4b840ce590 2024-05-30T17:02:35.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.156+0000 7f4b81600640 1 -- 172.21.15.67:0/4234348290 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f4b78044020 con 0x7f4b840ce590 2024-05-30T17:02:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.156+0000 7f4b81600640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4b6406e0c0 0x7f4b64070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.157+0000 7f4b81600640 1 -- 172.21.15.67:0/4234348290 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f4b780ab680 con 0x7f4b840ce590 2024-05-30T17:02:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.157+0000 7f4b89000640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4b6406e0c0 0x7f4b64070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.157+0000 7f4b89000640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4b6406e0c0 0x7f4b64070580 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4b8415b6c0 tx=0x7f4b700079b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:35.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.161+0000 7f4b81600640 1 -- 172.21.15.67:0/4234348290 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b7804b030 con 0x7f4b840ce590 2024-05-30T17:02:35.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.365+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4b84143e90 con 0x7f4b840ce590 2024-05-30T17:02:35.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.371+0000 7f4b81600640 1 -- 172.21.15.67:0/4234348290 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f4b7807b510 con 0x7f4b840ce590 2024-05-30T17:02:35.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.373+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4b6406e0c0 msgr2=0x7f4b64070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:35.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.373+0000 7f4b8c400640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4b6406e0c0 0x7f4b64070580 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4b8415b6c0 tx=0x7f4b700079b0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.373+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840ce590 msgr2=0x7f4b8409ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:35.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.373+0000 7f4b8c400640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840ce590 0x7f4b8409ecc0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4b78000f80 tx=0x7f4b78004a40 comp rx=0 tx=0).stop 2024-05-30T17:02:35.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.374+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 shutdown_connections 2024-05-30T17:02:35.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.374+0000 7f4b8c400640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4b6406e0c0 0x7f4b64070580 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.374+0000 7f4b8c400640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b8415a8a0 0x7f4b8415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.374+0000 7f4b8c400640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b840d1390 0x7f4b840b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.374+0000 7f4b8c400640 1 --2- 172.21.15.67:0/4234348290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840ce590 0x7f4b8409ecc0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:35.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.374+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 >> 172.21.15.67:0/4234348290 conn(0x7f4b840b6af0 msgr2=0x7f4b84143780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:35.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.374+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 shutdown_connections 2024-05-30T17:02:35.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:35.374+0000 7f4b8c400640 1 -- 172.21.15.67:0/4234348290 wait complete. 2024-05-30T17:02:35.395 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T17:02:35.395 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:02:35.511 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:02:35.642 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T17:02:35.642 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:02:35.642 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:35.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:02:35.703 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:02:35.703 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:35.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:02:35.761 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:02:35.761 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:35.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:02:35.822 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:02:35.822 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:35.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:02:35.880 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-30T17:02:35.880 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:02:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.101+0000 7fe7b2000640 1 Processor -- start 2024-05-30T17:02:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.101+0000 7fe7b2000640 1 -- start start 2024-05-30T17:02:36.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.102+0000 7fe7b2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7ac0d04a0 0x7fe7ac0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.102+0000 7fe7b2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe7ac0ad130 0x7fe7ac0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.102+0000 7fe7b2000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7ac06ed10 con 0x7fe7ac06f860 2024-05-30T17:02:36.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.102+0000 7fe7b2000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe7ac0d0e40 con 0x7fe7ac0ad130 2024-05-30T17:02:36.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.102+0000 7fe7b2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ac15cb00 con 0x7fe7ac0d04a0 2024-05-30T17:02:36.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.102+0000 7fe7ab400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe7ac06f860 0x7fe7ac0b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36908/0 (socket says 172.21.15.67:36908) 2024-05-30T17:02:36.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.102+0000 7fe7ab400640 1 -- 172.21.15.67:0/1766851473 learned_addr learned my addr 172.21.15.67:0/1766851473 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:36.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.102+0000 7fe7abe00640 1 --2- 172.21.15.67:0/1766851473 >> v2:172.21.15.67:3300/0 conn(0x7fe7ac0ad130 0x7fe7ac0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.103+0000 7fe7aaa00640 1 --2- 172.21.15.67:0/1766851473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7ac0d04a0 0x7fe7ac0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.103+0000 7fe7abe00640 1 -- 172.21.15.67:0/1766851473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7ac0d04a0 msgr2=0x7fe7ac0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.103+0000 7fe7abe00640 1 --2- 172.21.15.67:0/1766851473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7ac0d04a0 0x7fe7ac0d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.103+0000 7fe7abe00640 1 -- 172.21.15.67:0/1766851473 >> v1:172.21.15.67:6789/0 conn(0x7fe7ac06f860 legacy=0x7fe7ac0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.103+0000 7fe7abe00640 1 -- 172.21.15.67:0/1766851473 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac0b3cc0 con 0x7fe7ac0ad130 2024-05-30T17:02:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.103+0000 7fe7aa000640 1 -- 172.21.15.67:0/1766851473 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3025213758 0 0) 0x7fe79c001490 con 0x7fe7ac06f860 2024-05-30T17:02:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.104+0000 7fe7abe00640 1 --2- 172.21.15.67:0/1766851473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac0ad130 0x7fe7ac0ad5a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40097a0 tx=0x7fe7a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2362b5dbbdc672a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.104+0000 7fe7aa000640 1 -- 172.21.15.67:0/1766851473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4037070 con 0x7fe7ac0ad130 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.104+0000 7fe7aa000640 1 -- 172.21.15.67:0/1766851473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a4032440 con 0x7fe7ac0ad130 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.104+0000 7fe7aa000640 1 -- 172.21.15.67:0/1766851473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4032810 con 0x7fe7ac0ad130 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.105+0000 7fe7b2000640 1 -- 172.21.15.67:0/1766851473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac0ad130 msgr2=0x7fe7ac0ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.105+0000 7fe7b2000640 1 --2- 172.21.15.67:0/1766851473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac0ad130 0x7fe7ac0ad5a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40097a0 tx=0x7fe7a402a180 comp rx=0 tx=0).stop 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.105+0000 7fe7b2000640 1 -- 172.21.15.67:0/1766851473 shutdown_connections 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.105+0000 7fe7b2000640 1 --2- 172.21.15.67:0/1766851473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac0ad130 0x7fe7ac0ad5a0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.105+0000 7fe7b2000640 1 --2- 172.21.15.67:0/1766851473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7ac0d04a0 0x7fe7ac0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.105+0000 7fe7b2000640 1 -- 172.21.15.67:0/1766851473 >> 172.21.15.67:0/1766851473 conn(0x7fe7ac06bae0 msgr2=0x7fe7ac06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.105+0000 7fe7b2000640 1 -- 172.21.15.67:0/1766851473 shutdown_connections 2024-05-30T17:02:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.105+0000 7fe7b2000640 1 -- 172.21.15.67:0/1766851473 wait complete. 2024-05-30T17:02:36.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.107+0000 7fe7b2000640 1 Processor -- start 2024-05-30T17:02:36.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.107+0000 7fe7b2000640 1 -- start start 2024-05-30T17:02:36.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7b2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac06f860 0x7fe7ac0c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7b2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7ac0ad130 0x7fe7ac098920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7b2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7ac0d04a0 0x7fe7ac098e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7b2000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac0b4560 con 0x7fe7ac06f860 2024-05-30T17:02:36.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7b2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac0b4280 con 0x7fe7ac0ad130 2024-05-30T17:02:36.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7b2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ac0b4840 con 0x7fe7ac0d04a0 2024-05-30T17:02:36.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac06f860 0x7fe7ac0c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac06f860 0x7fe7ac0c2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59754/0 (socket says 172.21.15.67:59754) 2024-05-30T17:02:36.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7ab400640 1 -- 172.21.15.67:0/510449987 learned_addr learned my addr 172.21.15.67:0/510449987 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:36.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7ab400640 1 -- 172.21.15.67:0/510449987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7ac0d04a0 msgr2=0x7fe7ac098e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7ab400640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7ac0d04a0 0x7fe7ac098e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7ab400640 1 -- 172.21.15.67:0/510449987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7ac0ad130 msgr2=0x7fe7ac098920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7ab400640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7ac0ad130 0x7fe7ac098920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.108+0000 7fe7ab400640 1 -- 172.21.15.67:0/510449987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac15ef10 con 0x7fe7ac06f860 2024-05-30T17:02:36.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.109+0000 7fe7ab400640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac06f860 0x7fe7ac0c2f90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c850 tx=0x7fe79c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:36.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.109+0000 7fe7a3400640 1 -- 172.21.15.67:0/510449987 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c018070 con 0x7fe7ac06f860 2024-05-30T17:02:36.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.109+0000 7fe7a3400640 1 -- 172.21.15.67:0/510449987 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c00a640 con 0x7fe7ac06f860 2024-05-30T17:02:36.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.109+0000 7fe7a3400640 1 -- 172.21.15.67:0/510449987 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00aa70 con 0x7fe7ac06f860 2024-05-30T17:02:36.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.109+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac15f1f0 con 0x7fe7ac06f860 2024-05-30T17:02:36.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.109+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac06c050 con 0x7fe7ac06f860 2024-05-30T17:02:36.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.110+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac0ccb50 con 0x7fe7ac06f860 2024-05-30T17:02:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.111+0000 7fe7a3400640 1 -- 172.21.15.67:0/510449987 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fe79c00ac10 con 0x7fe7ac06f860 2024-05-30T17:02:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.112+0000 7fe7a3400640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe78806e0c0 0x7fe788070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.112+0000 7fe7a3400640 1 -- 172.21.15.67:0/510449987 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7fe79c089ee0 con 0x7fe7ac06f860 2024-05-30T17:02:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.113+0000 7fe7aaa00640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe78806e0c0 0x7fe788070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.113+0000 7fe7aaa00640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe78806e0c0 0x7fe788070580 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0b5040 tx=0x7fe794008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.117+0000 7fe7a3400640 1 -- 172.21.15.67:0/510449987 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7ac0ccb50 con 0x7fe7ac06f860 2024-05-30T17:02:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.365+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7ac0689e0 con 0x7fe7ac06f860 2024-05-30T17:02:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.366+0000 7fe7a3400640 1 -- 172.21.15.67:0/510449987 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+3875 (secure 0 0 0) 0x7fe7ac06ed10 con 0x7fe7ac06f860 2024-05-30T17:02:36.368 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:36.368 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2641805294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":2641805294},{"type":"v1","addr":"172.21.15.67:6837","nonce":2641805294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2609681849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2609681849},{"type":"v1","addr":"172.21.15.169:6834","nonce":2609681849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2372354106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2372354106},{"type":"v1","addr":"172.21.15.169:6835","nonce":2372354106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":8319,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/2198856612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2198856612},{"type":"v1","addr":"172.21.15.67:6836","nonce":2198856612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67}],"filesystems":[]} 2024-05-30T17:02:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 67 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe78806e0c0 msgr2=0x7fe788070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe78806e0c0 0x7fe788070580 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0b5040 tx=0x7fe794008040 comp rx=0 tx=0).stop 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac06f860 msgr2=0x7fe7ac0c2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac06f860 0x7fe7ac0c2f90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c850 tx=0x7fe79c00cd10 comp rx=0 tx=0).stop 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 shutdown_connections 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe78806e0c0 0x7fe788070580 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7ac0d04a0 0x7fe7ac098e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7ac0ad130 0x7fe7ac098920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 --2- 172.21.15.67:0/510449987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7ac06f860 0x7fe7ac0c2f90 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 >> 172.21.15.67:0/510449987 conn(0x7fe7ac06bae0 msgr2=0x7fe7ac0b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 shutdown_connections 2024-05-30T17:02:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.369+0000 7fe7b2000640 1 -- 172.21.15.67:0/510449987 wait complete. 2024-05-30T17:02:36.391 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-30T17:02:36.392 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-30T17:02:36.392 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-30T17:02:36.392 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:02:36.392 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-30T17:02:36.392 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-30T17:02:36.392 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-30T17:02:36.392 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:02:36.393 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-30T17:02:36.393 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-30T17:02:36.393 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-30T17:02:36.393 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:02:36.393 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-30T17:02:36.393 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-30T17:02:36.393 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-30T17:02:36.393 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:02:36.412 INFO:tasks.ceph.mds.a:Stopped 2024-05-30T17:02:36.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-30T17:02:36.413 INFO:tasks.ceph.mds.c:Stopped 2024-05-30T17:02:36.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-30T17:02:36.414 INFO:tasks.ceph.mds.d:Stopped 2024-05-30T17:02:36.414 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-30T17:02:36.426 INFO:tasks.ceph.mds.b:Stopped 2024-05-30T17:02:36.426 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-30T17:02:36.459 INFO:tasks.ceph.mds.a:Started 2024-05-30T17:02:36.459 DEBUG:teuthology.parallel:result is None 2024-05-30T17:02:36.464 INFO:tasks.ceph.mds.c:Started 2024-05-30T17:02:36.465 DEBUG:teuthology.parallel:result is None 2024-05-30T17:02:36.470 INFO:tasks.ceph.mds.d:Started 2024-05-30T17:02:36.471 DEBUG:teuthology.parallel:result is None 2024-05-30T17:02:36.477 INFO:tasks.ceph.mds.b:Started 2024-05-30T17:02:36.477 DEBUG:teuthology.parallel:result is None 2024-05-30T17:02:36.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-30T17:02:36.615 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T17:02:36.615 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T17:02:36.632 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T17:02:36.632 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T17:02:36.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.663+0000 7f565d600640 1 Processor -- start 2024-05-30T17:02:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.663+0000 7f565d600640 1 -- start start 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.664+0000 7f565d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f565815e870 0x7f56580d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.664+0000 7f565d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 0x7f56580d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f565d600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f565815ec50 con 0x7f56580d14d0 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f565d600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f56580d2150 con 0x7f565815e870 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f565d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56580cff80 con 0x7f56580cfa40 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f5657e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f565815e870 0x7f56580d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f5657e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f565815e870 0x7f56580d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59776/0 (socket says 172.21.15.67:59776) 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f5657e00640 1 -- 172.21.15.67:0/2039726072 learned_addr learned my addr 172.21.15.67:0/2039726072 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f564fe00640 1 --1- 172.21.15.67:0/2039726072 >> v1:172.21.15.67:6789/0 conn(0x7f56580d14d0 0x7f56580cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36932/0 (socket says 172.21.15.67:36932) 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f565cc00640 1 --2- 172.21.15.67:0/2039726072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 0x7f56580d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f5657e00640 1 -- 172.21.15.67:0/2039726072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 msgr2=0x7f56580d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f5657e00640 1 --2- 172.21.15.67:0/2039726072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 0x7f56580d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f5657e00640 1 -- 172.21.15.67:0/2039726072 >> v1:172.21.15.67:6789/0 conn(0x7f56580d14d0 legacy=0x7f56580cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f5657e00640 1 -- 172.21.15.67:0/2039726072 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56580d0260 con 0x7f565815e870 2024-05-30T17:02:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.665+0000 7f565cc00640 1 --2- 172.21.15.67:0/2039726072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 0x7f56580d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:02:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.666+0000 7f5657e00640 1 --2- 172.21.15.67:0/2039726072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f565815e870 0x7f56580d0f90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f5648009870 tx=0x7f564802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7647136c904567 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f5657400640 1 -- 172.21.15.67:0/2039726072 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648037070 con 0x7f565815e870 2024-05-30T17:02:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f5657400640 1 -- 172.21.15.67:0/2039726072 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5648032440 con 0x7f565815e870 2024-05-30T17:02:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f5657400640 1 -- 172.21.15.67:0/2039726072 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648032810 con 0x7f565815e870 2024-05-30T17:02:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f565d600640 1 -- 172.21.15.67:0/2039726072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f565815e870 msgr2=0x7f56580d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f565d600640 1 --2- 172.21.15.67:0/2039726072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f565815e870 0x7f56580d0f90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f5648009870 tx=0x7f564802a250 comp rx=0 tx=0).stop 2024-05-30T17:02:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f565d600640 1 -- 172.21.15.67:0/2039726072 shutdown_connections 2024-05-30T17:02:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f565d600640 1 --2- 172.21.15.67:0/2039726072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 0x7f56580d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f565d600640 1 --2- 172.21.15.67:0/2039726072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f565815e870 0x7f56580d0f90 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f565d600640 1 -- 172.21.15.67:0/2039726072 >> 172.21.15.67:0/2039726072 conn(0x7f565808bfc0 msgr2=0x7f56580d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.667+0000 7f565d600640 1 -- 172.21.15.67:0/2039726072 shutdown_connections 2024-05-30T17:02:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.668+0000 7f565d600640 1 -- 172.21.15.67:0/2039726072 wait complete. 2024-05-30T17:02:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.669+0000 7f565d600640 1 Processor -- start 2024-05-30T17:02:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.669+0000 7f565d600640 1 -- start start 2024-05-30T17:02:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.670+0000 7f565d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 0x7f5658159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.670+0000 7f565d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56580d14d0 0x7f56580b8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.670+0000 7f565d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f565815e870 0x7f56580cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.670+0000 7f565d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f56580d4fb0 con 0x7f56580d14d0 2024-05-30T17:02:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.670+0000 7f565d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56580d0820 con 0x7f565815e870 2024-05-30T17:02:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.670+0000 7f565d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56580d2b20 con 0x7f56580cfa40 2024-05-30T17:02:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f564fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56580d14d0 0x7f56580b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f564fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56580d14d0 0x7f56580b8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59792/0 (socket says 172.21.15.67:59792) 2024-05-30T17:02:36.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f564fe00640 1 -- 172.21.15.67:0/2244427369 learned_addr learned my addr 172.21.15.67:0/2244427369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:36.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f5657e00640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 0x7f5658159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f565cc00640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f565815e870 0x7f56580cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f564fe00640 1 -- 172.21.15.67:0/2244427369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 msgr2=0x7f5658159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f564fe00640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 0x7f5658159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f564fe00640 1 -- 172.21.15.67:0/2244427369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f565815e870 msgr2=0x7f56580cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f564fe00640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f565815e870 0x7f56580cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f564fe00640 1 -- 172.21.15.67:0/2244427369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56580b89c0 con 0x7f56580d14d0 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.671+0000 7f564fe00640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56580d14d0 0x7f56580b8090 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f564000c400 tx=0x7f564000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.672+0000 7f5654c00640 1 -- 172.21.15.67:0/2244427369 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640017070 con 0x7f56580d14d0 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.672+0000 7f5654c00640 1 -- 172.21.15.67:0/2244427369 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56400074e0 con 0x7f56580d14d0 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.672+0000 7f5654c00640 1 -- 172.21.15.67:0/2244427369 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56400059c0 con 0x7f56580d14d0 2024-05-30T17:02:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.672+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56580b8ca0 con 0x7f56580d14d0 2024-05-30T17:02:36.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.672+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56581f4b00 con 0x7f56580d14d0 2024-05-30T17:02:36.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.673+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56580cb3a0 con 0x7f56580d14d0 2024-05-30T17:02:36.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.674+0000 7f5654c00640 1 -- 172.21.15.67:0/2244427369 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f564001a070 con 0x7f56580d14d0 2024-05-30T17:02:36.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.675+0000 7f5654c00640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f563406e0c0 0x7f5634070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:36.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.675+0000 7f5654c00640 1 -- 172.21.15.67:0/2244427369 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5055+0+0 (secure 0 0 0) 0x7f5640089c60 con 0x7f56580d14d0 2024-05-30T17:02:36.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.675+0000 7f5657e00640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f563406e0c0 0x7f5634070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:36.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.676+0000 7f5657e00640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f563406e0c0 0x7f5634070580 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5648034000 tx=0x7f564802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:36.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.680+0000 7f5654c00640 1 -- 172.21.15.67:0/2244427369 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56580cb3a0 con 0x7f56580d14d0 2024-05-30T17:02:36.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.904+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f56580689e0 con 0x7f56580d14d0 2024-05-30T17:02:36.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.929+0000 7f5654c00640 1 -- 172.21.15.67:0/2244427369 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) ==== 97+0+0 (secure 0 0 0) 0x7f56400594a0 con 0x7f56580d14d0 2024-05-30T17:02:36.931 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.932+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f563406e0c0 msgr2=0x7f5634070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.932+0000 7f565d600640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f563406e0c0 0x7f5634070580 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5648034000 tx=0x7f564802e000 comp rx=0 tx=0).stop 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.932+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56580d14d0 msgr2=0x7f56580b8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.932+0000 7f565d600640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56580d14d0 0x7f56580b8090 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f564000c400 tx=0x7f564000c8c0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.933+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 shutdown_connections 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.933+0000 7f565d600640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f563406e0c0 0x7f5634070580 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.933+0000 7f565d600640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f565815e870 0x7f56580cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.933+0000 7f565d600640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56580d14d0 0x7f56580b8090 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.933+0000 7f565d600640 1 --2- 172.21.15.67:0/2244427369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f56580cfa40 0x7f5658159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.933+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 >> 172.21.15.67:0/2244427369 conn(0x7f565808bfc0 msgr2=0x7f56580b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.933+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 shutdown_connections 2024-05-30T17:02:36.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:36.933+0000 7f565d600640 1 -- 172.21.15.67:0/2244427369 wait complete. 2024-05-30T17:02:36.957 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-30T17:02:37.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.155+0000 7effd3e00640 1 Processor -- start 2024-05-30T17:02:37.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.155+0000 7effd3e00640 1 -- start start 2024-05-30T17:02:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.156+0000 7effd3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd4156b30 0x7effd40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.156+0000 7effd3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7effd406ee40 0x7effd40ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.156+0000 7effd3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7effd406f2c0 0x7effd40ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7effd4156f10 con 0x7effd406f2c0 2024-05-30T17:02:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7effd406dda0 con 0x7effd406ee40 2024-05-30T17:02:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7effd406e080 con 0x7effd4156b30 2024-05-30T17:02:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7effd406f2c0 0x7effd40ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7effd406f2c0 0x7effd40ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59806/0 (socket says 172.21.15.67:59806) 2024-05-30T17:02:37.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3400640 1 -- 172.21.15.67:0/3073923469 learned_addr learned my addr 172.21.15.67:0/3073923469 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd2a00640 1 --2- 172.21.15.67:0/3073923469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd4156b30 0x7effd40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd2000640 1 --2- 172.21.15.67:0/3073923469 >> v2:172.21.15.169:3300/0 conn(0x7effd406ee40 0x7effd40ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3400640 1 -- 172.21.15.67:0/3073923469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd4156b30 msgr2=0x7effd40af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3400640 1 --2- 172.21.15.67:0/3073923469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd4156b30 0x7effd40af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3400640 1 -- 172.21.15.67:0/3073923469 >> v2:172.21.15.169:3300/0 conn(0x7effd406ee40 msgr2=0x7effd40ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3400640 1 --2- 172.21.15.67:0/3073923469 >> v2:172.21.15.169:3300/0 conn(0x7effd406ee40 0x7effd40ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.157+0000 7effd3400640 1 -- 172.21.15.67:0/3073923469 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd406e360 con 0x7effd406f2c0 2024-05-30T17:02:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.159+0000 7effd3400640 1 --2- 172.21.15.67:0/3073923469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd406f2c0 0x7effd40ad770 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7effd406c860 tx=0x7effcc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c27e697ee92be6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.159+0000 7effd1600640 1 -- 172.21.15.67:0/3073923469 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc036070 con 0x7effd406f2c0 2024-05-30T17:02:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.159+0000 7effd1600640 1 -- 172.21.15.67:0/3073923469 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc031600 con 0x7effd406f2c0 2024-05-30T17:02:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.159+0000 7effd1600640 1 -- 172.21.15.67:0/3073923469 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0319d0 con 0x7effd406f2c0 2024-05-30T17:02:37.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.160+0000 7effd3e00640 1 -- 172.21.15.67:0/3073923469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd406f2c0 msgr2=0x7effd40ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.160+0000 7effd3e00640 1 --2- 172.21.15.67:0/3073923469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd406f2c0 0x7effd40ad770 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7effd406c860 tx=0x7effcc02a610 comp rx=0 tx=0).stop 2024-05-30T17:02:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.160+0000 7effd3e00640 1 -- 172.21.15.67:0/3073923469 shutdown_connections 2024-05-30T17:02:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.160+0000 7effd3e00640 1 --2- 172.21.15.67:0/3073923469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd406f2c0 0x7effd40ad770 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.160+0000 7effd3e00640 1 --2- 172.21.15.67:0/3073923469 >> v2:172.21.15.169:3300/0 conn(0x7effd406ee40 0x7effd40ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.160+0000 7effd3e00640 1 --2- 172.21.15.67:0/3073923469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd4156b30 0x7effd40af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.160+0000 7effd3e00640 1 -- 172.21.15.67:0/3073923469 >> 172.21.15.67:0/3073923469 conn(0x7effd40b1fc0 msgr2=0x7effd40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.160+0000 7effd3e00640 1 -- 172.21.15.67:0/3073923469 shutdown_connections 2024-05-30T17:02:37.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.160+0000 7effd3e00640 1 -- 172.21.15.67:0/3073923469 wait complete. 2024-05-30T17:02:37.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.162+0000 7effd3e00640 1 Processor -- start 2024-05-30T17:02:37.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.162+0000 7effd3e00640 1 -- start start 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.163+0000 7effd3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd406ee40 0x7effd4099090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd406f2c0 0x7effd415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd4156b30 0x7effd415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7effd40afc20 con 0x7effd4156b30 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7effd40b2c30 con 0x7effd406f2c0 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7effd415ccb0 con 0x7effd406ee40 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd4156b30 0x7effd415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd4156b30 0x7effd415abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59808/0 (socket says 172.21.15.67:59808) 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3400640 1 -- 172.21.15.67:0/374087240 learned_addr learned my addr 172.21.15.67:0/374087240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:37.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd2000640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd406f2c0 0x7effd415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd2a00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd406ee40 0x7effd4099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3400640 1 -- 172.21.15.67:0/374087240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd406ee40 msgr2=0x7effd4099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3400640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd406ee40 0x7effd4099090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3400640 1 -- 172.21.15.67:0/374087240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd406f2c0 msgr2=0x7effd415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3400640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd406f2c0 0x7effd415a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3400640 1 -- 172.21.15.67:0/374087240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd4159dc0 con 0x7effd4156b30 2024-05-30T17:02:37.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3400640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd4156b30 0x7effd415abf0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7effcc034040 tx=0x7effcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:37.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effcaa00640 1 -- 172.21.15.67:0/374087240 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc036050 con 0x7effd4156b30 2024-05-30T17:02:37.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effcaa00640 1 -- 172.21.15.67:0/374087240 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc042070 con 0x7effd4156b30 2024-05-30T17:02:37.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effcaa00640 1 -- 172.21.15.67:0/374087240 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc009510 con 0x7effd4156b30 2024-05-30T17:02:37.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd415cee0 con 0x7effd4156b30 2024-05-30T17:02:37.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.165+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd415d4a0 con 0x7effd4156b30 2024-05-30T17:02:37.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.166+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd40689e0 con 0x7effd4156b30 2024-05-30T17:02:37.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.168+0000 7effcaa00640 1 -- 172.21.15.67:0/374087240 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7effcc007bf0 con 0x7effd4156b30 2024-05-30T17:02:37.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.169+0000 7effcaa00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7effb406e090 0x7effb4070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.169+0000 7effcaa00640 1 -- 172.21.15.67:0/374087240 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4969+0+0 (secure 0 0 0) 0x7effcc0ab6d0 con 0x7effd4156b30 2024-05-30T17:02:37.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.169+0000 7effd2a00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7effb406e090 0x7effb4070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.170+0000 7effd2a00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7effb406e090 0x7effb4070550 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7effc4006fd0 tx=0x7effc4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:37.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.173+0000 7effcaa00640 1 -- 172.21.15.67:0/374087240 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effcc07b600 con 0x7effd4156b30 2024-05-30T17:02:37.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.458+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7effd406f7f0 con 0x7effd4156b30 2024-05-30T17:02:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.459+0000 7effcaa00640 1 -- 172.21.15.67:0/374087240 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7effcc07af00 con 0x7effd4156b30 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA==", 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:02:37.461 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw==", 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.462 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.463 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-05-30T17:02:37.463 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw==", 2024-05-30T17:02:37.463 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.463 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:02:37.463 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:02:37.463 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg==", 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.464 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng==", 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg==", 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.465 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w==", 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ==", 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:02:37.466 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A==", 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ==", 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w==", 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A==", 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.469 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg==", 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ==", 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.470 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.2", 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ==", 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T17:02:37.471 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.3", 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A==", 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T17:02:37.472 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCArFhmF5mhNhAA4H3cgcH6YdD53fANDW4seA==", 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.473 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ==", 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw==", 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T17:02:37.474 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:02:37.475 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.475 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.475 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-05-30T17:02:37.475 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7effb406e090 msgr2=0x7effb4070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7effb406e090 0x7effb4070550 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7effc4006fd0 tx=0x7effc4008040 comp rx=0 tx=0).stop 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd4156b30 msgr2=0x7effd415abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd4156b30 0x7effd415abf0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7effcc034040 tx=0x7effcc004a40 comp rx=0 tx=0).stop 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 shutdown_connections 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7effb406e090 0x7effb4070550 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effd4156b30 0x7effd415abf0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd406f2c0 0x7effd415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 --2- 172.21.15.67:0/374087240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd406ee40 0x7effd4099090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 >> 172.21.15.67:0/374087240 conn(0x7effd40b1fc0 msgr2=0x7effd40b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 shutdown_connections 2024-05-30T17:02:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.462+0000 7effd3e00640 1 -- 172.21.15.67:0/374087240 wait complete. 2024-05-30T17:02:37.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-30T17:02:37.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.691+0000 7f1af5800640 1 Processor -- start 2024-05-30T17:02:37.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.692+0000 7f1af5800640 1 -- start start 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.693+0000 7f1af5800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1af00d16a0 0x7f1af00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.693+0000 7f1af5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 0x7f1af00d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.693+0000 7f1af5800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af015ec50 con 0x7f1af015e870 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.693+0000 7f1af5800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1af00d2150 con 0x7f1af00d16a0 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.693+0000 7f1af5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1af00cfd20 con 0x7f1af00cf870 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.693+0000 7f1aef400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1af00d16a0 0x7f1af00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.694+0000 7f1aef400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1af00d16a0 0x7f1af00cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59810/0 (socket says 172.21.15.67:59810) 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.694+0000 7f1aef400640 1 -- 172.21.15.67:0/2220471460 learned_addr learned my addr 172.21.15.67:0/2220471460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.694+0000 7f1af4e00640 1 --2- 172.21.15.67:0/2220471460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 0x7f1af00d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.694+0000 7f1aef400640 1 -- 172.21.15.67:0/2220471460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 msgr2=0x7f1af00d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.694+0000 7f1aef400640 1 --2- 172.21.15.67:0/2220471460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 0x7f1af00d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.694+0000 7f1aef400640 1 -- 172.21.15.67:0/2220471460 >> v1:172.21.15.169:6789/0 conn(0x7f1af015e870 legacy=0x7f1af00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:37.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.694+0000 7f1aef400640 1 -- 172.21.15.67:0/2220471460 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af00cff80 con 0x7f1af00d16a0 2024-05-30T17:02:37.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.694+0000 7f1af4e00640 1 --2- 172.21.15.67:0/2220471460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 0x7f1af00d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.695+0000 7f1aef400640 1 --2- 172.21.15.67:0/2220471460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af00d16a0 0x7f1af00cf330 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009850 tx=0x7f1adc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ea6d4835b32dcf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.695+0000 7f1aeea00640 1 -- 172.21.15.67:0/2220471460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc004030 con 0x7f1af00d16a0 2024-05-30T17:02:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1aeea00640 1 -- 172.21.15.67:0/2220471460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc0041d0 con 0x7f1af00d16a0 2024-05-30T17:02:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1aeea00640 1 -- 172.21.15.67:0/2220471460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc003610 con 0x7f1af00d16a0 2024-05-30T17:02:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1af5800640 1 -- 172.21.15.67:0/2220471460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af00d16a0 msgr2=0x7f1af00cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1af5800640 1 --2- 172.21.15.67:0/2220471460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af00d16a0 0x7f1af00cf330 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009850 tx=0x7f1adc02a250 comp rx=0 tx=0).stop 2024-05-30T17:02:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1af5800640 1 -- 172.21.15.67:0/2220471460 shutdown_connections 2024-05-30T17:02:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1af5800640 1 --2- 172.21.15.67:0/2220471460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 0x7f1af00d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1af5800640 1 --2- 172.21.15.67:0/2220471460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af00d16a0 0x7f1af00cf330 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1af5800640 1 -- 172.21.15.67:0/2220471460 >> 172.21.15.67:0/2220471460 conn(0x7f1af008bfc0 msgr2=0x7f1af00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1af5800640 1 -- 172.21.15.67:0/2220471460 shutdown_connections 2024-05-30T17:02:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.696+0000 7f1af5800640 1 -- 172.21.15.67:0/2220471460 wait complete. 2024-05-30T17:02:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.698+0000 7f1af5800640 1 Processor -- start 2024-05-30T17:02:37.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.698+0000 7f1af5800640 1 -- start start 2024-05-30T17:02:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.699+0000 7f1af5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 0x7f1af00b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.699+0000 7f1af5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00d16a0 0x7f1af00cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.699+0000 7f1af5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af015e870 0x7f1af00b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.699+0000 7f1af5800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00d0820 con 0x7f1af015e870 2024-05-30T17:02:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.699+0000 7f1af5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00d0540 con 0x7f1af00d16a0 2024-05-30T17:02:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.699+0000 7f1af5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1af00d4fb0 con 0x7f1af00cf870 2024-05-30T17:02:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.699+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af015e870 0x7f1af00b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.699+0000 7f1af4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af015e870 0x7f1af00b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59824/0 (socket says 172.21.15.67:59824) 2024-05-30T17:02:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.699+0000 7f1af4e00640 1 -- 172.21.15.67:0/3107334095 learned_addr learned my addr 172.21.15.67:0/3107334095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:37.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1aefe00640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 0x7f1af00b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1aef400640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00d16a0 0x7f1af00cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1af4e00640 1 -- 172.21.15.67:0/3107334095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 msgr2=0x7f1af00b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1af4e00640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 0x7f1af00b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1af4e00640 1 -- 172.21.15.67:0/3107334095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00d16a0 msgr2=0x7f1af00cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1af4e00640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00d16a0 0x7f1af00cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1af4e00640 1 -- 172.21.15.67:0/3107334095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af00b8220 con 0x7f1af015e870 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1aef400640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00d16a0 0x7f1af00cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1af4e00640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af015e870 0x7f1af00b6020 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800cef0 tx=0x7f1ae800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.700+0000 7f1ae7e00640 1 -- 172.21.15.67:0/3107334095 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae801a070 con 0x7f1af015e870 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.701+0000 7f1ae7e00640 1 -- 172.21.15.67:0/3107334095 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae800d860 con 0x7f1af015e870 2024-05-30T17:02:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.701+0000 7f1ae7e00640 1 -- 172.21.15.67:0/3107334095 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800dc50 con 0x7f1af015e870 2024-05-30T17:02:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.701+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af00b8500 con 0x7f1af015e870 2024-05-30T17:02:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.701+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af00b8ac0 con 0x7f1af015e870 2024-05-30T17:02:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.702+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af00689e0 con 0x7f1af015e870 2024-05-30T17:02:37.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.703+0000 7f1ae7e00640 1 -- 172.21.15.67:0/3107334095 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f1ae801d070 con 0x7f1af015e870 2024-05-30T17:02:37.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.704+0000 7f1ae7e00640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ad006e090 0x7f1ad0070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.704+0000 7f1ae7e00640 1 -- 172.21.15.67:0/3107334095 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4969+0+0 (secure 0 0 0) 0x7f1ae808b1a0 con 0x7f1af015e870 2024-05-30T17:02:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.704+0000 7f1aefe00640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ad006e090 0x7f1ad0070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:37.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.705+0000 7f1aefe00640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ad006e090 0x7f1ad0070550 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0007fd0 tx=0x7f1ae0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:37.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.708+0000 7f1ae7e00640 1 -- 172.21.15.67:0/3107334095 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ae805b190 con 0x7f1af015e870 2024-05-30T17:02:37.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.916+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1af006df40 con 0x7f1af015e870 2024-05-30T17:02:37.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.917+0000 7f1ae7e00640 1 -- 172.21.15.67:0/3107334095 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v112) ==== 84+0+65 (secure 0 0 0) 0x7f1ae805aa90 con 0x7f1af015e870 2024-05-30T17:02:37.918 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:37.918 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:02:37.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:37.918 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-05-30T17:02:37.918 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-05-30T17:02:37.918 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:37.919 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:02:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ad006e090 msgr2=0x7f1ad0070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ad006e090 0x7f1ad0070550 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0007fd0 tx=0x7f1ae0009040 comp rx=0 tx=0).stop 2024-05-30T17:02:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af015e870 msgr2=0x7f1af00b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af015e870 0x7f1af00b6020 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800cef0 tx=0x7f1ae800a710 comp rx=0 tx=0).stop 2024-05-30T17:02:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 shutdown_connections 2024-05-30T17:02:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1ad006e090 0x7f1ad0070550 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1af015e870 0x7f1af00b6020 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1af00d16a0 0x7f1af00cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 --2- 172.21.15.67:0/3107334095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1af00cf870 0x7f1af00b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 >> 172.21.15.67:0/3107334095 conn(0x7f1af008bfc0 msgr2=0x7f1af006e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 shutdown_connections 2024-05-30T17:02:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:37.920+0000 7f1af5800640 1 -- 172.21.15.67:0/3107334095 wait complete. 2024-05-30T17:02:37.944 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-30T17:02:37.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-30T17:02:38.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.151+0000 7eff46400640 1 Processor -- start 2024-05-30T17:02:38.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.152+0000 7eff46400640 1 -- start start 2024-05-30T17:02:38.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.153+0000 7eff46400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 0x7eff400d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:38.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.153+0000 7eff46400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff401625b0 0x7eff400d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:38.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.153+0000 7eff46400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff400d0e50 con 0x7eff4015ea10 2024-05-30T17:02:38.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.153+0000 7eff46400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7eff400d1a60 con 0x7eff401625b0 2024-05-30T17:02:38.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.153+0000 7eff46400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff400d1130 con 0x7eff401621d0 2024-05-30T17:02:38.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.154+0000 7eff45000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 0x7eff400d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:38.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.154+0000 7eff3fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff401625b0 0x7eff400d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:38.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.154+0000 7eff45a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7eff4015ea10 0x7eff4015eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:59192/0 (socket says 172.21.15.67:59192) 2024-05-30T17:02:38.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.154+0000 7eff45a00640 1 -- 172.21.15.67:0/759710115 learned_addr learned my addr 172.21.15.67:0/759710115 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:38.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.155+0000 7eff3f400640 1 -- 172.21.15.67:0/759710115 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2746773989 0 0) 0x7eff400d0e50 con 0x7eff4015ea10 2024-05-30T17:02:38.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.155+0000 7eff3f400640 1 -- 172.21.15.67:0/759710115 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff24003520 con 0x7eff4015ea10 2024-05-30T17:02:38.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.155+0000 7eff3fe00640 1 -- 172.21.15.67:0/759710115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 msgr2=0x7eff400d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:38.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.155+0000 7eff3fe00640 1 --2- 172.21.15.67:0/759710115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 0x7eff400d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:38.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.155+0000 7eff3fe00640 1 -- 172.21.15.67:0/759710115 >> v1:172.21.15.169:6789/0 conn(0x7eff4015ea10 legacy=0x7eff4015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:38.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.155+0000 7eff3fe00640 1 -- 172.21.15.67:0/759710115 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff400d1410 con 0x7eff401625b0 2024-05-30T17:02:38.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.155+0000 7eff45000640 1 --2- 172.21.15.67:0/759710115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 0x7eff400d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:02:38.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.157+0000 7eff3fe00640 1 --2- 172.21.15.67:0/759710115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff401625b0 0x7eff400d4f30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009790 tx=0x7eff2c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=829095883a5222c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:38.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.158+0000 7eff3f400640 1 -- 172.21.15.67:0/759710115 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c037070 con 0x7eff401625b0 2024-05-30T17:02:38.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.158+0000 7eff3f400640 1 -- 172.21.15.67:0/759710115 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff2c032440 con 0x7eff401625b0 2024-05-30T17:02:38.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.158+0000 7eff3f400640 1 -- 172.21.15.67:0/759710115 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c032810 con 0x7eff401625b0 2024-05-30T17:02:38.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.159+0000 7eff46400640 1 -- 172.21.15.67:0/759710115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff401625b0 msgr2=0x7eff400d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:38.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.159+0000 7eff46400640 1 --2- 172.21.15.67:0/759710115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff401625b0 0x7eff400d4f30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009790 tx=0x7eff2c02a180 comp rx=0 tx=0).stop 2024-05-30T17:02:38.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.159+0000 7eff46400640 1 -- 172.21.15.67:0/759710115 shutdown_connections 2024-05-30T17:02:38.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.159+0000 7eff46400640 1 --2- 172.21.15.67:0/759710115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff401625b0 0x7eff400d4f30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:38.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.159+0000 7eff46400640 1 --2- 172.21.15.67:0/759710115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 0x7eff400d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:38.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.159+0000 7eff46400640 1 -- 172.21.15.67:0/759710115 >> 172.21.15.67:0/759710115 conn(0x7eff4008bfc0 msgr2=0x7eff400d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:38.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.159+0000 7eff46400640 1 -- 172.21.15.67:0/759710115 shutdown_connections 2024-05-30T17:02:38.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.159+0000 7eff46400640 1 -- 172.21.15.67:0/759710115 wait complete. 2024-05-30T17:02:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.161+0000 7eff46400640 1 Processor -- start 2024-05-30T17:02:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.161+0000 7eff46400640 1 -- start start 2024-05-30T17:02:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff46400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff4015ea10 0x7eff400a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff46400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 0x7eff40161220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff46400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff401625b0 0x7eff40161760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff46400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7eff400d5470 con 0x7eff401625b0 2024-05-30T17:02:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff46400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff400d1ef0 con 0x7eff4015ea10 2024-05-30T17:02:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff46400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff40162a10 con 0x7eff401621d0 2024-05-30T17:02:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff45a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff401625b0 0x7eff40161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff45a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff401625b0 0x7eff40161760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59840/0 (socket says 172.21.15.67:59840) 2024-05-30T17:02:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff45a00640 1 -- 172.21.15.67:0/734851236 learned_addr learned my addr 172.21.15.67:0/734851236 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff45000640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff4015ea10 0x7eff400a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.163+0000 7eff3fe00640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 0x7eff40161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff45a00640 1 -- 172.21.15.67:0/734851236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 msgr2=0x7eff40161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff45a00640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 0x7eff40161220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff45a00640 1 -- 172.21.15.67:0/734851236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff4015ea10 msgr2=0x7eff400a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff45a00640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff4015ea10 0x7eff400a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff45a00640 1 -- 172.21.15.67:0/734851236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff400cdda0 con 0x7eff401625b0 2024-05-30T17:02:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff45a00640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff401625b0 0x7eff40161760 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7eff3800c390 tx=0x7eff3800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff3cc00640 1 -- 172.21.15.67:0/734851236 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38017070 con 0x7eff401625b0 2024-05-30T17:02:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff3cc00640 1 -- 172.21.15.67:0/734851236 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff38005ce0 con 0x7eff401625b0 2024-05-30T17:02:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff3cc00640 1 -- 172.21.15.67:0/734851236 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38007310 con 0x7eff401625b0 2024-05-30T17:02:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.164+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff400ce080 con 0x7eff401625b0 2024-05-30T17:02:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.165+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff400b6540 con 0x7eff401625b0 2024-05-30T17:02:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.166+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff400689e0 con 0x7eff401625b0 2024-05-30T17:02:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.167+0000 7eff3cc00640 1 -- 172.21.15.67:0/734851236 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7eff3801a070 con 0x7eff401625b0 2024-05-30T17:02:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.168+0000 7eff3cc00640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff2006e090 0x7eff20070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.168+0000 7eff3cc00640 1 -- 172.21.15.67:0/734851236 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4969+0+0 (secure 0 0 0) 0x7eff38089d90 con 0x7eff401625b0 2024-05-30T17:02:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.168+0000 7eff45000640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff2006e090 0x7eff20070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.169+0000 7eff45000640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff2006e090 0x7eff20070550 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7eff30007440 tx=0x7eff3000b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:38.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.172+0000 7eff3cc00640 1 -- 172.21.15.67:0/734851236 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff38059ba0 con 0x7eff401625b0 2024-05-30T17:02:38.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.366+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7eff400d4d10 con 0x7eff401625b0 2024-05-30T17:02:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.948+0000 7eff3cc00640 1 -- 172.21.15.67:0/734851236 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v113) ==== 138+0+0 (secure 0 0 0) 0x7eff380594a0 con 0x7eff401625b0 2024-05-30T17:02:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-05-30T17:02:38.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.952+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff2006e090 msgr2=0x7eff20070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:38.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.952+0000 7eff46400640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff2006e090 0x7eff20070550 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7eff30007440 tx=0x7eff3000b040 comp rx=0 tx=0).stop 2024-05-30T17:02:38.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.952+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff401625b0 msgr2=0x7eff40161760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:38.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.952+0000 7eff46400640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff401625b0 0x7eff40161760 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7eff3800c390 tx=0x7eff3800c850 comp rx=0 tx=0).stop 2024-05-30T17:02:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.953+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 shutdown_connections 2024-05-30T17:02:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.953+0000 7eff46400640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7eff2006e090 0x7eff20070550 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.953+0000 7eff46400640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff401625b0 0x7eff40161760 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.953+0000 7eff46400640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff401621d0 0x7eff40161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.953+0000 7eff46400640 1 --2- 172.21.15.67:0/734851236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff4015ea10 0x7eff400a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.953+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 >> 172.21.15.67:0/734851236 conn(0x7eff4008bfc0 msgr2=0x7eff4015da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.953+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 shutdown_connections 2024-05-30T17:02:38.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:38.953+0000 7eff46400640 1 -- 172.21.15.67:0/734851236 wait complete. 2024-05-30T17:02:38.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-30T17:02:39.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.181+0000 7f5229c00640 1 Processor -- start 2024-05-30T17:02:39.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.181+0000 7f5229c00640 1 -- start start 2024-05-30T17:02:39.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.183+0000 7f5229c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f52240ce800 0x7f52240cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:39.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.183+0000 7f5229c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f522415dd70 0x7f522415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:39.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.183+0000 7f5229c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52240cd8e0 con 0x7f522415a150 2024-05-30T17:02:39.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.183+0000 7f5229c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f522415fe00 con 0x7f52240ce800 2024-05-30T17:02:39.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.183+0000 7f5229c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f52240cf1c0 con 0x7f522415dd70 2024-05-30T17:02:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.184+0000 7f5223e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f522415a150 0x7f522415d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36938/0 (socket says 172.21.15.67:36938) 2024-05-30T17:02:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.184+0000 7f5223e00640 1 -- 172.21.15.67:0/316379230 learned_addr learned my addr 172.21.15.67:0/316379230 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.184+0000 7f521b400640 1 --2- 172.21.15.67:0/316379230 >> v2:172.21.15.67:3300/0 conn(0x7f52240ce800 0x7f52240cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.184+0000 7f5229200640 1 --2- 172.21.15.67:0/316379230 >> v2:172.21.15.169:3300/0 conn(0x7f522415dd70 0x7f522415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.184+0000 7f5223400640 1 -- 172.21.15.67:0/316379230 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800457847 0 0) 0x7f52240cd8e0 con 0x7f522415a150 2024-05-30T17:02:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.184+0000 7f521b400640 1 -- 172.21.15.67:0/316379230 >> v2:172.21.15.169:3300/0 conn(0x7f522415dd70 msgr2=0x7f522415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.184+0000 7f521b400640 1 --2- 172.21.15.67:0/316379230 >> v2:172.21.15.169:3300/0 conn(0x7f522415dd70 0x7f522415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.184+0000 7f521b400640 1 -- 172.21.15.67:0/316379230 >> v1:172.21.15.67:6789/0 conn(0x7f522415a150 legacy=0x7f522415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.184+0000 7f521b400640 1 -- 172.21.15.67:0/316379230 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52240cdbc0 con 0x7f52240ce800 2024-05-30T17:02:39.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.185+0000 7f521b400640 1 --2- 172.21.15.67:0/316379230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 0x7f52240cec80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f5224070db0 tx=0x7f520c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65dbda0ff224e95a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:39.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.186+0000 7f5223400640 1 -- 172.21.15.67:0/316379230 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c037070 con 0x7f52240ce800 2024-05-30T17:02:39.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.186+0000 7f5223400640 1 -- 172.21.15.67:0/316379230 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520c032440 con 0x7f52240ce800 2024-05-30T17:02:39.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.186+0000 7f5223400640 1 -- 172.21.15.67:0/316379230 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c032810 con 0x7f52240ce800 2024-05-30T17:02:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.186+0000 7f5229c00640 1 -- 172.21.15.67:0/316379230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 msgr2=0x7f52240cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.186+0000 7f5229c00640 1 --2- 172.21.15.67:0/316379230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 0x7f52240cec80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f5224070db0 tx=0x7f520c02a1a0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.186+0000 7f5229c00640 1 -- 172.21.15.67:0/316379230 shutdown_connections 2024-05-30T17:02:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.186+0000 7f5229c00640 1 --2- 172.21.15.67:0/316379230 >> v2:172.21.15.169:3300/0 conn(0x7f522415dd70 0x7f522415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.186+0000 7f5229c00640 1 --2- 172.21.15.67:0/316379230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 0x7f52240cec80 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.186+0000 7f5229c00640 1 -- 172.21.15.67:0/316379230 >> 172.21.15.67:0/316379230 conn(0x7f52240732f0 msgr2=0x7f522406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.187+0000 7f5229c00640 1 -- 172.21.15.67:0/316379230 shutdown_connections 2024-05-30T17:02:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.187+0000 7f5229c00640 1 -- 172.21.15.67:0/316379230 wait complete. 2024-05-30T17:02:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.188+0000 7f5229c00640 1 Processor -- start 2024-05-30T17:02:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.188+0000 7f5229c00640 1 -- start start 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.189+0000 7f5229c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 0x7f522414d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.189+0000 7f5229c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522415a150 0x7f52240c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5229c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522415dd70 0x7f52240c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5229c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f52240ce180 con 0x7f52240ce800 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5229c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f52240ce4f0 con 0x7f522415a150 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5229c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f52240cd5b0 con 0x7f522415dd70 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5223e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 0x7f522414d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5223e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 0x7f522414d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59870/0 (socket says 172.21.15.67:59870) 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5223e00640 1 -- 172.21.15.67:0/519562005 learned_addr learned my addr 172.21.15.67:0/519562005 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f521b400640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522415a150 0x7f52240c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5223e00640 1 -- 172.21.15.67:0/519562005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522415dd70 msgr2=0x7f52240c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5223e00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522415dd70 0x7f52240c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5223e00640 1 -- 172.21.15.67:0/519562005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522415a150 msgr2=0x7f52240c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5223e00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522415a150 0x7f52240c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f5223e00640 1 -- 172.21.15.67:0/519562005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52240c6520 con 0x7f52240ce800 2024-05-30T17:02:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.190+0000 7f521b400640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522415a150 0x7f52240c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.191+0000 7f5223e00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 0x7f522414d540 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f521400c8c0 tx=0x7f521400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.191+0000 7f5220c00640 1 -- 172.21.15.67:0/519562005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214018070 con 0x7f52240ce800 2024-05-30T17:02:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.191+0000 7f5220c00640 1 -- 172.21.15.67:0/519562005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521400a5e0 con 0x7f52240ce800 2024-05-30T17:02:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.191+0000 7f5220c00640 1 -- 172.21.15.67:0/519562005 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521400a9d0 con 0x7f52240ce800 2024-05-30T17:02:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.191+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52240c6800 con 0x7f52240ce800 2024-05-30T17:02:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.191+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52240c6dc0 con 0x7f52240ce800 2024-05-30T17:02:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.192+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5224068bc0 con 0x7f52240ce800 2024-05-30T17:02:39.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.193+0000 7f5220c00640 1 -- 172.21.15.67:0/519562005 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f521400ab70 con 0x7f52240ce800 2024-05-30T17:02:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.194+0000 7f5220c00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520006e090 0x7f5200070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.194+0000 7f5220c00640 1 -- 172.21.15.67:0/519562005 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5339+0+0 (secure 0 0 0) 0x7f521408a0c0 con 0x7f52240ce800 2024-05-30T17:02:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.195+0000 7f521b400640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520006e090 0x7f5200070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.195+0000 7f521b400640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520006e090 0x7f5200070550 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f520c034000 tx=0x7f520c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:39.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.199+0000 7f5220c00640 1 -- 172.21.15.67:0/519562005 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5224068bc0 con 0x7f52240ce800 2024-05-30T17:02:39.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.406+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f52240690f0 con 0x7f52240ce800 2024-05-30T17:02:39.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.966+0000 7f5220c00640 1 -- 172.21.15.67:0/519562005 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v114) ==== 170+0+0 (secure 0 0 0) 0x7f5214059f50 con 0x7f52240ce800 2024-05-30T17:02:39.967 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-05-30T17:02:39.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.970+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520006e090 msgr2=0x7f5200070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:39.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.970+0000 7f5229c00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520006e090 0x7f5200070550 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f520c034000 tx=0x7f520c02e000 comp rx=0 tx=0).stop 2024-05-30T17:02:39.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.970+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 msgr2=0x7f522414d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:39.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.970+0000 7f5229c00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 0x7f522414d540 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f521400c8c0 tx=0x7f521400cd80 comp rx=0 tx=0).stop 2024-05-30T17:02:39.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.971+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 shutdown_connections 2024-05-30T17:02:39.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.971+0000 7f5229c00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f520006e090 0x7f5200070550 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.971+0000 7f5229c00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f522415dd70 0x7f52240c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.971+0000 7f5229c00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522415a150 0x7f52240c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.971+0000 7f5229c00640 1 --2- 172.21.15.67:0/519562005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52240ce800 0x7f522414d540 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:39.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.971+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 >> 172.21.15.67:0/519562005 conn(0x7f52240732f0 msgr2=0x7f52240b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:39.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.971+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 shutdown_connections 2024-05-30T17:02:39.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:39.971+0000 7f5229c00640 1 -- 172.21.15.67:0/519562005 wait complete. 2024-05-30T17:02:40.007 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-30T17:02:40.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.189+0000 7fa563e00640 1 Processor -- start 2024-05-30T17:02:40.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.189+0000 7fa563e00640 1 -- start start 2024-05-30T17:02:40.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.191+0000 7fa563e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa5640ce5d0 0x7fa5640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:40.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.191+0000 7fa563e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640b3d60 0x7fa5640b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:40.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.191+0000 7fa563e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5640b3580 con 0x7fa5640ce1f0 2024-05-30T17:02:40.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.191+0000 7fa563e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa56415f490 con 0x7fa5640ce5d0 2024-05-30T17:02:40.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.191+0000 7fa563e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa56415f770 con 0x7fa5640b3d60 2024-05-30T17:02:40.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.192+0000 7fa563400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640b3d60 0x7fa5640b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:40.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.192+0000 7fa563400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640b3d60 0x7fa5640b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:54048/0 (socket says 172.21.15.67:54048) 2024-05-30T17:02:40.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.192+0000 7fa563400640 1 -- 172.21.15.67:0/948214668 learned_addr learned my addr 172.21.15.67:0/948214668 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:40.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.192+0000 7fa562000640 1 --2- 172.21.15.67:0/948214668 >> v2:172.21.15.169:3300/0 conn(0x7fa5640ce5d0 0x7fa5640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:40.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.192+0000 7fa563400640 1 -- 172.21.15.67:0/948214668 >> v2:172.21.15.169:3300/0 conn(0x7fa5640ce5d0 msgr2=0x7fa5640b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:40.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.192+0000 7fa563400640 1 --2- 172.21.15.67:0/948214668 >> v2:172.21.15.169:3300/0 conn(0x7fa5640ce5d0 0x7fa5640b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:40.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.192+0000 7fa563400640 1 -- 172.21.15.67:0/948214668 >> v1:172.21.15.169:6789/0 conn(0x7fa5640ce1f0 legacy=0x7fa56415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:40.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.192+0000 7fa563400640 1 -- 172.21.15.67:0/948214668 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56415fa50 con 0x7fa5640b3d60 2024-05-30T17:02:40.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.192+0000 7fa562000640 1 --2- 172.21.15.67:0/948214668 >> v2:172.21.15.169:3300/0 conn(0x7fa5640ce5d0 0x7fa5640b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:02:40.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.194+0000 7fa563400640 1 --2- 172.21.15.67:0/948214668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640b3d60 0x7fa5640b41e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009e50 tx=0x7fa55c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b74ab3dc360cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:40.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.194+0000 7fa561600640 1 -- 172.21.15.67:0/948214668 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c036070 con 0x7fa5640b3d60 2024-05-30T17:02:40.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.194+0000 7fa561600640 1 -- 172.21.15.67:0/948214668 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c031600 con 0x7fa5640b3d60 2024-05-30T17:02:40.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.194+0000 7fa561600640 1 -- 172.21.15.67:0/948214668 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0319d0 con 0x7fa5640b3d60 2024-05-30T17:02:40.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.195+0000 7fa563e00640 1 -- 172.21.15.67:0/948214668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640b3d60 msgr2=0x7fa5640b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:40.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.195+0000 7fa563e00640 1 --2- 172.21.15.67:0/948214668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640b3d60 0x7fa5640b41e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009e50 tx=0x7fa55c02a5f0 comp rx=0 tx=0).stop 2024-05-30T17:02:40.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.195+0000 7fa563e00640 1 -- 172.21.15.67:0/948214668 shutdown_connections 2024-05-30T17:02:40.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.195+0000 7fa563e00640 1 --2- 172.21.15.67:0/948214668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640b3d60 0x7fa5640b41e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:40.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.195+0000 7fa563e00640 1 --2- 172.21.15.67:0/948214668 >> v2:172.21.15.169:3300/0 conn(0x7fa5640ce5d0 0x7fa5640b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:40.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.195+0000 7fa563e00640 1 -- 172.21.15.67:0/948214668 >> 172.21.15.67:0/948214668 conn(0x7fa56406b900 msgr2=0x7fa56406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:40.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.195+0000 7fa563e00640 1 -- 172.21.15.67:0/948214668 shutdown_connections 2024-05-30T17:02:40.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.195+0000 7fa563e00640 1 -- 172.21.15.67:0/948214668 wait complete. 2024-05-30T17:02:40.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.197+0000 7fa563e00640 1 Processor -- start 2024-05-30T17:02:40.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.198+0000 7fa563e00640 1 -- start start 2024-05-30T17:02:40.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.199+0000 7fa563e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5640b3d60 0x7fa5641f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:40.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.199+0000 7fa563e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5640ce1f0 0x7fa564149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:40.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.199+0000 7fa563e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640ce5d0 0x7fa564072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:40.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.199+0000 7fa563e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640ccd10 con 0x7fa5640ce1f0 2024-05-30T17:02:40.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.199+0000 7fa563e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640ccb90 con 0x7fa5640b3d60 2024-05-30T17:02:40.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.199+0000 7fa563e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa5640cce90 con 0x7fa5640ce5d0 2024-05-30T17:02:40.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.199+0000 7fa562000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5640ce1f0 0x7fa564149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:40.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.199+0000 7fa562000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5640ce1f0 0x7fa564149530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59876/0 (socket says 172.21.15.67:59876) 2024-05-30T17:02:40.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.199+0000 7fa562000640 1 -- 172.21.15.67:0/2508568511 learned_addr learned my addr 172.21.15.67:0/2508568511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:40.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.200+0000 7fa563400640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640ce5d0 0x7fa564072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:40.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.200+0000 7fa562a00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5640b3d60 0x7fa5641f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.200+0000 7fa562000640 1 -- 172.21.15.67:0/2508568511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640ce5d0 msgr2=0x7fa564072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.200+0000 7fa562000640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640ce5d0 0x7fa564072e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.200+0000 7fa562000640 1 -- 172.21.15.67:0/2508568511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5640b3d60 msgr2=0x7fa5641f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.200+0000 7fa562000640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5640b3d60 0x7fa5641f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.200+0000 7fa562000640 1 -- 172.21.15.67:0/2508568511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564149a70 con 0x7fa5640ce1f0 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.200+0000 7fa562000640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5640ce1f0 0x7fa564149530 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa55000d7d0 tx=0x7fa55000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.200+0000 7fa55aa00640 1 -- 172.21.15.67:0/2508568511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550004030 con 0x7fa5640ce1f0 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.201+0000 7fa55aa00640 1 -- 172.21.15.67:0/2508568511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5500041d0 con 0x7fa5640ce1f0 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.201+0000 7fa55aa00640 1 -- 172.21.15.67:0/2508568511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5500096a0 con 0x7fa5640ce1f0 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.201+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa564160100 con 0x7fa5640ce1f0 2024-05-30T17:02:40.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.201+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5641f7be0 con 0x7fa5640ce1f0 2024-05-30T17:02:40.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.202+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5640689e0 con 0x7fa5640ce1f0 2024-05-30T17:02:40.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.204+0000 7fa55aa00640 1 -- 172.21.15.67:0/2508568511 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fa550009840 con 0x7fa5640ce1f0 2024-05-30T17:02:40.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.205+0000 7fa55aa00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa54406e090 0x7fa544070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:40.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.205+0000 7fa55aa00640 1 -- 172.21.15.67:0/2508568511 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5721+0+0 (secure 0 0 0) 0x7fa55008b5c0 con 0x7fa5640ce1f0 2024-05-30T17:02:40.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.206+0000 7fa562a00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa54406e090 0x7fa544070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:40.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.206+0000 7fa562a00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa54406e090 0x7fa544070550 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa554000f60 tx=0x7fa554009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:40.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.210+0000 7fa55aa00640 1 -- 172.21.15.67:0/2508568511 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa564160100 con 0x7fa5640ce1f0 2024-05-30T17:02:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.460+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa5640ca070 con 0x7fa5640ce1f0 2024-05-30T17:02:40.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.993+0000 7fa55aa00640 1 -- 172.21.15.67:0/2508568511 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v70) ==== 174+0+0 (secure 0 0 0) 0x7fa564149a70 con 0x7fa5640ce1f0 2024-05-30T17:02:40.998 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 16 and data pool 17 2024-05-30T17:02:41.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.999+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa54406e090 msgr2=0x7fa544070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.999+0000 7fa563e00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa54406e090 0x7fa544070550 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa554000f60 tx=0x7fa554009040 comp rx=0 tx=0).stop 2024-05-30T17:02:41.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.999+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5640ce1f0 msgr2=0x7fa564149530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:40.999+0000 7fa563e00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5640ce1f0 0x7fa564149530 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa55000d7d0 tx=0x7fa55000dc90 comp rx=0 tx=0).stop 2024-05-30T17:02:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.000+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 shutdown_connections 2024-05-30T17:02:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.000+0000 7fa563e00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa54406e090 0x7fa544070550 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.000+0000 7fa563e00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640ce5d0 0x7fa564072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.000+0000 7fa563e00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5640ce1f0 0x7fa564149530 secure :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa55000d7d0 tx=0x7fa55000dc90 comp rx=0 tx=0).stop 2024-05-30T17:02:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.000+0000 7fa563e00640 1 --2- 172.21.15.67:0/2508568511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5640b3d60 0x7fa5641f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.000+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 >> 172.21.15.67:0/2508568511 conn(0x7fa56406b900 msgr2=0x7fa56406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.001+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 shutdown_connections 2024-05-30T17:02:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.002+0000 7fa563e00640 1 -- 172.21.15.67:0/2508568511 wait complete. 2024-05-30T17:02:41.038 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:02:41.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.214+0000 7f4847200640 1 Processor -- start 2024-05-30T17:02:41.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.214+0000 7f4847200640 1 -- start start 2024-05-30T17:02:41.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.215+0000 7f4847200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4848156b30 0x7f48480af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.215+0000 7f4847200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f48480ad940 0x7f48480b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.215+0000 7f4847200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4848156f10 con 0x7f484806ee40 2024-05-30T17:02:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.215+0000 7f4847200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f484806dda0 con 0x7f48480ad940 2024-05-30T17:02:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.215+0000 7f4847200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f484806e080 con 0x7f4848156b30 2024-05-30T17:02:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.216+0000 7f4846800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f48480ad940 0x7f48480b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.216+0000 7f4846800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f48480ad940 0x7f48480b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60618/0 (socket says 172.21.15.67:60618) 2024-05-30T17:02:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.216+0000 7f4846800640 1 -- 172.21.15.67:0/2227147451 learned_addr learned my addr 172.21.15.67:0/2227147451 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.216+0000 7f4845e00640 1 --2- 172.21.15.67:0/2227147451 >> v2:172.21.15.169:3300/0 conn(0x7f4848156b30 0x7f48480af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.216+0000 7f4846800640 1 -- 172.21.15.67:0/2227147451 >> v2:172.21.15.169:3300/0 conn(0x7f4848156b30 msgr2=0x7f48480af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:41.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.216+0000 7f4846800640 1 --2- 172.21.15.67:0/2227147451 >> v2:172.21.15.169:3300/0 conn(0x7f4848156b30 0x7f48480af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.216+0000 7f4846800640 1 -- 172.21.15.67:0/2227147451 >> v1:172.21.15.169:6789/0 conn(0x7f484806ee40 legacy=0x7f48480ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:41.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.216+0000 7f4846800640 1 -- 172.21.15.67:0/2227147451 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484806e360 con 0x7f48480ad940 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.217+0000 7f4846800640 1 --2- 172.21.15.67:0/2227147451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 0x7f48480b32c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f484806c8b0 tx=0x7f484002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6328f866a12080c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.218+0000 7f4844a00640 1 -- 172.21.15.67:0/2227147451 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840036070 con 0x7f48480ad940 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.218+0000 7f4844a00640 1 -- 172.21.15.67:0/2227147451 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4840031600 con 0x7f48480ad940 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.218+0000 7f4844a00640 1 -- 172.21.15.67:0/2227147451 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48400319d0 con 0x7f48480ad940 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.218+0000 7f4847200640 1 -- 172.21.15.67:0/2227147451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 msgr2=0x7f48480b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.218+0000 7f4847200640 1 --2- 172.21.15.67:0/2227147451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 0x7f48480b32c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f484806c8b0 tx=0x7f484002a660 comp rx=0 tx=0).stop 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.218+0000 7f4847200640 1 -- 172.21.15.67:0/2227147451 shutdown_connections 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.218+0000 7f4847200640 1 --2- 172.21.15.67:0/2227147451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 0x7f48480b32c0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.218+0000 7f4847200640 1 --2- 172.21.15.67:0/2227147451 >> v2:172.21.15.169:3300/0 conn(0x7f4848156b30 0x7f48480af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.218+0000 7f4847200640 1 -- 172.21.15.67:0/2227147451 >> 172.21.15.67:0/2227147451 conn(0x7f48480b1fc0 msgr2=0x7f48480b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.219+0000 7f4847200640 1 -- 172.21.15.67:0/2227147451 shutdown_connections 2024-05-30T17:02:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.219+0000 7f4847200640 1 -- 172.21.15.67:0/2227147451 wait complete. 2024-05-30T17:02:41.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.220+0000 7f4847200640 1 Processor -- start 2024-05-30T17:02:41.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.220+0000 7f4847200640 1 -- start start 2024-05-30T17:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.221+0000 7f4847200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f484806ee40 0x7f48480b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.221+0000 7f4847200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 0x7f48481f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.221+0000 7f4847200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4848156b30 0x7f48481f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.221+0000 7f4847200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f48480afc20 con 0x7f48480ad940 2024-05-30T17:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.221+0000 7f4847200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f48480b2c30 con 0x7f484806ee40 2024-05-30T17:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.221+0000 7f4847200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f48480b52c0 con 0x7f4848156b30 2024-05-30T17:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 0x7f48481f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 0x7f48481f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60620/0 (socket says 172.21.15.67:60620) 2024-05-30T17:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845400640 1 -- 172.21.15.67:0/1573351227 learned_addr learned my addr 172.21.15.67:0/1573351227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845e00640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f484806ee40 0x7f48480b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4846800640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4848156b30 0x7f48481f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845400640 1 -- 172.21.15.67:0/1573351227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4848156b30 msgr2=0x7f48481f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845400640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4848156b30 0x7f48481f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845400640 1 -- 172.21.15.67:0/1573351227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f484806ee40 msgr2=0x7f48480b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845400640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f484806ee40 0x7f48480b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845400640 1 -- 172.21.15.67:0/1573351227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4848155cb0 con 0x7f48480ad940 2024-05-30T17:02:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.222+0000 7f4845e00640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f484806ee40 0x7f48480b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.223+0000 7f4845400640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 0x7f48481f54d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f483400c430 tx=0x7f483400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.223+0000 7f483e000640 1 -- 172.21.15.67:0/1573351227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834011790 con 0x7f48480ad940 2024-05-30T17:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.223+0000 7f483e000640 1 -- 172.21.15.67:0/1573351227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4834017070 con 0x7f48480ad940 2024-05-30T17:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.223+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4848155f90 con 0x7f48480ad940 2024-05-30T17:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.223+0000 7f483e000640 1 -- 172.21.15.67:0/1573351227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834011bb0 con 0x7f48480ad940 2024-05-30T17:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.223+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48481f5a10 con 0x7f48480ad940 2024-05-30T17:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.224+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4848156160 con 0x7f48480ad940 2024-05-30T17:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.225+0000 7f483e000640 1 -- 172.21.15.67:0/1573351227 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f483401c460 con 0x7f48480ad940 2024-05-30T17:02:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.226+0000 7f483e000640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f482806e090 0x7f4828070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.226+0000 7f483e000640 1 -- 172.21.15.67:0/1573351227 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5821+0+0 (secure 0 0 0) 0x7f483408a870 con 0x7f48480ad940 2024-05-30T17:02:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.227+0000 7f4845e00640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f482806e090 0x7f4828070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.227+0000 7f4845e00640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f482806e090 0x7f4828070550 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4838006fd0 tx=0x7f4838008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.231+0000 7f483e000640 1 -- 172.21.15.67:0/1573351227 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f483405a700 con 0x7f48480ad940 2024-05-30T17:02:41.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.439+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48480689e0 con 0x7f48480ad940 2024-05-30T17:02:41.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.442+0000 7f483e000640 1 -- 172.21.15.67:0/1573351227 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16805 (secure 0 0 0) 0x7f483405a000 con 0x7f48480ad940 2024-05-30T17:02:41.444 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:41.444 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":115,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:02:40.462479+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:02:38.380492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-30T17:02:39.422718+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:02:41.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.445+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f482806e090 msgr2=0x7f4828070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.445+0000 7f4847200640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f482806e090 0x7f4828070550 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4838006fd0 tx=0x7f4838008040 comp rx=0 tx=0).stop 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.445+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 msgr2=0x7f48481f54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.445+0000 7f4847200640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 0x7f48481f54d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f483400c430 tx=0x7f483400c8f0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.446+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 shutdown_connections 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.446+0000 7f4847200640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f482806e090 0x7f4828070550 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.446+0000 7f4847200640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4848156b30 0x7f48481f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.446+0000 7f4847200640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48480ad940 0x7f48481f54d0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.446+0000 7f4847200640 1 --2- 172.21.15.67:0/1573351227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f484806ee40 0x7f48480b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.446+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 >> 172.21.15.67:0/1573351227 conn(0x7f48480b1fc0 msgr2=0x7f4848160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.446+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 shutdown_connections 2024-05-30T17:02:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.446+0000 7f4847200640 1 -- 172.21.15.67:0/1573351227 wait complete. 2024-05-30T17:02:41.466 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:02:41.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.670+0000 7f85e3200640 1 Processor -- start 2024-05-30T17:02:41.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.671+0000 7f85e3200640 1 -- start start 2024-05-30T17:02:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.672+0000 7f85e3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85dc15b3a0 0x7f85dc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.672+0000 7f85e3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85dc0afd60 con 0x7f85dc073310 2024-05-30T17:02:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.672+0000 7f85e3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85dc0af7a0 con 0x7f85dc06dc20 2024-05-30T17:02:41.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.672+0000 7f85e3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f85dc06f980 con 0x7f85dc15b3a0 2024-05-30T17:02:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.673+0000 7f85e1e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85dc15b3a0 0x7f85dc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.673+0000 7f85e2800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f85dc073310 0x7f85dc0ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58606/0 (socket says 172.21.15.67:58606) 2024-05-30T17:02:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.673+0000 7f85e2800640 1 -- 172.21.15.67:0/2696400944 learned_addr learned my addr 172.21.15.67:0/2696400944 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.673+0000 7f85e1e00640 1 -- 172.21.15.67:0/2696400944 >> v1:172.21.15.169:6789/0 conn(0x7f85dc06dc20 legacy=0x7f85dc072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.673+0000 7f85e1e00640 1 -- 172.21.15.67:0/2696400944 >> v1:172.21.15.67:6789/0 conn(0x7f85dc073310 legacy=0x7f85dc0ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.673+0000 7f85e1e00640 1 -- 172.21.15.67:0/2696400944 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc0afa80 con 0x7f85dc15b3a0 2024-05-30T17:02:41.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.674+0000 7f85e1e00640 1 --2- 172.21.15.67:0/2696400944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc15b3a0 0x7f85dc0b32c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009d90 tx=0x7f85d402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903031b5d760e00e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.675+0000 7f85e0a00640 1 -- 172.21.15.67:0/2696400944 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4007740 con 0x7f85dc15b3a0 2024-05-30T17:02:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.675+0000 7f85e0a00640 1 -- 172.21.15.67:0/2696400944 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d40078e0 con 0x7f85dc15b3a0 2024-05-30T17:02:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.675+0000 7f85e0a00640 1 -- 172.21.15.67:0/2696400944 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4007cb0 con 0x7f85dc15b3a0 2024-05-30T17:02:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.675+0000 7f85e3200640 1 -- 172.21.15.67:0/2696400944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc15b3a0 msgr2=0x7f85dc0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.675+0000 7f85e3200640 1 --2- 172.21.15.67:0/2696400944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc15b3a0 0x7f85dc0b32c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009d90 tx=0x7f85d402a6a0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.675+0000 7f85e3200640 1 -- 172.21.15.67:0/2696400944 shutdown_connections 2024-05-30T17:02:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.675+0000 7f85e3200640 1 --2- 172.21.15.67:0/2696400944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc15b3a0 0x7f85dc0b32c0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.675+0000 7f85e3200640 1 -- 172.21.15.67:0/2696400944 >> 172.21.15.67:0/2696400944 conn(0x7f85dc0b1fc0 msgr2=0x7f85dc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.675+0000 7f85e3200640 1 -- 172.21.15.67:0/2696400944 shutdown_connections 2024-05-30T17:02:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.676+0000 7f85e3200640 1 -- 172.21.15.67:0/2696400944 wait complete. 2024-05-30T17:02:41.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.677+0000 7f85e3200640 1 Processor -- start 2024-05-30T17:02:41.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.677+0000 7f85e3200640 1 -- start start 2024-05-30T17:02:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.678+0000 7f85e3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc06dc20 0x7f85dc14ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.678+0000 7f85e3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85dc073310 0x7f85dc0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.678+0000 7f85e3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85dc15b3a0 0x7f85dc0b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.678+0000 7f85e3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc15f9a0 con 0x7f85dc06dc20 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.678+0000 7f85e3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc0ae0c0 con 0x7f85dc15b3a0 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.678+0000 7f85e3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f85dc15fb20 con 0x7f85dc073310 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc06dc20 0x7f85dc14ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc06dc20 0x7f85dc14ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60626/0 (socket says 172.21.15.67:60626) 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e1e00640 1 -- 172.21.15.67:0/3074560684 learned_addr learned my addr 172.21.15.67:0/3074560684 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e2800640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85dc15b3a0 0x7f85dc0b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e1e00640 1 -- 172.21.15.67:0/3074560684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85dc073310 msgr2=0x7f85dc0b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e1e00640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85dc073310 0x7f85dc0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e1e00640 1 -- 172.21.15.67:0/3074560684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85dc15b3a0 msgr2=0x7f85dc0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e1e00640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85dc15b3a0 0x7f85dc0b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e1e00640 1 -- 172.21.15.67:0/3074560684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc1f4b00 con 0x7f85dc06dc20 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e2800640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85dc15b3a0 0x7f85dc0b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.679+0000 7f85e1e00640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc06dc20 0x7f85dc14ad90 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009eb0 tx=0x7f85d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:41.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.680+0000 7f85d2000640 1 -- 172.21.15.67:0/3074560684 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d403f070 con 0x7f85dc06dc20 2024-05-30T17:02:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.680+0000 7f85d2000640 1 -- 172.21.15.67:0/3074560684 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d4031d50 con 0x7f85dc06dc20 2024-05-30T17:02:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.680+0000 7f85d2000640 1 -- 172.21.15.67:0/3074560684 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4044660 con 0x7f85dc06dc20 2024-05-30T17:02:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.680+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85dc15b990 con 0x7f85dc06dc20 2024-05-30T17:02:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.680+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85dc15bf50 con 0x7f85dc06dc20 2024-05-30T17:02:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.681+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85dc0689e0 con 0x7f85dc06dc20 2024-05-30T17:02:41.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.682+0000 7f85d2000640 1 -- 172.21.15.67:0/3074560684 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f85d4044800 con 0x7f85dc06dc20 2024-05-30T17:02:41.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.683+0000 7f85d2000640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f85c006e090 0x7f85c0070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.683+0000 7f85d2000640 1 -- 172.21.15.67:0/3074560684 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5821+0+0 (secure 0 0 0) 0x7f85d40ac720 con 0x7f85dc06dc20 2024-05-30T17:02:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.684+0000 7f85e1400640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f85c006e090 0x7f85c0070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:41.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.684+0000 7f85e1400640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f85c006e090 0x7f85c0070550 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f85dc156f10 tx=0x7f85cc0079b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:41.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.686+0000 7f85d2000640 1 -- 172.21.15.67:0/3074560684 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85d4053080 con 0x7f85dc06dc20 2024-05-30T17:02:41.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.899+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85dc156980 con 0x7f85dc06dc20 2024-05-30T17:02:41.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.902+0000 7f85d2000640 1 -- 172.21.15.67:0/3074560684 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16805 (secure 0 0 0) 0x7f85d407c630 con 0x7f85dc06dc20 2024-05-30T17:02:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:41.903 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":115,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:02:40.462479+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:02:38.380492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-30T17:02:39.422718+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:02:41.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f85c006e090 msgr2=0x7f85c0070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f85c006e090 0x7f85c0070550 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f85dc156f10 tx=0x7f85cc0079b0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc06dc20 msgr2=0x7f85dc14ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:41.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc06dc20 0x7f85dc14ad90 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009eb0 tx=0x7f85d4004a40 comp rx=0 tx=0).stop 2024-05-30T17:02:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 shutdown_connections 2024-05-30T17:02:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f85c006e090 0x7f85c0070550 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85dc15b3a0 0x7f85dc0b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85dc073310 0x7f85dc0b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 --2- 172.21.15.67:0/3074560684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85dc06dc20 0x7f85dc14ad90 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 >> 172.21.15.67:0/3074560684 conn(0x7f85dc0b1fc0 msgr2=0x7f85dc0ae550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 shutdown_connections 2024-05-30T17:02:41.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:41.905+0000 7f85e3200640 1 -- 172.21.15.67:0/3074560684 wait complete. 2024-05-30T17:02:41.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-30T17:02:42.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.133+0000 7fcbbe200640 1 Processor -- start 2024-05-30T17:02:42.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.133+0000 7fcbbe200640 1 -- start start 2024-05-30T17:02:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.134+0000 7fcbbe200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcbb815e870 0x7fcbb80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.134+0000 7fcbbe200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcbb80d14d0 0x7fcbb80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbe200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb80cfce0 con 0x7fcbb80cf870 2024-05-30T17:02:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbe200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb80d2150 con 0x7fcbb815e870 2024-05-30T17:02:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbe200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb815ec50 con 0x7fcbb80d14d0 2024-05-30T17:02:42.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcbb815e870 0x7fcbb80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:42.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbd800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcbb80cf870 0x7fcbb8162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58610/0 (socket says 172.21.15.67:58610) 2024-05-30T17:02:42.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbd800640 1 -- 172.21.15.67:0/458995042 learned_addr learned my addr 172.21.15.67:0/458995042 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:42.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcbb815e870 0x7fcbb80d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60628/0 (socket says 172.21.15.67:60628) 2024-05-30T17:02:42.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbb7e00640 1 --2- 172.21.15.67:0/458995042 >> v2:172.21.15.169:3300/0 conn(0x7fcbb80d14d0 0x7fcbb80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbce00640 1 -- 172.21.15.67:0/458995042 >> v2:172.21.15.169:3300/0 conn(0x7fcbb80d14d0 msgr2=0x7fcbb80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbce00640 1 --2- 172.21.15.67:0/458995042 >> v2:172.21.15.169:3300/0 conn(0x7fcbb80d14d0 0x7fcbb80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbbce00640 1 -- 172.21.15.67:0/458995042 >> v1:172.21.15.67:6789/0 conn(0x7fcbb80cf870 legacy=0x7fcbb8162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.136+0000 7fcbbce00640 1 -- 172.21.15.67:0/458995042 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb80cff80 con 0x7fcbb815e870 2024-05-30T17:02:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.135+0000 7fcbb7400640 1 -- 172.21.15.67:0/458995042 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2937407998 0 0) 0x7fcbb80cfce0 con 0x7fcbb80cf870 2024-05-30T17:02:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.137+0000 7fcbbce00640 1 --2- 172.21.15.67:0/458995042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb815e870 0x7fcbb80d0f90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fcbb808e520 tx=0x7fcba802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de66203c0c07bb9b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.137+0000 7fcbb7400640 1 -- 172.21.15.67:0/458995042 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8007740 con 0x7fcbb815e870 2024-05-30T17:02:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.137+0000 7fcbb7400640 1 -- 172.21.15.67:0/458995042 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba80078e0 con 0x7fcbb815e870 2024-05-30T17:02:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.137+0000 7fcbb7400640 1 -- 172.21.15.67:0/458995042 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8007cb0 con 0x7fcbb815e870 2024-05-30T17:02:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.137+0000 7fcbbe200640 1 -- 172.21.15.67:0/458995042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb815e870 msgr2=0x7fcbb80d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.137+0000 7fcbbe200640 1 --2- 172.21.15.67:0/458995042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb815e870 0x7fcbb80d0f90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fcbb808e520 tx=0x7fcba802a680 comp rx=0 tx=0).stop 2024-05-30T17:02:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.138+0000 7fcbbe200640 1 -- 172.21.15.67:0/458995042 shutdown_connections 2024-05-30T17:02:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.138+0000 7fcbbe200640 1 --2- 172.21.15.67:0/458995042 >> v2:172.21.15.169:3300/0 conn(0x7fcbb80d14d0 0x7fcbb80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.138+0000 7fcbbe200640 1 --2- 172.21.15.67:0/458995042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb815e870 0x7fcbb80d0f90 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.138+0000 7fcbbe200640 1 -- 172.21.15.67:0/458995042 >> 172.21.15.67:0/458995042 conn(0x7fcbb808bfc0 msgr2=0x7fcbb80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.138+0000 7fcbbe200640 1 -- 172.21.15.67:0/458995042 shutdown_connections 2024-05-30T17:02:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.138+0000 7fcbbe200640 1 -- 172.21.15.67:0/458995042 wait complete. 2024-05-30T17:02:42.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.139+0000 7fcbbe200640 1 Processor -- start 2024-05-30T17:02:42.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.140+0000 7fcbbe200640 1 -- start start 2024-05-30T17:02:42.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.140+0000 7fcbbe200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb80cf870 0x7fcbb8159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:42.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.140+0000 7fcbbe200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbb80d14d0 0x7fcbb80b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbe200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb815e870 0x7fcbb80b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbe200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb80d0820 con 0x7fcbb80cf870 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbe200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb80d0540 con 0x7fcbb815e870 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbe200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb80d4fb0 con 0x7fcbb80d14d0 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb80cf870 0x7fcbb8159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb80cf870 0x7fcbb8159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60632/0 (socket says 172.21.15.67:60632) 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbce00640 1 -- 172.21.15.67:0/3856633481 learned_addr learned my addr 172.21.15.67:0/3856633481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbb7e00640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbb80d14d0 0x7fcbb80b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbd800640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb815e870 0x7fcbb80b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbce00640 1 -- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbb80d14d0 msgr2=0x7fcbb80b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbce00640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbb80d14d0 0x7fcbb80b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbce00640 1 -- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb815e870 msgr2=0x7fcbb80b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbce00640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb815e870 0x7fcbb80b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:42.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbce00640 1 -- 172.21.15.67:0/3856633481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb8159f90 con 0x7fcbb80cf870 2024-05-30T17:02:42.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbb7e00640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbb80d14d0 0x7fcbb80b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:42.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.141+0000 7fcbbd800640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb815e870 0x7fcbb80b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:42.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.142+0000 7fcbbce00640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb80cf870 0x7fcbb8159a50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fcba802ac70 tx=0x7fcba8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:42.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.142+0000 7fcbb4c00640 1 -- 172.21.15.67:0/3856633481 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8040070 con 0x7fcbb80cf870 2024-05-30T17:02:42.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.142+0000 7fcbb4c00640 1 -- 172.21.15.67:0/3856633481 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba8031c40 con 0x7fcbb80cf870 2024-05-30T17:02:42.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.142+0000 7fcbb4c00640 1 -- 172.21.15.67:0/3856633481 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba803c420 con 0x7fcbb80cf870 2024-05-30T17:02:42.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.142+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb806dff0 con 0x7fcbb80cf870 2024-05-30T17:02:42.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.142+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb815fd80 con 0x7fcbb80cf870 2024-05-30T17:02:42.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.143+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb80689e0 con 0x7fcbb80cf870 2024-05-30T17:02:42.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.145+0000 7fcbb4c00640 1 -- 172.21.15.67:0/3856633481 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fcba8039040 con 0x7fcbb80cf870 2024-05-30T17:02:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.146+0000 7fcbb4c00640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb9806e0c0 0x7fcb98070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.146+0000 7fcbb4c00640 1 -- 172.21.15.67:0/3856633481 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fcba80ac2f0 con 0x7fcbb80cf870 2024-05-30T17:02:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.146+0000 7fcbb7e00640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb9806e0c0 0x7fcb98070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.147+0000 7fcbb7e00640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb9806e0c0 0x7fcb98070580 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcbb815f590 tx=0x7fcba4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:42.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.151+0000 7fcbb4c00640 1 -- 172.21.15.67:0/3856633481 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbb8159f90 con 0x7fcbb80cf870 2024-05-30T17:02:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:42.404+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcbb80d1180 con 0x7fcbb80cf870 2024-05-30T17:02:42.950 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-05-30T17:02:42.949+0000 7f129a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T17:02:43.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.015+0000 7fcbb4c00640 1 -- 172.21.15.67:0/3856633481 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) ==== 118+0+0 (secure 0 0 0) 0x7fcba807bd70 con 0x7fcbb80cf870 2024-05-30T17:02:43.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb9806e0c0 msgr2=0x7fcb98070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:43.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb9806e0c0 0x7fcb98070580 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcbb815f590 tx=0x7fcba4008040 comp rx=0 tx=0).stop 2024-05-30T17:02:43.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb80cf870 msgr2=0x7fcbb8159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:43.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb80cf870 0x7fcbb8159a50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fcba802ac70 tx=0x7fcba8004a40 comp rx=0 tx=0).stop 2024-05-30T17:02:43.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 shutdown_connections 2024-05-30T17:02:43.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcb9806e0c0 0x7fcb98070580 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb815e870 0x7fcbb80b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbb80d14d0 0x7fcbb80b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 --2- 172.21.15.67:0/3856633481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbb80cf870 0x7fcbb8159a50 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.019+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 >> 172.21.15.67:0/3856633481 conn(0x7fcbb808bfc0 msgr2=0x7fcbb80b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:43.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.020+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 shutdown_connections 2024-05-30T17:02:43.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.020+0000 7fcbbe200640 1 -- 172.21.15.67:0/3856633481 wait complete. 2024-05-30T17:02:43.042 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-30T17:02:43.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-30T17:02:43.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.251+0000 7f77d5200640 1 Processor -- start 2024-05-30T17:02:43.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.251+0000 7f77d5200640 1 -- start start 2024-05-30T17:02:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.252+0000 7f77d5200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f77d0153510 0x7f77d00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.252+0000 7f77d5200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d015d900 con 0x7f77d015d270 2024-05-30T17:02:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.252+0000 7f77d5200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d00b3f90 con 0x7f77d00ccb90 2024-05-30T17:02:43.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.252+0000 7f77d5200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f77d01538f0 con 0x7f77d0153510 2024-05-30T17:02:43.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.253+0000 7f77cfe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f77d015d270 0x7f77d006e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58612/0 (socket says 172.21.15.67:58612) 2024-05-30T17:02:43.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.253+0000 7f77cfe00640 1 -- 172.21.15.67:0/752419312 learned_addr learned my addr 172.21.15.67:0/752419312 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:43.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.253+0000 7f77cf400640 1 --2- 172.21.15.67:0/752419312 >> v2:172.21.15.169:3300/0 conn(0x7f77d0153510 0x7f77d00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:43.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177331930 0 0) 0x7f77d015d900 con 0x7f77d015d270 2024-05-30T17:02:43.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77b4003520 con 0x7f77d015d270 2024-05-30T17:02:43.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 58294077 0 0) 0x7f77b4003520 con 0x7f77d015d270 2024-05-30T17:02:43.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 >> v2:172.21.15.169:3300/0 conn(0x7f77d0153510 msgr2=0x7f77d00b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:43.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 --2- 172.21.15.67:0/752419312 >> v2:172.21.15.169:3300/0 conn(0x7f77d0153510 0x7f77d00b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 >> v1:172.21.15.169:6789/0 conn(0x7f77d00ccb90 legacy=0x7f77d015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d00b4270 con 0x7f77d015d270 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f77c80019f0 con 0x7f77d015d270 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 >> v1:172.21.15.67:6789/0 conn(0x7f77d015d270 legacy=0x7f77d006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77b4003a00 0x7f77b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77b4004800 0x7f77b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4005210 0x7f77b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f77b400c130 con 0x7f77b4003a00 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f77b400c2e0 con 0x7f77b4004800 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.254+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f77b400c4e0 con 0x7f77b4005210 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cf400640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77b4003a00 0x7f77b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:43.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cea00640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77b4004800 0x7f77b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:43.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cf400640 1 -- 172.21.15.67:0/752419312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4005210 msgr2=0x7f77b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:43.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cfe00640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4005210 0x7f77b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:43.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cf400640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4005210 0x7f77b400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cf400640 1 -- 172.21.15.67:0/752419312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77b4004800 msgr2=0x7f77b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:43.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cf400640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77b4004800 0x7f77b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cf400640 1 -- 172.21.15.67:0/752419312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b4003520 con 0x7f77b4003a00 2024-05-30T17:02:43.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cea00640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77b4004800 0x7f77b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:43.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.255+0000 7f77cfe00640 1 -- 172.21.15.67:0/752419312 reap_dead start 2024-05-30T17:02:43.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.256+0000 7f77cf400640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77b4003a00 0x7f77b4003e20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009a90 tx=0x7f77b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616cc69fd316b4c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:43.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.256+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0036070 con 0x7f77b4003a00 2024-05-30T17:02:43.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.256+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c0031400 con 0x7f77b4003a00 2024-05-30T17:02:43.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.256+0000 7f77ce000640 1 -- 172.21.15.67:0/752419312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0009690 con 0x7f77b4003a00 2024-05-30T17:02:43.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.257+0000 7f77d5200640 1 -- 172.21.15.67:0/752419312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77b4003a00 msgr2=0x7f77b4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:43.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.257+0000 7f77d5200640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77b4003a00 0x7f77b4003e20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009a90 tx=0x7f77b4005830 comp rx=0 tx=0).stop 2024-05-30T17:02:43.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.257+0000 7f77d5200640 1 -- 172.21.15.67:0/752419312 shutdown_connections 2024-05-30T17:02:43.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.257+0000 7f77d5200640 1 --2- 172.21.15.67:0/752419312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77b4003a00 0x7f77b4003e20 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.257+0000 7f77d5200640 1 -- 172.21.15.67:0/752419312 >> 172.21.15.67:0/752419312 conn(0x7f77d006b900 msgr2=0x7f77d006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:43.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.257+0000 7f77d5200640 1 -- 172.21.15.67:0/752419312 shutdown_connections 2024-05-30T17:02:43.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.257+0000 7f77d5200640 1 -- 172.21.15.67:0/752419312 wait complete. 2024-05-30T17:02:43.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.259+0000 7f77d5200640 1 Processor -- start 2024-05-30T17:02:43.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.259+0000 7f77d5200640 1 -- start start 2024-05-30T17:02:43.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.260+0000 7f77d5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4003a00 0x7f77d014cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:43.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.260+0000 7f77d5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00aea80 0x7f77d006ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:43.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.260+0000 7f77d5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d00aef00 0x7f77d006f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:43.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.260+0000 7f77d5200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f77d015d6a0 con 0x7f77d00aef00 2024-05-30T17:02:43.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.260+0000 7f77d5200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f77d00b48c0 con 0x7f77d00aea80 2024-05-30T17:02:43.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.260+0000 7f77d5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f77d00b4d50 con 0x7f77b4003a00 2024-05-30T17:02:43.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.260+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d00aef00 0x7f77d006f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:43.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.260+0000 7f77cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d00aef00 0x7f77d006f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60654/0 (socket says 172.21.15.67:60654) 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.260+0000 7f77cfe00640 1 -- 172.21.15.67:0/243679284 learned_addr learned my addr 172.21.15.67:0/243679284 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77cea00640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00aea80 0x7f77d006ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77cf400640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4003a00 0x7f77d014cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77cfe00640 1 -- 172.21.15.67:0/243679284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4003a00 msgr2=0x7f77d014cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77cfe00640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4003a00 0x7f77d014cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77cfe00640 1 -- 172.21.15.67:0/243679284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00aea80 msgr2=0x7f77d006ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77cfe00640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00aea80 0x7f77d006ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77cfe00640 1 -- 172.21.15.67:0/243679284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d014d040 con 0x7f77d00aef00 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77cf400640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4003a00 0x7f77d014cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77cfe00640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d00aef00 0x7f77d006f390 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f77d00b3c10 tx=0x7f77c800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.261+0000 7f77c7400640 1 -- 172.21.15.67:0/243679284 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8004d70 con 0x7f77d00aef00 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.262+0000 7f77c7400640 1 -- 172.21.15.67:0/243679284 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c800bb80 con 0x7f77d00aef00 2024-05-30T17:02:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.262+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d0072860 con 0x7f77d00aef00 2024-05-30T17:02:43.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.262+0000 7f77c7400640 1 -- 172.21.15.67:0/243679284 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8007110 con 0x7f77d00aef00 2024-05-30T17:02:43.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.262+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d0072e20 con 0x7f77d00aef00 2024-05-30T17:02:43.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.263+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d00b4a70 con 0x7f77d00aef00 2024-05-30T17:02:43.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.264+0000 7f77c7400640 1 -- 172.21.15.67:0/243679284 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f77c801b060 con 0x7f77d00aef00 2024-05-30T17:02:43.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.265+0000 7f77c7400640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77b006e090 0x7f77b0070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:43.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.265+0000 7f77c7400640 1 -- 172.21.15.67:0/243679284 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f77c808a980 con 0x7f77d00aef00 2024-05-30T17:02:43.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.265+0000 7f77cf400640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77b006e090 0x7f77b0070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:43.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.266+0000 7f77cf400640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77b006e090 0x7f77b0070550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f77c0033000 tx=0x7f77c0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:43.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.269+0000 7f77c7400640 1 -- 172.21.15.67:0/243679284 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77d00b4a70 con 0x7f77d00aef00 2024-05-30T17:02:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:43.517+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f77d00b3c40 con 0x7f77d00aef00 2024-05-30T17:02:44.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.019+0000 7f77c7400640 1 -- 172.21.15.67:0/243679284 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) ==== 122+0+0 (secure 0 0 0) 0x7f77d014d040 con 0x7f77d00aef00 2024-05-30T17:02:44.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.022+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77b006e090 msgr2=0x7f77b0070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.022+0000 7f77d5200640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77b006e090 0x7f77b0070550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f77c0033000 tx=0x7f77c0008030 comp rx=0 tx=0).stop 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.022+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d00aef00 msgr2=0x7f77d006f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.022+0000 7f77d5200640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d00aef00 0x7f77d006f390 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f77d00b3c10 tx=0x7f77c800b4a0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.023+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 shutdown_connections 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.023+0000 7f77d5200640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f77b006e090 0x7f77b0070550 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.023+0000 7f77d5200640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77d00aef00 0x7f77d006f390 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.023+0000 7f77d5200640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00aea80 0x7f77d006ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.023+0000 7f77d5200640 1 --2- 172.21.15.67:0/243679284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77b4003a00 0x7f77d014cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.023+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 >> 172.21.15.67:0/243679284 conn(0x7f77d006b900 msgr2=0x7f77d00cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.023+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 shutdown_connections 2024-05-30T17:02:44.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.023+0000 7f77d5200640 1 -- 172.21.15.67:0/243679284 wait complete. 2024-05-30T17:02:44.046 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-30T17:02:44.046 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:02:44.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.257+0000 7f5dca000640 1 Processor -- start 2024-05-30T17:02:44.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.257+0000 7f5dca000640 1 -- start start 2024-05-30T17:02:44.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.259+0000 7f5dca000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5dc406f860 0x7f5dc40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.259+0000 7f5dca000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40d0bb0 0x7f5dc40ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.259+0000 7f5dca000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc406ed10 con 0x7f5dc40b3690 2024-05-30T17:02:44.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.259+0000 7f5dca000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc415cb00 con 0x7f5dc406f860 2024-05-30T17:02:44.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.259+0000 7f5dca000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc40b3cc0 con 0x7f5dc40d0bb0 2024-05-30T17:02:44.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.259+0000 7f5dc3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5dc406f860 0x7f5dc40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.259+0000 7f5dc2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5dc40b3690 0x7f5dc40d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58624/0 (socket says 172.21.15.67:58624) 2024-05-30T17:02:44.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.259+0000 7f5dc2a00640 1 -- 172.21.15.67:0/1806865507 learned_addr learned my addr 172.21.15.67:0/1806865507 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:44.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.260+0000 7f5dc3e00640 1 --2- 172.21.15.67:0/1806865507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40d0bb0 0x7f5dc40ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.260+0000 7f5dc3400640 1 -- 172.21.15.67:0/1806865507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40d0bb0 msgr2=0x7f5dc40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:44.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.260+0000 7f5dc3400640 1 --2- 172.21.15.67:0/1806865507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40d0bb0 0x7f5dc40ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.260+0000 7f5dc3400640 1 -- 172.21.15.67:0/1806865507 >> v1:172.21.15.67:6789/0 conn(0x7f5dc40b3690 legacy=0x7f5dc40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.260+0000 7f5dc3400640 1 -- 172.21.15.67:0/1806865507 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc40b3fa0 con 0x7f5dc406f860 2024-05-30T17:02:44.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.260+0000 7f5dc3e00640 1 --2- 172.21.15.67:0/1806865507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40d0bb0 0x7f5dc40ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:44.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.261+0000 7f5dc3400640 1 --2- 172.21.15.67:0/1806865507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 0x7f5dc40b3150 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f5db40097a0 tx=0x7f5db402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f760baec01bb893 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:44.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.261+0000 7f5dc2000640 1 -- 172.21.15.67:0/1806865507 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4037070 con 0x7f5dc406f860 2024-05-30T17:02:44.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.261+0000 7f5dc2000640 1 -- 172.21.15.67:0/1806865507 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db4032440 con 0x7f5dc406f860 2024-05-30T17:02:44.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.261+0000 7f5dc2000640 1 -- 172.21.15.67:0/1806865507 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4032810 con 0x7f5dc406f860 2024-05-30T17:02:44.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.262+0000 7f5dca000640 1 -- 172.21.15.67:0/1806865507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 msgr2=0x7f5dc40b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.262+0000 7f5dca000640 1 --2- 172.21.15.67:0/1806865507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 0x7f5dc40b3150 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f5db40097a0 tx=0x7f5db402a180 comp rx=0 tx=0).stop 2024-05-30T17:02:44.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.262+0000 7f5dca000640 1 -- 172.21.15.67:0/1806865507 shutdown_connections 2024-05-30T17:02:44.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.262+0000 7f5dca000640 1 --2- 172.21.15.67:0/1806865507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40d0bb0 0x7f5dc40ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.262+0000 7f5dca000640 1 --2- 172.21.15.67:0/1806865507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 0x7f5dc40b3150 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.262+0000 7f5dca000640 1 -- 172.21.15.67:0/1806865507 >> 172.21.15.67:0/1806865507 conn(0x7f5dc406bae0 msgr2=0x7f5dc406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:44.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.262+0000 7f5dca000640 1 -- 172.21.15.67:0/1806865507 shutdown_connections 2024-05-30T17:02:44.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.262+0000 7f5dca000640 1 -- 172.21.15.67:0/1806865507 wait complete. 2024-05-30T17:02:44.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.264+0000 7f5dca000640 1 Processor -- start 2024-05-30T17:02:44.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.264+0000 7f5dca000640 1 -- start start 2024-05-30T17:02:44.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dca000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 0x7f5dc40c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dca000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40b3690 0x7f5dc415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dca000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40d0bb0 0x7f5dc41f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dca000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40b4840 con 0x7f5dc406f860 2024-05-30T17:02:44.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dca000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40b4560 con 0x7f5dc40d0bb0 2024-05-30T17:02:44.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dca000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc40ad670 con 0x7f5dc40b3690 2024-05-30T17:02:44.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dc3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 0x7f5dc40c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dc3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 0x7f5dc40c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60662/0 (socket says 172.21.15.67:60662) 2024-05-30T17:02:44.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dc3400640 1 -- 172.21.15.67:0/3821187662 learned_addr learned my addr 172.21.15.67:0/3821187662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:44.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.265+0000 7f5dc3400640 1 -- 172.21.15.67:0/3821187662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40b3690 msgr2=0x7f5dc415f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dc3400640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40b3690 0x7f5dc415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dc3400640 1 -- 172.21.15.67:0/3821187662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40d0bb0 msgr2=0x7f5dc41f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dc3400640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40d0bb0 0x7f5dc41f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dc3400640 1 -- 172.21.15.67:0/3821187662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc415fb20 con 0x7f5dc406f860 2024-05-30T17:02:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dc3400640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 0x7f5dc40c7330 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f5db4035040 tx=0x7f5db40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dbb400640 1 -- 172.21.15.67:0/3821187662 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4037040 con 0x7f5dc406f860 2024-05-30T17:02:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dbb400640 1 -- 172.21.15.67:0/3821187662 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db4030770 con 0x7f5dc406f860 2024-05-30T17:02:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc40c7910 con 0x7f5dc406f860 2024-05-30T17:02:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dbb400640 1 -- 172.21.15.67:0/3821187662 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4030b80 con 0x7f5dc406f860 2024-05-30T17:02:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.266+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc40c81b0 con 0x7f5dc406f860 2024-05-30T17:02:44.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.267+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc40689e0 con 0x7f5dc406f860 2024-05-30T17:02:44.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.269+0000 7f5dbb400640 1 -- 172.21.15.67:0/3821187662 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f5db4044020 con 0x7f5dc406f860 2024-05-30T17:02:44.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.270+0000 7f5dbb400640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406e090 0x7f5da4070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.270+0000 7f5dbb400640 1 -- 172.21.15.67:0/3821187662 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f5db40aaf30 con 0x7f5dc406f860 2024-05-30T17:02:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.270+0000 7f5dc2a00640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406e090 0x7f5da4070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.271+0000 7f5dc2a00640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406e090 0x7f5da4070550 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000f60 tx=0x7f5db0007a80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:44.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.275+0000 7f5dbb400640 1 -- 172.21.15.67:0/3821187662 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5db4032e20 con 0x7f5dc406f860 2024-05-30T17:02:44.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.517+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dc40cefc0 con 0x7f5dc406f860 2024-05-30T17:02:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.518+0000 7f5dbb400640 1 -- 172.21.15.67:0/3821187662 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5189 (secure 0 0 0) 0x7f5db407add0 con 0x7f5dc406f860 2024-05-30T17:02:44.519 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:44.519 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1973461214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1973461214},{"type":"v1","addr":"172.21.15.169:6835","nonce":1973461214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":8400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3884958123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3884958123},{"type":"v1","addr":"172.21.15.67:6836","nonce":3884958123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":8403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1126347083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1126347083},{"type":"v1","addr":"172.21.15.67:6837","nonce":1126347083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:02:40.462403+0000","modified":"2024-05-30T17:02:43.518891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3425947383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3425947383},{"type":"v1","addr":"172.21.15.169:6834","nonce":3425947383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":8}]} 2024-05-30T17:02:44.519 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 73 2024-05-30T17:02:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406e090 msgr2=0x7f5da4070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406e090 0x7f5da4070550 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000f60 tx=0x7f5db0007a80 comp rx=0 tx=0).stop 2024-05-30T17:02:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 msgr2=0x7f5dc40c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 0x7f5dc40c7330 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f5db4035040 tx=0x7f5db40046c0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 shutdown_connections 2024-05-30T17:02:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5da406e090 0x7f5da4070550 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40d0bb0 0x7f5dc41f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dc40b3690 0x7f5dc415f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 --2- 172.21.15.67:0/3821187662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dc406f860 0x7f5dc40c7330 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.521+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 >> 172.21.15.67:0/3821187662 conn(0x7f5dc406bae0 msgr2=0x7f5dc40d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.522+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 shutdown_connections 2024-05-30T17:02:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.522+0000 7f5dca000640 1 -- 172.21.15.67:0/3821187662 wait complete. 2024-05-30T17:02:44.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:02:44.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.743+0000 7fc186e00640 1 Processor -- start 2024-05-30T17:02:44.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.743+0000 7fc186e00640 1 -- start start 2024-05-30T17:02:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.745+0000 7fc186e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 0x7fc1800b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.745+0000 7fc186e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc18006dc20 0x7fc18015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.745+0000 7fc186e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1800afd60 con 0x7fc18015b3a0 2024-05-30T17:02:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.745+0000 7fc186e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc18006f980 con 0x7fc18006dc20 2024-05-30T17:02:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.745+0000 7fc186e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1800ae0c0 con 0x7fc180073310 2024-05-30T17:02:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.745+0000 7fc185a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc18015b3a0 0x7fc180072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58626/0 (socket says 172.21.15.67:58626) 2024-05-30T17:02:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.745+0000 7fc185a00640 1 -- 172.21.15.67:0/4076281163 learned_addr learned my addr 172.21.15.67:0/4076281163 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.745+0000 7fc186400640 1 --2- 172.21.15.67:0/4076281163 >> v2:172.21.15.67:3300/0 conn(0x7fc18006dc20 0x7fc18015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.746+0000 7fc185000640 1 --2- 172.21.15.67:0/4076281163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 0x7fc1800b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.746+0000 7fc17fe00640 1 -- 172.21.15.67:0/4076281163 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 80167840 0 0) 0x7fc1800afd60 con 0x7fc18015b3a0 2024-05-30T17:02:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.746+0000 7fc17fe00640 1 -- 172.21.15.67:0/4076281163 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc168003520 con 0x7fc18015b3a0 2024-05-30T17:02:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.746+0000 7fc186400640 1 -- 172.21.15.67:0/4076281163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 msgr2=0x7fc1800b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.746+0000 7fc186400640 1 --2- 172.21.15.67:0/4076281163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 0x7fc1800b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.746+0000 7fc186400640 1 -- 172.21.15.67:0/4076281163 >> v1:172.21.15.67:6789/0 conn(0x7fc18015b3a0 legacy=0x7fc180072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.746+0000 7fc186400640 1 -- 172.21.15.67:0/4076281163 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1800af7a0 con 0x7fc18006dc20 2024-05-30T17:02:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.747+0000 7fc186400640 1 --2- 172.21.15.67:0/4076281163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18006dc20 0x7fc18015f9a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc178009d90 tx=0x7fc17802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6006f1934fa1b83c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.747+0000 7fc17fe00640 1 -- 172.21.15.67:0/4076281163 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178007740 con 0x7fc18006dc20 2024-05-30T17:02:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.747+0000 7fc17fe00640 1 -- 172.21.15.67:0/4076281163 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1780078e0 con 0x7fc18006dc20 2024-05-30T17:02:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.747+0000 7fc17fe00640 1 -- 172.21.15.67:0/4076281163 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178007c90 con 0x7fc18006dc20 2024-05-30T17:02:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.748+0000 7fc186e00640 1 -- 172.21.15.67:0/4076281163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18006dc20 msgr2=0x7fc18015f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.748+0000 7fc186e00640 1 --2- 172.21.15.67:0/4076281163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18006dc20 0x7fc18015f9a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc178009d90 tx=0x7fc17802a660 comp rx=0 tx=0).stop 2024-05-30T17:02:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.748+0000 7fc186e00640 1 -- 172.21.15.67:0/4076281163 shutdown_connections 2024-05-30T17:02:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.748+0000 7fc186e00640 1 --2- 172.21.15.67:0/4076281163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18006dc20 0x7fc18015f9a0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.748+0000 7fc186e00640 1 --2- 172.21.15.67:0/4076281163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 0x7fc1800b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.748+0000 7fc186e00640 1 -- 172.21.15.67:0/4076281163 >> 172.21.15.67:0/4076281163 conn(0x7fc1800b1fc0 msgr2=0x7fc1800b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.748+0000 7fc186e00640 1 -- 172.21.15.67:0/4076281163 shutdown_connections 2024-05-30T17:02:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.748+0000 7fc186e00640 1 -- 172.21.15.67:0/4076281163 wait complete. 2024-05-30T17:02:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.750+0000 7fc186e00640 1 Processor -- start 2024-05-30T17:02:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.750+0000 7fc186e00640 1 -- start start 2024-05-30T17:02:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.751+0000 7fc186e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18006dc20 0x7fc1800ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.751+0000 7fc186e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 0x7fc1800ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.751+0000 7fc186e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18015b3a0 0x7fc1800cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.751+0000 7fc186e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc18015b780 con 0x7fc18015b3a0 2024-05-30T17:02:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.751+0000 7fc186e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc18015fee0 con 0x7fc18006dc20 2024-05-30T17:02:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.751+0000 7fc186e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1800ae550 con 0x7fc180073310 2024-05-30T17:02:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc186400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18015b3a0 0x7fc1800cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc186400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18015b3a0 0x7fc1800cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60682/0 (socket says 172.21.15.67:60682) 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc186400640 1 -- 172.21.15.67:0/985471426 learned_addr learned my addr 172.21.15.67:0/985471426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc185000640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 0x7fc1800ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc185a00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18006dc20 0x7fc1800ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc186400640 1 -- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 msgr2=0x7fc1800ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc186400640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 0x7fc1800ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc186400640 1 -- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18006dc20 msgr2=0x7fc1800ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc186400640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18006dc20 0x7fc1800ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc186400640 1 -- 172.21.15.67:0/985471426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18015a240 con 0x7fc18015b3a0 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc185000640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 0x7fc1800ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc185a00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18006dc20 0x7fc1800ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.752+0000 7fc186400640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18015b3a0 0x7fc1800cd440 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc178009eb0 tx=0x7fc178004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.753+0000 7fc17d600640 1 -- 172.21.15.67:0/985471426 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178040070 con 0x7fc18015b3a0 2024-05-30T17:02:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.753+0000 7fc17d600640 1 -- 172.21.15.67:0/985471426 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc178031c60 con 0x7fc18015b3a0 2024-05-30T17:02:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.753+0000 7fc17d600640 1 -- 172.21.15.67:0/985471426 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178009450 con 0x7fc18015b3a0 2024-05-30T17:02:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.753+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc180159f60 con 0x7fc18015b3a0 2024-05-30T17:02:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.753+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18014e7a0 con 0x7fc18015b3a0 2024-05-30T17:02:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.754+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18014e2e0 con 0x7fc18015b3a0 2024-05-30T17:02:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.755+0000 7fc17d600640 1 -- 172.21.15.67:0/985471426 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fc178039040 con 0x7fc18015b3a0 2024-05-30T17:02:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.756+0000 7fc17d600640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc16006e0c0 0x7fc160070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.756+0000 7fc17d600640 1 -- 172.21.15.67:0/985471426 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fc1780abda0 con 0x7fc18015b3a0 2024-05-30T17:02:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.756+0000 7fc185a00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc16006e0c0 0x7fc160070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.757+0000 7fc185a00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc16006e0c0 0x7fc160070580 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc174005880 tx=0x7fc174009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.761+0000 7fc17d600640 1 -- 172.21.15.67:0/985471426 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc18014e2e0 con 0x7fc18015b3a0 2024-05-30T17:02:44.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.965+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1800ce9f0 con 0x7fc18015b3a0 2024-05-30T17:02:44.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.969+0000 7fc17d600640 1 -- 172.21.15.67:0/985471426 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16792 (secure 0 0 0) 0x7fc17807bc30 con 0x7fc18015b3a0 2024-05-30T17:02:44.970 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:44.970 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":116,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:02:41.991177+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:02:38.380492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-30T17:02:39.422718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:02:44.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.971+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc16006e0c0 msgr2=0x7fc160070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.971+0000 7fc186e00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc16006e0c0 0x7fc160070580 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc174005880 tx=0x7fc174009040 comp rx=0 tx=0).stop 2024-05-30T17:02:44.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.971+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18015b3a0 msgr2=0x7fc1800cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:44.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.971+0000 7fc186e00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18015b3a0 0x7fc1800cd440 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc178009eb0 tx=0x7fc178004a40 comp rx=0 tx=0).stop 2024-05-30T17:02:44.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.972+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 shutdown_connections 2024-05-30T17:02:44.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.972+0000 7fc186e00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc16006e0c0 0x7fc160070580 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.972+0000 7fc186e00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc18015b3a0 0x7fc1800cd440 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.972+0000 7fc186e00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180073310 0x7fc1800ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.972+0000 7fc186e00640 1 --2- 172.21.15.67:0/985471426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc18006dc20 0x7fc1800ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:44.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.972+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 >> 172.21.15.67:0/985471426 conn(0x7fc1800b1fc0 msgr2=0x7fc180072c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:44.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.972+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 shutdown_connections 2024-05-30T17:02:44.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:44.972+0000 7fc186e00640 1 -- 172.21.15.67:0/985471426 wait complete. 2024-05-30T17:02:44.993 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T17:02:44.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:02:45.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.206+0000 7f1a63200640 1 Processor -- start 2024-05-30T17:02:45.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.206+0000 7f1a63200640 1 -- start start 2024-05-30T17:02:45.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.207+0000 7f1a63200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1a6415ea40 0x7f1a640cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.207+0000 7f1a63200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a640d5190 0x7f1a640d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.207+0000 7f1a63200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1a6415ee20 0x7f1a6415fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.207+0000 7f1a63200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f1a640cecf0 con 0x7f1a6415ee20 2024-05-30T17:02:45.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.207+0000 7f1a63200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1a640cefd0 con 0x7f1a6415ea40 2024-05-30T17:02:45.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.207+0000 7f1a63200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a640cf2b0 con 0x7f1a640d5190 2024-05-30T17:02:45.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a62800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1a6415ee20 0x7f1a6415fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a62800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1a6415ee20 0x7f1a6415fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60686/0 (socket says 172.21.15.67:60686) 2024-05-30T17:02:45.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a62800640 1 -- 172.21.15.67:0/2126474577 learned_addr learned my addr 172.21.15.67:0/2126474577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a61400640 1 --2- 172.21.15.67:0/2126474577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a640d5190 0x7f1a640d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a62800640 1 -- 172.21.15.67:0/2126474577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a640d5190 msgr2=0x7f1a640d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a61e00640 1 --2- 172.21.15.67:0/2126474577 >> v2:172.21.15.169:3300/0 conn(0x7f1a6415ea40 0x7f1a640cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a62800640 1 --2- 172.21.15.67:0/2126474577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a640d5190 0x7f1a640d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a62800640 1 -- 172.21.15.67:0/2126474577 >> v2:172.21.15.169:3300/0 conn(0x7f1a6415ea40 msgr2=0x7f1a640cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a62800640 1 --2- 172.21.15.67:0/2126474577 >> v2:172.21.15.169:3300/0 conn(0x7f1a6415ea40 0x7f1a640cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a62800640 1 -- 172.21.15.67:0/2126474577 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a640cf590 con 0x7f1a6415ee20 2024-05-30T17:02:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.208+0000 7f1a61400640 1 --2- 172.21.15.67:0/2126474577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a640d5190 0x7f1a640d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.209+0000 7f1a62800640 1 --2- 172.21.15.67:0/2126474577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ee20 0x7f1a6415fff0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1a6408cbe0 tx=0x7f1a5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3799dc3bd0cb6f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:45.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a60a00640 1 -- 172.21.15.67:0/2126474577 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c037070 con 0x7f1a6415ee20 2024-05-30T17:02:45.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a60a00640 1 -- 172.21.15.67:0/2126474577 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a5c032440 con 0x7f1a6415ee20 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a60a00640 1 -- 172.21.15.67:0/2126474577 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c032810 con 0x7f1a6415ee20 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a63200640 1 -- 172.21.15.67:0/2126474577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ee20 msgr2=0x7f1a6415fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a63200640 1 --2- 172.21.15.67:0/2126474577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ee20 0x7f1a6415fff0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1a6408cbe0 tx=0x7f1a5c02a1e0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a63200640 1 -- 172.21.15.67:0/2126474577 shutdown_connections 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a63200640 1 --2- 172.21.15.67:0/2126474577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ee20 0x7f1a6415fff0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a63200640 1 --2- 172.21.15.67:0/2126474577 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a640d5190 0x7f1a640d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a63200640 1 --2- 172.21.15.67:0/2126474577 >> v2:172.21.15.169:3300/0 conn(0x7f1a6415ea40 0x7f1a640cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a63200640 1 -- 172.21.15.67:0/2126474577 >> 172.21.15.67:0/2126474577 conn(0x7f1a6408bfc0 msgr2=0x7f1a640d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.210+0000 7f1a63200640 1 -- 172.21.15.67:0/2126474577 shutdown_connections 2024-05-30T17:02:45.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.211+0000 7f1a63200640 1 -- 172.21.15.67:0/2126474577 wait complete. 2024-05-30T17:02:45.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.212+0000 7f1a63200640 1 Processor -- start 2024-05-30T17:02:45.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.212+0000 7f1a63200640 1 -- start start 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.213+0000 7f1a63200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a640d5190 0x7f1a6414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.213+0000 7f1a63200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ea40 0x7f1a6414c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.213+0000 7f1a63200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6415ee20 0x7f1a640c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.213+0000 7f1a63200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1a640d3760 con 0x7f1a6415ea40 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.213+0000 7f1a63200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a640d35e0 con 0x7f1a640d5190 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.213+0000 7f1a63200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6415f2d0 con 0x7f1a6415ee20 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a61400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ea40 0x7f1a6414c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a61400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ea40 0x7f1a6414c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60698/0 (socket says 172.21.15.67:60698) 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a61400640 1 -- 172.21.15.67:0/3349112769 learned_addr learned my addr 172.21.15.67:0/3349112769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a61e00640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a640d5190 0x7f1a6414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a62800640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6415ee20 0x7f1a640c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a61400640 1 -- 172.21.15.67:0/3349112769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6415ee20 msgr2=0x7f1a640c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a61400640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6415ee20 0x7f1a640c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a61400640 1 -- 172.21.15.67:0/3349112769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a640d5190 msgr2=0x7f1a6414c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a61400640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a640d5190 0x7f1a6414c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.214+0000 7f1a61400640 1 -- 172.21.15.67:0/3349112769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a641f45d0 con 0x7f1a6415ea40 2024-05-30T17:02:45.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.215+0000 7f1a61400640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ea40 0x7f1a6414c930 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c7c0 tx=0x7f1a5000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:45.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.215+0000 7f1a5a000640 1 -- 172.21.15.67:0/3349112769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50005b90 con 0x7f1a6415ea40 2024-05-30T17:02:45.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.215+0000 7f1a5a000640 1 -- 172.21.15.67:0/3349112769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a50005d30 con 0x7f1a6415ea40 2024-05-30T17:02:45.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.215+0000 7f1a5a000640 1 -- 172.21.15.67:0/3349112769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50006250 con 0x7f1a6415ea40 2024-05-30T17:02:45.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.215+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a640ce630 con 0x7f1a6415ea40 2024-05-30T17:02:45.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.215+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a641e8830 con 0x7f1a6415ea40 2024-05-30T17:02:45.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.216+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a640690f0 con 0x7f1a6415ea40 2024-05-30T17:02:45.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.217+0000 7f1a5a000640 1 -- 172.21.15.67:0/3349112769 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f1a500063f0 con 0x7f1a6415ea40 2024-05-30T17:02:45.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.218+0000 7f1a5a000640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4406e090 0x7f1a44070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.218+0000 7f1a5a000640 1 -- 172.21.15.67:0/3349112769 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f1a5008a950 con 0x7f1a6415ea40 2024-05-30T17:02:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.218+0000 7f1a61e00640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4406e090 0x7f1a44070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.219+0000 7f1a61e00640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4406e090 0x7f1a44070550 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1a540070b0 tx=0x7f1a54007040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:45.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.223+0000 7f1a5a000640 1 -- 172.21.15.67:0/3349112769 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a50015020 con 0x7f1a6415ea40 2024-05-30T17:02:45.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.426+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a6408c650 con 0x7f1a4406e090 2024-05-30T17:02:45.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.444+0000 7f1a5a000640 1 -- 172.21.15.67:0/3349112769 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247323 (secure 0 0 0) 0x7f1a6408c650 con 0x7f1a4406e090 2024-05-30T17:02:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:45.450 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4406e090 msgr2=0x7f1a44070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4406e090 0x7f1a44070550 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1a540070b0 tx=0x7f1a54007040 comp rx=0 tx=0).stop 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ea40 msgr2=0x7f1a6414c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ea40 0x7f1a6414c930 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c7c0 tx=0x7f1a5000cc80 comp rx=0 tx=0).stop 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 shutdown_connections 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f1a4406e090 0x7f1a44070550 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a6415ee20 0x7f1a640c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a6415ea40 0x7f1a6414c930 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 --2- 172.21.15.67:0/3349112769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a640d5190 0x7f1a6414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 >> 172.21.15.67:0/3349112769 conn(0x7f1a6408bfc0 msgr2=0x7f1a640d25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 shutdown_connections 2024-05-30T17:02:45.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.449+0000 7f1a63200640 1 -- 172.21.15.67:0/3349112769 wait complete. 2024-05-30T17:02:45.470 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":682,"stamp":"2024-05-30T17:02:44.522439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":184,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":984432,"kb_used_data":12208,"kb_used_omap":12,"kb_used_meta":972083,"kb_avail":753990288,"statfs":{"total":773094113280,"available":772086054912,"internally_reserved":0,"allocated":12500992,"data_stored":6118735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":995413552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570340"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986625+0000","last_change":"2024-05-30T17:02:39.986625+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986625+0000","last_undegraded":"2024-05-30T17:02:39.986625+0000","last_fullsized":"2024-05-30T17:02:39.986625+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738383+0000","last_change":"2024-05-30T17:02:41.013698+0000","last_active":"2024-05-30T17:02:43.738383+0000","last_peered":"2024-05-30T17:02:43.738383+0000","last_clean":"2024-05-30T17:02:43.738383+0000","last_became_active":"2024-05-30T17:02:41.013407+0000","last_became_peered":"2024-05-30T17:02:41.013407+0000","last_unstale":"2024-05-30T17:02:43.738383+0000","last_undegraded":"2024-05-30T17:02:43.738383+0000","last_fullsized":"2024-05-30T17:02:43.738383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:56.046859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.008923+0000","last_change":"2024-05-30T17:02:41.006127+0000","last_active":"2024-05-30T17:02:41.008923+0000","last_peered":"2024-05-30T17:02:41.008923+0000","last_clean":"2024-05-30T17:02:41.008923+0000","last_became_active":"2024-05-30T17:02:41.005503+0000","last_became_peered":"2024-05-30T17:02:41.005503+0000","last_unstale":"2024-05-30T17:02:41.008923+0000","last_undegraded":"2024-05-30T17:02:41.008923+0000","last_fullsized":"2024-05-30T17:02:41.008923+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:01.078245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.982145+0000","last_change":"2024-05-30T17:02:39.982145+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.982145+0000","last_undegraded":"2024-05-30T17:02:39.982145+0000","last_fullsized":"2024-05-30T17:02:39.982145+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011985+0000","last_change":"2024-05-30T17:02:41.010886+0000","last_active":"2024-05-30T17:02:42.011985+0000","last_peered":"2024-05-30T17:02:42.011985+0000","last_clean":"2024-05-30T17:02:42.011985+0000","last_became_active":"2024-05-30T17:02:41.007377+0000","last_became_peered":"2024-05-30T17:02:41.007377+0000","last_unstale":"2024-05-30T17:02:42.011985+0000","last_undegraded":"2024-05-30T17:02:42.011985+0000","last_fullsized":"2024-05-30T17:02:42.011985+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:59.071677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738235+0000","last_change":"2024-05-30T17:02:41.014332+0000","last_active":"2024-05-30T17:02:43.738235+0000","last_peered":"2024-05-30T17:02:43.738235+0000","last_clean":"2024-05-30T17:02:43.738235+0000","last_became_active":"2024-05-30T17:02:41.014074+0000","last_became_peered":"2024-05-30T17:02:41.014074+0000","last_unstale":"2024-05-30T17:02:43.738235+0000","last_undegraded":"2024-05-30T17:02:43.738235+0000","last_fullsized":"2024-05-30T17:02:43.738235+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:00:36.629725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.987823+0000","last_change":"2024-05-30T17:02:39.987823+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.987823+0000","last_undegraded":"2024-05-30T17:02:39.987823+0000","last_fullsized":"2024-05-30T17:02:39.987823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738860+0000","last_change":"2024-05-30T17:02:41.011886+0000","last_active":"2024-05-30T17:02:43.738860+0000","last_peered":"2024-05-30T17:02:43.738860+0000","last_clean":"2024-05-30T17:02:43.738860+0000","last_became_active":"2024-05-30T17:02:41.010296+0000","last_became_peered":"2024-05-30T17:02:41.010296+0000","last_unstale":"2024-05-30T17:02:43.738860+0000","last_undegraded":"2024-05-30T17:02:43.738860+0000","last_fullsized":"2024-05-30T17:02:43.738860+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:20.421009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737964+0000","last_change":"2024-05-30T17:02:41.012428+0000","last_active":"2024-05-30T17:02:43.737964+0000","last_peered":"2024-05-30T17:02:43.737964+0000","last_clean":"2024-05-30T17:02:43.737964+0000","last_became_active":"2024-05-30T17:02:41.011764+0000","last_became_peered":"2024-05-30T17:02:41.011764+0000","last_unstale":"2024-05-30T17:02:43.737964+0000","last_undegraded":"2024-05-30T17:02:43.737964+0000","last_fullsized":"2024-05-30T17:02:43.737964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:11:05.228132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983128+0000","last_change":"2024-05-30T17:02:39.983128+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983128+0000","last_undegraded":"2024-05-30T17:02:39.983128+0000","last_fullsized":"2024-05-30T17:02:39.983128+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.991643+0000","last_change":"2024-05-30T17:02:39.991643+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.991643+0000","last_undegraded":"2024-05-30T17:02:39.991643+0000","last_fullsized":"2024-05-30T17:02:39.991643+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.993548+0000","last_change":"2024-05-30T17:02:39.993548+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.993548+0000","last_undegraded":"2024-05-30T17:02:39.993548+0000","last_fullsized":"2024-05-30T17:02:39.993548+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738507+0000","last_change":"2024-05-30T17:02:41.011007+0000","last_active":"2024-05-30T17:02:43.738507+0000","last_peered":"2024-05-30T17:02:43.738507+0000","last_clean":"2024-05-30T17:02:43.738507+0000","last_became_active":"2024-05-30T17:02:41.010322+0000","last_became_peered":"2024-05-30T17:02:41.010322+0000","last_unstale":"2024-05-30T17:02:43.738507+0000","last_undegraded":"2024-05-30T17:02:43.738507+0000","last_fullsized":"2024-05-30T17:02:43.738507+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:40:54.062584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.987711+0000","last_change":"2024-05-30T17:02:39.987711+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.987711+0000","last_undegraded":"2024-05-30T17:02:39.987711+0000","last_fullsized":"2024-05-30T17:02:39.987711+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986411+0000","last_change":"2024-05-30T17:02:39.986411+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986411+0000","last_undegraded":"2024-05-30T17:02:39.986411+0000","last_fullsized":"2024-05-30T17:02:39.986411+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986524+0000","last_change":"2024-05-30T17:02:39.986524+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986524+0000","last_undegraded":"2024-05-30T17:02:39.986524+0000","last_fullsized":"2024-05-30T17:02:39.986524+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983355+0000","last_change":"2024-05-30T17:02:39.983355+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983355+0000","last_undegraded":"2024-05-30T17:02:39.983355+0000","last_fullsized":"2024-05-30T17:02:39.983355+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.980347+0000","last_change":"2024-05-30T17:02:39.980347+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.980347+0000","last_undegraded":"2024-05-30T17:02:39.980347+0000","last_fullsized":"2024-05-30T17:02:39.980347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.988429+0000","last_change":"2024-05-30T17:02:39.988429+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.988429+0000","last_undegraded":"2024-05-30T17:02:39.988429+0000","last_fullsized":"2024-05-30T17:02:39.988429+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.990375+0000","last_change":"2024-05-30T17:02:39.990375+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.990375+0000","last_undegraded":"2024-05-30T17:02:39.990375+0000","last_fullsized":"2024-05-30T17:02:39.990375+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979183+0000","last_change":"2024-05-30T17:02:39.979183+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979183+0000","last_undegraded":"2024-05-30T17:02:39.979183+0000","last_fullsized":"2024-05-30T17:02:39.979183+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.989647+0000","last_change":"2024-05-30T17:02:39.989647+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.989647+0000","last_undegraded":"2024-05-30T17:02:39.989647+0000","last_fullsized":"2024-05-30T17:02:39.989647+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012127+0000","last_change":"2024-05-30T17:02:41.011572+0000","last_active":"2024-05-30T17:02:42.012127+0000","last_peered":"2024-05-30T17:02:42.012127+0000","last_clean":"2024-05-30T17:02:42.012127+0000","last_became_active":"2024-05-30T17:02:41.011197+0000","last_became_peered":"2024-05-30T17:02:41.011197+0000","last_unstale":"2024-05-30T17:02:42.012127+0000","last_undegraded":"2024-05-30T17:02:42.012127+0000","last_fullsized":"2024-05-30T17:02:42.012127+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:54.031517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.981377+0000","last_change":"2024-05-30T17:02:39.981377+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.981377+0000","last_undegraded":"2024-05-30T17:02:39.981377+0000","last_fullsized":"2024-05-30T17:02:39.981377+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983059+0000","last_change":"2024-05-30T17:02:39.983059+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983059+0000","last_undegraded":"2024-05-30T17:02:39.983059+0000","last_fullsized":"2024-05-30T17:02:39.983059+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"115'4","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012308+0000","last_change":"2024-05-30T17:02:39.992222+0000","last_active":"2024-05-30T17:02:42.012308+0000","last_peered":"2024-05-30T17:02:42.012308+0000","last_clean":"2024-05-30T17:02:42.012308+0000","last_became_active":"2024-05-30T17:02:39.991672+0000","last_became_peered":"2024-05-30T17:02:39.991672+0000","last_unstale":"2024-05-30T17:02:42.012308+0000","last_undegraded":"2024-05-30T17:02:42.012308+0000","last_fullsized":"2024-05-30T17:02:42.012308+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:17:06.803976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.984329+0000","last_change":"2024-05-30T17:02:39.984329+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.984329+0000","last_undegraded":"2024-05-30T17:02:39.984329+0000","last_fullsized":"2024-05-30T17:02:39.984329+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738709+0000","last_change":"2024-05-30T17:02:40.001257+0000","last_active":"2024-05-30T17:02:43.738709+0000","last_peered":"2024-05-30T17:02:43.738709+0000","last_clean":"2024-05-30T17:02:43.738709+0000","last_became_active":"2024-05-30T17:02:40.000956+0000","last_became_peered":"2024-05-30T17:02:40.000956+0000","last_unstale":"2024-05-30T17:02:43.738709+0000","last_undegraded":"2024-05-30T17:02:43.738709+0000","last_fullsized":"2024-05-30T17:02:43.738709+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:52.599362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.984765+0000","last_change":"2024-05-30T17:02:39.984765+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.984765+0000","last_undegraded":"2024-05-30T17:02:39.984765+0000","last_fullsized":"2024-05-30T17:02:39.984765+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.026697+0000","last_change":"2024-05-30T17:02:40.004186+0000","last_active":"2024-05-30T17:02:41.026697+0000","last_peered":"2024-05-30T17:02:41.026697+0000","last_clean":"2024-05-30T17:02:41.026697+0000","last_became_active":"2024-05-30T17:02:40.003831+0000","last_became_peered":"2024-05-30T17:02:40.003831+0000","last_unstale":"2024-05-30T17:02:41.026697+0000","last_undegraded":"2024-05-30T17:02:41.026697+0000","last_fullsized":"2024-05-30T17:02:41.026697+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:09.841780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738822+0000","last_change":"2024-05-30T17:02:41.011197+0000","last_active":"2024-05-30T17:02:43.738822+0000","last_peered":"2024-05-30T17:02:43.738822+0000","last_clean":"2024-05-30T17:02:43.738822+0000","last_became_active":"2024-05-30T17:02:41.010742+0000","last_became_peered":"2024-05-30T17:02:41.010742+0000","last_unstale":"2024-05-30T17:02:43.738822+0000","last_undegraded":"2024-05-30T17:02:43.738822+0000","last_fullsized":"2024-05-30T17:02:43.738822+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:46:14.746408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.029154+0000","last_change":"2024-05-30T17:02:39.999367+0000","last_active":"2024-05-30T17:02:41.029154+0000","last_peered":"2024-05-30T17:02:41.029154+0000","last_clean":"2024-05-30T17:02:41.029154+0000","last_became_active":"2024-05-30T17:02:39.998521+0000","last_became_peered":"2024-05-30T17:02:39.998521+0000","last_unstale":"2024-05-30T17:02:41.029154+0000","last_undegraded":"2024-05-30T17:02:41.029154+0000","last_fullsized":"2024-05-30T17:02:41.029154+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:34.293069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012030+0000","last_change":"2024-05-30T17:02:39.994746+0000","last_active":"2024-05-30T17:02:42.012030+0000","last_peered":"2024-05-30T17:02:42.012030+0000","last_clean":"2024-05-30T17:02:42.012030+0000","last_became_active":"2024-05-30T17:02:39.994412+0000","last_became_peered":"2024-05-30T17:02:39.994412+0000","last_unstale":"2024-05-30T17:02:42.012030+0000","last_undegraded":"2024-05-30T17:02:42.012030+0000","last_fullsized":"2024-05-30T17:02:42.012030+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:01:23.895654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.988793+0000","last_change":"2024-05-30T17:02:39.988793+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.988793+0000","last_undegraded":"2024-05-30T17:02:39.988793+0000","last_fullsized":"2024-05-30T17:02:39.988793+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.004920+0000","last_change":"2024-05-30T17:02:40.002354+0000","last_active":"2024-05-30T17:02:40.004920+0000","last_peered":"2024-05-30T17:02:40.004920+0000","last_clean":"2024-05-30T17:02:40.004920+0000","last_became_active":"2024-05-30T17:02:40.002062+0000","last_became_peered":"2024-05-30T17:02:40.002062+0000","last_unstale":"2024-05-30T17:02:40.004920+0000","last_undegraded":"2024-05-30T17:02:40.004920+0000","last_fullsized":"2024-05-30T17:02:40.004920+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:45:00.479727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.000314+0000","last_change":"2024-05-30T17:02:39.996962+0000","last_active":"2024-05-30T17:02:40.000314+0000","last_peered":"2024-05-30T17:02:40.000314+0000","last_clean":"2024-05-30T17:02:40.000314+0000","last_became_active":"2024-05-30T17:02:39.996677+0000","last_became_peered":"2024-05-30T17:02:39.996677+0000","last_unstale":"2024-05-30T17:02:40.000314+0000","last_undegraded":"2024-05-30T17:02:40.000314+0000","last_fullsized":"2024-05-30T17:02:40.000314+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:32.794292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012752+0000","last_change":"2024-05-30T17:02:41.010582+0000","last_active":"2024-05-30T17:02:42.012752+0000","last_peered":"2024-05-30T17:02:42.012752+0000","last_clean":"2024-05-30T17:02:42.012752+0000","last_became_active":"2024-05-30T17:02:41.006958+0000","last_became_peered":"2024-05-30T17:02:41.006958+0000","last_unstale":"2024-05-30T17:02:42.012752+0000","last_undegraded":"2024-05-30T17:02:42.012752+0000","last_fullsized":"2024-05-30T17:02:42.012752+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:42:06.872806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.962728+0000","last_change":"2024-05-30T17:02:38.962728+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.962728+0000","last_undegraded":"2024-05-30T17:02:38.962728+0000","last_fullsized":"2024-05-30T17:02:38.962728+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012844+0000","last_change":"2024-05-30T17:02:40.999926+0000","last_active":"2024-05-30T17:02:42.012844+0000","last_peered":"2024-05-30T17:02:42.012844+0000","last_clean":"2024-05-30T17:02:42.012844+0000","last_became_active":"2024-05-30T17:02:40.997094+0000","last_became_peered":"2024-05-30T17:02:40.997094+0000","last_unstale":"2024-05-30T17:02:42.012844+0000","last_undegraded":"2024-05-30T17:02:42.012844+0000","last_fullsized":"2024-05-30T17:02:42.012844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:15:52.585750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.991460+0000","last_change":"2024-05-30T17:02:40.000501+0000","last_active":"2024-05-30T17:02:40.991460+0000","last_peered":"2024-05-30T17:02:40.991460+0000","last_clean":"2024-05-30T17:02:40.991460+0000","last_became_active":"2024-05-30T17:02:39.999369+0000","last_became_peered":"2024-05-30T17:02:39.999369+0000","last_unstale":"2024-05-30T17:02:40.991460+0000","last_undegraded":"2024-05-30T17:02:40.991460+0000","last_fullsized":"2024-05-30T17:02:40.991460+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:02.236301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983068+0000","last_change":"2024-05-30T17:02:39.983068+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983068+0000","last_undegraded":"2024-05-30T17:02:39.983068+0000","last_fullsized":"2024-05-30T17:02:39.983068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.005002+0000","last_change":"2024-05-30T17:02:40.002597+0000","last_active":"2024-05-30T17:02:40.005002+0000","last_peered":"2024-05-30T17:02:40.005002+0000","last_clean":"2024-05-30T17:02:40.005002+0000","last_became_active":"2024-05-30T17:02:40.002351+0000","last_became_peered":"2024-05-30T17:02:40.002351+0000","last_unstale":"2024-05-30T17:02:40.005002+0000","last_undegraded":"2024-05-30T17:02:40.005002+0000","last_fullsized":"2024-05-30T17:02:40.005002+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:34:39.559523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.978444+0000","last_change":"2024-05-30T17:02:39.978444+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.978444+0000","last_undegraded":"2024-05-30T17:02:39.978444+0000","last_fullsized":"2024-05-30T17:02:39.978444+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.962097+0000","last_change":"2024-05-30T17:02:38.962097+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.962097+0000","last_undegraded":"2024-05-30T17:02:38.962097+0000","last_fullsized":"2024-05-30T17:02:38.962097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012385+0000","last_change":"2024-05-30T17:02:41.003790+0000","last_active":"2024-05-30T17:02:42.012385+0000","last_peered":"2024-05-30T17:02:42.012385+0000","last_clean":"2024-05-30T17:02:42.012385+0000","last_became_active":"2024-05-30T17:02:41.003476+0000","last_became_peered":"2024-05-30T17:02:41.003476+0000","last_unstale":"2024-05-30T17:02:42.012385+0000","last_undegraded":"2024-05-30T17:02:42.012385+0000","last_fullsized":"2024-05-30T17:02:42.012385+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:19.156205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961654+0000","last_change":"2024-05-30T17:02:38.961654+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961654+0000","last_undegraded":"2024-05-30T17:02:38.961654+0000","last_fullsized":"2024-05-30T17:02:38.961654+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979914+0000","last_change":"2024-05-30T17:02:39.979914+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979914+0000","last_undegraded":"2024-05-30T17:02:39.979914+0000","last_fullsized":"2024-05-30T17:02:39.979914+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:39.999619+0000","last_change":"2024-05-30T17:02:39.993748+0000","last_active":"2024-05-30T17:02:39.999619+0000","last_peered":"2024-05-30T17:02:39.999619+0000","last_clean":"2024-05-30T17:02:39.999619+0000","last_became_active":"2024-05-30T17:02:39.993315+0000","last_became_peered":"2024-05-30T17:02:39.993315+0000","last_unstale":"2024-05-30T17:02:39.999619+0000","last_undegraded":"2024-05-30T17:02:39.999619+0000","last_fullsized":"2024-05-30T17:02:39.999619+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:05:49.456837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.978924+0000","last_change":"2024-05-30T17:02:39.978924+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.978924+0000","last_undegraded":"2024-05-30T17:02:39.978924+0000","last_fullsized":"2024-05-30T17:02:39.978924+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.007827+0000","last_change":"2024-05-30T17:02:40.006813+0000","last_active":"2024-05-30T17:02:40.007827+0000","last_peered":"2024-05-30T17:02:40.007827+0000","last_clean":"2024-05-30T17:02:40.007827+0000","last_became_active":"2024-05-30T17:02:40.006466+0000","last_became_peered":"2024-05-30T17:02:40.006466+0000","last_unstale":"2024-05-30T17:02:40.007827+0000","last_undegraded":"2024-05-30T17:02:40.007827+0000","last_fullsized":"2024-05-30T17:02:40.007827+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:22:42.496097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.981115+0000","last_change":"2024-05-30T17:02:39.981115+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.981115+0000","last_undegraded":"2024-05-30T17:02:39.981115+0000","last_fullsized":"2024-05-30T17:02:39.981115+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"115'8","reported_seq":20,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.032368+0000","last_change":"2024-05-30T17:02:39.998093+0000","last_active":"2024-05-30T17:02:41.032368+0000","last_peered":"2024-05-30T17:02:41.032368+0000","last_clean":"2024-05-30T17:02:41.032368+0000","last_became_active":"2024-05-30T17:02:39.997282+0000","last_became_peered":"2024-05-30T17:02:39.997282+0000","last_unstale":"2024-05-30T17:02:41.032368+0000","last_undegraded":"2024-05-30T17:02:41.032368+0000","last_fullsized":"2024-05-30T17:02:41.032368+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:31.480091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.013524+0000","last_change":"2024-05-30T17:02:41.009258+0000","last_active":"2024-05-30T17:02:41.013524+0000","last_peered":"2024-05-30T17:02:41.013524+0000","last_clean":"2024-05-30T17:02:41.013524+0000","last_became_active":"2024-05-30T17:02:41.008966+0000","last_became_peered":"2024-05-30T17:02:41.008966+0000","last_unstale":"2024-05-30T17:02:41.013524+0000","last_undegraded":"2024-05-30T17:02:41.013524+0000","last_fullsized":"2024-05-30T17:02:41.013524+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:14.951477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961297+0000","last_change":"2024-05-30T17:02:38.961297+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961297+0000","last_undegraded":"2024-05-30T17:02:38.961297+0000","last_fullsized":"2024-05-30T17:02:38.961297+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.990550+0000","last_change":"2024-05-30T17:02:39.990550+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.990550+0000","last_undegraded":"2024-05-30T17:02:39.990550+0000","last_fullsized":"2024-05-30T17:02:39.990550+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.028767+0000","last_change":"2024-05-30T17:02:39.995669+0000","last_active":"2024-05-30T17:02:41.028767+0000","last_peered":"2024-05-30T17:02:41.028767+0000","last_clean":"2024-05-30T17:02:41.028767+0000","last_became_active":"2024-05-30T17:02:39.995397+0000","last_became_peered":"2024-05-30T17:02:39.995397+0000","last_unstale":"2024-05-30T17:02:41.028767+0000","last_undegraded":"2024-05-30T17:02:41.028767+0000","last_fullsized":"2024-05-30T17:02:41.028767+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:29.373763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738221+0000","last_change":"2024-05-30T17:02:41.022727+0000","last_active":"2024-05-30T17:02:43.738221+0000","last_peered":"2024-05-30T17:02:43.738221+0000","last_clean":"2024-05-30T17:02:43.738221+0000","last_became_active":"2024-05-30T17:02:41.022446+0000","last_became_peered":"2024-05-30T17:02:41.022446+0000","last_unstale":"2024-05-30T17:02:43.738221+0000","last_undegraded":"2024-05-30T17:02:43.738221+0000","last_fullsized":"2024-05-30T17:02:43.738221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:08.319197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001682+0000","last_change":"2024-05-30T17:02:39.998860+0000","last_active":"2024-05-30T17:02:40.001682+0000","last_peered":"2024-05-30T17:02:40.001682+0000","last_clean":"2024-05-30T17:02:40.001682+0000","last_became_active":"2024-05-30T17:02:39.997624+0000","last_became_peered":"2024-05-30T17:02:39.997624+0000","last_unstale":"2024-05-30T17:02:40.001682+0000","last_undegraded":"2024-05-30T17:02:40.001682+0000","last_fullsized":"2024-05-30T17:02:40.001682+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:48.019750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738864+0000","last_change":"2024-05-30T17:02:41.017247+0000","last_active":"2024-05-30T17:02:43.738864+0000","last_peered":"2024-05-30T17:02:43.738864+0000","last_clean":"2024-05-30T17:02:43.738864+0000","last_became_active":"2024-05-30T17:02:41.016943+0000","last_became_peered":"2024-05-30T17:02:41.016943+0000","last_unstale":"2024-05-30T17:02:43.738864+0000","last_undegraded":"2024-05-30T17:02:43.738864+0000","last_fullsized":"2024-05-30T17:02:43.738864+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:14.515707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011769+0000","last_change":"2024-05-30T17:02:40.001321+0000","last_active":"2024-05-30T17:02:42.011769+0000","last_peered":"2024-05-30T17:02:42.011769+0000","last_clean":"2024-05-30T17:02:42.011769+0000","last_became_active":"2024-05-30T17:02:40.000949+0000","last_became_peered":"2024-05-30T17:02:40.000949+0000","last_unstale":"2024-05-30T17:02:42.011769+0000","last_undegraded":"2024-05-30T17:02:42.011769+0000","last_fullsized":"2024-05-30T17:02:42.011769+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:11:12.879603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737942+0000","last_change":"2024-05-30T17:02:41.012668+0000","last_active":"2024-05-30T17:02:43.737942+0000","last_peered":"2024-05-30T17:02:43.737942+0000","last_clean":"2024-05-30T17:02:43.737942+0000","last_became_active":"2024-05-30T17:02:41.012148+0000","last_became_peered":"2024-05-30T17:02:41.012148+0000","last_unstale":"2024-05-30T17:02:43.737942+0000","last_undegraded":"2024-05-30T17:02:43.737942+0000","last_fullsized":"2024-05-30T17:02:43.737942+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:51:48.462366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:39.999598+0000","last_change":"2024-05-30T17:02:39.992411+0000","last_active":"2024-05-30T17:02:39.999598+0000","last_peered":"2024-05-30T17:02:39.999598+0000","last_clean":"2024-05-30T17:02:39.999598+0000","last_became_active":"2024-05-30T17:02:39.989335+0000","last_became_peered":"2024-05-30T17:02:39.989335+0000","last_unstale":"2024-05-30T17:02:39.999598+0000","last_undegraded":"2024-05-30T17:02:39.999598+0000","last_fullsized":"2024-05-30T17:02:39.999598+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:23:05.285368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.986537+0000","last_change":"2024-05-30T17:02:39.993424+0000","last_active":"2024-05-30T17:02:40.986537+0000","last_peered":"2024-05-30T17:02:40.986537+0000","last_clean":"2024-05-30T17:02:40.986537+0000","last_became_active":"2024-05-30T17:02:39.993119+0000","last_became_peered":"2024-05-30T17:02:39.993119+0000","last_unstale":"2024-05-30T17:02:40.986537+0000","last_undegraded":"2024-05-30T17:02:40.986537+0000","last_fullsized":"2024-05-30T17:02:40.986537+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:25:20.784382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737939+0000","last_change":"2024-05-30T17:02:41.008813+0000","last_active":"2024-05-30T17:02:43.737939+0000","last_peered":"2024-05-30T17:02:43.737939+0000","last_clean":"2024-05-30T17:02:43.737939+0000","last_became_active":"2024-05-30T17:02:41.008199+0000","last_became_peered":"2024-05-30T17:02:41.008199+0000","last_unstale":"2024-05-30T17:02:43.737939+0000","last_undegraded":"2024-05-30T17:02:43.737939+0000","last_fullsized":"2024-05-30T17:02:43.737939+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:34:19.448690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:39.999648+0000","last_change":"2024-05-30T17:02:39.996513+0000","last_active":"2024-05-30T17:02:39.999648+0000","last_peered":"2024-05-30T17:02:39.999648+0000","last_clean":"2024-05-30T17:02:39.999648+0000","last_became_active":"2024-05-30T17:02:39.995747+0000","last_became_peered":"2024-05-30T17:02:39.995747+0000","last_unstale":"2024-05-30T17:02:39.999648+0000","last_undegraded":"2024-05-30T17:02:39.999648+0000","last_fullsized":"2024-05-30T17:02:39.999648+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:37.471312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979476+0000","last_change":"2024-05-30T17:02:39.979476+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979476+0000","last_undegraded":"2024-05-30T17:02:39.979476+0000","last_fullsized":"2024-05-30T17:02:39.979476+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961768+0000","last_change":"2024-05-30T17:02:38.961768+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961768+0000","last_undegraded":"2024-05-30T17:02:38.961768+0000","last_fullsized":"2024-05-30T17:02:38.961768+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.993019+0000","last_change":"2024-05-30T17:02:39.993019+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.993019+0000","last_undegraded":"2024-05-30T17:02:39.993019+0000","last_fullsized":"2024-05-30T17:02:39.993019+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001739+0000","last_change":"2024-05-30T17:02:39.997156+0000","last_active":"2024-05-30T17:02:40.001739+0000","last_peered":"2024-05-30T17:02:40.001739+0000","last_clean":"2024-05-30T17:02:40.001739+0000","last_became_active":"2024-05-30T17:02:39.996869+0000","last_became_peered":"2024-05-30T17:02:39.996869+0000","last_unstale":"2024-05-30T17:02:40.001739+0000","last_undegraded":"2024-05-30T17:02:40.001739+0000","last_fullsized":"2024-05-30T17:02:40.001739+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:59:49.118926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.002277+0000","last_change":"2024-05-30T17:02:40.000552+0000","last_active":"2024-05-30T17:02:40.002277+0000","last_peered":"2024-05-30T17:02:40.002277+0000","last_clean":"2024-05-30T17:02:40.002277+0000","last_became_active":"2024-05-30T17:02:40.000240+0000","last_became_peered":"2024-05-30T17:02:40.000240+0000","last_unstale":"2024-05-30T17:02:40.002277+0000","last_undegraded":"2024-05-30T17:02:40.002277+0000","last_fullsized":"2024-05-30T17:02:40.002277+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:17:05.041359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961468+0000","last_change":"2024-05-30T17:02:38.961468+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961468+0000","last_undegraded":"2024-05-30T17:02:38.961468+0000","last_fullsized":"2024-05-30T17:02:38.961468+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986377+0000","last_change":"2024-05-30T17:02:39.986377+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986377+0000","last_undegraded":"2024-05-30T17:02:39.986377+0000","last_fullsized":"2024-05-30T17:02:39.986377+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001681+0000","last_change":"2024-05-30T17:02:39.997337+0000","last_active":"2024-05-30T17:02:40.001681+0000","last_peered":"2024-05-30T17:02:40.001681+0000","last_clean":"2024-05-30T17:02:40.001681+0000","last_became_active":"2024-05-30T17:02:39.996998+0000","last_became_peered":"2024-05-30T17:02:39.996998+0000","last_unstale":"2024-05-30T17:02:40.001681+0000","last_undegraded":"2024-05-30T17:02:40.001681+0000","last_fullsized":"2024-05-30T17:02:40.001681+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:28.496917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.984796+0000","last_change":"2024-05-30T17:02:39.984796+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.984796+0000","last_undegraded":"2024-05-30T17:02:39.984796+0000","last_fullsized":"2024-05-30T17:02:39.984796+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012323+0000","last_change":"2024-05-30T17:02:39.996922+0000","last_active":"2024-05-30T17:02:42.012323+0000","last_peered":"2024-05-30T17:02:42.012323+0000","last_clean":"2024-05-30T17:02:42.012323+0000","last_became_active":"2024-05-30T17:02:39.996561+0000","last_became_peered":"2024-05-30T17:02:39.996561+0000","last_unstale":"2024-05-30T17:02:42.012323+0000","last_undegraded":"2024-05-30T17:02:42.012323+0000","last_fullsized":"2024-05-30T17:02:42.012323+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:16:39.184990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.987678+0000","last_change":"2024-05-30T17:02:39.987678+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.987678+0000","last_undegraded":"2024-05-30T17:02:39.987678+0000","last_fullsized":"2024-05-30T17:02:39.987678+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001125+0000","last_change":"2024-05-30T17:02:39.999916+0000","last_active":"2024-05-30T17:02:40.001125+0000","last_peered":"2024-05-30T17:02:40.001125+0000","last_clean":"2024-05-30T17:02:40.001125+0000","last_became_active":"2024-05-30T17:02:39.999588+0000","last_became_peered":"2024-05-30T17:02:39.999588+0000","last_unstale":"2024-05-30T17:02:40.001125+0000","last_undegraded":"2024-05-30T17:02:40.001125+0000","last_fullsized":"2024-05-30T17:02:40.001125+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:26:08.382531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738381+0000","last_change":"2024-05-30T17:02:41.009038+0000","last_active":"2024-05-30T17:02:43.738381+0000","last_peered":"2024-05-30T17:02:43.738381+0000","last_clean":"2024-05-30T17:02:43.738381+0000","last_became_active":"2024-05-30T17:02:41.008540+0000","last_became_peered":"2024-05-30T17:02:41.008540+0000","last_unstale":"2024-05-30T17:02:43.738381+0000","last_undegraded":"2024-05-30T17:02:43.738381+0000","last_fullsized":"2024-05-30T17:02:43.738381+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:44.351484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.986268+0000","last_change":"2024-05-30T17:02:39.994753+0000","last_active":"2024-05-30T17:02:40.986268+0000","last_peered":"2024-05-30T17:02:40.986268+0000","last_clean":"2024-05-30T17:02:40.986268+0000","last_became_active":"2024-05-30T17:02:39.994218+0000","last_became_peered":"2024-05-30T17:02:39.994218+0000","last_unstale":"2024-05-30T17:02:40.986268+0000","last_undegraded":"2024-05-30T17:02:40.986268+0000","last_fullsized":"2024-05-30T17:02:40.986268+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:01.556854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961002+0000","last_change":"2024-05-30T17:02:38.961002+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961002+0000","last_undegraded":"2024-05-30T17:02:38.961002+0000","last_fullsized":"2024-05-30T17:02:38.961002+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979708+0000","last_change":"2024-05-30T17:02:39.979708+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979708+0000","last_undegraded":"2024-05-30T17:02:39.979708+0000","last_fullsized":"2024-05-30T17:02:39.979708+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012123+0000","last_change":"2024-05-30T17:02:39.990744+0000","last_active":"2024-05-30T17:02:42.012123+0000","last_peered":"2024-05-30T17:02:42.012123+0000","last_clean":"2024-05-30T17:02:42.012123+0000","last_became_active":"2024-05-30T17:02:39.990046+0000","last_became_peered":"2024-05-30T17:02:39.990046+0000","last_unstale":"2024-05-30T17:02:42.012123+0000","last_undegraded":"2024-05-30T17:02:42.012123+0000","last_fullsized":"2024-05-30T17:02:42.012123+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:00:30.204717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":352,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738594+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:02:43.738594+0000","last_peered":"2024-05-30T17:02:43.738594+0000","last_clean":"2024-05-30T17:02:43.738594+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:02:43.738594+0000","last_undegraded":"2024-05-30T17:02:43.738594+0000","last_fullsized":"2024-05-30T17:02:43.738594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":7},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131052,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240788,"statfs":{"total":96636764160,"available":96502566912,"internally_reserved":0,"allocated":1409024,"data_stored":615310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144944,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226896,"statfs":{"total":96636764160,"available":96488341504,"internally_reserved":0,"allocated":1409024,"data_stored":615310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509672,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131056,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240784,"statfs":{"total":96636764160,"available":96502562816,"internally_reserved":0,"allocated":1409024,"data_stored":615310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149116,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222724,"statfs":{"total":96636764160,"available":96484069376,"internally_reserved":0,"allocated":1421312,"data_stored":622197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144936,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226904,"statfs":{"total":96636764160,"available":96488349696,"internally_reserved":0,"allocated":1396736,"data_stored":608923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542375,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131652,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240188,"statfs":{"total":96636764160,"available":96501952512,"internally_reserved":0,"allocated":2023424,"data_stored":1213181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542376,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64468,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307372,"statfs":{"total":96636764160,"available":96570748928,"internally_reserved":0,"allocated":1441792,"data_stored":629213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542375,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87208,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284632,"statfs":{"total":96636764160,"available":96547463168,"internally_reserved":0,"allocated":1990656,"data_stored":1199291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:02:45.475 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:02:45.491 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:02:45.523 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:02:45.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.705+0000 7fea2fe00640 1 Processor -- start 2024-05-30T17:02:45.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.705+0000 7fea2fe00640 1 -- start start 2024-05-30T17:02:45.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.706+0000 7fea2fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea300ad940 0x7fea300b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.706+0000 7fea2fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea30156f10 con 0x7fea3006ee40 2024-05-30T17:02:45.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.706+0000 7fea2fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea3006dda0 con 0x7fea30156b30 2024-05-30T17:02:45.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.706+0000 7fea2fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fea3006e080 con 0x7fea300ad940 2024-05-30T17:02:45.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.706+0000 7fea2f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea300ad940 0x7fea300b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.707+0000 7fea2f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea300ad940 0x7fea300b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60712/0 (socket says 172.21.15.67:60712) 2024-05-30T17:02:45.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.707+0000 7fea2f400640 1 -- 172.21.15.67:0/831673247 learned_addr learned my addr 172.21.15.67:0/831673247 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:45.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.707+0000 7fea27e00640 1 --1- 172.21.15.67:0/831673247 >> v1:172.21.15.67:6789/0 conn(0x7fea3006ee40 0x7fea300ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58638/0 (socket says 172.21.15.67:58638) 2024-05-30T17:02:45.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.707+0000 7fea2f400640 1 -- 172.21.15.67:0/831673247 >> v1:172.21.15.169:6789/0 conn(0x7fea30156b30 legacy=0x7fea300af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:45.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.707+0000 7fea2f400640 1 -- 172.21.15.67:0/831673247 >> v1:172.21.15.67:6789/0 conn(0x7fea3006ee40 legacy=0x7fea300ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.707+0000 7fea2f400640 1 -- 172.21.15.67:0/831673247 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3006e360 con 0x7fea300ad940 2024-05-30T17:02:45.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.707+0000 7fea2e000640 1 -- 172.21.15.67:0/831673247 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7826114 0 0) 0x7fea30156f10 con 0x7fea3006ee40 2024-05-30T17:02:45.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.708+0000 7fea2f400640 1 --2- 172.21.15.67:0/831673247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 0x7fea300b32c0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fea28009e50 tx=0x7fea2802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66336bc6db056bf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:45.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.708+0000 7fea2e000640 1 -- 172.21.15.67:0/831673247 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28036070 con 0x7fea300ad940 2024-05-30T17:02:45.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.708+0000 7fea2e000640 1 -- 172.21.15.67:0/831673247 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea28031600 con 0x7fea300ad940 2024-05-30T17:02:45.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.709+0000 7fea2e000640 1 -- 172.21.15.67:0/831673247 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea280319d0 con 0x7fea300ad940 2024-05-30T17:02:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.709+0000 7fea2fe00640 1 -- 172.21.15.67:0/831673247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 msgr2=0x7fea300b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.709+0000 7fea2fe00640 1 --2- 172.21.15.67:0/831673247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 0x7fea300b32c0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fea28009e50 tx=0x7fea2802a6d0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.709+0000 7fea2fe00640 1 -- 172.21.15.67:0/831673247 shutdown_connections 2024-05-30T17:02:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.709+0000 7fea2fe00640 1 --2- 172.21.15.67:0/831673247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 0x7fea300b32c0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.709+0000 7fea2fe00640 1 -- 172.21.15.67:0/831673247 >> 172.21.15.67:0/831673247 conn(0x7fea300b1fc0 msgr2=0x7fea300b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.709+0000 7fea2fe00640 1 -- 172.21.15.67:0/831673247 shutdown_connections 2024-05-30T17:02:45.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.709+0000 7fea2fe00640 1 -- 172.21.15.67:0/831673247 wait complete. 2024-05-30T17:02:45.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.711+0000 7fea2fe00640 1 Processor -- start 2024-05-30T17:02:45.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.711+0000 7fea2fe00640 1 -- start start 2024-05-30T17:02:45.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.712+0000 7fea2fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea3006ee40 0x7fea3014ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.712+0000 7fea2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 0x7fea300aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.712+0000 7fea2fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea30156b30 0x7fea3014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.712+0000 7fea2fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fea300b2db0 con 0x7fea300ad940 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.712+0000 7fea2fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea300b2c30 con 0x7fea30156b30 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.712+0000 7fea2fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fea3006f2f0 con 0x7fea3006ee40 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.712+0000 7fea27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 0x7fea300aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 0x7fea300aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60728/0 (socket says 172.21.15.67:60728) 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea27e00640 1 -- 172.21.15.67:0/438883887 learned_addr learned my addr 172.21.15.67:0/438883887 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea2f400640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea30156b30 0x7fea3014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea27e00640 1 -- 172.21.15.67:0/438883887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea3006ee40 msgr2=0x7fea3014ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea2ea00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea3006ee40 0x7fea3014ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea27e00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea3006ee40 0x7fea3014ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea27e00640 1 -- 172.21.15.67:0/438883887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea30156b30 msgr2=0x7fea3014de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea27e00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea30156b30 0x7fea3014de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea27e00640 1 -- 172.21.15.67:0/438883887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300b0f20 con 0x7fea300ad940 2024-05-30T17:02:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.713+0000 7fea27e00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 0x7fea300aff30 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c4d0 tx=0x7fea1800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.714+0000 7fea26a00640 1 -- 172.21.15.67:0/438883887 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18018070 con 0x7fea300ad940 2024-05-30T17:02:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.714+0000 7fea26a00640 1 -- 172.21.15.67:0/438883887 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea18007500 con 0x7fea300ad940 2024-05-30T17:02:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.714+0000 7fea26a00640 1 -- 172.21.15.67:0/438883887 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18004a20 con 0x7fea300ad940 2024-05-30T17:02:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.714+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea300b0960 con 0x7fea300ad940 2024-05-30T17:02:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.714+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea30151df0 con 0x7fea300ad940 2024-05-30T17:02:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.715+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea300689e0 con 0x7fea300ad940 2024-05-30T17:02:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.716+0000 7fea26a00640 1 -- 172.21.15.67:0/438883887 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fea1801b070 con 0x7fea300ad940 2024-05-30T17:02:45.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.717+0000 7fea26a00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea0c06e090 0x7fea0c070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:45.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.717+0000 7fea26a00640 1 -- 172.21.15.67:0/438883887 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fea18089bd0 con 0x7fea300ad940 2024-05-30T17:02:45.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.718+0000 7fea2ea00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea0c06e090 0x7fea0c070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:45.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.718+0000 7fea2ea00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea0c06e090 0x7fea0c070550 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fea2000a750 tx=0x7fea20008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:45.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.722+0000 7fea26a00640 1 -- 172.21.15.67:0/438883887 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea18059b00 con 0x7fea300ad940 2024-05-30T17:02:45.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.923+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea30160de0 con 0x7fea0c06e090 2024-05-30T17:02:45.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.935+0000 7fea26a00640 1 -- 172.21.15.67:0/438883887 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247323 (secure 0 0 0) 0x7fea30160de0 con 0x7fea0c06e090 2024-05-30T17:02:45.939 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:45.940 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.940+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea0c06e090 msgr2=0x7fea0c070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.940+0000 7fea2fe00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea0c06e090 0x7fea0c070550 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fea2000a750 tx=0x7fea20008040 comp rx=0 tx=0).stop 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.940+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 msgr2=0x7fea300aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.940+0000 7fea2fe00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 0x7fea300aff30 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c4d0 tx=0x7fea1800c990 comp rx=0 tx=0).stop 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.941+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 shutdown_connections 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.941+0000 7fea2fe00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fea0c06e090 0x7fea0c070550 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.941+0000 7fea2fe00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea30156b30 0x7fea3014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.941+0000 7fea2fe00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea300ad940 0x7fea300aff30 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.941+0000 7fea2fe00640 1 --2- 172.21.15.67:0/438883887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea3006ee40 0x7fea3014ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.941+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 >> 172.21.15.67:0/438883887 conn(0x7fea300b1fc0 msgr2=0x7fea30160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:45.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.941+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 shutdown_connections 2024-05-30T17:02:45.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:45.941+0000 7fea2fe00640 1 -- 172.21.15.67:0/438883887 wait complete. 2024-05-30T17:02:45.962 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":682,"stamp":"2024-05-30T17:02:44.522439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":184,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":984432,"kb_used_data":12208,"kb_used_omap":12,"kb_used_meta":972083,"kb_avail":753990288,"statfs":{"total":773094113280,"available":772086054912,"internally_reserved":0,"allocated":12500992,"data_stored":6118735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":995413552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570340"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986625+0000","last_change":"2024-05-30T17:02:39.986625+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986625+0000","last_undegraded":"2024-05-30T17:02:39.986625+0000","last_fullsized":"2024-05-30T17:02:39.986625+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738383+0000","last_change":"2024-05-30T17:02:41.013698+0000","last_active":"2024-05-30T17:02:43.738383+0000","last_peered":"2024-05-30T17:02:43.738383+0000","last_clean":"2024-05-30T17:02:43.738383+0000","last_became_active":"2024-05-30T17:02:41.013407+0000","last_became_peered":"2024-05-30T17:02:41.013407+0000","last_unstale":"2024-05-30T17:02:43.738383+0000","last_undegraded":"2024-05-30T17:02:43.738383+0000","last_fullsized":"2024-05-30T17:02:43.738383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:56.046859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.008923+0000","last_change":"2024-05-30T17:02:41.006127+0000","last_active":"2024-05-30T17:02:41.008923+0000","last_peered":"2024-05-30T17:02:41.008923+0000","last_clean":"2024-05-30T17:02:41.008923+0000","last_became_active":"2024-05-30T17:02:41.005503+0000","last_became_peered":"2024-05-30T17:02:41.005503+0000","last_unstale":"2024-05-30T17:02:41.008923+0000","last_undegraded":"2024-05-30T17:02:41.008923+0000","last_fullsized":"2024-05-30T17:02:41.008923+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:01.078245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.982145+0000","last_change":"2024-05-30T17:02:39.982145+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.982145+0000","last_undegraded":"2024-05-30T17:02:39.982145+0000","last_fullsized":"2024-05-30T17:02:39.982145+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011985+0000","last_change":"2024-05-30T17:02:41.010886+0000","last_active":"2024-05-30T17:02:42.011985+0000","last_peered":"2024-05-30T17:02:42.011985+0000","last_clean":"2024-05-30T17:02:42.011985+0000","last_became_active":"2024-05-30T17:02:41.007377+0000","last_became_peered":"2024-05-30T17:02:41.007377+0000","last_unstale":"2024-05-30T17:02:42.011985+0000","last_undegraded":"2024-05-30T17:02:42.011985+0000","last_fullsized":"2024-05-30T17:02:42.011985+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:59.071677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738235+0000","last_change":"2024-05-30T17:02:41.014332+0000","last_active":"2024-05-30T17:02:43.738235+0000","last_peered":"2024-05-30T17:02:43.738235+0000","last_clean":"2024-05-30T17:02:43.738235+0000","last_became_active":"2024-05-30T17:02:41.014074+0000","last_became_peered":"2024-05-30T17:02:41.014074+0000","last_unstale":"2024-05-30T17:02:43.738235+0000","last_undegraded":"2024-05-30T17:02:43.738235+0000","last_fullsized":"2024-05-30T17:02:43.738235+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:00:36.629725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.987823+0000","last_change":"2024-05-30T17:02:39.987823+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.987823+0000","last_undegraded":"2024-05-30T17:02:39.987823+0000","last_fullsized":"2024-05-30T17:02:39.987823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738860+0000","last_change":"2024-05-30T17:02:41.011886+0000","last_active":"2024-05-30T17:02:43.738860+0000","last_peered":"2024-05-30T17:02:43.738860+0000","last_clean":"2024-05-30T17:02:43.738860+0000","last_became_active":"2024-05-30T17:02:41.010296+0000","last_became_peered":"2024-05-30T17:02:41.010296+0000","last_unstale":"2024-05-30T17:02:43.738860+0000","last_undegraded":"2024-05-30T17:02:43.738860+0000","last_fullsized":"2024-05-30T17:02:43.738860+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:20.421009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737964+0000","last_change":"2024-05-30T17:02:41.012428+0000","last_active":"2024-05-30T17:02:43.737964+0000","last_peered":"2024-05-30T17:02:43.737964+0000","last_clean":"2024-05-30T17:02:43.737964+0000","last_became_active":"2024-05-30T17:02:41.011764+0000","last_became_peered":"2024-05-30T17:02:41.011764+0000","last_unstale":"2024-05-30T17:02:43.737964+0000","last_undegraded":"2024-05-30T17:02:43.737964+0000","last_fullsized":"2024-05-30T17:02:43.737964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:11:05.228132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983128+0000","last_change":"2024-05-30T17:02:39.983128+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983128+0000","last_undegraded":"2024-05-30T17:02:39.983128+0000","last_fullsized":"2024-05-30T17:02:39.983128+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.991643+0000","last_change":"2024-05-30T17:02:39.991643+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.991643+0000","last_undegraded":"2024-05-30T17:02:39.991643+0000","last_fullsized":"2024-05-30T17:02:39.991643+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.993548+0000","last_change":"2024-05-30T17:02:39.993548+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.993548+0000","last_undegraded":"2024-05-30T17:02:39.993548+0000","last_fullsized":"2024-05-30T17:02:39.993548+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738507+0000","last_change":"2024-05-30T17:02:41.011007+0000","last_active":"2024-05-30T17:02:43.738507+0000","last_peered":"2024-05-30T17:02:43.738507+0000","last_clean":"2024-05-30T17:02:43.738507+0000","last_became_active":"2024-05-30T17:02:41.010322+0000","last_became_peered":"2024-05-30T17:02:41.010322+0000","last_unstale":"2024-05-30T17:02:43.738507+0000","last_undegraded":"2024-05-30T17:02:43.738507+0000","last_fullsized":"2024-05-30T17:02:43.738507+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:40:54.062584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.987711+0000","last_change":"2024-05-30T17:02:39.987711+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.987711+0000","last_undegraded":"2024-05-30T17:02:39.987711+0000","last_fullsized":"2024-05-30T17:02:39.987711+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986411+0000","last_change":"2024-05-30T17:02:39.986411+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986411+0000","last_undegraded":"2024-05-30T17:02:39.986411+0000","last_fullsized":"2024-05-30T17:02:39.986411+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986524+0000","last_change":"2024-05-30T17:02:39.986524+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986524+0000","last_undegraded":"2024-05-30T17:02:39.986524+0000","last_fullsized":"2024-05-30T17:02:39.986524+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983355+0000","last_change":"2024-05-30T17:02:39.983355+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983355+0000","last_undegraded":"2024-05-30T17:02:39.983355+0000","last_fullsized":"2024-05-30T17:02:39.983355+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.980347+0000","last_change":"2024-05-30T17:02:39.980347+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.980347+0000","last_undegraded":"2024-05-30T17:02:39.980347+0000","last_fullsized":"2024-05-30T17:02:39.980347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.988429+0000","last_change":"2024-05-30T17:02:39.988429+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.988429+0000","last_undegraded":"2024-05-30T17:02:39.988429+0000","last_fullsized":"2024-05-30T17:02:39.988429+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.990375+0000","last_change":"2024-05-30T17:02:39.990375+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.990375+0000","last_undegraded":"2024-05-30T17:02:39.990375+0000","last_fullsized":"2024-05-30T17:02:39.990375+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979183+0000","last_change":"2024-05-30T17:02:39.979183+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979183+0000","last_undegraded":"2024-05-30T17:02:39.979183+0000","last_fullsized":"2024-05-30T17:02:39.979183+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.989647+0000","last_change":"2024-05-30T17:02:39.989647+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.989647+0000","last_undegraded":"2024-05-30T17:02:39.989647+0000","last_fullsized":"2024-05-30T17:02:39.989647+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012127+0000","last_change":"2024-05-30T17:02:41.011572+0000","last_active":"2024-05-30T17:02:42.012127+0000","last_peered":"2024-05-30T17:02:42.012127+0000","last_clean":"2024-05-30T17:02:42.012127+0000","last_became_active":"2024-05-30T17:02:41.011197+0000","last_became_peered":"2024-05-30T17:02:41.011197+0000","last_unstale":"2024-05-30T17:02:42.012127+0000","last_undegraded":"2024-05-30T17:02:42.012127+0000","last_fullsized":"2024-05-30T17:02:42.012127+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:54.031517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.981377+0000","last_change":"2024-05-30T17:02:39.981377+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.981377+0000","last_undegraded":"2024-05-30T17:02:39.981377+0000","last_fullsized":"2024-05-30T17:02:39.981377+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983059+0000","last_change":"2024-05-30T17:02:39.983059+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983059+0000","last_undegraded":"2024-05-30T17:02:39.983059+0000","last_fullsized":"2024-05-30T17:02:39.983059+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"115'4","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012308+0000","last_change":"2024-05-30T17:02:39.992222+0000","last_active":"2024-05-30T17:02:42.012308+0000","last_peered":"2024-05-30T17:02:42.012308+0000","last_clean":"2024-05-30T17:02:42.012308+0000","last_became_active":"2024-05-30T17:02:39.991672+0000","last_became_peered":"2024-05-30T17:02:39.991672+0000","last_unstale":"2024-05-30T17:02:42.012308+0000","last_undegraded":"2024-05-30T17:02:42.012308+0000","last_fullsized":"2024-05-30T17:02:42.012308+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:17:06.803976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.984329+0000","last_change":"2024-05-30T17:02:39.984329+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.984329+0000","last_undegraded":"2024-05-30T17:02:39.984329+0000","last_fullsized":"2024-05-30T17:02:39.984329+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738709+0000","last_change":"2024-05-30T17:02:40.001257+0000","last_active":"2024-05-30T17:02:43.738709+0000","last_peered":"2024-05-30T17:02:43.738709+0000","last_clean":"2024-05-30T17:02:43.738709+0000","last_became_active":"2024-05-30T17:02:40.000956+0000","last_became_peered":"2024-05-30T17:02:40.000956+0000","last_unstale":"2024-05-30T17:02:43.738709+0000","last_undegraded":"2024-05-30T17:02:43.738709+0000","last_fullsized":"2024-05-30T17:02:43.738709+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:52.599362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.984765+0000","last_change":"2024-05-30T17:02:39.984765+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.984765+0000","last_undegraded":"2024-05-30T17:02:39.984765+0000","last_fullsized":"2024-05-30T17:02:39.984765+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.026697+0000","last_change":"2024-05-30T17:02:40.004186+0000","last_active":"2024-05-30T17:02:41.026697+0000","last_peered":"2024-05-30T17:02:41.026697+0000","last_clean":"2024-05-30T17:02:41.026697+0000","last_became_active":"2024-05-30T17:02:40.003831+0000","last_became_peered":"2024-05-30T17:02:40.003831+0000","last_unstale":"2024-05-30T17:02:41.026697+0000","last_undegraded":"2024-05-30T17:02:41.026697+0000","last_fullsized":"2024-05-30T17:02:41.026697+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:09.841780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738822+0000","last_change":"2024-05-30T17:02:41.011197+0000","last_active":"2024-05-30T17:02:43.738822+0000","last_peered":"2024-05-30T17:02:43.738822+0000","last_clean":"2024-05-30T17:02:43.738822+0000","last_became_active":"2024-05-30T17:02:41.010742+0000","last_became_peered":"2024-05-30T17:02:41.010742+0000","last_unstale":"2024-05-30T17:02:43.738822+0000","last_undegraded":"2024-05-30T17:02:43.738822+0000","last_fullsized":"2024-05-30T17:02:43.738822+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:46:14.746408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.029154+0000","last_change":"2024-05-30T17:02:39.999367+0000","last_active":"2024-05-30T17:02:41.029154+0000","last_peered":"2024-05-30T17:02:41.029154+0000","last_clean":"2024-05-30T17:02:41.029154+0000","last_became_active":"2024-05-30T17:02:39.998521+0000","last_became_peered":"2024-05-30T17:02:39.998521+0000","last_unstale":"2024-05-30T17:02:41.029154+0000","last_undegraded":"2024-05-30T17:02:41.029154+0000","last_fullsized":"2024-05-30T17:02:41.029154+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:34.293069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012030+0000","last_change":"2024-05-30T17:02:39.994746+0000","last_active":"2024-05-30T17:02:42.012030+0000","last_peered":"2024-05-30T17:02:42.012030+0000","last_clean":"2024-05-30T17:02:42.012030+0000","last_became_active":"2024-05-30T17:02:39.994412+0000","last_became_peered":"2024-05-30T17:02:39.994412+0000","last_unstale":"2024-05-30T17:02:42.012030+0000","last_undegraded":"2024-05-30T17:02:42.012030+0000","last_fullsized":"2024-05-30T17:02:42.012030+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:01:23.895654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.988793+0000","last_change":"2024-05-30T17:02:39.988793+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.988793+0000","last_undegraded":"2024-05-30T17:02:39.988793+0000","last_fullsized":"2024-05-30T17:02:39.988793+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.004920+0000","last_change":"2024-05-30T17:02:40.002354+0000","last_active":"2024-05-30T17:02:40.004920+0000","last_peered":"2024-05-30T17:02:40.004920+0000","last_clean":"2024-05-30T17:02:40.004920+0000","last_became_active":"2024-05-30T17:02:40.002062+0000","last_became_peered":"2024-05-30T17:02:40.002062+0000","last_unstale":"2024-05-30T17:02:40.004920+0000","last_undegraded":"2024-05-30T17:02:40.004920+0000","last_fullsized":"2024-05-30T17:02:40.004920+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:45:00.479727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.000314+0000","last_change":"2024-05-30T17:02:39.996962+0000","last_active":"2024-05-30T17:02:40.000314+0000","last_peered":"2024-05-30T17:02:40.000314+0000","last_clean":"2024-05-30T17:02:40.000314+0000","last_became_active":"2024-05-30T17:02:39.996677+0000","last_became_peered":"2024-05-30T17:02:39.996677+0000","last_unstale":"2024-05-30T17:02:40.000314+0000","last_undegraded":"2024-05-30T17:02:40.000314+0000","last_fullsized":"2024-05-30T17:02:40.000314+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:32.794292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012752+0000","last_change":"2024-05-30T17:02:41.010582+0000","last_active":"2024-05-30T17:02:42.012752+0000","last_peered":"2024-05-30T17:02:42.012752+0000","last_clean":"2024-05-30T17:02:42.012752+0000","last_became_active":"2024-05-30T17:02:41.006958+0000","last_became_peered":"2024-05-30T17:02:41.006958+0000","last_unstale":"2024-05-30T17:02:42.012752+0000","last_undegraded":"2024-05-30T17:02:42.012752+0000","last_fullsized":"2024-05-30T17:02:42.012752+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:42:06.872806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.962728+0000","last_change":"2024-05-30T17:02:38.962728+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.962728+0000","last_undegraded":"2024-05-30T17:02:38.962728+0000","last_fullsized":"2024-05-30T17:02:38.962728+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012844+0000","last_change":"2024-05-30T17:02:40.999926+0000","last_active":"2024-05-30T17:02:42.012844+0000","last_peered":"2024-05-30T17:02:42.012844+0000","last_clean":"2024-05-30T17:02:42.012844+0000","last_became_active":"2024-05-30T17:02:40.997094+0000","last_became_peered":"2024-05-30T17:02:40.997094+0000","last_unstale":"2024-05-30T17:02:42.012844+0000","last_undegraded":"2024-05-30T17:02:42.012844+0000","last_fullsized":"2024-05-30T17:02:42.012844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:15:52.585750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.991460+0000","last_change":"2024-05-30T17:02:40.000501+0000","last_active":"2024-05-30T17:02:40.991460+0000","last_peered":"2024-05-30T17:02:40.991460+0000","last_clean":"2024-05-30T17:02:40.991460+0000","last_became_active":"2024-05-30T17:02:39.999369+0000","last_became_peered":"2024-05-30T17:02:39.999369+0000","last_unstale":"2024-05-30T17:02:40.991460+0000","last_undegraded":"2024-05-30T17:02:40.991460+0000","last_fullsized":"2024-05-30T17:02:40.991460+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:02.236301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983068+0000","last_change":"2024-05-30T17:02:39.983068+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983068+0000","last_undegraded":"2024-05-30T17:02:39.983068+0000","last_fullsized":"2024-05-30T17:02:39.983068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.005002+0000","last_change":"2024-05-30T17:02:40.002597+0000","last_active":"2024-05-30T17:02:40.005002+0000","last_peered":"2024-05-30T17:02:40.005002+0000","last_clean":"2024-05-30T17:02:40.005002+0000","last_became_active":"2024-05-30T17:02:40.002351+0000","last_became_peered":"2024-05-30T17:02:40.002351+0000","last_unstale":"2024-05-30T17:02:40.005002+0000","last_undegraded":"2024-05-30T17:02:40.005002+0000","last_fullsized":"2024-05-30T17:02:40.005002+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:34:39.559523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.978444+0000","last_change":"2024-05-30T17:02:39.978444+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.978444+0000","last_undegraded":"2024-05-30T17:02:39.978444+0000","last_fullsized":"2024-05-30T17:02:39.978444+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.962097+0000","last_change":"2024-05-30T17:02:38.962097+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.962097+0000","last_undegraded":"2024-05-30T17:02:38.962097+0000","last_fullsized":"2024-05-30T17:02:38.962097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012385+0000","last_change":"2024-05-30T17:02:41.003790+0000","last_active":"2024-05-30T17:02:42.012385+0000","last_peered":"2024-05-30T17:02:42.012385+0000","last_clean":"2024-05-30T17:02:42.012385+0000","last_became_active":"2024-05-30T17:02:41.003476+0000","last_became_peered":"2024-05-30T17:02:41.003476+0000","last_unstale":"2024-05-30T17:02:42.012385+0000","last_undegraded":"2024-05-30T17:02:42.012385+0000","last_fullsized":"2024-05-30T17:02:42.012385+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:19.156205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961654+0000","last_change":"2024-05-30T17:02:38.961654+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961654+0000","last_undegraded":"2024-05-30T17:02:38.961654+0000","last_fullsized":"2024-05-30T17:02:38.961654+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979914+0000","last_change":"2024-05-30T17:02:39.979914+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979914+0000","last_undegraded":"2024-05-30T17:02:39.979914+0000","last_fullsized":"2024-05-30T17:02:39.979914+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:39.999619+0000","last_change":"2024-05-30T17:02:39.993748+0000","last_active":"2024-05-30T17:02:39.999619+0000","last_peered":"2024-05-30T17:02:39.999619+0000","last_clean":"2024-05-30T17:02:39.999619+0000","last_became_active":"2024-05-30T17:02:39.993315+0000","last_became_peered":"2024-05-30T17:02:39.993315+0000","last_unstale":"2024-05-30T17:02:39.999619+0000","last_undegraded":"2024-05-30T17:02:39.999619+0000","last_fullsized":"2024-05-30T17:02:39.999619+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:05:49.456837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.978924+0000","last_change":"2024-05-30T17:02:39.978924+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.978924+0000","last_undegraded":"2024-05-30T17:02:39.978924+0000","last_fullsized":"2024-05-30T17:02:39.978924+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.007827+0000","last_change":"2024-05-30T17:02:40.006813+0000","last_active":"2024-05-30T17:02:40.007827+0000","last_peered":"2024-05-30T17:02:40.007827+0000","last_clean":"2024-05-30T17:02:40.007827+0000","last_became_active":"2024-05-30T17:02:40.006466+0000","last_became_peered":"2024-05-30T17:02:40.006466+0000","last_unstale":"2024-05-30T17:02:40.007827+0000","last_undegraded":"2024-05-30T17:02:40.007827+0000","last_fullsized":"2024-05-30T17:02:40.007827+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:22:42.496097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.981115+0000","last_change":"2024-05-30T17:02:39.981115+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.981115+0000","last_undegraded":"2024-05-30T17:02:39.981115+0000","last_fullsized":"2024-05-30T17:02:39.981115+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"115'8","reported_seq":20,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.032368+0000","last_change":"2024-05-30T17:02:39.998093+0000","last_active":"2024-05-30T17:02:41.032368+0000","last_peered":"2024-05-30T17:02:41.032368+0000","last_clean":"2024-05-30T17:02:41.032368+0000","last_became_active":"2024-05-30T17:02:39.997282+0000","last_became_peered":"2024-05-30T17:02:39.997282+0000","last_unstale":"2024-05-30T17:02:41.032368+0000","last_undegraded":"2024-05-30T17:02:41.032368+0000","last_fullsized":"2024-05-30T17:02:41.032368+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:31.480091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.013524+0000","last_change":"2024-05-30T17:02:41.009258+0000","last_active":"2024-05-30T17:02:41.013524+0000","last_peered":"2024-05-30T17:02:41.013524+0000","last_clean":"2024-05-30T17:02:41.013524+0000","last_became_active":"2024-05-30T17:02:41.008966+0000","last_became_peered":"2024-05-30T17:02:41.008966+0000","last_unstale":"2024-05-30T17:02:41.013524+0000","last_undegraded":"2024-05-30T17:02:41.013524+0000","last_fullsized":"2024-05-30T17:02:41.013524+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:14.951477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961297+0000","last_change":"2024-05-30T17:02:38.961297+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961297+0000","last_undegraded":"2024-05-30T17:02:38.961297+0000","last_fullsized":"2024-05-30T17:02:38.961297+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.990550+0000","last_change":"2024-05-30T17:02:39.990550+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.990550+0000","last_undegraded":"2024-05-30T17:02:39.990550+0000","last_fullsized":"2024-05-30T17:02:39.990550+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.028767+0000","last_change":"2024-05-30T17:02:39.995669+0000","last_active":"2024-05-30T17:02:41.028767+0000","last_peered":"2024-05-30T17:02:41.028767+0000","last_clean":"2024-05-30T17:02:41.028767+0000","last_became_active":"2024-05-30T17:02:39.995397+0000","last_became_peered":"2024-05-30T17:02:39.995397+0000","last_unstale":"2024-05-30T17:02:41.028767+0000","last_undegraded":"2024-05-30T17:02:41.028767+0000","last_fullsized":"2024-05-30T17:02:41.028767+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:29.373763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738221+0000","last_change":"2024-05-30T17:02:41.022727+0000","last_active":"2024-05-30T17:02:43.738221+0000","last_peered":"2024-05-30T17:02:43.738221+0000","last_clean":"2024-05-30T17:02:43.738221+0000","last_became_active":"2024-05-30T17:02:41.022446+0000","last_became_peered":"2024-05-30T17:02:41.022446+0000","last_unstale":"2024-05-30T17:02:43.738221+0000","last_undegraded":"2024-05-30T17:02:43.738221+0000","last_fullsized":"2024-05-30T17:02:43.738221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:08.319197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001682+0000","last_change":"2024-05-30T17:02:39.998860+0000","last_active":"2024-05-30T17:02:40.001682+0000","last_peered":"2024-05-30T17:02:40.001682+0000","last_clean":"2024-05-30T17:02:40.001682+0000","last_became_active":"2024-05-30T17:02:39.997624+0000","last_became_peered":"2024-05-30T17:02:39.997624+0000","last_unstale":"2024-05-30T17:02:40.001682+0000","last_undegraded":"2024-05-30T17:02:40.001682+0000","last_fullsized":"2024-05-30T17:02:40.001682+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:48.019750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738864+0000","last_change":"2024-05-30T17:02:41.017247+0000","last_active":"2024-05-30T17:02:43.738864+0000","last_peered":"2024-05-30T17:02:43.738864+0000","last_clean":"2024-05-30T17:02:43.738864+0000","last_became_active":"2024-05-30T17:02:41.016943+0000","last_became_peered":"2024-05-30T17:02:41.016943+0000","last_unstale":"2024-05-30T17:02:43.738864+0000","last_undegraded":"2024-05-30T17:02:43.738864+0000","last_fullsized":"2024-05-30T17:02:43.738864+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:14.515707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011769+0000","last_change":"2024-05-30T17:02:40.001321+0000","last_active":"2024-05-30T17:02:42.011769+0000","last_peered":"2024-05-30T17:02:42.011769+0000","last_clean":"2024-05-30T17:02:42.011769+0000","last_became_active":"2024-05-30T17:02:40.000949+0000","last_became_peered":"2024-05-30T17:02:40.000949+0000","last_unstale":"2024-05-30T17:02:42.011769+0000","last_undegraded":"2024-05-30T17:02:42.011769+0000","last_fullsized":"2024-05-30T17:02:42.011769+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:11:12.879603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737942+0000","last_change":"2024-05-30T17:02:41.012668+0000","last_active":"2024-05-30T17:02:43.737942+0000","last_peered":"2024-05-30T17:02:43.737942+0000","last_clean":"2024-05-30T17:02:43.737942+0000","last_became_active":"2024-05-30T17:02:41.012148+0000","last_became_peered":"2024-05-30T17:02:41.012148+0000","last_unstale":"2024-05-30T17:02:43.737942+0000","last_undegraded":"2024-05-30T17:02:43.737942+0000","last_fullsized":"2024-05-30T17:02:43.737942+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:51:48.462366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:39.999598+0000","last_change":"2024-05-30T17:02:39.992411+0000","last_active":"2024-05-30T17:02:39.999598+0000","last_peered":"2024-05-30T17:02:39.999598+0000","last_clean":"2024-05-30T17:02:39.999598+0000","last_became_active":"2024-05-30T17:02:39.989335+0000","last_became_peered":"2024-05-30T17:02:39.989335+0000","last_unstale":"2024-05-30T17:02:39.999598+0000","last_undegraded":"2024-05-30T17:02:39.999598+0000","last_fullsized":"2024-05-30T17:02:39.999598+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:23:05.285368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.986537+0000","last_change":"2024-05-30T17:02:39.993424+0000","last_active":"2024-05-30T17:02:40.986537+0000","last_peered":"2024-05-30T17:02:40.986537+0000","last_clean":"2024-05-30T17:02:40.986537+0000","last_became_active":"2024-05-30T17:02:39.993119+0000","last_became_peered":"2024-05-30T17:02:39.993119+0000","last_unstale":"2024-05-30T17:02:40.986537+0000","last_undegraded":"2024-05-30T17:02:40.986537+0000","last_fullsized":"2024-05-30T17:02:40.986537+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:25:20.784382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737939+0000","last_change":"2024-05-30T17:02:41.008813+0000","last_active":"2024-05-30T17:02:43.737939+0000","last_peered":"2024-05-30T17:02:43.737939+0000","last_clean":"2024-05-30T17:02:43.737939+0000","last_became_active":"2024-05-30T17:02:41.008199+0000","last_became_peered":"2024-05-30T17:02:41.008199+0000","last_unstale":"2024-05-30T17:02:43.737939+0000","last_undegraded":"2024-05-30T17:02:43.737939+0000","last_fullsized":"2024-05-30T17:02:43.737939+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:34:19.448690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:39.999648+0000","last_change":"2024-05-30T17:02:39.996513+0000","last_active":"2024-05-30T17:02:39.999648+0000","last_peered":"2024-05-30T17:02:39.999648+0000","last_clean":"2024-05-30T17:02:39.999648+0000","last_became_active":"2024-05-30T17:02:39.995747+0000","last_became_peered":"2024-05-30T17:02:39.995747+0000","last_unstale":"2024-05-30T17:02:39.999648+0000","last_undegraded":"2024-05-30T17:02:39.999648+0000","last_fullsized":"2024-05-30T17:02:39.999648+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:37.471312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979476+0000","last_change":"2024-05-30T17:02:39.979476+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979476+0000","last_undegraded":"2024-05-30T17:02:39.979476+0000","last_fullsized":"2024-05-30T17:02:39.979476+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961768+0000","last_change":"2024-05-30T17:02:38.961768+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961768+0000","last_undegraded":"2024-05-30T17:02:38.961768+0000","last_fullsized":"2024-05-30T17:02:38.961768+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.993019+0000","last_change":"2024-05-30T17:02:39.993019+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.993019+0000","last_undegraded":"2024-05-30T17:02:39.993019+0000","last_fullsized":"2024-05-30T17:02:39.993019+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001739+0000","last_change":"2024-05-30T17:02:39.997156+0000","last_active":"2024-05-30T17:02:40.001739+0000","last_peered":"2024-05-30T17:02:40.001739+0000","last_clean":"2024-05-30T17:02:40.001739+0000","last_became_active":"2024-05-30T17:02:39.996869+0000","last_became_peered":"2024-05-30T17:02:39.996869+0000","last_unstale":"2024-05-30T17:02:40.001739+0000","last_undegraded":"2024-05-30T17:02:40.001739+0000","last_fullsized":"2024-05-30T17:02:40.001739+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:59:49.118926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.002277+0000","last_change":"2024-05-30T17:02:40.000552+0000","last_active":"2024-05-30T17:02:40.002277+0000","last_peered":"2024-05-30T17:02:40.002277+0000","last_clean":"2024-05-30T17:02:40.002277+0000","last_became_active":"2024-05-30T17:02:40.000240+0000","last_became_peered":"2024-05-30T17:02:40.000240+0000","last_unstale":"2024-05-30T17:02:40.002277+0000","last_undegraded":"2024-05-30T17:02:40.002277+0000","last_fullsized":"2024-05-30T17:02:40.002277+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:17:05.041359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961468+0000","last_change":"2024-05-30T17:02:38.961468+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961468+0000","last_undegraded":"2024-05-30T17:02:38.961468+0000","last_fullsized":"2024-05-30T17:02:38.961468+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986377+0000","last_change":"2024-05-30T17:02:39.986377+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986377+0000","last_undegraded":"2024-05-30T17:02:39.986377+0000","last_fullsized":"2024-05-30T17:02:39.986377+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001681+0000","last_change":"2024-05-30T17:02:39.997337+0000","last_active":"2024-05-30T17:02:40.001681+0000","last_peered":"2024-05-30T17:02:40.001681+0000","last_clean":"2024-05-30T17:02:40.001681+0000","last_became_active":"2024-05-30T17:02:39.996998+0000","last_became_peered":"2024-05-30T17:02:39.996998+0000","last_unstale":"2024-05-30T17:02:40.001681+0000","last_undegraded":"2024-05-30T17:02:40.001681+0000","last_fullsized":"2024-05-30T17:02:40.001681+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:28.496917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.984796+0000","last_change":"2024-05-30T17:02:39.984796+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.984796+0000","last_undegraded":"2024-05-30T17:02:39.984796+0000","last_fullsized":"2024-05-30T17:02:39.984796+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012323+0000","last_change":"2024-05-30T17:02:39.996922+0000","last_active":"2024-05-30T17:02:42.012323+0000","last_peered":"2024-05-30T17:02:42.012323+0000","last_clean":"2024-05-30T17:02:42.012323+0000","last_became_active":"2024-05-30T17:02:39.996561+0000","last_became_peered":"2024-05-30T17:02:39.996561+0000","last_unstale":"2024-05-30T17:02:42.012323+0000","last_undegraded":"2024-05-30T17:02:42.012323+0000","last_fullsized":"2024-05-30T17:02:42.012323+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:16:39.184990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.987678+0000","last_change":"2024-05-30T17:02:39.987678+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.987678+0000","last_undegraded":"2024-05-30T17:02:39.987678+0000","last_fullsized":"2024-05-30T17:02:39.987678+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001125+0000","last_change":"2024-05-30T17:02:39.999916+0000","last_active":"2024-05-30T17:02:40.001125+0000","last_peered":"2024-05-30T17:02:40.001125+0000","last_clean":"2024-05-30T17:02:40.001125+0000","last_became_active":"2024-05-30T17:02:39.999588+0000","last_became_peered":"2024-05-30T17:02:39.999588+0000","last_unstale":"2024-05-30T17:02:40.001125+0000","last_undegraded":"2024-05-30T17:02:40.001125+0000","last_fullsized":"2024-05-30T17:02:40.001125+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:26:08.382531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738381+0000","last_change":"2024-05-30T17:02:41.009038+0000","last_active":"2024-05-30T17:02:43.738381+0000","last_peered":"2024-05-30T17:02:43.738381+0000","last_clean":"2024-05-30T17:02:43.738381+0000","last_became_active":"2024-05-30T17:02:41.008540+0000","last_became_peered":"2024-05-30T17:02:41.008540+0000","last_unstale":"2024-05-30T17:02:43.738381+0000","last_undegraded":"2024-05-30T17:02:43.738381+0000","last_fullsized":"2024-05-30T17:02:43.738381+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:44.351484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.986268+0000","last_change":"2024-05-30T17:02:39.994753+0000","last_active":"2024-05-30T17:02:40.986268+0000","last_peered":"2024-05-30T17:02:40.986268+0000","last_clean":"2024-05-30T17:02:40.986268+0000","last_became_active":"2024-05-30T17:02:39.994218+0000","last_became_peered":"2024-05-30T17:02:39.994218+0000","last_unstale":"2024-05-30T17:02:40.986268+0000","last_undegraded":"2024-05-30T17:02:40.986268+0000","last_fullsized":"2024-05-30T17:02:40.986268+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:01.556854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961002+0000","last_change":"2024-05-30T17:02:38.961002+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961002+0000","last_undegraded":"2024-05-30T17:02:38.961002+0000","last_fullsized":"2024-05-30T17:02:38.961002+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979708+0000","last_change":"2024-05-30T17:02:39.979708+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979708+0000","last_undegraded":"2024-05-30T17:02:39.979708+0000","last_fullsized":"2024-05-30T17:02:39.979708+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012123+0000","last_change":"2024-05-30T17:02:39.990744+0000","last_active":"2024-05-30T17:02:42.012123+0000","last_peered":"2024-05-30T17:02:42.012123+0000","last_clean":"2024-05-30T17:02:42.012123+0000","last_became_active":"2024-05-30T17:02:39.990046+0000","last_became_peered":"2024-05-30T17:02:39.990046+0000","last_unstale":"2024-05-30T17:02:42.012123+0000","last_undegraded":"2024-05-30T17:02:42.012123+0000","last_fullsized":"2024-05-30T17:02:42.012123+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:00:30.204717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":352,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738594+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:02:43.738594+0000","last_peered":"2024-05-30T17:02:43.738594+0000","last_clean":"2024-05-30T17:02:43.738594+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:02:43.738594+0000","last_undegraded":"2024-05-30T17:02:43.738594+0000","last_fullsized":"2024-05-30T17:02:43.738594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":7},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131052,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240788,"statfs":{"total":96636764160,"available":96502566912,"internally_reserved":0,"allocated":1409024,"data_stored":615310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144944,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226896,"statfs":{"total":96636764160,"available":96488341504,"internally_reserved":0,"allocated":1409024,"data_stored":615310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509672,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131056,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240784,"statfs":{"total":96636764160,"available":96502562816,"internally_reserved":0,"allocated":1409024,"data_stored":615310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149116,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222724,"statfs":{"total":96636764160,"available":96484069376,"internally_reserved":0,"allocated":1421312,"data_stored":622197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144936,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226904,"statfs":{"total":96636764160,"available":96488349696,"internally_reserved":0,"allocated":1396736,"data_stored":608923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542375,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131652,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240188,"statfs":{"total":96636764160,"available":96501952512,"internally_reserved":0,"allocated":2023424,"data_stored":1213181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542376,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64468,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307372,"statfs":{"total":96636764160,"available":96570748928,"internally_reserved":0,"allocated":1441792,"data_stored":629213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542375,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87208,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284632,"statfs":{"total":96636764160,"available":96547463168,"internally_reserved":0,"allocated":1990656,"data_stored":1199291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:02:45.965 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:02:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.215+0000 7f8c14e00640 1 Processor -- start 2024-05-30T17:02:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.215+0000 7f8c14e00640 1 -- start start 2024-05-30T17:02:46.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.217+0000 7f8c14e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8c10156b30 0x7f8c100af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.217+0000 7f8c14e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c100ad940 0x7f8c100b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.217+0000 7f8c14e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c10156f10 con 0x7f8c1006ee40 2024-05-30T17:02:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.217+0000 7f8c14e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8c1006dda0 con 0x7f8c10156b30 2024-05-30T17:02:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.217+0000 7f8c14e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1006e080 con 0x7f8c100ad940 2024-05-30T17:02:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.217+0000 7f8c0ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8c1006ee40 0x7f8c100ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58640/0 (socket says 172.21.15.67:58640) 2024-05-30T17:02:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.217+0000 7f8c0ea00640 1 -- 172.21.15.67:0/1007255519 learned_addr learned my addr 172.21.15.67:0/1007255519 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0f400640 1 --2- 172.21.15.67:0/1007255519 >> v2:172.21.15.169:3300/0 conn(0x7f8c10156b30 0x7f8c100af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0fe00640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c100ad940 0x7f8c100b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775835531 0 0) 0x7f8c10156f10 con 0x7f8c1006ee40 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bf4003520 con 0x7f8c1006ee40 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 10469484 0 0) 0x7f8bf4003520 con 0x7f8c1006ee40 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c100ad940 msgr2=0x7f8c100b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c100ad940 0x7f8c100b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 >> v2:172.21.15.169:3300/0 conn(0x7f8c10156b30 msgr2=0x7f8c100af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 --2- 172.21.15.67:0/1007255519 >> v2:172.21.15.169:3300/0 conn(0x7f8c10156b30 0x7f8c100af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1006e360 con 0x7f8c1006ee40 2024-05-30T17:02:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f8bfc001670 con 0x7f8c1006ee40 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 >> v1:172.21.15.67:6789/0 conn(0x7f8c1006ee40 legacy=0x7f8c100ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bf40039b0 0x7f8bf4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c100ad940 0x7f8bf4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf4004d40 0x7f8bf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1006dda0 con 0x7f8bf40039b0 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1006e080 con 0x7f8bf4004d40 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.218+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf400ba40 con 0x7f8c100ad940 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0f400640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bf40039b0 0x7f8bf4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0ea00640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c100ad940 0x7f8bf4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0fe00640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf4004d40 0x7f8bf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0f400640 1 -- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c100ad940 msgr2=0x7f8bf4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0f400640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c100ad940 0x7f8bf4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0f400640 1 -- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf4004d40 msgr2=0x7f8bf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0f400640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf4004d40 0x7f8bf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0f400640 1 -- 172.21.15.67:0/1007255519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf4003520 con 0x7f8bf40039b0 2024-05-30T17:02:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0ea00640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c100ad940 0x7f8bf4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.219+0000 7f8c0fe00640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf4004d40 0x7f8bf40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.220+0000 7f8c0f400640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bf40039b0 0x7f8bf4003dd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009ac0 tx=0x7f8bf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4036d4331e70b835 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00036070 con 0x7f8bf40039b0 2024-05-30T17:02:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c00031400 con 0x7f8bf40039b0 2024-05-30T17:02:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c0e000640 1 -- 172.21.15.67:0/1007255519 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00009690 con 0x7f8bf40039b0 2024-05-30T17:02:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c14e00640 1 -- 172.21.15.67:0/1007255519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bf40039b0 msgr2=0x7f8bf4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c14e00640 1 --2- 172.21.15.67:0/1007255519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bf40039b0 0x7f8bf4003dd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009ac0 tx=0x7f8bf4005830 comp rx=0 tx=0).stop 2024-05-30T17:02:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c0fe00640 1 -- 172.21.15.67:0/1007255519 reap_dead start 2024-05-30T17:02:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c14e00640 1 -- 172.21.15.67:0/1007255519 shutdown_connections 2024-05-30T17:02:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c14e00640 1 -- 172.21.15.67:0/1007255519 >> 172.21.15.67:0/1007255519 conn(0x7f8c100b1fc0 msgr2=0x7f8c100b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c14e00640 1 -- 172.21.15.67:0/1007255519 shutdown_connections 2024-05-30T17:02:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.221+0000 7f8c14e00640 1 -- 172.21.15.67:0/1007255519 wait complete. 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.223+0000 7f8c14e00640 1 Processor -- start 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.223+0000 7f8c14e00640 1 -- start start 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.224+0000 7f8c14e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf40039b0 0x7f8c1014e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.224+0000 7f8c14e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c100b0400 0x7f8c10159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.224+0000 7f8c14e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c101456b0 0x7f8c10156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.224+0000 7f8c14e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8c100afc20 con 0x7f8c100b0400 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.224+0000 7f8c14e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c100b2c30 con 0x7f8bf40039b0 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.224+0000 7f8c14e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8c10159fe0 con 0x7f8c101456b0 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.224+0000 7f8c0ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c100b0400 0x7f8c10159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c100b0400 0x7f8c10159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60736/0 (socket says 172.21.15.67:60736) 2024-05-30T17:02:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0ea00640 1 -- 172.21.15.67:0/930050441 learned_addr learned my addr 172.21.15.67:0/930050441 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0f400640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf40039b0 0x7f8c1014e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0fe00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c101456b0 0x7f8c10156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0ea00640 1 -- 172.21.15.67:0/930050441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c101456b0 msgr2=0x7f8c10156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0ea00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c101456b0 0x7f8c10156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0ea00640 1 -- 172.21.15.67:0/930050441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf40039b0 msgr2=0x7f8c1014e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0ea00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf40039b0 0x7f8c1014e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0ea00640 1 -- 172.21.15.67:0/930050441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c10159d60 con 0x7f8c100b0400 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0f400640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf40039b0 0x7f8c1014e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.225+0000 7f8c0ea00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c100b0400 0x7f8c10159820 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00d7a0 tx=0x7f8bfc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.226+0000 7f8c07400640 1 -- 172.21.15.67:0/930050441 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc019070 con 0x7f8c100b0400 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.226+0000 7f8c07400640 1 -- 172.21.15.67:0/930050441 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc0074e0 con 0x7f8c100b0400 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.226+0000 7f8c07400640 1 -- 172.21.15.67:0/930050441 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc004a00 con 0x7f8c100b0400 2024-05-30T17:02:46.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.226+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1014d290 con 0x7f8c100b0400 2024-05-30T17:02:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.226+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1014d850 con 0x7f8c100b0400 2024-05-30T17:02:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.227+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c100689e0 con 0x7f8c100b0400 2024-05-30T17:02:46.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.228+0000 7f8c07400640 1 -- 172.21.15.67:0/930050441 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f8bfc01c070 con 0x7f8c100b0400 2024-05-30T17:02:46.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.229+0000 7f8c07400640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8bf006e090 0x7f8bf0070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.229+0000 7f8c07400640 1 -- 172.21.15.67:0/930050441 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f8bfc08a360 con 0x7f8c100b0400 2024-05-30T17:02:46.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.229+0000 7f8c0f400640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8bf006e090 0x7f8bf0070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.230+0000 7f8c0f400640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8bf006e090 0x7f8bf0070550 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8c00033000 tx=0x7f8c00008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:46.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.234+0000 7f8c07400640 1 -- 172.21.15.67:0/930050441 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c10159d60 con 0x7f8c100b0400 2024-05-30T17:02:46.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.431+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c1006f7f0 con 0x7f8bf006e090 2024-05-30T17:02:46.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.449+0000 7f8c07400640 1 -- 172.21.15.67:0/930050441 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247323 (secure 0 0 0) 0x7f8c1006f7f0 con 0x7f8bf006e090 2024-05-30T17:02:46.451 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:46.455 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:02:46.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.455+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8bf006e090 msgr2=0x7f8bf0070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.455+0000 7f8c14e00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8bf006e090 0x7f8bf0070550 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8c00033000 tx=0x7f8c00008030 comp rx=0 tx=0).stop 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.455+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c100b0400 msgr2=0x7f8c10159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.455+0000 7f8c14e00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c100b0400 0x7f8c10159820 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00d7a0 tx=0x7f8bfc00dc60 comp rx=0 tx=0).stop 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.456+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 shutdown_connections 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.456+0000 7f8c14e00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8bf006e090 0x7f8bf0070550 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.456+0000 7f8c14e00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c101456b0 0x7f8c10156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.456+0000 7f8c14e00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c100b0400 0x7f8c10159820 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.456+0000 7f8c14e00640 1 --2- 172.21.15.67:0/930050441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8bf40039b0 0x7f8c1014e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.456+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 >> 172.21.15.67:0/930050441 conn(0x7f8c100b1fc0 msgr2=0x7f8c1015bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.456+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 shutdown_connections 2024-05-30T17:02:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.456+0000 7f8c14e00640 1 -- 172.21.15.67:0/930050441 wait complete. 2024-05-30T17:02:46.474 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":682,"stamp":"2024-05-30T17:02:44.522439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":184,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":984432,"kb_used_data":12208,"kb_used_omap":12,"kb_used_meta":972083,"kb_avail":753990288,"statfs":{"total":773094113280,"available":772086054912,"internally_reserved":0,"allocated":12500992,"data_stored":6118735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":995413552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570340"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986625+0000","last_change":"2024-05-30T17:02:39.986625+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986625+0000","last_undegraded":"2024-05-30T17:02:39.986625+0000","last_fullsized":"2024-05-30T17:02:39.986625+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738383+0000","last_change":"2024-05-30T17:02:41.013698+0000","last_active":"2024-05-30T17:02:43.738383+0000","last_peered":"2024-05-30T17:02:43.738383+0000","last_clean":"2024-05-30T17:02:43.738383+0000","last_became_active":"2024-05-30T17:02:41.013407+0000","last_became_peered":"2024-05-30T17:02:41.013407+0000","last_unstale":"2024-05-30T17:02:43.738383+0000","last_undegraded":"2024-05-30T17:02:43.738383+0000","last_fullsized":"2024-05-30T17:02:43.738383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:56.046859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.008923+0000","last_change":"2024-05-30T17:02:41.006127+0000","last_active":"2024-05-30T17:02:41.008923+0000","last_peered":"2024-05-30T17:02:41.008923+0000","last_clean":"2024-05-30T17:02:41.008923+0000","last_became_active":"2024-05-30T17:02:41.005503+0000","last_became_peered":"2024-05-30T17:02:41.005503+0000","last_unstale":"2024-05-30T17:02:41.008923+0000","last_undegraded":"2024-05-30T17:02:41.008923+0000","last_fullsized":"2024-05-30T17:02:41.008923+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:01.078245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.982145+0000","last_change":"2024-05-30T17:02:39.982145+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.982145+0000","last_undegraded":"2024-05-30T17:02:39.982145+0000","last_fullsized":"2024-05-30T17:02:39.982145+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011985+0000","last_change":"2024-05-30T17:02:41.010886+0000","last_active":"2024-05-30T17:02:42.011985+0000","last_peered":"2024-05-30T17:02:42.011985+0000","last_clean":"2024-05-30T17:02:42.011985+0000","last_became_active":"2024-05-30T17:02:41.007377+0000","last_became_peered":"2024-05-30T17:02:41.007377+0000","last_unstale":"2024-05-30T17:02:42.011985+0000","last_undegraded":"2024-05-30T17:02:42.011985+0000","last_fullsized":"2024-05-30T17:02:42.011985+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:59.071677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738235+0000","last_change":"2024-05-30T17:02:41.014332+0000","last_active":"2024-05-30T17:02:43.738235+0000","last_peered":"2024-05-30T17:02:43.738235+0000","last_clean":"2024-05-30T17:02:43.738235+0000","last_became_active":"2024-05-30T17:02:41.014074+0000","last_became_peered":"2024-05-30T17:02:41.014074+0000","last_unstale":"2024-05-30T17:02:43.738235+0000","last_undegraded":"2024-05-30T17:02:43.738235+0000","last_fullsized":"2024-05-30T17:02:43.738235+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:00:36.629725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.987823+0000","last_change":"2024-05-30T17:02:39.987823+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.987823+0000","last_undegraded":"2024-05-30T17:02:39.987823+0000","last_fullsized":"2024-05-30T17:02:39.987823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738860+0000","last_change":"2024-05-30T17:02:41.011886+0000","last_active":"2024-05-30T17:02:43.738860+0000","last_peered":"2024-05-30T17:02:43.738860+0000","last_clean":"2024-05-30T17:02:43.738860+0000","last_became_active":"2024-05-30T17:02:41.010296+0000","last_became_peered":"2024-05-30T17:02:41.010296+0000","last_unstale":"2024-05-30T17:02:43.738860+0000","last_undegraded":"2024-05-30T17:02:43.738860+0000","last_fullsized":"2024-05-30T17:02:43.738860+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:20.421009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737964+0000","last_change":"2024-05-30T17:02:41.012428+0000","last_active":"2024-05-30T17:02:43.737964+0000","last_peered":"2024-05-30T17:02:43.737964+0000","last_clean":"2024-05-30T17:02:43.737964+0000","last_became_active":"2024-05-30T17:02:41.011764+0000","last_became_peered":"2024-05-30T17:02:41.011764+0000","last_unstale":"2024-05-30T17:02:43.737964+0000","last_undegraded":"2024-05-30T17:02:43.737964+0000","last_fullsized":"2024-05-30T17:02:43.737964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:11:05.228132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983128+0000","last_change":"2024-05-30T17:02:39.983128+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983128+0000","last_undegraded":"2024-05-30T17:02:39.983128+0000","last_fullsized":"2024-05-30T17:02:39.983128+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.991643+0000","last_change":"2024-05-30T17:02:39.991643+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.991643+0000","last_undegraded":"2024-05-30T17:02:39.991643+0000","last_fullsized":"2024-05-30T17:02:39.991643+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.993548+0000","last_change":"2024-05-30T17:02:39.993548+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.993548+0000","last_undegraded":"2024-05-30T17:02:39.993548+0000","last_fullsized":"2024-05-30T17:02:39.993548+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738507+0000","last_change":"2024-05-30T17:02:41.011007+0000","last_active":"2024-05-30T17:02:43.738507+0000","last_peered":"2024-05-30T17:02:43.738507+0000","last_clean":"2024-05-30T17:02:43.738507+0000","last_became_active":"2024-05-30T17:02:41.010322+0000","last_became_peered":"2024-05-30T17:02:41.010322+0000","last_unstale":"2024-05-30T17:02:43.738507+0000","last_undegraded":"2024-05-30T17:02:43.738507+0000","last_fullsized":"2024-05-30T17:02:43.738507+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:40:54.062584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.987711+0000","last_change":"2024-05-30T17:02:39.987711+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.987711+0000","last_undegraded":"2024-05-30T17:02:39.987711+0000","last_fullsized":"2024-05-30T17:02:39.987711+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986411+0000","last_change":"2024-05-30T17:02:39.986411+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986411+0000","last_undegraded":"2024-05-30T17:02:39.986411+0000","last_fullsized":"2024-05-30T17:02:39.986411+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986524+0000","last_change":"2024-05-30T17:02:39.986524+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986524+0000","last_undegraded":"2024-05-30T17:02:39.986524+0000","last_fullsized":"2024-05-30T17:02:39.986524+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983355+0000","last_change":"2024-05-30T17:02:39.983355+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983355+0000","last_undegraded":"2024-05-30T17:02:39.983355+0000","last_fullsized":"2024-05-30T17:02:39.983355+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.980347+0000","last_change":"2024-05-30T17:02:39.980347+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.980347+0000","last_undegraded":"2024-05-30T17:02:39.980347+0000","last_fullsized":"2024-05-30T17:02:39.980347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.988429+0000","last_change":"2024-05-30T17:02:39.988429+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.988429+0000","last_undegraded":"2024-05-30T17:02:39.988429+0000","last_fullsized":"2024-05-30T17:02:39.988429+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.990375+0000","last_change":"2024-05-30T17:02:39.990375+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.990375+0000","last_undegraded":"2024-05-30T17:02:39.990375+0000","last_fullsized":"2024-05-30T17:02:39.990375+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979183+0000","last_change":"2024-05-30T17:02:39.979183+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979183+0000","last_undegraded":"2024-05-30T17:02:39.979183+0000","last_fullsized":"2024-05-30T17:02:39.979183+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.989647+0000","last_change":"2024-05-30T17:02:39.989647+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.989647+0000","last_undegraded":"2024-05-30T17:02:39.989647+0000","last_fullsized":"2024-05-30T17:02:39.989647+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012127+0000","last_change":"2024-05-30T17:02:41.011572+0000","last_active":"2024-05-30T17:02:42.012127+0000","last_peered":"2024-05-30T17:02:42.012127+0000","last_clean":"2024-05-30T17:02:42.012127+0000","last_became_active":"2024-05-30T17:02:41.011197+0000","last_became_peered":"2024-05-30T17:02:41.011197+0000","last_unstale":"2024-05-30T17:02:42.012127+0000","last_undegraded":"2024-05-30T17:02:42.012127+0000","last_fullsized":"2024-05-30T17:02:42.012127+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:54.031517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.981377+0000","last_change":"2024-05-30T17:02:39.981377+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.981377+0000","last_undegraded":"2024-05-30T17:02:39.981377+0000","last_fullsized":"2024-05-30T17:02:39.981377+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983059+0000","last_change":"2024-05-30T17:02:39.983059+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983059+0000","last_undegraded":"2024-05-30T17:02:39.983059+0000","last_fullsized":"2024-05-30T17:02:39.983059+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"115'4","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012308+0000","last_change":"2024-05-30T17:02:39.992222+0000","last_active":"2024-05-30T17:02:42.012308+0000","last_peered":"2024-05-30T17:02:42.012308+0000","last_clean":"2024-05-30T17:02:42.012308+0000","last_became_active":"2024-05-30T17:02:39.991672+0000","last_became_peered":"2024-05-30T17:02:39.991672+0000","last_unstale":"2024-05-30T17:02:42.012308+0000","last_undegraded":"2024-05-30T17:02:42.012308+0000","last_fullsized":"2024-05-30T17:02:42.012308+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:17:06.803976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.984329+0000","last_change":"2024-05-30T17:02:39.984329+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.984329+0000","last_undegraded":"2024-05-30T17:02:39.984329+0000","last_fullsized":"2024-05-30T17:02:39.984329+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738709+0000","last_change":"2024-05-30T17:02:40.001257+0000","last_active":"2024-05-30T17:02:43.738709+0000","last_peered":"2024-05-30T17:02:43.738709+0000","last_clean":"2024-05-30T17:02:43.738709+0000","last_became_active":"2024-05-30T17:02:40.000956+0000","last_became_peered":"2024-05-30T17:02:40.000956+0000","last_unstale":"2024-05-30T17:02:43.738709+0000","last_undegraded":"2024-05-30T17:02:43.738709+0000","last_fullsized":"2024-05-30T17:02:43.738709+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:52.599362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.984765+0000","last_change":"2024-05-30T17:02:39.984765+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.984765+0000","last_undegraded":"2024-05-30T17:02:39.984765+0000","last_fullsized":"2024-05-30T17:02:39.984765+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.026697+0000","last_change":"2024-05-30T17:02:40.004186+0000","last_active":"2024-05-30T17:02:41.026697+0000","last_peered":"2024-05-30T17:02:41.026697+0000","last_clean":"2024-05-30T17:02:41.026697+0000","last_became_active":"2024-05-30T17:02:40.003831+0000","last_became_peered":"2024-05-30T17:02:40.003831+0000","last_unstale":"2024-05-30T17:02:41.026697+0000","last_undegraded":"2024-05-30T17:02:41.026697+0000","last_fullsized":"2024-05-30T17:02:41.026697+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:09.841780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738822+0000","last_change":"2024-05-30T17:02:41.011197+0000","last_active":"2024-05-30T17:02:43.738822+0000","last_peered":"2024-05-30T17:02:43.738822+0000","last_clean":"2024-05-30T17:02:43.738822+0000","last_became_active":"2024-05-30T17:02:41.010742+0000","last_became_peered":"2024-05-30T17:02:41.010742+0000","last_unstale":"2024-05-30T17:02:43.738822+0000","last_undegraded":"2024-05-30T17:02:43.738822+0000","last_fullsized":"2024-05-30T17:02:43.738822+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:46:14.746408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.029154+0000","last_change":"2024-05-30T17:02:39.999367+0000","last_active":"2024-05-30T17:02:41.029154+0000","last_peered":"2024-05-30T17:02:41.029154+0000","last_clean":"2024-05-30T17:02:41.029154+0000","last_became_active":"2024-05-30T17:02:39.998521+0000","last_became_peered":"2024-05-30T17:02:39.998521+0000","last_unstale":"2024-05-30T17:02:41.029154+0000","last_undegraded":"2024-05-30T17:02:41.029154+0000","last_fullsized":"2024-05-30T17:02:41.029154+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:34.293069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012030+0000","last_change":"2024-05-30T17:02:39.994746+0000","last_active":"2024-05-30T17:02:42.012030+0000","last_peered":"2024-05-30T17:02:42.012030+0000","last_clean":"2024-05-30T17:02:42.012030+0000","last_became_active":"2024-05-30T17:02:39.994412+0000","last_became_peered":"2024-05-30T17:02:39.994412+0000","last_unstale":"2024-05-30T17:02:42.012030+0000","last_undegraded":"2024-05-30T17:02:42.012030+0000","last_fullsized":"2024-05-30T17:02:42.012030+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:01:23.895654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.988793+0000","last_change":"2024-05-30T17:02:39.988793+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.988793+0000","last_undegraded":"2024-05-30T17:02:39.988793+0000","last_fullsized":"2024-05-30T17:02:39.988793+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.004920+0000","last_change":"2024-05-30T17:02:40.002354+0000","last_active":"2024-05-30T17:02:40.004920+0000","last_peered":"2024-05-30T17:02:40.004920+0000","last_clean":"2024-05-30T17:02:40.004920+0000","last_became_active":"2024-05-30T17:02:40.002062+0000","last_became_peered":"2024-05-30T17:02:40.002062+0000","last_unstale":"2024-05-30T17:02:40.004920+0000","last_undegraded":"2024-05-30T17:02:40.004920+0000","last_fullsized":"2024-05-30T17:02:40.004920+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:45:00.479727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.000314+0000","last_change":"2024-05-30T17:02:39.996962+0000","last_active":"2024-05-30T17:02:40.000314+0000","last_peered":"2024-05-30T17:02:40.000314+0000","last_clean":"2024-05-30T17:02:40.000314+0000","last_became_active":"2024-05-30T17:02:39.996677+0000","last_became_peered":"2024-05-30T17:02:39.996677+0000","last_unstale":"2024-05-30T17:02:40.000314+0000","last_undegraded":"2024-05-30T17:02:40.000314+0000","last_fullsized":"2024-05-30T17:02:40.000314+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:47:32.794292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012752+0000","last_change":"2024-05-30T17:02:41.010582+0000","last_active":"2024-05-30T17:02:42.012752+0000","last_peered":"2024-05-30T17:02:42.012752+0000","last_clean":"2024-05-30T17:02:42.012752+0000","last_became_active":"2024-05-30T17:02:41.006958+0000","last_became_peered":"2024-05-30T17:02:41.006958+0000","last_unstale":"2024-05-30T17:02:42.012752+0000","last_undegraded":"2024-05-30T17:02:42.012752+0000","last_fullsized":"2024-05-30T17:02:42.012752+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:42:06.872806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.962728+0000","last_change":"2024-05-30T17:02:38.962728+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.962728+0000","last_undegraded":"2024-05-30T17:02:38.962728+0000","last_fullsized":"2024-05-30T17:02:38.962728+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012844+0000","last_change":"2024-05-30T17:02:40.999926+0000","last_active":"2024-05-30T17:02:42.012844+0000","last_peered":"2024-05-30T17:02:42.012844+0000","last_clean":"2024-05-30T17:02:42.012844+0000","last_became_active":"2024-05-30T17:02:40.997094+0000","last_became_peered":"2024-05-30T17:02:40.997094+0000","last_unstale":"2024-05-30T17:02:42.012844+0000","last_undegraded":"2024-05-30T17:02:42.012844+0000","last_fullsized":"2024-05-30T17:02:42.012844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:15:52.585750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.991460+0000","last_change":"2024-05-30T17:02:40.000501+0000","last_active":"2024-05-30T17:02:40.991460+0000","last_peered":"2024-05-30T17:02:40.991460+0000","last_clean":"2024-05-30T17:02:40.991460+0000","last_became_active":"2024-05-30T17:02:39.999369+0000","last_became_peered":"2024-05-30T17:02:39.999369+0000","last_unstale":"2024-05-30T17:02:40.991460+0000","last_undegraded":"2024-05-30T17:02:40.991460+0000","last_fullsized":"2024-05-30T17:02:40.991460+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:02.236301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.983068+0000","last_change":"2024-05-30T17:02:39.983068+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.983068+0000","last_undegraded":"2024-05-30T17:02:39.983068+0000","last_fullsized":"2024-05-30T17:02:39.983068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.005002+0000","last_change":"2024-05-30T17:02:40.002597+0000","last_active":"2024-05-30T17:02:40.005002+0000","last_peered":"2024-05-30T17:02:40.005002+0000","last_clean":"2024-05-30T17:02:40.005002+0000","last_became_active":"2024-05-30T17:02:40.002351+0000","last_became_peered":"2024-05-30T17:02:40.002351+0000","last_unstale":"2024-05-30T17:02:40.005002+0000","last_undegraded":"2024-05-30T17:02:40.005002+0000","last_fullsized":"2024-05-30T17:02:40.005002+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:34:39.559523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.978444+0000","last_change":"2024-05-30T17:02:39.978444+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.978444+0000","last_undegraded":"2024-05-30T17:02:39.978444+0000","last_fullsized":"2024-05-30T17:02:39.978444+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.962097+0000","last_change":"2024-05-30T17:02:38.962097+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.962097+0000","last_undegraded":"2024-05-30T17:02:38.962097+0000","last_fullsized":"2024-05-30T17:02:38.962097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012385+0000","last_change":"2024-05-30T17:02:41.003790+0000","last_active":"2024-05-30T17:02:42.012385+0000","last_peered":"2024-05-30T17:02:42.012385+0000","last_clean":"2024-05-30T17:02:42.012385+0000","last_became_active":"2024-05-30T17:02:41.003476+0000","last_became_peered":"2024-05-30T17:02:41.003476+0000","last_unstale":"2024-05-30T17:02:42.012385+0000","last_undegraded":"2024-05-30T17:02:42.012385+0000","last_fullsized":"2024-05-30T17:02:42.012385+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:19.156205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961654+0000","last_change":"2024-05-30T17:02:38.961654+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961654+0000","last_undegraded":"2024-05-30T17:02:38.961654+0000","last_fullsized":"2024-05-30T17:02:38.961654+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979914+0000","last_change":"2024-05-30T17:02:39.979914+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979914+0000","last_undegraded":"2024-05-30T17:02:39.979914+0000","last_fullsized":"2024-05-30T17:02:39.979914+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:39.999619+0000","last_change":"2024-05-30T17:02:39.993748+0000","last_active":"2024-05-30T17:02:39.999619+0000","last_peered":"2024-05-30T17:02:39.999619+0000","last_clean":"2024-05-30T17:02:39.999619+0000","last_became_active":"2024-05-30T17:02:39.993315+0000","last_became_peered":"2024-05-30T17:02:39.993315+0000","last_unstale":"2024-05-30T17:02:39.999619+0000","last_undegraded":"2024-05-30T17:02:39.999619+0000","last_fullsized":"2024-05-30T17:02:39.999619+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:05:49.456837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.978924+0000","last_change":"2024-05-30T17:02:39.978924+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.978924+0000","last_undegraded":"2024-05-30T17:02:39.978924+0000","last_fullsized":"2024-05-30T17:02:39.978924+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.007827+0000","last_change":"2024-05-30T17:02:40.006813+0000","last_active":"2024-05-30T17:02:40.007827+0000","last_peered":"2024-05-30T17:02:40.007827+0000","last_clean":"2024-05-30T17:02:40.007827+0000","last_became_active":"2024-05-30T17:02:40.006466+0000","last_became_peered":"2024-05-30T17:02:40.006466+0000","last_unstale":"2024-05-30T17:02:40.007827+0000","last_undegraded":"2024-05-30T17:02:40.007827+0000","last_fullsized":"2024-05-30T17:02:40.007827+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:22:42.496097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.981115+0000","last_change":"2024-05-30T17:02:39.981115+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.981115+0000","last_undegraded":"2024-05-30T17:02:39.981115+0000","last_fullsized":"2024-05-30T17:02:39.981115+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"115'8","reported_seq":20,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.032368+0000","last_change":"2024-05-30T17:02:39.998093+0000","last_active":"2024-05-30T17:02:41.032368+0000","last_peered":"2024-05-30T17:02:41.032368+0000","last_clean":"2024-05-30T17:02:41.032368+0000","last_became_active":"2024-05-30T17:02:39.997282+0000","last_became_peered":"2024-05-30T17:02:39.997282+0000","last_unstale":"2024-05-30T17:02:41.032368+0000","last_undegraded":"2024-05-30T17:02:41.032368+0000","last_fullsized":"2024-05-30T17:02:41.032368+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:31.480091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.013524+0000","last_change":"2024-05-30T17:02:41.009258+0000","last_active":"2024-05-30T17:02:41.013524+0000","last_peered":"2024-05-30T17:02:41.013524+0000","last_clean":"2024-05-30T17:02:41.013524+0000","last_became_active":"2024-05-30T17:02:41.008966+0000","last_became_peered":"2024-05-30T17:02:41.008966+0000","last_unstale":"2024-05-30T17:02:41.013524+0000","last_undegraded":"2024-05-30T17:02:41.013524+0000","last_fullsized":"2024-05-30T17:02:41.013524+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:56:14.951477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961297+0000","last_change":"2024-05-30T17:02:38.961297+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961297+0000","last_undegraded":"2024-05-30T17:02:38.961297+0000","last_fullsized":"2024-05-30T17:02:38.961297+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.990550+0000","last_change":"2024-05-30T17:02:39.990550+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.990550+0000","last_undegraded":"2024-05-30T17:02:39.990550+0000","last_fullsized":"2024-05-30T17:02:39.990550+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:41.028767+0000","last_change":"2024-05-30T17:02:39.995669+0000","last_active":"2024-05-30T17:02:41.028767+0000","last_peered":"2024-05-30T17:02:41.028767+0000","last_clean":"2024-05-30T17:02:41.028767+0000","last_became_active":"2024-05-30T17:02:39.995397+0000","last_became_peered":"2024-05-30T17:02:39.995397+0000","last_unstale":"2024-05-30T17:02:41.028767+0000","last_undegraded":"2024-05-30T17:02:41.028767+0000","last_fullsized":"2024-05-30T17:02:41.028767+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:29.373763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738221+0000","last_change":"2024-05-30T17:02:41.022727+0000","last_active":"2024-05-30T17:02:43.738221+0000","last_peered":"2024-05-30T17:02:43.738221+0000","last_clean":"2024-05-30T17:02:43.738221+0000","last_became_active":"2024-05-30T17:02:41.022446+0000","last_became_peered":"2024-05-30T17:02:41.022446+0000","last_unstale":"2024-05-30T17:02:43.738221+0000","last_undegraded":"2024-05-30T17:02:43.738221+0000","last_fullsized":"2024-05-30T17:02:43.738221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:08.319197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001682+0000","last_change":"2024-05-30T17:02:39.998860+0000","last_active":"2024-05-30T17:02:40.001682+0000","last_peered":"2024-05-30T17:02:40.001682+0000","last_clean":"2024-05-30T17:02:40.001682+0000","last_became_active":"2024-05-30T17:02:39.997624+0000","last_became_peered":"2024-05-30T17:02:39.997624+0000","last_unstale":"2024-05-30T17:02:40.001682+0000","last_undegraded":"2024-05-30T17:02:40.001682+0000","last_fullsized":"2024-05-30T17:02:40.001682+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:53:48.019750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738864+0000","last_change":"2024-05-30T17:02:41.017247+0000","last_active":"2024-05-30T17:02:43.738864+0000","last_peered":"2024-05-30T17:02:43.738864+0000","last_clean":"2024-05-30T17:02:43.738864+0000","last_became_active":"2024-05-30T17:02:41.016943+0000","last_became_peered":"2024-05-30T17:02:41.016943+0000","last_unstale":"2024-05-30T17:02:43.738864+0000","last_undegraded":"2024-05-30T17:02:43.738864+0000","last_fullsized":"2024-05-30T17:02:43.738864+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:14.515707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011769+0000","last_change":"2024-05-30T17:02:40.001321+0000","last_active":"2024-05-30T17:02:42.011769+0000","last_peered":"2024-05-30T17:02:42.011769+0000","last_clean":"2024-05-30T17:02:42.011769+0000","last_became_active":"2024-05-30T17:02:40.000949+0000","last_became_peered":"2024-05-30T17:02:40.000949+0000","last_unstale":"2024-05-30T17:02:42.011769+0000","last_undegraded":"2024-05-30T17:02:42.011769+0000","last_fullsized":"2024-05-30T17:02:42.011769+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:11:12.879603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737942+0000","last_change":"2024-05-30T17:02:41.012668+0000","last_active":"2024-05-30T17:02:43.737942+0000","last_peered":"2024-05-30T17:02:43.737942+0000","last_clean":"2024-05-30T17:02:43.737942+0000","last_became_active":"2024-05-30T17:02:41.012148+0000","last_became_peered":"2024-05-30T17:02:41.012148+0000","last_unstale":"2024-05-30T17:02:43.737942+0000","last_undegraded":"2024-05-30T17:02:43.737942+0000","last_fullsized":"2024-05-30T17:02:43.737942+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:51:48.462366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:39.999598+0000","last_change":"2024-05-30T17:02:39.992411+0000","last_active":"2024-05-30T17:02:39.999598+0000","last_peered":"2024-05-30T17:02:39.999598+0000","last_clean":"2024-05-30T17:02:39.999598+0000","last_became_active":"2024-05-30T17:02:39.989335+0000","last_became_peered":"2024-05-30T17:02:39.989335+0000","last_unstale":"2024-05-30T17:02:39.999598+0000","last_undegraded":"2024-05-30T17:02:39.999598+0000","last_fullsized":"2024-05-30T17:02:39.999598+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:23:05.285368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.986537+0000","last_change":"2024-05-30T17:02:39.993424+0000","last_active":"2024-05-30T17:02:40.986537+0000","last_peered":"2024-05-30T17:02:40.986537+0000","last_clean":"2024-05-30T17:02:40.986537+0000","last_became_active":"2024-05-30T17:02:39.993119+0000","last_became_peered":"2024-05-30T17:02:39.993119+0000","last_unstale":"2024-05-30T17:02:40.986537+0000","last_undegraded":"2024-05-30T17:02:40.986537+0000","last_fullsized":"2024-05-30T17:02:40.986537+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:25:20.784382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737939+0000","last_change":"2024-05-30T17:02:41.008813+0000","last_active":"2024-05-30T17:02:43.737939+0000","last_peered":"2024-05-30T17:02:43.737939+0000","last_clean":"2024-05-30T17:02:43.737939+0000","last_became_active":"2024-05-30T17:02:41.008199+0000","last_became_peered":"2024-05-30T17:02:41.008199+0000","last_unstale":"2024-05-30T17:02:43.737939+0000","last_undegraded":"2024-05-30T17:02:43.737939+0000","last_fullsized":"2024-05-30T17:02:43.737939+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:34:19.448690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:39.999648+0000","last_change":"2024-05-30T17:02:39.996513+0000","last_active":"2024-05-30T17:02:39.999648+0000","last_peered":"2024-05-30T17:02:39.999648+0000","last_clean":"2024-05-30T17:02:39.999648+0000","last_became_active":"2024-05-30T17:02:39.995747+0000","last_became_peered":"2024-05-30T17:02:39.995747+0000","last_unstale":"2024-05-30T17:02:39.999648+0000","last_undegraded":"2024-05-30T17:02:39.999648+0000","last_fullsized":"2024-05-30T17:02:39.999648+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:48:37.471312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979476+0000","last_change":"2024-05-30T17:02:39.979476+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979476+0000","last_undegraded":"2024-05-30T17:02:39.979476+0000","last_fullsized":"2024-05-30T17:02:39.979476+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961768+0000","last_change":"2024-05-30T17:02:38.961768+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961768+0000","last_undegraded":"2024-05-30T17:02:38.961768+0000","last_fullsized":"2024-05-30T17:02:38.961768+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.993019+0000","last_change":"2024-05-30T17:02:39.993019+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.993019+0000","last_undegraded":"2024-05-30T17:02:39.993019+0000","last_fullsized":"2024-05-30T17:02:39.993019+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001739+0000","last_change":"2024-05-30T17:02:39.997156+0000","last_active":"2024-05-30T17:02:40.001739+0000","last_peered":"2024-05-30T17:02:40.001739+0000","last_clean":"2024-05-30T17:02:40.001739+0000","last_became_active":"2024-05-30T17:02:39.996869+0000","last_became_peered":"2024-05-30T17:02:39.996869+0000","last_unstale":"2024-05-30T17:02:40.001739+0000","last_undegraded":"2024-05-30T17:02:40.001739+0000","last_fullsized":"2024-05-30T17:02:40.001739+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:59:49.118926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.002277+0000","last_change":"2024-05-30T17:02:40.000552+0000","last_active":"2024-05-30T17:02:40.002277+0000","last_peered":"2024-05-30T17:02:40.002277+0000","last_clean":"2024-05-30T17:02:40.002277+0000","last_became_active":"2024-05-30T17:02:40.000240+0000","last_became_peered":"2024-05-30T17:02:40.000240+0000","last_unstale":"2024-05-30T17:02:40.002277+0000","last_undegraded":"2024-05-30T17:02:40.002277+0000","last_fullsized":"2024-05-30T17:02:40.002277+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:17:05.041359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961468+0000","last_change":"2024-05-30T17:02:38.961468+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961468+0000","last_undegraded":"2024-05-30T17:02:38.961468+0000","last_fullsized":"2024-05-30T17:02:38.961468+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.986377+0000","last_change":"2024-05-30T17:02:39.986377+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.986377+0000","last_undegraded":"2024-05-30T17:02:39.986377+0000","last_fullsized":"2024-05-30T17:02:39.986377+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001681+0000","last_change":"2024-05-30T17:02:39.997337+0000","last_active":"2024-05-30T17:02:40.001681+0000","last_peered":"2024-05-30T17:02:40.001681+0000","last_clean":"2024-05-30T17:02:40.001681+0000","last_became_active":"2024-05-30T17:02:39.996998+0000","last_became_peered":"2024-05-30T17:02:39.996998+0000","last_unstale":"2024-05-30T17:02:40.001681+0000","last_undegraded":"2024-05-30T17:02:40.001681+0000","last_fullsized":"2024-05-30T17:02:40.001681+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:49:28.496917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.984796+0000","last_change":"2024-05-30T17:02:39.984796+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.984796+0000","last_undegraded":"2024-05-30T17:02:39.984796+0000","last_fullsized":"2024-05-30T17:02:39.984796+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012323+0000","last_change":"2024-05-30T17:02:39.996922+0000","last_active":"2024-05-30T17:02:42.012323+0000","last_peered":"2024-05-30T17:02:42.012323+0000","last_clean":"2024-05-30T17:02:42.012323+0000","last_became_active":"2024-05-30T17:02:39.996561+0000","last_became_peered":"2024-05-30T17:02:39.996561+0000","last_unstale":"2024-05-30T17:02:42.012323+0000","last_undegraded":"2024-05-30T17:02:42.012323+0000","last_fullsized":"2024-05-30T17:02:42.012323+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:16:39.184990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.987678+0000","last_change":"2024-05-30T17:02:39.987678+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.987678+0000","last_undegraded":"2024-05-30T17:02:39.987678+0000","last_fullsized":"2024-05-30T17:02:39.987678+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.001125+0000","last_change":"2024-05-30T17:02:39.999916+0000","last_active":"2024-05-30T17:02:40.001125+0000","last_peered":"2024-05-30T17:02:40.001125+0000","last_clean":"2024-05-30T17:02:40.001125+0000","last_became_active":"2024-05-30T17:02:39.999588+0000","last_became_peered":"2024-05-30T17:02:39.999588+0000","last_unstale":"2024-05-30T17:02:40.001125+0000","last_undegraded":"2024-05-30T17:02:40.001125+0000","last_fullsized":"2024-05-30T17:02:40.001125+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:26:08.382531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738381+0000","last_change":"2024-05-30T17:02:41.009038+0000","last_active":"2024-05-30T17:02:43.738381+0000","last_peered":"2024-05-30T17:02:43.738381+0000","last_clean":"2024-05-30T17:02:43.738381+0000","last_became_active":"2024-05-30T17:02:41.008540+0000","last_became_peered":"2024-05-30T17:02:41.008540+0000","last_unstale":"2024-05-30T17:02:43.738381+0000","last_undegraded":"2024-05-30T17:02:43.738381+0000","last_fullsized":"2024-05-30T17:02:43.738381+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:44.351484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-30T17:02:40.986268+0000","last_change":"2024-05-30T17:02:39.994753+0000","last_active":"2024-05-30T17:02:40.986268+0000","last_peered":"2024-05-30T17:02:40.986268+0000","last_clean":"2024-05-30T17:02:40.986268+0000","last_became_active":"2024-05-30T17:02:39.994218+0000","last_became_peered":"2024-05-30T17:02:39.994218+0000","last_unstale":"2024-05-30T17:02:40.986268+0000","last_undegraded":"2024-05-30T17:02:40.986268+0000","last_fullsized":"2024-05-30T17:02:40.986268+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:01.556854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-30T17:02:39.950253+0000","last_change":"2024-05-30T17:02:39.950253+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.950253+0000","last_undegraded":"2024-05-30T17:02:39.950253+0000","last_fullsized":"2024-05-30T17:02:39.950253+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2024-05-30T17:02:38.961002+0000","last_change":"2024-05-30T17:02:38.961002+0000","last_active":"2024-05-30T17:02:38.933146+0000","last_peered":"2024-05-30T17:02:38.933146+0000","last_clean":"2024-05-30T17:02:38.933146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:38.961002+0000","last_undegraded":"2024-05-30T17:02:38.961002+0000","last_fullsized":"2024-05-30T17:02:38.961002+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-05-30T17:02:39.979708+0000","last_change":"2024-05-30T17:02:39.979708+0000","last_active":"2024-05-30T17:02:39.950253+0000","last_peered":"2024-05-30T17:02:39.950253+0000","last_clean":"2024-05-30T17:02:39.950253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-30T17:02:39.979708+0000","last_undegraded":"2024-05-30T17:02:39.979708+0000","last_fullsized":"2024-05-30T17:02:39.979708+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012123+0000","last_change":"2024-05-30T17:02:39.990744+0000","last_active":"2024-05-30T17:02:42.012123+0000","last_peered":"2024-05-30T17:02:42.012123+0000","last_clean":"2024-05-30T17:02:42.012123+0000","last_became_active":"2024-05-30T17:02:39.990046+0000","last_became_peered":"2024-05-30T17:02:39.990046+0000","last_unstale":"2024-05-30T17:02:42.012123+0000","last_undegraded":"2024-05-30T17:02:42.012123+0000","last_fullsized":"2024-05-30T17:02:42.012123+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:00:30.204717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":352,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738594+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:02:43.738594+0000","last_peered":"2024-05-30T17:02:43.738594+0000","last_clean":"2024-05-30T17:02:43.738594+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:02:43.738594+0000","last_undegraded":"2024-05-30T17:02:43.738594+0000","last_fullsized":"2024-05-30T17:02:43.738594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":7},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131052,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240788,"statfs":{"total":96636764160,"available":96502566912,"internally_reserved":0,"allocated":1409024,"data_stored":615310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144944,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226896,"statfs":{"total":96636764160,"available":96488341504,"internally_reserved":0,"allocated":1409024,"data_stored":615310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509672,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131056,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240784,"statfs":{"total":96636764160,"available":96502562816,"internally_reserved":0,"allocated":1409024,"data_stored":615310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149116,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222724,"statfs":{"total":96636764160,"available":96484069376,"internally_reserved":0,"allocated":1421312,"data_stored":622197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144936,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226904,"statfs":{"total":96636764160,"available":96488349696,"internally_reserved":0,"allocated":1396736,"data_stored":608923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542375,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131652,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240188,"statfs":{"total":96636764160,"available":96501952512,"internally_reserved":0,"allocated":2023424,"data_stored":1213181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542376,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64468,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307372,"statfs":{"total":96636764160,"available":96570748928,"internally_reserved":0,"allocated":1441792,"data_stored":629213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542375,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87208,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284632,"statfs":{"total":96636764160,"available":96547463168,"internally_reserved":0,"allocated":1990656,"data_stored":1199291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:02:46.478 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3f is not active+clean 2024-05-30T17:02:46.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.478 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3e is not active+clean 2024-05-30T17:02:46.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.986625+0000', 'last_change': '2024-05-30T17:02:39.986625+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.986625+0000', 'last_undegraded': '2024-05-30T17:02:39.986625+0000', 'last_fullsized': '2024-05-30T17:02:39.986625+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.478 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3c is not active+clean 2024-05-30T17:02:46.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.478 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3a is not active+clean 2024-05-30T17:02:46.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.982145+0000', 'last_change': '2024-05-30T17:02:39.982145+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.982145+0000', 'last_undegraded': '2024-05-30T17:02:39.982145+0000', 'last_fullsized': '2024-05-30T17:02:39.982145+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:PG 17.37 is not active+clean 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:PG 17.36 is not active+clean 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:PG 17.35 is not active+clean 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.987823+0000', 'last_change': '2024-05-30T17:02:39.987823+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.987823+0000', 'last_undegraded': '2024-05-30T17:02:39.987823+0000', 'last_fullsized': '2024-05-30T17:02:39.987823+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:PG 17.34 is not active+clean 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:PG 17.31 is not active+clean 2024-05-30T17:02:46.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.983128+0000', 'last_change': '2024-05-30T17:02:39.983128+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.983128+0000', 'last_undegraded': '2024-05-30T17:02:39.983128+0000', 'last_fullsized': '2024-05-30T17:02:39.983128+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:PG 17.30 is not active+clean 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2f is not active+clean 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.991643+0000', 'last_change': '2024-05-30T17:02:39.991643+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.991643+0000', 'last_undegraded': '2024-05-30T17:02:39.991643+0000', 'last_fullsized': '2024-05-30T17:02:39.991643+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2e is not active+clean 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.993548+0000', 'last_change': '2024-05-30T17:02:39.993548+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.993548+0000', 'last_undegraded': '2024-05-30T17:02:39.993548+0000', 'last_fullsized': '2024-05-30T17:02:39.993548+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2c is not active+clean 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.987711+0000', 'last_change': '2024-05-30T17:02:39.987711+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.987711+0000', 'last_undegraded': '2024-05-30T17:02:39.987711+0000', 'last_fullsized': '2024-05-30T17:02:39.987711+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2b is not active+clean 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.986411+0000', 'last_change': '2024-05-30T17:02:39.986411+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.986411+0000', 'last_undegraded': '2024-05-30T17:02:39.986411+0000', 'last_fullsized': '2024-05-30T17:02:39.986411+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2a is not active+clean 2024-05-30T17:02:46.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.986524+0000', 'last_change': '2024-05-30T17:02:39.986524+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.986524+0000', 'last_undegraded': '2024-05-30T17:02:39.986524+0000', 'last_fullsized': '2024-05-30T17:02:39.986524+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:PG 17.29 is not active+clean 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.983355+0000', 'last_change': '2024-05-30T17:02:39.983355+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.983355+0000', 'last_undegraded': '2024-05-30T17:02:39.983355+0000', 'last_fullsized': '2024-05-30T17:02:39.983355+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:PG 17.28 is not active+clean 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.980347+0000', 'last_change': '2024-05-30T17:02:39.980347+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.980347+0000', 'last_undegraded': '2024-05-30T17:02:39.980347+0000', 'last_fullsized': '2024-05-30T17:02:39.980347+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:PG 17.27 is not active+clean 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.988429+0000', 'last_change': '2024-05-30T17:02:39.988429+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.988429+0000', 'last_undegraded': '2024-05-30T17:02:39.988429+0000', 'last_fullsized': '2024-05-30T17:02:39.988429+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:PG 17.26 is not active+clean 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:PG 17.25 is not active+clean 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.990375+0000', 'last_change': '2024-05-30T17:02:39.990375+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.990375+0000', 'last_undegraded': '2024-05-30T17:02:39.990375+0000', 'last_fullsized': '2024-05-30T17:02:39.990375+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:PG 17.24 is not active+clean 2024-05-30T17:02:46.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.979183+0000', 'last_change': '2024-05-30T17:02:39.979183+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.979183+0000', 'last_undegraded': '2024-05-30T17:02:39.979183+0000', 'last_fullsized': '2024-05-30T17:02:39.979183+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:PG 17.23 is not active+clean 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.989647+0000', 'last_change': '2024-05-30T17:02:39.989647+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.989647+0000', 'last_undegraded': '2024-05-30T17:02:39.989647+0000', 'last_fullsized': '2024-05-30T17:02:39.989647+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:PG 17.21 is not active+clean 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:PG 17.20 is not active+clean 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.981377+0000', 'last_change': '2024-05-30T17:02:39.981377+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.981377+0000', 'last_undegraded': '2024-05-30T17:02:39.981377+0000', 'last_fullsized': '2024-05-30T17:02:39.981377+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1e is not active+clean 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.983059+0000', 'last_change': '2024-05-30T17:02:39.983059+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.983059+0000', 'last_undegraded': '2024-05-30T17:02:39.983059+0000', 'last_fullsized': '2024-05-30T17:02:39.983059+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1f is not active+clean 2024-05-30T17:02:46.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.984329+0000', 'last_change': '2024-05-30T17:02:39.984329+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.984329+0000', 'last_undegraded': '2024-05-30T17:02:39.984329+0000', 'last_fullsized': '2024-05-30T17:02:39.984329+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1c is not active+clean 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.984765+0000', 'last_change': '2024-05-30T17:02:39.984765+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.984765+0000', 'last_undegraded': '2024-05-30T17:02:39.984765+0000', 'last_fullsized': '2024-05-30T17:02:39.984765+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1a is not active+clean 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1b is not active+clean 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.988793+0000', 'last_change': '2024-05-30T17:02:39.988793+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.988793+0000', 'last_undegraded': '2024-05-30T17:02:39.988793+0000', 'last_fullsized': '2024-05-30T17:02:39.988793+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:PG 17.18 is not active+clean 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:PG 16.18 is not active+clean 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:38.962728+0000', 'last_change': '2024-05-30T17:02:38.962728+0000', 'last_active': '2024-05-30T17:02:38.933146+0000', 'last_peered': '2024-05-30T17:02:38.933146+0000', 'last_clean': '2024-05-30T17:02:38.933146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:38.962728+0000', 'last_undegraded': '2024-05-30T17:02:38.962728+0000', 'last_fullsized': '2024-05-30T17:02:38.962728+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-30T17:02:46.483 INFO:tasks.ceph.ceph_manager.ceph:PG 17.17 is not active+clean 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.983068+0000', 'last_change': '2024-05-30T17:02:39.983068+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.983068+0000', 'last_undegraded': '2024-05-30T17:02:39.983068+0000', 'last_fullsized': '2024-05-30T17:02:39.983068+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:PG 17.14 is not active+clean 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.978444+0000', 'last_change': '2024-05-30T17:02:39.978444+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.978444+0000', 'last_undegraded': '2024-05-30T17:02:39.978444+0000', 'last_fullsized': '2024-05-30T17:02:39.978444+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:PG 16.15 is not active+clean 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:38.962097+0000', 'last_change': '2024-05-30T17:02:38.962097+0000', 'last_active': '2024-05-30T17:02:38.933146+0000', 'last_peered': '2024-05-30T17:02:38.933146+0000', 'last_clean': '2024-05-30T17:02:38.933146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:38.962097+0000', 'last_undegraded': '2024-05-30T17:02:38.962097+0000', 'last_fullsized': '2024-05-30T17:02:38.962097+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:PG 16.14 is not active+clean 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:38.961654+0000', 'last_change': '2024-05-30T17:02:38.961654+0000', 'last_active': '2024-05-30T17:02:38.933146+0000', 'last_peered': '2024-05-30T17:02:38.933146+0000', 'last_clean': '2024-05-30T17:02:38.933146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:38.961654+0000', 'last_undegraded': '2024-05-30T17:02:38.961654+0000', 'last_fullsized': '2024-05-30T17:02:38.961654+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:PG 17.12 is not active+clean 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.979914+0000', 'last_change': '2024-05-30T17:02:39.979914+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.979914+0000', 'last_undegraded': '2024-05-30T17:02:39.979914+0000', 'last_fullsized': '2024-05-30T17:02:39.979914+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:PG 17.13 is not active+clean 2024-05-30T17:02:46.484 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.978924+0000', 'last_change': '2024-05-30T17:02:39.978924+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.978924+0000', 'last_undegraded': '2024-05-30T17:02:39.978924+0000', 'last_fullsized': '2024-05-30T17:02:39.978924+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:PG 17.11 is not active+clean 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.981115+0000', 'last_change': '2024-05-30T17:02:39.981115+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.981115+0000', 'last_undegraded': '2024-05-30T17:02:39.981115+0000', 'last_fullsized': '2024-05-30T17:02:39.981115+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:PG 16.f is not active+clean 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:38.961297+0000', 'last_change': '2024-05-30T17:02:38.961297+0000', 'last_active': '2024-05-30T17:02:38.933146+0000', 'last_peered': '2024-05-30T17:02:38.933146+0000', 'last_clean': '2024-05-30T17:02:38.933146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:38.961297+0000', 'last_undegraded': '2024-05-30T17:02:38.961297+0000', 'last_fullsized': '2024-05-30T17:02:38.961297+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:PG 17.f is not active+clean 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.990550+0000', 'last_change': '2024-05-30T17:02:39.990550+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.990550+0000', 'last_undegraded': '2024-05-30T17:02:39.990550+0000', 'last_fullsized': '2024-05-30T17:02:39.990550+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:PG 17.b is not active+clean 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:PG 17.9 is not active+clean 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.979476+0000', 'last_change': '2024-05-30T17:02:39.979476+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.979476+0000', 'last_undegraded': '2024-05-30T17:02:39.979476+0000', 'last_fullsized': '2024-05-30T17:02:39.979476+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:PG 16.8 is not active+clean 2024-05-30T17:02:46.485 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:38.961768+0000', 'last_change': '2024-05-30T17:02:38.961768+0000', 'last_active': '2024-05-30T17:02:38.933146+0000', 'last_peered': '2024-05-30T17:02:38.933146+0000', 'last_clean': '2024-05-30T17:02:38.933146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:38.961768+0000', 'last_undegraded': '2024-05-30T17:02:38.961768+0000', 'last_fullsized': '2024-05-30T17:02:38.961768+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:PG 17.6 is not active+clean 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.993019+0000', 'last_change': '2024-05-30T17:02:39.993019+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.993019+0000', 'last_undegraded': '2024-05-30T17:02:39.993019+0000', 'last_fullsized': '2024-05-30T17:02:39.993019+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:PG 17.7 is not active+clean 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:PG 17.4 is not active+clean 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:PG 16.5 is not active+clean 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:38.961468+0000', 'last_change': '2024-05-30T17:02:38.961468+0000', 'last_active': '2024-05-30T17:02:38.933146+0000', 'last_peered': '2024-05-30T17:02:38.933146+0000', 'last_clean': '2024-05-30T17:02:38.933146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:38.961468+0000', 'last_undegraded': '2024-05-30T17:02:38.961468+0000', 'last_fullsized': '2024-05-30T17:02:38.961468+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:PG 17.5 is not active+clean 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.986377+0000', 'last_change': '2024-05-30T17:02:39.986377+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.986377+0000', 'last_undegraded': '2024-05-30T17:02:39.986377+0000', 'last_fullsized': '2024-05-30T17:02:39.986377+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2 is not active+clean 2024-05-30T17:02:46.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.984796+0000', 'last_change': '2024-05-30T17:02:39.984796+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.984796+0000', 'last_undegraded': '2024-05-30T17:02:39.984796+0000', 'last_fullsized': '2024-05-30T17:02:39.984796+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.487 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3 is not active+clean 2024-05-30T17:02:46.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.987678+0000', 'last_change': '2024-05-30T17:02:39.987678+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.987678+0000', 'last_undegraded': '2024-05-30T17:02:39.987678+0000', 'last_fullsized': '2024-05-30T17:02:39.987678+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-30T17:02:46.487 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1 is not active+clean 2024-05-30T17:02:46.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-30T17:02:39.950253+0000', 'last_change': '2024-05-30T17:02:39.950253+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.950253+0000', 'last_undegraded': '2024-05-30T17:02:39.950253+0000', 'last_fullsized': '2024-05-30T17:02:39.950253+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-30T17:02:46.487 INFO:tasks.ceph.ceph_manager.ceph:PG 16.0 is not active+clean 2024-05-30T17:02:46.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:38.961002+0000', 'last_change': '2024-05-30T17:02:38.961002+0000', 'last_active': '2024-05-30T17:02:38.933146+0000', 'last_peered': '2024-05-30T17:02:38.933146+0000', 'last_clean': '2024-05-30T17:02:38.933146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:38.961002+0000', 'last_undegraded': '2024-05-30T17:02:38.961002+0000', 'last_fullsized': '2024-05-30T17:02:38.961002+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:38.933146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-30T17:02:46.487 INFO:tasks.ceph.ceph_manager.ceph:PG 17.10 is not active+clean 2024-05-30T17:02:46.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-05-30T17:02:39.979708+0000', 'last_change': '2024-05-30T17:02:39.979708+0000', 'last_active': '2024-05-30T17:02:39.950253+0000', 'last_peered': '2024-05-30T17:02:39.950253+0000', 'last_clean': '2024-05-30T17:02:39.950253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-30T17:02:39.979708+0000', 'last_undegraded': '2024-05-30T17:02:39.979708+0000', 'last_fullsized': '2024-05-30T17:02:39.979708+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'last_clean_scrub_stamp': '2024-05-30T17:02:39.950253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-30T17:02:46.488 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-30T17:02:46.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.726+0000 7fa311600640 1 Processor -- start 2024-05-30T17:02:46.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.726+0000 7fa311600640 1 -- start start 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.727+0000 7fa311600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c15c6b0 0x7fa30c0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa311600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa30c0ccc30 0x7fa30c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa311600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa30c0d0460 0x7fa30c0d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa311600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa30c0cd1a0 con 0x7fa30c0ccc30 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa311600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa30c0cd480 con 0x7fa30c0d0460 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa311600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa30c0cd760 con 0x7fa30c15c6b0 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa30b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa30c0ccc30 0x7fa30c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa30b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa30c0ccc30 0x7fa30c06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60740/0 (socket says 172.21.15.67:60740) 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa30b400640 1 -- 172.21.15.67:0/3835710195 learned_addr learned my addr 172.21.15.67:0/3835710195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa30b400640 1 -- 172.21.15.67:0/3835710195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c15c6b0 msgr2=0x7fa30c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa30b400640 1 --2- 172.21.15.67:0/3835710195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c15c6b0 0x7fa30c0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa30b400640 1 -- 172.21.15.67:0/3835710195 >> v2:172.21.15.169:3300/0 conn(0x7fa30c0d0460 msgr2=0x7fa30c0d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa30b400640 1 --2- 172.21.15.67:0/3835710195 >> v2:172.21.15.169:3300/0 conn(0x7fa30c0d0460 0x7fa30c0d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.728+0000 7fa30b400640 1 -- 172.21.15.67:0/3835710195 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c0cda40 con 0x7fa30c0ccc30 2024-05-30T17:02:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.729+0000 7fa30b400640 1 --2- 172.21.15.67:0/3835710195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 0x7fa30c06dc90 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0ca7a0 tx=0x7fa2f802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0713240cd8ff92 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.730+0000 7fa30aa00640 1 -- 172.21.15.67:0/3835710195 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8037070 con 0x7fa30c0ccc30 2024-05-30T17:02:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.730+0000 7fa30aa00640 1 -- 172.21.15.67:0/3835710195 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f8032440 con 0x7fa30c0ccc30 2024-05-30T17:02:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.730+0000 7fa30aa00640 1 -- 172.21.15.67:0/3835710195 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8032810 con 0x7fa30c0ccc30 2024-05-30T17:02:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.731+0000 7fa311600640 1 -- 172.21.15.67:0/3835710195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 msgr2=0x7fa30c06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.731+0000 7fa311600640 1 --2- 172.21.15.67:0/3835710195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 0x7fa30c06dc90 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0ca7a0 tx=0x7fa2f802a1d0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.731+0000 7fa311600640 1 -- 172.21.15.67:0/3835710195 shutdown_connections 2024-05-30T17:02:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.731+0000 7fa311600640 1 --2- 172.21.15.67:0/3835710195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 0x7fa30c06dc90 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.731+0000 7fa311600640 1 --2- 172.21.15.67:0/3835710195 >> v2:172.21.15.169:3300/0 conn(0x7fa30c0d0460 0x7fa30c0d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.731+0000 7fa311600640 1 --2- 172.21.15.67:0/3835710195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c15c6b0 0x7fa30c0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.731+0000 7fa311600640 1 -- 172.21.15.67:0/3835710195 >> 172.21.15.67:0/3835710195 conn(0x7fa30c06bae0 msgr2=0x7fa30c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.731+0000 7fa311600640 1 -- 172.21.15.67:0/3835710195 shutdown_connections 2024-05-30T17:02:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.731+0000 7fa311600640 1 -- 172.21.15.67:0/3835710195 wait complete. 2024-05-30T17:02:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.733+0000 7fa311600640 1 Processor -- start 2024-05-30T17:02:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.733+0000 7fa311600640 1 -- start start 2024-05-30T17:02:46.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.734+0000 7fa311600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 0x7fa30c15f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.734+0000 7fa311600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0d0460 0x7fa30c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.734+0000 7fa311600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c15c6b0 0x7fa30c15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.734+0000 7fa311600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c0b4470 con 0x7fa30c0ccc30 2024-05-30T17:02:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.734+0000 7fa311600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c0b42f0 con 0x7fa30c0d0460 2024-05-30T17:02:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.734+0000 7fa311600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa30c06e1d0 con 0x7fa30c15c6b0 2024-05-30T17:02:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.734+0000 7fa30be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 0x7fa30c15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.734+0000 7fa30be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 0x7fa30c15f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60752/0 (socket says 172.21.15.67:60752) 2024-05-30T17:02:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.734+0000 7fa30be00640 1 -- 172.21.15.67:0/1492299601 learned_addr learned my addr 172.21.15.67:0/1492299601 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:46.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa30b400640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0d0460 0x7fa30c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa310c00640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c15c6b0 0x7fa30c15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa30be00640 1 -- 172.21.15.67:0/1492299601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c15c6b0 msgr2=0x7fa30c15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:46.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa30be00640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c15c6b0 0x7fa30c15ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa30be00640 1 -- 172.21.15.67:0/1492299601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0d0460 msgr2=0x7fa30c14e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:46.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa30be00640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0d0460 0x7fa30c14e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:46.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa30be00640 1 -- 172.21.15.67:0/1492299601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c0b0fb0 con 0x7fa30c0ccc30 2024-05-30T17:02:46.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa30be00640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 0x7fa30c15f410 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00b410 tx=0x7fa2fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:46.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa303e00640 1 -- 172.21.15.67:0/1492299601 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc015020 con 0x7fa30c0ccc30 2024-05-30T17:02:46.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa303e00640 1 -- 172.21.15.67:0/1492299601 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2fc011850 con 0x7fa30c0ccc30 2024-05-30T17:02:46.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa303e00640 1 -- 172.21.15.67:0/1492299601 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc011c40 con 0x7fa30c0ccc30 2024-05-30T17:02:46.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c0b1290 con 0x7fa30c0ccc30 2024-05-30T17:02:46.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.735+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c0b1850 con 0x7fa30c0ccc30 2024-05-30T17:02:46.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.737+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa30c0689e0 con 0x7fa30c0ccc30 2024-05-30T17:02:46.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.738+0000 7fa303e00640 1 -- 172.21.15.67:0/1492299601 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fa2fc018020 con 0x7fa30c0ccc30 2024-05-30T17:02:46.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.739+0000 7fa303e00640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2ec06e090 0x7fa2ec070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:46.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.739+0000 7fa303e00640 1 -- 172.21.15.67:0/1492299601 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fa2fc08a900 con 0x7fa30c0ccc30 2024-05-30T17:02:46.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.739+0000 7fa30b400640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2ec06e090 0x7fa2ec070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:46.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.740+0000 7fa30b400640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2ec06e090 0x7fa2ec070550 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8034000 tx=0x7fa2f802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:46.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:46.744+0000 7fa303e00640 1 -- 172.21.15.67:0/1492299601 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2fc01f020 con 0x7fa30c0ccc30 2024-05-30T17:02:47.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.020+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa30c099c70 con 0x7fa30c0ccc30 2024-05-30T17:02:47.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.021+0000 7fa303e00640 1 -- 172.21.15.67:0/1492299601 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1178 (secure 0 0 0) 0x7fa2fc0172c0 con 0x7fa30c0ccc30 2024-05-30T17:02:47.021 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:47.021 INFO:teuthology.orchestra.run.smithi067.stdout:{"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1160,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1717087419,"num_in_osds":8,"osd_in_since":1717087410,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":44},{"state_name":"creating+peering","count":39},{"state_name":"unknown","count":14}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":590950,"bytes_used":1008058368,"bytes_avail":772086054912,"bytes_total":773094113280,"unknown_pgs_ratio":0.14432989060878754,"inactive_pgs_ratio":0.40206184983253479,"write_bytes_sec":183,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":73,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8412}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-05-30T17:01:42.499404+0000","services":{}},"progress_events":{}} 2024-05-30T17:02:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.023+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2ec06e090 msgr2=0x7fa2ec070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.023+0000 7fa311600640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2ec06e090 0x7fa2ec070550 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8034000 tx=0x7fa2f802e000 comp rx=0 tx=0).stop 2024-05-30T17:02:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.023+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 msgr2=0x7fa30c15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.023+0000 7fa311600640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 0x7fa30c15f410 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00b410 tx=0x7fa2fc00b8d0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.024+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 shutdown_connections 2024-05-30T17:02:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.024+0000 7fa311600640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa2ec06e090 0x7fa2ec070550 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.024+0000 7fa311600640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa30c15c6b0 0x7fa30c15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.024+0000 7fa311600640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa30c0d0460 0x7fa30c14e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.024+0000 7fa311600640 1 --2- 172.21.15.67:0/1492299601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa30c0ccc30 0x7fa30c15f410 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.024+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 >> 172.21.15.67:0/1492299601 conn(0x7fa30c06bae0 msgr2=0x7fa30c0cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.024+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 shutdown_connections 2024-05-30T17:02:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.024+0000 7fa311600640 1 -- 172.21.15.67:0/1492299601 wait complete. 2024-05-30T17:02:47.044 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-30T17:02:47.044 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:02:47.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.243+0000 7f0e1fe00640 1 Processor -- start 2024-05-30T17:02:47.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.244+0000 7f0e1fe00640 1 -- start start 2024-05-30T17:02:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.245+0000 7f0e1fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2015e870 0x7f0e200d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.245+0000 7f0e1fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2015ec50 con 0x7f0e200cfa40 2024-05-30T17:02:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.245+0000 7f0e1fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e200cff80 con 0x7f0e200d14d0 2024-05-30T17:02:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.245+0000 7f0e1fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e200d2150 con 0x7f0e2015e870 2024-05-30T17:02:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.246+0000 7f0e1f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0e200cfa40 0x7f0e200d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58644/0 (socket says 172.21.15.67:58644) 2024-05-30T17:02:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.246+0000 7f0e1f400640 1 -- 172.21.15.67:0/1897793445 learned_addr learned my addr 172.21.15.67:0/1897793445 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.246+0000 7f0e1ea00640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2015e870 0x7f0e200d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.246+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4229099967 0 0) 0x7f0e2015ec50 con 0x7f0e200cfa40 2024-05-30T17:02:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.246+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e04003520 con 0x7f0e200cfa40 2024-05-30T17:02:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.246+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3780596526 0 0) 0x7f0e200cff80 con 0x7f0e200d14d0 2024-05-30T17:02:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.246+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e2015ec50 con 0x7f0e200d14d0 2024-05-30T17:02:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3464649625 0 0) 0x7f0e04003520 con 0x7f0e200cfa40 2024-05-30T17:02:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2015e870 msgr2=0x7f0e200d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2015e870 0x7f0e200d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 >> v1:172.21.15.169:6789/0 conn(0x7f0e200d14d0 legacy=0x7f0e200cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e200d0260 con 0x7f0e200cfa40 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f0e18001670 con 0x7f0e200cfa40 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 >> v1:172.21.15.67:6789/0 conn(0x7f0e200cfa40 legacy=0x7f0e200d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e04003b70 0x7f0e04003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2015e870 0x7f0e04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e04004d40 0x7f0e040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0e200d2150 con 0x7f0e04003b70 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0400ba70 con 0x7f0e04004d40 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0400bc20 con 0x7f0e2015e870 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1ea00640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e04003b70 0x7f0e04003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1ea00640 1 -- 172.21.15.67:0/1897793445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2015e870 msgr2=0x7f0e04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1ea00640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2015e870 0x7f0e04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.247+0000 7f0e1ea00640 1 -- 172.21.15.67:0/1897793445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e04004d40 msgr2=0x7f0e040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.248+0000 7f0e1ea00640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e04004d40 0x7f0e040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.248+0000 7f0e1ea00640 1 -- 172.21.15.67:0/1897793445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e04003520 con 0x7f0e04003b70 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.249+0000 7f0e1ea00640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e04003b70 0x7f0e04003f70 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009fc0 tx=0x7f0e04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53006cb6b5c26da3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.249+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10009c10 con 0x7f0e04003b70 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.249+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e10036070 con 0x7f0e04003b70 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.249+0000 7f0e1d600640 1 -- 172.21.15.67:0/1897793445 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1002d070 con 0x7f0e04003b70 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.249+0000 7f0e1fe00640 1 -- 172.21.15.67:0/1897793445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e04003b70 msgr2=0x7f0e04003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.249+0000 7f0e1fe00640 1 --2- 172.21.15.67:0/1897793445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e04003b70 0x7f0e04003f70 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009fc0 tx=0x7f0e04005830 comp rx=0 tx=0).stop 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.249+0000 7f0e1f400640 1 -- 172.21.15.67:0/1897793445 reap_dead start 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.250+0000 7f0e1fe00640 1 -- 172.21.15.67:0/1897793445 shutdown_connections 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.250+0000 7f0e1fe00640 1 -- 172.21.15.67:0/1897793445 >> 172.21.15.67:0/1897793445 conn(0x7f0e2008bfc0 msgr2=0x7f0e200d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.250+0000 7f0e1fe00640 1 -- 172.21.15.67:0/1897793445 shutdown_connections 2024-05-30T17:02:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.250+0000 7f0e1fe00640 1 -- 172.21.15.67:0/1897793445 wait complete. 2024-05-30T17:02:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.251+0000 7f0e1fe00640 1 Processor -- start 2024-05-30T17:02:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.252+0000 7f0e1fe00640 1 -- start start 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.252+0000 7f0e1fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e04003b70 0x7f0e2015b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2014f810 0x7f0e200c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e200ca250 0x7f0e200b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0e200d4fb0 con 0x7f0e200ca250 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e200d0820 con 0x7f0e2014f810 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e200d0f90 con 0x7f0e04003b70 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e200ca250 0x7f0e200b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e200ca250 0x7f0e200b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60762/0 (socket says 172.21.15.67:60762) 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1f400640 1 -- 172.21.15.67:0/3631009302 learned_addr learned my addr 172.21.15.67:0/3631009302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1ea00640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e04003b70 0x7f0e2015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1e000640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2014f810 0x7f0e200c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1f400640 1 -- 172.21.15.67:0/3631009302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e04003b70 msgr2=0x7f0e2015b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1f400640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e04003b70 0x7f0e2015b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1f400640 1 -- 172.21.15.67:0/3631009302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2014f810 msgr2=0x7f0e200c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1f400640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2014f810 0x7f0e200c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.253+0000 7f0e1f400640 1 -- 172.21.15.67:0/3631009302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2015a8f0 con 0x7f0e200ca250 2024-05-30T17:02:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.254+0000 7f0e1f400640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e200ca250 0x7f0e200b1e50 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0e20163400 tx=0x7f0e1800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.254+0000 7f0e17400640 1 -- 172.21.15.67:0/3631009302 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18004bf0 con 0x7f0e200ca250 2024-05-30T17:02:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.254+0000 7f0e17400640 1 -- 172.21.15.67:0/3631009302 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18004d90 con 0x7f0e200ca250 2024-05-30T17:02:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.254+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2015abd0 con 0x7f0e200ca250 2024-05-30T17:02:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.254+0000 7f0e17400640 1 -- 172.21.15.67:0/3631009302 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18007230 con 0x7f0e200ca250 2024-05-30T17:02:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.254+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2015b190 con 0x7f0e200ca250 2024-05-30T17:02:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.255+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e200689e0 con 0x7f0e200ca250 2024-05-30T17:02:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.256+0000 7f0e17400640 1 -- 172.21.15.67:0/3631009302 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f0e180073d0 con 0x7f0e200ca250 2024-05-30T17:02:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.257+0000 7f0e17400640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e0006e090 0x7f0e00070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.257+0000 7f0e17400640 1 -- 172.21.15.67:0/3631009302 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f0e18089ed0 con 0x7f0e200ca250 2024-05-30T17:02:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.258+0000 7f0e1ea00640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e0006e090 0x7f0e00070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.258+0000 7f0e1ea00640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e0006e090 0x7f0e00070550 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0e10035000 tx=0x7f0e1002f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:47.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.262+0000 7f0e17400640 1 -- 172.21.15.67:0/3631009302 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e18059d60 con 0x7f0e200ca250 2024-05-30T17:02:47.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.464+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e2006d9b0 con 0x7f0e0006e090 2024-05-30T17:02:47.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.483+0000 7f0e17400640 1 -- 172.21.15.67:0/3631009302 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252505 (secure 0 0 0) 0x7f0e2006d9b0 con 0x7f0e0006e090 2024-05-30T17:02:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:47.489 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:02:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e0006e090 msgr2=0x7f0e00070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e0006e090 0x7f0e00070550 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0e10035000 tx=0x7f0e1002f000 comp rx=0 tx=0).stop 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e200ca250 msgr2=0x7f0e200b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e200ca250 0x7f0e200b1e50 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0e20163400 tx=0x7f0e1800c940 comp rx=0 tx=0).stop 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 shutdown_connections 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0e0006e090 0x7f0e00070550 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e200ca250 0x7f0e200b1e50 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2014f810 0x7f0e200c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 --2- 172.21.15.67:0/3631009302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e04003b70 0x7f0e2015b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.489+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 >> 172.21.15.67:0/3631009302 conn(0x7f0e2008bfc0 msgr2=0x7f0e2006ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.490+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 shutdown_connections 2024-05-30T17:02:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:47.490+0000 7f0e1fe00640 1 -- 172.21.15.67:0/3631009302 wait complete. 2024-05-30T17:02:47.508 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":683,"stamp":"2024-05-30T17:02:46.523145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1002684,"kb_used_data":12412,"kb_used_omap":12,"kb_used_meta":990131,"kb_avail":753972036,"statfs":{"total":773094113280,"available":772067364864,"internally_reserved":0,"allocated":12709888,"data_stored":6208016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1013894704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.554130"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710331+0000","last_change":"2024-05-30T17:02:41.021236+0000","last_active":"2024-05-30T17:02:42.710331+0000","last_peered":"2024-05-30T17:02:42.710331+0000","last_clean":"2024-05-30T17:02:42.710331+0000","last_became_active":"2024-05-30T17:02:41.020953+0000","last_became_peered":"2024-05-30T17:02:41.020953+0000","last_unstale":"2024-05-30T17:02:42.710331+0000","last_undegraded":"2024-05-30T17:02:42.710331+0000","last_fullsized":"2024-05-30T17:02:42.710331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:08.262939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809486+0000","last_change":"2024-05-30T17:02:41.014728+0000","last_active":"2024-05-30T17:02:43.809486+0000","last_peered":"2024-05-30T17:02:43.809486+0000","last_clean":"2024-05-30T17:02:43.809486+0000","last_became_active":"2024-05-30T17:02:41.014423+0000","last_became_peered":"2024-05-30T17:02:41.014423+0000","last_unstale":"2024-05-30T17:02:43.809486+0000","last_undegraded":"2024-05-30T17:02:43.809486+0000","last_fullsized":"2024-05-30T17:02:43.809486+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:04:02.116807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738383+0000","last_change":"2024-05-30T17:02:41.013698+0000","last_active":"2024-05-30T17:02:43.738383+0000","last_peered":"2024-05-30T17:02:43.738383+0000","last_clean":"2024-05-30T17:02:43.738383+0000","last_became_active":"2024-05-30T17:02:41.013407+0000","last_became_peered":"2024-05-30T17:02:41.013407+0000","last_unstale":"2024-05-30T17:02:43.738383+0000","last_undegraded":"2024-05-30T17:02:43.738383+0000","last_fullsized":"2024-05-30T17:02:43.738383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:56.046859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709151+0000","last_change":"2024-05-30T17:02:41.020938+0000","last_active":"2024-05-30T17:02:42.709151+0000","last_peered":"2024-05-30T17:02:42.709151+0000","last_clean":"2024-05-30T17:02:42.709151+0000","last_became_active":"2024-05-30T17:02:41.017229+0000","last_became_peered":"2024-05-30T17:02:41.017229+0000","last_unstale":"2024-05-30T17:02:42.709151+0000","last_undegraded":"2024-05-30T17:02:42.709151+0000","last_fullsized":"2024-05-30T17:02:42.709151+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:39.295718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009700+0000","last_change":"2024-05-30T17:02:41.006127+0000","last_active":"2024-05-30T17:02:42.009700+0000","last_peered":"2024-05-30T17:02:42.009700+0000","last_clean":"2024-05-30T17:02:42.009700+0000","last_became_active":"2024-05-30T17:02:41.005503+0000","last_became_peered":"2024-05-30T17:02:41.005503+0000","last_unstale":"2024-05-30T17:02:42.009700+0000","last_undegraded":"2024-05-30T17:02:42.009700+0000","last_fullsized":"2024-05-30T17:02:42.009700+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:08.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188550+0000","last_change":"2024-05-30T17:02:41.006264+0000","last_active":"2024-05-30T17:02:42.188550+0000","last_peered":"2024-05-30T17:02:42.188550+0000","last_clean":"2024-05-30T17:02:42.188550+0000","last_became_active":"2024-05-30T17:02:41.005993+0000","last_became_peered":"2024-05-30T17:02:41.005993+0000","last_unstale":"2024-05-30T17:02:42.188550+0000","last_undegraded":"2024-05-30T17:02:42.188550+0000","last_fullsized":"2024-05-30T17:02:42.188550+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:52:18.051272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011985+0000","last_change":"2024-05-30T17:02:41.010886+0000","last_active":"2024-05-30T17:02:42.011985+0000","last_peered":"2024-05-30T17:02:42.011985+0000","last_clean":"2024-05-30T17:02:42.011985+0000","last_became_active":"2024-05-30T17:02:41.007377+0000","last_became_peered":"2024-05-30T17:02:41.007377+0000","last_unstale":"2024-05-30T17:02:42.011985+0000","last_undegraded":"2024-05-30T17:02:42.011985+0000","last_fullsized":"2024-05-30T17:02:42.011985+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:59.071677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738235+0000","last_change":"2024-05-30T17:02:41.014332+0000","last_active":"2024-05-30T17:02:43.738235+0000","last_peered":"2024-05-30T17:02:43.738235+0000","last_clean":"2024-05-30T17:02:43.738235+0000","last_became_active":"2024-05-30T17:02:41.014074+0000","last_became_peered":"2024-05-30T17:02:41.014074+0000","last_unstale":"2024-05-30T17:02:43.738235+0000","last_undegraded":"2024-05-30T17:02:43.738235+0000","last_fullsized":"2024-05-30T17:02:43.738235+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:00:36.629725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710477+0000","last_change":"2024-05-30T17:02:41.012156+0000","last_active":"2024-05-30T17:02:42.710477+0000","last_peered":"2024-05-30T17:02:42.710477+0000","last_clean":"2024-05-30T17:02:42.710477+0000","last_became_active":"2024-05-30T17:02:41.011380+0000","last_became_peered":"2024-05-30T17:02:41.011380+0000","last_unstale":"2024-05-30T17:02:42.710477+0000","last_undegraded":"2024-05-30T17:02:42.710477+0000","last_fullsized":"2024-05-30T17:02:42.710477+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:35:04.378307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710199+0000","last_change":"2024-05-30T17:02:41.021177+0000","last_active":"2024-05-30T17:02:42.710199+0000","last_peered":"2024-05-30T17:02:42.710199+0000","last_clean":"2024-05-30T17:02:42.710199+0000","last_became_active":"2024-05-30T17:02:41.015425+0000","last_became_peered":"2024-05-30T17:02:41.015425+0000","last_unstale":"2024-05-30T17:02:42.710199+0000","last_undegraded":"2024-05-30T17:02:42.710199+0000","last_fullsized":"2024-05-30T17:02:42.710199+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:34.132408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808924+0000","last_change":"2024-05-30T17:02:41.015419+0000","last_active":"2024-05-30T17:02:43.808924+0000","last_peered":"2024-05-30T17:02:43.808924+0000","last_clean":"2024-05-30T17:02:43.808924+0000","last_became_active":"2024-05-30T17:02:41.015134+0000","last_became_peered":"2024-05-30T17:02:41.015134+0000","last_unstale":"2024-05-30T17:02:43.808924+0000","last_undegraded":"2024-05-30T17:02:43.808924+0000","last_fullsized":"2024-05-30T17:02:43.808924+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:28:15.577874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709241+0000","last_change":"2024-05-30T17:02:41.017042+0000","last_active":"2024-05-30T17:02:42.709241+0000","last_peered":"2024-05-30T17:02:42.709241+0000","last_clean":"2024-05-30T17:02:42.709241+0000","last_became_active":"2024-05-30T17:02:41.013921+0000","last_became_peered":"2024-05-30T17:02:41.013921+0000","last_unstale":"2024-05-30T17:02:42.709241+0000","last_undegraded":"2024-05-30T17:02:42.709241+0000","last_fullsized":"2024-05-30T17:02:42.709241+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:56:21.972017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738860+0000","last_change":"2024-05-30T17:02:41.011886+0000","last_active":"2024-05-30T17:02:43.738860+0000","last_peered":"2024-05-30T17:02:43.738860+0000","last_clean":"2024-05-30T17:02:43.738860+0000","last_became_active":"2024-05-30T17:02:41.010296+0000","last_became_peered":"2024-05-30T17:02:41.010296+0000","last_unstale":"2024-05-30T17:02:43.738860+0000","last_undegraded":"2024-05-30T17:02:43.738860+0000","last_fullsized":"2024-05-30T17:02:43.738860+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:20.421009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737964+0000","last_change":"2024-05-30T17:02:41.012428+0000","last_active":"2024-05-30T17:02:43.737964+0000","last_peered":"2024-05-30T17:02:43.737964+0000","last_clean":"2024-05-30T17:02:43.737964+0000","last_became_active":"2024-05-30T17:02:41.011764+0000","last_became_peered":"2024-05-30T17:02:41.011764+0000","last_unstale":"2024-05-30T17:02:43.737964+0000","last_undegraded":"2024-05-30T17:02:43.737964+0000","last_fullsized":"2024-05-30T17:02:43.737964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:11:05.228132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188591+0000","last_change":"2024-05-30T17:02:41.005418+0000","last_active":"2024-05-30T17:02:42.188591+0000","last_peered":"2024-05-30T17:02:42.188591+0000","last_clean":"2024-05-30T17:02:42.188591+0000","last_became_active":"2024-05-30T17:02:41.005083+0000","last_became_peered":"2024-05-30T17:02:41.005083+0000","last_unstale":"2024-05-30T17:02:42.188591+0000","last_undegraded":"2024-05-30T17:02:42.188591+0000","last_fullsized":"2024-05-30T17:02:42.188591+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:32.601564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708052+0000","last_change":"2024-05-30T17:02:41.010971+0000","last_active":"2024-05-30T17:02:42.708052+0000","last_peered":"2024-05-30T17:02:42.708052+0000","last_clean":"2024-05-30T17:02:42.708052+0000","last_became_active":"2024-05-30T17:02:41.010647+0000","last_became_peered":"2024-05-30T17:02:41.010647+0000","last_unstale":"2024-05-30T17:02:42.708052+0000","last_undegraded":"2024-05-30T17:02:42.708052+0000","last_fullsized":"2024-05-30T17:02:42.708052+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:14.515707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809168+0000","last_change":"2024-05-30T17:02:41.012058+0000","last_active":"2024-05-30T17:02:43.809168+0000","last_peered":"2024-05-30T17:02:43.809168+0000","last_clean":"2024-05-30T17:02:43.809168+0000","last_became_active":"2024-05-30T17:02:41.011201+0000","last_became_peered":"2024-05-30T17:02:41.011201+0000","last_unstale":"2024-05-30T17:02:43.809168+0000","last_undegraded":"2024-05-30T17:02:43.809168+0000","last_fullsized":"2024-05-30T17:02:43.809168+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:51.244231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015163+0000","last_change":"2024-05-30T17:02:41.007534+0000","last_active":"2024-05-30T17:02:42.015163+0000","last_peered":"2024-05-30T17:02:42.015163+0000","last_clean":"2024-05-30T17:02:42.015163+0000","last_became_active":"2024-05-30T17:02:41.007238+0000","last_became_peered":"2024-05-30T17:02:41.007238+0000","last_unstale":"2024-05-30T17:02:42.015163+0000","last_undegraded":"2024-05-30T17:02:42.015163+0000","last_fullsized":"2024-05-30T17:02:42.015163+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:28:15.577874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738507+0000","last_change":"2024-05-30T17:02:41.011007+0000","last_active":"2024-05-30T17:02:43.738507+0000","last_peered":"2024-05-30T17:02:43.738507+0000","last_clean":"2024-05-30T17:02:43.738507+0000","last_became_active":"2024-05-30T17:02:41.010322+0000","last_became_peered":"2024-05-30T17:02:41.010322+0000","last_unstale":"2024-05-30T17:02:43.738507+0000","last_undegraded":"2024-05-30T17:02:43.738507+0000","last_fullsized":"2024-05-30T17:02:43.738507+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:40:54.062584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809392+0000","last_change":"2024-05-30T17:02:41.006384+0000","last_active":"2024-05-30T17:02:43.809392+0000","last_peered":"2024-05-30T17:02:43.809392+0000","last_clean":"2024-05-30T17:02:43.809392+0000","last_became_active":"2024-05-30T17:02:41.006075+0000","last_became_peered":"2024-05-30T17:02:41.006075+0000","last_unstale":"2024-05-30T17:02:43.809392+0000","last_undegraded":"2024-05-30T17:02:43.809392+0000","last_fullsized":"2024-05-30T17:02:43.809392+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:08:37.996204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809096+0000","last_change":"2024-05-30T17:02:41.008851+0000","last_active":"2024-05-30T17:02:43.809096+0000","last_peered":"2024-05-30T17:02:43.809096+0000","last_clean":"2024-05-30T17:02:43.809096+0000","last_became_active":"2024-05-30T17:02:41.008042+0000","last_became_peered":"2024-05-30T17:02:41.008042+0000","last_unstale":"2024-05-30T17:02:43.809096+0000","last_undegraded":"2024-05-30T17:02:43.809096+0000","last_fullsized":"2024-05-30T17:02:43.809096+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:16.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006668+0000","last_change":"2024-05-30T17:02:41.006733+0000","last_active":"2024-05-30T17:02:42.006668+0000","last_peered":"2024-05-30T17:02:42.006668+0000","last_clean":"2024-05-30T17:02:42.006668+0000","last_became_active":"2024-05-30T17:02:41.005818+0000","last_became_peered":"2024-05-30T17:02:41.005818+0000","last_unstale":"2024-05-30T17:02:42.006668+0000","last_undegraded":"2024-05-30T17:02:42.006668+0000","last_fullsized":"2024-05-30T17:02:42.006668+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:04:50.852844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808384+0000","last_change":"2024-05-30T17:02:41.004543+0000","last_active":"2024-05-30T17:02:43.808384+0000","last_peered":"2024-05-30T17:02:43.808384+0000","last_clean":"2024-05-30T17:02:43.808384+0000","last_became_active":"2024-05-30T17:02:41.004253+0000","last_became_peered":"2024-05-30T17:02:41.004253+0000","last_unstale":"2024-05-30T17:02:43.808384+0000","last_undegraded":"2024-05-30T17:02:43.808384+0000","last_fullsized":"2024-05-30T17:02:43.808384+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:45:01.496834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006559+0000","last_change":"2024-05-30T17:02:41.012901+0000","last_active":"2024-05-30T17:02:42.006559+0000","last_peered":"2024-05-30T17:02:42.006559+0000","last_clean":"2024-05-30T17:02:42.006559+0000","last_became_active":"2024-05-30T17:02:41.012234+0000","last_became_peered":"2024-05-30T17:02:41.012234+0000","last_unstale":"2024-05-30T17:02:42.006559+0000","last_undegraded":"2024-05-30T17:02:42.006559+0000","last_fullsized":"2024-05-30T17:02:42.006559+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:27:57.491125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809287+0000","last_change":"2024-05-30T17:02:41.011507+0000","last_active":"2024-05-30T17:02:43.809287+0000","last_peered":"2024-05-30T17:02:43.809287+0000","last_clean":"2024-05-30T17:02:43.809287+0000","last_became_active":"2024-05-30T17:02:41.011191+0000","last_became_peered":"2024-05-30T17:02:41.011191+0000","last_unstale":"2024-05-30T17:02:43.809287+0000","last_undegraded":"2024-05-30T17:02:43.809287+0000","last_fullsized":"2024-05-30T17:02:43.809287+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:35:55.671230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710128+0000","last_change":"2024-05-30T17:02:41.009883+0000","last_active":"2024-05-30T17:02:42.710128+0000","last_peered":"2024-05-30T17:02:42.710128+0000","last_clean":"2024-05-30T17:02:42.710128+0000","last_became_active":"2024-05-30T17:02:41.008338+0000","last_became_peered":"2024-05-30T17:02:41.008338+0000","last_unstale":"2024-05-30T17:02:42.710128+0000","last_undegraded":"2024-05-30T17:02:42.710128+0000","last_fullsized":"2024-05-30T17:02:42.710128+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:29:12.257884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.189393+0000","last_change":"2024-05-30T17:02:41.013990+0000","last_active":"2024-05-30T17:02:42.189393+0000","last_peered":"2024-05-30T17:02:42.189393+0000","last_clean":"2024-05-30T17:02:42.189393+0000","last_became_active":"2024-05-30T17:02:41.013699+0000","last_became_peered":"2024-05-30T17:02:41.013699+0000","last_unstale":"2024-05-30T17:02:42.189393+0000","last_undegraded":"2024-05-30T17:02:42.189393+0000","last_fullsized":"2024-05-30T17:02:42.189393+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:10.858887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007163+0000","last_change":"2024-05-30T17:02:41.016363+0000","last_active":"2024-05-30T17:02:42.007163+0000","last_peered":"2024-05-30T17:02:42.007163+0000","last_clean":"2024-05-30T17:02:42.007163+0000","last_became_active":"2024-05-30T17:02:41.016024+0000","last_became_peered":"2024-05-30T17:02:41.016024+0000","last_unstale":"2024-05-30T17:02:42.007163+0000","last_undegraded":"2024-05-30T17:02:42.007163+0000","last_fullsized":"2024-05-30T17:02:42.007163+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:12.804431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808497+0000","last_change":"2024-05-30T17:02:41.009285+0000","last_active":"2024-05-30T17:02:43.808497+0000","last_peered":"2024-05-30T17:02:43.808497+0000","last_clean":"2024-05-30T17:02:43.808497+0000","last_became_active":"2024-05-30T17:02:41.008928+0000","last_became_peered":"2024-05-30T17:02:41.008928+0000","last_unstale":"2024-05-30T17:02:43.808497+0000","last_undegraded":"2024-05-30T17:02:43.808497+0000","last_fullsized":"2024-05-30T17:02:43.808497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:52.817158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012127+0000","last_change":"2024-05-30T17:02:41.011572+0000","last_active":"2024-05-30T17:02:42.012127+0000","last_peered":"2024-05-30T17:02:42.012127+0000","last_clean":"2024-05-30T17:02:42.012127+0000","last_became_active":"2024-05-30T17:02:41.011197+0000","last_became_peered":"2024-05-30T17:02:41.011197+0000","last_unstale":"2024-05-30T17:02:42.012127+0000","last_undegraded":"2024-05-30T17:02:42.012127+0000","last_fullsized":"2024-05-30T17:02:42.012127+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:54.031517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708820+0000","last_change":"2024-05-30T17:02:41.014646+0000","last_active":"2024-05-30T17:02:42.708820+0000","last_peered":"2024-05-30T17:02:42.708820+0000","last_clean":"2024-05-30T17:02:42.708820+0000","last_became_active":"2024-05-30T17:02:41.014284+0000","last_became_peered":"2024-05-30T17:02:41.014284+0000","last_unstale":"2024-05-30T17:02:42.708820+0000","last_undegraded":"2024-05-30T17:02:42.708820+0000","last_fullsized":"2024-05-30T17:02:42.708820+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:15.241290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188844+0000","last_change":"2024-05-30T17:02:41.005765+0000","last_active":"2024-05-30T17:02:42.188844+0000","last_peered":"2024-05-30T17:02:42.188844+0000","last_clean":"2024-05-30T17:02:42.188844+0000","last_became_active":"2024-05-30T17:02:41.004818+0000","last_became_peered":"2024-05-30T17:02:41.004818+0000","last_unstale":"2024-05-30T17:02:42.188844+0000","last_undegraded":"2024-05-30T17:02:42.188844+0000","last_fullsized":"2024-05-30T17:02:42.188844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:32.497197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188893+0000","last_change":"2024-05-30T17:02:40.999105+0000","last_active":"2024-05-30T17:02:42.188893+0000","last_peered":"2024-05-30T17:02:42.188893+0000","last_clean":"2024-05-30T17:02:42.188893+0000","last_became_active":"2024-05-30T17:02:40.998808+0000","last_became_peered":"2024-05-30T17:02:40.998808+0000","last_unstale":"2024-05-30T17:02:42.188893+0000","last_undegraded":"2024-05-30T17:02:42.188893+0000","last_fullsized":"2024-05-30T17:02:42.188893+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:45:59.198214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"115'4","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012308+0000","last_change":"2024-05-30T17:02:39.992222+0000","last_active":"2024-05-30T17:02:42.012308+0000","last_peered":"2024-05-30T17:02:42.012308+0000","last_clean":"2024-05-30T17:02:42.012308+0000","last_became_active":"2024-05-30T17:02:39.991672+0000","last_became_peered":"2024-05-30T17:02:39.991672+0000","last_unstale":"2024-05-30T17:02:42.012308+0000","last_undegraded":"2024-05-30T17:02:42.012308+0000","last_fullsized":"2024-05-30T17:02:42.012308+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:17:06.803976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015760+0000","last_change":"2024-05-30T17:02:41.005694+0000","last_active":"2024-05-30T17:02:42.015760+0000","last_peered":"2024-05-30T17:02:42.015760+0000","last_clean":"2024-05-30T17:02:42.015760+0000","last_became_active":"2024-05-30T17:02:41.005391+0000","last_became_peered":"2024-05-30T17:02:41.005391+0000","last_unstale":"2024-05-30T17:02:42.015760+0000","last_undegraded":"2024-05-30T17:02:42.015760+0000","last_fullsized":"2024-05-30T17:02:42.015760+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:35:55.671230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738709+0000","last_change":"2024-05-30T17:02:40.001257+0000","last_active":"2024-05-30T17:02:43.738709+0000","last_peered":"2024-05-30T17:02:43.738709+0000","last_clean":"2024-05-30T17:02:43.738709+0000","last_became_active":"2024-05-30T17:02:40.000956+0000","last_became_peered":"2024-05-30T17:02:40.000956+0000","last_unstale":"2024-05-30T17:02:43.738709+0000","last_undegraded":"2024-05-30T17:02:43.738709+0000","last_fullsized":"2024-05-30T17:02:43.738709+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:52.599362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.189215+0000","last_change":"2024-05-30T17:02:41.007480+0000","last_active":"2024-05-30T17:02:42.189215+0000","last_peered":"2024-05-30T17:02:42.189215+0000","last_clean":"2024-05-30T17:02:42.189215+0000","last_became_active":"2024-05-30T17:02:41.004878+0000","last_became_peered":"2024-05-30T17:02:41.004878+0000","last_unstale":"2024-05-30T17:02:42.189215+0000","last_undegraded":"2024-05-30T17:02:42.189215+0000","last_fullsized":"2024-05-30T17:02:42.189215+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:25:21.801488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.010407+0000","last_change":"2024-05-30T17:02:40.004186+0000","last_active":"2024-05-30T17:02:42.010407+0000","last_peered":"2024-05-30T17:02:42.010407+0000","last_clean":"2024-05-30T17:02:42.010407+0000","last_became_active":"2024-05-30T17:02:40.003831+0000","last_became_peered":"2024-05-30T17:02:40.003831+0000","last_unstale":"2024-05-30T17:02:42.010407+0000","last_undegraded":"2024-05-30T17:02:42.010407+0000","last_fullsized":"2024-05-30T17:02:42.010407+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:09.841780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738822+0000","last_change":"2024-05-30T17:02:41.011197+0000","last_active":"2024-05-30T17:02:43.738822+0000","last_peered":"2024-05-30T17:02:43.738822+0000","last_clean":"2024-05-30T17:02:43.738822+0000","last_became_active":"2024-05-30T17:02:41.010742+0000","last_became_peered":"2024-05-30T17:02:41.010742+0000","last_unstale":"2024-05-30T17:02:43.738822+0000","last_undegraded":"2024-05-30T17:02:43.738822+0000","last_fullsized":"2024-05-30T17:02:43.738822+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:46:14.746408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.010031+0000","last_change":"2024-05-30T17:02:39.999367+0000","last_active":"2024-05-30T17:02:42.010031+0000","last_peered":"2024-05-30T17:02:42.010031+0000","last_clean":"2024-05-30T17:02:42.010031+0000","last_became_active":"2024-05-30T17:02:39.998521+0000","last_became_peered":"2024-05-30T17:02:39.998521+0000","last_unstale":"2024-05-30T17:02:42.010031+0000","last_undegraded":"2024-05-30T17:02:42.010031+0000","last_fullsized":"2024-05-30T17:02:42.010031+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:34.293069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708723+0000","last_change":"2024-05-30T17:02:41.012269+0000","last_active":"2024-05-30T17:02:42.708723+0000","last_peered":"2024-05-30T17:02:42.708723+0000","last_clean":"2024-05-30T17:02:42.708723+0000","last_became_active":"2024-05-30T17:02:41.011965+0000","last_became_peered":"2024-05-30T17:02:41.011965+0000","last_unstale":"2024-05-30T17:02:42.708723+0000","last_undegraded":"2024-05-30T17:02:42.708723+0000","last_fullsized":"2024-05-30T17:02:42.708723+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:28:07.302726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012030+0000","last_change":"2024-05-30T17:02:39.994746+0000","last_active":"2024-05-30T17:02:42.012030+0000","last_peered":"2024-05-30T17:02:42.012030+0000","last_clean":"2024-05-30T17:02:42.012030+0000","last_became_active":"2024-05-30T17:02:39.994412+0000","last_became_peered":"2024-05-30T17:02:39.994412+0000","last_unstale":"2024-05-30T17:02:42.012030+0000","last_undegraded":"2024-05-30T17:02:42.012030+0000","last_fullsized":"2024-05-30T17:02:42.012030+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:01:23.895654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808513+0000","last_change":"2024-05-30T17:02:41.014804+0000","last_active":"2024-05-30T17:02:43.808513+0000","last_peered":"2024-05-30T17:02:43.808513+0000","last_clean":"2024-05-30T17:02:43.808513+0000","last_became_active":"2024-05-30T17:02:41.014498+0000","last_became_peered":"2024-05-30T17:02:41.014498+0000","last_unstale":"2024-05-30T17:02:43.808513+0000","last_undegraded":"2024-05-30T17:02:43.808513+0000","last_fullsized":"2024-05-30T17:02:43.808513+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:22.624806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007547+0000","last_change":"2024-05-30T17:02:40.002354+0000","last_active":"2024-05-30T17:02:42.007547+0000","last_peered":"2024-05-30T17:02:42.007547+0000","last_clean":"2024-05-30T17:02:42.007547+0000","last_became_active":"2024-05-30T17:02:40.002062+0000","last_became_peered":"2024-05-30T17:02:40.002062+0000","last_unstale":"2024-05-30T17:02:42.007547+0000","last_undegraded":"2024-05-30T17:02:42.007547+0000","last_fullsized":"2024-05-30T17:02:42.007547+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:28:14.560768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708173+0000","last_change":"2024-05-30T17:02:41.008592+0000","last_active":"2024-05-30T17:02:42.708173+0000","last_peered":"2024-05-30T17:02:42.708173+0000","last_clean":"2024-05-30T17:02:42.708173+0000","last_became_active":"2024-05-30T17:02:41.008270+0000","last_became_peered":"2024-05-30T17:02:41.008270+0000","last_unstale":"2024-05-30T17:02:42.708173+0000","last_undegraded":"2024-05-30T17:02:42.708173+0000","last_fullsized":"2024-05-30T17:02:42.708173+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:35.433137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808933+0000","last_change":"2024-05-30T17:02:39.996962+0000","last_active":"2024-05-30T17:02:43.808933+0000","last_peered":"2024-05-30T17:02:43.808933+0000","last_clean":"2024-05-30T17:02:43.808933+0000","last_became_active":"2024-05-30T17:02:39.996677+0000","last_became_peered":"2024-05-30T17:02:39.996677+0000","last_unstale":"2024-05-30T17:02:43.808933+0000","last_undegraded":"2024-05-30T17:02:43.808933+0000","last_fullsized":"2024-05-30T17:02:43.808933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:28:38.120481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012752+0000","last_change":"2024-05-30T17:02:41.010582+0000","last_active":"2024-05-30T17:02:42.012752+0000","last_peered":"2024-05-30T17:02:42.012752+0000","last_clean":"2024-05-30T17:02:42.012752+0000","last_became_active":"2024-05-30T17:02:41.006958+0000","last_became_peered":"2024-05-30T17:02:41.006958+0000","last_unstale":"2024-05-30T17:02:42.012752+0000","last_undegraded":"2024-05-30T17:02:42.012752+0000","last_fullsized":"2024-05-30T17:02:42.012752+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:42:06.872806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708333+0000","last_change":"2024-05-30T17:02:39.996651+0000","last_active":"2024-05-30T17:02:42.708333+0000","last_peered":"2024-05-30T17:02:42.708333+0000","last_clean":"2024-05-30T17:02:42.708333+0000","last_became_active":"2024-05-30T17:02:39.996364+0000","last_became_peered":"2024-05-30T17:02:39.996364+0000","last_unstale":"2024-05-30T17:02:42.708333+0000","last_undegraded":"2024-05-30T17:02:42.708333+0000","last_fullsized":"2024-05-30T17:02:42.708333+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:25.128687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012844+0000","last_change":"2024-05-30T17:02:40.999926+0000","last_active":"2024-05-30T17:02:42.012844+0000","last_peered":"2024-05-30T17:02:42.012844+0000","last_clean":"2024-05-30T17:02:42.012844+0000","last_became_active":"2024-05-30T17:02:40.997094+0000","last_became_peered":"2024-05-30T17:02:40.997094+0000","last_unstale":"2024-05-30T17:02:42.012844+0000","last_undegraded":"2024-05-30T17:02:42.012844+0000","last_fullsized":"2024-05-30T17:02:42.012844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:15:52.585750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.010215+0000","last_change":"2024-05-30T17:02:40.000501+0000","last_active":"2024-05-30T17:02:42.010215+0000","last_peered":"2024-05-30T17:02:42.010215+0000","last_clean":"2024-05-30T17:02:42.010215+0000","last_became_active":"2024-05-30T17:02:39.999369+0000","last_became_peered":"2024-05-30T17:02:39.999369+0000","last_unstale":"2024-05-30T17:02:42.010215+0000","last_undegraded":"2024-05-30T17:02:42.010215+0000","last_fullsized":"2024-05-30T17:02:42.010215+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:02.236301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015857+0000","last_change":"2024-05-30T17:02:41.024527+0000","last_active":"2024-05-30T17:02:42.015857+0000","last_peered":"2024-05-30T17:02:42.015857+0000","last_clean":"2024-05-30T17:02:42.015857+0000","last_became_active":"2024-05-30T17:02:41.024226+0000","last_became_peered":"2024-05-30T17:02:41.024226+0000","last_unstale":"2024-05-30T17:02:42.015857+0000","last_undegraded":"2024-05-30T17:02:42.015857+0000","last_fullsized":"2024-05-30T17:02:42.015857+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:08:37.996204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006759+0000","last_change":"2024-05-30T17:02:40.002597+0000","last_active":"2024-05-30T17:02:42.006759+0000","last_peered":"2024-05-30T17:02:42.006759+0000","last_clean":"2024-05-30T17:02:42.006759+0000","last_became_active":"2024-05-30T17:02:40.002351+0000","last_became_peered":"2024-05-30T17:02:40.002351+0000","last_unstale":"2024-05-30T17:02:42.006759+0000","last_undegraded":"2024-05-30T17:02:42.006759+0000","last_fullsized":"2024-05-30T17:02:42.006759+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:15.687359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007338+0000","last_change":"2024-05-30T17:02:41.010196+0000","last_active":"2024-05-30T17:02:42.007338+0000","last_peered":"2024-05-30T17:02:42.007338+0000","last_clean":"2024-05-30T17:02:42.007338+0000","last_became_active":"2024-05-30T17:02:41.009903+0000","last_became_peered":"2024-05-30T17:02:41.009903+0000","last_unstale":"2024-05-30T17:02:42.007338+0000","last_undegraded":"2024-05-30T17:02:42.007338+0000","last_fullsized":"2024-05-30T17:02:42.007338+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:24.744651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"115'3","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709331+0000","last_change":"2024-05-30T17:02:39.999796+0000","last_active":"2024-05-30T17:02:42.709331+0000","last_peered":"2024-05-30T17:02:42.709331+0000","last_clean":"2024-05-30T17:02:42.709331+0000","last_became_active":"2024-05-30T17:02:39.999375+0000","last_became_peered":"2024-05-30T17:02:39.999375+0000","last_unstale":"2024-05-30T17:02:42.709331+0000","last_undegraded":"2024-05-30T17:02:42.709331+0000","last_fullsized":"2024-05-30T17:02:42.709331+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:11.895313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012385+0000","last_change":"2024-05-30T17:02:41.003790+0000","last_active":"2024-05-30T17:02:42.012385+0000","last_peered":"2024-05-30T17:02:42.012385+0000","last_clean":"2024-05-30T17:02:42.012385+0000","last_became_active":"2024-05-30T17:02:41.003476+0000","last_became_peered":"2024-05-30T17:02:41.003476+0000","last_unstale":"2024-05-30T17:02:42.012385+0000","last_undegraded":"2024-05-30T17:02:42.012385+0000","last_fullsized":"2024-05-30T17:02:42.012385+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:19.156205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"115'6","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709596+0000","last_change":"2024-05-30T17:02:40.003867+0000","last_active":"2024-05-30T17:02:42.709596+0000","last_peered":"2024-05-30T17:02:42.709596+0000","last_clean":"2024-05-30T17:02:42.709596+0000","last_became_active":"2024-05-30T17:02:39.996721+0000","last_became_peered":"2024-05-30T17:02:39.996721+0000","last_unstale":"2024-05-30T17:02:42.709596+0000","last_undegraded":"2024-05-30T17:02:42.709596+0000","last_fullsized":"2024-05-30T17:02:42.709596+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:08:31.053590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007408+0000","last_change":"2024-05-30T17:02:40.998234+0000","last_active":"2024-05-30T17:02:42.007408+0000","last_peered":"2024-05-30T17:02:42.007408+0000","last_clean":"2024-05-30T17:02:42.007408+0000","last_became_active":"2024-05-30T17:02:40.997241+0000","last_became_peered":"2024-05-30T17:02:40.997241+0000","last_unstale":"2024-05-30T17:02:42.007408+0000","last_undegraded":"2024-05-30T17:02:42.007408+0000","last_fullsized":"2024-05-30T17:02:42.007408+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:24:21.891014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.189204+0000","last_change":"2024-05-30T17:02:39.993748+0000","last_active":"2024-05-30T17:02:42.189204+0000","last_peered":"2024-05-30T17:02:42.189204+0000","last_clean":"2024-05-30T17:02:42.189204+0000","last_became_active":"2024-05-30T17:02:39.993315+0000","last_became_peered":"2024-05-30T17:02:39.993315+0000","last_unstale":"2024-05-30T17:02:42.189204+0000","last_undegraded":"2024-05-30T17:02:42.189204+0000","last_fullsized":"2024-05-30T17:02:42.189204+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:58.412522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007060+0000","last_change":"2024-05-30T17:02:41.003650+0000","last_active":"2024-05-30T17:02:42.007060+0000","last_peered":"2024-05-30T17:02:42.007060+0000","last_clean":"2024-05-30T17:02:42.007060+0000","last_became_active":"2024-05-30T17:02:41.002514+0000","last_became_peered":"2024-05-30T17:02:41.002514+0000","last_unstale":"2024-05-30T17:02:42.007060+0000","last_undegraded":"2024-05-30T17:02:42.007060+0000","last_fullsized":"2024-05-30T17:02:42.007060+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:41.488964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.014933+0000","last_change":"2024-05-30T17:02:40.006813+0000","last_active":"2024-05-30T17:02:42.014933+0000","last_peered":"2024-05-30T17:02:42.014933+0000","last_clean":"2024-05-30T17:02:42.014933+0000","last_became_active":"2024-05-30T17:02:40.006466+0000","last_became_peered":"2024-05-30T17:02:40.006466+0000","last_unstale":"2024-05-30T17:02:42.014933+0000","last_undegraded":"2024-05-30T17:02:42.014933+0000","last_fullsized":"2024-05-30T17:02:42.014933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:07.267691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006829+0000","last_change":"2024-05-30T17:02:41.000175+0000","last_active":"2024-05-30T17:02:42.006829+0000","last_peered":"2024-05-30T17:02:42.006829+0000","last_clean":"2024-05-30T17:02:42.006829+0000","last_became_active":"2024-05-30T17:02:40.999872+0000","last_became_peered":"2024-05-30T17:02:40.999872+0000","last_unstale":"2024-05-30T17:02:42.006829+0000","last_undegraded":"2024-05-30T17:02:42.006829+0000","last_fullsized":"2024-05-30T17:02:42.006829+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:36.737438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"115'8","reported_seq":22,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009351+0000","last_change":"2024-05-30T17:02:39.998093+0000","last_active":"2024-05-30T17:02:42.009351+0000","last_peered":"2024-05-30T17:02:42.009351+0000","last_clean":"2024-05-30T17:02:42.009351+0000","last_became_active":"2024-05-30T17:02:39.997282+0000","last_became_peered":"2024-05-30T17:02:39.997282+0000","last_unstale":"2024-05-30T17:02:42.009351+0000","last_undegraded":"2024-05-30T17:02:42.009351+0000","last_fullsized":"2024-05-30T17:02:42.009351+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:31.480091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.010013+0000","last_change":"2024-05-30T17:02:41.009258+0000","last_active":"2024-05-30T17:02:42.010013+0000","last_peered":"2024-05-30T17:02:42.010013+0000","last_clean":"2024-05-30T17:02:42.010013+0000","last_became_active":"2024-05-30T17:02:41.008966+0000","last_became_peered":"2024-05-30T17:02:41.008966+0000","last_unstale":"2024-05-30T17:02:42.010013+0000","last_undegraded":"2024-05-30T17:02:42.010013+0000","last_fullsized":"2024-05-30T17:02:42.010013+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:20.043628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710631+0000","last_change":"2024-05-30T17:02:40.002688+0000","last_active":"2024-05-30T17:02:42.710631+0000","last_peered":"2024-05-30T17:02:42.710631+0000","last_clean":"2024-05-30T17:02:42.710631+0000","last_became_active":"2024-05-30T17:02:40.000751+0000","last_became_peered":"2024-05-30T17:02:40.000751+0000","last_unstale":"2024-05-30T17:02:42.710631+0000","last_undegraded":"2024-05-30T17:02:42.710631+0000","last_fullsized":"2024-05-30T17:02:42.710631+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:34.156284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188976+0000","last_change":"2024-05-30T17:02:41.023475+0000","last_active":"2024-05-30T17:02:42.188976+0000","last_peered":"2024-05-30T17:02:42.188976+0000","last_clean":"2024-05-30T17:02:42.188976+0000","last_became_active":"2024-05-30T17:02:41.023183+0000","last_became_peered":"2024-05-30T17:02:41.023183+0000","last_unstale":"2024-05-30T17:02:42.188976+0000","last_undegraded":"2024-05-30T17:02:42.188976+0000","last_fullsized":"2024-05-30T17:02:42.188976+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:02.573961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009836+0000","last_change":"2024-05-30T17:02:39.995669+0000","last_active":"2024-05-30T17:02:42.009836+0000","last_peered":"2024-05-30T17:02:42.009836+0000","last_clean":"2024-05-30T17:02:42.009836+0000","last_became_active":"2024-05-30T17:02:39.995397+0000","last_became_peered":"2024-05-30T17:02:39.995397+0000","last_unstale":"2024-05-30T17:02:42.009836+0000","last_undegraded":"2024-05-30T17:02:42.009836+0000","last_fullsized":"2024-05-30T17:02:42.009836+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:29.373763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738221+0000","last_change":"2024-05-30T17:02:41.022727+0000","last_active":"2024-05-30T17:02:43.738221+0000","last_peered":"2024-05-30T17:02:43.738221+0000","last_clean":"2024-05-30T17:02:43.738221+0000","last_became_active":"2024-05-30T17:02:41.022446+0000","last_became_peered":"2024-05-30T17:02:41.022446+0000","last_unstale":"2024-05-30T17:02:43.738221+0000","last_undegraded":"2024-05-30T17:02:43.738221+0000","last_fullsized":"2024-05-30T17:02:43.738221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:08.319197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015437+0000","last_change":"2024-05-30T17:02:39.998860+0000","last_active":"2024-05-30T17:02:42.015437+0000","last_peered":"2024-05-30T17:02:42.015437+0000","last_clean":"2024-05-30T17:02:42.015437+0000","last_became_active":"2024-05-30T17:02:39.997624+0000","last_became_peered":"2024-05-30T17:02:39.997624+0000","last_unstale":"2024-05-30T17:02:42.015437+0000","last_undegraded":"2024-05-30T17:02:42.015437+0000","last_fullsized":"2024-05-30T17:02:42.015437+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:59:57.588303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738864+0000","last_change":"2024-05-30T17:02:41.017247+0000","last_active":"2024-05-30T17:02:43.738864+0000","last_peered":"2024-05-30T17:02:43.738864+0000","last_clean":"2024-05-30T17:02:43.738864+0000","last_became_active":"2024-05-30T17:02:41.016943+0000","last_became_peered":"2024-05-30T17:02:41.016943+0000","last_unstale":"2024-05-30T17:02:43.738864+0000","last_undegraded":"2024-05-30T17:02:43.738864+0000","last_fullsized":"2024-05-30T17:02:43.738864+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:14.515707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011769+0000","last_change":"2024-05-30T17:02:40.001321+0000","last_active":"2024-05-30T17:02:42.011769+0000","last_peered":"2024-05-30T17:02:42.011769+0000","last_clean":"2024-05-30T17:02:42.011769+0000","last_became_active":"2024-05-30T17:02:40.000949+0000","last_became_peered":"2024-05-30T17:02:40.000949+0000","last_unstale":"2024-05-30T17:02:42.011769+0000","last_undegraded":"2024-05-30T17:02:42.011769+0000","last_fullsized":"2024-05-30T17:02:42.011769+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:11:12.879603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737942+0000","last_change":"2024-05-30T17:02:41.012668+0000","last_active":"2024-05-30T17:02:43.737942+0000","last_peered":"2024-05-30T17:02:43.737942+0000","last_clean":"2024-05-30T17:02:43.737942+0000","last_became_active":"2024-05-30T17:02:41.012148+0000","last_became_peered":"2024-05-30T17:02:41.012148+0000","last_unstale":"2024-05-30T17:02:43.737942+0000","last_undegraded":"2024-05-30T17:02:43.737942+0000","last_fullsized":"2024-05-30T17:02:43.737942+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:51:48.462366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.189541+0000","last_change":"2024-05-30T17:02:39.992411+0000","last_active":"2024-05-30T17:02:42.189541+0000","last_peered":"2024-05-30T17:02:42.189541+0000","last_clean":"2024-05-30T17:02:42.189541+0000","last_became_active":"2024-05-30T17:02:39.989335+0000","last_became_peered":"2024-05-30T17:02:39.989335+0000","last_unstale":"2024-05-30T17:02:42.189541+0000","last_undegraded":"2024-05-30T17:02:42.189541+0000","last_fullsized":"2024-05-30T17:02:42.189541+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:35.765479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708659+0000","last_change":"2024-05-30T17:02:41.007384+0000","last_active":"2024-05-30T17:02:42.708659+0000","last_peered":"2024-05-30T17:02:42.708659+0000","last_clean":"2024-05-30T17:02:42.708659+0000","last_became_active":"2024-05-30T17:02:41.007093+0000","last_became_peered":"2024-05-30T17:02:41.007093+0000","last_unstale":"2024-05-30T17:02:42.708659+0000","last_undegraded":"2024-05-30T17:02:42.708659+0000","last_fullsized":"2024-05-30T17:02:42.708659+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:07.370463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009199+0000","last_change":"2024-05-30T17:02:39.993424+0000","last_active":"2024-05-30T17:02:42.009199+0000","last_peered":"2024-05-30T17:02:42.009199+0000","last_clean":"2024-05-30T17:02:42.009199+0000","last_became_active":"2024-05-30T17:02:39.993119+0000","last_became_peered":"2024-05-30T17:02:39.993119+0000","last_unstale":"2024-05-30T17:02:42.009199+0000","last_undegraded":"2024-05-30T17:02:42.009199+0000","last_fullsized":"2024-05-30T17:02:42.009199+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:25:20.784382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737939+0000","last_change":"2024-05-30T17:02:41.008813+0000","last_active":"2024-05-30T17:02:43.737939+0000","last_peered":"2024-05-30T17:02:43.737939+0000","last_clean":"2024-05-30T17:02:43.737939+0000","last_became_active":"2024-05-30T17:02:41.008199+0000","last_became_peered":"2024-05-30T17:02:41.008199+0000","last_unstale":"2024-05-30T17:02:43.737939+0000","last_undegraded":"2024-05-30T17:02:43.737939+0000","last_fullsized":"2024-05-30T17:02:43.737939+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:34:19.448690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188651+0000","last_change":"2024-05-30T17:02:39.996513+0000","last_active":"2024-05-30T17:02:42.188651+0000","last_peered":"2024-05-30T17:02:42.188651+0000","last_clean":"2024-05-30T17:02:42.188651+0000","last_became_active":"2024-05-30T17:02:39.995747+0000","last_became_peered":"2024-05-30T17:02:39.995747+0000","last_unstale":"2024-05-30T17:02:42.188651+0000","last_undegraded":"2024-05-30T17:02:42.188651+0000","last_fullsized":"2024-05-30T17:02:42.188651+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:26:38.544781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007292+0000","last_change":"2024-05-30T17:02:41.016390+0000","last_active":"2024-05-30T17:02:42.007292+0000","last_peered":"2024-05-30T17:02:42.007292+0000","last_clean":"2024-05-30T17:02:42.007292+0000","last_became_active":"2024-05-30T17:02:41.015267+0000","last_became_peered":"2024-05-30T17:02:41.015267+0000","last_unstale":"2024-05-30T17:02:42.007292+0000","last_undegraded":"2024-05-30T17:02:42.007292+0000","last_fullsized":"2024-05-30T17:02:42.007292+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:54:52.400582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708427+0000","last_change":"2024-05-30T17:02:40.006608+0000","last_active":"2024-05-30T17:02:42.708427+0000","last_peered":"2024-05-30T17:02:42.708427+0000","last_clean":"2024-05-30T17:02:42.708427+0000","last_became_active":"2024-05-30T17:02:40.005867+0000","last_became_peered":"2024-05-30T17:02:40.005867+0000","last_unstale":"2024-05-30T17:02:42.708427+0000","last_undegraded":"2024-05-30T17:02:42.708427+0000","last_fullsized":"2024-05-30T17:02:42.708427+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:35.438692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015333+0000","last_change":"2024-05-30T17:02:41.014137+0000","last_active":"2024-05-30T17:02:42.015333+0000","last_peered":"2024-05-30T17:02:42.015333+0000","last_clean":"2024-05-30T17:02:42.015333+0000","last_became_active":"2024-05-30T17:02:41.013859+0000","last_became_peered":"2024-05-30T17:02:41.013859+0000","last_unstale":"2024-05-30T17:02:42.015333+0000","last_undegraded":"2024-05-30T17:02:42.015333+0000","last_fullsized":"2024-05-30T17:02:42.015333+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:16.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007027+0000","last_change":"2024-05-30T17:02:39.997156+0000","last_active":"2024-05-30T17:02:42.007027+0000","last_peered":"2024-05-30T17:02:42.007027+0000","last_clean":"2024-05-30T17:02:42.007027+0000","last_became_active":"2024-05-30T17:02:39.996869+0000","last_became_peered":"2024-05-30T17:02:39.996869+0000","last_unstale":"2024-05-30T17:02:42.007027+0000","last_undegraded":"2024-05-30T17:02:42.007027+0000","last_fullsized":"2024-05-30T17:02:42.007027+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:21.607700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710551+0000","last_change":"2024-05-30T17:02:41.020925+0000","last_active":"2024-05-30T17:02:42.710551+0000","last_peered":"2024-05-30T17:02:42.710551+0000","last_clean":"2024-05-30T17:02:42.710551+0000","last_became_active":"2024-05-30T17:02:41.014575+0000","last_became_peered":"2024-05-30T17:02:41.014575+0000","last_unstale":"2024-05-30T17:02:42.710551+0000","last_undegraded":"2024-05-30T17:02:42.710551+0000","last_fullsized":"2024-05-30T17:02:42.710551+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:01:15.142676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"116'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015712+0000","last_change":"2024-05-30T17:02:40.000552+0000","last_active":"2024-05-30T17:02:42.015712+0000","last_peered":"2024-05-30T17:02:42.015712+0000","last_clean":"2024-05-30T17:02:42.015712+0000","last_became_active":"2024-05-30T17:02:40.000240+0000","last_became_peered":"2024-05-30T17:02:40.000240+0000","last_unstale":"2024-05-30T17:02:42.015712+0000","last_undegraded":"2024-05-30T17:02:42.015712+0000","last_fullsized":"2024-05-30T17:02:42.015712+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:28:05.119785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709580+0000","last_change":"2024-05-30T17:02:41.015414+0000","last_active":"2024-05-30T17:02:42.709580+0000","last_peered":"2024-05-30T17:02:42.709580+0000","last_clean":"2024-05-30T17:02:42.709580+0000","last_became_active":"2024-05-30T17:02:41.013542+0000","last_became_peered":"2024-05-30T17:02:41.013542+0000","last_unstale":"2024-05-30T17:02:42.709580+0000","last_undegraded":"2024-05-30T17:02:42.709580+0000","last_fullsized":"2024-05-30T17:02:42.709580+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:24:22.067418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709431+0000","last_change":"2024-05-30T17:02:40.004551+0000","last_active":"2024-05-30T17:02:42.709431+0000","last_peered":"2024-05-30T17:02:42.709431+0000","last_clean":"2024-05-30T17:02:42.709431+0000","last_became_active":"2024-05-30T17:02:40.004260+0000","last_became_peered":"2024-05-30T17:02:40.004260+0000","last_unstale":"2024-05-30T17:02:42.709431+0000","last_undegraded":"2024-05-30T17:02:42.709431+0000","last_fullsized":"2024-05-30T17:02:42.709431+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:05:51.093557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015455+0000","last_change":"2024-05-30T17:02:41.017162+0000","last_active":"2024-05-30T17:02:42.015455+0000","last_peered":"2024-05-30T17:02:42.015455+0000","last_clean":"2024-05-30T17:02:42.015455+0000","last_became_active":"2024-05-30T17:02:41.016552+0000","last_became_peered":"2024-05-30T17:02:41.016552+0000","last_unstale":"2024-05-30T17:02:42.015455+0000","last_undegraded":"2024-05-30T17:02:42.015455+0000","last_fullsized":"2024-05-30T17:02:42.015455+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:51.244231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007435+0000","last_change":"2024-05-30T17:02:39.997337+0000","last_active":"2024-05-30T17:02:42.007435+0000","last_peered":"2024-05-30T17:02:42.007435+0000","last_clean":"2024-05-30T17:02:42.007435+0000","last_became_active":"2024-05-30T17:02:39.996998+0000","last_became_peered":"2024-05-30T17:02:39.996998+0000","last_unstale":"2024-05-30T17:02:42.007435+0000","last_undegraded":"2024-05-30T17:02:42.007435+0000","last_fullsized":"2024-05-30T17:02:42.007435+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:50.227125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808410+0000","last_change":"2024-05-30T17:02:41.006346+0000","last_active":"2024-05-30T17:02:43.808410+0000","last_peered":"2024-05-30T17:02:43.808410+0000","last_clean":"2024-05-30T17:02:43.808410+0000","last_became_active":"2024-05-30T17:02:41.006009+0000","last_became_peered":"2024-05-30T17:02:41.006009+0000","last_unstale":"2024-05-30T17:02:43.808410+0000","last_undegraded":"2024-05-30T17:02:43.808410+0000","last_fullsized":"2024-05-30T17:02:43.808410+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:34:40.576629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012323+0000","last_change":"2024-05-30T17:02:39.996922+0000","last_active":"2024-05-30T17:02:42.012323+0000","last_peered":"2024-05-30T17:02:42.012323+0000","last_clean":"2024-05-30T17:02:42.012323+0000","last_became_active":"2024-05-30T17:02:39.996561+0000","last_became_peered":"2024-05-30T17:02:39.996561+0000","last_unstale":"2024-05-30T17:02:42.012323+0000","last_undegraded":"2024-05-30T17:02:42.012323+0000","last_fullsized":"2024-05-30T17:02:42.012323+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:16:39.184990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809408+0000","last_change":"2024-05-30T17:02:41.008494+0000","last_active":"2024-05-30T17:02:43.809408+0000","last_peered":"2024-05-30T17:02:43.809408+0000","last_clean":"2024-05-30T17:02:43.809408+0000","last_became_active":"2024-05-30T17:02:41.005828+0000","last_became_peered":"2024-05-30T17:02:41.005828+0000","last_unstale":"2024-05-30T17:02:43.809408+0000","last_undegraded":"2024-05-30T17:02:43.809408+0000","last_fullsized":"2024-05-30T17:02:43.809408+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:46.955577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808434+0000","last_change":"2024-05-30T17:02:39.999916+0000","last_active":"2024-05-30T17:02:43.808434+0000","last_peered":"2024-05-30T17:02:43.808434+0000","last_clean":"2024-05-30T17:02:43.808434+0000","last_became_active":"2024-05-30T17:02:39.999588+0000","last_became_peered":"2024-05-30T17:02:39.999588+0000","last_unstale":"2024-05-30T17:02:43.808434+0000","last_undegraded":"2024-05-30T17:02:43.808434+0000","last_fullsized":"2024-05-30T17:02:43.808434+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:58:40.604351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738381+0000","last_change":"2024-05-30T17:02:41.009038+0000","last_active":"2024-05-30T17:02:43.738381+0000","last_peered":"2024-05-30T17:02:43.738381+0000","last_clean":"2024-05-30T17:02:43.738381+0000","last_became_active":"2024-05-30T17:02:41.008540+0000","last_became_peered":"2024-05-30T17:02:41.008540+0000","last_unstale":"2024-05-30T17:02:43.738381+0000","last_undegraded":"2024-05-30T17:02:43.738381+0000","last_fullsized":"2024-05-30T17:02:43.738381+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:44.351484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009392+0000","last_change":"2024-05-30T17:02:39.994753+0000","last_active":"2024-05-30T17:02:42.009392+0000","last_peered":"2024-05-30T17:02:42.009392+0000","last_clean":"2024-05-30T17:02:42.009392+0000","last_became_active":"2024-05-30T17:02:39.994218+0000","last_became_peered":"2024-05-30T17:02:39.994218+0000","last_unstale":"2024-05-30T17:02:42.009392+0000","last_undegraded":"2024-05-30T17:02:42.009392+0000","last_fullsized":"2024-05-30T17:02:42.009392+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:01.556854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708964+0000","last_change":"2024-05-30T17:02:41.012720+0000","last_active":"2024-05-30T17:02:42.708964+0000","last_peered":"2024-05-30T17:02:42.708964+0000","last_clean":"2024-05-30T17:02:42.708964+0000","last_became_active":"2024-05-30T17:02:41.010803+0000","last_became_peered":"2024-05-30T17:02:41.010803+0000","last_unstale":"2024-05-30T17:02:42.708964+0000","last_undegraded":"2024-05-30T17:02:42.708964+0000","last_fullsized":"2024-05-30T17:02:42.708964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:08.401182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708658+0000","last_change":"2024-05-30T17:02:40.006576+0000","last_active":"2024-05-30T17:02:42.708658+0000","last_peered":"2024-05-30T17:02:42.708658+0000","last_clean":"2024-05-30T17:02:42.708658+0000","last_became_active":"2024-05-30T17:02:40.005830+0000","last_became_peered":"2024-05-30T17:02:40.005830+0000","last_unstale":"2024-05-30T17:02:42.708658+0000","last_undegraded":"2024-05-30T17:02:42.708658+0000","last_fullsized":"2024-05-30T17:02:42.708658+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:19.943415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006825+0000","last_change":"2024-05-30T17:02:41.015001+0000","last_active":"2024-05-30T17:02:42.006825+0000","last_peered":"2024-05-30T17:02:42.006825+0000","last_clean":"2024-05-30T17:02:42.006825+0000","last_became_active":"2024-05-30T17:02:41.012609+0000","last_became_peered":"2024-05-30T17:02:41.012609+0000","last_unstale":"2024-05-30T17:02:42.006825+0000","last_undegraded":"2024-05-30T17:02:42.006825+0000","last_fullsized":"2024-05-30T17:02:42.006825+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:40:37.646707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012123+0000","last_change":"2024-05-30T17:02:39.990744+0000","last_active":"2024-05-30T17:02:42.012123+0000","last_peered":"2024-05-30T17:02:42.012123+0000","last_clean":"2024-05-30T17:02:42.012123+0000","last_became_active":"2024-05-30T17:02:39.990046+0000","last_became_peered":"2024-05-30T17:02:39.990046+0000","last_unstale":"2024-05-30T17:02:42.012123+0000","last_undegraded":"2024-05-30T17:02:42.012123+0000","last_fullsized":"2024-05-30T17:02:42.012123+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:00:30.204717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":352,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738594+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:02:43.738594+0000","last_peered":"2024-05-30T17:02:43.738594+0000","last_clean":"2024-05-30T17:02:43.738594+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:02:43.738594+0000","last_undegraded":"2024-05-30T17:02:43.738594+0000","last_fullsized":"2024-05-30T17:02:43.738594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509672,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131088,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240752,"statfs":{"total":96636764160,"available":96502530048,"internally_reserved":0,"allocated":1437696,"data_stored":628679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509672,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144964,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226876,"statfs":{"total":96636764160,"available":96488321024,"internally_reserved":0,"allocated":1433600,"data_stored":628609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509673,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131092,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240748,"statfs":{"total":96636764160,"available":96502525952,"internally_reserved":0,"allocated":1449984,"data_stored":630291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509673,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149132,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222708,"statfs":{"total":96636764160,"available":96484052992,"internally_reserved":0,"allocated":1433600,"data_stored":628609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509672,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144980,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226860,"statfs":{"total":96636764160,"available":96488304640,"internally_reserved":0,"allocated":1441792,"data_stored":628733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542375,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131652,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240188,"statfs":{"total":96636764160,"available":96501952512,"internally_reserved":0,"allocated":2023424,"data_stored":1213181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542376,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64468,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307372,"statfs":{"total":96636764160,"available":96570748928,"internally_reserved":0,"allocated":1441792,"data_stored":629213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542376,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105308,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266532,"statfs":{"total":96636764160,"available":96528928768,"internally_reserved":0,"allocated":2048000,"data_stored":1220701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:02:50.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:02:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.734+0000 7f034ca00640 1 Processor -- start 2024-05-30T17:02:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.734+0000 7f034ca00640 1 -- start start 2024-05-30T17:02:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.735+0000 7f034ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f034815e870 0x7f03480d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.735+0000 7f034ca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f034815ec50 con 0x7f03480d14d0 2024-05-30T17:02:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.735+0000 7f034ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03480cff80 con 0x7f03480cfa40 2024-05-30T17:02:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.735+0000 7f034ca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f03480d2150 con 0x7f034815e870 2024-05-30T17:02:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.736+0000 7f0347400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f034815e870 0x7f03480d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.736+0000 7f0347400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f034815e870 0x7f03480d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60778/0 (socket says 172.21.15.67:60778) 2024-05-30T17:02:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.736+0000 7f0347400640 1 -- 172.21.15.67:0/3313426323 learned_addr learned my addr 172.21.15.67:0/3313426323 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.736+0000 7f0346a00640 1 --1- 172.21.15.67:0/3313426323 >> v1:172.21.15.67:6789/0 conn(0x7f03480d14d0 0x7f03480cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58656/0 (socket says 172.21.15.67:58656) 2024-05-30T17:02:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.736+0000 7f0347400640 1 -- 172.21.15.67:0/3313426323 >> v1:172.21.15.169:6789/0 conn(0x7f03480cfa40 legacy=0x7f03480d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.736+0000 7f0347400640 1 -- 172.21.15.67:0/3313426323 >> v1:172.21.15.67:6789/0 conn(0x7f03480d14d0 legacy=0x7f03480cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.736+0000 7f0347400640 1 -- 172.21.15.67:0/3313426323 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03480d0260 con 0x7f034815e870 2024-05-30T17:02:50.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.736+0000 7f0346000640 1 -- 172.21.15.67:0/3313426323 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237893131 0 0) 0x7f034815ec50 con 0x7f03480d14d0 2024-05-30T17:02:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.737+0000 7f0347400640 1 --2- 172.21.15.67:0/3313426323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 0x7f03480d0f90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f0338009870 tx=0x7f033802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d662ead116c43fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.738+0000 7f0346000640 1 -- 172.21.15.67:0/3313426323 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338037070 con 0x7f034815e870 2024-05-30T17:02:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.738+0000 7f0346000640 1 -- 172.21.15.67:0/3313426323 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0338032440 con 0x7f034815e870 2024-05-30T17:02:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.738+0000 7f0346000640 1 -- 172.21.15.67:0/3313426323 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338032810 con 0x7f034815e870 2024-05-30T17:02:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.738+0000 7f034ca00640 1 -- 172.21.15.67:0/3313426323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 msgr2=0x7f03480d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.738+0000 7f034ca00640 1 --2- 172.21.15.67:0/3313426323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 0x7f03480d0f90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f0338009870 tx=0x7f033802a2a0 comp rx=0 tx=0).stop 2024-05-30T17:02:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.739+0000 7f034ca00640 1 -- 172.21.15.67:0/3313426323 shutdown_connections 2024-05-30T17:02:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.739+0000 7f034ca00640 1 --2- 172.21.15.67:0/3313426323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 0x7f03480d0f90 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.739+0000 7f034ca00640 1 -- 172.21.15.67:0/3313426323 >> 172.21.15.67:0/3313426323 conn(0x7f034808bfc0 msgr2=0x7f03480d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.739+0000 7f034ca00640 1 -- 172.21.15.67:0/3313426323 shutdown_connections 2024-05-30T17:02:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.739+0000 7f034ca00640 1 -- 172.21.15.67:0/3313426323 wait complete. 2024-05-30T17:02:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.740+0000 7f034ca00640 1 Processor -- start 2024-05-30T17:02:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.741+0000 7f034ca00640 1 -- start start 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.741+0000 7f034ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03480cfa40 0x7f0348159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.741+0000 7f034ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03480d14d0 0x7f03480b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.741+0000 7f034ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 0x7f03480b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.741+0000 7f034ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f03480d4fb0 con 0x7f034815e870 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.741+0000 7f034ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f03480d0820 con 0x7f03480cfa40 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.741+0000 7f034ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f03480d2cf0 con 0x7f03480d14d0 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 0x7f03480b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 0x7f03480b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60780/0 (socket says 172.21.15.67:60780) 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347e00640 1 -- 172.21.15.67:0/2202973342 learned_addr learned my addr 172.21.15.67:0/2202973342 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347400640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03480cfa40 0x7f0348159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0346a00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03480d14d0 0x7f03480b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347e00640 1 -- 172.21.15.67:0/2202973342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03480d14d0 msgr2=0x7f03480b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347e00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03480d14d0 0x7f03480b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347e00640 1 -- 172.21.15.67:0/2202973342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03480cfa40 msgr2=0x7f0348159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347e00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03480cfa40 0x7f0348159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347e00640 1 -- 172.21.15.67:0/2202973342 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03480b5bd0 con 0x7f034815e870 2024-05-30T17:02:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.742+0000 7f0347400640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03480cfa40 0x7f0348159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.743+0000 7f0347e00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 0x7f03480b3c80 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f034000c730 tx=0x7f034000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.743+0000 7f033f400640 1 -- 172.21.15.67:0/2202973342 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340016020 con 0x7f034815e870 2024-05-30T17:02:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.743+0000 7f033f400640 1 -- 172.21.15.67:0/2202973342 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034000aa50 con 0x7f034815e870 2024-05-30T17:02:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.743+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03480b45f0 con 0x7f034815e870 2024-05-30T17:02:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.743+0000 7f033f400640 1 -- 172.21.15.67:0/2202973342 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034000ce40 con 0x7f034815e870 2024-05-30T17:02:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.743+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03480b4bb0 con 0x7f034815e870 2024-05-30T17:02:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.744+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03480689e0 con 0x7f034815e870 2024-05-30T17:02:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.745+0000 7f033f400640 1 -- 172.21.15.67:0/2202973342 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f0340019020 con 0x7f034815e870 2024-05-30T17:02:50.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.746+0000 7f033f400640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f032806e090 0x7f0328070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.746+0000 7f033f400640 1 -- 172.21.15.67:0/2202973342 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f034008a880 con 0x7f034815e870 2024-05-30T17:02:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.746+0000 7f0347400640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f032806e090 0x7f0328070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.747+0000 7f0347400640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f032806e090 0x7f0328070550 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0338034000 tx=0x7f033802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:50.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.751+0000 7f033f400640 1 -- 172.21.15.67:0/2202973342 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0340020030 con 0x7f034815e870 2024-05-30T17:02:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.952+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03480ba500 con 0x7f032806e090 2024-05-30T17:02:50.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.963+0000 7f033f400640 1 -- 172.21.15.67:0/2202973342 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252446 (secure 0 0 0) 0x7f03480ba500 con 0x7f032806e090 2024-05-30T17:02:50.967 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:50.969 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f032806e090 msgr2=0x7f0328070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f032806e090 0x7f0328070550 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0338034000 tx=0x7f033802e000 comp rx=0 tx=0).stop 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 msgr2=0x7f03480b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 0x7f03480b3c80 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f034000c730 tx=0x7f034000cbf0 comp rx=0 tx=0).stop 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 shutdown_connections 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f032806e090 0x7f0328070550 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f034815e870 0x7f03480b3c80 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03480d14d0 0x7f03480b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 --2- 172.21.15.67:0/2202973342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03480cfa40 0x7f0348159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.969+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 >> 172.21.15.67:0/2202973342 conn(0x7f034808bfc0 msgr2=0x7f034806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.970+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 shutdown_connections 2024-05-30T17:02:50.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:50.970+0000 7f034ca00640 1 -- 172.21.15.67:0/2202973342 wait complete. 2024-05-30T17:02:50.990 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":685,"stamp":"2024-05-30T17:02:50.524580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1002696,"kb_used_data":12424,"kb_used_omap":12,"kb_used_meta":990131,"kb_avail":753972024,"statfs":{"total":773094113280,"available":772067352576,"internally_reserved":0,"allocated":12722176,"data_stored":6214428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1013894704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.551484"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710331+0000","last_change":"2024-05-30T17:02:41.021236+0000","last_active":"2024-05-30T17:02:42.710331+0000","last_peered":"2024-05-30T17:02:42.710331+0000","last_clean":"2024-05-30T17:02:42.710331+0000","last_became_active":"2024-05-30T17:02:41.020953+0000","last_became_peered":"2024-05-30T17:02:41.020953+0000","last_unstale":"2024-05-30T17:02:42.710331+0000","last_undegraded":"2024-05-30T17:02:42.710331+0000","last_fullsized":"2024-05-30T17:02:42.710331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:12:08.262939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809486+0000","last_change":"2024-05-30T17:02:41.014728+0000","last_active":"2024-05-30T17:02:43.809486+0000","last_peered":"2024-05-30T17:02:43.809486+0000","last_clean":"2024-05-30T17:02:43.809486+0000","last_became_active":"2024-05-30T17:02:41.014423+0000","last_became_peered":"2024-05-30T17:02:41.014423+0000","last_unstale":"2024-05-30T17:02:43.809486+0000","last_undegraded":"2024-05-30T17:02:43.809486+0000","last_fullsized":"2024-05-30T17:02:43.809486+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:04:02.116807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738383+0000","last_change":"2024-05-30T17:02:41.013698+0000","last_active":"2024-05-30T17:02:43.738383+0000","last_peered":"2024-05-30T17:02:43.738383+0000","last_clean":"2024-05-30T17:02:43.738383+0000","last_became_active":"2024-05-30T17:02:41.013407+0000","last_became_peered":"2024-05-30T17:02:41.013407+0000","last_unstale":"2024-05-30T17:02:43.738383+0000","last_undegraded":"2024-05-30T17:02:43.738383+0000","last_fullsized":"2024-05-30T17:02:43.738383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:56.046859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709151+0000","last_change":"2024-05-30T17:02:41.020938+0000","last_active":"2024-05-30T17:02:42.709151+0000","last_peered":"2024-05-30T17:02:42.709151+0000","last_clean":"2024-05-30T17:02:42.709151+0000","last_became_active":"2024-05-30T17:02:41.017229+0000","last_became_peered":"2024-05-30T17:02:41.017229+0000","last_unstale":"2024-05-30T17:02:42.709151+0000","last_undegraded":"2024-05-30T17:02:42.709151+0000","last_fullsized":"2024-05-30T17:02:42.709151+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:45:39.295718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009700+0000","last_change":"2024-05-30T17:02:41.006127+0000","last_active":"2024-05-30T17:02:42.009700+0000","last_peered":"2024-05-30T17:02:42.009700+0000","last_clean":"2024-05-30T17:02:42.009700+0000","last_became_active":"2024-05-30T17:02:41.005503+0000","last_became_peered":"2024-05-30T17:02:41.005503+0000","last_unstale":"2024-05-30T17:02:42.009700+0000","last_undegraded":"2024-05-30T17:02:42.009700+0000","last_fullsized":"2024-05-30T17:02:42.009700+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:08.569678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188550+0000","last_change":"2024-05-30T17:02:41.006264+0000","last_active":"2024-05-30T17:02:42.188550+0000","last_peered":"2024-05-30T17:02:42.188550+0000","last_clean":"2024-05-30T17:02:42.188550+0000","last_became_active":"2024-05-30T17:02:41.005993+0000","last_became_peered":"2024-05-30T17:02:41.005993+0000","last_unstale":"2024-05-30T17:02:42.188550+0000","last_undegraded":"2024-05-30T17:02:42.188550+0000","last_fullsized":"2024-05-30T17:02:42.188550+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:52:18.051272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011985+0000","last_change":"2024-05-30T17:02:41.010886+0000","last_active":"2024-05-30T17:02:42.011985+0000","last_peered":"2024-05-30T17:02:42.011985+0000","last_clean":"2024-05-30T17:02:42.011985+0000","last_became_active":"2024-05-30T17:02:41.007377+0000","last_became_peered":"2024-05-30T17:02:41.007377+0000","last_unstale":"2024-05-30T17:02:42.011985+0000","last_undegraded":"2024-05-30T17:02:42.011985+0000","last_fullsized":"2024-05-30T17:02:42.011985+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:59.071677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738235+0000","last_change":"2024-05-30T17:02:41.014332+0000","last_active":"2024-05-30T17:02:43.738235+0000","last_peered":"2024-05-30T17:02:43.738235+0000","last_clean":"2024-05-30T17:02:43.738235+0000","last_became_active":"2024-05-30T17:02:41.014074+0000","last_became_peered":"2024-05-30T17:02:41.014074+0000","last_unstale":"2024-05-30T17:02:43.738235+0000","last_undegraded":"2024-05-30T17:02:43.738235+0000","last_fullsized":"2024-05-30T17:02:43.738235+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:00:36.629725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710477+0000","last_change":"2024-05-30T17:02:41.012156+0000","last_active":"2024-05-30T17:02:42.710477+0000","last_peered":"2024-05-30T17:02:42.710477+0000","last_clean":"2024-05-30T17:02:42.710477+0000","last_became_active":"2024-05-30T17:02:41.011380+0000","last_became_peered":"2024-05-30T17:02:41.011380+0000","last_unstale":"2024-05-30T17:02:42.710477+0000","last_undegraded":"2024-05-30T17:02:42.710477+0000","last_fullsized":"2024-05-30T17:02:42.710477+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:35:04.378307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710199+0000","last_change":"2024-05-30T17:02:41.021177+0000","last_active":"2024-05-30T17:02:42.710199+0000","last_peered":"2024-05-30T17:02:42.710199+0000","last_clean":"2024-05-30T17:02:42.710199+0000","last_became_active":"2024-05-30T17:02:41.015425+0000","last_became_peered":"2024-05-30T17:02:41.015425+0000","last_unstale":"2024-05-30T17:02:42.710199+0000","last_undegraded":"2024-05-30T17:02:42.710199+0000","last_fullsized":"2024-05-30T17:02:42.710199+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:59:34.132408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808924+0000","last_change":"2024-05-30T17:02:41.015419+0000","last_active":"2024-05-30T17:02:43.808924+0000","last_peered":"2024-05-30T17:02:43.808924+0000","last_clean":"2024-05-30T17:02:43.808924+0000","last_became_active":"2024-05-30T17:02:41.015134+0000","last_became_peered":"2024-05-30T17:02:41.015134+0000","last_unstale":"2024-05-30T17:02:43.808924+0000","last_undegraded":"2024-05-30T17:02:43.808924+0000","last_fullsized":"2024-05-30T17:02:43.808924+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:28:15.577874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709241+0000","last_change":"2024-05-30T17:02:41.017042+0000","last_active":"2024-05-30T17:02:42.709241+0000","last_peered":"2024-05-30T17:02:42.709241+0000","last_clean":"2024-05-30T17:02:42.709241+0000","last_became_active":"2024-05-30T17:02:41.013921+0000","last_became_peered":"2024-05-30T17:02:41.013921+0000","last_unstale":"2024-05-30T17:02:42.709241+0000","last_undegraded":"2024-05-30T17:02:42.709241+0000","last_fullsized":"2024-05-30T17:02:42.709241+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:56:21.972017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738860+0000","last_change":"2024-05-30T17:02:41.011886+0000","last_active":"2024-05-30T17:02:43.738860+0000","last_peered":"2024-05-30T17:02:43.738860+0000","last_clean":"2024-05-30T17:02:43.738860+0000","last_became_active":"2024-05-30T17:02:41.010296+0000","last_became_peered":"2024-05-30T17:02:41.010296+0000","last_unstale":"2024-05-30T17:02:43.738860+0000","last_undegraded":"2024-05-30T17:02:43.738860+0000","last_fullsized":"2024-05-30T17:02:43.738860+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:23:20.421009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737964+0000","last_change":"2024-05-30T17:02:41.012428+0000","last_active":"2024-05-30T17:02:43.737964+0000","last_peered":"2024-05-30T17:02:43.737964+0000","last_clean":"2024-05-30T17:02:43.737964+0000","last_became_active":"2024-05-30T17:02:41.011764+0000","last_became_peered":"2024-05-30T17:02:41.011764+0000","last_unstale":"2024-05-30T17:02:43.737964+0000","last_undegraded":"2024-05-30T17:02:43.737964+0000","last_fullsized":"2024-05-30T17:02:43.737964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:11:05.228132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188591+0000","last_change":"2024-05-30T17:02:41.005418+0000","last_active":"2024-05-30T17:02:42.188591+0000","last_peered":"2024-05-30T17:02:42.188591+0000","last_clean":"2024-05-30T17:02:42.188591+0000","last_became_active":"2024-05-30T17:02:41.005083+0000","last_became_peered":"2024-05-30T17:02:41.005083+0000","last_unstale":"2024-05-30T17:02:42.188591+0000","last_undegraded":"2024-05-30T17:02:42.188591+0000","last_fullsized":"2024-05-30T17:02:42.188591+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:13:32.601564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708052+0000","last_change":"2024-05-30T17:02:41.010971+0000","last_active":"2024-05-30T17:02:42.708052+0000","last_peered":"2024-05-30T17:02:42.708052+0000","last_clean":"2024-05-30T17:02:42.708052+0000","last_became_active":"2024-05-30T17:02:41.010647+0000","last_became_peered":"2024-05-30T17:02:41.010647+0000","last_unstale":"2024-05-30T17:02:42.708052+0000","last_undegraded":"2024-05-30T17:02:42.708052+0000","last_fullsized":"2024-05-30T17:02:42.708052+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:14.515707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809168+0000","last_change":"2024-05-30T17:02:41.012058+0000","last_active":"2024-05-30T17:02:43.809168+0000","last_peered":"2024-05-30T17:02:43.809168+0000","last_clean":"2024-05-30T17:02:43.809168+0000","last_became_active":"2024-05-30T17:02:41.011201+0000","last_became_peered":"2024-05-30T17:02:41.011201+0000","last_unstale":"2024-05-30T17:02:43.809168+0000","last_undegraded":"2024-05-30T17:02:43.809168+0000","last_fullsized":"2024-05-30T17:02:43.809168+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:51.244231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015163+0000","last_change":"2024-05-30T17:02:41.007534+0000","last_active":"2024-05-30T17:02:42.015163+0000","last_peered":"2024-05-30T17:02:42.015163+0000","last_clean":"2024-05-30T17:02:42.015163+0000","last_became_active":"2024-05-30T17:02:41.007238+0000","last_became_peered":"2024-05-30T17:02:41.007238+0000","last_unstale":"2024-05-30T17:02:42.015163+0000","last_undegraded":"2024-05-30T17:02:42.015163+0000","last_fullsized":"2024-05-30T17:02:42.015163+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:28:15.577874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738507+0000","last_change":"2024-05-30T17:02:41.011007+0000","last_active":"2024-05-30T17:02:43.738507+0000","last_peered":"2024-05-30T17:02:43.738507+0000","last_clean":"2024-05-30T17:02:43.738507+0000","last_became_active":"2024-05-30T17:02:41.010322+0000","last_became_peered":"2024-05-30T17:02:41.010322+0000","last_unstale":"2024-05-30T17:02:43.738507+0000","last_undegraded":"2024-05-30T17:02:43.738507+0000","last_fullsized":"2024-05-30T17:02:43.738507+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:40:54.062584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809392+0000","last_change":"2024-05-30T17:02:41.006384+0000","last_active":"2024-05-30T17:02:43.809392+0000","last_peered":"2024-05-30T17:02:43.809392+0000","last_clean":"2024-05-30T17:02:43.809392+0000","last_became_active":"2024-05-30T17:02:41.006075+0000","last_became_peered":"2024-05-30T17:02:41.006075+0000","last_unstale":"2024-05-30T17:02:43.809392+0000","last_undegraded":"2024-05-30T17:02:43.809392+0000","last_fullsized":"2024-05-30T17:02:43.809392+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:08:37.996204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809096+0000","last_change":"2024-05-30T17:02:41.008851+0000","last_active":"2024-05-30T17:02:43.809096+0000","last_peered":"2024-05-30T17:02:43.809096+0000","last_clean":"2024-05-30T17:02:43.809096+0000","last_became_active":"2024-05-30T17:02:41.008042+0000","last_became_peered":"2024-05-30T17:02:41.008042+0000","last_unstale":"2024-05-30T17:02:43.809096+0000","last_undegraded":"2024-05-30T17:02:43.809096+0000","last_fullsized":"2024-05-30T17:02:43.809096+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:16.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006668+0000","last_change":"2024-05-30T17:02:41.006733+0000","last_active":"2024-05-30T17:02:42.006668+0000","last_peered":"2024-05-30T17:02:42.006668+0000","last_clean":"2024-05-30T17:02:42.006668+0000","last_became_active":"2024-05-30T17:02:41.005818+0000","last_became_peered":"2024-05-30T17:02:41.005818+0000","last_unstale":"2024-05-30T17:02:42.006668+0000","last_undegraded":"2024-05-30T17:02:42.006668+0000","last_fullsized":"2024-05-30T17:02:42.006668+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:04:50.852844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808384+0000","last_change":"2024-05-30T17:02:41.004543+0000","last_active":"2024-05-30T17:02:43.808384+0000","last_peered":"2024-05-30T17:02:43.808384+0000","last_clean":"2024-05-30T17:02:43.808384+0000","last_became_active":"2024-05-30T17:02:41.004253+0000","last_became_peered":"2024-05-30T17:02:41.004253+0000","last_unstale":"2024-05-30T17:02:43.808384+0000","last_undegraded":"2024-05-30T17:02:43.808384+0000","last_fullsized":"2024-05-30T17:02:43.808384+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:45:01.496834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006559+0000","last_change":"2024-05-30T17:02:41.012901+0000","last_active":"2024-05-30T17:02:42.006559+0000","last_peered":"2024-05-30T17:02:42.006559+0000","last_clean":"2024-05-30T17:02:42.006559+0000","last_became_active":"2024-05-30T17:02:41.012234+0000","last_became_peered":"2024-05-30T17:02:41.012234+0000","last_unstale":"2024-05-30T17:02:42.006559+0000","last_undegraded":"2024-05-30T17:02:42.006559+0000","last_fullsized":"2024-05-30T17:02:42.006559+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:27:57.491125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809287+0000","last_change":"2024-05-30T17:02:41.011507+0000","last_active":"2024-05-30T17:02:43.809287+0000","last_peered":"2024-05-30T17:02:43.809287+0000","last_clean":"2024-05-30T17:02:43.809287+0000","last_became_active":"2024-05-30T17:02:41.011191+0000","last_became_peered":"2024-05-30T17:02:41.011191+0000","last_unstale":"2024-05-30T17:02:43.809287+0000","last_undegraded":"2024-05-30T17:02:43.809287+0000","last_fullsized":"2024-05-30T17:02:43.809287+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:35:55.671230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710128+0000","last_change":"2024-05-30T17:02:41.009883+0000","last_active":"2024-05-30T17:02:42.710128+0000","last_peered":"2024-05-30T17:02:42.710128+0000","last_clean":"2024-05-30T17:02:42.710128+0000","last_became_active":"2024-05-30T17:02:41.008338+0000","last_became_peered":"2024-05-30T17:02:41.008338+0000","last_unstale":"2024-05-30T17:02:42.710128+0000","last_undegraded":"2024-05-30T17:02:42.710128+0000","last_fullsized":"2024-05-30T17:02:42.710128+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:29:12.257884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.189393+0000","last_change":"2024-05-30T17:02:41.013990+0000","last_active":"2024-05-30T17:02:42.189393+0000","last_peered":"2024-05-30T17:02:42.189393+0000","last_clean":"2024-05-30T17:02:42.189393+0000","last_became_active":"2024-05-30T17:02:41.013699+0000","last_became_peered":"2024-05-30T17:02:41.013699+0000","last_unstale":"2024-05-30T17:02:42.189393+0000","last_undegraded":"2024-05-30T17:02:42.189393+0000","last_fullsized":"2024-05-30T17:02:42.189393+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:10.858887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007163+0000","last_change":"2024-05-30T17:02:41.016363+0000","last_active":"2024-05-30T17:02:42.007163+0000","last_peered":"2024-05-30T17:02:42.007163+0000","last_clean":"2024-05-30T17:02:42.007163+0000","last_became_active":"2024-05-30T17:02:41.016024+0000","last_became_peered":"2024-05-30T17:02:41.016024+0000","last_unstale":"2024-05-30T17:02:42.007163+0000","last_undegraded":"2024-05-30T17:02:42.007163+0000","last_fullsized":"2024-05-30T17:02:42.007163+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:12.804431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808497+0000","last_change":"2024-05-30T17:02:41.009285+0000","last_active":"2024-05-30T17:02:43.808497+0000","last_peered":"2024-05-30T17:02:43.808497+0000","last_clean":"2024-05-30T17:02:43.808497+0000","last_became_active":"2024-05-30T17:02:41.008928+0000","last_became_peered":"2024-05-30T17:02:41.008928+0000","last_unstale":"2024-05-30T17:02:43.808497+0000","last_undegraded":"2024-05-30T17:02:43.808497+0000","last_fullsized":"2024-05-30T17:02:43.808497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:39:52.817158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012127+0000","last_change":"2024-05-30T17:02:41.011572+0000","last_active":"2024-05-30T17:02:42.012127+0000","last_peered":"2024-05-30T17:02:42.012127+0000","last_clean":"2024-05-30T17:02:42.012127+0000","last_became_active":"2024-05-30T17:02:41.011197+0000","last_became_peered":"2024-05-30T17:02:41.011197+0000","last_unstale":"2024-05-30T17:02:42.012127+0000","last_undegraded":"2024-05-30T17:02:42.012127+0000","last_fullsized":"2024-05-30T17:02:42.012127+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:23:54.031517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708820+0000","last_change":"2024-05-30T17:02:41.014646+0000","last_active":"2024-05-30T17:02:42.708820+0000","last_peered":"2024-05-30T17:02:42.708820+0000","last_clean":"2024-05-30T17:02:42.708820+0000","last_became_active":"2024-05-30T17:02:41.014284+0000","last_became_peered":"2024-05-30T17:02:41.014284+0000","last_unstale":"2024-05-30T17:02:42.708820+0000","last_undegraded":"2024-05-30T17:02:42.708820+0000","last_fullsized":"2024-05-30T17:02:42.708820+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:20:15.241290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188844+0000","last_change":"2024-05-30T17:02:41.005765+0000","last_active":"2024-05-30T17:02:42.188844+0000","last_peered":"2024-05-30T17:02:42.188844+0000","last_clean":"2024-05-30T17:02:42.188844+0000","last_became_active":"2024-05-30T17:02:41.004818+0000","last_became_peered":"2024-05-30T17:02:41.004818+0000","last_unstale":"2024-05-30T17:02:42.188844+0000","last_undegraded":"2024-05-30T17:02:42.188844+0000","last_fullsized":"2024-05-30T17:02:42.188844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:32.497197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188893+0000","last_change":"2024-05-30T17:02:40.999105+0000","last_active":"2024-05-30T17:02:42.188893+0000","last_peered":"2024-05-30T17:02:42.188893+0000","last_clean":"2024-05-30T17:02:42.188893+0000","last_became_active":"2024-05-30T17:02:40.998808+0000","last_became_peered":"2024-05-30T17:02:40.998808+0000","last_unstale":"2024-05-30T17:02:42.188893+0000","last_undegraded":"2024-05-30T17:02:42.188893+0000","last_fullsized":"2024-05-30T17:02:42.188893+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:45:59.198214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"115'4","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012308+0000","last_change":"2024-05-30T17:02:39.992222+0000","last_active":"2024-05-30T17:02:42.012308+0000","last_peered":"2024-05-30T17:02:42.012308+0000","last_clean":"2024-05-30T17:02:42.012308+0000","last_became_active":"2024-05-30T17:02:39.991672+0000","last_became_peered":"2024-05-30T17:02:39.991672+0000","last_unstale":"2024-05-30T17:02:42.012308+0000","last_undegraded":"2024-05-30T17:02:42.012308+0000","last_fullsized":"2024-05-30T17:02:42.012308+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:17:06.803976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015760+0000","last_change":"2024-05-30T17:02:41.005694+0000","last_active":"2024-05-30T17:02:42.015760+0000","last_peered":"2024-05-30T17:02:42.015760+0000","last_clean":"2024-05-30T17:02:42.015760+0000","last_became_active":"2024-05-30T17:02:41.005391+0000","last_became_peered":"2024-05-30T17:02:41.005391+0000","last_unstale":"2024-05-30T17:02:42.015760+0000","last_undegraded":"2024-05-30T17:02:42.015760+0000","last_fullsized":"2024-05-30T17:02:42.015760+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:35:55.671230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738709+0000","last_change":"2024-05-30T17:02:40.001257+0000","last_active":"2024-05-30T17:02:43.738709+0000","last_peered":"2024-05-30T17:02:43.738709+0000","last_clean":"2024-05-30T17:02:43.738709+0000","last_became_active":"2024-05-30T17:02:40.000956+0000","last_became_peered":"2024-05-30T17:02:40.000956+0000","last_unstale":"2024-05-30T17:02:43.738709+0000","last_undegraded":"2024-05-30T17:02:43.738709+0000","last_fullsized":"2024-05-30T17:02:43.738709+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:18:52.599362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.189215+0000","last_change":"2024-05-30T17:02:41.007480+0000","last_active":"2024-05-30T17:02:42.189215+0000","last_peered":"2024-05-30T17:02:42.189215+0000","last_clean":"2024-05-30T17:02:42.189215+0000","last_became_active":"2024-05-30T17:02:41.004878+0000","last_became_peered":"2024-05-30T17:02:41.004878+0000","last_unstale":"2024-05-30T17:02:42.189215+0000","last_undegraded":"2024-05-30T17:02:42.189215+0000","last_fullsized":"2024-05-30T17:02:42.189215+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:25:21.801488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.010407+0000","last_change":"2024-05-30T17:02:40.004186+0000","last_active":"2024-05-30T17:02:42.010407+0000","last_peered":"2024-05-30T17:02:42.010407+0000","last_clean":"2024-05-30T17:02:42.010407+0000","last_became_active":"2024-05-30T17:02:40.003831+0000","last_became_peered":"2024-05-30T17:02:40.003831+0000","last_unstale":"2024-05-30T17:02:42.010407+0000","last_undegraded":"2024-05-30T17:02:42.010407+0000","last_fullsized":"2024-05-30T17:02:42.010407+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:09.841780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738822+0000","last_change":"2024-05-30T17:02:41.011197+0000","last_active":"2024-05-30T17:02:43.738822+0000","last_peered":"2024-05-30T17:02:43.738822+0000","last_clean":"2024-05-30T17:02:43.738822+0000","last_became_active":"2024-05-30T17:02:41.010742+0000","last_became_peered":"2024-05-30T17:02:41.010742+0000","last_unstale":"2024-05-30T17:02:43.738822+0000","last_undegraded":"2024-05-30T17:02:43.738822+0000","last_fullsized":"2024-05-30T17:02:43.738822+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:46:14.746408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.010031+0000","last_change":"2024-05-30T17:02:39.999367+0000","last_active":"2024-05-30T17:02:42.010031+0000","last_peered":"2024-05-30T17:02:42.010031+0000","last_clean":"2024-05-30T17:02:42.010031+0000","last_became_active":"2024-05-30T17:02:39.998521+0000","last_became_peered":"2024-05-30T17:02:39.998521+0000","last_unstale":"2024-05-30T17:02:42.010031+0000","last_undegraded":"2024-05-30T17:02:42.010031+0000","last_fullsized":"2024-05-30T17:02:42.010031+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:19:34.293069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708723+0000","last_change":"2024-05-30T17:02:41.012269+0000","last_active":"2024-05-30T17:02:42.708723+0000","last_peered":"2024-05-30T17:02:42.708723+0000","last_clean":"2024-05-30T17:02:42.708723+0000","last_became_active":"2024-05-30T17:02:41.011965+0000","last_became_peered":"2024-05-30T17:02:41.011965+0000","last_unstale":"2024-05-30T17:02:42.708723+0000","last_undegraded":"2024-05-30T17:02:42.708723+0000","last_fullsized":"2024-05-30T17:02:42.708723+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:28:07.302726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012030+0000","last_change":"2024-05-30T17:02:39.994746+0000","last_active":"2024-05-30T17:02:42.012030+0000","last_peered":"2024-05-30T17:02:42.012030+0000","last_clean":"2024-05-30T17:02:42.012030+0000","last_became_active":"2024-05-30T17:02:39.994412+0000","last_became_peered":"2024-05-30T17:02:39.994412+0000","last_unstale":"2024-05-30T17:02:42.012030+0000","last_undegraded":"2024-05-30T17:02:42.012030+0000","last_fullsized":"2024-05-30T17:02:42.012030+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:01:23.895654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808513+0000","last_change":"2024-05-30T17:02:41.014804+0000","last_active":"2024-05-30T17:02:43.808513+0000","last_peered":"2024-05-30T17:02:43.808513+0000","last_clean":"2024-05-30T17:02:43.808513+0000","last_became_active":"2024-05-30T17:02:41.014498+0000","last_became_peered":"2024-05-30T17:02:41.014498+0000","last_unstale":"2024-05-30T17:02:43.808513+0000","last_undegraded":"2024-05-30T17:02:43.808513+0000","last_fullsized":"2024-05-30T17:02:43.808513+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:22.624806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007547+0000","last_change":"2024-05-30T17:02:40.002354+0000","last_active":"2024-05-30T17:02:42.007547+0000","last_peered":"2024-05-30T17:02:42.007547+0000","last_clean":"2024-05-30T17:02:42.007547+0000","last_became_active":"2024-05-30T17:02:40.002062+0000","last_became_peered":"2024-05-30T17:02:40.002062+0000","last_unstale":"2024-05-30T17:02:42.007547+0000","last_undegraded":"2024-05-30T17:02:42.007547+0000","last_fullsized":"2024-05-30T17:02:42.007547+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:28:14.560768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708173+0000","last_change":"2024-05-30T17:02:41.008592+0000","last_active":"2024-05-30T17:02:42.708173+0000","last_peered":"2024-05-30T17:02:42.708173+0000","last_clean":"2024-05-30T17:02:42.708173+0000","last_became_active":"2024-05-30T17:02:41.008270+0000","last_became_peered":"2024-05-30T17:02:41.008270+0000","last_unstale":"2024-05-30T17:02:42.708173+0000","last_undegraded":"2024-05-30T17:02:42.708173+0000","last_fullsized":"2024-05-30T17:02:42.708173+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:23:35.433137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808933+0000","last_change":"2024-05-30T17:02:39.996962+0000","last_active":"2024-05-30T17:02:43.808933+0000","last_peered":"2024-05-30T17:02:43.808933+0000","last_clean":"2024-05-30T17:02:43.808933+0000","last_became_active":"2024-05-30T17:02:39.996677+0000","last_became_peered":"2024-05-30T17:02:39.996677+0000","last_unstale":"2024-05-30T17:02:43.808933+0000","last_undegraded":"2024-05-30T17:02:43.808933+0000","last_fullsized":"2024-05-30T17:02:43.808933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:28:38.120481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012752+0000","last_change":"2024-05-30T17:02:41.010582+0000","last_active":"2024-05-30T17:02:42.012752+0000","last_peered":"2024-05-30T17:02:42.012752+0000","last_clean":"2024-05-30T17:02:42.012752+0000","last_became_active":"2024-05-30T17:02:41.006958+0000","last_became_peered":"2024-05-30T17:02:41.006958+0000","last_unstale":"2024-05-30T17:02:42.012752+0000","last_undegraded":"2024-05-30T17:02:42.012752+0000","last_fullsized":"2024-05-30T17:02:42.012752+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:42:06.872806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708333+0000","last_change":"2024-05-30T17:02:39.996651+0000","last_active":"2024-05-30T17:02:42.708333+0000","last_peered":"2024-05-30T17:02:42.708333+0000","last_clean":"2024-05-30T17:02:42.708333+0000","last_became_active":"2024-05-30T17:02:39.996364+0000","last_became_peered":"2024-05-30T17:02:39.996364+0000","last_unstale":"2024-05-30T17:02:42.708333+0000","last_undegraded":"2024-05-30T17:02:42.708333+0000","last_fullsized":"2024-05-30T17:02:42.708333+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:50:25.128687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012844+0000","last_change":"2024-05-30T17:02:40.999926+0000","last_active":"2024-05-30T17:02:42.012844+0000","last_peered":"2024-05-30T17:02:42.012844+0000","last_clean":"2024-05-30T17:02:42.012844+0000","last_became_active":"2024-05-30T17:02:40.997094+0000","last_became_peered":"2024-05-30T17:02:40.997094+0000","last_unstale":"2024-05-30T17:02:42.012844+0000","last_undegraded":"2024-05-30T17:02:42.012844+0000","last_fullsized":"2024-05-30T17:02:42.012844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:15:52.585750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.010215+0000","last_change":"2024-05-30T17:02:40.000501+0000","last_active":"2024-05-30T17:02:42.010215+0000","last_peered":"2024-05-30T17:02:42.010215+0000","last_clean":"2024-05-30T17:02:42.010215+0000","last_became_active":"2024-05-30T17:02:39.999369+0000","last_became_peered":"2024-05-30T17:02:39.999369+0000","last_unstale":"2024-05-30T17:02:42.010215+0000","last_undegraded":"2024-05-30T17:02:42.010215+0000","last_fullsized":"2024-05-30T17:02:42.010215+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:53:02.236301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015857+0000","last_change":"2024-05-30T17:02:41.024527+0000","last_active":"2024-05-30T17:02:42.015857+0000","last_peered":"2024-05-30T17:02:42.015857+0000","last_clean":"2024-05-30T17:02:42.015857+0000","last_became_active":"2024-05-30T17:02:41.024226+0000","last_became_peered":"2024-05-30T17:02:41.024226+0000","last_unstale":"2024-05-30T17:02:42.015857+0000","last_undegraded":"2024-05-30T17:02:42.015857+0000","last_fullsized":"2024-05-30T17:02:42.015857+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:08:37.996204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006759+0000","last_change":"2024-05-30T17:02:40.002597+0000","last_active":"2024-05-30T17:02:42.006759+0000","last_peered":"2024-05-30T17:02:42.006759+0000","last_clean":"2024-05-30T17:02:42.006759+0000","last_became_active":"2024-05-30T17:02:40.002351+0000","last_became_peered":"2024-05-30T17:02:40.002351+0000","last_unstale":"2024-05-30T17:02:42.006759+0000","last_undegraded":"2024-05-30T17:02:42.006759+0000","last_fullsized":"2024-05-30T17:02:42.006759+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:15.687359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007338+0000","last_change":"2024-05-30T17:02:41.010196+0000","last_active":"2024-05-30T17:02:42.007338+0000","last_peered":"2024-05-30T17:02:42.007338+0000","last_clean":"2024-05-30T17:02:42.007338+0000","last_became_active":"2024-05-30T17:02:41.009903+0000","last_became_peered":"2024-05-30T17:02:41.009903+0000","last_unstale":"2024-05-30T17:02:42.007338+0000","last_undegraded":"2024-05-30T17:02:42.007338+0000","last_fullsized":"2024-05-30T17:02:42.007338+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:24.744651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"115'3","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709331+0000","last_change":"2024-05-30T17:02:39.999796+0000","last_active":"2024-05-30T17:02:42.709331+0000","last_peered":"2024-05-30T17:02:42.709331+0000","last_clean":"2024-05-30T17:02:42.709331+0000","last_became_active":"2024-05-30T17:02:39.999375+0000","last_became_peered":"2024-05-30T17:02:39.999375+0000","last_unstale":"2024-05-30T17:02:42.709331+0000","last_undegraded":"2024-05-30T17:02:42.709331+0000","last_fullsized":"2024-05-30T17:02:42.709331+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:46:11.895313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012385+0000","last_change":"2024-05-30T17:02:41.003790+0000","last_active":"2024-05-30T17:02:42.012385+0000","last_peered":"2024-05-30T17:02:42.012385+0000","last_clean":"2024-05-30T17:02:42.012385+0000","last_became_active":"2024-05-30T17:02:41.003476+0000","last_became_peered":"2024-05-30T17:02:41.003476+0000","last_unstale":"2024-05-30T17:02:42.012385+0000","last_undegraded":"2024-05-30T17:02:42.012385+0000","last_fullsized":"2024-05-30T17:02:42.012385+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:19.156205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"115'6","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709596+0000","last_change":"2024-05-30T17:02:40.003867+0000","last_active":"2024-05-30T17:02:42.709596+0000","last_peered":"2024-05-30T17:02:42.709596+0000","last_clean":"2024-05-30T17:02:42.709596+0000","last_became_active":"2024-05-30T17:02:39.996721+0000","last_became_peered":"2024-05-30T17:02:39.996721+0000","last_unstale":"2024-05-30T17:02:42.709596+0000","last_undegraded":"2024-05-30T17:02:42.709596+0000","last_fullsized":"2024-05-30T17:02:42.709596+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:08:31.053590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007408+0000","last_change":"2024-05-30T17:02:40.998234+0000","last_active":"2024-05-30T17:02:42.007408+0000","last_peered":"2024-05-30T17:02:42.007408+0000","last_clean":"2024-05-30T17:02:42.007408+0000","last_became_active":"2024-05-30T17:02:40.997241+0000","last_became_peered":"2024-05-30T17:02:40.997241+0000","last_unstale":"2024-05-30T17:02:42.007408+0000","last_undegraded":"2024-05-30T17:02:42.007408+0000","last_fullsized":"2024-05-30T17:02:42.007408+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:24:21.891014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.189204+0000","last_change":"2024-05-30T17:02:39.993748+0000","last_active":"2024-05-30T17:02:42.189204+0000","last_peered":"2024-05-30T17:02:42.189204+0000","last_clean":"2024-05-30T17:02:42.189204+0000","last_became_active":"2024-05-30T17:02:39.993315+0000","last_became_peered":"2024-05-30T17:02:39.993315+0000","last_unstale":"2024-05-30T17:02:42.189204+0000","last_undegraded":"2024-05-30T17:02:42.189204+0000","last_fullsized":"2024-05-30T17:02:42.189204+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:34:58.412522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007060+0000","last_change":"2024-05-30T17:02:41.003650+0000","last_active":"2024-05-30T17:02:42.007060+0000","last_peered":"2024-05-30T17:02:42.007060+0000","last_clean":"2024-05-30T17:02:42.007060+0000","last_became_active":"2024-05-30T17:02:41.002514+0000","last_became_peered":"2024-05-30T17:02:41.002514+0000","last_unstale":"2024-05-30T17:02:42.007060+0000","last_undegraded":"2024-05-30T17:02:42.007060+0000","last_fullsized":"2024-05-30T17:02:42.007060+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:41.488964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.014933+0000","last_change":"2024-05-30T17:02:40.006813+0000","last_active":"2024-05-30T17:02:42.014933+0000","last_peered":"2024-05-30T17:02:42.014933+0000","last_clean":"2024-05-30T17:02:42.014933+0000","last_became_active":"2024-05-30T17:02:40.006466+0000","last_became_peered":"2024-05-30T17:02:40.006466+0000","last_unstale":"2024-05-30T17:02:42.014933+0000","last_undegraded":"2024-05-30T17:02:42.014933+0000","last_fullsized":"2024-05-30T17:02:42.014933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:07.267691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006829+0000","last_change":"2024-05-30T17:02:41.000175+0000","last_active":"2024-05-30T17:02:42.006829+0000","last_peered":"2024-05-30T17:02:42.006829+0000","last_clean":"2024-05-30T17:02:42.006829+0000","last_became_active":"2024-05-30T17:02:40.999872+0000","last_became_peered":"2024-05-30T17:02:40.999872+0000","last_unstale":"2024-05-30T17:02:42.006829+0000","last_undegraded":"2024-05-30T17:02:42.006829+0000","last_fullsized":"2024-05-30T17:02:42.006829+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:25:36.737438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"115'8","reported_seq":22,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009351+0000","last_change":"2024-05-30T17:02:39.998093+0000","last_active":"2024-05-30T17:02:42.009351+0000","last_peered":"2024-05-30T17:02:42.009351+0000","last_clean":"2024-05-30T17:02:42.009351+0000","last_became_active":"2024-05-30T17:02:39.997282+0000","last_became_peered":"2024-05-30T17:02:39.997282+0000","last_unstale":"2024-05-30T17:02:42.009351+0000","last_undegraded":"2024-05-30T17:02:42.009351+0000","last_fullsized":"2024-05-30T17:02:42.009351+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:45:31.480091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.010013+0000","last_change":"2024-05-30T17:02:41.009258+0000","last_active":"2024-05-30T17:02:42.010013+0000","last_peered":"2024-05-30T17:02:42.010013+0000","last_clean":"2024-05-30T17:02:42.010013+0000","last_became_active":"2024-05-30T17:02:41.008966+0000","last_became_peered":"2024-05-30T17:02:41.008966+0000","last_unstale":"2024-05-30T17:02:42.010013+0000","last_undegraded":"2024-05-30T17:02:42.010013+0000","last_fullsized":"2024-05-30T17:02:42.010013+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:10:20.043628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710631+0000","last_change":"2024-05-30T17:02:40.002688+0000","last_active":"2024-05-30T17:02:42.710631+0000","last_peered":"2024-05-30T17:02:42.710631+0000","last_clean":"2024-05-30T17:02:42.710631+0000","last_became_active":"2024-05-30T17:02:40.000751+0000","last_became_peered":"2024-05-30T17:02:40.000751+0000","last_unstale":"2024-05-30T17:02:42.710631+0000","last_undegraded":"2024-05-30T17:02:42.710631+0000","last_fullsized":"2024-05-30T17:02:42.710631+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:25:34.156284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188976+0000","last_change":"2024-05-30T17:02:41.023475+0000","last_active":"2024-05-30T17:02:42.188976+0000","last_peered":"2024-05-30T17:02:42.188976+0000","last_clean":"2024-05-30T17:02:42.188976+0000","last_became_active":"2024-05-30T17:02:41.023183+0000","last_became_peered":"2024-05-30T17:02:41.023183+0000","last_unstale":"2024-05-30T17:02:42.188976+0000","last_undegraded":"2024-05-30T17:02:42.188976+0000","last_fullsized":"2024-05-30T17:02:42.188976+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:02.573961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009836+0000","last_change":"2024-05-30T17:02:39.995669+0000","last_active":"2024-05-30T17:02:42.009836+0000","last_peered":"2024-05-30T17:02:42.009836+0000","last_clean":"2024-05-30T17:02:42.009836+0000","last_became_active":"2024-05-30T17:02:39.995397+0000","last_became_peered":"2024-05-30T17:02:39.995397+0000","last_unstale":"2024-05-30T17:02:42.009836+0000","last_undegraded":"2024-05-30T17:02:42.009836+0000","last_fullsized":"2024-05-30T17:02:42.009836+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:37:29.373763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738221+0000","last_change":"2024-05-30T17:02:41.022727+0000","last_active":"2024-05-30T17:02:43.738221+0000","last_peered":"2024-05-30T17:02:43.738221+0000","last_clean":"2024-05-30T17:02:43.738221+0000","last_became_active":"2024-05-30T17:02:41.022446+0000","last_became_peered":"2024-05-30T17:02:41.022446+0000","last_unstale":"2024-05-30T17:02:43.738221+0000","last_undegraded":"2024-05-30T17:02:43.738221+0000","last_fullsized":"2024-05-30T17:02:43.738221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:08.319197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015437+0000","last_change":"2024-05-30T17:02:39.998860+0000","last_active":"2024-05-30T17:02:42.015437+0000","last_peered":"2024-05-30T17:02:42.015437+0000","last_clean":"2024-05-30T17:02:42.015437+0000","last_became_active":"2024-05-30T17:02:39.997624+0000","last_became_peered":"2024-05-30T17:02:39.997624+0000","last_unstale":"2024-05-30T17:02:42.015437+0000","last_undegraded":"2024-05-30T17:02:42.015437+0000","last_fullsized":"2024-05-30T17:02:42.015437+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:59:57.588303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738864+0000","last_change":"2024-05-30T17:02:41.017247+0000","last_active":"2024-05-30T17:02:43.738864+0000","last_peered":"2024-05-30T17:02:43.738864+0000","last_clean":"2024-05-30T17:02:43.738864+0000","last_became_active":"2024-05-30T17:02:41.016943+0000","last_became_peered":"2024-05-30T17:02:41.016943+0000","last_unstale":"2024-05-30T17:02:43.738864+0000","last_undegraded":"2024-05-30T17:02:43.738864+0000","last_fullsized":"2024-05-30T17:02:43.738864+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:33:14.515707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.011769+0000","last_change":"2024-05-30T17:02:40.001321+0000","last_active":"2024-05-30T17:02:42.011769+0000","last_peered":"2024-05-30T17:02:42.011769+0000","last_clean":"2024-05-30T17:02:42.011769+0000","last_became_active":"2024-05-30T17:02:40.000949+0000","last_became_peered":"2024-05-30T17:02:40.000949+0000","last_unstale":"2024-05-30T17:02:42.011769+0000","last_undegraded":"2024-05-30T17:02:42.011769+0000","last_fullsized":"2024-05-30T17:02:42.011769+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:11:12.879603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737942+0000","last_change":"2024-05-30T17:02:41.012668+0000","last_active":"2024-05-30T17:02:43.737942+0000","last_peered":"2024-05-30T17:02:43.737942+0000","last_clean":"2024-05-30T17:02:43.737942+0000","last_became_active":"2024-05-30T17:02:41.012148+0000","last_became_peered":"2024-05-30T17:02:41.012148+0000","last_unstale":"2024-05-30T17:02:43.737942+0000","last_undegraded":"2024-05-30T17:02:43.737942+0000","last_fullsized":"2024-05-30T17:02:43.737942+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:51:48.462366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.189541+0000","last_change":"2024-05-30T17:02:39.992411+0000","last_active":"2024-05-30T17:02:42.189541+0000","last_peered":"2024-05-30T17:02:42.189541+0000","last_clean":"2024-05-30T17:02:42.189541+0000","last_became_active":"2024-05-30T17:02:39.989335+0000","last_became_peered":"2024-05-30T17:02:39.989335+0000","last_unstale":"2024-05-30T17:02:42.189541+0000","last_undegraded":"2024-05-30T17:02:42.189541+0000","last_fullsized":"2024-05-30T17:02:42.189541+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:39:35.765479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708659+0000","last_change":"2024-05-30T17:02:41.007384+0000","last_active":"2024-05-30T17:02:42.708659+0000","last_peered":"2024-05-30T17:02:42.708659+0000","last_clean":"2024-05-30T17:02:42.708659+0000","last_became_active":"2024-05-30T17:02:41.007093+0000","last_became_peered":"2024-05-30T17:02:41.007093+0000","last_unstale":"2024-05-30T17:02:42.708659+0000","last_undegraded":"2024-05-30T17:02:42.708659+0000","last_fullsized":"2024-05-30T17:02:42.708659+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:05:07.370463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009199+0000","last_change":"2024-05-30T17:02:39.993424+0000","last_active":"2024-05-30T17:02:42.009199+0000","last_peered":"2024-05-30T17:02:42.009199+0000","last_clean":"2024-05-30T17:02:42.009199+0000","last_became_active":"2024-05-30T17:02:39.993119+0000","last_became_peered":"2024-05-30T17:02:39.993119+0000","last_unstale":"2024-05-30T17:02:42.009199+0000","last_undegraded":"2024-05-30T17:02:42.009199+0000","last_fullsized":"2024-05-30T17:02:42.009199+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:25:20.784382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.737939+0000","last_change":"2024-05-30T17:02:41.008813+0000","last_active":"2024-05-30T17:02:43.737939+0000","last_peered":"2024-05-30T17:02:43.737939+0000","last_clean":"2024-05-30T17:02:43.737939+0000","last_became_active":"2024-05-30T17:02:41.008199+0000","last_became_peered":"2024-05-30T17:02:41.008199+0000","last_unstale":"2024-05-30T17:02:43.737939+0000","last_undegraded":"2024-05-30T17:02:43.737939+0000","last_fullsized":"2024-05-30T17:02:43.737939+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:34:19.448690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.188651+0000","last_change":"2024-05-30T17:02:39.996513+0000","last_active":"2024-05-30T17:02:42.188651+0000","last_peered":"2024-05-30T17:02:42.188651+0000","last_clean":"2024-05-30T17:02:42.188651+0000","last_became_active":"2024-05-30T17:02:39.995747+0000","last_became_peered":"2024-05-30T17:02:39.995747+0000","last_unstale":"2024-05-30T17:02:42.188651+0000","last_undegraded":"2024-05-30T17:02:42.188651+0000","last_fullsized":"2024-05-30T17:02:42.188651+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:26:38.544781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007292+0000","last_change":"2024-05-30T17:02:41.016390+0000","last_active":"2024-05-30T17:02:42.007292+0000","last_peered":"2024-05-30T17:02:42.007292+0000","last_clean":"2024-05-30T17:02:42.007292+0000","last_became_active":"2024-05-30T17:02:41.015267+0000","last_became_peered":"2024-05-30T17:02:41.015267+0000","last_unstale":"2024-05-30T17:02:42.007292+0000","last_undegraded":"2024-05-30T17:02:42.007292+0000","last_fullsized":"2024-05-30T17:02:42.007292+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:54:52.400582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708427+0000","last_change":"2024-05-30T17:02:40.006608+0000","last_active":"2024-05-30T17:02:42.708427+0000","last_peered":"2024-05-30T17:02:42.708427+0000","last_clean":"2024-05-30T17:02:42.708427+0000","last_became_active":"2024-05-30T17:02:40.005867+0000","last_became_peered":"2024-05-30T17:02:40.005867+0000","last_unstale":"2024-05-30T17:02:42.708427+0000","last_undegraded":"2024-05-30T17:02:42.708427+0000","last_fullsized":"2024-05-30T17:02:42.708427+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:35.438692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015333+0000","last_change":"2024-05-30T17:02:41.014137+0000","last_active":"2024-05-30T17:02:42.015333+0000","last_peered":"2024-05-30T17:02:42.015333+0000","last_clean":"2024-05-30T17:02:42.015333+0000","last_became_active":"2024-05-30T17:02:41.013859+0000","last_became_peered":"2024-05-30T17:02:41.013859+0000","last_unstale":"2024-05-30T17:02:42.015333+0000","last_undegraded":"2024-05-30T17:02:42.015333+0000","last_fullsized":"2024-05-30T17:02:42.015333+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:57:16.704465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007027+0000","last_change":"2024-05-30T17:02:39.997156+0000","last_active":"2024-05-30T17:02:42.007027+0000","last_peered":"2024-05-30T17:02:42.007027+0000","last_clean":"2024-05-30T17:02:42.007027+0000","last_became_active":"2024-05-30T17:02:39.996869+0000","last_became_peered":"2024-05-30T17:02:39.996869+0000","last_unstale":"2024-05-30T17:02:42.007027+0000","last_undegraded":"2024-05-30T17:02:42.007027+0000","last_fullsized":"2024-05-30T17:02:42.007027+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:16:21.607700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.710551+0000","last_change":"2024-05-30T17:02:41.020925+0000","last_active":"2024-05-30T17:02:42.710551+0000","last_peered":"2024-05-30T17:02:42.710551+0000","last_clean":"2024-05-30T17:02:42.710551+0000","last_became_active":"2024-05-30T17:02:41.014575+0000","last_became_peered":"2024-05-30T17:02:41.014575+0000","last_unstale":"2024-05-30T17:02:42.710551+0000","last_undegraded":"2024-05-30T17:02:42.710551+0000","last_fullsized":"2024-05-30T17:02:42.710551+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:01:15.142676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"116'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015712+0000","last_change":"2024-05-30T17:02:40.000552+0000","last_active":"2024-05-30T17:02:42.015712+0000","last_peered":"2024-05-30T17:02:42.015712+0000","last_clean":"2024-05-30T17:02:42.015712+0000","last_became_active":"2024-05-30T17:02:40.000240+0000","last_became_peered":"2024-05-30T17:02:40.000240+0000","last_unstale":"2024-05-30T17:02:42.015712+0000","last_undegraded":"2024-05-30T17:02:42.015712+0000","last_fullsized":"2024-05-30T17:02:42.015712+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:28:05.119785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709580+0000","last_change":"2024-05-30T17:02:41.015414+0000","last_active":"2024-05-30T17:02:42.709580+0000","last_peered":"2024-05-30T17:02:42.709580+0000","last_clean":"2024-05-30T17:02:42.709580+0000","last_became_active":"2024-05-30T17:02:41.013542+0000","last_became_peered":"2024-05-30T17:02:41.013542+0000","last_unstale":"2024-05-30T17:02:42.709580+0000","last_undegraded":"2024-05-30T17:02:42.709580+0000","last_fullsized":"2024-05-30T17:02:42.709580+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:24:22.067418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.709431+0000","last_change":"2024-05-30T17:02:40.004551+0000","last_active":"2024-05-30T17:02:42.709431+0000","last_peered":"2024-05-30T17:02:42.709431+0000","last_clean":"2024-05-30T17:02:42.709431+0000","last_became_active":"2024-05-30T17:02:40.004260+0000","last_became_peered":"2024-05-30T17:02:40.004260+0000","last_unstale":"2024-05-30T17:02:42.709431+0000","last_undegraded":"2024-05-30T17:02:42.709431+0000","last_fullsized":"2024-05-30T17:02:42.709431+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:05:51.093557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.015455+0000","last_change":"2024-05-30T17:02:41.017162+0000","last_active":"2024-05-30T17:02:42.015455+0000","last_peered":"2024-05-30T17:02:42.015455+0000","last_clean":"2024-05-30T17:02:42.015455+0000","last_became_active":"2024-05-30T17:02:41.016552+0000","last_became_peered":"2024-05-30T17:02:41.016552+0000","last_unstale":"2024-05-30T17:02:42.015455+0000","last_undegraded":"2024-05-30T17:02:42.015455+0000","last_fullsized":"2024-05-30T17:02:42.015455+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:51.244231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.007435+0000","last_change":"2024-05-30T17:02:39.997337+0000","last_active":"2024-05-30T17:02:42.007435+0000","last_peered":"2024-05-30T17:02:42.007435+0000","last_clean":"2024-05-30T17:02:42.007435+0000","last_became_active":"2024-05-30T17:02:39.996998+0000","last_became_peered":"2024-05-30T17:02:39.996998+0000","last_unstale":"2024-05-30T17:02:42.007435+0000","last_undegraded":"2024-05-30T17:02:42.007435+0000","last_fullsized":"2024-05-30T17:02:42.007435+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:48:50.227125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808410+0000","last_change":"2024-05-30T17:02:41.006346+0000","last_active":"2024-05-30T17:02:43.808410+0000","last_peered":"2024-05-30T17:02:43.808410+0000","last_clean":"2024-05-30T17:02:43.808410+0000","last_became_active":"2024-05-30T17:02:41.006009+0000","last_became_peered":"2024-05-30T17:02:41.006009+0000","last_unstale":"2024-05-30T17:02:43.808410+0000","last_undegraded":"2024-05-30T17:02:43.808410+0000","last_fullsized":"2024-05-30T17:02:43.808410+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:34:40.576629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012323+0000","last_change":"2024-05-30T17:02:39.996922+0000","last_active":"2024-05-30T17:02:42.012323+0000","last_peered":"2024-05-30T17:02:42.012323+0000","last_clean":"2024-05-30T17:02:42.012323+0000","last_became_active":"2024-05-30T17:02:39.996561+0000","last_became_peered":"2024-05-30T17:02:39.996561+0000","last_unstale":"2024-05-30T17:02:42.012323+0000","last_undegraded":"2024-05-30T17:02:42.012323+0000","last_fullsized":"2024-05-30T17:02:42.012323+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:16:39.184990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.809408+0000","last_change":"2024-05-30T17:02:41.008494+0000","last_active":"2024-05-30T17:02:43.809408+0000","last_peered":"2024-05-30T17:02:43.809408+0000","last_clean":"2024-05-30T17:02:43.809408+0000","last_became_active":"2024-05-30T17:02:41.005828+0000","last_became_peered":"2024-05-30T17:02:41.005828+0000","last_unstale":"2024-05-30T17:02:43.809408+0000","last_undegraded":"2024-05-30T17:02:43.809408+0000","last_fullsized":"2024-05-30T17:02:43.809408+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:23:46.955577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.808434+0000","last_change":"2024-05-30T17:02:39.999916+0000","last_active":"2024-05-30T17:02:43.808434+0000","last_peered":"2024-05-30T17:02:43.808434+0000","last_clean":"2024-05-30T17:02:43.808434+0000","last_became_active":"2024-05-30T17:02:39.999588+0000","last_became_peered":"2024-05-30T17:02:39.999588+0000","last_unstale":"2024-05-30T17:02:43.808434+0000","last_undegraded":"2024-05-30T17:02:43.808434+0000","last_fullsized":"2024-05-30T17:02:43.808434+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:58:40.604351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738381+0000","last_change":"2024-05-30T17:02:41.009038+0000","last_active":"2024-05-30T17:02:43.738381+0000","last_peered":"2024-05-30T17:02:43.738381+0000","last_clean":"2024-05-30T17:02:43.738381+0000","last_became_active":"2024-05-30T17:02:41.008540+0000","last_became_peered":"2024-05-30T17:02:41.008540+0000","last_unstale":"2024-05-30T17:02:43.738381+0000","last_undegraded":"2024-05-30T17:02:43.738381+0000","last_fullsized":"2024-05-30T17:02:43.738381+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:44.351484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.009392+0000","last_change":"2024-05-30T17:02:39.994753+0000","last_active":"2024-05-30T17:02:42.009392+0000","last_peered":"2024-05-30T17:02:42.009392+0000","last_clean":"2024-05-30T17:02:42.009392+0000","last_became_active":"2024-05-30T17:02:39.994218+0000","last_became_peered":"2024-05-30T17:02:39.994218+0000","last_unstale":"2024-05-30T17:02:42.009392+0000","last_undegraded":"2024-05-30T17:02:42.009392+0000","last_fullsized":"2024-05-30T17:02:42.009392+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:08:01.556854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708964+0000","last_change":"2024-05-30T17:02:41.012720+0000","last_active":"2024-05-30T17:02:42.708964+0000","last_peered":"2024-05-30T17:02:42.708964+0000","last_clean":"2024-05-30T17:02:42.708964+0000","last_became_active":"2024-05-30T17:02:41.010803+0000","last_became_peered":"2024-05-30T17:02:41.010803+0000","last_unstale":"2024-05-30T17:02:42.708964+0000","last_undegraded":"2024-05-30T17:02:42.708964+0000","last_fullsized":"2024-05-30T17:02:42.708964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:08:08.401182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.708658+0000","last_change":"2024-05-30T17:02:40.006576+0000","last_active":"2024-05-30T17:02:42.708658+0000","last_peered":"2024-05-30T17:02:42.708658+0000","last_clean":"2024-05-30T17:02:42.708658+0000","last_became_active":"2024-05-30T17:02:40.005830+0000","last_became_peered":"2024-05-30T17:02:40.005830+0000","last_unstale":"2024-05-30T17:02:42.708658+0000","last_undegraded":"2024-05-30T17:02:42.708658+0000","last_fullsized":"2024-05-30T17:02:42.708658+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:04:19.943415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.006825+0000","last_change":"2024-05-30T17:02:41.015001+0000","last_active":"2024-05-30T17:02:42.006825+0000","last_peered":"2024-05-30T17:02:42.006825+0000","last_clean":"2024-05-30T17:02:42.006825+0000","last_became_active":"2024-05-30T17:02:41.012609+0000","last_became_peered":"2024-05-30T17:02:41.012609+0000","last_unstale":"2024-05-30T17:02:42.006825+0000","last_undegraded":"2024-05-30T17:02:42.006825+0000","last_fullsized":"2024-05-30T17:02:42.006825+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:39.950253+0000","last_clean_scrub_stamp":"2024-05-30T17:02:39.950253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:40:37.646707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:42.012123+0000","last_change":"2024-05-30T17:02:39.990744+0000","last_active":"2024-05-30T17:02:42.012123+0000","last_peered":"2024-05-30T17:02:42.012123+0000","last_clean":"2024-05-30T17:02:42.012123+0000","last_became_active":"2024-05-30T17:02:39.990046+0000","last_became_peered":"2024-05-30T17:02:39.990046+0000","last_unstale":"2024-05-30T17:02:42.012123+0000","last_undegraded":"2024-05-30T17:02:42.012123+0000","last_fullsized":"2024-05-30T17:02:42.012123+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:02:38.933146+0000","last_clean_scrub_stamp":"2024-05-30T17:02:38.933146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:00:30.204717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":352,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-30T17:02:43.738594+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:02:43.738594+0000","last_peered":"2024-05-30T17:02:43.738594+0000","last_clean":"2024-05-30T17:02:43.738594+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:02:43.738594+0000","last_undegraded":"2024-05-30T17:02:43.738594+0000","last_fullsized":"2024-05-30T17:02:43.738594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509673,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131088,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240752,"statfs":{"total":96636764160,"available":96502530048,"internally_reserved":0,"allocated":1437696,"data_stored":628679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":132774342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509673,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144964,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226876,"statfs":{"total":96636764160,"available":96488321024,"internally_reserved":0,"allocated":1433600,"data_stored":628609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509673,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131092,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240748,"statfs":{"total":96636764160,"available":96502525952,"internally_reserved":0,"allocated":1449984,"data_stored":630291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509673,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149132,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222708,"statfs":{"total":96636764160,"available":96484052992,"internally_reserved":0,"allocated":1433600,"data_stored":628609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509672,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144980,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226860,"statfs":{"total":96636764160,"available":96488304640,"internally_reserved":0,"allocated":1441792,"data_stored":628733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542376,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131664,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240176,"statfs":{"total":96636764160,"available":96501940224,"internally_reserved":0,"allocated":2035712,"data_stored":1219593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64468,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307372,"statfs":{"total":96636764160,"available":96570748928,"internally_reserved":0,"allocated":1441792,"data_stored":629213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542377,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105308,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94266532,"statfs":{"total":96636764160,"available":96528928768,"internally_reserved":0,"allocated":2048000,"data_stored":1220701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:02:50.994 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T17:02:50.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-30T17:02:51.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.256+0000 7f64a9400640 1 Processor -- start 2024-05-30T17:02:51.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.256+0000 7f64a9400640 1 -- start start 2024-05-30T17:02:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.257+0000 7f64a9400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f64a40caa40 0x7f64a40ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.257+0000 7f64a9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a41524c0 0x7f64a40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.257+0000 7f64a9400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a40b3e80 con 0x7f64a40cae20 2024-05-30T17:02:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.257+0000 7f64a9400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f64a40d0b30 con 0x7f64a40caa40 2024-05-30T17:02:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.257+0000 7f64a9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64a40b4160 con 0x7f64a41524c0 2024-05-30T17:02:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.258+0000 7f64a3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f64a40cae20 0x7f64a4151db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55944/0 (socket says 172.21.15.67:55944) 2024-05-30T17:02:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.258+0000 7f64a3400640 1 -- 172.21.15.67:0/963993032 learned_addr learned my addr 172.21.15.67:0/963993032 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.258+0000 7f64a8a00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a41524c0 0x7f64a40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.258+0000 7f64a3e00640 1 --2- 172.21.15.67:0/963993032 >> v2:172.21.15.169:3300/0 conn(0x7f64a40caa40 0x7f64a40ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.258+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156449329 0 0) 0x7f64a40b3e80 con 0x7f64a40cae20 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.258+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6488003520 con 0x7f64a40cae20 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.258+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3567755095 0 0) 0x7f6488003520 con 0x7f64a40cae20 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.258+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a41524c0 msgr2=0x7f64a40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.258+0000 7f64a2a00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a41524c0 0x7f64a40cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 >> v2:172.21.15.169:3300/0 conn(0x7f64a40caa40 msgr2=0x7f64a40ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 --2- 172.21.15.67:0/963993032 >> v2:172.21.15.169:3300/0 conn(0x7f64a40caa40 0x7f64a40ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a40cde80 con 0x7f64a40cae20 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f6490001a10 con 0x7f64a40cae20 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 >> v1:172.21.15.67:6789/0 conn(0x7f64a40cae20 legacy=0x7f64a4151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64880039b0 0x7f6488003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a41524c0 0x7f6488004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6488004d40 0x7f64880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64a40d0b30 con 0x7f64880039b0 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64a40b4160 con 0x7f6488004d40 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f648800dc00 con 0x7f64a41524c0 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a3e00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64880039b0 0x7f6488003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a8a00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6488004d40 0x7f64880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a3e00640 1 -- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a41524c0 msgr2=0x7f6488004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a3e00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64a41524c0 0x7f6488004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a3e00640 1 -- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6488004d40 msgr2=0x7f64880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a3e00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6488004d40 0x7f64880051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.259+0000 7f64a3e00640 1 -- 172.21.15.67:0/963993032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6488003520 con 0x7f64880039b0 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.260+0000 7f64a3e00640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64880039b0 0x7f6488003dd0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f6494009ac0 tx=0x7f6488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5233cd26c4f59d31 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494036070 con 0x7f64880039b0 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494031400 con 0x7f64880039b0 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a2a00640 1 -- 172.21.15.67:0/963993032 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494009690 con 0x7f64880039b0 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a9400640 1 -- 172.21.15.67:0/963993032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64880039b0 msgr2=0x7f6488003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a9400640 1 --2- 172.21.15.67:0/963993032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64880039b0 0x7f6488003dd0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f6494009ac0 tx=0x7f6488005830 comp rx=0 tx=0).stop 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a8a00640 1 -- 172.21.15.67:0/963993032 reap_dead start 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a9400640 1 -- 172.21.15.67:0/963993032 shutdown_connections 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a9400640 1 -- 172.21.15.67:0/963993032 >> 172.21.15.67:0/963993032 conn(0x7f64a406b900 msgr2=0x7f64a406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a9400640 1 -- 172.21.15.67:0/963993032 shutdown_connections 2024-05-30T17:02:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.261+0000 7f64a9400640 1 -- 172.21.15.67:0/963993032 wait complete. 2024-05-30T17:02:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.263+0000 7f64a9400640 1 Processor -- start 2024-05-30T17:02:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.263+0000 7f64a9400640 1 -- start start 2024-05-30T17:02:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.264+0000 7f64a9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64880039b0 0x7f64a41456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.264+0000 7f64a9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64a4071a80 0x7f64a40aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.264+0000 7f64a9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a40c94d0 0x7f64a40c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.264+0000 7f64a9400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64a40ceea0 con 0x7f64a4071a80 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.264+0000 7f64a9400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64a40ce440 con 0x7f64a40c94d0 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.264+0000 7f64a9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64a40cf020 con 0x7f64880039b0 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64a4071a80 0x7f64a40aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64a4071a80 0x7f64a40aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52574/0 (socket says 172.21.15.67:52574) 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a3400640 1 -- 172.21.15.67:0/1632303114 learned_addr learned my addr 172.21.15.67:0/1632303114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a8a00640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a40c94d0 0x7f64a40c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a3400640 1 -- 172.21.15.67:0/1632303114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64880039b0 msgr2=0x7f64a41456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a3400640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64880039b0 0x7f64a41456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a3400640 1 -- 172.21.15.67:0/1632303114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a40c94d0 msgr2=0x7f64a40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a3400640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a40c94d0 0x7f64a40c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a3400640 1 -- 172.21.15.67:0/1632303114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a4071f10 con 0x7f64a4071a80 2024-05-30T17:02:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.265+0000 7f64a3400640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64a4071a80 0x7f64a40aef20 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f649000ca70 tx=0x7f649000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.266+0000 7f649be00640 1 -- 172.21.15.67:0/1632303114 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490018070 con 0x7f64a4071a80 2024-05-30T17:02:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.266+0000 7f649be00640 1 -- 172.21.15.67:0/1632303114 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649000a760 con 0x7f64a4071a80 2024-05-30T17:02:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.266+0000 7f649be00640 1 -- 172.21.15.67:0/1632303114 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649000ab50 con 0x7f64a4071a80 2024-05-30T17:02:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.266+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a40721f0 con 0x7f64a4071a80 2024-05-30T17:02:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.266+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a40727b0 con 0x7f64a4071a80 2024-05-30T17:02:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.267+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a40689e0 con 0x7f64a4071a80 2024-05-30T17:02:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.268+0000 7f649be00640 1 -- 172.21.15.67:0/1632303114 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f649001b070 con 0x7f64a4071a80 2024-05-30T17:02:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.269+0000 7f649be00640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f648406e0c0 0x7f6484070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.269+0000 7f649be00640 1 -- 172.21.15.67:0/1632303114 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f649008a230 con 0x7f64a4071a80 2024-05-30T17:02:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.269+0000 7f64a3e00640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f648406e0c0 0x7f6484070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.270+0000 7f64a3e00640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f648406e0c0 0x7f6484070580 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6494033000 tx=0x7f649402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.273+0000 7f649be00640 1 -- 172.21.15.67:0/1632303114 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f649005a0c0 con 0x7f64a4071a80 2024-05-30T17:02:51.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.542+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f64a40b4ee0 con 0x7f64a4071a80 2024-05-30T17:02:51.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.548+0000 7f649be00640 1 -- 172.21.15.67:0/1632303114 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f64900599c0 con 0x7f64a4071a80 2024-05-30T17:02:51.549 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-05-30T17:02:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T17:02:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-05-30T17:02:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-05-30T17:02:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-30T17:02:51.549 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-05-30T17:02:51.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.550+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f648406e0c0 msgr2=0x7f6484070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f648406e0c0 0x7f6484070580 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6494033000 tx=0x7f649402e000 comp rx=0 tx=0).stop 2024-05-30T17:02:51.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64a4071a80 msgr2=0x7f64a40aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64a4071a80 0x7f64a40aef20 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f649000ca70 tx=0x7f649000cf30 comp rx=0 tx=0).stop 2024-05-30T17:02:51.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 shutdown_connections 2024-05-30T17:02:51.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f648406e0c0 0x7f6484070580 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64a40c94d0 0x7f64a40c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64a4071a80 0x7f64a40aef20 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 --2- 172.21.15.67:0/1632303114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64880039b0 0x7f64a41456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 >> 172.21.15.67:0/1632303114 conn(0x7f64a406b900 msgr2=0x7f64a406e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:51.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 shutdown_connections 2024-05-30T17:02:51.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.551+0000 7f64a9400640 1 -- 172.21.15.67:0/1632303114 wait complete. 2024-05-30T17:02:51.570 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:02:51.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.775+0000 7fede4a00640 1 Processor -- start 2024-05-30T17:02:51.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.776+0000 7fede4a00640 1 -- start start 2024-05-30T17:02:51.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.777+0000 7fede4a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fede00d16a0 0x7fede00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.777+0000 7fede4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00cf870 0x7fede00d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.777+0000 7fede4a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede015ec50 con 0x7fede015e870 2024-05-30T17:02:51.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.777+0000 7fede4a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fede00d2150 con 0x7fede00d16a0 2024-05-30T17:02:51.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.777+0000 7fede4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fede00cfd20 con 0x7fede00cf870 2024-05-30T17:02:51.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.777+0000 7feddf400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fede015e870 0x7fede00d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55950/0 (socket says 172.21.15.67:55950) 2024-05-30T17:02:51.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7feddf400640 1 -- 172.21.15.67:0/3591773315 learned_addr learned my addr 172.21.15.67:0/3591773315 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:51.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7feddfe00640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00cf870 0x7fede00d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7feddea00640 1 --2- 172.21.15.67:0/3591773315 >> v2:172.21.15.169:3300/0 conn(0x7fede00d16a0 0x7fede00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 116988522 0 0) 0x7fede015ec50 con 0x7fede015e870 2024-05-30T17:02:51.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedc4003520 con 0x7fede015e870 2024-05-30T17:02:51.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3066466221 0 0) 0x7fedc4003520 con 0x7fede015e870 2024-05-30T17:02:51.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00cf870 msgr2=0x7fede00d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:51.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7fedde000640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00cf870 0x7fede00d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 >> v2:172.21.15.169:3300/0 conn(0x7fede00d16a0 msgr2=0x7fede00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7fedde000640 1 --2- 172.21.15.67:0/3591773315 >> v2:172.21.15.169:3300/0 conn(0x7fede00d16a0 0x7fede00cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede00cff80 con 0x7fede015e870 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.778+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fedd0001670 con 0x7fede015e870 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 >> v1:172.21.15.67:6789/0 conn(0x7fede015e870 legacy=0x7fede00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7fedde000640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc40039b0 0x7fedc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7fedde000640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fedc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7fedde000640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00cf870 0x7fedc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fede00d2150 con 0x7fedc4004800 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fede00cfd20 con 0x7fedc40039b0 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fedc400ba10 con 0x7fede00cf870 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7feddea00640 1 --2- 172.21.15.67:0/3591773315 >> v2:172.21.15.169:3300/0 conn(0x7fede00d16a0 0x7fede00cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7feddea00640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fedc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7feddf400640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc40039b0 0x7fedc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7feddea00640 1 -- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00cf870 msgr2=0x7fedc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:51.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.779+0000 7feddea00640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00cf870 0x7fedc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.780+0000 7feddea00640 1 -- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc40039b0 msgr2=0x7fedc4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:51.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.780+0000 7feddea00640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc40039b0 0x7fedc4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.780+0000 7feddea00640 1 -- 172.21.15.67:0/3591773315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc4003520 con 0x7fedc4004800 2024-05-30T17:02:51.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.780+0000 7feddf400640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc40039b0 0x7fedc4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.781+0000 7feddea00640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fedc4004c80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fedc80098a0 tx=0x7fedc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466970f44f49d80 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.781+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8037070 con 0x7fedc4004800 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.781+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc802ad20 con 0x7fedc4004800 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.781+0000 7fedde000640 1 -- 172.21.15.67:0/3591773315 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc800f040 con 0x7fedc4004800 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.781+0000 7fede4a00640 1 -- 172.21.15.67:0/3591773315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 msgr2=0x7fedc4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.781+0000 7fede4a00640 1 --2- 172.21.15.67:0/3591773315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fedc4004c80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fedc80098a0 tx=0x7fedc4005830 comp rx=0 tx=0).stop 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.781+0000 7feddfe00640 1 -- 172.21.15.67:0/3591773315 reap_dead start 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.781+0000 7fede4a00640 1 -- 172.21.15.67:0/3591773315 shutdown_connections 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.782+0000 7fede4a00640 1 -- 172.21.15.67:0/3591773315 >> 172.21.15.67:0/3591773315 conn(0x7fede008bfc0 msgr2=0x7fede00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.782+0000 7fede4a00640 1 -- 172.21.15.67:0/3591773315 shutdown_connections 2024-05-30T17:02:51.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.782+0000 7fede4a00640 1 -- 172.21.15.67:0/3591773315 wait complete. 2024-05-30T17:02:51.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.783+0000 7fede4a00640 1 Processor -- start 2024-05-30T17:02:51.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.783+0000 7fede4a00640 1 -- start start 2024-05-30T17:02:51.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.784+0000 7fede4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fede0159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.784+0000 7fede4a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00b98e0 0x7fede00ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.784+0000 7fede4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00b9d40 0x7fede00ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.784+0000 7fede4a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fede00d0820 con 0x7fedc4004800 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.784+0000 7fede4a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fede00d0540 con 0x7fede00b98e0 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.784+0000 7fede4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fede00d4fb0 con 0x7fede00b9d40 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fede0159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fede0159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52582/0 (socket says 172.21.15.67:52582) 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddf400640 1 -- 172.21.15.67:0/1332382677 learned_addr learned my addr 172.21.15.67:0/1332382677 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddea00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00b98e0 0x7fede00ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddfe00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00b9d40 0x7fede00ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddf400640 1 -- 172.21.15.67:0/1332382677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00b9d40 msgr2=0x7fede00ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddf400640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00b9d40 0x7fede00ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddf400640 1 -- 172.21.15.67:0/1332382677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00b98e0 msgr2=0x7fede00ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddf400640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00b98e0 0x7fede00ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:51.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.785+0000 7feddf400640 1 -- 172.21.15.67:0/1332382677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede01605c0 con 0x7fedc4004800 2024-05-30T17:02:51.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.786+0000 7feddf400640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fede0159d70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fedd000d8e0 tx=0x7fedd000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:51.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.786+0000 7fedd7400640 1 -- 172.21.15.67:0/1332382677 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0019070 con 0x7fedc4004800 2024-05-30T17:02:51.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.786+0000 7fedd7400640 1 -- 172.21.15.67:0/1332382677 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd00074e0 con 0x7fedc4004800 2024-05-30T17:02:51.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.786+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede01608a0 con 0x7fedc4004800 2024-05-30T17:02:51.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.786+0000 7fedd7400640 1 -- 172.21.15.67:0/1332382677 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd00049e0 con 0x7fedc4004800 2024-05-30T17:02:51.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.786+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede0160e60 con 0x7fedc4004800 2024-05-30T17:02:51.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.787+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede00689e0 con 0x7fedc4004800 2024-05-30T17:02:51.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.788+0000 7fedd7400640 1 -- 172.21.15.67:0/1332382677 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fedd001c070 con 0x7fedc4004800 2024-05-30T17:02:51.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.789+0000 7fedd7400640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fedbc06e0c0 0x7fedbc070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:51.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.789+0000 7fedd7400640 1 -- 172.21.15.67:0/1332382677 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fedd008a570 con 0x7fedc4004800 2024-05-30T17:02:51.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.789+0000 7feddea00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fedbc06e0c0 0x7fedbc070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:51.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.790+0000 7feddea00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fedbc06e0c0 0x7fedbc070580 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fedc8034000 tx=0x7fedc802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:51.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:51.794+0000 7fedd7400640 1 -- 172.21.15.67:0/1332382677 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedd005a400 con 0x7fedc4004800 2024-05-30T17:02:52.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.037+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fede006e550 con 0x7fedc4004800 2024-05-30T17:02:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.039+0000 7fedd7400640 1 -- 172.21.15.67:0/1332382677 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5189 (secure 0 0 0) 0x7fedd0059d00 con 0x7fedc4004800 2024-05-30T17:02:52.039 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:02:52.039 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1973461214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1973461214},{"type":"v1","addr":"172.21.15.169:6835","nonce":1973461214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":8400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3884958123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3884958123},{"type":"v1","addr":"172.21.15.67:6836","nonce":3884958123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":8403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1126347083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1126347083},{"type":"v1","addr":"172.21.15.67:6837","nonce":1126347083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:02:40.462403+0000","modified":"2024-05-30T17:02:43.518891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3425947383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3425947383},{"type":"v1","addr":"172.21.15.169:6834","nonce":3425947383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":8}]} 2024-05-30T17:02:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 73 2024-05-30T17:02:52.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.041+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fedbc06e0c0 msgr2=0x7fedbc070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.041+0000 7fede4a00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fedbc06e0c0 0x7fedbc070580 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fedc8034000 tx=0x7fedc802e000 comp rx=0 tx=0).stop 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 msgr2=0x7fede0159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fede0159d70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fedd000d8e0 tx=0x7fedd000dda0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 shutdown_connections 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fedbc06e0c0 0x7fedbc070580 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fede00b9d40 0x7fede00ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00b98e0 0x7fede00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 --2- 172.21.15.67:0/1332382677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedc4004800 0x7fede0159d70 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 >> 172.21.15.67:0/1332382677 conn(0x7fede008bfc0 msgr2=0x7fede00b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 shutdown_connections 2024-05-30T17:02:52.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.042+0000 7fede4a00640 1 -- 172.21.15.67:0/1332382677 wait complete. 2024-05-30T17:02:52.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-30T17:02:40.462403+0000', 'modified': '2024-05-30T17:02:43.518891+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8412}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8412': {'gid': 8412, 'name': 'd', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6834/3425947383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 3425947383}, {'type': 'v1', 'addr': '172.21.15.169:6834', 'nonce': 3425947383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8412, 'qdb_cluster': [8412]} 2024-05-30T17:02:52.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-30T17:02:52.464 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-30T17:02:52.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.684+0000 7f307e200640 1 Processor -- start 2024-05-30T17:02:52.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.685+0000 7f307e200640 1 -- start start 2024-05-30T17:02:52.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.686+0000 7f307e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f307806ee40 0x7f30780ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.686+0000 7f307e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f307806f2c0 0x7f30780ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.686+0000 7f307e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3078156f10 con 0x7f3078156b30 2024-05-30T17:02:52.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.686+0000 7f307e200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f307806dda0 con 0x7f307806f2c0 2024-05-30T17:02:52.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.686+0000 7f307e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f307806e080 con 0x7f307806ee40 2024-05-30T17:02:52.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.687+0000 7f3077e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f307806ee40 0x7f30780ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.687+0000 7f307ce00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f3078156b30 0x7f30780af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:35156/0 (socket says 172.21.15.67:35156) 2024-05-30T17:02:52.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.687+0000 7f307ce00640 1 -- 172.21.15.67:0/777037374 learned_addr learned my addr 172.21.15.67:0/777037374 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:52.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.687+0000 7f307d800640 1 --2- 172.21.15.67:0/777037374 >> v2:172.21.15.169:3300/0 conn(0x7f307806f2c0 0x7f30780ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.688+0000 7f3077e00640 1 -- 172.21.15.67:0/777037374 >> v2:172.21.15.169:3300/0 conn(0x7f307806f2c0 msgr2=0x7f30780ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.688+0000 7f3077e00640 1 --2- 172.21.15.67:0/777037374 >> v2:172.21.15.169:3300/0 conn(0x7f307806f2c0 0x7f30780ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.688+0000 7f3077e00640 1 -- 172.21.15.67:0/777037374 >> v1:172.21.15.169:6789/0 conn(0x7f3078156b30 legacy=0x7f30780af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.688+0000 7f3077e00640 1 -- 172.21.15.67:0/777037374 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307806e360 con 0x7f307806ee40 2024-05-30T17:02:52.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.688+0000 7f307d800640 1 --2- 172.21.15.67:0/777037374 >> v2:172.21.15.169:3300/0 conn(0x7f307806f2c0 0x7f30780ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:02:52.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.689+0000 7f3077e00640 1 --2- 172.21.15.67:0/777037374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f307806ee40 0x7f30780ad230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f30640097a0 tx=0x7f306402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38b259aa2770e83e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f3077400640 1 -- 172.21.15.67:0/777037374 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064037070 con 0x7f307806ee40 2024-05-30T17:02:52.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f3077400640 1 -- 172.21.15.67:0/777037374 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306402ae50 con 0x7f307806ee40 2024-05-30T17:02:52.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f3077400640 1 -- 172.21.15.67:0/777037374 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064032660 con 0x7f307806ee40 2024-05-30T17:02:52.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f307e200640 1 -- 172.21.15.67:0/777037374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f307806ee40 msgr2=0x7f30780ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f307e200640 1 --2- 172.21.15.67:0/777037374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f307806ee40 0x7f30780ad230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f30640097a0 tx=0x7f306402a100 comp rx=0 tx=0).stop 2024-05-30T17:02:52.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f307e200640 1 -- 172.21.15.67:0/777037374 shutdown_connections 2024-05-30T17:02:52.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f307e200640 1 --2- 172.21.15.67:0/777037374 >> v2:172.21.15.169:3300/0 conn(0x7f307806f2c0 0x7f30780ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f307e200640 1 --2- 172.21.15.67:0/777037374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f307806ee40 0x7f30780ad230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f307e200640 1 -- 172.21.15.67:0/777037374 >> 172.21.15.67:0/777037374 conn(0x7f30780b1fc0 msgr2=0x7f30780b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:52.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f307e200640 1 -- 172.21.15.67:0/777037374 shutdown_connections 2024-05-30T17:02:52.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.690+0000 7f307e200640 1 -- 172.21.15.67:0/777037374 wait complete. 2024-05-30T17:02:52.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.692+0000 7f307e200640 1 Processor -- start 2024-05-30T17:02:52.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.692+0000 7f307e200640 1 -- start start 2024-05-30T17:02:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.693+0000 7f307e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f307806ee40 0x7f30781f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.693+0000 7f307e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307806f2c0 0x7f307815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.693+0000 7f307e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3078156b30 0x7f307815eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.693+0000 7f307e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f30780b2db0 con 0x7f307806ee40 2024-05-30T17:02:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.693+0000 7f307e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30780b2c30 con 0x7f307806f2c0 2024-05-30T17:02:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.693+0000 7f307e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30780b32c0 con 0x7f3078156b30 2024-05-30T17:02:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.693+0000 7f307ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f307806ee40 0x7f30781f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f307806ee40 0x7f30781f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52596/0 (socket says 172.21.15.67:52596) 2024-05-30T17:02:52.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307ce00640 1 -- 172.21.15.67:0/3255549449 learned_addr learned my addr 172.21.15.67:0/3255549449 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f3077e00640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307806f2c0 0x7f307815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307d800640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3078156b30 0x7f307815eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307ce00640 1 -- 172.21.15.67:0/3255549449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3078156b30 msgr2=0x7f307815eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307ce00640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3078156b30 0x7f307815eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307ce00640 1 -- 172.21.15.67:0/3255549449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307806f2c0 msgr2=0x7f307815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307ce00640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307806f2c0 0x7f307815e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307ce00640 1 -- 172.21.15.67:0/3255549449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3078154fc0 con 0x7f307806ee40 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307d800640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3078156b30 0x7f307815eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.694+0000 7f307ce00640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f307806ee40 0x7f30781f4490 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f306c00c3c0 tx=0x7f306c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.695+0000 7f3074c00640 1 -- 172.21.15.67:0/3255549449 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c017070 con 0x7f307806ee40 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.695+0000 7f3074c00640 1 -- 172.21.15.67:0/3255549449 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c005ce0 con 0x7f307806ee40 2024-05-30T17:02:52.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.695+0000 7f307e200640 1 -- 172.21.15.67:0/3255549449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307815f150 con 0x7f307806ee40 2024-05-30T17:02:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.695+0000 7f3074c00640 1 -- 172.21.15.67:0/3255549449 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c0072f0 con 0x7f307806ee40 2024-05-30T17:02:52.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.695+0000 7f307e200640 1 -- 172.21.15.67:0/3255549449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3078157560 con 0x7f307806ee40 2024-05-30T17:02:52.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.698+0000 7f3074c00640 1 -- 172.21.15.67:0/3255549449 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f306c01a070 con 0x7f307806ee40 2024-05-30T17:02:52.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.699+0000 7f3074c00640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f305806e090 0x7f3058070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.699+0000 7f3074c00640 1 -- 172.21.15.67:0/3255549449 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f306c089ce0 con 0x7f307806ee40 2024-05-30T17:02:52.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.699+0000 7f3077e00640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f305806e090 0x7f3058070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.699+0000 7f30821fd740 1 Processor -- start 2024-05-30T17:02:52.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.700+0000 7f30821fd740 1 -- start start 2024-05-30T17:02:52.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.700+0000 7f3077e00640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f305806e090 0x7f3058070550 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3064034000 tx=0x7f306402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f30821fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157a490 0x55eb2157dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f30821fd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2157e150 0x55eb2159a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f30821fd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb2159aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f30821fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb214b7b10 con 0x55eb2157a490 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f30821fd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eb2145e2b0 con 0x55eb2159a5d0 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f30821fd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eb214b3ad0 con 0x55eb2157e150 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f307ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157a490 0x55eb2157dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f307ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157a490 0x55eb2157dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52602/0 (socket says 172.21.15.67:52602) 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f307ce00640 1 -- 172.21.15.67:0/3998557454 learned_addr learned my addr 172.21.15.67:0/3998557454 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f3077e00640 1 --2- 172.21.15.67:0/3998557454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2157e150 0x55eb2159a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f307ce00640 1 -- 172.21.15.67:0/3998557454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2157e150 msgr2=0x55eb2159a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f307ce00640 1 --2- 172.21.15.67:0/3998557454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2157e150 0x55eb2159a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f307ce00640 1 -- 172.21.15.67:0/3998557454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 msgr2=0x55eb2159aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f307ce00640 1 --2- 172.21.15.67:0/3998557454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb2159aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f307ce00640 1 -- 172.21.15.67:0/3998557454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb211ca2d0 con 0x55eb2157a490 2024-05-30T17:02:52.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.701+0000 7f307ce00640 1 --2- 172.21.15.67:0/3998557454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157a490 0x55eb2157dc10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f306c00cd80 tx=0x7f306c0590e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2918580456142a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f3069600640 1 -- 172.21.15.67:0/3998557454 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c055ae0 con 0x55eb2157a490 2024-05-30T17:02:52.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f3069600640 1 -- 172.21.15.67:0/3998557454 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c007490 con 0x55eb2157a490 2024-05-30T17:02:52.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f3069600640 1 -- 172.21.15.67:0/3998557454 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c055c80 con 0x55eb2157a490 2024-05-30T17:02:52.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f30821fd740 1 -- 172.21.15.67:0/3998557454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157a490 msgr2=0x55eb2157dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f30821fd740 1 --2- 172.21.15.67:0/3998557454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157a490 0x55eb2157dc10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f306c00cd80 tx=0x7f306c0590e0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f30821fd740 1 -- 172.21.15.67:0/3998557454 shutdown_connections 2024-05-30T17:02:52.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f30821fd740 1 --2- 172.21.15.67:0/3998557454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb2159aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f30821fd740 1 --2- 172.21.15.67:0/3998557454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2157e150 0x55eb2159a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f30821fd740 1 --2- 172.21.15.67:0/3998557454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157a490 0x55eb2157dc10 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f30821fd740 1 -- 172.21.15.67:0/3998557454 >> 172.21.15.67:0/3998557454 conn(0x55eb21550470 msgr2=0x55eb21551470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:52.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.702+0000 7f30821fd740 1 -- 172.21.15.67:0/3998557454 shutdown_connections 2024-05-30T17:02:52.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.703+0000 7f30821fd740 1 -- 172.21.15.67:0/3998557454 wait complete. 2024-05-30T17:02:52.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.703+0000 7f30821fd740 1 -- start start 2024-05-30T17:02:52.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.703+0000 7f30821fd740 1 Processor -- start 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.704+0000 7f30821fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb215764f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.704+0000 7f30821fd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb21576f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.704+0000 7f30821fd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb21577450 0x55eb215778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.704+0000 7f30821fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb214b7b10 con 0x55eb2157e150 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.704+0000 7f30821fd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eb21471570 con 0x55eb2159a5d0 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.704+0000 7f30821fd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eb2154e2a0 con 0x55eb21577450 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.704+0000 7f30821fd740 10 client.0 _ll_register_callbacks cb 0x55eb2143ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.704+0000 7f307ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb215764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb215764f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52608/0 (socket says 172.21.15.67:52608) 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307ce00640 1 -- 172.21.15.67:0/2840262878 learned_addr learned my addr 172.21.15.67:0/2840262878 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307d800640 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb21577450 0x55eb215778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f3077e00640 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb21576f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307ce00640 1 -- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb21577450 msgr2=0x55eb215778d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307ce00640 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb21577450 0x55eb215778d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307ce00640 1 -- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 msgr2=0x55eb21576f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307ce00640 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb21576f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307ce00640 1 -- 172.21.15.67:0/2840262878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb212946b0 con 0x55eb2157e150 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307d800640 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb21577450 0x55eb215778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f3077e00640 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb21576f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f307ce00640 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb215764f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f306c057720 tx=0x7f306c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b168c891e6fbc28 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.705+0000 7f305e000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f305e000640 1 -- 172.21.15.67:0/2840262878 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c075030 con 0x55eb2157e150 2024-05-30T17:02:52.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f305e000640 1 -- 172.21.15.67:0/2840262878 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f306c088070 con 0x55eb2157e150 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f30821fd740 1 -- 172.21.15.67:0/2840262878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f306c007490 con 0x55eb2157e150 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f305e000640 1 -- 172.21.15.67:0/2840262878 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f306c058ad0 con 0x55eb2157e150 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f30821fd740 10 client.8562 fetch_fsmap learned FSMap version 73 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f30821fd740 1 -- 172.21.15.67:0/2840262878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55eb2157f450 con 0x55eb2157e150 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f305e000640 1 -- 172.21.15.67:0/2840262878 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c007490 con 0x55eb2157e150 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f305e000640 1 -- 172.21.15.67:0/2840262878 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f306c0551c0 con 0x55eb2157e150 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f30821fd740 10 client.8562 fetch_fsmap finished waiting for FSMap version 73 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f30821fd740 10 client.8562 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f30821fd740 20 client.8562 populate_metadata read hostname 'smithi067' 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f30821fd740 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] conn(0x55eb21594e00 0x55eb215951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f30821fd740 4 client.8562 mds_command: new command op to 8412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.706+0000 7f30821fd740 1 -- 172.21.15.67:0/2840262878 --> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb214c5530 con 0x55eb21594e00 2024-05-30T17:02:52.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.707+0000 7f3077e00640 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] conn(0x55eb21594e00 0x55eb215951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.708+0000 7f3077e00640 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] conn(0x55eb21594e00 0x55eb215951e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb366c82d74b09a server_cookie=3caa55b133175b56 in_seq=0 out_seq=0 2024-05-30T17:02:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.708+0000 7f305e000640 10 client.8562 ms_handle_connect on v2:172.21.15.169:6832/3425947383 2024-05-30T17:02:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f305e000640 1 -- 172.21.15.67:0/2840262878 <== mds.0 v2:172.21.15.169:6832/3425947383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb214c5530 con 0x55eb21594e00 2024-05-30T17:02:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f305e000640 10 client.8562 handle_command_reply: tid=0 2024-05-30T17:02:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 client.8562 shutdown 2024-05-30T17:02:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 -- 172.21.15.67:0/2840262878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 msgr2=0x55eb215764f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb215764f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f306c057720 tx=0x7f306c000ed0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 -- 172.21.15.67:0/2840262878 shutdown_connections 2024-05-30T17:02:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] conn(0x55eb21594e00 0x55eb215951e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb21577450 0x55eb215778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb21576f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 --2- 172.21.15.67:0/2840262878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb215764f0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 -- 172.21.15.67:0/2840262878 >> 172.21.15.67:0/2840262878 conn(0x55eb2157a490 msgr2=0x55eb21553e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:52.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f305e000640 0 client.8562 ms_handle_reset on v2:172.21.15.169:6832/3425947383 2024-05-30T17:02:52.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 -- 172.21.15.67:0/2840262878 shutdown_connections 2024-05-30T17:02:52.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.711+0000 7f30821fd740 1 -- 172.21.15.67:0/2840262878 wait complete. 2024-05-30T17:02:52.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.712+0000 7f30821fd740 20 client.8562 trim_cache size 0 max 16384 2024-05-30T17:02:52.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.731+0000 7f30821fd740 1 Processor -- start 2024-05-30T17:02:52.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.731+0000 7f30821fd740 1 -- start start 2024-05-30T17:02:52.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.732+0000 7f30821fd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb21577450 0x55eb21577890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.732+0000 7f30821fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb21577dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.732+0000 7f30821fd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2159a5d0 0x55eb21595d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.732+0000 7f30821fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f30781ffee0 con 0x55eb2157e150 2024-05-30T17:02:52.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.732+0000 7f30821fd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30781ff340 con 0x55eb21577450 2024-05-30T17:02:52.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.732+0000 7f30821fd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30781ff890 con 0x55eb2159a5d0 2024-05-30T17:02:52.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.732+0000 7f3077e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb21577dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.732+0000 7f3077e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb21577dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52610/0 (socket says 172.21.15.67:52610) 2024-05-30T17:02:52.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.733+0000 7f3077e00640 1 -- 172.21.15.67:0/1585108820 learned_addr learned my addr 172.21.15.67:0/1585108820 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.733+0000 7f307ce00640 1 --2- 172.21.15.67:0/1585108820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb21577450 0x55eb21577890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.733+0000 7f307d800640 1 --2- 172.21.15.67:0/1585108820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2159a5d0 0x55eb21595d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.733+0000 7f3077e00640 1 -- 172.21.15.67:0/1585108820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2159a5d0 msgr2=0x55eb21595d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.733+0000 7f3077e00640 1 --2- 172.21.15.67:0/1585108820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2159a5d0 0x55eb21595d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.733+0000 7f3077e00640 1 -- 172.21.15.67:0/1585108820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb21577450 msgr2=0x55eb21577890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.733+0000 7f3077e00640 1 --2- 172.21.15.67:0/1585108820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb21577450 0x55eb21577890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.733+0000 7f3077e00640 1 -- 172.21.15.67:0/1585108820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb21553fd0 con 0x55eb2157e150 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.733+0000 7f3077e00640 1 --2- 172.21.15.67:0/1585108820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb21577dd0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f3064009560 tx=0x7f306402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914184d403eb41d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f3068c00640 1 -- 172.21.15.67:0/1585108820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306404f030 con 0x55eb2157e150 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f3068c00640 1 -- 172.21.15.67:0/1585108820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306404b050 con 0x55eb2157e150 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f3068c00640 1 -- 172.21.15.67:0/1585108820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064037040 con 0x55eb2157e150 2024-05-30T17:02:52.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f30821fd740 1 -- 172.21.15.67:0/1585108820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 msgr2=0x55eb21577dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f30821fd740 1 --2- 172.21.15.67:0/1585108820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb21577dd0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f3064009560 tx=0x7f306402c000 comp rx=0 tx=0).stop 2024-05-30T17:02:52.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f30821fd740 1 -- 172.21.15.67:0/1585108820 shutdown_connections 2024-05-30T17:02:52.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f30821fd740 1 --2- 172.21.15.67:0/1585108820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2159a5d0 0x55eb21595d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f30821fd740 1 --2- 172.21.15.67:0/1585108820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb2157e150 0x55eb21577dd0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f30821fd740 1 --2- 172.21.15.67:0/1585108820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb21577450 0x55eb21577890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f30821fd740 1 -- 172.21.15.67:0/1585108820 >> 172.21.15.67:0/1585108820 conn(0x55eb2157a490 msgr2=0x55eb21598be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:52.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f30821fd740 1 -- 172.21.15.67:0/1585108820 shutdown_connections 2024-05-30T17:02:52.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.734+0000 7f30821fd740 1 -- 172.21.15.67:0/1585108820 wait complete. 2024-05-30T17:02:52.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.735+0000 7f30821fd740 1 -- start start 2024-05-30T17:02:52.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.735+0000 7f30821fd740 1 Processor -- start 2024-05-30T17:02:52.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f30821fd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb21577450 0x55eb21596460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f30821fd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2157e150 0x55eb216118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f30821fd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb21577890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f30821fd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eb214b7b10 con 0x55eb21577450 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f30821fd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eb2154e2a0 con 0x55eb2159a5d0 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f30821fd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eb21471570 con 0x55eb2157e150 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f30821fd740 10 client.0 _ll_register_callbacks cb 0x55eb2143ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f307ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb21577450 0x55eb21596460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f307ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb21577450 0x55eb21596460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52616/0 (socket says 172.21.15.67:52616) 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.736+0000 7f307ce00640 1 -- 172.21.15.67:0/2826537516 learned_addr learned my addr 172.21.15.67:0/2826537516 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f307ce00640 1 -- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2157e150 msgr2=0x55eb216118e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f307ce00640 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2157e150 0x55eb216118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f307ce00640 1 -- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 msgr2=0x55eb21577890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f307ce00640 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb21577890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f307ce00640 1 -- 172.21.15.67:0/2826537516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2158e5a0 con 0x55eb21577450 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f307ce00640 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb21577450 0x55eb21596460 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f306c0593b0 tx=0x7f306c0557d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7ad5f8cd03b699 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f305e000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f305e000640 1 -- 172.21.15.67:0/2826537516 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c075070 con 0x55eb21577450 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f305e000640 1 -- 172.21.15.67:0/2826537516 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f306c05e9b0 con 0x55eb21577450 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f30821fd740 1 -- 172.21.15.67:0/2826537516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f306404b050 con 0x55eb21577450 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f305e000640 1 -- 172.21.15.67:0/2826537516 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f306c0833c0 con 0x55eb21577450 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f30821fd740 10 client.8568 fetch_fsmap learned FSMap version 73 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.737+0000 7f30821fd740 1 -- 172.21.15.67:0/2826537516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55eb2158e5a0 con 0x55eb21577450 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.738+0000 7f305e000640 1 -- 172.21.15.67:0/2826537516 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306404b050 con 0x55eb21577450 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.738+0000 7f305e000640 1 -- 172.21.15.67:0/2826537516 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f306c064060 con 0x55eb21577450 2024-05-30T17:02:52.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.738+0000 7f30821fd740 10 client.8568 fetch_fsmap finished waiting for FSMap version 73 2024-05-30T17:02:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.738+0000 7f30821fd740 10 client.8568 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-30T17:02:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.738+0000 7f30821fd740 20 client.8568 populate_metadata read hostname 'smithi067' 2024-05-30T17:02:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.738+0000 7f30821fd740 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] conn(0x55eb21577dd0 0x55eb2159c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.738+0000 7f30821fd740 4 client.8568 mds_command: new command op to 8412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-30T17:02:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.738+0000 7f30821fd740 1 -- 172.21.15.67:0/2826537516 --> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] -- command(tid 0: {"prefix": "status"}) -- 0x55eb21593670 con 0x55eb21577dd0 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.738+0000 7f3077e00640 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] conn(0x55eb21577dd0 0x55eb2159c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.739+0000 7f3077e00640 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] conn(0x55eb21577dd0 0x55eb2159c190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99878196b8b338f server_cookie=3e66e0052e605566 in_seq=0 out_seq=0 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.739+0000 7f305e000640 10 client.8568 ms_handle_connect on v2:172.21.15.169:6832/3425947383 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f305e000640 1 -- 172.21.15.67:0/2826537516 <== mds.0 v2:172.21.15.169:6832/3425947383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55eb21593670 con 0x55eb21577dd0 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f305e000640 10 client.8568 handle_command_reply: tid=0 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 client.8568 shutdown 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 -- 172.21.15.67:0/2826537516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb21577450 msgr2=0x55eb21596460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb21577450 0x55eb21596460 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f306c0593b0 tx=0x7f306c0557d0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 -- 172.21.15.67:0/2826537516 shutdown_connections 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:6832/3425947383,v1:172.21.15.169:6834/3425947383] conn(0x55eb21577dd0 0x55eb2159c190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb2159a5d0 0x55eb21577890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb2157e150 0x55eb216118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 --2- 172.21.15.67:0/2826537516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eb21577450 0x55eb21596460 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f305e000640 0 client.8568 ms_handle_reset on v2:172.21.15.169:6832/3425947383 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 -- 172.21.15.67:0/2826537516 >> 172.21.15.67:0/2826537516 conn(0x55eb2157a490 msgr2=0x55eb21581410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.740+0000 7f30821fd740 1 -- 172.21.15.67:0/2826537516 shutdown_connections 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f30821fd740 1 -- 172.21.15.67:0/2826537516 wait complete. 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f30821fd740 20 client.8568 trim_cache size 0 max 16384 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-05-30T17:02:52.744 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8412, 2024-05-30T17:02:52.745 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-05-30T17:02:52.745 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-05-30T17:02:52.745 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-05-30T17:02:52.745 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 11.746965021999999, 2024-05-30T17:02:52.745 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 73, 2024-05-30T17:02:52.745 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 116, 2024-05-30T17:02:52.745 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 116, 2024-05-30T17:02:52.745 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 16.104288842999999 2024-05-30T17:02:52.745 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 -- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f305806e090 msgr2=0x7f3058070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f305806e090 0x7f3058070550 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3064034000 tx=0x7f306402e000 comp rx=0 tx=0).stop 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 -- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f307806ee40 msgr2=0x7f30781f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f307806ee40 0x7f30781f4490 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f306c00c3c0 tx=0x7f306c00c880 comp rx=0 tx=0).stop 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 -- 172.21.15.67:0/3255549449 shutdown_connections 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f305806e090 0x7f3058070550 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3078156b30 0x7f307815eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f307806f2c0 0x7f307815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 --2- 172.21.15.67:0/3255549449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f307806ee40 0x7f30781f4490 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.741+0000 7f307e200640 1 -- 172.21.15.67:0/3255549449 >> 172.21.15.67:0/3255549449 conn(0x7f30780b1fc0 msgr2=0x7f30780b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:52.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.742+0000 7f307e200640 1 -- 172.21.15.67:0/3255549449 shutdown_connections 2024-05-30T17:02:52.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.742+0000 7f307e200640 1 -- 172.21.15.67:0/3255549449 wait complete. 2024-05-30T17:02:52.758 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T17:02:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.952+0000 7efee2600640 1 Processor -- start 2024-05-30T17:02:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.953+0000 7efee2600640 1 -- start start 2024-05-30T17:02:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.954+0000 7efee2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efedc161d00 0x7efedc0d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.954+0000 7efee2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efedc0cfc70 0x7efedc0b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.954+0000 7efee2600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efedc0d5360 con 0x7efedc0d3940 2024-05-30T17:02:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.954+0000 7efee2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7efedc0d2670 con 0x7efedc0cfc70 2024-05-30T17:02:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.954+0000 7efee2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efedc0d34b0 con 0x7efedc161d00 2024-05-30T17:02:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.955+0000 7efee1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efedc0cfc70 0x7efedc0b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.955+0000 7efee1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efedc0cfc70 0x7efedc0b38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52620/0 (socket says 172.21.15.67:52620) 2024-05-30T17:02:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.955+0000 7efee1c00640 1 -- 172.21.15.67:0/3973642173 learned_addr learned my addr 172.21.15.67:0/3973642173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.955+0000 7efee1200640 1 --2- 172.21.15.67:0/3973642173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efedc161d00 0x7efedc0d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.955+0000 7efee1c00640 1 -- 172.21.15.67:0/3973642173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efedc161d00 msgr2=0x7efedc0d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.955+0000 7efee1c00640 1 --2- 172.21.15.67:0/3973642173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efedc161d00 0x7efedc0d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.955+0000 7efee1c00640 1 -- 172.21.15.67:0/3973642173 >> v1:172.21.15.169:6789/0 conn(0x7efedc0d3940 legacy=0x7efedc0cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.955+0000 7efee1c00640 1 -- 172.21.15.67:0/3973642173 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc0d28a0 con 0x7efedc0cfc70 2024-05-30T17:02:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.956+0000 7efee1c00640 1 --2- 172.21.15.67:0/3973642173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 0x7efedc0b38f0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7efed4009870 tx=0x7efed402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed2b3685aba1cd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.957+0000 7efedb400640 1 -- 172.21.15.67:0/3973642173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4037070 con 0x7efedc0cfc70 2024-05-30T17:02:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.957+0000 7efedb400640 1 -- 172.21.15.67:0/3973642173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed4032440 con 0x7efedc0cfc70 2024-05-30T17:02:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.957+0000 7efedb400640 1 -- 172.21.15.67:0/3973642173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4032810 con 0x7efedc0cfc70 2024-05-30T17:02:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.957+0000 7efee2600640 1 -- 172.21.15.67:0/3973642173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 msgr2=0x7efedc0b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.958+0000 7efee2600640 1 --2- 172.21.15.67:0/3973642173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 0x7efedc0b38f0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7efed4009870 tx=0x7efed402a180 comp rx=0 tx=0).stop 2024-05-30T17:02:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.958+0000 7efee2600640 1 -- 172.21.15.67:0/3973642173 shutdown_connections 2024-05-30T17:02:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.958+0000 7efee2600640 1 --2- 172.21.15.67:0/3973642173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 0x7efedc0b38f0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.958+0000 7efee2600640 1 --2- 172.21.15.67:0/3973642173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efedc161d00 0x7efedc0d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.958+0000 7efee2600640 1 -- 172.21.15.67:0/3973642173 >> 172.21.15.67:0/3973642173 conn(0x7efedc08bfc0 msgr2=0x7efedc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:52.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.958+0000 7efee2600640 1 -- 172.21.15.67:0/3973642173 shutdown_connections 2024-05-30T17:02:52.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.958+0000 7efee2600640 1 -- 172.21.15.67:0/3973642173 wait complete. 2024-05-30T17:02:52.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.959+0000 7efee2600640 1 Processor -- start 2024-05-30T17:02:52.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.960+0000 7efee2600640 1 -- start start 2024-05-30T17:02:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.960+0000 7efee2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 0x7efedc1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efedc0d3940 0x7efedc147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efedc161d00 0x7efedc15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0d3140 con 0x7efedc0cfc70 2024-05-30T17:02:52.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0d2e60 con 0x7efedc0d3940 2024-05-30T17:02:52.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efedc1620e0 con 0x7efedc161d00 2024-05-30T17:02:52.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 0x7efedc1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 0x7efedc1590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52636/0 (socket says 172.21.15.67:52636) 2024-05-30T17:02:52.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee1200640 1 -- 172.21.15.67:0/275306776 learned_addr learned my addr 172.21.15.67:0/275306776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:52.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee1200640 1 -- 172.21.15.67:0/275306776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efedc161d00 msgr2=0x7efedc15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee1200640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efedc161d00 0x7efedc15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee1200640 1 -- 172.21.15.67:0/275306776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efedc0d3940 msgr2=0x7efedc147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efedbe00640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efedc0d3940 0x7efedc147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee1200640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efedc0d3940 0x7efedc147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efee1200640 1 -- 172.21.15.67:0/275306776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc158650 con 0x7efedc0cfc70 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.961+0000 7efedbe00640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efedc0d3940 0x7efedc147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.962+0000 7efee1200640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 0x7efedc1590f0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7efec400e8a0 tx=0x7efec400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.962+0000 7efed8c00640 1 -- 172.21.15.67:0/275306776 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4017070 con 0x7efedc0cfc70 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.962+0000 7efed8c00640 1 -- 172.21.15.67:0/275306776 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec400c940 con 0x7efedc0cfc70 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.962+0000 7efed8c00640 1 -- 172.21.15.67:0/275306776 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec400cd30 con 0x7efedc0cfc70 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.962+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc147d80 con 0x7efedc0cfc70 2024-05-30T17:02:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.962+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc1588f0 con 0x7efedc0cfc70 2024-05-30T17:02:52.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.963+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efedc0cf680 con 0x7efedc0cfc70 2024-05-30T17:02:52.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.965+0000 7efed8c00640 1 -- 172.21.15.67:0/275306776 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7efec401a070 con 0x7efedc0cfc70 2024-05-30T17:02:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.965+0000 7efed8c00640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efeb806e090 0x7efeb8070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.966+0000 7efed8c00640 1 -- 172.21.15.67:0/275306776 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7efec408a780 con 0x7efedc0cfc70 2024-05-30T17:02:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.966+0000 7efedbe00640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efeb806e090 0x7efeb8070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.966+0000 7efedbe00640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efeb806e090 0x7efeb8070550 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efecc007fd0 tx=0x7efecc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:52.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:52.970+0000 7efed8c00640 1 -- 172.21.15.67:0/275306776 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efedc0cf680 con 0x7efedc0cfc70 2024-05-30T17:02:53.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.219+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efedc0689e0 con 0x7efedc0cfc70 2024-05-30T17:02:53.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.220+0000 7efed8c00640 1 -- 172.21.15.67:0/275306776 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) ==== 53+0+73 (secure 0 0 0) 0x7efec405a6b0 con 0x7efedc0cfc70 2024-05-30T17:02:53.221 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efeb806e090 msgr2=0x7efeb8070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efeb806e090 0x7efeb8070550 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efecc007fd0 tx=0x7efecc009040 comp rx=0 tx=0).stop 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 msgr2=0x7efedc1590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 0x7efedc1590f0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7efec400e8a0 tx=0x7efec400ed60 comp rx=0 tx=0).stop 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 shutdown_connections 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efeb806e090 0x7efeb8070550 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efedc161d00 0x7efedc15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efedc0d3940 0x7efedc147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 --2- 172.21.15.67:0/275306776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efedc0cfc70 0x7efedc1590f0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 >> 172.21.15.67:0/275306776 conn(0x7efedc08bfc0 msgr2=0x7efedc06ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 shutdown_connections 2024-05-30T17:02:53.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53.223+0000 7efee2600640 1 -- 172.21.15.67:0/275306776 wait complete. 2024-05-30T17:02:53.243 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T17:02:53.243 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T17:02:53.243 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T17:02:53.243 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-30T17:02:53.243 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T17:02:53.243 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T17:02:53.243 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T17:02:53.243 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T17:02:53.243 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T17:02:53.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:53.244 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T17:02:53.298 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T17:02:53.298 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T17:02:53.299 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41399sec preferred_lft 41399sec 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::74f5:cfff:fec0:693d/64 scope link 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::a8fe:aeff:fef7:f6a0/64 scope link 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:02:53.300 INFO:teuthology.orchestra.run.smithi067.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T17:02:53.301 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:02:53.301 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::185e:a4ff:fec6:7d8c/64 scope link 2024-05-30T17:02:53.301 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:02:53.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:53.302 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T17:02:53.388 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T17:02:53.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:53.393 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T17:02:53.449 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T17:02:53.449 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T17:02:53.449 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T17:02:53.449 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T17:02:53.450 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T17:02:53.450 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T17:02:53.450 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T17:02:53.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T17:02:53.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T17:02:53.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T17:02:53.450 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T17:02:53.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T17:02:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T17:02:53.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:53.672 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T17:02:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T17:02:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-30T17:02:53.723 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T17:02:53.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:53.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-05-30T17:02:53.811 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-30T17:02:53.811 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T17:02:53.811 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-30T17:02:53.864 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-30T17:02:53.864 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T17:02:53.864 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-30T17:02:53.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T17:02:54.002 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T17:02:54.062 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T17:02:54.062 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T17:02:54.062 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T17:02:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:02:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:02:54.065 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:02:54.065 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:02:54.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:54.066 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T17:02:54.118 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T17:02:54.119 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-30T17:02:54.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T17:02:54.189 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T17:02:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T17:02:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:02:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:02:54.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:54.232 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T17:02:54.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-05-30T17:02:54.292+0000 7fdc5c579480 -1 init, newargv = 0x55d8be632b60 newargc=15 2024-05-30T17:02:54.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[74543]: starting ceph client 2024-05-30T17:02:54.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[74543]: starting fuse 2024-05-30T17:02:54.319 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T17:02:54.319 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T17:02:54.320 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-05-30T17:02:54.321 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T17:02:54.510 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.74543.asok 2024-05-30T17:02:54.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T17:02:54.527 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74543.asok 2024-05-30T17:02:54.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:54.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74543.asok status 2024-05-30T17:02:54.767 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "74543", 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8580, 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T17:02:54.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8580 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 541401971 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 541401971 2024-05-30T17:02:54.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:02:54.770 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.8580 192.168.0.1:0/541401971", 2024-05-30T17:02:54.770 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/541401971", 2024-05-30T17:02:54.770 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T17:02:54.770 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 73, 2024-05-30T17:02:54.770 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 116, 2024-05-30T17:02:54.770 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T17:02:54.770 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T17:02:54.770 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T17:02:54.770 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:02:54.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:54.786 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T17:02:54.786 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-30T17:02:54.842 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T17:02:54.842 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-30T17:02:54.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:54.842 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-30T17:02:54.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-30T17:02:55.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.133+0000 7fbe9fe00640 1 Processor -- start 2024-05-30T17:02:55.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.133+0000 7fbe9fe00640 1 -- start start 2024-05-30T17:02:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.134+0000 7fbe9fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815b340 0x7fbe9806f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.134+0000 7fbe9fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbe9815c290 0x7fbe980b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.135+0000 7fbe9fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe9806fa90 con 0x7fbe9815b720 2024-05-30T17:02:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.135+0000 7fbe9fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbe980af9b0 con 0x7fbe9815c290 2024-05-30T17:02:55.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.135+0000 7fbe9fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe980aeda0 con 0x7fbe9815b340 2024-05-30T17:02:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.135+0000 7fbe9d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815b340 0x7fbe9806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.135+0000 7fbe9d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815b340 0x7fbe9806f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:40522/0 (socket says 172.21.15.67:40522) 2024-05-30T17:02:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.135+0000 7fbe9d400640 1 -- 172.21.15.67:0/3232267204 learned_addr learned my addr 172.21.15.67:0/3232267204 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.135+0000 7fbe9de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbe9815c290 0x7fbe980b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:55.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.135+0000 7fbe9ca00640 1 --1- 172.21.15.67:0/3232267204 >> v1:172.21.15.169:6789/0 conn(0x7fbe9815b720 0x7fbe9815bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:35168/0 (socket says 172.21.15.67:35168) 2024-05-30T17:02:55.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.136+0000 7fbe9de00640 1 -- 172.21.15.67:0/3232267204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815b340 msgr2=0x7fbe9806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:55.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.136+0000 7fbe9de00640 1 --2- 172.21.15.67:0/3232267204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815b340 0x7fbe9806f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:55.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.136+0000 7fbe9de00640 1 -- 172.21.15.67:0/3232267204 >> v1:172.21.15.169:6789/0 conn(0x7fbe9815b720 legacy=0x7fbe9815bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:55.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.136+0000 7fbe9de00640 1 -- 172.21.15.67:0/3232267204 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe980af080 con 0x7fbe9815c290 2024-05-30T17:02:55.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.136+0000 7fbe9d400640 1 --2- 172.21.15.67:0/3232267204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815b340 0x7fbe9806f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:02:55.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.137+0000 7fbe9de00640 1 --2- 172.21.15.67:0/3232267204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815c290 0x7fbe980b3300 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d90 tx=0x7fbe9002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbead4edbcdac8de server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.138+0000 7fbe97e00640 1 -- 172.21.15.67:0/3232267204 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90007740 con 0x7fbe9815c290 2024-05-30T17:02:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.138+0000 7fbe97e00640 1 -- 172.21.15.67:0/3232267204 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe900078e0 con 0x7fbe9815c290 2024-05-30T17:02:55.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.138+0000 7fbe97e00640 1 -- 172.21.15.67:0/3232267204 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90007cb0 con 0x7fbe9815c290 2024-05-30T17:02:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.138+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3232267204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815c290 msgr2=0x7fbe980b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.138+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3232267204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815c290 0x7fbe980b3300 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d90 tx=0x7fbe9002a660 comp rx=0 tx=0).stop 2024-05-30T17:02:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.139+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3232267204 shutdown_connections 2024-05-30T17:02:55.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.139+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3232267204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815c290 0x7fbe980b3300 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:55.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.139+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3232267204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815b340 0x7fbe9806f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:55.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.139+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3232267204 >> 172.21.15.67:0/3232267204 conn(0x7fbe980b1e50 msgr2=0x7fbe980b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:55.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.139+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3232267204 shutdown_connections 2024-05-30T17:02:55.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.139+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3232267204 wait complete. 2024-05-30T17:02:55.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.140+0000 7fbe9fe00640 1 Processor -- start 2024-05-30T17:02:55.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.141+0000 7fbe9fe00640 1 -- start start 2024-05-30T17:02:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.141+0000 7fbe9fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe9815b340 0x7fbe980ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.141+0000 7fbe9fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815b720 0x7fbe980cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815c290 0x7fbe9815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe980aea30 con 0x7fbe9815b340 2024-05-30T17:02:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe980af640 con 0x7fbe9815b720 2024-05-30T17:02:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9815c8c0 con 0x7fbe9815c290 2024-05-30T17:02:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe9815b340 0x7fbe980ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe9815b340 0x7fbe980ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52646/0 (socket says 172.21.15.67:52646) 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9d400640 1 -- 172.21.15.67:0/3789834816 learned_addr learned my addr 172.21.15.67:0/3789834816 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9ca00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815b720 0x7fbe980cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9d400640 1 -- 172.21.15.67:0/3789834816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815c290 msgr2=0x7fbe9815d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9de00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815c290 0x7fbe9815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9d400640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815c290 0x7fbe9815d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9d400640 1 -- 172.21.15.67:0/3789834816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815b720 msgr2=0x7fbe980cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9d400640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815b720 0x7fbe980cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9d400640 1 -- 172.21.15.67:0/3789834816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9809d3f0 con 0x7fbe9815b340 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.142+0000 7fbe9ca00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815b720 0x7fbe980cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.143+0000 7fbe9d400640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe9815b340 0x7fbe980ccb20 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00ea00 tx=0x7fbe8c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.143+0000 7fbe95600640 1 -- 172.21.15.67:0/3789834816 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c017070 con 0x7fbe9815b340 2024-05-30T17:02:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.143+0000 7fbe95600640 1 -- 172.21.15.67:0/3789834816 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8c00ca60 con 0x7fbe9815b340 2024-05-30T17:02:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.143+0000 7fbe95600640 1 -- 172.21.15.67:0/3789834816 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00ce10 con 0x7fbe9815b340 2024-05-30T17:02:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.143+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe981f2ad0 con 0x7fbe9815b340 2024-05-30T17:02:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.143+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe981f3090 con 0x7fbe9815b340 2024-05-30T17:02:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.144+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe980689e0 con 0x7fbe9815b340 2024-05-30T17:02:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.145+0000 7fbe95600640 1 -- 172.21.15.67:0/3789834816 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fbe8c01a070 con 0x7fbe9815b340 2024-05-30T17:02:55.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.147+0000 7fbe95600640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7806e0c0 0x7fbe78070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.147+0000 7fbe95600640 1 -- 172.21.15.67:0/3789834816 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fbe8c08ac50 con 0x7fbe9815b340 2024-05-30T17:02:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.147+0000 7fbe9ca00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7806e0c0 0x7fbe78070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:55.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.148+0000 7fbe9ca00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7806e0c0 0x7fbe78070580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbe980b1e20 tx=0x7fbe84009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:55.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.152+0000 7fbe95600640 1 -- 172.21.15.67:0/3789834816 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe8c05aae0 con 0x7fbe9815b340 2024-05-30T17:02:55.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:55.400+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fbe980b4e60 con 0x7fbe9815b340 2024-05-30T17:02:56.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.129+0000 7fbe95600640 1 -- 172.21.15.67:0/3789834816 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v74) ==== 136+0+0 (secure 0 0 0) 0x7fbe8c05a3e0 con 0x7fbe9815b340 2024-05-30T17:02:56.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7806e0c0 msgr2=0x7fbe78070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7806e0c0 0x7fbe78070580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbe980b1e20 tx=0x7fbe84009040 comp rx=0 tx=0).stop 2024-05-30T17:02:56.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe9815b340 msgr2=0x7fbe980ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe9815b340 0x7fbe980ccb20 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00ea00 tx=0x7fbe8c00eec0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 shutdown_connections 2024-05-30T17:02:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe7806e0c0 0x7fbe78070580 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe9815c290 0x7fbe9815d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe9815b720 0x7fbe980cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/3789834816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe9815b340 0x7fbe980ccb20 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 >> 172.21.15.67:0/3789834816 conn(0x7fbe980b1e50 msgr2=0x7fbe980b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:56.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 shutdown_connections 2024-05-30T17:02:56.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.133+0000 7fbe9fe00640 1 -- 172.21.15.67:0/3789834816 wait complete. 2024-05-30T17:02:56.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T17:02:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.369+0000 7fdb43000640 1 Processor -- start 2024-05-30T17:02:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.369+0000 7fdb43000640 1 -- start start 2024-05-30T17:02:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.370+0000 7fdb43000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdb3c06f860 0x7fdb3c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.370+0000 7fdb43000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.370+0000 7fdb43000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb3c06ed10 con 0x7fdb3c0b3690 2024-05-30T17:02:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.370+0000 7fdb43000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fdb3c15cb00 con 0x7fdb3c06f860 2024-05-30T17:02:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.370+0000 7fdb43000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3c0b3cc0 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.370+0000 7fdb3b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdb3c0b3690 0x7fdb3c0d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55960/0 (socket says 172.21.15.67:55960) 2024-05-30T17:02:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.370+0000 7fdb3be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdb3c06f860 0x7fdb3c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.370+0000 7fdb3b400640 1 -- 172.21.15.67:0/2884244341 learned_addr learned my addr 172.21.15.67:0/2884244341 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:56.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.371+0000 7fdb41000640 1 --2- 172.21.15.67:0/2884244341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.371+0000 7fdb3aa00640 1 -- 172.21.15.67:0/2884244341 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761232620 0 0) 0x7fdb24001490 con 0x7fdb3c0b3690 2024-05-30T17:02:56.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.371+0000 7fdb3aa00640 1 -- 172.21.15.67:0/2884244341 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb1c003520 con 0x7fdb3c0b3690 2024-05-30T17:02:56.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.371+0000 7fdb3be00640 1 -- 172.21.15.67:0/2884244341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0d0bb0 msgr2=0x7fdb3c0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:56.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.371+0000 7fdb3be00640 1 --2- 172.21.15.67:0/2884244341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.371+0000 7fdb3be00640 1 -- 172.21.15.67:0/2884244341 >> v1:172.21.15.67:6789/0 conn(0x7fdb3c0b3690 legacy=0x7fdb3c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.371+0000 7fdb3be00640 1 -- 172.21.15.67:0/2884244341 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c0b3fa0 con 0x7fdb3c06f860 2024-05-30T17:02:56.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.372+0000 7fdb3be00640 1 --2- 172.21.15.67:0/2884244341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c06f860 0x7fdb3c0b3150 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0097a0 tx=0x7fdb2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cef709e43ff762 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:56.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb3aa00640 1 -- 172.21.15.67:0/2884244341 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c037070 con 0x7fdb3c06f860 2024-05-30T17:02:56.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb3aa00640 1 -- 172.21.15.67:0/2884244341 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2c032440 con 0x7fdb3c06f860 2024-05-30T17:02:56.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb3aa00640 1 -- 172.21.15.67:0/2884244341 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c032810 con 0x7fdb3c06f860 2024-05-30T17:02:56.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb43000640 1 -- 172.21.15.67:0/2884244341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c06f860 msgr2=0x7fdb3c0b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb43000640 1 --2- 172.21.15.67:0/2884244341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c06f860 0x7fdb3c0b3150 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0097a0 tx=0x7fdb2c02a180 comp rx=0 tx=0).stop 2024-05-30T17:02:56.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb43000640 1 -- 172.21.15.67:0/2884244341 shutdown_connections 2024-05-30T17:02:56.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb43000640 1 --2- 172.21.15.67:0/2884244341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb43000640 1 --2- 172.21.15.67:0/2884244341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c06f860 0x7fdb3c0b3150 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb43000640 1 -- 172.21.15.67:0/2884244341 >> 172.21.15.67:0/2884244341 conn(0x7fdb3c06bae0 msgr2=0x7fdb3c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:56.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb43000640 1 -- 172.21.15.67:0/2884244341 shutdown_connections 2024-05-30T17:02:56.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.373+0000 7fdb43000640 1 -- 172.21.15.67:0/2884244341 wait complete. 2024-05-30T17:02:56.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.375+0000 7fdb43000640 1 Processor -- start 2024-05-30T17:02:56.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.375+0000 7fdb43000640 1 -- start start 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.376+0000 7fdb43000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c06f860 0x7fdb3c1f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.376+0000 7fdb43000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0b3690 0x7fdb3c15f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.376+0000 7fdb43000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.376+0000 7fdb43000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c0b4840 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.376+0000 7fdb43000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c0b4560 con 0x7fdb3c06f860 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.376+0000 7fdb43000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3c0ad670 con 0x7fdb3c0b3690 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb41000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb41000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52662/0 (socket says 172.21.15.67:52662) 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb41000640 1 -- 172.21.15.67:0/2799653648 learned_addr learned my addr 172.21.15.67:0/2799653648 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb3b400640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0b3690 0x7fdb3c15f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb41000640 1 -- 172.21.15.67:0/2799653648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0b3690 msgr2=0x7fdb3c15f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb41000640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0b3690 0x7fdb3c15f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb41000640 1 -- 172.21.15.67:0/2799653648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c06f860 msgr2=0x7fdb3c1f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb3be00640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c06f860 0x7fdb3c1f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb41000640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c06f860 0x7fdb3c1f57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb41000640 1 -- 172.21.15.67:0/2799653648 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c0ce080 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb3b400640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0b3690 0x7fdb3c15f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:56.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.377+0000 7fdb41000640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0cc7f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400e7f0 tx=0x7fdb3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:56.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.378+0000 7fdb33e00640 1 -- 172.21.15.67:0/2799653648 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34017070 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.378+0000 7fdb33e00640 1 -- 172.21.15.67:0/2799653648 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3400c730 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.378+0000 7fdb33e00640 1 -- 172.21.15.67:0/2799653648 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3400cb20 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.378+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3c0ce360 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.378+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3c0ce920 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.379+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3c0cee10 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.380+0000 7fdb33e00640 1 -- 172.21.15.67:0/2799653648 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fdb3401a070 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.381+0000 7fdb33e00640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdb1806e090 0x7fdb18070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.381+0000 7fdb33e00640 1 -- 172.21.15.67:0/2799653648 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fdb3408a800 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.381+0000 7fdb3be00640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdb1806e090 0x7fdb18070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.382+0000 7fdb3be00640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdb1806e090 0x7fdb18070550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c034000 tx=0x7fdb2c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.386+0000 7fdb33e00640 1 -- 172.21.15.67:0/2799653648 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb3405a690 con 0x7fdb3c0d0bb0 2024-05-30T17:02:56.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.616+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fdb3c0cde30 con 0x7fdb1806e090 2024-05-30T17:02:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.619+0000 7fdb33e00640 1 -- 172.21.15.67:0/2799653648 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fdb3c0cde30 con 0x7fdb1806e090 2024-05-30T17:02:56.619 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:02:56.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:02:56.620 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T17:02:56.620 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:02:56.620 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:02:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdb1806e090 msgr2=0x7fdb18070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdb1806e090 0x7fdb18070550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c034000 tx=0x7fdb2c02e000 comp rx=0 tx=0).stop 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c0d0bb0 msgr2=0x7fdb3c0cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0cc7f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400e7f0 tx=0x7fdb3400ecb0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 shutdown_connections 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdb1806e090 0x7fdb18070550 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb3c0d0bb0 0x7fdb3c0cc7f0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c0b3690 0x7fdb3c15f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 --2- 172.21.15.67:0/2799653648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c06f860 0x7fdb3c1f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 >> 172.21.15.67:0/2799653648 conn(0x7fdb3c06bae0 msgr2=0x7fdb3c0af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 shutdown_connections 2024-05-30T17:02:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.622+0000 7fdb43000640 1 -- 172.21.15.67:0/2799653648 wait complete. 2024-05-30T17:02:56.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T17:02:56.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.853+0000 7fc024800640 1 Processor -- start 2024-05-30T17:02:56.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.853+0000 7fc024800640 1 -- start start 2024-05-30T17:02:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.854+0000 7fc024800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc01c0d0640 0x7fc01c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.854+0000 7fc024800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c0d0060 con 0x7fc01c0d0a20 2024-05-30T17:02:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.854+0000 7fc024800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c15cb10 con 0x7fc01c0b32c0 2024-05-30T17:02:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.854+0000 7fc024800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc01c0adbb0 con 0x7fc01c0d0640 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.855+0000 7fc021400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc01c0d0a20 0x7fc01c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55970/0 (socket says 172.21.15.67:55970) 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.855+0000 7fc021400640 1 -- 172.21.15.67:0/3040428344 learned_addr learned my addr 172.21.15.67:0/3040428344 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.855+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3086586186 0 0) 0x7fc01c0d0060 con 0x7fc01c0d0a20 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.855+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc008003520 con 0x7fc01c0d0a20 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.855+0000 7fc021e00640 1 --2- 172.21.15.67:0/3040428344 >> v2:172.21.15.169:3300/0 conn(0x7fc01c0d0640 0x7fc01c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1921157295 0 0) 0x7fc008003520 con 0x7fc01c0d0a20 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 >> v2:172.21.15.169:3300/0 conn(0x7fc01c0d0640 msgr2=0x7fc01c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 --2- 172.21.15.67:0/3040428344 >> v2:172.21.15.169:3300/0 conn(0x7fc01c0d0640 0x7fc01c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 >> v1:172.21.15.169:6789/0 conn(0x7fc01c0b32c0 legacy=0x7fc01c0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0ad5f0 con 0x7fc01c0d0a20 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fc00c001670 con 0x7fc01c0d0a20 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 >> v1:172.21.15.67:6789/0 conn(0x7fc01c0d0a20 legacy=0x7fc01c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc008003a00 0x7fc008003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc008004800 0x7fc008004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0080052e0 0x7fc00800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc00800c1d0 con 0x7fc008004800 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc00800c380 con 0x7fc0080052e0 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc00800c580 con 0x7fc008003a00 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc021400640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc008004800 0x7fc008004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.856+0000 7fc021e00640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc008003a00 0x7fc008003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.857+0000 7fc021400640 1 -- 172.21.15.67:0/3040428344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc008003a00 msgr2=0x7fc008003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.857+0000 7fc021400640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc008003a00 0x7fc008003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.857+0000 7fc021400640 1 -- 172.21.15.67:0/3040428344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0080052e0 msgr2=0x7fc00800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.857+0000 7fc021400640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0080052e0 0x7fc00800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.857+0000 7fc021400640 1 -- 172.21.15.67:0/3040428344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc008003520 con 0x7fc008004800 2024-05-30T17:02:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.857+0000 7fc022800640 1 -- 172.21.15.67:0/3040428344 reap_dead start 2024-05-30T17:02:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.858+0000 7fc021400640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc008004800 0x7fc008004c80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c110 tx=0x7fc008005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cf375383387e44 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.858+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c007c40 con 0x7fc008004800 2024-05-30T17:02:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.858+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c005af0 con 0x7fc008004800 2024-05-30T17:02:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.858+0000 7fc020a00640 1 -- 172.21.15.67:0/3040428344 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c007070 con 0x7fc008004800 2024-05-30T17:02:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.859+0000 7fc024800640 1 -- 172.21.15.67:0/3040428344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc008004800 msgr2=0x7fc008004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.859+0000 7fc024800640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc008004800 0x7fc008004c80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c110 tx=0x7fc008005830 comp rx=0 tx=0).stop 2024-05-30T17:02:56.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.859+0000 7fc024800640 1 -- 172.21.15.67:0/3040428344 shutdown_connections 2024-05-30T17:02:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.859+0000 7fc024800640 1 --2- 172.21.15.67:0/3040428344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc008004800 0x7fc008004c80 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.859+0000 7fc024800640 1 -- 172.21.15.67:0/3040428344 >> 172.21.15.67:0/3040428344 conn(0x7fc01c06bd20 msgr2=0x7fc01c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.859+0000 7fc024800640 1 -- 172.21.15.67:0/3040428344 shutdown_connections 2024-05-30T17:02:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.859+0000 7fc024800640 1 -- 172.21.15.67:0/3040428344 wait complete. 2024-05-30T17:02:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.860+0000 7fc024800640 1 Processor -- start 2024-05-30T17:02:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.861+0000 7fc024800640 1 -- start start 2024-05-30T17:02:56.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc024800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc008004800 0x7fc01c0b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc024800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc01c154eb0 0x7fc01c153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc024800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc01c1f4e30 0x7fc01c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc024800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c15d0b0 con 0x7fc01c1f4e30 2024-05-30T17:02:56.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc024800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c15cdd0 con 0x7fc008004800 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc024800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c0ad2a0 con 0x7fc01c154eb0 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc022800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc01c1f4e30 0x7fc01c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc022800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc01c1f4e30 0x7fc01c15b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52680/0 (socket says 172.21.15.67:52680) 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc022800640 1 -- 172.21.15.67:0/311280957 learned_addr learned my addr 172.21.15.67:0/311280957 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc021400640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc01c154eb0 0x7fc01c153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.862+0000 7fc021e00640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc008004800 0x7fc01c0b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc022800640 1 -- 172.21.15.67:0/311280957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc01c154eb0 msgr2=0x7fc01c153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc022800640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc01c154eb0 0x7fc01c153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc022800640 1 -- 172.21.15.67:0/311280957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc008004800 msgr2=0x7fc01c0b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc022800640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc008004800 0x7fc01c0b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc022800640 1 -- 172.21.15.67:0/311280957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c153a60 con 0x7fc01c1f4e30 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc021e00640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc008004800 0x7fc01c0b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc022800640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc01c1f4e30 0x7fc01c15b240 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0aff00 tx=0x7fc01800c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc012000640 1 -- 172.21.15.67:0/311280957 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018017070 con 0x7fc01c1f4e30 2024-05-30T17:02:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc012000640 1 -- 172.21.15.67:0/311280957 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc018007310 con 0x7fc01c1f4e30 2024-05-30T17:02:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc012000640 1 -- 172.21.15.67:0/311280957 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180057c0 con 0x7fc01c1f4e30 2024-05-30T17:02:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c1f4650 con 0x7fc01c1f4e30 2024-05-30T17:02:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.863+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c155830 con 0x7fc01c1f4e30 2024-05-30T17:02:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.864+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c152850 con 0x7fc01c1f4e30 2024-05-30T17:02:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.865+0000 7fc012000640 1 -- 172.21.15.67:0/311280957 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7fc01801a070 con 0x7fc01c1f4e30 2024-05-30T17:02:56.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.866+0000 7fc012000640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc00006e090 0x7fc000070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:56.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.867+0000 7fc012000640 1 -- 172.21.15.67:0/311280957 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7fc018089810 con 0x7fc01c1f4e30 2024-05-30T17:02:56.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.867+0000 7fc021e00640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc00006e090 0x7fc000070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.867+0000 7fc021e00640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc00006e090 0x7fc000070550 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc01c1f21b0 tx=0x7fc01400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:56.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:56.871+0000 7fc012000640 1 -- 172.21.15.67:0/311280957 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc01c152850 con 0x7fc01c1f4e30 2024-05-30T17:02:57.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.080+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc01c0689e0 con 0x7fc01c1f4e30 2024-05-30T17:02:57.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.081+0000 7fc012000640 1 -- 172.21.15.67:0/311280957 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fc018059050 con 0x7fc01c1f4e30 2024-05-30T17:02:57.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.083+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc00006e090 msgr2=0x7fc000070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.083+0000 7fc024800640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc00006e090 0x7fc000070550 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc01c1f21b0 tx=0x7fc01400b040 comp rx=0 tx=0).stop 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.083+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc01c1f4e30 msgr2=0x7fc01c15b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.083+0000 7fc024800640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc01c1f4e30 0x7fc01c15b240 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0aff00 tx=0x7fc01800c570 comp rx=0 tx=0).stop 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.084+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 shutdown_connections 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.084+0000 7fc024800640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc00006e090 0x7fc000070550 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.084+0000 7fc024800640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc01c1f4e30 0x7fc01c15b240 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.084+0000 7fc024800640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc01c154eb0 0x7fc01c153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.084+0000 7fc024800640 1 --2- 172.21.15.67:0/311280957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc008004800 0x7fc01c0b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.084+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 >> 172.21.15.67:0/311280957 conn(0x7fc01c06bd20 msgr2=0x7fc01c15e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.084+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 shutdown_connections 2024-05-30T17:02:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.084+0000 7fc024800640 1 -- 172.21.15.67:0/311280957 wait complete. 2024-05-30T17:02:57.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-30T17:02:57.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.312+0000 7efc5e600640 1 Processor -- start 2024-05-30T17:02:57.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.313+0000 7efc5e600640 1 -- start start 2024-05-30T17:02:57.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.314+0000 7efc5e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 0x7efc580d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.314+0000 7efc5e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc580cf180 0x7efc580cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.314+0000 7efc5e600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc580d0e50 con 0x7efc581621d0 2024-05-30T17:02:57.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc5e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7efc580d1a60 con 0x7efc580cf180 2024-05-30T17:02:57.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc5e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efc580d1130 con 0x7efc580d3220 2024-05-30T17:02:57.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc5dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc580cf180 0x7efc580cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc5dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc580cf180 0x7efc580cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52690/0 (socket says 172.21.15.67:52690) 2024-05-30T17:02:57.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc5dc00640 1 -- 172.21.15.67:0/1290399358 learned_addr learned my addr 172.21.15.67:0/1290399358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:57.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc57e00640 1 --2- 172.21.15.67:0/1290399358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 0x7efc580d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc5dc00640 1 -- 172.21.15.67:0/1290399358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 msgr2=0x7efc580d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc5dc00640 1 --2- 172.21.15.67:0/1290399358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 0x7efc580d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc5dc00640 1 -- 172.21.15.67:0/1290399358 >> v1:172.21.15.169:6789/0 conn(0x7efc581621d0 legacy=0x7efc581625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.315+0000 7efc5dc00640 1 -- 172.21.15.67:0/1290399358 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc580d1410 con 0x7efc580cf180 2024-05-30T17:02:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.317+0000 7efc5dc00640 1 --2- 172.21.15.67:0/1290399358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 0x7efc580cf610 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7efc48009870 tx=0x7efc4802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734a8e547a64206e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.317+0000 7efc57400640 1 -- 172.21.15.67:0/1290399358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48037070 con 0x7efc580cf180 2024-05-30T17:02:57.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.317+0000 7efc57400640 1 -- 172.21.15.67:0/1290399358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc48032440 con 0x7efc580cf180 2024-05-30T17:02:57.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.317+0000 7efc57400640 1 -- 172.21.15.67:0/1290399358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48032810 con 0x7efc580cf180 2024-05-30T17:02:57.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.318+0000 7efc5e600640 1 -- 172.21.15.67:0/1290399358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 msgr2=0x7efc580cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.318+0000 7efc5e600640 1 --2- 172.21.15.67:0/1290399358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 0x7efc580cf610 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7efc48009870 tx=0x7efc4802a210 comp rx=0 tx=0).stop 2024-05-30T17:02:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.318+0000 7efc5e600640 1 -- 172.21.15.67:0/1290399358 shutdown_connections 2024-05-30T17:02:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.318+0000 7efc5e600640 1 --2- 172.21.15.67:0/1290399358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 0x7efc580cf610 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.318+0000 7efc5e600640 1 --2- 172.21.15.67:0/1290399358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 0x7efc580d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.318+0000 7efc5e600640 1 -- 172.21.15.67:0/1290399358 >> 172.21.15.67:0/1290399358 conn(0x7efc5808bfc0 msgr2=0x7efc580d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.318+0000 7efc5e600640 1 -- 172.21.15.67:0/1290399358 shutdown_connections 2024-05-30T17:02:57.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.318+0000 7efc5e600640 1 -- 172.21.15.67:0/1290399358 wait complete. 2024-05-30T17:02:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.320+0000 7efc5e600640 1 Processor -- start 2024-05-30T17:02:57.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.320+0000 7efc5e600640 1 -- start start 2024-05-30T17:02:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.321+0000 7efc5e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 0x7efc58159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.321+0000 7efc5e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 0x7efc5814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.321+0000 7efc5e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc581621d0 0x7efc580c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.321+0000 7efc5e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efc580d5470 con 0x7efc580cf180 2024-05-30T17:02:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.321+0000 7efc5e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efc580d1ef0 con 0x7efc581621d0 2024-05-30T17:02:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.321+0000 7efc5e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efc5815fa20 con 0x7efc580d3220 2024-05-30T17:02:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.321+0000 7efc5d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 0x7efc58159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.321+0000 7efc5d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 0x7efc58159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52698/0 (socket says 172.21.15.67:52698) 2024-05-30T17:02:57.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.321+0000 7efc5d200640 1 -- 172.21.15.67:0/2947672077 learned_addr learned my addr 172.21.15.67:0/2947672077 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc57e00640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 0x7efc5814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc5dc00640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc581621d0 0x7efc580c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc5d200640 1 -- 172.21.15.67:0/2947672077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 msgr2=0x7efc5814ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc5d200640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 0x7efc5814ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc5d200640 1 -- 172.21.15.67:0/2947672077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc581621d0 msgr2=0x7efc580c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc5d200640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc581621d0 0x7efc580c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc5d200640 1 -- 172.21.15.67:0/2947672077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc581e8ea0 con 0x7efc580cf180 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc5d200640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 0x7efc58159db0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b420 tx=0x7efc5000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc54c00640 1 -- 172.21.15.67:0/2947672077 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50011830 con 0x7efc580cf180 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc54c00640 1 -- 172.21.15.67:0/2947672077 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc50016070 con 0x7efc580cf180 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.322+0000 7efc54c00640 1 -- 172.21.15.67:0/2947672077 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50011c10 con 0x7efc580cf180 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.323+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc580b7db0 con 0x7efc580cf180 2024-05-30T17:02:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.323+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc580b8370 con 0x7efc580cf180 2024-05-30T17:02:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.324+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc580689e0 con 0x7efc580cf180 2024-05-30T17:02:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.325+0000 7efc54c00640 1 -- 172.21.15.67:0/2947672077 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7efc5001b460 con 0x7efc580cf180 2024-05-30T17:02:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.326+0000 7efc54c00640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc3806e0c0 0x7efc38070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.326+0000 7efc54c00640 1 -- 172.21.15.67:0/2947672077 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7efc5008a360 con 0x7efc580cf180 2024-05-30T17:02:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.327+0000 7efc57e00640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc3806e0c0 0x7efc38070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.327+0000 7efc57e00640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc3806e0c0 0x7efc38070580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efc580cb360 tx=0x7efc440039b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.331+0000 7efc54c00640 1 -- 172.21.15.67:0/2947672077 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc50020020 con 0x7efc580cf180 2024-05-30T17:02:57.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.564+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7efc580d4d10 con 0x7efc3806e0c0 2024-05-30T17:02:57.579 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:57.578+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:57.579 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:57.578+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:57.579 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:57.578+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:57.579 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:57.578+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:57.579 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:02:57.578+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:02:57.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.671+0000 7efc54c00640 1 -- 172.21.15.67:0/2947672077 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc580d4d10 con 0x7efc3806e0c0 2024-05-30T17:02:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc3806e0c0 msgr2=0x7efc38070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc3806e0c0 0x7efc38070580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efc580cb360 tx=0x7efc440039b0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 msgr2=0x7efc58159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 0x7efc58159db0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b420 tx=0x7efc5000b8e0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 shutdown_connections 2024-05-30T17:02:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc3806e0c0 0x7efc38070580 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc581621d0 0x7efc580c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc580d3220 0x7efc5814ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 --2- 172.21.15.67:0/2947672077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc580cf180 0x7efc58159db0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.674+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 >> 172.21.15.67:0/2947672077 conn(0x7efc5808bfc0 msgr2=0x7efc5806e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.675+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 shutdown_connections 2024-05-30T17:02:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.675+0000 7efc5e600640 1 -- 172.21.15.67:0/2947672077 wait complete. 2024-05-30T17:02:57.698 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T17:02:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.908+0000 7f84c5200640 1 Processor -- start 2024-05-30T17:02:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.908+0000 7f84c5200640 1 -- start start 2024-05-30T17:02:57.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.910+0000 7f84c5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c00b32c0 0x7f84c00b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.910+0000 7f84c5200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c00d0060 con 0x7f84c00d0640 2024-05-30T17:02:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.910+0000 7f84c5200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c015cb10 con 0x7f84c015d5d0 2024-05-30T17:02:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.910+0000 7f84c5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84c00adbb0 con 0x7f84c00b32c0 2024-05-30T17:02:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.910+0000 7f84bde00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f84c00d0640 0x7f84c00d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55986/0 (socket says 172.21.15.67:55986) 2024-05-30T17:02:57.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.910+0000 7f84bde00640 1 -- 172.21.15.67:0/625750117 learned_addr learned my addr 172.21.15.67:0/625750117 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:02:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84be800640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c00b32c0 0x7f84c00b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306055339 0 0) 0x7f84c00d0060 con 0x7f84c00d0640 2024-05-30T17:02:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84a4003520 con 0x7f84c00d0640 2024-05-30T17:02:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2635855451 0 0) 0x7f84a4003520 con 0x7f84c00d0640 2024-05-30T17:02:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c00b32c0 msgr2=0x7f84c00b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:57.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c00b32c0 0x7f84c00b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 >> v1:172.21.15.169:6789/0 conn(0x7f84c015d5d0 legacy=0x7f84c00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c00ad5f0 con 0x7f84c00d0640 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f84b00019f0 con 0x7f84c00d0640 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 >> v1:172.21.15.67:6789/0 conn(0x7f84c00d0640 legacy=0x7f84c00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a40039e0 0x7f84a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.911+0000 7f84bca00640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a4004800 0x7f84a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bca00640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c00b32c0 0x7f84a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f84a400ba70 con 0x7f84a4004800 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84a400bbf0 con 0x7f84a40039e0 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84a400bdf0 con 0x7f84c00b32c0 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bd400640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a4004800 0x7f84a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bde00640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a40039e0 0x7f84a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84be800640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c00b32c0 0x7f84a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bd400640 1 -- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c00b32c0 msgr2=0x7f84a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bd400640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c00b32c0 0x7f84a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bd400640 1 -- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a40039e0 msgr2=0x7f84a4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bd400640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a40039e0 0x7f84a4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84bd400640 1 -- 172.21.15.67:0/625750117 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a4003520 con 0x7f84a4004800 2024-05-30T17:02:57.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.912+0000 7f84be800640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c00b32c0 0x7f84a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:57.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.913+0000 7f84bd400640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a4004800 0x7f84a4004c80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001720 tx=0x7f84a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a898db76f482a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.914+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8001370 con 0x7f84a4004800 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.914+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a80052d0 con 0x7f84a4004800 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.914+0000 7f84bca00640 1 -- 172.21.15.67:0/625750117 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a802aba0 con 0x7f84a4004800 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.914+0000 7f84c5200640 1 -- 172.21.15.67:0/625750117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a4004800 msgr2=0x7f84a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.914+0000 7f84c5200640 1 --2- 172.21.15.67:0/625750117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84a4004800 0x7f84a4004c80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001720 tx=0x7f84a4005830 comp rx=0 tx=0).stop 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.914+0000 7f84be800640 1 -- 172.21.15.67:0/625750117 reap_dead start 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.914+0000 7f84c5200640 1 -- 172.21.15.67:0/625750117 shutdown_connections 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.914+0000 7f84c5200640 1 -- 172.21.15.67:0/625750117 >> 172.21.15.67:0/625750117 conn(0x7f84c006bd20 msgr2=0x7f84c006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.914+0000 7f84c5200640 1 -- 172.21.15.67:0/625750117 shutdown_connections 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.915+0000 7f84c5200640 1 -- 172.21.15.67:0/625750117 wait complete. 2024-05-30T17:02:57.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.916+0000 7f84c5200640 1 Processor -- start 2024-05-30T17:02:57.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.916+0000 7f84c5200640 1 -- start start 2024-05-30T17:02:57.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.917+0000 7f84c5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4004800 0x7f84c00b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.917+0000 7f84c5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84c006f260 0x7f84c00aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.917+0000 7f84c5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c015b240 0x7f84c015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.917+0000 7f84c5200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f84c015d0b0 con 0x7f84c006f260 2024-05-30T17:02:57.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.917+0000 7f84c5200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84c015cdd0 con 0x7f84a4004800 2024-05-30T17:02:57.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.917+0000 7f84c5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84c015da50 con 0x7f84c015b240 2024-05-30T17:02:57.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84bd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84c006f260 0x7f84c00aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84bd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84c006f260 0x7f84c00aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52732/0 (socket says 172.21.15.67:52732) 2024-05-30T17:02:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84bd400640 1 -- 172.21.15.67:0/2128882424 learned_addr learned my addr 172.21.15.67:0/2128882424 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84be800640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c015b240 0x7f84c015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84bde00640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4004800 0x7f84c00b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84bd400640 1 -- 172.21.15.67:0/2128882424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c015b240 msgr2=0x7f84c015b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84bd400640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c015b240 0x7f84c015b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84bd400640 1 -- 172.21.15.67:0/2128882424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4004800 msgr2=0x7f84c00b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84bd400640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4004800 0x7f84c00b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.918+0000 7f84bd400640 1 -- 172.21.15.67:0/2128882424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c00b08a0 con 0x7f84c006f260 2024-05-30T17:02:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.919+0000 7f84bd400640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84c006f260 0x7f84c00aff30 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001840 tx=0x7f84a8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.919+0000 7f84b6000640 1 -- 172.21.15.67:0/2128882424 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8037040 con 0x7f84c006f260 2024-05-30T17:02:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.919+0000 7f84b6000640 1 -- 172.21.15.67:0/2128882424 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a8005480 con 0x7f84c006f260 2024-05-30T17:02:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.919+0000 7f84b6000640 1 -- 172.21.15.67:0/2128882424 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8007a30 con 0x7f84c006f260 2024-05-30T17:02:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.919+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c00b0b80 con 0x7f84c006f260 2024-05-30T17:02:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.919+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c00b1140 con 0x7f84c006f260 2024-05-30T17:02:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.920+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c014db30 con 0x7f84c006f260 2024-05-30T17:02:57.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.922+0000 7f84b6000640 1 -- 172.21.15.67:0/2128882424 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f84a8044020 con 0x7f84c006f260 2024-05-30T17:02:57.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.922+0000 7f84b6000640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f849c06e090 0x7f849c070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:57.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.923+0000 7f84b6000640 1 -- 172.21.15.67:0/2128882424 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f84a80aa640 con 0x7f84c006f260 2024-05-30T17:02:57.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.923+0000 7f84bde00640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f849c06e090 0x7f849c070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:57.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.923+0000 7f84bde00640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f849c06e090 0x7f849c070550 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f84b00013d0 tx=0x7f84b00017b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:57.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:57.927+0000 7f84b6000640 1 -- 172.21.15.67:0/2128882424 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84a80b5080 con 0x7f84c006f260 2024-05-30T17:02:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.169+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f84c00689e0 con 0x7f849c06e090 2024-05-30T17:02:58.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.180+0000 7f84b6000640 1 -- 172.21.15.67:0/2128882424 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f84c00689e0 con 0x7f849c06e090 2024-05-30T17:02:58.181 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd 2024-05-30T17:02:58.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f849c06e090 msgr2=0x7f849c070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:58.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f849c06e090 0x7f849c070550 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f84b00013d0 tx=0x7f84b00017b0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84c006f260 msgr2=0x7f84c00aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:58.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84c006f260 0x7f84c00aff30 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001840 tx=0x7f84a8001120 comp rx=0 tx=0).stop 2024-05-30T17:02:58.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 shutdown_connections 2024-05-30T17:02:58.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f849c06e090 0x7f849c070550 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c015b240 0x7f84c015b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84c006f260 0x7f84c00aff30 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 --2- 172.21.15.67:0/2128882424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84a4004800 0x7f84c00b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.183+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 >> 172.21.15.67:0/2128882424 conn(0x7f84c006bd20 msgr2=0x7f84c00cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:58.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.184+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 shutdown_connections 2024-05-30T17:02:58.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.184+0000 7f84c5200640 1 -- 172.21.15.67:0/2128882424 wait complete. 2024-05-30T17:02:58.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T17:02:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.424+0000 7f8c5d000640 1 Processor -- start 2024-05-30T17:02:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.424+0000 7f8c5d000640 1 -- start start 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c5d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8c580caa40 0x7f8c580ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c5d000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8c580cae20 0x7f8c58151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c5d000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c580b3e80 con 0x7f8c581522f0 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c5d000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f8c580d0b30 con 0x7f8c580caa40 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c5d000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8c580b4160 con 0x7f8c580cae20 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c57400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8c580caa40 0x7f8c580ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c57400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8c580caa40 0x7f8c580ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52748/0 (socket says 172.21.15.67:52748) 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c57400640 1 -- 172.21.15.67:0/52893520 learned_addr learned my addr 172.21.15.67:0/52893520 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c57400640 1 -- 172.21.15.67:0/52893520 >> v2:172.21.15.169:3300/0 conn(0x7f8c580cae20 msgr2=0x7f8c58151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.426+0000 7f8c57400640 1 --2- 172.21.15.67:0/52893520 >> v2:172.21.15.169:3300/0 conn(0x7f8c580cae20 0x7f8c58151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.427+0000 7f8c57400640 1 -- 172.21.15.67:0/52893520 >> v1:172.21.15.169:6789/0 conn(0x7f8c581522f0 legacy=0x7f8c580cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:02:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.427+0000 7f8c57400640 1 -- 172.21.15.67:0/52893520 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c580cde80 con 0x7f8c580caa40 2024-05-30T17:02:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.428+0000 7f8c57400640 1 --2- 172.21.15.67:0/52893520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 0x7f8c580ce960 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f8c580d0fb0 tx=0x7f8c4802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a07a7b4710a4137 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.428+0000 7f8c56000640 1 -- 172.21.15.67:0/52893520 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48036070 con 0x7f8c580caa40 2024-05-30T17:02:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.428+0000 7f8c56000640 1 -- 172.21.15.67:0/52893520 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c48031600 con 0x7f8c580caa40 2024-05-30T17:02:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.428+0000 7f8c56000640 1 -- 172.21.15.67:0/52893520 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480319d0 con 0x7f8c580caa40 2024-05-30T17:02:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.428+0000 7f8c5d000640 1 -- 172.21.15.67:0/52893520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 msgr2=0x7f8c580ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.428+0000 7f8c5d000640 1 --2- 172.21.15.67:0/52893520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 0x7f8c580ce960 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f8c580d0fb0 tx=0x7f8c4802a660 comp rx=0 tx=0).stop 2024-05-30T17:02:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.429+0000 7f8c5d000640 1 -- 172.21.15.67:0/52893520 shutdown_connections 2024-05-30T17:02:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.429+0000 7f8c5d000640 1 --2- 172.21.15.67:0/52893520 >> v2:172.21.15.169:3300/0 conn(0x7f8c580cae20 0x7f8c58151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.429+0000 7f8c5d000640 1 --2- 172.21.15.67:0/52893520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 0x7f8c580ce960 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.429+0000 7f8c5d000640 1 -- 172.21.15.67:0/52893520 >> 172.21.15.67:0/52893520 conn(0x7f8c5806b900 msgr2=0x7f8c5806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.429+0000 7f8c5d000640 1 -- 172.21.15.67:0/52893520 shutdown_connections 2024-05-30T17:02:58.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.429+0000 7f8c5d000640 1 -- 172.21.15.67:0/52893520 wait complete. 2024-05-30T17:02:58.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.430+0000 7f8c5d000640 1 Processor -- start 2024-05-30T17:02:58.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.431+0000 7f8c5d000640 1 -- start start 2024-05-30T17:02:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.431+0000 7f8c5d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 0x7f8c58073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.431+0000 7f8c5d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cae20 0x7f8c580735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c5d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c581522f0 0x7f8c580ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c5d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58152710 con 0x7f8c580caa40 2024-05-30T17:02:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c5d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c580ce440 con 0x7f8c581522f0 2024-05-30T17:02:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c5d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8c580ceea0 con 0x7f8c580cae20 2024-05-30T17:02:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c57400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 0x7f8c58073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c57400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 0x7f8c58073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52752/0 (socket says 172.21.15.67:52752) 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c57400640 1 -- 172.21.15.67:0/1381164691 learned_addr learned my addr 172.21.15.67:0/1381164691 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c57400640 1 -- 172.21.15.67:0/1381164691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cae20 msgr2=0x7f8c580735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c56a00640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cae20 0x7f8c580735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c57e00640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c581522f0 0x7f8c580ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c57400640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cae20 0x7f8c580735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c57400640 1 -- 172.21.15.67:0/1381164691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c581522f0 msgr2=0x7f8c580ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c57400640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c581522f0 0x7f8c580ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c57400640 1 -- 172.21.15.67:0/1381164691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c580b1500 con 0x7f8c580caa40 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.432+0000 7f8c56a00640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cae20 0x7f8c580735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:02:58.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.433+0000 7f8c57400640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 0x7f8c58073090 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f8c48034040 tx=0x7f8c48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:58.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.433+0000 7f8c4f400640 1 -- 172.21.15.67:0/1381164691 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48041020 con 0x7f8c580caa40 2024-05-30T17:02:58.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.433+0000 7f8c4f400640 1 -- 172.21.15.67:0/1381164691 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c48007800 con 0x7f8c580caa40 2024-05-30T17:02:58.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.433+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c580b0460 con 0x7f8c580caa40 2024-05-30T17:02:58.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.433+0000 7f8c4f400640 1 -- 172.21.15.67:0/1381164691 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48036040 con 0x7f8c580caa40 2024-05-30T17:02:58.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.433+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c580b0d00 con 0x7f8c580caa40 2024-05-30T17:02:58.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.435+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c580ce960 con 0x7f8c580caa40 2024-05-30T17:02:58.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.436+0000 7f8c4f400640 1 -- 172.21.15.67:0/1381164691 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 38) ==== 93182+0+0 (secure 0 0 0) 0x7f8c48044020 con 0x7f8c580caa40 2024-05-30T17:02:58.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.436+0000 7f8c4f400640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406e090 0x7f8c34070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:02:58.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.437+0000 7f8c4f400640 1 -- 172.21.15.67:0/1381164691 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5821+0+0 (secure 0 0 0) 0x7f8c480ab870 con 0x7f8c580caa40 2024-05-30T17:02:58.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.437+0000 7f8c56a00640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406e090 0x7f8c34070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:02:58.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.438+0000 7f8c56a00640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406e090 0x7f8c34070550 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006fd0 tx=0x7f8c40008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:02:58.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.441+0000 7f8c4f400640 1 -- 172.21.15.67:0/1381164691 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c580ce960 con 0x7f8c580caa40 2024-05-30T17:02:58.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.679+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f8c580b4ee0 con 0x7f8c3406e090 2024-05-30T17:02:58.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.687+0000 7f8c4f400640 1 -- 172.21.15.67:0/1381164691 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f8c580b4ee0 con 0x7f8c3406e090 2024-05-30T17:02:58.687 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd 2024-05-30T17:02:58.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406e090 msgr2=0x7f8c34070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:58.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406e090 0x7f8c34070550 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006fd0 tx=0x7f8c40008040 comp rx=0 tx=0).stop 2024-05-30T17:02:58.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 msgr2=0x7f8c58073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:02:58.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 0x7f8c58073090 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f8c48034040 tx=0x7f8c48004a40 comp rx=0 tx=0).stop 2024-05-30T17:02:58.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 shutdown_connections 2024-05-30T17:02:58.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8c3406e090 0x7f8c34070550 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c581522f0 0x7f8c580ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c580cae20 0x7f8c580735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 --2- 172.21.15.67:0/1381164691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c580caa40 0x7f8c58073090 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:02:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 >> 172.21.15.67:0/1381164691 conn(0x7f8c5806b900 msgr2=0x7f8c5806bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:02:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.689+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 shutdown_connections 2024-05-30T17:02:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:02:58.690+0000 7f8c5d000640 1 -- 172.21.15.67:0/1381164691 wait complete. 2024-05-30T17:02:58.711 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd 2024-05-30T17:02:58.711 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:02:58.711 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:02:58.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:02:58.764 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T17:03:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T17:03:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T17:03:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.3105 s, 65.8 MB/s 2024-05-30T17:03:15.165 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:03:15.165 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:03:15.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:03:15.222 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T17:03:15.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:03:15.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:03:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T17:03:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T17:03:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.6724 s, 78.5 MB/s 2024-05-30T17:03:28.963 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:03:28.963 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:03:29.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:03:29.024 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T17:03:43.997 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T17:03:43.997 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T17:03:43.997 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.9112 s, 72.0 MB/s 2024-05-30T17:03:43.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-30T17:03:44.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.217+0000 7f84a3200640 1 Processor -- start 2024-05-30T17:03:44.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.217+0000 7f84a3200640 1 -- start start 2024-05-30T17:03:44.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.218+0000 7f84a3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f849c15fa00 0x7f849c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.219+0000 7f84a3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c1600f0 0x7f849c0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.219+0000 7f84a3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f849c0d0d90 con 0x7f849c0d0970 2024-05-30T17:03:44.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.219+0000 7f84a3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f849c06fa00 con 0x7f849c15fa00 2024-05-30T17:03:44.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.219+0000 7f84a3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f849c06eeb0 con 0x7f849c1600f0 2024-05-30T17:03:44.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.219+0000 7f84a1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c1600f0 0x7f849c0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.219+0000 7f84a1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c1600f0 0x7f849c0d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:53438/0 (socket says 172.21.15.67:53438) 2024-05-30T17:03:44.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.219+0000 7f84a1400640 1 -- 172.21.15.67:0/439990164 learned_addr learned my addr 172.21.15.67:0/439990164 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:44.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.220+0000 7f84a1e00640 1 --2- 172.21.15.67:0/439990164 >> v2:172.21.15.169:3300/0 conn(0x7f849c15fa00 0x7f849c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.220+0000 7f84a1400640 1 -- 172.21.15.67:0/439990164 >> v2:172.21.15.169:3300/0 conn(0x7f849c15fa00 msgr2=0x7f849c06e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:44.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.220+0000 7f84a1400640 1 --2- 172.21.15.67:0/439990164 >> v2:172.21.15.169:3300/0 conn(0x7f849c15fa00 0x7f849c06e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.220+0000 7f84a1400640 1 -- 172.21.15.67:0/439990164 >> v1:172.21.15.169:6789/0 conn(0x7f849c0d0970 legacy=0x7f849c0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.220+0000 7f84a1400640 1 -- 172.21.15.67:0/439990164 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c06f160 con 0x7f849c1600f0 2024-05-30T17:03:44.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.220+0000 7f84a1e00640 1 --2- 172.21.15.67:0/439990164 >> v2:172.21.15.169:3300/0 conn(0x7f849c15fa00 0x7f849c06e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:03:44.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.222+0000 7f84a1400640 1 --2- 172.21.15.67:0/439990164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c1600f0 0x7f849c0d0430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f848c009870 tx=0x7f848c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef774d1b452c050 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:44.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.223+0000 7f84a0a00640 1 -- 172.21.15.67:0/439990164 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c037070 con 0x7f849c1600f0 2024-05-30T17:03:44.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.223+0000 7f84a0a00640 1 -- 172.21.15.67:0/439990164 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848c032440 con 0x7f849c1600f0 2024-05-30T17:03:44.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.223+0000 7f84a0a00640 1 -- 172.21.15.67:0/439990164 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c032810 con 0x7f849c1600f0 2024-05-30T17:03:44.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.223+0000 7f84a3200640 1 -- 172.21.15.67:0/439990164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c1600f0 msgr2=0x7f849c0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.223+0000 7f84a3200640 1 --2- 172.21.15.67:0/439990164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c1600f0 0x7f849c0d0430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f848c009870 tx=0x7f848c02a1c0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.224+0000 7f84a3200640 1 -- 172.21.15.67:0/439990164 shutdown_connections 2024-05-30T17:03:44.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.224+0000 7f84a3200640 1 --2- 172.21.15.67:0/439990164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c1600f0 0x7f849c0d0430 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.224+0000 7f84a3200640 1 --2- 172.21.15.67:0/439990164 >> v2:172.21.15.169:3300/0 conn(0x7f849c15fa00 0x7f849c06e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.224+0000 7f84a3200640 1 -- 172.21.15.67:0/439990164 >> 172.21.15.67:0/439990164 conn(0x7f849c06bae0 msgr2=0x7f849c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.224+0000 7f84a3200640 1 -- 172.21.15.67:0/439990164 shutdown_connections 2024-05-30T17:03:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.224+0000 7f84a3200640 1 -- 172.21.15.67:0/439990164 wait complete. 2024-05-30T17:03:44.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.226+0000 7f84a3200640 1 Processor -- start 2024-05-30T17:03:44.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.226+0000 7f84a3200640 1 -- start start 2024-05-30T17:03:44.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.227+0000 7f84a3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c0d0970 0x7f849c1f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c15fa00 0x7f849c0af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f849c1600f0 0x7f849c148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f849c071c70 con 0x7f849c0d0970 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f849c06f720 con 0x7f849c1600f0 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f849c15fde0 con 0x7f849c15fa00 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c0d0970 0x7f849c1f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c0d0970 0x7f849c1f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52460/0 (socket says 172.21.15.67:52460) 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a1e00640 1 -- 172.21.15.67:0/1881754426 learned_addr learned my addr 172.21.15.67:0/1881754426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a1400640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c15fa00 0x7f849c0af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.228+0000 7f84a2800640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f849c1600f0 0x7f849c148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f84a1e00640 1 -- 172.21.15.67:0/1881754426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c15fa00 msgr2=0x7f849c0af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f84a1e00640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c15fa00 0x7f849c0af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f84a1e00640 1 -- 172.21.15.67:0/1881754426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f849c1600f0 msgr2=0x7f849c148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f84a1e00640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f849c1600f0 0x7f849c148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f84a1e00640 1 -- 172.21.15.67:0/1881754426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c158a10 con 0x7f849c0d0970 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f84a2800640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f849c1600f0 0x7f849c148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f84a1e00640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c0d0970 0x7f849c1f2320 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f849000e960 tx=0x7f849000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f8496000640 1 -- 172.21.15.67:0/1881754426 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490017070 con 0x7f849c0d0970 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f8496000640 1 -- 172.21.15.67:0/1881754426 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f849000ca00 con 0x7f849c0d0970 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f8496000640 1 -- 172.21.15.67:0/1881754426 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849000cdd0 con 0x7f849c0d0970 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.229+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c158490 con 0x7f849c0d0970 2024-05-30T17:03:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.230+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c158730 con 0x7f849c0d0970 2024-05-30T17:03:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.231+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f849c0689e0 con 0x7f849c0d0970 2024-05-30T17:03:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.232+0000 7f8496000640 1 -- 172.21.15.67:0/1881754426 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 39) ==== 93233+0+0 (secure 0 0 0) 0x7f849001a070 con 0x7f849c0d0970 2024-05-30T17:03:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.233+0000 7f8496000640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f848006e150 0x7f8480070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.233+0000 7f8496000640 1 -- 172.21.15.67:0/1881754426 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f849008a880 con 0x7f849c0d0970 2024-05-30T17:03:44.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.233+0000 7f84a1400640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f848006e150 0x7f8480070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.234+0000 7f84a1400640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f848006e150 0x7f8480070610 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f848c034000 tx=0x7f848c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:44.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.238+0000 7f8496000640 1 -- 172.21.15.67:0/1881754426 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f849005a710 con 0x7f849c0d0970 2024-05-30T17:03:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.478+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f849c152f90 con 0x7f848006e150 2024-05-30T17:03:44.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.506+0000 7f8496000640 1 -- 172.21.15.67:0/1881754426 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f848c0326f0 con 0x7f848006e150 2024-05-30T17:03:44.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f848006e150 msgr2=0x7f8480070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f848006e150 0x7f8480070610 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f848c034000 tx=0x7f848c02e000 comp rx=0 tx=0).stop 2024-05-30T17:03:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c0d0970 msgr2=0x7f849c1f2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c0d0970 0x7f849c1f2320 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f849000e960 tx=0x7f849000ee20 comp rx=0 tx=0).stop 2024-05-30T17:03:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 shutdown_connections 2024-05-30T17:03:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f848006e150 0x7f8480070610 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f849c1600f0 0x7f849c148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f849c15fa00 0x7f849c0af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 --2- 172.21.15.67:0/1881754426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f849c0d0970 0x7f849c1f2320 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 >> 172.21.15.67:0/1881754426 conn(0x7f849c06bae0 msgr2=0x7f849c0b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:44.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 shutdown_connections 2024-05-30T17:03:44.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.509+0000 7f84a3200640 1 -- 172.21.15.67:0/1881754426 wait complete. 2024-05-30T17:03:44.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T17:03:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.742+0000 7ff56fe00640 1 Processor -- start 2024-05-30T17:03:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.742+0000 7ff56fe00640 1 -- start start 2024-05-30T17:03:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.743+0000 7ff56fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff570153510 0x7ff5700b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.743+0000 7ff56fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57015d900 con 0x7ff5700ccb90 2024-05-30T17:03:44.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.743+0000 7ff56fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5700b3f90 con 0x7ff57015d270 2024-05-30T17:03:44.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.743+0000 7ff56fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff5701538f0 con 0x7ff570153510 2024-05-30T17:03:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.744+0000 7ff56ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff570153510 0x7ff5700b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.744+0000 7ff56ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff570153510 0x7ff5700b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52476/0 (socket says 172.21.15.67:52476) 2024-05-30T17:03:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.744+0000 7ff56ea00640 1 -- 172.21.15.67:0/2149964651 learned_addr learned my addr 172.21.15.67:0/2149964651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.745+0000 7ff56ea00640 1 -- 172.21.15.67:0/2149964651 >> v1:172.21.15.169:6789/0 conn(0x7ff57015d270 legacy=0x7ff57006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.745+0000 7ff56ea00640 1 -- 172.21.15.67:0/2149964651 >> v1:172.21.15.67:6789/0 conn(0x7ff5700ccb90 legacy=0x7ff57015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.745+0000 7ff56ea00640 1 -- 172.21.15.67:0/2149964651 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5700b4270 con 0x7ff570153510 2024-05-30T17:03:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.747+0000 7ff56ea00640 1 --2- 172.21.15.67:0/2149964651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 0x7ff5700b4d50 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff560009870 tx=0x7ff56002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b3fabe6b59709f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.747+0000 7ff56e000640 1 -- 172.21.15.67:0/2149964651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560037070 con 0x7ff570153510 2024-05-30T17:03:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.747+0000 7ff56e000640 1 -- 172.21.15.67:0/2149964651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff560032440 con 0x7ff570153510 2024-05-30T17:03:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.747+0000 7ff56e000640 1 -- 172.21.15.67:0/2149964651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560032810 con 0x7ff570153510 2024-05-30T17:03:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.748+0000 7ff56fe00640 1 -- 172.21.15.67:0/2149964651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 msgr2=0x7ff5700b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.748+0000 7ff56fe00640 1 --2- 172.21.15.67:0/2149964651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 0x7ff5700b4d50 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff560009870 tx=0x7ff56002a270 comp rx=0 tx=0).stop 2024-05-30T17:03:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.748+0000 7ff56fe00640 1 -- 172.21.15.67:0/2149964651 shutdown_connections 2024-05-30T17:03:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.748+0000 7ff56fe00640 1 --2- 172.21.15.67:0/2149964651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 0x7ff5700b4d50 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.748+0000 7ff56fe00640 1 -- 172.21.15.67:0/2149964651 >> 172.21.15.67:0/2149964651 conn(0x7ff57006b900 msgr2=0x7ff57006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.748+0000 7ff56fe00640 1 -- 172.21.15.67:0/2149964651 shutdown_connections 2024-05-30T17:03:44.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.748+0000 7ff56fe00640 1 -- 172.21.15.67:0/2149964651 wait complete. 2024-05-30T17:03:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.750+0000 7ff56fe00640 1 Processor -- start 2024-05-30T17:03:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.750+0000 7ff56fe00640 1 -- start start 2024-05-30T17:03:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.751+0000 7ff56fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5700ccb90 0x7ff57014e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.751+0000 7ff56fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 0x7ff57015b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.751+0000 7ff56fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57015d270 0x7ff57015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.751+0000 7ff56fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff57015d6a0 con 0x7ff570153510 2024-05-30T17:03:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.751+0000 7ff56fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5700b48c0 con 0x7ff57015d270 2024-05-30T17:03:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.751+0000 7ff56fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff5700ccff0 con 0x7ff5700ccb90 2024-05-30T17:03:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.751+0000 7ff567e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 0x7ff57015b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.751+0000 7ff567e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 0x7ff57015b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52478/0 (socket says 172.21.15.67:52478) 2024-05-30T17:03:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.751+0000 7ff567e00640 1 -- 172.21.15.67:0/533247380 learned_addr learned my addr 172.21.15.67:0/533247380 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff56ea00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5700ccb90 0x7ff57014e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff56f400640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57015d270 0x7ff57015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff567e00640 1 -- 172.21.15.67:0/533247380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5700ccb90 msgr2=0x7ff57014e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff567e00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5700ccb90 0x7ff57014e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff567e00640 1 -- 172.21.15.67:0/533247380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57015d270 msgr2=0x7ff57015b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff567e00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57015d270 0x7ff57015b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff567e00640 1 -- 172.21.15.67:0/533247380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57014f4d0 con 0x7ff570153510 2024-05-30T17:03:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff567e00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 0x7ff57015b430 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c430 tx=0x7ff55800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff566a00640 1 -- 172.21.15.67:0/533247380 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5580052d0 con 0x7ff570153510 2024-05-30T17:03:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff566a00640 1 -- 172.21.15.67:0/533247380 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff558005490 con 0x7ff570153510 2024-05-30T17:03:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff566a00640 1 -- 172.21.15.67:0/533247380 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558007260 con 0x7ff570153510 2024-05-30T17:03:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.752+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff570072a90 con 0x7ff570153510 2024-05-30T17:03:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.753+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff570073050 con 0x7ff570153510 2024-05-30T17:03:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.754+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5700689e0 con 0x7ff570153510 2024-05-30T17:03:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.755+0000 7ff566a00640 1 -- 172.21.15.67:0/533247380 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 39) ==== 93233+0+0 (secure 0 0 0) 0x7ff558007400 con 0x7ff570153510 2024-05-30T17:03:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.756+0000 7ff566a00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff54c06e150 0x7ff54c070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.756+0000 7ff566a00640 1 -- 172.21.15.67:0/533247380 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7ff558089ef0 con 0x7ff570153510 2024-05-30T17:03:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.756+0000 7ff56ea00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff54c06e150 0x7ff54c070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.757+0000 7ff56ea00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff54c06e150 0x7ff54c070610 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff560034000 tx=0x7ff56002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:44.760+0000 7ff566a00640 1 -- 172.21.15.67:0/533247380 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff558059d80 con 0x7ff570153510 2024-05-30T17:03:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.004+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff5700b3c40 con 0x7ff54c06e150 2024-05-30T17:03:45.015 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd 2024-05-30T17:03:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.015+0000 7ff566a00640 1 -- 172.21.15.67:0/533247380 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7ff5700b3c40 con 0x7ff54c06e150 2024-05-30T17:03:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff54c06e150 msgr2=0x7ff54c070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff54c06e150 0x7ff54c070610 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff560034000 tx=0x7ff56002e000 comp rx=0 tx=0).stop 2024-05-30T17:03:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 msgr2=0x7ff57015b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 0x7ff57015b430 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c430 tx=0x7ff55800c8f0 comp rx=0 tx=0).stop 2024-05-30T17:03:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 shutdown_connections 2024-05-30T17:03:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff54c06e150 0x7ff54c070610 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57015d270 0x7ff57015b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff570153510 0x7ff57015b430 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 --2- 172.21.15.67:0/533247380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5700ccb90 0x7ff57014e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.018+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 >> 172.21.15.67:0/533247380 conn(0x7ff57006b900 msgr2=0x7ff57015c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.019+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 shutdown_connections 2024-05-30T17:03:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:45.019+0000 7ff56fe00640 1 -- 172.21.15.67:0/533247380 wait complete. 2024-05-30T17:03:45.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:03:45.039 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd 2024-05-30T17:03:45.039 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T17:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd 2024-05-30T17:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3138256896" 2024-05-30T17:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:03:45.103 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-30T17:03:45.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:03:45.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:03:47.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:03:47.104 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd 2024-05-30T17:03:47.104 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T17:03:47.173 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd 2024-05-30T17:03:47.173 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3221225472" 2024-05-30T17:03:47.173 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:03:47.174 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-30T17:03:47.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone93311 2024-05-30T17:03:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.391+0000 7f9987a00640 1 Processor -- start 2024-05-30T17:03:47.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.391+0000 7f9987a00640 1 -- start start 2024-05-30T17:03:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.393+0000 7f9987a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998015e810 0x7f9980072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.393+0000 7f9987a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9980072c80 0x7f99800d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.393+0000 7f9987a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f99800d0c10 0x7f99800b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.393+0000 7f9987a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f99800ad1a0 con 0x7f9980072c80 2024-05-30T17:03:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.393+0000 7f9987a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f998015ebf0 con 0x7f99800d0c10 2024-05-30T17:03:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.393+0000 7f9987a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f998006dbc0 con 0x7f998015e810 2024-05-30T17:03:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.393+0000 7f9977e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9980072c80 0x7f99800d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.393+0000 7f9977e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9980072c80 0x7f99800d06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52488/0 (socket says 172.21.15.67:52488) 2024-05-30T17:03:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.393+0000 7f9977e00640 1 -- 172.21.15.67:0/352600324 learned_addr learned my addr 172.21.15.67:0/352600324 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.394+0000 7f9985a00640 1 --2- 172.21.15.67:0/352600324 >> v2:172.21.15.169:3300/0 conn(0x7f99800d0c10 0x7f99800b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.394+0000 7f9985000640 1 --2- 172.21.15.67:0/352600324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998015e810 0x7f9980072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.394+0000 7f9977e00640 1 -- 172.21.15.67:0/352600324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998015e810 msgr2=0x7f9980072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.394+0000 7f9977e00640 1 --2- 172.21.15.67:0/352600324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998015e810 0x7f9980072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.394+0000 7f9977e00640 1 -- 172.21.15.67:0/352600324 >> v2:172.21.15.169:3300/0 conn(0x7f99800d0c10 msgr2=0x7f99800b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.394+0000 7f9977e00640 1 --2- 172.21.15.67:0/352600324 >> v2:172.21.15.169:3300/0 conn(0x7f99800d0c10 0x7f99800b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.394+0000 7f9977e00640 1 -- 172.21.15.67:0/352600324 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998006dea0 con 0x7f9980072c80 2024-05-30T17:03:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.395+0000 7f9977e00640 1 --2- 172.21.15.67:0/352600324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 0x7f99800d06d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f99800b4f90 tx=0x7f996802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbb116531fb8fe server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f997fe00640 1 -- 172.21.15.67:0/352600324 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968037070 con 0x7f9980072c80 2024-05-30T17:03:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f997fe00640 1 -- 172.21.15.67:0/352600324 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9968032440 con 0x7f9980072c80 2024-05-30T17:03:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f997fe00640 1 -- 172.21.15.67:0/352600324 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968032810 con 0x7f9980072c80 2024-05-30T17:03:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f9987a00640 1 -- 172.21.15.67:0/352600324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 msgr2=0x7f99800d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f9987a00640 1 --2- 172.21.15.67:0/352600324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 0x7f99800d06d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f99800b4f90 tx=0x7f996802a1e0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f9987a00640 1 -- 172.21.15.67:0/352600324 shutdown_connections 2024-05-30T17:03:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f9987a00640 1 --2- 172.21.15.67:0/352600324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 0x7f99800d06d0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f9987a00640 1 --2- 172.21.15.67:0/352600324 >> v2:172.21.15.169:3300/0 conn(0x7f99800d0c10 0x7f99800b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f9987a00640 1 --2- 172.21.15.67:0/352600324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998015e810 0x7f9980072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.396+0000 7f9987a00640 1 -- 172.21.15.67:0/352600324 >> 172.21.15.67:0/352600324 conn(0x7f998006bd20 msgr2=0x7f998006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.397+0000 7f9987a00640 1 -- 172.21.15.67:0/352600324 shutdown_connections 2024-05-30T17:03:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.397+0000 7f9987a00640 1 -- 172.21.15.67:0/352600324 wait complete. 2024-05-30T17:03:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.399+0000 7f9987a00640 1 Processor -- start 2024-05-30T17:03:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.399+0000 7f9987a00640 1 -- start start 2024-05-30T17:03:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.400+0000 7f9987a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 0x7f99801484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.400+0000 7f9987a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99800d0c10 0x7f9980072010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.400+0000 7f9987a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f998015e810 0x7f9980072550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.400+0000 7f9987a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f998006e740 con 0x7f9980072c80 2024-05-30T17:03:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.400+0000 7f9987a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f998006e460 con 0x7f998015e810 2024-05-30T17:03:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.400+0000 7f9987a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f99800b3700 con 0x7f99800d0c10 2024-05-30T17:03:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.400+0000 7f9985000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 0x7f99801484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.400+0000 7f9985000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 0x7f99801484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52490/0 (socket says 172.21.15.67:52490) 2024-05-30T17:03:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.400+0000 7f9985000640 1 -- 172.21.15.67:0/3578755278 learned_addr learned my addr 172.21.15.67:0/3578755278 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.401+0000 7f9985a00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f998015e810 0x7f9980072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.401+0000 7f9977e00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99800d0c10 0x7f9980072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.401+0000 7f9985000640 1 -- 172.21.15.67:0/3578755278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99800d0c10 msgr2=0x7f9980072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.401+0000 7f9985000640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99800d0c10 0x7f9980072010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.401+0000 7f9985000640 1 -- 172.21.15.67:0/3578755278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f998015e810 msgr2=0x7f9980072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.401+0000 7f9985000640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f998015e810 0x7f9980072550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.401+0000 7f9985000640 1 -- 172.21.15.67:0/3578755278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99801605d0 con 0x7f9980072c80 2024-05-30T17:03:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.401+0000 7f9985000640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 0x7f99801484f0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f997000d620 tx=0x7f997000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.402+0000 7f997d600640 1 -- 172.21.15.67:0/3578755278 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997000b620 con 0x7f9980072c80 2024-05-30T17:03:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.402+0000 7f997d600640 1 -- 172.21.15.67:0/3578755278 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9970019070 con 0x7f9980072c80 2024-05-30T17:03:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.402+0000 7f997d600640 1 -- 172.21.15.67:0/3578755278 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997000ba00 con 0x7f9980072c80 2024-05-30T17:03:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.402+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99801608b0 con 0x7f9980072c80 2024-05-30T17:03:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.402+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9980155db0 con 0x7f9980072c80 2024-05-30T17:03:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.403+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99800689e0 con 0x7f9980072c80 2024-05-30T17:03:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.404+0000 7f997d600640 1 -- 172.21.15.67:0/3578755278 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 39) ==== 93233+0+0 (secure 0 0 0) 0x7f997000bba0 con 0x7f9980072c80 2024-05-30T17:03:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.405+0000 7f997d600640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f995c06e150 0x7f995c070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.405+0000 7f997d600640 1 -- 172.21.15.67:0/3578755278 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f997008acf0 con 0x7f9980072c80 2024-05-30T17:03:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.405+0000 7f9977e00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f995c06e150 0x7f995c070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.406+0000 7f9977e00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f995c06e150 0x7f995c070610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9968034000 tx=0x7f996802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:47.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.411+0000 7f997d600640 1 -- 172.21.15.67:0/3578755278 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f997005ab80 con 0x7f9980072c80 2024-05-30T17:03:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.656+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone93311", "target": ["mon-mgr", ""]}) -- 0x7f9980155ba0 con 0x7f995c06e150 2024-05-30T17:03:47.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.844+0000 7f997d600640 1 -- 172.21.15.67:0/3578755278 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9980155ba0 con 0x7f995c06e150 2024-05-30T17:03:47.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.847+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f995c06e150 msgr2=0x7f995c070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:47.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.847+0000 7f9987a00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f995c06e150 0x7f995c070610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9968034000 tx=0x7f996802e000 comp rx=0 tx=0).stop 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.847+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 msgr2=0x7f99801484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.848+0000 7f9987a00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 0x7f99801484f0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f997000d620 tx=0x7f997000dae0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.848+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 shutdown_connections 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.848+0000 7f9987a00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f995c06e150 0x7f995c070610 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.848+0000 7f9987a00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f998015e810 0x7f9980072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.848+0000 7f9987a00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99800d0c10 0x7f9980072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.848+0000 7f9987a00640 1 --2- 172.21.15.67:0/3578755278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9980072c80 0x7f99801484f0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.848+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 >> 172.21.15.67:0/3578755278 conn(0x7f998006bd20 msgr2=0x7f99800b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.848+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 shutdown_connections 2024-05-30T17:03:47.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:47.848+0000 7f9987a00640 1 -- 172.21.15.67:0/3578755278 wait complete. 2024-05-30T17:03:47.851 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.851+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:47.851 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.851+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:47.851 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.851+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:47.851 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.851+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:47.851 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.851+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:47.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone93312 2024-05-30T17:03:47.884 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.884+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:47.884 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.884+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:47.884 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.884+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:47.885 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.884+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:47.885 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:47.884+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:48.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.070+0000 7fdbdb800640 1 Processor -- start 2024-05-30T17:03:48.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.071+0000 7fdbdb800640 1 -- start start 2024-05-30T17:03:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.072+0000 7fdbdb800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd4156bd0 0x7fdbd40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.072+0000 7fdbdb800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdbd40b4e30 0x7fdbd415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.072+0000 7fdbdb800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd4156fb0 con 0x7fdbd415d030 2024-05-30T17:03:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.072+0000 7fdbdb800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd415dae0 con 0x7fdbd40b4e30 2024-05-30T17:03:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.072+0000 7fdbdb800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd415d520 con 0x7fdbd4156bd0 2024-05-30T17:03:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.072+0000 7fdbd3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdbd40b4e30 0x7fdbd415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.073+0000 7fdbd3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdbd40b4e30 0x7fdbd415caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52518/0 (socket says 172.21.15.67:52518) 2024-05-30T17:03:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.073+0000 7fdbd3e00640 1 -- 172.21.15.67:0/1261352282 learned_addr learned my addr 172.21.15.67:0/1261352282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:48.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.073+0000 7fdbd3e00640 1 -- 172.21.15.67:0/1261352282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd4156bd0 msgr2=0x7fdbd40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:48.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.073+0000 7fdbd3e00640 1 --2- 172.21.15.67:0/1261352282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd4156bd0 0x7fdbd40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:48.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.073+0000 7fdbd3e00640 1 -- 172.21.15.67:0/1261352282 >> v1:172.21.15.169:6789/0 conn(0x7fdbd415d030 legacy=0x7fdbd40d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:48.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.073+0000 7fdbd3e00640 1 -- 172.21.15.67:0/1261352282 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd415d800 con 0x7fdbd40b4e30 2024-05-30T17:03:48.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.074+0000 7fdbd3e00640 1 --2- 172.21.15.67:0/1261352282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 0x7fdbd415caf0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4074150 tx=0x7fdbbc02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03de46d9dcbd8b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:48.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.074+0000 7fdbd3400640 1 -- 172.21.15.67:0/1261352282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc037070 con 0x7fdbd40b4e30 2024-05-30T17:03:48.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.075+0000 7fdbd3400640 1 -- 172.21.15.67:0/1261352282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbbc032440 con 0x7fdbd40b4e30 2024-05-30T17:03:48.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.075+0000 7fdbd3400640 1 -- 172.21.15.67:0/1261352282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc032810 con 0x7fdbd40b4e30 2024-05-30T17:03:48.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.075+0000 7fdbdb800640 1 -- 172.21.15.67:0/1261352282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 msgr2=0x7fdbd415caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:48.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.075+0000 7fdbdb800640 1 --2- 172.21.15.67:0/1261352282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 0x7fdbd415caf0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4074150 tx=0x7fdbbc02a160 comp rx=0 tx=0).stop 2024-05-30T17:03:48.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.075+0000 7fdbdb800640 1 -- 172.21.15.67:0/1261352282 shutdown_connections 2024-05-30T17:03:48.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.075+0000 7fdbdb800640 1 --2- 172.21.15.67:0/1261352282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 0x7fdbd415caf0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:48.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.075+0000 7fdbdb800640 1 --2- 172.21.15.67:0/1261352282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd4156bd0 0x7fdbd40b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:48.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.075+0000 7fdbdb800640 1 -- 172.21.15.67:0/1261352282 >> 172.21.15.67:0/1261352282 conn(0x7fdbd406bd20 msgr2=0x7fdbd406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:48.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.075+0000 7fdbdb800640 1 -- 172.21.15.67:0/1261352282 shutdown_connections 2024-05-30T17:03:48.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.076+0000 7fdbdb800640 1 -- 172.21.15.67:0/1261352282 wait complete. 2024-05-30T17:03:48.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.077+0000 7fdbdb800640 1 Processor -- start 2024-05-30T17:03:48.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.077+0000 7fdbdb800640 1 -- start start 2024-05-30T17:03:48.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.078+0000 7fdbdb800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 0x7fdbd4073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.078+0000 7fdbdb800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd4156bd0 0x7fdbd414e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.078+0000 7fdbdb800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd415d030 0x7fdbd4157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbdb800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd40b4030 con 0x7fdbd40b4e30 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbdb800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd40b35d0 con 0x7fdbd415d030 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbdb800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd40b41b0 con 0x7fdbd4156bd0 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbd8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 0x7fdbd4073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbd8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 0x7fdbd4073000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52532/0 (socket says 172.21.15.67:52532) 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbd8e00640 1 -- 172.21.15.67:0/4050148135 learned_addr learned my addr 172.21.15.67:0/4050148135 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbd8e00640 1 -- 172.21.15.67:0/4050148135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd4156bd0 msgr2=0x7fdbd414e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbd3e00640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd4156bd0 0x7fdbd414e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbd8e00640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd4156bd0 0x7fdbd414e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:48.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbd8e00640 1 -- 172.21.15.67:0/4050148135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd415d030 msgr2=0x7fdbd4157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:48.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbd8e00640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd415d030 0x7fdbd4157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:48.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.079+0000 7fdbd8e00640 1 -- 172.21.15.67:0/4050148135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd4071dd0 con 0x7fdbd40b4e30 2024-05-30T17:03:48.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.080+0000 7fdbd8e00640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 0x7fdbd4073000 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400ba50 tx=0x7fdbc400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:48.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.080+0000 7fdbd0c00640 1 -- 172.21.15.67:0/4050148135 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4019070 con 0x7fdbd40b4e30 2024-05-30T17:03:48.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.080+0000 7fdbd0c00640 1 -- 172.21.15.67:0/4050148135 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc400cc00 con 0x7fdbd40b4e30 2024-05-30T17:03:48.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.080+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd4073700 con 0x7fdbd40b4e30 2024-05-30T17:03:48.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.080+0000 7fdbd0c00640 1 -- 172.21.15.67:0/4050148135 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4014440 con 0x7fdbd40b4e30 2024-05-30T17:03:48.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.080+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd41580f0 con 0x7fdbd40b4e30 2024-05-30T17:03:48.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.081+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd40689e0 con 0x7fdbd40b4e30 2024-05-30T17:03:48.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.082+0000 7fdbd0c00640 1 -- 172.21.15.67:0/4050148135 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 39) ==== 93233+0+0 (secure 0 0 0) 0x7fdbc401c070 con 0x7fdbd40b4e30 2024-05-30T17:03:48.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.083+0000 7fdbd0c00640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbb006e120 0x7fdbb00705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:48.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.083+0000 7fdbd0c00640 1 -- 172.21.15.67:0/4050148135 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7fdbc408a760 con 0x7fdbd40b4e30 2024-05-30T17:03:48.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.084+0000 7fdbd3e00640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbb006e120 0x7fdbb00705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:48.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.084+0000 7fdbd3e00640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbb006e120 0x7fdbb00705e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc034000 tx=0x7fdbbc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:48.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.088+0000 7fdbd0c00640 1 -- 172.21.15.67:0/4050148135 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbc405a5f0 con 0x7fdbd40b4e30 2024-05-30T17:03:48.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.331+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone93312", "target": ["mon-mgr", ""]}) -- 0x7fdbd4073c20 con 0x7fdbb006e120 2024-05-30T17:03:48.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:48.851+0000 7f0c1d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:48.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:48.851+0000 7f0c1d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:48.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:48.851+0000 7f0c1d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:48.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:48.852+0000 7f0c1d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:48.852 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:48.852+0000 7f0c1d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:48.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:48.860+0000 7fdbd0c00640 1 -- 172.21.15.67:0/4050148135 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 40) ==== 93335+0+0 (secure 0 0 0) 0x7fdbc4056560 con 0x7fdbd40b4e30 2024-05-30T17:03:49.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.085+0000 7fdbd0c00640 1 -- 172.21.15.67:0/4050148135 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdbd4073c20 con 0x7fdbb006e120 2024-05-30T17:03:49.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.088+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbb006e120 msgr2=0x7fdbb00705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.088+0000 7fdbdb800640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbb006e120 0x7fdbb00705e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc034000 tx=0x7fdbbc02e000 comp rx=0 tx=0).stop 2024-05-30T17:03:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.088+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 msgr2=0x7fdbd4073000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.088+0000 7fdbdb800640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 0x7fdbd4073000 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400ba50 tx=0x7fdbc400bf10 comp rx=0 tx=0).stop 2024-05-30T17:03:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.089+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 shutdown_connections 2024-05-30T17:03:49.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.089+0000 7fdbdb800640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdbb006e120 0x7fdbb00705e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:49.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.089+0000 7fdbdb800640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd415d030 0x7fdbd4157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:49.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.089+0000 7fdbdb800640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbd4156bd0 0x7fdbd414e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:49.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.089+0000 7fdbdb800640 1 --2- 172.21.15.67:0/4050148135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdbd40b4e30 0x7fdbd4073000 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:49.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.089+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 >> 172.21.15.67:0/4050148135 conn(0x7fdbd406bd20 msgr2=0x7fdbd415bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:49.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.089+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 shutdown_connections 2024-05-30T17:03:49.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.089+0000 7fdbdb800640 1 -- 172.21.15.67:0/4050148135 wait complete. 2024-05-30T17:03:49.117 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:49.117+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:49.117 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:49.117+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:49.117 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:49.117+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:49.117 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:49.117+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:49.117 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:49.117+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:49.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone93313 2024-05-30T17:03:49.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.476+0000 7f9b2ea00640 1 Processor -- start 2024-05-30T17:03:49.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.476+0000 7f9b2ea00640 1 -- start start 2024-05-30T17:03:49.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.479+0000 7f9b2ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9b28155000 0x7f9b2815b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:49.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.480+0000 7f9b2ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b280d0e10 0x7f9b2806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:49.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.480+0000 7f9b2ea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2815bdd0 con 0x7f9b280b3c30 2024-05-30T17:03:49.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.480+0000 7f9b2ea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9b280b3510 con 0x7f9b280d0e10 2024-05-30T17:03:49.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.480+0000 7f9b2ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9b2806e530 con 0x7f9b28155000 2024-05-30T17:03:49.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.481+0000 7f9b27400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9b280b3c30 0x7f9b280d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45350/0 (socket says 172.21.15.67:45350) 2024-05-30T17:03:49.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.481+0000 7f9b27400640 1 -- 172.21.15.67:0/2550435113 learned_addr learned my addr 172.21.15.67:0/2550435113 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:03:49.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.482+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2550435113 >> v2:172.21.15.169:3300/0 conn(0x7f9b28155000 0x7f9b2815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:49.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.482+0000 7f9b2ca00640 1 --2- 172.21.15.67:0/2550435113 >> v2:172.21.15.67:3300/0 conn(0x7f9b280d0e10 0x7f9b2806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:49.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.483+0000 7f9b26a00640 1 -- 172.21.15.67:0/2550435113 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1949545578 0 0) 0x7f9b14001110 con 0x7f9b280b3c30 2024-05-30T17:03:49.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.484+0000 7f9b26a00640 1 -- 172.21.15.67:0/2550435113 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b0c003520 con 0x7f9b280b3c30 2024-05-30T17:03:49.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.484+0000 7f9b2ca00640 1 -- 172.21.15.67:0/2550435113 >> v2:172.21.15.169:3300/0 conn(0x7f9b28155000 msgr2=0x7f9b2815b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:49.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.484+0000 7f9b2ca00640 1 --2- 172.21.15.67:0/2550435113 >> v2:172.21.15.169:3300/0 conn(0x7f9b28155000 0x7f9b2815b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:49.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.484+0000 7f9b2ca00640 1 -- 172.21.15.67:0/2550435113 >> v1:172.21.15.67:6789/0 conn(0x7f9b280b3c30 legacy=0x7f9b280d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:49.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.484+0000 7f9b2ca00640 1 -- 172.21.15.67:0/2550435113 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b280b40d0 con 0x7f9b280d0e10 2024-05-30T17:03:49.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.486+0000 7f9b2ca00640 1 --2- 172.21.15.67:0/2550435113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b280d0e10 0x7f9b2806e9c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9b28071590 tx=0x7f9b2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa07031abcb15f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:49.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.487+0000 7f9b26a00640 1 -- 172.21.15.67:0/2550435113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20007740 con 0x7f9b280d0e10 2024-05-30T17:03:49.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.487+0000 7f9b26a00640 1 -- 172.21.15.67:0/2550435113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b200078e0 con 0x7f9b280d0e10 2024-05-30T17:03:49.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.487+0000 7f9b26a00640 1 -- 172.21.15.67:0/2550435113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20007740 con 0x7f9b280d0e10 2024-05-30T17:03:49.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.488+0000 7f9b2ea00640 1 -- 172.21.15.67:0/2550435113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b280d0e10 msgr2=0x7f9b2806e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:49.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.488+0000 7f9b2ea00640 1 --2- 172.21.15.67:0/2550435113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b280d0e10 0x7f9b2806e9c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9b28071590 tx=0x7f9b2002a680 comp rx=0 tx=0).stop 2024-05-30T17:03:49.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.489+0000 7f9b2ea00640 1 -- 172.21.15.67:0/2550435113 shutdown_connections 2024-05-30T17:03:49.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.489+0000 7f9b2ea00640 1 --2- 172.21.15.67:0/2550435113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b280d0e10 0x7f9b2806e9c0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:49.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.489+0000 7f9b2ea00640 1 --2- 172.21.15.67:0/2550435113 >> v2:172.21.15.169:3300/0 conn(0x7f9b28155000 0x7f9b2815b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:49.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.489+0000 7f9b2ea00640 1 -- 172.21.15.67:0/2550435113 >> 172.21.15.67:0/2550435113 conn(0x7f9b280b56e0 msgr2=0x7f9b280b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:49.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.489+0000 7f9b2ea00640 1 -- 172.21.15.67:0/2550435113 shutdown_connections 2024-05-30T17:03:49.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.490+0000 7f9b2ea00640 1 -- 172.21.15.67:0/2550435113 wait complete. 2024-05-30T17:03:49.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.492+0000 7f9b2ea00640 1 Processor -- start 2024-05-30T17:03:49.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.493+0000 7f9b2ea00640 1 -- start start 2024-05-30T17:03:49.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.495+0000 7f9b2ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b280b3c30 0x7f9b2815d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:49.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.495+0000 7f9b2ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b280d0e10 0x7f9b281f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:49.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.495+0000 7f9b2ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b28155000 0x7f9b2815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:49.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.495+0000 7f9b2ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9b280d02b0 con 0x7f9b28155000 2024-05-30T17:03:49.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.495+0000 7f9b2ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2806e1c0 con 0x7f9b280d0e10 2024-05-30T17:03:49.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.495+0000 7f9b2ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9b280d0430 con 0x7f9b280b3c30 2024-05-30T17:03:49.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.496+0000 7f9b27e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b280b3c30 0x7f9b2815d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:49.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.496+0000 7f9b27e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b280b3c30 0x7f9b2815d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:53592/0 (socket says 172.21.15.67:53592) 2024-05-30T17:03:49.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.496+0000 7f9b27e00640 1 -- 172.21.15.67:0/1636938837 learned_addr learned my addr 172.21.15.67:0/1636938837 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:49.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.497+0000 7f9b2ca00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b28155000 0x7f9b2815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:49.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.498+0000 7f9b27e00640 1 -- 172.21.15.67:0/1636938837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b280d0e10 msgr2=0x7f9b281f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:49.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.498+0000 7f9b27e00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b280d0e10 0x7f9b281f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:49.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.498+0000 7f9b27e00640 1 -- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b28155000 msgr2=0x7f9b2815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.500+0000 7f9b27e00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b28155000 0x7f9b2815ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:49.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.500+0000 7f9b27e00640 1 -- 172.21.15.67:0/1636938837 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2815dc10 con 0x7f9b280b3c30 2024-05-30T17:03:49.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.501+0000 7f9b2ca00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b28155000 0x7f9b2815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:03:49.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.502+0000 7f9b27e00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b280b3c30 0x7f9b2815d6d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800d5f0 tx=0x7f9b1800dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:49.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.502+0000 7f9b1fe00640 1 -- 172.21.15.67:0/1636938837 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b180125f0 con 0x7f9b280b3c30 2024-05-30T17:03:49.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.503+0000 7f9b1fe00640 1 -- 172.21.15.67:0/1636938837 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b18019070 con 0x7f9b280b3c30 2024-05-30T17:03:49.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.503+0000 7f9b1fe00640 1 -- 172.21.15.67:0/1636938837 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b180129d0 con 0x7f9b280b3c30 2024-05-30T17:03:49.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.504+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b281f4880 con 0x7f9b280b3c30 2024-05-30T17:03:49.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.504+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b280b1ce0 con 0x7f9b280b3c30 2024-05-30T17:03:49.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.507+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b280689e0 con 0x7f9b280b3c30 2024-05-30T17:03:49.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.510+0000 7f9b1fe00640 1 -- 172.21.15.67:0/1636938837 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 40) ==== 93335+0+0 (secure 0 0 0) 0x7f9b18012b70 con 0x7f9b280b3c30 2024-05-30T17:03:49.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.512+0000 7f9b1fe00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b0806e240 0x7f9b08070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:49.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.512+0000 7f9b1fe00640 1 -- 172.21.15.67:0/1636938837 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f9b1808b060 con 0x7f9b280b3c30 2024-05-30T17:03:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.520+0000 7f9b1fe00640 1 -- 172.21.15.67:0/1636938837 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b1805af70 con 0x7f9b280b3c30 2024-05-30T17:03:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.524+0000 7f9b27400640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b0806e240 0x7f9b08070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:49.533+0000 7f9b27400640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b0806e240 0x7f9b08070700 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9b14000f60 tx=0x7f9b14001110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.064+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone93313", "target": ["mon-mgr", ""]}) -- 0x7f9b28149780 con 0x7f9b0806e240 2024-05-30T17:03:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.714+0000 7f9b1fe00640 1 -- 172.21.15.67:0/1636938837 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9b28149780 con 0x7f9b0806e240 2024-05-30T17:03:50.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b0806e240 msgr2=0x7f9b08070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b0806e240 0x7f9b08070700 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9b14000f60 tx=0x7f9b14001110 comp rx=0 tx=0).stop 2024-05-30T17:03:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b280b3c30 msgr2=0x7f9b2815d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b280b3c30 0x7f9b2815d6d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800d5f0 tx=0x7f9b1800dab0 comp rx=0 tx=0).stop 2024-05-30T17:03:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 shutdown_connections 2024-05-30T17:03:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b0806e240 0x7f9b08070700 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b28155000 0x7f9b2815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b280d0e10 0x7f9b281f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 --2- 172.21.15.67:0/1636938837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b280b3c30 0x7f9b2815d6d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.727+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 >> 172.21.15.67:0/1636938837 conn(0x7f9b280b56e0 msgr2=0x7f9b2806eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.731+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 shutdown_connections 2024-05-30T17:03:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:50.731+0000 7f9b2ea00640 1 -- 172.21.15.67:0/1636938837 wait complete. 2024-05-30T17:03:50.782 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:50.782+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:50.782 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:50.782+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:50.782 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:50.782+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:50.783 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:50.782+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:50.783 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:03:50.782+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:03:51.771 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone93311 2024-05-30T17:03:52.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.222+0000 7f31b9c00640 1 Processor -- start 2024-05-30T17:03:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.223+0000 7f31b9c00640 1 -- start start 2024-05-30T17:03:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.226+0000 7f31b9c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f31b415e720 0x7f31b40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:52.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.227+0000 7f31b9c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f31b40d1560 0x7f31b40d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.227+0000 7f31b9c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b40d2280 con 0x7f31b40d3320 2024-05-30T17:03:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.227+0000 7f31b9c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f31b40d2050 con 0x7f31b415e720 2024-05-30T17:03:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.227+0000 7f31b9c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f31b40d2e90 con 0x7f31b40d1560 2024-05-30T17:03:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.227+0000 7f31ab400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f31b40d3320 0x7f31b40d0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43480/0 (socket says 172.21.15.67:43480) 2024-05-30T17:03:52.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.227+0000 7f31ab400640 1 -- 172.21.15.67:0/873462316 learned_addr learned my addr 172.21.15.67:0/873462316 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:03:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.228+0000 7f31b3e00640 1 --2- 172.21.15.67:0/873462316 >> v2:172.21.15.67:3300/0 conn(0x7f31b415e720 0x7f31b40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:52.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.228+0000 7f31b9200640 1 --2- 172.21.15.67:0/873462316 >> v2:172.21.15.169:3300/0 conn(0x7f31b40d1560 0x7f31b40d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:52.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.229+0000 7f31b3e00640 1 -- 172.21.15.67:0/873462316 >> v2:172.21.15.169:3300/0 conn(0x7f31b40d1560 msgr2=0x7f31b40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:52.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.229+0000 7f31b3e00640 1 --2- 172.21.15.67:0/873462316 >> v2:172.21.15.169:3300/0 conn(0x7f31b40d1560 0x7f31b40d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:52.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.229+0000 7f31b3e00640 1 -- 172.21.15.67:0/873462316 >> v1:172.21.15.67:6789/0 conn(0x7f31b40d3320 legacy=0x7f31b40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:52.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.229+0000 7f31b3e00640 1 -- 172.21.15.67:0/873462316 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b40d2560 con 0x7f31b415e720 2024-05-30T17:03:52.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.232+0000 7f31b3e00640 1 --2- 172.21.15.67:0/873462316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 0x7f31b40cfb80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f31b408e150 tx=0x7f31a402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722d2ec5a00052f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.235+0000 7f31b3400640 1 -- 172.21.15.67:0/873462316 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4037070 con 0x7f31b415e720 2024-05-30T17:03:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.235+0000 7f31b3400640 1 -- 172.21.15.67:0/873462316 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a4032440 con 0x7f31b415e720 2024-05-30T17:03:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.235+0000 7f31b3400640 1 -- 172.21.15.67:0/873462316 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4032850 con 0x7f31b415e720 2024-05-30T17:03:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.236+0000 7f31b9c00640 1 -- 172.21.15.67:0/873462316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 msgr2=0x7f31b40cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:52.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.236+0000 7f31b9c00640 1 --2- 172.21.15.67:0/873462316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 0x7f31b40cfb80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f31b408e150 tx=0x7f31a402a130 comp rx=0 tx=0).stop 2024-05-30T17:03:52.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.236+0000 7f31b9c00640 1 -- 172.21.15.67:0/873462316 shutdown_connections 2024-05-30T17:03:52.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.236+0000 7f31b9c00640 1 --2- 172.21.15.67:0/873462316 >> v2:172.21.15.169:3300/0 conn(0x7f31b40d1560 0x7f31b40d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:52.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.236+0000 7f31b9c00640 1 --2- 172.21.15.67:0/873462316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 0x7f31b40cfb80 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:52.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.236+0000 7f31b9c00640 1 -- 172.21.15.67:0/873462316 >> 172.21.15.67:0/873462316 conn(0x7f31b408bfc0 msgr2=0x7f31b40d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:52.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.237+0000 7f31b9c00640 1 -- 172.21.15.67:0/873462316 shutdown_connections 2024-05-30T17:03:52.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.237+0000 7f31b9c00640 1 -- 172.21.15.67:0/873462316 wait complete. 2024-05-30T17:03:52.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.240+0000 7f31b9c00640 1 Processor -- start 2024-05-30T17:03:52.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.240+0000 7f31b9c00640 1 -- start start 2024-05-30T17:03:52.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.242+0000 7f31b9c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40d1560 0x7f31b40c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:52.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.242+0000 7f31b9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40d3320 0x7f31b40cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:52.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.242+0000 7f31b9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 0x7f31b40cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:52.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.242+0000 7f31b9c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f31b415eb00 con 0x7f31b415e720 2024-05-30T17:03:52.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.242+0000 7f31b9c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40d2b20 con 0x7f31b40d1560 2024-05-30T17:03:52.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.242+0000 7f31b9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f31b40b72d0 con 0x7f31b40d3320 2024-05-30T17:03:52.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.244+0000 7f31b9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 0x7f31b40cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:52.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.245+0000 7f31b3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40d1560 0x7f31b40c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:52.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.245+0000 7f31b3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40d1560 0x7f31b40c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:49738/0 (socket says 172.21.15.67:49738) 2024-05-30T17:03:52.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.245+0000 7f31b3e00640 1 -- 172.21.15.67:0/943999375 learned_addr learned my addr 172.21.15.67:0/943999375 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:52.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.245+0000 7f31b9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 0x7f31b40cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54692/0 (socket says 172.21.15.67:54692) 2024-05-30T17:03:52.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.246+0000 7f31ab400640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40d3320 0x7f31b40cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:52.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.247+0000 7f31b9200640 1 -- 172.21.15.67:0/943999375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40d3320 msgr2=0x7f31b40cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:52.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.247+0000 7f31b9200640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40d3320 0x7f31b40cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:52.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.247+0000 7f31b9200640 1 -- 172.21.15.67:0/943999375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40d1560 msgr2=0x7f31b40c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:52.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.247+0000 7f31b9200640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40d1560 0x7f31b40c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:52.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.247+0000 7f31b9200640 1 -- 172.21.15.67:0/943999375 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b40c8df0 con 0x7f31b415e720 2024-05-30T17:03:52.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.247+0000 7f31b3e00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40d1560 0x7f31b40c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:03:52.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.248+0000 7f31b9200640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 0x7f31b40cbdf0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f31b40d1970 tx=0x7f31ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:52.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.250+0000 7f31b0c00640 1 -- 172.21.15.67:0/943999375 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00edd0 con 0x7f31b415e720 2024-05-30T17:03:52.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.250+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b40c90d0 con 0x7f31b415e720 2024-05-30T17:03:52.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.250+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b40cb8d0 con 0x7f31b415e720 2024-05-30T17:03:52.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.253+0000 7f31b0c00640 1 -- 172.21.15.67:0/943999375 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac003990 con 0x7f31b415e720 2024-05-30T17:03:52.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.253+0000 7f31b0c00640 1 -- 172.21.15.67:0/943999375 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac003d80 con 0x7f31b415e720 2024-05-30T17:03:52.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.256+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c005290 con 0x7f31b415e720 2024-05-30T17:03:52.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.259+0000 7f31b0c00640 1 -- 172.21.15.67:0/943999375 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 41) ==== 93437+0+0 (secure 0 0 0) 0x7f31ac01b020 con 0x7f31b415e720 2024-05-30T17:03:52.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.280+0000 7f31b0c00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f319006e360 0x7f3190070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:52.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.282+0000 7f31b0c00640 1 -- 172.21.15.67:0/943999375 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f31ac08aa40 con 0x7f31b415e720 2024-05-30T17:03:52.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.283+0000 7f31b0c00640 1 -- 172.21.15.67:0/943999375 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31ac05a280 con 0x7f31b415e720 2024-05-30T17:03:52.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.283+0000 7f31b3e00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f319006e360 0x7f3190070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:52.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.288+0000 7f31b3e00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f319006e360 0x7f3190070820 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f31a40325c0 tx=0x7f31a402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:52.816+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone93311", "target": ["mon-mgr", ""]}) -- 0x7f318c000bc0 con 0x7f319006e360 2024-05-30T17:03:53.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.037+0000 7f31b0c00640 1 -- 172.21.15.67:0/943999375 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7f31ac0568f0 con 0x7f31b415e720 2024-05-30T17:03:53.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.402+0000 7f31b0c00640 1 -- 172.21.15.67:0/943999375 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f318c000bc0 con 0x7f319006e360 2024-05-30T17:03:53.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f319006e360 msgr2=0x7f3190070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:53.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f319006e360 0x7f3190070820 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f31a40325c0 tx=0x7f31a402e000 comp rx=0 tx=0).stop 2024-05-30T17:03:53.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 msgr2=0x7f31b40cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:53.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 0x7f31b40cbdf0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f31b40d1970 tx=0x7f31ac00e4b0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 shutdown_connections 2024-05-30T17:03:53.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f319006e360 0x7f3190070820 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31b415e720 0x7f31b40cbdf0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40d3320 0x7f31b40cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 --2- 172.21.15.67:0/943999375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40d1560 0x7f31b40c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.405+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 >> 172.21.15.67:0/943999375 conn(0x7f31b408bfc0 msgr2=0x7f31b40d3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:53.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.406+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 shutdown_connections 2024-05-30T17:03:53.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.406+0000 7f31b9c00640 1 -- 172.21.15.67:0/943999375 wait complete. 2024-05-30T17:03:53.449 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone93312 2024-05-30T17:03:53.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.847+0000 7fb6a7800640 1 Processor -- start 2024-05-30T17:03:53.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.848+0000 7fb6a7800640 1 -- start start 2024-05-30T17:03:53.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.854+0000 7fb6a7800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb6a0162ac0 0x7fb6a00d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:53.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.854+0000 7fb6a7800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6a00b4120 0x7fb6a00d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:53.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.854+0000 7fb6a7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a01633b0 0x7fb6a0163800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:53.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.854+0000 7fb6a7800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a00b3c90 con 0x7fb6a0162ac0 2024-05-30T17:03:53.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.854+0000 7fb6a7800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a01640e0 con 0x7fb6a00b4120 2024-05-30T17:03:53.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.854+0000 7fb6a7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a00b3360 con 0x7fb6a01633b0 2024-05-30T17:03:53.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.855+0000 7fb69fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6a00b4120 0x7fb6a00d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:53.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.855+0000 7fb69fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6a00b4120 0x7fb6a00d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:49742/0 (socket says 172.21.15.67:49742) 2024-05-30T17:03:53.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.855+0000 7fb69fe00640 1 -- 172.21.15.67:0/2364927001 learned_addr learned my addr 172.21.15.67:0/2364927001 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:53.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.855+0000 7fb6a5800640 1 --2- 172.21.15.67:0/2364927001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a01633b0 0x7fb6a0163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:53.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.857+0000 7fb69fe00640 1 -- 172.21.15.67:0/2364927001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a01633b0 msgr2=0x7fb6a0163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:53.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.857+0000 7fb69fe00640 1 --2- 172.21.15.67:0/2364927001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a01633b0 0x7fb6a0163800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.857+0000 7fb69fe00640 1 -- 172.21.15.67:0/2364927001 >> v2:172.21.15.67:3300/0 conn(0x7fb6a0162ac0 msgr2=0x7fb6a00d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:53.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.860+0000 7fb6a4e00640 1 --2- 172.21.15.67:0/2364927001 >> v2:172.21.15.67:3300/0 conn(0x7fb6a0162ac0 0x7fb6a00d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:53.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.860+0000 7fb69fe00640 1 --2- 172.21.15.67:0/2364927001 >> v2:172.21.15.67:3300/0 conn(0x7fb6a0162ac0 0x7fb6a00d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.861+0000 7fb69fe00640 1 -- 172.21.15.67:0/2364927001 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a00b3640 con 0x7fb6a00b4120 2024-05-30T17:03:53.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.861+0000 7fb6a4e00640 1 --2- 172.21.15.67:0/2364927001 >> v2:172.21.15.67:3300/0 conn(0x7fb6a0162ac0 0x7fb6a00d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:03:53.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.865+0000 7fb69fe00640 1 --2- 172.21.15.67:0/2364927001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a00b4120 0x7fb6a00d0b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00b3c00 tx=0x7fb68c02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4478adfd8bec925 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:53.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.867+0000 7fb69f400640 1 -- 172.21.15.67:0/2364927001 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c037070 con 0x7fb6a00b4120 2024-05-30T17:03:53.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.867+0000 7fb69f400640 1 -- 172.21.15.67:0/2364927001 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c032440 con 0x7fb6a00b4120 2024-05-30T17:03:53.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.867+0000 7fb69f400640 1 -- 172.21.15.67:0/2364927001 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c032810 con 0x7fb6a00b4120 2024-05-30T17:03:53.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.873+0000 7fb6a7800640 1 -- 172.21.15.67:0/2364927001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a00b4120 msgr2=0x7fb6a00d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:53.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.873+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2364927001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a00b4120 0x7fb6a00d0b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00b3c00 tx=0x7fb68c02a1e0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.874+0000 7fb6a7800640 1 -- 172.21.15.67:0/2364927001 shutdown_connections 2024-05-30T17:03:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.874+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2364927001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a01633b0 0x7fb6a0163800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.874+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2364927001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a00b4120 0x7fb6a00d0b80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.874+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2364927001 >> v2:172.21.15.67:3300/0 conn(0x7fb6a0162ac0 0x7fb6a00d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.874+0000 7fb6a7800640 1 -- 172.21.15.67:0/2364927001 >> 172.21.15.67:0/2364927001 conn(0x7fb6a006bd20 msgr2=0x7fb6a006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:53.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.876+0000 7fb6a7800640 1 -- 172.21.15.67:0/2364927001 shutdown_connections 2024-05-30T17:03:53.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.878+0000 7fb6a7800640 1 -- 172.21.15.67:0/2364927001 wait complete. 2024-05-30T17:03:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.881+0000 7fb6a7800640 1 Processor -- start 2024-05-30T17:03:53.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.883+0000 7fb6a7800640 1 -- start start 2024-05-30T17:03:53.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.885+0000 7fb6a7800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a00b4120 0x7fb6a01603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:53.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.885+0000 7fb6a7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a0162ac0 0x7fb6a00cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:53.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.885+0000 7fb6a7800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a01633b0 0x7fb6a015d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:53.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.885+0000 7fb6a7800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a0163ec0 con 0x7fb6a00b4120 2024-05-30T17:03:53.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.885+0000 7fb6a7800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a0163d40 con 0x7fb6a01633b0 2024-05-30T17:03:53.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.885+0000 7fb6a7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a0162ea0 con 0x7fb6a0162ac0 2024-05-30T17:03:53.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.886+0000 7fb6a4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a00b4120 0x7fb6a01603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:53.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.886+0000 7fb6a4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a00b4120 0x7fb6a01603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54700/0 (socket says 172.21.15.67:54700) 2024-05-30T17:03:53.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.886+0000 7fb6a4e00640 1 -- 172.21.15.67:0/2248690308 learned_addr learned my addr 172.21.15.67:0/2248690308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:53.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.886+0000 7fb69fe00640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a0162ac0 0x7fb6a00cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:53.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.886+0000 7fb6a5800640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a01633b0 0x7fb6a015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:53.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.886+0000 7fb6a4e00640 1 -- 172.21.15.67:0/2248690308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a0162ac0 msgr2=0x7fb6a00cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:53.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.886+0000 7fb6a4e00640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a0162ac0 0x7fb6a00cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.887+0000 7fb6a4e00640 1 -- 172.21.15.67:0/2248690308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a01633b0 msgr2=0x7fb6a015d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:53.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.887+0000 7fb6a4e00640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a01633b0 0x7fb6a015d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:53.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.887+0000 7fb6a4e00640 1 -- 172.21.15.67:0/2248690308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a01608e0 con 0x7fb6a00b4120 2024-05-30T17:03:53.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.888+0000 7fb6a4e00640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a00b4120 0x7fb6a01603a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c7c0 tx=0x7fb69400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.890+0000 7fb69cc00640 1 -- 172.21.15.67:0/2248690308 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694015070 con 0x7fb6a00b4120 2024-05-30T17:03:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.890+0000 7fb69cc00640 1 -- 172.21.15.67:0/2248690308 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb694007a30 con 0x7fb6a00b4120 2024-05-30T17:03:53.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.890+0000 7fb69cc00640 1 -- 172.21.15.67:0/2248690308 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694007e00 con 0x7fb6a00b4120 2024-05-30T17:03:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.890+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a00cf490 con 0x7fb6a00b4120 2024-05-30T17:03:53.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.890+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a00cfa50 con 0x7fb6a00b4120 2024-05-30T17:03:53.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.892+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb67c005290 con 0x7fb6a00b4120 2024-05-30T17:03:53.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.895+0000 7fb69cc00640 1 -- 172.21.15.67:0/2248690308 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7fb694018070 con 0x7fb6a00b4120 2024-05-30T17:03:53.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.898+0000 7fb69cc00640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb68006e440 0x7fb680070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:53.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.898+0000 7fb69cc00640 1 -- 172.21.15.67:0/2248690308 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7fb69408ac90 con 0x7fb6a00b4120 2024-05-30T17:03:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.908+0000 7fb69fe00640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb68006e440 0x7fb680070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:53.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.912+0000 7fb69cc00640 1 -- 172.21.15.67:0/2248690308 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb69405a650 con 0x7fb6a00b4120 2024-05-30T17:03:53.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:53.938+0000 7fb69fe00640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb68006e440 0x7fb680070900 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb68c034000 tx=0x7fb68c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:54.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.458+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone93312", "target": ["mon-mgr", ""]}) -- 0x7fb67c000bc0 con 0x7fb68006e440 2024-05-30T17:03:54.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.761+0000 7fb69cc00640 1 -- 172.21.15.67:0/2248690308 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb67c000bc0 con 0x7fb68006e440 2024-05-30T17:03:54.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.763+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb68006e440 msgr2=0x7fb680070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:54.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.763+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb68006e440 0x7fb680070900 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb68c034000 tx=0x7fb68c02e000 comp rx=0 tx=0).stop 2024-05-30T17:03:54.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.763+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a00b4120 msgr2=0x7fb6a01603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.763+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a00b4120 0x7fb6a01603a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c7c0 tx=0x7fb69400cc80 comp rx=0 tx=0).stop 2024-05-30T17:03:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.764+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 shutdown_connections 2024-05-30T17:03:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.764+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb68006e440 0x7fb680070900 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.764+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6a01633b0 0x7fb6a015d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.764+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6a0162ac0 0x7fb6a00cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.764+0000 7fb6a7800640 1 --2- 172.21.15.67:0/2248690308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a00b4120 0x7fb6a01603a0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.764+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 >> 172.21.15.67:0/2248690308 conn(0x7fb6a006bd20 msgr2=0x7fb6a00ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.764+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 shutdown_connections 2024-05-30T17:03:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:54.764+0000 7fb6a7800640 1 -- 172.21.15.67:0/2248690308 wait complete. 2024-05-30T17:03:54.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone93313 2024-05-30T17:03:55.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.001+0000 7f02f6000640 1 Processor -- start 2024-05-30T17:03:55.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.001+0000 7f02f6000640 1 -- start start 2024-05-30T17:03:55.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.003+0000 7f02f6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f02f00ce5d0 0x7f02f00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:55.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.003+0000 7f02f6000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02f00b3580 con 0x7f02f00b3d60 2024-05-30T17:03:55.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.003+0000 7f02f6000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02f015f490 con 0x7f02f00ce1f0 2024-05-30T17:03:55.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.003+0000 7f02f6000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f02f015f770 con 0x7f02f00ce5d0 2024-05-30T17:03:55.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.003+0000 7f02efe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f02f00ce5d0 0x7f02f00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:55.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.003+0000 7f02efe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f02f00ce5d0 0x7f02f00b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54706/0 (socket says 172.21.15.67:54706) 2024-05-30T17:03:55.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.003+0000 7f02efe00640 1 -- 172.21.15.67:0/3676230792 learned_addr learned my addr 172.21.15.67:0/3676230792 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:55.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.004+0000 7f02efe00640 1 -- 172.21.15.67:0/3676230792 >> v1:172.21.15.169:6789/0 conn(0x7f02f00ce1f0 legacy=0x7f02f015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:03:55.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.004+0000 7f02efe00640 1 -- 172.21.15.67:0/3676230792 >> v1:172.21.15.67:6789/0 conn(0x7f02f00b3d60 legacy=0x7f02f00ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:55.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.004+0000 7f02efe00640 1 -- 172.21.15.67:0/3676230792 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f015fa50 con 0x7f02f00ce5d0 2024-05-30T17:03:55.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.005+0000 7f02efe00640 1 --2- 172.21.15.67:0/3676230792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 0x7f02f00b4dd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009870 tx=0x7f02e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2fe8adbdebb39 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:55.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.005+0000 7f02ef400640 1 -- 172.21.15.67:0/3676230792 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0037070 con 0x7f02f00ce5d0 2024-05-30T17:03:55.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.005+0000 7f02ef400640 1 -- 172.21.15.67:0/3676230792 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e0032440 con 0x7f02f00ce5d0 2024-05-30T17:03:55.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.005+0000 7f02ef400640 1 -- 172.21.15.67:0/3676230792 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0032810 con 0x7f02f00ce5d0 2024-05-30T17:03:55.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.006+0000 7f02f6000640 1 -- 172.21.15.67:0/3676230792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 msgr2=0x7f02f00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.006+0000 7f02f6000640 1 --2- 172.21.15.67:0/3676230792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 0x7f02f00b4dd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009870 tx=0x7f02e002a180 comp rx=0 tx=0).stop 2024-05-30T17:03:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.006+0000 7f02f6000640 1 -- 172.21.15.67:0/3676230792 shutdown_connections 2024-05-30T17:03:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.006+0000 7f02f6000640 1 --2- 172.21.15.67:0/3676230792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 0x7f02f00b4dd0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.006+0000 7f02f6000640 1 -- 172.21.15.67:0/3676230792 >> 172.21.15.67:0/3676230792 conn(0x7f02f006b900 msgr2=0x7f02f006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.006+0000 7f02f6000640 1 -- 172.21.15.67:0/3676230792 shutdown_connections 2024-05-30T17:03:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.006+0000 7f02f6000640 1 -- 172.21.15.67:0/3676230792 wait complete. 2024-05-30T17:03:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.008+0000 7f02f6000640 1 Processor -- start 2024-05-30T17:03:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.008+0000 7f02f6000640 1 -- start start 2024-05-30T17:03:55.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.009+0000 7f02f6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f00b3d60 0x7f02f00b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:55.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.009+0000 7f02f6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02f00ce1f0 0x7f02f0151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:55.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.009+0000 7f02f6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 0x7f02f01f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:55.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.009+0000 7f02f6000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00cd420 con 0x7f02f00ce5d0 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.009+0000 7f02f6000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00cd2a0 con 0x7f02f00b3d60 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.009+0000 7f02f6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02f00b4210 con 0x7f02f00ce1f0 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.010+0000 7f02efe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02f00ce1f0 0x7f02f0151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.010+0000 7f02f5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 0x7f02f01f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.010+0000 7f02f4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f00b3d60 0x7f02f00b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.010+0000 7f02efe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02f00ce1f0 0x7f02f0151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:57806/0 (socket says 172.21.15.67:57806) 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.010+0000 7f02efe00640 1 -- 172.21.15.67:0/1066030121 learned_addr learned my addr 172.21.15.67:0/1066030121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.010+0000 7f02f4c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f00b3d60 0x7f02f00b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:49766/0 (socket says 172.21.15.67:49766) 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.011+0000 7f02f5600640 1 -- 172.21.15.67:0/1066030121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02f00ce1f0 msgr2=0x7f02f0151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.011+0000 7f02f5600640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02f00ce1f0 0x7f02f0151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.011+0000 7f02f5600640 1 -- 172.21.15.67:0/1066030121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f00b3d60 msgr2=0x7f02f00b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.011+0000 7f02f5600640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f00b3d60 0x7f02f00b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:55.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.011+0000 7f02f5600640 1 -- 172.21.15.67:0/1066030121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f0159810 con 0x7f02f00ce5d0 2024-05-30T17:03:55.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.011+0000 7f02f5600640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 0x7f02f01f77a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f02e800c690 tx=0x7f02e800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:55.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.011+0000 7f02ecc00640 1 -- 172.21.15.67:0/1066030121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8003b70 con 0x7f02f00ce5d0 2024-05-30T17:03:55.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.011+0000 7f02ecc00640 1 -- 172.21.15.67:0/1066030121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e8003d10 con 0x7f02f00ce5d0 2024-05-30T17:03:55.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.011+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f0159af0 con 0x7f02f00ce5d0 2024-05-30T17:03:55.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.012+0000 7f02ecc00640 1 -- 172.21.15.67:0/1066030121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8005310 con 0x7f02f00ce5d0 2024-05-30T17:03:55.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.012+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f015a0b0 con 0x7f02f00ce5d0 2024-05-30T17:03:55.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.015+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f00689e0 con 0x7f02f00ce5d0 2024-05-30T17:03:55.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.017+0000 7f02ecc00640 1 -- 172.21.15.67:0/1066030121 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7f02e80054b0 con 0x7f02f00ce5d0 2024-05-30T17:03:55.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.018+0000 7f02ecc00640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f02d406e3f0 0x7f02d40708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:03:55.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.018+0000 7f02ecc00640 1 -- 172.21.15.67:0/1066030121 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f02e808a720 con 0x7f02f00ce5d0 2024-05-30T17:03:55.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.020+0000 7f02f4c00640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f02d406e3f0 0x7f02d40708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:03:55.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.021+0000 7f02f4c00640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f02d406e3f0 0x7f02d40708b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f02e4004f60 tx=0x7f02e4007e30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:03:55.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.024+0000 7f02ecc00640 1 -- 172.21.15.67:0/1066030121 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02e805a000 con 0x7f02f00ce5d0 2024-05-30T17:03:55.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.461+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone93313", "target": ["mon-mgr", ""]}) -- 0x7f02f00ca070 con 0x7f02d406e3f0 2024-05-30T17:03:55.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.653+0000 7f02ecc00640 1 -- 172.21.15.67:0/1066030121 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02f00ca070 con 0x7f02d406e3f0 2024-05-30T17:03:55.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.656+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f02d406e3f0 msgr2=0x7f02d40708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:55.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.656+0000 7f02f6000640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f02d406e3f0 0x7f02d40708b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f02e4004f60 tx=0x7f02e4007e30 comp rx=0 tx=0).stop 2024-05-30T17:03:55.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.656+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 msgr2=0x7f02f01f77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:03:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.656+0000 7f02f6000640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 0x7f02f01f77a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f02e800c690 tx=0x7f02e800cb50 comp rx=0 tx=0).stop 2024-05-30T17:03:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.657+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 shutdown_connections 2024-05-30T17:03:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.657+0000 7f02f6000640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f02d406e3f0 0x7f02d40708b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.657+0000 7f02f6000640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02f00ce5d0 0x7f02f01f77a0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.657+0000 7f02f6000640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02f00ce1f0 0x7f02f0151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.657+0000 7f02f6000640 1 --2- 172.21.15.67:0/1066030121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f00b3d60 0x7f02f00b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:03:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.657+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 >> 172.21.15.67:0/1066030121 conn(0x7f02f006b900 msgr2=0x7f02f015d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:03:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.657+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 shutdown_connections 2024-05-30T17:03:55.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:03:55.657+0000 7f02f6000640 1 -- 172.21.15.67:0/1066030121 wait complete. 2024-05-30T17:04:00.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:04:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.897+0000 7f5ede800640 1 Processor -- start 2024-05-30T17:04:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.897+0000 7f5ede800640 1 -- start start 2024-05-30T17:04:00.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5ede800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5ed80ccc30 0x7f5ed806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:00.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5ede800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80d0460 0x7f5ed80d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:00.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5ede800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed80cd1a0 con 0x7f5ed815c6b0 2024-05-30T17:04:00.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5ede800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed80cd480 con 0x7f5ed80ccc30 2024-05-30T17:04:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5ede800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed80cd760 con 0x7f5ed80d0460 2024-05-30T17:04:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5edd400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5ed815c6b0 0x7f5ed80b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33158/0 (socket says 172.21.15.67:33158) 2024-05-30T17:04:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5edd400640 1 -- 172.21.15.67:0/111494217 learned_addr learned my addr 172.21.15.67:0/111494217 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:00.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5edca00640 1 --2- 172.21.15.67:0/111494217 >> v2:172.21.15.169:3300/0 conn(0x7f5ed80ccc30 0x7f5ed806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:00.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5edde00640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80d0460 0x7f5ed80d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:00.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.899+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1331250624 0 0) 0x7f5ed80cd1a0 con 0x7f5ed815c6b0 2024-05-30T17:04:00.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ec0003520 con 0x7f5ed815c6b0 2024-05-30T17:04:00.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 193478256 0 0) 0x7f5ec0003520 con 0x7f5ed815c6b0 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80d0460 msgr2=0x7f5ed80d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80d0460 0x7f5ed80d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 >> v2:172.21.15.169:3300/0 conn(0x7f5ed80ccc30 msgr2=0x7f5ed806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 --2- 172.21.15.67:0/111494217 >> v2:172.21.15.169:3300/0 conn(0x7f5ed80ccc30 0x7f5ed806dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed80cda40 con 0x7f5ed815c6b0 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5ecc001670 con 0x7f5ed815c6b0 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5edde00640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80d0460 0x7f5ed80d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 >> v1:172.21.15.67:6789/0 conn(0x7f5ed815c6b0 legacy=0x7f5ed80b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ec0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0004800 0x7f5ec0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80d0460 0x7f5ec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed80cd480 con 0x7f5ec00039b0 2024-05-30T17:04:00.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed80cd760 con 0x7f5ec0004800 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec000b9f0 con 0x7f5ed80d0460 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.900+0000 7f5edd400640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ec0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.901+0000 7f5edd400640 1 -- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80d0460 msgr2=0x7f5ec00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.901+0000 7f5edd400640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80d0460 0x7f5ec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.901+0000 7f5edd400640 1 -- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0004800 msgr2=0x7f5ec0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.901+0000 7f5edd400640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec0004800 0x7f5ec0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.901+0000 7f5edd400640 1 -- 172.21.15.67:0/111494217 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec0003520 con 0x7f5ec00039b0 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.902+0000 7f5edd400640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ec0003dd0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c110 tx=0x7f5ec0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669b07f7fde5e72e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.902+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc007c40 con 0x7f5ec00039b0 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.902+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ecc005af0 con 0x7f5ec00039b0 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.902+0000 7f5ed7e00640 1 -- 172.21.15.67:0/111494217 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc007070 con 0x7f5ec00039b0 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.903+0000 7f5ede800640 1 -- 172.21.15.67:0/111494217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 msgr2=0x7f5ec0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.903+0000 7f5ede800640 1 --2- 172.21.15.67:0/111494217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ec0003dd0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c110 tx=0x7f5ec0005830 comp rx=0 tx=0).stop 2024-05-30T17:04:00.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.903+0000 7f5edde00640 1 -- 172.21.15.67:0/111494217 reap_dead start 2024-05-30T17:04:00.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.903+0000 7f5ede800640 1 -- 172.21.15.67:0/111494217 shutdown_connections 2024-05-30T17:04:00.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.903+0000 7f5ede800640 1 -- 172.21.15.67:0/111494217 >> 172.21.15.67:0/111494217 conn(0x7f5ed806bae0 msgr2=0x7f5ed806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:00.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.903+0000 7f5ede800640 1 -- 172.21.15.67:0/111494217 shutdown_connections 2024-05-30T17:04:00.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.903+0000 7f5ede800640 1 -- 172.21.15.67:0/111494217 wait complete. 2024-05-30T17:04:00.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.905+0000 7f5ede800640 1 Processor -- start 2024-05-30T17:04:00.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.906+0000 7f5ede800640 1 -- start start 2024-05-30T17:04:00.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.906+0000 7f5ede800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ed80c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:00.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.906+0000 7f5ede800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80c9760 0x7f5ed814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:00.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.906+0000 7f5ede800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed814c9f0 0x7f5ed81e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:00.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5ede800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed806e350 con 0x7f5ec00039b0 2024-05-30T17:04:00.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5ede800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed806e1d0 con 0x7f5ed814c9f0 2024-05-30T17:04:00.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5ede800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed80d08f0 con 0x7f5ed80c9760 2024-05-30T17:04:00.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ed80c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:00.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ed80c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55110/0 (socket says 172.21.15.67:55110) 2024-05-30T17:04:00.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edd400640 1 -- 172.21.15.67:0/4254468645 learned_addr learned my addr 172.21.15.67:0/4254468645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:00.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edca00640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80c9760 0x7f5ed814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:00.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edde00640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed814c9f0 0x7f5ed81e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:00.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edd400640 1 -- 172.21.15.67:0/4254468645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80c9760 msgr2=0x7f5ed814c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:00.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edd400640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80c9760 0x7f5ed814c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:00.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edd400640 1 -- 172.21.15.67:0/4254468645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed814c9f0 msgr2=0x7f5ed81e17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:00.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edd400640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed814c9f0 0x7f5ed81e17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:00.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.907+0000 7f5edd400640 1 -- 172.21.15.67:0/4254468645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed80c86c0 con 0x7f5ec00039b0 2024-05-30T17:04:00.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.908+0000 7f5edd400640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ed80c7dc0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c230 tx=0x7f5ed80b3b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:00.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.908+0000 7f5ed5600640 1 -- 172.21.15.67:0/4254468645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc043070 con 0x7f5ec00039b0 2024-05-30T17:04:00.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.908+0000 7f5ed5600640 1 -- 172.21.15.67:0/4254468645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ecc038040 con 0x7f5ec00039b0 2024-05-30T17:04:00.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.908+0000 7f5ed5600640 1 -- 172.21.15.67:0/4254468645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc007290 con 0x7f5ec00039b0 2024-05-30T17:04:00.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.908+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed80c89a0 con 0x7f5ec00039b0 2024-05-30T17:04:00.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.908+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed80c8f60 con 0x7f5ec00039b0 2024-05-30T17:04:00.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.910+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed80689e0 con 0x7f5ec00039b0 2024-05-30T17:04:00.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.911+0000 7f5ed5600640 1 -- 172.21.15.67:0/4254468645 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7f5ecc007430 con 0x7f5ec00039b0 2024-05-30T17:04:00.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.912+0000 7f5ed5600640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eb806e3f0 0x7f5eb80708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:00.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.912+0000 7f5ed5600640 1 -- 172.21.15.67:0/4254468645 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f5ecc0aa4c0 con 0x7f5ec00039b0 2024-05-30T17:04:00.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.912+0000 7f5edca00640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eb806e3f0 0x7f5eb80708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:00.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.913+0000 7f5edca00640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eb806e3f0 0x7f5eb80708b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80c9b60 tx=0x7f5ec4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:00.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:00.916+0000 7f5ed5600640 1 -- 172.21.15.67:0/4254468645 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ecc07a3f0 con 0x7f5ec00039b0 2024-05-30T17:04:01.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.199+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5ed8073b90 con 0x7f5ec00039b0 2024-05-30T17:04:01.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.200+0000 7f5ed5600640 1 -- 172.21.15.67:0/4254468645 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2125 (secure 0 0 0) 0x7f5ecc079cf0 con 0x7f5ec00039b0 2024-05-30T17:04:01.201 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:01.201 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:04:01.202 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:01.203 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:04:01.203 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:04:01.203 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:04:01.203 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:04:01.203 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:01.203 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1234, 2024-05-30T17:04:01.204 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:04:01.204 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:04:01.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:04:01.204 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:04:01.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 118, 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:01.205 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1750, 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 7207108447, 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 15456149504, 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 757637963776, 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 335295561, 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 331848460, 2024-05-30T17:04:01.206 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 82, 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 94 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 74, 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8, 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 8, 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 8412 2024-05-30T17:04:01.207 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:03:48.549378+0000", 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "87f1ded5-ae09-4729-991d-ae326047c9cc": { 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 26.0 seconds before exiting (11s)\n [============================] ", 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:04:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:01.210 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eb806e3f0 msgr2=0x7f5eb80708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eb806e3f0 0x7f5eb80708b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80c9b60 tx=0x7f5ec4008040 comp rx=0 tx=0).stop 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 msgr2=0x7f5ed80c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ed80c7dc0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c230 tx=0x7f5ed80b3b70 comp rx=0 tx=0).stop 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 shutdown_connections 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eb806e3f0 0x7f5eb80708b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed814c9f0 0x7f5ed81e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ed80c9760 0x7f5ed814c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 --2- 172.21.15.67:0/4254468645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ec00039b0 0x7f5ed80c7dc0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.202+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 >> 172.21.15.67:0/4254468645 conn(0x7f5ed806bae0 msgr2=0x7f5ed8072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.203+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 shutdown_connections 2024-05-30T17:04:01.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.203+0000 7f5ede800640 1 -- 172.21.15.67:0/4254468645 wait complete. 2024-05-30T17:04:01.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:04:01.222 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone93311/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd 2024-05-30T17:04:01.222 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T17:04:01.285 INFO:teuthology.orchestra.run.smithi067.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone93311/15bad52f-e9e6-4b9e-abfa-9f26f81cbcdd': No such file or directory 2024-05-30T17:04:01.286 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-30T17:04:01.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T17:04:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.494+0000 7fa92e800640 1 Processor -- start 2024-05-30T17:04:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.494+0000 7fa92e800640 1 -- start start 2024-05-30T17:04:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.495+0000 7fa92e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9280d0a20 0x7fa9280997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.495+0000 7fa92e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9280b51f0 con 0x7fa9280b4e10 2024-05-30T17:04:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.495+0000 7fa92e800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9280aef60 con 0x7fa9280d0590 2024-05-30T17:04:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.495+0000 7fa92e800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa9280ad1d0 con 0x7fa9280d0a20 2024-05-30T17:04:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.495+0000 7fa927400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa9280b4e10 0x7fa9280cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33166/0 (socket says 172.21.15.67:33166) 2024-05-30T17:04:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.495+0000 7fa927400640 1 -- 172.21.15.67:0/2943840212 learned_addr learned my addr 172.21.15.67:0/2943840212 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.496+0000 7fa927e00640 1 --2- 172.21.15.67:0/2943840212 >> v2:172.21.15.67:3300/0 conn(0x7fa9280d0a20 0x7fa9280997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.496+0000 7fa927e00640 1 -- 172.21.15.67:0/2943840212 >> v1:172.21.15.169:6789/0 conn(0x7fa9280d0590 legacy=0x7fa928099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:01.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.496+0000 7fa927e00640 1 -- 172.21.15.67:0/2943840212 >> v1:172.21.15.67:6789/0 conn(0x7fa9280b4e10 legacy=0x7fa9280cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.496+0000 7fa927e00640 1 -- 172.21.15.67:0/2943840212 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9280ad470 con 0x7fa9280d0a20 2024-05-30T17:04:01.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.497+0000 7fa927e00640 1 --2- 172.21.15.67:0/2943840212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0a20 0x7fa9280997a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa9200097a0 tx=0x7fa92002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364462ccae5537e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:01.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.497+0000 7fa926000640 1 -- 172.21.15.67:0/2943840212 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920037070 con 0x7fa9280d0a20 2024-05-30T17:04:01.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.497+0000 7fa926000640 1 -- 172.21.15.67:0/2943840212 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92002ae50 con 0x7fa9280d0a20 2024-05-30T17:04:01.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.497+0000 7fa926000640 1 -- 172.21.15.67:0/2943840212 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920032660 con 0x7fa9280d0a20 2024-05-30T17:04:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.498+0000 7fa92e800640 1 -- 172.21.15.67:0/2943840212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0a20 msgr2=0x7fa9280997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.498+0000 7fa92e800640 1 --2- 172.21.15.67:0/2943840212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0a20 0x7fa9280997a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa9200097a0 tx=0x7fa92002a140 comp rx=0 tx=0).stop 2024-05-30T17:04:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.498+0000 7fa92e800640 1 -- 172.21.15.67:0/2943840212 shutdown_connections 2024-05-30T17:04:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.498+0000 7fa92e800640 1 --2- 172.21.15.67:0/2943840212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0a20 0x7fa9280997a0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.498+0000 7fa92e800640 1 -- 172.21.15.67:0/2943840212 >> 172.21.15.67:0/2943840212 conn(0x7fa92806bae0 msgr2=0x7fa92806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.498+0000 7fa92e800640 1 -- 172.21.15.67:0/2943840212 shutdown_connections 2024-05-30T17:04:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.498+0000 7fa92e800640 1 -- 172.21.15.67:0/2943840212 wait complete. 2024-05-30T17:04:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.500+0000 7fa92e800640 1 Processor -- start 2024-05-30T17:04:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.500+0000 7fa92e800640 1 -- start start 2024-05-30T17:04:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.501+0000 7fa92e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280b4e10 0x7fa92815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.501+0000 7fa92e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0590 0x7fa92815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.501+0000 7fa92e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9280d0a20 0x7fa92815d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.501+0000 7fa92e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa9280ce540 con 0x7fa9280d0590 2024-05-30T17:04:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.501+0000 7fa92e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9280d0e30 con 0x7fa9280d0a20 2024-05-30T17:04:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.501+0000 7fa92e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9280ce880 con 0x7fa9280b4e10 2024-05-30T17:04:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.501+0000 7fa926a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0590 0x7fa92815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.501+0000 7fa926a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0590 0x7fa92815d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55128/0 (socket says 172.21.15.67:55128) 2024-05-30T17:04:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.501+0000 7fa926a00640 1 -- 172.21.15.67:0/1839391971 learned_addr learned my addr 172.21.15.67:0/1839391971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:01.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa927e00640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9280d0a20 0x7fa92815d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa926a00640 1 -- 172.21.15.67:0/1839391971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280b4e10 msgr2=0x7fa92815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa926a00640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280b4e10 0x7fa92815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa926a00640 1 -- 172.21.15.67:0/1839391971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9280d0a20 msgr2=0x7fa92815d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa926a00640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9280d0a20 0x7fa92815d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa926a00640 1 -- 172.21.15.67:0/1839391971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9281f4e30 con 0x7fa9280d0590 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa927e00640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9280d0a20 0x7fa92815d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa926a00640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0590 0x7fa92815d110 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fa91400c7c0 tx=0x7fa91400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa91f400640 1 -- 172.21.15.67:0/1839391971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914016020 con 0x7fa9280d0590 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa91f400640 1 -- 172.21.15.67:0/1839391971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91400aa50 con 0x7fa9280d0590 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa91f400640 1 -- 172.21.15.67:0/1839391971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91400ae40 con 0x7fa9280d0590 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.502+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9280c9af0 con 0x7fa9280d0590 2024-05-30T17:04:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.503+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9280b0160 con 0x7fa9280d0590 2024-05-30T17:04:01.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.504+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9280689e0 con 0x7fa9280d0590 2024-05-30T17:04:01.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.505+0000 7fa91f400640 1 -- 172.21.15.67:0/1839391971 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7fa914019020 con 0x7fa9280d0590 2024-05-30T17:04:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.506+0000 7fa91f400640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa90806e3f0 0x7fa9080708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.506+0000 7fa91f400640 1 -- 172.21.15.67:0/1839391971 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7fa91408ab30 con 0x7fa9280d0590 2024-05-30T17:04:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.506+0000 7fa927400640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa90806e3f0 0x7fa9080708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.507+0000 7fa927400640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa90806e3f0 0x7fa9080708b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa918000f60 tx=0x7fa918007a80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:01.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.510+0000 7fa91f400640 1 -- 172.21.15.67:0/1839391971 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa914020030 con 0x7fa9280d0590 2024-05-30T17:04:01.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.742+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa92815b660 con 0x7fa90806e3f0 2024-05-30T17:04:01.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.745+0000 7fa91f400640 1 -- 172.21.15.67:0/1839391971 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa92815b660 con 0x7fa90806e3f0 2024-05-30T17:04:01.746 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:04:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T17:04:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:01.747 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:04:01.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.747+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa90806e3f0 msgr2=0x7fa9080708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.747+0000 7fa92e800640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa90806e3f0 0x7fa9080708b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa918000f60 tx=0x7fa918007a80 comp rx=0 tx=0).stop 2024-05-30T17:04:01.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.747+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0590 msgr2=0x7fa92815d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.747+0000 7fa92e800640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0590 0x7fa92815d110 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fa91400c7c0 tx=0x7fa91400cc80 comp rx=0 tx=0).stop 2024-05-30T17:04:01.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.748+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 shutdown_connections 2024-05-30T17:04:01.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.748+0000 7fa92e800640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa90806e3f0 0x7fa9080708b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.748+0000 7fa92e800640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9280d0a20 0x7fa92815d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.748+0000 7fa92e800640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9280d0590 0x7fa92815d110 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.748+0000 7fa92e800640 1 --2- 172.21.15.67:0/1839391971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9280b4e10 0x7fa92815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.748+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 >> 172.21.15.67:0/1839391971 conn(0x7fa92806bae0 msgr2=0x7fa92806e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:01.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.748+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 shutdown_connections 2024-05-30T17:04:01.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.748+0000 7fa92e800640 1 -- 172.21.15.67:0/1839391971 wait complete. 2024-05-30T17:04:01.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:04:01.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.977+0000 7f514fe00640 1 Processor -- start 2024-05-30T17:04:01.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.978+0000 7f514fe00640 1 -- start start 2024-05-30T17:04:01.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.979+0000 7f514fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015e690 0x7f51500d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.979+0000 7f514fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f51500d1ca0 0x7f51500d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.979+0000 7f514fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51500d3060 con 0x7f515015ea70 2024-05-30T17:04:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.979+0000 7f514fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f51500d3340 con 0x7f51500d1ca0 2024-05-30T17:04:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.979+0000 7f514fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f51500d3620 con 0x7f515015e690 2024-05-30T17:04:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.979+0000 7f514e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f51500d1ca0 0x7f51500d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.979+0000 7f514e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f51500d1ca0 0x7f51500d4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55142/0 (socket says 172.21.15.67:55142) 2024-05-30T17:04:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.979+0000 7f514e000640 1 -- 172.21.15.67:0/197777865 learned_addr learned my addr 172.21.15.67:0/197777865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.980+0000 7f514ea00640 1 --2- 172.21.15.67:0/197777865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015e690 0x7f51500d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.980+0000 7f514e000640 1 -- 172.21.15.67:0/197777865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015e690 msgr2=0x7f51500d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.980+0000 7f514e000640 1 --2- 172.21.15.67:0/197777865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015e690 0x7f51500d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.980+0000 7f514e000640 1 -- 172.21.15.67:0/197777865 >> v1:172.21.15.169:6789/0 conn(0x7f515015ea70 legacy=0x7f51500d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.980+0000 7f514e000640 1 -- 172.21.15.67:0/197777865 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51500cf770 con 0x7f51500d1ca0 2024-05-30T17:04:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.981+0000 7f514e000640 1 --2- 172.21.15.67:0/197777865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 0x7f51500d4a70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f513c009850 tx=0x7f513c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf051c9d7557582 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:01.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.982+0000 7f514d600640 1 -- 172.21.15.67:0/197777865 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c004030 con 0x7f51500d1ca0 2024-05-30T17:04:01.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.982+0000 7f514d600640 1 -- 172.21.15.67:0/197777865 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513c0041d0 con 0x7f51500d1ca0 2024-05-30T17:04:01.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.982+0000 7f514d600640 1 -- 172.21.15.67:0/197777865 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c003610 con 0x7f51500d1ca0 2024-05-30T17:04:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.982+0000 7f514fe00640 1 -- 172.21.15.67:0/197777865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 msgr2=0x7f51500d4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.982+0000 7f514fe00640 1 --2- 172.21.15.67:0/197777865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 0x7f51500d4a70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f513c009850 tx=0x7f513c02a1e0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.982+0000 7f514fe00640 1 -- 172.21.15.67:0/197777865 shutdown_connections 2024-05-30T17:04:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.982+0000 7f514fe00640 1 --2- 172.21.15.67:0/197777865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 0x7f51500d4a70 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.982+0000 7f514fe00640 1 --2- 172.21.15.67:0/197777865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015e690 0x7f51500d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.982+0000 7f514fe00640 1 -- 172.21.15.67:0/197777865 >> 172.21.15.67:0/197777865 conn(0x7f515009d910 msgr2=0x7f515008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.983+0000 7f514fe00640 1 -- 172.21.15.67:0/197777865 shutdown_connections 2024-05-30T17:04:01.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.983+0000 7f514fe00640 1 -- 172.21.15.67:0/197777865 wait complete. 2024-05-30T17:04:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.984+0000 7f514fe00640 1 Processor -- start 2024-05-30T17:04:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.984+0000 7f514fe00640 1 -- start start 2024-05-30T17:04:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.985+0000 7f514fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 0x7f515015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.985+0000 7f514fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515015e690 0x7f5150161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.985+0000 7f514fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015ea70 0x7f5150157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.985+0000 7f514fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f51500d2bc0 con 0x7f51500d1ca0 2024-05-30T17:04:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.985+0000 7f514fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f51500d2a40 con 0x7f515015e690 2024-05-30T17:04:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.985+0000 7f514fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f51500d2d40 con 0x7f515015ea70 2024-05-30T17:04:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 0x7f515015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 0x7f515015ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55158/0 (socket says 172.21.15.67:55158) 2024-05-30T17:04:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514ea00640 1 -- 172.21.15.67:0/1922341777 learned_addr learned my addr 172.21.15.67:0/1922341777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514e000640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515015e690 0x7f5150161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514f400640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015ea70 0x7f5150157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514ea00640 1 -- 172.21.15.67:0/1922341777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015ea70 msgr2=0x7f5150157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514ea00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015ea70 0x7f5150157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514ea00640 1 -- 172.21.15.67:0/1922341777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515015e690 msgr2=0x7f5150161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514ea00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515015e690 0x7f5150161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514ea00640 1 -- 172.21.15.67:0/1922341777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51500b9160 con 0x7f51500d1ca0 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.986+0000 7f514ea00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 0x7f515015ef10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f514000d5f0 tx=0x7f514000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.987+0000 7f5146a00640 1 -- 172.21.15.67:0/1922341777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51400125f0 con 0x7f51500d1ca0 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.987+0000 7f5146a00640 1 -- 172.21.15.67:0/1922341777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5140019070 con 0x7f51500d1ca0 2024-05-30T17:04:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.987+0000 7f5146a00640 1 -- 172.21.15.67:0/1922341777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51400129d0 con 0x7f51500d1ca0 2024-05-30T17:04:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.987+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51500b9440 con 0x7f51500d1ca0 2024-05-30T17:04:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.987+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51500cbaf0 con 0x7f51500d1ca0 2024-05-30T17:04:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.988+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51500cb610 con 0x7f51500d1ca0 2024-05-30T17:04:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.989+0000 7f5146a00640 1 -- 172.21.15.67:0/1922341777 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7f5140012b70 con 0x7f51500d1ca0 2024-05-30T17:04:01.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.990+0000 7f5146a00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f513006e3f0 0x7f51300708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:01.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.990+0000 7f5146a00640 1 -- 172.21.15.67:0/1922341777 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f514008ade0 con 0x7f51500d1ca0 2024-05-30T17:04:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.990+0000 7f514e000640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f513006e3f0 0x7f51300708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.991+0000 7f514e000640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f513006e3f0 0x7f51300708b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f513c033000 tx=0x7f513c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:01.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:01.995+0000 7f5146a00640 1 -- 172.21.15.67:0/1922341777 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f514005ad10 con 0x7f51500d1ca0 2024-05-30T17:04:02.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.281+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f515006cbc0 con 0x7f51500d1ca0 2024-05-30T17:04:02.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.281+0000 7f5146a00640 1 -- 172.21.15.67:0/1922341777 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2125 (secure 0 0 0) 0x7f514005a610 con 0x7f51500d1ca0 2024-05-30T17:04:02.283 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:02.283 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:04:02.283 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:04:02.283 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:04:02.283 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:02.284 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1235, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 118, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:02.285 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1750, 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 7207108447, 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 15456149504, 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 757637963776, 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:04:02.286 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 335295561, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 331848460, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 82, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 94 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 74, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 8, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:04:02.287 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 8412 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:02.288 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:03:48.549378+0000", 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: "87f1ded5-ae09-4729-991d-ae326047c9cc": { 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 26.0 seconds before exiting (11s)\n [============================] ", 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:02.289 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:04:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.284+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f513006e3f0 msgr2=0x7f51300708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.284+0000 7f514fe00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f513006e3f0 0x7f51300708b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f513c033000 tx=0x7f513c02e000 comp rx=0 tx=0).stop 2024-05-30T17:04:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 msgr2=0x7f515015ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 0x7f515015ef10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f514000d5f0 tx=0x7f514000dab0 comp rx=0 tx=0).stop 2024-05-30T17:04:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 shutdown_connections 2024-05-30T17:04:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f513006e3f0 0x7f51300708b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515015ea70 0x7f5150157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515015e690 0x7f5150161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 --2- 172.21.15.67:0/1922341777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51500d1ca0 0x7f515015ef10 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 >> 172.21.15.67:0/1922341777 conn(0x7f515009d910 msgr2=0x7f51500b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 shutdown_connections 2024-05-30T17:04:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:02.285+0000 7f514fe00640 1 -- 172.21.15.67:0/1922341777 wait complete. 2024-05-30T17:04:07.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:04:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.524+0000 7fd09b200640 1 Processor -- start 2024-05-30T17:04:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.525+0000 7fd09b200640 1 -- start start 2024-05-30T17:04:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.526+0000 7fd09b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15e690 0x7fd09c0d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.526+0000 7fd09b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd09c15ea70 0x7fd09c0d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.526+0000 7fd09b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd09c0d3060 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.526+0000 7fd09b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd09c0d3340 con 0x7fd09c15ea70 2024-05-30T17:04:07.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.526+0000 7fd09b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd09c0d3620 con 0x7fd09c15e690 2024-05-30T17:04:07.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.527+0000 7fd099e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15e690 0x7fd09c0d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:07.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.527+0000 7fd099e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15e690 0x7fd09c0d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:42400/0 (socket says 172.21.15.67:42400) 2024-05-30T17:04:07.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.527+0000 7fd099e00640 1 -- 172.21.15.67:0/2395737731 learned_addr learned my addr 172.21.15.67:0/2395737731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:07.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.527+0000 7fd09a800640 1 --2- 172.21.15.67:0/2395737731 >> v2:172.21.15.169:3300/0 conn(0x7fd09c15ea70 0x7fd09c0d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:07.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.528+0000 7fd099e00640 1 -- 172.21.15.67:0/2395737731 >> v2:172.21.15.169:3300/0 conn(0x7fd09c15ea70 msgr2=0x7fd09c0d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:07.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.528+0000 7fd099e00640 1 --2- 172.21.15.67:0/2395737731 >> v2:172.21.15.169:3300/0 conn(0x7fd09c15ea70 0x7fd09c0d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:07.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.528+0000 7fd099e00640 1 -- 172.21.15.67:0/2395737731 >> v1:172.21.15.169:6789/0 conn(0x7fd09c0d1ca0 legacy=0x7fd09c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:07.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.528+0000 7fd099e00640 1 -- 172.21.15.67:0/2395737731 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c0cf770 con 0x7fd09c15e690 2024-05-30T17:04:07.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.530+0000 7fd099e00640 1 --2- 172.21.15.67:0/2395737731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15e690 0x7fd09c0d4530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009e50 tx=0x7fd08c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c29a782f4c4e098 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:07.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.530+0000 7fd098a00640 1 -- 172.21.15.67:0/2395737731 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c036070 con 0x7fd09c15e690 2024-05-30T17:04:07.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.531+0000 7fd098a00640 1 -- 172.21.15.67:0/2395737731 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08c031600 con 0x7fd09c15e690 2024-05-30T17:04:07.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.531+0000 7fd098a00640 1 -- 172.21.15.67:0/2395737731 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0319d0 con 0x7fd09c15e690 2024-05-30T17:04:07.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.531+0000 7fd09b200640 1 -- 172.21.15.67:0/2395737731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15e690 msgr2=0x7fd09c0d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:07.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.531+0000 7fd09b200640 1 --2- 172.21.15.67:0/2395737731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15e690 0x7fd09c0d4530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009e50 tx=0x7fd08c02a710 comp rx=0 tx=0).stop 2024-05-30T17:04:07.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.531+0000 7fd09b200640 1 -- 172.21.15.67:0/2395737731 shutdown_connections 2024-05-30T17:04:07.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.531+0000 7fd09b200640 1 --2- 172.21.15.67:0/2395737731 >> v2:172.21.15.169:3300/0 conn(0x7fd09c15ea70 0x7fd09c0d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:07.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.531+0000 7fd09b200640 1 --2- 172.21.15.67:0/2395737731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15e690 0x7fd09c0d4530 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:07.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.531+0000 7fd09b200640 1 -- 172.21.15.67:0/2395737731 >> 172.21.15.67:0/2395737731 conn(0x7fd09c09d910 msgr2=0x7fd09c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:07.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.531+0000 7fd09b200640 1 -- 172.21.15.67:0/2395737731 shutdown_connections 2024-05-30T17:04:07.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.532+0000 7fd09b200640 1 -- 172.21.15.67:0/2395737731 wait complete. 2024-05-30T17:04:07.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.533+0000 7fd09b200640 1 Processor -- start 2024-05-30T17:04:07.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.533+0000 7fd09b200640 1 -- start start 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.534+0000 7fd09b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd09c0d1ca0 0x7fd09c151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.534+0000 7fd09b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd09c15e690 0x7fd09c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.534+0000 7fd09b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15ea70 0x7fd09c0b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.534+0000 7fd09b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c0d4bf0 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.534+0000 7fd09b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c0d4a70 con 0x7fd09c15e690 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.534+0000 7fd09b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd09c0d4d70 con 0x7fd09c15ea70 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd099e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd09c0d1ca0 0x7fd09c151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd099e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd09c0d1ca0 0x7fd09c151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55172/0 (socket says 172.21.15.67:55172) 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd099e00640 1 -- 172.21.15.67:0/3946678924 learned_addr learned my addr 172.21.15.67:0/3946678924 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd099e00640 1 -- 172.21.15.67:0/3946678924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15ea70 msgr2=0x7fd09c0b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd09a800640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15ea70 0x7fd09c0b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:07.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd099400640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd09c15e690 0x7fd09c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:07.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd099e00640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15ea70 0x7fd09c0b9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:07.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd099e00640 1 -- 172.21.15.67:0/3946678924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd09c15e690 msgr2=0x7fd09c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:07.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd099e00640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd09c15e690 0x7fd09c0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:07.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.535+0000 7fd099e00640 1 -- 172.21.15.67:0/3946678924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c0cca40 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.536+0000 7fd099e00640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd09c0d1ca0 0x7fd09c151870 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd08c034040 tx=0x7fd08c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:07.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.536+0000 7fd092000640 1 -- 172.21.15.67:0/3946678924 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c036050 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.536+0000 7fd092000640 1 -- 172.21.15.67:0/3946678924 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08c042070 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.536+0000 7fd092000640 1 -- 172.21.15.67:0/3946678924 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c009510 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.536+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09c0ccd20 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.536+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd09c0cd2e0 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.537+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd09c0cc7d0 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.538+0000 7fd092000640 1 -- 172.21.15.67:0/3946678924 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7fd08c007c50 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.539+0000 7fd092000640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd07806e420 0x7fd0780708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:07.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.540+0000 7fd092000640 1 -- 172.21.15.67:0/3946678924 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7fd08c0ab860 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.540+0000 7fd099400640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd07806e420 0x7fd0780708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:07.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.541+0000 7fd099400640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd07806e420 0x7fd0780708e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd09c15f6e0 tx=0x7fd084009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:07.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.545+0000 7fd092000640 1 -- 172.21.15.67:0/3946678924 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd08c07b6f0 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.823+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd09c0689e0 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.824+0000 7fd092000640 1 -- 172.21.15.67:0/3946678924 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2125 (secure 0 0 0) 0x7fd08c07aff0 con 0x7fd09c0d1ca0 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:04:07.825 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1241, 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:04:07.826 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 118, 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:07.827 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1749, 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 7207144428, 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 15494037504, 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 757600075776, 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 211708647, 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 214538597, 2024-05-30T17:04:07.828 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 52, 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 62 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 74, 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8, 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 8, 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:04:07.829 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 8412 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:04:07.830 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:03:48.549378+0000", 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: "87f1ded5-ae09-4729-991d-ae326047c9cc": { 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 20.0 seconds before exiting (17s)\n [============================] ", 2024-05-30T17:04:07.831 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:04:07.832 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:04:07.832 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:07.832 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:07.832 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:04:07.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd07806e420 msgr2=0x7fd0780708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:07.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd07806e420 0x7fd0780708e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd09c15f6e0 tx=0x7fd084009040 comp rx=0 tx=0).stop 2024-05-30T17:04:07.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd09c0d1ca0 msgr2=0x7fd09c151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:07.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd09c0d1ca0 0x7fd09c151870 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd08c034040 tx=0x7fd08c004a40 comp rx=0 tx=0).stop 2024-05-30T17:04:07.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 shutdown_connections 2024-05-30T17:04:07.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd07806e420 0x7fd0780708e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:07.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd09c15ea70 0x7fd09c0b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:07.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd09c15e690 0x7fd09c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:07.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 --2- 172.21.15.67:0/3946678924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd09c0d1ca0 0x7fd09c151870 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:07.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 >> 172.21.15.67:0/3946678924 conn(0x7fd09c09d910 msgr2=0x7fd09c0d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:07.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 shutdown_connections 2024-05-30T17:04:07.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:07.827+0000 7fd09b200640 1 -- 172.21.15.67:0/3946678924 wait complete. 2024-05-30T17:04:12.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:04:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.069+0000 7fcc2f000640 1 Processor -- start 2024-05-30T17:04:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.069+0000 7fcc2f000640 1 -- start start 2024-05-30T17:04:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.070+0000 7fcc2f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc2815c6b0 0x7fcc280b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.070+0000 7fcc2f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc280cd1a0 con 0x7fcc280d0460 2024-05-30T17:04:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.070+0000 7fcc2f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc280cd480 con 0x7fcc280ccc30 2024-05-30T17:04:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.070+0000 7fcc2f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fcc2806e3a0 con 0x7fcc2815c6b0 2024-05-30T17:04:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.071+0000 7fcc2dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc2815c6b0 0x7fcc280b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.071+0000 7fcc2dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc2815c6b0 0x7fcc280b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53582/0 (socket says 172.21.15.67:53582) 2024-05-30T17:04:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.071+0000 7fcc2dc00640 1 -- 172.21.15.67:0/2478430599 learned_addr learned my addr 172.21.15.67:0/2478430599 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.071+0000 7fcc2dc00640 1 -- 172.21.15.67:0/2478430599 >> v1:172.21.15.169:6789/0 conn(0x7fcc280ccc30 legacy=0x7fcc2806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.071+0000 7fcc2dc00640 1 -- 172.21.15.67:0/2478430599 >> v1:172.21.15.67:6789/0 conn(0x7fcc280d0460 legacy=0x7fcc280cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.071+0000 7fcc2dc00640 1 -- 172.21.15.67:0/2478430599 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc280cd760 con 0x7fcc2815c6b0 2024-05-30T17:04:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.072+0000 7fcc2dc00640 1 --2- 172.21.15.67:0/2478430599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc2815c6b0 0x7fcc280b3db0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098a0 tx=0x7fcc1c02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5220658c51559af4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.073+0000 7fcc27e00640 1 -- 172.21.15.67:0/2478430599 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c037070 con 0x7fcc2815c6b0 2024-05-30T17:04:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.073+0000 7fcc27e00640 1 -- 172.21.15.67:0/2478430599 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc1c032440 con 0x7fcc2815c6b0 2024-05-30T17:04:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.073+0000 7fcc27e00640 1 -- 172.21.15.67:0/2478430599 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c032810 con 0x7fcc2815c6b0 2024-05-30T17:04:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.073+0000 7fcc2f000640 1 -- 172.21.15.67:0/2478430599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc2815c6b0 msgr2=0x7fcc280b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.073+0000 7fcc2f000640 1 --2- 172.21.15.67:0/2478430599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc2815c6b0 0x7fcc280b3db0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098a0 tx=0x7fcc1c02a2e0 comp rx=0 tx=0).stop 2024-05-30T17:04:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.073+0000 7fcc2f000640 1 -- 172.21.15.67:0/2478430599 shutdown_connections 2024-05-30T17:04:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.073+0000 7fcc2f000640 1 --2- 172.21.15.67:0/2478430599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc2815c6b0 0x7fcc280b3db0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.073+0000 7fcc2f000640 1 -- 172.21.15.67:0/2478430599 >> 172.21.15.67:0/2478430599 conn(0x7fcc2806bae0 msgr2=0x7fcc2806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:13.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.074+0000 7fcc2f000640 1 -- 172.21.15.67:0/2478430599 shutdown_connections 2024-05-30T17:04:13.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.074+0000 7fcc2f000640 1 -- 172.21.15.67:0/2478430599 wait complete. 2024-05-30T17:04:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.075+0000 7fcc2f000640 1 Processor -- start 2024-05-30T17:04:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.075+0000 7fcc2f000640 1 -- start start 2024-05-30T17:04:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.076+0000 7fcc2f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280ccc30 0x7fcc281f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.076+0000 7fcc2f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280d0460 0x7fcc28155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.076+0000 7fcc2f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815c6b0 0x7fcc280c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.076+0000 7fcc2f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcc280b42f0 con 0x7fcc280d0460 2024-05-30T17:04:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.076+0000 7fcc2f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcc280cba10 con 0x7fcc280ccc30 2024-05-30T17:04:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.076+0000 7fcc2f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc280b4470 con 0x7fcc2815c6b0 2024-05-30T17:04:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280d0460 0x7fcc28155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280d0460 0x7fcc28155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53584/0 (socket says 172.21.15.67:53584) 2024-05-30T17:04:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2d200640 1 -- 172.21.15.67:0/61435707 learned_addr learned my addr 172.21.15.67:0/61435707 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2e600640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815c6b0 0x7fcc280c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2dc00640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280ccc30 0x7fcc281f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2d200640 1 -- 172.21.15.67:0/61435707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815c6b0 msgr2=0x7fcc280c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2d200640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815c6b0 0x7fcc280c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2d200640 1 -- 172.21.15.67:0/61435707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280ccc30 msgr2=0x7fcc281f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2d200640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280ccc30 0x7fcc281f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2d200640 1 -- 172.21.15.67:0/61435707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc280c8180 con 0x7fcc280d0460 2024-05-30T17:04:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.077+0000 7fcc2dc00640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280ccc30 0x7fcc281f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.078+0000 7fcc2d200640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280d0460 0x7fcc28155750 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c430 tx=0x7fcc1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.078+0000 7fcc25600640 1 -- 172.21.15.67:0/61435707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14011790 con 0x7fcc280d0460 2024-05-30T17:04:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.078+0000 7fcc25600640 1 -- 172.21.15.67:0/61435707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc14017070 con 0x7fcc280d0460 2024-05-30T17:04:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.078+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc280c8460 con 0x7fcc280d0460 2024-05-30T17:04:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.078+0000 7fcc25600640 1 -- 172.21.15.67:0/61435707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14011bb0 con 0x7fcc280d0460 2024-05-30T17:04:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.078+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc280c8a20 con 0x7fcc280d0460 2024-05-30T17:04:13.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.079+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc280689e0 con 0x7fcc280d0460 2024-05-30T17:04:13.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.080+0000 7fcc25600640 1 -- 172.21.15.67:0/61435707 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7fcc1401c460 con 0x7fcc280d0460 2024-05-30T17:04:13.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.081+0000 7fcc25600640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc0806e3f0 0x7fcc080708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:13.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.081+0000 7fcc25600640 1 -- 172.21.15.67:0/61435707 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7fcc1408a920 con 0x7fcc280d0460 2024-05-30T17:04:13.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.082+0000 7fcc2dc00640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc0806e3f0 0x7fcc080708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:13.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.082+0000 7fcc2dc00640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc0806e3f0 0x7fcc080708b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c034000 tx=0x7fcc1c02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:13.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.086+0000 7fcc25600640 1 -- 172.21.15.67:0/61435707 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc1405a7b0 con 0x7fcc280d0460 2024-05-30T17:04:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.368+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcc2815be60 con 0x7fcc280d0460 2024-05-30T17:04:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.368+0000 7fcc25600640 1 -- 172.21.15.67:0/61435707 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2112 (secure 0 0 0) 0x7fcc1405a0b0 con 0x7fcc280d0460 2024-05-30T17:04:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:13.369 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:04:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1246, 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:04:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 118, 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:04:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1749, 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 7207149636, 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 15494045696, 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 757600067584, 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 341, 2024-05-30T17:04:13.373 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 4009, 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 1 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 74, 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8, 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:13.374 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 8, 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 8412 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:04:13.375 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:03:48.549378+0000", 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "87f1ded5-ae09-4729-991d-ae326047c9cc": { 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 14.0 seconds before exiting (23s)\n [============================] ", 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:04:13.376 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:13.377 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:13.377 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc0806e3f0 msgr2=0x7fcc080708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc0806e3f0 0x7fcc080708b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c034000 tx=0x7fcc1c02e000 comp rx=0 tx=0).stop 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280d0460 msgr2=0x7fcc28155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280d0460 0x7fcc28155750 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c430 tx=0x7fcc1400c8f0 comp rx=0 tx=0).stop 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 shutdown_connections 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcc0806e3f0 0x7fcc080708b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc2815c6b0 0x7fcc280c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280d0460 0x7fcc28155750 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 --2- 172.21.15.67:0/61435707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc280ccc30 0x7fcc281f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 >> 172.21.15.67:0/61435707 conn(0x7fcc2806bae0 msgr2=0x7fcc280b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.371+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 shutdown_connections 2024-05-30T17:04:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:13.372+0000 7fcc2f000640 1 -- 172.21.15.67:0/61435707 wait complete. 2024-05-30T17:04:16.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:04:16.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:04:18.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:04:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.610+0000 7fa007000640 1 Processor -- start 2024-05-30T17:04:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.610+0000 7fa007000640 1 -- start start 2024-05-30T17:04:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.611+0000 7fa007000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa00015e690 0x7fa0000d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.611+0000 7fa007000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa00015ea70 0x7fa0000d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.611+0000 7fa007000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0000d3060 con 0x7fa0000d1ca0 2024-05-30T17:04:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.611+0000 7fa007000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa0000d3340 con 0x7fa00015ea70 2024-05-30T17:04:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.611+0000 7fa007000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa0000d3620 con 0x7fa00015e690 2024-05-30T17:04:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.612+0000 7fa005200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa0000d1ca0 0x7fa0000d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58630/0 (socket says 172.21.15.67:58630) 2024-05-30T17:04:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.612+0000 7fa005200640 1 -- 172.21.15.67:0/1364271250 learned_addr learned my addr 172.21.15.67:0/1364271250 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.612+0000 7fa006600640 1 --2- 172.21.15.67:0/1364271250 >> v2:172.21.15.67:3300/0 conn(0x7fa00015ea70 0x7fa0000d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.612+0000 7fa005c00640 1 --2- 172.21.15.67:0/1364271250 >> v2:172.21.15.169:3300/0 conn(0x7fa00015e690 0x7fa0000d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.612+0000 7f9fffe00640 1 -- 172.21.15.67:0/1364271250 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1923512453 0 0) 0x7fa0000d3060 con 0x7fa0000d1ca0 2024-05-30T17:04:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.612+0000 7f9fffe00640 1 -- 172.21.15.67:0/1364271250 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fe8003520 con 0x7fa0000d1ca0 2024-05-30T17:04:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.613+0000 7fa006600640 1 -- 172.21.15.67:0/1364271250 >> v2:172.21.15.169:3300/0 conn(0x7fa00015e690 msgr2=0x7fa0000d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.613+0000 7fa006600640 1 --2- 172.21.15.67:0/1364271250 >> v2:172.21.15.169:3300/0 conn(0x7fa00015e690 0x7fa0000d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.613+0000 7fa006600640 1 -- 172.21.15.67:0/1364271250 >> v1:172.21.15.67:6789/0 conn(0x7fa0000d1ca0 legacy=0x7fa0000d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.613+0000 7fa006600640 1 -- 172.21.15.67:0/1364271250 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0000cf770 con 0x7fa00015ea70 2024-05-30T17:04:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.614+0000 7fa006600640 1 --2- 172.21.15.67:0/1364271250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 0x7fa0000d2a40 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009d90 tx=0x7f9ff802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84335c84ff332098 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.614+0000 7f9fffe00640 1 -- 172.21.15.67:0/1364271250 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8007740 con 0x7fa00015ea70 2024-05-30T17:04:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.614+0000 7f9fffe00640 1 -- 172.21.15.67:0/1364271250 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff80078e0 con 0x7fa00015ea70 2024-05-30T17:04:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.614+0000 7f9fffe00640 1 -- 172.21.15.67:0/1364271250 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8007cb0 con 0x7fa00015ea70 2024-05-30T17:04:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.615+0000 7fa007000640 1 -- 172.21.15.67:0/1364271250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 msgr2=0x7fa0000d2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.615+0000 7fa007000640 1 --2- 172.21.15.67:0/1364271250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 0x7fa0000d2a40 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009d90 tx=0x7f9ff802a6a0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.615+0000 7fa007000640 1 -- 172.21.15.67:0/1364271250 shutdown_connections 2024-05-30T17:04:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.615+0000 7fa007000640 1 --2- 172.21.15.67:0/1364271250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 0x7fa0000d2a40 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.615+0000 7fa007000640 1 --2- 172.21.15.67:0/1364271250 >> v2:172.21.15.169:3300/0 conn(0x7fa00015e690 0x7fa0000d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.615+0000 7fa007000640 1 -- 172.21.15.67:0/1364271250 >> 172.21.15.67:0/1364271250 conn(0x7fa00009d910 msgr2=0x7fa00008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.615+0000 7fa007000640 1 -- 172.21.15.67:0/1364271250 shutdown_connections 2024-05-30T17:04:18.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.615+0000 7fa007000640 1 -- 172.21.15.67:0/1364271250 wait complete. 2024-05-30T17:04:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.618+0000 7fa007000640 1 Processor -- start 2024-05-30T17:04:18.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.618+0000 7fa007000640 1 -- start start 2024-05-30T17:04:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.619+0000 7fa007000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0000d1ca0 0x7fa0000b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.619+0000 7fa007000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa00015e690 0x7fa000158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.619+0000 7fa007000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 0x7fa0001600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.619+0000 7fa007000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa0000d4bf0 con 0x7fa00015ea70 2024-05-30T17:04:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.619+0000 7fa007000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa0000d4a70 con 0x7fa00015e690 2024-05-30T17:04:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.619+0000 7fa007000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa0000d4d70 con 0x7fa0000d1ca0 2024-05-30T17:04:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa006600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 0x7fa0001600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa006600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 0x7fa0001600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53604/0 (socket says 172.21.15.67:53604) 2024-05-30T17:04:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa006600640 1 -- 172.21.15.67:0/227659430 learned_addr learned my addr 172.21.15.67:0/227659430 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:18.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa005200640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa00015e690 0x7fa000158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa005c00640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0000d1ca0 0x7fa0000b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa006600640 1 -- 172.21.15.67:0/227659430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0000d1ca0 msgr2=0x7fa0000b9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa006600640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0000d1ca0 0x7fa0000b9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa006600640 1 -- 172.21.15.67:0/227659430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa00015e690 msgr2=0x7fa000158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa006600640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa00015e690 0x7fa000158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa006600640 1 -- 172.21.15.67:0/227659430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa000159250 con 0x7fa00015ea70 2024-05-30T17:04:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.620+0000 7fa005200640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa00015e690 0x7fa000158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:04:18.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.621+0000 7fa006600640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 0x7fa0001600f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9ff802ac90 tx=0x7f9ff8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.621+0000 7f9ffd600640 1 -- 172.21.15.67:0/227659430 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8040070 con 0x7fa00015ea70 2024-05-30T17:04:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.621+0000 7f9ffd600640 1 -- 172.21.15.67:0/227659430 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff8031bc0 con 0x7fa00015ea70 2024-05-30T17:04:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.621+0000 7f9ffd600640 1 -- 172.21.15.67:0/227659430 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8009450 con 0x7fa00015ea70 2024-05-30T17:04:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.621+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0000b8b40 con 0x7fa00015ea70 2024-05-30T17:04:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.621+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0000b9100 con 0x7fa00015ea70 2024-05-30T17:04:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.622+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0000689e0 con 0x7fa00015ea70 2024-05-30T17:04:18.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.623+0000 7f9ffd600640 1 -- 172.21.15.67:0/227659430 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7f9ff8039040 con 0x7fa00015ea70 2024-05-30T17:04:18.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.625+0000 7f9ffd600640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fe006e3f0 0x7f9fe00708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:18.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.625+0000 7f9ffd600640 1 -- 172.21.15.67:0/227659430 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f9ff80ac360 con 0x7fa00015ea70 2024-05-30T17:04:18.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.625+0000 7fa005c00640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fe006e3f0 0x7f9fe00708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:18.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.625+0000 7fa005c00640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fe006e3f0 0x7f9fe00708b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4009a30 tx=0x7f9ff4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:18.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.629+0000 7f9ffd600640 1 -- 172.21.15.67:0/227659430 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa000159250 con 0x7fa00015ea70 2024-05-30T17:04:18.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.910+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa0000b5ed0 con 0x7fa00015ea70 2024-05-30T17:04:18.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.911+0000 7f9ffd600640 1 -- 172.21.15.67:0/227659430 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2079 (secure 0 0 0) 0x7f9ff807c310 con 0x7fa00015ea70 2024-05-30T17:04:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:18.912 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:04:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:04:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:04:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:04:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:04:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:04:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1252, 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:04:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 118, 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1749, 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 7207149636, 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 15494045696, 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 757600067584, 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 511, 2024-05-30T17:04:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 74, 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8, 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 8, 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 8412 2024-05-30T17:04:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:03:48.549378+0000", 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "87f1ded5-ae09-4729-991d-ae326047c9cc": { 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 8.0 seconds before exiting (29s)\n [============================] ", 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:18.918 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:04:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fe006e3f0 msgr2=0x7f9fe00708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fe006e3f0 0x7f9fe00708b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4009a30 tx=0x7f9ff4008040 comp rx=0 tx=0).stop 2024-05-30T17:04:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 msgr2=0x7fa0001600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 0x7fa0001600f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9ff802ac90 tx=0x7f9ff8004a40 comp rx=0 tx=0).stop 2024-05-30T17:04:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 shutdown_connections 2024-05-30T17:04:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fe006e3f0 0x7f9fe00708b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa00015ea70 0x7fa0001600f0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa00015e690 0x7fa000158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 --2- 172.21.15.67:0/227659430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0000d1ca0 0x7fa0000b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 >> 172.21.15.67:0/227659430 conn(0x7fa00009d910 msgr2=0x7fa0000b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 shutdown_connections 2024-05-30T17:04:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:18.914+0000 7fa007000640 1 -- 172.21.15.67:0/227659430 wait complete. 2024-05-30T17:04:23.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:04:24.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.162+0000 7fa893200640 1 Processor -- start 2024-05-30T17:04:24.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.162+0000 7fa893200640 1 -- start start 2024-05-30T17:04:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.164+0000 7fa893200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8940d1ca0 0x7fa8940d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.164+0000 7fa893200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8940d3060 con 0x7fa8940d4c40 2024-05-30T17:04:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.164+0000 7fa893200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8940d3340 con 0x7fa89415e690 2024-05-30T17:04:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.164+0000 7fa893200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa8940d3620 con 0x7fa8940d1ca0 2024-05-30T17:04:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.164+0000 7fa891400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa8940d4c40 0x7fa8940d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56708/0 (socket says 172.21.15.67:56708) 2024-05-30T17:04:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.164+0000 7fa891400640 1 -- 172.21.15.67:0/155747959 learned_addr learned my addr 172.21.15.67:0/155747959 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009622874 0 0) 0x7fa8940d3060 con 0x7fa8940d4c40 2024-05-30T17:04:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa878003520 con 0x7fa8940d4c40 2024-05-30T17:04:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa892800640 1 --2- 172.21.15.67:0/155747959 >> v2:172.21.15.169:3300/0 conn(0x7fa8940d1ca0 0x7fa8940d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3320821856 0 0) 0x7fa878003520 con 0x7fa8940d4c40 2024-05-30T17:04:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 >> v2:172.21.15.169:3300/0 conn(0x7fa8940d1ca0 msgr2=0x7fa8940d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 --2- 172.21.15.67:0/155747959 >> v2:172.21.15.169:3300/0 conn(0x7fa8940d1ca0 0x7fa8940d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 >> v1:172.21.15.169:6789/0 conn(0x7fa89415e690 legacy=0x7fa8940d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8940cf770 con 0x7fa8940d4c40 2024-05-30T17:04:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fa8800017f0 con 0x7fa8940d4c40 2024-05-30T17:04:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 >> v1:172.21.15.67:6789/0 conn(0x7fa8940d4c40 legacy=0x7fa8940d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa878003a00 0x7fa878003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878004800 0x7fa878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8780052e0 0x7fa87800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa8940d3620 con 0x7fa878003a00 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.165+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa87800c230 con 0x7fa878004800 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa87800c430 con 0x7fa8780052e0 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa892800640 1 --2- 172.21.15.67:0/155747959 >> v2:172.21.15.169:3300/0 conn(0x7fa8940d1ca0 0x7fa8940d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa891e00640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa878003a00 0x7fa878003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa891e00640 1 -- 172.21.15.67:0/155747959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8780052e0 msgr2=0x7fa87800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa891e00640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8780052e0 0x7fa87800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa891e00640 1 -- 172.21.15.67:0/155747959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878004800 msgr2=0x7fa878004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa891400640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878004800 0x7fa878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa891e00640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878004800 0x7fa878004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa891e00640 1 -- 172.21.15.67:0/155747959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa878003520 con 0x7fa878003a00 2024-05-30T17:04:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa892800640 1 -- 172.21.15.67:0/155747959 reap_dead start 2024-05-30T17:04:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.166+0000 7fa891400640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878004800 0x7fa878004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.168+0000 7fa891e00640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa878003a00 0x7fa878003e00 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa884003d80 tx=0x7fa878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24920ef04fddca2f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.168+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884037070 con 0x7fa878003a00 2024-05-30T17:04:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.168+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88402a9c0 con 0x7fa878003a00 2024-05-30T17:04:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.168+0000 7fa890a00640 1 -- 172.21.15.67:0/155747959 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884003980 con 0x7fa878003a00 2024-05-30T17:04:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.169+0000 7fa893200640 1 -- 172.21.15.67:0/155747959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa878003a00 msgr2=0x7fa878003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.169+0000 7fa893200640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa878003a00 0x7fa878003e00 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa884003d80 tx=0x7fa878005830 comp rx=0 tx=0).stop 2024-05-30T17:04:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.169+0000 7fa893200640 1 -- 172.21.15.67:0/155747959 shutdown_connections 2024-05-30T17:04:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.169+0000 7fa893200640 1 --2- 172.21.15.67:0/155747959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa878003a00 0x7fa878003e00 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.169+0000 7fa893200640 1 -- 172.21.15.67:0/155747959 >> 172.21.15.67:0/155747959 conn(0x7fa89409d910 msgr2=0x7fa89408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.169+0000 7fa893200640 1 -- 172.21.15.67:0/155747959 shutdown_connections 2024-05-30T17:04:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.169+0000 7fa893200640 1 -- 172.21.15.67:0/155747959 wait complete. 2024-05-30T17:04:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.171+0000 7fa893200640 1 Processor -- start 2024-05-30T17:04:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.171+0000 7fa893200640 1 -- start start 2024-05-30T17:04:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa893200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878003a00 0x7fa89415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa893200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8940ce270 0x7fa8940ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa893200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa89415fdc0 0x7fa8940cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa893200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa89415ebf0 con 0x7fa8940ce270 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa893200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa89415ea70 con 0x7fa878003a00 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa893200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa89415ed70 con 0x7fa89415fdc0 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa891400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8940ce270 0x7fa8940ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa891400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8940ce270 0x7fa8940ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49276/0 (socket says 172.21.15.67:49276) 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa891400640 1 -- 172.21.15.67:0/4264491294 learned_addr learned my addr 172.21.15.67:0/4264491294 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa891e00640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878003a00 0x7fa89415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa892800640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa89415fdc0 0x7fa8940cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa891400640 1 -- 172.21.15.67:0/4264491294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa89415fdc0 msgr2=0x7fa8940cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa891400640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa89415fdc0 0x7fa8940cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa891400640 1 -- 172.21.15.67:0/4264491294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878003a00 msgr2=0x7fa89415aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.173+0000 7fa891400640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878003a00 0x7fa89415aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.174+0000 7fa891400640 1 -- 172.21.15.67:0/4264491294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8940cb670 con 0x7fa8940ce270 2024-05-30T17:04:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.174+0000 7fa891400640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8940ce270 0x7fa8940ca0c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa88000d7a0 tx=0x7fa88000dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.174+0000 7fa88a000640 1 -- 172.21.15.67:0/4264491294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880007270 con 0x7fa8940ce270 2024-05-30T17:04:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.174+0000 7fa88a000640 1 -- 172.21.15.67:0/4264491294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa880017070 con 0x7fa8940ce270 2024-05-30T17:04:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.174+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8940cb950 con 0x7fa8940ce270 2024-05-30T17:04:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.174+0000 7fa88a000640 1 -- 172.21.15.67:0/4264491294 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8800057a0 con 0x7fa8940ce270 2024-05-30T17:04:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.174+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8940cbf10 con 0x7fa8940ce270 2024-05-30T17:04:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.176+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8940689e0 con 0x7fa8940ce270 2024-05-30T17:04:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.177+0000 7fa88a000640 1 -- 172.21.15.67:0/4264491294 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7fa880005940 con 0x7fa8940ce270 2024-05-30T17:04:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.178+0000 7fa88a000640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa87406e3f0 0x7fa8740708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.178+0000 7fa88a000640 1 -- 172.21.15.67:0/4264491294 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7fa88008a9b0 con 0x7fa8940ce270 2024-05-30T17:04:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.179+0000 7fa891e00640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa87406e3f0 0x7fa8740708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.179+0000 7fa891e00640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa87406e3f0 0x7fa8740708b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa884034000 tx=0x7fa88402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.183+0000 7fa88a000640 1 -- 172.21.15.67:0/4264491294 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa88005a8e0 con 0x7fa8940ce270 2024-05-30T17:04:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.464+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa89406cbc0 con 0x7fa8940ce270 2024-05-30T17:04:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.465+0000 7fa88a000640 1 -- 172.21.15.67:0/4264491294 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1986 (secure 0 0 0) 0x7fa88001ae40 con 0x7fa8940ce270 2024-05-30T17:04:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:24.466 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:04:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:04:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:04:24.466 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:04:24.467 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1257, 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 118, 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:04:24.468 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1749, 2024-05-30T17:04:24.469 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 7207149636, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 15494045696, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 757600067584, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 74, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 8, 2024-05-30T17:04:24.470 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 8412 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:24.471 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:03:48.549378+0000", 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "87f1ded5-ae09-4729-991d-ae326047c9cc": { 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "All cloning finished, waiting 4.0 seconds before exiting (33s)\n [============================] ", 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 1, 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:24.472 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:04:24.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa87406e3f0 msgr2=0x7fa8740708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:24.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa87406e3f0 0x7fa8740708b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa884034000 tx=0x7fa88402e000 comp rx=0 tx=0).stop 2024-05-30T17:04:24.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8940ce270 msgr2=0x7fa8940ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8940ce270 0x7fa8940ca0c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa88000d7a0 tx=0x7fa88000dba0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 shutdown_connections 2024-05-30T17:04:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa87406e3f0 0x7fa8740708b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa89415fdc0 0x7fa8940cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8940ce270 0x7fa8940ca0c0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 --2- 172.21.15.67:0/4264491294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa878003a00 0x7fa89415aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.468+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 >> 172.21.15.67:0/4264491294 conn(0x7fa89409d910 msgr2=0x7fa8940d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.469+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 shutdown_connections 2024-05-30T17:04:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:24.469+0000 7fa893200640 1 -- 172.21.15.67:0/4264491294 wait complete. 2024-05-30T17:04:29.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:04:29.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.714+0000 7f95eec00640 1 Processor -- start 2024-05-30T17:04:29.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.714+0000 7f95eec00640 1 -- start start 2024-05-30T17:04:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.715+0000 7f95eec00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f95e815d170 0x7f95e80c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.715+0000 7f95eec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e80d0b70 0x7f95e80b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95eec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e815d550 con 0x7f95e8163dc0 2024-05-30T17:04:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95eec00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f95e80adb00 con 0x7f95e815d170 2024-05-30T17:04:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95eec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f95e80aefa0 con 0x7f95e80d0b70 2024-05-30T17:04:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95ed800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f95e815d170 0x7f95e80c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95ed800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f95e815d170 0x7f95e80c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49282/0 (socket says 172.21.15.67:49282) 2024-05-30T17:04:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95ed800640 1 -- 172.21.15.67:0/2432641348 learned_addr learned my addr 172.21.15.67:0/2432641348 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95ed800640 1 -- 172.21.15.67:0/2432641348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e80d0b70 msgr2=0x7f95e80b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95ed800640 1 --2- 172.21.15.67:0/2432641348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e80d0b70 0x7f95e80b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95ed800640 1 -- 172.21.15.67:0/2432641348 >> v1:172.21.15.169:6789/0 conn(0x7f95e8163dc0 legacy=0x7f95e80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.716+0000 7f95ed800640 1 -- 172.21.15.67:0/2432641348 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e80ad1d0 con 0x7f95e815d170 2024-05-30T17:04:29.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.717+0000 7f95ed800640 1 --2- 172.21.15.67:0/2432641348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e815d170 0x7f95e80c50c0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009870 tx=0x7f95dc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1891bb552cfd4212 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:29.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.718+0000 7f95e7e00640 1 -- 172.21.15.67:0/2432641348 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc037070 con 0x7f95e815d170 2024-05-30T17:04:29.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.718+0000 7f95e7e00640 1 -- 172.21.15.67:0/2432641348 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95dc032440 con 0x7f95e815d170 2024-05-30T17:04:29.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.718+0000 7f95e7e00640 1 -- 172.21.15.67:0/2432641348 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc032810 con 0x7f95e815d170 2024-05-30T17:04:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.718+0000 7f95eec00640 1 -- 172.21.15.67:0/2432641348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e815d170 msgr2=0x7f95e80c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.718+0000 7f95eec00640 1 --2- 172.21.15.67:0/2432641348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e815d170 0x7f95e80c50c0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009870 tx=0x7f95dc02a210 comp rx=0 tx=0).stop 2024-05-30T17:04:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.719+0000 7f95eec00640 1 -- 172.21.15.67:0/2432641348 shutdown_connections 2024-05-30T17:04:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.719+0000 7f95eec00640 1 --2- 172.21.15.67:0/2432641348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e80d0b70 0x7f95e80b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.719+0000 7f95eec00640 1 --2- 172.21.15.67:0/2432641348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e815d170 0x7f95e80c50c0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.719+0000 7f95eec00640 1 -- 172.21.15.67:0/2432641348 >> 172.21.15.67:0/2432641348 conn(0x7f95e806bae0 msgr2=0x7f95e806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.719+0000 7f95eec00640 1 -- 172.21.15.67:0/2432641348 shutdown_connections 2024-05-30T17:04:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.719+0000 7f95eec00640 1 -- 172.21.15.67:0/2432641348 wait complete. 2024-05-30T17:04:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.720+0000 7f95eec00640 1 Processor -- start 2024-05-30T17:04:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.721+0000 7f95eec00640 1 -- start start 2024-05-30T17:04:29.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95eec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95e80d0b70 0x7f95e80b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:29.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95eec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e815d170 0x7f95e80cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:29.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95eec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e8163dc0 0x7f95e80cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:29.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95eec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f95e80b3150 con 0x7f95e8163dc0 2024-05-30T17:04:29.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95eec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f95e80ad790 con 0x7f95e80d0b70 2024-05-30T17:04:29.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95eec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f95e80cc4a0 con 0x7f95e815d170 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ee200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e8163dc0 0x7f95e80cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ee200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e8163dc0 0x7f95e80cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49288/0 (socket says 172.21.15.67:49288) 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ee200640 1 -- 172.21.15.67:0/443283830 learned_addr learned my addr 172.21.15.67:0/443283830 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ed800640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95e80d0b70 0x7f95e80b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ece00640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e815d170 0x7f95e80cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ee200640 1 -- 172.21.15.67:0/443283830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e815d170 msgr2=0x7f95e80cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ee200640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e815d170 0x7f95e80cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ee200640 1 -- 172.21.15.67:0/443283830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95e80d0b70 msgr2=0x7f95e80b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ee200640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95e80d0b70 0x7f95e80b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ee200640 1 -- 172.21.15.67:0/443283830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e80cb890 con 0x7f95e8163dc0 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.722+0000 7f95ece00640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e815d170 0x7f95e80cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.723+0000 7f95ee200640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e8163dc0 0x7f95e80cb050 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f95e000b970 tx=0x7f95e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.723+0000 7f95e5600640 1 -- 172.21.15.67:0/443283830 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0019070 con 0x7f95e8163dc0 2024-05-30T17:04:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.723+0000 7f95e5600640 1 -- 172.21.15.67:0/443283830 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e000cb60 con 0x7f95e8163dc0 2024-05-30T17:04:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.723+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e80cbbd0 con 0x7f95e8163dc0 2024-05-30T17:04:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.723+0000 7f95e5600640 1 -- 172.21.15.67:0/443283830 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0014440 con 0x7f95e8163dc0 2024-05-30T17:04:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.723+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e80cb590 con 0x7f95e8163dc0 2024-05-30T17:04:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.724+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e80689e0 con 0x7f95e8163dc0 2024-05-30T17:04:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.725+0000 7f95e5600640 1 -- 172.21.15.67:0/443283830 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7f95e001c070 con 0x7f95e8163dc0 2024-05-30T17:04:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.726+0000 7f95e5600640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95c806e420 0x7f95c80708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.726+0000 7f95e5600640 1 -- 172.21.15.67:0/443283830 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f95e008a7a0 con 0x7f95e8163dc0 2024-05-30T17:04:29.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.727+0000 7f95ed800640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95c806e420 0x7f95c80708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:29.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.727+0000 7f95ed800640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95c806e420 0x7f95c80708e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f95dc034000 tx=0x7f95dc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:29.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:29.731+0000 7f95e5600640 1 -- 172.21.15.67:0/443283830 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95e80cbbd0 con 0x7f95e8163dc0 2024-05-30T17:04:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.010+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f95e80af2f0 con 0x7f95e8163dc0 2024-05-30T17:04:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.011+0000 7f95e5600640 1 -- 172.21.15.67:0/443283830 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1732 (secure 0 0 0) 0x7f95e005a630 con 0x7f95e8163dc0 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:04:30.012 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1263, 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:04:30.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 118, 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 1749, 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 7207149636, 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 15494045696, 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 757600067584, 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 74, 2024-05-30T17:04:30.015 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 8, 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 8, 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "d", 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 8412 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:04:30.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.017 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:04:30.018 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 21, 2024-05-30T17:04:30.018 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:03:48.549378+0000", 2024-05-30T17:04:30.018 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:04:30.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.018 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": {} 2024-05-30T17:04:30.018 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95c806e420 msgr2=0x7f95c80708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95c806e420 0x7f95c80708e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f95dc034000 tx=0x7f95dc02e000 comp rx=0 tx=0).stop 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e8163dc0 msgr2=0x7f95e80cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e8163dc0 0x7f95e80cb050 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f95e000b970 tx=0x7f95e000be30 comp rx=0 tx=0).stop 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 shutdown_connections 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f95c806e420 0x7f95c80708e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95e8163dc0 0x7f95e80cb050 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95e815d170 0x7f95e80cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 --2- 172.21.15.67:0/443283830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95e80d0b70 0x7f95e80b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 >> 172.21.15.67:0/443283830 conn(0x7f95e806bae0 msgr2=0x7f95e80b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.014+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 shutdown_connections 2024-05-30T17:04:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.015+0000 7f95eec00640 1 -- 172.21.15.67:0/443283830 wait complete. 2024-05-30T17:04:30.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-30T17:04:30.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.246+0000 7f1476600640 1 Processor -- start 2024-05-30T17:04:30.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.247+0000 7f1476600640 1 -- start start 2024-05-30T17:04:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.248+0000 7f1476600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14700d16a0 0x7f14700cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.248+0000 7f1476600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14700cf870 0x7f14700d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.248+0000 7f1476600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14700cfce0 con 0x7f147015e870 2024-05-30T17:04:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.248+0000 7f1476600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f14700d2150 con 0x7f14700cf870 2024-05-30T17:04:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.248+0000 7f1476600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f147015ec50 con 0x7f14700d16a0 2024-05-30T17:04:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.249+0000 7f1475c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14700cf870 0x7f14700d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.249+0000 7f1475c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14700cf870 0x7f14700d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49304/0 (socket says 172.21.15.67:49304) 2024-05-30T17:04:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.249+0000 7f1475c00640 1 -- 172.21.15.67:0/2444035404 learned_addr learned my addr 172.21.15.67:0/2444035404 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:30.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.249+0000 7f1475c00640 1 -- 172.21.15.67:0/2444035404 >> v2:172.21.15.169:3300/0 conn(0x7f14700d16a0 msgr2=0x7f14700cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:30.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.249+0000 7f1475c00640 1 --2- 172.21.15.67:0/2444035404 >> v2:172.21.15.169:3300/0 conn(0x7f14700d16a0 0x7f14700cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.249+0000 7f1475c00640 1 -- 172.21.15.67:0/2444035404 >> v1:172.21.15.169:6789/0 conn(0x7f147015e870 legacy=0x7f14700d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:30.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.249+0000 7f1475c00640 1 -- 172.21.15.67:0/2444035404 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14700cff80 con 0x7f14700cf870 2024-05-30T17:04:30.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.251+0000 7f1475c00640 1 --2- 172.21.15.67:0/2444035404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14700cf870 0x7f14700d25e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f147008e520 tx=0x7f146802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31619b0938b8368 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:30.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.251+0000 7f146f400640 1 -- 172.21.15.67:0/2444035404 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468037070 con 0x7f14700cf870 2024-05-30T17:04:30.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.251+0000 7f146f400640 1 -- 172.21.15.67:0/2444035404 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1468032440 con 0x7f14700cf870 2024-05-30T17:04:30.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.251+0000 7f146f400640 1 -- 172.21.15.67:0/2444035404 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468032810 con 0x7f14700cf870 2024-05-30T17:04:30.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.252+0000 7f1476600640 1 -- 172.21.15.67:0/2444035404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14700cf870 msgr2=0x7f14700d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.252+0000 7f1476600640 1 --2- 172.21.15.67:0/2444035404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14700cf870 0x7f14700d25e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f147008e520 tx=0x7f146802a230 comp rx=0 tx=0).stop 2024-05-30T17:04:30.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.252+0000 7f1476600640 1 -- 172.21.15.67:0/2444035404 shutdown_connections 2024-05-30T17:04:30.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.252+0000 7f1476600640 1 --2- 172.21.15.67:0/2444035404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14700cf870 0x7f14700d25e0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.252+0000 7f1476600640 1 --2- 172.21.15.67:0/2444035404 >> v2:172.21.15.169:3300/0 conn(0x7f14700d16a0 0x7f14700cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.252+0000 7f1476600640 1 -- 172.21.15.67:0/2444035404 >> 172.21.15.67:0/2444035404 conn(0x7f147008bfc0 msgr2=0x7f14700d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:30.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.252+0000 7f1476600640 1 -- 172.21.15.67:0/2444035404 shutdown_connections 2024-05-30T17:04:30.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.252+0000 7f1476600640 1 -- 172.21.15.67:0/2444035404 wait complete. 2024-05-30T17:04:30.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.254+0000 7f1476600640 1 Processor -- start 2024-05-30T17:04:30.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.254+0000 7f1476600640 1 -- start start 2024-05-30T17:04:30.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.255+0000 7f1476600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14700cf870 0x7f1470150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.255+0000 7f1476600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14700d16a0 0x7f14700ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.255+0000 7f1476600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f147015e870 0x7f14700caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.255+0000 7f1476600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14700d0820 con 0x7f147015e870 2024-05-30T17:04:30.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.255+0000 7f1476600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14700d0540 con 0x7f14700d16a0 2024-05-30T17:04:30.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.255+0000 7f1476600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14700d4fb0 con 0x7f14700cf870 2024-05-30T17:04:30.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f1475c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f147015e870 0x7f14700caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f1475c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f147015e870 0x7f14700caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49316/0 (socket says 172.21.15.67:49316) 2024-05-30T17:04:30.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f1475c00640 1 -- 172.21.15.67:0/2924564074 learned_addr learned my addr 172.21.15.67:0/2924564074 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f1475200640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14700cf870 0x7f1470150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f146fe00640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14700d16a0 0x7f14700ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f1475c00640 1 -- 172.21.15.67:0/2924564074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14700cf870 msgr2=0x7f1470150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f1475c00640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14700cf870 0x7f1470150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f1475c00640 1 -- 172.21.15.67:0/2924564074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14700d16a0 msgr2=0x7f14700ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f1475c00640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14700d16a0 0x7f14700ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.256+0000 7f1475c00640 1 -- 172.21.15.67:0/2924564074 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14700b75a0 con 0x7f147015e870 2024-05-30T17:04:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.257+0000 7f1475c00640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f147015e870 0x7f14700caed0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f14680097c0 tx=0x7f14680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.257+0000 7f146cc00640 1 -- 172.21.15.67:0/2924564074 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468041070 con 0x7f147015e870 2024-05-30T17:04:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.257+0000 7f146cc00640 1 -- 172.21.15.67:0/2924564074 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f146803c870 con 0x7f147015e870 2024-05-30T17:04:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.257+0000 7f146cc00640 1 -- 172.21.15.67:0/2924564074 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468037040 con 0x7f147015e870 2024-05-30T17:04:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.257+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14700b7840 con 0x7f147015e870 2024-05-30T17:04:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.257+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14700b7e00 con 0x7f147015e870 2024-05-30T17:04:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.258+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14700689e0 con 0x7f147015e870 2024-05-30T17:04:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.259+0000 7f146cc00640 1 -- 172.21.15.67:0/2924564074 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7f1468044070 con 0x7f147015e870 2024-05-30T17:04:30.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.260+0000 7f146cc00640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f144c06e3f0 0x7f144c0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.260+0000 7f146cc00640 1 -- 172.21.15.67:0/2924564074 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7f14680ab570 con 0x7f147015e870 2024-05-30T17:04:30.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.261+0000 7f1475200640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f144c06e3f0 0x7f144c0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.261+0000 7f1475200640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f144c06e3f0 0x7f144c0708b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1460001db0 tx=0x7f1460009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.265+0000 7f146cc00640 1 -- 172.21.15.67:0/2924564074 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f146807b400 con 0x7f147015e870 2024-05-30T17:04:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.496+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f14700ba500 con 0x7f144c06e3f0 2024-05-30T17:04:30.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.511+0000 7f146cc00640 1 -- 172.21.15.67:0/2924564074 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+185 (secure 0 0 0) 0x7f14700ba500 con 0x7f144c06e3f0 2024-05-30T17:04:30.511 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sv1" 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone93313" 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone93311" 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subvol_clone93312" 2024-05-30T17:04:30.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:30.513 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:04:30.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f144c06e3f0 msgr2=0x7f144c0708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f144c06e3f0 0x7f144c0708b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1460001db0 tx=0x7f1460009040 comp rx=0 tx=0).stop 2024-05-30T17:04:30.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f147015e870 msgr2=0x7f14700caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f147015e870 0x7f14700caed0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f14680097c0 tx=0x7f14680046c0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 shutdown_connections 2024-05-30T17:04:30.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f144c06e3f0 0x7f144c0708b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f147015e870 0x7f14700caed0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14700d16a0 0x7f14700ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 --2- 172.21.15.67:0/2924564074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14700cf870 0x7f1470150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.513+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 >> 172.21.15.67:0/2924564074 conn(0x7f147008bfc0 msgr2=0x7f14700d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:30.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.514+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 shutdown_connections 2024-05-30T17:04:30.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.514+0000 7f1476600640 1 -- 172.21.15.67:0/2924564074 wait complete. 2024-05-30T17:04:30.533 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-30T17:04:30.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.745+0000 7fb0bda00640 1 Processor -- start 2024-05-30T17:04:30.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.746+0000 7fb0bda00640 1 -- start start 2024-05-30T17:04:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.747+0000 7fb0bda00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb0b815fa00 0x7fb0b806e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.747+0000 7fb0bda00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0b80d0b40 0x7fb0b8072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.747+0000 7fb0bda00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b806fa00 con 0x7fb0b81600f0 2024-05-30T17:04:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.747+0000 7fb0bda00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b806eeb0 con 0x7fb0b80d0b40 2024-05-30T17:04:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.747+0000 7fb0bda00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b806f160 con 0x7fb0b815fa00 2024-05-30T17:04:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.747+0000 7fb0bd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0b80d0b40 0x7fb0b8072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.747+0000 7fb0bd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0b80d0b40 0x7fb0b8072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49318/0 (socket says 172.21.15.67:49318) 2024-05-30T17:04:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.747+0000 7fb0bd000640 1 -- 172.21.15.67:0/1345367434 learned_addr learned my addr 172.21.15.67:0/1345367434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:30.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.748+0000 7fb0bd000640 1 -- 172.21.15.67:0/1345367434 >> v2:172.21.15.169:3300/0 conn(0x7fb0b815fa00 msgr2=0x7fb0b806e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:30.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.748+0000 7fb0bd000640 1 --2- 172.21.15.67:0/1345367434 >> v2:172.21.15.169:3300/0 conn(0x7fb0b815fa00 0x7fb0b806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.748+0000 7fb0bd000640 1 -- 172.21.15.67:0/1345367434 >> v1:172.21.15.169:6789/0 conn(0x7fb0b81600f0 legacy=0x7fb0b80d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:30.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.748+0000 7fb0bd000640 1 -- 172.21.15.67:0/1345367434 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b806f440 con 0x7fb0b80d0b40 2024-05-30T17:04:30.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.749+0000 7fb0bd000640 1 --2- 172.21.15.67:0/1345367434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b80d0b40 0x7fb0b8072190 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb0b806e8c0 tx=0x7fb0b002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e2d3dba7fd4438 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:30.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.749+0000 7fb0b6a00640 1 -- 172.21.15.67:0/1345367434 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0036070 con 0x7fb0b80d0b40 2024-05-30T17:04:30.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.749+0000 7fb0b6a00640 1 -- 172.21.15.67:0/1345367434 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b0031600 con 0x7fb0b80d0b40 2024-05-30T17:04:30.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.749+0000 7fb0b6a00640 1 -- 172.21.15.67:0/1345367434 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00319d0 con 0x7fb0b80d0b40 2024-05-30T17:04:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.750+0000 7fb0bda00640 1 -- 172.21.15.67:0/1345367434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b80d0b40 msgr2=0x7fb0b8072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.750+0000 7fb0bda00640 1 --2- 172.21.15.67:0/1345367434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b80d0b40 0x7fb0b8072190 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb0b806e8c0 tx=0x7fb0b002a5b0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.750+0000 7fb0bda00640 1 -- 172.21.15.67:0/1345367434 shutdown_connections 2024-05-30T17:04:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.750+0000 7fb0bda00640 1 --2- 172.21.15.67:0/1345367434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b80d0b40 0x7fb0b8072190 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.750+0000 7fb0bda00640 1 --2- 172.21.15.67:0/1345367434 >> v2:172.21.15.169:3300/0 conn(0x7fb0b815fa00 0x7fb0b806e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.750+0000 7fb0bda00640 1 -- 172.21.15.67:0/1345367434 >> 172.21.15.67:0/1345367434 conn(0x7fb0b806bae0 msgr2=0x7fb0b806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.750+0000 7fb0bda00640 1 -- 172.21.15.67:0/1345367434 shutdown_connections 2024-05-30T17:04:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.750+0000 7fb0bda00640 1 -- 172.21.15.67:0/1345367434 wait complete. 2024-05-30T17:04:30.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.752+0000 7fb0bda00640 1 Processor -- start 2024-05-30T17:04:30.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.752+0000 7fb0bda00640 1 -- start start 2024-05-30T17:04:30.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bda00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0b80d0b40 0x7fb0b8152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bda00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0b815fa00 0x7fb0b81477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81600f0 0x7fb0b81f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b815fde0 con 0x7fb0b81600f0 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bda00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8071c70 con 0x7fb0b815fa00 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bda00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b80726d0 con 0x7fb0b80d0b40 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81600f0 0x7fb0b81f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81600f0 0x7fb0b81f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49320/0 (socket says 172.21.15.67:49320) 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bd000640 1 -- 172.21.15.67:0/4090190070 learned_addr learned my addr 172.21.15.67:0/4090190070 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0b7e00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0b80d0b40 0x7fb0b8152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0b7400640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0b815fa00 0x7fb0b81477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bd000640 1 -- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0b80d0b40 msgr2=0x7fb0b8152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bd000640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0b80d0b40 0x7fb0b8152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bd000640 1 -- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0b815fa00 msgr2=0x7fb0b81477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bd000640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0b815fa00 0x7fb0b81477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0bd000640 1 -- 172.21.15.67:0/4090190070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b80adc50 con 0x7fb0b81600f0 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.754+0000 7fb0b7e00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0b80d0b40 0x7fb0b8152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.755+0000 7fb0b7400640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0b815fa00 0x7fb0b81477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.755+0000 7fb0bd000640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81600f0 0x7fb0b81f2490 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0034040 tx=0x7fb0b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.755+0000 7fb0afe00640 1 -- 172.21.15.67:0/4090190070 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0041020 con 0x7fb0b81600f0 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.755+0000 7fb0afe00640 1 -- 172.21.15.67:0/4090190070 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b0007780 con 0x7fb0b81600f0 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.755+0000 7fb0afe00640 1 -- 172.21.15.67:0/4090190070 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0036040 con 0x7fb0b81600f0 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.755+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b815d6c0 con 0x7fb0b81600f0 2024-05-30T17:04:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.755+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b815dc80 con 0x7fb0b81600f0 2024-05-30T17:04:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.756+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b80689e0 con 0x7fb0b81600f0 2024-05-30T17:04:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.757+0000 7fb0afe00640 1 -- 172.21.15.67:0/4090190070 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7fb0b0044020 con 0x7fb0b81600f0 2024-05-30T17:04:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.758+0000 7fb0afe00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb09806e3f0 0x7fb0980708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.758+0000 7fb0afe00640 1 -- 172.21.15.67:0/4090190070 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7fb0b00ab900 con 0x7fb0b81600f0 2024-05-30T17:04:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.759+0000 7fb0b7e00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb09806e3f0 0x7fb0980708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.759+0000 7fb0b7e00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb09806e3f0 0x7fb0980708b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8006fd0 tx=0x7fb0a8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:30.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:30.764+0000 7fb0afe00640 1 -- 172.21.15.67:0/4090190070 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0b004b030 con 0x7fb0b81600f0 2024-05-30T17:04:31.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.005+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0b80cda60 con 0x7fb09806e3f0 2024-05-30T17:04:31.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.043+0000 7fb0afe00640 1 -- 172.21.15.67:0/4090190070 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb0b80cda60 con 0x7fb09806e3f0 2024-05-30T17:04:31.044 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:31.044 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:04:31.044 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:31.044 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ss1" 2024-05-30T17:04:31.044 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:31.044 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:04:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb09806e3f0 msgr2=0x7fb0980708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb09806e3f0 0x7fb0980708b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8006fd0 tx=0x7fb0a8008040 comp rx=0 tx=0).stop 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81600f0 msgr2=0x7fb0b81f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81600f0 0x7fb0b81f2490 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0034040 tx=0x7fb0b0004a40 comp rx=0 tx=0).stop 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 shutdown_connections 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb09806e3f0 0x7fb0980708b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0b81600f0 0x7fb0b81f2490 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0b815fa00 0x7fb0b81477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 --2- 172.21.15.67:0/4090190070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0b80d0b40 0x7fb0b8152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.045+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 >> 172.21.15.67:0/4090190070 conn(0x7fb0b806bae0 msgr2=0x7fb0b80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.046+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 shutdown_connections 2024-05-30T17:04:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.046+0000 7fb0bda00640 1 -- 172.21.15.67:0/4090190070 wait complete. 2024-05-30T17:04:31.066 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-30T17:04:31.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.283+0000 7fe723200640 1 Processor -- start 2024-05-30T17:04:31.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.283+0000 7fe723200640 1 -- start start 2024-05-30T17:04:31.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.284+0000 7fe723200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe72415e870 0x7fe7240d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.284+0000 7fe723200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe7240d14d0 0x7fe7240cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.285+0000 7fe723200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7240cfce0 con 0x7fe7240cf870 2024-05-30T17:04:31.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.285+0000 7fe723200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe7240d2150 con 0x7fe72415e870 2024-05-30T17:04:31.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.285+0000 7fe723200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe72415ec50 con 0x7fe7240d14d0 2024-05-30T17:04:31.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.285+0000 7fe721e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe72415e870 0x7fe7240d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.285+0000 7fe721e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe72415e870 0x7fe7240d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52012/0 (socket says 172.21.15.67:52012) 2024-05-30T17:04:31.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.285+0000 7fe721e00640 1 -- 172.21.15.67:0/3518971460 learned_addr learned my addr 172.21.15.67:0/3518971460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:31.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.285+0000 7fe721400640 1 --2- 172.21.15.67:0/3518971460 >> v2:172.21.15.169:3300/0 conn(0x7fe7240d14d0 0x7fe7240cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.285+0000 7fe721e00640 1 -- 172.21.15.67:0/3518971460 >> v2:172.21.15.169:3300/0 conn(0x7fe7240d14d0 msgr2=0x7fe7240cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:31.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.285+0000 7fe721e00640 1 --2- 172.21.15.67:0/3518971460 >> v2:172.21.15.169:3300/0 conn(0x7fe7240d14d0 0x7fe7240cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.286+0000 7fe721e00640 1 -- 172.21.15.67:0/3518971460 >> v1:172.21.15.169:6789/0 conn(0x7fe7240cf870 legacy=0x7fe724162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:31.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.286+0000 7fe721e00640 1 -- 172.21.15.67:0/3518971460 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7240cff80 con 0x7fe72415e870 2024-05-30T17:04:31.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.286+0000 7fe721400640 1 --2- 172.21.15.67:0/3518971460 >> v2:172.21.15.169:3300/0 conn(0x7fe7240d14d0 0x7fe7240cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:31.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.287+0000 7fe721e00640 1 --2- 172.21.15.67:0/3518971460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe72415e870 0x7fe7240d0f90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fe72408e520 tx=0x7fe71402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff500c0fc3759ff server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:31.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.287+0000 7fe720a00640 1 -- 172.21.15.67:0/3518971460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714037070 con 0x7fe72415e870 2024-05-30T17:04:31.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.287+0000 7fe720a00640 1 -- 172.21.15.67:0/3518971460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe714032440 con 0x7fe72415e870 2024-05-30T17:04:31.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.287+0000 7fe720a00640 1 -- 172.21.15.67:0/3518971460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714032810 con 0x7fe72415e870 2024-05-30T17:04:31.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.288+0000 7fe723200640 1 -- 172.21.15.67:0/3518971460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe72415e870 msgr2=0x7fe7240d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.288+0000 7fe723200640 1 --2- 172.21.15.67:0/3518971460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe72415e870 0x7fe7240d0f90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fe72408e520 tx=0x7fe71402a230 comp rx=0 tx=0).stop 2024-05-30T17:04:31.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.288+0000 7fe723200640 1 -- 172.21.15.67:0/3518971460 shutdown_connections 2024-05-30T17:04:31.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.288+0000 7fe723200640 1 --2- 172.21.15.67:0/3518971460 >> v2:172.21.15.169:3300/0 conn(0x7fe7240d14d0 0x7fe7240cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.288+0000 7fe723200640 1 --2- 172.21.15.67:0/3518971460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe72415e870 0x7fe7240d0f90 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.288+0000 7fe723200640 1 -- 172.21.15.67:0/3518971460 >> 172.21.15.67:0/3518971460 conn(0x7fe72408bfc0 msgr2=0x7fe7240d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:31.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.288+0000 7fe723200640 1 -- 172.21.15.67:0/3518971460 shutdown_connections 2024-05-30T17:04:31.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.288+0000 7fe723200640 1 -- 172.21.15.67:0/3518971460 wait complete. 2024-05-30T17:04:31.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.290+0000 7fe723200640 1 Processor -- start 2024-05-30T17:04:31.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.290+0000 7fe723200640 1 -- start start 2024-05-30T17:04:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe723200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7240cf870 0x7fe72414a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe723200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7240d14d0 0x7fe72415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe723200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe72415e870 0x7fe7240c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe723200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe7240d0820 con 0x7fe7240cf870 2024-05-30T17:04:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe723200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7240d0540 con 0x7fe7240d14d0 2024-05-30T17:04:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe723200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7240d4fb0 con 0x7fe72415e870 2024-05-30T17:04:31.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe721e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7240cf870 0x7fe72414a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe721e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7240cf870 0x7fe72414a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52026/0 (socket says 172.21.15.67:52026) 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe721e00640 1 -- 172.21.15.67:0/1586434930 learned_addr learned my addr 172.21.15.67:0/1586434930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe721400640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7240d14d0 0x7fe72415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe721e00640 1 -- 172.21.15.67:0/1586434930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe72415e870 msgr2=0x7fe7240c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe721e00640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe72415e870 0x7fe7240c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe721e00640 1 -- 172.21.15.67:0/1586434930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7240d14d0 msgr2=0x7fe72415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe721e00640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7240d14d0 0x7fe72415f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.291+0000 7fe721e00640 1 -- 172.21.15.67:0/1586434930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7240cc970 con 0x7fe7240cf870 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.292+0000 7fe721400640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7240d14d0 0x7fe72415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:31.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.292+0000 7fe721e00640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7240cf870 0x7fe72414a9f0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe714000c00 tx=0x7fe7140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:31.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.292+0000 7fe71a000640 1 -- 172.21.15.67:0/1586434930 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714037040 con 0x7fe7240cf870 2024-05-30T17:04:31.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.292+0000 7fe71a000640 1 -- 172.21.15.67:0/1586434930 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe714042070 con 0x7fe7240cf870 2024-05-30T17:04:31.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.292+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7240ccc50 con 0x7fe7240cf870 2024-05-30T17:04:31.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.292+0000 7fe71a000640 1 -- 172.21.15.67:0/1586434930 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714030bb0 con 0x7fe7240cf870 2024-05-30T17:04:31.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.292+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72414af30 con 0x7fe7240cf870 2024-05-30T17:04:31.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.294+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7240689e0 con 0x7fe7240cf870 2024-05-30T17:04:31.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.295+0000 7fe71a000640 1 -- 172.21.15.67:0/1586434930 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7fe714047460 con 0x7fe7240cf870 2024-05-30T17:04:31.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.296+0000 7fe71a000640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe70006e3f0 0x7fe7000708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.296+0000 7fe71a000640 1 -- 172.21.15.67:0/1586434930 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5879+0+0 (secure 0 0 0) 0x7fe7140ab630 con 0x7fe7240cf870 2024-05-30T17:04:31.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.297+0000 7fe721400640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe70006e3f0 0x7fe7000708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.297+0000 7fe721400640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe70006e3f0 0x7fe7000708b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe72415dff0 tx=0x7fe70c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:31.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.301+0000 7fe71a000640 1 -- 172.21.15.67:0/1586434930 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe714045360 con 0x7fe7240cf870 2024-05-30T17:04:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.546+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7240b37d0 con 0x7fe70006e3f0 2024-05-30T17:04:31.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.632+0000 7fe71a000640 1 -- 172.21.15.67:0/1586434930 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7240b37d0 con 0x7fe70006e3f0 2024-05-30T17:04:31.633 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:31.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.635+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe70006e3f0 msgr2=0x7fe7000708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.635+0000 7fe723200640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe70006e3f0 0x7fe7000708b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe72415dff0 tx=0x7fe70c008040 comp rx=0 tx=0).stop 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.635+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7240cf870 msgr2=0x7fe72414a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.635+0000 7fe723200640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7240cf870 0x7fe72414a9f0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe714000c00 tx=0x7fe7140046c0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.635+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 shutdown_connections 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.635+0000 7fe723200640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe70006e3f0 0x7fe7000708b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.636+0000 7fe723200640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe72415e870 0x7fe7240c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.636+0000 7fe723200640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7240d14d0 0x7fe72415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.636+0000 7fe723200640 1 --2- 172.21.15.67:0/1586434930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7240cf870 0x7fe72414a9f0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.636+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 >> 172.21.15.67:0/1586434930 conn(0x7fe72408bfc0 msgr2=0x7fe7240d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.636+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 shutdown_connections 2024-05-30T17:04:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.636+0000 7fe723200640 1 -- 172.21.15.67:0/1586434930 wait complete. 2024-05-30T17:04:31.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-30T17:04:31.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.865+0000 7fa001e00640 1 Processor -- start 2024-05-30T17:04:31.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.865+0000 7fa001e00640 1 -- start start 2024-05-30T17:04:31.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7fa001e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ffc0d0640 0x7f9ffc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7fa001e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0d0a20 0x7f9ffc0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7fa001e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ffc15cb10 con 0x7f9ffc0b32c0 2024-05-30T17:04:31.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7fa001e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9ffc0adbb0 con 0x7f9ffc0d0640 2024-05-30T17:04:31.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7fa001e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ffc15db10 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7f9ffaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ffc0d0640 0x7f9ffc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7f9ffaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ffc0d0640 0x7f9ffc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52042/0 (socket says 172.21.15.67:52042) 2024-05-30T17:04:31.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7f9ffaa00640 1 -- 172.21.15.67:0/1783517067 learned_addr learned my addr 172.21.15.67:0/1783517067 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:31.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7f9ffb400640 1 --1- 172.21.15.67:0/1783517067 >> v1:172.21.15.67:6789/0 conn(0x7f9ffc0b32c0 0x7f9ffc0b3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42328/0 (socket says 172.21.15.67:42328) 2024-05-30T17:04:31.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.867+0000 7f9ffa000640 1 --2- 172.21.15.67:0/1783517067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0d0a20 0x7f9ffc0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.868+0000 7f9ffaa00640 1 -- 172.21.15.67:0/1783517067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0d0a20 msgr2=0x7f9ffc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:31.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.868+0000 7f9ffaa00640 1 --2- 172.21.15.67:0/1783517067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0d0a20 0x7f9ffc0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.868+0000 7f9ffaa00640 1 -- 172.21.15.67:0/1783517067 >> v1:172.21.15.67:6789/0 conn(0x7f9ffc0b32c0 legacy=0x7f9ffc0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.868+0000 7f9ffaa00640 1 -- 172.21.15.67:0/1783517067 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc0ad5f0 con 0x7f9ffc0d0640 2024-05-30T17:04:31.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.869+0000 7f9ffaa00640 1 --2- 172.21.15.67:0/1783517067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0640 0x7f9ffc15d5d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009870 tx=0x7f9fec02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe05af29564ab532 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:31.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.869+0000 7f9ff9600640 1 -- 172.21.15.67:0/1783517067 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec037070 con 0x7f9ffc0d0640 2024-05-30T17:04:31.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.869+0000 7f9ff9600640 1 -- 172.21.15.67:0/1783517067 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec032440 con 0x7f9ffc0d0640 2024-05-30T17:04:31.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.869+0000 7f9ff9600640 1 -- 172.21.15.67:0/1783517067 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec032810 con 0x7f9ffc0d0640 2024-05-30T17:04:31.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.869+0000 7fa001e00640 1 -- 172.21.15.67:0/1783517067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0640 msgr2=0x7f9ffc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.869+0000 7fa001e00640 1 --2- 172.21.15.67:0/1783517067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0640 0x7f9ffc15d5d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009870 tx=0x7f9fec02a250 comp rx=0 tx=0).stop 2024-05-30T17:04:31.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.870+0000 7fa001e00640 1 -- 172.21.15.67:0/1783517067 shutdown_connections 2024-05-30T17:04:31.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.870+0000 7fa001e00640 1 --2- 172.21.15.67:0/1783517067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0d0a20 0x7f9ffc0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.870+0000 7fa001e00640 1 --2- 172.21.15.67:0/1783517067 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0640 0x7f9ffc15d5d0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.870+0000 7fa001e00640 1 -- 172.21.15.67:0/1783517067 >> 172.21.15.67:0/1783517067 conn(0x7f9ffc06bd20 msgr2=0x7f9ffc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:31.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.870+0000 7fa001e00640 1 -- 172.21.15.67:0/1783517067 shutdown_connections 2024-05-30T17:04:31.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.870+0000 7fa001e00640 1 -- 172.21.15.67:0/1783517067 wait complete. 2024-05-30T17:04:31.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.871+0000 7fa001e00640 1 Processor -- start 2024-05-30T17:04:31.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.871+0000 7fa001e00640 1 -- start start 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.872+0000 7fa001e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0b32c0 0x7f9ffc148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.872+0000 7fa001e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ffc0d0640 0x7f9ffc0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.872+0000 7fa001e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0a20 0x7f9ffc1e6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.872+0000 7fa001e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc15d0b0 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7fa001e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc15cdd0 con 0x7f9ffc0d0640 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7fa001e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ffc0b3e60 con 0x7f9ffc0b32c0 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0a20 0x7f9ffc1e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0a20 0x7f9ffc1e6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52044/0 (socket says 172.21.15.67:52044) 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffb400640 1 -- 172.21.15.67:0/2733422531 learned_addr learned my addr 172.21.15.67:0/2733422531 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffaa00640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0b32c0 0x7f9ffc148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffa000640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ffc0d0640 0x7f9ffc0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffb400640 1 -- 172.21.15.67:0/2733422531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0b32c0 msgr2=0x7f9ffc148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffb400640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0b32c0 0x7f9ffc148770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffb400640 1 -- 172.21.15.67:0/2733422531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ffc0d0640 msgr2=0x7f9ffc0ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffb400640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ffc0d0640 0x7f9ffc0ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffb400640 1 -- 172.21.15.67:0/2733422531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc0ceea0 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.873+0000 7f9ffa000640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ffc0d0640 0x7f9ffc0ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.874+0000 7f9ffb400640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0a20 0x7f9ffc1e6760 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400c780 tx=0x7f9ff400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.874+0000 7f9ff3400640 1 -- 172.21.15.67:0/2733422531 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4017070 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.874+0000 7f9ff3400640 1 -- 172.21.15.67:0/2733422531 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff400a640 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.874+0000 7f9ff3400640 1 -- 172.21.15.67:0/2733422531 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff400aa30 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.874+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc0cf180 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.874+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc06df60 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.875+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ffc06e160 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.876+0000 7f9ff3400640 1 -- 172.21.15.67:0/2733422531 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7f9ff401a070 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.877+0000 7f9ff3400640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fdc06e3f0 0x7f9fdc0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:31.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.877+0000 7f9ff3400640 1 -- 172.21.15.67:0/2733422531 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5935+0+0 (secure 0 0 0) 0x7f9ff4089fd0 con 0x7f9ffc0d0a20 2024-05-30T17:04:31.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.878+0000 7f9ffaa00640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fdc06e3f0 0x7f9fdc0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:31.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.878+0000 7f9ffaa00640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fdc06e3f0 0x7f9fdc0708b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9fec034000 tx=0x7f9fec02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:31.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:31.882+0000 7f9ff3400640 1 -- 172.21.15.67:0/2733422531 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ffc06e160 con 0x7f9ffc0d0a20 2024-05-30T17:04:32.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.118+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f9ffc0689e0 con 0x7f9fdc06e3f0 2024-05-30T17:04:32.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.164+0000 7f9ff3400640 1 -- 172.21.15.67:0/2733422531 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ffc0689e0 con 0x7f9fdc06e3f0 2024-05-30T17:04:32.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.166+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fdc06e3f0 msgr2=0x7f9fdc0708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.166+0000 7fa001e00640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fdc06e3f0 0x7f9fdc0708b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9fec034000 tx=0x7f9fec02e000 comp rx=0 tx=0).stop 2024-05-30T17:04:32.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.166+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0a20 msgr2=0x7f9ffc1e6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.166+0000 7fa001e00640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0a20 0x7f9ffc1e6760 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400c780 tx=0x7f9ff400cc40 comp rx=0 tx=0).stop 2024-05-30T17:04:32.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.167+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 shutdown_connections 2024-05-30T17:04:32.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.167+0000 7fa001e00640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9fdc06e3f0 0x7f9fdc0708b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.167+0000 7fa001e00640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ffc0d0a20 0x7f9ffc1e6760 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.167+0000 7fa001e00640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ffc0d0640 0x7f9ffc0ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.167+0000 7fa001e00640 1 --2- 172.21.15.67:0/2733422531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ffc0b32c0 0x7f9ffc148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.167+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 >> 172.21.15.67:0/2733422531 conn(0x7f9ffc06bd20 msgr2=0x7f9ffc15efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:32.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.167+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 shutdown_connections 2024-05-30T17:04:32.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.167+0000 7fa001e00640 1 -- 172.21.15.67:0/2733422531 wait complete. 2024-05-30T17:04:32.170 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.170+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.170 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.170+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.171 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.170+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.171 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.170+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.171 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.170+0000 7f0c8c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone93313 2024-05-30T17:04:32.199 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.198+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.199 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.198+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.199 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.198+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.199 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.198+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.199 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:04:32.198+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:04:32.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.372+0000 7fcfa4600640 1 Processor -- start 2024-05-30T17:04:32.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.373+0000 7fcfa4600640 1 -- start start 2024-05-30T17:04:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.374+0000 7fcfa4600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf9c0d34e0 0x7fcf9c0d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.374+0000 7fcfa4600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcf9c0b5570 0x7fcf9c0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.374+0000 7fcfa4600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf9c0d5500 con 0x7fcf9c1622b0 2024-05-30T17:04:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.374+0000 7fcfa4600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fcf9c0b5f00 con 0x7fcf9c0d34e0 2024-05-30T17:04:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.374+0000 7fcfa4600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcf9c0b4f50 con 0x7fcf9c0b5570 2024-05-30T17:04:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.375+0000 7fcfa1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf9c0d34e0 0x7fcf9c0d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.375+0000 7fcfa1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf9c0d34e0 0x7fcf9c0d4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52112/0 (socket says 172.21.15.67:52112) 2024-05-30T17:04:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.375+0000 7fcfa1200640 1 -- 172.21.15.67:0/4123795841 learned_addr learned my addr 172.21.15.67:0/4123795841 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.375+0000 7fcfa2600640 1 --2- 172.21.15.67:0/4123795841 >> v2:172.21.15.169:3300/0 conn(0x7fcf9c0b5570 0x7fcf9c0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.375+0000 7fcfa1200640 1 -- 172.21.15.67:0/4123795841 >> v2:172.21.15.169:3300/0 conn(0x7fcf9c0b5570 msgr2=0x7fcf9c0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.375+0000 7fcfa1200640 1 --2- 172.21.15.67:0/4123795841 >> v2:172.21.15.169:3300/0 conn(0x7fcf9c0b5570 0x7fcf9c0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.375+0000 7fcfa1200640 1 -- 172.21.15.67:0/4123795841 >> v1:172.21.15.169:6789/0 conn(0x7fcf9c1622b0 legacy=0x7fcf9c0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:32.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.375+0000 7fcfa1200640 1 -- 172.21.15.67:0/4123795841 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c0d2720 con 0x7fcf9c0d34e0 2024-05-30T17:04:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.376+0000 7fcfa1200640 1 --2- 172.21.15.67:0/4123795841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c0d34e0 0x7fcf9c0d4e00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0d0d20 tx=0x7fcf8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7063a7a37b99e1f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcf9be00640 1 -- 172.21.15.67:0/4123795841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c037070 con 0x7fcf9c0d34e0 2024-05-30T17:04:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcf9be00640 1 -- 172.21.15.67:0/4123795841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf8c032440 con 0x7fcf9c0d34e0 2024-05-30T17:04:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcf9be00640 1 -- 172.21.15.67:0/4123795841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c032810 con 0x7fcf9c0d34e0 2024-05-30T17:04:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcfa4600640 1 -- 172.21.15.67:0/4123795841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c0d34e0 msgr2=0x7fcf9c0d4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcfa4600640 1 --2- 172.21.15.67:0/4123795841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c0d34e0 0x7fcf9c0d4e00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0d0d20 tx=0x7fcf8c02a1a0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcfa4600640 1 -- 172.21.15.67:0/4123795841 shutdown_connections 2024-05-30T17:04:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcfa4600640 1 --2- 172.21.15.67:0/4123795841 >> v2:172.21.15.169:3300/0 conn(0x7fcf9c0b5570 0x7fcf9c0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcfa4600640 1 --2- 172.21.15.67:0/4123795841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c0d34e0 0x7fcf9c0d4e00 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcfa4600640 1 -- 172.21.15.67:0/4123795841 >> 172.21.15.67:0/4123795841 conn(0x7fcf9c08bfc0 msgr2=0x7fcf9c0d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.377+0000 7fcfa4600640 1 -- 172.21.15.67:0/4123795841 shutdown_connections 2024-05-30T17:04:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.378+0000 7fcfa4600640 1 -- 172.21.15.67:0/4123795841 wait complete. 2024-05-30T17:04:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.379+0000 7fcfa4600640 1 Processor -- start 2024-05-30T17:04:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.379+0000 7fcfa4600640 1 -- start start 2024-05-30T17:04:32.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.380+0000 7fcfa4600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf9c0b5570 0x7fcf9c150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.380+0000 7fcfa4600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf9c0d34e0 0x7fcf9c0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.380+0000 7fcfa4600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c1622b0 0x7fcf9c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.380+0000 7fcfa4600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c0b3f40 con 0x7fcf9c1622b0 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.380+0000 7fcfa4600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c0d2ce0 con 0x7fcf9c0d34e0 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.380+0000 7fcfa4600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c0b40c0 con 0x7fcf9c0b5570 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.380+0000 7fcfa2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c1622b0 0x7fcf9c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.380+0000 7fcfa2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c1622b0 0x7fcf9c0cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52114/0 (socket says 172.21.15.67:52114) 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.380+0000 7fcfa2600640 1 -- 172.21.15.67:0/178073049 learned_addr learned my addr 172.21.15.67:0/178073049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.381+0000 7fcfa1c00640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf9c0b5570 0x7fcf9c150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.381+0000 7fcfa1200640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf9c0d34e0 0x7fcf9c0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.381+0000 7fcfa2600640 1 -- 172.21.15.67:0/178073049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf9c0b5570 msgr2=0x7fcf9c150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.381+0000 7fcfa2600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf9c0b5570 0x7fcf9c150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.381+0000 7fcfa2600640 1 -- 172.21.15.67:0/178073049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf9c0d34e0 msgr2=0x7fcf9c0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.381+0000 7fcfa2600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf9c0d34e0 0x7fcf9c0ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.381+0000 7fcfa2600640 1 -- 172.21.15.67:0/178073049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c0cbb30 con 0x7fcf9c1622b0 2024-05-30T17:04:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.381+0000 7fcfa2600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c1622b0 0x7fcf9c0cdaf0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fcf94007e40 tx=0x7fcf9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.382+0000 7fcf99600640 1 -- 172.21.15.67:0/178073049 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94014620 con 0x7fcf9c1622b0 2024-05-30T17:04:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.382+0000 7fcf99600640 1 -- 172.21.15.67:0/178073049 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf94019070 con 0x7fcf9c1622b0 2024-05-30T17:04:32.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.382+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c0cbe10 con 0x7fcf9c1622b0 2024-05-30T17:04:32.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.382+0000 7fcf99600640 1 -- 172.21.15.67:0/178073049 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94014a20 con 0x7fcf9c1622b0 2024-05-30T17:04:32.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.382+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c0cc3d0 con 0x7fcf9c1622b0 2024-05-30T17:04:32.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.383+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c0689e0 con 0x7fcf9c1622b0 2024-05-30T17:04:32.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.384+0000 7fcf99600640 1 -- 172.21.15.67:0/178073049 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7fcf94014bc0 con 0x7fcf9c1622b0 2024-05-30T17:04:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.385+0000 7fcf99600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcf8006e420 0x7fcf800708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.385+0000 7fcf99600640 1 -- 172.21.15.67:0/178073049 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5935+0+0 (secure 0 0 0) 0x7fcf9408b3d0 con 0x7fcf9c1622b0 2024-05-30T17:04:32.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.385+0000 7fcfa1c00640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcf8006e420 0x7fcf800708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.386+0000 7fcfa1c00640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcf8006e420 0x7fcf800708e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcf90006fd0 tx=0x7fcf90008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:32.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.389+0000 7fcf99600640 1 -- 172.21.15.67:0/178073049 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf9405b1d0 con 0x7fcf9c1622b0 2024-05-30T17:04:32.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.632+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone93313", "target": ["mon-mgr", ""]}) -- 0x7fcf9c08c6c0 con 0x7fcf8006e420 2024-05-30T17:04:32.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.680+0000 7fcf99600640 1 -- 172.21.15.67:0/178073049 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcf9c08c6c0 con 0x7fcf8006e420 2024-05-30T17:04:32.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcf8006e420 msgr2=0x7fcf800708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcf8006e420 0x7fcf800708e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcf90006fd0 tx=0x7fcf90008040 comp rx=0 tx=0).stop 2024-05-30T17:04:32.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c1622b0 msgr2=0x7fcf9c0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c1622b0 0x7fcf9c0cdaf0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fcf94007e40 tx=0x7fcf9400e4b0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 shutdown_connections 2024-05-30T17:04:32.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcf8006e420 0x7fcf800708e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf9c1622b0 0x7fcf9c0cdaf0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf9c0d34e0 0x7fcf9c0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 --2- 172.21.15.67:0/178073049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf9c0b5570 0x7fcf9c150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.683+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 >> 172.21.15.67:0/178073049 conn(0x7fcf9c08bfc0 msgr2=0x7fcf9c06d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.684+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 shutdown_connections 2024-05-30T17:04:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.684+0000 7fcfa4600640 1 -- 172.21.15.67:0/178073049 wait complete. 2024-05-30T17:04:32.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone93311 2024-05-30T17:04:32.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.933+0000 7fe3b7e00640 1 Processor -- start 2024-05-30T17:04:32.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.933+0000 7fe3b7e00640 1 -- start start 2024-05-30T17:04:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.935+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3b80ce560 0x7fe3b80b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.935+0000 7fe3b7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe3b815f0c0 0x7fe3b815f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.935+0000 7fe3b7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b815bb30 con 0x7fe3b80d0430 2024-05-30T17:04:32.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.935+0000 7fe3b7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b80b3ba0 con 0x7fe3b815f0c0 2024-05-30T17:04:32.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.935+0000 7fe3b7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b815bdd0 con 0x7fe3b80ce560 2024-05-30T17:04:32.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.935+0000 7fe3afe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe3b80d0430 0x7fe3b80d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42334/0 (socket says 172.21.15.67:42334) 2024-05-30T17:04:32.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.935+0000 7fe3afe00640 1 -- 172.21.15.67:0/3335996145 learned_addr learned my addr 172.21.15.67:0/3335996145 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:32.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b7400640 1 --2- 172.21.15.67:0/3335996145 >> v2:172.21.15.169:3300/0 conn(0x7fe3b815f0c0 0x7fe3b815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175286870 0 0) 0x7fe3b815bb30 con 0x7fe3b80d0430 2024-05-30T17:04:32.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe39c003520 con 0x7fe3b80d0430 2024-05-30T17:04:32.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6a00640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3b80ce560 0x7fe3b80b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2785716112 0 0) 0x7fe39c003520 con 0x7fe3b80d0430 2024-05-30T17:04:32.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3b80ce560 msgr2=0x7fe3b80b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:32.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3b80ce560 0x7fe3b80b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 >> v2:172.21.15.169:3300/0 conn(0x7fe3b815f0c0 msgr2=0x7fe3b815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 --2- 172.21.15.67:0/3335996145 >> v2:172.21.15.169:3300/0 conn(0x7fe3b815f0c0 0x7fe3b815f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b815c0b0 con 0x7fe3b80d0430 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fe3a0001670 con 0x7fe3b80d0430 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 >> v1:172.21.15.67:6789/0 conn(0x7fe3b80d0430 legacy=0x7fe3b80d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3b80ce560 0x7fe39c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe39c004d40 0x7fe39c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80b3ba0 con 0x7fe39c004d40 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b815bdd0 con 0x7fe39c0039b0 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe39c00baf0 con 0x7fe3b80ce560 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.936+0000 7fe3b7400640 1 --2- 172.21.15.67:0/3335996145 >> v2:172.21.15.169:3300/0 conn(0x7fe3b815f0c0 0x7fe3b815f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.937+0000 7fe3b7400640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe39c004d40 0x7fe39c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.937+0000 7fe3afe00640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3b80ce560 0x7fe39c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.937+0000 7fe3b6a00640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.937+0000 7fe3b7400640 1 -- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3b80ce560 msgr2=0x7fe39c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:32.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.937+0000 7fe3b7400640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3b80ce560 0x7fe39c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.937+0000 7fe3b7400640 1 -- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe39c0039b0 msgr2=0x7fe39c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:32.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.937+0000 7fe3b7400640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.937+0000 7fe3b7400640 1 -- 172.21.15.67:0/3335996145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c003520 con 0x7fe39c004d40 2024-05-30T17:04:32.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.937+0000 7fe3afe00640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3b80ce560 0x7fe39c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.938+0000 7fe3b7400640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe39c004d40 0x7fe39c0051c0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00073e0 tx=0x7fe39c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92944b38c15e4b60 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0009c70 con 0x7fe39c004d40 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b002c670 con 0x7fe39c004d40 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b6000640 1 -- 172.21.15.67:0/3335996145 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b002ca20 con 0x7fe39c004d40 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b7e00640 1 -- 172.21.15.67:0/3335996145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe39c004d40 msgr2=0x7fe39c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b7e00640 1 --2- 172.21.15.67:0/3335996145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe39c004d40 0x7fe39c0051c0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00073e0 tx=0x7fe39c005830 comp rx=0 tx=0).stop 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b7400640 1 -- 172.21.15.67:0/3335996145 reap_dead start 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b7e00640 1 -- 172.21.15.67:0/3335996145 shutdown_connections 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b7e00640 1 -- 172.21.15.67:0/3335996145 >> 172.21.15.67:0/3335996145 conn(0x7fe3b806bae0 msgr2=0x7fe3b806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:32.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b7e00640 1 -- 172.21.15.67:0/3335996145 shutdown_connections 2024-05-30T17:04:32.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.939+0000 7fe3b7e00640 1 -- 172.21.15.67:0/3335996145 wait complete. 2024-05-30T17:04:32.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.941+0000 7fe3b7e00640 1 Processor -- start 2024-05-30T17:04:32.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.941+0000 7fe3b7e00640 1 -- start start 2024-05-30T17:04:32.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.942+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe39c004d40 0x7fe3b8151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.942+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b8152c00 0x7fe3b815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.942+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3b8153080 0x7fe3b815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.942+0000 7fe3b7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b806f630 con 0x7fe3b8153080 2024-05-30T17:04:32.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.942+0000 7fe3b7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b815c670 con 0x7fe3b8152c00 2024-05-30T17:04:32.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.942+0000 7fe3b7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b806f7b0 con 0x7fe39c004d40 2024-05-30T17:04:32.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.942+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3b8153080 0x7fe3b815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.942+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3b8153080 0x7fe3b815a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52134/0 (socket says 172.21.15.67:52134) 2024-05-30T17:04:32.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.942+0000 7fe3b7400640 1 -- 172.21.15.67:0/1653551287 learned_addr learned my addr 172.21.15.67:0/1653551287 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3b6a00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe39c004d40 0x7fe3b8151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3afe00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b8152c00 0x7fe3b815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3b7400640 1 -- 172.21.15.67:0/1653551287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe39c004d40 msgr2=0x7fe3b8151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3b7400640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe39c004d40 0x7fe3b8151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3b7400640 1 -- 172.21.15.67:0/1653551287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b8152c00 msgr2=0x7fe3b815a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3b7400640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b8152c00 0x7fe3b815a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3b7400640 1 -- 172.21.15.67:0/1653551287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b8154ab0 con 0x7fe3b8153080 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3afe00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b8152c00 0x7fe3b815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3b7400640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3b8153080 0x7fe3b815a540 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00074d0 tx=0x7fe3b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3aea00640 1 -- 172.21.15.67:0/1653551287 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0009c70 con 0x7fe3b8153080 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3aea00640 1 -- 172.21.15.67:0/1653551287 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b002e950 con 0x7fe3b8153080 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.943+0000 7fe3aea00640 1 -- 172.21.15.67:0/1653551287 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b002ed20 con 0x7fe3b8153080 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.944+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b8154d90 con 0x7fe3b8153080 2024-05-30T17:04:32.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.944+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b8155350 con 0x7fe3b8153080 2024-05-30T17:04:32.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.945+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b8155730 con 0x7fe3b8153080 2024-05-30T17:04:32.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.946+0000 7fe3aea00640 1 -- 172.21.15.67:0/1653551287 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7fe3b0041420 con 0x7fe3b8153080 2024-05-30T17:04:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.947+0000 7fe3aea00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe39406e3f0 0x7fe3940708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.947+0000 7fe3aea00640 1 -- 172.21.15.67:0/1653551287 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5935+0+0 (secure 0 0 0) 0x7fe3b00ab470 con 0x7fe3b8153080 2024-05-30T17:04:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.948+0000 7fe3b6a00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe39406e3f0 0x7fe3940708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:32.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.948+0000 7fe3b6a00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe39406e3f0 0x7fe3940708b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8006fd0 tx=0x7fe3a8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:32.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:32.952+0000 7fe3aea00640 1 -- 172.21.15.67:0/1653551287 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3b8155730 con 0x7fe3b8153080 2024-05-30T17:04:33.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.192+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone93311", "target": ["mon-mgr", ""]}) -- 0x7fe3b815e1d0 con 0x7fe39406e3f0 2024-05-30T17:04:33.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.241+0000 7fe3aea00640 1 -- 172.21.15.67:0/1653551287 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe3b815e1d0 con 0x7fe39406e3f0 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe39406e3f0 msgr2=0x7fe3940708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe39406e3f0 0x7fe3940708b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8006fd0 tx=0x7fe3a8008040 comp rx=0 tx=0).stop 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3b8153080 msgr2=0x7fe3b815a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3b8153080 0x7fe3b815a540 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00074d0 tx=0x7fe3b0004ab0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 shutdown_connections 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe39406e3f0 0x7fe3940708b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3b8153080 0x7fe3b815a540 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b8152c00 0x7fe3b815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 --2- 172.21.15.67:0/1653551287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe39c004d40 0x7fe3b8151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.244+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 >> 172.21.15.67:0/1653551287 conn(0x7fe3b806bae0 msgr2=0x7fe3b815cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.245+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 shutdown_connections 2024-05-30T17:04:33.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.245+0000 7fe3b7e00640 1 -- 172.21.15.67:0/1653551287 wait complete. 2024-05-30T17:04:33.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone93312 2024-05-30T17:04:33.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.460+0000 7efd43e00640 1 Processor -- start 2024-05-30T17:04:33.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.460+0000 7efd43e00640 1 -- start start 2024-05-30T17:04:33.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3c0cf2f0 0x7efd3c0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:33.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd43e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd3c06edf0 0x7efd3c13fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:33.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd43e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd3c06dd50 con 0x7efd3c06f250 2024-05-30T17:04:33.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd43e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7efd3c06e030 con 0x7efd3c06edf0 2024-05-30T17:04:33.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd43e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd3c06e310 con 0x7efd3c0cf2f0 2024-05-30T17:04:33.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd40a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd3c06edf0 0x7efd3c13fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:33.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd40a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd3c06edf0 0x7efd3c13fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52150/0 (socket says 172.21.15.67:52150) 2024-05-30T17:04:33.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd40a00640 1 -- 172.21.15.67:0/743539163 learned_addr learned my addr 172.21.15.67:0/743539163 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:33.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd41e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efd3c06f250 0x7efd3c0ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42348/0 (socket says 172.21.15.67:42348) 2024-05-30T17:04:33.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.462+0000 7efd41400640 1 --2- 172.21.15.67:0/743539163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3c0cf2f0 0x7efd3c0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:33.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.463+0000 7efd40a00640 1 -- 172.21.15.67:0/743539163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3c0cf2f0 msgr2=0x7efd3c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:33.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.463+0000 7efd40a00640 1 --2- 172.21.15.67:0/743539163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3c0cf2f0 0x7efd3c0cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.463+0000 7efd40a00640 1 -- 172.21.15.67:0/743539163 >> v1:172.21.15.67:6789/0 conn(0x7efd3c06f250 legacy=0x7efd3c0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:33.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.463+0000 7efd3be00640 1 -- 172.21.15.67:0/743539163 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2181831770 0 0) 0x7efd3c06dd50 con 0x7efd3c06f250 2024-05-30T17:04:33.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.463+0000 7efd40a00640 1 -- 172.21.15.67:0/743539163 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c06e5f0 con 0x7efd3c06edf0 2024-05-30T17:04:33.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.464+0000 7efd40a00640 1 --2- 172.21.15.67:0/743539163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 0x7efd3c13fdf0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7efd28009870 tx=0x7efd2802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42c08cf5d40249b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.464+0000 7efd3be00640 1 -- 172.21.15.67:0/743539163 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28037070 con 0x7efd3c06edf0 2024-05-30T17:04:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.464+0000 7efd3be00640 1 -- 172.21.15.67:0/743539163 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd28032440 con 0x7efd3c06edf0 2024-05-30T17:04:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.464+0000 7efd3be00640 1 -- 172.21.15.67:0/743539163 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28032810 con 0x7efd3c06edf0 2024-05-30T17:04:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.465+0000 7efd43e00640 1 -- 172.21.15.67:0/743539163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 msgr2=0x7efd3c13fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.465+0000 7efd43e00640 1 --2- 172.21.15.67:0/743539163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 0x7efd3c13fdf0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7efd28009870 tx=0x7efd2802a1c0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.465+0000 7efd43e00640 1 -- 172.21.15.67:0/743539163 shutdown_connections 2024-05-30T17:04:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.465+0000 7efd43e00640 1 --2- 172.21.15.67:0/743539163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 0x7efd3c13fdf0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.465+0000 7efd43e00640 1 --2- 172.21.15.67:0/743539163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3c0cf2f0 0x7efd3c0cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.465+0000 7efd43e00640 1 -- 172.21.15.67:0/743539163 >> 172.21.15.67:0/743539163 conn(0x7efd3c0b1e50 msgr2=0x7efd3c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.465+0000 7efd43e00640 1 -- 172.21.15.67:0/743539163 shutdown_connections 2024-05-30T17:04:33.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.465+0000 7efd43e00640 1 -- 172.21.15.67:0/743539163 wait complete. 2024-05-30T17:04:33.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.467+0000 7efd43e00640 1 Processor -- start 2024-05-30T17:04:33.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.467+0000 7efd43e00640 1 -- start start 2024-05-30T17:04:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.468+0000 7efd43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 0x7efd3c1f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.468+0000 7efd43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3c06f250 0x7efd3c1f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.468+0000 7efd43e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3c0cf2f0 0x7efd3c0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.468+0000 7efd43e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c1461a0 con 0x7efd3c06edf0 2024-05-30T17:04:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.468+0000 7efd43e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c146020 con 0x7efd3c0cf2f0 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.468+0000 7efd43e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd3c0cfe50 con 0x7efd3c06f250 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.468+0000 7efd41400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 0x7efd3c1f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.468+0000 7efd41400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 0x7efd3c1f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52166/0 (socket says 172.21.15.67:52166) 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.468+0000 7efd41400640 1 -- 172.21.15.67:0/621929294 learned_addr learned my addr 172.21.15.67:0/621929294 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd41400640 1 -- 172.21.15.67:0/621929294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3c06f250 msgr2=0x7efd3c1f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd41400640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3c06f250 0x7efd3c1f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd41400640 1 -- 172.21.15.67:0/621929294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3c0cf2f0 msgr2=0x7efd3c0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd41400640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3c0cf2f0 0x7efd3c0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd41400640 1 -- 172.21.15.67:0/621929294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c0cafc0 con 0x7efd3c06edf0 2024-05-30T17:04:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd41400640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 0x7efd3c1f5610 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7efd3000d800 tx=0x7efd3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:33.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd39600640 1 -- 172.21.15.67:0/621929294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3000b6c0 con 0x7efd3c06edf0 2024-05-30T17:04:33.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd39600640 1 -- 172.21.15.67:0/621929294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30019070 con 0x7efd3c06edf0 2024-05-30T17:04:33.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd39600640 1 -- 172.21.15.67:0/621929294 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3000ba80 con 0x7efd3c06edf0 2024-05-30T17:04:33.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.469+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c0c99b0 con 0x7efd3c06edf0 2024-05-30T17:04:33.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.470+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c15f540 con 0x7efd3c06edf0 2024-05-30T17:04:33.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.471+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c0689e0 con 0x7efd3c06edf0 2024-05-30T17:04:33.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.472+0000 7efd39600640 1 -- 172.21.15.67:0/621929294 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 42) ==== 93488+0+0 (secure 0 0 0) 0x7efd3000bc20 con 0x7efd3c06edf0 2024-05-30T17:04:33.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.472+0000 7efd39600640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd1c06e3f0 0x7efd1c0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:33.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.473+0000 7efd39600640 1 -- 172.21.15.67:0/621929294 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5935+0+0 (secure 0 0 0) 0x7efd3008af70 con 0x7efd3c06edf0 2024-05-30T17:04:33.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.473+0000 7efd40a00640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd1c06e3f0 0x7efd1c0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:33.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.473+0000 7efd40a00640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd1c06e3f0 0x7efd1c0708b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7efd28034000 tx=0x7efd2802c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:33.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.477+0000 7efd39600640 1 -- 172.21.15.67:0/621929294 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd3c0cafc0 con 0x7efd3c06edf0 2024-05-30T17:04:33.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.710+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone93312", "target": ["mon-mgr", ""]}) -- 0x7efd3c073f30 con 0x7efd1c06e3f0 2024-05-30T17:04:33.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.756+0000 7efd39600640 1 -- 172.21.15.67:0/621929294 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7efd30056d70 con 0x7efd3c06edf0 2024-05-30T17:04:33.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.767+0000 7efd39600640 1 -- 172.21.15.67:0/621929294 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd3c073f30 con 0x7efd1c06e3f0 2024-05-30T17:04:33.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.769+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd1c06e3f0 msgr2=0x7efd1c0708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:33.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.769+0000 7efd43e00640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd1c06e3f0 0x7efd1c0708b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7efd28034000 tx=0x7efd2802c000 comp rx=0 tx=0).stop 2024-05-30T17:04:33.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.769+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 msgr2=0x7efd3c1f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.769+0000 7efd43e00640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 0x7efd3c1f5610 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7efd3000d800 tx=0x7efd3000dcc0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.770+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 shutdown_connections 2024-05-30T17:04:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.770+0000 7efd43e00640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efd1c06e3f0 0x7efd1c0708b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.770+0000 7efd43e00640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd3c0cf2f0 0x7efd3c0b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.770+0000 7efd43e00640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3c06f250 0x7efd3c1f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.770+0000 7efd43e00640 1 --2- 172.21.15.67:0/621929294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd3c06edf0 0x7efd3c1f5610 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.770+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 >> 172.21.15.67:0/621929294 conn(0x7efd3c0b1e50 msgr2=0x7efd3c154c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.770+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 shutdown_connections 2024-05-30T17:04:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33.770+0000 7efd43e00640 1 -- 172.21.15.67:0/621929294 wait complete. 2024-05-30T17:04:33.796 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-30T17:04:33.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-30T17:04:33.923 INFO:teuthology.orchestra.run.smithi067.stdout:2 2024-05-30T17:04:33.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T17:04:33.933 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T17:04:33.933 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:04:34.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:04:34.145 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T17:04:34.145 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:04:34.145 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:04:34.145 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:04:34.208 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:04:34.208 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:04:34.269 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T17:04:34.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.498+0000 7f3060c00640 1 Processor -- start 2024-05-30T17:04:34.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.498+0000 7f3060c00640 1 -- start start 2024-05-30T17:04:34.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.499+0000 7f3060c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f305c0d0de0 0x7f305c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:34.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.499+0000 7f3060c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f305c15cbc0 con 0x7f305c0d0210 2024-05-30T17:04:34.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.499+0000 7f3060c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f305c1605a0 con 0x7f305c15c7e0 2024-05-30T17:04:34.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.499+0000 7f3060c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f305c0b3cd0 con 0x7f305c0d0de0 2024-05-30T17:04:34.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.499+0000 7f305a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f305c0d0210 0x7f305c0cd7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42358/0 (socket says 172.21.15.67:42358) 2024-05-30T17:04:34.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.499+0000 7f305a800640 1 -- 172.21.15.67:0/274301706 learned_addr learned my addr 172.21.15.67:0/274301706 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3059400640 1 --2- 172.21.15.67:0/274301706 >> v2:172.21.15.169:3300/0 conn(0x7f305c0d0de0 0x7f305c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834461934 0 0) 0x7f305c15cbc0 con 0x7f305c0d0210 2024-05-30T17:04:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3040003520 con 0x7f305c0d0210 2024-05-30T17:04:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3500562444 0 0) 0x7f3040003520 con 0x7f305c0d0210 2024-05-30T17:04:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 >> v2:172.21.15.169:3300/0 conn(0x7f305c0d0de0 msgr2=0x7f305c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 --2- 172.21.15.67:0/274301706 >> v2:172.21.15.169:3300/0 conn(0x7f305c0d0de0 0x7f305c0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 >> v1:172.21.15.169:6789/0 conn(0x7f305c15c7e0 legacy=0x7f305c0d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c160880 con 0x7f305c0d0210 2024-05-30T17:04:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f30540019f0 con 0x7f305c0d0210 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 >> v1:172.21.15.67:6789/0 conn(0x7f305c0d0210 legacy=0x7f305c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30400039b0 0x7f3040003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.500+0000 7f3058a00640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3040004800 0x7f3040004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f3058a00640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f304000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f304000c180 con 0x7f30400052e0 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f304000c330 con 0x7f30400039b0 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f304000c530 con 0x7f3040004800 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f305a800640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f304000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f3059400640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3040004800 0x7f3040004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f3059e00640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30400039b0 0x7f3040003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f305a800640 1 -- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3040004800 msgr2=0x7f3040004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f305a800640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3040004800 0x7f3040004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f305a800640 1 -- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30400039b0 msgr2=0x7f3040003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f305a800640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30400039b0 0x7f3040003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f305a800640 1 -- 172.21.15.67:0/274301706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3040003520 con 0x7f30400052e0 2024-05-30T17:04:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f3059400640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3040004800 0x7f3040004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.501+0000 7f3059e00640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30400039b0 0x7f3040003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.502+0000 7f305a800640 1 -- 172.21.15.67:0/274301706 reap_dead start 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.502+0000 7f305a800640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f304000b9c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f305c0d0680 tx=0x7f3040005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ed7b1168a1fb40 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.502+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305400a460 con 0x7f30400052e0 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.503+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f305402f070 con 0x7f30400052e0 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.503+0000 7f3058a00640 1 -- 172.21.15.67:0/274301706 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305402bc20 con 0x7f30400052e0 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.503+0000 7f3060c00640 1 -- 172.21.15.67:0/274301706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 msgr2=0x7f304000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.503+0000 7f3060c00640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f304000b9c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f305c0d0680 tx=0x7f3040005830 comp rx=0 tx=0).stop 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.503+0000 7f3060c00640 1 -- 172.21.15.67:0/274301706 shutdown_connections 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.503+0000 7f3060c00640 1 --2- 172.21.15.67:0/274301706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f304000b9c0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.503+0000 7f3060c00640 1 -- 172.21.15.67:0/274301706 >> 172.21.15.67:0/274301706 conn(0x7f305c0b2f40 msgr2=0x7f305c0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:34.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.503+0000 7f3060c00640 1 -- 172.21.15.67:0/274301706 shutdown_connections 2024-05-30T17:04:34.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.503+0000 7f3060c00640 1 -- 172.21.15.67:0/274301706 wait complete. 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.505+0000 7f3060c00640 1 Processor -- start 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.505+0000 7f3060c00640 1 -- start start 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.506+0000 7f3060c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f305c0c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.506+0000 7f3060c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f305c15fc00 0x7f305c15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.506+0000 7f3060c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c15d2f0 0x7f305c14ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.506+0000 7f3060c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f305c160e40 con 0x7f30400052e0 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.506+0000 7f3060c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f305c160350 con 0x7f305c15d2f0 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.506+0000 7f3060c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f305c06b900 con 0x7f305c15fc00 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.506+0000 7f3059e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f305c0c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.506+0000 7f3059e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f305c0c5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52194/0 (socket says 172.21.15.67:52194) 2024-05-30T17:04:34.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.506+0000 7f3059e00640 1 -- 172.21.15.67:0/397653754 learned_addr learned my addr 172.21.15.67:0/397653754 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:34.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3059e00640 1 -- 172.21.15.67:0/397653754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f305c15fc00 msgr2=0x7f305c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:34.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3059e00640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f305c15fc00 0x7f305c15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3059e00640 1 -- 172.21.15.67:0/397653754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c15d2f0 msgr2=0x7f305c14ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:34.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3059e00640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c15d2f0 0x7f305c14ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3059e00640 1 -- 172.21.15.67:0/397653754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c0c65d0 con 0x7f30400052e0 2024-05-30T17:04:34.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3059e00640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f305c0c5f10 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f304c00d770 tx=0x7f304c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:34.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3052000640 1 -- 172.21.15.67:0/397653754 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c005270 con 0x7f30400052e0 2024-05-30T17:04:34.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3052000640 1 -- 172.21.15.67:0/397653754 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304c017070 con 0x7f30400052e0 2024-05-30T17:04:34.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3052000640 1 -- 172.21.15.67:0/397653754 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c00b650 con 0x7f30400052e0 2024-05-30T17:04:34.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.507+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c0c68b0 con 0x7f30400052e0 2024-05-30T17:04:34.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.508+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c0c6e70 con 0x7f30400052e0 2024-05-30T17:04:34.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.509+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f305c15d710 con 0x7f30400052e0 2024-05-30T17:04:34.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.510+0000 7f3052000640 1 -- 172.21.15.67:0/397653754 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f304c00b7f0 con 0x7f30400052e0 2024-05-30T17:04:34.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.511+0000 7f3052000640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f303c06e540 0x7f303c070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:34.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.511+0000 7f3052000640 1 -- 172.21.15.67:0/397653754 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5935+0+0 (secure 0 0 0) 0x7f304c08a990 con 0x7f30400052e0 2024-05-30T17:04:34.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.511+0000 7f3059400640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f303c06e540 0x7f303c070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:34.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.512+0000 7f3059400640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f303c06e540 0x7f303c070a00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f305c0c7bf0 tx=0x7f3048009350 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:34.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.515+0000 7f3052000640 1 -- 172.21.15.67:0/397653754 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f304c05a830 con 0x7f30400052e0 2024-05-30T17:04:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.730+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f305c0690f0 con 0x7f30400052e0 2024-05-30T17:04:34.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.731+0000 7f3052000640 1 -- 172.21.15.67:0/397653754 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7f304c01ae40 con 0x7f30400052e0 2024-05-30T17:04:34.732 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T17:04:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.733+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f303c06e540 msgr2=0x7f303c070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.733+0000 7f3060c00640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f303c06e540 0x7f303c070a00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f305c0c7bf0 tx=0x7f3048009350 comp rx=0 tx=0).stop 2024-05-30T17:04:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.733+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 msgr2=0x7f305c0c5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:34.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.733+0000 7f3060c00640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f305c0c5f10 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f304c00d770 tx=0x7f304c00dc30 comp rx=0 tx=0).stop 2024-05-30T17:04:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.734+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 shutdown_connections 2024-05-30T17:04:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.734+0000 7f3060c00640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f303c06e540 0x7f303c070a00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.734+0000 7f3060c00640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c15d2f0 0x7f305c14ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.734+0000 7f3060c00640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f305c15fc00 0x7f305c15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.734+0000 7f3060c00640 1 --2- 172.21.15.67:0/397653754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30400052e0 0x7f305c0c5f10 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.734+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 >> 172.21.15.67:0/397653754 conn(0x7f305c0b2f40 msgr2=0x7f305c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.734+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 shutdown_connections 2024-05-30T17:04:34.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:34.734+0000 7f3060c00640 1 -- 172.21.15.67:0/397653754 wait complete. 2024-05-30T17:04:34.752 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T17:04:34.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:04:34.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-30T17:04:34.842 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T17:04:34.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:04:34.843 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-30T17:04:34.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[74543]: fuse finished with error 0 and tester_r 0 2024-05-30T17:04:35.027 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:04:35.027 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:04:35.028 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:04:35.072 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:04:35.072 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:04:35.072 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:04:35.072 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:04:35.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:04:35.127 INFO:tasks.cephfs.mount:Terminating background process 2024-05-30T17:04:35.128 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:04:35.128 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:04:35.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:04:35.183 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:04:35.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.462+0000 7f6d6b400640 1 Processor -- start 2024-05-30T17:04:35.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.462+0000 7f6d6b400640 1 -- start start 2024-05-30T17:04:35.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.464+0000 7f6d6b400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d6406f860 0x7f6d640b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:35.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.464+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d640d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.464+0000 7f6d6b400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d6406ed10 con 0x7f6d640d09e0 2024-05-30T17:04:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.464+0000 7f6d6b400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6d640d0e50 con 0x7f6d6406f860 2024-05-30T17:04:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.464+0000 7f6d6b400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6d640cfe40 con 0x7f6d640b3690 2024-05-30T17:04:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.465+0000 7f6d63e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d640d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.465+0000 7f6d63e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d640d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:54088/0 (socket says 172.21.15.67:54088) 2024-05-30T17:04:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.465+0000 7f6d63e00640 1 -- 172.21.15.67:0/3756195114 learned_addr learned my addr 172.21.15.67:0/3756195114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.465+0000 7f6d69400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f6d640d09e0 0x7f6d64160210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:37344/0 (socket says 172.21.15.67:37344) 2024-05-30T17:04:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.466+0000 7f6d63e00640 1 -- 172.21.15.67:0/3756195114 >> v2:172.21.15.169:3300/0 conn(0x7f6d6406f860 msgr2=0x7f6d640b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.467+0000 7f6d68a00640 1 --2- 172.21.15.67:0/3756195114 >> v2:172.21.15.169:3300/0 conn(0x7f6d6406f860 0x7f6d640b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.467+0000 7f6d63e00640 1 --2- 172.21.15.67:0/3756195114 >> v2:172.21.15.169:3300/0 conn(0x7f6d6406f860 0x7f6d640b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.467+0000 7f6d63e00640 1 -- 172.21.15.67:0/3756195114 >> v1:172.21.15.169:6789/0 conn(0x7f6d640d09e0 legacy=0x7f6d64160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.467+0000 7f6d63400640 1 -- 172.21.15.67:0/3756195114 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352704663 0 0) 0x7f6d5c001490 con 0x7f6d640d09e0 2024-05-30T17:04:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.467+0000 7f6d63e00640 1 -- 172.21.15.67:0/3756195114 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6415cb00 con 0x7f6d640b3690 2024-05-30T17:04:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.467+0000 7f6d68a00640 1 --2- 172.21.15.67:0/3756195114 >> v2:172.21.15.169:3300/0 conn(0x7f6d6406f860 0x7f6d640b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:35.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.469+0000 7f6d63e00640 1 --2- 172.21.15.67:0/3756195114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d640d04a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009870 tx=0x7f6d5002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a2e76c03833e630 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:35.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.470+0000 7f6d63400640 1 -- 172.21.15.67:0/3756195114 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50037070 con 0x7f6d640b3690 2024-05-30T17:04:35.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.470+0000 7f6d63400640 1 -- 172.21.15.67:0/3756195114 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d50032440 con 0x7f6d640b3690 2024-05-30T17:04:35.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.470+0000 7f6d63400640 1 -- 172.21.15.67:0/3756195114 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50037070 con 0x7f6d640b3690 2024-05-30T17:04:35.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.471+0000 7f6d6b400640 1 -- 172.21.15.67:0/3756195114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 msgr2=0x7f6d640d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:35.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.472+0000 7f6d6b400640 1 --2- 172.21.15.67:0/3756195114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d640d04a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009870 tx=0x7f6d5002a250 comp rx=0 tx=0).stop 2024-05-30T17:04:35.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.473+0000 7f6d6b400640 1 -- 172.21.15.67:0/3756195114 shutdown_connections 2024-05-30T17:04:35.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.473+0000 7f6d6b400640 1 --2- 172.21.15.67:0/3756195114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d640d04a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.473+0000 7f6d6b400640 1 --2- 172.21.15.67:0/3756195114 >> v2:172.21.15.169:3300/0 conn(0x7f6d6406f860 0x7f6d640b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.473+0000 7f6d6b400640 1 -- 172.21.15.67:0/3756195114 >> 172.21.15.67:0/3756195114 conn(0x7f6d6406bae0 msgr2=0x7f6d6406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:35.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.474+0000 7f6d6b400640 1 -- 172.21.15.67:0/3756195114 shutdown_connections 2024-05-30T17:04:35.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.474+0000 7f6d6b400640 1 -- 172.21.15.67:0/3756195114 wait complete. 2024-05-30T17:04:35.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.478+0000 7f6d6b400640 1 Processor -- start 2024-05-30T17:04:35.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.478+0000 7f6d6b400640 1 -- start start 2024-05-30T17:04:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.480+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d6406f860 0x7f6d640c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.481+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d6415e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.481+0000 7f6d6b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d640d09e0 0x7f6d640c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.481+0000 7f6d6b400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6d640b4280 con 0x7f6d6406f860 2024-05-30T17:04:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.482+0000 7f6d68a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d6406f860 0x7f6d640c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:35.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.482+0000 7f6d68a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d6406f860 0x7f6d640c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52208/0 (socket says 172.21.15.67:52208) 2024-05-30T17:04:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.482+0000 7f6d68a00640 1 -- 172.21.15.67:0/63825404 learned_addr learned my addr 172.21.15.67:0/63825404 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.482+0000 7f6d69400640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d640d09e0 0x7f6d640c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.482+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d640b3fa0 con 0x7f6d640d09e0 2024-05-30T17:04:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.482+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6d640b4560 con 0x7f6d640b3690 2024-05-30T17:04:35.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.482+0000 7f6d63e00640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d6415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.484+0000 7f6d63e00640 1 -- 172.21.15.67:0/63825404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d640d09e0 msgr2=0x7f6d640c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.484+0000 7f6d63e00640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d640d09e0 0x7f6d640c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.484+0000 7f6d63e00640 1 -- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d6406f860 msgr2=0x7f6d640c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.484+0000 7f6d63e00640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d6406f860 0x7f6d640c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.484+0000 7f6d63e00640 1 -- 172.21.15.67:0/63825404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d640c4410 con 0x7f6d640b3690 2024-05-30T17:04:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.484+0000 7f6d68a00640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d6406f860 0x7f6d640c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:04:35.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.485+0000 7f6d63e00640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d6415e540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d50031000 tx=0x7f6d500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:35.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.486+0000 7f6d60c00640 1 -- 172.21.15.67:0/63825404 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50042070 con 0x7f6d640b3690 2024-05-30T17:04:35.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.486+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d640c5e90 con 0x7f6d640b3690 2024-05-30T17:04:35.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.486+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d640c6450 con 0x7f6d640b3690 2024-05-30T17:04:35.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.487+0000 7f6d60c00640 1 -- 172.21.15.67:0/63825404 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d50004020 con 0x7f6d640b3690 2024-05-30T17:04:35.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.487+0000 7f6d60c00640 1 -- 172.21.15.67:0/63825404 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50037070 con 0x7f6d640b3690 2024-05-30T17:04:35.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.488+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d40005290 con 0x7f6d640b3690 2024-05-30T17:04:35.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.491+0000 7f6d60c00640 1 -- 172.21.15.67:0/63825404 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f6d50030710 con 0x7f6d640b3690 2024-05-30T17:04:35.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.493+0000 7f6d60c00640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d4406e510 0x7f6d440709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:35.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.493+0000 7f6d60c00640 1 -- 172.21.15.67:0/63825404 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(119..119 src has 1..119) ==== 5935+0+0 (secure 0 0 0) 0x7f6d500ab7f0 con 0x7f6d640b3690 2024-05-30T17:04:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.495+0000 7f6d68a00640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d4406e510 0x7f6d440709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:35.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.496+0000 7f6d68a00640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d4406e510 0x7f6d440709d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800f240 tx=0x7f6d58008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.501+0000 7f6d60c00640 1 -- 172.21.15.67:0/63825404 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d640c4410 con 0x7f6d640b3690 2024-05-30T17:04:35.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.946+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d40005580 con 0x7f6d640b3690 2024-05-30T17:04:35.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.948+0000 7f6d60c00640 1 -- 172.21.15.67:0/63825404 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5189 (secure 0 0 0) 0x7f6d5007b1d0 con 0x7f6d640b3690 2024-05-30T17:04:35.950 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:35.950 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":74,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1973461214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1973461214},{"type":"v1","addr":"172.21.15.169:6835","nonce":1973461214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":8400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3884958123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3884958123},{"type":"v1","addr":"172.21.15.67:6836","nonce":3884958123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":8403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1126347083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1126347083},{"type":"v1","addr":"172.21.15.67:6837","nonce":1126347083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:02:40.462403+0000","modified":"2024-05-30T17:02:43.518891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3425947383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3425947383},{"type":"v1","addr":"172.21.15.169:6834","nonce":3425947383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":8}]} 2024-05-30T17:04:35.950 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 74 2024-05-30T17:04:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d4406e510 msgr2=0x7f6d440709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d4406e510 0x7f6d440709d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800f240 tx=0x7f6d58008040 comp rx=0 tx=0).stop 2024-05-30T17:04:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 msgr2=0x7f6d6415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d6415e540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d50031000 tx=0x7f6d500046c0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 shutdown_connections 2024-05-30T17:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f6d4406e510 0x7f6d440709d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d640d09e0 0x7f6d640c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d640b3690 0x7f6d6415e540 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 --2- 172.21.15.67:0/63825404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d6406f860 0x7f6d640c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:35.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 >> 172.21.15.67:0/63825404 conn(0x7f6d6406bae0 msgr2=0x7f6d6415c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 shutdown_connections 2024-05-30T17:04:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:35.952+0000 7f6d6b400640 1 -- 172.21.15.67:0/63825404 wait complete. 2024-05-30T17:04:35.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:04:36.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.150+0000 7fb4dfe00640 1 Processor -- start 2024-05-30T17:04:36.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.150+0000 7fb4dfe00640 1 -- start start 2024-05-30T17:04:36.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.151+0000 7fb4dfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4e0071bc0 0x7fb4e0072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.151+0000 7fb4dfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4e00b32c0 0x7fb4e00b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.151+0000 7fb4dfe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4e006e6e0 con 0x7fb4e0158b80 2024-05-30T17:04:36.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.151+0000 7fb4dfe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e006e9c0 con 0x7fb4e00b32c0 2024-05-30T17:04:36.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.151+0000 7fb4dfe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e006eca0 con 0x7fb4e0071bc0 2024-05-30T17:04:36.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.151+0000 7fb4df400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4e00b32c0 0x7fb4e00b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:36.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4dea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb4e0158b80 0x7fb4e006dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42372/0 (socket says 172.21.15.67:42372) 2024-05-30T17:04:36.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4dea00640 1 -- 172.21.15.67:0/2179043415 learned_addr learned my addr 172.21.15.67:0/2179043415 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:36.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4df400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4e00b32c0 0x7fb4e00b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52210/0 (socket says 172.21.15.67:52210) 2024-05-30T17:04:36.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4d7e00640 1 --2- 172.21.15.67:0/2179043415 >> v2:172.21.15.169:3300/0 conn(0x7fb4e0071bc0 0x7fb4e0072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:36.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4df400640 1 -- 172.21.15.67:0/2179043415 >> v2:172.21.15.169:3300/0 conn(0x7fb4e0071bc0 msgr2=0x7fb4e0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:36.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4df400640 1 --2- 172.21.15.67:0/2179043415 >> v2:172.21.15.169:3300/0 conn(0x7fb4e0071bc0 0x7fb4e0072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4df400640 1 -- 172.21.15.67:0/2179043415 >> v1:172.21.15.67:6789/0 conn(0x7fb4e0158b80 legacy=0x7fb4e006dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4df400640 1 -- 172.21.15.67:0/2179043415 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e006ef80 con 0x7fb4e00b32c0 2024-05-30T17:04:36.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4d7e00640 1 --2- 172.21.15.67:0/2179043415 >> v2:172.21.15.169:3300/0 conn(0x7fb4e0071bc0 0x7fb4e0072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:36.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.152+0000 7fb4de000640 1 -- 172.21.15.67:0/2179043415 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3767698640 0 0) 0x7fb4e006e6e0 con 0x7fb4e0158b80 2024-05-30T17:04:36.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.153+0000 7fb4df400640 1 --2- 172.21.15.67:0/2179043415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e00b32c0 0x7fb4e00b36a0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0162130 tx=0x7fb4d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902d2cc54afa6106 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:36.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.153+0000 7fb4de000640 1 -- 172.21.15.67:0/2179043415 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8007740 con 0x7fb4e00b32c0 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.153+0000 7fb4de000640 1 -- 172.21.15.67:0/2179043415 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d80078e0 con 0x7fb4e00b32c0 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.153+0000 7fb4de000640 1 -- 172.21.15.67:0/2179043415 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8007cb0 con 0x7fb4e00b32c0 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.154+0000 7fb4dfe00640 1 -- 172.21.15.67:0/2179043415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e00b32c0 msgr2=0x7fb4e00b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.154+0000 7fb4dfe00640 1 --2- 172.21.15.67:0/2179043415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e00b32c0 0x7fb4e00b36a0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0162130 tx=0x7fb4d802a680 comp rx=0 tx=0).stop 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.154+0000 7fb4dfe00640 1 -- 172.21.15.67:0/2179043415 shutdown_connections 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.154+0000 7fb4dfe00640 1 --2- 172.21.15.67:0/2179043415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e00b32c0 0x7fb4e00b36a0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.154+0000 7fb4dfe00640 1 --2- 172.21.15.67:0/2179043415 >> v2:172.21.15.169:3300/0 conn(0x7fb4e0071bc0 0x7fb4e0072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.154+0000 7fb4dfe00640 1 -- 172.21.15.67:0/2179043415 >> 172.21.15.67:0/2179043415 conn(0x7fb4e00b1fc0 msgr2=0x7fb4e00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.154+0000 7fb4dfe00640 1 -- 172.21.15.67:0/2179043415 shutdown_connections 2024-05-30T17:04:36.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.154+0000 7fb4dfe00640 1 -- 172.21.15.67:0/2179043415 wait complete. 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.156+0000 7fb4dfe00640 1 Processor -- start 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.156+0000 7fb4dfe00640 1 -- start start 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.157+0000 7fb4dfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4e0071bc0 0x7fb4e01f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.157+0000 7fb4dfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4e00b32c0 0x7fb4e01f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.157+0000 7fb4dfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e0158b80 0x7fb4e01f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.157+0000 7fb4dfe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e006f780 con 0x7fb4e0158b80 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.157+0000 7fb4dfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e00b2d30 con 0x7fb4e00b32c0 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.157+0000 7fb4dfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e01f4d50 con 0x7fb4e0071bc0 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.157+0000 7fb4df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e0158b80 0x7fb4e01f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.157+0000 7fb4df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e0158b80 0x7fb4e01f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52228/0 (socket says 172.21.15.67:52228) 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.157+0000 7fb4df400640 1 -- 172.21.15.67:0/240490852 learned_addr learned my addr 172.21.15.67:0/240490852 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4df400640 1 -- 172.21.15.67:0/240490852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4e0071bc0 msgr2=0x7fb4e01f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4df400640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4e0071bc0 0x7fb4e01f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4df400640 1 -- 172.21.15.67:0/240490852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4e00b32c0 msgr2=0x7fb4e01f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4d7e00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4e00b32c0 0x7fb4e01f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4df400640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4e00b32c0 0x7fb4e01f3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4df400640 1 -- 172.21.15.67:0/240490852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e00b1750 con 0x7fb4e0158b80 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4df400640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e0158b80 0x7fb4e01f4f10 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009e50 tx=0x7fb4d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4d6a00640 1 -- 172.21.15.67:0/240490852 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8040070 con 0x7fb4e0158b80 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4d6a00640 1 -- 172.21.15.67:0/240490852 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d8031bf0 con 0x7fb4e0158b80 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4d6a00640 1 -- 172.21.15.67:0/240490852 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8045460 con 0x7fb4e0158b80 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e01f5450 con 0x7fb4e0158b80 2024-05-30T17:04:36.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.158+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e014f7b0 con 0x7fb4e0158b80 2024-05-30T17:04:36.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.160+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e00689e0 con 0x7fb4e0158b80 2024-05-30T17:04:36.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.161+0000 7fb4d6a00640 1 -- 172.21.15.67:0/240490852 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fb4d8039040 con 0x7fb4e0158b80 2024-05-30T17:04:36.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.161+0000 7fb4d6a00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4bc06e510 0x7fb4bc0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.162+0000 7fb4d6a00640 1 -- 172.21.15.67:0/240490852 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5935+0+0 (secure 0 0 0) 0x7fb4d80ac450 con 0x7fb4e0158b80 2024-05-30T17:04:36.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.162+0000 7fb4dea00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4bc06e510 0x7fb4bc0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:36.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.162+0000 7fb4dea00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4bc06e510 0x7fb4bc0709d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0007fd0 tx=0x7fb4d0007960 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:36.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.166+0000 7fb4d6a00640 1 -- 172.21.15.67:0/240490852 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4d8043360 con 0x7fb4e0158b80 2024-05-30T17:04:36.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.402+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4e006e010 con 0x7fb4e0158b80 2024-05-30T17:04:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.403+0000 7fb4d6a00640 1 -- 172.21.15.67:0/240490852 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5189 (secure 0 0 0) 0x7fb4d807c3a0 con 0x7fb4e0158b80 2024-05-30T17:04:36.404 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:36.404 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":74,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1973461214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1973461214},{"type":"v1","addr":"172.21.15.169:6835","nonce":1973461214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":8400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3884958123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3884958123},{"type":"v1","addr":"172.21.15.67:6836","nonce":3884958123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":8403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1126347083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1126347083},{"type":"v1","addr":"172.21.15.67:6837","nonce":1126347083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:02:40.462403+0000","modified":"2024-05-30T17:02:43.518891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3425947383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3425947383},{"type":"v1","addr":"172.21.15.169:6834","nonce":3425947383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":8}]} 2024-05-30T17:04:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 74 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4bc06e510 msgr2=0x7fb4bc0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4bc06e510 0x7fb4bc0709d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0007fd0 tx=0x7fb4d0007960 comp rx=0 tx=0).stop 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e0158b80 msgr2=0x7fb4e01f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e0158b80 0x7fb4e01f4f10 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009e50 tx=0x7fb4d8004a40 comp rx=0 tx=0).stop 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 shutdown_connections 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb4bc06e510 0x7fb4bc0709d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e0158b80 0x7fb4e01f4f10 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4e00b32c0 0x7fb4e01f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 --2- 172.21.15.67:0/240490852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4e0071bc0 0x7fb4e01f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 >> 172.21.15.67:0/240490852 conn(0x7fb4e00b1fc0 msgr2=0x7fb4e00b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:36.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 shutdown_connections 2024-05-30T17:04:36.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.406+0000 7fb4dfe00640 1 -- 172.21.15.67:0/240490852 wait complete. 2024-05-30T17:04:36.424 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:04:36.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.629+0000 7f63aa600640 1 Processor -- start 2024-05-30T17:04:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.629+0000 7f63aa600640 1 -- start start 2024-05-30T17:04:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.630+0000 7f63aa600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f63a4071bc0 0x7f63a4072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.630+0000 7f63aa600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63a40b32c0 0x7f63a40b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.630+0000 7f63aa600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a406e6e0 con 0x7f63a4158b80 2024-05-30T17:04:36.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.630+0000 7f63aa600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f63a406e9c0 con 0x7f63a4071bc0 2024-05-30T17:04:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.630+0000 7f63aa600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f63a406eca0 con 0x7f63a40b32c0 2024-05-30T17:04:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.631+0000 7f63a3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f63a4071bc0 0x7f63a4072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.631+0000 7f63a9200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f63a4158b80 0x7f63a406dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42384/0 (socket says 172.21.15.67:42384) 2024-05-30T17:04:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.631+0000 7f63a9200640 1 -- 172.21.15.67:0/1401525188 learned_addr learned my addr 172.21.15.67:0/1401525188 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.631+0000 7f63a3e00640 1 -- 172.21.15.67:0/1401525188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63a40b32c0 msgr2=0x7f63a40b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.631+0000 7f63a3e00640 1 --2- 172.21.15.67:0/1401525188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63a40b32c0 0x7f63a40b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.631+0000 7f63a3e00640 1 -- 172.21.15.67:0/1401525188 >> v1:172.21.15.67:6789/0 conn(0x7f63a4158b80 legacy=0x7f63a406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.631+0000 7f63a3e00640 1 -- 172.21.15.67:0/1401525188 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a406ef80 con 0x7f63a4071bc0 2024-05-30T17:04:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.632+0000 7f63a3e00640 1 --2- 172.21.15.67:0/1401525188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4071bc0 0x7f63a4072070 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f6390009d90 tx=0x7f639002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd3ecacfd615c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63a3400640 1 -- 172.21.15.67:0/1401525188 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390007740 con 0x7f63a4071bc0 2024-05-30T17:04:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63a3400640 1 -- 172.21.15.67:0/1401525188 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900078e0 con 0x7f63a4071bc0 2024-05-30T17:04:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63a3400640 1 -- 172.21.15.67:0/1401525188 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390007cb0 con 0x7f63a4071bc0 2024-05-30T17:04:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63aa600640 1 -- 172.21.15.67:0/1401525188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4071bc0 msgr2=0x7f63a4072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63aa600640 1 --2- 172.21.15.67:0/1401525188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4071bc0 0x7f63a4072070 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f6390009d90 tx=0x7f639002a660 comp rx=0 tx=0).stop 2024-05-30T17:04:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63aa600640 1 -- 172.21.15.67:0/1401525188 shutdown_connections 2024-05-30T17:04:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63aa600640 1 --2- 172.21.15.67:0/1401525188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63a40b32c0 0x7f63a40b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63aa600640 1 --2- 172.21.15.67:0/1401525188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4071bc0 0x7f63a4072070 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63aa600640 1 -- 172.21.15.67:0/1401525188 >> 172.21.15.67:0/1401525188 conn(0x7f63a40b1fc0 msgr2=0x7f63a40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63aa600640 1 -- 172.21.15.67:0/1401525188 shutdown_connections 2024-05-30T17:04:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.633+0000 7f63aa600640 1 -- 172.21.15.67:0/1401525188 wait complete. 2024-05-30T17:04:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.635+0000 7f63aa600640 1 Processor -- start 2024-05-30T17:04:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.635+0000 7f63aa600640 1 -- start start 2024-05-30T17:04:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.636+0000 7f63aa600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a4071bc0 0x7f63a41f1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.636+0000 7f63aa600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63a40b32c0 0x7f63a41f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.636+0000 7f63aa600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4158b80 0x7f63a41f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.636+0000 7f63aa600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f63a406f780 con 0x7f63a4158b80 2024-05-30T17:04:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.636+0000 7f63aa600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63a40b2d30 con 0x7f63a4071bc0 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.636+0000 7f63aa600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f63a414ee50 con 0x7f63a40b32c0 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.636+0000 7f63a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4158b80 0x7f63a41f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.636+0000 7f63a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4158b80 0x7f63a41f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52246/0 (socket says 172.21.15.67:52246) 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.636+0000 7f63a9c00640 1 -- 172.21.15.67:0/1475624699 learned_addr learned my addr 172.21.15.67:0/1475624699 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.637+0000 7f63a9c00640 1 -- 172.21.15.67:0/1475624699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63a40b32c0 msgr2=0x7f63a41f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.637+0000 7f63a9c00640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63a40b32c0 0x7f63a41f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.637+0000 7f63a9200640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a4071bc0 0x7f63a41f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.637+0000 7f63a9c00640 1 -- 172.21.15.67:0/1475624699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a4071bc0 msgr2=0x7f63a41f1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.637+0000 7f63a9c00640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a4071bc0 0x7f63a41f1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.637+0000 7f63a9c00640 1 -- 172.21.15.67:0/1475624699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a414db20 con 0x7f63a4158b80 2024-05-30T17:04:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.637+0000 7f63a9c00640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4158b80 0x7f63a41f76c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f639c00b3e0 tx=0x7f639c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.637+0000 7f63a0c00640 1 -- 172.21.15.67:0/1475624699 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c015020 con 0x7f63a4158b80 2024-05-30T17:04:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.637+0000 7f63a0c00640 1 -- 172.21.15.67:0/1475624699 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639c011830 con 0x7f63a4158b80 2024-05-30T17:04:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.638+0000 7f63a0c00640 1 -- 172.21.15.67:0/1475624699 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c011c00 con 0x7f63a4158b80 2024-05-30T17:04:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.638+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a414ddc0 con 0x7f63a4158b80 2024-05-30T17:04:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.638+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a414e0a0 con 0x7f63a4158b80 2024-05-30T17:04:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.639+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a4073910 con 0x7f63a4158b80 2024-05-30T17:04:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.640+0000 7f63a0c00640 1 -- 172.21.15.67:0/1475624699 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f639c018020 con 0x7f63a4158b80 2024-05-30T17:04:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.640+0000 7f63a0c00640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f638406e510 0x7f63840709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.641+0000 7f63a0c00640 1 -- 172.21.15.67:0/1475624699 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5935+0+0 (secure 0 0 0) 0x7f639c08a9a0 con 0x7f63a4158b80 2024-05-30T17:04:36.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.641+0000 7f63a9200640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f638406e510 0x7f63840709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:36.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.641+0000 7f63a9200640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f638406e510 0x7f63840709d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6394000f60 tx=0x7f6394009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:36.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.645+0000 7f63a0c00640 1 -- 172.21.15.67:0/1475624699 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63a4073910 con 0x7f63a4158b80 2024-05-30T17:04:36.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.846+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63a414efd0 con 0x7f63a4158b80 2024-05-30T17:04:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.848+0000 7f63a0c00640 1 -- 172.21.15.67:0/1475624699 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+17012 (secure 0 0 0) 0x7f639c01f020 con 0x7f63a4158b80 2024-05-30T17:04:36.849 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:36.849 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":120,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:04:36.769499+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:02:38.380492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-30T17:02:39.422718+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.9","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f638406e510 msgr2=0x7f63840709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f638406e510 0x7f63840709d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6394000f60 tx=0x7f6394009040 comp rx=0 tx=0).stop 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4158b80 msgr2=0x7f63a41f76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4158b80 0x7f63a41f76c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f639c00b3e0 tx=0x7f639c009f90 comp rx=0 tx=0).stop 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 shutdown_connections 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f638406e510 0x7f63840709d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63a4158b80 0x7f63a41f76c0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63a40b32c0 0x7f63a41f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 --2- 172.21.15.67:0/1475624699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a4071bc0 0x7f63a41f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 >> 172.21.15.67:0/1475624699 conn(0x7f63a40b1fc0 msgr2=0x7f63a40b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.850+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 shutdown_connections 2024-05-30T17:04:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:36.851+0000 7f63aa600640 1 -- 172.21.15.67:0/1475624699 wait complete. 2024-05-30T17:04:36.869 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-30T17:04:36.869 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:04:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.070+0000 7f12d7800640 1 Processor -- start 2024-05-30T17:04:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.071+0000 7f12d7800640 1 -- start start 2024-05-30T17:04:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.072+0000 7f12d7800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12d015dca0 0x7f12d006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.072+0000 7f12d7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 0x7f12d00cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.072+0000 7f12d7800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12d015e130 con 0x7f12d01536f0 2024-05-30T17:04:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.072+0000 7f12d7800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f12d0161030 con 0x7f12d015dca0 2024-05-30T17:04:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.072+0000 7f12d7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12d01604b0 con 0x7f12d006ce50 2024-05-30T17:04:37.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.072+0000 7f12c7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12d015dca0 0x7f12d006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:37.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.072+0000 7f12c7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12d015dca0 0x7f12d006c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52260/0 (socket says 172.21.15.67:52260) 2024-05-30T17:04:37.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.072+0000 7f12c7e00640 1 -- 172.21.15.67:0/3666291816 learned_addr learned my addr 172.21.15.67:0/3666291816 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:37.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.073+0000 7f12d5800640 1 --2- 172.21.15.67:0/3666291816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 0x7f12d00cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:37.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.073+0000 7f12c7e00640 1 -- 172.21.15.67:0/3666291816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 msgr2=0x7f12d00cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:37.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.073+0000 7f12c7e00640 1 --2- 172.21.15.67:0/3666291816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 0x7f12d00cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.073+0000 7f12c7e00640 1 -- 172.21.15.67:0/3666291816 >> v1:172.21.15.169:6789/0 conn(0x7f12d01536f0 legacy=0x7f12d00b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:37.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.073+0000 7f12c7e00640 1 -- 172.21.15.67:0/3666291816 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d0160790 con 0x7f12d015dca0 2024-05-30T17:04:37.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.073+0000 7f12d5800640 1 --2- 172.21.15.67:0/3666291816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 0x7f12d00cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.074+0000 7f12c7e00640 1 --2- 172.21.15.67:0/3666291816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d015dca0 0x7f12d006c910 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f12b80098a0 tx=0x7f12b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f52c60f896cd04 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.074+0000 7f12cfe00640 1 -- 172.21.15.67:0/3666291816 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8037070 con 0x7f12d015dca0 2024-05-30T17:04:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.074+0000 7f12cfe00640 1 -- 172.21.15.67:0/3666291816 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b8032440 con 0x7f12d015dca0 2024-05-30T17:04:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.074+0000 7f12cfe00640 1 -- 172.21.15.67:0/3666291816 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8032810 con 0x7f12d015dca0 2024-05-30T17:04:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.075+0000 7f12d7800640 1 -- 172.21.15.67:0/3666291816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d015dca0 msgr2=0x7f12d006c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.075+0000 7f12d7800640 1 --2- 172.21.15.67:0/3666291816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d015dca0 0x7f12d006c910 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f12b80098a0 tx=0x7f12b802a1a0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.075+0000 7f12d7800640 1 -- 172.21.15.67:0/3666291816 shutdown_connections 2024-05-30T17:04:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.075+0000 7f12d7800640 1 --2- 172.21.15.67:0/3666291816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 0x7f12d00cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.075+0000 7f12d7800640 1 --2- 172.21.15.67:0/3666291816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d015dca0 0x7f12d006c910 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.075+0000 7f12d7800640 1 -- 172.21.15.67:0/3666291816 >> 172.21.15.67:0/3666291816 conn(0x7f12d00724f0 msgr2=0x7f12d00728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.075+0000 7f12d7800640 1 -- 172.21.15.67:0/3666291816 shutdown_connections 2024-05-30T17:04:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.075+0000 7f12d7800640 1 -- 172.21.15.67:0/3666291816 wait complete. 2024-05-30T17:04:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.076+0000 7f12d7800640 1 Processor -- start 2024-05-30T17:04:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.077+0000 7f12d7800640 1 -- start start 2024-05-30T17:04:37.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.077+0000 7f12d7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 0x7f12d0140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.077+0000 7f12d7800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01536f0 0x7f12d01e6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.077+0000 7f12d7800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12d015dca0 0x7f12d01e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.077+0000 7f12d7800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f12d0160180 con 0x7f12d01536f0 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12d7800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12d0160d50 con 0x7f12d015dca0 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12d7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12d0070b00 con 0x7f12d006ce50 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01536f0 0x7f12d01e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01536f0 0x7f12d01e6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52264/0 (socket says 172.21.15.67:52264) 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12c7e00640 1 -- 172.21.15.67:0/668337645 learned_addr learned my addr 172.21.15.67:0/668337645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12c7e00640 1 -- 172.21.15.67:0/668337645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 msgr2=0x7f12d0140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12c7e00640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 0x7f12d0140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12c7e00640 1 -- 172.21.15.67:0/668337645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12d015dca0 msgr2=0x7f12d01e6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12c7e00640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12d015dca0 0x7f12d01e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.078+0000 7f12c7e00640 1 -- 172.21.15.67:0/668337645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d0143800 con 0x7f12d01536f0 2024-05-30T17:04:37.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.079+0000 7f12c7e00640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01536f0 0x7f12d01e6740 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f12b80090b0 tx=0x7f12b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:37.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.079+0000 7f12cd600640 1 -- 172.21.15.67:0/668337645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8037040 con 0x7f12d01536f0 2024-05-30T17:04:37.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.079+0000 7f12cd600640 1 -- 172.21.15.67:0/668337645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b8004230 con 0x7f12d01536f0 2024-05-30T17:04:37.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.079+0000 7f12cd600640 1 -- 172.21.15.67:0/668337645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8003610 con 0x7f12d01536f0 2024-05-30T17:04:37.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.079+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d01f7b50 con 0x7f12d01536f0 2024-05-30T17:04:37.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.079+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d01f8110 con 0x7f12d01536f0 2024-05-30T17:04:37.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.080+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d00689e0 con 0x7f12d01536f0 2024-05-30T17:04:37.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.081+0000 7f12cd600640 1 -- 172.21.15.67:0/668337645 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f12b80037b0 con 0x7f12d01536f0 2024-05-30T17:04:37.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.082+0000 7f12cd600640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12ac06e510 0x7f12ac0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.082+0000 7f12cd600640 1 -- 172.21.15.67:0/668337645 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5907+0+0 (secure 0 0 0) 0x7f12b80ab9f0 con 0x7f12d01536f0 2024-05-30T17:04:37.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.082+0000 7f12d4e00640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12ac06e510 0x7f12ac0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:37.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.083+0000 7f12d4e00640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12ac06e510 0x7f12ac0709d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f12c0006fd0 tx=0x7f12c0006bc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:37.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.087+0000 7f12cd600640 1 -- 172.21.15.67:0/668337645 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12b807b880 con 0x7f12d01536f0 2024-05-30T17:04:37.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.327+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12d0154d90 con 0x7f12d01536f0 2024-05-30T17:04:37.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.328+0000 7f12cd600640 1 -- 172.21.15.67:0/668337645 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5189 (secure 0 0 0) 0x7f12b807b180 con 0x7f12d01536f0 2024-05-30T17:04:37.329 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:37.329 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":74,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1973461214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1973461214},{"type":"v1","addr":"172.21.15.169:6835","nonce":1973461214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":8400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3884958123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3884958123},{"type":"v1","addr":"172.21.15.67:6836","nonce":3884958123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":8403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1126347083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1126347083},{"type":"v1","addr":"172.21.15.67:6837","nonce":1126347083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:02:40.462403+0000","modified":"2024-05-30T17:02:43.518891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3425947383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3425947383},{"type":"v1","addr":"172.21.15.169:6834","nonce":3425947383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":8}]} 2024-05-30T17:04:37.329 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 74 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12ac06e510 msgr2=0x7f12ac0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12ac06e510 0x7f12ac0709d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f12c0006fd0 tx=0x7f12c0006bc0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01536f0 msgr2=0x7f12d01e6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01536f0 0x7f12d01e6740 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f12b80090b0 tx=0x7f12b80046c0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 shutdown_connections 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f12ac06e510 0x7f12ac0709d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12d015dca0 0x7f12d01e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01536f0 0x7f12d01e6740 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 --2- 172.21.15.67:0/668337645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12d006ce50 0x7f12d0140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 >> 172.21.15.67:0/668337645 conn(0x7f12d00724f0 msgr2=0x7f12d006fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:37.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 shutdown_connections 2024-05-30T17:04:37.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.331+0000 7f12d7800640 1 -- 172.21.15.67:0/668337645 wait complete. 2024-05-30T17:04:37.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:04:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.547+0000 7fd0f4400640 1 Processor -- start 2024-05-30T17:04:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.547+0000 7fd0f4400640 1 -- start start 2024-05-30T17:04:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.548+0000 7fd0f4400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0ec0b4b60 0x7fd0ec0cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.549+0000 7fd0f4400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd0ec0b3b70 0x7fd0ec0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.549+0000 7fd0f4400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ec0ced40 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.549+0000 7fd0f4400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd0ec0b3f90 con 0x7fd0ec0b4b60 2024-05-30T17:04:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.549+0000 7fd0f4400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd0ec15fbb0 con 0x7fd0ec0b3b70 2024-05-30T17:04:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.549+0000 7fd0f1a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0ec0b4b60 0x7fd0ec0cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.549+0000 7fd0f1a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0ec0b4b60 0x7fd0ec0cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52270/0 (socket says 172.21.15.67:52270) 2024-05-30T17:04:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.549+0000 7fd0f1a00640 1 -- 172.21.15.67:0/3582980751 learned_addr learned my addr 172.21.15.67:0/3582980751 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.550+0000 7fd0f1a00640 1 -- 172.21.15.67:0/3582980751 >> v2:172.21.15.169:3300/0 conn(0x7fd0ec0b3b70 msgr2=0x7fd0ec0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.550+0000 7fd0f1a00640 1 --2- 172.21.15.67:0/3582980751 >> v2:172.21.15.169:3300/0 conn(0x7fd0ec0b3b70 0x7fd0ec0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.550+0000 7fd0f1a00640 1 -- 172.21.15.67:0/3582980751 >> v1:172.21.15.169:6789/0 conn(0x7fd0ec0d04a0 legacy=0x7fd0ec0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.550+0000 7fd0f1a00640 1 -- 172.21.15.67:0/3582980751 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec0cefc0 con 0x7fd0ec0b4b60 2024-05-30T17:04:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.551+0000 7fd0f1a00640 1 --2- 172.21.15.67:0/3582980751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0b4b60 0x7fd0ec0cfaa0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0af320 tx=0x7fd0e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa334f6e94b8cc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:37.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.551+0000 7fd0ebe00640 1 -- 172.21.15.67:0/3582980751 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0037070 con 0x7fd0ec0b4b60 2024-05-30T17:04:37.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0ebe00640 1 -- 172.21.15.67:0/3582980751 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e0032440 con 0x7fd0ec0b4b60 2024-05-30T17:04:37.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0ebe00640 1 -- 172.21.15.67:0/3582980751 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0032810 con 0x7fd0ec0b4b60 2024-05-30T17:04:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0f4400640 1 -- 172.21.15.67:0/3582980751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0b4b60 msgr2=0x7fd0ec0cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0f4400640 1 --2- 172.21.15.67:0/3582980751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0b4b60 0x7fd0ec0cfaa0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0af320 tx=0x7fd0e002a230 comp rx=0 tx=0).stop 2024-05-30T17:04:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0f4400640 1 -- 172.21.15.67:0/3582980751 shutdown_connections 2024-05-30T17:04:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0f4400640 1 --2- 172.21.15.67:0/3582980751 >> v2:172.21.15.169:3300/0 conn(0x7fd0ec0b3b70 0x7fd0ec0d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0f4400640 1 --2- 172.21.15.67:0/3582980751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0b4b60 0x7fd0ec0cfaa0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0f4400640 1 -- 172.21.15.67:0/3582980751 >> 172.21.15.67:0/3582980751 conn(0x7fd0ec06bae0 msgr2=0x7fd0ec06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0f4400640 1 -- 172.21.15.67:0/3582980751 shutdown_connections 2024-05-30T17:04:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.552+0000 7fd0f4400640 1 -- 172.21.15.67:0/3582980751 wait complete. 2024-05-30T17:04:37.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.554+0000 7fd0f4400640 1 Processor -- start 2024-05-30T17:04:37.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.554+0000 7fd0f4400640 1 -- start start 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.555+0000 7fd0f4400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0ec0b3b70 0x7fd0ec0738c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.555+0000 7fd0f4400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0ec0b4b60 0x7fd0ec072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.555+0000 7fd0f4400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0d04a0 0x7fd0ec1f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.555+0000 7fd0f4400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec0cffe0 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.555+0000 7fd0f4400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec0cf580 con 0x7fd0ec0b3b70 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.555+0000 7fd0f4400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ec15ede0 con 0x7fd0ec0b4b60 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f2400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0d04a0 0x7fd0ec1f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f2400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0d04a0 0x7fd0ec1f7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52280/0 (socket says 172.21.15.67:52280) 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f2400640 1 -- 172.21.15.67:0/3100702871 learned_addr learned my addr 172.21.15.67:0/3100702871 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f1a00640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0ec0b3b70 0x7fd0ec0738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f1000640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0ec0b4b60 0x7fd0ec072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f2400640 1 -- 172.21.15.67:0/3100702871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0ec0b4b60 msgr2=0x7fd0ec072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f2400640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0ec0b4b60 0x7fd0ec072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f2400640 1 -- 172.21.15.67:0/3100702871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0ec0b3b70 msgr2=0x7fd0ec0738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f2400640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0ec0b3b70 0x7fd0ec0738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f2400640 1 -- 172.21.15.67:0/3100702871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec15d970 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.556+0000 7fd0f1000640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0ec0b4b60 0x7fd0ec072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.557+0000 7fd0f2400640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0d04a0 0x7fd0ec1f7f30 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400b9a0 tx=0x7fd0e400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.557+0000 7fd0e9600640 1 -- 172.21.15.67:0/3100702871 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4019070 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.557+0000 7fd0e9600640 1 -- 172.21.15.67:0/3100702871 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e400cbc0 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.557+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ec0c6630 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.557+0000 7fd0e9600640 1 -- 172.21.15.67:0/3100702871 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4014440 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.557+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ec159020 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.558+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0ec0689e0 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.559+0000 7fd0e9600640 1 -- 172.21.15.67:0/3100702871 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fd0e401c070 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.560+0000 7fd0e9600640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd0cc06e510 0x7fd0cc0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:37.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.560+0000 7fd0e9600640 1 -- 172.21.15.67:0/3100702871 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5907+0+0 (secure 0 0 0) 0x7fd0e408a870 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.560+0000 7fd0f1a00640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd0cc06e510 0x7fd0cc0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:37.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.561+0000 7fd0f1a00640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd0cc06e510 0x7fd0cc0709d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0034000 tx=0x7fd0e002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:37.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.567+0000 7fd0e9600640 1 -- 172.21.15.67:0/3100702871 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0e405a700 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.812+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0ec161920 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.813+0000 7fd0e9600640 1 -- 172.21.15.67:0/3100702871 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5189 (secure 0 0 0) 0x7fd0e405a000 con 0x7fd0ec0d04a0 2024-05-30T17:04:37.814 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:37.814 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":74,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1973461214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1973461214},{"type":"v1","addr":"172.21.15.169:6835","nonce":1973461214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":8400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3884958123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3884958123},{"type":"v1","addr":"172.21.15.67:6836","nonce":3884958123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":8403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1126347083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1126347083},{"type":"v1","addr":"172.21.15.67:6837","nonce":1126347083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:02:40.462403+0000","modified":"2024-05-30T17:02:43.518891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6834/3425947383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3425947383},{"type":"v1","addr":"172.21.15.169:6834","nonce":3425947383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":8}]} 2024-05-30T17:04:37.814 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 74 2024-05-30T17:04:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd0cc06e510 msgr2=0x7fd0cc0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd0cc06e510 0x7fd0cc0709d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0034000 tx=0x7fd0e002e000 comp rx=0 tx=0).stop 2024-05-30T17:04:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0d04a0 msgr2=0x7fd0ec1f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0d04a0 0x7fd0ec1f7f30 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400b9a0 tx=0x7fd0e400be60 comp rx=0 tx=0).stop 2024-05-30T17:04:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 shutdown_connections 2024-05-30T17:04:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd0cc06e510 0x7fd0cc0709d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ec0d04a0 0x7fd0ec1f7f30 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0ec0b4b60 0x7fd0ec072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 --2- 172.21.15.67:0/3100702871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0ec0b3b70 0x7fd0ec0738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 >> 172.21.15.67:0/3100702871 conn(0x7fd0ec06bae0 msgr2=0x7fd0ec0b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 shutdown_connections 2024-05-30T17:04:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:37.816+0000 7fd0f4400640 1 -- 172.21.15.67:0/3100702871 wait complete. 2024-05-30T17:04:37.836 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:04:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.049+0000 7fe302000640 1 Processor -- start 2024-05-30T17:04:38.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.049+0000 7fe302000640 1 -- start start 2024-05-30T17:04:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.050+0000 7fe302000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2fc06ca10 0x7fe2fc0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.050+0000 7fe302000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2fc09abd0 con 0x7fe2fc0c52c0 2024-05-30T17:04:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.051+0000 7fe302000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2fc09aeb0 con 0x7fe2fc06ce90 2024-05-30T17:04:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.051+0000 7fe302000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc09b190 con 0x7fe2fc06ca10 2024-05-30T17:04:38.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.051+0000 7fe300c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe2fc0c52c0 0x7fe2fc0c56a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42388/0 (socket says 172.21.15.67:42388) 2024-05-30T17:04:38.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.051+0000 7fe300c00640 1 -- 172.21.15.67:0/1740209775 learned_addr learned my addr 172.21.15.67:0/1740209775 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.051+0000 7fe2fbe00640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2fc06ca10 0x7fe2fc0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.051+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194074848 0 0) 0x7fe2fc09abd0 con 0x7fe2fc0c52c0 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.051+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2e0003520 con 0x7fe2fc0c52c0 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3333719861 0 0) 0x7fe2e0003520 con 0x7fe2fc0c52c0 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2fc06ca10 msgr2=0x7fe2fc0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2fc06ca10 0x7fe2fc0cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 >> v1:172.21.15.169:6789/0 conn(0x7fe2fc06ce90 legacy=0x7fe2fc0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc09b470 con 0x7fe2fc0c52c0 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fe2e4001670 con 0x7fe2fc0c52c0 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 >> v1:172.21.15.67:6789/0 conn(0x7fe2fc0c52c0 legacy=0x7fe2fc0c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2fc06ca10 0x7fe2e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2e0003fb0 0x7fe2e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2e0004d40 0x7fe2e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc09b190 con 0x7fe2e0003fb0 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e000bb00 con 0x7fe2e0004d40 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e000bd00 con 0x7fe2fc06ca10 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.052+0000 7fe2fbe00640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2e0003fb0 0x7fe2e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.053+0000 7fe2fbe00640 1 -- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2fc06ca10 msgr2=0x7fe2e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.053+0000 7fe300c00640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2fc06ca10 0x7fe2e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.053+0000 7fe2fbe00640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2fc06ca10 0x7fe2e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.053+0000 7fe2fbe00640 1 -- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2e0004d40 msgr2=0x7fe2e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.053+0000 7fe2fbe00640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2e0004d40 0x7fe2e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.053+0000 7fe2fbe00640 1 -- 172.21.15.67:0/1740209775 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e0003520 con 0x7fe2e0003fb0 2024-05-30T17:04:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.053+0000 7fe300c00640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2fc06ca10 0x7fe2e00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.054+0000 7fe2fbe00640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2e0003fb0 0x7fe2e0004800 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009fc0 tx=0x7fe2e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9aceb0c88461259 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.054+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec037070 con 0x7fe2e0003fb0 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.054+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ec0325f0 con 0x7fe2e0003fb0 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.054+0000 7fe2fb400640 1 -- 172.21.15.67:0/1740209775 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec009c10 con 0x7fe2e0003fb0 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.055+0000 7fe302000640 1 -- 172.21.15.67:0/1740209775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2e0003fb0 msgr2=0x7fe2e0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.055+0000 7fe302000640 1 --2- 172.21.15.67:0/1740209775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2e0003fb0 0x7fe2e0004800 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009fc0 tx=0x7fe2e0005830 comp rx=0 tx=0).stop 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.055+0000 7fe301600640 1 -- 172.21.15.67:0/1740209775 reap_dead start 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.055+0000 7fe302000640 1 -- 172.21.15.67:0/1740209775 shutdown_connections 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.055+0000 7fe302000640 1 -- 172.21.15.67:0/1740209775 >> 172.21.15.67:0/1740209775 conn(0x7fe2fc0b3de0 msgr2=0x7fe2fc0b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.055+0000 7fe302000640 1 -- 172.21.15.67:0/1740209775 shutdown_connections 2024-05-30T17:04:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.055+0000 7fe302000640 1 -- 172.21.15.67:0/1740209775 wait complete. 2024-05-30T17:04:38.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.057+0000 7fe302000640 1 Processor -- start 2024-05-30T17:04:38.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.057+0000 7fe302000640 1 -- start start 2024-05-30T17:04:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.058+0000 7fe302000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2e0003fb0 0x7fe2fc15f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.058+0000 7fe302000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2fc15fbc0 0x7fe2fc0b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.058+0000 7fe302000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2fc0b6120 0x7fe2fc161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.058+0000 7fe302000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc0b4a90 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.058+0000 7fe302000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc09bc70 con 0x7fe2fc0b6120 2024-05-30T17:04:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.058+0000 7fe302000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc0ce640 con 0x7fe2e0003fb0 2024-05-30T17:04:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.058+0000 7fe2fbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2fc15fbc0 0x7fe2fc0b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.058+0000 7fe2fbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2fc15fbc0 0x7fe2fc0b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52304/0 (socket says 172.21.15.67:52304) 2024-05-30T17:04:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.058+0000 7fe2fbe00640 1 -- 172.21.15.67:0/364574130 learned_addr learned my addr 172.21.15.67:0/364574130 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:38.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe2fbe00640 1 -- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2e0003fb0 msgr2=0x7fe2fc15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe301600640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2fc0b6120 0x7fe2fc161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe300c00640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2e0003fb0 0x7fe2fc15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe2fbe00640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2e0003fb0 0x7fe2fc15f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe2fbe00640 1 -- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2fc0b6120 msgr2=0x7fe2fc161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe2fbe00640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2fc0b6120 0x7fe2fc161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe2fbe00640 1 -- 172.21.15.67:0/364574130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc0b65c0 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe301600640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2fc0b6120 0x7fe2fc161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe300c00640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2e0003fb0 0x7fe2fc15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe2fbe00640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2fc15fbc0 0x7fe2fc0b5be0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec035040 tx=0x7fe2ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.059+0000 7fe2f8c00640 1 -- 172.21.15.67:0/364574130 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec042030 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.060+0000 7fe2f8c00640 1 -- 172.21.15.67:0/364574130 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ec0323f0 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.060+0000 7fe2f8c00640 1 -- 172.21.15.67:0/364574130 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec037040 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.060+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2fc0ca780 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.060+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2fc0b68a0 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.061+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2fc0c7f00 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.063+0000 7fe2f8c00640 1 -- 172.21.15.67:0/364574130 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fe2ec045070 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.064+0000 7fe2f8c00640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d806e510 0x7fe2d80709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.064+0000 7fe2f8c00640 1 -- 172.21.15.67:0/364574130 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5907+0+0 (secure 0 0 0) 0x7fe2ec0abfa0 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.064+0000 7fe300c00640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d806e510 0x7fe2d80709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.065+0000 7fe300c00640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d806e510 0x7fe2d80709d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc06e7f0 tx=0x7fe2e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:38.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.068+0000 7fe2f8c00640 1 -- 172.21.15.67:0/364574130 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2ec02f070 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.279+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2fc0690f0 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.281+0000 7fe2f8c00640 1 -- 172.21.15.67:0/364574130 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+17012 (secure 0 0 0) 0x7fe2ec07beb0 con 0x7fe2fc15fbc0 2024-05-30T17:04:38.282 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:38.282 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":120,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:04:36.769499+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:02:38.380492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-30T17:02:39.422718+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.9","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:04:38.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d806e510 msgr2=0x7fe2d80709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d806e510 0x7fe2d80709d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc06e7f0 tx=0x7fe2e4009040 comp rx=0 tx=0).stop 2024-05-30T17:04:38.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2fc15fbc0 msgr2=0x7fe2fc0b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2fc15fbc0 0x7fe2fc0b5be0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec035040 tx=0x7fe2ec009750 comp rx=0 tx=0).stop 2024-05-30T17:04:38.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 shutdown_connections 2024-05-30T17:04:38.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d806e510 0x7fe2d80709d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2fc0b6120 0x7fe2fc161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2fc15fbc0 0x7fe2fc0b5be0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 --2- 172.21.15.67:0/364574130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2e0003fb0 0x7fe2fc15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 >> 172.21.15.67:0/364574130 conn(0x7fe2fc0b3de0 msgr2=0x7fe2fc0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:38.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.284+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 shutdown_connections 2024-05-30T17:04:38.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.285+0000 7fe302000640 1 -- 172.21.15.67:0/364574130 wait complete. 2024-05-30T17:04:38.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-30T17:04:38.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.523+0000 7f212f000640 1 Processor -- start 2024-05-30T17:04:38.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.523+0000 7f212f000640 1 -- start start 2024-05-30T17:04:38.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.525+0000 7f212f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f212806dc20 0x7f21280b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.525+0000 7f212f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21280afd60 con 0x7f212815b3a0 2024-05-30T17:04:38.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.525+0000 7f212f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21280af7a0 con 0x7f2128073310 2024-05-30T17:04:38.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.525+0000 7f212f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f212806f980 con 0x7f212806dc20 2024-05-30T17:04:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.525+0000 7f212e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f212806dc20 0x7f21280b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.525+0000 7f212dc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f212815b3a0 0x7f2128072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42394/0 (socket says 172.21.15.67:42394) 2024-05-30T17:04:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.525+0000 7f212dc00640 1 -- 172.21.15.67:0/971240566 learned_addr learned my addr 172.21.15.67:0/971240566 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.526+0000 7f212e600640 1 -- 172.21.15.67:0/971240566 >> v1:172.21.15.169:6789/0 conn(0x7f2128073310 legacy=0x7f21280ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.526+0000 7f212e600640 1 -- 172.21.15.67:0/971240566 >> v1:172.21.15.67:6789/0 conn(0x7f212815b3a0 legacy=0x7f2128072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.526+0000 7f212e600640 1 -- 172.21.15.67:0/971240566 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21280afa80 con 0x7f212806dc20 2024-05-30T17:04:38.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.527+0000 7f212e600640 1 --2- 172.21.15.67:0/971240566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212806dc20 0x7f21280b32c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f21200097a0 tx=0x7f212002c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7163d466dcc2695 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:38.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.527+0000 7f2127e00640 1 -- 172.21.15.67:0/971240566 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120039070 con 0x7f212806dc20 2024-05-30T17:04:38.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.527+0000 7f2127e00640 1 -- 172.21.15.67:0/971240566 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2120034640 con 0x7f212806dc20 2024-05-30T17:04:38.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.527+0000 7f2127e00640 1 -- 172.21.15.67:0/971240566 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120034a10 con 0x7f212806dc20 2024-05-30T17:04:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.528+0000 7f212f000640 1 -- 172.21.15.67:0/971240566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212806dc20 msgr2=0x7f21280b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.528+0000 7f212f000640 1 --2- 172.21.15.67:0/971240566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212806dc20 0x7f21280b32c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f21200097a0 tx=0x7f212002c3f0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.528+0000 7f212f000640 1 -- 172.21.15.67:0/971240566 shutdown_connections 2024-05-30T17:04:38.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.528+0000 7f212f000640 1 --2- 172.21.15.67:0/971240566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212806dc20 0x7f21280b32c0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.528+0000 7f212f000640 1 -- 172.21.15.67:0/971240566 >> 172.21.15.67:0/971240566 conn(0x7f21280b1fc0 msgr2=0x7f21280b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:38.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.528+0000 7f212f000640 1 -- 172.21.15.67:0/971240566 shutdown_connections 2024-05-30T17:04:38.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.528+0000 7f212f000640 1 -- 172.21.15.67:0/971240566 wait complete. 2024-05-30T17:04:38.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.530+0000 7f212f000640 1 Processor -- start 2024-05-30T17:04:38.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.530+0000 7f212f000640 1 -- start start 2024-05-30T17:04:38.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212806dc20 0x7f212814c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2128073310 0x7f212815b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212815b3a0 0x7f21280cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f212815f9a0 con 0x7f212815b3a0 2024-05-30T17:04:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21280ae0c0 con 0x7f212806dc20 2024-05-30T17:04:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f212815fb20 con 0x7f2128073310 2024-05-30T17:04:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212815b3a0 0x7f21280cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212815b3a0 0x7f21280cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52332/0 (socket says 172.21.15.67:52332) 2024-05-30T17:04:38.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212e600640 1 -- 172.21.15.67:0/2722331745 learned_addr learned my addr 172.21.15.67:0/2722331745 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.531+0000 7f212dc00640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212806dc20 0x7f212814c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f212e600640 1 -- 172.21.15.67:0/2722331745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2128073310 msgr2=0x7f212815b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f212e600640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2128073310 0x7f212815b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f212e600640 1 -- 172.21.15.67:0/2722331745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212806dc20 msgr2=0x7f212814c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f212e600640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212806dc20 0x7f212814c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f212e600640 1 -- 172.21.15.67:0/2722331745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212815bed0 con 0x7f212815b3a0 2024-05-30T17:04:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f212dc00640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212806dc20 0x7f212814c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f212e600640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212815b3a0 0x7f21280cdcf0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f21200098c0 tx=0x7f21200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f2125600640 1 -- 172.21.15.67:0/2722331745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120044070 con 0x7f212815b3a0 2024-05-30T17:04:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f2125600640 1 -- 172.21.15.67:0/2722331745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2120032900 con 0x7f212815b3a0 2024-05-30T17:04:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f2125600640 1 -- 172.21.15.67:0/2722331745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120039070 con 0x7f212815b3a0 2024-05-30T17:04:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.532+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212815e5e0 con 0x7f212815b3a0 2024-05-30T17:04:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.533+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212815eba0 con 0x7f212815b3a0 2024-05-30T17:04:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.534+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21280689e0 con 0x7f212815b3a0 2024-05-30T17:04:38.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.535+0000 7f2125600640 1 -- 172.21.15.67:0/2722331745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f2120047070 con 0x7f212815b3a0 2024-05-30T17:04:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.536+0000 7f2125600640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f210806e540 0x7f2108070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.536+0000 7f2125600640 1 -- 172.21.15.67:0/2722331745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5907+0+0 (secure 0 0 0) 0x7f21200ab220 con 0x7f212815b3a0 2024-05-30T17:04:38.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.536+0000 7f212dc00640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f210806e540 0x7f2108070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:38.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.537+0000 7f212dc00640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f210806e540 0x7f2108070a00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f211c0012e0 tx=0x7f211c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:38.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.539+0000 7f2125600640 1 -- 172.21.15.67:0/2722331745 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f212004e020 con 0x7f212815b3a0 2024-05-30T17:04:38.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.791+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f21280b02b0 con 0x7f212815b3a0 2024-05-30T17:04:38.793 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-05-30T17:04:38.792+0000 7f4982000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-30T17:04:38.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.797+0000 7f2125600640 1 -- 172.21.15.67:0/2722331745 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7f212007b130 con 0x7f212815b3a0 2024-05-30T17:04:38.797 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-30T17:04:38.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f210806e540 msgr2=0x7f2108070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f210806e540 0x7f2108070a00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f211c0012e0 tx=0x7f211c009040 comp rx=0 tx=0).stop 2024-05-30T17:04:38.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212815b3a0 msgr2=0x7f21280cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:38.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212815b3a0 0x7f21280cdcf0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f21200098c0 tx=0x7f21200046c0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 shutdown_connections 2024-05-30T17:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f210806e540 0x7f2108070a00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f212815b3a0 0x7f21280cdcf0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2128073310 0x7f212815b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 --2- 172.21.15.67:0/2722331745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212806dc20 0x7f212814c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 >> 172.21.15.67:0/2722331745 conn(0x7f21280b1fc0 msgr2=0x7f21280b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 shutdown_connections 2024-05-30T17:04:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:38.800+0000 7f212f000640 1 -- 172.21.15.67:0/2722331745 wait complete. 2024-05-30T17:04:38.822 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-30T17:04:38.923 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T17:04:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.020+0000 7fc98c000640 1 Processor -- start 2024-05-30T17:04:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.020+0000 7fc98c000640 1 -- start start 2024-05-30T17:04:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.022+0000 7fc98c000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc9840d0640 0x7fc98415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:39.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.022+0000 7fc98c000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc9840b32c0 0x7fc9840b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:39.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.022+0000 7fc98c000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9840d0060 con 0x7fc9840d0a20 2024-05-30T17:04:39.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.022+0000 7fc98c000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fc9840adbb0 con 0x7fc9840b32c0 2024-05-30T17:04:39.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.022+0000 7fc98c000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc98415db10 con 0x7fc9840d0640 2024-05-30T17:04:39.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.022+0000 7fc98a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc9840b32c0 0x7fc9840b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:39.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.022+0000 7fc988c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc9840d0a20 0x7fc9840cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42398/0 (socket says 172.21.15.67:42398) 2024-05-30T17:04:39.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.023+0000 7fc988c00640 1 -- 172.21.15.67:0/1919289185 learned_addr learned my addr 172.21.15.67:0/1919289185 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:39.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.023+0000 7fc989600640 1 --2- 172.21.15.67:0/1919289185 >> v2:172.21.15.169:3300/0 conn(0x7fc9840d0640 0x7fc98415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:39.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.023+0000 7fc983e00640 1 -- 172.21.15.67:0/1919289185 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592235523 0 0) 0x7fc9840d0060 con 0x7fc9840d0a20 2024-05-30T17:04:39.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.023+0000 7fc983e00640 1 -- 172.21.15.67:0/1919289185 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc96c003520 con 0x7fc9840d0a20 2024-05-30T17:04:39.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.023+0000 7fc98a000640 1 -- 172.21.15.67:0/1919289185 >> v2:172.21.15.169:3300/0 conn(0x7fc9840d0640 msgr2=0x7fc98415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:39.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.023+0000 7fc98a000640 1 --2- 172.21.15.67:0/1919289185 >> v2:172.21.15.169:3300/0 conn(0x7fc9840d0640 0x7fc98415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:39.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.023+0000 7fc98a000640 1 -- 172.21.15.67:0/1919289185 >> v1:172.21.15.67:6789/0 conn(0x7fc9840d0a20 legacy=0x7fc9840cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:39.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.023+0000 7fc98a000640 1 -- 172.21.15.67:0/1919289185 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98415cb10 con 0x7fc9840b32c0 2024-05-30T17:04:39.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.024+0000 7fc98a000640 1 --2- 172.21.15.67:0/1919289185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840b32c0 0x7fc9840b3730 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc984074150 tx=0x7fc97c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61c43799ab5759c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:39.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.025+0000 7fc983e00640 1 -- 172.21.15.67:0/1919289185 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c031600 con 0x7fc9840b32c0 2024-05-30T17:04:39.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.025+0000 7fc983e00640 1 -- 172.21.15.67:0/1919289185 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c0317a0 con 0x7fc9840b32c0 2024-05-30T17:04:39.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.025+0000 7fc983e00640 1 -- 172.21.15.67:0/1919289185 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c031b70 con 0x7fc9840b32c0 2024-05-30T17:04:39.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.025+0000 7fc98c000640 1 -- 172.21.15.67:0/1919289185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840b32c0 msgr2=0x7fc9840b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:39.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.025+0000 7fc98c000640 1 --2- 172.21.15.67:0/1919289185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840b32c0 0x7fc9840b3730 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc984074150 tx=0x7fc97c02a610 comp rx=0 tx=0).stop 2024-05-30T17:04:39.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.026+0000 7fc98c000640 1 -- 172.21.15.67:0/1919289185 shutdown_connections 2024-05-30T17:04:39.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.026+0000 7fc98c000640 1 --2- 172.21.15.67:0/1919289185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840b32c0 0x7fc9840b3730 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:39.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.026+0000 7fc98c000640 1 --2- 172.21.15.67:0/1919289185 >> v2:172.21.15.169:3300/0 conn(0x7fc9840d0640 0x7fc98415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:39.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.026+0000 7fc98c000640 1 -- 172.21.15.67:0/1919289185 >> 172.21.15.67:0/1919289185 conn(0x7fc98406bd20 msgr2=0x7fc98406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:39.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.026+0000 7fc98c000640 1 -- 172.21.15.67:0/1919289185 shutdown_connections 2024-05-30T17:04:39.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.026+0000 7fc98c000640 1 -- 172.21.15.67:0/1919289185 wait complete. 2024-05-30T17:04:39.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.027+0000 7fc98c000640 1 Processor -- start 2024-05-30T17:04:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.028+0000 7fc98c000640 1 -- start start 2024-05-30T17:04:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.028+0000 7fc98c000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9840b32c0 0x7fc98414c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.028+0000 7fc98c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9840d0640 0x7fc98409d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98c000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840d0a20 0x7fc98409d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98c000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc98415cdd0 con 0x7fc9840d0a20 2024-05-30T17:04:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98c000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc9840ad8d0 con 0x7fc9840b32c0 2024-05-30T17:04:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc98415d0b0 con 0x7fc9840d0640 2024-05-30T17:04:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840d0a20 0x7fc98409d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840d0a20 0x7fc98409d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52352/0 (socket says 172.21.15.67:52352) 2024-05-30T17:04:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98a000640 1 -- 172.21.15.67:0/3614833251 learned_addr learned my addr 172.21.15.67:0/3614833251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:39.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc989600640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9840b32c0 0x7fc98414c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:39.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc988c00640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9840d0640 0x7fc98409d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:39.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98a000640 1 -- 172.21.15.67:0/3614833251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9840d0640 msgr2=0x7fc98409d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:39.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98a000640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9840d0640 0x7fc98409d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:39.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98a000640 1 -- 172.21.15.67:0/3614833251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9840b32c0 msgr2=0x7fc98414c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:39.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98a000640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9840b32c0 0x7fc98414c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:39.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.029+0000 7fc98a000640 1 -- 172.21.15.67:0/3614833251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98409a830 con 0x7fc9840d0a20 2024-05-30T17:04:39.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.030+0000 7fc98a000640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840d0a20 0x7fc98409d9d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009eb0 tx=0x7fc97c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:39.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.030+0000 7fc981600640 1 -- 172.21.15.67:0/3614833251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c040070 con 0x7fc9840d0a20 2024-05-30T17:04:39.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.030+0000 7fc981600640 1 -- 172.21.15.67:0/3614833251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c03c830 con 0x7fc9840d0a20 2024-05-30T17:04:39.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.030+0000 7fc981600640 1 -- 172.21.15.67:0/3614833251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c03cbe0 con 0x7fc9840d0a20 2024-05-30T17:04:39.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.030+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc984155a50 con 0x7fc9840d0a20 2024-05-30T17:04:39.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.030+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc984156010 con 0x7fc9840d0a20 2024-05-30T17:04:39.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.031+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9840689e0 con 0x7fc9840d0a20 2024-05-30T17:04:39.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.033+0000 7fc981600640 1 -- 172.21.15.67:0/3614833251 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fc97c039040 con 0x7fc9840d0a20 2024-05-30T17:04:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.033+0000 7fc981600640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc96406e510 0x7fc9640709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.034+0000 7fc981600640 1 -- 172.21.15.67:0/3614833251 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5965+0+0 (secure 0 0 0) 0x7fc97c0ac200 con 0x7fc9840d0a20 2024-05-30T17:04:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.034+0000 7fc989600640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc96406e510 0x7fc9640709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.034+0000 7fc989600640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc96406e510 0x7fc9640709d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc978009ad0 tx=0x7fc978008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.038+0000 7fc981600640 1 -- 172.21.15.67:0/3614833251 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc97c07c1b0 con 0x7fc9840d0a20 2024-05-30T17:04:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.285+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc98409aa00 con 0x7fc9840d0a20 2024-05-30T17:04:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.802+0000 7fc981600640 1 -- 172.21.15.67:0/3614833251 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7fc97c07bab0 con 0x7fc9840d0a20 2024-05-30T17:04:39.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.805+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc96406e510 msgr2=0x7fc9640709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:39.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.805+0000 7fc98c000640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc96406e510 0x7fc9640709d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc978009ad0 tx=0x7fc978008040 comp rx=0 tx=0).stop 2024-05-30T17:04:39.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.805+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840d0a20 msgr2=0x7fc98409d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:39.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.805+0000 7fc98c000640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840d0a20 0x7fc98409d9d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009eb0 tx=0x7fc97c004a40 comp rx=0 tx=0).stop 2024-05-30T17:04:39.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.805+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 shutdown_connections 2024-05-30T17:04:39.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.805+0000 7fc98c000640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc96406e510 0x7fc9640709d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:39.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.806+0000 7fc98c000640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9840d0a20 0x7fc98409d9d0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:39.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.806+0000 7fc98c000640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9840d0640 0x7fc98409d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:39.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.806+0000 7fc98c000640 1 --2- 172.21.15.67:0/3614833251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9840b32c0 0x7fc98414c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:39.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.806+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 >> 172.21.15.67:0/3614833251 conn(0x7fc98406bd20 msgr2=0x7fc98415efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:39.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.806+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 shutdown_connections 2024-05-30T17:04:39.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:39.806+0000 7fc98c000640 1 -- 172.21.15.67:0/3614833251 wait complete. 2024-05-30T17:04:39.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-30T17:04:40.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.052+0000 7f3ae5800640 1 Processor -- start 2024-05-30T17:04:40.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.053+0000 7f3ae5800640 1 -- start start 2024-05-30T17:04:40.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.054+0000 7f3ae5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00af6f0 0x7f3ae01623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:40.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.054+0000 7f3ae5800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3ae01480a0 0x7f3ae00d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:40.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.054+0000 7f3ae5800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae0162cb0 con 0x7f3ae00d0970 2024-05-30T17:04:40.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.054+0000 7f3ae5800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae00d0de0 con 0x7f3ae01480a0 2024-05-30T17:04:40.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.054+0000 7f3ae5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae0162f90 con 0x7f3ae00af6f0 2024-05-30T17:04:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3ae4e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3ae00d0970 0x7f3ae0144060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42406/0 (socket says 172.21.15.67:42406) 2024-05-30T17:04:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3ae4e00640 1 -- 172.21.15.67:0/1876632814 learned_addr learned my addr 172.21.15.67:0/1876632814 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3ad7e00640 1 --2- 172.21.15.67:0/1876632814 >> v2:172.21.15.67:3300/0 conn(0x7f3ae01480a0 0x7f3ae00d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3adfe00640 1 --2- 172.21.15.67:0/1876632814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00af6f0 0x7f3ae01623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3adf400640 1 -- 172.21.15.67:0/1876632814 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1935542398 0 0) 0x7f3ae0162cb0 con 0x7f3ae00d0970 2024-05-30T17:04:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3adf400640 1 -- 172.21.15.67:0/1876632814 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ac4003520 con 0x7f3ae00d0970 2024-05-30T17:04:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3ad7e00640 1 -- 172.21.15.67:0/1876632814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00af6f0 msgr2=0x7f3ae01623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3ad7e00640 1 --2- 172.21.15.67:0/1876632814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00af6f0 0x7f3ae01623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3ad7e00640 1 -- 172.21.15.67:0/1876632814 >> v1:172.21.15.67:6789/0 conn(0x7f3ae00d0970 legacy=0x7f3ae0144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3ad7e00640 1 -- 172.21.15.67:0/1876632814 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae0163270 con 0x7f3ae01480a0 2024-05-30T17:04:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.055+0000 7f3adfe00640 1 --2- 172.21.15.67:0/1876632814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00af6f0 0x7f3ae01623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:04:40.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.057+0000 7f3ad7e00640 1 --2- 172.21.15.67:0/1876632814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae01480a0 0x7f3ae00d0430 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80097a0 tx=0x7f3ac802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1f9f34039c53a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:40.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.057+0000 7f3adf400640 1 -- 172.21.15.67:0/1876632814 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8037070 con 0x7f3ae01480a0 2024-05-30T17:04:40.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.057+0000 7f3adf400640 1 -- 172.21.15.67:0/1876632814 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac8032440 con 0x7f3ae01480a0 2024-05-30T17:04:40.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.057+0000 7f3adf400640 1 -- 172.21.15.67:0/1876632814 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8032810 con 0x7f3ae01480a0 2024-05-30T17:04:40.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.058+0000 7f3ae5800640 1 -- 172.21.15.67:0/1876632814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae01480a0 msgr2=0x7f3ae00d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:40.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.058+0000 7f3ae5800640 1 --2- 172.21.15.67:0/1876632814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae01480a0 0x7f3ae00d0430 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80097a0 tx=0x7f3ac802a180 comp rx=0 tx=0).stop 2024-05-30T17:04:40.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.058+0000 7f3ae5800640 1 -- 172.21.15.67:0/1876632814 shutdown_connections 2024-05-30T17:04:40.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.058+0000 7f3ae5800640 1 --2- 172.21.15.67:0/1876632814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae01480a0 0x7f3ae00d0430 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:40.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.058+0000 7f3ae5800640 1 --2- 172.21.15.67:0/1876632814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00af6f0 0x7f3ae01623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:40.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.058+0000 7f3ae5800640 1 -- 172.21.15.67:0/1876632814 >> 172.21.15.67:0/1876632814 conn(0x7f3ae006bae0 msgr2=0x7f3ae006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:40.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.058+0000 7f3ae5800640 1 -- 172.21.15.67:0/1876632814 shutdown_connections 2024-05-30T17:04:40.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.058+0000 7f3ae5800640 1 -- 172.21.15.67:0/1876632814 wait complete. 2024-05-30T17:04:40.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.060+0000 7f3ae5800640 1 Processor -- start 2024-05-30T17:04:40.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.061+0000 7f3ae5800640 1 -- start start 2024-05-30T17:04:40.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.062+0000 7f3ae5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae00af6f0 0x7f3ae00cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:40.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.062+0000 7f3ae5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00d0970 0x7f3ae01f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:40.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.062+0000 7f3ae5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae01480a0 0x7f3ae00cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:40.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.062+0000 7f3ae5800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01628f0 con 0x7f3ae00af6f0 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.062+0000 7f3ae5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01423b0 con 0x7f3ae01480a0 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.062+0000 7f3ae5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae0162a70 con 0x7f3ae00d0970 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.062+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae00af6f0 0x7f3ae00cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.062+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae00af6f0 0x7f3ae00cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52368/0 (socket says 172.21.15.67:52368) 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.062+0000 7f3adfe00640 1 -- 172.21.15.67:0/1712459085 learned_addr learned my addr 172.21.15.67:0/1712459085 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3ae4e00640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae01480a0 0x7f3ae00cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3adfe00640 1 -- 172.21.15.67:0/1712459085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00d0970 msgr2=0x7f3ae01f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3ad7e00640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00d0970 0x7f3ae01f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3adfe00640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00d0970 0x7f3ae01f7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3adfe00640 1 -- 172.21.15.67:0/1712459085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae01480a0 msgr2=0x7f3ae00cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3adfe00640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae01480a0 0x7f3ae00cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:40.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3adfe00640 1 -- 172.21.15.67:0/1712459085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae01599e0 con 0x7f3ae00af6f0 2024-05-30T17:04:40.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3ae4e00640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae01480a0 0x7f3ae00cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:40.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3adfe00640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae00af6f0 0x7f3ae00cb720 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000d7d0 tx=0x7f3ad000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:40.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.063+0000 7f3adcc00640 1 -- 172.21.15.67:0/1712459085 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0004030 con 0x7f3ae00af6f0 2024-05-30T17:04:40.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.064+0000 7f3adcc00640 1 -- 172.21.15.67:0/1712459085 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad00041d0 con 0x7f3ae00af6f0 2024-05-30T17:04:40.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.064+0000 7f3adcc00640 1 -- 172.21.15.67:0/1712459085 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00096a0 con 0x7f3ae00af6f0 2024-05-30T17:04:40.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.064+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae01f7740 con 0x7f3ae00af6f0 2024-05-30T17:04:40.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.064+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae00cc0b0 con 0x7f3ae00af6f0 2024-05-30T17:04:40.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.065+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae0147ec0 con 0x7f3ae00af6f0 2024-05-30T17:04:40.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.066+0000 7f3adcc00640 1 -- 172.21.15.67:0/1712459085 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f3ad0009840 con 0x7f3ae00af6f0 2024-05-30T17:04:40.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.067+0000 7f3adcc00640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3abc06e540 0x7f3abc070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:40.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.067+0000 7f3adcc00640 1 -- 172.21.15.67:0/1712459085 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5965+0+0 (secure 0 0 0) 0x7f3ad008ad50 con 0x7f3ae00af6f0 2024-05-30T17:04:40.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.067+0000 7f3ad7e00640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3abc06e540 0x7f3abc070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:40.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.068+0000 7f3ad7e00640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3abc06e540 0x7f3abc070a00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8034000 tx=0x7f3ac802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:40.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.071+0000 7f3adcc00640 1 -- 172.21.15.67:0/1712459085 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ae0147ec0 con 0x7f3ae00af6f0 2024-05-30T17:04:40.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.278+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3ae00689e0 con 0x7f3ae00af6f0 2024-05-30T17:04:40.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.812+0000 7f3adcc00640 1 -- 172.21.15.67:0/1712459085 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) ==== 181+0+0 (secure 0 0 0) 0x7f3ad005abe0 con 0x7f3ae00af6f0 2024-05-30T17:04:40.814 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-05-30T17:04:40.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3abc06e540 msgr2=0x7f3abc070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:40.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3abc06e540 0x7f3abc070a00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8034000 tx=0x7f3ac802e000 comp rx=0 tx=0).stop 2024-05-30T17:04:40.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae00af6f0 msgr2=0x7f3ae00cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:40.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae00af6f0 0x7f3ae00cb720 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000d7d0 tx=0x7f3ad000dc90 comp rx=0 tx=0).stop 2024-05-30T17:04:40.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 shutdown_connections 2024-05-30T17:04:40.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f3abc06e540 0x7f3abc070a00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:40.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae01480a0 0x7f3ae00cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:40.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00d0970 0x7f3ae01f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:40.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 --2- 172.21.15.67:0/1712459085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ae00af6f0 0x7f3ae00cb720 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:40.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 >> 172.21.15.67:0/1712459085 conn(0x7f3ae006bae0 msgr2=0x7f3ae006e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:40.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 shutdown_connections 2024-05-30T17:04:40.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:40.817+0000 7f3ae5800640 1 -- 172.21.15.67:0/1712459085 wait complete. 2024-05-30T17:04:40.846 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-30T17:04:41.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.059+0000 7f9728e00640 1 Processor -- start 2024-05-30T17:04:41.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.059+0000 7f9728e00640 1 -- start start 2024-05-30T17:04:41.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.060+0000 7f9728e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f972415e830 0x7f972406dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:41.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.060+0000 7f9728e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f97240b3510 0x7f972406f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:41.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.060+0000 7f9728e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f972406e190 con 0x7f9724071c00 2024-05-30T17:04:41.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.060+0000 7f9728e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f972406f770 con 0x7f97240b3510 2024-05-30T17:04:41.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.060+0000 7f9728e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f972406e470 con 0x7f972415e830 2024-05-30T17:04:41.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.061+0000 7f971be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f97240b3510 0x7f972406f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:41.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.061+0000 7f971be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f97240b3510 0x7f972406f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:48374/0 (socket says 172.21.15.67:48374) 2024-05-30T17:04:41.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.061+0000 7f971be00640 1 -- 172.21.15.67:0/3260506951 learned_addr learned my addr 172.21.15.67:0/3260506951 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.062+0000 7f971be00640 1 -- 172.21.15.67:0/3260506951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f972415e830 msgr2=0x7f972406dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.062+0000 7f971be00640 1 --2- 172.21.15.67:0/3260506951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f972415e830 0x7f972406dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.062+0000 7f971be00640 1 -- 172.21.15.67:0/3260506951 >> v1:172.21.15.169:6789/0 conn(0x7f9724071c00 legacy=0x7f97240ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.062+0000 7f971be00640 1 -- 172.21.15.67:0/3260506951 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972406e750 con 0x7f97240b3510 2024-05-30T17:04:41.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.064+0000 7f971be00640 1 --2- 172.21.15.67:0/3260506951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97240b3510 0x7f972406f230 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f970c009870 tx=0x7f970c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f672bf15f4f74d4a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:41.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.064+0000 7f9721400640 1 -- 172.21.15.67:0/3260506951 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c037070 con 0x7f97240b3510 2024-05-30T17:04:41.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.064+0000 7f9721400640 1 -- 172.21.15.67:0/3260506951 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970c032440 con 0x7f97240b3510 2024-05-30T17:04:41.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.064+0000 7f9721400640 1 -- 172.21.15.67:0/3260506951 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c032810 con 0x7f97240b3510 2024-05-30T17:04:41.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.065+0000 7f9728e00640 1 -- 172.21.15.67:0/3260506951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97240b3510 msgr2=0x7f972406f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:41.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.065+0000 7f9728e00640 1 --2- 172.21.15.67:0/3260506951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97240b3510 0x7f972406f230 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f970c009870 tx=0x7f970c02a210 comp rx=0 tx=0).stop 2024-05-30T17:04:41.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.065+0000 7f9728e00640 1 -- 172.21.15.67:0/3260506951 shutdown_connections 2024-05-30T17:04:41.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.065+0000 7f9728e00640 1 --2- 172.21.15.67:0/3260506951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97240b3510 0x7f972406f230 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.065+0000 7f9728e00640 1 --2- 172.21.15.67:0/3260506951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f972415e830 0x7f972406dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.065+0000 7f9728e00640 1 -- 172.21.15.67:0/3260506951 >> 172.21.15.67:0/3260506951 conn(0x7f97240b1e50 msgr2=0x7f97240b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:41.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.065+0000 7f9728e00640 1 -- 172.21.15.67:0/3260506951 shutdown_connections 2024-05-30T17:04:41.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.065+0000 7f9728e00640 1 -- 172.21.15.67:0/3260506951 wait complete. 2024-05-30T17:04:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.067+0000 7f9728e00640 1 Processor -- start 2024-05-30T17:04:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.067+0000 7f9728e00640 1 -- start start 2024-05-30T17:04:41.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.068+0000 7f9728e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9724071c00 0x7f972414ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.068+0000 7f9728e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97240b3510 0x7f972415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.068+0000 7f9728e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f972415e830 0x7f9724154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.068+0000 7f9728e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f972415ec10 con 0x7f972415e830 2024-05-30T17:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.068+0000 7f9728e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f972406ed10 con 0x7f9724071c00 2024-05-30T17:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.068+0000 7f9728e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f97240728c0 con 0x7f97240b3510 2024-05-30T17:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.068+0000 7f9722800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f972415e830 0x7f9724154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.068+0000 7f9722800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f972415e830 0x7f9724154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59648/0 (socket says 172.21.15.67:59648) 2024-05-30T17:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.068+0000 7f9722800640 1 -- 172.21.15.67:0/4166683421 learned_addr learned my addr 172.21.15.67:0/4166683421 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f971be00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97240b3510 0x7f972415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f9722800640 1 -- 172.21.15.67:0/4166683421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97240b3510 msgr2=0x7f972415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f9722800640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97240b3510 0x7f972415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f9722800640 1 -- 172.21.15.67:0/4166683421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9724071c00 msgr2=0x7f972414ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f9722800640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9724071c00 0x7f972414ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f9722800640 1 -- 172.21.15.67:0/4166683421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972415fb30 con 0x7f972415e830 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f971be00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97240b3510 0x7f972415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f9722800640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f972415e830 0x7f9724154280 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f971c00c780 tx=0x7f971c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f971a000640 1 -- 172.21.15.67:0/4166683421 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c017070 con 0x7f972415e830 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f971a000640 1 -- 172.21.15.67:0/4166683421 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971c003da0 con 0x7f972415e830 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f971a000640 1 -- 172.21.15.67:0/4166683421 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c0052b0 con 0x7f972415e830 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.069+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972415f2d0 con 0x7f972415e830 2024-05-30T17:04:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.070+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972415f850 con 0x7f972415e830 2024-05-30T17:04:41.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.071+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97240689e0 con 0x7f972415e830 2024-05-30T17:04:41.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.072+0000 7f971a000640 1 -- 172.21.15.67:0/4166683421 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f971c01a070 con 0x7f972415e830 2024-05-30T17:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.073+0000 7f971a000640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f970006e510 0x7f97000709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.073+0000 7f971a000640 1 -- 172.21.15.67:0/4166683421 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5527+0+0 (secure 0 0 0) 0x7f971c08a180 con 0x7f972415e830 2024-05-30T17:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.073+0000 7f9721e00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f970006e510 0x7f97000709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.074+0000 7f9721e00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f970006e510 0x7f97000709d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9714009670 tx=0x7f9714006bf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:41.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.077+0000 7f971a000640 1 -- 172.21.15.67:0/4166683421 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f971c05a010 con 0x7f972415e830 2024-05-30T17:04:41.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.281+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f97240b0e90 con 0x7f972415e830 2024-05-30T17:04:41.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.830+0000 7f971a000640 1 -- 172.21.15.67:0/4166683421 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) ==== 169+0+0 (secure 0 0 0) 0x7f971c059910 con 0x7f972415e830 2024-05-30T17:04:41.832 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-05-30T17:04:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.833+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f970006e510 msgr2=0x7f97000709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.833+0000 7f9728e00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f970006e510 0x7f97000709d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9714009670 tx=0x7f9714006bf0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.833+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f972415e830 msgr2=0x7f9724154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.833+0000 7f9728e00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f972415e830 0x7f9724154280 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f971c00c780 tx=0x7f971c00cc40 comp rx=0 tx=0).stop 2024-05-30T17:04:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.833+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 shutdown_connections 2024-05-30T17:04:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.834+0000 7f9728e00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f970006e510 0x7f97000709d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.834+0000 7f9728e00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f972415e830 0x7f9724154280 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.834+0000 7f9728e00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97240b3510 0x7f972415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.834+0000 7f9728e00640 1 --2- 172.21.15.67:0/4166683421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9724071c00 0x7f972414ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:41.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.834+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 >> 172.21.15.67:0/4166683421 conn(0x7f97240b1e50 msgr2=0x7f97240b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:41.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.835+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 shutdown_connections 2024-05-30T17:04:41.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:41.836+0000 7f9728e00640 1 -- 172.21.15.67:0/4166683421 wait complete. 2024-05-30T17:04:41.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-30T17:04:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.060+0000 7fe91ea00640 1 Processor -- start 2024-05-30T17:04:42.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.061+0000 7fe91ea00640 1 -- start start 2024-05-30T17:04:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.062+0000 7fe91ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe918162a90 0x7fe9180703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.062+0000 7fe91ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9180cd390 0x7fe9180708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe91806b900 0x7fe91806bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91ea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe9180ceae0 con 0x7fe918162a90 2024-05-30T17:04:42.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe9180ce650 con 0x7fe91806b900 2024-05-30T17:04:42.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9180cda80 con 0x7fe9180cd390 2024-05-30T17:04:42.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe918162a90 0x7fe9180703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:42.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe918162a90 0x7fe9180703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59654/0 (socket says 172.21.15.67:59654) 2024-05-30T17:04:42.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91d600640 1 -- 172.21.15.67:0/3384664954 learned_addr learned my addr 172.21.15.67:0/3384664954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:42.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91d600640 1 -- 172.21.15.67:0/3384664954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9180cd390 msgr2=0x7fe9180708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:42.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91d600640 1 --2- 172.21.15.67:0/3384664954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9180cd390 0x7fe9180708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91d600640 1 -- 172.21.15.67:0/3384664954 >> v2:172.21.15.169:3300/0 conn(0x7fe91806b900 msgr2=0x7fe91806bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:42.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91d600640 1 --2- 172.21.15.67:0/3384664954 >> v2:172.21.15.169:3300/0 conn(0x7fe91806b900 0x7fe91806bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.063+0000 7fe91d600640 1 -- 172.21.15.67:0/3384664954 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9180cdd20 con 0x7fe918162a90 2024-05-30T17:04:42.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.065+0000 7fe91d600640 1 --2- 172.21.15.67:0/3384664954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918162a90 0x7fe9180703a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009870 tx=0x7fe90c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de6a3026a162904 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:42.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.065+0000 7fe917e00640 1 -- 172.21.15.67:0/3384664954 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c037070 con 0x7fe918162a90 2024-05-30T17:04:42.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.065+0000 7fe917e00640 1 -- 172.21.15.67:0/3384664954 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c032440 con 0x7fe918162a90 2024-05-30T17:04:42.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.065+0000 7fe917e00640 1 -- 172.21.15.67:0/3384664954 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c032810 con 0x7fe918162a90 2024-05-30T17:04:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.066+0000 7fe91ea00640 1 -- 172.21.15.67:0/3384664954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918162a90 msgr2=0x7fe9180703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.066+0000 7fe91ea00640 1 --2- 172.21.15.67:0/3384664954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918162a90 0x7fe9180703a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009870 tx=0x7fe90c02a210 comp rx=0 tx=0).stop 2024-05-30T17:04:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.066+0000 7fe91ea00640 1 -- 172.21.15.67:0/3384664954 shutdown_connections 2024-05-30T17:04:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.066+0000 7fe91ea00640 1 --2- 172.21.15.67:0/3384664954 >> v2:172.21.15.169:3300/0 conn(0x7fe91806b900 0x7fe91806bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.066+0000 7fe91ea00640 1 --2- 172.21.15.67:0/3384664954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9180cd390 0x7fe9180708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.066+0000 7fe91ea00640 1 --2- 172.21.15.67:0/3384664954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918162a90 0x7fe9180703a0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.066+0000 7fe91ea00640 1 -- 172.21.15.67:0/3384664954 >> 172.21.15.67:0/3384664954 conn(0x7fe9180b2ed0 msgr2=0x7fe9180b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.066+0000 7fe91ea00640 1 -- 172.21.15.67:0/3384664954 shutdown_connections 2024-05-30T17:04:42.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.066+0000 7fe91ea00640 1 -- 172.21.15.67:0/3384664954 wait complete. 2024-05-30T17:04:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.068+0000 7fe91ea00640 1 Processor -- start 2024-05-30T17:04:42.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.069+0000 7fe91ea00640 1 -- start start 2024-05-30T17:04:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.070+0000 7fe91ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe91806b900 0x7fe91815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.070+0000 7fe91ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9180cd390 0x7fe918157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.070+0000 7fe91ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918162a90 0x7fe918157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.070+0000 7fe91ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe918162e70 con 0x7fe91806b900 2024-05-30T17:04:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.070+0000 7fe91ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe9180ce2e0 con 0x7fe918162a90 2024-05-30T17:04:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.070+0000 7fe91ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9180b3c70 con 0x7fe9180cd390 2024-05-30T17:04:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.070+0000 7fe91d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe91806b900 0x7fe91815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.070+0000 7fe91d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe91806b900 0x7fe91815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59670/0 (socket says 172.21.15.67:59670) 2024-05-30T17:04:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.070+0000 7fe91d600640 1 -- 172.21.15.67:0/2961371694 learned_addr learned my addr 172.21.15.67:0/2961371694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.071+0000 7fe91e000640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918162a90 0x7fe918157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.071+0000 7fe91cc00640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9180cd390 0x7fe918157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.071+0000 7fe91d600640 1 -- 172.21.15.67:0/2961371694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9180cd390 msgr2=0x7fe918157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.071+0000 7fe91d600640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9180cd390 0x7fe918157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.071+0000 7fe91d600640 1 -- 172.21.15.67:0/2961371694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918162a90 msgr2=0x7fe918157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.071+0000 7fe91d600640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918162a90 0x7fe918157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.071+0000 7fe91d600640 1 -- 172.21.15.67:0/2961371694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9181fc310 con 0x7fe91806b900 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.071+0000 7fe91e000640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918162a90 0x7fe918157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.071+0000 7fe91d600640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe91806b900 0x7fe91815bf30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009870 tx=0x7fe90c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.072+0000 7fe915600640 1 -- 172.21.15.67:0/2961371694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c041070 con 0x7fe91806b900 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.072+0000 7fe915600640 1 -- 172.21.15.67:0/2961371694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c03c870 con 0x7fe91806b900 2024-05-30T17:04:42.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.072+0000 7fe915600640 1 -- 172.21.15.67:0/2961371694 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c037040 con 0x7fe91806b900 2024-05-30T17:04:42.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.072+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9180b1710 con 0x7fe91806b900 2024-05-30T17:04:42.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.072+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9181fc890 con 0x7fe91806b900 2024-05-30T17:04:42.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.073+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9180b3610 con 0x7fe91806b900 2024-05-30T17:04:42.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.074+0000 7fe915600640 1 -- 172.21.15.67:0/2961371694 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fe90c044070 con 0x7fe91806b900 2024-05-30T17:04:42.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.075+0000 7fe915600640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8fc06e540 0x7fe8fc070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.076+0000 7fe915600640 1 -- 172.21.15.67:0/2961371694 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5055+0+0 (secure 0 0 0) 0x7fe90c0ab690 con 0x7fe91806b900 2024-05-30T17:04:42.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.076+0000 7fe91cc00640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8fc06e540 0x7fe8fc070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:42.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.077+0000 7fe91cc00640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8fc06e540 0x7fe8fc070a00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe91815c810 tx=0x7fe908008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:42.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.081+0000 7fe915600640 1 -- 172.21.15.67:0/2961371694 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9180b3610 con 0x7fe91806b900 2024-05-30T17:04:42.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.288+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe9180690f0 con 0x7fe91806b900 2024-05-30T17:04:42.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.295+0000 7fe915600640 1 -- 172.21.15.67:0/2961371694 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fe90c07b6e0 con 0x7fe91806b900 2024-05-30T17:04:42.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.298+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8fc06e540 msgr2=0x7fe8fc070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:42.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.298+0000 7fe91ea00640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8fc06e540 0x7fe8fc070a00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe91815c810 tx=0x7fe908008040 comp rx=0 tx=0).stop 2024-05-30T17:04:42.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.298+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe91806b900 msgr2=0x7fe91815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:42.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.298+0000 7fe91ea00640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe91806b900 0x7fe91815bf30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009870 tx=0x7fe90c0046c0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.299+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 shutdown_connections 2024-05-30T17:04:42.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.299+0000 7fe91ea00640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8fc06e540 0x7fe8fc070a00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.299+0000 7fe91ea00640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918162a90 0x7fe918157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.299+0000 7fe91ea00640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9180cd390 0x7fe918157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.299+0000 7fe91ea00640 1 --2- 172.21.15.67:0/2961371694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe91806b900 0x7fe91815bf30 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.299+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 >> 172.21.15.67:0/2961371694 conn(0x7fe9180b2ed0 msgr2=0x7fe9180d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:42.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.299+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 shutdown_connections 2024-05-30T17:04:42.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.299+0000 7fe91ea00640 1 -- 172.21.15.67:0/2961371694 wait complete. 2024-05-30T17:04:42.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled' 2024-05-30T17:04:42.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.531+0000 7fe7a1000640 1 Processor -- start 2024-05-30T17:04:42.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.532+0000 7fe7a1000640 1 -- start start 2024-05-30T17:04:42.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.533+0000 7fe7a1000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe79c162d20 0x7fe79c071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.533+0000 7fe7a1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe79c072100 0x7fe79c0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.533+0000 7fe7a1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79c06f590 con 0x7fe79c162940 2024-05-30T17:04:42.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe7a1000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe79c072530 con 0x7fe79c162d20 2024-05-30T17:04:42.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe7a1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe79c06f870 con 0x7fe79c072100 2024-05-30T17:04:42.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe793e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe79c162d20 0x7fe79c071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:42.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe793e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe79c162d20 0x7fe79c071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59674/0 (socket says 172.21.15.67:59674) 2024-05-30T17:04:42.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe793e00640 1 -- 172.21.15.67:0/2666534382 learned_addr learned my addr 172.21.15.67:0/2666534382 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:42.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe79be00640 1 --2- 172.21.15.67:0/2666534382 >> v2:172.21.15.169:3300/0 conn(0x7fe79c072100 0x7fe79c0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:42.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe793e00640 1 -- 172.21.15.67:0/2666534382 >> v2:172.21.15.169:3300/0 conn(0x7fe79c072100 msgr2=0x7fe79c0adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:42.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe793e00640 1 --2- 172.21.15.67:0/2666534382 >> v2:172.21.15.169:3300/0 conn(0x7fe79c072100 0x7fe79c0adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe793e00640 1 -- 172.21.15.67:0/2666534382 >> v1:172.21.15.169:6789/0 conn(0x7fe79c162940 legacy=0x7fe79c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:42.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe793e00640 1 -- 172.21.15.67:0/2666534382 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c0ad3d0 con 0x7fe79c162d20 2024-05-30T17:04:42.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.534+0000 7fe79be00640 1 --2- 172.21.15.67:0/2666534382 >> v2:172.21.15.169:3300/0 conn(0x7fe79c072100 0x7fe79c0adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:42.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.536+0000 7fe793e00640 1 --2- 172.21.15.67:0/2666534382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c162d20 0x7fe79c071bc0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0738a0 tx=0x7fe78402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44523219cf9a5ff server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:42.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.536+0000 7fe79aa00640 1 -- 172.21.15.67:0/2666534382 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784037070 con 0x7fe79c162d20 2024-05-30T17:04:42.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.536+0000 7fe79aa00640 1 -- 172.21.15.67:0/2666534382 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78402ae10 con 0x7fe79c162d20 2024-05-30T17:04:42.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.536+0000 7fe79aa00640 1 -- 172.21.15.67:0/2666534382 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784032660 con 0x7fe79c162d20 2024-05-30T17:04:42.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.537+0000 7fe7a1000640 1 -- 172.21.15.67:0/2666534382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c162d20 msgr2=0x7fe79c071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:42.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.537+0000 7fe7a1000640 1 --2- 172.21.15.67:0/2666534382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c162d20 0x7fe79c071bc0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0738a0 tx=0x7fe78402a100 comp rx=0 tx=0).stop 2024-05-30T17:04:42.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.537+0000 7fe7a1000640 1 -- 172.21.15.67:0/2666534382 shutdown_connections 2024-05-30T17:04:42.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.537+0000 7fe7a1000640 1 --2- 172.21.15.67:0/2666534382 >> v2:172.21.15.169:3300/0 conn(0x7fe79c072100 0x7fe79c0adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.537+0000 7fe7a1000640 1 --2- 172.21.15.67:0/2666534382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c162d20 0x7fe79c071bc0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.537+0000 7fe7a1000640 1 -- 172.21.15.67:0/2666534382 >> 172.21.15.67:0/2666534382 conn(0x7fe79c0b1fc0 msgr2=0x7fe79c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:42.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.537+0000 7fe7a1000640 1 -- 172.21.15.67:0/2666534382 shutdown_connections 2024-05-30T17:04:42.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.537+0000 7fe7a1000640 1 -- 172.21.15.67:0/2666534382 wait complete. 2024-05-30T17:04:42.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.539+0000 7fe7a1000640 1 Processor -- start 2024-05-30T17:04:42.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.539+0000 7fe7a1000640 1 -- start start 2024-05-30T17:04:42.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe7a1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c072100 0x7fe79c1fbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe7a1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79c162940 0x7fe79c09c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe7a1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe79c162d20 0x7fe79c098980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe7a1000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c06df60 con 0x7fe79c072100 2024-05-30T17:04:42.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe7a1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c06dde0 con 0x7fe79c162940 2024-05-30T17:04:42.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe7a1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c0ae0f0 con 0x7fe79c162d20 2024-05-30T17:04:42.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe79b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c072100 0x7fe79c1fbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:42.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe79b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c072100 0x7fe79c1fbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59680/0 (socket says 172.21.15.67:59680) 2024-05-30T17:04:42.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe79b400640 1 -- 172.21.15.67:0/1511752494 learned_addr learned my addr 172.21.15.67:0/1511752494 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:42.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe79b400640 1 -- 172.21.15.67:0/1511752494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe79c162d20 msgr2=0x7fe79c098980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:42.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe79b400640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe79c162d20 0x7fe79c098980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.540+0000 7fe79b400640 1 -- 172.21.15.67:0/1511752494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79c162940 msgr2=0x7fe79c09c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:42.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.541+0000 7fe79b400640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79c162940 0x7fe79c09c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:42.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.541+0000 7fe79b400640 1 -- 172.21.15.67:0/1511752494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c09a450 con 0x7fe79c072100 2024-05-30T17:04:42.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.541+0000 7fe79b400640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c072100 0x7fe79c1fbb60 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c460 tx=0x7fe78c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:42.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.541+0000 7fe793400640 1 -- 172.21.15.67:0/1511752494 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c016020 con 0x7fe79c072100 2024-05-30T17:04:42.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.541+0000 7fe793400640 1 -- 172.21.15.67:0/1511752494 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c012870 con 0x7fe79c072100 2024-05-30T17:04:42.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.541+0000 7fe793400640 1 -- 172.21.15.67:0/1511752494 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c012c60 con 0x7fe79c072100 2024-05-30T17:04:42.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.541+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c09a730 con 0x7fe79c072100 2024-05-30T17:04:42.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.541+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c09acf0 con 0x7fe79c072100 2024-05-30T17:04:42.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.542+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c0689e0 con 0x7fe79c072100 2024-05-30T17:04:42.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.544+0000 7fe793400640 1 -- 172.21.15.67:0/1511752494 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fe78c019020 con 0x7fe79c072100 2024-05-30T17:04:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.545+0000 7fe793400640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77806e510 0x7fe7780709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.545+0000 7fe793400640 1 -- 172.21.15.67:0/1511752494 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5055+0+0 (secure 0 0 0) 0x7fe78c08a7b0 con 0x7fe79c072100 2024-05-30T17:04:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.545+0000 7fe793e00640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77806e510 0x7fe7780709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.546+0000 7fe793e00640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77806e510 0x7fe7780709d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe784034000 tx=0x7fe78402c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.549+0000 7fe793400640 1 -- 172.21.15.67:0/1511752494 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe78c020030 con 0x7fe79c072100 2024-05-30T17:04:42.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:42.853+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]} v 0) -- 0x7fe79c0b4ee0 con 0x7fe79c072100 2024-05-30T17:04:43.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.836+0000 7fe793400640 1 -- 172.21.15.67:0/1511752494 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]}]=0 v872) ==== 163+0+0 (secure 0 0 0) 0x7fe78c05a6e0 con 0x7fe79c072100 2024-05-30T17:04:43.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77806e510 msgr2=0x7fe7780709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77806e510 0x7fe7780709d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe784034000 tx=0x7fe78402c000 comp rx=0 tx=0).stop 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c072100 msgr2=0x7fe79c1fbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c072100 0x7fe79c1fbb60 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c460 tx=0x7fe78c00c920 comp rx=0 tx=0).stop 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 shutdown_connections 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe77806e510 0x7fe7780709d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe79c162d20 0x7fe79c098980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe79c162940 0x7fe79c09c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 --2- 172.21.15.67:0/1511752494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe79c072100 0x7fe79c1fbb60 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 >> 172.21.15.67:0/1511752494 conn(0x7fe79c0b1fc0 msgr2=0x7fe79c0b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 shutdown_connections 2024-05-30T17:04:43.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:43.840+0000 7fe7a1000640 1 -- 172.21.15.67:0/1511752494 wait complete. 2024-05-30T17:04:43.865 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-05-30T17:04:43.865 INFO:tasks.cephfs_test_runner:Starting test: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-05-30T17:04:43.865 INFO:tasks.cephfs_test_runner:test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T17:04:43.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:04:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.076+0000 7f29c1400640 1 Processor -- start 2024-05-30T17:04:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.077+0000 7f29c1400640 1 -- start start 2024-05-30T17:04:44.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.078+0000 7f29c1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f29bc0b5560 0x7f29bc0ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.078+0000 7f29c1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29bc0b47f0 0x7f29bc0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.078+0000 7f29c1400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29bc09ada0 con 0x7f29bc15b7c0 2024-05-30T17:04:44.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.078+0000 7f29c1400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f29bc0ce3b0 con 0x7f29bc0b5560 2024-05-30T17:04:44.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.078+0000 7f29c1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f29bc0b36e0 con 0x7f29bc0b47f0 2024-05-30T17:04:44.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.079+0000 7f29b3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f29bc0b5560 0x7f29bc0ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.079+0000 7f29b3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f29bc0b5560 0x7f29bc0ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59684/0 (socket says 172.21.15.67:59684) 2024-05-30T17:04:44.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.079+0000 7f29b3e00640 1 -- 172.21.15.67:0/880290645 learned_addr learned my addr 172.21.15.67:0/880290645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:44.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.079+0000 7f29bbe00640 1 --1- 172.21.15.67:0/880290645 >> v1:172.21.15.67:6789/0 conn(0x7f29bc15b7c0 0x7f29bc0b4e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58970/0 (socket says 172.21.15.67:58970) 2024-05-30T17:04:44.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.079+0000 7f29c0a00640 1 --2- 172.21.15.67:0/880290645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29bc0b47f0 0x7f29bc0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.079+0000 7f29b3e00640 1 -- 172.21.15.67:0/880290645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29bc0b47f0 msgr2=0x7f29bc0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.079+0000 7f29b3e00640 1 --2- 172.21.15.67:0/880290645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29bc0b47f0 0x7f29bc0cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.079+0000 7f29b3e00640 1 -- 172.21.15.67:0/880290645 >> v1:172.21.15.67:6789/0 conn(0x7f29bc15b7c0 legacy=0x7f29bc0b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.079+0000 7f29b3e00640 1 -- 172.21.15.67:0/880290645 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc0b39c0 con 0x7f29bc0b5560 2024-05-30T17:04:44.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.080+0000 7f29b3e00640 1 --2- 172.21.15.67:0/880290645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc0b5560 0x7f29bc0ceb20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009870 tx=0x7f29a402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c71ccbd54ac82e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:44.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.081+0000 7f29bb400640 1 -- 172.21.15.67:0/880290645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4037070 con 0x7f29bc0b5560 2024-05-30T17:04:44.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.081+0000 7f29bb400640 1 -- 172.21.15.67:0/880290645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a4032440 con 0x7f29bc0b5560 2024-05-30T17:04:44.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.081+0000 7f29bb400640 1 -- 172.21.15.67:0/880290645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4032810 con 0x7f29bc0b5560 2024-05-30T17:04:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.081+0000 7f29c1400640 1 -- 172.21.15.67:0/880290645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc0b5560 msgr2=0x7f29bc0ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.081+0000 7f29c1400640 1 --2- 172.21.15.67:0/880290645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc0b5560 0x7f29bc0ceb20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009870 tx=0x7f29a402a1c0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.081+0000 7f29c1400640 1 -- 172.21.15.67:0/880290645 shutdown_connections 2024-05-30T17:04:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.081+0000 7f29c1400640 1 --2- 172.21.15.67:0/880290645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29bc0b47f0 0x7f29bc0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.081+0000 7f29c1400640 1 --2- 172.21.15.67:0/880290645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc0b5560 0x7f29bc0ceb20 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.082+0000 7f29c1400640 1 -- 172.21.15.67:0/880290645 >> 172.21.15.67:0/880290645 conn(0x7f29bc0b6a10 msgr2=0x7f29bc0b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.082+0000 7f29c1400640 1 -- 172.21.15.67:0/880290645 shutdown_connections 2024-05-30T17:04:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.082+0000 7f29c1400640 1 -- 172.21.15.67:0/880290645 wait complete. 2024-05-30T17:04:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.083+0000 7f29c1400640 1 Processor -- start 2024-05-30T17:04:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.083+0000 7f29c1400640 1 -- start start 2024-05-30T17:04:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.084+0000 7f29c1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29bc0b47f0 0x7f29bc0af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.084+0000 7f29c1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29bc0b5560 0x7f29bc1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.084+0000 7f29c1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc15b7c0 0x7f29bc0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.084+0000 7f29c1400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc0b42d0 con 0x7f29bc15b7c0 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.084+0000 7f29c1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc09aa30 con 0x7f29bc0b47f0 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.084+0000 7f29c1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f29bc0b4450 con 0x7f29bc0b5560 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.085+0000 7f29c0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc15b7c0 0x7f29bc0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.085+0000 7f29c0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc15b7c0 0x7f29bc0cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59688/0 (socket says 172.21.15.67:59688) 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.085+0000 7f29c0a00640 1 -- 172.21.15.67:0/1851182675 learned_addr learned my addr 172.21.15.67:0/1851182675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.085+0000 7f29bbe00640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29bc0b47f0 0x7f29bc0af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.085+0000 7f29c0a00640 1 -- 172.21.15.67:0/1851182675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29bc0b5560 msgr2=0x7f29bc1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.085+0000 7f29c0a00640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29bc0b5560 0x7f29bc1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.085+0000 7f29c0a00640 1 -- 172.21.15.67:0/1851182675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29bc0b47f0 msgr2=0x7f29bc0af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.085+0000 7f29c0a00640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29bc0b47f0 0x7f29bc0af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.085+0000 7f29c0a00640 1 -- 172.21.15.67:0/1851182675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc0af9b0 con 0x7f29bc15b7c0 2024-05-30T17:04:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.086+0000 7f29c0a00640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc15b7c0 0x7f29bc0cbe70 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f29b4007e70 tx=0x7f29b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.086+0000 7f29b8c00640 1 -- 172.21.15.67:0/1851182675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4014620 con 0x7f29bc15b7c0 2024-05-30T17:04:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.086+0000 7f29b8c00640 1 -- 172.21.15.67:0/1851182675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b4019070 con 0x7f29bc15b7c0 2024-05-30T17:04:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.086+0000 7f29b8c00640 1 -- 172.21.15.67:0/1851182675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4014a20 con 0x7f29bc15b7c0 2024-05-30T17:04:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.086+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc0afc90 con 0x7f29bc15b7c0 2024-05-30T17:04:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.086+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1f80d0 con 0x7f29bc15b7c0 2024-05-30T17:04:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.087+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc0690f0 con 0x7f29bc15b7c0 2024-05-30T17:04:44.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.088+0000 7f29b8c00640 1 -- 172.21.15.67:0/1851182675 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f29b4014bc0 con 0x7f29bc15b7c0 2024-05-30T17:04:44.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.089+0000 7f29b8c00640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f299806e510 0x7f29980709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.089+0000 7f29b8c00640 1 -- 172.21.15.67:0/1851182675 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5055+0+0 (secure 0 0 0) 0x7f29b408b440 con 0x7f29bc15b7c0 2024-05-30T17:04:44.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.089+0000 7f29bbe00640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f299806e510 0x7f29980709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.090+0000 7f29bbe00640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f299806e510 0x7f29980709d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f29ac000f60 tx=0x7f29ac007ce0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:44.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.093+0000 7f29b8c00640 1 -- 172.21.15.67:0/1851182675 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29b405b2d0 con 0x7f29bc15b7c0 2024-05-30T17:04:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.302+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29bc0b3e80 con 0x7f29bc15b7c0 2024-05-30T17:04:44.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.305+0000 7f29b8c00640 1 -- 172.21.15.67:0/1851182675 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+13197 (secure 0 0 0) 0x7f29b405abd0 con 0x7f29bc15b7c0 2024-05-30T17:04:44.307 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:44.307 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":123,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:04:41.814973+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6834/3425947383":"2024-05-31T17:04:38.792705+0000","172.21.15.169:6832/3425947383":"2024-05-31T17:04:38.792705+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:04:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f299806e510 msgr2=0x7f29980709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f299806e510 0x7f29980709d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f29ac000f60 tx=0x7f29ac007ce0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc15b7c0 msgr2=0x7f29bc0cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc15b7c0 0x7f29bc0cbe70 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f29b4007e70 tx=0x7f29b400e4b0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 shutdown_connections 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f299806e510 0x7f29980709d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29bc15b7c0 0x7f29bc0cbe70 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29bc0b5560 0x7f29bc1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 --2- 172.21.15.67:0/1851182675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29bc0b47f0 0x7f29bc0af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 >> 172.21.15.67:0/1851182675 conn(0x7f29bc0b6a10 msgr2=0x7f29bc1584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 shutdown_connections 2024-05-30T17:04:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.308+0000 7f29c1400640 1 -- 172.21.15.67:0/1851182675 wait complete. 2024-05-30T17:04:44.330 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-30T16:43:40.009725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-30T17:04:44.331 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-30T17:04:44.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.546+0000 7f5ecb200640 1 Processor -- start 2024-05-30T17:04:44.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.546+0000 7f5ecb200640 1 -- start start 2024-05-30T17:04:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.548+0000 7f5ecb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ecc0d16a0 0x7f5ecc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.548+0000 7f5ecb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5ecc0cf870 0x7f5ecc0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.548+0000 7f5ecb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ecc15ec50 con 0x7f5ecc15e870 2024-05-30T17:04:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.548+0000 7f5ecb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5ecc0d2150 con 0x7f5ecc0cf870 2024-05-30T17:04:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.548+0000 7f5ecb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5ecc0cfd20 con 0x7f5ecc0d16a0 2024-05-30T17:04:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.548+0000 7f5ec9e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5ecc15e870 0x7f5ecc0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58974/0 (socket says 172.21.15.67:58974) 2024-05-30T17:04:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.548+0000 7f5ec9e00640 1 -- 172.21.15.67:0/1906056714 learned_addr learned my addr 172.21.15.67:0/1906056714 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:44.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec9400640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ecc0d16a0 0x7f5ecc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5eca800640 1 --2- 172.21.15.67:0/1906056714 >> v2:172.21.15.169:3300/0 conn(0x7f5ecc0cf870 0x7f5ecc0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322982118 0 0) 0x7f5ecc15ec50 con 0x7f5ecc15e870 2024-05-30T17:04:44.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5eb0003520 con 0x7f5ecc15e870 2024-05-30T17:04:44.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1488651711 0 0) 0x7f5eb0003520 con 0x7f5ecc15e870 2024-05-30T17:04:44.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ecc0d16a0 msgr2=0x7f5ecc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ecc0d16a0 0x7f5ecc0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 >> v2:172.21.15.169:3300/0 conn(0x7f5ecc0cf870 msgr2=0x7f5ecc0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 --2- 172.21.15.67:0/1906056714 >> v2:172.21.15.169:3300/0 conn(0x7f5ecc0cf870 0x7f5ecc0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc0cff80 con 0x7f5ecc15e870 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f5ebc001670 con 0x7f5ecc15e870 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 >> v1:172.21.15.67:6789/0 conn(0x7f5ecc15e870 legacy=0x7f5ecc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb00039b0 0x7f5eb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5eb0004800 0x7f5eb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.549+0000 7f5ec8a00640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ecc0d16a0 0x7f5eb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0d2150 con 0x7f5eb00039b0 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0cfd20 con 0x7f5eb0004800 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb000b9c0 con 0x7f5ecc0d16a0 2024-05-30T17:04:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec9e00640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb00039b0 0x7f5eb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec9400640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5eb0004800 0x7f5eb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5eca800640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ecc0d16a0 0x7f5eb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec9e00640 1 -- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ecc0d16a0 msgr2=0x7f5eb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec9e00640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ecc0d16a0 0x7f5eb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec9e00640 1 -- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5eb0004800 msgr2=0x7f5eb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec9e00640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5eb0004800 0x7f5eb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5ec9e00640 1 -- 172.21.15.67:0/1906056714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb0003520 con 0x7f5eb00039b0 2024-05-30T17:04:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.550+0000 7f5eca800640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ecc0d16a0 0x7f5eb00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.551+0000 7f5ec9e00640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb00039b0 0x7f5eb0003dd0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c190 tx=0x7f5eb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cf67832ded4626 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc007c80 con 0x7f5eb00039b0 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc02e070 con 0x7f5eb00039b0 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5ec8a00640 1 -- 172.21.15.67:0/1906056714 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc005d30 con 0x7f5eb00039b0 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5ecb200640 1 -- 172.21.15.67:0/1906056714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb00039b0 msgr2=0x7f5eb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5ecb200640 1 --2- 172.21.15.67:0/1906056714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb00039b0 0x7f5eb0003dd0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c190 tx=0x7f5eb0005830 comp rx=0 tx=0).stop 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5eca800640 1 -- 172.21.15.67:0/1906056714 reap_dead start 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5ecb200640 1 -- 172.21.15.67:0/1906056714 shutdown_connections 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5ecb200640 1 -- 172.21.15.67:0/1906056714 >> 172.21.15.67:0/1906056714 conn(0x7f5ecc08bfc0 msgr2=0x7f5ecc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5ecb200640 1 -- 172.21.15.67:0/1906056714 shutdown_connections 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.552+0000 7f5ecb200640 1 -- 172.21.15.67:0/1906056714 wait complete. 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.554+0000 7f5ecb200640 1 Processor -- start 2024-05-30T17:04:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.554+0000 7f5ecb200640 1 -- start start 2024-05-30T17:04:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.555+0000 7f5ecb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eb00039b0 0x7f5ecc1f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.555+0000 7f5ecb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ecc0b7bb0 0x7f5ecc0b9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.555+0000 7f5ecb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ecc1600f0 0x7f5ecc160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.555+0000 7f5ecb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0d0820 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.555+0000 7f5ecb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0d0540 con 0x7f5ecc1600f0 2024-05-30T17:04:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.555+0000 7f5ecb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5ecc0d4fb0 con 0x7f5eb00039b0 2024-05-30T17:04:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5ec9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ecc0b7bb0 0x7f5ecc0b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5ec9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ecc0b7bb0 0x7f5ecc0b9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59700/0 (socket says 172.21.15.67:59700) 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5ec9400640 1 -- 172.21.15.67:0/880373387 learned_addr learned my addr 172.21.15.67:0/880373387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5eca800640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ecc1600f0 0x7f5ecc160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5ec9400640 1 -- 172.21.15.67:0/880373387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eb00039b0 msgr2=0x7f5ecc1f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5ec9400640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eb00039b0 0x7f5ecc1f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5ec9400640 1 -- 172.21.15.67:0/880373387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ecc1600f0 msgr2=0x7f5ecc160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5ec9400640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ecc1600f0 0x7f5ecc160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5ec9400640 1 -- 172.21.15.67:0/880373387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc0b8120 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.556+0000 7f5ec9400640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ecc0b7bb0 0x7f5ecc0b9000 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8011570 tx=0x7f5eb8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.557+0000 7f5ec2000640 1 -- 172.21.15.67:0/880373387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8015870 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.557+0000 7f5ec2000640 1 -- 172.21.15.67:0/880373387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb8015a10 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.557+0000 7f5ec2000640 1 -- 172.21.15.67:0/880373387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8015dc0 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.557+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc0b8400 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.557+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc0b89c0 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.558+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ecc0689e0 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.559+0000 7f5ec2000640 1 -- 172.21.15.67:0/880373387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f5eb800f460 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.561+0000 7f5ec2000640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eac06e510 0x7f5eac0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.561+0000 7f5ec2000640 1 -- 172.21.15.67:0/880373387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5055+0+0 (secure 0 0 0) 0x7f5eb808b480 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.561+0000 7f5ec9e00640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eac06e510 0x7f5eac0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.562+0000 7f5ec9e00640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eac06e510 0x7f5eac0709d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc032000 tx=0x7f5ebc02c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.565+0000 7f5ec2000640 1 -- 172.21.15.67:0/880373387 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5eb8019030 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.771+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ecc0ba500 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.772+0000 7f5ec2000640 1 -- 172.21.15.67:0/880373387 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7f5eb805b310 con 0x7f5ecc0b7bb0 2024-05-30T17:04:44.773 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-05-30T17:04:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.774+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eac06e510 msgr2=0x7f5eac0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.774+0000 7f5ecb200640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eac06e510 0x7f5eac0709d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc032000 tx=0x7f5ebc02c000 comp rx=0 tx=0).stop 2024-05-30T17:04:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.774+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ecc0b7bb0 msgr2=0x7f5ecc0b9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.774+0000 7f5ecb200640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ecc0b7bb0 0x7f5ecc0b9000 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8011570 tx=0x7f5eb8011a30 comp rx=0 tx=0).stop 2024-05-30T17:04:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.774+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 shutdown_connections 2024-05-30T17:04:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.774+0000 7f5ecb200640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5eac06e510 0x7f5eac0709d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.774+0000 7f5ecb200640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ecc1600f0 0x7f5ecc160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.774+0000 7f5ecb200640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ecc0b7bb0 0x7f5ecc0b9000 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.774+0000 7f5ecb200640 1 --2- 172.21.15.67:0/880373387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5eb00039b0 0x7f5ecc1f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.775+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 >> 172.21.15.67:0/880373387 conn(0x7f5ecc08bfc0 msgr2=0x7f5ecc06db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.775+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 shutdown_connections 2024-05-30T17:04:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:44.775+0000 7f5ecb200640 1 -- 172.21.15.67:0/880373387 wait complete. 2024-05-30T17:04:44.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_7_clones_are_cancelled' 2024-05-30T17:04:45.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.007+0000 7f8b97200640 1 Processor -- start 2024-05-30T17:04:45.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.007+0000 7f8b97200640 1 -- start start 2024-05-30T17:04:45.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.009+0000 7f8b97200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8b98156b30 0x7f8b980af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:45.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.009+0000 7f8b97200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 0x7f8b980ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.009+0000 7f8b97200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b9806f2c0 0x7f8b980ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.009+0000 7f8b97200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f8b98156f10 con 0x7f8b98156b30 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.009+0000 7f8b97200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8b9806dda0 con 0x7f8b9806f2c0 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.009+0000 7f8b97200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9806e080 con 0x7f8b9806ee40 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.009+0000 7f8b95e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8b98156b30 0x7f8b980af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.009+0000 7f8b95e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8b98156b30 0x7f8b980af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59708/0 (socket says 172.21.15.67:59708) 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.009+0000 7f8b95e00640 1 -- 172.21.15.67:0/3230489335 learned_addr learned my addr 172.21.15.67:0/3230489335 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.010+0000 7f8b95400640 1 --2- 172.21.15.67:0/3230489335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 0x7f8b980ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.010+0000 7f8b95e00640 1 -- 172.21.15.67:0/3230489335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 msgr2=0x7f8b980ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.010+0000 7f8b95e00640 1 --2- 172.21.15.67:0/3230489335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 0x7f8b980ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.010+0000 7f8b95e00640 1 -- 172.21.15.67:0/3230489335 >> v2:172.21.15.169:3300/0 conn(0x7f8b9806f2c0 msgr2=0x7f8b980ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.010+0000 7f8b96800640 1 --2- 172.21.15.67:0/3230489335 >> v2:172.21.15.169:3300/0 conn(0x7f8b9806f2c0 0x7f8b980ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.010+0000 7f8b95e00640 1 --2- 172.21.15.67:0/3230489335 >> v2:172.21.15.169:3300/0 conn(0x7f8b9806f2c0 0x7f8b980ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.010+0000 7f8b95e00640 1 -- 172.21.15.67:0/3230489335 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9806e360 con 0x7f8b98156b30 2024-05-30T17:04:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.010+0000 7f8b96800640 1 --2- 172.21.15.67:0/3230489335 >> v2:172.21.15.169:3300/0 conn(0x7f8b9806f2c0 0x7f8b980ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.011+0000 7f8b95e00640 1 --2- 172.21.15.67:0/3230489335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b98156b30 0x7f8b980af6e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8b98071410 tx=0x7f8b8802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed6e18b7f8a555b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b94a00640 1 -- 172.21.15.67:0/3230489335 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88037070 con 0x7f8b98156b30 2024-05-30T17:04:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b94a00640 1 -- 172.21.15.67:0/3230489335 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b88032440 con 0x7f8b98156b30 2024-05-30T17:04:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b94a00640 1 -- 172.21.15.67:0/3230489335 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88032810 con 0x7f8b98156b30 2024-05-30T17:04:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b97200640 1 -- 172.21.15.67:0/3230489335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b98156b30 msgr2=0x7f8b980af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b97200640 1 --2- 172.21.15.67:0/3230489335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b98156b30 0x7f8b980af6e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8b98071410 tx=0x7f8b8802a140 comp rx=0 tx=0).stop 2024-05-30T17:04:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b97200640 1 -- 172.21.15.67:0/3230489335 shutdown_connections 2024-05-30T17:04:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b97200640 1 --2- 172.21.15.67:0/3230489335 >> v2:172.21.15.169:3300/0 conn(0x7f8b9806f2c0 0x7f8b980ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b97200640 1 --2- 172.21.15.67:0/3230489335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 0x7f8b980ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b97200640 1 --2- 172.21.15.67:0/3230489335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b98156b30 0x7f8b980af6e0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.012+0000 7f8b97200640 1 -- 172.21.15.67:0/3230489335 >> 172.21.15.67:0/3230489335 conn(0x7f8b980b1fc0 msgr2=0x7f8b980b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.013+0000 7f8b97200640 1 -- 172.21.15.67:0/3230489335 shutdown_connections 2024-05-30T17:04:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.013+0000 7f8b97200640 1 -- 172.21.15.67:0/3230489335 wait complete. 2024-05-30T17:04:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.014+0000 7f8b97200640 1 Processor -- start 2024-05-30T17:04:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.015+0000 7f8b97200640 1 -- start start 2024-05-30T17:04:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.015+0000 7f8b97200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 0x7f8b9814b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.015+0000 7f8b97200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b9806f2c0 0x7f8b9809b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b97200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b98156b30 0x7f8b981f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b97200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8b980afc20 con 0x7f8b9806f2c0 2024-05-30T17:04:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b97200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8b980b2c30 con 0x7f8b98156b30 2024-05-30T17:04:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b97200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9815e020 con 0x7f8b9806ee40 2024-05-30T17:04:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b95400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b9806f2c0 0x7f8b9809b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b95400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b9806f2c0 0x7f8b9809b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59718/0 (socket says 172.21.15.67:59718) 2024-05-30T17:04:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b95400640 1 -- 172.21.15.67:0/1140434886 learned_addr learned my addr 172.21.15.67:0/1140434886 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b95e00640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 0x7f8b9814b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b96800640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b98156b30 0x7f8b981f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b95400640 1 -- 172.21.15.67:0/1140434886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 msgr2=0x7f8b9814b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b95400640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 0x7f8b9814b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b95400640 1 -- 172.21.15.67:0/1140434886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b98156b30 msgr2=0x7f8b981f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b95400640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b98156b30 0x7f8b981f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.016+0000 7f8b95400640 1 -- 172.21.15.67:0/1140434886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b981f4de0 con 0x7f8b9806f2c0 2024-05-30T17:04:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.017+0000 7f8b95400640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b9806f2c0 0x7f8b9809b8f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d830 tx=0x7f8b8400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.017+0000 7f8b8e000640 1 -- 172.21.15.67:0/1140434886 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84018030 con 0x7f8b9806f2c0 2024-05-30T17:04:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.017+0000 7f8b8e000640 1 -- 172.21.15.67:0/1140434886 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8400be10 con 0x7f8b9806f2c0 2024-05-30T17:04:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.017+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9815d410 con 0x7f8b9806f2c0 2024-05-30T17:04:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.017+0000 7f8b8e000640 1 -- 172.21.15.67:0/1140434886 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b840098b0 con 0x7f8b9806f2c0 2024-05-30T17:04:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.017+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9815d9d0 con 0x7f8b9806f2c0 2024-05-30T17:04:45.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.018+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b980689e0 con 0x7f8b9806f2c0 2024-05-30T17:04:45.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.019+0000 7f8b8e000640 1 -- 172.21.15.67:0/1140434886 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f8b8401b070 con 0x7f8b9806f2c0 2024-05-30T17:04:45.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.022+0000 7f8b8e000640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8b7806e510 0x7f8b780709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:45.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.022+0000 7f8b8e000640 1 -- 172.21.15.67:0/1140434886 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5055+0+0 (secure 0 0 0) 0x7f8b8408ac00 con 0x7f8b9806f2c0 2024-05-30T17:04:45.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.022+0000 7f8b95e00640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8b7806e510 0x7f8b780709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:45.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.023+0000 7f8b95e00640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8b7806e510 0x7f8b780709d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b8802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:45.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.026+0000 7f8b8e000640 1 -- 172.21.15.67:0/1140434886 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b981f4de0 con 0x7f8b9806f2c0 2024-05-30T17:04:45.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.306+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_7_clones_are_cancelled"]} v 0) -- 0x7f8b980ada90 con 0x7f8b9806f2c0 2024-05-30T17:04:45.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.848+0000 7f8b8e000640 1 -- 172.21.15.67:0/1140434886 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_7_clones_are_cancelled"]}]=0 v874) ==== 166+0+0 (secure 0 0 0) 0x7f8b8405aa90 con 0x7f8b9806f2c0 2024-05-30T17:04:45.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8b7806e510 msgr2=0x7f8b780709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8b7806e510 0x7f8b780709d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b8802e000 comp rx=0 tx=0).stop 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b9806f2c0 msgr2=0x7f8b9809b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b9806f2c0 0x7f8b9809b8f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d830 tx=0x7f8b8400dcf0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 shutdown_connections 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8b7806e510 0x7f8b780709d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b98156b30 0x7f8b981f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b9806f2c0 0x7f8b9809b8f0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 --2- 172.21.15.67:0/1140434886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b9806ee40 0x7f8b9814b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 >> 172.21.15.67:0/1140434886 conn(0x7f8b980b1fc0 msgr2=0x7f8b98072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 shutdown_connections 2024-05-30T17:04:45.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:45.852+0000 7f8b97200640 1 -- 172.21.15.67:0/1140434886 wait complete. 2024-05-30T17:04:45.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T17:04:46.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.092+0000 7f42c7e00640 1 Processor -- start 2024-05-30T17:04:46.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.092+0000 7f42c7e00640 1 -- start start 2024-05-30T17:04:46.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.094+0000 7f42c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42c815e870 0x7f42c80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:46.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.094+0000 7f42c7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42c80d14d0 0x7f42c80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.094+0000 7f42c7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42c80cf870 0x7f42c8162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.094+0000 7f42c7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f42c80d2150 con 0x7f42c80cf870 2024-05-30T17:04:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.094+0000 7f42c7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f42c80cfd00 con 0x7f42c80d14d0 2024-05-30T17:04:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.094+0000 7f42c7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42c815ec50 con 0x7f42c815e870 2024-05-30T17:04:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.094+0000 7f42c7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42c80cf870 0x7f42c8162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.094+0000 7f42c7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42c80cf870 0x7f42c8162360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59726/0 (socket says 172.21.15.67:59726) 2024-05-30T17:04:46.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.094+0000 7f42c7400640 1 -- 172.21.15.67:0/2332951246 learned_addr learned my addr 172.21.15.67:0/2332951246 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.095+0000 7f42c7400640 1 -- 172.21.15.67:0/2332951246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42c815e870 msgr2=0x7f42c80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.095+0000 7f42c7400640 1 --2- 172.21.15.67:0/2332951246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42c815e870 0x7f42c80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.095+0000 7f42c6000640 1 --2- 172.21.15.67:0/2332951246 >> v2:172.21.15.169:3300/0 conn(0x7f42c80d14d0 0x7f42c80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.095+0000 7f42c7400640 1 -- 172.21.15.67:0/2332951246 >> v2:172.21.15.169:3300/0 conn(0x7f42c80d14d0 msgr2=0x7f42c80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.095+0000 7f42c7400640 1 --2- 172.21.15.67:0/2332951246 >> v2:172.21.15.169:3300/0 conn(0x7f42c80d14d0 0x7f42c80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.095+0000 7f42c7400640 1 -- 172.21.15.67:0/2332951246 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c80cff80 con 0x7f42c80cf870 2024-05-30T17:04:46.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.095+0000 7f42c6000640 1 --2- 172.21.15.67:0/2332951246 >> v2:172.21.15.169:3300/0 conn(0x7f42c80d14d0 0x7f42c80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:46.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.096+0000 7f42c7400640 1 --2- 172.21.15.67:0/2332951246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c80cf870 0x7f42c8162360 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f42c80b7170 tx=0x7f42c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6fbe2c86a18306 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:46.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.096+0000 7f42c5600640 1 -- 172.21.15.67:0/2332951246 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0036070 con 0x7f42c80cf870 2024-05-30T17:04:46.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.096+0000 7f42c5600640 1 -- 172.21.15.67:0/2332951246 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c0031600 con 0x7f42c80cf870 2024-05-30T17:04:46.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.096+0000 7f42c5600640 1 -- 172.21.15.67:0/2332951246 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c00319d0 con 0x7f42c80cf870 2024-05-30T17:04:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.097+0000 7f42c7e00640 1 -- 172.21.15.67:0/2332951246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c80cf870 msgr2=0x7f42c8162360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.097+0000 7f42c7e00640 1 --2- 172.21.15.67:0/2332951246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c80cf870 0x7f42c8162360 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f42c80b7170 tx=0x7f42c002a6a0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.097+0000 7f42c7e00640 1 -- 172.21.15.67:0/2332951246 shutdown_connections 2024-05-30T17:04:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.097+0000 7f42c7e00640 1 --2- 172.21.15.67:0/2332951246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c80cf870 0x7f42c8162360 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.097+0000 7f42c7e00640 1 --2- 172.21.15.67:0/2332951246 >> v2:172.21.15.169:3300/0 conn(0x7f42c80d14d0 0x7f42c80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.097+0000 7f42c7e00640 1 --2- 172.21.15.67:0/2332951246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42c815e870 0x7f42c80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.097+0000 7f42c7e00640 1 -- 172.21.15.67:0/2332951246 >> 172.21.15.67:0/2332951246 conn(0x7f42c808bfc0 msgr2=0x7f42c80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.097+0000 7f42c7e00640 1 -- 172.21.15.67:0/2332951246 shutdown_connections 2024-05-30T17:04:46.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.097+0000 7f42c7e00640 1 -- 172.21.15.67:0/2332951246 wait complete. 2024-05-30T17:04:46.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.099+0000 7f42c7e00640 1 Processor -- start 2024-05-30T17:04:46.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.099+0000 7f42c7e00640 1 -- start start 2024-05-30T17:04:46.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.100+0000 7f42c7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42c80cf870 0x7f42c8159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.100+0000 7f42c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42c80d14d0 0x7f42c815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.100+0000 7f42c7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c815e870 0x7f42c80cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.100+0000 7f42c7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f42c80d0820 con 0x7f42c815e870 2024-05-30T17:04:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.100+0000 7f42c7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42c80d0540 con 0x7f42c80cf870 2024-05-30T17:04:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.100+0000 7f42c7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42c80d4fb0 con 0x7f42c80d14d0 2024-05-30T17:04:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.100+0000 7f42c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c815e870 0x7f42c80cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.100+0000 7f42c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c815e870 0x7f42c80cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59730/0 (socket says 172.21.15.67:59730) 2024-05-30T17:04:46.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.100+0000 7f42c7400640 1 -- 172.21.15.67:0/736760845 learned_addr learned my addr 172.21.15.67:0/736760845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42c7400640 1 -- 172.21.15.67:0/736760845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42c80d14d0 msgr2=0x7f42c815bb40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T17:04:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42c7400640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42c80d14d0 0x7f42c815bb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42c7400640 1 -- 172.21.15.67:0/736760845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42c80cf870 msgr2=0x7f42c8159ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T17:04:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42c7400640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42c80cf870 0x7f42c8159ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42c7400640 1 -- 172.21.15.67:0/736760845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c815a020 con 0x7f42c815e870 2024-05-30T17:04:46.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42c7400640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c815e870 0x7f42c80cbdb0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009f40 tx=0x7f42c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:46.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42bea00640 1 -- 172.21.15.67:0/736760845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0041020 con 0x7f42c815e870 2024-05-30T17:04:46.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42bea00640 1 -- 172.21.15.67:0/736760845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c00077a0 con 0x7f42c815e870 2024-05-30T17:04:46.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c815c470 con 0x7f42c815e870 2024-05-30T17:04:46.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.101+0000 7f42bea00640 1 -- 172.21.15.67:0/736760845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0036040 con 0x7f42c815e870 2024-05-30T17:04:46.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.102+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c815ca30 con 0x7f42c815e870 2024-05-30T17:04:46.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.103+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c80689e0 con 0x7f42c815e870 2024-05-30T17:04:46.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.104+0000 7f42bea00640 1 -- 172.21.15.67:0/736760845 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f42c0044020 con 0x7f42c815e870 2024-05-30T17:04:46.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.105+0000 7f42bea00640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42a806e510 0x7f42a80709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:46.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.105+0000 7f42bea00640 1 -- 172.21.15.67:0/736760845 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5055+0+0 (secure 0 0 0) 0x7f42c00ab850 con 0x7f42c815e870 2024-05-30T17:04:46.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.105+0000 7f42c6a00640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42a806e510 0x7f42a80709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:46.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.106+0000 7f42c6a00640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42a806e510 0x7f42a80709d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f42b80096b0 tx=0x7f42b8006c20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:46.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.109+0000 7f42bea00640 1 -- 172.21.15.67:0/736760845 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42c004b030 con 0x7f42c815e870 2024-05-30T17:04:46.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:04:46.150 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:04:46.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.317+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f42c80b40c0 con 0x7f42c815e870 2024-05-30T17:04:46.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.323+0000 7f42bea00640 1 -- 172.21.15.67:0/736760845 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f42c815a020 con 0x7f42c815e870 2024-05-30T17:04:46.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42a806e510 msgr2=0x7f42a80709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:46.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42a806e510 0x7f42a80709d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f42b80096b0 tx=0x7f42b8006c20 comp rx=0 tx=0).stop 2024-05-30T17:04:46.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c815e870 msgr2=0x7f42c80cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:46.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c815e870 0x7f42c80cbdb0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009f40 tx=0x7f42c0004a40 comp rx=0 tx=0).stop 2024-05-30T17:04:46.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 shutdown_connections 2024-05-30T17:04:46.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42a806e510 0x7f42a80709d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42c815e870 0x7f42c80cbdb0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42c80d14d0 0x7f42c815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 --2- 172.21.15.67:0/736760845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42c80cf870 0x7f42c8159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:46.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.326+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 >> 172.21.15.67:0/736760845 conn(0x7f42c808bfc0 msgr2=0x7f42c80b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:46.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.327+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 shutdown_connections 2024-05-30T17:04:46.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:46.327+0000 7f42c7e00640 1 -- 172.21.15.67:0/736760845 wait complete. 2024-05-30T17:04:46.349 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-30T17:04:46.349 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:04:46.464 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-30T17:04:46.580 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-30T17:04:46.580 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:04:46.580 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:04:46.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:04:46.642 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:04:46.642 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:04:46.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:04:46.700 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:04:46.700 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:04:46.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:04:46.760 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:04:46.760 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:04:46.817 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:04:46.818 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-30T17:04:46.818 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:04:47.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.037+0000 7fa760600640 1 Processor -- start 2024-05-30T17:04:47.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.037+0000 7fa760600640 1 -- start start 2024-05-30T17:04:47.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.039+0000 7fa760600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7580d0590 0x7fa7580d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.039+0000 7fa760600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa758099090 0x7fa7580994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.039+0000 7fa760600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7580b51f0 con 0x7fa7580b4e10 2024-05-30T17:04:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.039+0000 7fa760600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa7580d0f30 con 0x7fa758099090 2024-05-30T17:04:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.039+0000 7fa760600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa7580aef60 con 0x7fa7580d0590 2024-05-30T17:04:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.039+0000 7fa75dc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa7580b4e10 0x7fa7580cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58980/0 (socket says 172.21.15.67:58980) 2024-05-30T17:04:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.039+0000 7fa75dc00640 1 -- 172.21.15.67:0/1260138889 learned_addr learned my addr 172.21.15.67:0/1260138889 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.039+0000 7fa75e600640 1 --2- 172.21.15.67:0/1260138889 >> v2:172.21.15.67:3300/0 conn(0x7fa758099090 0x7fa7580994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.040+0000 7fa75d200640 1 --2- 172.21.15.67:0/1260138889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7580d0590 0x7fa7580d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.040+0000 7fa75e600640 1 -- 172.21.15.67:0/1260138889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7580d0590 msgr2=0x7fa7580d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.040+0000 7fa75e600640 1 --2- 172.21.15.67:0/1260138889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7580d0590 0x7fa7580d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.040+0000 7fa757e00640 1 -- 172.21.15.67:0/1260138889 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194304559 0 0) 0x7fa7580b51f0 con 0x7fa7580b4e10 2024-05-30T17:04:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.040+0000 7fa75e600640 1 -- 172.21.15.67:0/1260138889 >> v1:172.21.15.67:6789/0 conn(0x7fa7580b4e10 legacy=0x7fa7580cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.040+0000 7fa75e600640 1 -- 172.21.15.67:0/1260138889 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7580ad1d0 con 0x7fa758099090 2024-05-30T17:04:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.041+0000 7fa75e600640 1 --2- 172.21.15.67:0/1260138889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa758099090 0x7fa7580994e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa750009d90 tx=0x7fa75002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9991bb21f6e62f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.041+0000 7fa757e00640 1 -- 172.21.15.67:0/1260138889 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750031600 con 0x7fa758099090 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.041+0000 7fa757e00640 1 -- 172.21.15.67:0/1260138889 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7500317a0 con 0x7fa758099090 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.041+0000 7fa757e00640 1 -- 172.21.15.67:0/1260138889 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750031b70 con 0x7fa758099090 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.042+0000 7fa760600640 1 -- 172.21.15.67:0/1260138889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa758099090 msgr2=0x7fa7580994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.042+0000 7fa760600640 1 --2- 172.21.15.67:0/1260138889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa758099090 0x7fa7580994e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa750009d90 tx=0x7fa75002a5d0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.042+0000 7fa760600640 1 -- 172.21.15.67:0/1260138889 shutdown_connections 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.042+0000 7fa760600640 1 --2- 172.21.15.67:0/1260138889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa758099090 0x7fa7580994e0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.042+0000 7fa760600640 1 --2- 172.21.15.67:0/1260138889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7580d0590 0x7fa7580d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.042+0000 7fa760600640 1 -- 172.21.15.67:0/1260138889 >> 172.21.15.67:0/1260138889 conn(0x7fa75806bae0 msgr2=0x7fa75806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.042+0000 7fa760600640 1 -- 172.21.15.67:0/1260138889 shutdown_connections 2024-05-30T17:04:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.042+0000 7fa760600640 1 -- 172.21.15.67:0/1260138889 wait complete. 2024-05-30T17:04:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.044+0000 7fa760600640 1 Processor -- start 2024-05-30T17:04:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.044+0000 7fa760600640 1 -- start start 2024-05-30T17:04:47.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.045+0000 7fa760600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa758099090 0x7fa7580b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.045+0000 7fa760600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7580b4e10 0x7fa7580b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.045+0000 7fa760600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7580d0590 0x7fa7581595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.045+0000 7fa760600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580ce540 con 0x7fa7580d0590 2024-05-30T17:04:47.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.045+0000 7fa760600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580ad750 con 0x7fa7580b4e10 2024-05-30T17:04:47.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.045+0000 7fa760600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa7580ce880 con 0x7fa758099090 2024-05-30T17:04:47.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.045+0000 7fa75e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7580d0590 0x7fa7581595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.045+0000 7fa75e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7580d0590 0x7fa7581595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59756/0 (socket says 172.21.15.67:59756) 2024-05-30T17:04:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.045+0000 7fa75e600640 1 -- 172.21.15.67:0/3077104226 learned_addr learned my addr 172.21.15.67:0/3077104226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa75dc00640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa758099090 0x7fa7580b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa75e600640 1 -- 172.21.15.67:0/3077104226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa758099090 msgr2=0x7fa7580b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa75d200640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7580b4e10 0x7fa7580b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa75e600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa758099090 0x7fa7580b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa75e600640 1 -- 172.21.15.67:0/3077104226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7580b4e10 msgr2=0x7fa7580b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa75e600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7580b4e10 0x7fa7580b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa75e600640 1 -- 172.21.15.67:0/3077104226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7580b1000 con 0x7fa7580d0590 2024-05-30T17:04:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa75e600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7580d0590 0x7fa7581595b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fa75002abc0 tx=0x7fa750004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa755600640 1 -- 172.21.15.67:0/3077104226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75003f070 con 0x7fa7580d0590 2024-05-30T17:04:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa755600640 1 -- 172.21.15.67:0/3077104226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa750030b60 con 0x7fa7580d0590 2024-05-30T17:04:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7580b12e0 con 0x7fa7580d0590 2024-05-30T17:04:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.046+0000 7fa755600640 1 -- 172.21.15.67:0/3077104226 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750009450 con 0x7fa7580d0590 2024-05-30T17:04:47.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.047+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7580b18a0 con 0x7fa7580d0590 2024-05-30T17:04:47.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.048+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa758068240 con 0x7fa7580d0590 2024-05-30T17:04:47.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.049+0000 7fa755600640 1 -- 172.21.15.67:0/3077104226 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fa75003b410 con 0x7fa7580d0590 2024-05-30T17:04:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.051+0000 7fa755600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa73806e510 0x7fa7380709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.051+0000 7fa755600640 1 -- 172.21.15.67:0/3077104226 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5055+0+0 (secure 0 0 0) 0x7fa7500ac140 con 0x7fa7580d0590 2024-05-30T17:04:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.051+0000 7fa75dc00640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa73806e510 0x7fa7380709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.052+0000 7fa75dc00640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa73806e510 0x7fa7380709d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa74c007fd0 tx=0x7fa74c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.055+0000 7fa755600640 1 -- 172.21.15.67:0/3077104226 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa758068240 con 0x7fa7580d0590 2024-05-30T17:04:47.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.301+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7580689e0 con 0x7fa7580d0590 2024-05-30T17:04:47.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.302+0000 7fa755600640 1 -- 172.21.15.67:0/3077104226 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3875 (secure 0 0 0) 0x7fa75007b980 con 0x7fa7580d0590 2024-05-30T17:04:47.304 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:47.304 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6667,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1973461214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1973461214},{"type":"v1","addr":"172.21.15.169:6835","nonce":1973461214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2944363648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2944363648},{"type":"v1","addr":"172.21.15.169:6834","nonce":2944363648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6836/3884958123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":3884958123},{"type":"v1","addr":"172.21.15.67:6836","nonce":3884958123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":8403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/1126347083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6835","nonce":1126347083},{"type":"v1","addr":"172.21.15.67:6837","nonce":1126347083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-05-30T17:04:47.304 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 76 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa73806e510 msgr2=0x7fa7380709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa73806e510 0x7fa7380709d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa74c007fd0 tx=0x7fa74c009040 comp rx=0 tx=0).stop 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7580d0590 msgr2=0x7fa7581595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7580d0590 0x7fa7581595b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fa75002abc0 tx=0x7fa750004a40 comp rx=0 tx=0).stop 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 shutdown_connections 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa73806e510 0x7fa7380709d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7580d0590 0x7fa7581595b0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7580b4e10 0x7fa7580b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 --2- 172.21.15.67:0/3077104226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa758099090 0x7fa7580b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 >> 172.21.15.67:0/3077104226 conn(0x7fa75806bae0 msgr2=0x7fa7580cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:47.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 shutdown_connections 2024-05-30T17:04:47.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.305+0000 7fa760600640 1 -- 172.21.15.67:0/3077104226 wait complete. 2024-05-30T17:04:47.326 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-30T17:04:47.326 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-30T17:04:47.326 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-30T17:04:47.326 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:04:47.326 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-30T17:04:47.326 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-30T17:04:47.327 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-30T17:04:47.327 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:04:47.327 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-30T17:04:47.327 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-30T17:04:47.327 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-30T17:04:47.327 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:04:47.328 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-30T17:04:47.328 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-30T17:04:47.328 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-30T17:04:47.328 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:04:47.347 INFO:tasks.ceph.mds.c:Stopped 2024-05-30T17:04:47.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-30T17:04:47.348 INFO:tasks.ceph.mds.d:Stopped 2024-05-30T17:04:47.348 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-30T17:04:47.349 INFO:tasks.ceph.mds.a:Stopped 2024-05-30T17:04:47.349 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-30T17:04:47.358 INFO:tasks.ceph.mds.b:Stopped 2024-05-30T17:04:47.358 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-30T17:04:47.396 INFO:tasks.ceph.mds.c:Started 2024-05-30T17:04:47.396 DEBUG:teuthology.parallel:result is None 2024-05-30T17:04:47.402 INFO:tasks.ceph.mds.a:Started 2024-05-30T17:04:47.402 DEBUG:teuthology.parallel:result is None 2024-05-30T17:04:47.406 INFO:tasks.ceph.mds.d:Started 2024-05-30T17:04:47.407 DEBUG:teuthology.parallel:result is None 2024-05-30T17:04:47.412 INFO:tasks.ceph.mds.b:Started 2024-05-30T17:04:47.412 DEBUG:teuthology.parallel:result is None 2024-05-30T17:04:47.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-30T17:04:47.548 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-05-30T17:04:47.554 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-05-30T17:04:47.560 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-05-30T17:04:47.563 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-05-30T17:04:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.601+0000 7f5517e00640 1 Processor -- start 2024-05-30T17:04:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.601+0000 7f5517e00640 1 -- start start 2024-05-30T17:04:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.602+0000 7f5517e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815fae0 0x7f55180d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.602+0000 7f5517e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55180d3300 con 0x7f5518160380 2024-05-30T17:04:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.602+0000 7f5517e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55180d35e0 con 0x7f55180b6570 2024-05-30T17:04:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.602+0000 7f5517e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f55180ced60 con 0x7f551815fae0 2024-05-30T17:04:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.603+0000 7f550fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5518160380 0x7f55180b5e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59008/0 (socket says 172.21.15.67:59008) 2024-05-30T17:04:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.603+0000 7f550fe00640 1 -- 172.21.15.67:0/4213965535 learned_addr learned my addr 172.21.15.67:0/4213965535 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.603+0000 7f5516a00640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815fae0 0x7f55180d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.603+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422353764 0 0) 0x7f55180d3300 con 0x7f5518160380 2024-05-30T17:04:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.603+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54fc003520 con 0x7f5518160380 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3551163541 0 0) 0x7f54fc003520 con 0x7f5518160380 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815fae0 msgr2=0x7f55180d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815fae0 0x7f55180d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 >> v1:172.21.15.169:6789/0 conn(0x7f55180b6570 legacy=0x7f55180cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55180d38c0 con 0x7f5518160380 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f55000017f0 con 0x7f5518160380 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 >> v1:172.21.15.67:6789/0 conn(0x7f5518160380 legacy=0x7f55180b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc0039e0 0x7f54fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54fc004800 0x7f54fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815fae0 0x7f54fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55180ced60 con 0x7f54fc004800 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc00ba70 con 0x7f54fc0039e0 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc00bc40 con 0x7f551815fae0 2024-05-30T17:04:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f550fe00640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54fc004800 0x7f54fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5516a00640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc0039e0 0x7f54fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f5517400640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815fae0 0x7f54fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f550fe00640 1 -- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815fae0 msgr2=0x7f54fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f550fe00640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f551815fae0 0x7f54fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f550fe00640 1 -- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc0039e0 msgr2=0x7f54fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.604+0000 7f550fe00640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc0039e0 0x7f54fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.605+0000 7f550fe00640 1 -- 172.21.15.67:0/4213965535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc003520 con 0x7f54fc004800 2024-05-30T17:04:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.605+0000 7f550fe00640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54fc004800 0x7f54fc004c80 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f550000c0e0 tx=0x7f54fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375bd8fe8097b126 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.606+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500007c30 con 0x7f54fc004800 2024-05-30T17:04:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.606+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5500007290 con 0x7f54fc004800 2024-05-30T17:04:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.606+0000 7f5516000640 1 -- 172.21.15.67:0/4213965535 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55000057a0 con 0x7f54fc004800 2024-05-30T17:04:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.606+0000 7f5517e00640 1 -- 172.21.15.67:0/4213965535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54fc004800 msgr2=0x7f54fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.606+0000 7f5517e00640 1 --2- 172.21.15.67:0/4213965535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54fc004800 0x7f54fc004c80 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f550000c0e0 tx=0x7f54fc005830 comp rx=0 tx=0).stop 2024-05-30T17:04:47.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.606+0000 7f5517400640 1 -- 172.21.15.67:0/4213965535 reap_dead start 2024-05-30T17:04:47.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.606+0000 7f5517e00640 1 -- 172.21.15.67:0/4213965535 shutdown_connections 2024-05-30T17:04:47.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.606+0000 7f5517e00640 1 -- 172.21.15.67:0/4213965535 >> 172.21.15.67:0/4213965535 conn(0x7f551808bfc0 msgr2=0x7f55180d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:47.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.607+0000 7f5517e00640 1 -- 172.21.15.67:0/4213965535 shutdown_connections 2024-05-30T17:04:47.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.607+0000 7f5517e00640 1 -- 172.21.15.67:0/4213965535 wait complete. 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.608+0000 7f5517e00640 1 Processor -- start 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.608+0000 7f5517e00640 1 -- start start 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.609+0000 7f5517e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc004800 0x7f55180c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.609+0000 7f5517e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cd360 0x7f55180cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.609+0000 7f5517e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cc3f0 0x7f55180cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.609+0000 7f5517e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55180b55b0 con 0x7f55180cd360 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.609+0000 7f5517e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55180b5430 con 0x7f54fc004800 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.609+0000 7f5517e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f55180d4e20 con 0x7f55180cc3f0 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f550fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cd360 0x7f55180cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f5516a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc004800 0x7f55180c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f550fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cd360 0x7f55180cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59818/0 (socket says 172.21.15.67:59818) 2024-05-30T17:04:47.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f550fe00640 1 -- 172.21.15.67:0/1557305117 learned_addr learned my addr 172.21.15.67:0/1557305117 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:47.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f5517400640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cc3f0 0x7f55180cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f550fe00640 1 -- 172.21.15.67:0/1557305117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cc3f0 msgr2=0x7f55180cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f550fe00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cc3f0 0x7f55180cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f550fe00640 1 -- 172.21.15.67:0/1557305117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc004800 msgr2=0x7f55180c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:47.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f550fe00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc004800 0x7f55180c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:47.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f550fe00640 1 -- 172.21.15.67:0/1557305117 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55180cb120 con 0x7f55180cd360 2024-05-30T17:04:47.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.610+0000 7f5516a00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc004800 0x7f55180c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:04:47.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.611+0000 7f550fe00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cd360 0x7f55180cbeb0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5500000ef0 tx=0x7f55180d34d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:47.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.611+0000 7f550ea00640 1 -- 172.21.15.67:0/1557305117 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55000017b0 con 0x7f55180cd360 2024-05-30T17:04:47.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.611+0000 7f550ea00640 1 -- 172.21.15.67:0/1557305117 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5500042070 con 0x7f55180cd360 2024-05-30T17:04:47.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.611+0000 7f550ea00640 1 -- 172.21.15.67:0/1557305117 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55000059a0 con 0x7f55180cd360 2024-05-30T17:04:47.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.611+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55180b7920 con 0x7f55180cd360 2024-05-30T17:04:47.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.611+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55180cd7e0 con 0x7f55180cd360 2024-05-30T17:04:47.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.612+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55180689e0 con 0x7f55180cd360 2024-05-30T17:04:47.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.613+0000 7f550ea00640 1 -- 172.21.15.67:0/1557305117 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f5500032430 con 0x7f55180cd360 2024-05-30T17:04:47.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.614+0000 7f550ea00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f406e540 0x7f54f4070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:47.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.614+0000 7f550ea00640 1 -- 172.21.15.67:0/1557305117 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5055+0+0 (secure 0 0 0) 0x7f55000aa6a0 con 0x7f55180cd360 2024-05-30T17:04:47.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.614+0000 7f5516a00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f406e540 0x7f54f4070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:47.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.615+0000 7f5516a00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f406e540 0x7f54f4070a00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5508009fb0 tx=0x7f5508009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:47.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.618+0000 7f550ea00640 1 -- 172.21.15.67:0/1557305117 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f550004b020 con 0x7f55180cd360 2024-05-30T17:04:47.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:47.835+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f551806e870 con 0x7f55180cd360 2024-05-30T17:04:48.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.888+0000 7f550ea00640 1 -- 172.21.15.67:0/1557305117 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v128) ==== 97+0+0 (secure 0 0 0) 0x7f550007a530 con 0x7f55180cd360 2024-05-30T17:04:48.890 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-05-30T17:04:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.891+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f406e540 msgr2=0x7f54f4070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.891+0000 7f5517e00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f406e540 0x7f54f4070a00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5508009fb0 tx=0x7f5508009190 comp rx=0 tx=0).stop 2024-05-30T17:04:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.891+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cd360 msgr2=0x7f55180cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.891+0000 7f5517e00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cd360 0x7f55180cbeb0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5500000ef0 tx=0x7f55180d34d0 comp rx=0 tx=0).stop 2024-05-30T17:04:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.891+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 shutdown_connections 2024-05-30T17:04:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.891+0000 7f5517e00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f54f406e540 0x7f54f4070a00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.891+0000 7f5517e00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55180cc3f0 0x7f55180cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.891+0000 7f5517e00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55180cd360 0x7f55180cbeb0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.891+0000 7f5517e00640 1 --2- 172.21.15.67:0/1557305117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54fc004800 0x7f55180c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.892+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 >> 172.21.15.67:0/1557305117 conn(0x7f551808bfc0 msgr2=0x7f55180d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.892+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 shutdown_connections 2024-05-30T17:04:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:48.892+0000 7f5517e00640 1 -- 172.21.15.67:0/1557305117 wait complete. 2024-05-30T17:04:48.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-30T17:04:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.109+0000 7f5b79a00640 1 Processor -- start 2024-05-30T17:04:49.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.109+0000 7f5b79a00640 1 -- start start 2024-05-30T17:04:49.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.111+0000 7f5b79a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b7415e870 0x7f5b740d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.111+0000 7f5b79a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5b740d14d0 0x7f5b740cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.111+0000 7f5b79a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b740cfce0 con 0x7f5b740cf870 2024-05-30T17:04:49.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.111+0000 7f5b79a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5b740d2150 con 0x7f5b7415e870 2024-05-30T17:04:49.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.111+0000 7f5b79a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5b7415ec50 con 0x7f5b740d14d0 2024-05-30T17:04:49.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.112+0000 7f5b73e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b7415e870 0x7f5b740d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:49.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.112+0000 7f5b73e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b7415e870 0x7f5b740d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59828/0 (socket says 172.21.15.67:59828) 2024-05-30T17:04:49.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.112+0000 7f5b73e00640 1 -- 172.21.15.67:0/4131292950 learned_addr learned my addr 172.21.15.67:0/4131292950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:49.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.112+0000 7f5b6be00640 1 --2- 172.21.15.67:0/4131292950 >> v2:172.21.15.169:3300/0 conn(0x7f5b740d14d0 0x7f5b740cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:49.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.112+0000 7f5b73e00640 1 -- 172.21.15.67:0/4131292950 >> v2:172.21.15.169:3300/0 conn(0x7f5b740d14d0 msgr2=0x7f5b740cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:49.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.112+0000 7f5b73e00640 1 --2- 172.21.15.67:0/4131292950 >> v2:172.21.15.169:3300/0 conn(0x7f5b740d14d0 0x7f5b740cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.112+0000 7f5b73e00640 1 -- 172.21.15.67:0/4131292950 >> v1:172.21.15.169:6789/0 conn(0x7f5b740cf870 legacy=0x7f5b74162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:49.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.112+0000 7f5b73e00640 1 -- 172.21.15.67:0/4131292950 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b740cff80 con 0x7f5b7415e870 2024-05-30T17:04:49.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.114+0000 7f5b73e00640 1 --2- 172.21.15.67:0/4131292950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7415e870 0x7f5b740d0f90 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5b7408e520 tx=0x7f5b6402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70307d573e297b78 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:49.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.114+0000 7f5b73400640 1 -- 172.21.15.67:0/4131292950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64037070 con 0x7f5b7415e870 2024-05-30T17:04:49.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b73400640 1 -- 172.21.15.67:0/4131292950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b64032440 con 0x7f5b7415e870 2024-05-30T17:04:49.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b73400640 1 -- 172.21.15.67:0/4131292950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64032810 con 0x7f5b7415e870 2024-05-30T17:04:49.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b79a00640 1 -- 172.21.15.67:0/4131292950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7415e870 msgr2=0x7f5b740d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b79a00640 1 --2- 172.21.15.67:0/4131292950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7415e870 0x7f5b740d0f90 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5b7408e520 tx=0x7f5b6402a230 comp rx=0 tx=0).stop 2024-05-30T17:04:49.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b79a00640 1 -- 172.21.15.67:0/4131292950 shutdown_connections 2024-05-30T17:04:49.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b79a00640 1 --2- 172.21.15.67:0/4131292950 >> v2:172.21.15.169:3300/0 conn(0x7f5b740d14d0 0x7f5b740cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b79a00640 1 --2- 172.21.15.67:0/4131292950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b7415e870 0x7f5b740d0f90 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b79a00640 1 -- 172.21.15.67:0/4131292950 >> 172.21.15.67:0/4131292950 conn(0x7f5b7408bfc0 msgr2=0x7f5b740d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b79a00640 1 -- 172.21.15.67:0/4131292950 shutdown_connections 2024-05-30T17:04:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.115+0000 7f5b79a00640 1 -- 172.21.15.67:0/4131292950 wait complete. 2024-05-30T17:04:49.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.117+0000 7f5b79a00640 1 Processor -- start 2024-05-30T17:04:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.117+0000 7f5b79a00640 1 -- start start 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.118+0000 7f5b79a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b740cf870 0x7f5b741593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.118+0000 7f5b79a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b740d14d0 0x7f5b740c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.118+0000 7f5b79a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415e870 0x7f5b740c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.118+0000 7f5b79a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5b740d0820 con 0x7f5b740cf870 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.118+0000 7f5b79a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b740d0540 con 0x7f5b740d14d0 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.118+0000 7f5b79a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b740d4fb0 con 0x7f5b7415e870 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.119+0000 7f5b73e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b740cf870 0x7f5b741593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.119+0000 7f5b73e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b740cf870 0x7f5b741593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59838/0 (socket says 172.21.15.67:59838) 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.119+0000 7f5b73e00640 1 -- 172.21.15.67:0/3261306322 learned_addr learned my addr 172.21.15.67:0/3261306322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.119+0000 7f5b73e00640 1 -- 172.21.15.67:0/3261306322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415e870 msgr2=0x7f5b740c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.119+0000 7f5b73e00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415e870 0x7f5b740c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.119+0000 7f5b73e00640 1 -- 172.21.15.67:0/3261306322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b740d14d0 msgr2=0x7f5b740c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.119+0000 7f5b73e00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b740d14d0 0x7f5b740c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.119+0000 7f5b73e00640 1 -- 172.21.15.67:0/3261306322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b740ca250 con 0x7f5b740cf870 2024-05-30T17:04:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.119+0000 7f5b73e00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b740cf870 0x7f5b741593f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f5b64000c00 tx=0x7f5b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:49.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.120+0000 7f5b70c00640 1 -- 172.21.15.67:0/3261306322 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64041020 con 0x7f5b740cf870 2024-05-30T17:04:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.120+0000 7f5b70c00640 1 -- 172.21.15.67:0/3261306322 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b640306c0 con 0x7f5b740cf870 2024-05-30T17:04:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.120+0000 7f5b70c00640 1 -- 172.21.15.67:0/3261306322 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64037040 con 0x7f5b740cf870 2024-05-30T17:04:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.120+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b740ca530 con 0x7f5b740cf870 2024-05-30T17:04:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.120+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b740caaf0 con 0x7f5b740cf870 2024-05-30T17:04:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.121+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b740689e0 con 0x7f5b740cf870 2024-05-30T17:04:49.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.122+0000 7f5b70c00640 1 -- 172.21.15.67:0/3261306322 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f5b64044020 con 0x7f5b740cf870 2024-05-30T17:04:49.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.123+0000 7f5b70c00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5006e510 0x7f5b500709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.123+0000 7f5b70c00640 1 -- 172.21.15.67:0/3261306322 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4969+0+0 (secure 0 0 0) 0x7f5b640ab360 con 0x7f5b740cf870 2024-05-30T17:04:49.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.123+0000 7f5b6be00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5006e510 0x7f5b500709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:49.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.124+0000 7f5b6be00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5006e510 0x7f5b500709d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c006e20 tx=0x7f5b5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:49.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.128+0000 7f5b70c00640 1 -- 172.21.15.67:0/3261306322 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b6404b030 con 0x7f5b740cf870 2024-05-30T17:04:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.395+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5b740ba500 con 0x7f5b740cf870 2024-05-30T17:04:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.396+0000 7f5b70c00640 1 -- 172.21.15.67:0/3261306322 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7f5b6407be20 con 0x7f5b740cf870 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmLwAJARAAFWE89nAtIALpU5GiqXOgKA==", 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:04:49.397 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmuDE3HxAAaRWgS7wGI6izAI2s+D5nrw==", 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmUJB5DxAAchsOUkIejoous3nihMyOAw==", 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhmhfFNLhAAeL19U06bgRBop974SfJOIg==", 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.399 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCGrFhmjnEiOhAAgMmdebX/WP4y5NNbgJx/ng==", 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCKrFhmzndmChAA1dGLTDk4GA6UXRbiGA0PGg==", 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCNrFhmscPDFRAAJ7q/PMgzSkBZW9Updbaz4w==", 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.401 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCQrFhm64PqMRAAbsUK6CS9Njc4N1REPoq3RQ==", 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCXrFhmPKENCRAA/uUn9mAW73DO+eQtxRgB0A==", 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCarFhm5R3kExAALR8NvTgXSiMOv0KlohPwSQ==", 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.403 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCdrFhm05aaHxAABc6+wHOwezFP3SQ8tLs12w==", 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCgrFhmn6DgOBAAVOmgnUH9z8wnSbDitOaN1A==", 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg==", 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:04:49.405 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhm2ej0ExAAf65mlxJLvbFw7lS7z4evBQ==", 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.2", 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmiYspChAA9HrsjaVgRHxDEjkfxsHhaQ==", 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.3", 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCDrFhmkht/HRAAfW7X6CeJW+0dUhI79g0C3A==", 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.407 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCArFhmF5mhNhAA4H3cgcH6YdD53fANDW4seA==", 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmWzphMRAAooM2Syk8tnn5WseNktbZOQ==", 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCBrFhmxORKJxAAMVSAkTrBIGaqxktZvE/Kvw==", 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-05-30T17:04:49.409 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-05-30T17:04:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-05-30T17:04:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-05-30T17:04:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-05-30T17:04:49.410 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.398+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5006e510 msgr2=0x7f5b500709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5006e510 0x7f5b500709d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c006e20 tx=0x7f5b5c008040 comp rx=0 tx=0).stop 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b740cf870 msgr2=0x7f5b741593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b740cf870 0x7f5b741593f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f5b64000c00 tx=0x7f5b640046c0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 shutdown_connections 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5b5006e510 0x7f5b500709d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415e870 0x7f5b740c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b740d14d0 0x7f5b740c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 --2- 172.21.15.67:0/3261306322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b740cf870 0x7f5b741593f0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 >> 172.21.15.67:0/3261306322 conn(0x7f5b7408bfc0 msgr2=0x7f5b740d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 shutdown_connections 2024-05-30T17:04:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.399+0000 7f5b79a00640 1 -- 172.21.15.67:0/3261306322 wait complete. 2024-05-30T17:04:49.419 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-30T17:04:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.617+0000 7f101c000640 1 Processor -- start 2024-05-30T17:04:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.618+0000 7f101c000640 1 -- start start 2024-05-30T17:04:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.619+0000 7f101c000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f10140d04a0 0x7f10140d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.619+0000 7f101c000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10140b3460 0x7f10140b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.619+0000 7f101c000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10140ced40 con 0x7f10140b4b60 2024-05-30T17:04:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.619+0000 7f101c000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f101415fbb0 con 0x7f10140d04a0 2024-05-30T17:04:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.619+0000 7f101c000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f10140cefc0 con 0x7f10140b3460 2024-05-30T17:04:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.619+0000 7f1018c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f10140d04a0 0x7f10140d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.620+0000 7f1018c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f10140d04a0 0x7f10140d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59850/0 (socket says 172.21.15.67:59850) 2024-05-30T17:04:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.620+0000 7f1018c00640 1 -- 172.21.15.67:0/2452282065 learned_addr learned my addr 172.21.15.67:0/2452282065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.620+0000 7f1018c00640 1 -- 172.21.15.67:0/2452282065 >> v2:172.21.15.169:3300/0 conn(0x7f10140b3460 msgr2=0x7f10140b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.620+0000 7f1018c00640 1 --2- 172.21.15.67:0/2452282065 >> v2:172.21.15.169:3300/0 conn(0x7f10140b3460 0x7f10140b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.620+0000 7f1018c00640 1 -- 172.21.15.67:0/2452282065 >> v1:172.21.15.169:6789/0 conn(0x7f10140b4b60 legacy=0x7f10140cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.620+0000 7f1018c00640 1 -- 172.21.15.67:0/2452282065 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10140cf2a0 con 0x7f10140d04a0 2024-05-30T17:04:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.621+0000 7f1018c00640 1 --2- 172.21.15.67:0/2452282065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140d04a0 0x7f10140d0930 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f10140af320 tx=0x7f100002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e61a83a4152fe6a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.621+0000 7f1013e00640 1 -- 172.21.15.67:0/2452282065 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000037070 con 0x7f10140d04a0 2024-05-30T17:04:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.621+0000 7f1013e00640 1 -- 172.21.15.67:0/2452282065 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1000032440 con 0x7f10140d04a0 2024-05-30T17:04:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.621+0000 7f1013e00640 1 -- 172.21.15.67:0/2452282065 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000032810 con 0x7f10140d04a0 2024-05-30T17:04:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.622+0000 7f101c000640 1 -- 172.21.15.67:0/2452282065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140d04a0 msgr2=0x7f10140d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.622+0000 7f101c000640 1 --2- 172.21.15.67:0/2452282065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140d04a0 0x7f10140d0930 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f10140af320 tx=0x7f100002a230 comp rx=0 tx=0).stop 2024-05-30T17:04:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.622+0000 7f101c000640 1 -- 172.21.15.67:0/2452282065 shutdown_connections 2024-05-30T17:04:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.622+0000 7f101c000640 1 --2- 172.21.15.67:0/2452282065 >> v2:172.21.15.169:3300/0 conn(0x7f10140b3460 0x7f10140b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.622+0000 7f101c000640 1 --2- 172.21.15.67:0/2452282065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140d04a0 0x7f10140d0930 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.622+0000 7f101c000640 1 -- 172.21.15.67:0/2452282065 >> 172.21.15.67:0/2452282065 conn(0x7f101406bae0 msgr2=0x7f101406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.622+0000 7f101c000640 1 -- 172.21.15.67:0/2452282065 shutdown_connections 2024-05-30T17:04:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.622+0000 7f101c000640 1 -- 172.21.15.67:0/2452282065 wait complete. 2024-05-30T17:04:49.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.624+0000 7f101c000640 1 Processor -- start 2024-05-30T17:04:49.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.624+0000 7f101c000640 1 -- start start 2024-05-30T17:04:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.625+0000 7f101c000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10140b3460 0x7f101409ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.625+0000 7f101c000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140b4b60 0x7f10140c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.625+0000 7f101c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10140d04a0 0x7f1014154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.625+0000 7f101c000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f10140b3f40 con 0x7f10140b4b60 2024-05-30T17:04:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.625+0000 7f101c000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10140b3dc0 con 0x7f10140b3460 2024-05-30T17:04:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.625+0000 7f101c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f101415ede0 con 0x7f10140d04a0 2024-05-30T17:04:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1018c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140b4b60 0x7f10140c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:49.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1018c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140b4b60 0x7f10140c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59858/0 (socket says 172.21.15.67:59858) 2024-05-30T17:04:49.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1018c00640 1 -- 172.21.15.67:0/128688220 learned_addr learned my addr 172.21.15.67:0/128688220 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1019600640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10140b3460 0x7f101409ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f101a000640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10140d04a0 0x7f1014154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1018c00640 1 -- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10140d04a0 msgr2=0x7f1014154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1018c00640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10140d04a0 0x7f1014154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1018c00640 1 -- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10140b3460 msgr2=0x7f101409ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1018c00640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10140b3460 0x7f101409ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1018c00640 1 -- 172.21.15.67:0/128688220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10140c6bb0 con 0x7f10140b4b60 2024-05-30T17:04:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f1019600640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10140b3460 0x7f101409ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.626+0000 7f101a000640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10140d04a0 0x7f1014154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.627+0000 7f1018c00640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140b4b60 0x7f10140c5ef0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f100002a270 tx=0x7f10000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.627+0000 7f1011600640 1 -- 172.21.15.67:0/128688220 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000041070 con 0x7f10140b4b60 2024-05-30T17:04:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.627+0000 7f1011600640 1 -- 172.21.15.67:0/128688220 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100003c870 con 0x7f10140b4b60 2024-05-30T17:04:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.627+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101409a3e0 con 0x7f10140b4b60 2024-05-30T17:04:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.627+0000 7f1011600640 1 -- 172.21.15.67:0/128688220 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000037040 con 0x7f10140b4b60 2024-05-30T17:04:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.627+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10140c68d0 con 0x7f10140b4b60 2024-05-30T17:04:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.628+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10140689e0 con 0x7f10140b4b60 2024-05-30T17:04:49.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.630+0000 7f1011600640 1 -- 172.21.15.67:0/128688220 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f1000044070 con 0x7f10140b4b60 2024-05-30T17:04:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.631+0000 7f1011600640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ff406e510 0x7f0ff40709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.631+0000 7f1011600640 1 -- 172.21.15.67:0/128688220 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4969+0+0 (secure 0 0 0) 0x7f10000ab720 con 0x7f10140b4b60 2024-05-30T17:04:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.631+0000 7f1019600640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ff406e510 0x7f0ff40709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.632+0000 7f1019600640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ff406e510 0x7f0ff40709d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1004001dc0 tx=0x7f1004009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.636+0000 7f1011600640 1 -- 172.21.15.67:0/128688220 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10140ced40 con 0x7f10140b4b60 2024-05-30T17:04:49.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.843+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f10140b0b30 con 0x7f10140b4b60 2024-05-30T17:04:49.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.844+0000 7f1011600640 1 -- 172.21.15.67:0/128688220 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) ==== 84+0+65 (secure 0 0 0) 0x7f100007b5b0 con 0x7f10140b4b60 2024-05-30T17:04:49.845 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:49.845 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:04:49.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:04:49.845 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-05-30T17:04:49.845 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-05-30T17:04:49.845 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:04:49.845 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ff406e510 msgr2=0x7f0ff40709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ff406e510 0x7f0ff40709d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1004001dc0 tx=0x7f1004009040 comp rx=0 tx=0).stop 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140b4b60 msgr2=0x7f10140c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140b4b60 0x7f10140c5ef0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f100002a270 tx=0x7f10000046c0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 shutdown_connections 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0ff406e510 0x7f0ff40709d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10140d04a0 0x7f1014154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10140b4b60 0x7f10140c5ef0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 --2- 172.21.15.67:0/128688220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10140b3460 0x7f101409ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 >> 172.21.15.67:0/128688220 conn(0x7f101406bae0 msgr2=0x7f10140b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:49.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 shutdown_connections 2024-05-30T17:04:49.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:49.847+0000 7f101c000640 1 -- 172.21.15.67:0/128688220 wait complete. 2024-05-30T17:04:49.867 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-30T17:04:49.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-30T17:04:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.080+0000 7f5a03e00640 1 Processor -- start 2024-05-30T17:04:50.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.080+0000 7f5a03e00640 1 -- start start 2024-05-30T17:04:50.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.081+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a040d0320 0x7f5a040d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:50.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.081+0000 7f5a03e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a0415eef0 con 0x7f5a040af4a0 2024-05-30T17:04:50.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.081+0000 7f5a03e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a0415fdc0 con 0x7f5a040b4a70 2024-05-30T17:04:50.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.081+0000 7f5a03e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0415c1c0 con 0x7f5a040d0320 2024-05-30T17:04:50.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.082+0000 7f5a02a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5a040af4a0 0x7f5a0415b8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59022/0 (socket says 172.21.15.67:59022) 2024-05-30T17:04:50.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.082+0000 7f5a02a00640 1 -- 172.21.15.67:0/1398355437 learned_addr learned my addr 172.21.15.67:0/1398355437 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:50.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.082+0000 7f5a03400640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a040d0320 0x7f5a040d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:50.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.082+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356128939 0 0) 0x7f5a0415eef0 con 0x7f5a040af4a0 2024-05-30T17:04:50.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.082+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59e8005730 con 0x7f5a040af4a0 2024-05-30T17:04:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.082+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3209233402 0 0) 0x7f59e8005730 con 0x7f5a040af4a0 2024-05-30T17:04:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a040d0320 msgr2=0x7f5a040d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a040d0320 0x7f5a040d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 >> v1:172.21.15.169:6789/0 conn(0x7f5a040b4a70 legacy=0x7f5a0415f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0415c4a0 con 0x7f5a040af4a0 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f59f4001670 con 0x7f5a040af4a0 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 >> v1:172.21.15.67:6789/0 conn(0x7f5a040af4a0 legacy=0x7f5a0415b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a040d0320 0x7f59e8005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59e80061c0 0x7f59e8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f59e80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0415c1c0 con 0x7f59e8006f50 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f59e800dd10 con 0x7f59e80061c0 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f59e800df10 con 0x7f5a040d0320 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a03400640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f59e80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.083+0000 7f5a02a00640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a040d0320 0x7f59e8005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.084+0000 7f59fbe00640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59e80061c0 0x7f59e8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.084+0000 7f5a03400640 1 -- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a040d0320 msgr2=0x7f59e8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.084+0000 7f5a03400640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a040d0320 0x7f59e8005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.084+0000 7f5a03400640 1 -- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59e80061c0 msgr2=0x7f59e8006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.084+0000 7f5a03400640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59e80061c0 0x7f59e8006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.084+0000 7f5a03400640 1 -- 172.21.15.67:0/1398355437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e8005730 con 0x7f59e8006f50 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.084+0000 7f59fbe00640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59e80061c0 0x7f59e8006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.085+0000 7f5a03400640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f59e80073d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009fc0 tx=0x7f59e8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878a88d092109458 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.085+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc009c10 con 0x7f59e8006f50 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.085+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59fc036070 con 0x7f59e8006f50 2024-05-30T17:04:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.085+0000 7f5a02000640 1 -- 172.21.15.67:0/1398355437 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc031880 con 0x7f59e8006f50 2024-05-30T17:04:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.085+0000 7f5a03e00640 1 -- 172.21.15.67:0/1398355437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 msgr2=0x7f59e80073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.086+0000 7f5a03e00640 1 --2- 172.21.15.67:0/1398355437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f59e80073d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009fc0 tx=0x7f59e8007a40 comp rx=0 tx=0).stop 2024-05-30T17:04:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.086+0000 7f5a03400640 1 -- 172.21.15.67:0/1398355437 reap_dead start 2024-05-30T17:04:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.086+0000 7f5a03e00640 1 -- 172.21.15.67:0/1398355437 shutdown_connections 2024-05-30T17:04:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.086+0000 7f5a03e00640 1 -- 172.21.15.67:0/1398355437 >> 172.21.15.67:0/1398355437 conn(0x7f5a0406bae0 msgr2=0x7f5a0406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.086+0000 7f5a03e00640 1 -- 172.21.15.67:0/1398355437 shutdown_connections 2024-05-30T17:04:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.086+0000 7f5a03e00640 1 -- 172.21.15.67:0/1398355437 wait complete. 2024-05-30T17:04:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.088+0000 7f5a03e00640 1 Processor -- start 2024-05-30T17:04:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.088+0000 7f5a03e00640 1 -- start start 2024-05-30T17:04:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.089+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f5a041f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.089+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a04151b90 0x7f5a0409a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.089+0000 7f5a03e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a041507e0 0x7f5a04150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.089+0000 7f5a03e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5a040af880 con 0x7f59e8006f50 2024-05-30T17:04:50.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.089+0000 7f5a03e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0415f190 con 0x7f5a041507e0 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.089+0000 7f5a03e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a040afa00 con 0x7f5a04151b90 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.089+0000 7f5a02a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f5a041f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.089+0000 7f5a02a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f5a041f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59876/0 (socket says 172.21.15.67:59876) 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.089+0000 7f5a02a00640 1 -- 172.21.15.67:0/4260456291 learned_addr learned my addr 172.21.15.67:0/4260456291 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f5a02a00640 1 -- 172.21.15.67:0/4260456291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a04151b90 msgr2=0x7f5a0409a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f5a02a00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a04151b90 0x7f5a0409a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f5a02a00640 1 -- 172.21.15.67:0/4260456291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a041507e0 msgr2=0x7f5a04150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f5a02a00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a041507e0 0x7f5a04150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f5a02a00640 1 -- 172.21.15.67:0/4260456291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0409a6e0 con 0x7f59e8006f50 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f5a02a00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f5a041f4000 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f59f400e820 tx=0x7f59f400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f59faa00640 1 -- 172.21.15.67:0/4260456291 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f400c870 con 0x7f59e8006f50 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f59faa00640 1 -- 172.21.15.67:0/4260456291 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f4018070 con 0x7f59e8006f50 2024-05-30T17:04:50.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f59faa00640 1 -- 172.21.15.67:0/4260456291 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f400cc70 con 0x7f59e8006f50 2024-05-30T17:04:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a04152100 con 0x7f59e8006f50 2024-05-30T17:04:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.090+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a041526c0 con 0x7f59e8006f50 2024-05-30T17:04:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.092+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a04073a50 con 0x7f59e8006f50 2024-05-30T17:04:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.093+0000 7f59faa00640 1 -- 172.21.15.67:0/4260456291 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f59f4013420 con 0x7f59e8006f50 2024-05-30T17:04:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.093+0000 7f59faa00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f59e006e540 0x7f59e0070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.094+0000 7f59faa00640 1 -- 172.21.15.67:0/4260456291 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4969+0+0 (secure 0 0 0) 0x7f59f408a5b0 con 0x7f59e8006f50 2024-05-30T17:04:50.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.094+0000 7f59fbe00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f59e006e540 0x7f59e0070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:50.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.094+0000 7f59fbe00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f59e006e540 0x7f59e0070a00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5a0414d9f0 tx=0x7f59ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:50.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.098+0000 7f59faa00640 1 -- 172.21.15.67:0/4260456291 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a04073a50 con 0x7f59e8006f50 2024-05-30T17:04:50.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.308+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5a040689e0 con 0x7f59e8006f50 2024-05-30T17:04:50.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.907+0000 7f59faa00640 1 -- 172.21.15.67:0/4260456291 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) ==== 138+0+0 (secure 0 0 0) 0x7f59f405a4e0 con 0x7f59e8006f50 2024-05-30T17:04:50.908 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-05-30T17:04:50.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f59e006e540 msgr2=0x7f59e0070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f59e006e540 0x7f59e0070a00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5a0414d9f0 tx=0x7f59ec009040 comp rx=0 tx=0).stop 2024-05-30T17:04:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 msgr2=0x7f5a041f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f5a041f4000 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f59f400e820 tx=0x7f59f400ece0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 shutdown_connections 2024-05-30T17:04:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f59e006e540 0x7f59e0070a00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a041507e0 0x7f5a04150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a04151b90 0x7f5a0409a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 --2- 172.21.15.67:0/4260456291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e8006f50 0x7f5a041f4000 secure :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f59f400e820 tx=0x7f59f400ece0 comp rx=0 tx=0).stop 2024-05-30T17:04:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.912+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 >> 172.21.15.67:0/4260456291 conn(0x7f5a0406bae0 msgr2=0x7f5a0415dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.913+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 shutdown_connections 2024-05-30T17:04:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:50.914+0000 7f5a03e00640 1 -- 172.21.15.67:0/4260456291 wait complete. 2024-05-30T17:04:50.941 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-30T17:04:51.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.145+0000 7f0022800640 1 Processor -- start 2024-05-30T17:04:51.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.145+0000 7f0022800640 1 -- start start 2024-05-30T17:04:51.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.147+0000 7f0022800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f001c0cd2a0 0x7f001c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:51.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.147+0000 7f0022800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 0x7f001c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:51.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.147+0000 7f0022800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f001c15d900 con 0x7f001c153510 2024-05-30T17:04:51.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.147+0000 7f0022800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f001c1538f0 con 0x7f001c0cd2a0 2024-05-30T17:04:51.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.147+0000 7f0022800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f001c0b48c0 con 0x7f001c15cb60 2024-05-30T17:04:51.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.147+0000 7f0020a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f001c0cd2a0 0x7f001c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:51.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.147+0000 7f0020a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f001c0cd2a0 0x7f001c0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54786/0 (socket says 172.21.15.67:54786) 2024-05-30T17:04:51.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.147+0000 7f0020a00640 1 -- 172.21.15.67:0/3695033164 learned_addr learned my addr 172.21.15.67:0/3695033164 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.148+0000 7f0021e00640 1 --2- 172.21.15.67:0/3695033164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 0x7f001c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.148+0000 7f0020a00640 1 -- 172.21.15.67:0/3695033164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 msgr2=0x7f001c15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.148+0000 7f0020a00640 1 --2- 172.21.15.67:0/3695033164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 0x7f001c15cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.148+0000 7f0020a00640 1 -- 172.21.15.67:0/3695033164 >> v1:172.21.15.169:6789/0 conn(0x7f001c153510 legacy=0x7f001c0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.148+0000 7f0020a00640 1 -- 172.21.15.67:0/3695033164 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c0b3f90 con 0x7f001c0cd2a0 2024-05-30T17:04:51.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.149+0000 7f0020a00640 1 --2- 172.21.15.67:0/3695033164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 0x7f001c0b4d50 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f0008009870 tx=0x7f000802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cf5dc740ee9994 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.149+0000 7f001be00640 1 -- 172.21.15.67:0/3695033164 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008037070 con 0x7f001c0cd2a0 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.149+0000 7f001be00640 1 -- 172.21.15.67:0/3695033164 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0008032440 con 0x7f001c0cd2a0 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.149+0000 7f001be00640 1 -- 172.21.15.67:0/3695033164 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008032810 con 0x7f001c0cd2a0 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.150+0000 7f0022800640 1 -- 172.21.15.67:0/3695033164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 msgr2=0x7f001c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.150+0000 7f0022800640 1 --2- 172.21.15.67:0/3695033164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 0x7f001c0b4d50 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f0008009870 tx=0x7f000802a180 comp rx=0 tx=0).stop 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.150+0000 7f0022800640 1 -- 172.21.15.67:0/3695033164 shutdown_connections 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.150+0000 7f0022800640 1 --2- 172.21.15.67:0/3695033164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 0x7f001c15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.150+0000 7f0022800640 1 --2- 172.21.15.67:0/3695033164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 0x7f001c0b4d50 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.150+0000 7f0022800640 1 -- 172.21.15.67:0/3695033164 >> 172.21.15.67:0/3695033164 conn(0x7f001c06b900 msgr2=0x7f001c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.150+0000 7f0022800640 1 -- 172.21.15.67:0/3695033164 shutdown_connections 2024-05-30T17:04:51.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.150+0000 7f0022800640 1 -- 172.21.15.67:0/3695033164 wait complete. 2024-05-30T17:04:51.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.152+0000 7f0022800640 1 Processor -- start 2024-05-30T17:04:51.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.152+0000 7f0022800640 1 -- start start 2024-05-30T17:04:51.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.153+0000 7f0022800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 0x7f001c150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:51.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.153+0000 7f0022800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f001c153510 0x7f001c0b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:51.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.153+0000 7f0022800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 0x7f001c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:51.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.153+0000 7f0022800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f001c15d4f0 con 0x7f001c0cd2a0 2024-05-30T17:04:51.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.153+0000 7f0022800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f001c0b4550 con 0x7f001c153510 2024-05-30T17:04:51.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.153+0000 7f0022800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f001c15d670 con 0x7f001c15cb60 2024-05-30T17:04:51.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.153+0000 7f0021400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 0x7f001c150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:51.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.153+0000 7f0021400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 0x7f001c150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54802/0 (socket says 172.21.15.67:54802) 2024-05-30T17:04:51.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.153+0000 7f0021400640 1 -- 172.21.15.67:0/201636918 learned_addr learned my addr 172.21.15.67:0/201636918 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.154+0000 7f0021e00640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 0x7f001c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.154+0000 7f0020a00640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f001c153510 0x7f001c0b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.154+0000 7f0021400640 1 -- 172.21.15.67:0/201636918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 msgr2=0x7f001c0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.154+0000 7f0021400640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 0x7f001c0b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.154+0000 7f0021400640 1 -- 172.21.15.67:0/201636918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f001c153510 msgr2=0x7f001c0b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.154+0000 7f0021400640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f001c153510 0x7f001c0b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.154+0000 7f0021400640 1 -- 172.21.15.67:0/201636918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c0af050 con 0x7f001c0cd2a0 2024-05-30T17:04:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.154+0000 7f0021400640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 0x7f001c150b20 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f001000c430 tx=0x7f001000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.155+0000 7f0019600640 1 -- 172.21.15.67:0/201636918 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010011790 con 0x7f001c0cd2a0 2024-05-30T17:04:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.155+0000 7f0019600640 1 -- 172.21.15.67:0/201636918 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0010017070 con 0x7f001c0cd2a0 2024-05-30T17:04:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.155+0000 7f0019600640 1 -- 172.21.15.67:0/201636918 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010011bb0 con 0x7f001c0cd2a0 2024-05-30T17:04:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.155+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001c0af330 con 0x7f001c0cd2a0 2024-05-30T17:04:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.155+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f001c0ce0d0 con 0x7f001c0cd2a0 2024-05-30T17:04:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.156+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f001c0689e0 con 0x7f001c0cd2a0 2024-05-30T17:04:51.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.158+0000 7f0019600640 1 -- 172.21.15.67:0/201636918 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f001001c460 con 0x7f001c0cd2a0 2024-05-30T17:04:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.159+0000 7f0019600640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efffc06e510 0x7efffc0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.159+0000 7f0019600640 1 -- 172.21.15.67:0/201636918 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5339+0+0 (secure 0 0 0) 0x7f001008b4f0 con 0x7f001c0cd2a0 2024-05-30T17:04:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.159+0000 7f0020a00640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efffc06e510 0x7efffc0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:51.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.160+0000 7f0020a00640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efffc06e510 0x7efffc0709d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0008034000 tx=0x7f000802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:51.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.163+0000 7f0019600640 1 -- 172.21.15.67:0/201636918 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f001005b400 con 0x7f001c0cd2a0 2024-05-30T17:04:51.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.369+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f001c160c60 con 0x7f001c0cd2a0 2024-05-30T17:04:51.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.915+0000 7f0019600640 1 -- 172.21.15.67:0/201636918 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) ==== 170+0+0 (secure 0 0 0) 0x7f001005ad00 con 0x7f001c0cd2a0 2024-05-30T17:04:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-05-30T17:04:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.919+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efffc06e510 msgr2=0x7efffc0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.919+0000 7f0022800640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efffc06e510 0x7efffc0709d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0008034000 tx=0x7f000802e000 comp rx=0 tx=0).stop 2024-05-30T17:04:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.919+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 msgr2=0x7f001c150b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.919+0000 7f0022800640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 0x7f001c150b20 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f001000c430 tx=0x7f001000c8f0 comp rx=0 tx=0).stop 2024-05-30T17:04:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.920+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 shutdown_connections 2024-05-30T17:04:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.920+0000 7f0022800640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efffc06e510 0x7efffc0709d0 secure :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0008034000 tx=0x7f000802e000 comp rx=0 tx=0).stop 2024-05-30T17:04:51.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.920+0000 7f0022800640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f001c15cb60 0x7f001c0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:51.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.920+0000 7f0022800640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f001c153510 0x7f001c0b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:51.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.920+0000 7f0022800640 1 --2- 172.21.15.67:0/201636918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cd2a0 0x7f001c150b20 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:51.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.920+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 >> 172.21.15.67:0/201636918 conn(0x7f001c06b900 msgr2=0x7f001c0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.922+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 shutdown_connections 2024-05-30T17:04:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:51.923+0000 7f0022800640 1 -- 172.21.15.67:0/201636918 wait complete. 2024-05-30T17:04:51.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-30T17:04:52.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.143+0000 7f15bb600640 1 Processor -- start 2024-05-30T17:04:52.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.143+0000 7f15bb600640 1 -- start start 2024-05-30T17:04:52.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.145+0000 7f15bb600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f15b40d0640 0x7f15b415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:52.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.145+0000 7f15bb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 0x7f15b40b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:52.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.145+0000 7f15bb600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b40d0060 con 0x7f15b40d0a20 2024-05-30T17:04:52.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.145+0000 7f15bb600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f15b40adbb0 con 0x7f15b40d0640 2024-05-30T17:04:52.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.145+0000 7f15bb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15b415db10 con 0x7f15b40b32c0 2024-05-30T17:04:52.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.145+0000 7f15b8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f15b40d0640 0x7f15b415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:52.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.145+0000 7f15b8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f15b40d0640 0x7f15b415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54810/0 (socket says 172.21.15.67:54810) 2024-05-30T17:04:52.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.145+0000 7f15b8c00640 1 -- 172.21.15.67:0/1700258576 learned_addr learned my addr 172.21.15.67:0/1700258576 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:52.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.146+0000 7f15b9600640 1 --2- 172.21.15.67:0/1700258576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 0x7f15b40b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:52.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.146+0000 7f15b8c00640 1 -- 172.21.15.67:0/1700258576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 msgr2=0x7f15b40b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:52.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.146+0000 7f15b8c00640 1 --2- 172.21.15.67:0/1700258576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 0x7f15b40b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.146+0000 7f15b8c00640 1 -- 172.21.15.67:0/1700258576 >> v1:172.21.15.67:6789/0 conn(0x7f15b40d0a20 legacy=0x7f15b40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:52.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.146+0000 7f15b8c00640 1 -- 172.21.15.67:0/1700258576 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b415cb10 con 0x7f15b40d0640 2024-05-30T17:04:52.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.146+0000 7f15b9600640 1 --2- 172.21.15.67:0/1700258576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 0x7f15b40b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:52.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.147+0000 7f15b8c00640 1 --2- 172.21.15.67:0/1700258576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 0x7f15b415d5d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009870 tx=0x7f15a402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed2ca2b46652188 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:52.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.147+0000 7f15b3e00640 1 -- 172.21.15.67:0/1700258576 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4037070 con 0x7f15b40d0640 2024-05-30T17:04:52.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15b3e00640 1 -- 172.21.15.67:0/1700258576 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a4032440 con 0x7f15b40d0640 2024-05-30T17:04:52.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15b3e00640 1 -- 172.21.15.67:0/1700258576 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4032810 con 0x7f15b40d0640 2024-05-30T17:04:52.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15bb600640 1 -- 172.21.15.67:0/1700258576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 msgr2=0x7f15b415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:52.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15bb600640 1 --2- 172.21.15.67:0/1700258576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 0x7f15b415d5d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009870 tx=0x7f15a402a250 comp rx=0 tx=0).stop 2024-05-30T17:04:52.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15bb600640 1 -- 172.21.15.67:0/1700258576 shutdown_connections 2024-05-30T17:04:52.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15bb600640 1 --2- 172.21.15.67:0/1700258576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 0x7f15b40b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15bb600640 1 --2- 172.21.15.67:0/1700258576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 0x7f15b415d5d0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15bb600640 1 -- 172.21.15.67:0/1700258576 >> 172.21.15.67:0/1700258576 conn(0x7f15b406bd20 msgr2=0x7f15b406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:52.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15bb600640 1 -- 172.21.15.67:0/1700258576 shutdown_connections 2024-05-30T17:04:52.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.148+0000 7f15bb600640 1 -- 172.21.15.67:0/1700258576 wait complete. 2024-05-30T17:04:52.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.150+0000 7f15bb600640 1 Processor -- start 2024-05-30T17:04:52.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.150+0000 7f15bb600640 1 -- start start 2024-05-30T17:04:52.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.151+0000 7f15bb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 0x7f15b41f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.151+0000 7f15bb600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 0x7f15b4099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.151+0000 7f15bb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b40d0a20 0x7f15b41f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.151+0000 7f15bb600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f15b415cdd0 con 0x7f15b40d0640 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.151+0000 7f15bb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40ad8d0 con 0x7f15b40d0a20 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.151+0000 7f15bb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15b415d0b0 con 0x7f15b40b32c0 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 0x7f15b4099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 0x7f15b4099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54816/0 (socket says 172.21.15.67:54816) 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15abe00640 1 -- 172.21.15.67:0/1487128361 learned_addr learned my addr 172.21.15.67:0/1487128361 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15b9600640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b40d0a20 0x7f15b41f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15b8c00640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 0x7f15b41f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15abe00640 1 -- 172.21.15.67:0/1487128361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 msgr2=0x7f15b41f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15abe00640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 0x7f15b41f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15abe00640 1 -- 172.21.15.67:0/1487128361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b40d0a20 msgr2=0x7f15b41f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15abe00640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b40d0a20 0x7f15b41f7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15abe00640 1 -- 172.21.15.67:0/1487128361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b41f44d0 con 0x7f15b40d0640 2024-05-30T17:04:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.152+0000 7f15b9600640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b40d0a20 0x7f15b41f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:04:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.153+0000 7f15abe00640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 0x7f15b4099850 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c3f0 tx=0x7f159c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.153+0000 7f15b1600640 1 -- 172.21.15.67:0/1487128361 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c0052d0 con 0x7f15b40d0640 2024-05-30T17:04:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.153+0000 7f15b1600640 1 -- 172.21.15.67:0/1487128361 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c005490 con 0x7f15b40d0640 2024-05-30T17:04:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.153+0000 7f15b1600640 1 -- 172.21.15.67:0/1487128361 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c007260 con 0x7f15b40d0640 2024-05-30T17:04:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.153+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b41f84b0 con 0x7f15b40d0640 2024-05-30T17:04:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.153+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b41f4790 con 0x7f15b40d0640 2024-05-30T17:04:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.154+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b40689e0 con 0x7f15b40d0640 2024-05-30T17:04:52.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.156+0000 7f15b1600640 1 -- 172.21.15.67:0/1487128361 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f159c007400 con 0x7f15b40d0640 2024-05-30T17:04:52.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.157+0000 7f15b1600640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f159006e510 0x7f15900709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:52.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.157+0000 7f15b1600640 1 -- 172.21.15.67:0/1487128361 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5721+0+0 (secure 0 0 0) 0x7f159c08a2e0 con 0x7f15b40d0640 2024-05-30T17:04:52.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.158+0000 7f15b8c00640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f159006e510 0x7f15900709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:52.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.158+0000 7f15b8c00640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f159006e510 0x7f15900709d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f15a4034000 tx=0x7f15a402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.161+0000 7f15b1600640 1 -- 172.21.15.67:0/1487128361 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f159c05a2f0 con 0x7f15b40d0640 2024-05-30T17:04:52.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.407+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f15b4071ef0 con 0x7f15b40d0640 2024-05-30T17:04:52.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.968+0000 7f15b1600640 1 -- 172.21.15.67:0/1487128361 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v78) ==== 174+0+0 (secure 0 0 0) 0x7f159c059bf0 con 0x7f15b40d0640 2024-05-30T17:04:52.972 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 18 and data pool 19 2024-05-30T17:04:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f159006e510 msgr2=0x7f15900709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f159006e510 0x7f15900709d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f15a4034000 tx=0x7f15a402e000 comp rx=0 tx=0).stop 2024-05-30T17:04:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 msgr2=0x7f15b4099850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 0x7f15b4099850 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c3f0 tx=0x7f159c00c8b0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 shutdown_connections 2024-05-30T17:04:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f159006e510 0x7f15900709d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b40d0a20 0x7f15b41f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15b40d0640 0x7f15b4099850 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 --2- 172.21.15.67:0/1487128361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b40b32c0 0x7f15b41f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.978+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 >> 172.21.15.67:0/1487128361 conn(0x7f15b406bd20 msgr2=0x7f15b40aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.979+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 shutdown_connections 2024-05-30T17:04:52.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:52.979+0000 7f15bb600640 1 -- 172.21.15.67:0/1487128361 wait complete. 2024-05-30T17:04:53.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:04:53.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.192+0000 7f5079400640 1 Processor -- start 2024-05-30T17:04:53.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.192+0000 7f5079400640 1 -- start start 2024-05-30T17:04:53.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.194+0000 7f5079400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f507406bb70 0x7f507409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.194+0000 7f5079400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f507406bf50 0x7f50740b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5079400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50740ae220 con 0x7f5074073230 2024-05-30T17:04:53.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5079400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f50740cfef0 con 0x7f507406bb70 2024-05-30T17:04:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5079400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50740d01d0 con 0x7f507406bf50 2024-05-30T17:04:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5073e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f507406bb70 0x7f507409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5073e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f507406bb70 0x7f507409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54826/0 (socket says 172.21.15.67:54826) 2024-05-30T17:04:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5073e00640 1 -- 172.21.15.67:0/347968576 learned_addr learned my addr 172.21.15.67:0/347968576 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5073e00640 1 -- 172.21.15.67:0/347968576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f507406bf50 msgr2=0x7f50740b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:53.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5078a00640 1 --2- 172.21.15.67:0/347968576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f507406bf50 0x7f50740b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5073e00640 1 --2- 172.21.15.67:0/347968576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f507406bf50 0x7f50740b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.195+0000 7f5073e00640 1 -- 172.21.15.67:0/347968576 >> v1:172.21.15.169:6789/0 conn(0x7f5074073230 legacy=0x7f507415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.196+0000 7f5073e00640 1 -- 172.21.15.67:0/347968576 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50740d04b0 con 0x7f507406bb70 2024-05-30T17:04:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.197+0000 7f5073e00640 1 --2- 172.21.15.67:0/347968576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bb70 0x7f507409aa30 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5064009870 tx=0x7f506402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34851ea2cd28fd4a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.197+0000 7f5073400640 1 -- 172.21.15.67:0/347968576 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064037070 con 0x7f507406bb70 2024-05-30T17:04:53.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.197+0000 7f5073400640 1 -- 172.21.15.67:0/347968576 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5064032440 con 0x7f507406bb70 2024-05-30T17:04:53.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.197+0000 7f5073400640 1 -- 172.21.15.67:0/347968576 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064032810 con 0x7f507406bb70 2024-05-30T17:04:53.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.198+0000 7f5079400640 1 -- 172.21.15.67:0/347968576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bb70 msgr2=0x7f507409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.198+0000 7f5079400640 1 --2- 172.21.15.67:0/347968576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bb70 0x7f507409aa30 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5064009870 tx=0x7f506402a210 comp rx=0 tx=0).stop 2024-05-30T17:04:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.198+0000 7f5079400640 1 -- 172.21.15.67:0/347968576 shutdown_connections 2024-05-30T17:04:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.198+0000 7f5079400640 1 --2- 172.21.15.67:0/347968576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f507406bf50 0x7f50740b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.198+0000 7f5079400640 1 --2- 172.21.15.67:0/347968576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bb70 0x7f507409aa30 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.198+0000 7f5079400640 1 -- 172.21.15.67:0/347968576 >> 172.21.15.67:0/347968576 conn(0x7f50740b6980 msgr2=0x7f50740b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.198+0000 7f5079400640 1 -- 172.21.15.67:0/347968576 shutdown_connections 2024-05-30T17:04:53.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.198+0000 7f5079400640 1 -- 172.21.15.67:0/347968576 wait complete. 2024-05-30T17:04:53.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.200+0000 7f5079400640 1 Processor -- start 2024-05-30T17:04:53.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.200+0000 7f5079400640 1 -- start start 2024-05-30T17:04:53.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.201+0000 7f5079400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406bb70 0x7f5074155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.201+0000 7f5079400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bf50 0x7f507415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.201+0000 7f5079400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5074073230 0x7f5074150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.201+0000 7f5079400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f50740b4dd0 con 0x7f507406bf50 2024-05-30T17:04:53.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.201+0000 7f5079400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50740b4c50 con 0x7f507406bb70 2024-05-30T17:04:53.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.201+0000 7f5079400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f507415c7e0 con 0x7f5074073230 2024-05-30T17:04:53.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.201+0000 7f506be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bf50 0x7f507415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.201+0000 7f506be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bf50 0x7f507415d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54832/0 (socket says 172.21.15.67:54832) 2024-05-30T17:04:53.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.201+0000 7f506be00640 1 -- 172.21.15.67:0/3158983378 learned_addr learned my addr 172.21.15.67:0/3158983378 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:53.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f5073e00640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406bb70 0x7f5074155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f506be00640 1 -- 172.21.15.67:0/3158983378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5074073230 msgr2=0x7f5074150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f506be00640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5074073230 0x7f5074150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f506be00640 1 -- 172.21.15.67:0/3158983378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406bb70 msgr2=0x7f5074155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f506be00640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406bb70 0x7f5074155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f506be00640 1 -- 172.21.15.67:0/3158983378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507415de20 con 0x7f507406bf50 2024-05-30T17:04:53.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f506be00640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bf50 0x7f507415d5a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f505c00b420 tx=0x7f505c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:53.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f5070c00640 1 -- 172.21.15.67:0/3158983378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c011830 con 0x7f507406bf50 2024-05-30T17:04:53.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f5070c00640 1 -- 172.21.15.67:0/3158983378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505c016070 con 0x7f507406bf50 2024-05-30T17:04:53.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f5070c00640 1 -- 172.21.15.67:0/3158983378 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c011c10 con 0x7f507406bf50 2024-05-30T17:04:53.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.202+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507415e100 con 0x7f507406bf50 2024-05-30T17:04:53.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.203+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5074153240 con 0x7f507406bf50 2024-05-30T17:04:53.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.204+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507414e630 con 0x7f507406bf50 2024-05-30T17:04:53.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.205+0000 7f5070c00640 1 -- 172.21.15.67:0/3158983378 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f505c01b460 con 0x7f507406bf50 2024-05-30T17:04:53.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.206+0000 7f5070c00640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f505006e510 0x7f50500709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.206+0000 7f5070c00640 1 -- 172.21.15.67:0/3158983378 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f505c08a610 con 0x7f507406bf50 2024-05-30T17:04:53.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.206+0000 7f5073e00640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f505006e510 0x7f50500709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.207+0000 7f5073e00640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f505006e510 0x7f50500709d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5064034000 tx=0x7f506402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:53.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.210+0000 7f5070c00640 1 -- 172.21.15.67:0/3158983378 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f507414e630 con 0x7f507406bf50 2024-05-30T17:04:53.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.419+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5074153440 con 0x7f507406bf50 2024-05-30T17:04:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.421+0000 7f5070c00640 1 -- 172.21.15.67:0/3158983378 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16857 (secure 0 0 0) 0x7f5074153440 con 0x7f507406bf50 2024-05-30T17:04:53.422 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:53.423 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":131,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:04:52.409318+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:04:50.327312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-30T17:04:51.388842+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:04:53.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f505006e510 msgr2=0x7f50500709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f505006e510 0x7f50500709d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5064034000 tx=0x7f506402e000 comp rx=0 tx=0).stop 2024-05-30T17:04:53.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bf50 msgr2=0x7f507415d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bf50 0x7f507415d5a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f505c00b420 tx=0x7f505c00b8e0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 shutdown_connections 2024-05-30T17:04:53.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f505006e510 0x7f50500709d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5074073230 0x7f5074150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406bf50 0x7f507415d5a0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 --2- 172.21.15.67:0/3158983378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406bb70 0x7f5074155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 >> 172.21.15.67:0/3158983378 conn(0x7f50740b6980 msgr2=0x7f5074074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 shutdown_connections 2024-05-30T17:04:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.424+0000 7f5079400640 1 -- 172.21.15.67:0/3158983378 wait complete. 2024-05-30T17:04:53.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:04:53.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.646+0000 7f431ae00640 1 Processor -- start 2024-05-30T17:04:53.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.647+0000 7f431ae00640 1 -- start start 2024-05-30T17:04:53.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.648+0000 7f431ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 0x7f43140af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.648+0000 7f431ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f431406ee40 0x7f43140ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.648+0000 7f431ae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4314156f10 con 0x7f431406f2c0 2024-05-30T17:04:53.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.648+0000 7f431ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f431406dda0 con 0x7f431406ee40 2024-05-30T17:04:53.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.648+0000 7f431ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f431406e080 con 0x7f4314156b30 2024-05-30T17:04:53.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.649+0000 7f4319000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f431406ee40 0x7f43140ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.649+0000 7f4319a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 0x7f43140af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.649+0000 7f4319a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 0x7f43140af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:55720/0 (socket says 172.21.15.67:55720) 2024-05-30T17:04:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.649+0000 7f4319a00640 1 -- 172.21.15.67:0/3516051008 learned_addr learned my addr 172.21.15.67:0/3516051008 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.650+0000 7f4319000640 1 -- 172.21.15.67:0/3516051008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 msgr2=0x7f43140af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.650+0000 7f4319000640 1 --2- 172.21.15.67:0/3516051008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 0x7f43140af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.650+0000 7f4319000640 1 -- 172.21.15.67:0/3516051008 >> v1:172.21.15.169:6789/0 conn(0x7f431406f2c0 legacy=0x7f43140b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.650+0000 7f4319000640 1 -- 172.21.15.67:0/3516051008 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431406e360 con 0x7f431406ee40 2024-05-30T17:04:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.650+0000 7f4319a00640 1 --2- 172.21.15.67:0/3516051008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 0x7f43140af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:04:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.651+0000 7f4319000640 1 --2- 172.21.15.67:0/3516051008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431406ee40 0x7f43140ad230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f43000097a0 tx=0x7f430002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63cd99c68f20b57f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:53.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.651+0000 7f4313e00640 1 -- 172.21.15.67:0/3516051008 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300037070 con 0x7f431406ee40 2024-05-30T17:04:53.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.651+0000 7f4313e00640 1 -- 172.21.15.67:0/3516051008 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430002ae40 con 0x7f431406ee40 2024-05-30T17:04:53.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.652+0000 7f4313e00640 1 -- 172.21.15.67:0/3516051008 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300032660 con 0x7f431406ee40 2024-05-30T17:04:53.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.652+0000 7f431ae00640 1 -- 172.21.15.67:0/3516051008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431406ee40 msgr2=0x7f43140ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.652+0000 7f431ae00640 1 --2- 172.21.15.67:0/3516051008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431406ee40 0x7f43140ad230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f43000097a0 tx=0x7f430002a0f0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.652+0000 7f431ae00640 1 -- 172.21.15.67:0/3516051008 shutdown_connections 2024-05-30T17:04:53.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.652+0000 7f431ae00640 1 --2- 172.21.15.67:0/3516051008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431406ee40 0x7f43140ad230 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.652+0000 7f431ae00640 1 --2- 172.21.15.67:0/3516051008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 0x7f43140af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.652+0000 7f431ae00640 1 -- 172.21.15.67:0/3516051008 >> 172.21.15.67:0/3516051008 conn(0x7f43140b1fc0 msgr2=0x7f43140b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:53.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.652+0000 7f431ae00640 1 -- 172.21.15.67:0/3516051008 shutdown_connections 2024-05-30T17:04:53.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.653+0000 7f431ae00640 1 -- 172.21.15.67:0/3516051008 wait complete. 2024-05-30T17:04:53.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.654+0000 7f431ae00640 1 Processor -- start 2024-05-30T17:04:53.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.654+0000 7f431ae00640 1 -- start start 2024-05-30T17:04:53.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.655+0000 7f431ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431406ee40 0x7f4314159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.655+0000 7f431ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431406f2c0 0x7f43141f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f431ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 0x7f431414a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f431ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f43140afc20 con 0x7f431406ee40 2024-05-30T17:04:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f431ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43140b2c30 con 0x7f431406f2c0 2024-05-30T17:04:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f431ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4314150de0 con 0x7f4314156b30 2024-05-30T17:04:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f4319a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431406ee40 0x7f4314159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f4319a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431406ee40 0x7f4314159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54834/0 (socket says 172.21.15.67:54834) 2024-05-30T17:04:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f4319a00640 1 -- 172.21.15.67:0/3622262643 learned_addr learned my addr 172.21.15.67:0/3622262643 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f4319000640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431406f2c0 0x7f43141f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f4319a00640 1 -- 172.21.15.67:0/3622262643 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 msgr2=0x7f431414a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f4319a00640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 0x7f431414a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f4319a00640 1 -- 172.21.15.67:0/3622262643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431406f2c0 msgr2=0x7f43141f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f4319a00640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431406f2c0 0x7f43141f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.656+0000 7f4319a00640 1 -- 172.21.15.67:0/3622262643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4314071fa0 con 0x7f431406ee40 2024-05-30T17:04:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.657+0000 7f4319a00640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431406ee40 0x7f4314159b20 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f430800e9b0 tx=0x7f430800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.657+0000 7f4311600640 1 -- 172.21.15.67:0/3622262643 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308017070 con 0x7f431406ee40 2024-05-30T17:04:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.657+0000 7f4311600640 1 -- 172.21.15.67:0/3622262643 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430800c9a0 con 0x7f431406ee40 2024-05-30T17:04:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.657+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4314072280 con 0x7f431406ee40 2024-05-30T17:04:53.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.657+0000 7f4311600640 1 -- 172.21.15.67:0/3622262643 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430800cd50 con 0x7f431406ee40 2024-05-30T17:04:53.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.657+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4314072840 con 0x7f431406ee40 2024-05-30T17:04:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.658+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43140689e0 con 0x7f431406ee40 2024-05-30T17:04:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.660+0000 7f4311600640 1 -- 172.21.15.67:0/3622262643 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f430801a070 con 0x7f431406ee40 2024-05-30T17:04:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.661+0000 7f4311600640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f406e510 0x7f42f40709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.661+0000 7f4311600640 1 -- 172.21.15.67:0/3622262643 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5821+0+0 (secure 0 0 0) 0x7f430808ad30 con 0x7f431406ee40 2024-05-30T17:04:53.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.661+0000 7f4319000640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f406e510 0x7f42f40709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:53.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.662+0000 7f4319000640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f406e510 0x7f42f40709d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4300034000 tx=0x7f430002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:53.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.665+0000 7f4311600640 1 -- 172.21.15.67:0/3622262643 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f430805abd0 con 0x7f431406ee40 2024-05-30T17:04:53.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.870+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f431406f7f0 con 0x7f431406ee40 2024-05-30T17:04:53.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.873+0000 7f4311600640 1 -- 172.21.15.67:0/3622262643 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16857 (secure 0 0 0) 0x7f430805a4d0 con 0x7f431406ee40 2024-05-30T17:04:53.874 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:53.875 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":131,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:04:52.409318+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:04:50.327312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-30T17:04:51.388842+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:04:53.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f406e510 msgr2=0x7f42f40709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f406e510 0x7f42f40709d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4300034000 tx=0x7f430002e000 comp rx=0 tx=0).stop 2024-05-30T17:04:53.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431406ee40 msgr2=0x7f4314159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:53.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431406ee40 0x7f4314159b20 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f430800e9b0 tx=0x7f430800ee70 comp rx=0 tx=0).stop 2024-05-30T17:04:53.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 shutdown_connections 2024-05-30T17:04:53.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f42f406e510 0x7f42f40709d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4314156b30 0x7f431414a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431406f2c0 0x7f43141f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 --2- 172.21.15.67:0/3622262643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431406ee40 0x7f4314159b20 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:53.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.875+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 >> 172.21.15.67:0/3622262643 conn(0x7f43140b1fc0 msgr2=0x7f4314155920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:53.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.876+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 shutdown_connections 2024-05-30T17:04:53.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:53.876+0000 7f431ae00640 1 -- 172.21.15.67:0/3622262643 wait complete. 2024-05-30T17:04:53.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-30T17:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.110+0000 7f16e0c00640 1 Processor -- start 2024-05-30T17:04:54.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.111+0000 7f16e0c00640 1 -- start start 2024-05-30T17:04:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.112+0000 7f16e0c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f16dc0cd2a0 0x7f16dc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.112+0000 7f16e0c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16dc15cb60 0x7f16dc15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.112+0000 7f16e0c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16dc15d900 con 0x7f16dc153510 2024-05-30T17:04:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.112+0000 7f16e0c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f16dc1538f0 con 0x7f16dc0cd2a0 2024-05-30T17:04:54.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.112+0000 7f16e0c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f16dc0b48c0 con 0x7f16dc15cb60 2024-05-30T17:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.113+0000 7f16d3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f16dc0cd2a0 0x7f16dc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.113+0000 7f16d3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f16dc0cd2a0 0x7f16dc0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54846/0 (socket says 172.21.15.67:54846) 2024-05-30T17:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.113+0000 7f16d3e00640 1 -- 172.21.15.67:0/1687274348 learned_addr learned my addr 172.21.15.67:0/1687274348 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.113+0000 7f16dbe00640 1 --2- 172.21.15.67:0/1687274348 >> v2:172.21.15.169:3300/0 conn(0x7f16dc15cb60 0x7f16dc15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.113+0000 7f16d3e00640 1 -- 172.21.15.67:0/1687274348 >> v2:172.21.15.169:3300/0 conn(0x7f16dc15cb60 msgr2=0x7f16dc15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.113+0000 7f16d3e00640 1 --2- 172.21.15.67:0/1687274348 >> v2:172.21.15.169:3300/0 conn(0x7f16dc15cb60 0x7f16dc15cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.113+0000 7f16d3e00640 1 -- 172.21.15.67:0/1687274348 >> v1:172.21.15.169:6789/0 conn(0x7f16dc153510 legacy=0x7f16dc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:54.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.113+0000 7f16d3e00640 1 -- 172.21.15.67:0/1687274348 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16dc0b3f90 con 0x7f16dc0cd2a0 2024-05-30T17:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.114+0000 7f16d3e00640 1 --2- 172.21.15.67:0/1687274348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc0cd2a0 0x7f16dc0b4d50 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0b4a40 tx=0x7f16c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d84ced8de51ead server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.115+0000 7f16daa00640 1 -- 172.21.15.67:0/1687274348 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4037070 con 0x7f16dc0cd2a0 2024-05-30T17:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.115+0000 7f16daa00640 1 -- 172.21.15.67:0/1687274348 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c4032440 con 0x7f16dc0cd2a0 2024-05-30T17:04:54.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.115+0000 7f16daa00640 1 -- 172.21.15.67:0/1687274348 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4032810 con 0x7f16dc0cd2a0 2024-05-30T17:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.115+0000 7f16e0c00640 1 -- 172.21.15.67:0/1687274348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc0cd2a0 msgr2=0x7f16dc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.116+0000 7f16e0c00640 1 --2- 172.21.15.67:0/1687274348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc0cd2a0 0x7f16dc0b4d50 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0b4a40 tx=0x7f16c402a180 comp rx=0 tx=0).stop 2024-05-30T17:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.116+0000 7f16e0c00640 1 -- 172.21.15.67:0/1687274348 shutdown_connections 2024-05-30T17:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.116+0000 7f16e0c00640 1 --2- 172.21.15.67:0/1687274348 >> v2:172.21.15.169:3300/0 conn(0x7f16dc15cb60 0x7f16dc15cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.116+0000 7f16e0c00640 1 --2- 172.21.15.67:0/1687274348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc0cd2a0 0x7f16dc0b4d50 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.116+0000 7f16e0c00640 1 -- 172.21.15.67:0/1687274348 >> 172.21.15.67:0/1687274348 conn(0x7f16dc06b900 msgr2=0x7f16dc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:54.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.116+0000 7f16e0c00640 1 -- 172.21.15.67:0/1687274348 shutdown_connections 2024-05-30T17:04:54.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.116+0000 7f16e0c00640 1 -- 172.21.15.67:0/1687274348 wait complete. 2024-05-30T17:04:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.119+0000 7f16e0c00640 1 Processor -- start 2024-05-30T17:04:54.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.119+0000 7f16e0c00640 1 -- start start 2024-05-30T17:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.120+0000 7f16e0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16dc0cd2a0 0x7f16dc149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.120+0000 7f16e0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16dc153510 0x7f16dc0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.120+0000 7f16e0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc15cb60 0x7f16dc0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.120+0000 7f16e0c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc15d4b0 con 0x7f16dc15cb60 2024-05-30T17:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.120+0000 7f16e0c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc0b4550 con 0x7f16dc0cd2a0 2024-05-30T17:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.120+0000 7f16e0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16dc15d630 con 0x7f16dc153510 2024-05-30T17:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.120+0000 7f16dbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc15cb60 0x7f16dc0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.121+0000 7f16dbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc15cb60 0x7f16dc0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54854/0 (socket says 172.21.15.67:54854) 2024-05-30T17:04:54.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.121+0000 7f16dbe00640 1 -- 172.21.15.67:0/182833869 learned_addr learned my addr 172.21.15.67:0/182833869 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.121+0000 7f16dbe00640 1 -- 172.21.15.67:0/182833869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16dc153510 msgr2=0x7f16dc0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.121+0000 7f16dbe00640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16dc153510 0x7f16dc0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.121+0000 7f16dbe00640 1 -- 172.21.15.67:0/182833869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16dc0cd2a0 msgr2=0x7f16dc149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.121+0000 7f16dbe00640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16dc0cd2a0 0x7f16dc149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.121+0000 7f16dbe00640 1 -- 172.21.15.67:0/182833869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16dc0ce960 con 0x7f16dc15cb60 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.121+0000 7f16dbe00640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc15cb60 0x7f16dc0ce290 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f16d4007e40 tx=0x7f16d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.122+0000 7f16d3400640 1 -- 172.21.15.67:0/182833869 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4014620 con 0x7f16dc15cb60 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.122+0000 7f16d3400640 1 -- 172.21.15.67:0/182833869 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d4019070 con 0x7f16dc15cb60 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.122+0000 7f16d3400640 1 -- 172.21.15.67:0/182833869 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4014a20 con 0x7f16dc15cb60 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.122+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16dc0cec40 con 0x7f16dc15cb60 2024-05-30T17:04:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.122+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16dc0cf200 con 0x7f16dc15cb60 2024-05-30T17:04:54.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.123+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16dc0689e0 con 0x7f16dc15cb60 2024-05-30T17:04:54.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.124+0000 7f16d3400640 1 -- 172.21.15.67:0/182833869 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f16d4014bc0 con 0x7f16dc15cb60 2024-05-30T17:04:54.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.125+0000 7f16d3400640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16b806e540 0x7f16b8070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:54.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.125+0000 7f16d3400640 1 -- 172.21.15.67:0/182833869 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f16d408b340 con 0x7f16dc15cb60 2024-05-30T17:04:54.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.125+0000 7f16db400640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16b806e540 0x7f16b8070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:54.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.126+0000 7f16db400640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16b806e540 0x7f16b8070a00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f16cc001560 tx=0x7f16cc007990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:54.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.129+0000 7f16d3400640 1 -- 172.21.15.67:0/182833869 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16d405b270 con 0x7f16dc15cb60 2024-05-30T17:04:54.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.371+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f16dc0b3c40 con 0x7f16dc15cb60 2024-05-30T17:04:54.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.994+0000 7f16d3400640 1 -- 172.21.15.67:0/182833869 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7f16d405ab70 con 0x7f16dc15cb60 2024-05-30T17:04:54.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16b806e540 msgr2=0x7f16b8070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16b806e540 0x7f16b8070a00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f16cc001560 tx=0x7f16cc007990 comp rx=0 tx=0).stop 2024-05-30T17:04:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc15cb60 msgr2=0x7f16dc0ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc15cb60 0x7f16dc0ce290 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f16d4007e40 tx=0x7f16d400e4b0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 shutdown_connections 2024-05-30T17:04:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f16b806e540 0x7f16b8070a00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16dc15cb60 0x7f16dc0ce290 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16dc153510 0x7f16dc0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 --2- 172.21.15.67:0/182833869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16dc0cd2a0 0x7f16dc149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.998+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 >> 172.21.15.67:0/182833869 conn(0x7f16dc06b900 msgr2=0x7f16dc06f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:55.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.999+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 shutdown_connections 2024-05-30T17:04:55.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:54.999+0000 7f16e0c00640 1 -- 172.21.15.67:0/182833869 wait complete. 2024-05-30T17:04:55.021 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-30T17:04:55.021 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.238+0000 7fee96600640 1 Processor -- start 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.238+0000 7fee96600640 1 -- start start 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.239+0000 7fee96600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9006fa70 0x7fee900ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.239+0000 7fee96600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fee900b2970 0x7fee900ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.239+0000 7fee96600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee900ca590 con 0x7fee90098f10 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.239+0000 7fee96600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fee900ce600 con 0x7fee900b2970 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.239+0000 7fee96600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fee90099cd0 con 0x7fee9006fa70 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.240+0000 7fee95c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fee900b2970 0x7fee900ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.240+0000 7fee95c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fee900b2970 0x7fee900ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54864/0 (socket says 172.21.15.67:54864) 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.240+0000 7fee95c00640 1 -- 172.21.15.67:0/3259916850 learned_addr learned my addr 172.21.15.67:0/3259916850 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:55.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.241+0000 7fee8f400640 1 -- 172.21.15.67:0/3259916850 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1196523849 0 0) 0x7fee900ca590 con 0x7fee90098f10 2024-05-30T17:04:55.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.240+0000 7fee8fe00640 1 --2- 172.21.15.67:0/3259916850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9006fa70 0x7fee900ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:55.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.246+0000 7fee8f400640 1 -- 172.21.15.67:0/3259916850 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee74003500 con 0x7fee90098f10 2024-05-30T17:04:55.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.246+0000 7fee95c00640 1 -- 172.21.15.67:0/3259916850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9006fa70 msgr2=0x7fee900ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:55.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.246+0000 7fee95c00640 1 --2- 172.21.15.67:0/3259916850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9006fa70 0x7fee900ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:55.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.246+0000 7fee95c00640 1 -- 172.21.15.67:0/3259916850 >> v1:172.21.15.169:6789/0 conn(0x7fee90098f10 legacy=0x7fee900ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:55.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.246+0000 7fee95c00640 1 -- 172.21.15.67:0/3259916850 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee900cfd60 con 0x7fee900b2970 2024-05-30T17:04:55.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.247+0000 7fee95c00640 1 --2- 172.21.15.67:0/3259916850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee900b2970 0x7fee900ad840 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fee8800b970 tx=0x7fee88033070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd4db4601c009b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:55.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.247+0000 7fee8f400640 1 -- 172.21.15.67:0/3259916850 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88039070 con 0x7fee900b2970 2024-05-30T17:04:55.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.247+0000 7fee8f400640 1 -- 172.21.15.67:0/3259916850 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee88033dc0 con 0x7fee900b2970 2024-05-30T17:04:55.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.247+0000 7fee8f400640 1 -- 172.21.15.67:0/3259916850 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8803d620 con 0x7fee900b2970 2024-05-30T17:04:55.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.248+0000 7fee96600640 1 -- 172.21.15.67:0/3259916850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee900b2970 msgr2=0x7fee900ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:55.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.248+0000 7fee96600640 1 --2- 172.21.15.67:0/3259916850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee900b2970 0x7fee900ad840 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fee8800b970 tx=0x7fee88033070 comp rx=0 tx=0).stop 2024-05-30T17:04:55.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.248+0000 7fee96600640 1 -- 172.21.15.67:0/3259916850 shutdown_connections 2024-05-30T17:04:55.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.248+0000 7fee96600640 1 --2- 172.21.15.67:0/3259916850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee900b2970 0x7fee900ad840 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:55.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.248+0000 7fee96600640 1 --2- 172.21.15.67:0/3259916850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9006fa70 0x7fee900ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:55.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.248+0000 7fee96600640 1 -- 172.21.15.67:0/3259916850 >> 172.21.15.67:0/3259916850 conn(0x7fee900b1e50 msgr2=0x7fee900b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:55.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.248+0000 7fee96600640 1 -- 172.21.15.67:0/3259916850 shutdown_connections 2024-05-30T17:04:55.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.248+0000 7fee96600640 1 -- 172.21.15.67:0/3259916850 wait complete. 2024-05-30T17:04:55.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.250+0000 7fee96600640 1 Processor -- start 2024-05-30T17:04:55.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.250+0000 7fee96600640 1 -- start start 2024-05-30T17:04:55.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.251+0000 7fee96600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee9006fa70 0x7fee9016d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:55.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.251+0000 7fee96600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee90098f10 0x7fee9008fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:55.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.251+0000 7fee96600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee900b2970 0x7fee90090330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:55.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.251+0000 7fee96600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fee900992f0 con 0x7fee9006fa70 2024-05-30T17:04:55.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.251+0000 7fee96600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee900999f0 con 0x7fee90098f10 2024-05-30T17:04:55.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.251+0000 7fee96600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fee900add80 con 0x7fee900b2970 2024-05-30T17:04:55.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.251+0000 7fee95200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee9006fa70 0x7fee9016d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:55.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.251+0000 7fee95200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee9006fa70 0x7fee9016d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54880/0 (socket says 172.21.15.67:54880) 2024-05-30T17:04:55.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.251+0000 7fee95200640 1 -- 172.21.15.67:0/2565799073 learned_addr learned my addr 172.21.15.67:0/2565799073 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:55.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee95200640 1 -- 172.21.15.67:0/2565799073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee900b2970 msgr2=0x7fee90090330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:55.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee95200640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee900b2970 0x7fee90090330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:55.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee95200640 1 -- 172.21.15.67:0/2565799073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee90098f10 msgr2=0x7fee9008fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:55.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee95200640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee90098f10 0x7fee9008fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:55.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee95200640 1 -- 172.21.15.67:0/2565799073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9008d8f0 con 0x7fee9006fa70 2024-05-30T17:04:55.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee95200640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee9006fa70 0x7fee9016d390 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fee7800c530 tx=0x7fee7800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:55.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee8cc00640 1 -- 172.21.15.67:0/2565799073 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780052d0 con 0x7fee9006fa70 2024-05-30T17:04:55.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee8cc00640 1 -- 172.21.15.67:0/2565799073 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee78005490 con 0x7fee9006fa70 2024-05-30T17:04:55.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee8cc00640 1 -- 172.21.15.67:0/2565799073 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78012870 con 0x7fee9006fa70 2024-05-30T17:04:55.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9008dbd0 con 0x7fee9006fa70 2024-05-30T17:04:55.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.252+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9008e190 con 0x7fee9006fa70 2024-05-30T17:04:55.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.254+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee900689e0 con 0x7fee9006fa70 2024-05-30T17:04:55.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.255+0000 7fee8cc00640 1 -- 172.21.15.67:0/2565799073 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fee78012a10 con 0x7fee9006fa70 2024-05-30T17:04:55.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.256+0000 7fee8cc00640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee6c06e510 0x7fee6c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:55.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.256+0000 7fee8cc00640 1 -- 172.21.15.67:0/2565799073 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7fee7808a7e0 con 0x7fee9006fa70 2024-05-30T17:04:55.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.256+0000 7fee8fe00640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee6c06e510 0x7fee6c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:55.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.257+0000 7fee8fe00640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee6c06e510 0x7fee6c0709d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fee800060d0 tx=0x7fee80006020 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.261+0000 7fee8cc00640 1 -- 172.21.15.67:0/2565799073 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee7805a5f0 con 0x7fee9006fa70 2024-05-30T17:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.497+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fee900b2480 con 0x7fee9006fa70 2024-05-30T17:04:55.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:55.997+0000 7fee8cc00640 1 -- 172.21.15.67:0/2565799073 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7fee78059ef0 con 0x7fee9006fa70 2024-05-30T17:04:56.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee6c06e510 msgr2=0x7fee6c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee6c06e510 0x7fee6c0709d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fee800060d0 tx=0x7fee80006020 comp rx=0 tx=0).stop 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee9006fa70 msgr2=0x7fee9016d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee9006fa70 0x7fee9016d390 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fee7800c530 tx=0x7fee7800c9f0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 shutdown_connections 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fee6c06e510 0x7fee6c0709d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee900b2970 0x7fee90090330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee90098f10 0x7fee9008fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 --2- 172.21.15.67:0/2565799073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee9006fa70 0x7fee9016d390 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 >> 172.21.15.67:0/2565799073 conn(0x7fee900b1e50 msgr2=0x7fee900977a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 shutdown_connections 2024-05-30T17:04:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.001+0000 7fee96600640 1 -- 172.21.15.67:0/2565799073 wait complete. 2024-05-30T17:04:56.024 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-30T17:04:56.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:04:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.239+0000 7f9bf9a00640 1 Processor -- start 2024-05-30T17:04:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.239+0000 7f9bf9a00640 1 -- start start 2024-05-30T17:04:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.240+0000 7f9bf9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9bf406f860 0x7f9bf40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.240+0000 7f9bf9a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf406ed10 con 0x7f9bf40d0bb0 2024-05-30T17:04:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.240+0000 7f9bf9a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf415cb00 con 0x7f9bf40b3690 2024-05-30T17:04:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.240+0000 7f9bf9a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf40b3cc0 con 0x7f9bf406f860 2024-05-30T17:04:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.240+0000 7f9bf2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9bf406f860 0x7f9bf40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.240+0000 7f9bf2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9bf406f860 0x7f9bf40b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54890/0 (socket says 172.21.15.67:54890) 2024-05-30T17:04:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.240+0000 7f9bf2a00640 1 -- 172.21.15.67:0/877187640 learned_addr learned my addr 172.21.15.67:0/877187640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.241+0000 7f9bf2a00640 1 -- 172.21.15.67:0/877187640 >> v1:172.21.15.169:6789/0 conn(0x7f9bf40b3690 legacy=0x7f9bf40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.241+0000 7f9bf2a00640 1 -- 172.21.15.67:0/877187640 >> v1:172.21.15.67:6789/0 conn(0x7f9bf40d0bb0 legacy=0x7f9bf40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.241+0000 7f9bf2a00640 1 -- 172.21.15.67:0/877187640 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf40b3fa0 con 0x7f9bf406f860 2024-05-30T17:04:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.242+0000 7f9bf2a00640 1 --2- 172.21.15.67:0/877187640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf406f860 0x7f9bf40b3150 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009870 tx=0x7f9be402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8197a3bf0ea262af server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.242+0000 7f9bf1600640 1 -- 172.21.15.67:0/877187640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4037070 con 0x7f9bf406f860 2024-05-30T17:04:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.242+0000 7f9bf1600640 1 -- 172.21.15.67:0/877187640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be4032440 con 0x7f9bf406f860 2024-05-30T17:04:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.242+0000 7f9bf1600640 1 -- 172.21.15.67:0/877187640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4032810 con 0x7f9bf406f860 2024-05-30T17:04:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.243+0000 7f9bf9a00640 1 -- 172.21.15.67:0/877187640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf406f860 msgr2=0x7f9bf40b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.243+0000 7f9bf9a00640 1 --2- 172.21.15.67:0/877187640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf406f860 0x7f9bf40b3150 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009870 tx=0x7f9be402a2a0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.243+0000 7f9bf9a00640 1 -- 172.21.15.67:0/877187640 shutdown_connections 2024-05-30T17:04:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.243+0000 7f9bf9a00640 1 --2- 172.21.15.67:0/877187640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf406f860 0x7f9bf40b3150 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.243+0000 7f9bf9a00640 1 -- 172.21.15.67:0/877187640 >> 172.21.15.67:0/877187640 conn(0x7f9bf406bae0 msgr2=0x7f9bf406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.243+0000 7f9bf9a00640 1 -- 172.21.15.67:0/877187640 shutdown_connections 2024-05-30T17:04:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.243+0000 7f9bf9a00640 1 -- 172.21.15.67:0/877187640 wait complete. 2024-05-30T17:04:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.245+0000 7f9bf9a00640 1 Processor -- start 2024-05-30T17:04:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.245+0000 7f9bf9a00640 1 -- start start 2024-05-30T17:04:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.246+0000 7f9bf9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf406f860 0x7f9bf415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.246+0000 7f9bf9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf40b3690 0x7f9bf40c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.246+0000 7f9bf9a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf40d0bb0 0x7f9bf4072420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.246+0000 7f9bf9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf40b4840 con 0x7f9bf40b3690 2024-05-30T17:04:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.246+0000 7f9bf9a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf40b4560 con 0x7f9bf40d0bb0 2024-05-30T17:04:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.246+0000 7f9bf9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf40ad840 con 0x7f9bf406f860 2024-05-30T17:04:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.246+0000 7f9bf2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf40b3690 0x7f9bf40c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.246+0000 7f9bf2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf40b3690 0x7f9bf40c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54898/0 (socket says 172.21.15.67:54898) 2024-05-30T17:04:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.246+0000 7f9bf2000640 1 -- 172.21.15.67:0/1700050274 learned_addr learned my addr 172.21.15.67:0/1700050274 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9bf2000640 1 -- 172.21.15.67:0/1700050274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf406f860 msgr2=0x7f9bf415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9bf2000640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf406f860 0x7f9bf415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9bf2000640 1 -- 172.21.15.67:0/1700050274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf40d0bb0 msgr2=0x7f9bf4072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9bf2000640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf40d0bb0 0x7f9bf4072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9bf2000640 1 -- 172.21.15.67:0/1700050274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf41f5860 con 0x7f9bf40b3690 2024-05-30T17:04:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9bf2000640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf40b3690 0x7f9bf40c52a0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00c7c0 tx=0x7f9bdc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9beaa00640 1 -- 172.21.15.67:0/1700050274 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc016020 con 0x7f9bf40b3690 2024-05-30T17:04:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9beaa00640 1 -- 172.21.15.67:0/1700050274 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bdc00aa50 con 0x7f9bf40b3690 2024-05-30T17:04:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9beaa00640 1 -- 172.21.15.67:0/1700050274 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc012420 con 0x7f9bf40b3690 2024-05-30T17:04:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.247+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf41f5b40 con 0x7f9bf40b3690 2024-05-30T17:04:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.248+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf40c7880 con 0x7f9bf40b3690 2024-05-30T17:04:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.249+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf40689e0 con 0x7f9bf40b3690 2024-05-30T17:04:56.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.250+0000 7f9beaa00640 1 -- 172.21.15.67:0/1700050274 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f9bdc019020 con 0x7f9bf40b3690 2024-05-30T17:04:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.251+0000 7f9beaa00640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006e510 0x7f9bd00709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.251+0000 7f9beaa00640 1 -- 172.21.15.67:0/1700050274 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f9bdc08a880 con 0x7f9bf40b3690 2024-05-30T17:04:56.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.252+0000 7f9bf2a00640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006e510 0x7f9bd00709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:56.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.252+0000 7f9bf2a00640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006e510 0x7f9bd00709d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9be4034000 tx=0x7f9be402c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:56.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.256+0000 7f9beaa00640 1 -- 172.21.15.67:0/1700050274 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bdc020070 con 0x7f9bf40b3690 2024-05-30T17:04:56.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.497+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bf406e2f0 con 0x7f9bf40b3690 2024-05-30T17:04:56.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.499+0000 7f9beaa00640 1 -- 172.21.15.67:0/1700050274 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5186 (secure 0 0 0) 0x7f9bdc05a710 con 0x7f9bf40b3690 2024-05-30T17:04:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:56.499 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2461528462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2461528462},{"type":"v1","addr":"172.21.15.67:6837","nonce":2461528462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/4229871041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4229871041},{"type":"v1","addr":"172.21.15.169:6833","nonce":4229871041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/217648840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":217648840},{"type":"v1","addr":"172.21.15.67:6835","nonce":217648840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:04:52.409278+0000","modified":"2024-05-30T17:04:55.498697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6937":{"gid":6937,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2504652568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2504652568},{"type":"v1","addr":"172.21.15.169:6835","nonce":2504652568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6937,"qdb_cluster":[6937]},"id":9}]} 2024-05-30T17:04:56.500 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 82 2024-05-30T17:04:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006e510 msgr2=0x7f9bd00709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006e510 0x7f9bd00709d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9be4034000 tx=0x7f9be402c000 comp rx=0 tx=0).stop 2024-05-30T17:04:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf40b3690 msgr2=0x7f9bf40c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf40b3690 0x7f9bf40c52a0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00c7c0 tx=0x7f9bdc00cc80 comp rx=0 tx=0).stop 2024-05-30T17:04:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 shutdown_connections 2024-05-30T17:04:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd006e510 0x7f9bd00709d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf40d0bb0 0x7f9bf4072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf40b3690 0x7f9bf40c52a0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 --2- 172.21.15.67:0/1700050274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf406f860 0x7f9bf415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.501+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 >> 172.21.15.67:0/1700050274 conn(0x7f9bf406bae0 msgr2=0x7f9bf40cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:56.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.502+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 shutdown_connections 2024-05-30T17:04:56.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.502+0000 7f9bf9a00640 1 -- 172.21.15.67:0/1700050274 wait complete. 2024-05-30T17:04:56.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:04:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.726+0000 7f719fe00640 1 Processor -- start 2024-05-30T17:04:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.727+0000 7f719fe00640 1 -- start start 2024-05-30T17:04:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.728+0000 7f719fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a0156b30 0x7f71a00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.728+0000 7f719fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71a00ad940 0x7f71a00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.728+0000 7f719fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a0156f10 con 0x7f71a006ee40 2024-05-30T17:04:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.728+0000 7f719fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f71a006dda0 con 0x7f71a00ad940 2024-05-30T17:04:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.728+0000 7f719fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71a006e080 con 0x7f71a0156b30 2024-05-30T17:04:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.729+0000 7f719f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71a00ad940 0x7f71a00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.729+0000 7f719f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71a00ad940 0x7f71a00b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54902/0 (socket says 172.21.15.67:54902) 2024-05-30T17:04:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.729+0000 7f719f400640 1 -- 172.21.15.67:0/2025733259 learned_addr learned my addr 172.21.15.67:0/2025733259 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.729+0000 7f719ea00640 1 --2- 172.21.15.67:0/2025733259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a0156b30 0x7f71a00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.729+0000 7f719f400640 1 -- 172.21.15.67:0/2025733259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a0156b30 msgr2=0x7f71a00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.729+0000 7f719f400640 1 --2- 172.21.15.67:0/2025733259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a0156b30 0x7f71a00af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.729+0000 7f719f400640 1 -- 172.21.15.67:0/2025733259 >> v1:172.21.15.169:6789/0 conn(0x7f71a006ee40 legacy=0x7f71a00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.729+0000 7f719f400640 1 -- 172.21.15.67:0/2025733259 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a006e360 con 0x7f71a00ad940 2024-05-30T17:04:56.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.730+0000 7f719f400640 1 --2- 172.21.15.67:0/2025733259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ad940 0x7f71a00b32c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f7198009e50 tx=0x7f719802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49952dce1b90f99f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719e000640 1 -- 172.21.15.67:0/2025733259 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198036070 con 0x7f71a00ad940 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719e000640 1 -- 172.21.15.67:0/2025733259 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7198031600 con 0x7f71a00ad940 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719e000640 1 -- 172.21.15.67:0/2025733259 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71980319d0 con 0x7f71a00ad940 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719fe00640 1 -- 172.21.15.67:0/2025733259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ad940 msgr2=0x7f71a00b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719fe00640 1 --2- 172.21.15.67:0/2025733259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ad940 0x7f71a00b32c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f7198009e50 tx=0x7f719802a630 comp rx=0 tx=0).stop 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719fe00640 1 -- 172.21.15.67:0/2025733259 shutdown_connections 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719fe00640 1 --2- 172.21.15.67:0/2025733259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ad940 0x7f71a00b32c0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719fe00640 1 --2- 172.21.15.67:0/2025733259 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a0156b30 0x7f71a00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719fe00640 1 -- 172.21.15.67:0/2025733259 >> 172.21.15.67:0/2025733259 conn(0x7f71a00b1fc0 msgr2=0x7f71a00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719fe00640 1 -- 172.21.15.67:0/2025733259 shutdown_connections 2024-05-30T17:04:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.731+0000 7f719fe00640 1 -- 172.21.15.67:0/2025733259 wait complete. 2024-05-30T17:04:56.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.733+0000 7f719fe00640 1 Processor -- start 2024-05-30T17:04:56.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.733+0000 7f719fe00640 1 -- start start 2024-05-30T17:04:56.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.734+0000 7f719fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a006ee40 0x7f71a01458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.734+0000 7f719fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a00ad940 0x7f71a00b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.734+0000 7f719fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0156b30 0x7f71a0155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.734+0000 7f719fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f71a00afc20 con 0x7f71a0156b30 2024-05-30T17:04:56.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.734+0000 7f719fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71a00b2c30 con 0x7f71a00ad940 2024-05-30T17:04:56.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.734+0000 7f719fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71a0158820 con 0x7f71a006ee40 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.734+0000 7f719f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0156b30 0x7f71a0155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.734+0000 7f719f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0156b30 0x7f71a0155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54906/0 (socket says 172.21.15.67:54906) 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.734+0000 7f719f400640 1 -- 172.21.15.67:0/723012762 learned_addr learned my addr 172.21.15.67:0/723012762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f719ea00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a006ee40 0x7f71a01458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f7197e00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a00ad940 0x7f71a00b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f719f400640 1 -- 172.21.15.67:0/723012762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a006ee40 msgr2=0x7f71a01458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f719f400640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a006ee40 0x7f71a01458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f719f400640 1 -- 172.21.15.67:0/723012762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a00ad940 msgr2=0x7f71a00b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f719f400640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a00ad940 0x7f71a00b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f719f400640 1 -- 172.21.15.67:0/723012762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a0158ac0 con 0x7f71a0156b30 2024-05-30T17:04:56.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f7197e00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a00ad940 0x7f71a00b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f719f400640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0156b30 0x7f71a0155830 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f7198034040 tx=0x7f7198004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.735+0000 7f7196a00640 1 -- 172.21.15.67:0/723012762 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198036050 con 0x7f71a0156b30 2024-05-30T17:04:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.736+0000 7f7196a00640 1 -- 172.21.15.67:0/723012762 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7198042070 con 0x7f71a0156b30 2024-05-30T17:04:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.736+0000 7f7196a00640 1 -- 172.21.15.67:0/723012762 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198009510 con 0x7f71a0156b30 2024-05-30T17:04:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.736+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a0158da0 con 0x7f71a0156b30 2024-05-30T17:04:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.736+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a0159360 con 0x7f71a0156b30 2024-05-30T17:04:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.737+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a015bdd0 con 0x7f71a0156b30 2024-05-30T17:04:56.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.738+0000 7f7196a00640 1 -- 172.21.15.67:0/723012762 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f7198007bf0 con 0x7f71a0156b30 2024-05-30T17:04:56.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.739+0000 7f7196a00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f717c06e510 0x7f717c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:56.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.739+0000 7f7196a00640 1 -- 172.21.15.67:0/723012762 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f71980aba40 con 0x7f71a0156b30 2024-05-30T17:04:56.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.739+0000 7f719ea00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f717c06e510 0x7f717c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:56.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.740+0000 7f719ea00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f717c06e510 0x7f717c0709d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f719000a970 tx=0x7f7190008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:56.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.744+0000 7f7196a00640 1 -- 172.21.15.67:0/723012762 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71a015bdd0 con 0x7f71a0156b30 2024-05-30T17:04:56.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.938+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71a00689e0 con 0x7f71a0156b30 2024-05-30T17:04:56.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.941+0000 7f7196a00640 1 -- 172.21.15.67:0/723012762 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16844 (secure 0 0 0) 0x7f7198045360 con 0x7f71a0156b30 2024-05-30T17:04:56.942 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:56.942 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":132,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:04:53.946964+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:04:50.327312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-30T17:04:51.388842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:04:56.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.943+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f717c06e510 msgr2=0x7f717c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.943+0000 7f719fe00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f717c06e510 0x7f717c0709d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f719000a970 tx=0x7f7190008040 comp rx=0 tx=0).stop 2024-05-30T17:04:56.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.943+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0156b30 msgr2=0x7f71a0155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:56.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.943+0000 7f719fe00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0156b30 0x7f71a0155830 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f7198034040 tx=0x7f7198004a40 comp rx=0 tx=0).stop 2024-05-30T17:04:56.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.944+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 shutdown_connections 2024-05-30T17:04:56.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.944+0000 7f719fe00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f717c06e510 0x7f717c0709d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.944+0000 7f719fe00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0156b30 0x7f71a0155830 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.944+0000 7f719fe00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a00ad940 0x7f71a00b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.944+0000 7f719fe00640 1 --2- 172.21.15.67:0/723012762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71a006ee40 0x7f71a01458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:56.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.944+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 >> 172.21.15.67:0/723012762 conn(0x7f71a00b1fc0 msgr2=0x7f71a00b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:56.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.944+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 shutdown_connections 2024-05-30T17:04:56.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:56.944+0000 7f719fe00640 1 -- 172.21.15.67:0/723012762 wait complete. 2024-05-30T17:04:56.967 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T17:04:56.967 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:04:57.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.159+0000 7f984ec00640 1 Processor -- start 2024-05-30T17:04:57.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.160+0000 7f984ec00640 1 -- start start 2024-05-30T17:04:57.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.161+0000 7f984ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9848156b30 0x7f98480af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.161+0000 7f984ec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f984806ee40 0x7f98480ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.161+0000 7f984ec00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9848156f10 con 0x7f984806f2c0 2024-05-30T17:04:57.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.161+0000 7f984ec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f984806dda0 con 0x7f984806ee40 2024-05-30T17:04:57.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.161+0000 7f984ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f984806e080 con 0x7f9848156b30 2024-05-30T17:04:57.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.162+0000 7f984e200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f984806f2c0 0x7f98480b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44064/0 (socket says 172.21.15.67:44064) 2024-05-30T17:04:57.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.162+0000 7f984e200640 1 -- 172.21.15.67:0/1664002937 learned_addr learned my addr 172.21.15.67:0/1664002937 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:57.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.162+0000 7f984ce00640 1 --2- 172.21.15.67:0/1664002937 >> v2:172.21.15.169:3300/0 conn(0x7f984806ee40 0x7f98480ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.162+0000 7f984d800640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9848156b30 0x7f98480af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.162+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522703577 0 0) 0x7f9848156f10 con 0x7f984806f2c0 2024-05-30T17:04:57.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.162+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9830003520 con 0x7f984806f2c0 2024-05-30T17:04:57.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4138372752 0 0) 0x7f9830003520 con 0x7f984806f2c0 2024-05-30T17:04:57.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9848156b30 msgr2=0x7f98480af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9848156b30 0x7f98480af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 >> v2:172.21.15.169:3300/0 conn(0x7f984806ee40 msgr2=0x7f98480ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 --2- 172.21.15.67:0/1664002937 >> v2:172.21.15.169:3300/0 conn(0x7f984806ee40 0x7f98480ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984806e360 con 0x7f984806f2c0 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f98400019f0 con 0x7f984806f2c0 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 >> v1:172.21.15.67:6789/0 conn(0x7f984806f2c0 legacy=0x7f98480b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98300039b0 0x7f9830003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9848156b30 0x7f98300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f984806dda0 con 0x7f9830004800 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f984806e080 con 0x7f98300039b0 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f983000b9c0 con 0x7f9848156b30 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f984ce00640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f984ce00640 1 -- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9848156b30 msgr2=0x7f98300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:57.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f984e200640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9848156b30 0x7f98300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.163+0000 7f984d800640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98300039b0 0x7f9830003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.164+0000 7f984ce00640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9848156b30 0x7f98300051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.164+0000 7f984ce00640 1 -- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98300039b0 msgr2=0x7f9830003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:57.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.164+0000 7f984ce00640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98300039b0 0x7f9830003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.164+0000 7f984ce00640 1 -- 172.21.15.67:0/1664002937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9830003520 con 0x7f9830004800 2024-05-30T17:04:57.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.164+0000 7f984d800640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98300039b0 0x7f9830003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.165+0000 7f984ce00640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9830004c80 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f9834009af0 tx=0x7f9830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887c7486239c9847 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.165+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834036070 con 0x7f9830004800 2024-05-30T17:04:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.165+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9834031400 con 0x7f9830004800 2024-05-30T17:04:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.165+0000 7f9847e00640 1 -- 172.21.15.67:0/1664002937 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834009690 con 0x7f9830004800 2024-05-30T17:04:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.165+0000 7f984ec00640 1 -- 172.21.15.67:0/1664002937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 msgr2=0x7f9830004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.165+0000 7f984ec00640 1 --2- 172.21.15.67:0/1664002937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9830004c80 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f9834009af0 tx=0x7f9830005830 comp rx=0 tx=0).stop 2024-05-30T17:04:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.165+0000 7f984e200640 1 -- 172.21.15.67:0/1664002937 reap_dead start 2024-05-30T17:04:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.166+0000 7f984ec00640 1 -- 172.21.15.67:0/1664002937 shutdown_connections 2024-05-30T17:04:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.166+0000 7f984ec00640 1 -- 172.21.15.67:0/1664002937 >> 172.21.15.67:0/1664002937 conn(0x7f98480b1fc0 msgr2=0x7f98480b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.166+0000 7f984ec00640 1 -- 172.21.15.67:0/1664002937 shutdown_connections 2024-05-30T17:04:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.166+0000 7f984ec00640 1 -- 172.21.15.67:0/1664002937 wait complete. 2024-05-30T17:04:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.167+0000 7f984ec00640 1 Processor -- start 2024-05-30T17:04:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.168+0000 7f984ec00640 1 -- start start 2024-05-30T17:04:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.168+0000 7f984ec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9848156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.168+0000 7f984ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98480b4fb0 0x7f98481f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98480728b0 0x7f984814de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984ec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98480b2db0 con 0x7f9830004800 2024-05-30T17:04:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f98480b2c30 con 0x7f98480b4fb0 2024-05-30T17:04:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f98480b39d0 con 0x7f98480728b0 2024-05-30T17:04:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9848156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9848156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54936/0 (socket says 172.21.15.67:54936) 2024-05-30T17:04:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984d800640 1 -- 172.21.15.67:0/120536805 learned_addr learned my addr 172.21.15.67:0/120536805 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984ce00640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98480b4fb0 0x7f98481f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984d800640 1 -- 172.21.15.67:0/120536805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98480728b0 msgr2=0x7f984814de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984e200640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98480728b0 0x7f984814de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984d800640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98480728b0 0x7f984814de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984d800640 1 -- 172.21.15.67:0/120536805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98480b4fb0 msgr2=0x7f98481f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984d800640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98480b4fb0 0x7f98481f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984d800640 1 -- 172.21.15.67:0/120536805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98480b46c0 con 0x7f9830004800 2024-05-30T17:04:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.169+0000 7f984e200640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98480728b0 0x7f984814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.170+0000 7f984d800640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9848156f10 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f983c0108c0 tx=0x7f983c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.170+0000 7f9845600640 1 -- 172.21.15.67:0/120536805 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c00f040 con 0x7f9830004800 2024-05-30T17:04:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.170+0000 7f9845600640 1 -- 172.21.15.67:0/120536805 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983c00eb00 con 0x7f9830004800 2024-05-30T17:04:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.170+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9848073840 con 0x7f9830004800 2024-05-30T17:04:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.170+0000 7f9845600640 1 -- 172.21.15.67:0/120536805 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c01c400 con 0x7f9830004800 2024-05-30T17:04:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.170+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98480b4c10 con 0x7f9830004800 2024-05-30T17:04:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.171+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98480689e0 con 0x7f9830004800 2024-05-30T17:04:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.172+0000 7f9845600640 1 -- 172.21.15.67:0/120536805 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f983c01c5a0 con 0x7f9830004800 2024-05-30T17:04:57.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.173+0000 7f9845600640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f982806e510 0x7f98280709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.173+0000 7f9845600640 1 -- 172.21.15.67:0/120536805 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f983c08aee0 con 0x7f9830004800 2024-05-30T17:04:57.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.173+0000 7f984ce00640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f982806e510 0x7f98280709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.174+0000 7f984ce00640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f982806e510 0x7f98280709d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9834033000 tx=0x7f9834008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:57.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.178+0000 7f9845600640 1 -- 172.21.15.67:0/120536805 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f983c018030 con 0x7f9830004800 2024-05-30T17:04:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.378+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f984806f7f0 con 0x7f982806e510 2024-05-30T17:04:57.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.397+0000 7f9845600640 1 -- 172.21.15.67:0/120536805 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252507 (secure 0 0 0) 0x7f984806f7f0 con 0x7f982806e510 2024-05-30T17:04:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:57.403 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:04:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f982806e510 msgr2=0x7f98280709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f982806e510 0x7f98280709d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9834033000 tx=0x7f9834008030 comp rx=0 tx=0).stop 2024-05-30T17:04:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 msgr2=0x7f9848156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9848156f10 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f983c0108c0 tx=0x7f983c010d80 comp rx=0 tx=0).stop 2024-05-30T17:04:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 shutdown_connections 2024-05-30T17:04:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f982806e510 0x7f98280709d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98480728b0 0x7f984814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98480b4fb0 0x7f98481f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 --2- 172.21.15.67:0/120536805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9830004800 0x7f9848156f10 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.403+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 >> 172.21.15.67:0/120536805 conn(0x7f98480b1fc0 msgr2=0x7f98480af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.404+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 shutdown_connections 2024-05-30T17:04:57.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.404+0000 7f984ec00640 1 -- 172.21.15.67:0/120536805 wait complete. 2024-05-30T17:04:57.424 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":764,"stamp":"2024-05-30T17:04:56.578595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1023712,"kb_used_data":14000,"kb_used_omap":12,"kb_used_meta":1009587,"kb_avail":753951008,"statfs":{"total":773094113280,"available":772045832192,"internally_reserved":0,"allocated":14336000,"data_stored":6961044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1033817648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.668918"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.072117+0000","last_change":"2024-05-30T17:04:52.998366+0000","last_active":"2024-05-30T17:04:54.072117+0000","last_peered":"2024-05-30T17:04:54.072117+0000","last_clean":"2024-05-30T17:04:54.072117+0000","last_became_active":"2024-05-30T17:04:52.997618+0000","last_became_peered":"2024-05-30T17:04:52.997618+0000","last_unstale":"2024-05-30T17:04:54.072117+0000","last_undegraded":"2024-05-30T17:04:54.072117+0000","last_fullsized":"2024-05-30T17:04:54.072117+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:57.883038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119766+0000","last_change":"2024-05-30T17:04:52.984295+0000","last_active":"2024-05-30T17:04:54.119766+0000","last_peered":"2024-05-30T17:04:54.119766+0000","last_clean":"2024-05-30T17:04:54.119766+0000","last_became_active":"2024-05-30T17:04:52.982524+0000","last_became_peered":"2024-05-30T17:04:52.982524+0000","last_unstale":"2024-05-30T17:04:54.119766+0000","last_undegraded":"2024-05-30T17:04:54.119766+0000","last_fullsized":"2024-05-30T17:04:54.119766+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:49.622879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123582+0000","last_change":"2024-05-30T17:04:52.984783+0000","last_active":"2024-05-30T17:04:54.123582+0000","last_peered":"2024-05-30T17:04:54.123582+0000","last_clean":"2024-05-30T17:04:54.123582+0000","last_became_active":"2024-05-30T17:04:52.984493+0000","last_became_peered":"2024-05-30T17:04:52.984493+0000","last_unstale":"2024-05-30T17:04:54.123582+0000","last_undegraded":"2024-05-30T17:04:54.123582+0000","last_fullsized":"2024-05-30T17:04:54.123582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:21.682021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071315+0000","last_change":"2024-05-30T17:04:52.997076+0000","last_active":"2024-05-30T17:04:54.071315+0000","last_peered":"2024-05-30T17:04:54.071315+0000","last_clean":"2024-05-30T17:04:54.071315+0000","last_became_active":"2024-05-30T17:04:52.996675+0000","last_became_peered":"2024-05-30T17:04:52.996675+0000","last_unstale":"2024-05-30T17:04:54.071315+0000","last_undegraded":"2024-05-30T17:04:54.071315+0000","last_fullsized":"2024-05-30T17:04:54.071315+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:49.622879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999332+0000","last_change":"2024-05-30T17:04:52.990350+0000","last_active":"2024-05-30T17:04:53.999332+0000","last_peered":"2024-05-30T17:04:53.999332+0000","last_clean":"2024-05-30T17:04:53.999332+0000","last_became_active":"2024-05-30T17:04:52.990018+0000","last_became_peered":"2024-05-30T17:04:52.990018+0000","last_unstale":"2024-05-30T17:04:53.999332+0000","last_undegraded":"2024-05-30T17:04:53.999332+0000","last_fullsized":"2024-05-30T17:04:53.999332+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:21.257549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961410+0000","last_change":"2024-05-30T17:04:52.992047+0000","last_active":"2024-05-30T17:04:53.961410+0000","last_peered":"2024-05-30T17:04:53.961410+0000","last_clean":"2024-05-30T17:04:53.961410+0000","last_became_active":"2024-05-30T17:04:52.991748+0000","last_became_peered":"2024-05-30T17:04:52.991748+0000","last_unstale":"2024-05-30T17:04:53.961410+0000","last_undegraded":"2024-05-30T17:04:53.961410+0000","last_fullsized":"2024-05-30T17:04:53.961410+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:06.346572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960723+0000","last_change":"2024-05-30T17:04:52.984245+0000","last_active":"2024-05-30T17:04:53.960723+0000","last_peered":"2024-05-30T17:04:53.960723+0000","last_clean":"2024-05-30T17:04:53.960723+0000","last_became_active":"2024-05-30T17:04:52.983960+0000","last_became_peered":"2024-05-30T17:04:52.983960+0000","last_unstale":"2024-05-30T17:04:53.960723+0000","last_undegraded":"2024-05-30T17:04:53.960723+0000","last_fullsized":"2024-05-30T17:04:53.960723+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:40:33.491032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.122274+0000","last_change":"2024-05-30T17:04:52.988608+0000","last_active":"2024-05-30T17:04:54.122274+0000","last_peered":"2024-05-30T17:04:54.122274+0000","last_clean":"2024-05-30T17:04:54.122274+0000","last_became_active":"2024-05-30T17:04:52.988277+0000","last_became_peered":"2024-05-30T17:04:52.988277+0000","last_unstale":"2024-05-30T17:04:54.122274+0000","last_undegraded":"2024-05-30T17:04:54.122274+0000","last_fullsized":"2024-05-30T17:04:54.122274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:34.512494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123990+0000","last_change":"2024-05-30T17:04:52.993014+0000","last_active":"2024-05-30T17:04:54.123990+0000","last_peered":"2024-05-30T17:04:54.123990+0000","last_clean":"2024-05-30T17:04:54.123990+0000","last_became_active":"2024-05-30T17:04:52.990543+0000","last_became_peered":"2024-05-30T17:04:52.990543+0000","last_unstale":"2024-05-30T17:04:54.123990+0000","last_undegraded":"2024-05-30T17:04:54.123990+0000","last_fullsized":"2024-05-30T17:04:54.123990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:27:15.135800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119758+0000","last_change":"2024-05-30T17:04:52.979749+0000","last_active":"2024-05-30T17:04:54.119758+0000","last_peered":"2024-05-30T17:04:54.119758+0000","last_clean":"2024-05-30T17:04:54.119758+0000","last_became_active":"2024-05-30T17:04:52.979390+0000","last_became_peered":"2024-05-30T17:04:52.979390+0000","last_unstale":"2024-05-30T17:04:54.119758+0000","last_undegraded":"2024-05-30T17:04:54.119758+0000","last_fullsized":"2024-05-30T17:04:54.119758+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119955+0000","last_change":"2024-05-30T17:04:52.983598+0000","last_active":"2024-05-30T17:04:54.119955+0000","last_peered":"2024-05-30T17:04:54.119955+0000","last_clean":"2024-05-30T17:04:54.119955+0000","last_became_active":"2024-05-30T17:04:52.983250+0000","last_became_peered":"2024-05-30T17:04:52.983250+0000","last_unstale":"2024-05-30T17:04:54.119955+0000","last_undegraded":"2024-05-30T17:04:54.119955+0000","last_fullsized":"2024-05-30T17:04:54.119955+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:20.889101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.122809+0000","last_change":"2024-05-30T17:04:52.993606+0000","last_active":"2024-05-30T17:04:54.122809+0000","last_peered":"2024-05-30T17:04:54.122809+0000","last_clean":"2024-05-30T17:04:54.122809+0000","last_became_active":"2024-05-30T17:04:52.993151+0000","last_became_peered":"2024-05-30T17:04:52.993151+0000","last_unstale":"2024-05-30T17:04:54.122809+0000","last_undegraded":"2024-05-30T17:04:54.122809+0000","last_fullsized":"2024-05-30T17:04:54.122809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:14.893435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071930+0000","last_change":"2024-05-30T17:04:52.998102+0000","last_active":"2024-05-30T17:04:54.071930+0000","last_peered":"2024-05-30T17:04:54.071930+0000","last_clean":"2024-05-30T17:04:54.071930+0000","last_became_active":"2024-05-30T17:04:52.997770+0000","last_became_peered":"2024-05-30T17:04:52.997770+0000","last_unstale":"2024-05-30T17:04:54.071930+0000","last_undegraded":"2024-05-30T17:04:54.071930+0000","last_fullsized":"2024-05-30T17:04:54.071930+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119872+0000","last_change":"2024-05-30T17:04:52.992617+0000","last_active":"2024-05-30T17:04:54.119872+0000","last_peered":"2024-05-30T17:04:54.119872+0000","last_clean":"2024-05-30T17:04:54.119872+0000","last_became_active":"2024-05-30T17:04:52.992221+0000","last_became_peered":"2024-05-30T17:04:52.992221+0000","last_unstale":"2024-05-30T17:04:54.119872+0000","last_undegraded":"2024-05-30T17:04:54.119872+0000","last_fullsized":"2024-05-30T17:04:54.119872+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.070601+0000","last_change":"2024-05-30T17:04:52.992819+0000","last_active":"2024-05-30T17:04:54.070601+0000","last_peered":"2024-05-30T17:04:54.070601+0000","last_clean":"2024-05-30T17:04:54.070601+0000","last_became_active":"2024-05-30T17:04:52.991693+0000","last_became_peered":"2024-05-30T17:04:52.991693+0000","last_unstale":"2024-05-30T17:04:54.070601+0000","last_undegraded":"2024-05-30T17:04:54.070601+0000","last_fullsized":"2024-05-30T17:04:54.070601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.952890+0000","last_change":"2024-05-30T17:04:52.991328+0000","last_active":"2024-05-30T17:04:53.952890+0000","last_peered":"2024-05-30T17:04:53.952890+0000","last_clean":"2024-05-30T17:04:53.952890+0000","last_became_active":"2024-05-30T17:04:52.990334+0000","last_became_peered":"2024-05-30T17:04:52.990334+0000","last_unstale":"2024-05-30T17:04:53.952890+0000","last_undegraded":"2024-05-30T17:04:53.952890+0000","last_fullsized":"2024-05-30T17:04:53.952890+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:23.528030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.962307+0000","last_change":"2024-05-30T17:04:52.967087+0000","last_active":"2024-05-30T17:04:53.962307+0000","last_peered":"2024-05-30T17:04:53.962307+0000","last_clean":"2024-05-30T17:04:53.962307+0000","last_became_active":"2024-05-30T17:04:52.966026+0000","last_became_peered":"2024-05-30T17:04:52.966026+0000","last_unstale":"2024-05-30T17:04:53.962307+0000","last_undegraded":"2024-05-30T17:04:53.962307+0000","last_fullsized":"2024-05-30T17:04:53.962307+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:12.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953508+0000","last_change":"2024-05-30T17:04:52.986386+0000","last_active":"2024-05-30T17:04:53.953508+0000","last_peered":"2024-05-30T17:04:53.953508+0000","last_clean":"2024-05-30T17:04:53.953508+0000","last_became_active":"2024-05-30T17:04:52.985602+0000","last_became_peered":"2024-05-30T17:04:52.985602+0000","last_unstale":"2024-05-30T17:04:53.953508+0000","last_undegraded":"2024-05-30T17:04:53.953508+0000","last_fullsized":"2024-05-30T17:04:53.953508+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:31.682869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120117+0000","last_change":"2024-05-30T17:04:52.963412+0000","last_active":"2024-05-30T17:04:54.120117+0000","last_peered":"2024-05-30T17:04:54.120117+0000","last_clean":"2024-05-30T17:04:54.120117+0000","last_became_active":"2024-05-30T17:04:52.962582+0000","last_became_peered":"2024-05-30T17:04:52.962582+0000","last_unstale":"2024-05-30T17:04:54.120117+0000","last_undegraded":"2024-05-30T17:04:54.120117+0000","last_fullsized":"2024-05-30T17:04:54.120117+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:52.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953269+0000","last_change":"2024-05-30T17:04:52.987028+0000","last_active":"2024-05-30T17:04:53.953269+0000","last_peered":"2024-05-30T17:04:53.953269+0000","last_clean":"2024-05-30T17:04:53.953269+0000","last_became_active":"2024-05-30T17:04:52.986701+0000","last_became_peered":"2024-05-30T17:04:52.986701+0000","last_unstale":"2024-05-30T17:04:53.953269+0000","last_undegraded":"2024-05-30T17:04:53.953269+0000","last_fullsized":"2024-05-30T17:04:53.953269+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:50:37.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999097+0000","last_change":"2024-05-30T17:04:52.984763+0000","last_active":"2024-05-30T17:04:53.999097+0000","last_peered":"2024-05-30T17:04:53.999097+0000","last_clean":"2024-05-30T17:04:53.999097+0000","last_became_active":"2024-05-30T17:04:52.984462+0000","last_became_peered":"2024-05-30T17:04:52.984462+0000","last_unstale":"2024-05-30T17:04:53.999097+0000","last_undegraded":"2024-05-30T17:04:53.999097+0000","last_fullsized":"2024-05-30T17:04:53.999097+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:09.221732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119653+0000","last_change":"2024-05-30T17:04:52.980574+0000","last_active":"2024-05-30T17:04:54.119653+0000","last_peered":"2024-05-30T17:04:54.119653+0000","last_clean":"2024-05-30T17:04:54.119653+0000","last_became_active":"2024-05-30T17:04:52.978968+0000","last_became_peered":"2024-05-30T17:04:52.978968+0000","last_unstale":"2024-05-30T17:04:54.119653+0000","last_undegraded":"2024-05-30T17:04:54.119653+0000","last_fullsized":"2024-05-30T17:04:54.119653+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:30:51.419747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123031+0000","last_change":"2024-05-30T17:04:52.992275+0000","last_active":"2024-05-30T17:04:54.123031+0000","last_peered":"2024-05-30T17:04:54.123031+0000","last_clean":"2024-05-30T17:04:54.123031+0000","last_became_active":"2024-05-30T17:04:52.991966+0000","last_became_peered":"2024-05-30T17:04:52.991966+0000","last_unstale":"2024-05-30T17:04:54.123031+0000","last_undegraded":"2024-05-30T17:04:54.123031+0000","last_fullsized":"2024-05-30T17:04:54.123031+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:32.227484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119833+0000","last_change":"2024-05-30T17:04:52.965882+0000","last_active":"2024-05-30T17:04:54.119833+0000","last_peered":"2024-05-30T17:04:54.119833+0000","last_clean":"2024-05-30T17:04:54.119833+0000","last_became_active":"2024-05-30T17:04:52.965116+0000","last_became_peered":"2024-05-30T17:04:52.965116+0000","last_unstale":"2024-05-30T17:04:54.119833+0000","last_undegraded":"2024-05-30T17:04:54.119833+0000","last_fullsized":"2024-05-30T17:04:54.119833+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:35:54.995662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119957+0000","last_change":"2024-05-30T17:04:52.983400+0000","last_active":"2024-05-30T17:04:54.119957+0000","last_peered":"2024-05-30T17:04:54.119957+0000","last_clean":"2024-05-30T17:04:54.119957+0000","last_became_active":"2024-05-30T17:04:52.981867+0000","last_became_peered":"2024-05-30T17:04:52.981867+0000","last_unstale":"2024-05-30T17:04:54.119957+0000","last_undegraded":"2024-05-30T17:04:54.119957+0000","last_fullsized":"2024-05-30T17:04:54.119957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953764+0000","last_change":"2024-05-30T17:04:52.989413+0000","last_active":"2024-05-30T17:04:53.953764+0000","last_peered":"2024-05-30T17:04:53.953764+0000","last_clean":"2024-05-30T17:04:53.953764+0000","last_became_active":"2024-05-30T17:04:52.989141+0000","last_became_peered":"2024-05-30T17:04:52.989141+0000","last_unstale":"2024-05-30T17:04:53.953764+0000","last_undegraded":"2024-05-30T17:04:53.953764+0000","last_fullsized":"2024-05-30T17:04:53.953764+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:15.844444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071623+0000","last_change":"2024-05-30T17:04:52.992667+0000","last_active":"2024-05-30T17:04:54.071623+0000","last_peered":"2024-05-30T17:04:54.071623+0000","last_clean":"2024-05-30T17:04:54.071623+0000","last_became_active":"2024-05-30T17:04:52.988618+0000","last_became_peered":"2024-05-30T17:04:52.988618+0000","last_unstale":"2024-05-30T17:04:54.071623+0000","last_undegraded":"2024-05-30T17:04:54.071623+0000","last_fullsized":"2024-05-30T17:04:54.071623+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.958939+0000","last_change":"2024-05-30T17:04:52.977678+0000","last_active":"2024-05-30T17:04:53.958939+0000","last_peered":"2024-05-30T17:04:53.958939+0000","last_clean":"2024-05-30T17:04:53.958939+0000","last_became_active":"2024-05-30T17:04:52.977378+0000","last_became_peered":"2024-05-30T17:04:52.977378+0000","last_unstale":"2024-05-30T17:04:53.958939+0000","last_undegraded":"2024-05-30T17:04:53.958939+0000","last_fullsized":"2024-05-30T17:04:53.958939+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.220555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071535+0000","last_change":"2024-05-30T17:04:52.998322+0000","last_active":"2024-05-30T17:04:54.071535+0000","last_peered":"2024-05-30T17:04:54.071535+0000","last_clean":"2024-05-30T17:04:54.071535+0000","last_became_active":"2024-05-30T17:04:52.997323+0000","last_became_peered":"2024-05-30T17:04:52.997323+0000","last_unstale":"2024-05-30T17:04:54.071535+0000","last_undegraded":"2024-05-30T17:04:54.071535+0000","last_fullsized":"2024-05-30T17:04:54.071535+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071181+0000","last_change":"2024-05-30T17:04:52.995308+0000","last_active":"2024-05-30T17:04:54.071181+0000","last_peered":"2024-05-30T17:04:54.071181+0000","last_clean":"2024-05-30T17:04:54.071181+0000","last_became_active":"2024-05-30T17:04:52.992733+0000","last_became_peered":"2024-05-30T17:04:52.992733+0000","last_unstale":"2024-05-30T17:04:54.071181+0000","last_undegraded":"2024-05-30T17:04:54.071181+0000","last_fullsized":"2024-05-30T17:04:54.071181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998865+0000","last_change":"2024-05-30T17:04:52.993970+0000","last_active":"2024-05-30T17:04:53.998865+0000","last_peered":"2024-05-30T17:04:53.998865+0000","last_clean":"2024-05-30T17:04:53.998865+0000","last_became_active":"2024-05-30T17:04:52.993214+0000","last_became_peered":"2024-05-30T17:04:52.993214+0000","last_unstale":"2024-05-30T17:04:53.998865+0000","last_undegraded":"2024-05-30T17:04:53.998865+0000","last_fullsized":"2024-05-30T17:04:53.998865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:21:06.134079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960921+0000","last_change":"2024-05-30T17:04:52.980018+0000","last_active":"2024-05-30T17:04:53.960921+0000","last_peered":"2024-05-30T17:04:53.960921+0000","last_clean":"2024-05-30T17:04:53.960921+0000","last_became_active":"2024-05-30T17:04:52.979697+0000","last_became_peered":"2024-05-30T17:04:52.979697+0000","last_unstale":"2024-05-30T17:04:53.960921+0000","last_undegraded":"2024-05-30T17:04:53.960921+0000","last_fullsized":"2024-05-30T17:04:53.960921+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:30:44.014138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953739+0000","last_change":"2024-05-30T17:04:52.978441+0000","last_active":"2024-05-30T17:04:53.953739+0000","last_peered":"2024-05-30T17:04:53.953739+0000","last_clean":"2024-05-30T17:04:53.953739+0000","last_became_active":"2024-05-30T17:04:52.978065+0000","last_became_peered":"2024-05-30T17:04:52.978065+0000","last_unstale":"2024-05-30T17:04:53.953739+0000","last_undegraded":"2024-05-30T17:04:53.953739+0000","last_fullsized":"2024-05-30T17:04:53.953739+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:21:00.756549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999395+0000","last_change":"2024-05-30T17:04:51.944167+0000","last_active":"2024-05-30T17:04:53.999395+0000","last_peered":"2024-05-30T17:04:53.999395+0000","last_clean":"2024-05-30T17:04:53.999395+0000","last_became_active":"2024-05-30T17:04:51.943796+0000","last_became_peered":"2024-05-30T17:04:51.943796+0000","last_unstale":"2024-05-30T17:04:53.999395+0000","last_undegraded":"2024-05-30T17:04:53.999395+0000","last_fullsized":"2024-05-30T17:04:53.999395+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:50.016832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.124052+0000","last_change":"2024-05-30T17:04:52.998976+0000","last_active":"2024-05-30T17:04:54.124052+0000","last_peered":"2024-05-30T17:04:54.124052+0000","last_clean":"2024-05-30T17:04:54.124052+0000","last_became_active":"2024-05-30T17:04:52.998673+0000","last_became_peered":"2024-05-30T17:04:52.998673+0000","last_unstale":"2024-05-30T17:04:54.124052+0000","last_undegraded":"2024-05-30T17:04:54.124052+0000","last_fullsized":"2024-05-30T17:04:54.124052+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:58.331966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.959283+0000","last_change":"2024-05-30T17:04:51.939570+0000","last_active":"2024-05-30T17:04:53.959283+0000","last_peered":"2024-05-30T17:04:53.959283+0000","last_clean":"2024-05-30T17:04:53.959283+0000","last_became_active":"2024-05-30T17:04:51.937127+0000","last_became_peered":"2024-05-30T17:04:51.937127+0000","last_unstale":"2024-05-30T17:04:53.959283+0000","last_undegraded":"2024-05-30T17:04:53.959283+0000","last_fullsized":"2024-05-30T17:04:53.959283+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:18.245355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999390+0000","last_change":"2024-05-30T17:04:52.983816+0000","last_active":"2024-05-30T17:04:53.999390+0000","last_peered":"2024-05-30T17:04:53.999390+0000","last_clean":"2024-05-30T17:04:53.999390+0000","last_became_active":"2024-05-30T17:04:52.981316+0000","last_became_peered":"2024-05-30T17:04:52.981316+0000","last_unstale":"2024-05-30T17:04:53.999390+0000","last_undegraded":"2024-05-30T17:04:53.999390+0000","last_fullsized":"2024-05-30T17:04:53.999390+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:13.610126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120082+0000","last_change":"2024-05-30T17:04:51.938960+0000","last_active":"2024-05-30T17:04:54.120082+0000","last_peered":"2024-05-30T17:04:54.120082+0000","last_clean":"2024-05-30T17:04:54.120082+0000","last_became_active":"2024-05-30T17:04:51.938644+0000","last_became_peered":"2024-05-30T17:04:51.938644+0000","last_unstale":"2024-05-30T17:04:54.120082+0000","last_undegraded":"2024-05-30T17:04:54.120082+0000","last_fullsized":"2024-05-30T17:04:54.120082+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:12.595138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999528+0000","last_change":"2024-05-30T17:04:52.988274+0000","last_active":"2024-05-30T17:04:53.999528+0000","last_peered":"2024-05-30T17:04:53.999528+0000","last_clean":"2024-05-30T17:04:53.999528+0000","last_became_active":"2024-05-30T17:04:52.987961+0000","last_became_peered":"2024-05-30T17:04:52.987961+0000","last_unstale":"2024-05-30T17:04:53.999528+0000","last_undegraded":"2024-05-30T17:04:53.999528+0000","last_fullsized":"2024-05-30T17:04:53.999528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:21.613683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120166+0000","last_change":"2024-05-30T17:04:51.930251+0000","last_active":"2024-05-30T17:04:54.120166+0000","last_peered":"2024-05-30T17:04:54.120166+0000","last_clean":"2024-05-30T17:04:54.120166+0000","last_became_active":"2024-05-30T17:04:51.929298+0000","last_became_peered":"2024-05-30T17:04:51.929298+0000","last_unstale":"2024-05-30T17:04:54.120166+0000","last_undegraded":"2024-05-30T17:04:54.120166+0000","last_fullsized":"2024-05-30T17:04:54.120166+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:35.687280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953143+0000","last_change":"2024-05-30T17:04:52.986145+0000","last_active":"2024-05-30T17:04:53.953143+0000","last_peered":"2024-05-30T17:04:53.953143+0000","last_clean":"2024-05-30T17:04:53.953143+0000","last_became_active":"2024-05-30T17:04:52.985811+0000","last_became_peered":"2024-05-30T17:04:52.985811+0000","last_unstale":"2024-05-30T17:04:53.953143+0000","last_undegraded":"2024-05-30T17:04:53.953143+0000","last_fullsized":"2024-05-30T17:04:53.953143+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:55.493534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119735+0000","last_change":"2024-05-30T17:04:51.943821+0000","last_active":"2024-05-30T17:04:54.119735+0000","last_peered":"2024-05-30T17:04:54.119735+0000","last_clean":"2024-05-30T17:04:54.119735+0000","last_became_active":"2024-05-30T17:04:51.942597+0000","last_became_peered":"2024-05-30T17:04:51.942597+0000","last_unstale":"2024-05-30T17:04:54.119735+0000","last_undegraded":"2024-05-30T17:04:54.119735+0000","last_fullsized":"2024-05-30T17:04:54.119735+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:47.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961034+0000","last_change":"2024-05-30T17:04:52.967603+0000","last_active":"2024-05-30T17:04:53.961034+0000","last_peered":"2024-05-30T17:04:53.961034+0000","last_clean":"2024-05-30T17:04:53.961034+0000","last_became_active":"2024-05-30T17:04:52.967033+0000","last_became_peered":"2024-05-30T17:04:52.967033+0000","last_unstale":"2024-05-30T17:04:53.961034+0000","last_undegraded":"2024-05-30T17:04:53.961034+0000","last_fullsized":"2024-05-30T17:04:53.961034+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:18.064738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.952911+0000","last_change":"2024-05-30T17:04:51.942553+0000","last_active":"2024-05-30T17:04:53.952911+0000","last_peered":"2024-05-30T17:04:53.952911+0000","last_clean":"2024-05-30T17:04:53.952911+0000","last_became_active":"2024-05-30T17:04:51.941974+0000","last_became_peered":"2024-05-30T17:04:51.941974+0000","last_unstale":"2024-05-30T17:04:53.952911+0000","last_undegraded":"2024-05-30T17:04:53.952911+0000","last_fullsized":"2024-05-30T17:04:53.952911+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:40.877465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123264+0000","last_change":"2024-05-30T17:04:52.988985+0000","last_active":"2024-05-30T17:04:54.123264+0000","last_peered":"2024-05-30T17:04:54.123264+0000","last_clean":"2024-05-30T17:04:54.123264+0000","last_became_active":"2024-05-30T17:04:52.988645+0000","last_became_peered":"2024-05-30T17:04:52.988645+0000","last_unstale":"2024-05-30T17:04:54.123264+0000","last_undegraded":"2024-05-30T17:04:54.123264+0000","last_fullsized":"2024-05-30T17:04:54.123264+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:40.533224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960536+0000","last_change":"2024-05-30T17:04:51.951513+0000","last_active":"2024-05-30T17:04:53.960536+0000","last_peered":"2024-05-30T17:04:53.960536+0000","last_clean":"2024-05-30T17:04:53.960536+0000","last_became_active":"2024-05-30T17:04:51.951223+0000","last_became_peered":"2024-05-30T17:04:51.951223+0000","last_unstale":"2024-05-30T17:04:53.960536+0000","last_undegraded":"2024-05-30T17:04:53.960536+0000","last_fullsized":"2024-05-30T17:04:53.960536+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:13.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119495+0000","last_change":"2024-05-30T17:04:52.986452+0000","last_active":"2024-05-30T17:04:54.119495+0000","last_peered":"2024-05-30T17:04:54.119495+0000","last_clean":"2024-05-30T17:04:54.119495+0000","last_became_active":"2024-05-30T17:04:52.986164+0000","last_became_peered":"2024-05-30T17:04:52.986164+0000","last_unstale":"2024-05-30T17:04:54.119495+0000","last_undegraded":"2024-05-30T17:04:54.119495+0000","last_fullsized":"2024-05-30T17:04:54.119495+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119766+0000","last_change":"2024-05-30T17:04:51.955992+0000","last_active":"2024-05-30T17:04:54.119766+0000","last_peered":"2024-05-30T17:04:54.119766+0000","last_clean":"2024-05-30T17:04:54.119766+0000","last_became_active":"2024-05-30T17:04:51.955666+0000","last_became_peered":"2024-05-30T17:04:51.955666+0000","last_unstale":"2024-05-30T17:04:54.119766+0000","last_undegraded":"2024-05-30T17:04:54.119766+0000","last_fullsized":"2024-05-30T17:04:54.119766+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:03.343211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071910+0000","last_change":"2024-05-30T17:04:52.984705+0000","last_active":"2024-05-30T17:04:54.071910+0000","last_peered":"2024-05-30T17:04:54.071910+0000","last_clean":"2024-05-30T17:04:54.071910+0000","last_became_active":"2024-05-30T17:04:52.984409+0000","last_became_peered":"2024-05-30T17:04:52.984409+0000","last_unstale":"2024-05-30T17:04:54.071910+0000","last_undegraded":"2024-05-30T17:04:54.071910+0000","last_fullsized":"2024-05-30T17:04:54.071910+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960019+0000","last_change":"2024-05-30T17:04:51.952573+0000","last_active":"2024-05-30T17:04:53.960019+0000","last_peered":"2024-05-30T17:04:53.960019+0000","last_clean":"2024-05-30T17:04:53.960019+0000","last_became_active":"2024-05-30T17:04:51.948281+0000","last_became_peered":"2024-05-30T17:04:51.948281+0000","last_unstale":"2024-05-30T17:04:53.960019+0000","last_undegraded":"2024-05-30T17:04:53.960019+0000","last_fullsized":"2024-05-30T17:04:53.960019+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:26.183919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.959771+0000","last_change":"2024-05-30T17:04:52.986184+0000","last_active":"2024-05-30T17:04:53.959771+0000","last_peered":"2024-05-30T17:04:53.959771+0000","last_clean":"2024-05-30T17:04:53.959771+0000","last_became_active":"2024-05-30T17:04:52.985889+0000","last_became_peered":"2024-05-30T17:04:52.985889+0000","last_unstale":"2024-05-30T17:04:53.959771+0000","last_undegraded":"2024-05-30T17:04:53.959771+0000","last_fullsized":"2024-05-30T17:04:53.959771+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:27.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953802+0000","last_change":"2024-05-30T17:04:51.954829+0000","last_active":"2024-05-30T17:04:53.953802+0000","last_peered":"2024-05-30T17:04:53.953802+0000","last_clean":"2024-05-30T17:04:53.953802+0000","last_became_active":"2024-05-30T17:04:51.954434+0000","last_became_peered":"2024-05-30T17:04:51.954434+0000","last_unstale":"2024-05-30T17:04:53.953802+0000","last_undegraded":"2024-05-30T17:04:53.953802+0000","last_fullsized":"2024-05-30T17:04:53.953802+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:23.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999639+0000","last_change":"2024-05-30T17:04:52.989346+0000","last_active":"2024-05-30T17:04:53.999639+0000","last_peered":"2024-05-30T17:04:53.999639+0000","last_clean":"2024-05-30T17:04:53.999639+0000","last_became_active":"2024-05-30T17:04:52.989051+0000","last_became_peered":"2024-05-30T17:04:52.989051+0000","last_unstale":"2024-05-30T17:04:53.999639+0000","last_undegraded":"2024-05-30T17:04:53.999639+0000","last_fullsized":"2024-05-30T17:04:53.999639+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:36.128167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"131'3","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.958419+0000","last_change":"2024-05-30T17:04:51.938158+0000","last_active":"2024-05-30T17:04:53.958419+0000","last_peered":"2024-05-30T17:04:53.958419+0000","last_clean":"2024-05-30T17:04:53.958419+0000","last_became_active":"2024-05-30T17:04:51.937857+0000","last_became_peered":"2024-05-30T17:04:51.937857+0000","last_unstale":"2024-05-30T17:04:53.958419+0000","last_undegraded":"2024-05-30T17:04:53.958419+0000","last_fullsized":"2024-05-30T17:04:53.958419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:25.458335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999409+0000","last_change":"2024-05-30T17:04:52.995423+0000","last_active":"2024-05-30T17:04:53.999409+0000","last_peered":"2024-05-30T17:04:53.999409+0000","last_clean":"2024-05-30T17:04:53.999409+0000","last_became_active":"2024-05-30T17:04:52.995128+0000","last_became_peered":"2024-05-30T17:04:52.995128+0000","last_unstale":"2024-05-30T17:04:53.999409+0000","last_undegraded":"2024-05-30T17:04:53.999409+0000","last_fullsized":"2024-05-30T17:04:53.999409+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:47.501828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"131'6","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123102+0000","last_change":"2024-05-30T17:04:51.953550+0000","last_active":"2024-05-30T17:04:54.123102+0000","last_peered":"2024-05-30T17:04:54.123102+0000","last_clean":"2024-05-30T17:04:54.123102+0000","last_became_active":"2024-05-30T17:04:51.953276+0000","last_became_peered":"2024-05-30T17:04:51.953276+0000","last_unstale":"2024-05-30T17:04:54.123102+0000","last_undegraded":"2024-05-30T17:04:54.123102+0000","last_fullsized":"2024-05-30T17:04:54.123102+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:07:44.468185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071949+0000","last_change":"2024-05-30T17:04:52.988845+0000","last_active":"2024-05-30T17:04:54.071949+0000","last_peered":"2024-05-30T17:04:54.071949+0000","last_clean":"2024-05-30T17:04:54.071949+0000","last_became_active":"2024-05-30T17:04:52.986757+0000","last_became_peered":"2024-05-30T17:04:52.986757+0000","last_unstale":"2024-05-30T17:04:54.071949+0000","last_undegraded":"2024-05-30T17:04:54.071949+0000","last_fullsized":"2024-05-30T17:04:54.071949+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071262+0000","last_change":"2024-05-30T17:04:51.944552+0000","last_active":"2024-05-30T17:04:54.071262+0000","last_peered":"2024-05-30T17:04:54.071262+0000","last_clean":"2024-05-30T17:04:54.071262+0000","last_became_active":"2024-05-30T17:04:51.944167+0000","last_became_peered":"2024-05-30T17:04:51.944167+0000","last_unstale":"2024-05-30T17:04:54.071262+0000","last_undegraded":"2024-05-30T17:04:54.071262+0000","last_fullsized":"2024-05-30T17:04:54.071262+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:17:06.906161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119484+0000","last_change":"2024-05-30T17:04:52.991186+0000","last_active":"2024-05-30T17:04:54.119484+0000","last_peered":"2024-05-30T17:04:54.119484+0000","last_clean":"2024-05-30T17:04:54.119484+0000","last_became_active":"2024-05-30T17:04:52.990049+0000","last_became_peered":"2024-05-30T17:04:52.990049+0000","last_unstale":"2024-05-30T17:04:54.119484+0000","last_undegraded":"2024-05-30T17:04:54.119484+0000","last_fullsized":"2024-05-30T17:04:54.119484+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:28:23.896291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119598+0000","last_change":"2024-05-30T17:04:51.947216+0000","last_active":"2024-05-30T17:04:54.119598+0000","last_peered":"2024-05-30T17:04:54.119598+0000","last_clean":"2024-05-30T17:04:54.119598+0000","last_became_active":"2024-05-30T17:04:51.946868+0000","last_became_peered":"2024-05-30T17:04:51.946868+0000","last_unstale":"2024-05-30T17:04:54.119598+0000","last_undegraded":"2024-05-30T17:04:54.119598+0000","last_fullsized":"2024-05-30T17:04:54.119598+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:42:48.589336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.122520+0000","last_change":"2024-05-30T17:04:52.986670+0000","last_active":"2024-05-30T17:04:54.122520+0000","last_peered":"2024-05-30T17:04:54.122520+0000","last_clean":"2024-05-30T17:04:54.122520+0000","last_became_active":"2024-05-30T17:04:52.986293+0000","last_became_peered":"2024-05-30T17:04:52.986293+0000","last_unstale":"2024-05-30T17:04:54.122520+0000","last_undegraded":"2024-05-30T17:04:54.122520+0000","last_fullsized":"2024-05-30T17:04:54.122520+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:28.097184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.122953+0000","last_change":"2024-05-30T17:04:51.955924+0000","last_active":"2024-05-30T17:04:54.122953+0000","last_peered":"2024-05-30T17:04:54.122953+0000","last_clean":"2024-05-30T17:04:54.122953+0000","last_became_active":"2024-05-30T17:04:51.955542+0000","last_became_peered":"2024-05-30T17:04:51.955542+0000","last_unstale":"2024-05-30T17:04:54.122953+0000","last_undegraded":"2024-05-30T17:04:54.122953+0000","last_fullsized":"2024-05-30T17:04:54.122953+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:38.543340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123820+0000","last_change":"2024-05-30T17:04:53.001644+0000","last_active":"2024-05-30T17:04:54.123820+0000","last_peered":"2024-05-30T17:04:54.123820+0000","last_clean":"2024-05-30T17:04:54.123820+0000","last_became_active":"2024-05-30T17:04:52.999924+0000","last_became_peered":"2024-05-30T17:04:52.999924+0000","last_unstale":"2024-05-30T17:04:54.123820+0000","last_undegraded":"2024-05-30T17:04:54.123820+0000","last_fullsized":"2024-05-30T17:04:54.123820+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:17:36.947839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960175+0000","last_change":"2024-05-30T17:04:51.948540+0000","last_active":"2024-05-30T17:04:53.960175+0000","last_peered":"2024-05-30T17:04:53.960175+0000","last_clean":"2024-05-30T17:04:53.960175+0000","last_became_active":"2024-05-30T17:04:51.947928+0000","last_became_peered":"2024-05-30T17:04:51.947928+0000","last_unstale":"2024-05-30T17:04:53.960175+0000","last_undegraded":"2024-05-30T17:04:53.960175+0000","last_fullsized":"2024-05-30T17:04:53.960175+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:10:19.343810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.959882+0000","last_change":"2024-05-30T17:04:52.976220+0000","last_active":"2024-05-30T17:04:53.959882+0000","last_peered":"2024-05-30T17:04:53.959882+0000","last_clean":"2024-05-30T17:04:53.959882+0000","last_became_active":"2024-05-30T17:04:52.974987+0000","last_became_peered":"2024-05-30T17:04:52.974987+0000","last_unstale":"2024-05-30T17:04:53.959882+0000","last_undegraded":"2024-05-30T17:04:53.959882+0000","last_fullsized":"2024-05-30T17:04:53.959882+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:15.443712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120349+0000","last_change":"2024-05-30T17:04:51.950294+0000","last_active":"2024-05-30T17:04:54.120349+0000","last_peered":"2024-05-30T17:04:54.120349+0000","last_clean":"2024-05-30T17:04:54.120349+0000","last_became_active":"2024-05-30T17:04:51.949704+0000","last_became_peered":"2024-05-30T17:04:51.949704+0000","last_unstale":"2024-05-30T17:04:54.120349+0000","last_undegraded":"2024-05-30T17:04:54.120349+0000","last_fullsized":"2024-05-30T17:04:54.120349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:46.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953390+0000","last_change":"2024-05-30T17:04:52.989374+0000","last_active":"2024-05-30T17:04:53.953390+0000","last_peered":"2024-05-30T17:04:53.953390+0000","last_clean":"2024-05-30T17:04:53.953390+0000","last_became_active":"2024-05-30T17:04:52.989083+0000","last_became_peered":"2024-05-30T17:04:52.989083+0000","last_unstale":"2024-05-30T17:04:53.953390+0000","last_undegraded":"2024-05-30T17:04:53.953390+0000","last_fullsized":"2024-05-30T17:04:53.953390+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:35.524809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961842+0000","last_change":"2024-05-30T17:04:51.942441+0000","last_active":"2024-05-30T17:04:53.961842+0000","last_peered":"2024-05-30T17:04:53.961842+0000","last_clean":"2024-05-30T17:04:53.961842+0000","last_became_active":"2024-05-30T17:04:51.942121+0000","last_became_peered":"2024-05-30T17:04:51.942121+0000","last_unstale":"2024-05-30T17:04:53.961842+0000","last_undegraded":"2024-05-30T17:04:53.961842+0000","last_fullsized":"2024-05-30T17:04:53.961842+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:27.082196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120288+0000","last_change":"2024-05-30T17:04:52.994569+0000","last_active":"2024-05-30T17:04:54.120288+0000","last_peered":"2024-05-30T17:04:54.120288+0000","last_clean":"2024-05-30T17:04:54.120288+0000","last_became_active":"2024-05-30T17:04:52.994290+0000","last_became_peered":"2024-05-30T17:04:52.994290+0000","last_unstale":"2024-05-30T17:04:54.120288+0000","last_undegraded":"2024-05-30T17:04:54.120288+0000","last_fullsized":"2024-05-30T17:04:54.120288+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.958530+0000","last_change":"2024-05-30T17:04:51.945428+0000","last_active":"2024-05-30T17:04:53.958530+0000","last_peered":"2024-05-30T17:04:53.958530+0000","last_clean":"2024-05-30T17:04:53.958530+0000","last_became_active":"2024-05-30T17:04:51.945142+0000","last_became_peered":"2024-05-30T17:04:51.945142+0000","last_unstale":"2024-05-30T17:04:53.958530+0000","last_undegraded":"2024-05-30T17:04:53.958530+0000","last_fullsized":"2024-05-30T17:04:53.958530+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:25:46.375765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071194+0000","last_change":"2024-05-30T17:04:52.993514+0000","last_active":"2024-05-30T17:04:54.071194+0000","last_peered":"2024-05-30T17:04:54.071194+0000","last_clean":"2024-05-30T17:04:54.071194+0000","last_became_active":"2024-05-30T17:04:52.988660+0000","last_became_peered":"2024-05-30T17:04:52.988660+0000","last_unstale":"2024-05-30T17:04:54.071194+0000","last_undegraded":"2024-05-30T17:04:54.071194+0000","last_fullsized":"2024-05-30T17:04:54.071194+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.958417+0000","last_change":"2024-05-30T17:04:51.943410+0000","last_active":"2024-05-30T17:04:53.958417+0000","last_peered":"2024-05-30T17:04:53.958417+0000","last_clean":"2024-05-30T17:04:53.958417+0000","last_became_active":"2024-05-30T17:04:51.943085+0000","last_became_peered":"2024-05-30T17:04:51.943085+0000","last_unstale":"2024-05-30T17:04:53.958417+0000","last_undegraded":"2024-05-30T17:04:53.958417+0000","last_fullsized":"2024-05-30T17:04:53.958417+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:18.313092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961619+0000","last_change":"2024-05-30T17:04:52.992939+0000","last_active":"2024-05-30T17:04:53.961619+0000","last_peered":"2024-05-30T17:04:53.961619+0000","last_clean":"2024-05-30T17:04:53.961619+0000","last_became_active":"2024-05-30T17:04:52.989409+0000","last_became_peered":"2024-05-30T17:04:52.989409+0000","last_unstale":"2024-05-30T17:04:53.961619+0000","last_undegraded":"2024-05-30T17:04:53.961619+0000","last_fullsized":"2024-05-30T17:04:53.961619+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:41.892452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953068+0000","last_change":"2024-05-30T17:04:51.948194+0000","last_active":"2024-05-30T17:04:53.953068+0000","last_peered":"2024-05-30T17:04:53.953068+0000","last_clean":"2024-05-30T17:04:53.953068+0000","last_became_active":"2024-05-30T17:04:51.947919+0000","last_became_peered":"2024-05-30T17:04:51.947919+0000","last_unstale":"2024-05-30T17:04:53.953068+0000","last_undegraded":"2024-05-30T17:04:53.953068+0000","last_fullsized":"2024-05-30T17:04:53.953068+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:58.823357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119648+0000","last_change":"2024-05-30T17:04:52.981350+0000","last_active":"2024-05-30T17:04:54.119648+0000","last_peered":"2024-05-30T17:04:54.119648+0000","last_clean":"2024-05-30T17:04:54.119648+0000","last_became_active":"2024-05-30T17:04:52.978531+0000","last_became_peered":"2024-05-30T17:04:52.978531+0000","last_unstale":"2024-05-30T17:04:54.119648+0000","last_undegraded":"2024-05-30T17:04:54.119648+0000","last_fullsized":"2024-05-30T17:04:54.119648+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.952968+0000","last_change":"2024-05-30T17:04:51.950726+0000","last_active":"2024-05-30T17:04:53.952968+0000","last_peered":"2024-05-30T17:04:53.952968+0000","last_clean":"2024-05-30T17:04:53.952968+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:04:53.952968+0000","last_undegraded":"2024-05-30T17:04:53.952968+0000","last_fullsized":"2024-05-30T17:04:53.952968+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:17.049750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998962+0000","last_change":"2024-05-30T17:04:52.989896+0000","last_active":"2024-05-30T17:04:53.998962+0000","last_peered":"2024-05-30T17:04:53.998962+0000","last_clean":"2024-05-30T17:04:53.998962+0000","last_became_active":"2024-05-30T17:04:52.989566+0000","last_became_peered":"2024-05-30T17:04:52.989566+0000","last_unstale":"2024-05-30T17:04:53.998962+0000","last_undegraded":"2024-05-30T17:04:53.998962+0000","last_fullsized":"2024-05-30T17:04:53.998962+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:44.437259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119958+0000","last_change":"2024-05-30T17:04:51.951054+0000","last_active":"2024-05-30T17:04:54.119958+0000","last_peered":"2024-05-30T17:04:54.119958+0000","last_clean":"2024-05-30T17:04:54.119958+0000","last_became_active":"2024-05-30T17:04:51.950364+0000","last_became_peered":"2024-05-30T17:04:51.950364+0000","last_unstale":"2024-05-30T17:04:54.119958+0000","last_undegraded":"2024-05-30T17:04:54.119958+0000","last_fullsized":"2024-05-30T17:04:54.119958+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:11.010697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.962353+0000","last_change":"2024-05-30T17:04:52.967099+0000","last_active":"2024-05-30T17:04:53.962353+0000","last_peered":"2024-05-30T17:04:53.962353+0000","last_clean":"2024-05-30T17:04:53.962353+0000","last_became_active":"2024-05-30T17:04:52.966035+0000","last_became_peered":"2024-05-30T17:04:52.966035+0000","last_unstale":"2024-05-30T17:04:53.962353+0000","last_undegraded":"2024-05-30T17:04:53.962353+0000","last_fullsized":"2024-05-30T17:04:53.962353+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:24.565166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.072251+0000","last_change":"2024-05-30T17:04:51.949867+0000","last_active":"2024-05-30T17:04:54.072251+0000","last_peered":"2024-05-30T17:04:54.072251+0000","last_clean":"2024-05-30T17:04:54.072251+0000","last_became_active":"2024-05-30T17:04:51.949575+0000","last_became_peered":"2024-05-30T17:04:51.949575+0000","last_unstale":"2024-05-30T17:04:54.072251+0000","last_undegraded":"2024-05-30T17:04:54.072251+0000","last_fullsized":"2024-05-30T17:04:54.072251+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:58:11.353946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.072207+0000","last_change":"2024-05-30T17:04:52.998729+0000","last_active":"2024-05-30T17:04:54.072207+0000","last_peered":"2024-05-30T17:04:54.072207+0000","last_clean":"2024-05-30T17:04:54.072207+0000","last_became_active":"2024-05-30T17:04:52.997726+0000","last_became_peered":"2024-05-30T17:04:52.997726+0000","last_unstale":"2024-05-30T17:04:54.072207+0000","last_undegraded":"2024-05-30T17:04:54.072207+0000","last_fullsized":"2024-05-30T17:04:54.072207+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:32:14.601932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.001835+0000","last_change":"2024-05-30T17:04:51.949751+0000","last_active":"2024-05-30T17:04:54.001835+0000","last_peered":"2024-05-30T17:04:54.001835+0000","last_clean":"2024-05-30T17:04:54.001835+0000","last_became_active":"2024-05-30T17:04:51.949295+0000","last_became_peered":"2024-05-30T17:04:51.949295+0000","last_unstale":"2024-05-30T17:04:54.001835+0000","last_undegraded":"2024-05-30T17:04:54.001835+0000","last_fullsized":"2024-05-30T17:04:54.001835+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:41.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120054+0000","last_change":"2024-05-30T17:04:52.985036+0000","last_active":"2024-05-30T17:04:54.120054+0000","last_peered":"2024-05-30T17:04:54.120054+0000","last_clean":"2024-05-30T17:04:54.120054+0000","last_became_active":"2024-05-30T17:04:52.984675+0000","last_became_peered":"2024-05-30T17:04:52.984675+0000","last_unstale":"2024-05-30T17:04:54.120054+0000","last_undegraded":"2024-05-30T17:04:54.120054+0000","last_fullsized":"2024-05-30T17:04:54.120054+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120280+0000","last_change":"2024-05-30T17:04:51.952967+0000","last_active":"2024-05-30T17:04:54.120280+0000","last_peered":"2024-05-30T17:04:54.120280+0000","last_clean":"2024-05-30T17:04:54.120280+0000","last_became_active":"2024-05-30T17:04:51.952631+0000","last_became_peered":"2024-05-30T17:04:51.952631+0000","last_unstale":"2024-05-30T17:04:54.120280+0000","last_undegraded":"2024-05-30T17:04:54.120280+0000","last_fullsized":"2024-05-30T17:04:54.120280+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:23.747059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961783+0000","last_change":"2024-05-30T17:04:52.990963+0000","last_active":"2024-05-30T17:04:53.961783+0000","last_peered":"2024-05-30T17:04:53.961783+0000","last_clean":"2024-05-30T17:04:53.961783+0000","last_became_active":"2024-05-30T17:04:52.990476+0000","last_became_peered":"2024-05-30T17:04:52.990476+0000","last_unstale":"2024-05-30T17:04:53.961783+0000","last_undegraded":"2024-05-30T17:04:53.961783+0000","last_fullsized":"2024-05-30T17:04:53.961783+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:59.838344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120391+0000","last_change":"2024-05-30T17:04:51.933352+0000","last_active":"2024-05-30T17:04:54.120391+0000","last_peered":"2024-05-30T17:04:54.120391+0000","last_clean":"2024-05-30T17:04:54.120391+0000","last_became_active":"2024-05-30T17:04:51.933061+0000","last_became_peered":"2024-05-30T17:04:51.933061+0000","last_unstale":"2024-05-30T17:04:54.120391+0000","last_undegraded":"2024-05-30T17:04:54.120391+0000","last_fullsized":"2024-05-30T17:04:54.120391+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:20.242562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999136+0000","last_change":"2024-05-30T17:04:52.982699+0000","last_active":"2024-05-30T17:04:53.999136+0000","last_peered":"2024-05-30T17:04:53.999136+0000","last_clean":"2024-05-30T17:04:53.999136+0000","last_became_active":"2024-05-30T17:04:52.981542+0000","last_became_peered":"2024-05-30T17:04:52.981542+0000","last_unstale":"2024-05-30T17:04:53.999136+0000","last_undegraded":"2024-05-30T17:04:53.999136+0000","last_fullsized":"2024-05-30T17:04:53.999136+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:12.025684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119855+0000","last_change":"2024-05-30T17:04:51.947181+0000","last_active":"2024-05-30T17:04:54.119855+0000","last_peered":"2024-05-30T17:04:54.119855+0000","last_clean":"2024-05-30T17:04:54.119855+0000","last_became_active":"2024-05-30T17:04:51.946882+0000","last_became_peered":"2024-05-30T17:04:51.946882+0000","last_unstale":"2024-05-30T17:04:54.119855+0000","last_undegraded":"2024-05-30T17:04:54.119855+0000","last_fullsized":"2024-05-30T17:04:54.119855+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:47:52.431592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.959012+0000","last_change":"2024-05-30T17:04:52.992399+0000","last_active":"2024-05-30T17:04:53.959012+0000","last_peered":"2024-05-30T17:04:53.959012+0000","last_clean":"2024-05-30T17:04:53.959012+0000","last_became_active":"2024-05-30T17:04:52.992040+0000","last_became_peered":"2024-05-30T17:04:52.992040+0000","last_unstale":"2024-05-30T17:04:53.959012+0000","last_undegraded":"2024-05-30T17:04:53.959012+0000","last_fullsized":"2024-05-30T17:04:53.959012+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:16.335923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998986+0000","last_change":"2024-05-30T17:04:51.952089+0000","last_active":"2024-05-30T17:04:53.998986+0000","last_peered":"2024-05-30T17:04:53.998986+0000","last_clean":"2024-05-30T17:04:53.998986+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:04:53.998986+0000","last_undegraded":"2024-05-30T17:04:53.998986+0000","last_fullsized":"2024-05-30T17:04:53.998986+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:03:20.313618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953083+0000","last_change":"2024-05-30T17:04:52.988572+0000","last_active":"2024-05-30T17:04:53.953083+0000","last_peered":"2024-05-30T17:04:53.953083+0000","last_clean":"2024-05-30T17:04:53.953083+0000","last_became_active":"2024-05-30T17:04:52.987684+0000","last_became_peered":"2024-05-30T17:04:52.987684+0000","last_unstale":"2024-05-30T17:04:53.953083+0000","last_undegraded":"2024-05-30T17:04:53.953083+0000","last_fullsized":"2024-05-30T17:04:53.953083+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:26:52.920204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.952762+0000","last_change":"2024-05-30T17:04:51.949334+0000","last_active":"2024-05-30T17:04:53.952762+0000","last_peered":"2024-05-30T17:04:53.952762+0000","last_clean":"2024-05-30T17:04:53.952762+0000","last_became_active":"2024-05-30T17:04:51.948995+0000","last_became_peered":"2024-05-30T17:04:51.948995+0000","last_unstale":"2024-05-30T17:04:53.952762+0000","last_undegraded":"2024-05-30T17:04:53.952762+0000","last_fullsized":"2024-05-30T17:04:53.952762+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:05.331584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120171+0000","last_change":"2024-05-30T17:04:52.992411+0000","last_active":"2024-05-30T17:04:54.120171+0000","last_peered":"2024-05-30T17:04:54.120171+0000","last_clean":"2024-05-30T17:04:54.120171+0000","last_became_active":"2024-05-30T17:04:52.991997+0000","last_became_peered":"2024-05-30T17:04:52.991997+0000","last_unstale":"2024-05-30T17:04:54.120171+0000","last_undegraded":"2024-05-30T17:04:54.120171+0000","last_fullsized":"2024-05-30T17:04:54.120171+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120061+0000","last_change":"2024-05-30T17:04:51.946511+0000","last_active":"2024-05-30T17:04:54.120061+0000","last_peered":"2024-05-30T17:04:54.120061+0000","last_clean":"2024-05-30T17:04:54.120061+0000","last_became_active":"2024-05-30T17:04:51.946173+0000","last_became_peered":"2024-05-30T17:04:51.946173+0000","last_unstale":"2024-05-30T17:04:54.120061+0000","last_undegraded":"2024-05-30T17:04:54.120061+0000","last_fullsized":"2024-05-30T17:04:54.120061+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:08.206744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998869+0000","last_change":"2024-05-30T17:04:52.992639+0000","last_active":"2024-05-30T17:04:53.998869+0000","last_peered":"2024-05-30T17:04:53.998869+0000","last_clean":"2024-05-30T17:04:53.998869+0000","last_became_active":"2024-05-30T17:04:52.992202+0000","last_became_peered":"2024-05-30T17:04:52.992202+0000","last_unstale":"2024-05-30T17:04:53.998869+0000","last_undegraded":"2024-05-30T17:04:53.998869+0000","last_fullsized":"2024-05-30T17:04:53.998869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:34.869318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999432+0000","last_change":"2024-05-30T17:04:51.951966+0000","last_active":"2024-05-30T17:04:53.999432+0000","last_peered":"2024-05-30T17:04:53.999432+0000","last_clean":"2024-05-30T17:04:53.999432+0000","last_became_active":"2024-05-30T17:04:51.951484+0000","last_became_peered":"2024-05-30T17:04:51.951484+0000","last_unstale":"2024-05-30T17:04:53.999432+0000","last_undegraded":"2024-05-30T17:04:53.999432+0000","last_fullsized":"2024-05-30T17:04:53.999432+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:41.627294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":382,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998290+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:04:53.998290+0000","last_peered":"2024-05-30T17:04:53.998290+0000","last_clean":"2024-05-30T17:04:53.998290+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:04:53.998290+0000","last_undegraded":"2024-05-30T17:04:53.998290+0000","last_fullsized":"2024-05-30T17:04:53.998290+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509698,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118812,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253028,"statfs":{"total":96636764160,"available":96515100672,"internally_reserved":0,"allocated":1650688,"data_stored":725151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509698,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149328,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222512,"statfs":{"total":96636764160,"available":96483852288,"internally_reserved":0,"allocated":1642496,"data_stored":723609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118804,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253036,"statfs":{"total":96636764160,"available":96515108864,"internally_reserved":0,"allocated":1646592,"data_stored":723699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509699,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153496,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218344,"statfs":{"total":96636764160,"available":96479584256,"internally_reserved":0,"allocated":1646592,"data_stored":724155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509698,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149328,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222512,"statfs":{"total":96636764160,"available":96483852288,"internally_reserved":0,"allocated":1638400,"data_stored":723539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542401,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119400,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252440,"statfs":{"total":96636764160,"available":96514498560,"internally_reserved":0,"allocated":2256896,"data_stored":1316157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542402,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68808,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303032,"statfs":{"total":96636764160,"available":96566304768,"internally_reserved":0,"allocated":1634304,"data_stored":717239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542402,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145736,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226104,"statfs":{"total":96636764160,"available":96487530496,"internally_reserved":0,"allocated":2220032,"data_stored":1307495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:04:57.428 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:04:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.675+0000 7f2efca00640 1 Processor -- start 2024-05-30T17:04:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.676+0000 7f2efca00640 1 -- start start 2024-05-30T17:04:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.677+0000 7f2efca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2ef80d16a0 0x7f2ef80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2efca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cf870 0x7f2ef80d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2efca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef815ec50 con 0x7f2ef815e870 2024-05-30T17:04:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2efca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2ef80d2150 con 0x7f2ef80d16a0 2024-05-30T17:04:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2efca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef80cfd20 con 0x7f2ef80cf870 2024-05-30T17:04:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2ef7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2ef815e870 0x7f2ef80d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44076/0 (socket says 172.21.15.67:44076) 2024-05-30T17:04:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2ef7400640 1 -- 172.21.15.67:0/3385709820 learned_addr learned my addr 172.21.15.67:0/3385709820 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2ef6a00640 1 --2- 172.21.15.67:0/3385709820 >> v2:172.21.15.169:3300/0 conn(0x7f2ef80d16a0 0x7f2ef80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2ef7e00640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cf870 0x7f2ef80d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445112691 0 0) 0x7f2ef815ec50 con 0x7f2ef815e870 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.678+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2edc003520 con 0x7f2ef815e870 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 412630707 0 0) 0x7f2edc003520 con 0x7f2ef815e870 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cf870 msgr2=0x7f2ef80d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cf870 0x7f2ef80d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 >> v2:172.21.15.169:3300/0 conn(0x7f2ef80d16a0 msgr2=0x7f2ef80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 --2- 172.21.15.67:0/3385709820 >> v2:172.21.15.169:3300/0 conn(0x7f2ef80d16a0 0x7f2ef80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef80cff80 con 0x7f2ef815e870 2024-05-30T17:04:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f2ee8001670 con 0x7f2ef815e870 2024-05-30T17:04:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 >> v1:172.21.15.67:6789/0 conn(0x7f2ef815e870 legacy=0x7f2ef80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2edc0039b0 0x7f2edc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc004800 0x7f2edc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cf870 0x7f2edc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef80d2150 con 0x7f2edc004800 2024-05-30T17:04:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef80cfd20 con 0x7f2edc0039b0 2024-05-30T17:04:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc00b9c0 con 0x7f2ef80cf870 2024-05-30T17:04:57.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.679+0000 7f2ef6a00640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc004800 0x7f2edc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.680+0000 7f2ef7400640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2edc0039b0 0x7f2edc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.680+0000 7f2ef7e00640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cf870 0x7f2edc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.680+0000 7f2ef6a00640 1 -- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cf870 msgr2=0x7f2edc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.680+0000 7f2ef6a00640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cf870 0x7f2edc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.680+0000 7f2ef6a00640 1 -- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2edc0039b0 msgr2=0x7f2edc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.680+0000 7f2ef6a00640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2edc0039b0 0x7f2edc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.680+0000 7f2ef6a00640 1 -- 172.21.15.67:0/3385709820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc003520 con 0x7f2edc004800 2024-05-30T17:04:57.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.680+0000 7f2ef7e00640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cf870 0x7f2edc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.681+0000 7f2ef6a00640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc004800 0x7f2edc004c80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400e170 tx=0x7f2edc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce49509aa573c03f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.681+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4034020 con 0x7f2edc004800 2024-05-30T17:04:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.681+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee4030610 con 0x7f2edc004800 2024-05-30T17:04:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.681+0000 7f2ef6000640 1 -- 172.21.15.67:0/3385709820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4009c60 con 0x7f2edc004800 2024-05-30T17:04:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.682+0000 7f2efca00640 1 -- 172.21.15.67:0/3385709820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc004800 msgr2=0x7f2edc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.682+0000 7f2efca00640 1 --2- 172.21.15.67:0/3385709820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc004800 0x7f2edc004c80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400e170 tx=0x7f2edc005830 comp rx=0 tx=0).stop 2024-05-30T17:04:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.682+0000 7f2ef7e00640 1 -- 172.21.15.67:0/3385709820 reap_dead start 2024-05-30T17:04:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.682+0000 7f2efca00640 1 -- 172.21.15.67:0/3385709820 shutdown_connections 2024-05-30T17:04:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.682+0000 7f2efca00640 1 -- 172.21.15.67:0/3385709820 >> 172.21.15.67:0/3385709820 conn(0x7f2ef808bfc0 msgr2=0x7f2ef80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.682+0000 7f2efca00640 1 -- 172.21.15.67:0/3385709820 shutdown_connections 2024-05-30T17:04:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.682+0000 7f2efca00640 1 -- 172.21.15.67:0/3385709820 wait complete. 2024-05-30T17:04:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.684+0000 7f2efca00640 1 Processor -- start 2024-05-30T17:04:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.684+0000 7f2efca00640 1 -- start start 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.685+0000 7f2efca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2edc004800 0x7f2ef8159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.685+0000 7f2efca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ef8160f30 0x7f2ef81613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.685+0000 7f2efca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cc060 0x7f2ef80cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.685+0000 7f2efca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef80d0820 con 0x7f2ef8160f30 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.685+0000 7f2efca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef80d0540 con 0x7f2edc004800 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.685+0000 7f2efca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef80d4fb0 con 0x7f2ef80cc060 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ef8160f30 0x7f2ef81613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ef8160f30 0x7f2ef81613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54946/0 (socket says 172.21.15.67:54946) 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef6a00640 1 -- 172.21.15.67:0/237835312 learned_addr learned my addr 172.21.15.67:0/237835312 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef7400640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2edc004800 0x7f2ef8159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef7e00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cc060 0x7f2ef80cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef6a00640 1 -- 172.21.15.67:0/237835312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cc060 msgr2=0x7f2ef80cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef6a00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cc060 0x7f2ef80cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef6a00640 1 -- 172.21.15.67:0/237835312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2edc004800 msgr2=0x7f2ef8159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef6a00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2edc004800 0x7f2ef8159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef6a00640 1 -- 172.21.15.67:0/237835312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef80cd8d0 con 0x7f2ef8160f30 2024-05-30T17:04:57.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.686+0000 7f2ef6a00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ef8160f30 0x7f2ef81613b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4033040 tx=0x7f2ee4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:57.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.687+0000 7f2eef400640 1 -- 172.21.15.67:0/237835312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4040070 con 0x7f2ef8160f30 2024-05-30T17:04:57.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.687+0000 7f2eef400640 1 -- 172.21.15.67:0/237835312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee4039a00 con 0x7f2ef8160f30 2024-05-30T17:04:57.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.687+0000 7f2eef400640 1 -- 172.21.15.67:0/237835312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4034040 con 0x7f2ef8160f30 2024-05-30T17:04:57.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.687+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef80ccd50 con 0x7f2ef8160f30 2024-05-30T17:04:57.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.687+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef80cd310 con 0x7f2ef8160f30 2024-05-30T17:04:57.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.688+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef80689e0 con 0x7f2ef8160f30 2024-05-30T17:04:57.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.689+0000 7f2eef400640 1 -- 172.21.15.67:0/237835312 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f2ee4037020 con 0x7f2ef8160f30 2024-05-30T17:04:57.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.690+0000 7f2eef400640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ed806e510 0x7f2ed80709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:57.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.690+0000 7f2eef400640 1 -- 172.21.15.67:0/237835312 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f2ee40ab6d0 con 0x7f2ef8160f30 2024-05-30T17:04:57.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.691+0000 7f2ef7400640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ed806e510 0x7f2ed80709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:57.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.691+0000 7f2ef7400640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ed806e510 0x7f2ed80709d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8000c00 tx=0x7f2ee8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.695+0000 7f2eef400640 1 -- 172.21.15.67:0/237835312 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ee402d070 con 0x7f2ef8160f30 2024-05-30T17:04:57.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.895+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ef815a040 con 0x7f2ed806e510 2024-05-30T17:04:57.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.913+0000 7f2eef400640 1 -- 172.21.15.67:0/237835312 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252507 (secure 0 0 0) 0x7f2ee8010830 con 0x7f2ed806e510 2024-05-30T17:04:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:57.919 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:04:57.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ed806e510 msgr2=0x7f2ed80709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ed806e510 0x7f2ed80709d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8000c00 tx=0x7f2ee8009040 comp rx=0 tx=0).stop 2024-05-30T17:04:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ef8160f30 msgr2=0x7f2ef81613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ef8160f30 0x7f2ef81613b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4033040 tx=0x7f2ee4009750 comp rx=0 tx=0).stop 2024-05-30T17:04:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 shutdown_connections 2024-05-30T17:04:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2ed806e510 0x7f2ed80709d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ef80cc060 0x7f2ef80cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ef8160f30 0x7f2ef81613b0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 --2- 172.21.15.67:0/237835312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2edc004800 0x7f2ef8159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:57.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 >> 172.21.15.67:0/237835312 conn(0x7f2ef808bfc0 msgr2=0x7f2ef80b6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:57.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 shutdown_connections 2024-05-30T17:04:57.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:57.920+0000 7f2efca00640 1 -- 172.21.15.67:0/237835312 wait complete. 2024-05-30T17:04:57.939 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":764,"stamp":"2024-05-30T17:04:56.578595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1023712,"kb_used_data":14000,"kb_used_omap":12,"kb_used_meta":1009587,"kb_avail":753951008,"statfs":{"total":773094113280,"available":772045832192,"internally_reserved":0,"allocated":14336000,"data_stored":6961044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1033817648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.668918"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.072117+0000","last_change":"2024-05-30T17:04:52.998366+0000","last_active":"2024-05-30T17:04:54.072117+0000","last_peered":"2024-05-30T17:04:54.072117+0000","last_clean":"2024-05-30T17:04:54.072117+0000","last_became_active":"2024-05-30T17:04:52.997618+0000","last_became_peered":"2024-05-30T17:04:52.997618+0000","last_unstale":"2024-05-30T17:04:54.072117+0000","last_undegraded":"2024-05-30T17:04:54.072117+0000","last_fullsized":"2024-05-30T17:04:54.072117+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:57.883038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119766+0000","last_change":"2024-05-30T17:04:52.984295+0000","last_active":"2024-05-30T17:04:54.119766+0000","last_peered":"2024-05-30T17:04:54.119766+0000","last_clean":"2024-05-30T17:04:54.119766+0000","last_became_active":"2024-05-30T17:04:52.982524+0000","last_became_peered":"2024-05-30T17:04:52.982524+0000","last_unstale":"2024-05-30T17:04:54.119766+0000","last_undegraded":"2024-05-30T17:04:54.119766+0000","last_fullsized":"2024-05-30T17:04:54.119766+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:49.622879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123582+0000","last_change":"2024-05-30T17:04:52.984783+0000","last_active":"2024-05-30T17:04:54.123582+0000","last_peered":"2024-05-30T17:04:54.123582+0000","last_clean":"2024-05-30T17:04:54.123582+0000","last_became_active":"2024-05-30T17:04:52.984493+0000","last_became_peered":"2024-05-30T17:04:52.984493+0000","last_unstale":"2024-05-30T17:04:54.123582+0000","last_undegraded":"2024-05-30T17:04:54.123582+0000","last_fullsized":"2024-05-30T17:04:54.123582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:21.682021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071315+0000","last_change":"2024-05-30T17:04:52.997076+0000","last_active":"2024-05-30T17:04:54.071315+0000","last_peered":"2024-05-30T17:04:54.071315+0000","last_clean":"2024-05-30T17:04:54.071315+0000","last_became_active":"2024-05-30T17:04:52.996675+0000","last_became_peered":"2024-05-30T17:04:52.996675+0000","last_unstale":"2024-05-30T17:04:54.071315+0000","last_undegraded":"2024-05-30T17:04:54.071315+0000","last_fullsized":"2024-05-30T17:04:54.071315+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:49.622879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999332+0000","last_change":"2024-05-30T17:04:52.990350+0000","last_active":"2024-05-30T17:04:53.999332+0000","last_peered":"2024-05-30T17:04:53.999332+0000","last_clean":"2024-05-30T17:04:53.999332+0000","last_became_active":"2024-05-30T17:04:52.990018+0000","last_became_peered":"2024-05-30T17:04:52.990018+0000","last_unstale":"2024-05-30T17:04:53.999332+0000","last_undegraded":"2024-05-30T17:04:53.999332+0000","last_fullsized":"2024-05-30T17:04:53.999332+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:21.257549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961410+0000","last_change":"2024-05-30T17:04:52.992047+0000","last_active":"2024-05-30T17:04:53.961410+0000","last_peered":"2024-05-30T17:04:53.961410+0000","last_clean":"2024-05-30T17:04:53.961410+0000","last_became_active":"2024-05-30T17:04:52.991748+0000","last_became_peered":"2024-05-30T17:04:52.991748+0000","last_unstale":"2024-05-30T17:04:53.961410+0000","last_undegraded":"2024-05-30T17:04:53.961410+0000","last_fullsized":"2024-05-30T17:04:53.961410+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:06.346572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960723+0000","last_change":"2024-05-30T17:04:52.984245+0000","last_active":"2024-05-30T17:04:53.960723+0000","last_peered":"2024-05-30T17:04:53.960723+0000","last_clean":"2024-05-30T17:04:53.960723+0000","last_became_active":"2024-05-30T17:04:52.983960+0000","last_became_peered":"2024-05-30T17:04:52.983960+0000","last_unstale":"2024-05-30T17:04:53.960723+0000","last_undegraded":"2024-05-30T17:04:53.960723+0000","last_fullsized":"2024-05-30T17:04:53.960723+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:40:33.491032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.122274+0000","last_change":"2024-05-30T17:04:52.988608+0000","last_active":"2024-05-30T17:04:54.122274+0000","last_peered":"2024-05-30T17:04:54.122274+0000","last_clean":"2024-05-30T17:04:54.122274+0000","last_became_active":"2024-05-30T17:04:52.988277+0000","last_became_peered":"2024-05-30T17:04:52.988277+0000","last_unstale":"2024-05-30T17:04:54.122274+0000","last_undegraded":"2024-05-30T17:04:54.122274+0000","last_fullsized":"2024-05-30T17:04:54.122274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:34.512494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123990+0000","last_change":"2024-05-30T17:04:52.993014+0000","last_active":"2024-05-30T17:04:54.123990+0000","last_peered":"2024-05-30T17:04:54.123990+0000","last_clean":"2024-05-30T17:04:54.123990+0000","last_became_active":"2024-05-30T17:04:52.990543+0000","last_became_peered":"2024-05-30T17:04:52.990543+0000","last_unstale":"2024-05-30T17:04:54.123990+0000","last_undegraded":"2024-05-30T17:04:54.123990+0000","last_fullsized":"2024-05-30T17:04:54.123990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:27:15.135800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119758+0000","last_change":"2024-05-30T17:04:52.979749+0000","last_active":"2024-05-30T17:04:54.119758+0000","last_peered":"2024-05-30T17:04:54.119758+0000","last_clean":"2024-05-30T17:04:54.119758+0000","last_became_active":"2024-05-30T17:04:52.979390+0000","last_became_peered":"2024-05-30T17:04:52.979390+0000","last_unstale":"2024-05-30T17:04:54.119758+0000","last_undegraded":"2024-05-30T17:04:54.119758+0000","last_fullsized":"2024-05-30T17:04:54.119758+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119955+0000","last_change":"2024-05-30T17:04:52.983598+0000","last_active":"2024-05-30T17:04:54.119955+0000","last_peered":"2024-05-30T17:04:54.119955+0000","last_clean":"2024-05-30T17:04:54.119955+0000","last_became_active":"2024-05-30T17:04:52.983250+0000","last_became_peered":"2024-05-30T17:04:52.983250+0000","last_unstale":"2024-05-30T17:04:54.119955+0000","last_undegraded":"2024-05-30T17:04:54.119955+0000","last_fullsized":"2024-05-30T17:04:54.119955+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:20.889101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.122809+0000","last_change":"2024-05-30T17:04:52.993606+0000","last_active":"2024-05-30T17:04:54.122809+0000","last_peered":"2024-05-30T17:04:54.122809+0000","last_clean":"2024-05-30T17:04:54.122809+0000","last_became_active":"2024-05-30T17:04:52.993151+0000","last_became_peered":"2024-05-30T17:04:52.993151+0000","last_unstale":"2024-05-30T17:04:54.122809+0000","last_undegraded":"2024-05-30T17:04:54.122809+0000","last_fullsized":"2024-05-30T17:04:54.122809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:14.893435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071930+0000","last_change":"2024-05-30T17:04:52.998102+0000","last_active":"2024-05-30T17:04:54.071930+0000","last_peered":"2024-05-30T17:04:54.071930+0000","last_clean":"2024-05-30T17:04:54.071930+0000","last_became_active":"2024-05-30T17:04:52.997770+0000","last_became_peered":"2024-05-30T17:04:52.997770+0000","last_unstale":"2024-05-30T17:04:54.071930+0000","last_undegraded":"2024-05-30T17:04:54.071930+0000","last_fullsized":"2024-05-30T17:04:54.071930+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119872+0000","last_change":"2024-05-30T17:04:52.992617+0000","last_active":"2024-05-30T17:04:54.119872+0000","last_peered":"2024-05-30T17:04:54.119872+0000","last_clean":"2024-05-30T17:04:54.119872+0000","last_became_active":"2024-05-30T17:04:52.992221+0000","last_became_peered":"2024-05-30T17:04:52.992221+0000","last_unstale":"2024-05-30T17:04:54.119872+0000","last_undegraded":"2024-05-30T17:04:54.119872+0000","last_fullsized":"2024-05-30T17:04:54.119872+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.070601+0000","last_change":"2024-05-30T17:04:52.992819+0000","last_active":"2024-05-30T17:04:54.070601+0000","last_peered":"2024-05-30T17:04:54.070601+0000","last_clean":"2024-05-30T17:04:54.070601+0000","last_became_active":"2024-05-30T17:04:52.991693+0000","last_became_peered":"2024-05-30T17:04:52.991693+0000","last_unstale":"2024-05-30T17:04:54.070601+0000","last_undegraded":"2024-05-30T17:04:54.070601+0000","last_fullsized":"2024-05-30T17:04:54.070601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.952890+0000","last_change":"2024-05-30T17:04:52.991328+0000","last_active":"2024-05-30T17:04:53.952890+0000","last_peered":"2024-05-30T17:04:53.952890+0000","last_clean":"2024-05-30T17:04:53.952890+0000","last_became_active":"2024-05-30T17:04:52.990334+0000","last_became_peered":"2024-05-30T17:04:52.990334+0000","last_unstale":"2024-05-30T17:04:53.952890+0000","last_undegraded":"2024-05-30T17:04:53.952890+0000","last_fullsized":"2024-05-30T17:04:53.952890+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:23.528030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.962307+0000","last_change":"2024-05-30T17:04:52.967087+0000","last_active":"2024-05-30T17:04:53.962307+0000","last_peered":"2024-05-30T17:04:53.962307+0000","last_clean":"2024-05-30T17:04:53.962307+0000","last_became_active":"2024-05-30T17:04:52.966026+0000","last_became_peered":"2024-05-30T17:04:52.966026+0000","last_unstale":"2024-05-30T17:04:53.962307+0000","last_undegraded":"2024-05-30T17:04:53.962307+0000","last_fullsized":"2024-05-30T17:04:53.962307+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:12.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953508+0000","last_change":"2024-05-30T17:04:52.986386+0000","last_active":"2024-05-30T17:04:53.953508+0000","last_peered":"2024-05-30T17:04:53.953508+0000","last_clean":"2024-05-30T17:04:53.953508+0000","last_became_active":"2024-05-30T17:04:52.985602+0000","last_became_peered":"2024-05-30T17:04:52.985602+0000","last_unstale":"2024-05-30T17:04:53.953508+0000","last_undegraded":"2024-05-30T17:04:53.953508+0000","last_fullsized":"2024-05-30T17:04:53.953508+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:31.682869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120117+0000","last_change":"2024-05-30T17:04:52.963412+0000","last_active":"2024-05-30T17:04:54.120117+0000","last_peered":"2024-05-30T17:04:54.120117+0000","last_clean":"2024-05-30T17:04:54.120117+0000","last_became_active":"2024-05-30T17:04:52.962582+0000","last_became_peered":"2024-05-30T17:04:52.962582+0000","last_unstale":"2024-05-30T17:04:54.120117+0000","last_undegraded":"2024-05-30T17:04:54.120117+0000","last_fullsized":"2024-05-30T17:04:54.120117+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:52.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953269+0000","last_change":"2024-05-30T17:04:52.987028+0000","last_active":"2024-05-30T17:04:53.953269+0000","last_peered":"2024-05-30T17:04:53.953269+0000","last_clean":"2024-05-30T17:04:53.953269+0000","last_became_active":"2024-05-30T17:04:52.986701+0000","last_became_peered":"2024-05-30T17:04:52.986701+0000","last_unstale":"2024-05-30T17:04:53.953269+0000","last_undegraded":"2024-05-30T17:04:53.953269+0000","last_fullsized":"2024-05-30T17:04:53.953269+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:50:37.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999097+0000","last_change":"2024-05-30T17:04:52.984763+0000","last_active":"2024-05-30T17:04:53.999097+0000","last_peered":"2024-05-30T17:04:53.999097+0000","last_clean":"2024-05-30T17:04:53.999097+0000","last_became_active":"2024-05-30T17:04:52.984462+0000","last_became_peered":"2024-05-30T17:04:52.984462+0000","last_unstale":"2024-05-30T17:04:53.999097+0000","last_undegraded":"2024-05-30T17:04:53.999097+0000","last_fullsized":"2024-05-30T17:04:53.999097+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:09.221732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119653+0000","last_change":"2024-05-30T17:04:52.980574+0000","last_active":"2024-05-30T17:04:54.119653+0000","last_peered":"2024-05-30T17:04:54.119653+0000","last_clean":"2024-05-30T17:04:54.119653+0000","last_became_active":"2024-05-30T17:04:52.978968+0000","last_became_peered":"2024-05-30T17:04:52.978968+0000","last_unstale":"2024-05-30T17:04:54.119653+0000","last_undegraded":"2024-05-30T17:04:54.119653+0000","last_fullsized":"2024-05-30T17:04:54.119653+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:30:51.419747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123031+0000","last_change":"2024-05-30T17:04:52.992275+0000","last_active":"2024-05-30T17:04:54.123031+0000","last_peered":"2024-05-30T17:04:54.123031+0000","last_clean":"2024-05-30T17:04:54.123031+0000","last_became_active":"2024-05-30T17:04:52.991966+0000","last_became_peered":"2024-05-30T17:04:52.991966+0000","last_unstale":"2024-05-30T17:04:54.123031+0000","last_undegraded":"2024-05-30T17:04:54.123031+0000","last_fullsized":"2024-05-30T17:04:54.123031+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:32.227484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119833+0000","last_change":"2024-05-30T17:04:52.965882+0000","last_active":"2024-05-30T17:04:54.119833+0000","last_peered":"2024-05-30T17:04:54.119833+0000","last_clean":"2024-05-30T17:04:54.119833+0000","last_became_active":"2024-05-30T17:04:52.965116+0000","last_became_peered":"2024-05-30T17:04:52.965116+0000","last_unstale":"2024-05-30T17:04:54.119833+0000","last_undegraded":"2024-05-30T17:04:54.119833+0000","last_fullsized":"2024-05-30T17:04:54.119833+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:35:54.995662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119957+0000","last_change":"2024-05-30T17:04:52.983400+0000","last_active":"2024-05-30T17:04:54.119957+0000","last_peered":"2024-05-30T17:04:54.119957+0000","last_clean":"2024-05-30T17:04:54.119957+0000","last_became_active":"2024-05-30T17:04:52.981867+0000","last_became_peered":"2024-05-30T17:04:52.981867+0000","last_unstale":"2024-05-30T17:04:54.119957+0000","last_undegraded":"2024-05-30T17:04:54.119957+0000","last_fullsized":"2024-05-30T17:04:54.119957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953764+0000","last_change":"2024-05-30T17:04:52.989413+0000","last_active":"2024-05-30T17:04:53.953764+0000","last_peered":"2024-05-30T17:04:53.953764+0000","last_clean":"2024-05-30T17:04:53.953764+0000","last_became_active":"2024-05-30T17:04:52.989141+0000","last_became_peered":"2024-05-30T17:04:52.989141+0000","last_unstale":"2024-05-30T17:04:53.953764+0000","last_undegraded":"2024-05-30T17:04:53.953764+0000","last_fullsized":"2024-05-30T17:04:53.953764+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:15.844444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071623+0000","last_change":"2024-05-30T17:04:52.992667+0000","last_active":"2024-05-30T17:04:54.071623+0000","last_peered":"2024-05-30T17:04:54.071623+0000","last_clean":"2024-05-30T17:04:54.071623+0000","last_became_active":"2024-05-30T17:04:52.988618+0000","last_became_peered":"2024-05-30T17:04:52.988618+0000","last_unstale":"2024-05-30T17:04:54.071623+0000","last_undegraded":"2024-05-30T17:04:54.071623+0000","last_fullsized":"2024-05-30T17:04:54.071623+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.958939+0000","last_change":"2024-05-30T17:04:52.977678+0000","last_active":"2024-05-30T17:04:53.958939+0000","last_peered":"2024-05-30T17:04:53.958939+0000","last_clean":"2024-05-30T17:04:53.958939+0000","last_became_active":"2024-05-30T17:04:52.977378+0000","last_became_peered":"2024-05-30T17:04:52.977378+0000","last_unstale":"2024-05-30T17:04:53.958939+0000","last_undegraded":"2024-05-30T17:04:53.958939+0000","last_fullsized":"2024-05-30T17:04:53.958939+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.220555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071535+0000","last_change":"2024-05-30T17:04:52.998322+0000","last_active":"2024-05-30T17:04:54.071535+0000","last_peered":"2024-05-30T17:04:54.071535+0000","last_clean":"2024-05-30T17:04:54.071535+0000","last_became_active":"2024-05-30T17:04:52.997323+0000","last_became_peered":"2024-05-30T17:04:52.997323+0000","last_unstale":"2024-05-30T17:04:54.071535+0000","last_undegraded":"2024-05-30T17:04:54.071535+0000","last_fullsized":"2024-05-30T17:04:54.071535+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071181+0000","last_change":"2024-05-30T17:04:52.995308+0000","last_active":"2024-05-30T17:04:54.071181+0000","last_peered":"2024-05-30T17:04:54.071181+0000","last_clean":"2024-05-30T17:04:54.071181+0000","last_became_active":"2024-05-30T17:04:52.992733+0000","last_became_peered":"2024-05-30T17:04:52.992733+0000","last_unstale":"2024-05-30T17:04:54.071181+0000","last_undegraded":"2024-05-30T17:04:54.071181+0000","last_fullsized":"2024-05-30T17:04:54.071181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998865+0000","last_change":"2024-05-30T17:04:52.993970+0000","last_active":"2024-05-30T17:04:53.998865+0000","last_peered":"2024-05-30T17:04:53.998865+0000","last_clean":"2024-05-30T17:04:53.998865+0000","last_became_active":"2024-05-30T17:04:52.993214+0000","last_became_peered":"2024-05-30T17:04:52.993214+0000","last_unstale":"2024-05-30T17:04:53.998865+0000","last_undegraded":"2024-05-30T17:04:53.998865+0000","last_fullsized":"2024-05-30T17:04:53.998865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:21:06.134079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960921+0000","last_change":"2024-05-30T17:04:52.980018+0000","last_active":"2024-05-30T17:04:53.960921+0000","last_peered":"2024-05-30T17:04:53.960921+0000","last_clean":"2024-05-30T17:04:53.960921+0000","last_became_active":"2024-05-30T17:04:52.979697+0000","last_became_peered":"2024-05-30T17:04:52.979697+0000","last_unstale":"2024-05-30T17:04:53.960921+0000","last_undegraded":"2024-05-30T17:04:53.960921+0000","last_fullsized":"2024-05-30T17:04:53.960921+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:30:44.014138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953739+0000","last_change":"2024-05-30T17:04:52.978441+0000","last_active":"2024-05-30T17:04:53.953739+0000","last_peered":"2024-05-30T17:04:53.953739+0000","last_clean":"2024-05-30T17:04:53.953739+0000","last_became_active":"2024-05-30T17:04:52.978065+0000","last_became_peered":"2024-05-30T17:04:52.978065+0000","last_unstale":"2024-05-30T17:04:53.953739+0000","last_undegraded":"2024-05-30T17:04:53.953739+0000","last_fullsized":"2024-05-30T17:04:53.953739+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:21:00.756549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999395+0000","last_change":"2024-05-30T17:04:51.944167+0000","last_active":"2024-05-30T17:04:53.999395+0000","last_peered":"2024-05-30T17:04:53.999395+0000","last_clean":"2024-05-30T17:04:53.999395+0000","last_became_active":"2024-05-30T17:04:51.943796+0000","last_became_peered":"2024-05-30T17:04:51.943796+0000","last_unstale":"2024-05-30T17:04:53.999395+0000","last_undegraded":"2024-05-30T17:04:53.999395+0000","last_fullsized":"2024-05-30T17:04:53.999395+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:50.016832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.124052+0000","last_change":"2024-05-30T17:04:52.998976+0000","last_active":"2024-05-30T17:04:54.124052+0000","last_peered":"2024-05-30T17:04:54.124052+0000","last_clean":"2024-05-30T17:04:54.124052+0000","last_became_active":"2024-05-30T17:04:52.998673+0000","last_became_peered":"2024-05-30T17:04:52.998673+0000","last_unstale":"2024-05-30T17:04:54.124052+0000","last_undegraded":"2024-05-30T17:04:54.124052+0000","last_fullsized":"2024-05-30T17:04:54.124052+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:58.331966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.959283+0000","last_change":"2024-05-30T17:04:51.939570+0000","last_active":"2024-05-30T17:04:53.959283+0000","last_peered":"2024-05-30T17:04:53.959283+0000","last_clean":"2024-05-30T17:04:53.959283+0000","last_became_active":"2024-05-30T17:04:51.937127+0000","last_became_peered":"2024-05-30T17:04:51.937127+0000","last_unstale":"2024-05-30T17:04:53.959283+0000","last_undegraded":"2024-05-30T17:04:53.959283+0000","last_fullsized":"2024-05-30T17:04:53.959283+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:18.245355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999390+0000","last_change":"2024-05-30T17:04:52.983816+0000","last_active":"2024-05-30T17:04:53.999390+0000","last_peered":"2024-05-30T17:04:53.999390+0000","last_clean":"2024-05-30T17:04:53.999390+0000","last_became_active":"2024-05-30T17:04:52.981316+0000","last_became_peered":"2024-05-30T17:04:52.981316+0000","last_unstale":"2024-05-30T17:04:53.999390+0000","last_undegraded":"2024-05-30T17:04:53.999390+0000","last_fullsized":"2024-05-30T17:04:53.999390+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:13.610126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120082+0000","last_change":"2024-05-30T17:04:51.938960+0000","last_active":"2024-05-30T17:04:54.120082+0000","last_peered":"2024-05-30T17:04:54.120082+0000","last_clean":"2024-05-30T17:04:54.120082+0000","last_became_active":"2024-05-30T17:04:51.938644+0000","last_became_peered":"2024-05-30T17:04:51.938644+0000","last_unstale":"2024-05-30T17:04:54.120082+0000","last_undegraded":"2024-05-30T17:04:54.120082+0000","last_fullsized":"2024-05-30T17:04:54.120082+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:12.595138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999528+0000","last_change":"2024-05-30T17:04:52.988274+0000","last_active":"2024-05-30T17:04:53.999528+0000","last_peered":"2024-05-30T17:04:53.999528+0000","last_clean":"2024-05-30T17:04:53.999528+0000","last_became_active":"2024-05-30T17:04:52.987961+0000","last_became_peered":"2024-05-30T17:04:52.987961+0000","last_unstale":"2024-05-30T17:04:53.999528+0000","last_undegraded":"2024-05-30T17:04:53.999528+0000","last_fullsized":"2024-05-30T17:04:53.999528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:21.613683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120166+0000","last_change":"2024-05-30T17:04:51.930251+0000","last_active":"2024-05-30T17:04:54.120166+0000","last_peered":"2024-05-30T17:04:54.120166+0000","last_clean":"2024-05-30T17:04:54.120166+0000","last_became_active":"2024-05-30T17:04:51.929298+0000","last_became_peered":"2024-05-30T17:04:51.929298+0000","last_unstale":"2024-05-30T17:04:54.120166+0000","last_undegraded":"2024-05-30T17:04:54.120166+0000","last_fullsized":"2024-05-30T17:04:54.120166+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:35.687280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953143+0000","last_change":"2024-05-30T17:04:52.986145+0000","last_active":"2024-05-30T17:04:53.953143+0000","last_peered":"2024-05-30T17:04:53.953143+0000","last_clean":"2024-05-30T17:04:53.953143+0000","last_became_active":"2024-05-30T17:04:52.985811+0000","last_became_peered":"2024-05-30T17:04:52.985811+0000","last_unstale":"2024-05-30T17:04:53.953143+0000","last_undegraded":"2024-05-30T17:04:53.953143+0000","last_fullsized":"2024-05-30T17:04:53.953143+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:55.493534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119735+0000","last_change":"2024-05-30T17:04:51.943821+0000","last_active":"2024-05-30T17:04:54.119735+0000","last_peered":"2024-05-30T17:04:54.119735+0000","last_clean":"2024-05-30T17:04:54.119735+0000","last_became_active":"2024-05-30T17:04:51.942597+0000","last_became_peered":"2024-05-30T17:04:51.942597+0000","last_unstale":"2024-05-30T17:04:54.119735+0000","last_undegraded":"2024-05-30T17:04:54.119735+0000","last_fullsized":"2024-05-30T17:04:54.119735+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:47.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961034+0000","last_change":"2024-05-30T17:04:52.967603+0000","last_active":"2024-05-30T17:04:53.961034+0000","last_peered":"2024-05-30T17:04:53.961034+0000","last_clean":"2024-05-30T17:04:53.961034+0000","last_became_active":"2024-05-30T17:04:52.967033+0000","last_became_peered":"2024-05-30T17:04:52.967033+0000","last_unstale":"2024-05-30T17:04:53.961034+0000","last_undegraded":"2024-05-30T17:04:53.961034+0000","last_fullsized":"2024-05-30T17:04:53.961034+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:18.064738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.952911+0000","last_change":"2024-05-30T17:04:51.942553+0000","last_active":"2024-05-30T17:04:53.952911+0000","last_peered":"2024-05-30T17:04:53.952911+0000","last_clean":"2024-05-30T17:04:53.952911+0000","last_became_active":"2024-05-30T17:04:51.941974+0000","last_became_peered":"2024-05-30T17:04:51.941974+0000","last_unstale":"2024-05-30T17:04:53.952911+0000","last_undegraded":"2024-05-30T17:04:53.952911+0000","last_fullsized":"2024-05-30T17:04:53.952911+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:40.877465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123264+0000","last_change":"2024-05-30T17:04:52.988985+0000","last_active":"2024-05-30T17:04:54.123264+0000","last_peered":"2024-05-30T17:04:54.123264+0000","last_clean":"2024-05-30T17:04:54.123264+0000","last_became_active":"2024-05-30T17:04:52.988645+0000","last_became_peered":"2024-05-30T17:04:52.988645+0000","last_unstale":"2024-05-30T17:04:54.123264+0000","last_undegraded":"2024-05-30T17:04:54.123264+0000","last_fullsized":"2024-05-30T17:04:54.123264+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:40.533224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960536+0000","last_change":"2024-05-30T17:04:51.951513+0000","last_active":"2024-05-30T17:04:53.960536+0000","last_peered":"2024-05-30T17:04:53.960536+0000","last_clean":"2024-05-30T17:04:53.960536+0000","last_became_active":"2024-05-30T17:04:51.951223+0000","last_became_peered":"2024-05-30T17:04:51.951223+0000","last_unstale":"2024-05-30T17:04:53.960536+0000","last_undegraded":"2024-05-30T17:04:53.960536+0000","last_fullsized":"2024-05-30T17:04:53.960536+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:13.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119495+0000","last_change":"2024-05-30T17:04:52.986452+0000","last_active":"2024-05-30T17:04:54.119495+0000","last_peered":"2024-05-30T17:04:54.119495+0000","last_clean":"2024-05-30T17:04:54.119495+0000","last_became_active":"2024-05-30T17:04:52.986164+0000","last_became_peered":"2024-05-30T17:04:52.986164+0000","last_unstale":"2024-05-30T17:04:54.119495+0000","last_undegraded":"2024-05-30T17:04:54.119495+0000","last_fullsized":"2024-05-30T17:04:54.119495+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119766+0000","last_change":"2024-05-30T17:04:51.955992+0000","last_active":"2024-05-30T17:04:54.119766+0000","last_peered":"2024-05-30T17:04:54.119766+0000","last_clean":"2024-05-30T17:04:54.119766+0000","last_became_active":"2024-05-30T17:04:51.955666+0000","last_became_peered":"2024-05-30T17:04:51.955666+0000","last_unstale":"2024-05-30T17:04:54.119766+0000","last_undegraded":"2024-05-30T17:04:54.119766+0000","last_fullsized":"2024-05-30T17:04:54.119766+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:03.343211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071910+0000","last_change":"2024-05-30T17:04:52.984705+0000","last_active":"2024-05-30T17:04:54.071910+0000","last_peered":"2024-05-30T17:04:54.071910+0000","last_clean":"2024-05-30T17:04:54.071910+0000","last_became_active":"2024-05-30T17:04:52.984409+0000","last_became_peered":"2024-05-30T17:04:52.984409+0000","last_unstale":"2024-05-30T17:04:54.071910+0000","last_undegraded":"2024-05-30T17:04:54.071910+0000","last_fullsized":"2024-05-30T17:04:54.071910+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960019+0000","last_change":"2024-05-30T17:04:51.952573+0000","last_active":"2024-05-30T17:04:53.960019+0000","last_peered":"2024-05-30T17:04:53.960019+0000","last_clean":"2024-05-30T17:04:53.960019+0000","last_became_active":"2024-05-30T17:04:51.948281+0000","last_became_peered":"2024-05-30T17:04:51.948281+0000","last_unstale":"2024-05-30T17:04:53.960019+0000","last_undegraded":"2024-05-30T17:04:53.960019+0000","last_fullsized":"2024-05-30T17:04:53.960019+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:26.183919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.959771+0000","last_change":"2024-05-30T17:04:52.986184+0000","last_active":"2024-05-30T17:04:53.959771+0000","last_peered":"2024-05-30T17:04:53.959771+0000","last_clean":"2024-05-30T17:04:53.959771+0000","last_became_active":"2024-05-30T17:04:52.985889+0000","last_became_peered":"2024-05-30T17:04:52.985889+0000","last_unstale":"2024-05-30T17:04:53.959771+0000","last_undegraded":"2024-05-30T17:04:53.959771+0000","last_fullsized":"2024-05-30T17:04:53.959771+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:27.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953802+0000","last_change":"2024-05-30T17:04:51.954829+0000","last_active":"2024-05-30T17:04:53.953802+0000","last_peered":"2024-05-30T17:04:53.953802+0000","last_clean":"2024-05-30T17:04:53.953802+0000","last_became_active":"2024-05-30T17:04:51.954434+0000","last_became_peered":"2024-05-30T17:04:51.954434+0000","last_unstale":"2024-05-30T17:04:53.953802+0000","last_undegraded":"2024-05-30T17:04:53.953802+0000","last_fullsized":"2024-05-30T17:04:53.953802+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:23.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999639+0000","last_change":"2024-05-30T17:04:52.989346+0000","last_active":"2024-05-30T17:04:53.999639+0000","last_peered":"2024-05-30T17:04:53.999639+0000","last_clean":"2024-05-30T17:04:53.999639+0000","last_became_active":"2024-05-30T17:04:52.989051+0000","last_became_peered":"2024-05-30T17:04:52.989051+0000","last_unstale":"2024-05-30T17:04:53.999639+0000","last_undegraded":"2024-05-30T17:04:53.999639+0000","last_fullsized":"2024-05-30T17:04:53.999639+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:36.128167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"131'3","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.958419+0000","last_change":"2024-05-30T17:04:51.938158+0000","last_active":"2024-05-30T17:04:53.958419+0000","last_peered":"2024-05-30T17:04:53.958419+0000","last_clean":"2024-05-30T17:04:53.958419+0000","last_became_active":"2024-05-30T17:04:51.937857+0000","last_became_peered":"2024-05-30T17:04:51.937857+0000","last_unstale":"2024-05-30T17:04:53.958419+0000","last_undegraded":"2024-05-30T17:04:53.958419+0000","last_fullsized":"2024-05-30T17:04:53.958419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:25.458335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999409+0000","last_change":"2024-05-30T17:04:52.995423+0000","last_active":"2024-05-30T17:04:53.999409+0000","last_peered":"2024-05-30T17:04:53.999409+0000","last_clean":"2024-05-30T17:04:53.999409+0000","last_became_active":"2024-05-30T17:04:52.995128+0000","last_became_peered":"2024-05-30T17:04:52.995128+0000","last_unstale":"2024-05-30T17:04:53.999409+0000","last_undegraded":"2024-05-30T17:04:53.999409+0000","last_fullsized":"2024-05-30T17:04:53.999409+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:47.501828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"131'6","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123102+0000","last_change":"2024-05-30T17:04:51.953550+0000","last_active":"2024-05-30T17:04:54.123102+0000","last_peered":"2024-05-30T17:04:54.123102+0000","last_clean":"2024-05-30T17:04:54.123102+0000","last_became_active":"2024-05-30T17:04:51.953276+0000","last_became_peered":"2024-05-30T17:04:51.953276+0000","last_unstale":"2024-05-30T17:04:54.123102+0000","last_undegraded":"2024-05-30T17:04:54.123102+0000","last_fullsized":"2024-05-30T17:04:54.123102+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:07:44.468185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071949+0000","last_change":"2024-05-30T17:04:52.988845+0000","last_active":"2024-05-30T17:04:54.071949+0000","last_peered":"2024-05-30T17:04:54.071949+0000","last_clean":"2024-05-30T17:04:54.071949+0000","last_became_active":"2024-05-30T17:04:52.986757+0000","last_became_peered":"2024-05-30T17:04:52.986757+0000","last_unstale":"2024-05-30T17:04:54.071949+0000","last_undegraded":"2024-05-30T17:04:54.071949+0000","last_fullsized":"2024-05-30T17:04:54.071949+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071262+0000","last_change":"2024-05-30T17:04:51.944552+0000","last_active":"2024-05-30T17:04:54.071262+0000","last_peered":"2024-05-30T17:04:54.071262+0000","last_clean":"2024-05-30T17:04:54.071262+0000","last_became_active":"2024-05-30T17:04:51.944167+0000","last_became_peered":"2024-05-30T17:04:51.944167+0000","last_unstale":"2024-05-30T17:04:54.071262+0000","last_undegraded":"2024-05-30T17:04:54.071262+0000","last_fullsized":"2024-05-30T17:04:54.071262+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:17:06.906161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119484+0000","last_change":"2024-05-30T17:04:52.991186+0000","last_active":"2024-05-30T17:04:54.119484+0000","last_peered":"2024-05-30T17:04:54.119484+0000","last_clean":"2024-05-30T17:04:54.119484+0000","last_became_active":"2024-05-30T17:04:52.990049+0000","last_became_peered":"2024-05-30T17:04:52.990049+0000","last_unstale":"2024-05-30T17:04:54.119484+0000","last_undegraded":"2024-05-30T17:04:54.119484+0000","last_fullsized":"2024-05-30T17:04:54.119484+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:28:23.896291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119598+0000","last_change":"2024-05-30T17:04:51.947216+0000","last_active":"2024-05-30T17:04:54.119598+0000","last_peered":"2024-05-30T17:04:54.119598+0000","last_clean":"2024-05-30T17:04:54.119598+0000","last_became_active":"2024-05-30T17:04:51.946868+0000","last_became_peered":"2024-05-30T17:04:51.946868+0000","last_unstale":"2024-05-30T17:04:54.119598+0000","last_undegraded":"2024-05-30T17:04:54.119598+0000","last_fullsized":"2024-05-30T17:04:54.119598+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:42:48.589336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.122520+0000","last_change":"2024-05-30T17:04:52.986670+0000","last_active":"2024-05-30T17:04:54.122520+0000","last_peered":"2024-05-30T17:04:54.122520+0000","last_clean":"2024-05-30T17:04:54.122520+0000","last_became_active":"2024-05-30T17:04:52.986293+0000","last_became_peered":"2024-05-30T17:04:52.986293+0000","last_unstale":"2024-05-30T17:04:54.122520+0000","last_undegraded":"2024-05-30T17:04:54.122520+0000","last_fullsized":"2024-05-30T17:04:54.122520+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:28.097184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.122953+0000","last_change":"2024-05-30T17:04:51.955924+0000","last_active":"2024-05-30T17:04:54.122953+0000","last_peered":"2024-05-30T17:04:54.122953+0000","last_clean":"2024-05-30T17:04:54.122953+0000","last_became_active":"2024-05-30T17:04:51.955542+0000","last_became_peered":"2024-05-30T17:04:51.955542+0000","last_unstale":"2024-05-30T17:04:54.122953+0000","last_undegraded":"2024-05-30T17:04:54.122953+0000","last_fullsized":"2024-05-30T17:04:54.122953+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:38.543340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.123820+0000","last_change":"2024-05-30T17:04:53.001644+0000","last_active":"2024-05-30T17:04:54.123820+0000","last_peered":"2024-05-30T17:04:54.123820+0000","last_clean":"2024-05-30T17:04:54.123820+0000","last_became_active":"2024-05-30T17:04:52.999924+0000","last_became_peered":"2024-05-30T17:04:52.999924+0000","last_unstale":"2024-05-30T17:04:54.123820+0000","last_undegraded":"2024-05-30T17:04:54.123820+0000","last_fullsized":"2024-05-30T17:04:54.123820+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:17:36.947839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.960175+0000","last_change":"2024-05-30T17:04:51.948540+0000","last_active":"2024-05-30T17:04:53.960175+0000","last_peered":"2024-05-30T17:04:53.960175+0000","last_clean":"2024-05-30T17:04:53.960175+0000","last_became_active":"2024-05-30T17:04:51.947928+0000","last_became_peered":"2024-05-30T17:04:51.947928+0000","last_unstale":"2024-05-30T17:04:53.960175+0000","last_undegraded":"2024-05-30T17:04:53.960175+0000","last_fullsized":"2024-05-30T17:04:53.960175+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:10:19.343810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.959882+0000","last_change":"2024-05-30T17:04:52.976220+0000","last_active":"2024-05-30T17:04:53.959882+0000","last_peered":"2024-05-30T17:04:53.959882+0000","last_clean":"2024-05-30T17:04:53.959882+0000","last_became_active":"2024-05-30T17:04:52.974987+0000","last_became_peered":"2024-05-30T17:04:52.974987+0000","last_unstale":"2024-05-30T17:04:53.959882+0000","last_undegraded":"2024-05-30T17:04:53.959882+0000","last_fullsized":"2024-05-30T17:04:53.959882+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:15.443712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120349+0000","last_change":"2024-05-30T17:04:51.950294+0000","last_active":"2024-05-30T17:04:54.120349+0000","last_peered":"2024-05-30T17:04:54.120349+0000","last_clean":"2024-05-30T17:04:54.120349+0000","last_became_active":"2024-05-30T17:04:51.949704+0000","last_became_peered":"2024-05-30T17:04:51.949704+0000","last_unstale":"2024-05-30T17:04:54.120349+0000","last_undegraded":"2024-05-30T17:04:54.120349+0000","last_fullsized":"2024-05-30T17:04:54.120349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:46.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953390+0000","last_change":"2024-05-30T17:04:52.989374+0000","last_active":"2024-05-30T17:04:53.953390+0000","last_peered":"2024-05-30T17:04:53.953390+0000","last_clean":"2024-05-30T17:04:53.953390+0000","last_became_active":"2024-05-30T17:04:52.989083+0000","last_became_peered":"2024-05-30T17:04:52.989083+0000","last_unstale":"2024-05-30T17:04:53.953390+0000","last_undegraded":"2024-05-30T17:04:53.953390+0000","last_fullsized":"2024-05-30T17:04:53.953390+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:35.524809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961842+0000","last_change":"2024-05-30T17:04:51.942441+0000","last_active":"2024-05-30T17:04:53.961842+0000","last_peered":"2024-05-30T17:04:53.961842+0000","last_clean":"2024-05-30T17:04:53.961842+0000","last_became_active":"2024-05-30T17:04:51.942121+0000","last_became_peered":"2024-05-30T17:04:51.942121+0000","last_unstale":"2024-05-30T17:04:53.961842+0000","last_undegraded":"2024-05-30T17:04:53.961842+0000","last_fullsized":"2024-05-30T17:04:53.961842+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:27.082196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120288+0000","last_change":"2024-05-30T17:04:52.994569+0000","last_active":"2024-05-30T17:04:54.120288+0000","last_peered":"2024-05-30T17:04:54.120288+0000","last_clean":"2024-05-30T17:04:54.120288+0000","last_became_active":"2024-05-30T17:04:52.994290+0000","last_became_peered":"2024-05-30T17:04:52.994290+0000","last_unstale":"2024-05-30T17:04:54.120288+0000","last_undegraded":"2024-05-30T17:04:54.120288+0000","last_fullsized":"2024-05-30T17:04:54.120288+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.958530+0000","last_change":"2024-05-30T17:04:51.945428+0000","last_active":"2024-05-30T17:04:53.958530+0000","last_peered":"2024-05-30T17:04:53.958530+0000","last_clean":"2024-05-30T17:04:53.958530+0000","last_became_active":"2024-05-30T17:04:51.945142+0000","last_became_peered":"2024-05-30T17:04:51.945142+0000","last_unstale":"2024-05-30T17:04:53.958530+0000","last_undegraded":"2024-05-30T17:04:53.958530+0000","last_fullsized":"2024-05-30T17:04:53.958530+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:25:46.375765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.071194+0000","last_change":"2024-05-30T17:04:52.993514+0000","last_active":"2024-05-30T17:04:54.071194+0000","last_peered":"2024-05-30T17:04:54.071194+0000","last_clean":"2024-05-30T17:04:54.071194+0000","last_became_active":"2024-05-30T17:04:52.988660+0000","last_became_peered":"2024-05-30T17:04:52.988660+0000","last_unstale":"2024-05-30T17:04:54.071194+0000","last_undegraded":"2024-05-30T17:04:54.071194+0000","last_fullsized":"2024-05-30T17:04:54.071194+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.958417+0000","last_change":"2024-05-30T17:04:51.943410+0000","last_active":"2024-05-30T17:04:53.958417+0000","last_peered":"2024-05-30T17:04:53.958417+0000","last_clean":"2024-05-30T17:04:53.958417+0000","last_became_active":"2024-05-30T17:04:51.943085+0000","last_became_peered":"2024-05-30T17:04:51.943085+0000","last_unstale":"2024-05-30T17:04:53.958417+0000","last_undegraded":"2024-05-30T17:04:53.958417+0000","last_fullsized":"2024-05-30T17:04:53.958417+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:18.313092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961619+0000","last_change":"2024-05-30T17:04:52.992939+0000","last_active":"2024-05-30T17:04:53.961619+0000","last_peered":"2024-05-30T17:04:53.961619+0000","last_clean":"2024-05-30T17:04:53.961619+0000","last_became_active":"2024-05-30T17:04:52.989409+0000","last_became_peered":"2024-05-30T17:04:52.989409+0000","last_unstale":"2024-05-30T17:04:53.961619+0000","last_undegraded":"2024-05-30T17:04:53.961619+0000","last_fullsized":"2024-05-30T17:04:53.961619+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:41.892452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953068+0000","last_change":"2024-05-30T17:04:51.948194+0000","last_active":"2024-05-30T17:04:53.953068+0000","last_peered":"2024-05-30T17:04:53.953068+0000","last_clean":"2024-05-30T17:04:53.953068+0000","last_became_active":"2024-05-30T17:04:51.947919+0000","last_became_peered":"2024-05-30T17:04:51.947919+0000","last_unstale":"2024-05-30T17:04:53.953068+0000","last_undegraded":"2024-05-30T17:04:53.953068+0000","last_fullsized":"2024-05-30T17:04:53.953068+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:58.823357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119648+0000","last_change":"2024-05-30T17:04:52.981350+0000","last_active":"2024-05-30T17:04:54.119648+0000","last_peered":"2024-05-30T17:04:54.119648+0000","last_clean":"2024-05-30T17:04:54.119648+0000","last_became_active":"2024-05-30T17:04:52.978531+0000","last_became_peered":"2024-05-30T17:04:52.978531+0000","last_unstale":"2024-05-30T17:04:54.119648+0000","last_undegraded":"2024-05-30T17:04:54.119648+0000","last_fullsized":"2024-05-30T17:04:54.119648+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.952968+0000","last_change":"2024-05-30T17:04:51.950726+0000","last_active":"2024-05-30T17:04:53.952968+0000","last_peered":"2024-05-30T17:04:53.952968+0000","last_clean":"2024-05-30T17:04:53.952968+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:04:53.952968+0000","last_undegraded":"2024-05-30T17:04:53.952968+0000","last_fullsized":"2024-05-30T17:04:53.952968+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:17.049750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998962+0000","last_change":"2024-05-30T17:04:52.989896+0000","last_active":"2024-05-30T17:04:53.998962+0000","last_peered":"2024-05-30T17:04:53.998962+0000","last_clean":"2024-05-30T17:04:53.998962+0000","last_became_active":"2024-05-30T17:04:52.989566+0000","last_became_peered":"2024-05-30T17:04:52.989566+0000","last_unstale":"2024-05-30T17:04:53.998962+0000","last_undegraded":"2024-05-30T17:04:53.998962+0000","last_fullsized":"2024-05-30T17:04:53.998962+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:00:44.437259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119958+0000","last_change":"2024-05-30T17:04:51.951054+0000","last_active":"2024-05-30T17:04:54.119958+0000","last_peered":"2024-05-30T17:04:54.119958+0000","last_clean":"2024-05-30T17:04:54.119958+0000","last_became_active":"2024-05-30T17:04:51.950364+0000","last_became_peered":"2024-05-30T17:04:51.950364+0000","last_unstale":"2024-05-30T17:04:54.119958+0000","last_undegraded":"2024-05-30T17:04:54.119958+0000","last_fullsized":"2024-05-30T17:04:54.119958+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:11.010697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.962353+0000","last_change":"2024-05-30T17:04:52.967099+0000","last_active":"2024-05-30T17:04:53.962353+0000","last_peered":"2024-05-30T17:04:53.962353+0000","last_clean":"2024-05-30T17:04:53.962353+0000","last_became_active":"2024-05-30T17:04:52.966035+0000","last_became_peered":"2024-05-30T17:04:52.966035+0000","last_unstale":"2024-05-30T17:04:53.962353+0000","last_undegraded":"2024-05-30T17:04:53.962353+0000","last_fullsized":"2024-05-30T17:04:53.962353+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:24.565166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.072251+0000","last_change":"2024-05-30T17:04:51.949867+0000","last_active":"2024-05-30T17:04:54.072251+0000","last_peered":"2024-05-30T17:04:54.072251+0000","last_clean":"2024-05-30T17:04:54.072251+0000","last_became_active":"2024-05-30T17:04:51.949575+0000","last_became_peered":"2024-05-30T17:04:51.949575+0000","last_unstale":"2024-05-30T17:04:54.072251+0000","last_undegraded":"2024-05-30T17:04:54.072251+0000","last_fullsized":"2024-05-30T17:04:54.072251+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:58:11.353946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.072207+0000","last_change":"2024-05-30T17:04:52.998729+0000","last_active":"2024-05-30T17:04:54.072207+0000","last_peered":"2024-05-30T17:04:54.072207+0000","last_clean":"2024-05-30T17:04:54.072207+0000","last_became_active":"2024-05-30T17:04:52.997726+0000","last_became_peered":"2024-05-30T17:04:52.997726+0000","last_unstale":"2024-05-30T17:04:54.072207+0000","last_undegraded":"2024-05-30T17:04:54.072207+0000","last_fullsized":"2024-05-30T17:04:54.072207+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:32:14.601932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.001835+0000","last_change":"2024-05-30T17:04:51.949751+0000","last_active":"2024-05-30T17:04:54.001835+0000","last_peered":"2024-05-30T17:04:54.001835+0000","last_clean":"2024-05-30T17:04:54.001835+0000","last_became_active":"2024-05-30T17:04:51.949295+0000","last_became_peered":"2024-05-30T17:04:51.949295+0000","last_unstale":"2024-05-30T17:04:54.001835+0000","last_undegraded":"2024-05-30T17:04:54.001835+0000","last_fullsized":"2024-05-30T17:04:54.001835+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:41.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120054+0000","last_change":"2024-05-30T17:04:52.985036+0000","last_active":"2024-05-30T17:04:54.120054+0000","last_peered":"2024-05-30T17:04:54.120054+0000","last_clean":"2024-05-30T17:04:54.120054+0000","last_became_active":"2024-05-30T17:04:52.984675+0000","last_became_peered":"2024-05-30T17:04:52.984675+0000","last_unstale":"2024-05-30T17:04:54.120054+0000","last_undegraded":"2024-05-30T17:04:54.120054+0000","last_fullsized":"2024-05-30T17:04:54.120054+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120280+0000","last_change":"2024-05-30T17:04:51.952967+0000","last_active":"2024-05-30T17:04:54.120280+0000","last_peered":"2024-05-30T17:04:54.120280+0000","last_clean":"2024-05-30T17:04:54.120280+0000","last_became_active":"2024-05-30T17:04:51.952631+0000","last_became_peered":"2024-05-30T17:04:51.952631+0000","last_unstale":"2024-05-30T17:04:54.120280+0000","last_undegraded":"2024-05-30T17:04:54.120280+0000","last_fullsized":"2024-05-30T17:04:54.120280+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:23.747059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.961783+0000","last_change":"2024-05-30T17:04:52.990963+0000","last_active":"2024-05-30T17:04:53.961783+0000","last_peered":"2024-05-30T17:04:53.961783+0000","last_clean":"2024-05-30T17:04:53.961783+0000","last_became_active":"2024-05-30T17:04:52.990476+0000","last_became_peered":"2024-05-30T17:04:52.990476+0000","last_unstale":"2024-05-30T17:04:53.961783+0000","last_undegraded":"2024-05-30T17:04:53.961783+0000","last_fullsized":"2024-05-30T17:04:53.961783+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:59.838344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120391+0000","last_change":"2024-05-30T17:04:51.933352+0000","last_active":"2024-05-30T17:04:54.120391+0000","last_peered":"2024-05-30T17:04:54.120391+0000","last_clean":"2024-05-30T17:04:54.120391+0000","last_became_active":"2024-05-30T17:04:51.933061+0000","last_became_peered":"2024-05-30T17:04:51.933061+0000","last_unstale":"2024-05-30T17:04:54.120391+0000","last_undegraded":"2024-05-30T17:04:54.120391+0000","last_fullsized":"2024-05-30T17:04:54.120391+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:20.242562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999136+0000","last_change":"2024-05-30T17:04:52.982699+0000","last_active":"2024-05-30T17:04:53.999136+0000","last_peered":"2024-05-30T17:04:53.999136+0000","last_clean":"2024-05-30T17:04:53.999136+0000","last_became_active":"2024-05-30T17:04:52.981542+0000","last_became_peered":"2024-05-30T17:04:52.981542+0000","last_unstale":"2024-05-30T17:04:53.999136+0000","last_undegraded":"2024-05-30T17:04:53.999136+0000","last_fullsized":"2024-05-30T17:04:53.999136+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:12.025684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.119855+0000","last_change":"2024-05-30T17:04:51.947181+0000","last_active":"2024-05-30T17:04:54.119855+0000","last_peered":"2024-05-30T17:04:54.119855+0000","last_clean":"2024-05-30T17:04:54.119855+0000","last_became_active":"2024-05-30T17:04:51.946882+0000","last_became_peered":"2024-05-30T17:04:51.946882+0000","last_unstale":"2024-05-30T17:04:54.119855+0000","last_undegraded":"2024-05-30T17:04:54.119855+0000","last_fullsized":"2024-05-30T17:04:54.119855+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:47:52.431592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.959012+0000","last_change":"2024-05-30T17:04:52.992399+0000","last_active":"2024-05-30T17:04:53.959012+0000","last_peered":"2024-05-30T17:04:53.959012+0000","last_clean":"2024-05-30T17:04:53.959012+0000","last_became_active":"2024-05-30T17:04:52.992040+0000","last_became_peered":"2024-05-30T17:04:52.992040+0000","last_unstale":"2024-05-30T17:04:53.959012+0000","last_undegraded":"2024-05-30T17:04:53.959012+0000","last_fullsized":"2024-05-30T17:04:53.959012+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:16.335923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998986+0000","last_change":"2024-05-30T17:04:51.952089+0000","last_active":"2024-05-30T17:04:53.998986+0000","last_peered":"2024-05-30T17:04:53.998986+0000","last_clean":"2024-05-30T17:04:53.998986+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:04:53.998986+0000","last_undegraded":"2024-05-30T17:04:53.998986+0000","last_fullsized":"2024-05-30T17:04:53.998986+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:03:20.313618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.953083+0000","last_change":"2024-05-30T17:04:52.988572+0000","last_active":"2024-05-30T17:04:53.953083+0000","last_peered":"2024-05-30T17:04:53.953083+0000","last_clean":"2024-05-30T17:04:53.953083+0000","last_became_active":"2024-05-30T17:04:52.987684+0000","last_became_peered":"2024-05-30T17:04:52.987684+0000","last_unstale":"2024-05-30T17:04:53.953083+0000","last_undegraded":"2024-05-30T17:04:53.953083+0000","last_fullsized":"2024-05-30T17:04:53.953083+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:26:52.920204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.952762+0000","last_change":"2024-05-30T17:04:51.949334+0000","last_active":"2024-05-30T17:04:53.952762+0000","last_peered":"2024-05-30T17:04:53.952762+0000","last_clean":"2024-05-30T17:04:53.952762+0000","last_became_active":"2024-05-30T17:04:51.948995+0000","last_became_peered":"2024-05-30T17:04:51.948995+0000","last_unstale":"2024-05-30T17:04:53.952762+0000","last_undegraded":"2024-05-30T17:04:53.952762+0000","last_fullsized":"2024-05-30T17:04:53.952762+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:05.331584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120171+0000","last_change":"2024-05-30T17:04:52.992411+0000","last_active":"2024-05-30T17:04:54.120171+0000","last_peered":"2024-05-30T17:04:54.120171+0000","last_clean":"2024-05-30T17:04:54.120171+0000","last_became_active":"2024-05-30T17:04:52.991997+0000","last_became_peered":"2024-05-30T17:04:52.991997+0000","last_unstale":"2024-05-30T17:04:54.120171+0000","last_undegraded":"2024-05-30T17:04:54.120171+0000","last_fullsized":"2024-05-30T17:04:54.120171+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:54.120061+0000","last_change":"2024-05-30T17:04:51.946511+0000","last_active":"2024-05-30T17:04:54.120061+0000","last_peered":"2024-05-30T17:04:54.120061+0000","last_clean":"2024-05-30T17:04:54.120061+0000","last_became_active":"2024-05-30T17:04:51.946173+0000","last_became_peered":"2024-05-30T17:04:51.946173+0000","last_unstale":"2024-05-30T17:04:54.120061+0000","last_undegraded":"2024-05-30T17:04:54.120061+0000","last_fullsized":"2024-05-30T17:04:54.120061+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:08.206744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998869+0000","last_change":"2024-05-30T17:04:52.992639+0000","last_active":"2024-05-30T17:04:53.998869+0000","last_peered":"2024-05-30T17:04:53.998869+0000","last_clean":"2024-05-30T17:04:53.998869+0000","last_became_active":"2024-05-30T17:04:52.992202+0000","last_became_peered":"2024-05-30T17:04:52.992202+0000","last_unstale":"2024-05-30T17:04:53.998869+0000","last_undegraded":"2024-05-30T17:04:53.998869+0000","last_fullsized":"2024-05-30T17:04:53.998869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:34.869318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.999432+0000","last_change":"2024-05-30T17:04:51.951966+0000","last_active":"2024-05-30T17:04:53.999432+0000","last_peered":"2024-05-30T17:04:53.999432+0000","last_clean":"2024-05-30T17:04:53.999432+0000","last_became_active":"2024-05-30T17:04:51.951484+0000","last_became_peered":"2024-05-30T17:04:51.951484+0000","last_unstale":"2024-05-30T17:04:53.999432+0000","last_undegraded":"2024-05-30T17:04:53.999432+0000","last_fullsized":"2024-05-30T17:04:53.999432+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:41.627294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":382,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-30T17:04:53.998290+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:04:53.998290+0000","last_peered":"2024-05-30T17:04:53.998290+0000","last_clean":"2024-05-30T17:04:53.998290+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:04:53.998290+0000","last_undegraded":"2024-05-30T17:04:53.998290+0000","last_fullsized":"2024-05-30T17:04:53.998290+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509698,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118812,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253028,"statfs":{"total":96636764160,"available":96515100672,"internally_reserved":0,"allocated":1650688,"data_stored":725151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":119994822},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509698,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149328,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222512,"statfs":{"total":96636764160,"available":96483852288,"internally_reserved":0,"allocated":1642496,"data_stored":723609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118804,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253036,"statfs":{"total":96636764160,"available":96515108864,"internally_reserved":0,"allocated":1646592,"data_stored":723699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509699,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153496,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218344,"statfs":{"total":96636764160,"available":96479584256,"internally_reserved":0,"allocated":1646592,"data_stored":724155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509698,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149328,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222512,"statfs":{"total":96636764160,"available":96483852288,"internally_reserved":0,"allocated":1638400,"data_stored":723539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542401,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119400,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252440,"statfs":{"total":96636764160,"available":96514498560,"internally_reserved":0,"allocated":2256896,"data_stored":1316157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542402,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68808,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303032,"statfs":{"total":96636764160,"available":96566304768,"internally_reserved":0,"allocated":1634304,"data_stored":717239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542402,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145736,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226104,"statfs":{"total":96636764160,"available":96487530496,"internally_reserved":0,"allocated":2220032,"data_stored":1307495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:04:57.943 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T17:04:57.943 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-30T17:04:58.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.205+0000 7f7ad2600640 1 Processor -- start 2024-05-30T17:04:58.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.206+0000 7f7ad2600640 1 -- start start 2024-05-30T17:04:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.207+0000 7f7ad2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc15e870 0x7f7acc0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.207+0000 7f7ad2600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7acc0d14d0 0x7f7acc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.207+0000 7f7ad2600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7acc0cfd00 con 0x7f7acc0cf870 2024-05-30T17:04:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.207+0000 7f7ad2600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7acc0d2150 con 0x7f7acc0d14d0 2024-05-30T17:04:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.207+0000 7f7ad2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc15ec50 con 0x7f7acc15e870 2024-05-30T17:04:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.207+0000 7f7ad1c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7acc0cf870 0x7f7acc162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44086/0 (socket says 172.21.15.67:44086) 2024-05-30T17:04:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.207+0000 7f7ad1c00640 1 -- 172.21.15.67:0/3897141518 learned_addr learned my addr 172.21.15.67:0/3897141518 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7ad1200640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc15e870 0x7f7acc0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acbe00640 1 --2- 172.21.15.67:0/3897141518 >> v2:172.21.15.169:3300/0 conn(0x7f7acc0d14d0 0x7f7acc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 685620292 0 0) 0x7f7ac4001490 con 0x7f7acc0cf870 2024-05-30T17:04:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ab4003520 con 0x7f7acc0cf870 2024-05-30T17:04:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1972957123 0 0) 0x7f7ab4003520 con 0x7f7acc0cf870 2024-05-30T17:04:58.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc15e870 msgr2=0x7f7acc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc15e870 0x7f7acc0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 >> v2:172.21.15.169:3300/0 conn(0x7f7acc0d14d0 msgr2=0x7f7acc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 --2- 172.21.15.67:0/3897141518 >> v2:172.21.15.169:3300/0 conn(0x7f7acc0d14d0 0x7f7acc0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc0cff80 con 0x7f7acc0cf870 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f7ac4001bc0 con 0x7f7acc0cf870 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 >> v1:172.21.15.67:6789/0 conn(0x7f7acc0cf870 legacy=0x7f7acc162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.208+0000 7f7acb400640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc15e870 0x7f7ab40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7acb400640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4003f30 0x7f7ab4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7acb400640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7ab40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc0d2150 con 0x7f7ab4004d40 2024-05-30T17:04:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc15ec50 con 0x7f7ab4003f30 2024-05-30T17:04:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab400b9c0 con 0x7f7acc15e870 2024-05-30T17:04:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7ad1c00640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7ab40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7ad1200640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc15e870 0x7f7ab40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7acbe00640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4003f30 0x7f7ab4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7ad1c00640 1 -- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc15e870 msgr2=0x7f7ab40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7ad1c00640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc15e870 0x7f7ab40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7ad1c00640 1 -- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4003f30 msgr2=0x7f7ab4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7ad1c00640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4003f30 0x7f7ab4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.209+0000 7f7ad1c00640 1 -- 172.21.15.67:0/3897141518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab4003520 con 0x7f7ab4004d40 2024-05-30T17:04:58.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.210+0000 7f7ad1c00640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7ab40051c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400aa70 tx=0x7f7ab4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb08d659964d203 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:58.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.211+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400a690 con 0x7f7ab4004d40 2024-05-30T17:04:58.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.211+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac402f070 con 0x7f7ab4004d40 2024-05-30T17:04:58.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.211+0000 7f7acb400640 1 -- 172.21.15.67:0/3897141518 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4034470 con 0x7f7ab4004d40 2024-05-30T17:04:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.211+0000 7f7ad2600640 1 -- 172.21.15.67:0/3897141518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 msgr2=0x7f7ab40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.211+0000 7f7ad2600640 1 --2- 172.21.15.67:0/3897141518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7ab40051c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400aa70 tx=0x7f7ab4005830 comp rx=0 tx=0).stop 2024-05-30T17:04:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.211+0000 7f7ad1c00640 1 -- 172.21.15.67:0/3897141518 reap_dead start 2024-05-30T17:04:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.211+0000 7f7ad2600640 1 -- 172.21.15.67:0/3897141518 shutdown_connections 2024-05-30T17:04:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.211+0000 7f7ad2600640 1 -- 172.21.15.67:0/3897141518 >> 172.21.15.67:0/3897141518 conn(0x7f7acc08bfc0 msgr2=0x7f7acc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.211+0000 7f7ad2600640 1 -- 172.21.15.67:0/3897141518 shutdown_connections 2024-05-30T17:04:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.212+0000 7f7ad2600640 1 -- 172.21.15.67:0/3897141518 wait complete. 2024-05-30T17:04:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.213+0000 7f7ad2600640 1 Processor -- start 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.213+0000 7f7ad2600640 1 -- start start 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.214+0000 7f7ad2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7acc0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.214+0000 7f7ad2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc159940 0x7f7acc0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.214+0000 7f7ad2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc159dc0 0x7f7acc0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.214+0000 7f7ad2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc0d0820 con 0x7f7ab4004d40 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.214+0000 7f7ad2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc0d0540 con 0x7f7acc159940 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.214+0000 7f7ad2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc0d4fb0 con 0x7f7acc159dc0 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7acc0cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7acc0cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54964/0 (socket says 172.21.15.67:54964) 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1200640 1 -- 172.21.15.67:0/1652609728 learned_addr learned my addr 172.21.15.67:0/1652609728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7acbe00640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc159940 0x7f7acc0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1c00640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc159dc0 0x7f7acc0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1200640 1 -- 172.21.15.67:0/1652609728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc159dc0 msgr2=0x7f7acc0b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1200640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc159dc0 0x7f7acc0b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1200640 1 -- 172.21.15.67:0/1652609728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc159940 msgr2=0x7f7acc0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1200640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc159940 0x7f7acc0b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1200640 1 -- 172.21.15.67:0/1652609728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc0cc340 con 0x7f7ab4004d40 2024-05-30T17:04:58.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7acbe00640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc159940 0x7f7acc0b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:58.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.215+0000 7f7ad1200640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7acc0cd3e0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0011600 tx=0x7f7ac0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:58.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.216+0000 7f7ac8c00640 1 -- 172.21.15.67:0/1652609728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000f920 con 0x7f7ab4004d40 2024-05-30T17:04:58.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.216+0000 7f7ac8c00640 1 -- 172.21.15.67:0/1652609728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac000fac0 con 0x7f7ab4004d40 2024-05-30T17:04:58.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.216+0000 7f7ac8c00640 1 -- 172.21.15.67:0/1652609728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000fe70 con 0x7f7ab4004d40 2024-05-30T17:04:58.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.216+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7acc0cc620 con 0x7f7ab4004d40 2024-05-30T17:04:58.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.216+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7acc0ccbe0 con 0x7f7ab4004d40 2024-05-30T17:04:58.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.218+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7acc0689e0 con 0x7f7ab4004d40 2024-05-30T17:04:58.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.219+0000 7f7ac8c00640 1 -- 172.21.15.67:0/1652609728 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f7ac0015460 con 0x7f7ab4004d40 2024-05-30T17:04:58.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.220+0000 7f7ac8c00640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7aac06e540 0x7f7aac070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.220+0000 7f7ac8c00640 1 -- 172.21.15.67:0/1652609728 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f7ac008b4c0 con 0x7f7ab4004d40 2024-05-30T17:04:58.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.220+0000 7f7acbe00640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7aac06e540 0x7f7aac070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.221+0000 7f7acbe00640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7aac06e540 0x7f7aac070a00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0b5a40 tx=0x7f7ab8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:58.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.225+0000 7f7ac8c00640 1 -- 172.21.15.67:0/1652609728 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ac0019030 con 0x7f7ab4004d40 2024-05-30T17:04:58.347 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-05-30T17:04:58.347+0000 7f7524c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-30T17:04:58.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.503+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7acc0d3600 con 0x7f7ab4004d40 2024-05-30T17:04:58.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.508+0000 7f7ac8c00640 1 -- 172.21.15.67:0/1652609728 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f7ac005b350 con 0x7f7ab4004d40 2024-05-30T17:04:58.509 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-05-30T17:04:58.509 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCCrFhm0ghMOxAAaD5TuhSV62odnIS/dKj0tg== 2024-05-30T17:04:58.509 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-05-30T17:04:58.509 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-05-30T17:04:58.509 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-30T17:04:58.509 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-05-30T17:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7aac06e540 msgr2=0x7f7aac070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7aac06e540 0x7f7aac070a00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0b5a40 tx=0x7f7ab8008040 comp rx=0 tx=0).stop 2024-05-30T17:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 msgr2=0x7f7acc0cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7acc0cd3e0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0011600 tx=0x7f7ac0011ac0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 shutdown_connections 2024-05-30T17:04:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7aac06e540 0x7f7aac070a00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7acc159dc0 0x7f7acc0b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7acc159940 0x7f7acc0b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 --2- 172.21.15.67:0/1652609728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ab4004d40 0x7f7acc0cd3e0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 >> 172.21.15.67:0/1652609728 conn(0x7f7acc08bfc0 msgr2=0x7f7acc0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 shutdown_connections 2024-05-30T17:04:58.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.511+0000 7f7ad2600640 1 -- 172.21.15.67:0/1652609728 wait complete. 2024-05-30T17:04:58.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:04:58.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.730+0000 7f2f4d400640 1 Processor -- start 2024-05-30T17:04:58.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.731+0000 7f2f4d400640 1 -- start start 2024-05-30T17:04:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.732+0000 7f2f4d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 0x7f2f4815f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.732+0000 7f2f4d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2f480cebf0 0x7f2f480cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.732+0000 7f2f4d400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f480d2bd0 con 0x7f2f480d2610 2024-05-30T17:04:58.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.732+0000 7f2f4d400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2f480d37e0 con 0x7f2f480cebf0 2024-05-30T17:04:58.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.732+0000 7f2f4d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2f480d2eb0 con 0x7f2f480b5f90 2024-05-30T17:04:58.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.732+0000 7f2f4ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2f480cebf0 0x7f2f480cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.732+0000 7f2f4ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2f480cebf0 0x7f2f480cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54970/0 (socket says 172.21.15.67:54970) 2024-05-30T17:04:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.732+0000 7f2f4ca00640 1 -- 172.21.15.67:0/129845506 learned_addr learned my addr 172.21.15.67:0/129845506 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.733+0000 7f2f47e00640 1 --2- 172.21.15.67:0/129845506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 0x7f2f4815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.733+0000 7f2f4ca00640 1 -- 172.21.15.67:0/129845506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 msgr2=0x7f2f4815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.733+0000 7f2f4ca00640 1 --2- 172.21.15.67:0/129845506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 0x7f2f4815f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.733+0000 7f2f4ca00640 1 -- 172.21.15.67:0/129845506 >> v1:172.21.15.169:6789/0 conn(0x7f2f480d2610 legacy=0x7f2f480b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:58.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.733+0000 7f2f4ca00640 1 -- 172.21.15.67:0/129845506 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f480d3190 con 0x7f2f480cebf0 2024-05-30T17:04:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.734+0000 7f2f4ca00640 1 --2- 172.21.15.67:0/129845506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480cebf0 0x7f2f480cf080 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009e50 tx=0x7f2f4002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9632910fcadb843c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.734+0000 7f2f46a00640 1 -- 172.21.15.67:0/129845506 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40036070 con 0x7f2f480cebf0 2024-05-30T17:04:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.734+0000 7f2f46a00640 1 -- 172.21.15.67:0/129845506 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f40031600 con 0x7f2f480cebf0 2024-05-30T17:04:58.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.734+0000 7f2f46a00640 1 -- 172.21.15.67:0/129845506 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f400319d0 con 0x7f2f480cebf0 2024-05-30T17:04:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.735+0000 7f2f4d400640 1 -- 172.21.15.67:0/129845506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480cebf0 msgr2=0x7f2f480cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.735+0000 7f2f4d400640 1 --2- 172.21.15.67:0/129845506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480cebf0 0x7f2f480cf080 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009e50 tx=0x7f2f4002a6c0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.735+0000 7f2f4d400640 1 -- 172.21.15.67:0/129845506 shutdown_connections 2024-05-30T17:04:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.735+0000 7f2f4d400640 1 --2- 172.21.15.67:0/129845506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480cebf0 0x7f2f480cf080 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.735+0000 7f2f4d400640 1 --2- 172.21.15.67:0/129845506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 0x7f2f4815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.735+0000 7f2f4d400640 1 -- 172.21.15.67:0/129845506 >> 172.21.15.67:0/129845506 conn(0x7f2f4808bfc0 msgr2=0x7f2f480d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.735+0000 7f2f4d400640 1 -- 172.21.15.67:0/129845506 shutdown_connections 2024-05-30T17:04:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.735+0000 7f2f4d400640 1 -- 172.21.15.67:0/129845506 wait complete. 2024-05-30T17:04:58.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.737+0000 7f2f4d400640 1 Processor -- start 2024-05-30T17:04:58.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.737+0000 7f2f4d400640 1 -- start start 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 0x7f2f480c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f480cebf0 0x7f2f480cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480d2610 0x7f2f481f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4d400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4815eac0 con 0x7f2f480d2610 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4d400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f480d4ee0 con 0x7f2f480cebf0 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2f480cf5c0 con 0x7f2f480b5f90 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480d2610 0x7f2f481f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480d2610 0x7f2f481f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54974/0 (socket says 172.21.15.67:54974) 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4ca00640 1 -- 172.21.15.67:0/3476089930 learned_addr learned my addr 172.21.15.67:0/3476089930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4ca00640 1 -- 172.21.15.67:0/3476089930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 msgr2=0x7f2f480c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f47e00640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 0x7f2f480c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.738+0000 7f2f4ca00640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 0x7f2f480c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.739+0000 7f2f4ca00640 1 -- 172.21.15.67:0/3476089930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f480cebf0 msgr2=0x7f2f480cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.739+0000 7f2f4ca00640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f480cebf0 0x7f2f480cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.739+0000 7f2f4ca00640 1 -- 172.21.15.67:0/3476089930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f480c8500 con 0x7f2f480d2610 2024-05-30T17:04:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.739+0000 7f2f4ca00640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480d2610 0x7f2f481f7b50 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2f40034040 tx=0x7f2f40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.739+0000 7f2f3fe00640 1 -- 172.21.15.67:0/3476089930 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40041020 con 0x7f2f480d2610 2024-05-30T17:04:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.739+0000 7f2f3fe00640 1 -- 172.21.15.67:0/3476089930 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f400077a0 con 0x7f2f480d2610 2024-05-30T17:04:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.739+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f480c87e0 con 0x7f2f480d2610 2024-05-30T17:04:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.739+0000 7f2f3fe00640 1 -- 172.21.15.67:0/3476089930 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40036040 con 0x7f2f480d2610 2024-05-30T17:04:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.739+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f480c8da0 con 0x7f2f480d2610 2024-05-30T17:04:58.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.740+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f480689e0 con 0x7f2f480d2610 2024-05-30T17:04:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.742+0000 7f2f3fe00640 1 -- 172.21.15.67:0/3476089930 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f2f40044020 con 0x7f2f480d2610 2024-05-30T17:04:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.743+0000 7f2f3fe00640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f2406e510 0x7f2f240709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.743+0000 7f2f3fe00640 1 -- 172.21.15.67:0/3476089930 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f2f400ab7d0 con 0x7f2f480d2610 2024-05-30T17:04:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.743+0000 7f2f47e00640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f2406e510 0x7f2f240709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:58.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.744+0000 7f2f47e00640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f2406e510 0x7f2f240709d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:58.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.747+0000 7f2f3fe00640 1 -- 172.21.15.67:0/3476089930 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f4004b030 con 0x7f2f480d2610 2024-05-30T17:04:58.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.983+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f480b6e40 con 0x7f2f480d2610 2024-05-30T17:04:58.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.984+0000 7f2f3fe00640 1 -- 172.21.15.67:0/3476089930 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5186 (secure 0 0 0) 0x7f2f4007b700 con 0x7f2f480d2610 2024-05-30T17:04:58.985 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:04:58.985 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2461528462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2461528462},{"type":"v1","addr":"172.21.15.67:6837","nonce":2461528462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/4229871041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4229871041},{"type":"v1","addr":"172.21.15.169:6833","nonce":4229871041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/217648840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":217648840},{"type":"v1","addr":"172.21.15.67:6835","nonce":217648840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:04:52.409278+0000","modified":"2024-05-30T17:04:55.498697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6937":{"gid":6937,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2504652568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2504652568},{"type":"v1","addr":"172.21.15.169:6835","nonce":2504652568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6937,"qdb_cluster":[6937]},"id":9}]} 2024-05-30T17:04:58.985 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 82 2024-05-30T17:04:58.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f2406e510 msgr2=0x7f2f240709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f2406e510 0x7f2f240709d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38008040 comp rx=0 tx=0).stop 2024-05-30T17:04:58.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480d2610 msgr2=0x7f2f481f7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:58.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480d2610 0x7f2f481f7b50 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2f40034040 tx=0x7f2f40004a40 comp rx=0 tx=0).stop 2024-05-30T17:04:58.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 shutdown_connections 2024-05-30T17:04:58.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2f2406e510 0x7f2f240709d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f480d2610 0x7f2f481f7b50 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f480cebf0 0x7f2f480cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 --2- 172.21.15.67:0/3476089930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f480b5f90 0x7f2f480c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:58.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 >> 172.21.15.67:0/3476089930 conn(0x7f2f4808bfc0 msgr2=0x7f2f480b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:58.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 shutdown_connections 2024-05-30T17:04:58.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:58.987+0000 7f2f4d400640 1 -- 172.21.15.67:0/3476089930 wait complete. 2024-05-30T17:04:59.409 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-30T17:04:52.409278+0000', 'modified': '2024-05-30T17:04:55.498697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6937}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6937': {'gid': 6937, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/2504652568', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2504652568}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2504652568}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6937, 'qdb_cluster': [6937]} 2024-05-30T17:04:59.409 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-30T17:04:59.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-30T17:04:59.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.628+0000 7f0a9e600640 1 Processor -- start 2024-05-30T17:04:59.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.629+0000 7f0a9e600640 1 -- start start 2024-05-30T17:04:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.630+0000 7f0a9e600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0a9806f860 0x7f0a980b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.630+0000 7f0a9e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0a980b3690 0x7f0a980d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.630+0000 7f0a9e600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a9806ed10 con 0x7f0a980d09e0 2024-05-30T17:04:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.630+0000 7f0a9e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f0a980d0df0 con 0x7f0a980b3690 2024-05-30T17:04:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.630+0000 7f0a9e600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9815cb00 con 0x7f0a9806f860 2024-05-30T17:04:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.630+0000 7f0a8fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0a980b3690 0x7f0a980d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.630+0000 7f0a8fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0a980b3690 0x7f0a980d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54982/0 (socket says 172.21.15.67:54982) 2024-05-30T17:04:59.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.630+0000 7f0a8fe00640 1 -- 172.21.15.67:0/523375869 learned_addr learned my addr 172.21.15.67:0/523375869 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.631+0000 7f0a8fe00640 1 -- 172.21.15.67:0/523375869 >> v2:172.21.15.169:3300/0 conn(0x7f0a9806f860 msgr2=0x7f0a980b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.631+0000 7f0a97400640 1 --2- 172.21.15.67:0/523375869 >> v2:172.21.15.169:3300/0 conn(0x7f0a9806f860 0x7f0a980b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.631+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/523375869 >> v2:172.21.15.169:3300/0 conn(0x7f0a9806f860 0x7f0a980b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.631+0000 7f0a8fe00640 1 -- 172.21.15.67:0/523375869 >> v1:172.21.15.169:6789/0 conn(0x7f0a980d09e0 legacy=0x7f0a98160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.631+0000 7f0a8fe00640 1 -- 172.21.15.67:0/523375869 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980b3cc0 con 0x7f0a980b3690 2024-05-30T17:04:59.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.631+0000 7f0a97400640 1 --2- 172.21.15.67:0/523375869 >> v2:172.21.15.169:3300/0 conn(0x7f0a9806f860 0x7f0a980b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:59.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.632+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/523375869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980b3690 0x7f0a980d04a0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0a9806c8d0 tx=0x7f0a8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28e13885113434d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.632+0000 7f0a96a00640 1 -- 172.21.15.67:0/523375869 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80037070 con 0x7f0a980b3690 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.632+0000 7f0a96a00640 1 -- 172.21.15.67:0/523375869 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a80032440 con 0x7f0a980b3690 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.632+0000 7f0a96a00640 1 -- 172.21.15.67:0/523375869 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032810 con 0x7f0a980b3690 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.633+0000 7f0a9e600640 1 -- 172.21.15.67:0/523375869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980b3690 msgr2=0x7f0a980d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.633+0000 7f0a9e600640 1 --2- 172.21.15.67:0/523375869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980b3690 0x7f0a980d04a0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0a9806c8d0 tx=0x7f0a8002a230 comp rx=0 tx=0).stop 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.633+0000 7f0a9e600640 1 -- 172.21.15.67:0/523375869 shutdown_connections 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.633+0000 7f0a9e600640 1 --2- 172.21.15.67:0/523375869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980b3690 0x7f0a980d04a0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.633+0000 7f0a9e600640 1 --2- 172.21.15.67:0/523375869 >> v2:172.21.15.169:3300/0 conn(0x7f0a9806f860 0x7f0a980b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.633+0000 7f0a9e600640 1 -- 172.21.15.67:0/523375869 >> 172.21.15.67:0/523375869 conn(0x7f0a9806bae0 msgr2=0x7f0a9806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.633+0000 7f0a9e600640 1 -- 172.21.15.67:0/523375869 shutdown_connections 2024-05-30T17:04:59.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.633+0000 7f0a9e600640 1 -- 172.21.15.67:0/523375869 wait complete. 2024-05-30T17:04:59.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.635+0000 7f0a9e600640 1 Processor -- start 2024-05-30T17:04:59.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.635+0000 7f0a9e600640 1 -- start start 2024-05-30T17:04:59.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.636+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a9806f860 0x7f0a98157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.636+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a980b3690 0x7f0a980cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.636+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a980d09e0 0x7f0a980c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.636+0000 7f0a9e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980b4560 con 0x7f0a9806f860 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.636+0000 7f0a9e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980b4280 con 0x7f0a980d09e0 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.636+0000 7f0a9e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0a980b4840 con 0x7f0a980b3690 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.636+0000 7f0a97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a9806f860 0x7f0a98157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.636+0000 7f0a97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a9806f860 0x7f0a98157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54992/0 (socket says 172.21.15.67:54992) 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.636+0000 7f0a97400640 1 -- 172.21.15.67:0/1173333522 learned_addr learned my addr 172.21.15.67:0/1173333522 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.637+0000 7f0a97400640 1 -- 172.21.15.67:0/1173333522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a980b3690 msgr2=0x7f0a980cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.637+0000 7f0a97400640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a980b3690 0x7f0a980cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.637+0000 7f0a97400640 1 -- 172.21.15.67:0/1173333522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a980d09e0 msgr2=0x7f0a980c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.637+0000 7f0a97400640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a980d09e0 0x7f0a980c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.637+0000 7f0a97400640 1 -- 172.21.15.67:0/1173333522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980c9930 con 0x7f0a9806f860 2024-05-30T17:04:59.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.637+0000 7f0a97400640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a9806f860 0x7f0a98157c70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800c7f0 tx=0x7f0a8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.637+0000 7f0a8f400640 1 -- 172.21.15.67:0/1173333522 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88015070 con 0x7f0a9806f860 2024-05-30T17:04:59.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.637+0000 7f0a8f400640 1 -- 172.21.15.67:0/1173333522 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a88007820 con 0x7f0a9806f860 2024-05-30T17:04:59.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.637+0000 7f0a8f400640 1 -- 172.21.15.67:0/1173333522 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88007bd0 con 0x7f0a9806f860 2024-05-30T17:04:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.638+0000 7f0a9e600640 1 -- 172.21.15.67:0/1173333522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9815df00 con 0x7f0a9806f860 2024-05-30T17:04:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.638+0000 7f0a9e600640 1 -- 172.21.15.67:0/1173333522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9815e4c0 con 0x7f0a9806f860 2024-05-30T17:04:59.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.640+0000 7f0a8f400640 1 -- 172.21.15.67:0/1173333522 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f0a88018070 con 0x7f0a9806f860 2024-05-30T17:04:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.641+0000 7f0a8f400640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a7406e510 0x7f0a740709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.641+0000 7f0a8f400640 1 -- 172.21.15.67:0/1173333522 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f0a8808aa80 con 0x7f0a9806f860 2024-05-30T17:04:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.641+0000 7f0aa1443740 1 Processor -- start 2024-05-30T17:04:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.641+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a7406e510 0x7f0a740709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.642+0000 7f0aa1443740 1 -- start start 2024-05-30T17:04:59.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.642+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a7406e510 0x7f0a740709d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0a80034000 tx=0x7f0a8002c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d589666b0 0x563d58969e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d5896a370 0x563d589862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d589867f0 0x563d58986c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0aa1443740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563d5889fb10 con 0x563d589666b0 2024-05-30T17:04:59.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0aa1443740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563d588462b0 con 0x563d589867f0 2024-05-30T17:04:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0aa1443740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563d5889bad0 con 0x563d5896a370 2024-05-30T17:04:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0a97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d589666b0 0x563d58969e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0a97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d589666b0 0x563d58969e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55004/0 (socket says 172.21.15.67:55004) 2024-05-30T17:04:59.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0a97400640 1 -- 172.21.15.67:0/3425049360 learned_addr learned my addr 172.21.15.67:0/3425049360 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0a97400640 1 -- 172.21.15.67:0/3425049360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d5896a370 msgr2=0x563d589862b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0a97400640 1 --2- 172.21.15.67:0/3425049360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d5896a370 0x563d589862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0a97400640 1 -- 172.21.15.67:0/3425049360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d589867f0 msgr2=0x563d58986c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0a97400640 1 --2- 172.21.15.67:0/3425049360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d589867f0 0x563d58986c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.643+0000 7f0a97400640 1 -- 172.21.15.67:0/3425049360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d585b22d0 con 0x563d589666b0 2024-05-30T17:04:59.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.644+0000 7f0a97400640 1 --2- 172.21.15.67:0/3425049360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d589666b0 0x563d58969e30 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007410 tx=0x7f0a8805a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dd229f70a45a21 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.644+0000 7f0a87e00640 1 -- 172.21.15.67:0/3425049360 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8801f040 con 0x563d589666b0 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.644+0000 7f0a87e00640 1 -- 172.21.15.67:0/3425049360 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a88056880 con 0x563d589666b0 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.644+0000 7f0a87e00640 1 -- 172.21.15.67:0/3425049360 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8805e370 con 0x563d589666b0 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.644+0000 7f0aa1443740 1 -- 172.21.15.67:0/3425049360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d589666b0 msgr2=0x563d58969e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.644+0000 7f0aa1443740 1 --2- 172.21.15.67:0/3425049360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d589666b0 0x563d58969e30 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007410 tx=0x7f0a8805a350 comp rx=0 tx=0).stop 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.645+0000 7f0aa1443740 1 -- 172.21.15.67:0/3425049360 shutdown_connections 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.645+0000 7f0aa1443740 1 --2- 172.21.15.67:0/3425049360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d589867f0 0x563d58986c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.645+0000 7f0aa1443740 1 --2- 172.21.15.67:0/3425049360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d5896a370 0x563d589862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.645+0000 7f0aa1443740 1 --2- 172.21.15.67:0/3425049360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d589666b0 0x563d58969e30 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.645+0000 7f0aa1443740 1 -- 172.21.15.67:0/3425049360 >> 172.21.15.67:0/3425049360 conn(0x563d5893c690 msgr2=0x563d5893d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.645+0000 7f0aa1443740 1 -- 172.21.15.67:0/3425049360 shutdown_connections 2024-05-30T17:04:59.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.645+0000 7f0aa1443740 1 -- 172.21.15.67:0/3425049360 wait complete. 2024-05-30T17:04:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.645+0000 7f0aa1443740 1 -- start start 2024-05-30T17:04:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.646+0000 7f0aa1443740 1 Processor -- start 2024-05-30T17:04:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.646+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d58981df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.646+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 0x563d5896da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.646+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896cb80 0x563d5896afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.646+0000 7f0aa1443740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563d5893a4c0 con 0x563d5896a370 2024-05-30T17:04:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0aa1443740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563d58859570 con 0x563d5896cb80 2024-05-30T17:04:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0aa1443740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563d5893a7a0 con 0x563d589867f0 2024-05-30T17:04:59.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0aa1443740 10 client.0 _ll_register_callbacks cb 0x563d58827fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d58981df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d58981df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55020/0 (socket says 172.21.15.67:55020) 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a97400640 1 -- 172.21.15.67:0/4144377986 learned_addr learned my addr 172.21.15.67:0/4144377986 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a97400640 1 -- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 msgr2=0x563d5896da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a97400640 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 0x563d5896da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a97400640 1 -- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896cb80 msgr2=0x563d5896afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a97400640 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896cb80 0x563d5896afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a97400640 1 -- 172.21.15.67:0/4144377986 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5867c6b0 con 0x563d5896a370 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a97400640 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d58981df0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007410 tx=0x7f0a8800a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595eeafd67b149c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a84c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.647+0000 7f0a84c00640 1 -- 172.21.15.67:0/4144377986 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8801f040 con 0x563d5896a370 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0a84c00640 1 -- 172.21.15.67:0/4144377986 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f0a88059870 con 0x563d5896a370 2024-05-30T17:04:59.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0aa1443740 1 -- 172.21.15.67:0/4144377986 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a88056880 con 0x563d5896a370 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0a84c00640 1 -- 172.21.15.67:0/4144377986 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f0a88079b30 con 0x563d5896a370 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0aa1443740 10 client.9069 fetch_fsmap learned FSMap version 82 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0aa1443740 1 -- 172.21.15.67:0/4144377986 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x563d589818d0 con 0x563d5896a370 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0a84c00640 1 -- 172.21.15.67:0/4144377986 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a88056880 con 0x563d5896a370 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0a84c00640 1 -- 172.21.15.67:0/4144377986 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f0a88055f60 con 0x563d5896a370 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0aa1443740 10 client.9069 fetch_fsmap finished waiting for FSMap version 82 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0aa1443740 10 client.9069 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0aa1443740 20 client.9069 populate_metadata read hostname 'smithi067' 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0aa1443740 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] conn(0x563d589623d0 0x563d58994be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0aa1443740 4 client.9069 mds_command: new command op to 6937 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.648+0000 7f0aa1443740 1 -- 172.21.15.67:0/4144377986 --> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d588ad530 con 0x563d589623d0 2024-05-30T17:04:59.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.649+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] conn(0x563d589623d0 0x563d58994be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.650+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] conn(0x563d589623d0 0x563d58994be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb24b1953f7e5767 server_cookie=15ee19622dd1161f in_seq=0 out_seq=0 2024-05-30T17:04:59.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.650+0000 7f0a84c00640 10 client.9069 ms_handle_connect on v2:172.21.15.169:6834/2504652568 2024-05-30T17:04:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0a84c00640 1 -- 172.21.15.67:0/4144377986 <== mds.0 v2:172.21.15.169:6834/2504652568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d588ad530 con 0x563d589623d0 2024-05-30T17:04:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0a84c00640 10 client.9069 handle_command_reply: tid=0 2024-05-30T17:04:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 client.9069 shutdown 2024-05-30T17:04:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 -- 172.21.15.67:0/4144377986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 msgr2=0x563d58981df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d58981df0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007410 tx=0x7f0a8800a4d0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 -- 172.21.15.67:0/4144377986 shutdown_connections 2024-05-30T17:04:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] conn(0x563d589623d0 0x563d58994be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896cb80 0x563d5896afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 0x563d5896da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 --2- 172.21.15.67:0/4144377986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d58981df0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0a84c00640 0 client.9069 ms_handle_reset on v2:172.21.15.169:6834/2504652568 2024-05-30T17:04:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 -- 172.21.15.67:0/4144377986 >> 172.21.15.67:0/4144377986 conn(0x563d589666b0 msgr2=0x563d58934ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 -- 172.21.15.67:0/4144377986 shutdown_connections 2024-05-30T17:04:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.652+0000 7f0aa1443740 1 -- 172.21.15.67:0/4144377986 wait complete. 2024-05-30T17:04:59.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.653+0000 7f0aa1443740 20 client.9069 trim_cache size 0 max 16384 2024-05-30T17:04:59.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.672+0000 7f0aa1443740 1 Processor -- start 2024-05-30T17:04:59.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.672+0000 7f0aa1443740 1 -- start start 2024-05-30T17:04:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896a370 0x563d58964fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896cb80 0x563d589a9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 0x563d5897e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0aa1443740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980cfce0 con 0x563d5896cb80 2024-05-30T17:04:59.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0aa1443740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980cf040 con 0x563d5896a370 2024-05-30T17:04:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0aa1443740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0a980cf690 con 0x563d589867f0 2024-05-30T17:04:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896cb80 0x563d589a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896cb80 0x563d589a9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55036/0 (socket says 172.21.15.67:55036) 2024-05-30T17:04:59.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0a8fe00640 1 -- 172.21.15.67:0/2882500826 learned_addr learned my addr 172.21.15.67:0/2882500826 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0a8fe00640 1 -- 172.21.15.67:0/2882500826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 msgr2=0x563d5897e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/2882500826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 0x563d5897e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0a8fe00640 1 -- 172.21.15.67:0/2882500826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896a370 msgr2=0x563d58964fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/2882500826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896a370 0x563d58964fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.673+0000 7f0a8fe00640 1 -- 172.21.15.67:0/2882500826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d58997c50 con 0x563d5896cb80 2024-05-30T17:04:59.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.674+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/2882500826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896cb80 0x563d589a9570 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c00 tx=0x7f0a8002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f703f028eb0c365d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.674+0000 7f0a87400640 1 -- 172.21.15.67:0/2882500826 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80037040 con 0x563d5896cb80 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.674+0000 7f0a87400640 1 -- 172.21.15.67:0/2882500826 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8004a640 con 0x563d5896cb80 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.674+0000 7f0a87400640 1 -- 172.21.15.67:0/2882500826 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032880 con 0x563d5896cb80 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 -- 172.21.15.67:0/2882500826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896cb80 msgr2=0x563d589a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 --2- 172.21.15.67:0/2882500826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896cb80 0x563d589a9570 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c00 tx=0x7f0a8002c070 comp rx=0 tx=0).stop 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 -- 172.21.15.67:0/2882500826 shutdown_connections 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 --2- 172.21.15.67:0/2882500826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 0x563d5897e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 --2- 172.21.15.67:0/2882500826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896cb80 0x563d589a9570 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 --2- 172.21.15.67:0/2882500826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896a370 0x563d58964fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 -- 172.21.15.67:0/2882500826 >> 172.21.15.67:0/2882500826 conn(0x563d589666b0 msgr2=0x563d58981bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 -- 172.21.15.67:0/2882500826 shutdown_connections 2024-05-30T17:04:59.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 -- 172.21.15.67:0/2882500826 wait complete. 2024-05-30T17:04:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.675+0000 7f0aa1443740 1 -- start start 2024-05-30T17:04:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.676+0000 7f0aa1443740 1 Processor -- start 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.676+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d5893e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896cb80 0x563d5896d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0aa1443740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 0x563d5896dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0aa1443740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563d5893a4c0 con 0x563d5896a370 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0aa1443740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563d5893a7a0 con 0x563d5896cb80 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0aa1443740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563d58859570 con 0x563d589867f0 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0aa1443740 10 client.0 _ll_register_callbacks cb 0x7f0a8805a350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d5893e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d5893e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55040/0 (socket says 172.21.15.67:55040) 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a97400640 1 -- 172.21.15.67:0/49605971 learned_addr learned my addr 172.21.15.67:0/49605971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a97400640 1 -- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 msgr2=0x563d5896dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a97400640 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 0x563d5896dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a97400640 1 -- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896cb80 msgr2=0x563d5896d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a97400640 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896cb80 0x563d5896d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a97400640 1 -- 172.21.15.67:0/49605971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5896d310 con 0x563d5896a370 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a97400640 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d5893e3e0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f0a880628c0 tx=0x7f0a8808ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f761bda3864f4ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.677+0000 7f0a84c00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0a84c00640 1 -- 172.21.15.67:0/49605971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8801f040 con 0x563d5896a370 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0a84c00640 1 -- 172.21.15.67:0/49605971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f0a880592e0 con 0x563d5896a370 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0aa1443740 1 -- 172.21.15.67:0/49605971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a8004a640 con 0x563d5896a370 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0a84c00640 1 -- 172.21.15.67:0/49605971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f0a88068050 con 0x563d5896a370 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0aa1443740 10 client.9075 fetch_fsmap learned FSMap version 82 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0aa1443740 1 -- 172.21.15.67:0/49605971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x563d5896d310 con 0x563d5896a370 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0a84c00640 1 -- 172.21.15.67:0/49605971 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8004a640 con 0x563d5896a370 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0a84c00640 1 -- 172.21.15.67:0/49605971 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f0a88080070 con 0x563d5896a370 2024-05-30T17:04:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0aa1443740 10 client.9075 fetch_fsmap finished waiting for FSMap version 82 2024-05-30T17:04:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0aa1443740 10 client.9075 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-30T17:04:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0aa1443740 20 client.9075 populate_metadata read hostname 'smithi067' 2024-05-30T17:04:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0aa1443740 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] conn(0x563d58a03160 0x563d58968aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0aa1443740 4 client.9075 mds_command: new command op to 6937 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-30T17:04:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.678+0000 7f0aa1443740 1 -- 172.21.15.67:0/49605971 --> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] -- command(tid 0: {"prefix": "status"}) -- 0x563d58965af0 con 0x563d58a03160 2024-05-30T17:04:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.679+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] conn(0x563d58a03160 0x563d58968aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.680+0000 7f0a8fe00640 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] conn(0x563d58a03160 0x563d58968aa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1731be5a28c9af1b server_cookie=263913f9c1b91d20 in_seq=0 out_seq=0 2024-05-30T17:04:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.680+0000 7f0a84c00640 10 client.9075 ms_handle_connect on v2:172.21.15.169:6834/2504652568 2024-05-30T17:04:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0a84c00640 1 -- 172.21.15.67:0/49605971 <== mds.0 v2:172.21.15.169:6834/2504652568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563d58965af0 con 0x563d58a03160 2024-05-30T17:04:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0a84c00640 10 client.9075 handle_command_reply: tid=0 2024-05-30T17:04:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 client.9075 shutdown 2024-05-30T17:04:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 -- 172.21.15.67:0/49605971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 msgr2=0x563d5893e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d5893e3e0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f0a880628c0 tx=0x7f0a8808ad50 comp rx=0 tx=0).stop 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 -- 172.21.15.67:0/49605971 shutdown_connections 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:6834/2504652568,v1:172.21.15.169:6835/2504652568] conn(0x563d58a03160 0x563d58968aa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563d589867f0 0x563d5896dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563d5896cb80 0x563d5896d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 --2- 172.21.15.67:0/49605971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d5896a370 0x563d5893e3e0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0a84c00640 0 client.9075 ms_handle_reset on v2:172.21.15.169:6834/2504652568 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 -- 172.21.15.67:0/49605971 >> 172.21.15.67:0/49605971 conn(0x563d589666b0 msgr2=0x563d58964d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 -- 172.21.15.67:0/49605971 shutdown_connections 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 1 -- 172.21.15.67:0/49605971 wait complete. 2024-05-30T17:04:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.681+0000 7f0aa1443740 20 client.9075 trim_cache size 0 max 16384 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6937, 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 6.7064759760000001, 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 82, 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 132, 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 132, 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 12.112444018 2024-05-30T17:04:59.684 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:04:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 -- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a7406e510 msgr2=0x7f0a740709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a7406e510 0x7f0a740709d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0a80034000 tx=0x7f0a8002c000 comp rx=0 tx=0).stop 2024-05-30T17:04:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 -- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a9806f860 msgr2=0x7f0a98157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a9806f860 0x7f0a98157c70 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800c7f0 tx=0x7f0a8800ccb0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 -- 172.21.15.67:0/1173333522 shutdown_connections 2024-05-30T17:04:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0a7406e510 0x7f0a740709d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a980d09e0 0x7f0a980c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a980b3690 0x7f0a980cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 --2- 172.21.15.67:0/1173333522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a9806f860 0x7f0a98157c70 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 -- 172.21.15.67:0/1173333522 >> 172.21.15.67:0/1173333522 conn(0x7f0a9806bae0 msgr2=0x7f0a980af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 -- 172.21.15.67:0/1173333522 shutdown_connections 2024-05-30T17:04:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.682+0000 7f0a9e600640 1 -- 172.21.15.67:0/1173333522 wait complete. 2024-05-30T17:04:59.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-30T17:04:59.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.892+0000 7f4fb8600640 1 Processor -- start 2024-05-30T17:04:59.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.892+0000 7f4fb8600640 1 -- start start 2024-05-30T17:04:59.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.893+0000 7f4fb8600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4fb015fd30 0x7f4fb01699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.893+0000 7f4fb8600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb0163500 con 0x7f4fb00d2db0 2024-05-30T17:04:59.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.893+0000 7f4fb8600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb0162740 con 0x7f4fb00b3b50 2024-05-30T17:04:59.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.893+0000 7f4fb8600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb00d5540 con 0x7f4fb015fd30 2024-05-30T17:04:59.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.894+0000 7f4fb5c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4fb00d2db0 0x7f4fb00d4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44092/0 (socket says 172.21.15.67:44092) 2024-05-30T17:04:59.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.894+0000 7f4fb5c00640 1 -- 172.21.15.67:0/2657593048 learned_addr learned my addr 172.21.15.67:0/2657593048 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:04:59.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.894+0000 7f4fb6600640 1 --2- 172.21.15.67:0/2657593048 >> v2:172.21.15.67:3300/0 conn(0x7f4fb015fd30 0x7f4fb01699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.894+0000 7f4fb6600640 1 -- 172.21.15.67:0/2657593048 >> v1:172.21.15.169:6789/0 conn(0x7f4fb00b3b50 legacy=0x7f4fb00d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:04:59.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.894+0000 7f4fb6600640 1 -- 172.21.15.67:0/2657593048 >> v1:172.21.15.67:6789/0 conn(0x7f4fb00d2db0 legacy=0x7f4fb00d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.894+0000 7f4fb6600640 1 -- 172.21.15.67:0/2657593048 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb0162a20 con 0x7f4fb015fd30 2024-05-30T17:04:59.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.895+0000 7f4fb6600640 1 --2- 172.21.15.67:0/2657593048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb015fd30 0x7f4fb01699a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009d90 tx=0x7f4fa802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd056bc8e9c8e64 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fafe00640 1 -- 172.21.15.67:0/2657593048 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8031600 con 0x7f4fb015fd30 2024-05-30T17:04:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fafe00640 1 -- 172.21.15.67:0/2657593048 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa80317a0 con 0x7f4fb015fd30 2024-05-30T17:04:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fafe00640 1 -- 172.21.15.67:0/2657593048 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8031b70 con 0x7f4fb015fd30 2024-05-30T17:04:59.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fb8600640 1 -- 172.21.15.67:0/2657593048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb015fd30 msgr2=0x7f4fb01699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fb8600640 1 --2- 172.21.15.67:0/2657593048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb015fd30 0x7f4fb01699a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009d90 tx=0x7f4fa802a610 comp rx=0 tx=0).stop 2024-05-30T17:04:59.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fb8600640 1 -- 172.21.15.67:0/2657593048 shutdown_connections 2024-05-30T17:04:59.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fb8600640 1 --2- 172.21.15.67:0/2657593048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb015fd30 0x7f4fb01699a0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fb8600640 1 -- 172.21.15.67:0/2657593048 >> 172.21.15.67:0/2657593048 conn(0x7f4fb008bfc0 msgr2=0x7f4fb00d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:04:59.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fb8600640 1 -- 172.21.15.67:0/2657593048 shutdown_connections 2024-05-30T17:04:59.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.896+0000 7f4fb8600640 1 -- 172.21.15.67:0/2657593048 wait complete. 2024-05-30T17:04:59.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.898+0000 7f4fb8600640 1 Processor -- start 2024-05-30T17:04:59.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.898+0000 7f4fb8600640 1 -- start start 2024-05-30T17:04:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.899+0000 7f4fb8600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00b3b50 0x7f4fb00c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.899+0000 7f4fb8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb00d2db0 0x7f4fb00b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.899+0000 7f4fb8600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb00cc1a0 0x7f4fb01f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.899+0000 7f4fb8600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb00d04a0 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.899+0000 7f4fb8600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0162fe0 con 0x7f4fb00b3b50 2024-05-30T17:04:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.899+0000 7f4fb8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb00d0620 con 0x7f4fb00d2db0 2024-05-30T17:04:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.899+0000 7f4fb6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb00cc1a0 0x7f4fb01f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.899+0000 7f4fb6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb00cc1a0 0x7f4fb01f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55054/0 (socket says 172.21.15.67:55054) 2024-05-30T17:04:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.899+0000 7f4fb6600640 1 -- 172.21.15.67:0/542192014 learned_addr learned my addr 172.21.15.67:0/542192014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:04:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb5200640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb00d2db0 0x7f4fb00b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb6600640 1 -- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb00d2db0 msgr2=0x7f4fb00b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb5c00640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00b3b50 0x7f4fb00c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb6600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb00d2db0 0x7f4fb00b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb6600640 1 -- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00b3b50 msgr2=0x7f4fb00c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:04:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb6600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00b3b50 0x7f4fb00c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:04:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb6600640 1 -- 172.21.15.67:0/542192014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb00cc650 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb5200640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb00d2db0 0x7f4fb00b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb5c00640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00b3b50 0x7f4fb00c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:04:59.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.900+0000 7f4fb6600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb00cc1a0 0x7f4fb01f4bd0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009d90 tx=0x7f4fa8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.901+0000 7f4fad600640 1 -- 172.21.15.67:0/542192014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8040070 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.901+0000 7f4fad600640 1 -- 172.21.15.67:0/542192014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa8030b60 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.901+0000 7f4fad600640 1 -- 172.21.15.67:0/542192014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa803c420 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.901+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb01f5110 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.901+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb00b1da0 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.902+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb00689e0 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.903+0000 7f4fad600640 1 -- 172.21.15.67:0/542192014 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f4fa8039040 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.904+0000 7f4fad600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f9006e590 0x7f4f90070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:04:59.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.904+0000 7f4fad600640 1 -- 172.21.15.67:0/542192014 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f4fa80abde0 con 0x7f4fb00cc1a0 2024-05-30T17:04:59.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.905+0000 7f4fb5c00640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f9006e590 0x7f4f90070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:04:59.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.905+0000 7f4fb5c00640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f9006e590 0x7f4f90070a50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4000f60 tx=0x7f4fa4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:04:59.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:04:59.909+0000 7f4fad600640 1 -- 172.21.15.67:0/542192014 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4fa80b6020 con 0x7f4fb00cc1a0 2024-05-30T17:05:00.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.161+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4fb00d2af0 con 0x7f4fb00cc1a0 2024-05-30T17:05:00.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.162+0000 7f4fad600640 1 -- 172.21.15.67:0/542192014 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f4fa807beb0 con 0x7f4fb00cc1a0 2024-05-30T17:05:00.162 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f9006e590 msgr2=0x7f4f90070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f9006e590 0x7f4f90070a50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4000f60 tx=0x7f4fa4009040 comp rx=0 tx=0).stop 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb00cc1a0 msgr2=0x7f4fb01f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb00cc1a0 0x7f4fb01f4bd0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009d90 tx=0x7f4fa8004a40 comp rx=0 tx=0).stop 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 shutdown_connections 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4f9006e590 0x7f4f90070a50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb00cc1a0 0x7f4fb01f4bd0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb00d2db0 0x7f4fb00b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 --2- 172.21.15.67:0/542192014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00b3b50 0x7f4fb00c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 >> 172.21.15.67:0/542192014 conn(0x7f4fb008bfc0 msgr2=0x7f4fb0154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.164+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 shutdown_connections 2024-05-30T17:05:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00.165+0000 7f4fb8600640 1 -- 172.21.15.67:0/542192014 wait complete. 2024-05-30T17:05:00.184 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-30T17:05:00.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-30T17:05:00.184 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-05-30T17:05:00.184 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-30T17:05:00.184 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-30T17:05:00.184 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-30T17:05:00.185 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-30T17:05:00.185 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-30T17:05:00.185 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T17:05:00.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:00.185 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-05-30T17:05:00.240 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-30T17:05:00.240 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-30T17:05:00.240 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-05-30T17:05:00.240 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:05:00.240 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-05-30T17:05:00.240 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:05:00.240 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41272sec preferred_lft 41272sec 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-05-30T17:05:00.241 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::74f5:cfff:fec0:693d/64 scope link 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether aa:fe:ae:f7:f6:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::a8fe:aeff:fef7:f6a0/64 scope link 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 1a:5e:a4:c6:7d:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:05:00.242 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::185e:a4ff:fec6:7d8c/64 scope link 2024-05-30T17:05:00.243 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-05-30T17:05:00.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:00.244 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-30T17:05:00.329 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-05-30T17:05:00.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:00.334 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-05-30T17:05:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-05-30T17:05:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-30T17:05:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-30T17:05:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-30T17:05:00.390 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-30T17:05:00.390 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-30T17:05:00.390 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-05-30T17:05:00.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-30T17:05:00.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-30T17:05:00.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-30T17:05:00.391 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-05-30T17:05:00.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-30T17:05:00.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T17:05:00.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:00.618 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T17:05:00.671 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T17:05:00.671 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-30T17:05:00.672 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-30T17:05:00.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:00.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-05-30T17:05:00.753 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-30T17:05:00.753 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T17:05:00.754 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-30T17:05:00.807 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-30T17:05:00.807 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-30T17:05:00.808 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-30T17:05:00.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T17:05:00.950 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T17:05:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T17:05:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T17:05:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T17:05:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T17:05:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T17:05:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T17:05:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T17:05:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T17:05:01.007 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:05:01.008 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:05:01.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:01.009 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T17:05:01.062 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-30T17:05:01.063 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-30T17:05:01.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-05-30T17:05:01.141 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-05-30T17:05:01.185 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-30T17:05:01.186 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-0 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-1 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-2 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph/osd/ceph-3 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:05:01.187 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:05:01.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:01.188 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-05-30T17:05:01.245 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2024-05-30T17:05:01.245+0000 7fcd587ee480 -1 init, newargv = 0x557b6479fc80 newargc=15 2024-05-30T17:05:01.246 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[78266]: starting ceph client 2024-05-30T17:05:01.258 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[78266]: starting fuse 2024-05-30T17:05:01.275 INFO:teuthology.orchestra.run.smithi067.stdout:41 2024-05-30T17:05:01.276 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-30T17:05:01.276 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-05-30T17:05:01.276 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-05-30T17:05:01.276 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-05-30T17:05:01.277 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-30T17:05:01.278 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-30T17:05:01.278 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-30T17:05:01.278 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-05-30T17:05:01.278 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-05-30T17:05:01.278 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-05-30T17:05:01.278 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-30T17:05:01.278 DEBUG:teuthology.orchestra.run.smithi067:> 2024-05-30T17:05:01.278 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2024-05-30T17:05:01.278 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-05-30T17:05:01.477 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.78266.asok 2024-05-30T17:05:01.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-30T17:05:01.495 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.78266.asok 2024-05-30T17:05:01.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:01.495 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.78266.asok status 2024-05-30T17:05:01.731 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:05:01.731 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "9ed025fbf4406fb54980a0647506b15895b6d863", 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-3154-g9ed025fb (9ed025fbf4406fb54980a0647506b15895b6d863) squid (dev)", 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "78266", 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9090, 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-05-30T17:05:01.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 9090 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2653532824 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2653532824 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.9090 192.168.0.1:0/2653532824", 2024-05-30T17:05:01.733 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/2653532824", 2024-05-30T17:05:01.734 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-05-30T17:05:01.734 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 82, 2024-05-30T17:05:01.734 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 132, 2024-05-30T17:05:01.734 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-05-30T17:05:01.734 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-05-30T17:05:01.734 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-05-30T17:05:01.734 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:05:01.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:01.746 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-05-30T17:05:01.746 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-30T17:05:01.800 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-05-30T17:05:01.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-30T17:05:01.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:01.801 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-30T17:05:01.897 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-30T17:05:02.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.120+0000 7fc3c5400640 1 Processor -- start 2024-05-30T17:05:02.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.120+0000 7fc3c5400640 1 -- start start 2024-05-30T17:05:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.123+0000 7fc3c5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c00d1d90 0x7fc3c00d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.123+0000 7fc3c5400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c00cfba0 con 0x7fc3c00ce6f0 2024-05-30T17:05:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.123+0000 7fc3c5400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c0071a80 con 0x7fc3c009aa30 2024-05-30T17:05:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.123+0000 7fc3c5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c00cf670 con 0x7fc3c00d1d90 2024-05-30T17:05:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.123+0000 7fc3b7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc3c00ce6f0 0x7fc3c00d0250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60022/0 (socket says 172.21.15.67:60022) 2024-05-30T17:05:02.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.123+0000 7fc3b7e00640 1 -- 172.21.15.67:0/812550596 learned_addr learned my addr 172.21.15.67:0/812550596 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:05:02.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.123+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188862572 0 0) 0x7fc3c00cfba0 con 0x7fc3c00ce6f0 2024-05-30T17:05:02.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.123+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3a4003520 con 0x7fc3c00ce6f0 2024-05-30T17:05:02.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3c4a00640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c00d1d90 0x7fc3c00d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3659963168 0 0) 0x7fc3a4003520 con 0x7fc3c00ce6f0 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c00d1d90 msgr2=0x7fc3c00d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c00d1d90 0x7fc3c00d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 >> v1:172.21.15.169:6789/0 conn(0x7fc3c009aa30 legacy=0x7fc3c009ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c00cf8c0 con 0x7fc3c00ce6f0 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fc3a8001670 con 0x7fc3c00ce6f0 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 >> v1:172.21.15.67:6789/0 conn(0x7fc3c00ce6f0 legacy=0x7fc3c00d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3a4004800 0x7fc3a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c00d1d90 0x7fc3a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c00cf670 con 0x7fc3a40039e0 2024-05-30T17:05:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a4005520 con 0x7fc3a4004800 2024-05-30T17:05:02.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a400bf00 con 0x7fc3c00d1d90 2024-05-30T17:05:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.124+0000 7fc3bfe00640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.125+0000 7fc3b7e00640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3a4004800 0x7fc3a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.125+0000 7fc3c4a00640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c00d1d90 0x7fc3a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.125+0000 7fc3bfe00640 1 -- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c00d1d90 msgr2=0x7fc3a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.125+0000 7fc3bfe00640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c00d1d90 0x7fc3a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.125+0000 7fc3bfe00640 1 -- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3a4004800 msgr2=0x7fc3a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.125+0000 7fc3bfe00640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3a4004800 0x7fc3a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.125+0000 7fc3bfe00640 1 -- 172.21.15.67:0/812550596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a4003520 con 0x7fc3a40039e0 2024-05-30T17:05:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.125+0000 7fc3c4a00640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c00d1d90 0x7fc3a400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.126+0000 7fc3bfe00640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3a4003e00 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0001750 tx=0x7fc3a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cfc2192c644ce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.126+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0001370 con 0x7fc3a40039e0 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.126+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b00052d0 con 0x7fc3a40039e0 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.126+0000 7fc3bf400640 1 -- 172.21.15.67:0/812550596 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b002abe0 con 0x7fc3a40039e0 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.127+0000 7fc3c5400640 1 -- 172.21.15.67:0/812550596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 msgr2=0x7fc3a4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.127+0000 7fc3c5400640 1 --2- 172.21.15.67:0/812550596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3a4003e00 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0001750 tx=0x7fc3a4005830 comp rx=0 tx=0).stop 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.127+0000 7fc3c4a00640 1 -- 172.21.15.67:0/812550596 reap_dead start 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.127+0000 7fc3c5400640 1 -- 172.21.15.67:0/812550596 shutdown_connections 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.127+0000 7fc3c5400640 1 -- 172.21.15.67:0/812550596 >> 172.21.15.67:0/812550596 conn(0x7fc3c00b3ec0 msgr2=0x7fc3c00b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.127+0000 7fc3c5400640 1 -- 172.21.15.67:0/812550596 shutdown_connections 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.127+0000 7fc3c5400640 1 -- 172.21.15.67:0/812550596 wait complete. 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.129+0000 7fc3c5400640 1 Processor -- start 2024-05-30T17:05:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.129+0000 7fc3c5400640 1 -- start start 2024-05-30T17:05:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.130+0000 7fc3c5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3c00cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.130+0000 7fc3c5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c00cd840 0x7fc3c0150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.130+0000 7fc3c5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c015e070 0x7fc3c015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.130+0000 7fc3c5400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c00716a0 con 0x7fc3a40039e0 2024-05-30T17:05:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.130+0000 7fc3c5400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c006fc60 con 0x7fc3c00cd840 2024-05-30T17:05:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.130+0000 7fc3c5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c0071820 con 0x7fc3c015e070 2024-05-30T17:05:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.130+0000 7fc3bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3c00cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.130+0000 7fc3bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3c00cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56900/0 (socket says 172.21.15.67:56900) 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.130+0000 7fc3bfe00640 1 -- 172.21.15.67:0/289353998 learned_addr learned my addr 172.21.15.67:0/289353998 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.131+0000 7fc3c4a00640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c015e070 0x7fc3c015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.131+0000 7fc3bfe00640 1 -- 172.21.15.67:0/289353998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c015e070 msgr2=0x7fc3c015e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.131+0000 7fc3bfe00640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c015e070 0x7fc3c015e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.131+0000 7fc3b7e00640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c00cd840 0x7fc3c0150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.131+0000 7fc3bfe00640 1 -- 172.21.15.67:0/289353998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c00cd840 msgr2=0x7fc3c0150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.131+0000 7fc3bfe00640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c00cd840 0x7fc3c0150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.131+0000 7fc3bfe00640 1 -- 172.21.15.67:0/289353998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c0150580 con 0x7fc3a40039e0 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.131+0000 7fc3bfe00640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3c00cbf90 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0008fd0 tx=0x7fc3b0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.131+0000 7fc3bcc00640 1 -- 172.21.15.67:0/289353998 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0040020 con 0x7fc3a40039e0 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.132+0000 7fc3bcc00640 1 -- 172.21.15.67:0/289353998 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b0007680 con 0x7fc3a40039e0 2024-05-30T17:05:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.132+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c00cc570 con 0x7fc3a40039e0 2024-05-30T17:05:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.132+0000 7fc3bcc00640 1 -- 172.21.15.67:0/289353998 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0007a30 con 0x7fc3a40039e0 2024-05-30T17:05:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.132+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c00ccb30 con 0x7fc3a40039e0 2024-05-30T17:05:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.133+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c00689e0 con 0x7fc3a40039e0 2024-05-30T17:05:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.134+0000 7fc3bcc00640 1 -- 172.21.15.67:0/289353998 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fc3b0043020 con 0x7fc3a40039e0 2024-05-30T17:05:02.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.135+0000 7fc3bcc00640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc39c06e540 0x7fc39c070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:02.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.135+0000 7fc3bcc00640 1 -- 172.21.15.67:0/289353998 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7fc3b00aab50 con 0x7fc3a40039e0 2024-05-30T17:05:02.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.135+0000 7fc3b7e00640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc39c06e540 0x7fc39c070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:02.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.136+0000 7fc3b7e00640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc39c06e540 0x7fc39c070a00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8001eb0 tx=0x7fc3a8000ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:02.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.140+0000 7fc3bcc00640 1 -- 172.21.15.67:0/289353998 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3b004a030 con 0x7fc3a40039e0 2024-05-30T17:05:02.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:02.391+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fc3c00b4d60 con 0x7fc3a40039e0 2024-05-30T17:05:03.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.055+0000 7fc3bcc00640 1 -- 172.21.15.67:0/289353998 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v83) ==== 136+0+0 (secure 0 0 0) 0x7fc3c0150580 con 0x7fc3a40039e0 2024-05-30T17:05:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc39c06e540 msgr2=0x7fc39c070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc39c06e540 0x7fc39c070a00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8001eb0 tx=0x7fc3a8000ed0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 msgr2=0x7fc3c00cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3c00cbf90 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0008fd0 tx=0x7fc3b0001120 comp rx=0 tx=0).stop 2024-05-30T17:05:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 shutdown_connections 2024-05-30T17:05:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc39c06e540 0x7fc39c070a00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3c015e070 0x7fc3c015e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3c00cd840 0x7fc3c0150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 --2- 172.21.15.67:0/289353998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3a40039e0 0x7fc3c00cbf90 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.059+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 >> 172.21.15.67:0/289353998 conn(0x7fc3c00b3ec0 msgr2=0x7fc3c00b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.060+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 shutdown_connections 2024-05-30T17:05:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.060+0000 7fc3c5400640 1 -- 172.21.15.67:0/289353998 wait complete. 2024-05-30T17:05:03.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T17:05:03.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.290+0000 7fd5f8c00640 1 Processor -- start 2024-05-30T17:05:03.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.290+0000 7fd5f8c00640 1 -- start start 2024-05-30T17:05:03.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.292+0000 7fd5f8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162940 0x7fd5f406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.292+0000 7fd5f8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5f4162d20 0x7fd5f4071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.292+0000 7fd5f8c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5f406f590 con 0x7fd5f406f040 2024-05-30T17:05:03.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.292+0000 7fd5f8c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd5f406f870 con 0x7fd5f4162d20 2024-05-30T17:05:03.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.292+0000 7fd5f8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f40ad3d0 con 0x7fd5f4162940 2024-05-30T17:05:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.292+0000 7fd5f2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5f4162d20 0x7fd5f4071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.292+0000 7fd5f2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5f4162d20 0x7fd5f4071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56910/0 (socket says 172.21.15.67:56910) 2024-05-30T17:05:03.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.292+0000 7fd5f2a00640 1 -- 172.21.15.67:0/3213188645 learned_addr learned my addr 172.21.15.67:0/3213188645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:03.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.293+0000 7fd5f3400640 1 --2- 172.21.15.67:0/3213188645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162940 0x7fd5f406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.293+0000 7fd5f2a00640 1 -- 172.21.15.67:0/3213188645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162940 msgr2=0x7fd5f406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:03.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.293+0000 7fd5f2a00640 1 --2- 172.21.15.67:0/3213188645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162940 0x7fd5f406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.293+0000 7fd5f2a00640 1 -- 172.21.15.67:0/3213188645 >> v1:172.21.15.169:6789/0 conn(0x7fd5f406f040 legacy=0x7fd5f4166680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:03.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.293+0000 7fd5f2a00640 1 -- 172.21.15.67:0/3213188645 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f406dde0 con 0x7fd5f4162d20 2024-05-30T17:05:03.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.294+0000 7fd5f2a00640 1 --2- 172.21.15.67:0/3213188645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162d20 0x7fd5f4071bc0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009870 tx=0x7fd5e002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a89b93b65d80da server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:03.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.294+0000 7fd5f2000640 1 -- 172.21.15.67:0/3213188645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0037070 con 0x7fd5f4162d20 2024-05-30T17:05:03.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.294+0000 7fd5f2000640 1 -- 172.21.15.67:0/3213188645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e002ae10 con 0x7fd5f4162d20 2024-05-30T17:05:03.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.294+0000 7fd5f2000640 1 -- 172.21.15.67:0/3213188645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0032660 con 0x7fd5f4162d20 2024-05-30T17:05:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.295+0000 7fd5f8c00640 1 -- 172.21.15.67:0/3213188645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162d20 msgr2=0x7fd5f4071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.295+0000 7fd5f8c00640 1 --2- 172.21.15.67:0/3213188645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162d20 0x7fd5f4071bc0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009870 tx=0x7fd5e002a100 comp rx=0 tx=0).stop 2024-05-30T17:05:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.295+0000 7fd5f8c00640 1 -- 172.21.15.67:0/3213188645 shutdown_connections 2024-05-30T17:05:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.295+0000 7fd5f8c00640 1 --2- 172.21.15.67:0/3213188645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162d20 0x7fd5f4071bc0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.295+0000 7fd5f8c00640 1 --2- 172.21.15.67:0/3213188645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162940 0x7fd5f406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.295+0000 7fd5f8c00640 1 -- 172.21.15.67:0/3213188645 >> 172.21.15.67:0/3213188645 conn(0x7fd5f40b1fc0 msgr2=0x7fd5f40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.295+0000 7fd5f8c00640 1 -- 172.21.15.67:0/3213188645 shutdown_connections 2024-05-30T17:05:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.295+0000 7fd5f8c00640 1 -- 172.21.15.67:0/3213188645 wait complete. 2024-05-30T17:05:03.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.297+0000 7fd5f8c00640 1 Processor -- start 2024-05-30T17:05:03.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.297+0000 7fd5f8c00640 1 -- start start 2024-05-30T17:05:03.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.298+0000 7fd5f8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f406f040 0x7fd5f41608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.298+0000 7fd5f8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162940 0x7fd5f409ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.298+0000 7fd5f8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162d20 0x7fd5f40cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.298+0000 7fd5f8c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f4072100 con 0x7fd5f4162940 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.298+0000 7fd5f8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f40aee50 con 0x7fd5f406f040 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.298+0000 7fd5f8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f4072280 con 0x7fd5f4162d20 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.298+0000 7fd5f2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162940 0x7fd5f409ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.298+0000 7fd5f2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162940 0x7fd5f409ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56922/0 (socket says 172.21.15.67:56922) 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.298+0000 7fd5f2a00640 1 -- 172.21.15.67:0/4010289555 learned_addr learned my addr 172.21.15.67:0/4010289555 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f3400640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f406f040 0x7fd5f41608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f3e00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162d20 0x7fd5f40cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f2a00640 1 -- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162d20 msgr2=0x7fd5f40cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f2a00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162d20 0x7fd5f40cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f2a00640 1 -- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f406f040 msgr2=0x7fd5f41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f2a00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f406f040 0x7fd5f41608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f2a00640 1 -- 172.21.15.67:0/4010289555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f409b770 con 0x7fd5f4162940 2024-05-30T17:05:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f3e00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162d20 0x7fd5f40cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:03.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f3400640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f406f040 0x7fd5f41608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:03.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.299+0000 7fd5f2a00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162940 0x7fd5f409ad20 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000c00 tx=0x7fd5e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:03.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.300+0000 7fd5eb400640 1 -- 172.21.15.67:0/4010289555 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0041070 con 0x7fd5f4162940 2024-05-30T17:05:03.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.300+0000 7fd5eb400640 1 -- 172.21.15.67:0/4010289555 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e003c670 con 0x7fd5f4162940 2024-05-30T17:05:03.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.300+0000 7fd5eb400640 1 -- 172.21.15.67:0/4010289555 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0037040 con 0x7fd5f4162940 2024-05-30T17:05:03.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.300+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f409ba50 con 0x7fd5f4162940 2024-05-30T17:05:03.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.300+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f409c010 con 0x7fd5f4162940 2024-05-30T17:05:03.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.301+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f40689e0 con 0x7fd5f4162940 2024-05-30T17:05:03.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.302+0000 7fd5eb400640 1 -- 172.21.15.67:0/4010289555 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fd5e0044070 con 0x7fd5f4162940 2024-05-30T17:05:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.303+0000 7fd5eb400640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd5d406e510 0x7fd5d40709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.303+0000 7fd5eb400640 1 -- 172.21.15.67:0/4010289555 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7fd5e00ab240 con 0x7fd5f4162940 2024-05-30T17:05:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.303+0000 7fd5f3400640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd5d406e510 0x7fd5d40709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.304+0000 7fd5f3400640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd5d406e510 0x7fd5d40709d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:03.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.307+0000 7fd5eb400640 1 -- 172.21.15.67:0/4010289555 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd5e007b1f0 con 0x7fd5f4162940 2024-05-30T17:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.539+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fd5f409c250 con 0x7fd5d406e510 2024-05-30T17:05:03.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.543+0000 7fd5eb400640 1 -- 172.21.15.67:0/4010289555 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fd5f409c250 con 0x7fd5d406e510 2024-05-30T17:05:03.543 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:05:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:05:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T17:05:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:05:03.544 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:05:03.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd5d406e510 msgr2=0x7fd5d40709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd5d406e510 0x7fd5d40709d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4008040 comp rx=0 tx=0).stop 2024-05-30T17:05:03.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162940 msgr2=0x7fd5f409ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162940 0x7fd5f409ad20 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000c00 tx=0x7fd5e00046c0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 shutdown_connections 2024-05-30T17:05:03.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd5d406e510 0x7fd5d40709d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5f4162d20 0x7fd5f40cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f4162940 0x7fd5f409ad20 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 --2- 172.21.15.67:0/4010289555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5f406f040 0x7fd5f41608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.545+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 >> 172.21.15.67:0/4010289555 conn(0x7fd5f40b1fc0 msgr2=0x7fd5f406e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:03.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.546+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 shutdown_connections 2024-05-30T17:05:03.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.546+0000 7fd5f8c00640 1 -- 172.21.15.67:0/4010289555 wait complete. 2024-05-30T17:05:03.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-30T17:05:03.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.781+0000 7fcd38a00640 1 Processor -- start 2024-05-30T17:05:03.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.781+0000 7fcd38a00640 1 -- start start 2024-05-30T17:05:03.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.783+0000 7fcd38a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd3415e870 0x7fcd340d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.783+0000 7fcd38a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd340d14d0 0x7fcd340cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.783+0000 7fcd38a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd340cfd00 con 0x7fcd340cf870 2024-05-30T17:05:03.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.783+0000 7fcd38a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fcd340d2150 con 0x7fcd3415e870 2024-05-30T17:05:03.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.783+0000 7fcd38a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3415ec50 con 0x7fcd340d14d0 2024-05-30T17:05:03.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.783+0000 7fcd32800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd3415e870 0x7fcd340d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.783+0000 7fcd32800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd3415e870 0x7fcd340d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56924/0 (socket says 172.21.15.67:56924) 2024-05-30T17:05:03.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.783+0000 7fcd32800640 1 -- 172.21.15.67:0/265584287 learned_addr learned my addr 172.21.15.67:0/265584287 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.783+0000 7fcd31e00640 1 --2- 172.21.15.67:0/265584287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd340d14d0 0x7fcd340cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.784+0000 7fcd32800640 1 -- 172.21.15.67:0/265584287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd340d14d0 msgr2=0x7fcd340cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.784+0000 7fcd32800640 1 --2- 172.21.15.67:0/265584287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd340d14d0 0x7fcd340cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.784+0000 7fcd32800640 1 -- 172.21.15.67:0/265584287 >> v1:172.21.15.67:6789/0 conn(0x7fcd340cf870 legacy=0x7fcd34162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.784+0000 7fcd32800640 1 -- 172.21.15.67:0/265584287 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd340cff80 con 0x7fcd3415e870 2024-05-30T17:05:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.785+0000 7fcd32800640 1 --2- 172.21.15.67:0/265584287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3415e870 0x7fcd340d0f90 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009e50 tx=0x7fcd2402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0bbdb5497b82fe server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.785+0000 7fcd31400640 1 -- 172.21.15.67:0/265584287 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24036070 con 0x7fcd3415e870 2024-05-30T17:05:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.785+0000 7fcd31400640 1 -- 172.21.15.67:0/265584287 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd24031600 con 0x7fcd3415e870 2024-05-30T17:05:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.785+0000 7fcd31400640 1 -- 172.21.15.67:0/265584287 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd240319d0 con 0x7fcd3415e870 2024-05-30T17:05:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.786+0000 7fcd38a00640 1 -- 172.21.15.67:0/265584287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3415e870 msgr2=0x7fcd340d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.786+0000 7fcd38a00640 1 --2- 172.21.15.67:0/265584287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3415e870 0x7fcd340d0f90 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009e50 tx=0x7fcd2402a710 comp rx=0 tx=0).stop 2024-05-30T17:05:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.786+0000 7fcd38a00640 1 -- 172.21.15.67:0/265584287 shutdown_connections 2024-05-30T17:05:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.786+0000 7fcd38a00640 1 --2- 172.21.15.67:0/265584287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd340d14d0 0x7fcd340cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.786+0000 7fcd38a00640 1 --2- 172.21.15.67:0/265584287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd3415e870 0x7fcd340d0f90 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.786+0000 7fcd38a00640 1 -- 172.21.15.67:0/265584287 >> 172.21.15.67:0/265584287 conn(0x7fcd3408bfc0 msgr2=0x7fcd340d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.786+0000 7fcd38a00640 1 -- 172.21.15.67:0/265584287 shutdown_connections 2024-05-30T17:05:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.786+0000 7fcd38a00640 1 -- 172.21.15.67:0/265584287 wait complete. 2024-05-30T17:05:03.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.788+0000 7fcd38a00640 1 Processor -- start 2024-05-30T17:05:03.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.788+0000 7fcd38a00640 1 -- start start 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd38a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd340cf870 0x7fcd3414fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd38a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d14d0 0x7fcd3415ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd38a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3415e870 0x7fcd340b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd38a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd340d0820 con 0x7fcd340d14d0 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd38a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd340d0540 con 0x7fcd340cf870 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd38a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd340d4fb0 con 0x7fcd3415e870 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd31e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d14d0 0x7fcd3415ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd31e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d14d0 0x7fcd3415ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56938/0 (socket says 172.21.15.67:56938) 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd31e00640 1 -- 172.21.15.67:0/1034565740 learned_addr learned my addr 172.21.15.67:0/1034565740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd33200640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3415e870 0x7fcd340b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd32800640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd340cf870 0x7fcd3414fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd31e00640 1 -- 172.21.15.67:0/1034565740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3415e870 msgr2=0x7fcd340b8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd31e00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3415e870 0x7fcd340b8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd31e00640 1 -- 172.21.15.67:0/1034565740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd340cf870 msgr2=0x7fcd3414fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd31e00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd340cf870 0x7fcd3414fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd31e00640 1 -- 172.21.15.67:0/1034565740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd340cb8a0 con 0x7fcd340d14d0 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.789+0000 7fcd33200640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3415e870 0x7fcd340b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.790+0000 7fcd31e00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d14d0 0x7fcd3415ee60 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c7f0 tx=0x7fcd1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.790+0000 7fcd2aa00640 1 -- 172.21.15.67:0/1034565740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c015070 con 0x7fcd340d14d0 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.790+0000 7fcd2aa00640 1 -- 172.21.15.67:0/1034565740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c007820 con 0x7fcd340d14d0 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.790+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd340cbb80 con 0x7fcd340d14d0 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.790+0000 7fcd2aa00640 1 -- 172.21.15.67:0/1034565740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c007c10 con 0x7fcd340d14d0 2024-05-30T17:05:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.790+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd340cc140 con 0x7fcd340d14d0 2024-05-30T17:05:03.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.792+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd340cb630 con 0x7fcd340d14d0 2024-05-30T17:05:03.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.793+0000 7fcd2aa00640 1 -- 172.21.15.67:0/1034565740 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7fcd1c018070 con 0x7fcd340d14d0 2024-05-30T17:05:03.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.794+0000 7fcd2aa00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1006e510 0x7fcd100709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:03.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.794+0000 7fcd2aa00640 1 -- 172.21.15.67:0/1034565740 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7fcd1c08ab80 con 0x7fcd340d14d0 2024-05-30T17:05:03.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.794+0000 7fcd32800640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1006e510 0x7fcd100709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:03.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.795+0000 7fcd32800640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1006e510 0x7fcd100709d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcd24033000 tx=0x7fcd24009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:03.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:03.798+0000 7fcd2aa00640 1 -- 172.21.15.67:0/1034565740 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd1c05aab0 con 0x7fcd340d14d0 2024-05-30T17:05:04.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.005+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcd340689e0 con 0x7fcd340d14d0 2024-05-30T17:05:04.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.006+0000 7fcd2aa00640 1 -- 172.21.15.67:0/1034565740 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fcd1c05a3b0 con 0x7fcd340d14d0 2024-05-30T17:05:04.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1006e510 msgr2=0x7fcd100709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1006e510 0x7fcd100709d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcd24033000 tx=0x7fcd24009000 comp rx=0 tx=0).stop 2024-05-30T17:05:04.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d14d0 msgr2=0x7fcd3415ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d14d0 0x7fcd3415ee60 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c7f0 tx=0x7fcd1c00ccb0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 shutdown_connections 2024-05-30T17:05:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fcd1006e510 0x7fcd100709d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3415e870 0x7fcd340b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd340d14d0 0x7fcd3415ee60 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 --2- 172.21.15.67:0/1034565740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd340cf870 0x7fcd3414fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 >> 172.21.15.67:0/1034565740 conn(0x7fcd3408bfc0 msgr2=0x7fcd340b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 shutdown_connections 2024-05-30T17:05:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.009+0000 7fcd38a00640 1 -- 172.21.15.67:0/1034565740 wait complete. 2024-05-30T17:05:04.029 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-05-30T17:05:04.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.242+0000 7fadf8a00640 1 Processor -- start 2024-05-30T17:05:04.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.242+0000 7fadf8a00640 1 -- start start 2024-05-30T17:05:04.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.243+0000 7fadf8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fadf4159f00 0x7fadf4073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.244+0000 7fadf8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf40af330 0x7fadf4071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.244+0000 7fadf8a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf415a2e0 con 0x7fadf415d3b0 2024-05-30T17:05:04.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.244+0000 7fadf8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fadf415d860 con 0x7fadf4159f00 2024-05-30T17:05:04.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.244+0000 7fadf8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fadf40ad230 con 0x7fadf40af330 2024-05-30T17:05:04.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.244+0000 7fadf2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fadf415d3b0 0x7fadf40aec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60044/0 (socket says 172.21.15.67:60044) 2024-05-30T17:05:04.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.244+0000 7fadf2a00640 1 -- 172.21.15.67:0/1748465573 learned_addr learned my addr 172.21.15.67:0/1748465573 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:05:04.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.244+0000 7fadf3400640 1 --2- 172.21.15.67:0/1748465573 >> v2:172.21.15.169:3300/0 conn(0x7fadf4159f00 0x7fadf4073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.244+0000 7fadf3e00640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf40af330 0x7fadf4071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2926937089 0 0) 0x7faddc001110 con 0x7fadf415d3b0 2024-05-30T17:05:04.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadd8003520 con 0x7fadf415d3b0 2024-05-30T17:05:04.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 831019214 0 0) 0x7fadd8003520 con 0x7fadf415d3b0 2024-05-30T17:05:04.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf40af330 msgr2=0x7fadf4071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf40af330 0x7fadf4071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 >> v2:172.21.15.169:3300/0 conn(0x7fadf4159f00 msgr2=0x7fadf4073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 --2- 172.21.15.67:0/1748465573 >> v2:172.21.15.169:3300/0 conn(0x7fadf4159f00 0x7fadf4073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf40ad510 con 0x7fadf415d3b0 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7faddc001840 con 0x7fadf415d3b0 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 >> v1:172.21.15.67:6789/0 conn(0x7fadf415d3b0 legacy=0x7fadf40aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd80039b0 0x7fadd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadd8004800 0x7fadd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf40af330 0x7fadd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fadf40ad230 con 0x7fadd8004800 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fadd800ba70 con 0x7fadd80039b0 2024-05-30T17:05:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.245+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fadd800bc20 con 0x7fadf40af330 2024-05-30T17:05:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.246+0000 7fadf2a00640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadd8004800 0x7fadd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.246+0000 7fadf3400640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd80039b0 0x7fadd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.246+0000 7fadf2a00640 1 -- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf40af330 msgr2=0x7fadd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.246+0000 7fadf2a00640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf40af330 0x7fadd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.246+0000 7fadf2a00640 1 -- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd80039b0 msgr2=0x7fadd8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.246+0000 7fadf2a00640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd80039b0 0x7fadd8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.246+0000 7fadf2a00640 1 -- 172.21.15.67:0/1748465573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd8003520 con 0x7fadd8004800 2024-05-30T17:05:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.247+0000 7fadf2a00640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadd8004800 0x7fadd8004c80 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c2f0 tx=0x7fadd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431cc3df1e8d06e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:04.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.247+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc035070 con 0x7fadd8004800 2024-05-30T17:05:04.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.247+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faddc007e50 con 0x7fadd8004800 2024-05-30T17:05:04.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.247+0000 7fadf2000640 1 -- 172.21.15.67:0/1748465573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc005d30 con 0x7fadd8004800 2024-05-30T17:05:04.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.248+0000 7fadf8a00640 1 -- 172.21.15.67:0/1748465573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadd8004800 msgr2=0x7fadd8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.248+0000 7fadf8a00640 1 --2- 172.21.15.67:0/1748465573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadd8004800 0x7fadd8004c80 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c2f0 tx=0x7fadd8005830 comp rx=0 tx=0).stop 2024-05-30T17:05:04.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.248+0000 7fadf3e00640 1 -- 172.21.15.67:0/1748465573 reap_dead start 2024-05-30T17:05:04.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.248+0000 7fadf8a00640 1 -- 172.21.15.67:0/1748465573 shutdown_connections 2024-05-30T17:05:04.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.248+0000 7fadf8a00640 1 -- 172.21.15.67:0/1748465573 >> 172.21.15.67:0/1748465573 conn(0x7fadf40b1db0 msgr2=0x7fadf40b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:04.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.248+0000 7fadf8a00640 1 -- 172.21.15.67:0/1748465573 shutdown_connections 2024-05-30T17:05:04.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.248+0000 7fadf8a00640 1 -- 172.21.15.67:0/1748465573 wait complete. 2024-05-30T17:05:04.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.250+0000 7fadf8a00640 1 Processor -- start 2024-05-30T17:05:04.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.250+0000 7fadf8a00640 1 -- start start 2024-05-30T17:05:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.251+0000 7fadf8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd8004800 0x7fadf40b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.251+0000 7fadf8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadf4152600 0x7fadf40ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.251+0000 7fadf8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf4152a80 0x7fadf40adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.251+0000 7fadf8a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4071f00 con 0x7fadf4152600 2024-05-30T17:05:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.251+0000 7fadf8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fadf40cfbc0 con 0x7fadd8004800 2024-05-30T17:05:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.251+0000 7fadf8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fadf4072080 con 0x7fadf4152a80 2024-05-30T17:05:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.251+0000 7fadf2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadf4152600 0x7fadf40ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.251+0000 7fadf2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadf4152600 0x7fadf40ad9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56952/0 (socket says 172.21.15.67:56952) 2024-05-30T17:05:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.251+0000 7fadf2a00640 1 -- 172.21.15.67:0/1083895690 learned_addr learned my addr 172.21.15.67:0/1083895690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf3400640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd8004800 0x7fadf40b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf3e00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf4152a80 0x7fadf40adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf2a00640 1 -- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf4152a80 msgr2=0x7fadf40adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf2a00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf4152a80 0x7fadf40adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf2a00640 1 -- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd8004800 msgr2=0x7fadf40b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf2a00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd8004800 0x7fadf40b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf2a00640 1 -- 172.21.15.67:0/1083895690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf41f4aa0 con 0x7fadf4152600 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf3e00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf4152a80 0x7fadf40adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf3400640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd8004800 0x7fadf40b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.252+0000 7fadf2a00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadf4152600 0x7fadf40ad9a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7faddc033040 tx=0x7fadf406d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.253+0000 7fadeb400640 1 -- 172.21.15.67:0/1083895690 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc035040 con 0x7fadf4152600 2024-05-30T17:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.253+0000 7fadeb400640 1 -- 172.21.15.67:0/1083895690 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faddc005e20 con 0x7fadf4152600 2024-05-30T17:05:04.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.253+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf41f4d80 con 0x7fadf4152600 2024-05-30T17:05:04.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.253+0000 7fadeb400640 1 -- 172.21.15.67:0/1083895690 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc007270 con 0x7fadf4152600 2024-05-30T17:05:04.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.253+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf41f5340 con 0x7fadf4152600 2024-05-30T17:05:04.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.254+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf40689e0 con 0x7fadf4152600 2024-05-30T17:05:04.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.255+0000 7fadeb400640 1 -- 172.21.15.67:0/1083895690 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7faddc007410 con 0x7fadf4152600 2024-05-30T17:05:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.257+0000 7fadeb400640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fadd006e510 0x7fadd00709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.257+0000 7fadeb400640 1 -- 172.21.15.67:0/1083895690 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7faddc0aae40 con 0x7fadf4152600 2024-05-30T17:05:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.257+0000 7fadf3400640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fadd006e510 0x7fadd00709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.258+0000 7fadf3400640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fadd006e510 0x7fadd00709d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fadf415c040 tx=0x7fade400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:04.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.261+0000 7fadeb400640 1 -- 172.21.15.67:0/1083895690 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faddc040020 con 0x7fadf4152600 2024-05-30T17:05:04.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.464+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7fadf4072ec0 con 0x7fadf4152600 2024-05-30T17:05:04.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.472+0000 7fadeb400640 1 -- 172.21.15.67:0/1083895690 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v29)=0 v29) ==== 136+0+0 (secure 0 0 0) 0x7fadf415a2e0 con 0x7fadf4152600 2024-05-30T17:05:04.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fadd006e510 msgr2=0x7fadd00709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fadd006e510 0x7fadd00709d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fadf415c040 tx=0x7fade400b040 comp rx=0 tx=0).stop 2024-05-30T17:05:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadf4152600 msgr2=0x7fadf40ad9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadf4152600 0x7fadf40ad9a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7faddc033040 tx=0x7fadf406d870 comp rx=0 tx=0).stop 2024-05-30T17:05:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 shutdown_connections 2024-05-30T17:05:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fadd006e510 0x7fadd00709d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadf4152a80 0x7fadf40adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadf4152600 0x7fadf40ad9a0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 --2- 172.21.15.67:0/1083895690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd8004800 0x7fadf40b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 >> 172.21.15.67:0/1083895690 conn(0x7fadf40b1db0 msgr2=0x7fadf40cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:04.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 shutdown_connections 2024-05-30T17:05:04.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.476+0000 7fadf8a00640 1 -- 172.21.15.67:0/1083895690 wait complete. 2024-05-30T17:05:04.498 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-30T17:05:04.518 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.518+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:04.519 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.518+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:04.519 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.518+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:04.519 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.518+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:04.519 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.518+0000 7f0c8da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:04.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.679+0000 7f5ea5400640 1 Processor -- start 2024-05-30T17:05:04.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.679+0000 7f5ea5400640 1 -- start start 2024-05-30T17:05:04.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.681+0000 7f5ea5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea00d3220 0x7f5ea00d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.681+0000 7f5ea5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5ea00cf180 0x7f5ea00cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.681+0000 7f5ea5400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea00d0e50 con 0x7f5ea01621d0 2024-05-30T17:05:04.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.681+0000 7f5ea5400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea00d1a60 con 0x7f5ea00cf180 2024-05-30T17:05:04.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.681+0000 7f5ea5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea00d1130 con 0x7f5ea00d3220 2024-05-30T17:05:04.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.681+0000 7f5e9fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5ea01621d0 0x7f5ea01625b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60060/0 (socket says 172.21.15.67:60060) 2024-05-30T17:05:04.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.681+0000 7f5e9fe00640 1 -- 172.21.15.67:0/4100967178 learned_addr learned my addr 172.21.15.67:0/4100967178 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:05:04.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.681+0000 7f5ea4a00640 1 --2- 172.21.15.67:0/4100967178 >> v2:172.21.15.67:3300/0 conn(0x7f5ea00cf180 0x7f5ea00cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.682+0000 7f5e97400640 1 --2- 172.21.15.67:0/4100967178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea00d3220 0x7f5ea00d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.682+0000 7f5ea4a00640 1 -- 172.21.15.67:0/4100967178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea00d3220 msgr2=0x7f5ea00d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.682+0000 7f5ea4a00640 1 --2- 172.21.15.67:0/4100967178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea00d3220 0x7f5ea00d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.682+0000 7f5ea4a00640 1 -- 172.21.15.67:0/4100967178 >> v1:172.21.15.67:6789/0 conn(0x7f5ea01621d0 legacy=0x7f5ea01625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.682+0000 7f5ea4a00640 1 -- 172.21.15.67:0/4100967178 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea00d1410 con 0x7f5ea00cf180 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.682+0000 7f5e9f400640 1 -- 172.21.15.67:0/4100967178 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377707792 0 0) 0x7f5ea00d0e50 con 0x7f5ea01621d0 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.682+0000 7f5e97400640 1 --2- 172.21.15.67:0/4100967178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea00d3220 0x7f5ea00d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.683+0000 7f5ea4a00640 1 --2- 172.21.15.67:0/4100967178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00cf180 0x7f5ea00cf610 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009d90 tx=0x7f5e9802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe74cbf6d226318b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.683+0000 7f5e9f400640 1 -- 172.21.15.67:0/4100967178 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98007740 con 0x7f5ea00cf180 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.683+0000 7f5e9f400640 1 -- 172.21.15.67:0/4100967178 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e980078e0 con 0x7f5ea00cf180 2024-05-30T17:05:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.683+0000 7f5e9f400640 1 -- 172.21.15.67:0/4100967178 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98007cb0 con 0x7f5ea00cf180 2024-05-30T17:05:04.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.684+0000 7f5ea5400640 1 -- 172.21.15.67:0/4100967178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00cf180 msgr2=0x7f5ea00cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.684+0000 7f5ea5400640 1 --2- 172.21.15.67:0/4100967178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00cf180 0x7f5ea00cf610 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009d90 tx=0x7f5e9802a660 comp rx=0 tx=0).stop 2024-05-30T17:05:04.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.684+0000 7f5ea5400640 1 -- 172.21.15.67:0/4100967178 shutdown_connections 2024-05-30T17:05:04.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.684+0000 7f5ea5400640 1 --2- 172.21.15.67:0/4100967178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00cf180 0x7f5ea00cf610 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.684+0000 7f5ea5400640 1 --2- 172.21.15.67:0/4100967178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea00d3220 0x7f5ea00d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.684+0000 7f5ea5400640 1 -- 172.21.15.67:0/4100967178 >> 172.21.15.67:0/4100967178 conn(0x7f5ea008bfc0 msgr2=0x7f5ea00d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:04.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.684+0000 7f5ea5400640 1 -- 172.21.15.67:0/4100967178 shutdown_connections 2024-05-30T17:05:04.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.684+0000 7f5ea5400640 1 -- 172.21.15.67:0/4100967178 wait complete. 2024-05-30T17:05:04.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.686+0000 7f5ea5400640 1 Processor -- start 2024-05-30T17:05:04.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.686+0000 7f5ea5400640 1 -- start start 2024-05-30T17:05:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5ea5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ea00cf180 0x7f5ea00ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5ea5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00d3220 0x7f5ea015cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5ea5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea01621d0 0x7f5ea01f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5ea5400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00d5470 con 0x7f5ea00d3220 2024-05-30T17:05:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5ea5400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00d1ef0 con 0x7f5ea00cf180 2024-05-30T17:05:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5ea5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea015fa20 con 0x7f5ea01621d0 2024-05-30T17:05:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5e97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00d3220 0x7f5ea015cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5e97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00d3220 0x7f5ea015cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56974/0 (socket says 172.21.15.67:56974) 2024-05-30T17:05:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5e97400640 1 -- 172.21.15.67:0/3338712002 learned_addr learned my addr 172.21.15.67:0/3338712002 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:04.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5e97400640 1 -- 172.21.15.67:0/3338712002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea01621d0 msgr2=0x7f5ea01f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5e97400640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea01621d0 0x7f5ea01f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5e97400640 1 -- 172.21.15.67:0/3338712002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ea00cf180 msgr2=0x7f5ea00ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:04.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5e97400640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ea00cf180 0x7f5ea00ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:04.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.687+0000 7f5e97400640 1 -- 172.21.15.67:0/3338712002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea01f5870 con 0x7f5ea00d3220 2024-05-30T17:05:04.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.688+0000 7f5e97400640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00d3220 0x7f5ea015cd40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800c7f0 tx=0x7f5e8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:04.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.688+0000 7f5e9cc00640 1 -- 172.21.15.67:0/3338712002 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88004220 con 0x7f5ea00d3220 2024-05-30T17:05:04.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.688+0000 7f5e9cc00640 1 -- 172.21.15.67:0/3338712002 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e880043c0 con 0x7f5ea00d3220 2024-05-30T17:05:04.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.688+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea00cc720 con 0x7f5ea00d3220 2024-05-30T17:05:04.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.688+0000 7f5e9cc00640 1 -- 172.21.15.67:0/3338712002 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88005bb0 con 0x7f5ea00d3220 2024-05-30T17:05:04.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.688+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea00cc9e0 con 0x7f5ea00d3220 2024-05-30T17:05:04.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.689+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea00689e0 con 0x7f5ea00d3220 2024-05-30T17:05:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.690+0000 7f5e9cc00640 1 -- 172.21.15.67:0/3338712002 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f5e88006030 con 0x7f5ea00d3220 2024-05-30T17:05:04.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.691+0000 7f5e9cc00640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5e7c06e510 0x7f5e7c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:04.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.691+0000 7f5e9cc00640 1 -- 172.21.15.67:0/3338712002 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f5e8808b000 con 0x7f5ea00d3220 2024-05-30T17:05:04.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.692+0000 7f5e9fe00640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5e7c06e510 0x7f5e7c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:04.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.692+0000 7f5e9fe00640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5e7c06e510 0x7f5e7c0709d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007fd0 tx=0x7f5e90007960 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:04.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.695+0000 7f5e9cc00640 1 -- 172.21.15.67:0/3338712002 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e8801e020 con 0x7f5ea00d3220 2024-05-30T17:05:04.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:04.923+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f5ea00ba580 con 0x7f5e7c06e510 2024-05-30T17:05:04.931 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.931+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:04.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.931+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:04.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.931+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:04.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.931+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:04.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:04.931+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:05.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.009+0000 7f5e9cc00640 1 -- 172.21.15.67:0/3338712002 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ea00ba580 con 0x7f5e7c06e510 2024-05-30T17:05:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5e7c06e510 msgr2=0x7f5e7c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5e7c06e510 0x7f5e7c0709d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007fd0 tx=0x7f5e90007960 comp rx=0 tx=0).stop 2024-05-30T17:05:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00d3220 msgr2=0x7f5ea015cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00d3220 0x7f5ea015cd40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800c7f0 tx=0x7f5e8800ccb0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 shutdown_connections 2024-05-30T17:05:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f5e7c06e510 0x7f5e7c0709d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5ea01621d0 0x7f5ea01f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ea00d3220 0x7f5ea015cd40 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 --2- 172.21.15.67:0/3338712002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ea00cf180 0x7f5ea00ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.012+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 >> 172.21.15.67:0/3338712002 conn(0x7f5ea008bfc0 msgr2=0x7f5ea015dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.013+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 shutdown_connections 2024-05-30T17:05:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.013+0000 7f5ea5400640 1 -- 172.21.15.67:0/3338712002 wait complete. 2024-05-30T17:05:05.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T17:05:05.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.289+0000 7f9b91600640 1 Processor -- start 2024-05-30T17:05:05.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.289+0000 7f9b91600640 1 -- start start 2024-05-30T17:05:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.291+0000 7f9b91600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b8c0d0640 0x7f9b8c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.291+0000 7f9b91600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b8c0b32c0 0x7f9b8c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.291+0000 7f9b91600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8c0d0060 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.291+0000 7f9b91600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c0adbb0 con 0x7f9b8c0b32c0 2024-05-30T17:05:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.291+0000 7f9b91600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c15db10 con 0x7f9b8c0d0640 2024-05-30T17:05:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.291+0000 7f9b8b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b8c0b32c0 0x7f9b8c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.291+0000 7f9b8a000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9b8c0d0a20 0x7f9b8c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60070/0 (socket says 172.21.15.67:60070) 2024-05-30T17:05:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.291+0000 7f9b8b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b8c0b32c0 0x7f9b8c0b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57006/0 (socket says 172.21.15.67:57006) 2024-05-30T17:05:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.291+0000 7f9b8a000640 1 -- 172.21.15.67:0/857405900 learned_addr learned my addr 172.21.15.67:0/857405900 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:05:05.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.292+0000 7f9b8aa00640 1 --2- 172.21.15.67:0/857405900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b8c0d0640 0x7f9b8c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.292+0000 7f9b8b400640 1 -- 172.21.15.67:0/857405900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b8c0d0640 msgr2=0x7f9b8c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:05.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.292+0000 7f9b8b400640 1 --2- 172.21.15.67:0/857405900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b8c0d0640 0x7f9b8c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.292+0000 7f9b8b400640 1 -- 172.21.15.67:0/857405900 >> v1:172.21.15.67:6789/0 conn(0x7f9b8c0d0a20 legacy=0x7f9b8c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.292+0000 7f9b8b400640 1 -- 172.21.15.67:0/857405900 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c15cb10 con 0x7f9b8c0b32c0 2024-05-30T17:05:05.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.292+0000 7f9b89600640 1 -- 172.21.15.67:0/857405900 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2121543870 0 0) 0x7f9b8c0d0060 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.293+0000 7f9b8b400640 1 --2- 172.21.15.67:0/857405900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0b32c0 0x7f9b8c0b3750 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9b840097a0 tx=0x7f9b8402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d73ed8556200ac server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.293+0000 7f9b89600640 1 -- 172.21.15.67:0/857405900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84037070 con 0x7f9b8c0b32c0 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.293+0000 7f9b89600640 1 -- 172.21.15.67:0/857405900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b84032440 con 0x7f9b8c0b32c0 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.293+0000 7f9b89600640 1 -- 172.21.15.67:0/857405900 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84032810 con 0x7f9b8c0b32c0 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.294+0000 7f9b91600640 1 -- 172.21.15.67:0/857405900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0b32c0 msgr2=0x7f9b8c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.294+0000 7f9b91600640 1 --2- 172.21.15.67:0/857405900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0b32c0 0x7f9b8c0b3750 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9b840097a0 tx=0x7f9b8402a180 comp rx=0 tx=0).stop 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.294+0000 7f9b91600640 1 -- 172.21.15.67:0/857405900 shutdown_connections 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.294+0000 7f9b91600640 1 --2- 172.21.15.67:0/857405900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0b32c0 0x7f9b8c0b3750 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.294+0000 7f9b91600640 1 --2- 172.21.15.67:0/857405900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b8c0d0640 0x7f9b8c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.294+0000 7f9b91600640 1 -- 172.21.15.67:0/857405900 >> 172.21.15.67:0/857405900 conn(0x7f9b8c06bd20 msgr2=0x7f9b8c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:05.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.294+0000 7f9b91600640 1 -- 172.21.15.67:0/857405900 shutdown_connections 2024-05-30T17:05:05.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.294+0000 7f9b91600640 1 -- 172.21.15.67:0/857405900 wait complete. 2024-05-30T17:05:05.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.296+0000 7f9b91600640 1 Processor -- start 2024-05-30T17:05:05.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.296+0000 7f9b91600640 1 -- start start 2024-05-30T17:05:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.297+0000 7f9b91600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c0b32c0 0x7f9b8c0b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.297+0000 7f9b91600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b8c0d0640 0x7f9b8c152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.297+0000 7f9b91600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0d0a20 0x7f9b8c0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.297+0000 7f9b91600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c15cdd0 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.297+0000 7f9b91600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c0ad8d0 con 0x7f9b8c0b32c0 2024-05-30T17:05:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.297+0000 7f9b91600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c15d0b0 con 0x7f9b8c0d0640 2024-05-30T17:05:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0d0a20 0x7f9b8c0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0d0a20 0x7f9b8c0b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57014/0 (socket says 172.21.15.67:57014) 2024-05-30T17:05:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8b400640 1 -- 172.21.15.67:0/2912465113 learned_addr learned my addr 172.21.15.67:0/2912465113 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:05.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8aa00640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c0b32c0 0x7f9b8c0b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8b400640 1 -- 172.21.15.67:0/2912465113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b8c0d0640 msgr2=0x7f9b8c152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8b400640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b8c0d0640 0x7f9b8c152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8b400640 1 -- 172.21.15.67:0/2912465113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c0b32c0 msgr2=0x7f9b8c0b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8b400640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c0b32c0 0x7f9b8c0b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8b400640 1 -- 172.21.15.67:0/2912465113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c0b0060 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8aa00640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c0b32c0 0x7f9b8c0b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.298+0000 7f9b8b400640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0d0a20 0x7f9b8c0b0e20 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f9b840098c0 tx=0x7f9b840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.299+0000 7f9b82a00640 1 -- 172.21.15.67:0/2912465113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84037040 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.299+0000 7f9b82a00640 1 -- 172.21.15.67:0/2912465113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b84042070 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.299+0000 7f9b82a00640 1 -- 172.21.15.67:0/2912465113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84030b50 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.299+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c0b0340 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.299+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c0b0900 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.300+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c0689e0 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.301+0000 7f9b82a00640 1 -- 172.21.15.67:0/2912465113 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 43) ==== 93590+0+0 (secure 0 0 0) 0x7f9b84030cf0 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.302+0000 7f9b82a00640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b6c06e510 0x7f9b6c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.302+0000 7f9b82a00640 1 -- 172.21.15.67:0/2912465113 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f9b840ab640 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.302+0000 7f9b8aa00640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b6c06e510 0x7f9b6c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.303+0000 7f9b8aa00640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b6c06e510 0x7f9b6c0709d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c007330 tx=0x7f9b7c00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:05.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.306+0000 7f9b82a00640 1 -- 172.21.15.67:0/2912465113 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b84032e20 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.496+0000 7f9b82a00640 1 -- 172.21.15.67:0/2912465113 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 44) ==== 93641+0+0 (secure 0 0 0) 0x7f9b84077440 con 0x7f9b8c0d0a20 2024-05-30T17:05:05.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.549+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f9b8c0b2140 con 0x7f9b6c06e510 2024-05-30T17:05:05.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.559+0000 7f9b82a00640 1 -- 172.21.15.67:0/2912465113 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f9b8c0b2140 con 0x7f9b6c06e510 2024-05-30T17:05:05.560 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b6c06e510 msgr2=0x7f9b6c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b6c06e510 0x7f9b6c0709d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c007330 tx=0x7f9b7c00b040 comp rx=0 tx=0).stop 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0d0a20 msgr2=0x7f9b8c0b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0d0a20 0x7f9b8c0b0e20 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f9b840098c0 tx=0x7f9b840046c0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 shutdown_connections 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9b6c06e510 0x7f9b6c0709d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8c0d0a20 0x7f9b8c0b0e20 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b8c0d0640 0x7f9b8c152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 --2- 172.21.15.67:0/2912465113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c0b32c0 0x7f9b8c0b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 >> 172.21.15.67:0/2912465113 conn(0x7f9b8c06bd20 msgr2=0x7f9b8c150670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:05.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 shutdown_connections 2024-05-30T17:05:05.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.562+0000 7f9b91600640 1 -- 172.21.15.67:0/2912465113 wait complete. 2024-05-30T17:05:05.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T17:05:05.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.794+0000 7fe2b5800640 1 Processor -- start 2024-05-30T17:05:05.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.795+0000 7fe2b5800640 1 -- start start 2024-05-30T17:05:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.796+0000 7fe2b5800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe2b015e870 0x7fe2b00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.796+0000 7fe2b5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b00cfa40 0x7fe2b00d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.796+0000 7fe2b5800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b015ec50 con 0x7fe2b00d14d0 2024-05-30T17:05:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.796+0000 7fe2b5800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b00d2150 con 0x7fe2b015e870 2024-05-30T17:05:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.796+0000 7fe2b5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b00cff80 con 0x7fe2b00cfa40 2024-05-30T17:05:05.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.797+0000 7fe2afe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe2b015e870 0x7fe2b00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.797+0000 7fe2afe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe2b015e870 0x7fe2b00d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:51832/0 (socket says 172.21.15.67:51832) 2024-05-30T17:05:05.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.797+0000 7fe2afe00640 1 -- 172.21.15.67:0/1443736656 learned_addr learned my addr 172.21.15.67:0/1443736656 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:05.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.797+0000 7fe2b4e00640 1 --2- 172.21.15.67:0/1443736656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b00cfa40 0x7fe2b00d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.798+0000 7fe2b4e00640 1 -- 172.21.15.67:0/1443736656 >> v2:172.21.15.169:3300/0 conn(0x7fe2b015e870 msgr2=0x7fe2b00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:05.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.798+0000 7fe2b4e00640 1 --2- 172.21.15.67:0/1443736656 >> v2:172.21.15.169:3300/0 conn(0x7fe2b015e870 0x7fe2b00d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.798+0000 7fe2b4e00640 1 -- 172.21.15.67:0/1443736656 >> v1:172.21.15.169:6789/0 conn(0x7fe2b00d14d0 legacy=0x7fe2b00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.798+0000 7fe2b4e00640 1 -- 172.21.15.67:0/1443736656 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b00d0260 con 0x7fe2b00cfa40 2024-05-30T17:05:05.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.798+0000 7fe2afe00640 1 --2- 172.21.15.67:0/1443736656 >> v2:172.21.15.169:3300/0 conn(0x7fe2b015e870 0x7fe2b00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:05:05.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.800+0000 7fe2b4e00640 1 --2- 172.21.15.67:0/1443736656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b00cfa40 0x7fe2b00d25e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80097a0 tx=0x7fe2a802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100bc880ab0208c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:05.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.800+0000 7fe2aea00640 1 -- 172.21.15.67:0/1443736656 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8037070 con 0x7fe2b00cfa40 2024-05-30T17:05:05.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.800+0000 7fe2aea00640 1 -- 172.21.15.67:0/1443736656 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a8032440 con 0x7fe2b00cfa40 2024-05-30T17:05:05.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.800+0000 7fe2aea00640 1 -- 172.21.15.67:0/1443736656 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8032810 con 0x7fe2b00cfa40 2024-05-30T17:05:05.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.801+0000 7fe2b5800640 1 -- 172.21.15.67:0/1443736656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b00cfa40 msgr2=0x7fe2b00d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.801+0000 7fe2b5800640 1 --2- 172.21.15.67:0/1443736656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b00cfa40 0x7fe2b00d25e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80097a0 tx=0x7fe2a802a180 comp rx=0 tx=0).stop 2024-05-30T17:05:05.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.801+0000 7fe2b5800640 1 -- 172.21.15.67:0/1443736656 shutdown_connections 2024-05-30T17:05:05.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.801+0000 7fe2b5800640 1 --2- 172.21.15.67:0/1443736656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b00cfa40 0x7fe2b00d25e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.801+0000 7fe2b5800640 1 --2- 172.21.15.67:0/1443736656 >> v2:172.21.15.169:3300/0 conn(0x7fe2b015e870 0x7fe2b00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.801+0000 7fe2b5800640 1 -- 172.21.15.67:0/1443736656 >> 172.21.15.67:0/1443736656 conn(0x7fe2b008bfc0 msgr2=0x7fe2b00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:05.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.801+0000 7fe2b5800640 1 -- 172.21.15.67:0/1443736656 shutdown_connections 2024-05-30T17:05:05.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.801+0000 7fe2b5800640 1 -- 172.21.15.67:0/1443736656 wait complete. 2024-05-30T17:05:05.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.803+0000 7fe2b5800640 1 Processor -- start 2024-05-30T17:05:05.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.803+0000 7fe2b5800640 1 -- start start 2024-05-30T17:05:05.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.804+0000 7fe2b5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b00cfa40 0x7fe2b01593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.804+0000 7fe2b5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b00d14d0 0x7fe2b01618d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.804+0000 7fe2b5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b015e870 0x7fe2b015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.804+0000 7fe2b5800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b00d4fb0 con 0x7fe2b00d14d0 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.804+0000 7fe2b5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b00d0820 con 0x7fe2b00cfa40 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.804+0000 7fe2b5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b00d2b20 con 0x7fe2b015e870 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.804+0000 7fe2af400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b00d14d0 0x7fe2b01618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.804+0000 7fe2af400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b00d14d0 0x7fe2b01618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57016/0 (socket says 172.21.15.67:57016) 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.804+0000 7fe2af400640 1 -- 172.21.15.67:0/2230017090 learned_addr learned my addr 172.21.15.67:0/2230017090 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.805+0000 7fe2b4e00640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b015e870 0x7fe2b015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.805+0000 7fe2afe00640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b00cfa40 0x7fe2b01593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.805+0000 7fe2af400640 1 -- 172.21.15.67:0/2230017090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b015e870 msgr2=0x7fe2b015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.805+0000 7fe2af400640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b015e870 0x7fe2b015ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.805+0000 7fe2af400640 1 -- 172.21.15.67:0/2230017090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b00cfa40 msgr2=0x7fe2b01593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.805+0000 7fe2af400640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b00cfa40 0x7fe2b01593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.805+0000 7fe2af400640 1 -- 172.21.15.67:0/2230017090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b015eec0 con 0x7fe2b00d14d0 2024-05-30T17:05:05.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.805+0000 7fe2af400640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b00d14d0 0x7fe2b01618d0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe29800c7a0 tx=0x7fe29800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:05.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.805+0000 7fe2a7e00640 1 -- 172.21.15.67:0/2230017090 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298016020 con 0x7fe2b00d14d0 2024-05-30T17:05:05.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.806+0000 7fe2a7e00640 1 -- 172.21.15.67:0/2230017090 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe298006de0 con 0x7fe2b00d14d0 2024-05-30T17:05:05.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.806+0000 7fe2a7e00640 1 -- 172.21.15.67:0/2230017090 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298003c00 con 0x7fe2b00d14d0 2024-05-30T17:05:05.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.806+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b015f160 con 0x7fe2b00d14d0 2024-05-30T17:05:05.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.806+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b015f720 con 0x7fe2b00d14d0 2024-05-30T17:05:05.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.807+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b00689e0 con 0x7fe2b00d14d0 2024-05-30T17:05:05.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.808+0000 7fe2a7e00640 1 -- 172.21.15.67:0/2230017090 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 44) ==== 93641+0+0 (secure 0 0 0) 0x7fe29800a850 con 0x7fe2b00d14d0 2024-05-30T17:05:05.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.809+0000 7fe2a7e00640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe28c06e5a0 0x7fe28c070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:05.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.809+0000 7fe2a7e00640 1 -- 172.21.15.67:0/2230017090 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7fe29808a8d0 con 0x7fe2b00d14d0 2024-05-30T17:05:05.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.809+0000 7fe2afe00640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe28c06e5a0 0x7fe28c070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:05.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.810+0000 7fe2afe00640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe28c06e5a0 0x7fe28c070a60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ac0 tx=0x7fe2a0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:05.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:05.813+0000 7fe2a7e00640 1 -- 172.21.15.67:0/2230017090 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe298020020 con 0x7fe2b00d14d0 2024-05-30T17:05:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.049+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe2b00b1470 con 0x7fe28c06e5a0 2024-05-30T17:05:06.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.059+0000 7fe2a7e00640 1 -- 172.21.15.67:0/2230017090 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fe2b00b1470 con 0x7fe28c06e5a0 2024-05-30T17:05:06.060 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:06.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.062+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe28c06e5a0 msgr2=0x7fe28c070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.062+0000 7fe2b5800640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe28c06e5a0 0x7fe28c070a60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ac0 tx=0x7fe2a0008040 comp rx=0 tx=0).stop 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b00d14d0 msgr2=0x7fe2b01618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b00d14d0 0x7fe2b01618d0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe29800c7a0 tx=0x7fe29800cc60 comp rx=0 tx=0).stop 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 shutdown_connections 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe28c06e5a0 0x7fe28c070a60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2b015e870 0x7fe2b015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b00d14d0 0x7fe2b01618d0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 --2- 172.21.15.67:0/2230017090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2b00cfa40 0x7fe2b01593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 >> 172.21.15.67:0/2230017090 conn(0x7fe2b008bfc0 msgr2=0x7fe2b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 shutdown_connections 2024-05-30T17:05:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:06.063+0000 7fe2b5800640 1 -- 172.21.15.67:0/2230017090 wait complete. 2024-05-30T17:05:06.085 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:06.185 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:05:06.185 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:05:06.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:06.245 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T17:05:16.284 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:05:16.294 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:05:22.991 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T17:05:22.991 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T17:05:22.991 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.6521 s, 64.5 MB/s 2024-05-30T17:05:22.993 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:05:22.993 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:05:23.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:23.052 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T17:05:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T17:05:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T17:05:38.462 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.3454 s, 70.0 MB/s 2024-05-30T17:05:38.464 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:05:38.464 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:05:38.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:38.524 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-30T17:05:46.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:05:46.431 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:05:52.780 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records in 2024-05-30T17:05:52.780 INFO:teuthology.orchestra.run.smithi067.stderr:1024+0 records out 2024-05-30T17:05:52.781 INFO:teuthology.orchestra.run.smithi067.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.1885 s, 75.7 MB/s 2024-05-30T17:05:52.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-30T17:05:53.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.000+0000 7f4e85800640 1 Processor -- start 2024-05-30T17:05:53.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.001+0000 7f4e85800640 1 -- start start 2024-05-30T17:05:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.002+0000 7f4e85800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e80072310 0x7f4e8006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.002+0000 7f4e85800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4e800b3510 0x7f4e8006f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.002+0000 7f4e85800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e8006e190 con 0x7f4e8015e830 2024-05-30T17:05:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.002+0000 7f4e85800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8006f770 con 0x7f4e80072310 2024-05-30T17:05:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.002+0000 7f4e85800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8006e470 con 0x7f4e800b3510 2024-05-30T17:05:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.002+0000 7f4e76000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e80072310 0x7f4e8006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.003+0000 7f4e7ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4e8015e830 0x7f4e80071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37560/0 (socket says 172.21.15.67:37560) 2024-05-30T17:05:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.003+0000 7f4e7ea00640 1 -- 172.21.15.67:0/1009867031 learned_addr learned my addr 172.21.15.67:0/1009867031 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:05:53.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.003+0000 7f4e7f400640 1 --2- 172.21.15.67:0/1009867031 >> v2:172.21.15.169:3300/0 conn(0x7f4e800b3510 0x7f4e8006f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.003+0000 7f4e76000640 1 -- 172.21.15.67:0/1009867031 >> v2:172.21.15.169:3300/0 conn(0x7f4e800b3510 msgr2=0x7f4e8006f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:53.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.003+0000 7f4e76000640 1 --2- 172.21.15.67:0/1009867031 >> v2:172.21.15.169:3300/0 conn(0x7f4e800b3510 0x7f4e8006f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.003+0000 7f4e76000640 1 -- 172.21.15.67:0/1009867031 >> v1:172.21.15.67:6789/0 conn(0x7f4e8015e830 legacy=0x7f4e80071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.003+0000 7f4e76000640 1 -- 172.21.15.67:0/1009867031 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8006e750 con 0x7f4e80072310 2024-05-30T17:05:53.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.003+0000 7f4e7e000640 1 -- 172.21.15.67:0/1009867031 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713644385 0 0) 0x7f4e8006e190 con 0x7f4e8015e830 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.004+0000 7f4e76000640 1 --2- 172.21.15.67:0/1009867031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 0x7f4e8006dbb0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f4e8006fbe0 tx=0x7f4e6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fca1dcf9dae7063 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e7e000640 1 -- 172.21.15.67:0/1009867031 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68037070 con 0x7f4e80072310 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e7e000640 1 -- 172.21.15.67:0/1009867031 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e68032440 con 0x7f4e80072310 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e7e000640 1 -- 172.21.15.67:0/1009867031 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68032810 con 0x7f4e80072310 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e85800640 1 -- 172.21.15.67:0/1009867031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 msgr2=0x7f4e8006dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e85800640 1 --2- 172.21.15.67:0/1009867031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 0x7f4e8006dbb0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f4e8006fbe0 tx=0x7f4e6802a1a0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e85800640 1 -- 172.21.15.67:0/1009867031 shutdown_connections 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e85800640 1 --2- 172.21.15.67:0/1009867031 >> v2:172.21.15.169:3300/0 conn(0x7f4e800b3510 0x7f4e8006f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e85800640 1 --2- 172.21.15.67:0/1009867031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 0x7f4e8006dbb0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e85800640 1 -- 172.21.15.67:0/1009867031 >> 172.21.15.67:0/1009867031 conn(0x7f4e800b1e50 msgr2=0x7f4e800b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:53.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e85800640 1 -- 172.21.15.67:0/1009867031 shutdown_connections 2024-05-30T17:05:53.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.005+0000 7f4e85800640 1 -- 172.21.15.67:0/1009867031 wait complete. 2024-05-30T17:05:53.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.007+0000 7f4e85800640 1 Processor -- start 2024-05-30T17:05:53.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.007+0000 7f4e85800640 1 -- start start 2024-05-30T17:05:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.008+0000 7f4e85800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 0x7f4e801f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.008+0000 7f4e85800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e800b3510 0x7f4e8009cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.008+0000 7f4e85800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8015e830 0x7f4e8009d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.008+0000 7f4e85800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8015ec10 con 0x7f4e80072310 2024-05-30T17:05:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.008+0000 7f4e85800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8006ed10 con 0x7f4e8015e830 2024-05-30T17:05:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.008+0000 7f4e85800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4e800728c0 con 0x7f4e800b3510 2024-05-30T17:05:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e7ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 0x7f4e801f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e7ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 0x7f4e801f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45374/0 (socket says 172.21.15.67:45374) 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e7ea00640 1 -- 172.21.15.67:0/4148567071 learned_addr learned my addr 172.21.15.67:0/4148567071 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e76000640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e800b3510 0x7f4e8009cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e7f400640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8015e830 0x7f4e8009d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e7ea00640 1 -- 172.21.15.67:0/4148567071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e800b3510 msgr2=0x7f4e8009cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e7ea00640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e800b3510 0x7f4e8009cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e7ea00640 1 -- 172.21.15.67:0/4148567071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8015e830 msgr2=0x7f4e8009d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e7ea00640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8015e830 0x7f4e8009d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.009+0000 7f4e7ea00640 1 -- 172.21.15.67:0/4148567071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e801f41a0 con 0x7f4e80072310 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.010+0000 7f4e7ea00640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 0x7f4e801f3c60 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c460 tx=0x7f4e7000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.010+0000 7f4e77400640 1 -- 172.21.15.67:0/4148567071 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70018070 con 0x7f4e80072310 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.010+0000 7f4e77400640 1 -- 172.21.15.67:0/4148567071 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e70007500 con 0x7f4e80072310 2024-05-30T17:05:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.010+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e80160b30 con 0x7f4e80072310 2024-05-30T17:05:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.010+0000 7f4e77400640 1 -- 172.21.15.67:0/4148567071 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70004a20 con 0x7f4e80072310 2024-05-30T17:05:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.010+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8014c710 con 0x7f4e80072310 2024-05-30T17:05:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.011+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e800689e0 con 0x7f4e80072310 2024-05-30T17:05:53.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.012+0000 7f4e77400640 1 -- 172.21.15.67:0/4148567071 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 45) ==== 93692+0+0 (secure 0 0 0) 0x7f4e7001b070 con 0x7f4e80072310 2024-05-30T17:05:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.013+0000 7f4e77400640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e5c06e630 0x7f4e5c070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.014+0000 7f4e77400640 1 -- 172.21.15.67:0/4148567071 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f4e70089d20 con 0x7f4e80072310 2024-05-30T17:05:53.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.014+0000 7f4e76000640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e5c06e630 0x7f4e5c070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.015+0000 7f4e76000640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e5c06e630 0x7f4e5c070af0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4e68034000 tx=0x7f4e6802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:53.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.018+0000 7f4e77400640 1 -- 172.21.15.67:0/4148567071 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e70059560 con 0x7f4e80072310 2024-05-30T17:05:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.254+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f4e80071d00 con 0x7f4e5c06e630 2024-05-30T17:05:53.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.293+0000 7f4e77400640 1 -- 172.21.15.67:0/4148567071 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e80071d00 con 0x7f4e5c06e630 2024-05-30T17:05:53.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.296+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e5c06e630 msgr2=0x7f4e5c070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.296+0000 7f4e85800640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e5c06e630 0x7f4e5c070af0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4e68034000 tx=0x7f4e6802e000 comp rx=0 tx=0).stop 2024-05-30T17:05:53.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.296+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 msgr2=0x7f4e801f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.296+0000 7f4e85800640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 0x7f4e801f3c60 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c460 tx=0x7f4e7000c920 comp rx=0 tx=0).stop 2024-05-30T17:05:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.297+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 shutdown_connections 2024-05-30T17:05:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.297+0000 7f4e85800640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4e5c06e630 0x7f4e5c070af0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.297+0000 7f4e85800640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8015e830 0x7f4e8009d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.297+0000 7f4e85800640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e800b3510 0x7f4e8009cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.297+0000 7f4e85800640 1 --2- 172.21.15.67:0/4148567071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e80072310 0x7f4e801f3c60 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.297+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 >> 172.21.15.67:0/4148567071 conn(0x7f4e800b1e50 msgr2=0x7f4e8015d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.297+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 shutdown_connections 2024-05-30T17:05:53.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.297+0000 7f4e85800640 1 -- 172.21.15.67:0/4148567071 wait complete. 2024-05-30T17:05:53.316 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-30T17:05:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.530+0000 7f272f400640 1 Processor -- start 2024-05-30T17:05:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.530+0000 7f272f400640 1 -- start start 2024-05-30T17:05:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.532+0000 7f272f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27280d0640 0x7f272815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.532+0000 7f272f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 0x7f27280cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.532+0000 7f272f400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f272815cb10 con 0x7f27280b32c0 2024-05-30T17:05:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.532+0000 7f272f400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f27280adbb0 con 0x7f27280d0640 2024-05-30T17:05:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.532+0000 7f272f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f272815db10 con 0x7f27280d0a20 2024-05-30T17:05:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.532+0000 7f272ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27280d0640 0x7f272815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.532+0000 7f272ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27280d0640 0x7f272815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45376/0 (socket says 172.21.15.67:45376) 2024-05-30T17:05:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.532+0000 7f272ca00640 1 -- 172.21.15.67:0/4248580056 learned_addr learned my addr 172.21.15.67:0/4248580056 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.533+0000 7f2727e00640 1 --2- 172.21.15.67:0/4248580056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 0x7f27280cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.533+0000 7f272ca00640 1 -- 172.21.15.67:0/4248580056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 msgr2=0x7f27280cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.533+0000 7f272ca00640 1 --2- 172.21.15.67:0/4248580056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 0x7f27280cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.533+0000 7f272ca00640 1 -- 172.21.15.67:0/4248580056 >> v1:172.21.15.169:6789/0 conn(0x7f27280b32c0 legacy=0x7f27280b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.533+0000 7f272ca00640 1 -- 172.21.15.67:0/4248580056 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27280ad5f0 con 0x7f27280d0640 2024-05-30T17:05:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.533+0000 7f2727e00640 1 --2- 172.21.15.67:0/4248580056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 0x7f27280cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.534+0000 7f272ca00640 1 --2- 172.21.15.67:0/4248580056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280d0640 0x7f272815d5d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f271c009870 tx=0x7f271c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2685de7fa3ba141 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.534+0000 7f2727400640 1 -- 172.21.15.67:0/4248580056 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c037070 con 0x7f27280d0640 2024-05-30T17:05:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.534+0000 7f2727400640 1 -- 172.21.15.67:0/4248580056 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c032440 con 0x7f27280d0640 2024-05-30T17:05:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.534+0000 7f2727400640 1 -- 172.21.15.67:0/4248580056 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c032810 con 0x7f27280d0640 2024-05-30T17:05:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.535+0000 7f272f400640 1 -- 172.21.15.67:0/4248580056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280d0640 msgr2=0x7f272815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.535+0000 7f272f400640 1 --2- 172.21.15.67:0/4248580056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280d0640 0x7f272815d5d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f271c009870 tx=0x7f271c02a210 comp rx=0 tx=0).stop 2024-05-30T17:05:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.535+0000 7f272f400640 1 -- 172.21.15.67:0/4248580056 shutdown_connections 2024-05-30T17:05:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.535+0000 7f272f400640 1 --2- 172.21.15.67:0/4248580056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 0x7f27280cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.535+0000 7f272f400640 1 --2- 172.21.15.67:0/4248580056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280d0640 0x7f272815d5d0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.535+0000 7f272f400640 1 -- 172.21.15.67:0/4248580056 >> 172.21.15.67:0/4248580056 conn(0x7f272806bd20 msgr2=0x7f272806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.535+0000 7f272f400640 1 -- 172.21.15.67:0/4248580056 shutdown_connections 2024-05-30T17:05:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.535+0000 7f272f400640 1 -- 172.21.15.67:0/4248580056 wait complete. 2024-05-30T17:05:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.537+0000 7f272f400640 1 Processor -- start 2024-05-30T17:05:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.537+0000 7f272f400640 1 -- start start 2024-05-30T17:05:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.538+0000 7f272f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280b32c0 0x7f272806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.538+0000 7f272f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27280d0640 0x7f272815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.538+0000 7f272f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 0x7f27281f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.538+0000 7f272f400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f272815d0b0 con 0x7f27280b32c0 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.538+0000 7f272f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f272815cdd0 con 0x7f27280d0640 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.538+0000 7f272f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f27280b3e60 con 0x7f27280d0a20 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.538+0000 7f272ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280b32c0 0x7f272806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.538+0000 7f272ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280b32c0 0x7f272806d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45392/0 (socket says 172.21.15.67:45392) 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.538+0000 7f272ca00640 1 -- 172.21.15.67:0/2075225879 learned_addr learned my addr 172.21.15.67:0/2075225879 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.539+0000 7f272ca00640 1 -- 172.21.15.67:0/2075225879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 msgr2=0x7f27281f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.539+0000 7f272ca00640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 0x7f27281f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.539+0000 7f272ca00640 1 -- 172.21.15.67:0/2075225879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27280d0640 msgr2=0x7f272815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.539+0000 7f272ca00640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27280d0640 0x7f272815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.539+0000 7f272ca00640 1 -- 172.21.15.67:0/2075225879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272806dfa0 con 0x7f27280b32c0 2024-05-30T17:05:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.539+0000 7f272ca00640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280b32c0 0x7f272806d9c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f271c009870 tx=0x7f271c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.540+0000 7f2724c00640 1 -- 172.21.15.67:0/2075225879 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c041020 con 0x7f27280b32c0 2024-05-30T17:05:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.540+0000 7f2724c00640 1 -- 172.21.15.67:0/2075225879 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c030970 con 0x7f27280b32c0 2024-05-30T17:05:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.540+0000 7f2724c00640 1 -- 172.21.15.67:0/2075225879 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c037040 con 0x7f27280b32c0 2024-05-30T17:05:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.540+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272806e280 con 0x7f27280b32c0 2024-05-30T17:05:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.540+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272806e840 con 0x7f27280b32c0 2024-05-30T17:05:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.541+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27280689e0 con 0x7f27280b32c0 2024-05-30T17:05:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.542+0000 7f2724c00640 1 -- 172.21.15.67:0/2075225879 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 45) ==== 93692+0+0 (secure 0 0 0) 0x7f271c044020 con 0x7f27280b32c0 2024-05-30T17:05:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.543+0000 7f2724c00640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f270806e630 0x7f2708070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.543+0000 7f2724c00640 1 -- 172.21.15.67:0/2075225879 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f271c0ab580 con 0x7f27280b32c0 2024-05-30T17:05:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.543+0000 7f2727e00640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f270806e630 0x7f2708070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.544+0000 7f2727e00640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f270806e630 0x7f2708070af0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2714006fd0 tx=0x7f2714008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:53.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.548+0000 7f2724c00640 1 -- 172.21.15.67:0/2075225879 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f271c04b030 con 0x7f27280b32c0 2024-05-30T17:05:53.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.782+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f27281f4460 con 0x7f270806e630 2024-05-30T17:05:53.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.793+0000 7f2724c00640 1 -- 172.21.15.67:0/2075225879 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f271400c7a0 con 0x7f270806e630 2024-05-30T17:05:53.794 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f270806e630 msgr2=0x7f2708070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f270806e630 0x7f2708070af0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2714006fd0 tx=0x7f2714008040 comp rx=0 tx=0).stop 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280b32c0 msgr2=0x7f272806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280b32c0 0x7f272806d9c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f271c009870 tx=0x7f271c0046c0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 shutdown_connections 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f270806e630 0x7f2708070af0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27280d0a20 0x7f27281f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27280d0640 0x7f272815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 --2- 172.21.15.67:0/2075225879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27280b32c0 0x7f272806d9c0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:53.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 >> 172.21.15.67:0/2075225879 conn(0x7f272806bd20 msgr2=0x7f272815f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:53.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 shutdown_connections 2024-05-30T17:05:53.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:53.796+0000 7f272f400640 1 -- 172.21.15.67:0/2075225879 wait complete. 2024-05-30T17:05:53.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:53.818 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:53.818 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T17:05:53.880 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:53.880 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3138256896" 2024-05-30T17:05:53.880 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:05:53.881 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-30T17:05:55.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:55.883 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:55.883 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T17:05:55.953 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:55.953 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3138256896" 2024-05-30T17:05:55.953 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:05:55.954 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-30T17:05:57.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:05:57.957 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:57.957 DEBUG:teuthology.orchestra.run.smithi067:> ) 2024-05-30T17:05:58.026 INFO:teuthology.orchestra.run.smithi067.stdout:# file: volumes/_nogroup/sv1/add2f0a4-3bfd-4b21-bbf9-b5afefbad4b3 2024-05-30T17:05:58.026 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.rbytes="3221225472" 2024-05-30T17:05:58.026 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:05:58.027 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-30T17:05:58.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96441 2024-05-30T17:05:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.244+0000 7fb6bde00640 1 Processor -- start 2024-05-30T17:05:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.245+0000 7fb6bde00640 1 -- start start 2024-05-30T17:05:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.246+0000 7fb6bde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.246+0000 7fb6bde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6b80affb0 0x7fb6b80ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.247+0000 7fb6bde00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6b80b4720 con 0x7fb6b80afbd0 2024-05-30T17:05:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.247+0000 7fb6bde00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b815da10 con 0x7fb6b80affb0 2024-05-30T17:05:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.247+0000 7fb6bde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b80b4a00 con 0x7fb6b806d020 2024-05-30T17:05:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.247+0000 7fb6b7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.247+0000 7fb6bd400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6b80affb0 0x7fb6b80ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.248+0000 7fb6b7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:60492/0 (socket says 172.21.15.67:60492) 2024-05-30T17:05:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.248+0000 7fb6b7e00640 1 -- 172.21.15.67:0/3396420836 learned_addr learned my addr 172.21.15.67:0/3396420836 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:58.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.248+0000 7fb6b7e00640 1 -- 172.21.15.67:0/3396420836 >> v2:172.21.15.169:3300/0 conn(0x7fb6b80affb0 msgr2=0x7fb6b80ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:58.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.248+0000 7fb6b7e00640 1 --2- 172.21.15.67:0/3396420836 >> v2:172.21.15.169:3300/0 conn(0x7fb6b80affb0 0x7fb6b80ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.248+0000 7fb6b7e00640 1 -- 172.21.15.67:0/3396420836 >> v1:172.21.15.169:6789/0 conn(0x7fb6b80afbd0 legacy=0x7fb6b806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.248+0000 7fb6b7e00640 1 -- 172.21.15.67:0/3396420836 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b80b4ce0 con 0x7fb6b806d020 2024-05-30T17:05:58.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.250+0000 7fb6b7e00640 1 --2- 172.21.15.67:0/3396420836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b815d4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009870 tx=0x7fb6a002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8ce45b8d9587ff server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:58.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.251+0000 7fb6b7400640 1 -- 172.21.15.67:0/3396420836 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0037070 con 0x7fb6b806d020 2024-05-30T17:05:58.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.251+0000 7fb6b7400640 1 -- 172.21.15.67:0/3396420836 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a0032440 con 0x7fb6b806d020 2024-05-30T17:05:58.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.251+0000 7fb6b7400640 1 -- 172.21.15.67:0/3396420836 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0032810 con 0x7fb6b806d020 2024-05-30T17:05:58.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.251+0000 7fb6bde00640 1 -- 172.21.15.67:0/3396420836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 msgr2=0x7fb6b815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.251+0000 7fb6bde00640 1 --2- 172.21.15.67:0/3396420836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b815d4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009870 tx=0x7fb6a002a270 comp rx=0 tx=0).stop 2024-05-30T17:05:58.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.252+0000 7fb6bde00640 1 -- 172.21.15.67:0/3396420836 shutdown_connections 2024-05-30T17:05:58.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.252+0000 7fb6bde00640 1 --2- 172.21.15.67:0/3396420836 >> v2:172.21.15.169:3300/0 conn(0x7fb6b80affb0 0x7fb6b80ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.252+0000 7fb6bde00640 1 --2- 172.21.15.67:0/3396420836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b815d4d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.252+0000 7fb6bde00640 1 -- 172.21.15.67:0/3396420836 >> 172.21.15.67:0/3396420836 conn(0x7fb6b8072260 msgr2=0x7fb6b8072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:58.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.252+0000 7fb6bde00640 1 -- 172.21.15.67:0/3396420836 shutdown_connections 2024-05-30T17:05:58.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.252+0000 7fb6bde00640 1 -- 172.21.15.67:0/3396420836 wait complete. 2024-05-30T17:05:58.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.254+0000 7fb6bde00640 1 Processor -- start 2024-05-30T17:05:58.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.254+0000 7fb6bde00640 1 -- start start 2024-05-30T17:05:58.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6bde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b813f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6bde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6b80afbd0 0x7fb6b814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6bde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b80affb0 0x7fb6b81516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6bde00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b80ccc50 con 0x7fb6b80afbd0 2024-05-30T17:05:58.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6bde00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b80b52a0 con 0x7fb6b80affb0 2024-05-30T17:05:58.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6bde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b80ccdd0 con 0x7fb6b806d020 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6b80afbd0 0x7fb6b814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6b80afbd0 0x7fb6b814f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45402/0 (socket says 172.21.15.67:45402) 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6b7e00640 1 -- 172.21.15.67:0/12925606 learned_addr learned my addr 172.21.15.67:0/12925606 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.256+0000 7fb6bca00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b813f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6bd400640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b80affb0 0x7fb6b81516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6b7e00640 1 -- 172.21.15.67:0/12925606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 msgr2=0x7fb6b813f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6b7e00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b813f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6b7e00640 1 -- 172.21.15.67:0/12925606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b80affb0 msgr2=0x7fb6b81516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6b7e00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b80affb0 0x7fb6b81516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6b7e00640 1 -- 172.21.15.67:0/12925606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b8154ea0 con 0x7fb6b80afbd0 2024-05-30T17:05:58.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6bd400640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b80affb0 0x7fb6b81516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:05:58.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6b7e00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6b80afbd0 0x7fb6b814f2c0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0000c00 tx=0x7fb6a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:58.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6b4c00640 1 -- 172.21.15.67:0/12925606 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0037040 con 0x7fb6b80afbd0 2024-05-30T17:05:58.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6b4c00640 1 -- 172.21.15.67:0/12925606 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a0042070 con 0x7fb6b80afbd0 2024-05-30T17:05:58.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.257+0000 7fb6b4c00640 1 -- 172.21.15.67:0/12925606 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0030880 con 0x7fb6b80afbd0 2024-05-30T17:05:58.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.258+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b8155180 con 0x7fb6b80afbd0 2024-05-30T17:05:58.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.258+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b8155740 con 0x7fb6b80afbd0 2024-05-30T17:05:58.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.259+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b8156960 con 0x7fb6b80afbd0 2024-05-30T17:05:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.260+0000 7fb6b4c00640 1 -- 172.21.15.67:0/12925606 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 45) ==== 93692+0+0 (secure 0 0 0) 0x7fb6a0030a20 con 0x7fb6b80afbd0 2024-05-30T17:05:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.261+0000 7fb6b4c00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69406e630 0x7fb694070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.261+0000 7fb6b4c00640 1 -- 172.21.15.67:0/12925606 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fb6a00ab430 con 0x7fb6b80afbd0 2024-05-30T17:05:58.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.261+0000 7fb6bca00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69406e630 0x7fb694070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.262+0000 7fb6bca00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69406e630 0x7fb694070af0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb6b815fc50 tx=0x7fb6a8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:58.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.266+0000 7fb6b4c00640 1 -- 172.21.15.67:0/12925606 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6a007b2c0 con 0x7fb6b80afbd0 2024-05-30T17:05:58.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.511+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96441", "target": ["mon-mgr", ""]}) -- 0x7fb6b80d0230 con 0x7fb69406e630 2024-05-30T17:05:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.686+0000 7fb6b4c00640 1 -- 172.21.15.67:0/12925606 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb6b80d0230 con 0x7fb69406e630 2024-05-30T17:05:58.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69406e630 msgr2=0x7fb694070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69406e630 0x7fb694070af0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb6b815fc50 tx=0x7fb6a8009040 comp rx=0 tx=0).stop 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6b80afbd0 msgr2=0x7fb6b814f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6b80afbd0 0x7fb6b814f2c0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0000c00 tx=0x7fb6a00046c0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 shutdown_connections 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb69406e630 0x7fb694070af0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b80affb0 0x7fb6b81516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6b80afbd0 0x7fb6b814f2c0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 --2- 172.21.15.67:0/12925606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806d020 0x7fb6b813f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 >> 172.21.15.67:0/12925606 conn(0x7fb6b8072260 msgr2=0x7fb6b815cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 shutdown_connections 2024-05-30T17:05:58.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.690+0000 7fb6bde00640 1 -- 172.21.15.67:0/12925606 wait complete. 2024-05-30T17:05:58.694 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.694+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.694 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.694+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.694 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.694+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.694 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.694+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.694 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.694+0000 7f0c91600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96442 2024-05-30T17:05:58.725 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.724+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.725 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.724+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.725 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.724+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.725 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.724+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.725 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:58.724+0000 7f0c90200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:58.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.900+0000 7f9d17200640 1 Processor -- start 2024-05-30T17:05:58.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.900+0000 7f9d17200640 1 -- start start 2024-05-30T17:05:58.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.902+0000 7f9d17200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 0x7f9d180b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.902+0000 7f9d17200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9d1815f210 0x7f9d180ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.902+0000 7f9d17200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d180af200 con 0x7f9d1815edd0 2024-05-30T17:05:58.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.902+0000 7f9d17200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1815fc30 con 0x7f9d1815f210 2024-05-30T17:05:58.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.902+0000 7f9d17200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9d180b4a40 con 0x7f9d1815b180 2024-05-30T17:05:58.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.902+0000 7f9d16800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9d1815f210 0x7f9d180ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.902+0000 7f9d16800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9d1815f210 0x7f9d180ae660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45454/0 (socket says 172.21.15.67:45454) 2024-05-30T17:05:58.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.902+0000 7f9d16800640 1 -- 172.21.15.67:0/1353214337 learned_addr learned my addr 172.21.15.67:0/1353214337 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:58.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.902+0000 7f9d15e00640 1 --2- 172.21.15.67:0/1353214337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 0x7f9d180b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.903+0000 7f9d16800640 1 -- 172.21.15.67:0/1353214337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 msgr2=0x7f9d180b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:05:58.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.903+0000 7f9d16800640 1 --2- 172.21.15.67:0/1353214337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 0x7f9d180b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.903+0000 7f9d16800640 1 -- 172.21.15.67:0/1353214337 >> v1:172.21.15.169:6789/0 conn(0x7f9d1815edd0 legacy=0x7f9d180adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.903+0000 7f9d16800640 1 -- 172.21.15.67:0/1353214337 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d180b3e30 con 0x7f9d1815f210 2024-05-30T17:05:58.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.904+0000 7f9d16800640 1 --2- 172.21.15.67:0/1353214337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815f210 0x7f9d180ae660 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009e50 tx=0x7f9d0802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceb956829917f47 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:58.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.904+0000 7f9d14a00640 1 -- 172.21.15.67:0/1353214337 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08036070 con 0x7f9d1815f210 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.904+0000 7f9d14a00640 1 -- 172.21.15.67:0/1353214337 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d08031600 con 0x7f9d1815f210 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.904+0000 7f9d14a00640 1 -- 172.21.15.67:0/1353214337 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080319d0 con 0x7f9d1815f210 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.905+0000 7f9d17200640 1 -- 172.21.15.67:0/1353214337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815f210 msgr2=0x7f9d180ae660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.905+0000 7f9d17200640 1 --2- 172.21.15.67:0/1353214337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815f210 0x7f9d180ae660 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009e50 tx=0x7f9d0802a710 comp rx=0 tx=0).stop 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.905+0000 7f9d17200640 1 -- 172.21.15.67:0/1353214337 shutdown_connections 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.905+0000 7f9d17200640 1 --2- 172.21.15.67:0/1353214337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815f210 0x7f9d180ae660 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.905+0000 7f9d17200640 1 --2- 172.21.15.67:0/1353214337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 0x7f9d180b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.905+0000 7f9d17200640 1 -- 172.21.15.67:0/1353214337 >> 172.21.15.67:0/1353214337 conn(0x7f9d180b1db0 msgr2=0x7f9d180b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.905+0000 7f9d17200640 1 -- 172.21.15.67:0/1353214337 shutdown_connections 2024-05-30T17:05:58.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.905+0000 7f9d17200640 1 -- 172.21.15.67:0/1353214337 wait complete. 2024-05-30T17:05:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.907+0000 7f9d17200640 1 Processor -- start 2024-05-30T17:05:58.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.907+0000 7f9d17200640 1 -- start start 2024-05-30T17:05:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.908+0000 7f9d17200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 0x7f9d180cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.908+0000 7f9d17200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815edd0 0x7f9d181f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.908+0000 7f9d17200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1815f210 0x7f9d180b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.908+0000 7f9d17200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9d180b46d0 con 0x7f9d1815edd0 2024-05-30T17:05:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.908+0000 7f9d17200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9d180b43f0 con 0x7f9d1815f210 2024-05-30T17:05:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.908+0000 7f9d17200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1815f6a0 con 0x7f9d1815b180 2024-05-30T17:05:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.908+0000 7f9d15400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815edd0 0x7f9d181f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.908+0000 7f9d15400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815edd0 0x7f9d181f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45464/0 (socket says 172.21.15.67:45464) 2024-05-30T17:05:58.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.908+0000 7f9d15400640 1 -- 172.21.15.67:0/1233066524 learned_addr learned my addr 172.21.15.67:0/1233066524 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:05:58.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d15e00640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 0x7f9d180cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d16800640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1815f210 0x7f9d180b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d15400640 1 -- 172.21.15.67:0/1233066524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 msgr2=0x7f9d180cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d15400640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 0x7f9d180cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d15400640 1 -- 172.21.15.67:0/1233066524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1815f210 msgr2=0x7f9d180b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d15400640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1815f210 0x7f9d180b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d15400640 1 -- 172.21.15.67:0/1233066524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d181f4dd0 con 0x7f9d1815edd0 2024-05-30T17:05:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d15400640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815edd0 0x7f9d181f4390 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c450 tx=0x7f9d0000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d0e000640 1 -- 172.21.15.67:0/1233066524 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d000052d0 con 0x7f9d1815edd0 2024-05-30T17:05:58.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d0e000640 1 -- 172.21.15.67:0/1233066524 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d00005470 con 0x7f9d1815edd0 2024-05-30T17:05:58.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d0e000640 1 -- 172.21.15.67:0/1233066524 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00012870 con 0x7f9d1815edd0 2024-05-30T17:05:58.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.909+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d181f50b0 con 0x7f9d1815edd0 2024-05-30T17:05:58.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.910+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d181f5670 con 0x7f9d1815edd0 2024-05-30T17:05:58.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.911+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d180689e0 con 0x7f9d1815edd0 2024-05-30T17:05:58.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.912+0000 7f9d0e000640 1 -- 172.21.15.67:0/1233066524 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 45) ==== 93692+0+0 (secure 0 0 0) 0x7f9d00012a10 con 0x7f9d1815edd0 2024-05-30T17:05:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.913+0000 7f9d0e000640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cf406e630 0x7f9cf4070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:05:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.913+0000 7f9d0e000640 1 -- 172.21.15.67:0/1233066524 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f9d0008a690 con 0x7f9d1815edd0 2024-05-30T17:05:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.913+0000 7f9d15e00640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cf406e630 0x7f9cf4070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:05:58.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.913+0000 7f9d15e00640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cf406e630 0x7f9cf4070af0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9d10006fd0 tx=0x7f9d10008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:05:58.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.917+0000 7f9d0e000640 1 -- 172.21.15.67:0/1233066524 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d0005a5a0 con 0x7f9d1815edd0 2024-05-30T17:05:58.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:58.938+0000 7f9d0e000640 1 -- 172.21.15.67:0/1233066524 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 46) ==== 93794+0+0 (secure 0 0 0) 0x7f9d00012d00 con 0x7f9d1815edd0 2024-05-30T17:05:59.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.153+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96442", "target": ["mon-mgr", ""]}) -- 0x7f9d1806f790 con 0x7f9cf406e630 2024-05-30T17:05:59.695 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.695+0000 7f0c11e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.695 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.695+0000 7f0c11e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.696 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.695+0000 7f0c11e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.696 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.695+0000 7f0c11e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.696 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.695+0000 7f0c11e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.911+0000 7f9d0e000640 1 -- 172.21.15.67:0/1233066524 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d1806f790 con 0x7f9cf406e630 2024-05-30T17:05:59.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.914+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cf406e630 msgr2=0x7f9cf4070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.914+0000 7f9d17200640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cf406e630 0x7f9cf4070af0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9d10006fd0 tx=0x7f9d10008040 comp rx=0 tx=0).stop 2024-05-30T17:05:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.914+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815edd0 msgr2=0x7f9d181f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:05:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.914+0000 7f9d17200640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815edd0 0x7f9d181f4390 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c450 tx=0x7f9d0000c910 comp rx=0 tx=0).stop 2024-05-30T17:05:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.915+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 shutdown_connections 2024-05-30T17:05:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.915+0000 7f9d17200640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9cf406e630 0x7f9cf4070af0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.915+0000 7f9d17200640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1815f210 0x7f9d180b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.915+0000 7f9d17200640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d1815edd0 0x7f9d181f4390 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.915+0000 7f9d17200640 1 --2- 172.21.15.67:0/1233066524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1815b180 0x7f9d180cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:05:59.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.915+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 >> 172.21.15.67:0/1233066524 conn(0x7f9d180b1db0 msgr2=0x7f9d180b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:05:59.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.915+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 shutdown_connections 2024-05-30T17:05:59.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:05:59.916+0000 7f9d17200640 1 -- 172.21.15.67:0/1233066524 wait complete. 2024-05-30T17:05:59.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.932+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.932+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.932+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.932+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.932 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:05:59.932+0000 7f0c8f800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:05:59.943 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96443 2024-05-30T17:06:00.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.345+0000 7f0177e00640 1 Processor -- start 2024-05-30T17:06:00.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.345+0000 7f0177e00640 1 -- start start 2024-05-30T17:06:00.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.347+0000 7f0177e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0178153510 0x7f01780b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:00.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.347+0000 7f0177e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f017815d270 0x7f01780ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:00.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.347+0000 7f0177e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f017815d900 con 0x7f01780ccb90 2024-05-30T17:06:00.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.347+0000 7f0177e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f017815d6e0 con 0x7f0178153510 2024-05-30T17:06:00.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.347+0000 7f0177e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01781538f0 con 0x7f017815d270 2024-05-30T17:06:00.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.348+0000 7f016e000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f01780ccb90 0x7f017815cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37566/0 (socket says 172.21.15.67:37566) 2024-05-30T17:06:00.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.348+0000 7f016e000640 1 -- 172.21.15.67:0/321146840 learned_addr learned my addr 172.21.15.67:0/321146840 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:00.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.348+0000 7f0176a00640 1 --2- 172.21.15.67:0/321146840 >> v2:172.21.15.67:3300/0 conn(0x7f0178153510 0x7f01780b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.348+0000 7f0177400640 1 --2- 172.21.15.67:0/321146840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f017815d270 0x7f01780ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.348+0000 7f0176000640 1 -- 172.21.15.67:0/321146840 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2250872393 0 0) 0x7f017815d900 con 0x7f01780ccb90 2024-05-30T17:06:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.348+0000 7f0176000640 1 -- 172.21.15.67:0/321146840 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f015c003520 con 0x7f01780ccb90 2024-05-30T17:06:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.349+0000 7f0176a00640 1 -- 172.21.15.67:0/321146840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f017815d270 msgr2=0x7f01780ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.349+0000 7f0176a00640 1 --2- 172.21.15.67:0/321146840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f017815d270 0x7f01780ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.349+0000 7f0176a00640 1 -- 172.21.15.67:0/321146840 >> v1:172.21.15.67:6789/0 conn(0x7f01780ccb90 legacy=0x7f017815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.349+0000 7f0176000640 1 -- 172.21.15.67:0/321146840 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3989495100 0 0) 0x7f015c003520 con 0x7f01780ccb90 2024-05-30T17:06:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.349+0000 7f0176a00640 1 -- 172.21.15.67:0/321146840 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01780b3f90 con 0x7f0178153510 2024-05-30T17:06:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.351+0000 7f0176a00640 1 --2- 172.21.15.67:0/321146840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0178153510 0x7f01780b4d50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f01680097a0 tx=0x7f016802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d573dccc000774 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.351+0000 7f0176000640 1 -- 172.21.15.67:0/321146840 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168037070 con 0x7f0178153510 2024-05-30T17:06:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.351+0000 7f0176000640 1 -- 172.21.15.67:0/321146840 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f016802ab10 con 0x7f0178153510 2024-05-30T17:06:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.351+0000 7f0176000640 1 -- 172.21.15.67:0/321146840 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01600019f0 con 0x7f0178153510 2024-05-30T17:06:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.352+0000 7f0177e00640 1 -- 172.21.15.67:0/321146840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0178153510 msgr2=0x7f01780b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.352+0000 7f0177e00640 1 --2- 172.21.15.67:0/321146840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0178153510 0x7f01780b4d50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f01680097a0 tx=0x7f016802a080 comp rx=0 tx=0).stop 2024-05-30T17:06:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.352+0000 7f0177e00640 1 -- 172.21.15.67:0/321146840 shutdown_connections 2024-05-30T17:06:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.352+0000 7f0177e00640 1 --2- 172.21.15.67:0/321146840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f017815d270 0x7f01780ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.352+0000 7f0177e00640 1 --2- 172.21.15.67:0/321146840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0178153510 0x7f01780b4d50 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.352+0000 7f0177e00640 1 -- 172.21.15.67:0/321146840 >> 172.21.15.67:0/321146840 conn(0x7f017806b900 msgr2=0x7f017806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.352+0000 7f0177e00640 1 -- 172.21.15.67:0/321146840 shutdown_connections 2024-05-30T17:06:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.352+0000 7f0177e00640 1 -- 172.21.15.67:0/321146840 wait complete. 2024-05-30T17:06:00.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.355+0000 7f0177e00640 1 Processor -- start 2024-05-30T17:06:00.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.356+0000 7f0177e00640 1 -- start start 2024-05-30T17:06:00.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.358+0000 7f0177e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01780ccb90 0x7f01780cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:00.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.358+0000 7f0177e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0178153510 0x7f0178151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:00.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.358+0000 7f0177e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017815d270 0x7f01780ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:00.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.358+0000 7f0177e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f01780b48c0 con 0x7f01780ccb90 2024-05-30T17:06:00.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.358+0000 7f0177e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01780b4550 con 0x7f017815d270 2024-05-30T17:06:00.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.358+0000 7f0177e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f017806e940 con 0x7f0178153510 2024-05-30T17:06:00.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.358+0000 7f0176a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01780ccb90 0x7f01780cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:00.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.359+0000 7f0176a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01780ccb90 0x7f01780cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45530/0 (socket says 172.21.15.67:45530) 2024-05-30T17:06:00.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.359+0000 7f0176a00640 1 -- 172.21.15.67:0/2145134612 learned_addr learned my addr 172.21.15.67:0/2145134612 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:00.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.359+0000 7f016e000640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0178153510 0x7f0178151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:00.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.359+0000 7f0177400640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017815d270 0x7f01780ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:00.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.359+0000 7f0176a00640 1 -- 172.21.15.67:0/2145134612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0178153510 msgr2=0x7f0178151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:00.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.359+0000 7f0176a00640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0178153510 0x7f0178151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:00.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.359+0000 7f0176a00640 1 -- 172.21.15.67:0/2145134612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017815d270 msgr2=0x7f01780ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:00.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.359+0000 7f0176a00640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017815d270 0x7f01780ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:00.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.359+0000 7f0176a00640 1 -- 172.21.15.67:0/2145134612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01780cf460 con 0x7f01780ccb90 2024-05-30T17:06:00.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.360+0000 7f0177400640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017815d270 0x7f01780ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:00.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.360+0000 7f0176a00640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01780ccb90 0x7f01780cec00 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f0168000c00 tx=0x7f01680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:00.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.361+0000 7f016f400640 1 -- 172.21.15.67:0/2145134612 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168037040 con 0x7f01780ccb90 2024-05-30T17:06:00.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.361+0000 7f016f400640 1 -- 172.21.15.67:0/2145134612 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0168004030 con 0x7f01780ccb90 2024-05-30T17:06:00.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.361+0000 7f016f400640 1 -- 172.21.15.67:0/2145134612 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01680043e0 con 0x7f01780ccb90 2024-05-30T17:06:00.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.361+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01780cf740 con 0x7f01780ccb90 2024-05-30T17:06:00.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.362+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01780cfd00 con 0x7f01780ccb90 2024-05-30T17:06:00.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.364+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0178156de0 con 0x7f01780ccb90 2024-05-30T17:06:00.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.371+0000 7f016f400640 1 -- 172.21.15.67:0/2145134612 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 46) ==== 93794+0+0 (secure 0 0 0) 0x7f0168003610 con 0x7f01780ccb90 2024-05-30T17:06:00.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.373+0000 7f016f400640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f015406e7a0 0x7f0154070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:00.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.374+0000 7f016f400640 1 -- 172.21.15.67:0/2145134612 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f01680abde0 con 0x7f01780ccb90 2024-05-30T17:06:00.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.374+0000 7f016e000640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f015406e7a0 0x7f0154070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:00.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.377+0000 7f016f400640 1 -- 172.21.15.67:0/2145134612 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0178156de0 con 0x7f01780ccb90 2024-05-30T17:06:00.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.380+0000 7f016e000640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f015406e7a0 0x7f0154070c60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f017806f470 tx=0x7f0160009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:00.897+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96443", "target": ["mon-mgr", ""]}) -- 0x7f01780689e0 con 0x7f015406e7a0 2024-05-30T17:06:01.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.018+0000 7f016f400640 1 -- 172.21.15.67:0/2145134612 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 47) ==== 93896+0+0 (secure 0 0 0) 0x7f01680ac030 con 0x7f01780ccb90 2024-05-30T17:06:01.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.330+0000 7f016f400640 1 -- 172.21.15.67:0/2145134612 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f01780689e0 con 0x7f015406e7a0 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f015406e7a0 msgr2=0x7f0154070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f015406e7a0 0x7f0154070c60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f017806f470 tx=0x7f0160009040 comp rx=0 tx=0).stop 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01780ccb90 msgr2=0x7f01780cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01780ccb90 0x7f01780cec00 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f0168000c00 tx=0x7f01680046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 shutdown_connections 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f015406e7a0 0x7f0154070c60 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f017815d270 0x7f01780ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0178153510 0x7f0178151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 --2- 172.21.15.67:0/2145134612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01780ccb90 0x7f01780cec00 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.337+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 >> 172.21.15.67:0/2145134612 conn(0x7f017806b900 msgr2=0x7f017806bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.339+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 shutdown_connections 2024-05-30T17:06:01.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.339+0000 7f0177e00640 1 -- 172.21.15.67:0/2145134612 wait complete. 2024-05-30T17:06:01.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96444 2024-05-30T17:06:01.484 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:01.482+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:01.484 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:01.482+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:01.484 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:01.482+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:01.484 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:01.482+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:01.484 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:01.482+0000 7f0c90c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:01.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.804+0000 7f223fc00640 1 Processor -- start 2024-05-30T17:06:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.805+0000 7f223fc00640 1 -- start start 2024-05-30T17:06:01.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.811+0000 7f223fc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22380d0de0 0x7f2238072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:01.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.811+0000 7f223fc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f223806e450 0x7f22380ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:01.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.811+0000 7f223fc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2238072db0 con 0x7f2238162c20 2024-05-30T17:06:01.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.811+0000 7f223fc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2238072b80 con 0x7f22380d0de0 2024-05-30T17:06:01.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.811+0000 7f223fc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2238163000 con 0x7f223806e450 2024-05-30T17:06:01.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.812+0000 7f223d200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2238162c20 0x7f22380d06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56132/0 (socket says 172.21.15.67:56132) 2024-05-30T17:06:01.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.813+0000 7f223d200640 1 -- 172.21.15.67:0/2049330724 learned_addr learned my addr 172.21.15.67:0/2049330724 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:01.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.813+0000 7f2237e00640 1 --2- 172.21.15.67:0/2049330724 >> v2:172.21.15.67:3300/0 conn(0x7f22380d0de0 0x7f2238072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:01.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.814+0000 7f223dc00640 1 --2- 172.21.15.67:0/2049330724 >> v2:172.21.15.169:3300/0 conn(0x7f223806e450 0x7f22380ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:01.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.815+0000 7f2237e00640 1 -- 172.21.15.67:0/2049330724 >> v2:172.21.15.169:3300/0 conn(0x7f223806e450 msgr2=0x7f22380ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:01.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.815+0000 7f2237e00640 1 --2- 172.21.15.67:0/2049330724 >> v2:172.21.15.169:3300/0 conn(0x7f223806e450 0x7f22380ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.815+0000 7f2237e00640 1 -- 172.21.15.67:0/2049330724 >> v1:172.21.15.67:6789/0 conn(0x7f2238162c20 legacy=0x7f22380d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:01.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.816+0000 7f2237e00640 1 -- 172.21.15.67:0/2049330724 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2238073090 con 0x7f22380d0de0 2024-05-30T17:06:01.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.818+0000 7f2237e00640 1 --2- 172.21.15.67:0/2049330724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 0x7f2238072640 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f22380b53d0 tx=0x7f223002a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be126faa608c857 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:01.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.819+0000 7f2237400640 1 -- 172.21.15.67:0/2049330724 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230032640 con 0x7f22380d0de0 2024-05-30T17:06:01.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.819+0000 7f2237400640 1 -- 172.21.15.67:0/2049330724 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f223002e050 con 0x7f22380d0de0 2024-05-30T17:06:01.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.819+0000 7f2237400640 1 -- 172.21.15.67:0/2049330724 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230032a00 con 0x7f22380d0de0 2024-05-30T17:06:01.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.821+0000 7f223fc00640 1 -- 172.21.15.67:0/2049330724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 msgr2=0x7f2238072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:01.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.821+0000 7f223fc00640 1 --2- 172.21.15.67:0/2049330724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 0x7f2238072640 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f22380b53d0 tx=0x7f223002a850 comp rx=0 tx=0).stop 2024-05-30T17:06:01.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.822+0000 7f223fc00640 1 -- 172.21.15.67:0/2049330724 shutdown_connections 2024-05-30T17:06:01.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.822+0000 7f223fc00640 1 --2- 172.21.15.67:0/2049330724 >> v2:172.21.15.169:3300/0 conn(0x7f223806e450 0x7f22380ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.822+0000 7f223fc00640 1 --2- 172.21.15.67:0/2049330724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 0x7f2238072640 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.822+0000 7f223fc00640 1 -- 172.21.15.67:0/2049330724 >> 172.21.15.67:0/2049330724 conn(0x7f223806bd20 msgr2=0x7f223806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:01.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.824+0000 7f223fc00640 1 -- 172.21.15.67:0/2049330724 shutdown_connections 2024-05-30T17:06:01.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.824+0000 7f223fc00640 1 -- 172.21.15.67:0/2049330724 wait complete. 2024-05-30T17:06:01.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.827+0000 7f223fc00640 1 Processor -- start 2024-05-30T17:06:01.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.827+0000 7f223fc00640 1 -- start start 2024-05-30T17:06:01.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.829+0000 7f223fc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f223806e450 0x7f2238155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.829+0000 7f223fc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 0x7f223815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.829+0000 7f223fc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2238162c20 0x7f2238163b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.829+0000 7f223fc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2238073650 con 0x7f22380d0de0 2024-05-30T17:06:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.829+0000 7f223fc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f22380739c0 con 0x7f223806e450 2024-05-30T17:06:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.829+0000 7f223fc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2238073e50 con 0x7f2238162c20 2024-05-30T17:06:01.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.829+0000 7f2237e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 0x7f223815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:01.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.829+0000 7f2237e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 0x7f223815a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33368/0 (socket says 172.21.15.67:33368) 2024-05-30T17:06:01.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.830+0000 7f2237e00640 1 -- 172.21.15.67:0/3132252614 learned_addr learned my addr 172.21.15.67:0/3132252614 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:01.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.830+0000 7f2237e00640 1 -- 172.21.15.67:0/3132252614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2238162c20 msgr2=0x7f2238163b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:01.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.830+0000 7f2237e00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2238162c20 0x7f2238163b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.830+0000 7f2237e00640 1 -- 172.21.15.67:0/3132252614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f223806e450 msgr2=0x7f2238155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:01.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.830+0000 7f2237e00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f223806e450 0x7f2238155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:01.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.830+0000 7f2237e00640 1 -- 172.21.15.67:0/3132252614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2238156860 con 0x7f22380d0de0 2024-05-30T17:06:01.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.831+0000 7f2237e00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 0x7f223815a390 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f223000b140 tx=0x7f2230004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.831+0000 7f2234c00640 1 -- 172.21.15.67:0/3132252614 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230041070 con 0x7f22380d0de0 2024-05-30T17:06:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.831+0000 7f2234c00640 1 -- 172.21.15.67:0/3132252614 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2230031930 con 0x7f22380d0de0 2024-05-30T17:06:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.831+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2238156b40 con 0x7f22380d0de0 2024-05-30T17:06:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.831+0000 7f2234c00640 1 -- 172.21.15.67:0/3132252614 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230031ce0 con 0x7f22380d0de0 2024-05-30T17:06:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.832+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2238157100 con 0x7f22380d0de0 2024-05-30T17:06:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.834+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22380689e0 con 0x7f22380d0de0 2024-05-30T17:06:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.837+0000 7f2234c00640 1 -- 172.21.15.67:0/3132252614 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 47) ==== 93896+0+0 (secure 0 0 0) 0x7f223003a040 con 0x7f22380d0de0 2024-05-30T17:06:01.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.839+0000 7f2234c00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f221806e940 0x7f2218070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:01.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.839+0000 7f2234c00640 1 -- 172.21.15.67:0/3132252614 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f22300ac8d0 con 0x7f22380d0de0 2024-05-30T17:06:01.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.844+0000 7f223d200640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f221806e940 0x7f2218070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:01.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.846+0000 7f223d200640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f221806e940 0x7f2218070e00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2224000f60 tx=0x7f2224007a80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:01.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:01.846+0000 7f2234c00640 1 -- 172.21.15.67:0/3132252614 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f223007c7e0 con 0x7f22380d0de0 2024-05-30T17:06:02.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:02.393+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96444", "target": ["mon-mgr", ""]}) -- 0x7f22380cfd80 con 0x7f221806e940 2024-05-30T17:06:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.008+0000 7f2234c00640 1 -- 172.21.15.67:0/3132252614 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f22380cfd80 con 0x7f221806e940 2024-05-30T17:06:03.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.024+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f221806e940 msgr2=0x7f2218070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:03.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.024+0000 7f223fc00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f221806e940 0x7f2218070e00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2224000f60 tx=0x7f2224007a80 comp rx=0 tx=0).stop 2024-05-30T17:06:03.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.025+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 msgr2=0x7f223815a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:03.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.025+0000 7f223fc00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 0x7f223815a390 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f223000b140 tx=0x7f2230004a40 comp rx=0 tx=0).stop 2024-05-30T17:06:03.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.025+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 shutdown_connections 2024-05-30T17:06:03.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.025+0000 7f223fc00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f221806e940 0x7f2218070e00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:03.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.025+0000 7f223fc00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2238162c20 0x7f2238163b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:03.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.025+0000 7f223fc00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22380d0de0 0x7f223815a390 secure :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f223000b140 tx=0x7f2230004a40 comp rx=0 tx=0).stop 2024-05-30T17:06:03.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.025+0000 7f223fc00640 1 --2- 172.21.15.67:0/3132252614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f223806e450 0x7f2238155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:03.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.025+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 >> 172.21.15.67:0/3132252614 conn(0x7f223806bd20 msgr2=0x7f22380b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:03.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.026+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 shutdown_connections 2024-05-30T17:06:03.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.026+0000 7f223fc00640 1 -- 172.21.15.67:0/3132252614 wait complete. 2024-05-30T17:06:03.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96445 2024-05-30T17:06:03.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.490+0000 7fe521c00640 1 Processor -- start 2024-05-30T17:06:03.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.491+0000 7fe521c00640 1 -- start start 2024-05-30T17:06:03.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.493+0000 7fe521c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe51c0d06d0 0x7fe51c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:03.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.493+0000 7fe521c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c156fb0 con 0x7fe51c156bd0 2024-05-30T17:06:03.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.493+0000 7fe521c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c15d520 con 0x7fe51c0b4e30 2024-05-30T17:06:03.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.493+0000 7fe521c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe51c0d1060 con 0x7fe51c0d06d0 2024-05-30T17:06:03.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.494+0000 7fe51aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe51c156bd0 0x7fe51c0b3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56136/0 (socket says 172.21.15.67:56136) 2024-05-30T17:06:03.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.494+0000 7fe51aa00640 1 -- 172.21.15.67:0/2072274557 learned_addr learned my addr 172.21.15.67:0/2072274557 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:03.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.495+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3251955858 0 0) 0x7fe51c156fb0 con 0x7fe51c156bd0 2024-05-30T17:06:03.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.495+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4fc003520 con 0x7fe51c156bd0 2024-05-30T17:06:03.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.495+0000 7fe51b400640 1 --2- 172.21.15.67:0/2072274557 >> v2:172.21.15.169:3300/0 conn(0x7fe51c0d06d0 0x7fe51c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.495+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3284947199 0 0) 0x7fe51c15d520 con 0x7fe51c0b4e30 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe51c156fb0 con 0x7fe51c0b4e30 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 924386218 0 0) 0x7fe4fc003520 con 0x7fe51c156bd0 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 >> v2:172.21.15.169:3300/0 conn(0x7fe51c0d06d0 msgr2=0x7fe51c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 --2- 172.21.15.67:0/2072274557 >> v2:172.21.15.169:3300/0 conn(0x7fe51c0d06d0 0x7fe51c0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 >> v1:172.21.15.169:6789/0 conn(0x7fe51c0b4e30 legacy=0x7fe51c15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c15d800 con 0x7fe51c156bd0 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fe50c001670 con 0x7fe51c156bd0 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 >> v1:172.21.15.67:6789/0 conn(0x7fe51c156bd0 legacy=0x7fe51c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4fc003b40 0x7fe4fc003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc004800 0x7fe4fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc005210 0x7fe4fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c0d1060 con 0x7fe4fc004800 2024-05-30T17:06:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc00c140 con 0x7fe4fc005210 2024-05-30T17:06:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.496+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe4fc00c340 con 0x7fe4fc003b40 2024-05-30T17:06:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.497+0000 7fe51a000640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc004800 0x7fe4fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.499+0000 7fe51a000640 1 -- 172.21.15.67:0/2072274557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4fc003b40 msgr2=0x7fe4fc003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.499+0000 7fe51a000640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4fc003b40 0x7fe4fc003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.499+0000 7fe51a000640 1 -- 172.21.15.67:0/2072274557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc005210 msgr2=0x7fe4fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.499+0000 7fe51b400640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc005210 0x7fe4fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.499+0000 7fe51a000640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc005210 0x7fe4fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.499+0000 7fe51a000640 1 -- 172.21.15.67:0/2072274557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c15d800 con 0x7fe4fc004800 2024-05-30T17:06:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.499+0000 7fe51b400640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc005210 0x7fe4fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.499+0000 7fe51b400640 1 -- 172.21.15.67:0/2072274557 reap_dead start 2024-05-30T17:06:03.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.501+0000 7fe51a000640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc004800 0x7fe4fc004c80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe5040018c0 tx=0x7fe4fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a8382ccd63111a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.502+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5040014c0 con 0x7fe4fc004800 2024-05-30T17:06:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.503+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe50402ad30 con 0x7fe4fc004800 2024-05-30T17:06:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.503+0000 7fe519600640 1 -- 172.21.15.67:0/2072274557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504007260 con 0x7fe4fc004800 2024-05-30T17:06:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.504+0000 7fe521c00640 1 -- 172.21.15.67:0/2072274557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc004800 msgr2=0x7fe4fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:03.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.504+0000 7fe521c00640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc004800 0x7fe4fc004c80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe5040018c0 tx=0x7fe4fc005830 comp rx=0 tx=0).stop 2024-05-30T17:06:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.504+0000 7fe521c00640 1 -- 172.21.15.67:0/2072274557 shutdown_connections 2024-05-30T17:06:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.504+0000 7fe521c00640 1 --2- 172.21.15.67:0/2072274557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc004800 0x7fe4fc004c80 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:03.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.504+0000 7fe521c00640 1 -- 172.21.15.67:0/2072274557 >> 172.21.15.67:0/2072274557 conn(0x7fe51c06bd20 msgr2=0x7fe51c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.508+0000 7fe521c00640 1 -- 172.21.15.67:0/2072274557 shutdown_connections 2024-05-30T17:06:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.508+0000 7fe521c00640 1 -- 172.21.15.67:0/2072274557 wait complete. 2024-05-30T17:06:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.513+0000 7fe521c00640 1 Processor -- start 2024-05-30T17:06:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.514+0000 7fe521c00640 1 -- start start 2024-05-30T17:06:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.515+0000 7fe521c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc004800 0x7fe51c1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.515+0000 7fe521c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe51c1576f0 0x7fe51c1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.515+0000 7fe521c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c159600 0x7fe51c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.515+0000 7fe521c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c0b35d0 con 0x7fe51c159600 2024-05-30T17:06:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.515+0000 7fe521c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c15dae0 con 0x7fe4fc004800 2024-05-30T17:06:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.515+0000 7fe521c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe51c0d0b20 con 0x7fe51c1576f0 2024-05-30T17:06:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.516+0000 7fe51b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c159600 0x7fe51c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.516+0000 7fe51b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c159600 0x7fe51c159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33390/0 (socket says 172.21.15.67:33390) 2024-05-30T17:06:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.516+0000 7fe51b400640 1 -- 172.21.15.67:0/3736793866 learned_addr learned my addr 172.21.15.67:0/3736793866 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.517+0000 7fe51aa00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc004800 0x7fe51c1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.517+0000 7fe51b400640 1 -- 172.21.15.67:0/3736793866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe51c1576f0 msgr2=0x7fe51c1590c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T17:06:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.517+0000 7fe51b400640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe51c1576f0 0x7fe51c1590c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.517+0000 7fe51b400640 1 -- 172.21.15.67:0/3736793866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc004800 msgr2=0x7fe51c1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.517+0000 7fe51b400640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc004800 0x7fe51c1571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.517+0000 7fe51b400640 1 -- 172.21.15.67:0/3736793866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c156540 con 0x7fe51c159600 2024-05-30T17:06:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.517+0000 7fe51b400640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c159600 0x7fe51c159a80 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fe51c15a6c0 tx=0x7fe514009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.518+0000 7fe512a00640 1 -- 172.21.15.67:0/3736793866 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514019070 con 0x7fe51c159600 2024-05-30T17:06:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.518+0000 7fe512a00640 1 -- 172.21.15.67:0/3736793866 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe514007d20 con 0x7fe51c159600 2024-05-30T17:06:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.518+0000 7fe512a00640 1 -- 172.21.15.67:0/3736793866 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514014620 con 0x7fe51c159600 2024-05-30T17:06:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.518+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c159fc0 con 0x7fe51c159600 2024-05-30T17:06:03.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.519+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c155f80 con 0x7fe51c159600 2024-05-30T17:06:03.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.520+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51c0689e0 con 0x7fe51c159600 2024-05-30T17:06:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.537+0000 7fe512a00640 1 -- 172.21.15.67:0/3736793866 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 48) ==== 93947+0+0 (secure 0 0 0) 0x7fe514005d10 con 0x7fe51c159600 2024-05-30T17:06:03.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.543+0000 7fe512a00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe4f806ea00 0x7fe4f8070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:03.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.543+0000 7fe512a00640 1 -- 172.21.15.67:0/3736793866 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fe51408aa00 con 0x7fe51c159600 2024-05-30T17:06:03.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.549+0000 7fe512a00640 1 -- 172.21.15.67:0/3736793866 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe51405a910 con 0x7fe51c159600 2024-05-30T17:06:03.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.561+0000 7fe51aa00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe4f806ea00 0x7fe4f8070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:03.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:03.572+0000 7fe51aa00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe4f806ea00 0x7fe4f8070ec0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe51c051690 tx=0x7fe50c001760 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:04.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:04.141+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96445", "target": ["mon-mgr", ""]}) -- 0x7fe51c160f60 con 0x7fe4f806ea00 2024-05-30T17:06:05.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.304+0000 7fe512a00640 1 -- 172.21.15.67:0/3736793866 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe51c160f60 con 0x7fe4f806ea00 2024-05-30T17:06:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.315+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe4f806ea00 msgr2=0x7fe4f8070ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.315+0000 7fe521c00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe4f806ea00 0x7fe4f8070ec0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe51c051690 tx=0x7fe50c001760 comp rx=0 tx=0).stop 2024-05-30T17:06:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.315+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c159600 msgr2=0x7fe51c159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:05.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.315+0000 7fe521c00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c159600 0x7fe51c159a80 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fe51c15a6c0 tx=0x7fe514009cd0 comp rx=0 tx=0).stop 2024-05-30T17:06:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.316+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 shutdown_connections 2024-05-30T17:06:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.316+0000 7fe521c00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe4f806ea00 0x7fe4f8070ec0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.316+0000 7fe521c00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c159600 0x7fe51c159a80 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.316+0000 7fe521c00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe51c1576f0 0x7fe51c1590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.316+0000 7fe521c00640 1 --2- 172.21.15.67:0/3736793866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4fc004800 0x7fe51c1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.316+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 >> 172.21.15.67:0/3736793866 conn(0x7fe51c06bd20 msgr2=0x7fe51c15a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:05.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.317+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 shutdown_connections 2024-05-30T17:06:05.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.317+0000 7fe521c00640 1 -- 172.21.15.67:0/3736793866 wait complete. 2024-05-30T17:06:05.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96446 2024-05-30T17:06:05.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.822+0000 7f32ef800640 1 Processor -- start 2024-05-30T17:06:05.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.823+0000 7f32ef800640 1 -- start start 2024-05-30T17:06:05.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.825+0000 7f32ef800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e8155000 0x7f32e815b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:05.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.826+0000 7f32ef800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e815bdd0 con 0x7f32e80d0e10 2024-05-30T17:06:05.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.826+0000 7f32ef800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e80b40d0 con 0x7f32e80b3c30 2024-05-30T17:06:05.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.826+0000 7f32ef800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32e80b3510 con 0x7f32e8155000 2024-05-30T17:06:05.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.826+0000 7f32ece00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e8155000 0x7f32e815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:05.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.826+0000 7f32e7e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f32e80b3c30 0x7f32e80d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:44130/0 (socket says 172.21.15.67:44130) 2024-05-30T17:06:05.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.826+0000 7f32e7e00640 1 -- 172.21.15.67:0/3490714135 learned_addr learned my addr 172.21.15.67:0/3490714135 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:05.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.826+0000 7f32ece00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e8155000 0x7f32e815b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:49258/0 (socket says 172.21.15.67:49258) 2024-05-30T17:06:05.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.827+0000 7f32e7400640 1 -- 172.21.15.67:0/3490714135 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3161521803 0 0) 0x7f32e80b40d0 con 0x7f32e80b3c30 2024-05-30T17:06:05.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.827+0000 7f32e7400640 1 -- 172.21.15.67:0/3490714135 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32d0003440 con 0x7f32e80b3c30 2024-05-30T17:06:05.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.827+0000 7f32ece00640 1 -- 172.21.15.67:0/3490714135 >> v1:172.21.15.169:6789/0 conn(0x7f32e80b3c30 legacy=0x7f32e80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:05.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.827+0000 7f32ece00640 1 -- 172.21.15.67:0/3490714135 >> v1:172.21.15.67:6789/0 conn(0x7f32e80d0e10 legacy=0x7f32e806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:05.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.827+0000 7f32ece00640 1 -- 172.21.15.67:0/3490714135 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e806dee0 con 0x7f32e8155000 2024-05-30T17:06:05.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.828+0000 7f32ece00640 1 --2- 172.21.15.67:0/3490714135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e8155000 0x7f32e815b890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0097d0 tx=0x7f32dc02a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=983d2eb250973ecd server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:05.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.828+0000 7f32e7400640 1 -- 172.21.15.67:0/3490714135 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc037070 con 0x7f32e8155000 2024-05-30T17:06:05.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.829+0000 7f32e7400640 1 -- 172.21.15.67:0/3490714135 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc02ad40 con 0x7f32e8155000 2024-05-30T17:06:05.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.829+0000 7f32e7400640 1 -- 172.21.15.67:0/3490714135 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc032660 con 0x7f32e8155000 2024-05-30T17:06:05.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.829+0000 7f32ef800640 1 -- 172.21.15.67:0/3490714135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e8155000 msgr2=0x7f32e815b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:05.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.829+0000 7f32ef800640 1 --2- 172.21.15.67:0/3490714135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e8155000 0x7f32e815b890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0097d0 tx=0x7f32dc02a0b0 comp rx=0 tx=0).stop 2024-05-30T17:06:05.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.830+0000 7f32ef800640 1 -- 172.21.15.67:0/3490714135 shutdown_connections 2024-05-30T17:06:05.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.830+0000 7f32ef800640 1 --2- 172.21.15.67:0/3490714135 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e8155000 0x7f32e815b890 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:05.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.830+0000 7f32ef800640 1 -- 172.21.15.67:0/3490714135 >> 172.21.15.67:0/3490714135 conn(0x7f32e80b56e0 msgr2=0x7f32e80b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:05.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.830+0000 7f32ef800640 1 -- 172.21.15.67:0/3490714135 shutdown_connections 2024-05-30T17:06:05.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.830+0000 7f32ef800640 1 -- 172.21.15.67:0/3490714135 wait complete. 2024-05-30T17:06:05.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.832+0000 7f32ef800640 1 Processor -- start 2024-05-30T17:06:05.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.832+0000 7f32ef800640 1 -- start start 2024-05-30T17:06:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.834+0000 7f32ef800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e80b3c30 0x7f32e81f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.834+0000 7f32ef800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e80d0e10 0x7f32e81f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.834+0000 7f32ef800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e8155000 0x7f32e8157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.835+0000 7f32ef800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f32e80d02b0 con 0x7f32e8155000 2024-05-30T17:06:05.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.835+0000 7f32ef800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32e806e530 con 0x7f32e80b3c30 2024-05-30T17:06:05.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.835+0000 7f32ef800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32e80d0430 con 0x7f32e80d0e10 2024-05-30T17:06:05.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.835+0000 7f32ece00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e80b3c30 0x7f32e81f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.835+0000 7f32ece00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e80b3c30 0x7f32e81f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:36524/0 (socket says 172.21.15.67:36524) 2024-05-30T17:06:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.835+0000 7f32ece00640 1 -- 172.21.15.67:0/2863351373 learned_addr learned my addr 172.21.15.67:0/2863351373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:05.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.836+0000 7f32e7e00640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e80d0e10 0x7f32e81f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.836+0000 7f32ed800640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e8155000 0x7f32e8157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.837+0000 7f32ece00640 1 -- 172.21.15.67:0/2863351373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e80d0e10 msgr2=0x7f32e81f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.837+0000 7f32ece00640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e80d0e10 0x7f32e81f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.837+0000 7f32ece00640 1 -- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e8155000 msgr2=0x7f32e8157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.837+0000 7f32ece00640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e8155000 0x7f32e8157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:05.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.837+0000 7f32ece00640 1 -- 172.21.15.67:0/2863351373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e8155850 con 0x7f32e80b3c30 2024-05-30T17:06:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.837+0000 7f32ece00640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e80b3c30 0x7f32e81f29f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009720 tx=0x7f32dc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.838+0000 7f32e4c00640 1 -- 172.21.15.67:0/2863351373 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc037040 con 0x7f32e80b3c30 2024-05-30T17:06:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.838+0000 7f32e4c00640 1 -- 172.21.15.67:0/2863351373 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc004030 con 0x7f32e80b3c30 2024-05-30T17:06:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.838+0000 7f32e4c00640 1 -- 172.21.15.67:0/2863351373 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc004400 con 0x7f32e80b3c30 2024-05-30T17:06:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.838+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e8155b30 con 0x7f32e80b3c30 2024-05-30T17:06:05.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.838+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e81560f0 con 0x7f32e80b3c30 2024-05-30T17:06:05.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.842+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e80689e0 con 0x7f32e80b3c30 2024-05-30T17:06:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.842+0000 7f32e4c00640 1 -- 172.21.15.67:0/2863351373 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 48) ==== 93947+0+0 (secure 0 0 0) 0x7f32dc03a070 con 0x7f32e80b3c30 2024-05-30T17:06:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.844+0000 7f32e4c00640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32c806e9d0 0x7f32c8070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.844+0000 7f32e4c00640 1 -- 172.21.15.67:0/2863351373 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f32dc0ab120 con 0x7f32e80b3c30 2024-05-30T17:06:05.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.850+0000 7f32e7e00640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32c806e9d0 0x7f32c8070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:05.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.852+0000 7f32e7e00640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32c806e9d0 0x7f32c8070e90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f32e815a650 tx=0x7f32d4007d10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:05.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:05.852+0000 7f32e4c00640 1 -- 172.21.15.67:0/2863351373 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32dc041020 con 0x7f32e80b3c30 2024-05-30T17:06:06.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.368+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96446", "target": ["mon-mgr", ""]}) -- 0x7f32e806ef90 con 0x7f32c806e9d0 2024-05-30T17:06:06.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.892+0000 7f32e4c00640 1 -- 172.21.15.67:0/2863351373 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f32e806ef90 con 0x7f32c806e9d0 2024-05-30T17:06:06.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.904+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32c806e9d0 msgr2=0x7f32c8070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:06.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.904+0000 7f32ef800640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32c806e9d0 0x7f32c8070e90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f32e815a650 tx=0x7f32d4007d10 comp rx=0 tx=0).stop 2024-05-30T17:06:06.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.904+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e80b3c30 msgr2=0x7f32e81f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:06.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.904+0000 7f32ef800640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e80b3c30 0x7f32e81f29f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009720 tx=0x7f32dc0046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:06.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.905+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 shutdown_connections 2024-05-30T17:06:06.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.905+0000 7f32ef800640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f32c806e9d0 0x7f32c8070e90 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:06.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.905+0000 7f32ef800640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e8155000 0x7f32e8157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:06.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.905+0000 7f32ef800640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32e80d0e10 0x7f32e81f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:06.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.905+0000 7f32ef800640 1 --2- 172.21.15.67:0/2863351373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e80b3c30 0x7f32e81f29f0 secure :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009720 tx=0x7f32dc0046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:06.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.905+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 >> 172.21.15.67:0/2863351373 conn(0x7f32e80b56e0 msgr2=0x7f32e815d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:06.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.906+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 shutdown_connections 2024-05-30T17:06:06.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:06.908+0000 7f32ef800640 1 -- 172.21.15.67:0/2863351373 wait complete. 2024-05-30T17:06:06.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96447 2024-05-30T17:06:07.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.446+0000 7fd8a4e00640 1 Processor -- start 2024-05-30T17:06:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.446+0000 7fd8a4e00640 1 -- start start 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.449+0000 7fd8a4e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd8a0161fa0 0x7fd8a0162600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.449+0000 7fd8a4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 0x7fd8a00b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.449+0000 7fd8a4e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a0162380 con 0x7fd8a00b52a0 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.449+0000 7fd8a4e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd8a00d33e0 con 0x7fd8a0161fa0 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.449+0000 7fd8a4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a00b5bd0 con 0x7fd8a00b56d0 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.449+0000 7fd89ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd8a00b52a0 0x7fd8a0162b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56160/0 (socket says 172.21.15.67:56160) 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.449+0000 7fd89ea00640 1 -- 172.21.15.67:0/165125314 learned_addr learned my addr 172.21.15.67:0/165125314 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.450+0000 7fd89fe00640 1 --2- 172.21.15.67:0/165125314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 0x7fd8a00b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.450+0000 7fd89f400640 1 --2- 172.21.15.67:0/165125314 >> v2:172.21.15.67:3300/0 conn(0x7fd8a0161fa0 0x7fd8a0162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.450+0000 7fd89e000640 1 -- 172.21.15.67:0/165125314 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2264745211 0 0) 0x7fd8a0162380 con 0x7fd8a00b52a0 2024-05-30T17:06:07.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.450+0000 7fd89e000640 1 -- 172.21.15.67:0/165125314 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd884003520 con 0x7fd8a00b52a0 2024-05-30T17:06:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.451+0000 7fd89fe00640 1 -- 172.21.15.67:0/165125314 >> v2:172.21.15.67:3300/0 conn(0x7fd8a0161fa0 msgr2=0x7fd8a0162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.451+0000 7fd89fe00640 1 --2- 172.21.15.67:0/165125314 >> v2:172.21.15.67:3300/0 conn(0x7fd8a0161fa0 0x7fd8a0162600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.451+0000 7fd89fe00640 1 -- 172.21.15.67:0/165125314 >> v1:172.21.15.67:6789/0 conn(0x7fd8a00b52a0 legacy=0x7fd8a0162b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:07.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.451+0000 7fd89fe00640 1 -- 172.21.15.67:0/165125314 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a00d2e20 con 0x7fd8a00b56d0 2024-05-30T17:06:07.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.453+0000 7fd89f400640 1 --2- 172.21.15.67:0/165125314 >> v2:172.21.15.67:3300/0 conn(0x7fd8a0161fa0 0x7fd8a0162600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:06:07.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.454+0000 7fd89fe00640 1 --2- 172.21.15.67:0/165125314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 0x7fd8a00b3cd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd898009d90 tx=0x7fd89802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a00e8ae1b5483b0d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:07.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.455+0000 7fd89e000640 1 -- 172.21.15.67:0/165125314 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898007740 con 0x7fd8a00b56d0 2024-05-30T17:06:07.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.455+0000 7fd89e000640 1 -- 172.21.15.67:0/165125314 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8980078e0 con 0x7fd8a00b56d0 2024-05-30T17:06:07.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.455+0000 7fd89e000640 1 -- 172.21.15.67:0/165125314 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898007cf0 con 0x7fd8a00b56d0 2024-05-30T17:06:07.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.457+0000 7fd8a4e00640 1 -- 172.21.15.67:0/165125314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 msgr2=0x7fd8a00b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:07.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.457+0000 7fd8a4e00640 1 --2- 172.21.15.67:0/165125314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 0x7fd8a00b3cd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd898009d90 tx=0x7fd89802a5f0 comp rx=0 tx=0).stop 2024-05-30T17:06:07.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.459+0000 7fd8a4e00640 1 -- 172.21.15.67:0/165125314 shutdown_connections 2024-05-30T17:06:07.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.459+0000 7fd8a4e00640 1 --2- 172.21.15.67:0/165125314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 0x7fd8a00b3cd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:07.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.459+0000 7fd8a4e00640 1 --2- 172.21.15.67:0/165125314 >> v2:172.21.15.67:3300/0 conn(0x7fd8a0161fa0 0x7fd8a0162600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:07.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.459+0000 7fd8a4e00640 1 -- 172.21.15.67:0/165125314 >> 172.21.15.67:0/165125314 conn(0x7fd8a008bfc0 msgr2=0x7fd8a00d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:07.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.465+0000 7fd8a4e00640 1 -- 172.21.15.67:0/165125314 shutdown_connections 2024-05-30T17:06:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.467+0000 7fd8a4e00640 1 -- 172.21.15.67:0/165125314 wait complete. 2024-05-30T17:06:07.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.475+0000 7fd8a4e00640 1 Processor -- start 2024-05-30T17:06:07.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.476+0000 7fd8a4e00640 1 -- start start 2024-05-30T17:06:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.481+0000 7fd8a4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8a00b52a0 0x7fd8a015a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.481+0000 7fd8a4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 0x7fd8a00cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.481+0000 7fd8a4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0161fa0 0x7fd8a00b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.481+0000 7fd8a4e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00d51d0 con 0x7fd8a0161fa0 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.481+0000 7fd8a4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00d4cb0 con 0x7fd8a00b52a0 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.481+0000 7fd8a4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a00d5350 con 0x7fd8a00b56d0 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.482+0000 7fd89fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0161fa0 0x7fd8a00b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.482+0000 7fd89fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0161fa0 0x7fd8a00b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33422/0 (socket says 172.21.15.67:33422) 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.482+0000 7fd89fe00640 1 -- 172.21.15.67:0/2715829890 learned_addr learned my addr 172.21.15.67:0/2715829890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.483+0000 7fd89fe00640 1 -- 172.21.15.67:0/2715829890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 msgr2=0x7fd8a00cd650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.483+0000 7fd89fe00640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 0x7fd8a00cd650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.483+0000 7fd89fe00640 1 -- 172.21.15.67:0/2715829890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8a00b52a0 msgr2=0x7fd8a015a350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.483+0000 7fd89fe00640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8a00b52a0 0x7fd8a015a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.483+0000 7fd89fe00640 1 -- 172.21.15.67:0/2715829890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a015ac90 con 0x7fd8a0161fa0 2024-05-30T17:06:07.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.484+0000 7fd89fe00640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0161fa0 0x7fd8a00b2650 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fd898009eb0 tx=0x7fd898004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:07.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.485+0000 7fd897400640 1 -- 172.21.15.67:0/2715829890 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89803f070 con 0x7fd8a0161fa0 2024-05-30T17:06:07.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.485+0000 7fd897400640 1 -- 172.21.15.67:0/2715829890 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd898031bb0 con 0x7fd8a0161fa0 2024-05-30T17:06:07.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.486+0000 7fd897400640 1 -- 172.21.15.67:0/2715829890 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898044460 con 0x7fd8a0161fa0 2024-05-30T17:06:07.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.487+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a0160170 con 0x7fd8a0161fa0 2024-05-30T17:06:07.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.487+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a00cce50 con 0x7fd8a0161fa0 2024-05-30T17:06:07.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.490+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a00ceb60 con 0x7fd8a0161fa0 2024-05-30T17:06:07.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.491+0000 7fd897400640 1 -- 172.21.15.67:0/2715829890 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 48) ==== 93947+0+0 (secure 0 0 0) 0x7fd898044600 con 0x7fd8a0161fa0 2024-05-30T17:06:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.493+0000 7fd897400640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd88006e980 0x7fd880070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.493+0000 7fd897400640 1 -- 172.21.15.67:0/2715829890 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fd8980ac230 con 0x7fd8a0161fa0 2024-05-30T17:06:07.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.495+0000 7fd89f400640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd88006e980 0x7fd880070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:07.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.504+0000 7fd897400640 1 -- 172.21.15.67:0/2715829890 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8a00ceb60 con 0x7fd8a0161fa0 2024-05-30T17:06:07.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:07.510+0000 7fd89f400640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd88006e980 0x7fd880070e40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd890006fd0 tx=0x7fd890008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:08.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.043+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96447", "target": ["mon-mgr", ""]}) -- 0x7fd8a0160450 con 0x7fd88006e980 2024-05-30T17:06:08.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.940+0000 7fd897400640 1 -- 172.21.15.67:0/2715829890 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd890004020 con 0x7fd88006e980 2024-05-30T17:06:08.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.951+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd88006e980 msgr2=0x7fd880070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.951+0000 7fd8a4e00640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd88006e980 0x7fd880070e40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd890006fd0 tx=0x7fd890008040 comp rx=0 tx=0).stop 2024-05-30T17:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.951+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0161fa0 msgr2=0x7fd8a00b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.951+0000 7fd8a4e00640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0161fa0 0x7fd8a00b2650 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fd898009eb0 tx=0x7fd898004a40 comp rx=0 tx=0).stop 2024-05-30T17:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.952+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 shutdown_connections 2024-05-30T17:06:08.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.952+0000 7fd8a4e00640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd88006e980 0x7fd880070e40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:08.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.952+0000 7fd8a4e00640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8a0161fa0 0x7fd8a00b2650 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:08.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.952+0000 7fd8a4e00640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8a00b56d0 0x7fd8a00cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:08.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.952+0000 7fd8a4e00640 1 --2- 172.21.15.67:0/2715829890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8a00b52a0 0x7fd8a015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:08.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.952+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 >> 172.21.15.67:0/2715829890 conn(0x7fd8a008bfc0 msgr2=0x7fd8a015ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:08.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.955+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 shutdown_connections 2024-05-30T17:06:08.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:08.955+0000 7fd8a4e00640 1 -- 172.21.15.67:0/2715829890 wait complete. 2024-05-30T17:06:14.031 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96441 2024-05-30T17:06:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.454+0000 7fd32ce00640 1 Processor -- start 2024-05-30T17:06:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.454+0000 7fd32ce00640 1 -- start start 2024-05-30T17:06:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.459+0000 7fd32ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3280d0d50 0x7fd3281633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.459+0000 7fd32ce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3281640e0 con 0x7fd328162ac0 2024-05-30T17:06:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.459+0000 7fd32ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3280b3360 con 0x7fd3280b4120 2024-05-30T17:06:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.459+0000 7fd32ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd3280b3c90 con 0x7fd3280d0d50 2024-05-30T17:06:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.461+0000 7fd325e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd328162ac0 0x7fd3280d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42410/0 (socket says 172.21.15.67:42410) 2024-05-30T17:06:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.461+0000 7fd325e00640 1 -- 172.21.15.67:0/2856775702 learned_addr learned my addr 172.21.15.67:0/2856775702 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.463+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1727358792 0 0) 0x7fd3281640e0 con 0x7fd328162ac0 2024-05-30T17:06:14.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.464+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd30c003520 con 0x7fd328162ac0 2024-05-30T17:06:14.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.465+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1691353736 0 0) 0x7fd30c003520 con 0x7fd328162ac0 2024-05-30T17:06:14.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.465+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3280d0d50 msgr2=0x7fd3281633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:14.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.466+0000 7fd325400640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3280d0d50 0x7fd3281633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:14.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.466+0000 7fd324a00640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3280d0d50 0x7fd3281633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:14.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.467+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 >> v1:172.21.15.169:6789/0 conn(0x7fd3280b4120 legacy=0x7fd3281638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:14.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.467+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3280b3640 con 0x7fd328162ac0 2024-05-30T17:06:14.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.467+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fd3200019f0 con 0x7fd328162ac0 2024-05-30T17:06:14.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.467+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 >> v1:172.21.15.67:6789/0 conn(0x7fd328162ac0 legacy=0x7fd3280d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:14.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.467+0000 7fd324a00640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd30c0039e0 0x7fd30c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:14.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.467+0000 7fd324a00640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 0x7fd30c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:14.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.469+0000 7fd325e00640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd30c0039e0 0x7fd30c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:14.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.470+0000 7fd324a00640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3280d0d50 0x7fd30c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:14.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.470+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd3280b3c90 con 0x7fd30c004800 2024-05-30T17:06:14.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.470+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c005550 con 0x7fd30c0039e0 2024-05-30T17:06:14.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.470+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd30c00bf00 con 0x7fd3280d0d50 2024-05-30T17:06:14.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.471+0000 7fd325400640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 0x7fd30c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:14.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.471+0000 7fd326800640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3280d0d50 0x7fd30c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:14.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.473+0000 7fd325400640 1 -- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3280d0d50 msgr2=0x7fd30c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.473+0000 7fd325400640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3280d0d50 0x7fd30c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:14.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.473+0000 7fd325400640 1 -- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd30c0039e0 msgr2=0x7fd30c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.473+0000 7fd325400640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd30c0039e0 0x7fd30c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.473+0000 7fd325400640 1 -- 172.21.15.67:0/2856775702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3280b3640 con 0x7fd30c004800 2024-05-30T17:06:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.474+0000 7fd325e00640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd30c0039e0 0x7fd30c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:06:14.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.474+0000 7fd326800640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3280d0d50 0x7fd30c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:06:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.476+0000 7fd325400640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 0x7fd30c004c80 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fd31800a9a0 tx=0x7fd30c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec2f1ab46071cf server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:14.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.477+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318037070 con 0x7fd30c004800 2024-05-30T17:06:14.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.478+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31802ab70 con 0x7fd30c004800 2024-05-30T17:06:14.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.478+0000 7fd324a00640 1 -- 172.21.15.67:0/2856775702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31800a570 con 0x7fd30c004800 2024-05-30T17:06:14.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.480+0000 7fd32ce00640 1 -- 172.21.15.67:0/2856775702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 msgr2=0x7fd30c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:14.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.480+0000 7fd32ce00640 1 --2- 172.21.15.67:0/2856775702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 0x7fd30c004c80 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fd31800a9a0 tx=0x7fd30c005830 comp rx=0 tx=0).stop 2024-05-30T17:06:14.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.480+0000 7fd326800640 1 -- 172.21.15.67:0/2856775702 reap_dead start 2024-05-30T17:06:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.481+0000 7fd32ce00640 1 -- 172.21.15.67:0/2856775702 shutdown_connections 2024-05-30T17:06:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.482+0000 7fd32ce00640 1 -- 172.21.15.67:0/2856775702 >> 172.21.15.67:0/2856775702 conn(0x7fd32806bd20 msgr2=0x7fd32806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.482+0000 7fd32ce00640 1 -- 172.21.15.67:0/2856775702 shutdown_connections 2024-05-30T17:06:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.483+0000 7fd32ce00640 1 -- 172.21.15.67:0/2856775702 wait complete. 2024-05-30T17:06:14.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.491+0000 7fd32ce00640 1 Processor -- start 2024-05-30T17:06:14.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.496+0000 7fd32ce00640 1 -- start start 2024-05-30T17:06:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.499+0000 7fd32ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 0x7fd3281602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.499+0000 7fd32ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd32814dda0 0x7fd328160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.499+0000 7fd32ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3280cefc0 0x7fd3280cf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.500+0000 7fd32ce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd328163020 con 0x7fd30c004800 2024-05-30T17:06:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.500+0000 7fd325e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 0x7fd3281602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.500+0000 7fd325e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 0x7fd3281602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40322/0 (socket says 172.21.15.67:40322) 2024-05-30T17:06:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.500+0000 7fd325e00640 1 -- 172.21.15.67:0/3134069792 learned_addr learned my addr 172.21.15.67:0/3134069792 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.500+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd328162ea0 con 0x7fd3280cefc0 2024-05-30T17:06:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.500+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd3281633b0 con 0x7fd32814dda0 2024-05-30T17:06:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.500+0000 7fd325400640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd32814dda0 0x7fd328160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.501+0000 7fd326800640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3280cefc0 0x7fd3280cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:14.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.502+0000 7fd325400640 1 -- 172.21.15.67:0/3134069792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3280cefc0 msgr2=0x7fd3280cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.502+0000 7fd325400640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3280cefc0 0x7fd3280cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.502+0000 7fd325400640 1 -- 172.21.15.67:0/3134069792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 msgr2=0x7fd3281602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.502+0000 7fd325400640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 0x7fd3281602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.502+0000 7fd325400640 1 -- 172.21.15.67:0/3134069792 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd328160e80 con 0x7fd32814dda0 2024-05-30T17:06:14.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.503+0000 7fd325400640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd32814dda0 0x7fd328160820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd30c005520 tx=0x7fd318004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.509+0000 7fd31e000640 1 -- 172.21.15.67:0/3134069792 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318040020 con 0x7fd32814dda0 2024-05-30T17:06:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.511+0000 7fd31e000640 1 -- 172.21.15.67:0/3134069792 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd318038040 con 0x7fd32814dda0 2024-05-30T17:06:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.511+0000 7fd31e000640 1 -- 172.21.15.67:0/3134069792 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318037070 con 0x7fd32814dda0 2024-05-30T17:06:14.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.511+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd328161160 con 0x7fd32814dda0 2024-05-30T17:06:14.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.511+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd328150370 con 0x7fd32814dda0 2024-05-30T17:06:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.516+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd328161330 con 0x7fd32814dda0 2024-05-30T17:06:14.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.519+0000 7fd31e000640 1 -- 172.21.15.67:0/3134069792 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 48) ==== 93947+0+0 (secure 0 0 0) 0x7fd318043020 con 0x7fd32814dda0 2024-05-30T17:06:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.524+0000 7fd31e000640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd30406ea00 0x7fd304070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.525+0000 7fd31e000640 1 -- 172.21.15.67:0/3134069792 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fd3180aaba0 con 0x7fd32814dda0 2024-05-30T17:06:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.529+0000 7fd325e00640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd30406ea00 0x7fd304070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.534+0000 7fd31e000640 1 -- 172.21.15.67:0/3134069792 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd31804a030 con 0x7fd32814dda0 2024-05-30T17:06:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:14.541+0000 7fd325e00640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd30406ea00 0x7fd304070ec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd320000f80 tx=0x7fd320006040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.106+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96441", "target": ["mon-mgr", ""]}) -- 0x7fd3280689e0 con 0x7fd30406ea00 2024-05-30T17:06:15.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.694+0000 7fd31e000640 1 -- 172.21.15.67:0/3134069792 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd3280689e0 con 0x7fd30406ea00 2024-05-30T17:06:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.700+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd30406ea00 msgr2=0x7fd304070ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.700+0000 7fd32ce00640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd30406ea00 0x7fd304070ec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd320000f80 tx=0x7fd320006040 comp rx=0 tx=0).stop 2024-05-30T17:06:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.700+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd32814dda0 msgr2=0x7fd328160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.700+0000 7fd32ce00640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd32814dda0 0x7fd328160820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd30c005520 tx=0x7fd318004730 comp rx=0 tx=0).stop 2024-05-30T17:06:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.701+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 shutdown_connections 2024-05-30T17:06:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.701+0000 7fd32ce00640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd30406ea00 0x7fd304070ec0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.701+0000 7fd32ce00640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3280cefc0 0x7fd3280cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:15.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.701+0000 7fd32ce00640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd32814dda0 0x7fd328160820 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:15.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.701+0000 7fd32ce00640 1 --2- 172.21.15.67:0/3134069792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30c004800 0x7fd3281602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:15.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.701+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 >> 172.21.15.67:0/3134069792 conn(0x7fd32806bd20 msgr2=0x7fd32806ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:15.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.701+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 shutdown_connections 2024-05-30T17:06:15.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:15.701+0000 7fd32ce00640 1 -- 172.21.15.67:0/3134069792 wait complete. 2024-05-30T17:06:15.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96442 2024-05-30T17:06:16.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.160+0000 7fed37a00640 1 Processor -- start 2024-05-30T17:06:16.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.160+0000 7fed37a00640 1 -- start start 2024-05-30T17:06:16.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.163+0000 7fed37a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed300d17b0 0x7fed300cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:16.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.163+0000 7fed37a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fed300b2e50 0x7fed300cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:16.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.164+0000 7fed37a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed300b13d0 con 0x7fed30162130 2024-05-30T17:06:16.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.164+0000 7fed37a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fed300d33b0 con 0x7fed300b2e50 2024-05-30T17:06:16.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.164+0000 7fed37a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed300b1640 con 0x7fed300d17b0 2024-05-30T17:06:16.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.165+0000 7fed35a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fed300b2e50 0x7fed300cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:16.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.165+0000 7fed35a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fed300b2e50 0x7fed300cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40324/0 (socket says 172.21.15.67:40324) 2024-05-30T17:06:16.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.165+0000 7fed35a00640 1 -- 172.21.15.67:0/4218244276 learned_addr learned my addr 172.21.15.67:0/4218244276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:16.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.165+0000 7fed2fe00640 1 --2- 172.21.15.67:0/4218244276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed300d17b0 0x7fed300cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:16.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.168+0000 7fed35a00640 1 -- 172.21.15.67:0/4218244276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed300d17b0 msgr2=0x7fed300cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:16.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.168+0000 7fed35a00640 1 --2- 172.21.15.67:0/4218244276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed300d17b0 0x7fed300cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:16.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.168+0000 7fed35a00640 1 -- 172.21.15.67:0/4218244276 >> v1:172.21.15.67:6789/0 conn(0x7fed30162130 legacy=0x7fed300d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:16.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.168+0000 7fed35a00640 1 -- 172.21.15.67:0/4218244276 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed300b1920 con 0x7fed300b2e50 2024-05-30T17:06:16.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.168+0000 7fed2fe00640 1 --2- 172.21.15.67:0/4218244276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed300d17b0 0x7fed300cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:06:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.173+0000 7fed35a00640 1 --2- 172.21.15.67:0/4218244276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed300b2e50 0x7fed300cf810 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fed28009e50 tx=0x7fed2802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557a7cd8634c3e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.177+0000 7fed2f400640 1 -- 172.21.15.67:0/4218244276 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28036070 con 0x7fed300b2e50 2024-05-30T17:06:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.177+0000 7fed2f400640 1 -- 172.21.15.67:0/4218244276 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed28031600 con 0x7fed300b2e50 2024-05-30T17:06:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.177+0000 7fed2f400640 1 -- 172.21.15.67:0/4218244276 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28031a10 con 0x7fed300b2e50 2024-05-30T17:06:16.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.180+0000 7fed37a00640 1 -- 172.21.15.67:0/4218244276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed300b2e50 msgr2=0x7fed300cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:16.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.180+0000 7fed37a00640 1 --2- 172.21.15.67:0/4218244276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed300b2e50 0x7fed300cf810 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fed28009e50 tx=0x7fed2802a730 comp rx=0 tx=0).stop 2024-05-30T17:06:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.180+0000 7fed37a00640 1 -- 172.21.15.67:0/4218244276 shutdown_connections 2024-05-30T17:06:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.180+0000 7fed37a00640 1 --2- 172.21.15.67:0/4218244276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed300b2e50 0x7fed300cf810 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.180+0000 7fed37a00640 1 --2- 172.21.15.67:0/4218244276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed300d17b0 0x7fed300cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.180+0000 7fed37a00640 1 -- 172.21.15.67:0/4218244276 >> 172.21.15.67:0/4218244276 conn(0x7fed3008bfc0 msgr2=0x7fed300d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.181+0000 7fed37a00640 1 -- 172.21.15.67:0/4218244276 shutdown_connections 2024-05-30T17:06:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.181+0000 7fed37a00640 1 -- 172.21.15.67:0/4218244276 wait complete. 2024-05-30T17:06:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.184+0000 7fed37a00640 1 Processor -- start 2024-05-30T17:06:16.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.184+0000 7fed37a00640 1 -- start start 2024-05-30T17:06:16.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.186+0000 7fed37a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed300b2e50 0x7fed30159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:16.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.186+0000 7fed37a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed300d17b0 0x7fed300cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.186+0000 7fed37a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed30162130 0x7fed300d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.186+0000 7fed37a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fed30162690 con 0x7fed300d17b0 2024-05-30T17:06:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.186+0000 7fed37a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed30162510 con 0x7fed300b2e50 2024-05-30T17:06:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.186+0000 7fed37a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed300d3690 con 0x7fed30162130 2024-05-30T17:06:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.188+0000 7fed2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed300d17b0 0x7fed300cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.188+0000 7fed2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed300d17b0 0x7fed300cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40326/0 (socket says 172.21.15.67:40326) 2024-05-30T17:06:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.188+0000 7fed2fe00640 1 -- 172.21.15.67:0/4133573609 learned_addr learned my addr 172.21.15.67:0/4133573609 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.189+0000 7fed35a00640 1 --2- 172.21.15.67:0/4133573609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed30162130 0x7fed300d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.189+0000 7fed35000640 1 --2- 172.21.15.67:0/4133573609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed300b2e50 0x7fed30159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:16.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.190+0000 7fed2fe00640 1 -- 172.21.15.67:0/4133573609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed30162130 msgr2=0x7fed300d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:16.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.190+0000 7fed2fe00640 1 --2- 172.21.15.67:0/4133573609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed30162130 0x7fed300d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:16.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.190+0000 7fed2fe00640 1 -- 172.21.15.67:0/4133573609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed300b2e50 msgr2=0x7fed30159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:16.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.190+0000 7fed2fe00640 1 --2- 172.21.15.67:0/4133573609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed300b2e50 0x7fed30159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:16.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.190+0000 7fed2fe00640 1 -- 172.21.15.67:0/4133573609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed301e8880 con 0x7fed300d17b0 2024-05-30T17:06:16.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.193+0000 7fed2fe00640 1 --2- 172.21.15.67:0/4133573609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed300d17b0 0x7fed300cfec0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fed2000ebe0 tx=0x7fed2000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:16.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.195+0000 7fed2cc00640 1 -- 172.21.15.67:0/4133573609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20017070 con 0x7fed300d17b0 2024-05-30T17:06:16.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.195+0000 7fed2cc00640 1 -- 172.21.15.67:0/4133573609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed2000cc30 con 0x7fed300d17b0 2024-05-30T17:06:16.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.195+0000 7fed2cc00640 1 -- 172.21.15.67:0/4133573609 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20013470 con 0x7fed300d17b0 2024-05-30T17:06:16.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.196+0000 7fed37a00640 1 -- 172.21.15.67:0/4133573609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed301e8b60 con 0x7fed300d17b0 2024-05-30T17:06:16.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.196+0000 7fed37a00640 1 -- 172.21.15.67:0/4133573609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed301e9120 con 0x7fed300d17b0 2024-05-30T17:06:16.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.199+0000 7fed37a00640 1 -- 172.21.15.67:0/4133573609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed300690f0 con 0x7fed300d17b0 2024-05-30T17:06:16.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.206+0000 7fed2cc00640 1 -- 172.21.15.67:0/4133573609 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 48) ==== 93947+0+0 (secure 0 0 0) 0x7fed2001a070 con 0x7fed300d17b0 2024-05-30T17:06:16.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.209+0000 7fed2cc00640 1 --2- 172.21.15.67:0/4133573609 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fed0c06e900 0x7fed0c070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:16.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.209+0000 7fed2cc00640 1 -- 172.21.15.67:0/4133573609 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fed2008aed0 con 0x7fed300d17b0 2024-05-30T17:06:16.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.211+0000 7fed35000640 1 --2- 172.21.15.67:0/4133573609 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fed0c06e900 0x7fed0c070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:16.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.215+0000 7fed2cc00640 1 -- 172.21.15.67:0/4133573609 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed2005a710 con 0x7fed300d17b0 2024-05-30T17:06:16.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.231+0000 7fed35000640 1 --2- 172.21.15.67:0/4133573609 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fed0c06e900 0x7fed0c070dc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fed301e77d0 tx=0x7fed18009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:16.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:06:16.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:06:16.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:16.804+0000 7fed37a00640 1 -- 172.21.15.67:0/4133573609 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96442", "target": ["mon-mgr", ""]}) -- 0x7fed301618e0 con 0x7fed0c06e900 2024-05-30T17:06:17.130 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:17.129+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:17.130 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:17.129+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:17.130 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:17.129+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:17.130 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:17.129+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:17.130 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:17.129+0000 7f0c89e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-30T17:06:17.456 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:17.455+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T17:06:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.472+0000 7fed2cc00640 1 -- 172.21.15.67:0/4133573609 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fed301618e0 con 0x7fed0c06e900 2024-05-30T17:06:17.476 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T17:06:17.503 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T17:06:17.504 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96443 2024-05-30T17:06:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.949+0000 7f763fe00640 1 Processor -- start 2024-05-30T17:06:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.949+0000 7f763fe00640 1 -- start start 2024-05-30T17:06:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.951+0000 7f763fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f764015e990 0x7f764006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.951+0000 7f763fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400ae2b0 0x7f764006f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.951+0000 7f763fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76400ca5c0 con 0x7f764006ec00 2024-05-30T17:06:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.951+0000 7f763fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f76400cf2e0 con 0x7f764015e990 2024-05-30T17:06:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.951+0000 7f763fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76400cfe60 con 0x7f76400ae2b0 2024-05-30T17:06:17.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.953+0000 7f763f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400ae2b0 0x7f764006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:17.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.953+0000 7f763f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400ae2b0 0x7f764006f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:44952/0 (socket says 172.21.15.67:44952) 2024-05-30T17:06:17.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.953+0000 7f763f400640 1 -- 172.21.15.67:0/3810472159 learned_addr learned my addr 172.21.15.67:0/3810472159 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:17.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.953+0000 7f763ea00640 1 --2- 172.21.15.67:0/3810472159 >> v2:172.21.15.169:3300/0 conn(0x7f764015e990 0x7f764006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:17.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.954+0000 7f763f400640 1 -- 172.21.15.67:0/3810472159 >> v2:172.21.15.169:3300/0 conn(0x7f764015e990 msgr2=0x7f764006e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:17.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.954+0000 7f763f400640 1 --2- 172.21.15.67:0/3810472159 >> v2:172.21.15.169:3300/0 conn(0x7f764015e990 0x7f764006e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:17.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.954+0000 7f763f400640 1 -- 172.21.15.67:0/3810472159 >> v1:172.21.15.169:6789/0 conn(0x7f764006ec00 legacy=0x7f76400c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:17.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.954+0000 7f763f400640 1 -- 172.21.15.67:0/3810472159 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764015ed70 con 0x7f76400ae2b0 2024-05-30T17:06:17.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.956+0000 7f763f400640 1 --2- 172.21.15.67:0/3810472159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400ae2b0 0x7f764006f060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7638009e50 tx=0x7f763802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6363b843c21f7c34 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:17.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.957+0000 7f763e000640 1 -- 172.21.15.67:0/3810472159 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638036070 con 0x7f76400ae2b0 2024-05-30T17:06:17.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.957+0000 7f763e000640 1 -- 172.21.15.67:0/3810472159 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7638031600 con 0x7f76400ae2b0 2024-05-30T17:06:17.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.957+0000 7f763e000640 1 -- 172.21.15.67:0/3810472159 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76380319d0 con 0x7f76400ae2b0 2024-05-30T17:06:17.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.958+0000 7f763fe00640 1 -- 172.21.15.67:0/3810472159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400ae2b0 msgr2=0x7f764006f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:17.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.958+0000 7f763fe00640 1 --2- 172.21.15.67:0/3810472159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400ae2b0 0x7f764006f060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7638009e50 tx=0x7f763802a710 comp rx=0 tx=0).stop 2024-05-30T17:06:17.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.958+0000 7f763fe00640 1 -- 172.21.15.67:0/3810472159 shutdown_connections 2024-05-30T17:06:17.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.958+0000 7f763fe00640 1 --2- 172.21.15.67:0/3810472159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76400ae2b0 0x7f764006f060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:17.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.958+0000 7f763fe00640 1 --2- 172.21.15.67:0/3810472159 >> v2:172.21.15.169:3300/0 conn(0x7f764015e990 0x7f764006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:17.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.959+0000 7f763fe00640 1 -- 172.21.15.67:0/3810472159 >> 172.21.15.67:0/3810472159 conn(0x7f76400b1e50 msgr2=0x7f76400b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:17.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.960+0000 7f763fe00640 1 -- 172.21.15.67:0/3810472159 shutdown_connections 2024-05-30T17:06:17.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.962+0000 7f763fe00640 1 -- 172.21.15.67:0/3810472159 wait complete. 2024-05-30T17:06:17.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.965+0000 7f763fe00640 1 Processor -- start 2024-05-30T17:06:17.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.965+0000 7f763fe00640 1 -- start start 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.967+0000 7f763fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764006ec00 0x7f764014e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.968+0000 7f763fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76400ae2b0 0x7f7640158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.968+0000 7f763fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764015e990 0x7f7640158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.968+0000 7f763fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f76400cfaf0 con 0x7f76400ae2b0 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.968+0000 7f763fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76400cf810 con 0x7f764015e990 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.968+0000 7f763fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76400b2970 con 0x7f764006ec00 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.968+0000 7f763ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764006ec00 0x7f764014e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.969+0000 7f763ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764006ec00 0x7f764014e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:44954/0 (socket says 172.21.15.67:44954) 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.969+0000 7f763ea00640 1 -- 172.21.15.67:0/2273630379 learned_addr learned my addr 172.21.15.67:0/2273630379 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.969+0000 7f763f400640 1 --2- 172.21.15.67:0/2273630379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764015e990 0x7f7640158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.969+0000 7f7636000640 1 --2- 172.21.15.67:0/2273630379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76400ae2b0 0x7f7640158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.970+0000 7f763ea00640 1 -- 172.21.15.67:0/2273630379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764015e990 msgr2=0x7f7640158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.970+0000 7f763ea00640 1 --2- 172.21.15.67:0/2273630379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f764015e990 0x7f7640158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.971+0000 7f763ea00640 1 -- 172.21.15.67:0/2273630379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76400ae2b0 msgr2=0x7f7640158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.971+0000 7f763ea00640 1 --2- 172.21.15.67:0/2273630379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76400ae2b0 0x7f7640158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.971+0000 7f763ea00640 1 -- 172.21.15.67:0/2273630379 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76400b59b0 con 0x7f764006ec00 2024-05-30T17:06:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.971+0000 7f7636000640 1 --2- 172.21.15.67:0/2273630379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76400ae2b0 0x7f7640158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:06:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.975+0000 7f763ea00640 1 --2- 172.21.15.67:0/2273630379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f764006ec00 0x7f764014e920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f763000d620 tx=0x7f763000dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.976+0000 7f7637400640 1 -- 172.21.15.67:0/2273630379 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000b620 con 0x7f764006ec00 2024-05-30T17:06:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.976+0000 7f7637400640 1 -- 172.21.15.67:0/2273630379 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7630019070 con 0x7f764006ec00 2024-05-30T17:06:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.976+0000 7f7637400640 1 -- 172.21.15.67:0/2273630379 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000ba00 con 0x7f764006ec00 2024-05-30T17:06:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.976+0000 7f763fe00640 1 -- 172.21.15.67:0/2273630379 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764014ee60 con 0x7f764006ec00 2024-05-30T17:06:17.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.979+0000 7f763fe00640 1 -- 172.21.15.67:0/2273630379 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7640159470 con 0x7f764006ec00 2024-05-30T17:06:17.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.984+0000 7f763fe00640 1 -- 172.21.15.67:0/2273630379 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7640075560 con 0x7f764006ec00 2024-05-30T17:06:17.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.986+0000 7f7637400640 1 -- 172.21.15.67:0/2273630379 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 48) ==== 93947+0+0 (secure 0 0 0) 0x7f763000bba0 con 0x7f764006ec00 2024-05-30T17:06:17.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.987+0000 7f7637400640 1 --2- 172.21.15.67:0/2273630379 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f761c06e9d0 0x7f761c070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:17.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.988+0000 7f7637400640 1 -- 172.21.15.67:0/2273630379 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f763008af00 con 0x7f764006ec00 2024-05-30T17:06:17.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.988+0000 7f7636000640 1 --2- 172.21.15.67:0/2273630379 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f761c06e9d0 0x7f761c070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:17.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.990+0000 7f7636000640 1 --2- 172.21.15.67:0/2273630379 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f761c06e9d0 0x7f761c070e90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f764009e430 tx=0x7f7628009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:17.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:17.998+0000 7f7637400640 1 -- 172.21.15.67:0/2273630379 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f763005a7c0 con 0x7f764006ec00 2024-05-30T17:06:18.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:18.532+0000 7f763fe00640 1 -- 172.21.15.67:0/2273630379 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96443", "target": ["mon-mgr", ""]}) -- 0x7f76400689e0 con 0x7f761c06e9d0 2024-05-30T17:06:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.001+0000 7f7637400640 1 -- 172.21.15.67:0/2273630379 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mgrmap(e 49) ==== 93998+0+0 (secure 0 0 0) 0x7f7630056e30 con 0x7f764006ec00 2024-05-30T17:06:19.219 INFO:tasks.ceph.mgr.y.smithi067.stderr:Exception in thread Thread-14: 2024-05-30T17:06:19.219 INFO:tasks.ceph.mgr.y.smithi067.stderr:Traceback (most recent call last): 2024-05-30T17:06:19.220 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_v1.py", line 660, in _get_clone_source 2024-05-30T17:06:19.220 INFO:tasks.ceph.mgr.y.smithi067.stderr: 'volume' : self.metadata_mgr.get_option("source", "volume"), 2024-05-30T17:06:19.220 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/share/ceph/mgr/volumes/fs/operations/versions/metadata_manager.py", line 159, in get_option 2024-05-30T17:06:19.220 INFO:tasks.ceph.mgr.y.smithi067.stderr: raise MetadataMgrException(-errno.ENOENT, "section '{0}' does not exist".format(section)) 2024-05-30T17:06:19.220 INFO:tasks.ceph.mgr.y.smithi067.stderr:volumes.fs.exception.MetadataMgrException: -2 (section 'source' does not exist) 2024-05-30T17:06:19.220 INFO:tasks.ceph.mgr.y.smithi067.stderr: 2024-05-30T17:06:19.221 INFO:tasks.ceph.mgr.y.smithi067.stderr:During handling of the above exception, another exception occurred: 2024-05-30T17:06:19.221 INFO:tasks.ceph.mgr.y.smithi067.stderr: 2024-05-30T17:06:19.221 INFO:tasks.ceph.mgr.y.smithi067.stderr:Traceback (most recent call last): 2024-05-30T17:06:19.221 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-05-30T17:06:19.230 INFO:tasks.ceph.mgr.y.smithi067.stderr: self.run() 2024-05-30T17:06:19.230 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/share/ceph/mgr/mgr_util.py", line 82, in run 2024-05-30T17:06:19.230 INFO:tasks.ceph.mgr.y.smithi067.stderr: self.function(*self.args, **self.kwargs) 2024-05-30T17:06:19.230 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/share/ceph/mgr/volumes/fs/stats_util.py", line 262, in _update_progress_bars 2024-05-30T17:06:19.230 INFO:tasks.ceph.mgr.y.smithi067.stderr: clones = self._get_info_for_all_clones() 2024-05-30T17:06:19.230 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/share/ceph/mgr/volumes/fs/stats_util.py", line 158, in _get_info_for_all_clones 2024-05-30T17:06:19.230 INFO:tasks.ceph.mgr.y.smithi067.stderr: self._get_clone_src_info(fsh, ci) 2024-05-30T17:06:19.230 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/share/ceph/mgr/volumes/fs/stats_util.py", line 135, in _get_clone_src_info 2024-05-30T17:06:19.231 INFO:tasks.ceph.mgr.y.smithi067.stderr: with open_clone_sv_pair_in_vol( 2024-05-30T17:06:19.231 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/lib64/python3.9/contextlib.py", line 119, in __enter__ 2024-05-30T17:06:19.233 INFO:tasks.ceph.mgr.y.smithi067.stderr: return next(self.gen) 2024-05-30T17:06:19.233 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/share/ceph/mgr/volumes/fs/operations/subvolume.py", line 108, in open_clone_sv_pair_in_vol 2024-05-30T17:06:19.233 INFO:tasks.ceph.mgr.y.smithi067.stderr: dst_sv.get_clone_source() 2024-05-30T17:06:19.233 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_v1.py", line 677, in get_clone_source 2024-05-30T17:06:19.233 INFO:tasks.ceph.mgr.y.smithi067.stderr: src = self._get_clone_source() 2024-05-30T17:06:19.234 INFO:tasks.ceph.mgr.y.smithi067.stderr: File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_v1.py", line 673, in _get_clone_source 2024-05-30T17:06:19.234 INFO:tasks.ceph.mgr.y.smithi067.stderr: raise VolumeException(-errno.EINVAL, "error fetching subvolume metadata") 2024-05-30T17:06:19.234 INFO:tasks.ceph.mgr.y.smithi067.stderr:volumes.fs.exception.VolumeException: -22 (error fetching subvolume metadata) 2024-05-30T17:06:19.267 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-05-30T17:06:19.267+0000 7f0c89e00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-30T17:06:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.273+0000 7f7637400640 1 -- 172.21.15.67:0/2273630379 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f76400689e0 con 0x7f761c06e9d0 2024-05-30T17:06:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-30T17:06:19.298 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-30T17:06:19.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96444 2024-05-30T17:06:19.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.703+0000 7f0f43a00640 1 Processor -- start 2024-05-30T17:06:19.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.704+0000 7f0f43a00640 1 -- start start 2024-05-30T17:06:19.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.708+0000 7f0f43a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 0x7f0f3c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:19.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.708+0000 7f0f43a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f3c0d1b80 0x7f0f3c0b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:19.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.708+0000 7f0f43a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f3c0b8000 con 0x7f0f3c15e850 2024-05-30T17:06:19.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.708+0000 7f0f43a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f0f3c0d1ff0 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.708+0000 7f0f43a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3c15ec30 con 0x7f0f3c0cef30 2024-05-30T17:06:19.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.709+0000 7f0f41000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0f3c15e850 0x7f0f3c0b77b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42432/0 (socket says 172.21.15.67:42432) 2024-05-30T17:06:19.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.709+0000 7f0f41000640 1 -- 172.21.15.67:0/3264567726 learned_addr learned my addr 172.21.15.67:0/3264567726 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:19.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.709+0000 7f0f3be00640 1 --2- 172.21.15.67:0/3264567726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 0x7f0f3c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:19.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.710+0000 7f0f41a00640 1 --2- 172.21.15.67:0/3264567726 >> v2:172.21.15.67:3300/0 conn(0x7f0f3c0d1b80 0x7f0f3c0b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:19.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.711+0000 7f0f3be00640 1 -- 172.21.15.67:0/3264567726 >> v2:172.21.15.67:3300/0 conn(0x7f0f3c0d1b80 msgr2=0x7f0f3c0b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:19.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.711+0000 7f0f3be00640 1 --2- 172.21.15.67:0/3264567726 >> v2:172.21.15.67:3300/0 conn(0x7f0f3c0d1b80 0x7f0f3c0b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:19.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.711+0000 7f0f3be00640 1 -- 172.21.15.67:0/3264567726 >> v1:172.21.15.67:6789/0 conn(0x7f0f3c15e850 legacy=0x7f0f3c0b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:19.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.711+0000 7f0f3be00640 1 -- 172.21.15.67:0/3264567726 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c0b82e0 con 0x7f0f3c0cef30 2024-05-30T17:06:19.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.713+0000 7f0f3be00640 1 --2- 172.21.15.67:0/3264567726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 0x7f0f3c0d1640 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f280097a0 tx=0x7f0f2802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6337b552301815d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:19.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.714+0000 7f0f3b400640 1 -- 172.21.15.67:0/3264567726 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28037070 con 0x7f0f3c0cef30 2024-05-30T17:06:19.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.714+0000 7f0f3b400640 1 -- 172.21.15.67:0/3264567726 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2802ae40 con 0x7f0f3c0cef30 2024-05-30T17:06:19.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.714+0000 7f0f3b400640 1 -- 172.21.15.67:0/3264567726 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28032660 con 0x7f0f3c0cef30 2024-05-30T17:06:19.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.716+0000 7f0f43a00640 1 -- 172.21.15.67:0/3264567726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 msgr2=0x7f0f3c0d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:19.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.716+0000 7f0f43a00640 1 --2- 172.21.15.67:0/3264567726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 0x7f0f3c0d1640 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f280097a0 tx=0x7f0f2802a0f0 comp rx=0 tx=0).stop 2024-05-30T17:06:19.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.722+0000 7f0f43a00640 1 -- 172.21.15.67:0/3264567726 shutdown_connections 2024-05-30T17:06:19.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.722+0000 7f0f43a00640 1 --2- 172.21.15.67:0/3264567726 >> v2:172.21.15.67:3300/0 conn(0x7f0f3c0d1b80 0x7f0f3c0b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:19.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.722+0000 7f0f43a00640 1 --2- 172.21.15.67:0/3264567726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 0x7f0f3c0d1640 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:19.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.722+0000 7f0f43a00640 1 -- 172.21.15.67:0/3264567726 >> 172.21.15.67:0/3264567726 conn(0x7f0f3c067aa0 msgr2=0x7f0f3c067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:19.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.723+0000 7f0f43a00640 1 -- 172.21.15.67:0/3264567726 shutdown_connections 2024-05-30T17:06:19.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.726+0000 7f0f43a00640 1 -- 172.21.15.67:0/3264567726 wait complete. 2024-05-30T17:06:19.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.731+0000 7f0f43a00640 1 Processor -- start 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.732+0000 7f0f43a00640 1 -- start start 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.733+0000 7f0f43a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 0x7f0f3c1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.734+0000 7f0f43a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f3c0d1b80 0x7f0f3c0d3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.734+0000 7f0f43a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f3c15e850 0x7f0f3c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.734+0000 7f0f43a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0b95e0 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.734+0000 7f0f43a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0b88a0 con 0x7f0f3c15e850 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.734+0000 7f0f43a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3c0b9760 con 0x7f0f3c0cef30 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.734+0000 7f0f3be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f3c0d1b80 0x7f0f3c0d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.734+0000 7f0f3be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f3c0d1b80 0x7f0f3c0d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40362/0 (socket says 172.21.15.67:40362) 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.734+0000 7f0f3be00640 1 -- 172.21.15.67:0/3724679053 learned_addr learned my addr 172.21.15.67:0/3724679053 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.735+0000 7f0f41000640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 0x7f0f3c1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.735+0000 7f0f3be00640 1 -- 172.21.15.67:0/3724679053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 msgr2=0x7f0f3c1f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.735+0000 7f0f3be00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 0x7f0f3c1f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.735+0000 7f0f3be00640 1 -- 172.21.15.67:0/3724679053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f3c15e850 msgr2=0x7f0f3c0cd970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-30T17:06:19.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.735+0000 7f0f3be00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f3c15e850 0x7f0f3c0cd970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:19.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.735+0000 7f0f3be00640 1 -- 172.21.15.67:0/3724679053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c0d26e0 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.739+0000 7f0f3be00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f3c0d1b80 0x7f0f3c0d3800 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f0f28035040 tx=0x7f0f280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:19.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.743+0000 7f0f38c00640 1 -- 172.21.15.67:0/3724679053 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28041070 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.743+0000 7f0f38c00640 1 -- 172.21.15.67:0/3724679053 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2803c670 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.743+0000 7f0f38c00640 1 -- 172.21.15.67:0/3724679053 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28037040 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.744+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c0d29c0 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.745+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c0d2f80 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.748+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3c0689e0 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.759+0000 7f0f38c00640 1 -- 172.21.15.67:0/3724679053 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 49) ==== 93998+0+0 (secure 0 0 0) 0x7f0f28065070 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.761+0000 7f0f38c00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0f1c06ea60 0x7f0f1c070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.761+0000 7f0f38c00640 1 -- 172.21.15.67:0/3724679053 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f0f280ab850 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.766+0000 7f0f38c00640 1 -- 172.21.15.67:0/3724679053 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f2807b760 con 0x7f0f3c0d1b80 2024-05-30T17:06:19.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.775+0000 7f0f41000640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0f1c06ea60 0x7f0f1c070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:19.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:19.798+0000 7f0f41000640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0f1c06ea60 0x7f0f1c070f20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005880 tx=0x7f0f30009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:20.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:20.352+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96444", "target": ["mon-mgr", ""]}) -- 0x7f0f3c1f3f40 con 0x7f0f1c06ea60 2024-05-30T17:06:21.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.728+0000 7f0f38c00640 1 -- 172.21.15.67:0/3724679053 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0f3c1f3f40 con 0x7f0f1c06ea60 2024-05-30T17:06:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.735+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0f1c06ea60 msgr2=0x7f0f1c070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.735+0000 7f0f43a00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0f1c06ea60 0x7f0f1c070f20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005880 tx=0x7f0f30009040 comp rx=0 tx=0).stop 2024-05-30T17:06:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.735+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f3c0d1b80 msgr2=0x7f0f3c0d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.735+0000 7f0f43a00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f3c0d1b80 0x7f0f3c0d3800 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f0f28035040 tx=0x7f0f280046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.736+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 shutdown_connections 2024-05-30T17:06:21.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.736+0000 7f0f43a00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f0f1c06ea60 0x7f0f1c070f20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.736+0000 7f0f43a00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f3c15e850 0x7f0f3c0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.736+0000 7f0f43a00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f3c0d1b80 0x7f0f3c0d3800 secure :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f0f28035040 tx=0x7f0f280046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.736+0000 7f0f43a00640 1 --2- 172.21.15.67:0/3724679053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f3c0cef30 0x7f0f3c1f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.736+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 >> 172.21.15.67:0/3724679053 conn(0x7f0f3c067aa0 msgr2=0x7f0f3c0cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:21.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.739+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 shutdown_connections 2024-05-30T17:06:21.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:21.739+0000 7f0f43a00640 1 -- 172.21.15.67:0/3724679053 wait complete. 2024-05-30T17:06:21.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96445 2024-05-30T17:06:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.255+0000 7f20d9200640 1 Processor -- start 2024-05-30T17:06:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.256+0000 7f20d9200640 1 -- start start 2024-05-30T17:06:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.259+0000 7f20d9200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f20d4162b20 0x7f20d4167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.259+0000 7f20d9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d40d1570 0x7f20d40d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.259+0000 7f20d9200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d4162f00 con 0x7f20d40b7500 2024-05-30T17:06:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.259+0000 7f20d9200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f20d40d1f20 con 0x7f20d4162b20 2024-05-30T17:06:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.259+0000 7f20d9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20d40cf4f0 con 0x7f20d40d1570 2024-05-30T17:06:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.260+0000 7f20d2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d40d1570 0x7f20d40d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.260+0000 7f20d2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d40d1570 0x7f20d40d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:38440/0 (socket says 172.21.15.67:38440) 2024-05-30T17:06:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.260+0000 7f20d2800640 1 -- 172.21.15.67:0/1811336257 learned_addr learned my addr 172.21.15.67:0/1811336257 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.262+0000 7f20d1e00640 1 --2- 172.21.15.67:0/1811336257 >> v2:172.21.15.67:3300/0 conn(0x7f20d4162b20 0x7f20d4167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:22.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.266+0000 7f20d2800640 1 -- 172.21.15.67:0/1811336257 >> v2:172.21.15.67:3300/0 conn(0x7f20d4162b20 msgr2=0x7f20d4167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:22.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.266+0000 7f20d2800640 1 --2- 172.21.15.67:0/1811336257 >> v2:172.21.15.67:3300/0 conn(0x7f20d4162b20 0x7f20d4167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:22.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.266+0000 7f20d2800640 1 -- 172.21.15.67:0/1811336257 >> v1:172.21.15.67:6789/0 conn(0x7f20d40b7500 legacy=0x7f20d40d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:22.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.266+0000 7f20d2800640 1 -- 172.21.15.67:0/1811336257 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d40d4190 con 0x7f20d40d1570 2024-05-30T17:06:22.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.267+0000 7f20d1e00640 1 --2- 172.21.15.67:0/1811336257 >> v2:172.21.15.67:3300/0 conn(0x7f20d4162b20 0x7f20d4167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:06:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.270+0000 7f20d2800640 1 --2- 172.21.15.67:0/1811336257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d40d1570 0x7f20d40d19e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009e50 tx=0x7f20cc02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d6a87588411a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.271+0000 7f20d1400640 1 -- 172.21.15.67:0/1811336257 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc036070 con 0x7f20d40d1570 2024-05-30T17:06:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.272+0000 7f20d1400640 1 -- 172.21.15.67:0/1811336257 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20cc031600 con 0x7f20d40d1570 2024-05-30T17:06:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.272+0000 7f20d1400640 1 -- 172.21.15.67:0/1811336257 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc0319d0 con 0x7f20d40d1570 2024-05-30T17:06:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.273+0000 7f20d9200640 1 -- 172.21.15.67:0/1811336257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d40d1570 msgr2=0x7f20d40d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:22.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.273+0000 7f20d9200640 1 --2- 172.21.15.67:0/1811336257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d40d1570 0x7f20d40d19e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009e50 tx=0x7f20cc02a680 comp rx=0 tx=0).stop 2024-05-30T17:06:22.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.275+0000 7f20d9200640 1 -- 172.21.15.67:0/1811336257 shutdown_connections 2024-05-30T17:06:22.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.275+0000 7f20d9200640 1 --2- 172.21.15.67:0/1811336257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d40d1570 0x7f20d40d19e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:22.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.275+0000 7f20d9200640 1 --2- 172.21.15.67:0/1811336257 >> v2:172.21.15.67:3300/0 conn(0x7f20d4162b20 0x7f20d4167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:22.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.275+0000 7f20d9200640 1 -- 172.21.15.67:0/1811336257 >> 172.21.15.67:0/1811336257 conn(0x7f20d4067aa0 msgr2=0x7f20d4067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:22.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.279+0000 7f20d9200640 1 -- 172.21.15.67:0/1811336257 shutdown_connections 2024-05-30T17:06:22.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.280+0000 7f20d9200640 1 -- 172.21.15.67:0/1811336257 wait complete. 2024-05-30T17:06:22.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.283+0000 7f20d9200640 1 Processor -- start 2024-05-30T17:06:22.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.284+0000 7f20d9200640 1 -- start start 2024-05-30T17:06:22.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.286+0000 7f20d9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d40b7500 0x7f20d41ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:22.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.286+0000 7f20d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d40d1570 0x7f20d41f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:22.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.286+0000 7f20d9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d4162b20 0x7f20d41ec400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:22.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.286+0000 7f20d9200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f20d40d4670 con 0x7f20d40d1570 2024-05-30T17:06:22.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.286+0000 7f20d9200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20d40cf210 con 0x7f20d40b7500 2024-05-30T17:06:22.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.286+0000 7f20d9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20d40d47f0 con 0x7f20d4162b20 2024-05-30T17:06:22.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.287+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d40d1570 0x7f20d41f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:22.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.287+0000 7f20d1e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d40b7500 0x7f20d41ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:22.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.287+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d40d1570 0x7f20d41f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33022/0 (socket says 172.21.15.67:33022) 2024-05-30T17:06:22.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.287+0000 7f20cbe00640 1 -- 172.21.15.67:0/1160497107 learned_addr learned my addr 172.21.15.67:0/1160497107 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:22.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.287+0000 7f20d2800640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d4162b20 0x7f20d41ec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:22.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.288+0000 7f20d1e00640 1 -- 172.21.15.67:0/1160497107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d4162b20 msgr2=0x7f20d41ec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:22.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.288+0000 7f20d1e00640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d4162b20 0x7f20d41ec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:22.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.288+0000 7f20d1e00640 1 -- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d40d1570 msgr2=0x7f20d41f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:22.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.288+0000 7f20d1e00640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d40d1570 0x7f20d41f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:22.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.288+0000 7f20d1e00640 1 -- 172.21.15.67:0/1160497107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d41ec940 con 0x7f20d40b7500 2024-05-30T17:06:22.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.288+0000 7f20cbe00640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d40d1570 0x7f20d41f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:06:22.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.289+0000 7f20d1e00640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d40b7500 0x7f20d41ed0b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f20c400d5f0 tx=0x7f20c400dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.289+0000 7f20ca000640 1 -- 172.21.15.67:0/1160497107 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4004030 con 0x7f20d40b7500 2024-05-30T17:06:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.289+0000 7f20ca000640 1 -- 172.21.15.67:0/1160497107 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c40041d0 con 0x7f20d40b7500 2024-05-30T17:06:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.289+0000 7f20ca000640 1 -- 172.21.15.67:0/1160497107 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4014620 con 0x7f20d40b7500 2024-05-30T17:06:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.289+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d4163f40 con 0x7f20d40b7500 2024-05-30T17:06:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.290+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d4164500 con 0x7f20d40b7500 2024-05-30T17:06:22.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.292+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ac005290 con 0x7f20d40b7500 2024-05-30T17:06:22.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.298+0000 7f20ca000640 1 -- 172.21.15.67:0/1160497107 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 49) ==== 93998+0+0 (secure 0 0 0) 0x7f20c40147c0 con 0x7f20d40b7500 2024-05-30T17:06:22.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.300+0000 7f20ca000640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20b006e990 0x7f20b0070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:22.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.300+0000 7f20ca000640 1 -- 172.21.15.67:0/1160497107 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f20c408b1c0 con 0x7f20d40b7500 2024-05-30T17:06:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.300+0000 7f20cbe00640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20b006e990 0x7f20b0070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:22.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.301+0000 7f20cbe00640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20b006e990 0x7f20b0070e50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f20d41633a0 tx=0x7f20bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:22.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.308+0000 7f20ca000640 1 -- 172.21.15.67:0/1160497107 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20c405aa00 con 0x7f20d40b7500 2024-05-30T17:06:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:22.892+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96445", "target": ["mon-mgr", ""]}) -- 0x7f20ac000bc0 con 0x7f20b006e990 2024-05-30T17:06:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.669+0000 7f20ca000640 1 -- 172.21.15.67:0/1160497107 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20ac000bc0 con 0x7f20b006e990 2024-05-30T17:06:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.673+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20b006e990 msgr2=0x7f20b0070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.673+0000 7f20d9200640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20b006e990 0x7f20b0070e50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f20d41633a0 tx=0x7f20bc009040 comp rx=0 tx=0).stop 2024-05-30T17:06:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.674+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d40b7500 msgr2=0x7f20d41ed0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.674+0000 7f20d9200640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d40b7500 0x7f20d41ed0b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f20c400d5f0 tx=0x7f20c400dab0 comp rx=0 tx=0).stop 2024-05-30T17:06:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.675+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 shutdown_connections 2024-05-30T17:06:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.675+0000 7f20d9200640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f20b006e990 0x7f20b0070e50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.675+0000 7f20d9200640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20d4162b20 0x7f20d41ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.675+0000 7f20d9200640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20d40d1570 0x7f20d41f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.675+0000 7f20d9200640 1 --2- 172.21.15.67:0/1160497107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20d40b7500 0x7f20d41ed0b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.675+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 >> 172.21.15.67:0/1160497107 conn(0x7f20d4067aa0 msgr2=0x7f20d40b5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.675+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 shutdown_connections 2024-05-30T17:06:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:23.675+0000 7f20d9200640 1 -- 172.21.15.67:0/1160497107 wait complete. 2024-05-30T17:06:23.719 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96446 2024-05-30T17:06:24.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.094+0000 7f0669e00640 1 Processor -- start 2024-05-30T17:06:24.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.095+0000 7f0669e00640 1 -- start start 2024-05-30T17:06:24.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.097+0000 7f0669e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f06640b61e0 0x7f06640b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:24.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.097+0000 7f0669e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06640cf7d0 0x7f06640b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:24.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.097+0000 7f0669e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06640b6b00 con 0x7f06641634e0 2024-05-30T17:06:24.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.097+0000 7f0669e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f06640d43d0 con 0x7f06640cf7d0 2024-05-30T17:06:24.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.097+0000 7f0669e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f06640d3330 con 0x7f06640b61e0 2024-05-30T17:06:24.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.098+0000 7f0662a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f06640b61e0 0x7f06640b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:24.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.098+0000 7f0663400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06640cf7d0 0x7f06640b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:24.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.098+0000 7f0663400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06640cf7d0 0x7f06640b76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33032/0 (socket says 172.21.15.67:33032) 2024-05-30T17:06:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.098+0000 7f0663400640 1 -- 172.21.15.67:0/1516251531 learned_addr learned my addr 172.21.15.67:0/1516251531 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.103+0000 7f0662a00640 1 -- 172.21.15.67:0/1516251531 >> v2:172.21.15.67:3300/0 conn(0x7f06640cf7d0 msgr2=0x7f06640b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.103+0000 7f0662a00640 1 --2- 172.21.15.67:0/1516251531 >> v2:172.21.15.67:3300/0 conn(0x7f06640cf7d0 0x7f06640b76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:24.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.103+0000 7f0662a00640 1 -- 172.21.15.67:0/1516251531 >> v1:172.21.15.169:6789/0 conn(0x7f06641634e0 legacy=0x7f06640cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:24.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.100+0000 7f0661600640 1 -- 172.21.15.67:0/1516251531 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915940548 0 0) 0x7f06640b6b00 con 0x7f06641634e0 2024-05-30T17:06:24.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.103+0000 7f0662a00640 1 -- 172.21.15.67:0/1516251531 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06640d3610 con 0x7f06640b61e0 2024-05-30T17:06:24.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.108+0000 7f0662a00640 1 --2- 172.21.15.67:0/1516251531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640b61e0 0x7f06640b65c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f06640ced60 tx=0x7f065402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e151ba87f6db44 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:24.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.108+0000 7f0661600640 1 -- 172.21.15.67:0/1516251531 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654037070 con 0x7f06640b61e0 2024-05-30T17:06:24.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.108+0000 7f0661600640 1 -- 172.21.15.67:0/1516251531 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0654032440 con 0x7f06640b61e0 2024-05-30T17:06:24.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.108+0000 7f0661600640 1 -- 172.21.15.67:0/1516251531 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654032850 con 0x7f06640b61e0 2024-05-30T17:06:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.111+0000 7f0669e00640 1 -- 172.21.15.67:0/1516251531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640b61e0 msgr2=0x7f06640b65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:24.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.111+0000 7f0669e00640 1 --2- 172.21.15.67:0/1516251531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640b61e0 0x7f06640b65c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f06640ced60 tx=0x7f065402a1c0 comp rx=0 tx=0).stop 2024-05-30T17:06:24.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.113+0000 7f0669e00640 1 -- 172.21.15.67:0/1516251531 shutdown_connections 2024-05-30T17:06:24.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.113+0000 7f0669e00640 1 --2- 172.21.15.67:0/1516251531 >> v2:172.21.15.67:3300/0 conn(0x7f06640cf7d0 0x7f06640b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:24.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.113+0000 7f0669e00640 1 --2- 172.21.15.67:0/1516251531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640b61e0 0x7f06640b65c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:24.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.113+0000 7f0669e00640 1 -- 172.21.15.67:0/1516251531 >> 172.21.15.67:0/1516251531 conn(0x7f06640d13c0 msgr2=0x7f06640d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:24.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.113+0000 7f0669e00640 1 -- 172.21.15.67:0/1516251531 shutdown_connections 2024-05-30T17:06:24.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.114+0000 7f0669e00640 1 -- 172.21.15.67:0/1516251531 wait complete. 2024-05-30T17:06:24.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.117+0000 7f0669e00640 1 Processor -- start 2024-05-30T17:06:24.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.117+0000 7f0669e00640 1 -- start start 2024-05-30T17:06:24.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.119+0000 7f0669e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640b61e0 0x7f066414ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:24.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.119+0000 7f0669e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06640cf7d0 0x7f06640ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:24.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.119+0000 7f0669e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06641634e0 0x7f066414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:24.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.119+0000 7f0669e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f06640d3eb0 con 0x7f06641634e0 2024-05-30T17:06:24.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.119+0000 7f0669e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f06640d3bd0 con 0x7f06640b61e0 2024-05-30T17:06:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.120+0000 7f0669e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f06640b7bf0 con 0x7f06640cf7d0 2024-05-30T17:06:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.121+0000 7f0663400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06641634e0 0x7f066414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:24.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.121+0000 7f0662000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06640cf7d0 0x7f06640ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:24.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.121+0000 7f0662a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640b61e0 0x7f066414ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:24.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.121+0000 7f0662000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06640cf7d0 0x7f06640ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:38456/0 (socket says 172.21.15.67:38456) 2024-05-30T17:06:24.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.121+0000 7f0662000640 1 -- 172.21.15.67:0/640613424 learned_addr learned my addr 172.21.15.67:0/640613424 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:24.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.123+0000 7f0663400640 1 -- 172.21.15.67:0/640613424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06640cf7d0 msgr2=0x7f06640ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:24.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.123+0000 7f0663400640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06640cf7d0 0x7f06640ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:24.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.123+0000 7f0663400640 1 -- 172.21.15.67:0/640613424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640b61e0 msgr2=0x7f066414ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:24.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.123+0000 7f0663400640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640b61e0 0x7f066414ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:24.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.123+0000 7f0663400640 1 -- 172.21.15.67:0/640613424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066414f230 con 0x7f06641634e0 2024-05-30T17:06:24.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.124+0000 7f0663400640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06641634e0 0x7f066414e2b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f065c00efc0 tx=0x7f065c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:24.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.125+0000 7f065b400640 1 -- 172.21.15.67:0/640613424 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c00cd20 con 0x7f06641634e0 2024-05-30T17:06:24.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.125+0000 7f065b400640 1 -- 172.21.15.67:0/640613424 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c003990 con 0x7f06641634e0 2024-05-30T17:06:24.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.125+0000 7f065b400640 1 -- 172.21.15.67:0/640613424 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c003d80 con 0x7f06641634e0 2024-05-30T17:06:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.125+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066414f510 con 0x7f06641634e0 2024-05-30T17:06:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.125+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066414fad0 con 0x7f06641634e0 2024-05-30T17:06:24.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.128+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06640689e0 con 0x7f06641634e0 2024-05-30T17:06:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.134+0000 7f065b400640 1 -- 172.21.15.67:0/640613424 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 49) ==== 93998+0+0 (secure 0 0 0) 0x7f065c005d10 con 0x7f06641634e0 2024-05-30T17:06:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.137+0000 7f065b400640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f064006e990 0x7f0640070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.137+0000 7f065b400640 1 -- 172.21.15.67:0/640613424 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f065c08bf70 con 0x7f06641634e0 2024-05-30T17:06:24.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.145+0000 7f0662a00640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f064006e990 0x7f0640070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:24.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.152+0000 7f065b400640 1 -- 172.21.15.67:0/640613424 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f065c022020 con 0x7f06641634e0 2024-05-30T17:06:24.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.166+0000 7f0662a00640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f064006e990 0x7f0640070e50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0654034000 tx=0x7f065402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:24.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.621+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96446", "target": ["mon-mgr", ""]}) -- 0x7f06640b98e0 con 0x7f064006e990 2024-05-30T17:06:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:24.997+0000 7f065b400640 1 -- 172.21.15.67:0/640613424 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f06640b98e0 con 0x7f064006e990 2024-05-30T17:06:25.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f064006e990 msgr2=0x7f0640070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:25.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f064006e990 0x7f0640070e50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0654034000 tx=0x7f065402e000 comp rx=0 tx=0).stop 2024-05-30T17:06:25.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06641634e0 msgr2=0x7f066414e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:25.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06641634e0 0x7f066414e2b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f065c00efc0 tx=0x7f065c00c490 comp rx=0 tx=0).stop 2024-05-30T17:06:25.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 shutdown_connections 2024-05-30T17:06:25.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f064006e990 0x7f0640070e50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:25.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06641634e0 0x7f066414e2b0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:25.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06640cf7d0 0x7f06640ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:25.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 --2- 172.21.15.67:0/640613424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640b61e0 0x7f066414ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:25.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.003+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 >> 172.21.15.67:0/640613424 conn(0x7f06640d13c0 msgr2=0x7f06640b9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:25.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.006+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 shutdown_connections 2024-05-30T17:06:25.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.006+0000 7f0669e00640 1 -- 172.21.15.67:0/640613424 wait complete. 2024-05-30T17:06:25.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96447 2024-05-30T17:06:25.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.367+0000 7f791a000640 1 Processor -- start 2024-05-30T17:06:25.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.367+0000 7f791a000640 1 -- start start 2024-05-30T17:06:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.370+0000 7f791a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79140d0640 0x7f79140d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.370+0000 7f791a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f791415b6f0 0x7f791406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.370+0000 7f791a000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79140ade90 con 0x7f79140725e0 2024-05-30T17:06:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.370+0000 7f791a000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7914071c40 con 0x7f79140d0640 2024-05-30T17:06:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.370+0000 7f791a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f79140d0f60 con 0x7f791415b6f0 2024-05-30T17:06:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.371+0000 7f7913400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79140d0640 0x7f79140d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.371+0000 7f7913400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79140d0640 0x7f79140d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33054/0 (socket says 172.21.15.67:33054) 2024-05-30T17:06:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.371+0000 7f7913400640 1 -- 172.21.15.67:0/1734806942 learned_addr learned my addr 172.21.15.67:0/1734806942 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.371+0000 7f7913e00640 1 --1- 172.21.15.67:0/1734806942 >> v1:172.21.15.67:6789/0 conn(0x7f79140725e0 0x7f79140ae650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57056/0 (socket says 172.21.15.67:57056) 2024-05-30T17:06:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.371+0000 7f7913400640 1 -- 172.21.15.67:0/1734806942 >> v2:172.21.15.169:3300/0 conn(0x7f791415b6f0 msgr2=0x7f791406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.371+0000 7f7912a00640 1 --2- 172.21.15.67:0/1734806942 >> v2:172.21.15.169:3300/0 conn(0x7f791415b6f0 0x7f791406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.373+0000 7f7913400640 1 --2- 172.21.15.67:0/1734806942 >> v2:172.21.15.169:3300/0 conn(0x7f791415b6f0 0x7f791406e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.373+0000 7f7913400640 1 -- 172.21.15.67:0/1734806942 >> v1:172.21.15.67:6789/0 conn(0x7f79140725e0 legacy=0x7f79140ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.373+0000 7f7913400640 1 -- 172.21.15.67:0/1734806942 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7914071e70 con 0x7f79140d0640 2024-05-30T17:06:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.373+0000 7f7912a00640 1 --2- 172.21.15.67:0/1734806942 >> v2:172.21.15.169:3300/0 conn(0x7f791415b6f0 0x7f791406e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.375+0000 7f7913400640 1 --2- 172.21.15.67:0/1734806942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140d0640 0x7f79140d0a20 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f79140d0220 tx=0x7f790402a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac7b0eb2d436cc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:25.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.376+0000 7f7912000640 1 -- 172.21.15.67:0/1734806942 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904037070 con 0x7f79140d0640 2024-05-30T17:06:25.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.376+0000 7f7912000640 1 -- 172.21.15.67:0/1734806942 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7904032440 con 0x7f79140d0640 2024-05-30T17:06:25.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.376+0000 7f7912000640 1 -- 172.21.15.67:0/1734806942 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904032810 con 0x7f79140d0640 2024-05-30T17:06:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.377+0000 7f791a000640 1 -- 172.21.15.67:0/1734806942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140d0640 msgr2=0x7f79140d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.377+0000 7f791a000640 1 --2- 172.21.15.67:0/1734806942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140d0640 0x7f79140d0a20 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f79140d0220 tx=0x7f790402a280 comp rx=0 tx=0).stop 2024-05-30T17:06:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.378+0000 7f791a000640 1 -- 172.21.15.67:0/1734806942 shutdown_connections 2024-05-30T17:06:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.378+0000 7f791a000640 1 --2- 172.21.15.67:0/1734806942 >> v2:172.21.15.169:3300/0 conn(0x7f791415b6f0 0x7f791406e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:25.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.378+0000 7f791a000640 1 --2- 172.21.15.67:0/1734806942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140d0640 0x7f79140d0a20 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:25.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.378+0000 7f791a000640 1 -- 172.21.15.67:0/1734806942 >> 172.21.15.67:0/1734806942 conn(0x7f791406bd20 msgr2=0x7f791406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:25.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.378+0000 7f791a000640 1 -- 172.21.15.67:0/1734806942 shutdown_connections 2024-05-30T17:06:25.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.378+0000 7f791a000640 1 -- 172.21.15.67:0/1734806942 wait complete. 2024-05-30T17:06:25.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.381+0000 7f791a000640 1 Processor -- start 2024-05-30T17:06:25.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.381+0000 7f791a000640 1 -- start start 2024-05-30T17:06:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.382+0000 7f791a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140725e0 0x7f7914099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.382+0000 7f791a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79140d0640 0x7f7914149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.382+0000 7f791a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791415b6f0 0x7f79141499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.382+0000 7f791a000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f791406e890 con 0x7f79140725e0 2024-05-30T17:06:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.382+0000 7f791a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79140ae130 con 0x7f79140d0640 2024-05-30T17:06:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.382+0000 7f791a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79140aed60 con 0x7f791415b6f0 2024-05-30T17:06:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.383+0000 7f7913400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140725e0 0x7f7914099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.383+0000 7f7913400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140725e0 0x7f7914099980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33056/0 (socket says 172.21.15.67:33056) 2024-05-30T17:06:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.383+0000 7f7913400640 1 -- 172.21.15.67:0/2339198873 learned_addr learned my addr 172.21.15.67:0/2339198873 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.383+0000 7f7913e00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791415b6f0 0x7f79141499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.383+0000 7f7913400640 1 -- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791415b6f0 msgr2=0x7f79141499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.383+0000 7f7913400640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791415b6f0 0x7f79141499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.383+0000 7f7912a00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79140d0640 0x7f7914149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.383+0000 7f7913400640 1 -- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79140d0640 msgr2=0x7f7914149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.383+0000 7f7913400640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79140d0640 0x7f7914149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.384+0000 7f7913400640 1 -- 172.21.15.67:0/2339198873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7914099ec0 con 0x7f79140725e0 2024-05-30T17:06:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.384+0000 7f7913e00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791415b6f0 0x7f79141499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:06:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.384+0000 7f7912a00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79140d0640 0x7f7914149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.384+0000 7f7913400640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140725e0 0x7f7914099980 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f790402a2c0 tx=0x7f79040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:25.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.384+0000 7f790b400640 1 -- 172.21.15.67:0/2339198873 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904037040 con 0x7f79140725e0 2024-05-30T17:06:25.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.384+0000 7f790b400640 1 -- 172.21.15.67:0/2339198873 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7904042070 con 0x7f79140725e0 2024-05-30T17:06:25.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.384+0000 7f791a000640 1 -- 172.21.15.67:0/2339198873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791415c2e0 con 0x7f79140725e0 2024-05-30T17:06:25.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.384+0000 7f790b400640 1 -- 172.21.15.67:0/2339198873 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904030bb0 con 0x7f79140725e0 2024-05-30T17:06:25.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.385+0000 7f791a000640 1 -- 172.21.15.67:0/2339198873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791415c860 con 0x7f79140725e0 2024-05-30T17:06:25.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.386+0000 7f7908c00640 1 -- 172.21.15.67:0/2339198873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e8005290 con 0x7f79140725e0 2024-05-30T17:06:25.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.389+0000 7f790b400640 1 -- 172.21.15.67:0/2339198873 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 49) ==== 93998+0+0 (secure 0 0 0) 0x7f7904047460 con 0x7f79140725e0 2024-05-30T17:06:25.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.390+0000 7f790b400640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f78f406e990 0x7f78f4070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:25.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.390+0000 7f790b400640 1 -- 172.21.15.67:0/2339198873 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f79040ac310 con 0x7f79140725e0 2024-05-30T17:06:25.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.391+0000 7f7912a00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f78f406e990 0x7f78f4070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:25.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.392+0000 7f7912a00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f78f406e990 0x7f78f4070e50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7900006fd0 tx=0x7f7900008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:25.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.395+0000 7f790b400640 1 -- 172.21.15.67:0/2339198873 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7914099ec0 con 0x7f79140725e0 2024-05-30T17:06:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:25.838+0000 7f7908c00640 1 -- 172.21.15.67:0/2339198873 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96447", "target": ["mon-mgr", ""]}) -- 0x7f78e8000bc0 con 0x7f78f406e990 2024-05-30T17:06:26.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.028+0000 7f790b400640 1 -- 172.21.15.67:0/2339198873 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f78e8000bc0 con 0x7f78f406e990 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 -- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f78f406e990 msgr2=0x7f78f4070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f78f406e990 0x7f78f4070e50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7900006fd0 tx=0x7f7900008040 comp rx=0 tx=0).stop 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 -- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140725e0 msgr2=0x7f7914099980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140725e0 0x7f7914099980 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f790402a2c0 tx=0x7f79040046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 -- 172.21.15.67:0/2339198873 shutdown_connections 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f78f406e990 0x7f78f4070e50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f791415b6f0 0x7f79141499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79140d0640 0x7f7914149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 --2- 172.21.15.67:0/2339198873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140725e0 0x7f7914099980 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 -- 172.21.15.67:0/2339198873 >> 172.21.15.67:0/2339198873 conn(0x7f791406bd20 msgr2=0x7f79140b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 -- 172.21.15.67:0/2339198873 shutdown_connections 2024-05-30T17:06:26.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.031+0000 7f7908c00640 1 -- 172.21.15.67:0/2339198873 wait complete. 2024-05-30T17:06:26.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:26.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.240+0000 7f907dc00640 1 Processor -- start 2024-05-30T17:06:26.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.241+0000 7f907dc00640 1 -- start start 2024-05-30T17:06:26.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.242+0000 7f907dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f90780d4c40 0x7f90780d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:26.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.242+0000 7f907dc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90780d3060 con 0x7f907815e690 2024-05-30T17:06:26.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.242+0000 7f907dc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90780d3340 con 0x7f90780d1ca0 2024-05-30T17:06:26.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.242+0000 7f907dc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f90780d3620 con 0x7f90780d4c40 2024-05-30T17:06:26.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.242+0000 7f9077e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f907815e690 0x7f90780d4530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57068/0 (socket says 172.21.15.67:57068) 2024-05-30T17:06:26.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.242+0000 7f9077e00640 1 -- 172.21.15.67:0/982186773 learned_addr learned my addr 172.21.15.67:0/982186773 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1742988898 0 0) 0x7f90780d3060 con 0x7f907815e690 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f905c003520 con 0x7f907815e690 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9077400640 1 --2- 172.21.15.67:0/982186773 >> v2:172.21.15.169:3300/0 conn(0x7f90780d4c40 0x7f90780d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2844746051 0 0) 0x7f905c003520 con 0x7f907815e690 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 >> v2:172.21.15.169:3300/0 conn(0x7f90780d4c40 msgr2=0x7f90780d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 --2- 172.21.15.67:0/982186773 >> v2:172.21.15.169:3300/0 conn(0x7f90780d4c40 0x7f90780d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 >> v1:172.21.15.169:6789/0 conn(0x7f90780d1ca0 legacy=0x7f90780d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90780cf770 con 0x7f907815e690 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9068001670 con 0x7f907815e690 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 >> v1:172.21.15.67:6789/0 conn(0x7f907815e690 legacy=0x7f90780d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.243+0000 7f9076a00640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f905c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9076a00640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f905c004800 0x7f905c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9076a00640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f905c005210 0x7f905c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f90780d3620 con 0x7f905c003a00 2024-05-30T17:06:26.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f905c00c160 con 0x7f905c005210 2024-05-30T17:06:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f905c00c360 con 0x7f905c004800 2024-05-30T17:06:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9077e00640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f905c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9077e00640 1 -- 172.21.15.67:0/982186773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f905c004800 msgr2=0x7f905c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9077e00640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f905c004800 0x7f905c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f907d200640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f905c005210 0x7f905c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9077e00640 1 -- 172.21.15.67:0/982186773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f905c005210 msgr2=0x7f905c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9077e00640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f905c005210 0x7f905c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f9077e00640 1 -- 172.21.15.67:0/982186773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c003520 con 0x7f905c003a00 2024-05-30T17:06:26.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.244+0000 7f907d200640 1 -- 172.21.15.67:0/982186773 reap_dead start 2024-05-30T17:06:26.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.245+0000 7f9077e00640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f905c003e20 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f906800c0e0 tx=0x7f905c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e0f5541e949f16 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:26.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.245+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068007c20 con 0x7f905c003a00 2024-05-30T17:06:26.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.246+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9068005af0 con 0x7f905c003a00 2024-05-30T17:06:26.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.246+0000 7f9076a00640 1 -- 172.21.15.67:0/982186773 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068007070 con 0x7f905c003a00 2024-05-30T17:06:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.246+0000 7f907dc00640 1 -- 172.21.15.67:0/982186773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 msgr2=0x7f905c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.246+0000 7f907dc00640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f905c003e20 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f906800c0e0 tx=0x7f905c005830 comp rx=0 tx=0).stop 2024-05-30T17:06:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.246+0000 7f907dc00640 1 -- 172.21.15.67:0/982186773 shutdown_connections 2024-05-30T17:06:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.246+0000 7f907dc00640 1 --2- 172.21.15.67:0/982186773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f905c003e20 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.246+0000 7f907dc00640 1 -- 172.21.15.67:0/982186773 >> 172.21.15.67:0/982186773 conn(0x7f907809d910 msgr2=0x7f907808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.246+0000 7f907dc00640 1 -- 172.21.15.67:0/982186773 shutdown_connections 2024-05-30T17:06:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.246+0000 7f907dc00640 1 -- 172.21.15.67:0/982186773 wait complete. 2024-05-30T17:06:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.248+0000 7f907dc00640 1 Processor -- start 2024-05-30T17:06:26.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.248+0000 7f907dc00640 1 -- start start 2024-05-30T17:06:26.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.249+0000 7f907dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f90780b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:26.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.249+0000 7f907dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90780b58c0 0x7f90781f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:26.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.249+0000 7f907dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90780b7660 0x7f90780b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:26.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.249+0000 7f907dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f907815ebf0 con 0x7f905c003a00 2024-05-30T17:06:26.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.249+0000 7f907dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f907815ea70 con 0x7f90780b58c0 2024-05-30T17:06:26.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.249+0000 7f907dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f907815ed70 con 0x7f90780b7660 2024-05-30T17:06:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.249+0000 7f9077e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f90780b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.249+0000 7f9077e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f90780b98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33074/0 (socket says 172.21.15.67:33074) 2024-05-30T17:06:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.249+0000 7f9077e00640 1 -- 172.21.15.67:0/3820647663 learned_addr learned my addr 172.21.15.67:0/3820647663 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f9077400640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90780b58c0 0x7f90781f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f907d200640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90780b7660 0x7f90780b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:26.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f9077e00640 1 -- 172.21.15.67:0/3820647663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90780b7660 msgr2=0x7f90780b7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f9077e00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90780b7660 0x7f90780b7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f9077e00640 1 -- 172.21.15.67:0/3820647663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90780b58c0 msgr2=0x7f90781f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f9077e00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90780b58c0 0x7f90781f46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f9077e00640 1 -- 172.21.15.67:0/3820647663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90780b6900 con 0x7f905c003a00 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f907d200640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90780b7660 0x7f90780b7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f9077e00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f90780b98d0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f906800c200 tx=0x7f90780d3510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.250+0000 7f906fe00640 1 -- 172.21.15.67:0/3820647663 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068041020 con 0x7f905c003a00 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.251+0000 7f906fe00640 1 -- 172.21.15.67:0/3820647663 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9068038040 con 0x7f905c003a00 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.251+0000 7f906fe00640 1 -- 172.21.15.67:0/3820647663 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068001670 con 0x7f905c003a00 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.251+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90781f4c20 con 0x7f905c003a00 2024-05-30T17:06:26.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.251+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90780b6340 con 0x7f905c003a00 2024-05-30T17:06:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.252+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90780689e0 con 0x7f905c003a00 2024-05-30T17:06:26.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.253+0000 7f906fe00640 1 -- 172.21.15.67:0/3820647663 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 49) ==== 93998+0+0 (secure 0 0 0) 0x7f9068044020 con 0x7f905c003a00 2024-05-30T17:06:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.254+0000 7f906fe00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f905406e9c0 0x7f9054070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.254+0000 7f906fe00640 1 -- 172.21.15.67:0/3820647663 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f90680aaf00 con 0x7f905c003a00 2024-05-30T17:06:26.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.254+0000 7f9077400640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f905406e9c0 0x7f9054070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:26.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.255+0000 7f9077400640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f905406e9c0 0x7f9054070e80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f90780ce380 tx=0x7f9060008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:26.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.258+0000 7f906fe00640 1 -- 172.21.15.67:0/3820647663 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f906807af50 con 0x7f905c003a00 2024-05-30T17:06:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.529+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f907806cbc0 con 0x7f905c003a00 2024-05-30T17:06:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.530+0000 7f906fe00640 1 -- 172.21.15.67:0/3820647663 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2424 (secure 0 0 0) 0x7f906807a850 con 0x7f905c003a00 2024-05-30T17:06:26.531 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:26.531 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:26.532 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1379, 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:26.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:26.534 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:26.535 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:26.536 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4342, 2024-05-30T17:06:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 18058352739, 2024-05-30T17:06:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 36435222528, 2024-05-30T17:06:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 736658890752, 2024-05-30T17:06:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 638743057, 2024-05-30T17:06:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 644716085, 2024-05-30T17:06:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 154, 2024-05-30T17:06:26.537 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 168 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:26.538 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:26.539 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:26.540 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:26.541 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:26.541 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f905406e9c0 msgr2=0x7f9054070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f905406e9c0 0x7f9054070e80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f90780ce380 tx=0x7f9060008040 comp rx=0 tx=0).stop 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 msgr2=0x7f90780b98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f90780b98d0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f906800c200 tx=0x7f90780d3510 comp rx=0 tx=0).stop 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 shutdown_connections 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f905406e9c0 0x7f9054070e80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90780b7660 0x7f90780b7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90780b58c0 0x7f90781f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 --2- 172.21.15.67:0/3820647663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f905c003a00 0x7f90780b98d0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 >> 172.21.15.67:0/3820647663 conn(0x7f907809d910 msgr2=0x7f90780b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 shutdown_connections 2024-05-30T17:06:26.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:26.533+0000 7f907dc00640 1 -- 172.21.15.67:0/3820647663 wait complete. 2024-05-30T17:06:28.553 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:28.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.764+0000 7fe6e7e00640 1 Processor -- start 2024-05-30T17:06:28.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.765+0000 7fe6e7e00640 1 -- start start 2024-05-30T17:06:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.766+0000 7fe6e7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6e815c6b0 0x7fe6e80b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.766+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80ccc30 0x7fe6e806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.766+0000 7fe6e7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe6e80d0460 0x7fe6e80d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.766+0000 7fe6e7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e80cd1a0 con 0x7fe6e815c6b0 2024-05-30T17:06:28.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.766+0000 7fe6e7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e80cd480 con 0x7fe6e80d0460 2024-05-30T17:06:28.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.766+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e80cd760 con 0x7fe6e80ccc30 2024-05-30T17:06:28.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6e815c6b0 0x7fe6e80b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:28.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6e815c6b0 0x7fe6e80b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33084/0 (socket says 172.21.15.67:33084) 2024-05-30T17:06:28.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e6a00640 1 -- 172.21.15.67:0/4070118180 learned_addr learned my addr 172.21.15.67:0/4070118180 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e6000640 1 --2- 172.21.15.67:0/4070118180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80ccc30 0x7fe6e806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e7400640 1 --2- 172.21.15.67:0/4070118180 >> v2:172.21.15.169:3300/0 conn(0x7fe6e80d0460 0x7fe6e80d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e6a00640 1 -- 172.21.15.67:0/4070118180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80ccc30 msgr2=0x7fe6e806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/4070118180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80ccc30 0x7fe6e806dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e6a00640 1 -- 172.21.15.67:0/4070118180 >> v2:172.21.15.169:3300/0 conn(0x7fe6e80d0460 msgr2=0x7fe6e80d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/4070118180 >> v2:172.21.15.169:3300/0 conn(0x7fe6e80d0460 0x7fe6e80d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e6a00640 1 -- 172.21.15.67:0/4070118180 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e80cda40 con 0x7fe6e815c6b0 2024-05-30T17:06:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.767+0000 7fe6e7400640 1 --2- 172.21.15.67:0/4070118180 >> v2:172.21.15.169:3300/0 conn(0x7fe6e80d0460 0x7fe6e80d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.768+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/4070118180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e815c6b0 0x7fe6e80b3db0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80ca7a0 tx=0x7fe6d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c702030c0e276217 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.769+0000 7fe6e5600640 1 -- 172.21.15.67:0/4070118180 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8037070 con 0x7fe6e815c6b0 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.769+0000 7fe6e5600640 1 -- 172.21.15.67:0/4070118180 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d8032440 con 0x7fe6e815c6b0 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.769+0000 7fe6e5600640 1 -- 172.21.15.67:0/4070118180 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8032810 con 0x7fe6e815c6b0 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.769+0000 7fe6e7e00640 1 -- 172.21.15.67:0/4070118180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e815c6b0 msgr2=0x7fe6e80b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.769+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/4070118180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e815c6b0 0x7fe6e80b3db0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80ca7a0 tx=0x7fe6d802a1e0 comp rx=0 tx=0).stop 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.770+0000 7fe6e7e00640 1 -- 172.21.15.67:0/4070118180 shutdown_connections 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.770+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/4070118180 >> v2:172.21.15.169:3300/0 conn(0x7fe6e80d0460 0x7fe6e80d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.770+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/4070118180 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80ccc30 0x7fe6e806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.770+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/4070118180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e815c6b0 0x7fe6e80b3db0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.770+0000 7fe6e7e00640 1 -- 172.21.15.67:0/4070118180 >> 172.21.15.67:0/4070118180 conn(0x7fe6e806bae0 msgr2=0x7fe6e806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.770+0000 7fe6e7e00640 1 -- 172.21.15.67:0/4070118180 shutdown_connections 2024-05-30T17:06:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.770+0000 7fe6e7e00640 1 -- 172.21.15.67:0/4070118180 wait complete. 2024-05-30T17:06:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.771+0000 7fe6e7e00640 1 Processor -- start 2024-05-30T17:06:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.772+0000 7fe6e7e00640 1 -- start start 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ccc30 0x7fe6e80c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80d0460 0x7fe6e806f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e815c6b0 0x7fe6e80c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e80b4470 con 0x7fe6e80ccc30 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e80b42f0 con 0x7fe6e815c6b0 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e806e1d0 con 0x7fe6e80d0460 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ccc30 0x7fe6e80c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ccc30 0x7fe6e80c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33100/0 (socket says 172.21.15.67:33100) 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3897268877 learned_addr learned my addr 172.21.15.67:0/3897268877 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3897268877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80d0460 msgr2=0x7fe6e806f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e6000640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80d0460 0x7fe6e806f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80d0460 0x7fe6e806f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3897268877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e815c6b0 msgr2=0x7fe6e80c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e815c6b0 0x7fe6e80c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.773+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3897268877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e806fc00 con 0x7fe6e80ccc30 2024-05-30T17:06:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.774+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ccc30 0x7fe6e80c5ad0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe6d802a220 tx=0x7fe6d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.774+0000 7fe6dea00640 1 -- 172.21.15.67:0/3897268877 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8041020 con 0x7fe6e80ccc30 2024-05-30T17:06:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.774+0000 7fe6dea00640 1 -- 172.21.15.67:0/3897268877 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d80306c0 con 0x7fe6e80ccc30 2024-05-30T17:06:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.774+0000 7fe6dea00640 1 -- 172.21.15.67:0/3897268877 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8037040 con 0x7fe6e80ccc30 2024-05-30T17:06:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.774+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e80c60d0 con 0x7fe6e80ccc30 2024-05-30T17:06:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.774+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e80c6690 con 0x7fe6e80ccc30 2024-05-30T17:06:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.775+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e80689e0 con 0x7fe6e80ccc30 2024-05-30T17:06:28.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.777+0000 7fe6dea00640 1 -- 172.21.15.67:0/3897268877 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 49) ==== 93998+0+0 (secure 0 0 0) 0x7fe6d8044020 con 0x7fe6e80ccc30 2024-05-30T17:06:28.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.778+0000 7fe6dea00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe6c806e990 0x7fe6c8070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:28.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.778+0000 7fe6dea00640 1 -- 172.21.15.67:0/3897268877 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fe6d80ab480 con 0x7fe6e80ccc30 2024-05-30T17:06:28.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.778+0000 7fe6e6000640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe6c806e990 0x7fe6c8070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:28.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.779+0000 7fe6e6000640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe6c806e990 0x7fe6c8070e50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4006fd0 tx=0x7fe6d4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:28.783+0000 7fe6dea00640 1 -- 172.21.15.67:0/3897268877 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6d804b030 con 0x7fe6e80ccc30 2024-05-30T17:06:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.071+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe6e80af2c0 con 0x7fe6e80ccc30 2024-05-30T17:06:29.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.072+0000 7fe6dea00640 1 -- 172.21.15.67:0/3897268877 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2424 (secure 0 0 0) 0x7fe6d807b310 con 0x7fe6e80ccc30 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:29.073 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1382, 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:29.074 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:29.075 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 4934, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20529087242, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 41699459072, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 731394654208, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 788104295, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 787470893, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 191, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 208 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:29.076 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:29.078 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:29.079 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:29.080 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:29.080 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.074+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe6c806e990 msgr2=0x7fe6c8070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:29.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.074+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe6c806e990 0x7fe6c8070e50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4006fd0 tx=0x7fe6d4008040 comp rx=0 tx=0).stop 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.074+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ccc30 msgr2=0x7fe6e80c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.074+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ccc30 0x7fe6e80c5ad0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe6d802a220 tx=0x7fe6d80046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.075+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 shutdown_connections 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.075+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe6c806e990 0x7fe6c8070e50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.075+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e815c6b0 0x7fe6e80c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.075+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e80d0460 0x7fe6e806f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.075+0000 7fe6e7e00640 1 --2- 172.21.15.67:0/3897268877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ccc30 0x7fe6e80c5ad0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.075+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 >> 172.21.15.67:0/3897268877 conn(0x7fe6e806bae0 msgr2=0x7fe6e8072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.075+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 shutdown_connections 2024-05-30T17:06:29.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:29.075+0000 7fe6e7e00640 1 -- 172.21.15.67:0/3897268877 wait complete. 2024-05-30T17:06:31.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.313+0000 7f52c1000640 1 Processor -- start 2024-05-30T17:06:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.314+0000 7f52c1000640 1 -- start start 2024-05-30T17:06:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.315+0000 7f52c1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52bc0d5150 0x7f52bc0d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.315+0000 7f52c1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.315+0000 7f52c1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52bc0d2200 con 0x7f52bc15fc70 2024-05-30T17:06:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.315+0000 7f52c1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f52bc160050 con 0x7f52bc0d5150 2024-05-30T17:06:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.315+0000 7f52c1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f52bc0d2ac0 con 0x7f52bc0d3490 2024-05-30T17:06:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.316+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.316+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc0d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:37816/0 (socket says 172.21.15.67:37816) 2024-05-30T17:06:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.316+0000 7f52bbe00640 1 -- 172.21.15.67:0/3668903224 learned_addr learned my addr 172.21.15.67:0/3668903224 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.316+0000 7f52bb400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f52bc15fc70 0x7f52bc0cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:53440/0 (socket says 172.21.15.67:53440) 2024-05-30T17:06:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.316+0000 7f52baa00640 1 --2- 172.21.15.67:0/3668903224 >> v2:172.21.15.169:3300/0 conn(0x7f52bc0d5150 0x7f52bc0d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.317+0000 7f52bbe00640 1 -- 172.21.15.67:0/3668903224 >> v2:172.21.15.169:3300/0 conn(0x7f52bc0d5150 msgr2=0x7f52bc0d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.317+0000 7f52bbe00640 1 --2- 172.21.15.67:0/3668903224 >> v2:172.21.15.169:3300/0 conn(0x7f52bc0d5150 0x7f52bc0d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.317+0000 7f52bbe00640 1 -- 172.21.15.67:0/3668903224 >> v1:172.21.15.169:6789/0 conn(0x7f52bc15fc70 legacy=0x7f52bc0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.317+0000 7f52bbe00640 1 -- 172.21.15.67:0/3668903224 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc0d2470 con 0x7f52bc0d3490 2024-05-30T17:06:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.318+0000 7f52bbe00640 1 --2- 172.21.15.67:0/3668903224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc0d0f10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e50 tx=0x7f52b402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517d2e429cdcc0e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.319+0000 7f52ba000640 1 -- 172.21.15.67:0/3668903224 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4036070 con 0x7f52bc0d3490 2024-05-30T17:06:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.319+0000 7f52ba000640 1 -- 172.21.15.67:0/3668903224 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b4031600 con 0x7f52bc0d3490 2024-05-30T17:06:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.319+0000 7f52ba000640 1 -- 172.21.15.67:0/3668903224 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b40319d0 con 0x7f52bc0d3490 2024-05-30T17:06:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.320+0000 7f52c1000640 1 -- 172.21.15.67:0/3668903224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 msgr2=0x7f52bc0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.320+0000 7f52c1000640 1 --2- 172.21.15.67:0/3668903224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc0d0f10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e50 tx=0x7f52b402a710 comp rx=0 tx=0).stop 2024-05-30T17:06:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.320+0000 7f52c1000640 1 -- 172.21.15.67:0/3668903224 shutdown_connections 2024-05-30T17:06:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.320+0000 7f52c1000640 1 --2- 172.21.15.67:0/3668903224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc0d0f10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.320+0000 7f52c1000640 1 --2- 172.21.15.67:0/3668903224 >> v2:172.21.15.169:3300/0 conn(0x7f52bc0d5150 0x7f52bc0d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.320+0000 7f52c1000640 1 -- 172.21.15.67:0/3668903224 >> 172.21.15.67:0/3668903224 conn(0x7f52bc08bfc0 msgr2=0x7f52bc0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.320+0000 7f52c1000640 1 -- 172.21.15.67:0/3668903224 shutdown_connections 2024-05-30T17:06:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.320+0000 7f52c1000640 1 -- 172.21.15.67:0/3668903224 wait complete. 2024-05-30T17:06:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.322+0000 7f52c1000640 1 Processor -- start 2024-05-30T17:06:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.323+0000 7f52c1000640 1 -- start start 2024-05-30T17:06:31.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.324+0000 7f52c1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:31.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.324+0000 7f52c1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0d5150 0x7f52bc06cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:31.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.324+0000 7f52c1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc15fc70 0x7f52bc06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:31.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.324+0000 7f52c1000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc0d1450 con 0x7f52bc15fc70 2024-05-30T17:06:31.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.324+0000 7f52c1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc0d0570 con 0x7f52bc0d5150 2024-05-30T17:06:31.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.324+0000 7f52c1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f52bc0d15d0 con 0x7f52bc0d3490 2024-05-30T17:06:31.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.324+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc15fc70 0x7f52bc06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:31.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.324+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc15fc70 0x7f52bc06d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37964/0 (socket says 172.21.15.67:37964) 2024-05-30T17:06:31.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.324+0000 7f52bbe00640 1 -- 172.21.15.67:0/3132760480 learned_addr learned my addr 172.21.15.67:0/3132760480 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:31.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.325+0000 7f52baa00640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0d5150 0x7f52bc06cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:31.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.325+0000 7f52bb400640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:31.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.325+0000 7f52bbe00640 1 -- 172.21.15.67:0/3132760480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 msgr2=0x7f52bc151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:31.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.325+0000 7f52bbe00640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.325+0000 7f52bbe00640 1 -- 172.21.15.67:0/3132760480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0d5150 msgr2=0x7f52bc06cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:31.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.325+0000 7f52bbe00640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0d5150 0x7f52bc06cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.325+0000 7f52bbe00640 1 -- 172.21.15.67:0/3132760480 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc1f8b50 con 0x7f52bc15fc70 2024-05-30T17:06:31.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.325+0000 7f52bbe00640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc15fc70 0x7f52bc06d160 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e50 tx=0x7f52b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:31.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.325+0000 7f52b3400640 1 -- 172.21.15.67:0/3132760480 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4036050 con 0x7f52bc15fc70 2024-05-30T17:06:31.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.326+0000 7f52b3400640 1 -- 172.21.15.67:0/3132760480 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b4042070 con 0x7f52bc15fc70 2024-05-30T17:06:31.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.326+0000 7f52b3400640 1 -- 172.21.15.67:0/3132760480 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4009510 con 0x7f52bc15fc70 2024-05-30T17:06:31.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.326+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc0a0070 con 0x7f52bc15fc70 2024-05-30T17:06:31.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.326+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc0a0630 con 0x7f52bc15fc70 2024-05-30T17:06:31.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.327+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52bc0689e0 con 0x7f52bc15fc70 2024-05-30T17:06:31.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.328+0000 7f52b3400640 1 -- 172.21.15.67:0/3132760480 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 49) ==== 93998+0+0 (secure 0 0 0) 0x7f52b4007bf0 con 0x7f52bc15fc70 2024-05-30T17:06:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.329+0000 7f52b3400640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f529c06e990 0x7f529c070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.329+0000 7f52b3400640 1 -- 172.21.15.67:0/3132760480 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f52b40abab0 con 0x7f52bc15fc70 2024-05-30T17:06:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.329+0000 7f52bb400640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f529c06e990 0x7f529c070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.330+0000 7f52bb400640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f529c06e990 0x7f529c070e50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000fb0 tx=0x7f52ac007cd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:31.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.333+0000 7f52b3400640 1 -- 172.21.15.67:0/3132760480 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52b407b940 con 0x7f52bc15fc70 2024-05-30T17:06:31.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.616+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52bc0ba510 con 0x7f52bc15fc70 2024-05-30T17:06:31.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.617+0000 7f52b3400640 1 -- 172.21.15.67:0/3132760480 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2424 (secure 0 0 0) 0x7f52b407b240 con 0x7f52bc15fc70 2024-05-30T17:06:31.618 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:31.618 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:31.618 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:31.618 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:31.618 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:31.618 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1385, 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:31.619 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:31.620 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5018, 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20889876890, 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 42790281216, 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730303832064, 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 582065714, 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 580725647, 2024-05-30T17:06:31.621 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 142, 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 158 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:31.622 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:31.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:31.624 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:31.625 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:31.625 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:31.625 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:31.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f529c06e990 msgr2=0x7f529c070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f529c06e990 0x7f529c070e50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000fb0 tx=0x7f52ac007cd0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc15fc70 msgr2=0x7f52bc06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc15fc70 0x7f52bc06d160 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e50 tx=0x7f52b4004a40 comp rx=0 tx=0).stop 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 shutdown_connections 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f529c06e990 0x7f529c070e50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc15fc70 0x7f52bc06d160 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0d5150 0x7f52bc06cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 --2- 172.21.15.67:0/3132760480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc0d3490 0x7f52bc151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.620+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 >> 172.21.15.67:0/3132760480 conn(0x7f52bc08bfc0 msgr2=0x7f52bc161420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.621+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 shutdown_connections 2024-05-30T17:06:31.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:31.621+0000 7f52c1000640 1 -- 172.21.15.67:0/3132760480 wait complete. 2024-05-30T17:06:33.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:33.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.869+0000 7f4c0b200640 1 Processor -- start 2024-05-30T17:06:33.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.869+0000 7f4c0b200640 1 -- start start 2024-05-30T17:06:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.870+0000 7f4c0b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c0c0d0460 0x7f4c0c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.871+0000 7f4c0b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0c0cd1a0 con 0x7f4c0c0ccc30 2024-05-30T17:06:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.871+0000 7f4c0b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0c0cd480 con 0x7f4c0c15c6b0 2024-05-30T17:06:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.871+0000 7f4c0b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4c0c06e3a0 con 0x7f4c0c0d0460 2024-05-30T17:06:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.871+0000 7f4c01400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4c0c0ccc30 0x7f4c0c06dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36120/0 (socket says 172.21.15.67:36120) 2024-05-30T17:06:33.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.871+0000 7f4c01400640 1 -- 172.21.15.67:0/1004866792 learned_addr learned my addr 172.21.15.67:0/1004866792 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.871+0000 7f4c0a800640 1 --2- 172.21.15.67:0/1004866792 >> v2:172.21.15.169:3300/0 conn(0x7f4c0c0d0460 0x7f4c0c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.871+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3640113683 0 0) 0x7f4c0c0cd1a0 con 0x7f4c0c0ccc30 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.871+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bf0003520 con 0x7f4c0c0ccc30 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2808514556 0 0) 0x7f4bf0003520 con 0x7f4c0c0ccc30 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 >> v2:172.21.15.169:3300/0 conn(0x7f4c0c0d0460 msgr2=0x7f4c0c0d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 --2- 172.21.15.67:0/1004866792 >> v2:172.21.15.169:3300/0 conn(0x7f4c0c0d0460 0x7f4c0c0d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 >> v1:172.21.15.169:6789/0 conn(0x7f4c0c15c6b0 legacy=0x7f4c0c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c0cd760 con 0x7f4c0c0ccc30 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f4bf4001670 con 0x7f4c0c0ccc30 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 >> v1:172.21.15.67:6789/0 conn(0x7f4c0c0ccc30 legacy=0x7f4c0c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0003a00 0x7f4bf0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bf0005210 0x7f4bf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c06e3a0 con 0x7f4bf0005210 2024-05-30T17:06:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf000c160 con 0x7f4bf0003a00 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf000c360 con 0x7f4bf0004800 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.872+0000 7f4c0a800640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bf0005210 0x7f4bf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.873+0000 7f4c01400640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.873+0000 7f4c09e00640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0003a00 0x7f4bf0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.873+0000 7f4c0a800640 1 -- 172.21.15.67:0/1004866792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bf0004800 msgr2=0x7f4bf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.873+0000 7f4c0a800640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.873+0000 7f4c0a800640 1 -- 172.21.15.67:0/1004866792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0003a00 msgr2=0x7f4bf0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.873+0000 7f4c0a800640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0003a00 0x7f4bf0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.873+0000 7f4c0a800640 1 -- 172.21.15.67:0/1004866792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf0003520 con 0x7f4bf0005210 2024-05-30T17:06:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.873+0000 7f4c09e00640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0003a00 0x7f4bf0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.874+0000 7f4c0a800640 1 -- 172.21.15.67:0/1004866792 reap_dead start 2024-05-30T17:06:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.874+0000 7f4c0a800640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bf0005210 0x7f4bf000b9c0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f4c04007360 tx=0x7f4bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cfcf8b8d471f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.874+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04009be0 con 0x7f4bf0005210 2024-05-30T17:06:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.874+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c04003bd0 con 0x7f4bf0005210 2024-05-30T17:06:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.874+0000 7f4c09400640 1 -- 172.21.15.67:0/1004866792 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04004020 con 0x7f4bf0005210 2024-05-30T17:06:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.875+0000 7f4c0b200640 1 -- 172.21.15.67:0/1004866792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bf0005210 msgr2=0x7f4bf000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.875+0000 7f4c0b200640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bf0005210 0x7f4bf000b9c0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f4c04007360 tx=0x7f4bf0005830 comp rx=0 tx=0).stop 2024-05-30T17:06:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.875+0000 7f4c0b200640 1 -- 172.21.15.67:0/1004866792 shutdown_connections 2024-05-30T17:06:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.875+0000 7f4c0b200640 1 --2- 172.21.15.67:0/1004866792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bf0005210 0x7f4bf000b9c0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.875+0000 7f4c0b200640 1 -- 172.21.15.67:0/1004866792 >> 172.21.15.67:0/1004866792 conn(0x7f4c0c06bae0 msgr2=0x7f4c0c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.875+0000 7f4c0b200640 1 -- 172.21.15.67:0/1004866792 shutdown_connections 2024-05-30T17:06:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.875+0000 7f4c0b200640 1 -- 172.21.15.67:0/1004866792 wait complete. 2024-05-30T17:06:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.877+0000 7f4c0b200640 1 Processor -- start 2024-05-30T17:06:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.877+0000 7f4c0b200640 1 -- start start 2024-05-30T17:06:33.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.878+0000 7f4c0b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0005210 0x7f4c0c0b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:33.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.878+0000 7f4c0b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c0c0c5cd0 0x7f4c0c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:33.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.878+0000 7f4c0b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c0c06f2a0 0x7f4c0c06f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.878+0000 7f4c0b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c0d08b0 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.878+0000 7f4c0b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c0cba10 con 0x7f4bf0005210 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.878+0000 7f4c0b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0c0d0a30 con 0x7f4c0c06f2a0 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.878+0000 7f4c01400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c0c0c5cd0 0x7f4c0c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.878+0000 7f4c01400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c0c0c5cd0 0x7f4c0c155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37982/0 (socket says 172.21.15.67:37982) 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.878+0000 7f4c01400640 1 -- 172.21.15.67:0/2313339578 learned_addr learned my addr 172.21.15.67:0/2313339578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c0a800640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c0c06f2a0 0x7f4c0c06f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c09e00640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0005210 0x7f4c0c0b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c01400640 1 -- 172.21.15.67:0/2313339578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c0c06f2a0 msgr2=0x7f4c0c06f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c01400640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c0c06f2a0 0x7f4c0c06f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c01400640 1 -- 172.21.15.67:0/2313339578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0005210 msgr2=0x7f4c0c0b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c01400640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0005210 0x7f4c0c0b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c01400640 1 -- 172.21.15.67:0/2313339578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c158640 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c09e00640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0005210 0x7f4c0c0b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c01400640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c0c0c5cd0 0x7f4c0c155810 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4001200 tx=0x7f4bf400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c02a00640 1 -- 172.21.15.67:0/2313339578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf400c660 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c02a00640 1 -- 172.21.15.67:0/2313339578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf4018070 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.879+0000 7f4c02a00640 1 -- 172.21.15.67:0/2313339578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf400ca40 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.880+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c1572c0 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.880+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c157880 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.881+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0c0689e0 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.882+0000 7f4c02a00640 1 -- 172.21.15.67:0/2313339578 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7f4bf400cbe0 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.883+0000 7f4c02a00640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be806e930 0x7f4be8070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:33.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.883+0000 7f4c02a00640 1 -- 172.21.15.67:0/2313339578 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f4bf408a8b0 con 0x7f4c0c0c5cd0 2024-05-30T17:06:33.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.883+0000 7f4c09e00640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be806e930 0x7f4be8070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:33.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.884+0000 7f4c09e00640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be806e930 0x7f4be8070df0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc001690 tx=0x7f4bfc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:33.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:33.888+0000 7f4c02a00640 1 -- 172.21.15.67:0/2313339578 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bf405a7e0 con 0x7f4c0c0c5cd0 2024-05-30T17:06:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.170+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4c0c06dee0 con 0x7f4c0c0c5cd0 2024-05-30T17:06:34.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.171+0000 7f4c02a00640 1 -- 172.21.15.67:0/2313339578 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2424 (secure 0 0 0) 0x7f4bf405a0e0 con 0x7f4c0c0c5cd0 2024-05-30T17:06:34.172 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:34.172 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:34.172 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:34.172 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:34.173 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1387, 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:34.174 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915042971, 2024-05-30T17:06:34.175 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016757248, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077356032, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 429169985, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 425013945, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 105, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 115 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:34.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:34.177 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:34.178 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:34.179 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:34.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:34.179 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:34.179 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:34.179 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:34.179 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:34.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:34.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:34.179 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.174+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be806e930 msgr2=0x7f4be8070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.174+0000 7f4c0b200640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be806e930 0x7f4be8070df0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc001690 tx=0x7f4bfc009040 comp rx=0 tx=0).stop 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.174+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c0c0c5cd0 msgr2=0x7f4c0c155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.174+0000 7f4c0b200640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c0c0c5cd0 0x7f4c0c155810 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4001200 tx=0x7f4bf400ec10 comp rx=0 tx=0).stop 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.175+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 shutdown_connections 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.175+0000 7f4c0b200640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f4be806e930 0x7f4be8070df0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.175+0000 7f4c0b200640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c0c06f2a0 0x7f4c0c06f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.175+0000 7f4c0b200640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c0c0c5cd0 0x7f4c0c155810 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.175+0000 7f4c0b200640 1 --2- 172.21.15.67:0/2313339578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bf0005210 0x7f4c0c0b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.175+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 >> 172.21.15.67:0/2313339578 conn(0x7f4c0c06bae0 msgr2=0x7f4c0c06e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.175+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 shutdown_connections 2024-05-30T17:06:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:34.175+0000 7f4c0b200640 1 -- 172.21.15.67:0/2313339578 wait complete. 2024-05-30T17:06:36.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:36.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.417+0000 7f0526000640 1 Processor -- start 2024-05-30T17:06:36.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.417+0000 7f0526000640 1 -- start start 2024-05-30T17:06:36.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.418+0000 7f0526000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0520161df0 0x7f0520099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:36.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.418+0000 7f0526000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f05200d0b70 0x7f05200cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:36.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.418+0000 7f0526000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05201621d0 con 0x7f05200cc2b0 2024-05-30T17:06:36.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.418+0000 7f0526000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f05200cd840 con 0x7f05200d0b70 2024-05-30T17:06:36.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.418+0000 7f0526000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f05200ccf10 con 0x7f0520161df0 2024-05-30T17:06:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.419+0000 7f051fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f05200cc2b0 0x7f05200d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36132/0 (socket says 172.21.15.67:36132) 2024-05-30T17:06:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.419+0000 7f051fe00640 1 -- 172.21.15.67:0/2594888357 learned_addr learned my addr 172.21.15.67:0/2594888357 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.419+0000 7f0525600640 1 --2- 172.21.15.67:0/2594888357 >> v2:172.21.15.67:3300/0 conn(0x7f05200d0b70 0x7f05200cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.419+0000 7f0524c00640 1 --2- 172.21.15.67:0/2594888357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0520161df0 0x7f0520099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.419+0000 7f0525600640 1 -- 172.21.15.67:0/2594888357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0520161df0 msgr2=0x7f0520099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.419+0000 7f0525600640 1 --2- 172.21.15.67:0/2594888357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0520161df0 0x7f0520099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.419+0000 7f0525600640 1 -- 172.21.15.67:0/2594888357 >> v1:172.21.15.67:6789/0 conn(0x7f05200cc2b0 legacy=0x7f05200d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:36.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.419+0000 7f0525600640 1 -- 172.21.15.67:0/2594888357 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05200cd1f0 con 0x7f05200d0b70 2024-05-30T17:06:36.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.421+0000 7f0525600640 1 --2- 172.21.15.67:0/2594888357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05200d0b70 0x7f05200cefa0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f0518009d90 tx=0x7f051802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37978cec1eec3c8a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:36.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.421+0000 7f051f400640 1 -- 172.21.15.67:0/2594888357 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518007740 con 0x7f05200d0b70 2024-05-30T17:06:36.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.421+0000 7f051f400640 1 -- 172.21.15.67:0/2594888357 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05180078e0 con 0x7f05200d0b70 2024-05-30T17:06:36.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.421+0000 7f051f400640 1 -- 172.21.15.67:0/2594888357 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518007cb0 con 0x7f05200d0b70 2024-05-30T17:06:36.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.421+0000 7f0526000640 1 -- 172.21.15.67:0/2594888357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05200d0b70 msgr2=0x7f05200cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:36.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.421+0000 7f0526000640 1 --2- 172.21.15.67:0/2594888357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05200d0b70 0x7f05200cefa0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f0518009d90 tx=0x7f051802a660 comp rx=0 tx=0).stop 2024-05-30T17:06:36.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.422+0000 7f0526000640 1 -- 172.21.15.67:0/2594888357 shutdown_connections 2024-05-30T17:06:36.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.422+0000 7f0526000640 1 --2- 172.21.15.67:0/2594888357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05200d0b70 0x7f05200cefa0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:36.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.422+0000 7f0526000640 1 --2- 172.21.15.67:0/2594888357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0520161df0 0x7f0520099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:36.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.422+0000 7f0526000640 1 -- 172.21.15.67:0/2594888357 >> 172.21.15.67:0/2594888357 conn(0x7f052006bae0 msgr2=0x7f052006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:36.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.422+0000 7f0526000640 1 -- 172.21.15.67:0/2594888357 shutdown_connections 2024-05-30T17:06:36.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.422+0000 7f0526000640 1 -- 172.21.15.67:0/2594888357 wait complete. 2024-05-30T17:06:36.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.423+0000 7f0526000640 1 Processor -- start 2024-05-30T17:06:36.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.424+0000 7f0526000640 1 -- start start 2024-05-30T17:06:36.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0526000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05200cc2b0 0x7f052014e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0526000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05200d0b70 0x7f05200c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0526000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0520161df0 0x7f05200c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0526000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05200ccbc0 con 0x7f0520161df0 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0526000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0520099d40 con 0x7f05200cc2b0 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0526000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f05200ca870 con 0x7f05200d0b70 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0525600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0520161df0 0x7f05200c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0525600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0520161df0 0x7f05200c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38006/0 (socket says 172.21.15.67:38006) 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0525600640 1 -- 172.21.15.67:0/2621699091 learned_addr learned my addr 172.21.15.67:0/2621699091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f051fe00640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05200d0b70 0x7f05200c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0524c00640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05200cc2b0 0x7f052014e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0525600640 1 -- 172.21.15.67:0/2621699091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05200d0b70 msgr2=0x7f05200c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0525600640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05200d0b70 0x7f05200c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0525600640 1 -- 172.21.15.67:0/2621699091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05200cc2b0 msgr2=0x7f052014e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0525600640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05200cc2b0 0x7f052014e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0525600640 1 -- 172.21.15.67:0/2621699091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05200c95a0 con 0x7f0520161df0 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.425+0000 7f0524c00640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05200cc2b0 0x7f052014e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.426+0000 7f0525600640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0520161df0 0x7f05200c6b10 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f051802ac50 tx=0x7f0518004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.426+0000 7f051cc00640 1 -- 172.21.15.67:0/2621699091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518040070 con 0x7f0520161df0 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.426+0000 7f051cc00640 1 -- 172.21.15.67:0/2621699091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0518031bc0 con 0x7f0520161df0 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.426+0000 7f051cc00640 1 -- 172.21.15.67:0/2621699091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518045460 con 0x7f0520161df0 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.426+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05200c9880 con 0x7f0520161df0 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.426+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05200c9e40 con 0x7f0520161df0 2024-05-30T17:06:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.427+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05200ca040 con 0x7f0520161df0 2024-05-30T17:06:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.429+0000 7f051cc00640 1 -- 172.21.15.67:0/2621699091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7f0518039040 con 0x7f0520161df0 2024-05-30T17:06:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.429+0000 7f051cc00640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f050006e930 0x7f0500070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.430+0000 7f051cc00640 1 -- 172.21.15.67:0/2621699091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f05180ac650 con 0x7f0520161df0 2024-05-30T17:06:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.430+0000 7f0524c00640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f050006e930 0x7f0500070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:36.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.431+0000 7f0524c00640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f050006e930 0x7f0500070df0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0510006fd0 tx=0x7f0510008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.434+0000 7f051cc00640 1 -- 172.21.15.67:0/2621699091 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05200ca040 con 0x7f0520161df0 2024-05-30T17:06:36.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.715+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f05200689e0 con 0x7f0520161df0 2024-05-30T17:06:36.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.716+0000 7f051cc00640 1 -- 172.21.15.67:0/2621699091 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2422 (secure 0 0 0) 0x7f051807c600 con 0x7f0520161df0 2024-05-30T17:06:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1390, 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915042971, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016802304, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077310976, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 304849968, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 304535555, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 74, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 86 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:36.723 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.718+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f050006e930 msgr2=0x7f0500070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.718+0000 7f0526000640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f050006e930 0x7f0500070df0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0510006fd0 tx=0x7f0510008040 comp rx=0 tx=0).stop 2024-05-30T17:06:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.718+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0520161df0 msgr2=0x7f05200c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.718+0000 7f0526000640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0520161df0 0x7f05200c6b10 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f051802ac50 tx=0x7f0518004a40 comp rx=0 tx=0).stop 2024-05-30T17:06:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.719+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 shutdown_connections 2024-05-30T17:06:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.719+0000 7f0526000640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f050006e930 0x7f0500070df0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.719+0000 7f0526000640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0520161df0 0x7f05200c6b10 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.719+0000 7f0526000640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05200d0b70 0x7f05200c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.719+0000 7f0526000640 1 --2- 172.21.15.67:0/2621699091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05200cc2b0 0x7f052014e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.719+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 >> 172.21.15.67:0/2621699091 conn(0x7f052006bae0 msgr2=0x7f05200ad9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.719+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 shutdown_connections 2024-05-30T17:06:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:36.719+0000 7f0526000640 1 -- 172.21.15.67:0/2621699091 wait complete. 2024-05-30T17:06:38.741 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.960+0000 7fd477200640 1 Processor -- start 2024-05-30T17:06:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.960+0000 7fd477200640 1 -- start start 2024-05-30T17:06:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.962+0000 7fd477200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd47815fba0 0x7fd47809b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.962+0000 7fd477200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4780d0960 0x7fd4780cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.962+0000 7fd477200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4780d0f20 con 0x7fd4780ce050 2024-05-30T17:06:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.962+0000 7fd477200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fd4780cd030 con 0x7fd47815fba0 2024-05-30T17:06:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.962+0000 7fd477200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4780cd310 con 0x7fd4780d0960 2024-05-30T17:06:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.962+0000 7fd475e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd47815fba0 0x7fd47809b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.962+0000 7fd475e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd47815fba0 0x7fd47809b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38008/0 (socket says 172.21.15.67:38008) 2024-05-30T17:06:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.962+0000 7fd475e00640 1 -- 172.21.15.67:0/3169251480 learned_addr learned my addr 172.21.15.67:0/3169251480 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.963+0000 7fd46d400640 1 --2- 172.21.15.67:0/3169251480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4780d0960 0x7fd4780cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.963+0000 7fd475e00640 1 -- 172.21.15.67:0/3169251480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4780d0960 msgr2=0x7fd4780cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.963+0000 7fd475e00640 1 --2- 172.21.15.67:0/3169251480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4780d0960 0x7fd4780cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.963+0000 7fd475e00640 1 -- 172.21.15.67:0/3169251480 >> v1:172.21.15.67:6789/0 conn(0x7fd4780ce050 legacy=0x7fd47809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.963+0000 7fd475e00640 1 -- 172.21.15.67:0/3169251480 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4780cd5f0 con 0x7fd47815fba0 2024-05-30T17:06:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.964+0000 7fd475e00640 1 --2- 172.21.15.67:0/3169251480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd47815fba0 0x7fd47809b7b0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fd468009870 tx=0x7fd46802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a560fa9bee48fbec server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.964+0000 7fd475400640 1 -- 172.21.15.67:0/3169251480 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468037070 con 0x7fd47815fba0 2024-05-30T17:06:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.964+0000 7fd475400640 1 -- 172.21.15.67:0/3169251480 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46802ae50 con 0x7fd47815fba0 2024-05-30T17:06:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.964+0000 7fd475400640 1 -- 172.21.15.67:0/3169251480 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468032860 con 0x7fd47815fba0 2024-05-30T17:06:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.965+0000 7fd477200640 1 -- 172.21.15.67:0/3169251480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd47815fba0 msgr2=0x7fd47809b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.965+0000 7fd477200640 1 --2- 172.21.15.67:0/3169251480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd47815fba0 0x7fd47809b7b0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fd468009870 tx=0x7fd46802a300 comp rx=0 tx=0).stop 2024-05-30T17:06:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.965+0000 7fd477200640 1 -- 172.21.15.67:0/3169251480 shutdown_connections 2024-05-30T17:06:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.965+0000 7fd477200640 1 --2- 172.21.15.67:0/3169251480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4780d0960 0x7fd4780cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.965+0000 7fd477200640 1 --2- 172.21.15.67:0/3169251480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd47815fba0 0x7fd47809b7b0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.965+0000 7fd477200640 1 -- 172.21.15.67:0/3169251480 >> 172.21.15.67:0/3169251480 conn(0x7fd4780b3e70 msgr2=0x7fd4780b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.965+0000 7fd477200640 1 -- 172.21.15.67:0/3169251480 shutdown_connections 2024-05-30T17:06:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.965+0000 7fd477200640 1 -- 172.21.15.67:0/3169251480 wait complete. 2024-05-30T17:06:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.967+0000 7fd477200640 1 Processor -- start 2024-05-30T17:06:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.967+0000 7fd477200640 1 -- start start 2024-05-30T17:06:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.968+0000 7fd477200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4780ce050 0x7fd4780b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.968+0000 7fd477200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4780d0960 0x7fd4780b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.968+0000 7fd477200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47815fba0 0x7fd4780c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.968+0000 7fd477200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd47815ff80 con 0x7fd4780ce050 2024-05-30T17:06:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.968+0000 7fd477200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4780d1200 con 0x7fd47815fba0 2024-05-30T17:06:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.968+0000 7fd477200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4780b2230 con 0x7fd4780d0960 2024-05-30T17:06:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.968+0000 7fd475e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4780ce050 0x7fd4780b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.968+0000 7fd475e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4780ce050 0x7fd4780b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38010/0 (socket says 172.21.15.67:38010) 2024-05-30T17:06:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.968+0000 7fd475e00640 1 -- 172.21.15.67:0/2949029031 learned_addr learned my addr 172.21.15.67:0/2949029031 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd475e00640 1 -- 172.21.15.67:0/2949029031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4780d0960 msgr2=0x7fd4780b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd475e00640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4780d0960 0x7fd4780b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd475e00640 1 -- 172.21.15.67:0/2949029031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47815fba0 msgr2=0x7fd4780c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd475e00640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47815fba0 0x7fd4780c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd475e00640 1 -- 172.21.15.67:0/2949029031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4780b1620 con 0x7fd4780ce050 2024-05-30T17:06:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd475e00640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4780ce050 0x7fd4780b26c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fd468000c00 tx=0x7fd4680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd46ea00640 1 -- 172.21.15.67:0/2949029031 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468041070 con 0x7fd4780ce050 2024-05-30T17:06:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd46ea00640 1 -- 172.21.15.67:0/2949029031 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46803c890 con 0x7fd4780ce050 2024-05-30T17:06:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4780b1900 con 0x7fd4780ce050 2024-05-30T17:06:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd46ea00640 1 -- 172.21.15.67:0/2949029031 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468037040 con 0x7fd4780ce050 2024-05-30T17:06:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.969+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4780b1ec0 con 0x7fd4780ce050 2024-05-30T17:06:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.970+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4780689e0 con 0x7fd4780ce050 2024-05-30T17:06:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.972+0000 7fd46ea00640 1 -- 172.21.15.67:0/2949029031 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7fd468044070 con 0x7fd4780ce050 2024-05-30T17:06:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.973+0000 7fd46ea00640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd45406e900 0x7fd454070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.973+0000 7fd46ea00640 1 -- 172.21.15.67:0/2949029031 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fd4680ab520 con 0x7fd4780ce050 2024-05-30T17:06:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.973+0000 7fd46d400640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd45406e900 0x7fd454070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.974+0000 7fd46d400640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd45406e900 0x7fd454070dc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd460006fd0 tx=0x7fd460008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:38.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:38.978+0000 7fd46ea00640 1 -- 172.21.15.67:0/2949029031 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd46807b3b0 con 0x7fd4780ce050 2024-05-30T17:06:39.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.258+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd47815cfc0 con 0x7fd4780ce050 2024-05-30T17:06:39.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.259+0000 7fd46ea00640 1 -- 172.21.15.67:0/2949029031 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2419 (secure 0 0 0) 0x7fd46807acb0 con 0x7fd4780ce050 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:39.260 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1392, 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:39.261 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:39.262 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915085435, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 26203874, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 32154911, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 7, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 12 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:39.263 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:39.264 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:39.265 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:39.266 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:39.267 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:39.267 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:39.267 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd45406e900 msgr2=0x7fd454070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd45406e900 0x7fd454070dc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd460006fd0 tx=0x7fd460008040 comp rx=0 tx=0).stop 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4780ce050 msgr2=0x7fd4780b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4780ce050 0x7fd4780b26c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fd468000c00 tx=0x7fd4680046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 shutdown_connections 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd45406e900 0x7fd454070dc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47815fba0 0x7fd4780c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4780d0960 0x7fd4780b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 --2- 172.21.15.67:0/2949029031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4780ce050 0x7fd4780b26c0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 >> 172.21.15.67:0/2949029031 conn(0x7fd4780b3e70 msgr2=0x7fd47815d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 shutdown_connections 2024-05-30T17:06:39.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:39.262+0000 7fd477200640 1 -- 172.21.15.67:0/2949029031 wait complete. 2024-05-30T17:06:41.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:41.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.497+0000 7fad93200640 1 Processor -- start 2024-05-30T17:06:41.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.498+0000 7fad93200640 1 -- start start 2024-05-30T17:06:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.499+0000 7fad93200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 0x7fad940d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.499+0000 7fad93200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad940d1ca0 0x7fad940d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.499+0000 7fad93200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad940d3060 con 0x7fad9415ea70 2024-05-30T17:06:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.499+0000 7fad93200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fad940d3340 con 0x7fad940d1ca0 2024-05-30T17:06:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.499+0000 7fad93200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad940d3620 con 0x7fad9415e690 2024-05-30T17:06:41.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.500+0000 7fad91e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 0x7fad940d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:41.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.500+0000 7fad91e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 0x7fad940d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:39614/0 (socket says 172.21.15.67:39614) 2024-05-30T17:06:41.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.500+0000 7fad91e00640 1 -- 172.21.15.67:0/2139063719 learned_addr learned my addr 172.21.15.67:0/2139063719 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:41.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.500+0000 7fad91400640 1 --2- 172.21.15.67:0/2139063719 >> v2:172.21.15.169:3300/0 conn(0x7fad940d1ca0 0x7fad940d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:41.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.501+0000 7fad91e00640 1 -- 172.21.15.67:0/2139063719 >> v2:172.21.15.169:3300/0 conn(0x7fad940d1ca0 msgr2=0x7fad940d4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:41.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.501+0000 7fad91e00640 1 --2- 172.21.15.67:0/2139063719 >> v2:172.21.15.169:3300/0 conn(0x7fad940d1ca0 0x7fad940d4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.501+0000 7fad91e00640 1 -- 172.21.15.67:0/2139063719 >> v1:172.21.15.169:6789/0 conn(0x7fad9415ea70 legacy=0x7fad940d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:41.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.501+0000 7fad91e00640 1 -- 172.21.15.67:0/2139063719 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad940cf770 con 0x7fad9415e690 2024-05-30T17:06:41.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.503+0000 7fad91e00640 1 --2- 172.21.15.67:0/2139063719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 0x7fad940d4530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fad80009870 tx=0x7fad8002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742bbbdbf5c8660 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:41.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.503+0000 7fad90a00640 1 -- 172.21.15.67:0/2139063719 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80037070 con 0x7fad9415e690 2024-05-30T17:06:41.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.503+0000 7fad90a00640 1 -- 172.21.15.67:0/2139063719 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad80032440 con 0x7fad9415e690 2024-05-30T17:06:41.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.503+0000 7fad90a00640 1 -- 172.21.15.67:0/2139063719 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80032810 con 0x7fad9415e690 2024-05-30T17:06:41.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.504+0000 7fad93200640 1 -- 172.21.15.67:0/2139063719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 msgr2=0x7fad940d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:41.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.504+0000 7fad93200640 1 --2- 172.21.15.67:0/2139063719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 0x7fad940d4530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fad80009870 tx=0x7fad8002a270 comp rx=0 tx=0).stop 2024-05-30T17:06:41.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.504+0000 7fad93200640 1 -- 172.21.15.67:0/2139063719 shutdown_connections 2024-05-30T17:06:41.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.504+0000 7fad93200640 1 --2- 172.21.15.67:0/2139063719 >> v2:172.21.15.169:3300/0 conn(0x7fad940d1ca0 0x7fad940d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.504+0000 7fad93200640 1 --2- 172.21.15.67:0/2139063719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 0x7fad940d4530 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.504+0000 7fad93200640 1 -- 172.21.15.67:0/2139063719 >> 172.21.15.67:0/2139063719 conn(0x7fad9409d910 msgr2=0x7fad9408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:41.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.504+0000 7fad93200640 1 -- 172.21.15.67:0/2139063719 shutdown_connections 2024-05-30T17:06:41.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.504+0000 7fad93200640 1 -- 172.21.15.67:0/2139063719 wait complete. 2024-05-30T17:06:41.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.506+0000 7fad93200640 1 Processor -- start 2024-05-30T17:06:41.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.506+0000 7fad93200640 1 -- start start 2024-05-30T17:06:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.507+0000 7fad93200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad940d1ca0 0x7fad940cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.507+0000 7fad93200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 0x7fad94160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.507+0000 7fad93200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad9415ea70 0x7fad9415f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.507+0000 7fad93200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fad940d2bc0 con 0x7fad940d1ca0 2024-05-30T17:06:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.507+0000 7fad93200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad940d2a40 con 0x7fad9415ea70 2024-05-30T17:06:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.507+0000 7fad93200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad940d2d40 con 0x7fad9415e690 2024-05-30T17:06:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad91e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad940d1ca0 0x7fad940cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad91e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad940d1ca0 0x7fad940cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44696/0 (socket says 172.21.15.67:44696) 2024-05-30T17:06:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad91e00640 1 -- 172.21.15.67:0/695537862 learned_addr learned my addr 172.21.15.67:0/695537862 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:41.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad92800640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad9415ea70 0x7fad9415f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad91e00640 1 -- 172.21.15.67:0/695537862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 msgr2=0x7fad94160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad91e00640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 0x7fad94160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad91e00640 1 -- 172.21.15.67:0/695537862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad9415ea70 msgr2=0x7fad9415f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad91e00640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad9415ea70 0x7fad9415f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad91e00640 1 -- 172.21.15.67:0/695537862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad940cc910 con 0x7fad940d1ca0 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad92800640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad9415ea70 0x7fad9415f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.508+0000 7fad91e00640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad940d1ca0 0x7fad940cc3d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fad80000c00 tx=0x7fad800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.509+0000 7fad8a000640 1 -- 172.21.15.67:0/695537862 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80037040 con 0x7fad940d1ca0 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.509+0000 7fad8a000640 1 -- 172.21.15.67:0/695537862 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad80042070 con 0x7fad940d1ca0 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.509+0000 7fad8a000640 1 -- 172.21.15.67:0/695537862 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80030880 con 0x7fad940d1ca0 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.509+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad941615c0 con 0x7fad940d1ca0 2024-05-30T17:06:41.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.509+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad940c81d0 con 0x7fad940d1ca0 2024-05-30T17:06:41.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.510+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad940689e0 con 0x7fad940d1ca0 2024-05-30T17:06:41.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.511+0000 7fad8a000640 1 -- 172.21.15.67:0/695537862 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7fad80030a20 con 0x7fad940d1ca0 2024-05-30T17:06:41.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.513+0000 7fad8a000640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad7406e900 0x7fad74070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:41.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.513+0000 7fad8a000640 1 -- 172.21.15.67:0/695537862 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fad800ab4a0 con 0x7fad940d1ca0 2024-05-30T17:06:41.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.513+0000 7fad91400640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad7406e900 0x7fad74070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:41.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.514+0000 7fad91400640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad7406e900 0x7fad74070dc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fad84006a30 tx=0x7fad840069c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:41.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.517+0000 7fad8a000640 1 -- 172.21.15.67:0/695537862 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad8007b330 con 0x7fad940d1ca0 2024-05-30T17:06:41.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.795+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fad9406cbc0 con 0x7fad940d1ca0 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.796+0000 7fad8a000640 1 -- 172.21.15.67:0/695537862 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2419 (secure 0 0 0) 0x7fad8007ac30 con 0x7fad940d1ca0 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:41.797 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1395, 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:41.798 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:41.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 26204516, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 32155869, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 7, 2024-05-30T17:06:41.800 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 12 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:41.801 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:41.802 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:41.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:41.804 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:41.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad7406e900 msgr2=0x7fad74070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad7406e900 0x7fad74070dc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fad84006a30 tx=0x7fad840069c0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad940d1ca0 msgr2=0x7fad940cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad940d1ca0 0x7fad940cc3d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fad80000c00 tx=0x7fad800046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 shutdown_connections 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fad7406e900 0x7fad74070dc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad9415ea70 0x7fad9415f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad9415e690 0x7fad94160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 --2- 172.21.15.67:0/695537862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad940d1ca0 0x7fad940cc3d0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 >> 172.21.15.67:0/695537862 conn(0x7fad9409d910 msgr2=0x7fad940d4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 shutdown_connections 2024-05-30T17:06:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:41.799+0000 7fad93200640 1 -- 172.21.15.67:0/695537862 wait complete. 2024-05-30T17:06:43.819 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:44.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.040+0000 7fc5e6200640 1 Processor -- start 2024-05-30T17:06:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.040+0000 7fc5e6200640 1 -- start start 2024-05-30T17:06:44.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.041+0000 7fc5e6200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5e015ef60 0x7fc5e00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:44.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.041+0000 7fc5e6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e00d0460 0x7fc5e00d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:44.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.041+0000 7fc5e6200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5e015fe50 con 0x7fc5e015b380 2024-05-30T17:06:44.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.041+0000 7fc5e6200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e00b3310 con 0x7fc5e015ef60 2024-05-30T17:06:44.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.041+0000 7fc5e6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e015f5b0 con 0x7fc5e00d0460 2024-05-30T17:06:44.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.042+0000 7fc5e5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e00d0460 0x7fc5e00d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:44.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.042+0000 7fc5e5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e00d0460 0x7fc5e00d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:39632/0 (socket says 172.21.15.67:39632) 2024-05-30T17:06:44.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.042+0000 7fc5e5800640 1 -- 172.21.15.67:0/314100773 learned_addr learned my addr 172.21.15.67:0/314100773 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:44.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.042+0000 7fc5d7e00640 1 --2- 172.21.15.67:0/314100773 >> v2:172.21.15.169:3300/0 conn(0x7fc5e015ef60 0x7fc5e00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:44.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.043+0000 7fc5e5800640 1 -- 172.21.15.67:0/314100773 >> v2:172.21.15.169:3300/0 conn(0x7fc5e015ef60 msgr2=0x7fc5e00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:44.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.043+0000 7fc5e5800640 1 --2- 172.21.15.67:0/314100773 >> v2:172.21.15.169:3300/0 conn(0x7fc5e015ef60 0x7fc5e00ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:44.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.043+0000 7fc5e5800640 1 -- 172.21.15.67:0/314100773 >> v1:172.21.15.169:6789/0 conn(0x7fc5e015b380 legacy=0x7fc5e00ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:44.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.043+0000 7fc5e5800640 1 -- 172.21.15.67:0/314100773 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e015f890 con 0x7fc5e00d0460 2024-05-30T17:06:44.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.044+0000 7fc5e5800640 1 --2- 172.21.15.67:0/314100773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e00d0460 0x7fc5e00d08f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009e50 tx=0x7fc5d802c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87b452f680c18a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:44.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.045+0000 7fc5dfe00640 1 -- 172.21.15.67:0/314100773 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8039070 con 0x7fc5e00d0460 2024-05-30T17:06:44.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.045+0000 7fc5dfe00640 1 -- 172.21.15.67:0/314100773 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d80345f0 con 0x7fc5e00d0460 2024-05-30T17:06:44.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.045+0000 7fc5dfe00640 1 -- 172.21.15.67:0/314100773 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80349c0 con 0x7fc5e00d0460 2024-05-30T17:06:44.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.046+0000 7fc5e6200640 1 -- 172.21.15.67:0/314100773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e00d0460 msgr2=0x7fc5e00d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:44.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.046+0000 7fc5e6200640 1 --2- 172.21.15.67:0/314100773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e00d0460 0x7fc5e00d08f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009e50 tx=0x7fc5d802c800 comp rx=0 tx=0).stop 2024-05-30T17:06:44.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.046+0000 7fc5e6200640 1 -- 172.21.15.67:0/314100773 shutdown_connections 2024-05-30T17:06:44.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.046+0000 7fc5e6200640 1 --2- 172.21.15.67:0/314100773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e00d0460 0x7fc5e00d08f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:44.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.046+0000 7fc5e6200640 1 --2- 172.21.15.67:0/314100773 >> v2:172.21.15.169:3300/0 conn(0x7fc5e015ef60 0x7fc5e00ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:44.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.046+0000 7fc5e6200640 1 -- 172.21.15.67:0/314100773 >> 172.21.15.67:0/314100773 conn(0x7fc5e006bae0 msgr2=0x7fc5e006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:44.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.046+0000 7fc5e6200640 1 -- 172.21.15.67:0/314100773 shutdown_connections 2024-05-30T17:06:44.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.046+0000 7fc5e6200640 1 -- 172.21.15.67:0/314100773 wait complete. 2024-05-30T17:06:44.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.048+0000 7fc5e6200640 1 Processor -- start 2024-05-30T17:06:44.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.048+0000 7fc5e6200640 1 -- start start 2024-05-30T17:06:44.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5e6200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e00d0460 0x7fc5e0158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:44.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5e6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5e015b380 0x7fc5e00cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:44.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5e6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e015ef60 0x7fc5e00cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:44.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5e6200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e015b8e0 con 0x7fc5e015b380 2024-05-30T17:06:44.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5e6200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e015b760 con 0x7fc5e00d0460 2024-05-30T17:06:44.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5e6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e00b37a0 con 0x7fc5e015ef60 2024-05-30T17:06:44.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5e015b380 0x7fc5e00cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:44.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5e015b380 0x7fc5e00cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44700/0 (socket says 172.21.15.67:44700) 2024-05-30T17:06:44.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5d7e00640 1 -- 172.21.15.67:0/1218631765 learned_addr learned my addr 172.21.15.67:0/1218631765 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:44.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5e5800640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e015ef60 0x7fc5e00cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:44.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.049+0000 7fc5e4e00640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e00d0460 0x7fc5e0158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:44.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5d7e00640 1 -- 172.21.15.67:0/1218631765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e015ef60 msgr2=0x7fc5e00cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5d7e00640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e015ef60 0x7fc5e00cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5d7e00640 1 -- 172.21.15.67:0/1218631765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e00d0460 msgr2=0x7fc5e0158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5d7e00640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e00d0460 0x7fc5e0158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5d7e00640 1 -- 172.21.15.67:0/1218631765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e01f4920 con 0x7fc5e015b380 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5d7e00640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5e015b380 0x7fc5e00cba80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e840 tx=0x7fc5c800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5dd600640 1 -- 172.21.15.67:0/1218631765 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8017070 con 0x7fc5e015b380 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5dd600640 1 -- 172.21.15.67:0/1218631765 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c800c940 con 0x7fc5e015b380 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5dd600640 1 -- 172.21.15.67:0/1218631765 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800cd10 con 0x7fc5e015b380 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.050+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e01f4c00 con 0x7fc5e015b380 2024-05-30T17:06:44.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.051+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e01f51c0 con 0x7fc5e015b380 2024-05-30T17:06:44.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.052+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e00689e0 con 0x7fc5e015b380 2024-05-30T17:06:44.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.053+0000 7fc5dd600640 1 -- 172.21.15.67:0/1218631765 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7fc5c801a070 con 0x7fc5e015b380 2024-05-30T17:06:44.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.054+0000 7fc5dd600640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc5bc06e900 0x7fc5bc070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:44.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.054+0000 7fc5dd600640 1 -- 172.21.15.67:0/1218631765 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fc5c808ab00 con 0x7fc5e015b380 2024-05-30T17:06:44.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.054+0000 7fc5e4e00640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc5bc06e900 0x7fc5bc070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:44.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.055+0000 7fc5e4e00640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc5bc06e900 0x7fc5bc070dc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0001640 tx=0x7fc5d0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:44.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.059+0000 7fc5dd600640 1 -- 172.21.15.67:0/1218631765 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5c805a990 con 0x7fc5e015b380 2024-05-30T17:06:44.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.345+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc5e00b4fe0 con 0x7fc5e015b380 2024-05-30T17:06:44.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.347+0000 7fc5dd600640 1 -- 172.21.15.67:0/1218631765 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2415 (secure 0 0 0) 0x7fc5e00b4fe0 con 0x7fc5e015b380 2024-05-30T17:06:44.347 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:44.347 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:44.347 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:44.348 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1397, 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:44.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:06:44.350 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "read_bytes_sec": 699349, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 2100864, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 2 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:44.351 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:44.352 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:44.354 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:44.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.349+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc5bc06e900 msgr2=0x7fc5bc070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:44.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.349+0000 7fc5e6200640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc5bc06e900 0x7fc5bc070dc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0001640 tx=0x7fc5d0009040 comp rx=0 tx=0).stop 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.349+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5e015b380 msgr2=0x7fc5e00cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.349+0000 7fc5e6200640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5e015b380 0x7fc5e00cba80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e840 tx=0x7fc5c800ed00 comp rx=0 tx=0).stop 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.350+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 shutdown_connections 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.350+0000 7fc5e6200640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fc5bc06e900 0x7fc5bc070dc0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.350+0000 7fc5e6200640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5e015ef60 0x7fc5e00cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.350+0000 7fc5e6200640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5e015b380 0x7fc5e00cba80 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.350+0000 7fc5e6200640 1 --2- 172.21.15.67:0/1218631765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e00d0460 0x7fc5e0158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.350+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 >> 172.21.15.67:0/1218631765 conn(0x7fc5e006bae0 msgr2=0x7fc5e00ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.350+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 shutdown_connections 2024-05-30T17:06:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:44.350+0000 7fc5e6200640 1 -- 172.21.15.67:0/1218631765 wait complete. 2024-05-30T17:06:46.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.592+0000 7f71bce00640 1 Processor -- start 2024-05-30T17:06:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.592+0000 7f71bce00640 1 -- start start 2024-05-30T17:06:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.594+0000 7f71bce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71b80ccc30 0x7f71b806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.594+0000 7f71bce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f71b80d0460 0x7f71b80d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.594+0000 7f71bce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71b80cd1a0 con 0x7f71b815c6b0 2024-05-30T17:06:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.594+0000 7f71bce00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f71b80d0df0 con 0x7f71b80ccc30 2024-05-30T17:06:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.594+0000 7f71bce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f71b80cd480 con 0x7f71b80d0460 2024-05-30T17:06:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.594+0000 7f71aea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71b80ccc30 0x7f71b806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.594+0000 7f71aea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71b80ccc30 0x7f71b806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44710/0 (socket says 172.21.15.67:44710) 2024-05-30T17:06:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.594+0000 7f71aea00640 1 -- 172.21.15.67:0/2672220027 learned_addr learned my addr 172.21.15.67:0/2672220027 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.595+0000 7f71aea00640 1 -- 172.21.15.67:0/2672220027 >> v2:172.21.15.169:3300/0 conn(0x7f71b80d0460 msgr2=0x7f71b80d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.595+0000 7f71b7e00640 1 --2- 172.21.15.67:0/2672220027 >> v2:172.21.15.169:3300/0 conn(0x7f71b80d0460 0x7f71b80d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.595+0000 7f71aea00640 1 --2- 172.21.15.67:0/2672220027 >> v2:172.21.15.169:3300/0 conn(0x7f71b80d0460 0x7f71b80d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.595+0000 7f71aea00640 1 -- 172.21.15.67:0/2672220027 >> v1:172.21.15.169:6789/0 conn(0x7f71b815c6b0 legacy=0x7f71b80b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.595+0000 7f71aea00640 1 -- 172.21.15.67:0/2672220027 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b80cd760 con 0x7f71b80ccc30 2024-05-30T17:06:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.596+0000 7f71aea00640 1 --2- 172.21.15.67:0/2672220027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80ccc30 0x7f71b806dc90 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009870 tx=0x7f71a002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc0c8ebec8eac97 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.596+0000 7f71b6a00640 1 -- 172.21.15.67:0/2672220027 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0037070 con 0x7f71b80ccc30 2024-05-30T17:06:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.596+0000 7f71b6a00640 1 -- 172.21.15.67:0/2672220027 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a0032440 con 0x7f71b80ccc30 2024-05-30T17:06:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.596+0000 7f71b6a00640 1 -- 172.21.15.67:0/2672220027 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0032810 con 0x7f71b80ccc30 2024-05-30T17:06:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.597+0000 7f71bce00640 1 -- 172.21.15.67:0/2672220027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80ccc30 msgr2=0x7f71b806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.597+0000 7f71bce00640 1 --2- 172.21.15.67:0/2672220027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80ccc30 0x7f71b806dc90 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009870 tx=0x7f71a002a250 comp rx=0 tx=0).stop 2024-05-30T17:06:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.597+0000 7f71bce00640 1 -- 172.21.15.67:0/2672220027 shutdown_connections 2024-05-30T17:06:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.597+0000 7f71bce00640 1 --2- 172.21.15.67:0/2672220027 >> v2:172.21.15.169:3300/0 conn(0x7f71b80d0460 0x7f71b80d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.597+0000 7f71bce00640 1 --2- 172.21.15.67:0/2672220027 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80ccc30 0x7f71b806dc90 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.597+0000 7f71bce00640 1 -- 172.21.15.67:0/2672220027 >> 172.21.15.67:0/2672220027 conn(0x7f71b806bae0 msgr2=0x7f71b806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.597+0000 7f71bce00640 1 -- 172.21.15.67:0/2672220027 shutdown_connections 2024-05-30T17:06:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.597+0000 7f71bce00640 1 -- 172.21.15.67:0/2672220027 wait complete. 2024-05-30T17:06:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.599+0000 7f71bce00640 1 Processor -- start 2024-05-30T17:06:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.599+0000 7f71bce00640 1 -- start start 2024-05-30T17:06:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.600+0000 7f71bce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b80ccc30 0x7f71b81f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.600+0000 7f71bce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80d0460 0x7f71b80b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.600+0000 7f71bce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71b815c6b0 0x7f71b80ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.600+0000 7f71bce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f71b806e1d0 con 0x7f71b80d0460 2024-05-30T17:06:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.600+0000 7f71bce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71b80cba10 con 0x7f71b80ccc30 2024-05-30T17:06:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.600+0000 7f71bce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71b806e350 con 0x7f71b815c6b0 2024-05-30T17:06:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71aea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80d0460 0x7f71b80b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71aea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80d0460 0x7f71b80b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44720/0 (socket says 172.21.15.67:44720) 2024-05-30T17:06:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71aea00640 1 -- 172.21.15.67:0/307681928 learned_addr learned my addr 172.21.15.67:0/307681928 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71b7e00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71b815c6b0 0x7f71b80ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71b7400640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b80ccc30 0x7f71b81f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71aea00640 1 -- 172.21.15.67:0/307681928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71b815c6b0 msgr2=0x7f71b80ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71aea00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71b815c6b0 0x7f71b80ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71aea00640 1 -- 172.21.15.67:0/307681928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b80ccc30 msgr2=0x7f71b81f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71aea00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b80ccc30 0x7f71b81f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.601+0000 7f71aea00640 1 -- 172.21.15.67:0/307681928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b80c5430 con 0x7f71b80d0460 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.602+0000 7f71aea00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80d0460 0x7f71b80b1fc0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009870 tx=0x7f71a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.602+0000 7f71afe00640 1 -- 172.21.15.67:0/307681928 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0037040 con 0x7f71b80d0460 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.602+0000 7f71afe00640 1 -- 172.21.15.67:0/307681928 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a0004230 con 0x7f71b80d0460 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.602+0000 7f71afe00640 1 -- 172.21.15.67:0/307681928 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0030620 con 0x7f71b80d0460 2024-05-30T17:06:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.602+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b80c5710 con 0x7f71b80d0460 2024-05-30T17:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.602+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b80c5cd0 con 0x7f71b80d0460 2024-05-30T17:06:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.603+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b80c5f10 con 0x7f71b80d0460 2024-05-30T17:06:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.604+0000 7f71afe00640 1 -- 172.21.15.67:0/307681928 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7f71a00307c0 con 0x7f71b80d0460 2024-05-30T17:06:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.605+0000 7f71afe00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f719406e900 0x7f7194070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.606+0000 7f71afe00640 1 -- 172.21.15.67:0/307681928 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f71a00abba0 con 0x7f71b80d0460 2024-05-30T17:06:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.606+0000 7f71b7400640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f719406e900 0x7f7194070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.606+0000 7f71b7400640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f719406e900 0x7f7194070dc0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f71a8006fd0 tx=0x7f71a8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.610+0000 7f71afe00640 1 -- 172.21.15.67:0/307681928 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71b80c5f10 con 0x7f71b80d0460 2024-05-30T17:06:46.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:06:46.741 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:06:46.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.892+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f71b80689e0 con 0x7f71b80d0460 2024-05-30T17:06:46.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.893+0000 7f71afe00640 1 -- 172.21.15.67:0/307681928 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2378 (secure 0 0 0) 0x7f71a007ba30 con 0x7f71b80d0460 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:46.894 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1400, 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:46.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:46.896 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 3923, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:46.897 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:46.898 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:46.899 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:46.900 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:46.901 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:46.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f719406e900 msgr2=0x7f7194070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:46.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f719406e900 0x7f7194070dc0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f71a8006fd0 tx=0x7f71a8008040 comp rx=0 tx=0).stop 2024-05-30T17:06:46.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80d0460 msgr2=0x7f71b80b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:46.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80d0460 0x7f71b80b1fc0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009870 tx=0x7f71a00046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 shutdown_connections 2024-05-30T17:06:46.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f719406e900 0x7f7194070dc0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71b815c6b0 0x7f71b80ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b80d0460 0x7f71b80b1fc0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 --2- 172.21.15.67:0/307681928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71b80ccc30 0x7f71b81f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:46.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 >> 172.21.15.67:0/307681928 conn(0x7f71b806bae0 msgr2=0x7f71b80723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:46.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 shutdown_connections 2024-05-30T17:06:46.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:46.896+0000 7f71bce00640 1 -- 172.21.15.67:0/307681928 wait complete. 2024-05-30T17:06:48.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:49.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.137+0000 7fbeb6e00640 1 Processor -- start 2024-05-30T17:06:49.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.137+0000 7fbeb6e00640 1 -- start start 2024-05-30T17:06:49.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.139+0000 7fbeb6e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbeb00b1ce0 0x7fbeb00b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:49.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.139+0000 7fbeb6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00d1250 0x7fbeb0162420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:49.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.139+0000 7fbeb6e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb006d3f0 con 0x7fbeb0161dd0 2024-05-30T17:06:49.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.139+0000 7fbeb6e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fbeb00d06b0 con 0x7fbeb00b1ce0 2024-05-30T17:06:49.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.139+0000 7fbeb6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb01621b0 con 0x7fbeb00d1250 2024-05-30T17:06:49.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.139+0000 7fbeb5a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbeb0161dd0 0x7fbeb00d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51324/0 (socket says 172.21.15.67:51324) 2024-05-30T17:06:49.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.139+0000 7fbeb5a00640 1 -- 172.21.15.67:0/2221163533 learned_addr learned my addr 172.21.15.67:0/2221163533 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:06:49.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.139+0000 7fbeb5000640 1 --2- 172.21.15.67:0/2221163533 >> v2:172.21.15.67:3300/0 conn(0x7fbeb00b1ce0 0x7fbeb00b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:49.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.140+0000 7fbeb6400640 1 --2- 172.21.15.67:0/2221163533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00d1250 0x7fbeb0162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:49.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.140+0000 7fbeb5000640 1 -- 172.21.15.67:0/2221163533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00d1250 msgr2=0x7fbeb0162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:49.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.140+0000 7fbeb5000640 1 --2- 172.21.15.67:0/2221163533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00d1250 0x7fbeb0162420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.140+0000 7fbeb5000640 1 -- 172.21.15.67:0/2221163533 >> v1:172.21.15.67:6789/0 conn(0x7fbeb0161dd0 legacy=0x7fbeb00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:49.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.140+0000 7fbeb5000640 1 -- 172.21.15.67:0/2221163533 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb006d670 con 0x7fbeb00b1ce0 2024-05-30T17:06:49.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.140+0000 7fbeb6400640 1 --2- 172.21.15.67:0/2221163533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00d1250 0x7fbeb0162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:49.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.141+0000 7fbeb5000640 1 --2- 172.21.15.67:0/2221163533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00b1ce0 0x7fbeb00b4990 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0097a0 tx=0x7fbe9c02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3d68328013ae65 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.141+0000 7fbeafe00640 1 -- 172.21.15.67:0/2221163533 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c037070 con 0x7fbeb00b1ce0 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.141+0000 7fbeafe00640 1 -- 172.21.15.67:0/2221163533 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe9c02ae40 con 0x7fbeb00b1ce0 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.141+0000 7fbeafe00640 1 -- 172.21.15.67:0/2221163533 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c032660 con 0x7fbeb00b1ce0 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.142+0000 7fbeb6e00640 1 -- 172.21.15.67:0/2221163533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00b1ce0 msgr2=0x7fbeb00b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.142+0000 7fbeb6e00640 1 --2- 172.21.15.67:0/2221163533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00b1ce0 0x7fbeb00b4990 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0097a0 tx=0x7fbe9c02a0f0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.142+0000 7fbeb6e00640 1 -- 172.21.15.67:0/2221163533 shutdown_connections 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.142+0000 7fbeb6e00640 1 --2- 172.21.15.67:0/2221163533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00d1250 0x7fbeb0162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.142+0000 7fbeb6e00640 1 --2- 172.21.15.67:0/2221163533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00b1ce0 0x7fbeb00b4990 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.142+0000 7fbeb6e00640 1 -- 172.21.15.67:0/2221163533 >> 172.21.15.67:0/2221163533 conn(0x7fbeb00b3e70 msgr2=0x7fbeb00b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.142+0000 7fbeb6e00640 1 -- 172.21.15.67:0/2221163533 shutdown_connections 2024-05-30T17:06:49.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.142+0000 7fbeb6e00640 1 -- 172.21.15.67:0/2221163533 wait complete. 2024-05-30T17:06:49.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.144+0000 7fbeb6e00640 1 Processor -- start 2024-05-30T17:06:49.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.144+0000 7fbeb6e00640 1 -- start start 2024-05-30T17:06:49.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.145+0000 7fbeb6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00b1ce0 0x7fbeb0159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:49.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.145+0000 7fbeb6e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00d1250 0x7fbeb00c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.145+0000 7fbeb6e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbeb0161dd0 0x7fbeb00c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.145+0000 7fbeb6e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb006dc30 con 0x7fbeb00d1250 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.145+0000 7fbeb6e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb00b3550 con 0x7fbeb0161dd0 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.145+0000 7fbeb6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb006ddb0 con 0x7fbeb00b1ce0 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.145+0000 7fbeb5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00d1250 0x7fbeb00c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.145+0000 7fbeb5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00d1250 0x7fbeb00c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44744/0 (socket says 172.21.15.67:44744) 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.146+0000 7fbeb5000640 1 -- 172.21.15.67:0/1543529323 learned_addr learned my addr 172.21.15.67:0/1543529323 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.146+0000 7fbeb5a00640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00b1ce0 0x7fbeb0159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.146+0000 7fbeb6400640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbeb0161dd0 0x7fbeb00c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.146+0000 7fbeb5000640 1 -- 172.21.15.67:0/1543529323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00b1ce0 msgr2=0x7fbeb0159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.146+0000 7fbeb5000640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00b1ce0 0x7fbeb0159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.146+0000 7fbeb5000640 1 -- 172.21.15.67:0/1543529323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbeb0161dd0 msgr2=0x7fbeb00c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.146+0000 7fbeb5000640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbeb0161dd0 0x7fbeb00c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.146+0000 7fbeb5000640 1 -- 172.21.15.67:0/1543529323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb01595a0 con 0x7fbeb00d1250 2024-05-30T17:06:49.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.146+0000 7fbeb5000640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00d1250 0x7fbeb00c9450 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c035040 tx=0x7fbe9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:49.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.147+0000 7fbead600640 1 -- 172.21.15.67:0/1543529323 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c041070 con 0x7fbeb00d1250 2024-05-30T17:06:49.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.147+0000 7fbead600640 1 -- 172.21.15.67:0/1543529323 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe9c03c670 con 0x7fbeb00d1250 2024-05-30T17:06:49.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.147+0000 7fbead600640 1 -- 172.21.15.67:0/1543529323 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c037040 con 0x7fbeb00d1250 2024-05-30T17:06:49.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.147+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb00c6770 con 0x7fbeb00d1250 2024-05-30T17:06:49.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.147+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb00c6d30 con 0x7fbeb00d1250 2024-05-30T17:06:49.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.148+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb00689e0 con 0x7fbeb00d1250 2024-05-30T17:06:49.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.149+0000 7fbead600640 1 -- 172.21.15.67:0/1543529323 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7fbe9c03a070 con 0x7fbeb00d1250 2024-05-30T17:06:49.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.150+0000 7fbead600640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe9006e900 0x7fbe90070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.150+0000 7fbead600640 1 -- 172.21.15.67:0/1543529323 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fbe9c0ab0c0 con 0x7fbeb00d1250 2024-05-30T17:06:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.150+0000 7fbeb5a00640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe9006e900 0x7fbe90070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.151+0000 7fbeb5a00640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe9006e900 0x7fbe90070dc0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbeb015d0a0 tx=0x7fbea4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:49.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.155+0000 7fbead600640 1 -- 172.21.15.67:0/1543529323 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe9c07af50 con 0x7fbeb00d1250 2024-05-30T17:06:49.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.432+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbeb00c9990 con 0x7fbeb00d1250 2024-05-30T17:06:49.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.432+0000 7fbead600640 1 -- 172.21.15.67:0/1543529323 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2377 (secure 0 0 0) 0x7fbe9c07a850 con 0x7fbeb00d1250 2024-05-30T17:06:49.433 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:49.433 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:49.433 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:49.433 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:49.433 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:49.434 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1402, 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:49.435 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:06:49.436 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 170, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:49.438 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:49.439 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:49.440 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:49.441 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:49.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe9006e900 msgr2=0x7fbe90070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe9006e900 0x7fbe90070dc0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbeb015d0a0 tx=0x7fbea4009040 comp rx=0 tx=0).stop 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00d1250 msgr2=0x7fbeb00c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00d1250 0x7fbeb00c9450 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c035040 tx=0x7fbe9c0046c0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 shutdown_connections 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbe9006e900 0x7fbe90070dc0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbeb0161dd0 0x7fbeb00c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeb00d1250 0x7fbeb00c9450 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 --2- 172.21.15.67:0/1543529323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbeb00b1ce0 0x7fbeb0159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 >> 172.21.15.67:0/1543529323 conn(0x7fbeb00b3e70 msgr2=0x7fbeb00d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.435+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 shutdown_connections 2024-05-30T17:06:49.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:49.436+0000 7fbeb6e00640 1 -- 172.21.15.67:0/1543529323 wait complete. 2024-05-30T17:06:50.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-30T17:06:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.678+0000 7fe2fa800640 1 Processor -- start 2024-05-30T17:06:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.678+0000 7fe2fa800640 1 -- start start 2024-05-30T17:06:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.679+0000 7fe2fa800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe2f406ee40 0x7fe2f40ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.679+0000 7fe2fa800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe2f406f2f0 0x7fe2f40ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.679+0000 7fe2fa800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f4156f10 con 0x7fe2f4156b30 2024-05-30T17:06:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.680+0000 7fe2fa800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f406dda0 con 0x7fe2f406ee40 2024-05-30T17:06:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.680+0000 7fe2fa800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f406e080 con 0x7fe2f406f2f0 2024-05-30T17:06:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.680+0000 7fe2f8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe2f406ee40 0x7fe2f40ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.680+0000 7fe2f8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe2f406ee40 0x7fe2f40ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44746/0 (socket says 172.21.15.67:44746) 2024-05-30T17:06:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.680+0000 7fe2f8a00640 1 -- 172.21.15.67:0/4006411188 learned_addr learned my addr 172.21.15.67:0/4006411188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.680+0000 7fe2f8a00640 1 -- 172.21.15.67:0/4006411188 >> v2:172.21.15.169:3300/0 conn(0x7fe2f406f2f0 msgr2=0x7fe2f40ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.680+0000 7fe2f8a00640 1 --2- 172.21.15.67:0/4006411188 >> v2:172.21.15.169:3300/0 conn(0x7fe2f406f2f0 0x7fe2f40ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.680+0000 7fe2f8a00640 1 -- 172.21.15.67:0/4006411188 >> v1:172.21.15.169:6789/0 conn(0x7fe2f4156b30 legacy=0x7fe2f40af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:50.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.680+0000 7fe2f8a00640 1 -- 172.21.15.67:0/4006411188 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f406e360 con 0x7fe2f406ee40 2024-05-30T17:06:50.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.682+0000 7fe2f8a00640 1 --2- 172.21.15.67:0/4006411188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 0x7fe2f40ad230 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40d0180 tx=0x7fe2e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75b1a387ec336b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:50.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.682+0000 7fe2f3e00640 1 -- 172.21.15.67:0/4006411188 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0037070 con 0x7fe2f406ee40 2024-05-30T17:06:50.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.682+0000 7fe2f3e00640 1 -- 172.21.15.67:0/4006411188 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e0032440 con 0x7fe2f406ee40 2024-05-30T17:06:50.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.682+0000 7fe2f3e00640 1 -- 172.21.15.67:0/4006411188 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0032810 con 0x7fe2f406ee40 2024-05-30T17:06:50.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.683+0000 7fe2fa800640 1 -- 172.21.15.67:0/4006411188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 msgr2=0x7fe2f40ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:50.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.683+0000 7fe2fa800640 1 --2- 172.21.15.67:0/4006411188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 0x7fe2f40ad230 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40d0180 tx=0x7fe2e002a1a0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.683+0000 7fe2fa800640 1 -- 172.21.15.67:0/4006411188 shutdown_connections 2024-05-30T17:06:50.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.683+0000 7fe2fa800640 1 --2- 172.21.15.67:0/4006411188 >> v2:172.21.15.169:3300/0 conn(0x7fe2f406f2f0 0x7fe2f40ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.683+0000 7fe2fa800640 1 --2- 172.21.15.67:0/4006411188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 0x7fe2f40ad230 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.683+0000 7fe2fa800640 1 -- 172.21.15.67:0/4006411188 >> 172.21.15.67:0/4006411188 conn(0x7fe2f40b1fc0 msgr2=0x7fe2f40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:50.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.683+0000 7fe2fa800640 1 -- 172.21.15.67:0/4006411188 shutdown_connections 2024-05-30T17:06:50.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.683+0000 7fe2fa800640 1 -- 172.21.15.67:0/4006411188 wait complete. 2024-05-30T17:06:50.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.685+0000 7fe2fa800640 1 Processor -- start 2024-05-30T17:06:50.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.686+0000 7fe2fa800640 1 -- start start 2024-05-30T17:06:50.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.687+0000 7fe2fa800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 0x7fe2f4158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:50.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.687+0000 7fe2fa800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f406f2f0 0x7fe2f4158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:50.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.687+0000 7fe2fa800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f4156b30 0x7fe2f4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:50.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.687+0000 7fe2fa800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f40b2db0 con 0x7fe2f406ee40 2024-05-30T17:06:50.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.687+0000 7fe2fa800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f40b2c30 con 0x7fe2f4156b30 2024-05-30T17:06:50.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.687+0000 7fe2fa800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f40b32c0 con 0x7fe2f406f2f0 2024-05-30T17:06:50.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.687+0000 7fe2f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 0x7fe2f4158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:50.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.687+0000 7fe2f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 0x7fe2f4158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44760/0 (socket says 172.21.15.67:44760) 2024-05-30T17:06:50.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.687+0000 7fe2f9400640 1 -- 172.21.15.67:0/938231283 learned_addr learned my addr 172.21.15.67:0/938231283 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f8a00640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f406f2f0 0x7fe2f4158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f9e00640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f4156b30 0x7fe2f4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f9400640 1 -- 172.21.15.67:0/938231283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f406f2f0 msgr2=0x7fe2f4158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f9400640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f406f2f0 0x7fe2f4158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f9400640 1 -- 172.21.15.67:0/938231283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f4156b30 msgr2=0x7fe2f4158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f9400640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f4156b30 0x7fe2f4158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f9400640 1 -- 172.21.15.67:0/938231283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f4159530 con 0x7fe2f406ee40 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f9e00640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f4156b30 0x7fe2f4158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f9400640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 0x7fe2f4158400 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400ba80 tx=0x7fe2e400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f1600640 1 -- 172.21.15.67:0/938231283 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4019070 con 0x7fe2f406ee40 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f1600640 1 -- 172.21.15.67:0/938231283 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e400cc90 con 0x7fe2f406ee40 2024-05-30T17:06:50.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2f1600640 1 -- 172.21.15.67:0/938231283 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4014680 con 0x7fe2f406ee40 2024-05-30T17:06:50.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f415d580 con 0x7fe2f406ee40 2024-05-30T17:06:50.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.688+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f415db40 con 0x7fe2f406ee40 2024-05-30T17:06:50.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.690+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f40689e0 con 0x7fe2f406ee40 2024-05-30T17:06:50.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.691+0000 7fe2f1600640 1 -- 172.21.15.67:0/938231283 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7fe2e401c070 con 0x7fe2f406ee40 2024-05-30T17:06:50.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.692+0000 7fe2f1600640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406e900 0x7fe2d4070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:50.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.692+0000 7fe2f1600640 1 -- 172.21.15.67:0/938231283 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fe2e408aac0 con 0x7fe2f406ee40 2024-05-30T17:06:50.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.692+0000 7fe2f8a00640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406e900 0x7fe2d4070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:50.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.693+0000 7fe2f8a00640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406e900 0x7fe2d4070dc0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0034000 tx=0x7fe2e002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.696+0000 7fe2f1600640 1 -- 172.21.15.67:0/938231283 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2e405a950 con 0x7fe2f406ee40 2024-05-30T17:06:50.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.935+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe2f406f7f0 con 0x7fe2d406e900 2024-05-30T17:06:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.938+0000 7fe2f1600640 1 -- 172.21.15.67:0/938231283 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe2f406f7f0 con 0x7fe2d406e900 2024-05-30T17:06:50.939 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-05-30T17:06:50.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:50.940 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephfs" 2024-05-30T17:06:50.940 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:50.940 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-05-30T17:06:50.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.940+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406e900 msgr2=0x7fe2d4070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:50.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406e900 0x7fe2d4070dc0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0034000 tx=0x7fe2e002e000 comp rx=0 tx=0).stop 2024-05-30T17:06:50.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 msgr2=0x7fe2f4158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:50.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 0x7fe2f4158400 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400ba80 tx=0x7fe2e400bf40 comp rx=0 tx=0).stop 2024-05-30T17:06:50.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 shutdown_connections 2024-05-30T17:06:50.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe2d406e900 0x7fe2d4070dc0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f4156b30 0x7fe2f4158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f406f2f0 0x7fe2f4158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 --2- 172.21.15.67:0/938231283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2f406ee40 0x7fe2f4158400 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:50.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 >> 172.21.15.67:0/938231283 conn(0x7fe2f40b1fc0 msgr2=0x7fe2f40adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:50.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.941+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 shutdown_connections 2024-05-30T17:06:50.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:50.942+0000 7fe2fa800640 1 -- 172.21.15.67:0/938231283 wait complete. 2024-05-30T17:06:50.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.172+0000 7ff11ee00640 1 Processor -- start 2024-05-30T17:06:51.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.172+0000 7ff11ee00640 1 -- start start 2024-05-30T17:06:51.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11ee00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1180ccc30 0x7ff11806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:51.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11ee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1180cd1a0 con 0x7ff11815c6b0 2024-05-30T17:06:51.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1180cd480 con 0x7ff1180d0460 2024-05-30T17:06:51.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11ee00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff1180cd760 con 0x7ff1180ccc30 2024-05-30T17:06:51.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1180ccc30 0x7ff11806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:51.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff1180ccc30 0x7ff11806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38866/0 (socket says 172.21.15.67:38866) 2024-05-30T17:06:51.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11d000640 1 -- 172.21.15.67:0/2805681825 learned_addr learned my addr 172.21.15.67:0/2805681825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:51.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11d000640 1 -- 172.21.15.67:0/2805681825 >> v1:172.21.15.169:6789/0 conn(0x7ff1180d0460 legacy=0x7ff1180cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:06:51.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11d000640 1 -- 172.21.15.67:0/2805681825 >> v1:172.21.15.67:6789/0 conn(0x7ff11815c6b0 legacy=0x7ff1180b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:51.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.174+0000 7ff11d000640 1 -- 172.21.15.67:0/2805681825 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1180cda40 con 0x7ff1180ccc30 2024-05-30T17:06:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.175+0000 7ff11d000640 1 --2- 172.21.15.67:0/2805681825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180ccc30 0x7ff11806dc90 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff1040098a0 tx=0x7ff10402a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eb7fd5667e8dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.176+0000 7ff117e00640 1 -- 172.21.15.67:0/2805681825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104037070 con 0x7ff1180ccc30 2024-05-30T17:06:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.176+0000 7ff117e00640 1 -- 172.21.15.67:0/2805681825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff104032440 con 0x7ff1180ccc30 2024-05-30T17:06:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.176+0000 7ff117e00640 1 -- 172.21.15.67:0/2805681825 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104032810 con 0x7ff1180ccc30 2024-05-30T17:06:51.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.176+0000 7ff11ee00640 1 -- 172.21.15.67:0/2805681825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180ccc30 msgr2=0x7ff11806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:51.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.176+0000 7ff11ee00640 1 --2- 172.21.15.67:0/2805681825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180ccc30 0x7ff11806dc90 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff1040098a0 tx=0x7ff10402a2e0 comp rx=0 tx=0).stop 2024-05-30T17:06:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.177+0000 7ff11ee00640 1 -- 172.21.15.67:0/2805681825 shutdown_connections 2024-05-30T17:06:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.177+0000 7ff11ee00640 1 --2- 172.21.15.67:0/2805681825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1180ccc30 0x7ff11806dc90 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.177+0000 7ff11ee00640 1 -- 172.21.15.67:0/2805681825 >> 172.21.15.67:0/2805681825 conn(0x7ff11806bae0 msgr2=0x7ff11806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.177+0000 7ff11ee00640 1 -- 172.21.15.67:0/2805681825 shutdown_connections 2024-05-30T17:06:51.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.177+0000 7ff11ee00640 1 -- 172.21.15.67:0/2805681825 wait complete. 2024-05-30T17:06:51.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.178+0000 7ff11ee00640 1 Processor -- start 2024-05-30T17:06:51.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.179+0000 7ff11ee00640 1 -- start start 2024-05-30T17:06:51.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.179+0000 7ff11ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180ccc30 0x7ff1180c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:51.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.179+0000 7ff11ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1180d0460 0x7ff1181441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:51.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.179+0000 7ff11ee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11815c6b0 0x7ff118145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:51.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.179+0000 7ff11ee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff11806e350 con 0x7ff11815c6b0 2024-05-30T17:06:51.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.179+0000 7ff11ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff11806e1d0 con 0x7ff1180d0460 2024-05-30T17:06:51.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.179+0000 7ff11ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1180cc640 con 0x7ff1180ccc30 2024-05-30T17:06:51.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11815c6b0 0x7ff118145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:51.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11815c6b0 0x7ff118145c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38878/0 (socket says 172.21.15.67:38878) 2024-05-30T17:06:51.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11e400640 1 -- 172.21.15.67:0/2194285904 learned_addr learned my addr 172.21.15.67:0/2194285904 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:51.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11d000640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1180d0460 0x7ff1181441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11da00640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180ccc30 0x7ff1180c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11e400640 1 -- 172.21.15.67:0/2194285904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180ccc30 msgr2=0x7ff1180c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11e400640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180ccc30 0x7ff1180c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11e400640 1 -- 172.21.15.67:0/2194285904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1180d0460 msgr2=0x7ff1181441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11e400640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1180d0460 0x7ff1181441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.180+0000 7ff11e400640 1 -- 172.21.15.67:0/2194285904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1180738b0 con 0x7ff11815c6b0 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.181+0000 7ff11e400640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11815c6b0 0x7ff118145c30 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c780 tx=0x7ff11000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.181+0000 7ff115600640 1 -- 172.21.15.67:0/2194285904 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110017070 con 0x7ff11815c6b0 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.181+0000 7ff115600640 1 -- 172.21.15.67:0/2194285904 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff110003da0 con 0x7ff11815c6b0 2024-05-30T17:06:51.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.181+0000 7ff115600640 1 -- 172.21.15.67:0/2194285904 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1100052b0 con 0x7ff11815c6b0 2024-05-30T17:06:51.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.181+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff118073b90 con 0x7ff11815c6b0 2024-05-30T17:06:51.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.181+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11814f050 con 0x7ff11815c6b0 2024-05-30T17:06:51.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.182+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1180689e0 con 0x7ff11815c6b0 2024-05-30T17:06:51.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.183+0000 7ff115600640 1 -- 172.21.15.67:0/2194285904 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7ff110005450 con 0x7ff11815c6b0 2024-05-30T17:06:51.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.184+0000 7ff115600640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0f806e900 0x7ff0f8070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:51.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.185+0000 7ff115600640 1 -- 172.21.15.67:0/2194285904 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7ff11008a260 con 0x7ff11815c6b0 2024-05-30T17:06:51.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.185+0000 7ff11da00640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0f806e900 0x7ff0f8070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:51.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.185+0000 7ff11da00640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0f806e900 0x7ff0f8070dc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff10c007fd0 tx=0x7ff10c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:51.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.189+0000 7ff115600640 1 -- 172.21.15.67:0/2194285904 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff11005a0f0 con 0x7ff11815c6b0 2024-05-30T17:06:51.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.467+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff1180c50c0 con 0x7ff11815c6b0 2024-05-30T17:06:51.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.468+0000 7ff115600640 1 -- 172.21.15.67:0/2194285904 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2377 (secure 0 0 0) 0x7ff1100599f0 con 0x7ff11815c6b0 2024-05-30T17:06:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:51.470 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1404, 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 170, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:51.476 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:51.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0f806e900 msgr2=0x7ff0f8070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:51.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0f806e900 0x7ff0f8070dc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff10c007fd0 tx=0x7ff10c009040 comp rx=0 tx=0).stop 2024-05-30T17:06:51.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11815c6b0 msgr2=0x7ff118145c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:51.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11815c6b0 0x7ff118145c30 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c780 tx=0x7ff11000cc40 comp rx=0 tx=0).stop 2024-05-30T17:06:51.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 shutdown_connections 2024-05-30T17:06:51.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff0f806e900 0x7ff0f8070dc0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:51.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11815c6b0 0x7ff118145c30 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:51.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1180d0460 0x7ff1181441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:51.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 --2- 172.21.15.67:0/2194285904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1180ccc30 0x7ff1180c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:51.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 >> 172.21.15.67:0/2194285904 conn(0x7ff11806bae0 msgr2=0x7ff1180b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:51.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 shutdown_connections 2024-05-30T17:06:51.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:51.471+0000 7ff11ee00640 1 -- 172.21.15.67:0/2194285904 wait complete. 2024-05-30T17:06:56.491 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:06:56.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.709+0000 7fa98a800640 1 Processor -- start 2024-05-30T17:06:56.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.710+0000 7fa98a800640 1 -- start start 2024-05-30T17:06:56.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.711+0000 7fa98a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa98406c7b0 0x7fa98406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:56.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.711+0000 7fa98a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9840d50d0 con 0x7fa9840cfd60 2024-05-30T17:06:56.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.711+0000 7fa98a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9840d53b0 con 0x7fa984161b90 2024-05-30T17:06:56.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.711+0000 7fa98a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa9840d2040 con 0x7fa98406c7b0 2024-05-30T17:06:56.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.712+0000 7fa988a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa98406c7b0 0x7fa98406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:56.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.712+0000 7fa988a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa98406c7b0 0x7fa98406cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38880/0 (socket says 172.21.15.67:38880) 2024-05-30T17:06:56.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.712+0000 7fa988a00640 1 -- 172.21.15.67:0/1526719191 learned_addr learned my addr 172.21.15.67:0/1526719191 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:56.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.712+0000 7fa988a00640 1 -- 172.21.15.67:0/1526719191 >> v1:172.21.15.169:6789/0 conn(0x7fa984161b90 legacy=0x7fa9840d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:56.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.712+0000 7fa988a00640 1 -- 172.21.15.67:0/1526719191 >> v1:172.21.15.67:6789/0 conn(0x7fa9840cfd60 legacy=0x7fa9840d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:56.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.712+0000 7fa988a00640 1 -- 172.21.15.67:0/1526719191 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9840cf560 con 0x7fa98406c7b0 2024-05-30T17:06:56.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.713+0000 7fa988a00640 1 --2- 172.21.15.67:0/1526719191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98406c7b0 0x7fa98406cc10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fa974009e50 tx=0x7fa97402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d46b35184362ed server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:56.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.714+0000 7fa983e00640 1 -- 172.21.15.67:0/1526719191 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974036070 con 0x7fa98406c7b0 2024-05-30T17:06:56.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.714+0000 7fa983e00640 1 -- 172.21.15.67:0/1526719191 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa974031600 con 0x7fa98406c7b0 2024-05-30T17:06:56.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.714+0000 7fa983e00640 1 -- 172.21.15.67:0/1526719191 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9740319d0 con 0x7fa98406c7b0 2024-05-30T17:06:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.714+0000 7fa98a800640 1 -- 172.21.15.67:0/1526719191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98406c7b0 msgr2=0x7fa98406cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.714+0000 7fa98a800640 1 --2- 172.21.15.67:0/1526719191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98406c7b0 0x7fa98406cc10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fa974009e50 tx=0x7fa97402a6a0 comp rx=0 tx=0).stop 2024-05-30T17:06:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.715+0000 7fa98a800640 1 -- 172.21.15.67:0/1526719191 shutdown_connections 2024-05-30T17:06:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.715+0000 7fa98a800640 1 --2- 172.21.15.67:0/1526719191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa98406c7b0 0x7fa98406cc10 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.715+0000 7fa98a800640 1 -- 172.21.15.67:0/1526719191 >> 172.21.15.67:0/1526719191 conn(0x7fa98408bfc0 msgr2=0x7fa9840d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.715+0000 7fa98a800640 1 -- 172.21.15.67:0/1526719191 shutdown_connections 2024-05-30T17:06:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.715+0000 7fa98a800640 1 -- 172.21.15.67:0/1526719191 wait complete. 2024-05-30T17:06:56.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.716+0000 7fa98a800640 1 Processor -- start 2024-05-30T17:06:56.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.717+0000 7fa98a800640 1 -- start start 2024-05-30T17:06:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa98a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa98406c7b0 0x7fa9841569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa98a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9840cfd60 0x7fa9840b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa98a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984161b90 0x7fa9840b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa98a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840cf210 con 0x7fa9840cfd60 2024-05-30T17:06:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa98a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840cf8d0 con 0x7fa984161b90 2024-05-30T17:06:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa98a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9840d4d30 con 0x7fa98406c7b0 2024-05-30T17:06:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa988a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9840cfd60 0x7fa9840b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa988a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9840cfd60 0x7fa9840b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38882/0 (socket says 172.21.15.67:38882) 2024-05-30T17:06:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa988a00640 1 -- 172.21.15.67:0/3518482234 learned_addr learned my addr 172.21.15.67:0/3518482234 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:06:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa989e00640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984161b90 0x7fa9840b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa989400640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa98406c7b0 0x7fa9841569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa988a00640 1 -- 172.21.15.67:0/3518482234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa98406c7b0 msgr2=0x7fa9841569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa988a00640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa98406c7b0 0x7fa9841569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa988a00640 1 -- 172.21.15.67:0/3518482234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984161b90 msgr2=0x7fa9840b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa988a00640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984161b90 0x7fa9840b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa988a00640 1 -- 172.21.15.67:0/3518482234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9840ccc70 con 0x7fa9840cfd60 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.718+0000 7fa989e00640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984161b90 0x7fa9840b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.719+0000 7fa988a00640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9840cfd60 0x7fa9840b5160 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fa974009f70 tx=0x7fa974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.719+0000 7fa981600640 1 -- 172.21.15.67:0/3518482234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97403f020 con 0x7fa9840cfd60 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.719+0000 7fa981600640 1 -- 172.21.15.67:0/3518482234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa974037040 con 0x7fa9840cfd60 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.719+0000 7fa981600640 1 -- 172.21.15.67:0/3518482234 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974036070 con 0x7fa9840cfd60 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.719+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9840ccf50 con 0x7fa9840cfd60 2024-05-30T17:06:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.719+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9840cd510 con 0x7fa9840cfd60 2024-05-30T17:06:56.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.720+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9840cd820 con 0x7fa9840cfd60 2024-05-30T17:06:56.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.721+0000 7fa981600640 1 -- 172.21.15.67:0/3518482234 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7fa974042020 con 0x7fa9840cfd60 2024-05-30T17:06:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.723+0000 7fa981600640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa96806e900 0x7fa968070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:06:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.723+0000 7fa981600640 1 -- 172.21.15.67:0/3518482234 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fa9740ab250 con 0x7fa9840cfd60 2024-05-30T17:06:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.723+0000 7fa989400640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa96806e900 0x7fa968070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:06:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.724+0000 7fa989400640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa96806e900 0x7fa968070dc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa9840d38c0 tx=0x7fa978009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:06:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:56.728+0000 7fa981600640 1 -- 172.21.15.67:0/3518482234 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa974049030 con 0x7fa9840cfd60 2024-05-30T17:06:57.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.014+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa9840690f0 con 0x7fa9840cfd60 2024-05-30T17:06:57.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.015+0000 7fa981600640 1 -- 172.21.15.67:0/3518482234 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2284 (secure 0 0 0) 0x7fa97407b0e0 con 0x7fa9840cfd60 2024-05-30T17:06:57.016 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:57.017 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1410, 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:06:57.018 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:06:57.019 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:06:57.020 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:06:57.021 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:06:57.022 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:06:57.023 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:06:57.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa96806e900 msgr2=0x7fa968070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:57.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa96806e900 0x7fa968070dc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa9840d38c0 tx=0x7fa978009040 comp rx=0 tx=0).stop 2024-05-30T17:06:57.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9840cfd60 msgr2=0x7fa9840b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:06:57.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9840cfd60 0x7fa9840b5160 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fa974009f70 tx=0x7fa974004a40 comp rx=0 tx=0).stop 2024-05-30T17:06:57.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 shutdown_connections 2024-05-30T17:06:57.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa96806e900 0x7fa968070dc0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:57.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa984161b90 0x7fa9840b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:57.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9840cfd60 0x7fa9840b5160 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:57.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 --2- 172.21.15.67:0/3518482234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa98406c7b0 0x7fa9841569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:06:57.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 >> 172.21.15.67:0/3518482234 conn(0x7fa98408bfc0 msgr2=0x7fa9841574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:06:57.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 shutdown_connections 2024-05-30T17:06:57.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:06:57.018+0000 7fa98a800640 1 -- 172.21.15.67:0/3518482234 wait complete. 2024-05-30T17:07:02.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:07:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.259+0000 7f7ed7e00640 1 Processor -- start 2024-05-30T17:07:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.260+0000 7f7ed7e00640 1 -- start start 2024-05-30T17:07:02.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.261+0000 7f7ed7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7ed015ae10 0x7f7ed00cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:02.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.261+0000 7f7ed7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed006f6f0 con 0x7f7ed00723d0 2024-05-30T17:07:02.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.261+0000 7f7ed7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed006f9d0 con 0x7f7ed006f1a0 2024-05-30T17:07:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.261+0000 7f7ed7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed006fcb0 con 0x7f7ed015ae10 2024-05-30T17:07:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.261+0000 7f7ed5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7ed015ae10 0x7f7ed00cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.262+0000 7f7ed5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7ed015ae10 0x7f7ed00cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48394/0 (socket says 172.21.15.67:48394) 2024-05-30T17:07:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.262+0000 7f7ed5400640 1 -- 172.21.15.67:0/2039293920 learned_addr learned my addr 172.21.15.67:0/2039293920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.262+0000 7f7ed5400640 1 -- 172.21.15.67:0/2039293920 >> v1:172.21.15.169:6789/0 conn(0x7f7ed006f1a0 legacy=0x7f7ed0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.262+0000 7f7ed5400640 1 -- 172.21.15.67:0/2039293920 >> v1:172.21.15.67:6789/0 conn(0x7f7ed00723d0 legacy=0x7f7ed015ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.262+0000 7f7ed5400640 1 -- 172.21.15.67:0/2039293920 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed006ff90 con 0x7f7ed015ae10 2024-05-30T17:07:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.263+0000 7f7ed5400640 1 --2- 172.21.15.67:0/2039293920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 0x7f7ed00cd7b0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009870 tx=0x7f7ec402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24adaa8f787b64e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:02.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.263+0000 7f7ecfe00640 1 -- 172.21.15.67:0/2039293920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4037070 con 0x7f7ed015ae10 2024-05-30T17:07:02.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.264+0000 7f7ecfe00640 1 -- 172.21.15.67:0/2039293920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec4032440 con 0x7f7ed015ae10 2024-05-30T17:07:02.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.264+0000 7f7ecfe00640 1 -- 172.21.15.67:0/2039293920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4032810 con 0x7f7ed015ae10 2024-05-30T17:07:02.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.264+0000 7f7ed7e00640 1 -- 172.21.15.67:0/2039293920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 msgr2=0x7f7ed00cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:02.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.264+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/2039293920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 0x7f7ed00cd7b0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009870 tx=0x7f7ec402a2a0 comp rx=0 tx=0).stop 2024-05-30T17:07:02.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.264+0000 7f7ed7e00640 1 -- 172.21.15.67:0/2039293920 shutdown_connections 2024-05-30T17:07:02.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.264+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/2039293920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 0x7f7ed00cd7b0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:02.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.264+0000 7f7ed7e00640 1 -- 172.21.15.67:0/2039293920 >> 172.21.15.67:0/2039293920 conn(0x7f7ed0067ce0 msgr2=0x7f7ed006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:02.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.264+0000 7f7ed7e00640 1 -- 172.21.15.67:0/2039293920 shutdown_connections 2024-05-30T17:07:02.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.264+0000 7f7ed7e00640 1 -- 172.21.15.67:0/2039293920 wait complete. 2024-05-30T17:07:02.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.266+0000 7f7ed7e00640 1 Processor -- start 2024-05-30T17:07:02.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.266+0000 7f7ed7e00640 1 -- start start 2024-05-30T17:07:02.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.267+0000 7f7ed7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed006f1a0 0x7f7ed0155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:02.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.267+0000 7f7ed7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ed00723d0 0x7f7ed00ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:02.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.267+0000 7f7ed7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 0x7f7ed015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:02.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.267+0000 7f7ed7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00b3fd0 con 0x7f7ed015ae10 2024-05-30T17:07:02.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.267+0000 7f7ed7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00b3e50 con 0x7f7ed006f1a0 2024-05-30T17:07:02.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.267+0000 7f7ed7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed015f510 con 0x7f7ed00723d0 2024-05-30T17:07:02.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.267+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 0x7f7ed015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:02.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 0x7f7ed015dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48402/0 (socket says 172.21.15.67:48402) 2024-05-30T17:07:02.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed5e00640 1 -- 172.21.15.67:0/1918771928 learned_addr learned my addr 172.21.15.67:0/1918771928 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed4a00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ed00723d0 0x7f7ed00ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed5400640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed006f1a0 0x7f7ed0155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed5e00640 1 -- 172.21.15.67:0/1918771928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ed00723d0 msgr2=0x7f7ed00ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed5e00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ed00723d0 0x7f7ed00ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed5e00640 1 -- 172.21.15.67:0/1918771928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed006f1a0 msgr2=0x7f7ed0155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed5e00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed006f1a0 0x7f7ed0155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed5e00640 1 -- 172.21.15.67:0/1918771928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed015e270 con 0x7f7ed015ae10 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ed5e00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 0x7f7ed015dd30 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c730 tx=0x7f7ec800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.268+0000 7f7ecd600640 1 -- 172.21.15.67:0/1918771928 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8003b70 con 0x7f7ed015ae10 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.269+0000 7f7ecd600640 1 -- 172.21.15.67:0/1918771928 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec8003d10 con 0x7f7ed015ae10 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.269+0000 7f7ecd600640 1 -- 172.21.15.67:0/1918771928 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec800aa90 con 0x7f7ed015ae10 2024-05-30T17:07:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.269+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed00adff0 con 0x7f7ed015ae10 2024-05-30T17:07:02.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.269+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed00ae5b0 con 0x7f7ed015ae10 2024-05-30T17:07:02.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.270+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed00689e0 con 0x7f7ed015ae10 2024-05-30T17:07:02.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.271+0000 7f7ecd600640 1 -- 172.21.15.67:0/1918771928 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7f7ec800ac30 con 0x7f7ed015ae10 2024-05-30T17:07:02.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.272+0000 7f7ecd600640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7eb006e930 0x7f7eb0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:02.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.272+0000 7f7ecd600640 1 -- 172.21.15.67:0/1918771928 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f7ec808ab50 con 0x7f7ed015ae10 2024-05-30T17:07:02.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.272+0000 7f7ed5400640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7eb006e930 0x7f7eb0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:02.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.273+0000 7f7ed5400640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7eb006e930 0x7f7eb0070df0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4034000 tx=0x7f7ec402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:02.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.277+0000 7f7ecd600640 1 -- 172.21.15.67:0/1918771928 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ec805a9e0 con 0x7f7ed015ae10 2024-05-30T17:07:02.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.557+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7ed0072db0 con 0x7f7ed015ae10 2024-05-30T17:07:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.558+0000 7f7ecd600640 1 -- 172.21.15.67:0/1918771928 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2284 (secure 0 0 0) 0x7f7ec805a2e0 con 0x7f7ed015ae10 2024-05-30T17:07:02.558 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:07:02.559 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1415, 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:07:02.560 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:02.561 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:07:02.562 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:07:02.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:07:02.564 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:02.565 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:02.566 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:02.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.560+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7eb006e930 msgr2=0x7f7eb0070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.560+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7eb006e930 0x7f7eb0070df0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4034000 tx=0x7f7ec402e000 comp rx=0 tx=0).stop 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.560+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 msgr2=0x7f7ed015dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.560+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 0x7f7ed015dd30 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c730 tx=0x7f7ec800cbf0 comp rx=0 tx=0).stop 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.561+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 shutdown_connections 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.561+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7eb006e930 0x7f7eb0070df0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.561+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed015ae10 0x7f7ed015dd30 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.561+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ed00723d0 0x7f7ed00ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.561+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/1918771928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ed006f1a0 0x7f7ed0155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.561+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 >> 172.21.15.67:0/1918771928 conn(0x7f7ed0067ce0 msgr2=0x7f7ed0158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:02.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.561+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 shutdown_connections 2024-05-30T17:07:02.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:02.561+0000 7f7ed7e00640 1 -- 172.21.15.67:0/1918771928 wait complete. 2024-05-30T17:07:07.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:07:07.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.799+0000 7fbb61200640 1 Processor -- start 2024-05-30T17:07:07.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.799+0000 7fbb61200640 1 -- start start 2024-05-30T17:07:07.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.801+0000 7fbb61200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb5c15c6b0 0x7fbb5c0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:07.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.801+0000 7fbb61200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb5c0cd1a0 con 0x7fbb5c0ccc30 2024-05-30T17:07:07.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.801+0000 7fbb61200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb5c0cd480 con 0x7fbb5c0d0460 2024-05-30T17:07:07.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.801+0000 7fbb61200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5c06e3a0 con 0x7fbb5c15c6b0 2024-05-30T17:07:07.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.801+0000 7fbb52a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbb5c0ccc30 0x7fbb5c06dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48206/0 (socket says 172.21.15.67:48206) 2024-05-30T17:07:07.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.801+0000 7fbb52a00640 1 -- 172.21.15.67:0/889068384 learned_addr learned my addr 172.21.15.67:0/889068384 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3630592326 0 0) 0x7fbb5c0cd1a0 con 0x7fbb5c0ccc30 2024-05-30T17:07:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb3c003520 con 0x7fbb5c0ccc30 2024-05-30T17:07:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5b400640 1 --2- 172.21.15.67:0/889068384 >> v2:172.21.15.169:3300/0 conn(0x7fbb5c15c6b0 0x7fbb5c0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 385150200 0 0) 0x7fbb3c003520 con 0x7fbb5c0ccc30 2024-05-30T17:07:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 >> v2:172.21.15.169:3300/0 conn(0x7fbb5c15c6b0 msgr2=0x7fbb5c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 --2- 172.21.15.67:0/889068384 >> v2:172.21.15.169:3300/0 conn(0x7fbb5c15c6b0 0x7fbb5c0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 >> v1:172.21.15.169:6789/0 conn(0x7fbb5c0d0460 legacy=0x7fbb5c0cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c0cd760 con 0x7fbb5c0ccc30 2024-05-30T17:07:07.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fbb44001670 con 0x7fbb5c0ccc30 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 >> v1:172.21.15.67:6789/0 conn(0x7fbb5c0ccc30 legacy=0x7fbb5c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c003a00 0x7fbb3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3c004800 0x7fbb3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb3c005210 0x7fbb3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c06e3a0 con 0x7fbb3c004800 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c00c160 con 0x7fbb3c003a00 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.802+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c00c360 con 0x7fbb3c005210 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb52a00640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3c004800 0x7fbb3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb5b400640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c003a00 0x7fbb3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb52a00640 1 -- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb3c005210 msgr2=0x7fbb3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb5be00640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb3c005210 0x7fbb3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb52a00640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb3c005210 0x7fbb3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:07.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb52a00640 1 -- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c003a00 msgr2=0x7fbb3c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:07.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb52a00640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c003a00 0x7fbb3c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:07.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb52a00640 1 -- 172.21.15.67:0/889068384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c003520 con 0x7fbb3c004800 2024-05-30T17:07:07.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb5b400640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c003a00 0x7fbb3c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:07.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb5be00640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb3c005210 0x7fbb3c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:07.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.803+0000 7fbb5be00640 1 -- 172.21.15.67:0/889068384 reap_dead start 2024-05-30T17:07:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.804+0000 7fbb52a00640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3c004800 0x7fbb3c004c80 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c0e0 tx=0x7fbb3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6adacdc4dcc37a server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.804+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44007c20 con 0x7fbb3c004800 2024-05-30T17:07:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.804+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb44005af0 con 0x7fbb3c004800 2024-05-30T17:07:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.804+0000 7fbb5aa00640 1 -- 172.21.15.67:0/889068384 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44007070 con 0x7fbb3c004800 2024-05-30T17:07:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.805+0000 7fbb61200640 1 -- 172.21.15.67:0/889068384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3c004800 msgr2=0x7fbb3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.805+0000 7fbb61200640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3c004800 0x7fbb3c004c80 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c0e0 tx=0x7fbb3c005830 comp rx=0 tx=0).stop 2024-05-30T17:07:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.805+0000 7fbb61200640 1 -- 172.21.15.67:0/889068384 shutdown_connections 2024-05-30T17:07:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.805+0000 7fbb61200640 1 --2- 172.21.15.67:0/889068384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3c004800 0x7fbb3c004c80 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:07.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.805+0000 7fbb61200640 1 -- 172.21.15.67:0/889068384 >> 172.21.15.67:0/889068384 conn(0x7fbb5c06bae0 msgr2=0x7fbb5c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:07.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.805+0000 7fbb61200640 1 -- 172.21.15.67:0/889068384 shutdown_connections 2024-05-30T17:07:07.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.805+0000 7fbb61200640 1 -- 172.21.15.67:0/889068384 wait complete. 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.807+0000 7fbb61200640 1 Processor -- start 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.807+0000 7fbb61200640 1 -- start start 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.808+0000 7fbb61200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c004800 0x7fbb5c0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.808+0000 7fbb61200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15b7b0 0x7fbb5c0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.808+0000 7fbb61200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0cdd90 0x7fbb5c0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.808+0000 7fbb61200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0d08b0 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.808+0000 7fbb61200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0cba10 con 0x7fbb3c004800 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.808+0000 7fbb61200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0d0a30 con 0x7fbb5c0cdd90 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.808+0000 7fbb52a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15b7b0 0x7fbb5c0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.808+0000 7fbb52a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15b7b0 0x7fbb5c0c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48422/0 (socket says 172.21.15.67:48422) 2024-05-30T17:07:07.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.808+0000 7fbb52a00640 1 -- 172.21.15.67:0/2554827587 learned_addr learned my addr 172.21.15.67:0/2554827587 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.809+0000 7fbb5be00640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0cdd90 0x7fbb5c0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.809+0000 7fbb5b400640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c004800 0x7fbb5c0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.809+0000 7fbb52a00640 1 -- 172.21.15.67:0/2554827587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0cdd90 msgr2=0x7fbb5c0ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.809+0000 7fbb52a00640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0cdd90 0x7fbb5c0ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.809+0000 7fbb52a00640 1 -- 172.21.15.67:0/2554827587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c004800 msgr2=0x7fbb5c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.809+0000 7fbb52a00640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c004800 0x7fbb5c0b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.809+0000 7fbb52a00640 1 -- 172.21.15.67:0/2554827587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c0c7b60 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.809+0000 7fbb52a00640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15b7b0 0x7fbb5c0c72f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c200 tx=0x7fbb5c0cd650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.810+0000 7fbb53e00640 1 -- 172.21.15.67:0/2554827587 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44042070 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.810+0000 7fbb53e00640 1 -- 172.21.15.67:0/2554827587 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb44001630 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.810+0000 7fbb53e00640 1 -- 172.21.15.67:0/2554827587 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44037040 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.810+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c0c7e40 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.810+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c0c8620 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.811+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5c0689e0 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.812+0000 7fbb53e00640 1 -- 172.21.15.67:0/2554827587 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7fbb44045070 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.813+0000 7fbb53e00640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3806e900 0x7fbb38070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:07.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.813+0000 7fbb53e00640 1 -- 172.21.15.67:0/2554827587 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fbb440aa220 con 0x7fbb5c15b7b0 2024-05-30T17:07:07.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.814+0000 7fbb5b400640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3806e900 0x7fbb38070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:07.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.814+0000 7fbb5b400640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3806e900 0x7fbb38070dc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c006fd0 tx=0x7fbb4c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:07.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:07.819+0000 7fbb53e00640 1 -- 172.21.15.67:0/2554827587 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb44079b00 con 0x7fbb5c15b7b0 2024-05-30T17:07:08.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.098+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbb5c06fc50 con 0x7fbb5c15b7b0 2024-05-30T17:07:08.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.099+0000 7fbb53e00640 1 -- 172.21.15.67:0/2554827587 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2284 (secure 0 0 0) 0x7fbb4407de00 con 0x7fbb5c15b7b0 2024-05-30T17:07:08.100 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:08.100 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:07:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1421, 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:07:08.102 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:07:08.103 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:07:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:08.107 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:08.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3806e900 msgr2=0x7fbb38070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3806e900 0x7fbb38070dc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c006fd0 tx=0x7fbb4c008040 comp rx=0 tx=0).stop 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15b7b0 msgr2=0x7fbb5c0c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15b7b0 0x7fbb5c0c72f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c200 tx=0x7fbb5c0cd650 comp rx=0 tx=0).stop 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 shutdown_connections 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fbb3806e900 0x7fbb38070dc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb5c0cdd90 0x7fbb5c0ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb5c15b7b0 0x7fbb5c0c72f0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 --2- 172.21.15.67:0/2554827587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c004800 0x7fbb5c0b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 >> 172.21.15.67:0/2554827587 conn(0x7fbb5c06bae0 msgr2=0x7fbb5c0cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 shutdown_connections 2024-05-30T17:07:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:08.102+0000 7fbb61200640 1 -- 172.21.15.67:0/2554827587 wait complete. 2024-05-30T17:07:13.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:07:13.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.338+0000 7f7df1400640 1 Processor -- start 2024-05-30T17:07:13.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.338+0000 7f7df1400640 1 -- start start 2024-05-30T17:07:13.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.340+0000 7f7df1400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7dec161df0 0x7f7dec099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:13.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.340+0000 7f7df1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0d0b70 0x7f7dec0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:13.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.340+0000 7f7df1400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dec1621d0 con 0x7f7dec0cc2b0 2024-05-30T17:07:13.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.341+0000 7f7df1400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7dec0cd840 con 0x7f7dec161df0 2024-05-30T17:07:13.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.341+0000 7f7df1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7dec0ccf10 con 0x7f7dec0d0b70 2024-05-30T17:07:13.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.341+0000 7f7debe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7dec161df0 0x7f7dec099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:13.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.341+0000 7f7deb400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7dec0cc2b0 0x7f7dec0d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:51244/0 (socket says 172.21.15.67:51244) 2024-05-30T17:07:13.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.341+0000 7f7debe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7dec161df0 0x7f7dec099800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:43810/0 (socket says 172.21.15.67:43810) 2024-05-30T17:07:13.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.341+0000 7f7deb400640 1 -- 172.21.15.67:0/1436609350 learned_addr learned my addr 172.21.15.67:0/1436609350 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:13.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.341+0000 7f7df0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0d0b70 0x7f7dec0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:13.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.342+0000 7f7debe00640 1 -- 172.21.15.67:0/1436609350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0d0b70 msgr2=0x7f7dec0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:13.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.342+0000 7f7debe00640 1 --2- 172.21.15.67:0/1436609350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0d0b70 0x7f7dec0cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:13.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.342+0000 7f7debe00640 1 -- 172.21.15.67:0/1436609350 >> v1:172.21.15.169:6789/0 conn(0x7f7dec0cc2b0 legacy=0x7f7dec0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:13.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.342+0000 7f7debe00640 1 -- 172.21.15.67:0/1436609350 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec0cd1f0 con 0x7f7dec161df0 2024-05-30T17:07:13.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.343+0000 7f7debe00640 1 --2- 172.21.15.67:0/1436609350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dec161df0 0x7f7dec099800 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0097a0 tx=0x7f7ddc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52a9221771573cae server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:13.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.344+0000 7f7deaa00640 1 -- 172.21.15.67:0/1436609350 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc037070 con 0x7f7dec161df0 2024-05-30T17:07:13.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.344+0000 7f7deaa00640 1 -- 172.21.15.67:0/1436609350 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ddc032440 con 0x7f7dec161df0 2024-05-30T17:07:13.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.344+0000 7f7deaa00640 1 -- 172.21.15.67:0/1436609350 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc032810 con 0x7f7dec161df0 2024-05-30T17:07:13.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.345+0000 7f7df1400640 1 -- 172.21.15.67:0/1436609350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dec161df0 msgr2=0x7f7dec099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:13.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.345+0000 7f7df1400640 1 --2- 172.21.15.67:0/1436609350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dec161df0 0x7f7dec099800 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0097a0 tx=0x7f7ddc02a180 comp rx=0 tx=0).stop 2024-05-30T17:07:13.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.345+0000 7f7df1400640 1 -- 172.21.15.67:0/1436609350 shutdown_connections 2024-05-30T17:07:13.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.345+0000 7f7df1400640 1 --2- 172.21.15.67:0/1436609350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0d0b70 0x7f7dec0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:13.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.345+0000 7f7df1400640 1 --2- 172.21.15.67:0/1436609350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dec161df0 0x7f7dec099800 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:13.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.345+0000 7f7df1400640 1 -- 172.21.15.67:0/1436609350 >> 172.21.15.67:0/1436609350 conn(0x7f7dec06bae0 msgr2=0x7f7dec06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:13.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.345+0000 7f7df1400640 1 -- 172.21.15.67:0/1436609350 shutdown_connections 2024-05-30T17:07:13.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.345+0000 7f7df1400640 1 -- 172.21.15.67:0/1436609350 wait complete. 2024-05-30T17:07:13.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.347+0000 7f7df1400640 1 Processor -- start 2024-05-30T17:07:13.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.348+0000 7f7df1400640 1 -- start start 2024-05-30T17:07:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.349+0000 7f7df1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0cc2b0 0x7f7dec0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.349+0000 7f7df1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dec0d0b70 0x7f7dec15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.349+0000 7f7df1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dec161df0 0x7f7dec0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.349+0000 7f7df1400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec0ccbc0 con 0x7f7dec161df0 2024-05-30T17:07:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.349+0000 7f7df1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec099d40 con 0x7f7dec0d0b70 2024-05-30T17:07:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.349+0000 7f7df1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7dec0ca870 con 0x7f7dec0cc2b0 2024-05-30T17:07:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.349+0000 7f7df0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dec161df0 0x7f7dec0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.349+0000 7f7df0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dec161df0 0x7f7dec0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40560/0 (socket says 172.21.15.67:40560) 2024-05-30T17:07:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.349+0000 7f7df0a00640 1 -- 172.21.15.67:0/411415058 learned_addr learned my addr 172.21.15.67:0/411415058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:13.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.350+0000 7f7deb400640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dec0d0b70 0x7f7dec15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:13.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.350+0000 7f7debe00640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0cc2b0 0x7f7dec0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.350+0000 7f7df0a00640 1 -- 172.21.15.67:0/411415058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0cc2b0 msgr2=0x7f7dec0c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.350+0000 7f7df0a00640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0cc2b0 0x7f7dec0c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.350+0000 7f7df0a00640 1 -- 172.21.15.67:0/411415058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dec0d0b70 msgr2=0x7f7dec15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.350+0000 7f7df0a00640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dec0d0b70 0x7f7dec15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.350+0000 7f7df0a00640 1 -- 172.21.15.67:0/411415058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec0c8fe0 con 0x7f7dec161df0 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.350+0000 7f7df0a00640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dec161df0 0x7f7dec0c8aa0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f7de400efc0 tx=0x7f7de400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.351+0000 7f7de3e00640 1 -- 172.21.15.67:0/411415058 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4018020 con 0x7f7dec161df0 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.351+0000 7f7de3e00640 1 -- 172.21.15.67:0/411415058 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4014480 con 0x7f7dec161df0 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.351+0000 7f7de3e00640 1 -- 172.21.15.67:0/411415058 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4014870 con 0x7f7dec161df0 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.351+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec0c9280 con 0x7f7dec161df0 2024-05-30T17:07:13.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.351+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec0b00b0 con 0x7f7dec161df0 2024-05-30T17:07:13.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.352+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec0689e0 con 0x7f7dec161df0 2024-05-30T17:07:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.353+0000 7f7de3e00640 1 -- 172.21.15.67:0/411415058 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7f7de4005d10 con 0x7f7dec161df0 2024-05-30T17:07:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.354+0000 7f7de3e00640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7dcc06e900 0x7f7dcc070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:13.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.354+0000 7f7de3e00640 1 -- 172.21.15.67:0/411415058 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f7de408bb10 con 0x7f7dec161df0 2024-05-30T17:07:13.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.355+0000 7f7debe00640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7dcc06e900 0x7f7dcc070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:13.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.355+0000 7f7debe00640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7dcc06e900 0x7f7dcc070dc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc034000 tx=0x7f7ddc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:13.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.359+0000 7f7de3e00640 1 -- 172.21.15.67:0/411415058 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7de4022030 con 0x7f7dec161df0 2024-05-30T17:07:13.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.638+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7dec099800 con 0x7f7dec161df0 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.639+0000 7f7de3e00640 1 -- 172.21.15.67:0/411415058 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2284 (secure 0 0 0) 0x7f7de4010040 con 0x7f7dec161df0 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:13.640 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1427, 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:07:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:07:13.642 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:13.643 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:13.644 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:07:13.645 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:13.646 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:13.647 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:13.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7dcc06e900 msgr2=0x7f7dcc070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:13.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7dcc06e900 0x7f7dcc070dc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc034000 tx=0x7f7ddc02e000 comp rx=0 tx=0).stop 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dec161df0 msgr2=0x7f7dec0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dec161df0 0x7f7dec0c8aa0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f7de400efc0 tx=0x7f7de400c490 comp rx=0 tx=0).stop 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 shutdown_connections 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7dcc06e900 0x7f7dcc070dc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dec161df0 0x7f7dec0c8aa0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dec0d0b70 0x7f7dec15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 --2- 172.21.15.67:0/411415058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dec0cc2b0 0x7f7dec0c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 >> 172.21.15.67:0/411415058 conn(0x7f7dec06bae0 msgr2=0x7f7dec0cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 shutdown_connections 2024-05-30T17:07:13.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:13.642+0000 7f7df1400640 1 -- 172.21.15.67:0/411415058 wait complete. 2024-05-30T17:07:16.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:07:16.873 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:07:18.663 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:07:18.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.883+0000 7ffb07e00640 1 Processor -- start 2024-05-30T17:07:18.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.883+0000 7ffb07e00640 1 -- start start 2024-05-30T17:07:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.884+0000 7ffb07e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0815b380 0x7ffb0815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.884+0000 7ffb07e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb080d0460 0x7ffb080d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.884+0000 7ffb07e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb0815fe50 con 0x7ffb080ce800 2024-05-30T17:07:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb07e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ffb080b3310 con 0x7ffb080d0460 2024-05-30T17:07:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb07e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffb080cec60 con 0x7ffb0815b380 2024-05-30T17:07:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb07400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb080d0460 0x7ffb080d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb07400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb080d0460 0x7ffb080d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40566/0 (socket says 172.21.15.67:40566) 2024-05-30T17:07:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb07400640 1 -- 172.21.15.67:0/2177783312 learned_addr learned my addr 172.21.15.67:0/2177783312 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb06a00640 1 --2- 172.21.15.67:0/2177783312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0815b380 0x7ffb0815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb07400640 1 -- 172.21.15.67:0/2177783312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0815b380 msgr2=0x7ffb0815ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:18.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb07400640 1 --2- 172.21.15.67:0/2177783312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0815b380 0x7ffb0815ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:18.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb07400640 1 -- 172.21.15.67:0/2177783312 >> v1:172.21.15.169:6789/0 conn(0x7ffb080ce800 legacy=0x7ffb080ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:18.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.885+0000 7ffb07400640 1 -- 172.21.15.67:0/2177783312 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0815f5b0 con 0x7ffb080d0460 2024-05-30T17:07:18.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.886+0000 7ffb07400640 1 --2- 172.21.15.67:0/2177783312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb080d0460 0x7ffb080d08f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000d150 tx=0x7ffb0002c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d73b46a98c702ec server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:18.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.887+0000 7ffb06000640 1 -- 172.21.15.67:0/2177783312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00009600 con 0x7ffb080d0460 2024-05-30T17:07:18.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.887+0000 7ffb06000640 1 -- 172.21.15.67:0/2177783312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb000097a0 con 0x7ffb080d0460 2024-05-30T17:07:18.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.887+0000 7ffb06000640 1 -- 172.21.15.67:0/2177783312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00009b70 con 0x7ffb080d0460 2024-05-30T17:07:18.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.887+0000 7ffb07e00640 1 -- 172.21.15.67:0/2177783312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb080d0460 msgr2=0x7ffb080d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:18.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.887+0000 7ffb07e00640 1 --2- 172.21.15.67:0/2177783312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb080d0460 0x7ffb080d08f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000d150 tx=0x7ffb0002c7c0 comp rx=0 tx=0).stop 2024-05-30T17:07:18.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.888+0000 7ffb07e00640 1 -- 172.21.15.67:0/2177783312 shutdown_connections 2024-05-30T17:07:18.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.888+0000 7ffb07e00640 1 --2- 172.21.15.67:0/2177783312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb080d0460 0x7ffb080d08f0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:18.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.888+0000 7ffb07e00640 1 --2- 172.21.15.67:0/2177783312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0815b380 0x7ffb0815ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:18.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.888+0000 7ffb07e00640 1 -- 172.21.15.67:0/2177783312 >> 172.21.15.67:0/2177783312 conn(0x7ffb0806bae0 msgr2=0x7ffb0806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:18.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.888+0000 7ffb07e00640 1 -- 172.21.15.67:0/2177783312 shutdown_connections 2024-05-30T17:07:18.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.888+0000 7ffb07e00640 1 -- 172.21.15.67:0/2177783312 wait complete. 2024-05-30T17:07:18.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.890+0000 7ffb07e00640 1 Processor -- start 2024-05-30T17:07:18.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.890+0000 7ffb07e00640 1 -- start start 2024-05-30T17:07:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.891+0000 7ffb07e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb080ce800 0x7ffb0806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.891+0000 7ffb07e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb080d0460 0x7ffb0815a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.891+0000 7ffb07e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb0815b380 0x7ffb0815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.891+0000 7ffb07e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0815b760 con 0x7ffb0815b380 2024-05-30T17:07:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.891+0000 7ffb07e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0815fb70 con 0x7ffb080d0460 2024-05-30T17:07:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.891+0000 7ffb07e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0815b8e0 con 0x7ffb080ce800 2024-05-30T17:07:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.891+0000 7ffb07400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb0815b380 0x7ffb0815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.891+0000 7ffb07400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb0815b380 0x7ffb0815c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40570/0 (socket says 172.21.15.67:40570) 2024-05-30T17:07:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.891+0000 7ffb07400640 1 -- 172.21.15.67:0/4011307485 learned_addr learned my addr 172.21.15.67:0/4011307485 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.892+0000 7ffb06a00640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb080ce800 0x7ffb0806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.892+0000 7ffafe000640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb080d0460 0x7ffb0815a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.892+0000 7ffb07400640 1 -- 172.21.15.67:0/4011307485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb080ce800 msgr2=0x7ffb0806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.892+0000 7ffb07400640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb080ce800 0x7ffb0806dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.892+0000 7ffb07400640 1 -- 172.21.15.67:0/4011307485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb080d0460 msgr2=0x7ffb0815a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.892+0000 7ffb07400640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb080d0460 0x7ffb0815a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.892+0000 7ffb07400640 1 -- 172.21.15.67:0/4011307485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0815cf70 con 0x7ffb0815b380 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.892+0000 7ffafe000640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb080d0460 0x7ffb0815a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.892+0000 7ffb07400640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb0815b380 0x7ffb0815c6b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000d270 tx=0x7ffb00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.893+0000 7ffaff400640 1 -- 172.21.15.67:0/4011307485 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00042020 con 0x7ffb0815b380 2024-05-30T17:07:18.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.893+0000 7ffaff400640 1 -- 172.21.15.67:0/4011307485 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb00034bb0 con 0x7ffb0815b380 2024-05-30T17:07:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.893+0000 7ffaff400640 1 -- 172.21.15.67:0/4011307485 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00009600 con 0x7ffb0815b380 2024-05-30T17:07:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.893+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb080afe60 con 0x7ffb0815b380 2024-05-30T17:07:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.893+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0815cbf0 con 0x7ffb0815b380 2024-05-30T17:07:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.894+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb080689e0 con 0x7ffb0815b380 2024-05-30T17:07:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.895+0000 7ffaff400640 1 -- 172.21.15.67:0/4011307485 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7ffb00045020 con 0x7ffb0815b380 2024-05-30T17:07:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.896+0000 7ffaff400640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffae406e900 0x7ffae4070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.896+0000 7ffaff400640 1 -- 172.21.15.67:0/4011307485 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7ffb000abfe0 con 0x7ffb0815b380 2024-05-30T17:07:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.897+0000 7ffb06a00640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffae406e900 0x7ffae4070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:18.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.898+0000 7ffb06a00640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffae406e900 0x7ffae4070dc0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8006fd0 tx=0x7ffaf8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:18.901+0000 7ffaff400640 1 -- 172.21.15.67:0/4011307485 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb0004c030 con 0x7ffb0815b380 2024-05-30T17:07:19.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.185+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffb080b4fe0 con 0x7ffb0815b380 2024-05-30T17:07:19.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.186+0000 7ffaff400640 1 -- 172.21.15.67:0/4011307485 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2284 (secure 0 0 0) 0x7ffb0007be70 con 0x7ffb0815b380 2024-05-30T17:07:19.187 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:19.187 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:19.187 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:07:19.187 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:07:19.187 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:07:19.187 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:07:19.187 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:07:19.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1432, 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:07:19.188 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:07:19.189 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:19.190 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:19.191 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:07:19.192 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:19.193 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:19.194 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:19.194 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffae406e900 msgr2=0x7ffae4070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffae406e900 0x7ffae4070dc0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8006fd0 tx=0x7ffaf8008040 comp rx=0 tx=0).stop 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb0815b380 msgr2=0x7ffb0815c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb0815b380 0x7ffb0815c6b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000d270 tx=0x7ffb00004a40 comp rx=0 tx=0).stop 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 shutdown_connections 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ffae406e900 0x7ffae4070dc0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb0815b380 0x7ffb0815c6b0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb080d0460 0x7ffb0815a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 --2- 172.21.15.67:0/4011307485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb080ce800 0x7ffb0806dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:19.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 >> 172.21.15.67:0/4011307485 conn(0x7ffb0806bae0 msgr2=0x7ffb080ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 shutdown_connections 2024-05-30T17:07:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:19.189+0000 7ffb07e00640 1 -- 172.21.15.67:0/4011307485 wait complete. 2024-05-30T17:07:24.210 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:07:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.428+0000 7ff51f200640 1 Processor -- start 2024-05-30T17:07:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.428+0000 7ff51f200640 1 -- start start 2024-05-30T17:07:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.430+0000 7ff51f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff5200d0460 0x7ff5200d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.430+0000 7ff51f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5200cd1a0 con 0x7ff52015c6b0 2024-05-30T17:07:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.430+0000 7ff51f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5200cd480 con 0x7ff5200ccc30 2024-05-30T17:07:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.430+0000 7ff51f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7ff52006e3a0 con 0x7ff5200d0460 2024-05-30T17:07:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.430+0000 7ff51de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff52015c6b0 0x7ff5200b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60470/0 (socket says 172.21.15.67:60470) 2024-05-30T17:07:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.430+0000 7ff51de00640 1 -- 172.21.15.67:0/4031442931 learned_addr learned my addr 172.21.15.67:0/4031442931 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.430+0000 7ff51e800640 1 --2- 172.21.15.67:0/4031442931 >> v2:172.21.15.67:3300/0 conn(0x7ff5200d0460 0x7ff5200d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:24.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.431+0000 7ff51e800640 1 -- 172.21.15.67:0/4031442931 >> v1:172.21.15.169:6789/0 conn(0x7ff5200ccc30 legacy=0x7ff52006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:24.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.431+0000 7ff51e800640 1 -- 172.21.15.67:0/4031442931 >> v1:172.21.15.67:6789/0 conn(0x7ff52015c6b0 legacy=0x7ff5200b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:24.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.431+0000 7ff51e800640 1 -- 172.21.15.67:0/4031442931 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5200cd760 con 0x7ff5200d0460 2024-05-30T17:07:24.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.432+0000 7ff51e800640 1 --2- 172.21.15.67:0/4031442931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 0x7ff5200d08b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff518009dc0 tx=0x7ff51802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67473b95ce166be5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:24.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.432+0000 7ff51d400640 1 -- 172.21.15.67:0/4031442931 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518007740 con 0x7ff5200d0460 2024-05-30T17:07:24.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.433+0000 7ff51d400640 1 -- 172.21.15.67:0/4031442931 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5180078e0 con 0x7ff5200d0460 2024-05-30T17:07:24.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.433+0000 7ff51d400640 1 -- 172.21.15.67:0/4031442931 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518007cb0 con 0x7ff5200d0460 2024-05-30T17:07:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.433+0000 7ff51f200640 1 -- 172.21.15.67:0/4031442931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 msgr2=0x7ff5200d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.433+0000 7ff51f200640 1 --2- 172.21.15.67:0/4031442931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 0x7ff5200d08b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff518009dc0 tx=0x7ff51802a730 comp rx=0 tx=0).stop 2024-05-30T17:07:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.433+0000 7ff51f200640 1 -- 172.21.15.67:0/4031442931 shutdown_connections 2024-05-30T17:07:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.433+0000 7ff51f200640 1 --2- 172.21.15.67:0/4031442931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 0x7ff5200d08b0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.433+0000 7ff51f200640 1 -- 172.21.15.67:0/4031442931 >> 172.21.15.67:0/4031442931 conn(0x7ff52006bae0 msgr2=0x7ff52006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.433+0000 7ff51f200640 1 -- 172.21.15.67:0/4031442931 shutdown_connections 2024-05-30T17:07:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.433+0000 7ff51f200640 1 -- 172.21.15.67:0/4031442931 wait complete. 2024-05-30T17:07:24.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.435+0000 7ff51f200640 1 Processor -- start 2024-05-30T17:07:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.435+0000 7ff51f200640 1 -- start start 2024-05-30T17:07:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.436+0000 7ff51f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5200ccc30 0x7ff520156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.436+0000 7ff51f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 0x7ff5200c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.436+0000 7ff51f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff52015c6b0 0x7ff5200b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.436+0000 7ff51f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff5200cba10 con 0x7ff5200d0460 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.436+0000 7ff51f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5200d0df0 con 0x7ff5200ccc30 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.436+0000 7ff51f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff5200cbf30 con 0x7ff52015c6b0 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff517e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 0x7ff5200c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff517e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 0x7ff5200c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37030/0 (socket says 172.21.15.67:37030) 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff517e00640 1 -- 172.21.15.67:0/2452812399 learned_addr learned my addr 172.21.15.67:0/2452812399 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff51de00640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5200ccc30 0x7ff520156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff51e800640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff52015c6b0 0x7ff5200b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff517e00640 1 -- 172.21.15.67:0/2452812399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff52015c6b0 msgr2=0x7ff5200b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff517e00640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff52015c6b0 0x7ff5200b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff517e00640 1 -- 172.21.15.67:0/2452812399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5200ccc30 msgr2=0x7ff520156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:24.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff517e00640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5200ccc30 0x7ff520156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:24.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.437+0000 7ff517e00640 1 -- 172.21.15.67:0/2452812399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff520156640 con 0x7ff5200d0460 2024-05-30T17:07:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.438+0000 7ff517e00640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 0x7ff5200c7d00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c400 tx=0x7ff50800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.438+0000 7ff516000640 1 -- 172.21.15.67:0/2452812399 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5080052d0 con 0x7ff5200d0460 2024-05-30T17:07:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.438+0000 7ff516000640 1 -- 172.21.15.67:0/2452812399 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff508005490 con 0x7ff5200d0460 2024-05-30T17:07:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.438+0000 7ff516000640 1 -- 172.21.15.67:0/2452812399 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508007260 con 0x7ff5200d0460 2024-05-30T17:07:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.438+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5200afdc0 con 0x7ff5200d0460 2024-05-30T17:07:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.438+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5200b0380 con 0x7ff5200d0460 2024-05-30T17:07:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.439+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5200689e0 con 0x7ff5200d0460 2024-05-30T17:07:24.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.440+0000 7ff516000640 1 -- 172.21.15.67:0/2452812399 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7ff508007400 con 0x7ff5200d0460 2024-05-30T17:07:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.441+0000 7ff516000640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4fc06e930 0x7ff4fc070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.442+0000 7ff516000640 1 -- 172.21.15.67:0/2452812399 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7ff50808a150 con 0x7ff5200d0460 2024-05-30T17:07:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.442+0000 7ff51de00640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4fc06e930 0x7ff4fc070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.442+0000 7ff51de00640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4fc06e930 0x7ff4fc070df0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff510005880 tx=0x7ff510009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.446+0000 7ff516000640 1 -- 172.21.15.67:0/2452812399 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff520156640 con 0x7ff5200d0460 2024-05-30T17:07:24.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.731+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff52006dfb0 con 0x7ff5200d0460 2024-05-30T17:07:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.732+0000 7ff516000640 1 -- 172.21.15.67:0/2452812399 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2284 (secure 0 0 0) 0x7ff50805ee20 con 0x7ff5200d0460 2024-05-30T17:07:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:24.732 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:07:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:07:24.732 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:07:24.733 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1438, 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:07:24.734 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:07:24.735 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:07:24.736 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:24.737 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:07:24.738 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:07:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:07:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:24.739 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:24.739 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:24.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4fc06e930 msgr2=0x7ff4fc070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:24.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4fc06e930 0x7ff4fc070df0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff510005880 tx=0x7ff510009040 comp rx=0 tx=0).stop 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 msgr2=0x7ff5200c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 0x7ff5200c7d00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c400 tx=0x7ff50800c8c0 comp rx=0 tx=0).stop 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 shutdown_connections 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff4fc06e930 0x7ff4fc070df0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff52015c6b0 0x7ff5200b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5200d0460 0x7ff5200c7d00 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 --2- 172.21.15.67:0/2452812399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5200ccc30 0x7ff520156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.734+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 >> 172.21.15.67:0/2452812399 conn(0x7ff52006bae0 msgr2=0x7ff52006bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.735+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 shutdown_connections 2024-05-30T17:07:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:24.735+0000 7ff51f200640 1 -- 172.21.15.67:0/2452812399 wait complete. 2024-05-30T17:07:29.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:07:29.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.969+0000 7f2393e00640 1 Processor -- start 2024-05-30T17:07:29.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.970+0000 7f2393e00640 1 -- start start 2024-05-30T17:07:29.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.971+0000 7f2393e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2394161df0 0x7f2394099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:29.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.971+0000 7f2393e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f23940d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:29.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.971+0000 7f2393e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f23940d09a0 0x7f23940c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:29.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.971+0000 7f2393e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f23940cd840 con 0x7f23940d09a0 2024-05-30T17:07:29.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.971+0000 7f2393e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f23941621d0 con 0x7f2394161df0 2024-05-30T17:07:29.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.971+0000 7f2393e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f23940ccf10 con 0x7f23940cc2b0 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.971+0000 7f2393400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f23940d09a0 0x7f23940c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f2393400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f23940d09a0 0x7f23940c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37046/0 (socket says 172.21.15.67:37046) 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f2393400640 1 -- 172.21.15.67:0/2126207714 learned_addr learned my addr 172.21.15.67:0/2126207714 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f238be00640 1 --2- 172.21.15.67:0/2126207714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f23940d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f2393400640 1 -- 172.21.15.67:0/2126207714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 msgr2=0x7f23940d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f2392a00640 1 --2- 172.21.15.67:0/2126207714 >> v2:172.21.15.169:3300/0 conn(0x7f2394161df0 0x7f2394099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f2393400640 1 --2- 172.21.15.67:0/2126207714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f23940d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f2393400640 1 -- 172.21.15.67:0/2126207714 >> v2:172.21.15.169:3300/0 conn(0x7f2394161df0 msgr2=0x7f2394099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f2393400640 1 --2- 172.21.15.67:0/2126207714 >> v2:172.21.15.169:3300/0 conn(0x7f2394161df0 0x7f2394099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f2393400640 1 -- 172.21.15.67:0/2126207714 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23940cd1f0 con 0x7f23940d09a0 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f2392a00640 1 --2- 172.21.15.67:0/2126207714 >> v2:172.21.15.169:3300/0 conn(0x7f2394161df0 0x7f2394099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.972+0000 7f238be00640 1 --2- 172.21.15.67:0/2126207714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f23940d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:29.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.973+0000 7f2393400640 1 --2- 172.21.15.67:0/2126207714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 0x7f23940c50c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f23940b3770 tx=0x7f238c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c532fca218289df server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:29.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2392000640 1 -- 172.21.15.67:0/2126207714 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c036070 con 0x7f23940d09a0 2024-05-30T17:07:29.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2392000640 1 -- 172.21.15.67:0/2126207714 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f238c031600 con 0x7f23940d09a0 2024-05-30T17:07:29.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2392000640 1 -- 172.21.15.67:0/2126207714 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c0319d0 con 0x7f23940d09a0 2024-05-30T17:07:29.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2393e00640 1 -- 172.21.15.67:0/2126207714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 msgr2=0x7f23940c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:29.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2393e00640 1 --2- 172.21.15.67:0/2126207714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 0x7f23940c50c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f23940b3770 tx=0x7f238c02a6a0 comp rx=0 tx=0).stop 2024-05-30T17:07:29.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2393e00640 1 -- 172.21.15.67:0/2126207714 shutdown_connections 2024-05-30T17:07:29.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2393e00640 1 --2- 172.21.15.67:0/2126207714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 0x7f23940c50c0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:29.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2393e00640 1 --2- 172.21.15.67:0/2126207714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f23940d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:29.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2393e00640 1 --2- 172.21.15.67:0/2126207714 >> v2:172.21.15.169:3300/0 conn(0x7f2394161df0 0x7f2394099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:29.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2393e00640 1 -- 172.21.15.67:0/2126207714 >> 172.21.15.67:0/2126207714 conn(0x7f239406bae0 msgr2=0x7f239406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:29.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2393e00640 1 -- 172.21.15.67:0/2126207714 shutdown_connections 2024-05-30T17:07:29.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.974+0000 7f2393e00640 1 -- 172.21.15.67:0/2126207714 wait complete. 2024-05-30T17:07:29.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.976+0000 7f2393e00640 1 Processor -- start 2024-05-30T17:07:29.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.976+0000 7f2393e00640 1 -- start start 2024-05-30T17:07:29.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.977+0000 7f2393e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f239415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:29.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.977+0000 7f2393e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 0x7f2394152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:29.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.977+0000 7f2393e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2394161df0 0x7f23940c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:29.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.977+0000 7f2393e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f23940ccbc0 con 0x7f23940d09a0 2024-05-30T17:07:29.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.977+0000 7f2393e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2394099d40 con 0x7f2394161df0 2024-05-30T17:07:29.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.977+0000 7f2393e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f23940ca870 con 0x7f23940cc2b0 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f238be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 0x7f2394152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f238be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 0x7f2394152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37060/0 (socket says 172.21.15.67:37060) 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f238be00640 1 -- 172.21.15.67:0/599844618 learned_addr learned my addr 172.21.15.67:0/599844618 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f2393400640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2394161df0 0x7f23940c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f2392a00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f239415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f238be00640 1 -- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 msgr2=0x7f239415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f238be00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f239415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f238be00640 1 -- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2394161df0 msgr2=0x7f23940c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f238be00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2394161df0 0x7f23940c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f238be00640 1 -- 172.21.15.67:0/599844618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23940b19e0 con 0x7f23940d09a0 2024-05-30T17:07:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f2393400640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2394161df0 0x7f23940c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.978+0000 7f2392a00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f239415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.979+0000 7f238be00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 0x7f2394152740 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f237c00c7c0 tx=0x7f237c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.979+0000 7f238aa00640 1 -- 172.21.15.67:0/599844618 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c015070 con 0x7f23940d09a0 2024-05-30T17:07:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.979+0000 7f238aa00640 1 -- 172.21.15.67:0/599844618 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237c007a30 con 0x7f23940d09a0 2024-05-30T17:07:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.979+0000 7f238aa00640 1 -- 172.21.15.67:0/599844618 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c007e00 con 0x7f23940d09a0 2024-05-30T17:07:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.979+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23940b1140 con 0x7f23940d09a0 2024-05-30T17:07:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.979+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23940b1700 con 0x7f23940d09a0 2024-05-30T17:07:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.980+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23940689e0 con 0x7f23940d09a0 2024-05-30T17:07:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.981+0000 7f238aa00640 1 -- 172.21.15.67:0/599844618 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 50) ==== 93947+0+0 (secure 0 0 0) 0x7f237c018070 con 0x7f23940d09a0 2024-05-30T17:07:29.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.983+0000 7f238aa00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f237006e930 0x7f2370070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:29.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.983+0000 7f238aa00640 1 -- 172.21.15.67:0/599844618 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f237c08ab20 con 0x7f23940d09a0 2024-05-30T17:07:29.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.983+0000 7f2392a00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f237006e930 0x7f2370070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:29.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.984+0000 7f2392a00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f237006e930 0x7f2370070df0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2384006fd0 tx=0x7f2384008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:29.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:29.988+0000 7f238aa00640 1 -- 172.21.15.67:0/599844618 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f237c05a9b0 con 0x7f23940d09a0 2024-05-30T17:07:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.270+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f23940af410 con 0x7f23940d09a0 2024-05-30T17:07:30.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.271+0000 7f238aa00640 1 -- 172.21.15.67:0/599844618 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2284 (secure 0 0 0) 0x7f237c05a2b0 con 0x7f23940d09a0 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:07:30.272 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1443, 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:07:30.273 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:30.274 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016847360, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077265920, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:07:30.275 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:30.276 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:07:30.277 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:07:30.278 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:30.279 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:30.279 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:30.279 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:30.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f237006e930 msgr2=0x7f2370070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f237006e930 0x7f2370070df0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2384006fd0 tx=0x7f2384008040 comp rx=0 tx=0).stop 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 msgr2=0x7f2394152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 0x7f2394152740 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f237c00c7c0 tx=0x7f237c00cc80 comp rx=0 tx=0).stop 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 shutdown_connections 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f237006e930 0x7f2370070df0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2394161df0 0x7f23940c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23940d09a0 0x7f2394152740 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 --2- 172.21.15.67:0/599844618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23940cc2b0 0x7f239415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 >> 172.21.15.67:0/599844618 conn(0x7f239406bae0 msgr2=0x7f23940b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 shutdown_connections 2024-05-30T17:07:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:30.274+0000 7f2393e00640 1 -- 172.21.15.67:0/599844618 wait complete. 2024-05-30T17:07:35.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:07:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.514+0000 7f9bfd800640 1 Processor -- start 2024-05-30T17:07:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.514+0000 7f9bfd800640 1 -- start start 2024-05-30T17:07:35.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.515+0000 7f9bfd800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9bf8161df0 0x7f9bf8099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:35.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.515+0000 7f9bfd800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf81621d0 con 0x7f9bf80cc2b0 2024-05-30T17:07:35.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.515+0000 7f9bfd800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf80ccf10 con 0x7f9bf80d0b70 2024-05-30T17:07:35.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.515+0000 7f9bfd800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf80cd840 con 0x7f9bf8161df0 2024-05-30T17:07:35.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.516+0000 7f9befe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9bf80cc2b0 0x7f9bf80d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44464/0 (socket says 172.21.15.67:44464) 2024-05-30T17:07:35.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.516+0000 7f9befe00640 1 -- 172.21.15.67:0/1439101988 learned_addr learned my addr 172.21.15.67:0/1439101988 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:35.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.516+0000 7f9bf7e00640 1 --2- 172.21.15.67:0/1439101988 >> v2:172.21.15.67:3300/0 conn(0x7f9bf8161df0 0x7f9bf8099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:35.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.516+0000 7f9bf7e00640 1 -- 172.21.15.67:0/1439101988 >> v1:172.21.15.169:6789/0 conn(0x7f9bf80d0b70 legacy=0x7f9bf80cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:35.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.516+0000 7f9bf7e00640 1 -- 172.21.15.67:0/1439101988 >> v1:172.21.15.67:6789/0 conn(0x7f9bf80cc2b0 legacy=0x7f9bf80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:35.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.516+0000 7f9bf7e00640 1 -- 172.21.15.67:0/1439101988 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf80cd1f0 con 0x7f9bf8161df0 2024-05-30T17:07:35.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.518+0000 7f9bf7e00640 1 --2- 172.21.15.67:0/1439101988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf8161df0 0x7f9bf8099800 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f9be80097a0 tx=0x7f9be802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977139e7375e9a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:35.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.518+0000 7f9bf7400640 1 -- 172.21.15.67:0/1439101988 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8037070 con 0x7f9bf8161df0 2024-05-30T17:07:35.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.518+0000 7f9bf7400640 1 -- 172.21.15.67:0/1439101988 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be8032440 con 0x7f9bf8161df0 2024-05-30T17:07:35.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.518+0000 7f9bf7400640 1 -- 172.21.15.67:0/1439101988 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8032810 con 0x7f9bf8161df0 2024-05-30T17:07:35.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.519+0000 7f9bfd800640 1 -- 172.21.15.67:0/1439101988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf8161df0 msgr2=0x7f9bf8099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:35.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.519+0000 7f9bfd800640 1 --2- 172.21.15.67:0/1439101988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf8161df0 0x7f9bf8099800 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f9be80097a0 tx=0x7f9be802a1e0 comp rx=0 tx=0).stop 2024-05-30T17:07:35.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.519+0000 7f9bfd800640 1 -- 172.21.15.67:0/1439101988 shutdown_connections 2024-05-30T17:07:35.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.519+0000 7f9bfd800640 1 --2- 172.21.15.67:0/1439101988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf8161df0 0x7f9bf8099800 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:35.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.519+0000 7f9bfd800640 1 -- 172.21.15.67:0/1439101988 >> 172.21.15.67:0/1439101988 conn(0x7f9bf806bae0 msgr2=0x7f9bf806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:35.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.519+0000 7f9bfd800640 1 -- 172.21.15.67:0/1439101988 shutdown_connections 2024-05-30T17:07:35.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.519+0000 7f9bfd800640 1 -- 172.21.15.67:0/1439101988 wait complete. 2024-05-30T17:07:35.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.521+0000 7f9bfd800640 1 Processor -- start 2024-05-30T17:07:35.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.521+0000 7f9bfd800640 1 -- start start 2024-05-30T17:07:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.522+0000 7f9bfd800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf80cc2b0 0x7f9bf80ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.522+0000 7f9bfd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf80d0b70 0x7f9bf80c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.522+0000 7f9bfd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf8161df0 0x7f9bf814e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.522+0000 7f9bfd800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf80ccbc0 con 0x7f9bf80d0b70 2024-05-30T17:07:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.522+0000 7f9bfd800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8099d40 con 0x7f9bf80cc2b0 2024-05-30T17:07:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.522+0000 7f9bfd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf80ca870 con 0x7f9bf8161df0 2024-05-30T17:07:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.522+0000 7f9befe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf80d0b70 0x7f9bf80c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.522+0000 7f9befe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf80d0b70 0x7f9bf80c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57538/0 (socket says 172.21.15.67:57538) 2024-05-30T17:07:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.522+0000 7f9befe00640 1 -- 172.21.15.67:0/1628221290 learned_addr learned my addr 172.21.15.67:0/1628221290 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9bf7e00640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf80cc2b0 0x7f9bf80ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9bfce00640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf8161df0 0x7f9bf814e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9befe00640 1 -- 172.21.15.67:0/1628221290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf8161df0 msgr2=0x7f9bf814e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9befe00640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf8161df0 0x7f9bf814e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9befe00640 1 -- 172.21.15.67:0/1628221290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf80cc2b0 msgr2=0x7f9bf80ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9befe00640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf80cc2b0 0x7f9bf80ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9befe00640 1 -- 172.21.15.67:0/1628221290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf8151fc0 con 0x7f9bf80d0b70 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9bf7e00640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf80cc2b0 0x7f9bf80ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9befe00640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf80d0b70 0x7f9bf80c68c0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c3c0 tx=0x7f9be000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.523+0000 7f9bf4c00640 1 -- 172.21.15.67:0/1628221290 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0017070 con 0x7f9bf80d0b70 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.524+0000 7f9bf4c00640 1 -- 172.21.15.67:0/1628221290 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be0005ce0 con 0x7f9bf80d0b70 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.524+0000 7f9bf4c00640 1 -- 172.21.15.67:0/1628221290 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be00072f0 con 0x7f9bf80d0b70 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.524+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf80c9550 con 0x7f9bf80d0b70 2024-05-30T17:07:35.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.524+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf80c9b10 con 0x7f9bf80d0b70 2024-05-30T17:07:35.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.525+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf80689e0 con 0x7f9bf80d0b70 2024-05-30T17:07:35.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.526+0000 7f9bf4c00640 1 -- 172.21.15.67:0/1628221290 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f9be001a070 con 0x7f9bf80d0b70 2024-05-30T17:07:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.527+0000 7f9bf4c00640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd406e540 0x7f9bd4070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.527+0000 7f9bf4c00640 1 -- 172.21.15.67:0/1628221290 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f9be0089d60 con 0x7f9bf80d0b70 2024-05-30T17:07:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.527+0000 7f9bf7e00640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd406e540 0x7f9bd4070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:35.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.528+0000 7f9bf7e00640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd406e540 0x7f9bd4070a00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9be8034000 tx=0x7f9be802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:35.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.532+0000 7f9bf4c00640 1 -- 172.21.15.67:0/1628221290 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bf81621d0 con 0x7f9bf80d0b70 2024-05-30T17:07:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.818+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9bf80b4c90 con 0x7f9bf80d0b70 2024-05-30T17:07:35.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.819+0000 7f9bf4c00640 1 -- 172.21.15.67:0/1628221290 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2375 (secure 0 0 0) 0x7f9be0059c90 con 0x7f9bf80d0b70 2024-05-30T17:07:35.820 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:35.820 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:35.821 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1449, 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:07:35.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:35.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915087050, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43016855552, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730077257728, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 0, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 0 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:07:35.824 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:07:35.825 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:35.826 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:35.827 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd406e540 msgr2=0x7f9bd4070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd406e540 0x7f9bd4070a00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9be8034000 tx=0x7f9be802e000 comp rx=0 tx=0).stop 2024-05-30T17:07:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf80d0b70 msgr2=0x7f9bf80c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf80d0b70 0x7f9bf80c68c0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c3c0 tx=0x7f9be000c880 comp rx=0 tx=0).stop 2024-05-30T17:07:35.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 shutdown_connections 2024-05-30T17:07:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9bd406e540 0x7f9bd4070a00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf8161df0 0x7f9bf814e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bf80d0b70 0x7f9bf80c68c0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 --2- 172.21.15.67:0/1628221290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf80cc2b0 0x7f9bf80ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 >> 172.21.15.67:0/1628221290 conn(0x7f9bf806bae0 msgr2=0x7f9bf80ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 shutdown_connections 2024-05-30T17:07:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:35.822+0000 7f9bfd800640 1 -- 172.21.15.67:0/1628221290 wait complete. 2024-05-30T17:07:40.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-30T17:07:41.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.063+0000 7fb3f5600640 1 Processor -- start 2024-05-30T17:07:41.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.064+0000 7fb3f5600640 1 -- start start 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.066+0000 7fb3f5600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb3f009b140 0x7fb3f00c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.066+0000 7fb3f5600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3f015ff30 0x7fb3f00c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.066+0000 7fb3f5600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3f00cdf50 con 0x7fb3f015fb50 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.066+0000 7fb3f5600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f00cead0 con 0x7fb3f009b140 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.066+0000 7fb3f5600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f00ce230 con 0x7fb3f015ff30 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.066+0000 7fb3e7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb3f009b140 0x7fb3f00c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.066+0000 7fb3e7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb3f009b140 0x7fb3f00c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49174/0 (socket says 172.21.15.67:49174) 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.066+0000 7fb3e7e00640 1 -- 172.21.15.67:0/3674286953 learned_addr learned my addr 172.21.15.67:0/3674286953 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.066+0000 7fb3f4c00640 1 --2- 172.21.15.67:0/3674286953 >> v2:172.21.15.169:3300/0 conn(0x7fb3f015ff30 0x7fb3f00c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.067+0000 7fb3e7e00640 1 -- 172.21.15.67:0/3674286953 >> v2:172.21.15.169:3300/0 conn(0x7fb3f015ff30 msgr2=0x7fb3f00c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:41.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.067+0000 7fb3e7e00640 1 --2- 172.21.15.67:0/3674286953 >> v2:172.21.15.169:3300/0 conn(0x7fb3f015ff30 0x7fb3f00c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.067+0000 7fb3e7e00640 1 -- 172.21.15.67:0/3674286953 >> v1:172.21.15.169:6789/0 conn(0x7fb3f015fb50 legacy=0x7fb3f009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:41.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.067+0000 7fb3e7e00640 1 -- 172.21.15.67:0/3674286953 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f00ce510 con 0x7fb3f009b140 2024-05-30T17:07:41.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.068+0000 7fb3e7e00640 1 --2- 172.21.15.67:0/3674286953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f009b140 0x7fb3f00c9800 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00c5ec0 tx=0x7fb3d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675c07aef88a77da server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:41.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.068+0000 7fb3ef400640 1 -- 172.21.15.67:0/3674286953 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8037070 con 0x7fb3f009b140 2024-05-30T17:07:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.068+0000 7fb3ef400640 1 -- 172.21.15.67:0/3674286953 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d8032440 con 0x7fb3f009b140 2024-05-30T17:07:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.068+0000 7fb3ef400640 1 -- 172.21.15.67:0/3674286953 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8032810 con 0x7fb3f009b140 2024-05-30T17:07:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.069+0000 7fb3f5600640 1 -- 172.21.15.67:0/3674286953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f009b140 msgr2=0x7fb3f00c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.069+0000 7fb3f5600640 1 --2- 172.21.15.67:0/3674286953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f009b140 0x7fb3f00c9800 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00c5ec0 tx=0x7fb3d802a1a0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.069+0000 7fb3f5600640 1 -- 172.21.15.67:0/3674286953 shutdown_connections 2024-05-30T17:07:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.069+0000 7fb3f5600640 1 --2- 172.21.15.67:0/3674286953 >> v2:172.21.15.169:3300/0 conn(0x7fb3f015ff30 0x7fb3f00c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.069+0000 7fb3f5600640 1 --2- 172.21.15.67:0/3674286953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f009b140 0x7fb3f00c9800 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.069+0000 7fb3f5600640 1 -- 172.21.15.67:0/3674286953 >> 172.21.15.67:0/3674286953 conn(0x7fb3f00b3eb0 msgr2=0x7fb3f00b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.069+0000 7fb3f5600640 1 -- 172.21.15.67:0/3674286953 shutdown_connections 2024-05-30T17:07:41.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.069+0000 7fb3f5600640 1 -- 172.21.15.67:0/3674286953 wait complete. 2024-05-30T17:07:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.071+0000 7fb3f5600640 1 Processor -- start 2024-05-30T17:07:41.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.071+0000 7fb3f5600640 1 -- start start 2024-05-30T17:07:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3f5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3f009b140 0x7fb3f00b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3f5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f015fb50 0x7fb3f01f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3f5600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f015ff30 0x7fb3f01507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3f5600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f00d1240 con 0x7fb3f015fb50 2024-05-30T17:07:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3f5600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f00d10c0 con 0x7fb3f015ff30 2024-05-30T17:07:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3f5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f00cd9e0 con 0x7fb3f009b140 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f015fb50 0x7fb3f01f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f015fb50 0x7fb3f01f53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49190/0 (socket says 172.21.15.67:49190) 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3e7e00640 1 -- 172.21.15.67:0/2963736594 learned_addr learned my addr 172.21.15.67:0/2963736594 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3efe00640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3f009b140 0x7fb3f00b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3e7e00640 1 -- 172.21.15.67:0/2963736594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3f009b140 msgr2=0x7fb3f00b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3f4c00640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f015ff30 0x7fb3f01507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3e7e00640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3f009b140 0x7fb3f00b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3e7e00640 1 -- 172.21.15.67:0/2963736594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f015ff30 msgr2=0x7fb3f01507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3e7e00640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f015ff30 0x7fb3f01507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.072+0000 7fb3e7e00640 1 -- 172.21.15.67:0/2963736594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f01f5910 con 0x7fb3f015fb50 2024-05-30T17:07:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.073+0000 7fb3e7e00640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f015fb50 0x7fb3f01f53d0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8000c00 tx=0x7fb3d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.073+0000 7fb3ecc00640 1 -- 172.21.15.67:0/2963736594 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8041020 con 0x7fb3f015fb50 2024-05-30T17:07:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.073+0000 7fb3ecc00640 1 -- 172.21.15.67:0/2963736594 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d8030970 con 0x7fb3f015fb50 2024-05-30T17:07:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.073+0000 7fb3ecc00640 1 -- 172.21.15.67:0/2963736594 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8037040 con 0x7fb3f015fb50 2024-05-30T17:07:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.073+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f00b1f00 con 0x7fb3f015fb50 2024-05-30T17:07:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.073+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f00b24c0 con 0x7fb3f015fb50 2024-05-30T17:07:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.074+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f00689e0 con 0x7fb3f015fb50 2024-05-30T17:07:41.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.076+0000 7fb3ecc00640 1 -- 172.21.15.67:0/2963736594 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7fb3d8044020 con 0x7fb3f015fb50 2024-05-30T17:07:41.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.077+0000 7fb3ecc00640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3cc06e510 0x7fb3cc0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.077+0000 7fb3ecc00640 1 -- 172.21.15.67:0/2963736594 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fb3d80ab4e0 con 0x7fb3f015fb50 2024-05-30T17:07:41.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.078+0000 7fb3efe00640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3cc06e510 0x7fb3cc0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.078+0000 7fb3efe00640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3cc06e510 0x7fb3cc0709d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0003d90 tx=0x7fb3e0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:41.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.083+0000 7fb3ecc00640 1 -- 172.21.15.67:0/2963736594 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3d804b030 con 0x7fb3f015fb50 2024-05-30T17:07:41.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.365+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb3f00d14c0 con 0x7fb3f015fb50 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.366+0000 7fb3ecc00640 1 -- 172.21.15.67:0/2963736594 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2378 (secure 0 0 0) 0x7fb3d807b410 con 0x7fb3f015fb50 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d", 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout: "health": { 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "HEALTH_OK", 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout: "checks": {}, 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout: "mutes": [] 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-05-30T17:07:41.368 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_names": [ 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: "a", 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: "b", 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: "c" 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1454, 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-05-30T17:07:41.369 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "num_mons": 3 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap": { 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 134, 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "num_osds": 8, 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "num_up_osds": 8, 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_up_since": 1717087419, 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "num_in_osds": 8, 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_in_since": 1717087410, 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "num_remapped_pgs": 0 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:41.370 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap": { 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "pgs_by_state": [ 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "state_name": "active+clean", 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "count": 97 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pgs": 97, 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "num_pools": 3, 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "num_objects": 5021, 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "data_bytes": 20915095490, 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_used": 43029643264, 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_avail": 730064470016, 2024-05-30T17:07:41.371 INFO:teuthology.orchestra.run.smithi067.stdout: "bytes_total": 773094113280, 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "write_bytes_sec": 1705, 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "read_op_per_sec": 0, 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "write_op_per_sec": 1 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "fsmap": { 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 83, 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 9, 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "up": 1, 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "in": 1, 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1, 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "by_rank": [ 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-05-30T17:07:41.372 INFO:teuthology.orchestra.run.smithi067.stdout: "filesystem_id": 9, 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "up:active", 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 6937 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "up:standby": 3 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "mgrmap": { 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "num_standbys": 1, 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-05-30T17:07:41.373 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "servicemap": { 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 23, 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-05-30T17:05:58.606603+0000", 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "services": {} 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "progress_events": { 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "abe0c06d-6a7b-468a-a86b-94b24cb1b506": { 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "5 ongoing+pending clones; Avg progress is 31.458% (10s)\n [========....................] (remaining: 23s)", 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.31457999348640442, 2024-05-30T17:07:41.374 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:41.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-05-30T17:07:41.375 INFO:teuthology.orchestra.run.smithi067.stdout: "f5adaaf0-76c8-45a6-917a-e693b3266a76": { 2024-05-30T17:07:41.375 INFO:teuthology.orchestra.run.smithi067.stdout: "message": "4 ongoing clones; Avg progress is 14.323% (17s)\n [====........................] (remaining: 102s)", 2024-05-30T17:07:41.375 INFO:teuthology.orchestra.run.smithi067.stdout: "progress": 0.14323000609874725, 2024-05-30T17:07:41.375 INFO:teuthology.orchestra.run.smithi067.stdout: "add_to_ceph_s": true 2024-05-30T17:07:41.375 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:41.375 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-05-30T17:07:41.375 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3cc06e510 msgr2=0x7fb3cc0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3cc06e510 0x7fb3cc0709d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0003d90 tx=0x7fb3e0009040 comp rx=0 tx=0).stop 2024-05-30T17:07:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f015fb50 msgr2=0x7fb3f01f53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f015fb50 0x7fb3f01f53d0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8000c00 tx=0x7fb3d80046c0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 shutdown_connections 2024-05-30T17:07:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb3cc06e510 0x7fb3cc0709d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f015ff30 0x7fb3f01507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f015fb50 0x7fb3f01f53d0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 --2- 172.21.15.67:0/2963736594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3f009b140 0x7fb3f00b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.369+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 >> 172.21.15.67:0/2963736594 conn(0x7fb3f00b3eb0 msgr2=0x7fb3f006d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:41.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.370+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 shutdown_connections 2024-05-30T17:07:41.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.370+0000 7fb3f5600640 1 -- 172.21.15.67:0/2963736594 wait complete. 2024-05-30T17:07:41.393 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ERROR 2024-05-30T17:07:41.393 INFO:tasks.cephfs_test_runner:ERROR 2024-05-30T17:07:41.393 INFO:tasks.cephfs_test_runner: 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner:ERROR: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa/tasks/cephfs/test_volumes.py", line 8376, in test_when_7_clones_are_cancelled 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner: while proceed(): 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 134, in __call__ 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: reached maximum tries (11) after waiting for 10 seconds 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner: 2024-05-30T17:07:41.394 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner:ERROR: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa/tasks/cephfs/test_volumes.py", line 7778, in tearDown 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner: self.wait_if_cloning_is_ongoing() 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa/tasks/cephfs/test_volumes.py", line 7759, in wait_if_cloning_is_ongoing 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner: while proceed(): 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 134, in __call__ 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: reached maximum tries (11) after waiting for 50 seconds 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner: 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner:Ran 8 tests in 1399.244s 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner: 2024-05-30T17:07:41.395 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner: 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner:ERROR: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa/tasks/cephfs/test_volumes.py", line 8376, in test_when_7_clones_are_cancelled 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner: while proceed(): 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 134, in __call__ 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: reached maximum tries (11) after waiting for 10 seconds 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner: 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner:ERROR: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T17:07:41.396 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa/tasks/cephfs/test_volumes.py", line 7778, in tearDown 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner: self.wait_if_cloning_is_ongoing() 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa/tasks/cephfs/test_volumes.py", line 7759, in wait_if_cloning_is_ongoing 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner: while proceed(): 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/contextutil.py", line 134, in __call__ 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: reached maximum tries (11) after waiting for 50 seconds 2024-05-30T17:07:41.397 INFO:tasks.cephfs_test_runner: 2024-05-30T17:07:41.398 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter), test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T17:07:41.562 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=77fddf6f11784a40a513032d69b9eada Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e1979f064b0becde2020282c7e87cf3c5e98026c/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_019a0d1fdedb1237f85e8a3711761fae4e094286/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter), test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-30T17:07:41.564 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-30T17:07:41.572 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-30T17:07:41.588 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-30T17:07:41.588 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:07:41.589 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:07:41.652 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:07:41.652 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:07:41.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T17:07:41.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.926+0000 7f0608a00640 1 Processor -- start 2024-05-30T17:07:41.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.926+0000 7f0608a00640 1 -- start start 2024-05-30T17:07:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.927+0000 7f0608a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f060406ee40 0x7f06040ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.927+0000 7f0608a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06040ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.927+0000 7f0608a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0604156f10 con 0x7f0604156b30 2024-05-30T17:07:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.927+0000 7f0608a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f060406dda0 con 0x7f060406ee40 2024-05-30T17:07:41.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.927+0000 7f0608a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f060406e080 con 0x7f060406f2f0 2024-05-30T17:07:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.928+0000 7f0603e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06040ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.928+0000 7f05fbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f060406ee40 0x7f06040ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.928+0000 7f0603e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06040ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.67:39880/0 (socket says 172.21.15.67:39880) 2024-05-30T17:07:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.928+0000 7f0603e00640 1 -- 172.21.15.67:0/438161690 learned_addr learned my addr 172.21.15.67:0/438161690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:41.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.929+0000 7f0603e00640 1 -- 172.21.15.67:0/438161690 >> v2:172.21.15.169:3300/0 conn(0x7f060406ee40 msgr2=0x7f06040ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.929+0000 7f0603e00640 1 --2- 172.21.15.67:0/438161690 >> v2:172.21.15.169:3300/0 conn(0x7f060406ee40 0x7f06040ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.929+0000 7f0603e00640 1 -- 172.21.15.67:0/438161690 >> v1:172.21.15.169:6789/0 conn(0x7f0604156b30 legacy=0x7f06040af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.929+0000 7f0603e00640 1 -- 172.21.15.67:0/438161690 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060406e360 con 0x7f060406f2f0 2024-05-30T17:07:41.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.929+0000 7f05fbe00640 1 --2- 172.21.15.67:0/438161690 >> v2:172.21.15.169:3300/0 conn(0x7f060406ee40 0x7f06040ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:07:41.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.931+0000 7f0603e00640 1 --2- 172.21.15.67:0/438161690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06040ad770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009e50 tx=0x7f05fc02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8abd23ece43cdd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:41.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.932+0000 7f0602a00640 1 -- 172.21.15.67:0/438161690 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc036070 con 0x7f060406f2f0 2024-05-30T17:07:41.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.932+0000 7f0602a00640 1 -- 172.21.15.67:0/438161690 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc031600 con 0x7f060406f2f0 2024-05-30T17:07:41.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.932+0000 7f0602a00640 1 -- 172.21.15.67:0/438161690 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0319d0 con 0x7f060406f2f0 2024-05-30T17:07:41.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.932+0000 7f0608a00640 1 -- 172.21.15.67:0/438161690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 msgr2=0x7f06040ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:41.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.932+0000 7f0608a00640 1 --2- 172.21.15.67:0/438161690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06040ad770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009e50 tx=0x7f05fc02a680 comp rx=0 tx=0).stop 2024-05-30T17:07:41.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.932+0000 7f0608a00640 1 -- 172.21.15.67:0/438161690 shutdown_connections 2024-05-30T17:07:41.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.932+0000 7f0608a00640 1 --2- 172.21.15.67:0/438161690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06040ad770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.932+0000 7f0608a00640 1 --2- 172.21.15.67:0/438161690 >> v2:172.21.15.169:3300/0 conn(0x7f060406ee40 0x7f06040ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.932+0000 7f0608a00640 1 -- 172.21.15.67:0/438161690 >> 172.21.15.67:0/438161690 conn(0x7f06040b1fc0 msgr2=0x7f06040b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:41.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.933+0000 7f0608a00640 1 -- 172.21.15.67:0/438161690 shutdown_connections 2024-05-30T17:07:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.933+0000 7f0608a00640 1 -- 172.21.15.67:0/438161690 wait complete. 2024-05-30T17:07:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.934+0000 7f0608a00640 1 Processor -- start 2024-05-30T17:07:41.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.934+0000 7f0608a00640 1 -- start start 2024-05-30T17:07:41.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.935+0000 7f0608a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060406ee40 0x7f0604071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.935+0000 7f0608a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06041f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.935+0000 7f0608a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0604156b30 0x7f0604147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.935+0000 7f0608a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f06040b2db0 con 0x7f060406ee40 2024-05-30T17:07:41.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.935+0000 7f0608a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f06040b2c30 con 0x7f0604156b30 2024-05-30T17:07:41.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.935+0000 7f0608a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f06040b32c0 con 0x7f060406f2f0 2024-05-30T17:07:41.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f0603400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060406ee40 0x7f0604071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f0603400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060406ee40 0x7f0604071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49206/0 (socket says 172.21.15.67:49206) 2024-05-30T17:07:41.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f0603400640 1 -- 172.21.15.67:0/408805614 learned_addr learned my addr 172.21.15.67:0/408805614 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:41.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f05fbe00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06041f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f0603e00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0604156b30 0x7f0604147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f0603400640 1 -- 172.21.15.67:0/408805614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 msgr2=0x7f06041f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:41.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f0603400640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06041f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f0603400640 1 -- 172.21.15.67:0/408805614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0604156b30 msgr2=0x7f0604147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:41.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f0603400640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0604156b30 0x7f0604147a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:41.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.936+0000 7f0603400640 1 -- 172.21.15.67:0/408805614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060415ec60 con 0x7f060406ee40 2024-05-30T17:07:41.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.937+0000 7f0603400640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060406ee40 0x7f0604071bc0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c3c0 tx=0x7f05f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:41.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.937+0000 7f05fb400640 1 -- 172.21.15.67:0/408805614 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f40052d0 con 0x7f060406ee40 2024-05-30T17:07:41.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.937+0000 7f05fb400640 1 -- 172.21.15.67:0/408805614 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f4005490 con 0x7f060406ee40 2024-05-30T17:07:41.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.937+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060415ef40 con 0x7f060406ee40 2024-05-30T17:07:41.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.937+0000 7f05fb400640 1 -- 172.21.15.67:0/408805614 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4007260 con 0x7f060406ee40 2024-05-30T17:07:41.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.937+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06040b1cb0 con 0x7f060406ee40 2024-05-30T17:07:41.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.938+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06040689e0 con 0x7f060406ee40 2024-05-30T17:07:41.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.939+0000 7f05fb400640 1 -- 172.21.15.67:0/408805614 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f05f4007400 con 0x7f060406ee40 2024-05-30T17:07:41.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.940+0000 7f05fb400640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f05e006e540 0x7f05e0070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:41.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.940+0000 7f05fb400640 1 -- 172.21.15.67:0/408805614 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f05f4089f90 con 0x7f060406ee40 2024-05-30T17:07:41.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.941+0000 7f05fbe00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f05e006e540 0x7f05e0070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:41.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.941+0000 7f05fbe00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f05e006e540 0x7f05e0070a00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009560 tx=0x7f05ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:41.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:41.945+0000 7f05fb400640 1 -- 172.21.15.67:0/408805614 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05f4059e20 con 0x7f060406ee40 2024-05-30T17:07:42.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.161+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f060415a400 con 0x7f060406ee40 2024-05-30T17:07:42.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.162+0000 7f05fb400640 1 -- 172.21.15.67:0/408805614 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7f05f4059720 con 0x7f060406ee40 2024-05-30T17:07:42.164 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T17:07:42.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f05e006e540 msgr2=0x7f05e0070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:42.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f05e006e540 0x7f05e0070a00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009560 tx=0x7f05ec008040 comp rx=0 tx=0).stop 2024-05-30T17:07:42.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060406ee40 msgr2=0x7f0604071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:42.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060406ee40 0x7f0604071bc0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c3c0 tx=0x7f05f400c880 comp rx=0 tx=0).stop 2024-05-30T17:07:42.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 shutdown_connections 2024-05-30T17:07:42.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f05e006e540 0x7f05e0070a00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0604156b30 0x7f0604147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f060406f2f0 0x7f06041f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 --2- 172.21.15.67:0/408805614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060406ee40 0x7f0604071bc0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 >> 172.21.15.67:0/408805614 conn(0x7f06040b1fc0 msgr2=0x7f060415b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:42.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 shutdown_connections 2024-05-30T17:07:42.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.165+0000 7f0608a00640 1 -- 172.21.15.67:0/408805614 wait complete. 2024-05-30T17:07:42.187 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:07:42.187 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:07:42.243 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-30T17:07:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.456+0000 7fe97a600640 1 Processor -- start 2024-05-30T17:07:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.457+0000 7fe97a600640 1 -- start start 2024-05-30T17:07:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.458+0000 7fe97a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe97406ee40 0x7fe9740ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.458+0000 7fe97a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97406f2f0 0x7fe9740ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.458+0000 7fe97a600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe974156f10 con 0x7fe974156b30 2024-05-30T17:07:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.458+0000 7fe97a600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe97406dda0 con 0x7fe97406ee40 2024-05-30T17:07:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.458+0000 7fe97a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe97406e080 con 0x7fe97406f2f0 2024-05-30T17:07:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.459+0000 7fe973e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe97406ee40 0x7fe9740ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.459+0000 7fe973e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe97406ee40 0x7fe9740ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49222/0 (socket says 172.21.15.67:49222) 2024-05-30T17:07:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.459+0000 7fe973e00640 1 -- 172.21.15.67:0/993475957 learned_addr learned my addr 172.21.15.67:0/993475957 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.459+0000 7fe979c00640 1 --2- 172.21.15.67:0/993475957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97406f2f0 0x7fe9740ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.459+0000 7fe973e00640 1 -- 172.21.15.67:0/993475957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97406f2f0 msgr2=0x7fe9740ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.459+0000 7fe973e00640 1 --2- 172.21.15.67:0/993475957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97406f2f0 0x7fe9740ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.459+0000 7fe973e00640 1 -- 172.21.15.67:0/993475957 >> v1:172.21.15.169:6789/0 conn(0x7fe974156b30 legacy=0x7fe9740af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.459+0000 7fe973e00640 1 -- 172.21.15.67:0/993475957 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97406e360 con 0x7fe97406ee40 2024-05-30T17:07:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.461+0000 7fe973e00640 1 --2- 172.21.15.67:0/993475957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 0x7fe9740ad230 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fe960009870 tx=0x7fe96002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a422a59d2e15dc server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.461+0000 7fe973400640 1 -- 172.21.15.67:0/993475957 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960037070 con 0x7fe97406ee40 2024-05-30T17:07:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.461+0000 7fe973400640 1 -- 172.21.15.67:0/993475957 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe960032440 con 0x7fe97406ee40 2024-05-30T17:07:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.461+0000 7fe973400640 1 -- 172.21.15.67:0/993475957 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960032810 con 0x7fe97406ee40 2024-05-30T17:07:42.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.462+0000 7fe97a600640 1 -- 172.21.15.67:0/993475957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 msgr2=0x7fe9740ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:42.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.462+0000 7fe97a600640 1 --2- 172.21.15.67:0/993475957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 0x7fe9740ad230 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fe960009870 tx=0x7fe96002a1c0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.462+0000 7fe97a600640 1 -- 172.21.15.67:0/993475957 shutdown_connections 2024-05-30T17:07:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.462+0000 7fe97a600640 1 --2- 172.21.15.67:0/993475957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe97406f2f0 0x7fe9740ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.462+0000 7fe97a600640 1 --2- 172.21.15.67:0/993475957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 0x7fe9740ad230 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.462+0000 7fe97a600640 1 -- 172.21.15.67:0/993475957 >> 172.21.15.67:0/993475957 conn(0x7fe9740b1fc0 msgr2=0x7fe9740b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.462+0000 7fe97a600640 1 -- 172.21.15.67:0/993475957 shutdown_connections 2024-05-30T17:07:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.462+0000 7fe97a600640 1 -- 172.21.15.67:0/993475957 wait complete. 2024-05-30T17:07:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.464+0000 7fe97a600640 1 Processor -- start 2024-05-30T17:07:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.464+0000 7fe97a600640 1 -- start start 2024-05-30T17:07:42.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.465+0000 7fe97a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 0x7fe9741585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:42.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.465+0000 7fe97a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe97406f2f0 0x7fe9741e6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:42.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.465+0000 7fe97a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe974156b30 0x7fe9741f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.465+0000 7fe97a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe9740b2db0 con 0x7fe97406ee40 2024-05-30T17:07:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.465+0000 7fe97a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe9740b2c30 con 0x7fe97406f2f0 2024-05-30T17:07:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.465+0000 7fe97a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9740b32c0 con 0x7fe974156b30 2024-05-30T17:07:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.465+0000 7fe979200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 0x7fe9741585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.465+0000 7fe979200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 0x7fe9741585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49234/0 (socket says 172.21.15.67:49234) 2024-05-30T17:07:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.465+0000 7fe979200640 1 -- 172.21.15.67:0/676861109 learned_addr learned my addr 172.21.15.67:0/676861109 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe973e00640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe97406f2f0 0x7fe9741e6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:42.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe979c00640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe974156b30 0x7fe9741f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:42.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe979200640 1 -- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe974156b30 msgr2=0x7fe9741f8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:42.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe979200640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe974156b30 0x7fe9741f8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe979200640 1 -- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe97406f2f0 msgr2=0x7fe9741e6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:42.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe979200640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe97406f2f0 0x7fe9741e6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe979200640 1 -- 172.21.15.67:0/676861109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9741e7450 con 0x7fe97406ee40 2024-05-30T17:07:42.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe973e00640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe97406f2f0 0x7fe9741e6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:42.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe979c00640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe974156b30 0x7fe9741f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:42.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe979200640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 0x7fe9741585f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fe96800c400 tx=0x7fe96800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:42.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe970c00640 1 -- 172.21.15.67:0/676861109 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968017070 con 0x7fe97406ee40 2024-05-30T17:07:42.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe970c00640 1 -- 172.21.15.67:0/676861109 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9680052d0 con 0x7fe97406ee40 2024-05-30T17:07:42.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe970c00640 1 -- 172.21.15.67:0/676861109 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968003830 con 0x7fe97406ee40 2024-05-30T17:07:42.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.466+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9741e7720 con 0x7fe97406ee40 2024-05-30T17:07:42.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.467+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9741e7ce0 con 0x7fe97406ee40 2024-05-30T17:07:42.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.468+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9740689e0 con 0x7fe97406ee40 2024-05-30T17:07:42.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.469+0000 7fe970c00640 1 -- 172.21.15.67:0/676861109 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7fe96801a070 con 0x7fe97406ee40 2024-05-30T17:07:42.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.470+0000 7fe970c00640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95406e510 0x7fe9540709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:42.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.470+0000 7fe970c00640 1 -- 172.21.15.67:0/676861109 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fe968089dc0 con 0x7fe97406ee40 2024-05-30T17:07:42.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.470+0000 7fe973e00640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95406e510 0x7fe9540709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:42.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.471+0000 7fe973e00640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95406e510 0x7fe9540709d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe960034000 tx=0x7fe96002e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:42.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.474+0000 7fe970c00640 1 -- 172.21.15.67:0/676861109 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe968059cf0 con 0x7fe97406ee40 2024-05-30T17:07:42.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.699+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe9740b4950 con 0x7fe97406ee40 2024-05-30T17:07:42.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.700+0000 7fe970c00640 1 -- 172.21.15.67:0/676861109 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7fe9680595f0 con 0x7fe97406ee40 2024-05-30T17:07:42.702 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-05-30T17:07:42.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95406e510 msgr2=0x7fe9540709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:42.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95406e510 0x7fe9540709d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe960034000 tx=0x7fe96002e000 comp rx=0 tx=0).stop 2024-05-30T17:07:42.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 msgr2=0x7fe9741585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:42.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 0x7fe9741585f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fe96800c400 tx=0x7fe96800c8c0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 shutdown_connections 2024-05-30T17:07:42.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe95406e510 0x7fe9540709d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe974156b30 0x7fe9741f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe97406f2f0 0x7fe9741e6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 --2- 172.21.15.67:0/676861109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe97406ee40 0x7fe9741585f0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:42.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 >> 172.21.15.67:0/676861109 conn(0x7fe9740b1fc0 msgr2=0x7fe9740b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:42.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.703+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 shutdown_connections 2024-05-30T17:07:42.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:42.704+0000 7fe97a600640 1 -- 172.21.15.67:0/676861109 wait complete. 2024-05-30T17:07:42.723 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-05-30T17:07:42.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:42.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-30T17:07:42.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[78266]: fuse finished with error 0 and tester_r 0 2024-05-30T17:07:43.399 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:07:43.399 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T17:07:43.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:43.400 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-30T17:07:43.451 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-05-30T17:07:43.451 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:07:43.502 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052737,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-30T17:07:43.502 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:07:43.502 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:07:43.560 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:07:43.561 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-05-30T17:07:43.561 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-30T17:07:43.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-30T17:07:43.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:43.617 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-05-30T17:07:43.669 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-30T17:07:43.669 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-30T17:07:43.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:43.670 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-30T17:07:43.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:43.747 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-30T17:07:43.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:43.816 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-05-30T17:07:43.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:43.934 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-05-30T17:07:43.995 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-30T17:07:43.995 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-30T17:07:43.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:43.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-30T17:07:44.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:44.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-30T17:07:44.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-30T17:07:44.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-05-30T17:07:44.286 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-30T17:07:44.295 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-30T17:07:44.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:07:44.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.468+0000 7efcb9a00640 1 Processor -- start 2024-05-30T17:07:44.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.468+0000 7efcb9a00640 1 -- start start 2024-05-30T17:07:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.469+0000 7efcb9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb4162940 0x7efcb406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.469+0000 7efcb9a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efcb4162d20 0x7efcb4071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.469+0000 7efcb9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efcb406f040 0x7efcb4072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.469+0000 7efcb9a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7efcb406f590 con 0x7efcb406f040 2024-05-30T17:07:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.469+0000 7efcb9a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efcb406f870 con 0x7efcb4162d20 2024-05-30T17:07:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.469+0000 7efcb9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efcb40ad3d0 con 0x7efcb4162940 2024-05-30T17:07:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.469+0000 7efcb9000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efcb406f040 0x7efcb4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:44.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.469+0000 7efcb9000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efcb406f040 0x7efcb4072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49236/0 (socket says 172.21.15.67:49236) 2024-05-30T17:07:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.469+0000 7efcb9000640 1 -- 172.21.15.67:0/2899472143 learned_addr learned my addr 172.21.15.67:0/2899472143 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:44.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.470+0000 7efcb3e00640 1 --2- 172.21.15.67:0/2899472143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb4162940 0x7efcb406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:44.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.470+0000 7efcb3400640 1 --2- 172.21.15.67:0/2899472143 >> v2:172.21.15.169:3300/0 conn(0x7efcb4162d20 0x7efcb4071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:44.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.470+0000 7efcb9000640 1 -- 172.21.15.67:0/2899472143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb4162940 msgr2=0x7efcb406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:44.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.470+0000 7efcb9000640 1 --2- 172.21.15.67:0/2899472143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb4162940 0x7efcb406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.470+0000 7efcb9000640 1 -- 172.21.15.67:0/2899472143 >> v2:172.21.15.169:3300/0 conn(0x7efcb4162d20 msgr2=0x7efcb4071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:44.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.470+0000 7efcb9000640 1 --2- 172.21.15.67:0/2899472143 >> v2:172.21.15.169:3300/0 conn(0x7efcb4162d20 0x7efcb4071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.470+0000 7efcb9000640 1 -- 172.21.15.67:0/2899472143 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb406dde0 con 0x7efcb406f040 2024-05-30T17:07:44.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.471+0000 7efcb9000640 1 --2- 172.21.15.67:0/2899472143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 0x7efcb4072100 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7efcb406c860 tx=0x7efcac02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7da4790f7bb4067 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:44.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.471+0000 7efcb2a00640 1 -- 172.21.15.67:0/2899472143 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac036070 con 0x7efcb406f040 2024-05-30T17:07:44.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.471+0000 7efcb2a00640 1 -- 172.21.15.67:0/2899472143 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcac031600 con 0x7efcb406f040 2024-05-30T17:07:44.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.471+0000 7efcb2a00640 1 -- 172.21.15.67:0/2899472143 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac0319d0 con 0x7efcb406f040 2024-05-30T17:07:44.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.472+0000 7efcb9a00640 1 -- 172.21.15.67:0/2899472143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 msgr2=0x7efcb4072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:44.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.472+0000 7efcb9a00640 1 --2- 172.21.15.67:0/2899472143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 0x7efcb4072100 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7efcb406c860 tx=0x7efcac02a570 comp rx=0 tx=0).stop 2024-05-30T17:07:44.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.472+0000 7efcb9a00640 1 -- 172.21.15.67:0/2899472143 shutdown_connections 2024-05-30T17:07:44.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.472+0000 7efcb9a00640 1 --2- 172.21.15.67:0/2899472143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 0x7efcb4072100 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.472+0000 7efcb9a00640 1 --2- 172.21.15.67:0/2899472143 >> v2:172.21.15.169:3300/0 conn(0x7efcb4162d20 0x7efcb4071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.472+0000 7efcb9a00640 1 --2- 172.21.15.67:0/2899472143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb4162940 0x7efcb406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.472+0000 7efcb9a00640 1 -- 172.21.15.67:0/2899472143 >> 172.21.15.67:0/2899472143 conn(0x7efcb40b1fc0 msgr2=0x7efcb40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.472+0000 7efcb9a00640 1 -- 172.21.15.67:0/2899472143 shutdown_connections 2024-05-30T17:07:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.472+0000 7efcb9a00640 1 -- 172.21.15.67:0/2899472143 wait complete. 2024-05-30T17:07:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb9a00640 1 Processor -- start 2024-05-30T17:07:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb9a00640 1 -- start start 2024-05-30T17:07:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 0x7efcb415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb9a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb4162940 0x7efcb415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb9a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb4162d20 0x7efcb4164b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4166a10 con 0x7efcb406f040 2024-05-30T17:07:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb9a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4166890 con 0x7efcb4162940 2024-05-30T17:07:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb9a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efcb4166b90 con 0x7efcb4162d20 2024-05-30T17:07:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 0x7efcb415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 0x7efcb415b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49242/0 (socket says 172.21.15.67:49242) 2024-05-30T17:07:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb3e00640 1 -- 172.21.15.67:0/1063820269 learned_addr learned my addr 172.21.15.67:0/1063820269 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb3e00640 1 -- 172.21.15.67:0/1063820269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb4162d20 msgr2=0x7efcb4164b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb3e00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb4162d20 0x7efcb4164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb3e00640 1 -- 172.21.15.67:0/1063820269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb4162940 msgr2=0x7efcb415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.475+0000 7efcb3e00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb4162940 0x7efcb415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.476+0000 7efcb3e00640 1 -- 172.21.15.67:0/1063820269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb41650a0 con 0x7efcb406f040 2024-05-30T17:07:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.476+0000 7efcb3e00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 0x7efcb415b910 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00d800 tx=0x7efc9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.476+0000 7efcabe00640 1 -- 172.21.15.67:0/1063820269 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c018020 con 0x7efcb406f040 2024-05-30T17:07:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.476+0000 7efcabe00640 1 -- 172.21.15.67:0/1063820269 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc9c0096a0 con 0x7efcb406f040 2024-05-30T17:07:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.476+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb4165340 con 0x7efcb406f040 2024-05-30T17:07:44.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.476+0000 7efcabe00640 1 -- 172.21.15.67:0/1063820269 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c009a70 con 0x7efcb406f040 2024-05-30T17:07:44.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.476+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb4165900 con 0x7efcb406f040 2024-05-30T17:07:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.478+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb40cd950 con 0x7efcb406f040 2024-05-30T17:07:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.478+0000 7efcabe00640 1 -- 172.21.15.67:0/1063820269 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7efc9c01b020 con 0x7efcb406f040 2024-05-30T17:07:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.479+0000 7efcabe00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc9006e510 0x7efc900709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.479+0000 7efcabe00640 1 -- 172.21.15.67:0/1063820269 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7efc9c08b030 con 0x7efcb406f040 2024-05-30T17:07:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.480+0000 7efcb3400640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc9006e510 0x7efc900709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.480+0000 7efcb3400640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc9006e510 0x7efc900709d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7efca4006fd0 tx=0x7efca4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:44.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.484+0000 7efcabe00640 1 -- 172.21.15.67:0/1063820269 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc9c095020 con 0x7efcb406f040 2024-05-30T17:07:44.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.683+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efcb40689e0 con 0x7efc9006e510 2024-05-30T17:07:44.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.706+0000 7efcabe00640 1 -- 172.21.15.67:0/1063820269 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254973 (secure 0 0 0) 0x7efcb40689e0 con 0x7efc9006e510 2024-05-30T17:07:44.711 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:44.713 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:07:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc9006e510 msgr2=0x7efc900709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc9006e510 0x7efc900709d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7efca4006fd0 tx=0x7efca4008040 comp rx=0 tx=0).stop 2024-05-30T17:07:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 msgr2=0x7efcb415b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 0x7efcb415b910 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00d800 tx=0x7efc9c00dcc0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 shutdown_connections 2024-05-30T17:07:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7efc9006e510 0x7efc900709d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcb4162d20 0x7efcb4164b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb4162940 0x7efcb415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 --2- 172.21.15.67:0/1063820269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcb406f040 0x7efcb415b910 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 >> 172.21.15.67:0/1063820269 conn(0x7efcb40b1fc0 msgr2=0x7efcb40b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 shutdown_connections 2024-05-30T17:07:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.712+0000 7efcb9a00640 1 -- 172.21.15.67:0/1063820269 wait complete. 2024-05-30T17:07:44.732 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":850,"stamp":"2024-05-30T17:07:44.676607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20915095490,"num_objects":5021,"num_object_clones":1,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":1,"num_read":4440,"num_read_kb":17367224,"num_write":5865,"num_write_kb":20427047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8879,"ondisk_log_size":8879,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":42021136,"kb_used_data":40862928,"kb_used_omap":12,"kb_used_meta":1158131,"kb_avail":712953584,"statfs":{"total":773094113280,"available":730064470016,"internally_reserved":0,"allocated":41843638272,"data_stored":41836077397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1185926704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8440,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004676"},"pg_stats":[{"pgid":"19.3f","version":"134'118","reported_seq":213,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.596741+0000","last_change":"2024-05-30T17:04:52.998366+0000","last_active":"2024-05-30T17:07:34.596741+0000","last_peered":"2024-05-30T17:07:34.596741+0000","last_clean":"2024-05-30T17:07:34.596741+0000","last_became_active":"2024-05-30T17:04:52.997618+0000","last_became_peered":"2024-05-30T17:04:52.997618+0000","last_unstale":"2024-05-30T17:07:34.596741+0000","last_undegraded":"2024-05-30T17:07:34.596741+0000","last_fullsized":"2024-05-30T17:07:34.596741+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:57.883038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":71,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'119","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.587333+0000","last_change":"2024-05-30T17:04:52.984295+0000","last_active":"2024-05-30T17:07:34.587333+0000","last_peered":"2024-05-30T17:07:34.587333+0000","last_clean":"2024-05-30T17:07:34.587333+0000","last_became_active":"2024-05-30T17:04:52.982524+0000","last_became_peered":"2024-05-30T17:04:52.982524+0000","last_unstale":"2024-05-30T17:07:34.587333+0000","last_undegraded":"2024-05-30T17:07:34.587333+0000","last_fullsized":"2024-05-30T17:07:34.587333+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:49.622879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'141","reported_seq":233,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597096+0000","last_change":"2024-05-30T17:04:52.984783+0000","last_active":"2024-05-30T17:07:34.597096+0000","last_peered":"2024-05-30T17:07:34.597096+0000","last_clean":"2024-05-30T17:07:34.597096+0000","last_became_active":"2024-05-30T17:04:52.984493+0000","last_became_peered":"2024-05-30T17:04:52.984493+0000","last_unstale":"2024-05-30T17:07:34.597096+0000","last_undegraded":"2024-05-30T17:07:34.597096+0000","last_fullsized":"2024-05-30T17:07:34.597096+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:21.682021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904709,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":75,"num_read_kb":286725,"num_write":94,"num_write_kb":356356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'120","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592872+0000","last_change":"2024-05-30T17:05:34.963309+0000","last_active":"2024-05-30T17:07:34.592872+0000","last_peered":"2024-05-30T17:07:34.592872+0000","last_clean":"2024-05-30T17:07:34.592872+0000","last_became_active":"2024-05-30T17:05:34.859953+0000","last_became_peered":"2024-05-30T17:05:34.859953+0000","last_unstale":"2024-05-30T17:07:34.592872+0000","last_undegraded":"2024-05-30T17:07:34.592872+0000","last_fullsized":"2024-05-30T17:07:34.592872+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:31.835151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":70,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3b","version":"134'111","reported_seq":173,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.836580+0000","last_change":"2024-05-30T17:04:52.990350+0000","last_active":"2024-05-30T17:07:42.836580+0000","last_peered":"2024-05-30T17:07:42.836580+0000","last_clean":"2024-05-30T17:07:42.836580+0000","last_became_active":"2024-05-30T17:04:52.990018+0000","last_became_peered":"2024-05-30T17:04:52.990018+0000","last_unstale":"2024-05-30T17:07:42.836580+0000","last_undegraded":"2024-05-30T17:07:42.836580+0000","last_fullsized":"2024-05-30T17:07:42.836580+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:21.257549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":61,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'134","reported_seq":238,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600457+0000","last_change":"2024-05-30T17:04:52.992047+0000","last_active":"2024-05-30T17:07:34.600457+0000","last_peered":"2024-05-30T17:07:34.600457+0000","last_clean":"2024-05-30T17:07:34.600457+0000","last_became_active":"2024-05-30T17:04:52.991748+0000","last_became_peered":"2024-05-30T17:04:52.991748+0000","last_unstale":"2024-05-30T17:07:34.600457+0000","last_undegraded":"2024-05-30T17:07:34.600457+0000","last_fullsized":"2024-05-30T17:07:34.600457+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:06.346572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":84,"num_write_kb":335873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'126","reported_seq":194,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595157+0000","last_change":"2024-05-30T17:04:52.984245+0000","last_active":"2024-05-30T17:07:34.595157+0000","last_peered":"2024-05-30T17:07:34.595157+0000","last_clean":"2024-05-30T17:07:34.595157+0000","last_became_active":"2024-05-30T17:04:52.983960+0000","last_became_peered":"2024-05-30T17:04:52.983960+0000","last_unstale":"2024-05-30T17:07:34.595157+0000","last_undegraded":"2024-05-30T17:07:34.595157+0000","last_fullsized":"2024-05-30T17:07:34.595157+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:40:33.491032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":51,"num_read_kb":200706,"num_write":76,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'119","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592777+0000","last_change":"2024-05-30T17:04:52.988608+0000","last_active":"2024-05-30T17:07:34.592777+0000","last_peered":"2024-05-30T17:07:34.592777+0000","last_clean":"2024-05-30T17:07:34.592777+0000","last_became_active":"2024-05-30T17:04:52.988277+0000","last_became_peered":"2024-05-30T17:04:52.988277+0000","last_unstale":"2024-05-30T17:07:34.592777+0000","last_undegraded":"2024-05-30T17:07:34.592777+0000","last_fullsized":"2024-05-30T17:07:34.592777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:34.512494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":85,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'135","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.574879+0000","last_change":"2024-05-30T17:04:52.993014+0000","last_active":"2024-05-30T17:07:34.574879+0000","last_peered":"2024-05-30T17:07:34.574879+0000","last_clean":"2024-05-30T17:07:34.574879+0000","last_became_active":"2024-05-30T17:04:52.990543+0000","last_became_peered":"2024-05-30T17:04:52.990543+0000","last_unstale":"2024-05-30T17:07:34.574879+0000","last_undegraded":"2024-05-30T17:07:34.574879+0000","last_fullsized":"2024-05-30T17:07:34.574879+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:27:15.135800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":59,"num_read_kb":233474,"num_write":78,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'127","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.589282+0000","last_change":"2024-05-30T17:04:52.979749+0000","last_active":"2024-05-30T17:07:34.589282+0000","last_peered":"2024-05-30T17:07:34.589282+0000","last_clean":"2024-05-30T17:07:34.589282+0000","last_became_active":"2024-05-30T17:04:52.979390+0000","last_became_peered":"2024-05-30T17:04:52.979390+0000","last_unstale":"2024-05-30T17:07:34.589282+0000","last_undegraded":"2024-05-30T17:07:34.589282+0000","last_fullsized":"2024-05-30T17:07:34.589282+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":65,"num_read_kb":266240,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'122","reported_seq":245,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.599654+0000","last_change":"2024-05-30T17:04:52.983598+0000","last_active":"2024-05-30T17:07:34.599654+0000","last_peered":"2024-05-30T17:07:34.599654+0000","last_clean":"2024-05-30T17:07:34.599654+0000","last_became_active":"2024-05-30T17:04:52.983250+0000","last_became_peered":"2024-05-30T17:04:52.983250+0000","last_unstale":"2024-05-30T17:07:34.599654+0000","last_undegraded":"2024-05-30T17:07:34.599654+0000","last_fullsized":"2024-05-30T17:07:34.599654+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:20.889101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":106,"num_read_kb":434176,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'115","reported_seq":237,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.576009+0000","last_change":"2024-05-30T17:04:52.993606+0000","last_active":"2024-05-30T17:07:34.576009+0000","last_peered":"2024-05-30T17:07:34.576009+0000","last_clean":"2024-05-30T17:07:34.576009+0000","last_became_active":"2024-05-30T17:04:52.993151+0000","last_became_peered":"2024-05-30T17:04:52.993151+0000","last_unstale":"2024-05-30T17:07:34.576009+0000","last_undegraded":"2024-05-30T17:07:34.576009+0000","last_fullsized":"2024-05-30T17:07:34.576009+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:14.893435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":73,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'124","reported_seq":225,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.601181+0000","last_change":"2024-05-30T17:04:52.998102+0000","last_active":"2024-05-30T17:07:34.601181+0000","last_peered":"2024-05-30T17:07:34.601181+0000","last_clean":"2024-05-30T17:07:34.601181+0000","last_became_active":"2024-05-30T17:04:52.997770+0000","last_became_peered":"2024-05-30T17:04:52.997770+0000","last_unstale":"2024-05-30T17:07:34.601181+0000","last_undegraded":"2024-05-30T17:07:34.601181+0000","last_fullsized":"2024-05-30T17:07:34.601181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":84,"num_read_kb":344064,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'127","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.584319+0000","last_change":"2024-05-30T17:04:52.992617+0000","last_active":"2024-05-30T17:07:34.584319+0000","last_peered":"2024-05-30T17:07:34.584319+0000","last_clean":"2024-05-30T17:07:34.584319+0000","last_became_active":"2024-05-30T17:04:52.992221+0000","last_became_peered":"2024-05-30T17:04:52.992221+0000","last_unstale":"2024-05-30T17:07:34.584319+0000","last_undegraded":"2024-05-30T17:07:34.584319+0000","last_fullsized":"2024-05-30T17:07:34.584319+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127364,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":84,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'117","reported_seq":166,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597825+0000","last_change":"2024-05-30T17:04:52.992819+0000","last_active":"2024-05-30T17:07:34.597825+0000","last_peered":"2024-05-30T17:07:34.597825+0000","last_clean":"2024-05-30T17:07:34.597825+0000","last_became_active":"2024-05-30T17:04:52.991693+0000","last_became_peered":"2024-05-30T17:04:52.991693+0000","last_unstale":"2024-05-30T17:07:34.597825+0000","last_undegraded":"2024-05-30T17:07:34.597825+0000","last_fullsized":"2024-05-30T17:07:34.597825+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":32,"num_read_kb":131072,"num_write":72,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'129","reported_seq":198,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595356+0000","last_change":"2024-05-30T17:04:52.991328+0000","last_active":"2024-05-30T17:07:34.595356+0000","last_peered":"2024-05-30T17:07:34.595356+0000","last_clean":"2024-05-30T17:07:34.595356+0000","last_became_active":"2024-05-30T17:04:52.990334+0000","last_became_peered":"2024-05-30T17:04:52.990334+0000","last_unstale":"2024-05-30T17:07:34.595356+0000","last_undegraded":"2024-05-30T17:07:34.595356+0000","last_fullsized":"2024-05-30T17:07:34.595356+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:23.528030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'114","reported_seq":174,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595797+0000","last_change":"2024-05-30T17:04:52.967087+0000","last_active":"2024-05-30T17:07:34.595797+0000","last_peered":"2024-05-30T17:07:34.595797+0000","last_clean":"2024-05-30T17:07:34.595797+0000","last_became_active":"2024-05-30T17:04:52.966026+0000","last_became_peered":"2024-05-30T17:04:52.966026+0000","last_unstale":"2024-05-30T17:07:34.595797+0000","last_undegraded":"2024-05-30T17:07:34.595797+0000","last_fullsized":"2024-05-30T17:07:34.595797+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:12.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'104","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.596384+0000","last_change":"2024-05-30T17:04:52.986386+0000","last_active":"2024-05-30T17:07:34.596384+0000","last_peered":"2024-05-30T17:07:34.596384+0000","last_clean":"2024-05-30T17:07:34.596384+0000","last_became_active":"2024-05-30T17:04:52.985602+0000","last_became_peered":"2024-05-30T17:04:52.985602+0000","last_unstale":"2024-05-30T17:07:34.596384+0000","last_undegraded":"2024-05-30T17:07:34.596384+0000","last_fullsized":"2024-05-30T17:07:34.596384+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:31.682869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":91,"num_read_kb":372736,"num_write":71,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'128","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595466+0000","last_change":"2024-05-30T17:04:52.963412+0000","last_active":"2024-05-30T17:07:34.595466+0000","last_peered":"2024-05-30T17:07:34.595466+0000","last_clean":"2024-05-30T17:07:34.595466+0000","last_became_active":"2024-05-30T17:04:52.962582+0000","last_became_peered":"2024-05-30T17:04:52.962582+0000","last_unstale":"2024-05-30T17:07:34.595466+0000","last_undegraded":"2024-05-30T17:07:34.595466+0000","last_fullsized":"2024-05-30T17:07:34.595466+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:52.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369098752,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":90,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'140","reported_seq":219,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.591011+0000","last_change":"2024-05-30T17:04:52.987028+0000","last_active":"2024-05-30T17:07:34.591011+0000","last_peered":"2024-05-30T17:07:34.591011+0000","last_clean":"2024-05-30T17:07:34.591011+0000","last_became_active":"2024-05-30T17:04:52.986701+0000","last_became_peered":"2024-05-30T17:04:52.986701+0000","last_unstale":"2024-05-30T17:07:34.591011+0000","last_undegraded":"2024-05-30T17:07:34.591011+0000","last_fullsized":"2024-05-30T17:07:34.591011+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:50:37.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352321536,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":62,"num_read_kb":249857,"num_write":86,"num_write_kb":344065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'152","reported_seq":265,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.853599+0000","last_change":"2024-05-30T17:04:52.984763+0000","last_active":"2024-05-30T17:07:42.853599+0000","last_peered":"2024-05-30T17:07:42.853599+0000","last_clean":"2024-05-30T17:07:42.853599+0000","last_became_active":"2024-05-30T17:04:52.984462+0000","last_became_peered":"2024-05-30T17:04:52.984462+0000","last_unstale":"2024-05-30T17:07:42.853599+0000","last_undegraded":"2024-05-30T17:07:42.853599+0000","last_fullsized":"2024-05-30T17:07:42.853599+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:09.221732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293056,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":96,"num_read_kb":372741,"num_write":95,"num_write_kb":364547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'126","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.591148+0000","last_change":"2024-05-30T17:04:52.980574+0000","last_active":"2024-05-30T17:07:34.591148+0000","last_peered":"2024-05-30T17:07:34.591148+0000","last_clean":"2024-05-30T17:07:34.591148+0000","last_became_active":"2024-05-30T17:04:52.978968+0000","last_became_peered":"2024-05-30T17:04:52.978968+0000","last_unstale":"2024-05-30T17:07:34.591148+0000","last_undegraded":"2024-05-30T17:07:34.591148+0000","last_fullsized":"2024-05-30T17:07:34.591148+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:30:51.419747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":61,"num_read_kb":245761,"num_write":82,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'134","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592216+0000","last_change":"2024-05-30T17:04:52.992275+0000","last_active":"2024-05-30T17:07:34.592216+0000","last_peered":"2024-05-30T17:07:34.592216+0000","last_clean":"2024-05-30T17:07:34.592216+0000","last_became_active":"2024-05-30T17:04:52.991966+0000","last_became_peered":"2024-05-30T17:04:52.991966+0000","last_unstale":"2024-05-30T17:07:34.592216+0000","last_undegraded":"2024-05-30T17:07:34.592216+0000","last_fullsized":"2024-05-30T17:07:34.592216+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:32.227484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":65,"num_read_kb":253955,"num_write":87,"num_write_kb":348161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'128","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.590037+0000","last_change":"2024-05-30T17:04:52.965882+0000","last_active":"2024-05-30T17:07:34.590037+0000","last_peered":"2024-05-30T17:07:34.590037+0000","last_clean":"2024-05-30T17:07:34.590037+0000","last_became_active":"2024-05-30T17:04:52.965116+0000","last_became_peered":"2024-05-30T17:04:52.965116+0000","last_unstale":"2024-05-30T17:07:34.590037+0000","last_undegraded":"2024-05-30T17:07:34.590037+0000","last_fullsized":"2024-05-30T17:07:34.590037+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:35:54.995662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":57,"num_read_kb":229377,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'129","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593198+0000","last_change":"2024-05-30T17:04:52.983400+0000","last_active":"2024-05-30T17:07:34.593198+0000","last_peered":"2024-05-30T17:07:34.593198+0000","last_clean":"2024-05-30T17:07:34.593198+0000","last_became_active":"2024-05-30T17:04:52.981867+0000","last_became_peered":"2024-05-30T17:04:52.981867+0000","last_unstale":"2024-05-30T17:07:34.593198+0000","last_undegraded":"2024-05-30T17:07:34.593198+0000","last_fullsized":"2024-05-30T17:07:34.593198+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":66,"num_read_kb":266241,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'133","reported_seq":220,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.598173+0000","last_change":"2024-05-30T17:04:52.989413+0000","last_active":"2024-05-30T17:07:34.598173+0000","last_peered":"2024-05-30T17:07:34.598173+0000","last_clean":"2024-05-30T17:07:34.598173+0000","last_became_active":"2024-05-30T17:04:52.989141+0000","last_became_peered":"2024-05-30T17:04:52.989141+0000","last_unstale":"2024-05-30T17:07:34.598173+0000","last_undegraded":"2024-05-30T17:07:34.598173+0000","last_fullsized":"2024-05-30T17:07:34.598173+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:15.844444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'129","reported_seq":217,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597479+0000","last_change":"2024-05-30T17:04:52.992667+0000","last_active":"2024-05-30T17:07:34.597479+0000","last_peered":"2024-05-30T17:07:34.597479+0000","last_clean":"2024-05-30T17:07:34.597479+0000","last_became_active":"2024-05-30T17:04:52.988618+0000","last_became_peered":"2024-05-30T17:04:52.988618+0000","last_unstale":"2024-05-30T17:07:34.597479+0000","last_undegraded":"2024-05-30T17:07:34.597479+0000","last_fullsized":"2024-05-30T17:07:34.597479+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377487360,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":90,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'115","reported_seq":165,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.863460+0000","last_change":"2024-05-30T17:04:52.977678+0000","last_active":"2024-05-30T17:07:42.863460+0000","last_peered":"2024-05-30T17:07:42.863460+0000","last_clean":"2024-05-30T17:07:42.863460+0000","last_became_active":"2024-05-30T17:04:52.977378+0000","last_became_peered":"2024-05-30T17:04:52.977378+0000","last_unstale":"2024-05-30T17:07:42.863460+0000","last_undegraded":"2024-05-30T17:07:42.863460+0000","last_fullsized":"2024-05-30T17:07:42.863460+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.220555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'140","reported_seq":235,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600970+0000","last_change":"2024-05-30T17:04:52.998322+0000","last_active":"2024-05-30T17:07:34.600970+0000","last_peered":"2024-05-30T17:07:34.600970+0000","last_clean":"2024-05-30T17:07:34.600970+0000","last_became_active":"2024-05-30T17:04:52.997323+0000","last_became_peered":"2024-05-30T17:04:52.997323+0000","last_unstale":"2024-05-30T17:07:34.600970+0000","last_undegraded":"2024-05-30T17:07:34.600970+0000","last_fullsized":"2024-05-30T17:07:34.600970+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390070272,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":78,"num_read_kb":307203,"num_write":95,"num_write_kb":380929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'110","reported_seq":191,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.594854+0000","last_change":"2024-05-30T17:04:52.995308+0000","last_active":"2024-05-30T17:07:34.594854+0000","last_peered":"2024-05-30T17:07:34.594854+0000","last_clean":"2024-05-30T17:07:34.594854+0000","last_became_active":"2024-05-30T17:04:52.992733+0000","last_became_peered":"2024-05-30T17:04:52.992733+0000","last_unstale":"2024-05-30T17:07:34.594854+0000","last_undegraded":"2024-05-30T17:07:34.594854+0000","last_fullsized":"2024-05-30T17:07:34.594854+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'155","reported_seq":272,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.869426+0000","last_change":"2024-05-30T17:04:52.993970+0000","last_active":"2024-05-30T17:07:42.869426+0000","last_peered":"2024-05-30T17:07:42.869426+0000","last_clean":"2024-05-30T17:07:42.869426+0000","last_became_active":"2024-05-30T17:04:52.993214+0000","last_became_peered":"2024-05-30T17:04:52.993214+0000","last_unstale":"2024-05-30T17:07:42.869426+0000","last_undegraded":"2024-05-30T17:07:42.869426+0000","last_fullsized":"2024-05-30T17:07:42.869426+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:21:06.134079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'133","reported_seq":225,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.599575+0000","last_change":"2024-05-30T17:04:52.980018+0000","last_active":"2024-05-30T17:07:34.599575+0000","last_peered":"2024-05-30T17:07:34.599575+0000","last_clean":"2024-05-30T17:07:34.599575+0000","last_became_active":"2024-05-30T17:04:52.979697+0000","last_became_peered":"2024-05-30T17:04:52.979697+0000","last_unstale":"2024-05-30T17:07:34.599575+0000","last_undegraded":"2024-05-30T17:07:34.599575+0000","last_fullsized":"2024-05-30T17:07:34.599575+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:30:44.014138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":77,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'119","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.589562+0000","last_change":"2024-05-30T17:04:52.978441+0000","last_active":"2024-05-30T17:07:34.589562+0000","last_peered":"2024-05-30T17:07:34.589562+0000","last_clean":"2024-05-30T17:07:34.589562+0000","last_became_active":"2024-05-30T17:04:52.978065+0000","last_became_peered":"2024-05-30T17:04:52.978065+0000","last_unstale":"2024-05-30T17:07:34.589562+0000","last_undegraded":"2024-05-30T17:07:34.589562+0000","last_fullsized":"2024-05-30T17:07:34.589562+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:21:00.756549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289407237,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":70,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":23,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.856712+0000","last_change":"2024-05-30T17:04:51.944167+0000","last_active":"2024-05-30T17:05:34.856712+0000","last_peered":"2024-05-30T17:05:34.856712+0000","last_clean":"2024-05-30T17:05:34.856712+0000","last_became_active":"2024-05-30T17:04:51.943796+0000","last_became_peered":"2024-05-30T17:04:51.943796+0000","last_unstale":"2024-05-30T17:05:34.856712+0000","last_undegraded":"2024-05-30T17:05:34.856712+0000","last_fullsized":"2024-05-30T17:05:34.856712+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:50.016832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'126","reported_seq":213,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592533+0000","last_change":"2024-05-30T17:04:52.998976+0000","last_active":"2024-05-30T17:07:34.592533+0000","last_peered":"2024-05-30T17:07:34.592533+0000","last_clean":"2024-05-30T17:07:34.592533+0000","last_became_active":"2024-05-30T17:04:52.998673+0000","last_became_peered":"2024-05-30T17:04:52.998673+0000","last_unstale":"2024-05-30T17:07:34.592533+0000","last_undegraded":"2024-05-30T17:07:34.592533+0000","last_fullsized":"2024-05-30T17:07:34.592533+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:58.331966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849900+0000","last_change":"2024-05-30T17:04:51.939570+0000","last_active":"2024-05-30T17:05:34.849900+0000","last_peered":"2024-05-30T17:05:34.849900+0000","last_clean":"2024-05-30T17:05:34.849900+0000","last_became_active":"2024-05-30T17:04:51.937127+0000","last_became_peered":"2024-05-30T17:04:51.937127+0000","last_unstale":"2024-05-30T17:05:34.849900+0000","last_undegraded":"2024-05-30T17:05:34.849900+0000","last_fullsized":"2024-05-30T17:05:34.849900+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:18.245355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'141","reported_seq":232,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860777+0000","last_change":"2024-05-30T17:04:52.983816+0000","last_active":"2024-05-30T17:07:42.860777+0000","last_peered":"2024-05-30T17:07:42.860777+0000","last_clean":"2024-05-30T17:07:42.860777+0000","last_became_active":"2024-05-30T17:04:52.981316+0000","last_became_peered":"2024-05-30T17:04:52.981316+0000","last_unstale":"2024-05-30T17:07:42.860777+0000","last_undegraded":"2024-05-30T17:07:42.860777+0000","last_fullsized":"2024-05-30T17:07:42.860777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:13.610126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738885,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":74,"num_read_kb":303104,"num_write":82,"num_write_kb":331777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'3","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.113874+0000","last_change":"2024-05-30T17:04:51.938960+0000","last_active":"2024-05-30T17:06:00.113874+0000","last_peered":"2024-05-30T17:06:00.113874+0000","last_clean":"2024-05-30T17:06:00.113874+0000","last_became_active":"2024-05-30T17:04:51.938644+0000","last_became_peered":"2024-05-30T17:04:51.938644+0000","last_unstale":"2024-05-30T17:06:00.113874+0000","last_undegraded":"2024-05-30T17:06:00.113874+0000","last_fullsized":"2024-05-30T17:06:00.113874+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:12.595138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'148","reported_seq":253,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.871545+0000","last_change":"2024-05-30T17:04:52.988274+0000","last_active":"2024-05-30T17:07:42.871545+0000","last_peered":"2024-05-30T17:07:42.871545+0000","last_clean":"2024-05-30T17:07:42.871545+0000","last_became_active":"2024-05-30T17:04:52.987961+0000","last_became_peered":"2024-05-30T17:04:52.987961+0000","last_unstale":"2024-05-30T17:07:42.871545+0000","last_undegraded":"2024-05-30T17:07:42.871545+0000","last_fullsized":"2024-05-30T17:07:42.871545+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:21.613683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904448,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":87,"num_write_kb":356352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849476+0000","last_change":"2024-05-30T17:04:51.930251+0000","last_active":"2024-05-30T17:05:34.849476+0000","last_peered":"2024-05-30T17:05:34.849476+0000","last_clean":"2024-05-30T17:05:34.849476+0000","last_became_active":"2024-05-30T17:04:51.929298+0000","last_became_peered":"2024-05-30T17:04:51.929298+0000","last_unstale":"2024-05-30T17:05:34.849476+0000","last_undegraded":"2024-05-30T17:05:34.849476+0000","last_fullsized":"2024-05-30T17:05:34.849476+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:35.687280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'127","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600397+0000","last_change":"2024-05-30T17:04:52.986145+0000","last_active":"2024-05-30T17:07:34.600397+0000","last_peered":"2024-05-30T17:07:34.600397+0000","last_clean":"2024-05-30T17:07:34.600397+0000","last_became_active":"2024-05-30T17:04:52.985811+0000","last_became_peered":"2024-05-30T17:04:52.985811+0000","last_unstale":"2024-05-30T17:07:34.600397+0000","last_undegraded":"2024-05-30T17:07:34.600397+0000","last_fullsized":"2024-05-30T17:07:34.600397+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:55.493534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.848968+0000","last_change":"2024-05-30T17:04:51.943821+0000","last_active":"2024-05-30T17:05:34.848968+0000","last_peered":"2024-05-30T17:05:34.848968+0000","last_clean":"2024-05-30T17:05:34.848968+0000","last_became_active":"2024-05-30T17:04:51.942597+0000","last_became_peered":"2024-05-30T17:04:51.942597+0000","last_unstale":"2024-05-30T17:05:34.848968+0000","last_undegraded":"2024-05-30T17:05:34.848968+0000","last_fullsized":"2024-05-30T17:05:34.848968+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:47.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'119","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.598016+0000","last_change":"2024-05-30T17:04:52.967603+0000","last_active":"2024-05-30T17:07:34.598016+0000","last_peered":"2024-05-30T17:07:34.598016+0000","last_clean":"2024-05-30T17:07:34.598016+0000","last_became_active":"2024-05-30T17:04:52.967033+0000","last_became_peered":"2024-05-30T17:04:52.967033+0000","last_unstale":"2024-05-30T17:07:34.598016+0000","last_undegraded":"2024-05-30T17:07:34.598016+0000","last_fullsized":"2024-05-30T17:07:34.598016+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:18.064738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.948046+0000","last_change":"2024-05-30T17:04:51.942553+0000","last_active":"2024-05-30T17:05:34.948046+0000","last_peered":"2024-05-30T17:05:34.948046+0000","last_clean":"2024-05-30T17:05:34.948046+0000","last_became_active":"2024-05-30T17:04:51.941974+0000","last_became_peered":"2024-05-30T17:04:51.941974+0000","last_unstale":"2024-05-30T17:05:34.948046+0000","last_undegraded":"2024-05-30T17:05:34.948046+0000","last_fullsized":"2024-05-30T17:05:34.948046+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:40.877465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'136","reported_seq":229,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.594707+0000","last_change":"2024-05-30T17:04:52.988985+0000","last_active":"2024-05-30T17:07:34.594707+0000","last_peered":"2024-05-30T17:07:34.594707+0000","last_clean":"2024-05-30T17:07:34.594707+0000","last_became_active":"2024-05-30T17:04:52.988645+0000","last_became_peered":"2024-05-30T17:04:52.988645+0000","last_unstale":"2024-05-30T17:07:34.594707+0000","last_undegraded":"2024-05-30T17:07:34.594707+0000","last_fullsized":"2024-05-30T17:07:34.594707+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:40.533224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":76,"num_read_kb":303106,"num_write":79,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.934368+0000","last_change":"2024-05-30T17:04:51.951513+0000","last_active":"2024-05-30T17:05:34.934368+0000","last_peered":"2024-05-30T17:05:34.934368+0000","last_clean":"2024-05-30T17:05:34.934368+0000","last_became_active":"2024-05-30T17:04:51.951223+0000","last_became_peered":"2024-05-30T17:04:51.951223+0000","last_unstale":"2024-05-30T17:05:34.934368+0000","last_undegraded":"2024-05-30T17:05:34.934368+0000","last_fullsized":"2024-05-30T17:05:34.934368+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:13.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'121","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593857+0000","last_change":"2024-05-30T17:04:52.986452+0000","last_active":"2024-05-30T17:07:34.593857+0000","last_peered":"2024-05-30T17:07:34.593857+0000","last_clean":"2024-05-30T17:07:34.593857+0000","last_became_active":"2024-05-30T17:04:52.986164+0000","last_became_peered":"2024-05-30T17:04:52.986164+0000","last_unstale":"2024-05-30T17:07:34.593857+0000","last_undegraded":"2024-05-30T17:07:34.593857+0000","last_fullsized":"2024-05-30T17:07:34.593857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":66,"num_write_kb":253954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.848424+0000","last_change":"2024-05-30T17:04:51.955992+0000","last_active":"2024-05-30T17:05:34.848424+0000","last_peered":"2024-05-30T17:05:34.848424+0000","last_clean":"2024-05-30T17:05:34.848424+0000","last_became_active":"2024-05-30T17:04:51.955666+0000","last_became_peered":"2024-05-30T17:04:51.955666+0000","last_unstale":"2024-05-30T17:05:34.848424+0000","last_undegraded":"2024-05-30T17:05:34.848424+0000","last_fullsized":"2024-05-30T17:05:34.848424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:03.343211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'122","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.596305+0000","last_change":"2024-05-30T17:04:52.984705+0000","last_active":"2024-05-30T17:07:34.596305+0000","last_peered":"2024-05-30T17:07:34.596305+0000","last_clean":"2024-05-30T17:07:34.596305+0000","last_became_active":"2024-05-30T17:04:52.984409+0000","last_became_peered":"2024-05-30T17:04:52.984409+0000","last_unstale":"2024-05-30T17:07:34.596305+0000","last_undegraded":"2024-05-30T17:07:34.596305+0000","last_fullsized":"2024-05-30T17:07:34.596305+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":39,"num_read_kb":159744,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.123168+0000","last_change":"2024-05-30T17:04:51.952573+0000","last_active":"2024-05-30T17:06:00.123168+0000","last_peered":"2024-05-30T17:06:00.123168+0000","last_clean":"2024-05-30T17:06:00.123168+0000","last_became_active":"2024-05-30T17:04:51.948281+0000","last_became_peered":"2024-05-30T17:04:51.948281+0000","last_unstale":"2024-05-30T17:06:00.123168+0000","last_undegraded":"2024-05-30T17:06:00.123168+0000","last_fullsized":"2024-05-30T17:06:00.123168+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:26.183919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'118","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859139+0000","last_change":"2024-05-30T17:04:52.986184+0000","last_active":"2024-05-30T17:07:42.859139+0000","last_peered":"2024-05-30T17:07:42.859139+0000","last_clean":"2024-05-30T17:07:42.859139+0000","last_became_active":"2024-05-30T17:04:52.985889+0000","last_became_peered":"2024-05-30T17:04:52.985889+0000","last_unstale":"2024-05-30T17:07:42.859139+0000","last_undegraded":"2024-05-30T17:07:42.859139+0000","last_fullsized":"2024-05-30T17:07:42.859139+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:27.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881146,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":59,"num_write_kb":229378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.949232+0000","last_change":"2024-05-30T17:04:51.954829+0000","last_active":"2024-05-30T17:05:34.949232+0000","last_peered":"2024-05-30T17:05:34.949232+0000","last_clean":"2024-05-30T17:05:34.949232+0000","last_became_active":"2024-05-30T17:04:51.954434+0000","last_became_peered":"2024-05-30T17:04:51.954434+0000","last_unstale":"2024-05-30T17:05:34.949232+0000","last_undegraded":"2024-05-30T17:05:34.949232+0000","last_fullsized":"2024-05-30T17:05:34.949232+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:23.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'124","reported_seq":221,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.863792+0000","last_change":"2024-05-30T17:04:52.989346+0000","last_active":"2024-05-30T17:07:42.863792+0000","last_peered":"2024-05-30T17:07:42.863792+0000","last_clean":"2024-05-30T17:07:42.863792+0000","last_became_active":"2024-05-30T17:04:52.989051+0000","last_became_peered":"2024-05-30T17:04:52.989051+0000","last_unstale":"2024-05-30T17:07:42.863792+0000","last_undegraded":"2024-05-30T17:07:42.863792+0000","last_fullsized":"2024-05-30T17:07:42.863792+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:36.128167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'28","reported_seq":47,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:29.184425+0000","last_change":"2024-05-30T17:04:51.938158+0000","last_active":"2024-05-30T17:06:29.184425+0000","last_peered":"2024-05-30T17:06:29.184425+0000","last_clean":"2024-05-30T17:06:29.184425+0000","last_became_active":"2024-05-30T17:04:51.937857+0000","last_became_peered":"2024-05-30T17:04:51.937857+0000","last_unstale":"2024-05-30T17:06:29.184425+0000","last_undegraded":"2024-05-30T17:06:29.184425+0000","last_fullsized":"2024-05-30T17:06:29.184425+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:25.458335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'134","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.854431+0000","last_change":"2024-05-30T17:04:52.995423+0000","last_active":"2024-05-30T17:07:42.854431+0000","last_peered":"2024-05-30T17:07:42.854431+0000","last_clean":"2024-05-30T17:07:42.854431+0000","last_became_active":"2024-05-30T17:04:52.995128+0000","last_became_peered":"2024-05-30T17:04:52.995128+0000","last_unstale":"2024-05-30T17:07:42.854431+0000","last_undegraded":"2024-05-30T17:07:42.854431+0000","last_fullsized":"2024-05-30T17:07:42.854431+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:47.501828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":78,"num_write_kb":303106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'436","reported_seq":455,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593864+0000","last_change":"2024-05-30T17:04:51.953550+0000","last_active":"2024-05-30T17:07:34.593864+0000","last_peered":"2024-05-30T17:07:34.593864+0000","last_clean":"2024-05-30T17:07:34.593864+0000","last_became_active":"2024-05-30T17:04:51.953276+0000","last_became_peered":"2024-05-30T17:04:51.953276+0000","last_unstale":"2024-05-30T17:07:34.593864+0000","last_undegraded":"2024-05-30T17:07:34.593864+0000","last_fullsized":"2024-05-30T17:07:34.593864+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:07:44.468185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1698517,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":435,"num_write_kb":1831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'149","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600297+0000","last_change":"2024-05-30T17:04:52.988845+0000","last_active":"2024-05-30T17:07:34.600297+0000","last_peered":"2024-05-30T17:07:34.600297+0000","last_clean":"2024-05-30T17:07:34.600297+0000","last_became_active":"2024-05-30T17:04:52.986757+0000","last_became_peered":"2024-05-30T17:04:52.986757+0000","last_unstale":"2024-05-30T17:07:34.600297+0000","last_undegraded":"2024-05-30T17:07:34.600297+0000","last_fullsized":"2024-05-30T17:07:34.600297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385875968,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":92,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.853652+0000","last_change":"2024-05-30T17:04:51.944552+0000","last_active":"2024-05-30T17:05:34.853652+0000","last_peered":"2024-05-30T17:05:34.853652+0000","last_clean":"2024-05-30T17:05:34.853652+0000","last_became_active":"2024-05-30T17:04:51.944167+0000","last_became_peered":"2024-05-30T17:04:51.944167+0000","last_unstale":"2024-05-30T17:05:34.853652+0000","last_undegraded":"2024-05-30T17:05:34.853652+0000","last_fullsized":"2024-05-30T17:05:34.853652+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:17:06.906161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'121","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593998+0000","last_change":"2024-05-30T17:04:52.991186+0000","last_active":"2024-05-30T17:07:34.593998+0000","last_peered":"2024-05-30T17:07:34.593998+0000","last_clean":"2024-05-30T17:07:34.593998+0000","last_became_active":"2024-05-30T17:04:52.990049+0000","last_became_peered":"2024-05-30T17:04:52.990049+0000","last_unstale":"2024-05-30T17:07:34.593998+0000","last_undegraded":"2024-05-30T17:07:34.593998+0000","last_fullsized":"2024-05-30T17:07:34.593998+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:28:23.896291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.848017+0000","last_change":"2024-05-30T17:04:51.947216+0000","last_active":"2024-05-30T17:05:34.848017+0000","last_peered":"2024-05-30T17:05:34.848017+0000","last_clean":"2024-05-30T17:05:34.848017+0000","last_became_active":"2024-05-30T17:04:51.946868+0000","last_became_peered":"2024-05-30T17:04:51.946868+0000","last_unstale":"2024-05-30T17:05:34.848017+0000","last_undegraded":"2024-05-30T17:05:34.848017+0000","last_fullsized":"2024-05-30T17:05:34.848017+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:42:48.589336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'140","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592617+0000","last_change":"2024-05-30T17:04:52.986670+0000","last_active":"2024-05-30T17:07:34.592617+0000","last_peered":"2024-05-30T17:07:34.592617+0000","last_clean":"2024-05-30T17:07:34.592617+0000","last_became_active":"2024-05-30T17:04:52.986293+0000","last_became_peered":"2024-05-30T17:04:52.986293+0000","last_unstale":"2024-05-30T17:07:34.592617+0000","last_undegraded":"2024-05-30T17:07:34.592617+0000","last_fullsized":"2024-05-30T17:07:34.592617+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:28.097184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293317,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":90,"num_write_kb":364545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":27,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.955398+0000","last_change":"2024-05-30T17:04:51.955924+0000","last_active":"2024-05-30T17:05:34.955398+0000","last_peered":"2024-05-30T17:05:34.955398+0000","last_clean":"2024-05-30T17:05:34.955398+0000","last_became_active":"2024-05-30T17:04:51.955542+0000","last_became_peered":"2024-05-30T17:04:51.955542+0000","last_unstale":"2024-05-30T17:05:34.955398+0000","last_undegraded":"2024-05-30T17:05:34.955398+0000","last_fullsized":"2024-05-30T17:05:34.955398+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:38.543340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'107","reported_seq":149,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595390+0000","last_change":"2024-05-30T17:04:53.001644+0000","last_active":"2024-05-30T17:07:34.595390+0000","last_peered":"2024-05-30T17:07:34.595390+0000","last_clean":"2024-05-30T17:07:34.595390+0000","last_became_active":"2024-05-30T17:04:52.999924+0000","last_became_peered":"2024-05-30T17:04:52.999924+0000","last_unstale":"2024-05-30T17:07:34.595390+0000","last_undegraded":"2024-05-30T17:07:34.595390+0000","last_fullsized":"2024-05-30T17:07:34.595390+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:17:36.947839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.850216+0000","last_change":"2024-05-30T17:04:51.948540+0000","last_active":"2024-05-30T17:05:34.850216+0000","last_peered":"2024-05-30T17:05:34.850216+0000","last_clean":"2024-05-30T17:05:34.850216+0000","last_became_active":"2024-05-30T17:04:51.947928+0000","last_became_peered":"2024-05-30T17:04:51.947928+0000","last_unstale":"2024-05-30T17:05:34.850216+0000","last_undegraded":"2024-05-30T17:05:34.850216+0000","last_fullsized":"2024-05-30T17:05:34.850216+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:10:19.343810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'133","reported_seq":231,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.853021+0000","last_change":"2024-05-30T17:04:52.976220+0000","last_active":"2024-05-30T17:07:42.853021+0000","last_peered":"2024-05-30T17:07:42.853021+0000","last_clean":"2024-05-30T17:07:42.853021+0000","last_became_active":"2024-05-30T17:04:52.974987+0000","last_became_peered":"2024-05-30T17:04:52.974987+0000","last_unstale":"2024-05-30T17:07:42.853021+0000","last_undegraded":"2024-05-30T17:07:42.853021+0000","last_fullsized":"2024-05-30T17:07:42.853021+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:15.443712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360710144,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":81,"num_read_kb":331776,"num_write":86,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855580+0000","last_change":"2024-05-30T17:04:51.950294+0000","last_active":"2024-05-30T17:05:34.855580+0000","last_peered":"2024-05-30T17:05:34.855580+0000","last_clean":"2024-05-30T17:05:34.855580+0000","last_became_active":"2024-05-30T17:04:51.949704+0000","last_became_peered":"2024-05-30T17:04:51.949704+0000","last_unstale":"2024-05-30T17:05:34.855580+0000","last_undegraded":"2024-05-30T17:05:34.855580+0000","last_fullsized":"2024-05-30T17:05:34.855580+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:46.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'135","reported_seq":215,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.588249+0000","last_change":"2024-05-30T17:04:52.989374+0000","last_active":"2024-05-30T17:07:34.588249+0000","last_peered":"2024-05-30T17:07:34.588249+0000","last_clean":"2024-05-30T17:07:34.588249+0000","last_became_active":"2024-05-30T17:04:52.989083+0000","last_became_peered":"2024-05-30T17:04:52.989083+0000","last_unstale":"2024-05-30T17:07:34.588249+0000","last_undegraded":"2024-05-30T17:07:34.588249+0000","last_fullsized":"2024-05-30T17:07:34.588249+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:35.524809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":63,"num_read_kb":237573,"num_write":85,"num_write_kb":307205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.935270+0000","last_change":"2024-05-30T17:04:51.942441+0000","last_active":"2024-05-30T17:05:34.935270+0000","last_peered":"2024-05-30T17:05:34.935270+0000","last_clean":"2024-05-30T17:05:34.935270+0000","last_became_active":"2024-05-30T17:04:51.942121+0000","last_became_peered":"2024-05-30T17:04:51.942121+0000","last_unstale":"2024-05-30T17:05:34.935270+0000","last_undegraded":"2024-05-30T17:05:34.935270+0000","last_fullsized":"2024-05-30T17:05:34.935270+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:27.082196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'128","reported_seq":227,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.582512+0000","last_change":"2024-05-30T17:04:52.994569+0000","last_active":"2024-05-30T17:07:34.582512+0000","last_peered":"2024-05-30T17:07:34.582512+0000","last_clean":"2024-05-30T17:07:34.582512+0000","last_became_active":"2024-05-30T17:04:52.994290+0000","last_became_peered":"2024-05-30T17:04:52.994290+0000","last_unstale":"2024-05-30T17:07:34.582512+0000","last_undegraded":"2024-05-30T17:07:34.582512+0000","last_fullsized":"2024-05-30T17:07:34.582512+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849744+0000","last_change":"2024-05-30T17:04:51.945428+0000","last_active":"2024-05-30T17:05:34.849744+0000","last_peered":"2024-05-30T17:05:34.849744+0000","last_clean":"2024-05-30T17:05:34.849744+0000","last_became_active":"2024-05-30T17:04:51.945142+0000","last_became_peered":"2024-05-30T17:04:51.945142+0000","last_unstale":"2024-05-30T17:05:34.849744+0000","last_undegraded":"2024-05-30T17:05:34.849744+0000","last_fullsized":"2024-05-30T17:05:34.849744+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:25:46.375765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'128","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597686+0000","last_change":"2024-05-30T17:04:52.993514+0000","last_active":"2024-05-30T17:07:34.597686+0000","last_peered":"2024-05-30T17:07:34.597686+0000","last_clean":"2024-05-30T17:07:34.597686+0000","last_became_active":"2024-05-30T17:04:52.988660+0000","last_became_peered":"2024-05-30T17:04:52.988660+0000","last_unstale":"2024-05-30T17:07:34.597686+0000","last_undegraded":"2024-05-30T17:07:34.597686+0000","last_fullsized":"2024-05-30T17:07:34.597686+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'3","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.150462+0000","last_change":"2024-05-30T17:04:51.943410+0000","last_active":"2024-05-30T17:06:00.150462+0000","last_peered":"2024-05-30T17:06:00.150462+0000","last_clean":"2024-05-30T17:06:00.150462+0000","last_became_active":"2024-05-30T17:04:51.943085+0000","last_became_peered":"2024-05-30T17:04:51.943085+0000","last_unstale":"2024-05-30T17:06:00.150462+0000","last_undegraded":"2024-05-30T17:06:00.150462+0000","last_fullsized":"2024-05-30T17:06:00.150462+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:18.313092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'120","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.588879+0000","last_change":"2024-05-30T17:04:52.992939+0000","last_active":"2024-05-30T17:07:34.588879+0000","last_peered":"2024-05-30T17:07:34.588879+0000","last_clean":"2024-05-30T17:07:34.588879+0000","last_became_active":"2024-05-30T17:04:52.989409+0000","last_became_peered":"2024-05-30T17:04:52.989409+0000","last_unstale":"2024-05-30T17:07:34.588879+0000","last_undegraded":"2024-05-30T17:07:34.588879+0000","last_fullsized":"2024-05-30T17:07:34.588879+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:41.892452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795845,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":72,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.948183+0000","last_change":"2024-05-30T17:04:51.948194+0000","last_active":"2024-05-30T17:05:34.948183+0000","last_peered":"2024-05-30T17:05:34.948183+0000","last_clean":"2024-05-30T17:05:34.948183+0000","last_became_active":"2024-05-30T17:04:51.947919+0000","last_became_peered":"2024-05-30T17:04:51.947919+0000","last_unstale":"2024-05-30T17:05:34.948183+0000","last_undegraded":"2024-05-30T17:05:34.948183+0000","last_fullsized":"2024-05-30T17:05:34.948183+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:58.823357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'134","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600019+0000","last_change":"2024-05-30T17:04:52.981350+0000","last_active":"2024-05-30T17:07:34.600019+0000","last_peered":"2024-05-30T17:07:34.600019+0000","last_clean":"2024-05-30T17:07:34.600019+0000","last_became_active":"2024-05-30T17:04:52.978531+0000","last_became_peered":"2024-05-30T17:04:52.978531+0000","last_unstale":"2024-05-30T17:07:34.600019+0000","last_undegraded":"2024-05-30T17:07:34.600019+0000","last_fullsized":"2024-05-30T17:07:34.600019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":48,"num_read_kb":188418,"num_write":83,"num_write_kb":323586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.947541+0000","last_change":"2024-05-30T17:04:51.950726+0000","last_active":"2024-05-30T17:05:34.947541+0000","last_peered":"2024-05-30T17:05:34.947541+0000","last_clean":"2024-05-30T17:05:34.947541+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:05:34.947541+0000","last_undegraded":"2024-05-30T17:05:34.947541+0000","last_fullsized":"2024-05-30T17:05:34.947541+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:17.049750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'123","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859963+0000","last_change":"2024-05-30T17:05:35.214281+0000","last_active":"2024-05-30T17:07:42.859963+0000","last_peered":"2024-05-30T17:07:42.859963+0000","last_clean":"2024-05-30T17:07:42.859963+0000","last_became_active":"2024-05-30T17:05:34.860839+0000","last_became_peered":"2024-05-30T17:05:34.860839+0000","last_unstale":"2024-05-30T17:07:42.859963+0000","last_undegraded":"2024-05-30T17:07:42.859963+0000","last_fullsized":"2024-05-30T17:07:42.859963+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:11:46.508726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":69,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.854184+0000","last_change":"2024-05-30T17:04:51.951054+0000","last_active":"2024-05-30T17:05:34.854184+0000","last_peered":"2024-05-30T17:05:34.854184+0000","last_clean":"2024-05-30T17:05:34.854184+0000","last_became_active":"2024-05-30T17:04:51.950364+0000","last_became_peered":"2024-05-30T17:04:51.950364+0000","last_unstale":"2024-05-30T17:05:34.854184+0000","last_undegraded":"2024-05-30T17:05:34.854184+0000","last_fullsized":"2024-05-30T17:05:34.854184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:11.010697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'124","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593805+0000","last_change":"2024-05-30T17:04:52.967099+0000","last_active":"2024-05-30T17:07:34.593805+0000","last_peered":"2024-05-30T17:07:34.593805+0000","last_clean":"2024-05-30T17:07:34.593805+0000","last_became_active":"2024-05-30T17:04:52.966035+0000","last_became_peered":"2024-05-30T17:04:52.966035+0000","last_unstale":"2024-05-30T17:07:34.593805+0000","last_undegraded":"2024-05-30T17:07:34.593805+0000","last_fullsized":"2024-05-30T17:07:34.593805+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:24.565166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.853230+0000","last_change":"2024-05-30T17:04:51.949867+0000","last_active":"2024-05-30T17:05:34.853230+0000","last_peered":"2024-05-30T17:05:34.853230+0000","last_clean":"2024-05-30T17:05:34.853230+0000","last_became_active":"2024-05-30T17:04:51.949575+0000","last_became_peered":"2024-05-30T17:04:51.949575+0000","last_unstale":"2024-05-30T17:05:34.853230+0000","last_undegraded":"2024-05-30T17:05:34.853230+0000","last_fullsized":"2024-05-30T17:05:34.853230+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:58:11.353946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'126","reported_seq":230,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592727+0000","last_change":"2024-05-30T17:04:52.998729+0000","last_active":"2024-05-30T17:07:34.592727+0000","last_peered":"2024-05-30T17:07:34.592727+0000","last_clean":"2024-05-30T17:07:34.592727+0000","last_became_active":"2024-05-30T17:04:52.997726+0000","last_became_peered":"2024-05-30T17:04:52.997726+0000","last_unstale":"2024-05-30T17:07:34.592727+0000","last_undegraded":"2024-05-30T17:07:34.592727+0000","last_fullsized":"2024-05-30T17:07:34.592727+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:32:14.601932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"134'49","reported_seq":68,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:35.039807+0000","last_change":"2024-05-30T17:04:51.949751+0000","last_active":"2024-05-30T17:07:35.039807+0000","last_peered":"2024-05-30T17:07:35.039807+0000","last_clean":"2024-05-30T17:07:35.039807+0000","last_became_active":"2024-05-30T17:04:51.949295+0000","last_became_peered":"2024-05-30T17:04:51.949295+0000","last_unstale":"2024-05-30T17:07:35.039807+0000","last_undegraded":"2024-05-30T17:07:35.039807+0000","last_fullsized":"2024-05-30T17:07:35.039807+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:41.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'129","reported_seq":243,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595138+0000","last_change":"2024-05-30T17:04:52.985036+0000","last_active":"2024-05-30T17:07:34.595138+0000","last_peered":"2024-05-30T17:07:34.595138+0000","last_clean":"2024-05-30T17:07:34.595138+0000","last_became_active":"2024-05-30T17:04:52.984675+0000","last_became_peered":"2024-05-30T17:04:52.984675+0000","last_unstale":"2024-05-30T17:07:34.595138+0000","last_undegraded":"2024-05-30T17:07:34.595138+0000","last_fullsized":"2024-05-30T17:07:34.595138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":397312,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849721+0000","last_change":"2024-05-30T17:04:51.952967+0000","last_active":"2024-05-30T17:05:34.849721+0000","last_peered":"2024-05-30T17:05:34.849721+0000","last_clean":"2024-05-30T17:05:34.849721+0000","last_became_active":"2024-05-30T17:04:51.952631+0000","last_became_peered":"2024-05-30T17:04:51.952631+0000","last_unstale":"2024-05-30T17:05:34.849721+0000","last_undegraded":"2024-05-30T17:05:34.849721+0000","last_fullsized":"2024-05-30T17:05:34.849721+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:23.747059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'123","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592347+0000","last_change":"2024-05-30T17:04:52.990963+0000","last_active":"2024-05-30T17:07:34.592347+0000","last_peered":"2024-05-30T17:07:34.592347+0000","last_clean":"2024-05-30T17:07:34.592347+0000","last_became_active":"2024-05-30T17:04:52.990476+0000","last_became_peered":"2024-05-30T17:04:52.990476+0000","last_unstale":"2024-05-30T17:07:34.592347+0000","last_undegraded":"2024-05-30T17:07:34.592347+0000","last_fullsized":"2024-05-30T17:07:34.592347+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:59.838344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378750,"num_objects":77,"num_object_clones":1,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":1,"num_read":54,"num_read_kb":217089,"num_write":79,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855975+0000","last_change":"2024-05-30T17:04:51.933352+0000","last_active":"2024-05-30T17:05:34.855975+0000","last_peered":"2024-05-30T17:05:34.855975+0000","last_clean":"2024-05-30T17:05:34.855975+0000","last_became_active":"2024-05-30T17:04:51.933061+0000","last_became_peered":"2024-05-30T17:04:51.933061+0000","last_unstale":"2024-05-30T17:05:34.855975+0000","last_undegraded":"2024-05-30T17:05:34.855975+0000","last_fullsized":"2024-05-30T17:05:34.855975+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:20.242562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'134","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.870019+0000","last_change":"2024-05-30T17:04:52.982699+0000","last_active":"2024-05-30T17:07:42.870019+0000","last_peered":"2024-05-30T17:07:42.870019+0000","last_clean":"2024-05-30T17:07:42.870019+0000","last_became_active":"2024-05-30T17:04:52.981542+0000","last_became_peered":"2024-05-30T17:04:52.981542+0000","last_unstale":"2024-05-30T17:07:42.870019+0000","last_undegraded":"2024-05-30T17:07:42.870019+0000","last_fullsized":"2024-05-30T17:07:42.870019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:12.025684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":76,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849081+0000","last_change":"2024-05-30T17:04:51.947181+0000","last_active":"2024-05-30T17:05:34.849081+0000","last_peered":"2024-05-30T17:05:34.849081+0000","last_clean":"2024-05-30T17:05:34.849081+0000","last_became_active":"2024-05-30T17:04:51.946882+0000","last_became_peered":"2024-05-30T17:04:51.946882+0000","last_unstale":"2024-05-30T17:05:34.849081+0000","last_undegraded":"2024-05-30T17:05:34.849081+0000","last_fullsized":"2024-05-30T17:05:34.849081+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:47:52.431592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'157","reported_seq":257,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860002+0000","last_change":"2024-05-30T17:04:52.992399+0000","last_active":"2024-05-30T17:07:42.860002+0000","last_peered":"2024-05-30T17:07:42.860002+0000","last_clean":"2024-05-30T17:07:42.860002+0000","last_became_active":"2024-05-30T17:04:52.992040+0000","last_became_peered":"2024-05-30T17:04:52.992040+0000","last_unstale":"2024-05-30T17:07:42.860002+0000","last_undegraded":"2024-05-30T17:07:42.860002+0000","last_fullsized":"2024-05-30T17:07:42.860002+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:16.335923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":411041792,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":98,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855981+0000","last_change":"2024-05-30T17:04:51.952089+0000","last_active":"2024-05-30T17:05:34.855981+0000","last_peered":"2024-05-30T17:05:34.855981+0000","last_clean":"2024-05-30T17:05:34.855981+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:05:34.855981+0000","last_undegraded":"2024-05-30T17:05:34.855981+0000","last_fullsized":"2024-05-30T17:05:34.855981+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:03:20.313618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'131","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595785+0000","last_change":"2024-05-30T17:04:52.988572+0000","last_active":"2024-05-30T17:07:34.595785+0000","last_peered":"2024-05-30T17:07:34.595785+0000","last_clean":"2024-05-30T17:07:34.595785+0000","last_became_active":"2024-05-30T17:04:52.987684+0000","last_became_peered":"2024-05-30T17:04:52.987684+0000","last_unstale":"2024-05-30T17:07:34.595785+0000","last_undegraded":"2024-05-30T17:07:34.595785+0000","last_fullsized":"2024-05-30T17:07:34.595785+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:26:52.920204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":82,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'35","reported_seq":88,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:29.176344+0000","last_change":"2024-05-30T17:04:51.949334+0000","last_active":"2024-05-30T17:06:29.176344+0000","last_peered":"2024-05-30T17:06:29.176344+0000","last_clean":"2024-05-30T17:06:29.176344+0000","last_became_active":"2024-05-30T17:04:51.948995+0000","last_became_peered":"2024-05-30T17:04:51.948995+0000","last_unstale":"2024-05-30T17:06:29.176344+0000","last_undegraded":"2024-05-30T17:06:29.176344+0000","last_fullsized":"2024-05-30T17:06:29.176344+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:05.331584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4250,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'129","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593040+0000","last_change":"2024-05-30T17:04:52.992411+0000","last_active":"2024-05-30T17:07:34.593040+0000","last_peered":"2024-05-30T17:07:34.593040+0000","last_clean":"2024-05-30T17:07:34.593040+0000","last_became_active":"2024-05-30T17:04:52.991997+0000","last_became_peered":"2024-05-30T17:04:52.991997+0000","last_unstale":"2024-05-30T17:07:34.593040+0000","last_undegraded":"2024-05-30T17:07:34.593040+0000","last_fullsized":"2024-05-30T17:07:34.593040+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":402653184,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":96,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.854388+0000","last_change":"2024-05-30T17:04:51.946511+0000","last_active":"2024-05-30T17:05:34.854388+0000","last_peered":"2024-05-30T17:05:34.854388+0000","last_clean":"2024-05-30T17:05:34.854388+0000","last_became_active":"2024-05-30T17:04:51.946173+0000","last_became_peered":"2024-05-30T17:04:51.946173+0000","last_unstale":"2024-05-30T17:05:34.854388+0000","last_undegraded":"2024-05-30T17:05:34.854388+0000","last_fullsized":"2024-05-30T17:05:34.854388+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:08.206744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'122","reported_seq":192,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.864677+0000","last_change":"2024-05-30T17:04:52.992639+0000","last_active":"2024-05-30T17:07:42.864677+0000","last_peered":"2024-05-30T17:07:42.864677+0000","last_clean":"2024-05-30T17:07:42.864677+0000","last_became_active":"2024-05-30T17:04:52.992202+0000","last_became_peered":"2024-05-30T17:04:52.992202+0000","last_unstale":"2024-05-30T17:07:42.864677+0000","last_undegraded":"2024-05-30T17:07:42.864677+0000","last_fullsized":"2024-05-30T17:07:42.864677+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:34.869318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":53,"num_read_kb":217088,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.856152+0000","last_change":"2024-05-30T17:04:51.951966+0000","last_active":"2024-05-30T17:05:34.856152+0000","last_peered":"2024-05-30T17:05:34.856152+0000","last_clean":"2024-05-30T17:05:34.856152+0000","last_became_active":"2024-05-30T17:04:51.951484+0000","last_became_peered":"2024-05-30T17:04:51.951484+0000","last_unstale":"2024-05-30T17:05:34.856152+0000","last_undegraded":"2024-05-30T17:05:34.856152+0000","last_fullsized":"2024-05-30T17:05:34.856152+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:41.627294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":388,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:44.508422+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:05:44.508422+0000","last_peered":"2024-05-30T17:05:44.508422+0000","last_clean":"2024-05-30T17:05:44.508422+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:05:44.508422+0000","last_undegraded":"2024-05-30T17:05:44.508422+0000","last_fullsized":"2024-05-30T17:05:44.508422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":20912801557,"num_objects":4996,"num_object_clones":1,"num_object_copies":9992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":1,"num_read":4290,"num_read_kb":17367090,"num_write":5064,"num_write_kb":20422699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41825673216,"data_stored":41825603114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8182,"ondisk_log_size":8182,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1703565,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":642,"num_write_kb":1932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3473408,"data_stored":3407379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509731,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5955000,"kb_used_data":5811440,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":88416840,"statfs":{"total":96636764160,"available":90538844160,"internally_reserved":0,"allocated":5950914560,"data_stored":5949962393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509731,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6190836,"kb_used_data":6051436,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":88181004,"statfs":{"total":96636764160,"available":90297348096,"internally_reserved":0,"allocated":6196670464,"data_stored":6195723625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509732,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5015352,"kb_used_data":4871792,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":89356488,"statfs":{"total":96636764160,"available":91501043712,"internally_reserved":0,"allocated":4988715008,"data_stored":4987768020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509732,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6950260,"kb_used_data":6792812,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":87421580,"statfs":{"total":96636764160,"available":89519697920,"internally_reserved":0,"allocated":6955839488,"data_stored":6954892805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509731,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4728548,"kb_used_data":4589148,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89643292,"statfs":{"total":96636764160,"available":91794731008,"internally_reserved":0,"allocated":4699287552,"data_stored":4698356383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542435,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4575160,"kb_used_data":4435760,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89796680,"statfs":{"total":96636764160,"available":91951800320,"internally_reserved":0,"allocated":4542218240,"data_stored":4541262732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542436,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4261684,"kb_used_data":4118124,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90110156,"statfs":{"total":96636764160,"available":92272799744,"internally_reserved":0,"allocated":4216958976,"data_stored":4216012047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4344296,"kb_used_data":4192416,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":90027544,"statfs":{"total":96636764160,"available":92188205056,"internally_reserved":0,"allocated":4293033984,"data_stored":4292099392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1716224,"data_stored":1702767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1720320,"data_stored":1699404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4290777088,"data_stored":4290773253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4697624576,"data_stored":4697620612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6954168320,"data_stored":6954156418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4987035648,"data_stored":4987027839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6195003392,"data_stored":6194987784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4215287808,"data_stored":4215276164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5947535360,"data_stored":5947523855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4538241024,"data_stored":4538237189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:07:44.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:07:44.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.989+0000 7f9e73200640 1 Processor -- start 2024-05-30T17:07:44.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.990+0000 7f9e73200640 1 -- start start 2024-05-30T17:07:44.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.991+0000 7f9e73200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740ce3a0 0x7f9e7406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:44.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.991+0000 7f9e73200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e7406d960 0x7f9e7409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:44.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.991+0000 7f9e73200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e740d04c0 con 0x7f9e7409af70 2024-05-30T17:07:44.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.991+0000 7f9e73200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9e740d1350 con 0x7f9e7406d960 2024-05-30T17:07:44.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.991+0000 7f9e73200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e740ce780 con 0x7f9e740ce3a0 2024-05-30T17:07:44.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.992+0000 7f9e6be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e7406d960 0x7f9e7409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:44.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.992+0000 7f9e6be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e7406d960 0x7f9e7409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:35070/0 (socket says 172.21.15.67:35070) 2024-05-30T17:07:44.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.992+0000 7f9e71e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740ce3a0 0x7f9e7406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:44.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.992+0000 7f9e6be00640 1 -- 172.21.15.67:0/943261779 learned_addr learned my addr 172.21.15.67:0/943261779 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:44.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.993+0000 7f9e6be00640 1 -- 172.21.15.67:0/943261779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740ce3a0 msgr2=0x7f9e7406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:44.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.993+0000 7f9e6be00640 1 --2- 172.21.15.67:0/943261779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740ce3a0 0x7f9e7406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.993+0000 7f9e6be00640 1 -- 172.21.15.67:0/943261779 >> v1:172.21.15.169:6789/0 conn(0x7f9e7409af70 legacy=0x7f9e741624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:44.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.993+0000 7f9e6be00640 1 -- 172.21.15.67:0/943261779 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e740cd990 con 0x7f9e7406d960 2024-05-30T17:07:44.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.993+0000 7f9e71e00640 1 --2- 172.21.15.67:0/943261779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740ce3a0 0x7f9e7406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-30T17:07:44.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.993+0000 7f9e71400640 1 -- 172.21.15.67:0/943261779 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645618270 0 0) 0x7f9e6c001490 con 0x7f9e7409af70 2024-05-30T17:07:44.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.995+0000 7f9e6be00640 1 --2- 172.21.15.67:0/943261779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e7406d960 0x7f9e7409aa30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009870 tx=0x7f9e5c02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=390b22262fb25071 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:44.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.995+0000 7f9e71400640 1 -- 172.21.15.67:0/943261779 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c037070 con 0x7f9e7406d960 2024-05-30T17:07:44.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.996+0000 7f9e71400640 1 -- 172.21.15.67:0/943261779 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5c02ae50 con 0x7f9e7406d960 2024-05-30T17:07:44.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.996+0000 7f9e71400640 1 -- 172.21.15.67:0/943261779 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c032860 con 0x7f9e7406d960 2024-05-30T17:07:44.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.996+0000 7f9e73200640 1 -- 172.21.15.67:0/943261779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e7406d960 msgr2=0x7f9e7409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:44.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.996+0000 7f9e73200640 1 --2- 172.21.15.67:0/943261779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e7406d960 0x7f9e7409aa30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009870 tx=0x7f9e5c02a300 comp rx=0 tx=0).stop 2024-05-30T17:07:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.996+0000 7f9e73200640 1 -- 172.21.15.67:0/943261779 shutdown_connections 2024-05-30T17:07:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.996+0000 7f9e73200640 1 --2- 172.21.15.67:0/943261779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e7406d960 0x7f9e7409aa30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.996+0000 7f9e73200640 1 --2- 172.21.15.67:0/943261779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e740ce3a0 0x7f9e7406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.996+0000 7f9e73200640 1 -- 172.21.15.67:0/943261779 >> 172.21.15.67:0/943261779 conn(0x7f9e740d2330 msgr2=0x7f9e740d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.996+0000 7f9e73200640 1 -- 172.21.15.67:0/943261779 shutdown_connections 2024-05-30T17:07:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.997+0000 7f9e73200640 1 -- 172.21.15.67:0/943261779 wait complete. 2024-05-30T17:07:44.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.998+0000 7f9e73200640 1 Processor -- start 2024-05-30T17:07:44.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.998+0000 7f9e73200640 1 -- start start 2024-05-30T17:07:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.999+0000 7f9e73200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e7406d960 0x7f9e740cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.999+0000 7f9e73200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e7409af70 0x7f9e7415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.999+0000 7f9e73200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e740ce3a0 0x7f9e740b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.999+0000 7f9e73200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e740d0d00 con 0x7f9e740ce3a0 2024-05-30T17:07:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.999+0000 7f9e73200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e740d0a20 con 0x7f9e7409af70 2024-05-30T17:07:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:44.999+0000 7f9e73200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e740d0fe0 con 0x7f9e7406d960 2024-05-30T17:07:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e72800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e740ce3a0 0x7f9e740b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e72800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e740ce3a0 0x7f9e740b6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49250/0 (socket says 172.21.15.67:49250) 2024-05-30T17:07:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e72800640 1 -- 172.21.15.67:0/1985014886 learned_addr learned my addr 172.21.15.67:0/1985014886 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e6be00640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e7409af70 0x7f9e7415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e71e00640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e7406d960 0x7f9e740cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e72800640 1 -- 172.21.15.67:0/1985014886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e7406d960 msgr2=0x7f9e740cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e72800640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e7406d960 0x7f9e740cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e72800640 1 -- 172.21.15.67:0/1985014886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e7409af70 msgr2=0x7f9e7415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e72800640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e7409af70 0x7f9e7415c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e72800640 1 -- 172.21.15.67:0/1985014886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e740c6da0 con 0x7f9e740ce3a0 2024-05-30T17:07:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.000+0000 7f9e71e00640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e7406d960 0x7f9e740cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.001+0000 7f9e72800640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e740ce3a0 0x7f9e740b6a50 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c7c0 tx=0x7f9e6c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.001+0000 7f9e6a000640 1 -- 172.21.15.67:0/1985014886 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c003da0 con 0x7f9e740ce3a0 2024-05-30T17:07:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.001+0000 7f9e6a000640 1 -- 172.21.15.67:0/1985014886 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6c005070 con 0x7f9e740ce3a0 2024-05-30T17:07:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.001+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e740c70e0 con 0x7f9e740ce3a0 2024-05-30T17:07:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.001+0000 7f9e6a000640 1 -- 172.21.15.67:0/1985014886 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c005460 con 0x7f9e740ce3a0 2024-05-30T17:07:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.001+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e740c7920 con 0x7f9e740ce3a0 2024-05-30T17:07:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.002+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e740690f0 con 0x7f9e740ce3a0 2024-05-30T17:07:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.003+0000 7f9e6a000640 1 -- 172.21.15.67:0/1985014886 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f9e6c005600 con 0x7f9e740ce3a0 2024-05-30T17:07:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.004+0000 7f9e6a000640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e5006e540 0x7f9e50070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.004+0000 7f9e6a000640 1 -- 172.21.15.67:0/1985014886 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f9e6c08aaf0 con 0x7f9e740ce3a0 2024-05-30T17:07:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.005+0000 7f9e71e00640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e5006e540 0x7f9e50070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.005+0000 7f9e71e00640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e5006e540 0x7f9e50070a00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009a20 tx=0x7f9e64008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:45.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.009+0000 7f9e6a000640 1 -- 172.21.15.67:0/1985014886 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e740c70e0 con 0x7f9e740ce3a0 2024-05-30T17:07:45.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.209+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e740d1dc0 con 0x7f9e5006e540 2024-05-30T17:07:45.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.229+0000 7f9e6a000640 1 -- 172.21.15.67:0/1985014886 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254973 (secure 0 0 0) 0x7f9e740d1dc0 con 0x7f9e5006e540 2024-05-30T17:07:45.233 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:45.234 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:07:45.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e5006e540 msgr2=0x7f9e50070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e5006e540 0x7f9e50070a00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009a20 tx=0x7f9e64008040 comp rx=0 tx=0).stop 2024-05-30T17:07:45.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e740ce3a0 msgr2=0x7f9e740b6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e740ce3a0 0x7f9e740b6a50 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c7c0 tx=0x7f9e6c00cc80 comp rx=0 tx=0).stop 2024-05-30T17:07:45.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 shutdown_connections 2024-05-30T17:07:45.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9e5006e540 0x7f9e50070a00 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e740ce3a0 0x7f9e740b6a50 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e7409af70 0x7f9e7415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 --2- 172.21.15.67:0/1985014886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e7406d960 0x7f9e740cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 >> 172.21.15.67:0/1985014886 conn(0x7f9e740d2330 msgr2=0x7f9e7406fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:45.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 shutdown_connections 2024-05-30T17:07:45.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.235+0000 7f9e73200640 1 -- 172.21.15.67:0/1985014886 wait complete. 2024-05-30T17:07:45.258 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":850,"stamp":"2024-05-30T17:07:44.676607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20915095490,"num_objects":5021,"num_object_clones":1,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":1,"num_read":4440,"num_read_kb":17367224,"num_write":5865,"num_write_kb":20427047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8879,"ondisk_log_size":8879,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":42021136,"kb_used_data":40862928,"kb_used_omap":12,"kb_used_meta":1158131,"kb_avail":712953584,"statfs":{"total":773094113280,"available":730064470016,"internally_reserved":0,"allocated":41843638272,"data_stored":41836077397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1185926704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8440,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004676"},"pg_stats":[{"pgid":"19.3f","version":"134'118","reported_seq":213,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.596741+0000","last_change":"2024-05-30T17:04:52.998366+0000","last_active":"2024-05-30T17:07:34.596741+0000","last_peered":"2024-05-30T17:07:34.596741+0000","last_clean":"2024-05-30T17:07:34.596741+0000","last_became_active":"2024-05-30T17:04:52.997618+0000","last_became_peered":"2024-05-30T17:04:52.997618+0000","last_unstale":"2024-05-30T17:07:34.596741+0000","last_undegraded":"2024-05-30T17:07:34.596741+0000","last_fullsized":"2024-05-30T17:07:34.596741+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:57.883038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":71,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'119","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.587333+0000","last_change":"2024-05-30T17:04:52.984295+0000","last_active":"2024-05-30T17:07:34.587333+0000","last_peered":"2024-05-30T17:07:34.587333+0000","last_clean":"2024-05-30T17:07:34.587333+0000","last_became_active":"2024-05-30T17:04:52.982524+0000","last_became_peered":"2024-05-30T17:04:52.982524+0000","last_unstale":"2024-05-30T17:07:34.587333+0000","last_undegraded":"2024-05-30T17:07:34.587333+0000","last_fullsized":"2024-05-30T17:07:34.587333+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:49.622879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'141","reported_seq":233,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597096+0000","last_change":"2024-05-30T17:04:52.984783+0000","last_active":"2024-05-30T17:07:34.597096+0000","last_peered":"2024-05-30T17:07:34.597096+0000","last_clean":"2024-05-30T17:07:34.597096+0000","last_became_active":"2024-05-30T17:04:52.984493+0000","last_became_peered":"2024-05-30T17:04:52.984493+0000","last_unstale":"2024-05-30T17:07:34.597096+0000","last_undegraded":"2024-05-30T17:07:34.597096+0000","last_fullsized":"2024-05-30T17:07:34.597096+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:21.682021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904709,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":75,"num_read_kb":286725,"num_write":94,"num_write_kb":356356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'120","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592872+0000","last_change":"2024-05-30T17:05:34.963309+0000","last_active":"2024-05-30T17:07:34.592872+0000","last_peered":"2024-05-30T17:07:34.592872+0000","last_clean":"2024-05-30T17:07:34.592872+0000","last_became_active":"2024-05-30T17:05:34.859953+0000","last_became_peered":"2024-05-30T17:05:34.859953+0000","last_unstale":"2024-05-30T17:07:34.592872+0000","last_undegraded":"2024-05-30T17:07:34.592872+0000","last_fullsized":"2024-05-30T17:07:34.592872+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:31.835151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":70,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3b","version":"134'111","reported_seq":173,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.836580+0000","last_change":"2024-05-30T17:04:52.990350+0000","last_active":"2024-05-30T17:07:42.836580+0000","last_peered":"2024-05-30T17:07:42.836580+0000","last_clean":"2024-05-30T17:07:42.836580+0000","last_became_active":"2024-05-30T17:04:52.990018+0000","last_became_peered":"2024-05-30T17:04:52.990018+0000","last_unstale":"2024-05-30T17:07:42.836580+0000","last_undegraded":"2024-05-30T17:07:42.836580+0000","last_fullsized":"2024-05-30T17:07:42.836580+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:21.257549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":61,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'134","reported_seq":238,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600457+0000","last_change":"2024-05-30T17:04:52.992047+0000","last_active":"2024-05-30T17:07:34.600457+0000","last_peered":"2024-05-30T17:07:34.600457+0000","last_clean":"2024-05-30T17:07:34.600457+0000","last_became_active":"2024-05-30T17:04:52.991748+0000","last_became_peered":"2024-05-30T17:04:52.991748+0000","last_unstale":"2024-05-30T17:07:34.600457+0000","last_undegraded":"2024-05-30T17:07:34.600457+0000","last_fullsized":"2024-05-30T17:07:34.600457+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:06.346572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":84,"num_write_kb":335873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'126","reported_seq":194,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595157+0000","last_change":"2024-05-30T17:04:52.984245+0000","last_active":"2024-05-30T17:07:34.595157+0000","last_peered":"2024-05-30T17:07:34.595157+0000","last_clean":"2024-05-30T17:07:34.595157+0000","last_became_active":"2024-05-30T17:04:52.983960+0000","last_became_peered":"2024-05-30T17:04:52.983960+0000","last_unstale":"2024-05-30T17:07:34.595157+0000","last_undegraded":"2024-05-30T17:07:34.595157+0000","last_fullsized":"2024-05-30T17:07:34.595157+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:40:33.491032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":51,"num_read_kb":200706,"num_write":76,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'119","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592777+0000","last_change":"2024-05-30T17:04:52.988608+0000","last_active":"2024-05-30T17:07:34.592777+0000","last_peered":"2024-05-30T17:07:34.592777+0000","last_clean":"2024-05-30T17:07:34.592777+0000","last_became_active":"2024-05-30T17:04:52.988277+0000","last_became_peered":"2024-05-30T17:04:52.988277+0000","last_unstale":"2024-05-30T17:07:34.592777+0000","last_undegraded":"2024-05-30T17:07:34.592777+0000","last_fullsized":"2024-05-30T17:07:34.592777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:34.512494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":85,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'135","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.574879+0000","last_change":"2024-05-30T17:04:52.993014+0000","last_active":"2024-05-30T17:07:34.574879+0000","last_peered":"2024-05-30T17:07:34.574879+0000","last_clean":"2024-05-30T17:07:34.574879+0000","last_became_active":"2024-05-30T17:04:52.990543+0000","last_became_peered":"2024-05-30T17:04:52.990543+0000","last_unstale":"2024-05-30T17:07:34.574879+0000","last_undegraded":"2024-05-30T17:07:34.574879+0000","last_fullsized":"2024-05-30T17:07:34.574879+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:27:15.135800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":59,"num_read_kb":233474,"num_write":78,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'127","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.589282+0000","last_change":"2024-05-30T17:04:52.979749+0000","last_active":"2024-05-30T17:07:34.589282+0000","last_peered":"2024-05-30T17:07:34.589282+0000","last_clean":"2024-05-30T17:07:34.589282+0000","last_became_active":"2024-05-30T17:04:52.979390+0000","last_became_peered":"2024-05-30T17:04:52.979390+0000","last_unstale":"2024-05-30T17:07:34.589282+0000","last_undegraded":"2024-05-30T17:07:34.589282+0000","last_fullsized":"2024-05-30T17:07:34.589282+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":65,"num_read_kb":266240,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'122","reported_seq":245,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.599654+0000","last_change":"2024-05-30T17:04:52.983598+0000","last_active":"2024-05-30T17:07:34.599654+0000","last_peered":"2024-05-30T17:07:34.599654+0000","last_clean":"2024-05-30T17:07:34.599654+0000","last_became_active":"2024-05-30T17:04:52.983250+0000","last_became_peered":"2024-05-30T17:04:52.983250+0000","last_unstale":"2024-05-30T17:07:34.599654+0000","last_undegraded":"2024-05-30T17:07:34.599654+0000","last_fullsized":"2024-05-30T17:07:34.599654+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:20.889101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":106,"num_read_kb":434176,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'115","reported_seq":237,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.576009+0000","last_change":"2024-05-30T17:04:52.993606+0000","last_active":"2024-05-30T17:07:34.576009+0000","last_peered":"2024-05-30T17:07:34.576009+0000","last_clean":"2024-05-30T17:07:34.576009+0000","last_became_active":"2024-05-30T17:04:52.993151+0000","last_became_peered":"2024-05-30T17:04:52.993151+0000","last_unstale":"2024-05-30T17:07:34.576009+0000","last_undegraded":"2024-05-30T17:07:34.576009+0000","last_fullsized":"2024-05-30T17:07:34.576009+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:14.893435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":73,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'124","reported_seq":225,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.601181+0000","last_change":"2024-05-30T17:04:52.998102+0000","last_active":"2024-05-30T17:07:34.601181+0000","last_peered":"2024-05-30T17:07:34.601181+0000","last_clean":"2024-05-30T17:07:34.601181+0000","last_became_active":"2024-05-30T17:04:52.997770+0000","last_became_peered":"2024-05-30T17:04:52.997770+0000","last_unstale":"2024-05-30T17:07:34.601181+0000","last_undegraded":"2024-05-30T17:07:34.601181+0000","last_fullsized":"2024-05-30T17:07:34.601181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":84,"num_read_kb":344064,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'127","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.584319+0000","last_change":"2024-05-30T17:04:52.992617+0000","last_active":"2024-05-30T17:07:34.584319+0000","last_peered":"2024-05-30T17:07:34.584319+0000","last_clean":"2024-05-30T17:07:34.584319+0000","last_became_active":"2024-05-30T17:04:52.992221+0000","last_became_peered":"2024-05-30T17:04:52.992221+0000","last_unstale":"2024-05-30T17:07:34.584319+0000","last_undegraded":"2024-05-30T17:07:34.584319+0000","last_fullsized":"2024-05-30T17:07:34.584319+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127364,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":84,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'117","reported_seq":166,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597825+0000","last_change":"2024-05-30T17:04:52.992819+0000","last_active":"2024-05-30T17:07:34.597825+0000","last_peered":"2024-05-30T17:07:34.597825+0000","last_clean":"2024-05-30T17:07:34.597825+0000","last_became_active":"2024-05-30T17:04:52.991693+0000","last_became_peered":"2024-05-30T17:04:52.991693+0000","last_unstale":"2024-05-30T17:07:34.597825+0000","last_undegraded":"2024-05-30T17:07:34.597825+0000","last_fullsized":"2024-05-30T17:07:34.597825+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":32,"num_read_kb":131072,"num_write":72,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'129","reported_seq":198,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595356+0000","last_change":"2024-05-30T17:04:52.991328+0000","last_active":"2024-05-30T17:07:34.595356+0000","last_peered":"2024-05-30T17:07:34.595356+0000","last_clean":"2024-05-30T17:07:34.595356+0000","last_became_active":"2024-05-30T17:04:52.990334+0000","last_became_peered":"2024-05-30T17:04:52.990334+0000","last_unstale":"2024-05-30T17:07:34.595356+0000","last_undegraded":"2024-05-30T17:07:34.595356+0000","last_fullsized":"2024-05-30T17:07:34.595356+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:23.528030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'114","reported_seq":174,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595797+0000","last_change":"2024-05-30T17:04:52.967087+0000","last_active":"2024-05-30T17:07:34.595797+0000","last_peered":"2024-05-30T17:07:34.595797+0000","last_clean":"2024-05-30T17:07:34.595797+0000","last_became_active":"2024-05-30T17:04:52.966026+0000","last_became_peered":"2024-05-30T17:04:52.966026+0000","last_unstale":"2024-05-30T17:07:34.595797+0000","last_undegraded":"2024-05-30T17:07:34.595797+0000","last_fullsized":"2024-05-30T17:07:34.595797+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:12.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'104","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.596384+0000","last_change":"2024-05-30T17:04:52.986386+0000","last_active":"2024-05-30T17:07:34.596384+0000","last_peered":"2024-05-30T17:07:34.596384+0000","last_clean":"2024-05-30T17:07:34.596384+0000","last_became_active":"2024-05-30T17:04:52.985602+0000","last_became_peered":"2024-05-30T17:04:52.985602+0000","last_unstale":"2024-05-30T17:07:34.596384+0000","last_undegraded":"2024-05-30T17:07:34.596384+0000","last_fullsized":"2024-05-30T17:07:34.596384+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:31.682869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":91,"num_read_kb":372736,"num_write":71,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'128","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595466+0000","last_change":"2024-05-30T17:04:52.963412+0000","last_active":"2024-05-30T17:07:34.595466+0000","last_peered":"2024-05-30T17:07:34.595466+0000","last_clean":"2024-05-30T17:07:34.595466+0000","last_became_active":"2024-05-30T17:04:52.962582+0000","last_became_peered":"2024-05-30T17:04:52.962582+0000","last_unstale":"2024-05-30T17:07:34.595466+0000","last_undegraded":"2024-05-30T17:07:34.595466+0000","last_fullsized":"2024-05-30T17:07:34.595466+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:52.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369098752,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":90,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'140","reported_seq":219,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.591011+0000","last_change":"2024-05-30T17:04:52.987028+0000","last_active":"2024-05-30T17:07:34.591011+0000","last_peered":"2024-05-30T17:07:34.591011+0000","last_clean":"2024-05-30T17:07:34.591011+0000","last_became_active":"2024-05-30T17:04:52.986701+0000","last_became_peered":"2024-05-30T17:04:52.986701+0000","last_unstale":"2024-05-30T17:07:34.591011+0000","last_undegraded":"2024-05-30T17:07:34.591011+0000","last_fullsized":"2024-05-30T17:07:34.591011+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:50:37.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352321536,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":62,"num_read_kb":249857,"num_write":86,"num_write_kb":344065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'152","reported_seq":265,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.853599+0000","last_change":"2024-05-30T17:04:52.984763+0000","last_active":"2024-05-30T17:07:42.853599+0000","last_peered":"2024-05-30T17:07:42.853599+0000","last_clean":"2024-05-30T17:07:42.853599+0000","last_became_active":"2024-05-30T17:04:52.984462+0000","last_became_peered":"2024-05-30T17:04:52.984462+0000","last_unstale":"2024-05-30T17:07:42.853599+0000","last_undegraded":"2024-05-30T17:07:42.853599+0000","last_fullsized":"2024-05-30T17:07:42.853599+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:09.221732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293056,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":96,"num_read_kb":372741,"num_write":95,"num_write_kb":364547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'126","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.591148+0000","last_change":"2024-05-30T17:04:52.980574+0000","last_active":"2024-05-30T17:07:34.591148+0000","last_peered":"2024-05-30T17:07:34.591148+0000","last_clean":"2024-05-30T17:07:34.591148+0000","last_became_active":"2024-05-30T17:04:52.978968+0000","last_became_peered":"2024-05-30T17:04:52.978968+0000","last_unstale":"2024-05-30T17:07:34.591148+0000","last_undegraded":"2024-05-30T17:07:34.591148+0000","last_fullsized":"2024-05-30T17:07:34.591148+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:30:51.419747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":61,"num_read_kb":245761,"num_write":82,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'134","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592216+0000","last_change":"2024-05-30T17:04:52.992275+0000","last_active":"2024-05-30T17:07:34.592216+0000","last_peered":"2024-05-30T17:07:34.592216+0000","last_clean":"2024-05-30T17:07:34.592216+0000","last_became_active":"2024-05-30T17:04:52.991966+0000","last_became_peered":"2024-05-30T17:04:52.991966+0000","last_unstale":"2024-05-30T17:07:34.592216+0000","last_undegraded":"2024-05-30T17:07:34.592216+0000","last_fullsized":"2024-05-30T17:07:34.592216+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:32.227484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":65,"num_read_kb":253955,"num_write":87,"num_write_kb":348161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'128","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.590037+0000","last_change":"2024-05-30T17:04:52.965882+0000","last_active":"2024-05-30T17:07:34.590037+0000","last_peered":"2024-05-30T17:07:34.590037+0000","last_clean":"2024-05-30T17:07:34.590037+0000","last_became_active":"2024-05-30T17:04:52.965116+0000","last_became_peered":"2024-05-30T17:04:52.965116+0000","last_unstale":"2024-05-30T17:07:34.590037+0000","last_undegraded":"2024-05-30T17:07:34.590037+0000","last_fullsized":"2024-05-30T17:07:34.590037+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:35:54.995662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":57,"num_read_kb":229377,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'129","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593198+0000","last_change":"2024-05-30T17:04:52.983400+0000","last_active":"2024-05-30T17:07:34.593198+0000","last_peered":"2024-05-30T17:07:34.593198+0000","last_clean":"2024-05-30T17:07:34.593198+0000","last_became_active":"2024-05-30T17:04:52.981867+0000","last_became_peered":"2024-05-30T17:04:52.981867+0000","last_unstale":"2024-05-30T17:07:34.593198+0000","last_undegraded":"2024-05-30T17:07:34.593198+0000","last_fullsized":"2024-05-30T17:07:34.593198+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":66,"num_read_kb":266241,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'133","reported_seq":220,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.598173+0000","last_change":"2024-05-30T17:04:52.989413+0000","last_active":"2024-05-30T17:07:34.598173+0000","last_peered":"2024-05-30T17:07:34.598173+0000","last_clean":"2024-05-30T17:07:34.598173+0000","last_became_active":"2024-05-30T17:04:52.989141+0000","last_became_peered":"2024-05-30T17:04:52.989141+0000","last_unstale":"2024-05-30T17:07:34.598173+0000","last_undegraded":"2024-05-30T17:07:34.598173+0000","last_fullsized":"2024-05-30T17:07:34.598173+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:15.844444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'129","reported_seq":217,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597479+0000","last_change":"2024-05-30T17:04:52.992667+0000","last_active":"2024-05-30T17:07:34.597479+0000","last_peered":"2024-05-30T17:07:34.597479+0000","last_clean":"2024-05-30T17:07:34.597479+0000","last_became_active":"2024-05-30T17:04:52.988618+0000","last_became_peered":"2024-05-30T17:04:52.988618+0000","last_unstale":"2024-05-30T17:07:34.597479+0000","last_undegraded":"2024-05-30T17:07:34.597479+0000","last_fullsized":"2024-05-30T17:07:34.597479+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377487360,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":90,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'115","reported_seq":165,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.863460+0000","last_change":"2024-05-30T17:04:52.977678+0000","last_active":"2024-05-30T17:07:42.863460+0000","last_peered":"2024-05-30T17:07:42.863460+0000","last_clean":"2024-05-30T17:07:42.863460+0000","last_became_active":"2024-05-30T17:04:52.977378+0000","last_became_peered":"2024-05-30T17:04:52.977378+0000","last_unstale":"2024-05-30T17:07:42.863460+0000","last_undegraded":"2024-05-30T17:07:42.863460+0000","last_fullsized":"2024-05-30T17:07:42.863460+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.220555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'140","reported_seq":235,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600970+0000","last_change":"2024-05-30T17:04:52.998322+0000","last_active":"2024-05-30T17:07:34.600970+0000","last_peered":"2024-05-30T17:07:34.600970+0000","last_clean":"2024-05-30T17:07:34.600970+0000","last_became_active":"2024-05-30T17:04:52.997323+0000","last_became_peered":"2024-05-30T17:04:52.997323+0000","last_unstale":"2024-05-30T17:07:34.600970+0000","last_undegraded":"2024-05-30T17:07:34.600970+0000","last_fullsized":"2024-05-30T17:07:34.600970+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390070272,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":78,"num_read_kb":307203,"num_write":95,"num_write_kb":380929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'110","reported_seq":191,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.594854+0000","last_change":"2024-05-30T17:04:52.995308+0000","last_active":"2024-05-30T17:07:34.594854+0000","last_peered":"2024-05-30T17:07:34.594854+0000","last_clean":"2024-05-30T17:07:34.594854+0000","last_became_active":"2024-05-30T17:04:52.992733+0000","last_became_peered":"2024-05-30T17:04:52.992733+0000","last_unstale":"2024-05-30T17:07:34.594854+0000","last_undegraded":"2024-05-30T17:07:34.594854+0000","last_fullsized":"2024-05-30T17:07:34.594854+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'155","reported_seq":272,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.869426+0000","last_change":"2024-05-30T17:04:52.993970+0000","last_active":"2024-05-30T17:07:42.869426+0000","last_peered":"2024-05-30T17:07:42.869426+0000","last_clean":"2024-05-30T17:07:42.869426+0000","last_became_active":"2024-05-30T17:04:52.993214+0000","last_became_peered":"2024-05-30T17:04:52.993214+0000","last_unstale":"2024-05-30T17:07:42.869426+0000","last_undegraded":"2024-05-30T17:07:42.869426+0000","last_fullsized":"2024-05-30T17:07:42.869426+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:21:06.134079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'133","reported_seq":225,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.599575+0000","last_change":"2024-05-30T17:04:52.980018+0000","last_active":"2024-05-30T17:07:34.599575+0000","last_peered":"2024-05-30T17:07:34.599575+0000","last_clean":"2024-05-30T17:07:34.599575+0000","last_became_active":"2024-05-30T17:04:52.979697+0000","last_became_peered":"2024-05-30T17:04:52.979697+0000","last_unstale":"2024-05-30T17:07:34.599575+0000","last_undegraded":"2024-05-30T17:07:34.599575+0000","last_fullsized":"2024-05-30T17:07:34.599575+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:30:44.014138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":77,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'119","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.589562+0000","last_change":"2024-05-30T17:04:52.978441+0000","last_active":"2024-05-30T17:07:34.589562+0000","last_peered":"2024-05-30T17:07:34.589562+0000","last_clean":"2024-05-30T17:07:34.589562+0000","last_became_active":"2024-05-30T17:04:52.978065+0000","last_became_peered":"2024-05-30T17:04:52.978065+0000","last_unstale":"2024-05-30T17:07:34.589562+0000","last_undegraded":"2024-05-30T17:07:34.589562+0000","last_fullsized":"2024-05-30T17:07:34.589562+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:21:00.756549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289407237,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":70,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":23,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.856712+0000","last_change":"2024-05-30T17:04:51.944167+0000","last_active":"2024-05-30T17:05:34.856712+0000","last_peered":"2024-05-30T17:05:34.856712+0000","last_clean":"2024-05-30T17:05:34.856712+0000","last_became_active":"2024-05-30T17:04:51.943796+0000","last_became_peered":"2024-05-30T17:04:51.943796+0000","last_unstale":"2024-05-30T17:05:34.856712+0000","last_undegraded":"2024-05-30T17:05:34.856712+0000","last_fullsized":"2024-05-30T17:05:34.856712+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:50.016832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'126","reported_seq":213,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592533+0000","last_change":"2024-05-30T17:04:52.998976+0000","last_active":"2024-05-30T17:07:34.592533+0000","last_peered":"2024-05-30T17:07:34.592533+0000","last_clean":"2024-05-30T17:07:34.592533+0000","last_became_active":"2024-05-30T17:04:52.998673+0000","last_became_peered":"2024-05-30T17:04:52.998673+0000","last_unstale":"2024-05-30T17:07:34.592533+0000","last_undegraded":"2024-05-30T17:07:34.592533+0000","last_fullsized":"2024-05-30T17:07:34.592533+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:58.331966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849900+0000","last_change":"2024-05-30T17:04:51.939570+0000","last_active":"2024-05-30T17:05:34.849900+0000","last_peered":"2024-05-30T17:05:34.849900+0000","last_clean":"2024-05-30T17:05:34.849900+0000","last_became_active":"2024-05-30T17:04:51.937127+0000","last_became_peered":"2024-05-30T17:04:51.937127+0000","last_unstale":"2024-05-30T17:05:34.849900+0000","last_undegraded":"2024-05-30T17:05:34.849900+0000","last_fullsized":"2024-05-30T17:05:34.849900+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:18.245355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'141","reported_seq":232,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860777+0000","last_change":"2024-05-30T17:04:52.983816+0000","last_active":"2024-05-30T17:07:42.860777+0000","last_peered":"2024-05-30T17:07:42.860777+0000","last_clean":"2024-05-30T17:07:42.860777+0000","last_became_active":"2024-05-30T17:04:52.981316+0000","last_became_peered":"2024-05-30T17:04:52.981316+0000","last_unstale":"2024-05-30T17:07:42.860777+0000","last_undegraded":"2024-05-30T17:07:42.860777+0000","last_fullsized":"2024-05-30T17:07:42.860777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:13.610126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738885,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":74,"num_read_kb":303104,"num_write":82,"num_write_kb":331777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'3","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.113874+0000","last_change":"2024-05-30T17:04:51.938960+0000","last_active":"2024-05-30T17:06:00.113874+0000","last_peered":"2024-05-30T17:06:00.113874+0000","last_clean":"2024-05-30T17:06:00.113874+0000","last_became_active":"2024-05-30T17:04:51.938644+0000","last_became_peered":"2024-05-30T17:04:51.938644+0000","last_unstale":"2024-05-30T17:06:00.113874+0000","last_undegraded":"2024-05-30T17:06:00.113874+0000","last_fullsized":"2024-05-30T17:06:00.113874+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:12.595138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'148","reported_seq":253,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.871545+0000","last_change":"2024-05-30T17:04:52.988274+0000","last_active":"2024-05-30T17:07:42.871545+0000","last_peered":"2024-05-30T17:07:42.871545+0000","last_clean":"2024-05-30T17:07:42.871545+0000","last_became_active":"2024-05-30T17:04:52.987961+0000","last_became_peered":"2024-05-30T17:04:52.987961+0000","last_unstale":"2024-05-30T17:07:42.871545+0000","last_undegraded":"2024-05-30T17:07:42.871545+0000","last_fullsized":"2024-05-30T17:07:42.871545+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:21.613683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904448,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":87,"num_write_kb":356352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849476+0000","last_change":"2024-05-30T17:04:51.930251+0000","last_active":"2024-05-30T17:05:34.849476+0000","last_peered":"2024-05-30T17:05:34.849476+0000","last_clean":"2024-05-30T17:05:34.849476+0000","last_became_active":"2024-05-30T17:04:51.929298+0000","last_became_peered":"2024-05-30T17:04:51.929298+0000","last_unstale":"2024-05-30T17:05:34.849476+0000","last_undegraded":"2024-05-30T17:05:34.849476+0000","last_fullsized":"2024-05-30T17:05:34.849476+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:35.687280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'127","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600397+0000","last_change":"2024-05-30T17:04:52.986145+0000","last_active":"2024-05-30T17:07:34.600397+0000","last_peered":"2024-05-30T17:07:34.600397+0000","last_clean":"2024-05-30T17:07:34.600397+0000","last_became_active":"2024-05-30T17:04:52.985811+0000","last_became_peered":"2024-05-30T17:04:52.985811+0000","last_unstale":"2024-05-30T17:07:34.600397+0000","last_undegraded":"2024-05-30T17:07:34.600397+0000","last_fullsized":"2024-05-30T17:07:34.600397+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:55.493534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.848968+0000","last_change":"2024-05-30T17:04:51.943821+0000","last_active":"2024-05-30T17:05:34.848968+0000","last_peered":"2024-05-30T17:05:34.848968+0000","last_clean":"2024-05-30T17:05:34.848968+0000","last_became_active":"2024-05-30T17:04:51.942597+0000","last_became_peered":"2024-05-30T17:04:51.942597+0000","last_unstale":"2024-05-30T17:05:34.848968+0000","last_undegraded":"2024-05-30T17:05:34.848968+0000","last_fullsized":"2024-05-30T17:05:34.848968+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:47.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'119","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.598016+0000","last_change":"2024-05-30T17:04:52.967603+0000","last_active":"2024-05-30T17:07:34.598016+0000","last_peered":"2024-05-30T17:07:34.598016+0000","last_clean":"2024-05-30T17:07:34.598016+0000","last_became_active":"2024-05-30T17:04:52.967033+0000","last_became_peered":"2024-05-30T17:04:52.967033+0000","last_unstale":"2024-05-30T17:07:34.598016+0000","last_undegraded":"2024-05-30T17:07:34.598016+0000","last_fullsized":"2024-05-30T17:07:34.598016+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:18.064738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.948046+0000","last_change":"2024-05-30T17:04:51.942553+0000","last_active":"2024-05-30T17:05:34.948046+0000","last_peered":"2024-05-30T17:05:34.948046+0000","last_clean":"2024-05-30T17:05:34.948046+0000","last_became_active":"2024-05-30T17:04:51.941974+0000","last_became_peered":"2024-05-30T17:04:51.941974+0000","last_unstale":"2024-05-30T17:05:34.948046+0000","last_undegraded":"2024-05-30T17:05:34.948046+0000","last_fullsized":"2024-05-30T17:05:34.948046+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:40.877465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'136","reported_seq":229,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.594707+0000","last_change":"2024-05-30T17:04:52.988985+0000","last_active":"2024-05-30T17:07:34.594707+0000","last_peered":"2024-05-30T17:07:34.594707+0000","last_clean":"2024-05-30T17:07:34.594707+0000","last_became_active":"2024-05-30T17:04:52.988645+0000","last_became_peered":"2024-05-30T17:04:52.988645+0000","last_unstale":"2024-05-30T17:07:34.594707+0000","last_undegraded":"2024-05-30T17:07:34.594707+0000","last_fullsized":"2024-05-30T17:07:34.594707+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:40.533224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":76,"num_read_kb":303106,"num_write":79,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.934368+0000","last_change":"2024-05-30T17:04:51.951513+0000","last_active":"2024-05-30T17:05:34.934368+0000","last_peered":"2024-05-30T17:05:34.934368+0000","last_clean":"2024-05-30T17:05:34.934368+0000","last_became_active":"2024-05-30T17:04:51.951223+0000","last_became_peered":"2024-05-30T17:04:51.951223+0000","last_unstale":"2024-05-30T17:05:34.934368+0000","last_undegraded":"2024-05-30T17:05:34.934368+0000","last_fullsized":"2024-05-30T17:05:34.934368+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:13.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'121","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593857+0000","last_change":"2024-05-30T17:04:52.986452+0000","last_active":"2024-05-30T17:07:34.593857+0000","last_peered":"2024-05-30T17:07:34.593857+0000","last_clean":"2024-05-30T17:07:34.593857+0000","last_became_active":"2024-05-30T17:04:52.986164+0000","last_became_peered":"2024-05-30T17:04:52.986164+0000","last_unstale":"2024-05-30T17:07:34.593857+0000","last_undegraded":"2024-05-30T17:07:34.593857+0000","last_fullsized":"2024-05-30T17:07:34.593857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":66,"num_write_kb":253954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.848424+0000","last_change":"2024-05-30T17:04:51.955992+0000","last_active":"2024-05-30T17:05:34.848424+0000","last_peered":"2024-05-30T17:05:34.848424+0000","last_clean":"2024-05-30T17:05:34.848424+0000","last_became_active":"2024-05-30T17:04:51.955666+0000","last_became_peered":"2024-05-30T17:04:51.955666+0000","last_unstale":"2024-05-30T17:05:34.848424+0000","last_undegraded":"2024-05-30T17:05:34.848424+0000","last_fullsized":"2024-05-30T17:05:34.848424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:03.343211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'122","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.596305+0000","last_change":"2024-05-30T17:04:52.984705+0000","last_active":"2024-05-30T17:07:34.596305+0000","last_peered":"2024-05-30T17:07:34.596305+0000","last_clean":"2024-05-30T17:07:34.596305+0000","last_became_active":"2024-05-30T17:04:52.984409+0000","last_became_peered":"2024-05-30T17:04:52.984409+0000","last_unstale":"2024-05-30T17:07:34.596305+0000","last_undegraded":"2024-05-30T17:07:34.596305+0000","last_fullsized":"2024-05-30T17:07:34.596305+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":39,"num_read_kb":159744,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.123168+0000","last_change":"2024-05-30T17:04:51.952573+0000","last_active":"2024-05-30T17:06:00.123168+0000","last_peered":"2024-05-30T17:06:00.123168+0000","last_clean":"2024-05-30T17:06:00.123168+0000","last_became_active":"2024-05-30T17:04:51.948281+0000","last_became_peered":"2024-05-30T17:04:51.948281+0000","last_unstale":"2024-05-30T17:06:00.123168+0000","last_undegraded":"2024-05-30T17:06:00.123168+0000","last_fullsized":"2024-05-30T17:06:00.123168+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:26.183919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'118","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859139+0000","last_change":"2024-05-30T17:04:52.986184+0000","last_active":"2024-05-30T17:07:42.859139+0000","last_peered":"2024-05-30T17:07:42.859139+0000","last_clean":"2024-05-30T17:07:42.859139+0000","last_became_active":"2024-05-30T17:04:52.985889+0000","last_became_peered":"2024-05-30T17:04:52.985889+0000","last_unstale":"2024-05-30T17:07:42.859139+0000","last_undegraded":"2024-05-30T17:07:42.859139+0000","last_fullsized":"2024-05-30T17:07:42.859139+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:27.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881146,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":59,"num_write_kb":229378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.949232+0000","last_change":"2024-05-30T17:04:51.954829+0000","last_active":"2024-05-30T17:05:34.949232+0000","last_peered":"2024-05-30T17:05:34.949232+0000","last_clean":"2024-05-30T17:05:34.949232+0000","last_became_active":"2024-05-30T17:04:51.954434+0000","last_became_peered":"2024-05-30T17:04:51.954434+0000","last_unstale":"2024-05-30T17:05:34.949232+0000","last_undegraded":"2024-05-30T17:05:34.949232+0000","last_fullsized":"2024-05-30T17:05:34.949232+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:23.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'124","reported_seq":221,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.863792+0000","last_change":"2024-05-30T17:04:52.989346+0000","last_active":"2024-05-30T17:07:42.863792+0000","last_peered":"2024-05-30T17:07:42.863792+0000","last_clean":"2024-05-30T17:07:42.863792+0000","last_became_active":"2024-05-30T17:04:52.989051+0000","last_became_peered":"2024-05-30T17:04:52.989051+0000","last_unstale":"2024-05-30T17:07:42.863792+0000","last_undegraded":"2024-05-30T17:07:42.863792+0000","last_fullsized":"2024-05-30T17:07:42.863792+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:36.128167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'28","reported_seq":47,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:29.184425+0000","last_change":"2024-05-30T17:04:51.938158+0000","last_active":"2024-05-30T17:06:29.184425+0000","last_peered":"2024-05-30T17:06:29.184425+0000","last_clean":"2024-05-30T17:06:29.184425+0000","last_became_active":"2024-05-30T17:04:51.937857+0000","last_became_peered":"2024-05-30T17:04:51.937857+0000","last_unstale":"2024-05-30T17:06:29.184425+0000","last_undegraded":"2024-05-30T17:06:29.184425+0000","last_fullsized":"2024-05-30T17:06:29.184425+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:25.458335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'134","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.854431+0000","last_change":"2024-05-30T17:04:52.995423+0000","last_active":"2024-05-30T17:07:42.854431+0000","last_peered":"2024-05-30T17:07:42.854431+0000","last_clean":"2024-05-30T17:07:42.854431+0000","last_became_active":"2024-05-30T17:04:52.995128+0000","last_became_peered":"2024-05-30T17:04:52.995128+0000","last_unstale":"2024-05-30T17:07:42.854431+0000","last_undegraded":"2024-05-30T17:07:42.854431+0000","last_fullsized":"2024-05-30T17:07:42.854431+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:47.501828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":78,"num_write_kb":303106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'436","reported_seq":455,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593864+0000","last_change":"2024-05-30T17:04:51.953550+0000","last_active":"2024-05-30T17:07:34.593864+0000","last_peered":"2024-05-30T17:07:34.593864+0000","last_clean":"2024-05-30T17:07:34.593864+0000","last_became_active":"2024-05-30T17:04:51.953276+0000","last_became_peered":"2024-05-30T17:04:51.953276+0000","last_unstale":"2024-05-30T17:07:34.593864+0000","last_undegraded":"2024-05-30T17:07:34.593864+0000","last_fullsized":"2024-05-30T17:07:34.593864+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:07:44.468185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1698517,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":435,"num_write_kb":1831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'149","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600297+0000","last_change":"2024-05-30T17:04:52.988845+0000","last_active":"2024-05-30T17:07:34.600297+0000","last_peered":"2024-05-30T17:07:34.600297+0000","last_clean":"2024-05-30T17:07:34.600297+0000","last_became_active":"2024-05-30T17:04:52.986757+0000","last_became_peered":"2024-05-30T17:04:52.986757+0000","last_unstale":"2024-05-30T17:07:34.600297+0000","last_undegraded":"2024-05-30T17:07:34.600297+0000","last_fullsized":"2024-05-30T17:07:34.600297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385875968,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":92,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.853652+0000","last_change":"2024-05-30T17:04:51.944552+0000","last_active":"2024-05-30T17:05:34.853652+0000","last_peered":"2024-05-30T17:05:34.853652+0000","last_clean":"2024-05-30T17:05:34.853652+0000","last_became_active":"2024-05-30T17:04:51.944167+0000","last_became_peered":"2024-05-30T17:04:51.944167+0000","last_unstale":"2024-05-30T17:05:34.853652+0000","last_undegraded":"2024-05-30T17:05:34.853652+0000","last_fullsized":"2024-05-30T17:05:34.853652+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:17:06.906161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'121","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593998+0000","last_change":"2024-05-30T17:04:52.991186+0000","last_active":"2024-05-30T17:07:34.593998+0000","last_peered":"2024-05-30T17:07:34.593998+0000","last_clean":"2024-05-30T17:07:34.593998+0000","last_became_active":"2024-05-30T17:04:52.990049+0000","last_became_peered":"2024-05-30T17:04:52.990049+0000","last_unstale":"2024-05-30T17:07:34.593998+0000","last_undegraded":"2024-05-30T17:07:34.593998+0000","last_fullsized":"2024-05-30T17:07:34.593998+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:28:23.896291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.848017+0000","last_change":"2024-05-30T17:04:51.947216+0000","last_active":"2024-05-30T17:05:34.848017+0000","last_peered":"2024-05-30T17:05:34.848017+0000","last_clean":"2024-05-30T17:05:34.848017+0000","last_became_active":"2024-05-30T17:04:51.946868+0000","last_became_peered":"2024-05-30T17:04:51.946868+0000","last_unstale":"2024-05-30T17:05:34.848017+0000","last_undegraded":"2024-05-30T17:05:34.848017+0000","last_fullsized":"2024-05-30T17:05:34.848017+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:42:48.589336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'140","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592617+0000","last_change":"2024-05-30T17:04:52.986670+0000","last_active":"2024-05-30T17:07:34.592617+0000","last_peered":"2024-05-30T17:07:34.592617+0000","last_clean":"2024-05-30T17:07:34.592617+0000","last_became_active":"2024-05-30T17:04:52.986293+0000","last_became_peered":"2024-05-30T17:04:52.986293+0000","last_unstale":"2024-05-30T17:07:34.592617+0000","last_undegraded":"2024-05-30T17:07:34.592617+0000","last_fullsized":"2024-05-30T17:07:34.592617+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:28.097184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293317,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":90,"num_write_kb":364545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":27,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.955398+0000","last_change":"2024-05-30T17:04:51.955924+0000","last_active":"2024-05-30T17:05:34.955398+0000","last_peered":"2024-05-30T17:05:34.955398+0000","last_clean":"2024-05-30T17:05:34.955398+0000","last_became_active":"2024-05-30T17:04:51.955542+0000","last_became_peered":"2024-05-30T17:04:51.955542+0000","last_unstale":"2024-05-30T17:05:34.955398+0000","last_undegraded":"2024-05-30T17:05:34.955398+0000","last_fullsized":"2024-05-30T17:05:34.955398+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:38.543340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'107","reported_seq":149,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595390+0000","last_change":"2024-05-30T17:04:53.001644+0000","last_active":"2024-05-30T17:07:34.595390+0000","last_peered":"2024-05-30T17:07:34.595390+0000","last_clean":"2024-05-30T17:07:34.595390+0000","last_became_active":"2024-05-30T17:04:52.999924+0000","last_became_peered":"2024-05-30T17:04:52.999924+0000","last_unstale":"2024-05-30T17:07:34.595390+0000","last_undegraded":"2024-05-30T17:07:34.595390+0000","last_fullsized":"2024-05-30T17:07:34.595390+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:17:36.947839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.850216+0000","last_change":"2024-05-30T17:04:51.948540+0000","last_active":"2024-05-30T17:05:34.850216+0000","last_peered":"2024-05-30T17:05:34.850216+0000","last_clean":"2024-05-30T17:05:34.850216+0000","last_became_active":"2024-05-30T17:04:51.947928+0000","last_became_peered":"2024-05-30T17:04:51.947928+0000","last_unstale":"2024-05-30T17:05:34.850216+0000","last_undegraded":"2024-05-30T17:05:34.850216+0000","last_fullsized":"2024-05-30T17:05:34.850216+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:10:19.343810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'133","reported_seq":231,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.853021+0000","last_change":"2024-05-30T17:04:52.976220+0000","last_active":"2024-05-30T17:07:42.853021+0000","last_peered":"2024-05-30T17:07:42.853021+0000","last_clean":"2024-05-30T17:07:42.853021+0000","last_became_active":"2024-05-30T17:04:52.974987+0000","last_became_peered":"2024-05-30T17:04:52.974987+0000","last_unstale":"2024-05-30T17:07:42.853021+0000","last_undegraded":"2024-05-30T17:07:42.853021+0000","last_fullsized":"2024-05-30T17:07:42.853021+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:15.443712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360710144,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":81,"num_read_kb":331776,"num_write":86,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855580+0000","last_change":"2024-05-30T17:04:51.950294+0000","last_active":"2024-05-30T17:05:34.855580+0000","last_peered":"2024-05-30T17:05:34.855580+0000","last_clean":"2024-05-30T17:05:34.855580+0000","last_became_active":"2024-05-30T17:04:51.949704+0000","last_became_peered":"2024-05-30T17:04:51.949704+0000","last_unstale":"2024-05-30T17:05:34.855580+0000","last_undegraded":"2024-05-30T17:05:34.855580+0000","last_fullsized":"2024-05-30T17:05:34.855580+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:46.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'135","reported_seq":215,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.588249+0000","last_change":"2024-05-30T17:04:52.989374+0000","last_active":"2024-05-30T17:07:34.588249+0000","last_peered":"2024-05-30T17:07:34.588249+0000","last_clean":"2024-05-30T17:07:34.588249+0000","last_became_active":"2024-05-30T17:04:52.989083+0000","last_became_peered":"2024-05-30T17:04:52.989083+0000","last_unstale":"2024-05-30T17:07:34.588249+0000","last_undegraded":"2024-05-30T17:07:34.588249+0000","last_fullsized":"2024-05-30T17:07:34.588249+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:35.524809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":63,"num_read_kb":237573,"num_write":85,"num_write_kb":307205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.935270+0000","last_change":"2024-05-30T17:04:51.942441+0000","last_active":"2024-05-30T17:05:34.935270+0000","last_peered":"2024-05-30T17:05:34.935270+0000","last_clean":"2024-05-30T17:05:34.935270+0000","last_became_active":"2024-05-30T17:04:51.942121+0000","last_became_peered":"2024-05-30T17:04:51.942121+0000","last_unstale":"2024-05-30T17:05:34.935270+0000","last_undegraded":"2024-05-30T17:05:34.935270+0000","last_fullsized":"2024-05-30T17:05:34.935270+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:27.082196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'128","reported_seq":227,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.582512+0000","last_change":"2024-05-30T17:04:52.994569+0000","last_active":"2024-05-30T17:07:34.582512+0000","last_peered":"2024-05-30T17:07:34.582512+0000","last_clean":"2024-05-30T17:07:34.582512+0000","last_became_active":"2024-05-30T17:04:52.994290+0000","last_became_peered":"2024-05-30T17:04:52.994290+0000","last_unstale":"2024-05-30T17:07:34.582512+0000","last_undegraded":"2024-05-30T17:07:34.582512+0000","last_fullsized":"2024-05-30T17:07:34.582512+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849744+0000","last_change":"2024-05-30T17:04:51.945428+0000","last_active":"2024-05-30T17:05:34.849744+0000","last_peered":"2024-05-30T17:05:34.849744+0000","last_clean":"2024-05-30T17:05:34.849744+0000","last_became_active":"2024-05-30T17:04:51.945142+0000","last_became_peered":"2024-05-30T17:04:51.945142+0000","last_unstale":"2024-05-30T17:05:34.849744+0000","last_undegraded":"2024-05-30T17:05:34.849744+0000","last_fullsized":"2024-05-30T17:05:34.849744+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:25:46.375765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'128","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597686+0000","last_change":"2024-05-30T17:04:52.993514+0000","last_active":"2024-05-30T17:07:34.597686+0000","last_peered":"2024-05-30T17:07:34.597686+0000","last_clean":"2024-05-30T17:07:34.597686+0000","last_became_active":"2024-05-30T17:04:52.988660+0000","last_became_peered":"2024-05-30T17:04:52.988660+0000","last_unstale":"2024-05-30T17:07:34.597686+0000","last_undegraded":"2024-05-30T17:07:34.597686+0000","last_fullsized":"2024-05-30T17:07:34.597686+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'3","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.150462+0000","last_change":"2024-05-30T17:04:51.943410+0000","last_active":"2024-05-30T17:06:00.150462+0000","last_peered":"2024-05-30T17:06:00.150462+0000","last_clean":"2024-05-30T17:06:00.150462+0000","last_became_active":"2024-05-30T17:04:51.943085+0000","last_became_peered":"2024-05-30T17:04:51.943085+0000","last_unstale":"2024-05-30T17:06:00.150462+0000","last_undegraded":"2024-05-30T17:06:00.150462+0000","last_fullsized":"2024-05-30T17:06:00.150462+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:18.313092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'120","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.588879+0000","last_change":"2024-05-30T17:04:52.992939+0000","last_active":"2024-05-30T17:07:34.588879+0000","last_peered":"2024-05-30T17:07:34.588879+0000","last_clean":"2024-05-30T17:07:34.588879+0000","last_became_active":"2024-05-30T17:04:52.989409+0000","last_became_peered":"2024-05-30T17:04:52.989409+0000","last_unstale":"2024-05-30T17:07:34.588879+0000","last_undegraded":"2024-05-30T17:07:34.588879+0000","last_fullsized":"2024-05-30T17:07:34.588879+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:41.892452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795845,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":72,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.948183+0000","last_change":"2024-05-30T17:04:51.948194+0000","last_active":"2024-05-30T17:05:34.948183+0000","last_peered":"2024-05-30T17:05:34.948183+0000","last_clean":"2024-05-30T17:05:34.948183+0000","last_became_active":"2024-05-30T17:04:51.947919+0000","last_became_peered":"2024-05-30T17:04:51.947919+0000","last_unstale":"2024-05-30T17:05:34.948183+0000","last_undegraded":"2024-05-30T17:05:34.948183+0000","last_fullsized":"2024-05-30T17:05:34.948183+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:58.823357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'134","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600019+0000","last_change":"2024-05-30T17:04:52.981350+0000","last_active":"2024-05-30T17:07:34.600019+0000","last_peered":"2024-05-30T17:07:34.600019+0000","last_clean":"2024-05-30T17:07:34.600019+0000","last_became_active":"2024-05-30T17:04:52.978531+0000","last_became_peered":"2024-05-30T17:04:52.978531+0000","last_unstale":"2024-05-30T17:07:34.600019+0000","last_undegraded":"2024-05-30T17:07:34.600019+0000","last_fullsized":"2024-05-30T17:07:34.600019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":48,"num_read_kb":188418,"num_write":83,"num_write_kb":323586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.947541+0000","last_change":"2024-05-30T17:04:51.950726+0000","last_active":"2024-05-30T17:05:34.947541+0000","last_peered":"2024-05-30T17:05:34.947541+0000","last_clean":"2024-05-30T17:05:34.947541+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:05:34.947541+0000","last_undegraded":"2024-05-30T17:05:34.947541+0000","last_fullsized":"2024-05-30T17:05:34.947541+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:17.049750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'123","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859963+0000","last_change":"2024-05-30T17:05:35.214281+0000","last_active":"2024-05-30T17:07:42.859963+0000","last_peered":"2024-05-30T17:07:42.859963+0000","last_clean":"2024-05-30T17:07:42.859963+0000","last_became_active":"2024-05-30T17:05:34.860839+0000","last_became_peered":"2024-05-30T17:05:34.860839+0000","last_unstale":"2024-05-30T17:07:42.859963+0000","last_undegraded":"2024-05-30T17:07:42.859963+0000","last_fullsized":"2024-05-30T17:07:42.859963+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:11:46.508726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":69,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.854184+0000","last_change":"2024-05-30T17:04:51.951054+0000","last_active":"2024-05-30T17:05:34.854184+0000","last_peered":"2024-05-30T17:05:34.854184+0000","last_clean":"2024-05-30T17:05:34.854184+0000","last_became_active":"2024-05-30T17:04:51.950364+0000","last_became_peered":"2024-05-30T17:04:51.950364+0000","last_unstale":"2024-05-30T17:05:34.854184+0000","last_undegraded":"2024-05-30T17:05:34.854184+0000","last_fullsized":"2024-05-30T17:05:34.854184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:11.010697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'124","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593805+0000","last_change":"2024-05-30T17:04:52.967099+0000","last_active":"2024-05-30T17:07:34.593805+0000","last_peered":"2024-05-30T17:07:34.593805+0000","last_clean":"2024-05-30T17:07:34.593805+0000","last_became_active":"2024-05-30T17:04:52.966035+0000","last_became_peered":"2024-05-30T17:04:52.966035+0000","last_unstale":"2024-05-30T17:07:34.593805+0000","last_undegraded":"2024-05-30T17:07:34.593805+0000","last_fullsized":"2024-05-30T17:07:34.593805+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:24.565166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.853230+0000","last_change":"2024-05-30T17:04:51.949867+0000","last_active":"2024-05-30T17:05:34.853230+0000","last_peered":"2024-05-30T17:05:34.853230+0000","last_clean":"2024-05-30T17:05:34.853230+0000","last_became_active":"2024-05-30T17:04:51.949575+0000","last_became_peered":"2024-05-30T17:04:51.949575+0000","last_unstale":"2024-05-30T17:05:34.853230+0000","last_undegraded":"2024-05-30T17:05:34.853230+0000","last_fullsized":"2024-05-30T17:05:34.853230+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:58:11.353946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'126","reported_seq":230,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592727+0000","last_change":"2024-05-30T17:04:52.998729+0000","last_active":"2024-05-30T17:07:34.592727+0000","last_peered":"2024-05-30T17:07:34.592727+0000","last_clean":"2024-05-30T17:07:34.592727+0000","last_became_active":"2024-05-30T17:04:52.997726+0000","last_became_peered":"2024-05-30T17:04:52.997726+0000","last_unstale":"2024-05-30T17:07:34.592727+0000","last_undegraded":"2024-05-30T17:07:34.592727+0000","last_fullsized":"2024-05-30T17:07:34.592727+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:32:14.601932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"134'49","reported_seq":68,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:35.039807+0000","last_change":"2024-05-30T17:04:51.949751+0000","last_active":"2024-05-30T17:07:35.039807+0000","last_peered":"2024-05-30T17:07:35.039807+0000","last_clean":"2024-05-30T17:07:35.039807+0000","last_became_active":"2024-05-30T17:04:51.949295+0000","last_became_peered":"2024-05-30T17:04:51.949295+0000","last_unstale":"2024-05-30T17:07:35.039807+0000","last_undegraded":"2024-05-30T17:07:35.039807+0000","last_fullsized":"2024-05-30T17:07:35.039807+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:41.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'129","reported_seq":243,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595138+0000","last_change":"2024-05-30T17:04:52.985036+0000","last_active":"2024-05-30T17:07:34.595138+0000","last_peered":"2024-05-30T17:07:34.595138+0000","last_clean":"2024-05-30T17:07:34.595138+0000","last_became_active":"2024-05-30T17:04:52.984675+0000","last_became_peered":"2024-05-30T17:04:52.984675+0000","last_unstale":"2024-05-30T17:07:34.595138+0000","last_undegraded":"2024-05-30T17:07:34.595138+0000","last_fullsized":"2024-05-30T17:07:34.595138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":397312,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849721+0000","last_change":"2024-05-30T17:04:51.952967+0000","last_active":"2024-05-30T17:05:34.849721+0000","last_peered":"2024-05-30T17:05:34.849721+0000","last_clean":"2024-05-30T17:05:34.849721+0000","last_became_active":"2024-05-30T17:04:51.952631+0000","last_became_peered":"2024-05-30T17:04:51.952631+0000","last_unstale":"2024-05-30T17:05:34.849721+0000","last_undegraded":"2024-05-30T17:05:34.849721+0000","last_fullsized":"2024-05-30T17:05:34.849721+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:23.747059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'123","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592347+0000","last_change":"2024-05-30T17:04:52.990963+0000","last_active":"2024-05-30T17:07:34.592347+0000","last_peered":"2024-05-30T17:07:34.592347+0000","last_clean":"2024-05-30T17:07:34.592347+0000","last_became_active":"2024-05-30T17:04:52.990476+0000","last_became_peered":"2024-05-30T17:04:52.990476+0000","last_unstale":"2024-05-30T17:07:34.592347+0000","last_undegraded":"2024-05-30T17:07:34.592347+0000","last_fullsized":"2024-05-30T17:07:34.592347+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:59.838344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378750,"num_objects":77,"num_object_clones":1,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":1,"num_read":54,"num_read_kb":217089,"num_write":79,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855975+0000","last_change":"2024-05-30T17:04:51.933352+0000","last_active":"2024-05-30T17:05:34.855975+0000","last_peered":"2024-05-30T17:05:34.855975+0000","last_clean":"2024-05-30T17:05:34.855975+0000","last_became_active":"2024-05-30T17:04:51.933061+0000","last_became_peered":"2024-05-30T17:04:51.933061+0000","last_unstale":"2024-05-30T17:05:34.855975+0000","last_undegraded":"2024-05-30T17:05:34.855975+0000","last_fullsized":"2024-05-30T17:05:34.855975+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:20.242562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'134","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.870019+0000","last_change":"2024-05-30T17:04:52.982699+0000","last_active":"2024-05-30T17:07:42.870019+0000","last_peered":"2024-05-30T17:07:42.870019+0000","last_clean":"2024-05-30T17:07:42.870019+0000","last_became_active":"2024-05-30T17:04:52.981542+0000","last_became_peered":"2024-05-30T17:04:52.981542+0000","last_unstale":"2024-05-30T17:07:42.870019+0000","last_undegraded":"2024-05-30T17:07:42.870019+0000","last_fullsized":"2024-05-30T17:07:42.870019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:12.025684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":76,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849081+0000","last_change":"2024-05-30T17:04:51.947181+0000","last_active":"2024-05-30T17:05:34.849081+0000","last_peered":"2024-05-30T17:05:34.849081+0000","last_clean":"2024-05-30T17:05:34.849081+0000","last_became_active":"2024-05-30T17:04:51.946882+0000","last_became_peered":"2024-05-30T17:04:51.946882+0000","last_unstale":"2024-05-30T17:05:34.849081+0000","last_undegraded":"2024-05-30T17:05:34.849081+0000","last_fullsized":"2024-05-30T17:05:34.849081+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:47:52.431592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'157","reported_seq":257,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860002+0000","last_change":"2024-05-30T17:04:52.992399+0000","last_active":"2024-05-30T17:07:42.860002+0000","last_peered":"2024-05-30T17:07:42.860002+0000","last_clean":"2024-05-30T17:07:42.860002+0000","last_became_active":"2024-05-30T17:04:52.992040+0000","last_became_peered":"2024-05-30T17:04:52.992040+0000","last_unstale":"2024-05-30T17:07:42.860002+0000","last_undegraded":"2024-05-30T17:07:42.860002+0000","last_fullsized":"2024-05-30T17:07:42.860002+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:16.335923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":411041792,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":98,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855981+0000","last_change":"2024-05-30T17:04:51.952089+0000","last_active":"2024-05-30T17:05:34.855981+0000","last_peered":"2024-05-30T17:05:34.855981+0000","last_clean":"2024-05-30T17:05:34.855981+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:05:34.855981+0000","last_undegraded":"2024-05-30T17:05:34.855981+0000","last_fullsized":"2024-05-30T17:05:34.855981+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:03:20.313618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'131","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595785+0000","last_change":"2024-05-30T17:04:52.988572+0000","last_active":"2024-05-30T17:07:34.595785+0000","last_peered":"2024-05-30T17:07:34.595785+0000","last_clean":"2024-05-30T17:07:34.595785+0000","last_became_active":"2024-05-30T17:04:52.987684+0000","last_became_peered":"2024-05-30T17:04:52.987684+0000","last_unstale":"2024-05-30T17:07:34.595785+0000","last_undegraded":"2024-05-30T17:07:34.595785+0000","last_fullsized":"2024-05-30T17:07:34.595785+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:26:52.920204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":82,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'35","reported_seq":88,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:29.176344+0000","last_change":"2024-05-30T17:04:51.949334+0000","last_active":"2024-05-30T17:06:29.176344+0000","last_peered":"2024-05-30T17:06:29.176344+0000","last_clean":"2024-05-30T17:06:29.176344+0000","last_became_active":"2024-05-30T17:04:51.948995+0000","last_became_peered":"2024-05-30T17:04:51.948995+0000","last_unstale":"2024-05-30T17:06:29.176344+0000","last_undegraded":"2024-05-30T17:06:29.176344+0000","last_fullsized":"2024-05-30T17:06:29.176344+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:05.331584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4250,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'129","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593040+0000","last_change":"2024-05-30T17:04:52.992411+0000","last_active":"2024-05-30T17:07:34.593040+0000","last_peered":"2024-05-30T17:07:34.593040+0000","last_clean":"2024-05-30T17:07:34.593040+0000","last_became_active":"2024-05-30T17:04:52.991997+0000","last_became_peered":"2024-05-30T17:04:52.991997+0000","last_unstale":"2024-05-30T17:07:34.593040+0000","last_undegraded":"2024-05-30T17:07:34.593040+0000","last_fullsized":"2024-05-30T17:07:34.593040+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":402653184,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":96,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.854388+0000","last_change":"2024-05-30T17:04:51.946511+0000","last_active":"2024-05-30T17:05:34.854388+0000","last_peered":"2024-05-30T17:05:34.854388+0000","last_clean":"2024-05-30T17:05:34.854388+0000","last_became_active":"2024-05-30T17:04:51.946173+0000","last_became_peered":"2024-05-30T17:04:51.946173+0000","last_unstale":"2024-05-30T17:05:34.854388+0000","last_undegraded":"2024-05-30T17:05:34.854388+0000","last_fullsized":"2024-05-30T17:05:34.854388+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:08.206744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'122","reported_seq":192,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.864677+0000","last_change":"2024-05-30T17:04:52.992639+0000","last_active":"2024-05-30T17:07:42.864677+0000","last_peered":"2024-05-30T17:07:42.864677+0000","last_clean":"2024-05-30T17:07:42.864677+0000","last_became_active":"2024-05-30T17:04:52.992202+0000","last_became_peered":"2024-05-30T17:04:52.992202+0000","last_unstale":"2024-05-30T17:07:42.864677+0000","last_undegraded":"2024-05-30T17:07:42.864677+0000","last_fullsized":"2024-05-30T17:07:42.864677+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:34.869318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":53,"num_read_kb":217088,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.856152+0000","last_change":"2024-05-30T17:04:51.951966+0000","last_active":"2024-05-30T17:05:34.856152+0000","last_peered":"2024-05-30T17:05:34.856152+0000","last_clean":"2024-05-30T17:05:34.856152+0000","last_became_active":"2024-05-30T17:04:51.951484+0000","last_became_peered":"2024-05-30T17:04:51.951484+0000","last_unstale":"2024-05-30T17:05:34.856152+0000","last_undegraded":"2024-05-30T17:05:34.856152+0000","last_fullsized":"2024-05-30T17:05:34.856152+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:41.627294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":388,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:44.508422+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:05:44.508422+0000","last_peered":"2024-05-30T17:05:44.508422+0000","last_clean":"2024-05-30T17:05:44.508422+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:05:44.508422+0000","last_undegraded":"2024-05-30T17:05:44.508422+0000","last_fullsized":"2024-05-30T17:05:44.508422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":20912801557,"num_objects":4996,"num_object_clones":1,"num_object_copies":9992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":1,"num_read":4290,"num_read_kb":17367090,"num_write":5064,"num_write_kb":20422699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41825673216,"data_stored":41825603114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8182,"ondisk_log_size":8182,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1703565,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":642,"num_write_kb":1932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3473408,"data_stored":3407379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509731,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5955000,"kb_used_data":5811440,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":88416840,"statfs":{"total":96636764160,"available":90538844160,"internally_reserved":0,"allocated":5950914560,"data_stored":5949962393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509731,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6190836,"kb_used_data":6051436,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":88181004,"statfs":{"total":96636764160,"available":90297348096,"internally_reserved":0,"allocated":6196670464,"data_stored":6195723625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509732,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5015352,"kb_used_data":4871792,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":89356488,"statfs":{"total":96636764160,"available":91501043712,"internally_reserved":0,"allocated":4988715008,"data_stored":4987768020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509732,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6950260,"kb_used_data":6792812,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":87421580,"statfs":{"total":96636764160,"available":89519697920,"internally_reserved":0,"allocated":6955839488,"data_stored":6954892805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509731,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4728548,"kb_used_data":4589148,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89643292,"statfs":{"total":96636764160,"available":91794731008,"internally_reserved":0,"allocated":4699287552,"data_stored":4698356383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542435,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4575160,"kb_used_data":4435760,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89796680,"statfs":{"total":96636764160,"available":91951800320,"internally_reserved":0,"allocated":4542218240,"data_stored":4541262732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542436,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4261684,"kb_used_data":4118124,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90110156,"statfs":{"total":96636764160,"available":92272799744,"internally_reserved":0,"allocated":4216958976,"data_stored":4216012047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4344296,"kb_used_data":4192416,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":90027544,"statfs":{"total":96636764160,"available":92188205056,"internally_reserved":0,"allocated":4293033984,"data_stored":4292099392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1716224,"data_stored":1702767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1720320,"data_stored":1699404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4290777088,"data_stored":4290773253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4697624576,"data_stored":4697620612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6954168320,"data_stored":6954156418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4987035648,"data_stored":4987027839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6195003392,"data_stored":6194987784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4215287808,"data_stored":4215276164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5947535360,"data_stored":5947523855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4538241024,"data_stored":4538237189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:07:45.262 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-30T17:07:45.262 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:07:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.518+0000 7f7d1e200640 1 Processor -- start 2024-05-30T17:07:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.518+0000 7f7d1e200640 1 -- start start 2024-05-30T17:07:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.520+0000 7f7d1e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7d1815b3a0 0x7f7d180b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.520+0000 7f7d1e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d18073310 0x7f7d1815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.520+0000 7f7d1e200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d180afd60 con 0x7f7d1806dc20 2024-05-30T17:07:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.520+0000 7f7d1e200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f7d1806f980 con 0x7f7d18073310 2024-05-30T17:07:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.520+0000 7f7d1e200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7d180ae0c0 con 0x7f7d1815b3a0 2024-05-30T17:07:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.520+0000 7f7d17e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7d1806dc20 0x7f7d18072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43834/0 (socket says 172.21.15.67:43834) 2024-05-30T17:07:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.520+0000 7f7d17e00640 1 -- 172.21.15.67:0/2971430118 learned_addr learned my addr 172.21.15.67:0/2971430118 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:45.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.520+0000 7f7d1d800640 1 --2- 172.21.15.67:0/2971430118 >> v2:172.21.15.67:3300/0 conn(0x7f7d18073310 0x7f7d1815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.521+0000 7f7d1ce00640 1 --2- 172.21.15.67:0/2971430118 >> v2:172.21.15.169:3300/0 conn(0x7f7d1815b3a0 0x7f7d180b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.521+0000 7f7d1d800640 1 -- 172.21.15.67:0/2971430118 >> v2:172.21.15.169:3300/0 conn(0x7f7d1815b3a0 msgr2=0x7f7d180b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.521+0000 7f7d1d800640 1 --2- 172.21.15.67:0/2971430118 >> v2:172.21.15.169:3300/0 conn(0x7f7d1815b3a0 0x7f7d180b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.521+0000 7f7d1d800640 1 -- 172.21.15.67:0/2971430118 >> v1:172.21.15.67:6789/0 conn(0x7f7d1806dc20 legacy=0x7f7d18072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.521+0000 7f7d1d800640 1 -- 172.21.15.67:0/2971430118 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d180af7a0 con 0x7f7d18073310 2024-05-30T17:07:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.522+0000 7f7d1d800640 1 --2- 172.21.15.67:0/2971430118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d18073310 0x7f7d1815f9a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f7d18072bd0 tx=0x7f7d1002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55056d3b5a786382 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.522+0000 7f7d17400640 1 -- 172.21.15.67:0/2971430118 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10007740 con 0x7f7d18073310 2024-05-30T17:07:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d17400640 1 -- 172.21.15.67:0/2971430118 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d100078e0 con 0x7f7d18073310 2024-05-30T17:07:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d17400640 1 -- 172.21.15.67:0/2971430118 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10007cb0 con 0x7f7d18073310 2024-05-30T17:07:45.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d1e200640 1 -- 172.21.15.67:0/2971430118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d18073310 msgr2=0x7f7d1815f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d1e200640 1 --2- 172.21.15.67:0/2971430118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d18073310 0x7f7d1815f9a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f7d18072bd0 tx=0x7f7d1002a680 comp rx=0 tx=0).stop 2024-05-30T17:07:45.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d1e200640 1 -- 172.21.15.67:0/2971430118 shutdown_connections 2024-05-30T17:07:45.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d1e200640 1 --2- 172.21.15.67:0/2971430118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d18073310 0x7f7d1815f9a0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d1e200640 1 --2- 172.21.15.67:0/2971430118 >> v2:172.21.15.169:3300/0 conn(0x7f7d1815b3a0 0x7f7d180b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d1e200640 1 -- 172.21.15.67:0/2971430118 >> 172.21.15.67:0/2971430118 conn(0x7f7d180b1fc0 msgr2=0x7f7d180b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d1e200640 1 -- 172.21.15.67:0/2971430118 shutdown_connections 2024-05-30T17:07:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.523+0000 7f7d1e200640 1 -- 172.21.15.67:0/2971430118 wait complete. 2024-05-30T17:07:45.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.525+0000 7f7d1e200640 1 Processor -- start 2024-05-30T17:07:45.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.525+0000 7f7d1e200640 1 -- start start 2024-05-30T17:07:45.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.526+0000 7f7d1e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d1806dc20 0x7f7d18155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.526+0000 7f7d1e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d18073310 0x7f7d181f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.526+0000 7f7d1e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d1815b3a0 0x7f7d18150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.526+0000 7f7d1e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1815b780 con 0x7f7d1815b3a0 2024-05-30T17:07:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.526+0000 7f7d1e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1815fee0 con 0x7f7d18073310 2024-05-30T17:07:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.526+0000 7f7d1e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d180ae550 con 0x7f7d1806dc20 2024-05-30T17:07:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.526+0000 7f7d1d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d1815b3a0 0x7f7d18150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.526+0000 7f7d1d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d1815b3a0 0x7f7d18150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49268/0 (socket says 172.21.15.67:49268) 2024-05-30T17:07:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.526+0000 7f7d1d800640 1 -- 172.21.15.67:0/3584137997 learned_addr learned my addr 172.21.15.67:0/3584137997 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d1ce00640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d1806dc20 0x7f7d18155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d17e00640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d18073310 0x7f7d181f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d1d800640 1 -- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d1806dc20 msgr2=0x7f7d18155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d1d800640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d1806dc20 0x7f7d18155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d1d800640 1 -- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d18073310 msgr2=0x7f7d181f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d1d800640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d18073310 0x7f7d181f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d1d800640 1 -- 172.21.15.67:0/3584137997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d181f4d40 con 0x7f7d1815b3a0 2024-05-30T17:07:45.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d1ce00640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d1806dc20 0x7f7d18155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:45.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d17e00640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d18073310 0x7f7d181f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:45.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.527+0000 7f7d1d800640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d1815b3a0 0x7f7d18150b10 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f7d1002ac70 tx=0x7f7d10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:45.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.528+0000 7f7d14c00640 1 -- 172.21.15.67:0/3584137997 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10040070 con 0x7f7d1815b3a0 2024-05-30T17:07:45.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.528+0000 7f7d14c00640 1 -- 172.21.15.67:0/3584137997 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d10031bc0 con 0x7f7d1815b3a0 2024-05-30T17:07:45.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.528+0000 7f7d14c00640 1 -- 172.21.15.67:0/3584137997 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10009450 con 0x7f7d1815b3a0 2024-05-30T17:07:45.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.528+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d181f5020 con 0x7f7d1815b3a0 2024-05-30T17:07:45.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.528+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d181549d0 con 0x7f7d1815b3a0 2024-05-30T17:07:45.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.529+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d180689e0 con 0x7f7d1815b3a0 2024-05-30T17:07:45.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.530+0000 7f7d14c00640 1 -- 172.21.15.67:0/3584137997 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f7d10039040 con 0x7f7d1815b3a0 2024-05-30T17:07:45.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.532+0000 7f7d14c00640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cf806e510 0x7f7cf80709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:45.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.532+0000 7f7d14c00640 1 -- 172.21.15.67:0/3584137997 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f7d100ac330 con 0x7f7d1815b3a0 2024-05-30T17:07:45.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.532+0000 7f7d1ce00640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cf806e510 0x7f7cf80709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:45.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.533+0000 7f7d1ce00640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cf806e510 0x7f7cf80709d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:45.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.536+0000 7f7d14c00640 1 -- 172.21.15.67:0/3584137997 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d1007c260 con 0x7f7d1815b3a0 2024-05-30T17:07:45.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.739+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d180b02b0 con 0x7f7cf806e510 2024-05-30T17:07:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.758+0000 7f7d14c00640 1 -- 172.21.15.67:0/3584137997 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254973 (secure 0 0 0) 0x7f7d180b02b0 con 0x7f7cf806e510 2024-05-30T17:07:45.762 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:45.764 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:07:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cf806e510 msgr2=0x7f7cf80709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cf806e510 0x7f7cf80709d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08008040 comp rx=0 tx=0).stop 2024-05-30T17:07:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d1815b3a0 msgr2=0x7f7d18150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d1815b3a0 0x7f7d18150b10 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f7d1002ac70 tx=0x7f7d10004a40 comp rx=0 tx=0).stop 2024-05-30T17:07:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 shutdown_connections 2024-05-30T17:07:45.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f7cf806e510 0x7f7cf80709d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d1815b3a0 0x7f7d18150b10 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d18073310 0x7f7d181f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 --2- 172.21.15.67:0/3584137997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d1806dc20 0x7f7d18155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:45.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 >> 172.21.15.67:0/3584137997 conn(0x7f7d180b1fc0 msgr2=0x7f7d1806e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:45.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 shutdown_connections 2024-05-30T17:07:45.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:45.764+0000 7f7d1e200640 1 -- 172.21.15.67:0/3584137997 wait complete. 2024-05-30T17:07:45.785 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":850,"stamp":"2024-05-30T17:07:44.676607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20915095490,"num_objects":5021,"num_object_clones":1,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":1,"num_read":4440,"num_read_kb":17367224,"num_write":5865,"num_write_kb":20427047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8879,"ondisk_log_size":8879,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":42021136,"kb_used_data":40862928,"kb_used_omap":12,"kb_used_meta":1158131,"kb_avail":712953584,"statfs":{"total":773094113280,"available":730064470016,"internally_reserved":0,"allocated":41843638272,"data_stored":41836077397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1185926704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8440,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004676"},"pg_stats":[{"pgid":"19.3f","version":"134'118","reported_seq":213,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.596741+0000","last_change":"2024-05-30T17:04:52.998366+0000","last_active":"2024-05-30T17:07:34.596741+0000","last_peered":"2024-05-30T17:07:34.596741+0000","last_clean":"2024-05-30T17:07:34.596741+0000","last_became_active":"2024-05-30T17:04:52.997618+0000","last_became_peered":"2024-05-30T17:04:52.997618+0000","last_unstale":"2024-05-30T17:07:34.596741+0000","last_undegraded":"2024-05-30T17:07:34.596741+0000","last_fullsized":"2024-05-30T17:07:34.596741+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:27:57.883038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":71,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'119","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.587333+0000","last_change":"2024-05-30T17:04:52.984295+0000","last_active":"2024-05-30T17:07:34.587333+0000","last_peered":"2024-05-30T17:07:34.587333+0000","last_clean":"2024-05-30T17:07:34.587333+0000","last_became_active":"2024-05-30T17:04:52.982524+0000","last_became_peered":"2024-05-30T17:04:52.982524+0000","last_unstale":"2024-05-30T17:07:34.587333+0000","last_undegraded":"2024-05-30T17:07:34.587333+0000","last_fullsized":"2024-05-30T17:07:34.587333+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:49.622879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'141","reported_seq":233,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597096+0000","last_change":"2024-05-30T17:04:52.984783+0000","last_active":"2024-05-30T17:07:34.597096+0000","last_peered":"2024-05-30T17:07:34.597096+0000","last_clean":"2024-05-30T17:07:34.597096+0000","last_became_active":"2024-05-30T17:04:52.984493+0000","last_became_peered":"2024-05-30T17:04:52.984493+0000","last_unstale":"2024-05-30T17:07:34.597096+0000","last_undegraded":"2024-05-30T17:07:34.597096+0000","last_fullsized":"2024-05-30T17:07:34.597096+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:21.682021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904709,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":75,"num_read_kb":286725,"num_write":94,"num_write_kb":356356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'120","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592872+0000","last_change":"2024-05-30T17:05:34.963309+0000","last_active":"2024-05-30T17:07:34.592872+0000","last_peered":"2024-05-30T17:07:34.592872+0000","last_clean":"2024-05-30T17:07:34.592872+0000","last_became_active":"2024-05-30T17:05:34.859953+0000","last_became_peered":"2024-05-30T17:05:34.859953+0000","last_unstale":"2024-05-30T17:07:34.592872+0000","last_undegraded":"2024-05-30T17:07:34.592872+0000","last_fullsized":"2024-05-30T17:07:34.592872+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:50:31.835151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":70,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3b","version":"134'111","reported_seq":173,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.836580+0000","last_change":"2024-05-30T17:04:52.990350+0000","last_active":"2024-05-30T17:07:42.836580+0000","last_peered":"2024-05-30T17:07:42.836580+0000","last_clean":"2024-05-30T17:07:42.836580+0000","last_became_active":"2024-05-30T17:04:52.990018+0000","last_became_peered":"2024-05-30T17:04:52.990018+0000","last_unstale":"2024-05-30T17:07:42.836580+0000","last_undegraded":"2024-05-30T17:07:42.836580+0000","last_fullsized":"2024-05-30T17:07:42.836580+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:21.257549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":61,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'134","reported_seq":238,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600457+0000","last_change":"2024-05-30T17:04:52.992047+0000","last_active":"2024-05-30T17:07:34.600457+0000","last_peered":"2024-05-30T17:07:34.600457+0000","last_clean":"2024-05-30T17:07:34.600457+0000","last_became_active":"2024-05-30T17:04:52.991748+0000","last_became_peered":"2024-05-30T17:04:52.991748+0000","last_unstale":"2024-05-30T17:07:34.600457+0000","last_undegraded":"2024-05-30T17:07:34.600457+0000","last_fullsized":"2024-05-30T17:07:34.600457+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:06.346572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":84,"num_write_kb":335873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'126","reported_seq":194,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595157+0000","last_change":"2024-05-30T17:04:52.984245+0000","last_active":"2024-05-30T17:07:34.595157+0000","last_peered":"2024-05-30T17:07:34.595157+0000","last_clean":"2024-05-30T17:07:34.595157+0000","last_became_active":"2024-05-30T17:04:52.983960+0000","last_became_peered":"2024-05-30T17:04:52.983960+0000","last_unstale":"2024-05-30T17:07:34.595157+0000","last_undegraded":"2024-05-30T17:07:34.595157+0000","last_fullsized":"2024-05-30T17:07:34.595157+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:40:33.491032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":51,"num_read_kb":200706,"num_write":76,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'119","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592777+0000","last_change":"2024-05-30T17:04:52.988608+0000","last_active":"2024-05-30T17:07:34.592777+0000","last_peered":"2024-05-30T17:07:34.592777+0000","last_clean":"2024-05-30T17:07:34.592777+0000","last_became_active":"2024-05-30T17:04:52.988277+0000","last_became_peered":"2024-05-30T17:04:52.988277+0000","last_unstale":"2024-05-30T17:07:34.592777+0000","last_undegraded":"2024-05-30T17:07:34.592777+0000","last_fullsized":"2024-05-30T17:07:34.592777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:34.512494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":85,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'135","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.574879+0000","last_change":"2024-05-30T17:04:52.993014+0000","last_active":"2024-05-30T17:07:34.574879+0000","last_peered":"2024-05-30T17:07:34.574879+0000","last_clean":"2024-05-30T17:07:34.574879+0000","last_became_active":"2024-05-30T17:04:52.990543+0000","last_became_peered":"2024-05-30T17:04:52.990543+0000","last_unstale":"2024-05-30T17:07:34.574879+0000","last_undegraded":"2024-05-30T17:07:34.574879+0000","last_fullsized":"2024-05-30T17:07:34.574879+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:27:15.135800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":59,"num_read_kb":233474,"num_write":78,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'127","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.589282+0000","last_change":"2024-05-30T17:04:52.979749+0000","last_active":"2024-05-30T17:07:34.589282+0000","last_peered":"2024-05-30T17:07:34.589282+0000","last_clean":"2024-05-30T17:07:34.589282+0000","last_became_active":"2024-05-30T17:04:52.979390+0000","last_became_peered":"2024-05-30T17:04:52.979390+0000","last_unstale":"2024-05-30T17:07:34.589282+0000","last_undegraded":"2024-05-30T17:07:34.589282+0000","last_fullsized":"2024-05-30T17:07:34.589282+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":65,"num_read_kb":266240,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'122","reported_seq":245,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.599654+0000","last_change":"2024-05-30T17:04:52.983598+0000","last_active":"2024-05-30T17:07:34.599654+0000","last_peered":"2024-05-30T17:07:34.599654+0000","last_clean":"2024-05-30T17:07:34.599654+0000","last_became_active":"2024-05-30T17:04:52.983250+0000","last_became_peered":"2024-05-30T17:04:52.983250+0000","last_unstale":"2024-05-30T17:07:34.599654+0000","last_undegraded":"2024-05-30T17:07:34.599654+0000","last_fullsized":"2024-05-30T17:07:34.599654+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:52:20.889101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":106,"num_read_kb":434176,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'115","reported_seq":237,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.576009+0000","last_change":"2024-05-30T17:04:52.993606+0000","last_active":"2024-05-30T17:07:34.576009+0000","last_peered":"2024-05-30T17:07:34.576009+0000","last_clean":"2024-05-30T17:07:34.576009+0000","last_became_active":"2024-05-30T17:04:52.993151+0000","last_became_peered":"2024-05-30T17:04:52.993151+0000","last_unstale":"2024-05-30T17:07:34.576009+0000","last_undegraded":"2024-05-30T17:07:34.576009+0000","last_fullsized":"2024-05-30T17:07:34.576009+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:14.893435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":73,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'124","reported_seq":225,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.601181+0000","last_change":"2024-05-30T17:04:52.998102+0000","last_active":"2024-05-30T17:07:34.601181+0000","last_peered":"2024-05-30T17:07:34.601181+0000","last_clean":"2024-05-30T17:07:34.601181+0000","last_became_active":"2024-05-30T17:04:52.997770+0000","last_became_peered":"2024-05-30T17:04:52.997770+0000","last_unstale":"2024-05-30T17:07:34.601181+0000","last_undegraded":"2024-05-30T17:07:34.601181+0000","last_fullsized":"2024-05-30T17:07:34.601181+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":84,"num_read_kb":344064,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'127","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.584319+0000","last_change":"2024-05-30T17:04:52.992617+0000","last_active":"2024-05-30T17:07:34.584319+0000","last_peered":"2024-05-30T17:07:34.584319+0000","last_clean":"2024-05-30T17:07:34.584319+0000","last_became_active":"2024-05-30T17:04:52.992221+0000","last_became_peered":"2024-05-30T17:04:52.992221+0000","last_unstale":"2024-05-30T17:07:34.584319+0000","last_undegraded":"2024-05-30T17:07:34.584319+0000","last_fullsized":"2024-05-30T17:07:34.584319+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127364,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":84,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'117","reported_seq":166,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597825+0000","last_change":"2024-05-30T17:04:52.992819+0000","last_active":"2024-05-30T17:07:34.597825+0000","last_peered":"2024-05-30T17:07:34.597825+0000","last_clean":"2024-05-30T17:07:34.597825+0000","last_became_active":"2024-05-30T17:04:52.991693+0000","last_became_peered":"2024-05-30T17:04:52.991693+0000","last_unstale":"2024-05-30T17:07:34.597825+0000","last_undegraded":"2024-05-30T17:07:34.597825+0000","last_fullsized":"2024-05-30T17:07:34.597825+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":32,"num_read_kb":131072,"num_write":72,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'129","reported_seq":198,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595356+0000","last_change":"2024-05-30T17:04:52.991328+0000","last_active":"2024-05-30T17:07:34.595356+0000","last_peered":"2024-05-30T17:07:34.595356+0000","last_clean":"2024-05-30T17:07:34.595356+0000","last_became_active":"2024-05-30T17:04:52.990334+0000","last_became_peered":"2024-05-30T17:04:52.990334+0000","last_unstale":"2024-05-30T17:07:34.595356+0000","last_undegraded":"2024-05-30T17:07:34.595356+0000","last_fullsized":"2024-05-30T17:07:34.595356+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:38:23.528030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'114","reported_seq":174,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595797+0000","last_change":"2024-05-30T17:04:52.967087+0000","last_active":"2024-05-30T17:07:34.595797+0000","last_peered":"2024-05-30T17:07:34.595797+0000","last_clean":"2024-05-30T17:07:34.595797+0000","last_became_active":"2024-05-30T17:04:52.966026+0000","last_became_peered":"2024-05-30T17:04:52.966026+0000","last_unstale":"2024-05-30T17:07:34.595797+0000","last_undegraded":"2024-05-30T17:07:34.595797+0000","last_fullsized":"2024-05-30T17:07:34.595797+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:12.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'104","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.596384+0000","last_change":"2024-05-30T17:04:52.986386+0000","last_active":"2024-05-30T17:07:34.596384+0000","last_peered":"2024-05-30T17:07:34.596384+0000","last_clean":"2024-05-30T17:07:34.596384+0000","last_became_active":"2024-05-30T17:04:52.985602+0000","last_became_peered":"2024-05-30T17:04:52.985602+0000","last_unstale":"2024-05-30T17:07:34.596384+0000","last_undegraded":"2024-05-30T17:07:34.596384+0000","last_fullsized":"2024-05-30T17:07:34.596384+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:40:31.682869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":91,"num_read_kb":372736,"num_write":71,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'128","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595466+0000","last_change":"2024-05-30T17:04:52.963412+0000","last_active":"2024-05-30T17:07:34.595466+0000","last_peered":"2024-05-30T17:07:34.595466+0000","last_clean":"2024-05-30T17:07:34.595466+0000","last_became_active":"2024-05-30T17:04:52.962582+0000","last_became_peered":"2024-05-30T17:04:52.962582+0000","last_unstale":"2024-05-30T17:07:34.595466+0000","last_undegraded":"2024-05-30T17:07:34.595466+0000","last_fullsized":"2024-05-30T17:07:34.595466+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:07:52.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369098752,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":90,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'140","reported_seq":219,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.591011+0000","last_change":"2024-05-30T17:04:52.987028+0000","last_active":"2024-05-30T17:07:34.591011+0000","last_peered":"2024-05-30T17:07:34.591011+0000","last_clean":"2024-05-30T17:07:34.591011+0000","last_became_active":"2024-05-30T17:04:52.986701+0000","last_became_peered":"2024-05-30T17:04:52.986701+0000","last_unstale":"2024-05-30T17:07:34.591011+0000","last_undegraded":"2024-05-30T17:07:34.591011+0000","last_fullsized":"2024-05-30T17:07:34.591011+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:50:37.770982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352321536,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":62,"num_read_kb":249857,"num_write":86,"num_write_kb":344065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'152","reported_seq":265,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.853599+0000","last_change":"2024-05-30T17:04:52.984763+0000","last_active":"2024-05-30T17:07:42.853599+0000","last_peered":"2024-05-30T17:07:42.853599+0000","last_clean":"2024-05-30T17:07:42.853599+0000","last_became_active":"2024-05-30T17:04:52.984462+0000","last_became_peered":"2024-05-30T17:04:52.984462+0000","last_unstale":"2024-05-30T17:07:42.853599+0000","last_undegraded":"2024-05-30T17:07:42.853599+0000","last_fullsized":"2024-05-30T17:07:42.853599+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:09.221732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293056,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":96,"num_read_kb":372741,"num_write":95,"num_write_kb":364547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'126","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.591148+0000","last_change":"2024-05-30T17:04:52.980574+0000","last_active":"2024-05-30T17:07:34.591148+0000","last_peered":"2024-05-30T17:07:34.591148+0000","last_clean":"2024-05-30T17:07:34.591148+0000","last_became_active":"2024-05-30T17:04:52.978968+0000","last_became_peered":"2024-05-30T17:04:52.978968+0000","last_unstale":"2024-05-30T17:07:34.591148+0000","last_undegraded":"2024-05-30T17:07:34.591148+0000","last_fullsized":"2024-05-30T17:07:34.591148+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:30:51.419747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":61,"num_read_kb":245761,"num_write":82,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'134","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592216+0000","last_change":"2024-05-30T17:04:52.992275+0000","last_active":"2024-05-30T17:07:34.592216+0000","last_peered":"2024-05-30T17:07:34.592216+0000","last_clean":"2024-05-30T17:07:34.592216+0000","last_became_active":"2024-05-30T17:04:52.991966+0000","last_became_peered":"2024-05-30T17:04:52.991966+0000","last_unstale":"2024-05-30T17:07:34.592216+0000","last_undegraded":"2024-05-30T17:07:34.592216+0000","last_fullsized":"2024-05-30T17:07:34.592216+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:32.227484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":65,"num_read_kb":253955,"num_write":87,"num_write_kb":348161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'128","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.590037+0000","last_change":"2024-05-30T17:04:52.965882+0000","last_active":"2024-05-30T17:07:34.590037+0000","last_peered":"2024-05-30T17:07:34.590037+0000","last_clean":"2024-05-30T17:07:34.590037+0000","last_became_active":"2024-05-30T17:04:52.965116+0000","last_became_peered":"2024-05-30T17:04:52.965116+0000","last_unstale":"2024-05-30T17:07:34.590037+0000","last_undegraded":"2024-05-30T17:07:34.590037+0000","last_fullsized":"2024-05-30T17:07:34.590037+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:35:54.995662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":57,"num_read_kb":229377,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'129","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593198+0000","last_change":"2024-05-30T17:04:52.983400+0000","last_active":"2024-05-30T17:07:34.593198+0000","last_peered":"2024-05-30T17:07:34.593198+0000","last_clean":"2024-05-30T17:07:34.593198+0000","last_became_active":"2024-05-30T17:04:52.981867+0000","last_became_peered":"2024-05-30T17:04:52.981867+0000","last_unstale":"2024-05-30T17:07:34.593198+0000","last_undegraded":"2024-05-30T17:07:34.593198+0000","last_fullsized":"2024-05-30T17:07:34.593198+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":66,"num_read_kb":266241,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'133","reported_seq":220,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.598173+0000","last_change":"2024-05-30T17:04:52.989413+0000","last_active":"2024-05-30T17:07:34.598173+0000","last_peered":"2024-05-30T17:07:34.598173+0000","last_clean":"2024-05-30T17:07:34.598173+0000","last_became_active":"2024-05-30T17:04:52.989141+0000","last_became_peered":"2024-05-30T17:04:52.989141+0000","last_unstale":"2024-05-30T17:07:34.598173+0000","last_undegraded":"2024-05-30T17:07:34.598173+0000","last_fullsized":"2024-05-30T17:07:34.598173+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:10:15.844444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'129","reported_seq":217,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597479+0000","last_change":"2024-05-30T17:04:52.992667+0000","last_active":"2024-05-30T17:07:34.597479+0000","last_peered":"2024-05-30T17:07:34.597479+0000","last_clean":"2024-05-30T17:07:34.597479+0000","last_became_active":"2024-05-30T17:04:52.988618+0000","last_became_peered":"2024-05-30T17:04:52.988618+0000","last_unstale":"2024-05-30T17:07:34.597479+0000","last_undegraded":"2024-05-30T17:07:34.597479+0000","last_fullsized":"2024-05-30T17:07:34.597479+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:28:58.696434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377487360,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":90,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'115","reported_seq":165,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.863460+0000","last_change":"2024-05-30T17:04:52.977678+0000","last_active":"2024-05-30T17:07:42.863460+0000","last_peered":"2024-05-30T17:07:42.863460+0000","last_clean":"2024-05-30T17:07:42.863460+0000","last_became_active":"2024-05-30T17:04:52.977378+0000","last_became_peered":"2024-05-30T17:04:52.977378+0000","last_unstale":"2024-05-30T17:07:42.863460+0000","last_undegraded":"2024-05-30T17:07:42.863460+0000","last_fullsized":"2024-05-30T17:07:42.863460+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.220555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'140","reported_seq":235,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600970+0000","last_change":"2024-05-30T17:04:52.998322+0000","last_active":"2024-05-30T17:07:34.600970+0000","last_peered":"2024-05-30T17:07:34.600970+0000","last_clean":"2024-05-30T17:07:34.600970+0000","last_became_active":"2024-05-30T17:04:52.997323+0000","last_became_peered":"2024-05-30T17:04:52.997323+0000","last_unstale":"2024-05-30T17:07:34.600970+0000","last_undegraded":"2024-05-30T17:07:34.600970+0000","last_fullsized":"2024-05-30T17:07:34.600970+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:29:04.343909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390070272,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":78,"num_read_kb":307203,"num_write":95,"num_write_kb":380929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'110","reported_seq":191,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.594854+0000","last_change":"2024-05-30T17:04:52.995308+0000","last_active":"2024-05-30T17:07:34.594854+0000","last_peered":"2024-05-30T17:07:34.594854+0000","last_clean":"2024-05-30T17:07:34.594854+0000","last_became_active":"2024-05-30T17:04:52.992733+0000","last_became_peered":"2024-05-30T17:04:52.992733+0000","last_unstale":"2024-05-30T17:07:34.594854+0000","last_undegraded":"2024-05-30T17:07:34.594854+0000","last_fullsized":"2024-05-30T17:07:34.594854+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'155","reported_seq":272,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.869426+0000","last_change":"2024-05-30T17:04:52.993970+0000","last_active":"2024-05-30T17:07:42.869426+0000","last_peered":"2024-05-30T17:07:42.869426+0000","last_clean":"2024-05-30T17:07:42.869426+0000","last_became_active":"2024-05-30T17:04:52.993214+0000","last_became_peered":"2024-05-30T17:04:52.993214+0000","last_unstale":"2024-05-30T17:07:42.869426+0000","last_undegraded":"2024-05-30T17:07:42.869426+0000","last_fullsized":"2024-05-30T17:07:42.869426+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:21:06.134079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'133","reported_seq":225,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.599575+0000","last_change":"2024-05-30T17:04:52.980018+0000","last_active":"2024-05-30T17:07:34.599575+0000","last_peered":"2024-05-30T17:07:34.599575+0000","last_clean":"2024-05-30T17:07:34.599575+0000","last_became_active":"2024-05-30T17:04:52.979697+0000","last_became_peered":"2024-05-30T17:04:52.979697+0000","last_unstale":"2024-05-30T17:07:34.599575+0000","last_undegraded":"2024-05-30T17:07:34.599575+0000","last_fullsized":"2024-05-30T17:07:34.599575+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:30:44.014138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":77,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'119","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.589562+0000","last_change":"2024-05-30T17:04:52.978441+0000","last_active":"2024-05-30T17:07:34.589562+0000","last_peered":"2024-05-30T17:07:34.589562+0000","last_clean":"2024-05-30T17:07:34.589562+0000","last_became_active":"2024-05-30T17:04:52.978065+0000","last_became_peered":"2024-05-30T17:04:52.978065+0000","last_unstale":"2024-05-30T17:07:34.589562+0000","last_undegraded":"2024-05-30T17:07:34.589562+0000","last_fullsized":"2024-05-30T17:07:34.589562+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:21:00.756549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289407237,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":70,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":23,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.856712+0000","last_change":"2024-05-30T17:04:51.944167+0000","last_active":"2024-05-30T17:05:34.856712+0000","last_peered":"2024-05-30T17:05:34.856712+0000","last_clean":"2024-05-30T17:05:34.856712+0000","last_became_active":"2024-05-30T17:04:51.943796+0000","last_became_peered":"2024-05-30T17:04:51.943796+0000","last_unstale":"2024-05-30T17:05:34.856712+0000","last_undegraded":"2024-05-30T17:05:34.856712+0000","last_fullsized":"2024-05-30T17:05:34.856712+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:50.016832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'126","reported_seq":213,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592533+0000","last_change":"2024-05-30T17:04:52.998976+0000","last_active":"2024-05-30T17:07:34.592533+0000","last_peered":"2024-05-30T17:07:34.592533+0000","last_clean":"2024-05-30T17:07:34.592533+0000","last_became_active":"2024-05-30T17:04:52.998673+0000","last_became_peered":"2024-05-30T17:04:52.998673+0000","last_unstale":"2024-05-30T17:07:34.592533+0000","last_undegraded":"2024-05-30T17:07:34.592533+0000","last_fullsized":"2024-05-30T17:07:34.592533+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:58.331966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849900+0000","last_change":"2024-05-30T17:04:51.939570+0000","last_active":"2024-05-30T17:05:34.849900+0000","last_peered":"2024-05-30T17:05:34.849900+0000","last_clean":"2024-05-30T17:05:34.849900+0000","last_became_active":"2024-05-30T17:04:51.937127+0000","last_became_peered":"2024-05-30T17:04:51.937127+0000","last_unstale":"2024-05-30T17:05:34.849900+0000","last_undegraded":"2024-05-30T17:05:34.849900+0000","last_fullsized":"2024-05-30T17:05:34.849900+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:18.245355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'141","reported_seq":232,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860777+0000","last_change":"2024-05-30T17:04:52.983816+0000","last_active":"2024-05-30T17:07:42.860777+0000","last_peered":"2024-05-30T17:07:42.860777+0000","last_clean":"2024-05-30T17:07:42.860777+0000","last_became_active":"2024-05-30T17:04:52.981316+0000","last_became_peered":"2024-05-30T17:04:52.981316+0000","last_unstale":"2024-05-30T17:07:42.860777+0000","last_undegraded":"2024-05-30T17:07:42.860777+0000","last_fullsized":"2024-05-30T17:07:42.860777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:13.610126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738885,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":74,"num_read_kb":303104,"num_write":82,"num_write_kb":331777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'3","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.113874+0000","last_change":"2024-05-30T17:04:51.938960+0000","last_active":"2024-05-30T17:06:00.113874+0000","last_peered":"2024-05-30T17:06:00.113874+0000","last_clean":"2024-05-30T17:06:00.113874+0000","last_became_active":"2024-05-30T17:04:51.938644+0000","last_became_peered":"2024-05-30T17:04:51.938644+0000","last_unstale":"2024-05-30T17:06:00.113874+0000","last_undegraded":"2024-05-30T17:06:00.113874+0000","last_fullsized":"2024-05-30T17:06:00.113874+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:12.595138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'148","reported_seq":253,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.871545+0000","last_change":"2024-05-30T17:04:52.988274+0000","last_active":"2024-05-30T17:07:42.871545+0000","last_peered":"2024-05-30T17:07:42.871545+0000","last_clean":"2024-05-30T17:07:42.871545+0000","last_became_active":"2024-05-30T17:04:52.987961+0000","last_became_peered":"2024-05-30T17:04:52.987961+0000","last_unstale":"2024-05-30T17:07:42.871545+0000","last_undegraded":"2024-05-30T17:07:42.871545+0000","last_fullsized":"2024-05-30T17:07:42.871545+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:21.613683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904448,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":87,"num_write_kb":356352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849476+0000","last_change":"2024-05-30T17:04:51.930251+0000","last_active":"2024-05-30T17:05:34.849476+0000","last_peered":"2024-05-30T17:05:34.849476+0000","last_clean":"2024-05-30T17:05:34.849476+0000","last_became_active":"2024-05-30T17:04:51.929298+0000","last_became_peered":"2024-05-30T17:04:51.929298+0000","last_unstale":"2024-05-30T17:05:34.849476+0000","last_undegraded":"2024-05-30T17:05:34.849476+0000","last_fullsized":"2024-05-30T17:05:34.849476+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:52:35.687280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'127","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600397+0000","last_change":"2024-05-30T17:04:52.986145+0000","last_active":"2024-05-30T17:07:34.600397+0000","last_peered":"2024-05-30T17:07:34.600397+0000","last_clean":"2024-05-30T17:07:34.600397+0000","last_became_active":"2024-05-30T17:04:52.985811+0000","last_became_peered":"2024-05-30T17:04:52.985811+0000","last_unstale":"2024-05-30T17:07:34.600397+0000","last_undegraded":"2024-05-30T17:07:34.600397+0000","last_fullsized":"2024-05-30T17:07:34.600397+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:55.493534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.848968+0000","last_change":"2024-05-30T17:04:51.943821+0000","last_active":"2024-05-30T17:05:34.848968+0000","last_peered":"2024-05-30T17:05:34.848968+0000","last_clean":"2024-05-30T17:05:34.848968+0000","last_became_active":"2024-05-30T17:04:51.942597+0000","last_became_peered":"2024-05-30T17:04:51.942597+0000","last_unstale":"2024-05-30T17:05:34.848968+0000","last_undegraded":"2024-05-30T17:05:34.848968+0000","last_fullsized":"2024-05-30T17:05:34.848968+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:27:47.680066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'119","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.598016+0000","last_change":"2024-05-30T17:04:52.967603+0000","last_active":"2024-05-30T17:07:34.598016+0000","last_peered":"2024-05-30T17:07:34.598016+0000","last_clean":"2024-05-30T17:07:34.598016+0000","last_became_active":"2024-05-30T17:04:52.967033+0000","last_became_peered":"2024-05-30T17:04:52.967033+0000","last_unstale":"2024-05-30T17:07:34.598016+0000","last_undegraded":"2024-05-30T17:07:34.598016+0000","last_fullsized":"2024-05-30T17:07:34.598016+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:18.064738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.948046+0000","last_change":"2024-05-30T17:04:51.942553+0000","last_active":"2024-05-30T17:05:34.948046+0000","last_peered":"2024-05-30T17:05:34.948046+0000","last_clean":"2024-05-30T17:05:34.948046+0000","last_became_active":"2024-05-30T17:04:51.941974+0000","last_became_peered":"2024-05-30T17:04:51.941974+0000","last_unstale":"2024-05-30T17:05:34.948046+0000","last_undegraded":"2024-05-30T17:05:34.948046+0000","last_fullsized":"2024-05-30T17:05:34.948046+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:40.877465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'136","reported_seq":229,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.594707+0000","last_change":"2024-05-30T17:04:52.988985+0000","last_active":"2024-05-30T17:07:34.594707+0000","last_peered":"2024-05-30T17:07:34.594707+0000","last_clean":"2024-05-30T17:07:34.594707+0000","last_became_active":"2024-05-30T17:04:52.988645+0000","last_became_peered":"2024-05-30T17:04:52.988645+0000","last_unstale":"2024-05-30T17:07:34.594707+0000","last_undegraded":"2024-05-30T17:07:34.594707+0000","last_fullsized":"2024-05-30T17:07:34.594707+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:40.533224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":76,"num_read_kb":303106,"num_write":79,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.934368+0000","last_change":"2024-05-30T17:04:51.951513+0000","last_active":"2024-05-30T17:05:34.934368+0000","last_peered":"2024-05-30T17:05:34.934368+0000","last_clean":"2024-05-30T17:05:34.934368+0000","last_became_active":"2024-05-30T17:04:51.951223+0000","last_became_peered":"2024-05-30T17:04:51.951223+0000","last_unstale":"2024-05-30T17:05:34.934368+0000","last_undegraded":"2024-05-30T17:05:34.934368+0000","last_fullsized":"2024-05-30T17:05:34.934368+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:13.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'121","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593857+0000","last_change":"2024-05-30T17:04:52.986452+0000","last_active":"2024-05-30T17:07:34.593857+0000","last_peered":"2024-05-30T17:07:34.593857+0000","last_clean":"2024-05-30T17:07:34.593857+0000","last_became_active":"2024-05-30T17:04:52.986164+0000","last_became_peered":"2024-05-30T17:04:52.986164+0000","last_unstale":"2024-05-30T17:07:34.593857+0000","last_undegraded":"2024-05-30T17:07:34.593857+0000","last_fullsized":"2024-05-30T17:07:34.593857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:38:38.328901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":66,"num_write_kb":253954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.848424+0000","last_change":"2024-05-30T17:04:51.955992+0000","last_active":"2024-05-30T17:05:34.848424+0000","last_peered":"2024-05-30T17:05:34.848424+0000","last_clean":"2024-05-30T17:05:34.848424+0000","last_became_active":"2024-05-30T17:04:51.955666+0000","last_became_peered":"2024-05-30T17:04:51.955666+0000","last_unstale":"2024-05-30T17:05:34.848424+0000","last_undegraded":"2024-05-30T17:05:34.848424+0000","last_fullsized":"2024-05-30T17:05:34.848424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:57:03.343211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'122","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.596305+0000","last_change":"2024-05-30T17:04:52.984705+0000","last_active":"2024-05-30T17:07:34.596305+0000","last_peered":"2024-05-30T17:07:34.596305+0000","last_clean":"2024-05-30T17:07:34.596305+0000","last_became_active":"2024-05-30T17:04:52.984409+0000","last_became_peered":"2024-05-30T17:04:52.984409+0000","last_unstale":"2024-05-30T17:07:34.596305+0000","last_undegraded":"2024-05-30T17:07:34.596305+0000","last_fullsized":"2024-05-30T17:07:34.596305+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:45:56.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":39,"num_read_kb":159744,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.123168+0000","last_change":"2024-05-30T17:04:51.952573+0000","last_active":"2024-05-30T17:06:00.123168+0000","last_peered":"2024-05-30T17:06:00.123168+0000","last_clean":"2024-05-30T17:06:00.123168+0000","last_became_active":"2024-05-30T17:04:51.948281+0000","last_became_peered":"2024-05-30T17:04:51.948281+0000","last_unstale":"2024-05-30T17:06:00.123168+0000","last_undegraded":"2024-05-30T17:06:00.123168+0000","last_fullsized":"2024-05-30T17:06:00.123168+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:26.183919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'118","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859139+0000","last_change":"2024-05-30T17:04:52.986184+0000","last_active":"2024-05-30T17:07:42.859139+0000","last_peered":"2024-05-30T17:07:42.859139+0000","last_clean":"2024-05-30T17:07:42.859139+0000","last_became_active":"2024-05-30T17:04:52.985889+0000","last_became_peered":"2024-05-30T17:04:52.985889+0000","last_unstale":"2024-05-30T17:07:42.859139+0000","last_undegraded":"2024-05-30T17:07:42.859139+0000","last_fullsized":"2024-05-30T17:07:42.859139+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:27.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881146,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":59,"num_write_kb":229378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.949232+0000","last_change":"2024-05-30T17:04:51.954829+0000","last_active":"2024-05-30T17:05:34.949232+0000","last_peered":"2024-05-30T17:05:34.949232+0000","last_clean":"2024-05-30T17:05:34.949232+0000","last_became_active":"2024-05-30T17:04:51.954434+0000","last_became_peered":"2024-05-30T17:04:51.954434+0000","last_unstale":"2024-05-30T17:05:34.949232+0000","last_undegraded":"2024-05-30T17:05:34.949232+0000","last_fullsized":"2024-05-30T17:05:34.949232+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:23.550178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'124","reported_seq":221,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.863792+0000","last_change":"2024-05-30T17:04:52.989346+0000","last_active":"2024-05-30T17:07:42.863792+0000","last_peered":"2024-05-30T17:07:42.863792+0000","last_clean":"2024-05-30T17:07:42.863792+0000","last_became_active":"2024-05-30T17:04:52.989051+0000","last_became_peered":"2024-05-30T17:04:52.989051+0000","last_unstale":"2024-05-30T17:07:42.863792+0000","last_undegraded":"2024-05-30T17:07:42.863792+0000","last_fullsized":"2024-05-30T17:07:42.863792+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:36.128167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'28","reported_seq":47,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:29.184425+0000","last_change":"2024-05-30T17:04:51.938158+0000","last_active":"2024-05-30T17:06:29.184425+0000","last_peered":"2024-05-30T17:06:29.184425+0000","last_clean":"2024-05-30T17:06:29.184425+0000","last_became_active":"2024-05-30T17:04:51.937857+0000","last_became_peered":"2024-05-30T17:04:51.937857+0000","last_unstale":"2024-05-30T17:06:29.184425+0000","last_undegraded":"2024-05-30T17:06:29.184425+0000","last_fullsized":"2024-05-30T17:06:29.184425+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:25.458335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'134","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.854431+0000","last_change":"2024-05-30T17:04:52.995423+0000","last_active":"2024-05-30T17:07:42.854431+0000","last_peered":"2024-05-30T17:07:42.854431+0000","last_clean":"2024-05-30T17:07:42.854431+0000","last_became_active":"2024-05-30T17:04:52.995128+0000","last_became_peered":"2024-05-30T17:04:52.995128+0000","last_unstale":"2024-05-30T17:07:42.854431+0000","last_undegraded":"2024-05-30T17:07:42.854431+0000","last_fullsized":"2024-05-30T17:07:42.854431+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:47.501828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":78,"num_write_kb":303106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'436","reported_seq":455,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593864+0000","last_change":"2024-05-30T17:04:51.953550+0000","last_active":"2024-05-30T17:07:34.593864+0000","last_peered":"2024-05-30T17:07:34.593864+0000","last_clean":"2024-05-30T17:07:34.593864+0000","last_became_active":"2024-05-30T17:04:51.953276+0000","last_became_peered":"2024-05-30T17:04:51.953276+0000","last_unstale":"2024-05-30T17:07:34.593864+0000","last_undegraded":"2024-05-30T17:07:34.593864+0000","last_fullsized":"2024-05-30T17:07:34.593864+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:07:44.468185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1698517,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":435,"num_write_kb":1831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'149","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600297+0000","last_change":"2024-05-30T17:04:52.988845+0000","last_active":"2024-05-30T17:07:34.600297+0000","last_peered":"2024-05-30T17:07:34.600297+0000","last_clean":"2024-05-30T17:07:34.600297+0000","last_became_active":"2024-05-30T17:04:52.986757+0000","last_became_peered":"2024-05-30T17:04:52.986757+0000","last_unstale":"2024-05-30T17:07:34.600297+0000","last_undegraded":"2024-05-30T17:07:34.600297+0000","last_fullsized":"2024-05-30T17:07:34.600297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385875968,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":92,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.853652+0000","last_change":"2024-05-30T17:04:51.944552+0000","last_active":"2024-05-30T17:05:34.853652+0000","last_peered":"2024-05-30T17:05:34.853652+0000","last_clean":"2024-05-30T17:05:34.853652+0000","last_became_active":"2024-05-30T17:04:51.944167+0000","last_became_peered":"2024-05-30T17:04:51.944167+0000","last_unstale":"2024-05-30T17:05:34.853652+0000","last_undegraded":"2024-05-30T17:05:34.853652+0000","last_fullsized":"2024-05-30T17:05:34.853652+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:17:06.906161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'121","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593998+0000","last_change":"2024-05-30T17:04:52.991186+0000","last_active":"2024-05-30T17:07:34.593998+0000","last_peered":"2024-05-30T17:07:34.593998+0000","last_clean":"2024-05-30T17:07:34.593998+0000","last_became_active":"2024-05-30T17:04:52.990049+0000","last_became_peered":"2024-05-30T17:04:52.990049+0000","last_unstale":"2024-05-30T17:07:34.593998+0000","last_undegraded":"2024-05-30T17:07:34.593998+0000","last_fullsized":"2024-05-30T17:07:34.593998+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:28:23.896291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.848017+0000","last_change":"2024-05-30T17:04:51.947216+0000","last_active":"2024-05-30T17:05:34.848017+0000","last_peered":"2024-05-30T17:05:34.848017+0000","last_clean":"2024-05-30T17:05:34.848017+0000","last_became_active":"2024-05-30T17:04:51.946868+0000","last_became_peered":"2024-05-30T17:04:51.946868+0000","last_unstale":"2024-05-30T17:05:34.848017+0000","last_undegraded":"2024-05-30T17:05:34.848017+0000","last_fullsized":"2024-05-30T17:05:34.848017+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:42:48.589336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'140","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592617+0000","last_change":"2024-05-30T17:04:52.986670+0000","last_active":"2024-05-30T17:07:34.592617+0000","last_peered":"2024-05-30T17:07:34.592617+0000","last_clean":"2024-05-30T17:07:34.592617+0000","last_became_active":"2024-05-30T17:04:52.986293+0000","last_became_peered":"2024-05-30T17:04:52.986293+0000","last_unstale":"2024-05-30T17:07:34.592617+0000","last_undegraded":"2024-05-30T17:07:34.592617+0000","last_fullsized":"2024-05-30T17:07:34.592617+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:28.097184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293317,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":90,"num_write_kb":364545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":27,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.955398+0000","last_change":"2024-05-30T17:04:51.955924+0000","last_active":"2024-05-30T17:05:34.955398+0000","last_peered":"2024-05-30T17:05:34.955398+0000","last_clean":"2024-05-30T17:05:34.955398+0000","last_became_active":"2024-05-30T17:04:51.955542+0000","last_became_peered":"2024-05-30T17:04:51.955542+0000","last_unstale":"2024-05-30T17:05:34.955398+0000","last_undegraded":"2024-05-30T17:05:34.955398+0000","last_fullsized":"2024-05-30T17:05:34.955398+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:38.543340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'107","reported_seq":149,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595390+0000","last_change":"2024-05-30T17:04:53.001644+0000","last_active":"2024-05-30T17:07:34.595390+0000","last_peered":"2024-05-30T17:07:34.595390+0000","last_clean":"2024-05-30T17:07:34.595390+0000","last_became_active":"2024-05-30T17:04:52.999924+0000","last_became_peered":"2024-05-30T17:04:52.999924+0000","last_unstale":"2024-05-30T17:07:34.595390+0000","last_undegraded":"2024-05-30T17:07:34.595390+0000","last_fullsized":"2024-05-30T17:07:34.595390+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:17:36.947839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.850216+0000","last_change":"2024-05-30T17:04:51.948540+0000","last_active":"2024-05-30T17:05:34.850216+0000","last_peered":"2024-05-30T17:05:34.850216+0000","last_clean":"2024-05-30T17:05:34.850216+0000","last_became_active":"2024-05-30T17:04:51.947928+0000","last_became_peered":"2024-05-30T17:04:51.947928+0000","last_unstale":"2024-05-30T17:05:34.850216+0000","last_undegraded":"2024-05-30T17:05:34.850216+0000","last_fullsized":"2024-05-30T17:05:34.850216+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:10:19.343810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'133","reported_seq":231,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.853021+0000","last_change":"2024-05-30T17:04:52.976220+0000","last_active":"2024-05-30T17:07:42.853021+0000","last_peered":"2024-05-30T17:07:42.853021+0000","last_clean":"2024-05-30T17:07:42.853021+0000","last_became_active":"2024-05-30T17:04:52.974987+0000","last_became_peered":"2024-05-30T17:04:52.974987+0000","last_unstale":"2024-05-30T17:07:42.853021+0000","last_undegraded":"2024-05-30T17:07:42.853021+0000","last_fullsized":"2024-05-30T17:07:42.853021+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:15.443712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360710144,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":81,"num_read_kb":331776,"num_write":86,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855580+0000","last_change":"2024-05-30T17:04:51.950294+0000","last_active":"2024-05-30T17:05:34.855580+0000","last_peered":"2024-05-30T17:05:34.855580+0000","last_clean":"2024-05-30T17:05:34.855580+0000","last_became_active":"2024-05-30T17:04:51.949704+0000","last_became_peered":"2024-05-30T17:04:51.949704+0000","last_unstale":"2024-05-30T17:05:34.855580+0000","last_undegraded":"2024-05-30T17:05:34.855580+0000","last_fullsized":"2024-05-30T17:05:34.855580+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:46.486840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'135","reported_seq":215,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.588249+0000","last_change":"2024-05-30T17:04:52.989374+0000","last_active":"2024-05-30T17:07:34.588249+0000","last_peered":"2024-05-30T17:07:34.588249+0000","last_clean":"2024-05-30T17:07:34.588249+0000","last_became_active":"2024-05-30T17:04:52.989083+0000","last_became_peered":"2024-05-30T17:04:52.989083+0000","last_unstale":"2024-05-30T17:07:34.588249+0000","last_undegraded":"2024-05-30T17:07:34.588249+0000","last_fullsized":"2024-05-30T17:07:34.588249+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:35.524809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":63,"num_read_kb":237573,"num_write":85,"num_write_kb":307205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.935270+0000","last_change":"2024-05-30T17:04:51.942441+0000","last_active":"2024-05-30T17:05:34.935270+0000","last_peered":"2024-05-30T17:05:34.935270+0000","last_clean":"2024-05-30T17:05:34.935270+0000","last_became_active":"2024-05-30T17:04:51.942121+0000","last_became_peered":"2024-05-30T17:04:51.942121+0000","last_unstale":"2024-05-30T17:05:34.935270+0000","last_undegraded":"2024-05-30T17:05:34.935270+0000","last_fullsized":"2024-05-30T17:05:34.935270+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:27.082196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'128","reported_seq":227,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.582512+0000","last_change":"2024-05-30T17:04:52.994569+0000","last_active":"2024-05-30T17:07:34.582512+0000","last_peered":"2024-05-30T17:07:34.582512+0000","last_clean":"2024-05-30T17:07:34.582512+0000","last_became_active":"2024-05-30T17:04:52.994290+0000","last_became_peered":"2024-05-30T17:04:52.994290+0000","last_unstale":"2024-05-30T17:07:34.582512+0000","last_undegraded":"2024-05-30T17:07:34.582512+0000","last_fullsized":"2024-05-30T17:07:34.582512+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:28.557814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849744+0000","last_change":"2024-05-30T17:04:51.945428+0000","last_active":"2024-05-30T17:05:34.849744+0000","last_peered":"2024-05-30T17:05:34.849744+0000","last_clean":"2024-05-30T17:05:34.849744+0000","last_became_active":"2024-05-30T17:04:51.945142+0000","last_became_peered":"2024-05-30T17:04:51.945142+0000","last_unstale":"2024-05-30T17:05:34.849744+0000","last_undegraded":"2024-05-30T17:05:34.849744+0000","last_fullsized":"2024-05-30T17:05:34.849744+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:25:46.375765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'128","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.597686+0000","last_change":"2024-05-30T17:04:52.993514+0000","last_active":"2024-05-30T17:07:34.597686+0000","last_peered":"2024-05-30T17:07:34.597686+0000","last_clean":"2024-05-30T17:07:34.597686+0000","last_became_active":"2024-05-30T17:04:52.988660+0000","last_became_peered":"2024-05-30T17:04:52.988660+0000","last_unstale":"2024-05-30T17:07:34.597686+0000","last_undegraded":"2024-05-30T17:07:34.597686+0000","last_fullsized":"2024-05-30T17:07:34.597686+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'3","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.150462+0000","last_change":"2024-05-30T17:04:51.943410+0000","last_active":"2024-05-30T17:06:00.150462+0000","last_peered":"2024-05-30T17:06:00.150462+0000","last_clean":"2024-05-30T17:06:00.150462+0000","last_became_active":"2024-05-30T17:04:51.943085+0000","last_became_peered":"2024-05-30T17:04:51.943085+0000","last_unstale":"2024-05-30T17:06:00.150462+0000","last_undegraded":"2024-05-30T17:06:00.150462+0000","last_fullsized":"2024-05-30T17:06:00.150462+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:18.313092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'120","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.588879+0000","last_change":"2024-05-30T17:04:52.992939+0000","last_active":"2024-05-30T17:07:34.588879+0000","last_peered":"2024-05-30T17:07:34.588879+0000","last_clean":"2024-05-30T17:07:34.588879+0000","last_became_active":"2024-05-30T17:04:52.989409+0000","last_became_peered":"2024-05-30T17:04:52.989409+0000","last_unstale":"2024-05-30T17:07:34.588879+0000","last_undegraded":"2024-05-30T17:07:34.588879+0000","last_fullsized":"2024-05-30T17:07:34.588879+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:41.892452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795845,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":72,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.948183+0000","last_change":"2024-05-30T17:04:51.948194+0000","last_active":"2024-05-30T17:05:34.948183+0000","last_peered":"2024-05-30T17:05:34.948183+0000","last_clean":"2024-05-30T17:05:34.948183+0000","last_became_active":"2024-05-30T17:04:51.947919+0000","last_became_peered":"2024-05-30T17:04:51.947919+0000","last_unstale":"2024-05-30T17:05:34.948183+0000","last_undegraded":"2024-05-30T17:05:34.948183+0000","last_fullsized":"2024-05-30T17:05:34.948183+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:58.823357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'134","reported_seq":199,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.600019+0000","last_change":"2024-05-30T17:04:52.981350+0000","last_active":"2024-05-30T17:07:34.600019+0000","last_peered":"2024-05-30T17:07:34.600019+0000","last_clean":"2024-05-30T17:07:34.600019+0000","last_became_active":"2024-05-30T17:04:52.978531+0000","last_became_peered":"2024-05-30T17:04:52.978531+0000","last_unstale":"2024-05-30T17:07:34.600019+0000","last_undegraded":"2024-05-30T17:07:34.600019+0000","last_fullsized":"2024-05-30T17:07:34.600019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:23:00.650504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":48,"num_read_kb":188418,"num_write":83,"num_write_kb":323586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.947541+0000","last_change":"2024-05-30T17:04:51.950726+0000","last_active":"2024-05-30T17:05:34.947541+0000","last_peered":"2024-05-30T17:05:34.947541+0000","last_clean":"2024-05-30T17:05:34.947541+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:05:34.947541+0000","last_undegraded":"2024-05-30T17:05:34.947541+0000","last_fullsized":"2024-05-30T17:05:34.947541+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:17.049750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'123","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859963+0000","last_change":"2024-05-30T17:05:35.214281+0000","last_active":"2024-05-30T17:07:42.859963+0000","last_peered":"2024-05-30T17:07:42.859963+0000","last_clean":"2024-05-30T17:07:42.859963+0000","last_became_active":"2024-05-30T17:05:34.860839+0000","last_became_peered":"2024-05-30T17:05:34.860839+0000","last_unstale":"2024-05-30T17:07:42.859963+0000","last_undegraded":"2024-05-30T17:07:42.859963+0000","last_fullsized":"2024-05-30T17:07:42.859963+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:11:46.508726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":69,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.854184+0000","last_change":"2024-05-30T17:04:51.951054+0000","last_active":"2024-05-30T17:05:34.854184+0000","last_peered":"2024-05-30T17:05:34.854184+0000","last_clean":"2024-05-30T17:05:34.854184+0000","last_became_active":"2024-05-30T17:04:51.950364+0000","last_became_peered":"2024-05-30T17:04:51.950364+0000","last_unstale":"2024-05-30T17:05:34.854184+0000","last_undegraded":"2024-05-30T17:05:34.854184+0000","last_fullsized":"2024-05-30T17:05:34.854184+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:11.010697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'124","reported_seq":178,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593805+0000","last_change":"2024-05-30T17:04:52.967099+0000","last_active":"2024-05-30T17:07:34.593805+0000","last_peered":"2024-05-30T17:07:34.593805+0000","last_clean":"2024-05-30T17:07:34.593805+0000","last_became_active":"2024-05-30T17:04:52.966035+0000","last_became_peered":"2024-05-30T17:04:52.966035+0000","last_unstale":"2024-05-30T17:07:34.593805+0000","last_undegraded":"2024-05-30T17:07:34.593805+0000","last_fullsized":"2024-05-30T17:07:34.593805+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:24.565166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.853230+0000","last_change":"2024-05-30T17:04:51.949867+0000","last_active":"2024-05-30T17:05:34.853230+0000","last_peered":"2024-05-30T17:05:34.853230+0000","last_clean":"2024-05-30T17:05:34.853230+0000","last_became_active":"2024-05-30T17:04:51.949575+0000","last_became_peered":"2024-05-30T17:04:51.949575+0000","last_unstale":"2024-05-30T17:05:34.853230+0000","last_undegraded":"2024-05-30T17:05:34.853230+0000","last_fullsized":"2024-05-30T17:05:34.853230+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:58:11.353946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'126","reported_seq":230,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592727+0000","last_change":"2024-05-30T17:04:52.998729+0000","last_active":"2024-05-30T17:07:34.592727+0000","last_peered":"2024-05-30T17:07:34.592727+0000","last_clean":"2024-05-30T17:07:34.592727+0000","last_became_active":"2024-05-30T17:04:52.997726+0000","last_became_peered":"2024-05-30T17:04:52.997726+0000","last_unstale":"2024-05-30T17:07:34.592727+0000","last_undegraded":"2024-05-30T17:07:34.592727+0000","last_fullsized":"2024-05-30T17:07:34.592727+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:32:14.601932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"134'49","reported_seq":68,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:35.039807+0000","last_change":"2024-05-30T17:04:51.949751+0000","last_active":"2024-05-30T17:07:35.039807+0000","last_peered":"2024-05-30T17:07:35.039807+0000","last_clean":"2024-05-30T17:07:35.039807+0000","last_became_active":"2024-05-30T17:04:51.949295+0000","last_became_peered":"2024-05-30T17:04:51.949295+0000","last_unstale":"2024-05-30T17:07:35.039807+0000","last_undegraded":"2024-05-30T17:07:35.039807+0000","last_fullsized":"2024-05-30T17:07:35.039807+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:41.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'129","reported_seq":243,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595138+0000","last_change":"2024-05-30T17:04:52.985036+0000","last_active":"2024-05-30T17:07:34.595138+0000","last_peered":"2024-05-30T17:07:34.595138+0000","last_clean":"2024-05-30T17:07:34.595138+0000","last_became_active":"2024-05-30T17:04:52.984675+0000","last_became_peered":"2024-05-30T17:04:52.984675+0000","last_unstale":"2024-05-30T17:07:34.595138+0000","last_undegraded":"2024-05-30T17:07:34.595138+0000","last_fullsized":"2024-05-30T17:07:34.595138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:30:26.510444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":397312,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849721+0000","last_change":"2024-05-30T17:04:51.952967+0000","last_active":"2024-05-30T17:05:34.849721+0000","last_peered":"2024-05-30T17:05:34.849721+0000","last_clean":"2024-05-30T17:05:34.849721+0000","last_became_active":"2024-05-30T17:04:51.952631+0000","last_became_peered":"2024-05-30T17:04:51.952631+0000","last_unstale":"2024-05-30T17:05:34.849721+0000","last_undegraded":"2024-05-30T17:05:34.849721+0000","last_fullsized":"2024-05-30T17:05:34.849721+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:23.747059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'123","reported_seq":193,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.592347+0000","last_change":"2024-05-30T17:04:52.990963+0000","last_active":"2024-05-30T17:07:34.592347+0000","last_peered":"2024-05-30T17:07:34.592347+0000","last_clean":"2024-05-30T17:07:34.592347+0000","last_became_active":"2024-05-30T17:04:52.990476+0000","last_became_peered":"2024-05-30T17:04:52.990476+0000","last_unstale":"2024-05-30T17:07:34.592347+0000","last_undegraded":"2024-05-30T17:07:34.592347+0000","last_fullsized":"2024-05-30T17:07:34.592347+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:59.838344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378750,"num_objects":77,"num_object_clones":1,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":1,"num_read":54,"num_read_kb":217089,"num_write":79,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855975+0000","last_change":"2024-05-30T17:04:51.933352+0000","last_active":"2024-05-30T17:05:34.855975+0000","last_peered":"2024-05-30T17:05:34.855975+0000","last_clean":"2024-05-30T17:05:34.855975+0000","last_became_active":"2024-05-30T17:04:51.933061+0000","last_became_peered":"2024-05-30T17:04:51.933061+0000","last_unstale":"2024-05-30T17:05:34.855975+0000","last_undegraded":"2024-05-30T17:05:34.855975+0000","last_fullsized":"2024-05-30T17:05:34.855975+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:20.242562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'134","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.870019+0000","last_change":"2024-05-30T17:04:52.982699+0000","last_active":"2024-05-30T17:07:42.870019+0000","last_peered":"2024-05-30T17:07:42.870019+0000","last_clean":"2024-05-30T17:07:42.870019+0000","last_became_active":"2024-05-30T17:04:52.981542+0000","last_became_peered":"2024-05-30T17:04:52.981542+0000","last_unstale":"2024-05-30T17:07:42.870019+0000","last_undegraded":"2024-05-30T17:07:42.870019+0000","last_fullsized":"2024-05-30T17:07:42.870019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:12.025684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":76,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849081+0000","last_change":"2024-05-30T17:04:51.947181+0000","last_active":"2024-05-30T17:05:34.849081+0000","last_peered":"2024-05-30T17:05:34.849081+0000","last_clean":"2024-05-30T17:05:34.849081+0000","last_became_active":"2024-05-30T17:04:51.946882+0000","last_became_peered":"2024-05-30T17:04:51.946882+0000","last_unstale":"2024-05-30T17:05:34.849081+0000","last_undegraded":"2024-05-30T17:05:34.849081+0000","last_fullsized":"2024-05-30T17:05:34.849081+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:47:52.431592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'157","reported_seq":257,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860002+0000","last_change":"2024-05-30T17:04:52.992399+0000","last_active":"2024-05-30T17:07:42.860002+0000","last_peered":"2024-05-30T17:07:42.860002+0000","last_clean":"2024-05-30T17:07:42.860002+0000","last_became_active":"2024-05-30T17:04:52.992040+0000","last_became_peered":"2024-05-30T17:04:52.992040+0000","last_unstale":"2024-05-30T17:07:42.860002+0000","last_undegraded":"2024-05-30T17:07:42.860002+0000","last_fullsized":"2024-05-30T17:07:42.860002+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:16.335923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":411041792,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":98,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855981+0000","last_change":"2024-05-30T17:04:51.952089+0000","last_active":"2024-05-30T17:05:34.855981+0000","last_peered":"2024-05-30T17:05:34.855981+0000","last_clean":"2024-05-30T17:05:34.855981+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:05:34.855981+0000","last_undegraded":"2024-05-30T17:05:34.855981+0000","last_fullsized":"2024-05-30T17:05:34.855981+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:03:20.313618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'131","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.595785+0000","last_change":"2024-05-30T17:04:52.988572+0000","last_active":"2024-05-30T17:07:34.595785+0000","last_peered":"2024-05-30T17:07:34.595785+0000","last_clean":"2024-05-30T17:07:34.595785+0000","last_became_active":"2024-05-30T17:04:52.987684+0000","last_became_peered":"2024-05-30T17:04:52.987684+0000","last_unstale":"2024-05-30T17:07:34.595785+0000","last_undegraded":"2024-05-30T17:07:34.595785+0000","last_fullsized":"2024-05-30T17:07:34.595785+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:26:52.920204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":82,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'35","reported_seq":88,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:29.176344+0000","last_change":"2024-05-30T17:04:51.949334+0000","last_active":"2024-05-30T17:06:29.176344+0000","last_peered":"2024-05-30T17:06:29.176344+0000","last_clean":"2024-05-30T17:06:29.176344+0000","last_became_active":"2024-05-30T17:04:51.948995+0000","last_became_peered":"2024-05-30T17:04:51.948995+0000","last_unstale":"2024-05-30T17:06:29.176344+0000","last_undegraded":"2024-05-30T17:06:29.176344+0000","last_fullsized":"2024-05-30T17:06:29.176344+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:05.331584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4250,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'129","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:34.593040+0000","last_change":"2024-05-30T17:04:52.992411+0000","last_active":"2024-05-30T17:07:34.593040+0000","last_peered":"2024-05-30T17:07:34.593040+0000","last_clean":"2024-05-30T17:07:34.593040+0000","last_became_active":"2024-05-30T17:04:52.991997+0000","last_became_peered":"2024-05-30T17:04:52.991997+0000","last_unstale":"2024-05-30T17:07:34.593040+0000","last_undegraded":"2024-05-30T17:07:34.593040+0000","last_fullsized":"2024-05-30T17:07:34.593040+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:55:48.622932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":402653184,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":96,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.854388+0000","last_change":"2024-05-30T17:04:51.946511+0000","last_active":"2024-05-30T17:05:34.854388+0000","last_peered":"2024-05-30T17:05:34.854388+0000","last_clean":"2024-05-30T17:05:34.854388+0000","last_became_active":"2024-05-30T17:04:51.946173+0000","last_became_peered":"2024-05-30T17:04:51.946173+0000","last_unstale":"2024-05-30T17:05:34.854388+0000","last_undegraded":"2024-05-30T17:05:34.854388+0000","last_fullsized":"2024-05-30T17:05:34.854388+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:08.206744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'122","reported_seq":192,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.864677+0000","last_change":"2024-05-30T17:04:52.992639+0000","last_active":"2024-05-30T17:07:42.864677+0000","last_peered":"2024-05-30T17:07:42.864677+0000","last_clean":"2024-05-30T17:07:42.864677+0000","last_became_active":"2024-05-30T17:04:52.992202+0000","last_became_peered":"2024-05-30T17:04:52.992202+0000","last_unstale":"2024-05-30T17:07:42.864677+0000","last_undegraded":"2024-05-30T17:07:42.864677+0000","last_fullsized":"2024-05-30T17:07:42.864677+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:34.869318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":53,"num_read_kb":217088,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.856152+0000","last_change":"2024-05-30T17:04:51.951966+0000","last_active":"2024-05-30T17:05:34.856152+0000","last_peered":"2024-05-30T17:05:34.856152+0000","last_clean":"2024-05-30T17:05:34.856152+0000","last_became_active":"2024-05-30T17:04:51.951484+0000","last_became_peered":"2024-05-30T17:04:51.951484+0000","last_unstale":"2024-05-30T17:05:34.856152+0000","last_undegraded":"2024-05-30T17:05:34.856152+0000","last_fullsized":"2024-05-30T17:05:34.856152+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:41.627294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":388,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:44.508422+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:05:44.508422+0000","last_peered":"2024-05-30T17:05:44.508422+0000","last_clean":"2024-05-30T17:05:44.508422+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:05:44.508422+0000","last_undegraded":"2024-05-30T17:05:44.508422+0000","last_fullsized":"2024-05-30T17:05:44.508422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":20912801557,"num_objects":4996,"num_object_clones":1,"num_object_copies":9992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":1,"num_read":4290,"num_read_kb":17367090,"num_write":5064,"num_write_kb":20422699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41825673216,"data_stored":41825603114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8182,"ondisk_log_size":8182,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1703565,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":642,"num_write_kb":1932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3473408,"data_stored":3407379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":580,"ondisk_log_size":580,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509731,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5955000,"kb_used_data":5811440,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":88416840,"statfs":{"total":96636764160,"available":90538844160,"internally_reserved":0,"allocated":5950914560,"data_stored":5949962393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509731,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6190836,"kb_used_data":6051436,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":88181004,"statfs":{"total":96636764160,"available":90297348096,"internally_reserved":0,"allocated":6196670464,"data_stored":6195723625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509732,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5015352,"kb_used_data":4871792,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":89356488,"statfs":{"total":96636764160,"available":91501043712,"internally_reserved":0,"allocated":4988715008,"data_stored":4987768020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509732,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6950260,"kb_used_data":6792812,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":87421580,"statfs":{"total":96636764160,"available":89519697920,"internally_reserved":0,"allocated":6955839488,"data_stored":6954892805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509731,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4728548,"kb_used_data":4589148,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89643292,"statfs":{"total":96636764160,"available":91794731008,"internally_reserved":0,"allocated":4699287552,"data_stored":4698356383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542435,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4575160,"kb_used_data":4435760,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89796680,"statfs":{"total":96636764160,"available":91951800320,"internally_reserved":0,"allocated":4542218240,"data_stored":4541262732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542436,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4261684,"kb_used_data":4118124,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90110156,"statfs":{"total":96636764160,"available":92272799744,"internally_reserved":0,"allocated":4216958976,"data_stored":4216012047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4344296,"kb_used_data":4192416,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":90027544,"statfs":{"total":96636764160,"available":92188205056,"internally_reserved":0,"allocated":4293033984,"data_stored":4292099392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1716224,"data_stored":1702767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1720320,"data_stored":1699404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4290777088,"data_stored":4290773253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4697624576,"data_stored":4697620612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6954168320,"data_stored":6954156418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4987035648,"data_stored":4987027839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6195003392,"data_stored":6194987784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4215287808,"data_stored":4215276164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5947535360,"data_stored":5947523855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4538241024,"data_stored":4538237189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:07:45.789 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-30T17:07:46.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.035+0000 7fa3ffe00640 1 Processor -- start 2024-05-30T17:07:46.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.035+0000 7fa3ffe00640 1 -- start start 2024-05-30T17:07:46.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.037+0000 7fa3ffe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa400072070 0x7fa4000b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:46.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.037+0000 7fa3ffe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40006e6e0 con 0x7fa400158b80 2024-05-30T17:07:46.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.037+0000 7fa3ffe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40006e9c0 con 0x7fa400071bc0 2024-05-30T17:07:46.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.037+0000 7fa3ffe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa40006eca0 con 0x7fa400072070 2024-05-30T17:07:46.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.037+0000 7fa3fea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa400158b80 0x7fa40006dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43850/0 (socket says 172.21.15.67:43850) 2024-05-30T17:07:46.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.037+0000 7fa3fea00640 1 -- 172.21.15.67:0/4239983330 learned_addr learned my addr 172.21.15.67:0/4239983330 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:46.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061061478 0 0) 0x7fa40006e6e0 con 0x7fa400158b80 2024-05-30T17:07:46.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3ff400640 1 --2- 172.21.15.67:0/4239983330 >> v2:172.21.15.169:3300/0 conn(0x7fa400072070 0x7fa4000b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:46.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3e4003520 con 0x7fa400158b80 2024-05-30T17:07:46.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 143474577 0 0) 0x7fa3e4003520 con 0x7fa400158b80 2024-05-30T17:07:46.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 >> v2:172.21.15.169:3300/0 conn(0x7fa400072070 msgr2=0x7fa4000b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:46.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 --2- 172.21.15.67:0/4239983330 >> v2:172.21.15.169:3300/0 conn(0x7fa400072070 0x7fa4000b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 >> v1:172.21.15.169:6789/0 conn(0x7fa400071bc0 legacy=0x7fa4000b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40006ef80 con 0x7fa400158b80 2024-05-30T17:07:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fa3f0001670 con 0x7fa400158b80 2024-05-30T17:07:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 >> v1:172.21.15.67:6789/0 conn(0x7fa400158b80 legacy=0x7fa40006dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa3e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4004800 0x7fa3e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e40051c0 0x7fa3e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa40006eca0 con 0x7fa3e40039b0 2024-05-30T17:07:46.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e400c110 con 0x7fa3e4004800 2024-05-30T17:07:46.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e400c310 con 0x7fa3e40051c0 2024-05-30T17:07:46.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3ff400640 1 --2- 172.21.15.67:0/4239983330 >> v2:172.21.15.169:3300/0 conn(0x7fa400072070 0x7fa4000b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:07:46.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.038+0000 7fa3fea00640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa3e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:46.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.039+0000 7fa3fea00640 1 -- 172.21.15.67:0/4239983330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e40051c0 msgr2=0x7fa3e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:46.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.039+0000 7fa3f7e00640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4004800 0x7fa3e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:46.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.039+0000 7fa3fea00640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e40051c0 0x7fa3e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.039+0000 7fa3fea00640 1 -- 172.21.15.67:0/4239983330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4004800 msgr2=0x7fa3e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.039+0000 7fa3fea00640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4004800 0x7fa3e4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.039+0000 7fa3fea00640 1 -- 172.21.15.67:0/4239983330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e4003520 con 0x7fa3e40039b0 2024-05-30T17:07:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.039+0000 7fa3ff400640 1 -- 172.21.15.67:0/4239983330 reap_dead start 2024-05-30T17:07:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.040+0000 7fa3fea00640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa3e4003dd0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c240 tx=0x7fa3e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f81542849a842d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.040+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0007cf0 con 0x7fa3e40039b0 2024-05-30T17:07:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.040+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f00058d0 con 0x7fa3e40039b0 2024-05-30T17:07:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.040+0000 7fa3fe000640 1 -- 172.21.15.67:0/4239983330 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0005d20 con 0x7fa3e40039b0 2024-05-30T17:07:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.041+0000 7fa3ffe00640 1 -- 172.21.15.67:0/4239983330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 msgr2=0x7fa3e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.041+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa3e4003dd0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c240 tx=0x7fa3e4005830 comp rx=0 tx=0).stop 2024-05-30T17:07:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.041+0000 7fa3ffe00640 1 -- 172.21.15.67:0/4239983330 shutdown_connections 2024-05-30T17:07:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.041+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/4239983330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa3e4003dd0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.041+0000 7fa3ffe00640 1 -- 172.21.15.67:0/4239983330 >> 172.21.15.67:0/4239983330 conn(0x7fa4000b1fc0 msgr2=0x7fa4000b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.041+0000 7fa3ffe00640 1 -- 172.21.15.67:0/4239983330 shutdown_connections 2024-05-30T17:07:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.041+0000 7fa3ffe00640 1 -- 172.21.15.67:0/4239983330 wait complete. 2024-05-30T17:07:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.043+0000 7fa3ffe00640 1 Processor -- start 2024-05-30T17:07:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.043+0000 7fa3ffe00640 1 -- start start 2024-05-30T17:07:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.044+0000 7fa3ffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa40015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.044+0000 7fa3ffe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa40009d6e0 0x7fa40009db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.044+0000 7fa3ffe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4000af7a0 0x7fa4000afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.044+0000 7fa3ffe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa400072450 con 0x7fa3e40039b0 2024-05-30T17:07:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.044+0000 7fa3ffe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa4000b2d30 con 0x7fa40009d6e0 2024-05-30T17:07:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.044+0000 7fa3ffe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa40006f780 con 0x7fa4000af7a0 2024-05-30T17:07:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3fea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa40015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3fea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa40015d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49280/0 (socket says 172.21.15.67:49280) 2024-05-30T17:07:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3fea00640 1 -- 172.21.15.67:0/2311840754 learned_addr learned my addr 172.21.15.67:0/2311840754 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3ff400640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4000af7a0 0x7fa4000afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3f7e00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa40009d6e0 0x7fa40009db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3fea00640 1 -- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4000af7a0 msgr2=0x7fa4000afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3fea00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4000af7a0 0x7fa4000afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3fea00640 1 -- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa40009d6e0 msgr2=0x7fa40009db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3fea00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa40009d6e0 0x7fa40009db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3fea00640 1 -- 172.21.15.67:0/2311840754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40009e0a0 con 0x7fa3e40039b0 2024-05-30T17:07:46.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3ff400640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4000af7a0 0x7fa4000afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:46.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3f7e00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa40009d6e0 0x7fa40009db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:46.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.045+0000 7fa3fea00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa40015d940 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c330 tx=0x7fa40006eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:46.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.046+0000 7fa3f7400640 1 -- 172.21.15.67:0/2311840754 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0038040 con 0x7fa3e40039b0 2024-05-30T17:07:46.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.046+0000 7fa3f7400640 1 -- 172.21.15.67:0/2311840754 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f0000ed0 con 0x7fa3e40039b0 2024-05-30T17:07:46.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.046+0000 7fa3f7400640 1 -- 172.21.15.67:0/2311840754 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0001c30 con 0x7fa3e40039b0 2024-05-30T17:07:46.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.046+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40009add0 con 0x7fa3e40039b0 2024-05-30T17:07:46.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.046+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40009b350 con 0x7fa3e40039b0 2024-05-30T17:07:46.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.047+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4000689e0 con 0x7fa3e40039b0 2024-05-30T17:07:46.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.048+0000 7fa3f7400640 1 -- 172.21.15.67:0/2311840754 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7fa3f0007070 con 0x7fa3e40039b0 2024-05-30T17:07:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.049+0000 7fa3f7400640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa3dc06e510 0x7fa3dc0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.049+0000 7fa3f7400640 1 -- 172.21.15.67:0/2311840754 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fa3f00aa6f0 con 0x7fa3e40039b0 2024-05-30T17:07:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.050+0000 7fa3f7e00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa3dc06e510 0x7fa3dc0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.050+0000 7fa3f7e00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa3dc06e510 0x7fa3dc0709d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa4000b0f10 tx=0x7fa3e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:46.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.054+0000 7fa3f7400640 1 -- 172.21.15.67:0/2311840754 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa40009e0a0 con 0x7fa3e40039b0 2024-05-30T17:07:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.270+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa400157c40 con 0x7fa3e40039b0 2024-05-30T17:07:46.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.272+0000 7fa3f7400640 1 -- 172.21.15.67:0/2311840754 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16885 (secure 0 0 0) 0x7fa3f007c090 con 0x7fa3e40039b0 2024-05-30T17:07:46.273 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:46.274 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":134,"fsid":"faaf0c1f-b7d8-4c2c-86a6-1ebcd2bde04d","created":"2024-05-30T16:43:26.590972+0000","modified":"2024-05-30T17:05:34.829160+0000","last_up_change":"2024-05-30T16:43:39.428056+0000","last_in_change":"2024-05-30T16:43:30.736505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-30T16:43:40.009725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-30T17:04:50.327312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-30T17:04:51.388842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"841d017d-87f0-434e-bc72-699abeabcfd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6824","nonce":274279214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6827","nonce":274279214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6831","nonce":274279214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":274279214},{"type":"v1","addr":"172.21.15.67:6829","nonce":274279214}]},"public_addr":"172.21.15.67:6824/274279214","cluster_addr":"172.21.15.67:6827/274279214","heartbeat_back_addr":"172.21.15.67:6831/274279214","heartbeat_front_addr":"172.21.15.67:6829/274279214","state":["exists","up"]},{"osd":1,"uuid":"af68158f-3f57-4352-b975-983e7d597f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6809","nonce":3260209820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6811","nonce":3260209820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6817","nonce":3260209820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3260209820},{"type":"v1","addr":"172.21.15.67:6813","nonce":3260209820}]},"public_addr":"172.21.15.67:6809/3260209820","cluster_addr":"172.21.15.67:6811/3260209820","heartbeat_back_addr":"172.21.15.67:6817/3260209820","heartbeat_front_addr":"172.21.15.67:6813/3260209820","state":["exists","up"]},{"osd":2,"uuid":"d2137b46-ff54-4ae4-a961-cb73f43ba933","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6801","nonce":2549377238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6803","nonce":2549377238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6807","nonce":2549377238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2549377238},{"type":"v1","addr":"172.21.15.67:6805","nonce":2549377238}]},"public_addr":"172.21.15.67:6801/2549377238","cluster_addr":"172.21.15.67:6803/2549377238","heartbeat_back_addr":"172.21.15.67:6807/2549377238","heartbeat_front_addr":"172.21.15.67:6805/2549377238","state":["exists","up"]},{"osd":3,"uuid":"e129317f-acfd-4f49-9b2e-ebb28399566c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6816","nonce":2982751245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6819","nonce":2982751245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6825","nonce":2982751245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2982751245},{"type":"v1","addr":"172.21.15.67:6821","nonce":2982751245}]},"public_addr":"172.21.15.67:6816/2982751245","cluster_addr":"172.21.15.67:6819/2982751245","heartbeat_back_addr":"172.21.15.67:6825/2982751245","heartbeat_front_addr":"172.21.15.67:6821/2982751245","state":["exists","up"]},{"osd":4,"uuid":"45b623cb-599d-40fb-bf17-2eb0ae2476e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6814","nonce":432873034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6818","nonce":432873034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6823","nonce":432873034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":432873034},{"type":"v1","addr":"172.21.15.169:6821","nonce":432873034}]},"public_addr":"172.21.15.169:6814/432873034","cluster_addr":"172.21.15.169:6818/432873034","heartbeat_back_addr":"172.21.15.169:6823/432873034","heartbeat_front_addr":"172.21.15.169:6821/432873034","state":["exists","up"]},{"osd":5,"uuid":"e201087d-f95d-4cd6-b6c7-2fc0a89c6ff7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6825","nonce":3094236492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6827","nonce":3094236492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6831","nonce":3094236492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3094236492},{"type":"v1","addr":"172.21.15.169:6829","nonce":3094236492}]},"public_addr":"172.21.15.169:6825/3094236492","cluster_addr":"172.21.15.169:6827/3094236492","heartbeat_back_addr":"172.21.15.169:6831/3094236492","heartbeat_front_addr":"172.21.15.169:6829/3094236492","state":["exists","up"]},{"osd":6,"uuid":"0dd05747-ca8a-4e42-b854-abaa79659a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6801","nonce":2783087947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6803","nonce":2783087947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6807","nonce":2783087947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2783087947},{"type":"v1","addr":"172.21.15.169:6805","nonce":2783087947}]},"public_addr":"172.21.15.169:6801/2783087947","cluster_addr":"172.21.15.169:6803/2783087947","heartbeat_back_addr":"172.21.15.169:6807/2783087947","heartbeat_front_addr":"172.21.15.169:6805/2783087947","state":["exists","up"]},{"osd":7,"uuid":"bfddf4bf-a666-4bf3-97d5-93815ab9a518","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6809","nonce":4150327572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6811","nonce":4150327572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6819","nonce":4150327572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":4150327572},{"type":"v1","addr":"172.21.15.169:6815","nonce":4150327572}]},"public_addr":"172.21.15.169:6809/4150327572","cluster_addr":"172.21.15.169:6811/4150327572","heartbeat_back_addr":"172.21.15.169:6819/4150327572","heartbeat_front_addr":"172.21.15.169:6815/4150327572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.919123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.101696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.079636+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.133110+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.119542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.428362+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:36.060552+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-30T16:43:35.655003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.9","mappings":[{"from":2,"to":0}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":13,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-30T17:07:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.275+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa3dc06e510 msgr2=0x7fa3dc0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.275+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa3dc06e510 0x7fa3dc0709d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa4000b0f10 tx=0x7fa3e8009040 comp rx=0 tx=0).stop 2024-05-30T17:07:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.275+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 msgr2=0x7fa40015d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.275+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa40015d940 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c330 tx=0x7fa40006eb90 comp rx=0 tx=0).stop 2024-05-30T17:07:46.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.276+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 shutdown_connections 2024-05-30T17:07:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.276+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa3dc06e510 0x7fa3dc0709d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.276+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4000af7a0 0x7fa4000afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.276+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa40009d6e0 0x7fa40009db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.276+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/2311840754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3e40039b0 0x7fa40015d940 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.276+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 >> 172.21.15.67:0/2311840754 conn(0x7fa4000b1fc0 msgr2=0x7fa400157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.276+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 shutdown_connections 2024-05-30T17:07:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:46.276+0000 7fa3ffe00640 1 -- 172.21.15.67:0/2311840754 wait complete. 2024-05-30T17:07:47.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:07:47.043 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:07:47.298 INFO:tasks.ceph:Scrubbing osd.0 2024-05-30T17:07:47.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:07:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.522+0000 7f9921400640 1 Processor -- start 2024-05-30T17:07:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.523+0000 7f9921400640 1 -- start start 2024-05-30T17:07:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.524+0000 7f9921400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f991c06ecb0 0x7f991c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.524+0000 7f9921400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f991c06fc30 0x7f991c0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.524+0000 7f9921400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991c15af60 con 0x7f991c15a4d0 2024-05-30T17:07:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.524+0000 7f9921400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f991c15b240 con 0x7f991c06ecb0 2024-05-30T17:07:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.524+0000 7f9921400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f991c15b520 con 0x7f991c06fc30 2024-05-30T17:07:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.525+0000 7f9913e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f991c06ecb0 0x7f991c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.525+0000 7f9913e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f991c06ecb0 0x7f991c15a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49282/0 (socket says 172.21.15.67:49282) 2024-05-30T17:07:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.525+0000 7f9913e00640 1 -- 172.21.15.67:0/13783602 learned_addr learned my addr 172.21.15.67:0/13783602 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.525+0000 7f9920a00640 1 --2- 172.21.15.67:0/13783602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f991c06fc30 0x7f991c0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.525+0000 7f9913e00640 1 -- 172.21.15.67:0/13783602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f991c06fc30 msgr2=0x7f991c0cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.525+0000 7f9913e00640 1 --2- 172.21.15.67:0/13783602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f991c06fc30 0x7f991c0cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.525+0000 7f9913e00640 1 -- 172.21.15.67:0/13783602 >> v1:172.21.15.169:6789/0 conn(0x7f991c15a4d0 legacy=0x7f991c06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.525+0000 7f9913e00640 1 -- 172.21.15.67:0/13783602 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c15b800 con 0x7f991c06ecb0 2024-05-30T17:07:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.525+0000 7f9920a00640 1 --2- 172.21.15.67:0/13783602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f991c06fc30 0x7f991c0cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.526+0000 7f9913e00640 1 --2- 172.21.15.67:0/13783602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 0x7f991c15a8b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f9904009870 tx=0x7f990402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37ca887ac49d98f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:47.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.527+0000 7f991b400640 1 -- 172.21.15.67:0/13783602 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904037070 con 0x7f991c06ecb0 2024-05-30T17:07:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.527+0000 7f991b400640 1 -- 172.21.15.67:0/13783602 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904032440 con 0x7f991c06ecb0 2024-05-30T17:07:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.527+0000 7f991b400640 1 -- 172.21.15.67:0/13783602 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904032810 con 0x7f991c06ecb0 2024-05-30T17:07:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.528+0000 7f9921400640 1 -- 172.21.15.67:0/13783602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 msgr2=0x7f991c15a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.528+0000 7f9921400640 1 --2- 172.21.15.67:0/13783602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 0x7f991c15a8b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f9904009870 tx=0x7f990402a210 comp rx=0 tx=0).stop 2024-05-30T17:07:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.528+0000 7f9921400640 1 -- 172.21.15.67:0/13783602 shutdown_connections 2024-05-30T17:07:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.528+0000 7f9921400640 1 --2- 172.21.15.67:0/13783602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f991c06fc30 0x7f991c0cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.528+0000 7f9921400640 1 --2- 172.21.15.67:0/13783602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 0x7f991c15a8b0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.528+0000 7f9921400640 1 -- 172.21.15.67:0/13783602 >> 172.21.15.67:0/13783602 conn(0x7f991c06bae0 msgr2=0x7f991c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.528+0000 7f9921400640 1 -- 172.21.15.67:0/13783602 shutdown_connections 2024-05-30T17:07:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.528+0000 7f9921400640 1 -- 172.21.15.67:0/13783602 wait complete. 2024-05-30T17:07:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.530+0000 7f9921400640 1 Processor -- start 2024-05-30T17:07:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.530+0000 7f9921400640 1 -- start start 2024-05-30T17:07:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.531+0000 7f9921400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 0x7f991c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.531+0000 7f9921400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f991c06fc30 0x7f991c1f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.531+0000 7f9921400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f991c15a4d0 0x7f991c1f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.531+0000 7f9921400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0cfb40 con 0x7f991c06ecb0 2024-05-30T17:07:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.531+0000 7f9921400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0cf9c0 con 0x7f991c15a4d0 2024-05-30T17:07:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.531+0000 7f9921400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f991c0cfcc0 con 0x7f991c06fc30 2024-05-30T17:07:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.531+0000 7f991be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 0x7f991c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.531+0000 7f991be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 0x7f991c0ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49292/0 (socket says 172.21.15.67:49292) 2024-05-30T17:07:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.531+0000 7f991be00640 1 -- 172.21.15.67:0/3117595638 learned_addr learned my addr 172.21.15.67:0/3117595638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f991be00640 1 -- 172.21.15.67:0/3117595638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f991c06fc30 msgr2=0x7f991c1f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f991be00640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f991c06fc30 0x7f991c1f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f991be00640 1 -- 172.21.15.67:0/3117595638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f991c15a4d0 msgr2=0x7f991c1f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f991be00640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f991c15a4d0 0x7f991c1f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f991be00640 1 -- 172.21.15.67:0/3117595638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c0af2e0 con 0x7f991c06ecb0 2024-05-30T17:07:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f991be00640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 0x7f991c0ad8c0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f990c00b3f0 tx=0x7f990c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f9918c00640 1 -- 172.21.15.67:0/3117595638 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c015020 con 0x7f991c06ecb0 2024-05-30T17:07:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f9918c00640 1 -- 172.21.15.67:0/3117595638 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c011830 con 0x7f991c06ecb0 2024-05-30T17:07:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991c0ae240 con 0x7f991c06ecb0 2024-05-30T17:07:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.532+0000 7f9918c00640 1 -- 172.21.15.67:0/3117595638 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c011c20 con 0x7f991c06ecb0 2024-05-30T17:07:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.533+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991c0aeae0 con 0x7f991c06ecb0 2024-05-30T17:07:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.534+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f991c0ae630 con 0x7f991c06ecb0 2024-05-30T17:07:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.535+0000 7f9918c00640 1 -- 172.21.15.67:0/3117595638 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f990c018020 con 0x7f991c06ecb0 2024-05-30T17:07:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.536+0000 7f9918c00640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f98f806e510 0x7f98f80709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.536+0000 7f9918c00640 1 -- 172.21.15.67:0/3117595638 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f990c08a730 con 0x7f991c06ecb0 2024-05-30T17:07:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.537+0000 7f9913e00640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f98f806e510 0x7f98f80709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.537+0000 7f9918c00640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f98f8076910 0x7f98f8078db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.537+0000 7f9918c00640 1 -- 172.21.15.67:0/3117595638 --> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f990c001d30 con 0x7f98f8076910 2024-05-30T17:07:47.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.537+0000 7f9918c00640 1 -- 172.21.15.67:0/3117595638 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f990c096050 con 0x7f991c06ecb0 2024-05-30T17:07:47.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.537+0000 7f9920a00640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f98f8076910 0x7f98f8078db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.537+0000 7f9913e00640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f98f806e510 0x7f98f80709d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9904034000 tx=0x7f990402c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.538+0000 7f9920a00640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f98f8076910 0x7f98f8078db0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.538+0000 7f9918c00640 1 -- 172.21.15.67:0/3117595638 <== osd.0 v2:172.21.15.67:6823/274279214 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f990c001d30 con 0x7f98f8076910 2024-05-30T17:07:47.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.553+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 --> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f991c0689e0 con 0x7f98f8076910 2024-05-30T17:07:47.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.554+0000 7f9918c00640 1 -- 172.21.15.67:0/3117595638 <== osd.0 v2:172.21.15.67:6823/274279214 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f991c0689e0 con 0x7f98f8076910 2024-05-30T17:07:47.554 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:47.555 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:07:47.555 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.554+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f98f8076910 msgr2=0x7f98f8078db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.554+0000 7f9921400640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6823/274279214,v1:172.21.15.67:6824/274279214] conn(0x7f98f8076910 0x7f98f8078db0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.555+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f98f806e510 msgr2=0x7f98f80709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.555+0000 7f9921400640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f98f806e510 0x7f98f80709d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9904034000 tx=0x7f990402c000 comp rx=0 tx=0).stop 2024-05-30T17:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.555+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 msgr2=0x7f991c0ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.555+0000 7f9921400640 1 --2- 172.21.15.67:0/3117595638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c06ecb0 0x7f991c0ad8c0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f990c00b3f0 tx=0x7f990c00b8b0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.555+0000 7f9920a00640 1 -- 172.21.15.67:0/3117595638 reap_dead start 2024-05-30T17:07:47.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.555+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 shutdown_connections 2024-05-30T17:07:47.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.555+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 >> 172.21.15.67:0/3117595638 conn(0x7f991c06bae0 msgr2=0x7f991c15bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:47.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.555+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 shutdown_connections 2024-05-30T17:07:47.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.555+0000 7f9921400640 1 -- 172.21.15.67:0/3117595638 wait complete. 2024-05-30T17:07:47.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-30T17:07:47.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.784+0000 7fe8d9800640 1 Processor -- start 2024-05-30T17:07:47.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.784+0000 7fe8d9800640 1 -- start start 2024-05-30T17:07:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.786+0000 7fe8d9800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8d4073310 0x7fe8d40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.786+0000 7fe8d9800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe8d406dc20 0x7fe8d415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.786+0000 7fe8d9800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d40afd60 con 0x7fe8d415b3a0 2024-05-30T17:07:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.786+0000 7fe8d9800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d406f980 con 0x7fe8d4073310 2024-05-30T17:07:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.786+0000 7fe8d9800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d40ae0c0 con 0x7fe8d406dc20 2024-05-30T17:07:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.786+0000 7fe8d3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8d4073310 0x7fe8d40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.786+0000 7fe8d3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8d4073310 0x7fe8d40b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49306/0 (socket says 172.21.15.67:49306) 2024-05-30T17:07:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.786+0000 7fe8d3400640 1 -- 172.21.15.67:0/2442627503 learned_addr learned my addr 172.21.15.67:0/2442627503 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.786+0000 7fe8d8e00640 1 --2- 172.21.15.67:0/2442627503 >> v2:172.21.15.169:3300/0 conn(0x7fe8d406dc20 0x7fe8d415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.787+0000 7fe8d3400640 1 -- 172.21.15.67:0/2442627503 >> v2:172.21.15.169:3300/0 conn(0x7fe8d406dc20 msgr2=0x7fe8d415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.787+0000 7fe8d3400640 1 --2- 172.21.15.67:0/2442627503 >> v2:172.21.15.169:3300/0 conn(0x7fe8d406dc20 0x7fe8d415f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.787+0000 7fe8d3400640 1 -- 172.21.15.67:0/2442627503 >> v1:172.21.15.169:6789/0 conn(0x7fe8d415b3a0 legacy=0x7fe8d4072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.787+0000 7fe8d3400640 1 -- 172.21.15.67:0/2442627503 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d40af7a0 con 0x7fe8d4073310 2024-05-30T17:07:47.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.788+0000 7fe8d3400640 1 --2- 172.21.15.67:0/2442627503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d4073310 0x7fe8d40b32c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fe8d406d6e0 tx=0x7fe8c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c2c4f1f318434 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.788+0000 7fe8d2a00640 1 -- 172.21.15.67:0/2442627503 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0037070 con 0x7fe8d4073310 2024-05-30T17:07:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.788+0000 7fe8d2a00640 1 -- 172.21.15.67:0/2442627503 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c0032440 con 0x7fe8d4073310 2024-05-30T17:07:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.788+0000 7fe8d2a00640 1 -- 172.21.15.67:0/2442627503 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0032810 con 0x7fe8d4073310 2024-05-30T17:07:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.789+0000 7fe8d9800640 1 -- 172.21.15.67:0/2442627503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d4073310 msgr2=0x7fe8d40b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.789+0000 7fe8d9800640 1 --2- 172.21.15.67:0/2442627503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d4073310 0x7fe8d40b32c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fe8d406d6e0 tx=0x7fe8c002a230 comp rx=0 tx=0).stop 2024-05-30T17:07:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.789+0000 7fe8d9800640 1 -- 172.21.15.67:0/2442627503 shutdown_connections 2024-05-30T17:07:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.789+0000 7fe8d9800640 1 --2- 172.21.15.67:0/2442627503 >> v2:172.21.15.169:3300/0 conn(0x7fe8d406dc20 0x7fe8d415f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.789+0000 7fe8d9800640 1 --2- 172.21.15.67:0/2442627503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d4073310 0x7fe8d40b32c0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.789+0000 7fe8d9800640 1 -- 172.21.15.67:0/2442627503 >> 172.21.15.67:0/2442627503 conn(0x7fe8d40b1fc0 msgr2=0x7fe8d40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.789+0000 7fe8d9800640 1 -- 172.21.15.67:0/2442627503 shutdown_connections 2024-05-30T17:07:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.789+0000 7fe8d9800640 1 -- 172.21.15.67:0/2442627503 wait complete. 2024-05-30T17:07:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.791+0000 7fe8d9800640 1 Processor -- start 2024-05-30T17:07:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.791+0000 7fe8d9800640 1 -- start start 2024-05-30T17:07:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.792+0000 7fe8d9800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8d406dc20 0x7fe8d4155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.792+0000 7fe8d9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8d4073310 0x7fe8d415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.792+0000 7fe8d9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d415b3a0 0x7fe8d415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.792+0000 7fe8d9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d415b780 con 0x7fe8d415b3a0 2024-05-30T17:07:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.792+0000 7fe8d9800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d415fee0 con 0x7fe8d406dc20 2024-05-30T17:07:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.792+0000 7fe8d9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d40ae550 con 0x7fe8d4073310 2024-05-30T17:07:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.792+0000 7fe8d8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d415b3a0 0x7fe8d415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.792+0000 7fe8d8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d415b3a0 0x7fe8d415f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49312/0 (socket says 172.21.15.67:49312) 2024-05-30T17:07:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.792+0000 7fe8d8e00640 1 -- 172.21.15.67:0/1875037899 learned_addr learned my addr 172.21.15.67:0/1875037899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8d3e00640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8d406dc20 0x7fe8d4155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8d3400640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8d4073310 0x7fe8d415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8d8e00640 1 -- 172.21.15.67:0/1875037899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8d4073310 msgr2=0x7fe8d415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8d8e00640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8d4073310 0x7fe8d415ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8d8e00640 1 -- 172.21.15.67:0/1875037899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8d406dc20 msgr2=0x7fe8d4155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8d8e00640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8d406dc20 0x7fe8d4155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8d8e00640 1 -- 172.21.15.67:0/1875037899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d414e990 con 0x7fe8d415b3a0 2024-05-30T17:07:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8d3400640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8d4073310 0x7fe8d415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8d8e00640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d415b3a0 0x7fe8d415f460 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc007dc0 tx=0x7fe8cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:47.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8cbe00640 1 -- 172.21.15.67:0/1875037899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc014420 con 0x7fe8d415b3a0 2024-05-30T17:07:47.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.793+0000 7fe8cbe00640 1 -- 172.21.15.67:0/1875037899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc019070 con 0x7fe8d415b3a0 2024-05-30T17:07:47.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.794+0000 7fe8cbe00640 1 -- 172.21.15.67:0/1875037899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc014820 con 0x7fe8d415b3a0 2024-05-30T17:07:47.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.794+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d40ce950 con 0x7fe8d415b3a0 2024-05-30T17:07:47.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.794+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d40cef10 con 0x7fe8d415b3a0 2024-05-30T17:07:47.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.795+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d40689e0 con 0x7fe8d415b3a0 2024-05-30T17:07:47.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.796+0000 7fe8cbe00640 1 -- 172.21.15.67:0/1875037899 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7fe8cc0149c0 con 0x7fe8d415b3a0 2024-05-30T17:07:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.797+0000 7fe8cbe00640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8b406e510 0x7fe8b40709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.797+0000 7fe8cbe00640 1 -- 172.21.15.67:0/1875037899 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fe8cc08b350 con 0x7fe8d415b3a0 2024-05-30T17:07:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.797+0000 7fe8d3e00640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8b406e510 0x7fe8b40709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.798+0000 7fe8d3e00640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8b406e510 0x7fe8b40709d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400a750 tx=0x7fe8c4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:47.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:47.801+0000 7fe8cbe00640 1 -- 172.21.15.67:0/1875037899 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8d414e990 con 0x7fe8d415b3a0 2024-05-30T17:07:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.009+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fe8d414ebb0 con 0x7fe8b406e510 2024-05-30T17:07:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.010+0000 7fe8cbe00640 1 -- 172.21.15.67:0/1875037899 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe8d414ebb0 con 0x7fe8b406e510 2024-05-30T17:07:48.011 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 0 to deep-scrub 2024-05-30T17:07:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8b406e510 msgr2=0x7fe8b40709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8b406e510 0x7fe8b40709d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400a750 tx=0x7fe8c4008040 comp rx=0 tx=0).stop 2024-05-30T17:07:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d415b3a0 msgr2=0x7fe8d415f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d415b3a0 0x7fe8d415f460 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc007dc0 tx=0x7fe8cc00e4b0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 shutdown_connections 2024-05-30T17:07:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fe8b406e510 0x7fe8b40709d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d415b3a0 0x7fe8d415f460 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8d4073310 0x7fe8d415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 --2- 172.21.15.67:0/1875037899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8d406dc20 0x7fe8d4155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 >> 172.21.15.67:0/1875037899 conn(0x7fe8d40b1fc0 msgr2=0x7fe8d40b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.013+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 shutdown_connections 2024-05-30T17:07:48.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.014+0000 7fe8d9800640 1 -- 172.21.15.67:0/1875037899 wait complete. 2024-05-30T17:07:48.033 INFO:tasks.ceph:Scrubbing osd.1 2024-05-30T17:07:48.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:07:48.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.247+0000 7f7743200640 1 Processor -- start 2024-05-30T17:07:48.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.248+0000 7f7743200640 1 -- start start 2024-05-30T17:07:48.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.249+0000 7f7743200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f773c15e5a0 0x7f773c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.249+0000 7f7743200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f773c0d1420 0x7f773c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.249+0000 7f7743200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f773c0d3500 con 0x7f773c0cfb40 2024-05-30T17:07:48.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.249+0000 7f7743200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f773c0d2b30 con 0x7f773c0d1420 2024-05-30T17:07:48.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.249+0000 7f7743200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f773c0d2200 con 0x7f773c15e5a0 2024-05-30T17:07:48.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.249+0000 7f7742800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f773c0d1420 0x7f773c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.249+0000 7f7742800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f773c0d1420 0x7f773c06c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49314/0 (socket says 172.21.15.67:49314) 2024-05-30T17:07:48.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.249+0000 7f7742800640 1 -- 172.21.15.67:0/399047162 learned_addr learned my addr 172.21.15.67:0/399047162 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:48.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.250+0000 7f7741e00640 1 --2- 172.21.15.67:0/399047162 >> v2:172.21.15.169:3300/0 conn(0x7f773c15e5a0 0x7f773c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.250+0000 7f7742800640 1 -- 172.21.15.67:0/399047162 >> v2:172.21.15.169:3300/0 conn(0x7f773c15e5a0 msgr2=0x7f773c0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:48.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.250+0000 7f7742800640 1 --2- 172.21.15.67:0/399047162 >> v2:172.21.15.169:3300/0 conn(0x7f773c15e5a0 0x7f773c0d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.250+0000 7f7742800640 1 -- 172.21.15.67:0/399047162 >> v1:172.21.15.67:6789/0 conn(0x7f773c0cfb40 legacy=0x7f773c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.250+0000 7f7742800640 1 -- 172.21.15.67:0/399047162 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c0d24e0 con 0x7f773c0d1420 2024-05-30T17:07:48.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.250+0000 7f7741e00640 1 --2- 172.21.15.67:0/399047162 >> v2:172.21.15.169:3300/0 conn(0x7f773c15e5a0 0x7f773c0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.251+0000 7f7742800640 1 --2- 172.21.15.67:0/399047162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0d1420 0x7f773c06c740 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f773c08e450 tx=0x7f773802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b20c01259c1bccc server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.251+0000 7f7740a00640 1 -- 172.21.15.67:0/399047162 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738037070 con 0x7f773c0d1420 2024-05-30T17:07:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.251+0000 7f7740a00640 1 -- 172.21.15.67:0/399047162 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7738032440 con 0x7f773c0d1420 2024-05-30T17:07:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.251+0000 7f7740a00640 1 -- 172.21.15.67:0/399047162 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738032810 con 0x7f773c0d1420 2024-05-30T17:07:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.252+0000 7f7743200640 1 -- 172.21.15.67:0/399047162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0d1420 msgr2=0x7f773c06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.252+0000 7f7743200640 1 --2- 172.21.15.67:0/399047162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0d1420 0x7f773c06c740 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f773c08e450 tx=0x7f773802a270 comp rx=0 tx=0).stop 2024-05-30T17:07:48.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.252+0000 7f7743200640 1 -- 172.21.15.67:0/399047162 shutdown_connections 2024-05-30T17:07:48.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.252+0000 7f7743200640 1 --2- 172.21.15.67:0/399047162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0d1420 0x7f773c06c740 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.252+0000 7f7743200640 1 --2- 172.21.15.67:0/399047162 >> v2:172.21.15.169:3300/0 conn(0x7f773c15e5a0 0x7f773c0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.252+0000 7f7743200640 1 -- 172.21.15.67:0/399047162 >> 172.21.15.67:0/399047162 conn(0x7f773c08bfc0 msgr2=0x7f773c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:48.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.252+0000 7f7743200640 1 -- 172.21.15.67:0/399047162 shutdown_connections 2024-05-30T17:07:48.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.252+0000 7f7743200640 1 -- 172.21.15.67:0/399047162 wait complete. 2024-05-30T17:07:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.254+0000 7f7743200640 1 Processor -- start 2024-05-30T17:07:48.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.254+0000 7f7743200640 1 -- start start 2024-05-30T17:07:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.255+0000 7f7743200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0cfb40 0x7f773c0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.255+0000 7f7743200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f773c0d1420 0x7f773c0b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.255+0000 7f7743200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f773c15e5a0 0x7f773c1f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.255+0000 7f7743200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f773c15eb00 con 0x7f773c0cfb40 2024-05-30T17:07:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.255+0000 7f7743200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f773c15e980 con 0x7f773c0d1420 2024-05-30T17:07:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.255+0000 7f7743200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f773c0d5220 con 0x7f773c15e5a0 2024-05-30T17:07:48.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.255+0000 7f7741e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0cfb40 0x7f773c0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.255+0000 7f7741e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0cfb40 0x7f773c0ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49324/0 (socket says 172.21.15.67:49324) 2024-05-30T17:07:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.255+0000 7f7741e00640 1 -- 172.21.15.67:0/3561426176 learned_addr learned my addr 172.21.15.67:0/3561426176 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.256+0000 7f7741400640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f773c0d1420 0x7f773c0b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.256+0000 7f7741e00640 1 -- 172.21.15.67:0/3561426176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f773c15e5a0 msgr2=0x7f773c1f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.256+0000 7f7741e00640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f773c15e5a0 0x7f773c1f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.256+0000 7f7741e00640 1 -- 172.21.15.67:0/3561426176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f773c0d1420 msgr2=0x7f773c0b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.256+0000 7f7741e00640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f773c0d1420 0x7f773c0b80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.256+0000 7f7741e00640 1 -- 172.21.15.67:0/3561426176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c0b87b0 con 0x7f773c0cfb40 2024-05-30T17:07:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.256+0000 7f7741e00640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0cfb40 0x7f773c0ce1a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f773400ce30 tx=0x7f773400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.256+0000 7f7732000640 1 -- 172.21.15.67:0/3561426176 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773400f890 con 0x7f773c0cfb40 2024-05-30T17:07:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.257+0000 7f7732000640 1 -- 172.21.15.67:0/3561426176 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f773400fa30 con 0x7f773c0cfb40 2024-05-30T17:07:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.257+0000 7f7732000640 1 -- 172.21.15.67:0/3561426176 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773400fe00 con 0x7f773c0cfb40 2024-05-30T17:07:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.257+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c0b8a90 con 0x7f773c0cfb40 2024-05-30T17:07:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.257+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c0b9050 con 0x7f773c0cfb40 2024-05-30T17:07:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.258+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f773c0689e0 con 0x7f773c0cfb40 2024-05-30T17:07:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.259+0000 7f7732000640 1 -- 172.21.15.67:0/3561426176 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f7734018400 con 0x7f773c0cfb40 2024-05-30T17:07:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.260+0000 7f7732000640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f772006e510 0x7f77200709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.260+0000 7f7732000640 1 -- 172.21.15.67:0/3561426176 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f773408bf10 con 0x7f773c0cfb40 2024-05-30T17:07:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.260+0000 7f7741400640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f772006e510 0x7f77200709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.260+0000 7f7732000640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f7720076970 0x7f7720078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.260+0000 7f7732000640 1 -- 172.21.15.67:0/3561426176 --> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77200794e0 con 0x7f7720076970 2024-05-30T17:07:48.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.260+0000 7f7732000640 1 -- 172.21.15.67:0/3561426176 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f773408cea0 con 0x7f773c0cfb40 2024-05-30T17:07:48.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.261+0000 7f7742800640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f7720076970 0x7f7720078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.261+0000 7f7741400640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f772006e510 0x7f77200709d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f773c06db30 tx=0x7f772c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.262+0000 7f7742800640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f7720076970 0x7f7720078e10 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.262+0000 7f7732000640 1 -- 172.21.15.67:0/3561426176 <== osd.1 v2:172.21.15.67:6808/3260209820 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77200794e0 con 0x7f7720076970 2024-05-30T17:07:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.278+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 --> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f773c0d1d30 con 0x7f7720076970 2024-05-30T17:07:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.278+0000 7f7732000640 1 -- 172.21.15.67:0/3561426176 <== osd.1 v2:172.21.15.67:6808/3260209820 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f773c0d1d30 con 0x7f7720076970 2024-05-30T17:07:48.279 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:48.279 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:07:48.279 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:48.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f7720076970 msgr2=0x7f7720078e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6808/3260209820,v1:172.21.15.67:6809/3260209820] conn(0x7f7720076970 0x7f7720078e10 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f772006e510 msgr2=0x7f77200709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f772006e510 0x7f77200709d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f773c06db30 tx=0x7f772c009040 comp rx=0 tx=0).stop 2024-05-30T17:07:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0cfb40 msgr2=0x7f773c0ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 --2- 172.21.15.67:0/3561426176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f773c0cfb40 0x7f773c0ce1a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f773400ce30 tx=0x7f773400a650 comp rx=0 tx=0).stop 2024-05-30T17:07:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7742800640 1 -- 172.21.15.67:0/3561426176 reap_dead start 2024-05-30T17:07:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 shutdown_connections 2024-05-30T17:07:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 >> 172.21.15.67:0/3561426176 conn(0x7f773c08bfc0 msgr2=0x7f773c161630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 shutdown_connections 2024-05-30T17:07:48.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.279+0000 7f7743200640 1 -- 172.21.15.67:0/3561426176 wait complete. 2024-05-30T17:07:48.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-30T17:07:48.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.508+0000 7f01eb200640 1 Processor -- start 2024-05-30T17:07:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.508+0000 7f01eb200640 1 -- start start 2024-05-30T17:07:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.510+0000 7f01eb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01ec06ee40 0x7f01ec0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.510+0000 7f01eb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec06f2f0 0x7f01ec0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.510+0000 7f01eb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ec156f10 con 0x7f01ec156b30 2024-05-30T17:07:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.510+0000 7f01eb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f01ec06dda0 con 0x7f01ec06ee40 2024-05-30T17:07:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.510+0000 7f01eb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec06e080 con 0x7f01ec06f2f0 2024-05-30T17:07:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.511+0000 7f01e9400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01ec06ee40 0x7f01ec0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.511+0000 7f01e9400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01ec06ee40 0x7f01ec0ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.67:35190/0 (socket says 172.21.15.67:35190) 2024-05-30T17:07:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.511+0000 7f01e9400640 1 -- 172.21.15.67:0/796483309 learned_addr learned my addr 172.21.15.67:0/796483309 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.511+0000 7f01e9e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f01ec156b30 0x7f01ec0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.67:37800/0 (socket says 172.21.15.67:37800) 2024-05-30T17:07:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.511+0000 7f01ea800640 1 --2- 172.21.15.67:0/796483309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec06f2f0 0x7f01ec0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.511+0000 7f01e9400640 1 -- 172.21.15.67:0/796483309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec06f2f0 msgr2=0x7f01ec0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.511+0000 7f01e9400640 1 --2- 172.21.15.67:0/796483309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec06f2f0 0x7f01ec0ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.511+0000 7f01e9400640 1 -- 172.21.15.67:0/796483309 >> v1:172.21.15.169:6789/0 conn(0x7f01ec156b30 legacy=0x7f01ec0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.511+0000 7f01e9400640 1 -- 172.21.15.67:0/796483309 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec06e360 con 0x7f01ec06ee40 2024-05-30T17:07:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.513+0000 7f01e9400640 1 --2- 172.21.15.67:0/796483309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06ee40 0x7f01ec0ad230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009870 tx=0x7f01dc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d378869bd3120 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.513+0000 7f01e8a00640 1 -- 172.21.15.67:0/796483309 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc037070 con 0x7f01ec06ee40 2024-05-30T17:07:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.513+0000 7f01e8a00640 1 -- 172.21.15.67:0/796483309 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01dc032440 con 0x7f01ec06ee40 2024-05-30T17:07:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.513+0000 7f01e8a00640 1 -- 172.21.15.67:0/796483309 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc032810 con 0x7f01ec06ee40 2024-05-30T17:07:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.514+0000 7f01eb200640 1 -- 172.21.15.67:0/796483309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06ee40 msgr2=0x7f01ec0ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.514+0000 7f01eb200640 1 --2- 172.21.15.67:0/796483309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06ee40 0x7f01ec0ad230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009870 tx=0x7f01dc02a1c0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.514+0000 7f01eb200640 1 -- 172.21.15.67:0/796483309 shutdown_connections 2024-05-30T17:07:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.514+0000 7f01eb200640 1 --2- 172.21.15.67:0/796483309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec06f2f0 0x7f01ec0ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.514+0000 7f01eb200640 1 --2- 172.21.15.67:0/796483309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06ee40 0x7f01ec0ad230 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.514+0000 7f01eb200640 1 -- 172.21.15.67:0/796483309 >> 172.21.15.67:0/796483309 conn(0x7f01ec0b1fc0 msgr2=0x7f01ec0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.514+0000 7f01eb200640 1 -- 172.21.15.67:0/796483309 shutdown_connections 2024-05-30T17:07:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.514+0000 7f01eb200640 1 -- 172.21.15.67:0/796483309 wait complete. 2024-05-30T17:07:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.516+0000 7f01eb200640 1 Processor -- start 2024-05-30T17:07:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.516+0000 7f01eb200640 1 -- start start 2024-05-30T17:07:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.517+0000 7f01eb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ec06ee40 0x7f01ec153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.517+0000 7f01eb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06f2f0 0x7f01ec14c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.517+0000 7f01eb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec156b30 0x7f01ec155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.517+0000 7f01eb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec0b2db0 con 0x7f01ec06ee40 2024-05-30T17:07:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.517+0000 7f01eb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec0b2c30 con 0x7f01ec06f2f0 2024-05-30T17:07:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.517+0000 7f01eb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec0b32c0 con 0x7f01ec156b30 2024-05-30T17:07:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.517+0000 7f01e9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ec06ee40 0x7f01ec153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.517+0000 7f01e9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ec06ee40 0x7f01ec153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49338/0 (socket says 172.21.15.67:49338) 2024-05-30T17:07:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.517+0000 7f01e9e00640 1 -- 172.21.15.67:0/1832148790 learned_addr learned my addr 172.21.15.67:0/1832148790 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01e9400640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06f2f0 0x7f01ec14c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01ea800640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec156b30 0x7f01ec155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01e9e00640 1 -- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec156b30 msgr2=0x7f01ec155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01e9e00640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec156b30 0x7f01ec155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01e9e00640 1 -- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06f2f0 msgr2=0x7f01ec14c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01e9e00640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06f2f0 0x7f01ec14c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01e9e00640 1 -- 172.21.15.67:0/1832148790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec14ca30 con 0x7f01ec06ee40 2024-05-30T17:07:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01ea800640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec156b30 0x7f01ec155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01e9400640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06f2f0 0x7f01ec14c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-30T17:07:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.518+0000 7f01e9e00640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ec06ee40 0x7f01ec153640 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c760 tx=0x7f01e400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.519+0000 7f01e2000640 1 -- 172.21.15.67:0/1832148790 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4017070 con 0x7f01ec06ee40 2024-05-30T17:07:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.519+0000 7f01e2000640 1 -- 172.21.15.67:0/1832148790 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e400a640 con 0x7f01ec06ee40 2024-05-30T17:07:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.519+0000 7f01e2000640 1 -- 172.21.15.67:0/1832148790 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e400aa10 con 0x7f01ec06ee40 2024-05-30T17:07:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.519+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec151030 con 0x7f01ec06ee40 2024-05-30T17:07:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.519+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec1515f0 con 0x7f01ec06ee40 2024-05-30T17:07:48.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.520+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ec0689e0 con 0x7f01ec06ee40 2024-05-30T17:07:48.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.521+0000 7f01e2000640 1 -- 172.21.15.67:0/1832148790 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f01e401a070 con 0x7f01ec06ee40 2024-05-30T17:07:48.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.522+0000 7f01e2000640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f01c806e540 0x7f01c8070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.522+0000 7f01e2000640 1 -- 172.21.15.67:0/1832148790 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f01e408a090 con 0x7f01ec06ee40 2024-05-30T17:07:48.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.522+0000 7f01e9400640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f01c806e540 0x7f01c8070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.523+0000 7f01e9400640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f01c806e540 0x7f01c8070a00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f01dc034000 tx=0x7f01dc02e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.526+0000 7f01e2000640 1 -- 172.21.15.67:0/1832148790 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01e4092030 con 0x7f01ec06ee40 2024-05-30T17:07:48.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.731+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f01ec15fb50 con 0x7f01c806e540 2024-05-30T17:07:48.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.732+0000 7f01e2000640 1 -- 172.21.15.67:0/1832148790 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f01ec15fb50 con 0x7f01c806e540 2024-05-30T17:07:48.733 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 1 to deep-scrub 2024-05-30T17:07:48.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f01c806e540 msgr2=0x7f01c8070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f01c806e540 0x7f01c8070a00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f01dc034000 tx=0x7f01dc02e000 comp rx=0 tx=0).stop 2024-05-30T17:07:48.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ec06ee40 msgr2=0x7f01ec153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ec06ee40 0x7f01ec153640 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c760 tx=0x7f01e400cc20 comp rx=0 tx=0).stop 2024-05-30T17:07:48.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 shutdown_connections 2024-05-30T17:07:48.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f01c806e540 0x7f01c8070a00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ec156b30 0x7f01ec155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ec06f2f0 0x7f01ec14c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 --2- 172.21.15.67:0/1832148790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01ec06ee40 0x7f01ec153640 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 >> 172.21.15.67:0/1832148790 conn(0x7f01ec0b1fc0 msgr2=0x7f01ec0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:48.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 shutdown_connections 2024-05-30T17:07:48.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.735+0000 7f01eb200640 1 -- 172.21.15.67:0/1832148790 wait complete. 2024-05-30T17:07:48.754 INFO:tasks.ceph:Scrubbing osd.2 2024-05-30T17:07:48.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:07:48.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.971+0000 7f9e13e00640 1 Processor -- start 2024-05-30T17:07:48.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.971+0000 7f9e13e00640 1 -- start start 2024-05-30T17:07:48.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.973+0000 7f9e13e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0b5c80 0x7f9e0c0d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.973+0000 7f9e13e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0c0d5500 con 0x7f9e0c0d34e0 2024-05-30T17:07:48.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.973+0000 7f9e13e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0c0b4f50 con 0x7f9e0c1622b0 2024-05-30T17:07:48.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.973+0000 7f9e13e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0c0d2720 con 0x7f9e0c0b5c80 2024-05-30T17:07:48.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.973+0000 7f9e10a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9e0c0d34e0 0x7f9e0c0b5570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43872/0 (socket says 172.21.15.67:43872) 2024-05-30T17:07:48.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.973+0000 7f9e10a00640 1 -- 172.21.15.67:0/162005698 learned_addr learned my addr 172.21.15.67:0/162005698 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:48.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.973+0000 7f9e11e00640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0b5c80 0x7f9e0c0d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625762104 0 0) 0x7f9e0c0d5500 con 0x7f9e0c0d34e0 2024-05-30T17:07:48.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9df4003520 con 0x7f9e0c0d34e0 2024-05-30T17:07:48.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1137720711 0 0) 0x7f9df4003520 con 0x7f9e0c0d34e0 2024-05-30T17:07:48.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0b5c80 msgr2=0x7f9e0c0d4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:48.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0b5c80 0x7f9e0c0d4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 >> v1:172.21.15.169:6789/0 conn(0x7f9e0c1622b0 legacy=0x7f9e0c0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c0d2a00 con 0x7f9e0c0d34e0 2024-05-30T17:07:48.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f9e000019f0 con 0x7f9e0c0d34e0 2024-05-30T17:07:48.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 >> v1:172.21.15.67:6789/0 conn(0x7f9e0c0d34e0 legacy=0x7f9e0c0b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 0x7f9df4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4004800 0x7f9df4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0b5c80 0x7f9df4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.974+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c0d2720 con 0x7f9df40039e0 2024-05-30T17:07:48.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9df400bb20 con 0x7f9df4004800 2024-05-30T17:07:48.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9df400bd20 con 0x7f9e0c0b5c80 2024-05-30T17:07:48.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e11400640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 0x7f9df4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e10a00640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4004800 0x7f9df4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e11400640 1 -- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0b5c80 msgr2=0x7f9df4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e11400640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0b5c80 0x7f9df4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e11400640 1 -- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4004800 msgr2=0x7f9df4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e11400640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4004800 0x7f9df4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e11400640 1 -- 172.21.15.67:0/162005698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df4003520 con 0x7f9df40039e0 2024-05-30T17:07:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.975+0000 7f9e10a00640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9df4004800 0x7f9df4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:48.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.976+0000 7f9e11400640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 0x7f9df4003e00 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc001750 tx=0x7f9df4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f50142bc77a238e server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.977+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc001370 con 0x7f9df40039e0 2024-05-30T17:07:48.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.977+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dfc0052d0 con 0x7f9df40039e0 2024-05-30T17:07:48.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.977+0000 7f9e0be00640 1 -- 172.21.15.67:0/162005698 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc02abe0 con 0x7f9df40039e0 2024-05-30T17:07:48.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.977+0000 7f9e13e00640 1 -- 172.21.15.67:0/162005698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 msgr2=0x7f9df4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.977+0000 7f9e13e00640 1 --2- 172.21.15.67:0/162005698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 0x7f9df4003e00 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc001750 tx=0x7f9df4005830 comp rx=0 tx=0).stop 2024-05-30T17:07:48.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.977+0000 7f9e11e00640 1 -- 172.21.15.67:0/162005698 reap_dead start 2024-05-30T17:07:48.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.978+0000 7f9e13e00640 1 -- 172.21.15.67:0/162005698 shutdown_connections 2024-05-30T17:07:48.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.978+0000 7f9e13e00640 1 -- 172.21.15.67:0/162005698 >> 172.21.15.67:0/162005698 conn(0x7f9e0c08bfc0 msgr2=0x7f9e0c0d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:48.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.978+0000 7f9e13e00640 1 -- 172.21.15.67:0/162005698 shutdown_connections 2024-05-30T17:07:48.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.978+0000 7f9e13e00640 1 -- 172.21.15.67:0/162005698 wait complete. 2024-05-30T17:07:48.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.979+0000 7f9e13e00640 1 Processor -- start 2024-05-30T17:07:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.980+0000 7f9e13e00640 1 -- start start 2024-05-30T17:07:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e13e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 0x7f9e0c0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e13e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e0c157540 0x7f9e0c0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e13e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0cfbd0 0x7f9e0c0d0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e13e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c0d4f80 con 0x7f9df40039e0 2024-05-30T17:07:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e13e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c0d4e00 con 0x7f9e0c157540 2024-05-30T17:07:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e13e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0c0d5100 con 0x7f9e0c0cfbd0 2024-05-30T17:07:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e11400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 0x7f9e0c0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e11400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 0x7f9e0c0b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49352/0 (socket says 172.21.15.67:49352) 2024-05-30T17:07:48.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e11400640 1 -- 172.21.15.67:0/1791588387 learned_addr learned my addr 172.21.15.67:0/1791588387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:48.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e11400640 1 -- 172.21.15.67:0/1791588387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0cfbd0 msgr2=0x7f9e0c0d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e11400640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e0c0cfbd0 0x7f9e0c0d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e11400640 1 -- 172.21.15.67:0/1791588387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e0c157540 msgr2=0x7f9e0c0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:48.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e11400640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e0c157540 0x7f9e0c0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:48.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.981+0000 7f9e11400640 1 -- 172.21.15.67:0/1791588387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c0b8400 con 0x7f9df40039e0 2024-05-30T17:07:48.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.982+0000 7f9e11400640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 0x7f9e0c0b72a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc008fd0 tx=0x7f9dfc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.982+0000 7f9e09600640 1 -- 172.21.15.67:0/1791588387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc040020 con 0x7f9df40039e0 2024-05-30T17:07:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.982+0000 7f9e09600640 1 -- 172.21.15.67:0/1791588387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dfc005480 con 0x7f9df40039e0 2024-05-30T17:07:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.982+0000 7f9e09600640 1 -- 172.21.15.67:0/1791588387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc007a30 con 0x7f9df40039e0 2024-05-30T17:07:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.982+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0c1579a0 con 0x7f9df40039e0 2024-05-30T17:07:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.982+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0c0b7b60 con 0x7f9df40039e0 2024-05-30T17:07:48.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.983+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e0c0689e0 con 0x7f9df40039e0 2024-05-30T17:07:48.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.985+0000 7f9e09600640 1 -- 172.21.15.67:0/1791588387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f9dfc043020 con 0x7f9df40039e0 2024-05-30T17:07:48.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.986+0000 7f9e09600640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9df006e510 0x7f9df00709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.986+0000 7f9e09600640 1 -- 172.21.15.67:0/1791588387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f9dfc0aa9d0 con 0x7f9df40039e0 2024-05-30T17:07:48.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.986+0000 7f9e10a00640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9df006e510 0x7f9df00709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.986+0000 7f9e09600640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7f9df0076970 0x7f9df0078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:48.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.986+0000 7f9e09600640 1 -- 172.21.15.67:0/1791588387 --> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9dfc0052d0 con 0x7f9df0076970 2024-05-30T17:07:48.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.986+0000 7f9e09600640 1 -- 172.21.15.67:0/1791588387 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f9dfc0b6050 con 0x7f9df40039e0 2024-05-30T17:07:48.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.987+0000 7f9e11e00640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7f9df0076970 0x7f9df0078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:48.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.987+0000 7f9e10a00640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9df006e510 0x7f9df00709d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9e00007c40 tx=0x7f9e00009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.987+0000 7f9e11e00640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7f9df0076970 0x7f9df0078e10 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:48.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:48.988+0000 7f9e09600640 1 -- 172.21.15.67:0/1791588387 <== osd.2 v2:172.21.15.67:6800/2549377238 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9dfc0052d0 con 0x7f9df0076970 2024-05-30T17:07:49.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.008+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 --> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9e0c08c6c0 con 0x7f9df0076970 2024-05-30T17:07:49.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.008+0000 7f9e09600640 1 -- 172.21.15.67:0/1791588387 <== osd.2 v2:172.21.15.67:6800/2549377238 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9e0c08c6c0 con 0x7f9df0076970 2024-05-30T17:07:49.009 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:49.009 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:07:49.009 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:49.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7f9df0076970 msgr2=0x7f9df0078e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6800/2549377238,v1:172.21.15.67:6801/2549377238] conn(0x7f9df0076970 0x7f9df0078e10 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9df006e510 msgr2=0x7f9df00709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f9df006e510 0x7f9df00709d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9e00007c40 tx=0x7f9e00009040 comp rx=0 tx=0).stop 2024-05-30T17:07:49.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 msgr2=0x7f9e0c0b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 --2- 172.21.15.67:0/1791588387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9df40039e0 0x7f9e0c0b72a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc008fd0 tx=0x7f9dfc001120 comp rx=0 tx=0).stop 2024-05-30T17:07:49.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e11e00640 1 -- 172.21.15.67:0/1791588387 reap_dead start 2024-05-30T17:07:49.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 shutdown_connections 2024-05-30T17:07:49.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 >> 172.21.15.67:0/1791588387 conn(0x7f9e0c08bfc0 msgr2=0x7f9e0c161520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:49.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 shutdown_connections 2024-05-30T17:07:49.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.009+0000 7f9e13e00640 1 -- 172.21.15.67:0/1791588387 wait complete. 2024-05-30T17:07:49.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-30T17:07:49.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.233+0000 7f350b200640 1 Processor -- start 2024-05-30T17:07:49.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.233+0000 7f350b200640 1 -- start start 2024-05-30T17:07:49.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.235+0000 7f350b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f350c158b80 0x7f350c06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.235+0000 7f350b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f350c0722d0 0x7f350c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.235+0000 7f350b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350c06e6e0 con 0x7f350c06e160 2024-05-30T17:07:49.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.235+0000 7f350b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f350c06e9c0 con 0x7f350c0722d0 2024-05-30T17:07:49.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.235+0000 7f350b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f350c06eca0 con 0x7f350c158b80 2024-05-30T17:07:49.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.235+0000 7f3501400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f350c06e160 0x7f350c071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43874/0 (socket says 172.21.15.67:43874) 2024-05-30T17:07:49.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.235+0000 7f3501400640 1 -- 172.21.15.67:0/787834094 learned_addr learned my addr 172.21.15.67:0/787834094 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:49.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.235+0000 7f350a800640 1 --2- 172.21.15.67:0/787834094 >> v2:172.21.15.67:3300/0 conn(0x7f350c0722d0 0x7f350c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.236+0000 7f3509e00640 1 --2- 172.21.15.67:0/787834094 >> v2:172.21.15.169:3300/0 conn(0x7f350c158b80 0x7f350c06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.236+0000 7f350a800640 1 -- 172.21.15.67:0/787834094 >> v2:172.21.15.169:3300/0 conn(0x7f350c158b80 msgr2=0x7f350c06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:49.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.236+0000 7f350a800640 1 --2- 172.21.15.67:0/787834094 >> v2:172.21.15.169:3300/0 conn(0x7f350c158b80 0x7f350c06dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.236+0000 7f350a800640 1 -- 172.21.15.67:0/787834094 >> v1:172.21.15.67:6789/0 conn(0x7f350c06e160 legacy=0x7f350c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.236+0000 7f350a800640 1 -- 172.21.15.67:0/787834094 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c06ef80 con 0x7f350c0722d0 2024-05-30T17:07:49.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.236+0000 7f3509400640 1 -- 172.21.15.67:0/787834094 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699548504 0 0) 0x7f350c06e6e0 con 0x7f350c06e160 2024-05-30T17:07:49.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.237+0000 7f350a800640 1 --2- 172.21.15.67:0/787834094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c0722d0 0x7f350c0b32c0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f350c06c830 tx=0x7f350402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b29b81653442e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.237+0000 7f3509400640 1 -- 172.21.15.67:0/787834094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504037070 con 0x7f350c0722d0 2024-05-30T17:07:49.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f3509400640 1 -- 172.21.15.67:0/787834094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3504032440 con 0x7f350c0722d0 2024-05-30T17:07:49.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f3509400640 1 -- 172.21.15.67:0/787834094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504032810 con 0x7f350c0722d0 2024-05-30T17:07:49.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f350b200640 1 -- 172.21.15.67:0/787834094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c0722d0 msgr2=0x7f350c0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f350b200640 1 --2- 172.21.15.67:0/787834094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c0722d0 0x7f350c0b32c0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f350c06c830 tx=0x7f350402a1a0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f350b200640 1 -- 172.21.15.67:0/787834094 shutdown_connections 2024-05-30T17:07:49.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f350b200640 1 --2- 172.21.15.67:0/787834094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c0722d0 0x7f350c0b32c0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f350b200640 1 --2- 172.21.15.67:0/787834094 >> v2:172.21.15.169:3300/0 conn(0x7f350c158b80 0x7f350c06dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f350b200640 1 -- 172.21.15.67:0/787834094 >> 172.21.15.67:0/787834094 conn(0x7f350c0b1fc0 msgr2=0x7f350c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:49.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f350b200640 1 -- 172.21.15.67:0/787834094 shutdown_connections 2024-05-30T17:07:49.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.238+0000 7f350b200640 1 -- 172.21.15.67:0/787834094 wait complete. 2024-05-30T17:07:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.240+0000 7f350b200640 1 Processor -- start 2024-05-30T17:07:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.240+0000 7f350b200640 1 -- start start 2024-05-30T17:07:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.241+0000 7f350b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c06e160 0x7f350c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.241+0000 7f350b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f350c0722d0 0x7f350c156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.241+0000 7f350b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f350c158b80 0x7f350c0b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.241+0000 7f350b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f350c06f780 con 0x7f350c06e160 2024-05-30T17:07:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.241+0000 7f350b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f350c0b2d30 con 0x7f350c0722d0 2024-05-30T17:07:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.241+0000 7f350b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f350c09d740 con 0x7f350c158b80 2024-05-30T17:07:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.241+0000 7f3509e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c06e160 0x7f350c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.241+0000 7f3509e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c06e160 0x7f350c1563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49376/0 (socket says 172.21.15.67:49376) 2024-05-30T17:07:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.241+0000 7f3509e00640 1 -- 172.21.15.67:0/1454913249 learned_addr learned my addr 172.21.15.67:0/1454913249 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.242+0000 7f3501400640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f350c0722d0 0x7f350c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.242+0000 7f3509e00640 1 -- 172.21.15.67:0/1454913249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f350c158b80 msgr2=0x7f350c0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.242+0000 7f3509e00640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f350c158b80 0x7f350c0b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.242+0000 7f3509e00640 1 -- 172.21.15.67:0/1454913249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f350c0722d0 msgr2=0x7f350c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.242+0000 7f3509e00640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f350c0722d0 0x7f350c156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.242+0000 7f3509e00640 1 -- 172.21.15.67:0/1454913249 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c09da20 con 0x7f350c06e160 2024-05-30T17:07:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.242+0000 7f3509e00640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c06e160 0x7f350c1563d0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f34fc007e40 tx=0x7f34fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.242+0000 7f3502a00640 1 -- 172.21.15.67:0/1454913249 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc014620 con 0x7f350c06e160 2024-05-30T17:07:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.243+0000 7f3502a00640 1 -- 172.21.15.67:0/1454913249 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc019070 con 0x7f350c06e160 2024-05-30T17:07:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.243+0000 7f3502a00640 1 -- 172.21.15.67:0/1454913249 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc014a00 con 0x7f350c06e160 2024-05-30T17:07:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.243+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350c09dd00 con 0x7f350c06e160 2024-05-30T17:07:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.243+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350c0b3480 con 0x7f350c06e160 2024-05-30T17:07:49.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.244+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f350c0689e0 con 0x7f350c06e160 2024-05-30T17:07:49.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.245+0000 7f3502a00640 1 -- 172.21.15.67:0/1454913249 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f34fc014ba0 con 0x7f350c06e160 2024-05-30T17:07:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.246+0000 7f3502a00640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34e806e510 0x7f34e80709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.247+0000 7f3502a00640 1 -- 172.21.15.67:0/1454913249 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f34fc08b450 con 0x7f350c06e160 2024-05-30T17:07:49.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.247+0000 7f3501400640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34e806e510 0x7f34e80709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.248+0000 7f3501400640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34e806e510 0x7f34e80709d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f350c15eb90 tx=0x7f34f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.251+0000 7f3502a00640 1 -- 172.21.15.67:0/1454913249 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34fc05b2e0 con 0x7f350c06e160 2024-05-30T17:07:49.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.454+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f350c06cbc0 con 0x7f34e806e510 2024-05-30T17:07:49.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.455+0000 7f3502a00640 1 -- 172.21.15.67:0/1454913249 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f350c06cbc0 con 0x7f34e806e510 2024-05-30T17:07:49.456 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 2 to deep-scrub 2024-05-30T17:07:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34e806e510 msgr2=0x7f34e80709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34e806e510 0x7f34e80709d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f350c15eb90 tx=0x7f34f4009040 comp rx=0 tx=0).stop 2024-05-30T17:07:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c06e160 msgr2=0x7f350c1563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c06e160 0x7f350c1563d0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f34fc007e40 tx=0x7f34fc00e4b0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 shutdown_connections 2024-05-30T17:07:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f34e806e510 0x7f34e80709d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f350c158b80 0x7f350c0b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f350c0722d0 0x7f350c156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 --2- 172.21.15.67:0/1454913249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f350c06e160 0x7f350c1563d0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 >> 172.21.15.67:0/1454913249 conn(0x7f350c0b1fc0 msgr2=0x7f350c0d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 shutdown_connections 2024-05-30T17:07:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.458+0000 7f350b200640 1 -- 172.21.15.67:0/1454913249 wait complete. 2024-05-30T17:07:49.478 INFO:tasks.ceph:Scrubbing osd.3 2024-05-30T17:07:49.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:07:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.687+0000 7f667f200640 1 Processor -- start 2024-05-30T17:07:49.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.687+0000 7f667f200640 1 -- start start 2024-05-30T17:07:49.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6680162e00 0x7f6680166e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6680167490 con 0x7f6680163e60 2024-05-30T17:07:49.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6680163380 con 0x7f66800c5300 2024-05-30T17:07:49.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f6680163660 con 0x7f6680162e00 2024-05-30T17:07:49.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6680163e60 0x7f6680166760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43884/0 (socket says 172.21.15.67:43884) 2024-05-30T17:07:49.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667d400640 1 -- 172.21.15.67:0/3441066867 learned_addr learned my addr 172.21.15.67:0/3441066867 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:49.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667e800640 1 --2- 172.21.15.67:0/3441066867 >> v2:172.21.15.67:3300/0 conn(0x7f6680162e00 0x7f6680166e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667ca00640 1 -- 172.21.15.67:0/3441066867 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1093894890 0 0) 0x7f6680167490 con 0x7f6680163e60 2024-05-30T17:07:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667ca00640 1 -- 172.21.15.67:0/3441066867 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6664003520 con 0x7f6680163e60 2024-05-30T17:07:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.689+0000 7f667e800640 1 -- 172.21.15.67:0/3441066867 >> v1:172.21.15.169:6789/0 conn(0x7f66800c5300 legacy=0x7f66800c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.690+0000 7f667e800640 1 -- 172.21.15.67:0/3441066867 >> v1:172.21.15.67:6789/0 conn(0x7f6680163e60 legacy=0x7f6680166760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.690+0000 7f667e800640 1 -- 172.21.15.67:0/3441066867 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6680163940 con 0x7f6680162e00 2024-05-30T17:07:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.691+0000 7f667e800640 1 --2- 172.21.15.67:0/3441066867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680162e00 0x7f6680166e70 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f6678009d90 tx=0x7f667802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b7bcc10eb5004b server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.691+0000 7f667ca00640 1 -- 172.21.15.67:0/3441066867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678007740 con 0x7f6680162e00 2024-05-30T17:07:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.691+0000 7f667ca00640 1 -- 172.21.15.67:0/3441066867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66780078e0 con 0x7f6680162e00 2024-05-30T17:07:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.691+0000 7f667ca00640 1 -- 172.21.15.67:0/3441066867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678007cb0 con 0x7f6680162e00 2024-05-30T17:07:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.692+0000 7f667f200640 1 -- 172.21.15.67:0/3441066867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680162e00 msgr2=0x7f6680166e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.692+0000 7f667f200640 1 --2- 172.21.15.67:0/3441066867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680162e00 0x7f6680166e70 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f6678009d90 tx=0x7f667802a660 comp rx=0 tx=0).stop 2024-05-30T17:07:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.692+0000 7f667f200640 1 -- 172.21.15.67:0/3441066867 shutdown_connections 2024-05-30T17:07:49.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.692+0000 7f667f200640 1 --2- 172.21.15.67:0/3441066867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680162e00 0x7f6680166e70 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.692+0000 7f667f200640 1 -- 172.21.15.67:0/3441066867 >> 172.21.15.67:0/3441066867 conn(0x7f66800b3e20 msgr2=0x7f66800b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.692+0000 7f667f200640 1 -- 172.21.15.67:0/3441066867 shutdown_connections 2024-05-30T17:07:49.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.692+0000 7f667f200640 1 -- 172.21.15.67:0/3441066867 wait complete. 2024-05-30T17:07:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.694+0000 7f667f200640 1 Processor -- start 2024-05-30T17:07:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.695+0000 7f667f200640 1 -- start start 2024-05-30T17:07:49.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.696+0000 7f667f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66800c5300 0x7f6680161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.696+0000 7f667f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6680162e00 0x7f66800b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.696+0000 7f667f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680163e60 0x7f66800cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.696+0000 7f667f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66800af430 con 0x7f6680163e60 2024-05-30T17:07:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.696+0000 7f667f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66800af2b0 con 0x7f6680162e00 2024-05-30T17:07:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.696+0000 7f667f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66800af5b0 con 0x7f66800c5300 2024-05-30T17:07:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.696+0000 7f667e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680163e60 0x7f66800cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.696+0000 7f667e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680163e60 0x7f66800cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49396/0 (socket says 172.21.15.67:49396) 2024-05-30T17:07:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.696+0000 7f667e800640 1 -- 172.21.15.67:0/334393359 learned_addr learned my addr 172.21.15.67:0/334393359 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.697+0000 7f667d400640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6680162e00 0x7f66800b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.697+0000 7f667de00640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66800c5300 0x7f6680161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.697+0000 7f667e800640 1 -- 172.21.15.67:0/334393359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66800c5300 msgr2=0x7f6680161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.697+0000 7f667e800640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66800c5300 0x7f6680161bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.697+0000 7f667e800640 1 -- 172.21.15.67:0/334393359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6680162e00 msgr2=0x7f66800b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.697+0000 7f667e800640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6680162e00 0x7f66800b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.697+0000 7f667e800640 1 -- 172.21.15.67:0/334393359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668015fb50 con 0x7f6680163e60 2024-05-30T17:07:49.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.697+0000 7f667e800640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680163e60 0x7f66800cafc0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6678009eb0 tx=0x7f6678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.698+0000 7f6676000640 1 -- 172.21.15.67:0/334393359 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678040070 con 0x7f6680163e60 2024-05-30T17:07:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.698+0000 7f6676000640 1 -- 172.21.15.67:0/334393359 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6678031bc0 con 0x7f6680163e60 2024-05-30T17:07:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.698+0000 7f6676000640 1 -- 172.21.15.67:0/334393359 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678009450 con 0x7f6680163e60 2024-05-30T17:07:49.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.698+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66800b5eb0 con 0x7f6680163e60 2024-05-30T17:07:49.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.698+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66800b6470 con 0x7f6680163e60 2024-05-30T17:07:49.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.699+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66800690f0 con 0x7f6680163e60 2024-05-30T17:07:49.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.700+0000 7f6676000640 1 -- 172.21.15.67:0/334393359 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f6678039040 con 0x7f6680163e60 2024-05-30T17:07:49.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.701+0000 7f6676000640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f665c06e510 0x7f665c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.701+0000 7f6676000640 1 -- 172.21.15.67:0/334393359 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f66780ac6d0 con 0x7f6680163e60 2024-05-30T17:07:49.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.701+0000 7f667de00640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f665c06e510 0x7f665c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.701+0000 7f6676000640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f665c076970 0x7f665c078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.701+0000 7f6676000640 1 -- 172.21.15.67:0/334393359 --> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f667804cec0 con 0x7f665c076970 2024-05-30T17:07:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.701+0000 7f6676000640 1 -- 172.21.15.67:0/334393359 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f66780adbb0 con 0x7f6680163e60 2024-05-30T17:07:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.702+0000 7f667d400640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f665c076970 0x7f665c078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.702+0000 7f667de00640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f665c06e510 0x7f665c0709d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6670000f60 tx=0x7f6670009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.702+0000 7f667d400640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f665c076970 0x7f665c078e10 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.703+0000 7f6676000640 1 -- 172.21.15.67:0/334393359 <== osd.3 v2:172.21.15.67:6814/2982751245 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f667804cec0 con 0x7f665c076970 2024-05-30T17:07:49.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.718+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 --> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f66800b1680 con 0x7f665c076970 2024-05-30T17:07:49.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.719+0000 7f6676000640 1 -- 172.21.15.67:0/334393359 <== osd.3 v2:172.21.15.67:6814/2982751245 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f66800b1680 con 0x7f665c076970 2024-05-30T17:07:49.719 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:49.720 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:07:49.720 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.719+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f665c076970 msgr2=0x7f665c078e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.719+0000 7f667f200640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6814/2982751245,v1:172.21.15.67:6816/2982751245] conn(0x7f665c076970 0x7f665c078e10 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.719+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f665c06e510 msgr2=0x7f665c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.719+0000 7f667f200640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f665c06e510 0x7f665c0709d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6670000f60 tx=0x7f6670009040 comp rx=0 tx=0).stop 2024-05-30T17:07:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.719+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680163e60 msgr2=0x7f66800cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.719+0000 7f667f200640 1 --2- 172.21.15.67:0/334393359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6680163e60 0x7f66800cafc0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6678009eb0 tx=0x7f6678004a40 comp rx=0 tx=0).stop 2024-05-30T17:07:49.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.719+0000 7f667e800640 1 -- 172.21.15.67:0/334393359 reap_dead start 2024-05-30T17:07:49.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.720+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 shutdown_connections 2024-05-30T17:07:49.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.720+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 >> 172.21.15.67:0/334393359 conn(0x7f66800b3e20 msgr2=0x7f66800b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:49.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.720+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 shutdown_connections 2024-05-30T17:07:49.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.720+0000 7f667f200640 1 -- 172.21.15.67:0/334393359 wait complete. 2024-05-30T17:07:49.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-30T17:07:49.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.935+0000 7ff933200640 1 Processor -- start 2024-05-30T17:07:49.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.935+0000 7ff933200640 1 -- start start 2024-05-30T17:07:49.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.937+0000 7ff933200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340cfa40 0x7ff934169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.937+0000 7ff933200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff93415ec50 con 0x7ff9340d16a0 2024-05-30T17:07:49.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.937+0000 7ff933200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9340cff80 con 0x7ff93415e870 2024-05-30T17:07:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.937+0000 7ff933200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9340d2150 con 0x7ff9340cfa40 2024-05-30T17:07:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.937+0000 7ff929400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff9340d16a0 0x7ff9340cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43894/0 (socket says 172.21.15.67:43894) 2024-05-30T17:07:49.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.937+0000 7ff929400640 1 -- 172.21.15.67:0/3602136297 learned_addr learned my addr 172.21.15.67:0/3602136297 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.937+0000 7ff932800640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340cfa40 0x7ff934169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.937+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2173584849 0 0) 0x7ff93415ec50 con 0x7ff9340d16a0 2024-05-30T17:07:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff918003520 con 0x7ff9340d16a0 2024-05-30T17:07:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3524547856 0 0) 0x7ff918003520 con 0x7ff9340d16a0 2024-05-30T17:07:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340cfa40 msgr2=0x7ff934169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340cfa40 0x7ff934169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 >> v1:172.21.15.169:6789/0 conn(0x7ff93415e870 legacy=0x7ff9340d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9340d0260 con 0x7ff9340d16a0 2024-05-30T17:07:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7ff91c0017f0 con 0x7ff9340d16a0 2024-05-30T17:07:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 >> v1:172.21.15.67:6789/0 conn(0x7ff9340d16a0 legacy=0x7ff9340cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180039e0 0x7ff918003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff918004800 0x7ff918004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340cfa40 0x7ff9180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340d2150 con 0x7ff918004800 2024-05-30T17:07:49.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff91800ba70 con 0x7ff9180039e0 2024-05-30T17:07:49.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff91800bc70 con 0x7ff9340cfa40 2024-05-30T17:07:49.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.938+0000 7ff929400640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff918004800 0x7ff918004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.939+0000 7ff931e00640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180039e0 0x7ff918003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.939+0000 7ff932800640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340cfa40 0x7ff9180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.939+0000 7ff929400640 1 -- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340cfa40 msgr2=0x7ff9180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:49.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.939+0000 7ff929400640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340cfa40 0x7ff9180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.939+0000 7ff929400640 1 -- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180039e0 msgr2=0x7ff918003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:49.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.939+0000 7ff929400640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180039e0 0x7ff918003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.939+0000 7ff929400640 1 -- 172.21.15.67:0/3602136297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff918003520 con 0x7ff918004800 2024-05-30T17:07:49.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.939+0000 7ff932800640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340cfa40 0x7ff9180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:49.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.940+0000 7ff929400640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff918004800 0x7ff918004c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0011d0 tx=0x7ff918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752760f904033364 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.940+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c007d50 con 0x7ff918004800 2024-05-30T17:07:49.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.940+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91c007270 con 0x7ff918004800 2024-05-30T17:07:49.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.940+0000 7ff931400640 1 -- 172.21.15.67:0/3602136297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c0057a0 con 0x7ff918004800 2024-05-30T17:07:49.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.941+0000 7ff933200640 1 -- 172.21.15.67:0/3602136297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff918004800 msgr2=0x7ff918004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.941+0000 7ff933200640 1 --2- 172.21.15.67:0/3602136297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff918004800 0x7ff918004c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0011d0 tx=0x7ff918005830 comp rx=0 tx=0).stop 2024-05-30T17:07:49.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.941+0000 7ff932800640 1 -- 172.21.15.67:0/3602136297 reap_dead start 2024-05-30T17:07:49.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.941+0000 7ff933200640 1 -- 172.21.15.67:0/3602136297 shutdown_connections 2024-05-30T17:07:49.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.941+0000 7ff933200640 1 -- 172.21.15.67:0/3602136297 >> 172.21.15.67:0/3602136297 conn(0x7ff93408bfc0 msgr2=0x7ff9340d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:49.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.941+0000 7ff933200640 1 -- 172.21.15.67:0/3602136297 shutdown_connections 2024-05-30T17:07:49.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.941+0000 7ff933200640 1 -- 172.21.15.67:0/3602136297 wait complete. 2024-05-30T17:07:49.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.943+0000 7ff933200640 1 Processor -- start 2024-05-30T17:07:49.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.943+0000 7ff933200640 1 -- start start 2024-05-30T17:07:49.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.944+0000 7ff933200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff918004800 0x7ff934150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.944+0000 7ff933200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340ca1a0 0x7ff9341608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.944+0000 7ff933200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff93415f850 0x7ff934160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.944+0000 7ff933200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340d4fb0 con 0x7ff93415f850 2024-05-30T17:07:49.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.944+0000 7ff933200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340d0820 con 0x7ff918004800 2024-05-30T17:07:49.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.944+0000 7ff933200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9340d35f0 con 0x7ff9340ca1a0 2024-05-30T17:07:49.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.944+0000 7ff932800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff93415f850 0x7ff934160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.944+0000 7ff932800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff93415f850 0x7ff934160e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49426/0 (socket says 172.21.15.67:49426) 2024-05-30T17:07:49.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.944+0000 7ff932800640 1 -- 172.21.15.67:0/515968014 learned_addr learned my addr 172.21.15.67:0/515968014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:49.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.945+0000 7ff931e00640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff918004800 0x7ff934150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.945+0000 7ff929400640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340ca1a0 0x7ff9341608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.945+0000 7ff932800640 1 -- 172.21.15.67:0/515968014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340ca1a0 msgr2=0x7ff9341608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.945+0000 7ff932800640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340ca1a0 0x7ff9341608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.945+0000 7ff932800640 1 -- 172.21.15.67:0/515968014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff918004800 msgr2=0x7ff934150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:49.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.945+0000 7ff932800640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff918004800 0x7ff934150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:49.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.945+0000 7ff932800640 1 -- 172.21.15.67:0/515968014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9341606b0 con 0x7ff93415f850 2024-05-30T17:07:49.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.945+0000 7ff932800640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff93415f850 0x7ff934160e00 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009fd0 tx=0x7ff92c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.945+0000 7ff92aa00640 1 -- 172.21.15.67:0/515968014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c01c070 con 0x7ff93415f850 2024-05-30T17:07:49.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.946+0000 7ff92aa00640 1 -- 172.21.15.67:0/515968014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92c017400 con 0x7ff93415f850 2024-05-30T17:07:49.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.946+0000 7ff92aa00640 1 -- 172.21.15.67:0/515968014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0177f0 con 0x7ff93415f850 2024-05-30T17:07:49.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.946+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93415fe10 con 0x7ff93415f850 2024-05-30T17:07:49.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.946+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9341603d0 con 0x7ff93415f850 2024-05-30T17:07:49.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.947+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9340689e0 con 0x7ff93415f850 2024-05-30T17:07:49.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.948+0000 7ff92aa00640 1 -- 172.21.15.67:0/515968014 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7ff92c005d10 con 0x7ff93415f850 2024-05-30T17:07:49.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.949+0000 7ff92aa00640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff91006e540 0x7ff910070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:49.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.949+0000 7ff92aa00640 1 -- 172.21.15.67:0/515968014 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7ff92c08ace0 con 0x7ff93415f850 2024-05-30T17:07:49.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.949+0000 7ff931e00640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff91006e540 0x7ff910070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:49.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.950+0000 7ff931e00640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff91006e540 0x7ff910070a00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff924005490 tx=0x7ff924009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:49.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:49.954+0000 7ff92aa00640 1 -- 172.21.15.67:0/515968014 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff92c012070 con 0x7ff93415f850 2024-05-30T17:07:50.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.164+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7ff934150c60 con 0x7ff91006e540 2024-05-30T17:07:50.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.165+0000 7ff92aa00640 1 -- 172.21.15.67:0/515968014 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff924001350 con 0x7ff91006e540 2024-05-30T17:07:50.165 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 3 to deep-scrub 2024-05-30T17:07:50.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.167+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff91006e540 msgr2=0x7ff910070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.167+0000 7ff933200640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff91006e540 0x7ff910070a00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff924005490 tx=0x7ff924009040 comp rx=0 tx=0).stop 2024-05-30T17:07:50.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.167+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff93415f850 msgr2=0x7ff934160e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.167+0000 7ff933200640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff93415f850 0x7ff934160e00 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009fd0 tx=0x7ff92c00f4a0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.168+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 shutdown_connections 2024-05-30T17:07:50.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.168+0000 7ff933200640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7ff91006e540 0x7ff910070a00 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.168+0000 7ff933200640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff93415f850 0x7ff934160e00 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.168+0000 7ff933200640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9340ca1a0 0x7ff9341608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.168+0000 7ff933200640 1 --2- 172.21.15.67:0/515968014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff918004800 0x7ff934150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.168+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 >> 172.21.15.67:0/515968014 conn(0x7ff93408bfc0 msgr2=0x7ff934074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:50.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.168+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 shutdown_connections 2024-05-30T17:07:50.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.168+0000 7ff933200640 1 -- 172.21.15.67:0/515968014 wait complete. 2024-05-30T17:07:50.188 INFO:tasks.ceph:Scrubbing osd.4 2024-05-30T17:07:50.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:07:50.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.402+0000 7f3240c00640 1 Processor -- start 2024-05-30T17:07:50.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.402+0000 7f3240c00640 1 -- start start 2024-05-30T17:07:50.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.403+0000 7f3240c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c15e5a0 0x7f323c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.403+0000 7f3240c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f323c0cfb40 0x7f323c0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.403+0000 7f3240c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f323c0cffa0 0x7f323c0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.403+0000 7f3240c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f323c0d2b30 con 0x7f323c0cffa0 2024-05-30T17:07:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.403+0000 7f3240c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f323c0d3500 con 0x7f323c0cfb40 2024-05-30T17:07:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.403+0000 7f3240c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f323c0d2200 con 0x7f323c15e5a0 2024-05-30T17:07:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f323c0cffa0 0x7f323c0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f323c0cffa0 0x7f323c0d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49428/0 (socket says 172.21.15.67:49428) 2024-05-30T17:07:50.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323be00640 1 -- 172.21.15.67:0/3458300200 learned_addr learned my addr 172.21.15.67:0/3458300200 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:50.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323b400640 1 --2- 172.21.15.67:0/3458300200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c15e5a0 0x7f323c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323aa00640 1 --2- 172.21.15.67:0/3458300200 >> v2:172.21.15.169:3300/0 conn(0x7f323c0cfb40 0x7f323c0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323be00640 1 -- 172.21.15.67:0/3458300200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c15e5a0 msgr2=0x7f323c0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:50.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323be00640 1 --2- 172.21.15.67:0/3458300200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c15e5a0 0x7f323c0d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323be00640 1 -- 172.21.15.67:0/3458300200 >> v2:172.21.15.169:3300/0 conn(0x7f323c0cfb40 msgr2=0x7f323c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:50.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323be00640 1 --2- 172.21.15.67:0/3458300200 >> v2:172.21.15.169:3300/0 conn(0x7f323c0cfb40 0x7f323c0d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323be00640 1 -- 172.21.15.67:0/3458300200 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323c0d24e0 con 0x7f323c0cffa0 2024-05-30T17:07:50.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323b400640 1 --2- 172.21.15.67:0/3458300200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c15e5a0 0x7f323c0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:50.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.404+0000 7f323aa00640 1 --2- 172.21.15.67:0/3458300200 >> v2:172.21.15.169:3300/0 conn(0x7f323c0cfb40 0x7f323c0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:50.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.405+0000 7f323be00640 1 --2- 172.21.15.67:0/3458300200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cffa0 0x7f323c0d1250 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f323c073eb0 tx=0x7f323402a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a9f8ecb384fb6c server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:50.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f323a000640 1 -- 172.21.15.67:0/3458300200 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3234036070 con 0x7f323c0cffa0 2024-05-30T17:07:50.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f323a000640 1 -- 172.21.15.67:0/3458300200 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3234031600 con 0x7f323c0cffa0 2024-05-30T17:07:50.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f323a000640 1 -- 172.21.15.67:0/3458300200 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32340319d0 con 0x7f323c0cffa0 2024-05-30T17:07:50.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f3240c00640 1 -- 172.21.15.67:0/3458300200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cffa0 msgr2=0x7f323c0d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f3240c00640 1 --2- 172.21.15.67:0/3458300200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cffa0 0x7f323c0d1250 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f323c073eb0 tx=0x7f323402a690 comp rx=0 tx=0).stop 2024-05-30T17:07:50.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f3240c00640 1 -- 172.21.15.67:0/3458300200 shutdown_connections 2024-05-30T17:07:50.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f3240c00640 1 --2- 172.21.15.67:0/3458300200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cffa0 0x7f323c0d1250 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f3240c00640 1 --2- 172.21.15.67:0/3458300200 >> v2:172.21.15.169:3300/0 conn(0x7f323c0cfb40 0x7f323c0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f3240c00640 1 --2- 172.21.15.67:0/3458300200 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c15e5a0 0x7f323c0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f3240c00640 1 -- 172.21.15.67:0/3458300200 >> 172.21.15.67:0/3458300200 conn(0x7f323c08bfc0 msgr2=0x7f323c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:50.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f3240c00640 1 -- 172.21.15.67:0/3458300200 shutdown_connections 2024-05-30T17:07:50.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.406+0000 7f3240c00640 1 -- 172.21.15.67:0/3458300200 wait complete. 2024-05-30T17:07:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.408+0000 7f3240c00640 1 Processor -- start 2024-05-30T17:07:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.408+0000 7f3240c00640 1 -- start start 2024-05-30T17:07:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.409+0000 7f3240c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cfb40 0x7f323c1f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.409+0000 7f3240c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c0cffa0 0x7f323c0b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.409+0000 7f3240c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f323c15e5a0 0x7f323c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.409+0000 7f3240c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f323c15eb00 con 0x7f323c0cfb40 2024-05-30T17:07:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.409+0000 7f3240c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f323c15e980 con 0x7f323c15e5a0 2024-05-30T17:07:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.409+0000 7f3240c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f323c0d5220 con 0x7f323c0cffa0 2024-05-30T17:07:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cfb40 0x7f323c1f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cfb40 0x7f323c1f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49432/0 (socket says 172.21.15.67:49432) 2024-05-30T17:07:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323b400640 1 -- 172.21.15.67:0/1243201042 learned_addr learned my addr 172.21.15.67:0/1243201042 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323aa00640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c0cffa0 0x7f323c0b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323b400640 1 -- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c0cffa0 msgr2=0x7f323c0b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323b400640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f323c0cffa0 0x7f323c0b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323be00640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f323c15e5a0 0x7f323c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323b400640 1 -- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f323c15e5a0 msgr2=0x7f323c0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323b400640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f323c15e5a0 0x7f323c0b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.410+0000 7f323b400640 1 -- 172.21.15.67:0/1243201042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323c0b7a60 con 0x7f323c0cfb40 2024-05-30T17:07:50.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.411+0000 7f323b400640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cfb40 0x7f323c1f4b30 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c7f0 tx=0x7f322c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:50.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.411+0000 7f3233400640 1 -- 172.21.15.67:0/1243201042 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c015070 con 0x7f323c0cfb40 2024-05-30T17:07:50.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.411+0000 7f3233400640 1 -- 172.21.15.67:0/1243201042 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322c007a30 con 0x7f323c0cfb40 2024-05-30T17:07:50.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.411+0000 7f3233400640 1 -- 172.21.15.67:0/1243201042 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c007e00 con 0x7f323c0cfb40 2024-05-30T17:07:50.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.411+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323c0b7d40 con 0x7f323c0cfb40 2024-05-30T17:07:50.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.411+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323c1f4610 con 0x7f323c0cfb40 2024-05-30T17:07:50.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.412+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f323c1f4140 con 0x7f323c0cfb40 2024-05-30T17:07:50.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.413+0000 7f3233400640 1 -- 172.21.15.67:0/1243201042 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f322c018070 con 0x7f323c0cfb40 2024-05-30T17:07:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.414+0000 7f3233400640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f321c06e510 0x7f321c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.414+0000 7f3233400640 1 -- 172.21.15.67:0/1243201042 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f322c08aa10 con 0x7f323c0cfb40 2024-05-30T17:07:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.415+0000 7f323aa00640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f321c06e510 0x7f321c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.415+0000 7f3233400640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f321c076970 0x7f321c078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.415+0000 7f3233400640 1 -- 172.21.15.67:0/1243201042 --> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f322c00fe50 con 0x7f321c076970 2024-05-30T17:07:50.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.415+0000 7f3233400640 1 -- 172.21.15.67:0/1243201042 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f322c096050 con 0x7f323c0cfb40 2024-05-30T17:07:50.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.415+0000 7f323aa00640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f321c06e510 0x7f321c0709d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f323c15d6f0 tx=0x7f3228008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:50.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.415+0000 7f323be00640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f321c076970 0x7f321c078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.417+0000 7f323be00640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f321c076970 0x7f321c078e10 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:50.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.419+0000 7f3233400640 1 -- 172.21.15.67:0/1243201042 <== osd.4 v2:172.21.15.169:6812/432873034 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f322c00fe50 con 0x7f321c076970 2024-05-30T17:07:50.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.434+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 --> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f323c0689e0 con 0x7f321c076970 2024-05-30T17:07:50.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.435+0000 7f3233400640 1 -- 172.21.15.67:0/1243201042 <== osd.4 v2:172.21.15.169:6812/432873034 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f323c0689e0 con 0x7f321c076970 2024-05-30T17:07:50.436 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:50.436 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:07:50.436 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:50.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.435+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f321c076970 msgr2=0x7f321c078e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f3240c00640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.169:6812/432873034,v1:172.21.15.169:6814/432873034] conn(0x7f321c076970 0x7f321c078e10 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f321c06e510 msgr2=0x7f321c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f3240c00640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f321c06e510 0x7f321c0709d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f323c15d6f0 tx=0x7f3228008040 comp rx=0 tx=0).stop 2024-05-30T17:07:50.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cfb40 msgr2=0x7f323c1f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f3240c00640 1 --2- 172.21.15.67:0/1243201042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323c0cfb40 0x7f323c1f4b30 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c7f0 tx=0x7f322c00cbf0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f323be00640 1 -- 172.21.15.67:0/1243201042 reap_dead start 2024-05-30T17:07:50.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 shutdown_connections 2024-05-30T17:07:50.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 >> 172.21.15.67:0/1243201042 conn(0x7f323c08bfc0 msgr2=0x7f323c0b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:50.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 shutdown_connections 2024-05-30T17:07:50.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.436+0000 7f3240c00640 1 -- 172.21.15.67:0/1243201042 wait complete. 2024-05-30T17:07:50.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-30T17:07:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.664+0000 7f8ff3200640 1 Processor -- start 2024-05-30T17:07:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.665+0000 7f8ff3200640 1 -- start start 2024-05-30T17:07:50.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.666+0000 7f8ff3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff406c010 0x7f8ff40b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.666+0000 7f8ff3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ff40b2a60 0x7f8ff40b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.667+0000 7f8ff3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff409ae10 con 0x7f8ff415c6b0 2024-05-30T17:07:50.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.667+0000 7f8ff3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff40b3ee0 con 0x7f8ff40b2a60 2024-05-30T17:07:50.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.667+0000 7f8ff3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff40b2420 con 0x7f8ff406c010 2024-05-30T17:07:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.667+0000 7f8ff1e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8ff415c6b0 0x7f8ff406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43910/0 (socket says 172.21.15.67:43910) 2024-05-30T17:07:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.667+0000 7f8ff1e00640 1 -- 172.21.15.67:0/60558646 learned_addr learned my addr 172.21.15.67:0/60558646 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.667+0000 7f8febe00640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff406c010 0x7f8ff40b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.667+0000 7f8ff2800640 1 --2- 172.21.15.67:0/60558646 >> v2:172.21.15.169:3300/0 conn(0x7f8ff40b2a60 0x7f8ff40b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.667+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 275267171 0 0) 0x7f8ff409ae10 con 0x7f8ff415c6b0 2024-05-30T17:07:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.667+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fd8003520 con 0x7f8ff415c6b0 2024-05-30T17:07:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2080957839 0 0) 0x7f8fd8003520 con 0x7f8ff415c6b0 2024-05-30T17:07:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff406c010 msgr2=0x7f8ff40b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff406c010 0x7f8ff40b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 >> v2:172.21.15.169:3300/0 conn(0x7f8ff40b2a60 msgr2=0x7f8ff40b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 --2- 172.21.15.67:0/60558646 >> v2:172.21.15.169:3300/0 conn(0x7f8ff40b2a60 0x7f8ff40b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff40b2700 con 0x7f8ff415c6b0 2024-05-30T17:07:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7f8fe4001670 con 0x7f8ff415c6b0 2024-05-30T17:07:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 >> v1:172.21.15.67:6789/0 conn(0x7f8ff415c6b0 legacy=0x7f8ff406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8fd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd8004800 0x7f8fd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff406c010 0x7f8fd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff40b3ee0 con 0x7f8fd80039b0 2024-05-30T17:07:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff40b2420 con 0x7f8fd8004800 2024-05-30T17:07:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd800b9c0 con 0x7f8ff406c010 2024-05-30T17:07:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.668+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8fd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.669+0000 7f8febe00640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd8004800 0x7f8fd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.669+0000 7f8ff2800640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff406c010 0x7f8fd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.669+0000 7f8ff1e00640 1 -- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff406c010 msgr2=0x7f8fd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.669+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff406c010 0x7f8fd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.669+0000 7f8ff1e00640 1 -- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd8004800 msgr2=0x7f8fd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.669+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd8004800 0x7f8fd8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.669+0000 7f8ff1e00640 1 -- 172.21.15.67:0/60558646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd8003520 con 0x7f8fd80039b0 2024-05-30T17:07:50.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.670+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8fd8003dd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c190 tx=0x7f8fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c107f928e37c7f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:50.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.670+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4007c80 con 0x7f8fd80039b0 2024-05-30T17:07:50.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.670+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe402e070 con 0x7f8fd80039b0 2024-05-30T17:07:50.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.670+0000 7f8ff1400640 1 -- 172.21.15.67:0/60558646 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4005d50 con 0x7f8fd80039b0 2024-05-30T17:07:50.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.671+0000 7f8ff3200640 1 -- 172.21.15.67:0/60558646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 msgr2=0x7f8fd8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.671+0000 7f8ff3200640 1 --2- 172.21.15.67:0/60558646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8fd8003dd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c190 tx=0x7f8fd8005830 comp rx=0 tx=0).stop 2024-05-30T17:07:50.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.671+0000 7f8ff2800640 1 -- 172.21.15.67:0/60558646 reap_dead start 2024-05-30T17:07:50.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.671+0000 7f8ff3200640 1 -- 172.21.15.67:0/60558646 shutdown_connections 2024-05-30T17:07:50.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.671+0000 7f8ff3200640 1 -- 172.21.15.67:0/60558646 >> 172.21.15.67:0/60558646 conn(0x7f8ff40b6a10 msgr2=0x7f8ff40b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.671+0000 7f8ff3200640 1 -- 172.21.15.67:0/60558646 shutdown_connections 2024-05-30T17:07:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.671+0000 7f8ff3200640 1 -- 172.21.15.67:0/60558646 wait complete. 2024-05-30T17:07:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.673+0000 7f8ff3200640 1 Processor -- start 2024-05-30T17:07:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.673+0000 7f8ff3200640 1 -- start start 2024-05-30T17:07:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.674+0000 7f8ff3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8ff414ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.674+0000 7f8ff3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff40cf7a0 0x7f8ff4158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.674+0000 7f8ff3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ff4157880 0x7f8ff4157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.674+0000 7f8ff3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff415ca90 con 0x7f8fd80039b0 2024-05-30T17:07:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.674+0000 7f8ff3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff40b39c0 con 0x7f8ff4157880 2024-05-30T17:07:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.674+0000 7f8ff3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff409aa30 con 0x7f8ff40cf7a0 2024-05-30T17:07:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.674+0000 7f8ff1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8ff414ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.674+0000 7f8ff1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8ff414ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49452/0 (socket says 172.21.15.67:49452) 2024-05-30T17:07:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.674+0000 7f8ff1e00640 1 -- 172.21.15.67:0/3053887247 learned_addr learned my addr 172.21.15.67:0/3053887247 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.675+0000 7f8ff2800640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ff4157880 0x7f8ff4157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.675+0000 7f8febe00640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff40cf7a0 0x7f8ff4158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.675+0000 7f8ff1e00640 1 -- 172.21.15.67:0/3053887247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff40cf7a0 msgr2=0x7f8ff4158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.675+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff40cf7a0 0x7f8ff4158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.675+0000 7f8ff1e00640 1 -- 172.21.15.67:0/3053887247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ff4157880 msgr2=0x7f8ff4157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.675+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ff4157880 0x7f8ff4157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.675+0000 7f8ff1e00640 1 -- 172.21.15.67:0/3053887247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff415a2b0 con 0x7f8fd80039b0 2024-05-30T17:07:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.675+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8ff414ae80 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c2b0 tx=0x7f8fe4001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.675+0000 7f8fea000640 1 -- 172.21.15.67:0/3053887247 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4044070 con 0x7f8fd80039b0 2024-05-30T17:07:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.676+0000 7f8fea000640 1 -- 172.21.15.67:0/3053887247 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe4007480 con 0x7f8fd80039b0 2024-05-30T17:07:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.676+0000 7f8fea000640 1 -- 172.21.15.67:0/3053887247 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4007c80 con 0x7f8fd80039b0 2024-05-30T17:07:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.676+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff4158f90 con 0x7f8fd80039b0 2024-05-30T17:07:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.676+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff4159510 con 0x7f8fd80039b0 2024-05-30T17:07:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.677+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff4153610 con 0x7f8fd80039b0 2024-05-30T17:07:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.678+0000 7f8fea000640 1 -- 172.21.15.67:0/3053887247 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f8fe4033470 con 0x7f8fd80039b0 2024-05-30T17:07:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.679+0000 7f8fea000640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8fd006e510 0x7f8fd00709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.679+0000 7f8fea000640 1 -- 172.21.15.67:0/3053887247 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f8fe40aa430 con 0x7f8fd80039b0 2024-05-30T17:07:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.679+0000 7f8febe00640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8fd006e510 0x7f8fd00709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.680+0000 7f8febe00640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8fd006e510 0x7f8fd00709d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8ff41584a0 tx=0x7f8fdc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:50.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.683+0000 7f8fea000640 1 -- 172.21.15.67:0/3053887247 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ff4153610 con 0x7f8fd80039b0 2024-05-30T17:07:50.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.892+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f8ff40690f0 con 0x7f8fd006e510 2024-05-30T17:07:50.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.892+0000 7f8fea000640 1 -- 172.21.15.67:0/3053887247 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8ff40690f0 con 0x7f8fd006e510 2024-05-30T17:07:50.893 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 4 to deep-scrub 2024-05-30T17:07:50.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.895+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8fd006e510 msgr2=0x7f8fd00709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.895+0000 7f8ff3200640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8fd006e510 0x7f8fd00709d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8ff41584a0 tx=0x7f8fdc008040 comp rx=0 tx=0).stop 2024-05-30T17:07:50.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 msgr2=0x7f8ff414ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:50.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8ff414ae80 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c2b0 tx=0x7f8fe4001970 comp rx=0 tx=0).stop 2024-05-30T17:07:50.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 shutdown_connections 2024-05-30T17:07:50.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f8fd006e510 0x7f8fd00709d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ff4157880 0x7f8ff4157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ff40cf7a0 0x7f8ff4158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 --2- 172.21.15.67:0/3053887247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd80039b0 0x7f8ff414ae80 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:50.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 >> 172.21.15.67:0/3053887247 conn(0x7f8ff40b6a10 msgr2=0x7f8ff40b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:50.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 shutdown_connections 2024-05-30T17:07:50.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:50.896+0000 7f8ff3200640 1 -- 172.21.15.67:0/3053887247 wait complete. 2024-05-30T17:07:50.916 INFO:tasks.ceph:Scrubbing osd.5 2024-05-30T17:07:50.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:07:51.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.123+0000 7fa9b0a00640 1 Processor -- start 2024-05-30T17:07:51.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.124+0000 7fa9b0a00640 1 -- start start 2024-05-30T17:07:51.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.125+0000 7fa9b0a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa9ac15e5a0 0x7fa9ac0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.125+0000 7fa9b0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9ac0d1420 0x7fa9ac06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.125+0000 7fa9b0a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ac0d3500 con 0x7fa9ac0cfb40 2024-05-30T17:07:51.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.125+0000 7fa9b0a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ac0d2b30 con 0x7fa9ac0d1420 2024-05-30T17:07:51.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.125+0000 7fa9b0a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ac0d2200 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.126+0000 7fa9a3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa9ac0cfb40 0x7fa9ac0d0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33684/0 (socket says 172.21.15.67:33684) 2024-05-30T17:07:51.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.126+0000 7fa9a3e00640 1 -- 172.21.15.67:0/2696948034 learned_addr learned my addr 172.21.15.67:0/2696948034 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:51.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.126+0000 7fa9ab200640 1 --2- 172.21.15.67:0/2696948034 >> v2:172.21.15.67:3300/0 conn(0x7fa9ac0d1420 0x7fa9ac06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.126+0000 7fa9aa800640 1 --2- 172.21.15.67:0/2696948034 >> v2:172.21.15.169:3300/0 conn(0x7fa9ac15e5a0 0x7fa9ac0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.126+0000 7fa9ab200640 1 -- 172.21.15.67:0/2696948034 >> v2:172.21.15.169:3300/0 conn(0x7fa9ac15e5a0 msgr2=0x7fa9ac0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.126+0000 7fa9ab200640 1 --2- 172.21.15.67:0/2696948034 >> v2:172.21.15.169:3300/0 conn(0x7fa9ac15e5a0 0x7fa9ac0d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.126+0000 7fa9ab200640 1 -- 172.21.15.67:0/2696948034 >> v1:172.21.15.67:6789/0 conn(0x7fa9ac0cfb40 legacy=0x7fa9ac0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.126+0000 7fa9ab200640 1 -- 172.21.15.67:0/2696948034 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac0d24e0 con 0x7fa9ac0d1420 2024-05-30T17:07:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.128+0000 7fa9ab200640 1 --2- 172.21.15.67:0/2696948034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac0d1420 0x7fa9ac06c740 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac08e450 tx=0x7fa9a402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45e93be08f4bbb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.128+0000 7fa9a9e00640 1 -- 172.21.15.67:0/2696948034 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4007740 con 0x7fa9ac0d1420 2024-05-30T17:07:51.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.128+0000 7fa9a9e00640 1 -- 172.21.15.67:0/2696948034 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a40078e0 con 0x7fa9ac0d1420 2024-05-30T17:07:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.128+0000 7fa9a9e00640 1 -- 172.21.15.67:0/2696948034 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4007cb0 con 0x7fa9ac0d1420 2024-05-30T17:07:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.128+0000 7fa9b0a00640 1 -- 172.21.15.67:0/2696948034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac0d1420 msgr2=0x7fa9ac06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.128+0000 7fa9b0a00640 1 --2- 172.21.15.67:0/2696948034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac0d1420 0x7fa9ac06c740 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac08e450 tx=0x7fa9a402a680 comp rx=0 tx=0).stop 2024-05-30T17:07:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.129+0000 7fa9b0a00640 1 -- 172.21.15.67:0/2696948034 shutdown_connections 2024-05-30T17:07:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.129+0000 7fa9b0a00640 1 --2- 172.21.15.67:0/2696948034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac0d1420 0x7fa9ac06c740 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.129+0000 7fa9b0a00640 1 --2- 172.21.15.67:0/2696948034 >> v2:172.21.15.169:3300/0 conn(0x7fa9ac15e5a0 0x7fa9ac0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.129+0000 7fa9b0a00640 1 -- 172.21.15.67:0/2696948034 >> 172.21.15.67:0/2696948034 conn(0x7fa9ac08bfc0 msgr2=0x7fa9ac0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.129+0000 7fa9b0a00640 1 -- 172.21.15.67:0/2696948034 shutdown_connections 2024-05-30T17:07:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.129+0000 7fa9b0a00640 1 -- 172.21.15.67:0/2696948034 wait complete. 2024-05-30T17:07:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.130+0000 7fa9b0a00640 1 Processor -- start 2024-05-30T17:07:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.131+0000 7fa9b0a00640 1 -- start start 2024-05-30T17:07:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.131+0000 7fa9b0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9ac0cfb40 0x7fa9ac1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.131+0000 7fa9b0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9ac0d1420 0x7fa9ac0b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.131+0000 7fa9b0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac15e5a0 0x7fa9ac0b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.131+0000 7fa9b0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac15eb00 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.131+0000 7fa9b0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac15e980 con 0x7fa9ac0cfb40 2024-05-30T17:07:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.131+0000 7fa9b0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ac0d5220 con 0x7fa9ac0d1420 2024-05-30T17:07:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9ab200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac15e5a0 0x7fa9ac0b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9ab200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac15e5a0 0x7fa9ac0b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40290/0 (socket says 172.21.15.67:40290) 2024-05-30T17:07:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9ab200640 1 -- 172.21.15.67:0/4047135167 learned_addr learned my addr 172.21.15.67:0/4047135167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9aa800640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9ac0cfb40 0x7fa9ac1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9a3e00640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9ac0d1420 0x7fa9ac0b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9ab200640 1 -- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9ac0d1420 msgr2=0x7fa9ac0b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9ab200640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9ac0d1420 0x7fa9ac0b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9ab200640 1 -- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9ac0cfb40 msgr2=0x7fa9ac1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9ab200640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9ac0cfb40 0x7fa9ac1517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9ab200640 1 -- 172.21.15.67:0/4047135167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac151d30 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.132+0000 7fa9a3e00640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9ac0d1420 0x7fa9ac0b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:51.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.133+0000 7fa9ab200640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac15e5a0 0x7fa9ac0b2710 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa9a402ac70 tx=0x7fa9a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.133+0000 7fa9a2a00640 1 -- 172.21.15.67:0/4047135167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4040070 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.133+0000 7fa9a2a00640 1 -- 172.21.15.67:0/4047135167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a4031bc0 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.133+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ac0b9a80 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.133+0000 7fa9a2a00640 1 -- 172.21.15.67:0/4047135167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4045460 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.133+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ac0ba040 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.134+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9ac0689e0 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.136+0000 7fa9a2a00640 1 -- 172.21.15.67:0/4047135167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7fa9a4039040 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.137+0000 7fa9a2a00640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa98806e510 0x7fa9880709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.137+0000 7fa9a2a00640 1 -- 172.21.15.67:0/4047135167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fa9a40ac4c0 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.137+0000 7fa9aa800640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa98806e510 0x7fa9880709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.137+0000 7fa9a2a00640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7fa988076910 0x7fa988078db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.137+0000 7fa9a2a00640 1 -- 172.21.15.67:0/4047135167 --> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa988079480 con 0x7fa988076910 2024-05-30T17:07:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.137+0000 7fa9a2a00640 1 -- 172.21.15.67:0/4047135167 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7fa9a40b8050 con 0x7fa9ac15e5a0 2024-05-30T17:07:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.138+0000 7fa9aa800640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa98806e510 0x7fa9880709d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.138+0000 7fa9a3e00640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7fa988076910 0x7fa988078db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.139+0000 7fa9a3e00640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7fa988076910 0x7fa988078db0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.141+0000 7fa9a2a00640 1 -- 172.21.15.67:0/4047135167 <== osd.5 v2:172.21.15.169:6824/3094236492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa988079480 con 0x7fa988076910 2024-05-30T17:07:51.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.156+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 --> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa9ac0d1d30 con 0x7fa988076910 2024-05-30T17:07:51.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.157+0000 7fa9a2a00640 1 -- 172.21.15.67:0/4047135167 <== osd.5 v2:172.21.15.169:6824/3094236492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa9ac0d1d30 con 0x7fa988076910 2024-05-30T17:07:51.157 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:51.158 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:07:51.158 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.157+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7fa988076910 msgr2=0x7fa988078db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.157+0000 7fa9b0a00640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.169:6824/3094236492,v1:172.21.15.169:6825/3094236492] conn(0x7fa988076910 0x7fa988078db0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.158+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa98806e510 msgr2=0x7fa9880709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.158+0000 7fa9b0a00640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fa98806e510 0x7fa9880709d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c008040 comp rx=0 tx=0).stop 2024-05-30T17:07:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.158+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac15e5a0 msgr2=0x7fa9ac0b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.158+0000 7fa9b0a00640 1 --2- 172.21.15.67:0/4047135167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9ac15e5a0 0x7fa9ac0b2710 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa9a402ac70 tx=0x7fa9a4004a40 comp rx=0 tx=0).stop 2024-05-30T17:07:51.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.158+0000 7fa9ab200640 1 -- 172.21.15.67:0/4047135167 reap_dead start 2024-05-30T17:07:51.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.158+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 shutdown_connections 2024-05-30T17:07:51.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.158+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 >> 172.21.15.67:0/4047135167 conn(0x7fa9ac08bfc0 msgr2=0x7fa9ac1615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:51.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.158+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 shutdown_connections 2024-05-30T17:07:51.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.158+0000 7fa9b0a00640 1 -- 172.21.15.67:0/4047135167 wait complete. 2024-05-30T17:07:51.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-30T17:07:51.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.386+0000 7fb683600640 1 Processor -- start 2024-05-30T17:07:51.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.387+0000 7fb683600640 1 -- start start 2024-05-30T17:07:51.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.388+0000 7fb683600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb67c0d3350 0x7fb67c0cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.388+0000 7fb683600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0cf850 0x7fb67c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.388+0000 7fb683600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb67c162580 con 0x7fb67c1621a0 2024-05-30T17:07:51.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.388+0000 7fb683600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb67c0b39c0 con 0x7fb67c0d3350 2024-05-30T17:07:51.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.388+0000 7fb683600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb67c0d0b70 con 0x7fb67c0cf850 2024-05-30T17:07:51.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.388+0000 7fb680c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb67c1621a0 0x7fb67c0d2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33698/0 (socket says 172.21.15.67:33698) 2024-05-30T17:07:51.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.388+0000 7fb680c00640 1 -- 172.21.15.67:0/2277571518 learned_addr learned my addr 172.21.15.67:0/2277571518 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:07:51.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb681600640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0cf850 0x7fb67c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67be00640 1 --2- 172.21.15.67:0/2277571518 >> v2:172.21.15.169:3300/0 conn(0x7fb67c0d3350 0x7fb67c0cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132143197 0 0) 0x7fb67c162580 con 0x7fb67c1621a0 2024-05-30T17:07:51.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb660003520 con 0x7fb67c1621a0 2024-05-30T17:07:51.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2342816057 0 0) 0x7fb660003520 con 0x7fb67c1621a0 2024-05-30T17:07:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0cf850 msgr2=0x7fb67c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0cf850 0x7fb67c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 >> v2:172.21.15.169:3300/0 conn(0x7fb67c0d3350 msgr2=0x7fb67c0cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 --2- 172.21.15.67:0/2277571518 >> v2:172.21.15.169:3300/0 conn(0x7fb67c0d3350 0x7fb67c0cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c0cfff0 con 0x7fb67c1621a0 2024-05-30T17:07:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fb66c001670 con 0x7fb67c1621a0 2024-05-30T17:07:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 >> v1:172.21.15.67:6789/0 conn(0x7fb67c1621a0 legacy=0x7fb67c0d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0cf850 0x7fb6600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb660003f80 0x7fb660004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660004d40 0x7fb6600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c0d0b70 con 0x7fb660003f80 2024-05-30T17:07:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb66000baa0 con 0x7fb660004d40 2024-05-30T17:07:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.389+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb66000bca0 con 0x7fb67c0cf850 2024-05-30T17:07:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.390+0000 7fb67be00640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb660003f80 0x7fb660004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.390+0000 7fb67be00640 1 -- 172.21.15.67:0/2277571518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0cf850 msgr2=0x7fb6600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.390+0000 7fb67be00640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0cf850 0x7fb6600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.390+0000 7fb67be00640 1 -- 172.21.15.67:0/2277571518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660004d40 msgr2=0x7fb6600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:51.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.390+0000 7fb67be00640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660004d40 0x7fb6600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.390+0000 7fb67be00640 1 -- 172.21.15.67:0/2277571518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb660003520 con 0x7fb660003f80 2024-05-30T17:07:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.391+0000 7fb67be00640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb660003f80 0x7fb660004800 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fb66400ba30 tx=0x7fb660005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94116a75b6b0e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.391+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664030070 con 0x7fb660003f80 2024-05-30T17:07:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.391+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb66402ae10 con 0x7fb660003f80 2024-05-30T17:07:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.391+0000 7fb67b400640 1 -- 172.21.15.67:0/2277571518 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66400b600 con 0x7fb660003f80 2024-05-30T17:07:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.392+0000 7fb683600640 1 -- 172.21.15.67:0/2277571518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb660003f80 msgr2=0x7fb660004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.392+0000 7fb683600640 1 --2- 172.21.15.67:0/2277571518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb660003f80 0x7fb660004800 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fb66400ba30 tx=0x7fb660005830 comp rx=0 tx=0).stop 2024-05-30T17:07:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.392+0000 7fb681600640 1 -- 172.21.15.67:0/2277571518 reap_dead start 2024-05-30T17:07:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.392+0000 7fb683600640 1 -- 172.21.15.67:0/2277571518 shutdown_connections 2024-05-30T17:07:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.392+0000 7fb683600640 1 -- 172.21.15.67:0/2277571518 >> 172.21.15.67:0/2277571518 conn(0x7fb67c08bfc0 msgr2=0x7fb67c0d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.392+0000 7fb683600640 1 -- 172.21.15.67:0/2277571518 shutdown_connections 2024-05-30T17:07:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.392+0000 7fb683600640 1 -- 172.21.15.67:0/2277571518 wait complete. 2024-05-30T17:07:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.394+0000 7fb683600640 1 Processor -- start 2024-05-30T17:07:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.394+0000 7fb683600640 1 -- start start 2024-05-30T17:07:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.395+0000 7fb683600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660003f80 0x7fb67c15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.395+0000 7fb683600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0b9e50 0x7fb67c1f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.395+0000 7fb683600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c1f4c10 0x7fb67c0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.395+0000 7fb683600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c0d0890 con 0x7fb67c1f4c10 2024-05-30T17:07:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.395+0000 7fb683600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c0d05b0 con 0x7fb660003f80 2024-05-30T17:07:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.395+0000 7fb683600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb67c0cfca0 con 0x7fb67c0b9e50 2024-05-30T17:07:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb681600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c1f4c10 0x7fb67c0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb681600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c1f4c10 0x7fb67c0b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40320/0 (socket says 172.21.15.67:40320) 2024-05-30T17:07:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb681600640 1 -- 172.21.15.67:0/1421617176 learned_addr learned my addr 172.21.15.67:0/1421617176 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb680c00640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660003f80 0x7fb67c15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb681600640 1 -- 172.21.15.67:0/1421617176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0b9e50 msgr2=0x7fb67c1f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb67be00640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0b9e50 0x7fb67c1f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb681600640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0b9e50 0x7fb67c1f46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb681600640 1 -- 172.21.15.67:0/1421617176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660003f80 msgr2=0x7fb67c15bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb681600640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660003f80 0x7fb67c15bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb681600640 1 -- 172.21.15.67:0/1421617176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c1f5110 con 0x7fb67c1f4c10 2024-05-30T17:07:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.396+0000 7fb680c00640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660003f80 0x7fb67c15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.397+0000 7fb681600640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c1f4c10 0x7fb67c0b4590 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fb67c08db40 tx=0x7fb67400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.397+0000 7fb678c00640 1 -- 172.21.15.67:0/1421617176 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674018020 con 0x7fb67c1f4c10 2024-05-30T17:07:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.397+0000 7fb678c00640 1 -- 172.21.15.67:0/1421617176 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67400bb10 con 0x7fb67c1f4c10 2024-05-30T17:07:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.397+0000 7fb678c00640 1 -- 172.21.15.67:0/1421617176 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674014400 con 0x7fb67c1f4c10 2024-05-30T17:07:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.397+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67c1f53f0 con 0x7fb67c1f4c10 2024-05-30T17:07:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.397+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67c0ccd50 con 0x7fb67c1f4c10 2024-05-30T17:07:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.398+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb67c0d4220 con 0x7fb67c1f4c10 2024-05-30T17:07:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.399+0000 7fb678c00640 1 -- 172.21.15.67:0/1421617176 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7fb674005d10 con 0x7fb67c1f4c10 2024-05-30T17:07:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.400+0000 7fb678c00640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb65806e560 0x7fb658070a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.400+0000 7fb678c00640 1 -- 172.21.15.67:0/1421617176 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fb67408ac80 con 0x7fb67c1f4c10 2024-05-30T17:07:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.401+0000 7fb680c00640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb65806e560 0x7fb658070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.401+0000 7fb680c00640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb65806e560 0x7fb658070a20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb67c15fa10 tx=0x7fb66c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.405+0000 7fb678c00640 1 -- 172.21.15.67:0/1421617176 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb67c0d4220 con 0x7fb67c1f4c10 2024-05-30T17:07:51.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.614+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fb67c0689e0 con 0x7fb65806e560 2024-05-30T17:07:51.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.615+0000 7fb678c00640 1 -- 172.21.15.67:0/1421617176 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb67c0689e0 con 0x7fb65806e560 2024-05-30T17:07:51.615 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 5 to deep-scrub 2024-05-30T17:07:51.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb65806e560 msgr2=0x7fb658070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb65806e560 0x7fb658070a20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb67c15fa10 tx=0x7fb66c009040 comp rx=0 tx=0).stop 2024-05-30T17:07:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c1f4c10 msgr2=0x7fb67c0b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c1f4c10 0x7fb67c0b4590 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fb67c08db40 tx=0x7fb67400b480 comp rx=0 tx=0).stop 2024-05-30T17:07:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 shutdown_connections 2024-05-30T17:07:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb65806e560 0x7fb658070a20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb67c1f4c10 0x7fb67c0b4590 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb67c0b9e50 0x7fb67c1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 --2- 172.21.15.67:0/1421617176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb660003f80 0x7fb67c15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 >> 172.21.15.67:0/1421617176 conn(0x7fb67c08bfc0 msgr2=0x7fb67c06ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:51.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 shutdown_connections 2024-05-30T17:07:51.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.618+0000 7fb683600640 1 -- 172.21.15.67:0/1421617176 wait complete. 2024-05-30T17:07:51.639 INFO:tasks.ceph:Scrubbing osd.6 2024-05-30T17:07:51.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:07:51.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.847+0000 7f88ad800640 1 Processor -- start 2024-05-30T17:07:51.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.847+0000 7f88ad800640 1 -- start start 2024-05-30T17:07:51.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.848+0000 7f88ad800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88a815fa00 0x7f88a806e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.848+0000 7f88ad800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88a80d0b40 0x7f88a8072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.848+0000 7f88ad800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a806fa00 con 0x7f88a81600f0 2024-05-30T17:07:51.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.848+0000 7f88ad800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f88a806eeb0 con 0x7f88a815fa00 2024-05-30T17:07:51.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.848+0000 7f88ad800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88a806f160 con 0x7f88a80d0b40 2024-05-30T17:07:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.849+0000 7f88a7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88a815fa00 0x7f88a806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.849+0000 7f88a7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88a815fa00 0x7f88a806e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40332/0 (socket says 172.21.15.67:40332) 2024-05-30T17:07:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.849+0000 7f88a7e00640 1 -- 172.21.15.67:0/3393525217 learned_addr learned my addr 172.21.15.67:0/3393525217 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.849+0000 7f88ace00640 1 --2- 172.21.15.67:0/3393525217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88a80d0b40 0x7f88a8072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.850+0000 7f88a7e00640 1 -- 172.21.15.67:0/3393525217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88a80d0b40 msgr2=0x7f88a8072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.850+0000 7f88a7e00640 1 --2- 172.21.15.67:0/3393525217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88a80d0b40 0x7f88a8072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.850+0000 7f88a7e00640 1 -- 172.21.15.67:0/3393525217 >> v1:172.21.15.169:6789/0 conn(0x7f88a81600f0 legacy=0x7f88a80d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.850+0000 7f88a7e00640 1 -- 172.21.15.67:0/3393525217 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a806f440 con 0x7f88a815fa00 2024-05-30T17:07:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.852+0000 7f88a7e00640 1 --2- 172.21.15.67:0/3393525217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a815fa00 0x7f88a806e970 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8898009e50 tx=0x7f889802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70059062ba7fdf1f server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.852+0000 7f88a7400640 1 -- 172.21.15.67:0/3393525217 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898036070 con 0x7f88a815fa00 2024-05-30T17:07:51.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.852+0000 7f88a7400640 1 -- 172.21.15.67:0/3393525217 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8898031600 con 0x7f88a815fa00 2024-05-30T17:07:51.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.852+0000 7f88a7400640 1 -- 172.21.15.67:0/3393525217 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88980319d0 con 0x7f88a815fa00 2024-05-30T17:07:51.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.853+0000 7f88ad800640 1 -- 172.21.15.67:0/3393525217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a815fa00 msgr2=0x7f88a806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.853+0000 7f88ad800640 1 --2- 172.21.15.67:0/3393525217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a815fa00 0x7f88a806e970 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8898009e50 tx=0x7f889802a5f0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.853+0000 7f88ad800640 1 -- 172.21.15.67:0/3393525217 shutdown_connections 2024-05-30T17:07:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.853+0000 7f88ad800640 1 --2- 172.21.15.67:0/3393525217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88a80d0b40 0x7f88a8072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.853+0000 7f88ad800640 1 --2- 172.21.15.67:0/3393525217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a815fa00 0x7f88a806e970 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.853+0000 7f88ad800640 1 -- 172.21.15.67:0/3393525217 >> 172.21.15.67:0/3393525217 conn(0x7f88a806bae0 msgr2=0x7f88a806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.853+0000 7f88ad800640 1 -- 172.21.15.67:0/3393525217 shutdown_connections 2024-05-30T17:07:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.853+0000 7f88ad800640 1 -- 172.21.15.67:0/3393525217 wait complete. 2024-05-30T17:07:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.855+0000 7f88ad800640 1 Processor -- start 2024-05-30T17:07:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.855+0000 7f88ad800640 1 -- start start 2024-05-30T17:07:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.856+0000 7f88ad800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a80d0b40 0x7f88a8150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.856+0000 7f88ad800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88a81600f0 0x7f88a809d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.856+0000 7f88ad800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88a809d6d0 0x7f88a809b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.856+0000 7f88ad800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f88a815fde0 con 0x7f88a80d0b40 2024-05-30T17:07:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.856+0000 7f88ad800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8071c70 con 0x7f88a809d6d0 2024-05-30T17:07:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.856+0000 7f88ad800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88a80726d0 con 0x7f88a81600f0 2024-05-30T17:07:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.856+0000 7f88a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a80d0b40 0x7f88a8150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.856+0000 7f88a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a80d0b40 0x7f88a8150a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40348/0 (socket says 172.21.15.67:40348) 2024-05-30T17:07:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.856+0000 7f88a7e00640 1 -- 172.21.15.67:0/1361880919 learned_addr learned my addr 172.21.15.67:0/1361880919 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88a7e00640 1 -- 172.21.15.67:0/1361880919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88a81600f0 msgr2=0x7f88a809d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88a7e00640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88a81600f0 0x7f88a809d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88a7e00640 1 -- 172.21.15.67:0/1361880919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88a809d6d0 msgr2=0x7f88a809b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88a7e00640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88a809d6d0 0x7f88a809b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88a7e00640 1 -- 172.21.15.67:0/1361880919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a809cef0 con 0x7f88a80d0b40 2024-05-30T17:07:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88a7e00640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a80d0b40 0x7f88a8150a90 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f889802ab60 tx=0x7f8898004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88a4c00640 1 -- 172.21.15.67:0/1361880919 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898036070 con 0x7f88a80d0b40 2024-05-30T17:07:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88a4c00640 1 -- 172.21.15.67:0/1361880919 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8898040070 con 0x7f88a80d0b40 2024-05-30T17:07:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88a4c00640 1 -- 172.21.15.67:0/1361880919 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898007c70 con 0x7f88a80d0b40 2024-05-30T17:07:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.857+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a809be50 con 0x7f88a80d0b40 2024-05-30T17:07:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.858+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a809c410 con 0x7f88a80d0b40 2024-05-30T17:07:51.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.858+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88a80689e0 con 0x7f88a80d0b40 2024-05-30T17:07:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.860+0000 7f88a4c00640 1 -- 172.21.15.67:0/1361880919 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f8898045460 con 0x7f88a80d0b40 2024-05-30T17:07:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.861+0000 7f88a4c00640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f888406e540 0x7f8884070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.861+0000 7f88a4c00640 1 -- 172.21.15.67:0/1361880919 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f88980abcd0 con 0x7f88a80d0b40 2024-05-30T17:07:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.861+0000 7f88a4c00640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7f88840769a0 0x7f8884078e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.861+0000 7f889f400640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f888406e540 0x7f8884070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.861+0000 7f88a4c00640 1 -- 172.21.15.67:0/1361880919 --> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8898031600 con 0x7f88840769a0 2024-05-30T17:07:51.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.861+0000 7f88a4c00640 1 -- 172.21.15.67:0/1361880919 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f88980ad100 con 0x7f88a80d0b40 2024-05-30T17:07:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.862+0000 7f889f400640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f888406e540 0x7f8884070a00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f88a815e960 tx=0x7f8890007970 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.862+0000 7f88ace00640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7f88840769a0 0x7f8884078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:51.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.863+0000 7f88ace00640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7f88840769a0 0x7f8884078e40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:51.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.865+0000 7f88a4c00640 1 -- 172.21.15.67:0/1361880919 <== osd.6 v2:172.21.15.169:6800/2783087947 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8898031600 con 0x7f88840769a0 2024-05-30T17:07:51.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.879+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 --> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f88a80cda60 con 0x7f88840769a0 2024-05-30T17:07:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.880+0000 7f88a4c00640 1 -- 172.21.15.67:0/1361880919 <== osd.6 v2:172.21.15.169:6800/2783087947 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f88a80cda60 con 0x7f88840769a0 2024-05-30T17:07:51.881 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:51.881 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:07:51.881 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:51.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7f88840769a0 msgr2=0x7f8884078e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.169:6800/2783087947,v1:172.21.15.169:6801/2783087947] conn(0x7f88840769a0 0x7f8884078e40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:51.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f888406e540 msgr2=0x7f8884070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f888406e540 0x7f8884070a00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f88a815e960 tx=0x7f8890007970 comp rx=0 tx=0).stop 2024-05-30T17:07:51.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a80d0b40 msgr2=0x7f88a8150a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:51.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 --2- 172.21.15.67:0/1361880919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88a80d0b40 0x7f88a8150a90 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f889802ab60 tx=0x7f8898004a40 comp rx=0 tx=0).stop 2024-05-30T17:07:51.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ace00640 1 -- 172.21.15.67:0/1361880919 reap_dead start 2024-05-30T17:07:51.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 shutdown_connections 2024-05-30T17:07:51.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 >> 172.21.15.67:0/1361880919 conn(0x7f88a806bae0 msgr2=0x7f88a80cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:51.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 shutdown_connections 2024-05-30T17:07:51.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:51.881+0000 7f88ad800640 1 -- 172.21.15.67:0/1361880919 wait complete. 2024-05-30T17:07:51.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-30T17:07:52.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.107+0000 7f21a7400640 1 Processor -- start 2024-05-30T17:07:52.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.107+0000 7f21a7400640 1 -- start start 2024-05-30T17:07:52.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.108+0000 7f21a7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21a006f860 0x7f21a00b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.108+0000 7f21a7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a00d0bb0 0x7f21a00ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.108+0000 7f21a7400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a006ed10 con 0x7f21a00b3690 2024-05-30T17:07:52.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.108+0000 7f21a7400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f21a015cb00 con 0x7f21a006f860 2024-05-30T17:07:52.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.108+0000 7f21a7400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21a00b3cc0 con 0x7f21a00d0bb0 2024-05-30T17:07:52.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.108+0000 7f21a4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21a006f860 0x7f21a00b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.109+0000 7f21a4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21a006f860 0x7f21a00b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40350/0 (socket says 172.21.15.67:40350) 2024-05-30T17:07:52.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.109+0000 7f21a4a00640 1 -- 172.21.15.67:0/1022992031 learned_addr learned my addr 172.21.15.67:0/1022992031 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:52.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.109+0000 7f21a5400640 1 --2- 172.21.15.67:0/1022992031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a00d0bb0 0x7f21a00ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.109+0000 7f21a4a00640 1 -- 172.21.15.67:0/1022992031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a00d0bb0 msgr2=0x7f21a00ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:52.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.109+0000 7f21a4a00640 1 --2- 172.21.15.67:0/1022992031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a00d0bb0 0x7f21a00ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.109+0000 7f21a4a00640 1 -- 172.21.15.67:0/1022992031 >> v1:172.21.15.67:6789/0 conn(0x7f21a00b3690 legacy=0x7f21a00d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.109+0000 7f21a4a00640 1 -- 172.21.15.67:0/1022992031 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a00b3fa0 con 0x7f21a006f860 2024-05-30T17:07:52.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.111+0000 7f21a4a00640 1 --2- 172.21.15.67:0/1022992031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a006f860 0x7f21a00b3150 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f2194009870 tx=0x7f219402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cffb6631c40b732 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.111+0000 7f219f400640 1 -- 172.21.15.67:0/1022992031 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194037070 con 0x7f21a006f860 2024-05-30T17:07:52.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.111+0000 7f219f400640 1 -- 172.21.15.67:0/1022992031 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2194032440 con 0x7f21a006f860 2024-05-30T17:07:52.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.111+0000 7f219f400640 1 -- 172.21.15.67:0/1022992031 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194032810 con 0x7f21a006f860 2024-05-30T17:07:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.112+0000 7f21a7400640 1 -- 172.21.15.67:0/1022992031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a006f860 msgr2=0x7f21a00b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.112+0000 7f21a7400640 1 --2- 172.21.15.67:0/1022992031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a006f860 0x7f21a00b3150 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f2194009870 tx=0x7f219402a270 comp rx=0 tx=0).stop 2024-05-30T17:07:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.112+0000 7f21a7400640 1 -- 172.21.15.67:0/1022992031 shutdown_connections 2024-05-30T17:07:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.112+0000 7f21a7400640 1 --2- 172.21.15.67:0/1022992031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a00d0bb0 0x7f21a00ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.112+0000 7f21a7400640 1 --2- 172.21.15.67:0/1022992031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a006f860 0x7f21a00b3150 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.112+0000 7f21a7400640 1 -- 172.21.15.67:0/1022992031 >> 172.21.15.67:0/1022992031 conn(0x7f21a006bae0 msgr2=0x7f21a006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:52.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.112+0000 7f21a7400640 1 -- 172.21.15.67:0/1022992031 shutdown_connections 2024-05-30T17:07:52.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.112+0000 7f21a7400640 1 -- 172.21.15.67:0/1022992031 wait complete. 2024-05-30T17:07:52.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.114+0000 7f21a7400640 1 Processor -- start 2024-05-30T17:07:52.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.114+0000 7f21a7400640 1 -- start start 2024-05-30T17:07:52.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.115+0000 7f21a7400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a006f860 0x7f21a0098e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.115+0000 7f21a7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a00b3690 0x7f21a0152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.115+0000 7f21a7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a00d0bb0 0x7f21a0153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.115+0000 7f21a7400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f21a00b4560 con 0x7f21a00b3690 2024-05-30T17:07:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.115+0000 7f21a7400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21a00b3aa0 con 0x7f21a006f860 2024-05-30T17:07:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.115+0000 7f21a7400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21a00b4840 con 0x7f21a00d0bb0 2024-05-30T17:07:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.115+0000 7f219fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a00b3690 0x7f21a0152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.115+0000 7f219fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a00b3690 0x7f21a0152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40354/0 (socket says 172.21.15.67:40354) 2024-05-30T17:07:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.115+0000 7f219fe00640 1 -- 172.21.15.67:0/2844471145 learned_addr learned my addr 172.21.15.67:0/2844471145 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:52.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f219fe00640 1 -- 172.21.15.67:0/2844471145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a00d0bb0 msgr2=0x7f21a0153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f219fe00640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a00d0bb0 0x7f21a0153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f219fe00640 1 -- 172.21.15.67:0/2844471145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a006f860 msgr2=0x7f21a0098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f219fe00640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a006f860 0x7f21a0098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f219fe00640 1 -- 172.21.15.67:0/2844471145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a01574f0 con 0x7f21a00b3690 2024-05-30T17:07:52.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f219fe00640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a00b3690 0x7f21a0152c60 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f218c00c400 tx=0x7f218c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f219cc00640 1 -- 172.21.15.67:0/2844471145 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0052d0 con 0x7f21a00b3690 2024-05-30T17:07:52.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f219cc00640 1 -- 172.21.15.67:0/2844471145 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218c005490 con 0x7f21a00b3690 2024-05-30T17:07:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f219cc00640 1 -- 172.21.15.67:0/2844471145 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c007260 con 0x7f21a00b3690 2024-05-30T17:07:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a0151ea0 con 0x7f21a00b3690 2024-05-30T17:07:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.116+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a0152460 con 0x7f21a00b3690 2024-05-30T17:07:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.118+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a00689e0 con 0x7f21a00b3690 2024-05-30T17:07:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.119+0000 7f219cc00640 1 -- 172.21.15.67:0/2844471145 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f218c007400 con 0x7f21a00b3690 2024-05-30T17:07:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.120+0000 7f219cc00640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f218006e510 0x7f21800709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.120+0000 7f219cc00640 1 -- 172.21.15.67:0/2844471145 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f218c08a000 con 0x7f21a00b3690 2024-05-30T17:07:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.120+0000 7f21a4a00640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f218006e510 0x7f21800709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.121+0000 7f21a4a00640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f218006e510 0x7f21800709d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2194034000 tx=0x7f219402c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.125+0000 7f219cc00640 1 -- 172.21.15.67:0/2844471145 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21a006ed10 con 0x7f21a00b3690 2024-05-30T17:07:52.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.330+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f21a006e2f0 con 0x7f218006e510 2024-05-30T17:07:52.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.331+0000 7f219cc00640 1 -- 172.21.15.67:0/2844471145 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f21a006e2f0 con 0x7f218006e510 2024-05-30T17:07:52.331 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 6 to deep-scrub 2024-05-30T17:07:52.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f218006e510 msgr2=0x7f21800709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f218006e510 0x7f21800709d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2194034000 tx=0x7f219402c000 comp rx=0 tx=0).stop 2024-05-30T17:07:52.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a00b3690 msgr2=0x7f21a0152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a00b3690 0x7f21a0152c60 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f218c00c400 tx=0x7f218c00c8c0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 shutdown_connections 2024-05-30T17:07:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f218006e510 0x7f21800709d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21a00d0bb0 0x7f21a0153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21a00b3690 0x7f21a0152c60 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 --2- 172.21.15.67:0/2844471145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a006f860 0x7f21a0098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.333+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 >> 172.21.15.67:0/2844471145 conn(0x7f21a006bae0 msgr2=0x7f21a0072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.334+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 shutdown_connections 2024-05-30T17:07:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.334+0000 7f21a7400640 1 -- 172.21.15.67:0/2844471145 wait complete. 2024-05-30T17:07:52.354 INFO:tasks.ceph:Scrubbing osd.7 2024-05-30T17:07:52.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-30T17:07:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.601+0000 7f448de00640 1 Processor -- start 2024-05-30T17:07:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.601+0000 7f448de00640 1 -- start start 2024-05-30T17:07:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.603+0000 7f448de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f448809aa30 0x7f44880ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.603+0000 7f448de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44880d1d90 0x7f448809ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.603+0000 7f448de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44880cfba0 con 0x7f4488160140 2024-05-30T17:07:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.603+0000 7f448de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f44880cf670 con 0x7f44880d1d90 2024-05-30T17:07:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.603+0000 7f448de00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f448806fc60 con 0x7f448809aa30 2024-05-30T17:07:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.603+0000 7f4487e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44880d1d90 0x7f448809ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.603+0000 7f4487e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44880d1d90 0x7f448809ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40362/0 (socket says 172.21.15.67:40362) 2024-05-30T17:07:52.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.603+0000 7f4487e00640 1 -- 172.21.15.67:0/1810409802 learned_addr learned my addr 172.21.15.67:0/1810409802 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.604+0000 7f448ca00640 1 --2- 172.21.15.67:0/1810409802 >> v2:172.21.15.169:3300/0 conn(0x7f448809aa30 0x7f44880ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.604+0000 7f4487e00640 1 -- 172.21.15.67:0/1810409802 >> v2:172.21.15.169:3300/0 conn(0x7f448809aa30 msgr2=0x7f44880ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:52.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.604+0000 7f4487e00640 1 --2- 172.21.15.67:0/1810409802 >> v2:172.21.15.169:3300/0 conn(0x7f448809aa30 0x7f44880ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.604+0000 7f4487e00640 1 -- 172.21.15.67:0/1810409802 >> v1:172.21.15.67:6789/0 conn(0x7f4488160140 legacy=0x7f44881605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.604+0000 7f4487e00640 1 -- 172.21.15.67:0/1810409802 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448809b350 con 0x7f44880d1d90 2024-05-30T17:07:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.604+0000 7f448ca00640 1 --2- 172.21.15.67:0/1810409802 >> v2:172.21.15.169:3300/0 conn(0x7f448809aa30 0x7f44880ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:52.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.605+0000 7f4487e00640 1 --2- 172.21.15.67:0/1810409802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 0x7f448809ae10 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4488073660 tx=0x7f447402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1a2caf846f17b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.605+0000 7f4487400640 1 -- 172.21.15.67:0/1810409802 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474037070 con 0x7f44880d1d90 2024-05-30T17:07:52.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.605+0000 7f4487400640 1 -- 172.21.15.67:0/1810409802 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4474032440 con 0x7f44880d1d90 2024-05-30T17:07:52.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.605+0000 7f4487400640 1 -- 172.21.15.67:0/1810409802 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474032810 con 0x7f44880d1d90 2024-05-30T17:07:52.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.606+0000 7f448de00640 1 -- 172.21.15.67:0/1810409802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 msgr2=0x7f448809ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.606+0000 7f448de00640 1 --2- 172.21.15.67:0/1810409802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 0x7f448809ae10 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4488073660 tx=0x7f447402a1e0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.606+0000 7f448de00640 1 -- 172.21.15.67:0/1810409802 shutdown_connections 2024-05-30T17:07:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.606+0000 7f448de00640 1 --2- 172.21.15.67:0/1810409802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 0x7f448809ae10 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.606+0000 7f448de00640 1 --2- 172.21.15.67:0/1810409802 >> v2:172.21.15.169:3300/0 conn(0x7f448809aa30 0x7f44880ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.606+0000 7f448de00640 1 -- 172.21.15.67:0/1810409802 >> 172.21.15.67:0/1810409802 conn(0x7f44880b3ec0 msgr2=0x7f44880b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.606+0000 7f448de00640 1 -- 172.21.15.67:0/1810409802 shutdown_connections 2024-05-30T17:07:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.606+0000 7f448de00640 1 -- 172.21.15.67:0/1810409802 wait complete. 2024-05-30T17:07:52.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.608+0000 7f448de00640 1 Processor -- start 2024-05-30T17:07:52.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.609+0000 7f448de00640 1 -- start start 2024-05-30T17:07:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f448de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448809aa30 0x7f448814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f448de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 0x7f448815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f448de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4488160140 0x7f44881f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f448de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f44880d1260 con 0x7f44880d1d90 2024-05-30T17:07:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f448de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44880cf8c0 con 0x7f448809aa30 2024-05-30T17:07:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f448de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44880700f0 con 0x7f4488160140 2024-05-30T17:07:52.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f4487e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 0x7f448815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f4487e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 0x7f448815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40378/0 (socket says 172.21.15.67:40378) 2024-05-30T17:07:52.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f4487e00640 1 -- 172.21.15.67:0/2007924406 learned_addr learned my addr 172.21.15.67:0/2007924406 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:52.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f448ca00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448809aa30 0x7f448814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f4487e00640 1 -- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4488160140 msgr2=0x7f44881f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f4487e00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4488160140 0x7f44881f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f4487e00640 1 -- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448809aa30 msgr2=0x7f448814cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.610+0000 7f4487e00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448809aa30 0x7f448814cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.611+0000 7f4487e00640 1 -- 172.21.15.67:0/2007924406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44881591e0 con 0x7f44880d1d90 2024-05-30T17:07:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.611+0000 7f4487e00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 0x7f448815b850 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f447402a220 tx=0x7f44740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.611+0000 7f4484c00640 1 -- 172.21.15.67:0/2007924406 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474037040 con 0x7f44880d1d90 2024-05-30T17:07:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.611+0000 7f4484c00640 1 -- 172.21.15.67:0/2007924406 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4474042070 con 0x7f44880d1d90 2024-05-30T17:07:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.611+0000 7f4484c00640 1 -- 172.21.15.67:0/2007924406 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474030bb0 con 0x7f44880d1d90 2024-05-30T17:07:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.611+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44881594c0 con 0x7f44880d1d90 2024-05-30T17:07:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.611+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448815bd90 con 0x7f44880d1d90 2024-05-30T17:07:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.612+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4488158f90 con 0x7f44880d1d90 2024-05-30T17:07:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.614+0000 7f4484c00640 1 -- 172.21.15.67:0/2007924406 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f4474047460 con 0x7f44880d1d90 2024-05-30T17:07:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.615+0000 7f4484c00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f446806e510 0x7f44680709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.615+0000 7f4484c00640 1 -- 172.21.15.67:0/2007924406 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f44740ac0a0 con 0x7f44880d1d90 2024-05-30T17:07:52.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.616+0000 7f448ca00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f446806e510 0x7f44680709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.616+0000 7f4484c00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f4468076970 0x7f4468078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.616+0000 7f4484c00640 1 -- 172.21.15.67:0/2007924406 --> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4474032440 con 0x7f4468076970 2024-05-30T17:07:52.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.616+0000 7f4484c00640 1 -- 172.21.15.67:0/2007924406 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f44740ad500 con 0x7f44880d1d90 2024-05-30T17:07:52.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.616+0000 7f448ca00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f446806e510 0x7f44680709d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b840 tx=0x7f447c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.617+0000 7f448d400640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f4468076970 0x7f4468078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.618+0000 7f448d400640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f4468076970 0x7f4468078e10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.620+0000 7f4484c00640 1 -- 172.21.15.67:0/2007924406 <== osd.7 v2:172.21.15.169:6808/4150327572 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4474032440 con 0x7f4468076970 2024-05-30T17:07:52.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.636+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 --> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f44880689e0 con 0x7f4468076970 2024-05-30T17:07:52.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f4484c00640 1 -- 172.21.15.67:0/2007924406 <== osd.7 v2:172.21.15.169:6808/4150327572 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f44880689e0 con 0x7f4468076970 2024-05-30T17:07:52.637 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-05-30T17:07:52.638 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-30T17:07:52.638 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-05-30T17:07:52.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f4468076970 msgr2=0x7f4468078e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f448de00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.169:6808/4150327572,v1:172.21.15.169:6809/4150327572] conn(0x7f4468076970 0x7f4468078e10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f446806e510 msgr2=0x7f44680709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f448de00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f446806e510 0x7f44680709d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b840 tx=0x7f447c008040 comp rx=0 tx=0).stop 2024-05-30T17:07:52.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 msgr2=0x7f448815b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f448de00640 1 --2- 172.21.15.67:0/2007924406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44880d1d90 0x7f448815b850 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f447402a220 tx=0x7f44740046c0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f448d400640 1 -- 172.21.15.67:0/2007924406 reap_dead start 2024-05-30T17:07:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 shutdown_connections 2024-05-30T17:07:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.637+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 >> 172.21.15.67:0/2007924406 conn(0x7f44880b3ec0 msgr2=0x7f44880b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.638+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 shutdown_connections 2024-05-30T17:07:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.638+0000 7f448de00640 1 -- 172.21.15.67:0/2007924406 wait complete. 2024-05-30T17:07:52.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-30T17:07:52.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.872+0000 7f2d9ce00640 1 Processor -- start 2024-05-30T17:07:52.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.872+0000 7f2d9ce00640 1 -- start start 2024-05-30T17:07:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.874+0000 7f2d9ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2d9815e870 0x7f2d980d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.874+0000 7f2d9ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2d980cfa40 0x7f2d980d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.874+0000 7f2d9ce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9815ec50 con 0x7f2d980d14d0 2024-05-30T17:07:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.874+0000 7f2d9ce00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f2d980d2150 con 0x7f2d980cfa40 2024-05-30T17:07:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.874+0000 7f2d9ce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2d980cff80 con 0x7f2d9815e870 2024-05-30T17:07:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.874+0000 7f2d97e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2d980cfa40 0x7f2d980d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.874+0000 7f2d97e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2d980cfa40 0x7f2d980d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40380/0 (socket says 172.21.15.67:40380) 2024-05-30T17:07:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.874+0000 7f2d97e00640 1 -- 172.21.15.67:0/302261073 learned_addr learned my addr 172.21.15.67:0/302261073 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:52.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.875+0000 7f2d97e00640 1 -- 172.21.15.67:0/302261073 >> v2:172.21.15.169:3300/0 conn(0x7f2d9815e870 msgr2=0x7f2d980d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:52.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.875+0000 7f2d97e00640 1 --2- 172.21.15.67:0/302261073 >> v2:172.21.15.169:3300/0 conn(0x7f2d9815e870 0x7f2d980d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.875+0000 7f2d97e00640 1 -- 172.21.15.67:0/302261073 >> v1:172.21.15.67:6789/0 conn(0x7f2d980d14d0 legacy=0x7f2d980cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.875+0000 7f2d97e00640 1 -- 172.21.15.67:0/302261073 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d980d0260 con 0x7f2d980cfa40 2024-05-30T17:07:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.876+0000 7f2d97e00640 1 --2- 172.21.15.67:0/302261073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980cfa40 0x7f2d980d25e0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f2d9808e720 tx=0x7f2d9002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3be245ba1a9d64 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.876+0000 7f2d96000640 1 -- 172.21.15.67:0/302261073 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90036070 con 0x7f2d980cfa40 2024-05-30T17:07:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.877+0000 7f2d96000640 1 -- 172.21.15.67:0/302261073 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d90031600 con 0x7f2d980cfa40 2024-05-30T17:07:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.877+0000 7f2d96000640 1 -- 172.21.15.67:0/302261073 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d900319d0 con 0x7f2d980cfa40 2024-05-30T17:07:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.877+0000 7f2d9ce00640 1 -- 172.21.15.67:0/302261073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980cfa40 msgr2=0x7f2d980d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.877+0000 7f2d9ce00640 1 --2- 172.21.15.67:0/302261073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980cfa40 0x7f2d980d25e0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f2d9808e720 tx=0x7f2d9002a730 comp rx=0 tx=0).stop 2024-05-30T17:07:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.878+0000 7f2d9ce00640 1 -- 172.21.15.67:0/302261073 shutdown_connections 2024-05-30T17:07:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.878+0000 7f2d9ce00640 1 --2- 172.21.15.67:0/302261073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980cfa40 0x7f2d980d25e0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.878+0000 7f2d9ce00640 1 --2- 172.21.15.67:0/302261073 >> v2:172.21.15.169:3300/0 conn(0x7f2d9815e870 0x7f2d980d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.878+0000 7f2d9ce00640 1 -- 172.21.15.67:0/302261073 >> 172.21.15.67:0/302261073 conn(0x7f2d9808bfc0 msgr2=0x7f2d980d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.878+0000 7f2d9ce00640 1 -- 172.21.15.67:0/302261073 shutdown_connections 2024-05-30T17:07:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.878+0000 7f2d9ce00640 1 -- 172.21.15.67:0/302261073 wait complete. 2024-05-30T17:07:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.880+0000 7f2d9ce00640 1 Processor -- start 2024-05-30T17:07:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.880+0000 7f2d9ce00640 1 -- start start 2024-05-30T17:07:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.881+0000 7f2d9ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d980cfa40 0x7f2d98159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.881+0000 7f2d9ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980d14d0 0x7f2d980c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.881+0000 7f2d9ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d9815e870 0x7f2d980b9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.881+0000 7f2d9ce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2d980d4fb0 con 0x7f2d980d14d0 2024-05-30T17:07:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.881+0000 7f2d9ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2d980d0820 con 0x7f2d9815e870 2024-05-30T17:07:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.881+0000 7f2d9ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d980d2b20 con 0x7f2d980cfa40 2024-05-30T17:07:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.881+0000 7f2d96a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980d14d0 0x7f2d980c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d96a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980d14d0 0x7f2d980c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40386/0 (socket says 172.21.15.67:40386) 2024-05-30T17:07:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d96a00640 1 -- 172.21.15.67:0/3593072685 learned_addr learned my addr 172.21.15.67:0/3593072685 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d97e00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d9815e870 0x7f2d980b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d97400640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d980cfa40 0x7f2d98159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d96a00640 1 -- 172.21.15.67:0/3593072685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d980cfa40 msgr2=0x7f2d98159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d96a00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d980cfa40 0x7f2d98159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d96a00640 1 -- 172.21.15.67:0/3593072685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d9815e870 msgr2=0x7f2d980b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d96a00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d9815e870 0x7f2d980b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d96a00640 1 -- 172.21.15.67:0/3593072685 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d980c8dc0 con 0x7f2d980d14d0 2024-05-30T17:07:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.882+0000 7f2d97400640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d980cfa40 0x7f2d98159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.883+0000 7f2d96a00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980d14d0 0x7f2d980c8670 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400c430 tx=0x7f2d8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.883+0000 7f2d8f400640 1 -- 172.21.15.67:0/3593072685 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84017070 con 0x7f2d980d14d0 2024-05-30T17:07:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.883+0000 7f2d8f400640 1 -- 172.21.15.67:0/3593072685 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d840035f0 con 0x7f2d980d14d0 2024-05-30T17:07:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.883+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d980c90a0 con 0x7f2d980d14d0 2024-05-30T17:07:52.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.883+0000 7f2d8f400640 1 -- 172.21.15.67:0/3593072685 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84003a20 con 0x7f2d980d14d0 2024-05-30T17:07:52.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.883+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d980ca160 con 0x7f2d980d14d0 2024-05-30T17:07:52.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.884+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d980689e0 con 0x7f2d980d14d0 2024-05-30T17:07:52.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.886+0000 7f2d8f400640 1 -- 172.21.15.67:0/3593072685 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f2d8401a070 con 0x7f2d980d14d0 2024-05-30T17:07:52.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.887+0000 7f2d8f400640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d7806e540 0x7f2d78070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:52.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.887+0000 7f2d8f400640 1 -- 172.21.15.67:0/3593072685 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f2d84089e60 con 0x7f2d980d14d0 2024-05-30T17:07:52.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.887+0000 7f2d97400640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d7806e540 0x7f2d78070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:52.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.888+0000 7f2d97400640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d7806e540 0x7f2d78070a00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800a870 tx=0x7f2d88008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:52.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:52.892+0000 7f2d8f400640 1 -- 172.21.15.67:0/3593072685 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d84059cf0 con 0x7f2d980d14d0 2024-05-30T17:07:53.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.119+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f2d980ca420 con 0x7f2d7806e540 2024-05-30T17:07:53.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.120+0000 7f2d8f400640 1 -- 172.21.15.67:0/3593072685 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2d980ca420 con 0x7f2d7806e540 2024-05-30T17:07:53.120 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 7 to deep-scrub 2024-05-30T17:07:53.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d7806e540 msgr2=0x7f2d78070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:53.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d7806e540 0x7f2d78070a00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800a870 tx=0x7f2d88008040 comp rx=0 tx=0).stop 2024-05-30T17:07:53.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980d14d0 msgr2=0x7f2d980c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:53.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980d14d0 0x7f2d980c8670 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400c430 tx=0x7f2d8400c8f0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 shutdown_connections 2024-05-30T17:07:53.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f2d7806e540 0x7f2d78070a00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d9815e870 0x7f2d980b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d980d14d0 0x7f2d980c8670 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 --2- 172.21.15.67:0/3593072685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d980cfa40 0x7f2d98159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 >> 172.21.15.67:0/3593072685 conn(0x7f2d9808bfc0 msgr2=0x7f2d980b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:53.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 shutdown_connections 2024-05-30T17:07:53.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.123+0000 7f2d9ce00640 1 -- 172.21.15.67:0/3593072685 wait complete. 2024-05-30T17:07:53.143 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:07:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.343+0000 7fdfd7e00640 1 Processor -- start 2024-05-30T17:07:53.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.344+0000 7fdfd7e00640 1 -- start start 2024-05-30T17:07:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.345+0000 7fdfd7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdfd8160380 0x7fdfd80b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.345+0000 7fdfd7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 0x7fdfd80b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.345+0000 7fdfd7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd80d3300 con 0x7fdfd815fae0 2024-05-30T17:07:53.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.345+0000 7fdfd7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd80ced60 con 0x7fdfd8160380 2024-05-30T17:07:53.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.345+0000 7fdfd7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd80d35e0 con 0x7fdfd80b5430 2024-05-30T17:07:53.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.346+0000 7fdfd6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdfd8160380 0x7fdfd80b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:53.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.346+0000 7fdfd6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdfd8160380 0x7fdfd80b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40390/0 (socket says 172.21.15.67:40390) 2024-05-30T17:07:53.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.346+0000 7fdfd6000640 1 -- 172.21.15.67:0/1414605043 learned_addr learned my addr 172.21.15.67:0/1414605043 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:53.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.346+0000 7fdfd7400640 1 --2- 172.21.15.67:0/1414605043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 0x7fdfd80b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:53.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.346+0000 7fdfd6000640 1 -- 172.21.15.67:0/1414605043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 msgr2=0x7fdfd80b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:07:53.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.346+0000 7fdfd6000640 1 --2- 172.21.15.67:0/1414605043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 0x7fdfd80b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.346+0000 7fdfd6000640 1 -- 172.21.15.67:0/1414605043 >> v1:172.21.15.67:6789/0 conn(0x7fdfd815fae0 legacy=0x7fdfd80d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:53.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.346+0000 7fdfd6000640 1 -- 172.21.15.67:0/1414605043 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd80d38c0 con 0x7fdfd8160380 2024-05-30T17:07:53.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.346+0000 7fdfd7400640 1 --2- 172.21.15.67:0/1414605043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 0x7fdfd80b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:53.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.348+0000 7fdfd6000640 1 --2- 172.21.15.67:0/1414605043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd8160380 0x7fdfd80b5e60 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009850 tx=0x7fdfc402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c5b07bca76ec67 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:53.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.348+0000 7fdfd5600640 1 -- 172.21.15.67:0/1414605043 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4004030 con 0x7fdfd8160380 2024-05-30T17:07:53.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.348+0000 7fdfd5600640 1 -- 172.21.15.67:0/1414605043 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc40041d0 con 0x7fdfd8160380 2024-05-30T17:07:53.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.348+0000 7fdfd5600640 1 -- 172.21.15.67:0/1414605043 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4003610 con 0x7fdfd8160380 2024-05-30T17:07:53.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.349+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1414605043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd8160380 msgr2=0x7fdfd80b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:53.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.349+0000 7fdfd7e00640 1 --2- 172.21.15.67:0/1414605043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd8160380 0x7fdfd80b5e60 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009850 tx=0x7fdfc402a230 comp rx=0 tx=0).stop 2024-05-30T17:07:53.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.349+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1414605043 shutdown_connections 2024-05-30T17:07:53.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.349+0000 7fdfd7e00640 1 --2- 172.21.15.67:0/1414605043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 0x7fdfd80b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.349+0000 7fdfd7e00640 1 --2- 172.21.15.67:0/1414605043 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd8160380 0x7fdfd80b5e60 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.349+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1414605043 >> 172.21.15.67:0/1414605043 conn(0x7fdfd808bfc0 msgr2=0x7fdfd80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:53.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.349+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1414605043 shutdown_connections 2024-05-30T17:07:53.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.349+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1414605043 wait complete. 2024-05-30T17:07:53.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.351+0000 7fdfd7e00640 1 Processor -- start 2024-05-30T17:07:53.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.351+0000 7fdfd7e00640 1 -- start start 2024-05-30T17:07:53.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 0x7fdfd814f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:53.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd815fae0 0x7fdfd80cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:53.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfd8160380 0x7fdfd80cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:53.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd80cf370 con 0x7fdfd815fae0 2024-05-30T17:07:53.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd80cf1f0 con 0x7fdfd8160380 2024-05-30T17:07:53.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd80cf4f0 con 0x7fdfd80b5430 2024-05-30T17:07:53.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd815fae0 0x7fdfd80cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:53.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd815fae0 0x7fdfd80cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40400/0 (socket says 172.21.15.67:40400) 2024-05-30T17:07:53.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd6000640 1 -- 172.21.15.67:0/1682603248 learned_addr learned my addr 172.21.15.67:0/1682603248 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:07:53.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd6a00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 0x7fdfd814f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:53.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.352+0000 7fdfd7400640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfd8160380 0x7fdfd80cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:53.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.353+0000 7fdfd6000640 1 -- 172.21.15.67:0/1682603248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 msgr2=0x7fdfd814f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:53.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.353+0000 7fdfd6000640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 0x7fdfd814f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.353+0000 7fdfd6000640 1 -- 172.21.15.67:0/1682603248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfd8160380 msgr2=0x7fdfd80cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:53.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.353+0000 7fdfd6000640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfd8160380 0x7fdfd80cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.353+0000 7fdfd6000640 1 -- 172.21.15.67:0/1682603248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd80c8e90 con 0x7fdfd815fae0 2024-05-30T17:07:53.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.353+0000 7fdfd7400640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfd8160380 0x7fdfd80cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:07:53.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.353+0000 7fdfd6000640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd815fae0 0x7fdfd80cd970 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4034040 tx=0x7fdfc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:53.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.353+0000 7fdfcea00640 1 -- 172.21.15.67:0/1682603248 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4041030 con 0x7fdfd815fae0 2024-05-30T17:07:53.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.354+0000 7fdfcea00640 1 -- 172.21.15.67:0/1682603248 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc4003c50 con 0x7fdfd815fae0 2024-05-30T17:07:53.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.354+0000 7fdfcea00640 1 -- 172.21.15.67:0/1682603248 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4004030 con 0x7fdfd815fae0 2024-05-30T17:07:53.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.354+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd80c9170 con 0x7fdfd815fae0 2024-05-30T17:07:53.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.354+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd80b7cc0 con 0x7fdfd815fae0 2024-05-30T17:07:53.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.355+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd80cb5d0 con 0x7fdfd815fae0 2024-05-30T17:07:53.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.356+0000 7fdfcea00640 1 -- 172.21.15.67:0/1682603248 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7fdfc4044070 con 0x7fdfd815fae0 2024-05-30T17:07:53.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.357+0000 7fdfcea00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfb806e510 0x7fdfb80709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:07:53.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.357+0000 7fdfcea00640 1 -- 172.21.15.67:0/1682603248 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fdfc40ab750 con 0x7fdfd815fae0 2024-05-30T17:07:53.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.358+0000 7fdfd6a00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfb806e510 0x7fdfb80709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:07:53.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.358+0000 7fdfd6a00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfb806e510 0x7fdfb80709d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80012e0 tx=0x7fdfc8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:07:53.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.362+0000 7fdfcea00640 1 -- 172.21.15.67:0/1682603248 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdfc403b3b0 con 0x7fdfd815fae0 2024-05-30T17:07:53.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.634+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfd80ba480 con 0x7fdfb806e510 2024-05-30T17:07:53.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.643+0000 7fdfcea00640 1 -- 172.21.15.67:0/1682603248 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253285 (secure 0 0 0) 0x7fdfd80ba480 con 0x7fdfb806e510 2024-05-30T17:07:53.647 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:07:53.648 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:07:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfb806e510 msgr2=0x7fdfb80709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfb806e510 0x7fdfb80709d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80012e0 tx=0x7fdfc8009040 comp rx=0 tx=0).stop 2024-05-30T17:07:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd815fae0 msgr2=0x7fdfd80cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:07:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd815fae0 0x7fdfd80cd970 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4034040 tx=0x7fdfc40046c0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 shutdown_connections 2024-05-30T17:07:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fdfb806e510 0x7fdfb80709d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfd8160380 0x7fdfd80cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdfd815fae0 0x7fdfd80cd970 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 --2- 172.21.15.67:0/1682603248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfd80b5430 0x7fdfd814f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:07:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.648+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 >> 172.21.15.67:0/1682603248 conn(0x7fdfd808bfc0 msgr2=0x7fdfd806cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:07:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.649+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 shutdown_connections 2024-05-30T17:07:53.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:07:53.649+0000 7fdfd7e00640 1 -- 172.21.15.67:0/1682603248 wait complete. 2024-05-30T17:07:53.671 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":854,"stamp":"2024-05-30T17:07:52.679564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20915095739,"num_objects":5021,"num_object_clones":1,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":1,"num_read":4440,"num_read_kb":17367224,"num_write":5867,"num_write_kb":20427049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9274,"ondisk_log_size":9274,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":42021136,"kb_used_data":40862928,"kb_used_omap":12,"kb_used_meta":1158131,"kb_avail":712953584,"statfs":{"total":773094113280,"available":730064470016,"internally_reserved":0,"allocated":41843638272,"data_stored":41836077646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1185926704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004483"},"pg_stats":[{"pgid":"19.3f","version":"134'127","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.876279+0000","last_change":"2024-05-30T17:04:52.998366+0000","last_active":"2024-05-30T17:07:42.876279+0000","last_peered":"2024-05-30T17:07:42.876279+0000","last_clean":"2024-05-30T17:07:42.876279+0000","last_became_active":"2024-05-30T17:04:52.997618+0000","last_became_peered":"2024-05-30T17:04:52.997618+0000","last_unstale":"2024-05-30T17:07:42.876279+0000","last_undegraded":"2024-05-30T17:07:42.876279+0000","last_fullsized":"2024-05-30T17:07:42.876279+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":71,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'133","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.875159+0000","last_change":"2024-05-30T17:04:52.984295+0000","last_active":"2024-05-30T17:07:42.875159+0000","last_peered":"2024-05-30T17:07:42.875159+0000","last_clean":"2024-05-30T17:07:42.875159+0000","last_became_active":"2024-05-30T17:04:52.982524+0000","last_became_peered":"2024-05-30T17:04:52.982524+0000","last_unstale":"2024-05-30T17:07:42.875159+0000","last_undegraded":"2024-05-30T17:07:42.875159+0000","last_fullsized":"2024-05-30T17:07:42.875159+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'149","reported_seq":241,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859124+0000","last_change":"2024-05-30T17:04:52.984783+0000","last_active":"2024-05-30T17:07:42.859124+0000","last_peered":"2024-05-30T17:07:42.859124+0000","last_clean":"2024-05-30T17:07:42.859124+0000","last_became_active":"2024-05-30T17:04:52.984493+0000","last_became_peered":"2024-05-30T17:04:52.984493+0000","last_unstale":"2024-05-30T17:07:42.859124+0000","last_undegraded":"2024-05-30T17:07:42.859124+0000","last_fullsized":"2024-05-30T17:07:42.859124+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:59:21.682021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904709,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":75,"num_read_kb":286725,"num_write":94,"num_write_kb":356356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'127","reported_seq":186,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.856841+0000","last_change":"2024-05-30T17:05:34.963309+0000","last_active":"2024-05-30T17:07:42.856841+0000","last_peered":"2024-05-30T17:07:42.856841+0000","last_clean":"2024-05-30T17:07:42.856841+0000","last_became_active":"2024-05-30T17:05:34.859953+0000","last_became_peered":"2024-05-30T17:05:34.859953+0000","last_unstale":"2024-05-30T17:07:42.856841+0000","last_undegraded":"2024-05-30T17:07:42.856841+0000","last_fullsized":"2024-05-30T17:07:42.856841+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":70,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3b","version":"134'111","reported_seq":173,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.836580+0000","last_change":"2024-05-30T17:04:52.990350+0000","last_active":"2024-05-30T17:07:42.836580+0000","last_peered":"2024-05-30T17:07:42.836580+0000","last_clean":"2024-05-30T17:07:42.836580+0000","last_became_active":"2024-05-30T17:04:52.990018+0000","last_became_peered":"2024-05-30T17:04:52.990018+0000","last_unstale":"2024-05-30T17:07:42.836580+0000","last_undegraded":"2024-05-30T17:07:42.836580+0000","last_fullsized":"2024-05-30T17:07:42.836580+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:10:21.257549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":61,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'143","reported_seq":247,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.872885+0000","last_change":"2024-05-30T17:04:52.992047+0000","last_active":"2024-05-30T17:07:42.872885+0000","last_peered":"2024-05-30T17:07:42.872885+0000","last_clean":"2024-05-30T17:07:42.872885+0000","last_became_active":"2024-05-30T17:04:52.991748+0000","last_became_peered":"2024-05-30T17:04:52.991748+0000","last_unstale":"2024-05-30T17:07:42.872885+0000","last_undegraded":"2024-05-30T17:07:42.872885+0000","last_fullsized":"2024-05-30T17:07:42.872885+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:15:06.346572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":84,"num_write_kb":335873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'140","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.869195+0000","last_change":"2024-05-30T17:04:52.984245+0000","last_active":"2024-05-30T17:07:42.869195+0000","last_peered":"2024-05-30T17:07:42.869195+0000","last_clean":"2024-05-30T17:07:42.869195+0000","last_became_active":"2024-05-30T17:04:52.983960+0000","last_became_peered":"2024-05-30T17:04:52.983960+0000","last_unstale":"2024-05-30T17:07:42.869195+0000","last_undegraded":"2024-05-30T17:07:42.869195+0000","last_fullsized":"2024-05-30T17:07:42.869195+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:40:33.491032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":51,"num_read_kb":200706,"num_write":76,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'126","reported_seq":200,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.861586+0000","last_change":"2024-05-30T17:04:52.988608+0000","last_active":"2024-05-30T17:07:42.861586+0000","last_peered":"2024-05-30T17:07:42.861586+0000","last_clean":"2024-05-30T17:07:42.861586+0000","last_became_active":"2024-05-30T17:04:52.988277+0000","last_became_peered":"2024-05-30T17:04:52.988277+0000","last_unstale":"2024-05-30T17:07:42.861586+0000","last_undegraded":"2024-05-30T17:07:42.861586+0000","last_fullsized":"2024-05-30T17:07:42.861586+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:34.512494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":85,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'143","reported_seq":219,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.851819+0000","last_change":"2024-05-30T17:04:52.993014+0000","last_active":"2024-05-30T17:07:42.851819+0000","last_peered":"2024-05-30T17:07:42.851819+0000","last_clean":"2024-05-30T17:07:42.851819+0000","last_became_active":"2024-05-30T17:04:52.990543+0000","last_became_peered":"2024-05-30T17:04:52.990543+0000","last_unstale":"2024-05-30T17:07:42.851819+0000","last_undegraded":"2024-05-30T17:07:42.851819+0000","last_fullsized":"2024-05-30T17:07:42.851819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:27:15.135800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":59,"num_read_kb":233474,"num_write":78,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'133","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.872691+0000","last_change":"2024-05-30T17:04:52.979749+0000","last_active":"2024-05-30T17:07:42.872691+0000","last_peered":"2024-05-30T17:07:42.872691+0000","last_clean":"2024-05-30T17:07:42.872691+0000","last_became_active":"2024-05-30T17:04:52.979390+0000","last_became_peered":"2024-05-30T17:04:52.979390+0000","last_unstale":"2024-05-30T17:07:42.872691+0000","last_undegraded":"2024-05-30T17:07:42.872691+0000","last_fullsized":"2024-05-30T17:07:42.872691+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":65,"num_read_kb":266240,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'131","reported_seq":255,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.871438+0000","last_change":"2024-05-30T17:04:52.983598+0000","last_active":"2024-05-30T17:07:42.871438+0000","last_peered":"2024-05-30T17:07:42.871438+0000","last_clean":"2024-05-30T17:07:42.871438+0000","last_became_active":"2024-05-30T17:04:52.983250+0000","last_became_peered":"2024-05-30T17:04:52.983250+0000","last_unstale":"2024-05-30T17:07:42.871438+0000","last_undegraded":"2024-05-30T17:07:42.871438+0000","last_fullsized":"2024-05-30T17:07:42.871438+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":106,"num_read_kb":434176,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'118","reported_seq":240,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.838961+0000","last_change":"2024-05-30T17:04:52.993606+0000","last_active":"2024-05-30T17:07:42.838961+0000","last_peered":"2024-05-30T17:07:42.838961+0000","last_clean":"2024-05-30T17:07:42.838961+0000","last_became_active":"2024-05-30T17:04:52.993151+0000","last_became_peered":"2024-05-30T17:04:52.993151+0000","last_unstale":"2024-05-30T17:07:42.838961+0000","last_undegraded":"2024-05-30T17:07:42.838961+0000","last_fullsized":"2024-05-30T17:07:42.838961+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:14.893435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":73,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'129","reported_seq":231,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.876384+0000","last_change":"2024-05-30T17:04:52.998102+0000","last_active":"2024-05-30T17:07:42.876384+0000","last_peered":"2024-05-30T17:07:42.876384+0000","last_clean":"2024-05-30T17:07:42.876384+0000","last_became_active":"2024-05-30T17:04:52.997770+0000","last_became_peered":"2024-05-30T17:04:52.997770+0000","last_unstale":"2024-05-30T17:07:42.876384+0000","last_undegraded":"2024-05-30T17:07:42.876384+0000","last_fullsized":"2024-05-30T17:07:42.876384+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":84,"num_read_kb":344064,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'135","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.871736+0000","last_change":"2024-05-30T17:04:52.992617+0000","last_active":"2024-05-30T17:07:42.871736+0000","last_peered":"2024-05-30T17:07:42.871736+0000","last_clean":"2024-05-30T17:07:42.871736+0000","last_became_active":"2024-05-30T17:04:52.992221+0000","last_became_peered":"2024-05-30T17:04:52.992221+0000","last_unstale":"2024-05-30T17:07:42.871736+0000","last_undegraded":"2024-05-30T17:07:42.871736+0000","last_fullsized":"2024-05-30T17:07:42.871736+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127364,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":84,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'126","reported_seq":176,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.873157+0000","last_change":"2024-05-30T17:04:52.992819+0000","last_active":"2024-05-30T17:07:42.873157+0000","last_peered":"2024-05-30T17:07:42.873157+0000","last_clean":"2024-05-30T17:07:42.873157+0000","last_became_active":"2024-05-30T17:04:52.991693+0000","last_became_peered":"2024-05-30T17:04:52.991693+0000","last_unstale":"2024-05-30T17:07:42.873157+0000","last_undegraded":"2024-05-30T17:07:42.873157+0000","last_fullsized":"2024-05-30T17:07:42.873157+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":32,"num_read_kb":131072,"num_write":72,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'136","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.867754+0000","last_change":"2024-05-30T17:04:52.991328+0000","last_active":"2024-05-30T17:07:42.867754+0000","last_peered":"2024-05-30T17:07:42.867754+0000","last_clean":"2024-05-30T17:07:42.867754+0000","last_became_active":"2024-05-30T17:04:52.990334+0000","last_became_peered":"2024-05-30T17:04:52.990334+0000","last_unstale":"2024-05-30T17:07:42.867754+0000","last_undegraded":"2024-05-30T17:07:42.867754+0000","last_fullsized":"2024-05-30T17:07:42.867754+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'120","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.858857+0000","last_change":"2024-05-30T17:04:52.967087+0000","last_active":"2024-05-30T17:07:42.858857+0000","last_peered":"2024-05-30T17:07:42.858857+0000","last_clean":"2024-05-30T17:07:42.858857+0000","last_became_active":"2024-05-30T17:04:52.966026+0000","last_became_peered":"2024-05-30T17:04:52.966026+0000","last_unstale":"2024-05-30T17:07:42.858857+0000","last_undegraded":"2024-05-30T17:07:42.858857+0000","last_fullsized":"2024-05-30T17:07:42.858857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:12.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'115","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.876251+0000","last_change":"2024-05-30T17:04:52.986386+0000","last_active":"2024-05-30T17:07:42.876251+0000","last_peered":"2024-05-30T17:07:42.876251+0000","last_clean":"2024-05-30T17:07:42.876251+0000","last_became_active":"2024-05-30T17:04:52.985602+0000","last_became_peered":"2024-05-30T17:04:52.985602+0000","last_unstale":"2024-05-30T17:07:42.876251+0000","last_undegraded":"2024-05-30T17:07:42.876251+0000","last_fullsized":"2024-05-30T17:07:42.876251+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":91,"num_read_kb":372736,"num_write":71,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'135","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.861231+0000","last_change":"2024-05-30T17:04:52.963412+0000","last_active":"2024-05-30T17:07:42.861231+0000","last_peered":"2024-05-30T17:07:42.861231+0000","last_clean":"2024-05-30T17:07:42.861231+0000","last_became_active":"2024-05-30T17:04:52.962582+0000","last_became_peered":"2024-05-30T17:04:52.962582+0000","last_unstale":"2024-05-30T17:07:42.861231+0000","last_undegraded":"2024-05-30T17:07:42.861231+0000","last_fullsized":"2024-05-30T17:07:42.861231+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369098752,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":90,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'151","reported_seq":231,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.872323+0000","last_change":"2024-05-30T17:04:52.987028+0000","last_active":"2024-05-30T17:07:42.872323+0000","last_peered":"2024-05-30T17:07:42.872323+0000","last_clean":"2024-05-30T17:07:42.872323+0000","last_became_active":"2024-05-30T17:04:52.986701+0000","last_became_peered":"2024-05-30T17:04:52.986701+0000","last_unstale":"2024-05-30T17:07:42.872323+0000","last_undegraded":"2024-05-30T17:07:42.872323+0000","last_fullsized":"2024-05-30T17:07:42.872323+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352321536,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":62,"num_read_kb":249857,"num_write":86,"num_write_kb":344065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'152","reported_seq":265,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.853599+0000","last_change":"2024-05-30T17:04:52.984763+0000","last_active":"2024-05-30T17:07:42.853599+0000","last_peered":"2024-05-30T17:07:42.853599+0000","last_clean":"2024-05-30T17:07:42.853599+0000","last_became_active":"2024-05-30T17:04:52.984462+0000","last_became_peered":"2024-05-30T17:04:52.984462+0000","last_unstale":"2024-05-30T17:07:42.853599+0000","last_undegraded":"2024-05-30T17:07:42.853599+0000","last_fullsized":"2024-05-30T17:07:42.853599+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:09.221732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293056,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":96,"num_read_kb":372741,"num_write":95,"num_write_kb":364547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'132","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.870435+0000","last_change":"2024-05-30T17:04:52.980574+0000","last_active":"2024-05-30T17:07:42.870435+0000","last_peered":"2024-05-30T17:07:42.870435+0000","last_clean":"2024-05-30T17:07:42.870435+0000","last_became_active":"2024-05-30T17:04:52.978968+0000","last_became_peered":"2024-05-30T17:04:52.978968+0000","last_unstale":"2024-05-30T17:07:42.870435+0000","last_undegraded":"2024-05-30T17:07:42.870435+0000","last_fullsized":"2024-05-30T17:07:42.870435+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":61,"num_read_kb":245761,"num_write":82,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'142","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860783+0000","last_change":"2024-05-30T17:04:52.992275+0000","last_active":"2024-05-30T17:07:42.860783+0000","last_peered":"2024-05-30T17:07:42.860783+0000","last_clean":"2024-05-30T17:07:42.860783+0000","last_became_active":"2024-05-30T17:04:52.991966+0000","last_became_peered":"2024-05-30T17:04:52.991966+0000","last_unstale":"2024-05-30T17:07:42.860783+0000","last_undegraded":"2024-05-30T17:07:42.860783+0000","last_fullsized":"2024-05-30T17:07:42.860783+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:36:32.227484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":65,"num_read_kb":253955,"num_write":87,"num_write_kb":348161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'136","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860014+0000","last_change":"2024-05-30T17:04:52.965882+0000","last_active":"2024-05-30T17:07:42.860014+0000","last_peered":"2024-05-30T17:07:42.860014+0000","last_clean":"2024-05-30T17:07:42.860014+0000","last_became_active":"2024-05-30T17:04:52.965116+0000","last_became_peered":"2024-05-30T17:04:52.965116+0000","last_unstale":"2024-05-30T17:07:42.860014+0000","last_undegraded":"2024-05-30T17:07:42.860014+0000","last_fullsized":"2024-05-30T17:07:42.860014+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":57,"num_read_kb":229377,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'139","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.870813+0000","last_change":"2024-05-30T17:04:52.983400+0000","last_active":"2024-05-30T17:07:42.870813+0000","last_peered":"2024-05-30T17:07:42.870813+0000","last_clean":"2024-05-30T17:07:42.870813+0000","last_became_active":"2024-05-30T17:04:52.981867+0000","last_became_peered":"2024-05-30T17:04:52.981867+0000","last_unstale":"2024-05-30T17:07:42.870813+0000","last_undegraded":"2024-05-30T17:07:42.870813+0000","last_fullsized":"2024-05-30T17:07:42.870813+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":66,"num_read_kb":266241,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'141","reported_seq":229,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.876921+0000","last_change":"2024-05-30T17:04:52.989413+0000","last_active":"2024-05-30T17:07:42.876921+0000","last_peered":"2024-05-30T17:07:42.876921+0000","last_clean":"2024-05-30T17:07:42.876921+0000","last_became_active":"2024-05-30T17:04:52.989141+0000","last_became_peered":"2024-05-30T17:04:52.989141+0000","last_unstale":"2024-05-30T17:07:42.876921+0000","last_undegraded":"2024-05-30T17:07:42.876921+0000","last_fullsized":"2024-05-30T17:07:42.876921+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'134","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.875550+0000","last_change":"2024-05-30T17:04:52.992667+0000","last_active":"2024-05-30T17:07:42.875550+0000","last_peered":"2024-05-30T17:07:42.875550+0000","last_clean":"2024-05-30T17:07:42.875550+0000","last_became_active":"2024-05-30T17:04:52.988618+0000","last_became_peered":"2024-05-30T17:04:52.988618+0000","last_unstale":"2024-05-30T17:07:42.875550+0000","last_undegraded":"2024-05-30T17:07:42.875550+0000","last_fullsized":"2024-05-30T17:07:42.875550+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377487360,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":90,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'115","reported_seq":165,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.863460+0000","last_change":"2024-05-30T17:04:52.977678+0000","last_active":"2024-05-30T17:07:42.863460+0000","last_peered":"2024-05-30T17:07:42.863460+0000","last_clean":"2024-05-30T17:07:42.863460+0000","last_became_active":"2024-05-30T17:04:52.977378+0000","last_became_peered":"2024-05-30T17:04:52.977378+0000","last_unstale":"2024-05-30T17:07:42.863460+0000","last_undegraded":"2024-05-30T17:07:42.863460+0000","last_fullsized":"2024-05-30T17:07:42.863460+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:20.220555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'147","reported_seq":243,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.872620+0000","last_change":"2024-05-30T17:04:52.998322+0000","last_active":"2024-05-30T17:07:42.872620+0000","last_peered":"2024-05-30T17:07:42.872620+0000","last_clean":"2024-05-30T17:07:42.872620+0000","last_became_active":"2024-05-30T17:04:52.997323+0000","last_became_peered":"2024-05-30T17:04:52.997323+0000","last_unstale":"2024-05-30T17:07:42.872620+0000","last_undegraded":"2024-05-30T17:07:42.872620+0000","last_fullsized":"2024-05-30T17:07:42.872620+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390070272,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":78,"num_read_kb":307203,"num_write":95,"num_write_kb":380929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'121","reported_seq":203,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.874600+0000","last_change":"2024-05-30T17:04:52.995308+0000","last_active":"2024-05-30T17:07:42.874600+0000","last_peered":"2024-05-30T17:07:42.874600+0000","last_clean":"2024-05-30T17:07:42.874600+0000","last_became_active":"2024-05-30T17:04:52.992733+0000","last_became_peered":"2024-05-30T17:04:52.992733+0000","last_unstale":"2024-05-30T17:07:42.874600+0000","last_undegraded":"2024-05-30T17:07:42.874600+0000","last_fullsized":"2024-05-30T17:07:42.874600+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'155","reported_seq":272,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.869426+0000","last_change":"2024-05-30T17:04:52.993970+0000","last_active":"2024-05-30T17:07:42.869426+0000","last_peered":"2024-05-30T17:07:42.869426+0000","last_clean":"2024-05-30T17:07:42.869426+0000","last_became_active":"2024-05-30T17:04:52.993214+0000","last_became_peered":"2024-05-30T17:04:52.993214+0000","last_unstale":"2024-05-30T17:07:42.869426+0000","last_undegraded":"2024-05-30T17:07:42.869426+0000","last_fullsized":"2024-05-30T17:07:42.869426+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:21:06.134079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'143","reported_seq":235,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.874620+0000","last_change":"2024-05-30T17:04:52.980018+0000","last_active":"2024-05-30T17:07:42.874620+0000","last_peered":"2024-05-30T17:07:42.874620+0000","last_clean":"2024-05-30T17:07:42.874620+0000","last_became_active":"2024-05-30T17:04:52.979697+0000","last_became_peered":"2024-05-30T17:04:52.979697+0000","last_unstale":"2024-05-30T17:07:42.874620+0000","last_undegraded":"2024-05-30T17:07:42.874620+0000","last_fullsized":"2024-05-30T17:07:42.874620+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:30:44.014138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":77,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'123","reported_seq":185,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.858467+0000","last_change":"2024-05-30T17:04:52.978441+0000","last_active":"2024-05-30T17:07:42.858467+0000","last_peered":"2024-05-30T17:07:42.858467+0000","last_clean":"2024-05-30T17:07:42.858467+0000","last_became_active":"2024-05-30T17:04:52.978065+0000","last_became_peered":"2024-05-30T17:04:52.978065+0000","last_unstale":"2024-05-30T17:07:42.858467+0000","last_undegraded":"2024-05-30T17:07:42.858467+0000","last_fullsized":"2024-05-30T17:07:42.858467+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289407237,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":70,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":23,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.856712+0000","last_change":"2024-05-30T17:04:51.944167+0000","last_active":"2024-05-30T17:05:34.856712+0000","last_peered":"2024-05-30T17:05:34.856712+0000","last_clean":"2024-05-30T17:05:34.856712+0000","last_became_active":"2024-05-30T17:04:51.943796+0000","last_became_peered":"2024-05-30T17:04:51.943796+0000","last_unstale":"2024-05-30T17:05:34.856712+0000","last_undegraded":"2024-05-30T17:05:34.856712+0000","last_fullsized":"2024-05-30T17:05:34.856712+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:08:50.016832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'135","reported_seq":222,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.867563+0000","last_change":"2024-05-30T17:04:52.998976+0000","last_active":"2024-05-30T17:07:42.867563+0000","last_peered":"2024-05-30T17:07:42.867563+0000","last_clean":"2024-05-30T17:07:42.867563+0000","last_became_active":"2024-05-30T17:04:52.998673+0000","last_became_peered":"2024-05-30T17:04:52.998673+0000","last_unstale":"2024-05-30T17:07:42.867563+0000","last_undegraded":"2024-05-30T17:07:42.867563+0000","last_fullsized":"2024-05-30T17:07:42.867563+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:43:58.331966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849900+0000","last_change":"2024-05-30T17:04:51.939570+0000","last_active":"2024-05-30T17:05:34.849900+0000","last_peered":"2024-05-30T17:05:34.849900+0000","last_clean":"2024-05-30T17:05:34.849900+0000","last_became_active":"2024-05-30T17:04:51.937127+0000","last_became_peered":"2024-05-30T17:04:51.937127+0000","last_unstale":"2024-05-30T17:05:34.849900+0000","last_undegraded":"2024-05-30T17:05:34.849900+0000","last_fullsized":"2024-05-30T17:05:34.849900+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:30:18.245355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'141","reported_seq":232,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860777+0000","last_change":"2024-05-30T17:04:52.983816+0000","last_active":"2024-05-30T17:07:42.860777+0000","last_peered":"2024-05-30T17:07:42.860777+0000","last_clean":"2024-05-30T17:07:42.860777+0000","last_became_active":"2024-05-30T17:04:52.981316+0000","last_became_peered":"2024-05-30T17:04:52.981316+0000","last_unstale":"2024-05-30T17:07:42.860777+0000","last_undegraded":"2024-05-30T17:07:42.860777+0000","last_fullsized":"2024-05-30T17:07:42.860777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:26:13.610126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738885,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":74,"num_read_kb":303104,"num_write":82,"num_write_kb":331777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'3","reported_seq":23,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:24.706470+0000","last_change":"2024-05-30T17:04:51.938960+0000","last_active":"2024-05-30T17:07:24.706470+0000","last_peered":"2024-05-30T17:07:24.706470+0000","last_clean":"2024-05-30T17:07:24.706470+0000","last_became_active":"2024-05-30T17:04:51.938644+0000","last_became_peered":"2024-05-30T17:04:51.938644+0000","last_unstale":"2024-05-30T17:07:24.706470+0000","last_undegraded":"2024-05-30T17:07:24.706470+0000","last_fullsized":"2024-05-30T17:07:24.706470+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'148","reported_seq":253,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.871545+0000","last_change":"2024-05-30T17:04:52.988274+0000","last_active":"2024-05-30T17:07:42.871545+0000","last_peered":"2024-05-30T17:07:42.871545+0000","last_clean":"2024-05-30T17:07:42.871545+0000","last_became_active":"2024-05-30T17:04:52.987961+0000","last_became_peered":"2024-05-30T17:04:52.987961+0000","last_unstale":"2024-05-30T17:07:42.871545+0000","last_undegraded":"2024-05-30T17:07:42.871545+0000","last_fullsized":"2024-05-30T17:07:42.871545+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:21.613683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904448,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":87,"num_write_kb":356352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"134'3","reported_seq":31,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:49.130833+0000","last_change":"2024-05-30T17:07:49.130892+0000","last_active":"2024-05-30T17:07:49.130833+0000","last_peered":"2024-05-30T17:07:49.130833+0000","last_clean":"2024-05-30T17:07:49.130833+0000","last_became_active":"2024-05-30T17:04:51.929298+0000","last_became_peered":"2024-05-30T17:04:51.929298+0000","last_unstale":"2024-05-30T17:07:49.130833+0000","last_undegraded":"2024-05-30T17:07:49.130833+0000","last_fullsized":"2024-05-30T17:07:49.130833+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'3","last_scrub_stamp":"2024-05-30T17:07:49.130712+0000","last_deep_scrub":"134'3","last_deep_scrub_stamp":"2024-05-30T17:07:49.130712+0000","last_clean_scrub_stamp":"2024-05-30T17:07:49.130712+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:13.067287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'134","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.874012+0000","last_change":"2024-05-30T17:04:52.986145+0000","last_active":"2024-05-30T17:07:42.874012+0000","last_peered":"2024-05-30T17:07:42.874012+0000","last_clean":"2024-05-30T17:07:42.874012+0000","last_became_active":"2024-05-30T17:04:52.985811+0000","last_became_peered":"2024-05-30T17:04:52.985811+0000","last_unstale":"2024-05-30T17:07:42.874012+0000","last_undegraded":"2024-05-30T17:07:42.874012+0000","last_fullsized":"2024-05-30T17:07:42.874012+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:50.175063+0000","last_change":"2024-05-30T17:07:50.175119+0000","last_active":"2024-05-30T17:07:50.175063+0000","last_peered":"2024-05-30T17:07:50.175063+0000","last_clean":"2024-05-30T17:07:50.175063+0000","last_became_active":"2024-05-30T17:04:51.942597+0000","last_became_peered":"2024-05-30T17:04:51.942597+0000","last_unstale":"2024-05-30T17:07:50.175063+0000","last_undegraded":"2024-05-30T17:07:50.175063+0000","last_fullsized":"2024-05-30T17:07:50.175063+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'1","last_scrub_stamp":"2024-05-30T17:07:50.174913+0000","last_deep_scrub":"131'1","last_deep_scrub_stamp":"2024-05-30T17:07:50.174913+0000","last_clean_scrub_stamp":"2024-05-30T17:07:50.174913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:28.797745+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'131","reported_seq":240,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.868214+0000","last_change":"2024-05-30T17:04:52.967603+0000","last_active":"2024-05-30T17:07:42.868214+0000","last_peered":"2024-05-30T17:07:42.868214+0000","last_clean":"2024-05-30T17:07:42.868214+0000","last_became_active":"2024-05-30T17:04:52.967033+0000","last_became_peered":"2024-05-30T17:04:52.967033+0000","last_unstale":"2024-05-30T17:07:42.868214+0000","last_undegraded":"2024-05-30T17:07:42.868214+0000","last_fullsized":"2024-05-30T17:07:42.868214+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:47:18.064738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:50.646754+0000","last_change":"2024-05-30T17:07:50.646862+0000","last_active":"2024-05-30T17:07:50.646754+0000","last_peered":"2024-05-30T17:07:50.646754+0000","last_clean":"2024-05-30T17:07:50.646754+0000","last_became_active":"2024-05-30T17:04:51.941974+0000","last_became_peered":"2024-05-30T17:04:51.941974+0000","last_unstale":"2024-05-30T17:07:50.646754+0000","last_undegraded":"2024-05-30T17:07:50.646754+0000","last_fullsized":"2024-05-30T17:07:50.646754+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'1","last_scrub_stamp":"2024-05-30T17:07:50.646550+0000","last_deep_scrub":"131'1","last_deep_scrub_stamp":"2024-05-30T17:07:50.646550+0000","last_clean_scrub_stamp":"2024-05-30T17:07:50.646550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:04:41.381411+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'143","reported_seq":236,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.861865+0000","last_change":"2024-05-30T17:04:52.988985+0000","last_active":"2024-05-30T17:07:42.861865+0000","last_peered":"2024-05-30T17:07:42.861865+0000","last_clean":"2024-05-30T17:07:42.861865+0000","last_became_active":"2024-05-30T17:04:52.988645+0000","last_became_peered":"2024-05-30T17:04:52.988645+0000","last_unstale":"2024-05-30T17:07:42.861865+0000","last_undegraded":"2024-05-30T17:07:42.861865+0000","last_fullsized":"2024-05-30T17:07:42.861865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:01:40.533224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":76,"num_read_kb":303106,"num_write":79,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.934368+0000","last_change":"2024-05-30T17:04:51.951513+0000","last_active":"2024-05-30T17:05:34.934368+0000","last_peered":"2024-05-30T17:05:34.934368+0000","last_clean":"2024-05-30T17:05:34.934368+0000","last_became_active":"2024-05-30T17:04:51.951223+0000","last_became_peered":"2024-05-30T17:04:51.951223+0000","last_unstale":"2024-05-30T17:05:34.934368+0000","last_undegraded":"2024-05-30T17:05:34.934368+0000","last_fullsized":"2024-05-30T17:05:34.934368+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:35:13.878448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'131","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.863360+0000","last_change":"2024-05-30T17:04:52.986452+0000","last_active":"2024-05-30T17:07:42.863360+0000","last_peered":"2024-05-30T17:07:42.863360+0000","last_clean":"2024-05-30T17:07:42.863360+0000","last_became_active":"2024-05-30T17:04:52.986164+0000","last_became_peered":"2024-05-30T17:04:52.986164+0000","last_unstale":"2024-05-30T17:07:42.863360+0000","last_undegraded":"2024-05-30T17:07:42.863360+0000","last_fullsized":"2024-05-30T17:07:42.863360+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":66,"num_write_kb":253954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:20.773901+0000","last_change":"2024-05-30T17:04:51.955992+0000","last_active":"2024-05-30T17:07:20.773901+0000","last_peered":"2024-05-30T17:07:20.773901+0000","last_clean":"2024-05-30T17:07:20.773901+0000","last_became_active":"2024-05-30T17:04:51.955666+0000","last_became_peered":"2024-05-30T17:04:51.955666+0000","last_unstale":"2024-05-30T17:07:20.773901+0000","last_undegraded":"2024-05-30T17:07:20.773901+0000","last_fullsized":"2024-05-30T17:07:20.773901+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'133","reported_seq":190,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.876829+0000","last_change":"2024-05-30T17:04:52.984705+0000","last_active":"2024-05-30T17:07:42.876829+0000","last_peered":"2024-05-30T17:07:42.876829+0000","last_clean":"2024-05-30T17:07:42.876829+0000","last_became_active":"2024-05-30T17:04:52.984409+0000","last_became_peered":"2024-05-30T17:04:52.984409+0000","last_unstale":"2024-05-30T17:07:42.876829+0000","last_undegraded":"2024-05-30T17:07:42.876829+0000","last_fullsized":"2024-05-30T17:07:42.876829+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":39,"num_read_kb":159744,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.123168+0000","last_change":"2024-05-30T17:04:51.952573+0000","last_active":"2024-05-30T17:06:00.123168+0000","last_peered":"2024-05-30T17:06:00.123168+0000","last_clean":"2024-05-30T17:06:00.123168+0000","last_became_active":"2024-05-30T17:04:51.948281+0000","last_became_peered":"2024-05-30T17:04:51.948281+0000","last_unstale":"2024-05-30T17:06:00.123168+0000","last_undegraded":"2024-05-30T17:06:00.123168+0000","last_fullsized":"2024-05-30T17:06:00.123168+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:22:26.183919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'118","reported_seq":180,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859139+0000","last_change":"2024-05-30T17:04:52.986184+0000","last_active":"2024-05-30T17:07:42.859139+0000","last_peered":"2024-05-30T17:07:42.859139+0000","last_clean":"2024-05-30T17:07:42.859139+0000","last_became_active":"2024-05-30T17:04:52.985889+0000","last_became_peered":"2024-05-30T17:04:52.985889+0000","last_unstale":"2024-05-30T17:07:42.859139+0000","last_undegraded":"2024-05-30T17:07:42.859139+0000","last_fullsized":"2024-05-30T17:07:42.859139+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:03:27.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881146,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":59,"num_write_kb":229378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:16.059234+0000","last_change":"2024-05-30T17:04:51.954829+0000","last_active":"2024-05-30T17:07:16.059234+0000","last_peered":"2024-05-30T17:07:16.059234+0000","last_clean":"2024-05-30T17:07:16.059234+0000","last_became_active":"2024-05-30T17:04:51.954434+0000","last_became_peered":"2024-05-30T17:04:51.954434+0000","last_unstale":"2024-05-30T17:07:16.059234+0000","last_undegraded":"2024-05-30T17:07:16.059234+0000","last_fullsized":"2024-05-30T17:07:16.059234+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'124","reported_seq":221,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.863792+0000","last_change":"2024-05-30T17:04:52.989346+0000","last_active":"2024-05-30T17:07:42.863792+0000","last_peered":"2024-05-30T17:07:42.863792+0000","last_clean":"2024-05-30T17:07:42.863792+0000","last_became_active":"2024-05-30T17:04:52.989051+0000","last_became_peered":"2024-05-30T17:04:52.989051+0000","last_unstale":"2024-05-30T17:07:42.863792+0000","last_undegraded":"2024-05-30T17:07:42.863792+0000","last_fullsized":"2024-05-30T17:07:42.863792+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:37:36.128167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'28","reported_seq":47,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:29.184425+0000","last_change":"2024-05-30T17:04:51.938158+0000","last_active":"2024-05-30T17:06:29.184425+0000","last_peered":"2024-05-30T17:06:29.184425+0000","last_clean":"2024-05-30T17:06:29.184425+0000","last_became_active":"2024-05-30T17:04:51.937857+0000","last_became_peered":"2024-05-30T17:04:51.937857+0000","last_unstale":"2024-05-30T17:06:29.184425+0000","last_undegraded":"2024-05-30T17:06:29.184425+0000","last_fullsized":"2024-05-30T17:06:29.184425+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:35:25.458335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'134","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.854431+0000","last_change":"2024-05-30T17:04:52.995423+0000","last_active":"2024-05-30T17:07:42.854431+0000","last_peered":"2024-05-30T17:07:42.854431+0000","last_clean":"2024-05-30T17:07:42.854431+0000","last_became_active":"2024-05-30T17:04:52.995128+0000","last_became_peered":"2024-05-30T17:04:52.995128+0000","last_unstale":"2024-05-30T17:07:42.854431+0000","last_undegraded":"2024-05-30T17:07:42.854431+0000","last_fullsized":"2024-05-30T17:07:42.854431+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:53:47.501828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":78,"num_write_kb":303106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'438","reported_seq":457,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.804016+0000","last_change":"2024-05-30T17:04:51.953550+0000","last_active":"2024-05-30T17:07:42.804016+0000","last_peered":"2024-05-30T17:07:42.804016+0000","last_clean":"2024-05-30T17:07:42.804016+0000","last_became_active":"2024-05-30T17:04:51.953276+0000","last_became_peered":"2024-05-30T17:04:51.953276+0000","last_unstale":"2024-05-30T17:07:42.804016+0000","last_undegraded":"2024-05-30T17:07:42.804016+0000","last_fullsized":"2024-05-30T17:07:42.804016+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:07:44.468185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1698766,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":437,"num_write_kb":1833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'155","reported_seq":223,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.850609+0000","last_change":"2024-05-30T17:04:52.988845+0000","last_active":"2024-05-30T17:07:42.850609+0000","last_peered":"2024-05-30T17:07:42.850609+0000","last_clean":"2024-05-30T17:07:42.850609+0000","last_became_active":"2024-05-30T17:04:52.986757+0000","last_became_peered":"2024-05-30T17:04:52.986757+0000","last_unstale":"2024-05-30T17:07:42.850609+0000","last_undegraded":"2024-05-30T17:07:42.850609+0000","last_fullsized":"2024-05-30T17:07:42.850609+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385875968,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":92,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:50.402139+0000","last_change":"2024-05-30T17:07:50.402169+0000","last_active":"2024-05-30T17:07:50.402139+0000","last_peered":"2024-05-30T17:07:50.402139+0000","last_clean":"2024-05-30T17:07:50.402139+0000","last_became_active":"2024-05-30T17:04:51.944167+0000","last_became_peered":"2024-05-30T17:04:51.944167+0000","last_unstale":"2024-05-30T17:07:50.402139+0000","last_undegraded":"2024-05-30T17:07:50.402139+0000","last_fullsized":"2024-05-30T17:07:50.402139+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:50.402020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:50.402020+0000","last_clean_scrub_stamp":"2024-05-30T17:07:50.402020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:40:18.802356+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'125","reported_seq":204,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859817+0000","last_change":"2024-05-30T17:04:52.991186+0000","last_active":"2024-05-30T17:07:42.859817+0000","last_peered":"2024-05-30T17:07:42.859817+0000","last_clean":"2024-05-30T17:07:42.859817+0000","last_became_active":"2024-05-30T17:04:52.990049+0000","last_became_peered":"2024-05-30T17:04:52.990049+0000","last_unstale":"2024-05-30T17:07:42.859817+0000","last_undegraded":"2024-05-30T17:07:42.859817+0000","last_fullsized":"2024-05-30T17:07:42.859817+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:15.773081+0000","last_change":"2024-05-30T17:04:51.947216+0000","last_active":"2024-05-30T17:07:15.773081+0000","last_peered":"2024-05-30T17:07:15.773081+0000","last_clean":"2024-05-30T17:07:15.773081+0000","last_became_active":"2024-05-30T17:04:51.946868+0000","last_became_peered":"2024-05-30T17:04:51.946868+0000","last_unstale":"2024-05-30T17:07:15.773081+0000","last_undegraded":"2024-05-30T17:07:15.773081+0000","last_fullsized":"2024-05-30T17:07:15.773081+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'145","reported_seq":228,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.856742+0000","last_change":"2024-05-30T17:04:52.986670+0000","last_active":"2024-05-30T17:07:42.856742+0000","last_peered":"2024-05-30T17:07:42.856742+0000","last_clean":"2024-05-30T17:07:42.856742+0000","last_became_active":"2024-05-30T17:04:52.986293+0000","last_became_peered":"2024-05-30T17:04:52.986293+0000","last_unstale":"2024-05-30T17:07:42.856742+0000","last_undegraded":"2024-05-30T17:07:42.856742+0000","last_fullsized":"2024-05-30T17:07:42.856742+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:28.097184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293317,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":90,"num_write_kb":364545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":27,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.955398+0000","last_change":"2024-05-30T17:04:51.955924+0000","last_active":"2024-05-30T17:05:34.955398+0000","last_peered":"2024-05-30T17:05:34.955398+0000","last_clean":"2024-05-30T17:05:34.955398+0000","last_became_active":"2024-05-30T17:04:51.955542+0000","last_became_peered":"2024-05-30T17:04:51.955542+0000","last_unstale":"2024-05-30T17:05:34.955398+0000","last_undegraded":"2024-05-30T17:05:34.955398+0000","last_fullsized":"2024-05-30T17:05:34.955398+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:31:38.543340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'111","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.865827+0000","last_change":"2024-05-30T17:04:53.001644+0000","last_active":"2024-05-30T17:07:42.865827+0000","last_peered":"2024-05-30T17:07:42.865827+0000","last_clean":"2024-05-30T17:07:42.865827+0000","last_became_active":"2024-05-30T17:04:52.999924+0000","last_became_peered":"2024-05-30T17:04:52.999924+0000","last_unstale":"2024-05-30T17:07:42.865827+0000","last_undegraded":"2024-05-30T17:07:42.865827+0000","last_fullsized":"2024-05-30T17:07:42.865827+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:17:36.947839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.850216+0000","last_change":"2024-05-30T17:04:51.948540+0000","last_active":"2024-05-30T17:05:34.850216+0000","last_peered":"2024-05-30T17:05:34.850216+0000","last_clean":"2024-05-30T17:05:34.850216+0000","last_became_active":"2024-05-30T17:04:51.947928+0000","last_became_peered":"2024-05-30T17:04:51.947928+0000","last_unstale":"2024-05-30T17:05:34.850216+0000","last_undegraded":"2024-05-30T17:05:34.850216+0000","last_fullsized":"2024-05-30T17:05:34.850216+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:10:19.343810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'133","reported_seq":231,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.853021+0000","last_change":"2024-05-30T17:04:52.976220+0000","last_active":"2024-05-30T17:07:42.853021+0000","last_peered":"2024-05-30T17:07:42.853021+0000","last_clean":"2024-05-30T17:07:42.853021+0000","last_became_active":"2024-05-30T17:04:52.974987+0000","last_became_peered":"2024-05-30T17:04:52.974987+0000","last_unstale":"2024-05-30T17:07:42.853021+0000","last_undegraded":"2024-05-30T17:07:42.853021+0000","last_fullsized":"2024-05-30T17:07:42.853021+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:37:15.443712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360710144,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":81,"num_read_kb":331776,"num_write":86,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:09.704118+0000","last_change":"2024-05-30T17:04:51.950294+0000","last_active":"2024-05-30T17:07:09.704118+0000","last_peered":"2024-05-30T17:07:09.704118+0000","last_clean":"2024-05-30T17:07:09.704118+0000","last_became_active":"2024-05-30T17:04:51.949704+0000","last_became_peered":"2024-05-30T17:04:51.949704+0000","last_unstale":"2024-05-30T17:07:09.704118+0000","last_undegraded":"2024-05-30T17:07:09.704118+0000","last_fullsized":"2024-05-30T17:07:09.704118+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'145","reported_seq":226,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.874825+0000","last_change":"2024-05-30T17:04:52.989374+0000","last_active":"2024-05-30T17:07:42.874825+0000","last_peered":"2024-05-30T17:07:42.874825+0000","last_clean":"2024-05-30T17:07:42.874825+0000","last_became_active":"2024-05-30T17:04:52.989083+0000","last_became_peered":"2024-05-30T17:04:52.989083+0000","last_unstale":"2024-05-30T17:07:42.874825+0000","last_undegraded":"2024-05-30T17:07:42.874825+0000","last_fullsized":"2024-05-30T17:07:42.874825+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":63,"num_read_kb":237573,"num_write":85,"num_write_kb":307205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.935270+0000","last_change":"2024-05-30T17:04:51.942441+0000","last_active":"2024-05-30T17:05:34.935270+0000","last_peered":"2024-05-30T17:05:34.935270+0000","last_clean":"2024-05-30T17:05:34.935270+0000","last_became_active":"2024-05-30T17:04:51.942121+0000","last_became_peered":"2024-05-30T17:04:51.942121+0000","last_unstale":"2024-05-30T17:05:34.935270+0000","last_undegraded":"2024-05-30T17:05:34.935270+0000","last_fullsized":"2024-05-30T17:05:34.935270+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:27.082196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'133","reported_seq":233,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.869246+0000","last_change":"2024-05-30T17:04:52.994569+0000","last_active":"2024-05-30T17:07:42.869246+0000","last_peered":"2024-05-30T17:07:42.869246+0000","last_clean":"2024-05-30T17:07:42.869246+0000","last_became_active":"2024-05-30T17:04:52.994290+0000","last_became_peered":"2024-05-30T17:04:52.994290+0000","last_unstale":"2024-05-30T17:07:42.869246+0000","last_undegraded":"2024-05-30T17:07:42.869246+0000","last_fullsized":"2024-05-30T17:07:42.869246+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.849744+0000","last_change":"2024-05-30T17:04:51.945428+0000","last_active":"2024-05-30T17:05:34.849744+0000","last_peered":"2024-05-30T17:05:34.849744+0000","last_clean":"2024-05-30T17:05:34.849744+0000","last_became_active":"2024-05-30T17:04:51.945142+0000","last_became_peered":"2024-05-30T17:04:51.945142+0000","last_unstale":"2024-05-30T17:05:34.849744+0000","last_undegraded":"2024-05-30T17:05:34.849744+0000","last_fullsized":"2024-05-30T17:05:34.849744+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:25:46.375765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'132","reported_seq":233,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.876766+0000","last_change":"2024-05-30T17:04:52.993514+0000","last_active":"2024-05-30T17:07:42.876766+0000","last_peered":"2024-05-30T17:07:42.876766+0000","last_clean":"2024-05-30T17:07:42.876766+0000","last_became_active":"2024-05-30T17:04:52.988660+0000","last_became_peered":"2024-05-30T17:04:52.988660+0000","last_unstale":"2024-05-30T17:07:42.876766+0000","last_undegraded":"2024-05-30T17:07:42.876766+0000","last_fullsized":"2024-05-30T17:07:42.876766+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'3","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:00.150462+0000","last_change":"2024-05-30T17:04:51.943410+0000","last_active":"2024-05-30T17:06:00.150462+0000","last_peered":"2024-05-30T17:06:00.150462+0000","last_clean":"2024-05-30T17:06:00.150462+0000","last_became_active":"2024-05-30T17:04:51.943085+0000","last_became_peered":"2024-05-30T17:04:51.943085+0000","last_unstale":"2024-05-30T17:06:00.150462+0000","last_undegraded":"2024-05-30T17:06:00.150462+0000","last_fullsized":"2024-05-30T17:06:00.150462+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:07:18.313092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'128","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.868293+0000","last_change":"2024-05-30T17:04:52.992939+0000","last_active":"2024-05-30T17:07:42.868293+0000","last_peered":"2024-05-30T17:07:42.868293+0000","last_clean":"2024-05-30T17:07:42.868293+0000","last_became_active":"2024-05-30T17:04:52.989409+0000","last_became_peered":"2024-05-30T17:04:52.989409+0000","last_unstale":"2024-05-30T17:07:42.868293+0000","last_undegraded":"2024-05-30T17:07:42.868293+0000","last_fullsized":"2024-05-30T17:07:42.868293+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:43:41.892452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795845,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":72,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:01.056644+0000","last_change":"2024-05-30T17:04:51.948194+0000","last_active":"2024-05-30T17:07:01.056644+0000","last_peered":"2024-05-30T17:07:01.056644+0000","last_clean":"2024-05-30T17:07:01.056644+0000","last_became_active":"2024-05-30T17:04:51.947919+0000","last_became_peered":"2024-05-30T17:04:51.947919+0000","last_unstale":"2024-05-30T17:07:01.056644+0000","last_undegraded":"2024-05-30T17:07:01.056644+0000","last_fullsized":"2024-05-30T17:07:01.056644+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'145","reported_seq":211,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.869693+0000","last_change":"2024-05-30T17:04:52.981350+0000","last_active":"2024-05-30T17:07:42.869693+0000","last_peered":"2024-05-30T17:07:42.869693+0000","last_clean":"2024-05-30T17:07:42.869693+0000","last_became_active":"2024-05-30T17:04:52.978531+0000","last_became_peered":"2024-05-30T17:04:52.978531+0000","last_unstale":"2024-05-30T17:07:42.869693+0000","last_undegraded":"2024-05-30T17:07:42.869693+0000","last_fullsized":"2024-05-30T17:07:42.869693+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":48,"num_read_kb":188418,"num_write":83,"num_write_kb":323586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:01.056601+0000","last_change":"2024-05-30T17:04:51.950726+0000","last_active":"2024-05-30T17:07:01.056601+0000","last_peered":"2024-05-30T17:07:01.056601+0000","last_clean":"2024-05-30T17:07:01.056601+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:07:01.056601+0000","last_undegraded":"2024-05-30T17:07:01.056601+0000","last_fullsized":"2024-05-30T17:07:01.056601+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'123","reported_seq":224,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859963+0000","last_change":"2024-05-30T17:05:35.214281+0000","last_active":"2024-05-30T17:07:42.859963+0000","last_peered":"2024-05-30T17:07:42.859963+0000","last_clean":"2024-05-30T17:07:42.859963+0000","last_became_active":"2024-05-30T17:05:34.860839+0000","last_became_peered":"2024-05-30T17:05:34.860839+0000","last_unstale":"2024-05-30T17:07:42.859963+0000","last_undegraded":"2024-05-30T17:07:42.859963+0000","last_fullsized":"2024-05-30T17:07:42.859963+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:11:46.508726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":69,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:04.703280+0000","last_change":"2024-05-30T17:04:51.951054+0000","last_active":"2024-05-30T17:07:04.703280+0000","last_peered":"2024-05-30T17:07:04.703280+0000","last_clean":"2024-05-30T17:07:04.703280+0000","last_became_active":"2024-05-30T17:04:51.950364+0000","last_became_peered":"2024-05-30T17:04:51.950364+0000","last_unstale":"2024-05-30T17:07:04.703280+0000","last_undegraded":"2024-05-30T17:07:04.703280+0000","last_fullsized":"2024-05-30T17:07:04.703280+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'133","reported_seq":187,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.874209+0000","last_change":"2024-05-30T17:04:52.967099+0000","last_active":"2024-05-30T17:07:42.874209+0000","last_peered":"2024-05-30T17:07:42.874209+0000","last_clean":"2024-05-30T17:07:42.874209+0000","last_became_active":"2024-05-30T17:04:52.966035+0000","last_became_peered":"2024-05-30T17:04:52.966035+0000","last_unstale":"2024-05-30T17:07:42.874209+0000","last_undegraded":"2024-05-30T17:07:42.874209+0000","last_fullsized":"2024-05-30T17:07:42.874209+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:44:24.565166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"134'3","reported_seq":31,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:51.380874+0000","last_change":"2024-05-30T17:07:51.380930+0000","last_active":"2024-05-30T17:07:51.380874+0000","last_peered":"2024-05-30T17:07:51.380874+0000","last_clean":"2024-05-30T17:07:51.380874+0000","last_became_active":"2024-05-30T17:04:51.949575+0000","last_became_peered":"2024-05-30T17:04:51.949575+0000","last_unstale":"2024-05-30T17:07:51.380874+0000","last_undegraded":"2024-05-30T17:07:51.380874+0000","last_fullsized":"2024-05-30T17:07:51.380874+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'3","last_scrub_stamp":"2024-05-30T17:07:51.380713+0000","last_deep_scrub":"134'3","last_deep_scrub_stamp":"2024-05-30T17:07:51.380713+0000","last_clean_scrub_stamp":"2024-05-30T17:07:51.380713+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:02:43.008979+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'134","reported_seq":239,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.873656+0000","last_change":"2024-05-30T17:04:52.998729+0000","last_active":"2024-05-30T17:07:42.873656+0000","last_peered":"2024-05-30T17:07:42.873656+0000","last_clean":"2024-05-30T17:07:42.873656+0000","last_became_active":"2024-05-30T17:04:52.997726+0000","last_became_peered":"2024-05-30T17:04:52.997726+0000","last_unstale":"2024-05-30T17:07:42.873656+0000","last_undegraded":"2024-05-30T17:07:42.873656+0000","last_fullsized":"2024-05-30T17:07:42.873656+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"134'49","reported_seq":68,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:35.039807+0000","last_change":"2024-05-30T17:04:51.949751+0000","last_active":"2024-05-30T17:07:35.039807+0000","last_peered":"2024-05-30T17:07:35.039807+0000","last_clean":"2024-05-30T17:07:35.039807+0000","last_became_active":"2024-05-30T17:04:51.949295+0000","last_became_peered":"2024-05-30T17:04:51.949295+0000","last_unstale":"2024-05-30T17:07:35.039807+0000","last_undegraded":"2024-05-30T17:07:35.039807+0000","last_fullsized":"2024-05-30T17:07:35.039807+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:42:41.766069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'136","reported_seq":251,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.870600+0000","last_change":"2024-05-30T17:04:52.985036+0000","last_active":"2024-05-30T17:07:42.870600+0000","last_peered":"2024-05-30T17:07:42.870600+0000","last_clean":"2024-05-30T17:07:42.870600+0000","last_became_active":"2024-05-30T17:04:52.984675+0000","last_became_peered":"2024-05-30T17:04:52.984675+0000","last_unstale":"2024-05-30T17:07:42.870600+0000","last_undegraded":"2024-05-30T17:07:42.870600+0000","last_fullsized":"2024-05-30T17:07:42.870600+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":397312,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:00.770451+0000","last_change":"2024-05-30T17:04:51.952967+0000","last_active":"2024-05-30T17:07:00.770451+0000","last_peered":"2024-05-30T17:07:00.770451+0000","last_clean":"2024-05-30T17:07:00.770451+0000","last_became_active":"2024-05-30T17:04:51.952631+0000","last_became_peered":"2024-05-30T17:04:51.952631+0000","last_unstale":"2024-05-30T17:07:00.770451+0000","last_undegraded":"2024-05-30T17:07:00.770451+0000","last_fullsized":"2024-05-30T17:07:00.770451+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'130","reported_seq":200,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.874186+0000","last_change":"2024-05-30T17:04:52.990963+0000","last_active":"2024-05-30T17:07:42.874186+0000","last_peered":"2024-05-30T17:07:42.874186+0000","last_clean":"2024-05-30T17:07:42.874186+0000","last_became_active":"2024-05-30T17:04:52.990476+0000","last_became_peered":"2024-05-30T17:04:52.990476+0000","last_unstale":"2024-05-30T17:07:42.874186+0000","last_undegraded":"2024-05-30T17:07:42.874186+0000","last_fullsized":"2024-05-30T17:07:42.874186+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:22:59.838344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378750,"num_objects":77,"num_object_clones":1,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":1,"num_read":54,"num_read_kb":217089,"num_write":79,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:48.701144+0000","last_change":"2024-05-30T17:07:48.701181+0000","last_active":"2024-05-30T17:07:48.701144+0000","last_peered":"2024-05-30T17:07:48.701144+0000","last_clean":"2024-05-30T17:07:48.701144+0000","last_became_active":"2024-05-30T17:04:51.933061+0000","last_became_peered":"2024-05-30T17:04:51.933061+0000","last_unstale":"2024-05-30T17:07:48.701144+0000","last_undegraded":"2024-05-30T17:07:48.701144+0000","last_fullsized":"2024-05-30T17:07:48.701144+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:48.701020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:48.701020+0000","last_clean_scrub_stamp":"2024-05-30T17:07:48.701020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:51.079279+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'134","reported_seq":212,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.870019+0000","last_change":"2024-05-30T17:04:52.982699+0000","last_active":"2024-05-30T17:07:42.870019+0000","last_peered":"2024-05-30T17:07:42.870019+0000","last_clean":"2024-05-30T17:07:42.870019+0000","last_became_active":"2024-05-30T17:04:52.981542+0000","last_became_peered":"2024-05-30T17:04:52.981542+0000","last_unstale":"2024-05-30T17:07:42.870019+0000","last_undegraded":"2024-05-30T17:07:42.870019+0000","last_fullsized":"2024-05-30T17:07:42.870019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:09:12.025684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":76,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:00.770396+0000","last_change":"2024-05-30T17:04:51.947181+0000","last_active":"2024-05-30T17:07:00.770396+0000","last_peered":"2024-05-30T17:07:00.770396+0000","last_clean":"2024-05-30T17:07:00.770396+0000","last_became_active":"2024-05-30T17:04:51.946882+0000","last_became_peered":"2024-05-30T17:04:51.946882+0000","last_unstale":"2024-05-30T17:07:00.770396+0000","last_undegraded":"2024-05-30T17:07:00.770396+0000","last_fullsized":"2024-05-30T17:07:00.770396+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'157","reported_seq":257,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.860002+0000","last_change":"2024-05-30T17:04:52.992399+0000","last_active":"2024-05-30T17:07:42.860002+0000","last_peered":"2024-05-30T17:07:42.860002+0000","last_clean":"2024-05-30T17:07:42.860002+0000","last_became_active":"2024-05-30T17:04:52.992040+0000","last_became_peered":"2024-05-30T17:04:52.992040+0000","last_unstale":"2024-05-30T17:07:42.860002+0000","last_undegraded":"2024-05-30T17:07:42.860002+0000","last_fullsized":"2024-05-30T17:07:42.860002+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:37:16.335923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":411041792,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":98,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.855981+0000","last_change":"2024-05-30T17:04:51.952089+0000","last_active":"2024-05-30T17:05:34.855981+0000","last_peered":"2024-05-30T17:05:34.855981+0000","last_clean":"2024-05-30T17:05:34.855981+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:05:34.855981+0000","last_undegraded":"2024-05-30T17:05:34.855981+0000","last_fullsized":"2024-05-30T17:05:34.855981+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:03:20.313618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'136","reported_seq":234,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.859233+0000","last_change":"2024-05-30T17:04:52.988572+0000","last_active":"2024-05-30T17:07:42.859233+0000","last_peered":"2024-05-30T17:07:42.859233+0000","last_clean":"2024-05-30T17:07:42.859233+0000","last_became_active":"2024-05-30T17:04:52.987684+0000","last_became_peered":"2024-05-30T17:04:52.987684+0000","last_unstale":"2024-05-30T17:07:42.859233+0000","last_undegraded":"2024-05-30T17:07:42.859233+0000","last_fullsized":"2024-05-30T17:07:42.859233+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":82,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'35","reported_seq":89,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:56.055742+0000","last_change":"2024-05-30T17:04:51.949334+0000","last_active":"2024-05-30T17:06:56.055742+0000","last_peered":"2024-05-30T17:06:56.055742+0000","last_clean":"2024-05-30T17:06:56.055742+0000","last_became_active":"2024-05-30T17:04:51.948995+0000","last_became_peered":"2024-05-30T17:04:51.948995+0000","last_unstale":"2024-05-30T17:06:56.055742+0000","last_undegraded":"2024-05-30T17:06:56.055742+0000","last_fullsized":"2024-05-30T17:06:56.055742+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4250,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'136","reported_seq":232,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.872796+0000","last_change":"2024-05-30T17:04:52.992411+0000","last_active":"2024-05-30T17:07:42.872796+0000","last_peered":"2024-05-30T17:07:42.872796+0000","last_clean":"2024-05-30T17:07:42.872796+0000","last_became_active":"2024-05-30T17:04:52.991997+0000","last_became_peered":"2024-05-30T17:04:52.991997+0000","last_unstale":"2024-05-30T17:07:42.872796+0000","last_undegraded":"2024-05-30T17:07:42.872796+0000","last_fullsized":"2024-05-30T17:07:42.872796+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":402653184,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":96,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:06:54.701541+0000","last_change":"2024-05-30T17:04:51.946511+0000","last_active":"2024-05-30T17:06:54.701541+0000","last_peered":"2024-05-30T17:06:54.701541+0000","last_clean":"2024-05-30T17:06:54.701541+0000","last_became_active":"2024-05-30T17:04:51.946173+0000","last_became_peered":"2024-05-30T17:04:51.946173+0000","last_unstale":"2024-05-30T17:06:54.701541+0000","last_undegraded":"2024-05-30T17:06:54.701541+0000","last_fullsized":"2024-05-30T17:06:54.701541+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'122","reported_seq":192,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:42.864677+0000","last_change":"2024-05-30T17:04:52.992639+0000","last_active":"2024-05-30T17:07:42.864677+0000","last_peered":"2024-05-30T17:07:42.864677+0000","last_clean":"2024-05-30T17:07:42.864677+0000","last_became_active":"2024-05-30T17:04:52.992202+0000","last_became_peered":"2024-05-30T17:04:52.992202+0000","last_unstale":"2024-05-30T17:07:42.864677+0000","last_undegraded":"2024-05-30T17:07:42.864677+0000","last_fullsized":"2024-05-30T17:07:42.864677+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:51.907869+0000","last_clean_scrub_stamp":"2024-05-30T17:04:51.907869+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:24:34.869318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":53,"num_read_kb":217088,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":21,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:34.856152+0000","last_change":"2024-05-30T17:04:51.951966+0000","last_active":"2024-05-30T17:05:34.856152+0000","last_peered":"2024-05-30T17:05:34.856152+0000","last_clean":"2024-05-30T17:05:34.856152+0000","last_became_active":"2024-05-30T17:04:51.951484+0000","last_became_peered":"2024-05-30T17:04:51.951484+0000","last_unstale":"2024-05-30T17:05:34.856152+0000","last_undegraded":"2024-05-30T17:05:34.856152+0000","last_fullsized":"2024-05-30T17:05:34.856152+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:04:50.892881+0000","last_clean_scrub_stamp":"2024-05-30T17:04:50.892881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:37:41.627294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":388,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:05:44.508422+0000","last_change":"2024-05-30T16:43:41.958343+0000","last_active":"2024-05-30T17:05:44.508422+0000","last_peered":"2024-05-30T17:05:44.508422+0000","last_clean":"2024-05-30T17:05:44.508422+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:05:44.508422+0000","last_undegraded":"2024-05-30T17:05:44.508422+0000","last_fullsized":"2024-05-30T17:05:44.508422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T16:43:40.432932+0000","last_clean_scrub_stamp":"2024-05-30T16:43:40.432932+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:13:55.089963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":20912801557,"num_objects":4996,"num_object_clones":1,"num_object_copies":9992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":1,"num_read":4290,"num_read_kb":17367090,"num_write":5064,"num_write_kb":20422699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41825673216,"data_stored":41825603114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8573,"ondisk_log_size":8573,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1703814,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":644,"num_write_kb":1934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3473408,"data_stored":3407628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":584,"ondisk_log_size":584,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509733,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5955000,"kb_used_data":5811440,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":88416840,"statfs":{"total":96636764160,"available":90538844160,"internally_reserved":0,"allocated":5950914560,"data_stored":5949962642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509733,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6190836,"kb_used_data":6051436,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":88181004,"statfs":{"total":96636764160,"available":90297348096,"internally_reserved":0,"allocated":6196670464,"data_stored":6195723625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509734,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5015352,"kb_used_data":4871792,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":89356488,"statfs":{"total":96636764160,"available":91501043712,"internally_reserved":0,"allocated":4988715008,"data_stored":4987768020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509734,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6950260,"kb_used_data":6792812,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":87421580,"statfs":{"total":96636764160,"available":89519697920,"internally_reserved":0,"allocated":6955839488,"data_stored":6954892805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509733,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4728548,"kb_used_data":4589148,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89643292,"statfs":{"total":96636764160,"available":91794731008,"internally_reserved":0,"allocated":4699287552,"data_stored":4698356383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542436,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4575160,"kb_used_data":4435760,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89796680,"statfs":{"total":96636764160,"available":91951800320,"internally_reserved":0,"allocated":4542218240,"data_stored":4541262732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542437,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4261684,"kb_used_data":4118124,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90110156,"statfs":{"total":96636764160,"available":92272799744,"internally_reserved":0,"allocated":4216958976,"data_stored":4216012047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542437,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4344296,"kb_used_data":4192416,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":90027544,"statfs":{"total":96636764160,"available":92188205056,"internally_reserved":0,"allocated":4293033984,"data_stored":4292099392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":155515333},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1716224,"data_stored":1703016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1720320,"data_stored":1699404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4290777088,"data_stored":4290773253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4697624576,"data_stored":4697620612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6954168320,"data_stored":6954156418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4987035648,"data_stored":4987027839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6195003392,"data_stored":6194987784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4215287808,"data_stored":4215276164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5947535360,"data_stored":5947523855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4538241024,"data_stored":4538237189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:07:53.679 INFO:tasks.ceph:pgid 19.3f last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.679 INFO:tasks.ceph:pgid 19.3e last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.679 INFO:tasks.ceph:pgid 19.3d last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.680 INFO:tasks.ceph:pgid 19.3c last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.680 INFO:tasks.ceph:pgid 19.3b last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.680 INFO:tasks.ceph:pgid 19.3a last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.680 INFO:tasks.ceph:pgid 19.39 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.680 INFO:tasks.ceph:pgid 19.38 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.681 INFO:tasks.ceph:pgid 19.37 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.681 INFO:tasks.ceph:pgid 19.36 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.681 INFO:tasks.ceph:pgid 19.35 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.681 INFO:tasks.ceph:pgid 19.34 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.681 INFO:tasks.ceph:pgid 19.33 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.681 INFO:tasks.ceph:pgid 19.32 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.682 INFO:tasks.ceph:pgid 19.31 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.682 INFO:tasks.ceph:pgid 19.30 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.682 INFO:tasks.ceph:pgid 19.2f last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.682 INFO:tasks.ceph:pgid 19.2e last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.682 INFO:tasks.ceph:pgid 19.2d last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.682 INFO:tasks.ceph:pgid 19.2c last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.683 INFO:tasks.ceph:pgid 19.2b last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.683 INFO:tasks.ceph:pgid 19.2a last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.683 INFO:tasks.ceph:pgid 19.29 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.683 INFO:tasks.ceph:pgid 19.28 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.683 INFO:tasks.ceph:pgid 19.27 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.683 INFO:tasks.ceph:pgid 19.26 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.683 INFO:tasks.ceph:pgid 19.25 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.684 INFO:tasks.ceph:pgid 19.24 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.684 INFO:tasks.ceph:pgid 19.23 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.684 INFO:tasks.ceph:pgid 19.22 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.684 INFO:tasks.ceph:pgid 19.21 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.684 INFO:tasks.ceph:pgid 19.20 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.684 INFO:tasks.ceph:pgid 19.1e last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.685 INFO:tasks.ceph:pgid 18.1f last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.685 INFO:tasks.ceph:pgid 19.1f last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.685 INFO:tasks.ceph:pgid 18.1e last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.685 INFO:tasks.ceph:pgid 19.1c last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.685 INFO:tasks.ceph:pgid 18.1d last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.685 INFO:tasks.ceph:pgid 19.1d last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.685 INFO:tasks.ceph:pgid 19.1a last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.686 INFO:tasks.ceph:pgid 19.1b last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.686 INFO:tasks.ceph:pgid 19.18 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.686 INFO:tasks.ceph:pgid 18.19 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.686 INFO:tasks.ceph:pgid 19.19 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.686 INFO:tasks.ceph:pgid 18.18 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.686 INFO:tasks.ceph:pgid 19.16 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.687 INFO:tasks.ceph:pgid 18.17 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.687 INFO:tasks.ceph:pgid 19.17 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.687 INFO:tasks.ceph:pgid 18.16 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.687 INFO:tasks.ceph:pgid 19.14 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.687 INFO:tasks.ceph:pgid 18.15 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.687 INFO:tasks.ceph:pgid 19.15 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.687 INFO:tasks.ceph:pgid 18.14 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.688 INFO:tasks.ceph:pgid 19.13 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.688 INFO:tasks.ceph:pgid 19.10 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.688 INFO:tasks.ceph:pgid 18.11 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.688 INFO:tasks.ceph:pgid 19.11 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.688 INFO:tasks.ceph:pgid 18.10 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.688 INFO:tasks.ceph:pgid 19.e last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.689 INFO:tasks.ceph:pgid 18.f last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.689 INFO:tasks.ceph:pgid 19.f last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.689 INFO:tasks.ceph:pgid 18.e last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.689 INFO:tasks.ceph:pgid 19.c last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.689 INFO:tasks.ceph:pgid 18.d last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.689 INFO:tasks.ceph:pgid 19.d last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.689 INFO:tasks.ceph:pgid 18.c last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.690 INFO:tasks.ceph:pgid 19.a last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.690 INFO:tasks.ceph:pgid 18.b last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.690 INFO:tasks.ceph:pgid 19.b last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.690 INFO:tasks.ceph:pgid 18.a last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.690 INFO:tasks.ceph:pgid 19.8 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.690 INFO:tasks.ceph:pgid 18.9 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.691 INFO:tasks.ceph:pgid 19.9 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.691 INFO:tasks.ceph:pgid 18.8 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.691 INFO:tasks.ceph:pgid 19.6 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.691 INFO:tasks.ceph:pgid 19.7 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.691 INFO:tasks.ceph:pgid 18.6 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.691 INFO:tasks.ceph:pgid 19.4 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.691 INFO:tasks.ceph:pgid 18.5 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.692 INFO:tasks.ceph:pgid 19.5 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.692 INFO:tasks.ceph:pgid 19.2 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.692 INFO:tasks.ceph:pgid 18.3 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.692 INFO:tasks.ceph:pgid 19.3 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.692 INFO:tasks.ceph:pgid 18.2 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.692 INFO:tasks.ceph:pgid 19.0 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.693 INFO:tasks.ceph:pgid 18.1 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.693 INFO:tasks.ceph:pgid 19.1 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.693 INFO:tasks.ceph:pgid 18.0 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.693 INFO:tasks.ceph:pgid 19.12 last_scrub_stamp 2024-05-30T17:04:51.907869+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.693 INFO:tasks.ceph:pgid 18.13 last_scrub_stamp 2024-05-30T17:04:50.892881+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=4, tm_sec=50, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.693 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-30T16:43:40.432932+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=16, tm_min=43, tm_sec=40, tm_wday=3, tm_yday=151, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=30, tm_hour=17, tm_min=7, tm_sec=46, tm_wday=3, tm_yday=151, tm_isdst=0) 2024-05-30T17:07:53.693 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-30T17:08:13.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-30T17:08:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.919+0000 7f4936800640 1 Processor -- start 2024-05-30T17:08:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.919+0000 7f4936800640 1 -- start start 2024-05-30T17:08:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.921+0000 7f4936800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f49300b0570 0x7f4930070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.921+0000 7f4936800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f493006b900 0x7f493006bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.921+0000 7f4936800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49300b49c0 con 0x7f4930162c70 2024-05-30T17:08:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.921+0000 7f4936800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7f49300b0c80 con 0x7f493006b900 2024-05-30T17:08:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.921+0000 7f4936800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f49300b64f0 con 0x7f49300b0570 2024-05-30T17:08:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.921+0000 7f492fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f493006b900 0x7f493006bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.921+0000 7f492fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f493006b900 0x7f493006bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56956/0 (socket says 172.21.15.67:56956) 2024-05-30T17:08:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.921+0000 7f492fe00640 1 -- 172.21.15.67:0/782525258 learned_addr learned my addr 172.21.15.67:0/782525258 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:08:13.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.921+0000 7f492ea00640 1 --2- 172.21.15.67:0/782525258 >> v2:172.21.15.169:3300/0 conn(0x7f49300b0570 0x7f4930070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.922+0000 7f492fe00640 1 -- 172.21.15.67:0/782525258 >> v2:172.21.15.169:3300/0 conn(0x7f49300b0570 msgr2=0x7f4930070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:08:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.922+0000 7f492fe00640 1 --2- 172.21.15.67:0/782525258 >> v2:172.21.15.169:3300/0 conn(0x7f49300b0570 0x7f4930070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.922+0000 7f492fe00640 1 -- 172.21.15.67:0/782525258 >> v1:172.21.15.67:6789/0 conn(0x7f4930162c70 legacy=0x7f49300b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.922+0000 7f492fe00640 1 -- 172.21.15.67:0/782525258 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49300b6210 con 0x7f493006b900 2024-05-30T17:08:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.922+0000 7f492ea00640 1 --2- 172.21.15.67:0/782525258 >> v2:172.21.15.169:3300/0 conn(0x7f49300b0570 0x7f4930070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:08:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.923+0000 7f492fe00640 1 --2- 172.21.15.67:0/782525258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f493006b900 0x7f493006bd90 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f49300b58c0 tx=0x7f492802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e893107ffbf805b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:08:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.923+0000 7f492e000640 1 -- 172.21.15.67:0/782525258 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928037070 con 0x7f493006b900 2024-05-30T17:08:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.923+0000 7f492e000640 1 -- 172.21.15.67:0/782525258 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4928032440 con 0x7f493006b900 2024-05-30T17:08:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.923+0000 7f492e000640 1 -- 172.21.15.67:0/782525258 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928032810 con 0x7f493006b900 2024-05-30T17:08:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.924+0000 7f4936800640 1 -- 172.21.15.67:0/782525258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f493006b900 msgr2=0x7f493006bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.924+0000 7f4936800640 1 --2- 172.21.15.67:0/782525258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f493006b900 0x7f493006bd90 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f49300b58c0 tx=0x7f492802a200 comp rx=0 tx=0).stop 2024-05-30T17:08:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.924+0000 7f4936800640 1 -- 172.21.15.67:0/782525258 shutdown_connections 2024-05-30T17:08:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.924+0000 7f4936800640 1 --2- 172.21.15.67:0/782525258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f493006b900 0x7f493006bd90 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.924+0000 7f4936800640 1 --2- 172.21.15.67:0/782525258 >> v2:172.21.15.169:3300/0 conn(0x7f49300b0570 0x7f4930070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.924+0000 7f4936800640 1 -- 172.21.15.67:0/782525258 >> 172.21.15.67:0/782525258 conn(0x7f49300b6760 msgr2=0x7f49300b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:08:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.924+0000 7f4936800640 1 -- 172.21.15.67:0/782525258 shutdown_connections 2024-05-30T17:08:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.924+0000 7f4936800640 1 -- 172.21.15.67:0/782525258 wait complete. 2024-05-30T17:08:13.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.926+0000 7f4936800640 1 Processor -- start 2024-05-30T17:08:13.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.926+0000 7f4936800640 1 -- start start 2024-05-30T17:08:13.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.927+0000 7f4936800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493006b900 0x7f49301fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:13.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.927+0000 7f4936800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49300b0570 0x7f4930154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.927+0000 7f4936800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930162c70 0x7f4930157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.927+0000 7f4936800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f49300b5fb0 con 0x7f4930162c70 2024-05-30T17:08:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.927+0000 7f4936800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f49300b5e30 con 0x7f493006b900 2024-05-30T17:08:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.927+0000 7f4936800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4930163050 con 0x7f49300b0570 2024-05-30T17:08:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.927+0000 7f492fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930162c70 0x7f4930157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.927+0000 7f492fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930162c70 0x7f4930157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56964/0 (socket says 172.21.15.67:56964) 2024-05-30T17:08:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.927+0000 7f492fe00640 1 -- 172.21.15.67:0/350489882 learned_addr learned my addr 172.21.15.67:0/350489882 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:08:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f492fe00640 1 -- 172.21.15.67:0/350489882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49300b0570 msgr2=0x7f4930154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:13.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f492fe00640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49300b0570 0x7f4930154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:13.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f492f400640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493006b900 0x7f49301fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:13.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f492fe00640 1 -- 172.21.15.67:0/350489882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493006b900 msgr2=0x7f49301fc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:13.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f492fe00640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493006b900 0x7f49301fc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:13.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f492fe00640 1 -- 172.21.15.67:0/350489882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49301fbb20 con 0x7f4930162c70 2024-05-30T17:08:13.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f492f400640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493006b900 0x7f49301fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:08:13.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f492fe00640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930162c70 0x7f4930157c20 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f49280090b0 tx=0x7f49280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:08:13.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f4927400640 1 -- 172.21.15.67:0/350489882 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928037040 con 0x7f4930162c70 2024-05-30T17:08:13.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f4927400640 1 -- 172.21.15.67:0/350489882 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4928042070 con 0x7f4930162c70 2024-05-30T17:08:13.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4930158910 con 0x7f4930162c70 2024-05-30T17:08:13.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.928+0000 7f4927400640 1 -- 172.21.15.67:0/350489882 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928030bd0 con 0x7f4930162c70 2024-05-30T17:08:13.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.929+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49301fbde0 con 0x7f4930162c70 2024-05-30T17:08:13.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.930+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49300690f0 con 0x7f4930162c70 2024-05-30T17:08:13.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.931+0000 7f4927400640 1 -- 172.21.15.67:0/350489882 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7f4928047460 con 0x7f4930162c70 2024-05-30T17:08:13.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.932+0000 7f4927400640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f490c06e510 0x7f490c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:13.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.932+0000 7f4927400640 1 -- 172.21.15.67:0/350489882 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7f49280ab790 con 0x7f4930162c70 2024-05-30T17:08:13.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.932+0000 7f492f400640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f490c06e510 0x7f490c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:13.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.933+0000 7f492f400640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f490c06e510 0x7f490c0709d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f49180015e0 tx=0x7f4918009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:08:13.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:13.937+0000 7f4927400640 1 -- 172.21.15.67:0/350489882 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4928045360 con 0x7f4930162c70 2024-05-30T17:08:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.138+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 --> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4930154ca0 con 0x7f490c06e510 2024-05-30T17:08:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.148+0000 7f4927400640 1 -- 172.21.15.67:0/350489882 <== mgr.4098 v2:172.21.15.67:6832/46426377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255821 (secure 0 0 0) 0x7f491800d4f0 con 0x7f490c06e510 2024-05-30T17:08:14.153 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:08:14.154 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-05-30T17:08:14.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f490c06e510 msgr2=0x7f490c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f490c06e510 0x7f490c0709d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f49180015e0 tx=0x7f4918009040 comp rx=0 tx=0).stop 2024-05-30T17:08:14.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930162c70 msgr2=0x7f4930157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930162c70 0x7f4930157c20 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f49280090b0 tx=0x7f49280046c0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 shutdown_connections 2024-05-30T17:08:14.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7f490c06e510 0x7f490c0709d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930162c70 0x7f4930157c20 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49300b0570 0x7f4930154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 --2- 172.21.15.67:0/350489882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f493006b900 0x7f49301fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 >> 172.21.15.67:0/350489882 conn(0x7f49300b6760 msgr2=0x7f493006bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:08:14.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 shutdown_connections 2024-05-30T17:08:14.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.154+0000 7f4936800640 1 -- 172.21.15.67:0/350489882 wait complete. 2024-05-30T17:08:14.175 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":864,"stamp":"2024-05-30T17:08:12.686817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20915095739,"num_objects":5021,"num_object_clones":1,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":1,"num_read":4440,"num_read_kb":17367224,"num_write":5867,"num_write_kb":20427049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14290,"ondisk_log_size":14290,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":42033616,"kb_used_data":40862928,"kb_used_omap":12,"kb_used_meta":1170611,"kb_avail":712941104,"statfs":{"total":773094113280,"available":730051690496,"internally_reserved":0,"allocated":41843638272,"data_stored":41836077646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1198706224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004708"},"pg_stats":[{"pgid":"19.3f","version":"134'198","reported_seq":305,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.984674+0000","last_change":"2024-05-30T17:08:01.984817+0000","last_active":"2024-05-30T17:08:01.984674+0000","last_peered":"2024-05-30T17:08:01.984674+0000","last_clean":"2024-05-30T17:08:01.984674+0000","last_became_active":"2024-05-30T17:04:52.997618+0000","last_became_peered":"2024-05-30T17:04:52.997618+0000","last_unstale":"2024-05-30T17:08:01.984674+0000","last_undegraded":"2024-05-30T17:08:01.984674+0000","last_fullsized":"2024-05-30T17:08:01.984674+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'198","last_scrub_stamp":"2024-05-30T17:08:01.984558+0000","last_deep_scrub":"134'198","last_deep_scrub_stamp":"2024-05-30T17:08:01.984558+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.984558+0000","objects_scrubbed":71,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:34.009219+0000","scrub_duration":1450,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":71,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"134'211","reported_seq":313,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:59.205284+0000","last_change":"2024-05-30T17:07:59.205321+0000","last_active":"2024-05-30T17:07:59.205284+0000","last_peered":"2024-05-30T17:07:59.205284+0000","last_clean":"2024-05-30T17:07:59.205284+0000","last_became_active":"2024-05-30T17:04:52.982524+0000","last_became_peered":"2024-05-30T17:04:52.982524+0000","last_unstale":"2024-05-30T17:07:59.205284+0000","last_undegraded":"2024-05-30T17:07:59.205284+0000","last_fullsized":"2024-05-30T17:07:59.205284+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'211","last_scrub_stamp":"2024-05-30T17:07:59.205204+0000","last_deep_scrub":"134'211","last_deep_scrub_stamp":"2024-05-30T17:07:59.205204+0000","last_clean_scrub_stamp":"2024-05-30T17:07:59.205204+0000","objects_scrubbed":78,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:46:13.945805+0000","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"134'237","reported_seq":342,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:56.362517+0000","last_change":"2024-05-30T17:07:56.362581+0000","last_active":"2024-05-30T17:07:56.362517+0000","last_peered":"2024-05-30T17:07:56.362517+0000","last_clean":"2024-05-30T17:07:56.362517+0000","last_became_active":"2024-05-30T17:04:52.984493+0000","last_became_peered":"2024-05-30T17:04:52.984493+0000","last_unstale":"2024-05-30T17:07:56.362517+0000","last_undegraded":"2024-05-30T17:07:56.362517+0000","last_fullsized":"2024-05-30T17:07:56.362517+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'237","last_scrub_stamp":"2024-05-30T17:07:56.362436+0000","last_deep_scrub":"134'237","last_deep_scrub_stamp":"2024-05-30T17:07:56.362436+0000","last_clean_scrub_stamp":"2024-05-30T17:07:56.362436+0000","objects_scrubbed":88,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:18:46.975411+0000","scrub_duration":1135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904709,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":75,"num_read_kb":286725,"num_write":94,"num_write_kb":356356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"134'197","reported_seq":266,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:58.558266+0000","last_change":"2024-05-30T17:07:58.558375+0000","last_active":"2024-05-30T17:07:58.558266+0000","last_peered":"2024-05-30T17:07:58.558266+0000","last_clean":"2024-05-30T17:07:58.558266+0000","last_became_active":"2024-05-30T17:05:34.859953+0000","last_became_peered":"2024-05-30T17:05:34.859953+0000","last_unstale":"2024-05-30T17:07:58.558266+0000","last_undegraded":"2024-05-30T17:07:58.558266+0000","last_fullsized":"2024-05-30T17:07:58.558266+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'197","last_scrub_stamp":"2024-05-30T17:07:58.558186+0000","last_deep_scrub":"134'197","last_deep_scrub_stamp":"2024-05-30T17:07:58.558186+0000","last_clean_scrub_stamp":"2024-05-30T17:07:58.558186+0000","objects_scrubbed":70,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:14.569867+0000","scrub_duration":832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":70,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3b","version":"134'172","reported_seq":246,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:04.872850+0000","last_change":"2024-05-30T17:08:04.872957+0000","last_active":"2024-05-30T17:08:04.872850+0000","last_peered":"2024-05-30T17:08:04.872850+0000","last_clean":"2024-05-30T17:08:04.872850+0000","last_became_active":"2024-05-30T17:04:52.990018+0000","last_became_peered":"2024-05-30T17:04:52.990018+0000","last_unstale":"2024-05-30T17:08:04.872850+0000","last_undegraded":"2024-05-30T17:08:04.872850+0000","last_fullsized":"2024-05-30T17:08:04.872850+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'172","last_scrub_stamp":"2024-05-30T17:08:04.872606+0000","last_deep_scrub":"134'172","last_deep_scrub_stamp":"2024-05-30T17:08:04.872606+0000","last_clean_scrub_stamp":"2024-05-30T17:08:04.872606+0000","objects_scrubbed":61,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:28.966500+0000","scrub_duration":795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":45,"num_read_kb":184320,"num_write":61,"num_write_kb":249856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"134'225","reported_seq":342,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:00.494158+0000","last_change":"2024-05-30T17:08:00.494240+0000","last_active":"2024-05-30T17:08:00.494158+0000","last_peered":"2024-05-30T17:08:00.494158+0000","last_clean":"2024-05-30T17:08:00.494158+0000","last_became_active":"2024-05-30T17:04:52.991748+0000","last_became_peered":"2024-05-30T17:04:52.991748+0000","last_unstale":"2024-05-30T17:08:00.494158+0000","last_undegraded":"2024-05-30T17:08:00.494158+0000","last_fullsized":"2024-05-30T17:08:00.494158+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'225","last_scrub_stamp":"2024-05-30T17:08:00.494045+0000","last_deep_scrub":"134'225","last_deep_scrub_stamp":"2024-05-30T17:08:00.494045+0000","last_clean_scrub_stamp":"2024-05-30T17:08:00.494045+0000","objects_scrubbed":82,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:29:39.050108+0000","scrub_duration":1470,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":84,"num_write_kb":335873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"134'214","reported_seq":293,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:57.873460+0000","last_change":"2024-05-30T17:07:57.873572+0000","last_active":"2024-05-30T17:07:57.873460+0000","last_peered":"2024-05-30T17:07:57.873460+0000","last_clean":"2024-05-30T17:07:57.873460+0000","last_became_active":"2024-05-30T17:04:52.983960+0000","last_became_peered":"2024-05-30T17:04:52.983960+0000","last_unstale":"2024-05-30T17:07:57.873460+0000","last_undegraded":"2024-05-30T17:07:57.873460+0000","last_fullsized":"2024-05-30T17:07:57.873460+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'214","last_scrub_stamp":"2024-05-30T17:07:57.873335+0000","last_deep_scrub":"134'214","last_deep_scrub_stamp":"2024-05-30T17:07:57.873335+0000","last_clean_scrub_stamp":"2024-05-30T17:07:57.873335+0000","objects_scrubbed":74,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:20:43.997419+0000","scrub_duration":902,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":51,"num_read_kb":200706,"num_write":76,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"134'211","reported_seq":297,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:58.350215+0000","last_change":"2024-05-30T17:07:58.350307+0000","last_active":"2024-05-30T17:07:58.350215+0000","last_peered":"2024-05-30T17:07:58.350215+0000","last_clean":"2024-05-30T17:07:58.350215+0000","last_became_active":"2024-05-30T17:04:52.988277+0000","last_became_peered":"2024-05-30T17:04:52.988277+0000","last_unstale":"2024-05-30T17:07:58.350215+0000","last_undegraded":"2024-05-30T17:07:58.350215+0000","last_fullsized":"2024-05-30T17:07:58.350215+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'211","last_scrub_stamp":"2024-05-30T17:07:58.350074+0000","last_deep_scrub":"134'211","last_deep_scrub_stamp":"2024-05-30T17:07:58.350074+0000","last_clean_scrub_stamp":"2024-05-30T17:07:58.350074+0000","objects_scrubbed":85,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:49:18.689698+0000","scrub_duration":1029,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":85,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"134'219","reported_seq":308,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.981560+0000","last_change":"2024-05-30T17:08:01.981596+0000","last_active":"2024-05-30T17:08:01.981560+0000","last_peered":"2024-05-30T17:08:01.981560+0000","last_clean":"2024-05-30T17:08:01.981560+0000","last_became_active":"2024-05-30T17:04:52.990543+0000","last_became_peered":"2024-05-30T17:04:52.990543+0000","last_unstale":"2024-05-30T17:08:01.981560+0000","last_undegraded":"2024-05-30T17:08:01.981560+0000","last_fullsized":"2024-05-30T17:08:01.981560+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'219","last_scrub_stamp":"2024-05-30T17:08:01.981425+0000","last_deep_scrub":"134'219","last_deep_scrub_stamp":"2024-05-30T17:08:01.981425+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.981425+0000","objects_scrubbed":76,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:24.708121+0000","scrub_duration":1707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":59,"num_read_kb":233474,"num_write":78,"num_write_kb":311297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"134'208","reported_seq":303,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:56.182619+0000","last_change":"2024-05-30T17:07:56.182686+0000","last_active":"2024-05-30T17:07:56.182619+0000","last_peered":"2024-05-30T17:07:56.182619+0000","last_clean":"2024-05-30T17:07:56.182619+0000","last_became_active":"2024-05-30T17:04:52.979390+0000","last_became_peered":"2024-05-30T17:04:52.979390+0000","last_unstale":"2024-05-30T17:07:56.182619+0000","last_undegraded":"2024-05-30T17:07:56.182619+0000","last_fullsized":"2024-05-30T17:07:56.182619+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'208","last_scrub_stamp":"2024-05-30T17:07:56.182456+0000","last_deep_scrub":"134'208","last_deep_scrub_stamp":"2024-05-30T17:07:56.182456+0000","last_clean_scrub_stamp":"2024-05-30T17:07:56.182456+0000","objects_scrubbed":75,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:53:07.820041+0000","scrub_duration":946,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":65,"num_read_kb":266240,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"134'212","reported_seq":347,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:57.713662+0000","last_change":"2024-05-30T17:07:57.713733+0000","last_active":"2024-05-30T17:07:57.713662+0000","last_peered":"2024-05-30T17:07:57.713662+0000","last_clean":"2024-05-30T17:07:57.713662+0000","last_became_active":"2024-05-30T17:04:52.983250+0000","last_became_peered":"2024-05-30T17:04:52.983250+0000","last_unstale":"2024-05-30T17:07:57.713662+0000","last_undegraded":"2024-05-30T17:07:57.713662+0000","last_fullsized":"2024-05-30T17:07:57.713662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'212","last_scrub_stamp":"2024-05-30T17:07:57.713551+0000","last_deep_scrub":"134'212","last_deep_scrub_stamp":"2024-05-30T17:07:57.713551+0000","last_clean_scrub_stamp":"2024-05-30T17:07:57.713551+0000","objects_scrubbed":81,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:20:21.807445+0000","scrub_duration":1011,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":106,"num_read_kb":434176,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"134'189","reported_seq":322,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:02.548009+0000","last_change":"2024-05-30T17:08:02.548138+0000","last_active":"2024-05-30T17:08:02.548009+0000","last_peered":"2024-05-30T17:08:02.548009+0000","last_clean":"2024-05-30T17:08:02.548009+0000","last_became_active":"2024-05-30T17:04:52.993151+0000","last_became_peered":"2024-05-30T17:04:52.993151+0000","last_unstale":"2024-05-30T17:08:02.548009+0000","last_undegraded":"2024-05-30T17:08:02.548009+0000","last_fullsized":"2024-05-30T17:08:02.548009+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'189","last_scrub_stamp":"2024-05-30T17:08:02.547896+0000","last_deep_scrub":"134'189","last_deep_scrub_stamp":"2024-05-30T17:08:02.547896+0000","last_clean_scrub_stamp":"2024-05-30T17:08:02.547896+0000","objects_scrubbed":71,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:31.803525+0000","scrub_duration":1299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795584,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":105,"num_read_kb":413700,"num_write":73,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"134'212","reported_seq":325,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:59.534264+0000","last_change":"2024-05-30T17:07:59.534356+0000","last_active":"2024-05-30T17:07:59.534264+0000","last_peered":"2024-05-30T17:07:59.534264+0000","last_clean":"2024-05-30T17:07:59.534264+0000","last_became_active":"2024-05-30T17:04:52.997770+0000","last_became_peered":"2024-05-30T17:04:52.997770+0000","last_unstale":"2024-05-30T17:07:59.534264+0000","last_undegraded":"2024-05-30T17:07:59.534264+0000","last_fullsized":"2024-05-30T17:07:59.534264+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'212","last_scrub_stamp":"2024-05-30T17:07:59.534150+0000","last_deep_scrub":"134'212","last_deep_scrub_stamp":"2024-05-30T17:07:59.534150+0000","last_clean_scrub_stamp":"2024-05-30T17:07:59.534150+0000","objects_scrubbed":83,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:29.872097+0000","scrub_duration":1072,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":84,"num_read_kb":344064,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"134'219","reported_seq":311,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:04.388106+0000","last_change":"2024-05-30T17:08:04.388195+0000","last_active":"2024-05-30T17:08:04.388106+0000","last_peered":"2024-05-30T17:08:04.388106+0000","last_clean":"2024-05-30T17:08:04.388106+0000","last_became_active":"2024-05-30T17:04:52.992221+0000","last_became_peered":"2024-05-30T17:04:52.992221+0000","last_unstale":"2024-05-30T17:08:04.388106+0000","last_undegraded":"2024-05-30T17:08:04.388106+0000","last_fullsized":"2024-05-30T17:08:04.388106+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'219","last_scrub_stamp":"2024-05-30T17:08:04.387933+0000","last_deep_scrub":"134'219","last_deep_scrub_stamp":"2024-05-30T17:08:04.387933+0000","last_clean_scrub_stamp":"2024-05-30T17:08:04.387933+0000","objects_scrubbed":84,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:08:52.942615+0000","scrub_duration":977,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127364,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":84,"num_write_kb":339969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"134'198","reported_seq":258,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:56.323048+0000","last_change":"2024-05-30T17:07:56.323193+0000","last_active":"2024-05-30T17:07:56.323048+0000","last_peered":"2024-05-30T17:07:56.323048+0000","last_clean":"2024-05-30T17:07:56.323048+0000","last_became_active":"2024-05-30T17:04:52.991693+0000","last_became_peered":"2024-05-30T17:04:52.991693+0000","last_unstale":"2024-05-30T17:07:56.323048+0000","last_undegraded":"2024-05-30T17:07:56.323048+0000","last_fullsized":"2024-05-30T17:07:56.323048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'198","last_scrub_stamp":"2024-05-30T17:07:56.322959+0000","last_deep_scrub":"134'198","last_deep_scrub_stamp":"2024-05-30T17:07:56.322959+0000","last_clean_scrub_stamp":"2024-05-30T17:07:56.322959+0000","objects_scrubbed":72,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:53:07.960544+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":32,"num_read_kb":131072,"num_write":72,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"134'216","reported_seq":297,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:03.598529+0000","last_change":"2024-05-30T17:08:03.598596+0000","last_active":"2024-05-30T17:08:03.598529+0000","last_peered":"2024-05-30T17:08:03.598529+0000","last_clean":"2024-05-30T17:08:03.598529+0000","last_became_active":"2024-05-30T17:04:52.990334+0000","last_became_peered":"2024-05-30T17:04:52.990334+0000","last_unstale":"2024-05-30T17:08:03.598529+0000","last_undegraded":"2024-05-30T17:08:03.598529+0000","last_fullsized":"2024-05-30T17:08:03.598529+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'216","last_scrub_stamp":"2024-05-30T17:08:03.598441+0000","last_deep_scrub":"134'216","last_deep_scrub_stamp":"2024-05-30T17:08:03.598441+0000","last_clean_scrub_stamp":"2024-05-30T17:08:03.598441+0000","objects_scrubbed":80,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:57:50.004411+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"134'193","reported_seq":264,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:53.876582+0000","last_change":"2024-05-30T17:07:53.876742+0000","last_active":"2024-05-30T17:07:53.876582+0000","last_peered":"2024-05-30T17:07:53.876582+0000","last_clean":"2024-05-30T17:07:53.876582+0000","last_became_active":"2024-05-30T17:04:52.966026+0000","last_became_peered":"2024-05-30T17:04:52.966026+0000","last_unstale":"2024-05-30T17:07:53.876582+0000","last_undegraded":"2024-05-30T17:07:53.876582+0000","last_fullsized":"2024-05-30T17:07:53.876582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'193","last_scrub_stamp":"2024-05-30T17:07:53.876381+0000","last_deep_scrub":"134'193","last_deep_scrub_stamp":"2024-05-30T17:07:53.876381+0000","last_clean_scrub_stamp":"2024-05-30T17:07:53.876381+0000","objects_scrubbed":73,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:21:16.603077+0000","scrub_duration":860,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"134'184","reported_seq":303,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:58.487441+0000","last_change":"2024-05-30T17:07:58.487563+0000","last_active":"2024-05-30T17:07:58.487441+0000","last_peered":"2024-05-30T17:07:58.487441+0000","last_clean":"2024-05-30T17:07:58.487441+0000","last_became_active":"2024-05-30T17:04:52.985602+0000","last_became_peered":"2024-05-30T17:04:52.985602+0000","last_unstale":"2024-05-30T17:07:58.487441+0000","last_undegraded":"2024-05-30T17:07:58.487441+0000","last_fullsized":"2024-05-30T17:07:58.487441+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'184","last_scrub_stamp":"2024-05-30T17:07:58.487318+0000","last_deep_scrub":"134'184","last_deep_scrub_stamp":"2024-05-30T17:07:58.487318+0000","last_clean_scrub_stamp":"2024-05-30T17:07:58.487318+0000","objects_scrubbed":69,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:54:29.601009+0000","scrub_duration":822,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":91,"num_read_kb":372736,"num_write":71,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"134'223","reported_seq":324,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:54.787151+0000","last_change":"2024-05-30T17:07:54.787239+0000","last_active":"2024-05-30T17:07:54.787151+0000","last_peered":"2024-05-30T17:07:54.787151+0000","last_clean":"2024-05-30T17:07:54.787151+0000","last_became_active":"2024-05-30T17:04:52.962582+0000","last_became_peered":"2024-05-30T17:04:52.962582+0000","last_unstale":"2024-05-30T17:07:54.787151+0000","last_undegraded":"2024-05-30T17:07:54.787151+0000","last_fullsized":"2024-05-30T17:07:54.787151+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'223","last_scrub_stamp":"2024-05-30T17:07:54.787023+0000","last_deep_scrub":"134'223","last_deep_scrub_stamp":"2024-05-30T17:07:54.787023+0000","last_clean_scrub_stamp":"2024-05-30T17:07:54.787023+0000","objects_scrubbed":88,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:46.669165+0000","scrub_duration":1105,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369098752,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":90,"num_write_kb":360449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"134'235","reported_seq":326,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:59.714951+0000","last_change":"2024-05-30T17:07:59.715035+0000","last_active":"2024-05-30T17:07:59.714951+0000","last_peered":"2024-05-30T17:07:59.714951+0000","last_clean":"2024-05-30T17:07:59.714951+0000","last_became_active":"2024-05-30T17:04:52.986701+0000","last_became_peered":"2024-05-30T17:04:52.986701+0000","last_unstale":"2024-05-30T17:07:59.714951+0000","last_undegraded":"2024-05-30T17:07:59.714951+0000","last_fullsized":"2024-05-30T17:07:59.714951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'235","last_scrub_stamp":"2024-05-30T17:07:59.714831+0000","last_deep_scrub":"134'235","last_deep_scrub_stamp":"2024-05-30T17:07:59.714831+0000","last_clean_scrub_stamp":"2024-05-30T17:07:59.714831+0000","objects_scrubbed":84,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:07:53.555218+0000","scrub_duration":1071,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352321536,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":62,"num_read_kb":249857,"num_write":86,"num_write_kb":344065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"134'241","reported_seq":366,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:02.678991+0000","last_change":"2024-05-30T17:08:02.679061+0000","last_active":"2024-05-30T17:08:02.678991+0000","last_peered":"2024-05-30T17:08:02.678991+0000","last_clean":"2024-05-30T17:08:02.678991+0000","last_became_active":"2024-05-30T17:04:52.984462+0000","last_became_peered":"2024-05-30T17:04:52.984462+0000","last_unstale":"2024-05-30T17:08:02.678991+0000","last_undegraded":"2024-05-30T17:08:02.678991+0000","last_fullsized":"2024-05-30T17:08:02.678991+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'241","last_scrub_stamp":"2024-05-30T17:08:02.678911+0000","last_deep_scrub":"134'241","last_deep_scrub_stamp":"2024-05-30T17:08:02.678911+0000","last_clean_scrub_stamp":"2024-05-30T17:08:02.678911+0000","objects_scrubbed":89,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:24:54.561053+0000","scrub_duration":1640,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293056,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":96,"num_read_kb":372741,"num_write":95,"num_write_kb":364547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"134'212","reported_seq":302,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:56.757871+0000","last_change":"2024-05-30T17:07:56.757920+0000","last_active":"2024-05-30T17:07:56.757871+0000","last_peered":"2024-05-30T17:07:56.757871+0000","last_clean":"2024-05-30T17:07:56.757871+0000","last_became_active":"2024-05-30T17:04:52.978968+0000","last_became_peered":"2024-05-30T17:04:52.978968+0000","last_unstale":"2024-05-30T17:07:56.757871+0000","last_undegraded":"2024-05-30T17:07:56.757871+0000","last_fullsized":"2024-05-30T17:07:56.757871+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'212","last_scrub_stamp":"2024-05-30T17:07:56.757708+0000","last_deep_scrub":"134'212","last_deep_scrub_stamp":"2024-05-30T17:07:56.757708+0000","last_clean_scrub_stamp":"2024-05-30T17:07:56.757708+0000","objects_scrubbed":80,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:50:06.781685+0000","scrub_duration":1050,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":61,"num_read_kb":245761,"num_write":82,"num_write_kb":327681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"134'227","reported_seq":322,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:00.785961+0000","last_change":"2024-05-30T17:08:00.786034+0000","last_active":"2024-05-30T17:08:00.785961+0000","last_peered":"2024-05-30T17:08:00.785961+0000","last_clean":"2024-05-30T17:08:00.785961+0000","last_became_active":"2024-05-30T17:04:52.991966+0000","last_became_peered":"2024-05-30T17:04:52.991966+0000","last_unstale":"2024-05-30T17:08:00.785961+0000","last_undegraded":"2024-05-30T17:08:00.785961+0000","last_fullsized":"2024-05-30T17:08:00.785961+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'227","last_scrub_stamp":"2024-05-30T17:08:00.785864+0000","last_deep_scrub":"134'227","last_deep_scrub_stamp":"2024-05-30T17:08:00.785864+0000","last_clean_scrub_stamp":"2024-05-30T17:08:00.785864+0000","objects_scrubbed":85,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:24.284436+0000","scrub_duration":1531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":65,"num_read_kb":253955,"num_write":87,"num_write_kb":348161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"134'209","reported_seq":295,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:55.616094+0000","last_change":"2024-05-30T17:07:55.616190+0000","last_active":"2024-05-30T17:07:55.616094+0000","last_peered":"2024-05-30T17:07:55.616094+0000","last_clean":"2024-05-30T17:07:55.616094+0000","last_became_active":"2024-05-30T17:04:52.965116+0000","last_became_peered":"2024-05-30T17:04:52.965116+0000","last_unstale":"2024-05-30T17:07:55.616094+0000","last_undegraded":"2024-05-30T17:07:55.616094+0000","last_fullsized":"2024-05-30T17:07:55.616094+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'209","last_scrub_stamp":"2024-05-30T17:07:55.615990+0000","last_deep_scrub":"134'209","last_deep_scrub_stamp":"2024-05-30T17:07:55.615990+0000","last_clean_scrub_stamp":"2024-05-30T17:07:55.615990+0000","objects_scrubbed":73,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:11:32.516403+0000","scrub_duration":952,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":57,"num_read_kb":229377,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"134'217","reported_seq":312,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:58.209369+0000","last_change":"2024-05-30T17:07:58.209430+0000","last_active":"2024-05-30T17:07:58.209369+0000","last_peered":"2024-05-30T17:07:58.209369+0000","last_clean":"2024-05-30T17:07:58.209369+0000","last_became_active":"2024-05-30T17:04:52.981867+0000","last_became_peered":"2024-05-30T17:04:52.981867+0000","last_unstale":"2024-05-30T17:07:58.209369+0000","last_undegraded":"2024-05-30T17:07:58.209369+0000","last_fullsized":"2024-05-30T17:07:58.209369+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'217","last_scrub_stamp":"2024-05-30T17:07:58.209265+0000","last_deep_scrub":"134'217","last_deep_scrub_stamp":"2024-05-30T17:07:58.209265+0000","last_clean_scrub_stamp":"2024-05-30T17:07:58.209265+0000","objects_scrubbed":78,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:28:39.181620+0000","scrub_duration":939,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":66,"num_read_kb":266241,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"134'208","reported_seq":306,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:02.487564+0000","last_change":"2024-05-30T17:08:02.487682+0000","last_active":"2024-05-30T17:08:02.487564+0000","last_peered":"2024-05-30T17:08:02.487564+0000","last_clean":"2024-05-30T17:08:02.487564+0000","last_became_active":"2024-05-30T17:04:52.989141+0000","last_became_peered":"2024-05-30T17:04:52.989141+0000","last_unstale":"2024-05-30T17:08:02.487564+0000","last_undegraded":"2024-05-30T17:08:02.487564+0000","last_fullsized":"2024-05-30T17:08:02.487564+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'208","last_scrub_stamp":"2024-05-30T17:08:02.487415+0000","last_deep_scrub":"134'208","last_deep_scrub_stamp":"2024-05-30T17:08:02.487415+0000","last_clean_scrub_stamp":"2024-05-30T17:08:02.487415+0000","objects_scrubbed":67,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:11:52.233017+0000","scrub_duration":923,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"134'224","reported_seq":324,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:55.555378+0000","last_change":"2024-05-30T17:07:55.555444+0000","last_active":"2024-05-30T17:07:55.555378+0000","last_peered":"2024-05-30T17:07:55.555378+0000","last_clean":"2024-05-30T17:07:55.555378+0000","last_became_active":"2024-05-30T17:04:52.988618+0000","last_became_peered":"2024-05-30T17:04:52.988618+0000","last_unstale":"2024-05-30T17:07:55.555378+0000","last_undegraded":"2024-05-30T17:07:55.555378+0000","last_fullsized":"2024-05-30T17:07:55.555378+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'224","last_scrub_stamp":"2024-05-30T17:07:55.555294+0000","last_deep_scrub":"134'224","last_deep_scrub_stamp":"2024-05-30T17:07:55.555294+0000","last_clean_scrub_stamp":"2024-05-30T17:07:55.555294+0000","objects_scrubbed":90,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:43:36.624069+0000","scrub_duration":1167,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377487360,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":90,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"134'188","reported_seq":249,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.042319+0000","last_change":"2024-05-30T17:08:01.042428+0000","last_active":"2024-05-30T17:08:01.042319+0000","last_peered":"2024-05-30T17:08:01.042319+0000","last_clean":"2024-05-30T17:08:01.042319+0000","last_became_active":"2024-05-30T17:04:52.977378+0000","last_became_peered":"2024-05-30T17:04:52.977378+0000","last_unstale":"2024-05-30T17:08:01.042319+0000","last_undegraded":"2024-05-30T17:08:01.042319+0000","last_fullsized":"2024-05-30T17:08:01.042319+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'188","last_scrub_stamp":"2024-05-30T17:08:01.042232+0000","last_deep_scrub":"134'188","last_deep_scrub_stamp":"2024-05-30T17:08:01.042232+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.042232+0000","objects_scrubbed":73,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:37:33.550762+0000","scrub_duration":1434,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":75,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"134'240","reported_seq":347,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.481498+0000","last_change":"2024-05-30T17:08:01.481669+0000","last_active":"2024-05-30T17:08:01.481498+0000","last_peered":"2024-05-30T17:08:01.481498+0000","last_clean":"2024-05-30T17:08:01.481498+0000","last_became_active":"2024-05-30T17:04:52.997323+0000","last_became_peered":"2024-05-30T17:04:52.997323+0000","last_unstale":"2024-05-30T17:08:01.481498+0000","last_undegraded":"2024-05-30T17:08:01.481498+0000","last_fullsized":"2024-05-30T17:08:01.481498+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'240","last_scrub_stamp":"2024-05-30T17:08:01.481341+0000","last_deep_scrub":"134'240","last_deep_scrub_stamp":"2024-05-30T17:08:01.481341+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.481341+0000","objects_scrubbed":93,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:07:40.333355+0000","scrub_duration":1971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390070272,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":78,"num_read_kb":307203,"num_write":95,"num_write_kb":380929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"134'194","reported_seq":286,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:58.316435+0000","last_change":"2024-05-30T17:07:58.316565+0000","last_active":"2024-05-30T17:07:58.316435+0000","last_peered":"2024-05-30T17:07:58.316435+0000","last_clean":"2024-05-30T17:07:58.316435+0000","last_became_active":"2024-05-30T17:04:52.992733+0000","last_became_peered":"2024-05-30T17:04:52.992733+0000","last_unstale":"2024-05-30T17:07:58.316435+0000","last_undegraded":"2024-05-30T17:07:58.316435+0000","last_fullsized":"2024-05-30T17:07:58.316435+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'194","last_scrub_stamp":"2024-05-30T17:07:58.316311+0000","last_deep_scrub":"134'194","last_deep_scrub_stamp":"2024-05-30T17:07:58.316311+0000","last_clean_scrub_stamp":"2024-05-30T17:07:58.316311+0000","objects_scrubbed":73,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:41:07.332307+0000","scrub_duration":881,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306184192,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":73,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"134'234","reported_seq":363,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:07.029231+0000","last_change":"2024-05-30T17:08:07.029317+0000","last_active":"2024-05-30T17:08:07.029231+0000","last_peered":"2024-05-30T17:08:07.029231+0000","last_clean":"2024-05-30T17:08:07.029231+0000","last_became_active":"2024-05-30T17:04:52.993214+0000","last_became_peered":"2024-05-30T17:04:52.993214+0000","last_unstale":"2024-05-30T17:08:07.029231+0000","last_undegraded":"2024-05-30T17:08:07.029231+0000","last_fullsized":"2024-05-30T17:08:07.029231+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'234","last_scrub_stamp":"2024-05-30T17:08:07.029030+0000","last_deep_scrub":"134'234","last_deep_scrub_stamp":"2024-05-30T17:08:07.029030+0000","last_clean_scrub_stamp":"2024-05-30T17:08:07.029030+0000","objects_scrubbed":79,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:38:56.138608+0000","scrub_duration":899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":100,"num_read_kb":409600,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"134'220","reported_seq":324,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:57.015069+0000","last_change":"2024-05-30T17:07:57.015115+0000","last_active":"2024-05-30T17:07:57.015069+0000","last_peered":"2024-05-30T17:07:57.015069+0000","last_clean":"2024-05-30T17:07:57.015069+0000","last_became_active":"2024-05-30T17:04:52.979697+0000","last_became_peered":"2024-05-30T17:04:52.979697+0000","last_unstale":"2024-05-30T17:07:57.015069+0000","last_undegraded":"2024-05-30T17:07:57.015069+0000","last_fullsized":"2024-05-30T17:07:57.015069+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'220","last_scrub_stamp":"2024-05-30T17:07:57.014964+0000","last_deep_scrub":"134'220","last_deep_scrub_stamp":"2024-05-30T17:07:57.014964+0000","last_clean_scrub_stamp":"2024-05-30T17:07:57.014964+0000","objects_scrubbed":77,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:39:12.126701+0000","scrub_duration":1015,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":77,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"134'193","reported_seq":266,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:56.577661+0000","last_change":"2024-05-30T17:07:56.577790+0000","last_active":"2024-05-30T17:07:56.577661+0000","last_peered":"2024-05-30T17:07:56.577661+0000","last_clean":"2024-05-30T17:07:56.577661+0000","last_became_active":"2024-05-30T17:04:52.978065+0000","last_became_peered":"2024-05-30T17:04:52.978065+0000","last_unstale":"2024-05-30T17:07:56.577661+0000","last_undegraded":"2024-05-30T17:07:56.577661+0000","last_fullsized":"2024-05-30T17:07:56.577661+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'193","last_scrub_stamp":"2024-05-30T17:07:56.577578+0000","last_deep_scrub":"134'193","last_deep_scrub_stamp":"2024-05-30T17:07:56.577578+0000","last_clean_scrub_stamp":"2024-05-30T17:07:56.577578+0000","objects_scrubbed":70,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:43:27.435902+0000","scrub_duration":886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289407237,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":44,"num_read_kb":180224,"num_write":70,"num_write_kb":282625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"134'6","reported_seq":34,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:54.996576+0000","last_change":"2024-05-30T17:07:54.996645+0000","last_active":"2024-05-30T17:07:54.996576+0000","last_peered":"2024-05-30T17:07:54.996576+0000","last_clean":"2024-05-30T17:07:54.996576+0000","last_became_active":"2024-05-30T17:04:51.943796+0000","last_became_peered":"2024-05-30T17:04:51.943796+0000","last_unstale":"2024-05-30T17:07:54.996576+0000","last_undegraded":"2024-05-30T17:07:54.996576+0000","last_fullsized":"2024-05-30T17:07:54.996576+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'6","last_scrub_stamp":"2024-05-30T17:07:54.996464+0000","last_deep_scrub":"134'6","last_deep_scrub_stamp":"2024-05-30T17:07:54.996464+0000","last_clean_scrub_stamp":"2024-05-30T17:07:54.996464+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:05:51.978785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"134'216","reported_seq":315,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:03.327542+0000","last_change":"2024-05-30T17:08:03.327592+0000","last_active":"2024-05-30T17:08:03.327542+0000","last_peered":"2024-05-30T17:08:03.327542+0000","last_clean":"2024-05-30T17:08:03.327542+0000","last_became_active":"2024-05-30T17:04:52.998673+0000","last_became_peered":"2024-05-30T17:04:52.998673+0000","last_unstale":"2024-05-30T17:08:03.327542+0000","last_undegraded":"2024-05-30T17:08:03.327542+0000","last_fullsized":"2024-05-30T17:08:03.327542+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'216","last_scrub_stamp":"2024-05-30T17:08:03.327438+0000","last_deep_scrub":"134'216","last_deep_scrub_stamp":"2024-05-30T17:08:03.327438+0000","last_clean_scrub_stamp":"2024-05-30T17:08:03.327438+0000","objects_scrubbed":81,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:41:52.255591+0000","scrub_duration":1109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738624,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":70,"num_read_kb":286720,"num_write":81,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:53.551733+0000","last_change":"2024-05-30T17:07:53.551833+0000","last_active":"2024-05-30T17:07:53.551733+0000","last_peered":"2024-05-30T17:07:53.551733+0000","last_clean":"2024-05-30T17:07:53.551733+0000","last_became_active":"2024-05-30T17:04:51.937127+0000","last_became_peered":"2024-05-30T17:04:51.937127+0000","last_unstale":"2024-05-30T17:07:53.551733+0000","last_undegraded":"2024-05-30T17:07:53.551733+0000","last_fullsized":"2024-05-30T17:07:53.551733+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'1","last_scrub_stamp":"2024-05-30T17:07:53.551451+0000","last_deep_scrub":"131'1","last_deep_scrub_stamp":"2024-05-30T17:07:53.551451+0000","last_clean_scrub_stamp":"2024-05-30T17:07:53.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:03:46.080841+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"134'223","reported_seq":326,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.682919+0000","last_change":"2024-05-30T17:08:01.683005+0000","last_active":"2024-05-30T17:08:01.682919+0000","last_peered":"2024-05-30T17:08:01.682919+0000","last_clean":"2024-05-30T17:08:01.682919+0000","last_became_active":"2024-05-30T17:04:52.981316+0000","last_became_peered":"2024-05-30T17:04:52.981316+0000","last_unstale":"2024-05-30T17:08:01.682919+0000","last_undegraded":"2024-05-30T17:08:01.682919+0000","last_fullsized":"2024-05-30T17:08:01.682919+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'223","last_scrub_stamp":"2024-05-30T17:08:01.682761+0000","last_deep_scrub":"134'223","last_deep_scrub_stamp":"2024-05-30T17:08:01.682761+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.682761+0000","objects_scrubbed":82,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:32:26.500703+0000","scrub_duration":1678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339738885,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":74,"num_read_kb":303104,"num_write":82,"num_write_kb":331777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"134'4","reported_seq":32,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:49.733186+0000","last_change":"2024-05-30T17:07:49.733211+0000","last_active":"2024-05-30T17:07:49.733186+0000","last_peered":"2024-05-30T17:07:49.733186+0000","last_clean":"2024-05-30T17:07:49.733186+0000","last_became_active":"2024-05-30T17:04:51.938644+0000","last_became_peered":"2024-05-30T17:04:51.938644+0000","last_unstale":"2024-05-30T17:07:49.733186+0000","last_undegraded":"2024-05-30T17:07:49.733186+0000","last_fullsized":"2024-05-30T17:07:49.733186+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'4","last_scrub_stamp":"2024-05-30T17:07:49.733110+0000","last_deep_scrub":"134'4","last_deep_scrub_stamp":"2024-05-30T17:07:49.733110+0000","last_clean_scrub_stamp":"2024-05-30T17:07:49.733110+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:40:06.973624+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"134'235","reported_seq":352,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:03.246701+0000","last_change":"2024-05-30T17:08:03.246761+0000","last_active":"2024-05-30T17:08:03.246701+0000","last_peered":"2024-05-30T17:08:03.246701+0000","last_clean":"2024-05-30T17:08:03.246701+0000","last_became_active":"2024-05-30T17:04:52.987961+0000","last_became_peered":"2024-05-30T17:04:52.987961+0000","last_unstale":"2024-05-30T17:08:03.246701+0000","last_undegraded":"2024-05-30T17:08:03.246701+0000","last_fullsized":"2024-05-30T17:08:03.246701+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'235","last_scrub_stamp":"2024-05-30T17:08:03.246604+0000","last_deep_scrub":"134'235","last_deep_scrub_stamp":"2024-05-30T17:08:03.246604+0000","last_clean_scrub_stamp":"2024-05-30T17:08:03.246604+0000","objects_scrubbed":87,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:19:07.784610+0000","scrub_duration":1224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364904448,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":87,"num_write_kb":356352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"134'3","reported_seq":31,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:49.130833+0000","last_change":"2024-05-30T17:07:49.130892+0000","last_active":"2024-05-30T17:07:49.130833+0000","last_peered":"2024-05-30T17:07:49.130833+0000","last_clean":"2024-05-30T17:07:49.130833+0000","last_became_active":"2024-05-30T17:04:51.929298+0000","last_became_peered":"2024-05-30T17:04:51.929298+0000","last_unstale":"2024-05-30T17:07:49.130833+0000","last_undegraded":"2024-05-30T17:07:49.130833+0000","last_fullsized":"2024-05-30T17:07:49.130833+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'3","last_scrub_stamp":"2024-05-30T17:07:49.130712+0000","last_deep_scrub":"134'3","last_deep_scrub_stamp":"2024-05-30T17:07:49.130712+0000","last_clean_scrub_stamp":"2024-05-30T17:07:49.130712+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:13:13.067287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"134'209","reported_seq":298,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:57.580789+0000","last_change":"2024-05-30T17:07:57.580828+0000","last_active":"2024-05-30T17:07:57.580789+0000","last_peered":"2024-05-30T17:07:57.580789+0000","last_clean":"2024-05-30T17:07:57.580789+0000","last_became_active":"2024-05-30T17:04:52.985811+0000","last_became_peered":"2024-05-30T17:04:52.985811+0000","last_unstale":"2024-05-30T17:07:57.580789+0000","last_undegraded":"2024-05-30T17:07:57.580789+0000","last_fullsized":"2024-05-30T17:07:57.580789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'209","last_scrub_stamp":"2024-05-30T17:07:57.580680+0000","last_deep_scrub":"134'209","last_deep_scrub_stamp":"2024-05-30T17:07:57.580680+0000","last_clean_scrub_stamp":"2024-05-30T17:07:57.580680+0000","objects_scrubbed":75,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:40:26.740918+0000","scrub_duration":929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":75,"num_write_kb":307200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:50.175063+0000","last_change":"2024-05-30T17:07:50.175119+0000","last_active":"2024-05-30T17:07:50.175063+0000","last_peered":"2024-05-30T17:07:50.175063+0000","last_clean":"2024-05-30T17:07:50.175063+0000","last_became_active":"2024-05-30T17:04:51.942597+0000","last_became_peered":"2024-05-30T17:04:51.942597+0000","last_unstale":"2024-05-30T17:07:50.175063+0000","last_undegraded":"2024-05-30T17:07:50.175063+0000","last_fullsized":"2024-05-30T17:07:50.175063+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'1","last_scrub_stamp":"2024-05-30T17:07:50.174913+0000","last_deep_scrub":"131'1","last_deep_scrub_stamp":"2024-05-30T17:07:50.174913+0000","last_clean_scrub_stamp":"2024-05-30T17:07:50.174913+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:18:28.797745+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"134'209","reported_seq":331,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:56.018787+0000","last_change":"2024-05-30T17:07:56.018831+0000","last_active":"2024-05-30T17:07:56.018787+0000","last_peered":"2024-05-30T17:07:56.018787+0000","last_clean":"2024-05-30T17:07:56.018787+0000","last_became_active":"2024-05-30T17:04:52.967033+0000","last_became_peered":"2024-05-30T17:04:52.967033+0000","last_unstale":"2024-05-30T17:07:56.018787+0000","last_undegraded":"2024-05-30T17:07:56.018787+0000","last_fullsized":"2024-05-30T17:07:56.018787+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'209","last_scrub_stamp":"2024-05-30T17:07:56.018715+0000","last_deep_scrub":"134'209","last_deep_scrub_stamp":"2024-05-30T17:07:56.018715+0000","last_clean_scrub_stamp":"2024-05-30T17:07:56.018715+0000","objects_scrubbed":78,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:34.290339+0000","scrub_duration":998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:50.646754+0000","last_change":"2024-05-30T17:07:50.646862+0000","last_active":"2024-05-30T17:07:50.646754+0000","last_peered":"2024-05-30T17:07:50.646754+0000","last_clean":"2024-05-30T17:07:50.646754+0000","last_became_active":"2024-05-30T17:04:51.941974+0000","last_became_peered":"2024-05-30T17:04:51.941974+0000","last_unstale":"2024-05-30T17:07:50.646754+0000","last_undegraded":"2024-05-30T17:07:50.646754+0000","last_fullsized":"2024-05-30T17:07:50.646754+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'1","last_scrub_stamp":"2024-05-30T17:07:50.646550+0000","last_deep_scrub":"131'1","last_deep_scrub_stamp":"2024-05-30T17:07:50.646550+0000","last_clean_scrub_stamp":"2024-05-30T17:07:50.646550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:04:41.381411+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"134'220","reported_seq":325,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:59.219168+0000","last_change":"2024-05-30T17:07:59.219207+0000","last_active":"2024-05-30T17:07:59.219168+0000","last_peered":"2024-05-30T17:07:59.219168+0000","last_clean":"2024-05-30T17:07:59.219168+0000","last_became_active":"2024-05-30T17:04:52.988645+0000","last_became_peered":"2024-05-30T17:04:52.988645+0000","last_unstale":"2024-05-30T17:07:59.219168+0000","last_undegraded":"2024-05-30T17:07:59.219168+0000","last_fullsized":"2024-05-30T17:07:59.219168+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'220","last_scrub_stamp":"2024-05-30T17:07:59.219066+0000","last_deep_scrub":"134'220","last_deep_scrub_stamp":"2024-05-30T17:07:59.219066+0000","last_clean_scrub_stamp":"2024-05-30T17:07:59.219066+0000","objects_scrubbed":77,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:19.010549+0000","scrub_duration":942,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":76,"num_read_kb":303106,"num_write":79,"num_write_kb":315393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:52.033567+0000","last_change":"2024-05-30T17:07:52.033626+0000","last_active":"2024-05-30T17:07:52.033567+0000","last_peered":"2024-05-30T17:07:52.033567+0000","last_clean":"2024-05-30T17:07:52.033567+0000","last_became_active":"2024-05-30T17:04:51.951223+0000","last_became_peered":"2024-05-30T17:04:51.951223+0000","last_unstale":"2024-05-30T17:07:52.033567+0000","last_undegraded":"2024-05-30T17:07:52.033567+0000","last_fullsized":"2024-05-30T17:07:52.033567+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:52.033369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:52.033369+0000","last_clean_scrub_stamp":"2024-05-30T17:07:52.033369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:25:15.531940+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"134'193","reported_seq":289,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.151899+0000","last_change":"2024-05-30T17:08:01.151984+0000","last_active":"2024-05-30T17:08:01.151899+0000","last_peered":"2024-05-30T17:08:01.151899+0000","last_clean":"2024-05-30T17:08:01.151899+0000","last_became_active":"2024-05-30T17:04:52.986164+0000","last_became_peered":"2024-05-30T17:04:52.986164+0000","last_unstale":"2024-05-30T17:08:01.151899+0000","last_undegraded":"2024-05-30T17:08:01.151899+0000","last_fullsized":"2024-05-30T17:08:01.151899+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'193","last_scrub_stamp":"2024-05-30T17:08:01.151761+0000","last_deep_scrub":"134'193","last_deep_scrub_stamp":"2024-05-30T17:08:01.151761+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.151761+0000","objects_scrubbed":62,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:10:43.589420+0000","scrub_duration":854,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":67,"num_read_kb":258052,"num_write":66,"num_write_kb":253954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:54.194706+0000","last_change":"2024-05-30T17:07:54.194846+0000","last_active":"2024-05-30T17:07:54.194706+0000","last_peered":"2024-05-30T17:07:54.194706+0000","last_clean":"2024-05-30T17:07:54.194706+0000","last_became_active":"2024-05-30T17:04:51.955666+0000","last_became_peered":"2024-05-30T17:04:51.955666+0000","last_unstale":"2024-05-30T17:07:54.194706+0000","last_undegraded":"2024-05-30T17:07:54.194706+0000","last_fullsized":"2024-05-30T17:07:54.194706+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:54.194399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:54.194399+0000","last_clean_scrub_stamp":"2024-05-30T17:07:54.194399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:43:35.263174+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"134'200","reported_seq":267,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:52.955654+0000","last_change":"2024-05-30T17:07:52.955733+0000","last_active":"2024-05-30T17:07:52.955654+0000","last_peered":"2024-05-30T17:07:52.955654+0000","last_clean":"2024-05-30T17:07:52.955654+0000","last_became_active":"2024-05-30T17:04:52.984409+0000","last_became_peered":"2024-05-30T17:04:52.984409+0000","last_unstale":"2024-05-30T17:07:52.955654+0000","last_undegraded":"2024-05-30T17:07:52.955654+0000","last_fullsized":"2024-05-30T17:07:52.955654+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'200","last_scrub_stamp":"2024-05-30T17:07:52.955576+0000","last_deep_scrub":"134'200","last_deep_scrub_stamp":"2024-05-30T17:07:52.955576+0000","last_clean_scrub_stamp":"2024-05-30T17:07:52.955576+0000","objects_scrubbed":67,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:01:52.947220+0000","scrub_duration":611,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":39,"num_read_kb":159744,"num_write":67,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"134'1","reported_seq":29,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:57.661232+0000","last_change":"2024-05-30T17:07:57.661257+0000","last_active":"2024-05-30T17:07:57.661232+0000","last_peered":"2024-05-30T17:07:57.661232+0000","last_clean":"2024-05-30T17:07:57.661232+0000","last_became_active":"2024-05-30T17:04:51.948281+0000","last_became_peered":"2024-05-30T17:04:51.948281+0000","last_unstale":"2024-05-30T17:07:57.661232+0000","last_undegraded":"2024-05-30T17:07:57.661232+0000","last_fullsized":"2024-05-30T17:07:57.661232+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'1","last_scrub_stamp":"2024-05-30T17:07:57.661140+0000","last_deep_scrub":"134'1","last_deep_scrub_stamp":"2024-05-30T17:07:57.661140+0000","last_clean_scrub_stamp":"2024-05-30T17:07:57.661140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:13:01.470379+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"134'175","reported_seq":248,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.896526+0000","last_change":"2024-05-30T17:08:01.896609+0000","last_active":"2024-05-30T17:08:01.896526+0000","last_peered":"2024-05-30T17:08:01.896526+0000","last_clean":"2024-05-30T17:08:01.896526+0000","last_became_active":"2024-05-30T17:04:52.985889+0000","last_became_peered":"2024-05-30T17:04:52.985889+0000","last_unstale":"2024-05-30T17:08:01.896526+0000","last_undegraded":"2024-05-30T17:08:01.896526+0000","last_fullsized":"2024-05-30T17:08:01.896526+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'175","last_scrub_stamp":"2024-05-30T17:08:01.896385+0000","last_deep_scrub":"134'175","last_deep_scrub_stamp":"2024-05-30T17:08:01.896385+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.896385+0000","objects_scrubbed":57,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:48:53.681609+0000","scrub_duration":1241,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881146,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":59,"num_write_kb":229378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:54.694897+0000","last_change":"2024-05-30T17:07:54.695009+0000","last_active":"2024-05-30T17:07:54.694897+0000","last_peered":"2024-05-30T17:07:54.694897+0000","last_clean":"2024-05-30T17:07:54.694897+0000","last_became_active":"2024-05-30T17:04:51.954434+0000","last_became_peered":"2024-05-30T17:04:51.954434+0000","last_unstale":"2024-05-30T17:07:54.694897+0000","last_undegraded":"2024-05-30T17:07:54.694897+0000","last_fullsized":"2024-05-30T17:07:54.694897+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:54.694555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:54.694555+0000","last_clean_scrub_stamp":"2024-05-30T17:07:54.694555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:02:05.213234+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"134'202","reported_seq":311,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:04.059770+0000","last_change":"2024-05-30T17:08:04.059840+0000","last_active":"2024-05-30T17:08:04.059770+0000","last_peered":"2024-05-30T17:08:04.059770+0000","last_clean":"2024-05-30T17:08:04.059770+0000","last_became_active":"2024-05-30T17:04:52.989051+0000","last_became_peered":"2024-05-30T17:04:52.989051+0000","last_unstale":"2024-05-30T17:08:04.059770+0000","last_undegraded":"2024-05-30T17:08:04.059770+0000","last_fullsized":"2024-05-30T17:08:04.059770+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'202","last_scrub_stamp":"2024-05-30T17:08:04.059604+0000","last_deep_scrub":"134'202","last_deep_scrub_stamp":"2024-05-30T17:08:04.059604+0000","last_clean_scrub_stamp":"2024-05-30T17:08:04.059604+0000","objects_scrubbed":78,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:57:42.025083+0000","scrub_duration":988,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":78,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"134'30","reported_seq":58,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:52.601868+0000","last_change":"2024-05-30T17:07:52.601937+0000","last_active":"2024-05-30T17:07:52.601868+0000","last_peered":"2024-05-30T17:07:52.601868+0000","last_clean":"2024-05-30T17:07:52.601868+0000","last_became_active":"2024-05-30T17:04:51.937857+0000","last_became_peered":"2024-05-30T17:04:51.937857+0000","last_unstale":"2024-05-30T17:07:52.601868+0000","last_undegraded":"2024-05-30T17:07:52.601868+0000","last_fullsized":"2024-05-30T17:07:52.601868+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'30","last_scrub_stamp":"2024-05-30T17:07:52.601679+0000","last_deep_scrub":"134'30","last_deep_scrub_stamp":"2024-05-30T17:07:52.601679+0000","last_clean_scrub_stamp":"2024-05-30T17:07:52.601679+0000","objects_scrubbed":2,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:27:35.563129+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"134'208","reported_seq":301,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:08.096998+0000","last_change":"2024-05-30T17:08:08.097157+0000","last_active":"2024-05-30T17:08:08.096998+0000","last_peered":"2024-05-30T17:08:08.096998+0000","last_clean":"2024-05-30T17:08:08.096998+0000","last_became_active":"2024-05-30T17:04:52.995128+0000","last_became_peered":"2024-05-30T17:04:52.995128+0000","last_unstale":"2024-05-30T17:08:08.096998+0000","last_undegraded":"2024-05-30T17:08:08.096998+0000","last_fullsized":"2024-05-30T17:08:08.096998+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'208","last_scrub_stamp":"2024-05-30T17:08:08.096799+0000","last_deep_scrub":"134'208","last_deep_scrub_stamp":"2024-05-30T17:08:08.096799+0000","last_clean_scrub_stamp":"2024-05-30T17:08:08.096799+0000","objects_scrubbed":74,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:18:15.607797+0000","scrub_duration":956,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":78,"num_write_kb":303106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"134'440","reported_seq":468,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:53.248691+0000","last_change":"2024-05-30T17:07:53.248775+0000","last_active":"2024-05-30T17:07:53.248691+0000","last_peered":"2024-05-30T17:07:53.248691+0000","last_clean":"2024-05-30T17:07:53.248691+0000","last_became_active":"2024-05-30T17:04:51.953276+0000","last_became_peered":"2024-05-30T17:04:51.953276+0000","last_unstale":"2024-05-30T17:07:53.248691+0000","last_undegraded":"2024-05-30T17:07:53.248691+0000","last_fullsized":"2024-05-30T17:07:53.248691+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'440","last_scrub_stamp":"2024-05-30T17:07:53.248453+0000","last_deep_scrub":"134'440","last_deep_scrub_stamp":"2024-05-30T17:07:53.248453+0000","last_clean_scrub_stamp":"2024-05-30T17:07:53.248453+0000","objects_scrubbed":4,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:10:35.686112+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1698766,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":437,"num_write_kb":1833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"134'247","reported_seq":326,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:54.437829+0000","last_change":"2024-05-30T17:07:54.437920+0000","last_active":"2024-05-30T17:07:54.437829+0000","last_peered":"2024-05-30T17:07:54.437829+0000","last_clean":"2024-05-30T17:07:54.437829+0000","last_became_active":"2024-05-30T17:04:52.986757+0000","last_became_peered":"2024-05-30T17:04:52.986757+0000","last_unstale":"2024-05-30T17:07:54.437829+0000","last_undegraded":"2024-05-30T17:07:54.437829+0000","last_fullsized":"2024-05-30T17:07:54.437829+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'247","last_scrub_stamp":"2024-05-30T17:07:54.437701+0000","last_deep_scrub":"134'247","last_deep_scrub_stamp":"2024-05-30T17:07:54.437701+0000","last_clean_scrub_stamp":"2024-05-30T17:07:54.437701+0000","objects_scrubbed":92,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:27:46.535595+0000","scrub_duration":1059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385875968,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":50,"num_read_kb":204800,"num_write":92,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:50.402139+0000","last_change":"2024-05-30T17:07:50.402169+0000","last_active":"2024-05-30T17:07:50.402139+0000","last_peered":"2024-05-30T17:07:50.402139+0000","last_clean":"2024-05-30T17:07:50.402139+0000","last_became_active":"2024-05-30T17:04:51.944167+0000","last_became_peered":"2024-05-30T17:04:51.944167+0000","last_unstale":"2024-05-30T17:07:50.402139+0000","last_undegraded":"2024-05-30T17:07:50.402139+0000","last_fullsized":"2024-05-30T17:07:50.402139+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:50.402020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:50.402020+0000","last_clean_scrub_stamp":"2024-05-30T17:07:50.402020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:40:18.802356+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"134'204","reported_seq":294,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:02.725848+0000","last_change":"2024-05-30T17:08:02.725901+0000","last_active":"2024-05-30T17:08:02.725848+0000","last_peered":"2024-05-30T17:08:02.725848+0000","last_clean":"2024-05-30T17:08:02.725848+0000","last_became_active":"2024-05-30T17:04:52.990049+0000","last_became_peered":"2024-05-30T17:04:52.990049+0000","last_unstale":"2024-05-30T17:08:02.725848+0000","last_undegraded":"2024-05-30T17:08:02.725848+0000","last_fullsized":"2024-05-30T17:08:02.725848+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'204","last_scrub_stamp":"2024-05-30T17:08:02.725740+0000","last_deep_scrub":"134'204","last_deep_scrub_stamp":"2024-05-30T17:08:02.725740+0000","last_clean_scrub_stamp":"2024-05-30T17:08:02.725740+0000","objects_scrubbed":79,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:48:59.395643+0000","scrub_duration":1383,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:52.189468+0000","last_change":"2024-05-30T17:07:52.189519+0000","last_active":"2024-05-30T17:07:52.189468+0000","last_peered":"2024-05-30T17:07:52.189468+0000","last_clean":"2024-05-30T17:07:52.189468+0000","last_became_active":"2024-05-30T17:04:51.946868+0000","last_became_peered":"2024-05-30T17:04:51.946868+0000","last_unstale":"2024-05-30T17:07:52.189468+0000","last_undegraded":"2024-05-30T17:07:52.189468+0000","last_fullsized":"2024-05-30T17:07:52.189468+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:52.189272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:52.189272+0000","last_clean_scrub_stamp":"2024-05-30T17:07:52.189272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:52:29.739512+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"134'235","reported_seq":330,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:57.263768+0000","last_change":"2024-05-30T17:07:57.263875+0000","last_active":"2024-05-30T17:07:57.263768+0000","last_peered":"2024-05-30T17:07:57.263768+0000","last_clean":"2024-05-30T17:07:57.263768+0000","last_became_active":"2024-05-30T17:04:52.986293+0000","last_became_peered":"2024-05-30T17:04:52.986293+0000","last_unstale":"2024-05-30T17:07:57.263768+0000","last_undegraded":"2024-05-30T17:07:57.263768+0000","last_fullsized":"2024-05-30T17:07:57.263768+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'235","last_scrub_stamp":"2024-05-30T17:07:57.263675+0000","last_deep_scrub":"134'235","last_deep_scrub_stamp":"2024-05-30T17:07:57.263675+0000","last_clean_scrub_stamp":"2024-05-30T17:07:57.263675+0000","objects_scrubbed":90,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:32.556410+0000","scrub_duration":991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293317,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":66,"num_read_kb":270336,"num_write":90,"num_write_kb":364545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"134'12","reported_seq":40,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:54.231543+0000","last_change":"2024-05-30T17:07:54.231611+0000","last_active":"2024-05-30T17:07:54.231543+0000","last_peered":"2024-05-30T17:07:54.231543+0000","last_clean":"2024-05-30T17:07:54.231543+0000","last_became_active":"2024-05-30T17:04:51.955542+0000","last_became_peered":"2024-05-30T17:04:51.955542+0000","last_unstale":"2024-05-30T17:07:54.231543+0000","last_undegraded":"2024-05-30T17:07:54.231543+0000","last_fullsized":"2024-05-30T17:07:54.231543+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'12","last_scrub_stamp":"2024-05-30T17:07:54.231377+0000","last_deep_scrub":"134'12","last_deep_scrub_stamp":"2024-05-30T17:07:54.231377+0000","last_clean_scrub_stamp":"2024-05-30T17:07:54.231377+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:46:16.444017+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"134'177","reported_seq":230,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:04.045296+0000","last_change":"2024-05-30T17:08:04.045470+0000","last_active":"2024-05-30T17:08:04.045296+0000","last_peered":"2024-05-30T17:08:04.045296+0000","last_clean":"2024-05-30T17:08:04.045296+0000","last_became_active":"2024-05-30T17:04:52.999924+0000","last_became_peered":"2024-05-30T17:04:52.999924+0000","last_unstale":"2024-05-30T17:08:04.045296+0000","last_undegraded":"2024-05-30T17:08:04.045296+0000","last_fullsized":"2024-05-30T17:08:04.045296+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'177","last_scrub_stamp":"2024-05-30T17:08:04.045109+0000","last_deep_scrub":"134'177","last_deep_scrub_stamp":"2024-05-30T17:08:04.045109+0000","last_clean_scrub_stamp":"2024-05-30T17:08:04.045109+0000","objects_scrubbed":66,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:14:42.316734+0000","scrub_duration":861,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":25,"num_read_kb":102400,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"134'2","reported_seq":30,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:56.651906+0000","last_change":"2024-05-30T17:07:56.651944+0000","last_active":"2024-05-30T17:07:56.651906+0000","last_peered":"2024-05-30T17:07:56.651906+0000","last_clean":"2024-05-30T17:07:56.651906+0000","last_became_active":"2024-05-30T17:04:51.947928+0000","last_became_peered":"2024-05-30T17:04:51.947928+0000","last_unstale":"2024-05-30T17:07:56.651906+0000","last_undegraded":"2024-05-30T17:07:56.651906+0000","last_fullsized":"2024-05-30T17:07:56.651906+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'2","last_scrub_stamp":"2024-05-30T17:07:56.651779+0000","last_deep_scrub":"134'2","last_deep_scrub_stamp":"2024-05-30T17:07:56.651779+0000","last_clean_scrub_stamp":"2024-05-30T17:07:56.651779+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:59:47.557117+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"134'219","reported_seq":330,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:59.813398+0000","last_change":"2024-05-30T17:07:59.813466+0000","last_active":"2024-05-30T17:07:59.813398+0000","last_peered":"2024-05-30T17:07:59.813398+0000","last_clean":"2024-05-30T17:07:59.813398+0000","last_became_active":"2024-05-30T17:04:52.974987+0000","last_became_peered":"2024-05-30T17:04:52.974987+0000","last_unstale":"2024-05-30T17:07:59.813398+0000","last_undegraded":"2024-05-30T17:07:59.813398+0000","last_fullsized":"2024-05-30T17:07:59.813398+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'219","last_scrub_stamp":"2024-05-30T17:07:59.813287+0000","last_deep_scrub":"134'219","last_deep_scrub_stamp":"2024-05-30T17:07:59.813287+0000","last_clean_scrub_stamp":"2024-05-30T17:07:59.813287+0000","objects_scrubbed":86,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:55:28.794519+0000","scrub_duration":1187,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360710144,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":81,"num_read_kb":331776,"num_write":86,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"134'3","reported_seq":31,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:51.676608+0000","last_change":"2024-05-30T17:07:51.676658+0000","last_active":"2024-05-30T17:07:51.676608+0000","last_peered":"2024-05-30T17:07:51.676608+0000","last_clean":"2024-05-30T17:07:51.676608+0000","last_became_active":"2024-05-30T17:04:51.949704+0000","last_became_peered":"2024-05-30T17:04:51.949704+0000","last_unstale":"2024-05-30T17:07:51.676608+0000","last_undegraded":"2024-05-30T17:07:51.676608+0000","last_fullsized":"2024-05-30T17:07:51.676608+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'3","last_scrub_stamp":"2024-05-30T17:07:51.676502+0000","last_deep_scrub":"134'3","last_deep_scrub_stamp":"2024-05-30T17:07:51.676502+0000","last_clean_scrub_stamp":"2024-05-30T17:07:51.676502+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:41:58.256834+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"134'220","reported_seq":313,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.533152+0000","last_change":"2024-05-30T17:08:01.533181+0000","last_active":"2024-05-30T17:08:01.533152+0000","last_peered":"2024-05-30T17:08:01.533152+0000","last_clean":"2024-05-30T17:08:01.533152+0000","last_became_active":"2024-05-30T17:04:52.989083+0000","last_became_peered":"2024-05-30T17:04:52.989083+0000","last_unstale":"2024-05-30T17:08:01.533152+0000","last_undegraded":"2024-05-30T17:08:01.533152+0000","last_fullsized":"2024-05-30T17:08:01.533152+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'220","last_scrub_stamp":"2024-05-30T17:08:01.533064+0000","last_deep_scrub":"134'220","last_deep_scrub_stamp":"2024-05-30T17:08:01.533064+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.533064+0000","objects_scrubbed":75,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:32.839902+0000","scrub_duration":953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314572800,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":63,"num_read_kb":237573,"num_write":85,"num_write_kb":307205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"134'2","reported_seq":30,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:51.004912+0000","last_change":"2024-05-30T17:07:51.004984+0000","last_active":"2024-05-30T17:07:51.004912+0000","last_peered":"2024-05-30T17:07:51.004912+0000","last_clean":"2024-05-30T17:07:51.004912+0000","last_became_active":"2024-05-30T17:04:51.942121+0000","last_became_peered":"2024-05-30T17:04:51.942121+0000","last_unstale":"2024-05-30T17:07:51.004912+0000","last_undegraded":"2024-05-30T17:07:51.004912+0000","last_fullsized":"2024-05-30T17:07:51.004912+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'2","last_scrub_stamp":"2024-05-30T17:07:51.004712+0000","last_deep_scrub":"134'2","last_deep_scrub_stamp":"2024-05-30T17:07:51.004712+0000","last_clean_scrub_stamp":"2024-05-30T17:07:51.004712+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T02:58:10.796194+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"134'213","reported_seq":324,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:05.466888+0000","last_change":"2024-05-30T17:08:05.466940+0000","last_active":"2024-05-30T17:08:05.466888+0000","last_peered":"2024-05-30T17:08:05.466888+0000","last_clean":"2024-05-30T17:08:05.466888+0000","last_became_active":"2024-05-30T17:04:52.994290+0000","last_became_peered":"2024-05-30T17:04:52.994290+0000","last_unstale":"2024-05-30T17:08:05.466888+0000","last_undegraded":"2024-05-30T17:08:05.466888+0000","last_fullsized":"2024-05-30T17:08:05.466888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'213","last_scrub_stamp":"2024-05-30T17:08:05.466742+0000","last_deep_scrub":"134'213","last_deep_scrub_stamp":"2024-05-30T17:08:05.466742+0000","last_clean_scrub_stamp":"2024-05-30T17:08:05.466742+0000","objects_scrubbed":80,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:02:54.013069+0000","scrub_duration":1034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":82,"num_read_kb":335872,"num_write":80,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:55.601175+0000","last_change":"2024-05-30T17:07:55.601212+0000","last_active":"2024-05-30T17:07:55.601175+0000","last_peered":"2024-05-30T17:07:55.601175+0000","last_clean":"2024-05-30T17:07:55.601175+0000","last_became_active":"2024-05-30T17:04:51.945142+0000","last_became_peered":"2024-05-30T17:04:51.945142+0000","last_unstale":"2024-05-30T17:07:55.601175+0000","last_undegraded":"2024-05-30T17:07:55.601175+0000","last_fullsized":"2024-05-30T17:07:55.601175+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:55.601008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:55.601008+0000","last_clean_scrub_stamp":"2024-05-30T17:07:55.601008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:56:51.194967+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"134'215","reported_seq":328,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:57.444100+0000","last_change":"2024-05-30T17:07:57.444163+0000","last_active":"2024-05-30T17:07:57.444100+0000","last_peered":"2024-05-30T17:07:57.444100+0000","last_clean":"2024-05-30T17:07:57.444100+0000","last_became_active":"2024-05-30T17:04:52.988660+0000","last_became_peered":"2024-05-30T17:04:52.988660+0000","last_unstale":"2024-05-30T17:07:57.444100+0000","last_undegraded":"2024-05-30T17:07:57.444100+0000","last_fullsized":"2024-05-30T17:07:57.444100+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'215","last_scrub_stamp":"2024-05-30T17:07:57.444022+0000","last_deep_scrub":"134'215","last_deep_scrub_stamp":"2024-05-30T17:07:57.444022+0000","last_clean_scrub_stamp":"2024-05-30T17:07:57.444022+0000","objects_scrubbed":83,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:39:11.236018+0000","scrub_duration":1026,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"134'4","reported_seq":32,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:54.577233+0000","last_change":"2024-05-30T17:07:54.577302+0000","last_active":"2024-05-30T17:07:54.577233+0000","last_peered":"2024-05-30T17:07:54.577233+0000","last_clean":"2024-05-30T17:07:54.577233+0000","last_became_active":"2024-05-30T17:04:51.943085+0000","last_became_peered":"2024-05-30T17:04:51.943085+0000","last_unstale":"2024-05-30T17:07:54.577233+0000","last_undegraded":"2024-05-30T17:07:54.577233+0000","last_fullsized":"2024-05-30T17:07:54.577233+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'4","last_scrub_stamp":"2024-05-30T17:07:54.577016+0000","last_deep_scrub":"134'4","last_deep_scrub_stamp":"2024-05-30T17:07:54.577016+0000","last_clean_scrub_stamp":"2024-05-30T17:07:54.577016+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T18:13:23.926697+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"134'200","reported_seq":291,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.495006+0000","last_change":"2024-05-30T17:08:01.495141+0000","last_active":"2024-05-30T17:08:01.495006+0000","last_peered":"2024-05-30T17:08:01.495006+0000","last_clean":"2024-05-30T17:08:01.495006+0000","last_became_active":"2024-05-30T17:04:52.989409+0000","last_became_peered":"2024-05-30T17:04:52.989409+0000","last_unstale":"2024-05-30T17:08:01.495006+0000","last_undegraded":"2024-05-30T17:08:01.495006+0000","last_fullsized":"2024-05-30T17:08:01.495006+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'200","last_scrub_stamp":"2024-05-30T17:08:01.494880+0000","last_deep_scrub":"134'200","last_deep_scrub_stamp":"2024-05-30T17:08:01.494880+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.494880+0000","objects_scrubbed":72,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:33:38.009438+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297795845,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":62,"num_read_kb":253952,"num_write":72,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:51.688942+0000","last_change":"2024-05-30T17:07:51.688968+0000","last_active":"2024-05-30T17:07:51.688942+0000","last_peered":"2024-05-30T17:07:51.688942+0000","last_clean":"2024-05-30T17:07:51.688942+0000","last_became_active":"2024-05-30T17:04:51.947919+0000","last_became_peered":"2024-05-30T17:04:51.947919+0000","last_unstale":"2024-05-30T17:07:51.688942+0000","last_undegraded":"2024-05-30T17:07:51.688942+0000","last_fullsized":"2024-05-30T17:07:51.688942+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:51.688854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:51.688854+0000","last_clean_scrub_stamp":"2024-05-30T17:07:51.688854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:29:30.244918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"134'224","reported_seq":301,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:57.287393+0000","last_change":"2024-05-30T17:07:57.287449+0000","last_active":"2024-05-30T17:07:57.287393+0000","last_peered":"2024-05-30T17:07:57.287393+0000","last_clean":"2024-05-30T17:07:57.287393+0000","last_became_active":"2024-05-30T17:04:52.978531+0000","last_became_peered":"2024-05-30T17:04:52.978531+0000","last_unstale":"2024-05-30T17:07:57.287393+0000","last_undegraded":"2024-05-30T17:07:57.287393+0000","last_fullsized":"2024-05-30T17:07:57.287393+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'224","last_scrub_stamp":"2024-05-30T17:07:57.287312+0000","last_deep_scrub":"134'224","last_deep_scrub_stamp":"2024-05-30T17:07:57.287312+0000","last_clean_scrub_stamp":"2024-05-30T17:07:57.287312+0000","objects_scrubbed":79,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:39:11.079308+0000","scrub_duration":1024,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":48,"num_read_kb":188418,"num_write":83,"num_write_kb":323586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:53.656078+0000","last_change":"2024-05-30T17:07:53.656120+0000","last_active":"2024-05-30T17:07:53.656078+0000","last_peered":"2024-05-30T17:07:53.656078+0000","last_clean":"2024-05-30T17:07:53.656078+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:07:53.656078+0000","last_undegraded":"2024-05-30T17:07:53.656078+0000","last_fullsized":"2024-05-30T17:07:53.656078+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:53.655912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:53.655912+0000","last_clean_scrub_stamp":"2024-05-30T17:07:53.655912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:44:07.024395+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"134'192","reported_seq":304,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:08.957698+0000","last_change":"2024-05-30T17:08:08.957809+0000","last_active":"2024-05-30T17:08:08.957698+0000","last_peered":"2024-05-30T17:08:08.957698+0000","last_clean":"2024-05-30T17:08:08.957698+0000","last_became_active":"2024-05-30T17:05:34.860839+0000","last_became_peered":"2024-05-30T17:05:34.860839+0000","last_unstale":"2024-05-30T17:08:08.957698+0000","last_undegraded":"2024-05-30T17:08:08.957698+0000","last_fullsized":"2024-05-30T17:08:08.957698+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'192","last_scrub_stamp":"2024-05-30T17:08:08.957481+0000","last_deep_scrub":"134'192","last_deep_scrub_stamp":"2024-05-30T17:08:08.957481+0000","last_clean_scrub_stamp":"2024-05-30T17:08:08.957481+0000","objects_scrubbed":69,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:12:46.265928+0000","scrub_duration":814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289406976,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":69,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:52.657424+0000","last_change":"2024-05-30T17:07:52.657447+0000","last_active":"2024-05-30T17:07:52.657424+0000","last_peered":"2024-05-30T17:07:52.657424+0000","last_clean":"2024-05-30T17:07:52.657424+0000","last_became_active":"2024-05-30T17:04:51.950364+0000","last_became_peered":"2024-05-30T17:04:51.950364+0000","last_unstale":"2024-05-30T17:07:52.657424+0000","last_undegraded":"2024-05-30T17:07:52.657424+0000","last_fullsized":"2024-05-30T17:07:52.657424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:52.657348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:52.657348+0000","last_clean_scrub_stamp":"2024-05-30T17:07:52.657348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T20:37:38.383435+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"134'212","reported_seq":278,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:54.976892+0000","last_change":"2024-05-30T17:07:54.976962+0000","last_active":"2024-05-30T17:07:54.976892+0000","last_peered":"2024-05-30T17:07:54.976892+0000","last_clean":"2024-05-30T17:07:54.976892+0000","last_became_active":"2024-05-30T17:04:52.966035+0000","last_became_peered":"2024-05-30T17:04:52.966035+0000","last_unstale":"2024-05-30T17:07:54.976892+0000","last_undegraded":"2024-05-30T17:07:54.976892+0000","last_fullsized":"2024-05-30T17:07:54.976892+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'212","last_scrub_stamp":"2024-05-30T17:07:54.976782+0000","last_deep_scrub":"134'212","last_deep_scrub_stamp":"2024-05-30T17:07:54.976782+0000","last_clean_scrub_stamp":"2024-05-30T17:07:54.976782+0000","objects_scrubbed":79,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:41:24.232411+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331350016,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":79,"num_write_kb":323584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"134'3","reported_seq":31,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:51.380874+0000","last_change":"2024-05-30T17:07:51.380930+0000","last_active":"2024-05-30T17:07:51.380874+0000","last_peered":"2024-05-30T17:07:51.380874+0000","last_clean":"2024-05-30T17:07:51.380874+0000","last_became_active":"2024-05-30T17:04:51.949575+0000","last_became_peered":"2024-05-30T17:04:51.949575+0000","last_unstale":"2024-05-30T17:07:51.380874+0000","last_undegraded":"2024-05-30T17:07:51.380874+0000","last_fullsized":"2024-05-30T17:07:51.380874+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'3","last_scrub_stamp":"2024-05-30T17:07:51.380713+0000","last_deep_scrub":"134'3","last_deep_scrub_stamp":"2024-05-30T17:07:51.380713+0000","last_clean_scrub_stamp":"2024-05-30T17:07:51.380713+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:02:43.008979+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"134'217","reported_seq":333,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:02.870007+0000","last_change":"2024-05-30T17:08:02.870074+0000","last_active":"2024-05-30T17:08:02.870007+0000","last_peered":"2024-05-30T17:08:02.870007+0000","last_clean":"2024-05-30T17:08:02.870007+0000","last_became_active":"2024-05-30T17:04:52.997726+0000","last_became_peered":"2024-05-30T17:04:52.997726+0000","last_unstale":"2024-05-30T17:08:02.870007+0000","last_undegraded":"2024-05-30T17:08:02.870007+0000","last_fullsized":"2024-05-30T17:08:02.870007+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'217","last_scrub_stamp":"2024-05-30T17:08:02.869879+0000","last_deep_scrub":"134'217","last_deep_scrub_stamp":"2024-05-30T17:08:02.869879+0000","last_clean_scrub_stamp":"2024-05-30T17:08:02.869879+0000","objects_scrubbed":83,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T19:10:45.307538+0000","scrub_duration":1359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348127232,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":83,"num_write_kb":339968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"134'49","reported_seq":77,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:56.012432+0000","last_change":"2024-05-30T17:07:56.012458+0000","last_active":"2024-05-30T17:07:56.012432+0000","last_peered":"2024-05-30T17:07:56.012432+0000","last_clean":"2024-05-30T17:07:56.012432+0000","last_became_active":"2024-05-30T17:04:51.949295+0000","last_became_peered":"2024-05-30T17:04:51.949295+0000","last_unstale":"2024-05-30T17:07:56.012432+0000","last_undegraded":"2024-05-30T17:07:56.012432+0000","last_fullsized":"2024-05-30T17:07:56.012432+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'49","last_scrub_stamp":"2024-05-30T17:07:56.012326+0000","last_deep_scrub":"134'49","last_deep_scrub_stamp":"2024-05-30T17:07:56.012326+0000","last_clean_scrub_stamp":"2024-05-30T17:07:56.012326+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T03:08:16.869489+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"134'214","reported_seq":340,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:00.676729+0000","last_change":"2024-05-30T17:08:00.676806+0000","last_active":"2024-05-30T17:08:00.676729+0000","last_peered":"2024-05-30T17:08:00.676729+0000","last_clean":"2024-05-30T17:08:00.676729+0000","last_became_active":"2024-05-30T17:04:52.984675+0000","last_became_peered":"2024-05-30T17:04:52.984675+0000","last_unstale":"2024-05-30T17:08:00.676729+0000","last_undegraded":"2024-05-30T17:08:00.676729+0000","last_fullsized":"2024-05-30T17:08:00.676729+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'214","last_scrub_stamp":"2024-05-30T17:08:00.676585+0000","last_deep_scrub":"134'214","last_deep_scrub_stamp":"2024-05-30T17:08:00.676585+0000","last_clean_scrub_stamp":"2024-05-30T17:08:00.676585+0000","objects_scrubbed":78,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T22:11:32.701246+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327155712,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":397312,"num_write":80,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:53.159149+0000","last_change":"2024-05-30T17:07:53.159174+0000","last_active":"2024-05-30T17:07:53.159149+0000","last_peered":"2024-05-30T17:07:53.159149+0000","last_clean":"2024-05-30T17:07:53.159149+0000","last_became_active":"2024-05-30T17:04:51.952631+0000","last_became_peered":"2024-05-30T17:04:51.952631+0000","last_unstale":"2024-05-30T17:07:53.159149+0000","last_undegraded":"2024-05-30T17:07:53.159149+0000","last_fullsized":"2024-05-30T17:07:53.159149+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:53.159058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:53.159058+0000","last_clean_scrub_stamp":"2024-05-30T17:07:53.159058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:27:45.256951+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"134'207","reported_seq":289,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:58.898337+0000","last_change":"2024-05-30T17:07:58.898403+0000","last_active":"2024-05-30T17:07:58.898337+0000","last_peered":"2024-05-30T17:07:58.898337+0000","last_clean":"2024-05-30T17:07:58.898337+0000","last_became_active":"2024-05-30T17:04:52.990476+0000","last_became_peered":"2024-05-30T17:04:52.990476+0000","last_unstale":"2024-05-30T17:07:58.898337+0000","last_undegraded":"2024-05-30T17:07:58.898337+0000","last_fullsized":"2024-05-30T17:07:58.898337+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'207","last_scrub_stamp":"2024-05-30T17:07:58.898190+0000","last_deep_scrub":"134'207","last_deep_scrub_stamp":"2024-05-30T17:07:58.898190+0000","last_clean_scrub_stamp":"2024-05-30T17:07:58.898190+0000","objects_scrubbed":77,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:04:49.633051+0000","scrub_duration":901,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378750,"num_objects":77,"num_object_clones":1,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":1,"num_read":54,"num_read_kb":217089,"num_write":79,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:48.701144+0000","last_change":"2024-05-30T17:07:48.701181+0000","last_active":"2024-05-30T17:07:48.701144+0000","last_peered":"2024-05-30T17:07:48.701144+0000","last_clean":"2024-05-30T17:07:48.701144+0000","last_became_active":"2024-05-30T17:04:51.933061+0000","last_became_peered":"2024-05-30T17:04:51.933061+0000","last_unstale":"2024-05-30T17:07:48.701144+0000","last_undegraded":"2024-05-30T17:07:48.701144+0000","last_fullsized":"2024-05-30T17:07:48.701144+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:48.701020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:48.701020+0000","last_clean_scrub_stamp":"2024-05-30T17:07:48.701020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T21:51:51.079279+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"134'210","reported_seq":301,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:00.232004+0000","last_change":"2024-05-30T17:08:00.232057+0000","last_active":"2024-05-30T17:08:00.232004+0000","last_peered":"2024-05-30T17:08:00.232004+0000","last_clean":"2024-05-30T17:08:00.232004+0000","last_became_active":"2024-05-30T17:04:52.981542+0000","last_became_peered":"2024-05-30T17:04:52.981542+0000","last_unstale":"2024-05-30T17:08:00.232004+0000","last_undegraded":"2024-05-30T17:08:00.232004+0000","last_fullsized":"2024-05-30T17:08:00.232004+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'210","last_scrub_stamp":"2024-05-30T17:08:00.231849+0000","last_deep_scrub":"134'210","last_deep_scrub_stamp":"2024-05-30T17:08:00.231849+0000","last_clean_scrub_stamp":"2024-05-30T17:08:00.231849+0000","objects_scrubbed":76,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:16:34.724660+0000","scrub_duration":1211,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":61,"num_read_kb":249856,"num_write":76,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:51.209699+0000","last_change":"2024-05-30T17:07:51.209729+0000","last_active":"2024-05-30T17:07:51.209699+0000","last_peered":"2024-05-30T17:07:51.209699+0000","last_clean":"2024-05-30T17:07:51.209699+0000","last_became_active":"2024-05-30T17:04:51.946882+0000","last_became_peered":"2024-05-30T17:04:51.946882+0000","last_unstale":"2024-05-30T17:07:51.209699+0000","last_undegraded":"2024-05-30T17:07:51.209699+0000","last_fullsized":"2024-05-30T17:07:51.209699+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:51.209587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:51.209587+0000","last_clean_scrub_stamp":"2024-05-30T17:07:51.209587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T05:02:42.837854+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"134'255","reported_seq":367,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:03.146696+0000","last_change":"2024-05-30T17:08:03.146798+0000","last_active":"2024-05-30T17:08:03.146696+0000","last_peered":"2024-05-30T17:08:03.146696+0000","last_clean":"2024-05-30T17:08:03.146696+0000","last_became_active":"2024-05-30T17:04:52.992040+0000","last_became_peered":"2024-05-30T17:04:52.992040+0000","last_unstale":"2024-05-30T17:08:03.146696+0000","last_undegraded":"2024-05-30T17:08:03.146696+0000","last_fullsized":"2024-05-30T17:08:03.146696+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'255","last_scrub_stamp":"2024-05-30T17:08:03.146598+0000","last_deep_scrub":"134'255","last_deep_scrub_stamp":"2024-05-30T17:08:03.146598+0000","last_clean_scrub_stamp":"2024-05-30T17:08:03.146598+0000","objects_scrubbed":98,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:05:49.390392+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":411041792,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":83,"num_read_kb":339968,"num_write":98,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:58.009815+0000","last_change":"2024-05-30T17:07:58.009877+0000","last_active":"2024-05-30T17:07:58.009815+0000","last_peered":"2024-05-30T17:07:58.009815+0000","last_clean":"2024-05-30T17:07:58.009815+0000","last_became_active":"2024-05-30T17:04:51.950379+0000","last_became_peered":"2024-05-30T17:04:51.950379+0000","last_unstale":"2024-05-30T17:07:58.009815+0000","last_undegraded":"2024-05-30T17:07:58.009815+0000","last_fullsized":"2024-05-30T17:07:58.009815+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:58.009623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:58.009623+0000","last_clean_scrub_stamp":"2024-05-30T17:07:58.009623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:44.208426+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"134'218","reported_seq":328,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:01.387115+0000","last_change":"2024-05-30T17:08:01.387194+0000","last_active":"2024-05-30T17:08:01.387115+0000","last_peered":"2024-05-30T17:08:01.387115+0000","last_clean":"2024-05-30T17:08:01.387115+0000","last_became_active":"2024-05-30T17:04:52.987684+0000","last_became_peered":"2024-05-30T17:04:52.987684+0000","last_unstale":"2024-05-30T17:08:01.387115+0000","last_undegraded":"2024-05-30T17:08:01.387115+0000","last_fullsized":"2024-05-30T17:08:01.387115+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'218","last_scrub_stamp":"2024-05-30T17:08:01.386997+0000","last_deep_scrub":"134'218","last_deep_scrub_stamp":"2024-05-30T17:08:01.386997+0000","last_clean_scrub_stamp":"2024-05-30T17:08:01.386997+0000","objects_scrubbed":82,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:06:16.509118+0000","scrub_duration":1777,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343932928,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":82,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"134'36","reported_seq":98,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:52.703495+0000","last_change":"2024-05-30T17:07:52.703522+0000","last_active":"2024-05-30T17:07:52.703495+0000","last_peered":"2024-05-30T17:07:52.703495+0000","last_clean":"2024-05-30T17:07:52.703495+0000","last_became_active":"2024-05-30T17:04:51.948995+0000","last_became_peered":"2024-05-30T17:04:51.948995+0000","last_unstale":"2024-05-30T17:07:52.703495+0000","last_undegraded":"2024-05-30T17:07:52.703495+0000","last_fullsized":"2024-05-30T17:07:52.703495+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'36","last_scrub_stamp":"2024-05-30T17:07:52.703400+0000","last_deep_scrub":"134'36","last_deep_scrub_stamp":"2024-05-30T17:07:52.703400+0000","last_clean_scrub_stamp":"2024-05-30T17:07:52.703400+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:33:29.217958+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4250,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"134'232","reported_seq":339,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:03.652174+0000","last_change":"2024-05-30T17:08:03.652307+0000","last_active":"2024-05-30T17:08:03.652174+0000","last_peered":"2024-05-30T17:08:03.652174+0000","last_clean":"2024-05-30T17:08:03.652174+0000","last_became_active":"2024-05-30T17:04:52.991997+0000","last_became_peered":"2024-05-30T17:04:52.991997+0000","last_unstale":"2024-05-30T17:08:03.652174+0000","last_undegraded":"2024-05-30T17:08:03.652174+0000","last_fullsized":"2024-05-30T17:08:03.652174+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'232","last_scrub_stamp":"2024-05-30T17:08:03.651965+0000","last_deep_scrub":"134'232","last_deep_scrub_stamp":"2024-05-30T17:08:03.651965+0000","last_clean_scrub_stamp":"2024-05-30T17:08:03.651965+0000","objects_scrubbed":96,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T01:59:38.944699+0000","scrub_duration":1264,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":402653184,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":96,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:50.715488+0000","last_change":"2024-05-30T17:07:50.715534+0000","last_active":"2024-05-30T17:07:50.715488+0000","last_peered":"2024-05-30T17:07:50.715488+0000","last_clean":"2024-05-30T17:07:50.715488+0000","last_became_active":"2024-05-30T17:04:51.946173+0000","last_became_peered":"2024-05-30T17:04:51.946173+0000","last_unstale":"2024-05-30T17:07:50.715488+0000","last_undegraded":"2024-05-30T17:07:50.715488+0000","last_fullsized":"2024-05-30T17:07:50.715488+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-30T17:07:50.715308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-30T17:07:50.715308+0000","last_clean_scrub_stamp":"2024-05-30T17:07:50.715308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T00:19:36.914112+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"134'188","reported_seq":269,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:08:05.906024+0000","last_change":"2024-05-30T17:08:05.906129+0000","last_active":"2024-05-30T17:08:05.906024+0000","last_peered":"2024-05-30T17:08:05.906024+0000","last_clean":"2024-05-30T17:08:05.906024+0000","last_became_active":"2024-05-30T17:04:52.992202+0000","last_became_peered":"2024-05-30T17:04:52.992202+0000","last_unstale":"2024-05-30T17:08:05.906024+0000","last_undegraded":"2024-05-30T17:08:05.906024+0000","last_fullsized":"2024-05-30T17:08:05.906024+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'188","last_scrub_stamp":"2024-05-30T17:08:05.905846+0000","last_deep_scrub":"134'188","last_deep_scrub_stamp":"2024-05-30T17:08:05.905846+0000","last_clean_scrub_stamp":"2024-05-30T17:08:05.905846+0000","objects_scrubbed":66,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T17:25:21.917528+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":276824064,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":53,"num_read_kb":217088,"num_write":66,"num_write_kb":270336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"134'4","reported_seq":32,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:57.033642+0000","last_change":"2024-05-30T17:07:57.033704+0000","last_active":"2024-05-30T17:07:57.033642+0000","last_peered":"2024-05-30T17:07:57.033642+0000","last_clean":"2024-05-30T17:07:57.033642+0000","last_became_active":"2024-05-30T17:04:51.951484+0000","last_became_peered":"2024-05-30T17:04:51.951484+0000","last_unstale":"2024-05-30T17:07:57.033642+0000","last_undegraded":"2024-05-30T17:07:57.033642+0000","last_fullsized":"2024-05-30T17:07:57.033642+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'4","last_scrub_stamp":"2024-05-30T17:07:57.033538+0000","last_deep_scrub":"134'4","last_deep_scrub_stamp":"2024-05-30T17:07:57.033538+0000","last_clean_scrub_stamp":"2024-05-30T17:07:57.033538+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-01T04:20:39.139073+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"134'119","reported_seq":399,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-05-30T17:07:54.113009+0000","last_change":"2024-05-30T17:07:54.113044+0000","last_active":"2024-05-30T17:07:54.113009+0000","last_peered":"2024-05-30T17:07:54.113009+0000","last_clean":"2024-05-30T17:07:54.113009+0000","last_became_active":"2024-05-30T16:43:41.954942+0000","last_became_peered":"2024-05-30T16:43:41.954942+0000","last_unstale":"2024-05-30T17:07:54.113009+0000","last_undegraded":"2024-05-30T17:07:54.113009+0000","last_fullsized":"2024-05-30T17:07:54.113009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'119","last_scrub_stamp":"2024-05-30T17:07:54.112915+0000","last_deep_scrub":"134'119","last_deep_scrub_stamp":"2024-05-30T17:07:54.112915+0000","last_clean_scrub_stamp":"2024-05-30T17:07:54.112915+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-31T23:26:07.956278+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":20912801557,"num_objects":4996,"num_object_clones":1,"num_object_copies":9992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":1,"num_read":4290,"num_read_kb":17367090,"num_write":5064,"num_write_kb":20422699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":41943040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41825673216,"data_stored":41825603114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13569,"ondisk_log_size":13569,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1703814,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":644,"num_write_kb":1934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3473408,"data_stored":3407628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":602,"ondisk_log_size":602,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509737,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5959160,"kb_used_data":5811440,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":88412680,"statfs":{"total":96636764160,"available":90534584320,"internally_reserved":0,"allocated":5950914560,"data_stored":5949962642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509737,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6194996,"kb_used_data":6051436,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":88176844,"statfs":{"total":96636764160,"available":90293088256,"internally_reserved":0,"allocated":6196670464,"data_stored":6195723625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509738,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5015352,"kb_used_data":4871792,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":89356488,"statfs":{"total":96636764160,"available":91501043712,"internally_reserved":0,"allocated":4988715008,"data_stored":4987768020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509738,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6958580,"kb_used_data":6792812,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":87413260,"statfs":{"total":96636764160,"available":89511178240,"internally_reserved":0,"allocated":6955839488,"data_stored":6954892805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509737,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4728548,"kb_used_data":4589148,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":89643292,"statfs":{"total":96636764160,"available":91794731008,"internally_reserved":0,"allocated":4699287552,"data_stored":4698356383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542440,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4579320,"kb_used_data":4435760,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":89792520,"statfs":{"total":96636764160,"available":91947540480,"internally_reserved":0,"allocated":4542218240,"data_stored":4541262732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542441,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4265844,"kb_used_data":4118124,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90105996,"statfs":{"total":96636764160,"available":92268539904,"internally_reserved":0,"allocated":4216958976,"data_stored":4216012047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4331816,"kb_used_data":4192416,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":90040024,"statfs":{"total":96636764160,"available":92200984576,"internally_reserved":0,"allocated":4293033984,"data_stored":4292099392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1716224,"data_stored":1703016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1720320,"data_stored":1699404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4290777088,"data_stored":4290773253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4697624576,"data_stored":4697620612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6954168320,"data_stored":6954156418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4987035648,"data_stored":4987027839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6195003392,"data_stored":6194987784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4215287808,"data_stored":4215276164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5947535360,"data_stored":5947523855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4538241024,"data_stored":4538237189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-30T17:08:14.181 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-30T17:08:14.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.424+0000 7fd81fe00640 1 Processor -- start 2024-05-30T17:08:14.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.424+0000 7fd81fe00640 1 -- start start 2024-05-30T17:08:14.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.425+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8200d16a0 0x7fd8200cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.425+0000 7fd81fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd82015ec50 con 0x7fd8200cf870 2024-05-30T17:08:14.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.425+0000 7fd81fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8200cff80 con 0x7fd82015e870 2024-05-30T17:08:14.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.425+0000 7fd81fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd8200d2150 con 0x7fd8200d16a0 2024-05-30T17:08:14.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.425+0000 7fd81f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd8200cf870 0x7fd8200d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45648/0 (socket says 172.21.15.67:45648) 2024-05-30T17:08:14.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.425+0000 7fd81f400640 1 -- 172.21.15.67:0/1352609303 learned_addr learned my addr 172.21.15.67:0/1352609303 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:08:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81e000640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8200d16a0 0x7fd8200cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721204196 0 0) 0x7fd82015ec50 con 0x7fd8200cf870 2024-05-30T17:08:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd804003520 con 0x7fd8200cf870 2024-05-30T17:08:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3786503909 0 0) 0x7fd804003520 con 0x7fd8200cf870 2024-05-30T17:08:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8200d16a0 msgr2=0x7fd8200cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:08:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8200d16a0 0x7fd8200cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 >> v1:172.21.15.169:6789/0 conn(0x7fd82015e870 legacy=0x7fd8200d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8200d0260 con 0x7fd8200cf870 2024-05-30T17:08:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3734954716 0 0) 0x7fd8180019f0 con 0x7fd8200cf870 2024-05-30T17:08:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 >> v1:172.21.15.67:6789/0 conn(0x7fd8200cf870 legacy=0x7fd8200d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8200d16a0 0x7fd8040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804003f60 0x7fd804004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd804004d40 0x7fd8040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8200d2150 con 0x7fd804004d40 2024-05-30T17:08:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd80400baa0 con 0x7fd804003f60 2024-05-30T17:08:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.426+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd80400bca0 con 0x7fd8200d16a0 2024-05-30T17:08:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.427+0000 7fd81f400640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd804004d40 0x7fd8040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.427+0000 7fd81e000640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804003f60 0x7fd804004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.427+0000 7fd81ea00640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8200d16a0 0x7fd8040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.427+0000 7fd81f400640 1 -- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8200d16a0 msgr2=0x7fd8040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:08:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.427+0000 7fd81f400640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8200d16a0 0x7fd8040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.427+0000 7fd81f400640 1 -- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804003f60 msgr2=0x7fd804004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:08:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.427+0000 7fd81f400640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804003f60 0x7fd804004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.427+0000 7fd81f400640 1 -- 172.21.15.67:0/1352609303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd804003520 con 0x7fd804004d40 2024-05-30T17:08:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.428+0000 7fd81f400640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd804004d40 0x7fd8040051c0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fd81800a850 tx=0x7fd804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24098c33cd8189a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:08:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81800a4a0 con 0x7fd804004d40 2024-05-30T17:08:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd81802f070 con 0x7fd804004d40 2024-05-30T17:08:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81d600640 1 -- 172.21.15.67:0/1352609303 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81802bd20 con 0x7fd804004d40 2024-05-30T17:08:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81fe00640 1 -- 172.21.15.67:0/1352609303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd804004d40 msgr2=0x7fd8040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81fe00640 1 --2- 172.21.15.67:0/1352609303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd804004d40 0x7fd8040051c0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fd81800a850 tx=0x7fd804005830 comp rx=0 tx=0).stop 2024-05-30T17:08:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81f400640 1 -- 172.21.15.67:0/1352609303 reap_dead start 2024-05-30T17:08:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81fe00640 1 -- 172.21.15.67:0/1352609303 shutdown_connections 2024-05-30T17:08:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81fe00640 1 -- 172.21.15.67:0/1352609303 >> 172.21.15.67:0/1352609303 conn(0x7fd82008bfc0 msgr2=0x7fd8200d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:08:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81fe00640 1 -- 172.21.15.67:0/1352609303 shutdown_connections 2024-05-30T17:08:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.429+0000 7fd81fe00640 1 -- 172.21.15.67:0/1352609303 wait complete. 2024-05-30T17:08:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.431+0000 7fd81fe00640 1 Processor -- start 2024-05-30T17:08:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.431+0000 7fd81fe00640 1 -- start start 2024-05-30T17:08:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.432+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804004d40 0x7fd8200cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.432+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8200cb650 0x7fd8201f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.432+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8201f4a30 0x7fd8201f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.432+0000 7fd81fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd8200d0820 con 0x7fd8200cb650 2024-05-30T17:08:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.432+0000 7fd81fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8200cfd20 con 0x7fd804004d40 2024-05-30T17:08:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.432+0000 7fd81fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd8200d4fb0 con 0x7fd8201f4a30 2024-05-30T17:08:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8200cb650 0x7fd8201f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8200cb650 0x7fd8201f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56988/0 (socket says 172.21.15.67:56988) 2024-05-30T17:08:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81e000640 1 -- 172.21.15.67:0/1998230011 learned_addr learned my addr 172.21.15.67:0/1998230011 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:08:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81ea00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804004d40 0x7fd8200cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81e000640 1 -- 172.21.15.67:0/1998230011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8201f4a30 msgr2=0x7fd8201f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81f400640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8201f4a30 0x7fd8201f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81e000640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8201f4a30 0x7fd8201f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81e000640 1 -- 172.21.15.67:0/1998230011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804004d40 msgr2=0x7fd8200cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81e000640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804004d40 0x7fd8200cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81e000640 1 -- 172.21.15.67:0/1998230011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd820161f20 con 0x7fd8200cb650 2024-05-30T17:08:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81f400640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8201f4a30 0x7fd8201f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:08:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.433+0000 7fd81e000640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8200cb650 0x7fd8201f44f0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd8080115b0 tx=0x7fd808011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:08:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.434+0000 7fd816a00640 1 -- 172.21.15.67:0/1998230011 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808019070 con 0x7fd8200cb650 2024-05-30T17:08:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.434+0000 7fd816a00640 1 -- 172.21.15.67:0/1998230011 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd808005da0 con 0x7fd8200cb650 2024-05-30T17:08:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.434+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8200b7110 con 0x7fd8200cb650 2024-05-30T17:08:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.434+0000 7fd816a00640 1 -- 172.21.15.67:0/1998230011 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808006250 con 0x7fd8200cb650 2024-05-30T17:08:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.434+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8201f5390 con 0x7fd8200cb650 2024-05-30T17:08:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.435+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8200689e0 con 0x7fd8200cb650 2024-05-30T17:08:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.436+0000 7fd816a00640 1 -- 172.21.15.67:0/1998230011 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7fd80801c070 con 0x7fd8200cb650 2024-05-30T17:08:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.437+0000 7fd816a00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7fc06e510 0x7fd7fc0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.437+0000 7fd816a00640 1 -- 172.21.15.67:0/1998230011 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fd80808afa0 con 0x7fd8200cb650 2024-05-30T17:08:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.437+0000 7fd81ea00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7fc06e510 0x7fd7fc0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.438+0000 7fd81ea00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7fc06e510 0x7fd7fc0709d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd810007fd0 tx=0x7fd810009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:08:14.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.441+0000 7fd816a00640 1 -- 172.21.15.67:0/1998230011 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd80805ae30 con 0x7fd8200cb650 2024-05-30T17:08:14.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.645+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fd82006e350 con 0x7fd8200cb650 2024-05-30T17:08:14.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.653+0000 7fd816a00640 1 -- 172.21.15.67:0/1998230011 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v30)=0 v30) ==== 123+0+0 (secure 0 0 0) 0x7fd80805a730 con 0x7fd8200cb650 2024-05-30T17:08:14.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.656+0000 7fd816a00640 1 -- 172.21.15.67:0/1998230011 <== mon.0 v2:172.21.15.67:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd80805eae0 con 0x7fd8200cb650 2024-05-30T17:08:14.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.657+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7fc06e510 msgr2=0x7fd7fc0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.657+0000 7fd81fe00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7fc06e510 0x7fd7fc0709d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd810007fd0 tx=0x7fd810009040 comp rx=0 tx=0).stop 2024-05-30T17:08:14.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.657+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8200cb650 msgr2=0x7fd8201f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.657+0000 7fd81fe00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8200cb650 0x7fd8201f44f0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd8080115b0 tx=0x7fd808011a70 comp rx=0 tx=0).stop 2024-05-30T17:08:14.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.658+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 shutdown_connections 2024-05-30T17:08:14.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.658+0000 7fd81fe00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fd7fc06e510 0x7fd7fc0709d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.658+0000 7fd81fe00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8201f4a30 0x7fd8201f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.658+0000 7fd81fe00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8200cb650 0x7fd8201f44f0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.658+0000 7fd81fe00640 1 --2- 172.21.15.67:0/1998230011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd804004d40 0x7fd8200cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.658+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 >> 172.21.15.67:0/1998230011 conn(0x7fd82008bfc0 msgr2=0x7fd8200b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:08:14.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.658+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 shutdown_connections 2024-05-30T17:08:14.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.658+0000 7fd81fe00640 1 -- 172.21.15.67:0/1998230011 wait complete. 2024-05-30T17:08:14.679 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-30T17:08:14.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-30T17:08:14.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.878+0000 7fb57dc00640 1 Processor -- start 2024-05-30T17:08:14.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.879+0000 7fb57dc00640 1 -- start start 2024-05-30T17:08:14.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.880+0000 7fb57dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb5780d16a0 0x7fb5780cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.880+0000 7fb57dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5780cf870 0x7fb5780d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.880+0000 7fb57dc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5780cfce0 con 0x7fb57815e870 2024-05-30T17:08:14.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.880+0000 7fb57dc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 -- 0x7fb5780d2150 con 0x7fb5780d16a0 2024-05-30T17:08:14.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.880+0000 7fb57dc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb57815ec50 con 0x7fb5780cf870 2024-05-30T17:08:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.881+0000 7fb577e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb57815e870 0x7fb5780d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45652/0 (socket says 172.21.15.67:45652) 2024-05-30T17:08:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.881+0000 7fb577e00640 1 -- 172.21.15.67:0/1217565344 learned_addr learned my addr 172.21.15.67:0/1217565344 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-05-30T17:08:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.881+0000 7fb577400640 1 --2- 172.21.15.67:0/1217565344 >> v2:172.21.15.67:3300/0 conn(0x7fb5780d16a0 0x7fb5780cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.881+0000 7fb57d200640 1 --2- 172.21.15.67:0/1217565344 >> v2:172.21.15.169:3300/0 conn(0x7fb5780cf870 0x7fb5780d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.881+0000 7fb577400640 1 -- 172.21.15.67:0/1217565344 >> v2:172.21.15.169:3300/0 conn(0x7fb5780cf870 msgr2=0x7fb5780d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-30T17:08:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.881+0000 7fb577400640 1 --2- 172.21.15.67:0/1217565344 >> v2:172.21.15.169:3300/0 conn(0x7fb5780cf870 0x7fb5780d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.881+0000 7fb577400640 1 -- 172.21.15.67:0/1217565344 >> v1:172.21.15.67:6789/0 conn(0x7fb57815e870 legacy=0x7fb5780d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.881+0000 7fb577400640 1 -- 172.21.15.67:0/1217565344 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5780cff80 con 0x7fb5780d16a0 2024-05-30T17:08:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.882+0000 7fb577400640 1 --2- 172.21.15.67:0/1217565344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5780d16a0 0x7fb5780cf330 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fb578163400 tx=0x7fb56002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5723a56e6e3300d server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:08:14.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.883+0000 7fb576a00640 1 -- 172.21.15.67:0/1217565344 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560007740 con 0x7fb5780d16a0 2024-05-30T17:08:14.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.883+0000 7fb576a00640 1 -- 172.21.15.67:0/1217565344 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb5600078e0 con 0x7fb5780d16a0 2024-05-30T17:08:14.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.883+0000 7fb576a00640 1 -- 172.21.15.67:0/1217565344 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560007cd0 con 0x7fb5780d16a0 2024-05-30T17:08:14.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.883+0000 7fb57dc00640 1 -- 172.21.15.67:0/1217565344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5780d16a0 msgr2=0x7fb5780cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.883+0000 7fb57dc00640 1 --2- 172.21.15.67:0/1217565344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5780d16a0 0x7fb5780cf330 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fb578163400 tx=0x7fb56002a660 comp rx=0 tx=0).stop 2024-05-30T17:08:14.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.883+0000 7fb57dc00640 1 -- 172.21.15.67:0/1217565344 shutdown_connections 2024-05-30T17:08:14.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.883+0000 7fb57dc00640 1 --2- 172.21.15.67:0/1217565344 >> v2:172.21.15.169:3300/0 conn(0x7fb5780cf870 0x7fb5780d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.883+0000 7fb57dc00640 1 --2- 172.21.15.67:0/1217565344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5780d16a0 0x7fb5780cf330 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.883+0000 7fb57dc00640 1 -- 172.21.15.67:0/1217565344 >> 172.21.15.67:0/1217565344 conn(0x7fb57808bfc0 msgr2=0x7fb5780d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:08:14.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.884+0000 7fb57dc00640 1 -- 172.21.15.67:0/1217565344 shutdown_connections 2024-05-30T17:08:14.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.884+0000 7fb57dc00640 1 -- 172.21.15.67:0/1217565344 wait complete. 2024-05-30T17:08:14.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.885+0000 7fb57dc00640 1 Processor -- start 2024-05-30T17:08:14.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.885+0000 7fb57dc00640 1 -- start start 2024-05-30T17:08:14.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.886+0000 7fb57dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5780cf870 0x7fb57815f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.886+0000 7fb57dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5780d16a0 0x7fb5781599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.886+0000 7fb57dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb57815e870 0x7fb5780ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.886+0000 7fb57dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb5780d0820 con 0x7fb57815e870 2024-05-30T17:08:14.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.886+0000 7fb57dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5780d0540 con 0x7fb5780d16a0 2024-05-30T17:08:14.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.886+0000 7fb57dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb5780d4fb0 con 0x7fb5780cf870 2024-05-30T17:08:14.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb57d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb57815e870 0x7fb5780ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb57d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb57815e870 0x7fb5780ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57008/0 (socket says 172.21.15.67:57008) 2024-05-30T17:08:14.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb57d200640 1 -- 172.21.15.67:0/3152832327 learned_addr learned my addr 172.21.15.67:0/3152832327 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-05-30T17:08:14.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb577e00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5780cf870 0x7fb57815f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb577400640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5780d16a0 0x7fb5781599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb57d200640 1 -- 172.21.15.67:0/3152832327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5780cf870 msgr2=0x7fb57815f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb57d200640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5780cf870 0x7fb57815f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb57d200640 1 -- 172.21.15.67:0/3152832327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5780d16a0 msgr2=0x7fb5781599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:14.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb57d200640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5780d16a0 0x7fb5781599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:14.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb57d200640 1 -- 172.21.15.67:0/3152832327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5780b72b0 con 0x7fb57815e870 2024-05-30T17:08:14.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.887+0000 7fb577400640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5780d16a0 0x7fb5781599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-30T17:08:14.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.888+0000 7fb57d200640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb57815e870 0x7fb5780ca280 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fb57000d800 tx=0x7fb57000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:08:14.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.888+0000 7fb56fe00640 1 -- 172.21.15.67:0/3152832327 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000b6c0 con 0x7fb57815e870 2024-05-30T17:08:14.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.888+0000 7fb56fe00640 1 -- 172.21.15.67:0/3152832327 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb570019070 con 0x7fb57815e870 2024-05-30T17:08:14.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.888+0000 7fb56fe00640 1 -- 172.21.15.67:0/3152832327 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000bae0 con 0x7fb57815e870 2024-05-30T17:08:14.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.888+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb578160fe0 con 0x7fb57815e870 2024-05-30T17:08:14.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.888+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5780b7830 con 0x7fb57815e870 2024-05-30T17:08:14.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.889+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5780689e0 con 0x7fb57815e870 2024-05-30T17:08:14.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.890+0000 7fb56fe00640 1 -- 172.21.15.67:0/3152832327 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 52) ==== 93590+0+0 (secure 0 0 0) 0x7fb57000bc80 con 0x7fb57815e870 2024-05-30T17:08:14.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.891+0000 7fb56fe00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb55406e510 0x7fb5540709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-30T17:08:14.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.891+0000 7fb56fe00640 1 -- 172.21.15.67:0/3152832327 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5879+0+0 (secure 0 0 0) 0x7fb57008b080 con 0x7fb57815e870 2024-05-30T17:08:14.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.891+0000 7fb577e00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb55406e510 0x7fb5540709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-30T17:08:14.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.892+0000 7fb577e00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb55406e510 0x7fb5540709d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb568000f60 tx=0x7fb568009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-30T17:08:14.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:14.896+0000 7fb56fe00640 1 -- 172.21.15.67:0/3152832327 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb57005ae80 con 0x7fb57815e870 2024-05-30T17:08:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.139+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5780d4610 con 0x7fb57815e870 2024-05-30T17:08:15.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.140+0000 7fb56fe00640 1 -- 172.21.15.67:0/3152832327 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5186 (secure 0 0 0) 0x7fb57005a780 con 0x7fb57815e870 2024-05-30T17:08:15.140 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-05-30T17:08:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/2461528462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":2461528462},{"type":"v1","addr":"172.21.15.67:6837","nonce":2461528462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6248,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/4229871041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":4229871041},{"type":"v1","addr":"172.21.15.169:6833","nonce":4229871041}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/217648840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":217648840},{"type":"v1","addr":"172.21.15.67:6835","nonce":217648840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-30T17:04:52.409278+0000","modified":"2024-05-30T17:04:55.498697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6937":{"gid":6937,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2504652568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2504652568},{"type":"v1","addr":"172.21.15.169:6835","nonce":2504652568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6937,"qdb_cluster":[6937]},"id":9}]} 2024-05-30T17:08:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 83 2024-05-30T17:08:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb55406e510 msgr2=0x7fb5540709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb55406e510 0x7fb5540709d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb568000f60 tx=0x7fb568009040 comp rx=0 tx=0).stop 2024-05-30T17:08:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb57815e870 msgr2=0x7fb5780ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-30T17:08:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb57815e870 0x7fb5780ca280 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fb57000d800 tx=0x7fb57000dcc0 comp rx=0 tx=0).stop 2024-05-30T17:08:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 shutdown_connections 2024-05-30T17:08:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:6832/46426377,v1:172.21.15.67:6833/46426377] conn(0x7fb55406e510 0x7fb5540709d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb57815e870 0x7fb5780ca280 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5780d16a0 0x7fb5781599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 --2- 172.21.15.67:0/3152832327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5780cf870 0x7fb57815f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-30T17:08:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.142+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 >> 172.21.15.67:0/3152832327 conn(0x7fb57808bfc0 msgr2=0x7fb5780d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-30T17:08:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.143+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 shutdown_connections 2024-05-30T17:08:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-05-30T17:08:15.143+0000 7fb57dc00640 1 -- 172.21.15.67:0/3152832327 wait complete. 2024-05-30T17:08:15.163 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-30T17:08:15.163 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-30T17:08:15.163 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-30T17:08:15.163 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:15.184 INFO:tasks.ceph.mds.a:Stopped 2024-05-30T17:08:15.184 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-30T17:08:15.184 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:15.254 INFO:tasks.ceph.mds.c:Stopped 2024-05-30T17:08:15.255 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-30T17:08:15.255 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:15.281 INFO:tasks.ceph.mds.b:Stopped 2024-05-30T17:08:15.281 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-30T17:08:15.281 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:15.343 INFO:tasks.ceph.mds.d:Stopped 2024-05-30T17:08:15.343 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-30T17:08:15.343 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-30T17:08:15.343 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:17.173 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:08:17.182 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:08:21.445 INFO:tasks.ceph.osd.0:Stopped 2024-05-30T17:08:21.446 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-30T17:08:21.446 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:22.168 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-30T17:08:27.548 INFO:tasks.ceph.osd.1:Stopped 2024-05-30T17:08:27.548 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-30T17:08:27.548 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:28.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-30T17:08:28.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-05-30T17:08:33.651 INFO:tasks.ceph.osd.2:Stopped 2024-05-30T17:08:33.651 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-30T17:08:33.651 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:33.744 INFO:tasks.ceph.osd.3:Stopped 2024-05-30T17:08:33.744 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-30T17:08:33.745 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:34.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-30T17:08:34.286 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-05-30T17:08:39.846 INFO:tasks.ceph.osd.4:Stopped 2024-05-30T17:08:39.847 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-30T17:08:39.847 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:40.092 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-30T17:08:40.092 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-05-30T17:08:45.796 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-30T17:08:45.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-05-30T17:08:45.949 INFO:tasks.ceph.osd.5:Stopped 2024-05-30T17:08:45.949 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-30T17:08:45.949 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:47.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:08:47.319 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:08:51.402 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-30T17:08:51.402 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-05-30T17:08:52.050 INFO:tasks.ceph.osd.6:Stopped 2024-05-30T17:08:52.051 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-30T17:08:52.051 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:52.146 INFO:tasks.ceph.osd.7:Stopped 2024-05-30T17:08:52.146 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-30T17:08:52.147 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-30T17:08:52.147 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:56.805 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-05-30T17:08:56.806 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-05-30T17:08:58.249 INFO:tasks.ceph.mgr.y:Stopped 2024-05-30T17:08:58.249 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-30T17:08:58.249 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:58.347 INFO:tasks.ceph.mgr.x:Stopped 2024-05-30T17:08:58.347 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-30T17:08:58.347 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-30T17:08:58.348 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:58.389 INFO:tasks.ceph.mon.a:Stopped 2024-05-30T17:08:58.390 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-30T17:08:58.390 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:58.424 INFO:tasks.ceph.mon.b:Stopped 2024-05-30T17:08:58.424 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-30T17:08:58.425 INFO:teuthology.orchestra.run:waiting for 300 2024-05-30T17:08:58.511 INFO:tasks.ceph.mon.c:Stopped 2024-05-30T17:08:58.511 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-30T17:08:58.511 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-05-30T17:08:58.628 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T17:08:58.628 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-30T17:08:58.819 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T17:08:58.820 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-30T17:08:58.996 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T17:08:58.997 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-30T17:08:59.156 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T17:08:59.156 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-30T17:08:59.335 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T17:08:59.336 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-30T17:08:59.612 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T17:08:59.612 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-30T17:08:59.768 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T17:08:59.768 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-30T17:08:59.916 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T17:08:59.917 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-30T17:09:00.078 INFO:tasks.ceph:Archiving mon data... 2024-05-30T17:09:00.080 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/data/mon.a.tgz 2024-05-30T17:09:00.080 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-05-30T17:09:00.139 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.rQVPbXythW 2024-05-30T17:09:00.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.rQVPbXythW 2024-05-30T17:09:00.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.rQVPbXythW 2024-05-30T17:09:00.635 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.rQVPbXythW is 1MB 2024-05-30T17:09:00.764 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.rQVPbXythW 2024-05-30T17:09:00.823 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/data/mon.b.tgz 2024-05-30T17:09:00.823 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-05-30T17:09:00.880 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.UUE8jnaU7p 2024-05-30T17:09:00.881 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.UUE8jnaU7p 2024-05-30T17:09:01.226 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.UUE8jnaU7p 2024-05-30T17:09:01.368 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.UUE8jnaU7p is 1MB 2024-05-30T17:09:01.492 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.UUE8jnaU7p 2024-05-30T17:09:01.549 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/data/mon.c.tgz 2024-05-30T17:09:01.549 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-05-30T17:09:01.603 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.w4bne54Ch9 2024-05-30T17:09:01.603 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.w4bne54Ch9 2024-05-30T17:09:01.806 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-05-30T17:09:01.928 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.w4bne54Ch9 2024-05-30T17:09:02.058 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.w4bne54Ch9 is 1MB 2024-05-30T17:09:02.179 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.w4bne54Ch9 2024-05-30T17:09:02.237 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-30T17:09:02.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-30T17:09:02.245 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-30T17:09:02.388 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-30T17:09:02.444 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-30T17:09:02.445 INFO:tasks.ceph:Archiving crash dumps... 2024-05-30T17:09:02.447 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/remote/smithi067/crash 2024-05-30T17:09:02.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-30T17:09:02.539 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/remote/smithi169/crash 2024-05-30T17:09:02.540 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-30T17:09:02.629 INFO:tasks.ceph:Shutting down logrotate 2024-05-30T17:09:02.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:09:02.637 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-30T17:09:02.759 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-30T17:09:02.851 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-30T17:09:02.938 INFO:tasks.ceph:Compressing logs... 2024-05-30T17:09:02.938 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-30T17:09:02.946 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-30T17:09:03.024 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37244.log 2024-05-30T17:09:03.025 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-30T17:09:03.026 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.37244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41988.log 2024-05-30T17:09:03.026 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37244.log.gz 2024-05-30T17:09:03.027 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-30T17:09:03.028 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69487.log 2024-05-30T17:09:03.029 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.1.41988.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-30T17:09:03.029 INFO:teuthology.orchestra.run.smithi169.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.1.41988.log.gz 2024-05-30T17:09:03.030 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-30T17:09:03.030 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50740.log 2024-05-30T17:09:03.030 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69487.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.69487.log.gz 2024-05-30T17:09:03.031 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69403.log 2024-05-30T17:09:03.032 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-30T17:09:03.033 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71641.log 2024-05-30T17:09:03.033 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50740.log.gz 2024-05-30T17:09:03.033 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76289.log 2024-05-30T17:09:03.033 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69403.log.gz 2024-05-30T17:09:03.034 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-30T17:09:03.035 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37291.log 2024-05-30T17:09:03.035 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71641.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.71641.log.gz 2024-05-30T17:09:03.035 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63037.log 2024-05-30T17:09:03.035 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76551.log 2024-05-30T17:09:03.036 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76289.log.gz 2024-05-30T17:09:03.037 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-30T17:09:03.037 INFO:teuthology.orchestra.run.smithi169.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-30T17:09:03.037 INFO:teuthology.orchestra.run.smithi169.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-30T17:09:03.038 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.37291.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-30T17:09:03.038 INFO:teuthology.orchestra.run.smithi169.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37291.log.gz 2024-05-30T17:09:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63037.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.63037.log.gz 2024-05-30T17:09:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65171.log 2024-05-30T17:09:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76551.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76551.log.gz 2024-05-30T17:09:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43626.log 2024-05-30T17:09:03.039 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65171.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65171.log.gz -5 2024-05-30T17:09:03.039 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63391.log 2024-05-30T17:09:03.040 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-30T17:09:03.041 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37492.log 2024-05-30T17:09:03.041 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45503.log 2024-05-30T17:09:03.041 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43626.log.gz 2024-05-30T17:09:03.041 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63391.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63391.log.gz 2024-05-30T17:09:03.042 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65568.log 2024-05-30T17:09:03.042 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45503.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45503.log.gz 2024-05-30T17:09:03.042 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65610.log 2024-05-30T17:09:03.043 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63643.log 2024-05-30T17:09:03.044 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65568.log.gz 2024-05-30T17:09:03.044 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.37492.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37331.log 2024-05-30T17:09:03.046 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70391.log 2024-05-30T17:09:03.046 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65610.log.gz 2024-05-30T17:09:03.046 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63643.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63643.log.gz 2024-05-30T17:09:03.047 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60128.log 2024-05-30T17:09:03.047 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37200.log 2024-05-30T17:09:03.047 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37492.log.gz 2024-05-30T17:09:03.048 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70391.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70391.log.gz 2024-05-30T17:09:03.048 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46448.log 2024-05-30T17:09:03.049 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59266.log 2024-05-30T17:09:03.049 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60128.log.gz 2024-05-30T17:09:03.050 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46448.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46448.log.gz 2024-05-30T17:09:03.050 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62509.log 2024-05-30T17:09:03.050 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.37331.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37331.log.gz 2024-05-30T17:09:03.051 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42444.log 2024-05-30T17:09:03.051 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53512.log 2024-05-30T17:09:03.052 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59266.log.gz 2024-05-30T17:09:03.052 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57739.log 2024-05-30T17:09:03.053 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62509.log.gz 2024-05-30T17:09:03.054 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53512.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53512.log.gz 2024-05-30T17:09:03.054 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72555.log 2024-05-30T17:09:03.055 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.37200.log: /var/log/ceph/ceph-client.3.42444.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37371.log 2024-05-30T17:09:03.055 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37200.log.gz 2024-05-30T17:09:03.056 INFO:teuthology.orchestra.run.smithi169.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.3.42444.log.gz 2024-05-30T17:09:03.057 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50355.log 2024-05-30T17:09:03.057 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57739.log.gz 2024-05-30T17:09:03.057 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2024-05-30T17:09:03.057 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72555.log.gz 2024-05-30T17:09:03.057 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50355.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50355.log.gz 2024-05-30T17:09:03.057 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79547.log 2024-05-30T17:09:03.059 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37452.log 2024-05-30T17:09:03.059 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.37371.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37371.log.gz 2024-05-30T17:09:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77126.log 2024-05-30T17:09:03.060 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz 2024-05-30T17:09:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68542.log 2024-05-30T17:09:03.060 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79547.log.gz 2024-05-30T17:09:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77126.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77126.log.gz 2024-05-30T17:09:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69109.log 2024-05-30T17:09:03.062 INFO:teuthology.orchestra.run.smithi169.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-30T17:09:03.062 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.37452.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-30T17:09:03.062 INFO:teuthology.orchestra.run.smithi169.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37452.log.gz 2024-05-30T17:09:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80855.log 2024-05-30T17:09:03.063 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68542.log.gz 2024-05-30T17:09:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69109.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69109.log.gz 2024-05-30T17:09:03.064 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54307.log 2024-05-30T17:09:03.065 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42546.log 2024-05-30T17:09:03.065 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80855.log.gz 2024-05-30T17:09:03.066 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-30T17:09:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54307.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54307.log.gz 2024-05-30T17:09:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57496.log 2024-05-30T17:09:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71102.log 2024-05-30T17:09:03.067 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42546.log.gz 2024-05-30T17:09:03.068 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37411.log 2024-05-30T17:09:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-05-30T17:09:03.069 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57496.log.gz 2024-05-30T17:09:03.070 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.37411.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37411.log.gz 2024-05-30T17:09:03.070 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71102.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71102.log.gz 2024-05-30T17:09:03.070 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67194.log 2024-05-30T17:09:03.071 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73905.log 2024-05-30T17:09:03.071 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2024-05-30T17:09:03.072 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47805.log 2024-05-30T17:09:03.073 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67194.log.gz 2024-05-30T17:09:03.073 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75774.log 2024-05-30T17:09:03.074 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73905.log.gz 2024-05-30T17:09:03.075 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.47805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67875.log 2024-05-30T17:09:03.076 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62118.log 2024-05-30T17:09:03.076 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75774.log.gz 2024-05-30T17:09:03.077 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67875.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.67875.log.gz 2024-05-30T17:09:03.078 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64526.log 2024-05-30T17:09:03.079 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69662.log 2024-05-30T17:09:03.079 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62118.log.gz 2024-05-30T17:09:03.080 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64526.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64526.log.gz 2024-05-30T17:09:03.080 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55866.log 2024-05-30T17:09:03.081 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73550.log 2024-05-30T17:09:03.081 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69662.log.gz 2024-05-30T17:09:03.082 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72198.log 2024-05-30T17:09:03.083 INFO:teuthology.orchestra.run.smithi067.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55866.log.gz 2024-05-30T17:09:03.084 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41732.log 2024-05-30T17:09:03.084 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73550.log.gz 2024-05-30T17:09:03.085 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77586.log 2024-05-30T17:09:03.085 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72198.log.gz 2024-05-30T17:09:03.086 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79715.log 2024-05-30T17:09:03.087 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41732.log.gz 2024-05-30T17:09:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43458.log 2024-05-30T17:09:03.088 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77586.log.gz 2024-05-30T17:09:03.088 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73718.log 2024-05-30T17:09:03.089 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79715.log.gz 2024-05-30T17:09:03.090 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43458.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43458.log.gzgzip 2024-05-30T17:09:03.090 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53614.log 2024-05-30T17:09:03.091 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63703.log 2024-05-30T17:09:03.092 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73718.log.gz 2024-05-30T17:09:03.092 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64358.log 2024-05-30T17:09:03.093 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53614.log.gz 2024-05-30T17:09:03.093 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81125.log 2024-05-30T17:09:03.094 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63703.log.gz 2024-05-30T17:09:03.095 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46873.log 2024-05-30T17:09:03.095 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64358.log.gz 2024-05-30T17:09:03.096 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81125.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81125.log.gz 2024-05-30T17:09:03.096 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50835.log 2024-05-30T17:09:03.098 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46873.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46873.log.gz 2024-05-30T17:09:03.098 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69786.log 2024-05-30T17:09:03.099 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68857.log 2024-05-30T17:09:03.099 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50835.log.gz 2024-05-30T17:09:03.100 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76467.log 2024-05-30T17:09:03.101 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69786.log.gz 2024-05-30T17:09:03.101 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57580.log 2024-05-30T17:09:03.102 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68857.log.gz 2024-05-30T17:09:03.102 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46575.log 2024-05-30T17:09:03.103 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76467.log.gz 2024-05-30T17:09:03.104 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72681.log 2024-05-30T17:09:03.104 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57580.log.gz 2024-05-30T17:09:03.105 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49767.log 2024-05-30T17:09:03.105 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46575.log.gz 2024-05-30T17:09:03.106 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69967.log 2024-05-30T17:09:03.106 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72681.log.gz 2024-05-30T17:09:03.107 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63895.log 2024-05-30T17:09:03.107 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49767.log.gz 2024-05-30T17:09:03.108 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69967.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69967.log.gz 2024-05-30T17:09:03.108 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52951.log 2024-05-30T17:09:03.109 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80237.log 2024-05-30T17:09:03.109 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63895.log.gz 2024-05-30T17:09:03.110 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45061.log 2024-05-30T17:09:03.111 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52951.log.gz 2024-05-30T17:09:03.112 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48028.log 2024-05-30T17:09:03.112 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80237.log.gz 2024-05-30T17:09:03.113 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45061.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.45061.log.gz -5 2024-05-30T17:09:03.113 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58930.log 2024-05-30T17:09:03.114 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52773.log 2024-05-30T17:09:03.114 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48028.log.gz 2024-05-30T17:09:03.115 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45424.log 2024-05-30T17:09:03.115 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58930.log.gz 2024-05-30T17:09:03.116 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65129.log 2024-05-30T17:09:03.116 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52773.log.gz 2024-05-30T17:09:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45424.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45424.log.gz 2024-05-30T17:09:03.117 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61538.log 2024-05-30T17:09:03.118 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48900.log 2024-05-30T17:09:03.118 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65129.log.gz 2024-05-30T17:09:03.119 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62202.log 2024-05-30T17:09:03.120 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61538.log.gz 2024-05-30T17:09:03.121 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48900.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.48900.log.gz --verbose 2024-05-30T17:09:03.121 INFO:teuthology.orchestra.run.smithi067.stderr: -- /var/log/ceph/ceph-client.admin.46059.log 2024-05-30T17:09:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42867.log 2024-05-30T17:09:03.122 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62202.log.gz 2024-05-30T17:09:03.123 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62750.log 2024-05-30T17:09:03.123 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46059.log.gz 2024-05-30T17:09:03.124 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42867.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-30T17:09:03.124 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42867.log.gz 2024-05-30T17:09:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62750.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.62750.log.gz 2024-05-30T17:09:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64442.log 2024-05-30T17:09:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42548.log 2024-05-30T17:09:03.127 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55360.log 2024-05-30T17:09:03.127 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64442.log.gz 2024-05-30T17:09:03.128 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67248.log 2024-05-30T17:09:03.129 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42548.log.gz 2024-05-30T17:09:03.130 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55530.log 2024-05-30T17:09:03.130 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55360.log.gz 2024-05-30T17:09:03.131 INFO:teuthology.orchestra.run.smithi067.stderr: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-30T17:09:03.131 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67248.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67248.log.gz 2024-05-30T17:09:03.131 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55273.log 2024-05-30T17:09:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67068.log 2024-05-30T17:09:03.133 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55273.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55530.log.gz 2024-05-30T17:09:03.134 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55273.log.gz 2024-05-30T17:09:03.134 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80069.log 2024-05-30T17:09:03.135 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78995.log 2024-05-30T17:09:03.135 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67068.log.gz 2024-05-30T17:09:03.136 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66291.log 2024-05-30T17:09:03.136 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80069.log.gz 2024-05-30T17:09:03.137 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60170.log 2024-05-30T17:09:03.137 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78995.log.gz 2024-05-30T17:09:03.138 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67611.log 2024-05-30T17:09:03.139 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9%/var/log/ceph/ceph-client.admin.60170.log: -- replaced with /var/log/ceph/ceph-client.admin.66291.log.gz 2024-05-30T17:09:03.140 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60170.log.gz 2024-05-30T17:09:03.140 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42874.log 2024-05-30T17:09:03.141 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49326.log 2024-05-30T17:09:03.141 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67611.log.gz 2024-05-30T17:09:03.142 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45975.log 2024-05-30T17:09:03.142 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz 2024-05-30T17:09:03.143 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56896.log 2024-05-30T17:09:03.143 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49326.log.gz 2024-05-30T17:09:03.144 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79351.log 2024-05-30T17:09:03.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56896.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45975.log.gz 2024-05-30T17:09:03.146 INFO:teuthology.orchestra.run.smithi067.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56896.log.gz 2024-05-30T17:09:03.146 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67833.log 2024-05-30T17:09:03.147 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76593.log 2024-05-30T17:09:03.147 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79351.log.gz 2024-05-30T17:09:03.148 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69828.log 2024-05-30T17:09:03.148 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67833.log.gz 2024-05-30T17:09:03.149 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62914.log 2024-05-30T17:09:03.150 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69828.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76593.log.gz 2024-05-30T17:09:03.150 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69828.log.gz 2024-05-30T17:09:03.151 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54638.log 2024-05-30T17:09:03.151 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75443.log 2024-05-30T17:09:03.152 INFO:teuthology.orchestra.run.smithi067.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.62914.log.gz 2024-05-30T17:09:03.153 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60044.log 87.7% 2024-05-30T17:09:03.153 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54638.log.gz 2024-05-30T17:09:03.154 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51970.log 2024-05-30T17:09:03.155 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60044.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75443.log.gz 2024-05-30T17:09:03.155 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60044.log.gz 2024-05-30T17:09:03.155 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78447.log 2024-05-30T17:09:03.156 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69744.log 2024-05-30T17:09:03.156 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51970.log.gz 2024-05-30T17:09:03.157 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78447.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-30T17:09:03.158 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78447.log.gz 2024-05-30T17:09:03.158 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57925.log 2024-05-30T17:09:03.159 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69744.log.gz 2024-05-30T17:09:03.159 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56349.log 2024-05-30T17:09:03.161 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57925.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57925.log.gz 2024-05-30T17:09:03.161 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71574.log 2024-05-30T17:09:03.162 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50965.log 2024-05-30T17:09:03.162 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71574.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56349.log.gz 2024-05-30T17:09:03.163 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71574.log.gz 2024-05-30T17:09:03.163 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64316.log 2024-05-30T17:09:03.165 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65694.log 2024-05-30T17:09:03.165 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64316.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50965.log.gz 2024-05-30T17:09:03.166 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51511.log 2024-05-30T17:09:03.166 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64316.log.gz 2024-05-30T17:09:03.167 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81419.log 2024-05-30T17:09:03.168 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51511.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65694.log.gz 2024-05-30T17:09:03.168 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51511.log.gz 2024-05-30T17:09:03.169 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67426.log 2024-05-30T17:09:03.170 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81545.log 2024-05-30T17:09:03.170 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67426.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81419.log.gz 2024-05-30T17:09:03.171 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67426.log.gz 2024-05-30T17:09:03.171 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70577.log 2024-05-30T17:09:03.173 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54723.log 2024-05-30T17:09:03.173 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1%/var/log/ceph/ceph-client.admin.70577.log: -- replaced with /var/log/ceph/ceph-client.admin.81545.log.gz 2024-05-30T17:09:03.174 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66543.log 2024-05-30T17:09:03.174 INFO:teuthology.orchestra.run.smithi067.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.70577.log.gz 2024-05-30T17:09:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58051.log 2024-05-30T17:09:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66543.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54723.log.gz 2024-05-30T17:09:03.177 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66543.log.gz 2024-05-30T17:09:03.177 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53824.log 2024-05-30T17:09:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36980.log 2024-05-30T17:09:03.179 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53824.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58051.log.gz 2024-05-30T17:09:03.180 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53824.log.gz 2024-05-30T17:09:03.180 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63475.log 2024-05-30T17:09:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.36980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66651.log 2024-05-30T17:09:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63475.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.36980.log.gz 2024-05-30T17:09:03.183 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63475.log.gz 2024-05-30T17:09:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54536.log 2024-05-30T17:09:03.184 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57083.log 2024-05-30T17:09:03.184 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66651.log.gz 2024-05-30T17:09:03.185 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51928.log 2024-05-30T17:09:03.186 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57083.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54536.log.gz 2024-05-30T17:09:03.187 INFO:teuthology.orchestra.run.smithi067.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.57083.log.gz 2024-05-30T17:09:03.187 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66501.log 2024-05-30T17:09:03.188 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76971.log 2024-05-30T17:09:03.189 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51928.log.gz 2024-05-30T17:09:03.189 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66501.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66501.log.gz 2024-05-30T17:09:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42880.log 2024-05-30T17:09:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72156.log 2024-05-30T17:09:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42880.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76971.log.gz 2024-05-30T17:09:03.192 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42880.log.gz 2024-05-30T17:09:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47041.log 2024-05-30T17:09:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54016.log 2024-05-30T17:09:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47041.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72156.log.gz 2024-05-30T17:09:03.195 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47041.log.gz 2024-05-30T17:09:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68229.log 2024-05-30T17:09:03.196 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57125.log 2024-05-30T17:09:03.197 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68229.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54016.log.gz 2024-05-30T17:09:03.198 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74766.log 2024-05-30T17:09:03.198 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68229.log.gz 2024-05-30T17:09:03.199 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65992.log 2024-05-30T17:09:03.199 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57125.log.gz 2024-05-30T17:09:03.200 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77330.log 2024-05-30T17:09:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65992.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74766.log.gz 2024-05-30T17:09:03.201 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65992.log.gz 2024-05-30T17:09:03.202 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62873.log 2024-05-30T17:09:03.203 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42871.log 2024-05-30T17:09:03.204 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77330.log.gz 2024-05-30T17:09:03.205 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62873.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.62873.log.gz 2024-05-30T17:09:03.205 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65255.log 2024-05-30T17:09:03.206 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56210.log 2024-05-30T17:09:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65255.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42871.log.gz 2024-05-30T17:09:03.207 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65255.log.gz 2024-05-30T17:09:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61484.log 2024-05-30T17:09:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.56210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42879.log 2024-05-30T17:09:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61484.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61484.log.gz 2024-05-30T17:09:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71701.log 2024-05-30T17:09:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81209.log 2024-05-30T17:09:03.212 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42879.log.gz 2024-05-30T17:09:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52706.log 2024-05-30T17:09:03.213 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71701.log.gz 2024-05-30T17:09:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65358.log 2024-05-30T17:09:03.215 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81209.log.gz 2024-05-30T17:09:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52706.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52706.log.gz 2024-05-30T17:09:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46406.log 2024-05-30T17:09:03.217 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65358.log: gzip -5 --verbose 88.4% -- -- replaced with /var/log/ceph/ceph-client.admin.65358.log.gz /var/log/ceph/ceph-client.admin.61055.log 2024-05-30T17:09:03.218 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-05-30T17:09:03.218 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70265.log 2024-05-30T17:09:03.219 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46406.log.gz 2024-05-30T17:09:03.220 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61808.log 2024-05-30T17:09:03.220 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61055.log.gz 2024-05-30T17:09:03.221 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68043.log 2024-05-30T17:09:03.222 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70265.log.gz 2024-05-30T17:09:03.222 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60254.log 2024-05-30T17:09:03.223 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61808.log.gz 2024-05-30T17:09:03.224 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68043.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-30T17:09:03.224 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68043.log.gz 2024-05-30T17:09:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68398.log 2024-05-30T17:09:03.225 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60254.log.gz 2024-05-30T17:09:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78639.log 2024-05-30T17:09:03.228 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68398.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68398.log.gz 2024-05-30T17:09:03.228 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55740.log 2024-05-30T17:09:03.229 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69025.log 2024-05-30T17:09:03.229 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78639.log.gz 2024-05-30T17:09:03.230 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42542.log 2024-05-30T17:09:03.231 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55740.log.gz 2024-05-30T17:09:03.231 INFO:teuthology.orchestra.run.smithi067.stderr: 89.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-30T17:09:03.232 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79393.log 2024-05-30T17:09:03.232 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69025.log.gz 2024-05-30T17:09:03.232 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49368.log 2024-05-30T17:09:03.233 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42542.log.gz 2024-05-30T17:09:03.234 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70009.log 2024-05-30T17:09:03.234 INFO:teuthology.orchestra.run.smithi067.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79393.log.gz 2024-05-30T17:09:03.235 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77372.log 2024-05-30T17:09:03.235 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49368.log.gz 2024-05-30T17:09:03.236 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49182.log 2024-05-30T17:09:03.237 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70009.log.gz 2024-05-30T17:09:03.238 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48816.log 2024-05-30T17:09:03.238 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77372.log.gz 2024-05-30T17:09:03.239 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57269.log 2024-05-30T17:09:03.239 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49182.log.gz 2024-05-30T17:09:03.240 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58786.log 2024-05-30T17:09:03.240 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48816.log.gz 2024-05-30T17:09:03.242 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58093.log 2024-05-30T17:09:03.242 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57269.log.gz 2024-05-30T17:09:03.243 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43584.log 2024-05-30T17:09:03.243 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58786.log.gz 2024-05-30T17:09:03.244 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41904.log 2024-05-30T17:09:03.244 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58093.log.gz 2024-05-30T17:09:03.246 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59863.log 2024-05-30T17:09:03.246 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43584.log.gz 2024-05-30T17:09:03.247 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63559.log 2024-05-30T17:09:03.247 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41904.log.gz 2024-05-30T17:09:03.248 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53932.log 2024-05-30T17:09:03.248 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59863.log.gz 2024-05-30T17:09:03.249 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49725.log 2024-05-30T17:09:03.249 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63559.log.gz 2024-05-30T17:09:03.250 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46143.log 2024-05-30T17:09:03.251 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53932.log.gz 2024-05-30T17:09:03.251 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55572.log 2024-05-30T17:09:03.252 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49725.log.gz 2024-05-30T17:09:03.253 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59350.log 2024-05-30T17:09:03.253 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46143.log.gz 2024-05-30T17:09:03.254 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60296.log 2024-05-30T17:09:03.254 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55572.log.gz 2024-05-30T17:09:03.256 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74157.log 2024-05-30T17:09:03.256 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59350.log.gz 2024-05-30T17:09:03.257 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61580.log 2024-05-30T17:09:03.257 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60296.log.gz 2024-05-30T17:09:03.258 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2024-05-30T17:09:03.258 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74157.log.gz 2024-05-30T17:09:03.259 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79631.log 2024-05-30T17:09:03.259 INFO:teuthology.orchestra.run.smithi067.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61580.log.gz 2024-05-30T17:09:03.260 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73634.log 2024-05-30T17:09:03.261 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz 2024-05-30T17:09:03.262 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46491.log 2024-05-30T17:09:03.262 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79631.log.gz 2024-05-30T17:09:03.263 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42883.log 2024-05-30T17:09:03.263 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73634.log.gz 2024-05-30T17:09:03.264 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64736.log 2024-05-30T17:09:03.265 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46491.log.gz 2024-05-30T17:09:03.265 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48646.log 2024-05-30T17:09:03.266 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42883.log.gz 2024-05-30T17:09:03.266 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61622.log 2024-05-30T17:09:03.267 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64736.log.gz 2024-05-30T17:09:03.268 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71186.log 2024-05-30T17:09:03.268 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48646.log.gz 2024-05-30T17:09:03.269 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79309.log 2024-05-30T17:09:03.269 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61622.log.gz 2024-05-30T17:09:03.270 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63349.log 2024-05-30T17:09:03.271 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71186.log.gz 2024-05-30T17:09:03.272 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56475.log 2024-05-30T17:09:03.272 INFO:teuthology.orchestra.run.smithi067.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79309.log.gz 2024-05-30T17:09:03.273 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51217.log 2024-05-30T17:09:03.273 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63349.log.gz 2024-05-30T17:09:03.274 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73376.log 2024-05-30T17:09:03.275 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56475.log.gz 2024-05-30T17:09:03.276 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70307.log 2024-05-30T17:09:03.276 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51217.log.gz 2024-05-30T17:09:03.277 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77838.log 2024-05-30T17:09:03.277 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73376.log.gz 2024-05-30T17:09:03.278 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70097.log 2024-05-30T17:09:03.278 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70307.log.gz 2024-05-30T17:09:03.279 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55656.log 2024-05-30T17:09:03.279 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77838.log.gz 2024-05-30T17:09:03.280 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59434.log 2024-05-30T17:09:03.281 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70097.log.gz 2024-05-30T17:09:03.281 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79128.log 2024-05-30T17:09:03.282 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55656.log.gz 2024-05-30T17:09:03.283 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68584.log 2024-05-30T17:09:03.283 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59434.log.gz 2024-05-30T17:09:03.284 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41946.log 2024-05-30T17:09:03.284 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79128.log.gz 2024-05-30T17:09:03.285 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49098.log 2024-05-30T17:09:03.286 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68584.log.gz 2024-05-30T17:09:03.286 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41946.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-30T17:09:03.287 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41946.log.gz 2024-05-30T17:09:03.288 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80939.log 2024-05-30T17:09:03.288 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49098.log.gz 2024-05-30T17:09:03.289 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63307.log 2024-05-30T17:09:03.290 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77988.log 2024-05-30T17:09:03.291 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80939.log.gz 2024-05-30T17:09:03.292 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79464.log 2024-05-30T17:09:03.292 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63307.log.gz 2024-05-30T17:09:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43710.log 2024-05-30T17:09:03.293 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77988.log.gz 2024-05-30T17:09:03.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64891.log 2024-05-30T17:09:03.295 INFO:teuthology.orchestra.run.smithi067.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.79464.log.gz 2024-05-30T17:09:03.296 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47251.log 2024-05-30T17:09:03.297 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64891.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43710.log.gz 2024-05-30T17:09:03.297 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76092.log 2024-05-30T17:09:03.297 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64891.log.gz 2024-05-30T17:09:03.298 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74904.log 2024-05-30T17:09:03.298 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47251.log.gz 2024-05-30T17:09:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42536.log 2024-05-30T17:09:03.300 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76092.log.gz 2024-05-30T17:09:03.301 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64148.log 2024-05-30T17:09:03.301 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74904.log.gz 2024-05-30T17:09:03.302 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49851.log 2024-05-30T17:09:03.303 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42536.log.gz 2024-05-30T17:09:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51361.log 2024-05-30T17:09:03.304 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64148.log.gz 2024-05-30T17:09:03.305 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62555.log 2024-05-30T17:09:03.305 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49851.log.gz 2024-05-30T17:09:03.306 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72723.log 2024-05-30T17:09:03.306 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51361.log.gz 2024-05-30T17:09:03.307 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76176.log 2024-05-30T17:09:03.307 INFO:teuthology.orchestra.run.smithi067.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.62555.log.gz 2024-05-30T17:09:03.308 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72471.log 2024-05-30T17:09:03.309 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72723.log.gz 2024-05-30T17:09:03.310 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76176.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.67665.log 2024-05-30T17:09:03.310 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76176.log.gz 2024-05-30T17:09:03.311 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48590.log 2024-05-30T17:09:03.311 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72471.log.gz 2024-05-30T17:09:03.312 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67665.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.67665.log.gz -5 2024-05-30T17:09:03.313 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73462.log 2024-05-30T17:09:03.313 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73120.log 2024-05-30T17:09:03.314 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48590.log.gz 2024-05-30T17:09:03.315 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74808.log 2024-05-30T17:09:03.315 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73462.log.gz 2024-05-30T17:09:03.316 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80381.log 2024-05-30T17:09:03.317 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73120.log.gz 2024-05-30T17:09:03.318 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75401.log 2024-05-30T17:09:03.318 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74808.log.gz 2024-05-30T17:09:03.319 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61664.log 2024-05-30T17:09:03.319 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80381.log.gz 2024-05-30T17:09:03.320 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76026.log 2024-05-30T17:09:03.320 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75401.log.gz 2024-05-30T17:09:03.322 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70223.log 2024-05-30T17:09:03.322 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61664.log.gz 2024-05-30T17:09:03.323 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53698.log 2024-05-30T17:09:03.323 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76026.log.gz 2024-05-30T17:09:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50784.log 2024-05-30T17:09:03.324 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70223.log.gz 2024-05-30T17:09:03.326 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58009.log 2024-05-30T17:09:03.326 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53698.log.gz 2024-05-30T17:09:03.327 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50784.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58369.log 2024-05-30T17:09:03.327 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50784.log.gz 2024-05-30T17:09:03.328 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58009.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30965.log 2024-05-30T17:09:03.328 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58009.log.gz 2024-05-30T17:09:03.329 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63079.log 2024-05-30T17:09:03.330 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58369.log.gz 2024-05-30T17:09:03.330 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.tmp-client.admin.30965.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30965.log.gz 2024-05-30T17:09:03.331 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59182.log 2024-05-30T17:09:03.332 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73761.log 2024-05-30T17:09:03.332 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63079.log.gz 2024-05-30T17:09:03.333 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51091.log 2024-05-30T17:09:03.333 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59182.log.gz 2024-05-30T17:09:03.334 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44951.log 2024-05-30T17:09:03.335 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73761.log.gz 2024-05-30T17:09:03.336 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50877.log 2024-05-30T17:09:03.336 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51091.log.gz 2024-05-30T17:09:03.337 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79225.log 2024-05-30T17:09:03.337 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44951.log.gz 2024-05-30T17:09:03.338 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51007.log 2024-05-30T17:09:03.338 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50877.log.gz 2024-05-30T17:09:03.339 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61892.log 2024-05-30T17:09:03.339 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79225.log.gz 2024-05-30T17:09:03.340 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79841.log 2024-05-30T17:09:03.341 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51007.log.gz 2024-05-30T17:09:03.342 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64274.log 2024-05-30T17:09:03.342 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61892.log.gz 2024-05-30T17:09:03.343 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76509.log 2024-05-30T17:09:03.343 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79841.log.gz 2024-05-30T17:09:03.344 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2024-05-30T17:09:03.344 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64274.log.gz 2024-05-30T17:09:03.346 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56391.log 2024-05-30T17:09:03.346 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76509.log.gz 2024-05-30T17:09:03.347 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63433.log 2024-05-30T17:09:03.347 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2024-05-30T17:09:03.348 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42240.log 2024-05-30T17:09:03.348 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56391.log.gz 2024-05-30T17:09:03.349 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58285.log 2024-05-30T17:09:03.349 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63433.log.gz 2024-05-30T17:09:03.350 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55404.log 2024-05-30T17:09:03.350 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42240.log.gz 2024-05-30T17:09:03.352 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58285.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.56761.log -- replaced with /var/log/ceph/ceph-client.admin.58285.log.gz 2024-05-30T17:09:03.352 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-05-30T17:09:03.352 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45629.log 2024-05-30T17:09:03.353 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55404.log.gz 2024-05-30T17:09:03.354 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73036.log 2024-05-30T17:09:03.354 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56761.log.gz 2024-05-30T17:09:03.355 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70055.log 2024-05-30T17:09:03.355 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45629.log.gz 2024-05-30T17:09:03.356 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81587.log 2024-05-30T17:09:03.356 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73036.log.gz 2024-05-30T17:09:03.357 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72868.log 2024-05-30T17:09:03.357 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70055.log.gz 2024-05-30T17:09:03.358 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68127.log 2024-05-30T17:09:03.359 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81587.log.gz 2024-05-30T17:09:03.360 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72868.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-30T17:09:03.360 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72868.log.gz 2024-05-30T17:09:03.361 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68127.log: gzip -5 --verbose 88.6% -- -- replaced with /var/log/ceph/ceph-client.admin.68127.log.gz /var/log/ceph/ceph-client.admin.60821.log 2024-05-30T17:09:03.361 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-05-30T17:09:03.362 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80279.log 2024-05-30T17:09:03.363 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46617.log 2024-05-30T17:09:03.364 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60821.log.gz 2024-05-30T17:09:03.364 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76383.log 2024-05-30T17:09:03.365 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80279.log.gz 2024-05-30T17:09:03.366 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46617.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46617.log.gz 2024-05-30T17:09:03.366 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47527.log 2024-05-30T17:09:03.367 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76383.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76383.log.gz 2024-05-30T17:09:03.367 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54494.log 2024-05-30T17:09:03.368 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58972.log 2024-05-30T17:09:03.369 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47527.log.gz 2024-05-30T17:09:03.370 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54494.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54494.log.gz 2024-05-30T17:09:03.370 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53256.log 2024-05-30T17:09:03.371 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58972.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58972.log.gz 2024-05-30T17:09:03.372 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53428.log 2024-05-30T17:09:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50923.log 2024-05-30T17:09:03.373 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53256.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53256.log.gz 2024-05-30T17:09:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53428.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53428.log.gz 2024-05-30T17:09:03.375 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46743.log 2024-05-30T17:09:03.375 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50923.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50923.log.gz 2024-05-30T17:09:03.376 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71928.log 2024-05-30T17:09:03.377 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.46743.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69151.log 2024-05-30T17:09:03.378 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46743.log.gz 2024-05-30T17:09:03.378 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71928.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71928.log.gz 2024-05-30T17:09:03.379 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75078.log 2024-05-30T17:09:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69151.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69151.log.gz 2024-05-30T17:09:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53974.log 2024-05-30T17:09:03.381 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81251.log 2024-05-30T17:09:03.382 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75078.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75078.log.gz 2024-05-30T17:09:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53974.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53974.log.gz 2024-05-30T17:09:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58327.log 2024-05-30T17:09:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81251.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81251.log.gz 2024-05-30T17:09:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80681.log 2024-05-30T17:09:03.385 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75214.log 2024-05-30T17:09:03.386 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58327.log.gz 2024-05-30T17:09:03.387 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80681.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80681.log.gz 2024-05-30T17:09:03.387 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49518.log 2024-05-30T17:09:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75214.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75214.log.gz 2024-05-30T17:09:03.389 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76845.log 2024-05-30T17:09:03.390 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60905.log 2024-05-30T17:09:03.390 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49518.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49518.log.gz 2024-05-30T17:09:03.391 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76845.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76845.log.gz 2024-05-30T17:09:03.392 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42884.log 2024-05-30T17:09:03.393 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60905.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60905.log.gz 2024-05-30T17:09:03.393 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77502.log 2024-05-30T17:09:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42884.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-30T17:09:03.395 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42884.log.gz 2024-05-30T17:09:03.395 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77502.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77502.log.gz 2024-05-30T17:09:03.396 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62955.log 2024-05-30T17:09:03.398 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-30T17:09:03.399 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62955.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.62955.log.gz 2024-05-30T17:09:03.402 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48126.log 2024-05-30T17:09:03.405 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50450.log 2024-05-30T17:09:03.405 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50653.log 2024-05-30T17:09:03.405 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48126.log.gz 2024-05-30T17:09:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50450.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50450.log.gz 2024-05-30T17:09:03.409 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57865.log 2024-05-30T17:09:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57697.log 2024-05-30T17:09:03.410 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50653.log.gz 2024-05-30T17:09:03.413 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57865.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57865.log.gz 2024-05-30T17:09:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49452.log 2024-05-30T17:09:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57697.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57697.log.gz 2024-05-30T17:09:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49452.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2024-05-30T17:09:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81335.log 2024-05-30T17:09:03.424 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79267.log 2024-05-30T17:09:03.424 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76635.log 2024-05-30T17:09:03.424 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81335.log.gz 2024-05-30T17:09:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79267.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79267.log.gz 2024-05-30T17:09:03.430 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44255.log 2024-05-30T17:09:03.430 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76635.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76635.log.gz 2024-05-30T17:09:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.44255.log: 90.5% -- replaced with /var/log/ceph/ceph-client.0.44255.log.gz 2024-05-30T17:09:03.432 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43812.log 2024-05-30T17:09:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43812.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43812.log.gz 2024-05-30T17:09:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62060.log 2024-05-30T17:09:03.439 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77544.log 2024-05-30T17:09:03.439 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62060.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62060.log.gz 2024-05-30T17:09:03.440 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75529.log 2024-05-30T17:09:03.441 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77544.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77544.log.gz 2024-05-30T17:09:03.444 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69916.log 2024-05-30T17:09:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75529.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75529.log.gz 2024-05-30T17:09:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60212.log 2024-05-30T17:09:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69916.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69916.log.gz 2024-05-30T17:09:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64021.log 2024-05-30T17:09:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60212.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60212.log.gz 2024-05-30T17:09:03.457 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64021.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64021.log.gz 2024-05-30T17:09:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74073.log 2024-05-30T17:09:03.463 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65078.log 2024-05-30T17:09:03.463 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43668.log 2024-05-30T17:09:03.463 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74073.log.gz 2024-05-30T17:09:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65078.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65078.log.gz 2024-05-30T17:09:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-30T17:09:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43668.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43668.log.gz 2024-05-30T17:09:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41988.log 2024-05-30T17:09:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65789.log 2024-05-30T17:09:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41988.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41988.log.gz 2024-05-30T17:09:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47461.log 2024-05-30T17:09:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65789.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65789.log.gz 2024-05-30T17:09:03.481 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52012.log 2024-05-30T17:09:03.483 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47461.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.47461.log.gz 2024-05-30T17:09:03.485 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72597.log 2024-05-30T17:09:03.485 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52012.log.gz 2024-05-30T17:09:03.489 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55150.log 2024-05-30T17:09:03.491 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72597.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72597.log.gz 2024-05-30T17:09:03.494 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55150.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55150.log.gz 2024-05-30T17:09:03.496 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77628.log 2024-05-30T17:09:03.499 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54807.log 2024-05-30T17:09:03.500 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77628.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77628.log.gz 2024-05-30T17:09:03.501 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54975.log 2024-05-30T17:09:03.503 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54807.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54807.log.gz 2024-05-30T17:09:03.508 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81377.log 2024-05-30T17:09:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54975.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54975.log.gz 2024-05-30T17:09:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47125.log 2024-05-30T17:09:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81377.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81377.log.gz 2024-05-30T17:09:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76134.log 2024-05-30T17:09:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47125.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47125.log.gz 2024-05-30T17:09:03.523 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76134.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76134.log.gz 2024-05-30T17:09:03.525 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71498.log 2024-05-30T17:09:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53386.log 2024-05-30T17:09:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71498.log.gz 2024-05-30T17:09:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42114.log 2024-05-30T17:09:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53386.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53386.log.gz 2024-05-30T17:09:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64232.log 2024-05-30T17:09:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42114.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42114.log.gz 2024-05-30T17:09:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48774.log 2024-05-30T17:09:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64232.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64232.log.gz 2024-05-30T17:09:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66123.log 2024-05-30T17:09:03.543 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48774.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48774.log.gz 2024-05-30T17:09:03.545 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62018.log 2024-05-30T17:09:03.546 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66123.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66123.log.gz 2024-05-30T17:09:03.549 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59140.log 2024-05-30T17:09:03.549 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62018.log.gz 2024-05-30T17:09:03.551 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68440.log 2024-05-30T17:09:03.553 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43205.log 2024-05-30T17:09:03.555 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68440.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59140.log.gz 2024-05-30T17:09:03.555 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68440.log.gz 2024-05-30T17:09:03.558 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57017.log 2024-05-30T17:09:03.559 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47377.log 2024-05-30T17:09:03.560 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43205.log.gz 2024-05-30T17:09:03.561 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49977.log 2024-05-30T17:09:03.563 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47377.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47377.log.gz 2024-05-30T17:09:03.564 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57017.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57017.log.gz 2024-05-30T17:09:03.565 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76803.log 2024-05-30T17:09:03.567 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49977.log: 88.4%/var/log/ceph/ceph-client.admin.76803.log: -- replaced with /var/log/ceph/ceph-client.admin.49977.log.gz 2024-05-30T17:09:03.568 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43220.log 2024-05-30T17:09:03.571 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51789.log 2024-05-30T17:09:03.571 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76803.log.gz 2024-05-30T17:09:03.573 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43220.log: /var/log/ceph/ceph-client.0.51789.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43220.log.gz 2024-05-30T17:09:03.575 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51049.log 2024-05-30T17:09:03.577 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55488.log 2024-05-30T17:09:03.577 INFO:teuthology.orchestra.run.smithi169.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-30T17:09:03.578 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80423.log 2024-05-30T17:09:03.584 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51049.log: /var/log/ceph/ceph-client.admin.55488.log: /var/log/ceph/ceph-client.admin.80423.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80423.log.gz 2024-05-30T17:09:03.584 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51049.log.gz 2024-05-30T17:09:03.584 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55488.log.gz 2024-05-30T17:09:03.584 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67749.log 2024-05-30T17:09:03.589 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50611.log 2024-05-30T17:09:03.589 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67749.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67749.log.gz 2024-05-30T17:09:03.590 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61275.log 2024-05-30T17:09:03.592 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50611.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50611.log.gz 2024-05-30T17:09:03.597 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43213.log 2024-05-30T17:09:03.598 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61275.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61275.log.gz 2024-05-30T17:09:03.599 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58453.log 2024-05-30T17:09:03.601 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43213.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43213.log.gz 2024-05-30T17:09:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58453.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58453.log.gz 2024-05-30T17:09:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80507.log 2024-05-30T17:09:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46915.log 2024-05-30T17:09:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80507.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80507.log.gz 2024-05-30T17:09:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53108.log 2024-05-30T17:09:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46915.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46915.log.gz 2024-05-30T17:09:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49140.log 2024-05-30T17:09:03.619 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53108.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53108.log.gz 2024-05-30T17:09:03.621 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49140.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49140.log.gz 2024-05-30T17:09:03.622 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47335.log 2024-05-30T17:09:03.627 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54452.log 2024-05-30T17:09:03.628 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47335.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47335.log.gz 2024-05-30T17:09:03.631 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70643.log 2024-05-30T17:09:03.632 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54452.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54452.log.gz 2024-05-30T17:09:03.633 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79673.log 2024-05-30T17:09:03.634 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70643.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70643.log.gz 2024-05-30T17:09:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42366.log 2024-05-30T17:09:03.638 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79673.log.gz 2024-05-30T17:09:03.640 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47209.log 2024-05-30T17:09:03.641 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42366.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42366.log.gz 2024-05-30T17:09:03.643 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62832.log 2024-05-30T17:09:03.644 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47209.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47209.log.gz 2024-05-30T17:09:03.646 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70535.log 2024-05-30T17:09:03.646 INFO:teuthology.orchestra.run.smithi067.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.62832.log.gz 2024-05-30T17:09:03.649 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81629.log 2024-05-30T17:09:03.650 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70535.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70535.log.gz 2024-05-30T17:09:03.652 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81629.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81629.log.gz 2024-05-30T17:09:03.654 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78531.log 2024-05-30T17:09:03.656 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46701.log 2024-05-30T17:09:03.658 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52839.log 2024-05-30T17:09:03.658 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78531.log.gz 2024-05-30T17:09:03.659 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46701.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46701.log.gz 2024-05-30T17:09:03.663 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53890.log 2024-05-30T17:09:03.664 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52839.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52839.log.gz 2024-05-30T17:09:03.666 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42541.log 2024-05-30T17:09:03.667 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53890.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53890.log.gz 2024-05-30T17:09:03.669 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43216.log 2024-05-30T17:09:03.670 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42541.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42541.log.gz 2024-05-30T17:09:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64190.log 2024-05-30T17:09:03.675 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43216.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43216.log.gz 2024-05-30T17:09:03.677 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57311.log 2024-05-30T17:09:03.677 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64190.log.gz 2024-05-30T17:09:03.680 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74543.log 2024-05-30T17:09:03.682 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57311.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57311.log.gz 2024-05-30T17:09:03.685 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.74543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72639.log 2024-05-30T17:09:03.689 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-30T17:09:03.690 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72639.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72639.log.gz 2024-05-30T17:09:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62791.log 2024-05-30T17:09:03.694 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.62791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72072.log 2024-05-30T17:09:03.694 INFO:teuthology.orchestra.run.smithi067.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.62791.log.gz 2024-05-30T17:09:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68941.log 2024-05-30T17:09:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72072.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72072.log.gz 2024-05-30T17:09:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68941.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68941.log.gz 2024-05-30T17:09:03.701 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37245.log 2024-05-30T17:09:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66079.log 2024-05-30T17:09:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.37245.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37245.log.gz 2024-05-30T17:09:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81713.log 2024-05-30T17:09:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66079.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66079.log.gz 2024-05-30T17:09:03.709 INFO:teuthology.orchestra.run.smithi169.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-30T17:09:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73419.log 2024-05-30T17:09:03.710 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81713.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81713.log.gz 2024-05-30T17:09:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58828.log 2024-05-30T17:09:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73419.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73419.log.gz 2024-05-30T17:09:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58828.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58828.log.gz 2024-05-30T17:09:03.717 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59998.log 2024-05-30T17:09:03.721 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45271.log 2024-05-30T17:09:03.722 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59998.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59998.log.gz 2024-05-30T17:09:03.724 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71970.log 2024-05-30T17:09:03.725 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45271.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45271.log.gz 2024-05-30T17:09:03.728 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75858.log 2024-05-30T17:09:03.729 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71970.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71970.log.gz 2024-05-30T17:09:03.733 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72513.log 2024-05-30T17:09:03.733 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75858.log.gz 2024-05-30T17:09:03.736 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49809.log 2024-05-30T17:09:03.737 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72513.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72513.log.gz 2024-05-30T17:09:03.741 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49809.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49809.log.gz 2024-05-30T17:09:03.742 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72030.log 2024-05-30T17:09:03.746 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61001.log 2024-05-30T17:09:03.747 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72030.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72030.log.gz 2024-05-30T17:09:03.751 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64484.log 2024-05-30T17:09:03.752 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61001.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61001.log.gz 2024-05-30T17:09:03.755 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46957.log 2024-05-30T17:09:03.757 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64484.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64484.log.gz 2024-05-30T17:09:03.757 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73506.log 2024-05-30T17:09:03.758 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46957.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46957.log.gz 2024-05-30T17:09:03.762 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75281.log 2024-05-30T17:09:03.762 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73506.log.gz 2024-05-30T17:09:03.765 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81671.log 2024-05-30T17:09:03.766 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75281.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75281.log.gz 2024-05-30T17:09:03.768 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81671.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81671.log.gz 2024-05-30T17:09:03.770 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62305.log 2024-05-30T17:09:03.774 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64694.log 2024-05-30T17:09:03.775 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62305.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62305.log.gz 2024-05-30T17:09:03.776 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49266.log 2024-05-30T17:09:03.780 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.64694.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43542.log 2024-05-30T17:09:03.780 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64694.log.gz 2024-05-30T17:09:03.781 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49266.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49266.log.gz 2024-05-30T17:09:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61706.log 2024-05-30T17:09:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43542.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43542.log.gz 2024-05-30T17:09:03.788 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71760.log 2024-05-30T17:09:03.789 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61706.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61706.log.gz 2024-05-30T17:09:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71760.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71760.log.gz 2024-05-30T17:09:03.793 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71844.log 2024-05-30T17:09:03.795 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79883.log 2024-05-30T17:09:03.797 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71844.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71844.log.gz 2024-05-30T17:09:03.797 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73295.log 2024-05-30T17:09:03.798 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79883.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79883.log.gz 2024-05-30T17:09:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77796.log 2024-05-30T17:09:03.803 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67707.log 2024-05-30T17:09:03.804 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77796.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77796.log.gz 2024-05-30T17:09:03.805 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73295.log.gz 2024-05-30T17:09:03.806 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69445.log 2024-05-30T17:09:03.807 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67707.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67707.log.gz 2024-05-30T17:09:03.810 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58219.log 2024-05-30T17:09:03.811 INFO:teuthology.orchestra.run.smithi067.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-30T17:09:03.812 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69445.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69445.log.gz 2024-05-30T17:09:03.813 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58219.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58219.log.gz 2024-05-30T17:09:03.814 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65869.log 2024-05-30T17:09:03.816 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77013.log 2024-05-30T17:09:03.818 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76719.log 2024-05-30T17:09:03.819 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77013.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77013.log.gz 2024-05-30T17:09:03.821 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65869.log.gz 2024-05-30T17:09:03.823 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45366.log 2024-05-30T17:09:03.824 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76719.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76719.log.gz 2024-05-30T17:09:03.827 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45366.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45366.log.gz 2024-05-30T17:09:03.828 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79183.log 2024-05-30T17:09:03.832 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70451.log 2024-05-30T17:09:03.833 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79183.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79183.log.gz 2024-05-30T17:09:03.837 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60863.log 2024-05-30T17:09:03.838 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70451.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70451.log.gz 2024-05-30T17:09:03.839 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79799.log 2024-05-30T17:09:03.840 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60863.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60863.log.gz 2024-05-30T17:09:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73215.log 2024-05-30T17:09:03.844 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79799.log.gz 2024-05-30T17:09:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-05-30T17:09:03.849 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73215.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73215.log.gz 2024-05-30T17:09:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70181.log 2024-05-30T17:09:03.855 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51445.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-05-30T17:09:03.856 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70181.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70181.log.gz 2024-05-30T17:09:03.857 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54681.log 2024-05-30T17:09:03.860 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46787.log 2024-05-30T17:09:03.861 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54681.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54681.log.gz 2024-05-30T17:09:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75347.log 2024-05-30T17:09:03.865 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46787.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-05-30T17:09:03.866 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74199.log 2024-05-30T17:09:03.867 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75347.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.75347.log.gz 2024-05-30T17:09:03.870 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74199.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74199.log.gz 2024-05-30T17:09:03.872 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43500.log 2024-05-30T17:09:03.876 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52576.log 2024-05-30T17:09:03.877 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43500.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43500.log.gz 2024-05-30T17:09:03.881 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53572.log 2024-05-30T17:09:03.882 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52576.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52576.log.gz 2024-05-30T17:09:03.886 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-05-30T17:09:03.887 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53572.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53572.log.gz 2024-05-30T17:09:03.889 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61748.log 2024-05-30T17:09:03.890 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45688.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gz 2024-05-30T17:09:03.892 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-05-30T17:09:03.893 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61748.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61748.log.gz 2024-05-30T17:09:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42198.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-05-30T17:09:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67152.log 2024-05-30T17:09:03.901 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42324.log 2024-05-30T17:09:03.902 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67152.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67152.log.gz 2024-05-30T17:09:03.903 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51319.log 2024-05-30T17:09:03.905 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81837.log 2024-05-30T17:09:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51319.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51319.log.gz 2024-05-30T17:09:03.907 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56529.log 2024-05-30T17:09:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81837.log: /var/log/ceph/ceph-client.admin.42324.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81837.log.gz 2024-05-30T17:09:03.908 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42324.log.gz 2024-05-30T17:09:03.910 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59732.log 2024-05-30T17:09:03.911 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56529.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56529.log.gz 2024-05-30T17:09:03.913 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75984.log 2024-05-30T17:09:03.919 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59732.log: /var/log/ceph/ceph-client.admin.75984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43416.log 2024-05-30T17:09:03.919 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75984.log.gz 2024-05-30T17:09:03.919 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59732.log.gz 2024-05-30T17:09:03.920 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58411.log 2024-05-30T17:09:03.921 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43416.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43416.log.gz 2024-05-30T17:09:03.924 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72994.log 2024-05-30T17:09:03.924 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58411.log.gz 2024-05-30T17:09:03.926 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52108.log 2024-05-30T17:09:03.927 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72994.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72994.log.gz 2024-05-30T17:09:03.932 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70493.log 2024-05-30T17:09:03.934 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68085.log 2024-05-30T17:09:03.934 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52108.log.gz 2024-05-30T17:09:03.935 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70493.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70493.log.gz 2024-05-30T17:09:03.939 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78489.log 2024-05-30T17:09:03.940 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68085.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68085.log.gz 2024-05-30T17:09:03.943 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73676.log 2024-05-30T17:09:03.944 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78489.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78489.log.gz 2024-05-30T17:09:03.947 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70349.log 2024-05-30T17:09:03.948 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73676.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73676.log.gz 2024-05-30T17:09:03.949 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58650.log 2024-05-30T17:09:03.953 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77206.log 2024-05-30T17:09:03.955 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58650.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70349.log.gz 2024-05-30T17:09:03.955 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58650.log.gz 2024-05-30T17:09:03.958 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77206.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77206.log.gz 2024-05-30T17:09:03.959 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56963.log 2024-05-30T17:09:03.964 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78902.log 2024-05-30T17:09:03.965 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56963.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56963.log.gz 2024-05-30T17:09:03.969 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55231.log 2024-05-30T17:09:03.971 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78902.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78902.log.gz 2024-05-30T17:09:03.971 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77670.log 2024-05-30T17:09:03.973 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55231.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55231.log.gz 2024-05-30T17:09:03.973 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59529.log 2024-05-30T17:09:03.974 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77670.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77670.log.gz 2024-05-30T17:09:03.978 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59529.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59529.log.gz 2024-05-30T17:09:03.979 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68187.log 2024-05-30T17:09:03.981 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42547.log 2024-05-30T17:09:03.982 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68187.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68187.log.gz 2024-05-30T17:09:03.984 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49631.log 2024-05-30T17:09:03.985 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42547.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42547.log.gz 2024-05-30T17:09:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67544.log 2024-05-30T17:09:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49631.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49631.log.gz 2024-05-30T17:09:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67544.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67544.log.gz 2024-05-30T17:09:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54368.log 2024-05-30T17:09:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69872.log 2024-05-30T17:09:04.001 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53782.log 2024-05-30T17:09:04.001 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54368.log.gz 2024-05-30T17:09:04.003 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2024-05-30T17:09:04.006 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53782.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72775.log 2024-05-30T17:09:04.006 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69872.log.gz 2024-05-30T17:09:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50530.log: 87.6% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53782.log.gz 2024-05-30T17:09:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57823.log 2024-05-30T17:09:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72775.log: -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2024-05-30T17:09:04.012 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72775.log.gz 2024-05-30T17:09:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42156.log 2024-05-30T17:09:04.013 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57823.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57823.log.gz 2024-05-30T17:09:04.017 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42156.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42156.log.gz 2024-05-30T17:09:04.018 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67917.log 2024-05-30T17:09:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59098.log 2024-05-30T17:09:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67917.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67917.log.gz 2024-05-30T17:09:04.026 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59819.log 2024-05-30T17:09:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64610.log 2024-05-30T17:09:04.027 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59098.log.gz 2024-05-30T17:09:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59819.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59819.log.gz 2024-05-30T17:09:04.033 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41774.log 2024-05-30T17:09:04.033 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64610.log.gz 2024-05-30T17:09:04.036 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76677.log 2024-05-30T17:09:04.037 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41774.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41774.log.gz 2024-05-30T17:09:04.039 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64778.log 2024-05-30T17:09:04.040 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76677.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64105.log 2024-05-30T17:09:04.044 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48070.log 2024-05-30T17:09:04.044 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76677.log.gz 2024-05-30T17:09:04.045 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64778.log.gz/var/log/ceph/ceph-client.admin.64105.log: 2024-05-30T17:09:04.047 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71060.log 2024-05-30T17:09:04.047 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64105.log.gz 2024-05-30T17:09:04.049 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48070.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66417.log 2024-05-30T17:09:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71060.log: -- replaced with /var/log/ceph/ceph-client.admin.48070.log.gz 2024-05-30T17:09:04.052 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6%/var/log/ceph/ceph-client.admin.66417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54891.log 2024-05-30T17:09:04.054 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71060.log.gz 2024-05-30T17:09:04.055 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76425.log 2024-05-30T17:09:04.056 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54891.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66417.log.gz 2024-05-30T17:09:04.056 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54891.log.gz 2024-05-30T17:09:04.057 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61233.log 2024-05-30T17:09:04.058 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76425.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76425.log.gz 2024-05-30T17:09:04.063 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63601.log 2024-05-30T17:09:04.063 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61233.log.gz 2024-05-30T17:09:04.064 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68899.log 2024-05-30T17:09:04.065 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63601.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63601.log.gz 2024-05-30T17:09:04.067 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60338.log 2024-05-30T17:09:04.070 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44909.log 2024-05-30T17:09:04.072 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60338.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60338.log.gz 2024-05-30T17:09:04.072 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68899.log.gz 2024-05-30T17:09:04.076 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44909.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44909.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78681.log 2024-05-30T17:09:04.078 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 2024-05-30T17:09:04.078 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.78585.log 2024-05-30T17:09:04.079 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78681.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78681.log.gz 2024-05-30T17:09:04.080 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81795.log 2024-05-30T17:09:04.081 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78585.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78585.log.gz 2024-05-30T17:09:04.083 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63979.log 2024-05-30T17:09:04.085 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81795.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81795.log.gz 2024-05-30T17:09:04.085 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.74682.log 2024-05-30T17:09:04.086 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63979.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63979.log.gz 2024-05-30T17:09:04.090 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74682.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74682.log.gz 2024-05-30T17:09:04.091 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50696.log 2024-05-30T17:09:04.095 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70921.log 2024-05-30T17:09:04.096 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50696.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50696.log.gz 2024-05-30T17:09:04.098 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64985.log 2024-05-30T17:09:04.103 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.70921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46999.log 2024-05-30T17:09:04.105 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64985.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64985.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66333.log 2024-05-30T17:09:04.105 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-05-30T17:09:04.108 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46999.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46999.log.gz 2024-05-30T17:09:04.111 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66333.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66333.log.gz 2024-05-30T17:09:04.112 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73947.log 2024-05-30T17:09:04.117 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56583.log 2024-05-30T17:09:04.119 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73947.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73947.log.gz 2024-05-30T17:09:04.120 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45807.log 2024-05-30T17:09:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56583.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56583.log.gz 2024-05-30T17:09:04.126 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45807.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45807.log.gz 2024-05-30T17:09:04.128 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69193.log 2024-05-30T17:09:04.130 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42030.log 2024-05-30T17:09:04.132 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68650.log 2024-05-30T17:09:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42030.log: /var/log/ceph/ceph-client.admin.69193.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69193.log.gz 2024-05-30T17:09:04.135 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68314.log 2024-05-30T17:09:04.137 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68650.log: 87.7% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42030.log.gz 2024-05-30T17:09:04.138 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68650.log.gz 2024-05-30T17:09:04.139 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80465.log 2024-05-30T17:09:04.140 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68314.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68314.log.gz 2024-05-30T17:09:04.142 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61976.log 2024-05-30T17:09:04.143 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80465.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.80465.log.gz 2024-05-30T17:09:04.146 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50019.log 2024-05-30T17:09:04.148 INFO:teuthology.orchestra.run.smithi067.stderr:gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61976.log.gz 2024-05-30T17:09:04.148 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45849.log 2024-05-30T17:09:04.152 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50187.log 2024-05-30T17:09:04.152 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50019.log.gz 2024-05-30T17:09:04.153 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69582.log 2024-05-30T17:09:04.154 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45849.log.gz 2024-05-30T17:09:04.158 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50187.log: /var/log/ceph/ceph-client.admin.69582.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50187.log.gz 2024-05-30T17:09:04.159 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54578.log 2024-05-30T17:09:04.162 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.69582.log.gz 2024-05-30T17:09:04.162 INFO:teuthology.orchestra.run.smithi067.stderr: /var/log/ceph/ceph-client.admin.49410.log 2024-05-30T17:09:04.167 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54578.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54578.log.gz 2024-05-30T17:09:04.167 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48942.log 2024-05-30T17:09:04.168 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49410.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49410.log.gz 2024-05-30T17:09:04.171 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49056.log 2024-05-30T17:09:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48942.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48942.log.gz 2024-05-30T17:09:04.176 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51403.log 2024-05-30T17:09:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49056.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49056.log.gz 2024-05-30T17:09:04.181 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65213.log 2024-05-30T17:09:04.181 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51403.log.gz 2024-05-30T17:09:04.183 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43168.log 2024-05-30T17:09:04.186 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65213.log: /var/log/ceph/ceph-client.admin.43168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49224.log 2024-05-30T17:09:04.188 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65213.log.gz 2024-05-30T17:09:04.189 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43168.log.gz 2024-05-30T17:09:04.192 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49224.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49224.log.gz 2024-05-30T17:09:04.193 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59224.log 2024-05-30T17:09:04.195 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57167.log 2024-05-30T17:09:04.197 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67110.log 2024-05-30T17:09:04.199 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57167.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57167.log.gz 2024-05-30T17:09:04.200 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59224.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59224.log.gz 2024-05-30T17:09:04.202 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79967.log 2024-05-30T17:09:04.203 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67110.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67110.log.gz 2024-05-30T17:09:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61850.log 2024-05-30T17:09:04.207 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79967.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79967.log.gz 2024-05-30T17:09:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61850.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61850.log.gz 2024-05-30T17:09:04.209 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57365.log 2024-05-30T17:09:04.212 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81503.log 2024-05-30T17:09:04.213 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61351.log 2024-05-30T17:09:04.213 INFO:teuthology.orchestra.run.smithi067.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57365.log.gz 2024-05-30T17:09:04.214 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81503.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81503.log.gz 2024-05-30T17:09:04.218 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61351.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61351.log.gz 2024-05-30T17:09:04.220 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78405.log 2024-05-30T17:09:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47167.log 2024-05-30T17:09:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47293.log 2024-05-30T17:09:04.223 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78405.log.gz 2024-05-30T17:09:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47167.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47167.log.gz 2024-05-30T17:09:04.225 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65484.log 2024-05-30T17:09:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47293.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47293.log.gz 2024-05-30T17:09:04.229 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59308.log 2024-05-30T17:09:04.230 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65484.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65484.log.gz 2024-05-30T17:09:04.230 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63247.log 2024-05-30T17:09:04.232 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59308.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59308.log.gz 2024-05-30T17:09:04.233 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74115.log 2024-05-30T17:09:04.234 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63247.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63247.log.gz 2024-05-30T17:09:04.236 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74115.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74115.log.gz 2024-05-30T17:09:04.237 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79062.log 2024-05-30T17:09:04.241 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74724.log 2024-05-30T17:09:04.242 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79062.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79062.log.gz 2024-05-30T17:09:04.243 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73592.log 2024-05-30T17:09:04.244 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74724.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74724.log.gz 2024-05-30T17:09:04.247 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80009.log 2024-05-30T17:09:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73592.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73592.log.gz 2024-05-30T17:09:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55446.log 2024-05-30T17:09:04.250 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80009.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80009.log.gz 2024-05-30T17:09:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55446.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55446.log.gz 2024-05-30T17:09:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55698.log 2024-05-30T17:09:04.255 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71886.log 2024-05-30T17:09:04.256 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55698.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55698.log.gz 2024-05-30T17:09:04.259 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65400.log 2024-05-30T17:09:04.260 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71886.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71886.log.gz 2024-05-30T17:09:04.263 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67959.log 2024-05-30T17:09:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65400.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65400.log.gz 2024-05-30T17:09:04.268 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46227.log 2024-05-30T17:09:04.268 INFO:teuthology.orchestra.run.smithi067.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67959.log.gz 2024-05-30T17:09:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46227.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46227.log.gz 2024-05-30T17:09:04.274 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46101.log 2024-05-30T17:09:04.277 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59775.log 2024-05-30T17:09:04.278 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46101.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46101.log.gz 2024-05-30T17:09:04.281 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80549.log 2024-05-30T17:09:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59775.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59775.log.gz 2024-05-30T17:09:04.285 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66585.log 2024-05-30T17:09:04.286 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80549.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80549.log.gz 2024-05-30T17:09:04.291 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66585.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.66585.log.gz 2024-05-30T17:09:04.291 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77460.log 2024-05-30T17:09:04.293 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74265.log 2024-05-30T17:09:04.297 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77460.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77460.log.gz 2024-05-30T17:09:04.301 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41554.log 2024-05-30T17:09:04.302 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74265.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74265.log.gz 2024-05-30T17:09:04.306 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41554.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41554.log.gz 2024-05-30T17:09:04.307 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54213.log 2024-05-30T17:09:04.311 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46017.log 2024-05-30T17:09:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54213.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54213.log.gz 2024-05-30T17:09:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63163.log 2024-05-30T17:09:04.317 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46017.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46017.log.gz 2024-05-30T17:09:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52054.log 2024-05-30T17:09:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63163.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63163.log.gz 2024-05-30T17:09:04.324 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66375.log 2024-05-30T17:09:04.325 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52054.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52054.log.gz 2024-05-30T17:09:04.329 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66375.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66375.log.gz 2024-05-30T17:09:04.330 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44758.log 2024-05-30T17:09:04.332 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58537.log 2024-05-30T17:09:04.334 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.2.44758.log: 90.1% -- replaced with /var/log/ceph/ceph-client.2.44758.log.gz 2024-05-30T17:09:04.334 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67791.log 2024-05-30T17:09:04.336 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58537.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58537.log.gz 2024-05-30T17:09:04.336 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59956.log 2024-05-30T17:09:04.338 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67791.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67791.log.gz 2024-05-30T17:09:04.342 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59956.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59956.log.gz 2024-05-30T17:09:04.343 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68356.log 2024-05-30T17:09:04.346 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41816.log 2024-05-30T17:09:04.347 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68356.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68356.log.gz 2024-05-30T17:09:04.348 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62627.log 2024-05-30T17:09:04.349 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41816.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41816.log.gz 2024-05-30T17:09:04.352 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65442.log 2024-05-30T17:09:04.353 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62627.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62627.log.gz 2024-05-30T17:09:04.354 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47944.log 2024-05-30T17:09:04.355 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65442.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65442.log.gz 2024-05-30T17:09:04.359 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79925.log 2024-05-30T17:09:04.359 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47944.log.gz 2024-05-30T17:09:04.363 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49935.log 2024-05-30T17:09:04.364 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79925.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79925.log.gz 2024-05-30T17:09:04.367 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58177.log 2024-05-30T17:09:04.367 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49935.log.gz 2024-05-30T17:09:04.371 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42072.log 2024-05-30T17:09:04.372 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58177.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58177.log.gz 2024-05-30T17:09:04.375 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63121.log 2024-05-30T17:09:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42072.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42072.log.gz 2024-05-30T17:09:04.382 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45229.log 2024-05-30T17:09:04.383 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63121.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63121.log.gz 2024-05-30T17:09:04.385 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43217.log 2024-05-30T17:09:04.388 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45229.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45229.log.gz 2024-05-30T17:09:04.390 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81167.log 2024-05-30T17:09:04.391 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43217.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43217.log.gz 2024-05-30T17:09:04.394 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43218.log 2024-05-30T17:09:04.395 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81167.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81167.log.gz 2024-05-30T17:09:04.398 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81041.log 2024-05-30T17:09:04.399 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43218.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43218.log.gz 2024-05-30T17:09:04.402 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51259.log 2024-05-30T17:09:04.403 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81041.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81041.log.gz 2024-05-30T17:09:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77288.log 2024-05-30T17:09:04.407 INFO:teuthology.orchestra.run.smithi067.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51259.log.gz 2024-05-30T17:09:04.410 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80321.log/var/log/ceph/ceph-client.admin.77288.log: 2024-05-30T17:09:04.411 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77288.log.gz 2024-05-30T17:09:04.414 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75486.log 2024-05-30T17:09:04.415 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80321.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80321.log.gz 2024-05-30T17:09:04.419 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.75486.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73989.log 2024-05-30T17:09:04.419 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75486.log.gz 2024-05-30T17:09:04.422 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69067.log 2024-05-30T17:09:04.423 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73989.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73989.log.gz 2024-05-30T17:09:04.427 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69067.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69067.log.gz 2024-05-30T17:09:04.429 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51175.log 2024-05-30T17:09:04.433 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74862.log 2024-05-30T17:09:04.434 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51175.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51175.log.gz 2024-05-30T17:09:04.437 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72952.log 2024-05-30T17:09:04.438 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74862.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74862.log.gz 2024-05-30T17:09:04.441 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55824.log 2024-05-30T17:09:04.443 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72952.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72952.log.gz 2024-05-30T17:09:04.445 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65950.log 2024-05-30T17:09:04.446 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55824.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55824.log.gz 2024-05-30T17:09:04.449 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64652.log 2024-05-30T17:09:04.450 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65950.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65950.log.gz 2024-05-30T17:09:04.451 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65652.log 2024-05-30T17:09:04.454 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64652.log: /var/log/ceph/ceph-client.admin.65652.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65652.log.gz 2024-05-30T17:09:04.455 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60947.log 2024-05-30T17:09:04.455 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64652.log.gz 2024-05-30T17:09:04.459 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63205.log 2024-05-30T17:09:04.460 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60947.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60947.log.gz 2024-05-30T17:09:04.463 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50145.log 2024-05-30T17:09:04.464 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63205.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63205.log.gz 2024-05-30T17:09:04.468 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68500.log 2024-05-30T17:09:04.470 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50145.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50145.log.gz 2024-05-30T17:09:04.473 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68500.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68500.log.gz 2024-05-30T17:09:04.475 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70139.log 2024-05-30T17:09:04.478 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45891.log 2024-05-30T17:09:04.479 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70139.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70139.log.gz 2024-05-30T17:09:04.480 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44359.log 2024-05-30T17:09:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45891.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45891.log.gz 2024-05-30T17:09:04.484 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46185.log 2024-05-30T17:09:04.485 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44359.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44359.log.gz 2024-05-30T17:09:04.485 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75120.log 2024-05-30T17:09:04.487 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46185.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46185.log.gz 2024-05-30T17:09:04.490 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37348.log 2024-05-30T17:09:04.491 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75120.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75120.log.gz 2024-05-30T17:09:04.492 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77922.log 2024-05-30T17:09:04.493 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.37348.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37348.log.gz 2024-05-30T17:09:04.500 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72377.log 2024-05-30T17:09:04.501 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55614.log 2024-05-30T17:09:04.502 INFO:teuthology.orchestra.run.smithi067.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77922.log.gz 2024-05-30T17:09:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72377.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72377.log.gz 2024-05-30T17:09:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50103.log 2024-05-30T17:09:04.507 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55614.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55614.log.gz 2024-05-30T17:09:04.508 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49893.log 2024-05-30T17:09:04.510 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59914.log 2024-05-30T17:09:04.510 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50103.log.gz 2024-05-30T17:09:04.514 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42282.log 2024-05-30T17:09:04.515 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49893.log: /var/log/ceph/ceph-client.admin.59914.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49893.log.gz 2024-05-30T17:09:04.515 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59914.log.gz 2024-05-30T17:09:04.519 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54933.log 2024-05-30T17:09:04.522 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42282.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42282.log.gz 2024-05-30T17:09:04.523 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66165.log 2024-05-30T17:09:04.525 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54933.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54933.log.gz 2024-05-30T17:09:04.526 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41650.log 2024-05-30T17:09:04.529 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2024-05-30T17:09:04.530 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66165.log.gz/var/log/ceph/ceph-client.admin.41650.log: 2024-05-30T17:09:04.531 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65526.log 2024-05-30T17:09:04.534 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41650.log.gz 2024-05-30T17:09:04.535 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46307.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gz 2024-05-30T17:09:04.536 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65526.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65526.log.gz 2024-05-30T17:09:04.538 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68983.log 2024-05-30T17:09:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75942.log 2024-05-30T17:09:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68983.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68983.log.gz 2024-05-30T17:09:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58135.log 2024-05-30T17:09:04.546 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75942.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75942.log.gz 2024-05-30T17:09:04.549 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81755.log 2024-05-30T17:09:04.550 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58135.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58135.log.gz 2024-05-30T17:09:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50061.log 2024-05-30T17:09:04.553 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81755.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64063.log 2024-05-30T17:09:04.557 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50061.log: gzip 88.5%/var/log/ceph/ceph-client.admin.64063.log: -- replaced with /var/log/ceph/ceph-client.admin.81755.log.gz 2024-05-30T17:09:04.558 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.66035.log 2024-05-30T17:09:04.560 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50061.log.gz 2024-05-30T17:09:04.561 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64063.log.gz 2024-05-30T17:09:04.561 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57209.log 2024-05-30T17:09:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66035.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66035.log.gz 2024-05-30T17:09:04.565 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64568.log 2024-05-30T17:09:04.566 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57209.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57209.log.gz 2024-05-30T17:09:04.569 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53656.log 2024-05-30T17:09:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64568.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64568.log.gz 2024-05-30T17:09:04.572 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75672.log 2024-05-30T17:09:04.574 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53656.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53656.log.gz 2024-05-30T17:09:04.578 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58495.log 2024-05-30T17:09:04.578 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75672.log.gz 2024-05-30T17:09:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65027.log 2024-05-30T17:09:04.581 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58495.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58495.log.gz 2024-05-30T17:09:04.585 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65027.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65027.log.gz 2024-05-30T17:09:04.586 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71144.log 2024-05-30T17:09:04.590 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79589.log 2024-05-30T17:09:04.591 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67468.log 2024-05-30T17:09:04.591 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71144.log.gz 2024-05-30T17:09:04.592 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79589.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79589.log.gz 2024-05-30T17:09:04.596 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59392.log 2024-05-30T17:09:04.597 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67468.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67468.log.gz 2024-05-30T17:09:04.600 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48706.log 2024-05-30T17:09:04.601 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59392.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59392.log.gz 2024-05-30T17:09:04.605 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48706.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48706.log.gz 2024-05-30T17:09:04.606 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56433.log 2024-05-30T17:09:04.609 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79757.log 2024-05-30T17:09:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56433.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56433.log.gz 2024-05-30T17:09:04.612 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57781.log 2024-05-30T17:09:04.614 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79757.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62160.log 2024-05-30T17:09:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57781.log: gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.79757.log.gz 2024-05-30T17:09:04.617 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62160.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57781.log.gz 2024-05-30T17:09:04.618 INFO:teuthology.orchestra.run.smithi067.stderr: -- /var/log/ceph/ceph-client.admin.81083.log 2024-05-30T17:09:04.621 INFO:teuthology.orchestra.run.smithi067.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62160.log.gz 2024-05-30T17:09:04.621 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48858.log 2024-05-30T17:09:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77416.log 2024-05-30T17:09:04.623 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81083.log.gz 2024-05-30T17:09:04.625 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54765.log 2024-05-30T17:09:04.626 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77416.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77416.log.gz 2024-05-30T17:09:04.627 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42537.log 2024-05-30T17:09:04.629 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7%/var/log/ceph/ceph-client.admin.54765.log: -- replaced with /var/log/ceph/ceph-client.admin.48858.log.gz 2024-05-30T17:09:04.631 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4%/var/log/ceph/ceph-client.admin.42537.log: -- replaced with /var/log/ceph/ceph-client.admin.54765.log.gz 2024-05-30T17:09:04.632 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42537.log.gz 2024-05-30T17:09:04.633 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42869.log 2024-05-30T17:09:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62467.log 2024-05-30T17:09:04.637 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75714.log 2024-05-30T17:09:04.641 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62467.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42869.log.gz 2024-05-30T17:09:04.641 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-30T17:09:04.643 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75714.log: 88.1% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62467.log.gz 2024-05-30T17:09:04.643 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75714.log.gz 2024-05-30T17:09:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63745.log 2024-05-30T17:09:04.649 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80195.log 2024-05-30T17:09:04.650 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63745.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63745.log.gz 2024-05-30T17:09:04.652 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76761.log 2024-05-30T17:09:04.655 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80195.log: 88.2%/var/log/ceph/ceph-client.admin.76761.log: -- replaced with /var/log/ceph/ceph-client.admin.80195.log.gz 2024-05-30T17:09:04.658 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81461.log 2024-05-30T17:09:04.658 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76761.log.gz 2024-05-30T17:09:04.659 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76929.log 2024-05-30T17:09:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81461.log: /var/log/ceph/ceph-client.admin.76929.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81461.log.gz 2024-05-30T17:09:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55932.log 2024-05-30T17:09:04.667 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76929.log.gz 2024-05-30T17:09:04.670 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61418.log 2024-05-30T17:09:04.671 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55932.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55932.log.gz 2024-05-30T17:09:04.674 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77754.log 2024-05-30T17:09:04.675 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61418.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61418.log.gz 2024-05-30T17:09:04.676 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74031.log 2024-05-30T17:09:04.678 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77754.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77754.log.gz 2024-05-30T17:09:04.678 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75588.log 2024-05-30T17:09:04.679 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74031.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74031.log.gz 2024-05-30T17:09:04.682 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75900.log 2024-05-30T17:09:04.684 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55782.log 2024-05-30T17:09:04.685 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75900.log: /var/log/ceph/ceph-client.admin.75588.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75900.log.gz 2024-05-30T17:09:04.685 INFO:teuthology.orchestra.run.smithi067.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75588.log.gz 2024-05-30T17:09:04.687 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55782.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55782.log.gz 2024-05-30T17:09:04.690 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54100.log 2024-05-30T17:09:04.694 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77880.log 2024-05-30T17:09:04.695 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54100.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54100.log.gz 2024-05-30T17:09:04.699 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48984.log 2024-05-30T17:09:04.700 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77880.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77880.log.gz 2024-05-30T17:09:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53740.log 2024-05-30T17:09:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48984.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48984.log.gz 2024-05-30T17:09:04.710 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45103.log 2024-05-30T17:09:04.710 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53740.log.gz 2024-05-30T17:09:04.712 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59056.log 2024-05-30T17:09:04.714 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45765.log/var/log/ceph/ceph-client.admin.45103.log: 2024-05-30T17:09:04.714 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45103.log.gz 2024-05-30T17:09:04.715 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59056.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59056.log.gz 2024-05-30T17:09:04.719 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80111.log 2024-05-30T17:09:04.720 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45765.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45765.log.gz 2024-05-30T17:09:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80111.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80111.log.gz 2024-05-30T17:09:04.727 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80623.log 2024-05-30T17:09:04.731 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69361.log 2024-05-30T17:09:04.732 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80623.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80623.log.gz 2024-05-30T17:09:04.735 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52630.log 2024-05-30T17:09:04.736 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69361.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69361.log.gz 2024-05-30T17:09:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71802.log 2024-05-30T17:09:04.739 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52630.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52630.log.gz 2024-05-30T17:09:04.742 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71802.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71802.log.gz 2024-05-30T17:09:04.743 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73845.log 2024-05-30T17:09:04.747 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62996.log 2024-05-30T17:09:04.749 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73845.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73845.log.gz 2024-05-30T17:09:04.751 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55070.log 2024-05-30T17:09:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62996.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.62996.log.gz 2024-05-30T17:09:04.754 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45187.log 2024-05-30T17:09:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55070.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55070.log.gz 2024-05-30T17:09:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45187.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45187.log.gz 2024-05-30T17:09:04.759 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66929.log 2024-05-30T17:09:04.766 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75816.log 2024-05-30T17:09:04.768 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.66929.log: /var/log/ceph/ceph-client.admin.75816.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75816.log.gz 2024-05-30T17:09:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45933.log 2024-05-30T17:09:04.773 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45548.log 2024-05-30T17:09:04.774 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45933.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45933.log.gz 2024-05-30T17:09:04.778 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58870.log 2024-05-30T17:09:04.779 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41462.log 2024-05-30T17:09:04.780 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45548.log.gz 2024-05-30T17:09:04.782 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43214.log 2024-05-30T17:09:04.785 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41462.log: /var/log/ceph/ceph-client.admin.58870.log: 87.3% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58870.log.gz 2024-05-30T17:09:04.787 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41462.log.gz 2024-05-30T17:09:04.788 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43214.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43214.log.gz 2024-05-30T17:09:04.790 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60682.log 2024-05-30T17:09:04.791 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59014.log 2024-05-30T17:09:04.794 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.60682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50271.log 2024-05-30T17:09:04.795 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59014.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59014.log.gz 2024-05-30T17:09:04.798 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71456.log 2024-05-30T17:09:04.799 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50271.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50271.log.gz 2024-05-30T17:09:04.800 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50313.log 2024-05-30T17:09:04.801 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71456.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71456.log.gz 2024-05-30T17:09:04.804 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80153.log 2024-05-30T17:09:04.805 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50313.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50313.log.gz 2024-05-30T17:09:04.807 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72114.log 2024-05-30T17:09:04.809 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66459.log 2024-05-30T17:09:04.811 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80153.log.gz 2024-05-30T17:09:04.811 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68001.log 2024-05-30T17:09:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66459.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66459.log.gz 2024-05-30T17:09:04.816 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72114.log: 88.6%/var/log/ceph/ceph-client.admin.68001.log: -- replaced with /var/log/ceph/ceph-client.admin.72114.log.gz 2024-05-30T17:09:04.816 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68001.log.gz 2024-05-30T17:09:04.817 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73078.log 2024-05-30T17:09:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57967.log 2024-05-30T17:09:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42538.log 2024-05-30T17:09:04.823 INFO:teuthology.orchestra.run.smithi067.stderr: 96.2%/var/log/ceph/ceph-client.admin.73078.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73078.log.gz 2024-05-30T17:09:04.823 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.0.56210.log.gz 2024-05-30T17:09:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.57967.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63937.log 2024-05-30T17:09:04.825 INFO:teuthology.orchestra.run.smithi067.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57967.log.gz 2024-05-30T17:09:04.826 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42538.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42538.log.gz 2024-05-30T17:09:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57407.log 2024-05-30T17:09:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63937.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63937.log.gz 2024-05-30T17:09:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57407.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.57407.log.gz 2024-05-30T17:09:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51133.log 2024-05-30T17:09:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78266.log 2024-05-30T17:09:04.840 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51133.log.gz 2024-05-30T17:09:04.841 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65305.log 2024-05-30T17:09:04.843 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.78266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59609.log 2024-05-30T17:09:04.847 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65305.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65305.log.gz 2024-05-30T17:09:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59609.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62407.log 2024-05-30T17:09:04.849 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59609.log.gz 2024-05-30T17:09:04.852 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46831.log 2024-05-30T17:09:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62407.log: 87.4%/var/log/ceph/ceph-client.admin.46831.log: -- replaced with /var/log/ceph/ceph-client.admin.62407.log.gz 2024-05-30T17:09:04.858 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73803.log 2024-05-30T17:09:04.860 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46831.log.gz 2024-05-30T17:09:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78860.log 2024-05-30T17:09:04.863 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73803.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73803.log.gz 2024-05-30T17:09:04.868 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77712.log 2024-05-30T17:09:04.870 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60086.log 2024-05-30T17:09:04.870 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78860.log.gz 2024-05-30T17:09:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77712.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77712.log.gz 2024-05-30T17:09:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68763.log 2024-05-30T17:09:04.874 INFO:teuthology.orchestra.run.smithi169.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-30T17:09:04.875 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60086.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60086.log.gz 2024-05-30T17:09:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68763.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68763.log.gz 2024-05-30T17:09:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55316.log 2024-05-30T17:09:04.885 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55316.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55316.log.gz 2024-05-30T17:09:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54058.log 2024-05-30T17:09:04.887 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62709.log 2024-05-30T17:09:04.888 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54058.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54058.log.gz 2024-05-30T17:09:04.889 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54410.log 2024-05-30T17:09:04.894 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62709.log: 86.4%/var/log/ceph/ceph-client.admin.54410.log: -- replaced with /var/log/ceph/ceph-client.admin.62709.log.gz 2024-05-30T17:09:04.896 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69319.log 2024-05-30T17:09:04.897 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54410.log.gz 2024-05-30T17:09:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76887.log 2024-05-30T17:09:04.906 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69319.log: /var/log/ceph/ceph-client.admin.76887.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69319.log.gz 2024-05-30T17:09:04.906 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53341.log 2024-05-30T17:09:04.906 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76887.log.gz 2024-05-30T17:09:04.906 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53470.log 2024-05-30T17:09:04.907 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53341.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53341.log.gz 2024-05-30T17:09:04.912 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53470.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53470.log.gz 2024-05-30T17:09:04.913 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63829.log 2024-05-30T17:09:04.914 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66249.log 2024-05-30T17:09:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63829.log: /var/log/ceph/ceph-client.admin.66249.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63829.log.gz 2024-05-30T17:09:04.920 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80981.log 2024-05-30T17:09:04.920 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66249.log.gz 2024-05-30T17:09:04.926 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80981.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80981.log.gz 2024-05-30T17:09:04.926 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45145.log 2024-05-30T17:09:04.933 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45145.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45145.log.gz 2024-05-30T17:09:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71282.log 2024-05-30T17:09:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50229.log 2024-05-30T17:09:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50229.log: /var/log/ceph/ceph-client.admin.71282.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50229.log.gz 2024-05-30T17:09:04.940 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71282.log.gz 2024-05-30T17:09:04.940 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75630.log 2024-05-30T17:09:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63517.log 2024-05-30T17:09:04.943 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75630.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75630.log.gz 2024-05-30T17:09:04.949 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63517.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63517.log.gz 2024-05-30T17:09:04.950 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61934.log 2024-05-30T17:09:04.951 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46533.log 2024-05-30T17:09:04.956 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61934.log: 88.2%/var/log/ceph/ceph-client.admin.46533.log: -- replaced with /var/log/ceph/ceph-client.admin.61934.log.gz 2024-05-30T17:09:04.957 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64400.log 2024-05-30T17:09:04.957 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46533.log.gz 2024-05-30T17:09:04.962 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64400.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64400.log.gz 2024-05-30T17:09:04.963 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81293.log 2024-05-30T17:09:04.964 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69235.log 2024-05-30T17:09:04.966 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.81293.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81293.log.gz 2024-05-30T17:09:04.970 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69235.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69235.log.gz 2024-05-30T17:09:04.971 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71240.log 2024-05-30T17:09:04.973 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71240.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71240.log.gz 2024-05-30T17:09:04.974 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62352.log 2024-05-30T17:09:04.978 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62352.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62352.log.gz 2024-05-30T17:09:04.979 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56803.log 2024-05-30T17:09:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56803.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56803.log.gz 2024-05-30T17:09:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62668.log 2024-05-30T17:09:04.987 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58744.log 2024-05-30T17:09:04.992 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58744.log: /var/log/ceph/ceph-client.admin.62668.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58744.log.gz 2024-05-30T17:09:04.992 INFO:teuthology.orchestra.run.smithi067.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.62668.log.gz 2024-05-30T17:09:04.993 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47986.log 2024-05-30T17:09:04.994 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69277.log 2024-05-30T17:09:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47986.log: /var/log/ceph/ceph-client.admin.69277.log: 88.8% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47986.log.gz -- replaced with /var/log/ceph/ceph-client.admin.69277.log.gz 2024-05-30T17:09:04.999 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-05-30T17:09:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54849.log 2024-05-30T17:09:05.001 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72264.log 2024-05-30T17:09:05.003 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54849.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54849.log.gz 2024-05-30T17:09:05.007 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72264.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66207.log 2024-05-30T17:09:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80897.log 2024-05-30T17:09:05.009 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72264.log.gz 2024-05-30T17:09:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66207.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66207.log.gz 2024-05-30T17:09:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68271.log 2024-05-30T17:09:05.015 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.80897.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80897.log.gz 2024-05-30T17:09:05.017 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59690.log 2024-05-30T17:09:05.022 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68271.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68271.log.gz 2024-05-30T17:09:05.025 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59690.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59690.log.gz 2024-05-30T17:09:05.025 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63787.log 2024-05-30T17:09:05.026 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79505.log 2024-05-30T17:09:05.028 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63787.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63787.log.gz 2024-05-30T17:09:05.031 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72826.log 2024-05-30T17:09:05.032 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79505.log.gz 2024-05-30T17:09:05.033 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60404.log 2024-05-30T17:09:05.035 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72910.log 2024-05-30T17:09:05.039 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60404.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60404.log.gz 2024-05-30T17:09:05.040 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72826.log.gz 2024-05-30T17:09:05.041 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57655.log 2024-05-30T17:09:05.042 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72910.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53054.log 2024-05-30T17:09:05.045 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72910.log.gz 2024-05-30T17:09:05.047 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57655.log: /var/log/ceph/ceph-client.admin.53054.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57655.log.gz 2024-05-30T17:09:05.048 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53054.log.gz 2024-05-30T17:09:05.708 INFO:teuthology.orchestra.run.smithi067.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.74543.log.gz 2024-05-30T17:09:05.855 INFO:teuthology.orchestra.run.smithi169.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-30T17:09:06.511 INFO:teuthology.orchestra.run.smithi067.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.70921.log.gz 2024-05-30T17:09:06.579 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-30T17:09:06.805 INFO:teuthology.orchestra.run.smithi067.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.66929.log.gz 2024-05-30T17:09:06.927 INFO:teuthology.orchestra.run.smithi169.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-30T17:09:07.085 INFO:teuthology.orchestra.run.smithi067.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.78266.log.gz 2024-05-30T17:09:07.117 INFO:teuthology.orchestra.run.smithi067.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.60682.log.gz 2024-05-30T17:09:07.722 INFO:teuthology.orchestra.run.smithi169.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-30T17:09:08.199 INFO:teuthology.orchestra.run.smithi169.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-30T17:09:08.628 INFO:teuthology.orchestra.run.smithi067.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.47805.log.gz 2024-05-30T17:09:08.794 INFO:teuthology.orchestra.run.smithi067.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.51789.log.gz 2024-05-30T17:09:08.837 INFO:teuthology.orchestra.run.smithi169.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-30T17:09:08.841 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-05-30T17:09:08.841 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m5.847s 2024-05-30T17:09:08.841 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m23.474s 2024-05-30T17:09:08.842 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m2.037s 2024-05-30T17:09:10.674 INFO:teuthology.orchestra.run.smithi067.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-30T17:09:10.778 INFO:teuthology.orchestra.run.smithi067.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-30T17:09:11.002 INFO:teuthology.orchestra.run.smithi067.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-30T17:09:11.002 INFO:teuthology.orchestra.run.smithi067.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-30T17:09:11.515 INFO:teuthology.orchestra.run.smithi067.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-30T17:09:11.635 INFO:teuthology.orchestra.run.smithi067.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-30T17:09:11.638 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-05-30T17:09:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:real 0m8.644s 2024-05-30T17:09:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:user 0m52.638s 2024-05-30T17:09:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m5.892s 2024-05-30T17:09:11.640 INFO:tasks.ceph:Archiving logs... 2024-05-30T17:09:11.640 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/remote/smithi067/log 2024-05-30T17:09:11.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-30T17:09:16.908 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/remote/smithi169/log 2024-05-30T17:09:16.909 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-30T17:09:19.721 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-30T17:09:19.730 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-30T17:09:19.731 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-30T17:09:19.739 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-30T17:09:19.876 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-30T17:09:19.885 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-30T17:09:19.885 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-30T17:09:19.893 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-30T17:09:19.943 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-05-30T17:09:19.948 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-05-30T17:09:19.953 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-30T17:09:19.953 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-05-30T17:09:19.954 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 +1330us[+1344us] +/- 44ms 2024-05-30T17:09:19.954 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -1398us[-1383us] +/- 53ms 2024-05-30T17:09:19.954 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 -271us[ -271us] +/- 71ms 2024-05-30T17:09:19.954 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T17:09:19.959 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-30T17:09:19.959 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-05-30T17:09:19.959 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +1321us[+1298us] +/- 44ms 2024-05-30T17:09:19.959 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -1415us[-1438us] +/- 53ms 2024-05-30T17:09:19.959 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -256us[ -278us] +/- 71ms 2024-05-30T17:09:19.959 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-30T17:09:19.960 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-30T17:09:19.970 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-30T17:09:19.970 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-30T17:09:19.979 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-30T17:09:20.218 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-30T17:09:20.411 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-30T17:09:20.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T17:09:20.516 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-30T17:09:20.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T17:09:20.609 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-30T17:09:20.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-30T17:09:20.701 INFO:teuthology.task.internal:Duration was 2144.878832 seconds 2024-05-30T17:09:20.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-30T17:09:20.710 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-30T17:09:20.711 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-30T17:09:20.718 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-30T17:09:20.840 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-30T17:09:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-30T17:09:21.321 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-30T17:09:21.322 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-05-30T17:09:21.322 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-30T17:09:21.387 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-05-30T17:09:21.388 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-30T17:09:21.464 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-30T17:09:21.464 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-30T17:09:21.471 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-30T17:09:21.557 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-30T17:09:21.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-30T17:09:21.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-30T17:09:21.769 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-30T17:09:21.778 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-30T17:09:21.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-30T17:09:21.864 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-30T17:09:21.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-30T17:09:21.960 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-30T17:09:21.968 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-30T17:09:22.061 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-05-30T17:09:22.062 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-05-30T17:09:22.104 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-30T17:09:22.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T17:09:22.150 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-30T17:09:22.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-30T17:09:22.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-30T17:09:22.210 INFO:teuthology.task.internal:Transferring archived files... 2024-05-30T17:09:22.211 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/remote/smithi067 2024-05-30T17:09:22.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-30T17:09:22.311 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-30_16:10:36-fs:volumes-rishabh-clone-stats-testing-default-smithi/7734140/remote/smithi169 2024-05-30T17:09:22.311 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-30T17:09:22.409 INFO:teuthology.task.internal:Removing archive directory... 2024-05-30T17:09:22.409 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-30T17:09:22.417 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-30T17:09:22.472 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-30T17:09:22.483 INFO:teuthology.task.internal:Not uploading archives. 2024-05-30T17:09:22.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-30T17:09:22.546 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-30T17:09:22.547 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-30T17:09:22.555 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-30T17:09:22.609 INFO:teuthology.orchestra.run.smithi067.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 30 17:09 /home/ubuntu/cephtest 2024-05-30T17:09:22.613 INFO:teuthology.orchestra.run.smithi169.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 30 17:09 /home/ubuntu/cephtest 2024-05-30T17:09:22.616 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-30T17:09:22.625 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-30T17:09:22.647 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 2144.8788318634033 failure_reason: 'Test failure: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter), test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter)' flavor: default mon.a-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.b-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=77fddf6f11784a40a513032d69b9eada status: fail success: false 2024-05-30T17:09:22.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-30T17:09:22.737 INFO:teuthology.run:FAIL